2023-11-06T01:52:55.807 INFO:root:teuthology version: 0.0.1.dev203+g6899cd2 2023-11-06T01:52:55.808 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --archive /home/teuthworker/archive/yuriw-2023-11-05_23:44:49-fs-reef-release-distro-default-smithi/7448684 --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/forward-scrub} --owner scheduled_yuriw@teuthology --name yuriw-2023-11-05_23:44:49-fs-reef-release-distro-default-smithi -- /home/teuthworker/archive/yuriw-2023-11-05_23:44:49-fs-reef-release-distro-default-smithi/7448684/orig.config.yaml 2023-11-06T01:52:55.851 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-06T01:52:55.934 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2023-11-05_23:44:49-fs-reef-release-distro-default-smithi/7448684 branch: reef-release description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/forward-scrub} email: yweinste@redhat.com first_in_suite: false job_id: '7448684' kernel: kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: yuriw-2023-11-05_23:44:49-fs-reef-release-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: reef-release ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - inode wrongly marked free - bad backtrace on inode - inode table repaired for inode - Scrub error on inode - Scrub error on dir - Metadata damage detected sha1: f7e9f9af51acf98e0ccc3037487018e4ae40caa5 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: f7e9f9af51acf98e0ccc3037487018e4ae40caa5 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: reef-release sha1: f7e9f9af51acf98e0ccc3037487018e4ae40caa5 owner: scheduled_yuriw@teuthology priority: 71 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 1834 sha1: f7e9f9af51acf98e0ccc3037487018e4ae40caa5 sleep_before_teardown: 0 subset: 111/12000 suite: fs suite_branch: reef-release suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_f7e9f9af51acf98e0ccc3037487018e4ae40caa5/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: f7e9f9af51acf98e0ccc3037487018e4ae40caa5 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi049.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBExxTF5Kt7KZ8Ef/cLx/cqs3AWnVQ4ai7T1jEDiGZzy9Q5LtI0OPverOvitoCFdXpAojj0/ANPgMelc6AP0UzDA= smithi150.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHAJnIp8mtEt40gfMIRsjg5lCglKz1ohJ2WB8MsZn64LZXl/9/UGBQ4oYaVUsfH3FRK1MXjvSQFDp7zwxCN2mjg= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: null - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_forward_scrub teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 6899cd26fceddb2fec83dc1a1349394b28c8998e timestamp: 2023-11-05_23:44:49 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3084822 2023-11-06T01:52:55.935 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_f7e9f9af51acf98e0ccc3037487018e4ae40caa5/qa; will attempt to use it 2023-11-06T01:52:55.936 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_f7e9f9af51acf98e0ccc3037487018e4ae40caa5/qa/tasks 2023-11-06T01:52:55.936 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-11-06T01:52:55.938 INFO:teuthology.task.internal:Checking packages... 2023-11-06T01:52:55.962 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash 'f7e9f9af51acf98e0ccc3037487018e4ae40caa5' 2023-11-06T01:52:55.962 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-11-06T01:52:55.963 INFO:teuthology.packaging:ref: None 2023-11-06T01:52:55.963 INFO:teuthology.packaging:tag: None 2023-11-06T01:52:55.964 INFO:teuthology.packaging:branch: reef-release 2023-11-06T01:52:55.964 INFO:teuthology.packaging:sha1: f7e9f9af51acf98e0ccc3037487018e4ae40caa5 2023-11-06T01:52:55.964 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=reef-release 2023-11-06T01:52:56.183 INFO:teuthology.task.internal:Found packages for ceph version 18.2.0-1181-gf7e9f9af-1jammy 2023-11-06T01:52:56.184 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-11-06T01:52:56.190 INFO:teuthology.task.internal:no buildpackages task found 2023-11-06T01:52:56.190 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-11-06T01:52:56.225 INFO:teuthology.task.internal:Saving configuration 2023-11-06T01:52:56.240 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-11-06T01:52:56.246 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-06T01:52:56.270 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi049.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-11-05_23:44:49-fs-reef-release-distro-default-smithi/7448684', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-06 01:45:25.558695', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIN0A4fB812cGrFltWwhcoc7YVOteTrppjm94yCTXK+8i'} 2023-11-06T01:52:56.294 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi150.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-11-05_23:44:49-fs-reef-release-distro-default-smithi/7448684', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-06 01:45:25.557230', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIPoJTSTKffr49IfDn3suDNdnRf3UzWUMeQOGo8wd6fNM'} 2023-11-06T01:52:56.294 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-11-06T01:52:56.301 INFO:teuthology.task.internal:roles: ubuntu@smithi049.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-11-06T01:52:56.301 INFO:teuthology.task.internal:roles: ubuntu@smithi150.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-11-06T01:52:56.301 INFO:teuthology.run_tasks:Running task console_log... 2023-11-06T01:52:56.373 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f7e6b0daaf0>, signals=[15]) 2023-11-06T01:52:56.375 INFO:teuthology.run_tasks:Running task internal.connect... 2023-11-06T01:52:56.381 INFO:teuthology.task.internal:Opening connections... 2023-11-06T01:52:56.381 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi049.front.sepia.ceph.com 2023-11-06T01:52:56.383 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-06T01:52:56.463 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi150.front.sepia.ceph.com 2023-11-06T01:52:56.464 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-06T01:52:56.537 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-11-06T01:52:56.543 DEBUG:teuthology.orchestra.run.smithi049:> uname -m 2023-11-06T01:52:56.549 INFO:teuthology.orchestra.run.smithi049.stdout:x86_64 2023-11-06T01:52:56.549 DEBUG:teuthology.orchestra.run.smithi049:> cat /etc/os-release 2023-11-06T01:52:56.595 INFO:teuthology.orchestra.run.smithi049.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2023-11-06T01:52:56.595 INFO:teuthology.orchestra.run.smithi049.stdout:NAME="Ubuntu" 2023-11-06T01:52:56.595 INFO:teuthology.orchestra.run.smithi049.stdout:VERSION_ID="22.04" 2023-11-06T01:52:56.595 INFO:teuthology.orchestra.run.smithi049.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2023-11-06T01:52:56.595 INFO:teuthology.orchestra.run.smithi049.stdout:VERSION_CODENAME=jammy 2023-11-06T01:52:56.596 INFO:teuthology.orchestra.run.smithi049.stdout:ID=ubuntu 2023-11-06T01:52:56.596 INFO:teuthology.orchestra.run.smithi049.stdout:ID_LIKE=debian 2023-11-06T01:52:56.596 INFO:teuthology.orchestra.run.smithi049.stdout:HOME_URL="https://www.ubuntu.com/" 2023-11-06T01:52:56.596 INFO:teuthology.orchestra.run.smithi049.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-11-06T01:52:56.596 INFO:teuthology.orchestra.run.smithi049.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-11-06T01:52:56.597 INFO:teuthology.orchestra.run.smithi049.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-11-06T01:52:56.597 INFO:teuthology.orchestra.run.smithi049.stdout:UBUNTU_CODENAME=jammy 2023-11-06T01:52:56.597 INFO:teuthology.lock.ops:Updating smithi049.front.sepia.ceph.com on lock server 2023-11-06T01:52:56.618 DEBUG:teuthology.orchestra.run.smithi150:> uname -m 2023-11-06T01:52:56.623 INFO:teuthology.orchestra.run.smithi150.stdout:x86_64 2023-11-06T01:52:56.624 DEBUG:teuthology.orchestra.run.smithi150:> cat /etc/os-release 2023-11-06T01:52:56.670 INFO:teuthology.orchestra.run.smithi150.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2023-11-06T01:52:56.671 INFO:teuthology.orchestra.run.smithi150.stdout:NAME="Ubuntu" 2023-11-06T01:52:56.671 INFO:teuthology.orchestra.run.smithi150.stdout:VERSION_ID="22.04" 2023-11-06T01:52:56.671 INFO:teuthology.orchestra.run.smithi150.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2023-11-06T01:52:56.671 INFO:teuthology.orchestra.run.smithi150.stdout:VERSION_CODENAME=jammy 2023-11-06T01:52:56.672 INFO:teuthology.orchestra.run.smithi150.stdout:ID=ubuntu 2023-11-06T01:52:56.672 INFO:teuthology.orchestra.run.smithi150.stdout:ID_LIKE=debian 2023-11-06T01:52:56.672 INFO:teuthology.orchestra.run.smithi150.stdout:HOME_URL="https://www.ubuntu.com/" 2023-11-06T01:52:56.672 INFO:teuthology.orchestra.run.smithi150.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-11-06T01:52:56.672 INFO:teuthology.orchestra.run.smithi150.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-11-06T01:52:56.673 INFO:teuthology.orchestra.run.smithi150.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-11-06T01:52:56.673 INFO:teuthology.orchestra.run.smithi150.stdout:UBUNTU_CODENAME=jammy 2023-11-06T01:52:56.673 INFO:teuthology.lock.ops:Updating smithi150.front.sepia.ceph.com on lock server 2023-11-06T01:52:56.693 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-11-06T01:52:56.703 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-11-06T01:52:56.708 INFO:teuthology.task.internal:Checking for old test directory... 2023-11-06T01:52:56.709 DEBUG:teuthology.orchestra.run.smithi049:> test '!' -e /home/ubuntu/cephtest 2023-11-06T01:52:56.712 DEBUG:teuthology.orchestra.run.smithi150:> test '!' -e /home/ubuntu/cephtest 2023-11-06T01:52:56.718 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-11-06T01:52:56.724 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-11-06T01:52:56.724 DEBUG:teuthology.orchestra.run.smithi049:> test -z $(ls -A /var/lib/ceph) 2023-11-06T01:52:56.760 DEBUG:teuthology.orchestra.run.smithi150:> test -z $(ls -A /var/lib/ceph) 2023-11-06T01:52:56.847 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-11-06T01:52:56.901 INFO:teuthology.run_tasks:Running task kernel... 2023-11-06T01:52:56.916 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2023-11-06T01:52:56.916 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2023-11-06T01:52:56.917 DEBUG:teuthology.orchestra.run.smithi049:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-06T01:52:56.918 DEBUG:teuthology.orchestra.run.smithi150:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-06T01:52:56.922 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T01:52:56.923 DEBUG:teuthology.orchestra.run.smithi049:> uname -r 2023-11-06T01:52:56.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T01:52:56.924 DEBUG:teuthology.orchestra.run.smithi150:> uname -r 2023-11-06T01:52:56.971 INFO:teuthology.orchestra.run.smithi150.stdout:5.15.0-56-generic 2023-11-06T01:52:56.971 INFO:teuthology.task.kernel:Running kernel on smithi150: 5.15.0-56-generic 2023-11-06T01:52:56.971 DEBUG:teuthology.orchestra.run.smithi150:> sudo apt-get clean 2023-11-06T01:52:56.972 INFO:teuthology.orchestra.run.smithi049.stdout:5.15.0-56-generic 2023-11-06T01:52:56.973 INFO:teuthology.task.kernel:Running kernel on smithi049: 5.15.0-56-generic 2023-11-06T01:52:56.973 DEBUG:teuthology.orchestra.run.smithi049:> sudo apt-get clean 2023-11-06T01:52:57.084 DEBUG:teuthology.orchestra.run.smithi049:> sudo apt-get update 2023-11-06T01:52:57.109 DEBUG:teuthology.orchestra.run.smithi150:> sudo apt-get update 2023-11-06T01:52:57.251 INFO:teuthology.orchestra.run.smithi150.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2023-11-06T01:52:57.252 INFO:teuthology.orchestra.run.smithi150.stdout:Get:2 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2023-11-06T01:52:57.258 INFO:teuthology.orchestra.run.smithi150.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2023-11-06T01:52:57.274 INFO:teuthology.orchestra.run.smithi049.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2023-11-06T01:52:57.275 INFO:teuthology.orchestra.run.smithi049.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2023-11-06T01:52:57.275 INFO:teuthology.orchestra.run.smithi049.stdout:Get:3 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2023-11-06T01:52:57.333 INFO:teuthology.orchestra.run.smithi049.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2023-11-06T01:52:57.342 INFO:teuthology.orchestra.run.smithi150.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2023-11-06T01:52:57.482 INFO:teuthology.orchestra.run.smithi150.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [940 kB] 2023-11-06T01:52:57.538 INFO:teuthology.orchestra.run.smithi049.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [940 kB] 2023-11-06T01:52:57.553 INFO:teuthology.orchestra.run.smithi150.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [524 kB] 2023-11-06T01:52:57.584 INFO:teuthology.orchestra.run.smithi150.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [360 kB] 2023-11-06T01:52:57.590 INFO:teuthology.orchestra.run.smithi150.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [186 kB] 2023-11-06T01:52:57.592 INFO:teuthology.orchestra.run.smithi150.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2023-11-06T01:52:57.599 INFO:teuthology.orchestra.run.smithi150.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,085 kB] 2023-11-06T01:52:57.611 INFO:teuthology.orchestra.run.smithi049.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [524 kB] 2023-11-06T01:52:57.613 INFO:teuthology.orchestra.run.smithi150.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [32.4 kB] 2023-11-06T01:52:57.613 INFO:teuthology.orchestra.run.smithi150.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [176 kB] 2023-11-06T01:52:57.616 INFO:teuthology.orchestra.run.smithi150.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2023-11-06T01:52:57.617 INFO:teuthology.orchestra.run.smithi150.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [563 kB] 2023-11-06T01:52:57.619 INFO:teuthology.orchestra.run.smithi150.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,149 kB] 2023-11-06T01:52:57.640 INFO:teuthology.orchestra.run.smithi150.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [797 kB] 2023-11-06T01:52:57.642 INFO:teuthology.orchestra.run.smithi049.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [360 kB] 2023-11-06T01:52:57.645 INFO:teuthology.orchestra.run.smithi150.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [146 kB] 2023-11-06T01:52:57.646 INFO:teuthology.orchestra.run.smithi150.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2023-11-06T01:52:57.647 INFO:teuthology.orchestra.run.smithi150.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,032 B] 2023-11-06T01:52:57.647 INFO:teuthology.orchestra.run.smithi150.stdout:Get:20 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [36.5 kB] 2023-11-06T01:52:57.649 INFO:teuthology.orchestra.run.smithi049.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [186 kB] 2023-11-06T01:52:57.653 INFO:teuthology.orchestra.run.smithi049.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2023-11-06T01:52:57.653 INFO:teuthology.orchestra.run.smithi049.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [32.4 kB] 2023-11-06T01:52:57.654 INFO:teuthology.orchestra.run.smithi049.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,085 kB] 2023-11-06T01:52:57.654 INFO:teuthology.orchestra.run.smithi150.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [245 kB] 2023-11-06T01:52:57.655 INFO:teuthology.orchestra.run.smithi150.stdout:Get:22 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,060 B] 2023-11-06T01:52:57.655 INFO:teuthology.orchestra.run.smithi150.stdout:Get:23 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2023-11-06T01:52:57.660 INFO:teuthology.orchestra.run.smithi150.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2023-11-06T01:52:57.660 INFO:teuthology.orchestra.run.smithi150.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [32.8 kB] 2023-11-06T01:52:57.661 INFO:teuthology.orchestra.run.smithi150.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,105 kB] 2023-11-06T01:52:57.668 INFO:teuthology.orchestra.run.smithi049.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,149 kB] 2023-11-06T01:52:57.676 INFO:teuthology.orchestra.run.smithi049.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [176 kB] 2023-11-06T01:52:57.677 INFO:teuthology.orchestra.run.smithi150.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [179 kB] 2023-11-06T01:52:57.680 INFO:teuthology.orchestra.run.smithi150.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2023-11-06T01:52:57.680 INFO:teuthology.orchestra.run.smithi150.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [999 kB] 2023-11-06T01:52:57.680 INFO:teuthology.orchestra.run.smithi049.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2023-11-06T01:52:57.681 INFO:teuthology.orchestra.run.smithi049.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [563 kB] 2023-11-06T01:52:57.693 INFO:teuthology.orchestra.run.smithi049.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [797 kB] 2023-11-06T01:52:57.697 INFO:teuthology.orchestra.run.smithi150.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [661 kB] 2023-11-06T01:52:57.699 INFO:teuthology.orchestra.run.smithi049.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [245 kB] 2023-11-06T01:52:57.702 INFO:teuthology.orchestra.run.smithi049.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2023-11-06T01:52:57.709 INFO:teuthology.orchestra.run.smithi150.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [219 kB] 2023-11-06T01:52:57.710 INFO:teuthology.orchestra.run.smithi150.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2023-11-06T01:52:57.711 INFO:teuthology.orchestra.run.smithi150.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [41.6 kB] 2023-11-06T01:52:57.712 INFO:teuthology.orchestra.run.smithi150.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [3,888 B] 2023-11-06T01:52:57.712 INFO:teuthology.orchestra.run.smithi150.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [9,768 B] 2023-11-06T01:52:57.730 INFO:teuthology.orchestra.run.smithi049.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,105 kB] 2023-11-06T01:52:57.730 INFO:teuthology.orchestra.run.smithi049.stdout:Get:20 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [146 kB] 2023-11-06T01:52:57.731 INFO:teuthology.orchestra.run.smithi150.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2023-11-06T01:52:57.731 INFO:teuthology.orchestra.run.smithi150.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [56.3 kB] 2023-11-06T01:52:57.731 INFO:teuthology.orchestra.run.smithi150.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [64.2 kB] 2023-11-06T01:52:57.733 INFO:teuthology.orchestra.run.smithi150.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [10.5 kB] 2023-11-06T01:52:57.733 INFO:teuthology.orchestra.run.smithi150.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2023-11-06T01:52:57.733 INFO:teuthology.orchestra.run.smithi049.stdout:Get:21 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2023-11-06T01:52:57.734 INFO:teuthology.orchestra.run.smithi049.stdout:Get:22 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [36.5 kB] 2023-11-06T01:52:57.734 INFO:teuthology.orchestra.run.smithi049.stdout:Get:23 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,032 B] 2023-11-06T01:52:57.734 INFO:teuthology.orchestra.run.smithi049.stdout:Get:24 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,060 B] 2023-11-06T01:52:57.734 INFO:teuthology.orchestra.run.smithi049.stdout:Get:25 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2023-11-06T01:52:57.738 INFO:teuthology.orchestra.run.smithi049.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [32.8 kB] 2023-11-06T01:52:57.738 INFO:teuthology.orchestra.run.smithi049.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [179 kB] 2023-11-06T01:52:57.740 INFO:teuthology.orchestra.run.smithi049.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2023-11-06T01:52:57.740 INFO:teuthology.orchestra.run.smithi049.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [661 kB] 2023-11-06T01:52:57.744 INFO:teuthology.orchestra.run.smithi049.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [999 kB] 2023-11-06T01:52:57.750 INFO:teuthology.orchestra.run.smithi150.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [27.8 kB] 2023-11-06T01:52:57.751 INFO:teuthology.orchestra.run.smithi150.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [16.9 kB] 2023-11-06T01:52:57.751 INFO:teuthology.orchestra.run.smithi150.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.4 kB] 2023-11-06T01:52:57.751 INFO:teuthology.orchestra.run.smithi150.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2023-11-06T01:52:57.752 INFO:teuthology.orchestra.run.smithi049.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [219 kB] 2023-11-06T01:52:57.753 INFO:teuthology.orchestra.run.smithi049.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2023-11-06T01:52:57.762 INFO:teuthology.orchestra.run.smithi049.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [41.6 kB] 2023-11-06T01:52:57.763 INFO:teuthology.orchestra.run.smithi049.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [3,888 B] 2023-11-06T01:52:57.763 INFO:teuthology.orchestra.run.smithi049.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [9,768 B] 2023-11-06T01:52:57.764 INFO:teuthology.orchestra.run.smithi049.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2023-11-06T01:52:57.764 INFO:teuthology.orchestra.run.smithi049.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [56.3 kB] 2023-11-06T01:52:57.764 INFO:teuthology.orchestra.run.smithi049.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [64.2 kB] 2023-11-06T01:52:57.766 INFO:teuthology.orchestra.run.smithi049.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [10.5 kB] 2023-11-06T01:52:57.783 INFO:teuthology.orchestra.run.smithi049.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2023-11-06T01:52:57.783 INFO:teuthology.orchestra.run.smithi049.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [16.9 kB] 2023-11-06T01:52:57.784 INFO:teuthology.orchestra.run.smithi049.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [27.8 kB] 2023-11-06T01:52:57.784 INFO:teuthology.orchestra.run.smithi049.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.4 kB] 2023-11-06T01:52:57.784 INFO:teuthology.orchestra.run.smithi049.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2023-11-06T01:53:03.080 INFO:teuthology.orchestra.run.smithi150.stdout:Fetched 10.1 MB in 1s (6,810 kB/s) 2023-11-06T01:53:03.486 INFO:teuthology.orchestra.run.smithi049.stdout:Fetched 10.1 MB in 2s (6,300 kB/s) 2023-11-06T01:53:04.285 INFO:teuthology.orchestra.run.smithi150.stdout:Reading package lists... 2023-11-06T01:53:04.301 DEBUG:teuthology.orchestra.run.smithi150:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2023-11-06T01:53:04.360 INFO:teuthology.orchestra.run.smithi150.stdout:Reading package lists... 2023-11-06T01:53:04.535 INFO:teuthology.orchestra.run.smithi150.stdout:Building dependency tree... 2023-11-06T01:53:04.535 INFO:teuthology.orchestra.run.smithi150.stdout:Reading state information... 2023-11-06T01:53:04.679 INFO:teuthology.orchestra.run.smithi150.stdout:The following packages were automatically installed and are no longer required: 2023-11-06T01:53:04.679 INFO:teuthology.orchestra.run.smithi150.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-11-06T01:53:04.679 INFO:teuthology.orchestra.run.smithi150.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-11-06T01:53:04.680 INFO:teuthology.orchestra.run.smithi150.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-11-06T01:53:04.680 INFO:teuthology.orchestra.run.smithi150.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2023-11-06T01:53:04.680 INFO:teuthology.orchestra.run.smithi150.stdout: python2.7-minimal 2023-11-06T01:53:04.680 INFO:teuthology.orchestra.run.smithi150.stdout:Use 'sudo apt autoremove' to remove them. 2023-11-06T01:53:04.681 INFO:teuthology.orchestra.run.smithi150.stdout:The following additional packages will be installed: 2023-11-06T01:53:04.681 INFO:teuthology.orchestra.run.smithi150.stdout: linux-generic linux-headers-5.15.0-88 linux-headers-5.15.0-88-generic 2023-11-06T01:53:04.682 INFO:teuthology.orchestra.run.smithi150.stdout: linux-headers-generic linux-image-5.15.0-88-generic 2023-11-06T01:53:04.682 INFO:teuthology.orchestra.run.smithi150.stdout: linux-modules-5.15.0-88-generic linux-modules-extra-5.15.0-88-generic 2023-11-06T01:53:04.682 INFO:teuthology.orchestra.run.smithi150.stdout:Suggested packages: 2023-11-06T01:53:04.682 INFO:teuthology.orchestra.run.smithi150.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2023-11-06T01:53:04.683 INFO:teuthology.orchestra.run.smithi150.stdout:Recommended packages: 2023-11-06T01:53:04.683 INFO:teuthology.orchestra.run.smithi150.stdout: thermald 2023-11-06T01:53:04.732 INFO:teuthology.orchestra.run.smithi150.stdout:The following NEW packages will be installed: 2023-11-06T01:53:04.732 INFO:teuthology.orchestra.run.smithi150.stdout: linux-headers-5.15.0-88 linux-headers-5.15.0-88-generic 2023-11-06T01:53:04.733 INFO:teuthology.orchestra.run.smithi150.stdout: linux-image-5.15.0-88-generic linux-modules-5.15.0-88-generic 2023-11-06T01:53:04.733 INFO:teuthology.orchestra.run.smithi150.stdout: linux-modules-extra-5.15.0-88-generic 2023-11-06T01:53:04.734 INFO:teuthology.orchestra.run.smithi150.stdout:The following packages will be upgraded: 2023-11-06T01:53:04.734 INFO:teuthology.orchestra.run.smithi150.stdout: linux-generic linux-headers-generic linux-image-generic 2023-11-06T01:53:04.786 INFO:teuthology.orchestra.run.smithi150.stdout:3 upgraded, 5 newly installed, 0 to remove and 270 not upgraded. 2023-11-06T01:53:04.787 INFO:teuthology.orchestra.run.smithi150.stdout:Need to get 113 MB of archives. 2023-11-06T01:53:04.787 INFO:teuthology.orchestra.run.smithi150.stdout:After this operation, 582 MB of additional disk space will be used. 2023-11-06T01:53:04.787 INFO:teuthology.orchestra.run.smithi150.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-88-generic amd64 5.15.0-88.98 [22.4 MB] 2023-11-06T01:53:04.844 INFO:teuthology.orchestra.run.smithi049.stdout:Reading package lists... 2023-11-06T01:53:04.861 DEBUG:teuthology.orchestra.run.smithi049:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2023-11-06T01:53:04.927 INFO:teuthology.orchestra.run.smithi049.stdout:Reading package lists... 2023-11-06T01:53:05.110 INFO:teuthology.orchestra.run.smithi049.stdout:Building dependency tree... 2023-11-06T01:53:05.110 INFO:teuthology.orchestra.run.smithi049.stdout:Reading state information... 2023-11-06T01:53:05.182 INFO:teuthology.orchestra.run.smithi150.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-88-generic amd64 5.15.0-88.98 [11.5 MB] 2023-11-06T01:53:05.261 INFO:teuthology.orchestra.run.smithi049.stdout:The following packages were automatically installed and are no longer required: 2023-11-06T01:53:05.262 INFO:teuthology.orchestra.run.smithi049.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-11-06T01:53:05.262 INFO:teuthology.orchestra.run.smithi049.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-11-06T01:53:05.263 INFO:teuthology.orchestra.run.smithi049.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-11-06T01:53:05.263 INFO:teuthology.orchestra.run.smithi049.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2023-11-06T01:53:05.263 INFO:teuthology.orchestra.run.smithi049.stdout: python2.7-minimal 2023-11-06T01:53:05.263 INFO:teuthology.orchestra.run.smithi049.stdout:Use 'sudo apt autoremove' to remove them. 2023-11-06T01:53:05.264 INFO:teuthology.orchestra.run.smithi049.stdout:The following additional packages will be installed: 2023-11-06T01:53:05.264 INFO:teuthology.orchestra.run.smithi049.stdout: linux-generic linux-headers-5.15.0-88 linux-headers-5.15.0-88-generic 2023-11-06T01:53:05.264 INFO:teuthology.orchestra.run.smithi049.stdout: linux-headers-generic linux-image-5.15.0-88-generic 2023-11-06T01:53:05.265 INFO:teuthology.orchestra.run.smithi049.stdout: linux-modules-5.15.0-88-generic linux-modules-extra-5.15.0-88-generic 2023-11-06T01:53:05.265 INFO:teuthology.orchestra.run.smithi049.stdout:Suggested packages: 2023-11-06T01:53:05.265 INFO:teuthology.orchestra.run.smithi049.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2023-11-06T01:53:05.265 INFO:teuthology.orchestra.run.smithi049.stdout:Recommended packages: 2023-11-06T01:53:05.266 INFO:teuthology.orchestra.run.smithi049.stdout: thermald 2023-11-06T01:53:05.269 INFO:teuthology.orchestra.run.smithi150.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-88-generic amd64 5.15.0-88.98 [64.0 MB] 2023-11-06T01:53:05.311 INFO:teuthology.orchestra.run.smithi049.stdout:The following NEW packages will be installed: 2023-11-06T01:53:05.312 INFO:teuthology.orchestra.run.smithi049.stdout: linux-headers-5.15.0-88 linux-headers-5.15.0-88-generic 2023-11-06T01:53:05.312 INFO:teuthology.orchestra.run.smithi049.stdout: linux-image-5.15.0-88-generic linux-modules-5.15.0-88-generic 2023-11-06T01:53:05.312 INFO:teuthology.orchestra.run.smithi049.stdout: linux-modules-extra-5.15.0-88-generic 2023-11-06T01:53:05.313 INFO:teuthology.orchestra.run.smithi049.stdout:The following packages will be upgraded: 2023-11-06T01:53:05.313 INFO:teuthology.orchestra.run.smithi049.stdout: linux-generic linux-headers-generic linux-image-generic 2023-11-06T01:53:05.524 INFO:teuthology.orchestra.run.smithi049.stdout:3 upgraded, 5 newly installed, 0 to remove and 270 not upgraded. 2023-11-06T01:53:05.524 INFO:teuthology.orchestra.run.smithi049.stdout:Need to get 113 MB of archives. 2023-11-06T01:53:05.525 INFO:teuthology.orchestra.run.smithi049.stdout:After this operation, 582 MB of additional disk space will be used. 2023-11-06T01:53:05.525 INFO:teuthology.orchestra.run.smithi049.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-88-generic amd64 5.15.0-88.98 [22.4 MB] 2023-11-06T01:53:06.045 INFO:teuthology.orchestra.run.smithi150.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.88.85 [1,698 B] 2023-11-06T01:53:06.045 INFO:teuthology.orchestra.run.smithi150.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.88.85 [2,488 B] 2023-11-06T01:53:06.045 INFO:teuthology.orchestra.run.smithi150.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-88 all 5.15.0-88.98 [12.3 MB] 2023-11-06T01:53:06.163 INFO:teuthology.orchestra.run.smithi150.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-88-generic amd64 5.15.0-88.98 [2,878 kB] 2023-11-06T01:53:06.184 INFO:teuthology.orchestra.run.smithi150.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.88.85 [2,340 B] 2023-11-06T01:53:06.551 INFO:teuthology.orchestra.run.smithi150.stdout:Fetched 113 MB in 1s (78.3 MB/s) 2023-11-06T01:53:06.706 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package linux-modules-5.15.0-88-generic. 2023-11-06T01:53:07.170 INFO:teuthology.orchestra.run.smithi049.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-88-generic amd64 5.15.0-88.98 [11.5 MB] 2023-11-06T01:53:07.570 INFO:teuthology.orchestra.run.smithi049.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-88-generic amd64 5.15.0-88.98 [64.0 MB] 2023-11-06T01:53:08.762 INFO:teuthology.orchestra.run.smithi150.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2023-11-06T01:53:08.766 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../0-linux-modules-5.15.0-88-generic_5.15.0-88.98_amd64.deb ... 2023-11-06T01:53:08.820 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking linux-modules-5.15.0-88-generic (5.15.0-88.98) ... 2023-11-06T01:53:10.548 INFO:teuthology.orchestra.run.smithi049.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.88.85 [1,698 B] 2023-11-06T01:53:10.548 INFO:teuthology.orchestra.run.smithi049.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.88.85 [2,488 B] 2023-11-06T01:53:10.549 INFO:teuthology.orchestra.run.smithi049.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-88 all 5.15.0-88.98 [12.3 MB] 2023-11-06T01:53:10.796 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package linux-image-5.15.0-88-generic. 2023-11-06T01:53:10.817 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../1-linux-image-5.15.0-88-generic_5.15.0-88.98_amd64.deb ... 2023-11-06T01:53:10.896 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking linux-image-5.15.0-88-generic (5.15.0-88.98) ... 2023-11-06T01:53:10.983 INFO:teuthology.orchestra.run.smithi049.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-88-generic amd64 5.15.0-88.98 [2,878 kB] 2023-11-06T01:53:11.079 INFO:teuthology.orchestra.run.smithi049.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.88.85 [2,340 B] 2023-11-06T01:53:11.324 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-88-generic. 2023-11-06T01:53:11.345 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-88-generic_5.15.0-88.98_amd64.deb ... 2023-11-06T01:53:11.382 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking linux-modules-extra-5.15.0-88-generic (5.15.0-88.98) ... 2023-11-06T01:53:11.500 INFO:teuthology.orchestra.run.smithi049.stdout:Fetched 113 MB in 6s (19.6 MB/s) 2023-11-06T01:53:11.713 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package linux-modules-5.15.0-88-generic. 2023-11-06T01:53:13.787 INFO:teuthology.orchestra.run.smithi049.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2023-11-06T01:53:13.792 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../0-linux-modules-5.15.0-88-generic_5.15.0-88.98_amd64.deb ... 2023-11-06T01:53:13.919 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking linux-modules-5.15.0-88-generic (5.15.0-88.98) ... 2023-11-06T01:53:16.088 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package linux-image-5.15.0-88-generic. 2023-11-06T01:53:16.109 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../1-linux-image-5.15.0-88-generic_5.15.0-88.98_amd64.deb ... 2023-11-06T01:53:16.212 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking linux-image-5.15.0-88-generic (5.15.0-88.98) ... 2023-11-06T01:53:16.674 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-88-generic. 2023-11-06T01:53:16.695 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-88-generic_5.15.0-88.98_amd64.deb ... 2023-11-06T01:53:16.807 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking linux-modules-extra-5.15.0-88-generic (5.15.0-88.98) ... 2023-11-06T01:53:17.700 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../3-linux-generic_5.15.0.88.85_amd64.deb ... 2023-11-06T01:53:17.810 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking linux-generic (5.15.0.88.85) over (5.15.0.56.54) ... 2023-11-06T01:53:18.143 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.88.85_amd64.deb ... 2023-11-06T01:53:18.264 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking linux-image-generic (5.15.0.88.85) over (5.15.0.56.54) ... 2023-11-06T01:53:18.559 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package linux-headers-5.15.0-88. 2023-11-06T01:53:18.581 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../5-linux-headers-5.15.0-88_5.15.0-88.98_all.deb ... 2023-11-06T01:53:18.625 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking linux-headers-5.15.0-88 (5.15.0-88.98) ... 2023-11-06T01:53:23.913 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../3-linux-generic_5.15.0.88.85_amd64.deb ... 2023-11-06T01:53:24.012 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking linux-generic (5.15.0.88.85) over (5.15.0.56.54) ... 2023-11-06T01:53:24.346 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.88.85_amd64.deb ... 2023-11-06T01:53:24.474 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking linux-image-generic (5.15.0.88.85) over (5.15.0.56.54) ... 2023-11-06T01:53:24.488 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package linux-headers-5.15.0-88-generic. 2023-11-06T01:53:24.525 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../6-linux-headers-5.15.0-88-generic_5.15.0-88.98_amd64.deb ... 2023-11-06T01:53:24.563 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking linux-headers-5.15.0-88-generic (5.15.0-88.98) ... 2023-11-06T01:53:24.785 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package linux-headers-5.15.0-88. 2023-11-06T01:53:24.808 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../5-linux-headers-5.15.0-88_5.15.0-88.98_all.deb ... 2023-11-06T01:53:24.852 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking linux-headers-5.15.0-88 (5.15.0-88.98) ... 2023-11-06T01:53:27.116 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.88.85_amd64.deb ... 2023-11-06T01:53:27.242 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking linux-headers-generic (5.15.0.88.85) over (5.15.0.56.54) ... 2023-11-06T01:53:27.577 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up linux-headers-5.15.0-88 (5.15.0-88.98) ... 2023-11-06T01:53:27.697 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up linux-headers-5.15.0-88-generic (5.15.0-88.98) ... 2023-11-06T01:53:27.831 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up linux-headers-generic (5.15.0.88.85) ... 2023-11-06T01:53:27.960 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up linux-image-5.15.0-88-generic (5.15.0-88.98) ... 2023-11-06T01:53:29.321 INFO:teuthology.orchestra.run.smithi150.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2023-11-06T01:53:29.322 INFO:teuthology.orchestra.run.smithi150.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2023-11-06T01:53:29.322 INFO:teuthology.orchestra.run.smithi150.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-88-generic 2023-11-06T01:53:29.322 INFO:teuthology.orchestra.run.smithi150.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-88-generic 2023-11-06T01:53:29.612 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up linux-modules-extra-5.15.0-88-generic (5.15.0-88.98) ... 2023-11-06T01:53:30.550 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package linux-headers-5.15.0-88-generic. 2023-11-06T01:53:30.587 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../6-linux-headers-5.15.0-88-generic_5.15.0-88.98_amd64.deb ... 2023-11-06T01:53:30.624 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking linux-headers-5.15.0-88-generic (5.15.0-88.98) ... 2023-11-06T01:53:31.004 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up linux-modules-5.15.0-88-generic (5.15.0-88.98) ... 2023-11-06T01:53:32.414 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up linux-image-generic (5.15.0.88.85) ... 2023-11-06T01:53:32.540 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up linux-generic (5.15.0.88.85) ... 2023-11-06T01:53:32.666 INFO:teuthology.orchestra.run.smithi150.stdout:Processing triggers for linux-image-5.15.0-88-generic (5.15.0-88.98) ... 2023-11-06T01:53:32.728 INFO:teuthology.orchestra.run.smithi150.stdout:/etc/kernel/postinst.d/initramfs-tools: 2023-11-06T01:53:32.729 INFO:teuthology.orchestra.run.smithi150.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-88-generic 2023-11-06T01:53:34.630 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.88.85_amd64.deb ... 2023-11-06T01:53:34.768 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking linux-headers-generic (5.15.0.88.85) over (5.15.0.56.54) ... 2023-11-06T01:53:35.119 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up linux-headers-5.15.0-88 (5.15.0-88.98) ... 2023-11-06T01:53:35.215 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up linux-headers-5.15.0-88-generic (5.15.0-88.98) ... 2023-11-06T01:53:35.357 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up linux-headers-generic (5.15.0.88.85) ... 2023-11-06T01:53:35.495 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up linux-image-5.15.0-88-generic (5.15.0-88.98) ... 2023-11-06T01:53:36.978 INFO:teuthology.orchestra.run.smithi049.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2023-11-06T01:53:36.978 INFO:teuthology.orchestra.run.smithi049.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2023-11-06T01:53:36.979 INFO:teuthology.orchestra.run.smithi049.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-88-generic 2023-11-06T01:53:36.979 INFO:teuthology.orchestra.run.smithi049.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-88-generic 2023-11-06T01:53:37.166 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up linux-modules-extra-5.15.0-88-generic (5.15.0-88.98) ... 2023-11-06T01:53:39.466 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up linux-modules-5.15.0-88-generic (5.15.0-88.98) ... 2023-11-06T01:53:40.935 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up linux-image-generic (5.15.0.88.85) ... 2023-11-06T01:53:41.063 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up linux-generic (5.15.0.88.85) ... 2023-11-06T01:53:41.220 INFO:teuthology.orchestra.run.smithi049.stdout:Processing triggers for linux-image-5.15.0-88-generic (5.15.0-88.98) ... 2023-11-06T01:53:41.294 INFO:teuthology.orchestra.run.smithi049.stdout:/etc/kernel/postinst.d/initramfs-tools: 2023-11-06T01:53:41.294 INFO:teuthology.orchestra.run.smithi049.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-88-generic 2023-11-06T01:53:46.326 INFO:teuthology.orchestra.run.smithi150.stdout:/etc/kernel/postinst.d/zz-update-grub: 2023-11-06T01:53:46.327 INFO:teuthology.orchestra.run.smithi150.stdout:Sourcing file `/etc/default/grub' 2023-11-06T01:53:46.342 INFO:teuthology.orchestra.run.smithi150.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-11-06T01:53:46.357 INFO:teuthology.orchestra.run.smithi150.stdout:Generating grub configuration file ... 2023-11-06T01:53:46.621 INFO:teuthology.orchestra.run.smithi150.stdout:Found linux image: /boot/vmlinuz-5.15.0-88-generic 2023-11-06T01:53:46.639 INFO:teuthology.orchestra.run.smithi150.stdout:Found initrd image: /boot/initrd.img-5.15.0-88-generic 2023-11-06T01:53:46.986 INFO:teuthology.orchestra.run.smithi150.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2023-11-06T01:53:46.992 INFO:teuthology.orchestra.run.smithi150.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2023-11-06T01:53:47.063 INFO:teuthology.orchestra.run.smithi150.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2023-11-06T01:53:47.068 INFO:teuthology.orchestra.run.smithi150.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2023-11-06T01:53:47.231 INFO:teuthology.orchestra.run.smithi150.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2023-11-06T01:53:47.231 INFO:teuthology.orchestra.run.smithi150.stdout:Systems on them will not be added to the GRUB boot configuration. 2023-11-06T01:53:47.232 INFO:teuthology.orchestra.run.smithi150.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2023-11-06T01:53:47.277 INFO:teuthology.orchestra.run.smithi150.stdout:done 2023-11-06T01:53:47.755 DEBUG:teuthology.orchestra.run.smithi150:> dpkg -s linux-image-generic 2023-11-06T01:53:47.777 INFO:teuthology.orchestra.run.smithi150.stdout:Package: linux-image-generic 2023-11-06T01:53:47.778 INFO:teuthology.orchestra.run.smithi150.stdout:Status: install ok installed 2023-11-06T01:53:47.778 INFO:teuthology.orchestra.run.smithi150.stdout:Priority: optional 2023-11-06T01:53:47.778 INFO:teuthology.orchestra.run.smithi150.stdout:Section: kernel 2023-11-06T01:53:47.778 INFO:teuthology.orchestra.run.smithi150.stdout:Installed-Size: 20 2023-11-06T01:53:47.779 INFO:teuthology.orchestra.run.smithi150.stdout:Maintainer: Ubuntu Kernel Team 2023-11-06T01:53:47.779 INFO:teuthology.orchestra.run.smithi150.stdout:Architecture: amd64 2023-11-06T01:53:47.779 INFO:teuthology.orchestra.run.smithi150.stdout:Source: linux-meta 2023-11-06T01:53:47.779 INFO:teuthology.orchestra.run.smithi150.stdout:Version: 5.15.0.88.85 2023-11-06T01:53:47.779 INFO:teuthology.orchestra.run.smithi150.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.1), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-88), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.1) 2023-11-06T01:53:47.780 INFO:teuthology.orchestra.run.smithi150.stdout:Depends: linux-image-5.15.0-88-generic, linux-modules-extra-5.15.0-88-generic, linux-firmware, intel-microcode, amd64-microcode 2023-11-06T01:53:47.780 INFO:teuthology.orchestra.run.smithi150.stdout:Recommends: thermald 2023-11-06T01:53:47.780 INFO:teuthology.orchestra.run.smithi150.stdout:Description: Generic Linux kernel image 2023-11-06T01:53:47.780 INFO:teuthology.orchestra.run.smithi150.stdout: This package will always depend on the latest generic kernel image 2023-11-06T01:53:47.780 INFO:teuthology.orchestra.run.smithi150.stdout: available. 2023-11-06T01:53:47.781 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-88-generic 2023-11-06T01:53:47.781 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-11-06T01:53:47.781 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-11-06T01:53:47.782 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2023-11-06T01:53:47.782 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi150.front.sepia.ceph.com, path=None, version=distro) 2023-11-06T01:53:47.782 DEBUG:teuthology.orchestra.run.smithi150:> sudo apt-get clean 2023-11-06T01:53:47.955 DEBUG:teuthology.orchestra.run.smithi150:> sudo apt-get update 2023-11-06T01:53:48.078 INFO:teuthology.orchestra.run.smithi150.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2023-11-06T01:53:48.196 INFO:teuthology.orchestra.run.smithi150.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2023-11-06T01:53:48.300 INFO:teuthology.orchestra.run.smithi150.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2023-11-06T01:53:48.364 INFO:teuthology.orchestra.run.smithi150.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2023-11-06T01:53:49.453 INFO:teuthology.orchestra.run.smithi150.stdout:Reading package lists... 2023-11-06T01:53:49.469 DEBUG:teuthology.orchestra.run.smithi150:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2023-11-06T01:53:49.522 INFO:teuthology.orchestra.run.smithi150.stdout:Reading package lists... 2023-11-06T01:53:49.686 INFO:teuthology.orchestra.run.smithi150.stdout:Building dependency tree... 2023-11-06T01:53:49.687 INFO:teuthology.orchestra.run.smithi150.stdout:Reading state information... 2023-11-06T01:53:49.830 INFO:teuthology.orchestra.run.smithi150.stdout:linux-image-generic is already the newest version (5.15.0.88.85). 2023-11-06T01:53:49.831 INFO:teuthology.orchestra.run.smithi150.stdout:The following packages were automatically installed and are no longer required: 2023-11-06T01:53:49.831 INFO:teuthology.orchestra.run.smithi150.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-11-06T01:53:49.831 INFO:teuthology.orchestra.run.smithi150.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-11-06T01:53:49.831 INFO:teuthology.orchestra.run.smithi150.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-11-06T01:53:49.831 INFO:teuthology.orchestra.run.smithi150.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2023-11-06T01:53:49.832 INFO:teuthology.orchestra.run.smithi150.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2023-11-06T01:53:49.832 INFO:teuthology.orchestra.run.smithi150.stdout:Use 'sudo apt autoremove' to remove them. 2023-11-06T01:53:49.867 INFO:teuthology.orchestra.run.smithi150.stdout:0 upgraded, 0 newly installed, 0 to remove and 270 not upgraded. 2023-11-06T01:53:49.869 DEBUG:teuthology.orchestra.run.smithi150:> dpkg -s linux-image-generic 2023-11-06T01:53:49.928 INFO:teuthology.orchestra.run.smithi150.stdout:Package: linux-image-generic 2023-11-06T01:53:49.929 INFO:teuthology.orchestra.run.smithi150.stdout:Status: install ok installed 2023-11-06T01:53:49.929 INFO:teuthology.orchestra.run.smithi150.stdout:Priority: optional 2023-11-06T01:53:49.929 INFO:teuthology.orchestra.run.smithi150.stdout:Section: kernel 2023-11-06T01:53:49.929 INFO:teuthology.orchestra.run.smithi150.stdout:Installed-Size: 20 2023-11-06T01:53:49.930 INFO:teuthology.orchestra.run.smithi150.stdout:Maintainer: Ubuntu Kernel Team 2023-11-06T01:53:49.930 INFO:teuthology.orchestra.run.smithi150.stdout:Architecture: amd64 2023-11-06T01:53:49.930 INFO:teuthology.orchestra.run.smithi150.stdout:Source: linux-meta 2023-11-06T01:53:49.930 INFO:teuthology.orchestra.run.smithi150.stdout:Version: 5.15.0.88.85 2023-11-06T01:53:49.931 INFO:teuthology.orchestra.run.smithi150.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.1), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-88), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.1) 2023-11-06T01:53:49.931 INFO:teuthology.orchestra.run.smithi150.stdout:Depends: linux-image-5.15.0-88-generic, linux-modules-extra-5.15.0-88-generic, linux-firmware, intel-microcode, amd64-microcode 2023-11-06T01:53:49.931 INFO:teuthology.orchestra.run.smithi150.stdout:Recommends: thermald 2023-11-06T01:53:49.931 INFO:teuthology.orchestra.run.smithi150.stdout:Description: Generic Linux kernel image 2023-11-06T01:53:49.931 INFO:teuthology.orchestra.run.smithi150.stdout: This package will always depend on the latest generic kernel image 2023-11-06T01:53:49.932 INFO:teuthology.orchestra.run.smithi150.stdout: available. 2023-11-06T01:53:49.932 DEBUG:teuthology.orchestra.run.smithi150:> mktemp 2023-11-06T01:53:49.975 INFO:teuthology.orchestra.run.smithi150.stdout:/tmp/tmp.3TDEdbOHMR 2023-11-06T01:53:49.975 DEBUG:teuthology.orchestra.run.smithi150:> sudo cp /boot/grub/grub.cfg /tmp/tmp.3TDEdbOHMR 2023-11-06T01:53:50.034 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 0666 /tmp/tmp.3TDEdbOHMR 2023-11-06T01:53:50.150 DEBUG:teuthology.orchestra.remote:smithi150:/tmp/tmp.3TDEdbOHMR is 10KB 2023-11-06T01:53:50.163 DEBUG:teuthology.orchestra.run.smithi150:> rm -fr /tmp/tmp.3TDEdbOHMR 2023-11-06T01:53:50.168 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2023-11-06T01:53:50.221 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-11-06T01:53:50.221 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/grub.d/01_ceph_kernel 2023-11-06T01:53:50.222 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2023-11-06T01:53:50.285 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-88-generic 2023-11-06T01:53:50.285 DEBUG:teuthology.orchestra.run.smithi150:> sudo update-grub 2023-11-06T01:53:51.390 INFO:teuthology.orchestra.run.smithi150.stderr:Sourcing file `/etc/default/grub' 2023-11-06T01:53:51.392 INFO:teuthology.orchestra.run.smithi150.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-11-06T01:53:51.394 INFO:teuthology.orchestra.run.smithi150.stderr:Generating grub configuration file ... 2023-11-06T01:53:51.602 INFO:teuthology.orchestra.run.smithi150.stderr:Found linux image: /boot/vmlinuz-5.15.0-88-generic 2023-11-06T01:53:51.608 INFO:teuthology.orchestra.run.smithi150.stderr:Found initrd image: /boot/initrd.img-5.15.0-88-generic 2023-11-06T01:53:51.841 INFO:teuthology.orchestra.run.smithi150.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2023-11-06T01:53:51.846 INFO:teuthology.orchestra.run.smithi150.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2023-11-06T01:53:51.885 INFO:teuthology.orchestra.run.smithi150.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2023-11-06T01:53:51.890 INFO:teuthology.orchestra.run.smithi150.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2023-11-06T01:53:52.047 INFO:teuthology.orchestra.run.smithi150.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2023-11-06T01:53:52.047 INFO:teuthology.orchestra.run.smithi150.stderr:Systems on them will not be added to the GRUB boot configuration. 2023-11-06T01:53:52.048 INFO:teuthology.orchestra.run.smithi150.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2023-11-06T01:53:52.059 INFO:teuthology.orchestra.run.smithi150.stderr:done 2023-11-06T01:53:52.067 DEBUG:teuthology.orchestra.run.smithi150:> sudo shutdown -r now 2023-11-06T01:53:55.824 INFO:teuthology.orchestra.run.smithi049.stdout:/etc/kernel/postinst.d/zz-update-grub: 2023-11-06T01:53:55.825 INFO:teuthology.orchestra.run.smithi049.stdout:Sourcing file `/etc/default/grub' 2023-11-06T01:53:55.844 INFO:teuthology.orchestra.run.smithi049.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-11-06T01:53:55.860 INFO:teuthology.orchestra.run.smithi049.stdout:Generating grub configuration file ... 2023-11-06T01:53:56.146 INFO:teuthology.orchestra.run.smithi049.stdout:Found linux image: /boot/vmlinuz-5.15.0-88-generic 2023-11-06T01:53:56.169 INFO:teuthology.orchestra.run.smithi049.stdout:Found initrd image: /boot/initrd.img-5.15.0-88-generic 2023-11-06T01:53:56.498 INFO:teuthology.orchestra.run.smithi049.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2023-11-06T01:53:56.504 INFO:teuthology.orchestra.run.smithi049.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2023-11-06T01:53:56.569 INFO:teuthology.orchestra.run.smithi049.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2023-11-06T01:53:56.575 INFO:teuthology.orchestra.run.smithi049.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2023-11-06T01:53:56.740 INFO:teuthology.orchestra.run.smithi049.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2023-11-06T01:53:56.740 INFO:teuthology.orchestra.run.smithi049.stdout:Systems on them will not be added to the GRUB boot configuration. 2023-11-06T01:53:56.740 INFO:teuthology.orchestra.run.smithi049.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2023-11-06T01:53:56.780 INFO:teuthology.orchestra.run.smithi049.stdout:done 2023-11-06T01:53:57.346 DEBUG:teuthology.orchestra.run.smithi049:> dpkg -s linux-image-generic 2023-11-06T01:53:57.372 INFO:teuthology.orchestra.run.smithi049.stdout:Package: linux-image-generic 2023-11-06T01:53:57.372 INFO:teuthology.orchestra.run.smithi049.stdout:Status: install ok installed 2023-11-06T01:53:57.373 INFO:teuthology.orchestra.run.smithi049.stdout:Priority: optional 2023-11-06T01:53:57.373 INFO:teuthology.orchestra.run.smithi049.stdout:Section: kernel 2023-11-06T01:53:57.373 INFO:teuthology.orchestra.run.smithi049.stdout:Installed-Size: 20 2023-11-06T01:53:57.373 INFO:teuthology.orchestra.run.smithi049.stdout:Maintainer: Ubuntu Kernel Team 2023-11-06T01:53:57.374 INFO:teuthology.orchestra.run.smithi049.stdout:Architecture: amd64 2023-11-06T01:53:57.374 INFO:teuthology.orchestra.run.smithi049.stdout:Source: linux-meta 2023-11-06T01:53:57.374 INFO:teuthology.orchestra.run.smithi049.stdout:Version: 5.15.0.88.85 2023-11-06T01:53:57.374 INFO:teuthology.orchestra.run.smithi049.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.1), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-88), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.1) 2023-11-06T01:53:57.374 INFO:teuthology.orchestra.run.smithi049.stdout:Depends: linux-image-5.15.0-88-generic, linux-modules-extra-5.15.0-88-generic, linux-firmware, intel-microcode, amd64-microcode 2023-11-06T01:53:57.375 INFO:teuthology.orchestra.run.smithi049.stdout:Recommends: thermald 2023-11-06T01:53:57.375 INFO:teuthology.orchestra.run.smithi049.stdout:Description: Generic Linux kernel image 2023-11-06T01:53:57.375 INFO:teuthology.orchestra.run.smithi049.stdout: This package will always depend on the latest generic kernel image 2023-11-06T01:53:57.375 INFO:teuthology.orchestra.run.smithi049.stdout: available. 2023-11-06T01:53:57.375 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-88-generic 2023-11-06T01:53:57.376 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-11-06T01:53:57.376 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-11-06T01:53:57.376 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2023-11-06T01:53:57.376 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi049.front.sepia.ceph.com, path=None, version=distro) 2023-11-06T01:53:57.377 DEBUG:teuthology.orchestra.run.smithi049:> sudo apt-get clean 2023-11-06T01:53:57.551 DEBUG:teuthology.orchestra.run.smithi049:> sudo apt-get update 2023-11-06T01:53:57.688 INFO:teuthology.orchestra.run.smithi049.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2023-11-06T01:53:57.815 INFO:teuthology.orchestra.run.smithi049.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2023-11-06T01:53:57.900 INFO:teuthology.orchestra.run.smithi049.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2023-11-06T01:53:57.987 INFO:teuthology.orchestra.run.smithi049.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2023-11-06T01:53:59.048 INFO:teuthology.orchestra.run.smithi049.stdout:Reading package lists... 2023-11-06T01:53:59.065 DEBUG:teuthology.orchestra.run.smithi049:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2023-11-06T01:53:59.124 INFO:teuthology.orchestra.run.smithi049.stdout:Reading package lists... 2023-11-06T01:53:59.316 INFO:teuthology.orchestra.run.smithi049.stdout:Building dependency tree... 2023-11-06T01:53:59.317 INFO:teuthology.orchestra.run.smithi049.stdout:Reading state information... 2023-11-06T01:53:59.486 INFO:teuthology.orchestra.run.smithi049.stdout:linux-image-generic is already the newest version (5.15.0.88.85). 2023-11-06T01:53:59.486 INFO:teuthology.orchestra.run.smithi049.stdout:The following packages were automatically installed and are no longer required: 2023-11-06T01:53:59.486 INFO:teuthology.orchestra.run.smithi049.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-11-06T01:53:59.486 INFO:teuthology.orchestra.run.smithi049.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-11-06T01:53:59.487 INFO:teuthology.orchestra.run.smithi049.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-11-06T01:53:59.487 INFO:teuthology.orchestra.run.smithi049.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2023-11-06T01:53:59.487 INFO:teuthology.orchestra.run.smithi049.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2023-11-06T01:53:59.488 INFO:teuthology.orchestra.run.smithi049.stdout:Use 'sudo apt autoremove' to remove them. 2023-11-06T01:53:59.528 INFO:teuthology.orchestra.run.smithi049.stdout:0 upgraded, 0 newly installed, 0 to remove and 270 not upgraded. 2023-11-06T01:53:59.530 DEBUG:teuthology.orchestra.run.smithi049:> dpkg -s linux-image-generic 2023-11-06T01:53:59.546 INFO:teuthology.orchestra.run.smithi049.stdout:Package: linux-image-generic 2023-11-06T01:53:59.547 INFO:teuthology.orchestra.run.smithi049.stdout:Status: install ok installed 2023-11-06T01:53:59.547 INFO:teuthology.orchestra.run.smithi049.stdout:Priority: optional 2023-11-06T01:53:59.547 INFO:teuthology.orchestra.run.smithi049.stdout:Section: kernel 2023-11-06T01:53:59.547 INFO:teuthology.orchestra.run.smithi049.stdout:Installed-Size: 20 2023-11-06T01:53:59.547 INFO:teuthology.orchestra.run.smithi049.stdout:Maintainer: Ubuntu Kernel Team 2023-11-06T01:53:59.548 INFO:teuthology.orchestra.run.smithi049.stdout:Architecture: amd64 2023-11-06T01:53:59.548 INFO:teuthology.orchestra.run.smithi049.stdout:Source: linux-meta 2023-11-06T01:53:59.548 INFO:teuthology.orchestra.run.smithi049.stdout:Version: 5.15.0.88.85 2023-11-06T01:53:59.548 INFO:teuthology.orchestra.run.smithi049.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.1), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-88), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.1) 2023-11-06T01:53:59.548 INFO:teuthology.orchestra.run.smithi049.stdout:Depends: linux-image-5.15.0-88-generic, linux-modules-extra-5.15.0-88-generic, linux-firmware, intel-microcode, amd64-microcode 2023-11-06T01:53:59.549 INFO:teuthology.orchestra.run.smithi049.stdout:Recommends: thermald 2023-11-06T01:53:59.549 INFO:teuthology.orchestra.run.smithi049.stdout:Description: Generic Linux kernel image 2023-11-06T01:53:59.549 INFO:teuthology.orchestra.run.smithi049.stdout: This package will always depend on the latest generic kernel image 2023-11-06T01:53:59.549 INFO:teuthology.orchestra.run.smithi049.stdout: available. 2023-11-06T01:53:59.550 DEBUG:teuthology.orchestra.run.smithi049:> mktemp 2023-11-06T01:53:59.594 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/tmp.hpwIc4BtiY 2023-11-06T01:53:59.595 DEBUG:teuthology.orchestra.run.smithi049:> sudo cp /boot/grub/grub.cfg /tmp/tmp.hpwIc4BtiY 2023-11-06T01:53:59.650 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod 0666 /tmp/tmp.hpwIc4BtiY 2023-11-06T01:53:59.775 DEBUG:teuthology.orchestra.remote:smithi049:/tmp/tmp.hpwIc4BtiY is 10KB 2023-11-06T01:53:59.787 DEBUG:teuthology.orchestra.run.smithi049:> rm -fr /tmp/tmp.hpwIc4BtiY 2023-11-06T01:53:59.792 DEBUG:teuthology.orchestra.run.smithi049:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2023-11-06T01:53:59.845 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-11-06T01:53:59.845 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/etc/grub.d/01_ceph_kernel 2023-11-06T01:53:59.846 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2023-11-06T01:53:59.938 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-88-generic 2023-11-06T01:53:59.939 DEBUG:teuthology.orchestra.run.smithi049:> sudo update-grub 2023-11-06T01:54:01.356 INFO:teuthology.orchestra.run.smithi049.stderr:Sourcing file `/etc/default/grub' 2023-11-06T01:54:01.357 INFO:teuthology.orchestra.run.smithi049.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-11-06T01:54:01.364 INFO:teuthology.orchestra.run.smithi049.stderr:Generating grub configuration file ... 2023-11-06T01:54:01.558 INFO:teuthology.orchestra.run.smithi049.stderr:Found linux image: /boot/vmlinuz-5.15.0-88-generic 2023-11-06T01:54:01.564 INFO:teuthology.orchestra.run.smithi049.stderr:Found initrd image: /boot/initrd.img-5.15.0-88-generic 2023-11-06T01:54:01.864 INFO:teuthology.orchestra.run.smithi049.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2023-11-06T01:54:01.869 INFO:teuthology.orchestra.run.smithi049.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2023-11-06T01:54:01.911 INFO:teuthology.orchestra.run.smithi049.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2023-11-06T01:54:01.916 INFO:teuthology.orchestra.run.smithi049.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2023-11-06T01:54:02.043 INFO:teuthology.orchestra.run.smithi049.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2023-11-06T01:54:02.044 INFO:teuthology.orchestra.run.smithi049.stderr:Systems on them will not be added to the GRUB boot configuration. 2023-11-06T01:54:02.044 INFO:teuthology.orchestra.run.smithi049.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2023-11-06T01:54:02.071 INFO:teuthology.orchestra.run.smithi049.stderr:done 2023-11-06T01:54:02.073 DEBUG:teuthology.orchestra.run.smithi049:> sudo shutdown -r now 2023-11-06T01:54:22.085 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2023-11-06T01:54:22.086 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi150.front.sepia.ceph.com' 2023-11-06T01:54:22.087 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-06T01:54:32.128 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-11-06T01:54:32.128 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi049.front.sepia.ceph.com' 2023-11-06T01:54:32.129 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-06T01:54:56.766 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.150 2023-11-06T01:55:05.774 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi150.front.sepia.ceph.com' 2023-11-06T01:55:05.775 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-06T01:55:06.746 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.49 2023-11-06T01:55:15.754 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi049.front.sepia.ceph.com' 2023-11-06T01:55:15.755 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-06T01:55:18.810 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.49 2023-11-06T01:55:30.822 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi049.front.sepia.ceph.com' 2023-11-06T01:55:30.823 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-06T01:55:33.882 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.49 2023-11-06T01:55:48.897 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi049.front.sepia.ceph.com' 2023-11-06T01:55:48.898 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-06T01:55:51.962 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.49 2023-11-06T01:56:05.790 DEBUG:teuthology.orchestra.remote:timed out 2023-11-06T01:56:09.966 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi049.front.sepia.ceph.com' 2023-11-06T01:56:09.967 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-06T01:56:17.798 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi150.front.sepia.ceph.com' 2023-11-06T01:56:17.801 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-06T01:56:18.130 DEBUG:teuthology.orchestra.run.smithi150:> true 2023-11-06T01:56:18.672 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi150.front.sepia.ceph.com' 2023-11-06T01:56:18.673 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.15.0-88-generic"... 2023-11-06T01:56:18.673 DEBUG:teuthology.orchestra.run.smithi150:> uname -r 2023-11-06T01:56:18.720 INFO:teuthology.orchestra.run.smithi150.stdout:5.15.0-88-generic 2023-11-06T01:56:18.720 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-88-generic vs 5.15.0-88-generic 2023-11-06T01:56:18.721 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-11-06T01:56:18.721 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2023-11-06T01:56:19.722 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2023-11-06T01:56:19.722 DEBUG:teuthology.orchestra.run.smithi150:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-11-06T01:56:20.060 INFO:teuthology.orchestra.run.smithi150.stdout:ttyS1 2023-11-06T01:56:20.082 DEBUG:teuthology.parallel:result is None 2023-11-06T01:56:28.414 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.49 2023-11-06T01:56:49.415 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi049.front.sepia.ceph.com' 2023-11-06T01:56:49.416 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-06T01:56:49.714 DEBUG:teuthology.orchestra.run.smithi049:> true 2023-11-06T01:56:50.314 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi049.front.sepia.ceph.com' 2023-11-06T01:56:50.314 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-88-generic"... 2023-11-06T01:56:50.314 DEBUG:teuthology.orchestra.run.smithi049:> uname -r 2023-11-06T01:56:50.363 INFO:teuthology.orchestra.run.smithi049.stdout:5.15.0-88-generic 2023-11-06T01:56:50.363 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-88-generic vs 5.15.0-88-generic 2023-11-06T01:56:50.364 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-11-06T01:56:50.364 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2023-11-06T01:56:51.365 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-11-06T01:56:51.366 DEBUG:teuthology.orchestra.run.smithi049:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-11-06T01:56:51.434 INFO:teuthology.orchestra.run.smithi049.stdout:ttyS1 2023-11-06T01:56:51.472 DEBUG:teuthology.parallel:result is None 2023-11-06T01:56:51.473 INFO:teuthology.run_tasks:Running task internal.base... 2023-11-06T01:56:51.482 INFO:teuthology.task.internal:Creating test directory... 2023-11-06T01:56:51.483 DEBUG:teuthology.orchestra.run.smithi049:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-06T01:56:51.485 DEBUG:teuthology.orchestra.run.smithi150:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-06T01:56:51.492 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-11-06T01:56:51.499 INFO:teuthology.run_tasks:Running task internal.archive... 2023-11-06T01:56:51.506 INFO:teuthology.task.internal:Creating archive directory... 2023-11-06T01:56:51.506 DEBUG:teuthology.orchestra.run.smithi049:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-06T01:56:51.532 DEBUG:teuthology.orchestra.run.smithi150:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-06T01:56:51.638 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-11-06T01:56:51.645 INFO:teuthology.task.internal:Enabling coredump saving... 2023-11-06T01:56:51.646 DEBUG:teuthology.orchestra.run.smithi049:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-06T01:56:51.648 DEBUG:teuthology.orchestra.run.smithi150:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-06T01:56:51.669 INFO:teuthology.orchestra.run.smithi150.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-06T01:56:51.670 INFO:teuthology.orchestra.run.smithi049.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-06T01:56:51.677 INFO:teuthology.orchestra.run.smithi150.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-06T01:56:51.678 INFO:teuthology.orchestra.run.smithi049.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-06T01:56:51.680 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-11-06T01:56:51.687 INFO:teuthology.task.internal:Configuring sudo... 2023-11-06T01:56:51.687 DEBUG:teuthology.orchestra.run.smithi049:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-06T01:56:51.724 DEBUG:teuthology.orchestra.run.smithi150:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-06T01:56:51.752 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-11-06T01:56:51.760 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-11-06T01:56:51.761 DEBUG:teuthology.orchestra.run.smithi049:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-06T01:56:51.780 DEBUG:teuthology.orchestra.run.smithi150:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-06T01:56:51.804 DEBUG:teuthology.orchestra.run.smithi049:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-06T01:56:51.831 DEBUG:teuthology.orchestra.run.smithi049:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-06T01:56:51.879 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-11-06T01:56:51.879 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-06T01:56:51.942 DEBUG:teuthology.orchestra.run.smithi150:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-06T01:56:51.948 DEBUG:teuthology.orchestra.run.smithi150:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-06T01:56:51.996 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-11-06T01:56:51.996 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-06T01:56:52.056 DEBUG:teuthology.orchestra.run.smithi049:> sudo service rsyslog restart 2023-11-06T01:56:52.059 DEBUG:teuthology.orchestra.run.smithi150:> sudo service rsyslog restart 2023-11-06T01:56:52.167 INFO:teuthology.run_tasks:Running task internal.timer... 2023-11-06T01:56:52.174 INFO:teuthology.task.internal:Starting timer... 2023-11-06T01:56:52.175 INFO:teuthology.run_tasks:Running task pcp... 2023-11-06T01:56:52.184 INFO:teuthology.run_tasks:Running task selinux... 2023-11-06T01:56:52.215 DEBUG:teuthology.task.selinux:Excluding smithi049: OS 'ubuntu' does not support SELinux 2023-11-06T01:56:52.216 DEBUG:teuthology.task.selinux:Excluding smithi150: OS 'ubuntu' does not support SELinux 2023-11-06T01:56:52.216 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-11-06T01:56:52.217 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2023-11-06T01:56:52.217 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-11-06T01:56:52.217 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-11-06T01:56:52.227 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-11-06T01:56:52.227 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-11-06T01:56:52.245 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-11-06T01:56:52.246 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi049.front.sepia.ceph.com,smithi150.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-11-06T02:04:40.398 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi049.front.sepia.ceph.com'), Remote(name='ubuntu@smithi150.front.sepia.ceph.com')] 2023-11-06T02:04:40.400 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi049.front.sepia.ceph.com' 2023-11-06T02:04:40.401 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-06T02:04:40.481 DEBUG:teuthology.orchestra.run.smithi049:> true 2023-11-06T02:04:40.587 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi049.front.sepia.ceph.com' 2023-11-06T02:04:40.588 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi150.front.sepia.ceph.com' 2023-11-06T02:04:40.589 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-06T02:04:40.668 DEBUG:teuthology.orchestra.run.smithi150:> true 2023-11-06T02:04:40.770 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi150.front.sepia.ceph.com' 2023-11-06T02:04:40.771 INFO:teuthology.run_tasks:Running task clock... 2023-11-06T02:04:40.783 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-11-06T02:04:40.784 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-06T02:04:40.785 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-06T02:04:40.787 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-06T02:04:40.787 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-06T02:04:40.815 INFO:teuthology.orchestra.run.smithi049.stdout: 6 Nov 02:04:40 ntpd[27879]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2023-11-06T02:04:40.816 INFO:teuthology.orchestra.run.smithi049.stdout: 6 Nov 02:04:40 ntpd[27879]: Command line: ntpd -gq 2023-11-06T02:04:40.816 INFO:teuthology.orchestra.run.smithi049.stdout: 6 Nov 02:04:40 ntpd[27879]: ---------------------------------------------------- 2023-11-06T02:04:40.816 INFO:teuthology.orchestra.run.smithi049.stdout: 6 Nov 02:04:40 ntpd[27879]: ntp-4 is maintained by Network Time Foundation, 2023-11-06T02:04:40.816 INFO:teuthology.orchestra.run.smithi049.stdout: 6 Nov 02:04:40 ntpd[27879]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2023-11-06T02:04:40.816 INFO:teuthology.orchestra.run.smithi049.stdout: 6 Nov 02:04:40 ntpd[27879]: corporation. Support and training for ntp-4 are 2023-11-06T02:04:40.817 INFO:teuthology.orchestra.run.smithi049.stdout: 6 Nov 02:04:40 ntpd[27879]: available at https://www.nwtime.org/support 2023-11-06T02:04:40.817 INFO:teuthology.orchestra.run.smithi049.stdout: 6 Nov 02:04:40 ntpd[27879]: ---------------------------------------------------- 2023-11-06T02:04:40.817 INFO:teuthology.orchestra.run.smithi049.stdout: 6 Nov 02:04:40 ntpd[27879]: proto: precision = 0.046 usec (-24) 2023-11-06T02:04:40.817 INFO:teuthology.orchestra.run.smithi049.stdout: 6 Nov 02:04:40 ntpd[27879]: basedate set to 2022-02-04 2023-11-06T02:04:40.817 INFO:teuthology.orchestra.run.smithi049.stdout: 6 Nov 02:04:40 ntpd[27879]: gps base set to 2022-02-06 (week 2196) 2023-11-06T02:04:40.818 INFO:teuthology.orchestra.run.smithi049.stdout: 6 Nov 02:04:40 ntpd[27879]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-11-06T02:04:40.818 INFO:teuthology.orchestra.run.smithi049.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-11-06T02:04:40.818 INFO:teuthology.orchestra.run.smithi049.stderr:restrict ::: KOD does nothing without LIMITED. 2023-11-06T02:04:40.819 INFO:teuthology.orchestra.run.smithi049.stdout: 6 Nov 02:04:40 ntpd[27879]: restrict ::: KOD does nothing without LIMITED. 2023-11-06T02:04:40.820 INFO:teuthology.orchestra.run.smithi049.stdout: 6 Nov 02:04:40 ntpd[27879]: Listen and drop on 0 v6wildcard [::]:123 2023-11-06T02:04:40.820 INFO:teuthology.orchestra.run.smithi049.stdout: 6 Nov 02:04:40 ntpd[27879]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2023-11-06T02:04:40.820 INFO:teuthology.orchestra.run.smithi049.stdout: 6 Nov 02:04:40 ntpd[27879]: Listen normally on 2 lo 127.0.0.1:123 2023-11-06T02:04:40.820 INFO:teuthology.orchestra.run.smithi049.stdout: 6 Nov 02:04:40 ntpd[27879]: Listen normally on 3 ens1f0 172.21.15.49:123 2023-11-06T02:04:40.820 INFO:teuthology.orchestra.run.smithi049.stdout: 6 Nov 02:04:40 ntpd[27879]: Listen normally on 4 lo [::1]:123 2023-11-06T02:04:40.821 INFO:teuthology.orchestra.run.smithi049.stdout: 6 Nov 02:04:40 ntpd[27879]: Listen normally on 5 ens1f0 [fe80::ec4:7aff:febd:1566%4]:123 2023-11-06T02:04:40.821 INFO:teuthology.orchestra.run.smithi049.stdout: 6 Nov 02:04:40 ntpd[27879]: Listening on routing socket on fd #22 for interface updates 2023-11-06T02:04:40.839 INFO:teuthology.orchestra.run.smithi150.stdout: 6 Nov 02:04:40 ntpd[28008]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2023-11-06T02:04:40.839 INFO:teuthology.orchestra.run.smithi150.stdout: 6 Nov 02:04:40 ntpd[28008]: Command line: ntpd -gq 2023-11-06T02:04:40.840 INFO:teuthology.orchestra.run.smithi150.stdout: 6 Nov 02:04:40 ntpd[28008]: ---------------------------------------------------- 2023-11-06T02:04:40.840 INFO:teuthology.orchestra.run.smithi150.stdout: 6 Nov 02:04:40 ntpd[28008]: ntp-4 is maintained by Network Time Foundation, 2023-11-06T02:04:40.840 INFO:teuthology.orchestra.run.smithi150.stdout: 6 Nov 02:04:40 ntpd[28008]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2023-11-06T02:04:40.840 INFO:teuthology.orchestra.run.smithi150.stdout: 6 Nov 02:04:40 ntpd[28008]: corporation. Support and training for ntp-4 are 2023-11-06T02:04:40.840 INFO:teuthology.orchestra.run.smithi150.stdout: 6 Nov 02:04:40 ntpd[28008]: available at https://www.nwtime.org/support 2023-11-06T02:04:40.841 INFO:teuthology.orchestra.run.smithi150.stdout: 6 Nov 02:04:40 ntpd[28008]: ---------------------------------------------------- 2023-11-06T02:04:40.841 INFO:teuthology.orchestra.run.smithi150.stdout: 6 Nov 02:04:40 ntpd[28008]: proto: precision = 0.044 usec (-24) 2023-11-06T02:04:40.842 INFO:teuthology.orchestra.run.smithi150.stdout: 6 Nov 02:04:40 ntpd[28008]: basedate set to 2022-02-04 2023-11-06T02:04:40.842 INFO:teuthology.orchestra.run.smithi150.stdout: 6 Nov 02:04:40 ntpd[28008]: gps base set to 2022-02-06 (week 2196) 2023-11-06T02:04:40.842 INFO:teuthology.orchestra.run.smithi150.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-11-06T02:04:40.843 INFO:teuthology.orchestra.run.smithi150.stdout: 6 Nov 02:04:40 ntpd[28008]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-11-06T02:04:40.843 INFO:teuthology.orchestra.run.smithi150.stdout: 6 Nov 02:04:40 ntpd[28008]: restrict ::: KOD does nothing without LIMITED. 2023-11-06T02:04:40.843 INFO:teuthology.orchestra.run.smithi150.stderr:restrict ::: KOD does nothing without LIMITED. 2023-11-06T02:04:40.844 INFO:teuthology.orchestra.run.smithi150.stdout: 6 Nov 02:04:40 ntpd[28008]: Listen and drop on 0 v6wildcard [::]:123 2023-11-06T02:04:40.844 INFO:teuthology.orchestra.run.smithi150.stdout: 6 Nov 02:04:40 ntpd[28008]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2023-11-06T02:04:40.845 INFO:teuthology.orchestra.run.smithi150.stdout: 6 Nov 02:04:40 ntpd[28008]: Listen normally on 2 lo 127.0.0.1:123 2023-11-06T02:04:40.845 INFO:teuthology.orchestra.run.smithi150.stdout: 6 Nov 02:04:40 ntpd[28008]: Listen normally on 3 enp3s0f1 172.21.15.150:123 2023-11-06T02:04:40.845 INFO:teuthology.orchestra.run.smithi150.stdout: 6 Nov 02:04:40 ntpd[28008]: Listen normally on 4 lo [::1]:123 2023-11-06T02:04:40.845 INFO:teuthology.orchestra.run.smithi150.stdout: 6 Nov 02:04:40 ntpd[28008]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:6e35%5]:123 2023-11-06T02:04:40.845 INFO:teuthology.orchestra.run.smithi150.stdout: 6 Nov 02:04:40 ntpd[28008]: Listening on routing socket on fd #22 for interface updates 2023-11-06T02:04:41.815 INFO:teuthology.orchestra.run.smithi049.stderr: 6 Nov 02:04:41 ntpd[27879]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-06T02:04:41.815 INFO:teuthology.orchestra.run.smithi049.stderr: 6 Nov 02:04:41 ntpd[27879]: can't open /var/log/ntpstats/rawstats.20231106: Permission denied 2023-11-06T02:04:41.816 INFO:teuthology.orchestra.run.smithi049.stderr: 6 Nov 02:04:41 ntpd[27879]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-06T02:04:41.816 INFO:teuthology.orchestra.run.smithi049.stderr: 6 Nov 02:04:41 ntpd[27879]: can't open /var/log/ntpstats/rawstats.20231106: Permission denied 2023-11-06T02:04:41.816 INFO:teuthology.orchestra.run.smithi049.stderr: 6 Nov 02:04:41 ntpd[27879]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-06T02:04:41.816 INFO:teuthology.orchestra.run.smithi049.stderr: 6 Nov 02:04:41 ntpd[27879]: can't open /var/log/ntpstats/peerstats.20231106: Permission denied 2023-11-06T02:04:42.814 INFO:teuthology.orchestra.run.smithi049.stderr: 6 Nov 02:04:42 ntpd[27879]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-06T02:04:42.815 INFO:teuthology.orchestra.run.smithi049.stderr: 6 Nov 02:04:42 ntpd[27879]: can't open /var/log/ntpstats/rawstats.20231106: Permission denied 2023-11-06T02:04:43.815 INFO:teuthology.orchestra.run.smithi049.stderr: 6 Nov 02:04:43 ntpd[27879]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-06T02:04:43.816 INFO:teuthology.orchestra.run.smithi049.stderr: 6 Nov 02:04:43 ntpd[27879]: can't open /var/log/ntpstats/rawstats.20231106: Permission denied 2023-11-06T02:04:43.816 INFO:teuthology.orchestra.run.smithi049.stderr: 6 Nov 02:04:43 ntpd[27879]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-06T02:04:43.816 INFO:teuthology.orchestra.run.smithi049.stderr: 6 Nov 02:04:43 ntpd[27879]: can't open /var/log/ntpstats/rawstats.20231106: Permission denied 2023-11-06T02:04:43.816 INFO:teuthology.orchestra.run.smithi049.stderr: 6 Nov 02:04:43 ntpd[27879]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-06T02:04:43.817 INFO:teuthology.orchestra.run.smithi049.stderr: 6 Nov 02:04:43 ntpd[27879]: can't open /var/log/ntpstats/rawstats.20231106: Permission denied 2023-11-06T02:04:43.817 INFO:teuthology.orchestra.run.smithi049.stderr: 6 Nov 02:04:43 ntpd[27879]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-06T02:04:43.817 INFO:teuthology.orchestra.run.smithi049.stderr: 6 Nov 02:04:43 ntpd[27879]: can't open /var/log/ntpstats/peerstats.20231106: Permission denied 2023-11-06T02:04:43.817 INFO:teuthology.orchestra.run.smithi049.stderr: 6 Nov 02:04:43 ntpd[27879]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-06T02:04:43.817 INFO:teuthology.orchestra.run.smithi049.stderr: 6 Nov 02:04:43 ntpd[27879]: can't open /var/log/ntpstats/rawstats.20231106: Permission denied 2023-11-06T02:04:43.818 INFO:teuthology.orchestra.run.smithi049.stderr: 6 Nov 02:04:43 ntpd[27879]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-06T02:04:43.818 INFO:teuthology.orchestra.run.smithi049.stderr: 6 Nov 02:04:43 ntpd[27879]: can't open /var/log/ntpstats/peerstats.20231106: Permission denied 2023-11-06T02:04:43.840 INFO:teuthology.orchestra.run.smithi150.stderr: 6 Nov 02:04:43 ntpd[28008]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-06T02:04:43.841 INFO:teuthology.orchestra.run.smithi150.stderr: 6 Nov 02:04:43 ntpd[28008]: can't open /var/log/ntpstats/rawstats.20231106: Permission denied 2023-11-06T02:04:43.841 INFO:teuthology.orchestra.run.smithi150.stderr: 6 Nov 02:04:43 ntpd[28008]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-06T02:04:43.841 INFO:teuthology.orchestra.run.smithi150.stderr: 6 Nov 02:04:43 ntpd[28008]: can't open /var/log/ntpstats/rawstats.20231106: Permission denied 2023-11-06T02:04:43.841 INFO:teuthology.orchestra.run.smithi150.stderr: 6 Nov 02:04:43 ntpd[28008]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-06T02:04:43.842 INFO:teuthology.orchestra.run.smithi150.stderr: 6 Nov 02:04:43 ntpd[28008]: can't open /var/log/ntpstats/rawstats.20231106: Permission denied 2023-11-06T02:04:43.842 INFO:teuthology.orchestra.run.smithi150.stderr: 6 Nov 02:04:43 ntpd[28008]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-06T02:04:43.842 INFO:teuthology.orchestra.run.smithi150.stderr: 6 Nov 02:04:43 ntpd[28008]: can't open /var/log/ntpstats/peerstats.20231106: Permission denied 2023-11-06T02:04:43.842 INFO:teuthology.orchestra.run.smithi150.stderr: 6 Nov 02:04:43 ntpd[28008]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-06T02:04:43.842 INFO:teuthology.orchestra.run.smithi150.stderr: 6 Nov 02:04:43 ntpd[28008]: can't open /var/log/ntpstats/rawstats.20231106: Permission denied 2023-11-06T02:04:43.843 INFO:teuthology.orchestra.run.smithi150.stderr: 6 Nov 02:04:43 ntpd[28008]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-06T02:04:43.843 INFO:teuthology.orchestra.run.smithi150.stderr: 6 Nov 02:04:43 ntpd[28008]: can't open /var/log/ntpstats/peerstats.20231106: Permission denied 2023-11-06T02:04:44.841 INFO:teuthology.orchestra.run.smithi150.stderr: 6 Nov 02:04:44 ntpd[28008]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-06T02:04:44.841 INFO:teuthology.orchestra.run.smithi150.stderr: 6 Nov 02:04:44 ntpd[28008]: can't open /var/log/ntpstats/rawstats.20231106: Permission denied 2023-11-06T02:04:44.842 INFO:teuthology.orchestra.run.smithi150.stderr: 6 Nov 02:04:44 ntpd[28008]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-06T02:04:44.842 INFO:teuthology.orchestra.run.smithi150.stderr: 6 Nov 02:04:44 ntpd[28008]: can't open /var/log/ntpstats/rawstats.20231106: Permission denied 2023-11-06T02:04:44.842 INFO:teuthology.orchestra.run.smithi150.stderr: 6 Nov 02:04:44 ntpd[28008]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-06T02:04:44.843 INFO:teuthology.orchestra.run.smithi150.stderr: 6 Nov 02:04:44 ntpd[28008]: can't open /var/log/ntpstats/peerstats.20231106: Permission denied 2023-11-06T02:04:45.817 INFO:teuthology.orchestra.run.smithi049.stderr: 6 Nov 02:04:45 ntpd[27879]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-06T02:04:45.817 INFO:teuthology.orchestra.run.smithi049.stderr: 6 Nov 02:04:45 ntpd[27879]: can't open /var/log/ntpstats/rawstats.20231106: Permission denied 2023-11-06T02:04:45.817 INFO:teuthology.orchestra.run.smithi049.stderr: 6 Nov 02:04:45 ntpd[27879]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-06T02:04:45.818 INFO:teuthology.orchestra.run.smithi049.stderr: 6 Nov 02:04:45 ntpd[27879]: can't open /var/log/ntpstats/rawstats.20231106: Permission denied 2023-11-06T02:04:45.818 INFO:teuthology.orchestra.run.smithi049.stderr: 6 Nov 02:04:45 ntpd[27879]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-06T02:04:45.818 INFO:teuthology.orchestra.run.smithi049.stderr: 6 Nov 02:04:45 ntpd[27879]: can't open /var/log/ntpstats/rawstats.20231106: Permission denied 2023-11-06T02:04:45.819 INFO:teuthology.orchestra.run.smithi049.stderr: 6 Nov 02:04:45 ntpd[27879]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-06T02:04:45.819 INFO:teuthology.orchestra.run.smithi049.stderr: 6 Nov 02:04:45 ntpd[27879]: can't open /var/log/ntpstats/rawstats.20231106: Permission denied 2023-11-06T02:04:45.819 INFO:teuthology.orchestra.run.smithi049.stderr: 6 Nov 02:04:45 ntpd[27879]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-06T02:04:45.819 INFO:teuthology.orchestra.run.smithi049.stderr: 6 Nov 02:04:45 ntpd[27879]: can't open /var/log/ntpstats/peerstats.20231106: Permission denied 2023-11-06T02:04:45.820 INFO:teuthology.orchestra.run.smithi049.stderr: 6 Nov 02:04:45 ntpd[27879]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-06T02:04:45.820 INFO:teuthology.orchestra.run.smithi049.stderr: 6 Nov 02:04:45 ntpd[27879]: can't open /var/log/ntpstats/rawstats.20231106: Permission denied 2023-11-06T02:04:45.820 INFO:teuthology.orchestra.run.smithi049.stderr: 6 Nov 02:04:45 ntpd[27879]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-06T02:04:45.820 INFO:teuthology.orchestra.run.smithi049.stderr: 6 Nov 02:04:45 ntpd[27879]: can't open /var/log/ntpstats/peerstats.20231106: Permission denied 2023-11-06T02:04:45.820 INFO:teuthology.orchestra.run.smithi049.stderr: 6 Nov 02:04:45 ntpd[27879]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-06T02:04:45.821 INFO:teuthology.orchestra.run.smithi049.stderr: 6 Nov 02:04:45 ntpd[27879]: can't open /var/log/ntpstats/rawstats.20231106: Permission denied 2023-11-06T02:04:45.821 INFO:teuthology.orchestra.run.smithi049.stderr: 6 Nov 02:04:45 ntpd[27879]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-06T02:04:45.821 INFO:teuthology.orchestra.run.smithi049.stderr: 6 Nov 02:04:45 ntpd[27879]: can't open /var/log/ntpstats/peerstats.20231106: Permission denied 2023-11-06T02:04:45.841 INFO:teuthology.orchestra.run.smithi150.stderr: 6 Nov 02:04:45 ntpd[28008]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-06T02:04:45.841 INFO:teuthology.orchestra.run.smithi150.stderr: 6 Nov 02:04:45 ntpd[28008]: can't open /var/log/ntpstats/rawstats.20231106: Permission denied 2023-11-06T02:04:45.841 INFO:teuthology.orchestra.run.smithi150.stderr: 6 Nov 02:04:45 ntpd[28008]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-06T02:04:45.842 INFO:teuthology.orchestra.run.smithi150.stderr: 6 Nov 02:04:45 ntpd[28008]: can't open /var/log/ntpstats/rawstats.20231106: Permission denied 2023-11-06T02:04:45.842 INFO:teuthology.orchestra.run.smithi150.stderr: 6 Nov 02:04:45 ntpd[28008]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-06T02:04:45.842 INFO:teuthology.orchestra.run.smithi150.stderr: 6 Nov 02:04:45 ntpd[28008]: can't open /var/log/ntpstats/rawstats.20231106: Permission denied 2023-11-06T02:04:45.842 INFO:teuthology.orchestra.run.smithi150.stderr: 6 Nov 02:04:45 ntpd[28008]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-06T02:04:45.842 INFO:teuthology.orchestra.run.smithi150.stderr: 6 Nov 02:04:45 ntpd[28008]: can't open /var/log/ntpstats/rawstats.20231106: Permission denied 2023-11-06T02:04:45.843 INFO:teuthology.orchestra.run.smithi150.stderr: 6 Nov 02:04:45 ntpd[28008]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-06T02:04:45.843 INFO:teuthology.orchestra.run.smithi150.stderr: 6 Nov 02:04:45 ntpd[28008]: can't open /var/log/ntpstats/peerstats.20231106: Permission denied 2023-11-06T02:04:45.843 INFO:teuthology.orchestra.run.smithi150.stderr: 6 Nov 02:04:45 ntpd[28008]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-06T02:04:45.843 INFO:teuthology.orchestra.run.smithi150.stderr: 6 Nov 02:04:45 ntpd[28008]: can't open /var/log/ntpstats/rawstats.20231106: Permission denied 2023-11-06T02:04:45.843 INFO:teuthology.orchestra.run.smithi150.stderr: 6 Nov 02:04:45 ntpd[28008]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-06T02:04:45.844 INFO:teuthology.orchestra.run.smithi150.stderr: 6 Nov 02:04:45 ntpd[28008]: can't open /var/log/ntpstats/peerstats.20231106: Permission denied 2023-11-06T02:04:46.840 INFO:teuthology.orchestra.run.smithi150.stderr: 6 Nov 02:04:46 ntpd[28008]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-06T02:04:46.840 INFO:teuthology.orchestra.run.smithi150.stderr: 6 Nov 02:04:46 ntpd[28008]: can't open /var/log/ntpstats/rawstats.20231106: Permission denied 2023-11-06T02:04:46.840 INFO:teuthology.orchestra.run.smithi150.stderr: 6 Nov 02:04:46 ntpd[28008]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-06T02:04:46.841 INFO:teuthology.orchestra.run.smithi150.stderr: 6 Nov 02:04:46 ntpd[28008]: can't open /var/log/ntpstats/rawstats.20231106: Permission denied 2023-11-06T02:04:46.841 INFO:teuthology.orchestra.run.smithi150.stderr: 6 Nov 02:04:46 ntpd[28008]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-06T02:04:46.841 INFO:teuthology.orchestra.run.smithi150.stderr: 6 Nov 02:04:46 ntpd[28008]: can't open /var/log/ntpstats/peerstats.20231106: Permission denied 2023-11-06T02:04:47.815 INFO:teuthology.orchestra.run.smithi049.stderr: 6 Nov 02:04:47 ntpd[27879]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-06T02:04:47.816 INFO:teuthology.orchestra.run.smithi049.stderr: 6 Nov 02:04:47 ntpd[27879]: can't open /var/log/ntpstats/rawstats.20231106: Permission denied 2023-11-06T02:04:47.816 INFO:teuthology.orchestra.run.smithi049.stderr: 6 Nov 02:04:47 ntpd[27879]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-06T02:04:47.816 INFO:teuthology.orchestra.run.smithi049.stderr: 6 Nov 02:04:47 ntpd[27879]: can't open /var/log/ntpstats/rawstats.20231106: Permission denied 2023-11-06T02:04:47.816 INFO:teuthology.orchestra.run.smithi049.stderr: 6 Nov 02:04:47 ntpd[27879]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-06T02:04:47.816 INFO:teuthology.orchestra.run.smithi049.stderr: 6 Nov 02:04:47 ntpd[27879]: can't open /var/log/ntpstats/rawstats.20231106: Permission denied 2023-11-06T02:04:47.817 INFO:teuthology.orchestra.run.smithi049.stderr: 6 Nov 02:04:47 ntpd[27879]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-06T02:04:47.817 INFO:teuthology.orchestra.run.smithi049.stderr: 6 Nov 02:04:47 ntpd[27879]: can't open /var/log/ntpstats/rawstats.20231106: Permission denied 2023-11-06T02:04:47.817 INFO:teuthology.orchestra.run.smithi049.stderr: 6 Nov 02:04:47 ntpd[27879]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-06T02:04:47.817 INFO:teuthology.orchestra.run.smithi049.stderr: 6 Nov 02:04:47 ntpd[27879]: can't open /var/log/ntpstats/peerstats.20231106: Permission denied 2023-11-06T02:04:47.818 INFO:teuthology.orchestra.run.smithi049.stdout: 6 Nov 02:04:47 ntpd[27879]: ntpd: time slew -0.000391 s 2023-11-06T02:04:47.818 INFO:teuthology.orchestra.run.smithi049.stdout:ntpd: time slew -0.000391s 2023-11-06T02:04:47.818 INFO:teuthology.orchestra.run.smithi049.stderr: 6 Nov 02:04:47 ntpd[27879]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2023-11-06T02:04:47.819 INFO:teuthology.orchestra.run.smithi049.stderr: 6 Nov 02:04:47 ntpd[27879]: can't open /var/log/ntpstats/loopstats.20231106: Permission denied 2023-11-06T02:04:47.840 INFO:teuthology.orchestra.run.smithi150.stderr: 6 Nov 02:04:47 ntpd[28008]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-06T02:04:47.841 INFO:teuthology.orchestra.run.smithi150.stderr: 6 Nov 02:04:47 ntpd[28008]: can't open /var/log/ntpstats/rawstats.20231106: Permission denied 2023-11-06T02:04:47.841 INFO:teuthology.orchestra.run.smithi150.stderr: 6 Nov 02:04:47 ntpd[28008]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-06T02:04:47.841 INFO:teuthology.orchestra.run.smithi150.stderr: 6 Nov 02:04:47 ntpd[28008]: can't open /var/log/ntpstats/rawstats.20231106: Permission denied 2023-11-06T02:04:47.841 INFO:teuthology.orchestra.run.smithi150.stderr: 6 Nov 02:04:47 ntpd[28008]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-06T02:04:47.841 INFO:teuthology.orchestra.run.smithi150.stderr: 6 Nov 02:04:47 ntpd[28008]: can't open /var/log/ntpstats/rawstats.20231106: Permission denied 2023-11-06T02:04:47.842 INFO:teuthology.orchestra.run.smithi150.stderr: 6 Nov 02:04:47 ntpd[28008]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-06T02:04:47.842 INFO:teuthology.orchestra.run.smithi150.stderr: 6 Nov 02:04:47 ntpd[28008]: can't open /var/log/ntpstats/peerstats.20231106: Permission denied 2023-11-06T02:04:47.842 INFO:teuthology.orchestra.run.smithi150.stderr: 6 Nov 02:04:47 ntpd[28008]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-06T02:04:47.842 INFO:teuthology.orchestra.run.smithi150.stderr: 6 Nov 02:04:47 ntpd[28008]: can't open /var/log/ntpstats/rawstats.20231106: Permission denied 2023-11-06T02:04:47.842 INFO:teuthology.orchestra.run.smithi150.stderr: 6 Nov 02:04:47 ntpd[28008]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-06T02:04:47.843 INFO:teuthology.orchestra.run.smithi150.stderr: 6 Nov 02:04:47 ntpd[28008]: can't open /var/log/ntpstats/peerstats.20231106: Permission denied 2023-11-06T02:04:47.911 INFO:teuthology.orchestra.run.smithi049.stdout: remote refid st t when poll reach delay offset jitter 2023-11-06T02:04:47.911 INFO:teuthology.orchestra.run.smithi049.stdout:============================================================================== 2023-11-06T02:04:47.911 INFO:teuthology.orchestra.run.smithi049.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-11-06T02:04:47.911 INFO:teuthology.orchestra.run.smithi049.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-11-06T02:04:47.912 INFO:teuthology.orchestra.run.smithi049.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-11-06T02:04:47.913 INFO:teuthology.orchestra.run.smithi049.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-11-06T02:04:48.840 INFO:teuthology.orchestra.run.smithi150.stderr: 6 Nov 02:04:48 ntpd[28008]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-06T02:04:48.840 INFO:teuthology.orchestra.run.smithi150.stderr: 6 Nov 02:04:48 ntpd[28008]: can't open /var/log/ntpstats/rawstats.20231106: Permission denied 2023-11-06T02:04:48.840 INFO:teuthology.orchestra.run.smithi150.stderr: 6 Nov 02:04:48 ntpd[28008]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-06T02:04:48.840 INFO:teuthology.orchestra.run.smithi150.stderr: 6 Nov 02:04:48 ntpd[28008]: can't open /var/log/ntpstats/rawstats.20231106: Permission denied 2023-11-06T02:04:48.841 INFO:teuthology.orchestra.run.smithi150.stderr: 6 Nov 02:04:48 ntpd[28008]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-06T02:04:48.841 INFO:teuthology.orchestra.run.smithi150.stderr: 6 Nov 02:04:48 ntpd[28008]: can't open /var/log/ntpstats/peerstats.20231106: Permission denied 2023-11-06T02:04:49.840 INFO:teuthology.orchestra.run.smithi150.stderr: 6 Nov 02:04:49 ntpd[28008]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-06T02:04:49.840 INFO:teuthology.orchestra.run.smithi150.stderr: 6 Nov 02:04:49 ntpd[28008]: can't open /var/log/ntpstats/rawstats.20231106: Permission denied 2023-11-06T02:04:49.841 INFO:teuthology.orchestra.run.smithi150.stderr: 6 Nov 02:04:49 ntpd[28008]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-06T02:04:49.841 INFO:teuthology.orchestra.run.smithi150.stderr: 6 Nov 02:04:49 ntpd[28008]: can't open /var/log/ntpstats/rawstats.20231106: Permission denied 2023-11-06T02:04:49.841 INFO:teuthology.orchestra.run.smithi150.stderr: 6 Nov 02:04:49 ntpd[28008]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-06T02:04:49.841 INFO:teuthology.orchestra.run.smithi150.stderr: 6 Nov 02:04:49 ntpd[28008]: can't open /var/log/ntpstats/rawstats.20231106: Permission denied 2023-11-06T02:04:49.841 INFO:teuthology.orchestra.run.smithi150.stderr: 6 Nov 02:04:49 ntpd[28008]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-06T02:04:49.842 INFO:teuthology.orchestra.run.smithi150.stderr: 6 Nov 02:04:49 ntpd[28008]: can't open /var/log/ntpstats/peerstats.20231106: Permission denied 2023-11-06T02:04:49.842 INFO:teuthology.orchestra.run.smithi150.stdout: 6 Nov 02:04:49 ntpd[28008]: ntpd: time slew -0.000489 s 2023-11-06T02:04:49.842 INFO:teuthology.orchestra.run.smithi150.stdout:ntpd: time slew -0.000489s 2023-11-06T02:04:49.843 INFO:teuthology.orchestra.run.smithi150.stderr: 6 Nov 02:04:49 ntpd[28008]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2023-11-06T02:04:49.843 INFO:teuthology.orchestra.run.smithi150.stderr: 6 Nov 02:04:49 ntpd[28008]: can't open /var/log/ntpstats/loopstats.20231106: Permission denied 2023-11-06T02:04:50.214 INFO:teuthology.orchestra.run.smithi150.stdout: remote refid st t when poll reach delay offset jitter 2023-11-06T02:04:50.214 INFO:teuthology.orchestra.run.smithi150.stdout:============================================================================== 2023-11-06T02:04:50.214 INFO:teuthology.orchestra.run.smithi150.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-11-06T02:04:50.215 INFO:teuthology.orchestra.run.smithi150.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-11-06T02:04:50.215 INFO:teuthology.orchestra.run.smithi150.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-11-06T02:04:50.215 INFO:teuthology.orchestra.run.smithi150.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-11-06T02:04:50.216 INFO:teuthology.run_tasks:Running task install... 2023-11-06T02:04:50.225 DEBUG:teuthology.task.install:project ceph 2023-11-06T02:04:50.226 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'f7e9f9af51acf98e0ccc3037487018e4ae40caa5'}} 2023-11-06T02:04:50.226 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'f7e9f9af51acf98e0ccc3037487018e4ae40caa5'} 2023-11-06T02:04:50.226 INFO:teuthology.task.install:Using flavor: default 2023-11-06T02:04:50.233 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-11-06T02:04:50.233 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-11-06T02:04:50.234 DEBUG:teuthology.orchestra.run.smithi049:> sudo apt-key list | grep Ceph 2023-11-06T02:04:50.235 DEBUG:teuthology.orchestra.run.smithi150:> sudo apt-key list | grep Ceph 2023-11-06T02:04:50.302 INFO:teuthology.orchestra.run.smithi049.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2023-11-06T02:04:50.324 INFO:teuthology.orchestra.run.smithi150.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2023-11-06T02:04:50.334 INFO:teuthology.orchestra.run.smithi049.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2023-11-06T02:04:50.334 INFO:teuthology.orchestra.run.smithi049.stdout:uid [ unknown] Ceph.com (release key) 2023-11-06T02:04:50.335 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2023-11-06T02:04:50.335 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2023-11-06T02:04:50.336 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=f7e9f9af51acf98e0ccc3037487018e4ae40caa5 2023-11-06T02:04:50.354 INFO:teuthology.orchestra.run.smithi150.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2023-11-06T02:04:50.354 INFO:teuthology.orchestra.run.smithi150.stdout:uid [ unknown] Ceph.com (release key) 2023-11-06T02:04:50.354 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2023-11-06T02:04:50.354 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2023-11-06T02:04:50.355 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=f7e9f9af51acf98e0ccc3037487018e4ae40caa5 2023-11-06T02:04:50.505 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default/ 2023-11-06T02:04:50.505 INFO:teuthology.task.install.deb:Package version is 18.2.0-1181-gf7e9f9af-1jammy 2023-11-06T02:04:50.554 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default/ 2023-11-06T02:04:50.554 INFO:teuthology.task.install.deb:Package version is 18.2.0-1181-gf7e9f9af-1jammy 2023-11-06T02:04:50.666 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-11-06T02:04:50.666 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2023-11-06T02:04:50.677 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-11-06T02:04:50.677 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2023-11-06T02:04:50.681 DEBUG:teuthology.orchestra.run.smithi049:> sudo apt-get update 2023-11-06T02:04:50.691 DEBUG:teuthology.orchestra.run.smithi150:> sudo apt-get update 2023-11-06T02:04:50.849 INFO:teuthology.orchestra.run.smithi049.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2023-11-06T02:04:50.856 INFO:teuthology.orchestra.run.smithi150.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2023-11-06T02:04:50.861 INFO:teuthology.orchestra.run.smithi049.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2023-11-06T02:04:50.866 INFO:teuthology.orchestra.run.smithi150.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2023-11-06T02:04:50.881 INFO:teuthology.orchestra.run.smithi049.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2023-11-06T02:04:50.883 INFO:teuthology.orchestra.run.smithi150.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2023-11-06T02:04:50.906 INFO:teuthology.orchestra.run.smithi049.stdout:Ign:4 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy InRelease 2023-11-06T02:04:50.908 INFO:teuthology.orchestra.run.smithi150.stdout:Ign:4 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy InRelease 2023-11-06T02:04:50.930 INFO:teuthology.orchestra.run.smithi150.stdout:Get:5 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2023-11-06T02:04:50.932 INFO:teuthology.orchestra.run.smithi049.stdout:Get:5 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2023-11-06T02:04:50.951 INFO:teuthology.orchestra.run.smithi150.stdout:Ign:6 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy Release.gpg 2023-11-06T02:04:50.954 INFO:teuthology.orchestra.run.smithi049.stdout:Ign:6 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy Release.gpg 2023-11-06T02:04:50.973 INFO:teuthology.orchestra.run.smithi150.stdout:Get:7 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.4 kB] 2023-11-06T02:04:50.976 INFO:teuthology.orchestra.run.smithi049.stdout:Get:7 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.4 kB] 2023-11-06T02:04:51.000 INFO:teuthology.orchestra.run.smithi150.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,418 B] 2023-11-06T02:04:51.003 INFO:teuthology.orchestra.run.smithi049.stdout:Hit:8 http://security.ubuntu.com/ubuntu jammy-security InRelease 2023-11-06T02:04:51.004 INFO:teuthology.orchestra.run.smithi049.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,418 B] 2023-11-06T02:04:51.010 INFO:teuthology.orchestra.run.smithi150.stdout:Hit:9 http://security.ubuntu.com/ubuntu jammy-security InRelease 2023-11-06T02:04:51.210 INFO:teuthology.orchestra.run.smithi150.stdout:Fetched 28.5 kB in 0s (83.5 kB/s) 2023-11-06T02:04:51.214 INFO:teuthology.orchestra.run.smithi049.stdout:Fetched 28.5 kB in 0s (80.7 kB/s) 2023-11-06T02:04:52.045 INFO:teuthology.orchestra.run.smithi150.stdout:Reading package lists... 2023-11-06T02:04:52.061 DEBUG:teuthology.orchestra.run.smithi150:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.2.0-1181-gf7e9f9af-1jammy cephadm=18.2.0-1181-gf7e9f9af-1jammy ceph-mds=18.2.0-1181-gf7e9f9af-1jammy ceph-mgr=18.2.0-1181-gf7e9f9af-1jammy ceph-common=18.2.0-1181-gf7e9f9af-1jammy ceph-fuse=18.2.0-1181-gf7e9f9af-1jammy ceph-test=18.2.0-1181-gf7e9f9af-1jammy ceph-volume=18.2.0-1181-gf7e9f9af-1jammy radosgw=18.2.0-1181-gf7e9f9af-1jammy python3-rados=18.2.0-1181-gf7e9f9af-1jammy python3-rgw=18.2.0-1181-gf7e9f9af-1jammy python3-cephfs=18.2.0-1181-gf7e9f9af-1jammy python3-rbd=18.2.0-1181-gf7e9f9af-1jammy libcephfs2=18.2.0-1181-gf7e9f9af-1jammy libcephfs-dev=18.2.0-1181-gf7e9f9af-1jammy librados2=18.2.0-1181-gf7e9f9af-1jammy librbd1=18.2.0-1181-gf7e9f9af-1jammy rbd-fuse=18.2.0-1181-gf7e9f9af-1jammy python3-cephfs=18.2.0-1181-gf7e9f9af-1jammy cephfs-shell=18.2.0-1181-gf7e9f9af-1jammy cephfs-top=18.2.0-1181-gf7e9f9af-1jammy cephfs-mirror=18.2.0-1181-gf7e9f9af-1jammy 2023-11-06T02:04:52.113 INFO:teuthology.orchestra.run.smithi150.stdout:Reading package lists... 2023-11-06T02:04:52.119 INFO:teuthology.orchestra.run.smithi049.stdout:Reading package lists... 2023-11-06T02:04:52.134 DEBUG:teuthology.orchestra.run.smithi049:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.2.0-1181-gf7e9f9af-1jammy cephadm=18.2.0-1181-gf7e9f9af-1jammy ceph-mds=18.2.0-1181-gf7e9f9af-1jammy ceph-mgr=18.2.0-1181-gf7e9f9af-1jammy ceph-common=18.2.0-1181-gf7e9f9af-1jammy ceph-fuse=18.2.0-1181-gf7e9f9af-1jammy ceph-test=18.2.0-1181-gf7e9f9af-1jammy ceph-volume=18.2.0-1181-gf7e9f9af-1jammy radosgw=18.2.0-1181-gf7e9f9af-1jammy python3-rados=18.2.0-1181-gf7e9f9af-1jammy python3-rgw=18.2.0-1181-gf7e9f9af-1jammy python3-cephfs=18.2.0-1181-gf7e9f9af-1jammy python3-rbd=18.2.0-1181-gf7e9f9af-1jammy libcephfs2=18.2.0-1181-gf7e9f9af-1jammy libcephfs-dev=18.2.0-1181-gf7e9f9af-1jammy librados2=18.2.0-1181-gf7e9f9af-1jammy librbd1=18.2.0-1181-gf7e9f9af-1jammy rbd-fuse=18.2.0-1181-gf7e9f9af-1jammy python3-cephfs=18.2.0-1181-gf7e9f9af-1jammy cephfs-shell=18.2.0-1181-gf7e9f9af-1jammy cephfs-top=18.2.0-1181-gf7e9f9af-1jammy cephfs-mirror=18.2.0-1181-gf7e9f9af-1jammy 2023-11-06T02:04:52.199 INFO:teuthology.orchestra.run.smithi049.stdout:Reading package lists... 2023-11-06T02:04:52.279 INFO:teuthology.orchestra.run.smithi150.stdout:Building dependency tree... 2023-11-06T02:04:52.280 INFO:teuthology.orchestra.run.smithi150.stdout:Reading state information... 2023-11-06T02:04:52.389 INFO:teuthology.orchestra.run.smithi049.stdout:Building dependency tree... 2023-11-06T02:04:52.390 INFO:teuthology.orchestra.run.smithi049.stdout:Reading state information... 2023-11-06T02:04:52.429 INFO:teuthology.orchestra.run.smithi150.stdout:The following packages were automatically installed and are no longer required: 2023-11-06T02:04:52.429 INFO:teuthology.orchestra.run.smithi150.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-11-06T02:04:52.429 INFO:teuthology.orchestra.run.smithi150.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-11-06T02:04:52.430 INFO:teuthology.orchestra.run.smithi150.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-11-06T02:04:52.430 INFO:teuthology.orchestra.run.smithi150.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2023-11-06T02:04:52.431 INFO:teuthology.orchestra.run.smithi150.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2023-11-06T02:04:52.431 INFO:teuthology.orchestra.run.smithi150.stdout:Use 'sudo apt autoremove' to remove them. 2023-11-06T02:04:52.432 INFO:teuthology.orchestra.run.smithi150.stdout:The following additional packages will be installed: 2023-11-06T02:04:52.432 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2023-11-06T02:04:52.432 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2023-11-06T02:04:52.432 INFO:teuthology.orchestra.run.smithi150.stdout: libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 liboath0 libonig5 2023-11-06T02:04:52.433 INFO:teuthology.orchestra.run.smithi150.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2023-11-06T02:04:52.433 INFO:teuthology.orchestra.run.smithi150.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.16.0 nvme-cli 2023-11-06T02:04:52.433 INFO:teuthology.orchestra.run.smithi150.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2023-11-06T02:04:52.433 INFO:teuthology.orchestra.run.smithi150.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2023-11-06T02:04:52.433 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2023-11-06T02:04:52.434 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2023-11-06T02:04:52.434 INFO:teuthology.orchestra.run.smithi150.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2023-11-06T02:04:52.434 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2023-11-06T02:04:52.434 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2023-11-06T02:04:52.434 INFO:teuthology.orchestra.run.smithi150.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2023-11-06T02:04:52.435 INFO:teuthology.orchestra.run.smithi150.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2023-11-06T02:04:52.435 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2023-11-06T02:04:52.435 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2023-11-06T02:04:52.435 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2023-11-06T02:04:52.435 INFO:teuthology.orchestra.run.smithi150.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2023-11-06T02:04:52.435 INFO:teuthology.orchestra.run.smithi150.stdout: python3-simplejson python3-singledispatch python3-sklearn 2023-11-06T02:04:52.436 INFO:teuthology.orchestra.run.smithi150.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2023-11-06T02:04:52.436 INFO:teuthology.orchestra.run.smithi150.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2023-11-06T02:04:52.436 INFO:teuthology.orchestra.run.smithi150.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2023-11-06T02:04:52.436 INFO:teuthology.orchestra.run.smithi150.stdout: qttranslations5-l10n socat xmlstarlet 2023-11-06T02:04:52.437 INFO:teuthology.orchestra.run.smithi150.stdout:Suggested packages: 2023-11-06T02:04:52.437 INFO:teuthology.orchestra.run.smithi150.stdout: python3-influxdb python-jinja2-doc python3-beaker python-mako-doc 2023-11-06T02:04:52.437 INFO:teuthology.orchestra.run.smithi150.stdout: python-natsort-doc python-openssl-doc python3-openssl-dbg httpd-wsgi 2023-11-06T02:04:52.437 INFO:teuthology.orchestra.run.smithi150.stdout: libapache2-mod-python libapache2-mod-scgi libjs-mochikit python-pecan-doc 2023-11-06T02:04:52.438 INFO:teuthology.orchestra.run.smithi150.stdout: python-psutil-doc subversion python-pygments-doc python-pyinotify-doc 2023-11-06T02:04:52.438 INFO:teuthology.orchestra.run.smithi150.stdout: python3-socks python-requests-doc python3-dap python-sklearn-doc ipython3 2023-11-06T02:04:52.438 INFO:teuthology.orchestra.run.smithi150.stdout: python-waitress-doc python-webob-doc python-webtest-doc python-werkzeug-doc 2023-11-06T02:04:52.438 INFO:teuthology.orchestra.run.smithi150.stdout: python3-watchdog 2023-11-06T02:04:52.438 INFO:teuthology.orchestra.run.smithi150.stdout:Recommended packages: 2023-11-06T02:04:52.439 INFO:teuthology.orchestra.run.smithi150.stdout: btrfs-tools 2023-11-06T02:04:52.531 INFO:teuthology.orchestra.run.smithi150.stdout:The following NEW packages will be installed: 2023-11-06T02:04:52.531 INFO:teuthology.orchestra.run.smithi150.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2023-11-06T02:04:52.531 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2023-11-06T02:04:52.532 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2023-11-06T02:04:52.532 INFO:teuthology.orchestra.run.smithi150.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2023-11-06T02:04:52.532 INFO:teuthology.orchestra.run.smithi150.stdout: libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 liboath0 libonig5 2023-11-06T02:04:52.532 INFO:teuthology.orchestra.run.smithi150.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2023-11-06T02:04:52.532 INFO:teuthology.orchestra.run.smithi150.stdout: libradosstriper1 librbd1 librdkafka1 librgw2 libsqlite3-mod-ceph 2023-11-06T02:04:52.534 INFO:teuthology.orchestra.run.smithi150.stdout: libthrift-0.16.0 nvme-cli python-asyncssh-doc python-babel-localedata 2023-11-06T02:04:52.534 INFO:teuthology.orchestra.run.smithi150.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2023-11-06T02:04:52.534 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2023-11-06T02:04:52.534 INFO:teuthology.orchestra.run.smithi150.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2023-11-06T02:04:52.534 INFO:teuthology.orchestra.run.smithi150.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2023-11-06T02:04:52.535 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2023-11-06T02:04:52.535 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2023-11-06T02:04:52.535 INFO:teuthology.orchestra.run.smithi150.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2023-11-06T02:04:52.535 INFO:teuthology.orchestra.run.smithi150.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2023-11-06T02:04:52.535 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2023-11-06T02:04:52.536 INFO:teuthology.orchestra.run.smithi150.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2023-11-06T02:04:52.536 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2023-11-06T02:04:52.536 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2023-11-06T02:04:52.536 INFO:teuthology.orchestra.run.smithi150.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2023-11-06T02:04:52.536 INFO:teuthology.orchestra.run.smithi150.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2023-11-06T02:04:52.537 INFO:teuthology.orchestra.run.smithi150.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2023-11-06T02:04:52.537 INFO:teuthology.orchestra.run.smithi150.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2023-11-06T02:04:52.537 INFO:teuthology.orchestra.run.smithi150.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2023-11-06T02:04:52.537 INFO:teuthology.orchestra.run.smithi150.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2023-11-06T02:04:52.537 INFO:teuthology.orchestra.run.smithi150.stdout: socat xmlstarlet 2023-11-06T02:04:52.564 INFO:teuthology.orchestra.run.smithi049.stdout:The following packages were automatically installed and are no longer required: 2023-11-06T02:04:52.565 INFO:teuthology.orchestra.run.smithi049.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-11-06T02:04:52.565 INFO:teuthology.orchestra.run.smithi049.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-11-06T02:04:52.565 INFO:teuthology.orchestra.run.smithi049.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-11-06T02:04:52.566 INFO:teuthology.orchestra.run.smithi049.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2023-11-06T02:04:52.567 INFO:teuthology.orchestra.run.smithi049.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2023-11-06T02:04:52.567 INFO:teuthology.orchestra.run.smithi049.stdout:Use 'sudo apt autoremove' to remove them. 2023-11-06T02:04:52.567 INFO:teuthology.orchestra.run.smithi049.stdout:The following additional packages will be installed: 2023-11-06T02:04:52.567 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2023-11-06T02:04:52.568 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2023-11-06T02:04:52.568 INFO:teuthology.orchestra.run.smithi049.stdout: libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 liboath0 libonig5 2023-11-06T02:04:52.568 INFO:teuthology.orchestra.run.smithi049.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2023-11-06T02:04:52.569 INFO:teuthology.orchestra.run.smithi049.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.16.0 nvme-cli 2023-11-06T02:04:52.569 INFO:teuthology.orchestra.run.smithi049.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2023-11-06T02:04:52.569 INFO:teuthology.orchestra.run.smithi049.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2023-11-06T02:04:52.570 INFO:teuthology.orchestra.run.smithi049.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2023-11-06T02:04:52.570 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2023-11-06T02:04:52.570 INFO:teuthology.orchestra.run.smithi049.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2023-11-06T02:04:52.570 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2023-11-06T02:04:52.571 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2023-11-06T02:04:52.571 INFO:teuthology.orchestra.run.smithi049.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2023-11-06T02:04:52.571 INFO:teuthology.orchestra.run.smithi049.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2023-11-06T02:04:52.571 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2023-11-06T02:04:52.572 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2023-11-06T02:04:52.572 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2023-11-06T02:04:52.572 INFO:teuthology.orchestra.run.smithi049.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2023-11-06T02:04:52.572 INFO:teuthology.orchestra.run.smithi049.stdout: python3-simplejson python3-singledispatch python3-sklearn 2023-11-06T02:04:52.573 INFO:teuthology.orchestra.run.smithi049.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2023-11-06T02:04:52.573 INFO:teuthology.orchestra.run.smithi049.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2023-11-06T02:04:52.573 INFO:teuthology.orchestra.run.smithi049.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2023-11-06T02:04:52.573 INFO:teuthology.orchestra.run.smithi049.stdout: qttranslations5-l10n socat xmlstarlet 2023-11-06T02:04:52.574 INFO:teuthology.orchestra.run.smithi049.stdout:Suggested packages: 2023-11-06T02:04:52.574 INFO:teuthology.orchestra.run.smithi049.stdout: python3-influxdb python-jinja2-doc python3-beaker python-mako-doc 2023-11-06T02:04:52.574 INFO:teuthology.orchestra.run.smithi049.stdout: python-natsort-doc python-openssl-doc python3-openssl-dbg httpd-wsgi 2023-11-06T02:04:52.575 INFO:teuthology.orchestra.run.smithi049.stdout: libapache2-mod-python libapache2-mod-scgi libjs-mochikit python-pecan-doc 2023-11-06T02:04:52.575 INFO:teuthology.orchestra.run.smithi049.stdout: python-psutil-doc subversion python-pygments-doc python-pyinotify-doc 2023-11-06T02:04:52.575 INFO:teuthology.orchestra.run.smithi049.stdout: python3-socks python-requests-doc python3-dap python-sklearn-doc ipython3 2023-11-06T02:04:52.575 INFO:teuthology.orchestra.run.smithi049.stdout: python-waitress-doc python-webob-doc python-webtest-doc python-werkzeug-doc 2023-11-06T02:04:52.575 INFO:teuthology.orchestra.run.smithi049.stdout: python3-watchdog 2023-11-06T02:04:52.575 INFO:teuthology.orchestra.run.smithi049.stdout:Recommended packages: 2023-11-06T02:04:52.576 INFO:teuthology.orchestra.run.smithi049.stdout: btrfs-tools 2023-11-06T02:04:52.679 INFO:teuthology.orchestra.run.smithi049.stdout:The following NEW packages will be installed: 2023-11-06T02:04:52.680 INFO:teuthology.orchestra.run.smithi049.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2023-11-06T02:04:52.680 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2023-11-06T02:04:52.680 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2023-11-06T02:04:52.680 INFO:teuthology.orchestra.run.smithi049.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2023-11-06T02:04:52.681 INFO:teuthology.orchestra.run.smithi049.stdout: libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 liboath0 libonig5 2023-11-06T02:04:52.681 INFO:teuthology.orchestra.run.smithi049.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2023-11-06T02:04:52.681 INFO:teuthology.orchestra.run.smithi049.stdout: libradosstriper1 librbd1 librdkafka1 librgw2 libsqlite3-mod-ceph 2023-11-06T02:04:52.682 INFO:teuthology.orchestra.run.smithi049.stdout: libthrift-0.16.0 nvme-cli python-asyncssh-doc python-babel-localedata 2023-11-06T02:04:52.682 INFO:teuthology.orchestra.run.smithi049.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2023-11-06T02:04:52.682 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2023-11-06T02:04:52.683 INFO:teuthology.orchestra.run.smithi049.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2023-11-06T02:04:52.683 INFO:teuthology.orchestra.run.smithi049.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2023-11-06T02:04:52.683 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2023-11-06T02:04:52.683 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2023-11-06T02:04:52.683 INFO:teuthology.orchestra.run.smithi049.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2023-11-06T02:04:52.684 INFO:teuthology.orchestra.run.smithi049.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2023-11-06T02:04:52.684 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2023-11-06T02:04:52.684 INFO:teuthology.orchestra.run.smithi049.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2023-11-06T02:04:52.684 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2023-11-06T02:04:52.684 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2023-11-06T02:04:52.685 INFO:teuthology.orchestra.run.smithi049.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2023-11-06T02:04:52.685 INFO:teuthology.orchestra.run.smithi049.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2023-11-06T02:04:52.685 INFO:teuthology.orchestra.run.smithi049.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2023-11-06T02:04:52.685 INFO:teuthology.orchestra.run.smithi049.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2023-11-06T02:04:52.685 INFO:teuthology.orchestra.run.smithi049.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2023-11-06T02:04:52.686 INFO:teuthology.orchestra.run.smithi049.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2023-11-06T02:04:52.686 INFO:teuthology.orchestra.run.smithi049.stdout: socat xmlstarlet 2023-11-06T02:04:52.727 INFO:teuthology.orchestra.run.smithi150.stdout:0 upgraded, 115 newly installed, 0 to remove and 258 not upgraded. 2023-11-06T02:04:52.727 INFO:teuthology.orchestra.run.smithi150.stdout:Need to get 181 MB of archives. 2023-11-06T02:04:52.727 INFO:teuthology.orchestra.run.smithi150.stdout:After this operation, 812 MB of additional disk space will be used. 2023-11-06T02:04:52.727 INFO:teuthology.orchestra.run.smithi150.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2023-11-06T02:04:52.742 INFO:teuthology.orchestra.run.smithi049.stdout:0 upgraded, 115 newly installed, 0 to remove and 258 not upgraded. 2023-11-06T02:04:52.743 INFO:teuthology.orchestra.run.smithi049.stdout:Need to get 181 MB of archives. 2023-11-06T02:04:52.743 INFO:teuthology.orchestra.run.smithi049.stdout:After this operation, 812 MB of additional disk space will be used. 2023-11-06T02:04:52.743 INFO:teuthology.orchestra.run.smithi049.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2023-11-06T02:04:52.753 INFO:teuthology.orchestra.run.smithi150.stdout:Get:2 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 18.2.0-1181-gf7e9f9af-1jammy [3,417 kB] 2023-11-06T02:04:52.760 INFO:teuthology.orchestra.run.smithi049.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2023-11-06T02:04:52.770 INFO:teuthology.orchestra.run.smithi049.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2023-11-06T02:04:52.811 INFO:teuthology.orchestra.run.smithi049.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libfmt8 amd64 8.1.1+ds1-2 [60.2 kB] 2023-11-06T02:04:52.815 INFO:teuthology.orchestra.run.smithi150.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2023-11-06T02:04:52.820 INFO:teuthology.orchestra.run.smithi049.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2023-11-06T02:04:52.836 INFO:teuthology.orchestra.run.smithi049.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2023-11-06T02:04:52.839 INFO:teuthology.orchestra.run.smithi049.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2023-11-06T02:04:52.849 INFO:teuthology.orchestra.run.smithi049.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2023-11-06T02:04:52.864 INFO:teuthology.orchestra.run.smithi150.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2023-11-06T02:04:52.894 INFO:teuthology.orchestra.run.smithi049.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2023-11-06T02:04:52.898 INFO:teuthology.orchestra.run.smithi049.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2023-11-06T02:04:52.902 INFO:teuthology.orchestra.run.smithi049.stdout:Get:11 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 18.2.0-1181-gf7e9f9af-1jammy [3,417 kB] 2023-11-06T02:04:52.910 INFO:teuthology.orchestra.run.smithi049.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2023-11-06T02:04:52.914 INFO:teuthology.orchestra.run.smithi049.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2023-11-06T02:04:52.914 INFO:teuthology.orchestra.run.smithi049.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2023-11-06T02:04:52.923 INFO:teuthology.orchestra.run.smithi049.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2023-11-06T02:04:52.923 INFO:teuthology.orchestra.run.smithi049.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2023-11-06T02:04:52.924 INFO:teuthology.orchestra.run.smithi049.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2023-11-06T02:04:52.925 INFO:teuthology.orchestra.run.smithi049.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2023-11-06T02:04:52.925 INFO:teuthology.orchestra.run.smithi049.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2023-11-06T02:04:52.932 INFO:teuthology.orchestra.run.smithi049.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2023-11-06T02:04:52.932 INFO:teuthology.orchestra.run.smithi049.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2023-11-06T02:04:52.949 INFO:teuthology.orchestra.run.smithi049.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2023-11-06T02:04:52.949 INFO:teuthology.orchestra.run.smithi049.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2023-11-06T02:04:52.949 INFO:teuthology.orchestra.run.smithi049.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2023-11-06T02:04:52.952 INFO:teuthology.orchestra.run.smithi049.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2023-11-06T02:04:52.953 INFO:teuthology.orchestra.run.smithi049.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2023-11-06T02:04:52.953 INFO:teuthology.orchestra.run.smithi049.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2023-11-06T02:04:52.953 INFO:teuthology.orchestra.run.smithi049.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2023-11-06T02:04:52.954 INFO:teuthology.orchestra.run.smithi049.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2023-11-06T02:04:52.965 INFO:teuthology.orchestra.run.smithi049.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2023-11-06T02:04:52.966 INFO:teuthology.orchestra.run.smithi049.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2023-11-06T02:04:52.982 INFO:teuthology.orchestra.run.smithi049.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2023-11-06T02:04:52.983 INFO:teuthology.orchestra.run.smithi049.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2023-11-06T02:04:52.984 INFO:teuthology.orchestra.run.smithi049.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2023-11-06T02:04:52.989 INFO:teuthology.orchestra.run.smithi049.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2023-11-06T02:04:52.989 INFO:teuthology.orchestra.run.smithi049.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2023-11-06T02:04:52.990 INFO:teuthology.orchestra.run.smithi049.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2023-11-06T02:04:52.990 INFO:teuthology.orchestra.run.smithi049.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2023-11-06T02:04:52.991 INFO:teuthology.orchestra.run.smithi049.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2023-11-06T02:04:52.999 INFO:teuthology.orchestra.run.smithi049.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2023-11-06T02:04:53.000 INFO:teuthology.orchestra.run.smithi049.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-urllib3 all 1.26.5-1~exp1 [96.4 kB] 2023-11-06T02:04:53.011 INFO:teuthology.orchestra.run.smithi049.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2023-11-06T02:04:53.011 INFO:teuthology.orchestra.run.smithi049.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2023-11-06T02:04:53.014 INFO:teuthology.orchestra.run.smithi049.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2023-11-06T02:04:53.014 INFO:teuthology.orchestra.run.smithi049.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2023-11-06T02:04:53.017 INFO:teuthology.orchestra.run.smithi049.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2023-11-06T02:04:53.070 INFO:teuthology.orchestra.run.smithi049.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2023-11-06T02:04:53.070 INFO:teuthology.orchestra.run.smithi049.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jinja2 all 3.0.3-1 [108 kB] 2023-11-06T02:04:53.072 INFO:teuthology.orchestra.run.smithi049.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2023-11-06T02:04:53.072 INFO:teuthology.orchestra.run.smithi049.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2023-11-06T02:04:53.073 INFO:teuthology.orchestra.run.smithi049.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2023-11-06T02:04:53.074 INFO:teuthology.orchestra.run.smithi049.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2023-11-06T02:04:53.074 INFO:teuthology.orchestra.run.smithi049.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2023-11-06T02:04:53.074 INFO:teuthology.orchestra.run.smithi049.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2023-11-06T02:04:53.076 INFO:teuthology.orchestra.run.smithi049.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2023-11-06T02:04:53.077 INFO:teuthology.orchestra.run.smithi150.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libfmt8 amd64 8.1.1+ds1-2 [60.2 kB] 2023-11-06T02:04:53.088 INFO:teuthology.orchestra.run.smithi049.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2023-11-06T02:04:53.094 INFO:teuthology.orchestra.run.smithi049.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2023-11-06T02:04:53.112 INFO:teuthology.orchestra.run.smithi049.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2023-11-06T02:04:53.113 INFO:teuthology.orchestra.run.smithi049.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2023-11-06T02:04:53.114 INFO:teuthology.orchestra.run.smithi049.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2023-11-06T02:04:53.124 INFO:teuthology.orchestra.run.smithi150.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2023-11-06T02:04:53.130 INFO:teuthology.orchestra.run.smithi049.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2023-11-06T02:04:53.130 INFO:teuthology.orchestra.run.smithi049.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2023-11-06T02:04:53.130 INFO:teuthology.orchestra.run.smithi049.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2023-11-06T02:04:53.131 INFO:teuthology.orchestra.run.smithi049.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2023-11-06T02:04:53.131 INFO:teuthology.orchestra.run.smithi049.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2023-11-06T02:04:53.132 INFO:teuthology.orchestra.run.smithi049.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2023-11-06T02:04:53.138 INFO:teuthology.orchestra.run.smithi049.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2023-11-06T02:04:53.139 INFO:teuthology.orchestra.run.smithi049.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2023-11-06T02:04:53.142 INFO:teuthology.orchestra.run.smithi049.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2023-11-06T02:04:53.158 INFO:teuthology.orchestra.run.smithi049.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2023-11-06T02:04:53.158 INFO:teuthology.orchestra.run.smithi049.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2023-11-06T02:04:53.158 INFO:teuthology.orchestra.run.smithi049.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2023-11-06T02:04:53.161 INFO:teuthology.orchestra.run.smithi049.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2023-11-06T02:04:53.163 INFO:teuthology.orchestra.run.smithi049.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2023-11-06T02:04:53.163 INFO:teuthology.orchestra.run.smithi049.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2023-11-06T02:04:53.168 INFO:teuthology.orchestra.run.smithi049.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2023-11-06T02:04:53.172 INFO:teuthology.orchestra.run.smithi049.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2023-11-06T02:04:53.173 INFO:teuthology.orchestra.run.smithi049.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2023-11-06T02:04:53.174 INFO:teuthology.orchestra.run.smithi049.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2023-11-06T02:04:53.189 INFO:teuthology.orchestra.run.smithi049.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2023-11-06T02:04:53.189 INFO:teuthology.orchestra.run.smithi049.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2023-11-06T02:04:53.190 INFO:teuthology.orchestra.run.smithi049.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2023-11-06T02:04:53.208 INFO:teuthology.orchestra.run.smithi150.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2023-11-06T02:04:53.220 INFO:teuthology.orchestra.run.smithi150.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2023-11-06T02:04:53.266 INFO:teuthology.orchestra.run.smithi150.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 18.2.0-1181-gf7e9f9af-1jammy [3,104 kB] 2023-11-06T02:04:53.271 INFO:teuthology.orchestra.run.smithi150.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2023-11-06T02:04:53.419 INFO:teuthology.orchestra.run.smithi049.stdout:Get:83 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 18.2.0-1181-gf7e9f9af-1jammy [3,104 kB] 2023-11-06T02:04:53.461 INFO:teuthology.orchestra.run.smithi150.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2023-11-06T02:04:53.471 INFO:teuthology.orchestra.run.smithi150.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2023-11-06T02:04:53.494 INFO:teuthology.orchestra.run.smithi150.stdout:Get:13 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 18.2.0-1181-gf7e9f9af-1jammy [669 kB] 2023-11-06T02:04:53.499 INFO:teuthology.orchestra.run.smithi150.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2023-11-06T02:04:53.531 INFO:teuthology.orchestra.run.smithi150.stdout:Get:15 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 18.2.0-1181-gf7e9f9af-1jammy [356 kB] 2023-11-06T02:04:53.563 INFO:teuthology.orchestra.run.smithi150.stdout:Get:16 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 18.2.0-1181-gf7e9f9af-1jammy [32.8 kB] 2023-11-06T02:04:53.566 INFO:teuthology.orchestra.run.smithi150.stdout:Get:17 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 18.2.0-1181-gf7e9f9af-1jammy [180 kB] 2023-11-06T02:04:53.568 INFO:teuthology.orchestra.run.smithi150.stdout:Get:18 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 18.2.0-1181-gf7e9f9af-1jammy [61.5 kB] 2023-11-06T02:04:53.569 INFO:teuthology.orchestra.run.smithi150.stdout:Get:19 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 18.2.0-1181-gf7e9f9af-1jammy [330 kB] 2023-11-06T02:04:53.599 INFO:teuthology.orchestra.run.smithi150.stdout:Get:20 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 18.2.0-1181-gf7e9f9af-1jammy [5,886 kB] 2023-11-06T02:04:53.652 INFO:teuthology.orchestra.run.smithi049.stdout:Get:84 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 18.2.0-1181-gf7e9f9af-1jammy [669 kB] 2023-11-06T02:04:53.697 INFO:teuthology.orchestra.run.smithi049.stdout:Get:85 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 18.2.0-1181-gf7e9f9af-1jammy [356 kB] 2023-11-06T02:04:53.700 INFO:teuthology.orchestra.run.smithi150.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2023-11-06T02:04:53.731 INFO:teuthology.orchestra.run.smithi049.stdout:Get:86 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 18.2.0-1181-gf7e9f9af-1jammy [32.8 kB] 2023-11-06T02:04:53.733 INFO:teuthology.orchestra.run.smithi049.stdout:Get:87 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 18.2.0-1181-gf7e9f9af-1jammy [180 kB] 2023-11-06T02:04:53.738 INFO:teuthology.orchestra.run.smithi049.stdout:Get:88 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 18.2.0-1181-gf7e9f9af-1jammy [61.5 kB] 2023-11-06T02:04:53.740 INFO:teuthology.orchestra.run.smithi049.stdout:Get:89 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 18.2.0-1181-gf7e9f9af-1jammy [330 kB] 2023-11-06T02:04:53.776 INFO:teuthology.orchestra.run.smithi049.stdout:Get:90 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 18.2.0-1181-gf7e9f9af-1jammy [5,886 kB] 2023-11-06T02:04:53.933 INFO:teuthology.orchestra.run.smithi150.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2023-11-06T02:04:53.949 INFO:teuthology.orchestra.run.smithi150.stdout:Get:23 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 18.2.0-1181-gf7e9f9af-1jammy [112 kB] 2023-11-06T02:04:53.951 INFO:teuthology.orchestra.run.smithi150.stdout:Get:24 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 18.2.0-1181-gf7e9f9af-1jammy [442 kB] 2023-11-06T02:04:53.973 INFO:teuthology.orchestra.run.smithi150.stdout:Get:25 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 18.2.0-1181-gf7e9f9af-1jammy [22.0 MB] 2023-11-06T02:04:54.215 INFO:teuthology.orchestra.run.smithi049.stdout:Get:91 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 18.2.0-1181-gf7e9f9af-1jammy [112 kB] 2023-11-06T02:04:54.218 INFO:teuthology.orchestra.run.smithi049.stdout:Get:92 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 18.2.0-1181-gf7e9f9af-1jammy [442 kB] 2023-11-06T02:04:54.242 INFO:teuthology.orchestra.run.smithi049.stdout:Get:93 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 18.2.0-1181-gf7e9f9af-1jammy [22.0 MB] 2023-11-06T02:04:54.345 INFO:teuthology.orchestra.run.smithi150.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2023-11-06T02:04:54.355 INFO:teuthology.orchestra.run.smithi150.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2023-11-06T02:04:54.355 INFO:teuthology.orchestra.run.smithi150.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2023-11-06T02:04:54.368 INFO:teuthology.orchestra.run.smithi150.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2023-11-06T02:04:54.369 INFO:teuthology.orchestra.run.smithi150.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2023-11-06T02:04:54.371 INFO:teuthology.orchestra.run.smithi150.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2023-11-06T02:04:54.373 INFO:teuthology.orchestra.run.smithi150.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2023-11-06T02:04:54.375 INFO:teuthology.orchestra.run.smithi150.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2023-11-06T02:04:54.376 INFO:teuthology.orchestra.run.smithi150.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2023-11-06T02:04:54.569 INFO:teuthology.orchestra.run.smithi150.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2023-11-06T02:04:55.072 INFO:teuthology.orchestra.run.smithi150.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2023-11-06T02:04:55.083 INFO:teuthology.orchestra.run.smithi150.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2023-11-06T02:04:55.089 INFO:teuthology.orchestra.run.smithi150.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2023-11-06T02:04:55.094 INFO:teuthology.orchestra.run.smithi150.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2023-11-06T02:04:55.114 INFO:teuthology.orchestra.run.smithi150.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2023-11-06T02:04:55.119 INFO:teuthology.orchestra.run.smithi150.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2023-11-06T02:04:55.121 INFO:teuthology.orchestra.run.smithi150.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2023-11-06T02:04:55.171 INFO:teuthology.orchestra.run.smithi150.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2023-11-06T02:04:55.187 INFO:teuthology.orchestra.run.smithi150.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2023-11-06T02:04:55.192 INFO:teuthology.orchestra.run.smithi150.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2023-11-06T02:04:55.358 INFO:teuthology.orchestra.run.smithi150.stdout:Get:46 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 18.2.0-1181-gf7e9f9af-1jammy [6,838 kB] 2023-11-06T02:04:55.544 INFO:teuthology.orchestra.run.smithi150.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2023-11-06T02:04:55.640 INFO:teuthology.orchestra.run.smithi150.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2023-11-06T02:04:55.710 INFO:teuthology.orchestra.run.smithi150.stdout:Get:49 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 18.2.0-1181-gf7e9f9af-1jammy [237 kB] 2023-11-06T02:04:55.715 INFO:teuthology.orchestra.run.smithi150.stdout:Get:50 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 18.2.0-1181-gf7e9f9af-1jammy [129 kB] 2023-11-06T02:04:55.718 INFO:teuthology.orchestra.run.smithi150.stdout:Get:51 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 18.2.0-1181-gf7e9f9af-1jammy [1,483 kB] 2023-11-06T02:04:55.748 INFO:teuthology.orchestra.run.smithi150.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2023-11-06T02:04:55.800 INFO:teuthology.orchestra.run.smithi150.stdout:Get:53 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 18.2.0-1181-gf7e9f9af-1jammy [9,877 kB] 2023-11-06T02:04:55.812 INFO:teuthology.orchestra.run.smithi150.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2023-11-06T02:04:55.943 INFO:teuthology.orchestra.run.smithi049.stdout:Get:94 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 18.2.0-1181-gf7e9f9af-1jammy [6,838 kB] 2023-11-06T02:04:55.952 INFO:teuthology.orchestra.run.smithi150.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2023-11-06T02:04:56.055 INFO:teuthology.orchestra.run.smithi150.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2023-11-06T02:04:56.076 INFO:teuthology.orchestra.run.smithi150.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-urllib3 all 1.26.5-1~exp1 [96.4 kB] 2023-11-06T02:04:56.108 INFO:teuthology.orchestra.run.smithi150.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2023-11-06T02:04:56.122 INFO:teuthology.orchestra.run.smithi150.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2023-11-06T02:04:56.175 INFO:teuthology.orchestra.run.smithi150.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2023-11-06T02:04:56.183 INFO:teuthology.orchestra.run.smithi150.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2023-11-06T02:04:56.372 INFO:teuthology.orchestra.run.smithi049.stdout:Get:95 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 18.2.0-1181-gf7e9f9af-1jammy [237 kB] 2023-11-06T02:04:56.381 INFO:teuthology.orchestra.run.smithi049.stdout:Get:96 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 18.2.0-1181-gf7e9f9af-1jammy [129 kB] 2023-11-06T02:04:56.394 INFO:teuthology.orchestra.run.smithi150.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2023-11-06T02:04:56.417 INFO:teuthology.orchestra.run.smithi049.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 18.2.0-1181-gf7e9f9af-1jammy [1,483 kB] 2023-11-06T02:04:56.419 INFO:teuthology.orchestra.run.smithi150.stdout:Get:63 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 18.2.0-1181-gf7e9f9af-1jammy [31.5 MB] 2023-11-06T02:04:56.605 INFO:teuthology.orchestra.run.smithi049.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 18.2.0-1181-gf7e9f9af-1jammy [9,877 kB] 2023-11-06T02:04:57.243 INFO:teuthology.orchestra.run.smithi049.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 18.2.0-1181-gf7e9f9af-1jammy [31.5 MB] 2023-11-06T02:04:57.399 INFO:teuthology.orchestra.run.smithi150.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2023-11-06T02:04:57.401 INFO:teuthology.orchestra.run.smithi150.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jinja2 all 3.0.3-1 [108 kB] 2023-11-06T02:04:57.403 INFO:teuthology.orchestra.run.smithi150.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2023-11-06T02:04:57.405 INFO:teuthology.orchestra.run.smithi150.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2023-11-06T02:04:57.406 INFO:teuthology.orchestra.run.smithi150.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2023-11-06T02:04:57.408 INFO:teuthology.orchestra.run.smithi150.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2023-11-06T02:04:57.408 INFO:teuthology.orchestra.run.smithi150.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2023-11-06T02:04:57.409 INFO:teuthology.orchestra.run.smithi150.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2023-11-06T02:04:57.413 INFO:teuthology.orchestra.run.smithi150.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2023-11-06T02:04:57.490 INFO:teuthology.orchestra.run.smithi150.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2023-11-06T02:04:57.683 INFO:teuthology.orchestra.run.smithi150.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2023-11-06T02:04:58.342 INFO:teuthology.orchestra.run.smithi150.stdout:Get:75 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 18.2.0-1181-gf7e9f9af-1jammy [14.1 kB] 2023-11-06T02:04:58.343 INFO:teuthology.orchestra.run.smithi150.stdout:Get:76 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 18.2.0-1181-gf7e9f9af-1jammy [824 kB] 2023-11-06T02:04:58.359 INFO:teuthology.orchestra.run.smithi150.stdout:Get:77 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 18.2.0-1181-gf7e9f9af-1jammy [2,170 kB] 2023-11-06T02:04:58.428 INFO:teuthology.orchestra.run.smithi150.stdout:Get:78 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 18.2.0-1181-gf7e9f9af-1jammy [235 kB] 2023-11-06T02:04:58.436 INFO:teuthology.orchestra.run.smithi150.stdout:Get:79 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 18.2.0-1181-gf7e9f9af-1jammy [138 kB] 2023-11-06T02:04:58.456 INFO:teuthology.orchestra.run.smithi150.stdout:Get:80 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 18.2.0-1181-gf7e9f9af-1jammy [1,898 kB] 2023-11-06T02:04:58.506 INFO:teuthology.orchestra.run.smithi150.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2023-11-06T02:04:58.565 INFO:teuthology.orchestra.run.smithi150.stdout:Get:82 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 18.2.0-1181-gf7e9f9af-1jammy [8,625 kB] 2023-11-06T02:04:58.617 INFO:teuthology.orchestra.run.smithi150.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2023-11-06T02:04:58.617 INFO:teuthology.orchestra.run.smithi150.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2023-11-06T02:04:58.659 INFO:teuthology.orchestra.run.smithi150.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2023-11-06T02:04:58.659 INFO:teuthology.orchestra.run.smithi150.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2023-11-06T02:04:58.660 INFO:teuthology.orchestra.run.smithi150.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2023-11-06T02:04:58.661 INFO:teuthology.orchestra.run.smithi150.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2023-11-06T02:04:58.662 INFO:teuthology.orchestra.run.smithi150.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2023-11-06T02:04:58.663 INFO:teuthology.orchestra.run.smithi150.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2023-11-06T02:04:58.843 INFO:teuthology.orchestra.run.smithi150.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2023-11-06T02:04:58.977 INFO:teuthology.orchestra.run.smithi150.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2023-11-06T02:04:59.069 INFO:teuthology.orchestra.run.smithi150.stdout:Get:93 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 18.2.0-1181-gf7e9f9af-1jammy [14.2 kB] 2023-11-06T02:04:59.069 INFO:teuthology.orchestra.run.smithi150.stdout:Get:94 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 18.2.0-1181-gf7e9f9af-1jammy [46.5 MB] 2023-11-06T02:04:59.311 INFO:teuthology.orchestra.run.smithi150.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2023-11-06T02:04:59.362 INFO:teuthology.orchestra.run.smithi150.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2023-11-06T02:04:59.391 INFO:teuthology.orchestra.run.smithi150.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2023-11-06T02:04:59.400 INFO:teuthology.orchestra.run.smithi150.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2023-11-06T02:04:59.545 INFO:teuthology.orchestra.run.smithi150.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2023-11-06T02:04:59.552 INFO:teuthology.orchestra.run.smithi150.stdout:Get:100 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2023-11-06T02:04:59.553 INFO:teuthology.orchestra.run.smithi150.stdout:Get:101 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2023-11-06T02:04:59.610 INFO:teuthology.orchestra.run.smithi150.stdout:Get:102 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2023-11-06T02:04:59.627 INFO:teuthology.orchestra.run.smithi150.stdout:Get:103 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2023-11-06T02:04:59.646 INFO:teuthology.orchestra.run.smithi049.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 18.2.0-1181-gf7e9f9af-1jammy [14.1 kB] 2023-11-06T02:04:59.647 INFO:teuthology.orchestra.run.smithi049.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 18.2.0-1181-gf7e9f9af-1jammy [824 kB] 2023-11-06T02:04:59.664 INFO:teuthology.orchestra.run.smithi049.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 18.2.0-1181-gf7e9f9af-1jammy [2,170 kB] 2023-11-06T02:04:59.777 INFO:teuthology.orchestra.run.smithi049.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 18.2.0-1181-gf7e9f9af-1jammy [235 kB] 2023-11-06T02:04:59.787 INFO:teuthology.orchestra.run.smithi049.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 18.2.0-1181-gf7e9f9af-1jammy [138 kB] 2023-11-06T02:04:59.794 INFO:teuthology.orchestra.run.smithi150.stdout:Get:104 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2023-11-06T02:04:59.807 INFO:teuthology.orchestra.run.smithi049.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 18.2.0-1181-gf7e9f9af-1jammy [1,898 kB] 2023-11-06T02:04:59.934 INFO:teuthology.orchestra.run.smithi049.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 18.2.0-1181-gf7e9f9af-1jammy [8,625 kB] 2023-11-06T02:05:00.189 INFO:teuthology.orchestra.run.smithi150.stdout:Get:105 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2023-11-06T02:05:00.417 INFO:teuthology.orchestra.run.smithi150.stdout:Get:106 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2023-11-06T02:05:00.421 INFO:teuthology.orchestra.run.smithi150.stdout:Get:107 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2023-11-06T02:05:00.429 INFO:teuthology.orchestra.run.smithi150.stdout:Get:108 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2023-11-06T02:05:00.578 INFO:teuthology.orchestra.run.smithi049.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 18.2.0-1181-gf7e9f9af-1jammy [14.2 kB] 2023-11-06T02:05:00.578 INFO:teuthology.orchestra.run.smithi049.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 18.2.0-1181-gf7e9f9af-1jammy [46.5 MB] 2023-11-06T02:05:02.219 INFO:teuthology.orchestra.run.smithi150.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 18.2.0-1181-gf7e9f9af-1jammy [120 kB] 2023-11-06T02:05:02.221 INFO:teuthology.orchestra.run.smithi150.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 18.2.0-1181-gf7e9f9af-1jammy [228 kB] 2023-11-06T02:05:02.226 INFO:teuthology.orchestra.run.smithi150.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 18.2.0-1181-gf7e9f9af-1jammy [34.0 kB] 2023-11-06T02:05:02.227 INFO:teuthology.orchestra.run.smithi150.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 18.2.0-1181-gf7e9f9af-1jammy [23.3 kB] 2023-11-06T02:05:02.227 INFO:teuthology.orchestra.run.smithi150.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 18.2.0-1181-gf7e9f9af-1jammy [38.3 kB] 2023-11-06T02:05:02.228 INFO:teuthology.orchestra.run.smithi150.stdout:Get:114 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 18.2.0-1181-gf7e9f9af-1jammy [11.6 MB] 2023-11-06T02:05:02.768 INFO:teuthology.orchestra.run.smithi150.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 18.2.0-1181-gf7e9f9af-1jammy [92.9 kB] 2023-11-06T02:05:03.324 INFO:teuthology.orchestra.run.smithi150.stdout:Fetched 181 MB in 10s (17.7 MB/s) 2023-11-06T02:05:03.468 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-pyperclip. 2023-11-06T02:05:03.515 INFO:teuthology.orchestra.run.smithi049.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 18.2.0-1181-gf7e9f9af-1jammy [120 kB] 2023-11-06T02:05:03.517 INFO:teuthology.orchestra.run.smithi049.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 18.2.0-1181-gf7e9f9af-1jammy [228 kB] 2023-11-06T02:05:03.522 INFO:teuthology.orchestra.run.smithi049.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 18.2.0-1181-gf7e9f9af-1jammy [34.0 kB] 2023-11-06T02:05:03.523 INFO:teuthology.orchestra.run.smithi049.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 18.2.0-1181-gf7e9f9af-1jammy [23.3 kB] 2023-11-06T02:05:03.524 INFO:teuthology.orchestra.run.smithi049.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 18.2.0-1181-gf7e9f9af-1jammy [38.3 kB] 2023-11-06T02:05:03.525 INFO:teuthology.orchestra.run.smithi049.stdout:Get:114 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 18.2.0-1181-gf7e9f9af-1jammy [11.6 MB] 2023-11-06T02:05:03.565 INFO:teuthology.orchestra.run.smithi150.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165985 files and directories currently installed.) 2023-11-06T02:05:03.570 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2023-11-06T02:05:03.616 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2023-11-06T02:05:03.840 INFO:teuthology.orchestra.run.smithi049.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 18.2.0-1181-gf7e9f9af-1jammy [92.9 kB] 2023-11-06T02:05:03.894 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-wcwidth. 2023-11-06T02:05:03.909 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2023-11-06T02:05:03.952 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2023-11-06T02:05:04.288 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-cmd2. 2023-11-06T02:05:04.315 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2023-11-06T02:05:04.355 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2023-11-06T02:05:04.421 INFO:teuthology.orchestra.run.smithi049.stdout:Fetched 181 MB in 11s (16.2 MB/s) 2023-11-06T02:05:04.563 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package python3-pyperclip. 2023-11-06T02:05:04.654 INFO:teuthology.orchestra.run.smithi049.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165985 files and directories currently installed.) 2023-11-06T02:05:04.660 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2023-11-06T02:05:04.683 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package libfmt8:amd64. 2023-11-06T02:05:04.704 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2023-11-06T02:05:04.709 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../003-libfmt8_8.1.1+ds1-2_amd64.deb ... 2023-11-06T02:05:04.791 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking libfmt8:amd64 (8.1.1+ds1-2) ... 2023-11-06T02:05:04.956 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package python3-wcwidth. 2023-11-06T02:05:04.972 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2023-11-06T02:05:05.023 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2023-11-06T02:05:05.128 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2023-11-06T02:05:05.155 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../004-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2023-11-06T02:05:05.194 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2023-11-06T02:05:05.326 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package python3-cmd2. 2023-11-06T02:05:05.353 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2023-11-06T02:05:05.393 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2023-11-06T02:05:05.490 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2023-11-06T02:05:05.516 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../005-libdouble-conversion3_3.1.7-4_amd64.deb ... 2023-11-06T02:05:05.548 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2023-11-06T02:05:05.704 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package libfmt8:amd64. 2023-11-06T02:05:05.731 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../003-libfmt8_8.1.1+ds1-2_amd64.deb ... 2023-11-06T02:05:05.812 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking libfmt8:amd64 (8.1.1+ds1-2) ... 2023-11-06T02:05:05.884 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2023-11-06T02:05:05.910 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../006-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2023-11-06T02:05:05.951 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2023-11-06T02:05:06.149 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2023-11-06T02:05:06.176 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../004-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2023-11-06T02:05:06.215 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2023-11-06T02:05:06.396 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package libqt5core5a:amd64. 2023-11-06T02:05:06.423 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../007-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-11-06T02:05:06.470 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-11-06T02:05:06.735 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2023-11-06T02:05:06.762 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../005-libdouble-conversion3_3.1.7-4_amd64.deb ... 2023-11-06T02:05:06.793 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2023-11-06T02:05:06.841 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2023-11-06T02:05:06.867 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../008-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-11-06T02:05:06.899 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-11-06T02:05:07.088 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2023-11-06T02:05:07.115 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../006-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2023-11-06T02:05:07.146 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2023-11-06T02:05:07.252 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package libqt5network5:amd64. 2023-11-06T02:05:07.279 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../009-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-11-06T02:05:07.319 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-11-06T02:05:07.600 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package libqt5core5a:amd64. 2023-11-06T02:05:07.627 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../007-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-11-06T02:05:07.647 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2023-11-06T02:05:07.674 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../010-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2023-11-06T02:05:07.675 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-11-06T02:05:07.713 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2023-11-06T02:05:07.999 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package librados2. 2023-11-06T02:05:08.015 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../011-librados2_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T02:05:08.049 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking librados2 (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:05:08.069 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2023-11-06T02:05:08.096 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../008-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-11-06T02:05:08.136 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-11-06T02:05:08.431 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package libqt5network5:amd64. 2023-11-06T02:05:08.458 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../009-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-11-06T02:05:08.489 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-11-06T02:05:08.851 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2023-11-06T02:05:08.877 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../010-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2023-11-06T02:05:08.917 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2023-11-06T02:05:08.961 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package librbd1. 2023-11-06T02:05:08.987 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../012-librbd1_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T02:05:09.027 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking librbd1 (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:05:09.245 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package librados2. 2023-11-06T02:05:09.272 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../011-librados2_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T02:05:09.311 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking librados2 (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:05:09.480 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package libcephfs2. 2023-11-06T02:05:09.506 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../013-libcephfs2_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T02:05:09.547 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking libcephfs2 (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:05:09.748 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package librbd1. 2023-11-06T02:05:09.775 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../012-librbd1_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T02:05:09.814 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking librbd1 (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:05:09.858 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-rados. 2023-11-06T02:05:09.884 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../014-python3-rados_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T02:05:09.924 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-rados (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:05:10.228 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-ceph-argparse. 2023-11-06T02:05:10.251 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package libcephfs2. 2023-11-06T02:05:10.254 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../015-python3-ceph-argparse_18.2.0-1181-gf7e9f9af-1jammy_all.deb ... 2023-11-06T02:05:10.277 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../013-libcephfs2_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T02:05:10.294 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-ceph-argparse (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:05:10.317 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking libcephfs2 (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:05:10.546 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-cephfs. 2023-11-06T02:05:10.573 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../016-python3-cephfs_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T02:05:10.578 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package python3-rados. 2023-11-06T02:05:10.596 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../014-python3-rados_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T02:05:10.613 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-cephfs (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:05:10.628 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking python3-rados (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:05:10.907 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-ceph-common. 2023-11-06T02:05:10.930 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package python3-ceph-argparse. 2023-11-06T02:05:10.934 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../017-python3-ceph-common_18.2.0-1181-gf7e9f9af-1jammy_all.deb ... 2023-11-06T02:05:10.957 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../015-python3-ceph-argparse_18.2.0-1181-gf7e9f9af-1jammy_all.deb ... 2023-11-06T02:05:10.974 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-ceph-common (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:05:10.997 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking python3-ceph-argparse (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:05:11.261 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-prettytable. 2023-11-06T02:05:11.287 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../018-python3-prettytable_2.5.0-2_all.deb ... 2023-11-06T02:05:11.291 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package python3-cephfs. 2023-11-06T02:05:11.318 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../016-python3-cephfs_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T02:05:11.319 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2023-11-06T02:05:11.358 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking python3-cephfs (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:05:11.563 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-rbd. 2023-11-06T02:05:11.581 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../019-python3-rbd_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T02:05:11.638 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-rbd (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:05:11.669 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package python3-ceph-common. 2023-11-06T02:05:11.696 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../017-python3-ceph-common_18.2.0-1181-gf7e9f9af-1jammy_all.deb ... 2023-11-06T02:05:11.735 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking python3-ceph-common (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:05:11.941 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package librdkafka1:amd64. 2023-11-06T02:05:11.968 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../020-librdkafka1_1.8.0-1build1_amd64.deb ... 2023-11-06T02:05:12.007 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2023-11-06T02:05:12.063 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package python3-prettytable. 2023-11-06T02:05:12.090 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../018-python3-prettytable_2.5.0-2_all.deb ... 2023-11-06T02:05:12.130 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2023-11-06T02:05:12.335 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package librgw2. 2023-11-06T02:05:12.361 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../021-librgw2_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T02:05:12.399 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package python3-rbd. 2023-11-06T02:05:12.402 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking librgw2 (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:05:12.426 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../019-python3-rbd_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T02:05:12.466 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking python3-rbd (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:05:12.819 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package librdkafka1:amd64. 2023-11-06T02:05:12.846 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../020-librdkafka1_1.8.0-1build1_amd64.deb ... 2023-11-06T02:05:12.885 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2023-11-06T02:05:12.929 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-rgw. 2023-11-06T02:05:12.956 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../022-python3-rgw_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T02:05:12.996 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-rgw (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:05:13.213 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package librgw2. 2023-11-06T02:05:13.240 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../021-librgw2_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T02:05:13.280 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking librgw2 (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:05:13.307 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package liboath0:amd64. 2023-11-06T02:05:13.334 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../023-liboath0_2.6.7-3build1_amd64.deb ... 2023-11-06T02:05:13.374 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2023-11-06T02:05:13.660 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package libradosstriper1. 2023-11-06T02:05:13.686 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../024-libradosstriper1_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T02:05:13.726 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking libradosstriper1 (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:05:13.757 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package python3-rgw. 2023-11-06T02:05:13.784 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../022-python3-rgw_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T02:05:13.824 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking python3-rgw (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:05:14.152 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package liboath0:amd64. 2023-11-06T02:05:14.155 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package ceph-common. 2023-11-06T02:05:14.178 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../023-liboath0_2.6.7-3build1_amd64.deb ... 2023-11-06T02:05:14.182 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../025-ceph-common_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T02:05:14.218 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2023-11-06T02:05:14.221 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking ceph-common (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:05:14.504 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package libradosstriper1. 2023-11-06T02:05:14.531 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../024-libradosstriper1_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T02:05:14.571 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking libradosstriper1 (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:05:14.949 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package ceph-common. 2023-11-06T02:05:14.976 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../025-ceph-common_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T02:05:15.016 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking ceph-common (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:05:15.519 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package ceph-base. 2023-11-06T02:05:15.546 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../026-ceph-base_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T02:05:15.603 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking ceph-base (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:05:16.288 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package ceph-base. 2023-11-06T02:05:16.315 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-jaraco.functools. 2023-11-06T02:05:16.315 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../026-ceph-base_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T02:05:16.330 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../027-python3-jaraco.functools_3.4.0-2_all.deb ... 2023-11-06T02:05:16.364 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking ceph-base (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:05:16.364 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2023-11-06T02:05:16.667 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-cheroot. 2023-11-06T02:05:16.694 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../028-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2023-11-06T02:05:16.734 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2023-11-06T02:05:17.062 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-jaraco.classes. 2023-11-06T02:05:17.089 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../029-python3-jaraco.classes_3.2.1-3_all.deb ... 2023-11-06T02:05:17.128 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2023-11-06T02:05:17.175 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package python3-jaraco.functools. 2023-11-06T02:05:17.203 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../027-python3-jaraco.functools_3.4.0-2_all.deb ... 2023-11-06T02:05:17.233 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2023-11-06T02:05:17.406 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-jaraco.text. 2023-11-06T02:05:17.433 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../030-python3-jaraco.text_3.6.0-2_all.deb ... 2023-11-06T02:05:17.465 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2023-11-06T02:05:17.536 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package python3-cheroot. 2023-11-06T02:05:17.563 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../028-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2023-11-06T02:05:17.603 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2023-11-06T02:05:17.743 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-jaraco.collections. 2023-11-06T02:05:17.769 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../031-python3-jaraco.collections_3.4.0-2_all.deb ... 2023-11-06T02:05:17.809 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2023-11-06T02:05:17.922 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package python3-jaraco.classes. 2023-11-06T02:05:17.949 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../029-python3-jaraco.classes_3.2.1-3_all.deb ... 2023-11-06T02:05:17.989 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2023-11-06T02:05:18.121 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-tempora. 2023-11-06T02:05:18.147 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../032-python3-tempora_4.1.2-1_all.deb ... 2023-11-06T02:05:18.187 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-tempora (4.1.2-1) ... 2023-11-06T02:05:18.267 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package python3-jaraco.text. 2023-11-06T02:05:18.283 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../030-python3-jaraco.text_3.6.0-2_all.deb ... 2023-11-06T02:05:18.317 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2023-11-06T02:05:18.490 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-portend. 2023-11-06T02:05:18.517 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../033-python3-portend_3.0.0-1_all.deb ... 2023-11-06T02:05:18.556 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-portend (3.0.0-1) ... 2023-11-06T02:05:18.620 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package python3-jaraco.collections. 2023-11-06T02:05:18.647 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../031-python3-jaraco.collections_3.4.0-2_all.deb ... 2023-11-06T02:05:18.686 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2023-11-06T02:05:18.801 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-zc.lockfile. 2023-11-06T02:05:18.828 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../034-python3-zc.lockfile_2.0-1_all.deb ... 2023-11-06T02:05:18.859 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2023-11-06T02:05:19.006 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package python3-tempora. 2023-11-06T02:05:19.033 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../032-python3-tempora_4.1.2-1_all.deb ... 2023-11-06T02:05:19.072 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking python3-tempora (4.1.2-1) ... 2023-11-06T02:05:19.342 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package python3-portend. 2023-11-06T02:05:19.346 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-cherrypy3. 2023-11-06T02:05:19.369 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../033-python3-portend_3.0.0-1_all.deb ... 2023-11-06T02:05:19.373 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../035-python3-cherrypy3_18.6.1-4_all.deb ... 2023-11-06T02:05:19.408 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking python3-portend (3.0.0-1) ... 2023-11-06T02:05:19.412 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2023-11-06T02:05:19.670 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package python3-zc.lockfile. 2023-11-06T02:05:19.696 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../034-python3-zc.lockfile_2.0-1_all.deb ... 2023-11-06T02:05:19.732 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-natsort. 2023-11-06T02:05:19.736 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2023-11-06T02:05:19.759 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../036-python3-natsort_8.0.2-1_all.deb ... 2023-11-06T02:05:19.790 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-natsort (8.0.2-1) ... 2023-11-06T02:05:20.022 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package python3-cherrypy3. 2023-11-06T02:05:20.048 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../035-python3-cherrypy3_18.6.1-4_all.deb ... 2023-11-06T02:05:20.077 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-logutils. 2023-11-06T02:05:20.080 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2023-11-06T02:05:20.104 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../037-python3-logutils_0.3.3-8_all.deb ... 2023-11-06T02:05:20.144 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-logutils (0.3.3-8) ... 2023-11-06T02:05:20.429 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package python3-natsort. 2023-11-06T02:05:20.447 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-markupsafe. 2023-11-06T02:05:20.453 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../036-python3-natsort_8.0.2-1_all.deb ... 2023-11-06T02:05:20.474 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../038-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2023-11-06T02:05:20.492 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking python3-natsort (8.0.2-1) ... 2023-11-06T02:05:20.531 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2023-11-06T02:05:20.788 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package python3-logutils. 2023-11-06T02:05:20.814 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../037-python3-logutils_0.3.3-8_all.deb ... 2023-11-06T02:05:20.838 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-mako. 2023-11-06T02:05:20.854 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking python3-logutils (0.3.3-8) ... 2023-11-06T02:05:20.865 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../039-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2023-11-06T02:05:20.908 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2023-11-06T02:05:21.114 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package python3-markupsafe. 2023-11-06T02:05:21.130 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../038-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2023-11-06T02:05:21.173 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2023-11-06T02:05:21.202 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-simplegeneric. 2023-11-06T02:05:21.229 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../040-python3-simplegeneric_0.8.1-3_all.deb ... 2023-11-06T02:05:21.271 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2023-11-06T02:05:21.459 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package python3-mako. 2023-11-06T02:05:21.486 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../039-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2023-11-06T02:05:21.526 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2023-11-06T02:05:21.557 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-singledispatch. 2023-11-06T02:05:21.584 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../041-python3-singledispatch_3.4.0.3-3_all.deb ... 2023-11-06T02:05:21.624 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2023-11-06T02:05:21.854 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package python3-simplegeneric. 2023-11-06T02:05:21.881 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../040-python3-simplegeneric_0.8.1-3_all.deb ... 2023-11-06T02:05:21.893 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-webob. 2023-11-06T02:05:21.919 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../042-python3-webob_1%3a1.8.6-1.1_all.deb ... 2023-11-06T02:05:21.960 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2023-11-06T02:05:21.962 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2023-11-06T02:05:22.265 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package python3-singledispatch. 2023-11-06T02:05:22.279 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-waitress. 2023-11-06T02:05:22.292 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../041-python3-singledispatch_3.4.0.3-3_all.deb ... 2023-11-06T02:05:22.306 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../043-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2023-11-06T02:05:22.331 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2023-11-06T02:05:22.341 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2023-11-06T02:05:22.599 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-tempita. 2023-11-06T02:05:22.601 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package python3-webob. 2023-11-06T02:05:22.625 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../044-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2023-11-06T02:05:22.628 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../042-python3-webob_1%3a1.8.6-1.1_all.deb ... 2023-11-06T02:05:22.666 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2023-11-06T02:05:22.667 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2023-11-06T02:05:22.952 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-paste. 2023-11-06T02:05:22.962 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package python3-waitress. 2023-11-06T02:05:22.978 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../045-python3-paste_3.5.0+dfsg1-1_all.deb ... 2023-11-06T02:05:22.989 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../043-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2023-11-06T02:05:23.018 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2023-11-06T02:05:23.024 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2023-11-06T02:05:23.307 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package python3-tempita. 2023-11-06T02:05:23.333 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../044-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2023-11-06T02:05:23.346 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2023-11-06T02:05:23.361 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../046-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2023-11-06T02:05:23.373 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2023-11-06T02:05:23.404 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2023-11-06T02:05:23.651 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package python3-paste. 2023-11-06T02:05:23.678 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../045-python3-paste_3.5.0+dfsg1-1_all.deb ... 2023-11-06T02:05:23.683 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-pastedeploy. 2023-11-06T02:05:23.699 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../047-python3-pastedeploy_2.1.1-1_all.deb ... 2023-11-06T02:05:23.732 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2023-11-06T02:05:23.893 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2023-11-06T02:05:24.035 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-webtest. 2023-11-06T02:05:24.062 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../048-python3-webtest_2.0.35-1_all.deb ... 2023-11-06T02:05:24.102 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-webtest (2.0.35-1) ... 2023-11-06T02:05:24.421 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-pecan. 2023-11-06T02:05:24.448 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../049-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2023-11-06T02:05:24.488 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2023-11-06T02:05:24.843 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-certifi. 2023-11-06T02:05:24.870 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../050-python3-certifi_2020.6.20-1_all.deb ... 2023-11-06T02:05:24.909 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2023-11-06T02:05:24.946 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2023-11-06T02:05:24.973 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../046-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2023-11-06T02:05:25.012 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2023-11-06T02:05:25.262 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-idna. 2023-11-06T02:05:25.265 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package python3-pastedeploy. 2023-11-06T02:05:25.288 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../051-python3-idna_3.3-1_all.deb ... 2023-11-06T02:05:25.292 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../047-python3-pastedeploy_2.1.1-1_all.deb ... 2023-11-06T02:05:25.328 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-idna (3.3-1) ... 2023-11-06T02:05:25.332 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2023-11-06T02:05:25.615 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-urllib3. 2023-11-06T02:05:25.631 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../052-python3-urllib3_1.26.5-1~exp1_all.deb ... 2023-11-06T02:05:25.664 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1) ... 2023-11-06T02:05:25.668 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package python3-webtest. 2023-11-06T02:05:25.695 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../048-python3-webtest_2.0.35-1_all.deb ... 2023-11-06T02:05:25.735 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking python3-webtest (2.0.35-1) ... 2023-11-06T02:05:26.003 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-requests. 2023-11-06T02:05:26.030 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../053-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2023-11-06T02:05:26.054 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package python3-pecan. 2023-11-06T02:05:26.082 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../049-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2023-11-06T02:05:26.121 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2023-11-06T02:05:26.368 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2023-11-06T02:05:26.484 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package python3-certifi. 2023-11-06T02:05:26.512 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../050-python3-certifi_2020.6.20-1_all.deb ... 2023-11-06T02:05:26.550 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2023-11-06T02:05:26.878 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package python3-idna. 2023-11-06T02:05:26.905 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../051-python3-idna_3.3-1_all.deb ... 2023-11-06T02:05:26.945 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking python3-idna (3.3-1) ... 2023-11-06T02:05:27.029 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-werkzeug. 2023-11-06T02:05:27.056 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../054-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2023-11-06T02:05:27.096 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2023-11-06T02:05:27.198 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package python3-urllib3. 2023-11-06T02:05:27.211 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../052-python3-urllib3_1.26.5-1~exp1_all.deb ... 2023-11-06T02:05:27.247 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1) ... 2023-11-06T02:05:27.374 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2023-11-06T02:05:27.389 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../055-ceph-mgr-modules-core_18.2.0-1181-gf7e9f9af-1jammy_all.deb ... 2023-11-06T02:05:27.432 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking ceph-mgr-modules-core (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:05:27.576 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package python3-requests. 2023-11-06T02:05:27.597 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../053-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2023-11-06T02:05:27.634 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2023-11-06T02:05:27.820 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2023-11-06T02:05:27.846 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../056-libsqlite3-mod-ceph_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T02:05:27.886 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking libsqlite3-mod-ceph (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:05:27.953 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package python3-werkzeug. 2023-11-06T02:05:27.981 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../054-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2023-11-06T02:05:28.020 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2023-11-06T02:05:28.181 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-openssl. 2023-11-06T02:05:28.196 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../057-python3-openssl_21.0.0-1_all.deb ... 2023-11-06T02:05:28.231 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-openssl (21.0.0-1) ... 2023-11-06T02:05:28.265 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2023-11-06T02:05:28.281 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../055-ceph-mgr-modules-core_18.2.0-1181-gf7e9f9af-1jammy_all.deb ... 2023-11-06T02:05:28.323 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking ceph-mgr-modules-core (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:05:28.601 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package ceph-mgr. 2023-11-06T02:05:28.628 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../058-ceph-mgr_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T02:05:28.667 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking ceph-mgr (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:05:28.671 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2023-11-06T02:05:28.689 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../056-libsqlite3-mod-ceph_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T02:05:28.721 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking libsqlite3-mod-ceph (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:05:28.999 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package python3-openssl. 2023-11-06T02:05:29.026 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../057-python3-openssl_21.0.0-1_all.deb ... 2023-11-06T02:05:29.065 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking python3-openssl (21.0.0-1) ... 2023-11-06T02:05:29.070 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package ceph-mon. 2023-11-06T02:05:29.097 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../059-ceph-mon_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T02:05:29.137 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking ceph-mon (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:05:29.427 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package ceph-mgr. 2023-11-06T02:05:29.444 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../058-ceph-mgr_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T02:05:29.485 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking ceph-mgr (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:05:29.838 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package ceph-mon. 2023-11-06T02:05:29.865 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../059-ceph-mon_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T02:05:29.896 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking ceph-mon (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:05:30.182 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package ceph-osd. 2023-11-06T02:05:30.209 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../060-ceph-osd_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T02:05:30.249 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking ceph-osd (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:05:30.975 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package ceph-osd. 2023-11-06T02:05:31.002 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../060-ceph-osd_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T02:05:31.041 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking ceph-osd (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:05:32.110 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package ceph. 2023-11-06T02:05:32.137 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../061-ceph_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T02:05:32.177 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking ceph (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:05:32.446 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package ceph-fuse. 2023-11-06T02:05:32.473 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../062-ceph-fuse_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T02:05:32.513 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking ceph-fuse (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:05:32.891 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package ceph-mds. 2023-11-06T02:05:32.918 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../063-ceph-mds_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T02:05:32.957 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking ceph-mds (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:05:33.036 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package ceph. 2023-11-06T02:05:33.052 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../061-ceph_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T02:05:33.086 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking ceph (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:05:33.344 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package cephadm. 2023-11-06T02:05:33.371 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../064-cephadm_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T02:05:33.388 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package ceph-fuse. 2023-11-06T02:05:33.410 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking cephadm (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:05:33.415 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../062-ceph-fuse_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T02:05:33.455 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking ceph-fuse (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:05:33.672 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-asyncssh. 2023-11-06T02:05:33.687 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../065-python3-asyncssh_2.5.0-1_all.deb ... 2023-11-06T02:05:33.730 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2023-11-06T02:05:33.817 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package ceph-mds. 2023-11-06T02:05:33.844 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../063-ceph-mds_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T02:05:33.883 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking ceph-mds (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:05:34.024 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python-babel-localedata. 2023-11-06T02:05:34.040 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../066-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2023-11-06T02:05:34.082 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2023-11-06T02:05:34.236 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package cephadm. 2023-11-06T02:05:34.264 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../064-cephadm_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T02:05:34.294 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking cephadm (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:05:34.572 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package python3-asyncssh. 2023-11-06T02:05:34.599 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../065-python3-asyncssh_2.5.0-1_all.deb ... 2023-11-06T02:05:34.639 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2023-11-06T02:05:34.966 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package python-babel-localedata. 2023-11-06T02:05:34.982 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../066-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2023-11-06T02:05:35.025 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2023-11-06T02:05:35.096 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-babel. 2023-11-06T02:05:35.124 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../067-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2023-11-06T02:05:35.163 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2023-11-06T02:05:35.491 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-jinja2. 2023-11-06T02:05:35.518 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../068-python3-jinja2_3.0.3-1_all.deb ... 2023-11-06T02:05:35.557 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-jinja2 (3.0.3-1) ... 2023-11-06T02:05:35.911 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2023-11-06T02:05:35.938 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../069-ceph-mgr-cephadm_18.2.0-1181-gf7e9f9af-1jammy_all.deb ... 2023-11-06T02:05:35.977 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking ceph-mgr-cephadm (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:05:36.047 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package python3-babel. 2023-11-06T02:05:36.075 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../067-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2023-11-06T02:05:36.113 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2023-11-06T02:05:36.314 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-bcrypt. 2023-11-06T02:05:36.341 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../070-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2023-11-06T02:05:36.389 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2023-11-06T02:05:36.441 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package python3-jinja2. 2023-11-06T02:05:36.469 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../068-python3-jinja2_3.0.3-1_all.deb ... 2023-11-06T02:05:36.531 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking python3-jinja2 (3.0.3-1) ... 2023-11-06T02:05:36.650 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-pluggy. 2023-11-06T02:05:36.665 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../071-python3-pluggy_0.13.0-7.1_all.deb ... 2023-11-06T02:05:36.699 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2023-11-06T02:05:36.878 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2023-11-06T02:05:36.905 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../069-ceph-mgr-cephadm_18.2.0-1181-gf7e9f9af-1jammy_all.deb ... 2023-11-06T02:05:36.944 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking ceph-mgr-cephadm (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:05:36.986 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-py. 2023-11-06T02:05:37.013 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../072-python3-py_1.10.0-1_all.deb ... 2023-11-06T02:05:37.052 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-py (1.10.0-1) ... 2023-11-06T02:05:37.281 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package python3-bcrypt. 2023-11-06T02:05:37.308 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../070-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2023-11-06T02:05:37.347 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2023-11-06T02:05:37.356 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-iniconfig. 2023-11-06T02:05:37.383 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../073-python3-iniconfig_1.1.1-2_all.deb ... 2023-11-06T02:05:37.423 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2023-11-06T02:05:37.625 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package python3-pluggy. 2023-11-06T02:05:37.652 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../071-python3-pluggy_0.13.0-7.1_all.deb ... 2023-11-06T02:05:37.692 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2023-11-06T02:05:37.750 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-toml. 2023-11-06T02:05:37.778 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../074-python3-toml_0.10.2-1_all.deb ... 2023-11-06T02:05:37.842 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-toml (0.10.2-1) ... 2023-11-06T02:05:38.003 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package python3-py. 2023-11-06T02:05:38.031 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../072-python3-py_1.10.0-1_all.deb ... 2023-11-06T02:05:38.070 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking python3-py (1.10.0-1) ... 2023-11-06T02:05:38.145 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-pytest. 2023-11-06T02:05:38.172 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../075-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2023-11-06T02:05:38.211 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2023-11-06T02:05:38.398 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package python3-iniconfig. 2023-11-06T02:05:38.425 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../073-python3-iniconfig_1.1.1-2_all.deb ... 2023-11-06T02:05:38.464 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2023-11-06T02:05:38.565 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-repoze.lru. 2023-11-06T02:05:38.592 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../076-python3-repoze.lru_0.7-2_all.deb ... 2023-11-06T02:05:38.632 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2023-11-06T02:05:38.726 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package python3-toml. 2023-11-06T02:05:38.753 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../074-python3-toml_0.10.2-1_all.deb ... 2023-11-06T02:05:38.784 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking python3-toml (0.10.2-1) ... 2023-11-06T02:05:38.943 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-routes. 2023-11-06T02:05:38.970 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../077-python3-routes_2.5.1-1ubuntu1_all.deb ... 2023-11-06T02:05:39.009 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2023-11-06T02:05:39.062 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package python3-pytest. 2023-11-06T02:05:39.089 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../075-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2023-11-06T02:05:39.128 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2023-11-06T02:05:39.346 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2023-11-06T02:05:39.373 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../078-ceph-mgr-dashboard_18.2.0-1181-gf7e9f9af-1jammy_all.deb ... 2023-11-06T02:05:39.412 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking ceph-mgr-dashboard (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:05:39.457 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package python3-repoze.lru. 2023-11-06T02:05:39.484 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../076-python3-repoze.lru_0.7-2_all.deb ... 2023-11-06T02:05:39.515 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2023-11-06T02:05:39.785 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package python3-routes. 2023-11-06T02:05:39.812 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../077-python3-routes_2.5.1-1ubuntu1_all.deb ... 2023-11-06T02:05:39.833 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2023-11-06T02:05:39.848 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../079-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2023-11-06T02:05:39.851 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2023-11-06T02:05:39.891 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2023-11-06T02:05:40.188 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2023-11-06T02:05:40.216 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../078-ceph-mgr-dashboard_18.2.0-1181-gf7e9f9af-1jammy_all.deb ... 2023-11-06T02:05:40.255 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking ceph-mgr-dashboard (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:05:40.650 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2023-11-06T02:05:40.661 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-joblib. 2023-11-06T02:05:40.666 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../079-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2023-11-06T02:05:40.688 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../080-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2023-11-06T02:05:40.699 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2023-11-06T02:05:40.728 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2023-11-06T02:05:41.040 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-threadpoolctl. 2023-11-06T02:05:41.067 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../081-python3-threadpoolctl_3.1.0-1_all.deb ... 2023-11-06T02:05:41.106 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2023-11-06T02:05:41.178 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package python3-joblib. 2023-11-06T02:05:41.206 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../080-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2023-11-06T02:05:41.245 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2023-11-06T02:05:41.409 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-sklearn. 2023-11-06T02:05:41.436 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../082-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2023-11-06T02:05:41.475 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2023-11-06T02:05:41.615 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package python3-threadpoolctl. 2023-11-06T02:05:41.643 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../081-python3-threadpoolctl_3.1.0-1_all.deb ... 2023-11-06T02:05:41.681 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2023-11-06T02:05:41.959 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package python3-sklearn. 2023-11-06T02:05:41.987 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../082-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2023-11-06T02:05:42.025 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2023-11-06T02:05:42.253 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2023-11-06T02:05:42.280 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../083-ceph-mgr-diskprediction-local_18.2.0-1181-gf7e9f9af-1jammy_all.deb ... 2023-11-06T02:05:42.310 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking ceph-mgr-diskprediction-local (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:05:42.853 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2023-11-06T02:05:42.880 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../083-ceph-mgr-diskprediction-local_18.2.0-1181-gf7e9f9af-1jammy_all.deb ... 2023-11-06T02:05:42.919 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking ceph-mgr-diskprediction-local (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:05:43.305 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-cachetools. 2023-11-06T02:05:43.332 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../084-python3-cachetools_5.0.0-1_all.deb ... 2023-11-06T02:05:43.372 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2023-11-06T02:05:43.692 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-pyasn1. 2023-11-06T02:05:43.719 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../085-python3-pyasn1_0.4.8-1_all.deb ... 2023-11-06T02:05:43.758 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2023-11-06T02:05:43.822 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package python3-cachetools. 2023-11-06T02:05:43.850 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../084-python3-cachetools_5.0.0-1_all.deb ... 2023-11-06T02:05:43.888 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2023-11-06T02:05:44.044 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-pyasn1-modules. 2023-11-06T02:05:44.061 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../086-python3-pyasn1-modules_0.2.1-1_all.deb ... 2023-11-06T02:05:44.094 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2023-11-06T02:05:44.175 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package python3-pyasn1. 2023-11-06T02:05:44.202 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../085-python3-pyasn1_0.4.8-1_all.deb ... 2023-11-06T02:05:44.241 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2023-11-06T02:05:44.347 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-rsa. 2023-11-06T02:05:44.374 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../087-python3-rsa_4.8-1_all.deb ... 2023-11-06T02:05:44.414 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-rsa (4.8-1) ... 2023-11-06T02:05:44.586 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package python3-pyasn1-modules. 2023-11-06T02:05:44.614 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../086-python3-pyasn1-modules_0.2.1-1_all.deb ... 2023-11-06T02:05:44.653 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2023-11-06T02:05:44.734 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-google-auth. 2023-11-06T02:05:44.762 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../088-python3-google-auth_1.5.1-3_all.deb ... 2023-11-06T02:05:44.801 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2023-11-06T02:05:44.964 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package python3-rsa. 2023-11-06T02:05:44.992 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../087-python3-rsa_4.8-1_all.deb ... 2023-11-06T02:05:45.030 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking python3-rsa (4.8-1) ... 2023-11-06T02:05:45.121 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-requests-oauthlib. 2023-11-06T02:05:45.148 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../089-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2023-11-06T02:05:45.187 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2023-11-06T02:05:45.418 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package python3-google-auth. 2023-11-06T02:05:45.445 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../088-python3-google-auth_1.5.1-3_all.deb ... 2023-11-06T02:05:45.484 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2023-11-06T02:05:45.499 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-websocket. 2023-11-06T02:05:45.526 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../090-python3-websocket_1.2.3-1_all.deb ... 2023-11-06T02:05:45.565 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-websocket (1.2.3-1) ... 2023-11-06T02:05:45.804 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package python3-requests-oauthlib. 2023-11-06T02:05:45.832 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../089-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2023-11-06T02:05:45.843 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-kubernetes. 2023-11-06T02:05:45.871 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2023-11-06T02:05:45.871 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../091-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2023-11-06T02:05:45.902 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2023-11-06T02:05:46.182 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package python3-websocket. 2023-11-06T02:05:46.209 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../090-python3-websocket_1.2.3-1_all.deb ... 2023-11-06T02:05:46.248 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking python3-websocket (1.2.3-1) ... 2023-11-06T02:05:46.560 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package python3-kubernetes. 2023-11-06T02:05:46.587 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../091-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2023-11-06T02:05:46.599 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2023-11-06T02:05:46.615 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../092-ceph-mgr-k8sevents_18.2.0-1181-gf7e9f9af-1jammy_all.deb ... 2023-11-06T02:05:46.626 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2023-11-06T02:05:46.649 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking ceph-mgr-k8sevents (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:05:46.943 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package libonig5:amd64. 2023-11-06T02:05:46.971 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../093-libonig5_6.9.7.1-2build1_amd64.deb ... 2023-11-06T02:05:47.010 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2023-11-06T02:05:47.240 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2023-11-06T02:05:47.257 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../092-ceph-mgr-k8sevents_18.2.0-1181-gf7e9f9af-1jammy_all.deb ... 2023-11-06T02:05:47.321 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package libjq1:amd64. 2023-11-06T02:05:47.323 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking ceph-mgr-k8sevents (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:05:47.348 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../094-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2023-11-06T02:05:47.387 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2023-11-06T02:05:47.592 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package libonig5:amd64. 2023-11-06T02:05:47.609 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../093-libonig5_6.9.7.1-2build1_amd64.deb ... 2023-11-06T02:05:47.642 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2023-11-06T02:05:47.674 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package jq. 2023-11-06T02:05:47.701 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../095-jq_1.6-2.1ubuntu3_amd64.deb ... 2023-11-06T02:05:47.740 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2023-11-06T02:05:47.979 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package libjq1:amd64. 2023-11-06T02:05:47.993 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package socat. 2023-11-06T02:05:48.007 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../094-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2023-11-06T02:05:48.020 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../096-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2023-11-06T02:05:48.045 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2023-11-06T02:05:48.059 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2023-11-06T02:05:48.307 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package jq. 2023-11-06T02:05:48.334 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../095-jq_1.6-2.1ubuntu3_amd64.deb ... 2023-11-06T02:05:48.362 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package xmlstarlet. 2023-11-06T02:05:48.373 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2023-11-06T02:05:48.378 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../097-xmlstarlet_1.6.1-2.1_amd64.deb ... 2023-11-06T02:05:48.420 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2023-11-06T02:05:48.625 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package socat. 2023-11-06T02:05:48.642 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../096-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2023-11-06T02:05:48.684 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2023-11-06T02:05:48.791 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package ceph-test. 2023-11-06T02:05:48.807 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../098-ceph-test_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T02:05:48.850 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking ceph-test (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:05:48.962 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package xmlstarlet. 2023-11-06T02:05:48.978 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../097-xmlstarlet_1.6.1-2.1_amd64.deb ... 2023-11-06T02:05:49.013 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2023-11-06T02:05:49.358 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package ceph-test. 2023-11-06T02:05:49.385 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../098-ceph-test_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T02:05:49.424 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking ceph-test (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:05:51.171 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package ceph-volume. 2023-11-06T02:05:51.198 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../099-ceph-volume_18.2.0-1181-gf7e9f9af-1jammy_all.deb ... 2023-11-06T02:05:51.237 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking ceph-volume (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:05:51.608 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package cephfs-mirror. 2023-11-06T02:05:51.635 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../100-cephfs-mirror_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T02:05:51.674 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking cephfs-mirror (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:05:51.762 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package ceph-volume. 2023-11-06T02:05:51.790 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../099-ceph-volume_18.2.0-1181-gf7e9f9af-1jammy_all.deb ... 2023-11-06T02:05:51.828 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking ceph-volume (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:05:51.944 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-colorama. 2023-11-06T02:05:51.972 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../101-python3-colorama_0.4.4-1_all.deb ... 2023-11-06T02:05:52.002 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-colorama (0.4.4-1) ... 2023-11-06T02:05:52.182 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package cephfs-mirror. 2023-11-06T02:05:52.210 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../100-cephfs-mirror_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T02:05:52.238 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package cephfs-shell. 2023-11-06T02:05:52.248 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking cephfs-mirror (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:05:52.266 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../102-cephfs-shell_18.2.0-1181-gf7e9f9af-1jammy_all.deb ... 2023-11-06T02:05:52.305 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking cephfs-shell (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:05:52.549 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package cephfs-top. 2023-11-06T02:05:52.565 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../103-cephfs-top_18.2.0-1181-gf7e9f9af-1jammy_all.deb ... 2023-11-06T02:05:52.569 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package python3-colorama. 2023-11-06T02:05:52.597 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../101-python3-colorama_0.4.4-1_all.deb ... 2023-11-06T02:05:52.599 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking cephfs-top (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:05:52.638 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking python3-colorama (0.4.4-1) ... 2023-11-06T02:05:52.818 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package libcephfs-dev. 2023-11-06T02:05:52.834 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../104-libcephfs-dev_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T02:05:52.877 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking libcephfs-dev (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:05:52.888 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package cephfs-shell. 2023-11-06T02:05:52.904 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../102-cephfs-shell_18.2.0-1181-gf7e9f9af-1jammy_all.deb ... 2023-11-06T02:05:52.946 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking cephfs-shell (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:05:53.191 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package cephfs-top. 2023-11-06T02:05:53.218 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../103-cephfs-top_18.2.0-1181-gf7e9f9af-1jammy_all.deb ... 2023-11-06T02:05:53.255 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package nvme-cli. 2023-11-06T02:05:53.257 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking cephfs-top (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:05:53.282 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../105-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2023-11-06T02:05:53.321 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2023-11-06T02:05:53.510 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package libcephfs-dev. 2023-11-06T02:05:53.526 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../104-libcephfs-dev_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T02:05:53.568 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking libcephfs-dev (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:05:53.634 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python-asyncssh-doc. 2023-11-06T02:05:53.650 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../106-python-asyncssh-doc_2.5.0-1_all.deb ... 2023-11-06T02:05:53.692 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2023-11-06T02:05:53.946 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package nvme-cli. 2023-11-06T02:05:53.974 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../105-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2023-11-06T02:05:54.012 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2023-11-06T02:05:54.037 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-pastescript. 2023-11-06T02:05:54.053 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../107-python3-pastescript_2.0.2-4_all.deb ... 2023-11-06T02:05:54.095 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2023-11-06T02:05:54.351 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package python-asyncssh-doc. 2023-11-06T02:05:54.367 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../106-python-asyncssh-doc_2.5.0-1_all.deb ... 2023-11-06T02:05:54.407 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-psutil. 2023-11-06T02:05:54.408 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2023-11-06T02:05:54.434 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../108-python3-psutil_5.9.0-1build1_amd64.deb ... 2023-11-06T02:05:54.482 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2023-11-06T02:05:54.785 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-pygments. 2023-11-06T02:05:54.812 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../109-python3-pygments_2.11.2+dfsg-2_all.deb ... 2023-11-06T02:05:54.829 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package python3-pastescript. 2023-11-06T02:05:54.851 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2023-11-06T02:05:54.857 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../107-python3-pastescript_2.0.2-4_all.deb ... 2023-11-06T02:05:54.895 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2023-11-06T02:05:55.224 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package python3-psutil. 2023-11-06T02:05:55.252 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../108-python3-psutil_5.9.0-1build1_amd64.deb ... 2023-11-06T02:05:55.264 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-pyinotify. 2023-11-06T02:05:55.290 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2023-11-06T02:05:55.291 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../110-python3-pyinotify_0.9.6-1.3_all.deb ... 2023-11-06T02:05:55.338 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2023-11-06T02:05:55.635 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package python3-pygments. 2023-11-06T02:05:55.650 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-simplejson. 2023-11-06T02:05:55.663 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../109-python3-pygments_2.11.2+dfsg-2_all.deb ... 2023-11-06T02:05:55.678 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../111-python3-simplejson_3.17.6-1build1_amd64.deb ... 2023-11-06T02:05:55.701 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2023-11-06T02:05:55.716 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2023-11-06T02:05:56.027 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package qttranslations5-l10n. 2023-11-06T02:05:56.055 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../112-qttranslations5-l10n_5.15.3-1_all.deb ... 2023-11-06T02:05:56.094 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2023-11-06T02:05:56.105 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package python3-pyinotify. 2023-11-06T02:05:56.133 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../110-python3-pyinotify_0.9.6-1.3_all.deb ... 2023-11-06T02:05:56.171 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2023-11-06T02:05:56.457 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package python3-simplejson. 2023-11-06T02:05:56.486 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../111-python3-simplejson_3.17.6-1build1_amd64.deb ... 2023-11-06T02:05:56.524 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2023-11-06T02:05:56.776 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package qttranslations5-l10n. 2023-11-06T02:05:56.781 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package radosgw. 2023-11-06T02:05:56.793 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../112-qttranslations5-l10n_5.15.3-1_all.deb ... 2023-11-06T02:05:56.797 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../113-radosgw_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T02:05:56.830 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking radosgw (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:05:56.835 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2023-11-06T02:05:57.500 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package rbd-fuse. 2023-11-06T02:05:57.516 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../114-rbd-fuse_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T02:05:57.547 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package radosgw. 2023-11-06T02:05:57.558 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking rbd-fuse (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:05:57.576 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../113-radosgw_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T02:05:57.613 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking radosgw (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:05:57.862 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2023-11-06T02:05:58.113 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2023-11-06T02:05:58.239 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2023-11-06T02:05:58.375 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package rbd-fuse. 2023-11-06T02:05:58.392 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../114-rbd-fuse_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T02:05:58.432 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking rbd-fuse (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:05:58.484 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2023-11-06T02:05:58.729 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2023-11-06T02:05:58.842 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2023-11-06T02:05:59.063 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2023-11-06T02:05:59.163 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2023-11-06T02:05:59.306 INFO:teuthology.orchestra.run.smithi150.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2023-11-06T02:05:59.408 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up cephadm (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:05:59.437 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2023-11-06T02:05:59.798 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2023-11-06T02:05:59.950 INFO:teuthology.orchestra.run.smithi150.stdout:Adding system user cephadm....done 2023-11-06T02:06:00.105 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2023-11-06T02:06:00.275 INFO:teuthology.orchestra.run.smithi049.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2023-11-06T02:06:00.349 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up cephadm (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:06:00.373 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2023-11-06T02:06:00.624 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2023-11-06T02:06:00.726 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2023-11-06T02:06:00.977 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-repoze.lru (0.7-2) ... 2023-11-06T02:06:01.061 INFO:teuthology.orchestra.run.smithi049.stdout:Adding system user cephadm....done 2023-11-06T02:06:01.211 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2023-11-06T02:06:01.231 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2023-11-06T02:06:01.337 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-py (1.10.0-1) ... 2023-11-06T02:06:01.515 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2023-11-06T02:06:01.630 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2023-11-06T02:06:01.791 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2023-11-06T02:06:01.893 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2023-11-06T02:06:02.001 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-cachetools (5.0.0-1) ... 2023-11-06T02:06:02.144 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up python3-repoze.lru (0.7-2) ... 2023-11-06T02:06:02.260 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-colorama (0.4.4-1) ... 2023-11-06T02:06:02.403 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2023-11-06T02:06:02.512 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-openssl (21.0.0-1) ... 2023-11-06T02:06:02.529 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up python3-py (1.10.0-1) ... 2023-11-06T02:06:02.788 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2023-11-06T02:06:02.847 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2023-11-06T02:06:03.039 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2023-11-06T02:06:03.224 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up python3-cachetools (5.0.0-1) ... 2023-11-06T02:06:03.298 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2023-11-06T02:06:03.475 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up python3-colorama (0.4.4-1) ... 2023-11-06T02:06:03.533 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-ceph-argparse (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:06:03.718 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up python3-openssl (21.0.0-1) ... 2023-11-06T02:06:03.801 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2023-11-06T02:06:03.927 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2023-11-06T02:06:03.994 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2023-11-06T02:06:04.178 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2023-11-06T02:06:04.254 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2023-11-06T02:06:04.420 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2023-11-06T02:06:04.513 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2023-11-06T02:06:04.546 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2023-11-06T02:06:04.814 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up python3-ceph-argparse (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:06:04.914 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2023-11-06T02:06:05.040 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-natsort (8.0.2-1) ... 2023-11-06T02:06:05.099 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2023-11-06T02:06:05.250 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2023-11-06T02:06:05.324 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2023-11-06T02:06:05.518 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2023-11-06T02:06:05.584 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2023-11-06T02:06:05.777 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2023-11-06T02:06:05.868 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2023-11-06T02:06:05.903 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2023-11-06T02:06:06.279 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2023-11-06T02:06:06.388 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up python3-natsort (8.0.2-1) ... 2023-11-06T02:06:06.520 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2023-11-06T02:06:06.656 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2023-11-06T02:06:06.772 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2023-11-06T02:06:06.898 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2023-11-06T02:06:06.932 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2023-11-06T02:06:07.015 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2023-11-06T02:06:07.242 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2023-11-06T02:06:07.283 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2023-11-06T02:06:07.659 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-certifi (2020.6.20-1) ... 2023-11-06T02:06:07.910 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2023-11-06T02:06:07.919 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2023-11-06T02:06:08.178 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2023-11-06T02:06:08.287 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2023-11-06T02:06:08.288 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2023-11-06T02:06:08.405 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2023-11-06T02:06:08.580 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2023-11-06T02:06:08.698 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2023-11-06T02:06:08.881 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2023-11-06T02:06:09.124 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up python3-certifi (2020.6.20-1) ... 2023-11-06T02:06:09.224 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2023-11-06T02:06:09.375 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2023-11-06T02:06:09.475 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2023-11-06T02:06:09.601 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-ceph-common (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:06:09.777 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2023-11-06T02:06:09.885 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-idna (3.3-1) ... 2023-11-06T02:06:10.078 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2023-11-06T02:06:10.203 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2023-11-06T02:06:10.396 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2023-11-06T02:06:10.747 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2023-11-06T02:06:11.024 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2023-11-06T02:06:11.133 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up python3-ceph-common (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:06:11.342 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-11-06T02:06:11.417 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up python3-idna (3.3-1) ... 2023-11-06T02:06:11.452 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-toml (0.10.2-1) ... 2023-11-06T02:06:11.703 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2023-11-06T02:06:11.753 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2023-11-06T02:06:11.829 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2023-11-06T02:06:11.955 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-urllib3 (1.26.5-1~exp1) ... 2023-11-06T02:06:12.264 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2023-11-06T02:06:12.524 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2023-11-06T02:06:12.766 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-11-06T02:06:12.884 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up libfmt8:amd64 (8.1.1+ds1-2) ... 2023-11-06T02:06:12.941 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-11-06T02:06:13.010 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2023-11-06T02:06:13.075 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up python3-toml (0.10.2-1) ... 2023-11-06T02:06:13.328 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2023-11-06T02:06:13.345 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2023-11-06T02:06:13.446 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2023-11-06T02:06:13.572 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up python3-urllib3 (1.26.5-1~exp1) ... 2023-11-06T02:06:13.587 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-logutils (0.3.3-8) ... 2023-11-06T02:06:13.864 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-tempora (4.1.2-1) ... 2023-11-06T02:06:13.882 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2023-11-06T02:06:14.116 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2023-11-06T02:06:14.139 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2023-11-06T02:06:14.359 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-11-06T02:06:14.390 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-prettytable (2.5.0-2) ... 2023-11-06T02:06:14.485 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up libfmt8:amd64 (8.1.1+ds1-2) ... 2023-11-06T02:06:14.603 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2023-11-06T02:06:14.641 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2023-11-06T02:06:14.751 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-websocket (1.2.3-1) ... 2023-11-06T02:06:14.913 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2023-11-06T02:06:15.011 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2023-11-06T02:06:15.137 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2023-11-06T02:06:15.164 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up python3-logutils (0.3.3-8) ... 2023-11-06T02:06:15.432 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up python3-tempora (4.1.2-1) ... 2023-11-06T02:06:15.437 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2023-11-06T02:06:15.675 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2023-11-06T02:06:15.739 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2023-11-06T02:06:15.957 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up python3-prettytable (2.5.0-2) ... 2023-11-06T02:06:15.998 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2023-11-06T02:06:16.227 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2023-11-06T02:06:16.344 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up python3-websocket (1.2.3-1) ... 2023-11-06T02:06:16.411 INFO:teuthology.orchestra.run.smithi150.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2023-11-06T02:06:16.621 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2023-11-06T02:06:16.668 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2023-11-06T02:06:16.747 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2023-11-06T02:06:16.925 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2023-11-06T02:06:17.048 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2023-11-06T02:06:17.145 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2023-11-06T02:06:17.349 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2023-11-06T02:06:17.425 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-portend (3.0.0-1) ... 2023-11-06T02:06:17.600 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2023-11-06T02:06:17.690 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-11-06T02:06:17.815 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-jinja2 (3.0.3-1) ... 2023-11-06T02:06:18.045 INFO:teuthology.orchestra.run.smithi049.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2023-11-06T02:06:18.132 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2023-11-06T02:06:18.320 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2023-11-06T02:06:18.417 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2023-11-06T02:06:18.446 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2023-11-06T02:06:18.669 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2023-11-06T02:06:18.794 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-rsa (4.8-1) ... 2023-11-06T02:06:18.847 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2023-11-06T02:06:19.070 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-webtest (2.0.35-1) ... 2023-11-06T02:06:19.115 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up python3-portend (3.0.0-1) ... 2023-11-06T02:06:19.346 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2023-11-06T02:06:19.375 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-11-06T02:06:19.501 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up python3-jinja2 (3.0.3-1) ... 2023-11-06T02:06:19.606 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2023-11-06T02:06:19.827 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2023-11-06T02:06:20.007 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-pastescript (2.0.2-4) ... 2023-11-06T02:06:20.128 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2023-11-06T02:06:20.292 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2023-11-06T02:06:20.404 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2023-11-06T02:06:20.531 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up python3-rsa (4.8-1) ... 2023-11-06T02:06:20.644 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2023-11-06T02:06:20.770 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up librados2 (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:06:20.823 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up python3-webtest (2.0.35-1) ... 2023-11-06T02:06:20.896 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up librgw2 (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:06:21.099 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2023-11-06T02:06:21.107 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up libsqlite3-mod-ceph (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:06:21.234 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up libcephfs2 (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:06:21.335 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up libradosstriper1 (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:06:21.359 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2023-11-06T02:06:21.436 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-google-auth (1.5.1-3) ... 2023-11-06T02:06:21.704 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up librbd1 (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:06:21.935 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up python3-pastescript (2.0.2-4) ... 2023-11-06T02:06:22.190 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up ceph-mgr-modules-core (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:06:22.220 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2023-11-06T02:06:22.315 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up ceph-fuse (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:06:22.516 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-11-06T02:06:22.516 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-11-06T02:06:22.563 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2023-11-06T02:06:22.690 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up librados2 (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:06:22.816 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up librgw2 (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:06:23.010 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up libsqlite3-mod-ceph (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:06:23.016 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up libcephfs-dev (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:06:23.117 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-rados (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:06:23.145 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up libcephfs2 (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:06:23.235 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up cephfs-top (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:06:23.279 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up libradosstriper1 (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:06:23.361 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2023-11-06T02:06:23.405 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up python3-google-auth (1.5.1-3) ... 2023-11-06T02:06:23.682 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up librbd1 (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:06:23.808 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up ceph-mgr-modules-core (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:06:23.934 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up ceph-fuse (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:06:24.150 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-11-06T02:06:24.150 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-11-06T02:06:24.498 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-rbd (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:06:24.641 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up rbd-fuse (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:06:24.660 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up libcephfs-dev (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:06:24.759 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-rgw (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:06:24.787 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up python3-rados (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:06:24.860 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-cephfs (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:06:24.913 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up cephfs-top (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:06:24.961 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up cephfs-shell (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:06:25.039 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2023-11-06T02:06:25.100 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up ceph-common (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:06:25.647 INFO:teuthology.orchestra.run.smithi150.stdout:Adding group ceph....done 2023-11-06T02:06:26.040 INFO:teuthology.orchestra.run.smithi150.stdout:Adding system user ceph....done 2023-11-06T02:06:26.200 INFO:teuthology.orchestra.run.smithi150.stdout:Setting system user ceph properties....done 2023-11-06T02:06:26.216 INFO:teuthology.orchestra.run.smithi150.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2023-11-06T02:06:26.251 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up python3-rbd (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:06:26.347 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2023-11-06T02:06:26.361 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up rbd-fuse (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:06:26.479 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up python3-rgw (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:06:26.605 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up python3-cephfs (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:06:26.657 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2023-11-06T02:06:26.756 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up cephfs-shell (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:06:26.882 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up ceph-common (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:06:27.306 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up cephfs-mirror (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:06:27.450 INFO:teuthology.orchestra.run.smithi049.stdout:Adding group ceph....done 2023-11-06T02:06:27.495 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-11-06T02:06:27.495 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-11-06T02:06:27.826 INFO:teuthology.orchestra.run.smithi049.stdout:Adding system user ceph....done 2023-11-06T02:06:27.995 INFO:teuthology.orchestra.run.smithi049.stdout:Setting system user ceph properties....done 2023-11-06T02:06:28.012 INFO:teuthology.orchestra.run.smithi049.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2023-11-06T02:06:28.015 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up ceph-test (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:06:28.133 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up radosgw (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:06:28.153 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2023-11-06T02:06:28.494 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2023-11-06T02:06:28.607 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-11-06T02:06:28.608 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-11-06T02:06:29.067 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up cephfs-mirror (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:06:29.102 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up ceph-base (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:06:29.294 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-11-06T02:06:29.294 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-11-06T02:06:29.438 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2023-11-06T02:06:29.785 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up ceph-test (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:06:29.886 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up radosgw (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:06:29.937 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up ceph-mds (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:06:30.159 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-11-06T02:06:30.160 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-11-06T02:06:30.392 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-11-06T02:06:30.392 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-11-06T02:06:30.672 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up ceph-mgr (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:06:30.888 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up ceph-base (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:06:30.900 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-11-06T02:06:30.901 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-11-06T02:06:31.235 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2023-11-06T02:06:31.423 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up ceph-osd (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:06:31.730 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-11-06T02:06:31.730 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-11-06T02:06:31.747 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up ceph-mds (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:06:31.973 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-11-06T02:06:31.973 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-11-06T02:06:32.299 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up ceph-mgr-k8sevents (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:06:32.425 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up ceph-mgr-diskprediction-local (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:06:32.475 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up ceph-mgr (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:06:32.702 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-11-06T02:06:32.703 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-11-06T02:06:32.780 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up ceph-mon (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:06:32.983 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-11-06T02:06:32.983 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-11-06T02:06:33.226 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up ceph-osd (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:06:33.514 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up ceph-mgr-cephadm (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:06:33.550 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-11-06T02:06:33.551 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-11-06T02:06:33.632 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up ceph (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:06:33.750 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up ceph-mgr-dashboard (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:06:33.901 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up ceph-volume (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:06:34.119 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up ceph-mgr-k8sevents (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:06:34.245 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up ceph-mgr-diskprediction-local (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:06:34.253 INFO:teuthology.orchestra.run.smithi150.stdout:Processing triggers for man-db (2.10.2-1) ... 2023-11-06T02:06:34.396 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up ceph-mon (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:06:34.606 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-11-06T02:06:34.606 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-11-06T02:06:35.122 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up ceph-mgr-cephadm (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:06:35.248 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up ceph (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:06:35.371 INFO:teuthology.orchestra.run.smithi150.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2023-11-06T02:06:35.374 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up ceph-mgr-dashboard (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:06:35.526 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up ceph-volume (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T02:06:35.837 INFO:teuthology.orchestra.run.smithi150.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-11-06T02:06:35.840 DEBUG:teuthology.orchestra.run.smithi150:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2023-11-06T02:06:35.887 INFO:teuthology.orchestra.run.smithi049.stdout:Processing triggers for man-db (2.10.2-1) ... 2023-11-06T02:06:35.936 INFO:teuthology.orchestra.run.smithi150.stdout:Reading package lists... 2023-11-06T02:06:36.123 INFO:teuthology.orchestra.run.smithi150.stdout:Building dependency tree... 2023-11-06T02:06:36.124 INFO:teuthology.orchestra.run.smithi150.stdout:Reading state information... 2023-11-06T02:06:36.280 INFO:teuthology.orchestra.run.smithi150.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2023-11-06T02:06:36.280 INFO:teuthology.orchestra.run.smithi150.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2023-11-06T02:06:36.280 INFO:teuthology.orchestra.run.smithi150.stdout:util-linux is already the newest version (2.37.2-4ubuntu3). 2023-11-06T02:06:36.280 INFO:teuthology.orchestra.run.smithi150.stdout:uuid-dev is already the newest version (2.37.2-4ubuntu3). 2023-11-06T02:06:36.281 INFO:teuthology.orchestra.run.smithi150.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2023-11-06T02:06:36.281 INFO:teuthology.orchestra.run.smithi150.stdout:The following packages were automatically installed and are no longer required: 2023-11-06T02:06:36.281 INFO:teuthology.orchestra.run.smithi150.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-11-06T02:06:36.281 INFO:teuthology.orchestra.run.smithi150.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-11-06T02:06:36.281 INFO:teuthology.orchestra.run.smithi150.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-11-06T02:06:36.282 INFO:teuthology.orchestra.run.smithi150.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2023-11-06T02:06:36.282 INFO:teuthology.orchestra.run.smithi150.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2023-11-06T02:06:36.282 INFO:teuthology.orchestra.run.smithi150.stdout:Use 'sudo apt autoremove' to remove them. 2023-11-06T02:06:36.283 INFO:teuthology.orchestra.run.smithi150.stdout:The following additional packages will be installed: 2023-11-06T02:06:36.283 INFO:teuthology.orchestra.run.smithi150.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2023-11-06T02:06:36.283 INFO:teuthology.orchestra.run.smithi150.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2023-11-06T02:06:36.284 INFO:teuthology.orchestra.run.smithi150.stdout: libqmi-proxy libssl3 libteamdctl0 modemmanager network-manager-pptp 2023-11-06T02:06:36.284 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2023-11-06T02:06:36.284 INFO:teuthology.orchestra.run.smithi150.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2023-11-06T02:06:36.284 INFO:teuthology.orchestra.run.smithi150.stdout:Suggested packages: 2023-11-06T02:06:36.284 INFO:teuthology.orchestra.run.smithi150.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2023-11-06T02:06:36.285 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial wpagui 2023-11-06T02:06:36.285 INFO:teuthology.orchestra.run.smithi150.stdout: libengine-pkcs11-openssl 2023-11-06T02:06:36.319 INFO:teuthology.orchestra.run.smithi150.stdout:The following NEW packages will be installed: 2023-11-06T02:06:36.320 INFO:teuthology.orchestra.run.smithi150.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2023-11-06T02:06:36.320 INFO:teuthology.orchestra.run.smithi150.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2023-11-06T02:06:36.320 INFO:teuthology.orchestra.run.smithi150.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2023-11-06T02:06:36.321 INFO:teuthology.orchestra.run.smithi150.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2023-11-06T02:06:36.321 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2023-11-06T02:06:36.321 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2023-11-06T02:06:36.322 INFO:teuthology.orchestra.run.smithi150.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2023-11-06T02:06:36.322 INFO:teuthology.orchestra.run.smithi150.stdout:The following packages will be upgraded: 2023-11-06T02:06:36.322 INFO:teuthology.orchestra.run.smithi150.stdout: libssl3 2023-11-06T02:06:36.395 INFO:teuthology.orchestra.run.smithi150.stdout:1 upgraded, 39 newly installed, 0 to remove and 257 not upgraded. 2023-11-06T02:06:36.396 INFO:teuthology.orchestra.run.smithi150.stdout:Need to get 54.6 MB of archives. 2023-11-06T02:06:36.396 INFO:teuthology.orchestra.run.smithi150.stdout:After this operation, 202 MB of additional disk space will be used. 2023-11-06T02:06:36.396 INFO:teuthology.orchestra.run.smithi150.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2023-11-06T02:06:36.494 INFO:teuthology.orchestra.run.smithi150.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.12 [1,901 kB] 2023-11-06T02:06:36.554 INFO:teuthology.orchestra.run.smithi150.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2023-11-06T02:06:36.569 INFO:teuthology.orchestra.run.smithi150.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2023-11-06T02:06:36.570 INFO:teuthology.orchestra.run.smithi150.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2023-11-06T02:06:36.573 INFO:teuthology.orchestra.run.smithi150.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2023-11-06T02:06:36.574 INFO:teuthology.orchestra.run.smithi150.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2023-11-06T02:06:36.575 INFO:teuthology.orchestra.run.smithi150.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/main amd64 libbluetooth3 amd64 5.64-0ubuntu1 [86.9 kB] 2023-11-06T02:06:36.576 INFO:teuthology.orchestra.run.smithi150.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2023-11-06T02:06:36.578 INFO:teuthology.orchestra.run.smithi150.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2023-11-06T02:06:36.578 INFO:teuthology.orchestra.run.smithi150.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2023-11-06T02:06:36.579 INFO:teuthology.orchestra.run.smithi150.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2023-11-06T02:06:36.592 INFO:teuthology.orchestra.run.smithi150.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2023-11-06T02:06:36.922 INFO:teuthology.orchestra.run.smithi150.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2023-11-06T02:06:36.924 INFO:teuthology.orchestra.run.smithi150.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2023-11-06T02:06:36.924 INFO:teuthology.orchestra.run.smithi150.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.2 [263 kB] 2023-11-06T02:06:36.927 INFO:teuthology.orchestra.run.smithi150.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2023-11-06T02:06:36.927 INFO:teuthology.orchestra.run.smithi150.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2023-11-06T02:06:36.931 INFO:teuthology.orchestra.run.smithi150.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.9-0ubuntu0.22.04.1 [141 kB] 2023-11-06T02:06:36.933 INFO:teuthology.orchestra.run.smithi150.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2023-11-06T02:06:36.940 INFO:teuthology.orchestra.run.smithi150.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2023-11-06T02:06:36.940 INFO:teuthology.orchestra.run.smithi150.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.12 [2,373 kB] 2023-11-06T02:06:36.961 INFO:teuthology.orchestra.run.smithi150.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2023-11-06T02:06:36.961 INFO:teuthology.orchestra.run.smithi150.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.2 [1,094 kB] 2023-11-06T02:06:36.971 INFO:teuthology.orchestra.run.smithi150.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2023-11-06T02:06:36.990 INFO:teuthology.orchestra.run.smithi150.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2023-11-06T02:06:36.993 INFO:teuthology.orchestra.run.smithi150.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2023-11-06T02:06:36.994 INFO:teuthology.orchestra.run.smithi150.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2023-11-06T02:06:36.995 INFO:teuthology.orchestra.run.smithi150.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2023-11-06T02:06:36.995 INFO:teuthology.orchestra.run.smithi150.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.9-0ubuntu0.22.04.1 [1,222 kB] 2023-11-06T02:06:36.997 INFO:teuthology.orchestra.run.smithi049.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2023-11-06T02:06:37.005 INFO:teuthology.orchestra.run.smithi150.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2023-11-06T02:06:37.006 INFO:teuthology.orchestra.run.smithi150.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.9-0ubuntu0.22.04.1 [16.1 MB] 2023-11-06T02:06:37.198 INFO:teuthology.orchestra.run.smithi150.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2023-11-06T02:06:37.199 INFO:teuthology.orchestra.run.smithi150.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2023-11-06T02:06:37.199 INFO:teuthology.orchestra.run.smithi150.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2023-11-06T02:06:37.200 INFO:teuthology.orchestra.run.smithi150.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2023-11-06T02:06:37.200 INFO:teuthology.orchestra.run.smithi150.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2023-11-06T02:06:37.200 INFO:teuthology.orchestra.run.smithi150.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2023-11-06T02:06:37.210 INFO:teuthology.orchestra.run.smithi150.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2023-11-06T02:06:37.211 INFO:teuthology.orchestra.run.smithi150.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2023-11-06T02:06:37.511 INFO:teuthology.orchestra.run.smithi049.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-11-06T02:06:37.514 DEBUG:teuthology.orchestra.run.smithi049:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2023-11-06T02:06:37.577 INFO:teuthology.orchestra.run.smithi049.stdout:Reading package lists... 2023-11-06T02:06:37.610 INFO:teuthology.orchestra.run.smithi150.stdout:Preconfiguring packages ... 2023-11-06T02:06:37.769 INFO:teuthology.orchestra.run.smithi049.stdout:Building dependency tree... 2023-11-06T02:06:37.770 INFO:teuthology.orchestra.run.smithi049.stdout:Reading state information... 2023-11-06T02:06:37.788 INFO:teuthology.orchestra.run.smithi150.stdout:Fetched 54.6 MB in 1s (61.9 MB/s) 2023-11-06T02:06:37.940 INFO:teuthology.orchestra.run.smithi049.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2023-11-06T02:06:37.941 INFO:teuthology.orchestra.run.smithi049.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2023-11-06T02:06:37.941 INFO:teuthology.orchestra.run.smithi049.stdout:util-linux is already the newest version (2.37.2-4ubuntu3). 2023-11-06T02:06:37.941 INFO:teuthology.orchestra.run.smithi049.stdout:uuid-dev is already the newest version (2.37.2-4ubuntu3). 2023-11-06T02:06:37.941 INFO:teuthology.orchestra.run.smithi049.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2023-11-06T02:06:37.941 INFO:teuthology.orchestra.run.smithi049.stdout:The following packages were automatically installed and are no longer required: 2023-11-06T02:06:37.942 INFO:teuthology.orchestra.run.smithi049.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-11-06T02:06:37.942 INFO:teuthology.orchestra.run.smithi049.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-11-06T02:06:37.942 INFO:teuthology.orchestra.run.smithi049.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-11-06T02:06:37.942 INFO:teuthology.orchestra.run.smithi049.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2023-11-06T02:06:37.943 INFO:teuthology.orchestra.run.smithi049.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2023-11-06T02:06:37.944 INFO:teuthology.orchestra.run.smithi049.stdout:Use 'sudo apt autoremove' to remove them. 2023-11-06T02:06:37.944 INFO:teuthology.orchestra.run.smithi049.stdout:The following additional packages will be installed: 2023-11-06T02:06:37.944 INFO:teuthology.orchestra.run.smithi049.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2023-11-06T02:06:37.944 INFO:teuthology.orchestra.run.smithi049.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2023-11-06T02:06:37.944 INFO:teuthology.orchestra.run.smithi049.stdout: libqmi-proxy libssl3 libteamdctl0 modemmanager network-manager-pptp 2023-11-06T02:06:37.945 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2023-11-06T02:06:37.945 INFO:teuthology.orchestra.run.smithi049.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2023-11-06T02:06:37.945 INFO:teuthology.orchestra.run.smithi049.stdout:Suggested packages: 2023-11-06T02:06:37.945 INFO:teuthology.orchestra.run.smithi049.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2023-11-06T02:06:37.946 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial wpagui 2023-11-06T02:06:37.946 INFO:teuthology.orchestra.run.smithi049.stdout: libengine-pkcs11-openssl 2023-11-06T02:06:37.981 INFO:teuthology.orchestra.run.smithi049.stdout:The following NEW packages will be installed: 2023-11-06T02:06:37.982 INFO:teuthology.orchestra.run.smithi049.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2023-11-06T02:06:37.983 INFO:teuthology.orchestra.run.smithi049.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2023-11-06T02:06:37.983 INFO:teuthology.orchestra.run.smithi049.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2023-11-06T02:06:37.983 INFO:teuthology.orchestra.run.smithi049.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2023-11-06T02:06:37.983 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2023-11-06T02:06:37.984 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2023-11-06T02:06:37.984 INFO:teuthology.orchestra.run.smithi049.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2023-11-06T02:06:37.984 INFO:teuthology.orchestra.run.smithi049.stdout:The following packages will be upgraded: 2023-11-06T02:06:37.985 INFO:teuthology.orchestra.run.smithi049.stdout: libssl3 2023-11-06T02:06:38.074 INFO:teuthology.orchestra.run.smithi049.stdout:1 upgraded, 39 newly installed, 0 to remove and 257 not upgraded. 2023-11-06T02:06:38.075 INFO:teuthology.orchestra.run.smithi049.stdout:Need to get 54.6 MB of archives. 2023-11-06T02:06:38.075 INFO:teuthology.orchestra.run.smithi049.stdout:After this operation, 202 MB of additional disk space will be used. 2023-11-06T02:06:38.076 INFO:teuthology.orchestra.run.smithi049.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2023-11-06T02:06:38.209 INFO:teuthology.orchestra.run.smithi049.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.12 [1,901 kB] 2023-11-06T02:06:38.231 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package flex. 2023-11-06T02:06:38.280 INFO:teuthology.orchestra.run.smithi049.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2023-11-06T02:06:38.297 INFO:teuthology.orchestra.run.smithi049.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2023-11-06T02:06:38.298 INFO:teuthology.orchestra.run.smithi049.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2023-11-06T02:06:38.302 INFO:teuthology.orchestra.run.smithi049.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2023-11-06T02:06:38.303 INFO:teuthology.orchestra.run.smithi049.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2023-11-06T02:06:38.304 INFO:teuthology.orchestra.run.smithi049.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/main amd64 libbluetooth3 amd64 5.64-0ubuntu1 [86.9 kB] 2023-11-06T02:06:38.306 INFO:teuthology.orchestra.run.smithi049.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2023-11-06T02:06:38.308 INFO:teuthology.orchestra.run.smithi049.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2023-11-06T02:06:38.308 INFO:teuthology.orchestra.run.smithi049.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2023-11-06T02:06:38.309 INFO:teuthology.orchestra.run.smithi049.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2023-11-06T02:06:38.326 INFO:teuthology.orchestra.run.smithi049.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2023-11-06T02:06:38.338 INFO:teuthology.orchestra.run.smithi150.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 172902 files and directories currently installed.) 2023-11-06T02:06:38.344 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2023-11-06T02:06:38.437 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking flex (2.6.4-8build2) ... 2023-11-06T02:06:38.818 INFO:teuthology.orchestra.run.smithi049.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2023-11-06T02:06:38.821 INFO:teuthology.orchestra.run.smithi049.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2023-11-06T02:06:38.821 INFO:teuthology.orchestra.run.smithi049.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.2 [263 kB] 2023-11-06T02:06:38.825 INFO:teuthology.orchestra.run.smithi049.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2023-11-06T02:06:38.826 INFO:teuthology.orchestra.run.smithi049.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2023-11-06T02:06:38.833 INFO:teuthology.orchestra.run.smithi049.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.9-0ubuntu0.22.04.1 [141 kB] 2023-11-06T02:06:38.836 INFO:teuthology.orchestra.run.smithi049.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2023-11-06T02:06:38.848 INFO:teuthology.orchestra.run.smithi049.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2023-11-06T02:06:38.849 INFO:teuthology.orchestra.run.smithi049.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.12 [2,373 kB] 2023-11-06T02:06:38.882 INFO:teuthology.orchestra.run.smithi049.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2023-11-06T02:06:38.882 INFO:teuthology.orchestra.run.smithi049.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.2 [1,094 kB] 2023-11-06T02:06:38.893 INFO:teuthology.orchestra.run.smithi049.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2023-11-06T02:06:38.914 INFO:teuthology.orchestra.run.smithi049.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2023-11-06T02:06:38.917 INFO:teuthology.orchestra.run.smithi049.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2023-11-06T02:06:38.917 INFO:teuthology.orchestra.run.smithi049.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2023-11-06T02:06:38.918 INFO:teuthology.orchestra.run.smithi049.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2023-11-06T02:06:38.919 INFO:teuthology.orchestra.run.smithi049.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.9-0ubuntu0.22.04.1 [1,222 kB] 2023-11-06T02:06:38.928 INFO:teuthology.orchestra.run.smithi049.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2023-11-06T02:06:38.930 INFO:teuthology.orchestra.run.smithi049.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.9-0ubuntu0.22.04.1 [16.1 MB] 2023-11-06T02:06:39.012 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.12_amd64.deb ... 2023-11-06T02:06:39.184 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.12) over (3.0.2-0ubuntu1.7) ... 2023-11-06T02:06:39.236 INFO:teuthology.orchestra.run.smithi049.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2023-11-06T02:06:39.236 INFO:teuthology.orchestra.run.smithi049.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2023-11-06T02:06:39.237 INFO:teuthology.orchestra.run.smithi049.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2023-11-06T02:06:39.237 INFO:teuthology.orchestra.run.smithi049.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2023-11-06T02:06:39.238 INFO:teuthology.orchestra.run.smithi049.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2023-11-06T02:06:39.238 INFO:teuthology.orchestra.run.smithi049.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2023-11-06T02:06:39.258 INFO:teuthology.orchestra.run.smithi049.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2023-11-06T02:06:39.258 INFO:teuthology.orchestra.run.smithi049.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2023-11-06T02:06:39.569 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.12) ... 2023-11-06T02:06:39.680 INFO:teuthology.orchestra.run.smithi049.stdout:Preconfiguring packages ... 2023-11-06T02:06:39.813 INFO:teuthology.orchestra.run.smithi049.stdout:Fetched 54.6 MB in 1s (43.1 MB/s) 2023-11-06T02:06:39.954 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package flex. 2023-11-06T02:06:40.058 INFO:teuthology.orchestra.run.smithi049.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 172902 files and directories currently installed.) 2023-11-06T02:06:40.064 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2023-11-06T02:06:40.094 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package bison. 2023-11-06T02:06:40.102 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking flex (2.6.4-8build2) ... 2023-11-06T02:06:40.197 INFO:teuthology.orchestra.run.smithi150.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 172975 files and directories currently installed.) 2023-11-06T02:06:40.203 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2023-11-06T02:06:40.263 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2023-11-06T02:06:40.643 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.12_amd64.deb ... 2023-11-06T02:06:40.688 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package liblzo2-2:amd64. 2023-11-06T02:06:40.716 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2023-11-06T02:06:40.755 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2023-11-06T02:06:40.816 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.12) over (3.0.2-0ubuntu1.7) ... 2023-11-06T02:06:41.099 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package dump. 2023-11-06T02:06:41.127 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2023-11-06T02:06:41.166 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking dump (0.4b47-3) ... 2023-11-06T02:06:41.234 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.12) ... 2023-11-06T02:06:41.461 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package indent. 2023-11-06T02:06:41.477 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2023-11-06T02:06:41.511 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2023-11-06T02:06:41.722 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package libaio-dev:amd64. 2023-11-06T02:06:41.738 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2023-11-06T02:06:41.742 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package bison. 2023-11-06T02:06:41.780 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2023-11-06T02:06:41.845 INFO:teuthology.orchestra.run.smithi049.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 172975 files and directories currently installed.) 2023-11-06T02:06:41.851 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2023-11-06T02:06:41.903 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2023-11-06T02:06:42.134 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package libbluetooth3:amd64. 2023-11-06T02:06:42.161 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1_amd64.deb ... 2023-11-06T02:06:42.200 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1) ... 2023-11-06T02:06:42.287 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package liblzo2-2:amd64. 2023-11-06T02:06:42.321 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2023-11-06T02:06:42.438 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2023-11-06T02:06:42.470 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package libelf-dev:amd64. 2023-11-06T02:06:42.486 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2023-11-06T02:06:42.528 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2023-11-06T02:06:42.758 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package dump. 2023-11-06T02:06:42.786 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2023-11-06T02:06:42.814 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package libfl2:amd64. 2023-11-06T02:06:42.824 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking dump (0.4b47-3) ... 2023-11-06T02:06:42.842 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2023-11-06T02:06:42.872 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2023-11-06T02:06:43.103 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package indent. 2023-11-06T02:06:43.120 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2023-11-06T02:06:43.142 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package libfl-dev:amd64. 2023-11-06T02:06:43.169 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2023-11-06T02:06:43.170 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2023-11-06T02:06:43.208 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2023-11-06T02:06:43.422 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package libaio-dev:amd64. 2023-11-06T02:06:43.450 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2023-11-06T02:06:43.469 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package libjson-perl. 2023-11-06T02:06:43.486 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2023-11-06T02:06:43.489 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2023-11-06T02:06:43.528 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking libjson-perl (4.04000-1) ... 2023-11-06T02:06:43.792 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package libbluetooth3:amd64. 2023-11-06T02:06:43.821 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1_amd64.deb ... 2023-11-06T02:06:43.839 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package libllvm14:amd64. 2023-11-06T02:06:43.859 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1) ... 2023-11-06T02:06:43.867 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2023-11-06T02:06:43.905 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2023-11-06T02:06:44.120 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package libelf-dev:amd64. 2023-11-06T02:06:44.145 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2023-11-06T02:06:44.187 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2023-11-06T02:06:44.507 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package libfl2:amd64. 2023-11-06T02:06:44.534 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2023-11-06T02:06:44.573 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2023-11-06T02:06:44.842 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package libfl-dev:amd64. 2023-11-06T02:06:44.859 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2023-11-06T02:06:44.900 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2023-11-06T02:06:45.144 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package libjson-perl. 2023-11-06T02:06:45.170 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2023-11-06T02:06:45.204 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking libjson-perl (4.04000-1) ... 2023-11-06T02:06:45.481 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package libllvm14:amd64. 2023-11-06T02:06:45.509 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2023-11-06T02:06:45.516 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2023-11-06T02:06:45.533 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2023-11-06T02:06:45.547 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2023-11-06T02:06:45.566 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2023-11-06T02:06:45.836 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package libmbim-proxy. 2023-11-06T02:06:45.864 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2023-11-06T02:06:45.902 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2023-11-06T02:06:46.221 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package libmm-glib0:amd64. 2023-11-06T02:06:46.249 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.2_amd64.deb ... 2023-11-06T02:06:46.288 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.2) ... 2023-11-06T02:06:46.607 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package libndp0:amd64. 2023-11-06T02:06:46.635 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2023-11-06T02:06:46.665 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2023-11-06T02:06:46.960 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package libnm0:amd64. 2023-11-06T02:06:46.988 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2023-11-06T02:06:47.035 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2023-11-06T02:06:47.225 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2023-11-06T02:06:47.253 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2023-11-06T02:06:47.292 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2023-11-06T02:06:47.388 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package libpq5:amd64. 2023-11-06T02:06:47.415 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../16-libpq5_14.9-0ubuntu0.22.04.1_amd64.deb ... 2023-11-06T02:06:47.454 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking libpq5:amd64 (14.9-0ubuntu0.22.04.1) ... 2023-11-06T02:06:47.646 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package libmbim-proxy. 2023-11-06T02:06:47.662 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2023-11-06T02:06:47.704 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2023-11-06T02:06:47.799 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2023-11-06T02:06:47.826 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2023-11-06T02:06:47.874 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2023-11-06T02:06:47.990 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package libmm-glib0:amd64. 2023-11-06T02:06:48.018 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.2_amd64.deb ... 2023-11-06T02:06:48.056 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.2) ... 2023-11-06T02:06:48.185 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package libqmi-proxy. 2023-11-06T02:06:48.212 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2023-11-06T02:06:48.251 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2023-11-06T02:06:48.426 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package libndp0:amd64. 2023-11-06T02:06:48.454 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2023-11-06T02:06:48.479 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package libssl-dev:amd64. 2023-11-06T02:06:48.492 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2023-11-06T02:06:48.495 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.12_amd64.deb ... 2023-11-06T02:06:48.528 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.12) ... 2023-11-06T02:06:48.820 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package libnm0:amd64. 2023-11-06T02:06:48.849 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2023-11-06T02:06:48.887 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2023-11-06T02:06:49.057 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package libteamdctl0:amd64. 2023-11-06T02:06:49.085 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2023-11-06T02:06:49.124 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2023-11-06T02:06:49.190 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package libpq5:amd64. 2023-11-06T02:06:49.218 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../16-libpq5_14.9-0ubuntu0.22.04.1_amd64.deb ... 2023-11-06T02:06:49.273 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking libpq5:amd64 (14.9-0ubuntu0.22.04.1) ... 2023-11-06T02:06:49.418 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package modemmanager. 2023-11-06T02:06:49.446 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.2_amd64.deb ... 2023-11-06T02:06:49.485 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.2) ... 2023-11-06T02:06:49.617 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2023-11-06T02:06:49.646 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2023-11-06T02:06:49.684 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2023-11-06T02:06:50.012 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package libqmi-proxy. 2023-11-06T02:06:50.028 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2023-11-06T02:06:50.070 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2023-11-06T02:06:50.124 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package network-manager. 2023-11-06T02:06:50.152 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2023-11-06T02:06:50.199 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2023-11-06T02:06:50.281 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package libssl-dev:amd64. 2023-11-06T02:06:50.299 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.12_amd64.deb ... 2023-11-06T02:06:50.331 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.12) ... 2023-11-06T02:06:50.814 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package ppp. 2023-11-06T02:06:50.826 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package libteamdctl0:amd64. 2023-11-06T02:06:50.842 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2023-11-06T02:06:50.854 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2023-11-06T02:06:50.892 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2023-11-06T02:06:51.138 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2023-11-06T02:06:51.254 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package modemmanager. 2023-11-06T02:06:51.282 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.2_amd64.deb ... 2023-11-06T02:06:51.320 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.2) ... 2023-11-06T02:06:51.460 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package pptp-linux. 2023-11-06T02:06:51.476 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2023-11-06T02:06:51.511 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2023-11-06T02:06:51.713 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package network-manager-pptp. 2023-11-06T02:06:51.729 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2023-11-06T02:06:51.771 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2023-11-06T02:06:52.001 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package network-manager. 2023-11-06T02:06:52.030 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2023-11-06T02:06:52.074 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package postgresql-client-common. 2023-11-06T02:06:52.076 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2023-11-06T02:06:52.102 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2023-11-06T02:06:52.140 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking postgresql-client-common (238) ... 2023-11-06T02:06:52.453 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package postgresql-client-14. 2023-11-06T02:06:52.480 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../27-postgresql-client-14_14.9-0ubuntu0.22.04.1_amd64.deb ... 2023-11-06T02:06:52.519 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking postgresql-client-14 (14.9-0ubuntu0.22.04.1) ... 2023-11-06T02:06:53.032 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package postgresql-common. 2023-11-06T02:06:53.060 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2023-11-06T02:06:53.084 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package ppp. 2023-11-06T02:06:53.113 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2023-11-06T02:06:53.119 INFO:teuthology.orchestra.run.smithi150.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2023-11-06T02:06:53.139 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking postgresql-common (238) ... 2023-11-06T02:06:53.405 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2023-11-06T02:06:53.706 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package pptp-linux. 2023-11-06T02:06:53.734 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2023-11-06T02:06:53.742 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package postgresql-14. 2023-11-06T02:06:53.770 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../29-postgresql-14_14.9-0ubuntu0.22.04.1_amd64.deb ... 2023-11-06T02:06:53.772 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2023-11-06T02:06:53.812 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking postgresql-14 (14.9-0ubuntu0.22.04.1) ... 2023-11-06T02:06:54.025 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package network-manager-pptp. 2023-11-06T02:06:54.042 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2023-11-06T02:06:54.083 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2023-11-06T02:06:54.386 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package postgresql-client-common. 2023-11-06T02:06:54.415 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2023-11-06T02:06:54.453 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking postgresql-client-common (238) ... 2023-11-06T02:06:54.723 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package postgresql-client-14. 2023-11-06T02:06:54.740 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../27-postgresql-client-14_14.9-0ubuntu0.22.04.1_amd64.deb ... 2023-11-06T02:06:54.773 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking postgresql-client-14 (14.9-0ubuntu0.22.04.1) ... 2023-11-06T02:06:54.995 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package postgresql. 2023-11-06T02:06:55.011 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2023-11-06T02:06:55.053 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking postgresql (14+238) ... 2023-11-06T02:06:55.297 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package postgresql-client. 2023-11-06T02:06:55.325 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2023-11-06T02:06:55.345 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package postgresql-common. 2023-11-06T02:06:55.364 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking postgresql-client (14+238) ... 2023-11-06T02:06:55.374 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2023-11-06T02:06:55.438 INFO:teuthology.orchestra.run.smithi049.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2023-11-06T02:06:55.468 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking postgresql-common (238) ... 2023-11-06T02:06:55.608 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package postgresql-contrib. 2023-11-06T02:06:55.635 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2023-11-06T02:06:55.674 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking postgresql-contrib (14+238) ... 2023-11-06T02:06:55.927 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package postgresql-14. 2023-11-06T02:06:55.956 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../29-postgresql-14_14.9-0ubuntu0.22.04.1_amd64.deb ... 2023-11-06T02:06:55.960 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package tcl8.6. 2023-11-06T02:06:55.988 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2023-11-06T02:06:55.997 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking postgresql-14 (14.9-0ubuntu0.22.04.1) ... 2023-11-06T02:06:56.026 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2023-11-06T02:06:56.279 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package tcl. 2023-11-06T02:06:56.307 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2023-11-06T02:06:56.350 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking tcl (8.6.11+1build2) ... 2023-11-06T02:06:56.699 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package wpasupplicant. 2023-11-06T02:06:56.727 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2023-11-06T02:06:56.765 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2023-11-06T02:06:57.094 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package usb-modeswitch-data. 2023-11-06T02:06:57.110 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2023-11-06T02:06:57.147 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package postgresql. 2023-11-06T02:06:57.153 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2023-11-06T02:06:57.165 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2023-11-06T02:06:57.197 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking postgresql (14+238) ... 2023-11-06T02:06:57.424 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package postgresql-client. 2023-11-06T02:06:57.441 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2023-11-06T02:06:57.482 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking postgresql-client (14+238) ... 2023-11-06T02:06:57.564 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package usb-modeswitch. 2023-11-06T02:06:57.592 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2023-11-06T02:06:57.640 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2023-11-06T02:06:57.726 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package postgresql-contrib. 2023-11-06T02:06:57.743 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2023-11-06T02:06:57.785 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking postgresql-contrib (14+238) ... 2023-11-06T02:06:57.943 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up postgresql-client-common (238) ... 2023-11-06T02:06:58.113 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package tcl8.6. 2023-11-06T02:06:58.141 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2023-11-06T02:06:58.162 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up flex (2.6.4-8build2) ... 2023-11-06T02:06:58.288 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2023-11-06T02:06:58.296 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2023-11-06T02:06:58.431 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up libpq5:amd64 (14.9-0ubuntu0.22.04.1) ... 2023-11-06T02:06:58.557 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2023-11-06T02:06:58.667 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package tcl. 2023-11-06T02:06:58.683 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2023-11-06T02:06:58.696 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2023-11-06T02:06:58.738 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking tcl (8.6.11+1build2) ... 2023-11-06T02:06:58.834 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up postgresql-client-14 (14.9-0ubuntu0.22.04.1) ... 2023-11-06T02:06:59.103 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package wpasupplicant. 2023-11-06T02:06:59.132 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2023-11-06T02:06:59.170 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2023-11-06T02:06:59.329 INFO:teuthology.orchestra.run.smithi150.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2023-11-06T02:06:59.406 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2023-11-06T02:06:59.509 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package usb-modeswitch-data. 2023-11-06T02:06:59.538 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2023-11-06T02:06:59.576 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2023-11-06T02:06:59.791 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2023-11-06T02:06:59.791 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2023-11-06T02:06:59.954 INFO:teuthology.orchestra.run.smithi049.stdout:Selecting previously unselected package usb-modeswitch. 2023-11-06T02:06:59.983 INFO:teuthology.orchestra.run.smithi049.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2023-11-06T02:07:00.103 INFO:teuthology.orchestra.run.smithi049.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2023-11-06T02:07:00.326 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2023-11-06T02:07:00.375 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up postgresql-client-common (238) ... 2023-11-06T02:07:00.502 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.12) ... 2023-11-06T02:07:00.568 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up flex (2.6.4-8build2) ... 2023-11-06T02:07:00.636 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2023-11-06T02:07:00.686 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2023-11-06T02:07:00.762 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2023-11-06T02:07:00.812 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up libpq5:amd64 (14.9-0ubuntu0.22.04.1) ... 2023-11-06T02:07:00.889 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2023-11-06T02:07:00.938 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2023-11-06T02:07:01.015 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2023-11-06T02:07:01.073 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2023-11-06T02:07:01.141 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2023-11-06T02:07:01.199 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up postgresql-client-14 (14.9-0ubuntu0.22.04.1) ... 2023-11-06T02:07:01.773 INFO:teuthology.orchestra.run.smithi049.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2023-11-06T02:07:01.846 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2023-11-06T02:07:01.863 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2023-11-06T02:07:01.981 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up dump (0.4b47-3) ... 2023-11-06T02:07:02.074 INFO:teuthology.orchestra.run.smithi150.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2023-11-06T02:07:02.141 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.2) ... 2023-11-06T02:07:02.267 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1) ... 2023-11-06T02:07:02.283 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2023-11-06T02:07:02.283 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2023-11-06T02:07:02.409 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up tcl (8.6.11+1build2) ... 2023-11-06T02:07:02.560 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up libjson-perl (4.04000-1) ... 2023-11-06T02:07:02.711 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2023-11-06T02:07:02.802 INFO:teuthology.orchestra.run.smithi150.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2023-11-06T02:07:02.871 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2023-11-06T02:07:02.874 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2023-11-06T02:07:02.992 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.12) ... 2023-11-06T02:07:02.997 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2023-11-06T02:07:03.102 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2023-11-06T02:07:03.174 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2023-11-06T02:07:03.211 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2023-11-06T02:07:03.308 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2023-11-06T02:07:03.337 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2023-11-06T02:07:03.434 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2023-11-06T02:07:03.463 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2023-11-06T02:07:03.597 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2023-11-06T02:07:03.661 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up postgresql-client (14+238) ... 2023-11-06T02:07:03.904 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2023-11-06T02:07:04.219 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2023-11-06T02:07:04.346 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up dump (0.4b47-3) ... 2023-11-06T02:07:04.387 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:07:04.387 INFO:teuthology.orchestra.run.smithi150.stdout:The following network interfaces were found in /etc/network/interfaces 2023-11-06T02:07:04.388 INFO:teuthology.orchestra.run.smithi150.stdout:which means they are currently configured by ifupdown: 2023-11-06T02:07:04.388 INFO:teuthology.orchestra.run.smithi150.stdout:- enp3s0f1 2023-11-06T02:07:04.388 INFO:teuthology.orchestra.run.smithi150.stdout:If you want to manage those interfaces with NetworkManager instead 2023-11-06T02:07:04.388 INFO:teuthology.orchestra.run.smithi150.stdout:remove their configuration from /etc/network/interfaces. 2023-11-06T02:07:04.388 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:07:04.439 INFO:teuthology.orchestra.run.smithi049.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2023-11-06T02:07:04.502 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2023-11-06T02:07:04.505 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.2) ... 2023-11-06T02:07:04.640 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1) ... 2023-11-06T02:07:04.766 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up tcl (8.6.11+1build2) ... 2023-11-06T02:07:04.839 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2023-11-06T02:07:04.892 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up libjson-perl (4.04000-1) ... 2023-11-06T02:07:05.018 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2023-11-06T02:07:05.101 INFO:teuthology.orchestra.run.smithi049.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2023-11-06T02:07:05.153 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2023-11-06T02:07:05.170 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2023-11-06T02:07:05.270 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2023-11-06T02:07:05.447 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2023-11-06T02:07:05.573 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2023-11-06T02:07:05.699 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2023-11-06T02:07:05.731 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2023-11-06T02:07:05.866 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2023-11-06T02:07:05.867 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up postgresql-client (14+238) ... 2023-11-06T02:07:05.984 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2023-11-06T02:07:05.993 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2023-11-06T02:07:06.093 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up postgresql-common (238) ... 2023-11-06T02:07:06.501 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:07:06.502 INFO:teuthology.orchestra.run.smithi049.stdout:The following network interfaces were found in /etc/network/interfaces 2023-11-06T02:07:06.502 INFO:teuthology.orchestra.run.smithi049.stdout:which means they are currently configured by ifupdown: 2023-11-06T02:07:06.502 INFO:teuthology.orchestra.run.smithi049.stdout:- ens1f0 2023-11-06T02:07:06.502 INFO:teuthology.orchestra.run.smithi049.stdout:If you want to manage those interfaces with NetworkManager instead 2023-11-06T02:07:06.502 INFO:teuthology.orchestra.run.smithi049.stdout:remove their configuration from /etc/network/interfaces. 2023-11-06T02:07:06.503 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:07:06.625 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2023-11-06T02:07:06.964 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2023-11-06T02:07:06.988 INFO:teuthology.orchestra.run.smithi150.stdout:Adding user postgres to group ssl-cert 2023-11-06T02:07:07.230 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:07:07.230 INFO:teuthology.orchestra.run.smithi150.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2023-11-06T02:07:07.314 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2023-11-06T02:07:07.414 INFO:teuthology.orchestra.run.smithi150.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2023-11-06T02:07:07.414 INFO:teuthology.orchestra.run.smithi150.stdout:Removing obsolete dictionary files: 2023-11-06T02:07:07.870 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2023-11-06T02:07:07.993 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2023-11-06T02:07:08.030 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2023-11-06T02:07:08.156 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2023-11-06T02:07:08.282 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up postgresql-common (238) ... 2023-11-06T02:07:08.412 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up postgresql-14 (14.9-0ubuntu0.22.04.1) ... 2023-11-06T02:07:08.732 INFO:teuthology.orchestra.run.smithi150.stdout:Creating new PostgreSQL cluster 14/main ... 2023-11-06T02:07:08.771 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2023-11-06T02:07:08.809 INFO:teuthology.orchestra.run.smithi150.stdout:The files belonging to this database system will be owned by user "postgres". 2023-11-06T02:07:08.809 INFO:teuthology.orchestra.run.smithi150.stdout:This user must also own the server process. 2023-11-06T02:07:08.809 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:07:08.810 INFO:teuthology.orchestra.run.smithi150.stdout:The database cluster will be initialized with locale "en_US". 2023-11-06T02:07:08.810 INFO:teuthology.orchestra.run.smithi150.stdout:The default database encoding has accordingly been set to "LATIN1". 2023-11-06T02:07:08.810 INFO:teuthology.orchestra.run.smithi150.stdout:The default text search configuration will be set to "english". 2023-11-06T02:07:08.810 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:07:08.810 INFO:teuthology.orchestra.run.smithi150.stdout:Data page checksums are disabled. 2023-11-06T02:07:08.810 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:07:08.811 INFO:teuthology.orchestra.run.smithi150.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2023-11-06T02:07:08.811 INFO:teuthology.orchestra.run.smithi150.stdout:creating subdirectories ... ok 2023-11-06T02:07:08.811 INFO:teuthology.orchestra.run.smithi150.stdout:selecting dynamic shared memory implementation ... posix 2023-11-06T02:07:08.822 INFO:teuthology.orchestra.run.smithi150.stdout:selecting default max_connections ... 100 2023-11-06T02:07:08.843 INFO:teuthology.orchestra.run.smithi150.stdout:selecting default shared_buffers ... 128MB 2023-11-06T02:07:08.853 INFO:teuthology.orchestra.run.smithi150.stdout:selecting default time zone ... Etc/UTC 2023-11-06T02:07:08.854 INFO:teuthology.orchestra.run.smithi150.stdout:creating configuration files ... ok 2023-11-06T02:07:08.967 INFO:teuthology.orchestra.run.smithi150.stdout:running bootstrap script ... ok 2023-11-06T02:07:09.497 INFO:teuthology.orchestra.run.smithi049.stdout:Adding user postgres to group ssl-cert 2023-11-06T02:07:09.626 INFO:teuthology.orchestra.run.smithi150.stdout:performing post-bootstrap initialization ... ok 2023-11-06T02:07:09.727 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:07:09.727 INFO:teuthology.orchestra.run.smithi049.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2023-11-06T02:07:09.903 INFO:teuthology.orchestra.run.smithi049.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2023-11-06T02:07:09.903 INFO:teuthology.orchestra.run.smithi049.stdout:Removing obsolete dictionary files: 2023-11-06T02:07:10.291 INFO:teuthology.orchestra.run.smithi150.stdout:syncing data to disk ... ok 2023-11-06T02:07:10.506 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2023-11-06T02:07:10.662 INFO:teuthology.orchestra.run.smithi150.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2023-11-06T02:07:10.926 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up postgresql-14 (14.9-0ubuntu0.22.04.1) ... 2023-11-06T02:07:11.252 INFO:teuthology.orchestra.run.smithi049.stdout:Creating new PostgreSQL cluster 14/main ... 2023-11-06T02:07:11.269 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2023-11-06T02:07:11.307 INFO:teuthology.orchestra.run.smithi049.stdout:The files belonging to this database system will be owned by user "postgres". 2023-11-06T02:07:11.307 INFO:teuthology.orchestra.run.smithi049.stdout:This user must also own the server process. 2023-11-06T02:07:11.307 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:07:11.308 INFO:teuthology.orchestra.run.smithi049.stdout:The database cluster will be initialized with locale "en_US". 2023-11-06T02:07:11.308 INFO:teuthology.orchestra.run.smithi049.stdout:The default database encoding has accordingly been set to "LATIN1". 2023-11-06T02:07:11.308 INFO:teuthology.orchestra.run.smithi049.stdout:The default text search configuration will be set to "english". 2023-11-06T02:07:11.308 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:07:11.308 INFO:teuthology.orchestra.run.smithi049.stdout:Data page checksums are disabled. 2023-11-06T02:07:11.308 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:07:11.309 INFO:teuthology.orchestra.run.smithi049.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2023-11-06T02:07:11.309 INFO:teuthology.orchestra.run.smithi049.stdout:creating subdirectories ... ok 2023-11-06T02:07:11.309 INFO:teuthology.orchestra.run.smithi049.stdout:selecting dynamic shared memory implementation ... posix 2023-11-06T02:07:11.324 INFO:teuthology.orchestra.run.smithi049.stdout:selecting default max_connections ... 100 2023-11-06T02:07:11.344 INFO:teuthology.orchestra.run.smithi049.stdout:selecting default shared_buffers ... 128MB 2023-11-06T02:07:11.351 INFO:teuthology.orchestra.run.smithi049.stdout:selecting default time zone ... Etc/UTC 2023-11-06T02:07:11.352 INFO:teuthology.orchestra.run.smithi049.stdout:creating configuration files ... ok 2023-11-06T02:07:11.470 INFO:teuthology.orchestra.run.smithi049.stdout:running bootstrap script ... ok 2023-11-06T02:07:12.098 INFO:teuthology.orchestra.run.smithi049.stdout:performing post-bootstrap initialization ... ok 2023-11-06T02:07:12.694 INFO:teuthology.orchestra.run.smithi049.stdout:syncing data to disk ... ok 2023-11-06T02:07:13.111 INFO:teuthology.orchestra.run.smithi049.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2023-11-06T02:07:13.493 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2023-11-06T02:07:13.636 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up postgresql-contrib (14+238) ... 2023-11-06T02:07:13.762 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up postgresql (14+238) ... 2023-11-06T02:07:13.897 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2023-11-06T02:07:14.023 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.2) ... 2023-11-06T02:07:14.277 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2023-11-06T02:07:14.278 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2023-11-06T02:07:14.876 INFO:teuthology.orchestra.run.smithi150.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2023-11-06T02:07:15.186 INFO:teuthology.orchestra.run.smithi150.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2023-11-06T02:07:15.329 INFO:teuthology.orchestra.run.smithi150.stdout:Processing triggers for man-db (2.10.2-1) ... 2023-11-06T02:07:16.024 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2023-11-06T02:07:16.142 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up postgresql-contrib (14+238) ... 2023-11-06T02:07:16.243 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up postgresql (14+238) ... 2023-11-06T02:07:16.360 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2023-11-06T02:07:16.486 INFO:teuthology.orchestra.run.smithi049.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.2) ... 2023-11-06T02:07:16.674 INFO:teuthology.orchestra.run.smithi150.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2023-11-06T02:07:16.744 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2023-11-06T02:07:16.745 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2023-11-06T02:07:16.774 INFO:teuthology.orchestra.run.smithi150.stdout:Processing triggers for install-info (6.8-4build1) ... 2023-11-06T02:07:17.331 INFO:teuthology.orchestra.run.smithi049.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2023-11-06T02:07:17.420 INFO:teuthology.orchestra.run.smithi150.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-11-06T02:07:17.423 DEBUG:teuthology.parallel:result is None 2023-11-06T02:07:17.649 INFO:teuthology.orchestra.run.smithi049.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2023-11-06T02:07:17.801 INFO:teuthology.orchestra.run.smithi049.stdout:Processing triggers for man-db (2.10.2-1) ... 2023-11-06T02:07:19.146 INFO:teuthology.orchestra.run.smithi049.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2023-11-06T02:07:19.238 INFO:teuthology.orchestra.run.smithi049.stdout:Processing triggers for install-info (6.8-4build1) ... 2023-11-06T02:07:19.877 INFO:teuthology.orchestra.run.smithi049.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-11-06T02:07:19.880 DEBUG:teuthology.parallel:result is None 2023-11-06T02:07:19.880 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=f7e9f9af51acf98e0ccc3037487018e4ae40caa5 2023-11-06T02:07:20.030 DEBUG:teuthology.orchestra.run.smithi049:> dpkg-query -W -f '${Version}' ceph 2023-11-06T02:07:20.050 INFO:teuthology.orchestra.run.smithi049.stdout:18.2.0-1181-gf7e9f9af-1jammy 2023-11-06T02:07:20.050 INFO:teuthology.packaging:The installed version of ceph is 18.2.0-1181-gf7e9f9af-1jammy 2023-11-06T02:07:20.051 INFO:teuthology.task.install:The correct ceph version 18.2.0-1181-gf7e9f9af-1jammy is installed. 2023-11-06T02:07:20.051 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=f7e9f9af51acf98e0ccc3037487018e4ae40caa5 2023-11-06T02:07:20.342 DEBUG:teuthology.orchestra.run.smithi150:> dpkg-query -W -f '${Version}' ceph 2023-11-06T02:07:20.362 INFO:teuthology.orchestra.run.smithi150.stdout:18.2.0-1181-gf7e9f9af-1jammy 2023-11-06T02:07:20.362 INFO:teuthology.packaging:The installed version of ceph is 18.2.0-1181-gf7e9f9af-1jammy 2023-11-06T02:07:20.363 INFO:teuthology.task.install:The correct ceph version 18.2.0-1181-gf7e9f9af-1jammy is installed. 2023-11-06T02:07:20.363 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-11-06T02:07:20.364 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-11-06T02:07:20.364 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-06T02:07:20.379 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-11-06T02:07:20.380 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-06T02:07:20.417 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-11-06T02:07:20.417 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-11-06T02:07:20.417 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/usr/bin/daemon-helper 2023-11-06T02:07:20.439 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-06T02:07:20.495 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-11-06T02:07:20.495 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/usr/bin/daemon-helper 2023-11-06T02:07:20.508 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-06T02:07:20.564 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-11-06T02:07:20.564 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-11-06T02:07:20.565 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-06T02:07:20.577 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-06T02:07:20.630 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-11-06T02:07:20.631 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-06T02:07:20.644 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-06T02:07:20.703 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-11-06T02:07:20.703 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-11-06T02:07:20.703 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/usr/bin/stdin-killer 2023-11-06T02:07:20.717 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-06T02:07:20.771 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-11-06T02:07:20.771 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/usr/bin/stdin-killer 2023-11-06T02:07:20.785 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-06T02:07:20.839 INFO:teuthology.run_tasks:Running task ceph... 2023-11-06T02:07:20.901 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-11-06T02:07:20.902 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod 777 /var/log/ceph 2023-11-06T02:07:20.904 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 777 /var/log/ceph 2023-11-06T02:07:20.917 INFO:tasks.ceph:Disabling ceph logrotate... 2023-11-06T02:07:20.917 DEBUG:teuthology.orchestra.run.smithi049:> sudo rm -f -- /etc/logrotate.d/ceph 2023-11-06T02:07:20.960 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f -- /etc/logrotate.d/ceph 2023-11-06T02:07:20.973 INFO:tasks.ceph:Creating extra log directories... 2023-11-06T02:07:20.973 DEBUG:teuthology.orchestra.run.smithi049:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-11-06T02:07:21.020 DEBUG:teuthology.orchestra.run.smithi150:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-11-06T02:07:21.046 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-11-06T02:07:21.046 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-11-06T02:07:21.047 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-11-06T02:07:21.047 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-11-06T02:07:21.047 DEBUG:teuthology.orchestra.run.smithi049:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-11-06T02:07:21.080 DEBUG:teuthology.orchestra.run.smithi049:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-11-06T02:07:21.155 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-11-06T02:07:21.156 DEBUG:teuthology.orchestra.run.smithi150:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-11-06T02:07:21.162 DEBUG:teuthology.orchestra.run.smithi150:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-11-06T02:07:21.242 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-11-06T02:07:21.243 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-11-06T02:07:21.243 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2023-11-05_23:44:49-fs-reef-release-distro-default-smithi/7448684', 'branch': 'reef-release', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/forward-scrub}', 'email': 'yweinste@redhat.com', 'first_in_suite': False, 'job_id': '7448684', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'yuriw-2023-11-05_23:44:49-fs-reef-release-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'reef-release'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'sha1': 'f7e9f9af51acf98e0ccc3037487018e4ae40caa5'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'f7e9f9af51acf98e0ccc3037487018e4ae40caa5'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'reef-release', 'sha1': 'f7e9f9af51acf98e0ccc3037487018e4ae40caa5'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 71, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 1834, 'sha1': 'f7e9f9af51acf98e0ccc3037487018e4ae40caa5', 'sleep_before_teardown': 0, 'subset': '111/12000', 'suite': 'fs', 'suite_branch': 'reef-release', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_f7e9f9af51acf98e0ccc3037487018e4ae40caa5/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': 'f7e9f9af51acf98e0ccc3037487018e4ae40caa5', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi049.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBExxTF5Kt7KZ8Ef/cLx/cqs3AWnVQ4ai7T1jEDiGZzy9Q5LtI0OPverOvitoCFdXpAojj0/ANPgMelc6AP0UzDA=', 'smithi150.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHAJnIp8mtEt40gfMIRsjg5lCglKz1ohJ2WB8MsZn64LZXl/9/UGBQ4oYaVUsfH3FRK1MXjvSQFDp7zwxCN2mjg='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'f7e9f9af51acf98e0ccc3037487018e4ae40caa5'}}, {'ceph': None}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_forward_scrub']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '6899cd26fceddb2fec83dc1a1349394b28c8998e', 'timestamp': '2023-11-05_23:44:49', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3084822'} 2023-11-06T02:07:21.244 DEBUG:teuthology.orchestra.run.smithi049:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-11-06T02:07:21.247 DEBUG:teuthology.orchestra.run.smithi150:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-11-06T02:07:21.284 DEBUG:teuthology.orchestra.run.smithi049:> sudo install -d -m0777 -- /var/run/ceph 2023-11-06T02:07:21.296 DEBUG:teuthology.orchestra.run.smithi150:> sudo install -d -m0777 -- /var/run/ceph 2023-11-06T02:07:21.342 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-11-06T02:07:21.342 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/scratch_devs of=/dev/stdout 2023-11-06T02:07:21.356 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-06T02:07:21.356 DEBUG:teuthology.orchestra.run.smithi049:> stat /dev/vg_nvme/lv_1 2023-11-06T02:07:21.404 INFO:teuthology.orchestra.run.smithi049.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-06T02:07:21.404 INFO:teuthology.orchestra.run.smithi049.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-06T02:07:21.404 INFO:teuthology.orchestra.run.smithi049.stdout:Device: 5h/5d Inode: 875 Links: 1 2023-11-06T02:07:21.405 INFO:teuthology.orchestra.run.smithi049.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-06T02:07:21.405 INFO:teuthology.orchestra.run.smithi049.stdout:Access: 2023-11-06 02:01:17.219422114 +0000 2023-11-06T02:07:21.405 INFO:teuthology.orchestra.run.smithi049.stdout:Modify: 2023-11-06 02:01:16.851425400 +0000 2023-11-06T02:07:21.405 INFO:teuthology.orchestra.run.smithi049.stdout:Change: 2023-11-06 02:01:16.851425400 +0000 2023-11-06T02:07:21.405 INFO:teuthology.orchestra.run.smithi049.stdout: Birth: - 2023-11-06T02:07:21.406 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-06T02:07:21.458 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records in 2023-11-06T02:07:21.458 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records out 2023-11-06T02:07:21.458 INFO:teuthology.orchestra.run.smithi049.stderr:512 bytes copied, 0.000322837 s, 1.6 MB/s 2023-11-06T02:07:21.459 DEBUG:teuthology.orchestra.run.smithi049:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-06T02:07:21.510 DEBUG:teuthology.orchestra.run.smithi049:> stat /dev/vg_nvme/lv_2 2023-11-06T02:07:21.555 INFO:teuthology.orchestra.run.smithi049.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-06T02:07:21.556 INFO:teuthology.orchestra.run.smithi049.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-06T02:07:21.556 INFO:teuthology.orchestra.run.smithi049.stdout:Device: 5h/5d Inode: 892 Links: 1 2023-11-06T02:07:21.556 INFO:teuthology.orchestra.run.smithi049.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-06T02:07:21.556 INFO:teuthology.orchestra.run.smithi049.stdout:Access: 2023-11-06 02:01:17.875416255 +0000 2023-11-06T02:07:21.556 INFO:teuthology.orchestra.run.smithi049.stdout:Modify: 2023-11-06 02:01:17.459419969 +0000 2023-11-06T02:07:21.557 INFO:teuthology.orchestra.run.smithi049.stdout:Change: 2023-11-06 02:01:17.459419969 +0000 2023-11-06T02:07:21.557 INFO:teuthology.orchestra.run.smithi049.stdout: Birth: - 2023-11-06T02:07:21.557 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-06T02:07:21.610 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records in 2023-11-06T02:07:21.610 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records out 2023-11-06T02:07:21.610 INFO:teuthology.orchestra.run.smithi049.stderr:512 bytes copied, 0.000308944 s, 1.7 MB/s 2023-11-06T02:07:21.611 DEBUG:teuthology.orchestra.run.smithi049:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-06T02:07:21.660 DEBUG:teuthology.orchestra.run.smithi049:> stat /dev/vg_nvme/lv_3 2023-11-06T02:07:21.707 INFO:teuthology.orchestra.run.smithi049.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-06T02:07:21.708 INFO:teuthology.orchestra.run.smithi049.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-06T02:07:21.708 INFO:teuthology.orchestra.run.smithi049.stdout:Device: 5h/5d Inode: 904 Links: 1 2023-11-06T02:07:21.708 INFO:teuthology.orchestra.run.smithi049.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-06T02:07:21.708 INFO:teuthology.orchestra.run.smithi049.stdout:Access: 2023-11-06 02:01:18.495410717 +0000 2023-11-06T02:07:21.709 INFO:teuthology.orchestra.run.smithi049.stdout:Modify: 2023-11-06 02:01:18.135413931 +0000 2023-11-06T02:07:21.709 INFO:teuthology.orchestra.run.smithi049.stdout:Change: 2023-11-06 02:01:18.135413931 +0000 2023-11-06T02:07:21.709 INFO:teuthology.orchestra.run.smithi049.stdout: Birth: - 2023-11-06T02:07:21.709 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-06T02:07:21.761 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records in 2023-11-06T02:07:21.761 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records out 2023-11-06T02:07:21.762 INFO:teuthology.orchestra.run.smithi049.stderr:512 bytes copied, 0.000201637 s, 2.5 MB/s 2023-11-06T02:07:21.763 DEBUG:teuthology.orchestra.run.smithi049:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-06T02:07:21.812 DEBUG:teuthology.orchestra.run.smithi049:> stat /dev/vg_nvme/lv_4 2023-11-06T02:07:21.859 INFO:teuthology.orchestra.run.smithi049.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-06T02:07:21.859 INFO:teuthology.orchestra.run.smithi049.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-06T02:07:21.860 INFO:teuthology.orchestra.run.smithi049.stdout:Device: 5h/5d Inode: 926 Links: 1 2023-11-06T02:07:21.860 INFO:teuthology.orchestra.run.smithi049.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-06T02:07:21.860 INFO:teuthology.orchestra.run.smithi049.stdout:Access: 2023-11-06 02:01:18.703408859 +0000 2023-11-06T02:07:21.860 INFO:teuthology.orchestra.run.smithi049.stdout:Modify: 2023-11-06 02:01:18.699408894 +0000 2023-11-06T02:07:21.860 INFO:teuthology.orchestra.run.smithi049.stdout:Change: 2023-11-06 02:01:18.699408894 +0000 2023-11-06T02:07:21.861 INFO:teuthology.orchestra.run.smithi049.stdout: Birth: - 2023-11-06T02:07:21.861 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-06T02:07:21.914 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records in 2023-11-06T02:07:21.914 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records out 2023-11-06T02:07:21.914 INFO:teuthology.orchestra.run.smithi049.stderr:512 bytes copied, 0.000289845 s, 1.8 MB/s 2023-11-06T02:07:21.915 DEBUG:teuthology.orchestra.run.smithi049:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-06T02:07:21.968 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-06T02:07:21.969 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-11-06T02:07:21.969 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/scratch_devs of=/dev/stdout 2023-11-06T02:07:21.975 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-06T02:07:21.975 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_1 2023-11-06T02:07:22.024 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-06T02:07:22.024 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-06T02:07:22.025 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 5h/5d Inode: 876 Links: 1 2023-11-06T02:07:22.025 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-06T02:07:22.025 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2023-11-06 02:01:15.562003789 +0000 2023-11-06T02:07:22.025 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2023-11-06 02:01:15.178007215 +0000 2023-11-06T02:07:22.025 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2023-11-06 02:01:15.178007215 +0000 2023-11-06T02:07:22.026 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: - 2023-11-06T02:07:22.026 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-06T02:07:22.078 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2023-11-06T02:07:22.078 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2023-11-06T02:07:22.079 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.000378886 s, 1.4 MB/s 2023-11-06T02:07:22.080 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-06T02:07:22.129 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_2 2023-11-06T02:07:22.176 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-06T02:07:22.176 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-06T02:07:22.176 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 5h/5d Inode: 887 Links: 1 2023-11-06T02:07:22.177 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-06T02:07:22.177 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2023-11-06 02:01:16.213997973 +0000 2023-11-06T02:07:22.177 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2023-11-06 02:01:15.802001649 +0000 2023-11-06T02:07:22.177 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2023-11-06 02:01:15.802001649 +0000 2023-11-06T02:07:22.177 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: - 2023-11-06T02:07:22.178 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-06T02:07:22.230 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2023-11-06T02:07:22.230 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2023-11-06T02:07:22.231 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.000357623 s, 1.4 MB/s 2023-11-06T02:07:22.232 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-06T02:07:22.281 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_3 2023-11-06T02:07:22.332 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-06T02:07:22.332 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-06T02:07:22.332 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 5h/5d Inode: 908 Links: 1 2023-11-06T02:07:22.332 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-06T02:07:22.332 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2023-11-06 02:01:16.833992442 +0000 2023-11-06T02:07:22.333 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2023-11-06 02:01:16.477995618 +0000 2023-11-06T02:07:22.333 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2023-11-06 02:01:16.477995618 +0000 2023-11-06T02:07:22.333 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: - 2023-11-06T02:07:22.333 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-06T02:07:22.386 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2023-11-06T02:07:22.386 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2023-11-06T02:07:22.387 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.000360766 s, 1.4 MB/s 2023-11-06T02:07:22.388 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-06T02:07:22.437 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_4 2023-11-06T02:07:22.484 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-06T02:07:22.484 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-06T02:07:22.484 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 5h/5d Inode: 920 Links: 1 2023-11-06T02:07:22.485 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-06T02:07:22.485 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2023-11-06 02:01:17.429987125 +0000 2023-11-06T02:07:22.485 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2023-11-06 02:01:17.061990408 +0000 2023-11-06T02:07:22.485 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2023-11-06 02:01:17.061990408 +0000 2023-11-06T02:07:22.485 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: - 2023-11-06T02:07:22.486 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-06T02:07:22.538 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2023-11-06T02:07:22.539 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2023-11-06T02:07:22.539 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.000345409 s, 1.5 MB/s 2023-11-06T02:07:22.540 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-06T02:07:22.589 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-06T02:07:22.589 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi049.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi150.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-11-06T02:07:22.590 INFO:tasks.ceph:Generating config... 2023-11-06T02:07:22.591 INFO:tasks.ceph:[client] client mount timeout = 600 2023-11-06T02:07:22.591 INFO:tasks.ceph:[client] debug client = 20 2023-11-06T02:07:22.592 INFO:tasks.ceph:[client] debug ms = 1 2023-11-06T02:07:22.592 INFO:tasks.ceph:[client] pid file = 2023-11-06T02:07:22.592 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-11-06T02:07:22.592 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-11-06T02:07:22.593 INFO:tasks.ceph:[mds] debug mds = 20 2023-11-06T02:07:22.593 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-11-06T02:07:22.593 INFO:tasks.ceph:[mds] debug ms = 1 2023-11-06T02:07:22.594 INFO:tasks.ceph:[mds] mds debug frag = True 2023-11-06T02:07:22.594 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-11-06T02:07:22.594 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-11-06T02:07:22.594 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-11-06T02:07:22.595 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-11-06T02:07:22.595 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-11-06T02:07:22.595 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-11-06T02:07:22.595 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-11-06T02:07:22.596 INFO:tasks.ceph:[mgr] debug ms = 1 2023-11-06T02:07:22.596 INFO:tasks.ceph:[mon] debug mon = 20 2023-11-06T02:07:22.596 INFO:tasks.ceph:[mon] debug ms = 1 2023-11-06T02:07:22.596 INFO:tasks.ceph:[mon] debug paxos = 20 2023-11-06T02:07:22.597 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-11-06T02:07:22.597 INFO:tasks.ceph:[osd] bdev async discard = True 2023-11-06T02:07:22.597 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-11-06T02:07:22.597 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-11-06T02:07:22.598 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-11-06T02:07:22.598 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-11-06T02:07:22.598 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-11-06T02:07:22.599 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-11-06T02:07:22.601 INFO:tasks.ceph:[osd] debug ms = 1 2023-11-06T02:07:22.601 INFO:tasks.ceph:[osd] debug osd = 20 2023-11-06T02:07:22.602 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-11-06T02:07:22.602 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-11-06T02:07:22.602 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-11-06T02:07:22.603 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-11-06T02:07:22.603 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-11-06T02:07:22.603 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-11-06T02:07:22.603 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-11-06T02:07:22.604 INFO:tasks.ceph:Setting up mon.a... 2023-11-06T02:07:22.604 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-11-06T02:07:22.640 INFO:teuthology.orchestra.run.smithi049.stdout:creating /etc/ceph/ceph.keyring 2023-11-06T02:07:22.660 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-11-06T02:07:22.743 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-06T02:07:22.806 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.49'), ('mon.b', '172.21.15.150'), ('mon.c', '[v2:172.21.15.150:3301,v1:172.21.15.150:6790]')] 2023-11-06T02:07:22.807 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.49,172.21.15.150,[v2:172.21.15.150:3301,v1:172.21.15.150:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-11-06T02:07:22.808 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-11-06T02:07:22.808 DEBUG:teuthology.orchestra.run.smithi049:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-11-06T02:07:22.851 DEBUG:teuthology.orchestra.run.smithi049:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.49 --add b 172.21.15.150 --addv c '[v2:172.21.15.150:3301,v1:172.21.15.150:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-11-06T02:07:22.937 INFO:teuthology.orchestra.run.smithi049.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-11-06T02:07:22.937 INFO:teuthology.orchestra.run.smithi049.stdout:monmaptool: generated fsid 1a8b035c-6ba4-499a-b876-aea47fd9c0d4 2023-11-06T02:07:22.937 INFO:teuthology.orchestra.run.smithi049.stdout:setting min_mon_release = pacific 2023-11-06T02:07:22.938 INFO:teuthology.orchestra.run.smithi049.stdout:epoch 0 2023-11-06T02:07:22.938 INFO:teuthology.orchestra.run.smithi049.stdout:fsid 1a8b035c-6ba4-499a-b876-aea47fd9c0d4 2023-11-06T02:07:22.938 INFO:teuthology.orchestra.run.smithi049.stdout:last_changed 2023-11-06T02:07:22.945285+0000 2023-11-06T02:07:22.938 INFO:teuthology.orchestra.run.smithi049.stdout:created 2023-11-06T02:07:22.945285+0000 2023-11-06T02:07:22.939 INFO:teuthology.orchestra.run.smithi049.stdout:min_mon_release 16 (pacific) 2023-11-06T02:07:22.939 INFO:teuthology.orchestra.run.smithi049.stdout:election_strategy: 1 2023-11-06T02:07:22.939 INFO:teuthology.orchestra.run.smithi049.stdout:0: [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] mon.a 2023-11-06T02:07:22.939 INFO:teuthology.orchestra.run.smithi049.stdout:1: [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] mon.b 2023-11-06T02:07:22.939 INFO:teuthology.orchestra.run.smithi049.stdout:2: [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] mon.c 2023-11-06T02:07:22.940 INFO:teuthology.orchestra.run.smithi049.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-11-06T02:07:22.941 DEBUG:teuthology.orchestra.run.smithi049:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-11-06T02:07:22.988 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 1a8b035c-6ba4-499a-b876-aea47fd9c0d4... 2023-11-06T02:07:22.989 DEBUG:teuthology.orchestra.run.smithi049:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-06T02:07:23.032 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-06T02:07:23.063 INFO:teuthology.orchestra.run.smithi150.stdout:[global] 2023-11-06T02:07:23.063 INFO:teuthology.orchestra.run.smithi150.stdout: chdir = "" 2023-11-06T02:07:23.063 INFO:teuthology.orchestra.run.smithi150.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-06T02:07:23.063 INFO:teuthology.orchestra.run.smithi150.stdout: auth supported = cephx 2023-11-06T02:07:23.063 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:07:23.064 INFO:teuthology.orchestra.run.smithi150.stdout: filestore xattr use omap = true 2023-11-06T02:07:23.064 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:07:23.064 INFO:teuthology.orchestra.run.smithi150.stdout: mon clock drift allowed = 1.000 2023-11-06T02:07:23.064 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:07:23.064 INFO:teuthology.orchestra.run.smithi150.stdout: osd crush chooseleaf type = 0 2023-11-06T02:07:23.065 INFO:teuthology.orchestra.run.smithi150.stdout: auth debug = true 2023-11-06T02:07:23.065 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:07:23.065 INFO:teuthology.orchestra.run.smithi150.stdout: ms die on old message = true 2023-11-06T02:07:23.065 INFO:teuthology.orchestra.run.smithi150.stdout: ms die on bug = true 2023-11-06T02:07:23.065 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:07:23.065 INFO:teuthology.orchestra.run.smithi150.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-06T02:07:23.066 INFO:teuthology.orchestra.run.smithi150.stdout: mon pg warn max object skew = 0 2023-11-06T02:07:23.066 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:07:23.066 INFO:teuthology.orchestra.run.smithi150.stdout: # disable pg_autoscaler by default for new pools 2023-11-06T02:07:23.066 INFO:teuthology.orchestra.run.smithi150.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-06T02:07:23.066 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:07:23.067 INFO:teuthology.orchestra.run.smithi150.stdout: osd pool default size = 2 2023-11-06T02:07:23.067 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:07:23.067 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd allow primary affinity = true 2023-11-06T02:07:23.067 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd allow pg remap = true 2023-11-06T02:07:23.067 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on legacy crush tunables = false 2023-11-06T02:07:23.067 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on crush straw calc version zero = false 2023-11-06T02:07:23.068 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on no sortbitwise = false 2023-11-06T02:07:23.068 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on osd down out interval zero = false 2023-11-06T02:07:23.068 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on too few osds = false 2023-11-06T02:07:23.068 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-06T02:07:23.068 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-06T02:07:23.069 INFO:teuthology.orchestra.run.smithi150.stdout: mon_allow_pool_size_one = true 2023-11-06T02:07:23.069 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:07:23.069 INFO:teuthology.orchestra.run.smithi150.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-06T02:07:23.069 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:07:23.069 INFO:teuthology.orchestra.run.smithi150.stdout: osd default data pool replay window = 5 2023-11-06T02:07:23.070 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:07:23.070 INFO:teuthology.orchestra.run.smithi150.stdout: mon allow pool delete = true 2023-11-06T02:07:23.070 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:07:23.070 INFO:teuthology.orchestra.run.smithi150.stdout: mon cluster log file level = debug 2023-11-06T02:07:23.070 INFO:teuthology.orchestra.run.smithi150.stdout: debug asserts on shutdown = true 2023-11-06T02:07:23.070 INFO:teuthology.orchestra.run.smithi150.stdout: mon health detail to clog = false 2023-11-06T02:07:23.071 INFO:teuthology.orchestra.run.smithi150.stdout: mon host = "172.21.15.49,172.21.15.150,[v2:172.21.15.150:3301,v1:172.21.15.150:6790]" 2023-11-06T02:07:23.071 INFO:teuthology.orchestra.run.smithi150.stdout: fsid = 1a8b035c-6ba4-499a-b876-aea47fd9c0d4 2023-11-06T02:07:23.071 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:07:23.071 INFO:teuthology.orchestra.run.smithi150.stdout:[osd] 2023-11-06T02:07:23.071 INFO:teuthology.orchestra.run.smithi150.stdout: osd journal size = 100 2023-11-06T02:07:23.072 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:07:23.072 INFO:teuthology.orchestra.run.smithi150.stdout: osd scrub load threshold = 5.0 2023-11-06T02:07:23.072 INFO:teuthology.orchestra.run.smithi150.stdout: osd scrub max interval = 600 2023-11-06T02:07:23.072 INFO:teuthology.orchestra.run.smithi150.stdout: osd mclock profile = high_recovery_ops 2023-11-06T02:07:23.072 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:07:23.073 INFO:teuthology.orchestra.run.smithi150.stdout: osd recover clone overlap = true 2023-11-06T02:07:23.073 INFO:teuthology.orchestra.run.smithi150.stdout: osd recovery max chunk = 1048576 2023-11-06T02:07:23.073 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:07:23.073 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug shutdown = true 2023-11-06T02:07:23.073 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug op order = true 2023-11-06T02:07:23.074 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug verify stray on activate = true 2023-11-06T02:07:23.074 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:07:23.074 INFO:teuthology.orchestra.run.smithi150.stdout: osd open classes on start = true 2023-11-06T02:07:23.074 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug pg log writeout = true 2023-11-06T02:07:23.074 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:07:23.075 INFO:teuthology.orchestra.run.smithi150.stdout: osd deep scrub update digest min age = 30 2023-11-06T02:07:23.075 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:07:23.075 INFO:teuthology.orchestra.run.smithi150.stdout: osd map max advance = 10 2023-11-06T02:07:23.075 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:07:23.075 INFO:teuthology.orchestra.run.smithi150.stdout: journal zero on create = true 2023-11-06T02:07:23.076 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:07:23.076 INFO:teuthology.orchestra.run.smithi150.stdout: filestore ondisk finisher threads = 3 2023-11-06T02:07:23.076 INFO:teuthology.orchestra.run.smithi150.stdout: filestore apply finisher threads = 3 2023-11-06T02:07:23.076 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:07:23.076 INFO:teuthology.orchestra.run.smithi150.stdout: bdev debug aio = true 2023-11-06T02:07:23.077 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug misdirected ops = true 2023-11-06T02:07:23.077 INFO:teuthology.orchestra.run.smithi150.stdout: bdev async discard = True 2023-11-06T02:07:23.077 INFO:teuthology.orchestra.run.smithi150.stdout: bdev enable discard = True 2023-11-06T02:07:23.077 INFO:teuthology.orchestra.run.smithi150.stdout: bluestore allocator = bitmap 2023-11-06T02:07:23.077 INFO:teuthology.orchestra.run.smithi150.stdout: bluestore block size = 96636764160 2023-11-06T02:07:23.078 INFO:teuthology.orchestra.run.smithi150.stdout: bluestore fsck on mount = True 2023-11-06T02:07:23.078 INFO:teuthology.orchestra.run.smithi150.stdout: debug bluefs = 1/20 2023-11-06T02:07:23.078 INFO:teuthology.orchestra.run.smithi150.stdout: debug bluestore = 1/20 2023-11-06T02:07:23.078 INFO:teuthology.orchestra.run.smithi150.stdout: debug ms = 1 2023-11-06T02:07:23.078 INFO:teuthology.orchestra.run.smithi150.stdout: debug osd = 20 2023-11-06T02:07:23.079 INFO:teuthology.orchestra.run.smithi150.stdout: debug rocksdb = 4/10 2023-11-06T02:07:23.079 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd backfillfull_ratio = 0.85 2023-11-06T02:07:23.079 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd full ratio = 0.9 2023-11-06T02:07:23.079 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd nearfull ratio = 0.8 2023-11-06T02:07:23.079 INFO:teuthology.orchestra.run.smithi150.stdout: osd failsafe full ratio = 0.95 2023-11-06T02:07:23.080 INFO:teuthology.orchestra.run.smithi150.stdout: osd objectstore = bluestore 2023-11-06T02:07:23.080 INFO:teuthology.orchestra.run.smithi150.stdout: osd op complaint time = 180 2023-11-06T02:07:23.080 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:07:23.080 INFO:teuthology.orchestra.run.smithi150.stdout:[mgr] 2023-11-06T02:07:23.080 INFO:teuthology.orchestra.run.smithi150.stdout: debug ms = 1 2023-11-06T02:07:23.081 INFO:teuthology.orchestra.run.smithi150.stdout: debug mgr = 20 2023-11-06T02:07:23.081 INFO:teuthology.orchestra.run.smithi150.stdout: debug mon = 20 2023-11-06T02:07:23.081 INFO:teuthology.orchestra.run.smithi150.stdout: debug auth = 20 2023-11-06T02:07:23.081 INFO:teuthology.orchestra.run.smithi150.stdout: mon reweight min pgs per osd = 4 2023-11-06T02:07:23.081 INFO:teuthology.orchestra.run.smithi150.stdout: mon reweight min bytes per osd = 10 2023-11-06T02:07:23.082 INFO:teuthology.orchestra.run.smithi150.stdout: mgr/telemetry/nag = false 2023-11-06T02:07:23.082 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:07:23.082 INFO:teuthology.orchestra.run.smithi150.stdout:[mon] 2023-11-06T02:07:23.082 INFO:teuthology.orchestra.run.smithi150.stdout: debug ms = 1 2023-11-06T02:07:23.082 INFO:teuthology.orchestra.run.smithi150.stdout: debug mon = 20 2023-11-06T02:07:23.083 INFO:teuthology.orchestra.run.smithi150.stdout: debug paxos = 20 2023-11-06T02:07:23.083 INFO:teuthology.orchestra.run.smithi150.stdout: debug auth = 20 2023-11-06T02:07:23.083 INFO:teuthology.orchestra.run.smithi150.stdout: mon data avail warn = 5 2023-11-06T02:07:23.083 INFO:teuthology.orchestra.run.smithi150.stdout: mon mgr mkfs grace = 240 2023-11-06T02:07:23.083 INFO:teuthology.orchestra.run.smithi150.stdout: mon reweight min pgs per osd = 4 2023-11-06T02:07:23.084 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd reporter subtree level = osd 2023-11-06T02:07:23.084 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd prime pg temp = true 2023-11-06T02:07:23.084 INFO:teuthology.orchestra.run.smithi150.stdout: mon reweight min bytes per osd = 10 2023-11-06T02:07:23.084 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:07:23.084 INFO:teuthology.orchestra.run.smithi150.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-06T02:07:23.084 INFO:teuthology.orchestra.run.smithi150.stdout: auth mon ticket ttl = 660 # 11m 2023-11-06T02:07:23.085 INFO:teuthology.orchestra.run.smithi150.stdout: auth service ticket ttl = 240 # 4m 2023-11-06T02:07:23.085 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:07:23.085 INFO:teuthology.orchestra.run.smithi150.stdout: # don't complain about insecure global_id in the test suite 2023-11-06T02:07:23.085 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-06T02:07:23.085 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-06T02:07:23.086 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:07:23.086 INFO:teuthology.orchestra.run.smithi150.stdout: # 1m isn't quite enough 2023-11-06T02:07:23.086 INFO:teuthology.orchestra.run.smithi150.stdout: mon_down_mkfs_grace = 2m 2023-11-06T02:07:23.086 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:07:23.086 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_filestore_osds = false 2023-11-06T02:07:23.087 INFO:teuthology.orchestra.run.smithi150.stdout: mon op complaint time = 120 2023-11-06T02:07:23.087 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:07:23.087 INFO:teuthology.orchestra.run.smithi150.stdout:[client] 2023-11-06T02:07:23.087 INFO:teuthology.orchestra.run.smithi150.stdout: rgw cache enabled = true 2023-11-06T02:07:23.087 INFO:teuthology.orchestra.run.smithi150.stdout: rgw enable ops log = true 2023-11-06T02:07:23.087 INFO:teuthology.orchestra.run.smithi150.stdout: rgw enable usage log = true 2023-11-06T02:07:23.088 INFO:teuthology.orchestra.run.smithi150.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-06T02:07:23.088 INFO:teuthology.orchestra.run.smithi150.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-06T02:07:23.088 INFO:teuthology.orchestra.run.smithi150.stdout: client mount timeout = 600 2023-11-06T02:07:23.088 INFO:teuthology.orchestra.run.smithi150.stdout: debug client = 20 2023-11-06T02:07:23.088 INFO:teuthology.orchestra.run.smithi150.stdout: debug ms = 1 2023-11-06T02:07:23.089 INFO:teuthology.orchestra.run.smithi150.stdout: pid file = "" 2023-11-06T02:07:23.089 INFO:teuthology.orchestra.run.smithi150.stdout: rados mon op timeout = 900 2023-11-06T02:07:23.089 INFO:teuthology.orchestra.run.smithi150.stdout: rados osd op timeout = 900 2023-11-06T02:07:23.089 INFO:teuthology.orchestra.run.smithi150.stdout:[mon.a] 2023-11-06T02:07:23.089 INFO:teuthology.orchestra.run.smithi150.stdout:[mon.b] 2023-11-06T02:07:23.089 INFO:teuthology.orchestra.run.smithi150.stdout:[mon.c] 2023-11-06T02:07:23.090 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.a] 2023-11-06T02:07:23.090 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.c] 2023-11-06T02:07:23.090 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.b] 2023-11-06T02:07:23.090 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.d] 2023-11-06T02:07:23.090 INFO:teuthology.orchestra.run.smithi150.stdout:[mds] 2023-11-06T02:07:23.091 INFO:teuthology.orchestra.run.smithi150.stdout: debug mds = 20 2023-11-06T02:07:23.091 INFO:teuthology.orchestra.run.smithi150.stdout: debug mds balancer = 20 2023-11-06T02:07:23.091 INFO:teuthology.orchestra.run.smithi150.stdout: debug ms = 1 2023-11-06T02:07:23.091 INFO:teuthology.orchestra.run.smithi150.stdout: mds debug frag = True 2023-11-06T02:07:23.091 INFO:teuthology.orchestra.run.smithi150.stdout: mds debug scatterstat = True 2023-11-06T02:07:23.092 INFO:teuthology.orchestra.run.smithi150.stdout: mds op complaint time = 180 2023-11-06T02:07:23.092 INFO:teuthology.orchestra.run.smithi150.stdout: mds verify scatter = True 2023-11-06T02:07:23.092 INFO:teuthology.orchestra.run.smithi150.stdout: osd op complaint time = 180 2023-11-06T02:07:23.092 INFO:teuthology.orchestra.run.smithi150.stdout: rados mon op timeout = 900 2023-11-06T02:07:23.092 INFO:teuthology.orchestra.run.smithi150.stdout: rados osd op timeout = 900 2023-11-06T02:07:23.094 INFO:teuthology.orchestra.run.smithi049.stdout:[global] 2023-11-06T02:07:23.094 INFO:teuthology.orchestra.run.smithi049.stdout: chdir = "" 2023-11-06T02:07:23.094 INFO:teuthology.orchestra.run.smithi049.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-06T02:07:23.094 INFO:teuthology.orchestra.run.smithi049.stdout: auth supported = cephx 2023-11-06T02:07:23.095 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:07:23.095 INFO:teuthology.orchestra.run.smithi049.stdout: filestore xattr use omap = true 2023-11-06T02:07:23.095 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:07:23.095 INFO:teuthology.orchestra.run.smithi049.stdout: mon clock drift allowed = 1.000 2023-11-06T02:07:23.095 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:07:23.096 INFO:teuthology.orchestra.run.smithi049.stdout: osd crush chooseleaf type = 0 2023-11-06T02:07:23.096 INFO:teuthology.orchestra.run.smithi049.stdout: auth debug = true 2023-11-06T02:07:23.096 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:07:23.096 INFO:teuthology.orchestra.run.smithi049.stdout: ms die on old message = true 2023-11-06T02:07:23.096 INFO:teuthology.orchestra.run.smithi049.stdout: ms die on bug = true 2023-11-06T02:07:23.097 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:07:23.097 INFO:teuthology.orchestra.run.smithi049.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-06T02:07:23.097 INFO:teuthology.orchestra.run.smithi049.stdout: mon pg warn max object skew = 0 2023-11-06T02:07:23.097 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:07:23.097 INFO:teuthology.orchestra.run.smithi049.stdout: # disable pg_autoscaler by default for new pools 2023-11-06T02:07:23.097 INFO:teuthology.orchestra.run.smithi049.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-06T02:07:23.098 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:07:23.098 INFO:teuthology.orchestra.run.smithi049.stdout: osd pool default size = 2 2023-11-06T02:07:23.098 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:07:23.098 INFO:teuthology.orchestra.run.smithi049.stdout: mon osd allow primary affinity = true 2023-11-06T02:07:23.098 INFO:teuthology.orchestra.run.smithi049.stdout: mon osd allow pg remap = true 2023-11-06T02:07:23.099 INFO:teuthology.orchestra.run.smithi049.stdout: mon warn on legacy crush tunables = false 2023-11-06T02:07:23.099 INFO:teuthology.orchestra.run.smithi049.stdout: mon warn on crush straw calc version zero = false 2023-11-06T02:07:23.099 INFO:teuthology.orchestra.run.smithi049.stdout: mon warn on no sortbitwise = false 2023-11-06T02:07:23.099 INFO:teuthology.orchestra.run.smithi049.stdout: mon warn on osd down out interval zero = false 2023-11-06T02:07:23.099 INFO:teuthology.orchestra.run.smithi049.stdout: mon warn on too few osds = false 2023-11-06T02:07:23.099 INFO:teuthology.orchestra.run.smithi049.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-06T02:07:23.100 INFO:teuthology.orchestra.run.smithi049.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-06T02:07:23.100 INFO:teuthology.orchestra.run.smithi049.stdout: mon_allow_pool_size_one = true 2023-11-06T02:07:23.100 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:07:23.100 INFO:teuthology.orchestra.run.smithi049.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-06T02:07:23.100 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:07:23.101 INFO:teuthology.orchestra.run.smithi049.stdout: osd default data pool replay window = 5 2023-11-06T02:07:23.101 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:07:23.101 INFO:teuthology.orchestra.run.smithi049.stdout: mon allow pool delete = true 2023-11-06T02:07:23.101 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:07:23.102 INFO:teuthology.orchestra.run.smithi049.stdout: mon cluster log file level = debug 2023-11-06T02:07:23.102 INFO:teuthology.orchestra.run.smithi049.stdout: debug asserts on shutdown = true 2023-11-06T02:07:23.102 INFO:teuthology.orchestra.run.smithi049.stdout: mon health detail to clog = false 2023-11-06T02:07:23.102 INFO:teuthology.orchestra.run.smithi049.stdout: mon host = "172.21.15.49,172.21.15.150,[v2:172.21.15.150:3301,v1:172.21.15.150:6790]" 2023-11-06T02:07:23.102 INFO:teuthology.orchestra.run.smithi049.stdout: fsid = 1a8b035c-6ba4-499a-b876-aea47fd9c0d4 2023-11-06T02:07:23.103 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:07:23.103 INFO:teuthology.orchestra.run.smithi049.stdout:[osd] 2023-11-06T02:07:23.103 INFO:teuthology.orchestra.run.smithi049.stdout: osd journal size = 100 2023-11-06T02:07:23.103 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:07:23.103 INFO:teuthology.orchestra.run.smithi049.stdout: osd scrub load threshold = 5.0 2023-11-06T02:07:23.104 INFO:teuthology.orchestra.run.smithi049.stdout: osd scrub max interval = 600 2023-11-06T02:07:23.104 INFO:teuthology.orchestra.run.smithi049.stdout: osd mclock profile = high_recovery_ops 2023-11-06T02:07:23.104 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:07:23.104 INFO:teuthology.orchestra.run.smithi049.stdout: osd recover clone overlap = true 2023-11-06T02:07:23.104 INFO:teuthology.orchestra.run.smithi049.stdout: osd recovery max chunk = 1048576 2023-11-06T02:07:23.104 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:07:23.105 INFO:teuthology.orchestra.run.smithi049.stdout: osd debug shutdown = true 2023-11-06T02:07:23.105 INFO:teuthology.orchestra.run.smithi049.stdout: osd debug op order = true 2023-11-06T02:07:23.105 INFO:teuthology.orchestra.run.smithi049.stdout: osd debug verify stray on activate = true 2023-11-06T02:07:23.105 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:07:23.105 INFO:teuthology.orchestra.run.smithi049.stdout: osd open classes on start = true 2023-11-06T02:07:23.106 INFO:teuthology.orchestra.run.smithi049.stdout: osd debug pg log writeout = true 2023-11-06T02:07:23.106 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:07:23.106 INFO:teuthology.orchestra.run.smithi049.stdout: osd deep scrub update digest min age = 30 2023-11-06T02:07:23.106 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:07:23.106 INFO:teuthology.orchestra.run.smithi049.stdout: osd map max advance = 10 2023-11-06T02:07:23.107 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:07:23.107 INFO:teuthology.orchestra.run.smithi049.stdout: journal zero on create = true 2023-11-06T02:07:23.107 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:07:23.107 INFO:teuthology.orchestra.run.smithi049.stdout: filestore ondisk finisher threads = 3 2023-11-06T02:07:23.107 INFO:teuthology.orchestra.run.smithi049.stdout: filestore apply finisher threads = 3 2023-11-06T02:07:23.108 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:07:23.108 INFO:teuthology.orchestra.run.smithi049.stdout: bdev debug aio = true 2023-11-06T02:07:23.108 INFO:teuthology.orchestra.run.smithi049.stdout: osd debug misdirected ops = true 2023-11-06T02:07:23.108 INFO:teuthology.orchestra.run.smithi049.stdout: bdev async discard = True 2023-11-06T02:07:23.108 INFO:teuthology.orchestra.run.smithi049.stdout: bdev enable discard = True 2023-11-06T02:07:23.109 INFO:teuthology.orchestra.run.smithi049.stdout: bluestore allocator = bitmap 2023-11-06T02:07:23.109 INFO:teuthology.orchestra.run.smithi049.stdout: bluestore block size = 96636764160 2023-11-06T02:07:23.109 INFO:teuthology.orchestra.run.smithi049.stdout: bluestore fsck on mount = True 2023-11-06T02:07:23.109 INFO:teuthology.orchestra.run.smithi049.stdout: debug bluefs = 1/20 2023-11-06T02:07:23.109 INFO:teuthology.orchestra.run.smithi049.stdout: debug bluestore = 1/20 2023-11-06T02:07:23.110 INFO:teuthology.orchestra.run.smithi049.stdout: debug ms = 1 2023-11-06T02:07:23.110 INFO:teuthology.orchestra.run.smithi049.stdout: debug osd = 20 2023-11-06T02:07:23.110 INFO:teuthology.orchestra.run.smithi049.stdout: debug rocksdb = 4/10 2023-11-06T02:07:23.110 INFO:teuthology.orchestra.run.smithi049.stdout: mon osd backfillfull_ratio = 0.85 2023-11-06T02:07:23.110 INFO:teuthology.orchestra.run.smithi049.stdout: mon osd full ratio = 0.9 2023-11-06T02:07:23.111 INFO:teuthology.orchestra.run.smithi049.stdout: mon osd nearfull ratio = 0.8 2023-11-06T02:07:23.111 INFO:teuthology.orchestra.run.smithi049.stdout: osd failsafe full ratio = 0.95 2023-11-06T02:07:23.111 INFO:teuthology.orchestra.run.smithi049.stdout: osd objectstore = bluestore 2023-11-06T02:07:23.111 INFO:teuthology.orchestra.run.smithi049.stdout: osd op complaint time = 180 2023-11-06T02:07:23.111 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:07:23.112 INFO:teuthology.orchestra.run.smithi049.stdout:[mgr] 2023-11-06T02:07:23.112 INFO:teuthology.orchestra.run.smithi049.stdout: debug ms = 1 2023-11-06T02:07:23.112 INFO:teuthology.orchestra.run.smithi049.stdout: debug mgr = 20 2023-11-06T02:07:23.112 INFO:teuthology.orchestra.run.smithi049.stdout: debug mon = 20 2023-11-06T02:07:23.112 INFO:teuthology.orchestra.run.smithi049.stdout: debug auth = 20 2023-11-06T02:07:23.112 INFO:teuthology.orchestra.run.smithi049.stdout: mon reweight min pgs per osd = 4 2023-11-06T02:07:23.113 INFO:teuthology.orchestra.run.smithi049.stdout: mon reweight min bytes per osd = 10 2023-11-06T02:07:23.113 INFO:teuthology.orchestra.run.smithi049.stdout: mgr/telemetry/nag = false 2023-11-06T02:07:23.113 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:07:23.113 INFO:teuthology.orchestra.run.smithi049.stdout:[mon] 2023-11-06T02:07:23.113 INFO:teuthology.orchestra.run.smithi049.stdout: debug ms = 1 2023-11-06T02:07:23.114 INFO:teuthology.orchestra.run.smithi049.stdout: debug mon = 20 2023-11-06T02:07:23.114 INFO:teuthology.orchestra.run.smithi049.stdout: debug paxos = 20 2023-11-06T02:07:23.114 INFO:teuthology.orchestra.run.smithi049.stdout: debug auth = 20 2023-11-06T02:07:23.114 INFO:teuthology.orchestra.run.smithi049.stdout: mon data avail warn = 5 2023-11-06T02:07:23.114 INFO:teuthology.orchestra.run.smithi049.stdout: mon mgr mkfs grace = 240 2023-11-06T02:07:23.114 INFO:teuthology.orchestra.run.smithi049.stdout: mon reweight min pgs per osd = 4 2023-11-06T02:07:23.115 INFO:teuthology.orchestra.run.smithi049.stdout: mon osd reporter subtree level = osd 2023-11-06T02:07:23.115 INFO:teuthology.orchestra.run.smithi049.stdout: mon osd prime pg temp = true 2023-11-06T02:07:23.115 INFO:teuthology.orchestra.run.smithi049.stdout: mon reweight min bytes per osd = 10 2023-11-06T02:07:23.115 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:07:23.115 INFO:teuthology.orchestra.run.smithi049.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-06T02:07:23.115 INFO:teuthology.orchestra.run.smithi049.stdout: auth mon ticket ttl = 660 # 11m 2023-11-06T02:07:23.116 INFO:teuthology.orchestra.run.smithi049.stdout: auth service ticket ttl = 240 # 4m 2023-11-06T02:07:23.116 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:07:23.116 INFO:teuthology.orchestra.run.smithi049.stdout: # don't complain about insecure global_id in the test suite 2023-11-06T02:07:23.116 INFO:teuthology.orchestra.run.smithi049.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-06T02:07:23.116 INFO:teuthology.orchestra.run.smithi049.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-06T02:07:23.117 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:07:23.117 INFO:teuthology.orchestra.run.smithi049.stdout: # 1m isn't quite enough 2023-11-06T02:07:23.117 INFO:teuthology.orchestra.run.smithi049.stdout: mon_down_mkfs_grace = 2m 2023-11-06T02:07:23.117 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:07:23.117 INFO:teuthology.orchestra.run.smithi049.stdout: mon_warn_on_filestore_osds = false 2023-11-06T02:07:23.118 INFO:teuthology.orchestra.run.smithi049.stdout: mon op complaint time = 120 2023-11-06T02:07:23.118 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:07:23.118 INFO:teuthology.orchestra.run.smithi049.stdout:[client] 2023-11-06T02:07:23.118 INFO:teuthology.orchestra.run.smithi049.stdout: rgw cache enabled = true 2023-11-06T02:07:23.118 INFO:teuthology.orchestra.run.smithi049.stdout: rgw enable ops log = true 2023-11-06T02:07:23.119 INFO:teuthology.orchestra.run.smithi049.stdout: rgw enable usage log = true 2023-11-06T02:07:23.119 INFO:teuthology.orchestra.run.smithi049.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-06T02:07:23.119 INFO:teuthology.orchestra.run.smithi049.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-06T02:07:23.119 INFO:teuthology.orchestra.run.smithi049.stdout: client mount timeout = 600 2023-11-06T02:07:23.119 INFO:teuthology.orchestra.run.smithi049.stdout: debug client = 20 2023-11-06T02:07:23.120 INFO:teuthology.orchestra.run.smithi049.stdout: debug ms = 1 2023-11-06T02:07:23.120 INFO:teuthology.orchestra.run.smithi049.stdout: pid file = "" 2023-11-06T02:07:23.120 INFO:teuthology.orchestra.run.smithi049.stdout: rados mon op timeout = 900 2023-11-06T02:07:23.120 INFO:teuthology.orchestra.run.smithi049.stdout: rados osd op timeout = 900 2023-11-06T02:07:23.120 INFO:teuthology.orchestra.run.smithi049.stdout:[mon.a] 2023-11-06T02:07:23.121 INFO:teuthology.orchestra.run.smithi049.stdout:[mon.b] 2023-11-06T02:07:23.121 INFO:teuthology.orchestra.run.smithi049.stdout:[mon.c] 2023-11-06T02:07:23.121 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.a] 2023-11-06T02:07:23.121 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.c] 2023-11-06T02:07:23.121 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.b] 2023-11-06T02:07:23.122 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.d] 2023-11-06T02:07:23.122 INFO:teuthology.orchestra.run.smithi049.stdout:[mds] 2023-11-06T02:07:23.122 INFO:teuthology.orchestra.run.smithi049.stdout: debug mds = 20 2023-11-06T02:07:23.122 INFO:teuthology.orchestra.run.smithi049.stdout: debug mds balancer = 20 2023-11-06T02:07:23.122 INFO:teuthology.orchestra.run.smithi049.stdout: debug ms = 1 2023-11-06T02:07:23.123 INFO:teuthology.orchestra.run.smithi049.stdout: mds debug frag = True 2023-11-06T02:07:23.123 INFO:teuthology.orchestra.run.smithi049.stdout: mds debug scatterstat = True 2023-11-06T02:07:23.123 INFO:teuthology.orchestra.run.smithi049.stdout: mds op complaint time = 180 2023-11-06T02:07:23.123 INFO:teuthology.orchestra.run.smithi049.stdout: mds verify scatter = True 2023-11-06T02:07:23.123 INFO:teuthology.orchestra.run.smithi049.stdout: osd op complaint time = 180 2023-11-06T02:07:23.124 INFO:teuthology.orchestra.run.smithi049.stdout: rados mon op timeout = 900 2023-11-06T02:07:23.124 INFO:teuthology.orchestra.run.smithi049.stdout: rados osd op timeout = 900 2023-11-06T02:07:23.124 INFO:tasks.ceph:Creating admin key on mon.a... 2023-11-06T02:07:23.124 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-11-06T02:07:23.185 INFO:tasks.ceph:Copying monmap to all nodes... 2023-11-06T02:07:23.185 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-11-06T02:07:23.186 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-11-06T02:07:23.192 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-11-06T02:07:23.192 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-11-06T02:07:23.239 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi049.front.sepia.ceph.com 2023-11-06T02:07:23.240 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-11-06T02:07:23.240 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/etc/ceph/ceph.keyring 2023-11-06T02:07:23.241 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-06T02:07:23.305 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-11-06T02:07:23.305 DEBUG:teuthology.orchestra.run.smithi049:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-11-06T02:07:23.351 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi150.front.sepia.ceph.com 2023-11-06T02:07:23.352 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-11-06T02:07:23.352 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/ceph/ceph.keyring 2023-11-06T02:07:23.352 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-06T02:07:23.376 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-11-06T02:07:23.376 DEBUG:teuthology.orchestra.run.smithi150:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-11-06T02:07:23.424 INFO:tasks.ceph:Setting up mon nodes... 2023-11-06T02:07:23.424 INFO:tasks.ceph:Setting up mgr nodes... 2023-11-06T02:07:23.424 DEBUG:teuthology.orchestra.run.smithi049:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-11-06T02:07:23.468 INFO:teuthology.orchestra.run.smithi049.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-11-06T02:07:23.472 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-11-06T02:07:23.515 INFO:teuthology.orchestra.run.smithi150.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-11-06T02:07:23.526 INFO:tasks.ceph:Setting up mds nodes... 2023-11-06T02:07:23.526 DEBUG:teuthology.orchestra.run.smithi049:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-11-06T02:07:23.569 INFO:teuthology.orchestra.run.smithi049.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-11-06T02:07:23.572 DEBUG:teuthology.orchestra.run.smithi049:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-11-06T02:07:23.628 DEBUG:teuthology.orchestra.run.smithi049:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-11-06T02:07:23.718 INFO:teuthology.orchestra.run.smithi049.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-11-06T02:07:23.721 DEBUG:teuthology.orchestra.run.smithi049:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-11-06T02:07:23.777 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-11-06T02:07:23.821 INFO:teuthology.orchestra.run.smithi150.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-11-06T02:07:23.824 DEBUG:teuthology.orchestra.run.smithi150:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-11-06T02:07:23.879 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-11-06T02:07:23.958 INFO:teuthology.orchestra.run.smithi150.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-11-06T02:07:23.961 DEBUG:teuthology.orchestra.run.smithi150:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-11-06T02:07:24.015 INFO:tasks.ceph_client:Setting up client nodes... 2023-11-06T02:07:24.016 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-11-06T02:07:24.053 INFO:teuthology.orchestra.run.smithi049.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-11-06T02:07:24.065 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-11-06T02:07:24.142 INFO:teuthology.orchestra.run.smithi049.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-11-06T02:07:24.154 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-11-06T02:07:24.189 INFO:teuthology.orchestra.run.smithi150.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-11-06T02:07:24.200 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-11-06T02:07:24.275 INFO:teuthology.orchestra.run.smithi150.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-11-06T02:07:24.285 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-11-06T02:07:24.285 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi049.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi150.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-11-06T02:07:24.285 DEBUG:teuthology.orchestra.run.smithi049:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-11-06T02:07:24.299 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-06T02:07:24.300 INFO:tasks.ceph:role: osd.0 2023-11-06T02:07:24.300 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi049.front.sepia.ceph.com 2023-11-06T02:07:24.300 DEBUG:teuthology.orchestra.run.smithi049:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-11-06T02:07:24.355 INFO:teuthology.orchestra.run.smithi049.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-11-06T02:07:24.355 INFO:teuthology.orchestra.run.smithi049.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-06T02:07:24.356 INFO:teuthology.orchestra.run.smithi049.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-06T02:07:24.356 INFO:teuthology.orchestra.run.smithi049.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-06T02:07:24.356 INFO:teuthology.orchestra.run.smithi049.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-06T02:07:24.356 INFO:teuthology.orchestra.run.smithi049.stdout: = sunit=0 swidth=0 blks 2023-11-06T02:07:24.357 INFO:teuthology.orchestra.run.smithi049.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-06T02:07:24.357 INFO:teuthology.orchestra.run.smithi049.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-06T02:07:24.357 INFO:teuthology.orchestra.run.smithi049.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-06T02:07:24.357 INFO:teuthology.orchestra.run.smithi049.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-06T02:07:24.379 INFO:teuthology.orchestra.run.smithi049.stdout:Discarding blocks...Done. 2023-11-06T02:07:24.381 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi049.front.sepia.ceph.com -o noatime 2023-11-06T02:07:24.381 DEBUG:teuthology.orchestra.run.smithi049:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-11-06T02:07:24.438 DEBUG:teuthology.orchestra.run.smithi049:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-11-06T02:07:24.487 INFO:teuthology.orchestra.run.smithi049.stderr:sudo: /sbin/restorecon: command not found 2023-11-06T02:07:24.488 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T02:07:24.489 DEBUG:teuthology.orchestra.run.smithi049:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-11-06T02:07:24.544 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-06T02:07:24.544 INFO:tasks.ceph:role: osd.1 2023-11-06T02:07:24.545 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi049.front.sepia.ceph.com 2023-11-06T02:07:24.545 DEBUG:teuthology.orchestra.run.smithi049:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-11-06T02:07:24.599 INFO:teuthology.orchestra.run.smithi049.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-11-06T02:07:24.600 INFO:teuthology.orchestra.run.smithi049.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-06T02:07:24.600 INFO:teuthology.orchestra.run.smithi049.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-06T02:07:24.600 INFO:teuthology.orchestra.run.smithi049.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-06T02:07:24.600 INFO:teuthology.orchestra.run.smithi049.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-06T02:07:24.601 INFO:teuthology.orchestra.run.smithi049.stdout: = sunit=0 swidth=0 blks 2023-11-06T02:07:24.601 INFO:teuthology.orchestra.run.smithi049.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-06T02:07:24.601 INFO:teuthology.orchestra.run.smithi049.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-06T02:07:24.601 INFO:teuthology.orchestra.run.smithi049.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-06T02:07:24.602 INFO:teuthology.orchestra.run.smithi049.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-06T02:07:24.624 INFO:teuthology.orchestra.run.smithi049.stdout:Discarding blocks...Done. 2023-11-06T02:07:24.626 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi049.front.sepia.ceph.com -o noatime 2023-11-06T02:07:24.626 DEBUG:teuthology.orchestra.run.smithi049:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-11-06T02:07:24.688 DEBUG:teuthology.orchestra.run.smithi049:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-11-06T02:07:24.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T02:07:24.740 INFO:teuthology.orchestra.run.smithi049.stderr:sudo: /sbin/restorecon: command not found 2023-11-06T02:07:24.740 DEBUG:teuthology.orchestra.run.smithi049:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-11-06T02:07:24.794 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-06T02:07:24.794 INFO:tasks.ceph:role: osd.2 2023-11-06T02:07:24.794 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi049.front.sepia.ceph.com 2023-11-06T02:07:24.795 DEBUG:teuthology.orchestra.run.smithi049:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-11-06T02:07:24.846 INFO:teuthology.orchestra.run.smithi049.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-11-06T02:07:24.846 INFO:teuthology.orchestra.run.smithi049.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-06T02:07:24.846 INFO:teuthology.orchestra.run.smithi049.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-06T02:07:24.846 INFO:teuthology.orchestra.run.smithi049.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-06T02:07:24.847 INFO:teuthology.orchestra.run.smithi049.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-06T02:07:24.847 INFO:teuthology.orchestra.run.smithi049.stdout: = sunit=0 swidth=0 blks 2023-11-06T02:07:24.847 INFO:teuthology.orchestra.run.smithi049.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-06T02:07:24.847 INFO:teuthology.orchestra.run.smithi049.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-06T02:07:24.847 INFO:teuthology.orchestra.run.smithi049.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-06T02:07:24.848 INFO:teuthology.orchestra.run.smithi049.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-06T02:07:24.869 INFO:teuthology.orchestra.run.smithi049.stdout:Discarding blocks...Done. 2023-11-06T02:07:24.871 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi049.front.sepia.ceph.com -o noatime 2023-11-06T02:07:24.872 DEBUG:teuthology.orchestra.run.smithi049:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-11-06T02:07:24.931 DEBUG:teuthology.orchestra.run.smithi049:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-11-06T02:07:24.985 INFO:teuthology.orchestra.run.smithi049.stderr:sudo: /sbin/restorecon: command not found 2023-11-06T02:07:24.986 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T02:07:24.987 DEBUG:teuthology.orchestra.run.smithi049:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-11-06T02:07:25.043 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-06T02:07:25.044 INFO:tasks.ceph:role: osd.3 2023-11-06T02:07:25.044 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi049.front.sepia.ceph.com 2023-11-06T02:07:25.044 DEBUG:teuthology.orchestra.run.smithi049:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-11-06T02:07:25.099 INFO:teuthology.orchestra.run.smithi049.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-11-06T02:07:25.100 INFO:teuthology.orchestra.run.smithi049.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-06T02:07:25.100 INFO:teuthology.orchestra.run.smithi049.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-06T02:07:25.100 INFO:teuthology.orchestra.run.smithi049.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-06T02:07:25.100 INFO:teuthology.orchestra.run.smithi049.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-06T02:07:25.100 INFO:teuthology.orchestra.run.smithi049.stdout: = sunit=0 swidth=0 blks 2023-11-06T02:07:25.101 INFO:teuthology.orchestra.run.smithi049.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-06T02:07:25.101 INFO:teuthology.orchestra.run.smithi049.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-06T02:07:25.101 INFO:teuthology.orchestra.run.smithi049.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-06T02:07:25.101 INFO:teuthology.orchestra.run.smithi049.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-06T02:07:25.123 INFO:teuthology.orchestra.run.smithi049.stdout:Discarding blocks...Done. 2023-11-06T02:07:25.125 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi049.front.sepia.ceph.com -o noatime 2023-11-06T02:07:25.126 DEBUG:teuthology.orchestra.run.smithi049:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-11-06T02:07:25.182 DEBUG:teuthology.orchestra.run.smithi049:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-11-06T02:07:25.236 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T02:07:25.237 INFO:teuthology.orchestra.run.smithi049.stderr:sudo: /sbin/restorecon: command not found 2023-11-06T02:07:25.237 DEBUG:teuthology.orchestra.run.smithi049:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-06T02:07:25.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:25.354+0000 7efce2bc0640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-11-06T02:07:25.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:25.354+0000 7efce2bc0640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-11-06T02:07:25.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:25.354+0000 7efce2bc0640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-06T02:07:25.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:25.354+0000 7efce2bc0640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-06T02:07:25.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:25.354+0000 7efce2bc0640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-06T02:07:25.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:25.354+0000 7efce2bc0640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-11-06T02:07:28.374 DEBUG:teuthology.orchestra.run.smithi049:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-11-06T02:07:28.388 DEBUG:teuthology.orchestra.run.smithi049:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-06T02:07:28.472 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:28.478+0000 7effbf798640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-11-06T02:07:28.472 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:28.478+0000 7effbf798640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-11-06T02:07:28.473 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:28.478+0000 7effbf798640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-06T02:07:28.473 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:28.478+0000 7effbf798640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-06T02:07:28.473 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:28.478+0000 7effbf798640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-06T02:07:28.474 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:28.478+0000 7effbf798640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-11-06T02:07:31.470 DEBUG:teuthology.orchestra.run.smithi049:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-11-06T02:07:31.523 DEBUG:teuthology.orchestra.run.smithi049:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-06T02:07:31.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:31.614+0000 7fa90e883640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-11-06T02:07:31.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:31.614+0000 7fa90e883640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-11-06T02:07:31.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:31.614+0000 7fa90e883640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-06T02:07:31.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:31.614+0000 7fa90e883640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-06T02:07:31.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:31.614+0000 7fa90e883640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-06T02:07:31.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:31.614+0000 7fa90e883640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-11-06T02:07:34.603 DEBUG:teuthology.orchestra.run.smithi049:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-11-06T02:07:34.664 DEBUG:teuthology.orchestra.run.smithi049:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-06T02:07:34.752 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:34.758+0000 7f49ebdd7640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-11-06T02:07:34.753 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:34.758+0000 7f49ebdd7640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-11-06T02:07:34.753 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:34.758+0000 7f49ebdd7640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-06T02:07:34.753 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:34.758+0000 7f49ebdd7640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-06T02:07:34.753 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:34.758+0000 7f49ebdd7640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-06T02:07:34.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:34.758+0000 7f49ebdd7640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-11-06T02:07:37.762 DEBUG:teuthology.orchestra.run.smithi049:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-11-06T02:07:37.824 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-11-06T02:07:37.838 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-06T02:07:37.838 INFO:tasks.ceph:role: osd.4 2023-11-06T02:07:37.839 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi150.front.sepia.ceph.com 2023-11-06T02:07:37.839 DEBUG:teuthology.orchestra.run.smithi150:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-11-06T02:07:37.899 INFO:teuthology.orchestra.run.smithi150.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-11-06T02:07:37.899 INFO:teuthology.orchestra.run.smithi150.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-06T02:07:37.900 INFO:teuthology.orchestra.run.smithi150.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-06T02:07:37.900 INFO:teuthology.orchestra.run.smithi150.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-06T02:07:37.900 INFO:teuthology.orchestra.run.smithi150.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-06T02:07:37.900 INFO:teuthology.orchestra.run.smithi150.stdout: = sunit=0 swidth=0 blks 2023-11-06T02:07:37.900 INFO:teuthology.orchestra.run.smithi150.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-06T02:07:37.901 INFO:teuthology.orchestra.run.smithi150.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-06T02:07:37.901 INFO:teuthology.orchestra.run.smithi150.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-06T02:07:37.901 INFO:teuthology.orchestra.run.smithi150.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-06T02:07:38.386 INFO:teuthology.orchestra.run.smithi150.stdout:Discarding blocks...Done. 2023-11-06T02:07:38.388 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi150.front.sepia.ceph.com -o noatime 2023-11-06T02:07:38.388 DEBUG:teuthology.orchestra.run.smithi150:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-11-06T02:07:38.555 DEBUG:teuthology.orchestra.run.smithi150:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-11-06T02:07:38.566 INFO:teuthology.orchestra.run.smithi150.stderr:sudo: /sbin/restorecon: command not found 2023-11-06T02:07:38.567 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T02:07:38.568 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-11-06T02:07:38.623 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-06T02:07:38.623 INFO:tasks.ceph:role: osd.5 2023-11-06T02:07:38.624 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi150.front.sepia.ceph.com 2023-11-06T02:07:38.624 DEBUG:teuthology.orchestra.run.smithi150:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-11-06T02:07:38.687 INFO:teuthology.orchestra.run.smithi150.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-11-06T02:07:38.688 INFO:teuthology.orchestra.run.smithi150.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-06T02:07:38.688 INFO:teuthology.orchestra.run.smithi150.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-06T02:07:38.688 INFO:teuthology.orchestra.run.smithi150.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-06T02:07:38.688 INFO:teuthology.orchestra.run.smithi150.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-06T02:07:38.688 INFO:teuthology.orchestra.run.smithi150.stdout: = sunit=0 swidth=0 blks 2023-11-06T02:07:38.689 INFO:teuthology.orchestra.run.smithi150.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-06T02:07:38.689 INFO:teuthology.orchestra.run.smithi150.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-06T02:07:38.689 INFO:teuthology.orchestra.run.smithi150.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-06T02:07:38.689 INFO:teuthology.orchestra.run.smithi150.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-06T02:07:39.124 INFO:teuthology.orchestra.run.smithi150.stdout:Discarding blocks...Done. 2023-11-06T02:07:39.126 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi150.front.sepia.ceph.com -o noatime 2023-11-06T02:07:39.126 DEBUG:teuthology.orchestra.run.smithi150:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-11-06T02:07:39.213 DEBUG:teuthology.orchestra.run.smithi150:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-11-06T02:07:39.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T02:07:39.223 INFO:teuthology.orchestra.run.smithi150.stderr:sudo: /sbin/restorecon: command not found 2023-11-06T02:07:39.223 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-11-06T02:07:39.277 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-06T02:07:39.277 INFO:tasks.ceph:role: osd.6 2023-11-06T02:07:39.277 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi150.front.sepia.ceph.com 2023-11-06T02:07:39.277 DEBUG:teuthology.orchestra.run.smithi150:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-11-06T02:07:39.340 INFO:teuthology.orchestra.run.smithi150.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-11-06T02:07:39.340 INFO:teuthology.orchestra.run.smithi150.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-06T02:07:39.340 INFO:teuthology.orchestra.run.smithi150.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-06T02:07:39.340 INFO:teuthology.orchestra.run.smithi150.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-06T02:07:39.340 INFO:teuthology.orchestra.run.smithi150.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-06T02:07:39.341 INFO:teuthology.orchestra.run.smithi150.stdout: = sunit=0 swidth=0 blks 2023-11-06T02:07:39.341 INFO:teuthology.orchestra.run.smithi150.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-06T02:07:39.341 INFO:teuthology.orchestra.run.smithi150.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-06T02:07:39.341 INFO:teuthology.orchestra.run.smithi150.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-06T02:07:39.341 INFO:teuthology.orchestra.run.smithi150.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-06T02:07:39.792 INFO:teuthology.orchestra.run.smithi150.stdout:Discarding blocks...Done. 2023-11-06T02:07:39.794 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi150.front.sepia.ceph.com -o noatime 2023-11-06T02:07:39.794 DEBUG:teuthology.orchestra.run.smithi150:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-11-06T02:07:39.882 DEBUG:teuthology.orchestra.run.smithi150:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-11-06T02:07:39.891 INFO:teuthology.orchestra.run.smithi150.stderr:sudo: /sbin/restorecon: command not found 2023-11-06T02:07:39.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T02:07:39.892 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-11-06T02:07:39.944 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-06T02:07:39.945 INFO:tasks.ceph:role: osd.7 2023-11-06T02:07:39.945 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi150.front.sepia.ceph.com 2023-11-06T02:07:39.945 DEBUG:teuthology.orchestra.run.smithi150:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-11-06T02:07:40.008 INFO:teuthology.orchestra.run.smithi150.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-11-06T02:07:40.008 INFO:teuthology.orchestra.run.smithi150.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-06T02:07:40.008 INFO:teuthology.orchestra.run.smithi150.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-06T02:07:40.008 INFO:teuthology.orchestra.run.smithi150.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-06T02:07:40.009 INFO:teuthology.orchestra.run.smithi150.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-06T02:07:40.009 INFO:teuthology.orchestra.run.smithi150.stdout: = sunit=0 swidth=0 blks 2023-11-06T02:07:40.009 INFO:teuthology.orchestra.run.smithi150.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-06T02:07:40.009 INFO:teuthology.orchestra.run.smithi150.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-06T02:07:40.009 INFO:teuthology.orchestra.run.smithi150.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-06T02:07:40.010 INFO:teuthology.orchestra.run.smithi150.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-06T02:07:40.489 INFO:teuthology.orchestra.run.smithi150.stdout:Discarding blocks...Done. 2023-11-06T02:07:40.491 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi150.front.sepia.ceph.com -o noatime 2023-11-06T02:07:40.491 DEBUG:teuthology.orchestra.run.smithi150:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-11-06T02:07:40.642 DEBUG:teuthology.orchestra.run.smithi150:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-11-06T02:07:40.654 INFO:teuthology.orchestra.run.smithi150.stderr:sudo: /sbin/restorecon: command not found 2023-11-06T02:07:40.655 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T02:07:40.655 DEBUG:teuthology.orchestra.run.smithi150:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-06T02:07:40.775 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:07:40.778+0000 7fce56e35640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-11-06T02:07:40.781 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:07:40.782+0000 7fce56e35640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-11-06T02:07:40.781 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:07:40.782+0000 7fce56e35640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-06T02:07:40.782 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:07:40.782+0000 7fce56e35640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-06T02:07:40.782 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:07:40.782+0000 7fce56e35640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-06T02:07:40.788 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:07:40.790+0000 7fce56e35640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-11-06T02:07:43.798 DEBUG:teuthology.orchestra.run.smithi150:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-11-06T02:07:43.811 DEBUG:teuthology.orchestra.run.smithi150:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-06T02:07:43.899 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:07:43.898+0000 7f8226cca640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-11-06T02:07:43.899 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:07:43.902+0000 7f8226cca640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-11-06T02:07:43.900 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:07:43.902+0000 7f8226cca640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-06T02:07:43.900 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:07:43.902+0000 7f8226cca640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-06T02:07:43.900 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:07:43.902+0000 7f8226cca640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-06T02:07:43.901 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:07:43.902+0000 7f8226cca640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-11-06T02:07:46.919 DEBUG:teuthology.orchestra.run.smithi150:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-11-06T02:07:46.976 DEBUG:teuthology.orchestra.run.smithi150:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-06T02:07:47.063 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:07:47.066+0000 7f1a963b7640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-11-06T02:07:47.064 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:07:47.066+0000 7f1a963b7640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-11-06T02:07:47.064 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:07:47.066+0000 7f1a963b7640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-06T02:07:47.064 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:07:47.066+0000 7f1a963b7640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-06T02:07:47.065 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:07:47.066+0000 7f1a963b7640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-06T02:07:47.065 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:07:47.066+0000 7f1a963b7640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-11-06T02:07:50.087 DEBUG:teuthology.orchestra.run.smithi150:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-11-06T02:07:50.142 DEBUG:teuthology.orchestra.run.smithi150:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-06T02:07:50.221 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:07:50.222+0000 7f233bbb0640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-11-06T02:07:50.221 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:07:50.222+0000 7f233bbb0640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-11-06T02:07:50.222 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:07:50.222+0000 7f233bbb0640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-06T02:07:50.222 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:07:50.222+0000 7f233bbb0640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-06T02:07:50.222 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:07:50.222+0000 7f233bbb0640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-06T02:07:50.223 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:07:50.222+0000 7f233bbb0640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-11-06T02:07:51.245 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T02:07:51.248 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T02:07:53.227 DEBUG:teuthology.orchestra.run.smithi150:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-11-06T02:07:53.241 INFO:tasks.ceph:Reading keys from all nodes... 2023-11-06T02:07:53.241 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-11-06T02:07:53.242 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-11-06T02:07:53.256 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-11-06T02:07:53.256 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-11-06T02:07:53.311 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-11-06T02:07:53.311 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-11-06T02:07:53.366 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-11-06T02:07:53.367 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-11-06T02:07:53.422 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-11-06T02:07:53.423 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-11-06T02:07:53.479 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-11-06T02:07:53.479 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-11-06T02:07:53.534 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-11-06T02:07:53.535 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-11-06T02:07:53.590 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-11-06T02:07:53.591 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-11-06T02:07:53.604 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-11-06T02:07:53.605 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-11-06T02:07:53.659 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-11-06T02:07:53.659 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-11-06T02:07:53.715 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-11-06T02:07:53.715 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-11-06T02:07:53.770 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-11-06T02:07:53.771 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-11-06T02:07:53.823 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-11-06T02:07:53.823 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-11-06T02:07:53.879 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-11-06T02:07:53.879 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-11-06T02:07:53.935 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-11-06T02:07:53.935 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-11-06T02:07:53.941 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-11-06T02:07:53.941 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-11-06T02:07:53.987 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-11-06T02:07:53.987 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-11-06T02:07:53.992 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-11-06T02:07:53.993 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-11-06T02:07:54.039 INFO:tasks.ceph:Adding keys to all mons... 2023-11-06T02:07:54.040 DEBUG:teuthology.orchestra.run.smithi049:> sudo tee -a /etc/ceph/ceph.keyring 2023-11-06T02:07:54.042 DEBUG:teuthology.orchestra.run.smithi150:> sudo tee -a /etc/ceph/ceph.keyring 2023-11-06T02:07:54.085 INFO:teuthology.orchestra.run.smithi049.stdout:[mgr.y] 2023-11-06T02:07:54.086 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQBbSkhln1JuHBAAymHRkniPpsI6pf9L2C2y+g== 2023-11-06T02:07:54.086 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.a] 2023-11-06T02:07:54.086 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQBbSkhlbRFwIhAApA6xaqdIIeKDsOsNhyWCaA== 2023-11-06T02:07:54.087 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.c] 2023-11-06T02:07:54.087 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQBbSkhlMoZHKxAATV62bAItdXt3ShpsT4oVbA== 2023-11-06T02:07:54.087 INFO:teuthology.orchestra.run.smithi049.stdout:[osd.0] 2023-11-06T02:07:54.087 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQBdSkhlxuQtFRAAPttjVsYpkwexC5x2Pu2b+Q== 2023-11-06T02:07:54.087 INFO:teuthology.orchestra.run.smithi049.stdout:[osd.1] 2023-11-06T02:07:54.088 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQBgSkhlCGeiHBAA8SofNV6C1+RQNajUsewVpw== 2023-11-06T02:07:54.088 INFO:teuthology.orchestra.run.smithi049.stdout:[osd.2] 2023-11-06T02:07:54.088 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQBjSkhlBju+JBAAPJeIFngKDoamUvEjv1h8tw== 2023-11-06T02:07:54.088 INFO:teuthology.orchestra.run.smithi049.stdout:[osd.3] 2023-11-06T02:07:54.088 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQBmSkhlMclTLRAABnbmQA9j3TLn40H1wYFx5g== 2023-11-06T02:07:54.089 INFO:teuthology.orchestra.run.smithi049.stdout:[mgr.x] 2023-11-06T02:07:54.089 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQBbSkhlpJ4iHxAAGEz4XS5fILy7/NZbAenPFw== 2023-11-06T02:07:54.089 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.b] 2023-11-06T02:07:54.089 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQBbSkhl6QhcMRAAgmOxZWghmuu11F0Gfut3sQ== 2023-11-06T02:07:54.089 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.d] 2023-11-06T02:07:54.089 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQBbSkhl376GORAAS2NIQJMzg6Es0JCYPak6fw== 2023-11-06T02:07:54.090 INFO:teuthology.orchestra.run.smithi049.stdout:[osd.4] 2023-11-06T02:07:54.090 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQBsSkhl7fmmLhAArv62qbJ+StgYMKhCftM5HA== 2023-11-06T02:07:54.090 INFO:teuthology.orchestra.run.smithi049.stdout:[osd.5] 2023-11-06T02:07:54.090 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQBvSkhl5pAANhAArNFy9/YuEcXTdzM5XL46Xw== 2023-11-06T02:07:54.090 INFO:teuthology.orchestra.run.smithi049.stdout:[osd.6] 2023-11-06T02:07:54.091 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQBzSkhlgfs2BBAAeHXVJ/tFiFagugkynGakFw== 2023-11-06T02:07:54.091 INFO:teuthology.orchestra.run.smithi049.stdout:[osd.7] 2023-11-06T02:07:54.091 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQB2SkhlQbCYDRAAhE7vwbjJjqZDHfgDfdy1cA== 2023-11-06T02:07:54.091 INFO:teuthology.orchestra.run.smithi049.stdout:[client.0] 2023-11-06T02:07:54.091 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQBcSkhlY46mAxAALClTH91L6USom3nVss6iiQ== 2023-11-06T02:07:54.092 INFO:teuthology.orchestra.run.smithi049.stdout:[client.2] 2023-11-06T02:07:54.092 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQBcSkhlURf6CBAA9g2y/UMD4x42IT0fgSgIcg== 2023-11-06T02:07:54.092 INFO:teuthology.orchestra.run.smithi049.stdout:[client.1] 2023-11-06T02:07:54.092 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQBcSkhllvSyCxAAfcSiBhHHNgaIr9mhpY11fg== 2023-11-06T02:07:54.092 INFO:teuthology.orchestra.run.smithi049.stdout:[client.3] 2023-11-06T02:07:54.093 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQBcSkhlbjXXEBAA0xQGCY/avAWL18GLQtIjgg== 2023-11-06T02:07:54.094 INFO:teuthology.orchestra.run.smithi150.stdout:[mgr.y] 2023-11-06T02:07:54.094 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQBbSkhln1JuHBAAymHRkniPpsI6pf9L2C2y+g== 2023-11-06T02:07:54.094 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.a] 2023-11-06T02:07:54.095 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQBbSkhlbRFwIhAApA6xaqdIIeKDsOsNhyWCaA== 2023-11-06T02:07:54.095 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.c] 2023-11-06T02:07:54.095 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQBbSkhlMoZHKxAATV62bAItdXt3ShpsT4oVbA== 2023-11-06T02:07:54.095 INFO:teuthology.orchestra.run.smithi150.stdout:[osd.0] 2023-11-06T02:07:54.095 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQBdSkhlxuQtFRAAPttjVsYpkwexC5x2Pu2b+Q== 2023-11-06T02:07:54.096 INFO:teuthology.orchestra.run.smithi150.stdout:[osd.1] 2023-11-06T02:07:54.096 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQBgSkhlCGeiHBAA8SofNV6C1+RQNajUsewVpw== 2023-11-06T02:07:54.096 INFO:teuthology.orchestra.run.smithi150.stdout:[osd.2] 2023-11-06T02:07:54.096 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQBjSkhlBju+JBAAPJeIFngKDoamUvEjv1h8tw== 2023-11-06T02:07:54.096 INFO:teuthology.orchestra.run.smithi150.stdout:[osd.3] 2023-11-06T02:07:54.097 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQBmSkhlMclTLRAABnbmQA9j3TLn40H1wYFx5g== 2023-11-06T02:07:54.097 INFO:teuthology.orchestra.run.smithi150.stdout:[mgr.x] 2023-11-06T02:07:54.097 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQBbSkhlpJ4iHxAAGEz4XS5fILy7/NZbAenPFw== 2023-11-06T02:07:54.097 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.b] 2023-11-06T02:07:54.097 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQBbSkhl6QhcMRAAgmOxZWghmuu11F0Gfut3sQ== 2023-11-06T02:07:54.098 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.d] 2023-11-06T02:07:54.098 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQBbSkhl376GORAAS2NIQJMzg6Es0JCYPak6fw== 2023-11-06T02:07:54.098 INFO:teuthology.orchestra.run.smithi150.stdout:[osd.4] 2023-11-06T02:07:54.098 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQBsSkhl7fmmLhAArv62qbJ+StgYMKhCftM5HA== 2023-11-06T02:07:54.098 INFO:teuthology.orchestra.run.smithi150.stdout:[osd.5] 2023-11-06T02:07:54.099 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQBvSkhl5pAANhAArNFy9/YuEcXTdzM5XL46Xw== 2023-11-06T02:07:54.099 INFO:teuthology.orchestra.run.smithi150.stdout:[osd.6] 2023-11-06T02:07:54.099 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQBzSkhlgfs2BBAAeHXVJ/tFiFagugkynGakFw== 2023-11-06T02:07:54.099 INFO:teuthology.orchestra.run.smithi150.stdout:[osd.7] 2023-11-06T02:07:54.099 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQB2SkhlQbCYDRAAhE7vwbjJjqZDHfgDfdy1cA== 2023-11-06T02:07:54.099 INFO:teuthology.orchestra.run.smithi150.stdout:[client.0] 2023-11-06T02:07:54.100 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQBcSkhlY46mAxAALClTH91L6USom3nVss6iiQ== 2023-11-06T02:07:54.100 INFO:teuthology.orchestra.run.smithi150.stdout:[client.2] 2023-11-06T02:07:54.100 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQBcSkhlURf6CBAA9g2y/UMD4x42IT0fgSgIcg== 2023-11-06T02:07:54.100 INFO:teuthology.orchestra.run.smithi150.stdout:[client.1] 2023-11-06T02:07:54.100 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQBcSkhllvSyCxAAfcSiBhHHNgaIr9mhpY11fg== 2023-11-06T02:07:54.100 INFO:teuthology.orchestra.run.smithi150.stdout:[client.3] 2023-11-06T02:07:54.101 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQBcSkhlbjXXEBAA0xQGCY/avAWL18GLQtIjgg== 2023-11-06T02:07:54.102 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-06T02:07:54.140 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-06T02:07:54.188 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-06T02:07:54.191 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-06T02:07:54.236 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-06T02:07:54.239 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-06T02:07:54.280 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-06T02:07:54.324 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-06T02:07:54.362 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-06T02:07:54.408 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-06T02:07:54.448 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-06T02:07:54.492 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-06T02:07:54.531 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-06T02:07:54.580 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-06T02:07:54.620 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-06T02:07:54.664 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-06T02:07:54.709 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-06T02:07:54.752 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-06T02:07:54.801 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-06T02:07:54.803 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-06T02:07:54.846 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-06T02:07:54.891 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-06T02:07:54.932 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-06T02:07:54.975 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-06T02:07:55.018 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-06T02:07:55.059 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-06T02:07:55.104 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-06T02:07:55.144 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-06T02:07:55.192 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-06T02:07:55.228 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-06T02:07:55.275 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-06T02:07:55.277 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-06T02:07:55.366 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-06T02:07:55.369 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-06T02:07:55.416 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-06T02:07:55.455 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-06T02:07:55.498 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-11-06T02:07:55.498 DEBUG:teuthology.orchestra.run.smithi049:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-11-06T02:07:55.554 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-06T02:07:55.654 DEBUG:teuthology.orchestra.run.smithi049:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-11-06T02:07:55.666 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-11-06T02:07:55.680 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-06T02:07:55.779 DEBUG:teuthology.orchestra.run.smithi150:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-11-06T02:07:55.790 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-11-06T02:07:55.840 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-06T02:07:55.935 DEBUG:teuthology.orchestra.run.smithi150:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-11-06T02:07:55.946 DEBUG:teuthology.orchestra.run.smithi049:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-11-06T02:07:55.949 DEBUG:teuthology.orchestra.run.smithi150:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-11-06T02:07:55.991 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-11-06T02:07:55.991 INFO:tasks.ceph.mon.a:Restarting daemon 2023-11-06T02:07:55.992 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-11-06T02:07:55.995 INFO:tasks.ceph.mon.a:Started 2023-11-06T02:07:55.996 INFO:tasks.ceph.mon.b:Restarting daemon 2023-11-06T02:07:55.996 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-11-06T02:07:56.036 INFO:tasks.ceph.mon.b:Started 2023-11-06T02:07:56.037 INFO:tasks.ceph.mon.c:Restarting daemon 2023-11-06T02:07:56.037 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-11-06T02:07:56.039 INFO:tasks.ceph.mon.c:Started 2023-11-06T02:07:56.039 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-11-06T02:07:56.040 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-11-06T02:07:56.040 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-11-06T02:07:56.043 INFO:tasks.ceph.mgr.y:Started 2023-11-06T02:07:56.043 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-11-06T02:07:56.043 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-11-06T02:07:56.046 INFO:tasks.ceph.mgr.x:Started 2023-11-06T02:07:56.046 INFO:tasks.ceph:Setting crush tunables to default 2023-11-06T02:07:56.046 DEBUG:teuthology.orchestra.run.smithi049:> sudo ceph --cluster ceph osd crush tunables default 2023-11-06T02:07:56.163 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:56.170+0000 7f1e411e7640 1 Processor -- start 2023-11-06T02:07:56.164 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:56.170+0000 7f1e411e7640 1 -- start start 2023-11-06T02:07:56.165 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:56.170+0000 7f1e411e7640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e3c15cbb0 0x7f1e3c0d1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:07:56.165 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:56.170+0000 7f1e411e7640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f1e3c0d24e0 0x7f1e3c0d2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:07:56.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:56.170+0000 7f1e411e7640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e3c0d3400 con 0x7f1e3c0d2fa0 2023-11-06T02:07:56.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:56.170+0000 7f1e411e7640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e3c15cf90 con 0x7f1e3c0d24e0 2023-11-06T02:07:56.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:56.170+0000 7f1e411e7640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e3c0d0b80 con 0x7f1e3c15cbb0 2023-11-06T02:07:56.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:56.170+0000 7f1e3ad9d640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e3c15cbb0 0x7f1e3c0d1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:07:56.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:56.170+0000 7f1e3b59e640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f1e3c0d2fa0 0x7f1e3c0b4550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.49:48564/0 (socket says 172.21.15.49:48564) 2023-11-06T02:07:56.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:56.170+0000 7f1e3b59e640 1 -- 172.21.15.49:0/135760884 learned_addr learned my addr 172.21.15.49:0/135760884 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:07:56.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:56.170+0000 7f1e3a59c640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f1e3c0d24e0 0x7f1e3c0d2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:07:56.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:56.170+0000 7f1e3ad9d640 1 -- 172.21.15.49:0/135760884 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e3c15cbb0 msgr2=0x7f1e3c0d1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-11-06T02:07:56.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:56.170+0000 7f1e3ad9d640 1 -- 172.21.15.49:0/135760884 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e3c15cbb0 msgr2=0x7f1e3c0d1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-06T02:07:56.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:56.170+0000 7f1e3ad9d640 1 --2- 172.21.15.49:0/135760884 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e3c15cbb0 0x7f1e3c0d1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-06T02:07:56.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:56.170+0000 7f1e3a59c640 1 -- 172.21.15.49:0/135760884 >> v2:172.21.15.150:3300/0 conn(0x7f1e3c0d24e0 msgr2=0x7f1e3c0d2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-06T02:07:56.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:56.170+0000 7f1e3a59c640 1 -- 172.21.15.49:0/135760884 >> v2:172.21.15.150:3300/0 conn(0x7f1e3c0d24e0 msgr2=0x7f1e3c0d2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-06T02:07:56.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:56.170+0000 7f1e3a59c640 1 --2- 172.21.15.49:0/135760884 >> v2:172.21.15.150:3300/0 conn(0x7f1e3c0d24e0 0x7f1e3c0d2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-06T02:07:56.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:56.170+0000 7f1e3a59c640 1 --2- 172.21.15.49:0/135760884 >> v2:172.21.15.150:3300/0 conn(0x7f1e3c0d24e0 0x7f1e3c0d2a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-06T02:07:56.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:56.170+0000 7f1e3ad9d640 1 --2- 172.21.15.49:0/135760884 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e3c15cbb0 0x7f1e3c0d1fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-06T02:07:56.367 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:56.374+0000 7f1e3ad9d640 1 --2- 172.21.15.49:0/135760884 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e3c15cbb0 0x7f1e3c0d1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:07:56.367 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:56.374+0000 7f1e3a59c640 1 --2- 172.21.15.49:0/135760884 >> v2:172.21.15.150:3300/0 conn(0x7f1e3c0d24e0 0x7f1e3c0d2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:07:56.368 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:56.374+0000 7f1e3ad9d640 1 -- 172.21.15.49:0/135760884 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e3c15cbb0 msgr2=0x7f1e3c0d1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-06T02:07:56.368 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:56.374+0000 7f1e3ad9d640 1 -- 172.21.15.49:0/135760884 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e3c15cbb0 msgr2=0x7f1e3c0d1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-06T02:07:56.368 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:56.374+0000 7f1e3ad9d640 1 --2- 172.21.15.49:0/135760884 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e3c15cbb0 0x7f1e3c0d1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-06T02:07:56.368 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:56.374+0000 7f1e3ad9d640 1 --2- 172.21.15.49:0/135760884 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e3c15cbb0 0x7f1e3c0d1fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-06T02:07:56.368 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:56.374+0000 7f1e3a59c640 1 -- 172.21.15.49:0/135760884 >> v2:172.21.15.150:3300/0 conn(0x7f1e3c0d24e0 msgr2=0x7f1e3c0d2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-11-06T02:07:56.369 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:56.374+0000 7f1e3a59c640 1 -- 172.21.15.49:0/135760884 >> v2:172.21.15.150:3300/0 conn(0x7f1e3c0d24e0 msgr2=0x7f1e3c0d2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-06T02:07:56.369 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:56.374+0000 7f1e3a59c640 1 --2- 172.21.15.49:0/135760884 >> v2:172.21.15.150:3300/0 conn(0x7f1e3c0d24e0 0x7f1e3c0d2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-06T02:07:56.369 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:56.374+0000 7f1e3a59c640 1 --2- 172.21.15.49:0/135760884 >> v2:172.21.15.150:3300/0 conn(0x7f1e3c0d24e0 0x7f1e3c0d2a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-06T02:07:56.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:56.778+0000 7f1e3ad9d640 1 --2- 172.21.15.49:0/135760884 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e3c15cbb0 0x7f1e3c0d1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:07:56.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:56.778+0000 7f1e3a59c640 1 --2- 172.21.15.49:0/135760884 >> v2:172.21.15.150:3300/0 conn(0x7f1e3c0d24e0 0x7f1e3c0d2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:07:56.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:56.778+0000 7f1e3ad9d640 1 -- 172.21.15.49:0/135760884 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e3c15cbb0 msgr2=0x7f1e3c0d1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-06T02:07:56.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:56.778+0000 7f1e3ad9d640 1 -- 172.21.15.49:0/135760884 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e3c15cbb0 msgr2=0x7f1e3c0d1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-06T02:07:56.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:56.778+0000 7f1e3ad9d640 1 --2- 172.21.15.49:0/135760884 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e3c15cbb0 0x7f1e3c0d1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-06T02:07:56.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:56.778+0000 7f1e3ad9d640 1 --2- 172.21.15.49:0/135760884 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e3c15cbb0 0x7f1e3c0d1fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-06T02:07:56.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:56.778+0000 7f1e3a59c640 1 -- 172.21.15.49:0/135760884 >> v2:172.21.15.150:3300/0 conn(0x7f1e3c0d24e0 msgr2=0x7f1e3c0d2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-11-06T02:07:56.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:56.778+0000 7f1e3a59c640 1 -- 172.21.15.49:0/135760884 >> v2:172.21.15.150:3300/0 conn(0x7f1e3c0d24e0 msgr2=0x7f1e3c0d2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-06T02:07:56.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:56.778+0000 7f1e3a59c640 1 --2- 172.21.15.49:0/135760884 >> v2:172.21.15.150:3300/0 conn(0x7f1e3c0d24e0 0x7f1e3c0d2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-06T02:07:56.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:56.778+0000 7f1e3a59c640 1 --2- 172.21.15.49:0/135760884 >> v2:172.21.15.150:3300/0 conn(0x7f1e3c0d24e0 0x7f1e3c0d2a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-06T02:07:57.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:57.582+0000 7f1e3ad9d640 1 --2- 172.21.15.49:0/135760884 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e3c15cbb0 0x7f1e3c0d1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:07:57.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:57.582+0000 7f1e3a59c640 1 --2- 172.21.15.49:0/135760884 >> v2:172.21.15.150:3300/0 conn(0x7f1e3c0d24e0 0x7f1e3c0d2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:07:57.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:57.582+0000 7f1e3ad9d640 1 -- 172.21.15.49:0/135760884 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e3c15cbb0 msgr2=0x7f1e3c0d1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-06T02:07:57.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:57.582+0000 7f1e3ad9d640 1 -- 172.21.15.49:0/135760884 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e3c15cbb0 msgr2=0x7f1e3c0d1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-06T02:07:57.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:57.582+0000 7f1e3ad9d640 1 --2- 172.21.15.49:0/135760884 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e3c15cbb0 0x7f1e3c0d1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-06T02:07:57.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:57.582+0000 7f1e3a59c640 1 -- 172.21.15.49:0/135760884 >> v2:172.21.15.150:3300/0 conn(0x7f1e3c0d24e0 msgr2=0x7f1e3c0d2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-11-06T02:07:57.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:57.582+0000 7f1e3a59c640 1 -- 172.21.15.49:0/135760884 >> v2:172.21.15.150:3300/0 conn(0x7f1e3c0d24e0 msgr2=0x7f1e3c0d2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-06T02:07:57.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:57.582+0000 7f1e3a59c640 1 --2- 172.21.15.49:0/135760884 >> v2:172.21.15.150:3300/0 conn(0x7f1e3c0d24e0 0x7f1e3c0d2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-06T02:07:57.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:57.582+0000 7f1e3ad9d640 1 --2- 172.21.15.49:0/135760884 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e3c15cbb0 0x7f1e3c0d1fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-06T02:07:57.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:57.582+0000 7f1e3a59c640 1 --2- 172.21.15.49:0/135760884 >> v2:172.21.15.150:3300/0 conn(0x7f1e3c0d24e0 0x7f1e3c0d2a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-06T02:07:59.165 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:59.170+0000 7f1e38d99640 1 -- 172.21.15.49:0/135760884 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e3c15cbb0 msgr2=0x7f1e3c0d1fa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-06T02:07:59.165 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:59.170+0000 7f1e38d99640 1 --2- 172.21.15.49:0/135760884 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e3c15cbb0 0x7f1e3c0d1fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:07:59.165 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:59.170+0000 7f1e38d99640 1 -- 172.21.15.49:0/135760884 >> v2:172.21.15.150:3300/0 conn(0x7f1e3c0d24e0 msgr2=0x7f1e3c0d2a60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-06T02:07:59.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:59.170+0000 7f1e38d99640 1 --2- 172.21.15.49:0/135760884 >> v2:172.21.15.150:3300/0 conn(0x7f1e3c0d24e0 0x7f1e3c0d2a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:07:59.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:59.170+0000 7f1e38d99640 1 -- 172.21.15.49:0/135760884 >> v1:172.21.15.150:6789/0 conn(0x7f1e3c0d2fa0 legacy=0x7f1e3c0b4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:07:59.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:59.170+0000 7f1e38d99640 1 --2- 172.21.15.49:0/135760884 >> v2:172.21.15.49:3300/0 conn(0x7f1e280036f0 0x7f1e28005b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:07:59.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:59.170+0000 7f1e38d99640 1 -- 172.21.15.49:0/135760884 --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e3c0d0e60 con 0x7f1e28006210 2023-11-06T02:07:59.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:59.170+0000 7f1e38d99640 1 -- 172.21.15.49:0/135760884 --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e3c15cf90 con 0x7f1e280036f0 2023-11-06T02:07:59.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:59.170+0000 7f1e3ad9d640 1 --2- 172.21.15.49:0/135760884 >> v2:172.21.15.49:3300/0 conn(0x7f1e280036f0 0x7f1e28005b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:07:59.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:59.170+0000 7f1e3ad9d640 1 -- 172.21.15.49:0/135760884 >> v2:172.21.15.49:3300/0 conn(0x7f1e280036f0 msgr2=0x7f1e28005b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-06T02:07:59.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:59.170+0000 7f1e3ad9d640 1 -- 172.21.15.49:0/135760884 >> v2:172.21.15.49:3300/0 conn(0x7f1e280036f0 msgr2=0x7f1e28005b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-06T02:07:59.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:59.170+0000 7f1e3ad9d640 1 --2- 172.21.15.49:0/135760884 >> v2:172.21.15.49:3300/0 conn(0x7f1e280036f0 0x7f1e28005b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-06T02:07:59.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:59.170+0000 7f1e3ad9d640 1 --2- 172.21.15.49:0/135760884 >> v2:172.21.15.49:3300/0 conn(0x7f1e280036f0 0x7f1e28005b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-06T02:07:59.367 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:59.374+0000 7f1e3ad9d640 1 --2- 172.21.15.49:0/135760884 >> v2:172.21.15.49:3300/0 conn(0x7f1e280036f0 0x7f1e28005b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:07:59.367 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:59.374+0000 7f1e3ad9d640 1 -- 172.21.15.49:0/135760884 >> v2:172.21.15.49:3300/0 conn(0x7f1e280036f0 msgr2=0x7f1e28005b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-06T02:07:59.367 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:59.374+0000 7f1e3ad9d640 1 -- 172.21.15.49:0/135760884 >> v2:172.21.15.49:3300/0 conn(0x7f1e280036f0 msgr2=0x7f1e28005b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-06T02:07:59.367 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:59.374+0000 7f1e3ad9d640 1 --2- 172.21.15.49:0/135760884 >> v2:172.21.15.49:3300/0 conn(0x7f1e280036f0 0x7f1e28005b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-06T02:07:59.368 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:59.374+0000 7f1e3ad9d640 1 --2- 172.21.15.49:0/135760884 >> v2:172.21.15.49:3300/0 conn(0x7f1e280036f0 0x7f1e28005b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-06T02:07:59.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:59.778+0000 7f1e3ad9d640 1 --2- 172.21.15.49:0/135760884 >> v2:172.21.15.49:3300/0 conn(0x7f1e280036f0 0x7f1e28005b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:07:59.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:59.778+0000 7f1e3ad9d640 1 -- 172.21.15.49:0/135760884 >> v2:172.21.15.49:3300/0 conn(0x7f1e280036f0 msgr2=0x7f1e28005b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-06T02:07:59.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:59.778+0000 7f1e3ad9d640 1 -- 172.21.15.49:0/135760884 >> v2:172.21.15.49:3300/0 conn(0x7f1e280036f0 msgr2=0x7f1e28005b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-06T02:07:59.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:59.778+0000 7f1e3ad9d640 1 --2- 172.21.15.49:0/135760884 >> v2:172.21.15.49:3300/0 conn(0x7f1e280036f0 0x7f1e28005b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-06T02:07:59.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:07:59.778+0000 7f1e3ad9d640 1 --2- 172.21.15.49:0/135760884 >> v2:172.21.15.49:3300/0 conn(0x7f1e280036f0 0x7f1e28005b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-06T02:08:00.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:00.582+0000 7f1e3ad9d640 1 --2- 172.21.15.49:0/135760884 >> v2:172.21.15.49:3300/0 conn(0x7f1e280036f0 0x7f1e28005b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:00.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:00.582+0000 7f1e3ad9d640 1 -- 172.21.15.49:0/135760884 >> v2:172.21.15.49:3300/0 conn(0x7f1e280036f0 msgr2=0x7f1e28005b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-06T02:08:00.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:00.582+0000 7f1e3ad9d640 1 -- 172.21.15.49:0/135760884 >> v2:172.21.15.49:3300/0 conn(0x7f1e280036f0 msgr2=0x7f1e28005b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-06T02:08:00.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:00.582+0000 7f1e3ad9d640 1 --2- 172.21.15.49:0/135760884 >> v2:172.21.15.49:3300/0 conn(0x7f1e280036f0 0x7f1e28005b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-06T02:08:00.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:00.582+0000 7f1e3ad9d640 1 --2- 172.21.15.49:0/135760884 >> v2:172.21.15.49:3300/0 conn(0x7f1e280036f0 0x7f1e28005b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-06T02:08:02.165 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:02.170+0000 7f1e38d99640 1 -- 172.21.15.49:0/135760884 >> v2:172.21.15.49:3300/0 conn(0x7f1e280036f0 msgr2=0x7f1e28005b50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-06T02:08:02.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:02.170+0000 7f1e38d99640 1 --2- 172.21.15.49:0/135760884 >> v2:172.21.15.49:3300/0 conn(0x7f1e280036f0 0x7f1e28005b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:02.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:02.170+0000 7f1e38d99640 1 -- 172.21.15.49:0/135760884 >> v1:172.21.15.49:6789/0 conn(0x7f1e28006210 legacy=0x7f1e280086d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:02.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:02.170+0000 7f1e3b59e640 1 -- 172.21.15.49:0/135760884 reap_dead start 2023-11-06T02:08:02.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:02.170+0000 7f1e38d99640 1 --2- 172.21.15.49:0/135760884 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e3c15cbb0 0x7f1e2800a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:02.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:02.170+0000 7f1e38d99640 1 -- 172.21.15.49:0/135760884 --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e2800af90 con 0x7f1e28005b50 2023-11-06T02:08:02.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:02.170+0000 7f1e38d99640 1 -- 172.21.15.49:0/135760884 --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e2800aa40 con 0x7f1e2800a590 2023-11-06T02:08:02.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:02.170+0000 7f1e38d99640 1 -- 172.21.15.49:0/135760884 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e3c15cf90 con 0x7f1e3c15cbb0 2023-11-06T02:08:02.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:02.170+0000 7f1e3ad9d640 1 --2- 172.21.15.49:0/135760884 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e3c15cbb0 0x7f1e2800a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:02.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:02.170+0000 7f1e3ad9d640 1 -- 172.21.15.49:0/135760884 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e3c15cbb0 msgr2=0x7f1e2800a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-06T02:08:02.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:02.170+0000 7f1e3ad9d640 1 -- 172.21.15.49:0/135760884 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e3c15cbb0 msgr2=0x7f1e2800a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-06T02:08:02.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:02.170+0000 7f1e3ad9d640 1 --2- 172.21.15.49:0/135760884 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e3c15cbb0 0x7f1e2800a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-06T02:08:02.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:02.170+0000 7f1e3ad9d640 1 --2- 172.21.15.49:0/135760884 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e3c15cbb0 0x7f1e2800a050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-06T02:08:02.367 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:02.374+0000 7f1e3ad9d640 1 --2- 172.21.15.49:0/135760884 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e3c15cbb0 0x7f1e2800a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:02.367 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:02.374+0000 7f1e3ad9d640 1 -- 172.21.15.49:0/135760884 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e3c15cbb0 msgr2=0x7f1e2800a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-06T02:08:02.367 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:02.374+0000 7f1e3ad9d640 1 -- 172.21.15.49:0/135760884 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e3c15cbb0 msgr2=0x7f1e2800a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-06T02:08:02.368 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:02.374+0000 7f1e3ad9d640 1 --2- 172.21.15.49:0/135760884 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e3c15cbb0 0x7f1e2800a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-06T02:08:02.368 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:02.374+0000 7f1e3ad9d640 1 --2- 172.21.15.49:0/135760884 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e3c15cbb0 0x7f1e2800a050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-06T02:08:02.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:02.778+0000 7f1e3ad9d640 1 --2- 172.21.15.49:0/135760884 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e3c15cbb0 0x7f1e2800a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:02.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:02.778+0000 7f1e3ad9d640 1 -- 172.21.15.49:0/135760884 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e3c15cbb0 msgr2=0x7f1e2800a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-06T02:08:02.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:02.778+0000 7f1e3ad9d640 1 -- 172.21.15.49:0/135760884 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e3c15cbb0 msgr2=0x7f1e2800a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-06T02:08:02.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:02.778+0000 7f1e3ad9d640 1 --2- 172.21.15.49:0/135760884 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e3c15cbb0 0x7f1e2800a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-06T02:08:02.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:02.778+0000 7f1e3ad9d640 1 --2- 172.21.15.49:0/135760884 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e3c15cbb0 0x7f1e2800a050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-06T02:08:03.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:03.582+0000 7f1e3ad9d640 1 --2- 172.21.15.49:0/135760884 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e3c15cbb0 0x7f1e2800a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:03.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:03.582+0000 7f1e3ad9d640 1 -- 172.21.15.49:0/135760884 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e3c15cbb0 msgr2=0x7f1e2800a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-06T02:08:03.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:03.582+0000 7f1e3ad9d640 1 -- 172.21.15.49:0/135760884 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e3c15cbb0 msgr2=0x7f1e2800a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-06T02:08:03.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:03.582+0000 7f1e3ad9d640 1 --2- 172.21.15.49:0/135760884 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e3c15cbb0 0x7f1e2800a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-06T02:08:03.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:03.582+0000 7f1e3ad9d640 1 --2- 172.21.15.49:0/135760884 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e3c15cbb0 0x7f1e2800a050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-06T02:08:05.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:05.170+0000 7f1e38d99640 1 -- 172.21.15.49:0/135760884 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e3c15cbb0 msgr2=0x7f1e2800a050 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-06T02:08:05.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:05.170+0000 7f1e38d99640 1 --2- 172.21.15.49:0/135760884 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e3c15cbb0 0x7f1e2800a050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:05.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:05.170+0000 7f1e38d99640 1 -- 172.21.15.49:0/135760884 >> v1:172.21.15.150:6789/0 conn(0x7f1e2800a590 legacy=0x7f1e280066c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:05.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:05.170+0000 7f1e38d99640 1 -- 172.21.15.49:0/135760884 >> v1:172.21.15.49:6789/0 conn(0x7f1e28005b50 legacy=0x7f1e2800d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:05.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:05.170+0000 7f1e38d99640 1 --2- 172.21.15.49:0/135760884 >> v2:172.21.15.150:3300/0 conn(0x7f1e2800eb30 0x7f1e28010fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:05.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:05.170+0000 7f1e38d99640 1 --2- 172.21.15.49:0/135760884 >> v2:172.21.15.49:3300/0 conn(0x7f1e28011680 0x7f1e28013ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:05.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:05.170+0000 7f1e38d99640 1 -- 172.21.15.49:0/135760884 --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e3c15cf90 con 0x7f1e28011680 2023-11-06T02:08:05.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:05.170+0000 7f1e38d99640 1 -- 172.21.15.49:0/135760884 --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e3c0d0b80 con 0x7f1e2800eb30 2023-11-06T02:08:05.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:05.170+0000 7f1e3a59c640 1 --2- 172.21.15.49:0/135760884 >> v2:172.21.15.49:3300/0 conn(0x7f1e28011680 0x7f1e28013ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:05.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:05.170+0000 7f1e3ad9d640 1 --2- 172.21.15.49:0/135760884 >> v2:172.21.15.150:3300/0 conn(0x7f1e2800eb30 0x7f1e28010fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:05.169 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:05.174+0000 7f1e3a59c640 1 -- 172.21.15.49:0/135760884 >> v2:172.21.15.49:3300/0 conn(0x7f1e28011680 msgr2=0x7f1e28013ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-11-06T02:08:05.169 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:05.174+0000 7f1e3a59c640 1 -- 172.21.15.49:0/135760884 >> v2:172.21.15.49:3300/0 conn(0x7f1e28011680 msgr2=0x7f1e28013ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-06T02:08:05.169 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:05.174+0000 7f1e3a59c640 1 --2- 172.21.15.49:0/135760884 >> v2:172.21.15.49:3300/0 conn(0x7f1e28011680 0x7f1e28013ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-06T02:08:05.169 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:05.174+0000 7f1e3a59c640 1 --2- 172.21.15.49:0/135760884 >> v2:172.21.15.49:3300/0 conn(0x7f1e28011680 0x7f1e28013ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-06T02:08:05.169 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:05.174+0000 7f1e3ad9d640 1 -- 172.21.15.49:0/135760884 >> v2:172.21.15.150:3300/0 conn(0x7f1e2800eb30 msgr2=0x7f1e28010fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-06T02:08:05.170 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:05.174+0000 7f1e3ad9d640 1 -- 172.21.15.49:0/135760884 >> v2:172.21.15.150:3300/0 conn(0x7f1e2800eb30 msgr2=0x7f1e28010fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-06T02:08:05.170 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:05.174+0000 7f1e3ad9d640 1 --2- 172.21.15.49:0/135760884 >> v2:172.21.15.150:3300/0 conn(0x7f1e2800eb30 0x7f1e28010fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-06T02:08:05.170 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:05.174+0000 7f1e3ad9d640 1 --2- 172.21.15.49:0/135760884 >> v2:172.21.15.150:3300/0 conn(0x7f1e2800eb30 0x7f1e28010fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-06T02:08:05.371 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:05.378+0000 7f1e3a59c640 1 --2- 172.21.15.49:0/135760884 >> v2:172.21.15.49:3300/0 conn(0x7f1e28011680 0x7f1e28013ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:05.371 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:05.378+0000 7f1e3ad9d640 1 --2- 172.21.15.49:0/135760884 >> v2:172.21.15.150:3300/0 conn(0x7f1e2800eb30 0x7f1e28010fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:05.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:05.378+0000 7f1e3a59c640 1 -- 172.21.15.49:0/135760884 >> v2:172.21.15.49:3300/0 conn(0x7f1e28011680 msgr2=0x7f1e28013ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-11-06T02:08:05.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:05.378+0000 7f1e3a59c640 1 -- 172.21.15.49:0/135760884 >> v2:172.21.15.49:3300/0 conn(0x7f1e28011680 msgr2=0x7f1e28013ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-06T02:08:05.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:05.378+0000 7f1e3a59c640 1 --2- 172.21.15.49:0/135760884 >> v2:172.21.15.49:3300/0 conn(0x7f1e28011680 0x7f1e28013ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-06T02:08:05.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:05.378+0000 7f1e3a59c640 1 --2- 172.21.15.49:0/135760884 >> v2:172.21.15.49:3300/0 conn(0x7f1e28011680 0x7f1e28013ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-06T02:08:05.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:05.378+0000 7f1e3ad9d640 1 -- 172.21.15.49:0/135760884 >> v2:172.21.15.150:3300/0 conn(0x7f1e2800eb30 msgr2=0x7f1e28010fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-06T02:08:05.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:05.378+0000 7f1e3ad9d640 1 -- 172.21.15.49:0/135760884 >> v2:172.21.15.150:3300/0 conn(0x7f1e2800eb30 msgr2=0x7f1e28010fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-06T02:08:05.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:05.378+0000 7f1e3ad9d640 1 --2- 172.21.15.49:0/135760884 >> v2:172.21.15.150:3300/0 conn(0x7f1e2800eb30 0x7f1e28010fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-06T02:08:05.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:05.378+0000 7f1e3ad9d640 1 --2- 172.21.15.49:0/135760884 >> v2:172.21.15.150:3300/0 conn(0x7f1e2800eb30 0x7f1e28010fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-06T02:08:05.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:05.782+0000 7f1e3a59c640 1 --2- 172.21.15.49:0/135760884 >> v2:172.21.15.49:3300/0 conn(0x7f1e28011680 0x7f1e28013ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:05.776 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:05.782+0000 7f1e3ad9d640 1 --2- 172.21.15.49:0/135760884 >> v2:172.21.15.150:3300/0 conn(0x7f1e2800eb30 0x7f1e28010fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:05.776 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:05.782+0000 7f1e3a59c640 1 -- 172.21.15.49:0/135760884 >> v2:172.21.15.49:3300/0 conn(0x7f1e28011680 msgr2=0x7f1e28013ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-06T02:08:05.776 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:05.782+0000 7f1e3a59c640 1 -- 172.21.15.49:0/135760884 >> v2:172.21.15.49:3300/0 conn(0x7f1e28011680 msgr2=0x7f1e28013ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-06T02:08:05.776 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:05.782+0000 7f1e3a59c640 1 --2- 172.21.15.49:0/135760884 >> v2:172.21.15.49:3300/0 conn(0x7f1e28011680 0x7f1e28013ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-06T02:08:05.777 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:05.782+0000 7f1e3a59c640 1 --2- 172.21.15.49:0/135760884 >> v2:172.21.15.49:3300/0 conn(0x7f1e28011680 0x7f1e28013ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-06T02:08:05.777 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:05.782+0000 7f1e3ad9d640 1 -- 172.21.15.49:0/135760884 >> v2:172.21.15.150:3300/0 conn(0x7f1e2800eb30 msgr2=0x7f1e28010fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-11-06T02:08:05.777 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:05.782+0000 7f1e3ad9d640 1 -- 172.21.15.49:0/135760884 >> v2:172.21.15.150:3300/0 conn(0x7f1e2800eb30 msgr2=0x7f1e28010fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-06T02:08:05.778 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:05.782+0000 7f1e3ad9d640 1 --2- 172.21.15.49:0/135760884 >> v2:172.21.15.150:3300/0 conn(0x7f1e2800eb30 0x7f1e28010fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-06T02:08:05.778 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:05.782+0000 7f1e3ad9d640 1 --2- 172.21.15.49:0/135760884 >> v2:172.21.15.150:3300/0 conn(0x7f1e2800eb30 0x7f1e28010fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-06T02:08:06.579 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:06.585+0000 7f1e3a59c640 1 --2- 172.21.15.49:0/135760884 >> v2:172.21.15.49:3300/0 conn(0x7f1e28011680 0x7f1e28013ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:06.579 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:06.585+0000 7f1e3ad9d640 1 --2- 172.21.15.49:0/135760884 >> v2:172.21.15.150:3300/0 conn(0x7f1e2800eb30 0x7f1e28010fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:06.579 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:06.585+0000 7f1e3a59c640 1 -- 172.21.15.49:0/135760884 >> v2:172.21.15.150:3300/0 conn(0x7f1e2800eb30 msgr2=0x7f1e28010fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:06.580 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:06.585+0000 7f1e3a59c640 1 --2- 172.21.15.49:0/135760884 >> v2:172.21.15.150:3300/0 conn(0x7f1e2800eb30 0x7f1e28010fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:06.580 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:06.585+0000 7f1e3a59c640 1 -- 172.21.15.49:0/135760884 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e2800acd0 con 0x7f1e28011680 2023-11-06T02:08:06.580 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:06.585+0000 7f1e3ad9d640 1 --2- 172.21.15.49:0/135760884 >> v2:172.21.15.150:3300/0 conn(0x7f1e2800eb30 0x7f1e28010fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:08:06.581 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:06.585+0000 7f1e3a59c640 1 --2- 172.21.15.49:0/135760884 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e28011680 0x7f1e28013ae0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1e2800eae0 tx=0x7f1e3001d7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d21359e4209b0fe1 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:06.581 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:06.585+0000 7f1e39d9b640 1 -- 172.21.15.49:0/135760884 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e300a6750 con 0x7f1e28011680 2023-11-06T02:08:06.582 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:06.585+0000 7f1e39d9b640 1 -- 172.21.15.49:0/135760884 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e300aa070 con 0x7f1e28011680 2023-11-06T02:08:06.582 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:06.585+0000 7f1e39d9b640 1 -- 172.21.15.49:0/135760884 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e300a6ad0 con 0x7f1e28011680 2023-11-06T02:08:06.583 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:06.585+0000 7f1e411e7640 1 -- 172.21.15.49:0/135760884 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e28011680 msgr2=0x7f1e28013ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:06.583 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:06.585+0000 7f1e411e7640 1 --2- 172.21.15.49:0/135760884 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e28011680 0x7f1e28013ae0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1e2800eae0 tx=0x7f1e3001d7e0 comp rx=0 tx=0).stop 2023-11-06T02:08:06.583 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:06.585+0000 7f1e3b59e640 1 -- 172.21.15.49:0/135760884 reap_dead start 2023-11-06T02:08:06.583 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:06.585+0000 7f1e411e7640 1 -- 172.21.15.49:0/135760884 shutdown_connections 2023-11-06T02:08:06.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:06.585+0000 7f1e411e7640 1 --2- 172.21.15.49:0/135760884 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e28011680 0x7f1e28013ae0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:06.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:06.585+0000 7f1e411e7640 1 -- 172.21.15.49:0/135760884 >> 172.21.15.49:0/135760884 conn(0x7f1e3c0718a0 msgr2=0x7f1e3c071cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:06.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:06.585+0000 7f1e411e7640 1 -- 172.21.15.49:0/135760884 shutdown_connections 2023-11-06T02:08:06.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:06.585+0000 7f1e411e7640 1 -- 172.21.15.49:0/135760884 wait complete. 2023-11-06T02:08:06.586 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:06.589+0000 7f1e411e7640 1 Processor -- start 2023-11-06T02:08:06.587 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:06.589+0000 7f1e411e7640 1 -- start start 2023-11-06T02:08:06.587 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:06.589+0000 7f1e411e7640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1e28011680 0x7f1e3c0c8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:06.587 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:06.589+0000 7f1e411e7640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e3c15dd70 0x7f1e3c15efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:06.587 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:06.589+0000 7f1e411e7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e3c146aa0 0x7f1e3c146f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:06.588 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:06.589+0000 7f1e411e7640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e3c0c6c30 con 0x7f1e3c146aa0 2023-11-06T02:08:06.588 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:06.589+0000 7f1e411e7640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e3c147460 con 0x7f1e28011680 2023-11-06T02:08:06.588 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:06.589+0000 7f1e411e7640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e3c0c6600 con 0x7f1e3c15dd70 2023-11-06T02:08:06.588 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:06.589+0000 7f1e3b59e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e3c146aa0 0x7f1e3c146f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:06.588 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:06.589+0000 7f1e3b59e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e3c146aa0 0x7f1e3c146f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42566/0 (socket says 172.21.15.49:42566) 2023-11-06T02:08:06.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:06.589+0000 7f1e3b59e640 1 -- 172.21.15.49:0/2708659405 learned_addr learned my addr 172.21.15.49:0/2708659405 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:06.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:06.589+0000 7f1e3ad9d640 1 --2- 172.21.15.49:0/2708659405 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1e28011680 0x7f1e3c0c8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:06.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:06.589+0000 7f1e3a59c640 1 --2- 172.21.15.49:0/2708659405 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e3c15dd70 0x7f1e3c15efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:06.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:06.589+0000 7f1e3b59e640 1 -- 172.21.15.49:0/2708659405 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e3c15dd70 msgr2=0x7f1e3c15efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:06.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:06.589+0000 7f1e3b59e640 1 --2- 172.21.15.49:0/2708659405 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e3c15dd70 0x7f1e3c15efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:06.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:06.589+0000 7f1e3b59e640 1 -- 172.21.15.49:0/2708659405 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1e28011680 msgr2=0x7f1e3c0c8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:06.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:06.589+0000 7f1e3b59e640 1 --2- 172.21.15.49:0/2708659405 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1e28011680 0x7f1e3c0c8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:06.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:06.589+0000 7f1e3b59e640 1 -- 172.21.15.49:0/2708659405 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e2800aa40 con 0x7f1e3c146aa0 2023-11-06T02:08:06.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:06.589+0000 7f1e3ad9d640 1 --2- 172.21.15.49:0/2708659405 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1e28011680 0x7f1e3c0c8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:08:06.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:06.589+0000 7f1e3b59e640 1 --2- 172.21.15.49:0/2708659405 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e3c146aa0 0x7f1e3c146f20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c0012f0 tx=0x7f1e2c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:06.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:06.589+0000 7f1e237fe640 1 -- 172.21.15.49:0/2708659405 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e2c007070 con 0x7f1e3c146aa0 2023-11-06T02:08:06.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:06.589+0000 7f1e237fe640 1 -- 172.21.15.49:0/2708659405 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e2c007210 con 0x7f1e3c146aa0 2023-11-06T02:08:06.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:06.589+0000 7f1e237fe640 1 -- 172.21.15.49:0/2708659405 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e2c0075c0 con 0x7f1e3c146aa0 2023-11-06T02:08:06.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:06.589+0000 7f1e411e7640 1 -- 172.21.15.49:0/2708659405 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e3c0c9110 con 0x7f1e3c146aa0 2023-11-06T02:08:06.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:06.589+0000 7f1e411e7640 1 -- 172.21.15.49:0/2708659405 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e3c0c6850 con 0x7f1e3c146aa0 2023-11-06T02:08:06.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:06.589+0000 7f1e237fe640 1 -- 172.21.15.49:0/2708659405 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f1e2c01d3f0 con 0x7f1e3c146aa0 2023-11-06T02:08:06.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:06.589+0000 7f1e237fe640 1 -- 172.21.15.49:0/2708659405 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f1e2c012360 con 0x7f1e3c146aa0 2023-11-06T02:08:06.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:06.593+0000 7f1e411e7640 1 -- 172.21.15.49:0/2708659405 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e3c0c6d70 con 0x7f1e3c146aa0 2023-11-06T02:08:06.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:06.593+0000 7f1e237fe640 1 -- 172.21.15.49:0/2708659405 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f1e3c0c6d70 con 0x7f1e3c146aa0 2023-11-06T02:08:06.659 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:06.665+0000 7f1e411e7640 1 -- 172.21.15.49:0/2708659405 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f1e3c074040 con 0x7f1e3c146aa0 2023-11-06T02:08:06.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:06.669+0000 7f1e237fe640 1 -- 172.21.15.49:0/2708659405 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f1e2c01d830 con 0x7f1e3c146aa0 2023-11-06T02:08:06.664 INFO:teuthology.orchestra.run.smithi049.stderr:adjusted tunables profile to default 2023-11-06T02:08:06.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:06.669+0000 7f1e411e7640 1 -- 172.21.15.49:0/2708659405 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e3c146aa0 msgr2=0x7f1e3c146f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:06.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:06.669+0000 7f1e411e7640 1 --2- 172.21.15.49:0/2708659405 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e3c146aa0 0x7f1e3c146f20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c0012f0 tx=0x7f1e2c00c8e0 comp rx=0 tx=0).stop 2023-11-06T02:08:06.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:06.669+0000 7f1e411e7640 1 -- 172.21.15.49:0/2708659405 shutdown_connections 2023-11-06T02:08:06.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:06.669+0000 7f1e411e7640 1 --2- 172.21.15.49:0/2708659405 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e3c146aa0 0x7f1e3c146f20 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:06.667 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:06.669+0000 7f1e411e7640 1 --2- 172.21.15.49:0/2708659405 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e3c15dd70 0x7f1e3c15efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:06.667 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:06.669+0000 7f1e411e7640 1 --2- 172.21.15.49:0/2708659405 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1e28011680 0x7f1e3c0c8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:06.667 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:06.669+0000 7f1e411e7640 1 -- 172.21.15.49:0/2708659405 >> 172.21.15.49:0/2708659405 conn(0x7f1e3c0718a0 msgr2=0x7f1e3c0cb0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:06.667 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:06.669+0000 7f1e411e7640 1 -- 172.21.15.49:0/2708659405 shutdown_connections 2023-11-06T02:08:06.667 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:06.669+0000 7f1e411e7640 1 -- 172.21.15.49:0/2708659405 wait complete. 2023-11-06T02:08:06.687 INFO:tasks.ceph:check_enable_crimson: False 2023-11-06T02:08:06.688 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-11-06T02:08:06.688 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-11-06T02:08:06.688 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-11-06T02:08:06.700 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-11-06T02:08:06.700 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-11-06T02:08:06.754 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-11-06T02:08:06.754 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-11-06T02:08:06.761 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-06T02:08:06.762+0000 7f4792183180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-06T02:08:06.798 INFO:tasks.ceph.mgr.y.smithi049.stderr:2023-11-06T02:08:06.801+0000 7f148018b180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-06T02:08:06.806 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-11-06T02:08:06.806 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-11-06T02:08:06.858 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-11-06T02:08:06.859 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-11-06T02:08:06.871 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-11-06T02:08:06.872 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-11-06T02:08:06.927 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-11-06T02:08:06.927 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-11-06T02:08:06.983 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-11-06T02:08:06.983 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-11-06T02:08:07.038 DEBUG:teuthology.orchestra.run.smithi150:> sudo ceph --cluster ceph osd new 6ad55af8-0571-4e6a-8722-b5d429b7496d 0 2023-11-06T02:08:07.202 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.202+0000 7fbfeb4bb640 1 Processor -- start 2023-11-06T02:08:07.202 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.202+0000 7fbfeb4bb640 1 -- start start 2023-11-06T02:08:07.205 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.206+0000 7fbfeb4bb640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fbfe40caec0 0x7fbfe40cb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:07.205 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.206+0000 7fbfeb4bb640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfe415ad50 con 0x7fbfe415dd10 2023-11-06T02:08:07.206 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.206+0000 7fbfeb4bb640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfe40cecf0 con 0x7fbfe40c9600 2023-11-06T02:08:07.206 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.206+0000 7fbfeb4bb640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfe415e280 con 0x7fbfe40caec0 2023-11-06T02:08:07.206 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.206+0000 7fbfe8a56640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7fbfe40c9600 0x7fbfe4069d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:45404/0 (socket says 172.21.15.150:45404) 2023-11-06T02:08:07.206 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.206+0000 7fbfe8a56640 1 -- 172.21.15.150:0/1954788251 learned_addr learned my addr 172.21.15.150:0/1954788251 (peer_addr_for_me v1:172.21.15.150:0/0) 2023-11-06T02:08:07.206 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.206+0000 7fbfe9257640 1 --2- 172.21.15.150:0/1954788251 >> v2:172.21.15.49:3300/0 conn(0x7fbfe40caec0 0x7fbfe40cb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:07.207 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.206+0000 7fbfdbfff640 1 -- 172.21.15.150:0/1954788251 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3956204432 0 0) 0x7fbfe40cecf0 con 0x7fbfe40c9600 2023-11-06T02:08:07.207 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.206+0000 7fbfdbfff640 1 -- 172.21.15.150:0/1954788251 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbfd0003530 con 0x7fbfe40c9600 2023-11-06T02:08:07.207 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.206+0000 7fbfdbfff640 1 -- 172.21.15.150:0/1954788251 <== mon.1 v1:172.21.15.150:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4164255924 0 0) 0x7fbfd0003530 con 0x7fbfe40c9600 2023-11-06T02:08:07.207 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.206+0000 7fbfdbfff640 1 -- 172.21.15.150:0/1954788251 >> v2:172.21.15.49:3300/0 conn(0x7fbfe40caec0 msgr2=0x7fbfe40cb2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:07.207 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.206+0000 7fbfdbfff640 1 --2- 172.21.15.150:0/1954788251 >> v2:172.21.15.49:3300/0 conn(0x7fbfe40caec0 0x7fbfe40cb2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:07.208 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.206+0000 7fbfdbfff640 1 -- 172.21.15.150:0/1954788251 >> v1:172.21.15.49:6789/0 conn(0x7fbfe415dd10 legacy=0x7fbfe40cd190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:07.208 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.206+0000 7fbfdbfff640 1 -- 172.21.15.150:0/1954788251 --> v1:172.21.15.150:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfe40cb7e0 con 0x7fbfe40c9600 2023-11-06T02:08:07.208 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.206+0000 7fbfdbfff640 1 -- 172.21.15.150:0/1954788251 <== mon.1 v1:172.21.15.150:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7fbfd4001680 con 0x7fbfe40c9600 2023-11-06T02:08:07.208 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.206+0000 7fbfdbfff640 1 -- 172.21.15.150:0/1954788251 >> v1:172.21.15.150:6789/0 conn(0x7fbfe40c9600 legacy=0x7fbfe4069d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:07.208 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.206+0000 7fbfdbfff640 1 --2- 172.21.15.150:0/1954788251 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbfd0003980 0x7fbfd0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:07.208 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.206+0000 7fbfdbfff640 1 --2- 172.21.15.150:0/1954788251 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbfd00047d0 0x7fbfd0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:07.209 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.206+0000 7fbfdbfff640 1 --2- 172.21.15.150:0/1954788251 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbfd0005190 0x7fbfd000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:07.209 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.206+0000 7fbfdbfff640 1 -- 172.21.15.150:0/1954788251 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe415e280 con 0x7fbfd00047d0 2023-11-06T02:08:07.209 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.206+0000 7fbfdbfff640 1 -- 172.21.15.150:0/1954788251 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd000c130 con 0x7fbfd0005190 2023-11-06T02:08:07.209 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.206+0000 7fbfdbfff640 1 -- 172.21.15.150:0/1954788251 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd000c2f0 con 0x7fbfd0003980 2023-11-06T02:08:07.209 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.206+0000 7fbfe9257640 1 --2- 172.21.15.150:0/1954788251 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbfd0003980 0x7fbfd0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:07.210 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.206+0000 7fbfe9a58640 1 --2- 172.21.15.150:0/1954788251 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbfd0005190 0x7fbfd000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:07.210 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.206+0000 7fbfe8a56640 1 --2- 172.21.15.150:0/1954788251 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbfd00047d0 0x7fbfd0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:07.210 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.206+0000 7fbfe9257640 1 -- 172.21.15.150:0/1954788251 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbfd0005190 msgr2=0x7fbfd000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:07.210 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.206+0000 7fbfe9257640 1 --2- 172.21.15.150:0/1954788251 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbfd0005190 0x7fbfd000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:07.210 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.206+0000 7fbfe9257640 1 -- 172.21.15.150:0/1954788251 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbfd00047d0 msgr2=0x7fbfd0004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:07.211 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.206+0000 7fbfe9257640 1 --2- 172.21.15.150:0/1954788251 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbfd00047d0 0x7fbfd0004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:07.211 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.206+0000 7fbfe9257640 1 -- 172.21.15.150:0/1954788251 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfd0003530 con 0x7fbfd0003980 2023-11-06T02:08:07.211 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.206+0000 7fbfe9a58640 1 -- 172.21.15.150:0/1954788251 reap_dead start 2023-11-06T02:08:07.214 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.206+0000 7fbfe9257640 1 --2- 172.21.15.150:0/1954788251 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbfd0003980 0x7fbfd0003da0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbfe0009ae0 tx=0x7fbfe001ca20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dfd698fb84541fb server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:07.215 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.210+0000 7fbfdbfff640 1 -- 172.21.15.150:0/1954788251 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe00a5590 con 0x7fbfd0003980 2023-11-06T02:08:07.215 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.210+0000 7fbfdbfff640 1 -- 172.21.15.150:0/1954788251 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfe00a56f0 con 0x7fbfd0003980 2023-11-06T02:08:07.215 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.210+0000 7fbfdbfff640 1 -- 172.21.15.150:0/1954788251 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe00a5aa0 con 0x7fbfd0003980 2023-11-06T02:08:07.215 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.210+0000 7fbfeb4bb640 1 -- 172.21.15.150:0/1954788251 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbfd0003980 msgr2=0x7fbfd0003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:07.215 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.210+0000 7fbfeb4bb640 1 --2- 172.21.15.150:0/1954788251 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbfd0003980 0x7fbfd0003da0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbfe0009ae0 tx=0x7fbfe001ca20 comp rx=0 tx=0).stop 2023-11-06T02:08:07.216 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.210+0000 7fbfeb4bb640 1 -- 172.21.15.150:0/1954788251 shutdown_connections 2023-11-06T02:08:07.216 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.210+0000 7fbfeb4bb640 1 --2- 172.21.15.150:0/1954788251 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbfd0003980 0x7fbfd0003da0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:07.216 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.210+0000 7fbfeb4bb640 1 -- 172.21.15.150:0/1954788251 >> 172.21.15.150:0/1954788251 conn(0x7fbfe40b0aa0 msgr2=0x7fbfe40b0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:07.216 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.210+0000 7fbfeb4bb640 1 -- 172.21.15.150:0/1954788251 shutdown_connections 2023-11-06T02:08:07.217 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.210+0000 7fbfeb4bb640 1 -- 172.21.15.150:0/1954788251 wait complete. 2023-11-06T02:08:07.217 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.210+0000 7fbfeb4bb640 1 Processor -- start 2023-11-06T02:08:07.217 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.210+0000 7fbfeb4bb640 1 -- start start 2023-11-06T02:08:07.217 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.210+0000 7fbfeb4bb640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbfd0003980 0x7fbfe41fa440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:07.217 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.210+0000 7fbfeb4bb640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbfe40b2220 0x7fbfe406d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:07.218 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.210+0000 7fbfeb4bb640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbfe41f5910 0x7fbfe41f5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:07.218 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.210+0000 7fbfeb4bb640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe41fa980 con 0x7fbfe40b2220 2023-11-06T02:08:07.218 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.210+0000 7fbfeb4bb640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe40b3d80 con 0x7fbfd0003980 2023-11-06T02:08:07.218 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.210+0000 7fbfeb4bb640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe40b4060 con 0x7fbfe41f5910 2023-11-06T02:08:07.218 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.210+0000 7fbfe9257640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbfd0003980 0x7fbfe41fa440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:07.219 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.210+0000 7fbfe9257640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbfd0003980 0x7fbfe41fa440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:43880/0 (socket says 172.21.15.150:43880) 2023-11-06T02:08:07.219 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.210+0000 7fbfe9257640 1 -- 172.21.15.150:0/3524978398 learned_addr learned my addr 172.21.15.150:0/3524978398 (peer_addr_for_me v2:172.21.15.150:0/0) 2023-11-06T02:08:07.219 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.210+0000 7fbfe9a58640 1 --2- 172.21.15.150:0/3524978398 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbfe41f5910 0x7fbfe41f5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:07.219 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.210+0000 7fbfe8a56640 1 --2- 172.21.15.150:0/3524978398 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbfe40b2220 0x7fbfe406d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:07.219 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.210+0000 7fbfe9257640 1 -- 172.21.15.150:0/3524978398 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbfe41f5910 msgr2=0x7fbfe41f5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:07.219 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.210+0000 7fbfe9257640 1 --2- 172.21.15.150:0/3524978398 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbfe41f5910 0x7fbfe41f5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:07.220 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.210+0000 7fbfe9257640 1 -- 172.21.15.150:0/3524978398 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbfe40b2220 msgr2=0x7fbfe406d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:07.220 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.210+0000 7fbfe9257640 1 --2- 172.21.15.150:0/3524978398 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbfe40b2220 0x7fbfe406d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:07.220 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.210+0000 7fbfe9257640 1 -- 172.21.15.150:0/3524978398 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfe40b4340 con 0x7fbfd0003980 2023-11-06T02:08:07.220 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.214+0000 7fbfe9257640 1 --2- 172.21.15.150:0/3524978398 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbfd0003980 0x7fbfe41fa440 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fbfe001c850 tx=0x7fbfe00197c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:07.220 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.214+0000 7fbfd9ffb640 1 -- 172.21.15.150:0/3524978398 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe00bd070 con 0x7fbfd0003980 2023-11-06T02:08:07.221 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.214+0000 7fbfd9ffb640 1 -- 172.21.15.150:0/3524978398 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfe000f000 con 0x7fbfd0003980 2023-11-06T02:08:07.221 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.214+0000 7fbfeb4bb640 1 -- 172.21.15.150:0/3524978398 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfe40b4620 con 0x7fbfd0003980 2023-11-06T02:08:07.221 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.214+0000 7fbfd9ffb640 1 -- 172.21.15.150:0/3524978398 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe0004260 con 0x7fbfd0003980 2023-11-06T02:08:07.221 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.214+0000 7fbfeb4bb640 1 -- 172.21.15.150:0/3524978398 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfe41f62d0 con 0x7fbfd0003980 2023-11-06T02:08:07.221 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.214+0000 7fbfd9ffb640 1 -- 172.21.15.150:0/3524978398 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fbfe00b4070 con 0x7fbfd0003980 2023-11-06T02:08:07.221 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.214+0000 7fbfd9ffb640 1 -- 172.21.15.150:0/3524978398 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7fbfe0004400 con 0x7fbfd0003980 2023-11-06T02:08:07.222 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.214+0000 7fbfeb4bb640 1 -- 172.21.15.150:0/3524978398 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfe406f500 con 0x7fbfd0003980 2023-11-06T02:08:07.229 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.230+0000 7fbfd9ffb640 1 -- 172.21.15.150:0/3524978398 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7fbfe00b2070 con 0x7fbfd0003980 2023-11-06T02:08:07.282 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-06T02:08:07.282+0000 7f4792183180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-06T02:08:07.296 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.298+0000 7fbfeb4bb640 1 -- 172.21.15.150:0/3524978398 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd new", "uuid": "6ad55af8-0571-4e6a-8722-b5d429b7496d", "id": 0} v 0) v1 -- 0x7fbfe40b1620 con 0x7fbfd0003980 2023-11-06T02:08:07.305 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.306+0000 7fbfd9ffb640 1 -- 172.21.15.150:0/3524978398 <== mon.1 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6ad55af8-0571-4e6a-8722-b5d429b7496d", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7fbfe00096b0 con 0x7fbfd0003980 2023-11-06T02:08:07.306 INFO:teuthology.orchestra.run.smithi150.stdout:0 2023-11-06T02:08:07.307 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.306+0000 7fbfeb4bb640 1 -- 172.21.15.150:0/3524978398 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbfd0003980 msgr2=0x7fbfe41fa440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:07.307 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.306+0000 7fbfeb4bb640 1 --2- 172.21.15.150:0/3524978398 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbfd0003980 0x7fbfe41fa440 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fbfe001c850 tx=0x7fbfe00197c0 comp rx=0 tx=0).stop 2023-11-06T02:08:07.307 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.306+0000 7fbfeb4bb640 1 -- 172.21.15.150:0/3524978398 shutdown_connections 2023-11-06T02:08:07.308 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.306+0000 7fbfeb4bb640 1 --2- 172.21.15.150:0/3524978398 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbfe40b2220 0x7fbfe406d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:07.308 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.306+0000 7fbfeb4bb640 1 --2- 172.21.15.150:0/3524978398 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbfe41f5910 0x7fbfe41f5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:07.308 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.306+0000 7fbfeb4bb640 1 --2- 172.21.15.150:0/3524978398 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbfd0003980 0x7fbfe41fa440 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:07.308 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.306+0000 7fbfeb4bb640 1 -- 172.21.15.150:0/3524978398 >> 172.21.15.150:0/3524978398 conn(0x7fbfe40b0aa0 msgr2=0x7fbfe40c9a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:07.308 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.310+0000 7fbfeb4bb640 1 -- 172.21.15.150:0/3524978398 shutdown_connections 2023-11-06T02:08:07.309 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.310+0000 7fbfeb4bb640 1 -- 172.21.15.150:0/3524978398 wait complete. 2023-11-06T02:08:07.323 DEBUG:teuthology.orchestra.run.smithi150:> sudo ceph --cluster ceph osd new c8b31f85-a757-43ed-9419-351acc46611b 1 2023-11-06T02:08:07.357 INFO:tasks.ceph.mgr.y.smithi049.stderr:2023-11-06T02:08:07.361+0000 7f148018b180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-06T02:08:07.446 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.446+0000 7f7fdf477640 1 Processor -- start 2023-11-06T02:08:07.447 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.446+0000 7f7fdf477640 1 -- start start 2023-11-06T02:08:07.449 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.450+0000 7f7fdf477640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f7fd8151800 0x7f7fd806af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:07.449 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.450+0000 7f7fdf477640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7fd8071520 0x7f7fd80cdfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:07.450 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.450+0000 7f7fdf477640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fd815baf0 con 0x7f7fd8151be0 2023-11-06T02:08:07.450 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.450+0000 7f7fdf477640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fd815af00 con 0x7f7fd8151800 2023-11-06T02:08:07.451 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.450+0000 7f7fdf477640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd815b1c0 con 0x7f7fd8071520 2023-11-06T02:08:07.451 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.450+0000 7f7fdda14640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7fd8071520 0x7f7fd80cdfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:07.451 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.450+0000 7f7fdda14640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7fd8071520 0x7f7fd80cdfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.150:55192/0 (socket says 172.21.15.150:55192) 2023-11-06T02:08:07.452 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.450+0000 7f7fdda14640 1 -- 172.21.15.150:0/4117615256 learned_addr learned my addr 172.21.15.150:0/4117615256 (peer_addr_for_me v2:172.21.15.150:0/0) 2023-11-06T02:08:07.452 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.450+0000 7f7fdd213640 1 --2- 172.21.15.150:0/4117615256 >> v2:172.21.15.49:3300/0 conn(0x7f7fd8151800 0x7f7fd806af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:07.452 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.450+0000 7f7fdda14640 1 -- 172.21.15.150:0/4117615256 >> v2:172.21.15.49:3300/0 conn(0x7f7fd8151800 msgr2=0x7f7fd806af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:07.453 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.450+0000 7f7fdda14640 1 --2- 172.21.15.150:0/4117615256 >> v2:172.21.15.49:3300/0 conn(0x7f7fd8151800 0x7f7fd806af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:07.453 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.450+0000 7f7fdda14640 1 -- 172.21.15.150:0/4117615256 >> v1:172.21.15.49:6789/0 conn(0x7f7fd8151be0 legacy=0x7f7fd8070e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:07.453 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.450+0000 7f7fdda14640 1 -- 172.21.15.150:0/4117615256 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fd815b4a0 con 0x7f7fd8071520 2023-11-06T02:08:07.458 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.450+0000 7f7fdda14640 1 --2- 172.21.15.150:0/4117615256 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7fd8071520 0x7f7fd80cdfa0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7fc8009870 tx=0x7f7fc801c510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=281e1ed5027c9505 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:07.458 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.450+0000 7f7fd7fff640 1 -- 172.21.15.150:0/4117615256 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc80aa650 con 0x7f7fd8071520 2023-11-06T02:08:07.458 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.450+0000 7f7fd7fff640 1 -- 172.21.15.150:0/4117615256 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fc80aa7b0 con 0x7f7fd8071520 2023-11-06T02:08:07.459 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.450+0000 7f7fd7fff640 1 -- 172.21.15.150:0/4117615256 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc80aab40 con 0x7f7fd8071520 2023-11-06T02:08:07.459 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.450+0000 7f7fdf477640 1 -- 172.21.15.150:0/4117615256 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7fd8071520 msgr2=0x7f7fd80cdfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:07.459 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.450+0000 7f7fdf477640 1 --2- 172.21.15.150:0/4117615256 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7fd8071520 0x7f7fd80cdfa0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7fc8009870 tx=0x7f7fc801c510 comp rx=0 tx=0).stop 2023-11-06T02:08:07.459 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.450+0000 7f7fdf477640 1 -- 172.21.15.150:0/4117615256 shutdown_connections 2023-11-06T02:08:07.459 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.450+0000 7f7fdf477640 1 --2- 172.21.15.150:0/4117615256 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7fd8071520 0x7f7fd80cdfa0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:07.460 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.450+0000 7f7fdf477640 1 --2- 172.21.15.150:0/4117615256 >> v2:172.21.15.49:3300/0 conn(0x7f7fd8151800 0x7f7fd806af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:07.460 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.450+0000 7f7fdf477640 1 -- 172.21.15.150:0/4117615256 >> 172.21.15.150:0/4117615256 conn(0x7f7fd80cf380 msgr2=0x7f7fd80cf790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:07.460 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.450+0000 7f7fdf477640 1 -- 172.21.15.150:0/4117615256 shutdown_connections 2023-11-06T02:08:07.460 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.450+0000 7f7fdf477640 1 -- 172.21.15.150:0/4117615256 wait complete. 2023-11-06T02:08:07.461 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.454+0000 7f7fdf477640 1 Processor -- start 2023-11-06T02:08:07.461 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.454+0000 7f7fdf477640 1 -- start start 2023-11-06T02:08:07.461 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.454+0000 7f7fdf477640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7fd8071520 0x7f7fd814bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:07.461 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.454+0000 7f7fdf477640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7fd8151800 0x7f7fd814a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:07.461 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.454+0000 7f7fdf477640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7fd8151be0 0x7f7fd8134670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:07.462 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.454+0000 7f7fdf477640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd814ae80 con 0x7f7fd8071520 2023-11-06T02:08:07.462 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.454+0000 7f7fdf477640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd814b160 con 0x7f7fd8151be0 2023-11-06T02:08:07.462 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.454+0000 7f7fdf477640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd814b440 con 0x7f7fd8151800 2023-11-06T02:08:07.462 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.454+0000 7f7fdda14640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7fd8151be0 0x7f7fd8134670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:07.463 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.454+0000 7f7fdda14640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7fd8151be0 0x7f7fd8134670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:43892/0 (socket says 172.21.15.150:43892) 2023-11-06T02:08:07.463 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.454+0000 7f7fdda14640 1 -- 172.21.15.150:0/1827475072 learned_addr learned my addr 172.21.15.150:0/1827475072 (peer_addr_for_me v2:172.21.15.150:0/0) 2023-11-06T02:08:07.463 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.454+0000 7f7fdca12640 1 --2- 172.21.15.150:0/1827475072 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7fd8151800 0x7f7fd814a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:07.463 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.454+0000 7f7fdd213640 1 --2- 172.21.15.150:0/1827475072 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7fd8071520 0x7f7fd814bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:07.464 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.454+0000 7f7fdca12640 1 -- 172.21.15.150:0/1827475072 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7fd8151be0 msgr2=0x7f7fd8134670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:07.464 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.454+0000 7f7fdca12640 1 --2- 172.21.15.150:0/1827475072 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7fd8151be0 0x7f7fd8134670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:07.464 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.454+0000 7f7fdca12640 1 -- 172.21.15.150:0/1827475072 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7fd8071520 msgr2=0x7f7fd814bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:07.464 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.454+0000 7f7fdca12640 1 --2- 172.21.15.150:0/1827475072 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7fd8071520 0x7f7fd814bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:07.464 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.454+0000 7f7fdca12640 1 -- 172.21.15.150:0/1827475072 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fc80094d0 con 0x7f7fd8151800 2023-11-06T02:08:07.465 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.454+0000 7f7fdd213640 1 --2- 172.21.15.150:0/1827475072 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7fd8071520 0x7f7fd814bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:08:07.465 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.454+0000 7f7fdda14640 1 --2- 172.21.15.150:0/1827475072 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7fd8151be0 0x7f7fd8134670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T02:08:07.465 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.454+0000 7f7fdca12640 1 --2- 172.21.15.150:0/1827475072 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7fd8151800 0x7f7fd814a530 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc000f70 tx=0x7f7fcc00c820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:07.465 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.454+0000 7f7fd5ffb640 1 -- 172.21.15.150:0/1827475072 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fcc0052d0 con 0x7f7fd8151800 2023-11-06T02:08:07.465 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.454+0000 7f7fd5ffb640 1 -- 172.21.15.150:0/1827475072 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fcc005430 con 0x7f7fd8151800 2023-11-06T02:08:07.465 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.454+0000 7f7fdf477640 1 -- 172.21.15.150:0/1827475072 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fd814b720 con 0x7f7fd8151800 2023-11-06T02:08:07.466 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.454+0000 7f7fd5ffb640 1 -- 172.21.15.150:0/1827475072 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fcc003840 con 0x7f7fd8151800 2023-11-06T02:08:07.466 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.454+0000 7f7fdf477640 1 -- 172.21.15.150:0/1827475072 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fd814aa70 con 0x7f7fd8151800 2023-11-06T02:08:07.466 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.454+0000 7f7fd5ffb640 1 -- 172.21.15.150:0/1827475072 <== mon.2 v2:172.21.15.150:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f7fcc003ca0 con 0x7f7fd8151800 2023-11-06T02:08:07.466 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.454+0000 7f7fd5ffb640 1 -- 172.21.15.150:0/1827475072 <== mon.2 v2:172.21.15.150:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f7fcc01c440 con 0x7f7fd8151800 2023-11-06T02:08:07.467 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.454+0000 7f7fdf477640 1 -- 172.21.15.150:0/1827475072 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fd8074040 con 0x7f7fd8151800 2023-11-06T02:08:07.467 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.458+0000 7f7fd5ffb640 1 -- 172.21.15.150:0/1827475072 <== mon.2 v2:172.21.15.150:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f7fcc017070 con 0x7f7fd8151800 2023-11-06T02:08:07.468 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-06T02:08:07.466+0000 7f4792183180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-06T02:08:07.525 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.526+0000 7f7fdf477640 1 -- 172.21.15.150:0/1827475072 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_command({"prefix": "osd new", "uuid": "c8b31f85-a757-43ed-9419-351acc46611b", "id": 1} v 0) v1 -- 0x7f7fd814bbd0 con 0x7f7fd8151800 2023-11-06T02:08:07.531 INFO:tasks.ceph.mgr.y.smithi049.stderr:2023-11-06T02:08:07.537+0000 7f148018b180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-06T02:08:07.535 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.538+0000 7f7fd5ffb640 1 -- 172.21.15.150:0/1827475072 <== mon.2 v2:172.21.15.150:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c8b31f85-a757-43ed-9419-351acc46611b", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f7fcc011630 con 0x7f7fd8151800 2023-11-06T02:08:07.536 INFO:teuthology.orchestra.run.smithi150.stdout:1 2023-11-06T02:08:07.537 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.538+0000 7f7fdf477640 1 -- 172.21.15.150:0/1827475072 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7fd8151800 msgr2=0x7f7fd814a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:07.538 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.538+0000 7f7fdf477640 1 --2- 172.21.15.150:0/1827475072 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7fd8151800 0x7f7fd814a530 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc000f70 tx=0x7f7fcc00c820 comp rx=0 tx=0).stop 2023-11-06T02:08:07.538 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.538+0000 7f7fdf477640 1 -- 172.21.15.150:0/1827475072 shutdown_connections 2023-11-06T02:08:07.539 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.538+0000 7f7fdf477640 1 --2- 172.21.15.150:0/1827475072 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7fd8151be0 0x7f7fd8134670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:07.539 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.538+0000 7f7fdf477640 1 --2- 172.21.15.150:0/1827475072 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7fd8151800 0x7f7fd814a530 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:07.539 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.538+0000 7f7fdf477640 1 --2- 172.21.15.150:0/1827475072 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7fd8071520 0x7f7fd814bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:07.540 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.538+0000 7f7fdf477640 1 -- 172.21.15.150:0/1827475072 >> 172.21.15.150:0/1827475072 conn(0x7f7fd80cf380 msgr2=0x7f7fd815c690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:07.540 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.538+0000 7f7fdf477640 1 -- 172.21.15.150:0/1827475072 shutdown_connections 2023-11-06T02:08:07.541 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.538+0000 7f7fdf477640 1 -- 172.21.15.150:0/1827475072 wait complete. 2023-11-06T02:08:07.553 DEBUG:teuthology.orchestra.run.smithi150:> sudo ceph --cluster ceph osd new 555b3bf3-0a70-4b91-8efd-e4204b5fc26e 2 2023-11-06T02:08:07.665 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.666+0000 7f62ee132640 1 Processor -- start 2023-11-06T02:08:07.666 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.666+0000 7f62ee132640 1 -- start start 2023-11-06T02:08:07.668 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.666+0000 7f62ee132640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f62e80cb2d0 0x7f62e80cca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:07.669 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.666+0000 7f62ee132640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62e80cc5c0 con 0x7f62e815a9e0 2023-11-06T02:08:07.669 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.666+0000 7f62ee132640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62e80cb760 con 0x7f62e80b04e0 2023-11-06T02:08:07.669 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.666+0000 7f62ee132640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62e80cb9b0 con 0x7f62e80cb2d0 2023-11-06T02:08:07.669 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.666+0000 7f62e7fff640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f62e80b04e0 0x7f62e80b08f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:45408/0 (socket says 172.21.15.150:45408) 2023-11-06T02:08:07.669 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.666+0000 7f62e7fff640 1 -- 172.21.15.150:0/4199160870 learned_addr learned my addr 172.21.15.150:0/4199160870 (peer_addr_for_me v1:172.21.15.150:0/0) 2023-11-06T02:08:07.670 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.670+0000 7f62e6ffd640 1 --2- 172.21.15.150:0/4199160870 >> v2:172.21.15.49:3300/0 conn(0x7f62e80cb2d0 0x7f62e80cca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:07.670 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.670+0000 7f62e67fc640 1 -- 172.21.15.150:0/4199160870 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2449084984 0 0) 0x7f62e80cb760 con 0x7f62e80b04e0 2023-11-06T02:08:07.670 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.670+0000 7f62e67fc640 1 -- 172.21.15.150:0/4199160870 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62d4003530 con 0x7f62e80b04e0 2023-11-06T02:08:07.670 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.670+0000 7f62e67fc640 1 -- 172.21.15.150:0/4199160870 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 398613267 0 0) 0x7f62e80cc5c0 con 0x7f62e815a9e0 2023-11-06T02:08:07.670 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.670+0000 7f62e67fc640 1 -- 172.21.15.150:0/4199160870 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62e80cb760 con 0x7f62e815a9e0 2023-11-06T02:08:07.671 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.670+0000 7f62e67fc640 1 -- 172.21.15.150:0/4199160870 <== mon.1 v1:172.21.15.150:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2022469604 0 0) 0x7f62d4003530 con 0x7f62e80b04e0 2023-11-06T02:08:07.671 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.670+0000 7f62e67fc640 1 -- 172.21.15.150:0/4199160870 >> v2:172.21.15.49:3300/0 conn(0x7f62e80cb2d0 msgr2=0x7f62e80cca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:07.671 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.670+0000 7f62e67fc640 1 --2- 172.21.15.150:0/4199160870 >> v2:172.21.15.49:3300/0 conn(0x7f62e80cb2d0 0x7f62e80cca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:07.671 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.670+0000 7f62e67fc640 1 -- 172.21.15.150:0/4199160870 >> v1:172.21.15.49:6789/0 conn(0x7f62e815a9e0 legacy=0x7f62e80afbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:07.671 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.670+0000 7f62e67fc640 1 -- 172.21.15.150:0/4199160870 --> v1:172.21.15.150:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62e80cbc90 con 0x7f62e80b04e0 2023-11-06T02:08:07.672 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.670+0000 7f62e67fc640 1 -- 172.21.15.150:0/4199160870 <== mon.1 v1:172.21.15.150:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7f62e0001a00 con 0x7f62e80b04e0 2023-11-06T02:08:07.672 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.670+0000 7f62e67fc640 1 -- 172.21.15.150:0/4199160870 >> v1:172.21.15.150:6789/0 conn(0x7f62e80b04e0 legacy=0x7f62e80b08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:07.672 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.670+0000 7f62e67fc640 1 --2- 172.21.15.150:0/4199160870 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62d4003b10 0x7f62d4003f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:07.672 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.670+0000 7f62e67fc640 1 --2- 172.21.15.150:0/4199160870 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f62d40047d0 0x7f62d4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:07.672 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.670+0000 7f62e67fc640 1 --2- 172.21.15.150:0/4199160870 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f62d4005190 0x7f62d400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:07.673 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.670+0000 7f62e67fc640 1 -- 172.21.15.150:0/4199160870 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62e80cb9b0 con 0x7f62d4003b10 2023-11-06T02:08:07.673 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.670+0000 7f62e67fc640 1 -- 172.21.15.150:0/4199160870 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d400c010 con 0x7f62d40047d0 2023-11-06T02:08:07.673 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.670+0000 7f62e67fc640 1 -- 172.21.15.150:0/4199160870 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62d400c1d0 con 0x7f62d4005190 2023-11-06T02:08:07.673 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.670+0000 7f62e6ffd640 1 --2- 172.21.15.150:0/4199160870 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f62d40047d0 0x7f62d4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:07.673 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.670+0000 7f62e7fff640 1 --2- 172.21.15.150:0/4199160870 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f62d4005190 0x7f62d400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:07.673 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.670+0000 7f62e77fe640 1 --2- 172.21.15.150:0/4199160870 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62d4003b10 0x7f62d4003f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:07.674 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.670+0000 7f62e6ffd640 1 -- 172.21.15.150:0/4199160870 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f62d4005190 msgr2=0x7f62d400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:07.674 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.670+0000 7f62e6ffd640 1 --2- 172.21.15.150:0/4199160870 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f62d4005190 0x7f62d400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:07.674 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.670+0000 7f62e6ffd640 1 -- 172.21.15.150:0/4199160870 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62d4003b10 msgr2=0x7f62d4003f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:07.674 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.670+0000 7f62e6ffd640 1 --2- 172.21.15.150:0/4199160870 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62d4003b10 0x7f62d4003f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:07.674 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.670+0000 7f62e6ffd640 1 -- 172.21.15.150:0/4199160870 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62d4003530 con 0x7f62d40047d0 2023-11-06T02:08:07.675 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.670+0000 7f62e7fff640 1 -- 172.21.15.150:0/4199160870 reap_dead start 2023-11-06T02:08:07.678 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.670+0000 7f62e6ffd640 1 --2- 172.21.15.150:0/4199160870 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f62d40047d0 0x7f62d4004c50 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f62d8009ae0 tx=0x7f62d801ca20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65763f7d56ad0173 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:07.678 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.670+0000 7f62e67fc640 1 -- 172.21.15.150:0/4199160870 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d80a55f0 con 0x7f62d40047d0 2023-11-06T02:08:07.678 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.670+0000 7f62e67fc640 1 -- 172.21.15.150:0/4199160870 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62d80a5750 con 0x7f62d40047d0 2023-11-06T02:08:07.679 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.670+0000 7f62e67fc640 1 -- 172.21.15.150:0/4199160870 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d80a5b00 con 0x7f62d40047d0 2023-11-06T02:08:07.679 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.670+0000 7f62ee132640 1 -- 172.21.15.150:0/4199160870 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f62d40047d0 msgr2=0x7f62d4004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:07.679 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.670+0000 7f62ee132640 1 --2- 172.21.15.150:0/4199160870 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f62d40047d0 0x7f62d4004c50 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f62d8009ae0 tx=0x7f62d801ca20 comp rx=0 tx=0).stop 2023-11-06T02:08:07.679 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.670+0000 7f62ee132640 1 -- 172.21.15.150:0/4199160870 shutdown_connections 2023-11-06T02:08:07.679 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.670+0000 7f62ee132640 1 --2- 172.21.15.150:0/4199160870 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f62d40047d0 0x7f62d4004c50 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:07.680 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.670+0000 7f62ee132640 1 -- 172.21.15.150:0/4199160870 >> 172.21.15.150:0/4199160870 conn(0x7f62e8098a40 msgr2=0x7f62e80cf860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:07.680 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.670+0000 7f62ee132640 1 -- 172.21.15.150:0/4199160870 shutdown_connections 2023-11-06T02:08:07.680 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.670+0000 7f62ee132640 1 -- 172.21.15.150:0/4199160870 wait complete. 2023-11-06T02:08:07.680 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.674+0000 7f62ee132640 1 Processor -- start 2023-11-06T02:08:07.680 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.674+0000 7f62ee132640 1 -- start start 2023-11-06T02:08:07.680 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.674+0000 7f62ee132640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f62d40047d0 0x7f62e81526a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:07.681 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.674+0000 7f62ee132640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f62e815d7c0 0x7f62e815dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:07.681 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.674+0000 7f62ee132640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62e815b460 0x7f62e815b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:07.681 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.674+0000 7f62ee132640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62e815e180 con 0x7f62e815b460 2023-11-06T02:08:07.681 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.674+0000 7f62ee132640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62e81f3030 con 0x7f62d40047d0 2023-11-06T02:08:07.681 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.674+0000 7f62ee132640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62e80c46a0 con 0x7f62e815d7c0 2023-11-06T02:08:07.682 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.674+0000 7f62e77fe640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f62d40047d0 0x7f62e81526a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:07.682 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.674+0000 7f62e6ffd640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f62e815d7c0 0x7f62e815dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:07.682 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.674+0000 7f62e77fe640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f62d40047d0 0x7f62e81526a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:43910/0 (socket says 172.21.15.150:43910) 2023-11-06T02:08:07.682 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.674+0000 7f62e77fe640 1 -- 172.21.15.150:0/2271741717 learned_addr learned my addr 172.21.15.150:0/2271741717 (peer_addr_for_me v2:172.21.15.150:0/0) 2023-11-06T02:08:07.682 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.674+0000 7f62e7fff640 1 --2- 172.21.15.150:0/2271741717 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62e815b460 0x7f62e815b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:07.683 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.674+0000 7f62e77fe640 1 -- 172.21.15.150:0/2271741717 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f62e815d7c0 msgr2=0x7f62e815dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:07.683 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.674+0000 7f62e77fe640 1 --2- 172.21.15.150:0/2271741717 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f62e815d7c0 0x7f62e815dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:07.683 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.674+0000 7f62e77fe640 1 -- 172.21.15.150:0/2271741717 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62e815b460 msgr2=0x7f62e815b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:07.683 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.674+0000 7f62e77fe640 1 --2- 172.21.15.150:0/2271741717 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62e815b460 0x7f62e815b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:07.683 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.674+0000 7f62e77fe640 1 -- 172.21.15.150:0/2271741717 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62e80b0140 con 0x7f62d40047d0 2023-11-06T02:08:07.684 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.674+0000 7f62e7fff640 1 --2- 172.21.15.150:0/2271741717 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62e815b460 0x7f62e815b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:08:07.684 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.674+0000 7f62e77fe640 1 --2- 172.21.15.150:0/2271741717 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f62d40047d0 0x7f62e81526a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f62d000e4b0 tx=0x7f62d000e970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:07.684 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.674+0000 7f62dffff640 1 -- 172.21.15.150:0/2271741717 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d00052e0 con 0x7f62d40047d0 2023-11-06T02:08:07.684 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.674+0000 7f62dffff640 1 -- 172.21.15.150:0/2271741717 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62d0005440 con 0x7f62d40047d0 2023-11-06T02:08:07.684 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.674+0000 7f62ee132640 1 -- 172.21.15.150:0/2271741717 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62e815c190 con 0x7f62d40047d0 2023-11-06T02:08:07.684 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.674+0000 7f62dffff640 1 -- 172.21.15.150:0/2271741717 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d0003840 con 0x7f62d40047d0 2023-11-06T02:08:07.685 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.674+0000 7f62ee132640 1 -- 172.21.15.150:0/2271741717 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62e815c750 con 0x7f62d40047d0 2023-11-06T02:08:07.685 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.674+0000 7f62dffff640 1 -- 172.21.15.150:0/2271741717 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f62d0003ca0 con 0x7f62d40047d0 2023-11-06T02:08:07.685 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.674+0000 7f62dffff640 1 -- 172.21.15.150:0/2271741717 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f62d001ddc0 con 0x7f62d40047d0 2023-11-06T02:08:07.685 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.674+0000 7f62ee132640 1 -- 172.21.15.150:0/2271741717 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62e80c2c60 con 0x7f62d40047d0 2023-11-06T02:08:07.685 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.678+0000 7f62dffff640 1 -- 172.21.15.150:0/2271741717 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f62d001d5a0 con 0x7f62d40047d0 2023-11-06T02:08:07.791 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.790+0000 7f62ee132640 1 -- 172.21.15.150:0/2271741717 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd new", "uuid": "555b3bf3-0a70-4b91-8efd-e4204b5fc26e", "id": 2} v 0) v1 -- 0x7f62e80c4860 con 0x7f62d40047d0 2023-11-06T02:08:07.801 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.802+0000 7f62dffff640 1 -- 172.21.15.150:0/2271741717 <== mon.1 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "555b3bf3-0a70-4b91-8efd-e4204b5fc26e", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f62d001d8d0 con 0x7f62d40047d0 2023-11-06T02:08:07.802 INFO:teuthology.orchestra.run.smithi150.stdout:2 2023-11-06T02:08:07.802 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.802+0000 7f62ee132640 1 -- 172.21.15.150:0/2271741717 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f62d40047d0 msgr2=0x7f62e81526a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:07.803 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.802+0000 7f62ee132640 1 --2- 172.21.15.150:0/2271741717 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f62d40047d0 0x7f62e81526a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f62d000e4b0 tx=0x7f62d000e970 comp rx=0 tx=0).stop 2023-11-06T02:08:07.803 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.802+0000 7f62ee132640 1 -- 172.21.15.150:0/2271741717 shutdown_connections 2023-11-06T02:08:07.803 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.802+0000 7f62ee132640 1 --2- 172.21.15.150:0/2271741717 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62e815b460 0x7f62e815b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:07.803 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.802+0000 7f62ee132640 1 --2- 172.21.15.150:0/2271741717 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f62e815d7c0 0x7f62e815dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:07.803 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.802+0000 7f62ee132640 1 --2- 172.21.15.150:0/2271741717 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f62d40047d0 0x7f62e81526a0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:07.804 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.802+0000 7f62ee132640 1 -- 172.21.15.150:0/2271741717 >> 172.21.15.150:0/2271741717 conn(0x7f62e8098a40 msgr2=0x7f62e80aeec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:07.804 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.802+0000 7f62ee132640 1 -- 172.21.15.150:0/2271741717 shutdown_connections 2023-11-06T02:08:07.804 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.802+0000 7f62ee132640 1 -- 172.21.15.150:0/2271741717 wait complete. 2023-11-06T02:08:07.818 DEBUG:teuthology.orchestra.run.smithi150:> sudo ceph --cluster ceph osd new 8d4e5d77-03ef-4b9b-bfcb-5ae93a46795c 3 2023-11-06T02:08:07.932 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.934+0000 7f5b6d9e2640 1 Processor -- start 2023-11-06T02:08:07.933 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.934+0000 7f5b6d9e2640 1 -- start start 2023-11-06T02:08:07.934 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.934+0000 7f5b6d9e2640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5b6815e0c0 0x7f5b680ce840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:07.935 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.934+0000 7f5b6d9e2640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f5b680b16e0 0x7f5b680b1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:07.935 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.934+0000 7f5b6d9e2640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b680d1430 con 0x7f5b6815e4a0 2023-11-06T02:08:07.935 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.934+0000 7f5b6d9e2640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b680cd8d0 con 0x7f5b680b16e0 2023-11-06T02:08:07.935 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.934+0000 7f5b6d9e2640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b680b20d0 con 0x7f5b6815e0c0 2023-11-06T02:08:07.935 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.934+0000 7f5b66ffd640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5b6815e0c0 0x7f5b680ce840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:07.936 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.934+0000 7f5b667fc640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f5b6815e4a0 0x7f5b680cdcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:45410/0 (socket says 172.21.15.150:45410) 2023-11-06T02:08:07.936 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.934+0000 7f5b667fc640 1 -- 172.21.15.150:0/4165796362 learned_addr learned my addr 172.21.15.150:0/4165796362 (peer_addr_for_me v1:172.21.15.150:0/0) 2023-11-06T02:08:07.936 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.934+0000 7f5b677fe640 1 --2- 172.21.15.150:0/4165796362 >> v2:172.21.15.150:3300/0 conn(0x7f5b680b16e0 0x7f5b680b1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:07.936 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.934+0000 7f5b65ffb640 1 -- 172.21.15.150:0/4165796362 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1211761011 0 0) 0x7f5b680d1430 con 0x7f5b6815e4a0 2023-11-06T02:08:07.936 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.934+0000 7f5b65ffb640 1 -- 172.21.15.150:0/4165796362 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b50003530 con 0x7f5b6815e4a0 2023-11-06T02:08:07.936 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.934+0000 7f5b66ffd640 1 -- 172.21.15.150:0/4165796362 >> v2:172.21.15.150:3300/0 conn(0x7f5b680b16e0 msgr2=0x7f5b680b1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:07.937 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.934+0000 7f5b66ffd640 1 --2- 172.21.15.150:0/4165796362 >> v2:172.21.15.150:3300/0 conn(0x7f5b680b16e0 0x7f5b680b1b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:07.937 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.934+0000 7f5b66ffd640 1 -- 172.21.15.150:0/4165796362 >> v1:172.21.15.150:6789/0 conn(0x7f5b6815e4a0 legacy=0x7f5b680cdcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:07.937 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.934+0000 7f5b66ffd640 1 -- 172.21.15.150:0/4165796362 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b680d1710 con 0x7f5b6815e0c0 2023-11-06T02:08:07.937 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.934+0000 7f5b677fe640 1 --2- 172.21.15.150:0/4165796362 >> v2:172.21.15.150:3300/0 conn(0x7f5b680b16e0 0x7f5b680b1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T02:08:07.940 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.938+0000 7f5b66ffd640 1 --2- 172.21.15.150:0/4165796362 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5b6815e0c0 0x7f5b680ce840 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5b580097c0 tx=0x7f5b5801c3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49a3d0ca9d334c2b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:07.941 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.938+0000 7f5b65ffb640 1 -- 172.21.15.150:0/4165796362 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b580ab600 con 0x7f5b6815e0c0 2023-11-06T02:08:07.941 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.938+0000 7f5b65ffb640 1 -- 172.21.15.150:0/4165796362 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b580a7050 con 0x7f5b6815e0c0 2023-11-06T02:08:07.941 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.938+0000 7f5b65ffb640 1 -- 172.21.15.150:0/4165796362 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b580ab980 con 0x7f5b6815e0c0 2023-11-06T02:08:07.941 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.938+0000 7f5b6d9e2640 1 -- 172.21.15.150:0/4165796362 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5b6815e0c0 msgr2=0x7f5b680ce840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:07.942 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.938+0000 7f5b6d9e2640 1 --2- 172.21.15.150:0/4165796362 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5b6815e0c0 0x7f5b680ce840 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5b580097c0 tx=0x7f5b5801c3f0 comp rx=0 tx=0).stop 2023-11-06T02:08:07.942 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.938+0000 7f5b6d9e2640 1 -- 172.21.15.150:0/4165796362 shutdown_connections 2023-11-06T02:08:07.942 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.938+0000 7f5b6d9e2640 1 --2- 172.21.15.150:0/4165796362 >> v2:172.21.15.150:3300/0 conn(0x7f5b680b16e0 0x7f5b680b1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:07.942 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.938+0000 7f5b6d9e2640 1 --2- 172.21.15.150:0/4165796362 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5b6815e0c0 0x7f5b680ce840 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:07.943 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.938+0000 7f5b6d9e2640 1 -- 172.21.15.150:0/4165796362 >> 172.21.15.150:0/4165796362 conn(0x7f5b68099c50 msgr2=0x7f5b680d0940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:07.943 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.938+0000 7f5b6d9e2640 1 -- 172.21.15.150:0/4165796362 shutdown_connections 2023-11-06T02:08:07.943 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.938+0000 7f5b6d9e2640 1 -- 172.21.15.150:0/4165796362 wait complete. 2023-11-06T02:08:07.944 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.938+0000 7f5b6d9e2640 1 Processor -- start 2023-11-06T02:08:07.944 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.938+0000 7f5b6d9e2640 1 -- start start 2023-11-06T02:08:07.944 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.942+0000 7f5b6d9e2640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5b680b16e0 0x7f5b68158c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:07.944 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.942+0000 7f5b6d9e2640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b6815e0c0 0x7f5b680c6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:07.944 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.942+0000 7f5b6d9e2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b6815e4a0 0x7f5b680addb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:07.945 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.942+0000 7f5b6d9e2640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b681507a0 con 0x7f5b6815e4a0 2023-11-06T02:08:07.945 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.942+0000 7f5b6d9e2640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b680c99d0 con 0x7f5b6815e0c0 2023-11-06T02:08:07.945 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.942+0000 7f5b6d9e2640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b680c9cb0 con 0x7f5b680b16e0 2023-11-06T02:08:07.945 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.942+0000 7f5b66ffd640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5b680b16e0 0x7f5b68158c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:07.945 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.942+0000 7f5b667fc640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b6815e0c0 0x7f5b680c6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:07.946 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.942+0000 7f5b66ffd640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5b680b16e0 0x7f5b68158c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.150:55226/0 (socket says 172.21.15.150:55226) 2023-11-06T02:08:07.946 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.942+0000 7f5b66ffd640 1 -- 172.21.15.150:0/305515916 learned_addr learned my addr 172.21.15.150:0/305515916 (peer_addr_for_me v2:172.21.15.150:0/0) 2023-11-06T02:08:07.946 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.942+0000 7f5b677fe640 1 --2- 172.21.15.150:0/305515916 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b6815e4a0 0x7f5b680addb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:07.946 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.942+0000 7f5b667fc640 1 -- 172.21.15.150:0/305515916 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5b680b16e0 msgr2=0x7f5b68158c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:07.947 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.942+0000 7f5b667fc640 1 --2- 172.21.15.150:0/305515916 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5b680b16e0 0x7f5b68158c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:07.947 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.942+0000 7f5b667fc640 1 -- 172.21.15.150:0/305515916 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b6815e4a0 msgr2=0x7f5b680addb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:07.947 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.942+0000 7f5b667fc640 1 --2- 172.21.15.150:0/305515916 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b6815e4a0 0x7f5b680addb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:07.947 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.942+0000 7f5b667fc640 1 -- 172.21.15.150:0/305515916 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b60009a70 con 0x7f5b6815e0c0 2023-11-06T02:08:07.947 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.942+0000 7f5b66ffd640 1 --2- 172.21.15.150:0/305515916 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5b680b16e0 0x7f5b68158c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T02:08:07.948 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.942+0000 7f5b677fe640 1 --2- 172.21.15.150:0/305515916 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b6815e4a0 0x7f5b680addb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:08:07.948 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.942+0000 7f5b667fc640 1 --2- 172.21.15.150:0/305515916 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b6815e0c0 0x7f5b680c6920 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c00c400 tx=0x7f5b5c00c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:07.948 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.942+0000 7f5b4ffff640 1 -- 172.21.15.150:0/305515916 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b5c005cf0 con 0x7f5b6815e0c0 2023-11-06T02:08:07.948 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.942+0000 7f5b4ffff640 1 -- 172.21.15.150:0/305515916 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b5c005e50 con 0x7f5b6815e0c0 2023-11-06T02:08:07.948 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.942+0000 7f5b4ffff640 1 -- 172.21.15.150:0/305515916 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b5c0072d0 con 0x7f5b6815e0c0 2023-11-06T02:08:07.948 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.942+0000 7f5b6d9e2640 1 -- 172.21.15.150:0/305515916 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b58009420 con 0x7f5b6815e0c0 2023-11-06T02:08:07.949 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.942+0000 7f5b6d9e2640 1 -- 172.21.15.150:0/305515916 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b68150cd0 con 0x7f5b6815e0c0 2023-11-06T02:08:07.950 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.942+0000 7f5b4ffff640 1 -- 172.21.15.150:0/305515916 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f5b5c01c440 con 0x7f5b6815e0c0 2023-11-06T02:08:07.950 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.942+0000 7f5b4ffff640 1 -- 172.21.15.150:0/305515916 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f5b5c012890 con 0x7f5b6815e0c0 2023-11-06T02:08:07.950 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.942+0000 7f5b6d9e2640 1 -- 172.21.15.150:0/305515916 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b680c9e70 con 0x7f5b6815e0c0 2023-11-06T02:08:07.950 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:07.946+0000 7f5b4ffff640 1 -- 172.21.15.150:0/305515916 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f5b680c9e70 con 0x7f5b6815e0c0 2023-11-06T02:08:08.011 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.010+0000 7f5b6d9e2640 1 -- 172.21.15.150:0/305515916 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd new", "uuid": "8d4e5d77-03ef-4b9b-bfcb-5ae93a46795c", "id": 3} v 0) v1 -- 0x7f5b68150ef0 con 0x7f5b6815e0c0 2023-11-06T02:08:08.020 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.022+0000 7f5b4ffff640 1 -- 172.21.15.150:0/305515916 <== mon.1 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "8d4e5d77-03ef-4b9b-bfcb-5ae93a46795c", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f5b5c021060 con 0x7f5b6815e0c0 2023-11-06T02:08:08.021 INFO:teuthology.orchestra.run.smithi150.stdout:3 2023-11-06T02:08:08.022 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.022+0000 7f5b6d9e2640 1 -- 172.21.15.150:0/305515916 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b6815e0c0 msgr2=0x7f5b680c6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:08.022 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.022+0000 7f5b6d9e2640 1 --2- 172.21.15.150:0/305515916 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b6815e0c0 0x7f5b680c6920 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c00c400 tx=0x7f5b5c00c8c0 comp rx=0 tx=0).stop 2023-11-06T02:08:08.022 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.022+0000 7f5b6d9e2640 1 -- 172.21.15.150:0/305515916 shutdown_connections 2023-11-06T02:08:08.022 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.022+0000 7f5b6d9e2640 1 --2- 172.21.15.150:0/305515916 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b6815e4a0 0x7f5b680addb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:08.023 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.022+0000 7f5b6d9e2640 1 --2- 172.21.15.150:0/305515916 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b6815e0c0 0x7f5b680c6920 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:08.023 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.022+0000 7f5b6d9e2640 1 --2- 172.21.15.150:0/305515916 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5b680b16e0 0x7f5b68158c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:08.023 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.022+0000 7f5b6d9e2640 1 -- 172.21.15.150:0/305515916 >> 172.21.15.150:0/305515916 conn(0x7f5b68099c50 msgr2=0x7f5b680b23c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:08.023 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.022+0000 7f5b6d9e2640 1 -- 172.21.15.150:0/305515916 shutdown_connections 2023-11-06T02:08:08.023 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.022+0000 7f5b6d9e2640 1 -- 172.21.15.150:0/305515916 wait complete. 2023-11-06T02:08:08.037 DEBUG:teuthology.orchestra.run.smithi150:> sudo ceph --cluster ceph osd new dba03ad4-68a3-4cf3-9218-ac611f02eacf 4 2023-11-06T02:08:08.103 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-06T02:08:08.106+0000 7f4792183180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-06T02:08:08.137 INFO:tasks.ceph.mgr.y.smithi049.stderr:2023-11-06T02:08:08.141+0000 7f148018b180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-06T02:08:08.149 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.150+0000 7fbf9b4dd640 1 Processor -- start 2023-11-06T02:08:08.149 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.150+0000 7fbf9b4dd640 1 -- start start 2023-11-06T02:08:08.151 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.150+0000 7fbf9b4dd640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fbf940c9600 0x7fbf940cb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:08.151 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.150+0000 7fbf9b4dd640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf94065930 0x7fbf9406e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:08.151 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.150+0000 7fbf9b4dd640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf940cabf0 con 0x7fbf940cb5c0 2023-11-06T02:08:08.151 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.150+0000 7fbf9b4dd640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf9406def0 con 0x7fbf940c9600 2023-11-06T02:08:08.152 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.150+0000 7fbf9b4dd640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9415b970 con 0x7fbf94065930 2023-11-06T02:08:08.152 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.150+0000 7fbf99a7a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf94065930 0x7fbf9406e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:08.152 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.150+0000 7fbf99a7a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf94065930 0x7fbf9406e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.150:55234/0 (socket says 172.21.15.150:55234) 2023-11-06T02:08:08.152 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.150+0000 7fbf99a7a640 1 -- 172.21.15.150:0/274865678 learned_addr learned my addr 172.21.15.150:0/274865678 (peer_addr_for_me v2:172.21.15.150:0/0) 2023-11-06T02:08:08.153 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.150+0000 7fbf98a78640 1 --1- 172.21.15.150:0/274865678 >> v1:172.21.15.150:6789/0 conn(0x7fbf940cb5c0 0x7fbf940c99e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:45426/0 (socket says 172.21.15.150:45426) 2023-11-06T02:08:08.154 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.150+0000 7fbf99279640 1 --2- 172.21.15.150:0/274865678 >> v2:172.21.15.49:3300/0 conn(0x7fbf940c9600 0x7fbf940cb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:08.154 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.150+0000 7fbf99a7a640 1 -- 172.21.15.150:0/274865678 >> v2:172.21.15.49:3300/0 conn(0x7fbf940c9600 msgr2=0x7fbf940cb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:08.154 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.150+0000 7fbf99a7a640 1 --2- 172.21.15.150:0/274865678 >> v2:172.21.15.49:3300/0 conn(0x7fbf940c9600 0x7fbf940cb080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:08.154 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.150+0000 7fbf99a7a640 1 -- 172.21.15.150:0/274865678 >> v1:172.21.15.150:6789/0 conn(0x7fbf940cb5c0 legacy=0x7fbf940c99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:08.155 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.150+0000 7fbf99a7a640 1 -- 172.21.15.150:0/274865678 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf940ca8c0 con 0x7fbf94065930 2023-11-06T02:08:08.155 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.150+0000 7fbf99279640 1 --2- 172.21.15.150:0/274865678 >> v2:172.21.15.49:3300/0 conn(0x7fbf940c9600 0x7fbf940cb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:08:08.158 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.154+0000 7fbf99a7a640 1 --2- 172.21.15.150:0/274865678 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf94065930 0x7fbf9406e3f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c009e50 tx=0x7fbf8c01ca10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6eab095d8374ab6 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:08.158 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.154+0000 7fbf8bfff640 1 -- 172.21.15.150:0/274865678 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf8c0a56e0 con 0x7fbf94065930 2023-11-06T02:08:08.158 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.154+0000 7fbf8bfff640 1 -- 172.21.15.150:0/274865678 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf8c0a5840 con 0x7fbf94065930 2023-11-06T02:08:08.158 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.154+0000 7fbf8bfff640 1 -- 172.21.15.150:0/274865678 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf8c0a5bd0 con 0x7fbf94065930 2023-11-06T02:08:08.159 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.154+0000 7fbf9b4dd640 1 -- 172.21.15.150:0/274865678 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf94065930 msgr2=0x7fbf9406e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:08.159 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.154+0000 7fbf9b4dd640 1 --2- 172.21.15.150:0/274865678 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf94065930 0x7fbf9406e3f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c009e50 tx=0x7fbf8c01ca10 comp rx=0 tx=0).stop 2023-11-06T02:08:08.159 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.154+0000 7fbf9b4dd640 1 -- 172.21.15.150:0/274865678 shutdown_connections 2023-11-06T02:08:08.159 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.154+0000 7fbf9b4dd640 1 --2- 172.21.15.150:0/274865678 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf94065930 0x7fbf9406e3f0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:08.159 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.154+0000 7fbf9b4dd640 1 --2- 172.21.15.150:0/274865678 >> v2:172.21.15.49:3300/0 conn(0x7fbf940c9600 0x7fbf940cb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:08.160 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.154+0000 7fbf9b4dd640 1 -- 172.21.15.150:0/274865678 >> 172.21.15.150:0/274865678 conn(0x7fbf940b0aa0 msgr2=0x7fbf940b0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:08.160 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.154+0000 7fbf9b4dd640 1 -- 172.21.15.150:0/274865678 shutdown_connections 2023-11-06T02:08:08.160 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.154+0000 7fbf9b4dd640 1 -- 172.21.15.150:0/274865678 wait complete. 2023-11-06T02:08:08.160 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.154+0000 7fbf9b4dd640 1 Processor -- start 2023-11-06T02:08:08.160 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.154+0000 7fbf9b4dd640 1 -- start start 2023-11-06T02:08:08.160 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.158+0000 7fbf9b4dd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf94065930 0x7fbf94153af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:08.161 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.158+0000 7fbf9b4dd640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf940c9600 0x7fbf941f71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:08.161 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.158+0000 7fbf9b4dd640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf940cb5c0 0x7fbf941f76e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:08.161 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.158+0000 7fbf9b4dd640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf940b1bd0 con 0x7fbf94065930 2023-11-06T02:08:08.161 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.158+0000 7fbf9b4dd640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf940b49d0 con 0x7fbf940c9600 2023-11-06T02:08:08.161 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.158+0000 7fbf9b4dd640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf940afd90 con 0x7fbf940cb5c0 2023-11-06T02:08:08.162 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.158+0000 7fbf98a78640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf940c9600 0x7fbf941f71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:08.162 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.158+0000 7fbf98a78640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf940c9600 0x7fbf941f71a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:43944/0 (socket says 172.21.15.150:43944) 2023-11-06T02:08:08.162 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.158+0000 7fbf98a78640 1 -- 172.21.15.150:0/1448124710 learned_addr learned my addr 172.21.15.150:0/1448124710 (peer_addr_for_me v2:172.21.15.150:0/0) 2023-11-06T02:08:08.162 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.158+0000 7fbf99a7a640 1 --2- 172.21.15.150:0/1448124710 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf940cb5c0 0x7fbf941f76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:08.162 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.158+0000 7fbf99279640 1 --2- 172.21.15.150:0/1448124710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf94065930 0x7fbf94153af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:08.163 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.158+0000 7fbf98a78640 1 -- 172.21.15.150:0/1448124710 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf940cb5c0 msgr2=0x7fbf941f76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:08.163 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.158+0000 7fbf98a78640 1 --2- 172.21.15.150:0/1448124710 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf940cb5c0 0x7fbf941f76e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:08.163 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.158+0000 7fbf98a78640 1 -- 172.21.15.150:0/1448124710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf94065930 msgr2=0x7fbf94153af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:08.163 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.158+0000 7fbf98a78640 1 --2- 172.21.15.150:0/1448124710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf94065930 0x7fbf94153af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:08.163 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.158+0000 7fbf98a78640 1 -- 172.21.15.150:0/1448124710 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf8c009b30 con 0x7fbf940c9600 2023-11-06T02:08:08.164 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.158+0000 7fbf99a7a640 1 --2- 172.21.15.150:0/1448124710 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf940cb5c0 0x7fbf941f76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T02:08:08.164 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.158+0000 7fbf98a78640 1 --2- 172.21.15.150:0/1448124710 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf940c9600 0x7fbf941f71a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fbf8400c490 tx=0x7fbf8400c950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:08.164 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.158+0000 7fbf89ffb640 1 -- 172.21.15.150:0/1448124710 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf84007480 con 0x7fbf940c9600 2023-11-06T02:08:08.165 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.158+0000 7fbf89ffb640 1 -- 172.21.15.150:0/1448124710 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf840075e0 con 0x7fbf940c9600 2023-11-06T02:08:08.165 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.158+0000 7fbf9b4dd640 1 -- 172.21.15.150:0/1448124710 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf940c9b10 con 0x7fbf940c9600 2023-11-06T02:08:08.165 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.158+0000 7fbf89ffb640 1 -- 172.21.15.150:0/1448124710 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf84004a10 con 0x7fbf940c9600 2023-11-06T02:08:08.165 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.158+0000 7fbf9b4dd640 1 -- 172.21.15.150:0/1448124710 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf940b1e20 con 0x7fbf940c9600 2023-11-06T02:08:08.165 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.158+0000 7fbf89ffb640 1 -- 172.21.15.150:0/1448124710 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fbf8401d440 con 0x7fbf940c9600 2023-11-06T02:08:08.166 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.158+0000 7fbf89ffb640 1 -- 172.21.15.150:0/1448124710 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7fbf84012950 con 0x7fbf940c9600 2023-11-06T02:08:08.166 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.158+0000 7fbf9b4dd640 1 -- 172.21.15.150:0/1448124710 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf940b17e0 con 0x7fbf940c9600 2023-11-06T02:08:08.166 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.162+0000 7fbf89ffb640 1 -- 172.21.15.150:0/1448124710 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7fbf84018070 con 0x7fbf940c9600 2023-11-06T02:08:08.167 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-06T02:08:08.162+0000 7f4792183180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-06T02:08:08.194 INFO:tasks.ceph.mgr.y.smithi049.stderr:2023-11-06T02:08:08.197+0000 7f148018b180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-06T02:08:08.212 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-06T02:08:08.214+0000 7f4792183180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-06T02:08:08.226 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.226+0000 7fbf9b4dd640 1 -- 172.21.15.150:0/1448124710 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd new", "uuid": "dba03ad4-68a3-4cf3-9218-ac611f02eacf", "id": 4} v 0) v1 -- 0x7fbf940b4f30 con 0x7fbf940c9600 2023-11-06T02:08:08.234 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.234+0000 7fbf89ffb640 1 -- 172.21.15.150:0/1448124710 <== mon.1 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "dba03ad4-68a3-4cf3-9218-ac611f02eacf", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7fbf84012d00 con 0x7fbf940c9600 2023-11-06T02:08:08.234 INFO:teuthology.orchestra.run.smithi150.stdout:4 2023-11-06T02:08:08.235 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.238+0000 7fbf9b4dd640 1 -- 172.21.15.150:0/1448124710 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf940c9600 msgr2=0x7fbf941f71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:08.236 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.238+0000 7fbf9b4dd640 1 --2- 172.21.15.150:0/1448124710 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf940c9600 0x7fbf941f71a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fbf8400c490 tx=0x7fbf8400c950 comp rx=0 tx=0).stop 2023-11-06T02:08:08.236 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.238+0000 7fbf9b4dd640 1 -- 172.21.15.150:0/1448124710 shutdown_connections 2023-11-06T02:08:08.236 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.238+0000 7fbf9b4dd640 1 --2- 172.21.15.150:0/1448124710 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf940cb5c0 0x7fbf941f76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:08.236 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.238+0000 7fbf9b4dd640 1 --2- 172.21.15.150:0/1448124710 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf940c9600 0x7fbf941f71a0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:08.236 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.238+0000 7fbf9b4dd640 1 --2- 172.21.15.150:0/1448124710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf94065930 0x7fbf94153af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:08.237 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.238+0000 7fbf9b4dd640 1 -- 172.21.15.150:0/1448124710 >> 172.21.15.150:0/1448124710 conn(0x7fbf940b0aa0 msgr2=0x7fbf9406a110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:08.237 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.238+0000 7fbf9b4dd640 1 -- 172.21.15.150:0/1448124710 shutdown_connections 2023-11-06T02:08:08.237 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.238+0000 7fbf9b4dd640 1 -- 172.21.15.150:0/1448124710 wait complete. 2023-11-06T02:08:08.250 INFO:tasks.ceph.mgr.y.smithi049.stderr:2023-11-06T02:08:08.253+0000 7f148018b180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-06T02:08:08.251 DEBUG:teuthology.orchestra.run.smithi150:> sudo ceph --cluster ceph osd new 0df9c303-c255-4640-8e91-01f1c3efe18e 5 2023-11-06T02:08:08.293 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-06T02:08:08.294+0000 7f4792183180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-06T02:08:08.330 INFO:tasks.ceph.mgr.y.smithi049.stderr:2023-11-06T02:08:08.333+0000 7f148018b180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-06T02:08:08.346 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-06T02:08:08.346+0000 7f4792183180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-06T02:08:08.358 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.358+0000 7f5a6d89b640 1 Processor -- start 2023-11-06T02:08:08.358 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.358+0000 7f5a6d89b640 1 -- start start 2023-11-06T02:08:08.359 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.362+0000 7f5a6d89b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5a680cdea0 0x7f5a680cd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:08.360 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.362+0000 7f5a6d89b640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a6815e7e0 con 0x7f5a680ce280 2023-11-06T02:08:08.360 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.362+0000 7f5a6d89b640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a68099080 con 0x7f5a680ae760 2023-11-06T02:08:08.360 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.362+0000 7f5a6d89b640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a680b1100 con 0x7f5a680cdea0 2023-11-06T02:08:08.360 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.362+0000 7f5a66ffd640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5a680cdea0 0x7f5a680cd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:08.360 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.362+0000 7f5a677fe640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f5a680ae760 0x7f5a6815e0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:45436/0 (socket says 172.21.15.150:45436) 2023-11-06T02:08:08.361 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.362+0000 7f5a677fe640 1 -- 172.21.15.150:0/1690051212 learned_addr learned my addr 172.21.15.150:0/1690051212 (peer_addr_for_me v1:172.21.15.150:0/0) 2023-11-06T02:08:08.361 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.362+0000 7f5a66ffd640 1 -- 172.21.15.150:0/1690051212 >> v1:172.21.15.150:6789/0 conn(0x7f5a680ae760 legacy=0x7f5a6815e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:08.361 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.362+0000 7f5a66ffd640 1 -- 172.21.15.150:0/1690051212 >> v1:172.21.15.49:6789/0 conn(0x7f5a680ce280 legacy=0x7f5a6815d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:08.361 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.362+0000 7f5a65ffb640 1 -- 172.21.15.150:0/1690051212 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 85870029 0 0) 0x7f5a68099080 con 0x7f5a680ae760 2023-11-06T02:08:08.361 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.362+0000 7f5a66ffd640 1 -- 172.21.15.150:0/1690051212 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a680b0b40 con 0x7f5a680cdea0 2023-11-06T02:08:08.363 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.362+0000 7f5a66ffd640 1 --2- 172.21.15.150:0/1690051212 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5a680cdea0 0x7f5a680cd430 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5a580097f0 tx=0x7f5a5801c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e498fe2f5e4f4334 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:08.363 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.362+0000 7f5a65ffb640 1 -- 172.21.15.150:0/1690051212 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a580ab680 con 0x7f5a680cdea0 2023-11-06T02:08:08.363 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.362+0000 7f5a65ffb640 1 -- 172.21.15.150:0/1690051212 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a580ab7e0 con 0x7f5a680cdea0 2023-11-06T02:08:08.363 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.362+0000 7f5a65ffb640 1 -- 172.21.15.150:0/1690051212 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a580abb70 con 0x7f5a680cdea0 2023-11-06T02:08:08.364 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.362+0000 7f5a6d89b640 1 -- 172.21.15.150:0/1690051212 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5a680cdea0 msgr2=0x7f5a680cd430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:08.364 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.362+0000 7f5a6d89b640 1 --2- 172.21.15.150:0/1690051212 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5a680cdea0 0x7f5a680cd430 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5a580097f0 tx=0x7f5a5801c480 comp rx=0 tx=0).stop 2023-11-06T02:08:08.364 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.362+0000 7f5a6d89b640 1 -- 172.21.15.150:0/1690051212 shutdown_connections 2023-11-06T02:08:08.364 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.362+0000 7f5a6d89b640 1 --2- 172.21.15.150:0/1690051212 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5a680cdea0 0x7f5a680cd430 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:08.364 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.362+0000 7f5a6d89b640 1 -- 172.21.15.150:0/1690051212 >> 172.21.15.150:0/1690051212 conn(0x7f5a68098a40 msgr2=0x7f5a680cf860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:08.365 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.362+0000 7f5a6d89b640 1 -- 172.21.15.150:0/1690051212 shutdown_connections 2023-11-06T02:08:08.365 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.362+0000 7f5a6d89b640 1 -- 172.21.15.150:0/1690051212 wait complete. 2023-11-06T02:08:08.367 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.366+0000 7f5a6d89b640 1 Processor -- start 2023-11-06T02:08:08.367 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.366+0000 7f5a6d89b640 1 -- start start 2023-11-06T02:08:08.367 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.366+0000 7f5a6d89b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a680ae760 0x7f5a68157980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:08.368 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.366+0000 7f5a6d89b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5a680cdea0 0x7f5a680cc880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:08.368 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.366+0000 7f5a6d89b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5a680ce280 0x7f5a6814a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:08.368 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.366+0000 7f5a6d89b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a6814aad0 con 0x7f5a680ae760 2023-11-06T02:08:08.368 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.366+0000 7f5a6d89b640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a6814b540 con 0x7f5a680cdea0 2023-11-06T02:08:08.368 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.366+0000 7f5a6d89b640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a680ccf60 con 0x7f5a680ce280 2023-11-06T02:08:08.369 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.366+0000 7f5a677fe640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5a680ce280 0x7f5a6814a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:08.369 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.366+0000 7f5a667fc640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5a680cdea0 0x7f5a680cc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:08.369 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.366+0000 7f5a677fe640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5a680ce280 0x7f5a6814a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.150:55280/0 (socket says 172.21.15.150:55280) 2023-11-06T02:08:08.369 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.366+0000 7f5a677fe640 1 -- 172.21.15.150:0/4275407799 learned_addr learned my addr 172.21.15.150:0/4275407799 (peer_addr_for_me v2:172.21.15.150:0/0) 2023-11-06T02:08:08.369 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.366+0000 7f5a66ffd640 1 --2- 172.21.15.150:0/4275407799 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a680ae760 0x7f5a68157980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:08.370 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.366+0000 7f5a667fc640 1 -- 172.21.15.150:0/4275407799 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5a680ce280 msgr2=0x7f5a6814a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:08.370 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.366+0000 7f5a667fc640 1 --2- 172.21.15.150:0/4275407799 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5a680ce280 0x7f5a6814a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:08.370 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.366+0000 7f5a667fc640 1 -- 172.21.15.150:0/4275407799 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a680ae760 msgr2=0x7f5a68157980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:08.370 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.366+0000 7f5a667fc640 1 --2- 172.21.15.150:0/4275407799 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a680ae760 0x7f5a68157980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:08.370 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.366+0000 7f5a667fc640 1 -- 172.21.15.150:0/4275407799 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a58009420 con 0x7f5a680cdea0 2023-11-06T02:08:08.371 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.366+0000 7f5a66ffd640 1 --2- 172.21.15.150:0/4275407799 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a680ae760 0x7f5a68157980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:08:08.371 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.366+0000 7f5a677fe640 1 --2- 172.21.15.150:0/4275407799 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5a680ce280 0x7f5a6814a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T02:08:08.371 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.366+0000 7f5a667fc640 1 --2- 172.21.15.150:0/4275407799 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5a680cdea0 0x7f5a680cc880 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c00b410 tx=0x7f5a5c00b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:08.371 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.366+0000 7f5a4ffff640 1 -- 172.21.15.150:0/4275407799 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c004020 con 0x7f5a680cdea0 2023-11-06T02:08:08.371 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.366+0000 7f5a4ffff640 1 -- 172.21.15.150:0/4275407799 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a5c004180 con 0x7f5a680cdea0 2023-11-06T02:08:08.371 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.366+0000 7f5a6d89b640 1 -- 172.21.15.150:0/4275407799 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a680cad60 con 0x7f5a680cdea0 2023-11-06T02:08:08.372 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.366+0000 7f5a4ffff640 1 -- 172.21.15.150:0/4275407799 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c011630 con 0x7f5a680cdea0 2023-11-06T02:08:08.372 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.366+0000 7f5a6d89b640 1 -- 172.21.15.150:0/4275407799 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a68156bd0 con 0x7f5a680cdea0 2023-11-06T02:08:08.372 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.366+0000 7f5a4ffff640 1 -- 172.21.15.150:0/4275407799 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f5a5c011a90 con 0x7f5a680cdea0 2023-11-06T02:08:08.372 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.366+0000 7f5a4ffff640 1 -- 172.21.15.150:0/4275407799 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f5a5c003d80 con 0x7f5a680cdea0 2023-11-06T02:08:08.372 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.366+0000 7f5a6d89b640 1 -- 172.21.15.150:0/4275407799 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a681552f0 con 0x7f5a680cdea0 2023-11-06T02:08:08.373 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.370+0000 7f5a4ffff640 1 -- 172.21.15.150:0/4275407799 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f5a5c015030 con 0x7f5a680cdea0 2023-11-06T02:08:08.384 INFO:tasks.ceph.mgr.y.smithi049.stderr:2023-11-06T02:08:08.389+0000 7f148018b180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-06T02:08:08.441 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-06T02:08:08.442+0000 7f4792183180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-06T02:08:08.452 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.454+0000 7f5a6d89b640 1 -- 172.21.15.150:0/4275407799 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd new", "uuid": "0df9c303-c255-4640-8e91-01f1c3efe18e", "id": 5} v 0) v1 -- 0x7f5a68155010 con 0x7f5a680cdea0 2023-11-06T02:08:08.460 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.462+0000 7f5a4ffff640 1 -- 172.21.15.150:0/4275407799 <== mon.1 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0df9c303-c255-4640-8e91-01f1c3efe18e", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f5a5c003600 con 0x7f5a680cdea0 2023-11-06T02:08:08.460 INFO:teuthology.orchestra.run.smithi150.stdout:5 2023-11-06T02:08:08.462 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.462+0000 7f5a6d89b640 1 -- 172.21.15.150:0/4275407799 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5a680cdea0 msgr2=0x7f5a680cc880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:08.462 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.462+0000 7f5a6d89b640 1 --2- 172.21.15.150:0/4275407799 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5a680cdea0 0x7f5a680cc880 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c00b410 tx=0x7f5a5c00b8d0 comp rx=0 tx=0).stop 2023-11-06T02:08:08.462 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.462+0000 7f5a6d89b640 1 -- 172.21.15.150:0/4275407799 shutdown_connections 2023-11-06T02:08:08.462 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.462+0000 7f5a6d89b640 1 --2- 172.21.15.150:0/4275407799 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5a680ce280 0x7f5a6814a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:08.463 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.462+0000 7f5a6d89b640 1 --2- 172.21.15.150:0/4275407799 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5a680cdea0 0x7f5a680cc880 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:08.463 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.462+0000 7f5a6d89b640 1 --2- 172.21.15.150:0/4275407799 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a680ae760 0x7f5a68157980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:08.463 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.462+0000 7f5a6d89b640 1 -- 172.21.15.150:0/4275407799 >> 172.21.15.150:0/4275407799 conn(0x7f5a68098a40 msgr2=0x7f5a6806eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:08.463 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.462+0000 7f5a6d89b640 1 -- 172.21.15.150:0/4275407799 shutdown_connections 2023-11-06T02:08:08.463 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.462+0000 7f5a6d89b640 1 -- 172.21.15.150:0/4275407799 wait complete. 2023-11-06T02:08:08.466 INFO:tasks.ceph.mgr.y.smithi049.stderr:2023-11-06T02:08:08.469+0000 7f148018b180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-06T02:08:08.477 DEBUG:teuthology.orchestra.run.smithi150:> sudo ceph --cluster ceph osd new e59fa189-4b70-4c06-9722-c02b90828bf2 6 2023-11-06T02:08:08.497 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-06T02:08:08.498+0000 7f4792183180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-06T02:08:08.520 INFO:tasks.ceph.mgr.y.smithi049.stderr:2023-11-06T02:08:08.525+0000 7f148018b180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-06T02:08:08.608 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.610+0000 7f7884808640 1 Processor -- start 2023-11-06T02:08:08.609 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.610+0000 7f7884808640 1 -- start start 2023-11-06T02:08:08.610 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.610+0000 7f7884808640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7880065930 0x7f78800c99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:08.610 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.610+0000 7f7884808640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f78800c9f20 0x7f788006e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:08.611 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.610+0000 7f7884808640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78800cabf0 con 0x7f78800c9600 2023-11-06T02:08:08.611 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.610+0000 7f7884808640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f788006def0 con 0x7f78800c9f20 2023-11-06T02:08:08.611 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.610+0000 7f7884808640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f788015b970 con 0x7f7880065930 2023-11-06T02:08:08.611 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.610+0000 7f787dd9b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7880065930 0x7f78800c99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:08.611 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.610+0000 7f787dd9b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7880065930 0x7f78800c99e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.150:55296/0 (socket says 172.21.15.150:55296) 2023-11-06T02:08:08.612 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.610+0000 7f787dd9b640 1 -- 172.21.15.150:0/117771741 learned_addr learned my addr 172.21.15.150:0/117771741 (peer_addr_for_me v2:172.21.15.150:0/0) 2023-11-06T02:08:08.612 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.610+0000 7f787ed9d640 1 --2- 172.21.15.150:0/117771741 >> v2:172.21.15.49:3300/0 conn(0x7f78800c9f20 0x7f788006e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:08.612 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.610+0000 7f787dd9b640 1 -- 172.21.15.150:0/117771741 >> v2:172.21.15.49:3300/0 conn(0x7f78800c9f20 msgr2=0x7f788006e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:08.613 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.610+0000 7f787dd9b640 1 --2- 172.21.15.150:0/117771741 >> v2:172.21.15.49:3300/0 conn(0x7f78800c9f20 0x7f788006e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:08.613 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.610+0000 7f787dd9b640 1 -- 172.21.15.150:0/117771741 >> v1:172.21.15.49:6789/0 conn(0x7f78800c9600 legacy=0x7f78800cb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:08.613 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.610+0000 7f787dd9b640 1 -- 172.21.15.150:0/117771741 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78800ca8c0 con 0x7f7880065930 2023-11-06T02:08:08.619 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.614+0000 7f787dd9b640 1 --2- 172.21.15.150:0/117771741 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7880065930 0x7f78800c99e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7868009870 tx=0x7f786801c540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87d66258784e0df6 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:08.619 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.614+0000 7f787d59a640 1 -- 172.21.15.150:0/117771741 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78680a7070 con 0x7f7880065930 2023-11-06T02:08:08.619 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.614+0000 7f787d59a640 1 -- 172.21.15.150:0/117771741 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78680ab430 con 0x7f7880065930 2023-11-06T02:08:08.620 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.614+0000 7f787d59a640 1 -- 172.21.15.150:0/117771741 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78680ab7c0 con 0x7f7880065930 2023-11-06T02:08:08.620 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.614+0000 7f7884808640 1 -- 172.21.15.150:0/117771741 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7880065930 msgr2=0x7f78800c99e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:08.620 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.614+0000 7f7884808640 1 --2- 172.21.15.150:0/117771741 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7880065930 0x7f78800c99e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7868009870 tx=0x7f786801c540 comp rx=0 tx=0).stop 2023-11-06T02:08:08.620 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.614+0000 7f7884808640 1 -- 172.21.15.150:0/117771741 shutdown_connections 2023-11-06T02:08:08.620 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.614+0000 7f7884808640 1 --2- 172.21.15.150:0/117771741 >> v2:172.21.15.49:3300/0 conn(0x7f78800c9f20 0x7f788006e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:08.621 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.614+0000 7f7884808640 1 --2- 172.21.15.150:0/117771741 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7880065930 0x7f78800c99e0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:08.621 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.614+0000 7f7884808640 1 -- 172.21.15.150:0/117771741 >> 172.21.15.150:0/117771741 conn(0x7f78800b0aa0 msgr2=0x7f78800b0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:08.621 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.614+0000 7f7884808640 1 -- 172.21.15.150:0/117771741 shutdown_connections 2023-11-06T02:08:08.621 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.614+0000 7f7884808640 1 -- 172.21.15.150:0/117771741 wait complete. 2023-11-06T02:08:08.622 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.614+0000 7f7884808640 1 Processor -- start 2023-11-06T02:08:08.622 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.614+0000 7f7884808640 1 -- start start 2023-11-06T02:08:08.622 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.618+0000 7f7884808640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7880065930 0x7f78800b1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:08.622 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.618+0000 7f7884808640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f78800c9600 0x7f788015bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:08.622 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.618+0000 7f7884808640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78800c9f20 0x7f7880155c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:08.623 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.618+0000 7f7884808640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78800b4b90 con 0x7f78800c9f20 2023-11-06T02:08:08.623 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.618+0000 7f7884808640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78800c7fa0 con 0x7f78800c9600 2023-11-06T02:08:08.623 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.618+0000 7f7884808640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78800c76d0 con 0x7f7880065930 2023-11-06T02:08:08.623 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.618+0000 7f787e59c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7880065930 0x7f78800b1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:08.623 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.618+0000 7f787dd9b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f78800c9600 0x7f788015bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:08.624 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.618+0000 7f787e59c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7880065930 0x7f78800b1bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.150:55312/0 (socket says 172.21.15.150:55312) 2023-11-06T02:08:08.624 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.618+0000 7f787e59c640 1 -- 172.21.15.150:0/1557570960 learned_addr learned my addr 172.21.15.150:0/1557570960 (peer_addr_for_me v2:172.21.15.150:0/0) 2023-11-06T02:08:08.624 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.618+0000 7f787ed9d640 1 --2- 172.21.15.150:0/1557570960 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78800c9f20 0x7f7880155c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:08.624 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.618+0000 7f787e59c640 1 -- 172.21.15.150:0/1557570960 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f78800c9600 msgr2=0x7f788015bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:08.624 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.618+0000 7f787e59c640 1 --2- 172.21.15.150:0/1557570960 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f78800c9600 0x7f788015bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:08.624 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.618+0000 7f787e59c640 1 -- 172.21.15.150:0/1557570960 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78800c9f20 msgr2=0x7f7880155c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:08.625 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.618+0000 7f787e59c640 1 --2- 172.21.15.150:0/1557570960 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78800c9f20 0x7f7880155c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:08.625 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.618+0000 7f787e59c640 1 -- 172.21.15.150:0/1557570960 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78680094d0 con 0x7f7880065930 2023-11-06T02:08:08.625 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.618+0000 7f787dd9b640 1 --2- 172.21.15.150:0/1557570960 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f78800c9600 0x7f788015bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T02:08:08.625 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.618+0000 7f787e59c640 1 --2- 172.21.15.150:0/1557570960 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7880065930 0x7f78800b1bd0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f787000c3c0 tx=0x7f787000c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:08.625 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.618+0000 7f7876ffd640 1 -- 172.21.15.150:0/1557570960 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78700052d0 con 0x7f7880065930 2023-11-06T02:08:08.626 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.618+0000 7f7876ffd640 1 -- 172.21.15.150:0/1557570960 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7870005430 con 0x7f7880065930 2023-11-06T02:08:08.626 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.618+0000 7f7884808640 1 -- 172.21.15.150:0/1557570960 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78800b48b0 con 0x7f7880065930 2023-11-06T02:08:08.626 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.618+0000 7f7876ffd640 1 -- 172.21.15.150:0/1557570960 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7870003840 con 0x7f7880065930 2023-11-06T02:08:08.626 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.618+0000 7f7884808640 1 -- 172.21.15.150:0/1557570960 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78800c7c30 con 0x7f7880065930 2023-11-06T02:08:08.627 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.618+0000 7f7876ffd640 1 -- 172.21.15.150:0/1557570960 <== mon.2 v2:172.21.15.150:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f7870003ca0 con 0x7f7880065930 2023-11-06T02:08:08.627 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.618+0000 7f7876ffd640 1 -- 172.21.15.150:0/1557570960 <== mon.2 v2:172.21.15.150:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f7870012a10 con 0x7f7880065930 2023-11-06T02:08:08.627 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.618+0000 7f7884808640 1 -- 172.21.15.150:0/1557570960 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78800b17e0 con 0x7f7880065930 2023-11-06T02:08:08.627 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.622+0000 7f7876ffd640 1 -- 172.21.15.150:0/1557570960 <== mon.2 v2:172.21.15.150:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f7870017070 con 0x7f7880065930 2023-11-06T02:08:08.641 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-06T02:08:08.642+0000 7f4792183180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-06T02:08:08.667 INFO:tasks.ceph.mgr.y.smithi049.stderr:2023-11-06T02:08:08.673+0000 7f148018b180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-06T02:08:08.686 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.686+0000 7f7884808640 1 -- 172.21.15.150:0/1557570960 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_command({"prefix": "osd new", "uuid": "e59fa189-4b70-4c06-9722-c02b90828bf2", "id": 6} v 0) v1 -- 0x7f78800b4f30 con 0x7f7880065930 2023-11-06T02:08:08.694 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.694+0000 7f7876ffd640 1 -- 172.21.15.150:0/1557570960 <== mon.2 v2:172.21.15.150:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e59fa189-4b70-4c06-9722-c02b90828bf2", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f7870012420 con 0x7f7880065930 2023-11-06T02:08:08.694 INFO:teuthology.orchestra.run.smithi150.stdout:6 2023-11-06T02:08:08.695 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.698+0000 7f7884808640 1 -- 172.21.15.150:0/1557570960 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7880065930 msgr2=0x7f78800b1bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:08.695 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.698+0000 7f7884808640 1 --2- 172.21.15.150:0/1557570960 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7880065930 0x7f78800b1bd0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f787000c3c0 tx=0x7f787000c880 comp rx=0 tx=0).stop 2023-11-06T02:08:08.696 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.698+0000 7f7884808640 1 -- 172.21.15.150:0/1557570960 shutdown_connections 2023-11-06T02:08:08.696 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.698+0000 7f7884808640 1 --2- 172.21.15.150:0/1557570960 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78800c9f20 0x7f7880155c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:08.696 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.698+0000 7f7884808640 1 --2- 172.21.15.150:0/1557570960 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f78800c9600 0x7f788015bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:08.696 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.698+0000 7f7884808640 1 --2- 172.21.15.150:0/1557570960 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7880065930 0x7f78800b1bd0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:08.696 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.698+0000 7f7884808640 1 -- 172.21.15.150:0/1557570960 >> 172.21.15.150:0/1557570960 conn(0x7f78800b0aa0 msgr2=0x7f78800ce460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:08.697 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.698+0000 7f7884808640 1 -- 172.21.15.150:0/1557570960 shutdown_connections 2023-11-06T02:08:08.697 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.698+0000 7f7884808640 1 -- 172.21.15.150:0/1557570960 wait complete. 2023-11-06T02:08:08.702 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-06T02:08:08.702+0000 7f4792183180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-06T02:08:08.710 DEBUG:teuthology.orchestra.run.smithi150:> sudo ceph --cluster ceph osd new a64f57e1-ef98-46bd-b304-385759e95898 7 2023-11-06T02:08:08.729 INFO:tasks.ceph.mgr.y.smithi049.stderr:2023-11-06T02:08:08.733+0000 7f148018b180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-06T02:08:08.825 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.826+0000 7f97d9c21640 1 Processor -- start 2023-11-06T02:08:08.825 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.826+0000 7f97d9c21640 1 -- start start 2023-11-06T02:08:08.826 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.826+0000 7f97d9c21640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f97d415a9e0 0x7f97d40cb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:08.827 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.826+0000 7f97d9c21640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f97d40b04e0 0x7f97d40b08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:08.827 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.826+0000 7f97d9c21640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97d40cc5c0 con 0x7f97d40afbb0 2023-11-06T02:08:08.827 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.826+0000 7f97d9c21640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97d40cd160 con 0x7f97d415a9e0 2023-11-06T02:08:08.827 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.826+0000 7f97d9c21640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97d40b0e30 con 0x7f97d40b04e0 2023-11-06T02:08:08.828 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.826+0000 7f97d3fff640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f97d40b04e0 0x7f97d40b08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:08.828 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.826+0000 7f97d3fff640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f97d40b04e0 0x7f97d40b08f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:43964/0 (socket says 172.21.15.150:43964) 2023-11-06T02:08:08.828 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.826+0000 7f97d3fff640 1 -- 172.21.15.150:0/1183216839 learned_addr learned my addr 172.21.15.150:0/1183216839 (peer_addr_for_me v2:172.21.15.150:0/0) 2023-11-06T02:08:08.828 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.826+0000 7f97d37fe640 1 --2- 172.21.15.150:0/1183216839 >> v2:172.21.15.49:3300/0 conn(0x7f97d415a9e0 0x7f97d40cb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:08.828 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.826+0000 7f97d3fff640 1 -- 172.21.15.150:0/1183216839 >> v2:172.21.15.49:3300/0 conn(0x7f97d415a9e0 msgr2=0x7f97d40cb2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:08.829 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.826+0000 7f97d3fff640 1 --2- 172.21.15.150:0/1183216839 >> v2:172.21.15.49:3300/0 conn(0x7f97d415a9e0 0x7f97d40cb2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:08.829 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.826+0000 7f97d3fff640 1 -- 172.21.15.150:0/1183216839 >> v1:172.21.15.49:6789/0 conn(0x7f97d40afbb0 legacy=0x7f97d40cca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:08.829 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.826+0000 7f97d3fff640 1 -- 172.21.15.150:0/1183216839 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97d40cb9b0 con 0x7f97d40b04e0 2023-11-06T02:08:08.833 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.830+0000 7f97d3fff640 1 --2- 172.21.15.150:0/1183216839 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97d40b04e0 0x7f97d40b08f0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f97d40d05c0 tx=0x7f97cc01c9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c66f150733e2050e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:08.833 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.830+0000 7f97d27fc640 1 -- 172.21.15.150:0/1183216839 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97cc0a5860 con 0x7f97d40b04e0 2023-11-06T02:08:08.833 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.830+0000 7f97d27fc640 1 -- 172.21.15.150:0/1183216839 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97cc0a59c0 con 0x7f97d40b04e0 2023-11-06T02:08:08.833 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.830+0000 7f97d27fc640 1 -- 172.21.15.150:0/1183216839 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97cc0a5d50 con 0x7f97d40b04e0 2023-11-06T02:08:08.833 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.830+0000 7f97d9c21640 1 -- 172.21.15.150:0/1183216839 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97d40b04e0 msgr2=0x7f97d40b08f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:08.834 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.830+0000 7f97d9c21640 1 --2- 172.21.15.150:0/1183216839 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97d40b04e0 0x7f97d40b08f0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f97d40d05c0 tx=0x7f97cc01c9a0 comp rx=0 tx=0).stop 2023-11-06T02:08:08.834 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.830+0000 7f97d9c21640 1 -- 172.21.15.150:0/1183216839 shutdown_connections 2023-11-06T02:08:08.834 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.830+0000 7f97d9c21640 1 --2- 172.21.15.150:0/1183216839 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97d40b04e0 0x7f97d40b08f0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:08.834 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.830+0000 7f97d9c21640 1 --2- 172.21.15.150:0/1183216839 >> v2:172.21.15.49:3300/0 conn(0x7f97d415a9e0 0x7f97d40cb2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:08.835 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.830+0000 7f97d9c21640 1 -- 172.21.15.150:0/1183216839 >> 172.21.15.150:0/1183216839 conn(0x7f97d4098a40 msgr2=0x7f97d40cf860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:08.835 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.830+0000 7f97d9c21640 1 -- 172.21.15.150:0/1183216839 shutdown_connections 2023-11-06T02:08:08.835 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.830+0000 7f97d9c21640 1 -- 172.21.15.150:0/1183216839 wait complete. 2023-11-06T02:08:08.835 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.830+0000 7f97d9c21640 1 Processor -- start 2023-11-06T02:08:08.835 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.830+0000 7f97d9c21640 1 -- start start 2023-11-06T02:08:08.836 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.830+0000 7f97d9c21640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97d40afbb0 0x7f97d40c8730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:08.836 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.830+0000 7f97d9c21640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97d40b04e0 0x7f97d41427a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:08.836 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.830+0000 7f97d9c21640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f97d415a9e0 0x7f97d40c5d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:08.836 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.830+0000 7f97d9c21640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97d4142ce0 con 0x7f97d40b04e0 2023-11-06T02:08:08.837 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.830+0000 7f97d9c21640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97d40c8e10 con 0x7f97d40afbb0 2023-11-06T02:08:08.837 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.830+0000 7f97d9c21640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97d40c90f0 con 0x7f97d415a9e0 2023-11-06T02:08:08.837 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.834+0000 7f97d37fe640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97d40afbb0 0x7f97d40c8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:08.837 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.834+0000 7f97d37fe640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97d40afbb0 0x7f97d40c8730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:43972/0 (socket says 172.21.15.150:43972) 2023-11-06T02:08:08.837 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.834+0000 7f97d37fe640 1 -- 172.21.15.150:0/2894750229 learned_addr learned my addr 172.21.15.150:0/2894750229 (peer_addr_for_me v2:172.21.15.150:0/0) 2023-11-06T02:08:08.838 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.834+0000 7f97d3fff640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f97d415a9e0 0x7f97d40c5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:08.838 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.834+0000 7f97d2ffd640 1 --2- 172.21.15.150:0/2894750229 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97d40b04e0 0x7f97d41427a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:08.838 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.834+0000 7f97d37fe640 1 -- 172.21.15.150:0/2894750229 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f97d415a9e0 msgr2=0x7f97d40c5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:08.838 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.834+0000 7f97d37fe640 1 --2- 172.21.15.150:0/2894750229 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f97d415a9e0 0x7f97d40c5d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:08.839 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.834+0000 7f97d37fe640 1 -- 172.21.15.150:0/2894750229 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97d40b04e0 msgr2=0x7f97d41427a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:08.839 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.834+0000 7f97d37fe640 1 --2- 172.21.15.150:0/2894750229 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97d40b04e0 0x7f97d41427a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:08.839 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.834+0000 7f97d37fe640 1 -- 172.21.15.150:0/2894750229 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97cc009b30 con 0x7f97d40afbb0 2023-11-06T02:08:08.839 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.834+0000 7f97d3fff640 1 --2- 172.21.15.150:0/2894750229 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f97d415a9e0 0x7f97d40c5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T02:08:08.840 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.834+0000 7f97d37fe640 1 --2- 172.21.15.150:0/2894750229 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97d40afbb0 0x7f97d40c8730 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f97c400d800 tx=0x7f97c400dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:08.840 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.834+0000 7f97bbfff640 1 -- 172.21.15.150:0/2894750229 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c400b6c0 con 0x7f97d40afbb0 2023-11-06T02:08:08.840 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.834+0000 7f97bbfff640 1 -- 172.21.15.150:0/2894750229 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97c400b820 con 0x7f97d40afbb0 2023-11-06T02:08:08.840 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.834+0000 7f97d9c21640 1 -- 172.21.15.150:0/2894750229 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97d40c93d0 con 0x7f97d40afbb0 2023-11-06T02:08:08.840 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.834+0000 7f97bbfff640 1 -- 172.21.15.150:0/2894750229 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c400bb90 con 0x7f97d40afbb0 2023-11-06T02:08:08.841 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.834+0000 7f97d9c21640 1 -- 172.21.15.150:0/2894750229 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97d40c9990 con 0x7f97d40afbb0 2023-11-06T02:08:08.841 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.834+0000 7f97bbfff640 1 -- 172.21.15.150:0/2894750229 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f97c401e460 con 0x7f97d40afbb0 2023-11-06T02:08:08.841 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.834+0000 7f97bbfff640 1 -- 172.21.15.150:0/2894750229 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f97c4014bd0 con 0x7f97d40afbb0 2023-11-06T02:08:08.841 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.834+0000 7f97d9c21640 1 -- 172.21.15.150:0/2894750229 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97d40ccd10 con 0x7f97d40afbb0 2023-11-06T02:08:08.842 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.834+0000 7f97bbfff640 1 -- 172.21.15.150:0/2894750229 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f97c4014570 con 0x7f97d40afbb0 2023-11-06T02:08:08.897 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.898+0000 7f97d9c21640 1 -- 172.21.15.150:0/2894750229 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd new", "uuid": "a64f57e1-ef98-46bd-b304-385759e95898", "id": 7} v 0) v1 -- 0x7f97d40cc220 con 0x7f97d40afbb0 2023-11-06T02:08:08.906 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.906+0000 7f97bbfff640 1 -- 172.21.15.150:0/2894750229 <== mon.1 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a64f57e1-ef98-46bd-b304-385759e95898", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f97c4014710 con 0x7f97d40afbb0 2023-11-06T02:08:08.906 INFO:teuthology.orchestra.run.smithi150.stdout:7 2023-11-06T02:08:08.907 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.906+0000 7f97d9c21640 1 -- 172.21.15.150:0/2894750229 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97d40afbb0 msgr2=0x7f97d40c8730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:08.908 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.906+0000 7f97d9c21640 1 --2- 172.21.15.150:0/2894750229 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97d40afbb0 0x7f97d40c8730 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f97c400d800 tx=0x7f97c400dcc0 comp rx=0 tx=0).stop 2023-11-06T02:08:08.908 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.910+0000 7f97d9c21640 1 -- 172.21.15.150:0/2894750229 shutdown_connections 2023-11-06T02:08:08.908 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.910+0000 7f97d9c21640 1 --2- 172.21.15.150:0/2894750229 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97d40b04e0 0x7f97d41427a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:08.908 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.910+0000 7f97d9c21640 1 --2- 172.21.15.150:0/2894750229 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f97d415a9e0 0x7f97d40c5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:08.909 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.910+0000 7f97d9c21640 1 --2- 172.21.15.150:0/2894750229 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97d40afbb0 0x7f97d40c8730 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:08.909 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.910+0000 7f97d9c21640 1 -- 172.21.15.150:0/2894750229 >> 172.21.15.150:0/2894750229 conn(0x7f97d4098a40 msgr2=0x7f97d40b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:08.909 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.910+0000 7f97d9c21640 1 -- 172.21.15.150:0/2894750229 shutdown_connections 2023-11-06T02:08:08.909 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-06T02:08:08.910+0000 7f97d9c21640 1 -- 172.21.15.150:0/2894750229 wait complete. 2023-11-06T02:08:08.921 INFO:tasks.ceph.osd.0:Restarting daemon 2023-11-06T02:08:08.922 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-11-06T02:08:08.924 INFO:tasks.ceph.osd.0:Started 2023-11-06T02:08:08.925 INFO:tasks.ceph.osd.1:Restarting daemon 2023-11-06T02:08:08.925 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-11-06T02:08:08.927 INFO:tasks.ceph.osd.1:Started 2023-11-06T02:08:08.928 INFO:tasks.ceph.osd.2:Restarting daemon 2023-11-06T02:08:08.928 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-11-06T02:08:08.930 INFO:tasks.ceph.osd.2:Started 2023-11-06T02:08:08.931 INFO:tasks.ceph.osd.3:Restarting daemon 2023-11-06T02:08:08.931 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-11-06T02:08:08.933 INFO:tasks.ceph.osd.3:Started 2023-11-06T02:08:08.933 INFO:tasks.ceph.osd.4:Restarting daemon 2023-11-06T02:08:08.934 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-11-06T02:08:08.936 INFO:tasks.ceph.osd.4:Started 2023-11-06T02:08:08.936 INFO:tasks.ceph.osd.5:Restarting daemon 2023-11-06T02:08:08.937 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-11-06T02:08:08.939 INFO:tasks.ceph.osd.5:Started 2023-11-06T02:08:08.939 INFO:tasks.ceph.osd.6:Restarting daemon 2023-11-06T02:08:08.940 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-11-06T02:08:08.941 INFO:tasks.ceph.osd.6:Started 2023-11-06T02:08:08.942 INFO:tasks.ceph.osd.7:Restarting daemon 2023-11-06T02:08:08.942 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-11-06T02:08:08.944 INFO:tasks.ceph.osd.7:Started 2023-11-06T02:08:08.945 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T02:08:09.099 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:09.105+0000 7f4c78a93640 1 Processor -- start 2023-11-06T02:08:09.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:09.105+0000 7f4c78a93640 1 -- start start 2023-11-06T02:08:09.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:09.105+0000 7f4c78a93640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4c74071120 0x7f4c74071500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:09.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:09.105+0000 7f4c78a93640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f4c740c9c80 0x7f4c740b2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:09.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:09.105+0000 7f4c78a93640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c74071a40 con 0x7f4c740b5270 2023-11-06T02:08:09.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:09.105+0000 7f4c78a93640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c740ae9c0 con 0x7f4c740c9c80 2023-11-06T02:08:09.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:09.105+0000 7f4c78a93640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c740af010 con 0x7f4c74071120 2023-11-06T02:08:09.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:09.105+0000 7f4c72d9d640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f4c740b5270 0x7f4c740ca640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:54064/0 (socket says 172.21.15.49:54064) 2023-11-06T02:08:09.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:09.105+0000 7f4c72d9d640 1 -- 172.21.15.49:0/41237979 learned_addr learned my addr 172.21.15.49:0/41237979 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:08:09.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:09.105+0000 7f4c69d9b640 1 --2- 172.21.15.49:0/41237979 >> v2:172.21.15.49:3300/0 conn(0x7f4c740c9c80 0x7f4c740b2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:09.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:09.105+0000 7f4c7259c640 1 --2- 172.21.15.49:0/41237979 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4c74071120 0x7f4c74071500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:09.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:09.105+0000 7f4c71d9b640 1 -- 172.21.15.49:0/41237979 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3469662077 0 0) 0x7f4c74071a40 con 0x7f4c740b5270 2023-11-06T02:08:09.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:09.105+0000 7f4c71d9b640 1 -- 172.21.15.49:0/41237979 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c54003530 con 0x7f4c740b5270 2023-11-06T02:08:09.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:09.105+0000 7f4c69d9b640 1 -- 172.21.15.49:0/41237979 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4c74071120 msgr2=0x7f4c74071500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:09.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:09.105+0000 7f4c69d9b640 1 --2- 172.21.15.49:0/41237979 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4c74071120 0x7f4c74071500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:09.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:09.105+0000 7f4c69d9b640 1 -- 172.21.15.49:0/41237979 >> v1:172.21.15.49:6789/0 conn(0x7f4c740b5270 legacy=0x7f4c740ca640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:09.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:09.105+0000 7f4c69d9b640 1 -- 172.21.15.49:0/41237979 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c740ae6e0 con 0x7f4c740c9c80 2023-11-06T02:08:09.105 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:09.109+0000 7f4c69d9b640 1 --2- 172.21.15.49:0/41237979 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c740c9c80 0x7f4c740b2940 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f4c640097f0 tx=0x7f4c6401ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=740c0c7d35e1940f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:09.105 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:09.109+0000 7f4c71d9b640 1 -- 172.21.15.49:0/41237979 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c640a8970 con 0x7f4c740c9c80 2023-11-06T02:08:09.105 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:09.109+0000 7f4c71d9b640 1 -- 172.21.15.49:0/41237979 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c640a8ad0 con 0x7f4c740c9c80 2023-11-06T02:08:09.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:09.109+0000 7f4c71d9b640 1 -- 172.21.15.49:0/41237979 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c640a8e60 con 0x7f4c740c9c80 2023-11-06T02:08:09.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:09.109+0000 7f4c78a93640 1 -- 172.21.15.49:0/41237979 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c740c9c80 msgr2=0x7f4c740b2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:09.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:09.109+0000 7f4c78a93640 1 --2- 172.21.15.49:0/41237979 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c740c9c80 0x7f4c740b2940 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f4c640097f0 tx=0x7f4c6401ec70 comp rx=0 tx=0).stop 2023-11-06T02:08:09.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:09.109+0000 7f4c78a93640 1 -- 172.21.15.49:0/41237979 shutdown_connections 2023-11-06T02:08:09.107 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:09.109+0000 7f4c78a93640 1 --2- 172.21.15.49:0/41237979 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c740c9c80 0x7f4c740b2940 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:09.107 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:09.109+0000 7f4c78a93640 1 --2- 172.21.15.49:0/41237979 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4c74071120 0x7f4c74071500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:09.107 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:09.109+0000 7f4c78a93640 1 -- 172.21.15.49:0/41237979 >> 172.21.15.49:0/41237979 conn(0x7f4c740cc140 msgr2=0x7f4c740cc550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:09.107 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:09.109+0000 7f4c78a93640 1 -- 172.21.15.49:0/41237979 shutdown_connections 2023-11-06T02:08:09.107 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:09.109+0000 7f4c78a93640 1 -- 172.21.15.49:0/41237979 wait complete. 2023-11-06T02:08:09.107 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:09.109+0000 7f4c78a93640 1 Processor -- start 2023-11-06T02:08:09.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:09.113+0000 7f4c78a93640 1 -- start start 2023-11-06T02:08:09.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:09.113+0000 7f4c78a93640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4c74071120 0x7f4c74157030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:09.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:09.113+0000 7f4c78a93640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c740b5270 0x7f4c74149c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:09.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:09.113+0000 7f4c78a93640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c740c9c80 0x7f4c740ce680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:09.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:09.113+0000 7f4c78a93640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c740cd5e0 con 0x7f4c740c9c80 2023-11-06T02:08:09.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:09.113+0000 7f4c78a93640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c740cd8c0 con 0x7f4c740b5270 2023-11-06T02:08:09.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:09.113+0000 7f4c78a93640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c740cdba0 con 0x7f4c74071120 2023-11-06T02:08:09.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:09.113+0000 7f4c72d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c740c9c80 0x7f4c740ce680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:09.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:09.113+0000 7f4c72d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c740c9c80 0x7f4c740ce680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37512/0 (socket says 172.21.15.49:37512) 2023-11-06T02:08:09.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:09.113+0000 7f4c72d9d640 1 -- 172.21.15.49:0/1321827325 learned_addr learned my addr 172.21.15.49:0/1321827325 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:09.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:09.113+0000 7f4c7259c640 1 --2- 172.21.15.49:0/1321827325 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4c74071120 0x7f4c74157030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:09.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:09.113+0000 7f4c72d9d640 1 -- 172.21.15.49:0/1321827325 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4c74071120 msgr2=0x7f4c74157030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:09.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:09.113+0000 7f4c72d9d640 1 --2- 172.21.15.49:0/1321827325 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4c74071120 0x7f4c74157030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:09.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:09.113+0000 7f4c72d9d640 1 -- 172.21.15.49:0/1321827325 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c740b5270 msgr2=0x7f4c74149c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:09.113 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:09.113+0000 7f4c72d9d640 1 --2- 172.21.15.49:0/1321827325 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c740b5270 0x7f4c74149c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:09.113 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:09.113+0000 7f4c72d9d640 1 -- 172.21.15.49:0/1321827325 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c64009420 con 0x7f4c740c9c80 2023-11-06T02:08:09.113 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:09.113+0000 7f4c72d9d640 1 --2- 172.21.15.49:0/1321827325 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c740c9c80 0x7f4c740ce680 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c00cb40 tx=0x7f4c6c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:09.113 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:09.113+0000 7f4c6b7fe640 1 -- 172.21.15.49:0/1321827325 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c00aca0 con 0x7f4c740c9c80 2023-11-06T02:08:09.113 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:09.113+0000 7f4c6b7fe640 1 -- 172.21.15.49:0/1321827325 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c6c00ae00 con 0x7f4c740c9c80 2023-11-06T02:08:09.114 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:09.113+0000 7f4c78a93640 1 -- 172.21.15.49:0/1321827325 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c740cde80 con 0x7f4c740c9c80 2023-11-06T02:08:09.114 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:09.113+0000 7f4c6b7fe640 1 -- 172.21.15.49:0/1321827325 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c012620 con 0x7f4c740c9c80 2023-11-06T02:08:09.114 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:09.113+0000 7f4c78a93640 1 -- 172.21.15.49:0/1321827325 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c740cd200 con 0x7f4c740c9c80 2023-11-06T02:08:09.114 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:09.113+0000 7f4c6b7fe640 1 -- 172.21.15.49:0/1321827325 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f4c6c012a80 con 0x7f4c740c9c80 2023-11-06T02:08:09.114 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:09.113+0000 7f4c6b7fe640 1 -- 172.21.15.49:0/1321827325 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f4c6c012d30 con 0x7f4c740c9c80 2023-11-06T02:08:09.115 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:09.113+0000 7f4c78a93640 1 -- 172.21.15.49:0/1321827325 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c74067140 con 0x7f4c740c9c80 2023-11-06T02:08:09.115 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:09.117+0000 7f4c6b7fe640 1 -- 172.21.15.49:0/1321827325 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f4c6c017070 con 0x7f4c740c9c80 2023-11-06T02:08:09.173 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:09.177+0000 7f4c78a93640 1 -- 172.21.15.49:0/1321827325 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4c740ce330 con 0x7f4c740c9c80 2023-11-06T02:08:09.174 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:09.177+0000 7f4c6b7fe640 1 -- 172.21.15.49:0/1321827325 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f4c6c01c550 con 0x7f4c740c9c80 2023-11-06T02:08:09.174 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:08:09.174 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":10,"fsid":"1a8b035c-6ba4-499a-b876-aea47fd9c0d4","created":"2023-11-06T02:08:06.440857+0000","modified":"2023-11-06T02:08:08.906631+0000","last_up_change":"0.000000","last_in_change":"2023-11-06T02:08:08.906631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"6ad55af8-0571-4e6a-8722-b5d429b7496d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"c8b31f85-a757-43ed-9419-351acc46611b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"555b3bf3-0a70-4b91-8efd-e4204b5fc26e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"8d4e5d77-03ef-4b9b-bfcb-5ae93a46795c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"dba03ad4-68a3-4cf3-9218-ac611f02eacf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"0df9c303-c255-4640-8e91-01f1c3efe18e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"e59fa189-4b70-4c06-9722-c02b90828bf2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"a64f57e1-ef98-46bd-b304-385759e95898","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T02:08:09.175 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:09.177+0000 7f4c78a93640 1 -- 172.21.15.49:0/1321827325 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c740c9c80 msgr2=0x7f4c740ce680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:09.176 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:09.177+0000 7f4c78a93640 1 --2- 172.21.15.49:0/1321827325 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c740c9c80 0x7f4c740ce680 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c00cb40 tx=0x7f4c6c00a060 comp rx=0 tx=0).stop 2023-11-06T02:08:09.176 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:09.181+0000 7f4c78a93640 1 -- 172.21.15.49:0/1321827325 shutdown_connections 2023-11-06T02:08:09.176 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:09.181+0000 7f4c78a93640 1 --2- 172.21.15.49:0/1321827325 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c740c9c80 0x7f4c740ce680 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:09.176 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:09.181+0000 7f4c78a93640 1 --2- 172.21.15.49:0/1321827325 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c740b5270 0x7f4c74149c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:09.176 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:09.181+0000 7f4c78a93640 1 --2- 172.21.15.49:0/1321827325 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4c74071120 0x7f4c74157030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:09.177 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:09.181+0000 7f4c78a93640 1 -- 172.21.15.49:0/1321827325 >> 172.21.15.49:0/1321827325 conn(0x7f4c740cc140 msgr2=0x7f4c74158c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:09.177 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:09.181+0000 7f4c78a93640 1 -- 172.21.15.49:0/1321827325 shutdown_connections 2023-11-06T02:08:09.177 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:09.181+0000 7f4c78a93640 1 -- 172.21.15.49:0/1321827325 wait complete. 2023-11-06T02:08:09.192 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-11-06T02:08:09.193 INFO:tasks.ceph:Waiting for OSDs to come up 2023-11-06T02:08:09.312 INFO:tasks.ceph.osd.2.smithi049.stderr:2023-11-06T02:08:09.317+0000 7ff10758f640 -1 Falling back to public interface 2023-11-06T02:08:09.316 INFO:tasks.ceph.osd.5.smithi150.stderr:2023-11-06T02:08:09.318+0000 7fb747eec640 -1 Falling back to public interface 2023-11-06T02:08:09.326 INFO:tasks.ceph.mgr.y.smithi049.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-11-06T02:08:09.327 INFO:tasks.ceph.mgr.y.smithi049.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-11-06T02:08:09.327 INFO:tasks.ceph.mgr.y.smithi049.stderr: from numpy import show_config as show_numpy_config 2023-11-06T02:08:09.338 INFO:tasks.ceph.mgr.x.smithi150.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-11-06T02:08:09.338 INFO:tasks.ceph.mgr.x.smithi150.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-11-06T02:08:09.399 INFO:tasks.ceph.mgr.x.smithi150.stderr: from numpy import show_config as show_numpy_config 2023-11-06T02:08:09.400 INFO:tasks.ceph.mgr.y.smithi049.stderr:2023-11-06T02:08:09.381+0000 7f148018b180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-06T02:08:09.400 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-06T02:08:09.378+0000 7f4792183180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-06T02:08:09.506 INFO:tasks.ceph.mgr.y.smithi049.stderr:2023-11-06T02:08:09.509+0000 7f148018b180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-06T02:08:09.507 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-06T02:08:09.506+0000 7f4792183180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-06T02:08:09.549 INFO:tasks.ceph.osd.4.smithi150.stderr:2023-11-06T02:08:09.550+0000 7f43e00fa640 -1 Falling back to public interface 2023-11-06T02:08:09.557 INFO:tasks.ceph.osd.0.smithi049.stderr:2023-11-06T02:08:09.561+0000 7fd028a0c640 -1 Falling back to public interface 2023-11-06T02:08:09.557 INFO:tasks.ceph.osd.1.smithi049.stderr:2023-11-06T02:08:09.561+0000 7f4e7117c640 -1 Falling back to public interface 2023-11-06T02:08:09.558 INFO:tasks.ceph.osd.0.smithi049.stderr:2023-11-06T02:08:09.561+0000 7fd0279cd640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6808/0: (98) Address already in use 2023-11-06T02:08:09.564 INFO:tasks.ceph.osd.7.smithi150.stderr:2023-11-06T02:08:09.566+0000 7f2f443ef640 -1 Falling back to public interface 2023-11-06T02:08:09.580 INFO:tasks.ceph.osd.6.smithi150.stderr:2023-11-06T02:08:09.582+0000 7f388910a640 -1 Falling back to public interface 2023-11-06T02:08:09.587 INFO:tasks.ceph.osd.3.smithi049.stderr:2023-11-06T02:08:09.593+0000 7ff97c09f640 -1 Falling back to public interface 2023-11-06T02:08:09.662 INFO:tasks.ceph.mgr.y.smithi049.stderr:2023-11-06T02:08:09.665+0000 7f148018b180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-06T02:08:09.669 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-06T02:08:09.670+0000 7f4792183180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-06T02:08:09.728 INFO:tasks.ceph.mgr.y.smithi049.stderr:2023-11-06T02:08:09.733+0000 7f148018b180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-06T02:08:09.732 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-06T02:08:09.734+0000 7f4792183180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-06T02:08:09.855 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-06T02:08:09.854+0000 7f4792183180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-06T02:08:09.857 INFO:tasks.ceph.mgr.y.smithi049.stderr:2023-11-06T02:08:09.861+0000 7f148018b180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-06T02:08:10.003 DEBUG:teuthology.orchestra.run.smithi049:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-06T02:08:10.115 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:10.121+0000 7f2383434640 1 Processor -- start 2023-11-06T02:08:10.115 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:10.121+0000 7f2383434640 1 -- start start 2023-11-06T02:08:10.118 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:10.121+0000 7f2383434640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f237c15a830 0x7f237c15e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:10.118 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:10.121+0000 7f2383434640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f237c15f2d0 con 0x7f237c0b4fc0 2023-11-06T02:08:10.118 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:10.121+0000 7f2383434640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f237c15ac10 con 0x7f237c0b1780 2023-11-06T02:08:10.119 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:10.121+0000 7f2383434640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f237c15ea30 con 0x7f237c15a830 2023-11-06T02:08:10.119 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:10.121+0000 7f23809cf640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f237c0b4fc0 0x7f237c0cbde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:54076/0 (socket says 172.21.15.49:54076) 2023-11-06T02:08:10.119 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:10.121+0000 7f23809cf640 1 -- 172.21.15.49:0/3760026201 learned_addr learned my addr 172.21.15.49:0/3760026201 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:08:10.119 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:10.121+0000 7f23811d0640 1 --2- 172.21.15.49:0/3760026201 >> v2:172.21.15.150:3300/0 conn(0x7f237c15a830 0x7f237c15e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:10.120 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:10.121+0000 7f2373fff640 1 -- 172.21.15.49:0/3760026201 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 736752412 0 0) 0x7f237c15f2d0 con 0x7f237c0b4fc0 2023-11-06T02:08:10.120 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:10.121+0000 7f2373fff640 1 -- 172.21.15.49:0/3760026201 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2368003530 con 0x7f237c0b4fc0 2023-11-06T02:08:10.120 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:10.121+0000 7f2373fff640 1 -- 172.21.15.49:0/3760026201 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1313441371 0 0) 0x7f2368003530 con 0x7f237c0b4fc0 2023-11-06T02:08:10.120 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:10.121+0000 7f2373fff640 1 -- 172.21.15.49:0/3760026201 >> v2:172.21.15.150:3300/0 conn(0x7f237c15a830 msgr2=0x7f237c15e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:10.120 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:10.121+0000 7f2373fff640 1 --2- 172.21.15.49:0/3760026201 >> v2:172.21.15.150:3300/0 conn(0x7f237c15a830 0x7f237c15e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:10.120 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:10.121+0000 7f2373fff640 1 -- 172.21.15.49:0/3760026201 >> v1:172.21.15.150:6789/0 conn(0x7f237c0b1780 legacy=0x7f237c0cf8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:10.121 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:10.121+0000 7f2373fff640 1 -- 172.21.15.49:0/3760026201 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f237c15ed10 con 0x7f237c0b4fc0 2023-11-06T02:08:10.121 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:10.121+0000 7f2373fff640 1 -- 172.21.15.49:0/3760026201 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7f236c001680 con 0x7f237c0b4fc0 2023-11-06T02:08:10.121 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:10.121+0000 7f2373fff640 1 -- 172.21.15.49:0/3760026201 >> v1:172.21.15.49:6789/0 conn(0x7f237c0b4fc0 legacy=0x7f237c0cbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:10.121 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:10.121+0000 7f2373fff640 1 --2- 172.21.15.49:0/3760026201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2368003980 0x7f2368003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:10.121 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:10.121+0000 7f2373fff640 1 --2- 172.21.15.49:0/3760026201 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f23680047d0 0x7f2368004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:10.122 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:10.121+0000 7f2373fff640 1 --2- 172.21.15.49:0/3760026201 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2368005190 0x7f236800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:10.122 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:10.121+0000 7f2373fff640 1 -- 172.21.15.49:0/3760026201 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f237c15ea30 con 0x7f2368003980 2023-11-06T02:08:10.122 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:10.121+0000 7f23811d0640 1 --2- 172.21.15.49:0/3760026201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2368003980 0x7f2368003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:10.122 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:10.121+0000 7f2373fff640 1 -- 172.21.15.49:0/3760026201 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f236800c130 con 0x7f23680047d0 2023-11-06T02:08:10.122 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:10.121+0000 7f2373fff640 1 -- 172.21.15.49:0/3760026201 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f236800c2f0 con 0x7f2368005190 2023-11-06T02:08:10.123 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:10.121+0000 7f23811d0640 1 -- 172.21.15.49:0/3760026201 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2368005190 msgr2=0x7f236800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:10.123 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:10.121+0000 7f23811d0640 1 --2- 172.21.15.49:0/3760026201 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2368005190 0x7f236800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:10.123 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:10.121+0000 7f23811d0640 1 -- 172.21.15.49:0/3760026201 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f23680047d0 msgr2=0x7f2368004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:10.123 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:10.121+0000 7f23811d0640 1 --2- 172.21.15.49:0/3760026201 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f23680047d0 0x7f2368004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:10.123 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:10.121+0000 7f23811d0640 1 -- 172.21.15.49:0/3760026201 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2368003530 con 0x7f2368003980 2023-11-06T02:08:10.124 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:10.121+0000 7f23819d1640 1 -- 172.21.15.49:0/3760026201 reap_dead start 2023-11-06T02:08:10.127 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:10.125+0000 7f23811d0640 1 --2- 172.21.15.49:0/3760026201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2368003980 0x7f2368003da0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2378009ae0 tx=0x7f237801ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7145767e49b8b880 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:10.127 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:10.125+0000 7f2373fff640 1 -- 172.21.15.49:0/3760026201 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23780a5630 con 0x7f2368003980 2023-11-06T02:08:10.127 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:10.125+0000 7f2373fff640 1 -- 172.21.15.49:0/3760026201 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23780a5790 con 0x7f2368003980 2023-11-06T02:08:10.127 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:10.125+0000 7f2373fff640 1 -- 172.21.15.49:0/3760026201 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23780a5b40 con 0x7f2368003980 2023-11-06T02:08:10.128 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:10.125+0000 7f2383434640 1 -- 172.21.15.49:0/3760026201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2368003980 msgr2=0x7f2368003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:10.128 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:10.125+0000 7f2383434640 1 --2- 172.21.15.49:0/3760026201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2368003980 0x7f2368003da0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2378009ae0 tx=0x7f237801ca60 comp rx=0 tx=0).stop 2023-11-06T02:08:10.128 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:10.125+0000 7f2383434640 1 -- 172.21.15.49:0/3760026201 shutdown_connections 2023-11-06T02:08:10.128 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:10.125+0000 7f2383434640 1 --2- 172.21.15.49:0/3760026201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2368003980 0x7f2368003da0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:10.128 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:10.125+0000 7f2383434640 1 -- 172.21.15.49:0/3760026201 >> 172.21.15.49:0/3760026201 conn(0x7f237c06b080 msgr2=0x7f237c06b490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:10.129 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:10.125+0000 7f2383434640 1 -- 172.21.15.49:0/3760026201 shutdown_connections 2023-11-06T02:08:10.129 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:10.125+0000 7f2383434640 1 -- 172.21.15.49:0/3760026201 wait complete. 2023-11-06T02:08:10.129 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:10.125+0000 7f2383434640 1 Processor -- start 2023-11-06T02:08:10.129 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:10.125+0000 7f2383434640 1 -- start start 2023-11-06T02:08:10.129 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:10.125+0000 7f2383434640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2368003980 0x7f237c14ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:10.130 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:10.125+0000 7f2383434640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f237c0c66f0 0x7f237c15cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:10.130 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:10.125+0000 7f2383434640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f237c0c96a0 0x7f237c15d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:10.130 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:10.125+0000 7f2383434640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f237c0c8890 con 0x7f237c0c66f0 2023-11-06T02:08:10.130 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:10.125+0000 7f2383434640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f237c0ae3d0 con 0x7f2368003980 2023-11-06T02:08:10.130 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:10.125+0000 7f2383434640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f237c0ad7c0 con 0x7f237c0c96a0 2023-11-06T02:08:10.131 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:10.125+0000 7f23809cf640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f237c0c66f0 0x7f237c15cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:10.131 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:10.125+0000 7f23809cf640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f237c0c66f0 0x7f237c15cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37522/0 (socket says 172.21.15.49:37522) 2023-11-06T02:08:10.131 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:10.125+0000 7f23809cf640 1 -- 172.21.15.49:0/3772512928 learned_addr learned my addr 172.21.15.49:0/3772512928 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:10.131 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:10.125+0000 7f23811d0640 1 --2- 172.21.15.49:0/3772512928 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2368003980 0x7f237c14ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:10.131 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:10.125+0000 7f23819d1640 1 --2- 172.21.15.49:0/3772512928 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f237c0c96a0 0x7f237c15d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:10.132 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:10.129+0000 7f23809cf640 1 -- 172.21.15.49:0/3772512928 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f237c0c96a0 msgr2=0x7f237c15d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:10.132 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:10.129+0000 7f23809cf640 1 --2- 172.21.15.49:0/3772512928 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f237c0c96a0 0x7f237c15d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:10.132 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:10.129+0000 7f23809cf640 1 -- 172.21.15.49:0/3772512928 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2368003980 msgr2=0x7f237c14ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:10.132 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:10.129+0000 7f23809cf640 1 --2- 172.21.15.49:0/3772512928 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2368003980 0x7f237c14ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:10.132 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:10.129+0000 7f23809cf640 1 -- 172.21.15.49:0/3772512928 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f237c0adaa0 con 0x7f237c0c66f0 2023-11-06T02:08:10.133 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:10.129+0000 7f23819d1640 1 --2- 172.21.15.49:0/3772512928 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f237c0c96a0 0x7f237c15d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:08:10.133 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:10.129+0000 7f23809cf640 1 --2- 172.21.15.49:0/3772512928 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f237c0c66f0 0x7f237c15cfa0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f236c00c7c0 tx=0x7f236c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:10.133 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:10.129+0000 7f2371ffb640 1 -- 172.21.15.49:0/3772512928 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236c0074e0 con 0x7f237c0c66f0 2023-11-06T02:08:10.133 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:10.129+0000 7f2371ffb640 1 -- 172.21.15.49:0/3772512928 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f236c007640 con 0x7f237c0c66f0 2023-11-06T02:08:10.133 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:10.129+0000 7f2371ffb640 1 -- 172.21.15.49:0/3772512928 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236c004a30 con 0x7f237c0c66f0 2023-11-06T02:08:10.134 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:10.129+0000 7f2383434640 1 -- 172.21.15.49:0/3772512928 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f237c0add80 con 0x7f237c0c66f0 2023-11-06T02:08:10.134 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:10.129+0000 7f2383434640 1 -- 172.21.15.49:0/3772512928 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f237c14bfb0 con 0x7f237c0c66f0 2023-11-06T02:08:10.134 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:10.129+0000 7f2371ffb640 1 -- 172.21.15.49:0/3772512928 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f236c00a640 con 0x7f237c0c66f0 2023-11-06T02:08:10.134 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:10.129+0000 7f2371ffb640 1 -- 172.21.15.49:0/3772512928 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f236c012b70 con 0x7f237c0c66f0 2023-11-06T02:08:10.134 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:10.129+0000 7f2383434640 1 -- 172.21.15.49:0/3772512928 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f237c067140 con 0x7f237c0c66f0 2023-11-06T02:08:10.135 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:10.129+0000 7f2371ffb640 1 -- 172.21.15.49:0/3772512928 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f236c021060 con 0x7f237c0c66f0 2023-11-06T02:08:10.186 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:10.189+0000 7f2383434640 1 -- 172.21.15.49:0/3772512928 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f237c0ae510 con 0x7f237c0c66f0 2023-11-06T02:08:10.187 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:10.189+0000 7f2371ffb640 1 -- 172.21.15.49:0/3772512928 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f236c018070 con 0x7f237c0c66f0 2023-11-06T02:08:10.187 INFO:teuthology.misc.health.smithi049.stdout: 2023-11-06T02:08:10.188 INFO:teuthology.misc.health.smithi049.stdout:{"epoch":10,"fsid":"1a8b035c-6ba4-499a-b876-aea47fd9c0d4","created":"2023-11-06T02:08:06.440857+0000","modified":"2023-11-06T02:08:08.906631+0000","last_up_change":"0.000000","last_in_change":"2023-11-06T02:08:08.906631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"6ad55af8-0571-4e6a-8722-b5d429b7496d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"c8b31f85-a757-43ed-9419-351acc46611b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"555b3bf3-0a70-4b91-8efd-e4204b5fc26e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"8d4e5d77-03ef-4b9b-bfcb-5ae93a46795c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"dba03ad4-68a3-4cf3-9218-ac611f02eacf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"0df9c303-c255-4640-8e91-01f1c3efe18e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"e59fa189-4b70-4c06-9722-c02b90828bf2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"a64f57e1-ef98-46bd-b304-385759e95898","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T02:08:10.189 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:10.189+0000 7f2383434640 1 -- 172.21.15.49:0/3772512928 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f237c0c66f0 msgr2=0x7f237c15cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:10.189 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:10.189+0000 7f2383434640 1 --2- 172.21.15.49:0/3772512928 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f237c0c66f0 0x7f237c15cfa0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f236c00c7c0 tx=0x7f236c00cc80 comp rx=0 tx=0).stop 2023-11-06T02:08:10.189 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:10.189+0000 7f2383434640 1 -- 172.21.15.49:0/3772512928 shutdown_connections 2023-11-06T02:08:10.189 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:10.189+0000 7f2383434640 1 --2- 172.21.15.49:0/3772512928 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f237c0c66f0 0x7f237c15cfa0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:10.189 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:10.189+0000 7f2383434640 1 --2- 172.21.15.49:0/3772512928 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f237c0c96a0 0x7f237c15d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:10.190 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:10.189+0000 7f2383434640 1 --2- 172.21.15.49:0/3772512928 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2368003980 0x7f237c14ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:10.190 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:10.189+0000 7f2383434640 1 -- 172.21.15.49:0/3772512928 >> 172.21.15.49:0/3772512928 conn(0x7f237c06b080 msgr2=0x7f237c0b2770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:10.190 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:10.189+0000 7f2383434640 1 -- 172.21.15.49:0/3772512928 shutdown_connections 2023-11-06T02:08:10.190 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:10.189+0000 7f2383434640 1 -- 172.21.15.49:0/3772512928 wait complete. 2023-11-06T02:08:10.200 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-11-06T02:08:10.463 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-06T02:08:10.466+0000 7f4792183180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-06T02:08:10.493 INFO:tasks.ceph.mgr.y.smithi049.stderr:2023-11-06T02:08:10.497+0000 7f148018b180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-06T02:08:10.541 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-06T02:08:10.542+0000 7f4792183180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-06T02:08:10.572 INFO:tasks.ceph.mgr.y.smithi049.stderr:2023-11-06T02:08:10.577+0000 7f148018b180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-06T02:08:10.710 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-06T02:08:10.710+0000 7f4792183180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-06T02:08:10.751 INFO:tasks.ceph.mgr.y.smithi049.stderr:2023-11-06T02:08:10.757+0000 7f148018b180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-06T02:08:10.783 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-06T02:08:10.786+0000 7f4792183180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-06T02:08:10.829 INFO:tasks.ceph.mgr.y.smithi049.stderr:2023-11-06T02:08:10.833+0000 7f148018b180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-06T02:08:11.071 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-06T02:08:11.070+0000 7f4792183180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-06T02:08:11.132 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-06T02:08:11.134+0000 7f4792183180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-06T02:08:11.136 INFO:tasks.ceph.mgr.y.smithi049.stderr:2023-11-06T02:08:11.141+0000 7f148018b180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-06T02:08:11.200 INFO:tasks.ceph.mgr.y.smithi049.stderr:2023-11-06T02:08:11.205+0000 7f148018b180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-06T02:08:12.174 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-06T02:08:12.178+0000 7f4739a42640 -1 mgr.server handle_report got status from non-daemon mon.c 2023-11-06T02:08:12.178 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-06T02:08:12.182+0000 7f4739a42640 -1 mgr.server handle_report got status from non-daemon mon.b 2023-11-06T02:08:13.197 INFO:tasks.ceph.osd.5.smithi150.stderr:2023-11-06T02:08:13.202+0000 7fb747eec640 -1 osd.5 0 log_to_monitors true 2023-11-06T02:08:13.244 INFO:tasks.ceph.osd.2.smithi049.stderr:2023-11-06T02:08:13.249+0000 7ff10758f640 -1 osd.2 0 log_to_monitors true 2023-11-06T02:08:13.314 INFO:tasks.ceph.osd.4.smithi150.stderr:2023-11-06T02:08:13.318+0000 7f43e00fa640 -1 osd.4 0 log_to_monitors true 2023-11-06T02:08:13.424 INFO:tasks.ceph.osd.6.smithi150.stderr:2023-11-06T02:08:13.430+0000 7f388910a640 -1 osd.6 0 log_to_monitors true 2023-11-06T02:08:13.433 INFO:tasks.ceph.osd.7.smithi150.stderr:2023-11-06T02:08:13.438+0000 7f2f443ef640 -1 osd.7 0 log_to_monitors true 2023-11-06T02:08:13.440 INFO:tasks.ceph.osd.1.smithi049.stderr:2023-11-06T02:08:13.445+0000 7f4e7117c640 -1 osd.1 0 log_to_monitors true 2023-11-06T02:08:13.462 INFO:tasks.ceph.osd.3.smithi049.stderr:2023-11-06T02:08:13.465+0000 7ff97c09f640 -1 osd.3 0 log_to_monitors true 2023-11-06T02:08:13.466 INFO:tasks.ceph.osd.0.smithi049.stderr:2023-11-06T02:08:13.469+0000 7fd028a0c640 -1 osd.0 0 log_to_monitors true 2023-11-06T02:08:16.945 INFO:tasks.ceph.osd.2.smithi049.stderr:2023-11-06T02:08:16.948+0000 7ff10454c640 -1 osd.2 0 waiting for initial osdmap 2023-11-06T02:08:16.950 INFO:tasks.ceph.osd.3.smithi049.stderr:2023-11-06T02:08:16.952+0000 7ff97905c640 -1 osd.3 0 waiting for initial osdmap 2023-11-06T02:08:16.957 INFO:tasks.ceph.osd.1.smithi049.stderr:2023-11-06T02:08:16.960+0000 7f4e6c8fe640 -1 osd.1 0 waiting for initial osdmap 2023-11-06T02:08:16.971 INFO:tasks.ceph.osd.2.smithi049.stderr:2023-11-06T02:08:16.976+0000 7ff0fb356640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-06T02:08:16.983 INFO:tasks.ceph.osd.1.smithi049.stderr:2023-11-06T02:08:16.988+0000 7f4e64f43640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-06T02:08:16.984 INFO:tasks.ceph.osd.3.smithi049.stderr:2023-11-06T02:08:16.988+0000 7ff96fe66640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-06T02:08:17.006 DEBUG:teuthology.orchestra.run.smithi049:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-06T02:08:17.026 INFO:tasks.ceph.osd.0.smithi049.stderr:2023-11-06T02:08:17.028+0000 7fd02418e640 -1 osd.0 0 waiting for initial osdmap 2023-11-06T02:08:17.047 INFO:tasks.ceph.osd.0.smithi049.stderr:2023-11-06T02:08:17.052+0000 7fd01c7d3640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-06T02:08:17.116 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:17.120+0000 7f5d1133b640 1 Processor -- start 2023-11-06T02:08:17.116 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:17.120+0000 7f5d1133b640 1 -- start start 2023-11-06T02:08:17.118 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:17.120+0000 7f5d1133b640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f5d0c15dbd0 0x7f5d0c0ce020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:17.118 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:17.120+0000 7f5d1133b640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f5d0c0cf2b0 0x7f5d0c0cf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:17.118 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:17.120+0000 7f5d1133b640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d0c15e130 con 0x7f5d0c15d0e0 2023-11-06T02:08:17.118 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:17.120+0000 7f5d1133b640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d0c0ce560 con 0x7f5d0c15dbd0 2023-11-06T02:08:17.119 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:17.120+0000 7f5d1133b640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d0c15e410 con 0x7f5d0c0cf2b0 2023-11-06T02:08:17.119 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:17.120+0000 7f5d0a7fc640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f5d0c15dbd0 0x7f5d0c0ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:17.119 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:17.120+0000 7f5d0a7fc640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f5d0c15dbd0 0x7f5d0c0ce020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37572/0 (socket says 172.21.15.49:37572) 2023-11-06T02:08:17.119 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:17.120+0000 7f5d0a7fc640 1 -- 172.21.15.49:0/2133512924 learned_addr learned my addr 172.21.15.49:0/2133512924 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:17.119 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:17.120+0000 7f5d0b7fe640 1 --2- 172.21.15.49:0/2133512924 >> v2:172.21.15.150:3300/0 conn(0x7f5d0c0cf2b0 0x7f5d0c0cf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:17.120 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:17.120+0000 7f5d0a7fc640 1 -- 172.21.15.49:0/2133512924 >> v2:172.21.15.150:3300/0 conn(0x7f5d0c0cf2b0 msgr2=0x7f5d0c0cf6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:17.120 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:17.120+0000 7f5d0a7fc640 1 --2- 172.21.15.49:0/2133512924 >> v2:172.21.15.150:3300/0 conn(0x7f5d0c0cf2b0 0x7f5d0c0cf6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:17.120 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:17.120+0000 7f5d0a7fc640 1 -- 172.21.15.49:0/2133512924 >> v1:172.21.15.49:6789/0 conn(0x7f5d0c15d0e0 legacy=0x7f5d0c15d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:17.120 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:17.120+0000 7f5d0a7fc640 1 -- 172.21.15.49:0/2133512924 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d0c0cd540 con 0x7f5d0c15dbd0 2023-11-06T02:08:17.120 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:17.120+0000 7f5d09ffb640 1 -- 172.21.15.49:0/2133512924 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3458422051 0 0) 0x7f5d0c15e130 con 0x7f5d0c15d0e0 2023-11-06T02:08:17.124 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:17.124+0000 7f5d0a7fc640 1 --2- 172.21.15.49:0/2133512924 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d0c15dbd0 0x7f5d0c0ce020 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c0d03b0 tx=0x7f5d0001c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8981d91fb1a63a32 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:17.124 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:17.124+0000 7f5d09ffb640 1 -- 172.21.15.49:0/2133512924 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d000ab640 con 0x7f5d0c15dbd0 2023-11-06T02:08:17.124 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:17.124+0000 7f5d09ffb640 1 -- 172.21.15.49:0/2133512924 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d000a7070 con 0x7f5d0c15dbd0 2023-11-06T02:08:17.124 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:17.124+0000 7f5d09ffb640 1 -- 172.21.15.49:0/2133512924 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d000ab9c0 con 0x7f5d0c15dbd0 2023-11-06T02:08:17.124 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:17.124+0000 7f5d1133b640 1 -- 172.21.15.49:0/2133512924 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d0c15dbd0 msgr2=0x7f5d0c0ce020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:17.125 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:17.124+0000 7f5d1133b640 1 --2- 172.21.15.49:0/2133512924 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d0c15dbd0 0x7f5d0c0ce020 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c0d03b0 tx=0x7f5d0001c440 comp rx=0 tx=0).stop 2023-11-06T02:08:17.125 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:17.124+0000 7f5d1133b640 1 -- 172.21.15.49:0/2133512924 shutdown_connections 2023-11-06T02:08:17.125 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:17.124+0000 7f5d1133b640 1 --2- 172.21.15.49:0/2133512924 >> v2:172.21.15.150:3300/0 conn(0x7f5d0c0cf2b0 0x7f5d0c0cf6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:17.125 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:17.124+0000 7f5d1133b640 1 --2- 172.21.15.49:0/2133512924 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d0c15dbd0 0x7f5d0c0ce020 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:17.126 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:17.124+0000 7f5d1133b640 1 -- 172.21.15.49:0/2133512924 >> 172.21.15.49:0/2133512924 conn(0x7f5d0c06b000 msgr2=0x7f5d0c06b410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:17.126 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:17.124+0000 7f5d1133b640 1 -- 172.21.15.49:0/2133512924 shutdown_connections 2023-11-06T02:08:17.126 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:17.124+0000 7f5d1133b640 1 -- 172.21.15.49:0/2133512924 wait complete. 2023-11-06T02:08:17.126 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:17.124+0000 7f5d1133b640 1 Processor -- start 2023-11-06T02:08:17.126 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:17.124+0000 7f5d1133b640 1 -- start start 2023-11-06T02:08:17.127 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:17.124+0000 7f5d1133b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5d0c0cf2b0 0x7f5d0c152dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:17.127 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:17.124+0000 7f5d1133b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d0c15d0e0 0x7f5d0c0cbc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:17.127 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:17.124+0000 7f5d1133b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d0c15dbd0 0x7f5d0c154620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:17.127 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:17.124+0000 7f5d1133b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0c0caa00 con 0x7f5d0c15dbd0 2023-11-06T02:08:17.127 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:17.124+0000 7f5d1133b640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0c0caed0 con 0x7f5d0c15d0e0 2023-11-06T02:08:17.128 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:17.124+0000 7f5d1133b640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0c0cb1b0 con 0x7f5d0c0cf2b0 2023-11-06T02:08:17.128 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:17.124+0000 7f5d0b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d0c15dbd0 0x7f5d0c154620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:17.128 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:17.124+0000 7f5d0b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d0c15dbd0 0x7f5d0c154620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37578/0 (socket says 172.21.15.49:37578) 2023-11-06T02:08:17.128 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:17.124+0000 7f5d0b7fe640 1 -- 172.21.15.49:0/1793313771 learned_addr learned my addr 172.21.15.49:0/1793313771 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:17.128 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:17.124+0000 7f5d0a7fc640 1 --2- 172.21.15.49:0/1793313771 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d0c15d0e0 0x7f5d0c0cbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:17.128 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:17.124+0000 7f5d0affd640 1 --2- 172.21.15.49:0/1793313771 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5d0c0cf2b0 0x7f5d0c152dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:17.129 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:17.124+0000 7f5d0b7fe640 1 -- 172.21.15.49:0/1793313771 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5d0c0cf2b0 msgr2=0x7f5d0c152dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:17.129 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:17.124+0000 7f5d0b7fe640 1 --2- 172.21.15.49:0/1793313771 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5d0c0cf2b0 0x7f5d0c152dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:17.129 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:17.124+0000 7f5d0b7fe640 1 -- 172.21.15.49:0/1793313771 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d0c15d0e0 msgr2=0x7f5d0c0cbc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:17.129 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:17.124+0000 7f5d0b7fe640 1 --2- 172.21.15.49:0/1793313771 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d0c15d0e0 0x7f5d0c0cbc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:17.129 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:17.124+0000 7f5d0b7fe640 1 -- 172.21.15.49:0/1793313771 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d000094d0 con 0x7f5d0c15dbd0 2023-11-06T02:08:17.130 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:17.124+0000 7f5d0a7fc640 1 --2- 172.21.15.49:0/1793313771 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d0c15d0e0 0x7f5d0c0cbc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:08:17.130 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:17.124+0000 7f5d0affd640 1 --2- 172.21.15.49:0/1793313771 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5d0c0cf2b0 0x7f5d0c152dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:08:17.130 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:17.124+0000 7f5d0b7fe640 1 --2- 172.21.15.49:0/1793313771 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d0c15dbd0 0x7f5d0c154620 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5d04007e40 tx=0x7f5d0400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:17.130 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:17.128+0000 7f5cf37fe640 1 -- 172.21.15.49:0/1793313771 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d04014620 con 0x7f5d0c15dbd0 2023-11-06T02:08:17.130 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:17.128+0000 7f5cf37fe640 1 -- 172.21.15.49:0/1793313771 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d0400ee90 con 0x7f5d0c15dbd0 2023-11-06T02:08:17.130 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:17.128+0000 7f5d1133b640 1 -- 172.21.15.49:0/1793313771 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d0c0cb490 con 0x7f5d0c15dbd0 2023-11-06T02:08:17.131 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:17.128+0000 7f5cf37fe640 1 -- 172.21.15.49:0/1793313771 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d040149e0 con 0x7f5d0c15dbd0 2023-11-06T02:08:17.131 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:17.128+0000 7f5d1133b640 1 -- 172.21.15.49:0/1793313771 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d0c153310 con 0x7f5d0c15dbd0 2023-11-06T02:08:17.132 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:17.128+0000 7f5d1133b640 1 -- 172.21.15.49:0/1793313771 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d0c15b840 con 0x7f5d0c15dbd0 2023-11-06T02:08:17.132 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:17.128+0000 7f5cf37fe640 1 -- 172.21.15.49:0/1793313771 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 4) v1 ==== 85172+0+0 (secure 0 0 0) 0x7f5d04014b40 con 0x7f5d0c15dbd0 2023-11-06T02:08:17.132 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:17.128+0000 7f5cf37fe640 1 --2- 172.21.15.49:0/1793313771 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5cf80659d0 0x7f5cf8067e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:17.133 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:17.128+0000 7f5cf37fe640 1 -- 172.21.15.49:0/1793313771 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7f5d04082c70 con 0x7f5d0c15dbd0 2023-11-06T02:08:17.133 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:17.128+0000 7f5d0affd640 1 --2- 172.21.15.49:0/1793313771 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5cf80659d0 0x7f5cf8067e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:17.133 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:17.128+0000 7f5d0affd640 1 --2- 172.21.15.49:0/1793313771 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5cf80659d0 0x7f5cf8067e90 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc000f70 tx=0x7f5cfc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:17.133 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:17.132+0000 7f5cf37fe640 1 -- 172.21.15.49:0/1793313771 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f5d040514a0 con 0x7f5d0c15dbd0 2023-11-06T02:08:17.281 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:17.284+0000 7f5d1133b640 1 -- 172.21.15.49:0/1793313771 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5d0c073ff0 con 0x7f5d0c15dbd0 2023-11-06T02:08:17.282 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:17.284+0000 7f5cf37fe640 1 -- 172.21.15.49:0/1793313771 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f5d04058bb0 con 0x7f5d0c15dbd0 2023-11-06T02:08:17.283 INFO:teuthology.misc.health.smithi049.stdout: 2023-11-06T02:08:17.283 INFO:teuthology.misc.health.smithi049.stdout:{"epoch":12,"fsid":"1a8b035c-6ba4-499a-b876-aea47fd9c0d4","created":"2023-11-06T02:08:06.440857+0000","modified":"2023-11-06T02:08:15.189111+0000","last_up_change":"0.000000","last_in_change":"2023-11-06T02:08:08.906631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"6ad55af8-0571-4e6a-8722-b5d429b7496d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"c8b31f85-a757-43ed-9419-351acc46611b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"555b3bf3-0a70-4b91-8efd-e4204b5fc26e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"8d4e5d77-03ef-4b9b-bfcb-5ae93a46795c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"dba03ad4-68a3-4cf3-9218-ac611f02eacf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"0df9c303-c255-4640-8e91-01f1c3efe18e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"e59fa189-4b70-4c06-9722-c02b90828bf2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"a64f57e1-ef98-46bd-b304-385759e95898","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T02:08:17.285 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:17.288+0000 7f5d1133b640 1 -- 172.21.15.49:0/1793313771 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5cf80659d0 msgr2=0x7f5cf8067e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:17.285 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:17.288+0000 7f5d1133b640 1 --2- 172.21.15.49:0/1793313771 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5cf80659d0 0x7f5cf8067e90 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc000f70 tx=0x7f5cfc009040 comp rx=0 tx=0).stop 2023-11-06T02:08:17.285 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:17.288+0000 7f5d1133b640 1 -- 172.21.15.49:0/1793313771 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d0c15dbd0 msgr2=0x7f5d0c154620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:17.285 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:17.288+0000 7f5d1133b640 1 --2- 172.21.15.49:0/1793313771 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d0c15dbd0 0x7f5d0c154620 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5d04007e40 tx=0x7f5d0400e4b0 comp rx=0 tx=0).stop 2023-11-06T02:08:17.286 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:17.288+0000 7f5d1133b640 1 -- 172.21.15.49:0/1793313771 shutdown_connections 2023-11-06T02:08:17.286 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:17.288+0000 7f5d1133b640 1 --2- 172.21.15.49:0/1793313771 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5cf80659d0 0x7f5cf8067e90 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:17.286 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:17.288+0000 7f5d1133b640 1 --2- 172.21.15.49:0/1793313771 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d0c15dbd0 0x7f5d0c154620 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:17.286 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:17.288+0000 7f5d1133b640 1 --2- 172.21.15.49:0/1793313771 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d0c15d0e0 0x7f5d0c0cbc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:17.286 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:17.288+0000 7f5d1133b640 1 --2- 172.21.15.49:0/1793313771 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5d0c0cf2b0 0x7f5d0c152dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:17.287 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:17.288+0000 7f5d1133b640 1 -- 172.21.15.49:0/1793313771 >> 172.21.15.49:0/1793313771 conn(0x7f5d0c06b000 msgr2=0x7f5d0c0b7f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:17.287 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:17.288+0000 7f5d1133b640 1 -- 172.21.15.49:0/1793313771 shutdown_connections 2023-11-06T02:08:17.287 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:17.288+0000 7f5d1133b640 1 -- 172.21.15.49:0/1793313771 wait complete. 2023-11-06T02:08:17.299 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-11-06T02:08:17.461 INFO:tasks.ceph.osd.4.smithi150.stderr:2023-11-06T02:08:17.465+0000 7f43db87c640 -1 osd.4 0 waiting for initial osdmap 2023-11-06T02:08:17.472 INFO:tasks.ceph.osd.7.smithi150.stderr:2023-11-06T02:08:17.473+0000 7f2f413ac640 -1 osd.7 0 waiting for initial osdmap 2023-11-06T02:08:17.488 INFO:tasks.ceph.osd.5.smithi150.stderr:2023-11-06T02:08:17.489+0000 7fb744ea9640 -1 osd.5 0 waiting for initial osdmap 2023-11-06T02:08:17.496 INFO:tasks.ceph.osd.4.smithi150.stderr:2023-11-06T02:08:17.501+0000 7f43d3ec1640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-06T02:08:17.500 INFO:tasks.ceph.osd.7.smithi150.stderr:2023-11-06T02:08:17.505+0000 7f2f381b6640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-06T02:08:17.516 INFO:tasks.ceph.osd.5.smithi150.stderr:2023-11-06T02:08:17.521+0000 7fb73bcb3640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-06T02:08:17.538 INFO:tasks.ceph.osd.6.smithi150.stderr:2023-11-06T02:08:17.541+0000 7f388488c640 -1 osd.6 0 waiting for initial osdmap 2023-11-06T02:08:17.568 INFO:tasks.ceph.osd.6.smithi150.stderr:2023-11-06T02:08:17.573+0000 7f387ced1640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-06T02:08:21.367 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T02:08:21.371 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T02:08:24.106 DEBUG:teuthology.orchestra.run.smithi049:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-06T02:08:24.232 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:24.231+0000 7f3c6f56e640 1 Processor -- start 2023-11-06T02:08:24.232 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:24.235+0000 7f3c6f56e640 1 -- start start 2023-11-06T02:08:24.233 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:24.235+0000 7f3c6f56e640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f3c681599c0 0x7f3c680cdfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:24.233 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:24.235+0000 7f3c6f56e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3c680cafc0 0x7f3c680cb410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:24.234 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:24.235+0000 7f3c6f56e640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c680cb950 con 0x7f3c680711f0 2023-11-06T02:08:24.234 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:24.235+0000 7f3c6f56e640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c680b1b10 con 0x7f3c681599c0 2023-11-06T02:08:24.234 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:24.235+0000 7f3c6f56e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c68072370 con 0x7f3c680cafc0 2023-11-06T02:08:24.234 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:24.235+0000 7f3c6db0b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3c680cafc0 0x7f3c680cb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:24.234 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:24.235+0000 7f3c6db0b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3c680cafc0 0x7f3c680cb410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.49:47638/0 (socket says 172.21.15.49:47638) 2023-11-06T02:08:24.235 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:24.235+0000 7f3c6db0b640 1 -- 172.21.15.49:0/1161787886 learned_addr learned my addr 172.21.15.49:0/1161787886 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:24.235 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:24.235+0000 7f3c6d30a640 1 --2- 172.21.15.49:0/1161787886 >> v2:172.21.15.150:3300/0 conn(0x7f3c681599c0 0x7f3c680cdfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:24.235 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:24.235+0000 7f3c6db0b640 1 -- 172.21.15.49:0/1161787886 >> v2:172.21.15.150:3300/0 conn(0x7f3c681599c0 msgr2=0x7f3c680cdfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:24.236 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:24.235+0000 7f3c6db0b640 1 --2- 172.21.15.49:0/1161787886 >> v2:172.21.15.150:3300/0 conn(0x7f3c681599c0 0x7f3c680cdfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:24.236 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:24.235+0000 7f3c6db0b640 1 -- 172.21.15.49:0/1161787886 >> v1:172.21.15.150:6789/0 conn(0x7f3c680711f0 legacy=0x7f3c680b1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:24.236 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:24.235+0000 7f3c6db0b640 1 -- 172.21.15.49:0/1161787886 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c68071ad0 con 0x7f3c680cafc0 2023-11-06T02:08:24.237 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:24.239+0000 7f3c6db0b640 1 --2- 172.21.15.49:0/1161787886 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3c680cafc0 0x7f3c680cb410 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f3c60009e50 tx=0x7f3c6001c950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24f7b90bf34d4ab0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:24.237 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:24.239+0000 7f3c5ffff640 1 -- 172.21.15.49:0/1161787886 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c600a5810 con 0x7f3c680cafc0 2023-11-06T02:08:24.238 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:24.239+0000 7f3c5ffff640 1 -- 172.21.15.49:0/1161787886 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c600a5970 con 0x7f3c680cafc0 2023-11-06T02:08:24.238 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:24.239+0000 7f3c5ffff640 1 -- 172.21.15.49:0/1161787886 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c600a5d00 con 0x7f3c680cafc0 2023-11-06T02:08:24.238 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:24.239+0000 7f3c6f56e640 1 -- 172.21.15.49:0/1161787886 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3c680cafc0 msgr2=0x7f3c680cb410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:24.238 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:24.239+0000 7f3c6f56e640 1 --2- 172.21.15.49:0/1161787886 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3c680cafc0 0x7f3c680cb410 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f3c60009e50 tx=0x7f3c6001c950 comp rx=0 tx=0).stop 2023-11-06T02:08:24.238 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:24.239+0000 7f3c6f56e640 1 -- 172.21.15.49:0/1161787886 shutdown_connections 2023-11-06T02:08:24.239 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:24.239+0000 7f3c6f56e640 1 --2- 172.21.15.49:0/1161787886 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3c680cafc0 0x7f3c680cb410 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:24.239 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:24.239+0000 7f3c6f56e640 1 --2- 172.21.15.49:0/1161787886 >> v2:172.21.15.150:3300/0 conn(0x7f3c681599c0 0x7f3c680cdfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:24.239 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:24.239+0000 7f3c6f56e640 1 -- 172.21.15.49:0/1161787886 >> 172.21.15.49:0/1161787886 conn(0x7f3c6806a420 msgr2=0x7f3c6806c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:24.239 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:24.239+0000 7f3c6f56e640 1 -- 172.21.15.49:0/1161787886 shutdown_connections 2023-11-06T02:08:24.239 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:24.239+0000 7f3c6f56e640 1 -- 172.21.15.49:0/1161787886 wait complete. 2023-11-06T02:08:24.242 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:24.239+0000 7f3c6f56e640 1 Processor -- start 2023-11-06T02:08:24.242 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:24.239+0000 7f3c6f56e640 1 -- start start 2023-11-06T02:08:24.242 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:24.239+0000 7f3c6f56e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3c680711f0 0x7f3c6815b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:24.242 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:24.239+0000 7f3c6f56e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c680cafc0 0x7f3c681529c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:24.243 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:24.239+0000 7f3c6f56e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3c681599c0 0x7f3c68156e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:24.243 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:24.239+0000 7f3c6f56e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6815bd60 con 0x7f3c680cafc0 2023-11-06T02:08:24.243 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:24.239+0000 7f3c6f56e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c68158c00 con 0x7f3c680711f0 2023-11-06T02:08:24.243 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:24.239+0000 7f3c6f56e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c68158ea0 con 0x7f3c681599c0 2023-11-06T02:08:24.243 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:24.239+0000 7f3c6cb09640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c680cafc0 0x7f3c681529c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:24.244 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:24.239+0000 7f3c6cb09640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c680cafc0 0x7f3c681529c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41376/0 (socket says 172.21.15.49:41376) 2023-11-06T02:08:24.244 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:24.239+0000 7f3c6cb09640 1 -- 172.21.15.49:0/2363192691 learned_addr learned my addr 172.21.15.49:0/2363192691 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:24.244 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:24.239+0000 7f3c6db0b640 1 --2- 172.21.15.49:0/2363192691 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3c681599c0 0x7f3c68156e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:24.244 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:24.239+0000 7f3c6d30a640 1 --2- 172.21.15.49:0/2363192691 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3c680711f0 0x7f3c6815b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:24.244 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:24.239+0000 7f3c6cb09640 1 -- 172.21.15.49:0/2363192691 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3c681599c0 msgr2=0x7f3c68156e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:24.245 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:24.239+0000 7f3c6cb09640 1 --2- 172.21.15.49:0/2363192691 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3c681599c0 0x7f3c68156e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:24.245 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:24.239+0000 7f3c6cb09640 1 -- 172.21.15.49:0/2363192691 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3c680711f0 msgr2=0x7f3c6815b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:24.245 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:24.239+0000 7f3c6cb09640 1 --2- 172.21.15.49:0/2363192691 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3c680711f0 0x7f3c6815b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:24.245 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:24.239+0000 7f3c6cb09640 1 -- 172.21.15.49:0/2363192691 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c60009b30 con 0x7f3c680cafc0 2023-11-06T02:08:24.245 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:24.239+0000 7f3c6d30a640 1 --2- 172.21.15.49:0/2363192691 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3c680711f0 0x7f3c6815b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:08:24.246 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:24.243+0000 7f3c6cb09640 1 --2- 172.21.15.49:0/2363192691 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c680cafc0 0x7f3c681529c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3c5800c370 tx=0x7f3c5800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:24.246 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:24.243+0000 7f3c5dffb640 1 -- 172.21.15.49:0/2363192691 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c580052d0 con 0x7f3c680cafc0 2023-11-06T02:08:24.246 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:24.243+0000 7f3c5dffb640 1 -- 172.21.15.49:0/2363192691 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c58005430 con 0x7f3c680cafc0 2023-11-06T02:08:24.246 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:24.243+0000 7f3c6f56e640 1 -- 172.21.15.49:0/2363192691 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c68159180 con 0x7f3c680cafc0 2023-11-06T02:08:24.246 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:24.243+0000 7f3c5dffb640 1 -- 172.21.15.49:0/2363192691 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c58007260 con 0x7f3c680cafc0 2023-11-06T02:08:24.247 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:24.243+0000 7f3c6f56e640 1 -- 172.21.15.49:0/2363192691 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c680c23a0 con 0x7f3c680cafc0 2023-11-06T02:08:24.247 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:24.243+0000 7f3c6f56e640 1 -- 172.21.15.49:0/2363192691 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c6815bed0 con 0x7f3c680cafc0 2023-11-06T02:08:24.247 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:24.243+0000 7f3c5dffb640 1 -- 172.21.15.49:0/2363192691 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f3c580073c0 con 0x7f3c680cafc0 2023-11-06T02:08:24.247 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:24.243+0000 7f3c5dffb640 1 --2- 172.21.15.49:0/2363192691 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3c4c065a60 0x7f3c4c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:24.247 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:24.243+0000 7f3c5dffb640 1 -- 172.21.15.49:0/2363192691 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3c58081f90 con 0x7f3c680cafc0 2023-11-06T02:08:24.248 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:24.243+0000 7f3c6d30a640 1 --2- 172.21.15.49:0/2363192691 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3c4c065a60 0x7f3c4c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:24.248 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:24.243+0000 7f3c6d30a640 1 --2- 172.21.15.49:0/2363192691 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3c4c065a60 0x7f3c4c067f20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3c680cf410 tx=0x7f3c64008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:24.248 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:24.247+0000 7f3c5dffb640 1 -- 172.21.15.49:0/2363192691 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f3c580b3070 con 0x7f3c680cafc0 2023-11-06T02:08:24.408 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:24.411+0000 7f3c6f56e640 1 -- 172.21.15.49:0/2363192691 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3c6815ab20 con 0x7f3c680cafc0 2023-11-06T02:08:24.410 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:24.411+0000 7f3c5dffb640 1 -- 172.21.15.49:0/2363192691 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12382 (secure 0 0 0) 0x7f3c5806f070 con 0x7f3c680cafc0 2023-11-06T02:08:24.410 INFO:teuthology.misc.health.smithi049.stdout: 2023-11-06T02:08:24.411 INFO:teuthology.misc.health.smithi049.stdout:{"epoch":16,"fsid":"1a8b035c-6ba4-499a-b876-aea47fd9c0d4","created":"2023-11-06T02:08:06.440857+0000","modified":"2023-11-06T02:08:21.275020+0000","last_up_change":"2023-11-06T02:08:17.982372+0000","last_in_change":"2023-11-06T02:08:08.906631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T02:08:19.252222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ad55af8-0571-4e6a-8722-b5d429b7496d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6809","nonce":38116},{"type":"v1","addr":"172.21.15.49:6811","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6813","nonce":38116},{"type":"v1","addr":"172.21.15.49:6815","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6821","nonce":38116},{"type":"v1","addr":"172.21.15.49:6823","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6817","nonce":38116},{"type":"v1","addr":"172.21.15.49:6819","nonce":38116}]},"public_addr":"172.21.15.49:6811/38116","cluster_addr":"172.21.15.49:6815/38116","heartbeat_back_addr":"172.21.15.49:6823/38116","heartbeat_front_addr":"172.21.15.49:6819/38116","state":["exists","up"]},{"osd":1,"uuid":"c8b31f85-a757-43ed-9419-351acc46611b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":38113},{"type":"v1","addr":"172.21.15.49:6810","nonce":38113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":38113},{"type":"v1","addr":"172.21.15.49:6814","nonce":38113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":38113},{"type":"v1","addr":"172.21.15.49:6822","nonce":38113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":38113},{"type":"v1","addr":"172.21.15.49:6818","nonce":38113}]},"public_addr":"172.21.15.49:6810/38113","cluster_addr":"172.21.15.49:6814/38113","heartbeat_back_addr":"172.21.15.49:6822/38113","heartbeat_front_addr":"172.21.15.49:6818/38113","state":["exists","up"]},{"osd":2,"uuid":"555b3bf3-0a70-4b91-8efd-e4204b5fc26e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":38114},{"type":"v1","addr":"172.21.15.49:6801","nonce":38114}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":38114},{"type":"v1","addr":"172.21.15.49:6803","nonce":38114}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":38114},{"type":"v1","addr":"172.21.15.49:6807","nonce":38114}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":38114},{"type":"v1","addr":"172.21.15.49:6805","nonce":38114}]},"public_addr":"172.21.15.49:6801/38114","cluster_addr":"172.21.15.49:6803/38114","heartbeat_back_addr":"172.21.15.49:6807/38114","heartbeat_front_addr":"172.21.15.49:6805/38114","state":["exists","up"]},{"osd":3,"uuid":"8d4e5d77-03ef-4b9b-bfcb-5ae93a46795c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":38115},{"type":"v1","addr":"172.21.15.49:6825","nonce":38115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":38115},{"type":"v1","addr":"172.21.15.49:6827","nonce":38115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":38115},{"type":"v1","addr":"172.21.15.49:6831","nonce":38115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":38115},{"type":"v1","addr":"172.21.15.49:6829","nonce":38115}]},"public_addr":"172.21.15.49:6825/38115","cluster_addr":"172.21.15.49:6827/38115","heartbeat_back_addr":"172.21.15.49:6831/38115","heartbeat_front_addr":"172.21.15.49:6829/38115","state":["exists","up"]},{"osd":4,"uuid":"dba03ad4-68a3-4cf3-9218-ac611f02eacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":38417},{"type":"v1","addr":"172.21.15.150:6809","nonce":38417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":38417},{"type":"v1","addr":"172.21.15.150:6811","nonce":38417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":38417},{"type":"v1","addr":"172.21.15.150:6815","nonce":38417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":38417},{"type":"v1","addr":"172.21.15.150:6813","nonce":38417}]},"public_addr":"172.21.15.150:6809/38417","cluster_addr":"172.21.15.150:6811/38417","heartbeat_back_addr":"172.21.15.150:6815/38417","heartbeat_front_addr":"172.21.15.150:6813/38417","state":["exists","up"]},{"osd":5,"uuid":"0df9c303-c255-4640-8e91-01f1c3efe18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":38418},{"type":"v1","addr":"172.21.15.150:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":38418},{"type":"v1","addr":"172.21.15.150:6803","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":38418},{"type":"v1","addr":"172.21.15.150:6807","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":38418},{"type":"v1","addr":"172.21.15.150:6805","nonce":38418}]},"public_addr":"172.21.15.150:6801/38418","cluster_addr":"172.21.15.150:6803/38418","heartbeat_back_addr":"172.21.15.150:6807/38418","heartbeat_front_addr":"172.21.15.150:6805/38418","state":["exists","up"]},{"osd":6,"uuid":"e59fa189-4b70-4c06-9722-c02b90828bf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":38428},{"type":"v1","addr":"172.21.15.150:6825","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":38428},{"type":"v1","addr":"172.21.15.150:6827","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":38428},{"type":"v1","addr":"172.21.15.150:6831","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":38428},{"type":"v1","addr":"172.21.15.150:6829","nonce":38428}]},"public_addr":"172.21.15.150:6825/38428","cluster_addr":"172.21.15.150:6827/38428","heartbeat_back_addr":"172.21.15.150:6831/38428","heartbeat_front_addr":"172.21.15.150:6829/38428","state":["exists","up"]},{"osd":7,"uuid":"a64f57e1-ef98-46bd-b304-385759e95898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":38421},{"type":"v1","addr":"172.21.15.150:6817","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":38421},{"type":"v1","addr":"172.21.15.150:6819","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":38421},{"type":"v1","addr":"172.21.15.150:6823","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":38421},{"type":"v1","addr":"172.21.15.150:6821","nonce":38421}]},"public_addr":"172.21.15.150:6817/38421","cluster_addr":"172.21.15.150:6819/38421","heartbeat_back_addr":"172.21.15.150:6823/38421","heartbeat_front_addr":"172.21.15.150:6821/38421","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.438552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.413280+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.296466+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.424966+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.276953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.230984+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.410235+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.480084+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T02:08:24.413 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:24.415+0000 7f3c6f56e640 1 -- 172.21.15.49:0/2363192691 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3c4c065a60 msgr2=0x7f3c4c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:24.413 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:24.415+0000 7f3c6f56e640 1 --2- 172.21.15.49:0/2363192691 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3c4c065a60 0x7f3c4c067f20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3c680cf410 tx=0x7f3c64008040 comp rx=0 tx=0).stop 2023-11-06T02:08:24.414 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:24.415+0000 7f3c6f56e640 1 -- 172.21.15.49:0/2363192691 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c680cafc0 msgr2=0x7f3c681529c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:24.414 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:24.415+0000 7f3c6f56e640 1 --2- 172.21.15.49:0/2363192691 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c680cafc0 0x7f3c681529c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3c5800c370 tx=0x7f3c5800c830 comp rx=0 tx=0).stop 2023-11-06T02:08:24.414 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:24.415+0000 7f3c6f56e640 1 -- 172.21.15.49:0/2363192691 shutdown_connections 2023-11-06T02:08:24.414 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:24.415+0000 7f3c6f56e640 1 --2- 172.21.15.49:0/2363192691 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3c4c065a60 0x7f3c4c067f20 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:24.414 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:24.415+0000 7f3c6f56e640 1 --2- 172.21.15.49:0/2363192691 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3c680cafc0 0x7f3c681529c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:24.415 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:24.415+0000 7f3c6f56e640 1 --2- 172.21.15.49:0/2363192691 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3c681599c0 0x7f3c68156e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:24.415 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:24.415+0000 7f3c6f56e640 1 --2- 172.21.15.49:0/2363192691 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3c680711f0 0x7f3c6815b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:24.415 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:24.415+0000 7f3c6f56e640 1 -- 172.21.15.49:0/2363192691 >> 172.21.15.49:0/2363192691 conn(0x7f3c6806a420 msgr2=0x7f3c680c98c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:24.415 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:24.415+0000 7f3c6f56e640 1 -- 172.21.15.49:0/2363192691 shutdown_connections 2023-11-06T02:08:24.415 INFO:teuthology.misc.health.smithi049.stderr:2023-11-06T02:08:24.415+0000 7f3c6f56e640 1 -- 172.21.15.49:0/2363192691 wait complete. 2023-11-06T02:08:24.429 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-11-06T02:08:24.429 INFO:tasks.ceph:Creating RBD pool 2023-11-06T02:08:24.430 DEBUG:teuthology.orchestra.run.smithi049:> sudo ceph --cluster ceph osd pool create rbd 8 2023-11-06T02:08:24.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:24.559+0000 7fa236711640 1 Processor -- start 2023-11-06T02:08:24.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:24.559+0000 7fa236711640 1 -- start start 2023-11-06T02:08:24.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:24.559+0000 7fa236711640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa23015d160 0x7fa23015e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:24.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:24.559+0000 7fa236711640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fa23015e860 0x7fa2300707b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:24.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:24.559+0000 7fa236711640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa23015e100 con 0x7fa230153d50 2023-11-06T02:08:24.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:24.559+0000 7fa236711640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa23006e0a0 con 0x7fa23015e860 2023-11-06T02:08:24.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:24.559+0000 7fa236711640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa230154130 con 0x7fa23015d160 2023-11-06T02:08:24.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:24.559+0000 7fa22f7fe640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa23015d160 0x7fa23015e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:24.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:24.559+0000 7fa22f7fe640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa23015d160 0x7fa23015e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.49:47654/0 (socket says 172.21.15.49:47654) 2023-11-06T02:08:24.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:24.559+0000 7fa22f7fe640 1 -- 172.21.15.49:0/2391582644 learned_addr learned my addr 172.21.15.49:0/2391582644 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:24.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:24.559+0000 7fa234cae640 1 --2- 172.21.15.49:0/2391582644 >> v2:172.21.15.150:3300/0 conn(0x7fa23015e860 0x7fa2300707b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:24.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:24.559+0000 7fa22f7fe640 1 -- 172.21.15.49:0/2391582644 >> v2:172.21.15.150:3300/0 conn(0x7fa23015e860 msgr2=0x7fa2300707b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:24.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:24.559+0000 7fa22f7fe640 1 --2- 172.21.15.49:0/2391582644 >> v2:172.21.15.150:3300/0 conn(0x7fa23015e860 0x7fa2300707b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:24.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:24.559+0000 7fa22f7fe640 1 -- 172.21.15.49:0/2391582644 >> v1:172.21.15.150:6789/0 conn(0x7fa230153d50 legacy=0x7fa2300c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:24.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:24.559+0000 7fa22f7fe640 1 -- 172.21.15.49:0/2391582644 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa23015d5c0 con 0x7fa23015d160 2023-11-06T02:08:24.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:24.563+0000 7fa22f7fe640 1 --2- 172.21.15.49:0/2391582644 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa23015d160 0x7fa23015e320 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa224009870 tx=0x7fa22401c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f66399943089182 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:24.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:24.563+0000 7fa22effd640 1 -- 172.21.15.49:0/2391582644 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2240ad640 con 0x7fa23015d160 2023-11-06T02:08:24.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:24.563+0000 7fa22effd640 1 -- 172.21.15.49:0/2391582644 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2240ad7a0 con 0x7fa23015d160 2023-11-06T02:08:24.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:24.563+0000 7fa22effd640 1 -- 172.21.15.49:0/2391582644 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2240adb30 con 0x7fa23015d160 2023-11-06T02:08:24.563 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:24.563+0000 7fa236711640 1 -- 172.21.15.49:0/2391582644 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa23015d160 msgr2=0x7fa23015e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:24.563 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:24.563+0000 7fa236711640 1 --2- 172.21.15.49:0/2391582644 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa23015d160 0x7fa23015e320 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa224009870 tx=0x7fa22401c480 comp rx=0 tx=0).stop 2023-11-06T02:08:24.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:24.563+0000 7fa236711640 1 -- 172.21.15.49:0/2391582644 shutdown_connections 2023-11-06T02:08:24.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:24.563+0000 7fa236711640 1 --2- 172.21.15.49:0/2391582644 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa23015d160 0x7fa23015e320 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:24.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:24.563+0000 7fa236711640 1 --2- 172.21.15.49:0/2391582644 >> v2:172.21.15.150:3300/0 conn(0x7fa23015e860 0x7fa2300707b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:24.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:24.563+0000 7fa236711640 1 -- 172.21.15.49:0/2391582644 >> 172.21.15.49:0/2391582644 conn(0x7fa23006a370 msgr2=0x7fa2300ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:24.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:24.563+0000 7fa236711640 1 -- 172.21.15.49:0/2391582644 shutdown_connections 2023-11-06T02:08:24.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:24.563+0000 7fa236711640 1 -- 172.21.15.49:0/2391582644 wait complete. 2023-11-06T02:08:24.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:24.563+0000 7fa236711640 1 Processor -- start 2023-11-06T02:08:24.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:24.563+0000 7fa236711640 1 -- start start 2023-11-06T02:08:24.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:24.567+0000 7fa236711640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa230153d50 0x7fa230148a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:24.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:24.567+0000 7fa236711640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa23015d160 0x7fa230154b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:24.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:24.567+0000 7fa236711640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa23015e860 0x7fa230156660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:24.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:24.567+0000 7fa236711640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa230153950 con 0x7fa23015e860 2023-11-06T02:08:24.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:24.567+0000 7fa236711640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa23014b3d0 con 0x7fa23015d160 2023-11-06T02:08:24.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:24.567+0000 7fa236711640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa23014b620 con 0x7fa230153d50 2023-11-06T02:08:24.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:24.567+0000 7fa234cae640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa23015e860 0x7fa230156660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:24.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:24.567+0000 7fa234cae640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa23015e860 0x7fa230156660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41384/0 (socket says 172.21.15.49:41384) 2023-11-06T02:08:24.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:24.567+0000 7fa234cae640 1 -- 172.21.15.49:0/475102650 learned_addr learned my addr 172.21.15.49:0/475102650 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:24.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:24.567+0000 7fa22ffff640 1 --2- 172.21.15.49:0/475102650 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa230153d50 0x7fa230148a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:24.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:24.567+0000 7fa22f7fe640 1 --2- 172.21.15.49:0/475102650 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa23015d160 0x7fa230154b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:24.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:24.567+0000 7fa234cae640 1 -- 172.21.15.49:0/475102650 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa230153d50 msgr2=0x7fa230148a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:24.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:24.567+0000 7fa234cae640 1 --2- 172.21.15.49:0/475102650 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa230153d50 0x7fa230148a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:24.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:24.567+0000 7fa234cae640 1 -- 172.21.15.49:0/475102650 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa23015d160 msgr2=0x7fa230154b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:24.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:24.567+0000 7fa234cae640 1 --2- 172.21.15.49:0/475102650 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa23015d160 0x7fa230154b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:24.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:24.567+0000 7fa234cae640 1 -- 172.21.15.49:0/475102650 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2240094d0 con 0x7fa23015e860 2023-11-06T02:08:24.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:24.567+0000 7fa234cae640 1 --2- 172.21.15.49:0/475102650 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa23015e860 0x7fa230156660 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa22800efc0 tx=0x7fa22800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:24.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:24.567+0000 7fa22cff9640 1 -- 172.21.15.49:0/475102650 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa228009ba0 con 0x7fa23015e860 2023-11-06T02:08:24.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:24.567+0000 7fa22cff9640 1 -- 172.21.15.49:0/475102650 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa22800f040 con 0x7fa23015e860 2023-11-06T02:08:24.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:24.567+0000 7fa22cff9640 1 -- 172.21.15.49:0/475102650 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2280146c0 con 0x7fa23015e860 2023-11-06T02:08:24.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:24.567+0000 7fa236711640 1 -- 172.21.15.49:0/475102650 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa23014b900 con 0x7fa23015e860 2023-11-06T02:08:24.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:24.567+0000 7fa236711640 1 -- 172.21.15.49:0/475102650 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa23014bec0 con 0x7fa23015e860 2023-11-06T02:08:24.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:24.567+0000 7fa236711640 1 -- 172.21.15.49:0/475102650 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa230074040 con 0x7fa23015e860 2023-11-06T02:08:24.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:24.567+0000 7fa22cff9640 1 -- 172.21.15.49:0/475102650 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fa228014820 con 0x7fa23015e860 2023-11-06T02:08:24.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:24.571+0000 7fa22cff9640 1 --2- 172.21.15.49:0/475102650 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa21c065a60 0x7fa21c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:24.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:24.571+0000 7fa22cff9640 1 -- 172.21.15.49:0/475102650 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa2280838d0 con 0x7fa23015e860 2023-11-06T02:08:24.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:24.571+0000 7fa22ffff640 1 --2- 172.21.15.49:0/475102650 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa21c065a60 0x7fa21c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:24.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:24.571+0000 7fa22ffff640 1 --2- 172.21.15.49:0/475102650 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa21c065a60 0x7fa21c067f20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa220001810 tx=0x7fa220009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:24.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:24.575+0000 7fa22cff9640 1 -- 172.21.15.49:0/475102650 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fa2280515d0 con 0x7fa23015e860 2023-11-06T02:08:24.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:24.731+0000 7fa236711640 1 -- 172.21.15.49:0/475102650 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7fa23015cb80 con 0x7fa23015e860 2023-11-06T02:08:25.036 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:25.039+0000 7fa22cff9640 1 -- 172.21.15.49:0/475102650 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v17) v1 ==== 109+0+0 (secure 0 0 0) 0x7fa228054bf0 con 0x7fa23015e860 2023-11-06T02:08:25.037 INFO:teuthology.orchestra.run.smithi049.stderr:pool 'rbd' created 2023-11-06T02:08:25.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:25.039+0000 7fa236711640 1 -- 172.21.15.49:0/475102650 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa21c065a60 msgr2=0x7fa21c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:25.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:25.039+0000 7fa236711640 1 --2- 172.21.15.49:0/475102650 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa21c065a60 0x7fa21c067f20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa220001810 tx=0x7fa220009040 comp rx=0 tx=0).stop 2023-11-06T02:08:25.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:25.039+0000 7fa236711640 1 -- 172.21.15.49:0/475102650 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa23015e860 msgr2=0x7fa230156660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:25.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:25.039+0000 7fa236711640 1 --2- 172.21.15.49:0/475102650 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa23015e860 0x7fa230156660 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa22800efc0 tx=0x7fa22800c490 comp rx=0 tx=0).stop 2023-11-06T02:08:25.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:25.039+0000 7fa236711640 1 -- 172.21.15.49:0/475102650 shutdown_connections 2023-11-06T02:08:25.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:25.039+0000 7fa236711640 1 --2- 172.21.15.49:0/475102650 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa21c065a60 0x7fa21c067f20 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:25.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:25.039+0000 7fa236711640 1 --2- 172.21.15.49:0/475102650 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa23015e860 0x7fa230156660 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:25.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:25.039+0000 7fa236711640 1 --2- 172.21.15.49:0/475102650 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa23015d160 0x7fa230154b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:25.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:25.039+0000 7fa236711640 1 --2- 172.21.15.49:0/475102650 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa230153d50 0x7fa230148a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:25.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:25.039+0000 7fa236711640 1 -- 172.21.15.49:0/475102650 >> 172.21.15.49:0/475102650 conn(0x7fa23006a370 msgr2=0x7fa23015c470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:25.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:25.039+0000 7fa236711640 1 -- 172.21.15.49:0/475102650 shutdown_connections 2023-11-06T02:08:25.042 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:25.039+0000 7fa236711640 1 -- 172.21.15.49:0/475102650 wait complete. 2023-11-06T02:08:25.061 DEBUG:teuthology.orchestra.run.smithi049:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-11-06T02:08:25.185 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:25.187+0000 7f547c0fa640 1 Processor -- start 2023-11-06T02:08:25.185 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:25.187+0000 7f547c0fa640 1 -- start start 2023-11-06T02:08:25.186 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:25.187+0000 7f547c0fa640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f54740714a0 0x7f54740cd060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:25.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:25.187+0000 7f547c0fa640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f54740d25c0 0x7f547415d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:25.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:25.187+0000 7f547c0fa640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f547415da20 con 0x7f5474071880 2023-11-06T02:08:25.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:25.187+0000 7f547c0fa640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f547415d830 con 0x7f54740714a0 2023-11-06T02:08:25.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:25.187+0000 7f547c0fa640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54740d1d60 con 0x7f54740d25c0 2023-11-06T02:08:25.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:25.187+0000 7f5479e96640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f54740714a0 0x7f54740cd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:25.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:25.187+0000 7f547a697640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f5474071880 0x7f54740b4520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:53104/0 (socket says 172.21.15.49:53104) 2023-11-06T02:08:25.188 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:25.187+0000 7f547a697640 1 -- 172.21.15.49:0/1912204634 learned_addr learned my addr 172.21.15.49:0/1912204634 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:08:25.188 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:25.187+0000 7f5479695640 1 --2- 172.21.15.49:0/1912204634 >> v2:172.21.15.150:3300/0 conn(0x7f54740d25c0 0x7f547415d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:25.188 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:25.187+0000 7f5479e96640 1 -- 172.21.15.49:0/1912204634 >> v2:172.21.15.150:3300/0 conn(0x7f54740d25c0 msgr2=0x7f547415d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:25.188 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:25.187+0000 7f5479e96640 1 --2- 172.21.15.49:0/1912204634 >> v2:172.21.15.150:3300/0 conn(0x7f54740d25c0 0x7f547415d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:25.188 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:25.187+0000 7f5479e96640 1 -- 172.21.15.49:0/1912204634 >> v1:172.21.15.49:6789/0 conn(0x7f5474071880 legacy=0x7f54740b4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:25.189 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:25.187+0000 7f5479e96640 1 -- 172.21.15.49:0/1912204634 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f547415dd00 con 0x7f54740714a0 2023-11-06T02:08:25.189 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:25.187+0000 7f5478e94640 1 -- 172.21.15.49:0/1912204634 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1095126035 0 0) 0x7f547415da20 con 0x7f5474071880 2023-11-06T02:08:25.189 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:25.187+0000 7f5479695640 1 --2- 172.21.15.49:0/1912204634 >> v2:172.21.15.150:3300/0 conn(0x7f54740d25c0 0x7f547415d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:08:25.190 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:25.191+0000 7f5479e96640 1 --2- 172.21.15.49:0/1912204634 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54740714a0 0x7f54740cd060 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5474072090 tx=0x7f547001c310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=359f51eb86f9eae0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:25.190 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:25.191+0000 7f5478e94640 1 -- 172.21.15.49:0/1912204634 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54700ab600 con 0x7f54740714a0 2023-11-06T02:08:25.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:25.191+0000 7f5478e94640 1 -- 172.21.15.49:0/1912204634 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54700a7050 con 0x7f54740714a0 2023-11-06T02:08:25.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:25.191+0000 7f5478e94640 1 -- 172.21.15.49:0/1912204634 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54700ab980 con 0x7f54740714a0 2023-11-06T02:08:25.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:25.191+0000 7f547c0fa640 1 -- 172.21.15.49:0/1912204634 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54740714a0 msgr2=0x7f54740cd060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:25.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:25.191+0000 7f547c0fa640 1 --2- 172.21.15.49:0/1912204634 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54740714a0 0x7f54740cd060 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5474072090 tx=0x7f547001c310 comp rx=0 tx=0).stop 2023-11-06T02:08:25.192 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:25.191+0000 7f547c0fa640 1 -- 172.21.15.49:0/1912204634 shutdown_connections 2023-11-06T02:08:25.192 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:25.191+0000 7f547c0fa640 1 --2- 172.21.15.49:0/1912204634 >> v2:172.21.15.150:3300/0 conn(0x7f54740d25c0 0x7f547415d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:25.192 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:25.191+0000 7f547c0fa640 1 --2- 172.21.15.49:0/1912204634 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54740714a0 0x7f54740cd060 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:25.192 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:25.191+0000 7f547c0fa640 1 -- 172.21.15.49:0/1912204634 >> 172.21.15.49:0/1912204634 conn(0x7f54740d2aa0 msgr2=0x7f54740d2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:25.192 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:25.191+0000 7f547c0fa640 1 -- 172.21.15.49:0/1912204634 shutdown_connections 2023-11-06T02:08:25.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:25.191+0000 7f547c0fa640 1 -- 172.21.15.49:0/1912204634 wait complete. 2023-11-06T02:08:25.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:25.191+0000 7f547c0fa640 1 Processor -- start 2023-11-06T02:08:25.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:25.191+0000 7f547c0fa640 1 -- start start 2023-11-06T02:08:25.195 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:25.191+0000 7f547c0fa640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54740714a0 0x7f547415b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:25.196 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:25.191+0000 7f547c0fa640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5474071880 0x7f5474159500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:25.196 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:25.191+0000 7f547c0fa640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f54740d25c0 0x7f54740ca080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:25.196 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:25.191+0000 7f547c0fa640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f547415a5e0 con 0x7f54740714a0 2023-11-06T02:08:25.196 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:25.191+0000 7f547c0fa640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f547415a7f0 con 0x7f5474071880 2023-11-06T02:08:25.196 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:25.191+0000 7f547c0fa640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f547415aad0 con 0x7f54740d25c0 2023-11-06T02:08:25.197 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:25.191+0000 7f5479e96640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54740714a0 0x7f547415b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:25.197 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:25.191+0000 7f5479e96640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54740714a0 0x7f547415b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41394/0 (socket says 172.21.15.49:41394) 2023-11-06T02:08:25.197 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:25.191+0000 7f5479e96640 1 -- 172.21.15.49:0/1348735669 learned_addr learned my addr 172.21.15.49:0/1348735669 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:25.197 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:25.191+0000 7f547a697640 1 --2- 172.21.15.49:0/1348735669 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f54740d25c0 0x7f54740ca080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:25.197 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:25.191+0000 7f5479695640 1 --2- 172.21.15.49:0/1348735669 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5474071880 0x7f5474159500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:25.198 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:25.191+0000 7f5479e96640 1 -- 172.21.15.49:0/1348735669 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f54740d25c0 msgr2=0x7f54740ca080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:25.198 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:25.191+0000 7f5479e96640 1 --2- 172.21.15.49:0/1348735669 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f54740d25c0 0x7f54740ca080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:25.198 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:25.191+0000 7f5479e96640 1 -- 172.21.15.49:0/1348735669 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5474071880 msgr2=0x7f5474159500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:25.198 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:25.191+0000 7f5479e96640 1 --2- 172.21.15.49:0/1348735669 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5474071880 0x7f5474159500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:25.198 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:25.191+0000 7f5479e96640 1 -- 172.21.15.49:0/1348735669 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5470009420 con 0x7f54740714a0 2023-11-06T02:08:25.199 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:25.191+0000 7f5479695640 1 --2- 172.21.15.49:0/1348735669 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5474071880 0x7f5474159500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:08:25.199 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:25.195+0000 7f5479e96640 1 --2- 172.21.15.49:0/1348735669 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54740714a0 0x7f547415b890 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f54700ae040 tx=0x7f547001ab50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:25.199 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:25.195+0000 7f546a7fc640 1 -- 172.21.15.49:0/1348735669 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54700b7650 con 0x7f54740714a0 2023-11-06T02:08:25.199 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:25.195+0000 7f546a7fc640 1 -- 172.21.15.49:0/1348735669 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54700b77d0 con 0x7f54740714a0 2023-11-06T02:08:25.199 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:25.195+0000 7f547c0fa640 1 -- 172.21.15.49:0/1348735669 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f547415adb0 con 0x7f54740714a0 2023-11-06T02:08:25.200 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:25.195+0000 7f546a7fc640 1 -- 172.21.15.49:0/1348735669 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54700b7b80 con 0x7f54740714a0 2023-11-06T02:08:25.200 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:25.195+0000 7f547c0fa640 1 -- 172.21.15.49:0/1348735669 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f547415b370 con 0x7f54740714a0 2023-11-06T02:08:25.200 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:25.195+0000 7f547c0fa640 1 -- 172.21.15.49:0/1348735669 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5474074040 con 0x7f54740714a0 2023-11-06T02:08:25.200 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:25.195+0000 7f546a7fc640 1 -- 172.21.15.49:0/1348735669 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f54700b7ce0 con 0x7f54740714a0 2023-11-06T02:08:25.200 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:25.195+0000 7f546a7fc640 1 --2- 172.21.15.49:0/1348735669 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5458065a60 0x7f5458067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:25.201 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:25.195+0000 7f546a7fc640 1 -- 172.21.15.49:0/1348735669 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5199+0+0 (secure 0 0 0) 0x7f547011dae0 con 0x7f54740714a0 2023-11-06T02:08:25.201 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:25.195+0000 7f5479695640 1 --2- 172.21.15.49:0/1348735669 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5458065a60 0x7f5458067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:25.201 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:25.195+0000 7f5479695640 1 --2- 172.21.15.49:0/1348735669 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5458065a60 0x7f5458067f20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f547414cd60 tx=0x7f5464008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:25.201 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:25.199+0000 7f546a7fc640 1 -- 172.21.15.49:0/1348735669 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f54700eb730 con 0x7f54740714a0 2023-11-06T02:08:25.354 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:25.355+0000 7f547c0fa640 1 -- 172.21.15.49:0/1348735669 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f54740b79f0 con 0x7f54740714a0 2023-11-06T02:08:26.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.043+0000 7f546a7fc640 1 -- 172.21.15.49:0/1348735669 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v18) v1 ==== 173+0+0 (secure 0 0 0) 0x7f54700bb0c0 con 0x7f54740714a0 2023-11-06T02:08:26.042 INFO:teuthology.orchestra.run.smithi049.stderr:enabled application 'rbd' on pool 'rbd' 2023-11-06T02:08:26.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.047+0000 7f547c0fa640 1 -- 172.21.15.49:0/1348735669 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5458065a60 msgr2=0x7f5458067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:26.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.047+0000 7f547c0fa640 1 --2- 172.21.15.49:0/1348735669 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5458065a60 0x7f5458067f20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f547414cd60 tx=0x7f5464008040 comp rx=0 tx=0).stop 2023-11-06T02:08:26.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.047+0000 7f547c0fa640 1 -- 172.21.15.49:0/1348735669 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54740714a0 msgr2=0x7f547415b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:26.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.047+0000 7f547c0fa640 1 --2- 172.21.15.49:0/1348735669 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54740714a0 0x7f547415b890 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f54700ae040 tx=0x7f547001ab50 comp rx=0 tx=0).stop 2023-11-06T02:08:26.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.047+0000 7f547c0fa640 1 -- 172.21.15.49:0/1348735669 shutdown_connections 2023-11-06T02:08:26.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.047+0000 7f547c0fa640 1 --2- 172.21.15.49:0/1348735669 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5458065a60 0x7f5458067f20 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:26.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.047+0000 7f547c0fa640 1 --2- 172.21.15.49:0/1348735669 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f54740d25c0 0x7f54740ca080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:26.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.047+0000 7f547c0fa640 1 --2- 172.21.15.49:0/1348735669 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5474071880 0x7f5474159500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:26.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.047+0000 7f547c0fa640 1 --2- 172.21.15.49:0/1348735669 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54740714a0 0x7f547415b890 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:26.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.047+0000 7f547c0fa640 1 -- 172.21.15.49:0/1348735669 >> 172.21.15.49:0/1348735669 conn(0x7f54740d2aa0 msgr2=0x7f54740cfdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:26.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.047+0000 7f547c0fa640 1 -- 172.21.15.49:0/1348735669 shutdown_connections 2023-11-06T02:08:26.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.047+0000 7f547c0fa640 1 -- 172.21.15.49:0/1348735669 wait complete. 2023-11-06T02:08:26.067 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-11-06T02:08:26.068 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-06T02:08:26.068 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-06T02:08:26.070 INFO:tasks.ceph.mds.a:Started 2023-11-06T02:08:26.071 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-06T02:08:26.071 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-06T02:08:26.073 INFO:tasks.ceph.mds.c:Started 2023-11-06T02:08:26.073 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-06T02:08:26.074 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-06T02:08:26.077 INFO:tasks.ceph.mds.b:Started 2023-11-06T02:08:26.077 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-06T02:08:26.078 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-06T02:08:26.080 INFO:tasks.ceph.mds.d:Started 2023-11-06T02:08:26.081 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-11-06T02:08:26.081 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T02:08:26.155 INFO:tasks.ceph.mds.a.smithi049.stdout:starting mds.a at 2023-11-06T02:08:26.156 INFO:tasks.ceph.mds.c.smithi049.stdout:starting mds.c at 2023-11-06T02:08:26.159 INFO:tasks.ceph.mds.d.smithi150.stdout:starting mds.d at 2023-11-06T02:08:26.161 INFO:tasks.ceph.mds.b.smithi150.stdout:starting mds.b at 2023-11-06T02:08:26.197 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.199+0000 7efca15f2640 1 Processor -- start 2023-11-06T02:08:26.198 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.199+0000 7efca15f2640 1 -- start start 2023-11-06T02:08:26.199 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.199+0000 7efca15f2640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efc9c15ceb0 0x7efc9c0cb970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:26.199 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.199+0000 7efca15f2640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7efc9c0caf30 0x7efc9c071120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:26.199 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.199+0000 7efca15f2640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7efc9c0cb370 0x7efc9c071660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:26.200 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.199+0000 7efca15f2640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc9c0cbeb0 con 0x7efc9c0cb370 2023-11-06T02:08:26.200 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.199+0000 7efca15f2640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc9c0caaa0 con 0x7efc9c0caf30 2023-11-06T02:08:26.200 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.199+0000 7efca15f2640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc9c0ca1e0 con 0x7efc9c15ceb0 2023-11-06T02:08:26.200 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.199+0000 7efc9b7fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7efc9c0cb370 0x7efc9c071660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:26.200 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.199+0000 7efc9b7fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7efc9c0cb370 0x7efc9c071660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41416/0 (socket says 172.21.15.49:41416) 2023-11-06T02:08:26.201 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.199+0000 7efc9b7fe640 1 -- 172.21.15.49:0/2239979889 learned_addr learned my addr 172.21.15.49:0/2239979889 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:26.201 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.199+0000 7efc9affd640 1 --2- 172.21.15.49:0/2239979889 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efc9c15ceb0 0x7efc9c0cb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:26.201 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.199+0000 7efc9a7fc640 1 --2- 172.21.15.49:0/2239979889 >> v2:172.21.15.150:3300/0 conn(0x7efc9c0caf30 0x7efc9c071120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:26.201 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.199+0000 7efc9b7fe640 1 -- 172.21.15.49:0/2239979889 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efc9c15ceb0 msgr2=0x7efc9c0cb970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:26.201 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.199+0000 7efc9b7fe640 1 --2- 172.21.15.49:0/2239979889 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efc9c15ceb0 0x7efc9c0cb970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:26.202 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.199+0000 7efc9b7fe640 1 -- 172.21.15.49:0/2239979889 >> v2:172.21.15.150:3300/0 conn(0x7efc9c0caf30 msgr2=0x7efc9c071120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:26.202 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.199+0000 7efc9b7fe640 1 --2- 172.21.15.49:0/2239979889 >> v2:172.21.15.150:3300/0 conn(0x7efc9c0caf30 0x7efc9c071120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:26.202 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.199+0000 7efc9b7fe640 1 -- 172.21.15.49:0/2239979889 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc9c0ce490 con 0x7efc9c0cb370 2023-11-06T02:08:26.205 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.203+0000 7efc9b7fe640 1 --2- 172.21.15.49:0/2239979889 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc9c0cb370 0x7efc9c071660 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7efc9c066430 tx=0x7efc9401c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4db9c7b208e525c7 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:26.205 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.203+0000 7efc99ffb640 1 -- 172.21.15.49:0/2239979889 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc940a5740 con 0x7efc9c0cb370 2023-11-06T02:08:26.205 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.203+0000 7efc99ffb640 1 -- 172.21.15.49:0/2239979889 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc940a58a0 con 0x7efc9c0cb370 2023-11-06T02:08:26.205 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.203+0000 7efc99ffb640 1 -- 172.21.15.49:0/2239979889 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc940a5c30 con 0x7efc9c0cb370 2023-11-06T02:08:26.206 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.203+0000 7efca15f2640 1 -- 172.21.15.49:0/2239979889 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc9c0cb370 msgr2=0x7efc9c071660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:26.206 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.203+0000 7efca15f2640 1 --2- 172.21.15.49:0/2239979889 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc9c0cb370 0x7efc9c071660 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7efc9c066430 tx=0x7efc9401c9d0 comp rx=0 tx=0).stop 2023-11-06T02:08:26.206 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.203+0000 7efca15f2640 1 -- 172.21.15.49:0/2239979889 shutdown_connections 2023-11-06T02:08:26.206 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.203+0000 7efca15f2640 1 --2- 172.21.15.49:0/2239979889 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc9c0cb370 0x7efc9c071660 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:26.206 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.203+0000 7efca15f2640 1 --2- 172.21.15.49:0/2239979889 >> v2:172.21.15.150:3300/0 conn(0x7efc9c0caf30 0x7efc9c071120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:26.207 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.203+0000 7efca15f2640 1 --2- 172.21.15.49:0/2239979889 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efc9c15ceb0 0x7efc9c0cb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:26.207 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.203+0000 7efca15f2640 1 -- 172.21.15.49:0/2239979889 >> 172.21.15.49:0/2239979889 conn(0x7efc9c0cc140 msgr2=0x7efc9c0cc550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:26.207 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.203+0000 7efca15f2640 1 -- 172.21.15.49:0/2239979889 shutdown_connections 2023-11-06T02:08:26.207 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.203+0000 7efca15f2640 1 -- 172.21.15.49:0/2239979889 wait complete. 2023-11-06T02:08:26.207 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.203+0000 7efca15f2640 1 Processor -- start 2023-11-06T02:08:26.208 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.203+0000 7efca15f2640 1 -- start start 2023-11-06T02:08:26.208 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.203+0000 7efca15f2640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc9c0caf30 0x7efc9c1559e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:26.208 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.203+0000 7efca15f2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc9c15ceb0 0x7efc9c0adcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:26.208 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.203+0000 7efca15f2640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efc9c0c6bd0 0x7efc9c0ae200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:26.208 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.203+0000 7efca15f2640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc9c0c7da0 con 0x7efc9c15ceb0 2023-11-06T02:08:26.208 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.203+0000 7efca15f2640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc9c0c7120 con 0x7efc9c0caf30 2023-11-06T02:08:26.209 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.203+0000 7efca15f2640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc9c0c7400 con 0x7efc9c0c6bd0 2023-11-06T02:08:26.209 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.203+0000 7efc9a7fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc9c15ceb0 0x7efc9c0adcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:26.209 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.203+0000 7efc9a7fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc9c15ceb0 0x7efc9c0adcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41422/0 (socket says 172.21.15.49:41422) 2023-11-06T02:08:26.209 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.203+0000 7efc9a7fc640 1 -- 172.21.15.49:0/3111709203 learned_addr learned my addr 172.21.15.49:0/3111709203 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:26.209 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.203+0000 7efc9affd640 1 --2- 172.21.15.49:0/3111709203 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc9c0caf30 0x7efc9c1559e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:26.210 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.207+0000 7efc9b7fe640 1 --2- 172.21.15.49:0/3111709203 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efc9c0c6bd0 0x7efc9c0ae200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:26.210 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.207+0000 7efc9a7fc640 1 -- 172.21.15.49:0/3111709203 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efc9c0c6bd0 msgr2=0x7efc9c0ae200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:26.210 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.207+0000 7efc9a7fc640 1 --2- 172.21.15.49:0/3111709203 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efc9c0c6bd0 0x7efc9c0ae200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:26.210 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.207+0000 7efc9a7fc640 1 -- 172.21.15.49:0/3111709203 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc9c0caf30 msgr2=0x7efc9c1559e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:26.210 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.207+0000 7efc9a7fc640 1 --2- 172.21.15.49:0/3111709203 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc9c0caf30 0x7efc9c1559e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:26.211 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.207+0000 7efc9a7fc640 1 -- 172.21.15.49:0/3111709203 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc94009b30 con 0x7efc9c15ceb0 2023-11-06T02:08:26.211 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.207+0000 7efc9a7fc640 1 --2- 172.21.15.49:0/3111709203 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc9c15ceb0 0x7efc9c0adcc0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7efc9000e8b0 tx=0x7efc9000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:26.211 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.207+0000 7efc837fe640 1 -- 172.21.15.49:0/3111709203 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc9000c940 con 0x7efc9c15ceb0 2023-11-06T02:08:26.211 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.207+0000 7efc837fe640 1 -- 172.21.15.49:0/3111709203 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc9000caa0 con 0x7efc9c15ceb0 2023-11-06T02:08:26.211 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.207+0000 7efc837fe640 1 -- 172.21.15.49:0/3111709203 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc9000ce30 con 0x7efc9c15ceb0 2023-11-06T02:08:26.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.207+0000 7efca15f2640 1 -- 172.21.15.49:0/3111709203 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc9c0c76e0 con 0x7efc9c15ceb0 2023-11-06T02:08:26.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.207+0000 7efca15f2640 1 -- 172.21.15.49:0/3111709203 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc9c156060 con 0x7efc9c15ceb0 2023-11-06T02:08:26.213 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.207+0000 7efca15f2640 1 -- 172.21.15.49:0/3111709203 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc9c153a80 con 0x7efc9c15ceb0 2023-11-06T02:08:26.213 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.207+0000 7efc837fe640 1 -- 172.21.15.49:0/3111709203 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7efc90013470 con 0x7efc9c15ceb0 2023-11-06T02:08:26.213 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.207+0000 7efc837fe640 1 --2- 172.21.15.49:0/3111709203 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7efc88065a60 0x7efc88067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:26.214 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.207+0000 7efc837fe640 1 -- 172.21.15.49:0/3111709203 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5210+0+0 (secure 0 0 0) 0x7efc90083160 con 0x7efc9c15ceb0 2023-11-06T02:08:26.214 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.207+0000 7efc9affd640 1 --2- 172.21.15.49:0/3111709203 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7efc88065a60 0x7efc88067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:26.214 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.207+0000 7efc9affd640 1 --2- 172.21.15.49:0/3111709203 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7efc88065a60 0x7efc88067f20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7efc8c009a50 tx=0x7efc8c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:26.214 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.211+0000 7efc837fe640 1 -- 172.21.15.49:0/3111709203 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7efc90050d10 con 0x7efc9c15ceb0 2023-11-06T02:08:26.368 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.371+0000 7efca15f2640 1 -- 172.21.15.49:0/3111709203 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efc9c071e40 con 0x7efc9c15ceb0 2023-11-06T02:08:26.370 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.371+0000 7efc837fe640 1 -- 172.21.15.49:0/3111709203 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+14181 (secure 0 0 0) 0x7efc90054330 con 0x7efc9c15ceb0 2023-11-06T02:08:26.370 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:08:26.370 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":18,"fsid":"1a8b035c-6ba4-499a-b876-aea47fd9c0d4","created":"2023-11-06T02:08:06.440857+0000","modified":"2023-11-06T02:08:26.038425+0000","last_up_change":"2023-11-06T02:08:17.982372+0000","last_in_change":"2023-11-06T02:08:08.906631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T02:08:19.252222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T02:08:24.776211+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ad55af8-0571-4e6a-8722-b5d429b7496d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6809","nonce":38116},{"type":"v1","addr":"172.21.15.49:6811","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6813","nonce":38116},{"type":"v1","addr":"172.21.15.49:6815","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6821","nonce":38116},{"type":"v1","addr":"172.21.15.49:6823","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6817","nonce":38116},{"type":"v1","addr":"172.21.15.49:6819","nonce":38116}]},"public_addr":"172.21.15.49:6811/38116","cluster_addr":"172.21.15.49:6815/38116","heartbeat_back_addr":"172.21.15.49:6823/38116","heartbeat_front_addr":"172.21.15.49:6819/38116","state":["exists","up"]},{"osd":1,"uuid":"c8b31f85-a757-43ed-9419-351acc46611b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":38113},{"type":"v1","addr":"172.21.15.49:6810","nonce":38113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":38113},{"type":"v1","addr":"172.21.15.49:6814","nonce":38113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":38113},{"type":"v1","addr":"172.21.15.49:6822","nonce":38113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":38113},{"type":"v1","addr":"172.21.15.49:6818","nonce":38113}]},"public_addr":"172.21.15.49:6810/38113","cluster_addr":"172.21.15.49:6814/38113","heartbeat_back_addr":"172.21.15.49:6822/38113","heartbeat_front_addr":"172.21.15.49:6818/38113","state":["exists","up"]},{"osd":2,"uuid":"555b3bf3-0a70-4b91-8efd-e4204b5fc26e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":38114},{"type":"v1","addr":"172.21.15.49:6801","nonce":38114}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":38114},{"type":"v1","addr":"172.21.15.49:6803","nonce":38114}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":38114},{"type":"v1","addr":"172.21.15.49:6807","nonce":38114}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":38114},{"type":"v1","addr":"172.21.15.49:6805","nonce":38114}]},"public_addr":"172.21.15.49:6801/38114","cluster_addr":"172.21.15.49:6803/38114","heartbeat_back_addr":"172.21.15.49:6807/38114","heartbeat_front_addr":"172.21.15.49:6805/38114","state":["exists","up"]},{"osd":3,"uuid":"8d4e5d77-03ef-4b9b-bfcb-5ae93a46795c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":38115},{"type":"v1","addr":"172.21.15.49:6825","nonce":38115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":38115},{"type":"v1","addr":"172.21.15.49:6827","nonce":38115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":38115},{"type":"v1","addr":"172.21.15.49:6831","nonce":38115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":38115},{"type":"v1","addr":"172.21.15.49:6829","nonce":38115}]},"public_addr":"172.21.15.49:6825/38115","cluster_addr":"172.21.15.49:6827/38115","heartbeat_back_addr":"172.21.15.49:6831/38115","heartbeat_front_addr":"172.21.15.49:6829/38115","state":["exists","up"]},{"osd":4,"uuid":"dba03ad4-68a3-4cf3-9218-ac611f02eacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":38417},{"type":"v1","addr":"172.21.15.150:6809","nonce":38417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":38417},{"type":"v1","addr":"172.21.15.150:6811","nonce":38417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":38417},{"type":"v1","addr":"172.21.15.150:6815","nonce":38417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":38417},{"type":"v1","addr":"172.21.15.150:6813","nonce":38417}]},"public_addr":"172.21.15.150:6809/38417","cluster_addr":"172.21.15.150:6811/38417","heartbeat_back_addr":"172.21.15.150:6815/38417","heartbeat_front_addr":"172.21.15.150:6813/38417","state":["exists","up"]},{"osd":5,"uuid":"0df9c303-c255-4640-8e91-01f1c3efe18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":38418},{"type":"v1","addr":"172.21.15.150:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":38418},{"type":"v1","addr":"172.21.15.150:6803","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":38418},{"type":"v1","addr":"172.21.15.150:6807","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":38418},{"type":"v1","addr":"172.21.15.150:6805","nonce":38418}]},"public_addr":"172.21.15.150:6801/38418","cluster_addr":"172.21.15.150:6803/38418","heartbeat_back_addr":"172.21.15.150:6807/38418","heartbeat_front_addr":"172.21.15.150:6805/38418","state":["exists","up"]},{"osd":6,"uuid":"e59fa189-4b70-4c06-9722-c02b90828bf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":38428},{"type":"v1","addr":"172.21.15.150:6825","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":38428},{"type":"v1","addr":"172.21.15.150:6827","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":38428},{"type":"v1","addr":"172.21.15.150:6831","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":38428},{"type":"v1","addr":"172.21.15.150:6829","nonce":38428}]},"public_addr":"172.21.15.150:6825/38428","cluster_addr":"172.21.15.150:6827/38428","heartbeat_back_addr":"172.21.15.150:6831/38428","heartbeat_front_addr":"172.21.15.150:6829/38428","state":["exists","up"]},{"osd":7,"uuid":"a64f57e1-ef98-46bd-b304-385759e95898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":38421},{"type":"v1","addr":"172.21.15.150:6817","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":38421},{"type":"v1","addr":"172.21.15.150:6819","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":38421},{"type":"v1","addr":"172.21.15.150:6823","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":38421},{"type":"v1","addr":"172.21.15.150:6821","nonce":38421}]},"public_addr":"172.21.15.150:6817/38421","cluster_addr":"172.21.15.150:6819/38421","heartbeat_back_addr":"172.21.15.150:6823/38421","heartbeat_front_addr":"172.21.15.150:6821/38421","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.438552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.413280+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.296466+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.424966+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.276953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.230984+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.410235+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.480084+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T02:08:26.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.371+0000 7efca15f2640 1 -- 172.21.15.49:0/3111709203 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7efc88065a60 msgr2=0x7efc88067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:26.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.371+0000 7efca15f2640 1 --2- 172.21.15.49:0/3111709203 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7efc88065a60 0x7efc88067f20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7efc8c009a50 tx=0x7efc8c008040 comp rx=0 tx=0).stop 2023-11-06T02:08:26.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.371+0000 7efca15f2640 1 -- 172.21.15.49:0/3111709203 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc9c15ceb0 msgr2=0x7efc9c0adcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:26.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.371+0000 7efca15f2640 1 --2- 172.21.15.49:0/3111709203 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc9c15ceb0 0x7efc9c0adcc0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7efc9000e8b0 tx=0x7efc9000ed70 comp rx=0 tx=0).stop 2023-11-06T02:08:26.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.371+0000 7efca15f2640 1 -- 172.21.15.49:0/3111709203 shutdown_connections 2023-11-06T02:08:26.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.371+0000 7efca15f2640 1 --2- 172.21.15.49:0/3111709203 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7efc88065a60 0x7efc88067f20 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:26.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.371+0000 7efca15f2640 1 --2- 172.21.15.49:0/3111709203 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc9c15ceb0 0x7efc9c0adcc0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:26.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.371+0000 7efca15f2640 1 --2- 172.21.15.49:0/3111709203 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efc9c0c6bd0 0x7efc9c0ae200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:26.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.371+0000 7efca15f2640 1 --2- 172.21.15.49:0/3111709203 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc9c0caf30 0x7efc9c1559e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:26.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.375+0000 7efca15f2640 1 -- 172.21.15.49:0/3111709203 >> 172.21.15.49:0/3111709203 conn(0x7efc9c0cc140 msgr2=0x7efc9c158a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:26.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.375+0000 7efca15f2640 1 -- 172.21.15.49:0/3111709203 shutdown_connections 2023-11-06T02:08:26.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.375+0000 7efca15f2640 1 -- 172.21.15.49:0/3111709203 wait complete. 2023-11-06T02:08:26.389 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T02:08:19.252222+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T02:08:24.776211+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T02:08:26.389 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T02:08:26.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.515+0000 7fe535dc6640 1 Processor -- start 2023-11-06T02:08:26.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.515+0000 7fe535dc6640 1 -- start start 2023-11-06T02:08:26.514 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.515+0000 7fe535dc6640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe5300d2a60 0x7fe5300d2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:26.517 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.515+0000 7fe535dc6640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fe5300b5d40 0x7fe5300b61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:26.517 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.515+0000 7fe535dc6640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5300d33e0 con 0x7fe53015cbb0 2023-11-06T02:08:26.517 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.515+0000 7fe535dc6640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe53015cf90 con 0x7fe5300b5d40 2023-11-06T02:08:26.518 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.515+0000 7fe535dc6640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5300d0b80 con 0x7fe5300d2a60 2023-11-06T02:08:26.518 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.515+0000 7fe52ffff640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fe5300b5d40 0x7fe5300b61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:26.518 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.515+0000 7fe52effd640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe5300d2a60 0x7fe5300d2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:26.518 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.515+0000 7fe52ffff640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fe5300b5d40 0x7fe5300b61d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.49:36842/0 (socket says 172.21.15.49:36842) 2023-11-06T02:08:26.519 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.515+0000 7fe52f7fe640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7fe53015cbb0 0x7fe5300d1fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.49:45790/0 (socket says 172.21.15.49:45790) 2023-11-06T02:08:26.519 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.515+0000 7fe52ffff640 1 -- 172.21.15.49:0/3133207353 learned_addr learned my addr 172.21.15.49:0/3133207353 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:26.523 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.515+0000 7fe52ffff640 1 -- 172.21.15.49:0/3133207353 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe5300d2a60 msgr2=0x7fe5300d2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:26.523 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.515+0000 7fe52ffff640 1 --2- 172.21.15.49:0/3133207353 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe5300d2a60 0x7fe5300d2ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:26.523 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.515+0000 7fe52ffff640 1 -- 172.21.15.49:0/3133207353 >> v1:172.21.15.150:6789/0 conn(0x7fe53015cbb0 legacy=0x7fe5300d1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:26.523 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.515+0000 7fe52ffff640 1 -- 172.21.15.49:0/3133207353 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5300d0e60 con 0x7fe5300b5d40 2023-11-06T02:08:26.523 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.519+0000 7fe52ffff640 1 --2- 172.21.15.49:0/3133207353 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe5300b5d40 0x7fe5300b61d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe528009e50 tx=0x7fe52801f1f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2edab30774d922d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:26.524 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.519+0000 7fe52e7fc640 1 -- 172.21.15.49:0/3133207353 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5280af400 con 0x7fe5300b5d40 2023-11-06T02:08:26.524 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.519+0000 7fe52e7fc640 1 -- 172.21.15.49:0/3133207353 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5280ab050 con 0x7fe5300b5d40 2023-11-06T02:08:26.524 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.519+0000 7fe52e7fc640 1 -- 172.21.15.49:0/3133207353 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5280af780 con 0x7fe5300b5d40 2023-11-06T02:08:26.524 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.519+0000 7fe535dc6640 1 -- 172.21.15.49:0/3133207353 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe5300b5d40 msgr2=0x7fe5300b61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:26.524 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.519+0000 7fe535dc6640 1 --2- 172.21.15.49:0/3133207353 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe5300b5d40 0x7fe5300b61d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe528009e50 tx=0x7fe52801f1f0 comp rx=0 tx=0).stop 2023-11-06T02:08:26.525 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.519+0000 7fe535dc6640 1 -- 172.21.15.49:0/3133207353 shutdown_connections 2023-11-06T02:08:26.525 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.519+0000 7fe535dc6640 1 --2- 172.21.15.49:0/3133207353 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe5300d2a60 0x7fe5300d2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:26.525 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.519+0000 7fe535dc6640 1 --2- 172.21.15.49:0/3133207353 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe5300b5d40 0x7fe5300b61d0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:26.525 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.519+0000 7fe535dc6640 1 -- 172.21.15.49:0/3133207353 >> 172.21.15.49:0/3133207353 conn(0x7fe5300718a0 msgr2=0x7fe530071cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:26.525 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.519+0000 7fe535dc6640 1 -- 172.21.15.49:0/3133207353 shutdown_connections 2023-11-06T02:08:26.525 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.519+0000 7fe535dc6640 1 -- 172.21.15.49:0/3133207353 wait complete. 2023-11-06T02:08:26.526 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.519+0000 7fe535dc6640 1 Processor -- start 2023-11-06T02:08:26.526 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.519+0000 7fe535dc6640 1 -- start start 2023-11-06T02:08:26.526 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.523+0000 7fe535dc6640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe5300b5d40 0x7fe53015ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:26.526 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.523+0000 7fe535dc6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5300d2a60 0x7fe5300a9e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:26.527 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.523+0000 7fe535dc6640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe53015cbb0 0x7fe5301577a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:26.527 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.523+0000 7fe535dc6640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5300aa360 con 0x7fe5300d2a60 2023-11-06T02:08:26.527 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.523+0000 7fe535dc6640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5300aa5c0 con 0x7fe53015cbb0 2023-11-06T02:08:26.527 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.523+0000 7fe535dc6640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5300aa8a0 con 0x7fe5300b5d40 2023-11-06T02:08:26.527 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.523+0000 7fe52effd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5300d2a60 0x7fe5300a9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:26.527 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.523+0000 7fe52effd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5300d2a60 0x7fe5300a9e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41436/0 (socket says 172.21.15.49:41436) 2023-11-06T02:08:26.528 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.523+0000 7fe52effd640 1 -- 172.21.15.49:0/1957128529 learned_addr learned my addr 172.21.15.49:0/1957128529 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:26.528 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.523+0000 7fe52f7fe640 1 --2- 172.21.15.49:0/1957128529 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe5300b5d40 0x7fe53015ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:26.528 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.523+0000 7fe52ffff640 1 --2- 172.21.15.49:0/1957128529 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe53015cbb0 0x7fe5301577a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:26.528 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.523+0000 7fe52effd640 1 -- 172.21.15.49:0/1957128529 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe5300b5d40 msgr2=0x7fe53015ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:26.528 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.523+0000 7fe52effd640 1 --2- 172.21.15.49:0/1957128529 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe5300b5d40 0x7fe53015ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:26.529 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.523+0000 7fe52effd640 1 -- 172.21.15.49:0/1957128529 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe53015cbb0 msgr2=0x7fe5301577a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:26.529 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.523+0000 7fe52effd640 1 --2- 172.21.15.49:0/1957128529 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe53015cbb0 0x7fe5301577a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:26.529 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.523+0000 7fe52effd640 1 -- 172.21.15.49:0/1957128529 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe528009b30 con 0x7fe5300d2a60 2023-11-06T02:08:26.529 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.523+0000 7fe52f7fe640 1 --2- 172.21.15.49:0/1957128529 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe5300b5d40 0x7fe53015ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:08:26.529 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.523+0000 7fe52effd640 1 --2- 172.21.15.49:0/1957128529 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5300d2a60 0x7fe5300a9e20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe52000e930 tx=0x7fe52000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:26.530 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.523+0000 7fe527fff640 1 -- 172.21.15.49:0/1957128529 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe520009690 con 0x7fe5300d2a60 2023-11-06T02:08:26.530 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.523+0000 7fe527fff640 1 -- 172.21.15.49:0/1957128529 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe52000ca00 con 0x7fe5300d2a60 2023-11-06T02:08:26.530 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.523+0000 7fe535dc6640 1 -- 172.21.15.49:0/1957128529 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5300aab80 con 0x7fe5300d2a60 2023-11-06T02:08:26.530 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.523+0000 7fe527fff640 1 -- 172.21.15.49:0/1957128529 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe52000cd70 con 0x7fe5300d2a60 2023-11-06T02:08:26.530 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.523+0000 7fe535dc6640 1 -- 172.21.15.49:0/1957128529 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5300abac0 con 0x7fe5300d2a60 2023-11-06T02:08:26.531 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.523+0000 7fe535dc6640 1 -- 172.21.15.49:0/1957128529 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe530074040 con 0x7fe5300d2a60 2023-11-06T02:08:26.532 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.523+0000 7fe527fff640 1 -- 172.21.15.49:0/1957128529 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fe520013470 con 0x7fe5300d2a60 2023-11-06T02:08:26.532 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.523+0000 7fe527fff640 1 --2- 172.21.15.49:0/1957128529 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe510065a90 0x7fe510067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:26.532 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.523+0000 7fe527fff640 1 -- 172.21.15.49:0/1957128529 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fe520083170 con 0x7fe5300d2a60 2023-11-06T02:08:26.532 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.527+0000 7fe52f7fe640 1 --2- 172.21.15.49:0/1957128529 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe510065a90 0x7fe510067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:26.533 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.527+0000 7fe52f7fe640 1 --2- 172.21.15.49:0/1957128529 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe510065a90 0x7fe510067f50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe518000f70 tx=0x7fe518007d50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:26.533 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.531+0000 7fe527fff640 1 -- 172.21.15.49:0/1957128529 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fe520050e90 con 0x7fe5300d2a60 2023-11-06T02:08:26.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.687+0000 7fe535dc6640 1 -- 172.21.15.49:0/1957128529 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe5300c8ef0 con 0x7fe5300d2a60 2023-11-06T02:08:26.686 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.687+0000 7fe527fff640 1 -- 172.21.15.49:0/1957128529 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v18) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe5200544b0 con 0x7fe5300d2a60 2023-11-06T02:08:26.687 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2023-11-06T02:08:26.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.691+0000 7fe535dc6640 1 -- 172.21.15.49:0/1957128529 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe510065a90 msgr2=0x7fe510067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:26.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.691+0000 7fe535dc6640 1 --2- 172.21.15.49:0/1957128529 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe510065a90 0x7fe510067f50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe518000f70 tx=0x7fe518007d50 comp rx=0 tx=0).stop 2023-11-06T02:08:26.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.691+0000 7fe535dc6640 1 -- 172.21.15.49:0/1957128529 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5300d2a60 msgr2=0x7fe5300a9e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:26.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.691+0000 7fe535dc6640 1 --2- 172.21.15.49:0/1957128529 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5300d2a60 0x7fe5300a9e20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe52000e930 tx=0x7fe52000edf0 comp rx=0 tx=0).stop 2023-11-06T02:08:26.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.691+0000 7fe535dc6640 1 -- 172.21.15.49:0/1957128529 shutdown_connections 2023-11-06T02:08:26.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.691+0000 7fe535dc6640 1 --2- 172.21.15.49:0/1957128529 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe510065a90 0x7fe510067f50 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:26.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.691+0000 7fe535dc6640 1 --2- 172.21.15.49:0/1957128529 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5300d2a60 0x7fe5300a9e20 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:26.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.691+0000 7fe535dc6640 1 --2- 172.21.15.49:0/1957128529 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe53015cbb0 0x7fe5301577a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:26.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.691+0000 7fe535dc6640 1 --2- 172.21.15.49:0/1957128529 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe5300b5d40 0x7fe53015ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:26.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.691+0000 7fe535dc6640 1 -- 172.21.15.49:0/1957128529 >> 172.21.15.49:0/1957128529 conn(0x7fe5300718a0 msgr2=0x7fe5300cbd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:26.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.691+0000 7fe535dc6640 1 -- 172.21.15.49:0/1957128529 shutdown_connections 2023-11-06T02:08:26.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.691+0000 7fe535dc6640 1 -- 172.21.15.49:0/1957128529 wait complete. 2023-11-06T02:08:26.706 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T02:08:26.831 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.831+0000 7f0dc075e640 1 Processor -- start 2023-11-06T02:08:26.831 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.831+0000 7f0dc075e640 1 -- start start 2023-11-06T02:08:26.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.835+0000 7f0dc075e640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f0db814ffc0 0x7f0db80ada30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:26.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.835+0000 7f0dc075e640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f0db8070870 0x7f0db806c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:26.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.835+0000 7f0dc075e640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0db80c8c80 con 0x7f0db80adf70 2023-11-06T02:08:26.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.835+0000 7f0dc075e640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0db80ae4d0 con 0x7f0db814ffc0 2023-11-06T02:08:26.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.835+0000 7f0dc075e640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0db81594f0 con 0x7f0db8070870 2023-11-06T02:08:26.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.835+0000 7f0dbe4fa640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f0db814ffc0 0x7f0db80ada30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:26.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.835+0000 7f0dbe4fa640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f0db814ffc0 0x7f0db80ada30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41450/0 (socket says 172.21.15.49:41450) 2023-11-06T02:08:26.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.835+0000 7f0dbe4fa640 1 -- 172.21.15.49:0/758196160 learned_addr learned my addr 172.21.15.49:0/758196160 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:26.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.835+0000 7f0dbecfb640 1 --2- 172.21.15.49:0/758196160 >> v2:172.21.15.150:3300/0 conn(0x7f0db8070870 0x7f0db806c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:26.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.835+0000 7f0dbe4fa640 1 -- 172.21.15.49:0/758196160 >> v2:172.21.15.150:3300/0 conn(0x7f0db8070870 msgr2=0x7f0db806c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:26.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.835+0000 7f0dbe4fa640 1 --2- 172.21.15.49:0/758196160 >> v2:172.21.15.150:3300/0 conn(0x7f0db8070870 0x7f0db806c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:26.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.835+0000 7f0dbe4fa640 1 -- 172.21.15.49:0/758196160 >> v1:172.21.15.49:6789/0 conn(0x7f0db80adf70 legacy=0x7f0db8070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:26.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.835+0000 7f0dbe4fa640 1 -- 172.21.15.49:0/758196160 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0db8159700 con 0x7f0db814ffc0 2023-11-06T02:08:26.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.835+0000 7f0dbe4fa640 1 --2- 172.21.15.49:0/758196160 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0db814ffc0 0x7f0db80ada30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0db80ad7d0 tx=0x7f0db401c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b60d2c975dc75ab server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:26.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.835+0000 7f0dbd4f8640 1 -- 172.21.15.49:0/758196160 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db40ab5f0 con 0x7f0db814ffc0 2023-11-06T02:08:26.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.835+0000 7f0dbd4f8640 1 -- 172.21.15.49:0/758196160 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0db40ab750 con 0x7f0db814ffc0 2023-11-06T02:08:26.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.835+0000 7f0dbd4f8640 1 -- 172.21.15.49:0/758196160 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db40abae0 con 0x7f0db814ffc0 2023-11-06T02:08:26.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.835+0000 7f0dc075e640 1 -- 172.21.15.49:0/758196160 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0db814ffc0 msgr2=0x7f0db80ada30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:26.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.835+0000 7f0dc075e640 1 --2- 172.21.15.49:0/758196160 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0db814ffc0 0x7f0db80ada30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0db80ad7d0 tx=0x7f0db401c500 comp rx=0 tx=0).stop 2023-11-06T02:08:26.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.835+0000 7f0dc075e640 1 -- 172.21.15.49:0/758196160 shutdown_connections 2023-11-06T02:08:26.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.835+0000 7f0dc075e640 1 --2- 172.21.15.49:0/758196160 >> v2:172.21.15.150:3300/0 conn(0x7f0db8070870 0x7f0db806c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:26.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.835+0000 7f0dc075e640 1 --2- 172.21.15.49:0/758196160 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0db814ffc0 0x7f0db80ada30 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:26.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.835+0000 7f0dc075e640 1 -- 172.21.15.49:0/758196160 >> 172.21.15.49:0/758196160 conn(0x7f0db806a370 msgr2=0x7f0db80ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:26.839 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.835+0000 7f0dc075e640 1 -- 172.21.15.49:0/758196160 shutdown_connections 2023-11-06T02:08:26.839 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.835+0000 7f0dc075e640 1 -- 172.21.15.49:0/758196160 wait complete. 2023-11-06T02:08:26.841 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.839+0000 7f0dc075e640 1 Processor -- start 2023-11-06T02:08:26.842 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.839+0000 7f0dc075e640 1 -- start start 2023-11-06T02:08:26.842 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.839+0000 7f0dc075e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0db8070870 0x7f0db80c8340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:26.842 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.839+0000 7f0dc075e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0db80adf70 0x7f0db80c35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:26.842 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.839+0000 7f0dc075e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0db814ffc0 0x7f0db8146b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:26.843 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.839+0000 7f0dc075e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0db814deb0 con 0x7f0db8070870 2023-11-06T02:08:26.843 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.839+0000 7f0dc075e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0db8147050 con 0x7f0db80adf70 2023-11-06T02:08:26.843 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.839+0000 7f0dc075e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0db814d740 con 0x7f0db814ffc0 2023-11-06T02:08:26.843 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.839+0000 7f0dbe4fa640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0db8070870 0x7f0db80c8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:26.843 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.839+0000 7f0dbe4fa640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0db8070870 0x7f0db80c8340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41462/0 (socket says 172.21.15.49:41462) 2023-11-06T02:08:26.843 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.839+0000 7f0dbe4fa640 1 -- 172.21.15.49:0/2165160974 learned_addr learned my addr 172.21.15.49:0/2165160974 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:26.844 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.839+0000 7f0dbdcf9640 1 --2- 172.21.15.49:0/2165160974 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0db80adf70 0x7f0db80c35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:26.844 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.839+0000 7f0dbecfb640 1 --2- 172.21.15.49:0/2165160974 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0db814ffc0 0x7f0db8146b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:26.844 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.839+0000 7f0dbe4fa640 1 -- 172.21.15.49:0/2165160974 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0db814ffc0 msgr2=0x7f0db8146b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:26.844 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.839+0000 7f0dbe4fa640 1 --2- 172.21.15.49:0/2165160974 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0db814ffc0 0x7f0db8146b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:26.844 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.839+0000 7f0dbe4fa640 1 -- 172.21.15.49:0/2165160974 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0db80adf70 msgr2=0x7f0db80c35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:26.845 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.839+0000 7f0dbe4fa640 1 --2- 172.21.15.49:0/2165160974 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0db80adf70 0x7f0db80c35e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:26.845 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.839+0000 7f0dbe4fa640 1 -- 172.21.15.49:0/2165160974 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0db40094d0 con 0x7f0db8070870 2023-11-06T02:08:26.845 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.839+0000 7f0dbe4fa640 1 --2- 172.21.15.49:0/2165160974 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0db8070870 0x7f0db80c8340 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0db40ad040 tx=0x7f0db401ad40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:26.845 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.839+0000 7f0daeffd640 1 -- 172.21.15.49:0/2165160974 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db40be5f0 con 0x7f0db8070870 2023-11-06T02:08:26.845 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.839+0000 7f0daeffd640 1 -- 172.21.15.49:0/2165160974 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0db40be750 con 0x7f0db8070870 2023-11-06T02:08:26.846 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.839+0000 7f0dc075e640 1 -- 172.21.15.49:0/2165160974 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0db814d9e0 con 0x7f0db8070870 2023-11-06T02:08:26.846 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.839+0000 7f0daeffd640 1 -- 172.21.15.49:0/2165160974 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db40beae0 con 0x7f0db8070870 2023-11-06T02:08:26.846 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.839+0000 7f0dc075e640 1 -- 172.21.15.49:0/2165160974 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0db814e3e0 con 0x7f0db8070870 2023-11-06T02:08:26.846 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.843+0000 7f0dc075e640 1 -- 172.21.15.49:0/2165160974 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0db8074040 con 0x7f0db8070870 2023-11-06T02:08:26.846 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.843+0000 7f0daeffd640 1 -- 172.21.15.49:0/2165160974 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f0db40b43f0 con 0x7f0db8070870 2023-11-06T02:08:26.847 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.843+0000 7f0daeffd640 1 --2- 172.21.15.49:0/2165160974 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0d9c065a60 0x7f0d9c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:26.847 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.843+0000 7f0daeffd640 1 -- 172.21.15.49:0/2165160974 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f0db40a8050 con 0x7f0db8070870 2023-11-06T02:08:26.847 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.843+0000 7f0dbdcf9640 1 --2- 172.21.15.49:0/2165160974 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0d9c065a60 0x7f0d9c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:26.847 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.843+0000 7f0dbdcf9640 1 --2- 172.21.15.49:0/2165160974 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0d9c065a60 0x7f0d9c067f20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0db8149fa0 tx=0x7f0da8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:26.848 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.847+0000 7f0daeffd640 1 -- 172.21.15.49:0/2165160974 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f0db40eb7c0 con 0x7f0db8070870 2023-11-06T02:08:26.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.999+0000 7f0dc075e640 1 -- 172.21.15.49:0/2165160974 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f0db806d8d0 con 0x7f0db8070870 2023-11-06T02:08:26.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.999+0000 7f0daeffd640 1 -- 172.21.15.49:0/2165160974 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v18) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0db40eede0 con 0x7f0db8070870 2023-11-06T02:08:26.997 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 8 2023-11-06T02:08:26.999 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.999+0000 7f0dc075e640 1 -- 172.21.15.49:0/2165160974 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0d9c065a60 msgr2=0x7f0d9c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:26.999 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.999+0000 7f0dc075e640 1 --2- 172.21.15.49:0/2165160974 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0d9c065a60 0x7f0d9c067f20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0db8149fa0 tx=0x7f0da8009040 comp rx=0 tx=0).stop 2023-11-06T02:08:26.999 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.999+0000 7f0dc075e640 1 -- 172.21.15.49:0/2165160974 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0db8070870 msgr2=0x7f0db80c8340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:26.999 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.999+0000 7f0dc075e640 1 --2- 172.21.15.49:0/2165160974 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0db8070870 0x7f0db80c8340 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0db40ad040 tx=0x7f0db401ad40 comp rx=0 tx=0).stop 2023-11-06T02:08:26.999 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.999+0000 7f0dc075e640 1 -- 172.21.15.49:0/2165160974 shutdown_connections 2023-11-06T02:08:27.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.999+0000 7f0dc075e640 1 --2- 172.21.15.49:0/2165160974 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0d9c065a60 0x7f0d9c067f20 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:27.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.999+0000 7f0dc075e640 1 --2- 172.21.15.49:0/2165160974 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0db814ffc0 0x7f0db8146b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:27.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.999+0000 7f0dc075e640 1 --2- 172.21.15.49:0/2165160974 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0db80adf70 0x7f0db80c35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:27.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.999+0000 7f0dc075e640 1 --2- 172.21.15.49:0/2165160974 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0db8070870 0x7f0db80c8340 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:27.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.999+0000 7f0dc075e640 1 -- 172.21.15.49:0/2165160974 >> 172.21.15.49:0/2165160974 conn(0x7f0db806a370 msgr2=0x7f0db80c8e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:27.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.999+0000 7f0dc075e640 1 -- 172.21.15.49:0/2165160974 shutdown_connections 2023-11-06T02:08:27.001 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:26.999+0000 7f0dc075e640 1 -- 172.21.15.49:0/2165160974 wait complete. 2023-11-06T02:08:27.014 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T02:08:27.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.127+0000 7f3ea430c640 1 Processor -- start 2023-11-06T02:08:27.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.127+0000 7f3ea430c640 1 -- start start 2023-11-06T02:08:27.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.127+0000 7f3ea430c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f3e9c0c9690 0x7f3e9c0b04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:27.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.127+0000 7f3ea430c640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e9c0b0a20 con 0x7f3e9c15cc50 2023-11-06T02:08:27.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.127+0000 7f3ea430c640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e9c15d030 con 0x7f3e9c0b0080 2023-11-06T02:08:27.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.127+0000 7f3ea430c640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e9c0cc9c0 con 0x7f3e9c0c9690 2023-11-06T02:08:27.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.127+0000 7f3ea28a9640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f3e9c0c9690 0x7f3e9c0b04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:27.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.131+0000 7f3ea28a9640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f3e9c0c9690 0x7f3e9c0b04e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41472/0 (socket says 172.21.15.49:41472) 2023-11-06T02:08:27.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.131+0000 7f3ea28a9640 1 -- 172.21.15.49:0/4193872127 learned_addr learned my addr 172.21.15.49:0/4193872127 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:27.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.131+0000 7f3ea28a9640 1 -- 172.21.15.49:0/4193872127 >> v1:172.21.15.150:6789/0 conn(0x7f3e9c0b0080 legacy=0x7f3e9c0c8f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:27.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.131+0000 7f3ea28a9640 1 -- 172.21.15.49:0/4193872127 >> v1:172.21.15.49:6789/0 conn(0x7f3e9c15cc50 legacy=0x7f3e9c0701e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:27.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.131+0000 7f3ea28a9640 1 -- 172.21.15.49:0/4193872127 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e9c0ccc60 con 0x7f3e9c0c9690 2023-11-06T02:08:27.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.131+0000 7f3ea28a9640 1 --2- 172.21.15.49:0/4193872127 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3e9c0c9690 0x7f3e9c0b04e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3e94009e50 tx=0x7f3e9401c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e0d480f7bb82a0d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:27.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.131+0000 7f3ea10a6640 1 -- 172.21.15.49:0/4193872127 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e940a5810 con 0x7f3e9c0c9690 2023-11-06T02:08:27.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.131+0000 7f3ea10a6640 1 -- 172.21.15.49:0/4193872127 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e940a5970 con 0x7f3e9c0c9690 2023-11-06T02:08:27.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.131+0000 7f3ea10a6640 1 -- 172.21.15.49:0/4193872127 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e940a5d00 con 0x7f3e9c0c9690 2023-11-06T02:08:27.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.131+0000 7f3ea430c640 1 -- 172.21.15.49:0/4193872127 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3e9c0c9690 msgr2=0x7f3e9c0b04e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:27.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.131+0000 7f3ea430c640 1 --2- 172.21.15.49:0/4193872127 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3e9c0c9690 0x7f3e9c0b04e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3e94009e50 tx=0x7f3e9401c950 comp rx=0 tx=0).stop 2023-11-06T02:08:27.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.131+0000 7f3ea430c640 1 -- 172.21.15.49:0/4193872127 shutdown_connections 2023-11-06T02:08:27.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.131+0000 7f3ea430c640 1 --2- 172.21.15.49:0/4193872127 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3e9c0c9690 0x7f3e9c0b04e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:27.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.131+0000 7f3ea430c640 1 -- 172.21.15.49:0/4193872127 >> 172.21.15.49:0/4193872127 conn(0x7f3e9c06c0a0 msgr2=0x7f3e9c06c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:27.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.131+0000 7f3ea430c640 1 -- 172.21.15.49:0/4193872127 shutdown_connections 2023-11-06T02:08:27.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.131+0000 7f3ea430c640 1 -- 172.21.15.49:0/4193872127 wait complete. 2023-11-06T02:08:27.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.135+0000 7f3ea430c640 1 Processor -- start 2023-11-06T02:08:27.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.135+0000 7f3ea430c640 1 -- start start 2023-11-06T02:08:27.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.135+0000 7f3ea430c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e9c0b0080 0x7f3e9c14ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:27.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.135+0000 7f3ea430c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e9c0c9690 0x7f3e9c0ab8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:27.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.135+0000 7f3ea430c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3e9c15cc50 0x7f3e9c0abe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:27.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.135+0000 7f3ea430c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9c14d0a0 con 0x7f3e9c15cc50 2023-11-06T02:08:27.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.135+0000 7f3ea430c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9c1534f0 con 0x7f3e9c0b0080 2023-11-06T02:08:27.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.135+0000 7f3ea430c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9c15a380 con 0x7f3e9c0c9690 2023-11-06T02:08:27.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.135+0000 7f3ea28a9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3e9c15cc50 0x7f3e9c0abe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:27.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.135+0000 7f3ea28a9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3e9c15cc50 0x7f3e9c0abe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41478/0 (socket says 172.21.15.49:41478) 2023-11-06T02:08:27.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.135+0000 7f3ea28a9640 1 -- 172.21.15.49:0/1073287304 learned_addr learned my addr 172.21.15.49:0/1073287304 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:27.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.135+0000 7f3ea18a7640 1 --2- 172.21.15.49:0/1073287304 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e9c0c9690 0x7f3e9c0ab8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:27.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.135+0000 7f3ea20a8640 1 --2- 172.21.15.49:0/1073287304 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e9c0b0080 0x7f3e9c14ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:27.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.135+0000 7f3ea28a9640 1 -- 172.21.15.49:0/1073287304 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e9c0c9690 msgr2=0x7f3e9c0ab8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:27.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.135+0000 7f3ea28a9640 1 --2- 172.21.15.49:0/1073287304 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e9c0c9690 0x7f3e9c0ab8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:27.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.135+0000 7f3ea28a9640 1 -- 172.21.15.49:0/1073287304 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e9c0b0080 msgr2=0x7f3e9c14ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:27.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.135+0000 7f3ea28a9640 1 --2- 172.21.15.49:0/1073287304 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e9c0b0080 0x7f3e9c14ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:27.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.135+0000 7f3ea28a9640 1 -- 172.21.15.49:0/1073287304 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e94009b30 con 0x7f3e9c15cc50 2023-11-06T02:08:27.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.135+0000 7f3ea18a7640 1 --2- 172.21.15.49:0/1073287304 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e9c0c9690 0x7f3e9c0ab8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:08:27.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.135+0000 7f3ea28a9640 1 --2- 172.21.15.49:0/1073287304 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3e9c15cc50 0x7f3e9c0abe00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3e94019560 tx=0x7f3e940196f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:27.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.135+0000 7f3e92ffd640 1 -- 172.21.15.49:0/1073287304 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e940b8670 con 0x7f3e9c15cc50 2023-11-06T02:08:27.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.135+0000 7f3e92ffd640 1 -- 172.21.15.49:0/1073287304 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e940b87d0 con 0x7f3e9c15cc50 2023-11-06T02:08:27.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.135+0000 7f3ea430c640 1 -- 172.21.15.49:0/1073287304 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e9c15a660 con 0x7f3e9c15cc50 2023-11-06T02:08:27.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.135+0000 7f3e92ffd640 1 -- 172.21.15.49:0/1073287304 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e940b8b80 con 0x7f3e9c15cc50 2023-11-06T02:08:27.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.135+0000 7f3ea430c640 1 -- 172.21.15.49:0/1073287304 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e9c15ac20 con 0x7f3e9c15cc50 2023-11-06T02:08:27.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.139+0000 7f3ea430c640 1 -- 172.21.15.49:0/1073287304 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e9c074040 con 0x7f3e9c15cc50 2023-11-06T02:08:27.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.139+0000 7f3e92ffd640 1 -- 172.21.15.49:0/1073287304 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f3e940b8ce0 con 0x7f3e9c15cc50 2023-11-06T02:08:27.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.139+0000 7f3e92ffd640 1 --2- 172.21.15.49:0/1073287304 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3e80065a60 0x7f3e80067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:27.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.139+0000 7f3e92ffd640 1 -- 172.21.15.49:0/1073287304 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f3e9411e630 con 0x7f3e9c15cc50 2023-11-06T02:08:27.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.139+0000 7f3ea20a8640 1 --2- 172.21.15.49:0/1073287304 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3e80065a60 0x7f3e80067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:27.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.139+0000 7f3ea20a8640 1 --2- 172.21.15.49:0/1073287304 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3e80065a60 0x7f3e80067f20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c161690 tx=0x7f3e98009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:27.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.143+0000 7f3e92ffd640 1 -- 172.21.15.49:0/1073287304 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f3e940ec330 con 0x7f3e9c15cc50 2023-11-06T02:08:27.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.335+0000 7f3ea430c640 1 -- 172.21.15.49:0/1073287304 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3e9c0b0520 con 0x7f3e9c15cc50 2023-11-06T02:08:27.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.339+0000 7f3e92ffd640 1 -- 172.21.15.49:0/1073287304 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3530 (secure 0 0 0) 0x7f3e940ef950 con 0x7f3e9c15cc50 2023-11-06T02:08:27.337 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:08:27.337 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6834/1377586971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1377586971},{"type":"v1","addr":"172.21.15.49:6834","nonce":1377586971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4288,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2620584959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2620584959},{"type":"v1","addr":"172.21.15.150:6835","nonce":2620584959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4291,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/4272191418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":4272191418},{"type":"v1","addr":"172.21.15.150:6837","nonce":4272191418}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4293,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/4280448297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6833","nonce":4280448297},{"type":"v1","addr":"172.21.15.49:6835","nonce":4280448297}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-11-06T02:08:27.338 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 3 2023-11-06T02:08:27.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.339+0000 7f3ea430c640 1 -- 172.21.15.49:0/1073287304 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3e80065a60 msgr2=0x7f3e80067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:27.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.339+0000 7f3ea430c640 1 --2- 172.21.15.49:0/1073287304 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3e80065a60 0x7f3e80067f20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c161690 tx=0x7f3e98009040 comp rx=0 tx=0).stop 2023-11-06T02:08:27.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.339+0000 7f3ea430c640 1 -- 172.21.15.49:0/1073287304 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3e9c15cc50 msgr2=0x7f3e9c0abe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:27.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.339+0000 7f3ea430c640 1 --2- 172.21.15.49:0/1073287304 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3e9c15cc50 0x7f3e9c0abe00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3e94019560 tx=0x7f3e940196f0 comp rx=0 tx=0).stop 2023-11-06T02:08:27.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.339+0000 7f3ea430c640 1 -- 172.21.15.49:0/1073287304 shutdown_connections 2023-11-06T02:08:27.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.339+0000 7f3ea430c640 1 --2- 172.21.15.49:0/1073287304 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3e80065a60 0x7f3e80067f20 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:27.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.339+0000 7f3ea430c640 1 --2- 172.21.15.49:0/1073287304 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3e9c15cc50 0x7f3e9c0abe00 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:27.344 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.339+0000 7f3ea430c640 1 --2- 172.21.15.49:0/1073287304 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e9c0c9690 0x7f3e9c0ab8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:27.344 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.339+0000 7f3ea430c640 1 --2- 172.21.15.49:0/1073287304 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e9c0b0080 0x7f3e9c14ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:27.344 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.339+0000 7f3ea430c640 1 -- 172.21.15.49:0/1073287304 >> 172.21.15.49:0/1073287304 conn(0x7f3e9c06c0a0 msgr2=0x7f3e9c14ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:27.344 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.339+0000 7f3ea430c640 1 -- 172.21.15.49:0/1073287304 shutdown_connections 2023-11-06T02:08:27.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.339+0000 7f3ea430c640 1 -- 172.21.15.49:0/1073287304 wait complete. 2023-11-06T02:08:27.356 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T02:08:27.477 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.479+0000 7f0356d70640 1 Processor -- start 2023-11-06T02:08:27.477 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.479+0000 7f0356d70640 1 -- start start 2023-11-06T02:08:27.479 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.479+0000 7f0356d70640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f035015ceb0 0x7f03500cb970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:27.479 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.479+0000 7f0356d70640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f03500caf30 0x7f0350071120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:27.479 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.479+0000 7f0356d70640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f03500cb370 0x7f0350071660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:27.480 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.479+0000 7f0356d70640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03500cbeb0 con 0x7f03500caf30 2023-11-06T02:08:27.480 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.479+0000 7f0356d70640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03500caaa0 con 0x7f035015ceb0 2023-11-06T02:08:27.480 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.479+0000 7f0356d70640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03500ca1e0 con 0x7f03500cb370 2023-11-06T02:08:27.480 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.479+0000 7f034ffff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f03500caf30 0x7f0350071120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:27.481 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.479+0000 7f034ffff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f03500caf30 0x7f0350071120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41480/0 (socket says 172.21.15.49:41480) 2023-11-06T02:08:27.481 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.479+0000 7f034ffff640 1 -- 172.21.15.49:0/3357625801 learned_addr learned my addr 172.21.15.49:0/3357625801 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:27.481 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.479+0000 7f0354b0c640 1 --2- 172.21.15.49:0/3357625801 >> v2:172.21.15.150:3300/0 conn(0x7f035015ceb0 0x7f03500cb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:27.481 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.479+0000 7f035530d640 1 --2- 172.21.15.49:0/3357625801 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f03500cb370 0x7f0350071660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:27.481 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.479+0000 7f034ffff640 1 -- 172.21.15.49:0/3357625801 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f03500cb370 msgr2=0x7f0350071660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:27.482 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.479+0000 7f034ffff640 1 --2- 172.21.15.49:0/3357625801 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f03500cb370 0x7f0350071660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:27.482 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.479+0000 7f034ffff640 1 -- 172.21.15.49:0/3357625801 >> v2:172.21.15.150:3300/0 conn(0x7f035015ceb0 msgr2=0x7f03500cb970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:27.482 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.479+0000 7f034ffff640 1 --2- 172.21.15.49:0/3357625801 >> v2:172.21.15.150:3300/0 conn(0x7f035015ceb0 0x7f03500cb970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:27.482 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.479+0000 7f034ffff640 1 -- 172.21.15.49:0/3357625801 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03500ce490 con 0x7f03500caf30 2023-11-06T02:08:27.486 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.483+0000 7f034ffff640 1 --2- 172.21.15.49:0/3357625801 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03500caf30 0x7f0350071120 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0350068490 tx=0x7f034401c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d594d77067484397 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:27.486 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.483+0000 7f034f7fe640 1 -- 172.21.15.49:0/3357625801 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03440ac410 con 0x7f03500caf30 2023-11-06T02:08:27.487 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.483+0000 7f034f7fe640 1 -- 172.21.15.49:0/3357625801 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03440ac570 con 0x7f03500caf30 2023-11-06T02:08:27.487 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.483+0000 7f034f7fe640 1 -- 172.21.15.49:0/3357625801 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03440ac900 con 0x7f03500caf30 2023-11-06T02:08:27.487 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.483+0000 7f0356d70640 1 -- 172.21.15.49:0/3357625801 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03500caf30 msgr2=0x7f0350071120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:27.487 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.483+0000 7f0356d70640 1 --2- 172.21.15.49:0/3357625801 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03500caf30 0x7f0350071120 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0350068490 tx=0x7f034401c500 comp rx=0 tx=0).stop 2023-11-06T02:08:27.487 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.483+0000 7f0356d70640 1 -- 172.21.15.49:0/3357625801 shutdown_connections 2023-11-06T02:08:27.488 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.483+0000 7f0356d70640 1 --2- 172.21.15.49:0/3357625801 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f03500cb370 0x7f0350071660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:27.488 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.483+0000 7f0356d70640 1 --2- 172.21.15.49:0/3357625801 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03500caf30 0x7f0350071120 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:27.488 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.483+0000 7f0356d70640 1 --2- 172.21.15.49:0/3357625801 >> v2:172.21.15.150:3300/0 conn(0x7f035015ceb0 0x7f03500cb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:27.488 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.483+0000 7f0356d70640 1 -- 172.21.15.49:0/3357625801 >> 172.21.15.49:0/3357625801 conn(0x7f03500cc140 msgr2=0x7f03500cc550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:27.488 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.483+0000 7f0356d70640 1 -- 172.21.15.49:0/3357625801 shutdown_connections 2023-11-06T02:08:27.489 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.483+0000 7f0356d70640 1 -- 172.21.15.49:0/3357625801 wait complete. 2023-11-06T02:08:27.489 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.483+0000 7f0356d70640 1 Processor -- start 2023-11-06T02:08:27.489 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.483+0000 7f0356d70640 1 -- start start 2023-11-06T02:08:27.489 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.483+0000 7f0356d70640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03500caf30 0x7f03500c5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:27.490 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.483+0000 7f0356d70640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f035015ceb0 0x7f0350150a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:27.490 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.483+0000 7f0356d70640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0350150f70 0x7f03500c7560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:27.490 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.483+0000 7f0356d70640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0350151420 con 0x7f03500caf30 2023-11-06T02:08:27.490 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.483+0000 7f0356d70640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03500c3160 con 0x7f035015ceb0 2023-11-06T02:08:27.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.483+0000 7f0356d70640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03500c9750 con 0x7f0350150f70 2023-11-06T02:08:27.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.483+0000 7f0354b0c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03500caf30 0x7f03500c5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:27.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.483+0000 7f0354b0c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03500caf30 0x7f03500c5680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41496/0 (socket says 172.21.15.49:41496) 2023-11-06T02:08:27.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.483+0000 7f0354b0c640 1 -- 172.21.15.49:0/582146935 learned_addr learned my addr 172.21.15.49:0/582146935 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:27.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.483+0000 7f034ffff640 1 --2- 172.21.15.49:0/582146935 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f035015ceb0 0x7f0350150a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:27.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.483+0000 7f035530d640 1 --2- 172.21.15.49:0/582146935 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0350150f70 0x7f03500c7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:27.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.483+0000 7f0354b0c640 1 -- 172.21.15.49:0/582146935 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0350150f70 msgr2=0x7f03500c7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:27.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.483+0000 7f0354b0c640 1 --2- 172.21.15.49:0/582146935 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0350150f70 0x7f03500c7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:27.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.483+0000 7f0354b0c640 1 -- 172.21.15.49:0/582146935 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f035015ceb0 msgr2=0x7f0350150a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:27.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.483+0000 7f0354b0c640 1 --2- 172.21.15.49:0/582146935 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f035015ceb0 0x7f0350150a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:27.493 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.483+0000 7f0354b0c640 1 -- 172.21.15.49:0/582146935 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03440094d0 con 0x7f03500caf30 2023-11-06T02:08:27.493 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.487+0000 7f0354b0c640 1 --2- 172.21.15.49:0/582146935 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03500caf30 0x7f03500c5680 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f034000efc0 tx=0x7f034000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:27.493 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.487+0000 7f034d7fa640 1 -- 172.21.15.49:0/582146935 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0340009b80 con 0x7f03500caf30 2023-11-06T02:08:27.493 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.487+0000 7f034d7fa640 1 -- 172.21.15.49:0/582146935 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f034000f040 con 0x7f03500caf30 2023-11-06T02:08:27.494 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.487+0000 7f0356d70640 1 -- 172.21.15.49:0/582146935 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03500c8200 con 0x7f03500caf30 2023-11-06T02:08:27.494 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.487+0000 7f034d7fa640 1 -- 172.21.15.49:0/582146935 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03400146a0 con 0x7f03500caf30 2023-11-06T02:08:27.494 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.487+0000 7f0356d70640 1 -- 172.21.15.49:0/582146935 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03500c87c0 con 0x7f03500caf30 2023-11-06T02:08:27.495 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.487+0000 7f0356d70640 1 -- 172.21.15.49:0/582146935 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0350067140 con 0x7f03500caf30 2023-11-06T02:08:27.495 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.487+0000 7f034d7fa640 1 -- 172.21.15.49:0/582146935 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f0340014800 con 0x7f03500caf30 2023-11-06T02:08:27.495 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.487+0000 7f034d7fa640 1 --2- 172.21.15.49:0/582146935 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f033c065a60 0x7f033c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:27.496 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.487+0000 7f034d7fa640 1 -- 172.21.15.49:0/582146935 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f03400839a0 con 0x7f03500caf30 2023-11-06T02:08:27.496 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.487+0000 7f034ffff640 1 --2- 172.21.15.49:0/582146935 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f033c065a60 0x7f033c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:27.497 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.487+0000 7f034ffff640 1 --2- 172.21.15.49:0/582146935 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f033c065a60 0x7f033c067f20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f03440ae000 tx=0x7f0344009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:27.497 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.491+0000 7f034d7fa640 1 -- 172.21.15.49:0/582146935 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f03400515e0 con 0x7f03500caf30 2023-11-06T02:08:27.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.639+0000 7f0356d70640 1 -- 172.21.15.49:0/582146935 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0350071e40 con 0x7f03500caf30 2023-11-06T02:08:27.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.643+0000 7f034d7fa640 1 -- 172.21.15.49:0/582146935 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+14168 (secure 0 0 0) 0x7f0340054c00 con 0x7f03500caf30 2023-11-06T02:08:27.641 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:08:27.641 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":19,"fsid":"1a8b035c-6ba4-499a-b876-aea47fd9c0d4","created":"2023-11-06T02:08:06.440857+0000","modified":"2023-11-06T02:08:27.051411+0000","last_up_change":"2023-11-06T02:08:17.982372+0000","last_in_change":"2023-11-06T02:08:08.906631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T02:08:19.252222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T02:08:24.776211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ad55af8-0571-4e6a-8722-b5d429b7496d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6809","nonce":38116},{"type":"v1","addr":"172.21.15.49:6811","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6813","nonce":38116},{"type":"v1","addr":"172.21.15.49:6815","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6821","nonce":38116},{"type":"v1","addr":"172.21.15.49:6823","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6817","nonce":38116},{"type":"v1","addr":"172.21.15.49:6819","nonce":38116}]},"public_addr":"172.21.15.49:6811/38116","cluster_addr":"172.21.15.49:6815/38116","heartbeat_back_addr":"172.21.15.49:6823/38116","heartbeat_front_addr":"172.21.15.49:6819/38116","state":["exists","up"]},{"osd":1,"uuid":"c8b31f85-a757-43ed-9419-351acc46611b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":38113},{"type":"v1","addr":"172.21.15.49:6810","nonce":38113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":38113},{"type":"v1","addr":"172.21.15.49:6814","nonce":38113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":38113},{"type":"v1","addr":"172.21.15.49:6822","nonce":38113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":38113},{"type":"v1","addr":"172.21.15.49:6818","nonce":38113}]},"public_addr":"172.21.15.49:6810/38113","cluster_addr":"172.21.15.49:6814/38113","heartbeat_back_addr":"172.21.15.49:6822/38113","heartbeat_front_addr":"172.21.15.49:6818/38113","state":["exists","up"]},{"osd":2,"uuid":"555b3bf3-0a70-4b91-8efd-e4204b5fc26e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":38114},{"type":"v1","addr":"172.21.15.49:6801","nonce":38114}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":38114},{"type":"v1","addr":"172.21.15.49:6803","nonce":38114}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":38114},{"type":"v1","addr":"172.21.15.49:6807","nonce":38114}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":38114},{"type":"v1","addr":"172.21.15.49:6805","nonce":38114}]},"public_addr":"172.21.15.49:6801/38114","cluster_addr":"172.21.15.49:6803/38114","heartbeat_back_addr":"172.21.15.49:6807/38114","heartbeat_front_addr":"172.21.15.49:6805/38114","state":["exists","up"]},{"osd":3,"uuid":"8d4e5d77-03ef-4b9b-bfcb-5ae93a46795c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":38115},{"type":"v1","addr":"172.21.15.49:6825","nonce":38115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":38115},{"type":"v1","addr":"172.21.15.49:6827","nonce":38115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":38115},{"type":"v1","addr":"172.21.15.49:6831","nonce":38115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":38115},{"type":"v1","addr":"172.21.15.49:6829","nonce":38115}]},"public_addr":"172.21.15.49:6825/38115","cluster_addr":"172.21.15.49:6827/38115","heartbeat_back_addr":"172.21.15.49:6831/38115","heartbeat_front_addr":"172.21.15.49:6829/38115","state":["exists","up"]},{"osd":4,"uuid":"dba03ad4-68a3-4cf3-9218-ac611f02eacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":38417},{"type":"v1","addr":"172.21.15.150:6809","nonce":38417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":38417},{"type":"v1","addr":"172.21.15.150:6811","nonce":38417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":38417},{"type":"v1","addr":"172.21.15.150:6815","nonce":38417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":38417},{"type":"v1","addr":"172.21.15.150:6813","nonce":38417}]},"public_addr":"172.21.15.150:6809/38417","cluster_addr":"172.21.15.150:6811/38417","heartbeat_back_addr":"172.21.15.150:6815/38417","heartbeat_front_addr":"172.21.15.150:6813/38417","state":["exists","up"]},{"osd":5,"uuid":"0df9c303-c255-4640-8e91-01f1c3efe18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":38418},{"type":"v1","addr":"172.21.15.150:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":38418},{"type":"v1","addr":"172.21.15.150:6803","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":38418},{"type":"v1","addr":"172.21.15.150:6807","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":38418},{"type":"v1","addr":"172.21.15.150:6805","nonce":38418}]},"public_addr":"172.21.15.150:6801/38418","cluster_addr":"172.21.15.150:6803/38418","heartbeat_back_addr":"172.21.15.150:6807/38418","heartbeat_front_addr":"172.21.15.150:6805/38418","state":["exists","up"]},{"osd":6,"uuid":"e59fa189-4b70-4c06-9722-c02b90828bf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":38428},{"type":"v1","addr":"172.21.15.150:6825","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":38428},{"type":"v1","addr":"172.21.15.150:6827","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":38428},{"type":"v1","addr":"172.21.15.150:6831","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":38428},{"type":"v1","addr":"172.21.15.150:6829","nonce":38428}]},"public_addr":"172.21.15.150:6825/38428","cluster_addr":"172.21.15.150:6827/38428","heartbeat_back_addr":"172.21.15.150:6831/38428","heartbeat_front_addr":"172.21.15.150:6829/38428","state":["exists","up"]},{"osd":7,"uuid":"a64f57e1-ef98-46bd-b304-385759e95898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":38421},{"type":"v1","addr":"172.21.15.150:6817","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":38421},{"type":"v1","addr":"172.21.15.150:6819","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":38421},{"type":"v1","addr":"172.21.15.150:6823","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":38421},{"type":"v1","addr":"172.21.15.150:6821","nonce":38421}]},"public_addr":"172.21.15.150:6817/38421","cluster_addr":"172.21.15.150:6819/38421","heartbeat_back_addr":"172.21.15.150:6823/38421","heartbeat_front_addr":"172.21.15.150:6821/38421","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.438552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.413280+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.296466+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.424966+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.276953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.230984+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.410235+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.480084+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T02:08:27.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.643+0000 7f0356d70640 1 -- 172.21.15.49:0/582146935 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f033c065a60 msgr2=0x7f033c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:27.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.643+0000 7f0356d70640 1 --2- 172.21.15.49:0/582146935 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f033c065a60 0x7f033c067f20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f03440ae000 tx=0x7f0344009040 comp rx=0 tx=0).stop 2023-11-06T02:08:27.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.643+0000 7f0356d70640 1 -- 172.21.15.49:0/582146935 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03500caf30 msgr2=0x7f03500c5680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:27.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.643+0000 7f0356d70640 1 --2- 172.21.15.49:0/582146935 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03500caf30 0x7f03500c5680 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f034000efc0 tx=0x7f034000c490 comp rx=0 tx=0).stop 2023-11-06T02:08:27.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.643+0000 7f0356d70640 1 -- 172.21.15.49:0/582146935 shutdown_connections 2023-11-06T02:08:27.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.643+0000 7f0356d70640 1 --2- 172.21.15.49:0/582146935 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f033c065a60 0x7f033c067f20 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:27.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.643+0000 7f0356d70640 1 --2- 172.21.15.49:0/582146935 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0350150f70 0x7f03500c7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:27.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.643+0000 7f0356d70640 1 --2- 172.21.15.49:0/582146935 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f035015ceb0 0x7f0350150a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:27.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.643+0000 7f0356d70640 1 --2- 172.21.15.49:0/582146935 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03500caf30 0x7f03500c5680 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:27.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.643+0000 7f0356d70640 1 -- 172.21.15.49:0/582146935 >> 172.21.15.49:0/582146935 conn(0x7f03500cc140 msgr2=0x7f03500ae210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:27.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.643+0000 7f0356d70640 1 -- 172.21.15.49:0/582146935 shutdown_connections 2023-11-06T02:08:27.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.643+0000 7f0356d70640 1 -- 172.21.15.49:0/582146935 wait complete. 2023-11-06T02:08:27.658 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T02:08:19.252222+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T02:08:24.776211+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T02:08:27.659 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T02:08:27.784 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.787+0000 7fc12df99640 1 Processor -- start 2023-11-06T02:08:27.784 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.787+0000 7fc12df99640 1 -- start start 2023-11-06T02:08:27.787 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.787+0000 7fc12df99640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc1281595a0 0x7fc12815dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:27.787 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.787+0000 7fc12df99640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc12815a990 0x7fc1280c8e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:27.788 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.787+0000 7fc12df99640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc12806c5a0 con 0x7fc12806d640 2023-11-06T02:08:27.788 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.787+0000 7fc12df99640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc12815e250 con 0x7fc1281595a0 2023-11-06T02:08:27.788 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.787+0000 7fc12df99640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc12806c880 con 0x7fc12815a990 2023-11-06T02:08:27.788 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.787+0000 7fc126ffd640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fc12806d640 0x7fc1280b0220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:53138/0 (socket says 172.21.15.49:53138) 2023-11-06T02:08:27.788 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.787+0000 7fc126ffd640 1 -- 172.21.15.49:0/2261389862 learned_addr learned my addr 172.21.15.49:0/2261389862 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:08:27.789 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.787+0000 7fc1277fe640 1 --2- 172.21.15.49:0/2261389862 >> v2:172.21.15.150:3300/0 conn(0x7fc1281595a0 0x7fc12815dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:27.789 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.787+0000 7fc127fff640 1 --2- 172.21.15.49:0/2261389862 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc12815a990 0x7fc1280c8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:27.789 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.787+0000 7fc1267fc640 1 -- 172.21.15.49:0/2261389862 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4205101962 0 0) 0x7fc12806c5a0 con 0x7fc12806d640 2023-11-06T02:08:27.789 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.787+0000 7fc1267fc640 1 -- 172.21.15.49:0/2261389862 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc110003530 con 0x7fc12806d640 2023-11-06T02:08:27.789 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.787+0000 7fc1267fc640 1 -- 172.21.15.49:0/2261389862 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3501974314 0 0) 0x7fc110003530 con 0x7fc12806d640 2023-11-06T02:08:27.789 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.787+0000 7fc1267fc640 1 -- 172.21.15.49:0/2261389862 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc12815a990 msgr2=0x7fc1280c8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:27.790 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.787+0000 7fc1267fc640 1 --2- 172.21.15.49:0/2261389862 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc12815a990 0x7fc1280c8e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:27.790 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.787+0000 7fc1267fc640 1 -- 172.21.15.49:0/2261389862 >> v2:172.21.15.150:3300/0 conn(0x7fc1281595a0 msgr2=0x7fc12815dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:27.790 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.787+0000 7fc1267fc640 1 --2- 172.21.15.49:0/2261389862 >> v2:172.21.15.150:3300/0 conn(0x7fc1281595a0 0x7fc12815dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:27.790 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.787+0000 7fc1267fc640 1 -- 172.21.15.49:0/2261389862 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc12806cb60 con 0x7fc12806d640 2023-11-06T02:08:27.791 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.787+0000 7fc1267fc640 1 -- 172.21.15.49:0/2261389862 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7fc11c001680 con 0x7fc12806d640 2023-11-06T02:08:27.791 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.787+0000 7fc1267fc640 1 -- 172.21.15.49:0/2261389862 >> v1:172.21.15.49:6789/0 conn(0x7fc12806d640 legacy=0x7fc1280b0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:27.791 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.787+0000 7fc1267fc640 1 --2- 172.21.15.49:0/2261389862 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc12815a990 0x7fc110003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:27.791 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.787+0000 7fc1267fc640 1 --2- 172.21.15.49:0/2261389862 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc110003f00 0x7fc1100047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:27.791 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.787+0000 7fc1267fc640 1 --2- 172.21.15.49:0/2261389862 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc110004d10 0x7fc110005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:27.791 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.787+0000 7fc1267fc640 1 -- 172.21.15.49:0/2261389862 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc12815e250 con 0x7fc110003f00 2023-11-06T02:08:27.792 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.787+0000 7fc1267fc640 1 -- 172.21.15.49:0/2261389862 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc12806c880 con 0x7fc110004d10 2023-11-06T02:08:27.792 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.787+0000 7fc1267fc640 1 -- 172.21.15.49:0/2261389862 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc11000b990 con 0x7fc12815a990 2023-11-06T02:08:27.792 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.787+0000 7fc126ffd640 1 --2- 172.21.15.49:0/2261389862 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc110003f00 0x7fc1100047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:27.792 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.787+0000 7fc127fff640 1 --2- 172.21.15.49:0/2261389862 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc110004d10 0x7fc110005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:27.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.787+0000 7fc1277fe640 1 --2- 172.21.15.49:0/2261389862 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc12815a990 0x7fc110003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:27.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.787+0000 7fc126ffd640 1 -- 172.21.15.49:0/2261389862 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc12815a990 msgr2=0x7fc110003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:27.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.787+0000 7fc126ffd640 1 --2- 172.21.15.49:0/2261389862 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc12815a990 0x7fc110003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:27.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.787+0000 7fc126ffd640 1 -- 172.21.15.49:0/2261389862 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc110004d10 msgr2=0x7fc110005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:27.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.787+0000 7fc126ffd640 1 --2- 172.21.15.49:0/2261389862 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc110004d10 0x7fc110005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:27.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.787+0000 7fc126ffd640 1 -- 172.21.15.49:0/2261389862 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc110003530 con 0x7fc110003f00 2023-11-06T02:08:27.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.791+0000 7fc126ffd640 1 --2- 172.21.15.49:0/2261389862 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc110003f00 0x7fc1100047d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc11c00c240 tx=0x7fc11c01f370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c98293a8115105c0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:27.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.791+0000 7fc1267fc640 1 -- 172.21.15.49:0/2261389862 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc11c005ae0 con 0x7fc110003f00 2023-11-06T02:08:27.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.791+0000 7fc1267fc640 1 -- 172.21.15.49:0/2261389862 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc11c0ab070 con 0x7fc110003f00 2023-11-06T02:08:27.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.791+0000 7fc1267fc640 1 -- 172.21.15.49:0/2261389862 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc11c007070 con 0x7fc110003f00 2023-11-06T02:08:27.798 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.791+0000 7fc12df99640 1 -- 172.21.15.49:0/2261389862 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc110003f00 msgr2=0x7fc1100047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:27.798 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.791+0000 7fc12df99640 1 --2- 172.21.15.49:0/2261389862 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc110003f00 0x7fc1100047d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc11c00c240 tx=0x7fc11c01f370 comp rx=0 tx=0).stop 2023-11-06T02:08:27.798 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.791+0000 7fc127fff640 1 -- 172.21.15.49:0/2261389862 reap_dead start 2023-11-06T02:08:27.798 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.791+0000 7fc12df99640 1 -- 172.21.15.49:0/2261389862 shutdown_connections 2023-11-06T02:08:27.798 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.791+0000 7fc12df99640 1 -- 172.21.15.49:0/2261389862 >> 172.21.15.49:0/2261389862 conn(0x7fc12806a370 msgr2=0x7fc1280ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:27.799 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.791+0000 7fc12df99640 1 -- 172.21.15.49:0/2261389862 shutdown_connections 2023-11-06T02:08:27.799 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.791+0000 7fc12df99640 1 -- 172.21.15.49:0/2261389862 wait complete. 2023-11-06T02:08:27.799 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.791+0000 7fc12df99640 1 Processor -- start 2023-11-06T02:08:27.799 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.791+0000 7fc12df99640 1 -- start start 2023-11-06T02:08:27.799 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.791+0000 7fc12df99640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc110003f00 0x7fc12815c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:27.799 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.791+0000 7fc12df99640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc12814ecc0 0x7fc1281477b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:27.800 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.791+0000 7fc12df99640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc12814f140 0x7fc12814f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:27.800 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.791+0000 7fc12df99640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc12814e8a0 con 0x7fc110003f00 2023-11-06T02:08:27.800 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.791+0000 7fc12df99640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc12815ce90 con 0x7fc12814ecc0 2023-11-06T02:08:27.800 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.791+0000 7fc12df99640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc12815aff0 con 0x7fc12814f140 2023-11-06T02:08:27.800 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.791+0000 7fc1277fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc110003f00 0x7fc12815c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:27.801 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.791+0000 7fc1277fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc110003f00 0x7fc12815c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41518/0 (socket says 172.21.15.49:41518) 2023-11-06T02:08:27.801 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.791+0000 7fc1277fe640 1 -- 172.21.15.49:0/1517448927 learned_addr learned my addr 172.21.15.49:0/1517448927 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:27.801 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.791+0000 7fc126ffd640 1 --2- 172.21.15.49:0/1517448927 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc12814ecc0 0x7fc1281477b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:27.801 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.791+0000 7fc127fff640 1 --2- 172.21.15.49:0/1517448927 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc12814f140 0x7fc12814f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:27.801 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.791+0000 7fc1277fe640 1 -- 172.21.15.49:0/1517448927 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc12814f140 msgr2=0x7fc12814f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:27.801 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.791+0000 7fc1277fe640 1 --2- 172.21.15.49:0/1517448927 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc12814f140 0x7fc12814f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:27.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.791+0000 7fc1277fe640 1 -- 172.21.15.49:0/1517448927 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc12814ecc0 msgr2=0x7fc1281477b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:27.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.791+0000 7fc1277fe640 1 --2- 172.21.15.49:0/1517448927 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc12814ecc0 0x7fc1281477b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:27.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.791+0000 7fc1277fe640 1 -- 172.21.15.49:0/1517448927 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc12815b800 con 0x7fc110003f00 2023-11-06T02:08:27.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.791+0000 7fc1277fe640 1 --2- 172.21.15.49:0/1517448927 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc110003f00 0x7fc12815c790 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc118011610 tx=0x7fc118011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:27.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.791+0000 7fc10ffff640 1 -- 172.21.15.49:0/1517448927 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc11800f8f0 con 0x7fc110003f00 2023-11-06T02:08:27.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.791+0000 7fc10ffff640 1 -- 172.21.15.49:0/1517448927 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc11800fa50 con 0x7fc110003f00 2023-11-06T02:08:27.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.791+0000 7fc12df99640 1 -- 172.21.15.49:0/1517448927 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc12815bae0 con 0x7fc110003f00 2023-11-06T02:08:27.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.795+0000 7fc10ffff640 1 -- 172.21.15.49:0/1517448927 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc11800fdc0 con 0x7fc110003f00 2023-11-06T02:08:27.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.795+0000 7fc12df99640 1 -- 172.21.15.49:0/1517448927 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc128147cf0 con 0x7fc110003f00 2023-11-06T02:08:27.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.795+0000 7fc10ffff640 1 -- 172.21.15.49:0/1517448927 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fc118015460 con 0x7fc110003f00 2023-11-06T02:08:27.804 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.795+0000 7fc12df99640 1 -- 172.21.15.49:0/1517448927 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc128147fd0 con 0x7fc110003f00 2023-11-06T02:08:27.804 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.795+0000 7fc10ffff640 1 --2- 172.21.15.49:0/1517448927 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc114065a60 0x7fc114067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:27.804 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.795+0000 7fc10ffff640 1 -- 172.21.15.49:0/1517448927 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fc1180834e0 con 0x7fc110003f00 2023-11-06T02:08:27.804 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.795+0000 7fc126ffd640 1 --2- 172.21.15.49:0/1517448927 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc114065a60 0x7fc114067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:27.804 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.795+0000 7fc126ffd640 1 --2- 172.21.15.49:0/1517448927 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc114065a60 0x7fc114067f20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc11c0ae000 tx=0x7fc11c001a80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:27.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.799+0000 7fc10ffff640 1 -- 172.21.15.49:0/1517448927 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fc118051090 con 0x7fc110003f00 2023-11-06T02:08:27.950 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.951+0000 7fc12df99640 1 -- 172.21.15.49:0/1517448927 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc12806c210 con 0x7fc110003f00 2023-11-06T02:08:27.951 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.951+0000 7fc10ffff640 1 -- 172.21.15.49:0/1517448927 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v19) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc118022020 con 0x7fc110003f00 2023-11-06T02:08:27.951 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2023-11-06T02:08:27.953 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.955+0000 7fc12df99640 1 -- 172.21.15.49:0/1517448927 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc114065a60 msgr2=0x7fc114067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:27.953 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.955+0000 7fc12df99640 1 --2- 172.21.15.49:0/1517448927 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc114065a60 0x7fc114067f20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc11c0ae000 tx=0x7fc11c001a80 comp rx=0 tx=0).stop 2023-11-06T02:08:27.953 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.955+0000 7fc12df99640 1 -- 172.21.15.49:0/1517448927 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc110003f00 msgr2=0x7fc12815c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:27.953 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.955+0000 7fc12df99640 1 --2- 172.21.15.49:0/1517448927 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc110003f00 0x7fc12815c790 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc118011610 tx=0x7fc118011ad0 comp rx=0 tx=0).stop 2023-11-06T02:08:27.954 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.955+0000 7fc12df99640 1 -- 172.21.15.49:0/1517448927 shutdown_connections 2023-11-06T02:08:27.954 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.955+0000 7fc12df99640 1 --2- 172.21.15.49:0/1517448927 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc114065a60 0x7fc114067f20 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:27.954 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.955+0000 7fc12df99640 1 --2- 172.21.15.49:0/1517448927 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc12814f140 0x7fc12814f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:27.954 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.955+0000 7fc12df99640 1 --2- 172.21.15.49:0/1517448927 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc12814ecc0 0x7fc1281477b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:27.954 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.955+0000 7fc12df99640 1 --2- 172.21.15.49:0/1517448927 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc110003f00 0x7fc12815c790 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:27.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.955+0000 7fc12df99640 1 -- 172.21.15.49:0/1517448927 >> 172.21.15.49:0/1517448927 conn(0x7fc12806a370 msgr2=0x7fc1280b01e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:27.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.955+0000 7fc12df99640 1 -- 172.21.15.49:0/1517448927 shutdown_connections 2023-11-06T02:08:27.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:27.955+0000 7fc12df99640 1 -- 172.21.15.49:0/1517448927 wait complete. 2023-11-06T02:08:27.968 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T02:08:28.097 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.099+0000 7f9f790b8640 1 Processor -- start 2023-11-06T02:08:28.097 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.099+0000 7f9f790b8640 1 -- start start 2023-11-06T02:08:28.098 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.099+0000 7f9f790b8640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f9f7415ad90 0x7f9f740cde40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:28.099 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.099+0000 7f9f790b8640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f9f740cd2a0 0x7f9f740cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:28.099 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.099+0000 7f9f790b8640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f740b3720 con 0x7f9f7415a9b0 2023-11-06T02:08:28.099 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.099+0000 7f9f790b8640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f740cdc10 con 0x7f9f7415ad90 2023-11-06T02:08:28.099 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.099+0000 7f9f790b8640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f740cf690 con 0x7f9f740cd2a0 2023-11-06T02:08:28.099 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.099+0000 7f9f7259c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f9f7415ad90 0x7f9f740cde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:28.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.099+0000 7f9f7259c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f9f7415ad90 0x7f9f740cde40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41530/0 (socket says 172.21.15.49:41530) 2023-11-06T02:08:28.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.099+0000 7f9f7259c640 1 -- 172.21.15.49:0/3209797601 learned_addr learned my addr 172.21.15.49:0/3209797601 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:28.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.099+0000 7f9f7359e640 1 --2- 172.21.15.49:0/3209797601 >> v2:172.21.15.150:3300/0 conn(0x7f9f740cd2a0 0x7f9f740cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:28.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.099+0000 7f9f7259c640 1 -- 172.21.15.49:0/3209797601 >> v2:172.21.15.150:3300/0 conn(0x7f9f740cd2a0 msgr2=0x7f9f740cd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:28.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.099+0000 7f9f7259c640 1 --2- 172.21.15.49:0/3209797601 >> v2:172.21.15.150:3300/0 conn(0x7f9f740cd2a0 0x7f9f740cd6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:28.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.099+0000 7f9f7259c640 1 -- 172.21.15.49:0/3209797601 >> v1:172.21.15.49:6789/0 conn(0x7f9f7415a9b0 legacy=0x7f9f740cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:28.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.099+0000 7f9f7259c640 1 -- 172.21.15.49:0/3209797601 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f740b3970 con 0x7f9f7415ad90 2023-11-06T02:08:28.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.099+0000 7f9f71d9b640 1 -- 172.21.15.49:0/3209797601 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1384495611 0 0) 0x7f9f740b3720 con 0x7f9f7415a9b0 2023-11-06T02:08:28.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.103+0000 7f9f7259c640 1 --2- 172.21.15.49:0/3209797601 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9f7415ad90 0x7f9f740cde40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9f740caeb0 tx=0x7f9f6801c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37c8a7e4cdeb73b7 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:28.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.103+0000 7f9f71d9b640 1 -- 172.21.15.49:0/3209797601 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f680ab5f0 con 0x7f9f7415ad90 2023-11-06T02:08:28.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.103+0000 7f9f71d9b640 1 -- 172.21.15.49:0/3209797601 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f680ab750 con 0x7f9f7415ad90 2023-11-06T02:08:28.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.103+0000 7f9f71d9b640 1 -- 172.21.15.49:0/3209797601 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f680abae0 con 0x7f9f7415ad90 2023-11-06T02:08:28.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.103+0000 7f9f790b8640 1 -- 172.21.15.49:0/3209797601 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9f7415ad90 msgr2=0x7f9f740cde40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:28.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.103+0000 7f9f790b8640 1 --2- 172.21.15.49:0/3209797601 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9f7415ad90 0x7f9f740cde40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9f740caeb0 tx=0x7f9f6801c500 comp rx=0 tx=0).stop 2023-11-06T02:08:28.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.103+0000 7f9f790b8640 1 -- 172.21.15.49:0/3209797601 shutdown_connections 2023-11-06T02:08:28.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.103+0000 7f9f790b8640 1 --2- 172.21.15.49:0/3209797601 >> v2:172.21.15.150:3300/0 conn(0x7f9f740cd2a0 0x7f9f740cd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:28.105 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.103+0000 7f9f790b8640 1 --2- 172.21.15.49:0/3209797601 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9f7415ad90 0x7f9f740cde40 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:28.105 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.103+0000 7f9f790b8640 1 -- 172.21.15.49:0/3209797601 >> 172.21.15.49:0/3209797601 conn(0x7f9f740d08d0 msgr2=0x7f9f740d0cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:28.105 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.103+0000 7f9f790b8640 1 -- 172.21.15.49:0/3209797601 shutdown_connections 2023-11-06T02:08:28.105 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.103+0000 7f9f790b8640 1 -- 172.21.15.49:0/3209797601 wait complete. 2023-11-06T02:08:28.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.103+0000 7f9f790b8640 1 Processor -- start 2023-11-06T02:08:28.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.103+0000 7f9f790b8640 1 -- start start 2023-11-06T02:08:28.109 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.107+0000 7f9f790b8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9f740cd2a0 0x7f9f740c6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:28.109 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.107+0000 7f9f790b8640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f7415a9b0 0x7f9f740d0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:28.109 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.107+0000 7f9f790b8640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9f7415ad90 0x7f9f740d11f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:28.109 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.107+0000 7f9f790b8640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f740c80b0 con 0x7f9f740cd2a0 2023-11-06T02:08:28.109 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.107+0000 7f9f790b8640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f740c82d0 con 0x7f9f7415a9b0 2023-11-06T02:08:28.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.107+0000 7f9f790b8640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f740c85b0 con 0x7f9f7415ad90 2023-11-06T02:08:28.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.107+0000 7f9f72d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9f740cd2a0 0x7f9f740c6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:28.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.107+0000 7f9f72d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9f740cd2a0 0x7f9f740c6660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41536/0 (socket says 172.21.15.49:41536) 2023-11-06T02:08:28.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.107+0000 7f9f72d9d640 1 -- 172.21.15.49:0/3881807667 learned_addr learned my addr 172.21.15.49:0/3881807667 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:28.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.107+0000 7f9f7259c640 1 --2- 172.21.15.49:0/3881807667 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f7415a9b0 0x7f9f740d0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:28.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.107+0000 7f9f7359e640 1 --2- 172.21.15.49:0/3881807667 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9f7415ad90 0x7f9f740d11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:28.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.107+0000 7f9f72d9d640 1 -- 172.21.15.49:0/3881807667 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9f7415ad90 msgr2=0x7f9f740d11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:28.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.107+0000 7f9f72d9d640 1 --2- 172.21.15.49:0/3881807667 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9f7415ad90 0x7f9f740d11f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:28.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.107+0000 7f9f72d9d640 1 -- 172.21.15.49:0/3881807667 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f7415a9b0 msgr2=0x7f9f740d0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:28.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.107+0000 7f9f72d9d640 1 --2- 172.21.15.49:0/3881807667 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f7415a9b0 0x7f9f740d0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:28.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.107+0000 7f9f72d9d640 1 -- 172.21.15.49:0/3881807667 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f680094d0 con 0x7f9f740cd2a0 2023-11-06T02:08:28.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.107+0000 7f9f72d9d640 1 --2- 172.21.15.49:0/3881807667 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9f740cd2a0 0x7f9f740c6660 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9f6400c450 tx=0x7f9f6400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:28.113 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.107+0000 7f9f5b7fe640 1 -- 172.21.15.49:0/3881807667 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f640054e0 con 0x7f9f740cd2a0 2023-11-06T02:08:28.113 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.107+0000 7f9f5b7fe640 1 -- 172.21.15.49:0/3881807667 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f64003600 con 0x7f9f740cd2a0 2023-11-06T02:08:28.113 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.107+0000 7f9f790b8640 1 -- 172.21.15.49:0/3881807667 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f740c8890 con 0x7f9f740cd2a0 2023-11-06T02:08:28.113 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.107+0000 7f9f5b7fe640 1 -- 172.21.15.49:0/3881807667 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f640039b0 con 0x7f9f740cd2a0 2023-11-06T02:08:28.113 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.107+0000 7f9f790b8640 1 -- 172.21.15.49:0/3881807667 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f740c8e50 con 0x7f9f740cd2a0 2023-11-06T02:08:28.114 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.107+0000 7f9f790b8640 1 -- 172.21.15.49:0/3881807667 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f74074040 con 0x7f9f740cd2a0 2023-11-06T02:08:28.114 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.107+0000 7f9f5b7fe640 1 -- 172.21.15.49:0/3881807667 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f9f64003b10 con 0x7f9f740cd2a0 2023-11-06T02:08:28.114 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.107+0000 7f9f5b7fe640 1 --2- 172.21.15.49:0/3881807667 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9f60065a90 0x7f9f60067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:28.114 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.107+0000 7f9f5b7fe640 1 -- 172.21.15.49:0/3881807667 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f9f640824e0 con 0x7f9f740cd2a0 2023-11-06T02:08:28.114 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.107+0000 7f9f7259c640 1 --2- 172.21.15.49:0/3881807667 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9f60065a90 0x7f9f60067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:28.115 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.111+0000 7f9f7259c640 1 --2- 172.21.15.49:0/3881807667 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9f60065a90 0x7f9f60067f50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9f680af000 tx=0x7f9f680a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:28.115 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.111+0000 7f9f5b7fe640 1 -- 172.21.15.49:0/3881807667 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f9f64050120 con 0x7f9f740cd2a0 2023-11-06T02:08:28.266 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.267+0000 7f9f790b8640 1 -- 172.21.15.49:0/3881807667 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f9f740cbeb0 con 0x7f9f740cd2a0 2023-11-06T02:08:28.267 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.267+0000 7f9f5b7fe640 1 -- 172.21.15.49:0/3881807667 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v19) v1 ==== 92+0+10 (secure 0 0 0) 0x7f9f640170a0 con 0x7f9f740cd2a0 2023-11-06T02:08:28.268 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 8 2023-11-06T02:08:28.270 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.271+0000 7f9f790b8640 1 -- 172.21.15.49:0/3881807667 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9f60065a90 msgr2=0x7f9f60067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:28.270 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.271+0000 7f9f790b8640 1 --2- 172.21.15.49:0/3881807667 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9f60065a90 0x7f9f60067f50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9f680af000 tx=0x7f9f680a7000 comp rx=0 tx=0).stop 2023-11-06T02:08:28.271 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.271+0000 7f9f790b8640 1 -- 172.21.15.49:0/3881807667 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9f740cd2a0 msgr2=0x7f9f740c6660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:28.271 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.271+0000 7f9f790b8640 1 --2- 172.21.15.49:0/3881807667 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9f740cd2a0 0x7f9f740c6660 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9f6400c450 tx=0x7f9f6400c910 comp rx=0 tx=0).stop 2023-11-06T02:08:28.271 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.271+0000 7f9f790b8640 1 -- 172.21.15.49:0/3881807667 shutdown_connections 2023-11-06T02:08:28.272 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.271+0000 7f9f790b8640 1 --2- 172.21.15.49:0/3881807667 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9f60065a90 0x7f9f60067f50 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:28.272 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.271+0000 7f9f790b8640 1 --2- 172.21.15.49:0/3881807667 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9f7415ad90 0x7f9f740d11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:28.272 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.271+0000 7f9f790b8640 1 --2- 172.21.15.49:0/3881807667 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f7415a9b0 0x7f9f740d0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:28.273 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.271+0000 7f9f790b8640 1 --2- 172.21.15.49:0/3881807667 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9f740cd2a0 0x7f9f740c6660 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:28.273 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.271+0000 7f9f790b8640 1 -- 172.21.15.49:0/3881807667 >> 172.21.15.49:0/3881807667 conn(0x7f9f740d08d0 msgr2=0x7f9f7406e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:28.273 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.271+0000 7f9f790b8640 1 -- 172.21.15.49:0/3881807667 shutdown_connections 2023-11-06T02:08:28.273 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.271+0000 7f9f790b8640 1 -- 172.21.15.49:0/3881807667 wait complete. 2023-11-06T02:08:28.287 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-06T02:08:28.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.411+0000 7f84268f8640 1 Processor -- start 2023-11-06T02:08:28.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.411+0000 7f84268f8640 1 -- start start 2023-11-06T02:08:28.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.415+0000 7f84268f8640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f84201646f0 0x7f84200b0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:28.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.415+0000 7f84268f8640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f84200c8c50 0x7f84200b05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:28.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.415+0000 7f84268f8640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f84200adb00 0x7f84200cb010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:28.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.415+0000 7f84268f8640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8420165b80 con 0x7f84200c8c50 2023-11-06T02:08:28.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.415+0000 7f84268f8640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8420165600 con 0x7f84201646f0 2023-11-06T02:08:28.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.415+0000 7f84268f8640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84201658a0 con 0x7f84200adb00 2023-11-06T02:08:28.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.415+0000 7f841f7fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f84200c8c50 0x7f84200b05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:28.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.415+0000 7f841f7fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f84200c8c50 0x7f84200b05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41552/0 (socket says 172.21.15.49:41552) 2023-11-06T02:08:28.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.415+0000 7f841f7fe640 1 -- 172.21.15.49:0/3809746136 learned_addr learned my addr 172.21.15.49:0/3809746136 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:28.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.415+0000 7f8424e95640 1 --2- 172.21.15.49:0/3809746136 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f84200adb00 0x7f84200cb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:28.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.415+0000 7f841ffff640 1 --2- 172.21.15.49:0/3809746136 >> v2:172.21.15.150:3300/0 conn(0x7f84201646f0 0x7f84200b0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:28.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.415+0000 7f841f7fe640 1 -- 172.21.15.49:0/3809746136 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f84200adb00 msgr2=0x7f84200cb010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:28.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.415+0000 7f841f7fe640 1 --2- 172.21.15.49:0/3809746136 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f84200adb00 0x7f84200cb010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:28.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.415+0000 7f841f7fe640 1 -- 172.21.15.49:0/3809746136 >> v2:172.21.15.150:3300/0 conn(0x7f84201646f0 msgr2=0x7f84200b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:28.417 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.415+0000 7f841f7fe640 1 --2- 172.21.15.49:0/3809746136 >> v2:172.21.15.150:3300/0 conn(0x7f84201646f0 0x7f84200b0080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:28.417 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.415+0000 7f841f7fe640 1 -- 172.21.15.49:0/3809746136 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84200c96f0 con 0x7f84200c8c50 2023-11-06T02:08:28.417 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.415+0000 7f841ffff640 1 --2- 172.21.15.49:0/3809746136 >> v2:172.21.15.150:3300/0 conn(0x7f84201646f0 0x7f84200b0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:08:28.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.415+0000 7f841f7fe640 1 --2- 172.21.15.49:0/3809746136 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84200c8c50 0x7f84200b05c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f842006ceb0 tx=0x7f840801c240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89e6847b526f958c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:28.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.415+0000 7f841effd640 1 -- 172.21.15.49:0/3809746136 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84080ab600 con 0x7f84200c8c50 2023-11-06T02:08:28.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.415+0000 7f841effd640 1 -- 172.21.15.49:0/3809746136 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84080a7050 con 0x7f84200c8c50 2023-11-06T02:08:28.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.415+0000 7f841effd640 1 -- 172.21.15.49:0/3809746136 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84080ab980 con 0x7f84200c8c50 2023-11-06T02:08:28.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.419+0000 7f84268f8640 1 -- 172.21.15.49:0/3809746136 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84200c8c50 msgr2=0x7f84200b05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:28.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.419+0000 7f84268f8640 1 --2- 172.21.15.49:0/3809746136 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84200c8c50 0x7f84200b05c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f842006ceb0 tx=0x7f840801c240 comp rx=0 tx=0).stop 2023-11-06T02:08:28.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.419+0000 7f84268f8640 1 -- 172.21.15.49:0/3809746136 shutdown_connections 2023-11-06T02:08:28.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.419+0000 7f84268f8640 1 --2- 172.21.15.49:0/3809746136 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f84200adb00 0x7f84200cb010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:28.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.419+0000 7f84268f8640 1 --2- 172.21.15.49:0/3809746136 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84200c8c50 0x7f84200b05c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:28.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.419+0000 7f84268f8640 1 --2- 172.21.15.49:0/3809746136 >> v2:172.21.15.150:3300/0 conn(0x7f84201646f0 0x7f84200b0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:28.422 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.419+0000 7f84268f8640 1 -- 172.21.15.49:0/3809746136 >> 172.21.15.49:0/3809746136 conn(0x7f842006c0a0 msgr2=0x7f842006c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:28.422 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.419+0000 7f84268f8640 1 -- 172.21.15.49:0/3809746136 shutdown_connections 2023-11-06T02:08:28.422 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.419+0000 7f84268f8640 1 -- 172.21.15.49:0/3809746136 wait complete. 2023-11-06T02:08:28.422 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.419+0000 7f84268f8640 1 Processor -- start 2023-11-06T02:08:28.422 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.419+0000 7f84268f8640 1 -- start start 2023-11-06T02:08:28.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.419+0000 7f84268f8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84200adb00 0x7f8420161180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:28.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.419+0000 7f84268f8640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84200c8c50 0x7f84200c6d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:28.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.419+0000 7f84268f8640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f84201646f0 0x7f8420070aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:28.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.419+0000 7f84268f8640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8420160cf0 con 0x7f84200adb00 2023-11-06T02:08:28.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.419+0000 7f84268f8640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84201600e0 con 0x7f84200c8c50 2023-11-06T02:08:28.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.419+0000 7f84268f8640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84201603c0 con 0x7f84201646f0 2023-11-06T02:08:28.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.419+0000 7f841ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84200adb00 0x7f8420161180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:28.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.419+0000 7f841ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84200adb00 0x7f8420161180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41568/0 (socket says 172.21.15.49:41568) 2023-11-06T02:08:28.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.419+0000 7f841ffff640 1 -- 172.21.15.49:0/3331051208 learned_addr learned my addr 172.21.15.49:0/3331051208 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:28.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.419+0000 7f8424e95640 1 --2- 172.21.15.49:0/3331051208 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f84201646f0 0x7f8420070aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:28.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.419+0000 7f841f7fe640 1 --2- 172.21.15.49:0/3331051208 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84200c8c50 0x7f84200c6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:28.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.419+0000 7f841ffff640 1 -- 172.21.15.49:0/3331051208 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f84201646f0 msgr2=0x7f8420070aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:28.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.419+0000 7f841ffff640 1 --2- 172.21.15.49:0/3331051208 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f84201646f0 0x7f8420070aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:28.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.419+0000 7f841ffff640 1 -- 172.21.15.49:0/3331051208 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84200c8c50 msgr2=0x7f84200c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:28.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.419+0000 7f841ffff640 1 --2- 172.21.15.49:0/3331051208 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84200c8c50 0x7f84200c6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:28.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.419+0000 7f841ffff640 1 -- 172.21.15.49:0/3331051208 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84080094d0 con 0x7f84200adb00 2023-11-06T02:08:28.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.419+0000 7f841f7fe640 1 --2- 172.21.15.49:0/3331051208 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84200c8c50 0x7f84200c6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:08:28.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.419+0000 7f841ffff640 1 --2- 172.21.15.49:0/3331051208 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84200adb00 0x7f8420161180 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8418007ef0 tx=0x7f841800d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:28.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.419+0000 7f841cff9640 1 -- 172.21.15.49:0/3331051208 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8418003ba0 con 0x7f84200adb00 2023-11-06T02:08:28.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.419+0000 7f841cff9640 1 -- 172.21.15.49:0/3331051208 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f841800f040 con 0x7f84200adb00 2023-11-06T02:08:28.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.419+0000 7f84268f8640 1 -- 172.21.15.49:0/3331051208 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84201606a0 con 0x7f84200adb00 2023-11-06T02:08:28.427 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.419+0000 7f841cff9640 1 -- 172.21.15.49:0/3331051208 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8418004030 con 0x7f84200adb00 2023-11-06T02:08:28.427 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.419+0000 7f84268f8640 1 -- 172.21.15.49:0/3331051208 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8420157d70 con 0x7f84200adb00 2023-11-06T02:08:28.428 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.423+0000 7f84268f8640 1 -- 172.21.15.49:0/3331051208 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8420074040 con 0x7f84200adb00 2023-11-06T02:08:28.428 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.423+0000 7f841cff9640 1 -- 172.21.15.49:0/3331051208 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f8418006d50 con 0x7f84200adb00 2023-11-06T02:08:28.428 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.423+0000 7f841cff9640 1 --2- 172.21.15.49:0/3331051208 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8400065a60 0x7f8400067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:28.428 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.423+0000 7f841cff9640 1 -- 172.21.15.49:0/3331051208 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f8418082fe0 con 0x7f84200adb00 2023-11-06T02:08:28.429 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.423+0000 7f841f7fe640 1 --2- 172.21.15.49:0/3331051208 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8400065a60 0x7f8400067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:28.429 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.423+0000 7f841f7fe640 1 --2- 172.21.15.49:0/3331051208 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8400065a60 0x7f8400067f20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f84080ad000 tx=0x7f8408008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:28.429 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.427+0000 7f841cff9640 1 -- 172.21.15.49:0/3331051208 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f8418050c40 con 0x7f84200adb00 2023-11-06T02:08:28.580 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.583+0000 7f84268f8640 1 -- 172.21.15.49:0/3331051208 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f84200b4580 con 0x7f84200adb00 2023-11-06T02:08:28.581 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.583+0000 7f841cff9640 1 -- 172.21.15.49:0/3331051208 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v19) v1 ==== 84+0+126 (secure 0 0 0) 0x7f841801a090 con 0x7f84200adb00 2023-11-06T02:08:28.582 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:08:28.582 INFO:teuthology.orchestra.run.smithi049.stdout:[ 2023-11-06T02:08:28.582 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:08:28.583 INFO:teuthology.orchestra.run.smithi049.stdout: "poolnum": 1, 2023-11-06T02:08:28.583 INFO:teuthology.orchestra.run.smithi049.stdout: "poolname": ".mgr" 2023-11-06T02:08:28.583 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:08:28.583 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:08:28.583 INFO:teuthology.orchestra.run.smithi049.stdout: "poolnum": 2, 2023-11-06T02:08:28.583 INFO:teuthology.orchestra.run.smithi049.stdout: "poolname": "rbd" 2023-11-06T02:08:28.584 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:08:28.584 INFO:teuthology.orchestra.run.smithi049.stdout:] 2023-11-06T02:08:28.585 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.583+0000 7f84268f8640 1 -- 172.21.15.49:0/3331051208 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8400065a60 msgr2=0x7f8400067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:28.585 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.583+0000 7f84268f8640 1 --2- 172.21.15.49:0/3331051208 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8400065a60 0x7f8400067f20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f84080ad000 tx=0x7f8408008040 comp rx=0 tx=0).stop 2023-11-06T02:08:28.585 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.583+0000 7f84268f8640 1 -- 172.21.15.49:0/3331051208 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84200adb00 msgr2=0x7f8420161180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:28.586 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.583+0000 7f84268f8640 1 --2- 172.21.15.49:0/3331051208 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84200adb00 0x7f8420161180 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8418007ef0 tx=0x7f841800d4a0 comp rx=0 tx=0).stop 2023-11-06T02:08:28.586 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.583+0000 7f84268f8640 1 -- 172.21.15.49:0/3331051208 shutdown_connections 2023-11-06T02:08:28.586 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.583+0000 7f84268f8640 1 --2- 172.21.15.49:0/3331051208 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8400065a60 0x7f8400067f20 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:28.586 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.583+0000 7f84268f8640 1 --2- 172.21.15.49:0/3331051208 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f84201646f0 0x7f8420070aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:28.586 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.583+0000 7f84268f8640 1 --2- 172.21.15.49:0/3331051208 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84200c8c50 0x7f84200c6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:28.587 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.583+0000 7f84268f8640 1 --2- 172.21.15.49:0/3331051208 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84200adb00 0x7f8420161180 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:28.587 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.583+0000 7f84268f8640 1 -- 172.21.15.49:0/3331051208 >> 172.21.15.49:0/3331051208 conn(0x7f842006c0a0 msgr2=0x7f842015de10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:28.587 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.583+0000 7f84268f8640 1 -- 172.21.15.49:0/3331051208 shutdown_connections 2023-11-06T02:08:28.587 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.583+0000 7f84268f8640 1 -- 172.21.15.49:0/3331051208 wait complete. 2023-11-06T02:08:28.598 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-06T02:08:28.599 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-06T02:08:28.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.719+0000 7fe40480f640 1 Processor -- start 2023-11-06T02:08:28.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.719+0000 7fe40480f640 1 -- start start 2023-11-06T02:08:28.721 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.723+0000 7fe40480f640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fe4000cb8b0 0x7fe4000b22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:28.721 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.723+0000 7fe40480f640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4000b2c20 con 0x7fe40015a930 2023-11-06T02:08:28.721 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.723+0000 7fe40480f640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe40015e6f0 con 0x7fe4000b2810 2023-11-06T02:08:28.722 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.723+0000 7fe40480f640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4000b6830 con 0x7fe4000cb8b0 2023-11-06T02:08:28.722 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.723+0000 7fe3fe59c640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fe40015a930 0x7fe4000b6020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:44930/0 (socket says 172.21.15.49:44930) 2023-11-06T02:08:28.722 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.723+0000 7fe3fe59c640 1 -- 172.21.15.49:0/126999227 learned_addr learned my addr 172.21.15.49:0/126999227 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:08:28.722 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.723+0000 7fe3fdd9b640 1 --2- 172.21.15.49:0/126999227 >> v2:172.21.15.150:3300/0 conn(0x7fe4000cb8b0 0x7fe4000b22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:28.722 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.723+0000 7fe3fd59a640 1 -- 172.21.15.49:0/126999227 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 540440578 0 0) 0x7fe4000b2c20 con 0x7fe40015a930 2023-11-06T02:08:28.723 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.723+0000 7fe3fd59a640 1 -- 172.21.15.49:0/126999227 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3e8003530 con 0x7fe40015a930 2023-11-06T02:08:28.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.723+0000 7fe3fd59a640 1 -- 172.21.15.49:0/126999227 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3940950496 0 0) 0x7fe3e8003530 con 0x7fe40015a930 2023-11-06T02:08:28.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.723+0000 7fe3fd59a640 1 -- 172.21.15.49:0/126999227 >> v2:172.21.15.150:3300/0 conn(0x7fe4000cb8b0 msgr2=0x7fe4000b22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:28.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.723+0000 7fe3fd59a640 1 --2- 172.21.15.49:0/126999227 >> v2:172.21.15.150:3300/0 conn(0x7fe4000cb8b0 0x7fe4000b22d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:28.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.723+0000 7fe3fd59a640 1 -- 172.21.15.49:0/126999227 >> v1:172.21.15.150:6789/0 conn(0x7fe4000b2810 legacy=0x7fe4000ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:28.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.723+0000 7fe3fd59a640 1 -- 172.21.15.49:0/126999227 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe40015e9d0 con 0x7fe40015a930 2023-11-06T02:08:28.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.723+0000 7fe3fd59a640 1 -- 172.21.15.49:0/126999227 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7fe3f0001a00 con 0x7fe40015a930 2023-11-06T02:08:28.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.723+0000 7fe3fd59a640 1 -- 172.21.15.49:0/126999227 >> v1:172.21.15.49:6789/0 conn(0x7fe40015a930 legacy=0x7fe4000b6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:28.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.723+0000 7fe3fd59a640 1 --2- 172.21.15.49:0/126999227 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe3e8003980 0x7fe3e8003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:28.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.723+0000 7fe3fd59a640 1 --2- 172.21.15.49:0/126999227 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe3e80047d0 0x7fe3e8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:28.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.723+0000 7fe3fd59a640 1 --2- 172.21.15.49:0/126999227 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3e8005190 0x7fe3e800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:28.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.723+0000 7fe3fd59a640 1 -- 172.21.15.49:0/126999227 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4000b6830 con 0x7fe3e8005190 2023-11-06T02:08:28.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.723+0000 7fe3fd59a640 1 -- 172.21.15.49:0/126999227 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e800c130 con 0x7fe3e8003980 2023-11-06T02:08:28.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.723+0000 7fe3fd59a640 1 -- 172.21.15.49:0/126999227 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e800c2f0 con 0x7fe3e80047d0 2023-11-06T02:08:28.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.723+0000 7fe3fed9d640 1 --2- 172.21.15.49:0/126999227 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3e8005190 0x7fe3e800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:28.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.723+0000 7fe3fe59c640 1 --2- 172.21.15.49:0/126999227 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe3e8003980 0x7fe3e8003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:28.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.723+0000 7fe3fdd9b640 1 --2- 172.21.15.49:0/126999227 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe3e80047d0 0x7fe3e8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:28.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.723+0000 7fe3fed9d640 1 -- 172.21.15.49:0/126999227 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe3e80047d0 msgr2=0x7fe3e8004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:28.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.723+0000 7fe3fed9d640 1 --2- 172.21.15.49:0/126999227 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe3e80047d0 0x7fe3e8004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:28.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.723+0000 7fe3fed9d640 1 -- 172.21.15.49:0/126999227 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe3e8003980 msgr2=0x7fe3e8003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:28.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.723+0000 7fe3fed9d640 1 --2- 172.21.15.49:0/126999227 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe3e8003980 0x7fe3e8003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:28.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.723+0000 7fe3fed9d640 1 -- 172.21.15.49:0/126999227 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3e8003530 con 0x7fe3e8005190 2023-11-06T02:08:28.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.727+0000 7fe3fed9d640 1 -- 172.21.15.49:0/126999227 reap_dead start 2023-11-06T02:08:28.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.727+0000 7fe3fed9d640 1 --2- 172.21.15.49:0/126999227 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3e8005190 0x7fe3e800b990 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe3f8001780 tx=0x7fe3f801c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9989e8d4740473f6 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:28.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.727+0000 7fe3fd59a640 1 -- 172.21.15.49:0/126999227 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f80052d0 con 0x7fe3e8005190 2023-11-06T02:08:28.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.727+0000 7fe3fd59a640 1 -- 172.21.15.49:0/126999227 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3f80a7050 con 0x7fe3e8005190 2023-11-06T02:08:28.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.727+0000 7fe3fd59a640 1 -- 172.21.15.49:0/126999227 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f8003840 con 0x7fe3e8005190 2023-11-06T02:08:28.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.727+0000 7fe40480f640 1 -- 172.21.15.49:0/126999227 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3e8005190 msgr2=0x7fe3e800b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:28.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.727+0000 7fe40480f640 1 --2- 172.21.15.49:0/126999227 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3e8005190 0x7fe3e800b990 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe3f8001780 tx=0x7fe3f801c400 comp rx=0 tx=0).stop 2023-11-06T02:08:28.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.727+0000 7fe40480f640 1 -- 172.21.15.49:0/126999227 shutdown_connections 2023-11-06T02:08:28.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.727+0000 7fe40480f640 1 --2- 172.21.15.49:0/126999227 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3e8005190 0x7fe3e800b990 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:28.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.727+0000 7fe40480f640 1 -- 172.21.15.49:0/126999227 >> 172.21.15.49:0/126999227 conn(0x7fe4000cd2d0 msgr2=0x7fe4000cd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:28.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.727+0000 7fe40480f640 1 -- 172.21.15.49:0/126999227 shutdown_connections 2023-11-06T02:08:28.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.727+0000 7fe40480f640 1 -- 172.21.15.49:0/126999227 wait complete. 2023-11-06T02:08:28.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.727+0000 7fe40480f640 1 Processor -- start 2023-11-06T02:08:28.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.727+0000 7fe40480f640 1 -- start start 2023-11-06T02:08:28.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.731+0000 7fe40480f640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe3e8005190 0x7fe4001527e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:28.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.731+0000 7fe40480f640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe400151a30 0x7fe4000c8350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:28.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.731+0000 7fe40480f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe400158e00 0x7fe400159280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:28.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.731+0000 7fe40480f640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe400153a90 con 0x7fe400158e00 2023-11-06T02:08:28.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.731+0000 7fe40480f640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4001597c0 con 0x7fe400151a30 2023-11-06T02:08:28.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.731+0000 7fe40480f640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe400151e90 con 0x7fe3e8005190 2023-11-06T02:08:28.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.731+0000 7fe3fed9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe400158e00 0x7fe400159280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:28.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.731+0000 7fe3fed9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe400158e00 0x7fe400159280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47270/0 (socket says 172.21.15.49:47270) 2023-11-06T02:08:28.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.731+0000 7fe3fed9d640 1 -- 172.21.15.49:0/649019454 learned_addr learned my addr 172.21.15.49:0/649019454 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:28.736 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.731+0000 7fe3fdd9b640 1 --2- 172.21.15.49:0/649019454 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe400151a30 0x7fe4000c8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:28.736 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.731+0000 7fe3fe59c640 1 --2- 172.21.15.49:0/649019454 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe3e8005190 0x7fe4001527e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:28.736 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.731+0000 7fe3fed9d640 1 -- 172.21.15.49:0/649019454 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe3e8005190 msgr2=0x7fe4001527e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:28.736 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.731+0000 7fe3fed9d640 1 --2- 172.21.15.49:0/649019454 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe3e8005190 0x7fe4001527e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:28.736 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.731+0000 7fe3fed9d640 1 -- 172.21.15.49:0/649019454 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe400151a30 msgr2=0x7fe4000c8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:28.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.731+0000 7fe3fed9d640 1 --2- 172.21.15.49:0/649019454 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe400151a30 0x7fe4000c8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:28.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.731+0000 7fe3fed9d640 1 -- 172.21.15.49:0/649019454 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe400152e10 con 0x7fe400158e00 2023-11-06T02:08:28.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.731+0000 7fe3fed9d640 1 --2- 172.21.15.49:0/649019454 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe400158e00 0x7fe400159280 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe3f801cca0 tx=0x7fe3e8005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:28.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.731+0000 7fe3eeffd640 1 -- 172.21.15.49:0/649019454 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f80b0070 con 0x7fe400158e00 2023-11-06T02:08:28.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.731+0000 7fe3eeffd640 1 -- 172.21.15.49:0/649019454 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3f8007a40 con 0x7fe400158e00 2023-11-06T02:08:28.738 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.731+0000 7fe40480f640 1 -- 172.21.15.49:0/649019454 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4001530f0 con 0x7fe400158e00 2023-11-06T02:08:28.738 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.731+0000 7fe3eeffd640 1 -- 172.21.15.49:0/649019454 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f8004e70 con 0x7fe400158e00 2023-11-06T02:08:28.738 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.731+0000 7fe40480f640 1 -- 172.21.15.49:0/649019454 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4001536b0 con 0x7fe400158e00 2023-11-06T02:08:28.738 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.731+0000 7fe40480f640 1 -- 172.21.15.49:0/649019454 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe400153db0 con 0x7fe400158e00 2023-11-06T02:08:28.738 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.731+0000 7fe3eeffd640 1 -- 172.21.15.49:0/649019454 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fe3f80ab3f0 con 0x7fe400158e00 2023-11-06T02:08:28.739 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.731+0000 7fe3eeffd640 1 --2- 172.21.15.49:0/649019454 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe3e0065a90 0x7fe3e0067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:28.739 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.731+0000 7fe3eeffd640 1 -- 172.21.15.49:0/649019454 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fe3f811c040 con 0x7fe400158e00 2023-11-06T02:08:28.739 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.731+0000 7fe3fe59c640 1 --2- 172.21.15.49:0/649019454 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe3e0065a90 0x7fe3e0067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:28.739 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.731+0000 7fe3fe59c640 1 --2- 172.21.15.49:0/649019454 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe3e0065a90 0x7fe3e0067f50 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe3f0002240 tx=0x7fe3f0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:28.740 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.735+0000 7fe3eeffd640 1 -- 172.21.15.49:0/649019454 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fe3f80e9bf0 con 0x7fe400158e00 2023-11-06T02:08:28.890 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:28.891+0000 7fe40480f640 1 -- 172.21.15.49:0/649019454 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fe4000b31e0 con 0x7fe400158e00 2023-11-06T02:08:29.181 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.183+0000 7fe3eeffd640 1 -- 172.21.15.49:0/649019454 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v20) v1 ==== 138+0+0 (secure 0 0 0) 0x7fe3f80b9020 con 0x7fe400158e00 2023-11-06T02:08:29.181 INFO:teuthology.orchestra.run.smithi049.stderr:pool 'cephfs_metadata' created 2023-11-06T02:08:29.190 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.187+0000 7fe40480f640 1 -- 172.21.15.49:0/649019454 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe3e0065a90 msgr2=0x7fe3e0067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:29.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.187+0000 7fe40480f640 1 --2- 172.21.15.49:0/649019454 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe3e0065a90 0x7fe3e0067f50 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe3f0002240 tx=0x7fe3f0009040 comp rx=0 tx=0).stop 2023-11-06T02:08:29.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.187+0000 7fe40480f640 1 -- 172.21.15.49:0/649019454 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe400158e00 msgr2=0x7fe400159280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:29.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.187+0000 7fe40480f640 1 --2- 172.21.15.49:0/649019454 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe400158e00 0x7fe400159280 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe3f801cca0 tx=0x7fe3e8005800 comp rx=0 tx=0).stop 2023-11-06T02:08:29.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.187+0000 7fe40480f640 1 -- 172.21.15.49:0/649019454 shutdown_connections 2023-11-06T02:08:29.192 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.187+0000 7fe40480f640 1 --2- 172.21.15.49:0/649019454 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe3e0065a90 0x7fe3e0067f50 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:29.192 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.187+0000 7fe40480f640 1 --2- 172.21.15.49:0/649019454 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe400158e00 0x7fe400159280 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:29.192 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.187+0000 7fe40480f640 1 --2- 172.21.15.49:0/649019454 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe400151a30 0x7fe4000c8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:29.192 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.187+0000 7fe40480f640 1 --2- 172.21.15.49:0/649019454 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe3e8005190 0x7fe4001527e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:29.192 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.187+0000 7fe40480f640 1 -- 172.21.15.49:0/649019454 >> 172.21.15.49:0/649019454 conn(0x7fe4000cd2d0 msgr2=0x7fe4000cbc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:29.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.191+0000 7fe40480f640 1 -- 172.21.15.49:0/649019454 shutdown_connections 2023-11-06T02:08:29.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.191+0000 7fe40480f640 1 -- 172.21.15.49:0/649019454 wait complete. 2023-11-06T02:08:29.211 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-06T02:08:29.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.343+0000 7f07bc8f4640 1 Processor -- start 2023-11-06T02:08:29.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.343+0000 7f07bc8f4640 1 -- start start 2023-11-06T02:08:29.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.343+0000 7f07bc8f4640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f07b80d2a60 0x7f07b80d2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:29.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.343+0000 7f07bc8f4640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07b815cf90 con 0x7f07b815cbb0 2023-11-06T02:08:29.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.347+0000 7f07bc8f4640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07b80d0b80 con 0x7f07b80b5d40 2023-11-06T02:08:29.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.347+0000 7f07bc8f4640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07b80d3410 con 0x7f07b80d2a60 2023-11-06T02:08:29.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.347+0000 7f07b659c640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f07b815cbb0 0x7f07b80d1fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:44938/0 (socket says 172.21.15.49:44938) 2023-11-06T02:08:29.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.347+0000 7f07b659c640 1 -- 172.21.15.49:0/574919290 learned_addr learned my addr 172.21.15.49:0/574919290 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:08:29.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.347+0000 7f07b5d9b640 1 --2- 172.21.15.49:0/574919290 >> v2:172.21.15.150:3300/0 conn(0x7f07b80d2a60 0x7f07b80d2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:29.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.347+0000 7f07b559a640 1 -- 172.21.15.49:0/574919290 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3923107392 0 0) 0x7f07b815cf90 con 0x7f07b815cbb0 2023-11-06T02:08:29.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.347+0000 7f07b559a640 1 -- 172.21.15.49:0/574919290 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07a0003530 con 0x7f07b815cbb0 2023-11-06T02:08:29.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.347+0000 7f07b559a640 1 -- 172.21.15.49:0/574919290 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 231895593 0 0) 0x7f07a0003530 con 0x7f07b815cbb0 2023-11-06T02:08:29.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.347+0000 7f07b559a640 1 -- 172.21.15.49:0/574919290 >> v2:172.21.15.150:3300/0 conn(0x7f07b80d2a60 msgr2=0x7f07b80d2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:29.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.347+0000 7f07b559a640 1 --2- 172.21.15.49:0/574919290 >> v2:172.21.15.150:3300/0 conn(0x7f07b80d2a60 0x7f07b80d2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:29.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.347+0000 7f07b559a640 1 -- 172.21.15.49:0/574919290 >> v1:172.21.15.150:6789/0 conn(0x7f07b80b5d40 legacy=0x7f07b80b61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:29.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.347+0000 7f07b559a640 1 -- 172.21.15.49:0/574919290 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07b80d0e60 con 0x7f07b815cbb0 2023-11-06T02:08:29.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.347+0000 7f07b559a640 1 -- 172.21.15.49:0/574919290 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7f07ac001680 con 0x7f07b815cbb0 2023-11-06T02:08:29.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.347+0000 7f07b559a640 1 -- 172.21.15.49:0/574919290 >> v1:172.21.15.49:6789/0 conn(0x7f07b815cbb0 legacy=0x7f07b80d1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:29.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.347+0000 7f07b559a640 1 --2- 172.21.15.49:0/574919290 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f07a0003980 0x7f07a0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:29.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.347+0000 7f07b559a640 1 --2- 172.21.15.49:0/574919290 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07a00047d0 0x7f07a0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:29.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.347+0000 7f07b559a640 1 --2- 172.21.15.49:0/574919290 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f07a0005190 0x7f07a000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:29.350 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.347+0000 7f07b559a640 1 -- 172.21.15.49:0/574919290 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07b80d3410 con 0x7f07a00047d0 2023-11-06T02:08:29.350 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.347+0000 7f07b559a640 1 -- 172.21.15.49:0/574919290 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a000c040 con 0x7f07a0003980 2023-11-06T02:08:29.350 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.347+0000 7f07b559a640 1 -- 172.21.15.49:0/574919290 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07a000c200 con 0x7f07a0005190 2023-11-06T02:08:29.350 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.347+0000 7f07b5d9b640 1 --2- 172.21.15.49:0/574919290 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07a00047d0 0x7f07a0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:29.350 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.347+0000 7f07b659c640 1 --2- 172.21.15.49:0/574919290 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f07a0003980 0x7f07a0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:29.350 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.347+0000 7f07b6d9d640 1 --2- 172.21.15.49:0/574919290 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f07a0005190 0x7f07a000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:29.351 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.347+0000 7f07b5d9b640 1 -- 172.21.15.49:0/574919290 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f07a0005190 msgr2=0x7f07a000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:29.351 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.347+0000 7f07b5d9b640 1 --2- 172.21.15.49:0/574919290 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f07a0005190 0x7f07a000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:29.351 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.347+0000 7f07b5d9b640 1 -- 172.21.15.49:0/574919290 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f07a0003980 msgr2=0x7f07a0003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:29.351 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.347+0000 7f07b5d9b640 1 --2- 172.21.15.49:0/574919290 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f07a0003980 0x7f07a0003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:29.351 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.347+0000 7f07b5d9b640 1 -- 172.21.15.49:0/574919290 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07a0003530 con 0x7f07a00047d0 2023-11-06T02:08:29.351 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.347+0000 7f07b6d9d640 1 --2- 172.21.15.49:0/574919290 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f07a0005190 0x7f07a000b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:08:29.352 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.347+0000 7f07b6d9d640 1 -- 172.21.15.49:0/574919290 reap_dead start 2023-11-06T02:08:29.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.347+0000 7f07b5d9b640 1 --2- 172.21.15.49:0/574919290 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07a00047d0 0x7f07a0004c50 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f07a8009ae0 tx=0x7f07a801ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff2f057b5369a017 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:29.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.347+0000 7f07b559a640 1 -- 172.21.15.49:0/574919290 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a80a56d0 con 0x7f07a00047d0 2023-11-06T02:08:29.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.347+0000 7f07b559a640 1 -- 172.21.15.49:0/574919290 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07a80a5830 con 0x7f07a00047d0 2023-11-06T02:08:29.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.347+0000 7f07b559a640 1 -- 172.21.15.49:0/574919290 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a80a5be0 con 0x7f07a00047d0 2023-11-06T02:08:29.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.351+0000 7f07bc8f4640 1 -- 172.21.15.49:0/574919290 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07a00047d0 msgr2=0x7f07a0004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:29.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.351+0000 7f07bc8f4640 1 --2- 172.21.15.49:0/574919290 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07a00047d0 0x7f07a0004c50 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f07a8009ae0 tx=0x7f07a801ca50 comp rx=0 tx=0).stop 2023-11-06T02:08:29.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.351+0000 7f07bc8f4640 1 -- 172.21.15.49:0/574919290 shutdown_connections 2023-11-06T02:08:29.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.351+0000 7f07bc8f4640 1 --2- 172.21.15.49:0/574919290 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07a00047d0 0x7f07a0004c50 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:29.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.351+0000 7f07bc8f4640 1 -- 172.21.15.49:0/574919290 >> 172.21.15.49:0/574919290 conn(0x7f07b80718a0 msgr2=0x7f07b8071cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:29.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.351+0000 7f07bc8f4640 1 -- 172.21.15.49:0/574919290 shutdown_connections 2023-11-06T02:08:29.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.351+0000 7f07bc8f4640 1 -- 172.21.15.49:0/574919290 wait complete. 2023-11-06T02:08:29.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.351+0000 7f07bc8f4640 1 Processor -- start 2023-11-06T02:08:29.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.351+0000 7f07bc8f4640 1 -- start start 2023-11-06T02:08:29.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.351+0000 7f07bc8f4640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f07a00047d0 0x7f07b8153b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:29.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.351+0000 7f07bc8f4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07b81f3660 0x7f07b8158560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:29.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.351+0000 7f07bc8f4640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f07b8158aa0 0x7f07b80b6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:29.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.351+0000 7f07bc8f4640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07b8154040 con 0x7f07b81f3660 2023-11-06T02:08:29.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.351+0000 7f07bc8f4640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07b80c9590 con 0x7f07b8158aa0 2023-11-06T02:08:29.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.351+0000 7f07bc8f4640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07b81f3ae0 con 0x7f07a00047d0 2023-11-06T02:08:29.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.351+0000 7f07b5d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07b81f3660 0x7f07b8158560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:29.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.351+0000 7f07b5d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07b81f3660 0x7f07b8158560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47280/0 (socket says 172.21.15.49:47280) 2023-11-06T02:08:29.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.351+0000 7f07b5d9b640 1 -- 172.21.15.49:0/2292122019 learned_addr learned my addr 172.21.15.49:0/2292122019 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:29.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.351+0000 7f07b6d9d640 1 --2- 172.21.15.49:0/2292122019 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f07b8158aa0 0x7f07b80b6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:29.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.351+0000 7f07b659c640 1 --2- 172.21.15.49:0/2292122019 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f07a00047d0 0x7f07b8153b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:29.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.351+0000 7f07b5d9b640 1 -- 172.21.15.49:0/2292122019 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f07a00047d0 msgr2=0x7f07b8153b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:29.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.351+0000 7f07b5d9b640 1 --2- 172.21.15.49:0/2292122019 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f07a00047d0 0x7f07b8153b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:29.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.351+0000 7f07b5d9b640 1 -- 172.21.15.49:0/2292122019 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f07b8158aa0 msgr2=0x7f07b80b6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:29.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.351+0000 7f07b5d9b640 1 --2- 172.21.15.49:0/2292122019 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f07b8158aa0 0x7f07b80b6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:29.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.351+0000 7f07b5d9b640 1 -- 172.21.15.49:0/2292122019 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07b80c86a0 con 0x7f07b81f3660 2023-11-06T02:08:29.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.351+0000 7f07b5d9b640 1 --2- 172.21.15.49:0/2292122019 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07b81f3660 0x7f07b8158560 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f07a80b0040 tx=0x7f07a80197f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:29.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.351+0000 7f07a6ffd640 1 -- 172.21.15.49:0/2292122019 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a8004220 con 0x7f07b81f3660 2023-11-06T02:08:29.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.351+0000 7f07a6ffd640 1 -- 172.21.15.49:0/2292122019 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07a8004380 con 0x7f07b81f3660 2023-11-06T02:08:29.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.351+0000 7f07bc8f4640 1 -- 172.21.15.49:0/2292122019 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07b80c8980 con 0x7f07b81f3660 2023-11-06T02:08:29.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.351+0000 7f07a6ffd640 1 -- 172.21.15.49:0/2292122019 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a80b6970 con 0x7f07b81f3660 2023-11-06T02:08:29.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.351+0000 7f07bc8f4640 1 -- 172.21.15.49:0/2292122019 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07b80c8f40 con 0x7f07b81f3660 2023-11-06T02:08:29.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.355+0000 7f07bc8f4640 1 -- 172.21.15.49:0/2292122019 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07b8074040 con 0x7f07b81f3660 2023-11-06T02:08:29.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.355+0000 7f07a6ffd640 1 -- 172.21.15.49:0/2292122019 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f07a80b4070 con 0x7f07b81f3660 2023-11-06T02:08:29.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.355+0000 7f07a6ffd640 1 --2- 172.21.15.49:0/2292122019 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0798065a60 0x7f0798067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:29.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.355+0000 7f07a6ffd640 1 -- 172.21.15.49:0/2292122019 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5580+0+0 (secure 0 0 0) 0x7f07a811d3b0 con 0x7f07b81f3660 2023-11-06T02:08:29.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.355+0000 7f07b659c640 1 --2- 172.21.15.49:0/2292122019 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0798065a60 0x7f0798067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:29.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.355+0000 7f07b659c640 1 --2- 172.21.15.49:0/2292122019 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0798065a60 0x7f0798067f20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f07ac001be0 tx=0x7f07ac009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:29.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.359+0000 7f07a6ffd640 1 -- 172.21.15.49:0/2292122019 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f07a80eadf0 con 0x7f07b81f3660 2023-11-06T02:08:29.516 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:29.515+0000 7f07bc8f4640 1 -- 172.21.15.49:0/2292122019 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f07b80c63b0 con 0x7f07b81f3660 2023-11-06T02:08:30.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.187+0000 7f07a6ffd640 1 -- 172.21.15.49:0/2292122019 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v21) v1 ==== 170+0+0 (secure 0 0 0) 0x7f07a80044f0 con 0x7f07b81f3660 2023-11-06T02:08:30.188 INFO:teuthology.orchestra.run.smithi049.stderr:pool 'cephfs_data' created 2023-11-06T02:08:30.211 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.211+0000 7f07bc8f4640 1 -- 172.21.15.49:0/2292122019 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0798065a60 msgr2=0x7f0798067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:30.211 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.211+0000 7f07bc8f4640 1 --2- 172.21.15.49:0/2292122019 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0798065a60 0x7f0798067f20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f07ac001be0 tx=0x7f07ac009040 comp rx=0 tx=0).stop 2023-11-06T02:08:30.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.211+0000 7f07bc8f4640 1 -- 172.21.15.49:0/2292122019 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07b81f3660 msgr2=0x7f07b8158560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:30.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.211+0000 7f07bc8f4640 1 --2- 172.21.15.49:0/2292122019 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07b81f3660 0x7f07b8158560 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f07a80b0040 tx=0x7f07a80197f0 comp rx=0 tx=0).stop 2023-11-06T02:08:30.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.211+0000 7f07bc8f4640 1 -- 172.21.15.49:0/2292122019 shutdown_connections 2023-11-06T02:08:30.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.211+0000 7f07bc8f4640 1 --2- 172.21.15.49:0/2292122019 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0798065a60 0x7f0798067f20 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:30.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.211+0000 7f07bc8f4640 1 --2- 172.21.15.49:0/2292122019 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07b81f3660 0x7f07b8158560 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:30.213 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.211+0000 7f07bc8f4640 1 --2- 172.21.15.49:0/2292122019 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f07b8158aa0 0x7f07b80b6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:30.213 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.211+0000 7f07bc8f4640 1 --2- 172.21.15.49:0/2292122019 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f07a00047d0 0x7f07b8153b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:30.213 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.211+0000 7f07bc8f4640 1 -- 172.21.15.49:0/2292122019 >> 172.21.15.49:0/2292122019 conn(0x7f07b80718a0 msgr2=0x7f07b815f1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:30.214 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.215+0000 7f07bc8f4640 1 -- 172.21.15.49:0/2292122019 shutdown_connections 2023-11-06T02:08:30.214 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.215+0000 7f07bc8f4640 1 -- 172.21.15.49:0/2292122019 wait complete. 2023-11-06T02:08:30.242 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-06T02:08:30.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.375+0000 7ff4b915d640 1 Processor -- start 2023-11-06T02:08:30.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.375+0000 7ff4b915d640 1 -- start start 2023-11-06T02:08:30.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.375+0000 7ff4b915d640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7ff4b415ab60 0x7ff4b40ad520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:30.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.375+0000 7ff4b915d640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff4b40ada60 0x7ff4b40d08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:30.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.375+0000 7ff4b915d640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4b40d1240 con 0x7ff4b40d0df0 2023-11-06T02:08:30.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.375+0000 7ff4b915d640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4b40cb340 con 0x7ff4b40ada60 2023-11-06T02:08:30.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.375+0000 7ff4b915d640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4b40adf50 con 0x7ff4b415ab60 2023-11-06T02:08:30.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.375+0000 7ff4abfff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff4b40ada60 0x7ff4b40d08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:30.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.375+0000 7ff4abfff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff4b40ada60 0x7ff4b40d08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47282/0 (socket says 172.21.15.49:47282) 2023-11-06T02:08:30.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.375+0000 7ff4abfff640 1 -- 172.21.15.49:0/2963301248 learned_addr learned my addr 172.21.15.49:0/2963301248 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:30.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.375+0000 7ff4b2d9d640 1 --2- 172.21.15.49:0/2963301248 >> v2:172.21.15.150:3300/0 conn(0x7ff4b415ab60 0x7ff4b40ad520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:30.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.375+0000 7ff4abfff640 1 -- 172.21.15.49:0/2963301248 >> v2:172.21.15.150:3300/0 conn(0x7ff4b415ab60 msgr2=0x7ff4b40ad520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:30.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.375+0000 7ff4abfff640 1 --2- 172.21.15.49:0/2963301248 >> v2:172.21.15.150:3300/0 conn(0x7ff4b415ab60 0x7ff4b40ad520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:30.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.375+0000 7ff4abfff640 1 -- 172.21.15.49:0/2963301248 >> v1:172.21.15.150:6789/0 conn(0x7ff4b40d0df0 legacy=0x7ff4b40b08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:30.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.375+0000 7ff4abfff640 1 -- 172.21.15.49:0/2963301248 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4b40ae230 con 0x7ff4b40ada60 2023-11-06T02:08:30.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.375+0000 7ff4b2d9d640 1 --2- 172.21.15.49:0/2963301248 >> v2:172.21.15.150:3300/0 conn(0x7ff4b415ab60 0x7ff4b40ad520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:08:30.378 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.379+0000 7ff4abfff640 1 --2- 172.21.15.49:0/2963301248 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4b40ada60 0x7ff4b40d08b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff4b40ce420 tx=0x7ff49c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c65cf126e647c496 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:30.378 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.379+0000 7ff4b259c640 1 -- 172.21.15.49:0/2963301248 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49c0aa5f0 con 0x7ff4b40ada60 2023-11-06T02:08:30.378 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.379+0000 7ff4b259c640 1 -- 172.21.15.49:0/2963301248 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff49c0aa750 con 0x7ff4b40ada60 2023-11-06T02:08:30.379 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.379+0000 7ff4b259c640 1 -- 172.21.15.49:0/2963301248 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49c0aaae0 con 0x7ff4b40ada60 2023-11-06T02:08:30.379 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.379+0000 7ff4b915d640 1 -- 172.21.15.49:0/2963301248 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4b40ada60 msgr2=0x7ff4b40d08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:30.379 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.379+0000 7ff4b915d640 1 --2- 172.21.15.49:0/2963301248 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4b40ada60 0x7ff4b40d08b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff4b40ce420 tx=0x7ff49c01c490 comp rx=0 tx=0).stop 2023-11-06T02:08:30.379 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.379+0000 7ff4b915d640 1 -- 172.21.15.49:0/2963301248 shutdown_connections 2023-11-06T02:08:30.379 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.379+0000 7ff4b915d640 1 --2- 172.21.15.49:0/2963301248 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4b40ada60 0x7ff4b40d08b0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:30.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.379+0000 7ff4b915d640 1 --2- 172.21.15.49:0/2963301248 >> v2:172.21.15.150:3300/0 conn(0x7ff4b415ab60 0x7ff4b40ad520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:30.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.379+0000 7ff4b915d640 1 -- 172.21.15.49:0/2963301248 >> 172.21.15.49:0/2963301248 conn(0x7ff4b4089fd0 msgr2=0x7ff4b4099c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:30.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.379+0000 7ff4b915d640 1 -- 172.21.15.49:0/2963301248 shutdown_connections 2023-11-06T02:08:30.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.379+0000 7ff4b915d640 1 -- 172.21.15.49:0/2963301248 wait complete. 2023-11-06T02:08:30.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.379+0000 7ff4b915d640 1 Processor -- start 2023-11-06T02:08:30.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.379+0000 7ff4b915d640 1 -- start start 2023-11-06T02:08:30.383 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.379+0000 7ff4b915d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4b40ada60 0x7ff4b4159a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:30.383 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.379+0000 7ff4b915d640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4b40d0df0 0x7ff4b415b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:30.383 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.379+0000 7ff4b915d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4b406fff0 0x7ff4b415b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:30.383 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.379+0000 7ff4b915d640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b406fe70 con 0x7ff4b406fff0 2023-11-06T02:08:30.384 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.379+0000 7ff4b915d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b40b3610 con 0x7ff4b40ada60 2023-11-06T02:08:30.384 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.379+0000 7ff4b915d640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b415d470 con 0x7ff4b40d0df0 2023-11-06T02:08:30.384 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.379+0000 7ff4b359e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4b406fff0 0x7ff4b415b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:30.384 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.379+0000 7ff4b359e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4b406fff0 0x7ff4b415b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47292/0 (socket says 172.21.15.49:47292) 2023-11-06T02:08:30.384 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.379+0000 7ff4b359e640 1 -- 172.21.15.49:0/3650573687 learned_addr learned my addr 172.21.15.49:0/3650573687 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:30.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.379+0000 7ff4b2d9d640 1 --2- 172.21.15.49:0/3650573687 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4b40ada60 0x7ff4b4159a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:30.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.379+0000 7ff4abfff640 1 --2- 172.21.15.49:0/3650573687 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4b40d0df0 0x7ff4b415b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:30.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.379+0000 7ff4b359e640 1 -- 172.21.15.49:0/3650573687 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4b40d0df0 msgr2=0x7ff4b415b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:30.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.379+0000 7ff4b359e640 1 --2- 172.21.15.49:0/3650573687 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4b40d0df0 0x7ff4b415b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:30.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.379+0000 7ff4b359e640 1 -- 172.21.15.49:0/3650573687 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4b40ada60 msgr2=0x7ff4b4159a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:30.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.379+0000 7ff4b359e640 1 --2- 172.21.15.49:0/3650573687 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4b40ada60 0x7ff4b4159a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:30.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.379+0000 7ff4b359e640 1 -- 172.21.15.49:0/3650573687 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff49c0094d0 con 0x7ff4b406fff0 2023-11-06T02:08:30.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.379+0000 7ff4b2d9d640 1 --2- 172.21.15.49:0/3650573687 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4b40ada60 0x7ff4b4159a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:08:30.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.379+0000 7ff4abfff640 1 --2- 172.21.15.49:0/3650573687 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4b40d0df0 0x7ff4b415b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:08:30.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.379+0000 7ff4b359e640 1 --2- 172.21.15.49:0/3650573687 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4b406fff0 0x7ff4b415b7e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac00c7b0 tx=0x7ff4ac00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:30.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.379+0000 7ff4ab7fe640 1 -- 172.21.15.49:0/3650573687 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4ac004b70 con 0x7ff4b406fff0 2023-11-06T02:08:30.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.379+0000 7ff4ab7fe640 1 -- 172.21.15.49:0/3650573687 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4ac004cd0 con 0x7ff4b406fff0 2023-11-06T02:08:30.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.379+0000 7ff4b915d640 1 -- 172.21.15.49:0/3650573687 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4b415d750 con 0x7ff4b406fff0 2023-11-06T02:08:30.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.383+0000 7ff4ab7fe640 1 -- 172.21.15.49:0/3650573687 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4ac00aa90 con 0x7ff4b406fff0 2023-11-06T02:08:30.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.383+0000 7ff4b915d640 1 -- 172.21.15.49:0/3650573687 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4b415dd10 con 0x7ff4b406fff0 2023-11-06T02:08:30.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.383+0000 7ff4b915d640 1 -- 172.21.15.49:0/3650573687 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4b40b37c0 con 0x7ff4b406fff0 2023-11-06T02:08:30.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.383+0000 7ff4ab7fe640 1 -- 172.21.15.49:0/3650573687 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7ff4ac00abf0 con 0x7ff4b406fff0 2023-11-06T02:08:30.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.383+0000 7ff4ab7fe640 1 --2- 172.21.15.49:0/3650573687 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff494065a60 0x7ff494067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:30.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.383+0000 7ff4ab7fe640 1 -- 172.21.15.49:0/3650573687 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5962+0+0 (secure 0 0 0) 0x7ff4ac083510 con 0x7ff4b406fff0 2023-11-06T02:08:30.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.383+0000 7ff4b2d9d640 1 --2- 172.21.15.49:0/3650573687 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff494065a60 0x7ff494067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:30.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.383+0000 7ff4b2d9d640 1 --2- 172.21.15.49:0/3650573687 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff494065a60 0x7ff494067f20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff4a4008890 tx=0x7ff4a4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:30.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.395+0000 7ff4ab7fe640 1 -- 172.21.15.49:0/3650573687 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7ff4b40b37c0 con 0x7ff4b406fff0 2023-11-06T02:08:30.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.583+0000 7ff4b915d640 1 -- 172.21.15.49:0/3650573687 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7ff4b40b37c0 con 0x7ff4b406fff0 2023-11-06T02:08:30.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.595+0000 7ff4ab7fe640 1 -- 172.21.15.49:0/3650573687 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 3 and data pool 4 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7ff4b40b37c0 con 0x7ff4b406fff0 2023-11-06T02:08:30.593 INFO:teuthology.orchestra.run.smithi049.stderr:new fs with metadata pool 3 and data pool 4 2023-11-06T02:08:30.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.611+0000 7ff4b915d640 1 -- 172.21.15.49:0/3650573687 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff494065a60 msgr2=0x7ff494067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:30.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.611+0000 7ff4b915d640 1 --2- 172.21.15.49:0/3650573687 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff494065a60 0x7ff494067f20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff4a4008890 tx=0x7ff4a4008040 comp rx=0 tx=0).stop 2023-11-06T02:08:30.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.611+0000 7ff4b915d640 1 -- 172.21.15.49:0/3650573687 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4b406fff0 msgr2=0x7ff4b415b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:30.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.611+0000 7ff4b915d640 1 --2- 172.21.15.49:0/3650573687 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4b406fff0 0x7ff4b415b7e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac00c7b0 tx=0x7ff4ac00cc70 comp rx=0 tx=0).stop 2023-11-06T02:08:30.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.611+0000 7ff4b915d640 1 -- 172.21.15.49:0/3650573687 shutdown_connections 2023-11-06T02:08:30.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.611+0000 7ff4b915d640 1 --2- 172.21.15.49:0/3650573687 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff494065a60 0x7ff494067f20 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:30.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.611+0000 7ff4b915d640 1 --2- 172.21.15.49:0/3650573687 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4b406fff0 0x7ff4b415b7e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:30.612 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.611+0000 7ff4b915d640 1 --2- 172.21.15.49:0/3650573687 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4b40d0df0 0x7ff4b415b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:30.612 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.611+0000 7ff4b915d640 1 --2- 172.21.15.49:0/3650573687 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4b40ada60 0x7ff4b4159a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:30.612 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.611+0000 7ff4b915d640 1 -- 172.21.15.49:0/3650573687 >> 172.21.15.49:0/3650573687 conn(0x7ff4b4089fd0 msgr2=0x7ff4b406b360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:30.613 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.615+0000 7ff4b915d640 1 -- 172.21.15.49:0/3650573687 shutdown_connections 2023-11-06T02:08:30.613 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.615+0000 7ff4b915d640 1 -- 172.21.15.49:0/3650573687 wait complete. 2023-11-06T02:08:30.637 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T02:08:30.753 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.755+0000 7fe5b0834640 1 Processor -- start 2023-11-06T02:08:30.753 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.755+0000 7fe5b0834640 1 -- start start 2023-11-06T02:08:30.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.755+0000 7fe5b0834640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe5ac0b6100 0x7fe5ac0cb960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:30.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.755+0000 7fe5b0834640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe5ac15e440 0x7fe5ac15b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:30.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.755+0000 7fe5b0834640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5ac15f3c0 con 0x7fe5ac0cb580 2023-11-06T02:08:30.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.755+0000 7fe5b0834640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5ac15e8d0 con 0x7fe5ac0b6100 2023-11-06T02:08:30.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.755+0000 7fe5b0834640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5ac15eb20 con 0x7fe5ac15e440 2023-11-06T02:08:30.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.755+0000 7fe5a9d9b640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe5ac0b6100 0x7fe5ac0cb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:30.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.755+0000 7fe5a9d9b640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe5ac0b6100 0x7fe5ac0cb960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47300/0 (socket says 172.21.15.49:47300) 2023-11-06T02:08:30.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.755+0000 7fe5a9d9b640 1 -- 172.21.15.49:0/2640412334 learned_addr learned my addr 172.21.15.49:0/2640412334 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:30.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.755+0000 7fe5aad9d640 1 --2- 172.21.15.49:0/2640412334 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe5ac15e440 0x7fe5ac15b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:30.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.755+0000 7fe5a9d9b640 1 -- 172.21.15.49:0/2640412334 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe5ac15e440 msgr2=0x7fe5ac15b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:30.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.755+0000 7fe5a9d9b640 1 --2- 172.21.15.49:0/2640412334 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe5ac15e440 0x7fe5ac15b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:30.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.755+0000 7fe5a9d9b640 1 -- 172.21.15.49:0/2640412334 >> v1:172.21.15.150:6789/0 conn(0x7fe5ac0cb580 legacy=0x7fe5ac0ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:30.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.755+0000 7fe5a9d9b640 1 -- 172.21.15.49:0/2640412334 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5ac15ee00 con 0x7fe5ac0b6100 2023-11-06T02:08:30.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.755+0000 7fe5aad9d640 1 --2- 172.21.15.49:0/2640412334 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe5ac15e440 0x7fe5ac15b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:08:30.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.759+0000 7fe5a9d9b640 1 --2- 172.21.15.49:0/2640412334 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5ac0b6100 0x7fe5ac0cb960 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe59c009870 tx=0x7fe59c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f805ed26ba1d45f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:30.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.759+0000 7fe5a959a640 1 -- 172.21.15.49:0/2640412334 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe59c0ad640 con 0x7fe5ac0b6100 2023-11-06T02:08:30.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.759+0000 7fe5a959a640 1 -- 172.21.15.49:0/2640412334 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe59c0ad7a0 con 0x7fe5ac0b6100 2023-11-06T02:08:30.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.759+0000 7fe5a959a640 1 -- 172.21.15.49:0/2640412334 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe59c0adb30 con 0x7fe5ac0b6100 2023-11-06T02:08:30.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.759+0000 7fe5b0834640 1 -- 172.21.15.49:0/2640412334 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5ac0b6100 msgr2=0x7fe5ac0cb960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:30.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.759+0000 7fe5b0834640 1 --2- 172.21.15.49:0/2640412334 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5ac0b6100 0x7fe5ac0cb960 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe59c009870 tx=0x7fe59c01c480 comp rx=0 tx=0).stop 2023-11-06T02:08:30.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.759+0000 7fe5b0834640 1 -- 172.21.15.49:0/2640412334 shutdown_connections 2023-11-06T02:08:30.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.759+0000 7fe5b0834640 1 --2- 172.21.15.49:0/2640412334 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe5ac15e440 0x7fe5ac15b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:30.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.759+0000 7fe5b0834640 1 --2- 172.21.15.49:0/2640412334 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5ac0b6100 0x7fe5ac0cb960 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:30.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.759+0000 7fe5b0834640 1 -- 172.21.15.49:0/2640412334 >> 172.21.15.49:0/2640412334 conn(0x7fe5ac0cd2d0 msgr2=0x7fe5ac0cd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:30.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.759+0000 7fe5b0834640 1 -- 172.21.15.49:0/2640412334 shutdown_connections 2023-11-06T02:08:30.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.759+0000 7fe5b0834640 1 -- 172.21.15.49:0/2640412334 wait complete. 2023-11-06T02:08:30.763 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.759+0000 7fe5b0834640 1 Processor -- start 2023-11-06T02:08:30.763 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.759+0000 7fe5b0834640 1 -- start start 2023-11-06T02:08:30.764 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.759+0000 7fe5b0834640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe5ac0b6100 0x7fe5ac0c9410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:30.764 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.759+0000 7fe5b0834640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5ac0cb580 0x7fe5ac157290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:30.764 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.759+0000 7fe5b0834640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe5ac15e440 0x7fe5ac0c7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:30.764 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.759+0000 7fe5b0834640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5ac0c8f80 con 0x7fe5ac0cb580 2023-11-06T02:08:30.764 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.759+0000 7fe5b0834640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5ac1577d0 con 0x7fe5ac0b6100 2023-11-06T02:08:30.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.759+0000 7fe5b0834640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5ac0c8370 con 0x7fe5ac15e440 2023-11-06T02:08:30.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.763+0000 7fe5a9d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5ac0cb580 0x7fe5ac157290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:30.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.763+0000 7fe5a9d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5ac0cb580 0x7fe5ac157290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47302/0 (socket says 172.21.15.49:47302) 2023-11-06T02:08:30.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.763+0000 7fe5a9d9b640 1 -- 172.21.15.49:0/199702494 learned_addr learned my addr 172.21.15.49:0/199702494 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:30.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.763+0000 7fe5aa59c640 1 --2- 172.21.15.49:0/199702494 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe5ac0b6100 0x7fe5ac0c9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:30.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.763+0000 7fe5aad9d640 1 --2- 172.21.15.49:0/199702494 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe5ac15e440 0x7fe5ac0c7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:30.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.763+0000 7fe5a9d9b640 1 -- 172.21.15.49:0/199702494 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe5ac15e440 msgr2=0x7fe5ac0c7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:30.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.763+0000 7fe5a9d9b640 1 --2- 172.21.15.49:0/199702494 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe5ac15e440 0x7fe5ac0c7920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:30.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.763+0000 7fe5a9d9b640 1 -- 172.21.15.49:0/199702494 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe5ac0b6100 msgr2=0x7fe5ac0c9410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:30.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.763+0000 7fe5a9d9b640 1 --2- 172.21.15.49:0/199702494 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe5ac0b6100 0x7fe5ac0c9410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:30.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.763+0000 7fe5a9d9b640 1 -- 172.21.15.49:0/199702494 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe59c0094d0 con 0x7fe5ac0cb580 2023-11-06T02:08:30.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.763+0000 7fe5aad9d640 1 --2- 172.21.15.49:0/199702494 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe5ac15e440 0x7fe5ac0c7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:08:30.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.763+0000 7fe5a9d9b640 1 --2- 172.21.15.49:0/199702494 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5ac0cb580 0x7fe5ac157290 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe59c0b0040 tx=0x7fe59c019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:30.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.763+0000 7fe5a2ffd640 1 -- 172.21.15.49:0/199702494 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe59c0bc070 con 0x7fe5ac0cb580 2023-11-06T02:08:30.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.763+0000 7fe5a2ffd640 1 -- 172.21.15.49:0/199702494 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe59c0a6040 con 0x7fe5ac0cb580 2023-11-06T02:08:30.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.763+0000 7fe5b0834640 1 -- 172.21.15.49:0/199702494 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5ac0cecb0 con 0x7fe5ac0cb580 2023-11-06T02:08:30.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.763+0000 7fe5a2ffd640 1 -- 172.21.15.49:0/199702494 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe59c0b7850 con 0x7fe5ac0cb580 2023-11-06T02:08:30.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.763+0000 7fe5b0834640 1 -- 172.21.15.49:0/199702494 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5ac0c8930 con 0x7fe5ac0cb580 2023-11-06T02:08:30.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.763+0000 7fe5b0834640 1 -- 172.21.15.49:0/199702494 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5ac0ce170 con 0x7fe5ac0cb580 2023-11-06T02:08:30.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.763+0000 7fe5a2ffd640 1 -- 172.21.15.49:0/199702494 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fe59c0b79b0 con 0x7fe5ac0cb580 2023-11-06T02:08:30.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.763+0000 7fe5a2ffd640 1 --2- 172.21.15.49:0/199702494 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe58c065a60 0x7fe58c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:30.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.763+0000 7fe5a2ffd640 1 -- 172.21.15.49:0/199702494 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe59c11d710 con 0x7fe5ac0cb580 2023-11-06T02:08:30.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.763+0000 7fe5aa59c640 1 --2- 172.21.15.49:0/199702494 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe58c065a60 0x7fe58c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:30.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.763+0000 7fe5aa59c640 1 --2- 172.21.15.49:0/199702494 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe58c065a60 0x7fe58c067f20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe594000f70 tx=0x7fe594009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:30.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.767+0000 7fe5a2ffd640 1 -- 172.21.15.49:0/199702494 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fe59c0eaf70 con 0x7fe5ac0cb580 2023-11-06T02:08:30.917 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.919+0000 7fe5b0834640 1 -- 172.21.15.49:0/199702494 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe5ac0ccad0 con 0x7fe5ac0cb580 2023-11-06T02:08:30.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.923+0000 7fe5a2ffd640 1 -- 172.21.15.49:0/199702494 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+17824 (secure 0 0 0) 0x7fe59c0ee590 con 0x7fe5ac0cb580 2023-11-06T02:08:30.921 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:08:30.921 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":22,"fsid":"1a8b035c-6ba4-499a-b876-aea47fd9c0d4","created":"2023-11-06T02:08:06.440857+0000","modified":"2023-11-06T02:08:30.591607+0000","last_up_change":"2023-11-06T02:08:17.982372+0000","last_in_change":"2023-11-06T02:08:08.906631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T02:08:19.252222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T02:08:24.776211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-06T02:08:28.936178+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-06T02:08:29.563468+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ad55af8-0571-4e6a-8722-b5d429b7496d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6809","nonce":38116},{"type":"v1","addr":"172.21.15.49:6811","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6813","nonce":38116},{"type":"v1","addr":"172.21.15.49:6815","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6821","nonce":38116},{"type":"v1","addr":"172.21.15.49:6823","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6817","nonce":38116},{"type":"v1","addr":"172.21.15.49:6819","nonce":38116}]},"public_addr":"172.21.15.49:6811/38116","cluster_addr":"172.21.15.49:6815/38116","heartbeat_back_addr":"172.21.15.49:6823/38116","heartbeat_front_addr":"172.21.15.49:6819/38116","state":["exists","up"]},{"osd":1,"uuid":"c8b31f85-a757-43ed-9419-351acc46611b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":38113},{"type":"v1","addr":"172.21.15.49:6810","nonce":38113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":38113},{"type":"v1","addr":"172.21.15.49:6814","nonce":38113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":38113},{"type":"v1","addr":"172.21.15.49:6822","nonce":38113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":38113},{"type":"v1","addr":"172.21.15.49:6818","nonce":38113}]},"public_addr":"172.21.15.49:6810/38113","cluster_addr":"172.21.15.49:6814/38113","heartbeat_back_addr":"172.21.15.49:6822/38113","heartbeat_front_addr":"172.21.15.49:6818/38113","state":["exists","up"]},{"osd":2,"uuid":"555b3bf3-0a70-4b91-8efd-e4204b5fc26e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":38114},{"type":"v1","addr":"172.21.15.49:6801","nonce":38114}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":38114},{"type":"v1","addr":"172.21.15.49:6803","nonce":38114}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":38114},{"type":"v1","addr":"172.21.15.49:6807","nonce":38114}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":38114},{"type":"v1","addr":"172.21.15.49:6805","nonce":38114}]},"public_addr":"172.21.15.49:6801/38114","cluster_addr":"172.21.15.49:6803/38114","heartbeat_back_addr":"172.21.15.49:6807/38114","heartbeat_front_addr":"172.21.15.49:6805/38114","state":["exists","up"]},{"osd":3,"uuid":"8d4e5d77-03ef-4b9b-bfcb-5ae93a46795c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":38115},{"type":"v1","addr":"172.21.15.49:6825","nonce":38115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":38115},{"type":"v1","addr":"172.21.15.49:6827","nonce":38115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":38115},{"type":"v1","addr":"172.21.15.49:6831","nonce":38115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":38115},{"type":"v1","addr":"172.21.15.49:6829","nonce":38115}]},"public_addr":"172.21.15.49:6825/38115","cluster_addr":"172.21.15.49:6827/38115","heartbeat_back_addr":"172.21.15.49:6831/38115","heartbeat_front_addr":"172.21.15.49:6829/38115","state":["exists","up"]},{"osd":4,"uuid":"dba03ad4-68a3-4cf3-9218-ac611f02eacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":38417},{"type":"v1","addr":"172.21.15.150:6809","nonce":38417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":38417},{"type":"v1","addr":"172.21.15.150:6811","nonce":38417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":38417},{"type":"v1","addr":"172.21.15.150:6815","nonce":38417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":38417},{"type":"v1","addr":"172.21.15.150:6813","nonce":38417}]},"public_addr":"172.21.15.150:6809/38417","cluster_addr":"172.21.15.150:6811/38417","heartbeat_back_addr":"172.21.15.150:6815/38417","heartbeat_front_addr":"172.21.15.150:6813/38417","state":["exists","up"]},{"osd":5,"uuid":"0df9c303-c255-4640-8e91-01f1c3efe18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":38418},{"type":"v1","addr":"172.21.15.150:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":38418},{"type":"v1","addr":"172.21.15.150:6803","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":38418},{"type":"v1","addr":"172.21.15.150:6807","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":38418},{"type":"v1","addr":"172.21.15.150:6805","nonce":38418}]},"public_addr":"172.21.15.150:6801/38418","cluster_addr":"172.21.15.150:6803/38418","heartbeat_back_addr":"172.21.15.150:6807/38418","heartbeat_front_addr":"172.21.15.150:6805/38418","state":["exists","up"]},{"osd":6,"uuid":"e59fa189-4b70-4c06-9722-c02b90828bf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":38428},{"type":"v1","addr":"172.21.15.150:6825","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":38428},{"type":"v1","addr":"172.21.15.150:6827","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":38428},{"type":"v1","addr":"172.21.15.150:6831","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":38428},{"type":"v1","addr":"172.21.15.150:6829","nonce":38428}]},"public_addr":"172.21.15.150:6825/38428","cluster_addr":"172.21.15.150:6827/38428","heartbeat_back_addr":"172.21.15.150:6831/38428","heartbeat_front_addr":"172.21.15.150:6829/38428","state":["exists","up"]},{"osd":7,"uuid":"a64f57e1-ef98-46bd-b304-385759e95898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":38421},{"type":"v1","addr":"172.21.15.150:6817","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":38421},{"type":"v1","addr":"172.21.15.150:6819","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":38421},{"type":"v1","addr":"172.21.15.150:6823","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":38421},{"type":"v1","addr":"172.21.15.150:6821","nonce":38421}]},"public_addr":"172.21.15.150:6817/38421","cluster_addr":"172.21.15.150:6819/38421","heartbeat_back_addr":"172.21.15.150:6823/38421","heartbeat_front_addr":"172.21.15.150:6821/38421","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.438552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.413280+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.296466+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.424966+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.276953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.230984+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.410235+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.480084+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T02:08:30.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.923+0000 7fe5b0834640 1 -- 172.21.15.49:0/199702494 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe58c065a60 msgr2=0x7fe58c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:30.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.923+0000 7fe5b0834640 1 --2- 172.21.15.49:0/199702494 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe58c065a60 0x7fe58c067f20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe594000f70 tx=0x7fe594009040 comp rx=0 tx=0).stop 2023-11-06T02:08:30.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.923+0000 7fe5b0834640 1 -- 172.21.15.49:0/199702494 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5ac0cb580 msgr2=0x7fe5ac157290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:30.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.923+0000 7fe5b0834640 1 --2- 172.21.15.49:0/199702494 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5ac0cb580 0x7fe5ac157290 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe59c0b0040 tx=0x7fe59c019220 comp rx=0 tx=0).stop 2023-11-06T02:08:30.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.923+0000 7fe5b0834640 1 -- 172.21.15.49:0/199702494 shutdown_connections 2023-11-06T02:08:30.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.923+0000 7fe5b0834640 1 --2- 172.21.15.49:0/199702494 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe58c065a60 0x7fe58c067f20 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:30.924 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.923+0000 7fe5b0834640 1 --2- 172.21.15.49:0/199702494 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5ac0cb580 0x7fe5ac157290 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:30.924 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.923+0000 7fe5b0834640 1 --2- 172.21.15.49:0/199702494 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe5ac15e440 0x7fe5ac0c7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:30.924 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.923+0000 7fe5b0834640 1 --2- 172.21.15.49:0/199702494 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe5ac0b6100 0x7fe5ac0c9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:30.924 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.923+0000 7fe5b0834640 1 -- 172.21.15.49:0/199702494 >> 172.21.15.49:0/199702494 conn(0x7fe5ac0cd2d0 msgr2=0x7fe5ac0b19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:30.924 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.923+0000 7fe5b0834640 1 -- 172.21.15.49:0/199702494 shutdown_connections 2023-11-06T02:08:30.924 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:30.923+0000 7fe5b0834640 1 -- 172.21.15.49:0/199702494 wait complete. 2023-11-06T02:08:30.938 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T02:08:31.060 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.063+0000 7fc157cfe640 1 Processor -- start 2023-11-06T02:08:31.061 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.063+0000 7fc157cfe640 1 -- start start 2023-11-06T02:08:31.062 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.063+0000 7fc157cfe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc15015aae0 0x7fc150070a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:31.062 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.063+0000 7fc157cfe640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc1500b0790 0x7fc1500b1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:31.062 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.063+0000 7fc157cfe640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1500cacf0 con 0x7fc1500ca830 2023-11-06T02:08:31.063 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.063+0000 7fc157cfe640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc150070fc0 con 0x7fc15015aae0 2023-11-06T02:08:31.063 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.063+0000 7fc157cfe640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc15015aec0 con 0x7fc1500b0790 2023-11-06T02:08:31.063 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.063+0000 7fc155a9a640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc15015aae0 0x7fc150070a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:31.063 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.063+0000 7fc155a9a640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc15015aae0 0x7fc150070a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47306/0 (socket says 172.21.15.49:47306) 2023-11-06T02:08:31.063 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.063+0000 7fc155a9a640 1 -- 172.21.15.49:0/103479597 learned_addr learned my addr 172.21.15.49:0/103479597 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:31.064 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.063+0000 7fc15629b640 1 --2- 172.21.15.49:0/103479597 >> v2:172.21.15.150:3300/0 conn(0x7fc1500b0790 0x7fc1500b1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:31.064 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.063+0000 7fc155a9a640 1 -- 172.21.15.49:0/103479597 >> v2:172.21.15.150:3300/0 conn(0x7fc1500b0790 msgr2=0x7fc1500b1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:31.064 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.063+0000 7fc155a9a640 1 --2- 172.21.15.49:0/103479597 >> v2:172.21.15.150:3300/0 conn(0x7fc1500b0790 0x7fc1500b1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:31.064 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.063+0000 7fc155a9a640 1 -- 172.21.15.49:0/103479597 >> v1:172.21.15.150:6789/0 conn(0x7fc1500ca830 legacy=0x7fc1500b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:31.064 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.063+0000 7fc155a9a640 1 -- 172.21.15.49:0/103479597 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1500c8ff0 con 0x7fc15015aae0 2023-11-06T02:08:31.065 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.063+0000 7fc15629b640 1 --2- 172.21.15.49:0/103479597 >> v2:172.21.15.150:3300/0 conn(0x7fc1500b0790 0x7fc1500b1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:08:31.066 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.063+0000 7fc155a9a640 1 --2- 172.21.15.49:0/103479597 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc15015aae0 0x7fc150070a80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc1500adfd0 tx=0x7fc14c01c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=587f15201c8cdc9f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:31.066 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.063+0000 7fc154a98640 1 -- 172.21.15.49:0/103479597 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc14c0ab600 con 0x7fc15015aae0 2023-11-06T02:08:31.066 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.063+0000 7fc154a98640 1 -- 172.21.15.49:0/103479597 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc14c0ab760 con 0x7fc15015aae0 2023-11-06T02:08:31.067 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.063+0000 7fc154a98640 1 -- 172.21.15.49:0/103479597 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc14c0abaf0 con 0x7fc15015aae0 2023-11-06T02:08:31.067 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.067+0000 7fc157cfe640 1 -- 172.21.15.49:0/103479597 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc15015aae0 msgr2=0x7fc150070a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:31.067 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.067+0000 7fc157cfe640 1 --2- 172.21.15.49:0/103479597 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc15015aae0 0x7fc150070a80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc1500adfd0 tx=0x7fc14c01c400 comp rx=0 tx=0).stop 2023-11-06T02:08:31.067 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.067+0000 7fc157cfe640 1 -- 172.21.15.49:0/103479597 shutdown_connections 2023-11-06T02:08:31.067 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.067+0000 7fc157cfe640 1 --2- 172.21.15.49:0/103479597 >> v2:172.21.15.150:3300/0 conn(0x7fc1500b0790 0x7fc1500b1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:31.068 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.067+0000 7fc157cfe640 1 --2- 172.21.15.49:0/103479597 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc15015aae0 0x7fc150070a80 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:31.068 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.067+0000 7fc157cfe640 1 -- 172.21.15.49:0/103479597 >> 172.21.15.49:0/103479597 conn(0x7fc15006c0a0 msgr2=0x7fc15006c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:31.068 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.067+0000 7fc157cfe640 1 -- 172.21.15.49:0/103479597 shutdown_connections 2023-11-06T02:08:31.068 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.067+0000 7fc157cfe640 1 -- 172.21.15.49:0/103479597 wait complete. 2023-11-06T02:08:31.068 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.067+0000 7fc157cfe640 1 Processor -- start 2023-11-06T02:08:31.068 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.067+0000 7fc157cfe640 1 -- start start 2023-11-06T02:08:31.071 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.067+0000 7fc157cfe640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc1500b0790 0x7fc150154ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:31.071 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.067+0000 7fc157cfe640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1500ca830 0x7fc15015da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:31.071 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.067+0000 7fc157cfe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc15015aae0 0x7fc150152770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:31.072 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.067+0000 7fc157cfe640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc150152080 con 0x7fc15015aae0 2023-11-06T02:08:31.072 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.067+0000 7fc157cfe640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc150151da0 con 0x7fc1500ca830 2023-11-06T02:08:31.072 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.067+0000 7fc157cfe640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1501543e0 con 0x7fc1500b0790 2023-11-06T02:08:31.072 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.067+0000 7fc15629b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc15015aae0 0x7fc150152770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:31.072 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.067+0000 7fc15629b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc15015aae0 0x7fc150152770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47316/0 (socket says 172.21.15.49:47316) 2023-11-06T02:08:31.073 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.067+0000 7fc15629b640 1 -- 172.21.15.49:0/1451149143 learned_addr learned my addr 172.21.15.49:0/1451149143 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:31.073 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.067+0000 7fc155a9a640 1 --2- 172.21.15.49:0/1451149143 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc1500b0790 0x7fc150154ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:31.073 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.067+0000 7fc155299640 1 --2- 172.21.15.49:0/1451149143 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1500ca830 0x7fc15015da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:31.073 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.067+0000 7fc15629b640 1 -- 172.21.15.49:0/1451149143 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc1500b0790 msgr2=0x7fc150154ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:31.073 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.067+0000 7fc15629b640 1 --2- 172.21.15.49:0/1451149143 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc1500b0790 0x7fc150154ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:31.074 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.067+0000 7fc15629b640 1 -- 172.21.15.49:0/1451149143 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1500ca830 msgr2=0x7fc15015da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:31.074 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.067+0000 7fc15629b640 1 --2- 172.21.15.49:0/1451149143 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1500ca830 0x7fc15015da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:31.074 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.067+0000 7fc15629b640 1 -- 172.21.15.49:0/1451149143 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc14c009480 con 0x7fc15015aae0 2023-11-06T02:08:31.074 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.067+0000 7fc15629b640 1 --2- 172.21.15.49:0/1451149143 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc15015aae0 0x7fc150152770 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc14800cef0 tx=0x7fc14800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:31.074 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.067+0000 7fc1467fc640 1 -- 172.21.15.49:0/1451149143 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc14800d840 con 0x7fc15015aae0 2023-11-06T02:08:31.074 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.067+0000 7fc157cfe640 1 -- 172.21.15.49:0/1451149143 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1501546c0 con 0x7fc15015aae0 2023-11-06T02:08:31.075 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.067+0000 7fc157cfe640 1 -- 172.21.15.49:0/1451149143 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1501532b0 con 0x7fc15015aae0 2023-11-06T02:08:31.075 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.067+0000 7fc1467fc640 1 -- 172.21.15.49:0/1451149143 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc14800d9c0 con 0x7fc15015aae0 2023-11-06T02:08:31.075 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.067+0000 7fc1467fc640 1 -- 172.21.15.49:0/1451149143 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc14800dd70 con 0x7fc15015aae0 2023-11-06T02:08:31.075 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.071+0000 7fc157cfe640 1 -- 172.21.15.49:0/1451149143 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc138005260 con 0x7fc15015aae0 2023-11-06T02:08:31.075 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.071+0000 7fc1467fc640 1 -- 172.21.15.49:0/1451149143 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fc14801e3b0 con 0x7fc15015aae0 2023-11-06T02:08:31.076 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.071+0000 7fc1467fc640 1 --2- 172.21.15.49:0/1451149143 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc134065a60 0x7fc134067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:31.076 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.071+0000 7fc1467fc640 1 -- 172.21.15.49:0/1451149143 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc148083c00 con 0x7fc15015aae0 2023-11-06T02:08:31.076 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.071+0000 7fc155a9a640 1 --2- 172.21.15.49:0/1451149143 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc134065a60 0x7fc134067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:31.076 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.071+0000 7fc155a9a640 1 --2- 172.21.15.49:0/1451149143 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc134065a60 0x7fc134067f20 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc14c0af000 tx=0x7fc14c0a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:31.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.075+0000 7fc1467fc640 1 -- 172.21.15.49:0/1451149143 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fc14808a080 con 0x7fc15015aae0 2023-11-06T02:08:31.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.231+0000 7fc157cfe640 1 -- 172.21.15.49:0/1451149143 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc1380057e0 con 0x7fc15015aae0 2023-11-06T02:08:31.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.235+0000 7fc1467fc640 1 -- 172.21.15.49:0/1451149143 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+17824 (secure 0 0 0) 0x7fc14801a090 con 0x7fc15015aae0 2023-11-06T02:08:31.234 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:08:31.234 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":22,"fsid":"1a8b035c-6ba4-499a-b876-aea47fd9c0d4","created":"2023-11-06T02:08:06.440857+0000","modified":"2023-11-06T02:08:30.591607+0000","last_up_change":"2023-11-06T02:08:17.982372+0000","last_in_change":"2023-11-06T02:08:08.906631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T02:08:19.252222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T02:08:24.776211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-06T02:08:28.936178+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-06T02:08:29.563468+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ad55af8-0571-4e6a-8722-b5d429b7496d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6809","nonce":38116},{"type":"v1","addr":"172.21.15.49:6811","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6813","nonce":38116},{"type":"v1","addr":"172.21.15.49:6815","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6821","nonce":38116},{"type":"v1","addr":"172.21.15.49:6823","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6817","nonce":38116},{"type":"v1","addr":"172.21.15.49:6819","nonce":38116}]},"public_addr":"172.21.15.49:6811/38116","cluster_addr":"172.21.15.49:6815/38116","heartbeat_back_addr":"172.21.15.49:6823/38116","heartbeat_front_addr":"172.21.15.49:6819/38116","state":["exists","up"]},{"osd":1,"uuid":"c8b31f85-a757-43ed-9419-351acc46611b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":38113},{"type":"v1","addr":"172.21.15.49:6810","nonce":38113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":38113},{"type":"v1","addr":"172.21.15.49:6814","nonce":38113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":38113},{"type":"v1","addr":"172.21.15.49:6822","nonce":38113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":38113},{"type":"v1","addr":"172.21.15.49:6818","nonce":38113}]},"public_addr":"172.21.15.49:6810/38113","cluster_addr":"172.21.15.49:6814/38113","heartbeat_back_addr":"172.21.15.49:6822/38113","heartbeat_front_addr":"172.21.15.49:6818/38113","state":["exists","up"]},{"osd":2,"uuid":"555b3bf3-0a70-4b91-8efd-e4204b5fc26e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":38114},{"type":"v1","addr":"172.21.15.49:6801","nonce":38114}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":38114},{"type":"v1","addr":"172.21.15.49:6803","nonce":38114}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":38114},{"type":"v1","addr":"172.21.15.49:6807","nonce":38114}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":38114},{"type":"v1","addr":"172.21.15.49:6805","nonce":38114}]},"public_addr":"172.21.15.49:6801/38114","cluster_addr":"172.21.15.49:6803/38114","heartbeat_back_addr":"172.21.15.49:6807/38114","heartbeat_front_addr":"172.21.15.49:6805/38114","state":["exists","up"]},{"osd":3,"uuid":"8d4e5d77-03ef-4b9b-bfcb-5ae93a46795c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":38115},{"type":"v1","addr":"172.21.15.49:6825","nonce":38115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":38115},{"type":"v1","addr":"172.21.15.49:6827","nonce":38115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":38115},{"type":"v1","addr":"172.21.15.49:6831","nonce":38115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":38115},{"type":"v1","addr":"172.21.15.49:6829","nonce":38115}]},"public_addr":"172.21.15.49:6825/38115","cluster_addr":"172.21.15.49:6827/38115","heartbeat_back_addr":"172.21.15.49:6831/38115","heartbeat_front_addr":"172.21.15.49:6829/38115","state":["exists","up"]},{"osd":4,"uuid":"dba03ad4-68a3-4cf3-9218-ac611f02eacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":38417},{"type":"v1","addr":"172.21.15.150:6809","nonce":38417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":38417},{"type":"v1","addr":"172.21.15.150:6811","nonce":38417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":38417},{"type":"v1","addr":"172.21.15.150:6815","nonce":38417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":38417},{"type":"v1","addr":"172.21.15.150:6813","nonce":38417}]},"public_addr":"172.21.15.150:6809/38417","cluster_addr":"172.21.15.150:6811/38417","heartbeat_back_addr":"172.21.15.150:6815/38417","heartbeat_front_addr":"172.21.15.150:6813/38417","state":["exists","up"]},{"osd":5,"uuid":"0df9c303-c255-4640-8e91-01f1c3efe18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":38418},{"type":"v1","addr":"172.21.15.150:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":38418},{"type":"v1","addr":"172.21.15.150:6803","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":38418},{"type":"v1","addr":"172.21.15.150:6807","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":38418},{"type":"v1","addr":"172.21.15.150:6805","nonce":38418}]},"public_addr":"172.21.15.150:6801/38418","cluster_addr":"172.21.15.150:6803/38418","heartbeat_back_addr":"172.21.15.150:6807/38418","heartbeat_front_addr":"172.21.15.150:6805/38418","state":["exists","up"]},{"osd":6,"uuid":"e59fa189-4b70-4c06-9722-c02b90828bf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":38428},{"type":"v1","addr":"172.21.15.150:6825","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":38428},{"type":"v1","addr":"172.21.15.150:6827","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":38428},{"type":"v1","addr":"172.21.15.150:6831","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":38428},{"type":"v1","addr":"172.21.15.150:6829","nonce":38428}]},"public_addr":"172.21.15.150:6825/38428","cluster_addr":"172.21.15.150:6827/38428","heartbeat_back_addr":"172.21.15.150:6831/38428","heartbeat_front_addr":"172.21.15.150:6829/38428","state":["exists","up"]},{"osd":7,"uuid":"a64f57e1-ef98-46bd-b304-385759e95898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":38421},{"type":"v1","addr":"172.21.15.150:6817","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":38421},{"type":"v1","addr":"172.21.15.150:6819","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":38421},{"type":"v1","addr":"172.21.15.150:6823","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":38421},{"type":"v1","addr":"172.21.15.150:6821","nonce":38421}]},"public_addr":"172.21.15.150:6817/38421","cluster_addr":"172.21.15.150:6819/38421","heartbeat_back_addr":"172.21.15.150:6823/38421","heartbeat_front_addr":"172.21.15.150:6821/38421","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.438552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.413280+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.296466+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.424966+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.276953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.230984+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.410235+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.480084+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T02:08:31.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.235+0000 7fc157cfe640 1 -- 172.21.15.49:0/1451149143 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc134065a60 msgr2=0x7fc134067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:31.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.235+0000 7fc157cfe640 1 --2- 172.21.15.49:0/1451149143 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc134065a60 0x7fc134067f20 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc14c0af000 tx=0x7fc14c0a7000 comp rx=0 tx=0).stop 2023-11-06T02:08:31.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.235+0000 7fc157cfe640 1 -- 172.21.15.49:0/1451149143 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc15015aae0 msgr2=0x7fc150152770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:31.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.235+0000 7fc157cfe640 1 --2- 172.21.15.49:0/1451149143 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc15015aae0 0x7fc150152770 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc14800cef0 tx=0x7fc14800a710 comp rx=0 tx=0).stop 2023-11-06T02:08:31.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.235+0000 7fc157cfe640 1 -- 172.21.15.49:0/1451149143 shutdown_connections 2023-11-06T02:08:31.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.235+0000 7fc157cfe640 1 --2- 172.21.15.49:0/1451149143 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc134065a60 0x7fc134067f20 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:31.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.235+0000 7fc157cfe640 1 --2- 172.21.15.49:0/1451149143 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc15015aae0 0x7fc150152770 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:31.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.235+0000 7fc157cfe640 1 --2- 172.21.15.49:0/1451149143 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1500ca830 0x7fc15015da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:31.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.235+0000 7fc157cfe640 1 --2- 172.21.15.49:0/1451149143 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc1500b0790 0x7fc150154ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:31.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.235+0000 7fc157cfe640 1 -- 172.21.15.49:0/1451149143 >> 172.21.15.49:0/1451149143 conn(0x7fc15006c0a0 msgr2=0x7fc1500ce5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:31.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.235+0000 7fc157cfe640 1 -- 172.21.15.49:0/1451149143 shutdown_connections 2023-11-06T02:08:31.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.235+0000 7fc157cfe640 1 -- 172.21.15.49:0/1451149143 wait complete. 2023-11-06T02:08:31.253 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-06T02:08:31.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.383+0000 7fa3e731e640 1 Processor -- start 2023-11-06T02:08:31.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.383+0000 7fa3e731e640 1 -- start start 2023-11-06T02:08:31.383 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.383+0000 7fa3e731e640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa3e00b2310 0x7fa3e00cb070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:31.383 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.383+0000 7fa3e731e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa3e01607d0 0x7fa3e0160c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:31.383 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.383+0000 7fa3e731e640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3e00cf3a0 con 0x7fa3e00b26f0 2023-11-06T02:08:31.383 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.383+0000 7fa3e731e640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3e006c090 con 0x7fa3e00b2310 2023-11-06T02:08:31.384 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.383+0000 7fa3e731e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e015cee0 con 0x7fa3e01607d0 2023-11-06T02:08:31.384 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.383+0000 7fa3e50ba640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa3e00b2310 0x7fa3e00cb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:31.384 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.383+0000 7fa3e50ba640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa3e00b2310 0x7fa3e00cb070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47320/0 (socket says 172.21.15.49:47320) 2023-11-06T02:08:31.384 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.383+0000 7fa3e50ba640 1 -- 172.21.15.49:0/1434654804 learned_addr learned my addr 172.21.15.49:0/1434654804 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:31.384 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.383+0000 7fa3e58bb640 1 --2- 172.21.15.49:0/1434654804 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa3e01607d0 0x7fa3e0160c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:31.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.383+0000 7fa3e50ba640 1 -- 172.21.15.49:0/1434654804 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa3e01607d0 msgr2=0x7fa3e0160c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:31.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.383+0000 7fa3e50ba640 1 --2- 172.21.15.49:0/1434654804 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa3e01607d0 0x7fa3e0160c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:31.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.383+0000 7fa3e50ba640 1 -- 172.21.15.49:0/1434654804 >> v1:172.21.15.49:6789/0 conn(0x7fa3e00b26f0 legacy=0x7fa3e006c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:31.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.383+0000 7fa3e50ba640 1 -- 172.21.15.49:0/1434654804 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3e006b480 con 0x7fa3e00b2310 2023-11-06T02:08:31.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.383+0000 7fa3e58bb640 1 --2- 172.21.15.49:0/1434654804 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa3e01607d0 0x7fa3e0160c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:08:31.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.387+0000 7fa3e50ba640 1 --2- 172.21.15.49:0/1434654804 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa3e00b2310 0x7fa3e00cb070 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc009870 tx=0x7fa3dc01c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aec26ef0031e821 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:31.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.387+0000 7fa3d7fff640 1 -- 172.21.15.49:0/1434654804 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3dc0ab620 con 0x7fa3e00b2310 2023-11-06T02:08:31.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.387+0000 7fa3d7fff640 1 -- 172.21.15.49:0/1434654804 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3dc0ab780 con 0x7fa3e00b2310 2023-11-06T02:08:31.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.387+0000 7fa3d7fff640 1 -- 172.21.15.49:0/1434654804 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3dc0abb10 con 0x7fa3e00b2310 2023-11-06T02:08:31.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.387+0000 7fa3e731e640 1 -- 172.21.15.49:0/1434654804 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa3e00b2310 msgr2=0x7fa3e00cb070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:31.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.387+0000 7fa3e731e640 1 --2- 172.21.15.49:0/1434654804 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa3e00b2310 0x7fa3e00cb070 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc009870 tx=0x7fa3dc01c510 comp rx=0 tx=0).stop 2023-11-06T02:08:31.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.387+0000 7fa3e731e640 1 -- 172.21.15.49:0/1434654804 shutdown_connections 2023-11-06T02:08:31.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.387+0000 7fa3e731e640 1 --2- 172.21.15.49:0/1434654804 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa3e01607d0 0x7fa3e0160c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:31.389 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.387+0000 7fa3e731e640 1 --2- 172.21.15.49:0/1434654804 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa3e00b2310 0x7fa3e00cb070 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:31.389 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.387+0000 7fa3e731e640 1 -- 172.21.15.49:0/1434654804 >> 172.21.15.49:0/1434654804 conn(0x7fa3e00cd310 msgr2=0x7fa3e00cd720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:31.389 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.387+0000 7fa3e731e640 1 -- 172.21.15.49:0/1434654804 shutdown_connections 2023-11-06T02:08:31.389 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.387+0000 7fa3e731e640 1 -- 172.21.15.49:0/1434654804 wait complete. 2023-11-06T02:08:31.389 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.387+0000 7fa3e731e640 1 Processor -- start 2023-11-06T02:08:31.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.387+0000 7fa3e731e640 1 -- start start 2023-11-06T02:08:31.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.387+0000 7fa3e731e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3e00b2310 0x7fa3e015ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:31.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.387+0000 7fa3e731e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa3e00b26f0 0x7fa3e015f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:31.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.387+0000 7fa3e731e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa3e01607d0 0x7fa3e0154410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:31.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.387+0000 7fa3e731e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e015fe50 con 0x7fa3e00b26f0 2023-11-06T02:08:31.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.387+0000 7fa3e731e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e0159bc0 con 0x7fa3e00b2310 2023-11-06T02:08:31.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.387+0000 7fa3e731e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e0159ea0 con 0x7fa3e01607d0 2023-11-06T02:08:31.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.387+0000 7fa3e48b9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa3e00b26f0 0x7fa3e015f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:31.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.387+0000 7fa3e48b9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa3e00b26f0 0x7fa3e015f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47324/0 (socket says 172.21.15.49:47324) 2023-11-06T02:08:31.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.387+0000 7fa3e48b9640 1 -- 172.21.15.49:0/1892878655 learned_addr learned my addr 172.21.15.49:0/1892878655 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:31.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.387+0000 7fa3e58bb640 1 --2- 172.21.15.49:0/1892878655 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa3e01607d0 0x7fa3e0154410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:31.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.387+0000 7fa3e50ba640 1 --2- 172.21.15.49:0/1892878655 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3e00b2310 0x7fa3e015ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:31.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.391+0000 7fa3e48b9640 1 -- 172.21.15.49:0/1892878655 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa3e01607d0 msgr2=0x7fa3e0154410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:31.395 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.391+0000 7fa3e48b9640 1 --2- 172.21.15.49:0/1892878655 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa3e01607d0 0x7fa3e0154410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:31.395 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.391+0000 7fa3e48b9640 1 -- 172.21.15.49:0/1892878655 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3e00b2310 msgr2=0x7fa3e015ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:31.395 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.391+0000 7fa3e48b9640 1 --2- 172.21.15.49:0/1892878655 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3e00b2310 0x7fa3e015ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:31.395 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.391+0000 7fa3e48b9640 1 -- 172.21.15.49:0/1892878655 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3dc0094d0 con 0x7fa3e00b26f0 2023-11-06T02:08:31.395 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.391+0000 7fa3e50ba640 1 --2- 172.21.15.49:0/1892878655 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3e00b2310 0x7fa3e015ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:08:31.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.391+0000 7fa3e48b9640 1 --2- 172.21.15.49:0/1892878655 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa3e00b26f0 0x7fa3e015f910 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa3d000c500 tx=0x7fa3d000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:31.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.391+0000 7fa3d5ffb640 1 -- 172.21.15.49:0/1892878655 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d0005220 con 0x7fa3e00b26f0 2023-11-06T02:08:31.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.391+0000 7fa3d5ffb640 1 -- 172.21.15.49:0/1892878655 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3d0005380 con 0x7fa3e00b26f0 2023-11-06T02:08:31.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.391+0000 7fa3e731e640 1 -- 172.21.15.49:0/1892878655 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3e015a180 con 0x7fa3e00b26f0 2023-11-06T02:08:31.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.391+0000 7fa3d5ffb640 1 -- 172.21.15.49:0/1892878655 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d0007270 con 0x7fa3e00b26f0 2023-11-06T02:08:31.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.391+0000 7fa3e731e640 1 -- 172.21.15.49:0/1892878655 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3e015a740 con 0x7fa3e00b26f0 2023-11-06T02:08:31.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.391+0000 7fa3e731e640 1 -- 172.21.15.49:0/1892878655 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3e0067140 con 0x7fa3e00b26f0 2023-11-06T02:08:31.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.391+0000 7fa3d5ffb640 1 -- 172.21.15.49:0/1892878655 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fa3d00073d0 con 0x7fa3e00b26f0 2023-11-06T02:08:31.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.391+0000 7fa3d5ffb640 1 --2- 172.21.15.49:0/1892878655 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa3c4065a60 0x7fa3c4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:31.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.391+0000 7fa3d5ffb640 1 -- 172.21.15.49:0/1892878655 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa3d00826c0 con 0x7fa3e00b26f0 2023-11-06T02:08:31.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.391+0000 7fa3e50ba640 1 --2- 172.21.15.49:0/1892878655 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa3c4065a60 0x7fa3c4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:31.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.391+0000 7fa3e50ba640 1 --2- 172.21.15.49:0/1892878655 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa3c4065a60 0x7fa3c4067f20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc0af000 tx=0x7fa3dc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:31.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.395+0000 7fa3d5ffb640 1 -- 172.21.15.49:0/1892878655 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fa3d004ff30 con 0x7fa3e00b26f0 2023-11-06T02:08:31.586 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:31.587+0000 7fa3e731e640 1 -- 172.21.15.49:0/1892878655 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fa3e006b130 con 0x7fa3e00b26f0 2023-11-06T02:08:32.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:32.243+0000 7fa3d5ffb640 1 -- 172.21.15.49:0/1892878655 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v6) v1 ==== 118+0+0 (secure 0 0 0) 0x7fa3d0053550 con 0x7fa3e00b26f0 2023-11-06T02:08:32.247 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:32.247+0000 7fa3e731e640 1 -- 172.21.15.49:0/1892878655 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa3c4065a60 msgr2=0x7fa3c4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:32.248 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:32.247+0000 7fa3e731e640 1 --2- 172.21.15.49:0/1892878655 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa3c4065a60 0x7fa3c4067f20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc0af000 tx=0x7fa3dc008040 comp rx=0 tx=0).stop 2023-11-06T02:08:32.248 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:32.247+0000 7fa3e731e640 1 -- 172.21.15.49:0/1892878655 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa3e00b26f0 msgr2=0x7fa3e015f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:32.248 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:32.247+0000 7fa3e731e640 1 --2- 172.21.15.49:0/1892878655 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa3e00b26f0 0x7fa3e015f910 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa3d000c500 tx=0x7fa3d000c9c0 comp rx=0 tx=0).stop 2023-11-06T02:08:32.249 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:32.247+0000 7fa3e731e640 1 -- 172.21.15.49:0/1892878655 shutdown_connections 2023-11-06T02:08:32.249 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:32.247+0000 7fa3e731e640 1 --2- 172.21.15.49:0/1892878655 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa3c4065a60 0x7fa3c4067f20 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:32.249 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:32.247+0000 7fa3e731e640 1 --2- 172.21.15.49:0/1892878655 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa3e00b26f0 0x7fa3e015f910 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:32.250 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:32.247+0000 7fa3e731e640 1 --2- 172.21.15.49:0/1892878655 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa3e01607d0 0x7fa3e0154410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:32.250 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:32.247+0000 7fa3e731e640 1 --2- 172.21.15.49:0/1892878655 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3e00b2310 0x7fa3e015ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:32.250 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:32.247+0000 7fa3e731e640 1 -- 172.21.15.49:0/1892878655 >> 172.21.15.49:0/1892878655 conn(0x7fa3e00cd310 msgr2=0x7fa3e0155a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:32.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:32.247+0000 7fa3e731e640 1 -- 172.21.15.49:0/1892878655 shutdown_connections 2023-11-06T02:08:32.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:32.247+0000 7fa3e731e640 1 -- 172.21.15.49:0/1892878655 wait complete. 2023-11-06T02:08:32.267 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-06T02:08:32.267 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-06T02:08:32.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:32.395+0000 7f9a90421640 1 Processor -- start 2023-11-06T02:08:32.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:32.395+0000 7f9a90421640 1 -- start start 2023-11-06T02:08:32.395 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:32.395+0000 7f9a90421640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9a880d2a60 0x7f9a880d2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:32.395 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:32.395+0000 7f9a90421640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f9a880b5d40 0x7f9a880b61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:32.395 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:32.395+0000 7f9a90421640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a880d33e0 con 0x7f9a8815cbb0 2023-11-06T02:08:32.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:32.395+0000 7f9a90421640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a8815cf90 con 0x7f9a880b5d40 2023-11-06T02:08:32.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:32.395+0000 7f9a90421640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a880d0b80 con 0x7f9a880d2a60 2023-11-06T02:08:32.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:32.395+0000 7f9a8d9bc640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9a880d2a60 0x7f9a880d2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:32.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:32.395+0000 7f9a8d9bc640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9a880d2a60 0x7f9a880d2ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.49:60154/0 (socket says 172.21.15.49:60154) 2023-11-06T02:08:32.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:32.395+0000 7f9a8d9bc640 1 -- 172.21.15.49:0/10552072 learned_addr learned my addr 172.21.15.49:0/10552072 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:32.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:32.395+0000 7f9a8e9be640 1 --2- 172.21.15.49:0/10552072 >> v2:172.21.15.150:3300/0 conn(0x7f9a880b5d40 0x7f9a880b61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:32.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:32.395+0000 7f9a8e9be640 1 -- 172.21.15.49:0/10552072 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9a880d2a60 msgr2=0x7f9a880d2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:32.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:32.395+0000 7f9a8e9be640 1 --2- 172.21.15.49:0/10552072 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9a880d2a60 0x7f9a880d2ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:32.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:32.395+0000 7f9a8e9be640 1 -- 172.21.15.49:0/10552072 >> v1:172.21.15.150:6789/0 conn(0x7f9a8815cbb0 legacy=0x7f9a880d1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:32.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:32.395+0000 7f9a8e9be640 1 -- 172.21.15.49:0/10552072 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a880d0e60 con 0x7f9a880b5d40 2023-11-06T02:08:32.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:32.399+0000 7f9a8e9be640 1 --2- 172.21.15.49:0/10552072 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a880b5d40 0x7f9a880b61d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9a780097c0 tx=0x7f9a7801ec70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e21d848077598712 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:32.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:32.399+0000 7f9a8d1bb640 1 -- 172.21.15.49:0/10552072 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a780a8970 con 0x7f9a880b5d40 2023-11-06T02:08:32.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:32.399+0000 7f9a8d1bb640 1 -- 172.21.15.49:0/10552072 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a780a8ad0 con 0x7f9a880b5d40 2023-11-06T02:08:32.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:32.399+0000 7f9a8d1bb640 1 -- 172.21.15.49:0/10552072 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a780a8e60 con 0x7f9a880b5d40 2023-11-06T02:08:32.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:32.399+0000 7f9a90421640 1 -- 172.21.15.49:0/10552072 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a880b5d40 msgr2=0x7f9a880b61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:32.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:32.399+0000 7f9a90421640 1 --2- 172.21.15.49:0/10552072 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a880b5d40 0x7f9a880b61d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9a780097c0 tx=0x7f9a7801ec70 comp rx=0 tx=0).stop 2023-11-06T02:08:32.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:32.399+0000 7f9a90421640 1 -- 172.21.15.49:0/10552072 shutdown_connections 2023-11-06T02:08:32.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:32.399+0000 7f9a90421640 1 --2- 172.21.15.49:0/10552072 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9a880d2a60 0x7f9a880d2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:32.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:32.399+0000 7f9a90421640 1 --2- 172.21.15.49:0/10552072 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a880b5d40 0x7f9a880b61d0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:32.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:32.399+0000 7f9a90421640 1 -- 172.21.15.49:0/10552072 >> 172.21.15.49:0/10552072 conn(0x7f9a880718a0 msgr2=0x7f9a88071cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:32.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:32.399+0000 7f9a90421640 1 -- 172.21.15.49:0/10552072 shutdown_connections 2023-11-06T02:08:32.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:32.399+0000 7f9a90421640 1 -- 172.21.15.49:0/10552072 wait complete. 2023-11-06T02:08:32.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:32.399+0000 7f9a90421640 1 Processor -- start 2023-11-06T02:08:32.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:32.403+0000 7f9a90421640 1 -- start start 2023-11-06T02:08:32.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:32.403+0000 7f9a90421640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9a880b5d40 0x7f9a88156070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:32.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:32.403+0000 7f9a90421640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a880d2a60 0x7f9a88156630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:32.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:32.403+0000 7f9a90421640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a8815cbb0 0x7f9a880c8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:32.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:32.403+0000 7f9a90421640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a88157b00 con 0x7f9a880d2a60 2023-11-06T02:08:32.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:32.403+0000 7f9a90421640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a88156cf0 con 0x7f9a8815cbb0 2023-11-06T02:08:32.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:32.403+0000 7f9a90421640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a88156fd0 con 0x7f9a880b5d40 2023-11-06T02:08:32.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:32.403+0000 7f9a8d9bc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a880d2a60 0x7f9a88156630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:32.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:32.403+0000 7f9a8d9bc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a880d2a60 0x7f9a88156630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47332/0 (socket says 172.21.15.49:47332) 2023-11-06T02:08:32.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:32.403+0000 7f9a8d9bc640 1 -- 172.21.15.49:0/4215375740 learned_addr learned my addr 172.21.15.49:0/4215375740 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:32.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:32.403+0000 7f9a8e9be640 1 --2- 172.21.15.49:0/4215375740 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a8815cbb0 0x7f9a880c8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:32.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:32.403+0000 7f9a8e1bd640 1 --2- 172.21.15.49:0/4215375740 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9a880b5d40 0x7f9a88156070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:32.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:32.403+0000 7f9a8d9bc640 1 -- 172.21.15.49:0/4215375740 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9a880b5d40 msgr2=0x7f9a88156070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:32.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:32.403+0000 7f9a8d9bc640 1 --2- 172.21.15.49:0/4215375740 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9a880b5d40 0x7f9a88156070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:32.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:32.403+0000 7f9a8d9bc640 1 -- 172.21.15.49:0/4215375740 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a8815cbb0 msgr2=0x7f9a880c8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:32.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:32.403+0000 7f9a8d9bc640 1 --2- 172.21.15.49:0/4215375740 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a8815cbb0 0x7f9a880c8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:32.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:32.403+0000 7f9a8d9bc640 1 -- 172.21.15.49:0/4215375740 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a78009420 con 0x7f9a880d2a60 2023-11-06T02:08:32.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:32.403+0000 7f9a8d9bc640 1 --2- 172.21.15.49:0/4215375740 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a880d2a60 0x7f9a88156630 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c00efc0 tx=0x7f9a7c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:32.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:32.403+0000 7f9a86ffd640 1 -- 172.21.15.49:0/4215375740 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a7c00cda0 con 0x7f9a880d2a60 2023-11-06T02:08:32.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:32.403+0000 7f9a86ffd640 1 -- 172.21.15.49:0/4215375740 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a7c00f040 con 0x7f9a880d2a60 2023-11-06T02:08:32.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:32.403+0000 7f9a86ffd640 1 -- 172.21.15.49:0/4215375740 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a7c014680 con 0x7f9a880d2a60 2023-11-06T02:08:32.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:32.403+0000 7f9a90421640 1 -- 172.21.15.49:0/4215375740 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a881572b0 con 0x7f9a880d2a60 2023-11-06T02:08:32.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:32.403+0000 7f9a90421640 1 -- 172.21.15.49:0/4215375740 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a880c8ac0 con 0x7f9a880d2a60 2023-11-06T02:08:32.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:32.403+0000 7f9a90421640 1 -- 172.21.15.49:0/4215375740 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a88074040 con 0x7f9a880d2a60 2023-11-06T02:08:32.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:32.403+0000 7f9a86ffd640 1 -- 172.21.15.49:0/4215375740 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f9a7c0147e0 con 0x7f9a880d2a60 2023-11-06T02:08:32.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:32.403+0000 7f9a86ffd640 1 --2- 172.21.15.49:0/4215375740 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9a74065a60 0x7f9a74067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:32.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:32.403+0000 7f9a86ffd640 1 -- 172.21.15.49:0/4215375740 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9a7c083b40 con 0x7f9a880d2a60 2023-11-06T02:08:32.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:32.403+0000 7f9a8e1bd640 1 --2- 172.21.15.49:0/4215375740 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9a74065a60 0x7f9a74067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:32.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:32.407+0000 7f9a8e1bd640 1 --2- 172.21.15.49:0/4215375740 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9a74065a60 0x7f9a74067f20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9a80000f70 tx=0x7f9a80009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:32.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:32.411+0000 7f9a86ffd640 1 -- 172.21.15.49:0/4215375740 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f9a7c051300 con 0x7f9a880d2a60 2023-11-06T02:08:32.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:32.599+0000 7f9a90421640 1 -- 172.21.15.49:0/4215375740 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f9a88157c40 con 0x7f9a880d2a60 2023-11-06T02:08:33.178 INFO:tasks.ceph.mds.a.smithi049.stderr:2023-11-06T02:08:33.183+0000 7f5e10569640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-06T02:08:33.247 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.247+0000 7f9a86ffd640 1 -- 172.21.15.49:0/4215375740 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v7) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9a88157c40 con 0x7f9a880d2a60 2023-11-06T02:08:33.250 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.251+0000 7f9a90421640 1 -- 172.21.15.49:0/4215375740 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9a74065a60 msgr2=0x7f9a74067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:33.250 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.251+0000 7f9a90421640 1 --2- 172.21.15.49:0/4215375740 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9a74065a60 0x7f9a74067f20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9a80000f70 tx=0x7f9a80009040 comp rx=0 tx=0).stop 2023-11-06T02:08:33.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.251+0000 7f9a90421640 1 -- 172.21.15.49:0/4215375740 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a880d2a60 msgr2=0x7f9a88156630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:33.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.251+0000 7f9a90421640 1 --2- 172.21.15.49:0/4215375740 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a880d2a60 0x7f9a88156630 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c00efc0 tx=0x7f9a7c00c490 comp rx=0 tx=0).stop 2023-11-06T02:08:33.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.251+0000 7f9a90421640 1 -- 172.21.15.49:0/4215375740 shutdown_connections 2023-11-06T02:08:33.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.251+0000 7f9a90421640 1 --2- 172.21.15.49:0/4215375740 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9a74065a60 0x7f9a74067f20 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:33.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.251+0000 7f9a90421640 1 --2- 172.21.15.49:0/4215375740 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a880d2a60 0x7f9a88156630 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:33.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.251+0000 7f9a90421640 1 --2- 172.21.15.49:0/4215375740 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a8815cbb0 0x7f9a880c8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:33.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.251+0000 7f9a90421640 1 --2- 172.21.15.49:0/4215375740 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9a880b5d40 0x7f9a88156070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:33.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.251+0000 7f9a90421640 1 -- 172.21.15.49:0/4215375740 >> 172.21.15.49:0/4215375740 conn(0x7f9a880718a0 msgr2=0x7f9a880cd100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:33.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.251+0000 7f9a90421640 1 -- 172.21.15.49:0/4215375740 shutdown_connections 2023-11-06T02:08:33.253 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.251+0000 7f9a90421640 1 -- 172.21.15.49:0/4215375740 wait complete. 2023-11-06T02:08:33.273 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-06T02:08:33.273 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T02:08:33.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.403+0000 7ff7ef3bb640 1 Processor -- start 2023-11-06T02:08:33.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.403+0000 7ff7ef3bb640 1 -- start start 2023-11-06T02:08:33.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.403+0000 7ff7ef3bb640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff7e815ab50 0x7ff7e80c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:33.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.403+0000 7ff7ef3bb640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7e80cab80 con 0x7ff7e80cb010 2023-11-06T02:08:33.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.403+0000 7ff7ef3bb640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7e815af30 con 0x7ff7e80b0790 2023-11-06T02:08:33.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.403+0000 7ff7ef3bb640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7e80cb5b0 con 0x7ff7e815ab50 2023-11-06T02:08:33.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.403+0000 7ff7ed157640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff7e815ab50 0x7ff7e80c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:33.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.403+0000 7ff7ec956640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7ff7e80cb010 0x7ff7e80b0080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:44948/0 (socket says 172.21.15.49:44948) 2023-11-06T02:08:33.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.403+0000 7ff7ed157640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff7e815ab50 0x7ff7e80c9bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47336/0 (socket says 172.21.15.49:47336) 2023-11-06T02:08:33.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.403+0000 7ff7ec956640 1 -- 172.21.15.49:0/313703529 learned_addr learned my addr 172.21.15.49:0/313703529 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:08:33.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.403+0000 7ff7ed157640 1 -- 172.21.15.49:0/313703529 >> v1:172.21.15.150:6789/0 conn(0x7ff7e80b0790 legacy=0x7ff7e80c2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:33.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.403+0000 7ff7ed157640 1 -- 172.21.15.49:0/313703529 >> v1:172.21.15.49:6789/0 conn(0x7ff7e80cb010 legacy=0x7ff7e80b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:33.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.403+0000 7ff7ed157640 1 -- 172.21.15.49:0/313703529 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7e80cb890 con 0x7ff7e815ab50 2023-11-06T02:08:33.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.407+0000 7ff7ed157640 1 --2- 172.21.15.49:0/313703529 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7e815ab50 0x7ff7e80c9bb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff7e40097b0 tx=0x7ff7e401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5a096698d6734ac server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:33.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.407+0000 7ff7dffff640 1 -- 172.21.15.49:0/313703529 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e40ab650 con 0x7ff7e815ab50 2023-11-06T02:08:33.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.407+0000 7ff7dffff640 1 -- 172.21.15.49:0/313703529 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7e40ab7b0 con 0x7ff7e815ab50 2023-11-06T02:08:33.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.407+0000 7ff7dffff640 1 -- 172.21.15.49:0/313703529 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e40abb40 con 0x7ff7e815ab50 2023-11-06T02:08:33.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.407+0000 7ff7ef3bb640 1 -- 172.21.15.49:0/313703529 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7e815ab50 msgr2=0x7ff7e80c9bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:33.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.407+0000 7ff7ef3bb640 1 --2- 172.21.15.49:0/313703529 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7e815ab50 0x7ff7e80c9bb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff7e40097b0 tx=0x7ff7e401c450 comp rx=0 tx=0).stop 2023-11-06T02:08:33.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.407+0000 7ff7ef3bb640 1 -- 172.21.15.49:0/313703529 shutdown_connections 2023-11-06T02:08:33.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.407+0000 7ff7ef3bb640 1 --2- 172.21.15.49:0/313703529 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7e815ab50 0x7ff7e80c9bb0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:33.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.407+0000 7ff7ef3bb640 1 -- 172.21.15.49:0/313703529 >> 172.21.15.49:0/313703529 conn(0x7ff7e806c0a0 msgr2=0x7ff7e806c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:33.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.407+0000 7ff7ef3bb640 1 -- 172.21.15.49:0/313703529 shutdown_connections 2023-11-06T02:08:33.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.407+0000 7ff7ef3bb640 1 -- 172.21.15.49:0/313703529 wait complete. 2023-11-06T02:08:33.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.407+0000 7ff7ef3bb640 1 Processor -- start 2023-11-06T02:08:33.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.407+0000 7ff7ef3bb640 1 -- start start 2023-11-06T02:08:33.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.407+0000 7ff7ef3bb640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff7e80b0790 0x7ff7e814a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:33.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.407+0000 7ff7ef3bb640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7e80cb010 0x7ff7e8150fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:33.411 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.407+0000 7ff7ef3bb640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7e815ab50 0x7ff7e814e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:33.411 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.407+0000 7ff7ef3bb640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e814d060 con 0x7ff7e80cb010 2023-11-06T02:08:33.411 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.407+0000 7ff7ef3bb640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e814d300 con 0x7ff7e815ab50 2023-11-06T02:08:33.411 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.407+0000 7ff7ef3bb640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e814d5e0 con 0x7ff7e80b0790 2023-11-06T02:08:33.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.407+0000 7ff7ec956640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7e80cb010 0x7ff7e8150fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:33.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.407+0000 7ff7ec956640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7e80cb010 0x7ff7e8150fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47342/0 (socket says 172.21.15.49:47342) 2023-11-06T02:08:33.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.407+0000 7ff7ec956640 1 -- 172.21.15.49:0/2798849914 learned_addr learned my addr 172.21.15.49:0/2798849914 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:33.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.411+0000 7ff7ed157640 1 --2- 172.21.15.49:0/2798849914 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff7e80b0790 0x7ff7e814a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:33.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.411+0000 7ff7ed958640 1 --2- 172.21.15.49:0/2798849914 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7e815ab50 0x7ff7e814e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:33.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.411+0000 7ff7ec956640 1 -- 172.21.15.49:0/2798849914 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff7e80b0790 msgr2=0x7ff7e814a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:33.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.411+0000 7ff7ec956640 1 --2- 172.21.15.49:0/2798849914 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff7e80b0790 0x7ff7e814a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:33.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.411+0000 7ff7ec956640 1 -- 172.21.15.49:0/2798849914 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7e815ab50 msgr2=0x7ff7e814e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:33.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.411+0000 7ff7ec956640 1 --2- 172.21.15.49:0/2798849914 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7e815ab50 0x7ff7e814e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:33.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.411+0000 7ff7ec956640 1 -- 172.21.15.49:0/2798849914 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7e40093c0 con 0x7ff7e80cb010 2023-11-06T02:08:33.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.411+0000 7ff7ed157640 1 --2- 172.21.15.49:0/2798849914 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff7e80b0790 0x7ff7e814a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:08:33.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.411+0000 7ff7ec956640 1 --2- 172.21.15.49:0/2798849914 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7e80cb010 0x7ff7e8150fa0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff7d800c500 tx=0x7ff7d800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:33.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.411+0000 7ff7ddffb640 1 -- 172.21.15.49:0/2798849914 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d8005cf0 con 0x7ff7e80cb010 2023-11-06T02:08:33.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.411+0000 7ff7ddffb640 1 -- 172.21.15.49:0/2798849914 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7d8005e50 con 0x7ff7e80cb010 2023-11-06T02:08:33.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.411+0000 7ff7ef3bb640 1 -- 172.21.15.49:0/2798849914 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7e814d8c0 con 0x7ff7e80cb010 2023-11-06T02:08:33.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.411+0000 7ff7ddffb640 1 -- 172.21.15.49:0/2798849914 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d80072f0 con 0x7ff7e80cb010 2023-11-06T02:08:33.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.411+0000 7ff7ef3bb640 1 -- 172.21.15.49:0/2798849914 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7e814a560 con 0x7ff7e80cb010 2023-11-06T02:08:33.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.411+0000 7ff7ef3bb640 1 -- 172.21.15.49:0/2798849914 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7e80b0de0 con 0x7ff7e80cb010 2023-11-06T02:08:33.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.411+0000 7ff7ddffb640 1 -- 172.21.15.49:0/2798849914 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7ff7d8007450 con 0x7ff7e80cb010 2023-11-06T02:08:33.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.411+0000 7ff7ddffb640 1 --2- 172.21.15.49:0/2798849914 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff7cc065a60 0x7ff7cc067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:33.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.411+0000 7ff7ddffb640 1 -- 172.21.15.49:0/2798849914 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff7d8082740 con 0x7ff7e80cb010 2023-11-06T02:08:33.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.411+0000 7ff7ed157640 1 --2- 172.21.15.49:0/2798849914 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff7cc065a60 0x7ff7cc067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:33.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.411+0000 7ff7ed157640 1 --2- 172.21.15.49:0/2798849914 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff7cc065a60 0x7ff7cc067f20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff7e40af000 tx=0x7ff7e4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:33.417 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.415+0000 7ff7ddffb640 1 -- 172.21.15.49:0/2798849914 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7ff7d80500e0 con 0x7ff7e80cb010 2023-11-06T02:08:33.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.607+0000 7ff7ef3bb640 1 -- 172.21.15.49:0/2798849914 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff7e8074040 con 0x7ff7e80cb010 2023-11-06T02:08:33.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.607+0000 7ff7ddffb640 1 -- 172.21.15.49:0/2798849914 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4648 (secure 0 0 0) 0x7ff7d8053700 con 0x7ff7e80cb010 2023-11-06T02:08:33.607 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:08:33.608 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6834/1377586971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1377586971},{"type":"v1","addr":"172.21.15.49:6834","nonce":1377586971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4288,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2620584959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2620584959},{"type":"v1","addr":"172.21.15.150:6835","nonce":2620584959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4291,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/4272191418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":4272191418},{"type":"v1","addr":"172.21.15.150:6837","nonce":4272191418}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T02:08:30.592185+0000","modified":"2023-11-06T02:08:33.245446+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4293},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4293":{"gid":4293,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.49:6835/4280448297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6833","nonce":4280448297},{"type":"v1","addr":"172.21.15.49:6835","nonce":4280448297}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-06T02:08:33.608 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 7 2023-11-06T02:08:33.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.611+0000 7ff7ef3bb640 1 -- 172.21.15.49:0/2798849914 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff7cc065a60 msgr2=0x7ff7cc067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:33.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.611+0000 7ff7ef3bb640 1 --2- 172.21.15.49:0/2798849914 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff7cc065a60 0x7ff7cc067f20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff7e40af000 tx=0x7ff7e4008040 comp rx=0 tx=0).stop 2023-11-06T02:08:33.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.611+0000 7ff7ef3bb640 1 -- 172.21.15.49:0/2798849914 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7e80cb010 msgr2=0x7ff7e8150fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:33.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.611+0000 7ff7ef3bb640 1 --2- 172.21.15.49:0/2798849914 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7e80cb010 0x7ff7e8150fa0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff7d800c500 tx=0x7ff7d800c9c0 comp rx=0 tx=0).stop 2023-11-06T02:08:33.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.611+0000 7ff7ef3bb640 1 -- 172.21.15.49:0/2798849914 shutdown_connections 2023-11-06T02:08:33.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.611+0000 7ff7ef3bb640 1 --2- 172.21.15.49:0/2798849914 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff7cc065a60 0x7ff7cc067f20 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:33.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.611+0000 7ff7ef3bb640 1 --2- 172.21.15.49:0/2798849914 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7e80cb010 0x7ff7e8150fa0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:33.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.611+0000 7ff7ef3bb640 1 --2- 172.21.15.49:0/2798849914 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7e815ab50 0x7ff7e814e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:33.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.611+0000 7ff7ef3bb640 1 --2- 172.21.15.49:0/2798849914 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff7e80b0790 0x7ff7e814a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:33.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.611+0000 7ff7ef3bb640 1 -- 172.21.15.49:0/2798849914 >> 172.21.15.49:0/2798849914 conn(0x7ff7e806c0a0 msgr2=0x7ff7e80c9c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:33.612 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.611+0000 7ff7ef3bb640 1 -- 172.21.15.49:0/2798849914 shutdown_connections 2023-11-06T02:08:33.612 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.611+0000 7ff7ef3bb640 1 -- 172.21.15.49:0/2798849914 wait complete. 2023-11-06T02:08:33.627 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T02:08:33.753 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.755+0000 7fa24aa74640 1 Processor -- start 2023-11-06T02:08:33.753 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.755+0000 7fa24aa74640 1 -- start start 2023-11-06T02:08:33.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.755+0000 7fa24aa74640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fa2440b1f90 0x7fa2440cda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:33.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.755+0000 7fa24aa74640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa244161650 0x7fa2440d1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:33.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.755+0000 7fa24aa74640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa2440d2190 0x7fa2440714a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:33.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.755+0000 7fa24aa74640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2440d17c0 con 0x7fa2440d2190 2023-11-06T02:08:33.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.755+0000 7fa24aa74640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2440ce6c0 con 0x7fa2440b1f90 2023-11-06T02:08:33.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.755+0000 7fa24aa74640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa244161ad0 con 0x7fa244161650 2023-11-06T02:08:33.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.755+0000 7fa249011640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa2440d2190 0x7fa2440714a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:33.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.755+0000 7fa249011640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa2440d2190 0x7fa2440714a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47356/0 (socket says 172.21.15.49:47356) 2023-11-06T02:08:33.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.755+0000 7fa249011640 1 -- 172.21.15.49:0/3339920440 learned_addr learned my addr 172.21.15.49:0/3339920440 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:33.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.755+0000 7fa248810640 1 --2- 172.21.15.49:0/3339920440 >> v2:172.21.15.150:3300/0 conn(0x7fa2440b1f90 0x7fa2440cda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:33.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.755+0000 7fa243fff640 1 --2- 172.21.15.49:0/3339920440 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa244161650 0x7fa2440d1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:33.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.755+0000 7fa249011640 1 -- 172.21.15.49:0/3339920440 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa244161650 msgr2=0x7fa2440d1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:33.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.755+0000 7fa249011640 1 --2- 172.21.15.49:0/3339920440 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa244161650 0x7fa2440d1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:33.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.755+0000 7fa249011640 1 -- 172.21.15.49:0/3339920440 >> v2:172.21.15.150:3300/0 conn(0x7fa2440b1f90 msgr2=0x7fa2440cda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:33.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.755+0000 7fa249011640 1 --2- 172.21.15.49:0/3339920440 >> v2:172.21.15.150:3300/0 conn(0x7fa2440b1f90 0x7fa2440cda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:33.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.755+0000 7fa249011640 1 -- 172.21.15.49:0/3339920440 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2440ce100 con 0x7fa2440d2190 2023-11-06T02:08:33.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.759+0000 7fa249011640 1 --2- 172.21.15.49:0/3339920440 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2440d2190 0x7fa2440714a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa244072180 tx=0x7fa23c01c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dca4edb62d5e0a1 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:33.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.759+0000 7fa2437fe640 1 -- 172.21.15.49:0/3339920440 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa23c0a5730 con 0x7fa2440d2190 2023-11-06T02:08:33.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.759+0000 7fa2437fe640 1 -- 172.21.15.49:0/3339920440 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa23c0a5890 con 0x7fa2440d2190 2023-11-06T02:08:33.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.759+0000 7fa2437fe640 1 -- 172.21.15.49:0/3339920440 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa23c0a5c20 con 0x7fa2440d2190 2023-11-06T02:08:33.762 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.759+0000 7fa24aa74640 1 -- 172.21.15.49:0/3339920440 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2440d2190 msgr2=0x7fa2440714a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:33.762 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.759+0000 7fa24aa74640 1 --2- 172.21.15.49:0/3339920440 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2440d2190 0x7fa2440714a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa244072180 tx=0x7fa23c01c9a0 comp rx=0 tx=0).stop 2023-11-06T02:08:33.762 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.759+0000 7fa24aa74640 1 -- 172.21.15.49:0/3339920440 shutdown_connections 2023-11-06T02:08:33.762 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.759+0000 7fa24aa74640 1 --2- 172.21.15.49:0/3339920440 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa244161650 0x7fa2440d1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:33.762 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.759+0000 7fa24aa74640 1 --2- 172.21.15.49:0/3339920440 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2440d2190 0x7fa2440714a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:33.763 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.759+0000 7fa24aa74640 1 --2- 172.21.15.49:0/3339920440 >> v2:172.21.15.150:3300/0 conn(0x7fa2440b1f90 0x7fa2440cda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:33.763 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.759+0000 7fa24aa74640 1 -- 172.21.15.49:0/3339920440 >> 172.21.15.49:0/3339920440 conn(0x7fa2440d2aa0 msgr2=0x7fa2440d2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:33.763 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.759+0000 7fa24aa74640 1 -- 172.21.15.49:0/3339920440 shutdown_connections 2023-11-06T02:08:33.763 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.759+0000 7fa24aa74640 1 -- 172.21.15.49:0/3339920440 wait complete. 2023-11-06T02:08:33.763 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.759+0000 7fa24aa74640 1 Processor -- start 2023-11-06T02:08:33.764 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.759+0000 7fa24aa74640 1 -- start start 2023-11-06T02:08:33.764 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.759+0000 7fa24aa74640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2440b1f90 0x7fa24414e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:33.764 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.759+0000 7fa24aa74640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2440d2190 0x7fa24414fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:33.764 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.759+0000 7fa24aa74640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa244161650 0x7fa2440b7d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:33.764 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.759+0000 7fa24aa74640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa244152f80 con 0x7fa2440b1f90 2023-11-06T02:08:33.764 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.759+0000 7fa24aa74640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa244150680 con 0x7fa2440d2190 2023-11-06T02:08:33.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.759+0000 7fa24aa74640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa244150960 con 0x7fa244161650 2023-11-06T02:08:33.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.759+0000 7fa248810640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2440b1f90 0x7fa24414e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:33.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.759+0000 7fa248810640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2440b1f90 0x7fa24414e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47372/0 (socket says 172.21.15.49:47372) 2023-11-06T02:08:33.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.759+0000 7fa248810640 1 -- 172.21.15.49:0/8006098 learned_addr learned my addr 172.21.15.49:0/8006098 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:33.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.759+0000 7fa243fff640 1 --2- 172.21.15.49:0/8006098 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2440d2190 0x7fa24414fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:33.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.759+0000 7fa249011640 1 --2- 172.21.15.49:0/8006098 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa244161650 0x7fa2440b7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:33.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.759+0000 7fa248810640 1 -- 172.21.15.49:0/8006098 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa244161650 msgr2=0x7fa2440b7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:33.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.759+0000 7fa248810640 1 --2- 172.21.15.49:0/8006098 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa244161650 0x7fa2440b7d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:33.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.759+0000 7fa248810640 1 -- 172.21.15.49:0/8006098 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2440d2190 msgr2=0x7fa24414fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:33.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.759+0000 7fa248810640 1 --2- 172.21.15.49:0/8006098 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2440d2190 0x7fa24414fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:33.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.759+0000 7fa248810640 1 -- 172.21.15.49:0/8006098 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa23c009b30 con 0x7fa2440b1f90 2023-11-06T02:08:33.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.763+0000 7fa248810640 1 --2- 172.21.15.49:0/8006098 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2440b1f90 0x7fa24414e7c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa23800d800 tx=0x7fa23800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:33.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.763+0000 7fa2417fa640 1 -- 172.21.15.49:0/8006098 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2380096a0 con 0x7fa2440b1f90 2023-11-06T02:08:33.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.763+0000 7fa2417fa640 1 -- 172.21.15.49:0/8006098 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa238009800 con 0x7fa2440b1f90 2023-11-06T02:08:33.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.763+0000 7fa24aa74640 1 -- 172.21.15.49:0/8006098 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2440ca940 con 0x7fa2440b1f90 2023-11-06T02:08:33.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.763+0000 7fa2417fa640 1 -- 172.21.15.49:0/8006098 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa238009b70 con 0x7fa2440b1f90 2023-11-06T02:08:33.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.763+0000 7fa24aa74640 1 -- 172.21.15.49:0/8006098 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa244150c40 con 0x7fa2440b1f90 2023-11-06T02:08:33.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.763+0000 7fa24aa74640 1 -- 172.21.15.49:0/8006098 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2440cc2c0 con 0x7fa2440b1f90 2023-11-06T02:08:33.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.763+0000 7fa2417fa640 1 -- 172.21.15.49:0/8006098 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fa238009cd0 con 0x7fa2440b1f90 2023-11-06T02:08:33.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.763+0000 7fa2417fa640 1 --2- 172.21.15.49:0/8006098 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa230065a90 0x7fa230067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:33.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.763+0000 7fa2417fa640 1 -- 172.21.15.49:0/8006098 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa238083a80 con 0x7fa2440b1f90 2023-11-06T02:08:33.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.763+0000 7fa243fff640 1 --2- 172.21.15.49:0/8006098 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa230065a90 0x7fa230067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:33.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.763+0000 7fa243fff640 1 --2- 172.21.15.49:0/8006098 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa230065a90 0x7fa230067f50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa244158d80 tx=0x7fa234008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:33.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.767+0000 7fa2417fa640 1 -- 172.21.15.49:0/8006098 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fa2380512e0 con 0x7fa2440b1f90 2023-11-06T02:08:33.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.923+0000 7fa24aa74640 1 -- 172.21.15.49:0/8006098 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa244074040 con 0x7fa2440b1f90 2023-11-06T02:08:33.925 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.927+0000 7fa2417fa640 1 -- 172.21.15.49:0/8006098 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+17811 (secure 0 0 0) 0x7fa238054900 con 0x7fa2440b1f90 2023-11-06T02:08:33.925 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:08:33.925 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":23,"fsid":"1a8b035c-6ba4-499a-b876-aea47fd9c0d4","created":"2023-11-06T02:08:06.440857+0000","modified":"2023-11-06T02:08:31.594964+0000","last_up_change":"2023-11-06T02:08:17.982372+0000","last_in_change":"2023-11-06T02:08:08.906631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T02:08:19.252222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T02:08:24.776211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-06T02:08:28.936178+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-06T02:08:29.563468+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ad55af8-0571-4e6a-8722-b5d429b7496d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6809","nonce":38116},{"type":"v1","addr":"172.21.15.49:6811","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6813","nonce":38116},{"type":"v1","addr":"172.21.15.49:6815","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6821","nonce":38116},{"type":"v1","addr":"172.21.15.49:6823","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6817","nonce":38116},{"type":"v1","addr":"172.21.15.49:6819","nonce":38116}]},"public_addr":"172.21.15.49:6811/38116","cluster_addr":"172.21.15.49:6815/38116","heartbeat_back_addr":"172.21.15.49:6823/38116","heartbeat_front_addr":"172.21.15.49:6819/38116","state":["exists","up"]},{"osd":1,"uuid":"c8b31f85-a757-43ed-9419-351acc46611b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":38113},{"type":"v1","addr":"172.21.15.49:6810","nonce":38113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":38113},{"type":"v1","addr":"172.21.15.49:6814","nonce":38113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":38113},{"type":"v1","addr":"172.21.15.49:6822","nonce":38113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":38113},{"type":"v1","addr":"172.21.15.49:6818","nonce":38113}]},"public_addr":"172.21.15.49:6810/38113","cluster_addr":"172.21.15.49:6814/38113","heartbeat_back_addr":"172.21.15.49:6822/38113","heartbeat_front_addr":"172.21.15.49:6818/38113","state":["exists","up"]},{"osd":2,"uuid":"555b3bf3-0a70-4b91-8efd-e4204b5fc26e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":38114},{"type":"v1","addr":"172.21.15.49:6801","nonce":38114}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":38114},{"type":"v1","addr":"172.21.15.49:6803","nonce":38114}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":38114},{"type":"v1","addr":"172.21.15.49:6807","nonce":38114}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":38114},{"type":"v1","addr":"172.21.15.49:6805","nonce":38114}]},"public_addr":"172.21.15.49:6801/38114","cluster_addr":"172.21.15.49:6803/38114","heartbeat_back_addr":"172.21.15.49:6807/38114","heartbeat_front_addr":"172.21.15.49:6805/38114","state":["exists","up"]},{"osd":3,"uuid":"8d4e5d77-03ef-4b9b-bfcb-5ae93a46795c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":38115},{"type":"v1","addr":"172.21.15.49:6825","nonce":38115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":38115},{"type":"v1","addr":"172.21.15.49:6827","nonce":38115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":38115},{"type":"v1","addr":"172.21.15.49:6831","nonce":38115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":38115},{"type":"v1","addr":"172.21.15.49:6829","nonce":38115}]},"public_addr":"172.21.15.49:6825/38115","cluster_addr":"172.21.15.49:6827/38115","heartbeat_back_addr":"172.21.15.49:6831/38115","heartbeat_front_addr":"172.21.15.49:6829/38115","state":["exists","up"]},{"osd":4,"uuid":"dba03ad4-68a3-4cf3-9218-ac611f02eacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":38417},{"type":"v1","addr":"172.21.15.150:6809","nonce":38417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":38417},{"type":"v1","addr":"172.21.15.150:6811","nonce":38417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":38417},{"type":"v1","addr":"172.21.15.150:6815","nonce":38417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":38417},{"type":"v1","addr":"172.21.15.150:6813","nonce":38417}]},"public_addr":"172.21.15.150:6809/38417","cluster_addr":"172.21.15.150:6811/38417","heartbeat_back_addr":"172.21.15.150:6815/38417","heartbeat_front_addr":"172.21.15.150:6813/38417","state":["exists","up"]},{"osd":5,"uuid":"0df9c303-c255-4640-8e91-01f1c3efe18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":38418},{"type":"v1","addr":"172.21.15.150:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":38418},{"type":"v1","addr":"172.21.15.150:6803","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":38418},{"type":"v1","addr":"172.21.15.150:6807","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":38418},{"type":"v1","addr":"172.21.15.150:6805","nonce":38418}]},"public_addr":"172.21.15.150:6801/38418","cluster_addr":"172.21.15.150:6803/38418","heartbeat_back_addr":"172.21.15.150:6807/38418","heartbeat_front_addr":"172.21.15.150:6805/38418","state":["exists","up"]},{"osd":6,"uuid":"e59fa189-4b70-4c06-9722-c02b90828bf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":38428},{"type":"v1","addr":"172.21.15.150:6825","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":38428},{"type":"v1","addr":"172.21.15.150:6827","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":38428},{"type":"v1","addr":"172.21.15.150:6831","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":38428},{"type":"v1","addr":"172.21.15.150:6829","nonce":38428}]},"public_addr":"172.21.15.150:6825/38428","cluster_addr":"172.21.15.150:6827/38428","heartbeat_back_addr":"172.21.15.150:6831/38428","heartbeat_front_addr":"172.21.15.150:6829/38428","state":["exists","up"]},{"osd":7,"uuid":"a64f57e1-ef98-46bd-b304-385759e95898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":38421},{"type":"v1","addr":"172.21.15.150:6817","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":38421},{"type":"v1","addr":"172.21.15.150:6819","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":38421},{"type":"v1","addr":"172.21.15.150:6823","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":38421},{"type":"v1","addr":"172.21.15.150:6821","nonce":38421}]},"public_addr":"172.21.15.150:6817/38421","cluster_addr":"172.21.15.150:6819/38421","heartbeat_back_addr":"172.21.15.150:6823/38421","heartbeat_front_addr":"172.21.15.150:6821/38421","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.438552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.413280+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.296466+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.424966+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.276953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.230984+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.410235+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.480084+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T02:08:33.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.927+0000 7fa24aa74640 1 -- 172.21.15.49:0/8006098 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa230065a90 msgr2=0x7fa230067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:33.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.927+0000 7fa24aa74640 1 --2- 172.21.15.49:0/8006098 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa230065a90 0x7fa230067f50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa244158d80 tx=0x7fa234008040 comp rx=0 tx=0).stop 2023-11-06T02:08:33.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.927+0000 7fa24aa74640 1 -- 172.21.15.49:0/8006098 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2440b1f90 msgr2=0x7fa24414e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:33.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.927+0000 7fa24aa74640 1 --2- 172.21.15.49:0/8006098 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2440b1f90 0x7fa24414e7c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa23800d800 tx=0x7fa23800dcc0 comp rx=0 tx=0).stop 2023-11-06T02:08:33.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.927+0000 7fa24aa74640 1 -- 172.21.15.49:0/8006098 shutdown_connections 2023-11-06T02:08:33.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.927+0000 7fa24aa74640 1 --2- 172.21.15.49:0/8006098 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa230065a90 0x7fa230067f50 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:33.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.927+0000 7fa24aa74640 1 --2- 172.21.15.49:0/8006098 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa244161650 0x7fa2440b7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:33.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.927+0000 7fa24aa74640 1 --2- 172.21.15.49:0/8006098 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2440d2190 0x7fa24414fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:33.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.927+0000 7fa24aa74640 1 --2- 172.21.15.49:0/8006098 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2440b1f90 0x7fa24414e7c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:33.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.927+0000 7fa24aa74640 1 -- 172.21.15.49:0/8006098 >> 172.21.15.49:0/8006098 conn(0x7fa2440d2aa0 msgr2=0x7fa244158f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:33.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.927+0000 7fa24aa74640 1 -- 172.21.15.49:0/8006098 shutdown_connections 2023-11-06T02:08:33.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:33.927+0000 7fa24aa74640 1 -- 172.21.15.49:0/8006098 wait complete. 2023-11-06T02:08:33.943 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-06T02:08:33.944 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T02:08:34.068 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.071+0000 7f8ed4767640 1 Processor -- start 2023-11-06T02:08:34.069 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.071+0000 7f8ed4767640 1 -- start start 2023-11-06T02:08:34.070 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.071+0000 7f8ed4767640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8ecc159980 0x7f8ecc071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:34.071 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.071+0000 7f8ed4767640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f8ecc0b1450 0x7f8ecc0cdd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:34.071 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.071+0000 7f8ed4767640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ecc071890 con 0x7f8ecc0cb010 2023-11-06T02:08:34.071 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.071+0000 7f8ed4767640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ecc072410 con 0x7f8ecc0b1450 2023-11-06T02:08:34.071 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.071+0000 7f8ed4767640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ecc0b1890 con 0x7f8ecc159980 2023-11-06T02:08:34.072 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.071+0000 7f8ed1d02640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f8ecc0b1450 0x7f8ecc0cdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:34.072 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.071+0000 7f8ed1d02640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f8ecc0b1450 0x7f8ecc0cdd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47380/0 (socket says 172.21.15.49:47380) 2023-11-06T02:08:34.072 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.071+0000 7f8ed1d02640 1 -- 172.21.15.49:0/1217462253 learned_addr learned my addr 172.21.15.49:0/1217462253 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:34.072 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.071+0000 7f8ed2503640 1 --2- 172.21.15.49:0/1217462253 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8ecc159980 0x7f8ecc071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:34.072 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.071+0000 7f8ed1d02640 1 -- 172.21.15.49:0/1217462253 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8ecc159980 msgr2=0x7f8ecc071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:34.073 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.071+0000 7f8ed1d02640 1 --2- 172.21.15.49:0/1217462253 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8ecc159980 0x7f8ecc071290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:34.073 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.071+0000 7f8ed1d02640 1 -- 172.21.15.49:0/1217462253 >> v1:172.21.15.49:6789/0 conn(0x7f8ecc0cb010 legacy=0x7f8ecc0cc540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:34.073 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.071+0000 7f8ed1d02640 1 -- 172.21.15.49:0/1217462253 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ecc071b70 con 0x7f8ecc0b1450 2023-11-06T02:08:34.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.071+0000 7f8ed1d02640 1 --2- 172.21.15.49:0/1217462253 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ecc0b1450 0x7f8ecc0cdd30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8ec800bb10 tx=0x7f8ec801ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58fda73885e5a1b6 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:34.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.075+0000 7f8ed1501640 1 -- 172.21.15.49:0/1217462253 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec80a7c50 con 0x7f8ecc0b1450 2023-11-06T02:08:34.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.075+0000 7f8ed1501640 1 -- 172.21.15.49:0/1217462253 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ec80a7db0 con 0x7f8ecc0b1450 2023-11-06T02:08:34.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.075+0000 7f8ed1501640 1 -- 172.21.15.49:0/1217462253 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec80af660 con 0x7f8ecc0b1450 2023-11-06T02:08:34.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.075+0000 7f8ed4767640 1 -- 172.21.15.49:0/1217462253 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ecc0b1450 msgr2=0x7f8ecc0cdd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:34.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.075+0000 7f8ed4767640 1 --2- 172.21.15.49:0/1217462253 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ecc0b1450 0x7f8ecc0cdd30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8ec800bb10 tx=0x7f8ec801ec10 comp rx=0 tx=0).stop 2023-11-06T02:08:34.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.075+0000 7f8ed4767640 1 -- 172.21.15.49:0/1217462253 shutdown_connections 2023-11-06T02:08:34.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.075+0000 7f8ed4767640 1 --2- 172.21.15.49:0/1217462253 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ecc0b1450 0x7f8ecc0cdd30 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:34.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.075+0000 7f8ed4767640 1 --2- 172.21.15.49:0/1217462253 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8ecc159980 0x7f8ecc071290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:34.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.075+0000 7f8ed4767640 1 -- 172.21.15.49:0/1217462253 >> 172.21.15.49:0/1217462253 conn(0x7f8ecc06c0a0 msgr2=0x7f8ecc06c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:34.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.075+0000 7f8ed4767640 1 -- 172.21.15.49:0/1217462253 shutdown_connections 2023-11-06T02:08:34.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.075+0000 7f8ed4767640 1 -- 172.21.15.49:0/1217462253 wait complete. 2023-11-06T02:08:34.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.075+0000 7f8ed4767640 1 Processor -- start 2023-11-06T02:08:34.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.075+0000 7f8ed4767640 1 -- start start 2023-11-06T02:08:34.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.075+0000 7f8ed4767640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ecc0b1450 0x7f8ecc0b3520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:34.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.075+0000 7f8ed4767640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8ecc0cb010 0x7f8ecc146910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:34.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.075+0000 7f8ed4767640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ecc159980 0x7f8ecc156b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:34.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.075+0000 7f8ed4767640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ecc0b1e90 con 0x7f8ecc159980 2023-11-06T02:08:34.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.075+0000 7f8ed4767640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ecc0b2170 con 0x7f8ecc0b1450 2023-11-06T02:08:34.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.075+0000 7f8ed4767640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ecc0b2450 con 0x7f8ecc0cb010 2023-11-06T02:08:34.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.075+0000 7f8ed2d04640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ecc159980 0x7f8ecc156b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:34.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.075+0000 7f8ed2d04640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ecc159980 0x7f8ecc156b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47382/0 (socket says 172.21.15.49:47382) 2023-11-06T02:08:34.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.075+0000 7f8ed2d04640 1 -- 172.21.15.49:0/820989095 learned_addr learned my addr 172.21.15.49:0/820989095 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:34.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.075+0000 7f8ed1d02640 1 --2- 172.21.15.49:0/820989095 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8ecc0cb010 0x7f8ecc146910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:34.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.075+0000 7f8ed2503640 1 --2- 172.21.15.49:0/820989095 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ecc0b1450 0x7f8ecc0b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:34.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.075+0000 7f8ed2d04640 1 -- 172.21.15.49:0/820989095 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8ecc0cb010 msgr2=0x7f8ecc146910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:34.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.075+0000 7f8ed2d04640 1 --2- 172.21.15.49:0/820989095 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8ecc0cb010 0x7f8ecc146910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:34.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.075+0000 7f8ed2d04640 1 -- 172.21.15.49:0/820989095 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ecc0b1450 msgr2=0x7f8ecc0b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:34.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.075+0000 7f8ed2d04640 1 --2- 172.21.15.49:0/820989095 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ecc0b1450 0x7f8ecc0b3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:34.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.075+0000 7f8ed2d04640 1 -- 172.21.15.49:0/820989095 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ec800b770 con 0x7f8ecc159980 2023-11-06T02:08:34.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.075+0000 7f8ed2d04640 1 --2- 172.21.15.49:0/820989095 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ecc159980 0x7f8ecc156b10 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8ec400c770 tx=0x7f8ec400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:34.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.075+0000 7f8ec2ffd640 1 -- 172.21.15.49:0/820989095 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec4003b80 con 0x7f8ecc159980 2023-11-06T02:08:34.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.075+0000 7f8ec2ffd640 1 -- 172.21.15.49:0/820989095 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ec4003ce0 con 0x7f8ecc159980 2023-11-06T02:08:34.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.075+0000 7f8ed4767640 1 -- 172.21.15.49:0/820989095 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ecc0ce3f0 con 0x7f8ecc159980 2023-11-06T02:08:34.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.075+0000 7f8ec2ffd640 1 -- 172.21.15.49:0/820989095 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec400aa90 con 0x7f8ecc159980 2023-11-06T02:08:34.085 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.075+0000 7f8ed4767640 1 -- 172.21.15.49:0/820989095 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ecc0c2130 con 0x7f8ecc159980 2023-11-06T02:08:34.086 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.079+0000 7f8ed4767640 1 -- 172.21.15.49:0/820989095 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ecc074040 con 0x7f8ecc159980 2023-11-06T02:08:34.086 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.079+0000 7f8ec2ffd640 1 -- 172.21.15.49:0/820989095 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f8ec400abf0 con 0x7f8ecc159980 2023-11-06T02:08:34.086 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.079+0000 7f8ec2ffd640 1 --2- 172.21.15.49:0/820989095 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8eb0065a60 0x7f8eb0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:34.087 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.079+0000 7f8ec2ffd640 1 -- 172.21.15.49:0/820989095 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8ec4083090 con 0x7f8ecc159980 2023-11-06T02:08:34.087 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.079+0000 7f8ed2503640 1 --2- 172.21.15.49:0/820989095 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8eb0065a60 0x7f8eb0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:34.087 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.079+0000 7f8ed2503640 1 --2- 172.21.15.49:0/820989095 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8eb0065a60 0x7f8eb0067f20 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc006fd0 tx=0x7f8ebc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:34.087 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.083+0000 7f8ec2ffd640 1 -- 172.21.15.49:0/820989095 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f8ec4050980 con 0x7f8ecc159980 2023-11-06T02:08:34.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.235+0000 7f8ed4767640 1 -- 172.21.15.49:0/820989095 --> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8ecc0ce270 con 0x7f8eb0065a60 2023-11-06T02:08:34.256 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.255+0000 7f8ec2ffd640 1 -- 172.21.15.49:0/820989095 <== mgr.4100 v2:172.21.15.150:6832/38104 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+271909 (secure 0 0 0) 0x7f8ebc00b5e0 con 0x7f8eb0065a60 2023-11-06T02:08:34.259 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:08:34.260 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2023-11-06T02:08:34.262 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.263+0000 7f8ed4767640 1 -- 172.21.15.49:0/820989095 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8eb0065a60 msgr2=0x7f8eb0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:34.262 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.263+0000 7f8ed4767640 1 --2- 172.21.15.49:0/820989095 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8eb0065a60 0x7f8eb0067f20 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc006fd0 tx=0x7f8ebc008040 comp rx=0 tx=0).stop 2023-11-06T02:08:34.262 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.263+0000 7f8ed4767640 1 -- 172.21.15.49:0/820989095 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ecc159980 msgr2=0x7f8ecc156b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:34.263 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.263+0000 7f8ed4767640 1 --2- 172.21.15.49:0/820989095 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ecc159980 0x7f8ecc156b10 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8ec400c770 tx=0x7f8ec400cc30 comp rx=0 tx=0).stop 2023-11-06T02:08:34.263 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.263+0000 7f8ed4767640 1 -- 172.21.15.49:0/820989095 shutdown_connections 2023-11-06T02:08:34.263 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.263+0000 7f8ed4767640 1 --2- 172.21.15.49:0/820989095 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8eb0065a60 0x7f8eb0067f20 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:34.263 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.263+0000 7f8ed4767640 1 --2- 172.21.15.49:0/820989095 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ecc159980 0x7f8ecc156b10 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:34.263 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.263+0000 7f8ed4767640 1 --2- 172.21.15.49:0/820989095 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8ecc0cb010 0x7f8ecc146910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:34.264 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.263+0000 7f8ed4767640 1 --2- 172.21.15.49:0/820989095 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ecc0b1450 0x7f8ecc0b3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:34.264 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.263+0000 7f8ed4767640 1 -- 172.21.15.49:0/820989095 >> 172.21.15.49:0/820989095 conn(0x7f8ecc06c0a0 msgr2=0x7f8ecc154b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:34.264 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.263+0000 7f8ed4767640 1 -- 172.21.15.49:0/820989095 shutdown_connections 2023-11-06T02:08:34.264 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.263+0000 7f8ed4767640 1 -- 172.21.15.49:0/820989095 wait complete. 2023-11-06T02:08:34.279 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-11-06T02:08:33.175077+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":234612,"kb_used_data":2612,"kb_used_omap":0,"kb_used_meta":231936,"kb_avail":754740108,"statfs":{"total":773094113280,"available":772853870592,"internally_reserved":0,"allocated":2674688,"data_stored":1660949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":237502464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001250"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":163,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-11-06T02:08:26.053685+0000","last_change":"2023-11-06T02:08:20.523427+0000","last_active":"2023-11-06T02:08:26.053685+0000","last_peered":"2023-11-06T02:08:26.053685+0000","last_clean":"2023-11-06T02:08:26.053685+0000","last_became_active":"2023-11-06T02:08:20.521089+0000","last_became_peered":"2023-11-06T02:08:20.521089+0000","last_unstale":"2023-11-06T02:08:26.053685+0000","last_undegraded":"2023-11-06T02:08:26.053685+0000","last_fullsized":"2023-11-06T02:08:26.053685+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:19.252348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:19.252348+0000","last_clean_scrub_stamp":"2023-11-06T02:08:19.252348+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:30:15.566397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-06T02:08:27.711809+0000","last_change":"2023-11-06T02:08:26.059502+0000","last_active":"2023-11-06T02:08:27.711809+0000","last_peered":"2023-11-06T02:08:27.711809+0000","last_clean":"2023-11-06T02:08:27.711809+0000","last_became_active":"2023-11-06T02:08:26.058612+0000","last_became_peered":"2023-11-06T02:08:26.058612+0000","last_unstale":"2023-11-06T02:08:27.711809+0000","last_undegraded":"2023-11-06T02:08:27.711809+0000","last_fullsized":"2023-11-06T02:08:27.711809+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:10.626021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-06T02:08:28.038455+0000","last_change":"2023-11-06T02:08:26.060406+0000","last_active":"2023-11-06T02:08:28.038455+0000","last_peered":"2023-11-06T02:08:28.038455+0000","last_clean":"2023-11-06T02:08:28.038455+0000","last_became_active":"2023-11-06T02:08:26.059262+0000","last_became_peered":"2023-11-06T02:08:26.059262+0000","last_unstale":"2023-11-06T02:08:28.038455+0000","last_undegraded":"2023-11-06T02:08:28.038455+0000","last_fullsized":"2023-11-06T02:08:28.038455+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:10.626021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-06T02:08:28.038798+0000","last_change":"2023-11-06T02:08:26.559477+0000","last_active":"2023-11-06T02:08:28.038798+0000","last_peered":"2023-11-06T02:08:28.038798+0000","last_clean":"2023-11-06T02:08:28.038798+0000","last_became_active":"2023-11-06T02:08:26.559202+0000","last_became_peered":"2023-11-06T02:08:26.559202+0000","last_unstale":"2023-11-06T02:08:28.038798+0000","last_undegraded":"2023-11-06T02:08:28.038798+0000","last_fullsized":"2023-11-06T02:08:28.038798+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:55:04.552404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-06T02:08:27.711852+0000","last_change":"2023-11-06T02:08:26.059546+0000","last_active":"2023-11-06T02:08:27.711852+0000","last_peered":"2023-11-06T02:08:27.711852+0000","last_clean":"2023-11-06T02:08:27.711852+0000","last_became_active":"2023-11-06T02:08:26.058806+0000","last_became_peered":"2023-11-06T02:08:26.058806+0000","last_unstale":"2023-11-06T02:08:27.711852+0000","last_undegraded":"2023-11-06T02:08:27.711852+0000","last_fullsized":"2023-11-06T02:08:27.711852+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:23.468510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-06T02:08:27.955849+0000","last_change":"2023-11-06T02:08:26.436463+0000","last_active":"2023-11-06T02:08:27.955849+0000","last_peered":"2023-11-06T02:08:27.955849+0000","last_clean":"2023-11-06T02:08:27.955849+0000","last_became_active":"2023-11-06T02:08:26.436075+0000","last_became_peered":"2023-11-06T02:08:26.436075+0000","last_unstale":"2023-11-06T02:08:27.955849+0000","last_undegraded":"2023-11-06T02:08:27.955849+0000","last_fullsized":"2023-11-06T02:08:27.955849+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:23.468510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-06T02:08:28.038349+0000","last_change":"2023-11-06T02:08:26.436607+0000","last_active":"2023-11-06T02:08:28.038349+0000","last_peered":"2023-11-06T02:08:28.038349+0000","last_clean":"2023-11-06T02:08:28.038349+0000","last_became_active":"2023-11-06T02:08:26.435924+0000","last_became_peered":"2023-11-06T02:08:26.435924+0000","last_unstale":"2023-11-06T02:08:28.038349+0000","last_undegraded":"2023-11-06T02:08:28.038349+0000","last_fullsized":"2023-11-06T02:08:28.038349+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:23.468510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-06T02:08:27.955973+0000","last_change":"2023-11-06T02:08:26.057076+0000","last_active":"2023-11-06T02:08:27.955973+0000","last_peered":"2023-11-06T02:08:27.955973+0000","last_clean":"2023-11-06T02:08:27.955973+0000","last_became_active":"2023-11-06T02:08:26.056419+0000","last_became_peered":"2023-11-06T02:08:26.056419+0000","last_unstale":"2023-11-06T02:08:27.955973+0000","last_undegraded":"2023-11-06T02:08:27.955973+0000","last_fullsized":"2023-11-06T02:08:27.955973+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:10.626021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-06T02:08:27.059583+0000","last_change":"2023-11-06T02:08:26.058596+0000","last_active":"2023-11-06T02:08:27.059583+0000","last_peered":"2023-11-06T02:08:27.059583+0000","last_clean":"2023-11-06T02:08:27.059583+0000","last_became_active":"2023-11-06T02:08:26.057015+0000","last_became_peered":"2023-11-06T02:08:26.057015+0000","last_unstale":"2023-11-06T02:08:27.059583+0000","last_undegraded":"2023-11-06T02:08:27.059583+0000","last_fullsized":"2023-11-06T02:08:27.059583+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:55:04.552404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27708,"kb_used_data":756,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344132,"statfs":{"total":96636764160,"available":96608391168,"internally_reserved":0,"allocated":774144,"data_stored":647648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91700000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1240000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1699999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.206}]}]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27140,"kb_used_data":188,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344700,"statfs":{"total":96636764160,"available":96608972800,"internally_reserved":0,"allocated":192512,"data_stored":63028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.179}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.879}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2909999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4059999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.365}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.23}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27128,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344712,"statfs":{"total":96636764160,"available":96608985088,"internally_reserved":0,"allocated":180224,"data_stored":57280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95499999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0049999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1359999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.26}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90900000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27140,"kb_used_data":188,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344700,"statfs":{"total":96636764160,"available":96608972800,"internally_reserved":0,"allocated":192512,"data_stored":63028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2549999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90800000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95499999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85299999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0469999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.026}]}]},{"osd":3,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31224,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340616,"statfs":{"total":96636764160,"available":96604790784,"internally_reserved":0,"allocated":180224,"data_stored":57280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.079}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65700000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89000000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1160000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.161}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96499999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4910000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31236,"kb_used_data":188,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340604,"statfs":{"total":96636764160,"available":96604778496,"internally_reserved":0,"allocated":192512,"data_stored":63028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.371}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33000000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85799999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71599999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84199999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73399999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31224,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340616,"statfs":{"total":96636764160,"available":96604790784,"internally_reserved":0,"allocated":180224,"data_stored":57280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7210000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5620000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6279999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.486}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31812,"kb_used_data":764,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340028,"statfs":{"total":96636764160,"available":96604188672,"internally_reserved":0,"allocated":782336,"data_stored":652377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93400000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83099999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T02:08:34.287 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T02:08:34.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.439+0000 7f625efd0640 1 Processor -- start 2023-11-06T02:08:34.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.439+0000 7f625efd0640 1 -- start start 2023-11-06T02:08:34.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.439+0000 7f625efd0640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f625815ab50 0x7f62580c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:34.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.439+0000 7f625efd0640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f62580b0790 0x7f62580c2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:34.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.439+0000 7f625efd0640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62580cab80 con 0x7f62580cb010 2023-11-06T02:08:34.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.439+0000 7f625efd0640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f625815af30 con 0x7f62580b0790 2023-11-06T02:08:34.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.439+0000 7f625efd0640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62580cb5b0 con 0x7f625815ab50 2023-11-06T02:08:34.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.439+0000 7f625d56d640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f62580b0790 0x7f62580c2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:34.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.439+0000 7f625d56d640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f62580b0790 0x7f62580c2710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47398/0 (socket says 172.21.15.49:47398) 2023-11-06T02:08:34.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.439+0000 7f625d56d640 1 -- 172.21.15.49:0/686533016 learned_addr learned my addr 172.21.15.49:0/686533016 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:34.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.439+0000 7f625cd6c640 1 --2- 172.21.15.49:0/686533016 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f625815ab50 0x7f62580c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:34.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.439+0000 7f625d56d640 1 -- 172.21.15.49:0/686533016 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f625815ab50 msgr2=0x7f62580c9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:34.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.439+0000 7f625d56d640 1 --2- 172.21.15.49:0/686533016 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f625815ab50 0x7f62580c9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:34.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.439+0000 7f625d56d640 1 -- 172.21.15.49:0/686533016 >> v1:172.21.15.49:6789/0 conn(0x7f62580cb010 legacy=0x7f62580b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:34.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.439+0000 7f625d56d640 1 -- 172.21.15.49:0/686533016 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62580cb890 con 0x7f62580b0790 2023-11-06T02:08:34.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.443+0000 7f625d56d640 1 --2- 172.21.15.49:0/686533016 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62580b0790 0x7f62580c2710 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6250009e50 tx=0x7f625001c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92ddf8748a4fd15e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:34.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.443+0000 7f62577fe640 1 -- 172.21.15.49:0/686533016 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62500a58d0 con 0x7f62580b0790 2023-11-06T02:08:34.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.443+0000 7f62577fe640 1 -- 172.21.15.49:0/686533016 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62500a5a30 con 0x7f62580b0790 2023-11-06T02:08:34.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.443+0000 7f62577fe640 1 -- 172.21.15.49:0/686533016 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62500a5dc0 con 0x7f62580b0790 2023-11-06T02:08:34.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.443+0000 7f625efd0640 1 -- 172.21.15.49:0/686533016 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62580b0790 msgr2=0x7f62580c2710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:34.443 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.443+0000 7f625efd0640 1 --2- 172.21.15.49:0/686533016 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62580b0790 0x7f62580c2710 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6250009e50 tx=0x7f625001c9e0 comp rx=0 tx=0).stop 2023-11-06T02:08:34.443 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.443+0000 7f625efd0640 1 -- 172.21.15.49:0/686533016 shutdown_connections 2023-11-06T02:08:34.443 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.443+0000 7f625efd0640 1 --2- 172.21.15.49:0/686533016 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62580b0790 0x7f62580c2710 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:34.443 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.443+0000 7f625efd0640 1 --2- 172.21.15.49:0/686533016 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f625815ab50 0x7f62580c9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:34.443 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.443+0000 7f625efd0640 1 -- 172.21.15.49:0/686533016 >> 172.21.15.49:0/686533016 conn(0x7f625806c0a0 msgr2=0x7f625806c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:34.444 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.443+0000 7f625efd0640 1 -- 172.21.15.49:0/686533016 shutdown_connections 2023-11-06T02:08:34.444 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.443+0000 7f625efd0640 1 -- 172.21.15.49:0/686533016 wait complete. 2023-11-06T02:08:34.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.443+0000 7f625efd0640 1 Processor -- start 2023-11-06T02:08:34.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.443+0000 7f625efd0640 1 -- start start 2023-11-06T02:08:34.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.443+0000 7f625efd0640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f62580b0790 0x7f6258151440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:34.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.443+0000 7f625efd0640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f62580cb010 0x7f62580ab9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:34.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.443+0000 7f625efd0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f625815ab50 0x7f62580abf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:34.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.443+0000 7f625efd0640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62580c56d0 con 0x7f625815ab50 2023-11-06T02:08:34.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.443+0000 7f625efd0640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6258157d10 con 0x7f62580cb010 2023-11-06T02:08:34.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.443+0000 7f625efd0640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62580c5920 con 0x7f62580b0790 2023-11-06T02:08:34.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.443+0000 7f625d56d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f625815ab50 0x7f62580abf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:34.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.443+0000 7f625d56d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f625815ab50 0x7f62580abf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47404/0 (socket says 172.21.15.49:47404) 2023-11-06T02:08:34.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.443+0000 7f625d56d640 1 -- 172.21.15.49:0/2616871968 learned_addr learned my addr 172.21.15.49:0/2616871968 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:34.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.443+0000 7f625cd6c640 1 --2- 172.21.15.49:0/2616871968 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f62580b0790 0x7f6258151440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:34.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.443+0000 7f6257fff640 1 --2- 172.21.15.49:0/2616871968 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f62580cb010 0x7f62580ab9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:34.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.443+0000 7f625d56d640 1 -- 172.21.15.49:0/2616871968 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f62580b0790 msgr2=0x7f6258151440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:34.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.443+0000 7f625d56d640 1 --2- 172.21.15.49:0/2616871968 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f62580b0790 0x7f6258151440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:34.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.443+0000 7f625d56d640 1 -- 172.21.15.49:0/2616871968 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f62580cb010 msgr2=0x7f62580ab9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:34.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.443+0000 7f625d56d640 1 --2- 172.21.15.49:0/2616871968 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f62580cb010 0x7f62580ab9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:34.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.443+0000 7f625d56d640 1 -- 172.21.15.49:0/2616871968 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6250009b30 con 0x7f625815ab50 2023-11-06T02:08:34.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.447+0000 7f625d56d640 1 --2- 172.21.15.49:0/2616871968 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f625815ab50 0x7f62580abf00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f62500af040 tx=0x7f6250019780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:34.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.447+0000 7f62557fa640 1 -- 172.21.15.49:0/2616871968 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62500b8670 con 0x7f625815ab50 2023-11-06T02:08:34.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.447+0000 7f62557fa640 1 -- 172.21.15.49:0/2616871968 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62500b87d0 con 0x7f625815ab50 2023-11-06T02:08:34.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.447+0000 7f625efd0640 1 -- 172.21.15.49:0/2616871968 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62580c5c00 con 0x7f625815ab50 2023-11-06T02:08:34.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.447+0000 7f62557fa640 1 -- 172.21.15.49:0/2616871968 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62500b8b60 con 0x7f625815ab50 2023-11-06T02:08:34.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.447+0000 7f625efd0640 1 -- 172.21.15.49:0/2616871968 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6258151980 con 0x7f625815ab50 2023-11-06T02:08:34.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.447+0000 7f625efd0640 1 -- 172.21.15.49:0/2616871968 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62580b0de0 con 0x7f625815ab50 2023-11-06T02:08:34.451 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.447+0000 7f62557fa640 1 -- 172.21.15.49:0/2616871968 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f62500b8cc0 con 0x7f625815ab50 2023-11-06T02:08:34.451 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.447+0000 7f62557fa640 1 --2- 172.21.15.49:0/2616871968 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f623c065a60 0x7f623c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:34.452 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.447+0000 7f62557fa640 1 -- 172.21.15.49:0/2616871968 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f625011e850 con 0x7f625815ab50 2023-11-06T02:08:34.452 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.447+0000 7f625cd6c640 1 --2- 172.21.15.49:0/2616871968 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f623c065a60 0x7f623c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:34.452 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.447+0000 7f625cd6c640 1 --2- 172.21.15.49:0/2616871968 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f623c065a60 0x7f623c067f20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f624c006fd0 tx=0x7f624c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:34.452 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.451+0000 7f62557fa640 1 -- 172.21.15.49:0/2616871968 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f62500ec140 con 0x7f625815ab50 2023-11-06T02:08:34.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.603+0000 7f625efd0640 1 -- 172.21.15.49:0/2616871968 --> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f62580cbe50 con 0x7f623c065a60 2023-11-06T02:08:34.622 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.623+0000 7f62557fa640 1 -- 172.21.15.49:0/2616871968 <== mgr.4100 v2:172.21.15.150:6832/38104 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+271909 (secure 0 0 0) 0x7f624c00f860 con 0x7f623c065a60 2023-11-06T02:08:34.624 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:08:34.626 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2023-11-06T02:08:34.627 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.627+0000 7f625efd0640 1 -- 172.21.15.49:0/2616871968 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f623c065a60 msgr2=0x7f623c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:34.627 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.627+0000 7f625efd0640 1 --2- 172.21.15.49:0/2616871968 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f623c065a60 0x7f623c067f20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f624c006fd0 tx=0x7f624c008040 comp rx=0 tx=0).stop 2023-11-06T02:08:34.627 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.627+0000 7f625efd0640 1 -- 172.21.15.49:0/2616871968 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f625815ab50 msgr2=0x7f62580abf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:34.628 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.627+0000 7f625efd0640 1 --2- 172.21.15.49:0/2616871968 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f625815ab50 0x7f62580abf00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f62500af040 tx=0x7f6250019780 comp rx=0 tx=0).stop 2023-11-06T02:08:34.628 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.627+0000 7f625efd0640 1 -- 172.21.15.49:0/2616871968 shutdown_connections 2023-11-06T02:08:34.628 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.627+0000 7f625efd0640 1 --2- 172.21.15.49:0/2616871968 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f623c065a60 0x7f623c067f20 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:34.628 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.627+0000 7f625efd0640 1 --2- 172.21.15.49:0/2616871968 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f625815ab50 0x7f62580abf00 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:34.628 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.627+0000 7f625efd0640 1 --2- 172.21.15.49:0/2616871968 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f62580cb010 0x7f62580ab9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:34.629 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.627+0000 7f625efd0640 1 --2- 172.21.15.49:0/2616871968 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f62580b0790 0x7f6258151440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:34.629 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.627+0000 7f625efd0640 1 -- 172.21.15.49:0/2616871968 >> 172.21.15.49:0/2616871968 conn(0x7f625806c0a0 msgr2=0x7f62580c9bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:34.629 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.627+0000 7f625efd0640 1 -- 172.21.15.49:0/2616871968 shutdown_connections 2023-11-06T02:08:34.629 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.627+0000 7f625efd0640 1 -- 172.21.15.49:0/2616871968 wait complete. 2023-11-06T02:08:34.643 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-11-06T02:08:33.175077+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":234612,"kb_used_data":2612,"kb_used_omap":0,"kb_used_meta":231936,"kb_avail":754740108,"statfs":{"total":773094113280,"available":772853870592,"internally_reserved":0,"allocated":2674688,"data_stored":1660949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":237502464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001250"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":163,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-11-06T02:08:26.053685+0000","last_change":"2023-11-06T02:08:20.523427+0000","last_active":"2023-11-06T02:08:26.053685+0000","last_peered":"2023-11-06T02:08:26.053685+0000","last_clean":"2023-11-06T02:08:26.053685+0000","last_became_active":"2023-11-06T02:08:20.521089+0000","last_became_peered":"2023-11-06T02:08:20.521089+0000","last_unstale":"2023-11-06T02:08:26.053685+0000","last_undegraded":"2023-11-06T02:08:26.053685+0000","last_fullsized":"2023-11-06T02:08:26.053685+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:19.252348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:19.252348+0000","last_clean_scrub_stamp":"2023-11-06T02:08:19.252348+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:30:15.566397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-06T02:08:27.711809+0000","last_change":"2023-11-06T02:08:26.059502+0000","last_active":"2023-11-06T02:08:27.711809+0000","last_peered":"2023-11-06T02:08:27.711809+0000","last_clean":"2023-11-06T02:08:27.711809+0000","last_became_active":"2023-11-06T02:08:26.058612+0000","last_became_peered":"2023-11-06T02:08:26.058612+0000","last_unstale":"2023-11-06T02:08:27.711809+0000","last_undegraded":"2023-11-06T02:08:27.711809+0000","last_fullsized":"2023-11-06T02:08:27.711809+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:10.626021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-06T02:08:28.038455+0000","last_change":"2023-11-06T02:08:26.060406+0000","last_active":"2023-11-06T02:08:28.038455+0000","last_peered":"2023-11-06T02:08:28.038455+0000","last_clean":"2023-11-06T02:08:28.038455+0000","last_became_active":"2023-11-06T02:08:26.059262+0000","last_became_peered":"2023-11-06T02:08:26.059262+0000","last_unstale":"2023-11-06T02:08:28.038455+0000","last_undegraded":"2023-11-06T02:08:28.038455+0000","last_fullsized":"2023-11-06T02:08:28.038455+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:10.626021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-06T02:08:28.038798+0000","last_change":"2023-11-06T02:08:26.559477+0000","last_active":"2023-11-06T02:08:28.038798+0000","last_peered":"2023-11-06T02:08:28.038798+0000","last_clean":"2023-11-06T02:08:28.038798+0000","last_became_active":"2023-11-06T02:08:26.559202+0000","last_became_peered":"2023-11-06T02:08:26.559202+0000","last_unstale":"2023-11-06T02:08:28.038798+0000","last_undegraded":"2023-11-06T02:08:28.038798+0000","last_fullsized":"2023-11-06T02:08:28.038798+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:55:04.552404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-06T02:08:27.711852+0000","last_change":"2023-11-06T02:08:26.059546+0000","last_active":"2023-11-06T02:08:27.711852+0000","last_peered":"2023-11-06T02:08:27.711852+0000","last_clean":"2023-11-06T02:08:27.711852+0000","last_became_active":"2023-11-06T02:08:26.058806+0000","last_became_peered":"2023-11-06T02:08:26.058806+0000","last_unstale":"2023-11-06T02:08:27.711852+0000","last_undegraded":"2023-11-06T02:08:27.711852+0000","last_fullsized":"2023-11-06T02:08:27.711852+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:23.468510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-06T02:08:27.955849+0000","last_change":"2023-11-06T02:08:26.436463+0000","last_active":"2023-11-06T02:08:27.955849+0000","last_peered":"2023-11-06T02:08:27.955849+0000","last_clean":"2023-11-06T02:08:27.955849+0000","last_became_active":"2023-11-06T02:08:26.436075+0000","last_became_peered":"2023-11-06T02:08:26.436075+0000","last_unstale":"2023-11-06T02:08:27.955849+0000","last_undegraded":"2023-11-06T02:08:27.955849+0000","last_fullsized":"2023-11-06T02:08:27.955849+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:23.468510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-06T02:08:28.038349+0000","last_change":"2023-11-06T02:08:26.436607+0000","last_active":"2023-11-06T02:08:28.038349+0000","last_peered":"2023-11-06T02:08:28.038349+0000","last_clean":"2023-11-06T02:08:28.038349+0000","last_became_active":"2023-11-06T02:08:26.435924+0000","last_became_peered":"2023-11-06T02:08:26.435924+0000","last_unstale":"2023-11-06T02:08:28.038349+0000","last_undegraded":"2023-11-06T02:08:28.038349+0000","last_fullsized":"2023-11-06T02:08:28.038349+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:23.468510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-06T02:08:27.955973+0000","last_change":"2023-11-06T02:08:26.057076+0000","last_active":"2023-11-06T02:08:27.955973+0000","last_peered":"2023-11-06T02:08:27.955973+0000","last_clean":"2023-11-06T02:08:27.955973+0000","last_became_active":"2023-11-06T02:08:26.056419+0000","last_became_peered":"2023-11-06T02:08:26.056419+0000","last_unstale":"2023-11-06T02:08:27.955973+0000","last_undegraded":"2023-11-06T02:08:27.955973+0000","last_fullsized":"2023-11-06T02:08:27.955973+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:10.626021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-06T02:08:27.059583+0000","last_change":"2023-11-06T02:08:26.058596+0000","last_active":"2023-11-06T02:08:27.059583+0000","last_peered":"2023-11-06T02:08:27.059583+0000","last_clean":"2023-11-06T02:08:27.059583+0000","last_became_active":"2023-11-06T02:08:26.057015+0000","last_became_peered":"2023-11-06T02:08:26.057015+0000","last_unstale":"2023-11-06T02:08:27.059583+0000","last_undegraded":"2023-11-06T02:08:27.059583+0000","last_fullsized":"2023-11-06T02:08:27.059583+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:55:04.552404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27708,"kb_used_data":756,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344132,"statfs":{"total":96636764160,"available":96608391168,"internally_reserved":0,"allocated":774144,"data_stored":647648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91700000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1240000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1699999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.206}]}]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27140,"kb_used_data":188,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344700,"statfs":{"total":96636764160,"available":96608972800,"internally_reserved":0,"allocated":192512,"data_stored":63028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.179}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.879}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2909999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4059999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.365}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.23}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27128,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344712,"statfs":{"total":96636764160,"available":96608985088,"internally_reserved":0,"allocated":180224,"data_stored":57280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95499999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0049999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1359999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.26}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90900000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27140,"kb_used_data":188,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344700,"statfs":{"total":96636764160,"available":96608972800,"internally_reserved":0,"allocated":192512,"data_stored":63028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2549999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90800000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95499999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85299999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0469999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.026}]}]},{"osd":3,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31224,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340616,"statfs":{"total":96636764160,"available":96604790784,"internally_reserved":0,"allocated":180224,"data_stored":57280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.079}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65700000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89000000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1160000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.161}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96499999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4910000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31236,"kb_used_data":188,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340604,"statfs":{"total":96636764160,"available":96604778496,"internally_reserved":0,"allocated":192512,"data_stored":63028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.371}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33000000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85799999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71599999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84199999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73399999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31224,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340616,"statfs":{"total":96636764160,"available":96604790784,"internally_reserved":0,"allocated":180224,"data_stored":57280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7210000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5620000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6279999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.486}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31812,"kb_used_data":764,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340028,"statfs":{"total":96636764160,"available":96604188672,"internally_reserved":0,"allocated":782336,"data_stored":652377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93400000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83099999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T02:08:34.649 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T02:08:34.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.811+0000 7fb632255640 1 Processor -- start 2023-11-06T02:08:34.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.811+0000 7fb632255640 1 -- start start 2023-11-06T02:08:34.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.811+0000 7fb632255640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb62c0cd400 0x7fb62c160770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:34.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.811+0000 7fb632255640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb62c160cb0 con 0x7fb62c15cd10 2023-11-06T02:08:34.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.811+0000 7fb632255640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb62c0cd870 con 0x7fb62c0709b0 2023-11-06T02:08:34.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.811+0000 7fb632255640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb62c15e0f0 con 0x7fb62c0cd400 2023-11-06T02:08:34.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.811+0000 7fb62affd640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb62c0cd400 0x7fb62c160770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:34.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.811+0000 7fb62affd640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb62c0cd400 0x7fb62c160770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47418/0 (socket says 172.21.15.49:47418) 2023-11-06T02:08:34.812 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.811+0000 7fb62affd640 1 -- 172.21.15.49:0/2141525581 learned_addr learned my addr 172.21.15.49:0/2141525581 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:34.812 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.811+0000 7fb62affd640 1 -- 172.21.15.49:0/2141525581 >> v1:172.21.15.150:6789/0 conn(0x7fb62c0709b0 legacy=0x7fb62c070dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:34.812 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.811+0000 7fb62affd640 1 -- 172.21.15.49:0/2141525581 >> v1:172.21.15.49:6789/0 conn(0x7fb62c15cd10 legacy=0x7fb62c15d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:34.812 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.811+0000 7fb62affd640 1 -- 172.21.15.49:0/2141525581 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb62c160ee0 con 0x7fb62c0cd400 2023-11-06T02:08:34.813 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.815+0000 7fb62affd640 1 --2- 172.21.15.49:0/2141525581 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb62c0cd400 0x7fb62c160770 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb61c009870 tx=0x7fb61c01c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d150456e94d8eb2 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:34.814 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.815+0000 7fb62a7fc640 1 -- 172.21.15.49:0/2141525581 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61c0ab5f0 con 0x7fb62c0cd400 2023-11-06T02:08:34.814 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.815+0000 7fb62a7fc640 1 -- 172.21.15.49:0/2141525581 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb61c0ab750 con 0x7fb62c0cd400 2023-11-06T02:08:34.814 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.815+0000 7fb62a7fc640 1 -- 172.21.15.49:0/2141525581 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61c0abae0 con 0x7fb62c0cd400 2023-11-06T02:08:34.814 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.815+0000 7fb632255640 1 -- 172.21.15.49:0/2141525581 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb62c0cd400 msgr2=0x7fb62c160770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:34.814 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.815+0000 7fb632255640 1 --2- 172.21.15.49:0/2141525581 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb62c0cd400 0x7fb62c160770 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb61c009870 tx=0x7fb61c01c3f0 comp rx=0 tx=0).stop 2023-11-06T02:08:34.815 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.815+0000 7fb632255640 1 -- 172.21.15.49:0/2141525581 shutdown_connections 2023-11-06T02:08:34.815 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.815+0000 7fb632255640 1 --2- 172.21.15.49:0/2141525581 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb62c0cd400 0x7fb62c160770 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:34.815 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.815+0000 7fb632255640 1 -- 172.21.15.49:0/2141525581 >> 172.21.15.49:0/2141525581 conn(0x7fb62c0cee20 msgr2=0x7fb62c0cf230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:34.815 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.815+0000 7fb632255640 1 -- 172.21.15.49:0/2141525581 shutdown_connections 2023-11-06T02:08:34.815 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.815+0000 7fb632255640 1 -- 172.21.15.49:0/2141525581 wait complete. 2023-11-06T02:08:34.818 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.815+0000 7fb632255640 1 Processor -- start 2023-11-06T02:08:34.818 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.815+0000 7fb632255640 1 -- start start 2023-11-06T02:08:34.818 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.815+0000 7fb632255640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb62c0709b0 0x7fb62c0af7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:34.818 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.815+0000 7fb632255640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb62c0cd400 0x7fb62c0acd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:34.818 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.815+0000 7fb632255640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb62c15cd10 0x7fb62c153f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:34.819 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.815+0000 7fb632255640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb62c0b3160 con 0x7fb62c0cd400 2023-11-06T02:08:34.819 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.815+0000 7fb632255640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb62c0b3440 con 0x7fb62c0709b0 2023-11-06T02:08:34.819 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.815+0000 7fb632255640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb62c0b3720 con 0x7fb62c15cd10 2023-11-06T02:08:34.819 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.815+0000 7fb62affd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb62c0cd400 0x7fb62c0acd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:34.819 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.815+0000 7fb62affd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb62c0cd400 0x7fb62c0acd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47424/0 (socket says 172.21.15.49:47424) 2023-11-06T02:08:34.819 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.815+0000 7fb62affd640 1 -- 172.21.15.49:0/759338976 learned_addr learned my addr 172.21.15.49:0/759338976 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:34.820 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.815+0000 7fb62bfff640 1 --2- 172.21.15.49:0/759338976 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb62c15cd10 0x7fb62c153f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:34.820 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.815+0000 7fb62b7fe640 1 --2- 172.21.15.49:0/759338976 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb62c0709b0 0x7fb62c0af7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:34.820 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.815+0000 7fb62affd640 1 -- 172.21.15.49:0/759338976 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb62c15cd10 msgr2=0x7fb62c153f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:34.820 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.815+0000 7fb62affd640 1 --2- 172.21.15.49:0/759338976 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb62c15cd10 0x7fb62c153f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:34.820 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.815+0000 7fb62affd640 1 -- 172.21.15.49:0/759338976 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb62c0709b0 msgr2=0x7fb62c0af7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:34.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.815+0000 7fb62affd640 1 --2- 172.21.15.49:0/759338976 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb62c0709b0 0x7fb62c0af7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:34.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.815+0000 7fb62affd640 1 -- 172.21.15.49:0/759338976 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb61c0094d0 con 0x7fb62c0cd400 2023-11-06T02:08:34.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.815+0000 7fb62bfff640 1 --2- 172.21.15.49:0/759338976 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb62c15cd10 0x7fb62c153f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:08:34.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.815+0000 7fb62b7fe640 1 --2- 172.21.15.49:0/759338976 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb62c0709b0 0x7fb62c0af7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:08:34.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.815+0000 7fb62affd640 1 --2- 172.21.15.49:0/759338976 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb62c0cd400 0x7fb62c0acd70 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb61c0ad040 tx=0x7fb61c01ac30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:34.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.819+0000 7fb623fff640 1 -- 172.21.15.49:0/759338976 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61c0b4800 con 0x7fb62c0cd400 2023-11-06T02:08:34.822 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.819+0000 7fb623fff640 1 -- 172.21.15.49:0/759338976 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb61c0b4980 con 0x7fb62c0cd400 2023-11-06T02:08:34.822 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.819+0000 7fb632255640 1 -- 172.21.15.49:0/759338976 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb62c0b3a00 con 0x7fb62c0cd400 2023-11-06T02:08:34.822 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.819+0000 7fb623fff640 1 -- 172.21.15.49:0/759338976 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61c0b4d30 con 0x7fb62c0cd400 2023-11-06T02:08:34.822 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.819+0000 7fb632255640 1 -- 172.21.15.49:0/759338976 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb62c0b4200 con 0x7fb62c0cd400 2023-11-06T02:08:34.822 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.819+0000 7fb632255640 1 -- 172.21.15.49:0/759338976 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb62c074040 con 0x7fb62c0cd400 2023-11-06T02:08:34.823 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.819+0000 7fb623fff640 1 -- 172.21.15.49:0/759338976 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fb61c0c7440 con 0x7fb62c0cd400 2023-11-06T02:08:34.823 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.819+0000 7fb623fff640 1 --2- 172.21.15.49:0/759338976 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb60c065a60 0x7fb60c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:34.824 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.819+0000 7fb623fff640 1 -- 172.21.15.49:0/759338976 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb61c0a8050 con 0x7fb62c0cd400 2023-11-06T02:08:34.824 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.819+0000 7fb62b7fe640 1 --2- 172.21.15.49:0/759338976 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb60c065a60 0x7fb60c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:34.824 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.819+0000 7fb62b7fe640 1 --2- 172.21.15.49:0/759338976 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb60c065a60 0x7fb60c067f20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb614000f70 tx=0x7fb614009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:34.824 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.823+0000 7fb623fff640 1 -- 172.21.15.49:0/759338976 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fb61c0eb640 con 0x7fb62c0cd400 2023-11-06T02:08:34.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.971+0000 7fb632255640 1 -- 172.21.15.49:0/759338976 --> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb62c15d4c0 con 0x7fb60c065a60 2023-11-06T02:08:34.990 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.995+0000 7fb623fff640 1 -- 172.21.15.49:0/759338976 <== mgr.4100 v2:172.21.15.150:6832/38104 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+271909 (secure 0 0 0) 0x7fb61400d520 con 0x7fb60c065a60 2023-11-06T02:08:34.993 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:08:34.994 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2023-11-06T02:08:34.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.995+0000 7fb632255640 1 -- 172.21.15.49:0/759338976 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb60c065a60 msgr2=0x7fb60c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:34.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.995+0000 7fb632255640 1 --2- 172.21.15.49:0/759338976 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb60c065a60 0x7fb60c067f20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb614000f70 tx=0x7fb614009040 comp rx=0 tx=0).stop 2023-11-06T02:08:34.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.995+0000 7fb632255640 1 -- 172.21.15.49:0/759338976 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb62c0cd400 msgr2=0x7fb62c0acd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:34.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.995+0000 7fb632255640 1 --2- 172.21.15.49:0/759338976 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb62c0cd400 0x7fb62c0acd70 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb61c0ad040 tx=0x7fb61c01ac30 comp rx=0 tx=0).stop 2023-11-06T02:08:34.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.995+0000 7fb632255640 1 -- 172.21.15.49:0/759338976 shutdown_connections 2023-11-06T02:08:34.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.995+0000 7fb632255640 1 --2- 172.21.15.49:0/759338976 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb60c065a60 0x7fb60c067f20 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:34.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.995+0000 7fb632255640 1 --2- 172.21.15.49:0/759338976 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb62c0cd400 0x7fb62c0acd70 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:34.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.995+0000 7fb632255640 1 --2- 172.21.15.49:0/759338976 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb62c15cd10 0x7fb62c153f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:34.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.995+0000 7fb632255640 1 --2- 172.21.15.49:0/759338976 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb62c0709b0 0x7fb62c0af7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:34.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.995+0000 7fb632255640 1 -- 172.21.15.49:0/759338976 >> 172.21.15.49:0/759338976 conn(0x7fb62c0cee20 msgr2=0x7fb62c158190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:34.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.995+0000 7fb632255640 1 -- 172.21.15.49:0/759338976 shutdown_connections 2023-11-06T02:08:34.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:34.995+0000 7fb632255640 1 -- 172.21.15.49:0/759338976 wait complete. 2023-11-06T02:08:35.009 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-11-06T02:08:33.175077+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":234612,"kb_used_data":2612,"kb_used_omap":0,"kb_used_meta":231936,"kb_avail":754740108,"statfs":{"total":773094113280,"available":772853870592,"internally_reserved":0,"allocated":2674688,"data_stored":1660949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":237502464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001250"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":163,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-11-06T02:08:26.053685+0000","last_change":"2023-11-06T02:08:20.523427+0000","last_active":"2023-11-06T02:08:26.053685+0000","last_peered":"2023-11-06T02:08:26.053685+0000","last_clean":"2023-11-06T02:08:26.053685+0000","last_became_active":"2023-11-06T02:08:20.521089+0000","last_became_peered":"2023-11-06T02:08:20.521089+0000","last_unstale":"2023-11-06T02:08:26.053685+0000","last_undegraded":"2023-11-06T02:08:26.053685+0000","last_fullsized":"2023-11-06T02:08:26.053685+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:19.252348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:19.252348+0000","last_clean_scrub_stamp":"2023-11-06T02:08:19.252348+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:30:15.566397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-06T02:08:27.711809+0000","last_change":"2023-11-06T02:08:26.059502+0000","last_active":"2023-11-06T02:08:27.711809+0000","last_peered":"2023-11-06T02:08:27.711809+0000","last_clean":"2023-11-06T02:08:27.711809+0000","last_became_active":"2023-11-06T02:08:26.058612+0000","last_became_peered":"2023-11-06T02:08:26.058612+0000","last_unstale":"2023-11-06T02:08:27.711809+0000","last_undegraded":"2023-11-06T02:08:27.711809+0000","last_fullsized":"2023-11-06T02:08:27.711809+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:10.626021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-06T02:08:28.038455+0000","last_change":"2023-11-06T02:08:26.060406+0000","last_active":"2023-11-06T02:08:28.038455+0000","last_peered":"2023-11-06T02:08:28.038455+0000","last_clean":"2023-11-06T02:08:28.038455+0000","last_became_active":"2023-11-06T02:08:26.059262+0000","last_became_peered":"2023-11-06T02:08:26.059262+0000","last_unstale":"2023-11-06T02:08:28.038455+0000","last_undegraded":"2023-11-06T02:08:28.038455+0000","last_fullsized":"2023-11-06T02:08:28.038455+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:10.626021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-06T02:08:28.038798+0000","last_change":"2023-11-06T02:08:26.559477+0000","last_active":"2023-11-06T02:08:28.038798+0000","last_peered":"2023-11-06T02:08:28.038798+0000","last_clean":"2023-11-06T02:08:28.038798+0000","last_became_active":"2023-11-06T02:08:26.559202+0000","last_became_peered":"2023-11-06T02:08:26.559202+0000","last_unstale":"2023-11-06T02:08:28.038798+0000","last_undegraded":"2023-11-06T02:08:28.038798+0000","last_fullsized":"2023-11-06T02:08:28.038798+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:55:04.552404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-06T02:08:27.711852+0000","last_change":"2023-11-06T02:08:26.059546+0000","last_active":"2023-11-06T02:08:27.711852+0000","last_peered":"2023-11-06T02:08:27.711852+0000","last_clean":"2023-11-06T02:08:27.711852+0000","last_became_active":"2023-11-06T02:08:26.058806+0000","last_became_peered":"2023-11-06T02:08:26.058806+0000","last_unstale":"2023-11-06T02:08:27.711852+0000","last_undegraded":"2023-11-06T02:08:27.711852+0000","last_fullsized":"2023-11-06T02:08:27.711852+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:23.468510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-06T02:08:27.955849+0000","last_change":"2023-11-06T02:08:26.436463+0000","last_active":"2023-11-06T02:08:27.955849+0000","last_peered":"2023-11-06T02:08:27.955849+0000","last_clean":"2023-11-06T02:08:27.955849+0000","last_became_active":"2023-11-06T02:08:26.436075+0000","last_became_peered":"2023-11-06T02:08:26.436075+0000","last_unstale":"2023-11-06T02:08:27.955849+0000","last_undegraded":"2023-11-06T02:08:27.955849+0000","last_fullsized":"2023-11-06T02:08:27.955849+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:23.468510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-06T02:08:28.038349+0000","last_change":"2023-11-06T02:08:26.436607+0000","last_active":"2023-11-06T02:08:28.038349+0000","last_peered":"2023-11-06T02:08:28.038349+0000","last_clean":"2023-11-06T02:08:28.038349+0000","last_became_active":"2023-11-06T02:08:26.435924+0000","last_became_peered":"2023-11-06T02:08:26.435924+0000","last_unstale":"2023-11-06T02:08:28.038349+0000","last_undegraded":"2023-11-06T02:08:28.038349+0000","last_fullsized":"2023-11-06T02:08:28.038349+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:23.468510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-06T02:08:27.955973+0000","last_change":"2023-11-06T02:08:26.057076+0000","last_active":"2023-11-06T02:08:27.955973+0000","last_peered":"2023-11-06T02:08:27.955973+0000","last_clean":"2023-11-06T02:08:27.955973+0000","last_became_active":"2023-11-06T02:08:26.056419+0000","last_became_peered":"2023-11-06T02:08:26.056419+0000","last_unstale":"2023-11-06T02:08:27.955973+0000","last_undegraded":"2023-11-06T02:08:27.955973+0000","last_fullsized":"2023-11-06T02:08:27.955973+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:10.626021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-06T02:08:27.059583+0000","last_change":"2023-11-06T02:08:26.058596+0000","last_active":"2023-11-06T02:08:27.059583+0000","last_peered":"2023-11-06T02:08:27.059583+0000","last_clean":"2023-11-06T02:08:27.059583+0000","last_became_active":"2023-11-06T02:08:26.057015+0000","last_became_peered":"2023-11-06T02:08:26.057015+0000","last_unstale":"2023-11-06T02:08:27.059583+0000","last_undegraded":"2023-11-06T02:08:27.059583+0000","last_fullsized":"2023-11-06T02:08:27.059583+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:55:04.552404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:30.181639+0000","last_change":"2023-11-06T02:08:30.181639+0000","last_active":"2023-11-06T02:08:30.181639+0000","last_peered":"2023-11-06T02:08:30.181639+0000","last_clean":"2023-11-06T02:08:30.181639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:30.181639+0000","last_undegraded":"2023-11-06T02:08:30.181639+0000","last_fullsized":"2023-11-06T02:08:30.181639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:08:29.177803+0000","last_change":"2023-11-06T02:08:29.177803+0000","last_active":"2023-11-06T02:08:29.177803+0000","last_peered":"2023-11-06T02:08:29.177803+0000","last_clean":"2023-11-06T02:08:29.177803+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:08:29.177803+0000","last_undegraded":"2023-11-06T02:08:29.177803+0000","last_fullsized":"2023-11-06T02:08:29.177803+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27708,"kb_used_data":756,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344132,"statfs":{"total":96636764160,"available":96608391168,"internally_reserved":0,"allocated":774144,"data_stored":647648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91700000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1240000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1699999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.206}]}]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27140,"kb_used_data":188,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344700,"statfs":{"total":96636764160,"available":96608972800,"internally_reserved":0,"allocated":192512,"data_stored":63028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.179}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.879}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2909999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4059999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.365}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.23}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27128,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344712,"statfs":{"total":96636764160,"available":96608985088,"internally_reserved":0,"allocated":180224,"data_stored":57280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95499999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0049999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1359999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.26}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90900000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27140,"kb_used_data":188,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344700,"statfs":{"total":96636764160,"available":96608972800,"internally_reserved":0,"allocated":192512,"data_stored":63028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2549999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90800000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95499999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85299999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0469999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.026}]}]},{"osd":3,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31224,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340616,"statfs":{"total":96636764160,"available":96604790784,"internally_reserved":0,"allocated":180224,"data_stored":57280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.079}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65700000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89000000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1160000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.161}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96499999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4910000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31236,"kb_used_data":188,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340604,"statfs":{"total":96636764160,"available":96604778496,"internally_reserved":0,"allocated":192512,"data_stored":63028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.371}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33000000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85799999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71599999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84199999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73399999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31224,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340616,"statfs":{"total":96636764160,"available":96604790784,"internally_reserved":0,"allocated":180224,"data_stored":57280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7210000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5620000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6279999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.486}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31812,"kb_used_data":764,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340028,"statfs":{"total":96636764160,"available":96604188672,"internally_reserved":0,"allocated":782336,"data_stored":652377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93400000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83099999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T02:08:35.014 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.3f is not active+clean 2023-11-06T02:08:35.015 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:30.181639+0000', 'last_change': '2023-11-06T02:08:30.181639+0000', 'last_active': '2023-11-06T02:08:30.181639+0000', 'last_peered': '2023-11-06T02:08:30.181639+0000', 'last_clean': '2023-11-06T02:08:30.181639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:30.181639+0000', 'last_undegraded': '2023-11-06T02:08:30.181639+0000', 'last_fullsized': '2023-11-06T02:08:30.181639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.015 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.3e is not active+clean 2023-11-06T02:08:35.016 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:30.181639+0000', 'last_change': '2023-11-06T02:08:30.181639+0000', 'last_active': '2023-11-06T02:08:30.181639+0000', 'last_peered': '2023-11-06T02:08:30.181639+0000', 'last_clean': '2023-11-06T02:08:30.181639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:30.181639+0000', 'last_undegraded': '2023-11-06T02:08:30.181639+0000', 'last_fullsized': '2023-11-06T02:08:30.181639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.016 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.3d is not active+clean 2023-11-06T02:08:35.016 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:30.181639+0000', 'last_change': '2023-11-06T02:08:30.181639+0000', 'last_active': '2023-11-06T02:08:30.181639+0000', 'last_peered': '2023-11-06T02:08:30.181639+0000', 'last_clean': '2023-11-06T02:08:30.181639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:30.181639+0000', 'last_undegraded': '2023-11-06T02:08:30.181639+0000', 'last_fullsized': '2023-11-06T02:08:30.181639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.017 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.3c is not active+clean 2023-11-06T02:08:35.017 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:30.181639+0000', 'last_change': '2023-11-06T02:08:30.181639+0000', 'last_active': '2023-11-06T02:08:30.181639+0000', 'last_peered': '2023-11-06T02:08:30.181639+0000', 'last_clean': '2023-11-06T02:08:30.181639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:30.181639+0000', 'last_undegraded': '2023-11-06T02:08:30.181639+0000', 'last_fullsized': '2023-11-06T02:08:30.181639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.017 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.3b is not active+clean 2023-11-06T02:08:35.018 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:30.181639+0000', 'last_change': '2023-11-06T02:08:30.181639+0000', 'last_active': '2023-11-06T02:08:30.181639+0000', 'last_peered': '2023-11-06T02:08:30.181639+0000', 'last_clean': '2023-11-06T02:08:30.181639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:30.181639+0000', 'last_undegraded': '2023-11-06T02:08:30.181639+0000', 'last_fullsized': '2023-11-06T02:08:30.181639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.018 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.3a is not active+clean 2023-11-06T02:08:35.018 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:30.181639+0000', 'last_change': '2023-11-06T02:08:30.181639+0000', 'last_active': '2023-11-06T02:08:30.181639+0000', 'last_peered': '2023-11-06T02:08:30.181639+0000', 'last_clean': '2023-11-06T02:08:30.181639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:30.181639+0000', 'last_undegraded': '2023-11-06T02:08:30.181639+0000', 'last_fullsized': '2023-11-06T02:08:30.181639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.019 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.39 is not active+clean 2023-11-06T02:08:35.019 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:30.181639+0000', 'last_change': '2023-11-06T02:08:30.181639+0000', 'last_active': '2023-11-06T02:08:30.181639+0000', 'last_peered': '2023-11-06T02:08:30.181639+0000', 'last_clean': '2023-11-06T02:08:30.181639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:30.181639+0000', 'last_undegraded': '2023-11-06T02:08:30.181639+0000', 'last_fullsized': '2023-11-06T02:08:30.181639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.019 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.38 is not active+clean 2023-11-06T02:08:35.020 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:30.181639+0000', 'last_change': '2023-11-06T02:08:30.181639+0000', 'last_active': '2023-11-06T02:08:30.181639+0000', 'last_peered': '2023-11-06T02:08:30.181639+0000', 'last_clean': '2023-11-06T02:08:30.181639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:30.181639+0000', 'last_undegraded': '2023-11-06T02:08:30.181639+0000', 'last_fullsized': '2023-11-06T02:08:30.181639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.020 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.37 is not active+clean 2023-11-06T02:08:35.020 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:30.181639+0000', 'last_change': '2023-11-06T02:08:30.181639+0000', 'last_active': '2023-11-06T02:08:30.181639+0000', 'last_peered': '2023-11-06T02:08:30.181639+0000', 'last_clean': '2023-11-06T02:08:30.181639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:30.181639+0000', 'last_undegraded': '2023-11-06T02:08:30.181639+0000', 'last_fullsized': '2023-11-06T02:08:30.181639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.021 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.36 is not active+clean 2023-11-06T02:08:35.021 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:30.181639+0000', 'last_change': '2023-11-06T02:08:30.181639+0000', 'last_active': '2023-11-06T02:08:30.181639+0000', 'last_peered': '2023-11-06T02:08:30.181639+0000', 'last_clean': '2023-11-06T02:08:30.181639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:30.181639+0000', 'last_undegraded': '2023-11-06T02:08:30.181639+0000', 'last_fullsized': '2023-11-06T02:08:30.181639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.021 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.35 is not active+clean 2023-11-06T02:08:35.022 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:30.181639+0000', 'last_change': '2023-11-06T02:08:30.181639+0000', 'last_active': '2023-11-06T02:08:30.181639+0000', 'last_peered': '2023-11-06T02:08:30.181639+0000', 'last_clean': '2023-11-06T02:08:30.181639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:30.181639+0000', 'last_undegraded': '2023-11-06T02:08:30.181639+0000', 'last_fullsized': '2023-11-06T02:08:30.181639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.022 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.34 is not active+clean 2023-11-06T02:08:35.022 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:30.181639+0000', 'last_change': '2023-11-06T02:08:30.181639+0000', 'last_active': '2023-11-06T02:08:30.181639+0000', 'last_peered': '2023-11-06T02:08:30.181639+0000', 'last_clean': '2023-11-06T02:08:30.181639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:30.181639+0000', 'last_undegraded': '2023-11-06T02:08:30.181639+0000', 'last_fullsized': '2023-11-06T02:08:30.181639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.023 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.33 is not active+clean 2023-11-06T02:08:35.023 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:30.181639+0000', 'last_change': '2023-11-06T02:08:30.181639+0000', 'last_active': '2023-11-06T02:08:30.181639+0000', 'last_peered': '2023-11-06T02:08:30.181639+0000', 'last_clean': '2023-11-06T02:08:30.181639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:30.181639+0000', 'last_undegraded': '2023-11-06T02:08:30.181639+0000', 'last_fullsized': '2023-11-06T02:08:30.181639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.023 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.32 is not active+clean 2023-11-06T02:08:35.024 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:30.181639+0000', 'last_change': '2023-11-06T02:08:30.181639+0000', 'last_active': '2023-11-06T02:08:30.181639+0000', 'last_peered': '2023-11-06T02:08:30.181639+0000', 'last_clean': '2023-11-06T02:08:30.181639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:30.181639+0000', 'last_undegraded': '2023-11-06T02:08:30.181639+0000', 'last_fullsized': '2023-11-06T02:08:30.181639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.024 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.31 is not active+clean 2023-11-06T02:08:35.024 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:30.181639+0000', 'last_change': '2023-11-06T02:08:30.181639+0000', 'last_active': '2023-11-06T02:08:30.181639+0000', 'last_peered': '2023-11-06T02:08:30.181639+0000', 'last_clean': '2023-11-06T02:08:30.181639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:30.181639+0000', 'last_undegraded': '2023-11-06T02:08:30.181639+0000', 'last_fullsized': '2023-11-06T02:08:30.181639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.025 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.30 is not active+clean 2023-11-06T02:08:35.025 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:30.181639+0000', 'last_change': '2023-11-06T02:08:30.181639+0000', 'last_active': '2023-11-06T02:08:30.181639+0000', 'last_peered': '2023-11-06T02:08:30.181639+0000', 'last_clean': '2023-11-06T02:08:30.181639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:30.181639+0000', 'last_undegraded': '2023-11-06T02:08:30.181639+0000', 'last_fullsized': '2023-11-06T02:08:30.181639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.025 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.2f is not active+clean 2023-11-06T02:08:35.025 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:30.181639+0000', 'last_change': '2023-11-06T02:08:30.181639+0000', 'last_active': '2023-11-06T02:08:30.181639+0000', 'last_peered': '2023-11-06T02:08:30.181639+0000', 'last_clean': '2023-11-06T02:08:30.181639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:30.181639+0000', 'last_undegraded': '2023-11-06T02:08:30.181639+0000', 'last_fullsized': '2023-11-06T02:08:30.181639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.026 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.2e is not active+clean 2023-11-06T02:08:35.026 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:30.181639+0000', 'last_change': '2023-11-06T02:08:30.181639+0000', 'last_active': '2023-11-06T02:08:30.181639+0000', 'last_peered': '2023-11-06T02:08:30.181639+0000', 'last_clean': '2023-11-06T02:08:30.181639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:30.181639+0000', 'last_undegraded': '2023-11-06T02:08:30.181639+0000', 'last_fullsized': '2023-11-06T02:08:30.181639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.027 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.2d is not active+clean 2023-11-06T02:08:35.027 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:30.181639+0000', 'last_change': '2023-11-06T02:08:30.181639+0000', 'last_active': '2023-11-06T02:08:30.181639+0000', 'last_peered': '2023-11-06T02:08:30.181639+0000', 'last_clean': '2023-11-06T02:08:30.181639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:30.181639+0000', 'last_undegraded': '2023-11-06T02:08:30.181639+0000', 'last_fullsized': '2023-11-06T02:08:30.181639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.027 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.2c is not active+clean 2023-11-06T02:08:35.027 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:30.181639+0000', 'last_change': '2023-11-06T02:08:30.181639+0000', 'last_active': '2023-11-06T02:08:30.181639+0000', 'last_peered': '2023-11-06T02:08:30.181639+0000', 'last_clean': '2023-11-06T02:08:30.181639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:30.181639+0000', 'last_undegraded': '2023-11-06T02:08:30.181639+0000', 'last_fullsized': '2023-11-06T02:08:30.181639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.028 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.2b is not active+clean 2023-11-06T02:08:35.028 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:30.181639+0000', 'last_change': '2023-11-06T02:08:30.181639+0000', 'last_active': '2023-11-06T02:08:30.181639+0000', 'last_peered': '2023-11-06T02:08:30.181639+0000', 'last_clean': '2023-11-06T02:08:30.181639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:30.181639+0000', 'last_undegraded': '2023-11-06T02:08:30.181639+0000', 'last_fullsized': '2023-11-06T02:08:30.181639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.028 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.2a is not active+clean 2023-11-06T02:08:35.029 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:30.181639+0000', 'last_change': '2023-11-06T02:08:30.181639+0000', 'last_active': '2023-11-06T02:08:30.181639+0000', 'last_peered': '2023-11-06T02:08:30.181639+0000', 'last_clean': '2023-11-06T02:08:30.181639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:30.181639+0000', 'last_undegraded': '2023-11-06T02:08:30.181639+0000', 'last_fullsized': '2023-11-06T02:08:30.181639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.029 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.29 is not active+clean 2023-11-06T02:08:35.029 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:30.181639+0000', 'last_change': '2023-11-06T02:08:30.181639+0000', 'last_active': '2023-11-06T02:08:30.181639+0000', 'last_peered': '2023-11-06T02:08:30.181639+0000', 'last_clean': '2023-11-06T02:08:30.181639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:30.181639+0000', 'last_undegraded': '2023-11-06T02:08:30.181639+0000', 'last_fullsized': '2023-11-06T02:08:30.181639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.030 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.28 is not active+clean 2023-11-06T02:08:35.030 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:30.181639+0000', 'last_change': '2023-11-06T02:08:30.181639+0000', 'last_active': '2023-11-06T02:08:30.181639+0000', 'last_peered': '2023-11-06T02:08:30.181639+0000', 'last_clean': '2023-11-06T02:08:30.181639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:30.181639+0000', 'last_undegraded': '2023-11-06T02:08:30.181639+0000', 'last_fullsized': '2023-11-06T02:08:30.181639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.030 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.27 is not active+clean 2023-11-06T02:08:35.031 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:30.181639+0000', 'last_change': '2023-11-06T02:08:30.181639+0000', 'last_active': '2023-11-06T02:08:30.181639+0000', 'last_peered': '2023-11-06T02:08:30.181639+0000', 'last_clean': '2023-11-06T02:08:30.181639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:30.181639+0000', 'last_undegraded': '2023-11-06T02:08:30.181639+0000', 'last_fullsized': '2023-11-06T02:08:30.181639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.031 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.26 is not active+clean 2023-11-06T02:08:35.031 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:30.181639+0000', 'last_change': '2023-11-06T02:08:30.181639+0000', 'last_active': '2023-11-06T02:08:30.181639+0000', 'last_peered': '2023-11-06T02:08:30.181639+0000', 'last_clean': '2023-11-06T02:08:30.181639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:30.181639+0000', 'last_undegraded': '2023-11-06T02:08:30.181639+0000', 'last_fullsized': '2023-11-06T02:08:30.181639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.032 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.25 is not active+clean 2023-11-06T02:08:35.032 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:30.181639+0000', 'last_change': '2023-11-06T02:08:30.181639+0000', 'last_active': '2023-11-06T02:08:30.181639+0000', 'last_peered': '2023-11-06T02:08:30.181639+0000', 'last_clean': '2023-11-06T02:08:30.181639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:30.181639+0000', 'last_undegraded': '2023-11-06T02:08:30.181639+0000', 'last_fullsized': '2023-11-06T02:08:30.181639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.032 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.24 is not active+clean 2023-11-06T02:08:35.033 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:30.181639+0000', 'last_change': '2023-11-06T02:08:30.181639+0000', 'last_active': '2023-11-06T02:08:30.181639+0000', 'last_peered': '2023-11-06T02:08:30.181639+0000', 'last_clean': '2023-11-06T02:08:30.181639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:30.181639+0000', 'last_undegraded': '2023-11-06T02:08:30.181639+0000', 'last_fullsized': '2023-11-06T02:08:30.181639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.033 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.23 is not active+clean 2023-11-06T02:08:35.033 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:30.181639+0000', 'last_change': '2023-11-06T02:08:30.181639+0000', 'last_active': '2023-11-06T02:08:30.181639+0000', 'last_peered': '2023-11-06T02:08:30.181639+0000', 'last_clean': '2023-11-06T02:08:30.181639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:30.181639+0000', 'last_undegraded': '2023-11-06T02:08:30.181639+0000', 'last_fullsized': '2023-11-06T02:08:30.181639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.034 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.22 is not active+clean 2023-11-06T02:08:35.034 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:30.181639+0000', 'last_change': '2023-11-06T02:08:30.181639+0000', 'last_active': '2023-11-06T02:08:30.181639+0000', 'last_peered': '2023-11-06T02:08:30.181639+0000', 'last_clean': '2023-11-06T02:08:30.181639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:30.181639+0000', 'last_undegraded': '2023-11-06T02:08:30.181639+0000', 'last_fullsized': '2023-11-06T02:08:30.181639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.034 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.21 is not active+clean 2023-11-06T02:08:35.035 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:30.181639+0000', 'last_change': '2023-11-06T02:08:30.181639+0000', 'last_active': '2023-11-06T02:08:30.181639+0000', 'last_peered': '2023-11-06T02:08:30.181639+0000', 'last_clean': '2023-11-06T02:08:30.181639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:30.181639+0000', 'last_undegraded': '2023-11-06T02:08:30.181639+0000', 'last_fullsized': '2023-11-06T02:08:30.181639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.035 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.20 is not active+clean 2023-11-06T02:08:35.035 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:30.181639+0000', 'last_change': '2023-11-06T02:08:30.181639+0000', 'last_active': '2023-11-06T02:08:30.181639+0000', 'last_peered': '2023-11-06T02:08:30.181639+0000', 'last_clean': '2023-11-06T02:08:30.181639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:30.181639+0000', 'last_undegraded': '2023-11-06T02:08:30.181639+0000', 'last_fullsized': '2023-11-06T02:08:30.181639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.036 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.14 is not active+clean 2023-11-06T02:08:35.036 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:30.181639+0000', 'last_change': '2023-11-06T02:08:30.181639+0000', 'last_active': '2023-11-06T02:08:30.181639+0000', 'last_peered': '2023-11-06T02:08:30.181639+0000', 'last_clean': '2023-11-06T02:08:30.181639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:30.181639+0000', 'last_undegraded': '2023-11-06T02:08:30.181639+0000', 'last_fullsized': '2023-11-06T02:08:30.181639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.036 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.13 is not active+clean 2023-11-06T02:08:35.037 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:29.177803+0000', 'last_change': '2023-11-06T02:08:29.177803+0000', 'last_active': '2023-11-06T02:08:29.177803+0000', 'last_peered': '2023-11-06T02:08:29.177803+0000', 'last_clean': '2023-11-06T02:08:29.177803+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:29.177803+0000', 'last_undegraded': '2023-11-06T02:08:29.177803+0000', 'last_fullsized': '2023-11-06T02:08:29.177803+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.037 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.15 is not active+clean 2023-11-06T02:08:35.037 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:30.181639+0000', 'last_change': '2023-11-06T02:08:30.181639+0000', 'last_active': '2023-11-06T02:08:30.181639+0000', 'last_peered': '2023-11-06T02:08:30.181639+0000', 'last_clean': '2023-11-06T02:08:30.181639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:30.181639+0000', 'last_undegraded': '2023-11-06T02:08:30.181639+0000', 'last_fullsized': '2023-11-06T02:08:30.181639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.038 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.12 is not active+clean 2023-11-06T02:08:35.038 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:29.177803+0000', 'last_change': '2023-11-06T02:08:29.177803+0000', 'last_active': '2023-11-06T02:08:29.177803+0000', 'last_peered': '2023-11-06T02:08:29.177803+0000', 'last_clean': '2023-11-06T02:08:29.177803+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:29.177803+0000', 'last_undegraded': '2023-11-06T02:08:29.177803+0000', 'last_fullsized': '2023-11-06T02:08:29.177803+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.038 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.16 is not active+clean 2023-11-06T02:08:35.039 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:30.181639+0000', 'last_change': '2023-11-06T02:08:30.181639+0000', 'last_active': '2023-11-06T02:08:30.181639+0000', 'last_peered': '2023-11-06T02:08:30.181639+0000', 'last_clean': '2023-11-06T02:08:30.181639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:30.181639+0000', 'last_undegraded': '2023-11-06T02:08:30.181639+0000', 'last_fullsized': '2023-11-06T02:08:30.181639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.039 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.11 is not active+clean 2023-11-06T02:08:35.039 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:29.177803+0000', 'last_change': '2023-11-06T02:08:29.177803+0000', 'last_active': '2023-11-06T02:08:29.177803+0000', 'last_peered': '2023-11-06T02:08:29.177803+0000', 'last_clean': '2023-11-06T02:08:29.177803+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:29.177803+0000', 'last_undegraded': '2023-11-06T02:08:29.177803+0000', 'last_fullsized': '2023-11-06T02:08:29.177803+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.040 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.17 is not active+clean 2023-11-06T02:08:35.040 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:30.181639+0000', 'last_change': '2023-11-06T02:08:30.181639+0000', 'last_active': '2023-11-06T02:08:30.181639+0000', 'last_peered': '2023-11-06T02:08:30.181639+0000', 'last_clean': '2023-11-06T02:08:30.181639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:30.181639+0000', 'last_undegraded': '2023-11-06T02:08:30.181639+0000', 'last_fullsized': '2023-11-06T02:08:30.181639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.040 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.10 is not active+clean 2023-11-06T02:08:35.040 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:29.177803+0000', 'last_change': '2023-11-06T02:08:29.177803+0000', 'last_active': '2023-11-06T02:08:29.177803+0000', 'last_peered': '2023-11-06T02:08:29.177803+0000', 'last_clean': '2023-11-06T02:08:29.177803+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:29.177803+0000', 'last_undegraded': '2023-11-06T02:08:29.177803+0000', 'last_fullsized': '2023-11-06T02:08:29.177803+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.041 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.f is not active+clean 2023-11-06T02:08:35.041 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:29.177803+0000', 'last_change': '2023-11-06T02:08:29.177803+0000', 'last_active': '2023-11-06T02:08:29.177803+0000', 'last_peered': '2023-11-06T02:08:29.177803+0000', 'last_clean': '2023-11-06T02:08:29.177803+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:29.177803+0000', 'last_undegraded': '2023-11-06T02:08:29.177803+0000', 'last_fullsized': '2023-11-06T02:08:29.177803+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.041 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.8 is not active+clean 2023-11-06T02:08:35.042 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:30.181639+0000', 'last_change': '2023-11-06T02:08:30.181639+0000', 'last_active': '2023-11-06T02:08:30.181639+0000', 'last_peered': '2023-11-06T02:08:30.181639+0000', 'last_clean': '2023-11-06T02:08:30.181639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:30.181639+0000', 'last_undegraded': '2023-11-06T02:08:30.181639+0000', 'last_fullsized': '2023-11-06T02:08:30.181639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.042 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.e is not active+clean 2023-11-06T02:08:35.042 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:29.177803+0000', 'last_change': '2023-11-06T02:08:29.177803+0000', 'last_active': '2023-11-06T02:08:29.177803+0000', 'last_peered': '2023-11-06T02:08:29.177803+0000', 'last_clean': '2023-11-06T02:08:29.177803+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:29.177803+0000', 'last_undegraded': '2023-11-06T02:08:29.177803+0000', 'last_fullsized': '2023-11-06T02:08:29.177803+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.043 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.9 is not active+clean 2023-11-06T02:08:35.043 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:30.181639+0000', 'last_change': '2023-11-06T02:08:30.181639+0000', 'last_active': '2023-11-06T02:08:30.181639+0000', 'last_peered': '2023-11-06T02:08:30.181639+0000', 'last_clean': '2023-11-06T02:08:30.181639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:30.181639+0000', 'last_undegraded': '2023-11-06T02:08:30.181639+0000', 'last_fullsized': '2023-11-06T02:08:30.181639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.043 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.d is not active+clean 2023-11-06T02:08:35.044 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:29.177803+0000', 'last_change': '2023-11-06T02:08:29.177803+0000', 'last_active': '2023-11-06T02:08:29.177803+0000', 'last_peered': '2023-11-06T02:08:29.177803+0000', 'last_clean': '2023-11-06T02:08:29.177803+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:29.177803+0000', 'last_undegraded': '2023-11-06T02:08:29.177803+0000', 'last_fullsized': '2023-11-06T02:08:29.177803+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.044 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.a is not active+clean 2023-11-06T02:08:35.044 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:30.181639+0000', 'last_change': '2023-11-06T02:08:30.181639+0000', 'last_active': '2023-11-06T02:08:30.181639+0000', 'last_peered': '2023-11-06T02:08:30.181639+0000', 'last_clean': '2023-11-06T02:08:30.181639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:30.181639+0000', 'last_undegraded': '2023-11-06T02:08:30.181639+0000', 'last_fullsized': '2023-11-06T02:08:30.181639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.045 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.c is not active+clean 2023-11-06T02:08:35.045 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:29.177803+0000', 'last_change': '2023-11-06T02:08:29.177803+0000', 'last_active': '2023-11-06T02:08:29.177803+0000', 'last_peered': '2023-11-06T02:08:29.177803+0000', 'last_clean': '2023-11-06T02:08:29.177803+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:29.177803+0000', 'last_undegraded': '2023-11-06T02:08:29.177803+0000', 'last_fullsized': '2023-11-06T02:08:29.177803+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.045 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.b is not active+clean 2023-11-06T02:08:35.046 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:30.181639+0000', 'last_change': '2023-11-06T02:08:30.181639+0000', 'last_active': '2023-11-06T02:08:30.181639+0000', 'last_peered': '2023-11-06T02:08:30.181639+0000', 'last_clean': '2023-11-06T02:08:30.181639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:30.181639+0000', 'last_undegraded': '2023-11-06T02:08:30.181639+0000', 'last_fullsized': '2023-11-06T02:08:30.181639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.046 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.b is not active+clean 2023-11-06T02:08:35.046 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:29.177803+0000', 'last_change': '2023-11-06T02:08:29.177803+0000', 'last_active': '2023-11-06T02:08:29.177803+0000', 'last_peered': '2023-11-06T02:08:29.177803+0000', 'last_clean': '2023-11-06T02:08:29.177803+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:29.177803+0000', 'last_undegraded': '2023-11-06T02:08:29.177803+0000', 'last_fullsized': '2023-11-06T02:08:29.177803+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.047 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.c is not active+clean 2023-11-06T02:08:35.047 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:30.181639+0000', 'last_change': '2023-11-06T02:08:30.181639+0000', 'last_active': '2023-11-06T02:08:30.181639+0000', 'last_peered': '2023-11-06T02:08:30.181639+0000', 'last_clean': '2023-11-06T02:08:30.181639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:30.181639+0000', 'last_undegraded': '2023-11-06T02:08:30.181639+0000', 'last_fullsized': '2023-11-06T02:08:30.181639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.047 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.a is not active+clean 2023-11-06T02:08:35.048 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:29.177803+0000', 'last_change': '2023-11-06T02:08:29.177803+0000', 'last_active': '2023-11-06T02:08:29.177803+0000', 'last_peered': '2023-11-06T02:08:29.177803+0000', 'last_clean': '2023-11-06T02:08:29.177803+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:29.177803+0000', 'last_undegraded': '2023-11-06T02:08:29.177803+0000', 'last_fullsized': '2023-11-06T02:08:29.177803+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.048 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.d is not active+clean 2023-11-06T02:08:35.048 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:30.181639+0000', 'last_change': '2023-11-06T02:08:30.181639+0000', 'last_active': '2023-11-06T02:08:30.181639+0000', 'last_peered': '2023-11-06T02:08:30.181639+0000', 'last_clean': '2023-11-06T02:08:30.181639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:30.181639+0000', 'last_undegraded': '2023-11-06T02:08:30.181639+0000', 'last_fullsized': '2023-11-06T02:08:30.181639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.049 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.9 is not active+clean 2023-11-06T02:08:35.049 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:29.177803+0000', 'last_change': '2023-11-06T02:08:29.177803+0000', 'last_active': '2023-11-06T02:08:29.177803+0000', 'last_peered': '2023-11-06T02:08:29.177803+0000', 'last_clean': '2023-11-06T02:08:29.177803+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:29.177803+0000', 'last_undegraded': '2023-11-06T02:08:29.177803+0000', 'last_fullsized': '2023-11-06T02:08:29.177803+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.049 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.e is not active+clean 2023-11-06T02:08:35.050 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:30.181639+0000', 'last_change': '2023-11-06T02:08:30.181639+0000', 'last_active': '2023-11-06T02:08:30.181639+0000', 'last_peered': '2023-11-06T02:08:30.181639+0000', 'last_clean': '2023-11-06T02:08:30.181639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:30.181639+0000', 'last_undegraded': '2023-11-06T02:08:30.181639+0000', 'last_fullsized': '2023-11-06T02:08:30.181639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.050 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.8 is not active+clean 2023-11-06T02:08:35.050 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:29.177803+0000', 'last_change': '2023-11-06T02:08:29.177803+0000', 'last_active': '2023-11-06T02:08:29.177803+0000', 'last_peered': '2023-11-06T02:08:29.177803+0000', 'last_clean': '2023-11-06T02:08:29.177803+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:29.177803+0000', 'last_undegraded': '2023-11-06T02:08:29.177803+0000', 'last_fullsized': '2023-11-06T02:08:29.177803+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.051 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.f is not active+clean 2023-11-06T02:08:35.051 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:30.181639+0000', 'last_change': '2023-11-06T02:08:30.181639+0000', 'last_active': '2023-11-06T02:08:30.181639+0000', 'last_peered': '2023-11-06T02:08:30.181639+0000', 'last_clean': '2023-11-06T02:08:30.181639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:30.181639+0000', 'last_undegraded': '2023-11-06T02:08:30.181639+0000', 'last_fullsized': '2023-11-06T02:08:30.181639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.051 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2 is not active+clean 2023-11-06T02:08:35.052 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:29.177803+0000', 'last_change': '2023-11-06T02:08:29.177803+0000', 'last_active': '2023-11-06T02:08:29.177803+0000', 'last_peered': '2023-11-06T02:08:29.177803+0000', 'last_clean': '2023-11-06T02:08:29.177803+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:29.177803+0000', 'last_undegraded': '2023-11-06T02:08:29.177803+0000', 'last_fullsized': '2023-11-06T02:08:29.177803+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.052 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.5 is not active+clean 2023-11-06T02:08:35.052 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:30.181639+0000', 'last_change': '2023-11-06T02:08:30.181639+0000', 'last_active': '2023-11-06T02:08:30.181639+0000', 'last_peered': '2023-11-06T02:08:30.181639+0000', 'last_clean': '2023-11-06T02:08:30.181639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:30.181639+0000', 'last_undegraded': '2023-11-06T02:08:30.181639+0000', 'last_fullsized': '2023-11-06T02:08:30.181639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.053 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1 is not active+clean 2023-11-06T02:08:35.053 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:29.177803+0000', 'last_change': '2023-11-06T02:08:29.177803+0000', 'last_active': '2023-11-06T02:08:29.177803+0000', 'last_peered': '2023-11-06T02:08:29.177803+0000', 'last_clean': '2023-11-06T02:08:29.177803+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:29.177803+0000', 'last_undegraded': '2023-11-06T02:08:29.177803+0000', 'last_fullsized': '2023-11-06T02:08:29.177803+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.053 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.6 is not active+clean 2023-11-06T02:08:35.053 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:30.181639+0000', 'last_change': '2023-11-06T02:08:30.181639+0000', 'last_active': '2023-11-06T02:08:30.181639+0000', 'last_peered': '2023-11-06T02:08:30.181639+0000', 'last_clean': '2023-11-06T02:08:30.181639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:30.181639+0000', 'last_undegraded': '2023-11-06T02:08:30.181639+0000', 'last_fullsized': '2023-11-06T02:08:30.181639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.054 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.0 is not active+clean 2023-11-06T02:08:35.054 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:29.177803+0000', 'last_change': '2023-11-06T02:08:29.177803+0000', 'last_active': '2023-11-06T02:08:29.177803+0000', 'last_peered': '2023-11-06T02:08:29.177803+0000', 'last_clean': '2023-11-06T02:08:29.177803+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:29.177803+0000', 'last_undegraded': '2023-11-06T02:08:29.177803+0000', 'last_fullsized': '2023-11-06T02:08:29.177803+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.055 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.7 is not active+clean 2023-11-06T02:08:35.055 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:30.181639+0000', 'last_change': '2023-11-06T02:08:30.181639+0000', 'last_active': '2023-11-06T02:08:30.181639+0000', 'last_peered': '2023-11-06T02:08:30.181639+0000', 'last_clean': '2023-11-06T02:08:30.181639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:30.181639+0000', 'last_undegraded': '2023-11-06T02:08:30.181639+0000', 'last_fullsized': '2023-11-06T02:08:30.181639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.055 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3 is not active+clean 2023-11-06T02:08:35.055 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:29.177803+0000', 'last_change': '2023-11-06T02:08:29.177803+0000', 'last_active': '2023-11-06T02:08:29.177803+0000', 'last_peered': '2023-11-06T02:08:29.177803+0000', 'last_clean': '2023-11-06T02:08:29.177803+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:29.177803+0000', 'last_undegraded': '2023-11-06T02:08:29.177803+0000', 'last_fullsized': '2023-11-06T02:08:29.177803+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.056 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.4 is not active+clean 2023-11-06T02:08:35.056 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:30.181639+0000', 'last_change': '2023-11-06T02:08:30.181639+0000', 'last_active': '2023-11-06T02:08:30.181639+0000', 'last_peered': '2023-11-06T02:08:30.181639+0000', 'last_clean': '2023-11-06T02:08:30.181639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:30.181639+0000', 'last_undegraded': '2023-11-06T02:08:30.181639+0000', 'last_fullsized': '2023-11-06T02:08:30.181639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.056 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.5 is not active+clean 2023-11-06T02:08:35.057 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:29.177803+0000', 'last_change': '2023-11-06T02:08:29.177803+0000', 'last_active': '2023-11-06T02:08:29.177803+0000', 'last_peered': '2023-11-06T02:08:29.177803+0000', 'last_clean': '2023-11-06T02:08:29.177803+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:29.177803+0000', 'last_undegraded': '2023-11-06T02:08:29.177803+0000', 'last_fullsized': '2023-11-06T02:08:29.177803+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.057 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.2 is not active+clean 2023-11-06T02:08:35.057 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:30.181639+0000', 'last_change': '2023-11-06T02:08:30.181639+0000', 'last_active': '2023-11-06T02:08:30.181639+0000', 'last_peered': '2023-11-06T02:08:30.181639+0000', 'last_clean': '2023-11-06T02:08:30.181639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:30.181639+0000', 'last_undegraded': '2023-11-06T02:08:30.181639+0000', 'last_fullsized': '2023-11-06T02:08:30.181639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.058 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.4 is not active+clean 2023-11-06T02:08:35.058 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:29.177803+0000', 'last_change': '2023-11-06T02:08:29.177803+0000', 'last_active': '2023-11-06T02:08:29.177803+0000', 'last_peered': '2023-11-06T02:08:29.177803+0000', 'last_clean': '2023-11-06T02:08:29.177803+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:29.177803+0000', 'last_undegraded': '2023-11-06T02:08:29.177803+0000', 'last_fullsized': '2023-11-06T02:08:29.177803+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.059 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.3 is not active+clean 2023-11-06T02:08:35.059 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:30.181639+0000', 'last_change': '2023-11-06T02:08:30.181639+0000', 'last_active': '2023-11-06T02:08:30.181639+0000', 'last_peered': '2023-11-06T02:08:30.181639+0000', 'last_clean': '2023-11-06T02:08:30.181639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:30.181639+0000', 'last_undegraded': '2023-11-06T02:08:30.181639+0000', 'last_fullsized': '2023-11-06T02:08:30.181639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.059 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.7 is not active+clean 2023-11-06T02:08:35.060 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:29.177803+0000', 'last_change': '2023-11-06T02:08:29.177803+0000', 'last_active': '2023-11-06T02:08:29.177803+0000', 'last_peered': '2023-11-06T02:08:29.177803+0000', 'last_clean': '2023-11-06T02:08:29.177803+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:29.177803+0000', 'last_undegraded': '2023-11-06T02:08:29.177803+0000', 'last_fullsized': '2023-11-06T02:08:29.177803+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.060 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.0 is not active+clean 2023-11-06T02:08:35.061 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:30.181639+0000', 'last_change': '2023-11-06T02:08:30.181639+0000', 'last_active': '2023-11-06T02:08:30.181639+0000', 'last_peered': '2023-11-06T02:08:30.181639+0000', 'last_clean': '2023-11-06T02:08:30.181639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:30.181639+0000', 'last_undegraded': '2023-11-06T02:08:30.181639+0000', 'last_fullsized': '2023-11-06T02:08:30.181639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.061 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.6 is not active+clean 2023-11-06T02:08:35.061 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:29.177803+0000', 'last_change': '2023-11-06T02:08:29.177803+0000', 'last_active': '2023-11-06T02:08:29.177803+0000', 'last_peered': '2023-11-06T02:08:29.177803+0000', 'last_clean': '2023-11-06T02:08:29.177803+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:29.177803+0000', 'last_undegraded': '2023-11-06T02:08:29.177803+0000', 'last_fullsized': '2023-11-06T02:08:29.177803+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.062 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1 is not active+clean 2023-11-06T02:08:35.062 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:30.181639+0000', 'last_change': '2023-11-06T02:08:30.181639+0000', 'last_active': '2023-11-06T02:08:30.181639+0000', 'last_peered': '2023-11-06T02:08:30.181639+0000', 'last_clean': '2023-11-06T02:08:30.181639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:30.181639+0000', 'last_undegraded': '2023-11-06T02:08:30.181639+0000', 'last_fullsized': '2023-11-06T02:08:30.181639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.062 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.13 is not active+clean 2023-11-06T02:08:35.063 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:30.181639+0000', 'last_change': '2023-11-06T02:08:30.181639+0000', 'last_active': '2023-11-06T02:08:30.181639+0000', 'last_peered': '2023-11-06T02:08:30.181639+0000', 'last_clean': '2023-11-06T02:08:30.181639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:30.181639+0000', 'last_undegraded': '2023-11-06T02:08:30.181639+0000', 'last_fullsized': '2023-11-06T02:08:30.181639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.063 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.14 is not active+clean 2023-11-06T02:08:35.063 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:29.177803+0000', 'last_change': '2023-11-06T02:08:29.177803+0000', 'last_active': '2023-11-06T02:08:29.177803+0000', 'last_peered': '2023-11-06T02:08:29.177803+0000', 'last_clean': '2023-11-06T02:08:29.177803+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:29.177803+0000', 'last_undegraded': '2023-11-06T02:08:29.177803+0000', 'last_fullsized': '2023-11-06T02:08:29.177803+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.064 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.12 is not active+clean 2023-11-06T02:08:35.064 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:30.181639+0000', 'last_change': '2023-11-06T02:08:30.181639+0000', 'last_active': '2023-11-06T02:08:30.181639+0000', 'last_peered': '2023-11-06T02:08:30.181639+0000', 'last_clean': '2023-11-06T02:08:30.181639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:30.181639+0000', 'last_undegraded': '2023-11-06T02:08:30.181639+0000', 'last_fullsized': '2023-11-06T02:08:30.181639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.064 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.15 is not active+clean 2023-11-06T02:08:35.065 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:29.177803+0000', 'last_change': '2023-11-06T02:08:29.177803+0000', 'last_active': '2023-11-06T02:08:29.177803+0000', 'last_peered': '2023-11-06T02:08:29.177803+0000', 'last_clean': '2023-11-06T02:08:29.177803+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:29.177803+0000', 'last_undegraded': '2023-11-06T02:08:29.177803+0000', 'last_fullsized': '2023-11-06T02:08:29.177803+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.065 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.16 is not active+clean 2023-11-06T02:08:35.065 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:29.177803+0000', 'last_change': '2023-11-06T02:08:29.177803+0000', 'last_active': '2023-11-06T02:08:29.177803+0000', 'last_peered': '2023-11-06T02:08:29.177803+0000', 'last_clean': '2023-11-06T02:08:29.177803+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:29.177803+0000', 'last_undegraded': '2023-11-06T02:08:29.177803+0000', 'last_fullsized': '2023-11-06T02:08:29.177803+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.066 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.11 is not active+clean 2023-11-06T02:08:35.066 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:30.181639+0000', 'last_change': '2023-11-06T02:08:30.181639+0000', 'last_active': '2023-11-06T02:08:30.181639+0000', 'last_peered': '2023-11-06T02:08:30.181639+0000', 'last_clean': '2023-11-06T02:08:30.181639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:30.181639+0000', 'last_undegraded': '2023-11-06T02:08:30.181639+0000', 'last_fullsized': '2023-11-06T02:08:30.181639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.066 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.17 is not active+clean 2023-11-06T02:08:35.067 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:29.177803+0000', 'last_change': '2023-11-06T02:08:29.177803+0000', 'last_active': '2023-11-06T02:08:29.177803+0000', 'last_peered': '2023-11-06T02:08:29.177803+0000', 'last_clean': '2023-11-06T02:08:29.177803+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:29.177803+0000', 'last_undegraded': '2023-11-06T02:08:29.177803+0000', 'last_fullsized': '2023-11-06T02:08:29.177803+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.067 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.10 is not active+clean 2023-11-06T02:08:35.067 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:30.181639+0000', 'last_change': '2023-11-06T02:08:30.181639+0000', 'last_active': '2023-11-06T02:08:30.181639+0000', 'last_peered': '2023-11-06T02:08:30.181639+0000', 'last_clean': '2023-11-06T02:08:30.181639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:30.181639+0000', 'last_undegraded': '2023-11-06T02:08:30.181639+0000', 'last_fullsized': '2023-11-06T02:08:30.181639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.068 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1f is not active+clean 2023-11-06T02:08:35.068 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:30.181639+0000', 'last_change': '2023-11-06T02:08:30.181639+0000', 'last_active': '2023-11-06T02:08:30.181639+0000', 'last_peered': '2023-11-06T02:08:30.181639+0000', 'last_clean': '2023-11-06T02:08:30.181639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:30.181639+0000', 'last_undegraded': '2023-11-06T02:08:30.181639+0000', 'last_fullsized': '2023-11-06T02:08:30.181639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.068 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.18 is not active+clean 2023-11-06T02:08:35.069 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:29.177803+0000', 'last_change': '2023-11-06T02:08:29.177803+0000', 'last_active': '2023-11-06T02:08:29.177803+0000', 'last_peered': '2023-11-06T02:08:29.177803+0000', 'last_clean': '2023-11-06T02:08:29.177803+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:29.177803+0000', 'last_undegraded': '2023-11-06T02:08:29.177803+0000', 'last_fullsized': '2023-11-06T02:08:29.177803+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.069 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1e is not active+clean 2023-11-06T02:08:35.069 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:30.181639+0000', 'last_change': '2023-11-06T02:08:30.181639+0000', 'last_active': '2023-11-06T02:08:30.181639+0000', 'last_peered': '2023-11-06T02:08:30.181639+0000', 'last_clean': '2023-11-06T02:08:30.181639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:30.181639+0000', 'last_undegraded': '2023-11-06T02:08:30.181639+0000', 'last_fullsized': '2023-11-06T02:08:30.181639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.070 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.19 is not active+clean 2023-11-06T02:08:35.070 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:29.177803+0000', 'last_change': '2023-11-06T02:08:29.177803+0000', 'last_active': '2023-11-06T02:08:29.177803+0000', 'last_peered': '2023-11-06T02:08:29.177803+0000', 'last_clean': '2023-11-06T02:08:29.177803+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:29.177803+0000', 'last_undegraded': '2023-11-06T02:08:29.177803+0000', 'last_fullsized': '2023-11-06T02:08:29.177803+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.070 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1d is not active+clean 2023-11-06T02:08:35.071 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:30.181639+0000', 'last_change': '2023-11-06T02:08:30.181639+0000', 'last_active': '2023-11-06T02:08:30.181639+0000', 'last_peered': '2023-11-06T02:08:30.181639+0000', 'last_clean': '2023-11-06T02:08:30.181639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:30.181639+0000', 'last_undegraded': '2023-11-06T02:08:30.181639+0000', 'last_fullsized': '2023-11-06T02:08:30.181639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.071 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1a is not active+clean 2023-11-06T02:08:35.071 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:29.177803+0000', 'last_change': '2023-11-06T02:08:29.177803+0000', 'last_active': '2023-11-06T02:08:29.177803+0000', 'last_peered': '2023-11-06T02:08:29.177803+0000', 'last_clean': '2023-11-06T02:08:29.177803+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:29.177803+0000', 'last_undegraded': '2023-11-06T02:08:29.177803+0000', 'last_fullsized': '2023-11-06T02:08:29.177803+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.072 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1c is not active+clean 2023-11-06T02:08:35.072 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:30.181639+0000', 'last_change': '2023-11-06T02:08:30.181639+0000', 'last_active': '2023-11-06T02:08:30.181639+0000', 'last_peered': '2023-11-06T02:08:30.181639+0000', 'last_clean': '2023-11-06T02:08:30.181639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:30.181639+0000', 'last_undegraded': '2023-11-06T02:08:30.181639+0000', 'last_fullsized': '2023-11-06T02:08:30.181639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.072 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1b is not active+clean 2023-11-06T02:08:35.073 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:29.177803+0000', 'last_change': '2023-11-06T02:08:29.177803+0000', 'last_active': '2023-11-06T02:08:29.177803+0000', 'last_peered': '2023-11-06T02:08:29.177803+0000', 'last_clean': '2023-11-06T02:08:29.177803+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:29.177803+0000', 'last_undegraded': '2023-11-06T02:08:29.177803+0000', 'last_fullsized': '2023-11-06T02:08:29.177803+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.073 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1b is not active+clean 2023-11-06T02:08:35.073 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:30.181639+0000', 'last_change': '2023-11-06T02:08:30.181639+0000', 'last_active': '2023-11-06T02:08:30.181639+0000', 'last_peered': '2023-11-06T02:08:30.181639+0000', 'last_clean': '2023-11-06T02:08:30.181639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:30.181639+0000', 'last_undegraded': '2023-11-06T02:08:30.181639+0000', 'last_fullsized': '2023-11-06T02:08:30.181639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.074 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1c is not active+clean 2023-11-06T02:08:35.074 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:29.177803+0000', 'last_change': '2023-11-06T02:08:29.177803+0000', 'last_active': '2023-11-06T02:08:29.177803+0000', 'last_peered': '2023-11-06T02:08:29.177803+0000', 'last_clean': '2023-11-06T02:08:29.177803+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:29.177803+0000', 'last_undegraded': '2023-11-06T02:08:29.177803+0000', 'last_fullsized': '2023-11-06T02:08:29.177803+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.074 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1a is not active+clean 2023-11-06T02:08:35.074 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:30.181639+0000', 'last_change': '2023-11-06T02:08:30.181639+0000', 'last_active': '2023-11-06T02:08:30.181639+0000', 'last_peered': '2023-11-06T02:08:30.181639+0000', 'last_clean': '2023-11-06T02:08:30.181639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:30.181639+0000', 'last_undegraded': '2023-11-06T02:08:30.181639+0000', 'last_fullsized': '2023-11-06T02:08:30.181639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.075 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1d is not active+clean 2023-11-06T02:08:35.075 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:29.177803+0000', 'last_change': '2023-11-06T02:08:29.177803+0000', 'last_active': '2023-11-06T02:08:29.177803+0000', 'last_peered': '2023-11-06T02:08:29.177803+0000', 'last_clean': '2023-11-06T02:08:29.177803+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:29.177803+0000', 'last_undegraded': '2023-11-06T02:08:29.177803+0000', 'last_fullsized': '2023-11-06T02:08:29.177803+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.075 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.19 is not active+clean 2023-11-06T02:08:35.076 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:30.181639+0000', 'last_change': '2023-11-06T02:08:30.181639+0000', 'last_active': '2023-11-06T02:08:30.181639+0000', 'last_peered': '2023-11-06T02:08:30.181639+0000', 'last_clean': '2023-11-06T02:08:30.181639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:30.181639+0000', 'last_undegraded': '2023-11-06T02:08:30.181639+0000', 'last_fullsized': '2023-11-06T02:08:30.181639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.076 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1e is not active+clean 2023-11-06T02:08:35.076 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:29.177803+0000', 'last_change': '2023-11-06T02:08:29.177803+0000', 'last_active': '2023-11-06T02:08:29.177803+0000', 'last_peered': '2023-11-06T02:08:29.177803+0000', 'last_clean': '2023-11-06T02:08:29.177803+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:29.177803+0000', 'last_undegraded': '2023-11-06T02:08:29.177803+0000', 'last_fullsized': '2023-11-06T02:08:29.177803+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.077 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.18 is not active+clean 2023-11-06T02:08:35.077 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:30.181639+0000', 'last_change': '2023-11-06T02:08:30.181639+0000', 'last_active': '2023-11-06T02:08:30.181639+0000', 'last_peered': '2023-11-06T02:08:30.181639+0000', 'last_clean': '2023-11-06T02:08:30.181639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:30.181639+0000', 'last_undegraded': '2023-11-06T02:08:30.181639+0000', 'last_fullsized': '2023-11-06T02:08:30.181639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:30.181639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.077 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1f is not active+clean 2023-11-06T02:08:35.078 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:08:29.177803+0000', 'last_change': '2023-11-06T02:08:29.177803+0000', 'last_active': '2023-11-06T02:08:29.177803+0000', 'last_peered': '2023-11-06T02:08:29.177803+0000', 'last_clean': '2023-11-06T02:08:29.177803+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:08:29.177803+0000', 'last_undegraded': '2023-11-06T02:08:29.177803+0000', 'last_fullsized': '2023-11-06T02:08:29.177803+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'last_clean_scrub_stamp': '2023-11-06T02:08:29.177803+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:08:35.078 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-11-06T02:08:35.199 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.199+0000 7f262af67640 1 Processor -- start 2023-11-06T02:08:35.199 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.199+0000 7f262af67640 1 -- start start 2023-11-06T02:08:35.201 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.203+0000 7f262af67640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f262415ab50 0x7f26240c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:35.201 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.203+0000 7f262af67640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f26240b0790 0x7f26240c2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:35.201 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.203+0000 7f262af67640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26240cab80 con 0x7f26240cb010 2023-11-06T02:08:35.201 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.203+0000 7f262af67640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f262415af30 con 0x7f26240b0790 2023-11-06T02:08:35.202 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.203+0000 7f262af67640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26240cb5b0 con 0x7f262415ab50 2023-11-06T02:08:35.202 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.203+0000 7f2629504640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f26240b0790 0x7f26240c2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:35.202 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.203+0000 7f2629504640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f26240b0790 0x7f26240c2710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47434/0 (socket says 172.21.15.49:47434) 2023-11-06T02:08:35.202 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.203+0000 7f2629504640 1 -- 172.21.15.49:0/202868121 learned_addr learned my addr 172.21.15.49:0/202868121 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:35.202 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.203+0000 7f2628d03640 1 --2- 172.21.15.49:0/202868121 >> v2:172.21.15.150:3300/0 conn(0x7f262415ab50 0x7f26240c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:35.203 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.203+0000 7f2629504640 1 -- 172.21.15.49:0/202868121 >> v2:172.21.15.150:3300/0 conn(0x7f262415ab50 msgr2=0x7f26240c9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:35.203 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.203+0000 7f2629504640 1 --2- 172.21.15.49:0/202868121 >> v2:172.21.15.150:3300/0 conn(0x7f262415ab50 0x7f26240c9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:35.203 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.203+0000 7f2629504640 1 -- 172.21.15.49:0/202868121 >> v1:172.21.15.150:6789/0 conn(0x7f26240cb010 legacy=0x7f26240b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:35.203 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.203+0000 7f2629504640 1 -- 172.21.15.49:0/202868121 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26240cb890 con 0x7f26240b0790 2023-11-06T02:08:35.204 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.203+0000 7f2629504640 1 --2- 172.21.15.49:0/202868121 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26240b0790 0x7f26240c2710 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f26240725a0 tx=0x7f261c01c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dace1c5bb81ab786 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:35.205 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.203+0000 7f26237fe640 1 -- 172.21.15.49:0/202868121 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f261c0a5860 con 0x7f26240b0790 2023-11-06T02:08:35.205 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.203+0000 7f26237fe640 1 -- 172.21.15.49:0/202868121 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f261c0a59c0 con 0x7f26240b0790 2023-11-06T02:08:35.205 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.203+0000 7f26237fe640 1 -- 172.21.15.49:0/202868121 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f261c0a5d50 con 0x7f26240b0790 2023-11-06T02:08:35.205 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.203+0000 7f262af67640 1 -- 172.21.15.49:0/202868121 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26240b0790 msgr2=0x7f26240c2710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:35.205 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.203+0000 7f262af67640 1 --2- 172.21.15.49:0/202868121 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26240b0790 0x7f26240c2710 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f26240725a0 tx=0x7f261c01c9a0 comp rx=0 tx=0).stop 2023-11-06T02:08:35.205 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.203+0000 7f262af67640 1 -- 172.21.15.49:0/202868121 shutdown_connections 2023-11-06T02:08:35.206 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.203+0000 7f262af67640 1 --2- 172.21.15.49:0/202868121 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26240b0790 0x7f26240c2710 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:35.206 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.203+0000 7f262af67640 1 --2- 172.21.15.49:0/202868121 >> v2:172.21.15.150:3300/0 conn(0x7f262415ab50 0x7f26240c9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:35.206 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.203+0000 7f262af67640 1 -- 172.21.15.49:0/202868121 >> 172.21.15.49:0/202868121 conn(0x7f262406c0a0 msgr2=0x7f262406c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:35.206 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.203+0000 7f262af67640 1 -- 172.21.15.49:0/202868121 shutdown_connections 2023-11-06T02:08:35.206 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.203+0000 7f262af67640 1 -- 172.21.15.49:0/202868121 wait complete. 2023-11-06T02:08:35.208 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.207+0000 7f262af67640 1 Processor -- start 2023-11-06T02:08:35.209 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.207+0000 7f262af67640 1 -- start start 2023-11-06T02:08:35.209 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.207+0000 7f262af67640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f26240b0790 0x7f26241f6880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:35.209 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.207+0000 7f262af67640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26240cb010 0x7f26240aa1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:35.209 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.207+0000 7f262af67640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f262415ab50 0x7f262413f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:35.210 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.207+0000 7f262af67640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26240c3cd0 con 0x7f26240cb010 2023-11-06T02:08:35.210 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.207+0000 7f262af67640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26241422f0 con 0x7f262415ab50 2023-11-06T02:08:35.210 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.207+0000 7f262af67640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26241425d0 con 0x7f26240b0790 2023-11-06T02:08:35.210 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.207+0000 7f2623fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26240cb010 0x7f26240aa1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:35.210 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.207+0000 7f2623fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26240cb010 0x7f26240aa1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47448/0 (socket says 172.21.15.49:47448) 2023-11-06T02:08:35.211 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.207+0000 7f2623fff640 1 -- 172.21.15.49:0/757699825 learned_addr learned my addr 172.21.15.49:0/757699825 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:35.211 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.207+0000 7f2629504640 1 --2- 172.21.15.49:0/757699825 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f262415ab50 0x7f262413f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:35.211 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.207+0000 7f2628d03640 1 --2- 172.21.15.49:0/757699825 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f26240b0790 0x7f26241f6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:35.211 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.207+0000 7f2623fff640 1 -- 172.21.15.49:0/757699825 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f26240b0790 msgr2=0x7f26241f6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:35.211 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.207+0000 7f2623fff640 1 --2- 172.21.15.49:0/757699825 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f26240b0790 0x7f26241f6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:35.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.207+0000 7f2623fff640 1 -- 172.21.15.49:0/757699825 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f262415ab50 msgr2=0x7f262413f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:35.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.207+0000 7f2623fff640 1 --2- 172.21.15.49:0/757699825 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f262415ab50 0x7f262413f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:35.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.207+0000 7f2623fff640 1 -- 172.21.15.49:0/757699825 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f261c009b30 con 0x7f26240cb010 2023-11-06T02:08:35.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.207+0000 7f2623fff640 1 --2- 172.21.15.49:0/757699825 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26240cb010 0x7f26240aa1c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f261400c500 tx=0x7f261400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:35.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.207+0000 7f26217fa640 1 -- 172.21.15.49:0/757699825 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2614005220 con 0x7f26240cb010 2023-11-06T02:08:35.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.207+0000 7f26217fa640 1 -- 172.21.15.49:0/757699825 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2614005380 con 0x7f26240cb010 2023-11-06T02:08:35.213 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.207+0000 7f26217fa640 1 -- 172.21.15.49:0/757699825 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2614007270 con 0x7f26240cb010 2023-11-06T02:08:35.213 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.207+0000 7f262af67640 1 -- 172.21.15.49:0/757699825 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26241428b0 con 0x7f26240cb010 2023-11-06T02:08:35.213 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.207+0000 7f262af67640 1 -- 172.21.15.49:0/757699825 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26241f6dc0 con 0x7f26240cb010 2023-11-06T02:08:35.213 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.207+0000 7f262af67640 1 -- 172.21.15.49:0/757699825 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26240b0de0 con 0x7f26240cb010 2023-11-06T02:08:35.214 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.211+0000 7f26217fa640 1 -- 172.21.15.49:0/757699825 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f26140073d0 con 0x7f26240cb010 2023-11-06T02:08:35.214 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.211+0000 7f26217fa640 1 --2- 172.21.15.49:0/757699825 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2610065a60 0x7f2610067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:35.215 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.211+0000 7f26217fa640 1 -- 172.21.15.49:0/757699825 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2614083970 con 0x7f26240cb010 2023-11-06T02:08:35.215 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.211+0000 7f2628d03640 1 --2- 172.21.15.49:0/757699825 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2610065a60 0x7f2610067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:35.215 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.211+0000 7f2628d03640 1 --2- 172.21.15.49:0/757699825 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2610065a60 0x7f2610067f20 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2618006fd0 tx=0x7f2618008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:35.215 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.215+0000 7f26217fa640 1 -- 172.21.15.49:0/757699825 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f2614051260 con 0x7f26240cb010 2023-11-06T02:08:35.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.427+0000 7f262af67640 1 -- 172.21.15.49:0/757699825 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f2624142a40 con 0x7f26240cb010 2023-11-06T02:08:35.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.427+0000 7f26217fa640 1 -- 172.21.15.49:0/757699825 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1012 (secure 0 0 0) 0x7f2614054880 con 0x7f26240cb010 2023-11-06T02:08:35.427 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:08:35.427 INFO:teuthology.orchestra.run.smithi049.stdout:{"fsid":"1a8b035c-6ba4-499a-b876-aea47fd9c0d4","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":29,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":23,"num_osds":8,"num_up_osds":8,"osd_up_since":1699236497,"num_in_osds":8,"osd_in_since":1699236488,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":105}],"num_pgs":105,"num_pools":4,"num_objects":24,"data_bytes":592730,"bytes_used":240713728,"bytes_avail":772853399552,"bytes_total":773094113280,"write_bytes_sec":2392,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":7,"id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"a","status":"up:active","gid":4293}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2023-11-06T02:08:06.445048+0000","services":{}},"progress_events":{}} 2023-11-06T02:08:35.429 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.427+0000 7f262af67640 1 -- 172.21.15.49:0/757699825 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2610065a60 msgr2=0x7f2610067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:35.429 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.431+0000 7f262af67640 1 --2- 172.21.15.49:0/757699825 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2610065a60 0x7f2610067f20 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2618006fd0 tx=0x7f2618008040 comp rx=0 tx=0).stop 2023-11-06T02:08:35.429 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.431+0000 7f262af67640 1 -- 172.21.15.49:0/757699825 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26240cb010 msgr2=0x7f26240aa1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:35.429 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.431+0000 7f262af67640 1 --2- 172.21.15.49:0/757699825 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26240cb010 0x7f26240aa1c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f261400c500 tx=0x7f261400c9c0 comp rx=0 tx=0).stop 2023-11-06T02:08:35.429 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.431+0000 7f262af67640 1 -- 172.21.15.49:0/757699825 shutdown_connections 2023-11-06T02:08:35.430 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.431+0000 7f262af67640 1 --2- 172.21.15.49:0/757699825 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2610065a60 0x7f2610067f20 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:35.430 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.431+0000 7f262af67640 1 --2- 172.21.15.49:0/757699825 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26240cb010 0x7f26240aa1c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:35.430 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.431+0000 7f262af67640 1 --2- 172.21.15.49:0/757699825 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f262415ab50 0x7f262413f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:35.430 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.431+0000 7f262af67640 1 --2- 172.21.15.49:0/757699825 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f26240b0790 0x7f26241f6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:35.430 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.431+0000 7f262af67640 1 -- 172.21.15.49:0/757699825 >> 172.21.15.49:0/757699825 conn(0x7f262406c0a0 msgr2=0x7f26240c76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:35.431 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.431+0000 7f262af67640 1 -- 172.21.15.49:0/757699825 shutdown_connections 2023-11-06T02:08:35.431 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.431+0000 7f262af67640 1 -- 172.21.15.49:0/757699825 wait complete. 2023-11-06T02:08:35.445 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-11-06T02:08:35.446 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T02:08:35.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.563+0000 7ff8962bd640 1 Processor -- start 2023-11-06T02:08:35.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.563+0000 7ff8962bd640 1 -- start start 2023-11-06T02:08:35.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.567+0000 7ff8962bd640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7ff890159980 0x7ff890071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:35.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.567+0000 7ff8962bd640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff8900b1450 0x7ff8900cdd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:35.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.567+0000 7ff8962bd640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8900cb010 0x7ff8900cb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:35.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.567+0000 7ff8962bd640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff890072410 con 0x7ff8900b1450 2023-11-06T02:08:35.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.567+0000 7ff8962bd640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8900b1890 con 0x7ff890159980 2023-11-06T02:08:35.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.567+0000 7ff8962bd640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8900cb9e0 con 0x7ff8900cb010 2023-11-06T02:08:35.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.567+0000 7ff8877fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff8900b1450 0x7ff8900cdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:35.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.567+0000 7ff8877fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff8900b1450 0x7ff8900cdd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47456/0 (socket says 172.21.15.49:47456) 2023-11-06T02:08:35.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.567+0000 7ff8877fe640 1 -- 172.21.15.49:0/326247421 learned_addr learned my addr 172.21.15.49:0/326247421 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:35.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.567+0000 7ff88ffff640 1 --2- 172.21.15.49:0/326247421 >> v2:172.21.15.150:3300/0 conn(0x7ff890159980 0x7ff890071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:35.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.567+0000 7ff8877fe640 1 -- 172.21.15.49:0/326247421 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8900cb010 msgr2=0x7ff8900cb4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:35.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.567+0000 7ff89485a640 1 --2- 172.21.15.49:0/326247421 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8900cb010 0x7ff8900cb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:35.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.567+0000 7ff8877fe640 1 --2- 172.21.15.49:0/326247421 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8900cb010 0x7ff8900cb4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:35.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.567+0000 7ff8877fe640 1 -- 172.21.15.49:0/326247421 >> v2:172.21.15.150:3300/0 conn(0x7ff890159980 msgr2=0x7ff890071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:35.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.567+0000 7ff8877fe640 1 --2- 172.21.15.49:0/326247421 >> v2:172.21.15.150:3300/0 conn(0x7ff890159980 0x7ff890071290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:35.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.567+0000 7ff8877fe640 1 -- 172.21.15.49:0/326247421 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff890071890 con 0x7ff8900b1450 2023-11-06T02:08:35.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.567+0000 7ff89485a640 1 --2- 172.21.15.49:0/326247421 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8900cb010 0x7ff8900cb4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:08:35.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.567+0000 7ff88ffff640 1 --2- 172.21.15.49:0/326247421 >> v2:172.21.15.150:3300/0 conn(0x7ff890159980 0x7ff890071290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:08:35.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.567+0000 7ff8877fe640 1 --2- 172.21.15.49:0/326247421 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8900b1450 0x7ff8900cdd30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff8900cc140 tx=0x7ff87801c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b9c82cc522d350f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:35.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.567+0000 7ff88f7fe640 1 -- 172.21.15.49:0/326247421 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8780aa620 con 0x7ff8900b1450 2023-11-06T02:08:35.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.567+0000 7ff88f7fe640 1 -- 172.21.15.49:0/326247421 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8780aa780 con 0x7ff8900b1450 2023-11-06T02:08:35.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.567+0000 7ff88f7fe640 1 -- 172.21.15.49:0/326247421 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8780aab10 con 0x7ff8900b1450 2023-11-06T02:08:35.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.571+0000 7ff8962bd640 1 -- 172.21.15.49:0/326247421 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8900b1450 msgr2=0x7ff8900cdd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:35.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.571+0000 7ff8962bd640 1 --2- 172.21.15.49:0/326247421 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8900b1450 0x7ff8900cdd30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff8900cc140 tx=0x7ff87801c500 comp rx=0 tx=0).stop 2023-11-06T02:08:35.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.571+0000 7ff8962bd640 1 -- 172.21.15.49:0/326247421 shutdown_connections 2023-11-06T02:08:35.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.571+0000 7ff8962bd640 1 --2- 172.21.15.49:0/326247421 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8900cb010 0x7ff8900cb4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:35.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.571+0000 7ff8962bd640 1 --2- 172.21.15.49:0/326247421 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8900b1450 0x7ff8900cdd30 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:35.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.571+0000 7ff8962bd640 1 --2- 172.21.15.49:0/326247421 >> v2:172.21.15.150:3300/0 conn(0x7ff890159980 0x7ff890071290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:35.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.571+0000 7ff8962bd640 1 -- 172.21.15.49:0/326247421 >> 172.21.15.49:0/326247421 conn(0x7ff89006c0a0 msgr2=0x7ff89006c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:35.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.571+0000 7ff8962bd640 1 -- 172.21.15.49:0/326247421 shutdown_connections 2023-11-06T02:08:35.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.571+0000 7ff8962bd640 1 -- 172.21.15.49:0/326247421 wait complete. 2023-11-06T02:08:35.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.571+0000 7ff8962bd640 1 Processor -- start 2023-11-06T02:08:35.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.571+0000 7ff8962bd640 1 -- start start 2023-11-06T02:08:35.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.571+0000 7ff8962bd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8900b1450 0x7ff89014a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:35.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.571+0000 7ff8962bd640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8900cb010 0x7ff8900c0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:35.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.571+0000 7ff8962bd640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff890159980 0x7ff8900b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:35.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.571+0000 7ff8962bd640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8900ace50 con 0x7ff8900b1450 2023-11-06T02:08:35.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.571+0000 7ff8962bd640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8900b3a20 con 0x7ff890159980 2023-11-06T02:08:35.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.571+0000 7ff8962bd640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8900b3d00 con 0x7ff8900cb010 2023-11-06T02:08:35.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.571+0000 7ff88ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8900b1450 0x7ff89014a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:35.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.571+0000 7ff88ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8900b1450 0x7ff89014a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47462/0 (socket says 172.21.15.49:47462) 2023-11-06T02:08:35.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.571+0000 7ff88ffff640 1 -- 172.21.15.49:0/2418864856 learned_addr learned my addr 172.21.15.49:0/2418864856 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:35.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.571+0000 7ff8877fe640 1 --2- 172.21.15.49:0/2418864856 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8900cb010 0x7ff8900c0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:35.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.571+0000 7ff89485a640 1 --2- 172.21.15.49:0/2418864856 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff890159980 0x7ff8900b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:35.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.571+0000 7ff88ffff640 1 -- 172.21.15.49:0/2418864856 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8900cb010 msgr2=0x7ff8900c0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:35.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.571+0000 7ff88ffff640 1 --2- 172.21.15.49:0/2418864856 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8900cb010 0x7ff8900c0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:35.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.571+0000 7ff88ffff640 1 -- 172.21.15.49:0/2418864856 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff890159980 msgr2=0x7ff8900b2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:35.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.571+0000 7ff88ffff640 1 --2- 172.21.15.49:0/2418864856 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff890159980 0x7ff8900b2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:35.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.571+0000 7ff88ffff640 1 -- 172.21.15.49:0/2418864856 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8780094d0 con 0x7ff8900b1450 2023-11-06T02:08:35.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.571+0000 7ff88ffff640 1 --2- 172.21.15.49:0/2418864856 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8900b1450 0x7ff89014a780 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff88000c7f0 tx=0x7ff88000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:35.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.571+0000 7ff88d7fa640 1 -- 172.21.15.49:0/2418864856 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff880005ba0 con 0x7ff8900b1450 2023-11-06T02:08:35.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.571+0000 7ff88d7fa640 1 -- 172.21.15.49:0/2418864856 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff880005d00 con 0x7ff8900b1450 2023-11-06T02:08:35.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.571+0000 7ff88d7fa640 1 -- 172.21.15.49:0/2418864856 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff880006250 con 0x7ff8900b1450 2023-11-06T02:08:35.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.571+0000 7ff8962bd640 1 -- 172.21.15.49:0/2418864856 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8900ce3f0 con 0x7ff8900b1450 2023-11-06T02:08:35.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.571+0000 7ff8962bd640 1 -- 172.21.15.49:0/2418864856 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff89014acc0 con 0x7ff8900b1450 2023-11-06T02:08:35.579 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.575+0000 7ff8962bd640 1 -- 172.21.15.49:0/2418864856 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8900ce230 con 0x7ff8900b1450 2023-11-06T02:08:35.580 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.575+0000 7ff88d7fa640 1 -- 172.21.15.49:0/2418864856 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7ff8800065c0 con 0x7ff8900b1450 2023-11-06T02:08:35.580 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.575+0000 7ff88d7fa640 1 --2- 172.21.15.49:0/2418864856 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff870065a90 0x7ff870067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:35.580 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.575+0000 7ff88d7fa640 1 -- 172.21.15.49:0/2418864856 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff880083400 con 0x7ff8900b1450 2023-11-06T02:08:35.580 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.575+0000 7ff8877fe640 1 --2- 172.21.15.49:0/2418864856 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff870065a90 0x7ff870067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:35.580 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.575+0000 7ff8877fe640 1 --2- 172.21.15.49:0/2418864856 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff870065a90 0x7ff870067f50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff8780ae000 tx=0x7ff8780a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:35.580 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.579+0000 7ff88d7fa640 1 -- 172.21.15.49:0/2418864856 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7ff880050c60 con 0x7ff8900b1450 2023-11-06T02:08:35.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.727+0000 7ff8962bd640 1 -- 172.21.15.49:0/2418864856 --> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff8900b3b90 con 0x7ff870065a90 2023-11-06T02:08:35.748 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.751+0000 7ff88d7fa640 1 -- 172.21.15.49:0/2418864856 <== mgr.4100 v2:172.21.15.150:6832/38104 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286677 (secure 0 0 0) 0x7ff8900b3b90 con 0x7ff870065a90 2023-11-06T02:08:35.751 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:08:35.753 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2023-11-06T02:08:35.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.751+0000 7ff8962bd640 1 -- 172.21.15.49:0/2418864856 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff870065a90 msgr2=0x7ff870067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:35.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.751+0000 7ff8962bd640 1 --2- 172.21.15.49:0/2418864856 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff870065a90 0x7ff870067f50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff8780ae000 tx=0x7ff8780a7000 comp rx=0 tx=0).stop 2023-11-06T02:08:35.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.751+0000 7ff8962bd640 1 -- 172.21.15.49:0/2418864856 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8900b1450 msgr2=0x7ff89014a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:35.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.751+0000 7ff8962bd640 1 --2- 172.21.15.49:0/2418864856 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8900b1450 0x7ff89014a780 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff88000c7f0 tx=0x7ff88000ccb0 comp rx=0 tx=0).stop 2023-11-06T02:08:35.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.755+0000 7ff8962bd640 1 -- 172.21.15.49:0/2418864856 shutdown_connections 2023-11-06T02:08:35.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.755+0000 7ff8962bd640 1 --2- 172.21.15.49:0/2418864856 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff870065a90 0x7ff870067f50 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:35.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.755+0000 7ff8962bd640 1 --2- 172.21.15.49:0/2418864856 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff890159980 0x7ff8900b2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:35.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.755+0000 7ff8962bd640 1 --2- 172.21.15.49:0/2418864856 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8900cb010 0x7ff8900c0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:35.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.755+0000 7ff8962bd640 1 --2- 172.21.15.49:0/2418864856 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8900b1450 0x7ff89014a780 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:35.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.755+0000 7ff8962bd640 1 -- 172.21.15.49:0/2418864856 >> 172.21.15.49:0/2418864856 conn(0x7ff89006c0a0 msgr2=0x7ff8901583f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:35.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.755+0000 7ff8962bd640 1 -- 172.21.15.49:0/2418864856 shutdown_connections 2023-11-06T02:08:35.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:35.755+0000 7ff8962bd640 1 -- 172.21.15.49:0/2418864856 wait complete. 2023-11-06T02:08:35.768 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-11-06T02:08:35.175702+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":175,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235072,"kb_used_data":3072,"kb_used_omap":0,"kb_used_meta":231936,"kb_avail":754739648,"statfs":{"total":773094113280,"available":772853399552,"internally_reserved":0,"allocated":3145728,"data_stored":1881274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":237502464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":61,"apply_latency_ms":61,"commit_latency_ns":61000000,"apply_latency_ns":61000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.992103"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.655111+0000","last_change":"2023-11-06T02:08:30.608781+0000","last_active":"2023-11-06T02:08:32.655111+0000","last_peered":"2023-11-06T02:08:32.655111+0000","last_clean":"2023-11-06T02:08:32.655111+0000","last_became_active":"2023-11-06T02:08:30.606710+0000","last_became_peered":"2023-11-06T02:08:30.606710+0000","last_unstale":"2023-11-06T02:08:32.655111+0000","last_undegraded":"2023-11-06T02:08:32.655111+0000","last_fullsized":"2023-11-06T02:08:32.655111+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:43:57.021755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.413091+0000","last_change":"2023-11-06T02:08:30.646508+0000","last_active":"2023-11-06T02:08:32.413091+0000","last_peered":"2023-11-06T02:08:32.413091+0000","last_clean":"2023-11-06T02:08:32.413091+0000","last_became_active":"2023-11-06T02:08:30.645615+0000","last_became_peered":"2023-11-06T02:08:30.645615+0000","last_unstale":"2023-11-06T02:08:32.413091+0000","last_undegraded":"2023-11-06T02:08:32.413091+0000","last_fullsized":"2023-11-06T02:08:32.413091+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:49:05.460965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.610298+0000","last_change":"2023-11-06T02:08:30.658350+0000","last_active":"2023-11-06T02:08:31.610298+0000","last_peered":"2023-11-06T02:08:31.610298+0000","last_clean":"2023-11-06T02:08:31.610298+0000","last_became_active":"2023-11-06T02:08:30.651992+0000","last_became_peered":"2023-11-06T02:08:30.651992+0000","last_unstale":"2023-11-06T02:08:31.610298+0000","last_undegraded":"2023-11-06T02:08:31.610298+0000","last_fullsized":"2023-11-06T02:08:31.610298+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:09:56.455105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.610511+0000","last_change":"2023-11-06T02:08:30.627209+0000","last_active":"2023-11-06T02:08:31.610511+0000","last_peered":"2023-11-06T02:08:31.610511+0000","last_clean":"2023-11-06T02:08:31.610511+0000","last_became_active":"2023-11-06T02:08:30.626345+0000","last_became_peered":"2023-11-06T02:08:30.626345+0000","last_unstale":"2023-11-06T02:08:31.610511+0000","last_undegraded":"2023-11-06T02:08:31.610511+0000","last_fullsized":"2023-11-06T02:08:31.610511+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:46:38.867945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.652203+0000","last_change":"2023-11-06T02:08:30.619667+0000","last_active":"2023-11-06T02:08:32.652203+0000","last_peered":"2023-11-06T02:08:32.652203+0000","last_clean":"2023-11-06T02:08:32.652203+0000","last_became_active":"2023-11-06T02:08:30.619328+0000","last_became_peered":"2023-11-06T02:08:30.619328+0000","last_unstale":"2023-11-06T02:08:32.652203+0000","last_undegraded":"2023-11-06T02:08:32.652203+0000","last_fullsized":"2023-11-06T02:08:32.652203+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:26:58.090934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.608847+0000","last_change":"2023-11-06T02:08:30.657202+0000","last_active":"2023-11-06T02:08:31.608847+0000","last_peered":"2023-11-06T02:08:31.608847+0000","last_clean":"2023-11-06T02:08:31.608847+0000","last_became_active":"2023-11-06T02:08:30.652132+0000","last_became_peered":"2023-11-06T02:08:30.652132+0000","last_unstale":"2023-11-06T02:08:31.608847+0000","last_undegraded":"2023-11-06T02:08:31.608847+0000","last_fullsized":"2023-11-06T02:08:31.608847+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:35:19.038827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.149393+0000","last_change":"2023-11-06T02:08:30.645694+0000","last_active":"2023-11-06T02:08:32.149393+0000","last_peered":"2023-11-06T02:08:32.149393+0000","last_clean":"2023-11-06T02:08:32.149393+0000","last_became_active":"2023-11-06T02:08:30.645028+0000","last_became_peered":"2023-11-06T02:08:30.645028+0000","last_unstale":"2023-11-06T02:08:32.149393+0000","last_undegraded":"2023-11-06T02:08:32.149393+0000","last_fullsized":"2023-11-06T02:08:32.149393+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:26:58.090934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.412662+0000","last_change":"2023-11-06T02:08:30.652585+0000","last_active":"2023-11-06T02:08:32.412662+0000","last_peered":"2023-11-06T02:08:32.412662+0000","last_clean":"2023-11-06T02:08:32.412662+0000","last_became_active":"2023-11-06T02:08:30.652103+0000","last_became_peered":"2023-11-06T02:08:30.652103+0000","last_unstale":"2023-11-06T02:08:32.412662+0000","last_undegraded":"2023-11-06T02:08:32.412662+0000","last_fullsized":"2023-11-06T02:08:32.412662+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:57:32.079075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.651445+0000","last_change":"2023-11-06T02:08:30.618708+0000","last_active":"2023-11-06T02:08:32.651445+0000","last_peered":"2023-11-06T02:08:32.651445+0000","last_clean":"2023-11-06T02:08:32.651445+0000","last_became_active":"2023-11-06T02:08:30.618232+0000","last_became_peered":"2023-11-06T02:08:30.618232+0000","last_unstale":"2023-11-06T02:08:32.651445+0000","last_undegraded":"2023-11-06T02:08:32.651445+0000","last_fullsized":"2023-11-06T02:08:32.651445+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:31:28.301664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.413222+0000","last_change":"2023-11-06T02:08:30.640907+0000","last_active":"2023-11-06T02:08:32.413222+0000","last_peered":"2023-11-06T02:08:32.413222+0000","last_clean":"2023-11-06T02:08:32.413222+0000","last_became_active":"2023-11-06T02:08:30.626276+0000","last_became_peered":"2023-11-06T02:08:30.626276+0000","last_unstale":"2023-11-06T02:08:32.413222+0000","last_undegraded":"2023-11-06T02:08:32.413222+0000","last_fullsized":"2023-11-06T02:08:32.413222+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:56:51.565719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.610554+0000","last_change":"2023-11-06T02:08:30.658929+0000","last_active":"2023-11-06T02:08:31.610554+0000","last_peered":"2023-11-06T02:08:31.610554+0000","last_clean":"2023-11-06T02:08:31.610554+0000","last_became_active":"2023-11-06T02:08:30.657695+0000","last_became_peered":"2023-11-06T02:08:30.657695+0000","last_unstale":"2023-11-06T02:08:31.610554+0000","last_undegraded":"2023-11-06T02:08:31.610554+0000","last_fullsized":"2023-11-06T02:08:31.610554+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:57:32.079075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.654812+0000","last_change":"2023-11-06T02:08:30.660659+0000","last_active":"2023-11-06T02:08:32.654812+0000","last_peered":"2023-11-06T02:08:32.654812+0000","last_clean":"2023-11-06T02:08:32.654812+0000","last_became_active":"2023-11-06T02:08:30.660402+0000","last_became_peered":"2023-11-06T02:08:30.660402+0000","last_unstale":"2023-11-06T02:08:32.654812+0000","last_undegraded":"2023-11-06T02:08:32.654812+0000","last_fullsized":"2023-11-06T02:08:32.654812+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:26:50.302398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.654670+0000","last_change":"2023-11-06T02:08:30.608913+0000","last_active":"2023-11-06T02:08:32.654670+0000","last_peered":"2023-11-06T02:08:32.654670+0000","last_clean":"2023-11-06T02:08:32.654670+0000","last_became_active":"2023-11-06T02:08:30.606808+0000","last_became_peered":"2023-11-06T02:08:30.606808+0000","last_unstale":"2023-11-06T02:08:32.654670+0000","last_undegraded":"2023-11-06T02:08:32.654670+0000","last_fullsized":"2023-11-06T02:08:32.654670+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:24:10.805757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.412935+0000","last_change":"2023-11-06T02:08:30.653750+0000","last_active":"2023-11-06T02:08:32.412935+0000","last_peered":"2023-11-06T02:08:32.412935+0000","last_clean":"2023-11-06T02:08:32.412935+0000","last_became_active":"2023-11-06T02:08:30.653252+0000","last_became_peered":"2023-11-06T02:08:30.653252+0000","last_unstale":"2023-11-06T02:08:32.412935+0000","last_undegraded":"2023-11-06T02:08:32.412935+0000","last_fullsized":"2023-11-06T02:08:32.412935+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:26:58.090934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.149589+0000","last_change":"2023-11-06T02:08:30.644591+0000","last_active":"2023-11-06T02:08:32.149589+0000","last_peered":"2023-11-06T02:08:32.149589+0000","last_clean":"2023-11-06T02:08:32.149589+0000","last_became_active":"2023-11-06T02:08:30.643848+0000","last_became_peered":"2023-11-06T02:08:30.643848+0000","last_unstale":"2023-11-06T02:08:32.149589+0000","last_undegraded":"2023-11-06T02:08:32.149589+0000","last_fullsized":"2023-11-06T02:08:32.149589+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:49:05.460965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.608423+0000","last_change":"2023-11-06T02:08:30.634202+0000","last_active":"2023-11-06T02:08:31.608423+0000","last_peered":"2023-11-06T02:08:31.608423+0000","last_clean":"2023-11-06T02:08:31.608423+0000","last_became_active":"2023-11-06T02:08:30.633451+0000","last_became_peered":"2023-11-06T02:08:30.633451+0000","last_unstale":"2023-11-06T02:08:31.608423+0000","last_undegraded":"2023-11-06T02:08:31.608423+0000","last_fullsized":"2023-11-06T02:08:31.608423+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:13:14.793314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.651622+0000","last_change":"2023-11-06T02:08:30.618515+0000","last_active":"2023-11-06T02:08:32.651622+0000","last_peered":"2023-11-06T02:08:32.651622+0000","last_clean":"2023-11-06T02:08:32.651622+0000","last_became_active":"2023-11-06T02:08:30.617836+0000","last_became_peered":"2023-11-06T02:08:30.617836+0000","last_unstale":"2023-11-06T02:08:32.651622+0000","last_undegraded":"2023-11-06T02:08:32.651622+0000","last_fullsized":"2023-11-06T02:08:32.651622+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:17:50.007398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.609393+0000","last_change":"2023-11-06T02:08:30.659348+0000","last_active":"2023-11-06T02:08:31.609393+0000","last_peered":"2023-11-06T02:08:31.609393+0000","last_clean":"2023-11-06T02:08:31.609393+0000","last_became_active":"2023-11-06T02:08:30.652231+0000","last_became_peered":"2023-11-06T02:08:30.652231+0000","last_unstale":"2023-11-06T02:08:31.609393+0000","last_undegraded":"2023-11-06T02:08:31.609393+0000","last_fullsized":"2023-11-06T02:08:31.609393+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:13:26.942266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.655090+0000","last_change":"2023-11-06T02:08:30.627395+0000","last_active":"2023-11-06T02:08:32.655090+0000","last_peered":"2023-11-06T02:08:32.655090+0000","last_clean":"2023-11-06T02:08:32.655090+0000","last_became_active":"2023-11-06T02:08:30.627048+0000","last_became_peered":"2023-11-06T02:08:30.627048+0000","last_unstale":"2023-11-06T02:08:32.655090+0000","last_undegraded":"2023-11-06T02:08:32.655090+0000","last_fullsized":"2023-11-06T02:08:32.655090+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:15:59.045303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.609522+0000","last_change":"2023-11-06T02:08:30.658560+0000","last_active":"2023-11-06T02:08:31.609522+0000","last_peered":"2023-11-06T02:08:31.609522+0000","last_clean":"2023-11-06T02:08:31.609522+0000","last_became_active":"2023-11-06T02:08:30.657560+0000","last_became_peered":"2023-11-06T02:08:30.657560+0000","last_unstale":"2023-11-06T02:08:31.609522+0000","last_undegraded":"2023-11-06T02:08:31.609522+0000","last_fullsized":"2023-11-06T02:08:31.609522+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:26:06.040192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.149579+0000","last_change":"2023-11-06T02:08:30.650121+0000","last_active":"2023-11-06T02:08:32.149579+0000","last_peered":"2023-11-06T02:08:32.149579+0000","last_clean":"2023-11-06T02:08:32.149579+0000","last_became_active":"2023-11-06T02:08:30.649010+0000","last_became_peered":"2023-11-06T02:08:30.649010+0000","last_unstale":"2023-11-06T02:08:32.149579+0000","last_undegraded":"2023-11-06T02:08:32.149579+0000","last_fullsized":"2023-11-06T02:08:32.149579+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:56:51.565719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.651624+0000","last_change":"2023-11-06T02:08:30.622216+0000","last_active":"2023-11-06T02:08:32.651624+0000","last_peered":"2023-11-06T02:08:32.651624+0000","last_clean":"2023-11-06T02:08:32.651624+0000","last_became_active":"2023-11-06T02:08:30.621706+0000","last_became_peered":"2023-11-06T02:08:30.621706+0000","last_unstale":"2023-11-06T02:08:32.651624+0000","last_undegraded":"2023-11-06T02:08:32.651624+0000","last_fullsized":"2023-11-06T02:08:32.651624+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:52:47.174384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.604379+0000","last_change":"2023-11-06T02:08:30.623436+0000","last_active":"2023-11-06T02:08:31.604379+0000","last_peered":"2023-11-06T02:08:31.604379+0000","last_clean":"2023-11-06T02:08:31.604379+0000","last_became_active":"2023-11-06T02:08:30.623149+0000","last_became_peered":"2023-11-06T02:08:30.623149+0000","last_unstale":"2023-11-06T02:08:31.604379+0000","last_undegraded":"2023-11-06T02:08:31.604379+0000","last_fullsized":"2023-11-06T02:08:31.604379+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:47:30.859032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.149601+0000","last_change":"2023-11-06T02:08:30.656899+0000","last_active":"2023-11-06T02:08:32.149601+0000","last_peered":"2023-11-06T02:08:32.149601+0000","last_clean":"2023-11-06T02:08:32.149601+0000","last_became_active":"2023-11-06T02:08:30.656319+0000","last_became_peered":"2023-11-06T02:08:30.656319+0000","last_unstale":"2023-11-06T02:08:32.149601+0000","last_undegraded":"2023-11-06T02:08:32.149601+0000","last_fullsized":"2023-11-06T02:08:32.149601+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:57:32.079075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.652785+0000","last_change":"2023-11-06T02:08:30.627856+0000","last_active":"2023-11-06T02:08:32.652785+0000","last_peered":"2023-11-06T02:08:32.652785+0000","last_clean":"2023-11-06T02:08:32.652785+0000","last_became_active":"2023-11-06T02:08:30.627536+0000","last_became_peered":"2023-11-06T02:08:30.627536+0000","last_unstale":"2023-11-06T02:08:32.652785+0000","last_undegraded":"2023-11-06T02:08:32.652785+0000","last_fullsized":"2023-11-06T02:08:32.652785+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:53:11.188799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.652773+0000","last_change":"2023-11-06T02:08:30.613522+0000","last_active":"2023-11-06T02:08:32.652773+0000","last_peered":"2023-11-06T02:08:32.652773+0000","last_clean":"2023-11-06T02:08:32.652773+0000","last_became_active":"2023-11-06T02:08:30.612950+0000","last_became_peered":"2023-11-06T02:08:30.612950+0000","last_unstale":"2023-11-06T02:08:32.652773+0000","last_undegraded":"2023-11-06T02:08:32.652773+0000","last_fullsized":"2023-11-06T02:08:32.652773+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:26:58.090934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.413094+0000","last_change":"2023-11-06T02:08:30.653593+0000","last_active":"2023-11-06T02:08:32.413094+0000","last_peered":"2023-11-06T02:08:32.413094+0000","last_clean":"2023-11-06T02:08:32.413094+0000","last_became_active":"2023-11-06T02:08:30.652925+0000","last_became_peered":"2023-11-06T02:08:30.652925+0000","last_unstale":"2023-11-06T02:08:32.413094+0000","last_undegraded":"2023-11-06T02:08:32.413094+0000","last_fullsized":"2023-11-06T02:08:32.413094+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:17:50.007398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.652200+0000","last_change":"2023-11-06T02:08:30.645260+0000","last_active":"2023-11-06T02:08:32.652200+0000","last_peered":"2023-11-06T02:08:32.652200+0000","last_clean":"2023-11-06T02:08:32.652200+0000","last_became_active":"2023-11-06T02:08:30.644949+0000","last_became_peered":"2023-11-06T02:08:30.644949+0000","last_unstale":"2023-11-06T02:08:32.652200+0000","last_undegraded":"2023-11-06T02:08:32.652200+0000","last_fullsized":"2023-11-06T02:08:32.652200+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:41:00.973671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.604763+0000","last_change":"2023-11-06T02:08:30.649814+0000","last_active":"2023-11-06T02:08:31.604763+0000","last_peered":"2023-11-06T02:08:31.604763+0000","last_clean":"2023-11-06T02:08:31.604763+0000","last_became_active":"2023-11-06T02:08:30.649492+0000","last_became_peered":"2023-11-06T02:08:30.649492+0000","last_unstale":"2023-11-06T02:08:31.604763+0000","last_undegraded":"2023-11-06T02:08:31.604763+0000","last_fullsized":"2023-11-06T02:08:31.604763+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:26:58.090934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.609372+0000","last_change":"2023-11-06T02:08:30.632560+0000","last_active":"2023-11-06T02:08:31.609372+0000","last_peered":"2023-11-06T02:08:31.609372+0000","last_clean":"2023-11-06T02:08:31.609372+0000","last_became_active":"2023-11-06T02:08:30.631819+0000","last_became_peered":"2023-11-06T02:08:30.631819+0000","last_unstale":"2023-11-06T02:08:31.609372+0000","last_undegraded":"2023-11-06T02:08:31.609372+0000","last_fullsized":"2023-11-06T02:08:31.609372+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:43:23.215718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.604562+0000","last_change":"2023-11-06T02:08:30.624052+0000","last_active":"2023-11-06T02:08:31.604562+0000","last_peered":"2023-11-06T02:08:31.604562+0000","last_clean":"2023-11-06T02:08:31.604562+0000","last_became_active":"2023-11-06T02:08:30.623658+0000","last_became_peered":"2023-11-06T02:08:30.623658+0000","last_unstale":"2023-11-06T02:08:31.604562+0000","last_undegraded":"2023-11-06T02:08:31.604562+0000","last_fullsized":"2023-11-06T02:08:31.604562+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:17:50.007398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.609631+0000","last_change":"2023-11-06T02:08:30.614066+0000","last_active":"2023-11-06T02:08:31.609631+0000","last_peered":"2023-11-06T02:08:31.609631+0000","last_clean":"2023-11-06T02:08:31.609631+0000","last_became_active":"2023-11-06T02:08:30.613337+0000","last_became_peered":"2023-11-06T02:08:30.613337+0000","last_unstale":"2023-11-06T02:08:31.609631+0000","last_undegraded":"2023-11-06T02:08:31.609631+0000","last_fullsized":"2023-11-06T02:08:31.609631+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:33:39.359552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.604963+0000","last_change":"2023-11-06T02:08:30.621368+0000","last_active":"2023-11-06T02:08:31.604963+0000","last_peered":"2023-11-06T02:08:31.604963+0000","last_clean":"2023-11-06T02:08:31.604963+0000","last_became_active":"2023-11-06T02:08:30.620563+0000","last_became_peered":"2023-11-06T02:08:30.620563+0000","last_unstale":"2023-11-06T02:08:31.604963+0000","last_undegraded":"2023-11-06T02:08:31.604963+0000","last_fullsized":"2023-11-06T02:08:31.604963+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:56:51.565719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"22'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.149865+0000","last_change":"2023-11-06T02:08:30.229825+0000","last_active":"2023-11-06T02:08:32.149865+0000","last_peered":"2023-11-06T02:08:32.149865+0000","last_clean":"2023-11-06T02:08:32.149865+0000","last_became_active":"2023-11-06T02:08:30.229392+0000","last_became_peered":"2023-11-06T02:08:30.229392+0000","last_unstale":"2023-11-06T02:08:32.149865+0000","last_undegraded":"2023-11-06T02:08:32.149865+0000","last_fullsized":"2023-11-06T02:08:32.149865+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:33:38.355715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.413299+0000","last_change":"2023-11-06T02:08:30.646474+0000","last_active":"2023-11-06T02:08:32.413299+0000","last_peered":"2023-11-06T02:08:32.413299+0000","last_clean":"2023-11-06T02:08:32.413299+0000","last_became_active":"2023-11-06T02:08:30.645992+0000","last_became_peered":"2023-11-06T02:08:30.645992+0000","last_unstale":"2023-11-06T02:08:32.413299+0000","last_undegraded":"2023-11-06T02:08:32.413299+0000","last_fullsized":"2023-11-06T02:08:32.413299+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:47:30.859032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.651766+0000","last_change":"2023-11-06T02:08:30.225818+0000","last_active":"2023-11-06T02:08:32.651766+0000","last_peered":"2023-11-06T02:08:32.651766+0000","last_clean":"2023-11-06T02:08:32.651766+0000","last_became_active":"2023-11-06T02:08:30.225359+0000","last_became_peered":"2023-11-06T02:08:30.225359+0000","last_unstale":"2023-11-06T02:08:32.651766+0000","last_undegraded":"2023-11-06T02:08:32.651766+0000","last_fullsized":"2023-11-06T02:08:32.651766+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:52:19.592716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.653330+0000","last_change":"2023-11-06T02:08:30.618020+0000","last_active":"2023-11-06T02:08:32.653330+0000","last_peered":"2023-11-06T02:08:32.653330+0000","last_clean":"2023-11-06T02:08:32.653330+0000","last_became_active":"2023-11-06T02:08:30.617684+0000","last_became_peered":"2023-11-06T02:08:30.617684+0000","last_unstale":"2023-11-06T02:08:32.653330+0000","last_undegraded":"2023-11-06T02:08:32.653330+0000","last_fullsized":"2023-11-06T02:08:32.653330+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:56:51.565719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.149630+0000","last_change":"2023-11-06T02:08:30.230443+0000","last_active":"2023-11-06T02:08:32.149630+0000","last_peered":"2023-11-06T02:08:32.149630+0000","last_clean":"2023-11-06T02:08:32.149630+0000","last_became_active":"2023-11-06T02:08:30.229980+0000","last_became_peered":"2023-11-06T02:08:30.229980+0000","last_unstale":"2023-11-06T02:08:32.149630+0000","last_undegraded":"2023-11-06T02:08:32.149630+0000","last_fullsized":"2023-11-06T02:08:32.149630+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:46:37.864109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.605377+0000","last_change":"2023-11-06T02:08:30.647210+0000","last_active":"2023-11-06T02:08:31.605377+0000","last_peered":"2023-11-06T02:08:31.605377+0000","last_clean":"2023-11-06T02:08:31.605377+0000","last_became_active":"2023-11-06T02:08:30.646862+0000","last_became_peered":"2023-11-06T02:08:30.646862+0000","last_unstale":"2023-11-06T02:08:31.605377+0000","last_undegraded":"2023-11-06T02:08:31.605377+0000","last_fullsized":"2023-11-06T02:08:31.605377+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:43:23.215718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"22'8","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.609883+0000","last_change":"2023-11-06T02:08:30.218861+0000","last_active":"2023-11-06T02:08:31.609883+0000","last_peered":"2023-11-06T02:08:31.609883+0000","last_clean":"2023-11-06T02:08:31.609883+0000","last_became_active":"2023-11-06T02:08:30.218245+0000","last_became_peered":"2023-11-06T02:08:30.218245+0000","last_unstale":"2023-11-06T02:08:31.609883+0000","last_undegraded":"2023-11-06T02:08:31.609883+0000","last_fullsized":"2023-11-06T02:08:31.609883+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:04:42.682132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"22'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.150304+0000","last_change":"2023-11-06T02:08:30.229578+0000","last_active":"2023-11-06T02:08:32.150304+0000","last_peered":"2023-11-06T02:08:32.150304+0000","last_clean":"2023-11-06T02:08:32.150304+0000","last_became_active":"2023-11-06T02:08:30.229131+0000","last_became_peered":"2023-11-06T02:08:30.229131+0000","last_unstale":"2023-11-06T02:08:32.150304+0000","last_undegraded":"2023-11-06T02:08:32.150304+0000","last_fullsized":"2023-11-06T02:08:32.150304+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:30:49.856478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.412653+0000","last_change":"2023-11-06T02:08:30.647714+0000","last_active":"2023-11-06T02:08:32.412653+0000","last_peered":"2023-11-06T02:08:32.412653+0000","last_clean":"2023-11-06T02:08:32.412653+0000","last_became_active":"2023-11-06T02:08:30.647212+0000","last_became_peered":"2023-11-06T02:08:30.647212+0000","last_unstale":"2023-11-06T02:08:32.412653+0000","last_undegraded":"2023-11-06T02:08:32.412653+0000","last_fullsized":"2023-11-06T02:08:32.412653+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:13:37.616527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"22'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.150130+0000","last_change":"2023-11-06T02:08:30.230469+0000","last_active":"2023-11-06T02:08:32.150130+0000","last_peered":"2023-11-06T02:08:32.150130+0000","last_clean":"2023-11-06T02:08:32.150130+0000","last_became_active":"2023-11-06T02:08:30.230064+0000","last_became_peered":"2023-11-06T02:08:30.230064+0000","last_unstale":"2023-11-06T02:08:32.150130+0000","last_undegraded":"2023-11-06T02:08:32.150130+0000","last_fullsized":"2023-11-06T02:08:32.150130+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:19:59.347221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.608834+0000","last_change":"2023-11-06T02:08:30.632754+0000","last_active":"2023-11-06T02:08:31.608834+0000","last_peered":"2023-11-06T02:08:31.608834+0000","last_clean":"2023-11-06T02:08:31.608834+0000","last_became_active":"2023-11-06T02:08:30.631976+0000","last_became_peered":"2023-11-06T02:08:30.631976+0000","last_unstale":"2023-11-06T02:08:31.608834+0000","last_undegraded":"2023-11-06T02:08:31.608834+0000","last_fullsized":"2023-11-06T02:08:31.608834+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:56:51.565719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"22'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.149972+0000","last_change":"2023-11-06T02:08:30.231003+0000","last_active":"2023-11-06T02:08:32.149972+0000","last_peered":"2023-11-06T02:08:32.149972+0000","last_clean":"2023-11-06T02:08:32.149972+0000","last_became_active":"2023-11-06T02:08:30.229978+0000","last_became_peered":"2023-11-06T02:08:30.229978+0000","last_unstale":"2023-11-06T02:08:32.149972+0000","last_undegraded":"2023-11-06T02:08:32.149972+0000","last_fullsized":"2023-11-06T02:08:32.149972+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:24:09.801921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.609985+0000","last_change":"2023-11-06T02:08:30.619807+0000","last_active":"2023-11-06T02:08:31.609985+0000","last_peered":"2023-11-06T02:08:31.609985+0000","last_clean":"2023-11-06T02:08:31.609985+0000","last_became_active":"2023-11-06T02:08:30.619048+0000","last_became_peered":"2023-11-06T02:08:30.619048+0000","last_unstale":"2023-11-06T02:08:31.609985+0000","last_undegraded":"2023-11-06T02:08:31.609985+0000","last_fullsized":"2023-11-06T02:08:31.609985+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:15:59.045303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.412965+0000","last_change":"2023-11-06T02:08:30.218715+0000","last_active":"2023-11-06T02:08:32.412965+0000","last_peered":"2023-11-06T02:08:32.412965+0000","last_clean":"2023-11-06T02:08:32.412965+0000","last_became_active":"2023-11-06T02:08:30.217966+0000","last_became_peered":"2023-11-06T02:08:30.217966+0000","last_unstale":"2023-11-06T02:08:32.412965+0000","last_undegraded":"2023-11-06T02:08:32.412965+0000","last_fullsized":"2023-11-06T02:08:32.412965+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:30:21.274364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.652004+0000","last_change":"2023-11-06T02:08:30.620567+0000","last_active":"2023-11-06T02:08:32.652004+0000","last_peered":"2023-11-06T02:08:32.652004+0000","last_clean":"2023-11-06T02:08:32.652004+0000","last_became_active":"2023-11-06T02:08:30.619671+0000","last_became_peered":"2023-11-06T02:08:30.619671+0000","last_unstale":"2023-11-06T02:08:32.652004+0000","last_undegraded":"2023-11-06T02:08:32.652004+0000","last_fullsized":"2023-11-06T02:08:32.652004+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:57:32.079075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"22'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.605088+0000","last_change":"2023-11-06T02:08:30.220060+0000","last_active":"2023-11-06T02:08:31.605088+0000","last_peered":"2023-11-06T02:08:31.605088+0000","last_clean":"2023-11-06T02:08:31.605088+0000","last_became_active":"2023-11-06T02:08:30.218534+0000","last_became_peered":"2023-11-06T02:08:30.218534+0000","last_unstale":"2023-11-06T02:08:31.605088+0000","last_undegraded":"2023-11-06T02:08:31.605088+0000","last_fullsized":"2023-11-06T02:08:31.605088+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:30:21.274364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.609657+0000","last_change":"2023-11-06T02:08:30.658173+0000","last_active":"2023-11-06T02:08:31.609657+0000","last_peered":"2023-11-06T02:08:31.609657+0000","last_clean":"2023-11-06T02:08:31.609657+0000","last_became_active":"2023-11-06T02:08:30.656890+0000","last_became_peered":"2023-11-06T02:08:30.656890+0000","last_unstale":"2023-11-06T02:08:31.609657+0000","last_undegraded":"2023-11-06T02:08:31.609657+0000","last_fullsized":"2023-11-06T02:08:31.609657+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:23:59.639528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.610321+0000","last_change":"2023-11-06T02:08:30.222023+0000","last_active":"2023-11-06T02:08:31.610321+0000","last_peered":"2023-11-06T02:08:31.610321+0000","last_clean":"2023-11-06T02:08:31.610321+0000","last_became_active":"2023-11-06T02:08:30.220788+0000","last_became_peered":"2023-11-06T02:08:30.220788+0000","last_unstale":"2023-11-06T02:08:31.610321+0000","last_undegraded":"2023-11-06T02:08:31.610321+0000","last_fullsized":"2023-11-06T02:08:31.610321+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:31:31.882709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.610620+0000","last_change":"2023-11-06T02:08:30.635685+0000","last_active":"2023-11-06T02:08:31.610620+0000","last_peered":"2023-11-06T02:08:31.610620+0000","last_clean":"2023-11-06T02:08:31.610620+0000","last_became_active":"2023-11-06T02:08:30.634801+0000","last_became_peered":"2023-11-06T02:08:30.634801+0000","last_unstale":"2023-11-06T02:08:31.610620+0000","last_undegraded":"2023-11-06T02:08:31.610620+0000","last_fullsized":"2023-11-06T02:08:31.610620+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:37:47.953218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.609115+0000","last_change":"2023-11-06T02:08:30.222592+0000","last_active":"2023-11-06T02:08:31.609115+0000","last_peered":"2023-11-06T02:08:31.609115+0000","last_clean":"2023-11-06T02:08:31.609115+0000","last_became_active":"2023-11-06T02:08:30.222184+0000","last_became_peered":"2023-11-06T02:08:30.222184+0000","last_unstale":"2023-11-06T02:08:31.609115+0000","last_undegraded":"2023-11-06T02:08:31.609115+0000","last_fullsized":"2023-11-06T02:08:31.609115+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:31:31.882709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.609663+0000","last_change":"2023-11-06T02:08:30.651763+0000","last_active":"2023-11-06T02:08:31.609663+0000","last_peered":"2023-11-06T02:08:31.609663+0000","last_clean":"2023-11-06T02:08:31.609663+0000","last_became_active":"2023-11-06T02:08:30.651013+0000","last_became_peered":"2023-11-06T02:08:30.651013+0000","last_unstale":"2023-11-06T02:08:31.609663+0000","last_undegraded":"2023-11-06T02:08:31.609663+0000","last_fullsized":"2023-11-06T02:08:31.609663+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:41:51.822994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.604575+0000","last_change":"2023-11-06T02:08:30.225279+0000","last_active":"2023-11-06T02:08:31.604575+0000","last_peered":"2023-11-06T02:08:31.604575+0000","last_clean":"2023-11-06T02:08:31.604575+0000","last_became_active":"2023-11-06T02:08:30.224936+0000","last_became_peered":"2023-11-06T02:08:30.224936+0000","last_unstale":"2023-11-06T02:08:31.604575+0000","last_undegraded":"2023-11-06T02:08:31.604575+0000","last_fullsized":"2023-11-06T02:08:31.604575+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:52:19.592716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.653044+0000","last_change":"2023-11-06T02:08:30.619652+0000","last_active":"2023-11-06T02:08:32.653044+0000","last_peered":"2023-11-06T02:08:32.653044+0000","last_clean":"2023-11-06T02:08:32.653044+0000","last_became_active":"2023-11-06T02:08:30.619070+0000","last_became_peered":"2023-11-06T02:08:30.619070+0000","last_unstale":"2023-11-06T02:08:32.653044+0000","last_undegraded":"2023-11-06T02:08:32.653044+0000","last_fullsized":"2023-11-06T02:08:32.653044+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:56:51.565719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":167,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-06T02:08:30.628264+0000","last_change":"2023-11-06T02:08:20.523427+0000","last_active":"2023-11-06T02:08:30.628264+0000","last_peered":"2023-11-06T02:08:30.628264+0000","last_clean":"2023-11-06T02:08:30.628264+0000","last_became_active":"2023-11-06T02:08:20.521089+0000","last_became_peered":"2023-11-06T02:08:20.521089+0000","last_unstale":"2023-11-06T02:08:30.628264+0000","last_undegraded":"2023-11-06T02:08:30.628264+0000","last_fullsized":"2023-11-06T02:08:30.628264+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:19.252348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:19.252348+0000","last_clean_scrub_stamp":"2023-11-06T02:08:19.252348+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:30:15.566397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-06T02:08:30.640657+0000","last_change":"2023-11-06T02:08:26.059502+0000","last_active":"2023-11-06T02:08:30.640657+0000","last_peered":"2023-11-06T02:08:30.640657+0000","last_clean":"2023-11-06T02:08:30.640657+0000","last_became_active":"2023-11-06T02:08:26.058612+0000","last_became_peered":"2023-11-06T02:08:26.058612+0000","last_unstale":"2023-11-06T02:08:30.640657+0000","last_undegraded":"2023-11-06T02:08:30.640657+0000","last_fullsized":"2023-11-06T02:08:30.640657+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:10.626021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.604588+0000","last_change":"2023-11-06T02:08:30.219125+0000","last_active":"2023-11-06T02:08:31.604588+0000","last_peered":"2023-11-06T02:08:31.604588+0000","last_clean":"2023-11-06T02:08:31.604588+0000","last_became_active":"2023-11-06T02:08:30.217091+0000","last_became_peered":"2023-11-06T02:08:30.217091+0000","last_unstale":"2023-11-06T02:08:31.604588+0000","last_undegraded":"2023-11-06T02:08:31.604588+0000","last_fullsized":"2023-11-06T02:08:31.604588+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:04:42.682132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.610770+0000","last_change":"2023-11-06T02:08:30.660650+0000","last_active":"2023-11-06T02:08:31.610770+0000","last_peered":"2023-11-06T02:08:31.610770+0000","last_clean":"2023-11-06T02:08:31.610770+0000","last_became_active":"2023-11-06T02:08:30.660072+0000","last_became_peered":"2023-11-06T02:08:30.660072+0000","last_unstale":"2023-11-06T02:08:31.610770+0000","last_undegraded":"2023-11-06T02:08:31.610770+0000","last_fullsized":"2023-11-06T02:08:31.610770+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:30:50.860315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-06T02:08:30.626279+0000","last_change":"2023-11-06T02:08:26.060406+0000","last_active":"2023-11-06T02:08:30.626279+0000","last_peered":"2023-11-06T02:08:30.626279+0000","last_clean":"2023-11-06T02:08:30.626279+0000","last_became_active":"2023-11-06T02:08:26.059262+0000","last_became_peered":"2023-11-06T02:08:26.059262+0000","last_unstale":"2023-11-06T02:08:30.626279+0000","last_undegraded":"2023-11-06T02:08:30.626279+0000","last_fullsized":"2023-11-06T02:08:30.626279+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:10.626021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.651421+0000","last_change":"2023-11-06T02:08:30.227671+0000","last_active":"2023-11-06T02:08:32.651421+0000","last_peered":"2023-11-06T02:08:32.651421+0000","last_clean":"2023-11-06T02:08:32.651421+0000","last_became_active":"2023-11-06T02:08:30.227200+0000","last_became_peered":"2023-11-06T02:08:30.227200+0000","last_unstale":"2023-11-06T02:08:32.651421+0000","last_undegraded":"2023-11-06T02:08:32.651421+0000","last_fullsized":"2023-11-06T02:08:32.651421+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:26:49.298561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.653133+0000","last_change":"2023-11-06T02:08:30.619424+0000","last_active":"2023-11-06T02:08:32.653133+0000","last_peered":"2023-11-06T02:08:32.653133+0000","last_clean":"2023-11-06T02:08:32.653133+0000","last_became_active":"2023-11-06T02:08:30.619150+0000","last_became_peered":"2023-11-06T02:08:30.619150+0000","last_unstale":"2023-11-06T02:08:32.653133+0000","last_undegraded":"2023-11-06T02:08:32.653133+0000","last_fullsized":"2023-11-06T02:08:32.653133+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:47:30.859032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-06T02:08:30.597659+0000","last_change":"2023-11-06T02:08:26.559477+0000","last_active":"2023-11-06T02:08:30.597659+0000","last_peered":"2023-11-06T02:08:30.597659+0000","last_clean":"2023-11-06T02:08:30.597659+0000","last_became_active":"2023-11-06T02:08:26.559202+0000","last_became_peered":"2023-11-06T02:08:26.559202+0000","last_unstale":"2023-11-06T02:08:30.597659+0000","last_undegraded":"2023-11-06T02:08:30.597659+0000","last_fullsized":"2023-11-06T02:08:30.597659+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:55:04.552404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.651066+0000","last_change":"2023-11-06T02:08:30.219422+0000","last_active":"2023-11-06T02:08:32.651066+0000","last_peered":"2023-11-06T02:08:32.651066+0000","last_clean":"2023-11-06T02:08:32.651066+0000","last_became_active":"2023-11-06T02:08:30.219148+0000","last_became_peered":"2023-11-06T02:08:30.219148+0000","last_unstale":"2023-11-06T02:08:32.651066+0000","last_undegraded":"2023-11-06T02:08:32.651066+0000","last_fullsized":"2023-11-06T02:08:32.651066+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:52:19.592716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.653298+0000","last_change":"2023-11-06T02:08:30.655629+0000","last_active":"2023-11-06T02:08:32.653298+0000","last_peered":"2023-11-06T02:08:32.653298+0000","last_clean":"2023-11-06T02:08:32.653298+0000","last_became_active":"2023-11-06T02:08:30.655271+0000","last_became_peered":"2023-11-06T02:08:30.655271+0000","last_unstale":"2023-11-06T02:08:32.653298+0000","last_undegraded":"2023-11-06T02:08:32.653298+0000","last_fullsized":"2023-11-06T02:08:32.653298+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:10:14.974545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-06T02:08:30.641603+0000","last_change":"2023-11-06T02:08:26.059546+0000","last_active":"2023-11-06T02:08:30.641603+0000","last_peered":"2023-11-06T02:08:30.641603+0000","last_clean":"2023-11-06T02:08:30.641603+0000","last_became_active":"2023-11-06T02:08:26.058806+0000","last_became_peered":"2023-11-06T02:08:26.058806+0000","last_unstale":"2023-11-06T02:08:30.641603+0000","last_undegraded":"2023-11-06T02:08:30.641603+0000","last_fullsized":"2023-11-06T02:08:30.641603+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:23.468510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.609639+0000","last_change":"2023-11-06T02:08:30.223844+0000","last_active":"2023-11-06T02:08:31.609639+0000","last_peered":"2023-11-06T02:08:31.609639+0000","last_clean":"2023-11-06T02:08:31.609639+0000","last_became_active":"2023-11-06T02:08:30.222902+0000","last_became_peered":"2023-11-06T02:08:30.222902+0000","last_unstale":"2023-11-06T02:08:31.609639+0000","last_undegraded":"2023-11-06T02:08:31.609639+0000","last_fullsized":"2023-11-06T02:08:31.609639+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:52:19.592716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.651748+0000","last_change":"2023-11-06T02:08:30.619125+0000","last_active":"2023-11-06T02:08:32.651748+0000","last_peered":"2023-11-06T02:08:32.651748+0000","last_clean":"2023-11-06T02:08:32.651748+0000","last_became_active":"2023-11-06T02:08:30.617621+0000","last_became_peered":"2023-11-06T02:08:30.617621+0000","last_unstale":"2023-11-06T02:08:32.651748+0000","last_undegraded":"2023-11-06T02:08:32.651748+0000","last_fullsized":"2023-11-06T02:08:32.651748+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:13:37.616527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.413429+0000","last_change":"2023-11-06T02:08:30.225274+0000","last_active":"2023-11-06T02:08:32.413429+0000","last_peered":"2023-11-06T02:08:32.413429+0000","last_clean":"2023-11-06T02:08:32.413429+0000","last_became_active":"2023-11-06T02:08:30.224275+0000","last_became_peered":"2023-11-06T02:08:30.224275+0000","last_unstale":"2023-11-06T02:08:32.413429+0000","last_undegraded":"2023-11-06T02:08:32.413429+0000","last_fullsized":"2023-11-06T02:08:32.413429+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:26:49.298561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-06T02:08:30.614515+0000","last_change":"2023-11-06T02:08:26.436463+0000","last_active":"2023-11-06T02:08:30.614515+0000","last_peered":"2023-11-06T02:08:30.614515+0000","last_clean":"2023-11-06T02:08:30.614515+0000","last_became_active":"2023-11-06T02:08:26.436075+0000","last_became_peered":"2023-11-06T02:08:26.436075+0000","last_unstale":"2023-11-06T02:08:30.614515+0000","last_undegraded":"2023-11-06T02:08:30.614515+0000","last_fullsized":"2023-11-06T02:08:30.614515+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:23.468510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.651810+0000","last_change":"2023-11-06T02:08:30.655634+0000","last_active":"2023-11-06T02:08:32.651810+0000","last_peered":"2023-11-06T02:08:32.651810+0000","last_clean":"2023-11-06T02:08:32.651810+0000","last_became_active":"2023-11-06T02:08:30.655285+0000","last_became_peered":"2023-11-06T02:08:30.655285+0000","last_unstale":"2023-11-06T02:08:32.651810+0000","last_undegraded":"2023-11-06T02:08:32.651810+0000","last_fullsized":"2023-11-06T02:08:32.651810+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:49:05.460965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.651964+0000","last_change":"2023-11-06T02:08:30.220327+0000","last_active":"2023-11-06T02:08:32.651964+0000","last_peered":"2023-11-06T02:08:32.651964+0000","last_clean":"2023-11-06T02:08:32.651964+0000","last_became_active":"2023-11-06T02:08:30.219906+0000","last_became_peered":"2023-11-06T02:08:30.219906+0000","last_unstale":"2023-11-06T02:08:32.651964+0000","last_undegraded":"2023-11-06T02:08:32.651964+0000","last_fullsized":"2023-11-06T02:08:32.651964+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:43:56.017919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-06T02:08:30.636949+0000","last_change":"2023-11-06T02:08:26.436607+0000","last_active":"2023-11-06T02:08:30.636949+0000","last_peered":"2023-11-06T02:08:30.636949+0000","last_clean":"2023-11-06T02:08:30.636949+0000","last_became_active":"2023-11-06T02:08:26.435924+0000","last_became_peered":"2023-11-06T02:08:26.435924+0000","last_unstale":"2023-11-06T02:08:30.636949+0000","last_undegraded":"2023-11-06T02:08:30.636949+0000","last_fullsized":"2023-11-06T02:08:30.636949+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:23.468510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.652082+0000","last_change":"2023-11-06T02:08:30.654058+0000","last_active":"2023-11-06T02:08:32.652082+0000","last_peered":"2023-11-06T02:08:32.652082+0000","last_clean":"2023-11-06T02:08:32.652082+0000","last_became_active":"2023-11-06T02:08:30.653753+0000","last_became_peered":"2023-11-06T02:08:30.653753+0000","last_unstale":"2023-11-06T02:08:32.652082+0000","last_undegraded":"2023-11-06T02:08:32.652082+0000","last_fullsized":"2023-11-06T02:08:32.652082+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:13:37.616527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"22'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.605569+0000","last_change":"2023-11-06T02:08:30.231205+0000","last_active":"2023-11-06T02:08:31.605569+0000","last_peered":"2023-11-06T02:08:31.605569+0000","last_clean":"2023-11-06T02:08:31.605569+0000","last_became_active":"2023-11-06T02:08:30.230854+0000","last_became_peered":"2023-11-06T02:08:30.230854+0000","last_unstale":"2023-11-06T02:08:31.605569+0000","last_undegraded":"2023-11-06T02:08:31.605569+0000","last_fullsized":"2023-11-06T02:08:31.605569+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:26:49.298561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-06T02:08:30.614693+0000","last_change":"2023-11-06T02:08:26.057076+0000","last_active":"2023-11-06T02:08:30.614693+0000","last_peered":"2023-11-06T02:08:30.614693+0000","last_clean":"2023-11-06T02:08:30.614693+0000","last_became_active":"2023-11-06T02:08:26.056419+0000","last_became_peered":"2023-11-06T02:08:26.056419+0000","last_unstale":"2023-11-06T02:08:30.614693+0000","last_undegraded":"2023-11-06T02:08:30.614693+0000","last_fullsized":"2023-11-06T02:08:30.614693+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:10.626021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.604395+0000","last_change":"2023-11-06T02:08:30.647144+0000","last_active":"2023-11-06T02:08:31.604395+0000","last_peered":"2023-11-06T02:08:31.604395+0000","last_clean":"2023-11-06T02:08:31.604395+0000","last_became_active":"2023-11-06T02:08:30.646802+0000","last_became_peered":"2023-11-06T02:08:30.646802+0000","last_unstale":"2023-11-06T02:08:31.604395+0000","last_undegraded":"2023-11-06T02:08:31.604395+0000","last_fullsized":"2023-11-06T02:08:31.604395+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:49:05.460965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"22'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.653477+0000","last_change":"2023-11-06T02:08:30.225086+0000","last_active":"2023-11-06T02:08:32.653477+0000","last_peered":"2023-11-06T02:08:32.653477+0000","last_clean":"2023-11-06T02:08:32.653477+0000","last_became_active":"2023-11-06T02:08:30.224455+0000","last_became_peered":"2023-11-06T02:08:30.224455+0000","last_unstale":"2023-11-06T02:08:32.653477+0000","last_undegraded":"2023-11-06T02:08:32.653477+0000","last_fullsized":"2023-11-06T02:08:32.653477+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:26:21.361289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-06T02:08:30.606381+0000","last_change":"2023-11-06T02:08:26.058596+0000","last_active":"2023-11-06T02:08:30.606381+0000","last_peered":"2023-11-06T02:08:30.606381+0000","last_clean":"2023-11-06T02:08:30.606381+0000","last_became_active":"2023-11-06T02:08:26.057015+0000","last_became_peered":"2023-11-06T02:08:26.057015+0000","last_unstale":"2023-11-06T02:08:30.606381+0000","last_undegraded":"2023-11-06T02:08:30.606381+0000","last_fullsized":"2023-11-06T02:08:30.606381+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:55:04.552404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.610172+0000","last_change":"2023-11-06T02:08:30.656051+0000","last_active":"2023-11-06T02:08:31.610172+0000","last_peered":"2023-11-06T02:08:31.610172+0000","last_clean":"2023-11-06T02:08:31.610172+0000","last_became_active":"2023-11-06T02:08:30.650054+0000","last_became_peered":"2023-11-06T02:08:30.650054+0000","last_unstale":"2023-11-06T02:08:31.610172+0000","last_undegraded":"2023-11-06T02:08:31.610172+0000","last_fullsized":"2023-11-06T02:08:31.610172+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:17:50.007398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.609417+0000","last_change":"2023-11-06T02:08:30.633028+0000","last_active":"2023-11-06T02:08:31.609417+0000","last_peered":"2023-11-06T02:08:31.609417+0000","last_clean":"2023-11-06T02:08:31.609417+0000","last_became_active":"2023-11-06T02:08:30.632255+0000","last_became_peered":"2023-11-06T02:08:30.632255+0000","last_unstale":"2023-11-06T02:08:31.609417+0000","last_undegraded":"2023-11-06T02:08:31.609417+0000","last_fullsized":"2023-11-06T02:08:31.609417+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:47:30.859032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"22'6","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.609754+0000","last_change":"2023-11-06T02:08:30.230431+0000","last_active":"2023-11-06T02:08:31.609754+0000","last_peered":"2023-11-06T02:08:31.609754+0000","last_clean":"2023-11-06T02:08:31.609754+0000","last_became_active":"2023-11-06T02:08:30.229931+0000","last_became_peered":"2023-11-06T02:08:30.229931+0000","last_unstale":"2023-11-06T02:08:31.609754+0000","last_undegraded":"2023-11-06T02:08:31.609754+0000","last_fullsized":"2023-11-06T02:08:31.609754+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:26:21.361289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.651402+0000","last_change":"2023-11-06T02:08:30.628444+0000","last_active":"2023-11-06T02:08:32.651402+0000","last_peered":"2023-11-06T02:08:32.651402+0000","last_clean":"2023-11-06T02:08:32.651402+0000","last_became_active":"2023-11-06T02:08:30.628087+0000","last_became_peered":"2023-11-06T02:08:30.628087+0000","last_unstale":"2023-11-06T02:08:32.651402+0000","last_undegraded":"2023-11-06T02:08:32.651402+0000","last_fullsized":"2023-11-06T02:08:32.651402+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:57:32.079075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"22'3","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.149892+0000","last_change":"2023-11-06T02:08:30.230554+0000","last_active":"2023-11-06T02:08:32.149892+0000","last_peered":"2023-11-06T02:08:32.149892+0000","last_clean":"2023-11-06T02:08:32.149892+0000","last_became_active":"2023-11-06T02:08:30.229374+0000","last_became_peered":"2023-11-06T02:08:30.229374+0000","last_unstale":"2023-11-06T02:08:32.149892+0000","last_undegraded":"2023-11-06T02:08:32.149892+0000","last_fullsized":"2023-11-06T02:08:32.149892+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:15:58.041467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.413338+0000","last_change":"2023-11-06T02:08:30.231726+0000","last_active":"2023-11-06T02:08:32.413338+0000","last_peered":"2023-11-06T02:08:32.413338+0000","last_clean":"2023-11-06T02:08:32.413338+0000","last_became_active":"2023-11-06T02:08:30.230802+0000","last_became_peered":"2023-11-06T02:08:30.230802+0000","last_unstale":"2023-11-06T02:08:32.413338+0000","last_undegraded":"2023-11-06T02:08:32.413338+0000","last_fullsized":"2023-11-06T02:08:32.413338+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:15:58.041467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.604488+0000","last_change":"2023-11-06T02:08:30.644420+0000","last_active":"2023-11-06T02:08:31.604488+0000","last_peered":"2023-11-06T02:08:31.604488+0000","last_clean":"2023-11-06T02:08:31.604488+0000","last_became_active":"2023-11-06T02:08:30.644097+0000","last_became_peered":"2023-11-06T02:08:30.644097+0000","last_unstale":"2023-11-06T02:08:31.604488+0000","last_undegraded":"2023-11-06T02:08:31.604488+0000","last_fullsized":"2023-11-06T02:08:31.604488+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:31:28.301664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.651911+0000","last_change":"2023-11-06T02:08:30.218801+0000","last_active":"2023-11-06T02:08:32.651911+0000","last_peered":"2023-11-06T02:08:32.651911+0000","last_clean":"2023-11-06T02:08:32.651911+0000","last_became_active":"2023-11-06T02:08:30.217237+0000","last_became_peered":"2023-11-06T02:08:30.217237+0000","last_unstale":"2023-11-06T02:08:32.651911+0000","last_undegraded":"2023-11-06T02:08:32.651911+0000","last_fullsized":"2023-11-06T02:08:32.651911+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:40:04.297054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.604444+0000","last_change":"2023-11-06T02:08:30.619501+0000","last_active":"2023-11-06T02:08:31.604444+0000","last_peered":"2023-11-06T02:08:31.604444+0000","last_clean":"2023-11-06T02:08:31.604444+0000","last_became_active":"2023-11-06T02:08:30.619180+0000","last_became_peered":"2023-11-06T02:08:30.619180+0000","last_unstale":"2023-11-06T02:08:31.604444+0000","last_undegraded":"2023-11-06T02:08:31.604444+0000","last_fullsized":"2023-11-06T02:08:31.604444+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:13:37.616527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.611492+0000","last_change":"2023-11-06T02:08:30.655759+0000","last_active":"2023-11-06T02:08:31.611492+0000","last_peered":"2023-11-06T02:08:31.611492+0000","last_clean":"2023-11-06T02:08:31.611492+0000","last_became_active":"2023-11-06T02:08:30.655285+0000","last_became_peered":"2023-11-06T02:08:30.655285+0000","last_unstale":"2023-11-06T02:08:31.611492+0000","last_undegraded":"2023-11-06T02:08:31.611492+0000","last_fullsized":"2023-11-06T02:08:31.611492+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:20:00.351057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.604479+0000","last_change":"2023-11-06T02:08:30.222521+0000","last_active":"2023-11-06T02:08:31.604479+0000","last_peered":"2023-11-06T02:08:31.604479+0000","last_clean":"2023-11-06T02:08:31.604479+0000","last_became_active":"2023-11-06T02:08:30.221899+0000","last_became_peered":"2023-11-06T02:08:30.221899+0000","last_unstale":"2023-11-06T02:08:31.604479+0000","last_undegraded":"2023-11-06T02:08:31.604479+0000","last_fullsized":"2023-11-06T02:08:31.604479+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:43:56.017919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.652973+0000","last_change":"2023-11-06T02:08:30.644462+0000","last_active":"2023-11-06T02:08:32.652973+0000","last_peered":"2023-11-06T02:08:32.652973+0000","last_clean":"2023-11-06T02:08:32.652973+0000","last_became_active":"2023-11-06T02:08:30.644160+0000","last_became_peered":"2023-11-06T02:08:30.644160+0000","last_unstale":"2023-11-06T02:08:32.652973+0000","last_undegraded":"2023-11-06T02:08:32.652973+0000","last_fullsized":"2023-11-06T02:08:32.652973+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:43:23.215718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.652108+0000","last_change":"2023-11-06T02:08:30.214519+0000","last_active":"2023-11-06T02:08:32.652108+0000","last_peered":"2023-11-06T02:08:32.652108+0000","last_clean":"2023-11-06T02:08:32.652108+0000","last_became_active":"2023-11-06T02:08:30.214225+0000","last_became_peered":"2023-11-06T02:08:30.214225+0000","last_unstale":"2023-11-06T02:08:32.652108+0000","last_undegraded":"2023-11-06T02:08:32.652108+0000","last_fullsized":"2023-11-06T02:08:32.652108+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:30:21.274364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.605196+0000","last_change":"2023-11-06T02:08:30.617971+0000","last_active":"2023-11-06T02:08:31.605196+0000","last_peered":"2023-11-06T02:08:31.605196+0000","last_clean":"2023-11-06T02:08:31.605196+0000","last_became_active":"2023-11-06T02:08:30.617692+0000","last_became_peered":"2023-11-06T02:08:30.617692+0000","last_unstale":"2023-11-06T02:08:31.605196+0000","last_undegraded":"2023-11-06T02:08:31.605196+0000","last_fullsized":"2023-11-06T02:08:31.605196+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:23:59.639528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"22'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.609142+0000","last_change":"2023-11-06T02:08:30.224834+0000","last_active":"2023-11-06T02:08:31.609142+0000","last_peered":"2023-11-06T02:08:31.609142+0000","last_clean":"2023-11-06T02:08:31.609142+0000","last_became_active":"2023-11-06T02:08:30.224244+0000","last_became_peered":"2023-11-06T02:08:30.224244+0000","last_unstale":"2023-11-06T02:08:31.609142+0000","last_undegraded":"2023-11-06T02:08:31.609142+0000","last_fullsized":"2023-11-06T02:08:31.609142+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:40:04.297054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.654928+0000","last_change":"2023-11-06T02:08:30.619430+0000","last_active":"2023-11-06T02:08:32.654928+0000","last_peered":"2023-11-06T02:08:32.654928+0000","last_clean":"2023-11-06T02:08:32.654928+0000","last_became_active":"2023-11-06T02:08:30.619155+0000","last_became_peered":"2023-11-06T02:08:30.619155+0000","last_unstale":"2023-11-06T02:08:32.654928+0000","last_undegraded":"2023-11-06T02:08:32.654928+0000","last_fullsized":"2023-11-06T02:08:32.654928+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:30:22.278201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"22'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.652274+0000","last_change":"2023-11-06T02:08:30.227830+0000","last_active":"2023-11-06T02:08:32.652274+0000","last_peered":"2023-11-06T02:08:32.652274+0000","last_clean":"2023-11-06T02:08:32.652274+0000","last_became_active":"2023-11-06T02:08:30.227476+0000","last_became_peered":"2023-11-06T02:08:30.227476+0000","last_unstale":"2023-11-06T02:08:32.652274+0000","last_undegraded":"2023-11-06T02:08:32.652274+0000","last_fullsized":"2023-11-06T02:08:32.652274+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:30:21.274364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.609173+0000","last_change":"2023-11-06T02:08:30.651612+0000","last_active":"2023-11-06T02:08:31.609173+0000","last_peered":"2023-11-06T02:08:31.609173+0000","last_clean":"2023-11-06T02:08:31.609173+0000","last_became_active":"2023-11-06T02:08:30.650925+0000","last_became_peered":"2023-11-06T02:08:30.650925+0000","last_unstale":"2023-11-06T02:08:31.609173+0000","last_undegraded":"2023-11-06T02:08:31.609173+0000","last_fullsized":"2023-11-06T02:08:31.609173+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:31:28.301664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"22'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.413042+0000","last_change":"2023-11-06T02:08:30.217273+0000","last_active":"2023-11-06T02:08:32.413042+0000","last_peered":"2023-11-06T02:08:32.413042+0000","last_clean":"2023-11-06T02:08:32.413042+0000","last_became_active":"2023-11-06T02:08:30.216532+0000","last_became_peered":"2023-11-06T02:08:30.216532+0000","last_unstale":"2023-11-06T02:08:32.413042+0000","last_undegraded":"2023-11-06T02:08:32.413042+0000","last_fullsized":"2023-11-06T02:08:32.413042+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:33:38.355715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.608910+0000","last_change":"2023-11-06T02:08:30.651587+0000","last_active":"2023-11-06T02:08:31.608910+0000","last_peered":"2023-11-06T02:08:31.608910+0000","last_clean":"2023-11-06T02:08:31.608910+0000","last_became_active":"2023-11-06T02:08:30.650762+0000","last_became_peered":"2023-11-06T02:08:30.650762+0000","last_unstale":"2023-11-06T02:08:31.608910+0000","last_undegraded":"2023-11-06T02:08:31.608910+0000","last_fullsized":"2023-11-06T02:08:31.608910+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:52.535882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"22'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.413347+0000","last_change":"2023-11-06T02:08:30.220284+0000","last_active":"2023-11-06T02:08:32.413347+0000","last_peered":"2023-11-06T02:08:32.413347+0000","last_clean":"2023-11-06T02:08:32.413347+0000","last_became_active":"2023-11-06T02:08:30.219571+0000","last_became_peered":"2023-11-06T02:08:30.219571+0000","last_unstale":"2023-11-06T02:08:32.413347+0000","last_undegraded":"2023-11-06T02:08:32.413347+0000","last_fullsized":"2023-11-06T02:08:32.413347+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:24:09.801921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.604654+0000","last_change":"2023-11-06T02:08:30.621580+0000","last_active":"2023-11-06T02:08:31.604654+0000","last_peered":"2023-11-06T02:08:31.604654+0000","last_clean":"2023-11-06T02:08:31.604654+0000","last_became_active":"2023-11-06T02:08:30.618164+0000","last_became_peered":"2023-11-06T02:08:30.618164+0000","last_unstale":"2023-11-06T02:08:31.604654+0000","last_undegraded":"2023-11-06T02:08:31.604654+0000","last_fullsized":"2023-11-06T02:08:31.604654+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:57:32.079075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"22'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.604995+0000","last_change":"2023-11-06T02:08:30.208980+0000","last_active":"2023-11-06T02:08:31.604995+0000","last_peered":"2023-11-06T02:08:31.604995+0000","last_clean":"2023-11-06T02:08:31.604995+0000","last_became_active":"2023-11-06T02:08:30.208339+0000","last_became_peered":"2023-11-06T02:08:30.208339+0000","last_unstale":"2023-11-06T02:08:31.604995+0000","last_undegraded":"2023-11-06T02:08:31.604995+0000","last_fullsized":"2023-11-06T02:08:31.604995+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:31:31.882709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.608588+0000","last_change":"2023-11-06T02:08:30.634399+0000","last_active":"2023-11-06T02:08:31.608588+0000","last_peered":"2023-11-06T02:08:31.608588+0000","last_clean":"2023-11-06T02:08:31.608588+0000","last_became_active":"2023-11-06T02:08:30.633674+0000","last_became_peered":"2023-11-06T02:08:30.633674+0000","last_unstale":"2023-11-06T02:08:31.608588+0000","last_undegraded":"2023-11-06T02:08:31.608588+0000","last_fullsized":"2023-11-06T02:08:31.608588+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:14:45.248292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1f","version":"22'4","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.651772+0000","last_change":"2023-11-06T02:08:30.220947+0000","last_active":"2023-11-06T02:08:32.651772+0000","last_peered":"2023-11-06T02:08:32.651772+0000","last_clean":"2023-11-06T02:08:32.651772+0000","last_became_active":"2023-11-06T02:08:30.220379+0000","last_became_peered":"2023-11-06T02:08:30.220379+0000","last_unstale":"2023-11-06T02:08:32.651772+0000","last_undegraded":"2023-11-06T02:08:32.651772+0000","last_fullsized":"2023-11-06T02:08:32.651772+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:43:56.017919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27796,"kb_used_data":844,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344044,"statfs":{"total":96636764160,"available":96608301056,"internally_reserved":0,"allocated":864256,"data_stored":681747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73999999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.026}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97099999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83699999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1679999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3069999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.45}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27188,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":241664,"data_stored":89599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81200000000000006}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.087}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.268}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.474}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.347}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3180000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":237568,"data_stored":83598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.202}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1459999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3149999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":262144,"data_stored":91219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70399999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.026}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84899999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2210000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.157}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1919999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31288,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340552,"statfs":{"total":96636764160,"available":96604725248,"internally_reserved":0,"allocated":245760,"data_stored":89689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70399999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65800000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2929999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1899999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31284,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340556,"statfs":{"total":96636764160,"available":96604729344,"internally_reserved":0,"allocated":241664,"data_stored":89599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78300000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73999999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.625}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31272,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340568,"statfs":{"total":96636764160,"available":96604741632,"internally_reserved":0,"allocated":229376,"data_stored":82946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81999999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5349999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1459999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73599999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4690000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31852,"kb_used_data":804,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339988,"statfs":{"total":96636764160,"available":96604147712,"internally_reserved":0,"allocated":823296,"data_stored":672877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83299999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90200000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.052}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96999999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2230000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T02:08:38.775 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T02:08:38.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:38.914+0000 7fd9daefb640 1 Processor -- start 2023-11-06T02:08:38.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:38.914+0000 7fd9daefb640 1 -- start start 2023-11-06T02:08:38.912 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:38.914+0000 7fd9daefb640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd9d4159b40 0x7fd9d40ca980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:38.912 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:38.914+0000 7fd9daefb640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fd9d40cdd50 0x7fd9d40cc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:38.912 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:38.914+0000 7fd9daefb640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9d40723a0 con 0x7fd9d40cb010 2023-11-06T02:08:38.912 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:38.914+0000 7fd9daefb640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9d40715e0 con 0x7fd9d40cdd50 2023-11-06T02:08:38.913 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:38.914+0000 7fd9daefb640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d40718c0 con 0x7fd9d4159b40 2023-11-06T02:08:38.913 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:38.914+0000 7fd9d8c97640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fd9d40cb010 0x7fd9d40cb3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:53912/0 (socket says 172.21.15.49:53912) 2023-11-06T02:08:38.913 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:38.914+0000 7fd9d8c97640 1 -- 172.21.15.49:0/485095005 learned_addr learned my addr 172.21.15.49:0/485095005 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:08:38.913 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:38.914+0000 7fd9d3fff640 1 --2- 172.21.15.49:0/485095005 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd9d4159b40 0x7fd9d40ca980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:38.913 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:38.914+0000 7fd9d37fe640 1 -- 172.21.15.49:0/485095005 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1832459848 0 0) 0x7fd9d40723a0 con 0x7fd9d40cb010 2023-11-06T02:08:38.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:38.914+0000 7fd9d37fe640 1 -- 172.21.15.49:0/485095005 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9bc003530 con 0x7fd9d40cb010 2023-11-06T02:08:38.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:38.914+0000 7fd9d9498640 1 --2- 172.21.15.49:0/485095005 >> v2:172.21.15.150:3300/0 conn(0x7fd9d40cdd50 0x7fd9d40cc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:38.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:38.914+0000 7fd9d37fe640 1 -- 172.21.15.49:0/485095005 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3182532628 0 0) 0x7fd9bc003530 con 0x7fd9d40cb010 2023-11-06T02:08:38.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:38.914+0000 7fd9d37fe640 1 -- 172.21.15.49:0/485095005 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd9d4159b40 msgr2=0x7fd9d40ca980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:38.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:38.914+0000 7fd9d37fe640 1 --2- 172.21.15.49:0/485095005 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd9d4159b40 0x7fd9d40ca980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:38.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:38.914+0000 7fd9d37fe640 1 -- 172.21.15.49:0/485095005 >> v2:172.21.15.150:3300/0 conn(0x7fd9d40cdd50 msgr2=0x7fd9d40cc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:38.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:38.914+0000 7fd9d37fe640 1 --2- 172.21.15.49:0/485095005 >> v2:172.21.15.150:3300/0 conn(0x7fd9d40cdd50 0x7fd9d40cc1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:38.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:38.914+0000 7fd9d37fe640 1 -- 172.21.15.49:0/485095005 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9d4071ba0 con 0x7fd9d40cb010 2023-11-06T02:08:38.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:38.914+0000 7fd9d37fe640 1 -- 172.21.15.49:0/485095005 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7fd9c4001800 con 0x7fd9d40cb010 2023-11-06T02:08:38.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:38.914+0000 7fd9d37fe640 1 -- 172.21.15.49:0/485095005 >> v1:172.21.15.49:6789/0 conn(0x7fd9d40cb010 legacy=0x7fd9d40cb3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:38.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:38.914+0000 7fd9d37fe640 1 --2- 172.21.15.49:0/485095005 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9bc003980 0x7fd9bc003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:38.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:38.914+0000 7fd9d37fe640 1 --2- 172.21.15.49:0/485095005 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9bc0047d0 0x7fd9bc004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:38.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:38.914+0000 7fd9d37fe640 1 --2- 172.21.15.49:0/485095005 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd9d4159b40 0x7fd9bc005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:38.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:38.914+0000 7fd9d37fe640 1 -- 172.21.15.49:0/485095005 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d40715e0 con 0x7fd9bc003980 2023-11-06T02:08:38.917 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:38.914+0000 7fd9d37fe640 1 -- 172.21.15.49:0/485095005 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d40718c0 con 0x7fd9bc0047d0 2023-11-06T02:08:38.917 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:38.914+0000 7fd9d37fe640 1 -- 172.21.15.49:0/485095005 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9bc00b990 con 0x7fd9d4159b40 2023-11-06T02:08:38.917 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:38.914+0000 7fd9d8c97640 1 --2- 172.21.15.49:0/485095005 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9bc003980 0x7fd9bc003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:38.917 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:38.914+0000 7fd9d3fff640 1 --2- 172.21.15.49:0/485095005 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9bc0047d0 0x7fd9bc004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:38.918 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:38.914+0000 7fd9d9498640 1 --2- 172.21.15.49:0/485095005 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd9d4159b40 0x7fd9bc005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:38.918 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:38.914+0000 7fd9d8c97640 1 -- 172.21.15.49:0/485095005 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd9d4159b40 msgr2=0x7fd9bc005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:38.918 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:38.914+0000 7fd9d8c97640 1 --2- 172.21.15.49:0/485095005 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd9d4159b40 0x7fd9bc005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:38.918 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:38.914+0000 7fd9d8c97640 1 -- 172.21.15.49:0/485095005 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9bc0047d0 msgr2=0x7fd9bc004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:38.918 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:38.914+0000 7fd9d8c97640 1 --2- 172.21.15.49:0/485095005 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9bc0047d0 0x7fd9bc004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:38.919 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:38.914+0000 7fd9d8c97640 1 -- 172.21.15.49:0/485095005 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9bc003530 con 0x7fd9bc003980 2023-11-06T02:08:38.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:38.918+0000 7fd9d8c97640 1 --2- 172.21.15.49:0/485095005 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9bc003980 0x7fd9bc003da0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd9c400c240 tx=0x7fd9c401cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7748b07e2d5582ab server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:38.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:38.918+0000 7fd9d37fe640 1 -- 172.21.15.49:0/485095005 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c40070d0 con 0x7fd9bc003980 2023-11-06T02:08:38.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:38.918+0000 7fd9d37fe640 1 -- 172.21.15.49:0/485095005 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9c4007230 con 0x7fd9bc003980 2023-11-06T02:08:38.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:38.918+0000 7fd9d37fe640 1 -- 172.21.15.49:0/485095005 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c40075e0 con 0x7fd9bc003980 2023-11-06T02:08:38.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:38.918+0000 7fd9daefb640 1 -- 172.21.15.49:0/485095005 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9bc003980 msgr2=0x7fd9bc003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:38.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:38.918+0000 7fd9daefb640 1 --2- 172.21.15.49:0/485095005 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9bc003980 0x7fd9bc003da0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd9c400c240 tx=0x7fd9c401cc30 comp rx=0 tx=0).stop 2023-11-06T02:08:38.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:38.918+0000 7fd9d9498640 1 -- 172.21.15.49:0/485095005 reap_dead start 2023-11-06T02:08:38.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:38.918+0000 7fd9daefb640 1 -- 172.21.15.49:0/485095005 shutdown_connections 2023-11-06T02:08:38.924 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:38.918+0000 7fd9daefb640 1 -- 172.21.15.49:0/485095005 >> 172.21.15.49:0/485095005 conn(0x7fd9d406c0a0 msgr2=0x7fd9d406c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:38.924 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:38.918+0000 7fd9daefb640 1 -- 172.21.15.49:0/485095005 shutdown_connections 2023-11-06T02:08:38.924 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:38.918+0000 7fd9daefb640 1 -- 172.21.15.49:0/485095005 wait complete. 2023-11-06T02:08:38.924 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:38.918+0000 7fd9daefb640 1 Processor -- start 2023-11-06T02:08:38.924 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:38.918+0000 7fd9daefb640 1 -- start start 2023-11-06T02:08:38.925 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:38.922+0000 7fd9daefb640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9bc003980 0x7fd9d41f2900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:38.925 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:38.922+0000 7fd9daefb640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd9d415ee40 0x7fd9d415f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:38.925 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:38.922+0000 7fd9daefb640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9d41f6420 0x7fd9d41f68a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:38.925 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:38.922+0000 7fd9daefb640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d415a310 con 0x7fd9d41f6420 2023-11-06T02:08:38.925 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:38.922+0000 7fd9daefb640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d41f6de0 con 0x7fd9bc003980 2023-11-06T02:08:38.925 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:38.922+0000 7fd9daefb640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d415f8e0 con 0x7fd9d415ee40 2023-11-06T02:08:38.926 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:38.922+0000 7fd9d9498640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9d41f6420 0x7fd9d41f68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:38.926 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:38.922+0000 7fd9d9498640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9d41f6420 0x7fd9d41f68a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51954/0 (socket says 172.21.15.49:51954) 2023-11-06T02:08:38.926 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:38.922+0000 7fd9d9498640 1 -- 172.21.15.49:0/2226974078 learned_addr learned my addr 172.21.15.49:0/2226974078 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:38.926 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:38.922+0000 7fd9d8c97640 1 --2- 172.21.15.49:0/2226974078 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9bc003980 0x7fd9d41f2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:38.926 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:38.922+0000 7fd9d3fff640 1 --2- 172.21.15.49:0/2226974078 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd9d415ee40 0x7fd9d415f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:38.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:38.922+0000 7fd9d9498640 1 -- 172.21.15.49:0/2226974078 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd9d415ee40 msgr2=0x7fd9d415f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:38.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:38.922+0000 7fd9d9498640 1 --2- 172.21.15.49:0/2226974078 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd9d415ee40 0x7fd9d415f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:38.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:38.922+0000 7fd9d9498640 1 -- 172.21.15.49:0/2226974078 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9bc003980 msgr2=0x7fd9d41f2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:38.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:38.922+0000 7fd9d9498640 1 --2- 172.21.15.49:0/2226974078 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9bc003980 0x7fd9d41f2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:38.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:38.922+0000 7fd9d9498640 1 -- 172.21.15.49:0/2226974078 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9d415fbc0 con 0x7fd9d41f6420 2023-11-06T02:08:38.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:38.922+0000 7fd9d9498640 1 --2- 172.21.15.49:0/2226974078 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9d41f6420 0x7fd9d41f68a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd9d406e190 tx=0x7fd9cc0115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:38.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:38.922+0000 7fd9d17fa640 1 -- 172.21.15.49:0/2226974078 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9cc00e070 con 0x7fd9d41f6420 2023-11-06T02:08:38.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:38.922+0000 7fd9d17fa640 1 -- 172.21.15.49:0/2226974078 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9cc011ed0 con 0x7fd9d41f6420 2023-11-06T02:08:38.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:38.922+0000 7fd9d17fa640 1 -- 172.21.15.49:0/2226974078 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9cc0039a0 con 0x7fd9d41f6420 2023-11-06T02:08:38.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:38.922+0000 7fd9daefb640 1 -- 172.21.15.49:0/2226974078 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9d415a520 con 0x7fd9d41f6420 2023-11-06T02:08:38.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:38.922+0000 7fd9daefb640 1 -- 172.21.15.49:0/2226974078 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9d4154ff0 con 0x7fd9d41f6420 2023-11-06T02:08:38.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:38.922+0000 7fd9daefb640 1 -- 172.21.15.49:0/2226974078 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9d4074040 con 0x7fd9d41f6420 2023-11-06T02:08:38.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:38.922+0000 7fd9d17fa640 1 -- 172.21.15.49:0/2226974078 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fd9cc003b20 con 0x7fd9d41f6420 2023-11-06T02:08:38.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:38.922+0000 7fd9d17fa640 1 --2- 172.21.15.49:0/2226974078 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd9c0065a90 0x7fd9c0067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:38.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:38.922+0000 7fd9d17fa640 1 -- 172.21.15.49:0/2226974078 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd9cc083890 con 0x7fd9d41f6420 2023-11-06T02:08:38.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:38.922+0000 7fd9d8c97640 1 --2- 172.21.15.49:0/2226974078 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd9c0065a90 0x7fd9c0067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:38.930 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:38.926+0000 7fd9d8c97640 1 --2- 172.21.15.49:0/2226974078 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd9c0065a90 0x7fd9c0067f50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd9c40aa000 tx=0x7fd9c40b0040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:38.930 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:38.926+0000 7fd9d17fa640 1 -- 172.21.15.49:0/2226974078 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fd9cc0510f0 con 0x7fd9d41f6420 2023-11-06T02:08:39.076 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.078+0000 7fd9daefb640 1 -- 172.21.15.49:0/2226974078 --> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd9d40ccf20 con 0x7fd9c0065a90 2023-11-06T02:08:39.097 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.102+0000 7fd9d17fa640 1 -- 172.21.15.49:0/2226974078 <== mgr.4100 v2:172.21.15.150:6832/38104 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286677 (secure 0 0 0) 0x7fd9c40015c0 con 0x7fd9c0065a90 2023-11-06T02:08:39.100 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:08:39.102 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2023-11-06T02:08:39.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.106+0000 7fd9daefb640 1 -- 172.21.15.49:0/2226974078 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd9c0065a90 msgr2=0x7fd9c0067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:39.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.106+0000 7fd9daefb640 1 --2- 172.21.15.49:0/2226974078 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd9c0065a90 0x7fd9c0067f50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd9c40aa000 tx=0x7fd9c40b0040 comp rx=0 tx=0).stop 2023-11-06T02:08:39.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.106+0000 7fd9daefb640 1 -- 172.21.15.49:0/2226974078 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9d41f6420 msgr2=0x7fd9d41f68a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:39.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.106+0000 7fd9daefb640 1 --2- 172.21.15.49:0/2226974078 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9d41f6420 0x7fd9d41f68a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd9d406e190 tx=0x7fd9cc0115a0 comp rx=0 tx=0).stop 2023-11-06T02:08:39.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.106+0000 7fd9daefb640 1 -- 172.21.15.49:0/2226974078 shutdown_connections 2023-11-06T02:08:39.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.106+0000 7fd9daefb640 1 --2- 172.21.15.49:0/2226974078 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd9c0065a90 0x7fd9c0067f50 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:39.105 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.106+0000 7fd9daefb640 1 --2- 172.21.15.49:0/2226974078 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9d41f6420 0x7fd9d41f68a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:39.105 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.106+0000 7fd9daefb640 1 --2- 172.21.15.49:0/2226974078 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd9d415ee40 0x7fd9d415f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:39.105 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.106+0000 7fd9daefb640 1 --2- 172.21.15.49:0/2226974078 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9bc003980 0x7fd9d41f2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:39.105 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.106+0000 7fd9daefb640 1 -- 172.21.15.49:0/2226974078 >> 172.21.15.49:0/2226974078 conn(0x7fd9d406c0a0 msgr2=0x7fd9d406ffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:39.105 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.106+0000 7fd9daefb640 1 -- 172.21.15.49:0/2226974078 shutdown_connections 2023-11-06T02:08:39.105 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.106+0000 7fd9daefb640 1 -- 172.21.15.49:0/2226974078 wait complete. 2023-11-06T02:08:39.119 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2023-11-06T02:08:37.176545+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":175,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235072,"kb_used_data":3072,"kb_used_omap":0,"kb_used_meta":231936,"kb_avail":754739648,"statfs":{"total":773094113280,"available":772853399552,"internally_reserved":0,"allocated":3145728,"data_stored":1881274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":237502464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":61,"apply_latency_ms":61,"commit_latency_ns":61000000,"apply_latency_ns":61000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.985690"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.655111+0000","last_change":"2023-11-06T02:08:30.608781+0000","last_active":"2023-11-06T02:08:32.655111+0000","last_peered":"2023-11-06T02:08:32.655111+0000","last_clean":"2023-11-06T02:08:32.655111+0000","last_became_active":"2023-11-06T02:08:30.606710+0000","last_became_peered":"2023-11-06T02:08:30.606710+0000","last_unstale":"2023-11-06T02:08:32.655111+0000","last_undegraded":"2023-11-06T02:08:32.655111+0000","last_fullsized":"2023-11-06T02:08:32.655111+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:43:57.021755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.413091+0000","last_change":"2023-11-06T02:08:30.646508+0000","last_active":"2023-11-06T02:08:32.413091+0000","last_peered":"2023-11-06T02:08:32.413091+0000","last_clean":"2023-11-06T02:08:32.413091+0000","last_became_active":"2023-11-06T02:08:30.645615+0000","last_became_peered":"2023-11-06T02:08:30.645615+0000","last_unstale":"2023-11-06T02:08:32.413091+0000","last_undegraded":"2023-11-06T02:08:32.413091+0000","last_fullsized":"2023-11-06T02:08:32.413091+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:49:05.460965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.610298+0000","last_change":"2023-11-06T02:08:30.658350+0000","last_active":"2023-11-06T02:08:31.610298+0000","last_peered":"2023-11-06T02:08:31.610298+0000","last_clean":"2023-11-06T02:08:31.610298+0000","last_became_active":"2023-11-06T02:08:30.651992+0000","last_became_peered":"2023-11-06T02:08:30.651992+0000","last_unstale":"2023-11-06T02:08:31.610298+0000","last_undegraded":"2023-11-06T02:08:31.610298+0000","last_fullsized":"2023-11-06T02:08:31.610298+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:09:56.455105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.610511+0000","last_change":"2023-11-06T02:08:30.627209+0000","last_active":"2023-11-06T02:08:31.610511+0000","last_peered":"2023-11-06T02:08:31.610511+0000","last_clean":"2023-11-06T02:08:31.610511+0000","last_became_active":"2023-11-06T02:08:30.626345+0000","last_became_peered":"2023-11-06T02:08:30.626345+0000","last_unstale":"2023-11-06T02:08:31.610511+0000","last_undegraded":"2023-11-06T02:08:31.610511+0000","last_fullsized":"2023-11-06T02:08:31.610511+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:46:38.867945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.652203+0000","last_change":"2023-11-06T02:08:30.619667+0000","last_active":"2023-11-06T02:08:32.652203+0000","last_peered":"2023-11-06T02:08:32.652203+0000","last_clean":"2023-11-06T02:08:32.652203+0000","last_became_active":"2023-11-06T02:08:30.619328+0000","last_became_peered":"2023-11-06T02:08:30.619328+0000","last_unstale":"2023-11-06T02:08:32.652203+0000","last_undegraded":"2023-11-06T02:08:32.652203+0000","last_fullsized":"2023-11-06T02:08:32.652203+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:26:58.090934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.608847+0000","last_change":"2023-11-06T02:08:30.657202+0000","last_active":"2023-11-06T02:08:31.608847+0000","last_peered":"2023-11-06T02:08:31.608847+0000","last_clean":"2023-11-06T02:08:31.608847+0000","last_became_active":"2023-11-06T02:08:30.652132+0000","last_became_peered":"2023-11-06T02:08:30.652132+0000","last_unstale":"2023-11-06T02:08:31.608847+0000","last_undegraded":"2023-11-06T02:08:31.608847+0000","last_fullsized":"2023-11-06T02:08:31.608847+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:35:19.038827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.149393+0000","last_change":"2023-11-06T02:08:30.645694+0000","last_active":"2023-11-06T02:08:32.149393+0000","last_peered":"2023-11-06T02:08:32.149393+0000","last_clean":"2023-11-06T02:08:32.149393+0000","last_became_active":"2023-11-06T02:08:30.645028+0000","last_became_peered":"2023-11-06T02:08:30.645028+0000","last_unstale":"2023-11-06T02:08:32.149393+0000","last_undegraded":"2023-11-06T02:08:32.149393+0000","last_fullsized":"2023-11-06T02:08:32.149393+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:26:58.090934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.412662+0000","last_change":"2023-11-06T02:08:30.652585+0000","last_active":"2023-11-06T02:08:32.412662+0000","last_peered":"2023-11-06T02:08:32.412662+0000","last_clean":"2023-11-06T02:08:32.412662+0000","last_became_active":"2023-11-06T02:08:30.652103+0000","last_became_peered":"2023-11-06T02:08:30.652103+0000","last_unstale":"2023-11-06T02:08:32.412662+0000","last_undegraded":"2023-11-06T02:08:32.412662+0000","last_fullsized":"2023-11-06T02:08:32.412662+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:57:32.079075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.651445+0000","last_change":"2023-11-06T02:08:30.618708+0000","last_active":"2023-11-06T02:08:32.651445+0000","last_peered":"2023-11-06T02:08:32.651445+0000","last_clean":"2023-11-06T02:08:32.651445+0000","last_became_active":"2023-11-06T02:08:30.618232+0000","last_became_peered":"2023-11-06T02:08:30.618232+0000","last_unstale":"2023-11-06T02:08:32.651445+0000","last_undegraded":"2023-11-06T02:08:32.651445+0000","last_fullsized":"2023-11-06T02:08:32.651445+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:31:28.301664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.413222+0000","last_change":"2023-11-06T02:08:30.640907+0000","last_active":"2023-11-06T02:08:32.413222+0000","last_peered":"2023-11-06T02:08:32.413222+0000","last_clean":"2023-11-06T02:08:32.413222+0000","last_became_active":"2023-11-06T02:08:30.626276+0000","last_became_peered":"2023-11-06T02:08:30.626276+0000","last_unstale":"2023-11-06T02:08:32.413222+0000","last_undegraded":"2023-11-06T02:08:32.413222+0000","last_fullsized":"2023-11-06T02:08:32.413222+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:56:51.565719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.610554+0000","last_change":"2023-11-06T02:08:30.658929+0000","last_active":"2023-11-06T02:08:31.610554+0000","last_peered":"2023-11-06T02:08:31.610554+0000","last_clean":"2023-11-06T02:08:31.610554+0000","last_became_active":"2023-11-06T02:08:30.657695+0000","last_became_peered":"2023-11-06T02:08:30.657695+0000","last_unstale":"2023-11-06T02:08:31.610554+0000","last_undegraded":"2023-11-06T02:08:31.610554+0000","last_fullsized":"2023-11-06T02:08:31.610554+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:57:32.079075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.654812+0000","last_change":"2023-11-06T02:08:30.660659+0000","last_active":"2023-11-06T02:08:32.654812+0000","last_peered":"2023-11-06T02:08:32.654812+0000","last_clean":"2023-11-06T02:08:32.654812+0000","last_became_active":"2023-11-06T02:08:30.660402+0000","last_became_peered":"2023-11-06T02:08:30.660402+0000","last_unstale":"2023-11-06T02:08:32.654812+0000","last_undegraded":"2023-11-06T02:08:32.654812+0000","last_fullsized":"2023-11-06T02:08:32.654812+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:26:50.302398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.654670+0000","last_change":"2023-11-06T02:08:30.608913+0000","last_active":"2023-11-06T02:08:32.654670+0000","last_peered":"2023-11-06T02:08:32.654670+0000","last_clean":"2023-11-06T02:08:32.654670+0000","last_became_active":"2023-11-06T02:08:30.606808+0000","last_became_peered":"2023-11-06T02:08:30.606808+0000","last_unstale":"2023-11-06T02:08:32.654670+0000","last_undegraded":"2023-11-06T02:08:32.654670+0000","last_fullsized":"2023-11-06T02:08:32.654670+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:24:10.805757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.412935+0000","last_change":"2023-11-06T02:08:30.653750+0000","last_active":"2023-11-06T02:08:32.412935+0000","last_peered":"2023-11-06T02:08:32.412935+0000","last_clean":"2023-11-06T02:08:32.412935+0000","last_became_active":"2023-11-06T02:08:30.653252+0000","last_became_peered":"2023-11-06T02:08:30.653252+0000","last_unstale":"2023-11-06T02:08:32.412935+0000","last_undegraded":"2023-11-06T02:08:32.412935+0000","last_fullsized":"2023-11-06T02:08:32.412935+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:26:58.090934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.149589+0000","last_change":"2023-11-06T02:08:30.644591+0000","last_active":"2023-11-06T02:08:32.149589+0000","last_peered":"2023-11-06T02:08:32.149589+0000","last_clean":"2023-11-06T02:08:32.149589+0000","last_became_active":"2023-11-06T02:08:30.643848+0000","last_became_peered":"2023-11-06T02:08:30.643848+0000","last_unstale":"2023-11-06T02:08:32.149589+0000","last_undegraded":"2023-11-06T02:08:32.149589+0000","last_fullsized":"2023-11-06T02:08:32.149589+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:49:05.460965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.608423+0000","last_change":"2023-11-06T02:08:30.634202+0000","last_active":"2023-11-06T02:08:31.608423+0000","last_peered":"2023-11-06T02:08:31.608423+0000","last_clean":"2023-11-06T02:08:31.608423+0000","last_became_active":"2023-11-06T02:08:30.633451+0000","last_became_peered":"2023-11-06T02:08:30.633451+0000","last_unstale":"2023-11-06T02:08:31.608423+0000","last_undegraded":"2023-11-06T02:08:31.608423+0000","last_fullsized":"2023-11-06T02:08:31.608423+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:13:14.793314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.651622+0000","last_change":"2023-11-06T02:08:30.618515+0000","last_active":"2023-11-06T02:08:32.651622+0000","last_peered":"2023-11-06T02:08:32.651622+0000","last_clean":"2023-11-06T02:08:32.651622+0000","last_became_active":"2023-11-06T02:08:30.617836+0000","last_became_peered":"2023-11-06T02:08:30.617836+0000","last_unstale":"2023-11-06T02:08:32.651622+0000","last_undegraded":"2023-11-06T02:08:32.651622+0000","last_fullsized":"2023-11-06T02:08:32.651622+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:17:50.007398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.609393+0000","last_change":"2023-11-06T02:08:30.659348+0000","last_active":"2023-11-06T02:08:31.609393+0000","last_peered":"2023-11-06T02:08:31.609393+0000","last_clean":"2023-11-06T02:08:31.609393+0000","last_became_active":"2023-11-06T02:08:30.652231+0000","last_became_peered":"2023-11-06T02:08:30.652231+0000","last_unstale":"2023-11-06T02:08:31.609393+0000","last_undegraded":"2023-11-06T02:08:31.609393+0000","last_fullsized":"2023-11-06T02:08:31.609393+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:13:26.942266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.655090+0000","last_change":"2023-11-06T02:08:30.627395+0000","last_active":"2023-11-06T02:08:32.655090+0000","last_peered":"2023-11-06T02:08:32.655090+0000","last_clean":"2023-11-06T02:08:32.655090+0000","last_became_active":"2023-11-06T02:08:30.627048+0000","last_became_peered":"2023-11-06T02:08:30.627048+0000","last_unstale":"2023-11-06T02:08:32.655090+0000","last_undegraded":"2023-11-06T02:08:32.655090+0000","last_fullsized":"2023-11-06T02:08:32.655090+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:15:59.045303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.609522+0000","last_change":"2023-11-06T02:08:30.658560+0000","last_active":"2023-11-06T02:08:31.609522+0000","last_peered":"2023-11-06T02:08:31.609522+0000","last_clean":"2023-11-06T02:08:31.609522+0000","last_became_active":"2023-11-06T02:08:30.657560+0000","last_became_peered":"2023-11-06T02:08:30.657560+0000","last_unstale":"2023-11-06T02:08:31.609522+0000","last_undegraded":"2023-11-06T02:08:31.609522+0000","last_fullsized":"2023-11-06T02:08:31.609522+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:26:06.040192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.149579+0000","last_change":"2023-11-06T02:08:30.650121+0000","last_active":"2023-11-06T02:08:32.149579+0000","last_peered":"2023-11-06T02:08:32.149579+0000","last_clean":"2023-11-06T02:08:32.149579+0000","last_became_active":"2023-11-06T02:08:30.649010+0000","last_became_peered":"2023-11-06T02:08:30.649010+0000","last_unstale":"2023-11-06T02:08:32.149579+0000","last_undegraded":"2023-11-06T02:08:32.149579+0000","last_fullsized":"2023-11-06T02:08:32.149579+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:56:51.565719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.651624+0000","last_change":"2023-11-06T02:08:30.622216+0000","last_active":"2023-11-06T02:08:32.651624+0000","last_peered":"2023-11-06T02:08:32.651624+0000","last_clean":"2023-11-06T02:08:32.651624+0000","last_became_active":"2023-11-06T02:08:30.621706+0000","last_became_peered":"2023-11-06T02:08:30.621706+0000","last_unstale":"2023-11-06T02:08:32.651624+0000","last_undegraded":"2023-11-06T02:08:32.651624+0000","last_fullsized":"2023-11-06T02:08:32.651624+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:52:47.174384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.604379+0000","last_change":"2023-11-06T02:08:30.623436+0000","last_active":"2023-11-06T02:08:31.604379+0000","last_peered":"2023-11-06T02:08:31.604379+0000","last_clean":"2023-11-06T02:08:31.604379+0000","last_became_active":"2023-11-06T02:08:30.623149+0000","last_became_peered":"2023-11-06T02:08:30.623149+0000","last_unstale":"2023-11-06T02:08:31.604379+0000","last_undegraded":"2023-11-06T02:08:31.604379+0000","last_fullsized":"2023-11-06T02:08:31.604379+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:47:30.859032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.149601+0000","last_change":"2023-11-06T02:08:30.656899+0000","last_active":"2023-11-06T02:08:32.149601+0000","last_peered":"2023-11-06T02:08:32.149601+0000","last_clean":"2023-11-06T02:08:32.149601+0000","last_became_active":"2023-11-06T02:08:30.656319+0000","last_became_peered":"2023-11-06T02:08:30.656319+0000","last_unstale":"2023-11-06T02:08:32.149601+0000","last_undegraded":"2023-11-06T02:08:32.149601+0000","last_fullsized":"2023-11-06T02:08:32.149601+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:57:32.079075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.652785+0000","last_change":"2023-11-06T02:08:30.627856+0000","last_active":"2023-11-06T02:08:32.652785+0000","last_peered":"2023-11-06T02:08:32.652785+0000","last_clean":"2023-11-06T02:08:32.652785+0000","last_became_active":"2023-11-06T02:08:30.627536+0000","last_became_peered":"2023-11-06T02:08:30.627536+0000","last_unstale":"2023-11-06T02:08:32.652785+0000","last_undegraded":"2023-11-06T02:08:32.652785+0000","last_fullsized":"2023-11-06T02:08:32.652785+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:53:11.188799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.652773+0000","last_change":"2023-11-06T02:08:30.613522+0000","last_active":"2023-11-06T02:08:32.652773+0000","last_peered":"2023-11-06T02:08:32.652773+0000","last_clean":"2023-11-06T02:08:32.652773+0000","last_became_active":"2023-11-06T02:08:30.612950+0000","last_became_peered":"2023-11-06T02:08:30.612950+0000","last_unstale":"2023-11-06T02:08:32.652773+0000","last_undegraded":"2023-11-06T02:08:32.652773+0000","last_fullsized":"2023-11-06T02:08:32.652773+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:26:58.090934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.413094+0000","last_change":"2023-11-06T02:08:30.653593+0000","last_active":"2023-11-06T02:08:32.413094+0000","last_peered":"2023-11-06T02:08:32.413094+0000","last_clean":"2023-11-06T02:08:32.413094+0000","last_became_active":"2023-11-06T02:08:30.652925+0000","last_became_peered":"2023-11-06T02:08:30.652925+0000","last_unstale":"2023-11-06T02:08:32.413094+0000","last_undegraded":"2023-11-06T02:08:32.413094+0000","last_fullsized":"2023-11-06T02:08:32.413094+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:17:50.007398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.652200+0000","last_change":"2023-11-06T02:08:30.645260+0000","last_active":"2023-11-06T02:08:32.652200+0000","last_peered":"2023-11-06T02:08:32.652200+0000","last_clean":"2023-11-06T02:08:32.652200+0000","last_became_active":"2023-11-06T02:08:30.644949+0000","last_became_peered":"2023-11-06T02:08:30.644949+0000","last_unstale":"2023-11-06T02:08:32.652200+0000","last_undegraded":"2023-11-06T02:08:32.652200+0000","last_fullsized":"2023-11-06T02:08:32.652200+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:41:00.973671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.604763+0000","last_change":"2023-11-06T02:08:30.649814+0000","last_active":"2023-11-06T02:08:31.604763+0000","last_peered":"2023-11-06T02:08:31.604763+0000","last_clean":"2023-11-06T02:08:31.604763+0000","last_became_active":"2023-11-06T02:08:30.649492+0000","last_became_peered":"2023-11-06T02:08:30.649492+0000","last_unstale":"2023-11-06T02:08:31.604763+0000","last_undegraded":"2023-11-06T02:08:31.604763+0000","last_fullsized":"2023-11-06T02:08:31.604763+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:26:58.090934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.609372+0000","last_change":"2023-11-06T02:08:30.632560+0000","last_active":"2023-11-06T02:08:31.609372+0000","last_peered":"2023-11-06T02:08:31.609372+0000","last_clean":"2023-11-06T02:08:31.609372+0000","last_became_active":"2023-11-06T02:08:30.631819+0000","last_became_peered":"2023-11-06T02:08:30.631819+0000","last_unstale":"2023-11-06T02:08:31.609372+0000","last_undegraded":"2023-11-06T02:08:31.609372+0000","last_fullsized":"2023-11-06T02:08:31.609372+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:43:23.215718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.604562+0000","last_change":"2023-11-06T02:08:30.624052+0000","last_active":"2023-11-06T02:08:31.604562+0000","last_peered":"2023-11-06T02:08:31.604562+0000","last_clean":"2023-11-06T02:08:31.604562+0000","last_became_active":"2023-11-06T02:08:30.623658+0000","last_became_peered":"2023-11-06T02:08:30.623658+0000","last_unstale":"2023-11-06T02:08:31.604562+0000","last_undegraded":"2023-11-06T02:08:31.604562+0000","last_fullsized":"2023-11-06T02:08:31.604562+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:17:50.007398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.609631+0000","last_change":"2023-11-06T02:08:30.614066+0000","last_active":"2023-11-06T02:08:31.609631+0000","last_peered":"2023-11-06T02:08:31.609631+0000","last_clean":"2023-11-06T02:08:31.609631+0000","last_became_active":"2023-11-06T02:08:30.613337+0000","last_became_peered":"2023-11-06T02:08:30.613337+0000","last_unstale":"2023-11-06T02:08:31.609631+0000","last_undegraded":"2023-11-06T02:08:31.609631+0000","last_fullsized":"2023-11-06T02:08:31.609631+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:33:39.359552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.604963+0000","last_change":"2023-11-06T02:08:30.621368+0000","last_active":"2023-11-06T02:08:31.604963+0000","last_peered":"2023-11-06T02:08:31.604963+0000","last_clean":"2023-11-06T02:08:31.604963+0000","last_became_active":"2023-11-06T02:08:30.620563+0000","last_became_peered":"2023-11-06T02:08:30.620563+0000","last_unstale":"2023-11-06T02:08:31.604963+0000","last_undegraded":"2023-11-06T02:08:31.604963+0000","last_fullsized":"2023-11-06T02:08:31.604963+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:56:51.565719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"22'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.149865+0000","last_change":"2023-11-06T02:08:30.229825+0000","last_active":"2023-11-06T02:08:32.149865+0000","last_peered":"2023-11-06T02:08:32.149865+0000","last_clean":"2023-11-06T02:08:32.149865+0000","last_became_active":"2023-11-06T02:08:30.229392+0000","last_became_peered":"2023-11-06T02:08:30.229392+0000","last_unstale":"2023-11-06T02:08:32.149865+0000","last_undegraded":"2023-11-06T02:08:32.149865+0000","last_fullsized":"2023-11-06T02:08:32.149865+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:33:38.355715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.413299+0000","last_change":"2023-11-06T02:08:30.646474+0000","last_active":"2023-11-06T02:08:32.413299+0000","last_peered":"2023-11-06T02:08:32.413299+0000","last_clean":"2023-11-06T02:08:32.413299+0000","last_became_active":"2023-11-06T02:08:30.645992+0000","last_became_peered":"2023-11-06T02:08:30.645992+0000","last_unstale":"2023-11-06T02:08:32.413299+0000","last_undegraded":"2023-11-06T02:08:32.413299+0000","last_fullsized":"2023-11-06T02:08:32.413299+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:47:30.859032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.651766+0000","last_change":"2023-11-06T02:08:30.225818+0000","last_active":"2023-11-06T02:08:32.651766+0000","last_peered":"2023-11-06T02:08:32.651766+0000","last_clean":"2023-11-06T02:08:32.651766+0000","last_became_active":"2023-11-06T02:08:30.225359+0000","last_became_peered":"2023-11-06T02:08:30.225359+0000","last_unstale":"2023-11-06T02:08:32.651766+0000","last_undegraded":"2023-11-06T02:08:32.651766+0000","last_fullsized":"2023-11-06T02:08:32.651766+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:52:19.592716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.653330+0000","last_change":"2023-11-06T02:08:30.618020+0000","last_active":"2023-11-06T02:08:32.653330+0000","last_peered":"2023-11-06T02:08:32.653330+0000","last_clean":"2023-11-06T02:08:32.653330+0000","last_became_active":"2023-11-06T02:08:30.617684+0000","last_became_peered":"2023-11-06T02:08:30.617684+0000","last_unstale":"2023-11-06T02:08:32.653330+0000","last_undegraded":"2023-11-06T02:08:32.653330+0000","last_fullsized":"2023-11-06T02:08:32.653330+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:56:51.565719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.149630+0000","last_change":"2023-11-06T02:08:30.230443+0000","last_active":"2023-11-06T02:08:32.149630+0000","last_peered":"2023-11-06T02:08:32.149630+0000","last_clean":"2023-11-06T02:08:32.149630+0000","last_became_active":"2023-11-06T02:08:30.229980+0000","last_became_peered":"2023-11-06T02:08:30.229980+0000","last_unstale":"2023-11-06T02:08:32.149630+0000","last_undegraded":"2023-11-06T02:08:32.149630+0000","last_fullsized":"2023-11-06T02:08:32.149630+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:46:37.864109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.605377+0000","last_change":"2023-11-06T02:08:30.647210+0000","last_active":"2023-11-06T02:08:31.605377+0000","last_peered":"2023-11-06T02:08:31.605377+0000","last_clean":"2023-11-06T02:08:31.605377+0000","last_became_active":"2023-11-06T02:08:30.646862+0000","last_became_peered":"2023-11-06T02:08:30.646862+0000","last_unstale":"2023-11-06T02:08:31.605377+0000","last_undegraded":"2023-11-06T02:08:31.605377+0000","last_fullsized":"2023-11-06T02:08:31.605377+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:43:23.215718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"22'8","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.609883+0000","last_change":"2023-11-06T02:08:30.218861+0000","last_active":"2023-11-06T02:08:31.609883+0000","last_peered":"2023-11-06T02:08:31.609883+0000","last_clean":"2023-11-06T02:08:31.609883+0000","last_became_active":"2023-11-06T02:08:30.218245+0000","last_became_peered":"2023-11-06T02:08:30.218245+0000","last_unstale":"2023-11-06T02:08:31.609883+0000","last_undegraded":"2023-11-06T02:08:31.609883+0000","last_fullsized":"2023-11-06T02:08:31.609883+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:04:42.682132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"22'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.150304+0000","last_change":"2023-11-06T02:08:30.229578+0000","last_active":"2023-11-06T02:08:32.150304+0000","last_peered":"2023-11-06T02:08:32.150304+0000","last_clean":"2023-11-06T02:08:32.150304+0000","last_became_active":"2023-11-06T02:08:30.229131+0000","last_became_peered":"2023-11-06T02:08:30.229131+0000","last_unstale":"2023-11-06T02:08:32.150304+0000","last_undegraded":"2023-11-06T02:08:32.150304+0000","last_fullsized":"2023-11-06T02:08:32.150304+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:30:49.856478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.412653+0000","last_change":"2023-11-06T02:08:30.647714+0000","last_active":"2023-11-06T02:08:32.412653+0000","last_peered":"2023-11-06T02:08:32.412653+0000","last_clean":"2023-11-06T02:08:32.412653+0000","last_became_active":"2023-11-06T02:08:30.647212+0000","last_became_peered":"2023-11-06T02:08:30.647212+0000","last_unstale":"2023-11-06T02:08:32.412653+0000","last_undegraded":"2023-11-06T02:08:32.412653+0000","last_fullsized":"2023-11-06T02:08:32.412653+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:13:37.616527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"22'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.150130+0000","last_change":"2023-11-06T02:08:30.230469+0000","last_active":"2023-11-06T02:08:32.150130+0000","last_peered":"2023-11-06T02:08:32.150130+0000","last_clean":"2023-11-06T02:08:32.150130+0000","last_became_active":"2023-11-06T02:08:30.230064+0000","last_became_peered":"2023-11-06T02:08:30.230064+0000","last_unstale":"2023-11-06T02:08:32.150130+0000","last_undegraded":"2023-11-06T02:08:32.150130+0000","last_fullsized":"2023-11-06T02:08:32.150130+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:19:59.347221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.608834+0000","last_change":"2023-11-06T02:08:30.632754+0000","last_active":"2023-11-06T02:08:31.608834+0000","last_peered":"2023-11-06T02:08:31.608834+0000","last_clean":"2023-11-06T02:08:31.608834+0000","last_became_active":"2023-11-06T02:08:30.631976+0000","last_became_peered":"2023-11-06T02:08:30.631976+0000","last_unstale":"2023-11-06T02:08:31.608834+0000","last_undegraded":"2023-11-06T02:08:31.608834+0000","last_fullsized":"2023-11-06T02:08:31.608834+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:56:51.565719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"22'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.149972+0000","last_change":"2023-11-06T02:08:30.231003+0000","last_active":"2023-11-06T02:08:32.149972+0000","last_peered":"2023-11-06T02:08:32.149972+0000","last_clean":"2023-11-06T02:08:32.149972+0000","last_became_active":"2023-11-06T02:08:30.229978+0000","last_became_peered":"2023-11-06T02:08:30.229978+0000","last_unstale":"2023-11-06T02:08:32.149972+0000","last_undegraded":"2023-11-06T02:08:32.149972+0000","last_fullsized":"2023-11-06T02:08:32.149972+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:24:09.801921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.609985+0000","last_change":"2023-11-06T02:08:30.619807+0000","last_active":"2023-11-06T02:08:31.609985+0000","last_peered":"2023-11-06T02:08:31.609985+0000","last_clean":"2023-11-06T02:08:31.609985+0000","last_became_active":"2023-11-06T02:08:30.619048+0000","last_became_peered":"2023-11-06T02:08:30.619048+0000","last_unstale":"2023-11-06T02:08:31.609985+0000","last_undegraded":"2023-11-06T02:08:31.609985+0000","last_fullsized":"2023-11-06T02:08:31.609985+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:15:59.045303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.412965+0000","last_change":"2023-11-06T02:08:30.218715+0000","last_active":"2023-11-06T02:08:32.412965+0000","last_peered":"2023-11-06T02:08:32.412965+0000","last_clean":"2023-11-06T02:08:32.412965+0000","last_became_active":"2023-11-06T02:08:30.217966+0000","last_became_peered":"2023-11-06T02:08:30.217966+0000","last_unstale":"2023-11-06T02:08:32.412965+0000","last_undegraded":"2023-11-06T02:08:32.412965+0000","last_fullsized":"2023-11-06T02:08:32.412965+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:30:21.274364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.652004+0000","last_change":"2023-11-06T02:08:30.620567+0000","last_active":"2023-11-06T02:08:32.652004+0000","last_peered":"2023-11-06T02:08:32.652004+0000","last_clean":"2023-11-06T02:08:32.652004+0000","last_became_active":"2023-11-06T02:08:30.619671+0000","last_became_peered":"2023-11-06T02:08:30.619671+0000","last_unstale":"2023-11-06T02:08:32.652004+0000","last_undegraded":"2023-11-06T02:08:32.652004+0000","last_fullsized":"2023-11-06T02:08:32.652004+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:57:32.079075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"22'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.605088+0000","last_change":"2023-11-06T02:08:30.220060+0000","last_active":"2023-11-06T02:08:31.605088+0000","last_peered":"2023-11-06T02:08:31.605088+0000","last_clean":"2023-11-06T02:08:31.605088+0000","last_became_active":"2023-11-06T02:08:30.218534+0000","last_became_peered":"2023-11-06T02:08:30.218534+0000","last_unstale":"2023-11-06T02:08:31.605088+0000","last_undegraded":"2023-11-06T02:08:31.605088+0000","last_fullsized":"2023-11-06T02:08:31.605088+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:30:21.274364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.609657+0000","last_change":"2023-11-06T02:08:30.658173+0000","last_active":"2023-11-06T02:08:31.609657+0000","last_peered":"2023-11-06T02:08:31.609657+0000","last_clean":"2023-11-06T02:08:31.609657+0000","last_became_active":"2023-11-06T02:08:30.656890+0000","last_became_peered":"2023-11-06T02:08:30.656890+0000","last_unstale":"2023-11-06T02:08:31.609657+0000","last_undegraded":"2023-11-06T02:08:31.609657+0000","last_fullsized":"2023-11-06T02:08:31.609657+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:23:59.639528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.610321+0000","last_change":"2023-11-06T02:08:30.222023+0000","last_active":"2023-11-06T02:08:31.610321+0000","last_peered":"2023-11-06T02:08:31.610321+0000","last_clean":"2023-11-06T02:08:31.610321+0000","last_became_active":"2023-11-06T02:08:30.220788+0000","last_became_peered":"2023-11-06T02:08:30.220788+0000","last_unstale":"2023-11-06T02:08:31.610321+0000","last_undegraded":"2023-11-06T02:08:31.610321+0000","last_fullsized":"2023-11-06T02:08:31.610321+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:31:31.882709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.610620+0000","last_change":"2023-11-06T02:08:30.635685+0000","last_active":"2023-11-06T02:08:31.610620+0000","last_peered":"2023-11-06T02:08:31.610620+0000","last_clean":"2023-11-06T02:08:31.610620+0000","last_became_active":"2023-11-06T02:08:30.634801+0000","last_became_peered":"2023-11-06T02:08:30.634801+0000","last_unstale":"2023-11-06T02:08:31.610620+0000","last_undegraded":"2023-11-06T02:08:31.610620+0000","last_fullsized":"2023-11-06T02:08:31.610620+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:37:47.953218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.609115+0000","last_change":"2023-11-06T02:08:30.222592+0000","last_active":"2023-11-06T02:08:31.609115+0000","last_peered":"2023-11-06T02:08:31.609115+0000","last_clean":"2023-11-06T02:08:31.609115+0000","last_became_active":"2023-11-06T02:08:30.222184+0000","last_became_peered":"2023-11-06T02:08:30.222184+0000","last_unstale":"2023-11-06T02:08:31.609115+0000","last_undegraded":"2023-11-06T02:08:31.609115+0000","last_fullsized":"2023-11-06T02:08:31.609115+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:31:31.882709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.609663+0000","last_change":"2023-11-06T02:08:30.651763+0000","last_active":"2023-11-06T02:08:31.609663+0000","last_peered":"2023-11-06T02:08:31.609663+0000","last_clean":"2023-11-06T02:08:31.609663+0000","last_became_active":"2023-11-06T02:08:30.651013+0000","last_became_peered":"2023-11-06T02:08:30.651013+0000","last_unstale":"2023-11-06T02:08:31.609663+0000","last_undegraded":"2023-11-06T02:08:31.609663+0000","last_fullsized":"2023-11-06T02:08:31.609663+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:41:51.822994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.604575+0000","last_change":"2023-11-06T02:08:30.225279+0000","last_active":"2023-11-06T02:08:31.604575+0000","last_peered":"2023-11-06T02:08:31.604575+0000","last_clean":"2023-11-06T02:08:31.604575+0000","last_became_active":"2023-11-06T02:08:30.224936+0000","last_became_peered":"2023-11-06T02:08:30.224936+0000","last_unstale":"2023-11-06T02:08:31.604575+0000","last_undegraded":"2023-11-06T02:08:31.604575+0000","last_fullsized":"2023-11-06T02:08:31.604575+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:52:19.592716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.653044+0000","last_change":"2023-11-06T02:08:30.619652+0000","last_active":"2023-11-06T02:08:32.653044+0000","last_peered":"2023-11-06T02:08:32.653044+0000","last_clean":"2023-11-06T02:08:32.653044+0000","last_became_active":"2023-11-06T02:08:30.619070+0000","last_became_peered":"2023-11-06T02:08:30.619070+0000","last_unstale":"2023-11-06T02:08:32.653044+0000","last_undegraded":"2023-11-06T02:08:32.653044+0000","last_fullsized":"2023-11-06T02:08:32.653044+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:56:51.565719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":167,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-06T02:08:30.628264+0000","last_change":"2023-11-06T02:08:20.523427+0000","last_active":"2023-11-06T02:08:30.628264+0000","last_peered":"2023-11-06T02:08:30.628264+0000","last_clean":"2023-11-06T02:08:30.628264+0000","last_became_active":"2023-11-06T02:08:20.521089+0000","last_became_peered":"2023-11-06T02:08:20.521089+0000","last_unstale":"2023-11-06T02:08:30.628264+0000","last_undegraded":"2023-11-06T02:08:30.628264+0000","last_fullsized":"2023-11-06T02:08:30.628264+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:19.252348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:19.252348+0000","last_clean_scrub_stamp":"2023-11-06T02:08:19.252348+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:30:15.566397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-06T02:08:30.640657+0000","last_change":"2023-11-06T02:08:26.059502+0000","last_active":"2023-11-06T02:08:30.640657+0000","last_peered":"2023-11-06T02:08:30.640657+0000","last_clean":"2023-11-06T02:08:30.640657+0000","last_became_active":"2023-11-06T02:08:26.058612+0000","last_became_peered":"2023-11-06T02:08:26.058612+0000","last_unstale":"2023-11-06T02:08:30.640657+0000","last_undegraded":"2023-11-06T02:08:30.640657+0000","last_fullsized":"2023-11-06T02:08:30.640657+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:10.626021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.604588+0000","last_change":"2023-11-06T02:08:30.219125+0000","last_active":"2023-11-06T02:08:31.604588+0000","last_peered":"2023-11-06T02:08:31.604588+0000","last_clean":"2023-11-06T02:08:31.604588+0000","last_became_active":"2023-11-06T02:08:30.217091+0000","last_became_peered":"2023-11-06T02:08:30.217091+0000","last_unstale":"2023-11-06T02:08:31.604588+0000","last_undegraded":"2023-11-06T02:08:31.604588+0000","last_fullsized":"2023-11-06T02:08:31.604588+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:04:42.682132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.610770+0000","last_change":"2023-11-06T02:08:30.660650+0000","last_active":"2023-11-06T02:08:31.610770+0000","last_peered":"2023-11-06T02:08:31.610770+0000","last_clean":"2023-11-06T02:08:31.610770+0000","last_became_active":"2023-11-06T02:08:30.660072+0000","last_became_peered":"2023-11-06T02:08:30.660072+0000","last_unstale":"2023-11-06T02:08:31.610770+0000","last_undegraded":"2023-11-06T02:08:31.610770+0000","last_fullsized":"2023-11-06T02:08:31.610770+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:30:50.860315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-06T02:08:30.626279+0000","last_change":"2023-11-06T02:08:26.060406+0000","last_active":"2023-11-06T02:08:30.626279+0000","last_peered":"2023-11-06T02:08:30.626279+0000","last_clean":"2023-11-06T02:08:30.626279+0000","last_became_active":"2023-11-06T02:08:26.059262+0000","last_became_peered":"2023-11-06T02:08:26.059262+0000","last_unstale":"2023-11-06T02:08:30.626279+0000","last_undegraded":"2023-11-06T02:08:30.626279+0000","last_fullsized":"2023-11-06T02:08:30.626279+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:10.626021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.651421+0000","last_change":"2023-11-06T02:08:30.227671+0000","last_active":"2023-11-06T02:08:32.651421+0000","last_peered":"2023-11-06T02:08:32.651421+0000","last_clean":"2023-11-06T02:08:32.651421+0000","last_became_active":"2023-11-06T02:08:30.227200+0000","last_became_peered":"2023-11-06T02:08:30.227200+0000","last_unstale":"2023-11-06T02:08:32.651421+0000","last_undegraded":"2023-11-06T02:08:32.651421+0000","last_fullsized":"2023-11-06T02:08:32.651421+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:26:49.298561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.653133+0000","last_change":"2023-11-06T02:08:30.619424+0000","last_active":"2023-11-06T02:08:32.653133+0000","last_peered":"2023-11-06T02:08:32.653133+0000","last_clean":"2023-11-06T02:08:32.653133+0000","last_became_active":"2023-11-06T02:08:30.619150+0000","last_became_peered":"2023-11-06T02:08:30.619150+0000","last_unstale":"2023-11-06T02:08:32.653133+0000","last_undegraded":"2023-11-06T02:08:32.653133+0000","last_fullsized":"2023-11-06T02:08:32.653133+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:47:30.859032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-06T02:08:30.597659+0000","last_change":"2023-11-06T02:08:26.559477+0000","last_active":"2023-11-06T02:08:30.597659+0000","last_peered":"2023-11-06T02:08:30.597659+0000","last_clean":"2023-11-06T02:08:30.597659+0000","last_became_active":"2023-11-06T02:08:26.559202+0000","last_became_peered":"2023-11-06T02:08:26.559202+0000","last_unstale":"2023-11-06T02:08:30.597659+0000","last_undegraded":"2023-11-06T02:08:30.597659+0000","last_fullsized":"2023-11-06T02:08:30.597659+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:55:04.552404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.651066+0000","last_change":"2023-11-06T02:08:30.219422+0000","last_active":"2023-11-06T02:08:32.651066+0000","last_peered":"2023-11-06T02:08:32.651066+0000","last_clean":"2023-11-06T02:08:32.651066+0000","last_became_active":"2023-11-06T02:08:30.219148+0000","last_became_peered":"2023-11-06T02:08:30.219148+0000","last_unstale":"2023-11-06T02:08:32.651066+0000","last_undegraded":"2023-11-06T02:08:32.651066+0000","last_fullsized":"2023-11-06T02:08:32.651066+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:52:19.592716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.653298+0000","last_change":"2023-11-06T02:08:30.655629+0000","last_active":"2023-11-06T02:08:32.653298+0000","last_peered":"2023-11-06T02:08:32.653298+0000","last_clean":"2023-11-06T02:08:32.653298+0000","last_became_active":"2023-11-06T02:08:30.655271+0000","last_became_peered":"2023-11-06T02:08:30.655271+0000","last_unstale":"2023-11-06T02:08:32.653298+0000","last_undegraded":"2023-11-06T02:08:32.653298+0000","last_fullsized":"2023-11-06T02:08:32.653298+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:10:14.974545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-06T02:08:30.641603+0000","last_change":"2023-11-06T02:08:26.059546+0000","last_active":"2023-11-06T02:08:30.641603+0000","last_peered":"2023-11-06T02:08:30.641603+0000","last_clean":"2023-11-06T02:08:30.641603+0000","last_became_active":"2023-11-06T02:08:26.058806+0000","last_became_peered":"2023-11-06T02:08:26.058806+0000","last_unstale":"2023-11-06T02:08:30.641603+0000","last_undegraded":"2023-11-06T02:08:30.641603+0000","last_fullsized":"2023-11-06T02:08:30.641603+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:23.468510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.609639+0000","last_change":"2023-11-06T02:08:30.223844+0000","last_active":"2023-11-06T02:08:31.609639+0000","last_peered":"2023-11-06T02:08:31.609639+0000","last_clean":"2023-11-06T02:08:31.609639+0000","last_became_active":"2023-11-06T02:08:30.222902+0000","last_became_peered":"2023-11-06T02:08:30.222902+0000","last_unstale":"2023-11-06T02:08:31.609639+0000","last_undegraded":"2023-11-06T02:08:31.609639+0000","last_fullsized":"2023-11-06T02:08:31.609639+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:52:19.592716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.651748+0000","last_change":"2023-11-06T02:08:30.619125+0000","last_active":"2023-11-06T02:08:32.651748+0000","last_peered":"2023-11-06T02:08:32.651748+0000","last_clean":"2023-11-06T02:08:32.651748+0000","last_became_active":"2023-11-06T02:08:30.617621+0000","last_became_peered":"2023-11-06T02:08:30.617621+0000","last_unstale":"2023-11-06T02:08:32.651748+0000","last_undegraded":"2023-11-06T02:08:32.651748+0000","last_fullsized":"2023-11-06T02:08:32.651748+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:13:37.616527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.413429+0000","last_change":"2023-11-06T02:08:30.225274+0000","last_active":"2023-11-06T02:08:32.413429+0000","last_peered":"2023-11-06T02:08:32.413429+0000","last_clean":"2023-11-06T02:08:32.413429+0000","last_became_active":"2023-11-06T02:08:30.224275+0000","last_became_peered":"2023-11-06T02:08:30.224275+0000","last_unstale":"2023-11-06T02:08:32.413429+0000","last_undegraded":"2023-11-06T02:08:32.413429+0000","last_fullsized":"2023-11-06T02:08:32.413429+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:26:49.298561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-06T02:08:30.614515+0000","last_change":"2023-11-06T02:08:26.436463+0000","last_active":"2023-11-06T02:08:30.614515+0000","last_peered":"2023-11-06T02:08:30.614515+0000","last_clean":"2023-11-06T02:08:30.614515+0000","last_became_active":"2023-11-06T02:08:26.436075+0000","last_became_peered":"2023-11-06T02:08:26.436075+0000","last_unstale":"2023-11-06T02:08:30.614515+0000","last_undegraded":"2023-11-06T02:08:30.614515+0000","last_fullsized":"2023-11-06T02:08:30.614515+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:23.468510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.651810+0000","last_change":"2023-11-06T02:08:30.655634+0000","last_active":"2023-11-06T02:08:32.651810+0000","last_peered":"2023-11-06T02:08:32.651810+0000","last_clean":"2023-11-06T02:08:32.651810+0000","last_became_active":"2023-11-06T02:08:30.655285+0000","last_became_peered":"2023-11-06T02:08:30.655285+0000","last_unstale":"2023-11-06T02:08:32.651810+0000","last_undegraded":"2023-11-06T02:08:32.651810+0000","last_fullsized":"2023-11-06T02:08:32.651810+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:49:05.460965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.651964+0000","last_change":"2023-11-06T02:08:30.220327+0000","last_active":"2023-11-06T02:08:32.651964+0000","last_peered":"2023-11-06T02:08:32.651964+0000","last_clean":"2023-11-06T02:08:32.651964+0000","last_became_active":"2023-11-06T02:08:30.219906+0000","last_became_peered":"2023-11-06T02:08:30.219906+0000","last_unstale":"2023-11-06T02:08:32.651964+0000","last_undegraded":"2023-11-06T02:08:32.651964+0000","last_fullsized":"2023-11-06T02:08:32.651964+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:43:56.017919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-06T02:08:30.636949+0000","last_change":"2023-11-06T02:08:26.436607+0000","last_active":"2023-11-06T02:08:30.636949+0000","last_peered":"2023-11-06T02:08:30.636949+0000","last_clean":"2023-11-06T02:08:30.636949+0000","last_became_active":"2023-11-06T02:08:26.435924+0000","last_became_peered":"2023-11-06T02:08:26.435924+0000","last_unstale":"2023-11-06T02:08:30.636949+0000","last_undegraded":"2023-11-06T02:08:30.636949+0000","last_fullsized":"2023-11-06T02:08:30.636949+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:23.468510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.652082+0000","last_change":"2023-11-06T02:08:30.654058+0000","last_active":"2023-11-06T02:08:32.652082+0000","last_peered":"2023-11-06T02:08:32.652082+0000","last_clean":"2023-11-06T02:08:32.652082+0000","last_became_active":"2023-11-06T02:08:30.653753+0000","last_became_peered":"2023-11-06T02:08:30.653753+0000","last_unstale":"2023-11-06T02:08:32.652082+0000","last_undegraded":"2023-11-06T02:08:32.652082+0000","last_fullsized":"2023-11-06T02:08:32.652082+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:13:37.616527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"22'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.605569+0000","last_change":"2023-11-06T02:08:30.231205+0000","last_active":"2023-11-06T02:08:31.605569+0000","last_peered":"2023-11-06T02:08:31.605569+0000","last_clean":"2023-11-06T02:08:31.605569+0000","last_became_active":"2023-11-06T02:08:30.230854+0000","last_became_peered":"2023-11-06T02:08:30.230854+0000","last_unstale":"2023-11-06T02:08:31.605569+0000","last_undegraded":"2023-11-06T02:08:31.605569+0000","last_fullsized":"2023-11-06T02:08:31.605569+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:26:49.298561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-06T02:08:30.614693+0000","last_change":"2023-11-06T02:08:26.057076+0000","last_active":"2023-11-06T02:08:30.614693+0000","last_peered":"2023-11-06T02:08:30.614693+0000","last_clean":"2023-11-06T02:08:30.614693+0000","last_became_active":"2023-11-06T02:08:26.056419+0000","last_became_peered":"2023-11-06T02:08:26.056419+0000","last_unstale":"2023-11-06T02:08:30.614693+0000","last_undegraded":"2023-11-06T02:08:30.614693+0000","last_fullsized":"2023-11-06T02:08:30.614693+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:10.626021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.604395+0000","last_change":"2023-11-06T02:08:30.647144+0000","last_active":"2023-11-06T02:08:31.604395+0000","last_peered":"2023-11-06T02:08:31.604395+0000","last_clean":"2023-11-06T02:08:31.604395+0000","last_became_active":"2023-11-06T02:08:30.646802+0000","last_became_peered":"2023-11-06T02:08:30.646802+0000","last_unstale":"2023-11-06T02:08:31.604395+0000","last_undegraded":"2023-11-06T02:08:31.604395+0000","last_fullsized":"2023-11-06T02:08:31.604395+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:49:05.460965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"22'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.653477+0000","last_change":"2023-11-06T02:08:30.225086+0000","last_active":"2023-11-06T02:08:32.653477+0000","last_peered":"2023-11-06T02:08:32.653477+0000","last_clean":"2023-11-06T02:08:32.653477+0000","last_became_active":"2023-11-06T02:08:30.224455+0000","last_became_peered":"2023-11-06T02:08:30.224455+0000","last_unstale":"2023-11-06T02:08:32.653477+0000","last_undegraded":"2023-11-06T02:08:32.653477+0000","last_fullsized":"2023-11-06T02:08:32.653477+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:26:21.361289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-06T02:08:30.606381+0000","last_change":"2023-11-06T02:08:26.058596+0000","last_active":"2023-11-06T02:08:30.606381+0000","last_peered":"2023-11-06T02:08:30.606381+0000","last_clean":"2023-11-06T02:08:30.606381+0000","last_became_active":"2023-11-06T02:08:26.057015+0000","last_became_peered":"2023-11-06T02:08:26.057015+0000","last_unstale":"2023-11-06T02:08:30.606381+0000","last_undegraded":"2023-11-06T02:08:30.606381+0000","last_fullsized":"2023-11-06T02:08:30.606381+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:55:04.552404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.610172+0000","last_change":"2023-11-06T02:08:30.656051+0000","last_active":"2023-11-06T02:08:31.610172+0000","last_peered":"2023-11-06T02:08:31.610172+0000","last_clean":"2023-11-06T02:08:31.610172+0000","last_became_active":"2023-11-06T02:08:30.650054+0000","last_became_peered":"2023-11-06T02:08:30.650054+0000","last_unstale":"2023-11-06T02:08:31.610172+0000","last_undegraded":"2023-11-06T02:08:31.610172+0000","last_fullsized":"2023-11-06T02:08:31.610172+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:17:50.007398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.609417+0000","last_change":"2023-11-06T02:08:30.633028+0000","last_active":"2023-11-06T02:08:31.609417+0000","last_peered":"2023-11-06T02:08:31.609417+0000","last_clean":"2023-11-06T02:08:31.609417+0000","last_became_active":"2023-11-06T02:08:30.632255+0000","last_became_peered":"2023-11-06T02:08:30.632255+0000","last_unstale":"2023-11-06T02:08:31.609417+0000","last_undegraded":"2023-11-06T02:08:31.609417+0000","last_fullsized":"2023-11-06T02:08:31.609417+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:47:30.859032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"22'6","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.609754+0000","last_change":"2023-11-06T02:08:30.230431+0000","last_active":"2023-11-06T02:08:31.609754+0000","last_peered":"2023-11-06T02:08:31.609754+0000","last_clean":"2023-11-06T02:08:31.609754+0000","last_became_active":"2023-11-06T02:08:30.229931+0000","last_became_peered":"2023-11-06T02:08:30.229931+0000","last_unstale":"2023-11-06T02:08:31.609754+0000","last_undegraded":"2023-11-06T02:08:31.609754+0000","last_fullsized":"2023-11-06T02:08:31.609754+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:26:21.361289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.651402+0000","last_change":"2023-11-06T02:08:30.628444+0000","last_active":"2023-11-06T02:08:32.651402+0000","last_peered":"2023-11-06T02:08:32.651402+0000","last_clean":"2023-11-06T02:08:32.651402+0000","last_became_active":"2023-11-06T02:08:30.628087+0000","last_became_peered":"2023-11-06T02:08:30.628087+0000","last_unstale":"2023-11-06T02:08:32.651402+0000","last_undegraded":"2023-11-06T02:08:32.651402+0000","last_fullsized":"2023-11-06T02:08:32.651402+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:57:32.079075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"22'3","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.149892+0000","last_change":"2023-11-06T02:08:30.230554+0000","last_active":"2023-11-06T02:08:32.149892+0000","last_peered":"2023-11-06T02:08:32.149892+0000","last_clean":"2023-11-06T02:08:32.149892+0000","last_became_active":"2023-11-06T02:08:30.229374+0000","last_became_peered":"2023-11-06T02:08:30.229374+0000","last_unstale":"2023-11-06T02:08:32.149892+0000","last_undegraded":"2023-11-06T02:08:32.149892+0000","last_fullsized":"2023-11-06T02:08:32.149892+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:15:58.041467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.413338+0000","last_change":"2023-11-06T02:08:30.231726+0000","last_active":"2023-11-06T02:08:32.413338+0000","last_peered":"2023-11-06T02:08:32.413338+0000","last_clean":"2023-11-06T02:08:32.413338+0000","last_became_active":"2023-11-06T02:08:30.230802+0000","last_became_peered":"2023-11-06T02:08:30.230802+0000","last_unstale":"2023-11-06T02:08:32.413338+0000","last_undegraded":"2023-11-06T02:08:32.413338+0000","last_fullsized":"2023-11-06T02:08:32.413338+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:15:58.041467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.604488+0000","last_change":"2023-11-06T02:08:30.644420+0000","last_active":"2023-11-06T02:08:31.604488+0000","last_peered":"2023-11-06T02:08:31.604488+0000","last_clean":"2023-11-06T02:08:31.604488+0000","last_became_active":"2023-11-06T02:08:30.644097+0000","last_became_peered":"2023-11-06T02:08:30.644097+0000","last_unstale":"2023-11-06T02:08:31.604488+0000","last_undegraded":"2023-11-06T02:08:31.604488+0000","last_fullsized":"2023-11-06T02:08:31.604488+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:31:28.301664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.651911+0000","last_change":"2023-11-06T02:08:30.218801+0000","last_active":"2023-11-06T02:08:32.651911+0000","last_peered":"2023-11-06T02:08:32.651911+0000","last_clean":"2023-11-06T02:08:32.651911+0000","last_became_active":"2023-11-06T02:08:30.217237+0000","last_became_peered":"2023-11-06T02:08:30.217237+0000","last_unstale":"2023-11-06T02:08:32.651911+0000","last_undegraded":"2023-11-06T02:08:32.651911+0000","last_fullsized":"2023-11-06T02:08:32.651911+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:40:04.297054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.604444+0000","last_change":"2023-11-06T02:08:30.619501+0000","last_active":"2023-11-06T02:08:31.604444+0000","last_peered":"2023-11-06T02:08:31.604444+0000","last_clean":"2023-11-06T02:08:31.604444+0000","last_became_active":"2023-11-06T02:08:30.619180+0000","last_became_peered":"2023-11-06T02:08:30.619180+0000","last_unstale":"2023-11-06T02:08:31.604444+0000","last_undegraded":"2023-11-06T02:08:31.604444+0000","last_fullsized":"2023-11-06T02:08:31.604444+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:13:37.616527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.611492+0000","last_change":"2023-11-06T02:08:30.655759+0000","last_active":"2023-11-06T02:08:31.611492+0000","last_peered":"2023-11-06T02:08:31.611492+0000","last_clean":"2023-11-06T02:08:31.611492+0000","last_became_active":"2023-11-06T02:08:30.655285+0000","last_became_peered":"2023-11-06T02:08:30.655285+0000","last_unstale":"2023-11-06T02:08:31.611492+0000","last_undegraded":"2023-11-06T02:08:31.611492+0000","last_fullsized":"2023-11-06T02:08:31.611492+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:20:00.351057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.604479+0000","last_change":"2023-11-06T02:08:30.222521+0000","last_active":"2023-11-06T02:08:31.604479+0000","last_peered":"2023-11-06T02:08:31.604479+0000","last_clean":"2023-11-06T02:08:31.604479+0000","last_became_active":"2023-11-06T02:08:30.221899+0000","last_became_peered":"2023-11-06T02:08:30.221899+0000","last_unstale":"2023-11-06T02:08:31.604479+0000","last_undegraded":"2023-11-06T02:08:31.604479+0000","last_fullsized":"2023-11-06T02:08:31.604479+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:43:56.017919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.652973+0000","last_change":"2023-11-06T02:08:30.644462+0000","last_active":"2023-11-06T02:08:32.652973+0000","last_peered":"2023-11-06T02:08:32.652973+0000","last_clean":"2023-11-06T02:08:32.652973+0000","last_became_active":"2023-11-06T02:08:30.644160+0000","last_became_peered":"2023-11-06T02:08:30.644160+0000","last_unstale":"2023-11-06T02:08:32.652973+0000","last_undegraded":"2023-11-06T02:08:32.652973+0000","last_fullsized":"2023-11-06T02:08:32.652973+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:43:23.215718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.652108+0000","last_change":"2023-11-06T02:08:30.214519+0000","last_active":"2023-11-06T02:08:32.652108+0000","last_peered":"2023-11-06T02:08:32.652108+0000","last_clean":"2023-11-06T02:08:32.652108+0000","last_became_active":"2023-11-06T02:08:30.214225+0000","last_became_peered":"2023-11-06T02:08:30.214225+0000","last_unstale":"2023-11-06T02:08:32.652108+0000","last_undegraded":"2023-11-06T02:08:32.652108+0000","last_fullsized":"2023-11-06T02:08:32.652108+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:30:21.274364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.605196+0000","last_change":"2023-11-06T02:08:30.617971+0000","last_active":"2023-11-06T02:08:31.605196+0000","last_peered":"2023-11-06T02:08:31.605196+0000","last_clean":"2023-11-06T02:08:31.605196+0000","last_became_active":"2023-11-06T02:08:30.617692+0000","last_became_peered":"2023-11-06T02:08:30.617692+0000","last_unstale":"2023-11-06T02:08:31.605196+0000","last_undegraded":"2023-11-06T02:08:31.605196+0000","last_fullsized":"2023-11-06T02:08:31.605196+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:23:59.639528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"22'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.609142+0000","last_change":"2023-11-06T02:08:30.224834+0000","last_active":"2023-11-06T02:08:31.609142+0000","last_peered":"2023-11-06T02:08:31.609142+0000","last_clean":"2023-11-06T02:08:31.609142+0000","last_became_active":"2023-11-06T02:08:30.224244+0000","last_became_peered":"2023-11-06T02:08:30.224244+0000","last_unstale":"2023-11-06T02:08:31.609142+0000","last_undegraded":"2023-11-06T02:08:31.609142+0000","last_fullsized":"2023-11-06T02:08:31.609142+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:40:04.297054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.654928+0000","last_change":"2023-11-06T02:08:30.619430+0000","last_active":"2023-11-06T02:08:32.654928+0000","last_peered":"2023-11-06T02:08:32.654928+0000","last_clean":"2023-11-06T02:08:32.654928+0000","last_became_active":"2023-11-06T02:08:30.619155+0000","last_became_peered":"2023-11-06T02:08:30.619155+0000","last_unstale":"2023-11-06T02:08:32.654928+0000","last_undegraded":"2023-11-06T02:08:32.654928+0000","last_fullsized":"2023-11-06T02:08:32.654928+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:30:22.278201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"22'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.652274+0000","last_change":"2023-11-06T02:08:30.227830+0000","last_active":"2023-11-06T02:08:32.652274+0000","last_peered":"2023-11-06T02:08:32.652274+0000","last_clean":"2023-11-06T02:08:32.652274+0000","last_became_active":"2023-11-06T02:08:30.227476+0000","last_became_peered":"2023-11-06T02:08:30.227476+0000","last_unstale":"2023-11-06T02:08:32.652274+0000","last_undegraded":"2023-11-06T02:08:32.652274+0000","last_fullsized":"2023-11-06T02:08:32.652274+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:30:21.274364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.609173+0000","last_change":"2023-11-06T02:08:30.651612+0000","last_active":"2023-11-06T02:08:31.609173+0000","last_peered":"2023-11-06T02:08:31.609173+0000","last_clean":"2023-11-06T02:08:31.609173+0000","last_became_active":"2023-11-06T02:08:30.650925+0000","last_became_peered":"2023-11-06T02:08:30.650925+0000","last_unstale":"2023-11-06T02:08:31.609173+0000","last_undegraded":"2023-11-06T02:08:31.609173+0000","last_fullsized":"2023-11-06T02:08:31.609173+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:31:28.301664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"22'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.413042+0000","last_change":"2023-11-06T02:08:30.217273+0000","last_active":"2023-11-06T02:08:32.413042+0000","last_peered":"2023-11-06T02:08:32.413042+0000","last_clean":"2023-11-06T02:08:32.413042+0000","last_became_active":"2023-11-06T02:08:30.216532+0000","last_became_peered":"2023-11-06T02:08:30.216532+0000","last_unstale":"2023-11-06T02:08:32.413042+0000","last_undegraded":"2023-11-06T02:08:32.413042+0000","last_fullsized":"2023-11-06T02:08:32.413042+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:33:38.355715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.608910+0000","last_change":"2023-11-06T02:08:30.651587+0000","last_active":"2023-11-06T02:08:31.608910+0000","last_peered":"2023-11-06T02:08:31.608910+0000","last_clean":"2023-11-06T02:08:31.608910+0000","last_became_active":"2023-11-06T02:08:30.650762+0000","last_became_peered":"2023-11-06T02:08:30.650762+0000","last_unstale":"2023-11-06T02:08:31.608910+0000","last_undegraded":"2023-11-06T02:08:31.608910+0000","last_fullsized":"2023-11-06T02:08:31.608910+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:52.535882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"22'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.413347+0000","last_change":"2023-11-06T02:08:30.220284+0000","last_active":"2023-11-06T02:08:32.413347+0000","last_peered":"2023-11-06T02:08:32.413347+0000","last_clean":"2023-11-06T02:08:32.413347+0000","last_became_active":"2023-11-06T02:08:30.219571+0000","last_became_peered":"2023-11-06T02:08:30.219571+0000","last_unstale":"2023-11-06T02:08:32.413347+0000","last_undegraded":"2023-11-06T02:08:32.413347+0000","last_fullsized":"2023-11-06T02:08:32.413347+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:24:09.801921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.604654+0000","last_change":"2023-11-06T02:08:30.621580+0000","last_active":"2023-11-06T02:08:31.604654+0000","last_peered":"2023-11-06T02:08:31.604654+0000","last_clean":"2023-11-06T02:08:31.604654+0000","last_became_active":"2023-11-06T02:08:30.618164+0000","last_became_peered":"2023-11-06T02:08:30.618164+0000","last_unstale":"2023-11-06T02:08:31.604654+0000","last_undegraded":"2023-11-06T02:08:31.604654+0000","last_fullsized":"2023-11-06T02:08:31.604654+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:57:32.079075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"22'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.604995+0000","last_change":"2023-11-06T02:08:30.208980+0000","last_active":"2023-11-06T02:08:31.604995+0000","last_peered":"2023-11-06T02:08:31.604995+0000","last_clean":"2023-11-06T02:08:31.604995+0000","last_became_active":"2023-11-06T02:08:30.208339+0000","last_became_peered":"2023-11-06T02:08:30.208339+0000","last_unstale":"2023-11-06T02:08:31.604995+0000","last_undegraded":"2023-11-06T02:08:31.604995+0000","last_fullsized":"2023-11-06T02:08:31.604995+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:31:31.882709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.608588+0000","last_change":"2023-11-06T02:08:30.634399+0000","last_active":"2023-11-06T02:08:31.608588+0000","last_peered":"2023-11-06T02:08:31.608588+0000","last_clean":"2023-11-06T02:08:31.608588+0000","last_became_active":"2023-11-06T02:08:30.633674+0000","last_became_peered":"2023-11-06T02:08:30.633674+0000","last_unstale":"2023-11-06T02:08:31.608588+0000","last_undegraded":"2023-11-06T02:08:31.608588+0000","last_fullsized":"2023-11-06T02:08:31.608588+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:14:45.248292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1f","version":"22'4","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.651772+0000","last_change":"2023-11-06T02:08:30.220947+0000","last_active":"2023-11-06T02:08:32.651772+0000","last_peered":"2023-11-06T02:08:32.651772+0000","last_clean":"2023-11-06T02:08:32.651772+0000","last_became_active":"2023-11-06T02:08:30.220379+0000","last_became_peered":"2023-11-06T02:08:30.220379+0000","last_unstale":"2023-11-06T02:08:32.651772+0000","last_undegraded":"2023-11-06T02:08:32.651772+0000","last_fullsized":"2023-11-06T02:08:32.651772+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:43:56.017919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27796,"kb_used_data":844,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344044,"statfs":{"total":96636764160,"available":96608301056,"internally_reserved":0,"allocated":864256,"data_stored":681747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73999999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.026}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97099999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83699999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1679999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3069999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.45}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27188,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":241664,"data_stored":89599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81200000000000006}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.087}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.268}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.474}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.347}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3180000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":237568,"data_stored":83598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.202}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1459999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3149999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":262144,"data_stored":91219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70399999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.026}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84899999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2210000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.157}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1919999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31288,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340552,"statfs":{"total":96636764160,"available":96604725248,"internally_reserved":0,"allocated":245760,"data_stored":89689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70399999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65800000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2929999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1899999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31284,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340556,"statfs":{"total":96636764160,"available":96604729344,"internally_reserved":0,"allocated":241664,"data_stored":89599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78300000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73999999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.625}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31272,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340568,"statfs":{"total":96636764160,"available":96604741632,"internally_reserved":0,"allocated":229376,"data_stored":82946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81999999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5349999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1459999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73599999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4690000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31852,"kb_used_data":804,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339988,"statfs":{"total":96636764160,"available":96604147712,"internally_reserved":0,"allocated":823296,"data_stored":672877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83299999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90200000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.052}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96999999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2230000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T02:08:39.124 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-06T02:08:39.124 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-11-06T02:08:39.125 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-11-06T02:08:39.125 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-11-06T02:08:39.126 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-11-06T02:08:39.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.294+0000 7ff798286640 1 Processor -- start 2023-11-06T02:08:39.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.294+0000 7ff798286640 1 -- start start 2023-11-06T02:08:39.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.298+0000 7ff798286640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7ff7900adae0 0x7ff7900c8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:39.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.298+0000 7ff798286640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7901594d0 con 0x7ff7901590f0 2023-11-06T02:08:39.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.298+0000 7ff798286640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7901597a0 con 0x7ff7900b0790 2023-11-06T02:08:39.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.298+0000 7ff798286640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff79015a140 con 0x7ff7900adae0 2023-11-06T02:08:39.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.298+0000 7ff796022640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7ff7901590f0 0x7ff7900b0080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:53918/0 (socket says 172.21.15.49:53918) 2023-11-06T02:08:39.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.298+0000 7ff796022640 1 -- 172.21.15.49:0/2229895442 learned_addr learned my addr 172.21.15.49:0/2229895442 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:08:39.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.298+0000 7ff795020640 1 -- 172.21.15.49:0/2229895442 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 994156887 0 0) 0x7ff7901594d0 con 0x7ff7901590f0 2023-11-06T02:08:39.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.298+0000 7ff795020640 1 -- 172.21.15.49:0/2229895442 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff778003450 con 0x7ff7901590f0 2023-11-06T02:08:39.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.298+0000 7ff796823640 1 --2- 172.21.15.49:0/2229895442 >> v2:172.21.15.150:3300/0 conn(0x7ff7900adae0 0x7ff7900c8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:39.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.298+0000 7ff795020640 1 -- 172.21.15.49:0/2229895442 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1899724668 0 0) 0x7ff778003450 con 0x7ff7901590f0 2023-11-06T02:08:39.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.298+0000 7ff795020640 1 -- 172.21.15.49:0/2229895442 >> v2:172.21.15.150:3300/0 conn(0x7ff7900adae0 msgr2=0x7ff7900c8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:39.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.298+0000 7ff795020640 1 --2- 172.21.15.49:0/2229895442 >> v2:172.21.15.150:3300/0 conn(0x7ff7900adae0 0x7ff7900c8c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:39.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.298+0000 7ff795020640 1 -- 172.21.15.49:0/2229895442 >> v1:172.21.15.150:6789/0 conn(0x7ff7900b0790 legacy=0x7ff790070b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:39.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.298+0000 7ff795020640 1 -- 172.21.15.49:0/2229895442 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff790159a80 con 0x7ff7901590f0 2023-11-06T02:08:39.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.298+0000 7ff795020640 1 -- 172.21.15.49:0/2229895442 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7ff784001800 con 0x7ff7901590f0 2023-11-06T02:08:39.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.298+0000 7ff796823640 1 --2- 172.21.15.49:0/2229895442 >> v2:172.21.15.150:3300/0 conn(0x7ff7900adae0 0x7ff7900c8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:08:39.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.298+0000 7ff795020640 1 -- 172.21.15.49:0/2229895442 >> v1:172.21.15.49:6789/0 conn(0x7ff7901590f0 legacy=0x7ff7900b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:39.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.298+0000 7ff795020640 1 --2- 172.21.15.49:0/2229895442 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff778003a70 0x7ff778003e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:39.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.298+0000 7ff795020640 1 --2- 172.21.15.49:0/2229895442 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7780088b0 0x7ff77800ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:39.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.298+0000 7ff795020640 1 --2- 172.21.15.49:0/2229895442 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff77800b5e0 0x7ff77800daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:39.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.298+0000 7ff795020640 1 -- 172.21.15.49:0/2229895442 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff79015a140 con 0x7ff778003a70 2023-11-06T02:08:39.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.298+0000 7ff795020640 1 -- 172.21.15.49:0/2229895442 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff77800e380 con 0x7ff7780088b0 2023-11-06T02:08:39.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.298+0000 7ff795020640 1 -- 172.21.15.49:0/2229895442 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff77800e540 con 0x7ff77800b5e0 2023-11-06T02:08:39.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.298+0000 7ff796022640 1 --2- 172.21.15.49:0/2229895442 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff778003a70 0x7ff778003e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:39.300 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.298+0000 7ff795821640 1 --2- 172.21.15.49:0/2229895442 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7780088b0 0x7ff77800ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:39.300 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.298+0000 7ff796022640 1 -- 172.21.15.49:0/2229895442 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff77800b5e0 msgr2=0x7ff77800daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:39.300 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.298+0000 7ff796022640 1 --2- 172.21.15.49:0/2229895442 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff77800b5e0 0x7ff77800daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:39.300 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.298+0000 7ff796022640 1 -- 172.21.15.49:0/2229895442 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7780088b0 msgr2=0x7ff77800ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:39.300 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.298+0000 7ff796022640 1 --2- 172.21.15.49:0/2229895442 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7780088b0 0x7ff77800ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:39.301 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.298+0000 7ff796823640 1 -- 172.21.15.49:0/2229895442 reap_dead start 2023-11-06T02:08:39.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.298+0000 7ff796022640 1 -- 172.21.15.49:0/2229895442 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff778003450 con 0x7ff778003a70 2023-11-06T02:08:39.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.298+0000 7ff795821640 1 --2- 172.21.15.49:0/2229895442 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7780088b0 0x7ff77800ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:08:39.305 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.302+0000 7ff796022640 1 --2- 172.21.15.49:0/2229895442 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff778003a70 0x7ff778003e90 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff78400c240 tx=0x7ff78401cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1cc3907e63b5e79 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:39.305 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.302+0000 7ff795020640 1 -- 172.21.15.49:0/2229895442 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff784007270 con 0x7ff778003a70 2023-11-06T02:08:39.305 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.302+0000 7ff795020640 1 -- 172.21.15.49:0/2229895442 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7840073f0 con 0x7ff778003a70 2023-11-06T02:08:39.306 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.302+0000 7ff795020640 1 -- 172.21.15.49:0/2229895442 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7840a58a0 con 0x7ff778003a70 2023-11-06T02:08:39.306 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.302+0000 7ff798286640 1 -- 172.21.15.49:0/2229895442 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff778003a70 msgr2=0x7ff778003e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:39.306 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.302+0000 7ff798286640 1 --2- 172.21.15.49:0/2229895442 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff778003a70 0x7ff778003e90 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff78400c240 tx=0x7ff78401cc20 comp rx=0 tx=0).stop 2023-11-06T02:08:39.306 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.302+0000 7ff798286640 1 -- 172.21.15.49:0/2229895442 shutdown_connections 2023-11-06T02:08:39.306 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.302+0000 7ff798286640 1 --2- 172.21.15.49:0/2229895442 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff778003a70 0x7ff778003e90 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:39.307 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.302+0000 7ff798286640 1 -- 172.21.15.49:0/2229895442 >> 172.21.15.49:0/2229895442 conn(0x7ff79006c0a0 msgr2=0x7ff79006c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:39.307 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.302+0000 7ff798286640 1 -- 172.21.15.49:0/2229895442 shutdown_connections 2023-11-06T02:08:39.307 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.302+0000 7ff798286640 1 -- 172.21.15.49:0/2229895442 wait complete. 2023-11-06T02:08:39.307 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.302+0000 7ff798286640 1 Processor -- start 2023-11-06T02:08:39.307 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.302+0000 7ff798286640 1 -- start start 2023-11-06T02:08:39.308 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.302+0000 7ff798286640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff778003a70 0x7ff79014d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:39.308 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.302+0000 7ff798286640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff7900c62f0 0x7ff7900c6750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:39.308 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.302+0000 7ff798286640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7901f51a0 0x7ff7901f5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:39.308 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.302+0000 7ff798286640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7900c6c90 con 0x7ff778003a70 2023-11-06T02:08:39.308 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.302+0000 7ff798286640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7900aeef0 con 0x7ff7901f51a0 2023-11-06T02:08:39.308 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.302+0000 7ff798286640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7900af1d0 con 0x7ff7900c62f0 2023-11-06T02:08:39.309 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.302+0000 7ff796022640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff778003a70 0x7ff79014d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:39.309 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.302+0000 7ff796022640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff778003a70 0x7ff79014d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51980/0 (socket says 172.21.15.49:51980) 2023-11-06T02:08:39.309 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.302+0000 7ff796022640 1 -- 172.21.15.49:0/3454651115 learned_addr learned my addr 172.21.15.49:0/3454651115 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:39.309 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.302+0000 7ff795821640 1 --2- 172.21.15.49:0/3454651115 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff7900c62f0 0x7ff7900c6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:39.309 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.302+0000 7ff796823640 1 --2- 172.21.15.49:0/3454651115 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7901f51a0 0x7ff7901f5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:39.310 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.302+0000 7ff796022640 1 -- 172.21.15.49:0/3454651115 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff7900c62f0 msgr2=0x7ff7900c6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:39.310 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.302+0000 7ff796022640 1 --2- 172.21.15.49:0/3454651115 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff7900c62f0 0x7ff7900c6750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:39.310 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.302+0000 7ff796022640 1 -- 172.21.15.49:0/3454651115 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7901f51a0 msgr2=0x7ff7901f5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:39.310 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.302+0000 7ff796022640 1 --2- 172.21.15.49:0/3454651115 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7901f51a0 0x7ff7901f5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:39.310 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.302+0000 7ff796022640 1 -- 172.21.15.49:0/3454651115 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7900af4b0 con 0x7ff778003a70 2023-11-06T02:08:39.311 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.302+0000 7ff796022640 1 --2- 172.21.15.49:0/3454651115 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff778003a70 0x7ff79014d850 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff78401ca50 tx=0x7ff778003700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:39.311 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.302+0000 7ff78effd640 1 -- 172.21.15.49:0/3454651115 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7840b4070 con 0x7ff778003a70 2023-11-06T02:08:39.311 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.302+0000 7ff78effd640 1 -- 172.21.15.49:0/3454651115 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff784000f80 con 0x7ff778003a70 2023-11-06T02:08:39.311 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.302+0000 7ff798286640 1 -- 172.21.15.49:0/3454651115 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7900af790 con 0x7ff778003a70 2023-11-06T02:08:39.311 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.302+0000 7ff78effd640 1 -- 172.21.15.49:0/3454651115 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7840015c0 con 0x7ff778003a70 2023-11-06T02:08:39.311 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.302+0000 7ff798286640 1 -- 172.21.15.49:0/3454651115 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff79014dd90 con 0x7ff778003a70 2023-11-06T02:08:39.312 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.306+0000 7ff798286640 1 -- 172.21.15.49:0/3454651115 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff790074040 con 0x7ff778003a70 2023-11-06T02:08:39.312 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.306+0000 7ff78effd640 1 -- 172.21.15.49:0/3454651115 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7ff7840057b0 con 0x7ff778003a70 2023-11-06T02:08:39.312 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.306+0000 7ff78effd640 1 --2- 172.21.15.49:0/3454651115 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff77c065a90 0x7ff77c067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:39.312 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.306+0000 7ff78effd640 1 -- 172.21.15.49:0/3454651115 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff78411c790 con 0x7ff778003a70 2023-11-06T02:08:39.312 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.306+0000 7ff795821640 1 --2- 172.21.15.49:0/3454651115 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff77c065a90 0x7ff77c067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:39.313 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.306+0000 7ff795821640 1 --2- 172.21.15.49:0/3454651115 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff77c065a90 0x7ff77c067f50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff790158230 tx=0x7ff788009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:39.313 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.310+0000 7ff78effd640 1 -- 172.21.15.49:0/3454651115 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7ff7840e9ff0 con 0x7ff778003a70 2023-11-06T02:08:39.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.502+0000 7ff798286640 1 -- 172.21.15.49:0/3454651115 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7ff7900cba50 con 0x7ff778003a70 2023-11-06T02:08:39.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.510+0000 7ff78effd640 1 -- 172.21.15.49:0/3454651115 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+163370 (secure 0 0 0) 0x7ff7840ed610 con 0x7ff778003a70 2023-11-06T02:08:39.507 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:08:39.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.514+0000 7ff798286640 1 -- 172.21.15.49:0/3454651115 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff77c065a90 msgr2=0x7ff77c067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:39.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.514+0000 7ff798286640 1 --2- 172.21.15.49:0/3454651115 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff77c065a90 0x7ff77c067f50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff790158230 tx=0x7ff788009040 comp rx=0 tx=0).stop 2023-11-06T02:08:39.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.514+0000 7ff798286640 1 -- 172.21.15.49:0/3454651115 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff778003a70 msgr2=0x7ff79014d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:39.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.514+0000 7ff798286640 1 --2- 172.21.15.49:0/3454651115 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff778003a70 0x7ff79014d850 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff78401ca50 tx=0x7ff778003700 comp rx=0 tx=0).stop 2023-11-06T02:08:39.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.514+0000 7ff798286640 1 -- 172.21.15.49:0/3454651115 shutdown_connections 2023-11-06T02:08:39.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.514+0000 7ff798286640 1 --2- 172.21.15.49:0/3454651115 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff77c065a90 0x7ff77c067f50 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:39.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.514+0000 7ff798286640 1 --2- 172.21.15.49:0/3454651115 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7901f51a0 0x7ff7901f5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:39.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.514+0000 7ff798286640 1 --2- 172.21.15.49:0/3454651115 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff7900c62f0 0x7ff7900c6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:39.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.514+0000 7ff798286640 1 --2- 172.21.15.49:0/3454651115 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff778003a70 0x7ff79014d850 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:39.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.514+0000 7ff798286640 1 -- 172.21.15.49:0/3454651115 >> 172.21.15.49:0/3454651115 conn(0x7ff79006c0a0 msgr2=0x7ff790158410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:39.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.514+0000 7ff798286640 1 -- 172.21.15.49:0/3454651115 shutdown_connections 2023-11-06T02:08:39.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.514+0000 7ff798286640 1 -- 172.21.15.49:0/3454651115 wait complete. 2023-11-06T02:08:39.527 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":5,"active_gid":4100,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":38104},{"type":"v1","addr":"172.21.15.150:6833","nonce":38104}]},"active_addr":"172.21.15.150:6833/38104","active_change":"2023-11-06T02:08:11.161309+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4104,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.2","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.2","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.150:0","nonce":3686269222}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.150:0","nonce":3372342502}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.150:0","nonce":3233858927}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.150:0","nonce":3444730927}]}]} 2023-11-06T02:08:39.531 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-11-06T02:08:39.531 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-11-06T02:08:39.531 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T02:08:39.657 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.662+0000 7f7b8cde3640 1 Processor -- start 2023-11-06T02:08:39.657 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.662+0000 7f7b8cde3640 1 -- start start 2023-11-06T02:08:39.658 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.662+0000 7f7b8cde3640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f7b880b1f90 0x7f7b880cda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:39.658 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.662+0000 7f7b8cde3640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7b88161650 0x7f7b880d1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:39.658 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.662+0000 7f7b8cde3640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b880d17c0 con 0x7f7b880d2190 2023-11-06T02:08:39.658 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.662+0000 7f7b8cde3640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b880ce6c0 con 0x7f7b880b1f90 2023-11-06T02:08:39.659 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.662+0000 7f7b8cde3640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b88161ad0 con 0x7f7b88161650 2023-11-06T02:08:39.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.662+0000 7f7b85d9b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7b88161650 0x7f7b880d1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:39.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.662+0000 7f7b85d9b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7b88161650 0x7f7b880d1c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.49:33380/0 (socket says 172.21.15.49:33380) 2023-11-06T02:08:39.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.662+0000 7f7b85d9b640 1 -- 172.21.15.49:0/1532581594 learned_addr learned my addr 172.21.15.49:0/1532581594 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:39.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.662+0000 7f7b8659c640 1 --2- 172.21.15.49:0/1532581594 >> v2:172.21.15.150:3300/0 conn(0x7f7b880b1f90 0x7f7b880cda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:39.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.662+0000 7f7b8559a640 1 -- 172.21.15.49:0/1532581594 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1186312245 0 0) 0x7f7b880d17c0 con 0x7f7b880d2190 2023-11-06T02:08:39.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.662+0000 7f7b8559a640 1 -- 172.21.15.49:0/1532581594 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b70003530 con 0x7f7b880d2190 2023-11-06T02:08:39.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.662+0000 7f7b85d9b640 1 -- 172.21.15.49:0/1532581594 >> v2:172.21.15.150:3300/0 conn(0x7f7b880b1f90 msgr2=0x7f7b880cda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:39.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.662+0000 7f7b85d9b640 1 --2- 172.21.15.49:0/1532581594 >> v2:172.21.15.150:3300/0 conn(0x7f7b880b1f90 0x7f7b880cda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:39.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.662+0000 7f7b85d9b640 1 -- 172.21.15.49:0/1532581594 >> v1:172.21.15.150:6789/0 conn(0x7f7b880d2190 legacy=0x7f7b880714a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:39.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.662+0000 7f7b85d9b640 1 -- 172.21.15.49:0/1532581594 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b880ce100 con 0x7f7b88161650 2023-11-06T02:08:39.662 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.666+0000 7f7b85d9b640 1 --2- 172.21.15.49:0/1532581594 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7b88161650 0x7f7b880d1c50 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c009870 tx=0x7f7b7c01c510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=838fa07106a08628 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:39.662 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.666+0000 7f7b8559a640 1 -- 172.21.15.49:0/1532581594 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b7c0aa670 con 0x7f7b88161650 2023-11-06T02:08:39.662 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.666+0000 7f7b8559a640 1 -- 172.21.15.49:0/1532581594 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b7c0aa7d0 con 0x7f7b88161650 2023-11-06T02:08:39.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.666+0000 7f7b8559a640 1 -- 172.21.15.49:0/1532581594 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b7c0aab40 con 0x7f7b88161650 2023-11-06T02:08:39.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.666+0000 7f7b8cde3640 1 -- 172.21.15.49:0/1532581594 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7b88161650 msgr2=0x7f7b880d1c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:39.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.666+0000 7f7b8cde3640 1 --2- 172.21.15.49:0/1532581594 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7b88161650 0x7f7b880d1c50 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c009870 tx=0x7f7b7c01c510 comp rx=0 tx=0).stop 2023-11-06T02:08:39.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.666+0000 7f7b8cde3640 1 -- 172.21.15.49:0/1532581594 shutdown_connections 2023-11-06T02:08:39.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.666+0000 7f7b8cde3640 1 --2- 172.21.15.49:0/1532581594 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7b88161650 0x7f7b880d1c50 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:39.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.666+0000 7f7b8cde3640 1 --2- 172.21.15.49:0/1532581594 >> v2:172.21.15.150:3300/0 conn(0x7f7b880b1f90 0x7f7b880cda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:39.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.666+0000 7f7b8cde3640 1 -- 172.21.15.49:0/1532581594 >> 172.21.15.49:0/1532581594 conn(0x7f7b880d2aa0 msgr2=0x7f7b880d2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:39.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.666+0000 7f7b8cde3640 1 -- 172.21.15.49:0/1532581594 shutdown_connections 2023-11-06T02:08:39.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.666+0000 7f7b8cde3640 1 -- 172.21.15.49:0/1532581594 wait complete. 2023-11-06T02:08:39.667 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.666+0000 7f7b8cde3640 1 Processor -- start 2023-11-06T02:08:39.667 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.666+0000 7f7b8cde3640 1 -- start start 2023-11-06T02:08:39.667 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.670+0000 7f7b8cde3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b880b1f90 0x7f7b88158f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:39.668 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.670+0000 7f7b8cde3640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7b880d2190 0x7f7b88153d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:39.668 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.670+0000 7f7b8cde3640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b88161650 0x7f7b880c9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:39.668 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.670+0000 7f7b8cde3640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b880ca310 con 0x7f7b880b1f90 2023-11-06T02:08:39.668 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.670+0000 7f7b8cde3640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b880ca5f0 con 0x7f7b88161650 2023-11-06T02:08:39.668 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.670+0000 7f7b8cde3640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b880ca8d0 con 0x7f7b880d2190 2023-11-06T02:08:39.668 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.670+0000 7f7b8659c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b880b1f90 0x7f7b88158f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:39.669 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.670+0000 7f7b8659c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b880b1f90 0x7f7b88158f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51992/0 (socket says 172.21.15.49:51992) 2023-11-06T02:08:39.669 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.670+0000 7f7b8659c640 1 -- 172.21.15.49:0/3464614741 learned_addr learned my addr 172.21.15.49:0/3464614741 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:39.669 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.670+0000 7f7b85d9b640 1 --2- 172.21.15.49:0/3464614741 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7b880d2190 0x7f7b88153d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:39.669 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.670+0000 7f7b86d9d640 1 --2- 172.21.15.49:0/3464614741 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b88161650 0x7f7b880c9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:39.669 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.670+0000 7f7b8659c640 1 -- 172.21.15.49:0/3464614741 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7b880d2190 msgr2=0x7f7b88153d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:39.670 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.670+0000 7f7b8659c640 1 --2- 172.21.15.49:0/3464614741 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7b880d2190 0x7f7b88153d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:39.670 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.670+0000 7f7b8659c640 1 -- 172.21.15.49:0/3464614741 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b88161650 msgr2=0x7f7b880c9ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:39.670 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.670+0000 7f7b8659c640 1 --2- 172.21.15.49:0/3464614741 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b88161650 0x7f7b880c9ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:39.670 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.670+0000 7f7b8659c640 1 -- 172.21.15.49:0/3464614741 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b7c0094d0 con 0x7f7b880b1f90 2023-11-06T02:08:39.670 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.670+0000 7f7b86d9d640 1 --2- 172.21.15.49:0/3464614741 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b88161650 0x7f7b880c9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:08:39.671 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.670+0000 7f7b8659c640 1 --2- 172.21.15.49:0/3464614741 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b880b1f90 0x7f7b88158f80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7b7800e930 tx=0x7f7b7800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:39.671 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.670+0000 7f7b76ffd640 1 -- 172.21.15.49:0/3464614741 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b78009690 con 0x7f7b880b1f90 2023-11-06T02:08:39.671 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.670+0000 7f7b76ffd640 1 -- 172.21.15.49:0/3464614741 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b7800ca00 con 0x7f7b880b1f90 2023-11-06T02:08:39.671 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.670+0000 7f7b8cde3640 1 -- 172.21.15.49:0/3464614741 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b880cabb0 con 0x7f7b880b1f90 2023-11-06T02:08:39.671 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.670+0000 7f7b76ffd640 1 -- 172.21.15.49:0/3464614741 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b7800cd70 con 0x7f7b880b1f90 2023-11-06T02:08:39.671 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.670+0000 7f7b8cde3640 1 -- 172.21.15.49:0/3464614741 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b880ca020 con 0x7f7b880b1f90 2023-11-06T02:08:39.672 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.670+0000 7f7b8cde3640 1 -- 172.21.15.49:0/3464614741 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b88074040 con 0x7f7b880b1f90 2023-11-06T02:08:39.672 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.670+0000 7f7b76ffd640 1 -- 172.21.15.49:0/3464614741 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f7b78013470 con 0x7f7b880b1f90 2023-11-06T02:08:39.673 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.670+0000 7f7b76ffd640 1 --2- 172.21.15.49:0/3464614741 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7b68065a60 0x7f7b68067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:39.673 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.670+0000 7f7b76ffd640 1 -- 172.21.15.49:0/3464614741 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7b78083570 con 0x7f7b880b1f90 2023-11-06T02:08:39.673 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.670+0000 7f7b85d9b640 1 --2- 172.21.15.49:0/3464614741 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7b68065a60 0x7f7b68067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:39.673 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.674+0000 7f7b85d9b640 1 --2- 172.21.15.49:0/3464614741 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7b68065a60 0x7f7b68067f20 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c0ae000 tx=0x7f7b7c0a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:39.673 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.674+0000 7f7b76ffd640 1 -- 172.21.15.49:0/3464614741 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f7b78050e90 con 0x7f7b880b1f90 2023-11-06T02:08:39.826 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.830+0000 7f7b8cde3640 1 -- 172.21.15.49:0/3464614741 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7b880d0c30 con 0x7f7b880b1f90 2023-11-06T02:08:39.829 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.834+0000 7f7b76ffd640 1 -- 172.21.15.49:0/3464614741 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+17811 (secure 0 0 0) 0x7f7b780544b0 con 0x7f7b880b1f90 2023-11-06T02:08:39.830 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:08:39.830 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":23,"fsid":"1a8b035c-6ba4-499a-b876-aea47fd9c0d4","created":"2023-11-06T02:08:06.440857+0000","modified":"2023-11-06T02:08:31.594964+0000","last_up_change":"2023-11-06T02:08:17.982372+0000","last_in_change":"2023-11-06T02:08:08.906631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T02:08:19.252222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T02:08:24.776211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-06T02:08:28.936178+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-06T02:08:29.563468+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ad55af8-0571-4e6a-8722-b5d429b7496d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6809","nonce":38116},{"type":"v1","addr":"172.21.15.49:6811","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6813","nonce":38116},{"type":"v1","addr":"172.21.15.49:6815","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6821","nonce":38116},{"type":"v1","addr":"172.21.15.49:6823","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6817","nonce":38116},{"type":"v1","addr":"172.21.15.49:6819","nonce":38116}]},"public_addr":"172.21.15.49:6811/38116","cluster_addr":"172.21.15.49:6815/38116","heartbeat_back_addr":"172.21.15.49:6823/38116","heartbeat_front_addr":"172.21.15.49:6819/38116","state":["exists","up"]},{"osd":1,"uuid":"c8b31f85-a757-43ed-9419-351acc46611b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":38113},{"type":"v1","addr":"172.21.15.49:6810","nonce":38113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":38113},{"type":"v1","addr":"172.21.15.49:6814","nonce":38113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":38113},{"type":"v1","addr":"172.21.15.49:6822","nonce":38113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":38113},{"type":"v1","addr":"172.21.15.49:6818","nonce":38113}]},"public_addr":"172.21.15.49:6810/38113","cluster_addr":"172.21.15.49:6814/38113","heartbeat_back_addr":"172.21.15.49:6822/38113","heartbeat_front_addr":"172.21.15.49:6818/38113","state":["exists","up"]},{"osd":2,"uuid":"555b3bf3-0a70-4b91-8efd-e4204b5fc26e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":38114},{"type":"v1","addr":"172.21.15.49:6801","nonce":38114}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":38114},{"type":"v1","addr":"172.21.15.49:6803","nonce":38114}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":38114},{"type":"v1","addr":"172.21.15.49:6807","nonce":38114}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":38114},{"type":"v1","addr":"172.21.15.49:6805","nonce":38114}]},"public_addr":"172.21.15.49:6801/38114","cluster_addr":"172.21.15.49:6803/38114","heartbeat_back_addr":"172.21.15.49:6807/38114","heartbeat_front_addr":"172.21.15.49:6805/38114","state":["exists","up"]},{"osd":3,"uuid":"8d4e5d77-03ef-4b9b-bfcb-5ae93a46795c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":38115},{"type":"v1","addr":"172.21.15.49:6825","nonce":38115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":38115},{"type":"v1","addr":"172.21.15.49:6827","nonce":38115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":38115},{"type":"v1","addr":"172.21.15.49:6831","nonce":38115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":38115},{"type":"v1","addr":"172.21.15.49:6829","nonce":38115}]},"public_addr":"172.21.15.49:6825/38115","cluster_addr":"172.21.15.49:6827/38115","heartbeat_back_addr":"172.21.15.49:6831/38115","heartbeat_front_addr":"172.21.15.49:6829/38115","state":["exists","up"]},{"osd":4,"uuid":"dba03ad4-68a3-4cf3-9218-ac611f02eacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":38417},{"type":"v1","addr":"172.21.15.150:6809","nonce":38417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":38417},{"type":"v1","addr":"172.21.15.150:6811","nonce":38417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":38417},{"type":"v1","addr":"172.21.15.150:6815","nonce":38417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":38417},{"type":"v1","addr":"172.21.15.150:6813","nonce":38417}]},"public_addr":"172.21.15.150:6809/38417","cluster_addr":"172.21.15.150:6811/38417","heartbeat_back_addr":"172.21.15.150:6815/38417","heartbeat_front_addr":"172.21.15.150:6813/38417","state":["exists","up"]},{"osd":5,"uuid":"0df9c303-c255-4640-8e91-01f1c3efe18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":38418},{"type":"v1","addr":"172.21.15.150:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":38418},{"type":"v1","addr":"172.21.15.150:6803","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":38418},{"type":"v1","addr":"172.21.15.150:6807","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":38418},{"type":"v1","addr":"172.21.15.150:6805","nonce":38418}]},"public_addr":"172.21.15.150:6801/38418","cluster_addr":"172.21.15.150:6803/38418","heartbeat_back_addr":"172.21.15.150:6807/38418","heartbeat_front_addr":"172.21.15.150:6805/38418","state":["exists","up"]},{"osd":6,"uuid":"e59fa189-4b70-4c06-9722-c02b90828bf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":38428},{"type":"v1","addr":"172.21.15.150:6825","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":38428},{"type":"v1","addr":"172.21.15.150:6827","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":38428},{"type":"v1","addr":"172.21.15.150:6831","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":38428},{"type":"v1","addr":"172.21.15.150:6829","nonce":38428}]},"public_addr":"172.21.15.150:6825/38428","cluster_addr":"172.21.15.150:6827/38428","heartbeat_back_addr":"172.21.15.150:6831/38428","heartbeat_front_addr":"172.21.15.150:6829/38428","state":["exists","up"]},{"osd":7,"uuid":"a64f57e1-ef98-46bd-b304-385759e95898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":38421},{"type":"v1","addr":"172.21.15.150:6817","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":38421},{"type":"v1","addr":"172.21.15.150:6819","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":38421},{"type":"v1","addr":"172.21.15.150:6823","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":38421},{"type":"v1","addr":"172.21.15.150:6821","nonce":38421}]},"public_addr":"172.21.15.150:6817/38421","cluster_addr":"172.21.15.150:6819/38421","heartbeat_back_addr":"172.21.15.150:6823/38421","heartbeat_front_addr":"172.21.15.150:6821/38421","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.438552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.413280+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.296466+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.424966+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.276953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.230984+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.410235+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.480084+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T02:08:39.832 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.834+0000 7f7b8cde3640 1 -- 172.21.15.49:0/3464614741 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7b68065a60 msgr2=0x7f7b68067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:39.832 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.834+0000 7f7b8cde3640 1 --2- 172.21.15.49:0/3464614741 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7b68065a60 0x7f7b68067f20 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c0ae000 tx=0x7f7b7c0a7000 comp rx=0 tx=0).stop 2023-11-06T02:08:39.832 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.834+0000 7f7b8cde3640 1 -- 172.21.15.49:0/3464614741 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b880b1f90 msgr2=0x7f7b88158f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:39.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.834+0000 7f7b8cde3640 1 --2- 172.21.15.49:0/3464614741 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b880b1f90 0x7f7b88158f80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7b7800e930 tx=0x7f7b7800edf0 comp rx=0 tx=0).stop 2023-11-06T02:08:39.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.834+0000 7f7b8cde3640 1 -- 172.21.15.49:0/3464614741 shutdown_connections 2023-11-06T02:08:39.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.834+0000 7f7b8cde3640 1 --2- 172.21.15.49:0/3464614741 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7b68065a60 0x7f7b68067f20 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:39.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.834+0000 7f7b8cde3640 1 --2- 172.21.15.49:0/3464614741 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b88161650 0x7f7b880c9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:39.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.834+0000 7f7b8cde3640 1 --2- 172.21.15.49:0/3464614741 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7b880d2190 0x7f7b88153d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:39.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.834+0000 7f7b8cde3640 1 --2- 172.21.15.49:0/3464614741 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b880b1f90 0x7f7b88158f80 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:39.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.834+0000 7f7b8cde3640 1 -- 172.21.15.49:0/3464614741 >> 172.21.15.49:0/3464614741 conn(0x7f7b880d2aa0 msgr2=0x7f7b88157840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:39.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.834+0000 7f7b8cde3640 1 -- 172.21.15.49:0/3464614741 shutdown_connections 2023-11-06T02:08:39.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.834+0000 7f7b8cde3640 1 -- 172.21.15.49:0/3464614741 wait complete. 2023-11-06T02:08:39.847 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-11-06T02:08:39.848 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T02:08:39.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.970+0000 7f6a47a2b640 1 Processor -- start 2023-11-06T02:08:39.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.970+0000 7f6a47a2b640 1 -- start start 2023-11-06T02:08:39.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.974+0000 7f6a47a2b640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f6a400b0790 0x7f6a400c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:39.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.974+0000 7f6a47a2b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6a400cb010 0x7f6a400c2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:39.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.974+0000 7f6a47a2b640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a400cab80 con 0x7f6a4015ab50 2023-11-06T02:08:39.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.974+0000 7f6a47a2b640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a4015af30 con 0x7f6a400b0790 2023-11-06T02:08:39.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.974+0000 7f6a47a2b640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a400cb5b0 con 0x7f6a400cb010 2023-11-06T02:08:39.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.974+0000 7f6a457c7640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f6a4015ab50 0x7f6a400b0080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:53930/0 (socket says 172.21.15.49:53930) 2023-11-06T02:08:39.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.974+0000 7f6a457c7640 1 -- 172.21.15.49:0/2102217219 learned_addr learned my addr 172.21.15.49:0/2102217219 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:08:39.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.974+0000 7f6a44fc6640 1 --2- 172.21.15.49:0/2102217219 >> v2:172.21.15.150:3300/0 conn(0x7f6a400b0790 0x7f6a400c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:39.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.974+0000 7f6a45fc8640 1 --2- 172.21.15.49:0/2102217219 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6a400cb010 0x7f6a400c2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:39.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.974+0000 7f6a33fff640 1 -- 172.21.15.49:0/2102217219 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4225961278 0 0) 0x7f6a400cab80 con 0x7f6a4015ab50 2023-11-06T02:08:39.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.974+0000 7f6a33fff640 1 -- 172.21.15.49:0/2102217219 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a2c003530 con 0x7f6a4015ab50 2023-11-06T02:08:39.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.974+0000 7f6a33fff640 1 -- 172.21.15.49:0/2102217219 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3514305092 0 0) 0x7f6a2c003530 con 0x7f6a4015ab50 2023-11-06T02:08:39.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.974+0000 7f6a33fff640 1 -- 172.21.15.49:0/2102217219 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6a400cb010 msgr2=0x7f6a400c2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:39.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.974+0000 7f6a33fff640 1 --2- 172.21.15.49:0/2102217219 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6a400cb010 0x7f6a400c2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:39.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.974+0000 7f6a33fff640 1 -- 172.21.15.49:0/2102217219 >> v2:172.21.15.150:3300/0 conn(0x7f6a400b0790 msgr2=0x7f6a400c9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:39.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.974+0000 7f6a33fff640 1 --2- 172.21.15.49:0/2102217219 >> v2:172.21.15.150:3300/0 conn(0x7f6a400b0790 0x7f6a400c9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:39.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.974+0000 7f6a33fff640 1 -- 172.21.15.49:0/2102217219 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a400cb890 con 0x7f6a4015ab50 2023-11-06T02:08:39.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.974+0000 7f6a33fff640 1 -- 172.21.15.49:0/2102217219 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7f6a340050c0 con 0x7f6a4015ab50 2023-11-06T02:08:39.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.974+0000 7f6a33fff640 1 -- 172.21.15.49:0/2102217219 >> v1:172.21.15.49:6789/0 conn(0x7f6a4015ab50 legacy=0x7f6a400b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:39.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.974+0000 7f6a33fff640 1 --2- 172.21.15.49:0/2102217219 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6a400cb010 0x7f6a2c003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:39.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.974+0000 7f6a33fff640 1 --2- 172.21.15.49:0/2102217219 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a2c003f00 0x7f6a2c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:39.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.974+0000 7f6a33fff640 1 --2- 172.21.15.49:0/2102217219 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a2c004d10 0x7f6a2c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:39.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.974+0000 7f6a33fff640 1 -- 172.21.15.49:0/2102217219 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a4015af30 con 0x7f6a2c004d10 2023-11-06T02:08:39.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.974+0000 7f6a33fff640 1 -- 172.21.15.49:0/2102217219 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a400cb5b0 con 0x7f6a2c003f00 2023-11-06T02:08:39.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.974+0000 7f6a33fff640 1 -- 172.21.15.49:0/2102217219 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a2c00b990 con 0x7f6a400cb010 2023-11-06T02:08:39.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.974+0000 7f6a45fc8640 1 --2- 172.21.15.49:0/2102217219 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a2c004d10 0x7f6a2c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:39.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.974+0000 7f6a457c7640 1 --2- 172.21.15.49:0/2102217219 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6a400cb010 0x7f6a2c003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:39.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.974+0000 7f6a44fc6640 1 --2- 172.21.15.49:0/2102217219 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a2c003f00 0x7f6a2c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:39.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.974+0000 7f6a45fc8640 1 -- 172.21.15.49:0/2102217219 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6a400cb010 msgr2=0x7f6a2c003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:39.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.974+0000 7f6a45fc8640 1 --2- 172.21.15.49:0/2102217219 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6a400cb010 0x7f6a2c003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:39.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.974+0000 7f6a45fc8640 1 -- 172.21.15.49:0/2102217219 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a2c003f00 msgr2=0x7f6a2c0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:39.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.974+0000 7f6a45fc8640 1 --2- 172.21.15.49:0/2102217219 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a2c003f00 0x7f6a2c0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:39.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.974+0000 7f6a45fc8640 1 -- 172.21.15.49:0/2102217219 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a2c003530 con 0x7f6a2c004d10 2023-11-06T02:08:39.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.974+0000 7f6a44fc6640 1 --2- 172.21.15.49:0/2102217219 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a2c003f00 0x7f6a2c0047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:08:39.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.978+0000 7f6a45fc8640 1 --2- 172.21.15.49:0/2102217219 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a2c004d10 0x7f6a2c005190 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6a38009ae0 tx=0x7f6a3801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=724e374990e8553d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:39.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.978+0000 7f6a33fff640 1 -- 172.21.15.49:0/2102217219 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a380a5640 con 0x7f6a2c004d10 2023-11-06T02:08:39.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.978+0000 7f6a33fff640 1 -- 172.21.15.49:0/2102217219 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a380a57a0 con 0x7f6a2c004d10 2023-11-06T02:08:39.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.978+0000 7f6a33fff640 1 -- 172.21.15.49:0/2102217219 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a380a5b50 con 0x7f6a2c004d10 2023-11-06T02:08:39.980 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.978+0000 7f6a47a2b640 1 -- 172.21.15.49:0/2102217219 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a2c004d10 msgr2=0x7f6a2c005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:39.980 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.978+0000 7f6a47a2b640 1 --2- 172.21.15.49:0/2102217219 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a2c004d10 0x7f6a2c005190 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6a38009ae0 tx=0x7f6a3801c9f0 comp rx=0 tx=0).stop 2023-11-06T02:08:39.980 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.978+0000 7f6a45fc8640 1 -- 172.21.15.49:0/2102217219 reap_dead start 2023-11-06T02:08:39.980 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.978+0000 7f6a47a2b640 1 -- 172.21.15.49:0/2102217219 shutdown_connections 2023-11-06T02:08:39.980 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.978+0000 7f6a47a2b640 1 -- 172.21.15.49:0/2102217219 >> 172.21.15.49:0/2102217219 conn(0x7f6a4006c0a0 msgr2=0x7f6a4006c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:39.980 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.978+0000 7f6a47a2b640 1 -- 172.21.15.49:0/2102217219 shutdown_connections 2023-11-06T02:08:39.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.978+0000 7f6a47a2b640 1 -- 172.21.15.49:0/2102217219 wait complete. 2023-11-06T02:08:39.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.978+0000 7f6a47a2b640 1 Processor -- start 2023-11-06T02:08:39.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.978+0000 7f6a47a2b640 1 -- start start 2023-11-06T02:08:39.984 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.978+0000 7f6a47a2b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6a4014f7e0 0x7f6a401565a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:39.984 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.978+0000 7f6a47a2b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a400c2610 0x7f6a400c2a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:39.984 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.978+0000 7f6a47a2b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a401e4340 0x7f6a401e47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:39.984 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.978+0000 7f6a47a2b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a400c2fd0 con 0x7f6a400c2610 2023-11-06T02:08:39.984 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.978+0000 7f6a47a2b640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a401539e0 con 0x7f6a401e4340 2023-11-06T02:08:39.985 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.978+0000 7f6a47a2b640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a400c6c00 con 0x7f6a4014f7e0 2023-11-06T02:08:39.985 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.978+0000 7f6a44fc6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a400c2610 0x7f6a400c2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:39.985 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.978+0000 7f6a44fc6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a400c2610 0x7f6a400c2a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52022/0 (socket says 172.21.15.49:52022) 2023-11-06T02:08:39.985 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.978+0000 7f6a44fc6640 1 -- 172.21.15.49:0/3907616567 learned_addr learned my addr 172.21.15.49:0/3907616567 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:39.985 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.978+0000 7f6a457c7640 1 --2- 172.21.15.49:0/3907616567 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6a4014f7e0 0x7f6a401565a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:39.986 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.978+0000 7f6a45fc8640 1 --2- 172.21.15.49:0/3907616567 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a401e4340 0x7f6a401e47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:39.986 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.978+0000 7f6a44fc6640 1 -- 172.21.15.49:0/3907616567 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6a4014f7e0 msgr2=0x7f6a401565a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:39.986 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.978+0000 7f6a44fc6640 1 --2- 172.21.15.49:0/3907616567 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6a4014f7e0 0x7f6a401565a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:39.986 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.978+0000 7f6a44fc6640 1 -- 172.21.15.49:0/3907616567 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a401e4340 msgr2=0x7f6a401e47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:39.986 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.978+0000 7f6a44fc6640 1 --2- 172.21.15.49:0/3907616567 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a401e4340 0x7f6a401e47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:39.987 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.978+0000 7f6a44fc6640 1 -- 172.21.15.49:0/3907616567 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a400c6ee0 con 0x7f6a400c2610 2023-11-06T02:08:39.987 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.978+0000 7f6a44fc6640 1 --2- 172.21.15.49:0/3907616567 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a400c2610 0x7f6a400c2a90 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c007f80 tx=0x7f6a3c00f5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:39.987 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.978+0000 7f6a31ffb640 1 -- 172.21.15.49:0/3907616567 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a3c01a070 con 0x7f6a400c2610 2023-11-06T02:08:39.987 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.978+0000 7f6a31ffb640 1 -- 172.21.15.49:0/3907616567 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a3c0039a0 con 0x7f6a400c2610 2023-11-06T02:08:39.987 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.978+0000 7f6a47a2b640 1 -- 172.21.15.49:0/3907616567 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a400c6830 con 0x7f6a400c2610 2023-11-06T02:08:39.988 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.978+0000 7f6a31ffb640 1 -- 172.21.15.49:0/3907616567 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a3c003d10 con 0x7f6a400c2610 2023-11-06T02:08:39.988 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.978+0000 7f6a47a2b640 1 -- 172.21.15.49:0/3907616567 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a4015ded0 con 0x7f6a400c2610 2023-11-06T02:08:39.988 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.982+0000 7f6a47a2b640 1 -- 172.21.15.49:0/3907616567 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a400b0de0 con 0x7f6a400c2610 2023-11-06T02:08:39.988 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.982+0000 7f6a31ffb640 1 -- 172.21.15.49:0/3907616567 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f6a3c004020 con 0x7f6a400c2610 2023-11-06T02:08:39.988 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.982+0000 7f6a31ffb640 1 --2- 172.21.15.49:0/3907616567 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6a24065a60 0x7f6a24067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:39.989 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.982+0000 7f6a31ffb640 1 -- 172.21.15.49:0/3907616567 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6a3c083e70 con 0x7f6a400c2610 2023-11-06T02:08:39.989 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.982+0000 7f6a457c7640 1 --2- 172.21.15.49:0/3907616567 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6a24065a60 0x7f6a24067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:39.989 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.982+0000 7f6a457c7640 1 --2- 172.21.15.49:0/3907616567 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6a24065a60 0x7f6a24067f20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6a34004990 tx=0x7f6a34009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:39.989 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:39.986+0000 7f6a31ffb640 1 -- 172.21.15.49:0/3907616567 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f6a3c051760 con 0x7f6a400c2610 2023-11-06T02:08:40.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.146+0000 7f6a47a2b640 1 -- 172.21.15.49:0/3907616567 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6a400c7070 con 0x7f6a400c2610 2023-11-06T02:08:40.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.146+0000 7f6a31ffb640 1 -- 172.21.15.49:0/3907616567 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+17811 (secure 0 0 0) 0x7f6a3c054d80 con 0x7f6a400c2610 2023-11-06T02:08:40.143 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:08:40.143 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":23,"fsid":"1a8b035c-6ba4-499a-b876-aea47fd9c0d4","created":"2023-11-06T02:08:06.440857+0000","modified":"2023-11-06T02:08:31.594964+0000","last_up_change":"2023-11-06T02:08:17.982372+0000","last_in_change":"2023-11-06T02:08:08.906631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T02:08:19.252222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T02:08:24.776211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-06T02:08:28.936178+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-06T02:08:29.563468+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ad55af8-0571-4e6a-8722-b5d429b7496d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6809","nonce":38116},{"type":"v1","addr":"172.21.15.49:6811","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6813","nonce":38116},{"type":"v1","addr":"172.21.15.49:6815","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6821","nonce":38116},{"type":"v1","addr":"172.21.15.49:6823","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6817","nonce":38116},{"type":"v1","addr":"172.21.15.49:6819","nonce":38116}]},"public_addr":"172.21.15.49:6811/38116","cluster_addr":"172.21.15.49:6815/38116","heartbeat_back_addr":"172.21.15.49:6823/38116","heartbeat_front_addr":"172.21.15.49:6819/38116","state":["exists","up"]},{"osd":1,"uuid":"c8b31f85-a757-43ed-9419-351acc46611b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":38113},{"type":"v1","addr":"172.21.15.49:6810","nonce":38113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":38113},{"type":"v1","addr":"172.21.15.49:6814","nonce":38113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":38113},{"type":"v1","addr":"172.21.15.49:6822","nonce":38113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":38113},{"type":"v1","addr":"172.21.15.49:6818","nonce":38113}]},"public_addr":"172.21.15.49:6810/38113","cluster_addr":"172.21.15.49:6814/38113","heartbeat_back_addr":"172.21.15.49:6822/38113","heartbeat_front_addr":"172.21.15.49:6818/38113","state":["exists","up"]},{"osd":2,"uuid":"555b3bf3-0a70-4b91-8efd-e4204b5fc26e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":38114},{"type":"v1","addr":"172.21.15.49:6801","nonce":38114}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":38114},{"type":"v1","addr":"172.21.15.49:6803","nonce":38114}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":38114},{"type":"v1","addr":"172.21.15.49:6807","nonce":38114}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":38114},{"type":"v1","addr":"172.21.15.49:6805","nonce":38114}]},"public_addr":"172.21.15.49:6801/38114","cluster_addr":"172.21.15.49:6803/38114","heartbeat_back_addr":"172.21.15.49:6807/38114","heartbeat_front_addr":"172.21.15.49:6805/38114","state":["exists","up"]},{"osd":3,"uuid":"8d4e5d77-03ef-4b9b-bfcb-5ae93a46795c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":38115},{"type":"v1","addr":"172.21.15.49:6825","nonce":38115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":38115},{"type":"v1","addr":"172.21.15.49:6827","nonce":38115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":38115},{"type":"v1","addr":"172.21.15.49:6831","nonce":38115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":38115},{"type":"v1","addr":"172.21.15.49:6829","nonce":38115}]},"public_addr":"172.21.15.49:6825/38115","cluster_addr":"172.21.15.49:6827/38115","heartbeat_back_addr":"172.21.15.49:6831/38115","heartbeat_front_addr":"172.21.15.49:6829/38115","state":["exists","up"]},{"osd":4,"uuid":"dba03ad4-68a3-4cf3-9218-ac611f02eacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":38417},{"type":"v1","addr":"172.21.15.150:6809","nonce":38417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":38417},{"type":"v1","addr":"172.21.15.150:6811","nonce":38417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":38417},{"type":"v1","addr":"172.21.15.150:6815","nonce":38417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":38417},{"type":"v1","addr":"172.21.15.150:6813","nonce":38417}]},"public_addr":"172.21.15.150:6809/38417","cluster_addr":"172.21.15.150:6811/38417","heartbeat_back_addr":"172.21.15.150:6815/38417","heartbeat_front_addr":"172.21.15.150:6813/38417","state":["exists","up"]},{"osd":5,"uuid":"0df9c303-c255-4640-8e91-01f1c3efe18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":38418},{"type":"v1","addr":"172.21.15.150:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":38418},{"type":"v1","addr":"172.21.15.150:6803","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":38418},{"type":"v1","addr":"172.21.15.150:6807","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":38418},{"type":"v1","addr":"172.21.15.150:6805","nonce":38418}]},"public_addr":"172.21.15.150:6801/38418","cluster_addr":"172.21.15.150:6803/38418","heartbeat_back_addr":"172.21.15.150:6807/38418","heartbeat_front_addr":"172.21.15.150:6805/38418","state":["exists","up"]},{"osd":6,"uuid":"e59fa189-4b70-4c06-9722-c02b90828bf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":38428},{"type":"v1","addr":"172.21.15.150:6825","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":38428},{"type":"v1","addr":"172.21.15.150:6827","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":38428},{"type":"v1","addr":"172.21.15.150:6831","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":38428},{"type":"v1","addr":"172.21.15.150:6829","nonce":38428}]},"public_addr":"172.21.15.150:6825/38428","cluster_addr":"172.21.15.150:6827/38428","heartbeat_back_addr":"172.21.15.150:6831/38428","heartbeat_front_addr":"172.21.15.150:6829/38428","state":["exists","up"]},{"osd":7,"uuid":"a64f57e1-ef98-46bd-b304-385759e95898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":38421},{"type":"v1","addr":"172.21.15.150:6817","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":38421},{"type":"v1","addr":"172.21.15.150:6819","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":38421},{"type":"v1","addr":"172.21.15.150:6823","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":38421},{"type":"v1","addr":"172.21.15.150:6821","nonce":38421}]},"public_addr":"172.21.15.150:6817/38421","cluster_addr":"172.21.15.150:6819/38421","heartbeat_back_addr":"172.21.15.150:6823/38421","heartbeat_front_addr":"172.21.15.150:6821/38421","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.438552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.413280+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.296466+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.424966+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.276953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.230984+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.410235+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.480084+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T02:08:40.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.150+0000 7f6a47a2b640 1 -- 172.21.15.49:0/3907616567 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6a24065a60 msgr2=0x7f6a24067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:40.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.150+0000 7f6a47a2b640 1 --2- 172.21.15.49:0/3907616567 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6a24065a60 0x7f6a24067f20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6a34004990 tx=0x7f6a34009040 comp rx=0 tx=0).stop 2023-11-06T02:08:40.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.150+0000 7f6a47a2b640 1 -- 172.21.15.49:0/3907616567 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a400c2610 msgr2=0x7f6a400c2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:40.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.150+0000 7f6a47a2b640 1 --2- 172.21.15.49:0/3907616567 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a400c2610 0x7f6a400c2a90 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c007f80 tx=0x7f6a3c00f5e0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.146 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.150+0000 7f6a47a2b640 1 -- 172.21.15.49:0/3907616567 shutdown_connections 2023-11-06T02:08:40.146 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.150+0000 7f6a47a2b640 1 --2- 172.21.15.49:0/3907616567 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6a24065a60 0x7f6a24067f20 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.146 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.150+0000 7f6a47a2b640 1 --2- 172.21.15.49:0/3907616567 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a400c2610 0x7f6a400c2a90 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.150+0000 7f6a47a2b640 1 --2- 172.21.15.49:0/3907616567 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a401e4340 0x7f6a401e47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.150+0000 7f6a47a2b640 1 --2- 172.21.15.49:0/3907616567 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6a4014f7e0 0x7f6a401565a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.150+0000 7f6a47a2b640 1 -- 172.21.15.49:0/3907616567 >> 172.21.15.49:0/3907616567 conn(0x7f6a4006c0a0 msgr2=0x7f6a400c1530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:40.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.150+0000 7f6a47a2b640 1 -- 172.21.15.49:0/3907616567 shutdown_connections 2023-11-06T02:08:40.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.150+0000 7f6a47a2b640 1 -- 172.21.15.49:0/3907616567 wait complete. 2023-11-06T02:08:40.163 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-11-06T02:08:40.164 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-11-06T02:08:40.164 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-11-06T02:08:40.165 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-11-06T02:08:40.166 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-11-06T02:08:40.166 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-11-06T02:08:40.167 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-11-06T02:08:40.167 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-11-06T02:08:40.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.342+0000 7f5c1ca3c640 1 Processor -- start 2023-11-06T02:08:40.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.342+0000 7f5c1ca3c640 1 -- start start 2023-11-06T02:08:40.344 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.342+0000 7f5c1ca3c640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f5c1806bf20 0x7f5c1806c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.342+0000 7f5c1ca3c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f5c1806e320 0x7f5c180c9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.342+0000 7f5c1ca3c640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c1816ddf0 con 0x7f5c1806de90 2023-11-06T02:08:40.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.342+0000 7f5c1ca3c640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c1806c840 con 0x7f5c1806e320 2023-11-06T02:08:40.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.342+0000 7f5c1ca3c640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c1816d1e0 con 0x7f5c1806bf20 2023-11-06T02:08:40.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.342+0000 7f5c15d9b640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f5c1806de90 0x7f5c180c8a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:53936/0 (socket says 172.21.15.49:53936) 2023-11-06T02:08:40.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.342+0000 7f5c15d9b640 1 -- 172.21.15.49:0/2376104611 learned_addr learned my addr 172.21.15.49:0/2376104611 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:08:40.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.342+0000 7f5c16d9d640 1 --2- 172.21.15.49:0/2376104611 >> v2:172.21.15.49:3300/0 conn(0x7f5c1806e320 0x7f5c180c9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.342+0000 7f5c1659c640 1 --2- 172.21.15.49:0/2376104611 >> v2:172.21.15.150:3300/0 conn(0x7f5c1806bf20 0x7f5c1806c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.346+0000 7f5c16d9d640 1 -- 172.21.15.49:0/2376104611 >> v2:172.21.15.150:3300/0 conn(0x7f5c1806bf20 msgr2=0x7f5c1806c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:40.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.346+0000 7f5c16d9d640 1 --2- 172.21.15.49:0/2376104611 >> v2:172.21.15.150:3300/0 conn(0x7f5c1806bf20 0x7f5c1806c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.346+0000 7f5c16d9d640 1 -- 172.21.15.49:0/2376104611 >> v1:172.21.15.49:6789/0 conn(0x7f5c1806de90 legacy=0x7f5c180c8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:40.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.346+0000 7f5c16d9d640 1 -- 172.21.15.49:0/2376104611 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c1816d4c0 con 0x7f5c1806e320 2023-11-06T02:08:40.350 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.346+0000 7f5c16d9d640 1 --2- 172.21.15.49:0/2376104611 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c1806e320 0x7f5c180c9180 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5c1806cd20 tx=0x7f5c0c024cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69a760d930f2be58 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:40.350 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.346+0000 7f5c1559a640 1 -- 172.21.15.49:0/2376104611 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c0c0add40 con 0x7f5c1806e320 2023-11-06T02:08:40.350 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.346+0000 7f5c1559a640 1 -- 172.21.15.49:0/2376104611 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c0c0b1070 con 0x7f5c1806e320 2023-11-06T02:08:40.351 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.346+0000 7f5c1559a640 1 -- 172.21.15.49:0/2376104611 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c0c0adea0 con 0x7f5c1806e320 2023-11-06T02:08:40.351 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.346+0000 7f5c1ca3c640 1 -- 172.21.15.49:0/2376104611 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c1806e320 msgr2=0x7f5c180c9180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:40.351 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.346+0000 7f5c1ca3c640 1 --2- 172.21.15.49:0/2376104611 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c1806e320 0x7f5c180c9180 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5c1806cd20 tx=0x7f5c0c024cd0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.352 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.346+0000 7f5c1ca3c640 1 -- 172.21.15.49:0/2376104611 shutdown_connections 2023-11-06T02:08:40.352 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.346+0000 7f5c1ca3c640 1 --2- 172.21.15.49:0/2376104611 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c1806e320 0x7f5c180c9180 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.353 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.346+0000 7f5c1ca3c640 1 --2- 172.21.15.49:0/2376104611 >> v2:172.21.15.150:3300/0 conn(0x7f5c1806bf20 0x7f5c1806c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.353 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.346+0000 7f5c1ca3c640 1 -- 172.21.15.49:0/2376104611 >> 172.21.15.49:0/2376104611 conn(0x7f5c180ce420 msgr2=0x7f5c180ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:40.354 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.346+0000 7f5c1ca3c640 1 -- 172.21.15.49:0/2376104611 shutdown_connections 2023-11-06T02:08:40.354 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.346+0000 7f5c1ca3c640 1 -- 172.21.15.49:0/2376104611 wait complete. 2023-11-06T02:08:40.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.350+0000 7f5c1ca3c640 1 Processor -- start 2023-11-06T02:08:40.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.350+0000 7f5c1ca3c640 1 -- start start 2023-11-06T02:08:40.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.350+0000 7f5c1ca3c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c1806bf20 0x7f5c180c4120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.350+0000 7f5c1ca3c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5c1806de90 0x7f5c180c7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.350+0000 7f5c1ca3c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5c1806e320 0x7f5c18202080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.350+0000 7f5c1ca3c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c18203070 con 0x7f5c1806bf20 2023-11-06T02:08:40.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.350+0000 7f5c1ca3c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c180c7b60 con 0x7f5c1806de90 2023-11-06T02:08:40.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.350+0000 7f5c1ca3c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c18202ab0 con 0x7f5c1806e320 2023-11-06T02:08:40.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.350+0000 7f5c1659c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c1806bf20 0x7f5c180c4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.350+0000 7f5c1659c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c1806bf20 0x7f5c180c4120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52050/0 (socket says 172.21.15.49:52050) 2023-11-06T02:08:40.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.350+0000 7f5c1659c640 1 -- 172.21.15.49:0/3929206299 learned_addr learned my addr 172.21.15.49:0/3929206299 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:40.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.350+0000 7f5c15d9b640 1 --2- 172.21.15.49:0/3929206299 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5c1806de90 0x7f5c180c7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.350+0000 7f5c16d9d640 1 --2- 172.21.15.49:0/3929206299 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5c1806e320 0x7f5c18202080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.350+0000 7f5c1659c640 1 -- 172.21.15.49:0/3929206299 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5c1806e320 msgr2=0x7f5c18202080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:40.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.350+0000 7f5c1659c640 1 --2- 172.21.15.49:0/3929206299 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5c1806e320 0x7f5c18202080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.350+0000 7f5c1659c640 1 -- 172.21.15.49:0/3929206299 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5c1806de90 msgr2=0x7f5c180c7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:40.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.350+0000 7f5c1659c640 1 --2- 172.21.15.49:0/3929206299 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5c1806de90 0x7f5c180c7ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.350+0000 7f5c1659c640 1 -- 172.21.15.49:0/3929206299 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c0c008610 con 0x7f5c1806bf20 2023-11-06T02:08:40.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.350+0000 7f5c1659c640 1 --2- 172.21.15.49:0/3929206299 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c1806bf20 0x7f5c180c4120 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5c0800e9e0 tx=0x7f5c0800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:40.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.354+0000 7f5c06ffd640 1 -- 172.21.15.49:0/3929206299 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c080096b0 con 0x7f5c1806bf20 2023-11-06T02:08:40.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.354+0000 7f5c06ffd640 1 -- 172.21.15.49:0/3929206299 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c0800c9d0 con 0x7f5c1806bf20 2023-11-06T02:08:40.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.354+0000 7f5c1ca3c640 1 -- 172.21.15.49:0/3929206299 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c18202d90 con 0x7f5c1806bf20 2023-11-06T02:08:40.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.354+0000 7f5c06ffd640 1 -- 172.21.15.49:0/3929206299 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c0800cd60 con 0x7f5c1806bf20 2023-11-06T02:08:40.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.354+0000 7f5c1ca3c640 1 -- 172.21.15.49:0/3929206299 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c180c4660 con 0x7f5c1806bf20 2023-11-06T02:08:40.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.354+0000 7f5c1ca3c640 1 -- 172.21.15.49:0/3929206299 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5c181697c0 con 0x7f5c1806bf20 2023-11-06T02:08:40.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.354+0000 7f5c06ffd640 1 -- 172.21.15.49:0/3929206299 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f5c08013470 con 0x7f5c1806bf20 2023-11-06T02:08:40.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.354+0000 7f5c06ffd640 1 --2- 172.21.15.49:0/3929206299 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5bf8065ab0 0x7f5bf8067f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.354+0000 7f5c06ffd640 1 -- 172.21.15.49:0/3929206299 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5c08083450 con 0x7f5c1806bf20 2023-11-06T02:08:40.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.354+0000 7fb7c8c43640 1 Processor -- start 2023-11-06T02:08:40.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.354+0000 7f5c06ffd640 1 --2- 172.21.15.49:0/3929206299 >> [v2:172.21.15.49:6808/38113,v1:172.21.15.49:6810/38113] conn(0x7f5bf806e370 0x7f5bf8070790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.365 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.354+0000 7f5c06ffd640 1 -- 172.21.15.49:0/3929206299 --> [v2:172.21.15.49:6808/38113,v1:172.21.15.49:6810/38113] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5bf8070e40 con 0x7f5bf806e370 2023-11-06T02:08:40.365 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.354+0000 7f5c06ffd640 1 -- 172.21.15.49:0/3929206299 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c08085a30 con 0x7f5c1806bf20 2023-11-06T02:08:40.365 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.354+0000 7f5c15d9b640 1 --2- 172.21.15.49:0/3929206299 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5bf8065ab0 0x7f5bf8067f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.365 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.354+0000 7f5c16d9d640 1 --2- 172.21.15.49:0/3929206299 >> [v2:172.21.15.49:6808/38113,v1:172.21.15.49:6810/38113] conn(0x7f5bf806e370 0x7f5bf8070790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.366 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.354+0000 7fb7c8c43640 1 -- start start 2023-11-06T02:08:40.366 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.354+0000 7f09eecb8640 1 Processor -- start 2023-11-06T02:08:40.367 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.354+0000 7f09eecb8640 1 -- start start 2023-11-06T02:08:40.367 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.354+0000 7f4c3d8d7640 1 Processor -- start 2023-11-06T02:08:40.367 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.354+0000 7f5c16d9d640 1 --2- 172.21.15.49:0/3929206299 >> [v2:172.21.15.49:6808/38113,v1:172.21.15.49:6810/38113] conn(0x7f5bf806e370 0x7f5bf8070790 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:40.368 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.354+0000 7f4c3d8d7640 1 -- start start 2023-11-06T02:08:40.369 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.358+0000 7fb7c8c43640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fb7c40cb670 0x7fb7c4071270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.369 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.358+0000 7fb7c8c43640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7c40cd9d0 0x7fb7c40c8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.369 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.358+0000 7fb7c8c43640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7c40cc530 con 0x7fb7c4158e30 2023-11-06T02:08:40.369 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.358+0000 7fb7c8c43640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7c4159210 con 0x7fb7c40cb670 2023-11-06T02:08:40.370 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.358+0000 7f5c15d9b640 1 --2- 172.21.15.49:0/3929206299 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5bf8065ab0 0x7f5bf8067f70 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5c18166070 tx=0x7f5c100018c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:40.370 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.358+0000 7fb7c8c43640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c4071e10 con 0x7fb7c40cd9d0 2023-11-06T02:08:40.371 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.358+0000 7f5c06ffd640 1 -- 172.21.15.49:0/3929206299 <== osd.1 v2:172.21.15.49:6808/38113 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f5bf8070e40 con 0x7f5bf806e370 2023-11-06T02:08:40.371 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.358+0000 7fb7c37fe640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7fb7c4158e30 0x7fb7c40caf60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.49:43544/0 (socket says 172.21.15.49:43544) 2023-11-06T02:08:40.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.358+0000 7fb7c37fe640 1 -- 172.21.15.49:0/4285155950 learned_addr learned my addr 172.21.15.49:0/4285155950 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:08:40.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.358+0000 7fb7c3fff640 1 --2- 172.21.15.49:0/4285155950 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7c40cd9d0 0x7fb7c40c8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.358+0000 7f09eecb8640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f09e00e33c0 0x7f09e0038200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.358+0000 7f4c3d8d7640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4c380c92b0 0x7f4c380ce550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.358+0000 7f09eecb8640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09e0005ee0 con 0x7f09e0053ce0 2023-11-06T02:08:40.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.358+0000 7f4c3d8d7640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c3815db80 con 0x7f4c380cad40 2023-11-06T02:08:40.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.358+0000 7f4c3d8d7640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c380cb870 con 0x7f4c38158f40 2023-11-06T02:08:40.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.358+0000 7f4c3d8d7640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c380cbb10 con 0x7f4c380c92b0 2023-11-06T02:08:40.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.358+0000 7f09e7fff640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f09e0053ce0 0x7f09e00e2cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:53950/0 (socket says 172.21.15.49:53950) 2023-11-06T02:08:40.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.358+0000 7f09e7fff640 1 -- 172.21.15.49:0/2399613153 learned_addr learned my addr 172.21.15.49:0/2399613153 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:08:40.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.358+0000 7f09eecb8640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09e00e3a70 con 0x7f09e00df2f0 2023-11-06T02:08:40.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.358+0000 7f09eecb8640 1 -- 172.21.15.49:0/2399613153 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09e0006180 con 0x7f09e00e33c0 2023-11-06T02:08:40.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.358+0000 7fb7c3fff640 1 -- 172.21.15.49:0/4285155950 >> v2:172.21.15.150:3300/0 conn(0x7fb7c40cb670 msgr2=0x7fb7c4071270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:40.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.358+0000 7fb7c27fc640 1 -- 172.21.15.49:0/4285155950 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 675767309 0 0) 0x7fb7c40cc530 con 0x7fb7c4158e30 2023-11-06T02:08:40.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.358+0000 7f4c3d0d6640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f4c380cad40 0x7f4c380cb160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:53956/0 (socket says 172.21.15.49:53956) 2023-11-06T02:08:40.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.358+0000 7f4c3d0d6640 1 -- 172.21.15.49:0/1906535407 learned_addr learned my addr 172.21.15.49:0/1906535407 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:08:40.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.358+0000 7f09ed255640 1 --2- 172.21.15.49:0/2399613153 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f09e00e33c0 0x7f09e0038200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.378 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.358+0000 7f4c37fff640 1 --2- 172.21.15.49:0/1906535407 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4c380c92b0 0x7f4c380ce550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.378 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.358+0000 7fb7c2ffd640 1 --2- 172.21.15.49:0/4285155950 >> v2:172.21.15.150:3300/0 conn(0x7fb7c40cb670 0x7fb7c4071270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.378 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.358+0000 7fb7c3fff640 1 --2- 172.21.15.49:0/4285155950 >> v2:172.21.15.150:3300/0 conn(0x7fb7c40cb670 0x7fb7c4071270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.379 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.358+0000 7fb7c3fff640 1 -- 172.21.15.49:0/4285155950 >> v1:172.21.15.150:6789/0 conn(0x7fb7c4158e30 legacy=0x7fb7c40caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:40.379 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.358+0000 7fb7c3fff640 1 -- 172.21.15.49:0/4285155950 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7c4071850 con 0x7fb7c40cd9d0 2023-11-06T02:08:40.379 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.358+0000 7f4c377fe640 1 -- 172.21.15.49:0/1906535407 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2530996171 0 0) 0x7f4c3815db80 con 0x7f4c380cad40 2023-11-06T02:08:40.379 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.358+0000 7f4c377fe640 1 -- 172.21.15.49:0/1906535407 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c28003430 con 0x7f4c380cad40 2023-11-06T02:08:40.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.358+0000 7fb7c2ffd640 1 --2- 172.21.15.49:0/4285155950 >> v2:172.21.15.150:3300/0 conn(0x7fb7c40cb670 0x7fb7c4071270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:08:40.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.358+0000 7f4c377fe640 1 -- 172.21.15.49:0/1906535407 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1357634639 0 0) 0x7f4c28003430 con 0x7f4c380cad40 2023-11-06T02:08:40.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.358+0000 7f4c377fe640 1 -- 172.21.15.49:0/1906535407 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4c380c92b0 msgr2=0x7f4c380ce550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:40.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.358+0000 7f4c377fe640 1 --2- 172.21.15.49:0/1906535407 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4c380c92b0 0x7f4c380ce550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.358+0000 7f4c377fe640 1 -- 172.21.15.49:0/1906535407 >> v1:172.21.15.150:6789/0 conn(0x7f4c38158f40 legacy=0x7f4c380c8ba0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-06T02:08:40.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.358+0000 7f4c377fe640 1 -- 172.21.15.49:0/1906535407 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c3815cdc0 con 0x7f4c380cad40 2023-11-06T02:08:40.382 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.358+0000 7f09ed255640 1 -- 172.21.15.49:0/2399613153 >> v1:172.21.15.150:6789/0 conn(0x7f09e00df2f0 legacy=0x7f09e0007220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:40.382 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.358+0000 7f09ed255640 1 -- 172.21.15.49:0/2399613153 >> v1:172.21.15.49:6789/0 conn(0x7f09e0053ce0 legacy=0x7f09e00e2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:40.382 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.358+0000 7f09ed255640 1 -- 172.21.15.49:0/2399613153 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09e0006460 con 0x7f09e00e33c0 2023-11-06T02:08:40.383 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.358+0000 7f4c377fe640 1 -- 172.21.15.49:0/1906535407 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7f4c30001a00 con 0x7f4c380cad40 2023-11-06T02:08:40.383 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.358+0000 7f4c377fe640 1 -- 172.21.15.49:0/1906535407 >> v1:172.21.15.49:6789/0 conn(0x7f4c380cad40 legacy=0x7f4c380cb160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:40.383 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.358+0000 7f4c377fe640 1 --2- 172.21.15.49:0/1906535407 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4c380c92b0 0x7f4c28004a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.383 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.358+0000 7f4c377fe640 1 --2- 172.21.15.49:0/1906535407 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c28004ff0 0x7f4c28003670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.384 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.358+0000 7f4c377fe640 1 --2- 172.21.15.49:0/1906535407 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c28003cd0 0x7f4c2800fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.384 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.358+0000 7f4c377fe640 1 -- 172.21.15.49:0/1906535407 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c380cbb10 con 0x7f4c28003cd0 2023-11-06T02:08:40.384 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.358+0000 7f4c377fe640 1 -- 172.21.15.49:0/1906535407 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c280043d0 con 0x7f4c28004ff0 2023-11-06T02:08:40.384 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.358+0000 7f4c377fe640 1 -- 172.21.15.49:0/1906535407 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c280100c0 con 0x7f4c380c92b0 2023-11-06T02:08:40.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.358+0000 7fa8f359e640 1 Processor -- start 2023-11-06T02:08:40.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.358+0000 7f4c37fff640 1 --2- 172.21.15.49:0/1906535407 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c28004ff0 0x7f4c28003670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.358+0000 7f4c37fff640 1 -- 172.21.15.49:0/1906535407 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4c380c92b0 msgr2=0x7f4c28004a90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-06T02:08:40.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.358+0000 7f4c37fff640 1 --2- 172.21.15.49:0/1906535407 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4c380c92b0 0x7f4c28004a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.358+0000 7f4c37fff640 1 -- 172.21.15.49:0/1906535407 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c28003cd0 msgr2=0x7f4c2800fb80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-06T02:08:40.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.358+0000 7f4c37fff640 1 --2- 172.21.15.49:0/1906535407 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c28003cd0 0x7f4c2800fb80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.358+0000 7f4c37fff640 1 -- 172.21.15.49:0/1906535407 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c3815cdc0 con 0x7f4c28004ff0 2023-11-06T02:08:40.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.358+0000 7fe1e841c640 1 Processor -- start 2023-11-06T02:08:40.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.358+0000 7fe1e841c640 1 -- start start 2023-11-06T02:08:40.389 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.362+0000 7fe1e841c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe1e00cb580 0x7fe1e00cb960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.389 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.362+0000 7fe1e841c640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fe1e015e440 0x7fe1e015b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.389 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.362+0000 7fe1e59b7640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fe1e00b6100 0x7fe1e00ce420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:53960/0 (socket says 172.21.15.49:53960) 2023-11-06T02:08:40.389 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.362+0000 7fe1e59b7640 1 -- 172.21.15.49:0/3160449476 learned_addr learned my addr 172.21.15.49:0/3160449476 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:08:40.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.362+0000 7fe1e61b8640 1 --2- 172.21.15.49:0/3160449476 >> v2:172.21.15.49:3300/0 conn(0x7fe1e00cb580 0x7fe1e00cb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.362+0000 7fe1e841c640 1 -- 172.21.15.49:0/3160449476 --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1e015f3c0 con 0x7fe1e00b6100 2023-11-06T02:08:40.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.362+0000 7fe1e841c640 1 -- 172.21.15.49:0/3160449476 --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1e015e8d0 con 0x7fe1e00cb580 2023-11-06T02:08:40.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.362+0000 7fe1e841c640 1 -- 172.21.15.49:0/3160449476 --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1e015eb20 con 0x7fe1e015e440 2023-11-06T02:08:40.391 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.362+0000 7fe1e69b9640 1 --2- 172.21.15.49:0/3160449476 >> v2:172.21.15.150:3300/0 conn(0x7fe1e015e440 0x7fe1e015b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.391 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.362+0000 7fe1e51b6640 1 -- 172.21.15.49:0/3160449476 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2319777096 0 0) 0x7fe1e015f3c0 con 0x7fe1e00b6100 2023-11-06T02:08:40.391 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.362+0000 7fe1e51b6640 1 -- 172.21.15.49:0/3160449476 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1cc003530 con 0x7fe1e00b6100 2023-11-06T02:08:40.391 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.362+0000 7fa8f359e640 1 -- start start 2023-11-06T02:08:40.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.362+0000 7fb7c3fff640 1 --2- 172.21.15.49:0/4285155950 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7c40cd9d0 0x7fb7c40c8ba0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb7b4009fd0 tx=0x7fb7b401e690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be4c2f9b7e6525d6 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:40.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.362+0000 7f09ed255640 1 --2- 172.21.15.49:0/2399613153 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f09e00e33c0 0x7f09e0038200 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f09e8000b70 tx=0x7f09e8097570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e44fa7d6c1ff7a9 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:40.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.362+0000 7fb7c27fc640 1 -- 172.21.15.49:0/4285155950 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b40a8660 con 0x7fb7c40cd9d0 2023-11-06T02:08:40.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.362+0000 7efc61e62640 1 Processor -- start 2023-11-06T02:08:40.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.362+0000 7fe1e69b9640 1 -- 172.21.15.49:0/3160449476 >> v2:172.21.15.49:3300/0 conn(0x7fe1e00cb580 msgr2=0x7fe1e00cb960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:40.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.362+0000 7fb7c27fc640 1 -- 172.21.15.49:0/4285155950 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7b40a87c0 con 0x7fb7c40cd9d0 2023-11-06T02:08:40.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.362+0000 7fb7c27fc640 1 -- 172.21.15.49:0/4285155950 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b40a8b50 con 0x7fb7c40cd9d0 2023-11-06T02:08:40.395 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.362+0000 7f09e77fe640 1 -- 172.21.15.49:0/2399613153 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09e8126600 con 0x7f09e00e33c0 2023-11-06T02:08:40.395 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.362+0000 7f09e77fe640 1 -- 172.21.15.49:0/2399613153 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09e8122050 con 0x7f09e00e33c0 2023-11-06T02:08:40.395 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.362+0000 7f09e77fe640 1 -- 172.21.15.49:0/2399613153 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09e8126a60 con 0x7f09e00e33c0 2023-11-06T02:08:40.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.362+0000 7fb7c8c43640 1 -- 172.21.15.49:0/4285155950 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7c40cd9d0 msgr2=0x7fb7c40c8ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:40.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.362+0000 7fb7c8c43640 1 --2- 172.21.15.49:0/4285155950 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7c40cd9d0 0x7fb7c40c8ba0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb7b4009fd0 tx=0x7fb7b401e690 comp rx=0 tx=0).stop 2023-11-06T02:08:40.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.362+0000 7fe1e51b6640 1 -- 172.21.15.49:0/3160449476 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 287301914 0 0) 0x7fe1cc003530 con 0x7fe1e00b6100 2023-11-06T02:08:40.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.362+0000 7fa8f359e640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fa8f40cd9d0 0x7fa8f4071270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.362+0000 7fa8f359e640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8f40cc530 con 0x7fa8f40caf60 2023-11-06T02:08:40.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.362+0000 7fa8f359e640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8f4071850 con 0x7fa8f40b0c20 2023-11-06T02:08:40.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.362+0000 7fa8f359e640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8f415f930 con 0x7fa8f40cd9d0 2023-11-06T02:08:40.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.362+0000 7f09eecb8640 1 -- 172.21.15.49:0/2399613153 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f09e00e33c0 msgr2=0x7f09e0038200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:40.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.362+0000 7f09eecb8640 1 --2- 172.21.15.49:0/2399613153 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f09e00e33c0 0x7f09e0038200 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f09e8000b70 tx=0x7f09e8097570 comp rx=0 tx=0).stop 2023-11-06T02:08:40.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.362+0000 7fa8f2d9d640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fa8f40caf60 0x7fa8f40c8ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:53976/0 (socket says 172.21.15.49:53976) 2023-11-06T02:08:40.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.362+0000 7fa8f2d9d640 1 -- 172.21.15.49:0/2650604715 learned_addr learned my addr 172.21.15.49:0/2650604715 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:08:40.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.362+0000 7efc61e62640 1 -- start start 2023-11-06T02:08:40.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.362+0000 7efc61e62640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7efc5c161160 0x7efc5c0ceba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.362+0000 7efc61e62640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7efc5c0cdf30 0x7efc5c09a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.362+0000 7efc61e62640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc5c0b74f0 con 0x7efc5c0cd3c0 2023-11-06T02:08:40.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.362+0000 7efc61e62640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc5c0ce440 con 0x7efc5c161160 2023-11-06T02:08:40.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.362+0000 7efc61e62640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc5c0b6a10 con 0x7efc5c0cdf30 2023-11-06T02:08:40.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.362+0000 7fb7c8c43640 1 -- 172.21.15.49:0/4285155950 shutdown_connections 2023-11-06T02:08:40.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.362+0000 7fb7c8c43640 1 --2- 172.21.15.49:0/4285155950 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7c40cd9d0 0x7fb7c40c8ba0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.362+0000 7fb7c8c43640 1 --2- 172.21.15.49:0/4285155950 >> v2:172.21.15.150:3300/0 conn(0x7fb7c40cb670 0x7fb7c4071270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.362+0000 7fb7c8c43640 1 -- 172.21.15.49:0/4285155950 >> 172.21.15.49:0/4285155950 conn(0x7fb7c40b0100 msgr2=0x7fb7c40b0510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:40.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.362+0000 7fe1e69b9640 1 --2- 172.21.15.49:0/3160449476 >> v2:172.21.15.49:3300/0 conn(0x7fe1e00cb580 0x7fe1e00cb960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.362+0000 7fe1e69b9640 1 -- 172.21.15.49:0/3160449476 >> v1:172.21.15.49:6789/0 conn(0x7fe1e00b6100 legacy=0x7fe1e00ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:40.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.362+0000 7fe1e69b9640 1 -- 172.21.15.49:0/3160449476 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1e015ee00 con 0x7fe1e015e440 2023-11-06T02:08:40.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.362+0000 7f09eecb8640 1 -- 172.21.15.49:0/2399613153 shutdown_connections 2023-11-06T02:08:40.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.362+0000 7f09eecb8640 1 --2- 172.21.15.49:0/2399613153 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f09e00e33c0 0x7f09e0038200 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.362+0000 7f09eecb8640 1 -- 172.21.15.49:0/2399613153 >> 172.21.15.49:0/2399613153 conn(0x7f09e0054b10 msgr2=0x7f09e0054f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:40.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.362+0000 7efc5bfff640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7efc5c0cd3c0 0x7efc5c0cd820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.49:43562/0 (socket says 172.21.15.49:43562) 2023-11-06T02:08:40.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.362+0000 7fb7c8c43640 1 -- 172.21.15.49:0/4285155950 shutdown_connections 2023-11-06T02:08:40.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.362+0000 7efc5bfff640 1 -- 172.21.15.49:0/46678005 learned_addr learned my addr 172.21.15.49:0/46678005 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:08:40.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.362+0000 7efc60e60640 1 --2- 172.21.15.49:0/46678005 >> v2:172.21.15.49:3300/0 conn(0x7efc5c161160 0x7efc5c0ceba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.362+0000 7f4c37fff640 1 --2- 172.21.15.49:0/1906535407 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c28004ff0 0x7f4c28003670 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c009ae0 tx=0x7f4c2c01cab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf30ef40ee96d0ce server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:40.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.362+0000 7fa8f1d9b640 1 --2- 172.21.15.49:0/2650604715 >> v2:172.21.15.150:3300/0 conn(0x7fa8f40cd9d0 0x7fa8f4071270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.362+0000 7efc61661640 1 --2- 172.21.15.49:0/46678005 >> v2:172.21.15.150:3300/0 conn(0x7efc5c0cdf30 0x7efc5c09a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.366+0000 7f4c377fe640 1 -- 172.21.15.49:0/1906535407 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c2c0a56a0 con 0x7f4c28004ff0 2023-11-06T02:08:40.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.366+0000 7f09eecb8640 1 -- 172.21.15.49:0/2399613153 shutdown_connections 2023-11-06T02:08:40.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.366+0000 7f4c377fe640 1 -- 172.21.15.49:0/1906535407 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c2c0a5800 con 0x7f4c28004ff0 2023-11-06T02:08:40.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.366+0000 7f4c377fe640 1 -- 172.21.15.49:0/1906535407 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c2c0a5bf0 con 0x7f4c28004ff0 2023-11-06T02:08:40.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.362+0000 7fb7c8c43640 1 -- 172.21.15.49:0/4285155950 wait complete. 2023-11-06T02:08:40.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.366+0000 7efc5b7fe640 1 -- 172.21.15.49:0/46678005 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2381889075 0 0) 0x7efc540014f0 con 0x7efc5c0cd3c0 2023-11-06T02:08:40.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.366+0000 7efc61661640 1 -- 172.21.15.49:0/46678005 >> v2:172.21.15.49:3300/0 conn(0x7efc5c161160 msgr2=0x7efc5c0ceba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:40.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.366+0000 7efc61661640 1 --2- 172.21.15.49:0/46678005 >> v2:172.21.15.49:3300/0 conn(0x7efc5c161160 0x7efc5c0ceba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.366+0000 7efc61661640 1 -- 172.21.15.49:0/46678005 >> v1:172.21.15.150:6789/0 conn(0x7efc5c0cd3c0 legacy=0x7efc5c0cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:40.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.366+0000 7efc61661640 1 -- 172.21.15.49:0/46678005 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc5c0b6cf0 con 0x7efc5c0cdf30 2023-11-06T02:08:40.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.366+0000 7fa8f1d9b640 1 -- 172.21.15.49:0/2650604715 >> v1:172.21.15.150:6789/0 conn(0x7fa8f40b0c20 legacy=0x7fa8f4158bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:40.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.366+0000 7fa8f1d9b640 1 -- 172.21.15.49:0/2650604715 >> v1:172.21.15.49:6789/0 conn(0x7fa8f40caf60 legacy=0x7fa8f40c8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:40.411 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.366+0000 7fa8f1d9b640 1 -- 172.21.15.49:0/2650604715 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8f4071b30 con 0x7fa8f40cd9d0 2023-11-06T02:08:40.411 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.366+0000 7fb7c8c43640 1 Processor -- start 2023-11-06T02:08:40.411 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.366+0000 7fb7c8c43640 1 -- start start 2023-11-06T02:08:40.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.366+0000 7f09eecb8640 1 -- 172.21.15.49:0/2399613153 wait complete. 2023-11-06T02:08:40.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.366+0000 7f09eecb8640 1 Processor -- start 2023-11-06T02:08:40.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.366+0000 7f4c3d8d7640 1 -- 172.21.15.49:0/1906535407 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c28004ff0 msgr2=0x7f4c28003670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:40.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.366+0000 7f4c3d8d7640 1 --2- 172.21.15.49:0/1906535407 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c28004ff0 0x7f4c28003670 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c009ae0 tx=0x7f4c2c01cab0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.366+0000 7f4c3d0d6640 1 -- 172.21.15.49:0/1906535407 reap_dead start 2023-11-06T02:08:40.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.366+0000 7f4c3d8d7640 1 -- 172.21.15.49:0/1906535407 shutdown_connections 2023-11-06T02:08:40.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.366+0000 7f4c3d8d7640 1 -- 172.21.15.49:0/1906535407 >> 172.21.15.49:0/1906535407 conn(0x7f4c3806c050 msgr2=0x7f4c3806c460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:40.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.366+0000 7fe1e69b9640 1 --2- 172.21.15.49:0/3160449476 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe1e015e440 0x7fe1e015b510 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe1e0158a40 tx=0x7fe1d801c770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=654d21b9408f1aa1 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:40.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.366+0000 7f4c3d8d7640 1 -- 172.21.15.49:0/1906535407 shutdown_connections 2023-11-06T02:08:40.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.366+0000 7f4c3d8d7640 1 -- 172.21.15.49:0/1906535407 wait complete. 2023-11-06T02:08:40.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.366+0000 7fb7c8c43640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb7c40cb670 0x7fb7c4070720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.366+0000 7fb7c8c43640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7c40cd9d0 0x7fb7c4070c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.366+0000 7fb7c8c43640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7c4158e30 0x7fb7c4150e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.366+0000 7fb7c37fe640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb7c40cb670 0x7fb7c4070720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.366+0000 7fb7c8c43640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c4146a10 con 0x7fb7c40cd9d0 2023-11-06T02:08:40.417 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.366+0000 7fb7c37fe640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb7c40cb670 0x7fb7c4070720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.49:53422/0 (socket says 172.21.15.49:53422) 2023-11-06T02:08:40.417 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.366+0000 7fb7c8c43640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c4147830 con 0x7fb7c40cb670 2023-11-06T02:08:40.417 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.366+0000 7fb7c37fe640 1 -- 172.21.15.49:0/2075492357 learned_addr learned my addr 172.21.15.49:0/2075492357 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:40.418 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.366+0000 7f09eecb8640 1 -- start start 2023-11-06T02:08:40.418 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.366+0000 7f09eecb8640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09e0053ce0 0x7f09e0035e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.418 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.366+0000 7f09eecb8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09e00df2f0 0x7f09e004fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.418 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.366+0000 7f09eecb8640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f09e00e33c0 0x7f09e004d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.419 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.366+0000 7f09eecb8640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09e004cd90 con 0x7f09e00df2f0 2023-11-06T02:08:40.419 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.366+0000 7f09eecb8640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09e004c180 con 0x7f09e0053ce0 2023-11-06T02:08:40.419 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.366+0000 7f09eecb8640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09e004c460 con 0x7f09e00e33c0 2023-11-06T02:08:40.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.366+0000 7fb7c3fff640 1 --2- 172.21.15.49:0/2075492357 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7c4158e30 0x7fb7c4150e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.366+0000 7fb7c8c43640 1 -- 172.21.15.49:0/2075492357 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c4146c20 con 0x7fb7c4158e30 2023-11-06T02:08:40.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.366+0000 7fe1e51b6640 1 -- 172.21.15.49:0/3160449476 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1d0001680 con 0x7fe1e015e440 2023-11-06T02:08:40.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.366+0000 7fe1e51b6640 1 -- 172.21.15.49:0/3160449476 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1d80a64d0 con 0x7fe1e015e440 2023-11-06T02:08:40.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.366+0000 7fe1e51b6640 1 -- 172.21.15.49:0/3160449476 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1d80a6840 con 0x7fe1e015e440 2023-11-06T02:08:40.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.370+0000 7f09eca54640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09e0053ce0 0x7f09e0035e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.422 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.370+0000 7f09eca54640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09e0053ce0 0x7f09e0035e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.49:53438/0 (socket says 172.21.15.49:53438) 2023-11-06T02:08:40.422 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.370+0000 7f09ed255640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f09e00e33c0 0x7f09e004d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.422 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.370+0000 7f09eca54640 1 -- 172.21.15.49:0/2276228187 learned_addr learned my addr 172.21.15.49:0/2276228187 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:40.422 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.370+0000 7f09e7fff640 1 --2- 172.21.15.49:0/2276228187 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09e00df2f0 0x7f09e004fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.370+0000 7efc61661640 1 --2- 172.21.15.49:0/46678005 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc5c0cdf30 0x7efc5c09a7f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7efc5c073b00 tx=0x7efc5001c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd81065acd903d79 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:40.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.370+0000 7f4c3d8d7640 1 Processor -- start 2023-11-06T02:08:40.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.370+0000 7efc5b7fe640 1 -- 172.21.15.49:0/46678005 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc500a7070 con 0x7efc5c0cdf30 2023-11-06T02:08:40.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.370+0000 7fa8f1d9b640 1 --2- 172.21.15.49:0/2650604715 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa8f40cd9d0 0x7fa8f4071270 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa8e80097c0 tx=0x7fa8e801c1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c05aaf6a81fd8a15 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:40.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.370+0000 7efc5b7fe640 1 -- 172.21.15.49:0/46678005 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc500ab430 con 0x7efc5c0cdf30 2023-11-06T02:08:40.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.370+0000 7efc5b7fe640 1 -- 172.21.15.49:0/46678005 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc500ab7c0 con 0x7efc5c0cdf30 2023-11-06T02:08:40.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.370+0000 7efc61e62640 1 -- 172.21.15.49:0/46678005 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc5c0cdf30 msgr2=0x7efc5c09a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:40.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.370+0000 7efc61e62640 1 --2- 172.21.15.49:0/46678005 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc5c0cdf30 0x7efc5c09a7f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7efc5c073b00 tx=0x7efc5001c450 comp rx=0 tx=0).stop 2023-11-06T02:08:40.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.370+0000 7fe1e841c640 1 -- 172.21.15.49:0/3160449476 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe1e015e440 msgr2=0x7fe1e015b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:40.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.370+0000 7fe1e841c640 1 --2- 172.21.15.49:0/3160449476 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe1e015e440 0x7fe1e015b510 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe1e0158a40 tx=0x7fe1d801c770 comp rx=0 tx=0).stop 2023-11-06T02:08:40.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.370+0000 7f09e7fff640 1 -- 172.21.15.49:0/2276228187 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f09e00e33c0 msgr2=0x7f09e004d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:40.427 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.370+0000 7f09e7fff640 1 --2- 172.21.15.49:0/2276228187 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f09e00e33c0 0x7f09e004d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.427 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.370+0000 7f09e7fff640 1 -- 172.21.15.49:0/2276228187 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09e0053ce0 msgr2=0x7f09e0035e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:40.427 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.370+0000 7f09e7fff640 1 --2- 172.21.15.49:0/2276228187 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09e0053ce0 0x7f09e0035e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.427 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.370+0000 7f09e7fff640 1 -- 172.21.15.49:0/2276228187 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09e804e5c0 con 0x7f09e00df2f0 2023-11-06T02:08:40.427 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.370+0000 7f09e7fff640 1 --2- 172.21.15.49:0/2276228187 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09e00df2f0 0x7f09e004fa40 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f09dc009e00 tx=0x7f09dc00c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:40.428 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.370+0000 7fb7c2ffd640 1 --2- 172.21.15.49:0/2075492357 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7c40cd9d0 0x7fb7c4070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.428 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.370+0000 7fb7c37fe640 1 -- 172.21.15.49:0/2075492357 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7c4158e30 msgr2=0x7fb7c4150e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:40.428 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.370+0000 7fb7c37fe640 1 --2- 172.21.15.49:0/2075492357 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7c4158e30 0x7fb7c4150e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.429 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.370+0000 7fb7c37fe640 1 -- 172.21.15.49:0/2075492357 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7c40cd9d0 msgr2=0x7fb7c4070c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:40.429 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.370+0000 7fb7c37fe640 1 --2- 172.21.15.49:0/2075492357 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7c40cd9d0 0x7fb7c4070c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.429 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.370+0000 7fb7c37fe640 1 -- 172.21.15.49:0/2075492357 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7b4009cb0 con 0x7fb7c40cb670 2023-11-06T02:08:40.429 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.370+0000 7fb7c3fff640 1 --2- 172.21.15.49:0/2075492357 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7c4158e30 0x7fb7c4150e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T02:08:40.430 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.370+0000 7fb7c37fe640 1 --2- 172.21.15.49:0/2075492357 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb7c40cb670 0x7fb7c4070720 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb7b800c480 tx=0x7fb7b800c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:40.430 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.370+0000 7f4c3d8d7640 1 -- start start 2023-11-06T02:08:40.430 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.370+0000 7f4c3d8d7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c28004ff0 0x7f4c38152670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.431 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.370+0000 7fa8f159a640 1 -- 172.21.15.49:0/2650604715 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e80ab600 con 0x7fa8f40cd9d0 2023-11-06T02:08:40.431 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.370+0000 7fa8f159a640 1 -- 172.21.15.49:0/2650604715 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8e80a7050 con 0x7fa8f40cd9d0 2023-11-06T02:08:40.431 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.370+0000 7fa8f159a640 1 -- 172.21.15.49:0/2650604715 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e80ab980 con 0x7fa8f40cd9d0 2023-11-06T02:08:40.432 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.370+0000 7f09e57fa640 1 -- 172.21.15.49:0/2276228187 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09dc0078f0 con 0x7f09e00df2f0 2023-11-06T02:08:40.432 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.370+0000 7f09e57fa640 1 -- 172.21.15.49:0/2276228187 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09dc007a50 con 0x7f09e00df2f0 2023-11-06T02:08:40.432 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.370+0000 7f09e57fa640 1 -- 172.21.15.49:0/2276228187 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09dc007de0 con 0x7f09e00df2f0 2023-11-06T02:08:40.432 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.370+0000 7f09eecb8640 1 -- 172.21.15.49:0/2276228187 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09e004c740 con 0x7f09e00df2f0 2023-11-06T02:08:40.432 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.370+0000 7f09eecb8640 1 -- 172.21.15.49:0/2276228187 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09e004ff80 con 0x7f09e00df2f0 2023-11-06T02:08:40.433 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.370+0000 7fe1e841c640 1 -- 172.21.15.49:0/3160449476 shutdown_connections 2023-11-06T02:08:40.433 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.370+0000 7fe1e841c640 1 --2- 172.21.15.49:0/3160449476 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe1e015e440 0x7fe1e015b510 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.434 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.370+0000 7f4c3c8d5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c28004ff0 0x7f4c38152670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.434 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.370+0000 7f4c3c8d5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c28004ff0 0x7f4c38152670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52088/0 (socket says 172.21.15.49:52088) 2023-11-06T02:08:40.434 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.370+0000 7fe1e841c640 1 --2- 172.21.15.49:0/3160449476 >> v2:172.21.15.49:3300/0 conn(0x7fe1e00cb580 0x7fe1e00cb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.370+0000 7fe1e841c640 1 -- 172.21.15.49:0/3160449476 >> 172.21.15.49:0/3160449476 conn(0x7fe1e00cd2d0 msgr2=0x7fe1e00cd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:40.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.370+0000 7f4c3d8d7640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c380c7ce0 0x7f4c380afa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.370+0000 7f4c3d8d7640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4c3814c4c0 0x7f4c381fe640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.370+0000 7fa8f359e640 1 -- 172.21.15.49:0/2650604715 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa8f40cd9d0 msgr2=0x7fa8f4071270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:40.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.370+0000 7fa8f359e640 1 --2- 172.21.15.49:0/2650604715 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa8f40cd9d0 0x7fa8f4071270 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa8e80097c0 tx=0x7fa8e801c1c0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.370+0000 7fa8f359e640 1 -- 172.21.15.49:0/2650604715 shutdown_connections 2023-11-06T02:08:40.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.370+0000 7fa8f359e640 1 --2- 172.21.15.49:0/2650604715 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa8f40cd9d0 0x7fa8f4071270 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.370+0000 7fa8f359e640 1 -- 172.21.15.49:0/2650604715 >> 172.21.15.49:0/2650604715 conn(0x7fa8f40b0100 msgr2=0x7fa8f40b0510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:40.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.370+0000 7fa8f359e640 1 -- 172.21.15.49:0/2650604715 shutdown_connections 2023-11-06T02:08:40.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.370+0000 7fa8f359e640 1 -- 172.21.15.49:0/2650604715 wait complete. 2023-11-06T02:08:40.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.370+0000 7efc61e62640 1 -- 172.21.15.49:0/46678005 shutdown_connections 2023-11-06T02:08:40.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.370+0000 7efc61e62640 1 --2- 172.21.15.49:0/46678005 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc5c0cdf30 0x7efc5c09a7f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.370+0000 7efc61e62640 1 --2- 172.21.15.49:0/46678005 >> v2:172.21.15.49:3300/0 conn(0x7efc5c161160 0x7efc5c0ceba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.370+0000 7efc61e62640 1 -- 172.21.15.49:0/46678005 >> 172.21.15.49:0/46678005 conn(0x7efc5c0d28b0 msgr2=0x7efc5c0d2cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:40.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.370+0000 7efc61e62640 1 -- 172.21.15.49:0/46678005 shutdown_connections 2023-11-06T02:08:40.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.370+0000 7f09eecb8640 1 -- 172.21.15.49:0/2276228187 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f09e0006a20 con 0x7f09e00df2f0 2023-11-06T02:08:40.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.370+0000 7fb7abfff640 1 -- 172.21.15.49:0/2075492357 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b80074e0 con 0x7fb7c40cb670 2023-11-06T02:08:40.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.370+0000 7fb7abfff640 1 -- 172.21.15.49:0/2075492357 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7b8007660 con 0x7fb7c40cb670 2023-11-06T02:08:40.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.370+0000 7fb7abfff640 1 -- 172.21.15.49:0/2075492357 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b8004a30 con 0x7fb7c40cb670 2023-11-06T02:08:40.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.370+0000 7fb7c8c43640 1 -- 172.21.15.49:0/2075492357 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7c4146f00 con 0x7fb7c40cb670 2023-11-06T02:08:40.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.370+0000 7fb7c8c43640 1 -- 172.21.15.49:0/2075492357 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7c41474c0 con 0x7fb7c40cb670 2023-11-06T02:08:40.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.370+0000 7f4c3d8d7640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c38150350 con 0x7f4c28004ff0 2023-11-06T02:08:40.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.370+0000 7f4c3d8d7640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3814fa40 con 0x7f4c380c7ce0 2023-11-06T02:08:40.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.370+0000 7f4c3d8d7640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c380c8520 con 0x7f4c3814c4c0 2023-11-06T02:08:40.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.370+0000 7f4c3c8d5640 1 -- 172.21.15.49:0/366822995 learned_addr learned my addr 172.21.15.49:0/366822995 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:40.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.370+0000 7fe1e841c640 1 -- 172.21.15.49:0/3160449476 shutdown_connections 2023-11-06T02:08:40.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.370+0000 7f4c3d0d6640 1 --2- 172.21.15.49:0/366822995 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4c3814c4c0 0x7f4c381fe640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.443 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.370+0000 7efc61e62640 1 -- 172.21.15.49:0/46678005 wait complete. 2023-11-06T02:08:40.443 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.374+0000 7f4c3c8d5640 1 -- 172.21.15.49:0/366822995 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4c3814c4c0 msgr2=0x7f4c381fe640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:40.444 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.374+0000 7f4c3c8d5640 1 --2- 172.21.15.49:0/366822995 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4c3814c4c0 0x7f4c381fe640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.444 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.374+0000 7f4c3c8d5640 1 -- 172.21.15.49:0/366822995 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c380c7ce0 msgr2=0x7f4c380afa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:40.444 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.374+0000 7f4c3c8d5640 1 --2- 172.21.15.49:0/366822995 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c380c7ce0 0x7f4c380afa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.444 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.374+0000 7f4c3c8d5640 1 -- 172.21.15.49:0/366822995 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c3814fc90 con 0x7f4c28004ff0 2023-11-06T02:08:40.445 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.370+0000 7fb7c8c43640 1 -- 172.21.15.49:0/2075492357 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb7c40ada60 con 0x7fb7c40cb670 2023-11-06T02:08:40.445 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.374+0000 7f4c3c8d5640 1 --2- 172.21.15.49:0/366822995 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c28004ff0 0x7f4c38152670 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4c2400b410 tx=0x7f4c2400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:40.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.374+0000 7fa8f359e640 1 Processor -- start 2023-11-06T02:08:40.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.374+0000 7fa8f359e640 1 -- start start 2023-11-06T02:08:40.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.374+0000 7fa8f359e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa8f40b0c20 0x7fa8f4151c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.374+0000 7fa8f359e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa8f40caf60 0x7fa8f40b1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.374+0000 7fa8f359e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa8f40cd9d0 0x7fa8f40b22a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.374+0000 7fa8f359e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f41581f0 con 0x7fa8f40cd9d0 2023-11-06T02:08:40.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.374+0000 7fa8f359e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f4157620 con 0x7fa8f40caf60 2023-11-06T02:08:40.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.374+0000 7fa8f359e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f40b27e0 con 0x7fa8f40b0c20 2023-11-06T02:08:40.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.374+0000 7fa8f2d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa8f40cd9d0 0x7fa8f40b22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.374+0000 7fa8f2d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa8f40cd9d0 0x7fa8f40b22a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52100/0 (socket says 172.21.15.49:52100) 2023-11-06T02:08:40.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.374+0000 7fa8f2d9d640 1 -- 172.21.15.49:0/762720457 learned_addr learned my addr 172.21.15.49:0/762720457 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:40.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.374+0000 7efc61e62640 1 Processor -- start 2023-11-06T02:08:40.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.374+0000 7f4c357fa640 1 -- 172.21.15.49:0/366822995 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c24011830 con 0x7f4c28004ff0 2023-11-06T02:08:40.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.374+0000 7f4c357fa640 1 -- 172.21.15.49:0/366822995 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c24011990 con 0x7f4c28004ff0 2023-11-06T02:08:40.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.374+0000 7f4c357fa640 1 -- 172.21.15.49:0/366822995 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c24011dc0 con 0x7f4c28004ff0 2023-11-06T02:08:40.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.374+0000 7fe1e841c640 1 -- 172.21.15.49:0/3160449476 wait complete. 2023-11-06T02:08:40.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.374+0000 7fe1e841c640 1 Processor -- start 2023-11-06T02:08:40.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.374+0000 7f09e57fa640 1 -- 172.21.15.49:0/2276228187 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f09dc009070 con 0x7f09e00df2f0 2023-11-06T02:08:40.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.374+0000 7f09e57fa640 1 --2- 172.21.15.49:0/2276228187 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f09cc065ae0 0x7f09cc067fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.374+0000 7f09e57fa640 1 -- 172.21.15.49:0/2276228187 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f09dc0856c0 con 0x7f09e00df2f0 2023-11-06T02:08:40.451 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.374+0000 7f09e57fa640 1 --2- 172.21.15.49:0/2276228187 >> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] conn(0x7f09cc06e3a0 0x7f09cc0707c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.451 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.374+0000 7f09e57fa640 1 -- 172.21.15.49:0/2276228187 --> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f09cc070e70 con 0x7f09cc06e3a0 2023-11-06T02:08:40.451 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.374+0000 7f09e57fa640 1 -- 172.21.15.49:0/2276228187 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f09dc085b60 con 0x7f09e00df2f0 2023-11-06T02:08:40.451 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.374+0000 7f09eca54640 1 --2- 172.21.15.49:0/2276228187 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f09cc065ae0 0x7f09cc067fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.451 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.374+0000 7f09ed255640 1 --2- 172.21.15.49:0/2276228187 >> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] conn(0x7f09cc06e3a0 0x7f09cc0707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.452 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.374+0000 7fa8f1d9b640 1 --2- 172.21.15.49:0/762720457 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa8f40caf60 0x7fa8f40b1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.452 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.374+0000 7fe1e841c640 1 -- start start 2023-11-06T02:08:40.452 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.374+0000 7fe1e841c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe1e00b6100 0x7fe1e00b4220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.453 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.374+0000 7fe1e841c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1e00cb580 0x7fe1e006b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.453 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.374+0000 7fe1e841c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe1e015e440 0x7fe1e006bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.453 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.374+0000 7fe1e841c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e00b5110 con 0x7fe1e00cb580 2023-11-06T02:08:40.453 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.374+0000 7fe1e841c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e00c7ce0 con 0x7fe1e00b6100 2023-11-06T02:08:40.453 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.374+0000 7fe1e841c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e00c7fc0 con 0x7fe1e015e440 2023-11-06T02:08:40.454 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.374+0000 7fa8f259c640 1 --2- 172.21.15.49:0/762720457 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa8f40b0c20 0x7fa8f4151c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.454 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.374+0000 7fe1e61b8640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe1e00b6100 0x7fe1e00b4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.454 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.374+0000 7fe1e61b8640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe1e00b6100 0x7fe1e00b4220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.49:53466/0 (socket says 172.21.15.49:53466) 2023-11-06T02:08:40.455 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.374+0000 7fe1e61b8640 1 -- 172.21.15.49:0/2705881160 learned_addr learned my addr 172.21.15.49:0/2705881160 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:40.455 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.374+0000 7fb7abfff640 1 -- 172.21.15.49:0/2075492357 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fb7b8004b90 con 0x7fb7c40cb670 2023-11-06T02:08:40.455 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.374+0000 7fb7abfff640 1 --2- 172.21.15.49:0/2075492357 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb7b0065a90 0x7fb7b0067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.455 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.374+0000 7fb7abfff640 1 -- 172.21.15.49:0/2075492357 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb7b8082b70 con 0x7fb7c40cb670 2023-11-06T02:08:40.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.374+0000 7fb7abfff640 1 --2- 172.21.15.49:0/2075492357 >> [v2:172.21.15.150:6824/38428,v1:172.21.15.150:6825/38428] conn(0x7fb7b006e2d0 0x7fb7b00706f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.374+0000 7fb7abfff640 1 -- 172.21.15.49:0/2075492357 --> [v2:172.21.15.150:6824/38428,v1:172.21.15.150:6825/38428] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb7b0070da0 con 0x7fb7b006e2d0 2023-11-06T02:08:40.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.374+0000 7fb7abfff640 1 -- 172.21.15.49:0/2075492357 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb7b8082f20 con 0x7fb7c40cb670 2023-11-06T02:08:40.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.374+0000 7efc61e62640 1 -- start start 2023-11-06T02:08:40.457 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.374+0000 7efc61e62640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efc5c0cd3c0 0x7efc5c0b4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.457 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.374+0000 7efc61e62640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc5c0cdf30 0x7efc5c0b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.457 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.374+0000 7efc61e62640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc5c161160 0x7efc5c0cb100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.457 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.374+0000 7efc61e62640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc5c0c5d50 con 0x7efc5c161160 2023-11-06T02:08:40.458 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.374+0000 7efc61e62640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc5c0cc100 con 0x7efc5c0cdf30 2023-11-06T02:08:40.458 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.374+0000 7efc61e62640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc5c0cc3e0 con 0x7efc5c0cd3c0 2023-11-06T02:08:40.458 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.374+0000 7fa8f2d9d640 1 -- 172.21.15.49:0/762720457 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa8f40b0c20 msgr2=0x7fa8f4151c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:40.458 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.374+0000 7fa8f2d9d640 1 --2- 172.21.15.49:0/762720457 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa8f40b0c20 0x7fa8f4151c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.459 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.374+0000 7fa8f2d9d640 1 -- 172.21.15.49:0/762720457 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa8f40caf60 msgr2=0x7fa8f40b1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:40.459 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.374+0000 7fa8f2d9d640 1 --2- 172.21.15.49:0/762720457 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa8f40caf60 0x7fa8f40b1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.459 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.374+0000 7fa8f2d9d640 1 -- 172.21.15.49:0/762720457 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8e8009420 con 0x7fa8f40cd9d0 2023-11-06T02:08:40.459 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.374+0000 7fa8f259c640 1 --2- 172.21.15.49:0/762720457 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa8f40b0c20 0x7fa8f4151c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:08:40.460 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.374+0000 7fe1e69b9640 1 --2- 172.21.15.49:0/2705881160 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe1e015e440 0x7fe1e006bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.460 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.374+0000 7f4c3d8d7640 1 -- 172.21.15.49:0/366822995 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c3814ff70 con 0x7f4c28004ff0 2023-11-06T02:08:40.460 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.374+0000 7f4c3d8d7640 1 -- 172.21.15.49:0/366822995 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c380c82f0 con 0x7f4c28004ff0 2023-11-06T02:08:40.461 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.374+0000 7efc60e60640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efc5c0cd3c0 0x7efc5c0b4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.461 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.374+0000 7efc61661640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc5c161160 0x7efc5c0cb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.461 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.374+0000 7efc61661640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc5c161160 0x7efc5c0cb100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52126/0 (socket says 172.21.15.49:52126) 2023-11-06T02:08:40.461 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.374+0000 7efc61661640 1 -- 172.21.15.49:0/969359415 learned_addr learned my addr 172.21.15.49:0/969359415 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:40.462 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.374+0000 7fb7c3fff640 1 --2- 172.21.15.49:0/2075492357 >> [v2:172.21.15.150:6824/38428,v1:172.21.15.150:6825/38428] conn(0x7fb7b006e2d0 0x7fb7b00706f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.462 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.374+0000 7fb7c2ffd640 1 --2- 172.21.15.49:0/2075492357 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb7b0065a90 0x7fb7b0067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.463 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.374+0000 7fa8f2d9d640 1 --2- 172.21.15.49:0/762720457 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa8f40cd9d0 0x7fa8f40b22a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec00caf0 tx=0x7fa8ec0020f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:40.463 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.374+0000 7fe1e59b7640 1 --2- 172.21.15.49:0/2705881160 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1e00cb580 0x7fe1e006b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.463 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.374+0000 7efc5bfff640 1 --2- 172.21.15.49:0/969359415 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc5c0cdf30 0x7efc5c0b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.464 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.374+0000 7f09eca54640 1 --2- 172.21.15.49:0/2276228187 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f09cc065ae0 0x7f09cc067fa0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f09d4001a70 tx=0x7f09d4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:40.465 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.374+0000 7efc61661640 1 -- 172.21.15.49:0/969359415 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efc5c0cd3c0 msgr2=0x7efc5c0b4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:40.465 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.374+0000 7efc61661640 1 --2- 172.21.15.49:0/969359415 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efc5c0cd3c0 0x7efc5c0b4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.465 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.374+0000 7efc61661640 1 -- 172.21.15.49:0/969359415 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc5c0cdf30 msgr2=0x7efc5c0b4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:40.465 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.374+0000 7efc61661640 1 --2- 172.21.15.49:0/969359415 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc5c0cdf30 0x7efc5c0b4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.466 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.374+0000 7efc61661640 1 -- 172.21.15.49:0/969359415 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc50009420 con 0x7efc5c161160 2023-11-06T02:08:40.466 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.378+0000 7fe1e69b9640 1 -- 172.21.15.49:0/2705881160 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe1e00b6100 msgr2=0x7fe1e00b4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:40.466 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.378+0000 7fe1e69b9640 1 --2- 172.21.15.49:0/2705881160 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe1e00b6100 0x7fe1e00b4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.467 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.378+0000 7fe1e69b9640 1 -- 172.21.15.49:0/2705881160 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1e00cb580 msgr2=0x7fe1e006b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:40.467 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.378+0000 7fe1e69b9640 1 --2- 172.21.15.49:0/2705881160 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe1e00cb580 0x7fe1e006b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.469 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.378+0000 7fe1e69b9640 1 -- 172.21.15.49:0/2705881160 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1d8009a70 con 0x7fe1e015e440 2023-11-06T02:08:40.470 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.378+0000 7efc61661640 1 --2- 172.21.15.49:0/969359415 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc5c161160 0x7efc5c0cb100 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7efc500ad040 tx=0x7efc5001ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:40.470 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.374+0000 7f4c3d8d7640 1 -- 172.21.15.49:0/366822995 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4c3815d380 con 0x7f4c28004ff0 2023-11-06T02:08:40.471 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.378+0000 7efc597fa640 1 -- 172.21.15.49:0/969359415 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc500b5810 con 0x7efc5c161160 2023-11-06T02:08:40.471 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.378+0000 7efc597fa640 1 -- 172.21.15.49:0/969359415 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc500b5970 con 0x7efc5c161160 2023-11-06T02:08:40.471 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.378+0000 7f09ed255640 1 --2- 172.21.15.49:0/2276228187 >> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] conn(0x7f09cc06e3a0 0x7f09cc0707c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:40.472 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.378+0000 7fb7c3fff640 1 --2- 172.21.15.49:0/2075492357 >> [v2:172.21.15.150:6824/38428,v1:172.21.15.150:6825/38428] conn(0x7fb7b006e2d0 0x7fb7b00706f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:40.472 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.378+0000 7fb7c2ffd640 1 --2- 172.21.15.49:0/2075492357 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb7b0065a90 0x7fb7b0067f50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc0075b0 tx=0x7fb7bc007540 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:40.473 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.378+0000 7fa8e2ffd640 1 -- 172.21.15.49:0/762720457 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec00a760 con 0x7fa8f40cd9d0 2023-11-06T02:08:40.473 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.378+0000 7fa8e2ffd640 1 -- 172.21.15.49:0/762720457 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8ec00a8e0 con 0x7fa8f40cd9d0 2023-11-06T02:08:40.473 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.378+0000 7fa8f359e640 1 -- 172.21.15.49:0/762720457 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8f41578c0 con 0x7fa8f40cd9d0 2023-11-06T02:08:40.474 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.378+0000 7efc597fa640 1 -- 172.21.15.49:0/969359415 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc500b5d00 con 0x7efc5c161160 2023-11-06T02:08:40.474 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.378+0000 7efc61e62640 1 -- 172.21.15.49:0/969359415 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc5c0c75b0 con 0x7efc5c161160 2023-11-06T02:08:40.474 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.378+0000 7efc61e62640 1 -- 172.21.15.49:0/969359415 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc5c1605b0 con 0x7efc5c161160 2023-11-06T02:08:40.474 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.378+0000 7fa8e2ffd640 1 -- 172.21.15.49:0/762720457 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec00ac90 con 0x7fa8f40cd9d0 2023-11-06T02:08:40.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.378+0000 7fa8f359e640 1 -- 172.21.15.49:0/762720457 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8f4157e80 con 0x7fa8f40cd9d0 2023-11-06T02:08:40.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.378+0000 7fe13f59e640 1 Processor -- start 2023-11-06T02:08:40.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.378+0000 7efc61e62640 1 -- 172.21.15.49:0/969359415 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7efc5c0c7430 con 0x7efc5c161160 2023-11-06T02:08:40.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.378+0000 7fe1e69b9640 1 --2- 172.21.15.49:0/2705881160 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe1e015e440 0x7fe1e006bcf0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fe1d80ac000 tx=0x7fe1d8019510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:40.477 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.378+0000 7fe13f59e640 1 -- start start 2023-11-06T02:08:40.477 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.378+0000 7fe13f59e640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fe138035030 0x7fe138048d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.478 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.378+0000 7fe13f59e640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe138052280 0x7fe1380048d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.478 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.378+0000 7fe13f59e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe138004e10 0x7fe1380071d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.478 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.378+0000 7fe13f59e640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1380052c0 con 0x7fe138052280 2023-11-06T02:08:40.478 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.378+0000 7fe13f59e640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe138053b00 con 0x7fe138035030 2023-11-06T02:08:40.479 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.378+0000 7fe13f59e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe138008620 con 0x7fe138004e10 2023-11-06T02:08:40.479 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.378+0000 7fe13e59c640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fe138035030 0x7fe138048d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.479 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.378+0000 7fe13e59c640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fe138035030 0x7fe138048d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.49:53486/0 (socket says 172.21.15.49:53486) 2023-11-06T02:08:40.479 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.378+0000 7fe13e59c640 1 -- 172.21.15.49:0/2778718789 learned_addr learned my addr 172.21.15.49:0/2778718789 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:40.480 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.378+0000 7fa8f359e640 1 -- 172.21.15.49:0/762720457 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa8d8000fd0 con 0x7fa8f40cd9d0 2023-11-06T02:08:40.480 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.378+0000 7fe13e59c640 1 -- 172.21.15.49:0/2778718789 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe138004e10 msgr2=0x7fe1380071d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-06T02:08:40.481 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.378+0000 7fe13e59c640 1 --2- 172.21.15.49:0/2778718789 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe138004e10 0x7fe1380071d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.481 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.378+0000 7fe13e59c640 1 -- 172.21.15.49:0/2778718789 >> v2:172.21.15.49:3300/0 conn(0x7fe138052280 msgr2=0x7fe1380048d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:40.481 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.378+0000 7fe13e59c640 1 --2- 172.21.15.49:0/2778718789 >> v2:172.21.15.49:3300/0 conn(0x7fe138052280 0x7fe1380048d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.481 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.378+0000 7fe13e59c640 1 -- 172.21.15.49:0/2778718789 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe138008900 con 0x7fe138035030 2023-11-06T02:08:40.482 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.378+0000 7fb7abfff640 1 -- 172.21.15.49:0/2075492357 <== osd.6 v2:172.21.15.150:6824/38428 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fb7b0070da0 con 0x7fb7b006e2d0 2023-11-06T02:08:40.482 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.378+0000 7fe1d6ffd640 1 -- 172.21.15.49:0/2705881160 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1d80a6910 con 0x7fe1e015e440 2023-11-06T02:08:40.482 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.378+0000 7fe1e841c640 1 -- 172.21.15.49:0/2705881160 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1e00cecb0 con 0x7fe1e015e440 2023-11-06T02:08:40.483 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.378+0000 7fe1e841c640 1 -- 172.21.15.49:0/2705881160 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1e00c8580 con 0x7fe1e015e440 2023-11-06T02:08:40.483 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.378+0000 7fe1d6ffd640 1 -- 172.21.15.49:0/2705881160 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1d80a6a70 con 0x7fe1e015e440 2023-11-06T02:08:40.483 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.378+0000 7fe1d6ffd640 1 -- 172.21.15.49:0/2705881160 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1d80a6e20 con 0x7fe1e015e440 2023-11-06T02:08:40.483 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.378+0000 7fe1e841c640 1 -- 172.21.15.49:0/2705881160 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe1c8000fd0 con 0x7fe1e015e440 2023-11-06T02:08:40.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.378+0000 7f4c357fa640 1 -- 172.21.15.49:0/366822995 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f4c2401a400 con 0x7f4c28004ff0 2023-11-06T02:08:40.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.378+0000 7f4c357fa640 1 --2- 172.21.15.49:0/366822995 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4c1c065b30 0x7f4c1c067ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.378+0000 7f4c357fa640 1 -- 172.21.15.49:0/366822995 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4c24083990 con 0x7f4c28004ff0 2023-11-06T02:08:40.485 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.378+0000 7f4c357fa640 1 --2- 172.21.15.49:0/366822995 >> [v2:172.21.15.49:6824/38115,v1:172.21.15.49:6825/38115] conn(0x7f4c1c06e370 0x7f4c1c070790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.485 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.378+0000 7f4c357fa640 1 -- 172.21.15.49:0/366822995 --> [v2:172.21.15.49:6824/38115,v1:172.21.15.49:6825/38115] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4c1c070e40 con 0x7f4c1c06e370 2023-11-06T02:08:40.485 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.378+0000 7f4c3d0d6640 1 --2- 172.21.15.49:0/366822995 >> [v2:172.21.15.49:6824/38115,v1:172.21.15.49:6825/38115] conn(0x7f4c1c06e370 0x7f4c1c070790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.485 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.382+0000 7fa8e2ffd640 1 -- 172.21.15.49:0/762720457 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fa8ec0070d0 con 0x7fa8f40cd9d0 2023-11-06T02:08:40.486 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.382+0000 7fa8e2ffd640 1 --2- 172.21.15.49:0/762720457 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa8d4065a60 0x7fa8d4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.486 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.382+0000 7fa8e2ffd640 1 -- 172.21.15.49:0/762720457 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa8ec0831d0 con 0x7fa8f40cd9d0 2023-11-06T02:08:40.486 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.382+0000 7fa8e2ffd640 1 --2- 172.21.15.49:0/762720457 >> [v2:172.21.15.49:6809/38116,v1:172.21.15.49:6811/38116] conn(0x7fa8d406e220 0x7fa8d4070640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.486 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.382+0000 7fa8e2ffd640 1 -- 172.21.15.49:0/762720457 --> [v2:172.21.15.49:6809/38116,v1:172.21.15.49:6811/38116] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa8d4070cf0 con 0x7fa8d406e220 2023-11-06T02:08:40.486 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.382+0000 7fa8f259c640 1 --2- 172.21.15.49:0/762720457 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa8d4065a60 0x7fa8d4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.487 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.382+0000 7fa8f1d9b640 1 --2- 172.21.15.49:0/762720457 >> [v2:172.21.15.49:6809/38116,v1:172.21.15.49:6811/38116] conn(0x7fa8d406e220 0x7fa8d4070640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.487 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.382+0000 7fa8f1d9b640 1 --2- 172.21.15.49:0/762720457 >> [v2:172.21.15.49:6809/38116,v1:172.21.15.49:6811/38116] conn(0x7fa8d406e220 0x7fa8d4070640 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:40.487 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.382+0000 7fa8e2ffd640 1 -- 172.21.15.49:0/762720457 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa8ec001a10 con 0x7fa8f40cd9d0 2023-11-06T02:08:40.487 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.382+0000 7fa8f259c640 1 --2- 172.21.15.49:0/762720457 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa8d4065a60 0x7fa8d4067f20 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa8e4001850 tx=0x7fa8e4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:40.488 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.382+0000 7efc597fa640 1 -- 172.21.15.49:0/969359415 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7efc500c8420 con 0x7efc5c161160 2023-11-06T02:08:40.488 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.382+0000 7efc597fa640 1 --2- 172.21.15.49:0/969359415 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7efc48065a60 0x7efc48067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.488 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.382+0000 7efc597fa640 1 -- 172.21.15.49:0/969359415 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7efc5011dd60 con 0x7efc5c161160 2023-11-06T02:08:40.488 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.382+0000 7efc597fa640 1 --2- 172.21.15.49:0/969359415 >> [v2:172.21.15.49:6800/38114,v1:172.21.15.49:6801/38114] conn(0x7efc4806e2a0 0x7efc480706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.488 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.382+0000 7efc597fa640 1 -- 172.21.15.49:0/969359415 --> [v2:172.21.15.49:6800/38114,v1:172.21.15.49:6801/38114] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efc48070d70 con 0x7efc4806e2a0 2023-11-06T02:08:40.488 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.382+0000 7efc597fa640 1 -- 172.21.15.49:0/969359415 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7efc5011e110 con 0x7efc5c161160 2023-11-06T02:08:40.489 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.382+0000 7efc60e60640 1 --2- 172.21.15.49:0/969359415 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7efc48065a60 0x7efc48067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.489 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.382+0000 7efc5bfff640 1 --2- 172.21.15.49:0/969359415 >> [v2:172.21.15.49:6800/38114,v1:172.21.15.49:6801/38114] conn(0x7efc4806e2a0 0x7efc480706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.489 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.382+0000 7efc5bfff640 1 --2- 172.21.15.49:0/969359415 >> [v2:172.21.15.49:6800/38114,v1:172.21.15.49:6801/38114] conn(0x7efc4806e2a0 0x7efc480706c0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:40.489 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.382+0000 7efc60e60640 1 --2- 172.21.15.49:0/969359415 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7efc48065a60 0x7efc48067f20 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7efc4c006fd0 tx=0x7efc4c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:40.490 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.382+0000 7f4c37fff640 1 --2- 172.21.15.49:0/366822995 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4c1c065b30 0x7f4c1c067ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.490 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.382+0000 7f4c3d0d6640 1 --2- 172.21.15.49:0/366822995 >> [v2:172.21.15.49:6824/38115,v1:172.21.15.49:6825/38115] conn(0x7f4c1c06e370 0x7f4c1c070790 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:40.490 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.382+0000 7f4c357fa640 1 -- 172.21.15.49:0/366822995 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4c24051260 con 0x7f4c28004ff0 2023-11-06T02:08:40.490 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.382+0000 7f4c357fa640 1 -- 172.21.15.49:0/366822995 <== osd.3 v2:172.21.15.49:6824/38115 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f4c1c070e40 con 0x7f4c1c06e370 2023-11-06T02:08:40.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.382+0000 7fe1d6ffd640 1 -- 172.21.15.49:0/2705881160 <== mon.2 v2:172.21.15.150:3301/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fe1d8004040 con 0x7fe1e015e440 2023-11-06T02:08:40.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.382+0000 7fe1d6ffd640 1 --2- 172.21.15.49:0/2705881160 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe1c4065ab0 0x7fe1c4067f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.382+0000 7fe1d6ffd640 1 -- 172.21.15.49:0/2705881160 <== mon.2 v2:172.21.15.150:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe1d80aa070 con 0x7fe1e015e440 2023-11-06T02:08:40.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.382+0000 7fe1d6ffd640 1 --2- 172.21.15.49:0/2705881160 >> [v2:172.21.15.150:6808/38417,v1:172.21.15.150:6809/38417] conn(0x7fe1c406e200 0x7fe1c4070620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.382+0000 7fe1d6ffd640 1 -- 172.21.15.49:0/2705881160 --> [v2:172.21.15.150:6808/38417,v1:172.21.15.150:6809/38417] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe1c4070cd0 con 0x7fe1c406e200 2023-11-06T02:08:40.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.382+0000 7fe1d6ffd640 1 -- 172.21.15.49:0/2705881160 <== mon.2 v2:172.21.15.150:3301/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe1d811ee30 con 0x7fe1e015e440 2023-11-06T02:08:40.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.382+0000 7fe1e59b7640 1 --2- 172.21.15.49:0/2705881160 >> [v2:172.21.15.150:6808/38417,v1:172.21.15.150:6809/38417] conn(0x7fe1c406e200 0x7fe1c4070620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.382+0000 7fe1e61b8640 1 --2- 172.21.15.49:0/2705881160 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe1c4065ab0 0x7fe1c4067f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.493 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.382+0000 7f09e57fa640 1 -- 172.21.15.49:0/2276228187 <== osd.5 v2:172.21.15.150:6800/38418 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f09cc070e70 con 0x7f09cc06e3a0 2023-11-06T02:08:40.493 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.382+0000 7fe13e59c640 1 --2- 172.21.15.49:0/2778718789 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe138035030 0x7fe138048d20 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe1380059b0 tx=0x7fe13001c210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8738be57200b1d3 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:40.494 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.382+0000 7fe1e59b7640 1 --2- 172.21.15.49:0/2705881160 >> [v2:172.21.15.150:6808/38417,v1:172.21.15.150:6809/38417] conn(0x7fe1c406e200 0x7fe1c4070620 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:40.494 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.382+0000 7efc597fa640 1 -- 172.21.15.49:0/969359415 <== osd.2 v2:172.21.15.49:6800/38114 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7efc48070d70 con 0x7efc4806e2a0 2023-11-06T02:08:40.495 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.382+0000 7fe1e61b8640 1 --2- 172.21.15.49:0/2705881160 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe1c4065ab0 0x7fe1c4067f70 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc007b10 tx=0x7fe1dc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:40.495 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.386+0000 7fa8e2ffd640 1 -- 172.21.15.49:0/762720457 <== osd.0 v2:172.21.15.49:6809/38116 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fa8d4070cf0 con 0x7fa8d406e220 2023-11-06T02:08:40.496 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.386+0000 7fe13d59a640 1 -- 172.21.15.49:0/2778718789 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1300ab400 con 0x7fe138035030 2023-11-06T02:08:40.496 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.386+0000 7fe13d59a640 1 -- 172.21.15.49:0/2778718789 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1300a7050 con 0x7fe138035030 2023-11-06T02:08:40.496 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.386+0000 7fe13d59a640 1 -- 172.21.15.49:0/2778718789 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1300ab780 con 0x7fe138035030 2023-11-06T02:08:40.497 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.386+0000 7fe1d6ffd640 1 -- 172.21.15.49:0/2705881160 <== osd.4 v2:172.21.15.150:6808/38417 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fe1c4070cd0 con 0x7fe1c406e200 2023-11-06T02:08:40.497 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.386+0000 7fe13f59e640 1 -- 172.21.15.49:0/2778718789 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe138035030 msgr2=0x7fe138048d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:40.497 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.386+0000 7fe13f59e640 1 --2- 172.21.15.49:0/2778718789 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe138035030 0x7fe138048d20 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe1380059b0 tx=0x7fe13001c210 comp rx=0 tx=0).stop 2023-11-06T02:08:40.498 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.386+0000 7f4c37fff640 1 --2- 172.21.15.49:0/366822995 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4c1c065b30 0x7f4c1c067ff0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c0af000 tx=0x7f4c2c004740 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:40.498 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.390+0000 7f5c1ca3c640 1 -- 172.21.15.49:0/3929206299 --> [v2:172.21.15.49:6808/38113,v1:172.21.15.49:6810/38113] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5c180c7ca0 con 0x7f5bf806e370 2023-11-06T02:08:40.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.390+0000 7f5c06ffd640 1 -- 172.21.15.49:0/3929206299 <== osd.1 v2:172.21.15.49:6808/38113 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f5c0c004040 con 0x7f5bf806e370 2023-11-06T02:08:40.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.390+0000 7f5c1ca3c640 1 -- 172.21.15.49:0/3929206299 >> [v2:172.21.15.49:6808/38113,v1:172.21.15.49:6810/38113] conn(0x7f5bf806e370 msgr2=0x7f5bf8070790 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:40.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.390+0000 7f5c1ca3c640 1 --2- 172.21.15.49:0/3929206299 >> [v2:172.21.15.49:6808/38113,v1:172.21.15.49:6810/38113] conn(0x7f5bf806e370 0x7f5bf8070790 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.390+0000 7f5c1ca3c640 1 -- 172.21.15.49:0/3929206299 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5bf8065ab0 msgr2=0x7f5bf8067f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:40.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.390+0000 7f5c1ca3c640 1 --2- 172.21.15.49:0/3929206299 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5bf8065ab0 0x7f5bf8067f70 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5c18166070 tx=0x7f5c100018c0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.390+0000 7f5c1ca3c640 1 -- 172.21.15.49:0/3929206299 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c1806bf20 msgr2=0x7f5c180c4120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:40.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.390+0000 7f5c1ca3c640 1 --2- 172.21.15.49:0/3929206299 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c1806bf20 0x7f5c180c4120 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5c0800e9e0 tx=0x7f5c0800eea0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.390+0000 7f5c16d9d640 1 -- 172.21.15.49:0/3929206299 reap_dead start 2023-11-06T02:08:40.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.390+0000 7fe13f59e640 1 -- 172.21.15.49:0/2778718789 shutdown_connections 2023-11-06T02:08:40.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.390+0000 7fe13f59e640 1 --2- 172.21.15.49:0/2778718789 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe138004e10 0x7fe1380071d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.390+0000 7fe13f59e640 1 --2- 172.21.15.49:0/2778718789 >> v2:172.21.15.49:3300/0 conn(0x7fe138052280 0x7fe1380048d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.390+0000 7fe13f59e640 1 --2- 172.21.15.49:0/2778718789 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe138035030 0x7fe138048d20 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.390+0000 7fe13f59e640 1 -- 172.21.15.49:0/2778718789 >> 172.21.15.49:0/2778718789 conn(0x7fe138055e00 msgr2=0x7fe138056210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:40.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.390+0000 7fe13f59e640 1 -- 172.21.15.49:0/2778718789 shutdown_connections 2023-11-06T02:08:40.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.390+0000 7f5c1ca3c640 1 -- 172.21.15.49:0/3929206299 shutdown_connections 2023-11-06T02:08:40.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.390+0000 7f5c1ca3c640 1 -- 172.21.15.49:0/3929206299 >> 172.21.15.49:0/3929206299 conn(0x7f5c180ce420 msgr2=0x7f5c1816bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:40.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.390+0000 7f5c1ca3c640 1 -- 172.21.15.49:0/3929206299 shutdown_connections 2023-11-06T02:08:40.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.390+0000 7f5c1ca3c640 1 -- 172.21.15.49:0/3929206299 wait complete. 2023-11-06T02:08:40.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.390+0000 7fe13f59e640 1 -- 172.21.15.49:0/2778718789 wait complete. 2023-11-06T02:08:40.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.390+0000 7fe13f59e640 1 Processor -- start 2023-11-06T02:08:40.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.394+0000 7fe13f59e640 1 -- start start 2023-11-06T02:08:40.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.394+0000 7fe13f59e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe138004e10 0x7fe1380e86a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.394+0000 7fe13f59e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe138035030 0x7fe13818c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.394+0000 7fe13f59e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe138052280 0x7fe13818a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.394+0000 7fe13e59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe138004e10 0x7fe1380e86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.394+0000 7fe13e59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe138004e10 0x7fe1380e86a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52144/0 (socket says 172.21.15.49:52144) 2023-11-06T02:08:40.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.394+0000 7fe13dd9b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe138035030 0x7fe13818c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.394+0000 7fe13e59c640 1 -- 172.21.15.49:0/2496485476 learned_addr learned my addr 172.21.15.49:0/2496485476 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:40.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.394+0000 7fe13f59e640 1 -- 172.21.15.49:0/2496485476 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe138189e50 con 0x7fe138004e10 2023-11-06T02:08:40.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.394+0000 7fe144a67640 1 --2- 172.21.15.49:0/2496485476 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe138052280 0x7fe13818a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.394+0000 7fe13f59e640 1 -- 172.21.15.49:0/2496485476 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1380e8f90 con 0x7fe138052280 2023-11-06T02:08:40.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.394+0000 7fe13f59e640 1 -- 172.21.15.49:0/2496485476 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1380e9270 con 0x7fe138035030 2023-11-06T02:08:40.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.394+0000 7fe13dd9b640 1 -- 172.21.15.49:0/2496485476 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe138052280 msgr2=0x7fe13818a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:40.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.394+0000 7fe13dd9b640 1 --2- 172.21.15.49:0/2496485476 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe138052280 0x7fe13818a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.394+0000 7fe13dd9b640 1 -- 172.21.15.49:0/2496485476 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe138004e10 msgr2=0x7fe1380e86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:40.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.394+0000 7fe13dd9b640 1 --2- 172.21.15.49:0/2496485476 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe138004e10 0x7fe1380e86a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.394+0000 7fe13dd9b640 1 -- 172.21.15.49:0/2496485476 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1300094d0 con 0x7fe138035030 2023-11-06T02:08:40.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.398+0000 7fe13dd9b640 1 --2- 172.21.15.49:0/2496485476 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe138035030 0x7fe13818c040 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe13400b3e0 tx=0x7fe13400b7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:40.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.402+0000 7fe12effd640 1 -- 172.21.15.49:0/2496485476 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe134011830 con 0x7fe138035030 2023-11-06T02:08:40.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.402+0000 7fe13f59e640 1 -- 172.21.15.49:0/2496485476 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe138035590 con 0x7fe138035030 2023-11-06T02:08:40.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.402+0000 7fe13f59e640 1 -- 172.21.15.49:0/2496485476 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe138189ae0 con 0x7fe138035030 2023-11-06T02:08:40.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.402+0000 7fe12effd640 1 -- 172.21.15.49:0/2496485476 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1340119b0 con 0x7fe138035030 2023-11-06T02:08:40.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.402+0000 7fe13f59e640 1 -- 172.21.15.49:0/2496485476 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe138035410 con 0x7fe138035030 2023-11-06T02:08:40.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.402+0000 7fe12effd640 1 -- 172.21.15.49:0/2496485476 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe134011d60 con 0x7fe138035030 2023-11-06T02:08:40.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.402+0000 7fe12effd640 1 -- 172.21.15.49:0/2496485476 <== mon.2 v2:172.21.15.150:3301/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fe13401a400 con 0x7fe138035030 2023-11-06T02:08:40.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.402+0000 7fb7c8c43640 1 -- 172.21.15.49:0/2075492357 --> [v2:172.21.15.150:6824/38428,v1:172.21.15.150:6825/38428] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb7c4074040 con 0x7fb7b006e2d0 2023-11-06T02:08:40.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.402+0000 7f4c3d8d7640 1 -- 172.21.15.49:0/366822995 --> [v2:172.21.15.49:6824/38115,v1:172.21.15.49:6825/38115] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4c3815d0a0 con 0x7f4c1c06e370 2023-11-06T02:08:40.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.402+0000 7fe12effd640 1 --2- 172.21.15.49:0/2496485476 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe120065b60 0x7fe120068020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.402+0000 7fe12effd640 1 -- 172.21.15.49:0/2496485476 <== mon.2 v2:172.21.15.150:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe134083800 con 0x7fe138035030 2023-11-06T02:08:40.514 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.402+0000 7fe12effd640 1 --2- 172.21.15.49:0/2496485476 >> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] conn(0x7fe12006e2b0 0x7fe1200706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.514 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.402+0000 7fe12effd640 1 -- 172.21.15.49:0/2496485476 --> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe120070d80 con 0x7fe12006e2b0 2023-11-06T02:08:40.514 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.402+0000 7fe13e59c640 1 --2- 172.21.15.49:0/2496485476 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe120065b60 0x7fe120068020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.514 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.402+0000 7fe12effd640 1 -- 172.21.15.49:0/2496485476 <== mon.2 v2:172.21.15.150:3301/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe134083bb0 con 0x7fe138035030 2023-11-06T02:08:40.515 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.406+0000 7f4c357fa640 1 -- 172.21.15.49:0/366822995 <== osd.3 v2:172.21.15.49:6824/38115 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f4c30004d70 con 0x7f4c1c06e370 2023-11-06T02:08:40.515 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.406+0000 7fb7abfff640 1 -- 172.21.15.49:0/2075492357 <== osd.6 v2:172.21.15.150:6824/38428 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb7c4074040 con 0x7fb7b006e2d0 2023-11-06T02:08:40.516 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.406+0000 7fe144a67640 1 --2- 172.21.15.49:0/2496485476 >> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] conn(0x7fe12006e2b0 0x7fe1200706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.516 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.406+0000 7fb7c8c43640 1 -- 172.21.15.49:0/2075492357 >> [v2:172.21.15.150:6824/38428,v1:172.21.15.150:6825/38428] conn(0x7fb7b006e2d0 msgr2=0x7fb7b00706f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:40.517 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.406+0000 7fb7c8c43640 1 --2- 172.21.15.49:0/2075492357 >> [v2:172.21.15.150:6824/38428,v1:172.21.15.150:6825/38428] conn(0x7fb7b006e2d0 0x7fb7b00706f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.517 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.406+0000 7fe13e59c640 1 --2- 172.21.15.49:0/2496485476 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe120065b60 0x7fe120068020 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe13001c250 tx=0x7fe1300a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:40.517 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.406+0000 7fe144a67640 1 --2- 172.21.15.49:0/2496485476 >> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] conn(0x7fe12006e2b0 0x7fe1200706d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:40.518 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.406+0000 7fb7c8c43640 1 -- 172.21.15.49:0/2075492357 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb7b0065a90 msgr2=0x7fb7b0067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:40.518 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.406+0000 7fb7c8c43640 1 --2- 172.21.15.49:0/2075492357 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb7b0065a90 0x7fb7b0067f50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc0075b0 tx=0x7fb7bc007540 comp rx=0 tx=0).stop 2023-11-06T02:08:40.518 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.406+0000 7fb7c8c43640 1 -- 172.21.15.49:0/2075492357 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb7c40cb670 msgr2=0x7fb7c4070720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:40.519 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.406+0000 7fb7c8c43640 1 --2- 172.21.15.49:0/2075492357 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb7c40cb670 0x7fb7c4070720 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb7b800c480 tx=0x7fb7b800c940 comp rx=0 tx=0).stop 2023-11-06T02:08:40.519 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.406+0000 7fb7c3fff640 1 -- 172.21.15.49:0/2075492357 reap_dead start 2023-11-06T02:08:40.519 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.406+0000 7fb7c8c43640 1 -- 172.21.15.49:0/2075492357 shutdown_connections 2023-11-06T02:08:40.519 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.406+0000 7fb7c8c43640 1 -- 172.21.15.49:0/2075492357 >> 172.21.15.49:0/2075492357 conn(0x7fb7c40b0100 msgr2=0x7fb7c40caf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:40.519 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.406+0000 7fb7c8c43640 1 -- 172.21.15.49:0/2075492357 shutdown_connections 2023-11-06T02:08:40.520 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.406+0000 7fb7c8c43640 1 -- 172.21.15.49:0/2075492357 wait complete. 2023-11-06T02:08:40.521 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.406+0000 7f4c22ffd640 1 -- 172.21.15.49:0/366822995 >> [v2:172.21.15.49:6824/38115,v1:172.21.15.49:6825/38115] conn(0x7f4c1c06e370 msgr2=0x7f4c1c070790 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:40.521 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.406+0000 7f4c22ffd640 1 --2- 172.21.15.49:0/366822995 >> [v2:172.21.15.49:6824/38115,v1:172.21.15.49:6825/38115] conn(0x7f4c1c06e370 0x7f4c1c070790 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.521 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.406+0000 7f4c22ffd640 1 -- 172.21.15.49:0/366822995 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4c1c065b30 msgr2=0x7f4c1c067ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:40.521 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.406+0000 7f4c22ffd640 1 --2- 172.21.15.49:0/366822995 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4c1c065b30 0x7f4c1c067ff0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c0af000 tx=0x7f4c2c004740 comp rx=0 tx=0).stop 2023-11-06T02:08:40.522 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.406+0000 7f4c22ffd640 1 -- 172.21.15.49:0/366822995 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c28004ff0 msgr2=0x7f4c38152670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:40.522 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.406+0000 7f4c22ffd640 1 --2- 172.21.15.49:0/366822995 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c28004ff0 0x7f4c38152670 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4c2400b410 tx=0x7f4c2400b8d0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.522 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.406+0000 7f4c3d0d6640 1 -- 172.21.15.49:0/366822995 reap_dead start 2023-11-06T02:08:40.522 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.406+0000 7f4c22ffd640 1 -- 172.21.15.49:0/366822995 shutdown_connections 2023-11-06T02:08:40.522 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.406+0000 7f4c22ffd640 1 -- 172.21.15.49:0/366822995 >> 172.21.15.49:0/366822995 conn(0x7f4c3806c050 msgr2=0x7f4c381579d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:40.523 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.406+0000 7fe12effd640 1 -- 172.21.15.49:0/2496485476 <== osd.7 v2:172.21.15.150:6816/38421 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fe120070d80 con 0x7fe12006e2b0 2023-11-06T02:08:40.523 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.406+0000 7f4c22ffd640 1 -- 172.21.15.49:0/366822995 shutdown_connections 2023-11-06T02:08:40.524 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.406+0000 7f4c22ffd640 1 -- 172.21.15.49:0/366822995 wait complete. 2023-11-06T02:08:40.524 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.406+0000 7fa8f359e640 1 -- 172.21.15.49:0/762720457 --> [v2:172.21.15.49:6809/38116,v1:172.21.15.49:6811/38116] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa8d8002cc0 con 0x7fa8d406e220 2023-11-06T02:08:40.524 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.410+0000 7fa8e2ffd640 1 -- 172.21.15.49:0/762720457 <== osd.0 v2:172.21.15.49:6809/38116 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa8d8002cc0 con 0x7fa8d406e220 2023-11-06T02:08:40.525 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.410+0000 7fa8f359e640 1 -- 172.21.15.49:0/762720457 >> [v2:172.21.15.49:6809/38116,v1:172.21.15.49:6811/38116] conn(0x7fa8d406e220 msgr2=0x7fa8d4070640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:40.525 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.410+0000 7fa8f359e640 1 --2- 172.21.15.49:0/762720457 >> [v2:172.21.15.49:6809/38116,v1:172.21.15.49:6811/38116] conn(0x7fa8d406e220 0x7fa8d4070640 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.526 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.410+0000 7fa8f359e640 1 -- 172.21.15.49:0/762720457 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa8d4065a60 msgr2=0x7fa8d4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:40.526 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.410+0000 7fa8f359e640 1 --2- 172.21.15.49:0/762720457 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa8d4065a60 0x7fa8d4067f20 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa8e4001850 tx=0x7fa8e4009040 comp rx=0 tx=0).stop 2023-11-06T02:08:40.526 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.410+0000 7fa8f359e640 1 -- 172.21.15.49:0/762720457 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa8f40cd9d0 msgr2=0x7fa8f40b22a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:40.526 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.410+0000 7fa8f359e640 1 --2- 172.21.15.49:0/762720457 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa8f40cd9d0 0x7fa8f40b22a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec00caf0 tx=0x7fa8ec0020f0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.526 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.410+0000 7fa8f2d9d640 1 -- 172.21.15.49:0/762720457 reap_dead start 2023-11-06T02:08:40.527 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.410+0000 7fa8f359e640 1 -- 172.21.15.49:0/762720457 shutdown_connections 2023-11-06T02:08:40.527 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.410+0000 7fa8f359e640 1 -- 172.21.15.49:0/762720457 >> 172.21.15.49:0/762720457 conn(0x7fa8f40b0100 msgr2=0x7fa8f40af330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:40.527 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.410+0000 7fa8f359e640 1 -- 172.21.15.49:0/762720457 shutdown_connections 2023-11-06T02:08:40.527 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.410+0000 7fa8f359e640 1 -- 172.21.15.49:0/762720457 wait complete. 2023-11-06T02:08:40.528 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.410+0000 7f09eecb8640 1 -- 172.21.15.49:0/2276228187 --> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f09e004c5d0 con 0x7f09cc06e3a0 2023-11-06T02:08:40.528 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.410+0000 7efc61e62640 1 -- 172.21.15.49:0/969359415 --> [v2:172.21.15.49:6800/38114,v1:172.21.15.49:6801/38114] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7efc5c0d17d0 con 0x7efc4806e2a0 2023-11-06T02:08:40.529 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.410+0000 7f09e57fa640 1 -- 172.21.15.49:0/2276228187 <== osd.5 v2:172.21.15.150:6800/38418 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f09e004c5d0 con 0x7f09cc06e3a0 2023-11-06T02:08:40.530 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.410+0000 7efc597fa640 1 -- 172.21.15.49:0/969359415 <== osd.2 v2:172.21.15.49:6800/38114 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7efc54004b70 con 0x7efc4806e2a0 2023-11-06T02:08:40.531 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.414+0000 7f09eecb8640 1 -- 172.21.15.49:0/2276228187 >> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] conn(0x7f09cc06e3a0 msgr2=0x7f09cc0707c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:40.531 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.414+0000 7f09eecb8640 1 --2- 172.21.15.49:0/2276228187 >> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] conn(0x7f09cc06e3a0 0x7f09cc0707c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.531 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.414+0000 7f09eecb8640 1 -- 172.21.15.49:0/2276228187 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f09cc065ae0 msgr2=0x7f09cc067fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:40.532 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.414+0000 7f09eecb8640 1 --2- 172.21.15.49:0/2276228187 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f09cc065ae0 0x7f09cc067fa0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f09d4001a70 tx=0x7f09d4009040 comp rx=0 tx=0).stop 2023-11-06T02:08:40.532 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.414+0000 7f09eecb8640 1 -- 172.21.15.49:0/2276228187 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09e00df2f0 msgr2=0x7f09e004fa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:40.532 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.414+0000 7f09eecb8640 1 --2- 172.21.15.49:0/2276228187 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09e00df2f0 0x7f09e004fa40 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f09dc009e00 tx=0x7f09dc00c430 comp rx=0 tx=0).stop 2023-11-06T02:08:40.532 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.414+0000 7f09ed255640 1 -- 172.21.15.49:0/2276228187 reap_dead start 2023-11-06T02:08:40.532 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.414+0000 7f09eecb8640 1 -- 172.21.15.49:0/2276228187 shutdown_connections 2023-11-06T02:08:40.533 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.414+0000 7f09eecb8640 1 -- 172.21.15.49:0/2276228187 >> 172.21.15.49:0/2276228187 conn(0x7f09e0054b10 msgr2=0x7f09e00365e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:40.533 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.414+0000 7f09eecb8640 1 -- 172.21.15.49:0/2276228187 shutdown_connections 2023-11-06T02:08:40.533 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.414+0000 7f09eecb8640 1 -- 172.21.15.49:0/2276228187 wait complete. 2023-11-06T02:08:40.533 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.414+0000 7fe1e841c640 1 -- 172.21.15.49:0/2705881160 --> [v2:172.21.15.150:6808/38417,v1:172.21.15.150:6809/38417] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe1c8002cc0 con 0x7fe1c406e200 2023-11-06T02:08:40.534 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.414+0000 7efc61e62640 1 -- 172.21.15.49:0/969359415 >> [v2:172.21.15.49:6800/38114,v1:172.21.15.49:6801/38114] conn(0x7efc4806e2a0 msgr2=0x7efc480706c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:40.535 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.414+0000 7efc61e62640 1 --2- 172.21.15.49:0/969359415 >> [v2:172.21.15.49:6800/38114,v1:172.21.15.49:6801/38114] conn(0x7efc4806e2a0 0x7efc480706c0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.535 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.414+0000 7efc61e62640 1 -- 172.21.15.49:0/969359415 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7efc48065a60 msgr2=0x7efc48067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:40.535 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.414+0000 7efc61e62640 1 --2- 172.21.15.49:0/969359415 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7efc48065a60 0x7efc48067f20 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7efc4c006fd0 tx=0x7efc4c008040 comp rx=0 tx=0).stop 2023-11-06T02:08:40.535 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.414+0000 7efc61e62640 1 -- 172.21.15.49:0/969359415 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc5c161160 msgr2=0x7efc5c0cb100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:40.535 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.414+0000 7efc61e62640 1 --2- 172.21.15.49:0/969359415 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc5c161160 0x7efc5c0cb100 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7efc500ad040 tx=0x7efc5001ac90 comp rx=0 tx=0).stop 2023-11-06T02:08:40.536 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.418+0000 7fe1d6ffd640 1 -- 172.21.15.49:0/2705881160 <== osd.4 v2:172.21.15.150:6808/38417 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe1c8002cc0 con 0x7fe1c406e200 2023-11-06T02:08:40.537 INFO:teuthology.orchestra.run.smithi049.stdout:55834574855 2023-11-06T02:08:40.537 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-11-06T02:08:40.538 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.418+0000 7fe1e841c640 1 -- 172.21.15.49:0/2705881160 >> [v2:172.21.15.150:6808/38417,v1:172.21.15.150:6809/38417] conn(0x7fe1c406e200 msgr2=0x7fe1c4070620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:40.538 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.418+0000 7fe1e841c640 1 --2- 172.21.15.49:0/2705881160 >> [v2:172.21.15.150:6808/38417,v1:172.21.15.150:6809/38417] conn(0x7fe1c406e200 0x7fe1c4070620 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.418+0000 7fe1e841c640 1 -- 172.21.15.49:0/2705881160 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe1c4065ab0 msgr2=0x7fe1c4067f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:40.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.418+0000 7fe1e841c640 1 --2- 172.21.15.49:0/2705881160 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe1c4065ab0 0x7fe1c4067f70 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc007b10 tx=0x7fe1dc008040 comp rx=0 tx=0).stop 2023-11-06T02:08:40.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.418+0000 7efc61661640 1 -- 172.21.15.49:0/969359415 reap_dead start 2023-11-06T02:08:40.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.418+0000 7fe1e841c640 1 -- 172.21.15.49:0/2705881160 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe1e015e440 msgr2=0x7fe1e006bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:40.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.418+0000 7fe1e841c640 1 --2- 172.21.15.49:0/2705881160 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe1e015e440 0x7fe1e006bcf0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fe1d80ac000 tx=0x7fe1d8019510 comp rx=0 tx=0).stop 2023-11-06T02:08:40.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.418+0000 7fe1e69b9640 1 -- 172.21.15.49:0/2705881160 reap_dead start 2023-11-06T02:08:40.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.418+0000 7efc61e62640 1 -- 172.21.15.49:0/969359415 shutdown_connections 2023-11-06T02:08:40.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.418+0000 7fe1e841c640 1 -- 172.21.15.49:0/2705881160 shutdown_connections 2023-11-06T02:08:40.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.418+0000 7efc61e62640 1 -- 172.21.15.49:0/969359415 >> 172.21.15.49:0/969359415 conn(0x7efc5c0d28b0 msgr2=0x7efc5c06c720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:40.542 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.418+0000 7fe1e841c640 1 -- 172.21.15.49:0/2705881160 >> 172.21.15.49:0/2705881160 conn(0x7fe1e00cd2d0 msgr2=0x7fe1e00d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:40.542 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.418+0000 7fe1e841c640 1 -- 172.21.15.49:0/2705881160 shutdown_connections 2023-11-06T02:08:40.543 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.418+0000 7efc61e62640 1 -- 172.21.15.49:0/969359415 shutdown_connections 2023-11-06T02:08:40.543 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.418+0000 7efc61e62640 1 -- 172.21.15.49:0/969359415 wait complete. 2023-11-06T02:08:40.543 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.418+0000 7fe1e841c640 1 -- 172.21.15.49:0/2705881160 wait complete. 2023-11-06T02:08:40.544 INFO:teuthology.orchestra.run.smithi049.stdout:55834574854 2023-11-06T02:08:40.545 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-11-06T02:08:40.545 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.438+0000 7fe13f59e640 1 -- 172.21.15.49:0/2496485476 --> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe138049460 con 0x7fe12006e2b0 2023-11-06T02:08:40.546 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.438+0000 7fe12effd640 1 -- 172.21.15.49:0/2496485476 <== osd.7 v2:172.21.15.150:6816/38421 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe14004f040 con 0x7fe12006e2b0 2023-11-06T02:08:40.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.438+0000 7fe13f59e640 1 -- 172.21.15.49:0/2496485476 >> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] conn(0x7fe12006e2b0 msgr2=0x7fe1200706d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:40.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.438+0000 7fe13f59e640 1 --2- 172.21.15.49:0/2496485476 >> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] conn(0x7fe12006e2b0 0x7fe1200706d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.438+0000 7fe13f59e640 1 -- 172.21.15.49:0/2496485476 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe120065b60 msgr2=0x7fe120068020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:40.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.438+0000 7fe13f59e640 1 --2- 172.21.15.49:0/2496485476 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe120065b60 0x7fe120068020 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe13001c250 tx=0x7fe1300a6000 comp rx=0 tx=0).stop 2023-11-06T02:08:40.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.438+0000 7fe13f59e640 1 -- 172.21.15.49:0/2496485476 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe138035030 msgr2=0x7fe13818c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:40.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.438+0000 7fe13f59e640 1 --2- 172.21.15.49:0/2496485476 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe138035030 0x7fe13818c040 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe13400b3e0 tx=0x7fe13400b7e0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.438+0000 7fe144a67640 1 -- 172.21.15.49:0/2496485476 reap_dead start 2023-11-06T02:08:40.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.438+0000 7fe13f59e640 1 -- 172.21.15.49:0/2496485476 shutdown_connections 2023-11-06T02:08:40.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.438+0000 7fe13f59e640 1 -- 172.21.15.49:0/2496485476 >> 172.21.15.49:0/2496485476 conn(0x7fe138055e00 msgr2=0x7fe13803b380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:40.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.438+0000 7fe13f59e640 1 -- 172.21.15.49:0/2496485476 shutdown_connections 2023-11-06T02:08:40.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.438+0000 7fe13f59e640 1 -- 172.21.15.49:0/2496485476 wait complete. 2023-11-06T02:08:40.550 INFO:teuthology.orchestra.run.smithi049.stdout:55834574855 2023-11-06T02:08:40.550 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-11-06T02:08:40.551 INFO:teuthology.orchestra.run.smithi049.stdout:55834574855 2023-11-06T02:08:40.552 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-11-06T02:08:40.552 INFO:teuthology.orchestra.run.smithi049.stdout:55834574854 2023-11-06T02:08:40.553 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-11-06T02:08:40.554 INFO:teuthology.orchestra.run.smithi049.stdout:55834574855 2023-11-06T02:08:40.554 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-11-06T02:08:40.555 INFO:teuthology.orchestra.run.smithi049.stdout:55834574854 2023-11-06T02:08:40.556 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-11-06T02:08:40.557 INFO:teuthology.orchestra.run.smithi049.stdout:55834574855 2023-11-06T02:08:40.557 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-11-06T02:08:40.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.770+0000 7f8b1819d640 1 Processor -- start 2023-11-06T02:08:40.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.770+0000 7f8b1819d640 1 -- start start 2023-11-06T02:08:40.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.770+0000 7fdb159bc640 1 Processor -- start 2023-11-06T02:08:40.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.770+0000 7fdb159bc640 1 -- start start 2023-11-06T02:08:40.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.770+0000 7fdb159bc640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fdb08051b50 0x7fdb08051f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.770+0000 7f8b1819d640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f8b10159b60 0x7f8b10070f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.770+0000 7fdb159bc640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fdb080360d0 0x7fdb0803c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.770+0000 7f8b1819d640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f8b100b40a0 0x7f8b100b44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.770+0000 7fdb159bc640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb08055280 con 0x7fdb08039f60 2023-11-06T02:08:40.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.770+0000 7f8b15f39640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f8b10159b60 0x7f8b10070f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.770+0000 7f8b15f39640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f8b10159b60 0x7f8b10070f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52152/0 (socket says 172.21.15.49:52152) 2023-11-06T02:08:40.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.770+0000 7fdb159bc640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb08052470 con 0x7fdb080360d0 2023-11-06T02:08:40.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.770+0000 7f194c9a2640 1 Processor -- start 2023-11-06T02:08:40.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.770+0000 7fdb159bc640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb08055520 con 0x7fdb08051b50 2023-11-06T02:08:40.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.770+0000 7fdb149ba640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fdb08051b50 0x7fdb08051f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.770+0000 7fdb149ba640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fdb08051b50 0x7fdb08051f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.49:53502/0 (socket says 172.21.15.49:53502) 2023-11-06T02:08:40.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.770+0000 7fdb149ba640 1 -- 172.21.15.49:0/4205750715 learned_addr learned my addr 172.21.15.49:0/4205750715 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:40.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.770+0000 7f8b15f39640 1 -- 172.21.15.49:0/3064428572 learned_addr learned my addr 172.21.15.49:0/3064428572 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:40.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.770+0000 7f8b1673a640 1 --2- 172.21.15.49:0/3064428572 >> v2:172.21.15.150:3300/0 conn(0x7f8b100b40a0 0x7f8b100b44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.770+0000 7f8b1819d640 1 -- 172.21.15.49:0/3064428572 --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b100ce670 con 0x7f8b100ce1e0 2023-11-06T02:08:40.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.770+0000 7f8b1819d640 1 -- 172.21.15.49:0/3064428572 --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b100ae2d0 con 0x7f8b10159b60 2023-11-06T02:08:40.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.770+0000 7f8b1819d640 1 -- 172.21.15.49:0/3064428572 --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b100b4a30 con 0x7f8b100b40a0 2023-11-06T02:08:40.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.770+0000 7f8b14f37640 1 -- 172.21.15.49:0/3064428572 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 377397511 0 0) 0x7f8b100ce670 con 0x7f8b100ce1e0 2023-11-06T02:08:40.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.770+0000 7f8b14f37640 1 -- 172.21.15.49:0/3064428572 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8afc007940 con 0x7f8b100ce1e0 2023-11-06T02:08:40.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.770+0000 7f8b1673a640 1 -- 172.21.15.49:0/3064428572 >> v2:172.21.15.49:3300/0 conn(0x7f8b10159b60 msgr2=0x7f8b10070f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:40.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.770+0000 7f8b1673a640 1 --2- 172.21.15.49:0/3064428572 >> v2:172.21.15.49:3300/0 conn(0x7f8b10159b60 0x7f8b10070f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.770+0000 7f8b1673a640 1 -- 172.21.15.49:0/3064428572 >> v1:172.21.15.150:6789/0 conn(0x7f8b100ce1e0 legacy=0x7f8b1006df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:40.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.770+0000 7f8b1673a640 1 -- 172.21.15.49:0/3064428572 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b1006e770 con 0x7f8b100b40a0 2023-11-06T02:08:40.776 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.770+0000 7f8b15f39640 1 --2- 172.21.15.49:0/3064428572 >> v2:172.21.15.49:3300/0 conn(0x7f8b10159b60 0x7f8b10070f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T02:08:40.776 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.770+0000 7fd8e282e640 1 Processor -- start 2023-11-06T02:08:40.777 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.770+0000 7f194c9a2640 1 -- start start 2023-11-06T02:08:40.777 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.770+0000 7f194c9a2640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f19400df460 0x7f1940007410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.777 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.770+0000 7f194c9a2640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1940050280 0x7f1940054110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.777 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.770+0000 7f194c9a2640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1940006370 con 0x7f1940005ee0 2023-11-06T02:08:40.777 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.770+0000 7f194c9a2640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1940006650 con 0x7f19400df460 2023-11-06T02:08:40.778 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.770+0000 7f194c9a2640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1940006930 con 0x7f1940050280 2023-11-06T02:08:40.778 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.770+0000 7f194659c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f19400df460 0x7f1940007410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.778 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.770+0000 7f194659c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f19400df460 0x7f1940007410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52172/0 (socket says 172.21.15.49:52172) 2023-11-06T02:08:40.778 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.770+0000 7f194659c640 1 -- 172.21.15.49:0/1632210198 learned_addr learned my addr 172.21.15.49:0/1632210198 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:40.779 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.770+0000 7fdb0ffff640 1 --2- 172.21.15.49:0/4205750715 >> v2:172.21.15.49:3300/0 conn(0x7fdb080360d0 0x7fdb0803c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.779 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.770+0000 7fd8e282e640 1 -- start start 2023-11-06T02:08:40.780 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.770+0000 7f93c0813640 1 Processor -- start 2023-11-06T02:08:40.780 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.774+0000 7fdb0f7fe640 1 -- 172.21.15.49:0/4205750715 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 849026625 0 0) 0x7fdb08055280 con 0x7fdb08039f60 2023-11-06T02:08:40.780 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.774+0000 7fdb0f7fe640 1 -- 172.21.15.49:0/4205750715 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb00003530 con 0x7fdb08039f60 2023-11-06T02:08:40.781 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.774+0000 7fd8e282e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd8dc0cb810 0x7fd8dc0ca170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.781 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.774+0000 7fd8e282e640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8dc0ca770 con 0x7fd8dc164850 2023-11-06T02:08:40.782 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.774+0000 7fd8e282e640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8dc0caa50 con 0x7fd8dc0ce320 2023-11-06T02:08:40.782 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.774+0000 7fd8e282e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8dc0c2d60 con 0x7fd8dc0cb810 2023-11-06T02:08:40.782 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.774+0000 7f93c0813640 1 -- start start 2023-11-06T02:08:40.783 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.774+0000 7fdb0ffff640 1 -- 172.21.15.49:0/4205750715 >> v2:172.21.15.150:3300/0 conn(0x7fdb08051b50 msgr2=0x7fdb08051f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:40.783 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.774+0000 7fdb0ffff640 1 --2- 172.21.15.49:0/4205750715 >> v2:172.21.15.150:3300/0 conn(0x7fdb08051b50 0x7fdb08051f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.783 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.774+0000 7fdb0ffff640 1 -- 172.21.15.49:0/4205750715 >> v1:172.21.15.150:6789/0 conn(0x7fdb08039f60 legacy=0x7fdb080562e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:40.783 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.774+0000 7fdb0ffff640 1 -- 172.21.15.49:0/4205750715 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb08055800 con 0x7fdb080360d0 2023-11-06T02:08:40.784 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.774+0000 7fd8e0dcb640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd8dc0cb810 0x7fd8dc0ca170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.784 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.774+0000 7f93c0813640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f93b4055630 0x7f93b4007410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.785 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.774+0000 7f93c0813640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f93b40df460 0x7f93b4054110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.785 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.774+0000 7f93c0813640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93b4006370 con 0x7f93b4005ee0 2023-11-06T02:08:40.785 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.774+0000 7f93c0813640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93b4006650 con 0x7f93b40df460 2023-11-06T02:08:40.785 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.774+0000 7f93c0813640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93b4006930 con 0x7f93b4055630 2023-11-06T02:08:40.786 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.774+0000 7fd8e0dcb640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd8dc0cb810 0x7fd8dc0ca170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.49:33560/0 (socket says 172.21.15.49:33560) 2023-11-06T02:08:40.786 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.774+0000 7fd8e0dcb640 1 -- 172.21.15.49:0/1138642649 learned_addr learned my addr 172.21.15.49:0/1138642649 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:40.786 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.774+0000 7f93bbfff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f93b40df460 0x7f93b4054110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.787 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.774+0000 7fdf64950640 1 Processor -- start 2023-11-06T02:08:40.787 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.774+0000 7fdf64950640 1 -- start start 2023-11-06T02:08:40.787 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.774+0000 7fd8e0dcb640 1 -- 172.21.15.49:0/1138642649 >> v1:172.21.15.150:6789/0 conn(0x7fd8dc0ce320 legacy=0x7fd8dc069d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:40.788 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.774+0000 7fd8e0dcb640 1 -- 172.21.15.49:0/1138642649 >> v1:172.21.15.49:6789/0 conn(0x7fd8dc164850 legacy=0x7fd8dc0cdc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:40.788 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.774+0000 7fd8e0dcb640 1 -- 172.21.15.49:0/1138642649 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8dc0cad30 con 0x7fd8dc0cb810 2023-11-06T02:08:40.788 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.774+0000 7f93baffd640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f93b4055630 0x7f93b4007410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.789 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.774+0000 7f93baffd640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f93b4055630 0x7f93b4007410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.49:33576/0 (socket says 172.21.15.49:33576) 2023-11-06T02:08:40.789 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.774+0000 7f93baffd640 1 -- 172.21.15.49:0/3849703978 learned_addr learned my addr 172.21.15.49:0/3849703978 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:40.789 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.774+0000 7f194659c640 1 -- 172.21.15.49:0/1632210198 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1940050280 msgr2=0x7f1940054110 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-06T02:08:40.789 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.774+0000 7f194659c640 1 --2- 172.21.15.49:0/1632210198 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1940050280 0x7f1940054110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.790 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.774+0000 7f194659c640 1 -- 172.21.15.49:0/1632210198 >> v1:172.21.15.150:6789/0 conn(0x7f1940005ee0 legacy=0x7f194004fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:40.790 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.774+0000 7f194659c640 1 -- 172.21.15.49:0/1632210198 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1940006c10 con 0x7f19400df460 2023-11-06T02:08:40.790 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.774+0000 7f93baffd640 1 -- 172.21.15.49:0/3849703978 >> v2:172.21.15.49:3300/0 conn(0x7f93b40df460 msgr2=0x7f93b4054110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:40.791 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.774+0000 7f8b1673a640 1 --2- 172.21.15.49:0/3064428572 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b100b40a0 0x7f8b100b44f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8b100b05b0 tx=0x7f8b0801cb90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1078ef96258f55e8 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:40.791 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.774+0000 7f93ba7fc640 1 -- 172.21.15.49:0/3849703978 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 775229740 0 0) 0x7f93b4006370 con 0x7f93b4005ee0 2023-11-06T02:08:40.792 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.774+0000 7f8b14f37640 1 -- 172.21.15.49:0/3064428572 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b080a5c40 con 0x7f8b100b40a0 2023-11-06T02:08:40.792 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.774+0000 7f8b14f37640 1 -- 172.21.15.49:0/3064428572 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b080a5da0 con 0x7f8b100b40a0 2023-11-06T02:08:40.792 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.774+0000 7f8b14f37640 1 -- 172.21.15.49:0/3064428572 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b080ad640 con 0x7f8b100b40a0 2023-11-06T02:08:40.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.774+0000 7fdf64950640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fdf60153c30 0x7fdf600adef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.774+0000 7fdf64950640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf600c9060 0x7fdf60070420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.774+0000 7fdf64950640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf60159c10 con 0x7fdf60070960 2023-11-06T02:08:40.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.774+0000 7fdf64950640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf6015ab50 con 0x7fdf60153c30 2023-11-06T02:08:40.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.774+0000 7fdf64950640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf60070df0 con 0x7fdf600c9060 2023-11-06T02:08:40.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.774+0000 7fdf5ed9d640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fdf60153c30 0x7fdf600adef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.774+0000 7f93baffd640 1 --2- 172.21.15.49:0/3849703978 >> v2:172.21.15.49:3300/0 conn(0x7f93b40df460 0x7f93b4054110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.774+0000 7f93baffd640 1 -- 172.21.15.49:0/3849703978 >> v1:172.21.15.49:6789/0 conn(0x7f93b4005ee0 legacy=0x7f93b40e2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:40.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.774+0000 7f93baffd640 1 -- 172.21.15.49:0/3849703978 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93b4006c10 con 0x7f93b4055630 2023-11-06T02:08:40.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.774+0000 7f93bbfff640 1 --2- 172.21.15.49:0/3849703978 >> v2:172.21.15.49:3300/0 conn(0x7f93b40df460 0x7f93b4054110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:08:40.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.774+0000 7fdf5ed9d640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fdf60153c30 0x7fdf600adef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.49:53526/0 (socket says 172.21.15.49:53526) 2023-11-06T02:08:40.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.774+0000 7fdf5ed9d640 1 -- 172.21.15.49:0/4292040471 learned_addr learned my addr 172.21.15.49:0/4292040471 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:40.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.774+0000 7fdb0ffff640 1 --2- 172.21.15.49:0/4205750715 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb080360d0 0x7fdb0803c750 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fdb08051680 tx=0x7fdafc01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d363bc13241aa0d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:40.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.774+0000 7f194659c640 1 --2- 172.21.15.49:0/1632210198 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19400df460 0x7f1940007410 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f19380091e0 tx=0x7f193801d620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca4025a049cf2032 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:40.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.774+0000 7f8b1819d640 1 -- 172.21.15.49:0/3064428572 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b100b40a0 msgr2=0x7f8b100b44f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:40.798 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.774+0000 7f8b1819d640 1 --2- 172.21.15.49:0/3064428572 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b100b40a0 0x7f8b100b44f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8b100b05b0 tx=0x7f8b0801cb90 comp rx=0 tx=0).stop 2023-11-06T02:08:40.798 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.774+0000 7f8b1819d640 1 -- 172.21.15.49:0/3064428572 shutdown_connections 2023-11-06T02:08:40.798 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.774+0000 7f8b1819d640 1 --2- 172.21.15.49:0/3064428572 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b100b40a0 0x7f8b100b44f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.798 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.774+0000 7f8b1819d640 1 --2- 172.21.15.49:0/3064428572 >> v2:172.21.15.49:3300/0 conn(0x7f8b10159b60 0x7f8b10070f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.799 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.774+0000 7fdb0f7fe640 1 -- 172.21.15.49:0/4205750715 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdafc0a7070 con 0x7fdb080360d0 2023-11-06T02:08:40.799 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.778+0000 7f8b1819d640 1 -- 172.21.15.49:0/3064428572 >> 172.21.15.49:0/3064428572 conn(0x7f8b100cb4f0 msgr2=0x7f8b100cb900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:40.800 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.778+0000 7fdb0f7fe640 1 -- 172.21.15.49:0/4205750715 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdafc0ab430 con 0x7fdb080360d0 2023-11-06T02:08:40.800 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.778+0000 7fdb0f7fe640 1 -- 172.21.15.49:0/4205750715 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdafc0ab7c0 con 0x7fdb080360d0 2023-11-06T02:08:40.800 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.778+0000 7f8b1819d640 1 -- 172.21.15.49:0/3064428572 shutdown_connections 2023-11-06T02:08:40.801 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.778+0000 7f8b1819d640 1 -- 172.21.15.49:0/3064428572 wait complete. 2023-11-06T02:08:40.801 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.778+0000 7fdf5e59c640 1 --2- 172.21.15.49:0/4292040471 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf600c9060 0x7fdf60070420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.778+0000 7f194559a640 1 -- 172.21.15.49:0/1632210198 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19380a7660 con 0x7f19400df460 2023-11-06T02:08:40.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.778+0000 7f194559a640 1 -- 172.21.15.49:0/1632210198 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19380a77c0 con 0x7f19400df460 2023-11-06T02:08:40.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.778+0000 7f194559a640 1 -- 172.21.15.49:0/1632210198 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19380a7b50 con 0x7f19400df460 2023-11-06T02:08:40.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.778+0000 7fdf5dd9b640 1 -- 172.21.15.49:0/4292040471 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3247386603 0 0) 0x7fdf60159c10 con 0x7fdf60070960 2023-11-06T02:08:40.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.778+0000 7fdf5dd9b640 1 -- 172.21.15.49:0/4292040471 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf48003530 con 0x7fdf60070960 2023-11-06T02:08:40.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.778+0000 7fdb159bc640 1 -- 172.21.15.49:0/4205750715 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb080360d0 msgr2=0x7fdb0803c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:40.804 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.778+0000 7fdb159bc640 1 --2- 172.21.15.49:0/4205750715 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb080360d0 0x7fdb0803c750 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fdb08051680 tx=0x7fdafc01c450 comp rx=0 tx=0).stop 2023-11-06T02:08:40.804 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.778+0000 7fdf5dd9b640 1 -- 172.21.15.49:0/4292040471 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1866289460 0 0) 0x7fdf48003530 con 0x7fdf60070960 2023-11-06T02:08:40.804 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.778+0000 7fdf5dd9b640 1 -- 172.21.15.49:0/4292040471 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf600c9060 msgr2=0x7fdf60070420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:40.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.778+0000 7fdf5dd9b640 1 --2- 172.21.15.49:0/4292040471 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf600c9060 0x7fdf60070420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.778+0000 7fdb159bc640 1 -- 172.21.15.49:0/4205750715 shutdown_connections 2023-11-06T02:08:40.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.778+0000 7fdb159bc640 1 --2- 172.21.15.49:0/4205750715 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb080360d0 0x7fdb0803c750 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.778+0000 7fdb159bc640 1 --2- 172.21.15.49:0/4205750715 >> v2:172.21.15.150:3300/0 conn(0x7fdb08051b50 0x7fdb08051f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.778+0000 7fdb159bc640 1 -- 172.21.15.49:0/4205750715 >> 172.21.15.49:0/4205750715 conn(0x7fdb080538b0 msgr2=0x7fdb08053cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:40.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.778+0000 7fdf5dd9b640 1 -- 172.21.15.49:0/4292040471 >> v2:172.21.15.150:3300/0 conn(0x7fdf60153c30 msgr2=0x7fdf600adef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:40.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.778+0000 7fdf5dd9b640 1 --2- 172.21.15.49:0/4292040471 >> v2:172.21.15.150:3300/0 conn(0x7fdf60153c30 0x7fdf600adef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.807 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.778+0000 7fdf5dd9b640 1 -- 172.21.15.49:0/4292040471 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf60159ed0 con 0x7fdf60070960 2023-11-06T02:08:40.807 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.778+0000 7fdf5dd9b640 1 -- 172.21.15.49:0/4292040471 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7fdf58001a00 con 0x7fdf60070960 2023-11-06T02:08:40.807 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.778+0000 7fdf5dd9b640 1 -- 172.21.15.49:0/4292040471 >> v1:172.21.15.49:6789/0 conn(0x7fdf60070960 legacy=0x7fdf600c4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:40.807 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.778+0000 7fdf5dd9b640 1 --2- 172.21.15.49:0/4292040471 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdf480039b0 0x7fdf48003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.778+0000 7fdf5dd9b640 1 --2- 172.21.15.49:0/4292040471 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf480047d0 0x7fdf48004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.778+0000 7fdf5dd9b640 1 --2- 172.21.15.49:0/4292040471 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf600c9060 0x7fdf48005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.778+0000 7fdf5dd9b640 1 -- 172.21.15.49:0/4292040471 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf6015ab50 con 0x7fdf480039b0 2023-11-06T02:08:40.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.778+0000 7fdf5dd9b640 1 -- 172.21.15.49:0/4292040471 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf60070df0 con 0x7fdf480047d0 2023-11-06T02:08:40.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.778+0000 7fdf5dd9b640 1 -- 172.21.15.49:0/4292040471 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4800fda0 con 0x7fdf600c9060 2023-11-06T02:08:40.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.778+0000 7fdf5ed9d640 1 --2- 172.21.15.49:0/4292040471 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdf480039b0 0x7fdf48003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.778+0000 7fd8e0dcb640 1 --2- 172.21.15.49:0/1138642649 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd8dc0cb810 0x7fd8dc0ca170 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd8d4009e80 tx=0x7fd8d401c980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0f4be25a40f8e2a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:40.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.778+0000 7f194c9a2640 1 -- 172.21.15.49:0/1632210198 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19400df460 msgr2=0x7f1940007410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:40.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.778+0000 7f194c9a2640 1 --2- 172.21.15.49:0/1632210198 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19400df460 0x7f1940007410 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f19380091e0 tx=0x7f193801d620 comp rx=0 tx=0).stop 2023-11-06T02:08:40.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.778+0000 7f194c9a2640 1 -- 172.21.15.49:0/1632210198 shutdown_connections 2023-11-06T02:08:40.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.778+0000 7f194c9a2640 1 --2- 172.21.15.49:0/1632210198 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1940050280 0x7f1940054110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.778+0000 7f194c9a2640 1 --2- 172.21.15.49:0/1632210198 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19400df460 0x7f1940007410 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.778+0000 7f194c9a2640 1 -- 172.21.15.49:0/1632210198 >> 172.21.15.49:0/1632210198 conn(0x7f1940054b10 msgr2=0x7f1940054f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:40.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.778+0000 7fdf5e59c640 1 --2- 172.21.15.49:0/4292040471 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf480047d0 0x7fdf48004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.778+0000 7fdf5f59e640 1 --2- 172.21.15.49:0/4292040471 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf600c9060 0x7fdf48005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.812 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.778+0000 7fd8daffd640 1 -- 172.21.15.49:0/1138642649 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8d40a5800 con 0x7fd8dc0cb810 2023-11-06T02:08:40.812 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.778+0000 7fdf5ed9d640 1 -- 172.21.15.49:0/4292040471 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf600c9060 msgr2=0x7fdf48005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:40.813 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.778+0000 7fdf5ed9d640 1 --2- 172.21.15.49:0/4292040471 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf600c9060 0x7fdf48005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.813 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.778+0000 7fdf5ed9d640 1 -- 172.21.15.49:0/4292040471 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf480047d0 msgr2=0x7fdf48004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:40.813 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.778+0000 7fdf5ed9d640 1 --2- 172.21.15.49:0/4292040471 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf480047d0 0x7fdf48004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.813 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.778+0000 7fdf5ed9d640 1 -- 172.21.15.49:0/4292040471 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf60159ed0 con 0x7fdf480039b0 2023-11-06T02:08:40.814 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.778+0000 7f8b1819d640 1 Processor -- start 2023-11-06T02:08:40.814 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.778+0000 7f8b1819d640 1 -- start start 2023-11-06T02:08:40.815 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.778+0000 7f93baffd640 1 --2- 172.21.15.49:0/3849703978 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f93b4055630 0x7f93b4007410 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f93a8009870 tx=0x7f93a801d6b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c151b10faf3b73c4 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:40.815 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.778+0000 7f93ba7fc640 1 -- 172.21.15.49:0/3849703978 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a80a76d0 con 0x7f93b4055630 2023-11-06T02:08:40.815 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.778+0000 7f93ba7fc640 1 -- 172.21.15.49:0/3849703978 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93a80a7830 con 0x7f93b4055630 2023-11-06T02:08:40.815 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.778+0000 7f93ba7fc640 1 -- 172.21.15.49:0/3849703978 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a80a7bc0 con 0x7f93b4055630 2023-11-06T02:08:40.816 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.778+0000 7fd8daffd640 1 -- 172.21.15.49:0/1138642649 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8d40a5960 con 0x7fd8dc0cb810 2023-11-06T02:08:40.816 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.778+0000 7fd8daffd640 1 -- 172.21.15.49:0/1138642649 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8d40a5cf0 con 0x7fd8dc0cb810 2023-11-06T02:08:40.816 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.778+0000 7fd8e282e640 1 -- 172.21.15.49:0/1138642649 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd8dc0cb810 msgr2=0x7fd8dc0ca170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:40.816 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.778+0000 7fd8e282e640 1 --2- 172.21.15.49:0/1138642649 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd8dc0cb810 0x7fd8dc0ca170 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd8d4009e80 tx=0x7fd8d401c980 comp rx=0 tx=0).stop 2023-11-06T02:08:40.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.778+0000 7f8b1819d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b100b40a0 0x7f8b1014fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.778+0000 7f8b1819d640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8b100ce1e0 0x7f8b1006c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.778+0000 7f8b1819d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8b10159b60 0x7f8b1006ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.818 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.778+0000 7f8b1819d640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b100c3500 con 0x7f8b10159b60 2023-11-06T02:08:40.818 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.778+0000 7f8b1819d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b10150210 con 0x7f8b100b40a0 2023-11-06T02:08:40.818 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.778+0000 7f8b1819d640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b1015aa00 con 0x7f8b100ce1e0 2023-11-06T02:08:40.819 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.778+0000 7fd8e282e640 1 -- 172.21.15.49:0/1138642649 shutdown_connections 2023-11-06T02:08:40.819 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.778+0000 7fd8e282e640 1 --2- 172.21.15.49:0/1138642649 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd8dc0cb810 0x7fd8dc0ca170 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.819 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.778+0000 7fd8e282e640 1 -- 172.21.15.49:0/1138642649 >> 172.21.15.49:0/1138642649 conn(0x7fd8dc0cc140 msgr2=0x7fd8dc0cc550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:40.819 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.778+0000 7fd8e282e640 1 -- 172.21.15.49:0/1138642649 shutdown_connections 2023-11-06T02:08:40.820 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.778+0000 7f8b15f39640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b100b40a0 0x7f8b1014fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.820 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.778+0000 7f8b15f39640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b100b40a0 0x7f8b1014fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.49:53544/0 (socket says 172.21.15.49:53544) 2023-11-06T02:08:40.820 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.778+0000 7f8b15f39640 1 -- 172.21.15.49:0/1202269021 learned_addr learned my addr 172.21.15.49:0/1202269021 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:40.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.778+0000 7f8b15738640 1 --2- 172.21.15.49:0/1202269021 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8b100ce1e0 0x7f8b1006c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.778+0000 7f8b1673a640 1 --2- 172.21.15.49:0/1202269021 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8b10159b60 0x7f8b1006ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.778+0000 7f93c0813640 1 -- 172.21.15.49:0/3849703978 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f93b4055630 msgr2=0x7f93b4007410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:40.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.778+0000 7f93c0813640 1 --2- 172.21.15.49:0/3849703978 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f93b4055630 0x7f93b4007410 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f93a8009870 tx=0x7f93a801d6b0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.822 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.778+0000 7fdb159bc640 1 -- 172.21.15.49:0/4205750715 shutdown_connections 2023-11-06T02:08:40.822 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.778+0000 7fdb159bc640 1 -- 172.21.15.49:0/4205750715 wait complete. 2023-11-06T02:08:40.823 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.778+0000 7f93c0813640 1 -- 172.21.15.49:0/3849703978 shutdown_connections 2023-11-06T02:08:40.823 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.778+0000 7f93c0813640 1 --2- 172.21.15.49:0/3849703978 >> v2:172.21.15.49:3300/0 conn(0x7f93b40df460 0x7f93b4054110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.824 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.778+0000 7f93c0813640 1 --2- 172.21.15.49:0/3849703978 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f93b4055630 0x7f93b4007410 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.824 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.778+0000 7f93c0813640 1 -- 172.21.15.49:0/3849703978 >> 172.21.15.49:0/3849703978 conn(0x7f93b4054b10 msgr2=0x7f93b4054f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:40.824 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.782+0000 7f194c9a2640 1 -- 172.21.15.49:0/1632210198 shutdown_connections 2023-11-06T02:08:40.825 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.782+0000 7f93c0813640 1 -- 172.21.15.49:0/3849703978 shutdown_connections 2023-11-06T02:08:40.825 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.782+0000 7f8b15f39640 1 -- 172.21.15.49:0/1202269021 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8b100ce1e0 msgr2=0x7f8b1006c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:40.825 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.782+0000 7f8b15f39640 1 --2- 172.21.15.49:0/1202269021 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8b100ce1e0 0x7f8b1006c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.826 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.782+0000 7f8b15f39640 1 -- 172.21.15.49:0/1202269021 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8b10159b60 msgr2=0x7f8b1006ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:40.826 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.782+0000 7f8b15f39640 1 --2- 172.21.15.49:0/1202269021 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8b10159b60 0x7f8b1006ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.826 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.782+0000 7f8b15f39640 1 -- 172.21.15.49:0/1202269021 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b0c0094d0 con 0x7f8b100b40a0 2023-11-06T02:08:40.826 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.782+0000 7fdf5ed9d640 1 --2- 172.21.15.49:0/4292040471 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdf480039b0 0x7fdf48003dd0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fdf50009b90 tx=0x7fdf5001cb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae4eab78a47efa51 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:40.827 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.782+0000 7f8b15f39640 1 --2- 172.21.15.49:0/1202269021 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b100b40a0 0x7f8b1014fcd0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c00e960 tx=0x7f8b0c00ee20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:40.827 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.782+0000 7f93c0813640 1 -- 172.21.15.49:0/3849703978 wait complete. 2023-11-06T02:08:40.828 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.782+0000 7fdf5dd9b640 1 -- 172.21.15.49:0/4292040471 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf500a5610 con 0x7fdf480039b0 2023-11-06T02:08:40.828 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.782+0000 7fdf5dd9b640 1 -- 172.21.15.49:0/4292040471 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf500a5770 con 0x7fdf480039b0 2023-11-06T02:08:40.828 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.782+0000 7fdf5dd9b640 1 -- 172.21.15.49:0/4292040471 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf500a5b00 con 0x7fdf480039b0 2023-11-06T02:08:40.829 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.782+0000 7f8b067fc640 1 -- 172.21.15.49:0/1202269021 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b0c00ca00 con 0x7f8b100b40a0 2023-11-06T02:08:40.829 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.782+0000 7f8b1819d640 1 -- 172.21.15.49:0/1202269021 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b08009a70 con 0x7f8b100b40a0 2023-11-06T02:08:40.829 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.782+0000 7f8b1819d640 1 -- 172.21.15.49:0/1202269021 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b100c2e40 con 0x7f8b100b40a0 2023-11-06T02:08:40.829 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.782+0000 7f8b067fc640 1 -- 172.21.15.49:0/1202269021 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b0c00cb60 con 0x7f8b100b40a0 2023-11-06T02:08:40.830 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.782+0000 7f8b067fc640 1 -- 172.21.15.49:0/1202269021 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b0c00ced0 con 0x7f8b100b40a0 2023-11-06T02:08:40.830 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.778+0000 7fd8e282e640 1 -- 172.21.15.49:0/1138642649 wait complete. 2023-11-06T02:08:40.830 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.782+0000 7fd8e282e640 1 Processor -- start 2023-11-06T02:08:40.831 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.782+0000 7fdf64950640 1 -- 172.21.15.49:0/4292040471 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdf480039b0 msgr2=0x7fdf48003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:40.831 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.782+0000 7fdf64950640 1 --2- 172.21.15.49:0/4292040471 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdf480039b0 0x7fdf48003dd0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fdf50009b90 tx=0x7fdf5001cb20 comp rx=0 tx=0).stop 2023-11-06T02:08:40.832 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.782+0000 7fdf5f59e640 1 -- 172.21.15.49:0/4292040471 reap_dead start 2023-11-06T02:08:40.832 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.782+0000 7fdf64950640 1 -- 172.21.15.49:0/4292040471 shutdown_connections 2023-11-06T02:08:40.832 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.782+0000 7fdf64950640 1 -- 172.21.15.49:0/4292040471 >> 172.21.15.49:0/4292040471 conn(0x7fdf600ce9a0 msgr2=0x7fdf600cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:40.832 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.782+0000 7f194c9a2640 1 -- 172.21.15.49:0/1632210198 wait complete. 2023-11-06T02:08:40.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.782+0000 7f194c9a2640 1 Processor -- start 2023-11-06T02:08:40.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.782+0000 7f93c0813640 1 Processor -- start 2023-11-06T02:08:40.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.782+0000 7f8b1819d640 1 -- 172.21.15.49:0/1202269021 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8af8005260 con 0x7f8b100b40a0 2023-11-06T02:08:40.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.782+0000 7fd8e282e640 1 -- start start 2023-11-06T02:08:40.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.782+0000 7fd8e282e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd8dc0cb810 0x7fd8dc0c9110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.782+0000 7fd8e282e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8dc0ce320 0x7fd8dc156800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.782+0000 7fd8e282e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd8dc164850 0x7fd8dc202a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.782+0000 7fd8e282e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8dc0c8960 con 0x7fd8dc0ce320 2023-11-06T02:08:40.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.782+0000 7fd8e282e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8dc0c78c0 con 0x7fd8dc164850 2023-11-06T02:08:40.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.782+0000 7fd8e282e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8dc0c7ba0 con 0x7fd8dc0cb810 2023-11-06T02:08:40.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.782+0000 7fdf64950640 1 -- 172.21.15.49:0/4292040471 shutdown_connections 2023-11-06T02:08:40.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.782+0000 7fd8dbfff640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd8dc0cb810 0x7fd8dc0c9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.782+0000 7fdb159bc640 1 Processor -- start 2023-11-06T02:08:40.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.782+0000 7f93c0813640 1 -- start start 2023-11-06T02:08:40.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.782+0000 7fdb159bc640 1 -- start start 2023-11-06T02:08:40.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.782+0000 7f93c0813640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93b4055630 0x7f93b40d6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.782+0000 7fd8db7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8dc0ce320 0x7fd8dc156800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.839 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.782+0000 7fd8db7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8dc0ce320 0x7fd8dc156800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52216/0 (socket says 172.21.15.49:52216) 2023-11-06T02:08:40.839 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.782+0000 7fd8db7fe640 1 -- 172.21.15.49:0/4095349699 learned_addr learned my addr 172.21.15.49:0/4095349699 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:40.839 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.782+0000 7f194c9a2640 1 -- start start 2023-11-06T02:08:40.840 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.782+0000 7f93c0813640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93b40df460 0x7f93b404ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.840 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.782+0000 7f194c9a2640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1940050280 0x7f19400ebd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.841 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.782+0000 7f194c9a2640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f19400df460 0x7f194004af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.841 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.782+0000 7f194c9a2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f194004bbe0 0x7f194017c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.841 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.782+0000 7f194c9a2640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1940049510 con 0x7f194004bbe0 2023-11-06T02:08:40.841 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.782+0000 7f194c9a2640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f194004b4b0 con 0x7f1940050280 2023-11-06T02:08:40.841 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.782+0000 7f194c9a2640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19400eab90 con 0x7f19400df460 2023-11-06T02:08:40.842 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.782+0000 7f1946d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f194004bbe0 0x7f194017c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.842 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.782+0000 7f1946d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f194004bbe0 0x7f194017c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52234/0 (socket says 172.21.15.49:52234) 2023-11-06T02:08:40.842 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.782+0000 7f1946d9d640 1 -- 172.21.15.49:0/3289220166 learned_addr learned my addr 172.21.15.49:0/3289220166 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:40.842 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.782+0000 7f194659c640 1 --2- 172.21.15.49:0/3289220166 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1940050280 0x7f19400ebd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.843 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.782+0000 7f1945d9b640 1 --2- 172.21.15.49:0/3289220166 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f19400df460 0x7f194004af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.843 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.782+0000 7f93bb7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93b4055630 0x7f93b40d6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.843 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.782+0000 7f93bb7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93b4055630 0x7f93b40d6b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52218/0 (socket says 172.21.15.49:52218) 2023-11-06T02:08:40.843 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.782+0000 7f93bb7fe640 1 -- 172.21.15.49:0/1480523494 learned_addr learned my addr 172.21.15.49:0/1480523494 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:40.844 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.782+0000 7f93c0813640 1 --2- 172.21.15.49:0/1480523494 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f93b404e910 0x7f93b404c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.844 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.782+0000 7f93baffd640 1 --2- 172.21.15.49:0/1480523494 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93b40df460 0x7f93b404ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.844 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.782+0000 7fd8db7fe640 1 -- 172.21.15.49:0/4095349699 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd8dc0cb810 msgr2=0x7fd8dc0c9110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:40.844 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.782+0000 7fd8db7fe640 1 --2- 172.21.15.49:0/4095349699 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd8dc0cb810 0x7fd8dc0c9110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.845 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.782+0000 7fd8db7fe640 1 -- 172.21.15.49:0/4095349699 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd8dc164850 msgr2=0x7fd8dc202a10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T02:08:40.845 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.782+0000 7fd8db7fe640 1 --2- 172.21.15.49:0/4095349699 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd8dc164850 0x7fd8dc202a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.845 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.782+0000 7fd8db7fe640 1 -- 172.21.15.49:0/4095349699 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8d4009b30 con 0x7fd8dc0ce320 2023-11-06T02:08:40.846 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.786+0000 7f93c0813640 1 -- 172.21.15.49:0/1480523494 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93b404b650 con 0x7f93b4055630 2023-11-06T02:08:40.846 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.786+0000 7f93c0813640 1 -- 172.21.15.49:0/1480523494 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93b4034f50 con 0x7f93b40df460 2023-11-06T02:08:40.846 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.786+0000 7f93c0813640 1 -- 172.21.15.49:0/1480523494 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93b4035230 con 0x7f93b404e910 2023-11-06T02:08:40.847 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.786+0000 7f1946d9d640 1 -- 172.21.15.49:0/3289220166 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f19400df460 msgr2=0x7f194004af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:40.847 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.786+0000 7f1946d9d640 1 --2- 172.21.15.49:0/3289220166 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f19400df460 0x7f194004af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.847 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.786+0000 7f1946d9d640 1 -- 172.21.15.49:0/3289220166 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1940050280 msgr2=0x7f19400ebd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:40.847 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.786+0000 7f1946d9d640 1 --2- 172.21.15.49:0/3289220166 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1940050280 0x7f19400ebd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.847 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.786+0000 7f1946d9d640 1 -- 172.21.15.49:0/3289220166 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1938009d60 con 0x7f194004bbe0 2023-11-06T02:08:40.848 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.786+0000 7f1945d9b640 1 --2- 172.21.15.49:0/3289220166 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f19400df460 0x7f194004af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:08:40.848 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.786+0000 7f93bbfff640 1 --2- 172.21.15.49:0/1480523494 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f93b404e910 0x7f93b404c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.848 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.786+0000 7fd8db7fe640 1 --2- 172.21.15.49:0/4095349699 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8dc0ce320 0x7fd8dc156800 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd8d000c450 tx=0x7fd8d000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:40.849 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.782+0000 7f8b067fc640 1 -- 172.21.15.49:0/1202269021 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f8b0c026480 con 0x7f8b100b40a0 2023-11-06T02:08:40.849 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.786+0000 7f8b067fc640 1 --2- 172.21.15.49:0/1202269021 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8af4065a60 0x7f8af4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.849 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.786+0000 7f8b067fc640 1 -- 172.21.15.49:0/1202269021 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8b0c083d20 con 0x7f8b100b40a0 2023-11-06T02:08:40.850 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.786+0000 7fdf64950640 1 -- 172.21.15.49:0/4292040471 wait complete. 2023-11-06T02:08:40.850 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.786+0000 7f8b15738640 1 --2- 172.21.15.49:0/1202269021 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8af4065a60 0x7f8af4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.851 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.786+0000 7f93bb7fe640 1 -- 172.21.15.49:0/1480523494 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f93b404e910 msgr2=0x7f93b404c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:40.851 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.786+0000 7f93bb7fe640 1 --2- 172.21.15.49:0/1480523494 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f93b404e910 0x7f93b404c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.851 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.786+0000 7f93bb7fe640 1 -- 172.21.15.49:0/1480523494 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93b40df460 msgr2=0x7f93b404ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:40.851 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.786+0000 7f93bb7fe640 1 --2- 172.21.15.49:0/1480523494 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93b40df460 0x7f93b404ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.852 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.786+0000 7f93bb7fe640 1 -- 172.21.15.49:0/1480523494 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93a80094d0 con 0x7f93b4055630 2023-11-06T02:08:40.852 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.786+0000 7fdb159bc640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdb080360d0 0x7fdb0804a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.853 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.786+0000 7fd8d8ff9640 1 -- 172.21.15.49:0/4095349699 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8d0012830 con 0x7fd8dc0ce320 2023-11-06T02:08:40.853 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.786+0000 7f1946d9d640 1 --2- 172.21.15.49:0/3289220166 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f194004bbe0 0x7f194017c800 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1948069620 tx=0x7f19480733b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:40.853 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.786+0000 7fdb159bc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb08039f60 0x7fdb08057060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.854 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.786+0000 7fdb159bc640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb08051b50 0x7fdb0804c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.854 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.786+0000 7fdb149ba640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdb080360d0 0x7fdb0804a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.854 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.786+0000 7fdb149ba640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdb080360d0 0x7fdb0804a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.49:33652/0 (socket says 172.21.15.49:33652) 2023-11-06T02:08:40.854 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.786+0000 7fdb149ba640 1 -- 172.21.15.49:0/122718117 learned_addr learned my addr 172.21.15.49:0/122718117 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:40.854 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.786+0000 7fdb159bc640 1 -- 172.21.15.49:0/122718117 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0804bc00 con 0x7fdb08039f60 2023-11-06T02:08:40.855 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.786+0000 7fdb159bc640 1 -- 172.21.15.49:0/122718117 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0804aff0 con 0x7fdb08051b50 2023-11-06T02:08:40.855 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.786+0000 7fdb159bc640 1 -- 172.21.15.49:0/122718117 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0804b2d0 con 0x7fdb080360d0 2023-11-06T02:08:40.855 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.786+0000 7fdb151bb640 1 --2- 172.21.15.49:0/122718117 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb08051b50 0x7fdb0804c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.855 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.786+0000 7fdb0ffff640 1 --2- 172.21.15.49:0/122718117 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb08039f60 0x7fdb08057060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.856 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.786+0000 7fd8d8ff9640 1 -- 172.21.15.49:0/4095349699 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8d0012990 con 0x7fd8dc0ce320 2023-11-06T02:08:40.856 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.786+0000 7fd8e282e640 1 -- 172.21.15.49:0/4095349699 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8dc0c7e80 con 0x7fd8dc0ce320 2023-11-06T02:08:40.856 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.786+0000 7fd8e282e640 1 -- 172.21.15.49:0/4095349699 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8dc0c8440 con 0x7fd8dc0ce320 2023-11-06T02:08:40.856 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.786+0000 7fd8d8ff9640 1 -- 172.21.15.49:0/4095349699 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8d0012d40 con 0x7fd8dc0ce320 2023-11-06T02:08:40.857 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.786+0000 7f93bb7fe640 1 --2- 172.21.15.49:0/1480523494 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93b4055630 0x7f93b40d6b70 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f93b000c480 tx=0x7f93b000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:40.857 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.786+0000 7fdf64950640 1 Processor -- start 2023-11-06T02:08:40.857 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.786+0000 7f1936ffd640 1 -- 172.21.15.49:0/3289220166 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1948067260 con 0x7f194004bbe0 2023-11-06T02:08:40.858 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.786+0000 7f194c9a2640 1 -- 172.21.15.49:0/3289220166 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19400eade0 con 0x7f194004bbe0 2023-11-06T02:08:40.858 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.786+0000 7f194c9a2640 1 -- 172.21.15.49:0/3289220166 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19400eb3a0 con 0x7f194004bbe0 2023-11-06T02:08:40.858 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.786+0000 7f1936ffd640 1 -- 172.21.15.49:0/3289220166 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19480721c0 con 0x7f194004bbe0 2023-11-06T02:08:40.858 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.786+0000 7f1936ffd640 1 -- 172.21.15.49:0/3289220166 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1948091400 con 0x7f194004bbe0 2023-11-06T02:08:40.859 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.786+0000 7f32c4feb640 1 Processor -- start 2023-11-06T02:08:40.859 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.786+0000 7fdb0ffff640 1 -- 172.21.15.49:0/122718117 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdb080360d0 msgr2=0x7fdb0804a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:40.859 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.786+0000 7fdb0ffff640 1 --2- 172.21.15.49:0/122718117 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdb080360d0 0x7fdb0804a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.860 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.786+0000 7fdb0ffff640 1 -- 172.21.15.49:0/122718117 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb08051b50 msgr2=0x7fdb0804c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:40.860 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.786+0000 7fdb0ffff640 1 --2- 172.21.15.49:0/122718117 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb08051b50 0x7fdb0804c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.860 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.786+0000 7fdb0ffff640 1 -- 172.21.15.49:0/122718117 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdafc009420 con 0x7fdb08039f60 2023-11-06T02:08:40.860 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.786+0000 7fdb0ffff640 1 --2- 172.21.15.49:0/122718117 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb08039f60 0x7fdb08057060 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fdafc0ac000 tx=0x7fdafc01ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:40.861 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.786+0000 7fd8e282e640 1 -- 172.21.15.49:0/4095349699 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8bc005260 con 0x7fd8dc0ce320 2023-11-06T02:08:40.861 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.786+0000 7f8b15738640 1 --2- 172.21.15.49:0/1202269021 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8af4065a60 0x7f8af4067f20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8b100c1c70 tx=0x7f8b00009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:40.862 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.786+0000 7f93a7fff640 1 -- 172.21.15.49:0/1480523494 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b0005480 con 0x7f93b4055630 2023-11-06T02:08:40.862 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.786+0000 7f93a7fff640 1 -- 172.21.15.49:0/1480523494 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93b0003600 con 0x7f93b4055630 2023-11-06T02:08:40.862 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.786+0000 7f93a7fff640 1 -- 172.21.15.49:0/1480523494 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b00039b0 con 0x7f93b4055630 2023-11-06T02:08:40.862 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.786+0000 7f93c0813640 1 -- 172.21.15.49:0/1480523494 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93b4035510 con 0x7f93b4055630 2023-11-06T02:08:40.862 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.786+0000 7f93c0813640 1 -- 172.21.15.49:0/1480523494 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93b4035d10 con 0x7f93b4055630 2023-11-06T02:08:40.863 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.786+0000 7fdf64950640 1 -- start start 2023-11-06T02:08:40.863 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.786+0000 7f357a712640 1 Processor -- start 2023-11-06T02:08:40.864 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.786+0000 7fdf64950640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdf480039b0 0x7fdf6015d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.864 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.786+0000 7fdf64950640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf60152d30 0x7fdf600b3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.864 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.786+0000 7fdf64950640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf6006d5d0 0x7fdf60145790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.865 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.786+0000 7fdf64950640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf600c6d60 con 0x7fdf480039b0 2023-11-06T02:08:40.865 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.786+0000 7fdf64950640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf600c7040 con 0x7fdf60152d30 2023-11-06T02:08:40.865 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.786+0000 7fdf64950640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf600c7320 con 0x7fdf6006d5d0 2023-11-06T02:08:40.865 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.786+0000 7fdf5f59e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf6006d5d0 0x7fdf60145790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.865 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.786+0000 7fdf5f59e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf6006d5d0 0x7fdf60145790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.49:33666/0 (socket says 172.21.15.49:33666) 2023-11-06T02:08:40.866 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.786+0000 7fdf5f59e640 1 -- 172.21.15.49:0/1352908973 learned_addr learned my addr 172.21.15.49:0/1352908973 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:40.866 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.786+0000 7fdf5e59c640 1 --2- 172.21.15.49:0/1352908973 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf60152d30 0x7fdf600b3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.866 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.786+0000 7fd8d8ff9640 1 -- 172.21.15.49:0/4095349699 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fd8d001b3c0 con 0x7fd8dc0ce320 2023-11-06T02:08:40.867 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.790+0000 7fd8d8ff9640 1 --2- 172.21.15.49:0/4095349699 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd8c8065ab0 0x7fd8c8067f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.867 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.790+0000 7f357a712640 1 -- start start 2023-11-06T02:08:40.867 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.790+0000 7fdf5f59e640 1 -- 172.21.15.49:0/1352908973 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf60152d30 msgr2=0x7fdf600b3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:40.868 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.790+0000 7fdf5f59e640 1 --2- 172.21.15.49:0/1352908973 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf60152d30 0x7fdf600b3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.868 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.790+0000 7fdf5f59e640 1 -- 172.21.15.49:0/1352908973 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdf480039b0 msgr2=0x7fdf6015d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:40.868 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.790+0000 7fdf5f59e640 1 --2- 172.21.15.49:0/1352908973 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdf480039b0 0x7fdf6015d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.868 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.790+0000 7fdf5f59e640 1 -- 172.21.15.49:0/1352908973 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf600c7600 con 0x7fdf6006d5d0 2023-11-06T02:08:40.869 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.786+0000 7f32c4feb640 1 -- start start 2023-11-06T02:08:40.869 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.790+0000 7f32c4feb640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f32c0160da0 0x7f32c00b43e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.869 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.790+0000 7f32c4feb640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f32c00d0530 0x7f32c00b4920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.870 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.790+0000 7f32c4feb640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32c00d0ad0 con 0x7f32c00b4e60 2023-11-06T02:08:40.870 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.790+0000 7f32c4feb640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32c00b52b0 con 0x7f32c0160da0 2023-11-06T02:08:40.870 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.790+0000 7f32c4feb640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32c00d0db0 con 0x7f32c00d0530 2023-11-06T02:08:40.870 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.790+0000 7fd8d8ff9640 1 -- 172.21.15.49:0/4095349699 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd8d00833b0 con 0x7fd8dc0ce320 2023-11-06T02:08:40.871 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.790+0000 7f93c0813640 1 -- 172.21.15.49:0/1480523494 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93b403a670 con 0x7f93b4055630 2023-11-06T02:08:40.871 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.790+0000 7f32bdd9b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f32c00d0530 0x7f32c00b4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.872 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.790+0000 7fdf5f59e640 1 --2- 172.21.15.49:0/1352908973 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf6006d5d0 0x7fdf60145790 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fdf600cc4c0 tx=0x7fdf5800d520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:40.872 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.790+0000 7f32bdd9b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f32c00d0530 0x7f32c00b4920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.49:33672/0 (socket says 172.21.15.49:33672) 2023-11-06T02:08:40.872 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.790+0000 7f32bdd9b640 1 -- 172.21.15.49:0/4173035243 learned_addr learned my addr 172.21.15.49:0/4173035243 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:40.873 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.790+0000 7fdb0d7fa640 1 -- 172.21.15.49:0/122718117 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdafc0b89a0 con 0x7fdb08039f60 2023-11-06T02:08:40.873 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.790+0000 7fdb159bc640 1 -- 172.21.15.49:0/122718117 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb0804b5b0 con 0x7fdb08039f60 2023-11-06T02:08:40.873 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.790+0000 7fdb159bc640 1 -- 172.21.15.49:0/122718117 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb080daf60 con 0x7fdb08039f60 2023-11-06T02:08:40.874 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.790+0000 7fdf477fe640 1 -- 172.21.15.49:0/1352908973 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf58006b60 con 0x7fdf6006d5d0 2023-11-06T02:08:40.874 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.790+0000 7fdf64950640 1 -- 172.21.15.49:0/1352908973 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf601534b0 con 0x7fdf6006d5d0 2023-11-06T02:08:40.874 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.790+0000 7fdf64950640 1 -- 172.21.15.49:0/1352908973 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf6006dc20 con 0x7fdf6006d5d0 2023-11-06T02:08:40.875 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.790+0000 7f32bdd9b640 1 -- 172.21.15.49:0/4173035243 >> v2:172.21.15.49:3300/0 conn(0x7f32c0160da0 msgr2=0x7f32c00b43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:40.875 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.790+0000 7f32bdd9b640 1 --2- 172.21.15.49:0/4173035243 >> v2:172.21.15.49:3300/0 conn(0x7f32c0160da0 0x7f32c00b43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.875 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.790+0000 7f32bdd9b640 1 -- 172.21.15.49:0/4173035243 >> v1:172.21.15.150:6789/0 conn(0x7f32c00b4e60 legacy=0x7f32c00ced00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:40.875 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.790+0000 7f32bdd9b640 1 -- 172.21.15.49:0/4173035243 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32c00d1090 con 0x7f32c00d0530 2023-11-06T02:08:40.876 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.790+0000 7fdb159bc640 1 -- 172.21.15.49:0/122718117 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb0804bd70 con 0x7fdb08039f60 2023-11-06T02:08:40.876 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.790+0000 7fdf477fe640 1 -- 172.21.15.49:0/1352908973 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf58006cc0 con 0x7fdf6006d5d0 2023-11-06T02:08:40.877 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.790+0000 7fdf64950640 1 -- 172.21.15.49:0/1352908973 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf600c7840 con 0x7fdf6006d5d0 2023-11-06T02:08:40.877 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.790+0000 7f357a712640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f356c0df430 0x7f356c007360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.877 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.790+0000 7f357a712640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f356c0e3500 0x7f356c038340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.877 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.790+0000 7f357a712640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f356c006020 con 0x7f356c053e20 2023-11-06T02:08:40.878 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.790+0000 7f357a712640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f356c0e3bb0 con 0x7f356c0e3500 2023-11-06T02:08:40.878 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.790+0000 7f357a712640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f356c0062c0 con 0x7f356c0df430 2023-11-06T02:08:40.878 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.790+0000 7fdf477fe640 1 -- 172.21.15.49:0/1352908973 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf5800dd80 con 0x7fdf6006d5d0 2023-11-06T02:08:40.879 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.790+0000 7fd8dbfff640 1 --2- 172.21.15.49:0/4095349699 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd8c8065ab0 0x7fd8c8067f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.879 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.786+0000 7f194c9a2640 1 -- 172.21.15.49:0/3289220166 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f192c005260 con 0x7f194004bbe0 2023-11-06T02:08:40.879 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.790+0000 7f1936ffd640 1 -- 172.21.15.49:0/3289220166 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f1948091560 con 0x7f194004bbe0 2023-11-06T02:08:40.881 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.790+0000 7f1936ffd640 1 --2- 172.21.15.49:0/3289220166 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1928065a60 0x7f1928067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.790+0000 7f194659c640 1 --2- 172.21.15.49:0/3289220166 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1928065a60 0x7f1928067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.790+0000 7fdf477fe640 1 -- 172.21.15.49:0/1352908973 <== mon.2 v2:172.21.15.150:3301/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fdf58003b90 con 0x7fdf6006d5d0 2023-11-06T02:08:40.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.794+0000 7fdf477fe640 1 --2- 172.21.15.49:0/1352908973 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fdf4c065a90 0x7fdf4c067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.885 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.794+0000 7f32bdd9b640 1 --2- 172.21.15.49:0/4173035243 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f32c00d0530 0x7f32c00b4920 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f32b0009870 tx=0x7f32b001c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f99fd756736954b1 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:40.885 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.794+0000 7fdf477fe640 1 -- 172.21.15.49:0/1352908973 <== mon.2 v2:172.21.15.150:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdf580837d0 con 0x7fdf6006d5d0 2023-11-06T02:08:40.886 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.794+0000 7f3573fff640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f356c0df430 0x7f356c007360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.886 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.794+0000 7f3573fff640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f356c0df430 0x7f356c007360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.49:33684/0 (socket says 172.21.15.49:33684) 2023-11-06T02:08:40.886 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.794+0000 7f3573fff640 1 -- 172.21.15.49:0/3828396463 learned_addr learned my addr 172.21.15.49:0/3828396463 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:40.887 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.794+0000 7f35737fe640 1 --1- 172.21.15.49:0/3828396463 >> v1:172.21.15.150:6789/0 conn(0x7f356c053e20 0x7f356c0e2df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.49:43628/0 (socket says 172.21.15.49:43628) 2023-11-06T02:08:40.887 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.794+0000 7f3578caf640 1 --2- 172.21.15.49:0/3828396463 >> v2:172.21.15.150:3300/0 conn(0x7f356c0e3500 0x7f356c038340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.887 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.794+0000 7f194659c640 1 --2- 172.21.15.49:0/3289220166 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1928065a60 0x7f1928067f20 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f19380af000 tx=0x7f19380a8000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:40.888 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.794+0000 7fdf5ed9d640 1 --2- 172.21.15.49:0/1352908973 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fdf4c065a90 0x7fdf4c067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.888 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.794+0000 7fd8dbfff640 1 --2- 172.21.15.49:0/4095349699 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd8c8065ab0 0x7fd8c8067f70 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc007fd0 tx=0x7fd8cc007e40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:40.889 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.794+0000 7fdf5ed9d640 1 --2- 172.21.15.49:0/1352908973 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fdf4c065a90 0x7fdf4c067f50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fdf500af000 tx=0x7fdf50004740 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:40.889 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.794+0000 7f1936ffd640 1 -- 172.21.15.49:0/3289220166 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f19480ce770 con 0x7f194004bbe0 2023-11-06T02:08:40.889 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.794+0000 7fd8d8ff9640 1 -- 172.21.15.49:0/4095349699 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fd8d0050cf0 con 0x7fd8dc0ce320 2023-11-06T02:08:40.890 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.794+0000 7fdb0d7fa640 1 -- 172.21.15.49:0/122718117 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdafc0bc070 con 0x7fdb08039f60 2023-11-06T02:08:40.890 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.794+0000 7fdb0d7fa640 1 -- 172.21.15.49:0/122718117 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdafc0b8d00 con 0x7fdb08039f60 2023-11-06T02:08:40.891 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.794+0000 7f3573fff640 1 -- 172.21.15.49:0/3828396463 >> v2:172.21.15.150:3300/0 conn(0x7f356c0e3500 msgr2=0x7f356c038340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:40.891 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.794+0000 7f3573fff640 1 --2- 172.21.15.49:0/3828396463 >> v2:172.21.15.150:3300/0 conn(0x7f356c0e3500 0x7f356c038340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.891 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.794+0000 7f3573fff640 1 -- 172.21.15.49:0/3828396463 >> v1:172.21.15.150:6789/0 conn(0x7f356c053e20 legacy=0x7f356c0e2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:40.891 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.794+0000 7f3573fff640 1 -- 172.21.15.49:0/3828396463 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f356c0065a0 con 0x7f356c0df430 2023-11-06T02:08:40.891 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.798+0000 7f3573fff640 1 --2- 172.21.15.49:0/3828396463 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f356c0df430 0x7f356c007360 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f356800a910 tx=0x7f356801c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b16b9575569a6d2 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:40.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.798+0000 7f93a7fff640 1 -- 172.21.15.49:0/1480523494 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f93b0003b10 con 0x7f93b4055630 2023-11-06T02:08:40.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.798+0000 7f93a7fff640 1 --2- 172.21.15.49:0/1480523494 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f93a0065a60 0x7f93a0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.798+0000 7f93baffd640 1 --2- 172.21.15.49:0/1480523494 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f93a0065a60 0x7f93a0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.798+0000 7f93baffd640 1 --2- 172.21.15.49:0/1480523494 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f93a0065a60 0x7f93a0067f20 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f93a80b0000 tx=0x7f93a80a8000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:40.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.798+0000 7f1936ffd640 1 -- 172.21.15.49:0/3289220166 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f19480ca3b0 con 0x7f194004bbe0 2023-11-06T02:08:40.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.802+0000 7fdb0d7fa640 1 -- 172.21.15.49:0/122718117 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fdafc0cc420 con 0x7fdb08039f60 2023-11-06T02:08:40.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.802+0000 7f93a7fff640 1 -- 172.21.15.49:0/1480523494 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f93b0054ea0 con 0x7f93b4055630 2023-11-06T02:08:40.895 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.802+0000 7fdb0d7fa640 1 --2- 172.21.15.49:0/122718117 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fdaf4065a60 0x7fdaf4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.895 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.802+0000 7fdb0d7fa640 1 -- 172.21.15.49:0/122718117 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdafc0be070 con 0x7fdb08039f60 2023-11-06T02:08:40.895 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.802+0000 7fdb149ba640 1 --2- 172.21.15.49:0/122718117 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fdaf4065a60 0x7fdaf4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.895 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.802+0000 7fdb149ba640 1 --2- 172.21.15.49:0/122718117 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fdaf4065a60 0x7fdaf4067f20 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fdb080e3b20 tx=0x7fdb04008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:40.896 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.806+0000 7f8b067fc640 1 -- 172.21.15.49:0/1202269021 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f8b0c051580 con 0x7f8b100b40a0 2023-11-06T02:08:40.896 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.806+0000 7f93a7fff640 1 -- 172.21.15.49:0/1480523494 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f93b00511e0 con 0x7f93b4055630 2023-11-06T02:08:40.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.810+0000 7fdb0d7fa640 1 -- 172.21.15.49:0/122718117 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fdafc0ef7e0 con 0x7fdb08039f60 2023-11-06T02:08:40.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.810+0000 7f3572ffd640 1 -- 172.21.15.49:0/3828396463 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35680a5870 con 0x7f356c0df430 2023-11-06T02:08:40.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.810+0000 7f3572ffd640 1 -- 172.21.15.49:0/3828396463 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35680a9070 con 0x7f356c0df430 2023-11-06T02:08:40.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.810+0000 7f3572ffd640 1 -- 172.21.15.49:0/3828396463 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35680a5c30 con 0x7f356c0df430 2023-11-06T02:08:40.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.810+0000 7fdf477fe640 1 -- 172.21.15.49:0/1352908973 <== mon.2 v2:172.21.15.150:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fdf58051030 con 0x7fdf6006d5d0 2023-11-06T02:08:40.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.814+0000 7f357a712640 1 -- 172.21.15.49:0/3828396463 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f356c0df430 msgr2=0x7f356c007360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:40.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.814+0000 7f357a712640 1 --2- 172.21.15.49:0/3828396463 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f356c0df430 0x7f356c007360 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f356800a910 tx=0x7f356801c890 comp rx=0 tx=0).stop 2023-11-06T02:08:40.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.810+0000 7f32bd59a640 1 -- 172.21.15.49:0/4173035243 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b00a6040 con 0x7f32c00d0530 2023-11-06T02:08:40.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.810+0000 7f32bd59a640 1 -- 172.21.15.49:0/4173035243 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32b00aa600 con 0x7f32c00d0530 2023-11-06T02:08:40.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.810+0000 7f32bd59a640 1 -- 172.21.15.49:0/4173035243 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b00aa990 con 0x7f32c00d0530 2023-11-06T02:08:40.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.814+0000 7f357a712640 1 -- 172.21.15.49:0/3828396463 shutdown_connections 2023-11-06T02:08:40.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.814+0000 7f357a712640 1 --2- 172.21.15.49:0/3828396463 >> v2:172.21.15.150:3300/0 conn(0x7f356c0e3500 0x7f356c038340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.814+0000 7f357a712640 1 --2- 172.21.15.49:0/3828396463 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f356c0df430 0x7f356c007360 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.814+0000 7f357a712640 1 -- 172.21.15.49:0/3828396463 >> 172.21.15.49:0/3828396463 conn(0x7f356c054c50 msgr2=0x7f356c055060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:40.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.814+0000 7f32c4feb640 1 -- 172.21.15.49:0/4173035243 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f32c00d0530 msgr2=0x7f32c00b4920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:40.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.814+0000 7f32c4feb640 1 --2- 172.21.15.49:0/4173035243 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f32c00d0530 0x7f32c00b4920 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f32b0009870 tx=0x7f32b001c4a0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.814+0000 7f32c4feb640 1 -- 172.21.15.49:0/4173035243 shutdown_connections 2023-11-06T02:08:40.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.814+0000 7f32c4feb640 1 --2- 172.21.15.49:0/4173035243 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f32c00d0530 0x7f32c00b4920 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.814+0000 7f32c4feb640 1 --2- 172.21.15.49:0/4173035243 >> v2:172.21.15.49:3300/0 conn(0x7f32c0160da0 0x7f32c00b43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.814+0000 7f32c4feb640 1 -- 172.21.15.49:0/4173035243 >> 172.21.15.49:0/4173035243 conn(0x7f32c006af70 msgr2=0x7f32c006b380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:40.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.814+0000 7f357a712640 1 -- 172.21.15.49:0/3828396463 shutdown_connections 2023-11-06T02:08:40.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.814+0000 7f357a712640 1 -- 172.21.15.49:0/3828396463 wait complete. 2023-11-06T02:08:40.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.814+0000 7f32c4feb640 1 -- 172.21.15.49:0/4173035243 shutdown_connections 2023-11-06T02:08:40.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.814+0000 7f32c4feb640 1 -- 172.21.15.49:0/4173035243 wait complete. 2023-11-06T02:08:40.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.814+0000 7f32c4feb640 1 Processor -- start 2023-11-06T02:08:40.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.818+0000 7f357a712640 1 Processor -- start 2023-11-06T02:08:40.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.818+0000 7f357a712640 1 -- start start 2023-11-06T02:08:40.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.814+0000 7f32c4feb640 1 -- start start 2023-11-06T02:08:40.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.818+0000 7f32c4feb640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32c00b4e60 0x7f32c00cd040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.818+0000 7f32c4feb640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f32c00d0530 0x7f32c015d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.818+0000 7f32c4feb640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32c0160da0 0x7f32c00c86a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.818+0000 7f32c4feb640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c015dfe0 con 0x7f32c00b4e60 2023-11-06T02:08:40.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.818+0000 7f32c4feb640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c00cd580 con 0x7f32c0160da0 2023-11-06T02:08:40.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.818+0000 7f32c4feb640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32c015e280 con 0x7f32c00d0530 2023-11-06T02:08:40.908 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.818+0000 7f32be59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32c00b4e60 0x7f32c00cd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.908 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.818+0000 7f32bed9d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32c0160da0 0x7f32c00c86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.908 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.818+0000 7f32be59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32c00b4e60 0x7f32c00cd040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52270/0 (socket says 172.21.15.49:52270) 2023-11-06T02:08:40.908 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.818+0000 7f32be59c640 1 -- 172.21.15.49:0/2257924035 learned_addr learned my addr 172.21.15.49:0/2257924035 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:40.908 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.818+0000 7f32bdd9b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f32c00d0530 0x7f32c015d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.818+0000 7f32be59c640 1 -- 172.21.15.49:0/2257924035 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f32c00d0530 msgr2=0x7f32c015d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:40.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.818+0000 7f32be59c640 1 --2- 172.21.15.49:0/2257924035 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f32c00d0530 0x7f32c015d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.818+0000 7f32be59c640 1 -- 172.21.15.49:0/2257924035 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32c0160da0 msgr2=0x7f32c00c86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:40.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.818+0000 7f32be59c640 1 --2- 172.21.15.49:0/2257924035 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32c0160da0 0x7f32c00c86a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.818+0000 7f32be59c640 1 -- 172.21.15.49:0/2257924035 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32b00094d0 con 0x7f32c00b4e60 2023-11-06T02:08:40.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.818+0000 7f32be59c640 1 --2- 172.21.15.49:0/2257924035 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32c00b4e60 0x7f32c00cd040 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f32b400b410 tx=0x7f32b400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:40.911 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.818+0000 7f357a712640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f356c053e20 0x7f356c04a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.911 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.818+0000 7f357a712640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f356c0df430 0x7f356c04df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.911 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.818+0000 7f3573fff640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f356c053e20 0x7f356c04a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.911 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.818+0000 7f3573fff640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f356c053e20 0x7f356c04a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.49:53622/0 (socket says 172.21.15.49:53622) 2023-11-06T02:08:40.911 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.818+0000 7f3573fff640 1 -- 172.21.15.49:0/4065228037 learned_addr learned my addr 172.21.15.49:0/4065228037 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:40.912 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.818+0000 7f35737fe640 1 --2- 172.21.15.49:0/4065228037 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f356c0df430 0x7f356c04df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.912 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.818+0000 7f357a712640 1 --2- 172.21.15.49:0/4065228037 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f356c0e3500 0x7f356c04e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.912 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.818+0000 7f357a712640 1 -- 172.21.15.49:0/4065228037 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f356c0d96a0 con 0x7f356c0df430 2023-11-06T02:08:40.912 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.818+0000 7f357a712640 1 -- 172.21.15.49:0/4065228037 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f356c0d9980 con 0x7f356c053e20 2023-11-06T02:08:40.913 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.822+0000 7f357a712640 1 -- 172.21.15.49:0/4065228037 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f356c0d9c60 con 0x7f356c0e3500 2023-11-06T02:08:40.913 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.822+0000 7f3578caf640 1 --2- 172.21.15.49:0/4065228037 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f356c0e3500 0x7f356c04e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.913 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.822+0000 7f32aeffd640 1 -- 172.21.15.49:0/2257924035 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b4011830 con 0x7f32c00b4e60 2023-11-06T02:08:40.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.822+0000 7f35737fe640 1 -- 172.21.15.49:0/4065228037 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f356c0e3500 msgr2=0x7f356c04e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:40.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.822+0000 7f35737fe640 1 --2- 172.21.15.49:0/4065228037 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f356c0e3500 0x7f356c04e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.822+0000 7f35737fe640 1 -- 172.21.15.49:0/4065228037 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f356c053e20 msgr2=0x7f356c04a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:40.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.822+0000 7f35737fe640 1 --2- 172.21.15.49:0/4065228037 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f356c053e20 0x7f356c04a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:40.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.822+0000 7f32aeffd640 1 -- 172.21.15.49:0/2257924035 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32b4011990 con 0x7f32c00b4e60 2023-11-06T02:08:40.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.822+0000 7f35737fe640 1 -- 172.21.15.49:0/4065228037 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f356800a570 con 0x7f356c0df430 2023-11-06T02:08:40.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.822+0000 7f32c4feb640 1 -- 172.21.15.49:0/2257924035 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32c015e560 con 0x7f32c00b4e60 2023-11-06T02:08:40.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.822+0000 7f32c4feb640 1 -- 172.21.15.49:0/2257924035 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32c015eb20 con 0x7f32c00b4e60 2023-11-06T02:08:40.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.822+0000 7f35737fe640 1 --2- 172.21.15.49:0/4065228037 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f356c0df430 0x7f356c04df20 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f356000c430 tx=0x7f356000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:40.917 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.822+0000 7f32c4feb640 1 -- 172.21.15.49:0/2257924035 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32a4005260 con 0x7f32c00b4e60 2023-11-06T02:08:40.917 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.822+0000 7f32aeffd640 1 -- 172.21.15.49:0/2257924035 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b4011d00 con 0x7f32c00b4e60 2023-11-06T02:08:40.917 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.826+0000 7f32aeffd640 1 -- 172.21.15.49:0/2257924035 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f32b401a400 con 0x7f32c00b4e60 2023-11-06T02:08:40.917 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.826+0000 7f32aeffd640 1 --2- 172.21.15.49:0/2257924035 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f32a0065ab0 0x7f32a0067f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.918 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.826+0000 7f3570ff9640 1 -- 172.21.15.49:0/4065228037 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35600074e0 con 0x7f356c0df430 2023-11-06T02:08:40.918 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.826+0000 7f357a712640 1 -- 172.21.15.49:0/4065228037 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f356c0d9f40 con 0x7f356c0df430 2023-11-06T02:08:40.919 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.826+0000 7f32aeffd640 1 -- 172.21.15.49:0/2257924035 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f32b4054ac0 con 0x7f32c00b4e60 2023-11-06T02:08:40.919 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.826+0000 7f32bdd9b640 1 --2- 172.21.15.49:0/2257924035 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f32a0065ab0 0x7f32a0067f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.919 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.826+0000 7f357a712640 1 -- 172.21.15.49:0/4065228037 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f356c04acc0 con 0x7f356c0df430 2023-11-06T02:08:40.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.826+0000 7f3570ff9640 1 -- 172.21.15.49:0/4065228037 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3560007640 con 0x7f356c0df430 2023-11-06T02:08:40.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.826+0000 7f32bdd9b640 1 --2- 172.21.15.49:0/2257924035 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f32a0065ab0 0x7f32a0067f70 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f32b00ae000 tx=0x7f32b00b5040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:40.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.826+0000 7f3570ff9640 1 -- 172.21.15.49:0/4065228037 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35600049f0 con 0x7f356c0df430 2023-11-06T02:08:40.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.834+0000 7f357a712640 1 -- 172.21.15.49:0/4065228037 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3554005260 con 0x7f356c0df430 2023-11-06T02:08:40.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.834+0000 7f32aeffd640 1 -- 172.21.15.49:0/2257924035 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f32b4050e00 con 0x7f32c00b4e60 2023-11-06T02:08:40.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.838+0000 7f3570ff9640 1 -- 172.21.15.49:0/4065228037 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f3560004b50 con 0x7f356c0df430 2023-11-06T02:08:40.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.838+0000 7f3570ff9640 1 --2- 172.21.15.49:0/4065228037 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3558065a60 0x7f3558067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:40.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.838+0000 7f3570ff9640 1 -- 172.21.15.49:0/4065228037 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3560083390 con 0x7f356c0df430 2023-11-06T02:08:40.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.838+0000 7f3573fff640 1 --2- 172.21.15.49:0/4065228037 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3558065a60 0x7f3558067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:40.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.842+0000 7f3573fff640 1 --2- 172.21.15.49:0/4065228037 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3558065a60 0x7f3558067f20 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3568000c10 tx=0x7f35680a7040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:40.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:40.846+0000 7f3570ff9640 1 -- 172.21.15.49:0/4065228037 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f3560050c00 con 0x7f356c0df430 2023-11-06T02:08:41.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.110+0000 7f93c0813640 1 -- 172.21.15.49:0/1480523494 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f93b4008a20 con 0x7f93b4055630 2023-11-06T02:08:41.109 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.114+0000 7f93a7fff640 1 -- 172.21.15.49:0/1480523494 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f93b0054800 con 0x7f93b4055630 2023-11-06T02:08:41.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.114+0000 7fd8e282e640 1 -- 172.21.15.49:0/4095349699 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fd8bc0057e0 con 0x7fd8dc0ce320 2023-11-06T02:08:41.111 INFO:teuthology.orchestra.run.smithi049.stdout:55834574853 2023-11-06T02:08:41.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.114+0000 7fd8d8ff9640 1 -- 172.21.15.49:0/4095349699 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd8d0054310 con 0x7fd8dc0ce320 2023-11-06T02:08:41.112 INFO:teuthology.orchestra.run.smithi049.stdout:55834574854 2023-11-06T02:08:41.114 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.114+0000 7f93c0813640 1 -- 172.21.15.49:0/1480523494 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f93a0065a60 msgr2=0x7f93a0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:41.114 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.114+0000 7f93c0813640 1 --2- 172.21.15.49:0/1480523494 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f93a0065a60 0x7f93a0067f20 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f93a80b0000 tx=0x7f93a80a8000 comp rx=0 tx=0).stop 2023-11-06T02:08:41.114 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.114+0000 7f93c0813640 1 -- 172.21.15.49:0/1480523494 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93b4055630 msgr2=0x7f93b40d6b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:41.114 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.114+0000 7f93c0813640 1 --2- 172.21.15.49:0/1480523494 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93b4055630 0x7f93b40d6b70 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f93b000c480 tx=0x7f93b000c940 comp rx=0 tx=0).stop 2023-11-06T02:08:41.114 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.114+0000 7f93c0813640 1 -- 172.21.15.49:0/1480523494 shutdown_connections 2023-11-06T02:08:41.115 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.114+0000 7f93c0813640 1 --2- 172.21.15.49:0/1480523494 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f93a0065a60 0x7f93a0067f20 secure :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f93a80b0000 tx=0x7f93a80a8000 comp rx=0 tx=0).stop 2023-11-06T02:08:41.115 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.114+0000 7f93c0813640 1 --2- 172.21.15.49:0/1480523494 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f93b404e910 0x7f93b404c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:41.115 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.114+0000 7f93c0813640 1 --2- 172.21.15.49:0/1480523494 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93b40df460 0x7f93b404ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:41.115 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.118+0000 7f93c0813640 1 --2- 172.21.15.49:0/1480523494 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93b4055630 0x7f93b40d6b70 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:41.115 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.118+0000 7f93c0813640 1 -- 172.21.15.49:0/1480523494 >> 172.21.15.49:0/1480523494 conn(0x7f93b4054b10 msgr2=0x7f93b40dc280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:41.116 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.118+0000 7f93c0813640 1 -- 172.21.15.49:0/1480523494 shutdown_connections 2023-11-06T02:08:41.116 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.118+0000 7f93c0813640 1 -- 172.21.15.49:0/1480523494 wait complete. 2023-11-06T02:08:41.117 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.118+0000 7fd8e282e640 1 -- 172.21.15.49:0/4095349699 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd8c8065ab0 msgr2=0x7fd8c8067f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:41.117 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.118+0000 7fd8e282e640 1 --2- 172.21.15.49:0/4095349699 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd8c8065ab0 0x7fd8c8067f70 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc007fd0 tx=0x7fd8cc007e40 comp rx=0 tx=0).stop 2023-11-06T02:08:41.117 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.118+0000 7fd8e282e640 1 -- 172.21.15.49:0/4095349699 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8dc0ce320 msgr2=0x7fd8dc156800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:41.118 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.118+0000 7fd8e282e640 1 --2- 172.21.15.49:0/4095349699 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8dc0ce320 0x7fd8dc156800 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd8d000c450 tx=0x7fd8d000c910 comp rx=0 tx=0).stop 2023-11-06T02:08:41.118 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.118+0000 7fd8e282e640 1 -- 172.21.15.49:0/4095349699 shutdown_connections 2023-11-06T02:08:41.118 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.118+0000 7fd8e282e640 1 --2- 172.21.15.49:0/4095349699 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd8c8065ab0 0x7fd8c8067f70 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:41.118 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.118+0000 7fd8e282e640 1 --2- 172.21.15.49:0/4095349699 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8dc0ce320 0x7fd8dc156800 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:41.118 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.118+0000 7fd8e282e640 1 --2- 172.21.15.49:0/4095349699 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd8dc164850 0x7fd8dc202a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:41.119 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.118+0000 7fd8e282e640 1 --2- 172.21.15.49:0/4095349699 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd8dc0cb810 0x7fd8dc0c9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:41.119 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.118+0000 7fd8e282e640 1 -- 172.21.15.49:0/4095349699 >> 172.21.15.49:0/4095349699 conn(0x7fd8dc0cc140 msgr2=0x7fd8dc0cdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:41.119 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.118+0000 7fd8e282e640 1 -- 172.21.15.49:0/4095349699 shutdown_connections 2023-11-06T02:08:41.119 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.118+0000 7fd8e282e640 1 -- 172.21.15.49:0/4095349699 wait complete. 2023-11-06T02:08:41.120 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.122+0000 7fdb159bc640 1 -- 172.21.15.49:0/122718117 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fdb080058f0 con 0x7fdb08039f60 2023-11-06T02:08:41.120 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.122+0000 7fdb0d7fa640 1 -- 172.21.15.49:0/122718117 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fdafc0b8e60 con 0x7fdb08039f60 2023-11-06T02:08:41.120 INFO:teuthology.orchestra.run.smithi049.stdout:55834574854 2023-11-06T02:08:41.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.126+0000 7fdafaffd640 1 -- 172.21.15.49:0/122718117 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fdaf4065a60 msgr2=0x7fdaf4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:41.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.126+0000 7f8b1819d640 1 -- 172.21.15.49:0/1202269021 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f8af80057e0 con 0x7f8b100b40a0 2023-11-06T02:08:41.123 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.126+0000 7fdafaffd640 1 --2- 172.21.15.49:0/122718117 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fdaf4065a60 0x7fdaf4067f20 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fdb080e3b20 tx=0x7fdb04008040 comp rx=0 tx=0).stop 2023-11-06T02:08:41.123 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.126+0000 7fdafaffd640 1 -- 172.21.15.49:0/122718117 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb08039f60 msgr2=0x7fdb08057060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:41.123 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.126+0000 7fdafaffd640 1 --2- 172.21.15.49:0/122718117 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb08039f60 0x7fdb08057060 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fdafc0ac000 tx=0x7fdafc01ac90 comp rx=0 tx=0).stop 2023-11-06T02:08:41.123 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.126+0000 7fdafaffd640 1 -- 172.21.15.49:0/122718117 shutdown_connections 2023-11-06T02:08:41.124 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.126+0000 7fdafaffd640 1 --2- 172.21.15.49:0/122718117 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fdaf4065a60 0x7fdaf4067f20 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:41.124 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.126+0000 7fdafaffd640 1 --2- 172.21.15.49:0/122718117 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb08039f60 0x7fdb08057060 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:41.124 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.126+0000 7fdafaffd640 1 --2- 172.21.15.49:0/122718117 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb08051b50 0x7fdb0804c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:41.124 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.126+0000 7fdafaffd640 1 --2- 172.21.15.49:0/122718117 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdb080360d0 0x7fdb0804a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:41.124 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.126+0000 7fdafaffd640 1 -- 172.21.15.49:0/122718117 >> 172.21.15.49:0/122718117 conn(0x7fdb080538b0 msgr2=0x7fdb08053c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:41.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.126+0000 7fdafaffd640 1 -- 172.21.15.49:0/122718117 shutdown_connections 2023-11-06T02:08:41.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.126+0000 7fdafaffd640 1 -- 172.21.15.49:0/122718117 wait complete. 2023-11-06T02:08:41.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.126+0000 7f8b067fc640 1 -- 172.21.15.49:0/1202269021 <== mon.1 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8b0c021090 con 0x7f8b100b40a0 2023-11-06T02:08:41.126 INFO:teuthology.orchestra.run.smithi049.stdout:55834574853 2023-11-06T02:08:41.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.130+0000 7f194c9a2640 1 -- 172.21.15.49:0/3289220166 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f192c0057e0 con 0x7f194004bbe0 2023-11-06T02:08:41.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.134+0000 7f1936ffd640 1 -- 172.21.15.49:0/3289220166 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1948096090 con 0x7f194004bbe0 2023-11-06T02:08:41.131 INFO:teuthology.orchestra.run.smithi049.stdout:55834574854 2023-11-06T02:08:41.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.138+0000 7f8af3fff640 1 -- 172.21.15.49:0/1202269021 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8af4065a60 msgr2=0x7f8af4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:41.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.138+0000 7f8af3fff640 1 --2- 172.21.15.49:0/1202269021 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8af4065a60 0x7f8af4067f20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8b100c1c70 tx=0x7f8b00009040 comp rx=0 tx=0).stop 2023-11-06T02:08:41.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.138+0000 7f8af3fff640 1 -- 172.21.15.49:0/1202269021 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b100b40a0 msgr2=0x7f8b1014fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:41.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.138+0000 7f8af3fff640 1 --2- 172.21.15.49:0/1202269021 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b100b40a0 0x7f8b1014fcd0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c00e960 tx=0x7f8b0c00ee20 comp rx=0 tx=0).stop 2023-11-06T02:08:41.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.138+0000 7f8af3fff640 1 -- 172.21.15.49:0/1202269021 shutdown_connections 2023-11-06T02:08:41.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.138+0000 7f8af3fff640 1 --2- 172.21.15.49:0/1202269021 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8af4065a60 0x7f8af4067f20 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:41.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.138+0000 7f8af3fff640 1 --2- 172.21.15.49:0/1202269021 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8b10159b60 0x7f8b1006ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:41.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.138+0000 7f8af3fff640 1 --2- 172.21.15.49:0/1202269021 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8b100ce1e0 0x7f8b1006c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:41.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.138+0000 7f8af3fff640 1 --2- 172.21.15.49:0/1202269021 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b100b40a0 0x7f8b1014fcd0 secure :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c00e960 tx=0x7f8b0c00ee20 comp rx=0 tx=0).stop 2023-11-06T02:08:41.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.138+0000 7f8af3fff640 1 -- 172.21.15.49:0/1202269021 >> 172.21.15.49:0/1202269021 conn(0x7f8b100cb4f0 msgr2=0x7f8b100b3920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:41.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.138+0000 7f8af3fff640 1 -- 172.21.15.49:0/1202269021 shutdown_connections 2023-11-06T02:08:41.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.138+0000 7f8af3fff640 1 -- 172.21.15.49:0/1202269021 wait complete. 2023-11-06T02:08:41.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.138+0000 7f194c9a2640 1 -- 172.21.15.49:0/3289220166 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1928065a60 msgr2=0x7f1928067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:41.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.138+0000 7f194c9a2640 1 --2- 172.21.15.49:0/3289220166 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1928065a60 0x7f1928067f20 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f19380af000 tx=0x7f19380a8000 comp rx=0 tx=0).stop 2023-11-06T02:08:41.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.138+0000 7f194c9a2640 1 -- 172.21.15.49:0/3289220166 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f194004bbe0 msgr2=0x7f194017c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:41.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.138+0000 7f194c9a2640 1 --2- 172.21.15.49:0/3289220166 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f194004bbe0 0x7f194017c800 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1948069620 tx=0x7f19480733b0 comp rx=0 tx=0).stop 2023-11-06T02:08:41.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.138+0000 7f194c9a2640 1 -- 172.21.15.49:0/3289220166 shutdown_connections 2023-11-06T02:08:41.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.138+0000 7f194c9a2640 1 --2- 172.21.15.49:0/3289220166 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1928065a60 0x7f1928067f20 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:41.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.138+0000 7f194c9a2640 1 --2- 172.21.15.49:0/3289220166 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f194004bbe0 0x7f194017c800 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:41.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.138+0000 7f194c9a2640 1 --2- 172.21.15.49:0/3289220166 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f19400df460 0x7f194004af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:41.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.138+0000 7f194c9a2640 1 --2- 172.21.15.49:0/3289220166 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1940050280 0x7f19400ebd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:41.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.138+0000 7f194c9a2640 1 -- 172.21.15.49:0/3289220166 >> 172.21.15.49:0/3289220166 conn(0x7f1940054b10 msgr2=0x7f19400d3700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:41.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.138+0000 7f194c9a2640 1 -- 172.21.15.49:0/3289220166 shutdown_connections 2023-11-06T02:08:41.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.138+0000 7f194c9a2640 1 -- 172.21.15.49:0/3289220166 wait complete. 2023-11-06T02:08:41.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.138+0000 7fdf64950640 1 -- 172.21.15.49:0/1352908973 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fdf600c7460 con 0x7fdf6006d5d0 2023-11-06T02:08:41.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.142+0000 7fdf477fe640 1 -- 172.21.15.49:0/1352908973 <== mon.2 v2:172.21.15.150:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fdf600c7460 con 0x7fdf6006d5d0 2023-11-06T02:08:41.141 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.2 2023-11-06T02:08:41.141 INFO:teuthology.orchestra.run.smithi049.stdout:55834574854 2023-11-06T02:08:41.143 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.5 2023-11-06T02:08:41.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.146+0000 7fdf64950640 1 -- 172.21.15.49:0/1352908973 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fdf4c065a90 msgr2=0x7fdf4c067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:41.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.146+0000 7fdf64950640 1 --2- 172.21.15.49:0/1352908973 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fdf4c065a90 0x7fdf4c067f50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fdf500af000 tx=0x7fdf50004740 comp rx=0 tx=0).stop 2023-11-06T02:08:41.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.146+0000 7fdf64950640 1 -- 172.21.15.49:0/1352908973 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf6006d5d0 msgr2=0x7fdf60145790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:41.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.146+0000 7fdf64950640 1 --2- 172.21.15.49:0/1352908973 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf6006d5d0 0x7fdf60145790 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fdf600cc4c0 tx=0x7fdf5800d520 comp rx=0 tx=0).stop 2023-11-06T02:08:41.146 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.146+0000 7fdf64950640 1 -- 172.21.15.49:0/1352908973 shutdown_connections 2023-11-06T02:08:41.146 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.146+0000 7fdf64950640 1 --2- 172.21.15.49:0/1352908973 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fdf4c065a90 0x7fdf4c067f50 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:41.146 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.146+0000 7fdf64950640 1 --2- 172.21.15.49:0/1352908973 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf6006d5d0 0x7fdf60145790 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:41.146 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.146+0000 7fdf64950640 1 --2- 172.21.15.49:0/1352908973 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf60152d30 0x7fdf600b3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:41.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.146+0000 7fdf64950640 1 --2- 172.21.15.49:0/1352908973 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdf480039b0 0x7fdf6015d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:41.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.146+0000 7fdf64950640 1 -- 172.21.15.49:0/1352908973 >> 172.21.15.49:0/1352908973 conn(0x7fdf600ce9a0 msgr2=0x7fdf600cd310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:41.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.146+0000 7fdf64950640 1 -- 172.21.15.49:0/1352908973 shutdown_connections 2023-11-06T02:08:41.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.146+0000 7fdf64950640 1 -- 172.21.15.49:0/1352908973 wait complete. 2023-11-06T02:08:41.150 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.0 2023-11-06T02:08:41.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.154+0000 7f32c4feb640 1 -- 172.21.15.49:0/2257924035 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f32a40057e0 con 0x7f32c00b4e60 2023-11-06T02:08:41.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.154+0000 7f32aeffd640 1 -- 172.21.15.49:0/2257924035 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f32b4015020 con 0x7f32c00b4e60 2023-11-06T02:08:41.152 INFO:teuthology.orchestra.run.smithi049.stdout:55834574853 2023-11-06T02:08:41.156 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.158+0000 7f32c4feb640 1 -- 172.21.15.49:0/2257924035 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f32a0065ab0 msgr2=0x7f32a0067f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:41.156 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.158+0000 7f32c4feb640 1 --2- 172.21.15.49:0/2257924035 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f32a0065ab0 0x7f32a0067f70 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f32b00ae000 tx=0x7f32b00b5040 comp rx=0 tx=0).stop 2023-11-06T02:08:41.156 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.158+0000 7f32c4feb640 1 -- 172.21.15.49:0/2257924035 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32c00b4e60 msgr2=0x7f32c00cd040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:41.156 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.158+0000 7f32c4feb640 1 --2- 172.21.15.49:0/2257924035 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32c00b4e60 0x7f32c00cd040 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f32b400b410 tx=0x7f32b400b8d0 comp rx=0 tx=0).stop 2023-11-06T02:08:41.157 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.158+0000 7f32c4feb640 1 -- 172.21.15.49:0/2257924035 shutdown_connections 2023-11-06T02:08:41.157 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.158+0000 7f32c4feb640 1 --2- 172.21.15.49:0/2257924035 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f32a0065ab0 0x7f32a0067f70 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:41.157 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.158+0000 7f32c4feb640 1 --2- 172.21.15.49:0/2257924035 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32c0160da0 0x7f32c00c86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:41.157 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.158+0000 7f32c4feb640 1 --2- 172.21.15.49:0/2257924035 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f32c00d0530 0x7f32c015d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:41.157 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.158+0000 7f32c4feb640 1 --2- 172.21.15.49:0/2257924035 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32c00b4e60 0x7f32c00cd040 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:41.158 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.158+0000 7f32c4feb640 1 -- 172.21.15.49:0/2257924035 >> 172.21.15.49:0/2257924035 conn(0x7f32c006af70 msgr2=0x7f32c015c510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:41.158 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.158+0000 7f32c4feb640 1 -- 172.21.15.49:0/2257924035 shutdown_connections 2023-11-06T02:08:41.158 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.158+0000 7f32c4feb640 1 -- 172.21.15.49:0/2257924035 wait complete. 2023-11-06T02:08:41.159 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.162+0000 7f357a712640 1 -- 172.21.15.49:0/4065228037 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f35540057e0 con 0x7f356c0df430 2023-11-06T02:08:41.159 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.162+0000 7f3570ff9640 1 -- 172.21.15.49:0/4065228037 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3560054220 con 0x7f356c0df430 2023-11-06T02:08:41.159 INFO:teuthology.orchestra.run.smithi049.stdout:55834574854 2023-11-06T02:08:41.160 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.1 2023-11-06T02:08:41.161 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.162+0000 7f357a712640 1 -- 172.21.15.49:0/4065228037 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3558065a60 msgr2=0x7f3558067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:41.161 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.162+0000 7f357a712640 1 --2- 172.21.15.49:0/4065228037 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3558065a60 0x7f3558067f20 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3568000c10 tx=0x7f35680a7040 comp rx=0 tx=0).stop 2023-11-06T02:08:41.162 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.162+0000 7f357a712640 1 -- 172.21.15.49:0/4065228037 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f356c0df430 msgr2=0x7f356c04df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:41.162 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.162+0000 7f357a712640 1 --2- 172.21.15.49:0/4065228037 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f356c0df430 0x7f356c04df20 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f356000c430 tx=0x7f356000c8f0 comp rx=0 tx=0).stop 2023-11-06T02:08:41.162 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.162+0000 7f357a712640 1 -- 172.21.15.49:0/4065228037 shutdown_connections 2023-11-06T02:08:41.162 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.162+0000 7f357a712640 1 --2- 172.21.15.49:0/4065228037 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3558065a60 0x7f3558067f20 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:41.163 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.162+0000 7f357a712640 1 --2- 172.21.15.49:0/4065228037 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f356c0df430 0x7f356c04df20 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:41.163 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.162+0000 7f357a712640 1 --2- 172.21.15.49:0/4065228037 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f356c0e3500 0x7f356c04e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:41.163 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.162+0000 7f357a712640 1 --2- 172.21.15.49:0/4065228037 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f356c053e20 0x7f356c04a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:41.163 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.162+0000 7f357a712640 1 -- 172.21.15.49:0/4065228037 >> 172.21.15.49:0/4065228037 conn(0x7f356c054c50 msgr2=0x7f356c0e1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:41.163 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.162+0000 7f357a712640 1 -- 172.21.15.49:0/4065228037 shutdown_connections 2023-11-06T02:08:41.163 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:41.162+0000 7f357a712640 1 -- 172.21.15.49:0/4065228037 wait complete. 2023-11-06T02:08:41.167 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.3 2023-11-06T02:08:41.170 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.7 2023-11-06T02:08:41.179 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.4 2023-11-06T02:08:41.180 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.6 2023-11-06T02:08:42.142 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-11-06T02:08:42.144 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-11-06T02:08:42.151 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-11-06T02:08:42.162 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-11-06T02:08:42.167 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-11-06T02:08:42.171 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-11-06T02:08:42.180 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-11-06T02:08:42.180 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-11-06T02:08:42.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.330+0000 7ff0c64be640 1 Processor -- start 2023-11-06T02:08:42.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.334+0000 7ff0c64be640 1 -- start start 2023-11-06T02:08:42.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.334+0000 7ff0c64be640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7ff0c0157f70 0x7ff0c01651d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:42.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.334+0000 7ff0c64be640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff0c01611e0 0x7ff0c0161640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:42.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.334+0000 7ff0c64be640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff0c006bf20 0x7ff0c006c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:42.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.334+0000 7ff0c64be640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0c01622a0 con 0x7ff0c01611e0 2023-11-06T02:08:42.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.334+0000 7ff0c64be640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0c006c8d0 con 0x7ff0c0157f70 2023-11-06T02:08:42.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.334+0000 7ff0c64be640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c0164d40 con 0x7ff0c006bf20 2023-11-06T02:08:42.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.334+0000 7ff0bffff640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7ff0c0157f70 0x7ff0c01651d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:42.334 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.334+0000 7ff0c4a5b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff0c006bf20 0x7ff0c006c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:42.334 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.334+0000 7ff0bf7fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff0c01611e0 0x7ff0c0161640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:42.334 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.334+0000 7ff0bf7fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff0c01611e0 0x7ff0c0161640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52286/0 (socket says 172.21.15.49:52286) 2023-11-06T02:08:42.334 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.334+0000 7ff0bf7fe640 1 -- 172.21.15.49:0/3846307689 learned_addr learned my addr 172.21.15.49:0/3846307689 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:42.334 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.334+0000 7ff0bf7fe640 1 -- 172.21.15.49:0/3846307689 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff0c006bf20 msgr2=0x7ff0c006c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:42.335 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.334+0000 7ff0bf7fe640 1 --2- 172.21.15.49:0/3846307689 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff0c006bf20 0x7ff0c006c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.335 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.334+0000 7ff0bf7fe640 1 -- 172.21.15.49:0/3846307689 >> v2:172.21.15.150:3300/0 conn(0x7ff0c0157f70 msgr2=0x7ff0c01651d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:42.335 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.334+0000 7ff0bf7fe640 1 --2- 172.21.15.49:0/3846307689 >> v2:172.21.15.150:3300/0 conn(0x7ff0c0157f70 0x7ff0c01651d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.335 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.334+0000 7ff0bf7fe640 1 -- 172.21.15.49:0/3846307689 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0c0165710 con 0x7ff0c01611e0 2023-11-06T02:08:42.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.338+0000 7ff0bf7fe640 1 --2- 172.21.15.49:0/3846307689 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff0c01611e0 0x7ff0c0161640 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff0c006cd20 tx=0x7ff0b801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96cf4fd1adc7eef1 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:42.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.338+0000 7ff0beffd640 1 -- 172.21.15.49:0/3846307689 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b80a57b0 con 0x7ff0c01611e0 2023-11-06T02:08:42.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.338+0000 7ff0beffd640 1 -- 172.21.15.49:0/3846307689 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0b80a5910 con 0x7ff0c01611e0 2023-11-06T02:08:42.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.338+0000 7ff0c64be640 1 -- 172.21.15.49:0/3846307689 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff0c01611e0 msgr2=0x7ff0c0161640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:42.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.338+0000 7ff0c64be640 1 --2- 172.21.15.49:0/3846307689 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff0c01611e0 0x7ff0c0161640 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff0c006cd20 tx=0x7ff0b801c930 comp rx=0 tx=0).stop 2023-11-06T02:08:42.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.338+0000 7ff0c64be640 1 -- 172.21.15.49:0/3846307689 shutdown_connections 2023-11-06T02:08:42.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.338+0000 7ff0c64be640 1 --2- 172.21.15.49:0/3846307689 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff0c006bf20 0x7ff0c006c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.338+0000 7ff0c64be640 1 --2- 172.21.15.49:0/3846307689 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff0c01611e0 0x7ff0c0161640 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.338+0000 7ff0c64be640 1 --2- 172.21.15.49:0/3846307689 >> v2:172.21.15.150:3300/0 conn(0x7ff0c0157f70 0x7ff0c01651d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.338+0000 7ff0c64be640 1 -- 172.21.15.49:0/3846307689 >> 172.21.15.49:0/3846307689 conn(0x7ff0c00ce420 msgr2=0x7ff0c00ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:42.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.342+0000 7ff0c64be640 1 -- 172.21.15.49:0/3846307689 shutdown_connections 2023-11-06T02:08:42.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.342+0000 7ff0c64be640 1 -- 172.21.15.49:0/3846307689 wait complete. 2023-11-06T02:08:42.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.342+0000 7ff0c64be640 1 Processor -- start 2023-11-06T02:08:42.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.342+0000 7ff0c64be640 1 -- start start 2023-11-06T02:08:42.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.342+0000 7ff0c64be640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff0c006bf20 0x7ff0c00c63c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:42.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.342+0000 7ff0c64be640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0c01611e0 0x7ff0c0071070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:42.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.342+0000 7ff0c64be640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff0c00c49c0 0x7ff0c015b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:42.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.342+0000 7ff0c64be640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c0149910 con 0x7ff0c00c49c0 2023-11-06T02:08:42.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.342+0000 7ff0c64be640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c00c5f30 con 0x7ff0c01611e0 2023-11-06T02:08:42.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.342+0000 7ff0c64be640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c00c4ec0 con 0x7ff0c006bf20 2023-11-06T02:08:42.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.342+0000 7ff0c4a5b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff0c00c49c0 0x7ff0c015b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:42.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.342+0000 7ff0c4a5b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff0c00c49c0 0x7ff0c015b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52296/0 (socket says 172.21.15.49:52296) 2023-11-06T02:08:42.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.342+0000 7ff0c4a5b640 1 -- 172.21.15.49:0/2735891648 learned_addr learned my addr 172.21.15.49:0/2735891648 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:42.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.342+0000 7ff0bffff640 1 --2- 172.21.15.49:0/2735891648 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff0c006bf20 0x7ff0c00c63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:42.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.346+0000 7ff0c4a5b640 1 -- 172.21.15.49:0/2735891648 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff0c006bf20 msgr2=0x7ff0c00c63c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:42.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.346+0000 7ff0bf7fe640 1 --2- 172.21.15.49:0/2735891648 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0c01611e0 0x7ff0c0071070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:42.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.346+0000 7ff0c4a5b640 1 --2- 172.21.15.49:0/2735891648 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff0c006bf20 0x7ff0c00c63c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.346+0000 7ff0c4a5b640 1 -- 172.21.15.49:0/2735891648 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0c01611e0 msgr2=0x7ff0c0071070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:42.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.346+0000 7ff0c4a5b640 1 --2- 172.21.15.49:0/2735891648 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0c01611e0 0x7ff0c0071070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.346+0000 7ff0c4a5b640 1 -- 172.21.15.49:0/2735891648 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0b8009b30 con 0x7ff0c00c49c0 2023-11-06T02:08:42.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.346+0000 7ff0bf7fe640 1 --2- 172.21.15.49:0/2735891648 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0c01611e0 0x7ff0c0071070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:08:42.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.346+0000 7ff0c4a5b640 1 --2- 172.21.15.49:0/2735891648 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff0c00c49c0 0x7ff0c015b8d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff0b400e820 tx=0x7ff0b400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:42.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.346+0000 7ff0bcff9640 1 -- 172.21.15.49:0/2735891648 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b4009530 con 0x7ff0c00c49c0 2023-11-06T02:08:42.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.346+0000 7ff0c64be640 1 -- 172.21.15.49:0/2735891648 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0c00c5600 con 0x7ff0c00c49c0 2023-11-06T02:08:42.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.346+0000 7ff0c64be640 1 -- 172.21.15.49:0/2735891648 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0c00c5bc0 con 0x7ff0c00c49c0 2023-11-06T02:08:42.350 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.346+0000 7ff0bcff9640 1 -- 172.21.15.49:0/2735891648 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0b400eed0 con 0x7ff0c00c49c0 2023-11-06T02:08:42.350 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.346+0000 7ff0bcff9640 1 -- 172.21.15.49:0/2735891648 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b400cb60 con 0x7ff0c00c49c0 2023-11-06T02:08:42.351 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.346+0000 7ff0c64be640 1 -- 172.21.15.49:0/2735891648 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff09c005260 con 0x7ff0c00c49c0 2023-11-06T02:08:42.351 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.346+0000 7ff0bcff9640 1 -- 172.21.15.49:0/2735891648 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7ff0b4013470 con 0x7ff0c00c49c0 2023-11-06T02:08:42.351 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.350+0000 7ff0bcff9640 1 --2- 172.21.15.49:0/2735891648 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff0a4065ab0 0x7ff0a4067f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:42.351 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.350+0000 7ff0bcff9640 1 -- 172.21.15.49:0/2735891648 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff0b4083210 con 0x7ff0c00c49c0 2023-11-06T02:08:42.352 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.350+0000 7ff0bffff640 1 --2- 172.21.15.49:0/2735891648 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff0a4065ab0 0x7ff0a4067f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:42.353 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.350+0000 7ff0bffff640 1 --2- 172.21.15.49:0/2735891648 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff0a4065ab0 0x7ff0a4067f70 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff0b0009a40 tx=0x7ff0b0009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:42.353 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.358+0000 7ff0bcff9640 1 -- 172.21.15.49:0/2735891648 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7ff0b4050b90 con 0x7ff0c00c49c0 2023-11-06T02:08:42.371 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.374+0000 7faa77fff640 1 Processor -- start 2023-11-06T02:08:42.371 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.374+0000 7faa77fff640 1 -- start start 2023-11-06T02:08:42.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.374+0000 7faa77fff640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7faa780cca80 0x7faa7806d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:42.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.374+0000 7faa77fff640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa780ce690 0x7faa780ceaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:42.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.374+0000 7faa77fff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7faa780b18c0 0x7faa780b1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:42.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.374+0000 7faa77fff640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa780cf030 con 0x7faa780b18c0 2023-11-06T02:08:42.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.374+0000 7faa77fff640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa7806cfd0 con 0x7faa780cca80 2023-11-06T02:08:42.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.374+0000 7faa77fff640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa78159c70 con 0x7faa780ce690 2023-11-06T02:08:42.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.374+0000 7faa777fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7faa780b18c0 0x7faa780b1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:42.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.374+0000 7faa777fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7faa780b18c0 0x7faa780b1d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52306/0 (socket says 172.21.15.49:52306) 2023-11-06T02:08:42.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.374+0000 7faa777fe640 1 -- 172.21.15.49:0/99024099 learned_addr learned my addr 172.21.15.49:0/99024099 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:42.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.378+0000 7faa76ffd640 1 --2- 172.21.15.49:0/99024099 >> v2:172.21.15.150:3300/0 conn(0x7faa780cca80 0x7faa7806d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:42.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.378+0000 7faa767fc640 1 --2- 172.21.15.49:0/99024099 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa780ce690 0x7faa780ceaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:42.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.378+0000 7faa777fe640 1 -- 172.21.15.49:0/99024099 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa780ce690 msgr2=0x7faa780ceaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:42.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.378+0000 7faa777fe640 1 --2- 172.21.15.49:0/99024099 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa780ce690 0x7faa780ceaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.378+0000 7faa777fe640 1 -- 172.21.15.49:0/99024099 >> v2:172.21.15.150:3300/0 conn(0x7faa780cca80 msgr2=0x7faa7806d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:42.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.378+0000 7faa777fe640 1 --2- 172.21.15.49:0/99024099 >> v2:172.21.15.150:3300/0 conn(0x7faa780cca80 0x7faa7806d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.378+0000 7faa777fe640 1 -- 172.21.15.49:0/99024099 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa7806dde0 con 0x7faa780b18c0 2023-11-06T02:08:42.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.378+0000 7f7657b5a640 1 Processor -- start 2023-11-06T02:08:42.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.378+0000 7f7657b5a640 1 -- start start 2023-11-06T02:08:42.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.378+0000 7f7657b5a640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f76500cb970 0x7f7650071120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:42.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.378+0000 7f7657b5a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7650071660 0x7f7650069d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:42.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.378+0000 7f7657b5a640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76500caaa0 con 0x7f765015ceb0 2023-11-06T02:08:42.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.378+0000 7f7657b5a640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7650071b10 con 0x7f76500cb970 2023-11-06T02:08:42.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.378+0000 7f7657b5a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76500ca1e0 con 0x7f7650071660 2023-11-06T02:08:42.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.378+0000 7f76558f6640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f765015ceb0 0x7f76500caf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:53994/0 (socket says 172.21.15.49:53994) 2023-11-06T02:08:42.382 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.378+0000 7f76558f6640 1 -- 172.21.15.49:0/3666405527 learned_addr learned my addr 172.21.15.49:0/3666405527 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:08:42.382 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.378+0000 7f76550f5640 1 --2- 172.21.15.49:0/3666405527 >> v2:172.21.15.150:3300/0 conn(0x7f76500cb970 0x7f7650071120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:42.382 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.378+0000 7f76560f7640 1 --2- 172.21.15.49:0/3666405527 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7650071660 0x7f7650069d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:42.382 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.378+0000 7f76548f4640 1 -- 172.21.15.49:0/3666405527 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1558508928 0 0) 0x7f76500caaa0 con 0x7f765015ceb0 2023-11-06T02:08:42.383 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.378+0000 7f76548f4640 1 -- 172.21.15.49:0/3666405527 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f763c003530 con 0x7f765015ceb0 2023-11-06T02:08:42.383 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.378+0000 7f76550f5640 1 -- 172.21.15.49:0/3666405527 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7650071660 msgr2=0x7f7650069d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:42.383 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.378+0000 7f76548f4640 1 -- 172.21.15.49:0/3666405527 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3201352152 0 0) 0x7f763c003530 con 0x7f765015ceb0 2023-11-06T02:08:42.383 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.378+0000 7f76550f5640 1 --2- 172.21.15.49:0/3666405527 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7650071660 0x7f7650069d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.383 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.378+0000 7f76550f5640 1 -- 172.21.15.49:0/3666405527 >> v1:172.21.15.49:6789/0 conn(0x7f765015ceb0 legacy=0x7f76500caf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:42.384 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.378+0000 7f76550f5640 1 -- 172.21.15.49:0/3666405527 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76500ce490 con 0x7f76500cb970 2023-11-06T02:08:42.384 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.378+0000 7faa777fe640 1 --2- 172.21.15.49:0/99024099 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa780b18c0 0x7faa780b1d50 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7faa7806d830 tx=0x7faa7001ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=719718d227625f76 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:42.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.378+0000 7faa75ffb640 1 -- 172.21.15.49:0/99024099 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa7000b2d0 con 0x7faa780b18c0 2023-11-06T02:08:42.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.382+0000 7faa75ffb640 1 -- 172.21.15.49:0/99024099 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa7000b430 con 0x7faa780b18c0 2023-11-06T02:08:42.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.382+0000 7faa75ffb640 1 -- 172.21.15.49:0/99024099 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa700a7a70 con 0x7faa780b18c0 2023-11-06T02:08:42.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.378+0000 7f9f0dbd6640 1 Processor -- start 2023-11-06T02:08:42.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.378+0000 7f9f0dbd6640 1 -- start start 2023-11-06T02:08:42.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.382+0000 7f9f0dbd6640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f9f081643e0 0x7f9f080c16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:42.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.382+0000 7f9f0dbd6640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f9f080b0790 0x7f9f080cc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:42.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.382+0000 7f9f0dbd6640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f080ce350 con 0x7f9f081647c0 2023-11-06T02:08:42.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.382+0000 7f9f0dbd6640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f08164de0 con 0x7f9f081643e0 2023-11-06T02:08:42.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.382+0000 7f9f0dbd6640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f081650c0 con 0x7f9f080b0790 2023-11-06T02:08:42.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.382+0000 7f9f077fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f9f081643e0 0x7f9f080c16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:42.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.382+0000 7faa77fff640 1 -- 172.21.15.49:0/99024099 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa780b18c0 msgr2=0x7faa780b1d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:42.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.382+0000 7faa77fff640 1 --2- 172.21.15.49:0/99024099 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa780b18c0 0x7faa780b1d50 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7faa7806d830 tx=0x7faa7001ebd0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.382+0000 7faa77fff640 1 -- 172.21.15.49:0/99024099 shutdown_connections 2023-11-06T02:08:42.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.382+0000 7faa77fff640 1 --2- 172.21.15.49:0/99024099 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa780ce690 0x7faa780ceaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.382+0000 7faa77fff640 1 --2- 172.21.15.49:0/99024099 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa780b18c0 0x7faa780b1d50 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.389 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.382+0000 7faa77fff640 1 --2- 172.21.15.49:0/99024099 >> v2:172.21.15.150:3300/0 conn(0x7faa780cca80 0x7faa7806d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.389 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.382+0000 7faa77fff640 1 -- 172.21.15.49:0/99024099 >> 172.21.15.49:0/99024099 conn(0x7faa78069f40 msgr2=0x7faa7806a350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:42.389 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.382+0000 7faa77fff640 1 -- 172.21.15.49:0/99024099 shutdown_connections 2023-11-06T02:08:42.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.382+0000 7f9f077fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f9f081643e0 0x7f9f080c16c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52320/0 (socket says 172.21.15.49:52320) 2023-11-06T02:08:42.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.382+0000 7f9f077fe640 1 -- 172.21.15.49:0/2059517203 learned_addr learned my addr 172.21.15.49:0/2059517203 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:42.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.382+0000 7f9f07fff640 1 --2- 172.21.15.49:0/2059517203 >> v2:172.21.15.150:3300/0 conn(0x7f9f080b0790 0x7f9f080cc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:42.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.382+0000 7f9f077fe640 1 -- 172.21.15.49:0/2059517203 >> v2:172.21.15.150:3300/0 conn(0x7f9f080b0790 msgr2=0x7f9f080cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:42.391 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.382+0000 7f9f077fe640 1 --2- 172.21.15.49:0/2059517203 >> v2:172.21.15.150:3300/0 conn(0x7f9f080b0790 0x7f9f080cc7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.391 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.382+0000 7f9f077fe640 1 -- 172.21.15.49:0/2059517203 >> v1:172.21.15.49:6789/0 conn(0x7f9f081647c0 legacy=0x7f9f080b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:42.391 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.382+0000 7f9f077fe640 1 -- 172.21.15.49:0/2059517203 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f081653a0 con 0x7f9f081643e0 2023-11-06T02:08:42.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.382+0000 7f76550f5640 1 --2- 172.21.15.49:0/3666405527 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76500cb970 0x7f7650071120 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f764c0097c0 tx=0x7f764c01c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39fda0dd8ca35c5 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:42.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.382+0000 7f76548f4640 1 -- 172.21.15.49:0/3666405527 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7640001680 con 0x7f76500cb970 2023-11-06T02:08:42.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.382+0000 7f76548f4640 1 -- 172.21.15.49:0/3666405527 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f764c0a7050 con 0x7f76500cb970 2023-11-06T02:08:42.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.382+0000 7f76548f4640 1 -- 172.21.15.49:0/3666405527 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764c0ab600 con 0x7f76500cb970 2023-11-06T02:08:42.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.382+0000 7faa77fff640 1 -- 172.21.15.49:0/99024099 wait complete. 2023-11-06T02:08:42.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.382+0000 7faa77fff640 1 Processor -- start 2023-11-06T02:08:42.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.382+0000 7f7657b5a640 1 -- 172.21.15.49:0/3666405527 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76500cb970 msgr2=0x7f7650071120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:42.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.382+0000 7f7657b5a640 1 --2- 172.21.15.49:0/3666405527 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76500cb970 0x7f7650071120 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f764c0097c0 tx=0x7f764c01c3a0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.382+0000 7f7657b5a640 1 -- 172.21.15.49:0/3666405527 shutdown_connections 2023-11-06T02:08:42.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.382+0000 7f7657b5a640 1 --2- 172.21.15.49:0/3666405527 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7650071660 0x7f7650069d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.382+0000 7f7657b5a640 1 --2- 172.21.15.49:0/3666405527 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76500cb970 0x7f7650071120 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.395 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.382+0000 7f7657b5a640 1 -- 172.21.15.49:0/3666405527 >> 172.21.15.49:0/3666405527 conn(0x7f76500cc140 msgr2=0x7f76500cc550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:42.395 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.382+0000 7f7657b5a640 1 -- 172.21.15.49:0/3666405527 shutdown_connections 2023-11-06T02:08:42.395 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.382+0000 7f7657b5a640 1 -- 172.21.15.49:0/3666405527 wait complete. 2023-11-06T02:08:42.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.382+0000 7faa77fff640 1 -- start start 2023-11-06T02:08:42.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.382+0000 7faa77fff640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa780b18c0 0x7faa781580f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:42.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.382+0000 7faa77fff640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa780cca80 0x7faa78159000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:42.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.382+0000 7faa77fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa780ce690 0x7faa78159540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:42.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.382+0000 7faa77fff640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa78157ae0 con 0x7faa780ce690 2023-11-06T02:08:42.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.382+0000 7faa77fff640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa78152e20 con 0x7faa780cca80 2023-11-06T02:08:42.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.382+0000 7faa77fff640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa78153100 con 0x7faa780b18c0 2023-11-06T02:08:42.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.386+0000 7f9f077fe640 1 --2- 172.21.15.49:0/2059517203 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9f081643e0 0x7f9f080c16c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9f08165cd0 tx=0x7f9f0001c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfcf6fd69836bba server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:42.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.386+0000 7faa777fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa780ce690 0x7faa78159540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:42.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.386+0000 7faa777fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa780ce690 0x7faa78159540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52328/0 (socket says 172.21.15.49:52328) 2023-11-06T02:08:42.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.386+0000 7faa777fe640 1 -- 172.21.15.49:0/435333224 learned_addr learned my addr 172.21.15.49:0/435333224 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:42.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.386+0000 7faa767fc640 1 --2- 172.21.15.49:0/435333224 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa780cca80 0x7faa78159000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:42.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.386+0000 7faa76ffd640 1 --2- 172.21.15.49:0/435333224 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa780b18c0 0x7faa781580f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:42.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.386+0000 7f9f067fc640 1 -- 172.21.15.49:0/2059517203 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f000a5880 con 0x7f9f081643e0 2023-11-06T02:08:42.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.386+0000 7f9f067fc640 1 -- 172.21.15.49:0/2059517203 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f000a59e0 con 0x7f9f081643e0 2023-11-06T02:08:42.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.386+0000 7f9f067fc640 1 -- 172.21.15.49:0/2059517203 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f000a5d70 con 0x7f9f081643e0 2023-11-06T02:08:42.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.386+0000 7fdf48c01640 1 Processor -- start 2023-11-06T02:08:42.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.386+0000 7faa777fe640 1 -- 172.21.15.49:0/435333224 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa780b18c0 msgr2=0x7faa781580f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:42.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.386+0000 7faa777fe640 1 --2- 172.21.15.49:0/435333224 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa780b18c0 0x7faa781580f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.386+0000 7faa777fe640 1 -- 172.21.15.49:0/435333224 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa780cca80 msgr2=0x7faa78159000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:42.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.386+0000 7faa777fe640 1 --2- 172.21.15.49:0/435333224 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa780cca80 0x7faa78159000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.386+0000 7faa777fe640 1 -- 172.21.15.49:0/435333224 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa7000bd40 con 0x7faa780ce690 2023-11-06T02:08:42.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.386+0000 7faa777fe640 1 --2- 172.21.15.49:0/435333224 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa780ce690 0x7faa78159540 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7faa700b1040 tx=0x7faa7001b970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:42.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.386+0000 7f7207b4c640 1 Processor -- start 2023-11-06T02:08:42.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.386+0000 7f7657b5a640 1 Processor -- start 2023-11-06T02:08:42.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.386+0000 7f7657b5a640 1 -- start start 2023-11-06T02:08:42.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.386+0000 7f7657b5a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7650071660 0x7f7650156450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:42.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.386+0000 7f7657b5a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76500cb970 0x7f76500ae230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:42.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.386+0000 7f7657b5a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f765015ceb0 0x7f7650152b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:42.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.386+0000 7f7657b5a640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76500ac000 con 0x7f7650071660 2023-11-06T02:08:42.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.386+0000 7f7657b5a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7650156e10 con 0x7f76500cb970 2023-11-06T02:08:42.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.386+0000 7f7657b5a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76500ab9f0 con 0x7f765015ceb0 2023-11-06T02:08:42.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.386+0000 7f7207b4c640 1 -- start start 2023-11-06T02:08:42.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.386+0000 7f7207b4c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7200164850 0x7f72000ca170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:42.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.386+0000 7f7207b4c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f72000cb810 0x7f72001659f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:42.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.386+0000 7f7207b4c640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7200165f30 con 0x7f72000cdc10 2023-11-06T02:08:42.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.386+0000 7f7207b4c640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72000c2d60 con 0x7f72000cb810 2023-11-06T02:08:42.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.386+0000 7f7207b4c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72000ca770 con 0x7f7200164850 2023-11-06T02:08:42.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.386+0000 7faa5ffff640 1 -- 172.21.15.49:0/435333224 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa70009800 con 0x7faa780ce690 2023-11-06T02:08:42.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.386+0000 7faa5ffff640 1 -- 172.21.15.49:0/435333224 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa70009960 con 0x7faa780ce690 2023-11-06T02:08:42.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.386+0000 7faa5ffff640 1 -- 172.21.15.49:0/435333224 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa70009d50 con 0x7faa780ce690 2023-11-06T02:08:42.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.386+0000 7faa77fff640 1 -- 172.21.15.49:0/435333224 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa781533e0 con 0x7faa780ce690 2023-11-06T02:08:42.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.386+0000 7faa77fff640 1 -- 172.21.15.49:0/435333224 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa781539a0 con 0x7faa780ce690 2023-11-06T02:08:42.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.386+0000 7f9f0dbd6640 1 -- 172.21.15.49:0/2059517203 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9f081643e0 msgr2=0x7f9f080c16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:42.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.386+0000 7f9f0dbd6640 1 --2- 172.21.15.49:0/2059517203 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9f081643e0 0x7f9f080c16c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9f08165cd0 tx=0x7f9f0001c9a0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.386+0000 7f9f0dbd6640 1 -- 172.21.15.49:0/2059517203 shutdown_connections 2023-11-06T02:08:42.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.386+0000 7f9f0dbd6640 1 --2- 172.21.15.49:0/2059517203 >> v2:172.21.15.150:3300/0 conn(0x7f9f080b0790 0x7f9f080cc7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.386+0000 7f9f0dbd6640 1 --2- 172.21.15.49:0/2059517203 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9f081643e0 0x7f9f080c16c0 secure :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9f08165cd0 tx=0x7f9f0001c9a0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.386+0000 7f9f0dbd6640 1 -- 172.21.15.49:0/2059517203 >> 172.21.15.49:0/2059517203 conn(0x7f9f080cb080 msgr2=0x7f9f080cb490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:42.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.386+0000 7f9f0dbd6640 1 -- 172.21.15.49:0/2059517203 shutdown_connections 2023-11-06T02:08:42.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.386+0000 7f72050e7640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f72000cb810 0x7f72001659f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:42.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.386+0000 7f72050e7640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f72000cb810 0x7f72001659f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52346/0 (socket says 172.21.15.49:52346) 2023-11-06T02:08:42.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.386+0000 7f72050e7640 1 -- 172.21.15.49:0/1016159659 learned_addr learned my addr 172.21.15.49:0/1016159659 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:42.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.386+0000 7f76560f7640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f765015ceb0 0x7f7650152b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:42.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.386+0000 7f76560f7640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f765015ceb0 0x7f7650152b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.49:33756/0 (socket says 172.21.15.49:33756) 2023-11-06T02:08:42.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.386+0000 7f76560f7640 1 -- 172.21.15.49:0/2560671880 learned_addr learned my addr 172.21.15.49:0/2560671880 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:42.411 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.386+0000 7f72058e8640 1 --2- 172.21.15.49:0/1016159659 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7200164850 0x7f72000ca170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:42.411 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.386+0000 7f76558f6640 1 --2- 172.21.15.49:0/2560671880 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7650071660 0x7f7650156450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:42.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.386+0000 7f9f0dbd6640 1 -- 172.21.15.49:0/2059517203 wait complete. 2023-11-06T02:08:42.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.386+0000 7f9f0dbd6640 1 Processor -- start 2023-11-06T02:08:42.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.386+0000 7f76550f5640 1 --2- 172.21.15.49:0/2560671880 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76500cb970 0x7f76500ae230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:42.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.386+0000 7fdf48c01640 1 -- start start 2023-11-06T02:08:42.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.386+0000 7fdf48c01640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fdf3c005ee0 0x7fdf3c007410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:42.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.386+0000 7fdf48c01640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fdf3c050280 0x7fdf3c054110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:42.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.386+0000 7fdf48c01640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf3c006370 con 0x7fdf3c0df460 2023-11-06T02:08:42.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.386+0000 7fdf48c01640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf3c006650 con 0x7fdf3c005ee0 2023-11-06T02:08:42.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.386+0000 7fdf48c01640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf3c006930 con 0x7fdf3c050280 2023-11-06T02:08:42.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.386+0000 7fdf437fe640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fdf3c0df460 0x7fdf3c04fb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:54018/0 (socket says 172.21.15.49:54018) 2023-11-06T02:08:42.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.386+0000 7fdf437fe640 1 -- 172.21.15.49:0/1338280599 learned_addr learned my addr 172.21.15.49:0/1338280599 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:08:42.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.386+0000 7fdf42ffd640 1 --2- 172.21.15.49:0/1338280599 >> v2:172.21.15.49:3300/0 conn(0x7fdf3c005ee0 0x7fdf3c007410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:42.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.386+0000 7fdf42ffd640 1 -- 172.21.15.49:0/1338280599 >> v2:172.21.15.150:3300/0 conn(0x7fdf3c050280 msgr2=0x7fdf3c054110 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-06T02:08:42.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.386+0000 7fdf42ffd640 1 --2- 172.21.15.49:0/1338280599 >> v2:172.21.15.150:3300/0 conn(0x7fdf3c050280 0x7fdf3c054110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.386+0000 7fdf42ffd640 1 -- 172.21.15.49:0/1338280599 >> v1:172.21.15.49:6789/0 conn(0x7fdf3c0df460 legacy=0x7fdf3c04fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:42.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.386+0000 7fdf42ffd640 1 -- 172.21.15.49:0/1338280599 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf3c006c10 con 0x7fdf3c005ee0 2023-11-06T02:08:42.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.390+0000 7f72058e8640 1 -- 172.21.15.49:0/1016159659 >> v2:172.21.15.49:3300/0 conn(0x7f72000cb810 msgr2=0x7f72001659f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:42.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.390+0000 7f72058e8640 1 --2- 172.21.15.49:0/1016159659 >> v2:172.21.15.49:3300/0 conn(0x7f72000cb810 0x7f72001659f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.390+0000 7f72058e8640 1 -- 172.21.15.49:0/1016159659 >> v1:172.21.15.150:6789/0 conn(0x7f72000cdc10 legacy=0x7f7200071120 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-06T02:08:42.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.390+0000 7f72058e8640 1 -- 172.21.15.49:0/1016159659 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72000caa50 con 0x7f7200164850 2023-11-06T02:08:42.417 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.390+0000 7f76558f6640 1 -- 172.21.15.49:0/2560671880 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f765015ceb0 msgr2=0x7f7650152b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:42.417 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.390+0000 7f76558f6640 1 --2- 172.21.15.49:0/2560671880 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f765015ceb0 0x7f7650152b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.417 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.390+0000 7f76558f6640 1 -- 172.21.15.49:0/2560671880 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76500cb970 msgr2=0x7f76500ae230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:42.418 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.390+0000 7f76558f6640 1 --2- 172.21.15.49:0/2560671880 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76500cb970 0x7f76500ae230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.418 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.390+0000 7f76558f6640 1 -- 172.21.15.49:0/2560671880 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f764c009420 con 0x7f7650071660 2023-11-06T02:08:42.418 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.390+0000 7f76560f7640 1 --2- 172.21.15.49:0/2560671880 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f765015ceb0 0x7f7650152b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T02:08:42.418 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.390+0000 7f76558f6640 1 --2- 172.21.15.49:0/2560671880 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7650071660 0x7f7650156450 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7640001390 tx=0x7f764000cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:42.419 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.386+0000 7f9f0dbd6640 1 -- start start 2023-11-06T02:08:42.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.390+0000 7f9f0dbd6640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f080b0790 0x7f9f08155ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:42.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.390+0000 7f9f0dbd6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9f081647c0 0x7f9f080c78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:42.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.390+0000 7f9f0dbd6640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9f080c7de0 0x7f9f08165d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:42.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.390+0000 7f9f0dbd6640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f080c8270 con 0x7f9f081647c0 2023-11-06T02:08:42.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.390+0000 7f9f0dbd6640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f08156020 con 0x7f9f080b0790 2023-11-06T02:08:42.422 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.390+0000 7f9f0dbd6640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f08203ac0 con 0x7f9f080c7de0 2023-11-06T02:08:42.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.390+0000 7f76467fc640 1 -- 172.21.15.49:0/2560671880 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7640005ae0 con 0x7f7650071660 2023-11-06T02:08:42.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.390+0000 7f76467fc640 1 -- 172.21.15.49:0/2560671880 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7640005c60 con 0x7f7650071660 2023-11-06T02:08:42.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.390+0000 7f76467fc640 1 -- 172.21.15.49:0/2560671880 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7640007070 con 0x7f7650071660 2023-11-06T02:08:42.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.390+0000 7f7657b5a640 1 -- 172.21.15.49:0/2560671880 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7650156b30 con 0x7f7650071660 2023-11-06T02:08:42.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.390+0000 7f7657b5a640 1 -- 172.21.15.49:0/2560671880 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76501530c0 con 0x7f7650071660 2023-11-06T02:08:42.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.390+0000 7f7657b5a640 1 -- 172.21.15.49:0/2560671880 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7650067140 con 0x7f7650071660 2023-11-06T02:08:42.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.390+0000 7fdf42ffd640 1 --2- 172.21.15.49:0/1338280599 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdf3c005ee0 0x7fdf3c007410 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c007950 tx=0x7fdf3001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1ef56d3489fc549 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:42.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.390+0000 7f72058e8640 1 --2- 172.21.15.49:0/1016159659 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7200164850 0x7f72000ca170 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f71fc0097c0 tx=0x7f71fc01c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f9e8f6d473be6d5 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:42.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.390+0000 7f72048e6640 1 -- 172.21.15.49:0/1016159659 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71fc0ac630 con 0x7f7200164850 2023-11-06T02:08:42.427 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.390+0000 7f72048e6640 1 -- 172.21.15.49:0/1016159659 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71fc0ac790 con 0x7f7200164850 2023-11-06T02:08:42.427 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.390+0000 7f72048e6640 1 -- 172.21.15.49:0/1016159659 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71fc0acb20 con 0x7f7200164850 2023-11-06T02:08:42.428 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.390+0000 7f9f07fff640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9f080c7de0 0x7f9f08165d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:42.428 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.390+0000 7f9f07fff640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9f080c7de0 0x7f9f08165d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.49:33778/0 (socket says 172.21.15.49:33778) 2023-11-06T02:08:42.428 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.390+0000 7f9f07fff640 1 -- 172.21.15.49:0/965919672 learned_addr learned my addr 172.21.15.49:0/965919672 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:42.428 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.390+0000 7f9f077fe640 1 --2- 172.21.15.49:0/965919672 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f080b0790 0x7f9f08155ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:42.428 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.390+0000 7f9f06ffd640 1 --2- 172.21.15.49:0/965919672 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9f081647c0 0x7f9f080c78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:42.429 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.390+0000 7f9f07fff640 1 -- 172.21.15.49:0/965919672 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f080b0790 msgr2=0x7f9f08155ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:42.429 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.390+0000 7f9f07fff640 1 --2- 172.21.15.49:0/965919672 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f080b0790 0x7f9f08155ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.429 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.390+0000 7f9f07fff640 1 -- 172.21.15.49:0/965919672 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9f081647c0 msgr2=0x7f9f080c78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:42.429 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.390+0000 7f9f07fff640 1 --2- 172.21.15.49:0/965919672 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9f081647c0 0x7f9f080c78a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.429 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.390+0000 7f9f07fff640 1 -- 172.21.15.49:0/965919672 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f00009b30 con 0x7f9f080c7de0 2023-11-06T02:08:42.430 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.390+0000 7f9f07fff640 1 --2- 172.21.15.49:0/965919672 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9f080c7de0 0x7f9f08165d40 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9ef800d800 tx=0x7f9ef800dcc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:42.430 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.390+0000 7f9eeffff640 1 -- 172.21.15.49:0/965919672 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef800b660 con 0x7f9f080c7de0 2023-11-06T02:08:42.430 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.390+0000 7f9eeffff640 1 -- 172.21.15.49:0/965919672 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ef800b7c0 con 0x7f9f080c7de0 2023-11-06T02:08:42.430 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.390+0000 7f9eeffff640 1 -- 172.21.15.49:0/965919672 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef800bb70 con 0x7f9f080c7de0 2023-11-06T02:08:42.431 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.390+0000 7f76467fc640 1 -- 172.21.15.49:0/2560671880 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f76400071d0 con 0x7f7650071660 2023-11-06T02:08:42.431 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.394+0000 7f76467fc640 1 --2- 172.21.15.49:0/2560671880 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7634065a90 0x7f7634067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:42.431 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.394+0000 7f9f0dbd6640 1 -- 172.21.15.49:0/965919672 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f08203190 con 0x7f9f080c7de0 2023-11-06T02:08:42.431 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.394+0000 7f9f0dbd6640 1 -- 172.21.15.49:0/965919672 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f08203750 con 0x7f9f080c7de0 2023-11-06T02:08:42.432 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.394+0000 7f7207b4c640 1 -- 172.21.15.49:0/1016159659 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7200164850 msgr2=0x7f72000ca170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:42.432 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.394+0000 7f7207b4c640 1 --2- 172.21.15.49:0/1016159659 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7200164850 0x7f72000ca170 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f71fc0097c0 tx=0x7f71fc01c410 comp rx=0 tx=0).stop 2023-11-06T02:08:42.432 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.394+0000 7fdf427fc640 1 -- 172.21.15.49:0/1338280599 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf300ac410 con 0x7fdf3c005ee0 2023-11-06T02:08:42.433 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.394+0000 7f7207b4c640 1 -- 172.21.15.49:0/1016159659 shutdown_connections 2023-11-06T02:08:42.433 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.394+0000 7f7207b4c640 1 --2- 172.21.15.49:0/1016159659 >> v2:172.21.15.49:3300/0 conn(0x7f72000cb810 0x7f72001659f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.433 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.394+0000 7f7207b4c640 1 --2- 172.21.15.49:0/1016159659 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7200164850 0x7f72000ca170 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.434 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.394+0000 7f7207b4c640 1 -- 172.21.15.49:0/1016159659 >> 172.21.15.49:0/1016159659 conn(0x7f72000cc140 msgr2=0x7f72000cc550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:42.434 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.394+0000 7f7207b4c640 1 -- 172.21.15.49:0/1016159659 shutdown_connections 2023-11-06T02:08:42.434 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.394+0000 7f76550f5640 1 --2- 172.21.15.49:0/2560671880 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7634065a90 0x7f7634067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:42.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.394+0000 7f9f0dbd6640 1 -- 172.21.15.49:0/965919672 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f08074040 con 0x7f9f080c7de0 2023-11-06T02:08:42.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.386+0000 7f4a2c63f640 1 Processor -- start 2023-11-06T02:08:42.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.386+0000 7f4a2c63f640 1 -- start start 2023-11-06T02:08:42.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.386+0000 7f4a2c63f640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4a1c0e1090 0x7f4a1c03c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:42.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.386+0000 7f4a2c63f640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f4a1c0550d0 0x7f4a1c03a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:42.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.386+0000 7f4a2c63f640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a1c0e1470 con 0x7f4a1c0568e0 2023-11-06T02:08:42.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.386+0000 7f4a2c63f640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a1c055580 con 0x7f4a1c0550d0 2023-11-06T02:08:42.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.386+0000 7f4a2c63f640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1c055840 con 0x7f4a1c0e1090 2023-11-06T02:08:42.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.386+0000 7f4a2a3db640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4a1c0e1090 0x7f4a1c03c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:42.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.386+0000 7f4a2a3db640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4a1c0e1090 0x7f4a1c03c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.49:33766/0 (socket says 172.21.15.49:33766) 2023-11-06T02:08:42.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.386+0000 7f4a2a3db640 1 -- 172.21.15.49:0/1641500620 learned_addr learned my addr 172.21.15.49:0/1641500620 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:42.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.386+0000 7f4a2abdc640 1 --2- 172.21.15.49:0/1641500620 >> v2:172.21.15.150:3300/0 conn(0x7f4a1c0550d0 0x7f4a1c03a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:42.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.390+0000 7f4a2abdc640 1 -- 172.21.15.49:0/1641500620 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4a1c0e1090 msgr2=0x7f4a1c03c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:42.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.390+0000 7f4a2abdc640 1 --2- 172.21.15.49:0/1641500620 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4a1c0e1090 0x7f4a1c03c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.390+0000 7f4a2abdc640 1 -- 172.21.15.49:0/1641500620 >> v1:172.21.15.150:6789/0 conn(0x7f4a1c0568e0 legacy=0x7f4a1c039e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:42.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.390+0000 7f4a2abdc640 1 -- 172.21.15.49:0/1641500620 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a1c055b20 con 0x7f4a1c0550d0 2023-11-06T02:08:42.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.390+0000 7f4a2abdc640 1 --2- 172.21.15.49:0/1641500620 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a1c0550d0 0x7f4a1c03a560 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4a24000b70 tx=0x7f4a24098b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2d06a67d6b4beac server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:42.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.390+0000 7f4a293d9640 1 -- 172.21.15.49:0/1641500620 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a24123780 con 0x7f4a1c0550d0 2023-11-06T02:08:42.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.390+0000 7f4a293d9640 1 -- 172.21.15.49:0/1641500620 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a24123920 con 0x7f4a1c0550d0 2023-11-06T02:08:42.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.390+0000 7f4a293d9640 1 -- 172.21.15.49:0/1641500620 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a24123d50 con 0x7f4a1c0550d0 2023-11-06T02:08:42.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.390+0000 7f4a2c63f640 1 -- 172.21.15.49:0/1641500620 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a1c0550d0 msgr2=0x7f4a1c03a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:42.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.390+0000 7f4a2c63f640 1 --2- 172.21.15.49:0/1641500620 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a1c0550d0 0x7f4a1c03a560 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4a24000b70 tx=0x7f4a24098b90 comp rx=0 tx=0).stop 2023-11-06T02:08:42.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.390+0000 7f4a2c63f640 1 -- 172.21.15.49:0/1641500620 shutdown_connections 2023-11-06T02:08:42.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.390+0000 7f4a2c63f640 1 --2- 172.21.15.49:0/1641500620 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a1c0550d0 0x7f4a1c03a560 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.394+0000 7f4a2c63f640 1 --2- 172.21.15.49:0/1641500620 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4a1c0e1090 0x7f4a1c03c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.394+0000 7f4a2c63f640 1 -- 172.21.15.49:0/1641500620 >> 172.21.15.49:0/1641500620 conn(0x7f4a1c056d20 msgr2=0x7f4a1c057130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:42.443 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.394+0000 7f4a2c63f640 1 -- 172.21.15.49:0/1641500620 shutdown_connections 2023-11-06T02:08:42.443 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.394+0000 7f4a2c63f640 1 -- 172.21.15.49:0/1641500620 wait complete. 2023-11-06T02:08:42.444 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.394+0000 7f4a2c63f640 1 Processor -- start 2023-11-06T02:08:42.444 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.394+0000 7f4a2c63f640 1 -- start start 2023-11-06T02:08:42.444 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.394+0000 7f4a2c63f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a1c0550d0 0x7f4a1c0d0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:42.445 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.394+0000 7f4a2c63f640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a1c0568e0 0x7f4a1c0e3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:42.445 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.394+0000 7f4a2c63f640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4a1c0e1090 0x7f4a1c04bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:42.445 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.394+0000 7f4a2c63f640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1c0d1240 con 0x7f4a1c0550d0 2023-11-06T02:08:42.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.394+0000 7f4a2c63f640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1c0e2cc0 con 0x7f4a1c0568e0 2023-11-06T02:08:42.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.394+0000 7f4a2c63f640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1c1ba050 con 0x7f4a1c0e1090 2023-11-06T02:08:42.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.394+0000 7f4a2a3db640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a1c0550d0 0x7f4a1c0d0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:42.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.394+0000 7f4a2a3db640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a1c0550d0 0x7f4a1c0d0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52390/0 (socket says 172.21.15.49:52390) 2023-11-06T02:08:42.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.394+0000 7f4a2a3db640 1 -- 172.21.15.49:0/172701062 learned_addr learned my addr 172.21.15.49:0/172701062 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:42.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.394+0000 7f4a2abdc640 1 --2- 172.21.15.49:0/172701062 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4a1c0e1090 0x7f4a1c04bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:42.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.394+0000 7f4a29bda640 1 --2- 172.21.15.49:0/172701062 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a1c0568e0 0x7f4a1c0e3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:42.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.394+0000 7f76550f5640 1 --2- 172.21.15.49:0/2560671880 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7634065a90 0x7f7634067f50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f764c0ad000 tx=0x7f764c0a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:42.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.394+0000 7f7207b4c640 1 -- 172.21.15.49:0/1016159659 wait complete. 2023-11-06T02:08:42.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.394+0000 7f7207b4c640 1 Processor -- start 2023-11-06T02:08:42.451 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.394+0000 7fdf427fc640 1 -- 172.21.15.49:0/1338280599 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf300ac570 con 0x7fdf3c005ee0 2023-11-06T02:08:42.452 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.394+0000 7faa77fff640 1 -- 172.21.15.49:0/435333224 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa78074040 con 0x7faa780ce690 2023-11-06T02:08:42.453 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.394+0000 7f4a29bda640 1 -- 172.21.15.49:0/172701062 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4a1c0e1090 msgr2=0x7f4a1c04bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:42.453 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.394+0000 7f4a29bda640 1 --2- 172.21.15.49:0/172701062 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4a1c0e1090 0x7f4a1c04bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.453 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.394+0000 7f4a29bda640 1 -- 172.21.15.49:0/172701062 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a1c0550d0 msgr2=0x7f4a1c0d0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:42.454 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.394+0000 7f4a29bda640 1 --2- 172.21.15.49:0/172701062 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a1c0550d0 0x7f4a1c0d0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.454 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.394+0000 7f4a29bda640 1 -- 172.21.15.49:0/172701062 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a2404e5c0 con 0x7f4a1c0568e0 2023-11-06T02:08:42.455 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.394+0000 7f4a29bda640 1 --2- 172.21.15.49:0/172701062 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a1c0568e0 0x7f4a1c0e3440 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4a14010bb0 tx=0x7f4a1400e430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:42.455 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.394+0000 7faa5ffff640 1 -- 172.21.15.49:0/435333224 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7faa700c3460 con 0x7faa780ce690 2023-11-06T02:08:42.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.394+0000 7f4a1affd640 1 -- 172.21.15.49:0/172701062 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1400ed40 con 0x7f4a1c0568e0 2023-11-06T02:08:42.457 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.394+0000 7f4a1affd640 1 -- 172.21.15.49:0/172701062 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a140050d0 con 0x7f4a1c0568e0 2023-11-06T02:08:42.457 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.394+0000 7f4a1affd640 1 -- 172.21.15.49:0/172701062 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1400eea0 con 0x7f4a1c0568e0 2023-11-06T02:08:42.457 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.394+0000 7f4a2c63f640 1 -- 172.21.15.49:0/172701062 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a1c0e20b0 con 0x7f4a1c0568e0 2023-11-06T02:08:42.458 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.394+0000 7f9eeffff640 1 -- 172.21.15.49:0/965919672 <== mon.2 v2:172.21.15.150:3301/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f9ef800bcd0 con 0x7f9f080c7de0 2023-11-06T02:08:42.458 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.394+0000 7f9eeffff640 1 --2- 172.21.15.49:0/965919672 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9ef4065b30 0x7f9ef4067ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:42.459 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.394+0000 7f9eeffff640 1 -- 172.21.15.49:0/965919672 <== mon.2 v2:172.21.15.150:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9ef8084270 con 0x7f9f080c7de0 2023-11-06T02:08:42.459 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.398+0000 7f76467fc640 1 -- 172.21.15.49:0/2560671880 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7640053b80 con 0x7f7650071660 2023-11-06T02:08:42.460 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.398+0000 7fdf48c01640 1 -- 172.21.15.49:0/1338280599 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdf3c005ee0 msgr2=0x7fdf3c007410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:42.460 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.398+0000 7fdf48c01640 1 --2- 172.21.15.49:0/1338280599 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdf3c005ee0 0x7fdf3c007410 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c007950 tx=0x7fdf3001c4a0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.461 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.398+0000 7f9f077fe640 1 --2- 172.21.15.49:0/965919672 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9ef4065b30 0x7f9ef4067ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:42.462 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.398+0000 7f7207b4c640 1 -- start start 2023-11-06T02:08:42.462 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.398+0000 7faa5ffff640 1 --2- 172.21.15.49:0/435333224 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7faa64065b30 0x7faa64067ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:42.463 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.398+0000 7faa5ffff640 1 -- 172.21.15.49:0/435333224 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7faa7011e7e0 con 0x7faa780ce690 2023-11-06T02:08:42.463 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.398+0000 7f4a2c63f640 1 -- 172.21.15.49:0/172701062 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a1c0e2670 con 0x7f4a1c0568e0 2023-11-06T02:08:42.464 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.398+0000 7f4a2abdc640 1 --2- 172.21.15.49:0/172701062 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4a1c0e1090 0x7f4a1c04bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T02:08:42.464 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.398+0000 7faa76ffd640 1 --2- 172.21.15.49:0/435333224 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7faa64065b30 0x7faa64067ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:42.465 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.398+0000 7f4a2c63f640 1 -- 172.21.15.49:0/172701062 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a1c00ac50 con 0x7f4a1c0568e0 2023-11-06T02:08:42.465 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.398+0000 7f9f077fe640 1 --2- 172.21.15.49:0/965919672 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9ef4065b30 0x7f9ef4067ff0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9f000ae000 tx=0x7f9f000a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:42.465 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.398+0000 7faa76ffd640 1 --2- 172.21.15.49:0/435333224 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7faa64065b30 0x7faa64067ff0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7faa68009ae0 tx=0x7faa68009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:42.467 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.398+0000 7f7207b4c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72000cb810 0x7f72000c5f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:42.467 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.398+0000 7f7207b4c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f72000cdc10 0x7f72001579f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:42.467 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.398+0000 7f72050e7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f72000cdc10 0x7f72001579f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:42.467 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.398+0000 7f72050e7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f72000cdc10 0x7f72001579f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52398/0 (socket says 172.21.15.49:52398) 2023-11-06T02:08:42.468 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.398+0000 7f72050e7640 1 -- 172.21.15.49:0/62447651 learned_addr learned my addr 172.21.15.49:0/62447651 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:42.468 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.398+0000 7f72058e8640 1 --2- 172.21.15.49:0/62447651 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72000cb810 0x7f72000c5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:42.468 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.398+0000 7f7207b4c640 1 --2- 172.21.15.49:0/62447651 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7200164850 0x7f72000b1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:42.468 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.398+0000 7f7207b4c640 1 -- 172.21.15.49:0/62447651 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7200159010 con 0x7f72000cdc10 2023-11-06T02:08:42.468 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.398+0000 7f7207b4c640 1 -- 172.21.15.49:0/62447651 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7200158400 con 0x7f72000cb810 2023-11-06T02:08:42.469 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.398+0000 7f7207b4c640 1 -- 172.21.15.49:0/62447651 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72001586e0 con 0x7f7200164850 2023-11-06T02:08:42.469 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.398+0000 7f72050e7640 1 -- 172.21.15.49:0/62447651 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7200164850 msgr2=0x7f72000b1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:42.469 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.398+0000 7f72050e7640 1 --2- 172.21.15.49:0/62447651 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7200164850 0x7f72000b1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.469 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.398+0000 7f72050e7640 1 -- 172.21.15.49:0/62447651 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72000cb810 msgr2=0x7f72000c5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:42.469 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.398+0000 7f72050e7640 1 --2- 172.21.15.49:0/62447651 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72000cb810 0x7f72000c5f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.469 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.398+0000 7f72050e7640 1 -- 172.21.15.49:0/62447651 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71fc009420 con 0x7f72000cdc10 2023-11-06T02:08:42.470 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.398+0000 7fdf48c01640 1 -- 172.21.15.49:0/1338280599 shutdown_connections 2023-11-06T02:08:42.470 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.398+0000 7fdf48c01640 1 --2- 172.21.15.49:0/1338280599 >> v2:172.21.15.150:3300/0 conn(0x7fdf3c050280 0x7fdf3c054110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.470 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.398+0000 7fdf48c01640 1 --2- 172.21.15.49:0/1338280599 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdf3c005ee0 0x7fdf3c007410 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.470 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.398+0000 7fdf48c01640 1 -- 172.21.15.49:0/1338280599 >> 172.21.15.49:0/1338280599 conn(0x7fdf3c054b10 msgr2=0x7fdf3c054f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:42.471 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.398+0000 7f4a1affd640 1 -- 172.21.15.49:0/172701062 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f4a14003600 con 0x7f4a1c0568e0 2023-11-06T02:08:42.471 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.398+0000 7f72050e7640 1 --2- 172.21.15.49:0/62447651 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f72000cdc10 0x7f72001579f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f71f000d7d0 tx=0x7f71f000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:42.472 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.398+0000 7fdf48c01640 1 -- 172.21.15.49:0/1338280599 shutdown_connections 2023-11-06T02:08:42.472 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.398+0000 7f4a1affd640 1 --2- 172.21.15.49:0/172701062 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4a08065a90 0x7f4a08067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:42.472 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.398+0000 7f4a1affd640 1 -- 172.21.15.49:0/172701062 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4a14087dc0 con 0x7f4a1c0568e0 2023-11-06T02:08:42.473 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.398+0000 7f76467fc640 1 -- 172.21.15.49:0/2560671880 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f764004fec0 con 0x7f7650071660 2023-11-06T02:08:42.473 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.398+0000 7f4a2a3db640 1 --2- 172.21.15.49:0/172701062 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4a08065a90 0x7f4a08067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:42.474 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.398+0000 7fdf48c01640 1 -- 172.21.15.49:0/1338280599 wait complete. 2023-11-06T02:08:42.474 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.402+0000 7f2251a8c640 1 Processor -- start 2023-11-06T02:08:42.474 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.402+0000 7f2251a8c640 1 -- start start 2023-11-06T02:08:42.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.402+0000 7f4a2a3db640 1 --2- 172.21.15.49:0/172701062 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4a08065a90 0x7f4a08067f50 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4a20006f50 tx=0x7f4a20009240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:42.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.402+0000 7f2251a8c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f22440e3250 0x7f2244055380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:42.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.402+0000 7f2251a8c640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f22440e7a50 0x7f22440517d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:42.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.402+0000 7f2251a8c640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2244008ea0 con 0x7f2244051d10 2023-11-06T02:08:42.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.402+0000 7f2251a8c640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2244055cd0 con 0x7f22440e3250 2023-11-06T02:08:42.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.402+0000 7f2251a8c640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2244008570 con 0x7f22440e7a50 2023-11-06T02:08:42.477 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.402+0000 7fdf48c01640 1 Processor -- start 2023-11-06T02:08:42.478 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.402+0000 7f224bfff640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f22440e7a50 0x7f22440517d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:42.478 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.402+0000 7f224bfff640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f22440e7a50 0x7f22440517d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.49:53730/0 (socket says 172.21.15.49:53730) 2023-11-06T02:08:42.478 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.402+0000 7f224bfff640 1 -- 172.21.15.49:0/1105601824 learned_addr learned my addr 172.21.15.49:0/1105601824 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:42.478 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.402+0000 7f224bfff640 1 -- 172.21.15.49:0/1105601824 >> v2:172.21.15.49:3300/0 conn(0x7f22440e3250 msgr2=0x7f2244055380 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-06T02:08:42.478 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.402+0000 7f224bfff640 1 --2- 172.21.15.49:0/1105601824 >> v2:172.21.15.49:3300/0 conn(0x7f22440e3250 0x7f2244055380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.479 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.402+0000 7f224bfff640 1 -- 172.21.15.49:0/1105601824 >> v1:172.21.15.150:6789/0 conn(0x7f2244051d10 legacy=0x7f22440388b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:42.479 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.402+0000 7f224bfff640 1 -- 172.21.15.49:0/1105601824 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2244008850 con 0x7f22440e7a50 2023-11-06T02:08:42.480 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.402+0000 7fdf48c01640 1 -- start start 2023-11-06T02:08:42.481 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.402+0000 7fdf48c01640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf3c050280 0x7fdf3c0cd340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:42.481 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.402+0000 7fdf48c01640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdf3c0df460 0x7fdf3c0d5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:42.481 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.402+0000 7fdf48c01640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf3c0d1090 0x7fdf3c0dca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:42.481 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.402+0000 7fdf48c01640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3c0d2210 con 0x7fdf3c0df460 2023-11-06T02:08:42.481 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.402+0000 7fdf48c01640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3c0d1600 con 0x7fdf3c050280 2023-11-06T02:08:42.482 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.402+0000 7fdf48c01640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3c0d18e0 con 0x7fdf3c0d1090 2023-11-06T02:08:42.482 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.402+0000 7fdf437fe640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf3c050280 0x7fdf3c0cd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:42.482 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.402+0000 7fdf42ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdf3c0df460 0x7fdf3c0d5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:42.482 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.402+0000 7fdf437fe640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf3c050280 0x7fdf3c0cd340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.49:53734/0 (socket says 172.21.15.49:53734) 2023-11-06T02:08:42.482 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.402+0000 7fdf42ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdf3c0df460 0x7fdf3c0d5170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52412/0 (socket says 172.21.15.49:52412) 2023-11-06T02:08:42.483 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.402+0000 7fdf437fe640 1 -- 172.21.15.49:0/3470940351 learned_addr learned my addr 172.21.15.49:0/3470940351 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:42.483 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.402+0000 7fdf43fff640 1 --2- 172.21.15.49:0/3470940351 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf3c0d1090 0x7fdf3c0dca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:42.483 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.402+0000 7fdf437fe640 1 -- 172.21.15.49:0/3470940351 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf3c0d1090 msgr2=0x7fdf3c0dca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:42.483 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.402+0000 7fdf437fe640 1 --2- 172.21.15.49:0/3470940351 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf3c0d1090 0x7fdf3c0dca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.483 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.402+0000 7fdf437fe640 1 -- 172.21.15.49:0/3470940351 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdf3c0df460 msgr2=0x7fdf3c0d5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:42.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.402+0000 7fdf437fe640 1 --2- 172.21.15.49:0/3470940351 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdf3c0df460 0x7fdf3c0d5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.402+0000 7fdf437fe640 1 -- 172.21.15.49:0/3470940351 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf30009420 con 0x7fdf3c050280 2023-11-06T02:08:42.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.402+0000 7fdf42ffd640 1 --2- 172.21.15.49:0/3470940351 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdf3c0df460 0x7fdf3c0d5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T02:08:42.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.406+0000 7fdf437fe640 1 --2- 172.21.15.49:0/3470940351 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf3c050280 0x7fdf3c0cd340 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fdf4404f9f0 tx=0x7fdf440733b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:42.485 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.406+0000 7f224bfff640 1 --2- 172.21.15.49:0/1105601824 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22440e7a50 0x7f22440517d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2244056bc0 tx=0x7f223801c540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74d178f4f57415e4 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:42.485 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.406+0000 7f71f67fc640 1 -- 172.21.15.49:0/62447651 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f00096a0 con 0x7f72000cdc10 2023-11-06T02:08:42.485 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.406+0000 7f71f67fc640 1 -- 172.21.15.49:0/62447651 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71f0009800 con 0x7f72000cdc10 2023-11-06T02:08:42.486 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.406+0000 7f71f67fc640 1 -- 172.21.15.49:0/62447651 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f0009bb0 con 0x7f72000cdc10 2023-11-06T02:08:42.486 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.406+0000 7faa5ffff640 1 -- 172.21.15.49:0/435333224 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7faa700ec190 con 0x7faa780ce690 2023-11-06T02:08:42.486 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.406+0000 7f7207b4c640 1 -- 172.21.15.49:0/62447651 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72001589c0 con 0x7f72000cdc10 2023-11-06T02:08:42.487 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.406+0000 7f7207b4c640 1 -- 172.21.15.49:0/62447651 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72000c6450 con 0x7f72000cdc10 2023-11-06T02:08:42.487 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.410+0000 7f7207b4c640 1 -- 172.21.15.49:0/62447651 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72001591c0 con 0x7f72000cdc10 2023-11-06T02:08:42.487 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.410+0000 7f9eeffff640 1 -- 172.21.15.49:0/965919672 <== mon.2 v2:172.21.15.150:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f9ef8051b40 con 0x7f9f080c7de0 2023-11-06T02:08:42.488 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.410+0000 7f71f67fc640 1 -- 172.21.15.49:0/62447651 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f71f0009d10 con 0x7f72000cdc10 2023-11-06T02:08:42.488 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.410+0000 7f71f67fc640 1 --2- 172.21.15.49:0/62447651 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f71e4065a60 0x7f71e4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:42.488 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.410+0000 7f71f67fc640 1 -- 172.21.15.49:0/62447651 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f71f0083e90 con 0x7f72000cdc10 2023-11-06T02:08:42.489 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.410+0000 7f72058e8640 1 --2- 172.21.15.49:0/62447651 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f71e4065a60 0x7f71e4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:42.489 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.414+0000 7f72058e8640 1 --2- 172.21.15.49:0/62447651 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f71e4065a60 0x7f71e4067f20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f71fc0ae000 tx=0x7f71fc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:42.489 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.414+0000 7fdf2ffff640 1 -- 172.21.15.49:0/3470940351 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf44067270 con 0x7fdf3c050280 2023-11-06T02:08:42.490 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.414+0000 7fdf2ffff640 1 -- 172.21.15.49:0/3470940351 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf44073b50 con 0x7fdf3c050280 2023-11-06T02:08:42.490 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.414+0000 7fdf48c01640 1 -- 172.21.15.49:0/3470940351 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf3c0d1bc0 con 0x7fdf3c050280 2023-11-06T02:08:42.490 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.414+0000 7fdf2ffff640 1 -- 172.21.15.49:0/3470940351 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf44072440 con 0x7fdf3c050280 2023-11-06T02:08:42.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.414+0000 7f4a1affd640 1 -- 172.21.15.49:0/172701062 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f4a140557b0 con 0x7f4a1c0568e0 2023-11-06T02:08:42.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.414+0000 7fdf48c01640 1 -- 172.21.15.49:0/3470940351 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf3c0cd880 con 0x7fdf3c050280 2023-11-06T02:08:42.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.414+0000 7f224b7fe640 1 -- 172.21.15.49:0/1105601824 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22380ab5f0 con 0x7f22440e7a50 2023-11-06T02:08:42.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.414+0000 7f224b7fe640 1 -- 172.21.15.49:0/1105601824 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22380ab750 con 0x7f22440e7a50 2023-11-06T02:08:42.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.414+0000 7f224b7fe640 1 -- 172.21.15.49:0/1105601824 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22380abae0 con 0x7f22440e7a50 2023-11-06T02:08:42.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.414+0000 7fdf48c01640 1 -- 172.21.15.49:0/3470940351 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf3c008a20 con 0x7fdf3c050280 2023-11-06T02:08:42.493 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.414+0000 7f2251a8c640 1 -- 172.21.15.49:0/1105601824 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22440e7a50 msgr2=0x7f22440517d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:42.493 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.414+0000 7f2251a8c640 1 --2- 172.21.15.49:0/1105601824 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22440e7a50 0x7f22440517d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2244056bc0 tx=0x7f223801c540 comp rx=0 tx=0).stop 2023-11-06T02:08:42.494 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.414+0000 7f2251a8c640 1 -- 172.21.15.49:0/1105601824 shutdown_connections 2023-11-06T02:08:42.494 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.414+0000 7f2251a8c640 1 --2- 172.21.15.49:0/1105601824 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22440e7a50 0x7f22440517d0 secure :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2244056bc0 tx=0x7f223801c540 comp rx=0 tx=0).stop 2023-11-06T02:08:42.494 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.414+0000 7f2251a8c640 1 --2- 172.21.15.49:0/1105601824 >> v2:172.21.15.49:3300/0 conn(0x7f22440e3250 0x7f2244055380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.494 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.414+0000 7f2251a8c640 1 -- 172.21.15.49:0/1105601824 >> 172.21.15.49:0/1105601824 conn(0x7f22440538b0 msgr2=0x7f2244053cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:42.494 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.418+0000 7f2251a8c640 1 -- 172.21.15.49:0/1105601824 shutdown_connections 2023-11-06T02:08:42.495 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.418+0000 7fdf2ffff640 1 -- 172.21.15.49:0/3470940351 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fdf440923c0 con 0x7fdf3c050280 2023-11-06T02:08:42.495 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.418+0000 7fdf2ffff640 1 --2- 172.21.15.49:0/3470940351 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fdf28065a90 0x7fdf28067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:42.495 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.418+0000 7fdf2ffff640 1 -- 172.21.15.49:0/3470940351 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdf440b3020 con 0x7fdf3c050280 2023-11-06T02:08:42.496 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.418+0000 7f71f67fc640 1 -- 172.21.15.49:0/62447651 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f71f0051760 con 0x7f72000cdc10 2023-11-06T02:08:42.496 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.418+0000 7fdf42ffd640 1 --2- 172.21.15.49:0/3470940351 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fdf28065a90 0x7fdf28067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:42.496 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.418+0000 7f2251a8c640 1 -- 172.21.15.49:0/1105601824 wait complete. 2023-11-06T02:08:42.497 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.418+0000 7fdf42ffd640 1 --2- 172.21.15.49:0/3470940351 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fdf28065a90 0x7fdf28067f50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fdf300ae000 tx=0x7fdf300a6040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:42.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.418+0000 7f2251a8c640 1 Processor -- start 2023-11-06T02:08:42.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.418+0000 7f2251a8c640 1 -- start start 2023-11-06T02:08:42.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.418+0000 7f2251a8c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2244051d10 0x7f224404d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:42.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.418+0000 7f2251a8c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f22440e3250 0x7f22440ddd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:42.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.418+0000 7f2251a8c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22440de240 0x7f22440e5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:42.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.418+0000 7f2251a8c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22440e4770 con 0x7f2244051d10 2023-11-06T02:08:42.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.418+0000 7f2251a8c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22440de6d0 con 0x7f22440de240 2023-11-06T02:08:42.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.418+0000 7f2251a8c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22440e38a0 con 0x7f22440e3250 2023-11-06T02:08:42.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.418+0000 7f2250a8a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2244051d10 0x7f224404d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:42.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.418+0000 7f2250a8a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2244051d10 0x7f224404d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52426/0 (socket says 172.21.15.49:52426) 2023-11-06T02:08:42.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.418+0000 7f2250a8a640 1 -- 172.21.15.49:0/3555009280 learned_addr learned my addr 172.21.15.49:0/3555009280 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:42.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.422+0000 7f225128b640 1 --2- 172.21.15.49:0/3555009280 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22440de240 0x7f22440e5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:42.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.422+0000 7f224bfff640 1 --2- 172.21.15.49:0/3555009280 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f22440e3250 0x7f22440ddd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:42.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.422+0000 7f2250a8a640 1 -- 172.21.15.49:0/3555009280 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f22440e3250 msgr2=0x7f22440ddd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:42.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.422+0000 7f2250a8a640 1 --2- 172.21.15.49:0/3555009280 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f22440e3250 0x7f22440ddd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.422+0000 7f2250a8a640 1 -- 172.21.15.49:0/3555009280 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22440de240 msgr2=0x7f22440e5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:42.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.422+0000 7f2250a8a640 1 --2- 172.21.15.49:0/3555009280 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22440de240 0x7f22440e5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.422+0000 7f2250a8a640 1 -- 172.21.15.49:0/3555009280 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22380094d0 con 0x7f2244051d10 2023-11-06T02:08:42.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.422+0000 7f225128b640 1 --2- 172.21.15.49:0/3555009280 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22440de240 0x7f22440e5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:08:42.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.422+0000 7f2250a8a640 1 --2- 172.21.15.49:0/3555009280 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2244051d10 0x7f224404d5b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f224000b3f0 tx=0x7f224000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:42.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.422+0000 7f22497fa640 1 -- 172.21.15.49:0/3555009280 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2240011830 con 0x7f2244051d10 2023-11-06T02:08:42.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.422+0000 7f2251a8c640 1 -- 172.21.15.49:0/3555009280 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22440e3af0 con 0x7f2244051d10 2023-11-06T02:08:42.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.422+0000 7f2251a8c640 1 -- 172.21.15.49:0/3555009280 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22440e40b0 con 0x7f2244051d10 2023-11-06T02:08:42.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.422+0000 7f22497fa640 1 -- 172.21.15.49:0/3555009280 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22400119b0 con 0x7f2244051d10 2023-11-06T02:08:42.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.422+0000 7f22497fa640 1 -- 172.21.15.49:0/3555009280 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2240011d60 con 0x7f2244051d10 2023-11-06T02:08:42.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.422+0000 7f2251a8c640 1 -- 172.21.15.49:0/3555009280 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22440e48b0 con 0x7f2244051d10 2023-11-06T02:08:42.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.422+0000 7f22497fa640 1 -- 172.21.15.49:0/3555009280 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f224001a400 con 0x7f2244051d10 2023-11-06T02:08:42.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.422+0000 7f22497fa640 1 --2- 172.21.15.49:0/3555009280 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2230065a60 0x7f2230067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:42.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.426+0000 7f22497fa640 1 -- 172.21.15.49:0/3555009280 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f22400833f0 con 0x7f2244051d10 2023-11-06T02:08:42.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.426+0000 7fdf2ffff640 1 -- 172.21.15.49:0/3470940351 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fdf440ce2f0 con 0x7fdf3c050280 2023-11-06T02:08:42.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.434+0000 7f224bfff640 1 --2- 172.21.15.49:0/3555009280 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2230065a60 0x7f2230067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:42.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.434+0000 7f224bfff640 1 --2- 172.21.15.49:0/3555009280 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2230065a60 0x7f2230067f20 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f22380af000 tx=0x7f22380a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:42.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.438+0000 7f22497fa640 1 -- 172.21.15.49:0/3555009280 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f22440e48b0 con 0x7f2244051d10 2023-11-06T02:08:42.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.686+0000 7ff0c64be640 1 -- 172.21.15.49:0/2735891648 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7ff09c005510 con 0x7ff0c00c49c0 2023-11-06T02:08:42.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.686+0000 7ff0bcff9640 1 -- 172.21.15.49:0/2735891648 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff0b40541b0 con 0x7ff0c00c49c0 2023-11-06T02:08:42.684 INFO:teuthology.orchestra.run.smithi049.stdout:55834574854 2023-11-06T02:08:42.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.694+0000 7ff0ae7fc640 1 -- 172.21.15.49:0/2735891648 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff0a4065ab0 msgr2=0x7ff0a4067f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:42.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.694+0000 7ff0ae7fc640 1 --2- 172.21.15.49:0/2735891648 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff0a4065ab0 0x7ff0a4067f70 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff0b0009a40 tx=0x7ff0b0009190 comp rx=0 tx=0).stop 2023-11-06T02:08:42.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.694+0000 7ff0ae7fc640 1 -- 172.21.15.49:0/2735891648 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff0c00c49c0 msgr2=0x7ff0c015b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:42.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.694+0000 7ff0ae7fc640 1 --2- 172.21.15.49:0/2735891648 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff0c00c49c0 0x7ff0c015b8d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff0b400e820 tx=0x7ff0b400ece0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.694+0000 7ff0ae7fc640 1 -- 172.21.15.49:0/2735891648 shutdown_connections 2023-11-06T02:08:42.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.694+0000 7ff0ae7fc640 1 --2- 172.21.15.49:0/2735891648 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff0a4065ab0 0x7ff0a4067f70 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.694+0000 7ff0ae7fc640 1 --2- 172.21.15.49:0/2735891648 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff0c00c49c0 0x7ff0c015b8d0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.694+0000 7ff0ae7fc640 1 --2- 172.21.15.49:0/2735891648 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0c01611e0 0x7ff0c0071070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.694+0000 7ff0ae7fc640 1 --2- 172.21.15.49:0/2735891648 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff0c006bf20 0x7ff0c00c63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.694+0000 7ff0ae7fc640 1 -- 172.21.15.49:0/2735891648 >> 172.21.15.49:0/2735891648 conn(0x7ff0c00ce420 msgr2=0x7ff0c015a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:42.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.694+0000 7ff0ae7fc640 1 -- 172.21.15.49:0/2735891648 shutdown_connections 2023-11-06T02:08:42.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.694+0000 7ff0ae7fc640 1 -- 172.21.15.49:0/2735891648 wait complete. 2023-11-06T02:08:42.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.698+0000 7f7657b5a640 1 -- 172.21.15.49:0/2560671880 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f76500c9a60 con 0x7f7650071660 2023-11-06T02:08:42.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.698+0000 7f76467fc640 1 -- 172.21.15.49:0/2560671880 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f76400534e0 con 0x7f7650071660 2023-11-06T02:08:42.696 INFO:teuthology.orchestra.run.smithi049.stdout:55834574855 2023-11-06T02:08:42.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.702+0000 7f7633fff640 1 -- 172.21.15.49:0/2560671880 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7634065a90 msgr2=0x7f7634067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:42.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.702+0000 7f7633fff640 1 --2- 172.21.15.49:0/2560671880 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7634065a90 0x7f7634067f50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f764c0ad000 tx=0x7f764c0a6000 comp rx=0 tx=0).stop 2023-11-06T02:08:42.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.702+0000 7f7633fff640 1 -- 172.21.15.49:0/2560671880 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7650071660 msgr2=0x7f7650156450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:42.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.702+0000 7f7633fff640 1 --2- 172.21.15.49:0/2560671880 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7650071660 0x7f7650156450 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7640001390 tx=0x7f764000cbe0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.702+0000 7f7633fff640 1 -- 172.21.15.49:0/2560671880 shutdown_connections 2023-11-06T02:08:42.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.702+0000 7f7633fff640 1 --2- 172.21.15.49:0/2560671880 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7634065a90 0x7f7634067f50 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.701 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.702+0000 7f7633fff640 1 --2- 172.21.15.49:0/2560671880 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f765015ceb0 0x7f7650152b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.701 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.702+0000 7f7633fff640 1 --2- 172.21.15.49:0/2560671880 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76500cb970 0x7f76500ae230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.701 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.702+0000 7f7633fff640 1 --2- 172.21.15.49:0/2560671880 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7650071660 0x7f7650156450 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.701 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.702+0000 7f7633fff640 1 -- 172.21.15.49:0/2560671880 >> 172.21.15.49:0/2560671880 conn(0x7f76500cc140 msgr2=0x7f76500c8710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:42.702 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.702+0000 7f7633fff640 1 -- 172.21.15.49:0/2560671880 shutdown_connections 2023-11-06T02:08:42.702 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.702+0000 7f7633fff640 1 -- 172.21.15.49:0/2560671880 wait complete. 2023-11-06T02:08:42.714 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.2 2023-11-06T02:08:42.714 DEBUG:teuthology.parallel:result is None 2023-11-06T02:08:42.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.722+0000 7f4a2c63f640 1 -- 172.21.15.49:0/172701062 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f4a1c005350 con 0x7f4a1c0568e0 2023-11-06T02:08:42.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.722+0000 7f4a1affd640 1 -- 172.21.15.49:0/172701062 <== mon.1 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4a14058dd0 con 0x7f4a1c0568e0 2023-11-06T02:08:42.720 INFO:teuthology.orchestra.run.smithi049.stdout:55834574855 2023-11-06T02:08:42.722 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.726+0000 7f4a2c63f640 1 -- 172.21.15.49:0/172701062 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4a08065a90 msgr2=0x7f4a08067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:42.722 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.726+0000 7f4a2c63f640 1 --2- 172.21.15.49:0/172701062 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4a08065a90 0x7f4a08067f50 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4a20006f50 tx=0x7f4a20009240 comp rx=0 tx=0).stop 2023-11-06T02:08:42.723 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.726+0000 7f4a2c63f640 1 -- 172.21.15.49:0/172701062 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a1c0568e0 msgr2=0x7f4a1c0e3440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:42.723 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.726+0000 7f4a2c63f640 1 --2- 172.21.15.49:0/172701062 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a1c0568e0 0x7f4a1c0e3440 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4a14010bb0 tx=0x7f4a1400e430 comp rx=0 tx=0).stop 2023-11-06T02:08:42.723 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.726+0000 7f4a2c63f640 1 -- 172.21.15.49:0/172701062 shutdown_connections 2023-11-06T02:08:42.723 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.726+0000 7f4a2c63f640 1 --2- 172.21.15.49:0/172701062 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4a08065a90 0x7f4a08067f50 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.726+0000 7f4a2c63f640 1 --2- 172.21.15.49:0/172701062 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4a1c0e1090 0x7f4a1c04bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.726+0000 7f4a2c63f640 1 --2- 172.21.15.49:0/172701062 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a1c0568e0 0x7f4a1c0e3440 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.726+0000 7f4a2c63f640 1 --2- 172.21.15.49:0/172701062 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a1c0550d0 0x7f4a1c0d0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.726+0000 7f4a2c63f640 1 -- 172.21.15.49:0/172701062 >> 172.21.15.49:0/172701062 conn(0x7f4a1c056d20 msgr2=0x7f4a1c0dcff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:42.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.726+0000 7f4a2c63f640 1 -- 172.21.15.49:0/172701062 shutdown_connections 2023-11-06T02:08:42.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.726+0000 7f4a2c63f640 1 -- 172.21.15.49:0/172701062 wait complete. 2023-11-06T02:08:42.727 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.5 2023-11-06T02:08:42.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.730+0000 7f9f0dbd6640 1 -- 172.21.15.49:0/965919672 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f9f08203c00 con 0x7f9f080c7de0 2023-11-06T02:08:42.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.730+0000 7f9eeffff640 1 -- 172.21.15.49:0/965919672 <== mon.2 v2:172.21.15.150:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9ef8055160 con 0x7f9f080c7de0 2023-11-06T02:08:42.728 DEBUG:teuthology.parallel:result is None 2023-11-06T02:08:42.728 INFO:teuthology.orchestra.run.smithi049.stdout:55834574855 2023-11-06T02:08:42.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.730+0000 7faa77fff640 1 -- 172.21.15.49:0/435333224 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7faa780cec10 con 0x7faa780ce690 2023-11-06T02:08:42.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.730+0000 7f9f0dbd6640 1 -- 172.21.15.49:0/965919672 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9ef4065b30 msgr2=0x7f9ef4067ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:42.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.730+0000 7f9f0dbd6640 1 --2- 172.21.15.49:0/965919672 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9ef4065b30 0x7f9ef4067ff0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9f000ae000 tx=0x7f9f000a7070 comp rx=0 tx=0).stop 2023-11-06T02:08:42.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.730+0000 7f9f0dbd6640 1 -- 172.21.15.49:0/965919672 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9f080c7de0 msgr2=0x7f9f08165d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:42.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.730+0000 7f9f0dbd6640 1 --2- 172.21.15.49:0/965919672 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9f080c7de0 0x7f9f08165d40 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9ef800d800 tx=0x7f9ef800dcc0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.730+0000 7faa5ffff640 1 -- 172.21.15.49:0/435333224 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7faa700ef7b0 con 0x7faa780ce690 2023-11-06T02:08:42.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.730+0000 7f9f0dbd6640 1 -- 172.21.15.49:0/965919672 shutdown_connections 2023-11-06T02:08:42.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.730+0000 7f9f0dbd6640 1 --2- 172.21.15.49:0/965919672 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9ef4065b30 0x7f9ef4067ff0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.730+0000 7f9f0dbd6640 1 --2- 172.21.15.49:0/965919672 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9f081647c0 0x7f9f080c78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.730+0000 7f9f0dbd6640 1 --2- 172.21.15.49:0/965919672 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9f080c7de0 0x7f9f08165d40 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.730+0000 7f9f0dbd6640 1 --2- 172.21.15.49:0/965919672 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f080b0790 0x7f9f08155ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.730+0000 7f9f0dbd6640 1 -- 172.21.15.49:0/965919672 >> 172.21.15.49:0/965919672 conn(0x7f9f080cb080 msgr2=0x7f9f080cd900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:42.732 INFO:teuthology.orchestra.run.smithi049.stdout:55834574854 2023-11-06T02:08:42.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.730+0000 7f9f0dbd6640 1 -- 172.21.15.49:0/965919672 shutdown_connections 2023-11-06T02:08:42.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.730+0000 7f9f0dbd6640 1 -- 172.21.15.49:0/965919672 wait complete. 2023-11-06T02:08:42.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.734+0000 7faa77fff640 1 -- 172.21.15.49:0/435333224 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7faa64065b30 msgr2=0x7faa64067ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:42.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.734+0000 7faa77fff640 1 --2- 172.21.15.49:0/435333224 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7faa64065b30 0x7faa64067ff0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7faa68009ae0 tx=0x7faa68009190 comp rx=0 tx=0).stop 2023-11-06T02:08:42.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.734+0000 7faa77fff640 1 -- 172.21.15.49:0/435333224 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa780ce690 msgr2=0x7faa78159540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:42.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.734+0000 7faa77fff640 1 --2- 172.21.15.49:0/435333224 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa780ce690 0x7faa78159540 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7faa700b1040 tx=0x7faa7001b970 comp rx=0 tx=0).stop 2023-11-06T02:08:42.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.734+0000 7faa77fff640 1 -- 172.21.15.49:0/435333224 shutdown_connections 2023-11-06T02:08:42.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.734+0000 7faa77fff640 1 --2- 172.21.15.49:0/435333224 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7faa64065b30 0x7faa64067ff0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.736 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.734+0000 7faa77fff640 1 --2- 172.21.15.49:0/435333224 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa780ce690 0x7faa78159540 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.736 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.734+0000 7faa77fff640 1 --2- 172.21.15.49:0/435333224 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa780cca80 0x7faa78159000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.736 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.734+0000 7faa77fff640 1 --2- 172.21.15.49:0/435333224 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa780b18c0 0x7faa781580f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.736 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.734+0000 7faa77fff640 1 -- 172.21.15.49:0/435333224 >> 172.21.15.49:0/435333224 conn(0x7faa78069f40 msgr2=0x7faa78071570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:42.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.734+0000 7faa77fff640 1 -- 172.21.15.49:0/435333224 shutdown_connections 2023-11-06T02:08:42.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.734+0000 7faa77fff640 1 -- 172.21.15.49:0/435333224 wait complete. 2023-11-06T02:08:42.739 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.742+0000 7fdf48c01640 1 -- 172.21.15.49:0/3470940351 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fdf3c005350 con 0x7fdf3c050280 2023-11-06T02:08:42.740 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.742+0000 7f7207b4c640 1 -- 172.21.15.49:0/62447651 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f7200067140 con 0x7f72000cdc10 2023-11-06T02:08:42.740 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.742+0000 7f71f67fc640 1 -- 172.21.15.49:0/62447651 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f71f0054d80 con 0x7f72000cdc10 2023-11-06T02:08:42.740 INFO:teuthology.orchestra.run.smithi049.stdout:55834574855 2023-11-06T02:08:42.741 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.742+0000 7fdf2ffff640 1 -- 172.21.15.49:0/3470940351 <== mon.1 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fdf4408d020 con 0x7fdf3c050280 2023-11-06T02:08:42.741 INFO:teuthology.orchestra.run.smithi049.stdout:55834574855 2023-11-06T02:08:42.744 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.746+0000 7f7207b4c640 1 -- 172.21.15.49:0/62447651 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f71e4065a60 msgr2=0x7f71e4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:42.744 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.746+0000 7f7207b4c640 1 --2- 172.21.15.49:0/62447651 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f71e4065a60 0x7f71e4067f20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f71fc0ae000 tx=0x7f71fc008040 comp rx=0 tx=0).stop 2023-11-06T02:08:42.745 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.746+0000 7f7207b4c640 1 -- 172.21.15.49:0/62447651 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f72000cdc10 msgr2=0x7f72001579f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:42.745 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.746+0000 7f7207b4c640 1 --2- 172.21.15.49:0/62447651 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f72000cdc10 0x7f72001579f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f71f000d7d0 tx=0x7f71f000dc90 comp rx=0 tx=0).stop 2023-11-06T02:08:42.745 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.746+0000 7f7207b4c640 1 -- 172.21.15.49:0/62447651 shutdown_connections 2023-11-06T02:08:42.745 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.746+0000 7f7207b4c640 1 --2- 172.21.15.49:0/62447651 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f71e4065a60 0x7f71e4067f20 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.745 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.746+0000 7f7207b4c640 1 --2- 172.21.15.49:0/62447651 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f72000cdc10 0x7f72001579f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.746 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.746+0000 7f7207b4c640 1 --2- 172.21.15.49:0/62447651 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7200164850 0x7f72000b1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.746 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.746+0000 7f7207b4c640 1 --2- 172.21.15.49:0/62447651 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72000cb810 0x7f72000c5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.746 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.746+0000 7f7207b4c640 1 -- 172.21.15.49:0/62447651 >> 172.21.15.49:0/62447651 conn(0x7f72000cc140 msgr2=0x7f7200071120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:42.746 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.746+0000 7f7207b4c640 1 -- 172.21.15.49:0/62447651 shutdown_connections 2023-11-06T02:08:42.746 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.746+0000 7f7207b4c640 1 -- 172.21.15.49:0/62447651 wait complete. 2023-11-06T02:08:42.748 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.7 2023-11-06T02:08:42.748 DEBUG:teuthology.parallel:result is None 2023-11-06T02:08:42.749 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.750+0000 7fdf48c01640 1 -- 172.21.15.49:0/3470940351 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fdf28065a90 msgr2=0x7fdf28067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:42.749 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.750+0000 7fdf48c01640 1 --2- 172.21.15.49:0/3470940351 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fdf28065a90 0x7fdf28067f50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fdf300ae000 tx=0x7fdf300a6040 comp rx=0 tx=0).stop 2023-11-06T02:08:42.749 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.750+0000 7fdf48c01640 1 -- 172.21.15.49:0/3470940351 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf3c050280 msgr2=0x7fdf3c0cd340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:42.749 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.750+0000 7fdf48c01640 1 --2- 172.21.15.49:0/3470940351 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf3c050280 0x7fdf3c0cd340 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fdf4404f9f0 tx=0x7fdf440733b0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.749 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.750+0000 7fdf48c01640 1 -- 172.21.15.49:0/3470940351 shutdown_connections 2023-11-06T02:08:42.750 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.750+0000 7fdf48c01640 1 --2- 172.21.15.49:0/3470940351 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fdf28065a90 0x7fdf28067f50 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.750 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.750+0000 7fdf48c01640 1 --2- 172.21.15.49:0/3470940351 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdf3c0df460 0x7fdf3c0d5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.750 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.750+0000 7fdf48c01640 1 --2- 172.21.15.49:0/3470940351 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf3c0d1090 0x7fdf3c0dca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.750 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.750+0000 7fdf48c01640 1 --2- 172.21.15.49:0/3470940351 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf3c050280 0x7fdf3c0cd340 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.750 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.750+0000 7fdf48c01640 1 -- 172.21.15.49:0/3470940351 >> 172.21.15.49:0/3470940351 conn(0x7fdf3c054b10 msgr2=0x7fdf3c0da090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:42.751 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.750+0000 7fdf48c01640 1 -- 172.21.15.49:0/3470940351 shutdown_connections 2023-11-06T02:08:42.751 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.750+0000 7fdf48c01640 1 -- 172.21.15.49:0/3470940351 wait complete. 2023-11-06T02:08:42.751 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.750+0000 7f2251a8c640 1 -- 172.21.15.49:0/3555009280 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f22440e48b0 con 0x7f2244051d10 2023-11-06T02:08:42.752 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.750+0000 7f22497fa640 1 -- 172.21.15.49:0/3555009280 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2240050ce0 con 0x7f2244051d10 2023-11-06T02:08:42.752 INFO:teuthology.orchestra.run.smithi049.stdout:55834574854 2023-11-06T02:08:42.753 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.754+0000 7f2251a8c640 1 -- 172.21.15.49:0/3555009280 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2230065a60 msgr2=0x7f2230067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:42.753 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.754+0000 7f2251a8c640 1 --2- 172.21.15.49:0/3555009280 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2230065a60 0x7f2230067f20 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f22380af000 tx=0x7f22380a7000 comp rx=0 tx=0).stop 2023-11-06T02:08:42.753 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.754+0000 7f2251a8c640 1 -- 172.21.15.49:0/3555009280 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2244051d10 msgr2=0x7f224404d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:42.753 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.754+0000 7f2251a8c640 1 --2- 172.21.15.49:0/3555009280 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2244051d10 0x7f224404d5b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f224000b3f0 tx=0x7f224000b8b0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.754+0000 7f2251a8c640 1 -- 172.21.15.49:0/3555009280 shutdown_connections 2023-11-06T02:08:42.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.754+0000 7f2251a8c640 1 --2- 172.21.15.49:0/3555009280 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2230065a60 0x7f2230067f20 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.754+0000 7f2251a8c640 1 --2- 172.21.15.49:0/3555009280 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22440de240 0x7f22440e5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.754+0000 7f2251a8c640 1 --2- 172.21.15.49:0/3555009280 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f22440e3250 0x7f22440ddd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.754+0000 7f2251a8c640 1 --2- 172.21.15.49:0/3555009280 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2244051d10 0x7f224404d5b0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.754+0000 7f2251a8c640 1 -- 172.21.15.49:0/3555009280 >> 172.21.15.49:0/3555009280 conn(0x7f22440538b0 msgr2=0x7f2244038760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:42.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.754+0000 7f2251a8c640 1 -- 172.21.15.49:0/3555009280 shutdown_connections 2023-11-06T02:08:42.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.754+0000 7f2251a8c640 1 -- 172.21.15.49:0/3555009280 wait complete. 2023-11-06T02:08:42.756 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.4 2023-11-06T02:08:42.756 DEBUG:teuthology.parallel:result is None 2023-11-06T02:08:42.757 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.3 2023-11-06T02:08:42.758 DEBUG:teuthology.parallel:result is None 2023-11-06T02:08:42.767 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.0 2023-11-06T02:08:42.767 DEBUG:teuthology.parallel:result is None 2023-11-06T02:08:42.771 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.1 2023-11-06T02:08:42.772 DEBUG:teuthology.parallel:result is None 2023-11-06T02:08:42.775 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.6 2023-11-06T02:08:42.775 DEBUG:teuthology.parallel:result is None 2023-11-06T02:08:42.776 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-11-06T02:08:42.776 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T02:08:42.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.926+0000 7f6c75fa4640 1 Processor -- start 2023-11-06T02:08:42.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.926+0000 7f6c75fa4640 1 -- start start 2023-11-06T02:08:42.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.926+0000 7f6c75fa4640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f6c700b0790 0x7f6c700c8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:42.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.926+0000 7f6c75fa4640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f6c700adae0 0x7f6c70070b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:42.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.926+0000 7f6c75fa4640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c701594d0 con 0x7f6c701590f0 2023-11-06T02:08:42.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.926+0000 7f6c75fa4640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c7015a140 con 0x7f6c700b0790 2023-11-06T02:08:42.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.926+0000 7f6c75fa4640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c700c92e0 con 0x7f6c700adae0 2023-11-06T02:08:42.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.926+0000 7f6c6f7fe640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f6c701590f0 0x7f6c700b0080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:54030/0 (socket says 172.21.15.49:54030) 2023-11-06T02:08:42.924 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.926+0000 7f6c6f7fe640 1 -- 172.21.15.49:0/3229780762 learned_addr learned my addr 172.21.15.49:0/3229780762 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:08:42.924 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.926+0000 7f6c67fff640 1 --2- 172.21.15.49:0/3229780762 >> v2:172.21.15.49:3300/0 conn(0x7f6c700b0790 0x7f6c700c8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:42.924 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.926+0000 7f6c6ffff640 1 --2- 172.21.15.49:0/3229780762 >> v2:172.21.15.150:3300/0 conn(0x7f6c700adae0 0x7f6c70070b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:42.924 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.926+0000 7f6c67fff640 1 -- 172.21.15.49:0/3229780762 >> v2:172.21.15.150:3300/0 conn(0x7f6c700adae0 msgr2=0x7f6c70070b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:42.924 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.926+0000 7f6c67fff640 1 --2- 172.21.15.49:0/3229780762 >> v2:172.21.15.150:3300/0 conn(0x7f6c700adae0 0x7f6c70070b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.924 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.926+0000 7f6c67fff640 1 -- 172.21.15.49:0/3229780762 >> v1:172.21.15.49:6789/0 conn(0x7f6c701590f0 legacy=0x7f6c700b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:42.925 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.926+0000 7f6c67fff640 1 -- 172.21.15.49:0/3229780762 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c701597a0 con 0x7f6c700b0790 2023-11-06T02:08:42.925 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.926+0000 7f6c6ffff640 1 --2- 172.21.15.49:0/3229780762 >> v2:172.21.15.150:3300/0 conn(0x7f6c700adae0 0x7f6c70070b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:08:42.926 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.930+0000 7f6c67fff640 1 --2- 172.21.15.49:0/3229780762 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c700b0790 0x7f6c700c8c50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6c7015cac0 tx=0x7f6c6001c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49d93da18356c788 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:42.926 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.930+0000 7f6c6effd640 1 -- 172.21.15.49:0/3229780762 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c600ab600 con 0x7f6c700b0790 2023-11-06T02:08:42.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.930+0000 7f6c6effd640 1 -- 172.21.15.49:0/3229780762 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c600a7050 con 0x7f6c700b0790 2023-11-06T02:08:42.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.930+0000 7f6c6effd640 1 -- 172.21.15.49:0/3229780762 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c600ab980 con 0x7f6c700b0790 2023-11-06T02:08:42.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.930+0000 7f6c75fa4640 1 -- 172.21.15.49:0/3229780762 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c700b0790 msgr2=0x7f6c700c8c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:42.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.930+0000 7f6c75fa4640 1 --2- 172.21.15.49:0/3229780762 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c700b0790 0x7f6c700c8c50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6c7015cac0 tx=0x7f6c6001c3d0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.930+0000 7f6c75fa4640 1 -- 172.21.15.49:0/3229780762 shutdown_connections 2023-11-06T02:08:42.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.930+0000 7f6c75fa4640 1 --2- 172.21.15.49:0/3229780762 >> v2:172.21.15.150:3300/0 conn(0x7f6c700adae0 0x7f6c70070b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.930+0000 7f6c75fa4640 1 --2- 172.21.15.49:0/3229780762 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c700b0790 0x7f6c700c8c50 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.930+0000 7f6c75fa4640 1 -- 172.21.15.49:0/3229780762 >> 172.21.15.49:0/3229780762 conn(0x7f6c7006c0a0 msgr2=0x7f6c7006c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:42.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.930+0000 7f6c75fa4640 1 -- 172.21.15.49:0/3229780762 shutdown_connections 2023-11-06T02:08:42.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.930+0000 7f6c75fa4640 1 -- 172.21.15.49:0/3229780762 wait complete. 2023-11-06T02:08:42.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.930+0000 7f6c75fa4640 1 Processor -- start 2023-11-06T02:08:42.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.930+0000 7f6c75fa4640 1 -- start start 2023-11-06T02:08:42.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.930+0000 7f6c75fa4640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6c700adae0 0x7f6c700af170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:42.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.930+0000 7f6c75fa4640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6c701590f0 0x7f6c70152cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:42.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.930+0000 7f6c75fa4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c701566f0 0x7f6c70156b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:42.935 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.930+0000 7f6c75fa4640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c700aed50 con 0x7f6c701566f0 2023-11-06T02:08:42.935 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.930+0000 7f6c75fa4640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c700af6b0 con 0x7f6c701590f0 2023-11-06T02:08:42.935 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.930+0000 7f6c75fa4640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c701570c0 con 0x7f6c700adae0 2023-11-06T02:08:42.935 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.930+0000 7f6c6ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c701566f0 0x7f6c70156b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:42.935 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.930+0000 7f6c6ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c701566f0 0x7f6c70156b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52442/0 (socket says 172.21.15.49:52442) 2023-11-06T02:08:42.936 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.930+0000 7f6c6ffff640 1 -- 172.21.15.49:0/1624996573 learned_addr learned my addr 172.21.15.49:0/1624996573 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:42.936 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.930+0000 7f6c6f7fe640 1 --2- 172.21.15.49:0/1624996573 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6c700adae0 0x7f6c700af170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:42.936 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.934+0000 7f6c67fff640 1 --2- 172.21.15.49:0/1624996573 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6c701590f0 0x7f6c70152cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:42.936 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.934+0000 7f6c6ffff640 1 -- 172.21.15.49:0/1624996573 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6c700adae0 msgr2=0x7f6c700af170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:42.936 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.934+0000 7f6c6ffff640 1 --2- 172.21.15.49:0/1624996573 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6c700adae0 0x7f6c700af170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.937 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.934+0000 7f6c6ffff640 1 -- 172.21.15.49:0/1624996573 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6c701590f0 msgr2=0x7f6c70152cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:42.937 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.934+0000 7f6c6ffff640 1 --2- 172.21.15.49:0/1624996573 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6c701590f0 0x7f6c70152cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:42.937 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.934+0000 7f6c6ffff640 1 -- 172.21.15.49:0/1624996573 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c60009420 con 0x7f6c701566f0 2023-11-06T02:08:42.937 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.934+0000 7f6c6ffff640 1 --2- 172.21.15.49:0/1624996573 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c701566f0 0x7f6c70156b80 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6c6800cef0 tx=0x7f6c6800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:42.937 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.934+0000 7f6c6cff9640 1 -- 172.21.15.49:0/1624996573 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c6800d840 con 0x7f6c701566f0 2023-11-06T02:08:42.938 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.934+0000 7f6c6cff9640 1 -- 172.21.15.49:0/1624996573 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c6800d9c0 con 0x7f6c701566f0 2023-11-06T02:08:42.938 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.934+0000 7f6c75fa4640 1 -- 172.21.15.49:0/1624996573 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c70153230 con 0x7f6c701566f0 2023-11-06T02:08:42.938 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.934+0000 7f6c6cff9640 1 -- 172.21.15.49:0/1624996573 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c6800dd70 con 0x7f6c701566f0 2023-11-06T02:08:42.938 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.934+0000 7f6c75fa4640 1 -- 172.21.15.49:0/1624996573 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c700ae690 con 0x7f6c701566f0 2023-11-06T02:08:42.938 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.934+0000 7f6c75fa4640 1 -- 172.21.15.49:0/1624996573 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c700aee90 con 0x7f6c701566f0 2023-11-06T02:08:42.939 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.934+0000 7f6c6cff9640 1 -- 172.21.15.49:0/1624996573 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f6c6801e3b0 con 0x7f6c701566f0 2023-11-06T02:08:42.939 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.934+0000 7f6c6cff9640 1 --2- 172.21.15.49:0/1624996573 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6c50065a60 0x7f6c50067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:42.939 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.934+0000 7f6c6cff9640 1 -- 172.21.15.49:0/1624996573 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6c68083d90 con 0x7f6c701566f0 2023-11-06T02:08:42.939 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.934+0000 7f6c6f7fe640 1 --2- 172.21.15.49:0/1624996573 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6c50065a60 0x7f6c50067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:42.939 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.934+0000 7f6c6f7fe640 1 --2- 172.21.15.49:0/1624996573 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6c50065a60 0x7f6c50067f20 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6c580010f0 tx=0x7f6c58009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:42.940 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:42.938+0000 7f6c6cff9640 1 -- 172.21.15.49:0/1624996573 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f6c700aee90 con 0x7f6c701566f0 2023-11-06T02:08:43.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.086+0000 7f6c75fa4640 1 -- 172.21.15.49:0/1624996573 --> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6c700ae1a0 con 0x7f6c50065a60 2023-11-06T02:08:43.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.106+0000 7f6c6cff9640 1 -- 172.21.15.49:0/1624996573 <== mgr.4100 v2:172.21.15.150:6832/38104 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286584 (secure 0 0 0) 0x7f6c700ae1a0 con 0x7f6c50065a60 2023-11-06T02:08:43.105 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:08:43.107 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2023-11-06T02:08:43.109 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.110+0000 7f6c75fa4640 1 -- 172.21.15.49:0/1624996573 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6c50065a60 msgr2=0x7f6c50067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:43.109 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.110+0000 7f6c75fa4640 1 --2- 172.21.15.49:0/1624996573 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6c50065a60 0x7f6c50067f20 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6c580010f0 tx=0x7f6c58009040 comp rx=0 tx=0).stop 2023-11-06T02:08:43.109 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.110+0000 7f6c75fa4640 1 -- 172.21.15.49:0/1624996573 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c701566f0 msgr2=0x7f6c70156b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:43.109 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.110+0000 7f6c75fa4640 1 --2- 172.21.15.49:0/1624996573 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c701566f0 0x7f6c70156b80 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6c6800cef0 tx=0x7f6c6800a710 comp rx=0 tx=0).stop 2023-11-06T02:08:43.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.110+0000 7f6c75fa4640 1 -- 172.21.15.49:0/1624996573 shutdown_connections 2023-11-06T02:08:43.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.110+0000 7f6c75fa4640 1 --2- 172.21.15.49:0/1624996573 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6c50065a60 0x7f6c50067f20 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:43.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.110+0000 7f6c75fa4640 1 --2- 172.21.15.49:0/1624996573 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c701566f0 0x7f6c70156b80 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:43.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.110+0000 7f6c75fa4640 1 --2- 172.21.15.49:0/1624996573 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6c701590f0 0x7f6c70152cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:43.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.110+0000 7f6c75fa4640 1 --2- 172.21.15.49:0/1624996573 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6c700adae0 0x7f6c700af170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:43.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.110+0000 7f6c75fa4640 1 -- 172.21.15.49:0/1624996573 >> 172.21.15.49:0/1624996573 conn(0x7f6c7006c0a0 msgr2=0x7f6c70165f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:43.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.110+0000 7f6c75fa4640 1 -- 172.21.15.49:0/1624996573 shutdown_connections 2023-11-06T02:08:43.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.110+0000 7f6c75fa4640 1 -- 172.21.15.49:0/1624996573 wait complete. 2023-11-06T02:08:43.122 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2023-11-06T02:08:41.178078+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235108,"kb_used_data":3108,"kb_used_omap":0,"kb_used_meta":231936,"kb_avail":754739612,"statfs":{"total":773094113280,"available":772853362688,"internally_reserved":0,"allocated":3182592,"data_stored":1901233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":237502464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003635"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.655111+0000","last_change":"2023-11-06T02:08:30.608781+0000","last_active":"2023-11-06T02:08:32.655111+0000","last_peered":"2023-11-06T02:08:32.655111+0000","last_clean":"2023-11-06T02:08:32.655111+0000","last_became_active":"2023-11-06T02:08:30.606710+0000","last_became_peered":"2023-11-06T02:08:30.606710+0000","last_unstale":"2023-11-06T02:08:32.655111+0000","last_undegraded":"2023-11-06T02:08:32.655111+0000","last_fullsized":"2023-11-06T02:08:32.655111+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:43:57.021755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.413091+0000","last_change":"2023-11-06T02:08:30.646508+0000","last_active":"2023-11-06T02:08:32.413091+0000","last_peered":"2023-11-06T02:08:32.413091+0000","last_clean":"2023-11-06T02:08:32.413091+0000","last_became_active":"2023-11-06T02:08:30.645615+0000","last_became_peered":"2023-11-06T02:08:30.645615+0000","last_unstale":"2023-11-06T02:08:32.413091+0000","last_undegraded":"2023-11-06T02:08:32.413091+0000","last_fullsized":"2023-11-06T02:08:32.413091+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:49:05.460965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.610298+0000","last_change":"2023-11-06T02:08:30.658350+0000","last_active":"2023-11-06T02:08:31.610298+0000","last_peered":"2023-11-06T02:08:31.610298+0000","last_clean":"2023-11-06T02:08:31.610298+0000","last_became_active":"2023-11-06T02:08:30.651992+0000","last_became_peered":"2023-11-06T02:08:30.651992+0000","last_unstale":"2023-11-06T02:08:31.610298+0000","last_undegraded":"2023-11-06T02:08:31.610298+0000","last_fullsized":"2023-11-06T02:08:31.610298+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:09:56.455105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.610511+0000","last_change":"2023-11-06T02:08:30.627209+0000","last_active":"2023-11-06T02:08:31.610511+0000","last_peered":"2023-11-06T02:08:31.610511+0000","last_clean":"2023-11-06T02:08:31.610511+0000","last_became_active":"2023-11-06T02:08:30.626345+0000","last_became_peered":"2023-11-06T02:08:30.626345+0000","last_unstale":"2023-11-06T02:08:31.610511+0000","last_undegraded":"2023-11-06T02:08:31.610511+0000","last_fullsized":"2023-11-06T02:08:31.610511+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:46:38.867945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.652203+0000","last_change":"2023-11-06T02:08:30.619667+0000","last_active":"2023-11-06T02:08:32.652203+0000","last_peered":"2023-11-06T02:08:32.652203+0000","last_clean":"2023-11-06T02:08:32.652203+0000","last_became_active":"2023-11-06T02:08:30.619328+0000","last_became_peered":"2023-11-06T02:08:30.619328+0000","last_unstale":"2023-11-06T02:08:32.652203+0000","last_undegraded":"2023-11-06T02:08:32.652203+0000","last_fullsized":"2023-11-06T02:08:32.652203+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:26:58.090934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.608847+0000","last_change":"2023-11-06T02:08:30.657202+0000","last_active":"2023-11-06T02:08:31.608847+0000","last_peered":"2023-11-06T02:08:31.608847+0000","last_clean":"2023-11-06T02:08:31.608847+0000","last_became_active":"2023-11-06T02:08:30.652132+0000","last_became_peered":"2023-11-06T02:08:30.652132+0000","last_unstale":"2023-11-06T02:08:31.608847+0000","last_undegraded":"2023-11-06T02:08:31.608847+0000","last_fullsized":"2023-11-06T02:08:31.608847+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:35:19.038827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.149393+0000","last_change":"2023-11-06T02:08:30.645694+0000","last_active":"2023-11-06T02:08:32.149393+0000","last_peered":"2023-11-06T02:08:32.149393+0000","last_clean":"2023-11-06T02:08:32.149393+0000","last_became_active":"2023-11-06T02:08:30.645028+0000","last_became_peered":"2023-11-06T02:08:30.645028+0000","last_unstale":"2023-11-06T02:08:32.149393+0000","last_undegraded":"2023-11-06T02:08:32.149393+0000","last_fullsized":"2023-11-06T02:08:32.149393+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:26:58.090934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.412662+0000","last_change":"2023-11-06T02:08:30.652585+0000","last_active":"2023-11-06T02:08:32.412662+0000","last_peered":"2023-11-06T02:08:32.412662+0000","last_clean":"2023-11-06T02:08:32.412662+0000","last_became_active":"2023-11-06T02:08:30.652103+0000","last_became_peered":"2023-11-06T02:08:30.652103+0000","last_unstale":"2023-11-06T02:08:32.412662+0000","last_undegraded":"2023-11-06T02:08:32.412662+0000","last_fullsized":"2023-11-06T02:08:32.412662+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:57:32.079075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.651445+0000","last_change":"2023-11-06T02:08:30.618708+0000","last_active":"2023-11-06T02:08:32.651445+0000","last_peered":"2023-11-06T02:08:32.651445+0000","last_clean":"2023-11-06T02:08:32.651445+0000","last_became_active":"2023-11-06T02:08:30.618232+0000","last_became_peered":"2023-11-06T02:08:30.618232+0000","last_unstale":"2023-11-06T02:08:32.651445+0000","last_undegraded":"2023-11-06T02:08:32.651445+0000","last_fullsized":"2023-11-06T02:08:32.651445+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:31:28.301664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.413222+0000","last_change":"2023-11-06T02:08:30.640907+0000","last_active":"2023-11-06T02:08:32.413222+0000","last_peered":"2023-11-06T02:08:32.413222+0000","last_clean":"2023-11-06T02:08:32.413222+0000","last_became_active":"2023-11-06T02:08:30.626276+0000","last_became_peered":"2023-11-06T02:08:30.626276+0000","last_unstale":"2023-11-06T02:08:32.413222+0000","last_undegraded":"2023-11-06T02:08:32.413222+0000","last_fullsized":"2023-11-06T02:08:32.413222+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:56:51.565719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.610554+0000","last_change":"2023-11-06T02:08:30.658929+0000","last_active":"2023-11-06T02:08:31.610554+0000","last_peered":"2023-11-06T02:08:31.610554+0000","last_clean":"2023-11-06T02:08:31.610554+0000","last_became_active":"2023-11-06T02:08:30.657695+0000","last_became_peered":"2023-11-06T02:08:30.657695+0000","last_unstale":"2023-11-06T02:08:31.610554+0000","last_undegraded":"2023-11-06T02:08:31.610554+0000","last_fullsized":"2023-11-06T02:08:31.610554+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:57:32.079075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.654812+0000","last_change":"2023-11-06T02:08:30.660659+0000","last_active":"2023-11-06T02:08:32.654812+0000","last_peered":"2023-11-06T02:08:32.654812+0000","last_clean":"2023-11-06T02:08:32.654812+0000","last_became_active":"2023-11-06T02:08:30.660402+0000","last_became_peered":"2023-11-06T02:08:30.660402+0000","last_unstale":"2023-11-06T02:08:32.654812+0000","last_undegraded":"2023-11-06T02:08:32.654812+0000","last_fullsized":"2023-11-06T02:08:32.654812+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:26:50.302398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.654670+0000","last_change":"2023-11-06T02:08:30.608913+0000","last_active":"2023-11-06T02:08:32.654670+0000","last_peered":"2023-11-06T02:08:32.654670+0000","last_clean":"2023-11-06T02:08:32.654670+0000","last_became_active":"2023-11-06T02:08:30.606808+0000","last_became_peered":"2023-11-06T02:08:30.606808+0000","last_unstale":"2023-11-06T02:08:32.654670+0000","last_undegraded":"2023-11-06T02:08:32.654670+0000","last_fullsized":"2023-11-06T02:08:32.654670+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:24:10.805757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.412935+0000","last_change":"2023-11-06T02:08:30.653750+0000","last_active":"2023-11-06T02:08:32.412935+0000","last_peered":"2023-11-06T02:08:32.412935+0000","last_clean":"2023-11-06T02:08:32.412935+0000","last_became_active":"2023-11-06T02:08:30.653252+0000","last_became_peered":"2023-11-06T02:08:30.653252+0000","last_unstale":"2023-11-06T02:08:32.412935+0000","last_undegraded":"2023-11-06T02:08:32.412935+0000","last_fullsized":"2023-11-06T02:08:32.412935+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:26:58.090934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.149589+0000","last_change":"2023-11-06T02:08:30.644591+0000","last_active":"2023-11-06T02:08:32.149589+0000","last_peered":"2023-11-06T02:08:32.149589+0000","last_clean":"2023-11-06T02:08:32.149589+0000","last_became_active":"2023-11-06T02:08:30.643848+0000","last_became_peered":"2023-11-06T02:08:30.643848+0000","last_unstale":"2023-11-06T02:08:32.149589+0000","last_undegraded":"2023-11-06T02:08:32.149589+0000","last_fullsized":"2023-11-06T02:08:32.149589+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:49:05.460965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.608423+0000","last_change":"2023-11-06T02:08:30.634202+0000","last_active":"2023-11-06T02:08:31.608423+0000","last_peered":"2023-11-06T02:08:31.608423+0000","last_clean":"2023-11-06T02:08:31.608423+0000","last_became_active":"2023-11-06T02:08:30.633451+0000","last_became_peered":"2023-11-06T02:08:30.633451+0000","last_unstale":"2023-11-06T02:08:31.608423+0000","last_undegraded":"2023-11-06T02:08:31.608423+0000","last_fullsized":"2023-11-06T02:08:31.608423+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:13:14.793314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.651622+0000","last_change":"2023-11-06T02:08:30.618515+0000","last_active":"2023-11-06T02:08:32.651622+0000","last_peered":"2023-11-06T02:08:32.651622+0000","last_clean":"2023-11-06T02:08:32.651622+0000","last_became_active":"2023-11-06T02:08:30.617836+0000","last_became_peered":"2023-11-06T02:08:30.617836+0000","last_unstale":"2023-11-06T02:08:32.651622+0000","last_undegraded":"2023-11-06T02:08:32.651622+0000","last_fullsized":"2023-11-06T02:08:32.651622+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:17:50.007398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.609393+0000","last_change":"2023-11-06T02:08:30.659348+0000","last_active":"2023-11-06T02:08:31.609393+0000","last_peered":"2023-11-06T02:08:31.609393+0000","last_clean":"2023-11-06T02:08:31.609393+0000","last_became_active":"2023-11-06T02:08:30.652231+0000","last_became_peered":"2023-11-06T02:08:30.652231+0000","last_unstale":"2023-11-06T02:08:31.609393+0000","last_undegraded":"2023-11-06T02:08:31.609393+0000","last_fullsized":"2023-11-06T02:08:31.609393+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:13:26.942266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.655090+0000","last_change":"2023-11-06T02:08:30.627395+0000","last_active":"2023-11-06T02:08:32.655090+0000","last_peered":"2023-11-06T02:08:32.655090+0000","last_clean":"2023-11-06T02:08:32.655090+0000","last_became_active":"2023-11-06T02:08:30.627048+0000","last_became_peered":"2023-11-06T02:08:30.627048+0000","last_unstale":"2023-11-06T02:08:32.655090+0000","last_undegraded":"2023-11-06T02:08:32.655090+0000","last_fullsized":"2023-11-06T02:08:32.655090+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:15:59.045303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.609522+0000","last_change":"2023-11-06T02:08:30.658560+0000","last_active":"2023-11-06T02:08:31.609522+0000","last_peered":"2023-11-06T02:08:31.609522+0000","last_clean":"2023-11-06T02:08:31.609522+0000","last_became_active":"2023-11-06T02:08:30.657560+0000","last_became_peered":"2023-11-06T02:08:30.657560+0000","last_unstale":"2023-11-06T02:08:31.609522+0000","last_undegraded":"2023-11-06T02:08:31.609522+0000","last_fullsized":"2023-11-06T02:08:31.609522+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:26:06.040192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.149579+0000","last_change":"2023-11-06T02:08:30.650121+0000","last_active":"2023-11-06T02:08:32.149579+0000","last_peered":"2023-11-06T02:08:32.149579+0000","last_clean":"2023-11-06T02:08:32.149579+0000","last_became_active":"2023-11-06T02:08:30.649010+0000","last_became_peered":"2023-11-06T02:08:30.649010+0000","last_unstale":"2023-11-06T02:08:32.149579+0000","last_undegraded":"2023-11-06T02:08:32.149579+0000","last_fullsized":"2023-11-06T02:08:32.149579+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:56:51.565719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.651624+0000","last_change":"2023-11-06T02:08:30.622216+0000","last_active":"2023-11-06T02:08:32.651624+0000","last_peered":"2023-11-06T02:08:32.651624+0000","last_clean":"2023-11-06T02:08:32.651624+0000","last_became_active":"2023-11-06T02:08:30.621706+0000","last_became_peered":"2023-11-06T02:08:30.621706+0000","last_unstale":"2023-11-06T02:08:32.651624+0000","last_undegraded":"2023-11-06T02:08:32.651624+0000","last_fullsized":"2023-11-06T02:08:32.651624+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:52:47.174384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.604379+0000","last_change":"2023-11-06T02:08:30.623436+0000","last_active":"2023-11-06T02:08:31.604379+0000","last_peered":"2023-11-06T02:08:31.604379+0000","last_clean":"2023-11-06T02:08:31.604379+0000","last_became_active":"2023-11-06T02:08:30.623149+0000","last_became_peered":"2023-11-06T02:08:30.623149+0000","last_unstale":"2023-11-06T02:08:31.604379+0000","last_undegraded":"2023-11-06T02:08:31.604379+0000","last_fullsized":"2023-11-06T02:08:31.604379+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:47:30.859032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.149601+0000","last_change":"2023-11-06T02:08:30.656899+0000","last_active":"2023-11-06T02:08:32.149601+0000","last_peered":"2023-11-06T02:08:32.149601+0000","last_clean":"2023-11-06T02:08:32.149601+0000","last_became_active":"2023-11-06T02:08:30.656319+0000","last_became_peered":"2023-11-06T02:08:30.656319+0000","last_unstale":"2023-11-06T02:08:32.149601+0000","last_undegraded":"2023-11-06T02:08:32.149601+0000","last_fullsized":"2023-11-06T02:08:32.149601+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:57:32.079075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.652785+0000","last_change":"2023-11-06T02:08:30.627856+0000","last_active":"2023-11-06T02:08:32.652785+0000","last_peered":"2023-11-06T02:08:32.652785+0000","last_clean":"2023-11-06T02:08:32.652785+0000","last_became_active":"2023-11-06T02:08:30.627536+0000","last_became_peered":"2023-11-06T02:08:30.627536+0000","last_unstale":"2023-11-06T02:08:32.652785+0000","last_undegraded":"2023-11-06T02:08:32.652785+0000","last_fullsized":"2023-11-06T02:08:32.652785+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:53:11.188799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.652773+0000","last_change":"2023-11-06T02:08:30.613522+0000","last_active":"2023-11-06T02:08:32.652773+0000","last_peered":"2023-11-06T02:08:32.652773+0000","last_clean":"2023-11-06T02:08:32.652773+0000","last_became_active":"2023-11-06T02:08:30.612950+0000","last_became_peered":"2023-11-06T02:08:30.612950+0000","last_unstale":"2023-11-06T02:08:32.652773+0000","last_undegraded":"2023-11-06T02:08:32.652773+0000","last_fullsized":"2023-11-06T02:08:32.652773+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:26:58.090934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.413094+0000","last_change":"2023-11-06T02:08:30.653593+0000","last_active":"2023-11-06T02:08:32.413094+0000","last_peered":"2023-11-06T02:08:32.413094+0000","last_clean":"2023-11-06T02:08:32.413094+0000","last_became_active":"2023-11-06T02:08:30.652925+0000","last_became_peered":"2023-11-06T02:08:30.652925+0000","last_unstale":"2023-11-06T02:08:32.413094+0000","last_undegraded":"2023-11-06T02:08:32.413094+0000","last_fullsized":"2023-11-06T02:08:32.413094+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:17:50.007398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.652200+0000","last_change":"2023-11-06T02:08:30.645260+0000","last_active":"2023-11-06T02:08:32.652200+0000","last_peered":"2023-11-06T02:08:32.652200+0000","last_clean":"2023-11-06T02:08:32.652200+0000","last_became_active":"2023-11-06T02:08:30.644949+0000","last_became_peered":"2023-11-06T02:08:30.644949+0000","last_unstale":"2023-11-06T02:08:32.652200+0000","last_undegraded":"2023-11-06T02:08:32.652200+0000","last_fullsized":"2023-11-06T02:08:32.652200+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:41:00.973671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.604763+0000","last_change":"2023-11-06T02:08:30.649814+0000","last_active":"2023-11-06T02:08:31.604763+0000","last_peered":"2023-11-06T02:08:31.604763+0000","last_clean":"2023-11-06T02:08:31.604763+0000","last_became_active":"2023-11-06T02:08:30.649492+0000","last_became_peered":"2023-11-06T02:08:30.649492+0000","last_unstale":"2023-11-06T02:08:31.604763+0000","last_undegraded":"2023-11-06T02:08:31.604763+0000","last_fullsized":"2023-11-06T02:08:31.604763+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:26:58.090934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.609372+0000","last_change":"2023-11-06T02:08:30.632560+0000","last_active":"2023-11-06T02:08:31.609372+0000","last_peered":"2023-11-06T02:08:31.609372+0000","last_clean":"2023-11-06T02:08:31.609372+0000","last_became_active":"2023-11-06T02:08:30.631819+0000","last_became_peered":"2023-11-06T02:08:30.631819+0000","last_unstale":"2023-11-06T02:08:31.609372+0000","last_undegraded":"2023-11-06T02:08:31.609372+0000","last_fullsized":"2023-11-06T02:08:31.609372+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:43:23.215718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.604562+0000","last_change":"2023-11-06T02:08:30.624052+0000","last_active":"2023-11-06T02:08:31.604562+0000","last_peered":"2023-11-06T02:08:31.604562+0000","last_clean":"2023-11-06T02:08:31.604562+0000","last_became_active":"2023-11-06T02:08:30.623658+0000","last_became_peered":"2023-11-06T02:08:30.623658+0000","last_unstale":"2023-11-06T02:08:31.604562+0000","last_undegraded":"2023-11-06T02:08:31.604562+0000","last_fullsized":"2023-11-06T02:08:31.604562+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:17:50.007398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.609631+0000","last_change":"2023-11-06T02:08:30.614066+0000","last_active":"2023-11-06T02:08:31.609631+0000","last_peered":"2023-11-06T02:08:31.609631+0000","last_clean":"2023-11-06T02:08:31.609631+0000","last_became_active":"2023-11-06T02:08:30.613337+0000","last_became_peered":"2023-11-06T02:08:30.613337+0000","last_unstale":"2023-11-06T02:08:31.609631+0000","last_undegraded":"2023-11-06T02:08:31.609631+0000","last_fullsized":"2023-11-06T02:08:31.609631+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:33:39.359552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.604963+0000","last_change":"2023-11-06T02:08:30.621368+0000","last_active":"2023-11-06T02:08:31.604963+0000","last_peered":"2023-11-06T02:08:31.604963+0000","last_clean":"2023-11-06T02:08:31.604963+0000","last_became_active":"2023-11-06T02:08:30.620563+0000","last_became_peered":"2023-11-06T02:08:30.620563+0000","last_unstale":"2023-11-06T02:08:31.604963+0000","last_undegraded":"2023-11-06T02:08:31.604963+0000","last_fullsized":"2023-11-06T02:08:31.604963+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:56:51.565719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"22'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.149865+0000","last_change":"2023-11-06T02:08:30.229825+0000","last_active":"2023-11-06T02:08:32.149865+0000","last_peered":"2023-11-06T02:08:32.149865+0000","last_clean":"2023-11-06T02:08:32.149865+0000","last_became_active":"2023-11-06T02:08:30.229392+0000","last_became_peered":"2023-11-06T02:08:30.229392+0000","last_unstale":"2023-11-06T02:08:32.149865+0000","last_undegraded":"2023-11-06T02:08:32.149865+0000","last_fullsized":"2023-11-06T02:08:32.149865+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:33:38.355715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.413299+0000","last_change":"2023-11-06T02:08:30.646474+0000","last_active":"2023-11-06T02:08:32.413299+0000","last_peered":"2023-11-06T02:08:32.413299+0000","last_clean":"2023-11-06T02:08:32.413299+0000","last_became_active":"2023-11-06T02:08:30.645992+0000","last_became_peered":"2023-11-06T02:08:30.645992+0000","last_unstale":"2023-11-06T02:08:32.413299+0000","last_undegraded":"2023-11-06T02:08:32.413299+0000","last_fullsized":"2023-11-06T02:08:32.413299+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:47:30.859032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.651766+0000","last_change":"2023-11-06T02:08:30.225818+0000","last_active":"2023-11-06T02:08:32.651766+0000","last_peered":"2023-11-06T02:08:32.651766+0000","last_clean":"2023-11-06T02:08:32.651766+0000","last_became_active":"2023-11-06T02:08:30.225359+0000","last_became_peered":"2023-11-06T02:08:30.225359+0000","last_unstale":"2023-11-06T02:08:32.651766+0000","last_undegraded":"2023-11-06T02:08:32.651766+0000","last_fullsized":"2023-11-06T02:08:32.651766+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:52:19.592716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.653330+0000","last_change":"2023-11-06T02:08:30.618020+0000","last_active":"2023-11-06T02:08:32.653330+0000","last_peered":"2023-11-06T02:08:32.653330+0000","last_clean":"2023-11-06T02:08:32.653330+0000","last_became_active":"2023-11-06T02:08:30.617684+0000","last_became_peered":"2023-11-06T02:08:30.617684+0000","last_unstale":"2023-11-06T02:08:32.653330+0000","last_undegraded":"2023-11-06T02:08:32.653330+0000","last_fullsized":"2023-11-06T02:08:32.653330+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:56:51.565719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.149630+0000","last_change":"2023-11-06T02:08:30.230443+0000","last_active":"2023-11-06T02:08:32.149630+0000","last_peered":"2023-11-06T02:08:32.149630+0000","last_clean":"2023-11-06T02:08:32.149630+0000","last_became_active":"2023-11-06T02:08:30.229980+0000","last_became_peered":"2023-11-06T02:08:30.229980+0000","last_unstale":"2023-11-06T02:08:32.149630+0000","last_undegraded":"2023-11-06T02:08:32.149630+0000","last_fullsized":"2023-11-06T02:08:32.149630+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:46:37.864109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.605377+0000","last_change":"2023-11-06T02:08:30.647210+0000","last_active":"2023-11-06T02:08:31.605377+0000","last_peered":"2023-11-06T02:08:31.605377+0000","last_clean":"2023-11-06T02:08:31.605377+0000","last_became_active":"2023-11-06T02:08:30.646862+0000","last_became_peered":"2023-11-06T02:08:30.646862+0000","last_unstale":"2023-11-06T02:08:31.605377+0000","last_undegraded":"2023-11-06T02:08:31.605377+0000","last_fullsized":"2023-11-06T02:08:31.605377+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:43:23.215718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"22'8","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.609883+0000","last_change":"2023-11-06T02:08:30.218861+0000","last_active":"2023-11-06T02:08:31.609883+0000","last_peered":"2023-11-06T02:08:31.609883+0000","last_clean":"2023-11-06T02:08:31.609883+0000","last_became_active":"2023-11-06T02:08:30.218245+0000","last_became_peered":"2023-11-06T02:08:30.218245+0000","last_unstale":"2023-11-06T02:08:31.609883+0000","last_undegraded":"2023-11-06T02:08:31.609883+0000","last_fullsized":"2023-11-06T02:08:31.609883+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:04:42.682132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"22'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.150304+0000","last_change":"2023-11-06T02:08:30.229578+0000","last_active":"2023-11-06T02:08:32.150304+0000","last_peered":"2023-11-06T02:08:32.150304+0000","last_clean":"2023-11-06T02:08:32.150304+0000","last_became_active":"2023-11-06T02:08:30.229131+0000","last_became_peered":"2023-11-06T02:08:30.229131+0000","last_unstale":"2023-11-06T02:08:32.150304+0000","last_undegraded":"2023-11-06T02:08:32.150304+0000","last_fullsized":"2023-11-06T02:08:32.150304+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:30:49.856478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.412653+0000","last_change":"2023-11-06T02:08:30.647714+0000","last_active":"2023-11-06T02:08:32.412653+0000","last_peered":"2023-11-06T02:08:32.412653+0000","last_clean":"2023-11-06T02:08:32.412653+0000","last_became_active":"2023-11-06T02:08:30.647212+0000","last_became_peered":"2023-11-06T02:08:30.647212+0000","last_unstale":"2023-11-06T02:08:32.412653+0000","last_undegraded":"2023-11-06T02:08:32.412653+0000","last_fullsized":"2023-11-06T02:08:32.412653+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:13:37.616527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"22'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.150130+0000","last_change":"2023-11-06T02:08:30.230469+0000","last_active":"2023-11-06T02:08:32.150130+0000","last_peered":"2023-11-06T02:08:32.150130+0000","last_clean":"2023-11-06T02:08:32.150130+0000","last_became_active":"2023-11-06T02:08:30.230064+0000","last_became_peered":"2023-11-06T02:08:30.230064+0000","last_unstale":"2023-11-06T02:08:32.150130+0000","last_undegraded":"2023-11-06T02:08:32.150130+0000","last_fullsized":"2023-11-06T02:08:32.150130+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:19:59.347221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.608834+0000","last_change":"2023-11-06T02:08:30.632754+0000","last_active":"2023-11-06T02:08:31.608834+0000","last_peered":"2023-11-06T02:08:31.608834+0000","last_clean":"2023-11-06T02:08:31.608834+0000","last_became_active":"2023-11-06T02:08:30.631976+0000","last_became_peered":"2023-11-06T02:08:30.631976+0000","last_unstale":"2023-11-06T02:08:31.608834+0000","last_undegraded":"2023-11-06T02:08:31.608834+0000","last_fullsized":"2023-11-06T02:08:31.608834+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:56:51.565719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"22'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.149972+0000","last_change":"2023-11-06T02:08:30.231003+0000","last_active":"2023-11-06T02:08:32.149972+0000","last_peered":"2023-11-06T02:08:32.149972+0000","last_clean":"2023-11-06T02:08:32.149972+0000","last_became_active":"2023-11-06T02:08:30.229978+0000","last_became_peered":"2023-11-06T02:08:30.229978+0000","last_unstale":"2023-11-06T02:08:32.149972+0000","last_undegraded":"2023-11-06T02:08:32.149972+0000","last_fullsized":"2023-11-06T02:08:32.149972+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:24:09.801921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.609985+0000","last_change":"2023-11-06T02:08:30.619807+0000","last_active":"2023-11-06T02:08:31.609985+0000","last_peered":"2023-11-06T02:08:31.609985+0000","last_clean":"2023-11-06T02:08:31.609985+0000","last_became_active":"2023-11-06T02:08:30.619048+0000","last_became_peered":"2023-11-06T02:08:30.619048+0000","last_unstale":"2023-11-06T02:08:31.609985+0000","last_undegraded":"2023-11-06T02:08:31.609985+0000","last_fullsized":"2023-11-06T02:08:31.609985+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:15:59.045303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.412965+0000","last_change":"2023-11-06T02:08:30.218715+0000","last_active":"2023-11-06T02:08:32.412965+0000","last_peered":"2023-11-06T02:08:32.412965+0000","last_clean":"2023-11-06T02:08:32.412965+0000","last_became_active":"2023-11-06T02:08:30.217966+0000","last_became_peered":"2023-11-06T02:08:30.217966+0000","last_unstale":"2023-11-06T02:08:32.412965+0000","last_undegraded":"2023-11-06T02:08:32.412965+0000","last_fullsized":"2023-11-06T02:08:32.412965+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:30:21.274364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.652004+0000","last_change":"2023-11-06T02:08:30.620567+0000","last_active":"2023-11-06T02:08:32.652004+0000","last_peered":"2023-11-06T02:08:32.652004+0000","last_clean":"2023-11-06T02:08:32.652004+0000","last_became_active":"2023-11-06T02:08:30.619671+0000","last_became_peered":"2023-11-06T02:08:30.619671+0000","last_unstale":"2023-11-06T02:08:32.652004+0000","last_undegraded":"2023-11-06T02:08:32.652004+0000","last_fullsized":"2023-11-06T02:08:32.652004+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:57:32.079075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"22'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.605088+0000","last_change":"2023-11-06T02:08:30.220060+0000","last_active":"2023-11-06T02:08:31.605088+0000","last_peered":"2023-11-06T02:08:31.605088+0000","last_clean":"2023-11-06T02:08:31.605088+0000","last_became_active":"2023-11-06T02:08:30.218534+0000","last_became_peered":"2023-11-06T02:08:30.218534+0000","last_unstale":"2023-11-06T02:08:31.605088+0000","last_undegraded":"2023-11-06T02:08:31.605088+0000","last_fullsized":"2023-11-06T02:08:31.605088+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:30:21.274364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.609657+0000","last_change":"2023-11-06T02:08:30.658173+0000","last_active":"2023-11-06T02:08:31.609657+0000","last_peered":"2023-11-06T02:08:31.609657+0000","last_clean":"2023-11-06T02:08:31.609657+0000","last_became_active":"2023-11-06T02:08:30.656890+0000","last_became_peered":"2023-11-06T02:08:30.656890+0000","last_unstale":"2023-11-06T02:08:31.609657+0000","last_undegraded":"2023-11-06T02:08:31.609657+0000","last_fullsized":"2023-11-06T02:08:31.609657+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:23:59.639528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.610321+0000","last_change":"2023-11-06T02:08:30.222023+0000","last_active":"2023-11-06T02:08:31.610321+0000","last_peered":"2023-11-06T02:08:31.610321+0000","last_clean":"2023-11-06T02:08:31.610321+0000","last_became_active":"2023-11-06T02:08:30.220788+0000","last_became_peered":"2023-11-06T02:08:30.220788+0000","last_unstale":"2023-11-06T02:08:31.610321+0000","last_undegraded":"2023-11-06T02:08:31.610321+0000","last_fullsized":"2023-11-06T02:08:31.610321+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:31:31.882709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.610620+0000","last_change":"2023-11-06T02:08:30.635685+0000","last_active":"2023-11-06T02:08:31.610620+0000","last_peered":"2023-11-06T02:08:31.610620+0000","last_clean":"2023-11-06T02:08:31.610620+0000","last_became_active":"2023-11-06T02:08:30.634801+0000","last_became_peered":"2023-11-06T02:08:30.634801+0000","last_unstale":"2023-11-06T02:08:31.610620+0000","last_undegraded":"2023-11-06T02:08:31.610620+0000","last_fullsized":"2023-11-06T02:08:31.610620+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:37:47.953218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.609115+0000","last_change":"2023-11-06T02:08:30.222592+0000","last_active":"2023-11-06T02:08:31.609115+0000","last_peered":"2023-11-06T02:08:31.609115+0000","last_clean":"2023-11-06T02:08:31.609115+0000","last_became_active":"2023-11-06T02:08:30.222184+0000","last_became_peered":"2023-11-06T02:08:30.222184+0000","last_unstale":"2023-11-06T02:08:31.609115+0000","last_undegraded":"2023-11-06T02:08:31.609115+0000","last_fullsized":"2023-11-06T02:08:31.609115+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:31:31.882709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.609663+0000","last_change":"2023-11-06T02:08:30.651763+0000","last_active":"2023-11-06T02:08:31.609663+0000","last_peered":"2023-11-06T02:08:31.609663+0000","last_clean":"2023-11-06T02:08:31.609663+0000","last_became_active":"2023-11-06T02:08:30.651013+0000","last_became_peered":"2023-11-06T02:08:30.651013+0000","last_unstale":"2023-11-06T02:08:31.609663+0000","last_undegraded":"2023-11-06T02:08:31.609663+0000","last_fullsized":"2023-11-06T02:08:31.609663+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:41:51.822994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.604575+0000","last_change":"2023-11-06T02:08:30.225279+0000","last_active":"2023-11-06T02:08:31.604575+0000","last_peered":"2023-11-06T02:08:31.604575+0000","last_clean":"2023-11-06T02:08:31.604575+0000","last_became_active":"2023-11-06T02:08:30.224936+0000","last_became_peered":"2023-11-06T02:08:30.224936+0000","last_unstale":"2023-11-06T02:08:31.604575+0000","last_undegraded":"2023-11-06T02:08:31.604575+0000","last_fullsized":"2023-11-06T02:08:31.604575+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:52:19.592716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.653044+0000","last_change":"2023-11-06T02:08:30.619652+0000","last_active":"2023-11-06T02:08:32.653044+0000","last_peered":"2023-11-06T02:08:32.653044+0000","last_clean":"2023-11-06T02:08:32.653044+0000","last_became_active":"2023-11-06T02:08:30.619070+0000","last_became_peered":"2023-11-06T02:08:30.619070+0000","last_unstale":"2023-11-06T02:08:32.653044+0000","last_undegraded":"2023-11-06T02:08:32.653044+0000","last_fullsized":"2023-11-06T02:08:32.653044+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:56:51.565719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":167,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-06T02:08:30.628264+0000","last_change":"2023-11-06T02:08:20.523427+0000","last_active":"2023-11-06T02:08:30.628264+0000","last_peered":"2023-11-06T02:08:30.628264+0000","last_clean":"2023-11-06T02:08:30.628264+0000","last_became_active":"2023-11-06T02:08:20.521089+0000","last_became_peered":"2023-11-06T02:08:20.521089+0000","last_unstale":"2023-11-06T02:08:30.628264+0000","last_undegraded":"2023-11-06T02:08:30.628264+0000","last_fullsized":"2023-11-06T02:08:30.628264+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:19.252348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:19.252348+0000","last_clean_scrub_stamp":"2023-11-06T02:08:19.252348+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:30:15.566397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-06T02:08:30.640657+0000","last_change":"2023-11-06T02:08:26.059502+0000","last_active":"2023-11-06T02:08:30.640657+0000","last_peered":"2023-11-06T02:08:30.640657+0000","last_clean":"2023-11-06T02:08:30.640657+0000","last_became_active":"2023-11-06T02:08:26.058612+0000","last_became_peered":"2023-11-06T02:08:26.058612+0000","last_unstale":"2023-11-06T02:08:30.640657+0000","last_undegraded":"2023-11-06T02:08:30.640657+0000","last_fullsized":"2023-11-06T02:08:30.640657+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:10.626021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.604588+0000","last_change":"2023-11-06T02:08:30.219125+0000","last_active":"2023-11-06T02:08:31.604588+0000","last_peered":"2023-11-06T02:08:31.604588+0000","last_clean":"2023-11-06T02:08:31.604588+0000","last_became_active":"2023-11-06T02:08:30.217091+0000","last_became_peered":"2023-11-06T02:08:30.217091+0000","last_unstale":"2023-11-06T02:08:31.604588+0000","last_undegraded":"2023-11-06T02:08:31.604588+0000","last_fullsized":"2023-11-06T02:08:31.604588+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:04:42.682132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.610770+0000","last_change":"2023-11-06T02:08:30.660650+0000","last_active":"2023-11-06T02:08:31.610770+0000","last_peered":"2023-11-06T02:08:31.610770+0000","last_clean":"2023-11-06T02:08:31.610770+0000","last_became_active":"2023-11-06T02:08:30.660072+0000","last_became_peered":"2023-11-06T02:08:30.660072+0000","last_unstale":"2023-11-06T02:08:31.610770+0000","last_undegraded":"2023-11-06T02:08:31.610770+0000","last_fullsized":"2023-11-06T02:08:31.610770+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:30:50.860315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-06T02:08:30.626279+0000","last_change":"2023-11-06T02:08:26.060406+0000","last_active":"2023-11-06T02:08:30.626279+0000","last_peered":"2023-11-06T02:08:30.626279+0000","last_clean":"2023-11-06T02:08:30.626279+0000","last_became_active":"2023-11-06T02:08:26.059262+0000","last_became_peered":"2023-11-06T02:08:26.059262+0000","last_unstale":"2023-11-06T02:08:30.626279+0000","last_undegraded":"2023-11-06T02:08:30.626279+0000","last_fullsized":"2023-11-06T02:08:30.626279+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:10.626021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.651421+0000","last_change":"2023-11-06T02:08:30.227671+0000","last_active":"2023-11-06T02:08:32.651421+0000","last_peered":"2023-11-06T02:08:32.651421+0000","last_clean":"2023-11-06T02:08:32.651421+0000","last_became_active":"2023-11-06T02:08:30.227200+0000","last_became_peered":"2023-11-06T02:08:30.227200+0000","last_unstale":"2023-11-06T02:08:32.651421+0000","last_undegraded":"2023-11-06T02:08:32.651421+0000","last_fullsized":"2023-11-06T02:08:32.651421+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:26:49.298561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.653133+0000","last_change":"2023-11-06T02:08:30.619424+0000","last_active":"2023-11-06T02:08:32.653133+0000","last_peered":"2023-11-06T02:08:32.653133+0000","last_clean":"2023-11-06T02:08:32.653133+0000","last_became_active":"2023-11-06T02:08:30.619150+0000","last_became_peered":"2023-11-06T02:08:30.619150+0000","last_unstale":"2023-11-06T02:08:32.653133+0000","last_undegraded":"2023-11-06T02:08:32.653133+0000","last_fullsized":"2023-11-06T02:08:32.653133+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:47:30.859032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-06T02:08:30.597659+0000","last_change":"2023-11-06T02:08:26.559477+0000","last_active":"2023-11-06T02:08:30.597659+0000","last_peered":"2023-11-06T02:08:30.597659+0000","last_clean":"2023-11-06T02:08:30.597659+0000","last_became_active":"2023-11-06T02:08:26.559202+0000","last_became_peered":"2023-11-06T02:08:26.559202+0000","last_unstale":"2023-11-06T02:08:30.597659+0000","last_undegraded":"2023-11-06T02:08:30.597659+0000","last_fullsized":"2023-11-06T02:08:30.597659+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:55:04.552404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.651066+0000","last_change":"2023-11-06T02:08:30.219422+0000","last_active":"2023-11-06T02:08:32.651066+0000","last_peered":"2023-11-06T02:08:32.651066+0000","last_clean":"2023-11-06T02:08:32.651066+0000","last_became_active":"2023-11-06T02:08:30.219148+0000","last_became_peered":"2023-11-06T02:08:30.219148+0000","last_unstale":"2023-11-06T02:08:32.651066+0000","last_undegraded":"2023-11-06T02:08:32.651066+0000","last_fullsized":"2023-11-06T02:08:32.651066+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:52:19.592716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.653298+0000","last_change":"2023-11-06T02:08:30.655629+0000","last_active":"2023-11-06T02:08:32.653298+0000","last_peered":"2023-11-06T02:08:32.653298+0000","last_clean":"2023-11-06T02:08:32.653298+0000","last_became_active":"2023-11-06T02:08:30.655271+0000","last_became_peered":"2023-11-06T02:08:30.655271+0000","last_unstale":"2023-11-06T02:08:32.653298+0000","last_undegraded":"2023-11-06T02:08:32.653298+0000","last_fullsized":"2023-11-06T02:08:32.653298+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:10:14.974545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-06T02:08:30.641603+0000","last_change":"2023-11-06T02:08:26.059546+0000","last_active":"2023-11-06T02:08:30.641603+0000","last_peered":"2023-11-06T02:08:30.641603+0000","last_clean":"2023-11-06T02:08:30.641603+0000","last_became_active":"2023-11-06T02:08:26.058806+0000","last_became_peered":"2023-11-06T02:08:26.058806+0000","last_unstale":"2023-11-06T02:08:30.641603+0000","last_undegraded":"2023-11-06T02:08:30.641603+0000","last_fullsized":"2023-11-06T02:08:30.641603+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:23.468510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.609639+0000","last_change":"2023-11-06T02:08:30.223844+0000","last_active":"2023-11-06T02:08:31.609639+0000","last_peered":"2023-11-06T02:08:31.609639+0000","last_clean":"2023-11-06T02:08:31.609639+0000","last_became_active":"2023-11-06T02:08:30.222902+0000","last_became_peered":"2023-11-06T02:08:30.222902+0000","last_unstale":"2023-11-06T02:08:31.609639+0000","last_undegraded":"2023-11-06T02:08:31.609639+0000","last_fullsized":"2023-11-06T02:08:31.609639+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:52:19.592716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.651748+0000","last_change":"2023-11-06T02:08:30.619125+0000","last_active":"2023-11-06T02:08:32.651748+0000","last_peered":"2023-11-06T02:08:32.651748+0000","last_clean":"2023-11-06T02:08:32.651748+0000","last_became_active":"2023-11-06T02:08:30.617621+0000","last_became_peered":"2023-11-06T02:08:30.617621+0000","last_unstale":"2023-11-06T02:08:32.651748+0000","last_undegraded":"2023-11-06T02:08:32.651748+0000","last_fullsized":"2023-11-06T02:08:32.651748+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:13:37.616527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.413429+0000","last_change":"2023-11-06T02:08:30.225274+0000","last_active":"2023-11-06T02:08:32.413429+0000","last_peered":"2023-11-06T02:08:32.413429+0000","last_clean":"2023-11-06T02:08:32.413429+0000","last_became_active":"2023-11-06T02:08:30.224275+0000","last_became_peered":"2023-11-06T02:08:30.224275+0000","last_unstale":"2023-11-06T02:08:32.413429+0000","last_undegraded":"2023-11-06T02:08:32.413429+0000","last_fullsized":"2023-11-06T02:08:32.413429+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:26:49.298561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-06T02:08:30.614515+0000","last_change":"2023-11-06T02:08:26.436463+0000","last_active":"2023-11-06T02:08:30.614515+0000","last_peered":"2023-11-06T02:08:30.614515+0000","last_clean":"2023-11-06T02:08:30.614515+0000","last_became_active":"2023-11-06T02:08:26.436075+0000","last_became_peered":"2023-11-06T02:08:26.436075+0000","last_unstale":"2023-11-06T02:08:30.614515+0000","last_undegraded":"2023-11-06T02:08:30.614515+0000","last_fullsized":"2023-11-06T02:08:30.614515+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:23.468510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.651810+0000","last_change":"2023-11-06T02:08:30.655634+0000","last_active":"2023-11-06T02:08:32.651810+0000","last_peered":"2023-11-06T02:08:32.651810+0000","last_clean":"2023-11-06T02:08:32.651810+0000","last_became_active":"2023-11-06T02:08:30.655285+0000","last_became_peered":"2023-11-06T02:08:30.655285+0000","last_unstale":"2023-11-06T02:08:32.651810+0000","last_undegraded":"2023-11-06T02:08:32.651810+0000","last_fullsized":"2023-11-06T02:08:32.651810+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:49:05.460965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.651964+0000","last_change":"2023-11-06T02:08:30.220327+0000","last_active":"2023-11-06T02:08:32.651964+0000","last_peered":"2023-11-06T02:08:32.651964+0000","last_clean":"2023-11-06T02:08:32.651964+0000","last_became_active":"2023-11-06T02:08:30.219906+0000","last_became_peered":"2023-11-06T02:08:30.219906+0000","last_unstale":"2023-11-06T02:08:32.651964+0000","last_undegraded":"2023-11-06T02:08:32.651964+0000","last_fullsized":"2023-11-06T02:08:32.651964+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:43:56.017919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-06T02:08:30.636949+0000","last_change":"2023-11-06T02:08:26.436607+0000","last_active":"2023-11-06T02:08:30.636949+0000","last_peered":"2023-11-06T02:08:30.636949+0000","last_clean":"2023-11-06T02:08:30.636949+0000","last_became_active":"2023-11-06T02:08:26.435924+0000","last_became_peered":"2023-11-06T02:08:26.435924+0000","last_unstale":"2023-11-06T02:08:30.636949+0000","last_undegraded":"2023-11-06T02:08:30.636949+0000","last_fullsized":"2023-11-06T02:08:30.636949+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:23.468510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.652082+0000","last_change":"2023-11-06T02:08:30.654058+0000","last_active":"2023-11-06T02:08:32.652082+0000","last_peered":"2023-11-06T02:08:32.652082+0000","last_clean":"2023-11-06T02:08:32.652082+0000","last_became_active":"2023-11-06T02:08:30.653753+0000","last_became_peered":"2023-11-06T02:08:30.653753+0000","last_unstale":"2023-11-06T02:08:32.652082+0000","last_undegraded":"2023-11-06T02:08:32.652082+0000","last_fullsized":"2023-11-06T02:08:32.652082+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:13:37.616527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"22'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.605569+0000","last_change":"2023-11-06T02:08:30.231205+0000","last_active":"2023-11-06T02:08:31.605569+0000","last_peered":"2023-11-06T02:08:31.605569+0000","last_clean":"2023-11-06T02:08:31.605569+0000","last_became_active":"2023-11-06T02:08:30.230854+0000","last_became_peered":"2023-11-06T02:08:30.230854+0000","last_unstale":"2023-11-06T02:08:31.605569+0000","last_undegraded":"2023-11-06T02:08:31.605569+0000","last_fullsized":"2023-11-06T02:08:31.605569+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:26:49.298561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-06T02:08:30.614693+0000","last_change":"2023-11-06T02:08:26.057076+0000","last_active":"2023-11-06T02:08:30.614693+0000","last_peered":"2023-11-06T02:08:30.614693+0000","last_clean":"2023-11-06T02:08:30.614693+0000","last_became_active":"2023-11-06T02:08:26.056419+0000","last_became_peered":"2023-11-06T02:08:26.056419+0000","last_unstale":"2023-11-06T02:08:30.614693+0000","last_undegraded":"2023-11-06T02:08:30.614693+0000","last_fullsized":"2023-11-06T02:08:30.614693+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:10.626021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.604395+0000","last_change":"2023-11-06T02:08:30.647144+0000","last_active":"2023-11-06T02:08:31.604395+0000","last_peered":"2023-11-06T02:08:31.604395+0000","last_clean":"2023-11-06T02:08:31.604395+0000","last_became_active":"2023-11-06T02:08:30.646802+0000","last_became_peered":"2023-11-06T02:08:30.646802+0000","last_unstale":"2023-11-06T02:08:31.604395+0000","last_undegraded":"2023-11-06T02:08:31.604395+0000","last_fullsized":"2023-11-06T02:08:31.604395+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:49:05.460965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"22'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.653477+0000","last_change":"2023-11-06T02:08:30.225086+0000","last_active":"2023-11-06T02:08:32.653477+0000","last_peered":"2023-11-06T02:08:32.653477+0000","last_clean":"2023-11-06T02:08:32.653477+0000","last_became_active":"2023-11-06T02:08:30.224455+0000","last_became_peered":"2023-11-06T02:08:30.224455+0000","last_unstale":"2023-11-06T02:08:32.653477+0000","last_undegraded":"2023-11-06T02:08:32.653477+0000","last_fullsized":"2023-11-06T02:08:32.653477+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:26:21.361289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-06T02:08:30.606381+0000","last_change":"2023-11-06T02:08:26.058596+0000","last_active":"2023-11-06T02:08:30.606381+0000","last_peered":"2023-11-06T02:08:30.606381+0000","last_clean":"2023-11-06T02:08:30.606381+0000","last_became_active":"2023-11-06T02:08:26.057015+0000","last_became_peered":"2023-11-06T02:08:26.057015+0000","last_unstale":"2023-11-06T02:08:30.606381+0000","last_undegraded":"2023-11-06T02:08:30.606381+0000","last_fullsized":"2023-11-06T02:08:30.606381+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:55:04.552404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.610172+0000","last_change":"2023-11-06T02:08:30.656051+0000","last_active":"2023-11-06T02:08:31.610172+0000","last_peered":"2023-11-06T02:08:31.610172+0000","last_clean":"2023-11-06T02:08:31.610172+0000","last_became_active":"2023-11-06T02:08:30.650054+0000","last_became_peered":"2023-11-06T02:08:30.650054+0000","last_unstale":"2023-11-06T02:08:31.610172+0000","last_undegraded":"2023-11-06T02:08:31.610172+0000","last_fullsized":"2023-11-06T02:08:31.610172+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:17:50.007398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.609417+0000","last_change":"2023-11-06T02:08:30.633028+0000","last_active":"2023-11-06T02:08:31.609417+0000","last_peered":"2023-11-06T02:08:31.609417+0000","last_clean":"2023-11-06T02:08:31.609417+0000","last_became_active":"2023-11-06T02:08:30.632255+0000","last_became_peered":"2023-11-06T02:08:30.632255+0000","last_unstale":"2023-11-06T02:08:31.609417+0000","last_undegraded":"2023-11-06T02:08:31.609417+0000","last_fullsized":"2023-11-06T02:08:31.609417+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:47:30.859032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"22'6","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.609754+0000","last_change":"2023-11-06T02:08:30.230431+0000","last_active":"2023-11-06T02:08:31.609754+0000","last_peered":"2023-11-06T02:08:31.609754+0000","last_clean":"2023-11-06T02:08:31.609754+0000","last_became_active":"2023-11-06T02:08:30.229931+0000","last_became_peered":"2023-11-06T02:08:30.229931+0000","last_unstale":"2023-11-06T02:08:31.609754+0000","last_undegraded":"2023-11-06T02:08:31.609754+0000","last_fullsized":"2023-11-06T02:08:31.609754+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:26:21.361289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.651402+0000","last_change":"2023-11-06T02:08:30.628444+0000","last_active":"2023-11-06T02:08:32.651402+0000","last_peered":"2023-11-06T02:08:32.651402+0000","last_clean":"2023-11-06T02:08:32.651402+0000","last_became_active":"2023-11-06T02:08:30.628087+0000","last_became_peered":"2023-11-06T02:08:30.628087+0000","last_unstale":"2023-11-06T02:08:32.651402+0000","last_undegraded":"2023-11-06T02:08:32.651402+0000","last_fullsized":"2023-11-06T02:08:32.651402+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:57:32.079075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"22'3","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.149892+0000","last_change":"2023-11-06T02:08:30.230554+0000","last_active":"2023-11-06T02:08:32.149892+0000","last_peered":"2023-11-06T02:08:32.149892+0000","last_clean":"2023-11-06T02:08:32.149892+0000","last_became_active":"2023-11-06T02:08:30.229374+0000","last_became_peered":"2023-11-06T02:08:30.229374+0000","last_unstale":"2023-11-06T02:08:32.149892+0000","last_undegraded":"2023-11-06T02:08:32.149892+0000","last_fullsized":"2023-11-06T02:08:32.149892+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:15:58.041467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.413338+0000","last_change":"2023-11-06T02:08:30.231726+0000","last_active":"2023-11-06T02:08:32.413338+0000","last_peered":"2023-11-06T02:08:32.413338+0000","last_clean":"2023-11-06T02:08:32.413338+0000","last_became_active":"2023-11-06T02:08:30.230802+0000","last_became_peered":"2023-11-06T02:08:30.230802+0000","last_unstale":"2023-11-06T02:08:32.413338+0000","last_undegraded":"2023-11-06T02:08:32.413338+0000","last_fullsized":"2023-11-06T02:08:32.413338+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:15:58.041467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.604488+0000","last_change":"2023-11-06T02:08:30.644420+0000","last_active":"2023-11-06T02:08:31.604488+0000","last_peered":"2023-11-06T02:08:31.604488+0000","last_clean":"2023-11-06T02:08:31.604488+0000","last_became_active":"2023-11-06T02:08:30.644097+0000","last_became_peered":"2023-11-06T02:08:30.644097+0000","last_unstale":"2023-11-06T02:08:31.604488+0000","last_undegraded":"2023-11-06T02:08:31.604488+0000","last_fullsized":"2023-11-06T02:08:31.604488+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:31:28.301664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.651911+0000","last_change":"2023-11-06T02:08:30.218801+0000","last_active":"2023-11-06T02:08:32.651911+0000","last_peered":"2023-11-06T02:08:32.651911+0000","last_clean":"2023-11-06T02:08:32.651911+0000","last_became_active":"2023-11-06T02:08:30.217237+0000","last_became_peered":"2023-11-06T02:08:30.217237+0000","last_unstale":"2023-11-06T02:08:32.651911+0000","last_undegraded":"2023-11-06T02:08:32.651911+0000","last_fullsized":"2023-11-06T02:08:32.651911+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:40:04.297054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.604444+0000","last_change":"2023-11-06T02:08:30.619501+0000","last_active":"2023-11-06T02:08:31.604444+0000","last_peered":"2023-11-06T02:08:31.604444+0000","last_clean":"2023-11-06T02:08:31.604444+0000","last_became_active":"2023-11-06T02:08:30.619180+0000","last_became_peered":"2023-11-06T02:08:30.619180+0000","last_unstale":"2023-11-06T02:08:31.604444+0000","last_undegraded":"2023-11-06T02:08:31.604444+0000","last_fullsized":"2023-11-06T02:08:31.604444+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:13:37.616527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.611492+0000","last_change":"2023-11-06T02:08:30.655759+0000","last_active":"2023-11-06T02:08:31.611492+0000","last_peered":"2023-11-06T02:08:31.611492+0000","last_clean":"2023-11-06T02:08:31.611492+0000","last_became_active":"2023-11-06T02:08:30.655285+0000","last_became_peered":"2023-11-06T02:08:30.655285+0000","last_unstale":"2023-11-06T02:08:31.611492+0000","last_undegraded":"2023-11-06T02:08:31.611492+0000","last_fullsized":"2023-11-06T02:08:31.611492+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:20:00.351057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.604479+0000","last_change":"2023-11-06T02:08:30.222521+0000","last_active":"2023-11-06T02:08:31.604479+0000","last_peered":"2023-11-06T02:08:31.604479+0000","last_clean":"2023-11-06T02:08:31.604479+0000","last_became_active":"2023-11-06T02:08:30.221899+0000","last_became_peered":"2023-11-06T02:08:30.221899+0000","last_unstale":"2023-11-06T02:08:31.604479+0000","last_undegraded":"2023-11-06T02:08:31.604479+0000","last_fullsized":"2023-11-06T02:08:31.604479+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:43:56.017919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.652973+0000","last_change":"2023-11-06T02:08:30.644462+0000","last_active":"2023-11-06T02:08:32.652973+0000","last_peered":"2023-11-06T02:08:32.652973+0000","last_clean":"2023-11-06T02:08:32.652973+0000","last_became_active":"2023-11-06T02:08:30.644160+0000","last_became_peered":"2023-11-06T02:08:30.644160+0000","last_unstale":"2023-11-06T02:08:32.652973+0000","last_undegraded":"2023-11-06T02:08:32.652973+0000","last_fullsized":"2023-11-06T02:08:32.652973+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:43:23.215718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.652108+0000","last_change":"2023-11-06T02:08:30.214519+0000","last_active":"2023-11-06T02:08:32.652108+0000","last_peered":"2023-11-06T02:08:32.652108+0000","last_clean":"2023-11-06T02:08:32.652108+0000","last_became_active":"2023-11-06T02:08:30.214225+0000","last_became_peered":"2023-11-06T02:08:30.214225+0000","last_unstale":"2023-11-06T02:08:32.652108+0000","last_undegraded":"2023-11-06T02:08:32.652108+0000","last_fullsized":"2023-11-06T02:08:32.652108+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:30:21.274364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.605196+0000","last_change":"2023-11-06T02:08:30.617971+0000","last_active":"2023-11-06T02:08:31.605196+0000","last_peered":"2023-11-06T02:08:31.605196+0000","last_clean":"2023-11-06T02:08:31.605196+0000","last_became_active":"2023-11-06T02:08:30.617692+0000","last_became_peered":"2023-11-06T02:08:30.617692+0000","last_unstale":"2023-11-06T02:08:31.605196+0000","last_undegraded":"2023-11-06T02:08:31.605196+0000","last_fullsized":"2023-11-06T02:08:31.605196+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:23:59.639528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"22'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.609142+0000","last_change":"2023-11-06T02:08:30.224834+0000","last_active":"2023-11-06T02:08:31.609142+0000","last_peered":"2023-11-06T02:08:31.609142+0000","last_clean":"2023-11-06T02:08:31.609142+0000","last_became_active":"2023-11-06T02:08:30.224244+0000","last_became_peered":"2023-11-06T02:08:30.224244+0000","last_unstale":"2023-11-06T02:08:31.609142+0000","last_undegraded":"2023-11-06T02:08:31.609142+0000","last_fullsized":"2023-11-06T02:08:31.609142+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:40:04.297054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.654928+0000","last_change":"2023-11-06T02:08:30.619430+0000","last_active":"2023-11-06T02:08:32.654928+0000","last_peered":"2023-11-06T02:08:32.654928+0000","last_clean":"2023-11-06T02:08:32.654928+0000","last_became_active":"2023-11-06T02:08:30.619155+0000","last_became_peered":"2023-11-06T02:08:30.619155+0000","last_unstale":"2023-11-06T02:08:32.654928+0000","last_undegraded":"2023-11-06T02:08:32.654928+0000","last_fullsized":"2023-11-06T02:08:32.654928+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:30:22.278201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"22'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.652274+0000","last_change":"2023-11-06T02:08:30.227830+0000","last_active":"2023-11-06T02:08:32.652274+0000","last_peered":"2023-11-06T02:08:32.652274+0000","last_clean":"2023-11-06T02:08:32.652274+0000","last_became_active":"2023-11-06T02:08:30.227476+0000","last_became_peered":"2023-11-06T02:08:30.227476+0000","last_unstale":"2023-11-06T02:08:32.652274+0000","last_undegraded":"2023-11-06T02:08:32.652274+0000","last_fullsized":"2023-11-06T02:08:32.652274+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:30:21.274364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.609173+0000","last_change":"2023-11-06T02:08:30.651612+0000","last_active":"2023-11-06T02:08:31.609173+0000","last_peered":"2023-11-06T02:08:31.609173+0000","last_clean":"2023-11-06T02:08:31.609173+0000","last_became_active":"2023-11-06T02:08:30.650925+0000","last_became_peered":"2023-11-06T02:08:30.650925+0000","last_unstale":"2023-11-06T02:08:31.609173+0000","last_undegraded":"2023-11-06T02:08:31.609173+0000","last_fullsized":"2023-11-06T02:08:31.609173+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:31:28.301664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"22'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.413042+0000","last_change":"2023-11-06T02:08:30.217273+0000","last_active":"2023-11-06T02:08:32.413042+0000","last_peered":"2023-11-06T02:08:32.413042+0000","last_clean":"2023-11-06T02:08:32.413042+0000","last_became_active":"2023-11-06T02:08:30.216532+0000","last_became_peered":"2023-11-06T02:08:30.216532+0000","last_unstale":"2023-11-06T02:08:32.413042+0000","last_undegraded":"2023-11-06T02:08:32.413042+0000","last_fullsized":"2023-11-06T02:08:32.413042+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:33:38.355715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.608910+0000","last_change":"2023-11-06T02:08:30.651587+0000","last_active":"2023-11-06T02:08:31.608910+0000","last_peered":"2023-11-06T02:08:31.608910+0000","last_clean":"2023-11-06T02:08:31.608910+0000","last_became_active":"2023-11-06T02:08:30.650762+0000","last_became_peered":"2023-11-06T02:08:30.650762+0000","last_unstale":"2023-11-06T02:08:31.608910+0000","last_undegraded":"2023-11-06T02:08:31.608910+0000","last_fullsized":"2023-11-06T02:08:31.608910+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:52.535882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"22'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.413347+0000","last_change":"2023-11-06T02:08:30.220284+0000","last_active":"2023-11-06T02:08:32.413347+0000","last_peered":"2023-11-06T02:08:32.413347+0000","last_clean":"2023-11-06T02:08:32.413347+0000","last_became_active":"2023-11-06T02:08:30.219571+0000","last_became_peered":"2023-11-06T02:08:30.219571+0000","last_unstale":"2023-11-06T02:08:32.413347+0000","last_undegraded":"2023-11-06T02:08:32.413347+0000","last_fullsized":"2023-11-06T02:08:32.413347+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:24:09.801921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.604654+0000","last_change":"2023-11-06T02:08:30.621580+0000","last_active":"2023-11-06T02:08:31.604654+0000","last_peered":"2023-11-06T02:08:31.604654+0000","last_clean":"2023-11-06T02:08:31.604654+0000","last_became_active":"2023-11-06T02:08:30.618164+0000","last_became_peered":"2023-11-06T02:08:30.618164+0000","last_unstale":"2023-11-06T02:08:31.604654+0000","last_undegraded":"2023-11-06T02:08:31.604654+0000","last_fullsized":"2023-11-06T02:08:31.604654+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:57:32.079075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"22'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.604995+0000","last_change":"2023-11-06T02:08:30.208980+0000","last_active":"2023-11-06T02:08:31.604995+0000","last_peered":"2023-11-06T02:08:31.604995+0000","last_clean":"2023-11-06T02:08:31.604995+0000","last_became_active":"2023-11-06T02:08:30.208339+0000","last_became_peered":"2023-11-06T02:08:30.208339+0000","last_unstale":"2023-11-06T02:08:31.604995+0000","last_undegraded":"2023-11-06T02:08:31.604995+0000","last_fullsized":"2023-11-06T02:08:31.604995+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:31:31.882709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.608588+0000","last_change":"2023-11-06T02:08:30.634399+0000","last_active":"2023-11-06T02:08:31.608588+0000","last_peered":"2023-11-06T02:08:31.608588+0000","last_clean":"2023-11-06T02:08:31.608588+0000","last_became_active":"2023-11-06T02:08:30.633674+0000","last_became_peered":"2023-11-06T02:08:30.633674+0000","last_unstale":"2023-11-06T02:08:31.608588+0000","last_undegraded":"2023-11-06T02:08:31.608588+0000","last_fullsized":"2023-11-06T02:08:31.608588+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:14:45.248292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1f","version":"22'4","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.651772+0000","last_change":"2023-11-06T02:08:30.220947+0000","last_active":"2023-11-06T02:08:32.651772+0000","last_peered":"2023-11-06T02:08:32.651772+0000","last_clean":"2023-11-06T02:08:32.651772+0000","last_became_active":"2023-11-06T02:08:30.220379+0000","last_became_peered":"2023-11-06T02:08:30.220379+0000","last_unstale":"2023-11-06T02:08:32.651772+0000","last_undegraded":"2023-11-06T02:08:32.651772+0000","last_fullsized":"2023-11-06T02:08:32.651772+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:43:56.017919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27796,"kb_used_data":844,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344044,"statfs":{"total":96636764160,"available":96608301056,"internally_reserved":0,"allocated":864256,"data_stored":681747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69799999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0660000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0029999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83499999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91500000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95599999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0960000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27188,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":241664,"data_stored":89599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0820000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93400000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91900000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81399999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.997}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":244,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":249856,"data_stored":90251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83099999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78000000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0049999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.165}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91500000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1870000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":262144,"data_stored":91219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0129999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95599999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.145}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1299999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.101}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2150000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31288,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340552,"statfs":{"total":96636764160,"available":96604725248,"internally_reserved":0,"allocated":245760,"data_stored":89689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89200000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.266}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84899999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99399999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31284,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340556,"statfs":{"total":96636764160,"available":96604729344,"internally_reserved":0,"allocated":241664,"data_stored":89599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.311}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70399999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71399999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.625}]}]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31284,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340556,"statfs":{"total":96636764160,"available":96604729344,"internally_reserved":0,"allocated":241664,"data_stored":89599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.503}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73899999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84599999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31864,"kb_used_data":816,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339976,"statfs":{"total":96636764160,"available":96604135424,"internally_reserved":0,"allocated":835584,"data_stored":679530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71099999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0009999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95899999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79800000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2270000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.194}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T02:08:43.129 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T02:08:43.280 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.282+0000 7fb739edf640 1 Processor -- start 2023-11-06T02:08:43.280 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.282+0000 7fb739edf640 1 -- start start 2023-11-06T02:08:43.281 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.286+0000 7fb739edf640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fb7340b0790 0x7fb7340c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:43.282 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.286+0000 7fb739edf640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb7340cb010 0x7fb7340c2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:43.282 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.286+0000 7fb739edf640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7340cab80 con 0x7fb73415ab50 2023-11-06T02:08:43.282 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.286+0000 7fb739edf640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb73415af30 con 0x7fb7340cb010 2023-11-06T02:08:43.282 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.286+0000 7fb739edf640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7340cb5b0 con 0x7fb7340b0790 2023-11-06T02:08:43.283 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.286+0000 7fb733fff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb7340cb010 0x7fb7340c2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:43.283 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.286+0000 7fb733fff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb7340cb010 0x7fb7340c2710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52456/0 (socket says 172.21.15.49:52456) 2023-11-06T02:08:43.283 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.286+0000 7fb733fff640 1 -- 172.21.15.49:0/3999619328 learned_addr learned my addr 172.21.15.49:0/3999619328 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:43.283 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.286+0000 7fb732ffd640 1 --2- 172.21.15.49:0/3999619328 >> v2:172.21.15.150:3300/0 conn(0x7fb7340b0790 0x7fb7340c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:43.283 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.286+0000 7fb733fff640 1 -- 172.21.15.49:0/3999619328 >> v2:172.21.15.150:3300/0 conn(0x7fb7340b0790 msgr2=0x7fb7340c9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:43.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.286+0000 7fb733fff640 1 --2- 172.21.15.49:0/3999619328 >> v2:172.21.15.150:3300/0 conn(0x7fb7340b0790 0x7fb7340c9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:43.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.286+0000 7fb733fff640 1 -- 172.21.15.49:0/3999619328 >> v1:172.21.15.49:6789/0 conn(0x7fb73415ab50 legacy=0x7fb7340b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:43.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.286+0000 7fb733fff640 1 -- 172.21.15.49:0/3999619328 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7340cb890 con 0x7fb7340cb010 2023-11-06T02:08:43.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.286+0000 7fb733fff640 1 --2- 172.21.15.49:0/3999619328 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7340cb010 0x7fb7340c2710 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb73406dde0 tx=0x7fb72c01c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=802f9f836c6673e5 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:43.286 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.286+0000 7fb7327fc640 1 -- 172.21.15.49:0/3999619328 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb72c0a5860 con 0x7fb7340cb010 2023-11-06T02:08:43.286 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.286+0000 7fb7327fc640 1 -- 172.21.15.49:0/3999619328 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb72c0a59c0 con 0x7fb7340cb010 2023-11-06T02:08:43.286 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.286+0000 7fb7327fc640 1 -- 172.21.15.49:0/3999619328 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb72c0a5d50 con 0x7fb7340cb010 2023-11-06T02:08:43.286 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.286+0000 7fb739edf640 1 -- 172.21.15.49:0/3999619328 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7340cb010 msgr2=0x7fb7340c2710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:43.286 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.286+0000 7fb739edf640 1 --2- 172.21.15.49:0/3999619328 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7340cb010 0x7fb7340c2710 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb73406dde0 tx=0x7fb72c01c9a0 comp rx=0 tx=0).stop 2023-11-06T02:08:43.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.286+0000 7fb739edf640 1 -- 172.21.15.49:0/3999619328 shutdown_connections 2023-11-06T02:08:43.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.286+0000 7fb739edf640 1 --2- 172.21.15.49:0/3999619328 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7340cb010 0x7fb7340c2710 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:43.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.286+0000 7fb739edf640 1 --2- 172.21.15.49:0/3999619328 >> v2:172.21.15.150:3300/0 conn(0x7fb7340b0790 0x7fb7340c9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:43.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.286+0000 7fb739edf640 1 -- 172.21.15.49:0/3999619328 >> 172.21.15.49:0/3999619328 conn(0x7fb73406c0a0 msgr2=0x7fb73406c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:43.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.286+0000 7fb739edf640 1 -- 172.21.15.49:0/3999619328 shutdown_connections 2023-11-06T02:08:43.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.286+0000 7fb739edf640 1 -- 172.21.15.49:0/3999619328 wait complete. 2023-11-06T02:08:43.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.290+0000 7fb739edf640 1 Processor -- start 2023-11-06T02:08:43.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.290+0000 7fb739edf640 1 -- start start 2023-11-06T02:08:43.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.290+0000 7fb739edf640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7340b0790 0x7fb73415c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:43.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.290+0000 7fb739edf640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7340cb010 0x7fb734147a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:43.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.290+0000 7fb739edf640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb73415ab50 0x7fb73415d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:43.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.290+0000 7fb739edf640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb734156070 con 0x7fb7340b0790 2023-11-06T02:08:43.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.290+0000 7fb739edf640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb73415ca10 con 0x7fb73415ab50 2023-11-06T02:08:43.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.290+0000 7fb739edf640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb73415b430 con 0x7fb7340cb010 2023-11-06T02:08:43.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.290+0000 7fb7337fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7340b0790 0x7fb73415c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:43.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.290+0000 7fb7337fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7340b0790 0x7fb73415c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52472/0 (socket says 172.21.15.49:52472) 2023-11-06T02:08:43.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.290+0000 7fb7337fe640 1 -- 172.21.15.49:0/1935391993 learned_addr learned my addr 172.21.15.49:0/1935391993 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:43.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.290+0000 7fb733fff640 1 --2- 172.21.15.49:0/1935391993 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb73415ab50 0x7fb73415d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:43.293 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.290+0000 7fb732ffd640 1 --2- 172.21.15.49:0/1935391993 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7340cb010 0x7fb734147a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:43.293 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.290+0000 7fb7337fe640 1 -- 172.21.15.49:0/1935391993 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7340cb010 msgr2=0x7fb734147a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:43.293 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.290+0000 7fb7337fe640 1 --2- 172.21.15.49:0/1935391993 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7340cb010 0x7fb734147a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:43.293 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.290+0000 7fb7337fe640 1 -- 172.21.15.49:0/1935391993 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb73415ab50 msgr2=0x7fb73415d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:43.293 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.290+0000 7fb7337fe640 1 --2- 172.21.15.49:0/1935391993 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb73415ab50 0x7fb73415d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:43.294 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.290+0000 7fb7337fe640 1 -- 172.21.15.49:0/1935391993 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb72c009b30 con 0x7fb7340b0790 2023-11-06T02:08:43.294 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.290+0000 7fb733fff640 1 --2- 172.21.15.49:0/1935391993 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb73415ab50 0x7fb73415d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:08:43.294 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.290+0000 7fb7337fe640 1 --2- 172.21.15.49:0/1935391993 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7340b0790 0x7fb73415c4d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb72400c3c0 tx=0x7fb72400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:43.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.290+0000 7fb71bfff640 1 -- 172.21.15.49:0/1935391993 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7240054e0 con 0x7fb7340b0790 2023-11-06T02:08:43.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.290+0000 7fb71bfff640 1 -- 172.21.15.49:0/1935391993 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb724003600 con 0x7fb7340b0790 2023-11-06T02:08:43.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.290+0000 7fb739edf640 1 -- 172.21.15.49:0/1935391993 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb73415b710 con 0x7fb7340b0790 2023-11-06T02:08:43.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.290+0000 7fb71bfff640 1 -- 172.21.15.49:0/1935391993 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7240039d0 con 0x7fb7340b0790 2023-11-06T02:08:43.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.290+0000 7fb739edf640 1 -- 172.21.15.49:0/1935391993 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb73415bcd0 con 0x7fb7340b0790 2023-11-06T02:08:43.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.290+0000 7fb739edf640 1 -- 172.21.15.49:0/1935391993 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7340b0de0 con 0x7fb7340b0790 2023-11-06T02:08:43.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.290+0000 7fb71bfff640 1 -- 172.21.15.49:0/1935391993 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fb724003b30 con 0x7fb7340b0790 2023-11-06T02:08:43.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.294+0000 7fb71bfff640 1 --2- 172.21.15.49:0/1935391993 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb720065a60 0x7fb720067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:43.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.294+0000 7fb71bfff640 1 -- 172.21.15.49:0/1935391993 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb724082800 con 0x7fb7340b0790 2023-11-06T02:08:43.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.294+0000 7fb732ffd640 1 --2- 172.21.15.49:0/1935391993 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb720065a60 0x7fb720067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:43.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.294+0000 7fb732ffd640 1 --2- 172.21.15.49:0/1935391993 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb720065a60 0x7fb720067f20 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb7340acfb0 tx=0x7fb728008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:43.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.298+0000 7fb71bfff640 1 -- 172.21.15.49:0/1935391993 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fb724050060 con 0x7fb7340b0790 2023-11-06T02:08:43.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.442+0000 7fb739edf640 1 -- 172.21.15.49:0/1935391993 --> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb7340cbe50 con 0x7fb720065a60 2023-11-06T02:08:43.461 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.466+0000 7fb71bfff640 1 -- 172.21.15.49:0/1935391993 <== mgr.4100 v2:172.21.15.150:6832/38104 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286584 (secure 0 0 0) 0x7fb72800b670 con 0x7fb720065a60 2023-11-06T02:08:43.463 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:08:43.465 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2023-11-06T02:08:43.466 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.470+0000 7fb739edf640 1 -- 172.21.15.49:0/1935391993 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb720065a60 msgr2=0x7fb720067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:43.466 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.470+0000 7fb739edf640 1 --2- 172.21.15.49:0/1935391993 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb720065a60 0x7fb720067f20 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb7340acfb0 tx=0x7fb728008040 comp rx=0 tx=0).stop 2023-11-06T02:08:43.467 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.470+0000 7fb739edf640 1 -- 172.21.15.49:0/1935391993 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7340b0790 msgr2=0x7fb73415c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:43.467 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.470+0000 7fb739edf640 1 --2- 172.21.15.49:0/1935391993 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7340b0790 0x7fb73415c4d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb72400c3c0 tx=0x7fb72400c880 comp rx=0 tx=0).stop 2023-11-06T02:08:43.467 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.470+0000 7fb739edf640 1 -- 172.21.15.49:0/1935391993 shutdown_connections 2023-11-06T02:08:43.467 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.470+0000 7fb739edf640 1 --2- 172.21.15.49:0/1935391993 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb720065a60 0x7fb720067f20 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:43.467 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.470+0000 7fb739edf640 1 --2- 172.21.15.49:0/1935391993 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb73415ab50 0x7fb73415d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:43.468 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.470+0000 7fb739edf640 1 --2- 172.21.15.49:0/1935391993 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7340cb010 0x7fb734147a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:43.468 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.470+0000 7fb739edf640 1 --2- 172.21.15.49:0/1935391993 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7340b0790 0x7fb73415c4d0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:43.468 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.470+0000 7fb739edf640 1 -- 172.21.15.49:0/1935391993 >> 172.21.15.49:0/1935391993 conn(0x7fb73406c0a0 msgr2=0x7fb7340ad190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:43.468 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.470+0000 7fb739edf640 1 -- 172.21.15.49:0/1935391993 shutdown_connections 2023-11-06T02:08:43.469 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.470+0000 7fb739edf640 1 -- 172.21.15.49:0/1935391993 wait complete. 2023-11-06T02:08:43.481 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":29,"stamp":"2023-11-06T02:08:43.179093+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235108,"kb_used_data":3108,"kb_used_omap":0,"kb_used_meta":231936,"kb_avail":754739612,"statfs":{"total":773094113280,"available":772853362688,"internally_reserved":0,"allocated":3182592,"data_stored":1901233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":237502464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.573473"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.655111+0000","last_change":"2023-11-06T02:08:30.608781+0000","last_active":"2023-11-06T02:08:32.655111+0000","last_peered":"2023-11-06T02:08:32.655111+0000","last_clean":"2023-11-06T02:08:32.655111+0000","last_became_active":"2023-11-06T02:08:30.606710+0000","last_became_peered":"2023-11-06T02:08:30.606710+0000","last_unstale":"2023-11-06T02:08:32.655111+0000","last_undegraded":"2023-11-06T02:08:32.655111+0000","last_fullsized":"2023-11-06T02:08:32.655111+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:43:57.021755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.413091+0000","last_change":"2023-11-06T02:08:30.646508+0000","last_active":"2023-11-06T02:08:32.413091+0000","last_peered":"2023-11-06T02:08:32.413091+0000","last_clean":"2023-11-06T02:08:32.413091+0000","last_became_active":"2023-11-06T02:08:30.645615+0000","last_became_peered":"2023-11-06T02:08:30.645615+0000","last_unstale":"2023-11-06T02:08:32.413091+0000","last_undegraded":"2023-11-06T02:08:32.413091+0000","last_fullsized":"2023-11-06T02:08:32.413091+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:49:05.460965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.610298+0000","last_change":"2023-11-06T02:08:30.658350+0000","last_active":"2023-11-06T02:08:31.610298+0000","last_peered":"2023-11-06T02:08:31.610298+0000","last_clean":"2023-11-06T02:08:31.610298+0000","last_became_active":"2023-11-06T02:08:30.651992+0000","last_became_peered":"2023-11-06T02:08:30.651992+0000","last_unstale":"2023-11-06T02:08:31.610298+0000","last_undegraded":"2023-11-06T02:08:31.610298+0000","last_fullsized":"2023-11-06T02:08:31.610298+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:09:56.455105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.610511+0000","last_change":"2023-11-06T02:08:30.627209+0000","last_active":"2023-11-06T02:08:31.610511+0000","last_peered":"2023-11-06T02:08:31.610511+0000","last_clean":"2023-11-06T02:08:31.610511+0000","last_became_active":"2023-11-06T02:08:30.626345+0000","last_became_peered":"2023-11-06T02:08:30.626345+0000","last_unstale":"2023-11-06T02:08:31.610511+0000","last_undegraded":"2023-11-06T02:08:31.610511+0000","last_fullsized":"2023-11-06T02:08:31.610511+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:46:38.867945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.652203+0000","last_change":"2023-11-06T02:08:30.619667+0000","last_active":"2023-11-06T02:08:32.652203+0000","last_peered":"2023-11-06T02:08:32.652203+0000","last_clean":"2023-11-06T02:08:32.652203+0000","last_became_active":"2023-11-06T02:08:30.619328+0000","last_became_peered":"2023-11-06T02:08:30.619328+0000","last_unstale":"2023-11-06T02:08:32.652203+0000","last_undegraded":"2023-11-06T02:08:32.652203+0000","last_fullsized":"2023-11-06T02:08:32.652203+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:26:58.090934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.608847+0000","last_change":"2023-11-06T02:08:30.657202+0000","last_active":"2023-11-06T02:08:31.608847+0000","last_peered":"2023-11-06T02:08:31.608847+0000","last_clean":"2023-11-06T02:08:31.608847+0000","last_became_active":"2023-11-06T02:08:30.652132+0000","last_became_peered":"2023-11-06T02:08:30.652132+0000","last_unstale":"2023-11-06T02:08:31.608847+0000","last_undegraded":"2023-11-06T02:08:31.608847+0000","last_fullsized":"2023-11-06T02:08:31.608847+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:35:19.038827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.149393+0000","last_change":"2023-11-06T02:08:30.645694+0000","last_active":"2023-11-06T02:08:32.149393+0000","last_peered":"2023-11-06T02:08:32.149393+0000","last_clean":"2023-11-06T02:08:32.149393+0000","last_became_active":"2023-11-06T02:08:30.645028+0000","last_became_peered":"2023-11-06T02:08:30.645028+0000","last_unstale":"2023-11-06T02:08:32.149393+0000","last_undegraded":"2023-11-06T02:08:32.149393+0000","last_fullsized":"2023-11-06T02:08:32.149393+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:26:58.090934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.412662+0000","last_change":"2023-11-06T02:08:30.652585+0000","last_active":"2023-11-06T02:08:32.412662+0000","last_peered":"2023-11-06T02:08:32.412662+0000","last_clean":"2023-11-06T02:08:32.412662+0000","last_became_active":"2023-11-06T02:08:30.652103+0000","last_became_peered":"2023-11-06T02:08:30.652103+0000","last_unstale":"2023-11-06T02:08:32.412662+0000","last_undegraded":"2023-11-06T02:08:32.412662+0000","last_fullsized":"2023-11-06T02:08:32.412662+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:57:32.079075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.651445+0000","last_change":"2023-11-06T02:08:30.618708+0000","last_active":"2023-11-06T02:08:32.651445+0000","last_peered":"2023-11-06T02:08:32.651445+0000","last_clean":"2023-11-06T02:08:32.651445+0000","last_became_active":"2023-11-06T02:08:30.618232+0000","last_became_peered":"2023-11-06T02:08:30.618232+0000","last_unstale":"2023-11-06T02:08:32.651445+0000","last_undegraded":"2023-11-06T02:08:32.651445+0000","last_fullsized":"2023-11-06T02:08:32.651445+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:31:28.301664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.413222+0000","last_change":"2023-11-06T02:08:30.640907+0000","last_active":"2023-11-06T02:08:32.413222+0000","last_peered":"2023-11-06T02:08:32.413222+0000","last_clean":"2023-11-06T02:08:32.413222+0000","last_became_active":"2023-11-06T02:08:30.626276+0000","last_became_peered":"2023-11-06T02:08:30.626276+0000","last_unstale":"2023-11-06T02:08:32.413222+0000","last_undegraded":"2023-11-06T02:08:32.413222+0000","last_fullsized":"2023-11-06T02:08:32.413222+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:56:51.565719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.610554+0000","last_change":"2023-11-06T02:08:30.658929+0000","last_active":"2023-11-06T02:08:31.610554+0000","last_peered":"2023-11-06T02:08:31.610554+0000","last_clean":"2023-11-06T02:08:31.610554+0000","last_became_active":"2023-11-06T02:08:30.657695+0000","last_became_peered":"2023-11-06T02:08:30.657695+0000","last_unstale":"2023-11-06T02:08:31.610554+0000","last_undegraded":"2023-11-06T02:08:31.610554+0000","last_fullsized":"2023-11-06T02:08:31.610554+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:57:32.079075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.654812+0000","last_change":"2023-11-06T02:08:30.660659+0000","last_active":"2023-11-06T02:08:32.654812+0000","last_peered":"2023-11-06T02:08:32.654812+0000","last_clean":"2023-11-06T02:08:32.654812+0000","last_became_active":"2023-11-06T02:08:30.660402+0000","last_became_peered":"2023-11-06T02:08:30.660402+0000","last_unstale":"2023-11-06T02:08:32.654812+0000","last_undegraded":"2023-11-06T02:08:32.654812+0000","last_fullsized":"2023-11-06T02:08:32.654812+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:26:50.302398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.654670+0000","last_change":"2023-11-06T02:08:30.608913+0000","last_active":"2023-11-06T02:08:32.654670+0000","last_peered":"2023-11-06T02:08:32.654670+0000","last_clean":"2023-11-06T02:08:32.654670+0000","last_became_active":"2023-11-06T02:08:30.606808+0000","last_became_peered":"2023-11-06T02:08:30.606808+0000","last_unstale":"2023-11-06T02:08:32.654670+0000","last_undegraded":"2023-11-06T02:08:32.654670+0000","last_fullsized":"2023-11-06T02:08:32.654670+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:24:10.805757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.412935+0000","last_change":"2023-11-06T02:08:30.653750+0000","last_active":"2023-11-06T02:08:32.412935+0000","last_peered":"2023-11-06T02:08:32.412935+0000","last_clean":"2023-11-06T02:08:32.412935+0000","last_became_active":"2023-11-06T02:08:30.653252+0000","last_became_peered":"2023-11-06T02:08:30.653252+0000","last_unstale":"2023-11-06T02:08:32.412935+0000","last_undegraded":"2023-11-06T02:08:32.412935+0000","last_fullsized":"2023-11-06T02:08:32.412935+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:26:58.090934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.149589+0000","last_change":"2023-11-06T02:08:30.644591+0000","last_active":"2023-11-06T02:08:32.149589+0000","last_peered":"2023-11-06T02:08:32.149589+0000","last_clean":"2023-11-06T02:08:32.149589+0000","last_became_active":"2023-11-06T02:08:30.643848+0000","last_became_peered":"2023-11-06T02:08:30.643848+0000","last_unstale":"2023-11-06T02:08:32.149589+0000","last_undegraded":"2023-11-06T02:08:32.149589+0000","last_fullsized":"2023-11-06T02:08:32.149589+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:49:05.460965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.608423+0000","last_change":"2023-11-06T02:08:30.634202+0000","last_active":"2023-11-06T02:08:31.608423+0000","last_peered":"2023-11-06T02:08:31.608423+0000","last_clean":"2023-11-06T02:08:31.608423+0000","last_became_active":"2023-11-06T02:08:30.633451+0000","last_became_peered":"2023-11-06T02:08:30.633451+0000","last_unstale":"2023-11-06T02:08:31.608423+0000","last_undegraded":"2023-11-06T02:08:31.608423+0000","last_fullsized":"2023-11-06T02:08:31.608423+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:13:14.793314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.651622+0000","last_change":"2023-11-06T02:08:30.618515+0000","last_active":"2023-11-06T02:08:32.651622+0000","last_peered":"2023-11-06T02:08:32.651622+0000","last_clean":"2023-11-06T02:08:32.651622+0000","last_became_active":"2023-11-06T02:08:30.617836+0000","last_became_peered":"2023-11-06T02:08:30.617836+0000","last_unstale":"2023-11-06T02:08:32.651622+0000","last_undegraded":"2023-11-06T02:08:32.651622+0000","last_fullsized":"2023-11-06T02:08:32.651622+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:17:50.007398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.609393+0000","last_change":"2023-11-06T02:08:30.659348+0000","last_active":"2023-11-06T02:08:31.609393+0000","last_peered":"2023-11-06T02:08:31.609393+0000","last_clean":"2023-11-06T02:08:31.609393+0000","last_became_active":"2023-11-06T02:08:30.652231+0000","last_became_peered":"2023-11-06T02:08:30.652231+0000","last_unstale":"2023-11-06T02:08:31.609393+0000","last_undegraded":"2023-11-06T02:08:31.609393+0000","last_fullsized":"2023-11-06T02:08:31.609393+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:13:26.942266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.655090+0000","last_change":"2023-11-06T02:08:30.627395+0000","last_active":"2023-11-06T02:08:32.655090+0000","last_peered":"2023-11-06T02:08:32.655090+0000","last_clean":"2023-11-06T02:08:32.655090+0000","last_became_active":"2023-11-06T02:08:30.627048+0000","last_became_peered":"2023-11-06T02:08:30.627048+0000","last_unstale":"2023-11-06T02:08:32.655090+0000","last_undegraded":"2023-11-06T02:08:32.655090+0000","last_fullsized":"2023-11-06T02:08:32.655090+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:15:59.045303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.609522+0000","last_change":"2023-11-06T02:08:30.658560+0000","last_active":"2023-11-06T02:08:31.609522+0000","last_peered":"2023-11-06T02:08:31.609522+0000","last_clean":"2023-11-06T02:08:31.609522+0000","last_became_active":"2023-11-06T02:08:30.657560+0000","last_became_peered":"2023-11-06T02:08:30.657560+0000","last_unstale":"2023-11-06T02:08:31.609522+0000","last_undegraded":"2023-11-06T02:08:31.609522+0000","last_fullsized":"2023-11-06T02:08:31.609522+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:26:06.040192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.149579+0000","last_change":"2023-11-06T02:08:30.650121+0000","last_active":"2023-11-06T02:08:32.149579+0000","last_peered":"2023-11-06T02:08:32.149579+0000","last_clean":"2023-11-06T02:08:32.149579+0000","last_became_active":"2023-11-06T02:08:30.649010+0000","last_became_peered":"2023-11-06T02:08:30.649010+0000","last_unstale":"2023-11-06T02:08:32.149579+0000","last_undegraded":"2023-11-06T02:08:32.149579+0000","last_fullsized":"2023-11-06T02:08:32.149579+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:56:51.565719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.651624+0000","last_change":"2023-11-06T02:08:30.622216+0000","last_active":"2023-11-06T02:08:32.651624+0000","last_peered":"2023-11-06T02:08:32.651624+0000","last_clean":"2023-11-06T02:08:32.651624+0000","last_became_active":"2023-11-06T02:08:30.621706+0000","last_became_peered":"2023-11-06T02:08:30.621706+0000","last_unstale":"2023-11-06T02:08:32.651624+0000","last_undegraded":"2023-11-06T02:08:32.651624+0000","last_fullsized":"2023-11-06T02:08:32.651624+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:52:47.174384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.604379+0000","last_change":"2023-11-06T02:08:30.623436+0000","last_active":"2023-11-06T02:08:31.604379+0000","last_peered":"2023-11-06T02:08:31.604379+0000","last_clean":"2023-11-06T02:08:31.604379+0000","last_became_active":"2023-11-06T02:08:30.623149+0000","last_became_peered":"2023-11-06T02:08:30.623149+0000","last_unstale":"2023-11-06T02:08:31.604379+0000","last_undegraded":"2023-11-06T02:08:31.604379+0000","last_fullsized":"2023-11-06T02:08:31.604379+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:47:30.859032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.149601+0000","last_change":"2023-11-06T02:08:30.656899+0000","last_active":"2023-11-06T02:08:32.149601+0000","last_peered":"2023-11-06T02:08:32.149601+0000","last_clean":"2023-11-06T02:08:32.149601+0000","last_became_active":"2023-11-06T02:08:30.656319+0000","last_became_peered":"2023-11-06T02:08:30.656319+0000","last_unstale":"2023-11-06T02:08:32.149601+0000","last_undegraded":"2023-11-06T02:08:32.149601+0000","last_fullsized":"2023-11-06T02:08:32.149601+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:57:32.079075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.652785+0000","last_change":"2023-11-06T02:08:30.627856+0000","last_active":"2023-11-06T02:08:32.652785+0000","last_peered":"2023-11-06T02:08:32.652785+0000","last_clean":"2023-11-06T02:08:32.652785+0000","last_became_active":"2023-11-06T02:08:30.627536+0000","last_became_peered":"2023-11-06T02:08:30.627536+0000","last_unstale":"2023-11-06T02:08:32.652785+0000","last_undegraded":"2023-11-06T02:08:32.652785+0000","last_fullsized":"2023-11-06T02:08:32.652785+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:53:11.188799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.652773+0000","last_change":"2023-11-06T02:08:30.613522+0000","last_active":"2023-11-06T02:08:32.652773+0000","last_peered":"2023-11-06T02:08:32.652773+0000","last_clean":"2023-11-06T02:08:32.652773+0000","last_became_active":"2023-11-06T02:08:30.612950+0000","last_became_peered":"2023-11-06T02:08:30.612950+0000","last_unstale":"2023-11-06T02:08:32.652773+0000","last_undegraded":"2023-11-06T02:08:32.652773+0000","last_fullsized":"2023-11-06T02:08:32.652773+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:26:58.090934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.413094+0000","last_change":"2023-11-06T02:08:30.653593+0000","last_active":"2023-11-06T02:08:32.413094+0000","last_peered":"2023-11-06T02:08:32.413094+0000","last_clean":"2023-11-06T02:08:32.413094+0000","last_became_active":"2023-11-06T02:08:30.652925+0000","last_became_peered":"2023-11-06T02:08:30.652925+0000","last_unstale":"2023-11-06T02:08:32.413094+0000","last_undegraded":"2023-11-06T02:08:32.413094+0000","last_fullsized":"2023-11-06T02:08:32.413094+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:17:50.007398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.652200+0000","last_change":"2023-11-06T02:08:30.645260+0000","last_active":"2023-11-06T02:08:32.652200+0000","last_peered":"2023-11-06T02:08:32.652200+0000","last_clean":"2023-11-06T02:08:32.652200+0000","last_became_active":"2023-11-06T02:08:30.644949+0000","last_became_peered":"2023-11-06T02:08:30.644949+0000","last_unstale":"2023-11-06T02:08:32.652200+0000","last_undegraded":"2023-11-06T02:08:32.652200+0000","last_fullsized":"2023-11-06T02:08:32.652200+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:41:00.973671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.604763+0000","last_change":"2023-11-06T02:08:30.649814+0000","last_active":"2023-11-06T02:08:31.604763+0000","last_peered":"2023-11-06T02:08:31.604763+0000","last_clean":"2023-11-06T02:08:31.604763+0000","last_became_active":"2023-11-06T02:08:30.649492+0000","last_became_peered":"2023-11-06T02:08:30.649492+0000","last_unstale":"2023-11-06T02:08:31.604763+0000","last_undegraded":"2023-11-06T02:08:31.604763+0000","last_fullsized":"2023-11-06T02:08:31.604763+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:26:58.090934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.609372+0000","last_change":"2023-11-06T02:08:30.632560+0000","last_active":"2023-11-06T02:08:31.609372+0000","last_peered":"2023-11-06T02:08:31.609372+0000","last_clean":"2023-11-06T02:08:31.609372+0000","last_became_active":"2023-11-06T02:08:30.631819+0000","last_became_peered":"2023-11-06T02:08:30.631819+0000","last_unstale":"2023-11-06T02:08:31.609372+0000","last_undegraded":"2023-11-06T02:08:31.609372+0000","last_fullsized":"2023-11-06T02:08:31.609372+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:43:23.215718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.604562+0000","last_change":"2023-11-06T02:08:30.624052+0000","last_active":"2023-11-06T02:08:31.604562+0000","last_peered":"2023-11-06T02:08:31.604562+0000","last_clean":"2023-11-06T02:08:31.604562+0000","last_became_active":"2023-11-06T02:08:30.623658+0000","last_became_peered":"2023-11-06T02:08:30.623658+0000","last_unstale":"2023-11-06T02:08:31.604562+0000","last_undegraded":"2023-11-06T02:08:31.604562+0000","last_fullsized":"2023-11-06T02:08:31.604562+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:17:50.007398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.609631+0000","last_change":"2023-11-06T02:08:30.614066+0000","last_active":"2023-11-06T02:08:31.609631+0000","last_peered":"2023-11-06T02:08:31.609631+0000","last_clean":"2023-11-06T02:08:31.609631+0000","last_became_active":"2023-11-06T02:08:30.613337+0000","last_became_peered":"2023-11-06T02:08:30.613337+0000","last_unstale":"2023-11-06T02:08:31.609631+0000","last_undegraded":"2023-11-06T02:08:31.609631+0000","last_fullsized":"2023-11-06T02:08:31.609631+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:33:39.359552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.604963+0000","last_change":"2023-11-06T02:08:30.621368+0000","last_active":"2023-11-06T02:08:31.604963+0000","last_peered":"2023-11-06T02:08:31.604963+0000","last_clean":"2023-11-06T02:08:31.604963+0000","last_became_active":"2023-11-06T02:08:30.620563+0000","last_became_peered":"2023-11-06T02:08:30.620563+0000","last_unstale":"2023-11-06T02:08:31.604963+0000","last_undegraded":"2023-11-06T02:08:31.604963+0000","last_fullsized":"2023-11-06T02:08:31.604963+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:56:51.565719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"22'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.149865+0000","last_change":"2023-11-06T02:08:30.229825+0000","last_active":"2023-11-06T02:08:32.149865+0000","last_peered":"2023-11-06T02:08:32.149865+0000","last_clean":"2023-11-06T02:08:32.149865+0000","last_became_active":"2023-11-06T02:08:30.229392+0000","last_became_peered":"2023-11-06T02:08:30.229392+0000","last_unstale":"2023-11-06T02:08:32.149865+0000","last_undegraded":"2023-11-06T02:08:32.149865+0000","last_fullsized":"2023-11-06T02:08:32.149865+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:33:38.355715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.413299+0000","last_change":"2023-11-06T02:08:30.646474+0000","last_active":"2023-11-06T02:08:32.413299+0000","last_peered":"2023-11-06T02:08:32.413299+0000","last_clean":"2023-11-06T02:08:32.413299+0000","last_became_active":"2023-11-06T02:08:30.645992+0000","last_became_peered":"2023-11-06T02:08:30.645992+0000","last_unstale":"2023-11-06T02:08:32.413299+0000","last_undegraded":"2023-11-06T02:08:32.413299+0000","last_fullsized":"2023-11-06T02:08:32.413299+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:47:30.859032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.651766+0000","last_change":"2023-11-06T02:08:30.225818+0000","last_active":"2023-11-06T02:08:32.651766+0000","last_peered":"2023-11-06T02:08:32.651766+0000","last_clean":"2023-11-06T02:08:32.651766+0000","last_became_active":"2023-11-06T02:08:30.225359+0000","last_became_peered":"2023-11-06T02:08:30.225359+0000","last_unstale":"2023-11-06T02:08:32.651766+0000","last_undegraded":"2023-11-06T02:08:32.651766+0000","last_fullsized":"2023-11-06T02:08:32.651766+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:52:19.592716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.653330+0000","last_change":"2023-11-06T02:08:30.618020+0000","last_active":"2023-11-06T02:08:32.653330+0000","last_peered":"2023-11-06T02:08:32.653330+0000","last_clean":"2023-11-06T02:08:32.653330+0000","last_became_active":"2023-11-06T02:08:30.617684+0000","last_became_peered":"2023-11-06T02:08:30.617684+0000","last_unstale":"2023-11-06T02:08:32.653330+0000","last_undegraded":"2023-11-06T02:08:32.653330+0000","last_fullsized":"2023-11-06T02:08:32.653330+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:56:51.565719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.149630+0000","last_change":"2023-11-06T02:08:30.230443+0000","last_active":"2023-11-06T02:08:32.149630+0000","last_peered":"2023-11-06T02:08:32.149630+0000","last_clean":"2023-11-06T02:08:32.149630+0000","last_became_active":"2023-11-06T02:08:30.229980+0000","last_became_peered":"2023-11-06T02:08:30.229980+0000","last_unstale":"2023-11-06T02:08:32.149630+0000","last_undegraded":"2023-11-06T02:08:32.149630+0000","last_fullsized":"2023-11-06T02:08:32.149630+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:46:37.864109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.605377+0000","last_change":"2023-11-06T02:08:30.647210+0000","last_active":"2023-11-06T02:08:31.605377+0000","last_peered":"2023-11-06T02:08:31.605377+0000","last_clean":"2023-11-06T02:08:31.605377+0000","last_became_active":"2023-11-06T02:08:30.646862+0000","last_became_peered":"2023-11-06T02:08:30.646862+0000","last_unstale":"2023-11-06T02:08:31.605377+0000","last_undegraded":"2023-11-06T02:08:31.605377+0000","last_fullsized":"2023-11-06T02:08:31.605377+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:43:23.215718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"22'8","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.609883+0000","last_change":"2023-11-06T02:08:30.218861+0000","last_active":"2023-11-06T02:08:31.609883+0000","last_peered":"2023-11-06T02:08:31.609883+0000","last_clean":"2023-11-06T02:08:31.609883+0000","last_became_active":"2023-11-06T02:08:30.218245+0000","last_became_peered":"2023-11-06T02:08:30.218245+0000","last_unstale":"2023-11-06T02:08:31.609883+0000","last_undegraded":"2023-11-06T02:08:31.609883+0000","last_fullsized":"2023-11-06T02:08:31.609883+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:04:42.682132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"22'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.150304+0000","last_change":"2023-11-06T02:08:30.229578+0000","last_active":"2023-11-06T02:08:32.150304+0000","last_peered":"2023-11-06T02:08:32.150304+0000","last_clean":"2023-11-06T02:08:32.150304+0000","last_became_active":"2023-11-06T02:08:30.229131+0000","last_became_peered":"2023-11-06T02:08:30.229131+0000","last_unstale":"2023-11-06T02:08:32.150304+0000","last_undegraded":"2023-11-06T02:08:32.150304+0000","last_fullsized":"2023-11-06T02:08:32.150304+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:30:49.856478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.412653+0000","last_change":"2023-11-06T02:08:30.647714+0000","last_active":"2023-11-06T02:08:32.412653+0000","last_peered":"2023-11-06T02:08:32.412653+0000","last_clean":"2023-11-06T02:08:32.412653+0000","last_became_active":"2023-11-06T02:08:30.647212+0000","last_became_peered":"2023-11-06T02:08:30.647212+0000","last_unstale":"2023-11-06T02:08:32.412653+0000","last_undegraded":"2023-11-06T02:08:32.412653+0000","last_fullsized":"2023-11-06T02:08:32.412653+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:13:37.616527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"22'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.150130+0000","last_change":"2023-11-06T02:08:30.230469+0000","last_active":"2023-11-06T02:08:32.150130+0000","last_peered":"2023-11-06T02:08:32.150130+0000","last_clean":"2023-11-06T02:08:32.150130+0000","last_became_active":"2023-11-06T02:08:30.230064+0000","last_became_peered":"2023-11-06T02:08:30.230064+0000","last_unstale":"2023-11-06T02:08:32.150130+0000","last_undegraded":"2023-11-06T02:08:32.150130+0000","last_fullsized":"2023-11-06T02:08:32.150130+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:19:59.347221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.608834+0000","last_change":"2023-11-06T02:08:30.632754+0000","last_active":"2023-11-06T02:08:31.608834+0000","last_peered":"2023-11-06T02:08:31.608834+0000","last_clean":"2023-11-06T02:08:31.608834+0000","last_became_active":"2023-11-06T02:08:30.631976+0000","last_became_peered":"2023-11-06T02:08:30.631976+0000","last_unstale":"2023-11-06T02:08:31.608834+0000","last_undegraded":"2023-11-06T02:08:31.608834+0000","last_fullsized":"2023-11-06T02:08:31.608834+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:56:51.565719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"22'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.149972+0000","last_change":"2023-11-06T02:08:30.231003+0000","last_active":"2023-11-06T02:08:32.149972+0000","last_peered":"2023-11-06T02:08:32.149972+0000","last_clean":"2023-11-06T02:08:32.149972+0000","last_became_active":"2023-11-06T02:08:30.229978+0000","last_became_peered":"2023-11-06T02:08:30.229978+0000","last_unstale":"2023-11-06T02:08:32.149972+0000","last_undegraded":"2023-11-06T02:08:32.149972+0000","last_fullsized":"2023-11-06T02:08:32.149972+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:24:09.801921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.609985+0000","last_change":"2023-11-06T02:08:30.619807+0000","last_active":"2023-11-06T02:08:31.609985+0000","last_peered":"2023-11-06T02:08:31.609985+0000","last_clean":"2023-11-06T02:08:31.609985+0000","last_became_active":"2023-11-06T02:08:30.619048+0000","last_became_peered":"2023-11-06T02:08:30.619048+0000","last_unstale":"2023-11-06T02:08:31.609985+0000","last_undegraded":"2023-11-06T02:08:31.609985+0000","last_fullsized":"2023-11-06T02:08:31.609985+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:15:59.045303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.412965+0000","last_change":"2023-11-06T02:08:30.218715+0000","last_active":"2023-11-06T02:08:32.412965+0000","last_peered":"2023-11-06T02:08:32.412965+0000","last_clean":"2023-11-06T02:08:32.412965+0000","last_became_active":"2023-11-06T02:08:30.217966+0000","last_became_peered":"2023-11-06T02:08:30.217966+0000","last_unstale":"2023-11-06T02:08:32.412965+0000","last_undegraded":"2023-11-06T02:08:32.412965+0000","last_fullsized":"2023-11-06T02:08:32.412965+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:30:21.274364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.652004+0000","last_change":"2023-11-06T02:08:30.620567+0000","last_active":"2023-11-06T02:08:32.652004+0000","last_peered":"2023-11-06T02:08:32.652004+0000","last_clean":"2023-11-06T02:08:32.652004+0000","last_became_active":"2023-11-06T02:08:30.619671+0000","last_became_peered":"2023-11-06T02:08:30.619671+0000","last_unstale":"2023-11-06T02:08:32.652004+0000","last_undegraded":"2023-11-06T02:08:32.652004+0000","last_fullsized":"2023-11-06T02:08:32.652004+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:57:32.079075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"22'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.605088+0000","last_change":"2023-11-06T02:08:30.220060+0000","last_active":"2023-11-06T02:08:31.605088+0000","last_peered":"2023-11-06T02:08:31.605088+0000","last_clean":"2023-11-06T02:08:31.605088+0000","last_became_active":"2023-11-06T02:08:30.218534+0000","last_became_peered":"2023-11-06T02:08:30.218534+0000","last_unstale":"2023-11-06T02:08:31.605088+0000","last_undegraded":"2023-11-06T02:08:31.605088+0000","last_fullsized":"2023-11-06T02:08:31.605088+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:30:21.274364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.609657+0000","last_change":"2023-11-06T02:08:30.658173+0000","last_active":"2023-11-06T02:08:31.609657+0000","last_peered":"2023-11-06T02:08:31.609657+0000","last_clean":"2023-11-06T02:08:31.609657+0000","last_became_active":"2023-11-06T02:08:30.656890+0000","last_became_peered":"2023-11-06T02:08:30.656890+0000","last_unstale":"2023-11-06T02:08:31.609657+0000","last_undegraded":"2023-11-06T02:08:31.609657+0000","last_fullsized":"2023-11-06T02:08:31.609657+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:23:59.639528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.610321+0000","last_change":"2023-11-06T02:08:30.222023+0000","last_active":"2023-11-06T02:08:31.610321+0000","last_peered":"2023-11-06T02:08:31.610321+0000","last_clean":"2023-11-06T02:08:31.610321+0000","last_became_active":"2023-11-06T02:08:30.220788+0000","last_became_peered":"2023-11-06T02:08:30.220788+0000","last_unstale":"2023-11-06T02:08:31.610321+0000","last_undegraded":"2023-11-06T02:08:31.610321+0000","last_fullsized":"2023-11-06T02:08:31.610321+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:31:31.882709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.610620+0000","last_change":"2023-11-06T02:08:30.635685+0000","last_active":"2023-11-06T02:08:31.610620+0000","last_peered":"2023-11-06T02:08:31.610620+0000","last_clean":"2023-11-06T02:08:31.610620+0000","last_became_active":"2023-11-06T02:08:30.634801+0000","last_became_peered":"2023-11-06T02:08:30.634801+0000","last_unstale":"2023-11-06T02:08:31.610620+0000","last_undegraded":"2023-11-06T02:08:31.610620+0000","last_fullsized":"2023-11-06T02:08:31.610620+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:37:47.953218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.609115+0000","last_change":"2023-11-06T02:08:30.222592+0000","last_active":"2023-11-06T02:08:31.609115+0000","last_peered":"2023-11-06T02:08:31.609115+0000","last_clean":"2023-11-06T02:08:31.609115+0000","last_became_active":"2023-11-06T02:08:30.222184+0000","last_became_peered":"2023-11-06T02:08:30.222184+0000","last_unstale":"2023-11-06T02:08:31.609115+0000","last_undegraded":"2023-11-06T02:08:31.609115+0000","last_fullsized":"2023-11-06T02:08:31.609115+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:31:31.882709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.609663+0000","last_change":"2023-11-06T02:08:30.651763+0000","last_active":"2023-11-06T02:08:31.609663+0000","last_peered":"2023-11-06T02:08:31.609663+0000","last_clean":"2023-11-06T02:08:31.609663+0000","last_became_active":"2023-11-06T02:08:30.651013+0000","last_became_peered":"2023-11-06T02:08:30.651013+0000","last_unstale":"2023-11-06T02:08:31.609663+0000","last_undegraded":"2023-11-06T02:08:31.609663+0000","last_fullsized":"2023-11-06T02:08:31.609663+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:41:51.822994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.604575+0000","last_change":"2023-11-06T02:08:30.225279+0000","last_active":"2023-11-06T02:08:31.604575+0000","last_peered":"2023-11-06T02:08:31.604575+0000","last_clean":"2023-11-06T02:08:31.604575+0000","last_became_active":"2023-11-06T02:08:30.224936+0000","last_became_peered":"2023-11-06T02:08:30.224936+0000","last_unstale":"2023-11-06T02:08:31.604575+0000","last_undegraded":"2023-11-06T02:08:31.604575+0000","last_fullsized":"2023-11-06T02:08:31.604575+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:52:19.592716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.653044+0000","last_change":"2023-11-06T02:08:30.619652+0000","last_active":"2023-11-06T02:08:32.653044+0000","last_peered":"2023-11-06T02:08:32.653044+0000","last_clean":"2023-11-06T02:08:32.653044+0000","last_became_active":"2023-11-06T02:08:30.619070+0000","last_became_peered":"2023-11-06T02:08:30.619070+0000","last_unstale":"2023-11-06T02:08:32.653044+0000","last_undegraded":"2023-11-06T02:08:32.653044+0000","last_fullsized":"2023-11-06T02:08:32.653044+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:56:51.565719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":167,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-06T02:08:30.628264+0000","last_change":"2023-11-06T02:08:20.523427+0000","last_active":"2023-11-06T02:08:30.628264+0000","last_peered":"2023-11-06T02:08:30.628264+0000","last_clean":"2023-11-06T02:08:30.628264+0000","last_became_active":"2023-11-06T02:08:20.521089+0000","last_became_peered":"2023-11-06T02:08:20.521089+0000","last_unstale":"2023-11-06T02:08:30.628264+0000","last_undegraded":"2023-11-06T02:08:30.628264+0000","last_fullsized":"2023-11-06T02:08:30.628264+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:19.252348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:19.252348+0000","last_clean_scrub_stamp":"2023-11-06T02:08:19.252348+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:30:15.566397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-06T02:08:30.640657+0000","last_change":"2023-11-06T02:08:26.059502+0000","last_active":"2023-11-06T02:08:30.640657+0000","last_peered":"2023-11-06T02:08:30.640657+0000","last_clean":"2023-11-06T02:08:30.640657+0000","last_became_active":"2023-11-06T02:08:26.058612+0000","last_became_peered":"2023-11-06T02:08:26.058612+0000","last_unstale":"2023-11-06T02:08:30.640657+0000","last_undegraded":"2023-11-06T02:08:30.640657+0000","last_fullsized":"2023-11-06T02:08:30.640657+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:10.626021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.604588+0000","last_change":"2023-11-06T02:08:30.219125+0000","last_active":"2023-11-06T02:08:31.604588+0000","last_peered":"2023-11-06T02:08:31.604588+0000","last_clean":"2023-11-06T02:08:31.604588+0000","last_became_active":"2023-11-06T02:08:30.217091+0000","last_became_peered":"2023-11-06T02:08:30.217091+0000","last_unstale":"2023-11-06T02:08:31.604588+0000","last_undegraded":"2023-11-06T02:08:31.604588+0000","last_fullsized":"2023-11-06T02:08:31.604588+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:04:42.682132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.610770+0000","last_change":"2023-11-06T02:08:30.660650+0000","last_active":"2023-11-06T02:08:31.610770+0000","last_peered":"2023-11-06T02:08:31.610770+0000","last_clean":"2023-11-06T02:08:31.610770+0000","last_became_active":"2023-11-06T02:08:30.660072+0000","last_became_peered":"2023-11-06T02:08:30.660072+0000","last_unstale":"2023-11-06T02:08:31.610770+0000","last_undegraded":"2023-11-06T02:08:31.610770+0000","last_fullsized":"2023-11-06T02:08:31.610770+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:30:50.860315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-06T02:08:30.626279+0000","last_change":"2023-11-06T02:08:26.060406+0000","last_active":"2023-11-06T02:08:30.626279+0000","last_peered":"2023-11-06T02:08:30.626279+0000","last_clean":"2023-11-06T02:08:30.626279+0000","last_became_active":"2023-11-06T02:08:26.059262+0000","last_became_peered":"2023-11-06T02:08:26.059262+0000","last_unstale":"2023-11-06T02:08:30.626279+0000","last_undegraded":"2023-11-06T02:08:30.626279+0000","last_fullsized":"2023-11-06T02:08:30.626279+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:10.626021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.651421+0000","last_change":"2023-11-06T02:08:30.227671+0000","last_active":"2023-11-06T02:08:32.651421+0000","last_peered":"2023-11-06T02:08:32.651421+0000","last_clean":"2023-11-06T02:08:32.651421+0000","last_became_active":"2023-11-06T02:08:30.227200+0000","last_became_peered":"2023-11-06T02:08:30.227200+0000","last_unstale":"2023-11-06T02:08:32.651421+0000","last_undegraded":"2023-11-06T02:08:32.651421+0000","last_fullsized":"2023-11-06T02:08:32.651421+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:26:49.298561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.653133+0000","last_change":"2023-11-06T02:08:30.619424+0000","last_active":"2023-11-06T02:08:32.653133+0000","last_peered":"2023-11-06T02:08:32.653133+0000","last_clean":"2023-11-06T02:08:32.653133+0000","last_became_active":"2023-11-06T02:08:30.619150+0000","last_became_peered":"2023-11-06T02:08:30.619150+0000","last_unstale":"2023-11-06T02:08:32.653133+0000","last_undegraded":"2023-11-06T02:08:32.653133+0000","last_fullsized":"2023-11-06T02:08:32.653133+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:47:30.859032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-06T02:08:30.597659+0000","last_change":"2023-11-06T02:08:26.559477+0000","last_active":"2023-11-06T02:08:30.597659+0000","last_peered":"2023-11-06T02:08:30.597659+0000","last_clean":"2023-11-06T02:08:30.597659+0000","last_became_active":"2023-11-06T02:08:26.559202+0000","last_became_peered":"2023-11-06T02:08:26.559202+0000","last_unstale":"2023-11-06T02:08:30.597659+0000","last_undegraded":"2023-11-06T02:08:30.597659+0000","last_fullsized":"2023-11-06T02:08:30.597659+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:55:04.552404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.651066+0000","last_change":"2023-11-06T02:08:30.219422+0000","last_active":"2023-11-06T02:08:32.651066+0000","last_peered":"2023-11-06T02:08:32.651066+0000","last_clean":"2023-11-06T02:08:32.651066+0000","last_became_active":"2023-11-06T02:08:30.219148+0000","last_became_peered":"2023-11-06T02:08:30.219148+0000","last_unstale":"2023-11-06T02:08:32.651066+0000","last_undegraded":"2023-11-06T02:08:32.651066+0000","last_fullsized":"2023-11-06T02:08:32.651066+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:52:19.592716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.653298+0000","last_change":"2023-11-06T02:08:30.655629+0000","last_active":"2023-11-06T02:08:32.653298+0000","last_peered":"2023-11-06T02:08:32.653298+0000","last_clean":"2023-11-06T02:08:32.653298+0000","last_became_active":"2023-11-06T02:08:30.655271+0000","last_became_peered":"2023-11-06T02:08:30.655271+0000","last_unstale":"2023-11-06T02:08:32.653298+0000","last_undegraded":"2023-11-06T02:08:32.653298+0000","last_fullsized":"2023-11-06T02:08:32.653298+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:10:14.974545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-06T02:08:30.641603+0000","last_change":"2023-11-06T02:08:26.059546+0000","last_active":"2023-11-06T02:08:30.641603+0000","last_peered":"2023-11-06T02:08:30.641603+0000","last_clean":"2023-11-06T02:08:30.641603+0000","last_became_active":"2023-11-06T02:08:26.058806+0000","last_became_peered":"2023-11-06T02:08:26.058806+0000","last_unstale":"2023-11-06T02:08:30.641603+0000","last_undegraded":"2023-11-06T02:08:30.641603+0000","last_fullsized":"2023-11-06T02:08:30.641603+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:23.468510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.609639+0000","last_change":"2023-11-06T02:08:30.223844+0000","last_active":"2023-11-06T02:08:31.609639+0000","last_peered":"2023-11-06T02:08:31.609639+0000","last_clean":"2023-11-06T02:08:31.609639+0000","last_became_active":"2023-11-06T02:08:30.222902+0000","last_became_peered":"2023-11-06T02:08:30.222902+0000","last_unstale":"2023-11-06T02:08:31.609639+0000","last_undegraded":"2023-11-06T02:08:31.609639+0000","last_fullsized":"2023-11-06T02:08:31.609639+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:52:19.592716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.651748+0000","last_change":"2023-11-06T02:08:30.619125+0000","last_active":"2023-11-06T02:08:32.651748+0000","last_peered":"2023-11-06T02:08:32.651748+0000","last_clean":"2023-11-06T02:08:32.651748+0000","last_became_active":"2023-11-06T02:08:30.617621+0000","last_became_peered":"2023-11-06T02:08:30.617621+0000","last_unstale":"2023-11-06T02:08:32.651748+0000","last_undegraded":"2023-11-06T02:08:32.651748+0000","last_fullsized":"2023-11-06T02:08:32.651748+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:13:37.616527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.413429+0000","last_change":"2023-11-06T02:08:30.225274+0000","last_active":"2023-11-06T02:08:32.413429+0000","last_peered":"2023-11-06T02:08:32.413429+0000","last_clean":"2023-11-06T02:08:32.413429+0000","last_became_active":"2023-11-06T02:08:30.224275+0000","last_became_peered":"2023-11-06T02:08:30.224275+0000","last_unstale":"2023-11-06T02:08:32.413429+0000","last_undegraded":"2023-11-06T02:08:32.413429+0000","last_fullsized":"2023-11-06T02:08:32.413429+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:26:49.298561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-06T02:08:30.614515+0000","last_change":"2023-11-06T02:08:26.436463+0000","last_active":"2023-11-06T02:08:30.614515+0000","last_peered":"2023-11-06T02:08:30.614515+0000","last_clean":"2023-11-06T02:08:30.614515+0000","last_became_active":"2023-11-06T02:08:26.436075+0000","last_became_peered":"2023-11-06T02:08:26.436075+0000","last_unstale":"2023-11-06T02:08:30.614515+0000","last_undegraded":"2023-11-06T02:08:30.614515+0000","last_fullsized":"2023-11-06T02:08:30.614515+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:23.468510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.651810+0000","last_change":"2023-11-06T02:08:30.655634+0000","last_active":"2023-11-06T02:08:32.651810+0000","last_peered":"2023-11-06T02:08:32.651810+0000","last_clean":"2023-11-06T02:08:32.651810+0000","last_became_active":"2023-11-06T02:08:30.655285+0000","last_became_peered":"2023-11-06T02:08:30.655285+0000","last_unstale":"2023-11-06T02:08:32.651810+0000","last_undegraded":"2023-11-06T02:08:32.651810+0000","last_fullsized":"2023-11-06T02:08:32.651810+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:49:05.460965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.651964+0000","last_change":"2023-11-06T02:08:30.220327+0000","last_active":"2023-11-06T02:08:32.651964+0000","last_peered":"2023-11-06T02:08:32.651964+0000","last_clean":"2023-11-06T02:08:32.651964+0000","last_became_active":"2023-11-06T02:08:30.219906+0000","last_became_peered":"2023-11-06T02:08:30.219906+0000","last_unstale":"2023-11-06T02:08:32.651964+0000","last_undegraded":"2023-11-06T02:08:32.651964+0000","last_fullsized":"2023-11-06T02:08:32.651964+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:43:56.017919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-06T02:08:30.636949+0000","last_change":"2023-11-06T02:08:26.436607+0000","last_active":"2023-11-06T02:08:30.636949+0000","last_peered":"2023-11-06T02:08:30.636949+0000","last_clean":"2023-11-06T02:08:30.636949+0000","last_became_active":"2023-11-06T02:08:26.435924+0000","last_became_peered":"2023-11-06T02:08:26.435924+0000","last_unstale":"2023-11-06T02:08:30.636949+0000","last_undegraded":"2023-11-06T02:08:30.636949+0000","last_fullsized":"2023-11-06T02:08:30.636949+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:23.468510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.652082+0000","last_change":"2023-11-06T02:08:30.654058+0000","last_active":"2023-11-06T02:08:32.652082+0000","last_peered":"2023-11-06T02:08:32.652082+0000","last_clean":"2023-11-06T02:08:32.652082+0000","last_became_active":"2023-11-06T02:08:30.653753+0000","last_became_peered":"2023-11-06T02:08:30.653753+0000","last_unstale":"2023-11-06T02:08:32.652082+0000","last_undegraded":"2023-11-06T02:08:32.652082+0000","last_fullsized":"2023-11-06T02:08:32.652082+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:13:37.616527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"22'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.605569+0000","last_change":"2023-11-06T02:08:30.231205+0000","last_active":"2023-11-06T02:08:31.605569+0000","last_peered":"2023-11-06T02:08:31.605569+0000","last_clean":"2023-11-06T02:08:31.605569+0000","last_became_active":"2023-11-06T02:08:30.230854+0000","last_became_peered":"2023-11-06T02:08:30.230854+0000","last_unstale":"2023-11-06T02:08:31.605569+0000","last_undegraded":"2023-11-06T02:08:31.605569+0000","last_fullsized":"2023-11-06T02:08:31.605569+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:26:49.298561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-06T02:08:30.614693+0000","last_change":"2023-11-06T02:08:26.057076+0000","last_active":"2023-11-06T02:08:30.614693+0000","last_peered":"2023-11-06T02:08:30.614693+0000","last_clean":"2023-11-06T02:08:30.614693+0000","last_became_active":"2023-11-06T02:08:26.056419+0000","last_became_peered":"2023-11-06T02:08:26.056419+0000","last_unstale":"2023-11-06T02:08:30.614693+0000","last_undegraded":"2023-11-06T02:08:30.614693+0000","last_fullsized":"2023-11-06T02:08:30.614693+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:10.626021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.604395+0000","last_change":"2023-11-06T02:08:30.647144+0000","last_active":"2023-11-06T02:08:31.604395+0000","last_peered":"2023-11-06T02:08:31.604395+0000","last_clean":"2023-11-06T02:08:31.604395+0000","last_became_active":"2023-11-06T02:08:30.646802+0000","last_became_peered":"2023-11-06T02:08:30.646802+0000","last_unstale":"2023-11-06T02:08:31.604395+0000","last_undegraded":"2023-11-06T02:08:31.604395+0000","last_fullsized":"2023-11-06T02:08:31.604395+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:49:05.460965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"22'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.653477+0000","last_change":"2023-11-06T02:08:30.225086+0000","last_active":"2023-11-06T02:08:32.653477+0000","last_peered":"2023-11-06T02:08:32.653477+0000","last_clean":"2023-11-06T02:08:32.653477+0000","last_became_active":"2023-11-06T02:08:30.224455+0000","last_became_peered":"2023-11-06T02:08:30.224455+0000","last_unstale":"2023-11-06T02:08:32.653477+0000","last_undegraded":"2023-11-06T02:08:32.653477+0000","last_fullsized":"2023-11-06T02:08:32.653477+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:26:21.361289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-06T02:08:30.606381+0000","last_change":"2023-11-06T02:08:26.058596+0000","last_active":"2023-11-06T02:08:30.606381+0000","last_peered":"2023-11-06T02:08:30.606381+0000","last_clean":"2023-11-06T02:08:30.606381+0000","last_became_active":"2023-11-06T02:08:26.057015+0000","last_became_peered":"2023-11-06T02:08:26.057015+0000","last_unstale":"2023-11-06T02:08:30.606381+0000","last_undegraded":"2023-11-06T02:08:30.606381+0000","last_fullsized":"2023-11-06T02:08:30.606381+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:55:04.552404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.610172+0000","last_change":"2023-11-06T02:08:30.656051+0000","last_active":"2023-11-06T02:08:31.610172+0000","last_peered":"2023-11-06T02:08:31.610172+0000","last_clean":"2023-11-06T02:08:31.610172+0000","last_became_active":"2023-11-06T02:08:30.650054+0000","last_became_peered":"2023-11-06T02:08:30.650054+0000","last_unstale":"2023-11-06T02:08:31.610172+0000","last_undegraded":"2023-11-06T02:08:31.610172+0000","last_fullsized":"2023-11-06T02:08:31.610172+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:17:50.007398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.609417+0000","last_change":"2023-11-06T02:08:30.633028+0000","last_active":"2023-11-06T02:08:31.609417+0000","last_peered":"2023-11-06T02:08:31.609417+0000","last_clean":"2023-11-06T02:08:31.609417+0000","last_became_active":"2023-11-06T02:08:30.632255+0000","last_became_peered":"2023-11-06T02:08:30.632255+0000","last_unstale":"2023-11-06T02:08:31.609417+0000","last_undegraded":"2023-11-06T02:08:31.609417+0000","last_fullsized":"2023-11-06T02:08:31.609417+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:47:30.859032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"22'6","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.609754+0000","last_change":"2023-11-06T02:08:30.230431+0000","last_active":"2023-11-06T02:08:31.609754+0000","last_peered":"2023-11-06T02:08:31.609754+0000","last_clean":"2023-11-06T02:08:31.609754+0000","last_became_active":"2023-11-06T02:08:30.229931+0000","last_became_peered":"2023-11-06T02:08:30.229931+0000","last_unstale":"2023-11-06T02:08:31.609754+0000","last_undegraded":"2023-11-06T02:08:31.609754+0000","last_fullsized":"2023-11-06T02:08:31.609754+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:26:21.361289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.651402+0000","last_change":"2023-11-06T02:08:30.628444+0000","last_active":"2023-11-06T02:08:32.651402+0000","last_peered":"2023-11-06T02:08:32.651402+0000","last_clean":"2023-11-06T02:08:32.651402+0000","last_became_active":"2023-11-06T02:08:30.628087+0000","last_became_peered":"2023-11-06T02:08:30.628087+0000","last_unstale":"2023-11-06T02:08:32.651402+0000","last_undegraded":"2023-11-06T02:08:32.651402+0000","last_fullsized":"2023-11-06T02:08:32.651402+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:57:32.079075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"22'3","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.149892+0000","last_change":"2023-11-06T02:08:30.230554+0000","last_active":"2023-11-06T02:08:32.149892+0000","last_peered":"2023-11-06T02:08:32.149892+0000","last_clean":"2023-11-06T02:08:32.149892+0000","last_became_active":"2023-11-06T02:08:30.229374+0000","last_became_peered":"2023-11-06T02:08:30.229374+0000","last_unstale":"2023-11-06T02:08:32.149892+0000","last_undegraded":"2023-11-06T02:08:32.149892+0000","last_fullsized":"2023-11-06T02:08:32.149892+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:15:58.041467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.413338+0000","last_change":"2023-11-06T02:08:30.231726+0000","last_active":"2023-11-06T02:08:32.413338+0000","last_peered":"2023-11-06T02:08:32.413338+0000","last_clean":"2023-11-06T02:08:32.413338+0000","last_became_active":"2023-11-06T02:08:30.230802+0000","last_became_peered":"2023-11-06T02:08:30.230802+0000","last_unstale":"2023-11-06T02:08:32.413338+0000","last_undegraded":"2023-11-06T02:08:32.413338+0000","last_fullsized":"2023-11-06T02:08:32.413338+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:15:58.041467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.604488+0000","last_change":"2023-11-06T02:08:30.644420+0000","last_active":"2023-11-06T02:08:31.604488+0000","last_peered":"2023-11-06T02:08:31.604488+0000","last_clean":"2023-11-06T02:08:31.604488+0000","last_became_active":"2023-11-06T02:08:30.644097+0000","last_became_peered":"2023-11-06T02:08:30.644097+0000","last_unstale":"2023-11-06T02:08:31.604488+0000","last_undegraded":"2023-11-06T02:08:31.604488+0000","last_fullsized":"2023-11-06T02:08:31.604488+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:31:28.301664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.651911+0000","last_change":"2023-11-06T02:08:30.218801+0000","last_active":"2023-11-06T02:08:32.651911+0000","last_peered":"2023-11-06T02:08:32.651911+0000","last_clean":"2023-11-06T02:08:32.651911+0000","last_became_active":"2023-11-06T02:08:30.217237+0000","last_became_peered":"2023-11-06T02:08:30.217237+0000","last_unstale":"2023-11-06T02:08:32.651911+0000","last_undegraded":"2023-11-06T02:08:32.651911+0000","last_fullsized":"2023-11-06T02:08:32.651911+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:40:04.297054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.604444+0000","last_change":"2023-11-06T02:08:30.619501+0000","last_active":"2023-11-06T02:08:31.604444+0000","last_peered":"2023-11-06T02:08:31.604444+0000","last_clean":"2023-11-06T02:08:31.604444+0000","last_became_active":"2023-11-06T02:08:30.619180+0000","last_became_peered":"2023-11-06T02:08:30.619180+0000","last_unstale":"2023-11-06T02:08:31.604444+0000","last_undegraded":"2023-11-06T02:08:31.604444+0000","last_fullsized":"2023-11-06T02:08:31.604444+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:13:37.616527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.611492+0000","last_change":"2023-11-06T02:08:30.655759+0000","last_active":"2023-11-06T02:08:31.611492+0000","last_peered":"2023-11-06T02:08:31.611492+0000","last_clean":"2023-11-06T02:08:31.611492+0000","last_became_active":"2023-11-06T02:08:30.655285+0000","last_became_peered":"2023-11-06T02:08:30.655285+0000","last_unstale":"2023-11-06T02:08:31.611492+0000","last_undegraded":"2023-11-06T02:08:31.611492+0000","last_fullsized":"2023-11-06T02:08:31.611492+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:20:00.351057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.604479+0000","last_change":"2023-11-06T02:08:30.222521+0000","last_active":"2023-11-06T02:08:31.604479+0000","last_peered":"2023-11-06T02:08:31.604479+0000","last_clean":"2023-11-06T02:08:31.604479+0000","last_became_active":"2023-11-06T02:08:30.221899+0000","last_became_peered":"2023-11-06T02:08:30.221899+0000","last_unstale":"2023-11-06T02:08:31.604479+0000","last_undegraded":"2023-11-06T02:08:31.604479+0000","last_fullsized":"2023-11-06T02:08:31.604479+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:43:56.017919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.652973+0000","last_change":"2023-11-06T02:08:30.644462+0000","last_active":"2023-11-06T02:08:32.652973+0000","last_peered":"2023-11-06T02:08:32.652973+0000","last_clean":"2023-11-06T02:08:32.652973+0000","last_became_active":"2023-11-06T02:08:30.644160+0000","last_became_peered":"2023-11-06T02:08:30.644160+0000","last_unstale":"2023-11-06T02:08:32.652973+0000","last_undegraded":"2023-11-06T02:08:32.652973+0000","last_fullsized":"2023-11-06T02:08:32.652973+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:43:23.215718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.652108+0000","last_change":"2023-11-06T02:08:30.214519+0000","last_active":"2023-11-06T02:08:32.652108+0000","last_peered":"2023-11-06T02:08:32.652108+0000","last_clean":"2023-11-06T02:08:32.652108+0000","last_became_active":"2023-11-06T02:08:30.214225+0000","last_became_peered":"2023-11-06T02:08:30.214225+0000","last_unstale":"2023-11-06T02:08:32.652108+0000","last_undegraded":"2023-11-06T02:08:32.652108+0000","last_fullsized":"2023-11-06T02:08:32.652108+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:30:21.274364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.605196+0000","last_change":"2023-11-06T02:08:30.617971+0000","last_active":"2023-11-06T02:08:31.605196+0000","last_peered":"2023-11-06T02:08:31.605196+0000","last_clean":"2023-11-06T02:08:31.605196+0000","last_became_active":"2023-11-06T02:08:30.617692+0000","last_became_peered":"2023-11-06T02:08:30.617692+0000","last_unstale":"2023-11-06T02:08:31.605196+0000","last_undegraded":"2023-11-06T02:08:31.605196+0000","last_fullsized":"2023-11-06T02:08:31.605196+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:23:59.639528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"22'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.609142+0000","last_change":"2023-11-06T02:08:30.224834+0000","last_active":"2023-11-06T02:08:31.609142+0000","last_peered":"2023-11-06T02:08:31.609142+0000","last_clean":"2023-11-06T02:08:31.609142+0000","last_became_active":"2023-11-06T02:08:30.224244+0000","last_became_peered":"2023-11-06T02:08:30.224244+0000","last_unstale":"2023-11-06T02:08:31.609142+0000","last_undegraded":"2023-11-06T02:08:31.609142+0000","last_fullsized":"2023-11-06T02:08:31.609142+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:40:04.297054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.654928+0000","last_change":"2023-11-06T02:08:30.619430+0000","last_active":"2023-11-06T02:08:32.654928+0000","last_peered":"2023-11-06T02:08:32.654928+0000","last_clean":"2023-11-06T02:08:32.654928+0000","last_became_active":"2023-11-06T02:08:30.619155+0000","last_became_peered":"2023-11-06T02:08:30.619155+0000","last_unstale":"2023-11-06T02:08:32.654928+0000","last_undegraded":"2023-11-06T02:08:32.654928+0000","last_fullsized":"2023-11-06T02:08:32.654928+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:30:22.278201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"22'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.652274+0000","last_change":"2023-11-06T02:08:30.227830+0000","last_active":"2023-11-06T02:08:32.652274+0000","last_peered":"2023-11-06T02:08:32.652274+0000","last_clean":"2023-11-06T02:08:32.652274+0000","last_became_active":"2023-11-06T02:08:30.227476+0000","last_became_peered":"2023-11-06T02:08:30.227476+0000","last_unstale":"2023-11-06T02:08:32.652274+0000","last_undegraded":"2023-11-06T02:08:32.652274+0000","last_fullsized":"2023-11-06T02:08:32.652274+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:30:21.274364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.609173+0000","last_change":"2023-11-06T02:08:30.651612+0000","last_active":"2023-11-06T02:08:31.609173+0000","last_peered":"2023-11-06T02:08:31.609173+0000","last_clean":"2023-11-06T02:08:31.609173+0000","last_became_active":"2023-11-06T02:08:30.650925+0000","last_became_peered":"2023-11-06T02:08:30.650925+0000","last_unstale":"2023-11-06T02:08:31.609173+0000","last_undegraded":"2023-11-06T02:08:31.609173+0000","last_fullsized":"2023-11-06T02:08:31.609173+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:31:28.301664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"22'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.413042+0000","last_change":"2023-11-06T02:08:30.217273+0000","last_active":"2023-11-06T02:08:32.413042+0000","last_peered":"2023-11-06T02:08:32.413042+0000","last_clean":"2023-11-06T02:08:32.413042+0000","last_became_active":"2023-11-06T02:08:30.216532+0000","last_became_peered":"2023-11-06T02:08:30.216532+0000","last_unstale":"2023-11-06T02:08:32.413042+0000","last_undegraded":"2023-11-06T02:08:32.413042+0000","last_fullsized":"2023-11-06T02:08:32.413042+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:33:38.355715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.608910+0000","last_change":"2023-11-06T02:08:30.651587+0000","last_active":"2023-11-06T02:08:31.608910+0000","last_peered":"2023-11-06T02:08:31.608910+0000","last_clean":"2023-11-06T02:08:31.608910+0000","last_became_active":"2023-11-06T02:08:30.650762+0000","last_became_peered":"2023-11-06T02:08:30.650762+0000","last_unstale":"2023-11-06T02:08:31.608910+0000","last_undegraded":"2023-11-06T02:08:31.608910+0000","last_fullsized":"2023-11-06T02:08:31.608910+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:52.535882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"22'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.413347+0000","last_change":"2023-11-06T02:08:30.220284+0000","last_active":"2023-11-06T02:08:32.413347+0000","last_peered":"2023-11-06T02:08:32.413347+0000","last_clean":"2023-11-06T02:08:32.413347+0000","last_became_active":"2023-11-06T02:08:30.219571+0000","last_became_peered":"2023-11-06T02:08:30.219571+0000","last_unstale":"2023-11-06T02:08:32.413347+0000","last_undegraded":"2023-11-06T02:08:32.413347+0000","last_fullsized":"2023-11-06T02:08:32.413347+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:24:09.801921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.604654+0000","last_change":"2023-11-06T02:08:30.621580+0000","last_active":"2023-11-06T02:08:31.604654+0000","last_peered":"2023-11-06T02:08:31.604654+0000","last_clean":"2023-11-06T02:08:31.604654+0000","last_became_active":"2023-11-06T02:08:30.618164+0000","last_became_peered":"2023-11-06T02:08:30.618164+0000","last_unstale":"2023-11-06T02:08:31.604654+0000","last_undegraded":"2023-11-06T02:08:31.604654+0000","last_fullsized":"2023-11-06T02:08:31.604654+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:57:32.079075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"22'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.604995+0000","last_change":"2023-11-06T02:08:30.208980+0000","last_active":"2023-11-06T02:08:31.604995+0000","last_peered":"2023-11-06T02:08:31.604995+0000","last_clean":"2023-11-06T02:08:31.604995+0000","last_became_active":"2023-11-06T02:08:30.208339+0000","last_became_peered":"2023-11-06T02:08:30.208339+0000","last_unstale":"2023-11-06T02:08:31.604995+0000","last_undegraded":"2023-11-06T02:08:31.604995+0000","last_fullsized":"2023-11-06T02:08:31.604995+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:31:31.882709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:31.608588+0000","last_change":"2023-11-06T02:08:30.634399+0000","last_active":"2023-11-06T02:08:31.608588+0000","last_peered":"2023-11-06T02:08:31.608588+0000","last_clean":"2023-11-06T02:08:31.608588+0000","last_became_active":"2023-11-06T02:08:30.633674+0000","last_became_peered":"2023-11-06T02:08:30.633674+0000","last_unstale":"2023-11-06T02:08:31.608588+0000","last_undegraded":"2023-11-06T02:08:31.608588+0000","last_fullsized":"2023-11-06T02:08:31.608588+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:30.181639+0000","last_clean_scrub_stamp":"2023-11-06T02:08:30.181639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:14:45.248292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1f","version":"22'4","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T02:08:32.651772+0000","last_change":"2023-11-06T02:08:30.220947+0000","last_active":"2023-11-06T02:08:32.651772+0000","last_peered":"2023-11-06T02:08:32.651772+0000","last_clean":"2023-11-06T02:08:32.651772+0000","last_became_active":"2023-11-06T02:08:30.220379+0000","last_became_peered":"2023-11-06T02:08:30.220379+0000","last_unstale":"2023-11-06T02:08:32.651772+0000","last_undegraded":"2023-11-06T02:08:32.651772+0000","last_fullsized":"2023-11-06T02:08:32.651772+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:29.177803+0000","last_clean_scrub_stamp":"2023-11-06T02:08:29.177803+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:43:56.017919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27796,"kb_used_data":844,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344044,"statfs":{"total":96636764160,"available":96608301056,"internally_reserved":0,"allocated":864256,"data_stored":681747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69799999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0660000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0029999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83499999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91500000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95599999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0960000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27188,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":241664,"data_stored":89599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0820000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93400000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91900000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81399999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.997}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":244,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":249856,"data_stored":90251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83099999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78000000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0049999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.165}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91500000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1870000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":262144,"data_stored":91219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0129999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95599999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.145}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1299999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.101}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2150000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31288,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340552,"statfs":{"total":96636764160,"available":96604725248,"internally_reserved":0,"allocated":245760,"data_stored":89689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89200000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.266}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84899999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99399999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31284,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340556,"statfs":{"total":96636764160,"available":96604729344,"internally_reserved":0,"allocated":241664,"data_stored":89599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.311}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70399999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71399999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.625}]}]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31284,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340556,"statfs":{"total":96636764160,"available":96604729344,"internally_reserved":0,"allocated":241664,"data_stored":89599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.503}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73899999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84599999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31864,"kb_used_data":816,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339976,"statfs":{"total":96636764160,"available":96604135424,"internally_reserved":0,"allocated":835584,"data_stored":679530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71099999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0009999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95899999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79800000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2270000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.194}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T02:08:43.487 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-11-06T02:08:43.487 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-11-06T02:08:43.488 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-11-06T02:08:43.488 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-06T02:08:43.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.642+0000 7f3248d24640 1 Processor -- start 2023-11-06T02:08:43.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.642+0000 7f3248d24640 1 -- start start 2023-11-06T02:08:43.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.642+0000 7f3248d24640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f32440b0790 0x7f32440c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:43.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.642+0000 7f3248d24640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32440cab80 con 0x7f324415ab50 2023-11-06T02:08:43.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.642+0000 7f3248d24640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f324415af30 con 0x7f32440cb010 2023-11-06T02:08:43.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.642+0000 7f3248d24640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32440cb5b0 con 0x7f32440b0790 2023-11-06T02:08:43.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.642+0000 7f324259c640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f324415ab50 0x7f32440b0080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:54048/0 (socket says 172.21.15.49:54048) 2023-11-06T02:08:43.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.642+0000 7f324259c640 1 -- 172.21.15.49:0/349105850 learned_addr learned my addr 172.21.15.49:0/349105850 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:08:43.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.642+0000 7f324159a640 1 -- 172.21.15.49:0/349105850 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1429507395 0 0) 0x7f32440cab80 con 0x7f324415ab50 2023-11-06T02:08:43.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.642+0000 7f324159a640 1 -- 172.21.15.49:0/349105850 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3230003530 con 0x7f324415ab50 2023-11-06T02:08:43.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.642+0000 7f3241d9b640 1 --2- 172.21.15.49:0/349105850 >> v2:172.21.15.150:3300/0 conn(0x7f32440b0790 0x7f32440c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:43.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.646+0000 7f324159a640 1 -- 172.21.15.49:0/349105850 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3900970237 0 0) 0x7f3230003530 con 0x7f324415ab50 2023-11-06T02:08:43.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.646+0000 7f324159a640 1 -- 172.21.15.49:0/349105850 >> v2:172.21.15.150:3300/0 conn(0x7f32440b0790 msgr2=0x7f32440c9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:43.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.646+0000 7f324159a640 1 --2- 172.21.15.49:0/349105850 >> v2:172.21.15.150:3300/0 conn(0x7f32440b0790 0x7f32440c9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:43.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.646+0000 7f324159a640 1 -- 172.21.15.49:0/349105850 >> v1:172.21.15.150:6789/0 conn(0x7f32440cb010 legacy=0x7f32440c2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:43.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.646+0000 7f324159a640 1 -- 172.21.15.49:0/349105850 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32440cb890 con 0x7f324415ab50 2023-11-06T02:08:43.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.646+0000 7f324159a640 1 -- 172.21.15.49:0/349105850 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7f323c001800 con 0x7f324415ab50 2023-11-06T02:08:43.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.646+0000 7f324159a640 1 -- 172.21.15.49:0/349105850 >> v1:172.21.15.49:6789/0 conn(0x7f324415ab50 legacy=0x7f32440b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:43.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.646+0000 7f324159a640 1 --2- 172.21.15.49:0/349105850 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3230003980 0x7f3230003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:43.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.646+0000 7f324159a640 1 --2- 172.21.15.49:0/349105850 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f32300047d0 0x7f3230004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:43.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.646+0000 7f324159a640 1 --2- 172.21.15.49:0/349105850 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3230005190 0x7f323000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:43.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.646+0000 7f324159a640 1 -- 172.21.15.49:0/349105850 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32440cb5b0 con 0x7f3230005190 2023-11-06T02:08:43.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.646+0000 7f324159a640 1 -- 172.21.15.49:0/349105850 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f323000c0e0 con 0x7f3230003980 2023-11-06T02:08:43.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.646+0000 7f324159a640 1 -- 172.21.15.49:0/349105850 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f323000c2a0 con 0x7f32300047d0 2023-11-06T02:08:43.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.646+0000 7f3242d9d640 1 --2- 172.21.15.49:0/349105850 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3230005190 0x7f323000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:43.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.646+0000 7f3241d9b640 1 --2- 172.21.15.49:0/349105850 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f32300047d0 0x7f3230004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:43.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.646+0000 7f324259c640 1 --2- 172.21.15.49:0/349105850 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3230003980 0x7f3230003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:43.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.646+0000 7f3242d9d640 1 -- 172.21.15.49:0/349105850 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f32300047d0 msgr2=0x7f3230004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:43.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.646+0000 7f3242d9d640 1 --2- 172.21.15.49:0/349105850 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f32300047d0 0x7f3230004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:43.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.646+0000 7f3242d9d640 1 -- 172.21.15.49:0/349105850 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3230003980 msgr2=0x7f3230003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:43.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.646+0000 7f3242d9d640 1 --2- 172.21.15.49:0/349105850 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3230003980 0x7f3230003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:43.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.646+0000 7f3242d9d640 1 -- 172.21.15.49:0/349105850 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3230003530 con 0x7f3230005190 2023-11-06T02:08:43.650 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.646+0000 7f3242d9d640 1 -- 172.21.15.49:0/349105850 reap_dead start 2023-11-06T02:08:43.651 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.646+0000 7f3242d9d640 1 --2- 172.21.15.49:0/349105850 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3230005190 0x7f323000b990 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3234001780 tx=0x7f323401c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a5fd6410a5884ad server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:43.651 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.646+0000 7f324159a640 1 -- 172.21.15.49:0/349105850 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32340052d0 con 0x7f3230005190 2023-11-06T02:08:43.651 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.646+0000 7f324159a640 1 -- 172.21.15.49:0/349105850 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32340a7050 con 0x7f3230005190 2023-11-06T02:08:43.651 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.646+0000 7f324159a640 1 -- 172.21.15.49:0/349105850 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3234003840 con 0x7f3230005190 2023-11-06T02:08:43.651 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.646+0000 7f3248d24640 1 -- 172.21.15.49:0/349105850 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3230005190 msgr2=0x7f323000b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:43.652 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.646+0000 7f3248d24640 1 --2- 172.21.15.49:0/349105850 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3230005190 0x7f323000b990 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3234001780 tx=0x7f323401c400 comp rx=0 tx=0).stop 2023-11-06T02:08:43.652 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.646+0000 7f3248d24640 1 -- 172.21.15.49:0/349105850 shutdown_connections 2023-11-06T02:08:43.652 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.646+0000 7f3248d24640 1 --2- 172.21.15.49:0/349105850 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3230005190 0x7f323000b990 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:43.652 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.646+0000 7f3248d24640 1 -- 172.21.15.49:0/349105850 >> 172.21.15.49:0/349105850 conn(0x7f324406c0a0 msgr2=0x7f324406c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:43.652 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.646+0000 7f3248d24640 1 -- 172.21.15.49:0/349105850 shutdown_connections 2023-11-06T02:08:43.653 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.646+0000 7f3248d24640 1 -- 172.21.15.49:0/349105850 wait complete. 2023-11-06T02:08:43.653 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.650+0000 7f3248d24640 1 Processor -- start 2023-11-06T02:08:43.653 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.650+0000 7f3248d24640 1 -- start start 2023-11-06T02:08:43.653 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.650+0000 7f3248d24640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3230005190 0x7f32441e3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:43.653 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.650+0000 7f3248d24640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3244157fe0 0x7f32440c7a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:43.654 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.650+0000 7f3248d24640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32440c6700 0x7f32440ce570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:43.654 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.650+0000 7f3248d24640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32440c6270 con 0x7f3230005190 2023-11-06T02:08:43.654 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.650+0000 7f3248d24640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32440c8580 con 0x7f32440c6700 2023-11-06T02:08:43.654 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.650+0000 7f3248d24640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32440c5c60 con 0x7f3244157fe0 2023-11-06T02:08:43.654 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.650+0000 7f324259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3230005190 0x7f32441e3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:43.654 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.650+0000 7f324259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3230005190 0x7f32441e3e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52490/0 (socket says 172.21.15.49:52490) 2023-11-06T02:08:43.655 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.650+0000 7f324259c640 1 -- 172.21.15.49:0/3728374993 learned_addr learned my addr 172.21.15.49:0/3728374993 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:43.655 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.650+0000 7f3241d9b640 1 --2- 172.21.15.49:0/3728374993 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3244157fe0 0x7f32440c7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:43.655 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.650+0000 7f3242d9d640 1 --2- 172.21.15.49:0/3728374993 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32440c6700 0x7f32440ce570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:43.655 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.650+0000 7f324259c640 1 -- 172.21.15.49:0/3728374993 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3244157fe0 msgr2=0x7f32440c7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:43.655 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.650+0000 7f324259c640 1 --2- 172.21.15.49:0/3728374993 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3244157fe0 0x7f32440c7a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:43.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.650+0000 7f324259c640 1 -- 172.21.15.49:0/3728374993 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32440c6700 msgr2=0x7f32440ce570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:43.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.650+0000 7f324259c640 1 --2- 172.21.15.49:0/3728374993 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32440c6700 0x7f32440ce570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:43.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.650+0000 7f324259c640 1 -- 172.21.15.49:0/3728374993 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32440c82a0 con 0x7f3230005190 2023-11-06T02:08:43.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.650+0000 7f324259c640 1 --2- 172.21.15.49:0/3728374993 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3230005190 0x7f32441e3e50 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f323c00e800 tx=0x7f323c00ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:43.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.650+0000 7f323affd640 1 -- 172.21.15.49:0/3728374993 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f323c0070d0 con 0x7f3230005190 2023-11-06T02:08:43.657 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.650+0000 7f323affd640 1 -- 172.21.15.49:0/3728374993 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f323c007250 con 0x7f3230005190 2023-11-06T02:08:43.657 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.650+0000 7f3248d24640 1 -- 172.21.15.49:0/3728374993 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32440c5f00 con 0x7f3230005190 2023-11-06T02:08:43.657 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.650+0000 7f323affd640 1 -- 172.21.15.49:0/3728374993 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f323c007600 con 0x7f3230005190 2023-11-06T02:08:43.657 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.650+0000 7f3248d24640 1 -- 172.21.15.49:0/3728374993 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32440c7f90 con 0x7f3230005190 2023-11-06T02:08:43.657 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.650+0000 7f3248d24640 1 -- 172.21.15.49:0/3728374993 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32440c60c0 con 0x7f3230005190 2023-11-06T02:08:43.657 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.650+0000 7f323affd640 1 -- 172.21.15.49:0/3728374993 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f323c0057b0 con 0x7f3230005190 2023-11-06T02:08:43.658 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.650+0000 7f323affd640 1 --2- 172.21.15.49:0/3728374993 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3224065a90 0x7f3224067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:43.658 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.650+0000 7f323affd640 1 -- 172.21.15.49:0/3728374993 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f323c083340 con 0x7f3230005190 2023-11-06T02:08:43.658 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.654+0000 7f3241d9b640 1 --2- 172.21.15.49:0/3728374993 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3224065a90 0x7f3224067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:43.658 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.654+0000 7f3241d9b640 1 --2- 172.21.15.49:0/3728374993 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3224065a90 0x7f3224067f50 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f32441e1ec0 tx=0x7f322c00b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:43.659 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.654+0000 7f323affd640 1 -- 172.21.15.49:0/3728374993 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f323c050ba0 con 0x7f3230005190 2023-11-06T02:08:43.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.838+0000 7f3248d24640 1 -- 172.21.15.49:0/3728374993 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f32440b0de0 con 0x7f3230005190 2023-11-06T02:08:43.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.838+0000 7f323affd640 1 -- 172.21.15.49:0/3728374993 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f323c01a310 con 0x7f3230005190 2023-11-06T02:08:43.835 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:08:43.836 INFO:teuthology.orchestra.run.smithi049.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-11-06T02:08:43.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.842+0000 7f3248d24640 1 -- 172.21.15.49:0/3728374993 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3224065a90 msgr2=0x7f3224067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:43.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.842+0000 7f3248d24640 1 --2- 172.21.15.49:0/3728374993 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3224065a90 0x7f3224067f50 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f32441e1ec0 tx=0x7f322c00b040 comp rx=0 tx=0).stop 2023-11-06T02:08:43.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.842+0000 7f3248d24640 1 -- 172.21.15.49:0/3728374993 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3230005190 msgr2=0x7f32441e3e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:43.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.842+0000 7f3248d24640 1 --2- 172.21.15.49:0/3728374993 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3230005190 0x7f32441e3e50 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f323c00e800 tx=0x7f323c00ecc0 comp rx=0 tx=0).stop 2023-11-06T02:08:43.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.842+0000 7f3248d24640 1 -- 172.21.15.49:0/3728374993 shutdown_connections 2023-11-06T02:08:43.839 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.842+0000 7f3248d24640 1 --2- 172.21.15.49:0/3728374993 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3224065a90 0x7f3224067f50 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:43.839 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.842+0000 7f3248d24640 1 --2- 172.21.15.49:0/3728374993 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32440c6700 0x7f32440ce570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:43.839 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.842+0000 7f3248d24640 1 --2- 172.21.15.49:0/3728374993 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3244157fe0 0x7f32440c7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:43.839 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.842+0000 7f3248d24640 1 --2- 172.21.15.49:0/3728374993 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3230005190 0x7f32441e3e50 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:43.839 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.842+0000 7f3248d24640 1 -- 172.21.15.49:0/3728374993 >> 172.21.15.49:0/3728374993 conn(0x7f324406c0a0 msgr2=0x7f32440b0070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:43.839 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.842+0000 7f3248d24640 1 -- 172.21.15.49:0/3728374993 shutdown_connections 2023-11-06T02:08:43.840 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.842+0000 7f3248d24640 1 -- 172.21.15.49:0/3728374993 wait complete. 2023-11-06T02:08:43.853 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-11-06T02:08:43.854 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T02:08:43.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.970+0000 7ff90ee77640 1 Processor -- start 2023-11-06T02:08:43.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.970+0000 7ff90ee77640 1 -- start start 2023-11-06T02:08:43.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.970+0000 7ff90ee77640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff90815ab50 0x7ff9080c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:43.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.970+0000 7ff90ee77640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff9080b0790 0x7ff9080c2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:43.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.970+0000 7ff90ee77640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9080cab80 con 0x7ff9080cb010 2023-11-06T02:08:43.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.970+0000 7ff90ee77640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff90815af30 con 0x7ff9080b0790 2023-11-06T02:08:43.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.970+0000 7ff90ee77640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9080cb5b0 con 0x7ff90815ab50 2023-11-06T02:08:43.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.970+0000 7ff90d414640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff9080b0790 0x7ff9080c2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:43.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.970+0000 7ff90d414640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff9080b0790 0x7ff9080c2710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52506/0 (socket says 172.21.15.49:52506) 2023-11-06T02:08:43.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.970+0000 7ff90d414640 1 -- 172.21.15.49:0/2488540560 learned_addr learned my addr 172.21.15.49:0/2488540560 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:43.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.974+0000 7ff90cc13640 1 --2- 172.21.15.49:0/2488540560 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff90815ab50 0x7ff9080c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:43.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.974+0000 7ff90d414640 1 -- 172.21.15.49:0/2488540560 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff90815ab50 msgr2=0x7ff9080c9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:43.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.974+0000 7ff90d414640 1 --2- 172.21.15.49:0/2488540560 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff90815ab50 0x7ff9080c9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:43.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.974+0000 7ff90d414640 1 -- 172.21.15.49:0/2488540560 >> v1:172.21.15.150:6789/0 conn(0x7ff9080cb010 legacy=0x7ff9080b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:43.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.974+0000 7ff90d414640 1 -- 172.21.15.49:0/2488540560 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9080cb890 con 0x7ff9080b0790 2023-11-06T02:08:43.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.974+0000 7ff90d414640 1 --2- 172.21.15.49:0/2488540560 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9080b0790 0x7ff9080c2710 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff900009e50 tx=0x7ff90001c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=964b43e0757fe61a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:43.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.974+0000 7ff9077fe640 1 -- 172.21.15.49:0/2488540560 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9000a5820 con 0x7ff9080b0790 2023-11-06T02:08:43.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.974+0000 7ff9077fe640 1 -- 172.21.15.49:0/2488540560 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9000a5980 con 0x7ff9080b0790 2023-11-06T02:08:43.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.974+0000 7ff9077fe640 1 -- 172.21.15.49:0/2488540560 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9000a5d10 con 0x7ff9080b0790 2023-11-06T02:08:43.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.974+0000 7ff90ee77640 1 -- 172.21.15.49:0/2488540560 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9080b0790 msgr2=0x7ff9080c2710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:43.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.974+0000 7ff90ee77640 1 --2- 172.21.15.49:0/2488540560 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9080b0790 0x7ff9080c2710 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff900009e50 tx=0x7ff90001c9a0 comp rx=0 tx=0).stop 2023-11-06T02:08:43.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.974+0000 7ff90ee77640 1 -- 172.21.15.49:0/2488540560 shutdown_connections 2023-11-06T02:08:43.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.974+0000 7ff90ee77640 1 --2- 172.21.15.49:0/2488540560 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9080b0790 0x7ff9080c2710 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:43.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.974+0000 7ff90ee77640 1 --2- 172.21.15.49:0/2488540560 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff90815ab50 0x7ff9080c9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:43.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.974+0000 7ff90ee77640 1 -- 172.21.15.49:0/2488540560 >> 172.21.15.49:0/2488540560 conn(0x7ff90806c0a0 msgr2=0x7ff90806c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:43.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.974+0000 7ff90ee77640 1 -- 172.21.15.49:0/2488540560 shutdown_connections 2023-11-06T02:08:43.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.974+0000 7ff90ee77640 1 -- 172.21.15.49:0/2488540560 wait complete. 2023-11-06T02:08:43.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.978+0000 7ff90ee77640 1 Processor -- start 2023-11-06T02:08:43.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.978+0000 7ff90ee77640 1 -- start start 2023-11-06T02:08:43.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.978+0000 7ff90ee77640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff9080b0790 0x7ff908157670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:43.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.978+0000 7ff90ee77640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff9080cb010 0x7ff90815b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:43.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.978+0000 7ff90ee77640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff90815ab50 0x7ff90814c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:43.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.978+0000 7ff90ee77640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9080c7340 con 0x7ff90815ab50 2023-11-06T02:08:43.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.978+0000 7ff90ee77640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9081585a0 con 0x7ff9080b0790 2023-11-06T02:08:43.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.978+0000 7ff90ee77640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9080c1b80 con 0x7ff9080cb010 2023-11-06T02:08:43.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.978+0000 7ff90d414640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff90815ab50 0x7ff90814c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:43.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.978+0000 7ff90d414640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff90815ab50 0x7ff90814c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52512/0 (socket says 172.21.15.49:52512) 2023-11-06T02:08:43.980 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.978+0000 7ff90d414640 1 -- 172.21.15.49:0/2363470002 learned_addr learned my addr 172.21.15.49:0/2363470002 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:43.980 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.978+0000 7ff907fff640 1 --2- 172.21.15.49:0/2363470002 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff9080cb010 0x7ff90815b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:43.980 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.978+0000 7ff90cc13640 1 --2- 172.21.15.49:0/2363470002 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff9080b0790 0x7ff908157670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:43.980 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.978+0000 7ff90d414640 1 -- 172.21.15.49:0/2363470002 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff9080cb010 msgr2=0x7ff90815b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:43.980 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.978+0000 7ff90d414640 1 --2- 172.21.15.49:0/2363470002 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff9080cb010 0x7ff90815b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:43.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.978+0000 7ff90d414640 1 -- 172.21.15.49:0/2363470002 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff9080b0790 msgr2=0x7ff908157670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:43.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.978+0000 7ff90d414640 1 --2- 172.21.15.49:0/2363470002 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff9080b0790 0x7ff908157670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:43.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.978+0000 7ff90d414640 1 -- 172.21.15.49:0/2363470002 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff900009b30 con 0x7ff90815ab50 2023-11-06T02:08:43.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.978+0000 7ff90cc13640 1 --2- 172.21.15.49:0/2363470002 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff9080b0790 0x7ff908157670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:08:43.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.978+0000 7ff90d414640 1 --2- 172.21.15.49:0/2363470002 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff90815ab50 0x7ff90814c9c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff90001b050 tx=0x7ff900019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:43.982 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.978+0000 7ff9057fa640 1 -- 172.21.15.49:0/2363470002 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9000b8670 con 0x7ff90815ab50 2023-11-06T02:08:43.982 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.978+0000 7ff9057fa640 1 -- 172.21.15.49:0/2363470002 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9000b87d0 con 0x7ff90815ab50 2023-11-06T02:08:43.982 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.978+0000 7ff90ee77640 1 -- 172.21.15.49:0/2363470002 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff90815b5b0 con 0x7ff90815ab50 2023-11-06T02:08:43.982 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.978+0000 7ff9057fa640 1 -- 172.21.15.49:0/2363470002 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9000b8b80 con 0x7ff90815ab50 2023-11-06T02:08:43.982 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.978+0000 7ff90ee77640 1 -- 172.21.15.49:0/2363470002 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff908157fe0 con 0x7ff90815ab50 2023-11-06T02:08:43.983 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.978+0000 7ff90ee77640 1 -- 172.21.15.49:0/2363470002 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9080b0de0 con 0x7ff90815ab50 2023-11-06T02:08:43.983 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.978+0000 7ff9057fa640 1 -- 172.21.15.49:0/2363470002 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7ff9000b8ce0 con 0x7ff90815ab50 2023-11-06T02:08:43.983 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.978+0000 7ff9057fa640 1 --2- 172.21.15.49:0/2363470002 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff8e8065a90 0x7ff8e8067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:43.983 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.978+0000 7ff9057fa640 1 -- 172.21.15.49:0/2363470002 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff90011e930 con 0x7ff90815ab50 2023-11-06T02:08:43.983 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.978+0000 7ff90cc13640 1 --2- 172.21.15.49:0/2363470002 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff8e8065a90 0x7ff8e8067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:43.983 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.982+0000 7ff90cc13640 1 --2- 172.21.15.49:0/2363470002 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff8e8065a90 0x7ff8e8067f50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff8f0006fd0 tx=0x7ff8f0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:43.984 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:43.982+0000 7ff9057fa640 1 -- 172.21.15.49:0/2363470002 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7ff9000ec370 con 0x7ff90815ab50 2023-11-06T02:08:44.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.134+0000 7ff90ee77640 1 -- 172.21.15.49:0/2363470002 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff9080b0de0 con 0x7ff90815ab50 2023-11-06T02:08:44.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.138+0000 7ff9057fa640 1 -- 172.21.15.49:0/2363470002 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+17811 (secure 0 0 0) 0x7ff9000ef990 con 0x7ff90815ab50 2023-11-06T02:08:44.135 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:08:44.135 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":23,"fsid":"1a8b035c-6ba4-499a-b876-aea47fd9c0d4","created":"2023-11-06T02:08:06.440857+0000","modified":"2023-11-06T02:08:31.594964+0000","last_up_change":"2023-11-06T02:08:17.982372+0000","last_in_change":"2023-11-06T02:08:08.906631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T02:08:19.252222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T02:08:24.776211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-06T02:08:28.936178+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-06T02:08:29.563468+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ad55af8-0571-4e6a-8722-b5d429b7496d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6809","nonce":38116},{"type":"v1","addr":"172.21.15.49:6811","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6813","nonce":38116},{"type":"v1","addr":"172.21.15.49:6815","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6821","nonce":38116},{"type":"v1","addr":"172.21.15.49:6823","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6817","nonce":38116},{"type":"v1","addr":"172.21.15.49:6819","nonce":38116}]},"public_addr":"172.21.15.49:6811/38116","cluster_addr":"172.21.15.49:6815/38116","heartbeat_back_addr":"172.21.15.49:6823/38116","heartbeat_front_addr":"172.21.15.49:6819/38116","state":["exists","up"]},{"osd":1,"uuid":"c8b31f85-a757-43ed-9419-351acc46611b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":38113},{"type":"v1","addr":"172.21.15.49:6810","nonce":38113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":38113},{"type":"v1","addr":"172.21.15.49:6814","nonce":38113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":38113},{"type":"v1","addr":"172.21.15.49:6822","nonce":38113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":38113},{"type":"v1","addr":"172.21.15.49:6818","nonce":38113}]},"public_addr":"172.21.15.49:6810/38113","cluster_addr":"172.21.15.49:6814/38113","heartbeat_back_addr":"172.21.15.49:6822/38113","heartbeat_front_addr":"172.21.15.49:6818/38113","state":["exists","up"]},{"osd":2,"uuid":"555b3bf3-0a70-4b91-8efd-e4204b5fc26e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":38114},{"type":"v1","addr":"172.21.15.49:6801","nonce":38114}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":38114},{"type":"v1","addr":"172.21.15.49:6803","nonce":38114}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":38114},{"type":"v1","addr":"172.21.15.49:6807","nonce":38114}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":38114},{"type":"v1","addr":"172.21.15.49:6805","nonce":38114}]},"public_addr":"172.21.15.49:6801/38114","cluster_addr":"172.21.15.49:6803/38114","heartbeat_back_addr":"172.21.15.49:6807/38114","heartbeat_front_addr":"172.21.15.49:6805/38114","state":["exists","up"]},{"osd":3,"uuid":"8d4e5d77-03ef-4b9b-bfcb-5ae93a46795c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":38115},{"type":"v1","addr":"172.21.15.49:6825","nonce":38115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":38115},{"type":"v1","addr":"172.21.15.49:6827","nonce":38115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":38115},{"type":"v1","addr":"172.21.15.49:6831","nonce":38115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":38115},{"type":"v1","addr":"172.21.15.49:6829","nonce":38115}]},"public_addr":"172.21.15.49:6825/38115","cluster_addr":"172.21.15.49:6827/38115","heartbeat_back_addr":"172.21.15.49:6831/38115","heartbeat_front_addr":"172.21.15.49:6829/38115","state":["exists","up"]},{"osd":4,"uuid":"dba03ad4-68a3-4cf3-9218-ac611f02eacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":38417},{"type":"v1","addr":"172.21.15.150:6809","nonce":38417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":38417},{"type":"v1","addr":"172.21.15.150:6811","nonce":38417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":38417},{"type":"v1","addr":"172.21.15.150:6815","nonce":38417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":38417},{"type":"v1","addr":"172.21.15.150:6813","nonce":38417}]},"public_addr":"172.21.15.150:6809/38417","cluster_addr":"172.21.15.150:6811/38417","heartbeat_back_addr":"172.21.15.150:6815/38417","heartbeat_front_addr":"172.21.15.150:6813/38417","state":["exists","up"]},{"osd":5,"uuid":"0df9c303-c255-4640-8e91-01f1c3efe18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":38418},{"type":"v1","addr":"172.21.15.150:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":38418},{"type":"v1","addr":"172.21.15.150:6803","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":38418},{"type":"v1","addr":"172.21.15.150:6807","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":38418},{"type":"v1","addr":"172.21.15.150:6805","nonce":38418}]},"public_addr":"172.21.15.150:6801/38418","cluster_addr":"172.21.15.150:6803/38418","heartbeat_back_addr":"172.21.15.150:6807/38418","heartbeat_front_addr":"172.21.15.150:6805/38418","state":["exists","up"]},{"osd":6,"uuid":"e59fa189-4b70-4c06-9722-c02b90828bf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":38428},{"type":"v1","addr":"172.21.15.150:6825","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":38428},{"type":"v1","addr":"172.21.15.150:6827","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":38428},{"type":"v1","addr":"172.21.15.150:6831","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":38428},{"type":"v1","addr":"172.21.15.150:6829","nonce":38428}]},"public_addr":"172.21.15.150:6825/38428","cluster_addr":"172.21.15.150:6827/38428","heartbeat_back_addr":"172.21.15.150:6831/38428","heartbeat_front_addr":"172.21.15.150:6829/38428","state":["exists","up"]},{"osd":7,"uuid":"a64f57e1-ef98-46bd-b304-385759e95898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":38421},{"type":"v1","addr":"172.21.15.150:6817","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":38421},{"type":"v1","addr":"172.21.15.150:6819","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":38421},{"type":"v1","addr":"172.21.15.150:6823","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":38421},{"type":"v1","addr":"172.21.15.150:6821","nonce":38421}]},"public_addr":"172.21.15.150:6817/38421","cluster_addr":"172.21.15.150:6819/38421","heartbeat_back_addr":"172.21.15.150:6823/38421","heartbeat_front_addr":"172.21.15.150:6821/38421","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.438552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.413280+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.296466+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.424966+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.276953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.230984+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.410235+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.480084+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T02:08:44.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.142+0000 7ff90ee77640 1 -- 172.21.15.49:0/2363470002 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff8e8065a90 msgr2=0x7ff8e8067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:44.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.142+0000 7ff90ee77640 1 --2- 172.21.15.49:0/2363470002 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff8e8065a90 0x7ff8e8067f50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff8f0006fd0 tx=0x7ff8f0008040 comp rx=0 tx=0).stop 2023-11-06T02:08:44.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.142+0000 7ff90ee77640 1 -- 172.21.15.49:0/2363470002 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff90815ab50 msgr2=0x7ff90814c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:44.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.142+0000 7ff90ee77640 1 --2- 172.21.15.49:0/2363470002 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff90815ab50 0x7ff90814c9c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff90001b050 tx=0x7ff900019740 comp rx=0 tx=0).stop 2023-11-06T02:08:44.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.142+0000 7ff90ee77640 1 -- 172.21.15.49:0/2363470002 shutdown_connections 2023-11-06T02:08:44.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.142+0000 7ff90ee77640 1 --2- 172.21.15.49:0/2363470002 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff8e8065a90 0x7ff8e8067f50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:44.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.142+0000 7ff90ee77640 1 --2- 172.21.15.49:0/2363470002 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff90815ab50 0x7ff90814c9c0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:44.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.142+0000 7ff90ee77640 1 --2- 172.21.15.49:0/2363470002 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff9080cb010 0x7ff90815b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:44.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.142+0000 7ff90ee77640 1 --2- 172.21.15.49:0/2363470002 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff9080b0790 0x7ff908157670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:44.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.142+0000 7ff90ee77640 1 -- 172.21.15.49:0/2363470002 >> 172.21.15.49:0/2363470002 conn(0x7ff90806c0a0 msgr2=0x7ff9080c9bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:44.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.142+0000 7ff90ee77640 1 -- 172.21.15.49:0/2363470002 shutdown_connections 2023-11-06T02:08:44.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.142+0000 7ff90ee77640 1 -- 172.21.15.49:0/2363470002 wait complete. 2023-11-06T02:08:44.153 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T02:08:19.252222+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T02:08:24.776211+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T02:08:28.936178+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T02:08:29.563468+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T02:08:44.154 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T02:08:44.268 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.270+0000 7f576e65a640 1 Processor -- start 2023-11-06T02:08:44.268 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.270+0000 7f576e65a640 1 -- start start 2023-11-06T02:08:44.270 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.274+0000 7f576e65a640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f576815a880 0x7f57680b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:44.270 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.274+0000 7f576e65a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f576806d860 0x7f5768070610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:44.270 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.274+0000 7f576e65a640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57680ae360 con 0x7f5768070160 2023-11-06T02:08:44.270 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.274+0000 7f576e65a640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5768070b50 con 0x7f576815a880 2023-11-06T02:08:44.271 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.274+0000 7f576e65a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57680ae640 con 0x7f576806d860 2023-11-06T02:08:44.271 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.274+0000 7f575ffff640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f5768070160 0x7f576806d150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:54058/0 (socket says 172.21.15.49:54058) 2023-11-06T02:08:44.271 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.274+0000 7f575ffff640 1 -- 172.21.15.49:0/3181096507 learned_addr learned my addr 172.21.15.49:0/3181096507 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:08:44.271 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.274+0000 7f5767fff640 1 --2- 172.21.15.49:0/3181096507 >> v2:172.21.15.49:3300/0 conn(0x7f576815a880 0x7f57680b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:44.271 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.274+0000 7f576cbf7640 1 --2- 172.21.15.49:0/3181096507 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f576806d860 0x7f5768070610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:44.272 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.274+0000 7f57677fe640 1 -- 172.21.15.49:0/3181096507 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3650443399 0 0) 0x7f57680ae360 con 0x7f5768070160 2023-11-06T02:08:44.272 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.274+0000 7f57677fe640 1 -- 172.21.15.49:0/3181096507 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5754003530 con 0x7f5768070160 2023-11-06T02:08:44.272 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.274+0000 7f5767fff640 1 -- 172.21.15.49:0/3181096507 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f576806d860 msgr2=0x7f5768070610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:44.272 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.274+0000 7f5767fff640 1 --2- 172.21.15.49:0/3181096507 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f576806d860 0x7f5768070610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:44.272 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.274+0000 7f5767fff640 1 -- 172.21.15.49:0/3181096507 >> v1:172.21.15.49:6789/0 conn(0x7f5768070160 legacy=0x7f576806d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:44.273 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.274+0000 7f5767fff640 1 -- 172.21.15.49:0/3181096507 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57680ae920 con 0x7f576815a880 2023-11-06T02:08:44.274 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.274+0000 7f5767fff640 1 --2- 172.21.15.49:0/3181096507 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f576815a880 0x7f57680b15c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f57580097c0 tx=0x7f575801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=727fa9306c7c7736 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:44.274 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.274+0000 7f57677fe640 1 -- 172.21.15.49:0/3181096507 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57580ad640 con 0x7f576815a880 2023-11-06T02:08:44.274 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.274+0000 7f57677fe640 1 -- 172.21.15.49:0/3181096507 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57580ad7a0 con 0x7f576815a880 2023-11-06T02:08:44.275 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.274+0000 7f57677fe640 1 -- 172.21.15.49:0/3181096507 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57580adb30 con 0x7f576815a880 2023-11-06T02:08:44.275 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.278+0000 7f576e65a640 1 -- 172.21.15.49:0/3181096507 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f576815a880 msgr2=0x7f57680b15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:44.275 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.278+0000 7f576e65a640 1 --2- 172.21.15.49:0/3181096507 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f576815a880 0x7f57680b15c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f57580097c0 tx=0x7f575801c480 comp rx=0 tx=0).stop 2023-11-06T02:08:44.275 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.278+0000 7f576e65a640 1 -- 172.21.15.49:0/3181096507 shutdown_connections 2023-11-06T02:08:44.275 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.278+0000 7f576e65a640 1 --2- 172.21.15.49:0/3181096507 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f576806d860 0x7f5768070610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:44.276 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.278+0000 7f576e65a640 1 --2- 172.21.15.49:0/3181096507 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f576815a880 0x7f57680b15c0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:44.276 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.278+0000 7f576e65a640 1 -- 172.21.15.49:0/3181096507 >> 172.21.15.49:0/3181096507 conn(0x7f576806a370 msgr2=0x7f57680ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:44.276 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.278+0000 7f576e65a640 1 -- 172.21.15.49:0/3181096507 shutdown_connections 2023-11-06T02:08:44.276 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.278+0000 7f576e65a640 1 -- 172.21.15.49:0/3181096507 wait complete. 2023-11-06T02:08:44.276 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.278+0000 7f576e65a640 1 Processor -- start 2023-11-06T02:08:44.277 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.278+0000 7f576e65a640 1 -- start start 2023-11-06T02:08:44.279 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.278+0000 7f576e65a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f576806d860 0x7f57680cb690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:44.279 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.278+0000 7f576e65a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5768070160 0x7f57680cda60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:44.280 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.278+0000 7f576e65a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f576815a880 0x7f57680c7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:44.280 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.278+0000 7f576e65a640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57680cdfa0 con 0x7f576806d860 2023-11-06T02:08:44.280 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.278+0000 7f576e65a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57680c8e60 con 0x7f576815a880 2023-11-06T02:08:44.280 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.278+0000 7f576e65a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57680cbd00 con 0x7f5768070160 2023-11-06T02:08:44.280 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.278+0000 7f5767fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f576806d860 0x7f57680cb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:44.281 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.278+0000 7f5767fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f576806d860 0x7f57680cb690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52520/0 (socket says 172.21.15.49:52520) 2023-11-06T02:08:44.281 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.278+0000 7f5767fff640 1 -- 172.21.15.49:0/1275646372 learned_addr learned my addr 172.21.15.49:0/1275646372 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:44.281 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.278+0000 7f576cbf7640 1 --2- 172.21.15.49:0/1275646372 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f576815a880 0x7f57680c7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:44.281 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.278+0000 7f5767fff640 1 -- 172.21.15.49:0/1275646372 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5768070160 msgr2=0x7f57680cda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:44.281 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.278+0000 7f5767fff640 1 --2- 172.21.15.49:0/1275646372 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5768070160 0x7f57680cda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:44.282 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.278+0000 7f5767fff640 1 -- 172.21.15.49:0/1275646372 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f576815a880 msgr2=0x7f57680c7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:44.282 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.278+0000 7f5767fff640 1 --2- 172.21.15.49:0/1275646372 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f576815a880 0x7f57680c7ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:44.282 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.278+0000 7f5767fff640 1 -- 172.21.15.49:0/1275646372 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5758009420 con 0x7f576806d860 2023-11-06T02:08:44.282 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.278+0000 7f5767fff640 1 --2- 172.21.15.49:0/1275646372 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f576806d860 0x7f57680cb690 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f57580b0040 tx=0x7f5758019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:44.282 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.278+0000 7f57657fa640 1 -- 172.21.15.49:0/1275646372 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57580b6610 con 0x7f576806d860 2023-11-06T02:08:44.283 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.278+0000 7f57657fa640 1 -- 172.21.15.49:0/1275646372 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57580a6040 con 0x7f576806d860 2023-11-06T02:08:44.283 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.278+0000 7f57657fa640 1 -- 172.21.15.49:0/1275646372 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57580b69b0 con 0x7f576806d860 2023-11-06T02:08:44.283 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.278+0000 7f576e65a640 1 -- 172.21.15.49:0/1275646372 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57680cbfe0 con 0x7f576806d860 2023-11-06T02:08:44.283 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.278+0000 7f576e65a640 1 -- 172.21.15.49:0/1275646372 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57680cc2c0 con 0x7f576806d860 2023-11-06T02:08:44.283 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.278+0000 7f576e65a640 1 -- 172.21.15.49:0/1275646372 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5768074040 con 0x7f576806d860 2023-11-06T02:08:44.283 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.282+0000 7f57657fa640 1 -- 172.21.15.49:0/1275646372 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f57580b4040 con 0x7f576806d860 2023-11-06T02:08:44.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.282+0000 7f57657fa640 1 --2- 172.21.15.49:0/1275646372 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5748065a60 0x7f5748067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:44.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.282+0000 7f57657fa640 1 -- 172.21.15.49:0/1275646372 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f575811d5d0 con 0x7f576806d860 2023-11-06T02:08:44.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.282+0000 7f575ffff640 1 --2- 172.21.15.49:0/1275646372 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5748065a60 0x7f5748067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:44.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.282+0000 7f575ffff640 1 --2- 172.21.15.49:0/1275646372 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5748065a60 0x7f5748067f20 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5750007fd0 tx=0x7f5750007940 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:44.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.286+0000 7f57657fa640 1 -- 172.21.15.49:0/1275646372 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f57580eaec0 con 0x7f576806d860 2023-11-06T02:08:44.431 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.434+0000 7f576e65a640 1 -- 172.21.15.49:0/1275646372 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f57680c92d0 con 0x7f576806d860 2023-11-06T02:08:44.432 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.434+0000 7f57657fa640 1 -- 172.21.15.49:0/1275646372 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f57580ee4e0 con 0x7f576806d860 2023-11-06T02:08:44.432 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2023-11-06T02:08:44.434 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.438+0000 7f576e65a640 1 -- 172.21.15.49:0/1275646372 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5748065a60 msgr2=0x7f5748067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:44.434 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.438+0000 7f576e65a640 1 --2- 172.21.15.49:0/1275646372 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5748065a60 0x7f5748067f20 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5750007fd0 tx=0x7f5750007940 comp rx=0 tx=0).stop 2023-11-06T02:08:44.434 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.438+0000 7f576e65a640 1 -- 172.21.15.49:0/1275646372 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f576806d860 msgr2=0x7f57680cb690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:44.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.438+0000 7f576e65a640 1 --2- 172.21.15.49:0/1275646372 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f576806d860 0x7f57680cb690 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f57580b0040 tx=0x7f5758019220 comp rx=0 tx=0).stop 2023-11-06T02:08:44.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.438+0000 7f576e65a640 1 -- 172.21.15.49:0/1275646372 shutdown_connections 2023-11-06T02:08:44.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.438+0000 7f576e65a640 1 --2- 172.21.15.49:0/1275646372 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5748065a60 0x7f5748067f20 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:44.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.438+0000 7f576e65a640 1 --2- 172.21.15.49:0/1275646372 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f576815a880 0x7f57680c7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:44.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.438+0000 7f576e65a640 1 --2- 172.21.15.49:0/1275646372 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5768070160 0x7f57680cda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:44.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.438+0000 7f576e65a640 1 --2- 172.21.15.49:0/1275646372 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f576806d860 0x7f57680cb690 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:44.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.438+0000 7f576e65a640 1 -- 172.21.15.49:0/1275646372 >> 172.21.15.49:0/1275646372 conn(0x7f576806a370 msgr2=0x7f576815d670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:44.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.438+0000 7f576e65a640 1 -- 172.21.15.49:0/1275646372 shutdown_connections 2023-11-06T02:08:44.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.438+0000 7f576e65a640 1 -- 172.21.15.49:0/1275646372 wait complete. 2023-11-06T02:08:44.449 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T02:08:44.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.570+0000 7f0227172640 1 Processor -- start 2023-11-06T02:08:44.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.570+0000 7f0227172640 1 -- start start 2023-11-06T02:08:44.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.570+0000 7f0227172640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f02200b50a0 0x7f02200cff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:44.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.570+0000 7f0227172640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f02200d0450 0x7f0220069d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:44.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.570+0000 7f0227172640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02200cada0 con 0x7f02200ca9c0 2023-11-06T02:08:44.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.570+0000 7f0227172640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02200cf7e0 con 0x7f02200b50a0 2023-11-06T02:08:44.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.570+0000 7f0227172640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f022015b6a0 con 0x7f02200d0450 2023-11-06T02:08:44.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.570+0000 7f021ffff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f02200b50a0 0x7f02200cff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:44.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.570+0000 7f021ffff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f02200b50a0 0x7f02200cff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52532/0 (socket says 172.21.15.49:52532) 2023-11-06T02:08:44.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.570+0000 7f021ffff640 1 -- 172.21.15.49:0/956039081 learned_addr learned my addr 172.21.15.49:0/956039081 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:44.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.570+0000 7f022570f640 1 --2- 172.21.15.49:0/956039081 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f02200d0450 0x7f0220069d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:44.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.570+0000 7f021ffff640 1 -- 172.21.15.49:0/956039081 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f02200d0450 msgr2=0x7f0220069d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:44.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.570+0000 7f021ffff640 1 --2- 172.21.15.49:0/956039081 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f02200d0450 0x7f0220069d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:44.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.570+0000 7f021ffff640 1 -- 172.21.15.49:0/956039081 >> v1:172.21.15.150:6789/0 conn(0x7f02200ca9c0 legacy=0x7f022015ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:44.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.570+0000 7f021ffff640 1 -- 172.21.15.49:0/956039081 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f022015b980 con 0x7f02200b50a0 2023-11-06T02:08:44.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.574+0000 7f021ffff640 1 --2- 172.21.15.49:0/956039081 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02200b50a0 0x7f02200cff10 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0214009870 tx=0x7f021401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f57b466d0ab2c2a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:44.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.574+0000 7f021f7fe640 1 -- 172.21.15.49:0/956039081 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02140ac600 con 0x7f02200b50a0 2023-11-06T02:08:44.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.574+0000 7f021f7fe640 1 -- 172.21.15.49:0/956039081 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02140a8050 con 0x7f02200b50a0 2023-11-06T02:08:44.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.574+0000 7f021f7fe640 1 -- 172.21.15.49:0/956039081 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02140ac980 con 0x7f02200b50a0 2023-11-06T02:08:44.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.574+0000 7f0227172640 1 -- 172.21.15.49:0/956039081 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02200b50a0 msgr2=0x7f02200cff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:44.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.574+0000 7f0227172640 1 --2- 172.21.15.49:0/956039081 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02200b50a0 0x7f02200cff10 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0214009870 tx=0x7f021401c4d0 comp rx=0 tx=0).stop 2023-11-06T02:08:44.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.574+0000 7f0227172640 1 -- 172.21.15.49:0/956039081 shutdown_connections 2023-11-06T02:08:44.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.574+0000 7f0227172640 1 --2- 172.21.15.49:0/956039081 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f02200d0450 0x7f0220069d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:44.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.574+0000 7f0227172640 1 --2- 172.21.15.49:0/956039081 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02200b50a0 0x7f02200cff10 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:44.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.574+0000 7f0227172640 1 -- 172.21.15.49:0/956039081 >> 172.21.15.49:0/956039081 conn(0x7f022006d4b0 msgr2=0x7f022006d890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:44.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.574+0000 7f0227172640 1 -- 172.21.15.49:0/956039081 shutdown_connections 2023-11-06T02:08:44.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.574+0000 7f0227172640 1 -- 172.21.15.49:0/956039081 wait complete. 2023-11-06T02:08:44.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.574+0000 7f0227172640 1 Processor -- start 2023-11-06T02:08:44.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.574+0000 7f0227172640 1 -- start start 2023-11-06T02:08:44.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.574+0000 7f0227172640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f02200b50a0 0x7f02200c6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:44.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.574+0000 7f0227172640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f02200ca9c0 0x7f0220153350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:44.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.574+0000 7f0227172640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02200d0450 0x7f0220153890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:44.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.574+0000 7f0227172640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02201f74e0 con 0x7f02200d0450 2023-11-06T02:08:44.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.574+0000 7f0227172640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02201f6720 con 0x7f02200ca9c0 2023-11-06T02:08:44.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.574+0000 7f0227172640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02201f6a00 con 0x7f02200b50a0 2023-11-06T02:08:44.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.574+0000 7f022570f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02200d0450 0x7f0220153890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:44.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.574+0000 7f022570f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02200d0450 0x7f0220153890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52538/0 (socket says 172.21.15.49:52538) 2023-11-06T02:08:44.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.574+0000 7f022570f640 1 -- 172.21.15.49:0/2784033319 learned_addr learned my addr 172.21.15.49:0/2784033319 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:44.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.574+0000 7f021ffff640 1 --2- 172.21.15.49:0/2784033319 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f02200ca9c0 0x7f0220153350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:44.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.574+0000 7f0224f0e640 1 --2- 172.21.15.49:0/2784033319 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f02200b50a0 0x7f02200c6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:44.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.574+0000 7f022570f640 1 -- 172.21.15.49:0/2784033319 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f02200b50a0 msgr2=0x7f02200c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:44.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.574+0000 7f022570f640 1 --2- 172.21.15.49:0/2784033319 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f02200b50a0 0x7f02200c6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:44.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.574+0000 7f022570f640 1 -- 172.21.15.49:0/2784033319 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f02200ca9c0 msgr2=0x7f0220153350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:44.579 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.574+0000 7f022570f640 1 --2- 172.21.15.49:0/2784033319 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f02200ca9c0 0x7f0220153350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:44.579 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.574+0000 7f022570f640 1 -- 172.21.15.49:0/2784033319 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02140094d0 con 0x7f02200d0450 2023-11-06T02:08:44.579 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.574+0000 7f022570f640 1 --2- 172.21.15.49:0/2784033319 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02200d0450 0x7f0220153890 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0218007e60 tx=0x7f021800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:44.579 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.574+0000 7f021d7fa640 1 -- 172.21.15.49:0/2784033319 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0218014420 con 0x7f02200d0450 2023-11-06T02:08:44.579 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.578+0000 7f021d7fa640 1 -- 172.21.15.49:0/2784033319 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02180145a0 con 0x7f02200d0450 2023-11-06T02:08:44.580 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.578+0000 7f021d7fa640 1 -- 172.21.15.49:0/2784033319 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0218014950 con 0x7f02200d0450 2023-11-06T02:08:44.580 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.578+0000 7f0227172640 1 -- 172.21.15.49:0/2784033319 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02201f6ce0 con 0x7f02200d0450 2023-11-06T02:08:44.580 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.578+0000 7f0227172640 1 -- 172.21.15.49:0/2784033319 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02201ed400 con 0x7f02200d0450 2023-11-06T02:08:44.581 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.578+0000 7f0227172640 1 -- 172.21.15.49:0/2784033319 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0220067140 con 0x7f02200d0450 2023-11-06T02:08:44.581 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.578+0000 7f021d7fa640 1 -- 172.21.15.49:0/2784033319 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f0218014ab0 con 0x7f02200d0450 2023-11-06T02:08:44.582 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.578+0000 7f021d7fa640 1 --2- 172.21.15.49:0/2784033319 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f020c065a60 0x7f020c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:44.582 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.578+0000 7f021d7fa640 1 -- 172.21.15.49:0/2784033319 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0218083c30 con 0x7f02200d0450 2023-11-06T02:08:44.582 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.578+0000 7f0224f0e640 1 --2- 172.21.15.49:0/2784033319 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f020c065a60 0x7f020c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:44.582 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.578+0000 7f0224f0e640 1 --2- 172.21.15.49:0/2784033319 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f020c065a60 0x7f020c067f20 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f022015c7b0 tx=0x7f0210009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:44.582 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.582+0000 7f021d7fa640 1 -- 172.21.15.49:0/2784033319 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f0218051480 con 0x7f02200d0450 2023-11-06T02:08:44.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.734+0000 7f0227172640 1 -- 172.21.15.49:0/2784033319 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f02201f7190 con 0x7f02200d0450 2023-11-06T02:08:44.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.734+0000 7f021d7fa640 1 -- 172.21.15.49:0/2784033319 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0218054aa0 con 0x7f02200d0450 2023-11-06T02:08:44.730 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 8 2023-11-06T02:08:44.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.734+0000 7f0227172640 1 -- 172.21.15.49:0/2784033319 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f020c065a60 msgr2=0x7f020c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:44.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.734+0000 7f0227172640 1 --2- 172.21.15.49:0/2784033319 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f020c065a60 0x7f020c067f20 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f022015c7b0 tx=0x7f0210009040 comp rx=0 tx=0).stop 2023-11-06T02:08:44.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.734+0000 7f0227172640 1 -- 172.21.15.49:0/2784033319 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02200d0450 msgr2=0x7f0220153890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:44.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.734+0000 7f0227172640 1 --2- 172.21.15.49:0/2784033319 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02200d0450 0x7f0220153890 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0218007e60 tx=0x7f021800e4b0 comp rx=0 tx=0).stop 2023-11-06T02:08:44.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.734+0000 7f0227172640 1 -- 172.21.15.49:0/2784033319 shutdown_connections 2023-11-06T02:08:44.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.734+0000 7f0227172640 1 --2- 172.21.15.49:0/2784033319 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f020c065a60 0x7f020c067f20 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:44.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.734+0000 7f0227172640 1 --2- 172.21.15.49:0/2784033319 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02200d0450 0x7f0220153890 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:44.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.734+0000 7f0227172640 1 --2- 172.21.15.49:0/2784033319 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f02200ca9c0 0x7f0220153350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:44.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.734+0000 7f0227172640 1 --2- 172.21.15.49:0/2784033319 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f02200b50a0 0x7f02200c6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:44.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.734+0000 7f0227172640 1 -- 172.21.15.49:0/2784033319 >> 172.21.15.49:0/2784033319 conn(0x7f022006d4b0 msgr2=0x7f02200b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:44.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.734+0000 7f0227172640 1 -- 172.21.15.49:0/2784033319 shutdown_connections 2023-11-06T02:08:44.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.734+0000 7f0227172640 1 -- 172.21.15.49:0/2784033319 wait complete. 2023-11-06T02:08:44.747 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T02:08:44.873 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.878+0000 7f4963dff640 1 Processor -- start 2023-11-06T02:08:44.873 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.878+0000 7f4963dff640 1 -- start start 2023-11-06T02:08:44.874 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.878+0000 7f4963dff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f495c0b0690 0x7f495c0b0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:44.874 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.878+0000 7f4963dff640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f495c161620 con 0x7f495c160680 2023-11-06T02:08:44.875 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.878+0000 7f4963dff640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f495c0ca830 con 0x7f495c15ca20 2023-11-06T02:08:44.875 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.878+0000 7f4963dff640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f495c0b1060 con 0x7f495c0b0690 2023-11-06T02:08:44.875 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.878+0000 7f496139a640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f495c0b0690 0x7f495c0b0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:44.875 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.878+0000 7f496139a640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f495c0b0690 0x7f495c0b0b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52542/0 (socket says 172.21.15.49:52542) 2023-11-06T02:08:44.875 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.878+0000 7f496139a640 1 -- 172.21.15.49:0/3121466749 learned_addr learned my addr 172.21.15.49:0/3121466749 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:44.876 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.878+0000 7f496239c640 1 --1- 172.21.15.49:0/3121466749 >> v1:172.21.15.49:6789/0 conn(0x7f495c160680 0x7f495c160ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:54074/0 (socket says 172.21.15.49:54074) 2023-11-06T02:08:44.876 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.878+0000 7f496139a640 1 -- 172.21.15.49:0/3121466749 >> v1:172.21.15.150:6789/0 conn(0x7f495c15ca20 legacy=0x7f495c0cd980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:44.876 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.878+0000 7f496139a640 1 -- 172.21.15.49:0/3121466749 >> v1:172.21.15.49:6789/0 conn(0x7f495c160680 legacy=0x7f495c160ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:44.876 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.878+0000 7f496139a640 1 -- 172.21.15.49:0/3121466749 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f495c161340 con 0x7f495c0b0690 2023-11-06T02:08:44.877 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.878+0000 7f496139a640 1 --2- 172.21.15.49:0/3121466749 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f495c0b0690 0x7f495c0b0b20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4950009870 tx=0x7f495001c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0b781ee7c8c637e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:44.877 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.878+0000 7f4960b99640 1 -- 172.21.15.49:0/3121466749 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49500ab640 con 0x7f495c0b0690 2023-11-06T02:08:44.878 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.878+0000 7f4960b99640 1 -- 172.21.15.49:0/3121466749 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49500a7070 con 0x7f495c0b0690 2023-11-06T02:08:44.878 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.878+0000 7f4960b99640 1 -- 172.21.15.49:0/3121466749 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49500ab9c0 con 0x7f495c0b0690 2023-11-06T02:08:44.878 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.882+0000 7f4963dff640 1 -- 172.21.15.49:0/3121466749 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f495c0b0690 msgr2=0x7f495c0b0b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:44.878 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.882+0000 7f4963dff640 1 --2- 172.21.15.49:0/3121466749 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f495c0b0690 0x7f495c0b0b20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4950009870 tx=0x7f495001c440 comp rx=0 tx=0).stop 2023-11-06T02:08:44.878 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.882+0000 7f4963dff640 1 -- 172.21.15.49:0/3121466749 shutdown_connections 2023-11-06T02:08:44.879 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.882+0000 7f4963dff640 1 --2- 172.21.15.49:0/3121466749 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f495c0b0690 0x7f495c0b0b20 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:44.879 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.882+0000 7f4963dff640 1 -- 172.21.15.49:0/3121466749 >> 172.21.15.49:0/3121466749 conn(0x7f495c0ced00 msgr2=0x7f495c0cf110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:44.879 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.882+0000 7f4963dff640 1 -- 172.21.15.49:0/3121466749 shutdown_connections 2023-11-06T02:08:44.879 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.882+0000 7f4963dff640 1 -- 172.21.15.49:0/3121466749 wait complete. 2023-11-06T02:08:44.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.882+0000 7f4963dff640 1 Processor -- start 2023-11-06T02:08:44.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.882+0000 7f4963dff640 1 -- start start 2023-11-06T02:08:44.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.882+0000 7f4963dff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f495c0b0690 0x7f495c153de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:44.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.882+0000 7f4963dff640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f495c15ca20 0x7f495c0c3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:44.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.882+0000 7f4963dff640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f495c160680 0x7f495c158cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:44.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.882+0000 7f4963dff640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f495c0c7d90 con 0x7f495c0b0690 2023-11-06T02:08:44.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.882+0000 7f4963dff640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f495c0c7180 con 0x7f495c160680 2023-11-06T02:08:44.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.882+0000 7f4963dff640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f495c0c7460 con 0x7f495c15ca20 2023-11-06T02:08:44.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.882+0000 7f4961b9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f495c0b0690 0x7f495c153de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:44.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.882+0000 7f4961b9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f495c0b0690 0x7f495c153de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52546/0 (socket says 172.21.15.49:52546) 2023-11-06T02:08:44.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.882+0000 7f4961b9b640 1 -- 172.21.15.49:0/3289867745 learned_addr learned my addr 172.21.15.49:0/3289867745 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:44.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.882+0000 7f496239c640 1 --2- 172.21.15.49:0/3289867745 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f495c160680 0x7f495c158cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:44.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.882+0000 7f496139a640 1 --2- 172.21.15.49:0/3289867745 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f495c15ca20 0x7f495c0c3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:44.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.882+0000 7f4961b9b640 1 -- 172.21.15.49:0/3289867745 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f495c15ca20 msgr2=0x7f495c0c3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:44.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.882+0000 7f4961b9b640 1 --2- 172.21.15.49:0/3289867745 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f495c15ca20 0x7f495c0c3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:44.885 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.882+0000 7f4961b9b640 1 -- 172.21.15.49:0/3289867745 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f495c160680 msgr2=0x7f495c158cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:44.885 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.882+0000 7f4961b9b640 1 --2- 172.21.15.49:0/3289867745 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f495c160680 0x7f495c158cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:44.885 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.882+0000 7f4961b9b640 1 -- 172.21.15.49:0/3289867745 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49500094d0 con 0x7f495c0b0690 2023-11-06T02:08:44.885 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.882+0000 7f496139a640 1 --2- 172.21.15.49:0/3289867745 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f495c15ca20 0x7f495c0c3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:08:44.885 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.882+0000 7f496239c640 1 --2- 172.21.15.49:0/3289867745 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f495c160680 0x7f495c158cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:08:44.885 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.882+0000 7f4961b9b640 1 --2- 172.21.15.49:0/3289867745 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f495c0b0690 0x7f495c153de0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f495400c450 tx=0x7f495400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:44.886 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.882+0000 7f495a7fc640 1 -- 172.21.15.49:0/3289867745 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4954012830 con 0x7f495c0b0690 2023-11-06T02:08:44.886 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.882+0000 7f495a7fc640 1 -- 172.21.15.49:0/3289867745 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49540129b0 con 0x7f495c0b0690 2023-11-06T02:08:44.886 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.882+0000 7f495a7fc640 1 -- 172.21.15.49:0/3289867745 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4954012d60 con 0x7f495c0b0690 2023-11-06T02:08:44.886 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.882+0000 7f4963dff640 1 -- 172.21.15.49:0/3289867745 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f495c0c7740 con 0x7f495c0b0690 2023-11-06T02:08:44.886 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.882+0000 7f4963dff640 1 -- 172.21.15.49:0/3289867745 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f495c154320 con 0x7f495c0b0690 2023-11-06T02:08:44.887 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.886+0000 7f4963dff640 1 -- 172.21.15.49:0/3289867745 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f495c0ad880 con 0x7f495c0b0690 2023-11-06T02:08:44.887 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.886+0000 7f495a7fc640 1 -- 172.21.15.49:0/3289867745 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f495401c460 con 0x7f495c0b0690 2023-11-06T02:08:44.887 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.886+0000 7f495a7fc640 1 --2- 172.21.15.49:0/3289867745 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4948065a60 0x7f4948067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:44.887 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.886+0000 7f495a7fc640 1 -- 172.21.15.49:0/3289867745 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f49540830d0 con 0x7f495c0b0690 2023-11-06T02:08:44.887 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.886+0000 7f496139a640 1 --2- 172.21.15.49:0/3289867745 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4948065a60 0x7f4948067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:44.888 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.886+0000 7f496139a640 1 --2- 172.21.15.49:0/3289867745 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4948065a60 0x7f4948067f20 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f49500af000 tx=0x7f4950008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:44.888 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:44.890+0000 7f495a7fc640 1 -- 172.21.15.49:0/3289867745 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f49540509c0 con 0x7f495c0b0690 2023-11-06T02:08:45.036 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.038+0000 7f4963dff640 1 -- 172.21.15.49:0/3289867745 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f495c0b0b20 con 0x7f495c0b0690 2023-11-06T02:08:45.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.042+0000 7f495a7fc640 1 -- 172.21.15.49:0/3289867745 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4954053fe0 con 0x7f495c0b0690 2023-11-06T02:08:45.037 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2023-11-06T02:08:45.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.042+0000 7f4963dff640 1 -- 172.21.15.49:0/3289867745 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4948065a60 msgr2=0x7f4948067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:45.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.042+0000 7f4963dff640 1 --2- 172.21.15.49:0/3289867745 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4948065a60 0x7f4948067f20 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f49500af000 tx=0x7f4950008040 comp rx=0 tx=0).stop 2023-11-06T02:08:45.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.042+0000 7f4963dff640 1 -- 172.21.15.49:0/3289867745 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f495c0b0690 msgr2=0x7f495c153de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:45.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.042+0000 7f4963dff640 1 --2- 172.21.15.49:0/3289867745 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f495c0b0690 0x7f495c153de0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f495400c450 tx=0x7f495400c910 comp rx=0 tx=0).stop 2023-11-06T02:08:45.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.042+0000 7f4963dff640 1 -- 172.21.15.49:0/3289867745 shutdown_connections 2023-11-06T02:08:45.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.042+0000 7f4963dff640 1 --2- 172.21.15.49:0/3289867745 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4948065a60 0x7f4948067f20 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:45.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.042+0000 7f4963dff640 1 --2- 172.21.15.49:0/3289867745 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f495c160680 0x7f495c158cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:45.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.042+0000 7f4963dff640 1 --2- 172.21.15.49:0/3289867745 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f495c15ca20 0x7f495c0c3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:45.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.042+0000 7f4963dff640 1 --2- 172.21.15.49:0/3289867745 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f495c0b0690 0x7f495c153de0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:45.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.042+0000 7f4963dff640 1 -- 172.21.15.49:0/3289867745 >> 172.21.15.49:0/3289867745 conn(0x7f495c0ced00 msgr2=0x7f495c160ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:45.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.042+0000 7f4963dff640 1 -- 172.21.15.49:0/3289867745 shutdown_connections 2023-11-06T02:08:45.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.042+0000 7f4963dff640 1 -- 172.21.15.49:0/3289867745 wait complete. 2023-11-06T02:08:45.054 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T02:08:45.172 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.178+0000 7efef65b4640 1 Processor -- start 2023-11-06T02:08:45.173 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.178+0000 7efef65b4640 1 -- start start 2023-11-06T02:08:45.175 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.178+0000 7efef65b4640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efef0070160 0x7efef00705a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:45.176 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.178+0000 7efef65b4640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7efef006c230 0x7efef006c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:45.176 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.178+0000 7efef65b4640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efef00c8c80 con 0x7efef014ffc0 2023-11-06T02:08:45.176 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.178+0000 7efef65b4640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efef00ae4d0 con 0x7efef006c230 2023-11-06T02:08:45.176 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.178+0000 7efef65b4640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efef01594f0 con 0x7efef0070160 2023-11-06T02:08:45.176 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.178+0000 7efeeffff640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7efef014ffc0 0x7efef00ada30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:54078/0 (socket says 172.21.15.49:54078) 2023-11-06T02:08:45.176 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.178+0000 7efeeffff640 1 -- 172.21.15.49:0/327637229 learned_addr learned my addr 172.21.15.49:0/327637229 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:08:45.177 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.178+0000 7efeef7fe640 1 --2- 172.21.15.49:0/327637229 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efef0070160 0x7efef00705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:45.177 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.178+0000 7efef4b51640 1 --2- 172.21.15.49:0/327637229 >> v2:172.21.15.150:3300/0 conn(0x7efef006c230 0x7efef006c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:45.177 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.178+0000 7efeeeffd640 1 -- 172.21.15.49:0/327637229 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 573617506 0 0) 0x7efef00c8c80 con 0x7efef014ffc0 2023-11-06T02:08:45.177 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.178+0000 7efeeeffd640 1 -- 172.21.15.49:0/327637229 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efed8003530 con 0x7efef014ffc0 2023-11-06T02:08:45.177 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.178+0000 7efeeeffd640 1 -- 172.21.15.49:0/327637229 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2833375862 0 0) 0x7efed8003530 con 0x7efef014ffc0 2023-11-06T02:08:45.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.178+0000 7efeeeffd640 1 -- 172.21.15.49:0/327637229 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efef0070160 msgr2=0x7efef00705a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:45.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.178+0000 7efeeeffd640 1 --2- 172.21.15.49:0/327637229 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efef0070160 0x7efef00705a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:45.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.178+0000 7efeeeffd640 1 -- 172.21.15.49:0/327637229 >> v2:172.21.15.150:3300/0 conn(0x7efef006c230 msgr2=0x7efef006c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:45.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.178+0000 7efeeeffd640 1 --2- 172.21.15.49:0/327637229 >> v2:172.21.15.150:3300/0 conn(0x7efef006c230 0x7efef006c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:45.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.178+0000 7efeeeffd640 1 -- 172.21.15.49:0/327637229 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efef0070ae0 con 0x7efef014ffc0 2023-11-06T02:08:45.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.178+0000 7efeeeffd640 1 -- 172.21.15.49:0/327637229 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7efee0001a00 con 0x7efef014ffc0 2023-11-06T02:08:45.179 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.178+0000 7efef4b51640 1 --2- 172.21.15.49:0/327637229 >> v2:172.21.15.150:3300/0 conn(0x7efef006c230 0x7efef006c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:08:45.179 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.178+0000 7efeeeffd640 1 -- 172.21.15.49:0/327637229 >> v1:172.21.15.49:6789/0 conn(0x7efef014ffc0 legacy=0x7efef00ada30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:45.179 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.178+0000 7efeeeffd640 1 --2- 172.21.15.49:0/327637229 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efed8003980 0x7efed8003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:45.179 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.178+0000 7efeeeffd640 1 --2- 172.21.15.49:0/327637229 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efed80047d0 0x7efed8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:45.179 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.178+0000 7efeeeffd640 1 --2- 172.21.15.49:0/327637229 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efed8005190 0x7efed800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:45.179 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.178+0000 7efeeeffd640 1 -- 172.21.15.49:0/327637229 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef00ae4d0 con 0x7efed8005190 2023-11-06T02:08:45.180 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.178+0000 7efeeeffd640 1 -- 172.21.15.49:0/327637229 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef01594f0 con 0x7efed80047d0 2023-11-06T02:08:45.180 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.178+0000 7efeeeffd640 1 -- 172.21.15.49:0/327637229 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efed800bfd0 con 0x7efed8003980 2023-11-06T02:08:45.180 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.178+0000 7efeef7fe640 1 --2- 172.21.15.49:0/327637229 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efef0070160 0x7efef00705a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:08:45.180 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.178+0000 7efef4b51640 1 --2- 172.21.15.49:0/327637229 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efed8005190 0x7efed800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:45.180 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.178+0000 7efef4b51640 1 -- 172.21.15.49:0/327637229 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efed8003980 msgr2=0x7efed8003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:45.181 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.178+0000 7efef4b51640 1 --2- 172.21.15.49:0/327637229 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efed8003980 0x7efed8003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:45.181 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.178+0000 7efef4b51640 1 -- 172.21.15.49:0/327637229 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efed80047d0 msgr2=0x7efed8004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:45.181 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.178+0000 7efef4b51640 1 --2- 172.21.15.49:0/327637229 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efed80047d0 0x7efed8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:45.181 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.178+0000 7efef4b51640 1 -- 172.21.15.49:0/327637229 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efed8003530 con 0x7efed8005190 2023-11-06T02:08:45.184 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.182+0000 7efef4b51640 1 --2- 172.21.15.49:0/327637229 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efed8005190 0x7efed800b990 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7efee8009900 tx=0x7efee801c610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b48d631a0a97b9e8 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:45.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.182+0000 7efeeeffd640 1 -- 172.21.15.49:0/327637229 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee80a6620 con 0x7efed8005190 2023-11-06T02:08:45.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.182+0000 7efeeeffd640 1 -- 172.21.15.49:0/327637229 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efee80a6780 con 0x7efed8005190 2023-11-06T02:08:45.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.182+0000 7efeeeffd640 1 -- 172.21.15.49:0/327637229 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee80a6af0 con 0x7efed8005190 2023-11-06T02:08:45.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.182+0000 7efef65b4640 1 -- 172.21.15.49:0/327637229 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efed8005190 msgr2=0x7efed800b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:45.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.182+0000 7efef65b4640 1 --2- 172.21.15.49:0/327637229 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efed8005190 0x7efed800b990 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7efee8009900 tx=0x7efee801c610 comp rx=0 tx=0).stop 2023-11-06T02:08:45.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.182+0000 7efef4b51640 1 -- 172.21.15.49:0/327637229 reap_dead start 2023-11-06T02:08:45.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.182+0000 7efef65b4640 1 -- 172.21.15.49:0/327637229 shutdown_connections 2023-11-06T02:08:45.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.182+0000 7efef65b4640 1 -- 172.21.15.49:0/327637229 >> 172.21.15.49:0/327637229 conn(0x7efef006a370 msgr2=0x7efef00ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:45.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.182+0000 7efef65b4640 1 -- 172.21.15.49:0/327637229 shutdown_connections 2023-11-06T02:08:45.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.182+0000 7efef65b4640 1 -- 172.21.15.49:0/327637229 wait complete. 2023-11-06T02:08:45.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.182+0000 7efef65b4640 1 Processor -- start 2023-11-06T02:08:45.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.182+0000 7efef65b4640 1 -- start start 2023-11-06T02:08:45.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.182+0000 7efef65b4640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efed8005190 0x7efef014eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:45.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.182+0000 7efef65b4640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efef014acc0 0x7efef014f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:45.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.182+0000 7efef65b4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efef0149740 0x7efef00cd440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:45.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.182+0000 7efef65b4640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef014a830 con 0x7efef0149740 2023-11-06T02:08:45.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.182+0000 7efef65b4640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef0149c20 con 0x7efef014acc0 2023-11-06T02:08:45.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.182+0000 7efef65b4640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efef0149f00 con 0x7efed8005190 2023-11-06T02:08:45.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.182+0000 7efef4b51640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efef0149740 0x7efef00cd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:45.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.182+0000 7efef4b51640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efef0149740 0x7efef00cd440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52570/0 (socket says 172.21.15.49:52570) 2023-11-06T02:08:45.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.182+0000 7efef4b51640 1 -- 172.21.15.49:0/1230085330 learned_addr learned my addr 172.21.15.49:0/1230085330 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:45.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.182+0000 7efeef7fe640 1 --2- 172.21.15.49:0/1230085330 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efef014acc0 0x7efef014f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:45.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.182+0000 7efeeffff640 1 --2- 172.21.15.49:0/1230085330 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efed8005190 0x7efef014eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:45.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.182+0000 7efef4b51640 1 -- 172.21.15.49:0/1230085330 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efed8005190 msgr2=0x7efef014eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:45.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.182+0000 7efef4b51640 1 --2- 172.21.15.49:0/1230085330 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efed8005190 0x7efef014eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:45.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.182+0000 7efef4b51640 1 -- 172.21.15.49:0/1230085330 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efef014acc0 msgr2=0x7efef014f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:45.229 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.182+0000 7efef4b51640 1 --2- 172.21.15.49:0/1230085330 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efef014acc0 0x7efef014f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:45.229 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.182+0000 7efef4b51640 1 -- 172.21.15.49:0/1230085330 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efef014a1e0 con 0x7efef0149740 2023-11-06T02:08:45.229 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.186+0000 7efef4b51640 1 --2- 172.21.15.49:0/1230085330 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efef0149740 0x7efef00cd440 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7efee8009900 tx=0x7efed8005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:45.229 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.186+0000 7efeecff9640 1 -- 172.21.15.49:0/1230085330 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee80b6070 con 0x7efef0149740 2023-11-06T02:08:45.229 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.186+0000 7efeecff9640 1 -- 172.21.15.49:0/1230085330 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efee8004020 con 0x7efef0149740 2023-11-06T02:08:45.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.186+0000 7efef65b4640 1 -- 172.21.15.49:0/1230085330 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efef014a4c0 con 0x7efef0149740 2023-11-06T02:08:45.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.186+0000 7efeecff9640 1 -- 172.21.15.49:0/1230085330 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee8004390 con 0x7efef0149740 2023-11-06T02:08:45.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.186+0000 7efef65b4640 1 -- 172.21.15.49:0/1230085330 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efef00cdb50 con 0x7efef0149740 2023-11-06T02:08:45.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.186+0000 7efef65b4640 1 -- 172.21.15.49:0/1230085330 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efef014a650 con 0x7efef0149740 2023-11-06T02:08:45.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.186+0000 7efeecff9640 1 -- 172.21.15.49:0/1230085330 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7efee8003620 con 0x7efef0149740 2023-11-06T02:08:45.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.186+0000 7efeecff9640 1 --2- 172.21.15.49:0/1230085330 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7efedc065a60 0x7efedc067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:45.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.186+0000 7efeecff9640 1 -- 172.21.15.49:0/1230085330 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7efee811d210 con 0x7efef0149740 2023-11-06T02:08:45.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.186+0000 7efeeffff640 1 --2- 172.21.15.49:0/1230085330 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7efedc065a60 0x7efedc067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:45.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.186+0000 7efeeffff640 1 --2- 172.21.15.49:0/1230085330 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7efedc065a60 0x7efedc067f20 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7efee0001b30 tx=0x7efee00019e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:45.232 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.190+0000 7efeecff9640 1 -- 172.21.15.49:0/1230085330 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7efee80ea9e0 con 0x7efef0149740 2023-11-06T02:08:45.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.338+0000 7efef65b4640 1 -- 172.21.15.49:0/1230085330 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7efef0074040 con 0x7efef0149740 2023-11-06T02:08:45.334 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.338+0000 7efeecff9640 1 -- 172.21.15.49:0/1230085330 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7efee80ac0a0 con 0x7efef0149740 2023-11-06T02:08:45.335 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2023-11-06T02:08:45.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.338+0000 7efef65b4640 1 -- 172.21.15.49:0/1230085330 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7efedc065a60 msgr2=0x7efedc067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:45.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.338+0000 7efef65b4640 1 --2- 172.21.15.49:0/1230085330 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7efedc065a60 0x7efedc067f20 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7efee0001b30 tx=0x7efee00019e0 comp rx=0 tx=0).stop 2023-11-06T02:08:45.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.338+0000 7efef65b4640 1 -- 172.21.15.49:0/1230085330 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efef0149740 msgr2=0x7efef00cd440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:45.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.338+0000 7efef65b4640 1 --2- 172.21.15.49:0/1230085330 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efef0149740 0x7efef00cd440 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7efee8009900 tx=0x7efed8005800 comp rx=0 tx=0).stop 2023-11-06T02:08:45.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.338+0000 7efef65b4640 1 -- 172.21.15.49:0/1230085330 shutdown_connections 2023-11-06T02:08:45.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.338+0000 7efef65b4640 1 --2- 172.21.15.49:0/1230085330 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7efedc065a60 0x7efedc067f20 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:45.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.338+0000 7efef65b4640 1 --2- 172.21.15.49:0/1230085330 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efef0149740 0x7efef00cd440 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:45.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.338+0000 7efef65b4640 1 --2- 172.21.15.49:0/1230085330 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efef014acc0 0x7efef014f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:45.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.338+0000 7efef65b4640 1 --2- 172.21.15.49:0/1230085330 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efed8005190 0x7efef014eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:45.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.338+0000 7efef65b4640 1 -- 172.21.15.49:0/1230085330 >> 172.21.15.49:0/1230085330 conn(0x7efef006a370 msgr2=0x7efef00b0b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:45.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.338+0000 7efef65b4640 1 -- 172.21.15.49:0/1230085330 shutdown_connections 2023-11-06T02:08:45.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.338+0000 7efef65b4640 1 -- 172.21.15.49:0/1230085330 wait complete. 2023-11-06T02:08:45.351 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T02:08:45.469 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.474+0000 7f7b65d98640 1 Processor -- start 2023-11-06T02:08:45.469 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.474+0000 7f7b65d98640 1 -- start start 2023-11-06T02:08:45.472 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.474+0000 7f7b65d98640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7b6015cc50 0x7f7b600701e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:45.472 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.474+0000 7f7b65d98640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b6015d030 con 0x7f7b600c9690 2023-11-06T02:08:45.472 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.474+0000 7f7b65d98640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b600cc9c0 con 0x7f7b600b0080 2023-11-06T02:08:45.473 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.474+0000 7f7b65d98640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b600ccc60 con 0x7f7b6015cc50 2023-11-06T02:08:45.473 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.474+0000 7f7b5ffff640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f7b600c9690 0x7f7b600cd740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:54082/0 (socket says 172.21.15.49:54082) 2023-11-06T02:08:45.473 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.474+0000 7f7b5ffff640 1 -- 172.21.15.49:0/2676062430 learned_addr learned my addr 172.21.15.49:0/2676062430 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:08:45.473 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.474+0000 7f7b5f7fe640 1 --2- 172.21.15.49:0/2676062430 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7b6015cc50 0x7f7b600701e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:45.474 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.474+0000 7f7b5e7fc640 1 -- 172.21.15.49:0/2676062430 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2980303588 0 0) 0x7f7b6015d030 con 0x7f7b600c9690 2023-11-06T02:08:45.474 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.474+0000 7f7b5e7fc640 1 -- 172.21.15.49:0/2676062430 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b4c003530 con 0x7f7b600c9690 2023-11-06T02:08:45.474 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.474+0000 7f7b5e7fc640 1 -- 172.21.15.49:0/2676062430 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 364879961 0 0) 0x7f7b4c003530 con 0x7f7b600c9690 2023-11-06T02:08:45.474 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.474+0000 7f7b5e7fc640 1 -- 172.21.15.49:0/2676062430 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7b6015cc50 msgr2=0x7f7b600701e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:45.474 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.474+0000 7f7b5e7fc640 1 --2- 172.21.15.49:0/2676062430 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7b6015cc50 0x7f7b600701e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:45.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.474+0000 7f7b5e7fc640 1 -- 172.21.15.49:0/2676062430 >> v1:172.21.15.150:6789/0 conn(0x7f7b600b0080 legacy=0x7f7b600c8f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:45.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.474+0000 7f7b5e7fc640 1 -- 172.21.15.49:0/2676062430 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b600ccf40 con 0x7f7b600c9690 2023-11-06T02:08:45.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.474+0000 7f7b5e7fc640 1 -- 172.21.15.49:0/2676062430 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7f7b58001680 con 0x7f7b600c9690 2023-11-06T02:08:45.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.474+0000 7f7b5e7fc640 1 -- 172.21.15.49:0/2676062430 >> v1:172.21.15.49:6789/0 conn(0x7f7b600c9690 legacy=0x7f7b600cd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:45.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.474+0000 7f7b5e7fc640 1 --2- 172.21.15.49:0/2676062430 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b4c0039b0 0x7f7b4c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:45.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.474+0000 7f7b5e7fc640 1 --2- 172.21.15.49:0/2676062430 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b4c0047d0 0x7f7b4c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:45.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.474+0000 7f7b5e7fc640 1 --2- 172.21.15.49:0/2676062430 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7b6015cc50 0x7f7b4c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:45.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.474+0000 7f7b5e7fc640 1 -- 172.21.15.49:0/2676062430 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b600ccc60 con 0x7f7b4c0039b0 2023-11-06T02:08:45.477 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.474+0000 7f7b5e7fc640 1 -- 172.21.15.49:0/2676062430 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b4c00bac0 con 0x7f7b4c0047d0 2023-11-06T02:08:45.477 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.474+0000 7f7b5e7fc640 1 -- 172.21.15.49:0/2676062430 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b4c00bc80 con 0x7f7b6015cc50 2023-11-06T02:08:45.477 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.474+0000 7f7b5f7fe640 1 --2- 172.21.15.49:0/2676062430 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b4c0039b0 0x7f7b4c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:45.478 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.474+0000 7f7b5effd640 1 --2- 172.21.15.49:0/2676062430 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b4c0047d0 0x7f7b4c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:45.478 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.474+0000 7f7b5ffff640 1 --2- 172.21.15.49:0/2676062430 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7b6015cc50 0x7f7b4c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:45.478 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.474+0000 7f7b5f7fe640 1 -- 172.21.15.49:0/2676062430 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7b6015cc50 msgr2=0x7f7b4c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:45.479 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.474+0000 7f7b5f7fe640 1 --2- 172.21.15.49:0/2676062430 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7b6015cc50 0x7f7b4c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:45.479 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.474+0000 7f7b5f7fe640 1 -- 172.21.15.49:0/2676062430 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b4c0047d0 msgr2=0x7f7b4c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:45.479 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.474+0000 7f7b5f7fe640 1 --2- 172.21.15.49:0/2676062430 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b4c0047d0 0x7f7b4c004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:45.479 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.474+0000 7f7b5f7fe640 1 -- 172.21.15.49:0/2676062430 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b4c003530 con 0x7f7b4c0039b0 2023-11-06T02:08:45.485 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.478+0000 7f7b5f7fe640 1 --2- 172.21.15.49:0/2676062430 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b4c0039b0 0x7f7b4c003dd0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7b5000e170 tx=0x7f7b5001cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d800b282cb96388 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:45.485 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.478+0000 7f7b5e7fc640 1 -- 172.21.15.49:0/2676062430 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b500a5c00 con 0x7f7b4c0039b0 2023-11-06T02:08:45.485 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.478+0000 7f7b5e7fc640 1 -- 172.21.15.49:0/2676062430 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b5000f040 con 0x7f7b4c0039b0 2023-11-06T02:08:45.486 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.478+0000 7f7b5e7fc640 1 -- 172.21.15.49:0/2676062430 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b500ad440 con 0x7f7b4c0039b0 2023-11-06T02:08:45.486 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.478+0000 7f7b65d98640 1 -- 172.21.15.49:0/2676062430 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b4c0039b0 msgr2=0x7f7b4c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:45.486 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.478+0000 7f7b65d98640 1 --2- 172.21.15.49:0/2676062430 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b4c0039b0 0x7f7b4c003dd0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7b5000e170 tx=0x7f7b5001cc70 comp rx=0 tx=0).stop 2023-11-06T02:08:45.487 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.478+0000 7f7b5ffff640 1 -- 172.21.15.49:0/2676062430 reap_dead start 2023-11-06T02:08:45.487 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.478+0000 7f7b65d98640 1 -- 172.21.15.49:0/2676062430 shutdown_connections 2023-11-06T02:08:45.487 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.478+0000 7f7b65d98640 1 -- 172.21.15.49:0/2676062430 >> 172.21.15.49:0/2676062430 conn(0x7f7b6006c0a0 msgr2=0x7f7b6006c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:45.488 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.478+0000 7f7b65d98640 1 -- 172.21.15.49:0/2676062430 shutdown_connections 2023-11-06T02:08:45.488 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.478+0000 7f7b65d98640 1 -- 172.21.15.49:0/2676062430 wait complete. 2023-11-06T02:08:45.488 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.478+0000 7f7b65d98640 1 Processor -- start 2023-11-06T02:08:45.488 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.478+0000 7f7b65d98640 1 -- start start 2023-11-06T02:08:45.489 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.478+0000 7f7b65d98640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b4c0039b0 0x7f7b6015fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:45.489 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.478+0000 7f7b65d98640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b600c4550 0x7f7b601f48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:45.489 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.478+0000 7f7b65d98640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7b601f4e00 0x7f7b6015d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:45.490 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.478+0000 7f7b65d98640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b60160140 con 0x7f7b4c0039b0 2023-11-06T02:08:45.490 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.478+0000 7f7b65d98640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b600c40c0 con 0x7f7b600c4550 2023-11-06T02:08:45.490 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.478+0000 7f7b65d98640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b600c34b0 con 0x7f7b601f4e00 2023-11-06T02:08:45.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.478+0000 7f7b5f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b4c0039b0 0x7f7b6015fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:45.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.478+0000 7f7b5f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b4c0039b0 0x7f7b6015fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52594/0 (socket says 172.21.15.49:52594) 2023-11-06T02:08:45.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.478+0000 7f7b5f7fe640 1 -- 172.21.15.49:0/243002529 learned_addr learned my addr 172.21.15.49:0/243002529 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:45.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.478+0000 7f7b5effd640 1 --2- 172.21.15.49:0/243002529 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b600c4550 0x7f7b601f48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:45.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.478+0000 7f7b5ffff640 1 --2- 172.21.15.49:0/243002529 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7b601f4e00 0x7f7b6015d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:45.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.482+0000 7f7b5f7fe640 1 -- 172.21.15.49:0/243002529 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7b601f4e00 msgr2=0x7f7b6015d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:45.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.482+0000 7f7b5f7fe640 1 --2- 172.21.15.49:0/243002529 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7b601f4e00 0x7f7b6015d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:45.493 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.482+0000 7f7b5f7fe640 1 -- 172.21.15.49:0/243002529 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b600c4550 msgr2=0x7f7b601f48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:45.493 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.482+0000 7f7b5f7fe640 1 --2- 172.21.15.49:0/243002529 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b600c4550 0x7f7b601f48c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:45.493 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.482+0000 7f7b5f7fe640 1 -- 172.21.15.49:0/243002529 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b600c3790 con 0x7f7b4c0039b0 2023-11-06T02:08:45.494 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.482+0000 7f7b5ffff640 1 --2- 172.21.15.49:0/243002529 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7b601f4e00 0x7f7b6015d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:08:45.494 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.482+0000 7f7b5f7fe640 1 --2- 172.21.15.49:0/243002529 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b4c0039b0 0x7f7b6015fc00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7b5001caa0 tx=0x7f7b50009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:45.494 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.482+0000 7f7b57fff640 1 -- 172.21.15.49:0/243002529 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b500ad780 con 0x7f7b4c0039b0 2023-11-06T02:08:45.495 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.482+0000 7f7b57fff640 1 -- 172.21.15.49:0/243002529 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b500ad8e0 con 0x7f7b4c0039b0 2023-11-06T02:08:45.495 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.482+0000 7f7b65d98640 1 -- 172.21.15.49:0/243002529 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b600c3a70 con 0x7f7b4c0039b0 2023-11-06T02:08:45.495 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.482+0000 7f7b57fff640 1 -- 172.21.15.49:0/243002529 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b500adc90 con 0x7f7b4c0039b0 2023-11-06T02:08:45.495 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.482+0000 7f7b65d98640 1 -- 172.21.15.49:0/243002529 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b600c49d0 con 0x7f7b4c0039b0 2023-11-06T02:08:45.495 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.482+0000 7f7b65d98640 1 -- 172.21.15.49:0/243002529 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b60074040 con 0x7f7b4c0039b0 2023-11-06T02:08:45.496 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.482+0000 7f7b57fff640 1 -- 172.21.15.49:0/243002529 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f7b500b4040 con 0x7f7b4c0039b0 2023-11-06T02:08:45.496 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.482+0000 7f7b57fff640 1 --2- 172.21.15.49:0/243002529 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7b40065a60 0x7f7b40067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:45.496 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.482+0000 7f7b57fff640 1 -- 172.21.15.49:0/243002529 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7b5011d920 con 0x7f7b4c0039b0 2023-11-06T02:08:45.496 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.482+0000 7f7b5effd640 1 --2- 172.21.15.49:0/243002529 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7b40065a60 0x7f7b40067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:45.496 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.482+0000 7f7b5effd640 1 --2- 172.21.15.49:0/243002529 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7b40065a60 0x7f7b40067f20 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7b601f8730 tx=0x7f7b48009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:45.497 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.486+0000 7f7b57fff640 1 -- 172.21.15.49:0/243002529 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f7b500eb330 con 0x7f7b4c0039b0 2023-11-06T02:08:45.673 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.678+0000 7f7b65d98640 1 -- 172.21.15.49:0/243002529 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7b600b4580 con 0x7f7b4c0039b0 2023-11-06T02:08:45.674 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.678+0000 7f7b57fff640 1 -- 172.21.15.49:0/243002529 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4648 (secure 0 0 0) 0x7f7b500b2090 con 0x7f7b4c0039b0 2023-11-06T02:08:45.675 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:08:45.675 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6834/1377586971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1377586971},{"type":"v1","addr":"172.21.15.49:6834","nonce":1377586971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4288,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2620584959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2620584959},{"type":"v1","addr":"172.21.15.150:6835","nonce":2620584959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4291,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/4272191418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":4272191418},{"type":"v1","addr":"172.21.15.150:6837","nonce":4272191418}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T02:08:30.592185+0000","modified":"2023-11-06T02:08:33.245446+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4293},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4293":{"gid":4293,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.49:6835/4280448297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6833","nonce":4280448297},{"type":"v1","addr":"172.21.15.49:6835","nonce":4280448297}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-06T02:08:45.675 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 7 2023-11-06T02:08:45.677 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.682+0000 7f7b65d98640 1 -- 172.21.15.49:0/243002529 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7b40065a60 msgr2=0x7f7b40067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:45.677 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.682+0000 7f7b65d98640 1 --2- 172.21.15.49:0/243002529 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7b40065a60 0x7f7b40067f20 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7b601f8730 tx=0x7f7b48009040 comp rx=0 tx=0).stop 2023-11-06T02:08:45.677 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.682+0000 7f7b65d98640 1 -- 172.21.15.49:0/243002529 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b4c0039b0 msgr2=0x7f7b6015fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:45.677 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.682+0000 7f7b65d98640 1 --2- 172.21.15.49:0/243002529 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b4c0039b0 0x7f7b6015fc00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7b5001caa0 tx=0x7f7b50009750 comp rx=0 tx=0).stop 2023-11-06T02:08:45.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.682+0000 7f7b65d98640 1 -- 172.21.15.49:0/243002529 shutdown_connections 2023-11-06T02:08:45.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.682+0000 7f7b65d98640 1 --2- 172.21.15.49:0/243002529 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7b40065a60 0x7f7b40067f20 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:45.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.682+0000 7f7b65d98640 1 --2- 172.21.15.49:0/243002529 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7b601f4e00 0x7f7b6015d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:45.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.682+0000 7f7b65d98640 1 --2- 172.21.15.49:0/243002529 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b600c4550 0x7f7b601f48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:45.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.682+0000 7f7b65d98640 1 --2- 172.21.15.49:0/243002529 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b4c0039b0 0x7f7b6015fc00 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:45.679 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.682+0000 7f7b65d98640 1 -- 172.21.15.49:0/243002529 >> 172.21.15.49:0/243002529 conn(0x7f7b6006c0a0 msgr2=0x7f7b60160d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:45.679 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.682+0000 7f7b65d98640 1 -- 172.21.15.49:0/243002529 shutdown_connections 2023-11-06T02:08:45.679 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.682+0000 7f7b65d98640 1 -- 172.21.15.49:0/243002529 wait complete. 2023-11-06T02:08:45.694 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T02:08:45.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.810+0000 7f6dedd60640 1 Processor -- start 2023-11-06T02:08:45.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.810+0000 7f6dedd60640 1 -- start start 2023-11-06T02:08:45.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.814+0000 7f6dedd60640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f6de8159980 0x7f6de8071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:45.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.814+0000 7f6dedd60640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f6de80b1450 0x7f6de80cdd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:45.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.814+0000 7f6dedd60640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6de8071890 con 0x7f6de80cb010 2023-11-06T02:08:45.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.814+0000 7f6dedd60640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6de8072410 con 0x7f6de80b1450 2023-11-06T02:08:45.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.814+0000 7f6dedd60640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6de80b1890 con 0x7f6de8159980 2023-11-06T02:08:45.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.814+0000 7f6de6ffd640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f6de80b1450 0x7f6de80cdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:45.812 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.814+0000 7f6de6ffd640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f6de80b1450 0x7f6de80cdd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52600/0 (socket says 172.21.15.49:52600) 2023-11-06T02:08:45.812 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.814+0000 7f6de6ffd640 1 -- 172.21.15.49:0/3123363791 learned_addr learned my addr 172.21.15.49:0/3123363791 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:45.812 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.814+0000 7f6de77fe640 1 --2- 172.21.15.49:0/3123363791 >> v2:172.21.15.150:3300/0 conn(0x7f6de8159980 0x7f6de8071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:45.812 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.814+0000 7f6de6ffd640 1 -- 172.21.15.49:0/3123363791 >> v2:172.21.15.150:3300/0 conn(0x7f6de8159980 msgr2=0x7f6de8071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:45.812 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.814+0000 7f6de6ffd640 1 --2- 172.21.15.49:0/3123363791 >> v2:172.21.15.150:3300/0 conn(0x7f6de8159980 0x7f6de8071290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:45.813 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.814+0000 7f6de6ffd640 1 -- 172.21.15.49:0/3123363791 >> v1:172.21.15.150:6789/0 conn(0x7f6de80cb010 legacy=0x7f6de80cc540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:45.813 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.814+0000 7f6de6ffd640 1 -- 172.21.15.49:0/3123363791 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6de8071b70 con 0x7f6de80b1450 2023-11-06T02:08:45.814 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.818+0000 7f6de6ffd640 1 --2- 172.21.15.49:0/3123363791 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6de80b1450 0x7f6de80cdd30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6de80cc140 tx=0x7f6dd001c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b31aa0ff66180d43 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:45.814 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.818+0000 7f6de67fc640 1 -- 172.21.15.49:0/3123363791 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd00a7070 con 0x7f6de80b1450 2023-11-06T02:08:45.815 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.818+0000 7f6de67fc640 1 -- 172.21.15.49:0/3123363791 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dd00ab430 con 0x7f6de80b1450 2023-11-06T02:08:45.815 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.818+0000 7f6de67fc640 1 -- 172.21.15.49:0/3123363791 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd00ab7c0 con 0x7f6de80b1450 2023-11-06T02:08:45.815 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.818+0000 7f6dedd60640 1 -- 172.21.15.49:0/3123363791 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6de80b1450 msgr2=0x7f6de80cdd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:45.816 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.818+0000 7f6dedd60640 1 --2- 172.21.15.49:0/3123363791 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6de80b1450 0x7f6de80cdd30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6de80cc140 tx=0x7f6dd001c4c0 comp rx=0 tx=0).stop 2023-11-06T02:08:45.816 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.818+0000 7f6dedd60640 1 -- 172.21.15.49:0/3123363791 shutdown_connections 2023-11-06T02:08:45.816 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.818+0000 7f6dedd60640 1 --2- 172.21.15.49:0/3123363791 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6de80b1450 0x7f6de80cdd30 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:45.816 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.818+0000 7f6dedd60640 1 --2- 172.21.15.49:0/3123363791 >> v2:172.21.15.150:3300/0 conn(0x7f6de8159980 0x7f6de8071290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:45.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.818+0000 7f6dedd60640 1 -- 172.21.15.49:0/3123363791 >> 172.21.15.49:0/3123363791 conn(0x7f6de806c0a0 msgr2=0x7f6de806c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:45.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.818+0000 7f6dedd60640 1 -- 172.21.15.49:0/3123363791 shutdown_connections 2023-11-06T02:08:45.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.818+0000 7f6dedd60640 1 -- 172.21.15.49:0/3123363791 wait complete. 2023-11-06T02:08:45.822 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.818+0000 7f6dedd60640 1 Processor -- start 2023-11-06T02:08:45.822 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.818+0000 7f6dedd60640 1 -- start start 2023-11-06T02:08:45.822 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.818+0000 7f6dedd60640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6de80b1450 0x7f6de8151000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:45.823 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.818+0000 7f6dedd60640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6de80cb010 0x7f6de80c56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:45.823 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.818+0000 7f6dedd60640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6de8159980 0x7f6de80c46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:45.823 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.818+0000 7f6dedd60640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de8151540 con 0x7f6de80b1450 2023-11-06T02:08:45.823 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.818+0000 7f6dedd60640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de80c3350 con 0x7f6de8159980 2023-11-06T02:08:45.824 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.818+0000 7f6dedd60640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6de80c3630 con 0x7f6de80cb010 2023-11-06T02:08:45.824 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.818+0000 7f6de77fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6de80b1450 0x7f6de8151000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:45.824 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.818+0000 7f6de77fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6de80b1450 0x7f6de8151000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52614/0 (socket says 172.21.15.49:52614) 2023-11-06T02:08:45.825 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.822+0000 7f6de77fe640 1 -- 172.21.15.49:0/2832953732 learned_addr learned my addr 172.21.15.49:0/2832953732 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:45.825 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.822+0000 7f6de6ffd640 1 --2- 172.21.15.49:0/2832953732 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6de80cb010 0x7f6de80c56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:45.825 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.822+0000 7f6de7fff640 1 --2- 172.21.15.49:0/2832953732 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6de8159980 0x7f6de80c46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:45.825 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.822+0000 7f6de77fe640 1 -- 172.21.15.49:0/2832953732 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6de80cb010 msgr2=0x7f6de80c56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:45.825 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.822+0000 7f6de77fe640 1 --2- 172.21.15.49:0/2832953732 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6de80cb010 0x7f6de80c56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:45.826 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.822+0000 7f6de77fe640 1 -- 172.21.15.49:0/2832953732 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6de8159980 msgr2=0x7f6de80c46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:45.826 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.822+0000 7f6de77fe640 1 --2- 172.21.15.49:0/2832953732 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6de8159980 0x7f6de80c46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:45.826 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.822+0000 7f6de77fe640 1 -- 172.21.15.49:0/2832953732 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dd00094d0 con 0x7f6de80b1450 2023-11-06T02:08:45.826 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.822+0000 7f6de6ffd640 1 --2- 172.21.15.49:0/2832953732 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6de80cb010 0x7f6de80c56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:08:45.827 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.822+0000 7f6de77fe640 1 --2- 172.21.15.49:0/2832953732 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6de80b1450 0x7f6de8151000 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6dd800d7d0 tx=0x7f6dd800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:45.827 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.822+0000 7f6ddffff640 1 -- 172.21.15.49:0/2832953732 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd800b6c0 con 0x7f6de80b1450 2023-11-06T02:08:45.827 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.822+0000 7f6ddffff640 1 -- 172.21.15.49:0/2832953732 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dd800b820 con 0x7f6de80b1450 2023-11-06T02:08:45.827 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.822+0000 7f6dedd60640 1 -- 172.21.15.49:0/2832953732 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6de80ce3f0 con 0x7f6de80b1450 2023-11-06T02:08:45.827 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.822+0000 7f6ddffff640 1 -- 172.21.15.49:0/2832953732 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd800bbd0 con 0x7f6de80b1450 2023-11-06T02:08:45.827 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.822+0000 7f6dedd60640 1 -- 172.21.15.49:0/2832953732 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6de80c3bf0 con 0x7f6de80b1450 2023-11-06T02:08:45.828 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.822+0000 7f6dedd60640 1 -- 172.21.15.49:0/2832953732 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6de8074040 con 0x7f6de80b1450 2023-11-06T02:08:45.828 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.822+0000 7f6ddffff640 1 -- 172.21.15.49:0/2832953732 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f6dd800bd30 con 0x7f6de80b1450 2023-11-06T02:08:45.828 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.822+0000 7f6ddffff640 1 --2- 172.21.15.49:0/2832953732 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6dc8065a60 0x7f6dc8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:45.828 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.822+0000 7f6ddffff640 1 -- 172.21.15.49:0/2832953732 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6dd80837d0 con 0x7f6de80b1450 2023-11-06T02:08:45.828 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.822+0000 7f6de6ffd640 1 --2- 172.21.15.49:0/2832953732 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6dc8065a60 0x7f6dc8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:45.829 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.822+0000 7f6de6ffd640 1 --2- 172.21.15.49:0/2832953732 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6dc8065a60 0x7f6dc8067f20 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6dd00af000 tx=0x7f6dd0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:45.829 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.826+0000 7f6ddffff640 1 -- 172.21.15.49:0/2832953732 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f6dd8051030 con 0x7f6de80b1450 2023-11-06T02:08:45.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.982+0000 7f6dedd60640 1 -- 172.21.15.49:0/2832953732 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6de8156930 con 0x7f6de80b1450 2023-11-06T02:08:45.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.982+0000 7f6ddffff640 1 -- 172.21.15.49:0/2832953732 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+17811 (secure 0 0 0) 0x7f6dd8054650 con 0x7f6de80b1450 2023-11-06T02:08:45.979 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:08:45.979 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":23,"fsid":"1a8b035c-6ba4-499a-b876-aea47fd9c0d4","created":"2023-11-06T02:08:06.440857+0000","modified":"2023-11-06T02:08:31.594964+0000","last_up_change":"2023-11-06T02:08:17.982372+0000","last_in_change":"2023-11-06T02:08:08.906631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T02:08:19.252222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T02:08:24.776211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-06T02:08:28.936178+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-06T02:08:29.563468+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ad55af8-0571-4e6a-8722-b5d429b7496d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6809","nonce":38116},{"type":"v1","addr":"172.21.15.49:6811","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6813","nonce":38116},{"type":"v1","addr":"172.21.15.49:6815","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6821","nonce":38116},{"type":"v1","addr":"172.21.15.49:6823","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6817","nonce":38116},{"type":"v1","addr":"172.21.15.49:6819","nonce":38116}]},"public_addr":"172.21.15.49:6811/38116","cluster_addr":"172.21.15.49:6815/38116","heartbeat_back_addr":"172.21.15.49:6823/38116","heartbeat_front_addr":"172.21.15.49:6819/38116","state":["exists","up"]},{"osd":1,"uuid":"c8b31f85-a757-43ed-9419-351acc46611b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":38113},{"type":"v1","addr":"172.21.15.49:6810","nonce":38113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":38113},{"type":"v1","addr":"172.21.15.49:6814","nonce":38113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":38113},{"type":"v1","addr":"172.21.15.49:6822","nonce":38113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":38113},{"type":"v1","addr":"172.21.15.49:6818","nonce":38113}]},"public_addr":"172.21.15.49:6810/38113","cluster_addr":"172.21.15.49:6814/38113","heartbeat_back_addr":"172.21.15.49:6822/38113","heartbeat_front_addr":"172.21.15.49:6818/38113","state":["exists","up"]},{"osd":2,"uuid":"555b3bf3-0a70-4b91-8efd-e4204b5fc26e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":38114},{"type":"v1","addr":"172.21.15.49:6801","nonce":38114}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":38114},{"type":"v1","addr":"172.21.15.49:6803","nonce":38114}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":38114},{"type":"v1","addr":"172.21.15.49:6807","nonce":38114}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":38114},{"type":"v1","addr":"172.21.15.49:6805","nonce":38114}]},"public_addr":"172.21.15.49:6801/38114","cluster_addr":"172.21.15.49:6803/38114","heartbeat_back_addr":"172.21.15.49:6807/38114","heartbeat_front_addr":"172.21.15.49:6805/38114","state":["exists","up"]},{"osd":3,"uuid":"8d4e5d77-03ef-4b9b-bfcb-5ae93a46795c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":38115},{"type":"v1","addr":"172.21.15.49:6825","nonce":38115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":38115},{"type":"v1","addr":"172.21.15.49:6827","nonce":38115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":38115},{"type":"v1","addr":"172.21.15.49:6831","nonce":38115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":38115},{"type":"v1","addr":"172.21.15.49:6829","nonce":38115}]},"public_addr":"172.21.15.49:6825/38115","cluster_addr":"172.21.15.49:6827/38115","heartbeat_back_addr":"172.21.15.49:6831/38115","heartbeat_front_addr":"172.21.15.49:6829/38115","state":["exists","up"]},{"osd":4,"uuid":"dba03ad4-68a3-4cf3-9218-ac611f02eacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":38417},{"type":"v1","addr":"172.21.15.150:6809","nonce":38417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":38417},{"type":"v1","addr":"172.21.15.150:6811","nonce":38417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":38417},{"type":"v1","addr":"172.21.15.150:6815","nonce":38417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":38417},{"type":"v1","addr":"172.21.15.150:6813","nonce":38417}]},"public_addr":"172.21.15.150:6809/38417","cluster_addr":"172.21.15.150:6811/38417","heartbeat_back_addr":"172.21.15.150:6815/38417","heartbeat_front_addr":"172.21.15.150:6813/38417","state":["exists","up"]},{"osd":5,"uuid":"0df9c303-c255-4640-8e91-01f1c3efe18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":38418},{"type":"v1","addr":"172.21.15.150:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":38418},{"type":"v1","addr":"172.21.15.150:6803","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":38418},{"type":"v1","addr":"172.21.15.150:6807","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":38418},{"type":"v1","addr":"172.21.15.150:6805","nonce":38418}]},"public_addr":"172.21.15.150:6801/38418","cluster_addr":"172.21.15.150:6803/38418","heartbeat_back_addr":"172.21.15.150:6807/38418","heartbeat_front_addr":"172.21.15.150:6805/38418","state":["exists","up"]},{"osd":6,"uuid":"e59fa189-4b70-4c06-9722-c02b90828bf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":38428},{"type":"v1","addr":"172.21.15.150:6825","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":38428},{"type":"v1","addr":"172.21.15.150:6827","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":38428},{"type":"v1","addr":"172.21.15.150:6831","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":38428},{"type":"v1","addr":"172.21.15.150:6829","nonce":38428}]},"public_addr":"172.21.15.150:6825/38428","cluster_addr":"172.21.15.150:6827/38428","heartbeat_back_addr":"172.21.15.150:6831/38428","heartbeat_front_addr":"172.21.15.150:6829/38428","state":["exists","up"]},{"osd":7,"uuid":"a64f57e1-ef98-46bd-b304-385759e95898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":38421},{"type":"v1","addr":"172.21.15.150:6817","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":38421},{"type":"v1","addr":"172.21.15.150:6819","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":38421},{"type":"v1","addr":"172.21.15.150:6823","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":38421},{"type":"v1","addr":"172.21.15.150:6821","nonce":38421}]},"public_addr":"172.21.15.150:6817/38421","cluster_addr":"172.21.15.150:6819/38421","heartbeat_back_addr":"172.21.15.150:6823/38421","heartbeat_front_addr":"172.21.15.150:6821/38421","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.438552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.413280+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.296466+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.424966+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.276953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.230984+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.410235+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.480084+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T02:08:45.980 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.982+0000 7f6dedd60640 1 -- 172.21.15.49:0/2832953732 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6dc8065a60 msgr2=0x7f6dc8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:45.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.982+0000 7f6dedd60640 1 --2- 172.21.15.49:0/2832953732 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6dc8065a60 0x7f6dc8067f20 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6dd00af000 tx=0x7f6dd0008040 comp rx=0 tx=0).stop 2023-11-06T02:08:45.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.982+0000 7f6dedd60640 1 -- 172.21.15.49:0/2832953732 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6de80b1450 msgr2=0x7f6de8151000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:45.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.982+0000 7f6dedd60640 1 --2- 172.21.15.49:0/2832953732 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6de80b1450 0x7f6de8151000 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6dd800d7d0 tx=0x7f6dd800dc90 comp rx=0 tx=0).stop 2023-11-06T02:08:45.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.982+0000 7f6dedd60640 1 -- 172.21.15.49:0/2832953732 shutdown_connections 2023-11-06T02:08:45.982 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.982+0000 7f6dedd60640 1 --2- 172.21.15.49:0/2832953732 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6dc8065a60 0x7f6dc8067f20 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:45.982 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.982+0000 7f6dedd60640 1 --2- 172.21.15.49:0/2832953732 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6de8159980 0x7f6de80c46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:45.983 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.982+0000 7f6dedd60640 1 --2- 172.21.15.49:0/2832953732 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6de80cb010 0x7f6de80c56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:45.983 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.982+0000 7f6dedd60640 1 --2- 172.21.15.49:0/2832953732 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6de80b1450 0x7f6de8151000 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:45.983 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.986+0000 7f6dedd60640 1 -- 172.21.15.49:0/2832953732 >> 172.21.15.49:0/2832953732 conn(0x7f6de806c0a0 msgr2=0x7f6de8156220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:45.983 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.986+0000 7f6dedd60640 1 -- 172.21.15.49:0/2832953732 shutdown_connections 2023-11-06T02:08:45.984 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:45.986+0000 7f6dedd60640 1 -- 172.21.15.49:0/2832953732 wait complete. 2023-11-06T02:08:45.997 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T02:08:19.252222+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T02:08:24.776211+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T02:08:28.936178+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T02:08:29.563468+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T02:08:45.998 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T02:08:46.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.130+0000 7f61e314c640 1 Processor -- start 2023-11-06T02:08:46.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.130+0000 7f61e314c640 1 -- start start 2023-11-06T02:08:46.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.130+0000 7f61e314c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f61dc0cda10 0x7f61dc0cf5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:46.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.130+0000 7f61e314c640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61dc0d2810 con 0x7f61dc0cf1f0 2023-11-06T02:08:46.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.130+0000 7f61e314c640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61dc0d1b10 con 0x7f61dc0d1fa0 2023-11-06T02:08:46.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.130+0000 7f61e314c640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61dc0d0f70 con 0x7f61dc0cda10 2023-11-06T02:08:46.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.130+0000 7f61e0ee8640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f61dc0cf1f0 0x7f61dc15dc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:54092/0 (socket says 172.21.15.49:54092) 2023-11-06T02:08:46.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.130+0000 7f61e0ee8640 1 -- 172.21.15.49:0/3180591901 learned_addr learned my addr 172.21.15.49:0/3180591901 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:08:46.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.130+0000 7f61dbfff640 1 --2- 172.21.15.49:0/3180591901 >> v2:172.21.15.49:3300/0 conn(0x7f61dc0cda10 0x7f61dc0cf5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:46.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.130+0000 7f61db7fe640 1 -- 172.21.15.49:0/3180591901 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2793341542 0 0) 0x7f61dc0d2810 con 0x7f61dc0cf1f0 2023-11-06T02:08:46.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.130+0000 7f61db7fe640 1 -- 172.21.15.49:0/3180591901 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61c8003530 con 0x7f61dc0cf1f0 2023-11-06T02:08:46.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.130+0000 7f61dbfff640 1 -- 172.21.15.49:0/3180591901 >> v1:172.21.15.150:6789/0 conn(0x7f61dc0d1fa0 legacy=0x7f61dc15e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:46.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.130+0000 7f61dbfff640 1 -- 172.21.15.49:0/3180591901 >> v1:172.21.15.49:6789/0 conn(0x7f61dc0cf1f0 legacy=0x7f61dc15dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:46.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.130+0000 7f61dbfff640 1 -- 172.21.15.49:0/3180591901 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61dc0d11e0 con 0x7f61dc0cda10 2023-11-06T02:08:46.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.134+0000 7f61dbfff640 1 --2- 172.21.15.49:0/3180591901 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61dc0cda10 0x7f61dc0cf5d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f61c40097f0 tx=0x7f61c401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cab03347412f87eb server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:46.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.134+0000 7f61db7fe640 1 -- 172.21.15.49:0/3180591901 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c40ad660 con 0x7f61dc0cda10 2023-11-06T02:08:46.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.134+0000 7f61db7fe640 1 -- 172.21.15.49:0/3180591901 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61c40ad7c0 con 0x7f61dc0cda10 2023-11-06T02:08:46.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.134+0000 7f61db7fe640 1 -- 172.21.15.49:0/3180591901 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c40adb50 con 0x7f61dc0cda10 2023-11-06T02:08:46.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.134+0000 7f61e314c640 1 -- 172.21.15.49:0/3180591901 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61dc0cda10 msgr2=0x7f61dc0cf5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:46.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.134+0000 7f61e314c640 1 --2- 172.21.15.49:0/3180591901 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61dc0cda10 0x7f61dc0cf5d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f61c40097f0 tx=0x7f61c401c450 comp rx=0 tx=0).stop 2023-11-06T02:08:46.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.134+0000 7f61e314c640 1 -- 172.21.15.49:0/3180591901 shutdown_connections 2023-11-06T02:08:46.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.134+0000 7f61e314c640 1 --2- 172.21.15.49:0/3180591901 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61dc0cda10 0x7f61dc0cf5d0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:46.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.134+0000 7f61e314c640 1 -- 172.21.15.49:0/3180591901 >> 172.21.15.49:0/3180591901 conn(0x7f61dc071aa0 msgr2=0x7f61dc06b110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:46.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.134+0000 7f61e314c640 1 -- 172.21.15.49:0/3180591901 shutdown_connections 2023-11-06T02:08:46.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.134+0000 7f61e314c640 1 -- 172.21.15.49:0/3180591901 wait complete. 2023-11-06T02:08:46.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.134+0000 7f61e314c640 1 Processor -- start 2023-11-06T02:08:46.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.134+0000 7f61e314c640 1 -- start start 2023-11-06T02:08:46.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.134+0000 7f61e314c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61dc0cda10 0x7f61dc149740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:46.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.134+0000 7f61e314c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61dc0cf1f0 0x7f61dc14c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:46.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.134+0000 7f61e314c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61dc0d1fa0 0x7f61dc148b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:46.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.134+0000 7f61e314c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61dc14b4d0 con 0x7f61dc0cf1f0 2023-11-06T02:08:46.143 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.134+0000 7f61e314c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61dc14b7b0 con 0x7f61dc0cda10 2023-11-06T02:08:46.143 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.134+0000 7f61e314c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61dc14bb20 con 0x7f61dc0d1fa0 2023-11-06T02:08:46.143 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.134+0000 7f61dbfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61dc0cf1f0 0x7f61dc14c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:46.143 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.134+0000 7f61dbfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61dc0cf1f0 0x7f61dc14c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52624/0 (socket says 172.21.15.49:52624) 2023-11-06T02:08:46.144 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.134+0000 7f61dbfff640 1 -- 172.21.15.49:0/1845122802 learned_addr learned my addr 172.21.15.49:0/1845122802 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:46.144 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.134+0000 7f61e0ee8640 1 --2- 172.21.15.49:0/1845122802 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61dc0cda10 0x7f61dc149740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:46.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.134+0000 7f61dbfff640 1 -- 172.21.15.49:0/1845122802 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61dc0d1fa0 msgr2=0x7f61dc148b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:46.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.134+0000 7f61dbfff640 1 --2- 172.21.15.49:0/1845122802 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61dc0d1fa0 0x7f61dc148b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:46.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.134+0000 7f61dbfff640 1 -- 172.21.15.49:0/1845122802 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61dc0cda10 msgr2=0x7f61dc149740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:46.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.134+0000 7f61dbfff640 1 --2- 172.21.15.49:0/1845122802 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61dc0cda10 0x7f61dc149740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:46.146 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.134+0000 7f61dbfff640 1 -- 172.21.15.49:0/1845122802 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61c4009420 con 0x7f61dc0cf1f0 2023-11-06T02:08:46.146 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.134+0000 7f61dbfff640 1 --2- 172.21.15.49:0/1845122802 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61dc0cf1f0 0x7f61dc14c320 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f61c4019060 tx=0x7f61c40191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:46.146 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.138+0000 7f61d97fa640 1 -- 172.21.15.49:0/1845122802 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c40b6610 con 0x7f61dc0cf1f0 2023-11-06T02:08:46.146 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.138+0000 7f61d97fa640 1 -- 172.21.15.49:0/1845122802 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61c40b6790 con 0x7f61dc0cf1f0 2023-11-06T02:08:46.146 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.138+0000 7f61d97fa640 1 -- 172.21.15.49:0/1845122802 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c40b6b40 con 0x7f61dc0cf1f0 2023-11-06T02:08:46.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.138+0000 7f61e314c640 1 -- 172.21.15.49:0/1845122802 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61dc14be00 con 0x7f61dc0cf1f0 2023-11-06T02:08:46.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.138+0000 7f61e314c640 1 -- 172.21.15.49:0/1845122802 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61dc14b180 con 0x7f61dc0cf1f0 2023-11-06T02:08:46.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.138+0000 7f61e314c640 1 -- 172.21.15.49:0/1845122802 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61dc067140 con 0x7f61dc0cf1f0 2023-11-06T02:08:46.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.138+0000 7f61d97fa640 1 -- 172.21.15.49:0/1845122802 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f61c40b4040 con 0x7f61dc0cf1f0 2023-11-06T02:08:46.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.138+0000 7f61d97fa640 1 --2- 172.21.15.49:0/1845122802 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f61bc065a90 0x7f61bc067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:46.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.138+0000 7f61d97fa640 1 -- 172.21.15.49:0/1845122802 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f61c411d570 con 0x7f61dc0cf1f0 2023-11-06T02:08:46.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.138+0000 7f61e0ee8640 1 --2- 172.21.15.49:0/1845122802 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f61bc065a90 0x7f61bc067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:46.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.142+0000 7f61e0ee8640 1 --2- 172.21.15.49:0/1845122802 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f61bc065a90 0x7f61bc067f50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f61cc001570 tx=0x7f61cc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:46.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.142+0000 7f61d97fa640 1 -- 172.21.15.49:0/1845122802 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f61c40eaf80 con 0x7f61dc0cf1f0 2023-11-06T02:08:46.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.294+0000 7f61e314c640 1 -- 172.21.15.49:0/1845122802 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f61dc14b610 con 0x7f61dc0cf1f0 2023-11-06T02:08:46.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.294+0000 7f61d97fa640 1 -- 172.21.15.49:0/1845122802 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f61dc14b610 con 0x7f61dc0cf1f0 2023-11-06T02:08:46.292 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2023-11-06T02:08:46.294 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.298+0000 7f61e314c640 1 -- 172.21.15.49:0/1845122802 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f61bc065a90 msgr2=0x7f61bc067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:46.294 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.298+0000 7f61e314c640 1 --2- 172.21.15.49:0/1845122802 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f61bc065a90 0x7f61bc067f50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f61cc001570 tx=0x7f61cc009040 comp rx=0 tx=0).stop 2023-11-06T02:08:46.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.298+0000 7f61e314c640 1 -- 172.21.15.49:0/1845122802 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61dc0cf1f0 msgr2=0x7f61dc14c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:46.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.298+0000 7f61e314c640 1 --2- 172.21.15.49:0/1845122802 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61dc0cf1f0 0x7f61dc14c320 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f61c4019060 tx=0x7f61c40191f0 comp rx=0 tx=0).stop 2023-11-06T02:08:46.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.298+0000 7f61e314c640 1 -- 172.21.15.49:0/1845122802 shutdown_connections 2023-11-06T02:08:46.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.298+0000 7f61e314c640 1 --2- 172.21.15.49:0/1845122802 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f61bc065a90 0x7f61bc067f50 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:46.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.298+0000 7f61e314c640 1 --2- 172.21.15.49:0/1845122802 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61dc0cf1f0 0x7f61dc14c320 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:46.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.298+0000 7f61e314c640 1 --2- 172.21.15.49:0/1845122802 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61dc0d1fa0 0x7f61dc148b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:46.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.298+0000 7f61e314c640 1 --2- 172.21.15.49:0/1845122802 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61dc0cda10 0x7f61dc149740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:46.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.298+0000 7f61e314c640 1 -- 172.21.15.49:0/1845122802 >> 172.21.15.49:0/1845122802 conn(0x7f61dc071aa0 msgr2=0x7f61dc0b7de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:46.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.298+0000 7f61e314c640 1 -- 172.21.15.49:0/1845122802 shutdown_connections 2023-11-06T02:08:46.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.298+0000 7f61e314c640 1 -- 172.21.15.49:0/1845122802 wait complete. 2023-11-06T02:08:46.310 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T02:08:46.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.430+0000 7f25b6e69640 1 Processor -- start 2023-11-06T02:08:46.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.430+0000 7f25b6e69640 1 -- start start 2023-11-06T02:08:46.427 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.430+0000 7f25b6e69640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f25b00d08b0 0x7f25b00d0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:46.428 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.430+0000 7f25b6e69640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f25b00aeff0 0x7f25b00af440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:46.428 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.430+0000 7f25b6e69640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25b00d1230 con 0x7f25b015ab60 2023-11-06T02:08:46.428 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.430+0000 7f25b6e69640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25b00cb340 con 0x7f25b00d08b0 2023-11-06T02:08:46.428 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.430+0000 7f25b6e69640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25b00adf50 con 0x7f25b00aeff0 2023-11-06T02:08:46.428 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.430+0000 7f25affff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f25b00d08b0 0x7f25b00d0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:46.429 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.430+0000 7f25affff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f25b00d08b0 0x7f25b00d0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52632/0 (socket says 172.21.15.49:52632) 2023-11-06T02:08:46.429 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.430+0000 7f25affff640 1 -- 172.21.15.49:0/3541966621 learned_addr learned my addr 172.21.15.49:0/3541966621 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:46.429 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.430+0000 7f25b5406640 1 --2- 172.21.15.49:0/3541966621 >> v2:172.21.15.150:3300/0 conn(0x7f25b00aeff0 0x7f25b00af440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:46.429 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.430+0000 7f25affff640 1 -- 172.21.15.49:0/3541966621 >> v2:172.21.15.150:3300/0 conn(0x7f25b00aeff0 msgr2=0x7f25b00af440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:46.430 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.430+0000 7f25affff640 1 --2- 172.21.15.49:0/3541966621 >> v2:172.21.15.150:3300/0 conn(0x7f25b00aeff0 0x7f25b00af440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:46.430 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.430+0000 7f25affff640 1 -- 172.21.15.49:0/3541966621 >> v1:172.21.15.150:6789/0 conn(0x7f25b015ab60 legacy=0x7f25b00ad520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:46.430 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.430+0000 7f25affff640 1 -- 172.21.15.49:0/3541966621 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25b00ae230 con 0x7f25b00d08b0 2023-11-06T02:08:46.434 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.434+0000 7f25affff640 1 --2- 172.21.15.49:0/3541966621 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25b00d08b0 0x7f25b00d0cf0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f25b00b34d0 tx=0x7f25a401c1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72ba67970ce5f493 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:46.434 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.434+0000 7f25af7fe640 1 -- 172.21.15.49:0/3541966621 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a40ab600 con 0x7f25b00d08b0 2023-11-06T02:08:46.434 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.434+0000 7f25af7fe640 1 -- 172.21.15.49:0/3541966621 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25a40a7050 con 0x7f25b00d08b0 2023-11-06T02:08:46.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.434+0000 7f25af7fe640 1 -- 172.21.15.49:0/3541966621 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a40ab980 con 0x7f25b00d08b0 2023-11-06T02:08:46.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.434+0000 7f25b6e69640 1 -- 172.21.15.49:0/3541966621 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25b00d08b0 msgr2=0x7f25b00d0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:46.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.434+0000 7f25b6e69640 1 --2- 172.21.15.49:0/3541966621 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25b00d08b0 0x7f25b00d0cf0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f25b00b34d0 tx=0x7f25a401c1d0 comp rx=0 tx=0).stop 2023-11-06T02:08:46.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.434+0000 7f25b6e69640 1 -- 172.21.15.49:0/3541966621 shutdown_connections 2023-11-06T02:08:46.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.434+0000 7f25b6e69640 1 --2- 172.21.15.49:0/3541966621 >> v2:172.21.15.150:3300/0 conn(0x7f25b00aeff0 0x7f25b00af440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:46.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.434+0000 7f25b6e69640 1 --2- 172.21.15.49:0/3541966621 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25b00d08b0 0x7f25b00d0cf0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:46.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.434+0000 7f25b6e69640 1 -- 172.21.15.49:0/3541966621 >> 172.21.15.49:0/3541966621 conn(0x7f25b0089fd0 msgr2=0x7f25b0099c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:46.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.434+0000 7f25b6e69640 1 -- 172.21.15.49:0/3541966621 shutdown_connections 2023-11-06T02:08:46.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.434+0000 7f25b6e69640 1 -- 172.21.15.49:0/3541966621 wait complete. 2023-11-06T02:08:46.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.434+0000 7f25b6e69640 1 Processor -- start 2023-11-06T02:08:46.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.434+0000 7f25b6e69640 1 -- start start 2023-11-06T02:08:46.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.434+0000 7f25b6e69640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25b00aeff0 0x7f25b015d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:46.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.434+0000 7f25b6e69640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f25b00d08b0 0x7f25b0150200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:46.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.434+0000 7f25b6e69640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25b00c9700 0x7f25b00c9b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:46.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.434+0000 7f25b6e69640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b00ca760 con 0x7f25b00c9700 2023-11-06T02:08:46.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.434+0000 7f25b6e69640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b00c8e80 con 0x7f25b00aeff0 2023-11-06T02:08:46.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.434+0000 7f25b6e69640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25b0150740 con 0x7f25b00d08b0 2023-11-06T02:08:46.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.434+0000 7f25b5406640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25b00c9700 0x7f25b00c9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:46.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.434+0000 7f25b5406640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25b00c9700 0x7f25b00c9b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52638/0 (socket says 172.21.15.49:52638) 2023-11-06T02:08:46.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.434+0000 7f25b5406640 1 -- 172.21.15.49:0/3908238925 learned_addr learned my addr 172.21.15.49:0/3908238925 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:46.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.434+0000 7f25affff640 1 --2- 172.21.15.49:0/3908238925 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f25b00d08b0 0x7f25b0150200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:46.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.434+0000 7f25b4c05640 1 --2- 172.21.15.49:0/3908238925 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25b00aeff0 0x7f25b015d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:46.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.434+0000 7f25b5406640 1 -- 172.21.15.49:0/3908238925 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f25b00d08b0 msgr2=0x7f25b0150200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:46.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.434+0000 7f25b5406640 1 --2- 172.21.15.49:0/3908238925 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f25b00d08b0 0x7f25b0150200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:46.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.434+0000 7f25b5406640 1 -- 172.21.15.49:0/3908238925 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25b00aeff0 msgr2=0x7f25b015d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:46.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.434+0000 7f25b5406640 1 --2- 172.21.15.49:0/3908238925 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25b00aeff0 0x7f25b015d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:46.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.434+0000 7f25b5406640 1 -- 172.21.15.49:0/3908238925 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25a40094d0 con 0x7f25b00c9700 2023-11-06T02:08:46.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.434+0000 7f25b5406640 1 --2- 172.21.15.49:0/3908238925 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25b00c9700 0x7f25b00c9b90 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f25a8007e70 tx=0x7f25a800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:46.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.434+0000 7f25ad7fa640 1 -- 172.21.15.49:0/3908238925 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a8014420 con 0x7f25b00c9700 2023-11-06T02:08:46.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.434+0000 7f25ad7fa640 1 -- 172.21.15.49:0/3908238925 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25a80145a0 con 0x7f25b00c9700 2023-11-06T02:08:46.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.434+0000 7f25b6e69640 1 -- 172.21.15.49:0/3908238925 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25b00c85e0 con 0x7f25b00c9700 2023-11-06T02:08:46.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.434+0000 7f25ad7fa640 1 -- 172.21.15.49:0/3908238925 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a8014950 con 0x7f25b00c9700 2023-11-06T02:08:46.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.434+0000 7f25b6e69640 1 -- 172.21.15.49:0/3908238925 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25b00c8ba0 con 0x7f25b00c9700 2023-11-06T02:08:46.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.438+0000 7f25b6e69640 1 -- 172.21.15.49:0/3908238925 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25b0067140 con 0x7f25b00c9700 2023-11-06T02:08:46.443 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.438+0000 7f25ad7fa640 1 -- 172.21.15.49:0/3908238925 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f25a8014ab0 con 0x7f25b00c9700 2023-11-06T02:08:46.443 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.438+0000 7f25ad7fa640 1 --2- 172.21.15.49:0/3908238925 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f259c065a90 0x7f259c067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:46.443 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.438+0000 7f25ad7fa640 1 -- 172.21.15.49:0/3908238925 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f25a8083b40 con 0x7f25b00c9700 2023-11-06T02:08:46.444 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.438+0000 7f25b4c05640 1 --2- 172.21.15.49:0/3908238925 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f259c065a90 0x7f259c067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:46.444 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.438+0000 7f25b4c05640 1 --2- 172.21.15.49:0/3908238925 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f259c065a90 0x7f259c067f50 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f25b00d1200 tx=0x7f25a0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:46.445 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.442+0000 7f25ad7fa640 1 -- 172.21.15.49:0/3908238925 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f25a8051430 con 0x7f25b00c9700 2023-11-06T02:08:46.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.594+0000 7f25b6e69640 1 -- 172.21.15.49:0/3908238925 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f25b00cb520 con 0x7f25b00c9700 2023-11-06T02:08:46.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.594+0000 7f25ad7fa640 1 -- 172.21.15.49:0/3908238925 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f25a8054a50 con 0x7f25b00c9700 2023-11-06T02:08:46.591 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 8 2023-11-06T02:08:46.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.594+0000 7f25b6e69640 1 -- 172.21.15.49:0/3908238925 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f259c065a90 msgr2=0x7f259c067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:46.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.594+0000 7f25b6e69640 1 --2- 172.21.15.49:0/3908238925 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f259c065a90 0x7f259c067f50 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f25b00d1200 tx=0x7f25a0008040 comp rx=0 tx=0).stop 2023-11-06T02:08:46.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.594+0000 7f25b6e69640 1 -- 172.21.15.49:0/3908238925 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25b00c9700 msgr2=0x7f25b00c9b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:46.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.594+0000 7f25b6e69640 1 --2- 172.21.15.49:0/3908238925 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25b00c9700 0x7f25b00c9b90 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f25a8007e70 tx=0x7f25a800e4b0 comp rx=0 tx=0).stop 2023-11-06T02:08:46.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.594+0000 7f25b6e69640 1 -- 172.21.15.49:0/3908238925 shutdown_connections 2023-11-06T02:08:46.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.594+0000 7f25b6e69640 1 --2- 172.21.15.49:0/3908238925 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f259c065a90 0x7f259c067f50 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:46.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.594+0000 7f25b6e69640 1 --2- 172.21.15.49:0/3908238925 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25b00c9700 0x7f25b00c9b90 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:46.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.594+0000 7f25b6e69640 1 --2- 172.21.15.49:0/3908238925 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f25b00d08b0 0x7f25b0150200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:46.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.594+0000 7f25b6e69640 1 --2- 172.21.15.49:0/3908238925 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25b00aeff0 0x7f25b015d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:46.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.594+0000 7f25b6e69640 1 -- 172.21.15.49:0/3908238925 >> 172.21.15.49:0/3908238925 conn(0x7f25b0089fd0 msgr2=0x7f25b00b08d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:46.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.594+0000 7f25b6e69640 1 -- 172.21.15.49:0/3908238925 shutdown_connections 2023-11-06T02:08:46.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.594+0000 7f25b6e69640 1 -- 172.21.15.49:0/3908238925 wait complete. 2023-11-06T02:08:46.608 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T02:08:46.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.738+0000 7f449fbca640 1 Processor -- start 2023-11-06T02:08:46.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.738+0000 7f449fbca640 1 -- start start 2023-11-06T02:08:46.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.738+0000 7f449fbca640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f44980b04e0 0x7f44980b08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:46.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.738+0000 7f449fbca640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44980cc5c0 con 0x7f44980cb2d0 2023-11-06T02:08:46.736 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.738+0000 7f449fbca640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44980cb760 con 0x7f449815a9e0 2023-11-06T02:08:46.736 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.738+0000 7f449fbca640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44980cd160 con 0x7f44980b04e0 2023-11-06T02:08:46.736 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.738+0000 7f449d165640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f44980cb2d0 0x7f44980cca50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:54102/0 (socket says 172.21.15.49:54102) 2023-11-06T02:08:46.736 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.738+0000 7f449d165640 1 -- 172.21.15.49:0/3101126053 learned_addr learned my addr 172.21.15.49:0/3101126053 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:08:46.736 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.738+0000 7f449e167640 1 --2- 172.21.15.49:0/3101126053 >> v2:172.21.15.49:3300/0 conn(0x7f44980b04e0 0x7f44980b08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:46.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.738+0000 7f449c964640 1 -- 172.21.15.49:0/3101126053 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2079763973 0 0) 0x7f44980cc5c0 con 0x7f44980cb2d0 2023-11-06T02:08:46.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.738+0000 7f449c964640 1 -- 172.21.15.49:0/3101126053 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4484003530 con 0x7f44980cb2d0 2023-11-06T02:08:46.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.738+0000 7f449e167640 1 -- 172.21.15.49:0/3101126053 >> v1:172.21.15.150:6789/0 conn(0x7f449815a9e0 legacy=0x7f44980afbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:46.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.738+0000 7f449e167640 1 -- 172.21.15.49:0/3101126053 >> v1:172.21.15.49:6789/0 conn(0x7f44980cb2d0 legacy=0x7f44980cca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:46.738 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.738+0000 7f449e167640 1 -- 172.21.15.49:0/3101126053 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44980b0e30 con 0x7f44980b04e0 2023-11-06T02:08:46.739 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.742+0000 7f449e167640 1 --2- 172.21.15.49:0/3101126053 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44980b04e0 0x7f44980b08f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f44940097c0 tx=0x7f449401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe19262cd1d1e6c9 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:46.739 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.742+0000 7f449c964640 1 -- 172.21.15.49:0/3101126053 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44940ab650 con 0x7f44980b04e0 2023-11-06T02:08:46.739 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.742+0000 7f449c964640 1 -- 172.21.15.49:0/3101126053 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44940ab7b0 con 0x7f44980b04e0 2023-11-06T02:08:46.740 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.742+0000 7f449c964640 1 -- 172.21.15.49:0/3101126053 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44940abb20 con 0x7f44980b04e0 2023-11-06T02:08:46.740 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.742+0000 7f449fbca640 1 -- 172.21.15.49:0/3101126053 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44980b04e0 msgr2=0x7f44980b08f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:46.740 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.742+0000 7f449fbca640 1 --2- 172.21.15.49:0/3101126053 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44980b04e0 0x7f44980b08f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f44940097c0 tx=0x7f449401c450 comp rx=0 tx=0).stop 2023-11-06T02:08:46.740 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.742+0000 7f449fbca640 1 -- 172.21.15.49:0/3101126053 shutdown_connections 2023-11-06T02:08:46.740 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.742+0000 7f449fbca640 1 --2- 172.21.15.49:0/3101126053 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44980b04e0 0x7f44980b08f0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:46.741 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.742+0000 7f449fbca640 1 -- 172.21.15.49:0/3101126053 >> 172.21.15.49:0/3101126053 conn(0x7f4498098a40 msgr2=0x7f44980cf860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:46.741 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.742+0000 7f449fbca640 1 -- 172.21.15.49:0/3101126053 shutdown_connections 2023-11-06T02:08:46.741 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.742+0000 7f449fbca640 1 -- 172.21.15.49:0/3101126053 wait complete. 2023-11-06T02:08:46.743 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.742+0000 7f449fbca640 1 Processor -- start 2023-11-06T02:08:46.744 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.742+0000 7f449fbca640 1 -- start start 2023-11-06T02:08:46.744 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.742+0000 7f449fbca640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44980b04e0 0x7f4498159db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:46.744 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.742+0000 7f449fbca640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44980cb2d0 0x7f44980c5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:46.744 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.742+0000 7f449fbca640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f449815a9e0 0x7f44980c5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:46.745 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.742+0000 7f449fbca640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f449815a2f0 con 0x7f44980b04e0 2023-11-06T02:08:46.745 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.742+0000 7f449fbca640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f449813b690 con 0x7f44980cb2d0 2023-11-06T02:08:46.745 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.742+0000 7f449fbca640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f449813c210 con 0x7f449815a9e0 2023-11-06T02:08:46.745 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.742+0000 7f449d966640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44980b04e0 0x7f4498159db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:46.745 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.746+0000 7f449d966640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44980b04e0 0x7f4498159db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52644/0 (socket says 172.21.15.49:52644) 2023-11-06T02:08:46.746 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.746+0000 7f449d966640 1 -- 172.21.15.49:0/2480857804 learned_addr learned my addr 172.21.15.49:0/2480857804 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:46.746 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.746+0000 7f449e167640 1 --2- 172.21.15.49:0/2480857804 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f449815a9e0 0x7f44980c5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:46.746 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.746+0000 7f449d165640 1 --2- 172.21.15.49:0/2480857804 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44980cb2d0 0x7f44980c5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:46.746 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.746+0000 7f449d966640 1 -- 172.21.15.49:0/2480857804 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f449815a9e0 msgr2=0x7f44980c5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:46.746 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.746+0000 7f449d966640 1 --2- 172.21.15.49:0/2480857804 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f449815a9e0 0x7f44980c5cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:46.747 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.746+0000 7f449d966640 1 -- 172.21.15.49:0/2480857804 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44980cb2d0 msgr2=0x7f44980c5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:46.747 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.746+0000 7f449d966640 1 --2- 172.21.15.49:0/2480857804 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44980cb2d0 0x7f44980c5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:46.747 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.746+0000 7f449d966640 1 -- 172.21.15.49:0/2480857804 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4494009420 con 0x7f44980b04e0 2023-11-06T02:08:46.747 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.746+0000 7f449e167640 1 --2- 172.21.15.49:0/2480857804 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f449815a9e0 0x7f44980c5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:08:46.747 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.746+0000 7f449d165640 1 --2- 172.21.15.49:0/2480857804 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44980cb2d0 0x7f44980c5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:08:46.747 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.746+0000 7f449d966640 1 --2- 172.21.15.49:0/2480857804 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44980b04e0 0x7f4498159db0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f449000c7a0 tx=0x7f449000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:46.748 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.746+0000 7f448e7fc640 1 -- 172.21.15.49:0/2480857804 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4490003b80 con 0x7f44980b04e0 2023-11-06T02:08:46.748 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.746+0000 7f448e7fc640 1 -- 172.21.15.49:0/2480857804 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4490003ce0 con 0x7f44980b04e0 2023-11-06T02:08:46.748 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.746+0000 7f448e7fc640 1 -- 172.21.15.49:0/2480857804 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f449000aa90 con 0x7f44980b04e0 2023-11-06T02:08:46.748 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.746+0000 7f449fbca640 1 -- 172.21.15.49:0/2480857804 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f449813b8e0 con 0x7f44980b04e0 2023-11-06T02:08:46.749 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.746+0000 7f449fbca640 1 -- 172.21.15.49:0/2480857804 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f449813bea0 con 0x7f44980b04e0 2023-11-06T02:08:46.749 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.746+0000 7f449fbca640 1 -- 172.21.15.49:0/2480857804 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44980ca6a0 con 0x7f44980b04e0 2023-11-06T02:08:46.749 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.746+0000 7f448e7fc640 1 -- 172.21.15.49:0/2480857804 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f449000abf0 con 0x7f44980b04e0 2023-11-06T02:08:46.749 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.746+0000 7f448e7fc640 1 --2- 172.21.15.49:0/2480857804 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f447c065a60 0x7f447c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:46.749 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.746+0000 7f448e7fc640 1 -- 172.21.15.49:0/2480857804 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f44900830a0 con 0x7f44980b04e0 2023-11-06T02:08:46.750 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.746+0000 7f449d165640 1 --2- 172.21.15.49:0/2480857804 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f447c065a60 0x7f447c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:46.750 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.746+0000 7f449d165640 1 --2- 172.21.15.49:0/2480857804 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f447c065a60 0x7f447c067f20 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f44980ade50 tx=0x7f4488009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:46.750 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.750+0000 7f448e7fc640 1 -- 172.21.15.49:0/2480857804 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f4490050990 con 0x7f44980b04e0 2023-11-06T02:08:46.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.902+0000 7f449fbca640 1 -- 172.21.15.49:0/2480857804 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f449813e520 con 0x7f44980b04e0 2023-11-06T02:08:46.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.906+0000 7f448e7fc640 1 -- 172.21.15.49:0/2480857804 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4490053fb0 con 0x7f44980b04e0 2023-11-06T02:08:46.901 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2023-11-06T02:08:46.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.906+0000 7f449fbca640 1 -- 172.21.15.49:0/2480857804 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f447c065a60 msgr2=0x7f447c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:46.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.906+0000 7f449fbca640 1 --2- 172.21.15.49:0/2480857804 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f447c065a60 0x7f447c067f20 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f44980ade50 tx=0x7f4488009040 comp rx=0 tx=0).stop 2023-11-06T02:08:46.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.906+0000 7f449fbca640 1 -- 172.21.15.49:0/2480857804 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44980b04e0 msgr2=0x7f4498159db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:46.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.906+0000 7f449fbca640 1 --2- 172.21.15.49:0/2480857804 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44980b04e0 0x7f4498159db0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f449000c7a0 tx=0x7f449000cc60 comp rx=0 tx=0).stop 2023-11-06T02:08:46.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.906+0000 7f449fbca640 1 -- 172.21.15.49:0/2480857804 shutdown_connections 2023-11-06T02:08:46.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.906+0000 7f449fbca640 1 --2- 172.21.15.49:0/2480857804 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f447c065a60 0x7f447c067f20 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:46.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.906+0000 7f449fbca640 1 --2- 172.21.15.49:0/2480857804 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f449815a9e0 0x7f44980c5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:46.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.906+0000 7f449fbca640 1 --2- 172.21.15.49:0/2480857804 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44980cb2d0 0x7f44980c5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:46.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.906+0000 7f449fbca640 1 --2- 172.21.15.49:0/2480857804 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44980b04e0 0x7f4498159db0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:46.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.906+0000 7f449fbca640 1 -- 172.21.15.49:0/2480857804 >> 172.21.15.49:0/2480857804 conn(0x7f4498098a40 msgr2=0x7f44980cca50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:46.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.906+0000 7f449fbca640 1 -- 172.21.15.49:0/2480857804 shutdown_connections 2023-11-06T02:08:46.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:46.906+0000 7f449fbca640 1 -- 172.21.15.49:0/2480857804 wait complete. 2023-11-06T02:08:46.920 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T02:08:47.050 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.054+0000 7fe894cf5640 1 Processor -- start 2023-11-06T02:08:47.050 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.054+0000 7fe894cf5640 1 -- start start 2023-11-06T02:08:47.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.054+0000 7fe894cf5640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe89015a880 0x7fe8900b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:47.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.054+0000 7fe894cf5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe890070160 0x7fe890070610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:47.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.054+0000 7fe894cf5640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8900ae360 con 0x7fe89006d150 2023-11-06T02:08:47.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.054+0000 7fe894cf5640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe890070b50 con 0x7fe89015a880 2023-11-06T02:08:47.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.054+0000 7fe894cf5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8900ae640 con 0x7fe890070160 2023-11-06T02:08:47.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.054+0000 7fe88e59c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe89015a880 0x7fe8900b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:47.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.054+0000 7fe88e59c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe89015a880 0x7fe8900b15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52650/0 (socket says 172.21.15.49:52650) 2023-11-06T02:08:47.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.054+0000 7fe88e59c640 1 -- 172.21.15.49:0/1398011081 learned_addr learned my addr 172.21.15.49:0/1398011081 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:47.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.054+0000 7fe88ed9d640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fe89006d150 0x7fe89006d5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:54118/0 (socket says 172.21.15.49:54118) 2023-11-06T02:08:47.054 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.054+0000 7fe88dd9b640 1 --2- 172.21.15.49:0/1398011081 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe890070160 0x7fe890070610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:47.054 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.054+0000 7fe88e59c640 1 -- 172.21.15.49:0/1398011081 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe890070160 msgr2=0x7fe890070610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:47.054 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.054+0000 7fe88e59c640 1 --2- 172.21.15.49:0/1398011081 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe890070160 0x7fe890070610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:47.054 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.054+0000 7fe88e59c640 1 -- 172.21.15.49:0/1398011081 >> v1:172.21.15.49:6789/0 conn(0x7fe89006d150 legacy=0x7fe89006d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:47.054 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.054+0000 7fe88e59c640 1 -- 172.21.15.49:0/1398011081 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8900ae920 con 0x7fe89015a880 2023-11-06T02:08:47.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.054+0000 7fe88dd9b640 1 --2- 172.21.15.49:0/1398011081 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe890070160 0x7fe890070610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:08:47.058 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.058+0000 7fe88e59c640 1 --2- 172.21.15.49:0/1398011081 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe89015a880 0x7fe8900b15c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe880009e50 tx=0x7fe88001ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b2c13c0a8db3f61 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:47.059 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.058+0000 7fe88d59a640 1 -- 172.21.15.49:0/1398011081 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8800a57f0 con 0x7fe89015a880 2023-11-06T02:08:47.059 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.058+0000 7fe88d59a640 1 -- 172.21.15.49:0/1398011081 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8800a5950 con 0x7fe89015a880 2023-11-06T02:08:47.059 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.058+0000 7fe88d59a640 1 -- 172.21.15.49:0/1398011081 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8800a5ce0 con 0x7fe89015a880 2023-11-06T02:08:47.059 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.058+0000 7fe894cf5640 1 -- 172.21.15.49:0/1398011081 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe89015a880 msgr2=0x7fe8900b15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:47.059 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.058+0000 7fe894cf5640 1 --2- 172.21.15.49:0/1398011081 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe89015a880 0x7fe8900b15c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe880009e50 tx=0x7fe88001ca10 comp rx=0 tx=0).stop 2023-11-06T02:08:47.060 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.058+0000 7fe894cf5640 1 -- 172.21.15.49:0/1398011081 shutdown_connections 2023-11-06T02:08:47.060 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.058+0000 7fe894cf5640 1 --2- 172.21.15.49:0/1398011081 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe890070160 0x7fe890070610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:47.060 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.058+0000 7fe894cf5640 1 --2- 172.21.15.49:0/1398011081 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe89015a880 0x7fe8900b15c0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:47.060 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.058+0000 7fe894cf5640 1 -- 172.21.15.49:0/1398011081 >> 172.21.15.49:0/1398011081 conn(0x7fe89006a370 msgr2=0x7fe8900ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:47.061 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.058+0000 7fe894cf5640 1 -- 172.21.15.49:0/1398011081 shutdown_connections 2023-11-06T02:08:47.061 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.058+0000 7fe894cf5640 1 -- 172.21.15.49:0/1398011081 wait complete. 2023-11-06T02:08:47.061 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.058+0000 7fe894cf5640 1 Processor -- start 2023-11-06T02:08:47.061 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.058+0000 7fe894cf5640 1 -- start start 2023-11-06T02:08:47.061 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.062+0000 7fe894cf5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe89006d150 0x7fe8900cd420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:47.062 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.062+0000 7fe894cf5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe890070160 0x7fe8900c42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:47.062 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.062+0000 7fe894cf5640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe89015a880 0x7fe8900cbc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:47.062 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.062+0000 7fe894cf5640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8900cc170 con 0x7fe89006d150 2023-11-06T02:08:47.062 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.062+0000 7fe894cf5640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8900cc380 con 0x7fe89015a880 2023-11-06T02:08:47.062 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.062+0000 7fe894cf5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8900cc660 con 0x7fe890070160 2023-11-06T02:08:47.063 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.062+0000 7fe88e59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe89006d150 0x7fe8900cd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:47.063 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.062+0000 7fe88e59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe89006d150 0x7fe8900cd420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52656/0 (socket says 172.21.15.49:52656) 2023-11-06T02:08:47.063 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.062+0000 7fe88e59c640 1 -- 172.21.15.49:0/338580580 learned_addr learned my addr 172.21.15.49:0/338580580 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:47.063 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.062+0000 7fe88dd9b640 1 --2- 172.21.15.49:0/338580580 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe890070160 0x7fe8900c42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:47.064 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.062+0000 7fe88ed9d640 1 --2- 172.21.15.49:0/338580580 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe89015a880 0x7fe8900cbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:47.064 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.062+0000 7fe88e59c640 1 -- 172.21.15.49:0/338580580 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe890070160 msgr2=0x7fe8900c42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:47.064 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.062+0000 7fe88e59c640 1 --2- 172.21.15.49:0/338580580 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe890070160 0x7fe8900c42e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:47.064 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.062+0000 7fe88e59c640 1 -- 172.21.15.49:0/338580580 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe89015a880 msgr2=0x7fe8900cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:47.064 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.062+0000 7fe88e59c640 1 --2- 172.21.15.49:0/338580580 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe89015a880 0x7fe8900cbc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:47.065 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.062+0000 7fe88e59c640 1 -- 172.21.15.49:0/338580580 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe880009b30 con 0x7fe89006d150 2023-11-06T02:08:47.065 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.062+0000 7fe88ed9d640 1 --2- 172.21.15.49:0/338580580 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe89015a880 0x7fe8900cbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:08:47.065 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.062+0000 7fe88e59c640 1 --2- 172.21.15.49:0/338580580 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe89006d150 0x7fe8900cd420 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe88001a640 tx=0x7fe8800197b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:47.065 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.062+0000 7fe87effd640 1 -- 172.21.15.49:0/338580580 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8800bd070 con 0x7fe89006d150 2023-11-06T02:08:47.065 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.062+0000 7fe87effd640 1 -- 172.21.15.49:0/338580580 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8800b8470 con 0x7fe89006d150 2023-11-06T02:08:47.066 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.062+0000 7fe894cf5640 1 -- 172.21.15.49:0/338580580 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8900cc940 con 0x7fe89006d150 2023-11-06T02:08:47.066 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.062+0000 7fe87effd640 1 -- 172.21.15.49:0/338580580 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8800b8820 con 0x7fe89006d150 2023-11-06T02:08:47.066 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.062+0000 7fe894cf5640 1 -- 172.21.15.49:0/338580580 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8900ccf00 con 0x7fe89006d150 2023-11-06T02:08:47.066 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.062+0000 7fe894cf5640 1 -- 172.21.15.49:0/338580580 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe890074040 con 0x7fe89006d150 2023-11-06T02:08:47.067 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.062+0000 7fe87effd640 1 -- 172.21.15.49:0/338580580 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fe8800b8980 con 0x7fe89006d150 2023-11-06T02:08:47.068 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.062+0000 7fe87effd640 1 --2- 172.21.15.49:0/338580580 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe870065a60 0x7fe870067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:47.068 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.062+0000 7fe87effd640 1 -- 172.21.15.49:0/338580580 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe88011ded0 con 0x7fe89006d150 2023-11-06T02:08:47.068 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.062+0000 7fe88dd9b640 1 --2- 172.21.15.49:0/338580580 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe870065a60 0x7fe870067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:47.068 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.066+0000 7fe88dd9b640 1 --2- 172.21.15.49:0/338580580 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe870065a60 0x7fe870067f20 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe890165480 tx=0x7fe884008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:47.069 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.066+0000 7fe87effd640 1 -- 172.21.15.49:0/338580580 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fe8800eb730 con 0x7fe89006d150 2023-11-06T02:08:47.222 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.226+0000 7fe894cf5640 1 -- 172.21.15.49:0/338580580 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe8900cdff0 con 0x7fe89006d150 2023-11-06T02:08:47.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.226+0000 7fe87effd640 1 -- 172.21.15.49:0/338580580 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe8800eed50 con 0x7fe89006d150 2023-11-06T02:08:47.223 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2023-11-06T02:08:47.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.230+0000 7fe894cf5640 1 -- 172.21.15.49:0/338580580 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe870065a60 msgr2=0x7fe870067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:47.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.230+0000 7fe894cf5640 1 --2- 172.21.15.49:0/338580580 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe870065a60 0x7fe870067f20 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe890165480 tx=0x7fe884008040 comp rx=0 tx=0).stop 2023-11-06T02:08:47.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.230+0000 7fe894cf5640 1 -- 172.21.15.49:0/338580580 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe89006d150 msgr2=0x7fe8900cd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:47.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.230+0000 7fe894cf5640 1 --2- 172.21.15.49:0/338580580 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe89006d150 0x7fe8900cd420 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe88001a640 tx=0x7fe8800197b0 comp rx=0 tx=0).stop 2023-11-06T02:08:47.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.230+0000 7fe894cf5640 1 -- 172.21.15.49:0/338580580 shutdown_connections 2023-11-06T02:08:47.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.230+0000 7fe894cf5640 1 --2- 172.21.15.49:0/338580580 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe870065a60 0x7fe870067f20 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:47.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.230+0000 7fe894cf5640 1 --2- 172.21.15.49:0/338580580 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe89015a880 0x7fe8900cbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:47.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.230+0000 7fe894cf5640 1 --2- 172.21.15.49:0/338580580 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe890070160 0x7fe8900c42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:47.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.230+0000 7fe894cf5640 1 --2- 172.21.15.49:0/338580580 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe89006d150 0x7fe8900cd420 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:47.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.230+0000 7fe894cf5640 1 -- 172.21.15.49:0/338580580 >> 172.21.15.49:0/338580580 conn(0x7fe89006a370 msgr2=0x7fe8900b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:47.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.230+0000 7fe894cf5640 1 -- 172.21.15.49:0/338580580 shutdown_connections 2023-11-06T02:08:47.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.230+0000 7fe894cf5640 1 -- 172.21.15.49:0/338580580 wait complete. 2023-11-06T02:08:47.243 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T02:08:47.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.366+0000 7f8f387d1640 1 Processor -- start 2023-11-06T02:08:47.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.366+0000 7f8f387d1640 1 -- start start 2023-11-06T02:08:47.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.366+0000 7f8f387d1640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f8f300b0790 0x7f8f30070a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:47.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.366+0000 7f8f387d1640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8f300ca830 0x7f8f300b1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:47.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.366+0000 7f8f387d1640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f300cacf0 con 0x7f8f3015aae0 2023-11-06T02:08:47.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.366+0000 7f8f387d1640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f30070fc0 con 0x7f8f300b0790 2023-11-06T02:08:47.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.366+0000 7f8f387d1640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f3015aec0 con 0x7f8f300ca830 2023-11-06T02:08:47.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.366+0000 7f8f3656d640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f8f3015aae0 0x7f8f300b0080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:54130/0 (socket says 172.21.15.49:54130) 2023-11-06T02:08:47.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.366+0000 7f8f3656d640 1 -- 172.21.15.49:0/4144242782 learned_addr learned my addr 172.21.15.49:0/4144242782 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:08:47.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.366+0000 7f8f35d6c640 1 --2- 172.21.15.49:0/4144242782 >> v2:172.21.15.49:3300/0 conn(0x7f8f300b0790 0x7f8f30070a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:47.365 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.366+0000 7f8f36d6e640 1 --2- 172.21.15.49:0/4144242782 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8f300ca830 0x7f8f300b1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:47.365 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.366+0000 7f8f35d6c640 1 -- 172.21.15.49:0/4144242782 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8f300ca830 msgr2=0x7f8f300b1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:47.365 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.366+0000 7f8f35d6c640 1 --2- 172.21.15.49:0/4144242782 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8f300ca830 0x7f8f300b1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:47.365 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.366+0000 7f8f35d6c640 1 -- 172.21.15.49:0/4144242782 >> v1:172.21.15.49:6789/0 conn(0x7f8f3015aae0 legacy=0x7f8f300b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:47.365 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.366+0000 7f8f35d6c640 1 -- 172.21.15.49:0/4144242782 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f300c8ff0 con 0x7f8f300b0790 2023-11-06T02:08:47.367 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.370+0000 7f8f35d6c640 1 --2- 172.21.15.49:0/4144242782 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f300b0790 0x7f8f30070a80 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8f200097c0 tx=0x7f8f2001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=791861b0fb0b949f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:47.367 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.370+0000 7f8f3556b640 1 -- 172.21.15.49:0/4144242782 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f200ac620 con 0x7f8f300b0790 2023-11-06T02:08:47.367 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.370+0000 7f8f3556b640 1 -- 172.21.15.49:0/4144242782 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f200ac780 con 0x7f8f300b0790 2023-11-06T02:08:47.367 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.370+0000 7f8f3556b640 1 -- 172.21.15.49:0/4144242782 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f200acb10 con 0x7f8f300b0790 2023-11-06T02:08:47.367 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.370+0000 7f8f387d1640 1 -- 172.21.15.49:0/4144242782 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f300b0790 msgr2=0x7f8f30070a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:47.368 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.370+0000 7f8f387d1640 1 --2- 172.21.15.49:0/4144242782 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f300b0790 0x7f8f30070a80 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8f200097c0 tx=0x7f8f2001c3c0 comp rx=0 tx=0).stop 2023-11-06T02:08:47.368 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.370+0000 7f8f387d1640 1 -- 172.21.15.49:0/4144242782 shutdown_connections 2023-11-06T02:08:47.368 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.370+0000 7f8f387d1640 1 --2- 172.21.15.49:0/4144242782 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8f300ca830 0x7f8f300b1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:47.368 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.370+0000 7f8f387d1640 1 --2- 172.21.15.49:0/4144242782 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f300b0790 0x7f8f30070a80 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:47.368 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.370+0000 7f8f387d1640 1 -- 172.21.15.49:0/4144242782 >> 172.21.15.49:0/4144242782 conn(0x7f8f3006c0a0 msgr2=0x7f8f3006c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:47.369 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.370+0000 7f8f387d1640 1 -- 172.21.15.49:0/4144242782 shutdown_connections 2023-11-06T02:08:47.369 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.370+0000 7f8f387d1640 1 -- 172.21.15.49:0/4144242782 wait complete. 2023-11-06T02:08:47.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.370+0000 7f8f387d1640 1 Processor -- start 2023-11-06T02:08:47.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.370+0000 7f8f387d1640 1 -- start start 2023-11-06T02:08:47.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.370+0000 7f8f387d1640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8f300b0790 0x7f8f3015dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:47.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.370+0000 7f8f387d1640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f300ca830 0x7f8f3014cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:47.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.370+0000 7f8f387d1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f3015aae0 0x7f8f30153320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:47.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.370+0000 7f8f387d1640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f30154790 con 0x7f8f3015aae0 2023-11-06T02:08:47.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.370+0000 7f8f387d1640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f3015d740 con 0x7f8f300ca830 2023-11-06T02:08:47.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.370+0000 7f8f387d1640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f301541d0 con 0x7f8f300b0790 2023-11-06T02:08:47.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.370+0000 7f8f36d6e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f3015aae0 0x7f8f30153320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:47.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.370+0000 7f8f36d6e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f3015aae0 0x7f8f30153320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52678/0 (socket says 172.21.15.49:52678) 2023-11-06T02:08:47.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.370+0000 7f8f36d6e640 1 -- 172.21.15.49:0/554724230 learned_addr learned my addr 172.21.15.49:0/554724230 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:47.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.370+0000 7f8f35d6c640 1 --2- 172.21.15.49:0/554724230 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f300ca830 0x7f8f3014cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:47.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.370+0000 7f8f3656d640 1 --2- 172.21.15.49:0/554724230 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8f300b0790 0x7f8f3015dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:47.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.370+0000 7f8f36d6e640 1 -- 172.21.15.49:0/554724230 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8f300b0790 msgr2=0x7f8f3015dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:47.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.370+0000 7f8f36d6e640 1 --2- 172.21.15.49:0/554724230 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8f300b0790 0x7f8f3015dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:47.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.370+0000 7f8f36d6e640 1 -- 172.21.15.49:0/554724230 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f300ca830 msgr2=0x7f8f3014cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:47.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.370+0000 7f8f36d6e640 1 --2- 172.21.15.49:0/554724230 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f300ca830 0x7f8f3014cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:47.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.370+0000 7f8f36d6e640 1 -- 172.21.15.49:0/554724230 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f20009420 con 0x7f8f3015aae0 2023-11-06T02:08:47.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.374+0000 7f8f36d6e640 1 --2- 172.21.15.49:0/554724230 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f3015aae0 0x7f8f30153320 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8f28007e30 tx=0x7f8f2800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:47.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.374+0000 7f8f26ffd640 1 -- 172.21.15.49:0/554724230 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f28014620 con 0x7f8f3015aae0 2023-11-06T02:08:47.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.374+0000 7f8f26ffd640 1 -- 172.21.15.49:0/554724230 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f2800ee90 con 0x7f8f3015aae0 2023-11-06T02:08:47.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.374+0000 7f8f387d1640 1 -- 172.21.15.49:0/554724230 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f301544b0 con 0x7f8f3015aae0 2023-11-06T02:08:47.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.374+0000 7f8f26ffd640 1 -- 172.21.15.49:0/554724230 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f280149c0 con 0x7f8f3015aae0 2023-11-06T02:08:47.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.374+0000 7f8f387d1640 1 -- 172.21.15.49:0/554724230 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f3015d3d0 con 0x7f8f3015aae0 2023-11-06T02:08:47.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.374+0000 7f8f387d1640 1 -- 172.21.15.49:0/554724230 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f3015cf50 con 0x7f8f3015aae0 2023-11-06T02:08:47.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.374+0000 7f8f26ffd640 1 -- 172.21.15.49:0/554724230 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f8f28014b20 con 0x7f8f3015aae0 2023-11-06T02:08:47.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.374+0000 7f8f26ffd640 1 --2- 172.21.15.49:0/554724230 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8f14065a60 0x7f8f14067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:47.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.374+0000 7f8f26ffd640 1 -- 172.21.15.49:0/554724230 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8f28083be0 con 0x7f8f3015aae0 2023-11-06T02:08:47.378 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.374+0000 7f8f3656d640 1 --2- 172.21.15.49:0/554724230 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8f14065a60 0x7f8f14067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:47.378 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.374+0000 7f8f3656d640 1 --2- 172.21.15.49:0/554724230 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8f14065a60 0x7f8f14067f20 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c0010f0 tx=0x7f8f2c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:47.379 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.378+0000 7f8f26ffd640 1 -- 172.21.15.49:0/554724230 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f8f3015cf50 con 0x7f8f3015aae0 2023-11-06T02:08:47.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.562+0000 7f8f387d1640 1 -- 172.21.15.49:0/554724230 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8f3015cf50 con 0x7f8f3015aae0 2023-11-06T02:08:47.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.562+0000 7f8f26ffd640 1 -- 172.21.15.49:0/554724230 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4648 (secure 0 0 0) 0x7f8f3015cf50 con 0x7f8f3015aae0 2023-11-06T02:08:47.560 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:08:47.560 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6834/1377586971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1377586971},{"type":"v1","addr":"172.21.15.49:6834","nonce":1377586971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4288,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2620584959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2620584959},{"type":"v1","addr":"172.21.15.150:6835","nonce":2620584959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4291,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/4272191418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":4272191418},{"type":"v1","addr":"172.21.15.150:6837","nonce":4272191418}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T02:08:30.592185+0000","modified":"2023-11-06T02:08:33.245446+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4293},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4293":{"gid":4293,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.49:6835/4280448297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6833","nonce":4280448297},{"type":"v1","addr":"172.21.15.49:6835","nonce":4280448297}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-06T02:08:47.560 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 7 2023-11-06T02:08:47.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.566+0000 7f8f387d1640 1 -- 172.21.15.49:0/554724230 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8f14065a60 msgr2=0x7f8f14067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:47.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.566+0000 7f8f387d1640 1 --2- 172.21.15.49:0/554724230 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8f14065a60 0x7f8f14067f20 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c0010f0 tx=0x7f8f2c009040 comp rx=0 tx=0).stop 2023-11-06T02:08:47.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.566+0000 7f8f387d1640 1 -- 172.21.15.49:0/554724230 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f3015aae0 msgr2=0x7f8f30153320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:47.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.566+0000 7f8f387d1640 1 --2- 172.21.15.49:0/554724230 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f3015aae0 0x7f8f30153320 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8f28007e30 tx=0x7f8f2800e4b0 comp rx=0 tx=0).stop 2023-11-06T02:08:47.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.566+0000 7f8f387d1640 1 -- 172.21.15.49:0/554724230 shutdown_connections 2023-11-06T02:08:47.563 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.566+0000 7f8f387d1640 1 --2- 172.21.15.49:0/554724230 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8f14065a60 0x7f8f14067f20 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:47.563 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.566+0000 7f8f387d1640 1 --2- 172.21.15.49:0/554724230 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f3015aae0 0x7f8f30153320 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:47.563 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.566+0000 7f8f387d1640 1 --2- 172.21.15.49:0/554724230 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f300ca830 0x7f8f3014cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:47.563 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.566+0000 7f8f387d1640 1 --2- 172.21.15.49:0/554724230 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8f300b0790 0x7f8f3015dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:47.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.566+0000 7f8f387d1640 1 -- 172.21.15.49:0/554724230 >> 172.21.15.49:0/554724230 conn(0x7f8f3006c0a0 msgr2=0x7f8f300b24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:47.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.566+0000 7f8f387d1640 1 -- 172.21.15.49:0/554724230 shutdown_connections 2023-11-06T02:08:47.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.566+0000 7f8f387d1640 1 -- 172.21.15.49:0/554724230 wait complete. 2023-11-06T02:08:47.577 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T02:08:47.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.702+0000 7fb6db047640 1 Processor -- start 2023-11-06T02:08:47.701 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.702+0000 7fb6db047640 1 -- start start 2023-11-06T02:08:47.703 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.706+0000 7fb6db047640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb6d415a8c0 0x7fb6d415ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:47.703 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.706+0000 7fb6db047640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb6d40ca110 0x7fb6d415aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:47.703 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.706+0000 7fb6db047640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6d415b650 con 0x7fb6d415b1e0 2023-11-06T02:08:47.703 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.706+0000 7fb6db047640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6d415f390 con 0x7fb6d40ca110 2023-11-06T02:08:47.704 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.706+0000 7fb6db047640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d415b8c0 con 0x7fb6d415a8c0 2023-11-06T02:08:47.704 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.706+0000 7fb6d3fff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb6d40ca110 0x7fb6d415aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:47.704 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.706+0000 7fb6d3fff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb6d40ca110 0x7fb6d415aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52684/0 (socket says 172.21.15.49:52684) 2023-11-06T02:08:47.704 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.706+0000 7fb6d3fff640 1 -- 172.21.15.49:0/2833620840 learned_addr learned my addr 172.21.15.49:0/2833620840 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:47.704 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.706+0000 7fb6d8de3640 1 --2- 172.21.15.49:0/2833620840 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb6d415a8c0 0x7fb6d415ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:47.705 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.706+0000 7fb6d3fff640 1 -- 172.21.15.49:0/2833620840 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb6d415a8c0 msgr2=0x7fb6d415ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:47.705 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.706+0000 7fb6d3fff640 1 --2- 172.21.15.49:0/2833620840 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb6d415a8c0 0x7fb6d415ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:47.705 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.706+0000 7fb6d3fff640 1 -- 172.21.15.49:0/2833620840 >> v1:172.21.15.49:6789/0 conn(0x7fb6d415b1e0 legacy=0x7fb6d40cb830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:47.705 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.706+0000 7fb6d3fff640 1 -- 172.21.15.49:0/2833620840 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6d415bba0 con 0x7fb6d40ca110 2023-11-06T02:08:47.709 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.706+0000 7fb6d3fff640 1 --2- 172.21.15.49:0/2833620840 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6d40ca110 0x7fb6d415aca0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb6c8009870 tx=0x7fb6c801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5110605cd00ccba2 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:47.709 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.706+0000 7fb6d37fe640 1 -- 172.21.15.49:0/2833620840 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c80ab680 con 0x7fb6d40ca110 2023-11-06T02:08:47.709 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.706+0000 7fb6d37fe640 1 -- 172.21.15.49:0/2833620840 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6c80ab7e0 con 0x7fb6d40ca110 2023-11-06T02:08:47.710 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.706+0000 7fb6d37fe640 1 -- 172.21.15.49:0/2833620840 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c80abb70 con 0x7fb6d40ca110 2023-11-06T02:08:47.710 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.710+0000 7fb6db047640 1 -- 172.21.15.49:0/2833620840 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6d40ca110 msgr2=0x7fb6d415aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:47.710 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.710+0000 7fb6db047640 1 --2- 172.21.15.49:0/2833620840 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6d40ca110 0x7fb6d415aca0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb6c8009870 tx=0x7fb6c801c480 comp rx=0 tx=0).stop 2023-11-06T02:08:47.710 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.710+0000 7fb6db047640 1 -- 172.21.15.49:0/2833620840 shutdown_connections 2023-11-06T02:08:47.711 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.710+0000 7fb6db047640 1 --2- 172.21.15.49:0/2833620840 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6d40ca110 0x7fb6d415aca0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:47.711 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.710+0000 7fb6db047640 1 --2- 172.21.15.49:0/2833620840 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb6d415a8c0 0x7fb6d415ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:47.711 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.710+0000 7fb6db047640 1 -- 172.21.15.49:0/2833620840 >> 172.21.15.49:0/2833620840 conn(0x7fb6d40b0bd0 msgr2=0x7fb6d40b0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:47.711 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.710+0000 7fb6db047640 1 -- 172.21.15.49:0/2833620840 shutdown_connections 2023-11-06T02:08:47.711 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.710+0000 7fb6db047640 1 -- 172.21.15.49:0/2833620840 wait complete. 2023-11-06T02:08:47.712 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.710+0000 7fb6db047640 1 Processor -- start 2023-11-06T02:08:47.712 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.710+0000 7fb6db047640 1 -- start start 2023-11-06T02:08:47.712 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.710+0000 7fb6db047640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb6d40ca110 0x7fb6d4154f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:47.712 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.710+0000 7fb6db047640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb6d415a8c0 0x7fb6d40b3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:47.713 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.710+0000 7fb6db047640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6d415b1e0 0x7fb6d415cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:47.713 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.710+0000 7fb6db047640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d4154a90 con 0x7fb6d415b1e0 2023-11-06T02:08:47.713 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.710+0000 7fb6db047640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d4154180 con 0x7fb6d40ca110 2023-11-06T02:08:47.713 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.710+0000 7fb6db047640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d4154440 con 0x7fb6d415a8c0 2023-11-06T02:08:47.714 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.710+0000 7fb6d95e4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6d415b1e0 0x7fb6d415cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:47.714 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.710+0000 7fb6d95e4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6d415b1e0 0x7fb6d415cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52686/0 (socket says 172.21.15.49:52686) 2023-11-06T02:08:47.714 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.710+0000 7fb6d95e4640 1 -- 172.21.15.49:0/854009189 learned_addr learned my addr 172.21.15.49:0/854009189 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:47.714 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.710+0000 7fb6d8de3640 1 --2- 172.21.15.49:0/854009189 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb6d40ca110 0x7fb6d4154f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:47.715 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.710+0000 7fb6d3fff640 1 --2- 172.21.15.49:0/854009189 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb6d415a8c0 0x7fb6d40b3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:47.715 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.710+0000 7fb6d95e4640 1 -- 172.21.15.49:0/854009189 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb6d415a8c0 msgr2=0x7fb6d40b3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:47.715 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.710+0000 7fb6d95e4640 1 --2- 172.21.15.49:0/854009189 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb6d415a8c0 0x7fb6d40b3830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:47.715 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.710+0000 7fb6d95e4640 1 -- 172.21.15.49:0/854009189 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb6d40ca110 msgr2=0x7fb6d4154f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:47.715 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.710+0000 7fb6d95e4640 1 --2- 172.21.15.49:0/854009189 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb6d40ca110 0x7fb6d4154f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:47.716 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.710+0000 7fb6d95e4640 1 -- 172.21.15.49:0/854009189 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6c80094d0 con 0x7fb6d415b1e0 2023-11-06T02:08:47.716 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.710+0000 7fb6d95e4640 1 --2- 172.21.15.49:0/854009189 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6d415b1e0 0x7fb6d415cb20 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc00c430 tx=0x7fb6cc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:47.716 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.710+0000 7fb6d17fa640 1 -- 172.21.15.49:0/854009189 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6cc005220 con 0x7fb6d415b1e0 2023-11-06T02:08:47.716 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.710+0000 7fb6d17fa640 1 -- 172.21.15.49:0/854009189 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6cc005380 con 0x7fb6d415b1e0 2023-11-06T02:08:47.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.710+0000 7fb6d17fa640 1 -- 172.21.15.49:0/854009189 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6cc007270 con 0x7fb6d415b1e0 2023-11-06T02:08:47.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.710+0000 7fb6db047640 1 -- 172.21.15.49:0/854009189 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6d4154720 con 0x7fb6d415b1e0 2023-11-06T02:08:47.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.710+0000 7fb6db047640 1 -- 172.21.15.49:0/854009189 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6d40b3d70 con 0x7fb6d415b1e0 2023-11-06T02:08:47.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.714+0000 7fb6db047640 1 -- 172.21.15.49:0/854009189 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6d4154c00 con 0x7fb6d415b1e0 2023-11-06T02:08:47.718 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.714+0000 7fb6d17fa640 1 -- 172.21.15.49:0/854009189 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fb6cc0073d0 con 0x7fb6d415b1e0 2023-11-06T02:08:47.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.714+0000 7fb6d17fa640 1 --2- 172.21.15.49:0/854009189 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb6c0065a60 0x7fb6c0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:47.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.714+0000 7fb6d17fa640 1 -- 172.21.15.49:0/854009189 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb6cc082910 con 0x7fb6d415b1e0 2023-11-06T02:08:47.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.714+0000 7fb6d8de3640 1 --2- 172.21.15.49:0/854009189 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb6c0065a60 0x7fb6c0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:47.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.714+0000 7fb6d8de3640 1 --2- 172.21.15.49:0/854009189 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb6c0065a60 0x7fb6c0067f20 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb6c4003fd0 tx=0x7fb6c40039a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:47.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.718+0000 7fb6d17fa640 1 -- 172.21.15.49:0/854009189 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fb6cc050160 con 0x7fb6d415b1e0 2023-11-06T02:08:47.865 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.870+0000 7fb6db047640 1 -- 172.21.15.49:0/854009189 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb6d4067140 con 0x7fb6d415b1e0 2023-11-06T02:08:47.867 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.870+0000 7fb6d17fa640 1 -- 172.21.15.49:0/854009189 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+17811 (secure 0 0 0) 0x7fb6cc053780 con 0x7fb6d415b1e0 2023-11-06T02:08:47.867 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:08:47.868 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":23,"fsid":"1a8b035c-6ba4-499a-b876-aea47fd9c0d4","created":"2023-11-06T02:08:06.440857+0000","modified":"2023-11-06T02:08:31.594964+0000","last_up_change":"2023-11-06T02:08:17.982372+0000","last_in_change":"2023-11-06T02:08:08.906631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T02:08:19.252222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T02:08:24.776211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-06T02:08:28.936178+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-06T02:08:29.563468+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ad55af8-0571-4e6a-8722-b5d429b7496d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6809","nonce":38116},{"type":"v1","addr":"172.21.15.49:6811","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6813","nonce":38116},{"type":"v1","addr":"172.21.15.49:6815","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6821","nonce":38116},{"type":"v1","addr":"172.21.15.49:6823","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6817","nonce":38116},{"type":"v1","addr":"172.21.15.49:6819","nonce":38116}]},"public_addr":"172.21.15.49:6811/38116","cluster_addr":"172.21.15.49:6815/38116","heartbeat_back_addr":"172.21.15.49:6823/38116","heartbeat_front_addr":"172.21.15.49:6819/38116","state":["exists","up"]},{"osd":1,"uuid":"c8b31f85-a757-43ed-9419-351acc46611b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":38113},{"type":"v1","addr":"172.21.15.49:6810","nonce":38113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":38113},{"type":"v1","addr":"172.21.15.49:6814","nonce":38113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":38113},{"type":"v1","addr":"172.21.15.49:6822","nonce":38113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":38113},{"type":"v1","addr":"172.21.15.49:6818","nonce":38113}]},"public_addr":"172.21.15.49:6810/38113","cluster_addr":"172.21.15.49:6814/38113","heartbeat_back_addr":"172.21.15.49:6822/38113","heartbeat_front_addr":"172.21.15.49:6818/38113","state":["exists","up"]},{"osd":2,"uuid":"555b3bf3-0a70-4b91-8efd-e4204b5fc26e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":38114},{"type":"v1","addr":"172.21.15.49:6801","nonce":38114}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":38114},{"type":"v1","addr":"172.21.15.49:6803","nonce":38114}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":38114},{"type":"v1","addr":"172.21.15.49:6807","nonce":38114}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":38114},{"type":"v1","addr":"172.21.15.49:6805","nonce":38114}]},"public_addr":"172.21.15.49:6801/38114","cluster_addr":"172.21.15.49:6803/38114","heartbeat_back_addr":"172.21.15.49:6807/38114","heartbeat_front_addr":"172.21.15.49:6805/38114","state":["exists","up"]},{"osd":3,"uuid":"8d4e5d77-03ef-4b9b-bfcb-5ae93a46795c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":38115},{"type":"v1","addr":"172.21.15.49:6825","nonce":38115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":38115},{"type":"v1","addr":"172.21.15.49:6827","nonce":38115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":38115},{"type":"v1","addr":"172.21.15.49:6831","nonce":38115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":38115},{"type":"v1","addr":"172.21.15.49:6829","nonce":38115}]},"public_addr":"172.21.15.49:6825/38115","cluster_addr":"172.21.15.49:6827/38115","heartbeat_back_addr":"172.21.15.49:6831/38115","heartbeat_front_addr":"172.21.15.49:6829/38115","state":["exists","up"]},{"osd":4,"uuid":"dba03ad4-68a3-4cf3-9218-ac611f02eacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":38417},{"type":"v1","addr":"172.21.15.150:6809","nonce":38417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":38417},{"type":"v1","addr":"172.21.15.150:6811","nonce":38417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":38417},{"type":"v1","addr":"172.21.15.150:6815","nonce":38417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":38417},{"type":"v1","addr":"172.21.15.150:6813","nonce":38417}]},"public_addr":"172.21.15.150:6809/38417","cluster_addr":"172.21.15.150:6811/38417","heartbeat_back_addr":"172.21.15.150:6815/38417","heartbeat_front_addr":"172.21.15.150:6813/38417","state":["exists","up"]},{"osd":5,"uuid":"0df9c303-c255-4640-8e91-01f1c3efe18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":38418},{"type":"v1","addr":"172.21.15.150:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":38418},{"type":"v1","addr":"172.21.15.150:6803","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":38418},{"type":"v1","addr":"172.21.15.150:6807","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":38418},{"type":"v1","addr":"172.21.15.150:6805","nonce":38418}]},"public_addr":"172.21.15.150:6801/38418","cluster_addr":"172.21.15.150:6803/38418","heartbeat_back_addr":"172.21.15.150:6807/38418","heartbeat_front_addr":"172.21.15.150:6805/38418","state":["exists","up"]},{"osd":6,"uuid":"e59fa189-4b70-4c06-9722-c02b90828bf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":38428},{"type":"v1","addr":"172.21.15.150:6825","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":38428},{"type":"v1","addr":"172.21.15.150:6827","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":38428},{"type":"v1","addr":"172.21.15.150:6831","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":38428},{"type":"v1","addr":"172.21.15.150:6829","nonce":38428}]},"public_addr":"172.21.15.150:6825/38428","cluster_addr":"172.21.15.150:6827/38428","heartbeat_back_addr":"172.21.15.150:6831/38428","heartbeat_front_addr":"172.21.15.150:6829/38428","state":["exists","up"]},{"osd":7,"uuid":"a64f57e1-ef98-46bd-b304-385759e95898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":38421},{"type":"v1","addr":"172.21.15.150:6817","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":38421},{"type":"v1","addr":"172.21.15.150:6819","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":38421},{"type":"v1","addr":"172.21.15.150:6823","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":38421},{"type":"v1","addr":"172.21.15.150:6821","nonce":38421}]},"public_addr":"172.21.15.150:6817/38421","cluster_addr":"172.21.15.150:6819/38421","heartbeat_back_addr":"172.21.15.150:6823/38421","heartbeat_front_addr":"172.21.15.150:6821/38421","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.438552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.413280+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.296466+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.424966+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.276953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.230984+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.410235+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.480084+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T02:08:47.869 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.874+0000 7fb6db047640 1 -- 172.21.15.49:0/854009189 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb6c0065a60 msgr2=0x7fb6c0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:47.869 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.874+0000 7fb6db047640 1 --2- 172.21.15.49:0/854009189 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb6c0065a60 0x7fb6c0067f20 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb6c4003fd0 tx=0x7fb6c40039a0 comp rx=0 tx=0).stop 2023-11-06T02:08:47.870 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.874+0000 7fb6db047640 1 -- 172.21.15.49:0/854009189 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6d415b1e0 msgr2=0x7fb6d415cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:47.870 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.874+0000 7fb6db047640 1 --2- 172.21.15.49:0/854009189 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6d415b1e0 0x7fb6d415cb20 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc00c430 tx=0x7fb6cc00c8f0 comp rx=0 tx=0).stop 2023-11-06T02:08:47.870 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.874+0000 7fb6db047640 1 -- 172.21.15.49:0/854009189 shutdown_connections 2023-11-06T02:08:47.870 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.874+0000 7fb6db047640 1 --2- 172.21.15.49:0/854009189 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb6c0065a60 0x7fb6c0067f20 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:47.871 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.874+0000 7fb6db047640 1 --2- 172.21.15.49:0/854009189 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6d415b1e0 0x7fb6d415cb20 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:47.871 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.874+0000 7fb6db047640 1 --2- 172.21.15.49:0/854009189 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb6d415a8c0 0x7fb6d40b3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:47.871 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.874+0000 7fb6db047640 1 --2- 172.21.15.49:0/854009189 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb6d40ca110 0x7fb6d4154f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:47.871 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.874+0000 7fb6db047640 1 -- 172.21.15.49:0/854009189 >> 172.21.15.49:0/854009189 conn(0x7fb6d40b0bd0 msgr2=0x7fb6d40c90a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:47.871 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.874+0000 7fb6db047640 1 -- 172.21.15.49:0/854009189 shutdown_connections 2023-11-06T02:08:47.872 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:47.874+0000 7fb6db047640 1 -- 172.21.15.49:0/854009189 wait complete. 2023-11-06T02:08:47.885 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T02:08:47.999 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.002+0000 7ff80f367640 1 Processor -- start 2023-11-06T02:08:47.999 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.002+0000 7ff80f367640 1 -- start start 2023-11-06T02:08:47.999 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.002+0000 7ff80f367640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7ff8080b1f90 0x7ff8080cda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:48.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.002+0000 7ff80f367640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff8080714a0 0x7ff8080718b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:48.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.002+0000 7ff80f367640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8080d17c0 con 0x7ff808161650 2023-11-06T02:08:48.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.002+0000 7ff80f367640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff808161b00 con 0x7ff8080714a0 2023-11-06T02:08:48.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.002+0000 7ff80f367640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff808071df0 con 0x7ff8080b1f90 2023-11-06T02:08:48.001 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.002+0000 7ff80d904640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff8080714a0 0x7ff8080718b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:48.001 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.002+0000 7ff80d904640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff8080714a0 0x7ff8080718b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52696/0 (socket says 172.21.15.49:52696) 2023-11-06T02:08:48.001 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.002+0000 7ff80d904640 1 -- 172.21.15.49:0/32225141 learned_addr learned my addr 172.21.15.49:0/32225141 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:48.001 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.002+0000 7ff80d103640 1 --2- 172.21.15.49:0/32225141 >> v2:172.21.15.150:3300/0 conn(0x7ff8080b1f90 0x7ff8080cda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:48.001 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.002+0000 7ff80d904640 1 -- 172.21.15.49:0/32225141 >> v2:172.21.15.150:3300/0 conn(0x7ff8080b1f90 msgr2=0x7ff8080cda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:48.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.002+0000 7ff80d904640 1 --2- 172.21.15.49:0/32225141 >> v2:172.21.15.150:3300/0 conn(0x7ff8080b1f90 0x7ff8080cda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:48.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.002+0000 7ff80d904640 1 -- 172.21.15.49:0/32225141 >> v1:172.21.15.150:6789/0 conn(0x7ff808161650 legacy=0x7ff8080d1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:48.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.002+0000 7ff80d904640 1 -- 172.21.15.49:0/32225141 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8080ce6c0 con 0x7ff8080714a0 2023-11-06T02:08:48.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.002+0000 7ff80d103640 1 --2- 172.21.15.49:0/32225141 >> v2:172.21.15.150:3300/0 conn(0x7ff8080b1f90 0x7ff8080cda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:08:48.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.006+0000 7ff80d904640 1 --2- 172.21.15.49:0/32225141 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8080714a0 0x7ff8080718b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff808072180 tx=0x7ff80001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2558fccdd516e582 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:48.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.006+0000 7ff7fffff640 1 -- 172.21.15.49:0/32225141 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8000a5830 con 0x7ff8080714a0 2023-11-06T02:08:48.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.006+0000 7ff7fffff640 1 -- 172.21.15.49:0/32225141 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8000a5990 con 0x7ff8080714a0 2023-11-06T02:08:48.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.006+0000 7ff7fffff640 1 -- 172.21.15.49:0/32225141 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8000a5d20 con 0x7ff8080714a0 2023-11-06T02:08:48.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.006+0000 7ff80f367640 1 -- 172.21.15.49:0/32225141 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8080714a0 msgr2=0x7ff8080718b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:48.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.006+0000 7ff80f367640 1 --2- 172.21.15.49:0/32225141 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8080714a0 0x7ff8080718b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff808072180 tx=0x7ff80001c970 comp rx=0 tx=0).stop 2023-11-06T02:08:48.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.006+0000 7ff80f367640 1 -- 172.21.15.49:0/32225141 shutdown_connections 2023-11-06T02:08:48.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.006+0000 7ff80f367640 1 --2- 172.21.15.49:0/32225141 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8080714a0 0x7ff8080718b0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:48.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.006+0000 7ff80f367640 1 --2- 172.21.15.49:0/32225141 >> v2:172.21.15.150:3300/0 conn(0x7ff8080b1f90 0x7ff8080cda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:48.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.006+0000 7ff80f367640 1 -- 172.21.15.49:0/32225141 >> 172.21.15.49:0/32225141 conn(0x7ff8080d2aa0 msgr2=0x7ff8080d2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:48.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.006+0000 7ff80f367640 1 -- 172.21.15.49:0/32225141 shutdown_connections 2023-11-06T02:08:48.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.006+0000 7ff80f367640 1 -- 172.21.15.49:0/32225141 wait complete. 2023-11-06T02:08:48.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.006+0000 7ff80f367640 1 Processor -- start 2023-11-06T02:08:48.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.006+0000 7ff80f367640 1 -- start start 2023-11-06T02:08:48.009 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.006+0000 7ff80f367640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8080714a0 0x7ff80815b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:48.009 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.006+0000 7ff80f367640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8080b1f90 0x7ff80815dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:48.009 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.006+0000 7ff80f367640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff808161650 0x7ff8080c8350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:48.010 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.006+0000 7ff80f367640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff80815e850 con 0x7ff808161650 2023-11-06T02:08:48.010 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.006+0000 7ff80f367640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff80815eb30 con 0x7ff8080714a0 2023-11-06T02:08:48.010 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.006+0000 7ff80f367640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff80815ee10 con 0x7ff8080b1f90 2023-11-06T02:08:48.010 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.006+0000 7ff80d904640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff808161650 0x7ff8080c8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:48.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.006+0000 7ff80d904640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff808161650 0x7ff8080c8350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52704/0 (socket says 172.21.15.49:52704) 2023-11-06T02:08:48.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.010+0000 7ff80d904640 1 -- 172.21.15.49:0/3300413363 learned_addr learned my addr 172.21.15.49:0/3300413363 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:48.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.010+0000 7ff80d103640 1 --2- 172.21.15.49:0/3300413363 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8080714a0 0x7ff80815b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:48.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.010+0000 7ff80c902640 1 --2- 172.21.15.49:0/3300413363 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8080b1f90 0x7ff80815dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:48.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.010+0000 7ff80d904640 1 -- 172.21.15.49:0/3300413363 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8080b1f90 msgr2=0x7ff80815dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:48.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.010+0000 7ff80d904640 1 --2- 172.21.15.49:0/3300413363 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8080b1f90 0x7ff80815dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:48.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.010+0000 7ff80d904640 1 -- 172.21.15.49:0/3300413363 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8080714a0 msgr2=0x7ff80815b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:48.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.010+0000 7ff80d904640 1 --2- 172.21.15.49:0/3300413363 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8080714a0 0x7ff80815b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:48.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.010+0000 7ff80d904640 1 -- 172.21.15.49:0/3300413363 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff800009b30 con 0x7ff808161650 2023-11-06T02:08:48.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.010+0000 7ff80c902640 1 --2- 172.21.15.49:0/3300413363 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8080b1f90 0x7ff80815dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:08:48.013 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.010+0000 7ff80d904640 1 --2- 172.21.15.49:0/3300413363 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff808161650 0x7ff8080c8350 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff8000af040 tx=0x7ff800019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:48.013 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.010+0000 7ff7fdffb640 1 -- 172.21.15.49:0/3300413363 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8000b8880 con 0x7ff808161650 2023-11-06T02:08:48.013 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.010+0000 7ff7fdffb640 1 -- 172.21.15.49:0/3300413363 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8000b8a00 con 0x7ff808161650 2023-11-06T02:08:48.013 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.010+0000 7ff7fdffb640 1 -- 172.21.15.49:0/3300413363 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8000b8db0 con 0x7ff808161650 2023-11-06T02:08:48.013 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.010+0000 7ff80f367640 1 -- 172.21.15.49:0/3300413363 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff80815f0f0 con 0x7ff808161650 2023-11-06T02:08:48.014 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.010+0000 7ff80f367640 1 -- 172.21.15.49:0/3300413363 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff80815b5d0 con 0x7ff808161650 2023-11-06T02:08:48.014 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.010+0000 7ff80f367640 1 -- 172.21.15.49:0/3300413363 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff808074040 con 0x7ff808161650 2023-11-06T02:08:48.014 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.010+0000 7ff7fdffb640 1 -- 172.21.15.49:0/3300413363 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7ff8000c3460 con 0x7ff808161650 2023-11-06T02:08:48.014 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.010+0000 7ff7fdffb640 1 --2- 172.21.15.49:0/3300413363 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff7ec065a60 0x7ff7ec067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:48.015 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.010+0000 7ff7fdffb640 1 -- 172.21.15.49:0/3300413363 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff80011ea20 con 0x7ff808161650 2023-11-06T02:08:48.015 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.010+0000 7ff80d103640 1 --2- 172.21.15.49:0/3300413363 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff7ec065a60 0x7ff7ec067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:48.015 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.010+0000 7ff80d103640 1 --2- 172.21.15.49:0/3300413363 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff7ec065a60 0x7ff7ec067f20 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff8080b1970 tx=0x7ff804008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:48.015 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.014+0000 7ff7fdffb640 1 -- 172.21.15.49:0/3300413363 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7ff8000ec450 con 0x7ff808161650 2023-11-06T02:08:48.200 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.202+0000 7ff80f367640 1 -- 172.21.15.49:0/3300413363 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff80815ef50 con 0x7ff808161650 2023-11-06T02:08:48.201 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.206+0000 7ff7fdffb640 1 -- 172.21.15.49:0/3300413363 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4648 (secure 0 0 0) 0x7ff80815ef50 con 0x7ff808161650 2023-11-06T02:08:48.202 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:08:48.202 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6834/1377586971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1377586971},{"type":"v1","addr":"172.21.15.49:6834","nonce":1377586971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4288,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2620584959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2620584959},{"type":"v1","addr":"172.21.15.150:6835","nonce":2620584959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4291,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/4272191418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":4272191418},{"type":"v1","addr":"172.21.15.150:6837","nonce":4272191418}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T02:08:30.592185+0000","modified":"2023-11-06T02:08:33.245446+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4293},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4293":{"gid":4293,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.49:6835/4280448297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6833","nonce":4280448297},{"type":"v1","addr":"172.21.15.49:6835","nonce":4280448297}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-06T02:08:48.202 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 7 2023-11-06T02:08:48.203 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.206+0000 7ff80f367640 1 -- 172.21.15.49:0/3300413363 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff7ec065a60 msgr2=0x7ff7ec067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:48.204 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.206+0000 7ff80f367640 1 --2- 172.21.15.49:0/3300413363 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff7ec065a60 0x7ff7ec067f20 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff8080b1970 tx=0x7ff804008040 comp rx=0 tx=0).stop 2023-11-06T02:08:48.204 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.206+0000 7ff80f367640 1 -- 172.21.15.49:0/3300413363 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff808161650 msgr2=0x7ff8080c8350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:48.204 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.206+0000 7ff80f367640 1 --2- 172.21.15.49:0/3300413363 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff808161650 0x7ff8080c8350 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff8000af040 tx=0x7ff800019710 comp rx=0 tx=0).stop 2023-11-06T02:08:48.204 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.206+0000 7ff80f367640 1 -- 172.21.15.49:0/3300413363 shutdown_connections 2023-11-06T02:08:48.204 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.206+0000 7ff80f367640 1 --2- 172.21.15.49:0/3300413363 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff7ec065a60 0x7ff7ec067f20 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:48.205 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.206+0000 7ff80f367640 1 --2- 172.21.15.49:0/3300413363 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff808161650 0x7ff8080c8350 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:48.205 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.206+0000 7ff80f367640 1 --2- 172.21.15.49:0/3300413363 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8080b1f90 0x7ff80815dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:48.205 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.206+0000 7ff80f367640 1 --2- 172.21.15.49:0/3300413363 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8080714a0 0x7ff80815b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:48.205 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.206+0000 7ff80f367640 1 -- 172.21.15.49:0/3300413363 >> 172.21.15.49:0/3300413363 conn(0x7ff8080d2aa0 msgr2=0x7ff8080d1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:48.205 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.206+0000 7ff80f367640 1 -- 172.21.15.49:0/3300413363 shutdown_connections 2023-11-06T02:08:48.206 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.206+0000 7ff80f367640 1 -- 172.21.15.49:0/3300413363 wait complete. 2023-11-06T02:08:48.623 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-06T02:08:30.592185+0000', 'modified': '2023-11-06T02:08:33.245446+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4293}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4293': {'gid': 4293, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.49:6835/4280448297', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6833', 'nonce': 4280448297}, {'type': 'v1', 'addr': '172.21.15.49:6835', 'nonce': 4280448297}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-06T02:08:48.624 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-06T02:08:48.624 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-06T02:08:48.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.762+0000 7fbf0d53c640 1 Processor -- start 2023-11-06T02:08:48.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.762+0000 7fbf0d53c640 1 -- start start 2023-11-06T02:08:48.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.762+0000 7fbf0d53c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fbf080b1f90 0x7fbf080cda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:48.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.762+0000 7fbf0d53c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf080714a0 0x7fbf080718d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:48.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.762+0000 7fbf0d53c640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf080d17c0 con 0x7fbf08161650 2023-11-06T02:08:48.762 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.762+0000 7fbf0d53c640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf08071e10 con 0x7fbf080b1f90 2023-11-06T02:08:48.762 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.762+0000 7fbf0d53c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf08161b00 con 0x7fbf080714a0 2023-11-06T02:08:48.762 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.762+0000 7fbf06ffd640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fbf080b1f90 0x7fbf080cda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:48.762 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.766+0000 7fbf06ffd640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fbf080b1f90 0x7fbf080cda70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47782/0 (socket says 172.21.15.49:47782) 2023-11-06T02:08:48.763 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.766+0000 7fbf06ffd640 1 -- 172.21.15.49:0/2019551914 learned_addr learned my addr 172.21.15.49:0/2019551914 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:48.763 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.766+0000 7fbf077fe640 1 --2- 172.21.15.49:0/2019551914 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf080714a0 0x7fbf080718d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:48.763 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.766+0000 7fbf06ffd640 1 -- 172.21.15.49:0/2019551914 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf080714a0 msgr2=0x7fbf080718d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:48.763 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.766+0000 7fbf06ffd640 1 --2- 172.21.15.49:0/2019551914 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf080714a0 0x7fbf080718d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:48.763 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.766+0000 7fbf06ffd640 1 -- 172.21.15.49:0/2019551914 >> v1:172.21.15.150:6789/0 conn(0x7fbf08161650 legacy=0x7fbf080d1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:48.764 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.766+0000 7fbf06ffd640 1 -- 172.21.15.49:0/2019551914 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf080ce6c0 con 0x7fbf080b1f90 2023-11-06T02:08:48.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.766+0000 7fbf06ffd640 1 --2- 172.21.15.49:0/2019551914 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf080b1f90 0x7fbf080cda70 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fbef8009870 tx=0x7fbef801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c35189a62f0514d2 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:48.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.766+0000 7fbf067fc640 1 -- 172.21.15.49:0/2019551914 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef80aa680 con 0x7fbf080b1f90 2023-11-06T02:08:48.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.766+0000 7fbf067fc640 1 -- 172.21.15.49:0/2019551914 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbef80aa7e0 con 0x7fbf080b1f90 2023-11-06T02:08:48.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.766+0000 7fbf067fc640 1 -- 172.21.15.49:0/2019551914 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef80aab70 con 0x7fbf080b1f90 2023-11-06T02:08:48.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.766+0000 7fbf0d53c640 1 -- 172.21.15.49:0/2019551914 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf080b1f90 msgr2=0x7fbf080cda70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:48.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.766+0000 7fbf0d53c640 1 --2- 172.21.15.49:0/2019551914 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf080b1f90 0x7fbf080cda70 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fbef8009870 tx=0x7fbef801c520 comp rx=0 tx=0).stop 2023-11-06T02:08:48.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.766+0000 7fbf0d53c640 1 -- 172.21.15.49:0/2019551914 shutdown_connections 2023-11-06T02:08:48.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.766+0000 7fbf0d53c640 1 --2- 172.21.15.49:0/2019551914 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf080714a0 0x7fbf080718d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:48.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.766+0000 7fbf0d53c640 1 --2- 172.21.15.49:0/2019551914 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf080b1f90 0x7fbf080cda70 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:48.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.766+0000 7fbf0d53c640 1 -- 172.21.15.49:0/2019551914 >> 172.21.15.49:0/2019551914 conn(0x7fbf080d2aa0 msgr2=0x7fbf080d2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:48.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.766+0000 7fbf0d53c640 1 -- 172.21.15.49:0/2019551914 shutdown_connections 2023-11-06T02:08:48.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.766+0000 7fbf0d53c640 1 -- 172.21.15.49:0/2019551914 wait complete. 2023-11-06T02:08:48.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.770+0000 7fbf0d53c640 1 Processor -- start 2023-11-06T02:08:48.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.770+0000 7fbf0d53c640 1 -- start start 2023-11-06T02:08:48.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.770+0000 7fbf0d53c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf080714a0 0x7fbf080b3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:48.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.770+0000 7fbf0d53c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf080b1f90 0x7fbf080cc630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:48.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.770+0000 7fbf0d53c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf08161650 0x7fbf08155b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:48.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.770+0000 7fbf0d53c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf0815e270 con 0x7fbf080714a0 2023-11-06T02:08:48.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.770+0000 7fbf0d53c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf080cb190 con 0x7fbf08161650 2023-11-06T02:08:48.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.770+0000 7fbf0d53c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf080caeb0 con 0x7fbf080b1f90 2023-11-06T02:08:48.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.770+0000 7fbf06ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf080714a0 0x7fbf080b3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:48.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.770+0000 7fbf06ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf080714a0 0x7fbf080b3670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47786/0 (socket says 172.21.15.49:47786) 2023-11-06T02:08:48.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.770+0000 7fbf06ffd640 1 -- 172.21.15.49:0/2797887938 learned_addr learned my addr 172.21.15.49:0/2797887938 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:48.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.770+0000 7fbf077fe640 1 --2- 172.21.15.49:0/2797887938 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf08161650 0x7fbf08155b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:48.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.770+0000 7fbefffff640 1 --2- 172.21.15.49:0/2797887938 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf080b1f90 0x7fbf080cc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:48.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.770+0000 7fbf06ffd640 1 -- 172.21.15.49:0/2797887938 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf080b1f90 msgr2=0x7fbf080cc630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:48.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.770+0000 7fbf06ffd640 1 --2- 172.21.15.49:0/2797887938 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf080b1f90 0x7fbf080cc630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:48.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.770+0000 7fbf06ffd640 1 -- 172.21.15.49:0/2797887938 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf08161650 msgr2=0x7fbf08155b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:48.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.770+0000 7fbf06ffd640 1 --2- 172.21.15.49:0/2797887938 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf08161650 0x7fbf08155b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:48.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.770+0000 7fbf06ffd640 1 -- 172.21.15.49:0/2797887938 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbef80094d0 con 0x7fbf080714a0 2023-11-06T02:08:48.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.770+0000 7fbf06ffd640 1 --2- 172.21.15.49:0/2797887938 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf080714a0 0x7fbf080b3670 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fbef80ac040 tx=0x7fbef801ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:48.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.770+0000 7fbeff7fe640 1 -- 172.21.15.49:0/2797887938 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef80b3800 con 0x7fbf080714a0 2023-11-06T02:08:48.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.770+0000 7fbeff7fe640 1 -- 172.21.15.49:0/2797887938 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbef80b3980 con 0x7fbf080714a0 2023-11-06T02:08:48.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.770+0000 7fbf0d53c640 1 -- 172.21.15.49:0/2797887938 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf0815e510 con 0x7fbf080714a0 2023-11-06T02:08:48.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.770+0000 7fbeff7fe640 1 -- 172.21.15.49:0/2797887938 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef80b3d30 con 0x7fbf080714a0 2023-11-06T02:08:48.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.770+0000 7fbf0d53c640 1 -- 172.21.15.49:0/2797887938 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf08156700 con 0x7fbf080714a0 2023-11-06T02:08:48.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.770+0000 7fbeff7fe640 1 -- 172.21.15.49:0/2797887938 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fbef80c6390 con 0x7fbf080714a0 2023-11-06T02:08:48.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.774+0000 7fbeff7fe640 1 --2- 172.21.15.49:0/2797887938 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fbeec065a90 0x7fbeec067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:48.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.774+0000 7fbeff7fe640 1 -- 172.21.15.49:0/2797887938 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbef811e020 con 0x7fbf080714a0 2023-11-06T02:08:48.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.774+0000 7fbefffff640 1 --2- 172.21.15.49:0/2797887938 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fbeec065a90 0x7fbeec067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:48.781 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.774+0000 7fbf10664000 1 Processor -- start 2023-11-06T02:08:48.781 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.774+0000 7fbf10664000 1 -- start start 2023-11-06T02:08:48.781 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.774+0000 7fbefffff640 1 --2- 172.21.15.49:0/2797887938 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fbeec065a90 0x7fbeec067f50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbf080b18d0 tx=0x7fbef0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:48.782 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.774+0000 7fbf10664000 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5628651846e0 0x562865184ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:48.782 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.774+0000 7fbf10664000 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562865185000 0x56286513a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:48.782 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.774+0000 7fbf10664000 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56286513aa90 0x56286513ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:48.782 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.774+0000 7fbf10664000 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x562865002fc0 con 0x5628651846e0 2023-11-06T02:08:48.782 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.774+0000 7fbf10664000 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x562864f911a0 con 0x562865185000 2023-11-06T02:08:48.783 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.774+0000 7fbf10664000 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x562864f8e770 con 0x56286513aa90 2023-11-06T02:08:48.783 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.774+0000 7fbf06ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5628651846e0 0x562865184ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:48.783 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.774+0000 7fbf06ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5628651846e0 0x562865184ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47788/0 (socket says 172.21.15.49:47788) 2023-11-06T02:08:48.783 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.774+0000 7fbf06ffd640 1 -- 172.21.15.49:0/594769444 learned_addr learned my addr 172.21.15.49:0/594769444 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:48.783 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.774+0000 7fbf077fe640 1 --2- 172.21.15.49:0/594769444 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56286513aa90 0x56286513ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:48.783 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.774+0000 7fbefffff640 1 --2- 172.21.15.49:0/594769444 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562865185000 0x56286513a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:48.784 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.774+0000 7fbf06ffd640 1 -- 172.21.15.49:0/594769444 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56286513aa90 msgr2=0x56286513ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:48.784 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.774+0000 7fbf06ffd640 1 --2- 172.21.15.49:0/594769444 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56286513aa90 0x56286513ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:48.784 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.774+0000 7fbf06ffd640 1 -- 172.21.15.49:0/594769444 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562865185000 msgr2=0x56286513a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:48.784 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.774+0000 7fbf06ffd640 1 --2- 172.21.15.49:0/594769444 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562865185000 0x56286513a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:48.784 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.774+0000 7fbf06ffd640 1 -- 172.21.15.49:0/594769444 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562864c86890 con 0x5628651846e0 2023-11-06T02:08:48.785 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.774+0000 7fbf06ffd640 1 --2- 172.21.15.49:0/594769444 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5628651846e0 0x562865184ac0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fbef80ebad0 tx=0x7fbef80efcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcc0f710ce38f350 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:48.785 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.774+0000 7fbefcff9640 1 -- 172.21.15.49:0/594769444 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef80f75b0 con 0x5628651846e0 2023-11-06T02:08:48.785 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.774+0000 7fbefcff9640 1 -- 172.21.15.49:0/594769444 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbef80f5040 con 0x5628651846e0 2023-11-06T02:08:48.785 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.774+0000 7fbefcff9640 1 -- 172.21.15.49:0/594769444 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef80f3070 con 0x5628651846e0 2023-11-06T02:08:48.785 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.774+0000 7fbf10664000 1 -- 172.21.15.49:0/594769444 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5628651846e0 msgr2=0x562865184ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:48.785 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.774+0000 7fbf10664000 1 --2- 172.21.15.49:0/594769444 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5628651846e0 0x562865184ac0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fbef80ebad0 tx=0x7fbef80efcd0 comp rx=0 tx=0).stop 2023-11-06T02:08:48.786 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.774+0000 7fbf10664000 1 -- 172.21.15.49:0/594769444 shutdown_connections 2023-11-06T02:08:48.786 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.774+0000 7fbf10664000 1 --2- 172.21.15.49:0/594769444 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56286513aa90 0x56286513ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:48.786 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.774+0000 7fbf10664000 1 --2- 172.21.15.49:0/594769444 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562865185000 0x56286513a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:48.786 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.774+0000 7fbf10664000 1 --2- 172.21.15.49:0/594769444 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5628651846e0 0x562865184ac0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:48.786 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.774+0000 7fbf10664000 1 -- 172.21.15.49:0/594769444 >> 172.21.15.49:0/594769444 conn(0x562865140bf0 msgr2=0x562865138e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:48.787 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.774+0000 7fbf10664000 1 -- 172.21.15.49:0/594769444 shutdown_connections 2023-11-06T02:08:48.787 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.774+0000 7fbf10664000 1 -- 172.21.15.49:0/594769444 wait complete. 2023-11-06T02:08:48.787 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.778+0000 7fbf10664000 1 -- start start 2023-11-06T02:08:48.787 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.778+0000 7fbf10664000 1 Processor -- start 2023-11-06T02:08:48.787 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.778+0000 7fbf10664000 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56286513aa90 0x562865145880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:48.788 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.778+0000 7fbf10664000 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5628651846e0 0x562865145dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:48.788 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.778+0000 7fbf10664000 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562865185000 0x562865146300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:48.788 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.778+0000 7fbf10664000 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x562864f8a060 con 0x56286513aa90 2023-11-06T02:08:48.788 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.778+0000 7fbf10664000 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x562864f29290 con 0x562865185000 2023-11-06T02:08:48.788 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.778+0000 7fbf10664000 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x562864f476f0 con 0x5628651846e0 2023-11-06T02:08:48.789 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.778+0000 7fbf10664000 10 client.0 _ll_register_callbacks cb 0x5628651261d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T02:08:48.789 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.778+0000 7fbf06ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56286513aa90 0x562865145880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:48.789 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.778+0000 7fbf06ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56286513aa90 0x562865145880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47796/0 (socket says 172.21.15.49:47796) 2023-11-06T02:08:48.789 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.778+0000 7fbf06ffd640 1 -- 172.21.15.49:0/2242584209 learned_addr learned my addr 172.21.15.49:0/2242584209 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:48.789 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.778+0000 7fbf077fe640 1 --2- 172.21.15.49:0/2242584209 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562865185000 0x562865146300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:48.789 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.778+0000 7fbefffff640 1 --2- 172.21.15.49:0/2242584209 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5628651846e0 0x562865145dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:48.790 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.778+0000 7fbf06ffd640 1 -- 172.21.15.49:0/2242584209 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5628651846e0 msgr2=0x562865145dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:48.790 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.778+0000 7fbf06ffd640 1 --2- 172.21.15.49:0/2242584209 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5628651846e0 0x562865145dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:48.790 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.778+0000 7fbf06ffd640 1 -- 172.21.15.49:0/2242584209 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562865185000 msgr2=0x562865146300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:48.790 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.778+0000 7fbf06ffd640 1 --2- 172.21.15.49:0/2242584209 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562865185000 0x562865146300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:48.790 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.778+0000 7fbf06ffd640 1 -- 172.21.15.49:0/2242584209 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf08156700 con 0x56286513aa90 2023-11-06T02:08:48.791 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.778+0000 7fbf06ffd640 1 --2- 172.21.15.49:0/2242584209 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56286513aa90 0x562865145880 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fbef80efe80 tx=0x7fbef8000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1db5dc5ea68d4fc server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:48.791 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.778+0000 7fbef5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-11-06T02:08:48.791 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.778+0000 7fbef5ffb640 1 -- 172.21.15.49:0/2242584209 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef80ebce0 con 0x56286513aa90 2023-11-06T02:08:48.791 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.778+0000 7fbef5ffb640 1 -- 172.21.15.49:0/2242584209 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbef80ebe40 con 0x56286513aa90 2023-11-06T02:08:48.791 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.778+0000 7fbf10664000 1 -- 172.21.15.49:0/2242584209 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562864f889b0 con 0x56286513aa90 2023-11-06T02:08:48.792 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.778+0000 7fbef5ffb640 1 -- 172.21.15.49:0/2242584209 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbef810d0d0 con 0x56286513aa90 2023-11-06T02:08:48.792 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.778+0000 7fbf10664000 10 client.4728 fetch_fsmap learned FSMap version 7 2023-11-06T02:08:48.792 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.778+0000 7fbf10664000 1 -- 172.21.15.49:0/2242584209 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x562864f4f1e0 con 0x56286513aa90 2023-11-06T02:08:48.792 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.778+0000 7fbef5ffb640 1 -- 172.21.15.49:0/2242584209 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562864f889b0 con 0x56286513aa90 2023-11-06T02:08:48.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.778+0000 7fbef5ffb640 1 -- 172.21.15.49:0/2242584209 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 7) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fbef80f7120 con 0x56286513aa90 2023-11-06T02:08:48.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.778+0000 7fbf10664000 10 client.4728 fetch_fsmap finished waiting for FSMap version 7 2023-11-06T02:08:48.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.778+0000 7fbf10664000 10 client.4728 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-06T02:08:48.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.778+0000 7fbf10664000 20 client.4728 populate_metadata read hostname 'smithi049' 2023-11-06T02:08:48.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.778+0000 7fbf10664000 1 --2- 172.21.15.49:0/2242584209 >> [v2:172.21.15.49:6833/4280448297,v1:172.21.15.49:6835/4280448297] conn(0x562865193770 0x562865195b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:48.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.778+0000 7fbf10664000 4 client.4728 mds_command: new command op to 4293 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T02:08:48.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.778+0000 7fbf10664000 1 -- 172.21.15.49:0/2242584209 --> [v2:172.21.15.49:6833/4280448297,v1:172.21.15.49:6835/4280448297] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562864fb94d0 con 0x562865193770 2023-11-06T02:08:48.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.778+0000 7fbefffff640 1 --2- 172.21.15.49:0/2242584209 >> [v2:172.21.15.49:6833/4280448297,v1:172.21.15.49:6835/4280448297] conn(0x562865193770 0x562865195b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:48.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.778+0000 7fbefffff640 1 --2- 172.21.15.49:0/2242584209 >> [v2:172.21.15.49:6833/4280448297,v1:172.21.15.49:6835/4280448297] conn(0x562865193770 0x562865195b60 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=430ff5a00d4f2ba3 server_cookie=6ddd7bc8985f1a65 in_seq=0 out_seq=0 2023-11-06T02:08:48.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.778+0000 7fbef5ffb640 10 client.4728 ms_handle_connect on v2:172.21.15.49:6833/4280448297 2023-11-06T02:08:48.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.778+0000 7fbef5ffb640 1 -- 172.21.15.49:0/2242584209 <== mds.0 v2:172.21.15.49:6833/4280448297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x562864fb94d0 con 0x562865193770 2023-11-06T02:08:48.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.778+0000 7fbef5ffb640 10 client.4728 handle_command_reply: tid=0 2023-11-06T02:08:48.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.778+0000 7fbf10664000 1 client.4728 shutdown 2023-11-06T02:08:48.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.778+0000 7fbf10664000 1 -- 172.21.15.49:0/2242584209 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56286513aa90 msgr2=0x562865145880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:48.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.778+0000 7fbf10664000 1 --2- 172.21.15.49:0/2242584209 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56286513aa90 0x562865145880 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fbef80efe80 tx=0x7fbef8000bd0 comp rx=0 tx=0).stop 2023-11-06T02:08:48.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.778+0000 7fbf10664000 1 -- 172.21.15.49:0/2242584209 shutdown_connections 2023-11-06T02:08:48.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.778+0000 7fbf10664000 1 --2- 172.21.15.49:0/2242584209 >> [v2:172.21.15.49:6833/4280448297,v1:172.21.15.49:6835/4280448297] conn(0x562865193770 0x562865195b60 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:48.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.778+0000 7fbf10664000 1 --2- 172.21.15.49:0/2242584209 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562865185000 0x562865146300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:48.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.778+0000 7fbf10664000 1 --2- 172.21.15.49:0/2242584209 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5628651846e0 0x562865145dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:48.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.778+0000 7fbef5ffb640 0 client.4728 ms_handle_reset on v2:172.21.15.49:6833/4280448297 2023-11-06T02:08:48.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.778+0000 7fbf10664000 1 --2- 172.21.15.49:0/2242584209 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56286513aa90 0x562865145880 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:48.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.778+0000 7fbf10664000 1 -- 172.21.15.49:0/2242584209 >> 172.21.15.49:0/2242584209 conn(0x562865140bf0 msgr2=0x562865139530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:48.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.782+0000 7fbf10664000 1 -- 172.21.15.49:0/2242584209 shutdown_connections 2023-11-06T02:08:48.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.782+0000 7fbf10664000 1 -- 172.21.15.49:0/2242584209 wait complete. 2023-11-06T02:08:48.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.782+0000 7fbf10664000 20 client.4728 trim_cache size 0 max 16384 2023-11-06T02:08:48.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.794+0000 7fbf10664000 1 Processor -- start 2023-11-06T02:08:48.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.794+0000 7fbf10664000 1 -- start start 2023-11-06T02:08:48.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.794+0000 7fbf10664000 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56286513aa90 0x5628651973d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:48.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.794+0000 7fbf10664000 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5628651846e0 0x562865197910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:48.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.794+0000 7fbf10664000 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562865185000 0x56286517c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:48.804 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.794+0000 7fbf10664000 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x562864c47940 con 0x5628651846e0 2023-11-06T02:08:48.804 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.794+0000 7fbf10664000 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x562864f92600 con 0x562865185000 2023-11-06T02:08:48.804 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.794+0000 7fbf10664000 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x562864f2e000 con 0x56286513aa90 2023-11-06T02:08:48.804 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.794+0000 7fbefffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5628651846e0 0x562865197910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:48.804 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.794+0000 7fbefffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5628651846e0 0x562865197910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47804/0 (socket says 172.21.15.49:47804) 2023-11-06T02:08:48.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.794+0000 7fbefffff640 1 -- 172.21.15.49:0/203544962 learned_addr learned my addr 172.21.15.49:0/203544962 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:48.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.798+0000 7fbf077fe640 1 --2- 172.21.15.49:0/203544962 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562865185000 0x56286517c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:48.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.798+0000 7fbf06ffd640 1 --2- 172.21.15.49:0/203544962 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56286513aa90 0x5628651973d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:48.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.798+0000 7fbefffff640 1 -- 172.21.15.49:0/203544962 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56286513aa90 msgr2=0x5628651973d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:48.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.798+0000 7fbefffff640 1 --2- 172.21.15.49:0/203544962 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56286513aa90 0x5628651973d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:48.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.798+0000 7fbefffff640 1 -- 172.21.15.49:0/203544962 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562865185000 msgr2=0x56286517c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:48.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.798+0000 7fbefffff640 1 --2- 172.21.15.49:0/203544962 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562865185000 0x56286517c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:48.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.798+0000 7fbefffff640 1 -- 172.21.15.49:0/203544962 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562864f35330 con 0x5628651846e0 2023-11-06T02:08:48.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.798+0000 7fbefffff640 1 --2- 172.21.15.49:0/203544962 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5628651846e0 0x562865197910 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fbf0815c7a0 tx=0x7fbef0012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=232bd8829a780499 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:48.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.798+0000 7fbefd7fa640 1 -- 172.21.15.49:0/203544962 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef0005390 con 0x5628651846e0 2023-11-06T02:08:48.807 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.798+0000 7fbefd7fa640 1 -- 172.21.15.49:0/203544962 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbef0025c70 con 0x5628651846e0 2023-11-06T02:08:48.807 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.798+0000 7fbefd7fa640 1 -- 172.21.15.49:0/203544962 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef0025de0 con 0x5628651846e0 2023-11-06T02:08:48.807 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.798+0000 7fbf10664000 1 -- 172.21.15.49:0/203544962 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5628651846e0 msgr2=0x562865197910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:48.807 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.798+0000 7fbf10664000 1 --2- 172.21.15.49:0/203544962 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5628651846e0 0x562865197910 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fbf0815c7a0 tx=0x7fbef0012000 comp rx=0 tx=0).stop 2023-11-06T02:08:48.807 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.798+0000 7fbf10664000 1 -- 172.21.15.49:0/203544962 shutdown_connections 2023-11-06T02:08:48.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.798+0000 7fbf10664000 1 --2- 172.21.15.49:0/203544962 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5628651846e0 0x562865197910 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:48.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.798+0000 7fbf10664000 1 --2- 172.21.15.49:0/203544962 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562865185000 0x56286517c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:48.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.798+0000 7fbf10664000 1 --2- 172.21.15.49:0/203544962 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56286513aa90 0x5628651973d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:48.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.798+0000 7fbf10664000 1 -- 172.21.15.49:0/203544962 >> 172.21.15.49:0/203544962 conn(0x562865140bf0 msgr2=0x562865196cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:48.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.798+0000 7fbf10664000 1 -- 172.21.15.49:0/203544962 shutdown_connections 2023-11-06T02:08:48.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.798+0000 7fbf10664000 1 -- 172.21.15.49:0/203544962 wait complete. 2023-11-06T02:08:48.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.798+0000 7fbf10664000 1 -- start start 2023-11-06T02:08:48.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.798+0000 7fbf10664000 1 Processor -- start 2023-11-06T02:08:48.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.798+0000 7fbf10664000 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56286513aa90 0x5628651e0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:48.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.798+0000 7fbf10664000 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5628651846e0 0x5628651e1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:48.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.798+0000 7fbf10664000 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562865185000 0x56286517f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:48.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.798+0000 7fbf10664000 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x562864fa2910 con 0x56286513aa90 2023-11-06T02:08:48.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.798+0000 7fbf10664000 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x562864f85fd0 con 0x5628651846e0 2023-11-06T02:08:48.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.798+0000 7fbf10664000 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x562864fe28c0 con 0x562865185000 2023-11-06T02:08:48.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.798+0000 7fbf10664000 10 client.0 _ll_register_callbacks cb 0x5628651069e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T02:08:48.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.798+0000 7fbf06ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56286513aa90 0x5628651e0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:48.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.798+0000 7fbf06ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56286513aa90 0x5628651e0c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47812/0 (socket says 172.21.15.49:47812) 2023-11-06T02:08:48.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.798+0000 7fbf06ffd640 1 -- 172.21.15.49:0/127890945 learned_addr learned my addr 172.21.15.49:0/127890945 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:48.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.798+0000 7fbf06ffd640 1 -- 172.21.15.49:0/127890945 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562865185000 msgr2=0x56286517f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:48.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.798+0000 7fbf06ffd640 1 --2- 172.21.15.49:0/127890945 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562865185000 0x56286517f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:48.812 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.798+0000 7fbf06ffd640 1 -- 172.21.15.49:0/127890945 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5628651846e0 msgr2=0x5628651e1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:48.812 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.798+0000 7fbf06ffd640 1 --2- 172.21.15.49:0/127890945 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5628651846e0 0x5628651e1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:48.812 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.798+0000 7fbf06ffd640 1 -- 172.21.15.49:0/127890945 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbef001d040 con 0x56286513aa90 2023-11-06T02:08:48.812 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.798+0000 7fbf06ffd640 1 --2- 172.21.15.49:0/127890945 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56286513aa90 0x5628651e0c30 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbef801ca00 tx=0x7fbef80192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95011842a1bd6f1b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:48.812 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.798+0000 7fbef5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-11-06T02:08:48.813 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.798+0000 7fbef5ffb640 1 -- 172.21.15.49:0/127890945 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef810d0d0 con 0x56286513aa90 2023-11-06T02:08:48.813 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.798+0000 7fbef5ffb640 1 -- 172.21.15.49:0/127890945 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbef80f53b0 con 0x56286513aa90 2023-11-06T02:08:48.813 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.798+0000 7fbf10664000 1 -- 172.21.15.49:0/127890945 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562864f59b90 con 0x56286513aa90 2023-11-06T02:08:48.813 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.798+0000 7fbef5ffb640 1 -- 172.21.15.49:0/127890945 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbef80eb3c0 con 0x56286513aa90 2023-11-06T02:08:48.813 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.798+0000 7fbf10664000 10 client.4734 fetch_fsmap learned FSMap version 7 2023-11-06T02:08:48.814 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.798+0000 7fbf10664000 1 -- 172.21.15.49:0/127890945 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x562864c86cc0 con 0x56286513aa90 2023-11-06T02:08:48.814 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.798+0000 7fbef5ffb640 1 -- 172.21.15.49:0/127890945 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562864f59b90 con 0x56286513aa90 2023-11-06T02:08:48.814 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.798+0000 7fbef5ffb640 1 -- 172.21.15.49:0/127890945 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 7) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fbef8116240 con 0x56286513aa90 2023-11-06T02:08:48.814 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.798+0000 7fbf10664000 10 client.4734 fetch_fsmap finished waiting for FSMap version 7 2023-11-06T02:08:48.814 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.798+0000 7fbf10664000 10 client.4734 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-06T02:08:48.815 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.798+0000 7fbf10664000 20 client.4734 populate_metadata read hostname 'smithi049' 2023-11-06T02:08:48.815 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.798+0000 7fbf10664000 1 --2- 172.21.15.49:0/127890945 >> [v2:172.21.15.49:6833/4280448297,v1:172.21.15.49:6835/4280448297] conn(0x56286517fee0 0x5628651802c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:48.815 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.798+0000 7fbf10664000 4 client.4734 mds_command: new command op to 4293 tid=0[{"prefix": "status"}] 2023-11-06T02:08:48.815 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.798+0000 7fbf10664000 1 -- 172.21.15.49:0/127890945 --> [v2:172.21.15.49:6833/4280448297,v1:172.21.15.49:6835/4280448297] -- command(tid 0: {"prefix": "status"}) v1 -- 0x562864f830e0 con 0x56286517fee0 2023-11-06T02:08:48.815 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.798+0000 7fbefffff640 1 --2- 172.21.15.49:0/127890945 >> [v2:172.21.15.49:6833/4280448297,v1:172.21.15.49:6835/4280448297] conn(0x56286517fee0 0x5628651802c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:48.816 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.802+0000 7fbefffff640 1 --2- 172.21.15.49:0/127890945 >> [v2:172.21.15.49:6833/4280448297,v1:172.21.15.49:6835/4280448297] conn(0x56286517fee0 0x5628651802c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3636bc06ae0aa9ef server_cookie=610b16ebb9f0ce5c in_seq=0 out_seq=0 2023-11-06T02:08:48.816 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.802+0000 7fbef5ffb640 10 client.4734 ms_handle_connect on v2:172.21.15.49:6833/4280448297 2023-11-06T02:08:48.816 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.802+0000 7fbef5ffb640 1 -- 172.21.15.49:0/127890945 <== mds.0 v2:172.21.15.49:6833/4280448297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x562864f830e0 con 0x56286517fee0 2023-11-06T02:08:48.816 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.802+0000 7fbef5ffb640 10 client.4734 handle_command_reply: tid=0 2023-11-06T02:08:48.816 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.802+0000 7fbf10664000 1 client.4734 shutdown 2023-11-06T02:08:48.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.802+0000 7fbf10664000 1 -- 172.21.15.49:0/127890945 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56286513aa90 msgr2=0x5628651e0c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:48.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.802+0000 7fbf10664000 1 --2- 172.21.15.49:0/127890945 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56286513aa90 0x5628651e0c30 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbef801ca00 tx=0x7fbef80192c0 comp rx=0 tx=0).stop 2023-11-06T02:08:48.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.802+0000 7fbf10664000 1 -- 172.21.15.49:0/127890945 shutdown_connections 2023-11-06T02:08:48.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.802+0000 7fbf10664000 1 --2- 172.21.15.49:0/127890945 >> [v2:172.21.15.49:6833/4280448297,v1:172.21.15.49:6835/4280448297] conn(0x56286517fee0 0x5628651802c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:48.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.802+0000 7fbf10664000 1 --2- 172.21.15.49:0/127890945 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562865185000 0x56286517f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:48.818 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.802+0000 7fbf10664000 1 --2- 172.21.15.49:0/127890945 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5628651846e0 0x5628651e1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:48.818 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.802+0000 7fbef5ffb640 0 client.4734 ms_handle_reset on v2:172.21.15.49:6833/4280448297 2023-11-06T02:08:48.818 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.802+0000 7fbf10664000 1 --2- 172.21.15.49:0/127890945 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56286513aa90 0x5628651e0c30 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:48.818 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.802+0000 7fbf10664000 1 -- 172.21.15.49:0/127890945 >> 172.21.15.49:0/127890945 conn(0x562865140bf0 msgr2=0x562865196cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:48.818 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.802+0000 7fbf10664000 1 -- 172.21.15.49:0/127890945 shutdown_connections 2023-11-06T02:08:48.818 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.802+0000 7fbf10664000 1 -- 172.21.15.49:0/127890945 wait complete. 2023-11-06T02:08:48.819 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.802+0000 7fbf10664000 20 client.4734 trim_cache size 0 max 16384 2023-11-06T02:08:48.819 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-11-06T02:08:48.819 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_fsid": "1a8b035c-6ba4-499a-b876-aea47fd9c0d4", 2023-11-06T02:08:48.820 INFO:teuthology.orchestra.run.smithi049.stdout: "whoami": 0, 2023-11-06T02:08:48.820 INFO:teuthology.orchestra.run.smithi049.stdout: "id": 4293, 2023-11-06T02:08:48.820 INFO:teuthology.orchestra.run.smithi049.stdout: "want_state": "up:active", 2023-11-06T02:08:48.820 INFO:teuthology.orchestra.run.smithi049.stdout: "state": "up:active", 2023-11-06T02:08:48.820 INFO:teuthology.orchestra.run.smithi049.stdout: "fs_name": "cephfs", 2023-11-06T02:08:48.821 INFO:teuthology.orchestra.run.smithi049.stdout: "rank_uptime": 18.207603631000001, 2023-11-06T02:08:48.821 INFO:teuthology.orchestra.run.smithi049.stdout: "mdsmap_epoch": 7, 2023-11-06T02:08:48.821 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch": 23, 2023-11-06T02:08:48.821 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch_barrier": 23, 2023-11-06T02:08:48.821 INFO:teuthology.orchestra.run.smithi049.stdout: "uptime": 22.639239962000001 2023-11-06T02:08:48.822 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-11-06T02:08:48.823 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.802+0000 7fbf0d53c640 1 -- 172.21.15.49:0/2797887938 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fbeec065a90 msgr2=0x7fbeec067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:48.823 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.802+0000 7fbf0d53c640 1 --2- 172.21.15.49:0/2797887938 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fbeec065a90 0x7fbeec067f50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbf080b18d0 tx=0x7fbef0009040 comp rx=0 tx=0).stop 2023-11-06T02:08:48.823 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.802+0000 7fbf0d53c640 1 -- 172.21.15.49:0/2797887938 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf080714a0 msgr2=0x7fbf080b3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:48.824 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.802+0000 7fbf0d53c640 1 --2- 172.21.15.49:0/2797887938 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf080714a0 0x7fbf080b3670 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fbef80ac040 tx=0x7fbef801ad60 comp rx=0 tx=0).stop 2023-11-06T02:08:48.824 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.802+0000 7fbf0d53c640 1 -- 172.21.15.49:0/2797887938 shutdown_connections 2023-11-06T02:08:48.824 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.802+0000 7fbf0d53c640 1 --2- 172.21.15.49:0/2797887938 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fbeec065a90 0x7fbeec067f50 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:48.824 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.802+0000 7fbf0d53c640 1 --2- 172.21.15.49:0/2797887938 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf08161650 0x7fbf08155b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:48.824 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.802+0000 7fbf0d53c640 1 --2- 172.21.15.49:0/2797887938 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf080b1f90 0x7fbf080cc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:48.825 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.802+0000 7fbf0d53c640 1 --2- 172.21.15.49:0/2797887938 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf080714a0 0x7fbf080b3670 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:48.825 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.802+0000 7fbf0d53c640 1 -- 172.21.15.49:0/2797887938 >> 172.21.15.49:0/2797887938 conn(0x7fbf080d2aa0 msgr2=0x7fbf081600e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:48.825 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.802+0000 7fbf0d53c640 1 -- 172.21.15.49:0/2797887938 shutdown_connections 2023-11-06T02:08:48.825 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:48.802+0000 7fbf0d53c640 1 -- 172.21.15.49:0/2797887938 wait complete. 2023-11-06T02:08:48.826 INFO:teuthology.run_tasks:Running task ceph-fuse... 2023-11-06T02:08:48.848 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2023-11-06T02:08:48.849 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-11-06T02:08:48.849 INFO:tasks.ceph_fuse:client.0 config is {} 2023-11-06T02:08:48.849 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-06T02:08:48.850 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-06T02:08:48.850 INFO:tasks.ceph_fuse:client.2 config is {} 2023-11-06T02:08:48.851 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-06T02:08:48.851 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-11-06T02:08:48.851 INFO:tasks.ceph_fuse:client.1 config is {} 2023-11-06T02:08:48.851 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-06T02:08:48.852 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-06T02:08:48.852 INFO:tasks.ceph_fuse:client.3 config is {} 2023-11-06T02:08:48.852 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-06T02:08:48.852 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-11-06T02:08:48.853 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:08:48.853 DEBUG:teuthology.orchestra.run.smithi150:> ip netns list 2023-11-06T02:08:48.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:08:48.861 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link delete ceph-brx 2023-11-06T02:08:48.921 INFO:teuthology.orchestra.run.smithi150.stderr:Cannot find device "ceph-brx" 2023-11-06T02:08:48.922 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T02:08:48.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:08:48.923 DEBUG:teuthology.orchestra.run.smithi049:> ip netns list 2023-11-06T02:08:48.930 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:08:48.930 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link delete ceph-brx 2023-11-06T02:08:48.989 INFO:teuthology.orchestra.run.smithi049.stderr:Cannot find device "ceph-brx" 2023-11-06T02:08:48.990 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T02:08:48.991 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2023-11-06T02:08:48.991 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-06T02:08:49.169 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:49.174+0000 7fae84a2e640 1 Processor -- start 2023-11-06T02:08:49.169 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:49.174+0000 7fae84a2e640 1 -- start start 2023-11-06T02:08:49.170 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:49.174+0000 7fae84a2e640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fae800cb960 0x7fae8015e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:49.170 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:49.174+0000 7fae84a2e640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae8015f3c0 con 0x7fae800cb580 2023-11-06T02:08:49.170 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:49.174+0000 7fae84a2e640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae8015eb20 con 0x7fae800b6100 2023-11-06T02:08:49.171 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:49.174+0000 7fae84a2e640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae800cbdf0 con 0x7fae800cb960 2023-11-06T02:08:49.171 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:49.174+0000 7fae7e59c640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fae800cb580 0x7fae800ce420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:37198/0 (socket says 172.21.15.49:37198) 2023-11-06T02:08:49.171 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:49.174+0000 7fae7e59c640 1 -- 172.21.15.49:0/4244441915 learned_addr learned my addr 172.21.15.49:0/4244441915 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:08:49.171 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:49.174+0000 7fae7ed9d640 1 --2- 172.21.15.49:0/4244441915 >> v2:172.21.15.49:3300/0 conn(0x7fae800cb960 0x7fae8015e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:49.172 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:49.174+0000 7fae7ed9d640 1 -- 172.21.15.49:0/4244441915 >> v1:172.21.15.150:6789/0 conn(0x7fae800b6100 legacy=0x7fae8015b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:49.172 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:49.174+0000 7fae7ed9d640 1 -- 172.21.15.49:0/4244441915 >> v1:172.21.15.49:6789/0 conn(0x7fae800cb580 legacy=0x7fae800ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:49.172 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:49.174+0000 7fae7ed9d640 1 -- 172.21.15.49:0/4244441915 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae8015ee00 con 0x7fae800cb960 2023-11-06T02:08:49.174 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:49.178+0000 7fae7ed9d640 1 --2- 172.21.15.49:0/4244441915 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fae800cb960 0x7fae8015e440 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fae78009d90 tx=0x7fae7801c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca9bb5aabefc2319 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:49.174 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:49.178+0000 7fae7d59a640 1 -- 172.21.15.49:0/4244441915 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae780a5950 con 0x7fae800cb960 2023-11-06T02:08:49.174 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:49.178+0000 7fae7d59a640 1 -- 172.21.15.49:0/4244441915 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae780a5ab0 con 0x7fae800cb960 2023-11-06T02:08:49.174 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:49.178+0000 7fae7d59a640 1 -- 172.21.15.49:0/4244441915 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae780a5e40 con 0x7fae800cb960 2023-11-06T02:08:49.175 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:49.178+0000 7fae84a2e640 1 -- 172.21.15.49:0/4244441915 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fae800cb960 msgr2=0x7fae8015e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:49.175 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:49.178+0000 7fae84a2e640 1 --2- 172.21.15.49:0/4244441915 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fae800cb960 0x7fae8015e440 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fae78009d90 tx=0x7fae7801c890 comp rx=0 tx=0).stop 2023-11-06T02:08:49.175 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:49.178+0000 7fae84a2e640 1 -- 172.21.15.49:0/4244441915 shutdown_connections 2023-11-06T02:08:49.175 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:49.178+0000 7fae84a2e640 1 --2- 172.21.15.49:0/4244441915 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fae800cb960 0x7fae8015e440 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:49.175 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:49.178+0000 7fae84a2e640 1 -- 172.21.15.49:0/4244441915 >> 172.21.15.49:0/4244441915 conn(0x7fae800cd2d0 msgr2=0x7fae800cd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:49.176 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:49.178+0000 7fae84a2e640 1 -- 172.21.15.49:0/4244441915 shutdown_connections 2023-11-06T02:08:49.176 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:49.178+0000 7fae84a2e640 1 -- 172.21.15.49:0/4244441915 wait complete. 2023-11-06T02:08:49.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:49.178+0000 7fae84a2e640 1 Processor -- start 2023-11-06T02:08:49.179 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:49.178+0000 7fae84a2e640 1 -- start start 2023-11-06T02:08:49.179 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:49.178+0000 7fae84a2e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fae800b6100 0x7fae800b4220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:49.179 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:49.178+0000 7fae84a2e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fae800cb580 0x7fae8014e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:49.179 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:49.178+0000 7fae84a2e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fae800cb960 0x7fae801573c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:49.180 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:49.178+0000 7fae84a2e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae800b4760 con 0x7fae800b6100 2023-11-06T02:08:49.180 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:49.178+0000 7fae84a2e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae8014f0e0 con 0x7fae800cb580 2023-11-06T02:08:49.180 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:49.178+0000 7fae84a2e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae8014f3c0 con 0x7fae800cb960 2023-11-06T02:08:49.180 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:49.178+0000 7fae7e59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fae800b6100 0x7fae800b4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:49.180 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:49.178+0000 7fae7e59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fae800b6100 0x7fae800b4220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47830/0 (socket says 172.21.15.49:47830) 2023-11-06T02:08:49.181 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:49.178+0000 7fae7e59c640 1 -- 172.21.15.49:0/3076144874 learned_addr learned my addr 172.21.15.49:0/3076144874 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:49.181 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:49.178+0000 7fae7dd9b640 1 --2- 172.21.15.49:0/3076144874 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fae800cb580 0x7fae8014e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:49.181 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:49.178+0000 7fae7e59c640 1 -- 172.21.15.49:0/3076144874 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fae800cb960 msgr2=0x7fae801573c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:49.181 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:49.178+0000 7fae7ed9d640 1 --2- 172.21.15.49:0/3076144874 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fae800cb960 0x7fae801573c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:49.182 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:49.178+0000 7fae7e59c640 1 --2- 172.21.15.49:0/3076144874 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fae800cb960 0x7fae801573c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:49.182 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:49.178+0000 7fae7e59c640 1 -- 172.21.15.49:0/3076144874 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fae800cb580 msgr2=0x7fae8014e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:49.182 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:49.178+0000 7fae7e59c640 1 --2- 172.21.15.49:0/3076144874 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fae800cb580 0x7fae8014e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:49.182 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:49.178+0000 7fae7e59c640 1 -- 172.21.15.49:0/3076144874 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae78009a70 con 0x7fae800b6100 2023-11-06T02:08:49.182 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:49.178+0000 7fae7dd9b640 1 --2- 172.21.15.49:0/3076144874 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fae800cb580 0x7fae8014e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:08:49.183 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:49.178+0000 7fae7e59c640 1 --2- 172.21.15.49:0/3076144874 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fae800b6100 0x7fae800b4220 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fae7000c500 tx=0x7fae7000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:49.183 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:49.178+0000 7fae6effd640 1 -- 172.21.15.49:0/3076144874 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae70005cf0 con 0x7fae800b6100 2023-11-06T02:08:49.183 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:49.178+0000 7fae6effd640 1 -- 172.21.15.49:0/3076144874 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae70005e50 con 0x7fae800b6100 2023-11-06T02:08:49.183 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:49.178+0000 7fae84a2e640 1 -- 172.21.15.49:0/3076144874 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae800cecb0 con 0x7fae800b6100 2023-11-06T02:08:49.183 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:49.178+0000 7fae6effd640 1 -- 172.21.15.49:0/3076144874 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae700072f0 con 0x7fae800b6100 2023-11-06T02:08:49.184 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:49.178+0000 7fae84a2e640 1 -- 172.21.15.49:0/3076144874 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae8014f980 con 0x7fae800b6100 2023-11-06T02:08:49.184 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:49.182+0000 7fae84a2e640 1 -- 172.21.15.49:0/3076144874 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae800ce170 con 0x7fae800b6100 2023-11-06T02:08:49.184 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:49.182+0000 7fae6effd640 1 -- 172.21.15.49:0/3076144874 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fae70007450 con 0x7fae800b6100 2023-11-06T02:08:49.184 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:49.182+0000 7fae6effd640 1 --2- 172.21.15.49:0/3076144874 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fae60065a60 0x7fae60067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:49.184 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:49.182+0000 7fae6effd640 1 -- 172.21.15.49:0/3076144874 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fae70082740 con 0x7fae800b6100 2023-11-06T02:08:49.185 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:49.182+0000 7fae7dd9b640 1 --2- 172.21.15.49:0/3076144874 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fae60065a60 0x7fae60067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:49.185 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:49.182+0000 7fae7dd9b640 1 --2- 172.21.15.49:0/3076144874 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fae60065a60 0x7fae60067f20 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fae80166700 tx=0x7fae74009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:49.185 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:49.186+0000 7fae6effd640 1 -- 172.21.15.49:0/3076144874 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fae700500e0 con 0x7fae800b6100 2023-11-06T02:08:49.365 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:49.370+0000 7fae84a2e640 1 -- 172.21.15.49:0/3076144874 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fae8014e9f0 con 0x7fae800b6100 2023-11-06T02:08:49.366 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:49.370+0000 7fae6effd640 1 -- 172.21.15.49:0/3076144874 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7fae70017060 con 0x7fae800b6100 2023-11-06T02:08:49.366 INFO:teuthology.orchestra.run.smithi049.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-06T02:08:49.367 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:49.370+0000 7fae84a2e640 1 -- 172.21.15.49:0/3076144874 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fae60065a60 msgr2=0x7fae60067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:49.368 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:49.370+0000 7fae84a2e640 1 --2- 172.21.15.49:0/3076144874 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fae60065a60 0x7fae60067f20 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fae80166700 tx=0x7fae74009040 comp rx=0 tx=0).stop 2023-11-06T02:08:49.368 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:49.370+0000 7fae84a2e640 1 -- 172.21.15.49:0/3076144874 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fae800b6100 msgr2=0x7fae800b4220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:49.368 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:49.370+0000 7fae84a2e640 1 --2- 172.21.15.49:0/3076144874 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fae800b6100 0x7fae800b4220 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fae7000c500 tx=0x7fae7000c9c0 comp rx=0 tx=0).stop 2023-11-06T02:08:49.368 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:49.370+0000 7fae84a2e640 1 -- 172.21.15.49:0/3076144874 shutdown_connections 2023-11-06T02:08:49.368 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:49.370+0000 7fae84a2e640 1 --2- 172.21.15.49:0/3076144874 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fae60065a60 0x7fae60067f20 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:49.369 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:49.370+0000 7fae84a2e640 1 --2- 172.21.15.49:0/3076144874 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fae800cb960 0x7fae801573c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:49.369 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:49.370+0000 7fae84a2e640 1 --2- 172.21.15.49:0/3076144874 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fae800cb580 0x7fae8014e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:49.369 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:49.370+0000 7fae84a2e640 1 --2- 172.21.15.49:0/3076144874 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fae800b6100 0x7fae800b4220 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:49.369 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:49.370+0000 7fae84a2e640 1 -- 172.21.15.49:0/3076144874 >> 172.21.15.49:0/3076144874 conn(0x7fae800cd2d0 msgr2=0x7fae8015a360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:49.370 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:49.370+0000 7fae84a2e640 1 -- 172.21.15.49:0/3076144874 shutdown_connections 2023-11-06T02:08:49.370 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:49.370+0000 7fae84a2e640 1 -- 172.21.15.49:0/3076144874 wait complete. 2023-11-06T02:08:49.383 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-06T02:08:49.383 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-06T02:08:49.383 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi049.front.sepia.ceph.com 2023-11-06T02:08:49.384 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-06T02:08:49.384 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-06T02:08:49.384 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-06T02:08:49.384 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-06T02:08:49.385 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-06T02:08:49.385 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-06T02:08:49.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:08:49.385 DEBUG:teuthology.orchestra.run.smithi049:> ip addr 2023-11-06T02:08:49.391 INFO:teuthology.orchestra.run.smithi049.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-06T02:08:49.391 INFO:teuthology.orchestra.run.smithi049.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-06T02:08:49.391 INFO:teuthology.orchestra.run.smithi049.stdout: inet 127.0.0.1/8 scope host lo 2023-11-06T02:08:49.391 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:08:49.391 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 ::1/128 scope host 2023-11-06T02:08:49.392 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:08:49.392 INFO:teuthology.orchestra.run.smithi049.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-06T02:08:49.392 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:6b:ff:84 brd ff:ff:ff:ff:ff:ff 2023-11-06T02:08:49.392 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp7s0f0 2023-11-06T02:08:49.392 INFO:teuthology.orchestra.run.smithi049.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-06T02:08:49.393 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:6b:ff:85 brd ff:ff:ff:ff:ff:ff 2023-11-06T02:08:49.393 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp7s0f1 2023-11-06T02:08:49.393 INFO:teuthology.orchestra.run.smithi049.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-06T02:08:49.393 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:bd:15:66 brd ff:ff:ff:ff:ff:ff 2023-11-06T02:08:49.393 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp1s0f0 2023-11-06T02:08:49.394 INFO:teuthology.orchestra.run.smithi049.stdout: inet 172.21.15.49/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-11-06T02:08:49.394 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft 42468sec preferred_lft 42468sec 2023-11-06T02:08:49.394 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 fe80::ec4:7aff:febd:1566/64 scope link 2023-11-06T02:08:49.394 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:08:49.394 INFO:teuthology.orchestra.run.smithi049.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-06T02:08:49.395 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:bd:15:67 brd ff:ff:ff:ff:ff:ff 2023-11-06T02:08:49.395 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp1s0f1 2023-11-06T02:08:49.395 INFO:teuthology.orchestra.run.smithi049.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-06T02:08:49.395 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 02:42:47:49:33:69 brd ff:ff:ff:ff:ff:ff 2023-11-06T02:08:49.395 INFO:teuthology.orchestra.run.smithi049.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-06T02:08:49.396 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:08:49.396 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-06T02:08:49.396 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:08:49.397 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec bash -c ' 2023-11-06T02:08:49.397 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2023-11-06T02:08:49.397 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link add name ceph-brx type bridge 2023-11-06T02:08:49.398 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip addr flush dev ceph-brx 2023-11-06T02:08:49.398 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link set ceph-brx up 2023-11-06T02:08:49.398 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-06T02:08:49.399 DEBUG:teuthology.orchestra.run.smithi049:> ') 2023-11-06T02:08:49.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:08:49.467 DEBUG:teuthology.orchestra.run.smithi049:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-06T02:08:49.516 INFO:teuthology.orchestra.run.smithi049.stdout:1 2023-11-06T02:08:49.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:08:49.518 DEBUG:teuthology.orchestra.run.smithi049:> route 2023-11-06T02:08:49.580 INFO:teuthology.orchestra.run.smithi049.stdout:Kernel IP routing table 2023-11-06T02:08:49.580 INFO:teuthology.orchestra.run.smithi049.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-06T02:08:49.580 INFO:teuthology.orchestra.run.smithi049.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-11-06T02:08:49.580 INFO:teuthology.orchestra.run.smithi049.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-06T02:08:49.581 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-11-06T02:08:49.581 INFO:teuthology.orchestra.run.smithi049.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-06T02:08:49.582 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:08:49.582 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec bash -c ' 2023-11-06T02:08:49.583 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2023-11-06T02:08:49.583 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-06T02:08:49.584 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-06T02:08:49.584 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-06T02:08:49.585 DEBUG:teuthology.orchestra.run.smithi049:> ') 2023-11-06T02:08:49.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:08:49.652 DEBUG:teuthology.orchestra.run.smithi049:> ip netns list 2023-11-06T02:08:49.702 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:08:49.703 DEBUG:teuthology.orchestra.run.smithi049:> ip netns list-id 2023-11-06T02:08:49.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:08:49.750 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec bash -c ' 2023-11-06T02:08:49.751 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2023-11-06T02:08:49.751 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-06T02:08:49.752 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-11-06T02:08:49.752 DEBUG:teuthology.orchestra.run.smithi049:> ') 2023-11-06T02:08:49.821 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-11-06T02:08:49.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:08:49.822 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec bash -c ' 2023-11-06T02:08:49.822 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2023-11-06T02:08:49.822 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-11-06T02:08:49.823 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-06T02:08:49.823 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-11-06T02:08:49.823 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-11-06T02:08:49.824 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-11-06T02:08:49.824 DEBUG:teuthology.orchestra.run.smithi049:> ') 2023-11-06T02:08:49.936 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:08:49.937 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec bash -c ' 2023-11-06T02:08:49.937 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2023-11-06T02:08:49.937 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link set brx.0 up 2023-11-06T02:08:49.938 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link set dev brx.0 master ceph-brx 2023-11-06T02:08:49.938 DEBUG:teuthology.orchestra.run.smithi049:> ') 2023-11-06T02:08:49.958 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-06T02:08:49.958 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-06T02:08:49.959 DEBUG:teuthology.orchestra.run.smithi049:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-06T02:08:50.005 INFO:teuthology.orchestra.run.smithi049.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-06T02:08:50.006 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-06T02:08:50.006 DEBUG:teuthology.orchestra.run.smithi049:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-06T02:08:50.053 DEBUG:teuthology.orchestra.run.smithi049:> sudo modprobe fuse 2023-11-06T02:08:50.110 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-06T02:08:50.111 DEBUG:teuthology.orchestra.run.smithi049:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-06T02:08:50.165 INFO:teuthology.orchestra.run.smithi049.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-06T02:08:50.167 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-06T02:08:50.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:08:50.168 DEBUG:teuthology.orchestra.run.smithi049:> ls /sys/fs/fuse/connections 2023-11-06T02:08:50.217 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-06T02:08:50.218 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-06T02:08:50.262 DEBUG:teuthology.orchestra.run.smithi049:> sudo modprobe fuse 2023-11-06T02:08:50.278 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-06T02:08:50.278 DEBUG:teuthology.orchestra.run.smithi049:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-06T02:08:50.332 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-06T02:08:50.333 INFO:teuthology.orchestra.run.smithi049.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-06T02:08:50.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:08:50.335 DEBUG:teuthology.orchestra.run.smithi049:> ls /sys/fs/fuse/connections 2023-11-06T02:08:50.357 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi049.stderr:2023-11-06T02:08:50.358+0000 7f38e3631580 -1 init, newargv = 0x55792937bdc0 newargc=15 2023-11-06T02:08:50.357 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi049.stderr:ceph-fuse[44947]: starting ceph client 2023-11-06T02:08:50.363 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi049.stderr:ceph-fuse[44947]: starting fuse 2023-11-06T02:08:51.413 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T02:08:51.416 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T02:08:51.462 DEBUG:teuthology.orchestra.run.smithi049:> sudo modprobe fuse 2023-11-06T02:08:51.510 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-06T02:08:51.511 DEBUG:teuthology.orchestra.run.smithi049:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-06T02:08:51.565 INFO:teuthology.orchestra.run.smithi049.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-06T02:08:51.566 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-06T02:08:51.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:08:51.567 DEBUG:teuthology.orchestra.run.smithi049:> ls /sys/fs/fuse/connections 2023-11-06T02:08:51.614 INFO:teuthology.orchestra.run.smithi049.stdout:45 2023-11-06T02:08:51.614 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2023-11-06T02:08:51.615 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-11-06T02:08:51.615 DEBUG:teuthology.orchestra.run.smithi049:> import glob 2023-11-06T02:08:51.615 DEBUG:teuthology.orchestra.run.smithi049:> import re 2023-11-06T02:08:51.616 DEBUG:teuthology.orchestra.run.smithi049:> import os 2023-11-06T02:08:51.616 DEBUG:teuthology.orchestra.run.smithi049:> import subprocess 2023-11-06T02:08:51.616 DEBUG:teuthology.orchestra.run.smithi049:> 2023-11-06T02:08:51.616 DEBUG:teuthology.orchestra.run.smithi049:> def _find_admin_socket(client_name): 2023-11-06T02:08:51.617 DEBUG:teuthology.orchestra.run.smithi049:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-06T02:08:51.617 DEBUG:teuthology.orchestra.run.smithi049:> files = glob.glob(asok_path) 2023-11-06T02:08:51.617 DEBUG:teuthology.orchestra.run.smithi049:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-06T02:08:51.618 DEBUG:teuthology.orchestra.run.smithi049:> 2023-11-06T02:08:51.618 DEBUG:teuthology.orchestra.run.smithi049:> # Given a non-glob path, it better be there 2023-11-06T02:08:51.618 DEBUG:teuthology.orchestra.run.smithi049:> if "*" not in asok_path: 2023-11-06T02:08:51.618 DEBUG:teuthology.orchestra.run.smithi049:> assert(len(files) == 1) 2023-11-06T02:08:51.619 DEBUG:teuthology.orchestra.run.smithi049:> return files[0] 2023-11-06T02:08:51.619 DEBUG:teuthology.orchestra.run.smithi049:> 2023-11-06T02:08:51.619 DEBUG:teuthology.orchestra.run.smithi049:> for f in files: 2023-11-06T02:08:51.620 DEBUG:teuthology.orchestra.run.smithi049:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-06T02:08:51.620 DEBUG:teuthology.orchestra.run.smithi049:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-06T02:08:51.620 DEBUG:teuthology.orchestra.run.smithi049:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-06T02:08:51.621 DEBUG:teuthology.orchestra.run.smithi049:> contents = proc_f.read() 2023-11-06T02:08:51.621 DEBUG:teuthology.orchestra.run.smithi049:> if mountpoint in contents: 2023-11-06T02:08:51.621 DEBUG:teuthology.orchestra.run.smithi049:> return f 2023-11-06T02:08:51.622 DEBUG:teuthology.orchestra.run.smithi049:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-06T02:08:51.622 DEBUG:teuthology.orchestra.run.smithi049:> 2023-11-06T02:08:51.622 DEBUG:teuthology.orchestra.run.smithi049:> print(_find_admin_socket("client.0")) 2023-11-06T02:08:51.623 DEBUG:teuthology.orchestra.run.smithi049:> ' 2023-11-06T02:08:51.738 INFO:teuthology.orchestra.run.smithi049.stdout:/var/run/ceph/ceph-client.0.44947.asok 2023-11-06T02:08:51.920 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.44947.asok 2023-11-06T02:08:51.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:08:51.921 DEBUG:teuthology.orchestra.run.smithi049:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.44947.asok status 2023-11-06T02:08:52.083 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-11-06T02:08:52.083 INFO:teuthology.orchestra.run.smithi049.stdout: "metadata": { 2023-11-06T02:08:52.083 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_sha1": "f7e9f9af51acf98e0ccc3037487018e4ae40caa5", 2023-11-06T02:08:52.084 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_version": "ceph version 18.2.0-1181-gf7e9f9af (f7e9f9af51acf98e0ccc3037487018e4ae40caa5) reef (stable)", 2023-11-06T02:08:52.084 INFO:teuthology.orchestra.run.smithi049.stdout: "entity_id": "0", 2023-11-06T02:08:52.084 INFO:teuthology.orchestra.run.smithi049.stdout: "hostname": "smithi049", 2023-11-06T02:08:52.084 INFO:teuthology.orchestra.run.smithi049.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-06T02:08:52.084 INFO:teuthology.orchestra.run.smithi049.stdout: "pid": "44947", 2023-11-06T02:08:52.085 INFO:teuthology.orchestra.run.smithi049.stdout: "root": "/" 2023-11-06T02:08:52.085 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:08:52.085 INFO:teuthology.orchestra.run.smithi049.stdout: "dentry_count": 0, 2023-11-06T02:08:52.085 INFO:teuthology.orchestra.run.smithi049.stdout: "dentry_pinned_count": 0, 2023-11-06T02:08:52.086 INFO:teuthology.orchestra.run.smithi049.stdout: "id": 4752, 2023-11-06T02:08:52.086 INFO:teuthology.orchestra.run.smithi049.stdout: "inst": { 2023-11-06T02:08:52.086 INFO:teuthology.orchestra.run.smithi049.stdout: "name": { 2023-11-06T02:08:52.086 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "client", 2023-11-06T02:08:52.086 INFO:teuthology.orchestra.run.smithi049.stdout: "num": 4752 2023-11-06T02:08:52.087 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:08:52.087 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": { 2023-11-06T02:08:52.087 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2023-11-06T02:08:52.087 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "192.168.0.1:0", 2023-11-06T02:08:52.087 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 3887548866 2023-11-06T02:08:52.088 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:08:52.088 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:08:52.088 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": { 2023-11-06T02:08:52.088 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2023-11-06T02:08:52.088 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "192.168.0.1:0", 2023-11-06T02:08:52.089 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 3887548866 2023-11-06T02:08:52.089 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:08:52.089 INFO:teuthology.orchestra.run.smithi049.stdout: "inst_str": "client.4752 192.168.0.1:0/3887548866", 2023-11-06T02:08:52.089 INFO:teuthology.orchestra.run.smithi049.stdout: "addr_str": "192.168.0.1:0/3887548866", 2023-11-06T02:08:52.089 INFO:teuthology.orchestra.run.smithi049.stdout: "inode_count": 1, 2023-11-06T02:08:52.089 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_epoch": 7, 2023-11-06T02:08:52.090 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_epoch": 23, 2023-11-06T02:08:52.090 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_epoch_barrier": 0, 2023-11-06T02:08:52.090 INFO:teuthology.orchestra.run.smithi049.stdout: "blocklisted": false, 2023-11-06T02:08:52.090 INFO:teuthology.orchestra.run.smithi049.stdout: "fs_name": "cephfs" 2023-11-06T02:08:52.090 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-11-06T02:08:52.097 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-06T02:08:52.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:52.226+0000 7fe515d41640 1 Processor -- start 2023-11-06T02:08:52.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:52.226+0000 7fe515d41640 1 -- start start 2023-11-06T02:08:52.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:52.226+0000 7fe515d41640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe51015ab50 0x7fe5100c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:52.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:52.226+0000 7fe515d41640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe5100cb010 0x7fe5100b0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:52.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:52.226+0000 7fe515d41640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5100cab80 con 0x7fe5100b05c0 2023-11-06T02:08:52.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:52.226+0000 7fe515d41640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5100b0a50 con 0x7fe5100cb010 2023-11-06T02:08:52.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:52.226+0000 7fe515d41640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe51015af30 con 0x7fe51015ab50 2023-11-06T02:08:52.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:52.226+0000 7fe50ffff640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fe5100b05c0 0x7fe5100700a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:37200/0 (socket says 172.21.15.49:37200) 2023-11-06T02:08:52.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:52.226+0000 7fe50ffff640 1 -- 172.21.15.49:0/4053428766 learned_addr learned my addr 172.21.15.49:0/4053428766 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:08:52.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:52.226+0000 7fe50effd640 1 --2- 172.21.15.49:0/4053428766 >> v2:172.21.15.49:3300/0 conn(0x7fe5100cb010 0x7fe5100b0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:52.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:52.226+0000 7fe50f7fe640 1 --2- 172.21.15.49:0/4053428766 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe51015ab50 0x7fe5100c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:52.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:52.230+0000 7fe50e7fc640 1 -- 172.21.15.49:0/4053428766 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2499983898 0 0) 0x7fe5100cab80 con 0x7fe5100b05c0 2023-11-06T02:08:52.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:52.230+0000 7fe50e7fc640 1 -- 172.21.15.49:0/4053428766 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe4f8003530 con 0x7fe5100b05c0 2023-11-06T02:08:52.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:52.230+0000 7fe50effd640 1 -- 172.21.15.49:0/4053428766 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe51015ab50 msgr2=0x7fe5100c9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:52.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:52.230+0000 7fe50effd640 1 --2- 172.21.15.49:0/4053428766 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe51015ab50 0x7fe5100c9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:52.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:52.230+0000 7fe50effd640 1 -- 172.21.15.49:0/4053428766 >> v1:172.21.15.49:6789/0 conn(0x7fe5100b05c0 legacy=0x7fe5100700a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:52.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:52.230+0000 7fe50effd640 1 -- 172.21.15.49:0/4053428766 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5100cb5b0 con 0x7fe5100cb010 2023-11-06T02:08:52.229 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:52.230+0000 7fe50effd640 1 --2- 172.21.15.49:0/4053428766 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5100cb010 0x7fe5100b0080 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe5040097c0 tx=0x7fe50401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3c673d3e4a67169 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:52.229 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:52.230+0000 7fe50e7fc640 1 -- 172.21.15.49:0/4053428766 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5040ab650 con 0x7fe5100cb010 2023-11-06T02:08:52.229 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:52.230+0000 7fe50e7fc640 1 -- 172.21.15.49:0/4053428766 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5040ab7b0 con 0x7fe5100cb010 2023-11-06T02:08:52.229 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:52.230+0000 7fe50e7fc640 1 -- 172.21.15.49:0/4053428766 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5040abb20 con 0x7fe5100cb010 2023-11-06T02:08:52.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:52.230+0000 7fe515d41640 1 -- 172.21.15.49:0/4053428766 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5100cb010 msgr2=0x7fe5100b0080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:52.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:52.230+0000 7fe515d41640 1 --2- 172.21.15.49:0/4053428766 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5100cb010 0x7fe5100b0080 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe5040097c0 tx=0x7fe50401c450 comp rx=0 tx=0).stop 2023-11-06T02:08:52.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:52.234+0000 7fe515d41640 1 -- 172.21.15.49:0/4053428766 shutdown_connections 2023-11-06T02:08:52.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:52.234+0000 7fe515d41640 1 --2- 172.21.15.49:0/4053428766 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5100cb010 0x7fe5100b0080 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:52.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:52.234+0000 7fe515d41640 1 --2- 172.21.15.49:0/4053428766 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe51015ab50 0x7fe5100c9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:52.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:52.234+0000 7fe515d41640 1 -- 172.21.15.49:0/4053428766 >> 172.21.15.49:0/4053428766 conn(0x7fe51006c0a0 msgr2=0x7fe51006c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:52.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:52.234+0000 7fe515d41640 1 -- 172.21.15.49:0/4053428766 shutdown_connections 2023-11-06T02:08:52.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:52.234+0000 7fe515d41640 1 -- 172.21.15.49:0/4053428766 wait complete. 2023-11-06T02:08:52.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:52.234+0000 7fe515d41640 1 Processor -- start 2023-11-06T02:08:52.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:52.234+0000 7fe515d41640 1 -- start start 2023-11-06T02:08:52.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:52.234+0000 7fe515d41640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5100b05c0 0x7fe5100c8660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:52.234 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:52.234+0000 7fe515d41640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe5100cb010 0x7fe510159070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:52.234 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:52.234+0000 7fe515d41640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe51015ab50 0x7fe5101502d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:52.234 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:52.234+0000 7fe515d41640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe51015a2d0 con 0x7fe5100b05c0 2023-11-06T02:08:52.234 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:52.234+0000 7fe515d41640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5100c73c0 con 0x7fe51015ab50 2023-11-06T02:08:52.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:52.234+0000 7fe515d41640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5100c76a0 con 0x7fe5100cb010 2023-11-06T02:08:52.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:52.234+0000 7fe50f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5100b05c0 0x7fe5100c8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:52.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:52.234+0000 7fe50f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5100b05c0 0x7fe5100c8660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47846/0 (socket says 172.21.15.49:47846) 2023-11-06T02:08:52.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:52.234+0000 7fe50f7fe640 1 -- 172.21.15.49:0/1467168067 learned_addr learned my addr 172.21.15.49:0/1467168067 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:52.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:52.234+0000 7fe50ffff640 1 --2- 172.21.15.49:0/1467168067 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe51015ab50 0x7fe5101502d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:52.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:52.234+0000 7fe50effd640 1 --2- 172.21.15.49:0/1467168067 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe5100cb010 0x7fe510159070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:52.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:52.234+0000 7fe50f7fe640 1 -- 172.21.15.49:0/1467168067 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe5100cb010 msgr2=0x7fe510159070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:52.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:52.234+0000 7fe50f7fe640 1 --2- 172.21.15.49:0/1467168067 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe5100cb010 0x7fe510159070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:52.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:52.234+0000 7fe50f7fe640 1 -- 172.21.15.49:0/1467168067 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe51015ab50 msgr2=0x7fe5101502d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:52.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:52.234+0000 7fe50f7fe640 1 --2- 172.21.15.49:0/1467168067 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe51015ab50 0x7fe5101502d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:52.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:52.234+0000 7fe50f7fe640 1 -- 172.21.15.49:0/1467168067 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe504009420 con 0x7fe5100b05c0 2023-11-06T02:08:52.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:52.234+0000 7fe50f7fe640 1 --2- 172.21.15.49:0/1467168067 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5100b05c0 0x7fe5100c8660 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe50000e870 tx=0x7fe50000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:52.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:52.234+0000 7fe4f7fff640 1 -- 172.21.15.49:0/1467168067 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5000095d0 con 0x7fe5100b05c0 2023-11-06T02:08:52.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:52.234+0000 7fe4f7fff640 1 -- 172.21.15.49:0/1467168067 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe50000c940 con 0x7fe5100b05c0 2023-11-06T02:08:52.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:52.234+0000 7fe4f7fff640 1 -- 172.21.15.49:0/1467168067 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe50000ccb0 con 0x7fe5100b05c0 2023-11-06T02:08:52.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:52.234+0000 7fe515d41640 1 -- 172.21.15.49:0/1467168067 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5100c7980 con 0x7fe5100b05c0 2023-11-06T02:08:52.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:52.234+0000 7fe515d41640 1 -- 172.21.15.49:0/1467168067 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe510159f80 con 0x7fe5100b05c0 2023-11-06T02:08:52.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:52.234+0000 7fe515d41640 1 -- 172.21.15.49:0/1467168067 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5100b0de0 con 0x7fe5100b05c0 2023-11-06T02:08:52.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:52.238+0000 7fe4f7fff640 1 -- 172.21.15.49:0/1467168067 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fe500013470 con 0x7fe5100b05c0 2023-11-06T02:08:52.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:52.238+0000 7fe4f7fff640 1 --2- 172.21.15.49:0/1467168067 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe4fc065a60 0x7fe4fc067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:52.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:52.238+0000 7fe4f7fff640 1 -- 172.21.15.49:0/1467168067 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe500083830 con 0x7fe5100b05c0 2023-11-06T02:08:52.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:52.238+0000 7fe50effd640 1 --2- 172.21.15.49:0/1467168067 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe4fc065a60 0x7fe4fc067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:52.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:52.238+0000 7fe50effd640 1 --2- 172.21.15.49:0/1467168067 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe4fc065a60 0x7fe4fc067f20 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe5040af000 tx=0x7fe5040a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:52.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:52.242+0000 7fe4f7fff640 1 -- 172.21.15.49:0/1467168067 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fe500051150 con 0x7fe5100b05c0 2023-11-06T02:08:52.430 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:52.434+0000 7fe515d41640 1 -- 172.21.15.49:0/1467168067 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe510074040 con 0x7fe5100b05c0 2023-11-06T02:08:52.431 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:52.434+0000 7fe4f7fff640 1 -- 172.21.15.49:0/1467168067 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7fe500021070 con 0x7fe5100b05c0 2023-11-06T02:08:52.432 INFO:teuthology.orchestra.run.smithi049.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-06T02:08:52.434 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:52.438+0000 7fe515d41640 1 -- 172.21.15.49:0/1467168067 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe4fc065a60 msgr2=0x7fe4fc067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:52.434 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:52.438+0000 7fe515d41640 1 --2- 172.21.15.49:0/1467168067 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe4fc065a60 0x7fe4fc067f20 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe5040af000 tx=0x7fe5040a7000 comp rx=0 tx=0).stop 2023-11-06T02:08:52.434 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:52.438+0000 7fe515d41640 1 -- 172.21.15.49:0/1467168067 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5100b05c0 msgr2=0x7fe5100c8660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:52.434 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:52.438+0000 7fe515d41640 1 --2- 172.21.15.49:0/1467168067 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5100b05c0 0x7fe5100c8660 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe50000e870 tx=0x7fe50000ed30 comp rx=0 tx=0).stop 2023-11-06T02:08:52.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:52.438+0000 7fe515d41640 1 -- 172.21.15.49:0/1467168067 shutdown_connections 2023-11-06T02:08:52.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:52.438+0000 7fe515d41640 1 --2- 172.21.15.49:0/1467168067 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe4fc065a60 0x7fe4fc067f20 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:52.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:52.438+0000 7fe515d41640 1 --2- 172.21.15.49:0/1467168067 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe51015ab50 0x7fe5101502d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:52.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:52.438+0000 7fe515d41640 1 --2- 172.21.15.49:0/1467168067 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe5100cb010 0x7fe510159070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:52.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:52.438+0000 7fe515d41640 1 --2- 172.21.15.49:0/1467168067 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5100b05c0 0x7fe5100c8660 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:52.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:52.438+0000 7fe515d41640 1 -- 172.21.15.49:0/1467168067 >> 172.21.15.49:0/1467168067 conn(0x7fe51006c0a0 msgr2=0x7fe5100af030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:52.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:52.438+0000 7fe515d41640 1 -- 172.21.15.49:0/1467168067 shutdown_connections 2023-11-06T02:08:52.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:52.438+0000 7fe515d41640 1 -- 172.21.15.49:0/1467168067 wait complete. 2023-11-06T02:08:52.451 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-06T02:08:52.451 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-06T02:08:52.452 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi049.front.sepia.ceph.com 2023-11-06T02:08:52.452 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-11-06T02:08:52.452 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-11-06T02:08:52.452 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-06T02:08:52.453 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-06T02:08:52.453 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-06T02:08:52.453 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-11-06T02:08:52.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:08:52.454 DEBUG:teuthology.orchestra.run.smithi049:> ip addr 2023-11-06T02:08:52.461 INFO:teuthology.orchestra.run.smithi049.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-06T02:08:52.461 INFO:teuthology.orchestra.run.smithi049.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-06T02:08:52.461 INFO:teuthology.orchestra.run.smithi049.stdout: inet 127.0.0.1/8 scope host lo 2023-11-06T02:08:52.462 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:08:52.462 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 ::1/128 scope host 2023-11-06T02:08:52.462 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:08:52.462 INFO:teuthology.orchestra.run.smithi049.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-06T02:08:52.463 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:6b:ff:84 brd ff:ff:ff:ff:ff:ff 2023-11-06T02:08:52.463 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp7s0f0 2023-11-06T02:08:52.463 INFO:teuthology.orchestra.run.smithi049.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-06T02:08:52.463 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:6b:ff:85 brd ff:ff:ff:ff:ff:ff 2023-11-06T02:08:52.464 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp7s0f1 2023-11-06T02:08:52.464 INFO:teuthology.orchestra.run.smithi049.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-06T02:08:52.464 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:bd:15:66 brd ff:ff:ff:ff:ff:ff 2023-11-06T02:08:52.464 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp1s0f0 2023-11-06T02:08:52.464 INFO:teuthology.orchestra.run.smithi049.stdout: inet 172.21.15.49/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-11-06T02:08:52.465 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft 42465sec preferred_lft 42465sec 2023-11-06T02:08:52.465 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 fe80::ec4:7aff:febd:1566/64 scope link 2023-11-06T02:08:52.465 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:08:52.465 INFO:teuthology.orchestra.run.smithi049.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-06T02:08:52.465 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:bd:15:67 brd ff:ff:ff:ff:ff:ff 2023-11-06T02:08:52.466 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp1s0f1 2023-11-06T02:08:52.466 INFO:teuthology.orchestra.run.smithi049.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-06T02:08:52.466 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 02:42:47:49:33:69 brd ff:ff:ff:ff:ff:ff 2023-11-06T02:08:52.466 INFO:teuthology.orchestra.run.smithi049.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-06T02:08:52.466 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:08:52.467 INFO:teuthology.orchestra.run.smithi049.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-06T02:08:52.467 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-11-06T02:08:52.467 INFO:teuthology.orchestra.run.smithi049.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-06T02:08:52.467 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:08:52.468 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-11-06T02:08:52.468 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:08:52.468 INFO:teuthology.orchestra.run.smithi049.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-06T02:08:52.468 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-06T02:08:52.469 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-11-06T02:08:52.469 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:08:52.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:08:52.470 DEBUG:teuthology.orchestra.run.smithi049:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-06T02:08:52.516 INFO:teuthology.orchestra.run.smithi049.stdout:1 2023-11-06T02:08:52.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:08:52.518 DEBUG:teuthology.orchestra.run.smithi049:> route 2023-11-06T02:08:52.567 INFO:teuthology.orchestra.run.smithi049.stdout:Kernel IP routing table 2023-11-06T02:08:52.567 INFO:teuthology.orchestra.run.smithi049.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-06T02:08:52.568 INFO:teuthology.orchestra.run.smithi049.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-11-06T02:08:52.568 INFO:teuthology.orchestra.run.smithi049.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-06T02:08:52.568 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-11-06T02:08:52.568 INFO:teuthology.orchestra.run.smithi049.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-06T02:08:52.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:08:52.569 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec bash -c ' 2023-11-06T02:08:52.569 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2023-11-06T02:08:52.570 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-06T02:08:52.570 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-06T02:08:52.570 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-06T02:08:52.571 DEBUG:teuthology.orchestra.run.smithi049:> ') 2023-11-06T02:08:52.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:08:52.647 DEBUG:teuthology.orchestra.run.smithi049:> ip netns list 2023-11-06T02:08:52.694 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-06T02:08:52.695 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:08:52.695 DEBUG:teuthology.orchestra.run.smithi049:> ip netns list-id 2023-11-06T02:08:52.746 INFO:teuthology.orchestra.run.smithi049.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-11-06T02:08:52.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:08:52.747 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec bash -c ' 2023-11-06T02:08:52.748 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2023-11-06T02:08:52.748 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-06T02:08:52.748 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-11-06T02:08:52.749 DEBUG:teuthology.orchestra.run.smithi049:> ') 2023-11-06T02:08:52.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:08:52.820 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-11-06T02:08:52.872 INFO:teuthology.orchestra.run.smithi049.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-06T02:08:52.872 INFO:teuthology.orchestra.run.smithi049.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-06T02:08:52.873 INFO:teuthology.orchestra.run.smithi049.stdout: inet 127.0.0.1/8 scope host lo 2023-11-06T02:08:52.873 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:08:52.873 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 ::1/128 scope host 2023-11-06T02:08:52.873 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:08:52.874 INFO:teuthology.orchestra.run.smithi049.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-06T02:08:52.874 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 46:39:b4:90:68:b2 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-06T02:08:52.874 INFO:teuthology.orchestra.run.smithi049.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-06T02:08:52.874 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:08:52.874 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 fe80::4439:b4ff:fe90:68b2/64 scope link 2023-11-06T02:08:52.875 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:08:52.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:08:52.876 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-11-06T02:08:52.930 INFO:teuthology.orchestra.run.smithi049.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-06T02:08:52.931 INFO:teuthology.orchestra.run.smithi049.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-06T02:08:52.931 INFO:teuthology.orchestra.run.smithi049.stdout: inet 127.0.0.1/8 scope host lo 2023-11-06T02:08:52.931 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:08:52.932 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 ::1/128 scope host 2023-11-06T02:08:52.932 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:08:52.932 INFO:teuthology.orchestra.run.smithi049.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-06T02:08:52.932 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 46:39:b4:90:68:b2 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-06T02:08:52.932 INFO:teuthology.orchestra.run.smithi049.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-06T02:08:52.933 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:08:52.933 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 fe80::4439:b4ff:fe90:68b2/64 scope link 2023-11-06T02:08:52.933 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:08:52.934 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-11-06T02:08:52.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:08:52.935 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec bash -c ' 2023-11-06T02:08:52.935 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2023-11-06T02:08:52.936 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-11-06T02:08:52.936 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-11-06T02:08:52.936 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-11-06T02:08:52.937 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-11-06T02:08:52.937 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-11-06T02:08:52.937 DEBUG:teuthology.orchestra.run.smithi049:> ') 2023-11-06T02:08:53.028 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:08:53.028 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec bash -c ' 2023-11-06T02:08:53.028 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2023-11-06T02:08:53.029 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link set brx.1 up 2023-11-06T02:08:53.029 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link set dev brx.1 master ceph-brx 2023-11-06T02:08:53.029 DEBUG:teuthology.orchestra.run.smithi049:> ') 2023-11-06T02:08:53.050 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2023-11-06T02:08:53.050 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-06T02:08:53.051 DEBUG:teuthology.orchestra.run.smithi049:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-11-06T02:08:53.097 INFO:teuthology.orchestra.run.smithi049.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-11-06T02:08:53.097 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-06T02:08:53.097 DEBUG:teuthology.orchestra.run.smithi049:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-11-06T02:08:53.145 DEBUG:teuthology.orchestra.run.smithi049:> sudo modprobe fuse 2023-11-06T02:08:53.205 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-06T02:08:53.206 DEBUG:teuthology.orchestra.run.smithi049:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-06T02:08:53.265 INFO:teuthology.orchestra.run.smithi049.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-06T02:08:53.266 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-06T02:08:53.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:08:53.268 DEBUG:teuthology.orchestra.run.smithi049:> ls /sys/fs/fuse/connections 2023-11-06T02:08:53.313 INFO:teuthology.orchestra.run.smithi049.stdout:45 2023-11-06T02:08:53.314 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [45] 2023-11-06T02:08:53.314 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2023-11-06T02:08:53.358 DEBUG:teuthology.orchestra.run.smithi049:> sudo modprobe fuse 2023-11-06T02:08:53.373 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-06T02:08:53.374 DEBUG:teuthology.orchestra.run.smithi049:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-06T02:08:53.426 INFO:teuthology.orchestra.run.smithi049.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-06T02:08:53.428 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-06T02:08:53.428 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:08:53.429 DEBUG:teuthology.orchestra.run.smithi049:> ls /sys/fs/fuse/connections 2023-11-06T02:08:53.442 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi049.stderr:2023-11-06T02:08:53.446+0000 7f357ac22580 -1 init, newargv = 0x559816b0aae0 newargc=15 2023-11-06T02:08:53.442 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi049.stderr:ceph-fuse[45086]: starting ceph client 2023-11-06T02:08:53.446 INFO:teuthology.orchestra.run.smithi049.stdout:45 2023-11-06T02:08:53.449 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi049.stderr:ceph-fuse[45086]: starting fuse 2023-11-06T02:08:54.549 DEBUG:teuthology.orchestra.run.smithi049:> sudo modprobe fuse 2023-11-06T02:08:54.566 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-06T02:08:54.567 DEBUG:teuthology.orchestra.run.smithi049:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-06T02:08:54.622 INFO:teuthology.orchestra.run.smithi049.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-06T02:08:54.623 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-06T02:08:54.624 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:08:54.625 DEBUG:teuthology.orchestra.run.smithi049:> ls /sys/fs/fuse/connections 2023-11-06T02:08:54.674 INFO:teuthology.orchestra.run.smithi049.stdout:45 2023-11-06T02:08:54.674 INFO:teuthology.orchestra.run.smithi049.stdout:46 2023-11-06T02:08:54.674 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45, 46] 2023-11-06T02:08:54.675 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-11-06T02:08:54.675 DEBUG:teuthology.orchestra.run.smithi049:> import glob 2023-11-06T02:08:54.675 DEBUG:teuthology.orchestra.run.smithi049:> import re 2023-11-06T02:08:54.676 DEBUG:teuthology.orchestra.run.smithi049:> import os 2023-11-06T02:08:54.676 DEBUG:teuthology.orchestra.run.smithi049:> import subprocess 2023-11-06T02:08:54.676 DEBUG:teuthology.orchestra.run.smithi049:> 2023-11-06T02:08:54.677 DEBUG:teuthology.orchestra.run.smithi049:> def _find_admin_socket(client_name): 2023-11-06T02:08:54.677 DEBUG:teuthology.orchestra.run.smithi049:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2023-11-06T02:08:54.677 DEBUG:teuthology.orchestra.run.smithi049:> files = glob.glob(asok_path) 2023-11-06T02:08:54.678 DEBUG:teuthology.orchestra.run.smithi049:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2023-11-06T02:08:54.678 DEBUG:teuthology.orchestra.run.smithi049:> 2023-11-06T02:08:54.678 DEBUG:teuthology.orchestra.run.smithi049:> # Given a non-glob path, it better be there 2023-11-06T02:08:54.679 DEBUG:teuthology.orchestra.run.smithi049:> if "*" not in asok_path: 2023-11-06T02:08:54.679 DEBUG:teuthology.orchestra.run.smithi049:> assert(len(files) == 1) 2023-11-06T02:08:54.679 DEBUG:teuthology.orchestra.run.smithi049:> return files[0] 2023-11-06T02:08:54.679 DEBUG:teuthology.orchestra.run.smithi049:> 2023-11-06T02:08:54.680 DEBUG:teuthology.orchestra.run.smithi049:> for f in files: 2023-11-06T02:08:54.680 DEBUG:teuthology.orchestra.run.smithi049:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-06T02:08:54.680 DEBUG:teuthology.orchestra.run.smithi049:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-06T02:08:54.680 DEBUG:teuthology.orchestra.run.smithi049:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-06T02:08:54.681 DEBUG:teuthology.orchestra.run.smithi049:> contents = proc_f.read() 2023-11-06T02:08:54.681 DEBUG:teuthology.orchestra.run.smithi049:> if mountpoint in contents: 2023-11-06T02:08:54.681 DEBUG:teuthology.orchestra.run.smithi049:> return f 2023-11-06T02:08:54.682 DEBUG:teuthology.orchestra.run.smithi049:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-06T02:08:54.682 DEBUG:teuthology.orchestra.run.smithi049:> 2023-11-06T02:08:54.682 DEBUG:teuthology.orchestra.run.smithi049:> print(_find_admin_socket("client.2")) 2023-11-06T02:08:54.683 DEBUG:teuthology.orchestra.run.smithi049:> ' 2023-11-06T02:08:54.797 INFO:teuthology.orchestra.run.smithi049.stdout:/var/run/ceph/ceph-client.2.45086.asok 2023-11-06T02:08:54.979 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.45086.asok 2023-11-06T02:08:54.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:08:54.980 DEBUG:teuthology.orchestra.run.smithi049:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.45086.asok status 2023-11-06T02:08:55.144 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-11-06T02:08:55.145 INFO:teuthology.orchestra.run.smithi049.stdout: "metadata": { 2023-11-06T02:08:55.145 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_sha1": "f7e9f9af51acf98e0ccc3037487018e4ae40caa5", 2023-11-06T02:08:55.145 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_version": "ceph version 18.2.0-1181-gf7e9f9af (f7e9f9af51acf98e0ccc3037487018e4ae40caa5) reef (stable)", 2023-11-06T02:08:55.146 INFO:teuthology.orchestra.run.smithi049.stdout: "entity_id": "2", 2023-11-06T02:08:55.146 INFO:teuthology.orchestra.run.smithi049.stdout: "hostname": "smithi049", 2023-11-06T02:08:55.146 INFO:teuthology.orchestra.run.smithi049.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2023-11-06T02:08:55.146 INFO:teuthology.orchestra.run.smithi049.stdout: "pid": "45086", 2023-11-06T02:08:55.147 INFO:teuthology.orchestra.run.smithi049.stdout: "root": "/" 2023-11-06T02:08:55.147 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:08:55.147 INFO:teuthology.orchestra.run.smithi049.stdout: "dentry_count": 0, 2023-11-06T02:08:55.147 INFO:teuthology.orchestra.run.smithi049.stdout: "dentry_pinned_count": 0, 2023-11-06T02:08:55.147 INFO:teuthology.orchestra.run.smithi049.stdout: "id": 4767, 2023-11-06T02:08:55.147 INFO:teuthology.orchestra.run.smithi049.stdout: "inst": { 2023-11-06T02:08:55.148 INFO:teuthology.orchestra.run.smithi049.stdout: "name": { 2023-11-06T02:08:55.148 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "client", 2023-11-06T02:08:55.148 INFO:teuthology.orchestra.run.smithi049.stdout: "num": 4767 2023-11-06T02:08:55.148 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:08:55.148 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": { 2023-11-06T02:08:55.149 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2023-11-06T02:08:55.149 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "192.168.0.2:0", 2023-11-06T02:08:55.149 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 1222871560 2023-11-06T02:08:55.149 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:08:55.150 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:08:55.150 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": { 2023-11-06T02:08:55.150 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2023-11-06T02:08:55.150 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "192.168.0.2:0", 2023-11-06T02:08:55.150 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 1222871560 2023-11-06T02:08:55.151 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:08:55.151 INFO:teuthology.orchestra.run.smithi049.stdout: "inst_str": "client.4767 192.168.0.2:0/1222871560", 2023-11-06T02:08:55.151 INFO:teuthology.orchestra.run.smithi049.stdout: "addr_str": "192.168.0.2:0/1222871560", 2023-11-06T02:08:55.151 INFO:teuthology.orchestra.run.smithi049.stdout: "inode_count": 1, 2023-11-06T02:08:55.152 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_epoch": 7, 2023-11-06T02:08:55.152 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_epoch": 23, 2023-11-06T02:08:55.152 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_epoch_barrier": 0, 2023-11-06T02:08:55.152 INFO:teuthology.orchestra.run.smithi049.stdout: "blocklisted": false, 2023-11-06T02:08:55.152 INFO:teuthology.orchestra.run.smithi049.stdout: "fs_name": "cephfs" 2023-11-06T02:08:55.153 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-11-06T02:08:55.159 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-06T02:08:55.282 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.286+0000 7f70f6958640 1 Processor -- start 2023-11-06T02:08:55.282 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.286+0000 7f70f6958640 1 -- start start 2023-11-06T02:08:55.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.286+0000 7f70f6958640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f70f015ab50 0x7f70f00c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:55.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.286+0000 7f70f6958640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f70f00b0790 0x7f70f00c2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:55.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.286+0000 7f70f6958640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70f00cab80 con 0x7f70f00cb010 2023-11-06T02:08:55.286 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.286+0000 7f70f6958640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70f015af30 con 0x7f70f00b0790 2023-11-06T02:08:55.286 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.286+0000 7f70f6958640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70f00cb5b0 con 0x7f70f015ab50 2023-11-06T02:08:55.286 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.286+0000 7f70ef7fe640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f70f00cb010 0x7f70f00b0080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:37214/0 (socket says 172.21.15.49:37214) 2023-11-06T02:08:55.286 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.286+0000 7f70ef7fe640 1 -- 172.21.15.49:0/2132151865 learned_addr learned my addr 172.21.15.49:0/2132151865 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:08:55.286 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.286+0000 7f70effff640 1 --2- 172.21.15.49:0/2132151865 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f70f015ab50 0x7f70f00c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:55.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.286+0000 7f70f4ef5640 1 --2- 172.21.15.49:0/2132151865 >> v2:172.21.15.150:3300/0 conn(0x7f70f00b0790 0x7f70f00c2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:55.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.286+0000 7f70eeffd640 1 -- 172.21.15.49:0/2132151865 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1373247434 0 0) 0x7f70f00cab80 con 0x7f70f00cb010 2023-11-06T02:08:55.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.286+0000 7f70eeffd640 1 -- 172.21.15.49:0/2132151865 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70d8003530 con 0x7f70f00cb010 2023-11-06T02:08:55.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.286+0000 7f70eeffd640 1 -- 172.21.15.49:0/2132151865 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3285027766 0 0) 0x7f70d8003530 con 0x7f70f00cb010 2023-11-06T02:08:55.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.286+0000 7f70eeffd640 1 -- 172.21.15.49:0/2132151865 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f70f015ab50 msgr2=0x7f70f00c9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:55.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.286+0000 7f70eeffd640 1 --2- 172.21.15.49:0/2132151865 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f70f015ab50 0x7f70f00c9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:55.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.286+0000 7f70eeffd640 1 -- 172.21.15.49:0/2132151865 >> v2:172.21.15.150:3300/0 conn(0x7f70f00b0790 msgr2=0x7f70f00c2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:55.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.286+0000 7f70eeffd640 1 --2- 172.21.15.49:0/2132151865 >> v2:172.21.15.150:3300/0 conn(0x7f70f00b0790 0x7f70f00c2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:55.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.286+0000 7f70eeffd640 1 -- 172.21.15.49:0/2132151865 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70f00cb890 con 0x7f70f00cb010 2023-11-06T02:08:55.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.286+0000 7f70eeffd640 1 -- 172.21.15.49:0/2132151865 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7f70e4001680 con 0x7f70f00cb010 2023-11-06T02:08:55.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.286+0000 7f70eeffd640 1 -- 172.21.15.49:0/2132151865 >> v1:172.21.15.49:6789/0 conn(0x7f70f00cb010 legacy=0x7f70f00b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:55.289 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.286+0000 7f70eeffd640 1 --2- 172.21.15.49:0/2132151865 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f70f015ab50 0x7f70d8003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:55.289 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.286+0000 7f70eeffd640 1 --2- 172.21.15.49:0/2132151865 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70d8003f00 0x7f70d80047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:55.289 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.286+0000 7f70eeffd640 1 --2- 172.21.15.49:0/2132151865 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f70d8004d10 0x7f70d8005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:55.289 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.286+0000 7f70eeffd640 1 -- 172.21.15.49:0/2132151865 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70f015af30 con 0x7f70d8004d10 2023-11-06T02:08:55.289 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.286+0000 7f70eeffd640 1 -- 172.21.15.49:0/2132151865 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70f00cb5b0 con 0x7f70d8003f00 2023-11-06T02:08:55.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.286+0000 7f70eeffd640 1 -- 172.21.15.49:0/2132151865 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70d800b990 con 0x7f70f015ab50 2023-11-06T02:08:55.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.286+0000 7f70f4ef5640 1 --2- 172.21.15.49:0/2132151865 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f70d8004d10 0x7f70d8005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:55.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.286+0000 7f70ef7fe640 1 --2- 172.21.15.49:0/2132151865 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70d8003f00 0x7f70d80047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:55.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.286+0000 7f70effff640 1 --2- 172.21.15.49:0/2132151865 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f70f015ab50 0x7f70d8003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:55.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.286+0000 7f70f4ef5640 1 -- 172.21.15.49:0/2132151865 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f70f015ab50 msgr2=0x7f70d8003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:55.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.286+0000 7f70f4ef5640 1 --2- 172.21.15.49:0/2132151865 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f70f015ab50 0x7f70d8003980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:55.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.290+0000 7f70f4ef5640 1 -- 172.21.15.49:0/2132151865 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70d8003f00 msgr2=0x7f70d80047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:55.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.290+0000 7f70f4ef5640 1 --2- 172.21.15.49:0/2132151865 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70d8003f00 0x7f70d80047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:55.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.290+0000 7f70f4ef5640 1 -- 172.21.15.49:0/2132151865 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70d8003530 con 0x7f70d8004d10 2023-11-06T02:08:55.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.290+0000 7f70ef7fe640 1 --2- 172.21.15.49:0/2132151865 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70d8003f00 0x7f70d80047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:08:55.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.290+0000 7f70f4ef5640 1 --2- 172.21.15.49:0/2132151865 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f70d8004d10 0x7f70d8005190 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f70f0070050 tx=0x7f70e801cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82a8dc262d79668d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:55.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.290+0000 7f70eeffd640 1 -- 172.21.15.49:0/2132151865 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e80a5c50 con 0x7f70d8004d10 2023-11-06T02:08:55.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.290+0000 7f70eeffd640 1 -- 172.21.15.49:0/2132151865 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70e800f040 con 0x7f70d8004d10 2023-11-06T02:08:55.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.290+0000 7f70eeffd640 1 -- 172.21.15.49:0/2132151865 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e80ad440 con 0x7f70d8004d10 2023-11-06T02:08:55.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.290+0000 7f70f6958640 1 -- 172.21.15.49:0/2132151865 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f70d8004d10 msgr2=0x7f70d8005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:55.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.290+0000 7f70f6958640 1 --2- 172.21.15.49:0/2132151865 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f70d8004d10 0x7f70d8005190 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f70f0070050 tx=0x7f70e801cbe0 comp rx=0 tx=0).stop 2023-11-06T02:08:55.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.290+0000 7f70f4ef5640 1 -- 172.21.15.49:0/2132151865 reap_dead start 2023-11-06T02:08:55.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.290+0000 7f70f6958640 1 -- 172.21.15.49:0/2132151865 shutdown_connections 2023-11-06T02:08:55.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.290+0000 7f70f6958640 1 -- 172.21.15.49:0/2132151865 >> 172.21.15.49:0/2132151865 conn(0x7f70f006c0a0 msgr2=0x7f70f006c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:55.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.290+0000 7f70f6958640 1 -- 172.21.15.49:0/2132151865 shutdown_connections 2023-11-06T02:08:55.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.290+0000 7f70f6958640 1 -- 172.21.15.49:0/2132151865 wait complete. 2023-11-06T02:08:55.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.294+0000 7f70f6958640 1 Processor -- start 2023-11-06T02:08:55.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.294+0000 7f70f6958640 1 -- start start 2023-11-06T02:08:55.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.294+0000 7f70f6958640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f70f014da10 0x7f70f00c7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:55.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.294+0000 7f70f6958640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f70f015caa0 0x7f70f015cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:55.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.294+0000 7f70f6958640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70f01f5940 0x7f70f01f5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:55.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.294+0000 7f70f6958640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70f015d440 con 0x7f70f015caa0 2023-11-06T02:08:55.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.294+0000 7f70f6958640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70f015b1f0 con 0x7f70f01f5940 2023-11-06T02:08:55.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.294+0000 7f70f6958640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70f015b4d0 con 0x7f70f014da10 2023-11-06T02:08:55.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.294+0000 7f70ef7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f70f015caa0 0x7f70f015cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:55.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.294+0000 7f70ef7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f70f015caa0 0x7f70f015cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47876/0 (socket says 172.21.15.49:47876) 2023-11-06T02:08:55.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.294+0000 7f70ef7fe640 1 -- 172.21.15.49:0/1262690660 learned_addr learned my addr 172.21.15.49:0/1262690660 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:55.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.294+0000 7f70f4ef5640 1 --2- 172.21.15.49:0/1262690660 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70f01f5940 0x7f70f01f5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:55.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.294+0000 7f70effff640 1 --2- 172.21.15.49:0/1262690660 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f70f014da10 0x7f70f00c7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:55.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.294+0000 7f70ef7fe640 1 -- 172.21.15.49:0/1262690660 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f70f014da10 msgr2=0x7f70f00c7ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:55.300 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.294+0000 7f70ef7fe640 1 --2- 172.21.15.49:0/1262690660 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f70f014da10 0x7f70f00c7ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:55.300 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.294+0000 7f70ef7fe640 1 -- 172.21.15.49:0/1262690660 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70f01f5940 msgr2=0x7f70f01f5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:55.300 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.294+0000 7f70ef7fe640 1 --2- 172.21.15.49:0/1262690660 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70f01f5940 0x7f70f01f5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:55.300 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.294+0000 7f70ef7fe640 1 -- 172.21.15.49:0/1262690660 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70f015b7b0 con 0x7f70f015caa0 2023-11-06T02:08:55.300 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.294+0000 7f70effff640 1 --2- 172.21.15.49:0/1262690660 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f70f014da10 0x7f70f00c7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:08:55.300 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.294+0000 7f70ef7fe640 1 --2- 172.21.15.49:0/1262690660 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f70f015caa0 0x7f70f015cf00 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f70e400dae0 tx=0x7f70e4007e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:55.301 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.294+0000 7f70ecff9640 1 -- 172.21.15.49:0/1262690660 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e4004cc0 con 0x7f70f015caa0 2023-11-06T02:08:55.301 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.294+0000 7f70ecff9640 1 -- 172.21.15.49:0/1262690660 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70e4004e20 con 0x7f70f015caa0 2023-11-06T02:08:55.301 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.294+0000 7f70f6958640 1 -- 172.21.15.49:0/1262690660 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70f015ba90 con 0x7f70f015caa0 2023-11-06T02:08:55.301 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.294+0000 7f70ecff9640 1 -- 172.21.15.49:0/1262690660 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e40072f0 con 0x7f70f015caa0 2023-11-06T02:08:55.301 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.294+0000 7f70f6958640 1 -- 172.21.15.49:0/1262690660 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70f00c79c0 con 0x7f70f015caa0 2023-11-06T02:08:55.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.294+0000 7f70f6958640 1 -- 172.21.15.49:0/1262690660 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70f00b0de0 con 0x7f70f015caa0 2023-11-06T02:08:55.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.294+0000 7f70ecff9640 1 -- 172.21.15.49:0/1262690660 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f70e4007450 con 0x7f70f015caa0 2023-11-06T02:08:55.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.294+0000 7f70ecff9640 1 --2- 172.21.15.49:0/1262690660 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f70dc065a60 0x7f70dc067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:55.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.294+0000 7f70ecff9640 1 -- 172.21.15.49:0/1262690660 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f70e40831a0 con 0x7f70f015caa0 2023-11-06T02:08:55.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.298+0000 7f70effff640 1 --2- 172.21.15.49:0/1262690660 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f70dc065a60 0x7f70dc067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:55.303 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.298+0000 7f70effff640 1 --2- 172.21.15.49:0/1262690660 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f70dc065a60 0x7f70dc067f20 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f70e00099e0 tx=0x7f70e0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:55.303 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.298+0000 7f70ecff9640 1 -- 172.21.15.49:0/1262690660 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f70e4050a00 con 0x7f70f015caa0 2023-11-06T02:08:55.490 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.494+0000 7f70f6958640 1 -- 172.21.15.49:0/1262690660 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f70f0074040 con 0x7f70f015caa0 2023-11-06T02:08:55.490 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.494+0000 7f70ecff9640 1 -- 172.21.15.49:0/1262690660 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7f70e4019060 con 0x7f70f015caa0 2023-11-06T02:08:55.491 INFO:teuthology.orchestra.run.smithi049.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-06T02:08:55.493 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.494+0000 7f70f6958640 1 -- 172.21.15.49:0/1262690660 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f70dc065a60 msgr2=0x7f70dc067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:55.493 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.494+0000 7f70f6958640 1 --2- 172.21.15.49:0/1262690660 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f70dc065a60 0x7f70dc067f20 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f70e00099e0 tx=0x7f70e0008040 comp rx=0 tx=0).stop 2023-11-06T02:08:55.493 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.494+0000 7f70f6958640 1 -- 172.21.15.49:0/1262690660 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f70f015caa0 msgr2=0x7f70f015cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:55.493 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.494+0000 7f70f6958640 1 --2- 172.21.15.49:0/1262690660 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f70f015caa0 0x7f70f015cf00 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f70e400dae0 tx=0x7f70e4007e80 comp rx=0 tx=0).stop 2023-11-06T02:08:55.494 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.494+0000 7f70f6958640 1 -- 172.21.15.49:0/1262690660 shutdown_connections 2023-11-06T02:08:55.494 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.494+0000 7f70f6958640 1 --2- 172.21.15.49:0/1262690660 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f70dc065a60 0x7f70dc067f20 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:55.494 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.494+0000 7f70f6958640 1 --2- 172.21.15.49:0/1262690660 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f70f015caa0 0x7f70f015cf00 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:55.494 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.494+0000 7f70f6958640 1 --2- 172.21.15.49:0/1262690660 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70f01f5940 0x7f70f01f5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:55.495 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.494+0000 7f70f6958640 1 --2- 172.21.15.49:0/1262690660 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f70f014da10 0x7f70f00c7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:55.495 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.494+0000 7f70f6958640 1 -- 172.21.15.49:0/1262690660 >> 172.21.15.49:0/1262690660 conn(0x7f70f006c0a0 msgr2=0x7f70f00ac540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:55.495 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.498+0000 7f70f6958640 1 -- 172.21.15.49:0/1262690660 shutdown_connections 2023-11-06T02:08:55.495 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:55.498+0000 7f70f6958640 1 -- 172.21.15.49:0/1262690660 wait complete. 2023-11-06T02:08:55.510 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-06T02:08:55.511 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-06T02:08:55.511 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi150.front.sepia.ceph.com 2023-11-06T02:08:55.511 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-11-06T02:08:55.512 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-06T02:08:55.512 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-06T02:08:55.512 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-06T02:08:55.512 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-06T02:08:55.513 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-06T02:08:55.513 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:08:55.514 DEBUG:teuthology.orchestra.run.smithi150:> ip addr 2023-11-06T02:08:55.521 INFO:teuthology.orchestra.run.smithi150.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-06T02:08:55.521 INFO:teuthology.orchestra.run.smithi150.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-06T02:08:55.521 INFO:teuthology.orchestra.run.smithi150.stdout: inet 127.0.0.1/8 scope host lo 2023-11-06T02:08:55.522 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:08:55.522 INFO:teuthology.orchestra.run.smithi150.stdout: inet6 ::1/128 scope host 2023-11-06T02:08:55.522 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:08:55.523 INFO:teuthology.orchestra.run.smithi150.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-06T02:08:55.523 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 0c:c4:7a:d9:da:bc brd ff:ff:ff:ff:ff:ff 2023-11-06T02:08:55.523 INFO:teuthology.orchestra.run.smithi150.stdout: altname enp7s0f0 2023-11-06T02:08:55.524 INFO:teuthology.orchestra.run.smithi150.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-06T02:08:55.524 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 0c:c4:7a:d9:da:bd brd ff:ff:ff:ff:ff:ff 2023-11-06T02:08:55.524 INFO:teuthology.orchestra.run.smithi150.stdout: altname enp7s0f1 2023-11-06T02:08:55.525 INFO:teuthology.orchestra.run.smithi150.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-06T02:08:55.525 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 0c:c4:7a:88:6e:34 brd ff:ff:ff:ff:ff:ff 2023-11-06T02:08:55.525 INFO:teuthology.orchestra.run.smithi150.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-06T02:08:55.525 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 0c:c4:7a:88:6e:35 brd ff:ff:ff:ff:ff:ff 2023-11-06T02:08:55.526 INFO:teuthology.orchestra.run.smithi150.stdout: inet 172.21.15.150/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-11-06T02:08:55.526 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft 42405sec preferred_lft 42405sec 2023-11-06T02:08:55.526 INFO:teuthology.orchestra.run.smithi150.stdout: inet6 fe80::ec4:7aff:fe88:6e35/64 scope link 2023-11-06T02:08:55.526 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:08:55.526 INFO:teuthology.orchestra.run.smithi150.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-06T02:08:55.527 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 02:42:13:9f:6f:a7 brd ff:ff:ff:ff:ff:ff 2023-11-06T02:08:55.527 INFO:teuthology.orchestra.run.smithi150.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-06T02:08:55.527 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:08:55.527 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-06T02:08:55.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:08:55.528 DEBUG:teuthology.orchestra.run.smithi150:> (cd / && exec bash -c ' 2023-11-06T02:08:55.528 DEBUG:teuthology.orchestra.run.smithi150:> set -e 2023-11-06T02:08:55.528 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link add name ceph-brx type bridge 2023-11-06T02:08:55.529 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip addr flush dev ceph-brx 2023-11-06T02:08:55.529 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link set ceph-brx up 2023-11-06T02:08:55.530 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-06T02:08:55.530 DEBUG:teuthology.orchestra.run.smithi150:> ') 2023-11-06T02:08:55.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:08:55.608 DEBUG:teuthology.orchestra.run.smithi150:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-06T02:08:55.662 INFO:teuthology.orchestra.run.smithi150.stdout:1 2023-11-06T02:08:55.664 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:08:55.664 DEBUG:teuthology.orchestra.run.smithi150:> route 2023-11-06T02:08:55.740 INFO:teuthology.orchestra.run.smithi150.stdout:Kernel IP routing table 2023-11-06T02:08:55.740 INFO:teuthology.orchestra.run.smithi150.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-06T02:08:55.740 INFO:teuthology.orchestra.run.smithi150.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-11-06T02:08:55.740 INFO:teuthology.orchestra.run.smithi150.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-06T02:08:55.741 INFO:teuthology.orchestra.run.smithi150.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-06T02:08:55.741 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-06T02:08:55.741 INFO:teuthology.orchestra.run.smithi150.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-06T02:08:55.741 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-06T02:08:55.741 INFO:teuthology.orchestra.run.smithi150.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-06T02:08:55.742 INFO:teuthology.orchestra.run.smithi150.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-06T02:08:55.742 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:08:55.742 DEBUG:teuthology.orchestra.run.smithi150:> (cd / && exec bash -c ' 2023-11-06T02:08:55.743 DEBUG:teuthology.orchestra.run.smithi150:> set -e 2023-11-06T02:08:55.743 DEBUG:teuthology.orchestra.run.smithi150:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-06T02:08:55.743 DEBUG:teuthology.orchestra.run.smithi150:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-06T02:08:55.744 DEBUG:teuthology.orchestra.run.smithi150:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-06T02:08:55.744 DEBUG:teuthology.orchestra.run.smithi150:> ') 2023-11-06T02:08:55.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:08:55.815 DEBUG:teuthology.orchestra.run.smithi150:> ip netns list 2023-11-06T02:08:55.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:08:55.862 DEBUG:teuthology.orchestra.run.smithi150:> ip netns list-id 2023-11-06T02:08:55.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:08:55.909 DEBUG:teuthology.orchestra.run.smithi150:> (cd / && exec bash -c ' 2023-11-06T02:08:55.910 DEBUG:teuthology.orchestra.run.smithi150:> set -e 2023-11-06T02:08:55.910 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-06T02:08:55.910 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-11-06T02:08:55.910 DEBUG:teuthology.orchestra.run.smithi150:> ') 2023-11-06T02:08:55.979 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-11-06T02:08:55.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:08:55.980 DEBUG:teuthology.orchestra.run.smithi150:> (cd / && exec bash -c ' 2023-11-06T02:08:55.980 DEBUG:teuthology.orchestra.run.smithi150:> set -e 2023-11-06T02:08:55.981 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-11-06T02:08:55.981 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-06T02:08:55.981 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-11-06T02:08:55.982 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-11-06T02:08:55.982 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-11-06T02:08:55.982 DEBUG:teuthology.orchestra.run.smithi150:> ') 2023-11-06T02:08:56.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:08:56.080 DEBUG:teuthology.orchestra.run.smithi150:> (cd / && exec bash -c ' 2023-11-06T02:08:56.080 DEBUG:teuthology.orchestra.run.smithi150:> set -e 2023-11-06T02:08:56.080 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link set brx.0 up 2023-11-06T02:08:56.081 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link set dev brx.0 master ceph-brx 2023-11-06T02:08:56.081 DEBUG:teuthology.orchestra.run.smithi150:> ') 2023-11-06T02:08:56.100 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-11-06T02:08:56.100 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-06T02:08:56.100 DEBUG:teuthology.orchestra.run.smithi150:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-11-06T02:08:56.145 INFO:teuthology.orchestra.run.smithi150.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-11-06T02:08:56.145 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-06T02:08:56.146 DEBUG:teuthology.orchestra.run.smithi150:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-11-06T02:08:56.197 DEBUG:teuthology.orchestra.run.smithi150:> sudo modprobe fuse 2023-11-06T02:08:56.258 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-06T02:08:56.258 DEBUG:teuthology.orchestra.run.smithi150:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-06T02:08:56.312 INFO:teuthology.orchestra.run.smithi150.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-06T02:08:56.313 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-06T02:08:56.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:08:56.314 DEBUG:teuthology.orchestra.run.smithi150:> ls /sys/fs/fuse/connections 2023-11-06T02:08:56.361 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-06T02:08:56.361 DEBUG:teuthology.orchestra.run.smithi150:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-11-06T02:08:56.406 DEBUG:teuthology.orchestra.run.smithi150:> sudo modprobe fuse 2023-11-06T02:08:56.421 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-06T02:08:56.421 DEBUG:teuthology.orchestra.run.smithi150:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-06T02:08:56.481 INFO:teuthology.orchestra.run.smithi150.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-06T02:08:56.482 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-06T02:08:56.483 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:08:56.483 DEBUG:teuthology.orchestra.run.smithi150:> ls /sys/fs/fuse/connections 2023-11-06T02:08:56.487 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi150.stderr:2023-11-06T02:08:56.487+0000 7f8f89bc4580 -1 init, newargv = 0x55d19ba95e10 newargc=15 2023-11-06T02:08:56.488 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi150.stderr:ceph-fuse[44227]: starting ceph client 2023-11-06T02:08:56.495 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi150.stderr:ceph-fuse[44227]: starting fuse 2023-11-06T02:08:57.594 DEBUG:teuthology.orchestra.run.smithi150:> sudo modprobe fuse 2023-11-06T02:08:57.611 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-06T02:08:57.611 DEBUG:teuthology.orchestra.run.smithi150:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-06T02:08:57.669 INFO:teuthology.orchestra.run.smithi150.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-06T02:08:57.670 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-06T02:08:57.671 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:08:57.671 DEBUG:teuthology.orchestra.run.smithi150:> ls /sys/fs/fuse/connections 2023-11-06T02:08:57.722 INFO:teuthology.orchestra.run.smithi150.stdout:45 2023-11-06T02:08:57.722 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2023-11-06T02:08:57.722 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-11-06T02:08:57.723 DEBUG:teuthology.orchestra.run.smithi150:> import glob 2023-11-06T02:08:57.723 DEBUG:teuthology.orchestra.run.smithi150:> import re 2023-11-06T02:08:57.723 DEBUG:teuthology.orchestra.run.smithi150:> import os 2023-11-06T02:08:57.724 DEBUG:teuthology.orchestra.run.smithi150:> import subprocess 2023-11-06T02:08:57.724 DEBUG:teuthology.orchestra.run.smithi150:> 2023-11-06T02:08:57.724 DEBUG:teuthology.orchestra.run.smithi150:> def _find_admin_socket(client_name): 2023-11-06T02:08:57.724 DEBUG:teuthology.orchestra.run.smithi150:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-11-06T02:08:57.725 DEBUG:teuthology.orchestra.run.smithi150:> files = glob.glob(asok_path) 2023-11-06T02:08:57.725 DEBUG:teuthology.orchestra.run.smithi150:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-11-06T02:08:57.725 DEBUG:teuthology.orchestra.run.smithi150:> 2023-11-06T02:08:57.725 DEBUG:teuthology.orchestra.run.smithi150:> # Given a non-glob path, it better be there 2023-11-06T02:08:57.726 DEBUG:teuthology.orchestra.run.smithi150:> if "*" not in asok_path: 2023-11-06T02:08:57.726 DEBUG:teuthology.orchestra.run.smithi150:> assert(len(files) == 1) 2023-11-06T02:08:57.726 DEBUG:teuthology.orchestra.run.smithi150:> return files[0] 2023-11-06T02:08:57.727 DEBUG:teuthology.orchestra.run.smithi150:> 2023-11-06T02:08:57.727 DEBUG:teuthology.orchestra.run.smithi150:> for f in files: 2023-11-06T02:08:57.727 DEBUG:teuthology.orchestra.run.smithi150:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-06T02:08:57.727 DEBUG:teuthology.orchestra.run.smithi150:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-06T02:08:57.728 DEBUG:teuthology.orchestra.run.smithi150:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-06T02:08:57.728 DEBUG:teuthology.orchestra.run.smithi150:> contents = proc_f.read() 2023-11-06T02:08:57.728 DEBUG:teuthology.orchestra.run.smithi150:> if mountpoint in contents: 2023-11-06T02:08:57.728 DEBUG:teuthology.orchestra.run.smithi150:> return f 2023-11-06T02:08:57.729 DEBUG:teuthology.orchestra.run.smithi150:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-06T02:08:57.729 DEBUG:teuthology.orchestra.run.smithi150:> 2023-11-06T02:08:57.729 DEBUG:teuthology.orchestra.run.smithi150:> print(_find_admin_socket("client.1")) 2023-11-06T02:08:57.729 DEBUG:teuthology.orchestra.run.smithi150:> ' 2023-11-06T02:08:57.841 INFO:teuthology.orchestra.run.smithi150.stdout:/var/run/ceph/ceph-client.1.44227.asok 2023-11-06T02:08:58.024 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.44227.asok 2023-11-06T02:08:58.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:08:58.025 DEBUG:teuthology.orchestra.run.smithi150:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.44227.asok status 2023-11-06T02:08:58.184 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2023-11-06T02:08:58.184 INFO:teuthology.orchestra.run.smithi150.stdout: "metadata": { 2023-11-06T02:08:58.184 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_sha1": "f7e9f9af51acf98e0ccc3037487018e4ae40caa5", 2023-11-06T02:08:58.185 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_version": "ceph version 18.2.0-1181-gf7e9f9af (f7e9f9af51acf98e0ccc3037487018e4ae40caa5) reef (stable)", 2023-11-06T02:08:58.185 INFO:teuthology.orchestra.run.smithi150.stdout: "entity_id": "1", 2023-11-06T02:08:58.185 INFO:teuthology.orchestra.run.smithi150.stdout: "hostname": "smithi150", 2023-11-06T02:08:58.185 INFO:teuthology.orchestra.run.smithi150.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-11-06T02:08:58.185 INFO:teuthology.orchestra.run.smithi150.stdout: "pid": "44227", 2023-11-06T02:08:58.186 INFO:teuthology.orchestra.run.smithi150.stdout: "root": "/" 2023-11-06T02:08:58.186 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-06T02:08:58.187 INFO:teuthology.orchestra.run.smithi150.stdout: "dentry_count": 0, 2023-11-06T02:08:58.187 INFO:teuthology.orchestra.run.smithi150.stdout: "dentry_pinned_count": 0, 2023-11-06T02:08:58.187 INFO:teuthology.orchestra.run.smithi150.stdout: "id": 4648, 2023-11-06T02:08:58.187 INFO:teuthology.orchestra.run.smithi150.stdout: "inst": { 2023-11-06T02:08:58.187 INFO:teuthology.orchestra.run.smithi150.stdout: "name": { 2023-11-06T02:08:58.188 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "client", 2023-11-06T02:08:58.188 INFO:teuthology.orchestra.run.smithi150.stdout: "num": 4648 2023-11-06T02:08:58.188 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-06T02:08:58.189 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": { 2023-11-06T02:08:58.189 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2023-11-06T02:08:58.189 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "192.168.0.1:0", 2023-11-06T02:08:58.189 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 48018205 2023-11-06T02:08:58.190 INFO:teuthology.orchestra.run.smithi150.stdout: } 2023-11-06T02:08:58.190 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-06T02:08:58.190 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": { 2023-11-06T02:08:58.190 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2023-11-06T02:08:58.190 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "192.168.0.1:0", 2023-11-06T02:08:58.191 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 48018205 2023-11-06T02:08:58.191 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-06T02:08:58.191 INFO:teuthology.orchestra.run.smithi150.stdout: "inst_str": "client.4648 192.168.0.1:0/48018205", 2023-11-06T02:08:58.191 INFO:teuthology.orchestra.run.smithi150.stdout: "addr_str": "192.168.0.1:0/48018205", 2023-11-06T02:08:58.191 INFO:teuthology.orchestra.run.smithi150.stdout: "inode_count": 1, 2023-11-06T02:08:58.192 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_epoch": 7, 2023-11-06T02:08:58.192 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_epoch": 23, 2023-11-06T02:08:58.192 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_epoch_barrier": 0, 2023-11-06T02:08:58.192 INFO:teuthology.orchestra.run.smithi150.stdout: "blocklisted": false, 2023-11-06T02:08:58.192 INFO:teuthology.orchestra.run.smithi150.stdout: "fs_name": "cephfs" 2023-11-06T02:08:58.193 INFO:teuthology.orchestra.run.smithi150.stdout:} 2023-11-06T02:08:58.198 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-06T02:08:58.335 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:58.338+0000 7eff59570640 1 Processor -- start 2023-11-06T02:08:58.335 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:58.338+0000 7eff59570640 1 -- start start 2023-11-06T02:08:58.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:58.338+0000 7eff59570640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7eff5415ab50 0x7eff540c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:58.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:58.338+0000 7eff59570640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7eff540cb010 0x7eff540b0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:58.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:58.338+0000 7eff59570640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7eff540b05c0 0x7eff540700a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:58.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:58.338+0000 7eff59570640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff540b0a50 con 0x7eff540cb010 2023-11-06T02:08:58.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:58.338+0000 7eff59570640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff540cab80 con 0x7eff540b05c0 2023-11-06T02:08:58.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:58.338+0000 7eff59570640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff5415af30 con 0x7eff5415ab50 2023-11-06T02:08:58.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:58.338+0000 7eff527fc640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7eff540cb010 0x7eff540b0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:58.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:58.338+0000 7eff527fc640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7eff540cb010 0x7eff540b0080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47884/0 (socket says 172.21.15.49:47884) 2023-11-06T02:08:58.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:58.338+0000 7eff527fc640 1 -- 172.21.15.49:0/485781977 learned_addr learned my addr 172.21.15.49:0/485781977 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:58.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:58.338+0000 7eff52ffd640 1 --2- 172.21.15.49:0/485781977 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7eff5415ab50 0x7eff540c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:58.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:58.338+0000 7eff537fe640 1 --2- 172.21.15.49:0/485781977 >> v2:172.21.15.150:3300/0 conn(0x7eff540b05c0 0x7eff540700a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:58.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:58.342+0000 7eff527fc640 1 -- 172.21.15.49:0/485781977 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7eff5415ab50 msgr2=0x7eff540c9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:58.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:58.342+0000 7eff527fc640 1 --2- 172.21.15.49:0/485781977 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7eff5415ab50 0x7eff540c9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:58.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:58.342+0000 7eff527fc640 1 -- 172.21.15.49:0/485781977 >> v2:172.21.15.150:3300/0 conn(0x7eff540b05c0 msgr2=0x7eff540700a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:08:58.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:58.342+0000 7eff527fc640 1 --2- 172.21.15.49:0/485781977 >> v2:172.21.15.150:3300/0 conn(0x7eff540b05c0 0x7eff540700a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:58.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:58.342+0000 7eff527fc640 1 -- 172.21.15.49:0/485781977 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff540cb5b0 con 0x7eff540cb010 2023-11-06T02:08:58.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:58.342+0000 7eff527fc640 1 --2- 172.21.15.49:0/485781977 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff540cb010 0x7eff540b0080 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7eff540cae30 tx=0x7eff4801c210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c96603f26b859aac server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:58.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:58.342+0000 7eff51ffb640 1 -- 172.21.15.49:0/485781977 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff480ab600 con 0x7eff540cb010 2023-11-06T02:08:58.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:58.342+0000 7eff51ffb640 1 -- 172.21.15.49:0/485781977 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff480a7050 con 0x7eff540cb010 2023-11-06T02:08:58.344 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:58.342+0000 7eff51ffb640 1 -- 172.21.15.49:0/485781977 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff480ab980 con 0x7eff540cb010 2023-11-06T02:08:58.344 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:58.342+0000 7eff59570640 1 -- 172.21.15.49:0/485781977 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff540cb010 msgr2=0x7eff540b0080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:58.344 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:58.342+0000 7eff59570640 1 --2- 172.21.15.49:0/485781977 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff540cb010 0x7eff540b0080 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7eff540cae30 tx=0x7eff4801c210 comp rx=0 tx=0).stop 2023-11-06T02:08:58.344 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:58.342+0000 7eff59570640 1 -- 172.21.15.49:0/485781977 shutdown_connections 2023-11-06T02:08:58.344 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:58.342+0000 7eff59570640 1 --2- 172.21.15.49:0/485781977 >> v2:172.21.15.150:3300/0 conn(0x7eff540b05c0 0x7eff540700a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:58.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:58.342+0000 7eff59570640 1 --2- 172.21.15.49:0/485781977 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff540cb010 0x7eff540b0080 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:58.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:58.342+0000 7eff59570640 1 --2- 172.21.15.49:0/485781977 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7eff5415ab50 0x7eff540c9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:58.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:58.342+0000 7eff59570640 1 -- 172.21.15.49:0/485781977 >> 172.21.15.49:0/485781977 conn(0x7eff5406c0a0 msgr2=0x7eff5406c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:58.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:58.342+0000 7eff59570640 1 -- 172.21.15.49:0/485781977 shutdown_connections 2023-11-06T02:08:58.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:58.342+0000 7eff59570640 1 -- 172.21.15.49:0/485781977 wait complete. 2023-11-06T02:08:58.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:58.346+0000 7eff59570640 1 Processor -- start 2023-11-06T02:08:58.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:58.346+0000 7eff59570640 1 -- start start 2023-11-06T02:08:58.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:58.346+0000 7eff59570640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff540b05c0 0x7eff54152680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:58.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:58.346+0000 7eff59570640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7eff540cb010 0x7eff540c6530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:58.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:58.346+0000 7eff59570640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7eff5415ab50 0x7eff540c6a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:58.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:58.346+0000 7eff59570640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff540ad200 con 0x7eff540b05c0 2023-11-06T02:08:58.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:58.346+0000 7eff59570640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff540c7290 con 0x7eff540cb010 2023-11-06T02:08:58.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:58.346+0000 7eff59570640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff540c7570 con 0x7eff5415ab50 2023-11-06T02:08:58.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:58.346+0000 7eff52ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff540b05c0 0x7eff54152680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:58.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:58.346+0000 7eff52ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff540b05c0 0x7eff54152680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47898/0 (socket says 172.21.15.49:47898) 2023-11-06T02:08:58.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:58.346+0000 7eff52ffd640 1 -- 172.21.15.49:0/3584473866 learned_addr learned my addr 172.21.15.49:0/3584473866 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:08:58.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:58.346+0000 7eff537fe640 1 --2- 172.21.15.49:0/3584473866 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7eff5415ab50 0x7eff540c6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:58.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:58.346+0000 7eff527fc640 1 --2- 172.21.15.49:0/3584473866 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7eff540cb010 0x7eff540c6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:58.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:58.346+0000 7eff52ffd640 1 -- 172.21.15.49:0/3584473866 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7eff5415ab50 msgr2=0x7eff540c6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:58.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:58.346+0000 7eff52ffd640 1 --2- 172.21.15.49:0/3584473866 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7eff5415ab50 0x7eff540c6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:58.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:58.346+0000 7eff52ffd640 1 -- 172.21.15.49:0/3584473866 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7eff540cb010 msgr2=0x7eff540c6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:58.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:58.346+0000 7eff52ffd640 1 --2- 172.21.15.49:0/3584473866 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7eff540cb010 0x7eff540c6530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:58.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:58.346+0000 7eff52ffd640 1 -- 172.21.15.49:0/3584473866 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff480094d0 con 0x7eff540b05c0 2023-11-06T02:08:58.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:58.346+0000 7eff52ffd640 1 --2- 172.21.15.49:0/3584473866 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff540b05c0 0x7eff54152680 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7eff4400e870 tx=0x7eff4400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:58.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:58.346+0000 7eff3b7fe640 1 -- 172.21.15.49:0/3584473866 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff440095d0 con 0x7eff540b05c0 2023-11-06T02:08:58.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:58.346+0000 7eff3b7fe640 1 -- 172.21.15.49:0/3584473866 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff4400c940 con 0x7eff540b05c0 2023-11-06T02:08:58.350 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:58.346+0000 7eff59570640 1 -- 172.21.15.49:0/3584473866 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff540c7850 con 0x7eff540b05c0 2023-11-06T02:08:58.350 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:58.346+0000 7eff3b7fe640 1 -- 172.21.15.49:0/3584473866 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff4400ccb0 con 0x7eff540b05c0 2023-11-06T02:08:58.350 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:58.346+0000 7eff59570640 1 -- 172.21.15.49:0/3584473866 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff540c6fb0 con 0x7eff540b05c0 2023-11-06T02:08:58.351 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:58.346+0000 7eff59570640 1 -- 172.21.15.49:0/3584473866 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff540b0de0 con 0x7eff540b05c0 2023-11-06T02:08:58.351 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:58.346+0000 7eff3b7fe640 1 -- 172.21.15.49:0/3584473866 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7eff44013470 con 0x7eff540b05c0 2023-11-06T02:08:58.351 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:58.346+0000 7eff3b7fe640 1 --2- 172.21.15.49:0/3584473866 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7eff40065a60 0x7eff40067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:08:58.352 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:58.350+0000 7eff3b7fe640 1 -- 172.21.15.49:0/3584473866 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7eff440837f0 con 0x7eff540b05c0 2023-11-06T02:08:58.352 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:58.350+0000 7eff527fc640 1 --2- 172.21.15.49:0/3584473866 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7eff40065a60 0x7eff40067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:08:58.352 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:58.350+0000 7eff527fc640 1 --2- 172.21.15.49:0/3584473866 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7eff40065a60 0x7eff40067f20 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7eff480ad000 tx=0x7eff480a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:08:58.352 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:58.354+0000 7eff3b7fe640 1 -- 172.21.15.49:0/3584473866 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7eff440511a0 con 0x7eff540b05c0 2023-11-06T02:08:58.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:58.546+0000 7eff59570640 1 -- 172.21.15.49:0/3584473866 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7eff54074040 con 0x7eff540b05c0 2023-11-06T02:08:58.542 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:58.546+0000 7eff3b7fe640 1 -- 172.21.15.49:0/3584473866 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7eff44017070 con 0x7eff540b05c0 2023-11-06T02:08:58.542 INFO:teuthology.orchestra.run.smithi049.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-06T02:08:58.545 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:58.546+0000 7eff59570640 1 -- 172.21.15.49:0/3584473866 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7eff40065a60 msgr2=0x7eff40067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:58.545 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:58.546+0000 7eff59570640 1 --2- 172.21.15.49:0/3584473866 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7eff40065a60 0x7eff40067f20 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7eff480ad000 tx=0x7eff480a6000 comp rx=0 tx=0).stop 2023-11-06T02:08:58.545 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:58.546+0000 7eff59570640 1 -- 172.21.15.49:0/3584473866 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff540b05c0 msgr2=0x7eff54152680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:08:58.545 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:58.546+0000 7eff59570640 1 --2- 172.21.15.49:0/3584473866 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff540b05c0 0x7eff54152680 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7eff4400e870 tx=0x7eff4400ed30 comp rx=0 tx=0).stop 2023-11-06T02:08:58.545 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:58.546+0000 7eff59570640 1 -- 172.21.15.49:0/3584473866 shutdown_connections 2023-11-06T02:08:58.546 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:58.546+0000 7eff59570640 1 --2- 172.21.15.49:0/3584473866 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7eff40065a60 0x7eff40067f20 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:58.546 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:58.546+0000 7eff59570640 1 --2- 172.21.15.49:0/3584473866 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7eff5415ab50 0x7eff540c6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:58.546 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:58.546+0000 7eff59570640 1 --2- 172.21.15.49:0/3584473866 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7eff540cb010 0x7eff540c6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:58.546 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:58.546+0000 7eff59570640 1 --2- 172.21.15.49:0/3584473866 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff540b05c0 0x7eff54152680 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:08:58.546 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:58.546+0000 7eff59570640 1 -- 172.21.15.49:0/3584473866 >> 172.21.15.49:0/3584473866 conn(0x7eff5406c0a0 msgr2=0x7eff540c4080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:08:58.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:58.546+0000 7eff59570640 1 -- 172.21.15.49:0/3584473866 shutdown_connections 2023-11-06T02:08:58.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:08:58.546+0000 7eff59570640 1 -- 172.21.15.49:0/3584473866 wait complete. 2023-11-06T02:08:58.563 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-06T02:08:58.563 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-06T02:08:58.563 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi150.front.sepia.ceph.com 2023-11-06T02:08:58.563 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-11-06T02:08:58.564 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-11-06T02:08:58.564 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-06T02:08:58.564 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-06T02:08:58.564 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-06T02:08:58.565 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-11-06T02:08:58.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:08:58.565 DEBUG:teuthology.orchestra.run.smithi150:> ip addr 2023-11-06T02:08:58.572 INFO:teuthology.orchestra.run.smithi150.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-06T02:08:58.572 INFO:teuthology.orchestra.run.smithi150.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-06T02:08:58.573 INFO:teuthology.orchestra.run.smithi150.stdout: inet 127.0.0.1/8 scope host lo 2023-11-06T02:08:58.573 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:08:58.573 INFO:teuthology.orchestra.run.smithi150.stdout: inet6 ::1/128 scope host 2023-11-06T02:08:58.573 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:08:58.574 INFO:teuthology.orchestra.run.smithi150.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-06T02:08:58.574 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 0c:c4:7a:d9:da:bc brd ff:ff:ff:ff:ff:ff 2023-11-06T02:08:58.574 INFO:teuthology.orchestra.run.smithi150.stdout: altname enp7s0f0 2023-11-06T02:08:58.574 INFO:teuthology.orchestra.run.smithi150.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-06T02:08:58.574 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 0c:c4:7a:d9:da:bd brd ff:ff:ff:ff:ff:ff 2023-11-06T02:08:58.575 INFO:teuthology.orchestra.run.smithi150.stdout: altname enp7s0f1 2023-11-06T02:08:58.575 INFO:teuthology.orchestra.run.smithi150.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-06T02:08:58.575 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 0c:c4:7a:88:6e:34 brd ff:ff:ff:ff:ff:ff 2023-11-06T02:08:58.575 INFO:teuthology.orchestra.run.smithi150.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-06T02:08:58.575 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 0c:c4:7a:88:6e:35 brd ff:ff:ff:ff:ff:ff 2023-11-06T02:08:58.576 INFO:teuthology.orchestra.run.smithi150.stdout: inet 172.21.15.150/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-11-06T02:08:58.576 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft 42402sec preferred_lft 42402sec 2023-11-06T02:08:58.576 INFO:teuthology.orchestra.run.smithi150.stdout: inet6 fe80::ec4:7aff:fe88:6e35/64 scope link 2023-11-06T02:08:58.576 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:08:58.576 INFO:teuthology.orchestra.run.smithi150.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-06T02:08:58.576 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 02:42:13:9f:6f:a7 brd ff:ff:ff:ff:ff:ff 2023-11-06T02:08:58.577 INFO:teuthology.orchestra.run.smithi150.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-06T02:08:58.577 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:08:58.577 INFO:teuthology.orchestra.run.smithi150.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-06T02:08:58.577 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-11-06T02:08:58.577 INFO:teuthology.orchestra.run.smithi150.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-06T02:08:58.578 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:08:58.578 INFO:teuthology.orchestra.run.smithi150.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-11-06T02:08:58.578 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:08:58.578 INFO:teuthology.orchestra.run.smithi150.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-06T02:08:58.578 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-06T02:08:58.579 INFO:teuthology.orchestra.run.smithi150.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-11-06T02:08:58.579 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:08:58.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:08:58.579 DEBUG:teuthology.orchestra.run.smithi150:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-06T02:08:58.633 INFO:teuthology.orchestra.run.smithi150.stdout:1 2023-11-06T02:08:58.634 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:08:58.635 DEBUG:teuthology.orchestra.run.smithi150:> route 2023-11-06T02:08:58.684 INFO:teuthology.orchestra.run.smithi150.stdout:Kernel IP routing table 2023-11-06T02:08:58.685 INFO:teuthology.orchestra.run.smithi150.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-06T02:08:58.685 INFO:teuthology.orchestra.run.smithi150.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-11-06T02:08:58.685 INFO:teuthology.orchestra.run.smithi150.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-06T02:08:58.685 INFO:teuthology.orchestra.run.smithi150.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-06T02:08:58.685 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-06T02:08:58.685 INFO:teuthology.orchestra.run.smithi150.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-06T02:08:58.686 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-06T02:08:58.686 INFO:teuthology.orchestra.run.smithi150.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-06T02:08:58.686 INFO:teuthology.orchestra.run.smithi150.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-06T02:08:58.686 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:08:58.687 DEBUG:teuthology.orchestra.run.smithi150:> (cd / && exec bash -c ' 2023-11-06T02:08:58.687 DEBUG:teuthology.orchestra.run.smithi150:> set -e 2023-11-06T02:08:58.687 DEBUG:teuthology.orchestra.run.smithi150:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-06T02:08:58.688 DEBUG:teuthology.orchestra.run.smithi150:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-06T02:08:58.688 DEBUG:teuthology.orchestra.run.smithi150:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-06T02:08:58.688 DEBUG:teuthology.orchestra.run.smithi150:> ') 2023-11-06T02:08:58.765 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:08:58.765 DEBUG:teuthology.orchestra.run.smithi150:> ip netns list 2023-11-06T02:08:58.813 INFO:teuthology.orchestra.run.smithi150.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-06T02:08:58.814 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:08:58.814 DEBUG:teuthology.orchestra.run.smithi150:> ip netns list-id 2023-11-06T02:08:58.861 INFO:teuthology.orchestra.run.smithi150.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-11-06T02:08:58.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:08:58.862 DEBUG:teuthology.orchestra.run.smithi150:> (cd / && exec bash -c ' 2023-11-06T02:08:58.862 DEBUG:teuthology.orchestra.run.smithi150:> set -e 2023-11-06T02:08:58.862 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-06T02:08:58.863 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-11-06T02:08:58.863 DEBUG:teuthology.orchestra.run.smithi150:> ') 2023-11-06T02:08:58.930 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:08:58.931 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-11-06T02:08:58.985 INFO:teuthology.orchestra.run.smithi150.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-06T02:08:58.985 INFO:teuthology.orchestra.run.smithi150.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-06T02:08:58.985 INFO:teuthology.orchestra.run.smithi150.stdout: inet 127.0.0.1/8 scope host lo 2023-11-06T02:08:58.985 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:08:58.986 INFO:teuthology.orchestra.run.smithi150.stdout: inet6 ::1/128 scope host 2023-11-06T02:08:58.986 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:08:58.986 INFO:teuthology.orchestra.run.smithi150.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-06T02:08:58.986 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 42:5d:2c:16:6a:d4 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-06T02:08:58.986 INFO:teuthology.orchestra.run.smithi150.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-06T02:08:58.987 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:08:58.987 INFO:teuthology.orchestra.run.smithi150.stdout: inet6 fe80::405d:2cff:fe16:6ad4/64 scope link 2023-11-06T02:08:58.987 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:08:58.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:08:58.988 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-11-06T02:08:59.045 INFO:teuthology.orchestra.run.smithi150.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-06T02:08:59.046 INFO:teuthology.orchestra.run.smithi150.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-06T02:08:59.046 INFO:teuthology.orchestra.run.smithi150.stdout: inet 127.0.0.1/8 scope host lo 2023-11-06T02:08:59.046 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:08:59.046 INFO:teuthology.orchestra.run.smithi150.stdout: inet6 ::1/128 scope host 2023-11-06T02:08:59.047 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:08:59.047 INFO:teuthology.orchestra.run.smithi150.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-06T02:08:59.047 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 42:5d:2c:16:6a:d4 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-06T02:08:59.047 INFO:teuthology.orchestra.run.smithi150.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-06T02:08:59.047 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:08:59.047 INFO:teuthology.orchestra.run.smithi150.stdout: inet6 fe80::405d:2cff:fe16:6ad4/64 scope link 2023-11-06T02:08:59.048 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:08:59.049 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-11-06T02:08:59.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:08:59.049 DEBUG:teuthology.orchestra.run.smithi150:> (cd / && exec bash -c ' 2023-11-06T02:08:59.050 DEBUG:teuthology.orchestra.run.smithi150:> set -e 2023-11-06T02:08:59.050 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-11-06T02:08:59.050 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-11-06T02:08:59.051 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-11-06T02:08:59.051 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-11-06T02:08:59.051 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-11-06T02:08:59.052 DEBUG:teuthology.orchestra.run.smithi150:> ') 2023-11-06T02:08:59.147 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:08:59.147 DEBUG:teuthology.orchestra.run.smithi150:> (cd / && exec bash -c ' 2023-11-06T02:08:59.148 DEBUG:teuthology.orchestra.run.smithi150:> set -e 2023-11-06T02:08:59.148 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link set brx.1 up 2023-11-06T02:08:59.148 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link set dev brx.1 master ceph-brx 2023-11-06T02:08:59.148 DEBUG:teuthology.orchestra.run.smithi150:> ') 2023-11-06T02:08:59.169 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2023-11-06T02:08:59.169 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-06T02:08:59.169 DEBUG:teuthology.orchestra.run.smithi150:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-11-06T02:08:59.217 INFO:teuthology.orchestra.run.smithi150.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-11-06T02:08:59.217 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-06T02:08:59.218 DEBUG:teuthology.orchestra.run.smithi150:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-11-06T02:08:59.266 DEBUG:teuthology.orchestra.run.smithi150:> sudo modprobe fuse 2023-11-06T02:08:59.322 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-06T02:08:59.322 DEBUG:teuthology.orchestra.run.smithi150:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-06T02:08:59.377 INFO:teuthology.orchestra.run.smithi150.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-06T02:08:59.378 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-06T02:08:59.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:08:59.379 DEBUG:teuthology.orchestra.run.smithi150:> ls /sys/fs/fuse/connections 2023-11-06T02:08:59.426 INFO:teuthology.orchestra.run.smithi150.stdout:45 2023-11-06T02:08:59.426 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [45] 2023-11-06T02:08:59.426 DEBUG:teuthology.orchestra.run.smithi150:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2023-11-06T02:08:59.474 DEBUG:teuthology.orchestra.run.smithi150:> sudo modprobe fuse 2023-11-06T02:08:59.489 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-06T02:08:59.490 DEBUG:teuthology.orchestra.run.smithi150:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-06T02:08:59.543 INFO:teuthology.orchestra.run.smithi150.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-06T02:08:59.544 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-06T02:08:59.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:08:59.545 DEBUG:teuthology.orchestra.run.smithi150:> ls /sys/fs/fuse/connections 2023-11-06T02:08:59.551 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi150.stderr:2023-11-06T02:08:59.551+0000 7f005c466580 -1 init, newargv = 0x5587ae307e20 newargc=15 2023-11-06T02:08:59.551 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi150.stderr:ceph-fuse[44340]: starting ceph client 2023-11-06T02:08:59.555 INFO:teuthology.orchestra.run.smithi150.stdout:45 2023-11-06T02:08:59.558 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi150.stderr:ceph-fuse[44340]: starting fuse 2023-11-06T02:09:00.658 DEBUG:teuthology.orchestra.run.smithi150:> sudo modprobe fuse 2023-11-06T02:09:00.674 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-06T02:09:00.675 DEBUG:teuthology.orchestra.run.smithi150:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-06T02:09:00.728 INFO:teuthology.orchestra.run.smithi150.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-06T02:09:00.729 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-06T02:09:00.730 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:09:00.730 DEBUG:teuthology.orchestra.run.smithi150:> ls /sys/fs/fuse/connections 2023-11-06T02:09:00.777 INFO:teuthology.orchestra.run.smithi150.stdout:45 2023-11-06T02:09:00.778 INFO:teuthology.orchestra.run.smithi150.stdout:46 2023-11-06T02:09:00.778 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45, 46] 2023-11-06T02:09:00.778 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-11-06T02:09:00.779 DEBUG:teuthology.orchestra.run.smithi150:> import glob 2023-11-06T02:09:00.779 DEBUG:teuthology.orchestra.run.smithi150:> import re 2023-11-06T02:09:00.779 DEBUG:teuthology.orchestra.run.smithi150:> import os 2023-11-06T02:09:00.780 DEBUG:teuthology.orchestra.run.smithi150:> import subprocess 2023-11-06T02:09:00.780 DEBUG:teuthology.orchestra.run.smithi150:> 2023-11-06T02:09:00.780 DEBUG:teuthology.orchestra.run.smithi150:> def _find_admin_socket(client_name): 2023-11-06T02:09:00.781 DEBUG:teuthology.orchestra.run.smithi150:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2023-11-06T02:09:00.781 DEBUG:teuthology.orchestra.run.smithi150:> files = glob.glob(asok_path) 2023-11-06T02:09:00.781 DEBUG:teuthology.orchestra.run.smithi150:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2023-11-06T02:09:00.781 DEBUG:teuthology.orchestra.run.smithi150:> 2023-11-06T02:09:00.782 DEBUG:teuthology.orchestra.run.smithi150:> # Given a non-glob path, it better be there 2023-11-06T02:09:00.782 DEBUG:teuthology.orchestra.run.smithi150:> if "*" not in asok_path: 2023-11-06T02:09:00.782 DEBUG:teuthology.orchestra.run.smithi150:> assert(len(files) == 1) 2023-11-06T02:09:00.783 DEBUG:teuthology.orchestra.run.smithi150:> return files[0] 2023-11-06T02:09:00.783 DEBUG:teuthology.orchestra.run.smithi150:> 2023-11-06T02:09:00.783 DEBUG:teuthology.orchestra.run.smithi150:> for f in files: 2023-11-06T02:09:00.784 DEBUG:teuthology.orchestra.run.smithi150:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-06T02:09:00.784 DEBUG:teuthology.orchestra.run.smithi150:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-06T02:09:00.784 DEBUG:teuthology.orchestra.run.smithi150:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-06T02:09:00.784 DEBUG:teuthology.orchestra.run.smithi150:> contents = proc_f.read() 2023-11-06T02:09:00.785 DEBUG:teuthology.orchestra.run.smithi150:> if mountpoint in contents: 2023-11-06T02:09:00.785 DEBUG:teuthology.orchestra.run.smithi150:> return f 2023-11-06T02:09:00.785 DEBUG:teuthology.orchestra.run.smithi150:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-06T02:09:00.786 DEBUG:teuthology.orchestra.run.smithi150:> 2023-11-06T02:09:00.786 DEBUG:teuthology.orchestra.run.smithi150:> print(_find_admin_socket("client.3")) 2023-11-06T02:09:00.786 DEBUG:teuthology.orchestra.run.smithi150:> ' 2023-11-06T02:09:00.898 INFO:teuthology.orchestra.run.smithi150.stdout:/var/run/ceph/ceph-client.3.44340.asok 2023-11-06T02:09:01.080 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.44340.asok 2023-11-06T02:09:01.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:09:01.081 DEBUG:teuthology.orchestra.run.smithi150:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.44340.asok status 2023-11-06T02:09:01.241 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2023-11-06T02:09:01.241 INFO:teuthology.orchestra.run.smithi150.stdout: "metadata": { 2023-11-06T02:09:01.242 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_sha1": "f7e9f9af51acf98e0ccc3037487018e4ae40caa5", 2023-11-06T02:09:01.242 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_version": "ceph version 18.2.0-1181-gf7e9f9af (f7e9f9af51acf98e0ccc3037487018e4ae40caa5) reef (stable)", 2023-11-06T02:09:01.242 INFO:teuthology.orchestra.run.smithi150.stdout: "entity_id": "3", 2023-11-06T02:09:01.242 INFO:teuthology.orchestra.run.smithi150.stdout: "hostname": "smithi150", 2023-11-06T02:09:01.242 INFO:teuthology.orchestra.run.smithi150.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2023-11-06T02:09:01.243 INFO:teuthology.orchestra.run.smithi150.stdout: "pid": "44340", 2023-11-06T02:09:01.243 INFO:teuthology.orchestra.run.smithi150.stdout: "root": "/" 2023-11-06T02:09:01.243 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-06T02:09:01.243 INFO:teuthology.orchestra.run.smithi150.stdout: "dentry_count": 0, 2023-11-06T02:09:01.243 INFO:teuthology.orchestra.run.smithi150.stdout: "dentry_pinned_count": 0, 2023-11-06T02:09:01.244 INFO:teuthology.orchestra.run.smithi150.stdout: "id": 4544, 2023-11-06T02:09:01.244 INFO:teuthology.orchestra.run.smithi150.stdout: "inst": { 2023-11-06T02:09:01.244 INFO:teuthology.orchestra.run.smithi150.stdout: "name": { 2023-11-06T02:09:01.244 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "client", 2023-11-06T02:09:01.244 INFO:teuthology.orchestra.run.smithi150.stdout: "num": 4544 2023-11-06T02:09:01.245 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-06T02:09:01.245 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": { 2023-11-06T02:09:01.245 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2023-11-06T02:09:01.245 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "192.168.0.2:0", 2023-11-06T02:09:01.245 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 1763258648 2023-11-06T02:09:01.246 INFO:teuthology.orchestra.run.smithi150.stdout: } 2023-11-06T02:09:01.246 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-06T02:09:01.246 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": { 2023-11-06T02:09:01.246 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2023-11-06T02:09:01.246 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "192.168.0.2:0", 2023-11-06T02:09:01.247 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 1763258648 2023-11-06T02:09:01.247 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-06T02:09:01.247 INFO:teuthology.orchestra.run.smithi150.stdout: "inst_str": "client.4544 192.168.0.2:0/1763258648", 2023-11-06T02:09:01.247 INFO:teuthology.orchestra.run.smithi150.stdout: "addr_str": "192.168.0.2:0/1763258648", 2023-11-06T02:09:01.247 INFO:teuthology.orchestra.run.smithi150.stdout: "inode_count": 1, 2023-11-06T02:09:01.247 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_epoch": 7, 2023-11-06T02:09:01.248 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_epoch": 23, 2023-11-06T02:09:01.248 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_epoch_barrier": 0, 2023-11-06T02:09:01.248 INFO:teuthology.orchestra.run.smithi150.stdout: "blocklisted": false, 2023-11-06T02:09:01.248 INFO:teuthology.orchestra.run.smithi150.stdout: "fs_name": "cephfs" 2023-11-06T02:09:01.248 INFO:teuthology.orchestra.run.smithi150.stdout:} 2023-11-06T02:09:01.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:09:01.255 DEBUG:teuthology.orchestra.run.smithi049:> stat --file-system '--printf=%T 2023-11-06T02:09:01.255 DEBUG:teuthology.orchestra.run.smithi049:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-06T02:09:01.263 INFO:teuthology.orchestra.run.smithi049.stdout:fuseblk 2023-11-06T02:09:01.263 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-06T02:09:01.263 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:09:01.264 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-06T02:09:01.322 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:09:01.323 DEBUG:teuthology.orchestra.run.smithi049:> stat --file-system '--printf=%T 2023-11-06T02:09:01.323 DEBUG:teuthology.orchestra.run.smithi049:> ' -- /home/ubuntu/cephtest/mnt.2 2023-11-06T02:09:01.371 INFO:teuthology.orchestra.run.smithi049.stdout:fuseblk 2023-11-06T02:09:01.371 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2023-11-06T02:09:01.371 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:09:01.372 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-11-06T02:09:01.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:09:01.432 DEBUG:teuthology.orchestra.run.smithi150:> stat --file-system '--printf=%T 2023-11-06T02:09:01.433 DEBUG:teuthology.orchestra.run.smithi150:> ' -- /home/ubuntu/cephtest/mnt.1 2023-11-06T02:09:01.440 INFO:teuthology.orchestra.run.smithi150.stdout:fuseblk 2023-11-06T02:09:01.440 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-11-06T02:09:01.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:09:01.441 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-11-06T02:09:01.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:09:01.498 DEBUG:teuthology.orchestra.run.smithi150:> stat --file-system '--printf=%T 2023-11-06T02:09:01.498 DEBUG:teuthology.orchestra.run.smithi150:> ' -- /home/ubuntu/cephtest/mnt.3 2023-11-06T02:09:01.546 INFO:teuthology.orchestra.run.smithi150.stdout:fuseblk 2023-11-06T02:09:01.547 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2023-11-06T02:09:01.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:09:01.547 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-11-06T02:09:01.601 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-11-06T02:09:01.619 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T02:09:01.753 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:01.758+0000 7ff4bac03640 1 Processor -- start 2023-11-06T02:09:01.753 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:01.758+0000 7ff4bac03640 1 -- start start 2023-11-06T02:09:01.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:01.758+0000 7ff4bac03640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7ff4b415ab50 0x7ff4b40c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:01.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:01.758+0000 7ff4bac03640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4b40cb010 0x7ff4b40b0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:01.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:01.758+0000 7ff4bac03640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff4b40b05c0 0x7ff4b40700a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:01.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:01.758+0000 7ff4bac03640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4b40b0a50 con 0x7ff4b40b05c0 2023-11-06T02:09:01.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:01.758+0000 7ff4bac03640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4b40cab80 con 0x7ff4b415ab50 2023-11-06T02:09:01.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:01.758+0000 7ff4bac03640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b415af30 con 0x7ff4b40cb010 2023-11-06T02:09:01.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:01.758+0000 7ff4b91a0640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff4b40b05c0 0x7ff4b40700a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:01.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:01.758+0000 7ff4b91a0640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff4b40b05c0 0x7ff4b40700a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47566/0 (socket says 172.21.15.49:47566) 2023-11-06T02:09:01.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:01.758+0000 7ff4b91a0640 1 -- 172.21.15.49:0/4248199586 learned_addr learned my addr 172.21.15.49:0/4248199586 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:01.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:01.758+0000 7ff4b3fff640 1 --2- 172.21.15.49:0/4248199586 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4b40cb010 0x7ff4b40b0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:01.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:01.758+0000 7ff4b899f640 1 --2- 172.21.15.49:0/4248199586 >> v2:172.21.15.150:3300/0 conn(0x7ff4b415ab50 0x7ff4b40c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:01.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:01.758+0000 7ff4b91a0640 1 -- 172.21.15.49:0/4248199586 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4b40cb010 msgr2=0x7ff4b40b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:01.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:01.758+0000 7ff4b91a0640 1 --2- 172.21.15.49:0/4248199586 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4b40cb010 0x7ff4b40b0080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:01.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:01.758+0000 7ff4b91a0640 1 -- 172.21.15.49:0/4248199586 >> v2:172.21.15.150:3300/0 conn(0x7ff4b415ab50 msgr2=0x7ff4b40c9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:01.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:01.758+0000 7ff4b91a0640 1 --2- 172.21.15.49:0/4248199586 >> v2:172.21.15.150:3300/0 conn(0x7ff4b415ab50 0x7ff4b40c9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:01.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:01.758+0000 7ff4b91a0640 1 -- 172.21.15.49:0/4248199586 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4b40cb5b0 con 0x7ff4b40b05c0 2023-11-06T02:09:01.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:01.758+0000 7ff4b899f640 1 --2- 172.21.15.49:0/4248199586 >> v2:172.21.15.150:3300/0 conn(0x7ff4b415ab50 0x7ff4b40c9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:09:01.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:01.762+0000 7ff4b91a0640 1 --2- 172.21.15.49:0/4248199586 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4b40b05c0 0x7ff4b40700a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff4b40725a0 tx=0x7ff4a401c210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efa589cefb64566 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:01.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:01.762+0000 7ff4b37fe640 1 -- 172.21.15.49:0/4248199586 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a40ab600 con 0x7ff4b40b05c0 2023-11-06T02:09:01.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:01.762+0000 7ff4b37fe640 1 -- 172.21.15.49:0/4248199586 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4a40a7050 con 0x7ff4b40b05c0 2023-11-06T02:09:01.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:01.762+0000 7ff4b37fe640 1 -- 172.21.15.49:0/4248199586 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a40ab980 con 0x7ff4b40b05c0 2023-11-06T02:09:01.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:01.762+0000 7ff4bac03640 1 -- 172.21.15.49:0/4248199586 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4b40b05c0 msgr2=0x7ff4b40700a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:01.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:01.762+0000 7ff4bac03640 1 --2- 172.21.15.49:0/4248199586 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4b40b05c0 0x7ff4b40700a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff4b40725a0 tx=0x7ff4a401c210 comp rx=0 tx=0).stop 2023-11-06T02:09:01.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:01.762+0000 7ff4bac03640 1 -- 172.21.15.49:0/4248199586 shutdown_connections 2023-11-06T02:09:01.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:01.762+0000 7ff4bac03640 1 --2- 172.21.15.49:0/4248199586 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4b40cb010 0x7ff4b40b0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:01.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:01.762+0000 7ff4bac03640 1 --2- 172.21.15.49:0/4248199586 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4b40b05c0 0x7ff4b40700a0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:01.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:01.762+0000 7ff4bac03640 1 --2- 172.21.15.49:0/4248199586 >> v2:172.21.15.150:3300/0 conn(0x7ff4b415ab50 0x7ff4b40c9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:01.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:01.762+0000 7ff4bac03640 1 -- 172.21.15.49:0/4248199586 >> 172.21.15.49:0/4248199586 conn(0x7ff4b406c0a0 msgr2=0x7ff4b406c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:01.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:01.762+0000 7ff4bac03640 1 -- 172.21.15.49:0/4248199586 shutdown_connections 2023-11-06T02:09:01.762 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:01.762+0000 7ff4bac03640 1 -- 172.21.15.49:0/4248199586 wait complete. 2023-11-06T02:09:01.762 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:01.762+0000 7ff4bac03640 1 Processor -- start 2023-11-06T02:09:01.762 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:01.762+0000 7ff4bac03640 1 -- start start 2023-11-06T02:09:01.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:01.762+0000 7ff4bac03640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4b40b05c0 0x7ff4b4155db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:01.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:01.762+0000 7ff4bac03640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4b40cb010 0x7ff4b40cd870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:01.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:01.762+0000 7ff4bac03640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4b415ab50 0x7ff4b415d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:01.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:01.762+0000 7ff4bac03640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b41593a0 con 0x7ff4b40b05c0 2023-11-06T02:09:01.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:01.762+0000 7ff4bac03640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b41562f0 con 0x7ff4b40cb010 2023-11-06T02:09:01.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:01.762+0000 7ff4bac03640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b4158340 con 0x7ff4b415ab50 2023-11-06T02:09:01.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:01.762+0000 7ff4b899f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4b40b05c0 0x7ff4b4155db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:01.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:01.762+0000 7ff4b899f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4b40b05c0 0x7ff4b4155db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47570/0 (socket says 172.21.15.49:47570) 2023-11-06T02:09:01.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:01.762+0000 7ff4b899f640 1 -- 172.21.15.49:0/3423640703 learned_addr learned my addr 172.21.15.49:0/3423640703 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:01.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:01.762+0000 7ff4b91a0640 1 --2- 172.21.15.49:0/3423640703 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4b415ab50 0x7ff4b415d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:01.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:01.762+0000 7ff4b3fff640 1 --2- 172.21.15.49:0/3423640703 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4b40cb010 0x7ff4b40cd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:01.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:01.762+0000 7ff4b899f640 1 -- 172.21.15.49:0/3423640703 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4b415ab50 msgr2=0x7ff4b415d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:01.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:01.762+0000 7ff4b899f640 1 --2- 172.21.15.49:0/3423640703 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4b415ab50 0x7ff4b415d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:01.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:01.762+0000 7ff4b899f640 1 -- 172.21.15.49:0/3423640703 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4b40cb010 msgr2=0x7ff4b40cd870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:01.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:01.762+0000 7ff4b899f640 1 --2- 172.21.15.49:0/3423640703 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4b40cb010 0x7ff4b40cd870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:01.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:01.762+0000 7ff4b899f640 1 -- 172.21.15.49:0/3423640703 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4a40094d0 con 0x7ff4b40b05c0 2023-11-06T02:09:01.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:01.762+0000 7ff4b91a0640 1 --2- 172.21.15.49:0/3423640703 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4b415ab50 0x7ff4b415d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:09:01.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:01.762+0000 7ff4b3fff640 1 --2- 172.21.15.49:0/3423640703 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4b40cb010 0x7ff4b40cd870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:09:01.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:01.766+0000 7ff4b899f640 1 --2- 172.21.15.49:0/3423640703 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4b40b05c0 0x7ff4b4155db0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff49c00c7f0 tx=0x7ff49c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:01.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:01.766+0000 7ff4b17fa640 1 -- 172.21.15.49:0/3423640703 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49c005ba0 con 0x7ff4b40b05c0 2023-11-06T02:09:01.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:01.766+0000 7ff4b17fa640 1 -- 172.21.15.49:0/3423640703 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff49c005d00 con 0x7ff4b40b05c0 2023-11-06T02:09:01.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:01.766+0000 7ff4b17fa640 1 -- 172.21.15.49:0/3423640703 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49c006250 con 0x7ff4b40b05c0 2023-11-06T02:09:01.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:01.766+0000 7ff4bac03640 1 -- 172.21.15.49:0/3423640703 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4b41585e0 con 0x7ff4b40b05c0 2023-11-06T02:09:01.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:01.766+0000 7ff4bac03640 1 -- 172.21.15.49:0/3423640703 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4b4158ba0 con 0x7ff4b40b05c0 2023-11-06T02:09:01.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:01.766+0000 7ff4bac03640 1 -- 172.21.15.49:0/3423640703 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4b40b0de0 con 0x7ff4b40b05c0 2023-11-06T02:09:01.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:01.766+0000 7ff4b17fa640 1 -- 172.21.15.49:0/3423640703 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7ff49c0065c0 con 0x7ff4b40b05c0 2023-11-06T02:09:01.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:01.766+0000 7ff4b17fa640 1 --2- 172.21.15.49:0/3423640703 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff494065a60 0x7ff494067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:01.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:01.766+0000 7ff4b17fa640 1 -- 172.21.15.49:0/3423640703 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff49c083380 con 0x7ff4b40b05c0 2023-11-06T02:09:01.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:01.766+0000 7ff4b3fff640 1 --2- 172.21.15.49:0/3423640703 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff494065a60 0x7ff494067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:01.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:01.766+0000 7ff4b3fff640 1 --2- 172.21.15.49:0/3423640703 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff494065a60 0x7ff494067f20 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff4b40c8050 tx=0x7ff4ac00b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:01.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:01.770+0000 7ff4b17fa640 1 -- 172.21.15.49:0/3423640703 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7ff49c050c70 con 0x7ff4b40b05c0 2023-11-06T02:09:01.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:01.926+0000 7ff4bac03640 1 -- 172.21.15.49:0/3423640703 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff4b4074040 con 0x7ff4b40b05c0 2023-11-06T02:09:01.926 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:01.930+0000 7ff4b17fa640 1 -- 172.21.15.49:0/3423640703 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+17811 (secure 0 0 0) 0x7ff49c006870 con 0x7ff4b40b05c0 2023-11-06T02:09:01.926 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:09:01.926 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":23,"fsid":"1a8b035c-6ba4-499a-b876-aea47fd9c0d4","created":"2023-11-06T02:08:06.440857+0000","modified":"2023-11-06T02:08:31.594964+0000","last_up_change":"2023-11-06T02:08:17.982372+0000","last_in_change":"2023-11-06T02:08:08.906631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T02:08:19.252222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T02:08:24.776211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-06T02:08:28.936178+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-06T02:08:29.563468+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ad55af8-0571-4e6a-8722-b5d429b7496d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6809","nonce":38116},{"type":"v1","addr":"172.21.15.49:6811","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6813","nonce":38116},{"type":"v1","addr":"172.21.15.49:6815","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6821","nonce":38116},{"type":"v1","addr":"172.21.15.49:6823","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6817","nonce":38116},{"type":"v1","addr":"172.21.15.49:6819","nonce":38116}]},"public_addr":"172.21.15.49:6811/38116","cluster_addr":"172.21.15.49:6815/38116","heartbeat_back_addr":"172.21.15.49:6823/38116","heartbeat_front_addr":"172.21.15.49:6819/38116","state":["exists","up"]},{"osd":1,"uuid":"c8b31f85-a757-43ed-9419-351acc46611b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":38113},{"type":"v1","addr":"172.21.15.49:6810","nonce":38113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":38113},{"type":"v1","addr":"172.21.15.49:6814","nonce":38113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":38113},{"type":"v1","addr":"172.21.15.49:6822","nonce":38113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":38113},{"type":"v1","addr":"172.21.15.49:6818","nonce":38113}]},"public_addr":"172.21.15.49:6810/38113","cluster_addr":"172.21.15.49:6814/38113","heartbeat_back_addr":"172.21.15.49:6822/38113","heartbeat_front_addr":"172.21.15.49:6818/38113","state":["exists","up"]},{"osd":2,"uuid":"555b3bf3-0a70-4b91-8efd-e4204b5fc26e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":38114},{"type":"v1","addr":"172.21.15.49:6801","nonce":38114}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":38114},{"type":"v1","addr":"172.21.15.49:6803","nonce":38114}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":38114},{"type":"v1","addr":"172.21.15.49:6807","nonce":38114}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":38114},{"type":"v1","addr":"172.21.15.49:6805","nonce":38114}]},"public_addr":"172.21.15.49:6801/38114","cluster_addr":"172.21.15.49:6803/38114","heartbeat_back_addr":"172.21.15.49:6807/38114","heartbeat_front_addr":"172.21.15.49:6805/38114","state":["exists","up"]},{"osd":3,"uuid":"8d4e5d77-03ef-4b9b-bfcb-5ae93a46795c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":38115},{"type":"v1","addr":"172.21.15.49:6825","nonce":38115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":38115},{"type":"v1","addr":"172.21.15.49:6827","nonce":38115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":38115},{"type":"v1","addr":"172.21.15.49:6831","nonce":38115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":38115},{"type":"v1","addr":"172.21.15.49:6829","nonce":38115}]},"public_addr":"172.21.15.49:6825/38115","cluster_addr":"172.21.15.49:6827/38115","heartbeat_back_addr":"172.21.15.49:6831/38115","heartbeat_front_addr":"172.21.15.49:6829/38115","state":["exists","up"]},{"osd":4,"uuid":"dba03ad4-68a3-4cf3-9218-ac611f02eacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":38417},{"type":"v1","addr":"172.21.15.150:6809","nonce":38417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":38417},{"type":"v1","addr":"172.21.15.150:6811","nonce":38417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":38417},{"type":"v1","addr":"172.21.15.150:6815","nonce":38417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":38417},{"type":"v1","addr":"172.21.15.150:6813","nonce":38417}]},"public_addr":"172.21.15.150:6809/38417","cluster_addr":"172.21.15.150:6811/38417","heartbeat_back_addr":"172.21.15.150:6815/38417","heartbeat_front_addr":"172.21.15.150:6813/38417","state":["exists","up"]},{"osd":5,"uuid":"0df9c303-c255-4640-8e91-01f1c3efe18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":38418},{"type":"v1","addr":"172.21.15.150:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":38418},{"type":"v1","addr":"172.21.15.150:6803","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":38418},{"type":"v1","addr":"172.21.15.150:6807","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":38418},{"type":"v1","addr":"172.21.15.150:6805","nonce":38418}]},"public_addr":"172.21.15.150:6801/38418","cluster_addr":"172.21.15.150:6803/38418","heartbeat_back_addr":"172.21.15.150:6807/38418","heartbeat_front_addr":"172.21.15.150:6805/38418","state":["exists","up"]},{"osd":6,"uuid":"e59fa189-4b70-4c06-9722-c02b90828bf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":38428},{"type":"v1","addr":"172.21.15.150:6825","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":38428},{"type":"v1","addr":"172.21.15.150:6827","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":38428},{"type":"v1","addr":"172.21.15.150:6831","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":38428},{"type":"v1","addr":"172.21.15.150:6829","nonce":38428}]},"public_addr":"172.21.15.150:6825/38428","cluster_addr":"172.21.15.150:6827/38428","heartbeat_back_addr":"172.21.15.150:6831/38428","heartbeat_front_addr":"172.21.15.150:6829/38428","state":["exists","up"]},{"osd":7,"uuid":"a64f57e1-ef98-46bd-b304-385759e95898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":38421},{"type":"v1","addr":"172.21.15.150:6817","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":38421},{"type":"v1","addr":"172.21.15.150:6819","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":38421},{"type":"v1","addr":"172.21.15.150:6823","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":38421},{"type":"v1","addr":"172.21.15.150:6821","nonce":38421}]},"public_addr":"172.21.15.150:6817/38421","cluster_addr":"172.21.15.150:6819/38421","heartbeat_back_addr":"172.21.15.150:6823/38421","heartbeat_front_addr":"172.21.15.150:6821/38421","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.438552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.413280+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.296466+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.424966+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.276953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.230984+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.410235+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.480084+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T02:09:01.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:01.930+0000 7ff4bac03640 1 -- 172.21.15.49:0/3423640703 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff494065a60 msgr2=0x7ff494067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:01.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:01.930+0000 7ff4bac03640 1 --2- 172.21.15.49:0/3423640703 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff494065a60 0x7ff494067f20 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff4b40c8050 tx=0x7ff4ac00b040 comp rx=0 tx=0).stop 2023-11-06T02:09:01.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:01.930+0000 7ff4bac03640 1 -- 172.21.15.49:0/3423640703 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4b40b05c0 msgr2=0x7ff4b4155db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:01.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:01.930+0000 7ff4bac03640 1 --2- 172.21.15.49:0/3423640703 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4b40b05c0 0x7ff4b4155db0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff49c00c7f0 tx=0x7ff49c00ccb0 comp rx=0 tx=0).stop 2023-11-06T02:09:01.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:01.930+0000 7ff4bac03640 1 -- 172.21.15.49:0/3423640703 shutdown_connections 2023-11-06T02:09:01.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:01.930+0000 7ff4bac03640 1 --2- 172.21.15.49:0/3423640703 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff494065a60 0x7ff494067f20 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:01.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:01.930+0000 7ff4bac03640 1 --2- 172.21.15.49:0/3423640703 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4b415ab50 0x7ff4b415d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:01.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:01.930+0000 7ff4bac03640 1 --2- 172.21.15.49:0/3423640703 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4b40cb010 0x7ff4b40cd870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:01.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:01.930+0000 7ff4bac03640 1 --2- 172.21.15.49:0/3423640703 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4b40b05c0 0x7ff4b4155db0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:01.930 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:01.930+0000 7ff4bac03640 1 -- 172.21.15.49:0/3423640703 >> 172.21.15.49:0/3423640703 conn(0x7ff4b406c0a0 msgr2=0x7ff4b40ad460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:01.930 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:01.930+0000 7ff4bac03640 1 -- 172.21.15.49:0/3423640703 shutdown_connections 2023-11-06T02:09:01.930 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:01.930+0000 7ff4bac03640 1 -- 172.21.15.49:0/3423640703 wait complete. 2023-11-06T02:09:01.946 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T02:08:19.252222+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T02:08:24.776211+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T02:08:28.936178+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T02:08:29.563468+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T02:09:01.946 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T02:09:02.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.081+0000 7fa45447a640 1 Processor -- start 2023-11-06T02:09:02.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.081+0000 7fa45447a640 1 -- start start 2023-11-06T02:09:02.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.081+0000 7fa45447a640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa44c15aac0 0x7fa44c069d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:02.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.081+0000 7fa45447a640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fa44c0ca580 0x7fa44c0c3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:02.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.081+0000 7fa45447a640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa44c0cc1c0 con 0x7fa44c06a2b0 2023-11-06T02:09:02.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.081+0000 7fa45447a640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa44c0caba0 con 0x7fa44c15aac0 2023-11-06T02:09:02.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.081+0000 7fa45447a640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa44c06a710 con 0x7fa44c0ca580 2023-11-06T02:09:02.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.081+0000 7fa452216640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa44c15aac0 0x7fa44c069d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:02.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.081+0000 7fa451a15640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fa44c06a2b0 0x7fa44c0c9e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:48364/0 (socket says 172.21.15.49:48364) 2023-11-06T02:09:02.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.081+0000 7fa451a15640 1 -- 172.21.15.49:0/3369596265 learned_addr learned my addr 172.21.15.49:0/3369596265 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:09:02.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.081+0000 7fa452a17640 1 --2- 172.21.15.49:0/3369596265 >> v2:172.21.15.150:3300/0 conn(0x7fa44c0ca580 0x7fa44c0c3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:02.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.081+0000 7fa452216640 1 -- 172.21.15.49:0/3369596265 >> v2:172.21.15.150:3300/0 conn(0x7fa44c0ca580 msgr2=0x7fa44c0c3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:02.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.081+0000 7fa452216640 1 --2- 172.21.15.49:0/3369596265 >> v2:172.21.15.150:3300/0 conn(0x7fa44c0ca580 0x7fa44c0c3300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:02.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.081+0000 7fa452216640 1 -- 172.21.15.49:0/3369596265 >> v1:172.21.15.49:6789/0 conn(0x7fa44c06a2b0 legacy=0x7fa44c0c9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:02.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.081+0000 7fa452216640 1 -- 172.21.15.49:0/3369596265 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa44c0cc410 con 0x7fa44c15aac0 2023-11-06T02:09:02.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.081+0000 7fa451214640 1 -- 172.21.15.49:0/3369596265 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2088490208 0 0) 0x7fa44c0cc1c0 con 0x7fa44c06a2b0 2023-11-06T02:09:02.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.081+0000 7fa452a17640 1 --2- 172.21.15.49:0/3369596265 >> v2:172.21.15.150:3300/0 conn(0x7fa44c0ca580 0x7fa44c0c3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:09:02.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.085+0000 7fa452216640 1 --2- 172.21.15.49:0/3369596265 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa44c15aac0 0x7fa44c069d70 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa44c0b22e0 tx=0x7fa44801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=622f11cbf7b835ed server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:02.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.085+0000 7fa451214640 1 -- 172.21.15.49:0/3369596265 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa448004230 con 0x7fa44c15aac0 2023-11-06T02:09:02.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.085+0000 7fa451214640 1 -- 172.21.15.49:0/3369596265 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4480a7050 con 0x7fa44c15aac0 2023-11-06T02:09:02.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.085+0000 7fa451214640 1 -- 172.21.15.49:0/3369596265 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa448003620 con 0x7fa44c15aac0 2023-11-06T02:09:02.085 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.085+0000 7fa45447a640 1 -- 172.21.15.49:0/3369596265 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa44c15aac0 msgr2=0x7fa44c069d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:02.085 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.085+0000 7fa45447a640 1 --2- 172.21.15.49:0/3369596265 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa44c15aac0 0x7fa44c069d70 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa44c0b22e0 tx=0x7fa44801c460 comp rx=0 tx=0).stop 2023-11-06T02:09:02.085 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.085+0000 7fa45447a640 1 -- 172.21.15.49:0/3369596265 shutdown_connections 2023-11-06T02:09:02.085 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.085+0000 7fa45447a640 1 --2- 172.21.15.49:0/3369596265 >> v2:172.21.15.150:3300/0 conn(0x7fa44c0ca580 0x7fa44c0c3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:02.085 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.085+0000 7fa45447a640 1 --2- 172.21.15.49:0/3369596265 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa44c15aac0 0x7fa44c069d70 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:02.085 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.085+0000 7fa45447a640 1 -- 172.21.15.49:0/3369596265 >> 172.21.15.49:0/3369596265 conn(0x7fa44c06ae80 msgr2=0x7fa44c06b290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:02.086 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.085+0000 7fa45447a640 1 -- 172.21.15.49:0/3369596265 shutdown_connections 2023-11-06T02:09:02.086 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.085+0000 7fa45447a640 1 -- 172.21.15.49:0/3369596265 wait complete. 2023-11-06T02:09:02.086 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.085+0000 7fa45447a640 1 Processor -- start 2023-11-06T02:09:02.086 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.085+0000 7fa45447a640 1 -- start start 2023-11-06T02:09:02.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.085+0000 7fa45447a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa44c06a2b0 0x7fa44c0c92b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:02.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.085+0000 7fa45447a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa44c0ca580 0x7fa44c0a8f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:02.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.085+0000 7fa45447a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa44c15aac0 0x7fa44c0c4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:02.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.085+0000 7fa45447a640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa44c0c8740 con 0x7fa44c06a2b0 2023-11-06T02:09:02.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.085+0000 7fa45447a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa44c0a81a0 con 0x7fa44c0ca580 2023-11-06T02:09:02.090 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.085+0000 7fa45447a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa44c0a8480 con 0x7fa44c15aac0 2023-11-06T02:09:02.090 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.085+0000 7fa452216640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa44c06a2b0 0x7fa44c0c92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:02.090 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.085+0000 7fa452216640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa44c06a2b0 0x7fa44c0c92b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47584/0 (socket says 172.21.15.49:47584) 2023-11-06T02:09:02.090 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.085+0000 7fa452216640 1 -- 172.21.15.49:0/1222010559 learned_addr learned my addr 172.21.15.49:0/1222010559 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:02.090 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.089+0000 7fa451a15640 1 --2- 172.21.15.49:0/1222010559 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa44c0ca580 0x7fa44c0a8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:02.091 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.089+0000 7fa452216640 1 -- 172.21.15.49:0/1222010559 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa44c15aac0 msgr2=0x7fa44c0c4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:02.091 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.089+0000 7fa452216640 1 --2- 172.21.15.49:0/1222010559 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa44c15aac0 0x7fa44c0c4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:02.091 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.089+0000 7fa452216640 1 -- 172.21.15.49:0/1222010559 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa44c0ca580 msgr2=0x7fa44c0a8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:02.091 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.089+0000 7fa452216640 1 --2- 172.21.15.49:0/1222010559 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa44c0ca580 0x7fa44c0a8f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:02.091 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.089+0000 7fa452216640 1 -- 172.21.15.49:0/1222010559 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa448009420 con 0x7fa44c06a2b0 2023-11-06T02:09:02.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.089+0000 7fa452216640 1 --2- 172.21.15.49:0/1222010559 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa44c06a2b0 0x7fa44c0c92b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa448019070 tx=0x7fa448019200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:02.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.089+0000 7fa442ffd640 1 -- 172.21.15.49:0/1222010559 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa448003c70 con 0x7fa44c06a2b0 2023-11-06T02:09:02.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.089+0000 7fa442ffd640 1 -- 172.21.15.49:0/1222010559 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa448003dd0 con 0x7fa44c06a2b0 2023-11-06T02:09:02.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.089+0000 7fa442ffd640 1 -- 172.21.15.49:0/1222010559 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4480ab660 con 0x7fa44c06a2b0 2023-11-06T02:09:02.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.089+0000 7fa45447a640 1 -- 172.21.15.49:0/1222010559 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa44c0a8760 con 0x7fa44c06a2b0 2023-11-06T02:09:02.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.089+0000 7fa45447a640 1 -- 172.21.15.49:0/1222010559 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa44c0a7e90 con 0x7fa44c06a2b0 2023-11-06T02:09:02.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.089+0000 7fa45447a640 1 -- 172.21.15.49:0/1222010559 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa44c074040 con 0x7fa44c06a2b0 2023-11-06T02:09:02.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.089+0000 7fa442ffd640 1 -- 172.21.15.49:0/1222010559 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fa4480b2040 con 0x7fa44c06a2b0 2023-11-06T02:09:02.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.089+0000 7fa442ffd640 1 --2- 172.21.15.49:0/1222010559 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa430065a60 0x7fa430067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:02.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.089+0000 7fa442ffd640 1 -- 172.21.15.49:0/1222010559 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa44811d2d0 con 0x7fa44c06a2b0 2023-11-06T02:09:02.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.089+0000 7fa451a15640 1 --2- 172.21.15.49:0/1222010559 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa430065a60 0x7fa430067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:02.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.089+0000 7fa451a15640 1 --2- 172.21.15.49:0/1222010559 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa430065a60 0x7fa430067f20 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa44c0c4d00 tx=0x7fa43c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:02.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.093+0000 7fa442ffd640 1 -- 172.21.15.49:0/1222010559 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fa4480eab30 con 0x7fa44c06a2b0 2023-11-06T02:09:02.249 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.253+0000 7fa45447a640 1 -- 172.21.15.49:0/1222010559 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa44c06ec10 con 0x7fa44c06a2b0 2023-11-06T02:09:02.249 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.253+0000 7fa442ffd640 1 -- 172.21.15.49:0/1222010559 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa4480ee150 con 0x7fa44c06a2b0 2023-11-06T02:09:02.250 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2023-11-06T02:09:02.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.253+0000 7fa45447a640 1 -- 172.21.15.49:0/1222010559 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa430065a60 msgr2=0x7fa430067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:02.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.253+0000 7fa45447a640 1 --2- 172.21.15.49:0/1222010559 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa430065a60 0x7fa430067f20 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa44c0c4d00 tx=0x7fa43c009040 comp rx=0 tx=0).stop 2023-11-06T02:09:02.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.253+0000 7fa45447a640 1 -- 172.21.15.49:0/1222010559 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa44c06a2b0 msgr2=0x7fa44c0c92b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:02.253 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.253+0000 7fa45447a640 1 --2- 172.21.15.49:0/1222010559 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa44c06a2b0 0x7fa44c0c92b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa448019070 tx=0x7fa448019200 comp rx=0 tx=0).stop 2023-11-06T02:09:02.253 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.253+0000 7fa45447a640 1 -- 172.21.15.49:0/1222010559 shutdown_connections 2023-11-06T02:09:02.253 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.253+0000 7fa45447a640 1 --2- 172.21.15.49:0/1222010559 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa430065a60 0x7fa430067f20 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:02.253 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.253+0000 7fa45447a640 1 --2- 172.21.15.49:0/1222010559 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa44c15aac0 0x7fa44c0c4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:02.253 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.253+0000 7fa45447a640 1 --2- 172.21.15.49:0/1222010559 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa44c0ca580 0x7fa44c0a8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:02.254 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.253+0000 7fa45447a640 1 --2- 172.21.15.49:0/1222010559 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa44c06a2b0 0x7fa44c0c92b0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:02.254 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.253+0000 7fa45447a640 1 -- 172.21.15.49:0/1222010559 >> 172.21.15.49:0/1222010559 conn(0x7fa44c06ae80 msgr2=0x7fa44c0b0080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:02.254 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.253+0000 7fa45447a640 1 -- 172.21.15.49:0/1222010559 shutdown_connections 2023-11-06T02:09:02.254 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.253+0000 7fa45447a640 1 -- 172.21.15.49:0/1222010559 wait complete. 2023-11-06T02:09:02.269 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T02:09:02.389 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.393+0000 7f1cc6672640 1 Processor -- start 2023-11-06T02:09:02.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.393+0000 7f1cc6672640 1 -- start start 2023-11-06T02:09:02.391 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.393+0000 7f1cc6672640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f1cc00ca0f0 0x7f1cc006e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:02.391 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.393+0000 7f1cc6672640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cc00cabf0 con 0x7f1cc00c9600 2023-11-06T02:09:02.391 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.393+0000 7f1cc6672640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cc00ca8c0 con 0x7f1cc0065930 2023-11-06T02:09:02.391 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.393+0000 7f1cc6672640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cc006def0 con 0x7f1cc00ca0f0 2023-11-06T02:09:02.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.393+0000 7f1cbffff640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f1cc00c9600 0x7f1cc00cb080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:48378/0 (socket says 172.21.15.49:48378) 2023-11-06T02:09:02.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.393+0000 7f1cbffff640 1 -- 172.21.15.49:0/3096282641 learned_addr learned my addr 172.21.15.49:0/3096282641 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:09:02.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.393+0000 7f1cc4c0f640 1 --2- 172.21.15.49:0/3096282641 >> v2:172.21.15.150:3300/0 conn(0x7f1cc00ca0f0 0x7f1cc006e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:02.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.393+0000 7f1cbeffd640 1 -- 172.21.15.49:0/3096282641 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2881405664 0 0) 0x7f1cc00cabf0 con 0x7f1cc00c9600 2023-11-06T02:09:02.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.393+0000 7f1cbeffd640 1 -- 172.21.15.49:0/3096282641 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ca8003530 con 0x7f1cc00c9600 2023-11-06T02:09:02.395 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.393+0000 7f1cbeffd640 1 -- 172.21.15.49:0/3096282641 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2641225698 0 0) 0x7f1ca8003530 con 0x7f1cc00c9600 2023-11-06T02:09:02.395 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.393+0000 7f1cbeffd640 1 -- 172.21.15.49:0/3096282641 >> v2:172.21.15.150:3300/0 conn(0x7f1cc00ca0f0 msgr2=0x7f1cc006e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:02.395 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.393+0000 7f1cbeffd640 1 --2- 172.21.15.49:0/3096282641 >> v2:172.21.15.150:3300/0 conn(0x7f1cc00ca0f0 0x7f1cc006e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:02.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.393+0000 7f1cbeffd640 1 -- 172.21.15.49:0/3096282641 >> v1:172.21.15.150:6789/0 conn(0x7f1cc0065930 legacy=0x7f1cc00c99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:02.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.393+0000 7f1cbeffd640 1 -- 172.21.15.49:0/3096282641 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cc009fea0 con 0x7f1cc00c9600 2023-11-06T02:09:02.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.393+0000 7f1cbeffd640 1 -- 172.21.15.49:0/3096282641 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7f1cb0001680 con 0x7f1cc00c9600 2023-11-06T02:09:02.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.393+0000 7f1cbeffd640 1 -- 172.21.15.49:0/3096282641 >> v1:172.21.15.49:6789/0 conn(0x7f1cc00c9600 legacy=0x7f1cc00cb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:02.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.393+0000 7f1cbeffd640 1 --2- 172.21.15.49:0/3096282641 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1ca8003980 0x7f1ca8003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:02.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.393+0000 7f1cbeffd640 1 --2- 172.21.15.49:0/3096282641 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ca80047d0 0x7f1ca8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:02.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.393+0000 7f1cbeffd640 1 --2- 172.21.15.49:0/3096282641 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ca8005190 0x7f1ca800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:02.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.393+0000 7f1cbeffd640 1 -- 172.21.15.49:0/3096282641 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca800c000 con 0x7f1ca80047d0 2023-11-06T02:09:02.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.393+0000 7f1cbeffd640 1 -- 172.21.15.49:0/3096282641 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca800c170 con 0x7f1ca8005190 2023-11-06T02:09:02.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.393+0000 7f1cbeffd640 1 -- 172.21.15.49:0/3096282641 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca800c330 con 0x7f1ca8003980 2023-11-06T02:09:02.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.393+0000 7f1cbf7fe640 1 --2- 172.21.15.49:0/3096282641 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ca80047d0 0x7f1ca8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:02.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.393+0000 7f1cbffff640 1 --2- 172.21.15.49:0/3096282641 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1ca8003980 0x7f1ca8003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:02.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.393+0000 7f1cc4c0f640 1 --2- 172.21.15.49:0/3096282641 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ca8005190 0x7f1ca800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:02.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.393+0000 7f1cbf7fe640 1 -- 172.21.15.49:0/3096282641 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1ca8003980 msgr2=0x7f1ca8003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:02.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.393+0000 7f1cbf7fe640 1 --2- 172.21.15.49:0/3096282641 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1ca8003980 0x7f1ca8003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:02.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.393+0000 7f1cbf7fe640 1 -- 172.21.15.49:0/3096282641 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ca8005190 msgr2=0x7f1ca800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:02.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.393+0000 7f1cbf7fe640 1 --2- 172.21.15.49:0/3096282641 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ca8005190 0x7f1ca800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:02.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.393+0000 7f1cbf7fe640 1 -- 172.21.15.49:0/3096282641 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ca8003530 con 0x7f1ca80047d0 2023-11-06T02:09:02.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.393+0000 7f1cc4c0f640 1 -- 172.21.15.49:0/3096282641 reap_dead start 2023-11-06T02:09:02.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.397+0000 7f1cbf7fe640 1 --2- 172.21.15.49:0/3096282641 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ca80047d0 0x7f1ca8004c50 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1cb4001780 tx=0x7f1cb401c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0a1696bc79dd1a7 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:02.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.397+0000 7f1cbeffd640 1 -- 172.21.15.49:0/3096282641 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cb40a4ea0 con 0x7f1ca80047d0 2023-11-06T02:09:02.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.397+0000 7f1cbeffd640 1 -- 172.21.15.49:0/3096282641 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cb40a7050 con 0x7f1ca80047d0 2023-11-06T02:09:02.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.397+0000 7f1cbeffd640 1 -- 172.21.15.49:0/3096282641 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cb4003600 con 0x7f1ca80047d0 2023-11-06T02:09:02.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.397+0000 7f1cc6672640 1 -- 172.21.15.49:0/3096282641 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ca80047d0 msgr2=0x7f1ca8004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:02.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.397+0000 7f1cc6672640 1 --2- 172.21.15.49:0/3096282641 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ca80047d0 0x7f1ca8004c50 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1cb4001780 tx=0x7f1cb401c3e0 comp rx=0 tx=0).stop 2023-11-06T02:09:02.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.397+0000 7f1cc6672640 1 -- 172.21.15.49:0/3096282641 shutdown_connections 2023-11-06T02:09:02.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.397+0000 7f1cc6672640 1 --2- 172.21.15.49:0/3096282641 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ca80047d0 0x7f1ca8004c50 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:02.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.397+0000 7f1cc6672640 1 -- 172.21.15.49:0/3096282641 >> 172.21.15.49:0/3096282641 conn(0x7f1cc00b0aa0 msgr2=0x7f1cc00b0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:02.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.397+0000 7f1cc6672640 1 -- 172.21.15.49:0/3096282641 shutdown_connections 2023-11-06T02:09:02.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.397+0000 7f1cc6672640 1 -- 172.21.15.49:0/3096282641 wait complete. 2023-11-06T02:09:02.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.397+0000 7f1cc6672640 1 Processor -- start 2023-11-06T02:09:02.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.397+0000 7f1cc6672640 1 -- start start 2023-11-06T02:09:02.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.401+0000 7f1cc6672640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ca80047d0 0x7f1cc00c6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:02.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.401+0000 7f1cc6672640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1cc015a8e0 0x7f1cc00c8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:02.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.401+0000 7f1cc6672640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1cc00afa00 0x7f1cc00afeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:02.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.401+0000 7f1cc6672640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc00b03f0 con 0x7f1cc00afa00 2023-11-06T02:09:02.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.401+0000 7f1cc6672640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc00c69f0 con 0x7f1ca80047d0 2023-11-06T02:09:02.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.401+0000 7f1cc6672640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc00c6cd0 con 0x7f1cc015a8e0 2023-11-06T02:09:02.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.401+0000 7f1cc4c0f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1cc00afa00 0x7f1cc00afeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:02.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.401+0000 7f1cc4c0f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1cc00afa00 0x7f1cc00afeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47604/0 (socket says 172.21.15.49:47604) 2023-11-06T02:09:02.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.401+0000 7f1cc4c0f640 1 -- 172.21.15.49:0/2217031440 learned_addr learned my addr 172.21.15.49:0/2217031440 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:02.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.401+0000 7f1cbf7fe640 1 --2- 172.21.15.49:0/2217031440 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1cc015a8e0 0x7f1cc00c8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:02.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.401+0000 7f1cc4c0f640 1 -- 172.21.15.49:0/2217031440 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1cc015a8e0 msgr2=0x7f1cc00c8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:02.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.401+0000 7f1cc4c0f640 1 --2- 172.21.15.49:0/2217031440 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1cc015a8e0 0x7f1cc00c8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:02.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.401+0000 7f1cc4c0f640 1 -- 172.21.15.49:0/2217031440 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ca80047d0 msgr2=0x7f1cc00c6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:02.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.401+0000 7f1cc4c0f640 1 --2- 172.21.15.49:0/2217031440 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ca80047d0 0x7f1cc00c6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:02.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.401+0000 7f1cc4c0f640 1 -- 172.21.15.49:0/2217031440 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cc00c6fb0 con 0x7f1cc00afa00 2023-11-06T02:09:02.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.401+0000 7f1cc4c0f640 1 --2- 172.21.15.49:0/2217031440 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1cc00afa00 0x7f1cc00afeb0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1cc0070690 tx=0x7f1cb80115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:02.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.401+0000 7f1cbcff9640 1 -- 172.21.15.49:0/2217031440 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cb800e070 con 0x7f1cc00afa00 2023-11-06T02:09:02.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.401+0000 7f1cbcff9640 1 -- 172.21.15.49:0/2217031440 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cb8011e10 con 0x7f1cc00afa00 2023-11-06T02:09:02.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.401+0000 7f1cc6672640 1 -- 172.21.15.49:0/2217031440 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cc00c7290 con 0x7f1cc00afa00 2023-11-06T02:09:02.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.401+0000 7f1cbcff9640 1 -- 172.21.15.49:0/2217031440 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cb80039a0 con 0x7f1cc00afa00 2023-11-06T02:09:02.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.401+0000 7f1cc6672640 1 -- 172.21.15.49:0/2217031440 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cc00c7a70 con 0x7f1cc00afa00 2023-11-06T02:09:02.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.401+0000 7f1cc6672640 1 -- 172.21.15.49:0/2217031440 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cc00b17e0 con 0x7f1cc00afa00 2023-11-06T02:09:02.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.401+0000 7f1cbcff9640 1 -- 172.21.15.49:0/2217031440 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f1cb8003b00 con 0x7f1cc00afa00 2023-11-06T02:09:02.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.401+0000 7f1cbcff9640 1 --2- 172.21.15.49:0/2217031440 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1cac065a60 0x7f1cac067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:02.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.401+0000 7f1cbcff9640 1 -- 172.21.15.49:0/2217031440 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1cb8083b80 con 0x7f1cc00afa00 2023-11-06T02:09:02.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.401+0000 7f1cbffff640 1 --2- 172.21.15.49:0/2217031440 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1cac065a60 0x7f1cac067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:02.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.405+0000 7f1cbffff640 1 --2- 172.21.15.49:0/2217031440 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1cac065a60 0x7f1cac067f20 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1cb0004880 tx=0x7f1cb00011a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:02.411 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.405+0000 7f1cbcff9640 1 -- 172.21.15.49:0/2217031440 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f1cb80513e0 con 0x7f1cc00afa00 2023-11-06T02:09:02.563 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.565+0000 7f1cc6672640 1 -- 172.21.15.49:0/2217031440 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f1cc015be10 con 0x7f1cc00afa00 2023-11-06T02:09:02.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.565+0000 7f1cbcff9640 1 -- 172.21.15.49:0/2217031440 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1cb8054a00 con 0x7f1cc00afa00 2023-11-06T02:09:02.564 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 8 2023-11-06T02:09:02.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.569+0000 7f1cc6672640 1 -- 172.21.15.49:0/2217031440 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1cac065a60 msgr2=0x7f1cac067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:02.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.569+0000 7f1cc6672640 1 --2- 172.21.15.49:0/2217031440 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1cac065a60 0x7f1cac067f20 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1cb0004880 tx=0x7f1cb00011a0 comp rx=0 tx=0).stop 2023-11-06T02:09:02.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.569+0000 7f1cc6672640 1 -- 172.21.15.49:0/2217031440 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1cc00afa00 msgr2=0x7f1cc00afeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:02.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.569+0000 7f1cc6672640 1 --2- 172.21.15.49:0/2217031440 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1cc00afa00 0x7f1cc00afeb0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1cc0070690 tx=0x7f1cb80115a0 comp rx=0 tx=0).stop 2023-11-06T02:09:02.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.569+0000 7f1cc6672640 1 -- 172.21.15.49:0/2217031440 shutdown_connections 2023-11-06T02:09:02.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.569+0000 7f1cc6672640 1 --2- 172.21.15.49:0/2217031440 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1cac065a60 0x7f1cac067f20 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:02.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.569+0000 7f1cc6672640 1 --2- 172.21.15.49:0/2217031440 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1cc00afa00 0x7f1cc00afeb0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:02.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.569+0000 7f1cc6672640 1 --2- 172.21.15.49:0/2217031440 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1cc015a8e0 0x7f1cc00c8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:02.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.569+0000 7f1cc6672640 1 --2- 172.21.15.49:0/2217031440 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ca80047d0 0x7f1cc00c6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:02.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.569+0000 7f1cc6672640 1 -- 172.21.15.49:0/2217031440 >> 172.21.15.49:0/2217031440 conn(0x7f1cc00b0aa0 msgr2=0x7f1cc009ef00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:02.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.569+0000 7f1cc6672640 1 -- 172.21.15.49:0/2217031440 shutdown_connections 2023-11-06T02:09:02.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.569+0000 7f1cc6672640 1 -- 172.21.15.49:0/2217031440 wait complete. 2023-11-06T02:09:02.583 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T02:09:02.710 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.713+0000 7fb59a0a3640 1 Processor -- start 2023-11-06T02:09:02.710 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.713+0000 7fb59a0a3640 1 -- start start 2023-11-06T02:09:02.712 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.713+0000 7fb59a0a3640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fb59406b2f0 0x7fb5940b1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:02.712 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.713+0000 7fb59a0a3640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb5940b0810 0x7fb59415c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:02.712 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.713+0000 7fb59a0a3640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb59415cd40 0x7fb59406b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:02.712 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.713+0000 7fb59a0a3640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5940b1990 con 0x7fb59415cd40 2023-11-06T02:09:02.713 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.713+0000 7fb59a0a3640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb59415d1d0 con 0x7fb59406b2f0 2023-11-06T02:09:02.713 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.713+0000 7fb59a0a3640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5940b0d80 con 0x7fb5940b0810 2023-11-06T02:09:02.713 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.713+0000 7fb593fff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb59415cd40 0x7fb59406b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:02.713 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.713+0000 7fb593fff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb59415cd40 0x7fb59406b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47616/0 (socket says 172.21.15.49:47616) 2023-11-06T02:09:02.713 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.713+0000 7fb593fff640 1 -- 172.21.15.49:0/198176907 learned_addr learned my addr 172.21.15.49:0/198176907 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:02.714 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.713+0000 7fb5937fe640 1 --2- 172.21.15.49:0/198176907 >> v2:172.21.15.150:3300/0 conn(0x7fb59406b2f0 0x7fb5940b1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:02.714 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.713+0000 7fb592ffd640 1 --2- 172.21.15.49:0/198176907 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb5940b0810 0x7fb59415c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:02.714 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.713+0000 7fb593fff640 1 -- 172.21.15.49:0/198176907 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb5940b0810 msgr2=0x7fb59415c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:02.714 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.713+0000 7fb593fff640 1 --2- 172.21.15.49:0/198176907 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb5940b0810 0x7fb59415c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:02.714 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.713+0000 7fb593fff640 1 -- 172.21.15.49:0/198176907 >> v2:172.21.15.150:3300/0 conn(0x7fb59406b2f0 msgr2=0x7fb5940b1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:02.715 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.713+0000 7fb593fff640 1 --2- 172.21.15.49:0/198176907 >> v2:172.21.15.150:3300/0 conn(0x7fb59406b2f0 0x7fb5940b1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:02.715 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.713+0000 7fb593fff640 1 -- 172.21.15.49:0/198176907 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5940b1060 con 0x7fb59415cd40 2023-11-06T02:09:02.715 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.713+0000 7fb5937fe640 1 --2- 172.21.15.49:0/198176907 >> v2:172.21.15.150:3300/0 conn(0x7fb59406b2f0 0x7fb5940b1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:09:02.716 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.717+0000 7fb593fff640 1 --2- 172.21.15.49:0/198176907 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb59415cd40 0x7fb59406b6d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb594071a30 tx=0x7fb58c01f1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64c32d16807f3b9b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:02.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.717+0000 7fb5927fc640 1 -- 172.21.15.49:0/198176907 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb58c0af400 con 0x7fb59415cd40 2023-11-06T02:09:02.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.717+0000 7fb5927fc640 1 -- 172.21.15.49:0/198176907 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb58c0ab050 con 0x7fb59415cd40 2023-11-06T02:09:02.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.717+0000 7fb5927fc640 1 -- 172.21.15.49:0/198176907 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb58c0af780 con 0x7fb59415cd40 2023-11-06T02:09:02.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.717+0000 7fb59a0a3640 1 -- 172.21.15.49:0/198176907 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb59415cd40 msgr2=0x7fb59406b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:02.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.717+0000 7fb59a0a3640 1 --2- 172.21.15.49:0/198176907 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb59415cd40 0x7fb59406b6d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb594071a30 tx=0x7fb58c01f1c0 comp rx=0 tx=0).stop 2023-11-06T02:09:02.718 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.717+0000 7fb59a0a3640 1 -- 172.21.15.49:0/198176907 shutdown_connections 2023-11-06T02:09:02.718 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.717+0000 7fb59a0a3640 1 --2- 172.21.15.49:0/198176907 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb5940b0810 0x7fb59415c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:02.718 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.717+0000 7fb59a0a3640 1 --2- 172.21.15.49:0/198176907 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb59415cd40 0x7fb59406b6d0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:02.718 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.717+0000 7fb59a0a3640 1 --2- 172.21.15.49:0/198176907 >> v2:172.21.15.150:3300/0 conn(0x7fb59406b2f0 0x7fb5940b1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:02.718 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.717+0000 7fb59a0a3640 1 -- 172.21.15.49:0/198176907 >> 172.21.15.49:0/198176907 conn(0x7fb5940d08d0 msgr2=0x7fb5940d0cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:02.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.717+0000 7fb59a0a3640 1 -- 172.21.15.49:0/198176907 shutdown_connections 2023-11-06T02:09:02.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.717+0000 7fb59a0a3640 1 -- 172.21.15.49:0/198176907 wait complete. 2023-11-06T02:09:02.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.717+0000 7fb59a0a3640 1 Processor -- start 2023-11-06T02:09:02.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.717+0000 7fb59a0a3640 1 -- start start 2023-11-06T02:09:02.721 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.721+0000 7fb59a0a3640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb59406b2f0 0x7fb59406fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:02.721 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.721+0000 7fb59a0a3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5940b0810 0x7fb594148f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:02.722 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.721+0000 7fb59a0a3640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb59415cd40 0x7fb5940cd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:02.722 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.721+0000 7fb59a0a3640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5941494d0 con 0x7fb5940b0810 2023-11-06T02:09:02.722 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.721+0000 7fb59a0a3640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb59406f9c0 con 0x7fb59415cd40 2023-11-06T02:09:02.722 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.721+0000 7fb59a0a3640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb59406edf0 con 0x7fb59406b2f0 2023-11-06T02:09:02.722 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.721+0000 7fb592ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5940b0810 0x7fb594148f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:02.723 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.721+0000 7fb592ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5940b0810 0x7fb594148f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47622/0 (socket says 172.21.15.49:47622) 2023-11-06T02:09:02.723 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.721+0000 7fb592ffd640 1 -- 172.21.15.49:0/1001528916 learned_addr learned my addr 172.21.15.49:0/1001528916 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:02.723 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.721+0000 7fb592ffd640 1 -- 172.21.15.49:0/1001528916 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb59406b2f0 msgr2=0x7fb59406fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:02.723 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.721+0000 7fb592ffd640 1 --2- 172.21.15.49:0/1001528916 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb59406b2f0 0x7fb59406fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:02.723 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.721+0000 7fb592ffd640 1 -- 172.21.15.49:0/1001528916 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb59415cd40 msgr2=0x7fb5940cd6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:02.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.721+0000 7fb593fff640 1 --2- 172.21.15.49:0/1001528916 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb59415cd40 0x7fb5940cd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:02.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.721+0000 7fb592ffd640 1 --2- 172.21.15.49:0/1001528916 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb59415cd40 0x7fb5940cd6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:02.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.721+0000 7fb592ffd640 1 -- 172.21.15.49:0/1001528916 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb58c009b30 con 0x7fb5940b0810 2023-11-06T02:09:02.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.721+0000 7fb592ffd640 1 --2- 172.21.15.49:0/1001528916 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5940b0810 0x7fb594148f90 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb57c00d800 tx=0x7fb57c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:02.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.721+0000 7fb58bfff640 1 -- 172.21.15.49:0/1001528916 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb57c00b6c0 con 0x7fb5940b0810 2023-11-06T02:09:02.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.721+0000 7fb58bfff640 1 -- 172.21.15.49:0/1001528916 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb57c00b820 con 0x7fb5940b0810 2023-11-06T02:09:02.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.721+0000 7fb59a0a3640 1 -- 172.21.15.49:0/1001528916 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb59406f090 con 0x7fb5940b0810 2023-11-06T02:09:02.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.721+0000 7fb58bfff640 1 -- 172.21.15.49:0/1001528916 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb57c00bb90 con 0x7fb5940b0810 2023-11-06T02:09:02.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.721+0000 7fb59a0a3640 1 -- 172.21.15.49:0/1001528916 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb59406f650 con 0x7fb5940b0810 2023-11-06T02:09:02.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.721+0000 7fb59a0a3640 1 -- 172.21.15.49:0/1001528916 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb594074040 con 0x7fb5940b0810 2023-11-06T02:09:02.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.721+0000 7fb58bfff640 1 -- 172.21.15.49:0/1001528916 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fb57c00bcf0 con 0x7fb5940b0810 2023-11-06T02:09:02.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.725+0000 7fb58bfff640 1 --2- 172.21.15.49:0/1001528916 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb574065a90 0x7fb574067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:02.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.725+0000 7fb58bfff640 1 -- 172.21.15.49:0/1001528916 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb57c084120 con 0x7fb5940b0810 2023-11-06T02:09:02.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.725+0000 7fb5937fe640 1 --2- 172.21.15.49:0/1001528916 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb574065a90 0x7fb574067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:02.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.725+0000 7fb5937fe640 1 --2- 172.21.15.49:0/1001528916 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb574065a90 0x7fb574067f50 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb584006fd0 tx=0x7fb584008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:02.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.725+0000 7fb58bfff640 1 -- 172.21.15.49:0/1001528916 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fb57c051af0 con 0x7fb5940b0810 2023-11-06T02:09:02.873 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.877+0000 7fb59a0a3640 1 -- 172.21.15.49:0/1001528916 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb5940c7150 con 0x7fb5940b0810 2023-11-06T02:09:02.874 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.877+0000 7fb58bfff640 1 -- 172.21.15.49:0/1001528916 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb57c055110 con 0x7fb5940b0810 2023-11-06T02:09:02.874 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2023-11-06T02:09:02.876 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.877+0000 7fb59a0a3640 1 -- 172.21.15.49:0/1001528916 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb574065a90 msgr2=0x7fb574067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:02.876 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.877+0000 7fb59a0a3640 1 --2- 172.21.15.49:0/1001528916 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb574065a90 0x7fb574067f50 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb584006fd0 tx=0x7fb584008040 comp rx=0 tx=0).stop 2023-11-06T02:09:02.877 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.877+0000 7fb59a0a3640 1 -- 172.21.15.49:0/1001528916 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5940b0810 msgr2=0x7fb594148f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:02.877 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.877+0000 7fb59a0a3640 1 --2- 172.21.15.49:0/1001528916 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5940b0810 0x7fb594148f90 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb57c00d800 tx=0x7fb57c00dcc0 comp rx=0 tx=0).stop 2023-11-06T02:09:02.877 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.877+0000 7fb59a0a3640 1 -- 172.21.15.49:0/1001528916 shutdown_connections 2023-11-06T02:09:02.877 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.877+0000 7fb59a0a3640 1 --2- 172.21.15.49:0/1001528916 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb574065a90 0x7fb574067f50 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:02.877 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.877+0000 7fb59a0a3640 1 --2- 172.21.15.49:0/1001528916 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5940b0810 0x7fb594148f90 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:02.878 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.877+0000 7fb59a0a3640 1 --2- 172.21.15.49:0/1001528916 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb59415cd40 0x7fb5940cd6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:02.878 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.877+0000 7fb59a0a3640 1 --2- 172.21.15.49:0/1001528916 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb59406b2f0 0x7fb59406fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:02.878 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.877+0000 7fb59a0a3640 1 -- 172.21.15.49:0/1001528916 >> 172.21.15.49:0/1001528916 conn(0x7fb5940d08d0 msgr2=0x7fb59415bd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:02.878 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.877+0000 7fb59a0a3640 1 -- 172.21.15.49:0/1001528916 shutdown_connections 2023-11-06T02:09:02.878 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:02.877+0000 7fb59a0a3640 1 -- 172.21.15.49:0/1001528916 wait complete. 2023-11-06T02:09:02.892 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T02:09:03.019 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.021+0000 7f00627c3640 1 Processor -- start 2023-11-06T02:09:03.019 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.021+0000 7f00627c3640 1 -- start start 2023-11-06T02:09:03.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.021+0000 7f00627c3640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f005c15a910 0x7f005c070080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:03.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.021+0000 7f00627c3640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f005c0af9d0 0x7f005c0705c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:03.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.021+0000 7f00627c3640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f005c0b16d0 con 0x7f005c06d4b0 2023-11-06T02:09:03.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.021+0000 7f00627c3640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f005c0b19b0 con 0x7f005c15a910 2023-11-06T02:09:03.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.021+0000 7f00627c3640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f005c0b1c90 con 0x7f005c0af9d0 2023-11-06T02:09:03.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.021+0000 7f005bfff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f005c15a910 0x7f005c070080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:03.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.021+0000 7f0060d60640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f005c06d4b0 0x7f005c06c210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:48394/0 (socket says 172.21.15.49:48394) 2023-11-06T02:09:03.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.021+0000 7f0060d60640 1 -- 172.21.15.49:0/812464110 learned_addr learned my addr 172.21.15.49:0/812464110 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:09:03.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.021+0000 7f005b7fe640 1 --2- 172.21.15.49:0/812464110 >> v2:172.21.15.150:3300/0 conn(0x7f005c0af9d0 0x7f005c0705c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:03.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.021+0000 7f005bfff640 1 -- 172.21.15.49:0/812464110 >> v2:172.21.15.150:3300/0 conn(0x7f005c0af9d0 msgr2=0x7f005c0705c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:03.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.021+0000 7f005bfff640 1 --2- 172.21.15.49:0/812464110 >> v2:172.21.15.150:3300/0 conn(0x7f005c0af9d0 0x7f005c0705c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:03.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.021+0000 7f005bfff640 1 -- 172.21.15.49:0/812464110 >> v1:172.21.15.49:6789/0 conn(0x7f005c06d4b0 legacy=0x7f005c06c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:03.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.021+0000 7f005bfff640 1 -- 172.21.15.49:0/812464110 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f005c0b1f70 con 0x7f005c15a910 2023-11-06T02:09:03.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.021+0000 7f005affd640 1 -- 172.21.15.49:0/812464110 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2867982090 0 0) 0x7f005c0b16d0 con 0x7f005c06d4b0 2023-11-06T02:09:03.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.025+0000 7f005bfff640 1 --2- 172.21.15.49:0/812464110 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f005c15a910 0x7f005c070080 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f004c0097c0 tx=0x7f004c01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a491a00396fcc182 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:03.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.025+0000 7f005affd640 1 -- 172.21.15.49:0/812464110 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f004c0a7070 con 0x7f005c15a910 2023-11-06T02:09:03.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.025+0000 7f005affd640 1 -- 172.21.15.49:0/812464110 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f004c0ab430 con 0x7f005c15a910 2023-11-06T02:09:03.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.025+0000 7f005affd640 1 -- 172.21.15.49:0/812464110 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f004c0ab7c0 con 0x7f005c15a910 2023-11-06T02:09:03.026 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.025+0000 7f00627c3640 1 -- 172.21.15.49:0/812464110 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f005c15a910 msgr2=0x7f005c070080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:03.026 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.025+0000 7f00627c3640 1 --2- 172.21.15.49:0/812464110 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f005c15a910 0x7f005c070080 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f004c0097c0 tx=0x7f004c01c410 comp rx=0 tx=0).stop 2023-11-06T02:09:03.026 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.025+0000 7f00627c3640 1 -- 172.21.15.49:0/812464110 shutdown_connections 2023-11-06T02:09:03.026 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.025+0000 7f00627c3640 1 --2- 172.21.15.49:0/812464110 >> v2:172.21.15.150:3300/0 conn(0x7f005c0af9d0 0x7f005c0705c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:03.026 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.025+0000 7f00627c3640 1 --2- 172.21.15.49:0/812464110 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f005c15a910 0x7f005c070080 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:03.027 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.025+0000 7f00627c3640 1 -- 172.21.15.49:0/812464110 >> 172.21.15.49:0/812464110 conn(0x7f005c0cec90 msgr2=0x7f005c0cb0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:03.027 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.025+0000 7f00627c3640 1 -- 172.21.15.49:0/812464110 shutdown_connections 2023-11-06T02:09:03.027 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.025+0000 7f00627c3640 1 -- 172.21.15.49:0/812464110 wait complete. 2023-11-06T02:09:03.027 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.025+0000 7f00627c3640 1 Processor -- start 2023-11-06T02:09:03.028 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.025+0000 7f00627c3640 1 -- start start 2023-11-06T02:09:03.030 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.029+0000 7f00627c3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f005c06d4b0 0x7f005c156c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:03.031 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.029+0000 7f00627c3640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f005c0af9d0 0x7f005c13cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:03.031 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.029+0000 7f00627c3640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f005c14ce60 0x7f005c14d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:03.031 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.029+0000 7f00627c3640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f005c151ee0 con 0x7f005c06d4b0 2023-11-06T02:09:03.031 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.029+0000 7f00627c3640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f005c1571b0 con 0x7f005c14ce60 2023-11-06T02:09:03.032 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.029+0000 7f00627c3640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f005c14d850 con 0x7f005c0af9d0 2023-11-06T02:09:03.032 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.029+0000 7f005bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f005c06d4b0 0x7f005c156c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:03.032 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.029+0000 7f005bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f005c06d4b0 0x7f005c156c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47634/0 (socket says 172.21.15.49:47634) 2023-11-06T02:09:03.032 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.029+0000 7f005bfff640 1 -- 172.21.15.49:0/160672546 learned_addr learned my addr 172.21.15.49:0/160672546 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:03.032 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.029+0000 7f0060d60640 1 --2- 172.21.15.49:0/160672546 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f005c14ce60 0x7f005c14d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:03.033 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.029+0000 7f005b7fe640 1 --2- 172.21.15.49:0/160672546 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f005c0af9d0 0x7f005c13cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:03.033 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.029+0000 7f005bfff640 1 -- 172.21.15.49:0/160672546 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f005c0af9d0 msgr2=0x7f005c13cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:03.033 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.029+0000 7f005bfff640 1 --2- 172.21.15.49:0/160672546 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f005c0af9d0 0x7f005c13cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:03.033 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.029+0000 7f005bfff640 1 -- 172.21.15.49:0/160672546 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f005c14ce60 msgr2=0x7f005c14d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:03.033 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.029+0000 7f005bfff640 1 --2- 172.21.15.49:0/160672546 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f005c14ce60 0x7f005c14d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:03.034 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.029+0000 7f005bfff640 1 -- 172.21.15.49:0/160672546 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f004c009420 con 0x7f005c06d4b0 2023-11-06T02:09:03.034 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.029+0000 7f0060d60640 1 --2- 172.21.15.49:0/160672546 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f005c14ce60 0x7f005c14d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:09:03.034 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.029+0000 7f005bfff640 1 --2- 172.21.15.49:0/160672546 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f005c06d4b0 0x7f005c156c70 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f004c01a040 tx=0x7f004c01ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:03.034 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.029+0000 7f0058ff9640 1 -- 172.21.15.49:0/160672546 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f004c0b5810 con 0x7f005c06d4b0 2023-11-06T02:09:03.035 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.029+0000 7f0058ff9640 1 -- 172.21.15.49:0/160672546 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f004c0abe90 con 0x7f005c06d4b0 2023-11-06T02:09:03.035 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.029+0000 7f00627c3640 1 -- 172.21.15.49:0/160672546 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f005c151360 con 0x7f005c06d4b0 2023-11-06T02:09:03.035 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.029+0000 7f0058ff9640 1 -- 172.21.15.49:0/160672546 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f004c0b5bd0 con 0x7f005c06d4b0 2023-11-06T02:09:03.035 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.029+0000 7f00627c3640 1 -- 172.21.15.49:0/160672546 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f005c151920 con 0x7f005c06d4b0 2023-11-06T02:09:03.035 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.029+0000 7f00627c3640 1 -- 172.21.15.49:0/160672546 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f005c074040 con 0x7f005c06d4b0 2023-11-06T02:09:03.036 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.029+0000 7f0058ff9640 1 -- 172.21.15.49:0/160672546 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f004c0b5d30 con 0x7f005c06d4b0 2023-11-06T02:09:03.036 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.029+0000 7f0058ff9640 1 --2- 172.21.15.49:0/160672546 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0048065a60 0x7f0048067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:03.036 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.029+0000 7f0058ff9640 1 -- 172.21.15.49:0/160672546 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f004c11de00 con 0x7f005c06d4b0 2023-11-06T02:09:03.036 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.029+0000 7f005b7fe640 1 --2- 172.21.15.49:0/160672546 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0048065a60 0x7f0048067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:03.036 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.033+0000 7f005b7fe640 1 --2- 172.21.15.49:0/160672546 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0048065a60 0x7f0048067f20 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f005c0afe40 tx=0x7f0050008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:03.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.033+0000 7f0058ff9640 1 -- 172.21.15.49:0/160672546 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f004c0eb6f0 con 0x7f005c06d4b0 2023-11-06T02:09:03.188 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.189+0000 7f00627c3640 1 -- 172.21.15.49:0/160672546 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f005c06d980 con 0x7f005c06d4b0 2023-11-06T02:09:03.189 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.193+0000 7f0058ff9640 1 -- 172.21.15.49:0/160672546 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f004c0eed10 con 0x7f005c06d4b0 2023-11-06T02:09:03.190 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2023-11-06T02:09:03.192 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.193+0000 7f00627c3640 1 -- 172.21.15.49:0/160672546 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0048065a60 msgr2=0x7f0048067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:03.192 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.193+0000 7f00627c3640 1 --2- 172.21.15.49:0/160672546 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0048065a60 0x7f0048067f20 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f005c0afe40 tx=0x7f0050008040 comp rx=0 tx=0).stop 2023-11-06T02:09:03.192 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.193+0000 7f00627c3640 1 -- 172.21.15.49:0/160672546 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f005c06d4b0 msgr2=0x7f005c156c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:03.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.193+0000 7f00627c3640 1 --2- 172.21.15.49:0/160672546 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f005c06d4b0 0x7f005c156c70 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f004c01a040 tx=0x7f004c01ac50 comp rx=0 tx=0).stop 2023-11-06T02:09:03.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.193+0000 7f00627c3640 1 -- 172.21.15.49:0/160672546 shutdown_connections 2023-11-06T02:09:03.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.193+0000 7f00627c3640 1 --2- 172.21.15.49:0/160672546 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0048065a60 0x7f0048067f20 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:03.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.193+0000 7f00627c3640 1 --2- 172.21.15.49:0/160672546 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f005c14ce60 0x7f005c14d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:03.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.193+0000 7f00627c3640 1 --2- 172.21.15.49:0/160672546 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f005c0af9d0 0x7f005c13cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:03.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.193+0000 7f00627c3640 1 --2- 172.21.15.49:0/160672546 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f005c06d4b0 0x7f005c156c70 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:03.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.193+0000 7f00627c3640 1 -- 172.21.15.49:0/160672546 >> 172.21.15.49:0/160672546 conn(0x7f005c0cec90 msgr2=0x7f005c0ccad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:03.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.193+0000 7f00627c3640 1 -- 172.21.15.49:0/160672546 shutdown_connections 2023-11-06T02:09:03.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.193+0000 7f00627c3640 1 -- 172.21.15.49:0/160672546 wait complete. 2023-11-06T02:09:03.208 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T02:09:03.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.333+0000 7f06b4016640 1 Processor -- start 2023-11-06T02:09:03.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.333+0000 7f06b4016640 1 -- start start 2023-11-06T02:09:03.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.333+0000 7f06b4016640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f06ac15ab50 0x7f06ac0c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:03.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.333+0000 7f06b4016640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f06ac0b0790 0x7f06ac0c2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:03.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.333+0000 7f06b4016640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06ac0cab80 con 0x7f06ac0cb010 2023-11-06T02:09:03.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.333+0000 7f06b4016640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06ac15af30 con 0x7f06ac0b0790 2023-11-06T02:09:03.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.333+0000 7f06b4016640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06ac0cb5b0 con 0x7f06ac15ab50 2023-11-06T02:09:03.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.333+0000 7f06b25b3640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f06ac0b0790 0x7f06ac0c2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:03.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.333+0000 7f06b25b3640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f06ac0b0790 0x7f06ac0c2710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47636/0 (socket says 172.21.15.49:47636) 2023-11-06T02:09:03.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.333+0000 7f06b25b3640 1 -- 172.21.15.49:0/344146249 learned_addr learned my addr 172.21.15.49:0/344146249 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:03.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.333+0000 7f06b1db2640 1 --2- 172.21.15.49:0/344146249 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f06ac15ab50 0x7f06ac0c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:03.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.333+0000 7f06b25b3640 1 -- 172.21.15.49:0/344146249 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f06ac15ab50 msgr2=0x7f06ac0c9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:03.334 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.333+0000 7f06b25b3640 1 --2- 172.21.15.49:0/344146249 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f06ac15ab50 0x7f06ac0c9bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:03.334 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.333+0000 7f06b25b3640 1 -- 172.21.15.49:0/344146249 >> v1:172.21.15.150:6789/0 conn(0x7f06ac0cb010 legacy=0x7f06ac0b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:03.334 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.333+0000 7f06b25b3640 1 -- 172.21.15.49:0/344146249 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06ac0cb890 con 0x7f06ac0b0790 2023-11-06T02:09:03.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.337+0000 7f06b25b3640 1 --2- 172.21.15.49:0/344146249 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f06ac0b0790 0x7f06ac0c2710 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f06a4009e50 tx=0x7f06a401c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b3abef7eeb47945 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:03.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.337+0000 7f06b0db0640 1 -- 172.21.15.49:0/344146249 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a40a5820 con 0x7f06ac0b0790 2023-11-06T02:09:03.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.337+0000 7f06b0db0640 1 -- 172.21.15.49:0/344146249 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06a40a5980 con 0x7f06ac0b0790 2023-11-06T02:09:03.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.337+0000 7f06b0db0640 1 -- 172.21.15.49:0/344146249 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a40a5d10 con 0x7f06ac0b0790 2023-11-06T02:09:03.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.337+0000 7f06b4016640 1 -- 172.21.15.49:0/344146249 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f06ac0b0790 msgr2=0x7f06ac0c2710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:03.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.337+0000 7f06b4016640 1 --2- 172.21.15.49:0/344146249 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f06ac0b0790 0x7f06ac0c2710 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f06a4009e50 tx=0x7f06a401c9a0 comp rx=0 tx=0).stop 2023-11-06T02:09:03.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.337+0000 7f06b4016640 1 -- 172.21.15.49:0/344146249 shutdown_connections 2023-11-06T02:09:03.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.337+0000 7f06b4016640 1 --2- 172.21.15.49:0/344146249 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f06ac0b0790 0x7f06ac0c2710 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:03.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.337+0000 7f06b4016640 1 --2- 172.21.15.49:0/344146249 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f06ac15ab50 0x7f06ac0c9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:03.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.337+0000 7f06b4016640 1 -- 172.21.15.49:0/344146249 >> 172.21.15.49:0/344146249 conn(0x7f06ac06c0a0 msgr2=0x7f06ac06c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:03.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.337+0000 7f06b4016640 1 -- 172.21.15.49:0/344146249 shutdown_connections 2023-11-06T02:09:03.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.337+0000 7f06b4016640 1 -- 172.21.15.49:0/344146249 wait complete. 2023-11-06T02:09:03.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.337+0000 7f06b4016640 1 Processor -- start 2023-11-06T02:09:03.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.337+0000 7f06b4016640 1 -- start start 2023-11-06T02:09:03.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.337+0000 7f06b4016640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06ac0b0790 0x7f06ac157670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:03.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.337+0000 7f06b4016640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f06ac0cb010 0x7f06ac15b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:03.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.337+0000 7f06b4016640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f06ac15ab50 0x7f06ac14c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:03.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.337+0000 7f06b4016640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06ac0c7340 con 0x7f06ac0cb010 2023-11-06T02:09:03.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.337+0000 7f06b4016640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06ac1585a0 con 0x7f06ac0b0790 2023-11-06T02:09:03.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.337+0000 7f06b4016640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06ac0c1b80 con 0x7f06ac15ab50 2023-11-06T02:09:03.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.337+0000 7f06b15b1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f06ac0cb010 0x7f06ac15b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:03.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.337+0000 7f06b15b1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f06ac0cb010 0x7f06ac15b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47642/0 (socket says 172.21.15.49:47642) 2023-11-06T02:09:03.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.337+0000 7f06b15b1640 1 -- 172.21.15.49:0/3392660220 learned_addr learned my addr 172.21.15.49:0/3392660220 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:03.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.337+0000 7f06b25b3640 1 --2- 172.21.15.49:0/3392660220 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f06ac15ab50 0x7f06ac14c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:03.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.337+0000 7f06b1db2640 1 --2- 172.21.15.49:0/3392660220 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06ac0b0790 0x7f06ac157670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:03.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.337+0000 7f06b15b1640 1 -- 172.21.15.49:0/3392660220 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f06ac15ab50 msgr2=0x7f06ac14c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:03.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.337+0000 7f06b15b1640 1 --2- 172.21.15.49:0/3392660220 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f06ac15ab50 0x7f06ac14c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:03.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.337+0000 7f06b15b1640 1 -- 172.21.15.49:0/3392660220 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06ac0b0790 msgr2=0x7f06ac157670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:03.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.337+0000 7f06b15b1640 1 --2- 172.21.15.49:0/3392660220 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06ac0b0790 0x7f06ac157670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:03.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.337+0000 7f06b15b1640 1 -- 172.21.15.49:0/3392660220 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06a4009b30 con 0x7f06ac0cb010 2023-11-06T02:09:03.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.337+0000 7f06b15b1640 1 --2- 172.21.15.49:0/3392660220 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f06ac0cb010 0x7f06ac15b070 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f069c00c450 tx=0x7f069c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:03.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.337+0000 7f06a27fc640 1 -- 172.21.15.49:0/3392660220 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f069c012830 con 0x7f06ac0cb010 2023-11-06T02:09:03.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.341+0000 7f06a27fc640 1 -- 172.21.15.49:0/3392660220 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f069c0129b0 con 0x7f06ac0cb010 2023-11-06T02:09:03.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.341+0000 7f06b4016640 1 -- 172.21.15.49:0/3392660220 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06ac15b5b0 con 0x7f06ac0cb010 2023-11-06T02:09:03.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.341+0000 7f06a27fc640 1 -- 172.21.15.49:0/3392660220 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f069c012d60 con 0x7f06ac0cb010 2023-11-06T02:09:03.344 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.341+0000 7f06b4016640 1 -- 172.21.15.49:0/3392660220 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06ac157fe0 con 0x7f06ac0cb010 2023-11-06T02:09:03.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.341+0000 7f06b4016640 1 -- 172.21.15.49:0/3392660220 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06ac0b0de0 con 0x7f06ac0cb010 2023-11-06T02:09:03.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.341+0000 7f06a27fc640 1 -- 172.21.15.49:0/3392660220 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f069c01c460 con 0x7f06ac0cb010 2023-11-06T02:09:03.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.341+0000 7f06a27fc640 1 --2- 172.21.15.49:0/3392660220 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0690065a90 0x7f0690067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:03.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.341+0000 7f06a27fc640 1 -- 172.21.15.49:0/3392660220 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f069c0830f0 con 0x7f06ac0cb010 2023-11-06T02:09:03.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.341+0000 7f06b1db2640 1 --2- 172.21.15.49:0/3392660220 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0690065a90 0x7f0690067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:03.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.341+0000 7f06b1db2640 1 --2- 172.21.15.49:0/3392660220 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0690065a90 0x7f0690067f50 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f06a8006fd0 tx=0x7f06a8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:03.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.345+0000 7f06a27fc640 1 -- 172.21.15.49:0/3392660220 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f069c050950 con 0x7f06ac0cb010 2023-11-06T02:09:03.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.505+0000 7f06b4016640 1 -- 172.21.15.49:0/3392660220 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f06ac074040 con 0x7f06ac0cb010 2023-11-06T02:09:03.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.505+0000 7f06a27fc640 1 -- 172.21.15.49:0/3392660220 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+17811 (secure 0 0 0) 0x7f069c053f70 con 0x7f06ac0cb010 2023-11-06T02:09:03.503 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:09:03.504 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":23,"fsid":"1a8b035c-6ba4-499a-b876-aea47fd9c0d4","created":"2023-11-06T02:08:06.440857+0000","modified":"2023-11-06T02:08:31.594964+0000","last_up_change":"2023-11-06T02:08:17.982372+0000","last_in_change":"2023-11-06T02:08:08.906631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T02:08:19.252222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T02:08:24.776211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-06T02:08:28.936178+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-06T02:08:29.563468+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ad55af8-0571-4e6a-8722-b5d429b7496d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6809","nonce":38116},{"type":"v1","addr":"172.21.15.49:6811","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6813","nonce":38116},{"type":"v1","addr":"172.21.15.49:6815","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6821","nonce":38116},{"type":"v1","addr":"172.21.15.49:6823","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6817","nonce":38116},{"type":"v1","addr":"172.21.15.49:6819","nonce":38116}]},"public_addr":"172.21.15.49:6811/38116","cluster_addr":"172.21.15.49:6815/38116","heartbeat_back_addr":"172.21.15.49:6823/38116","heartbeat_front_addr":"172.21.15.49:6819/38116","state":["exists","up"]},{"osd":1,"uuid":"c8b31f85-a757-43ed-9419-351acc46611b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":38113},{"type":"v1","addr":"172.21.15.49:6810","nonce":38113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":38113},{"type":"v1","addr":"172.21.15.49:6814","nonce":38113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":38113},{"type":"v1","addr":"172.21.15.49:6822","nonce":38113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":38113},{"type":"v1","addr":"172.21.15.49:6818","nonce":38113}]},"public_addr":"172.21.15.49:6810/38113","cluster_addr":"172.21.15.49:6814/38113","heartbeat_back_addr":"172.21.15.49:6822/38113","heartbeat_front_addr":"172.21.15.49:6818/38113","state":["exists","up"]},{"osd":2,"uuid":"555b3bf3-0a70-4b91-8efd-e4204b5fc26e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":38114},{"type":"v1","addr":"172.21.15.49:6801","nonce":38114}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":38114},{"type":"v1","addr":"172.21.15.49:6803","nonce":38114}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":38114},{"type":"v1","addr":"172.21.15.49:6807","nonce":38114}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":38114},{"type":"v1","addr":"172.21.15.49:6805","nonce":38114}]},"public_addr":"172.21.15.49:6801/38114","cluster_addr":"172.21.15.49:6803/38114","heartbeat_back_addr":"172.21.15.49:6807/38114","heartbeat_front_addr":"172.21.15.49:6805/38114","state":["exists","up"]},{"osd":3,"uuid":"8d4e5d77-03ef-4b9b-bfcb-5ae93a46795c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":38115},{"type":"v1","addr":"172.21.15.49:6825","nonce":38115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":38115},{"type":"v1","addr":"172.21.15.49:6827","nonce":38115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":38115},{"type":"v1","addr":"172.21.15.49:6831","nonce":38115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":38115},{"type":"v1","addr":"172.21.15.49:6829","nonce":38115}]},"public_addr":"172.21.15.49:6825/38115","cluster_addr":"172.21.15.49:6827/38115","heartbeat_back_addr":"172.21.15.49:6831/38115","heartbeat_front_addr":"172.21.15.49:6829/38115","state":["exists","up"]},{"osd":4,"uuid":"dba03ad4-68a3-4cf3-9218-ac611f02eacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":38417},{"type":"v1","addr":"172.21.15.150:6809","nonce":38417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":38417},{"type":"v1","addr":"172.21.15.150:6811","nonce":38417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":38417},{"type":"v1","addr":"172.21.15.150:6815","nonce":38417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":38417},{"type":"v1","addr":"172.21.15.150:6813","nonce":38417}]},"public_addr":"172.21.15.150:6809/38417","cluster_addr":"172.21.15.150:6811/38417","heartbeat_back_addr":"172.21.15.150:6815/38417","heartbeat_front_addr":"172.21.15.150:6813/38417","state":["exists","up"]},{"osd":5,"uuid":"0df9c303-c255-4640-8e91-01f1c3efe18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":38418},{"type":"v1","addr":"172.21.15.150:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":38418},{"type":"v1","addr":"172.21.15.150:6803","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":38418},{"type":"v1","addr":"172.21.15.150:6807","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":38418},{"type":"v1","addr":"172.21.15.150:6805","nonce":38418}]},"public_addr":"172.21.15.150:6801/38418","cluster_addr":"172.21.15.150:6803/38418","heartbeat_back_addr":"172.21.15.150:6807/38418","heartbeat_front_addr":"172.21.15.150:6805/38418","state":["exists","up"]},{"osd":6,"uuid":"e59fa189-4b70-4c06-9722-c02b90828bf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":38428},{"type":"v1","addr":"172.21.15.150:6825","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":38428},{"type":"v1","addr":"172.21.15.150:6827","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":38428},{"type":"v1","addr":"172.21.15.150:6831","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":38428},{"type":"v1","addr":"172.21.15.150:6829","nonce":38428}]},"public_addr":"172.21.15.150:6825/38428","cluster_addr":"172.21.15.150:6827/38428","heartbeat_back_addr":"172.21.15.150:6831/38428","heartbeat_front_addr":"172.21.15.150:6829/38428","state":["exists","up"]},{"osd":7,"uuid":"a64f57e1-ef98-46bd-b304-385759e95898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":38421},{"type":"v1","addr":"172.21.15.150:6817","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":38421},{"type":"v1","addr":"172.21.15.150:6819","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":38421},{"type":"v1","addr":"172.21.15.150:6823","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":38421},{"type":"v1","addr":"172.21.15.150:6821","nonce":38421}]},"public_addr":"172.21.15.150:6817/38421","cluster_addr":"172.21.15.150:6819/38421","heartbeat_back_addr":"172.21.15.150:6823/38421","heartbeat_front_addr":"172.21.15.150:6821/38421","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.438552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.413280+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.296466+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.424966+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.276953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.230984+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.410235+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.480084+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T02:09:03.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.509+0000 7f06b4016640 1 -- 172.21.15.49:0/3392660220 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0690065a90 msgr2=0x7f0690067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:03.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.509+0000 7f06b4016640 1 --2- 172.21.15.49:0/3392660220 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0690065a90 0x7f0690067f50 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f06a8006fd0 tx=0x7f06a8008040 comp rx=0 tx=0).stop 2023-11-06T02:09:03.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.509+0000 7f06b4016640 1 -- 172.21.15.49:0/3392660220 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f06ac0cb010 msgr2=0x7f06ac15b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:03.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.509+0000 7f06b4016640 1 --2- 172.21.15.49:0/3392660220 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f06ac0cb010 0x7f06ac15b070 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f069c00c450 tx=0x7f069c00c910 comp rx=0 tx=0).stop 2023-11-06T02:09:03.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.509+0000 7f06b4016640 1 -- 172.21.15.49:0/3392660220 shutdown_connections 2023-11-06T02:09:03.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.509+0000 7f06b4016640 1 --2- 172.21.15.49:0/3392660220 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0690065a90 0x7f0690067f50 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:03.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.509+0000 7f06b4016640 1 --2- 172.21.15.49:0/3392660220 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f06ac0cb010 0x7f06ac15b070 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:03.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.509+0000 7f06b4016640 1 --2- 172.21.15.49:0/3392660220 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f06ac15ab50 0x7f06ac14c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:03.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.509+0000 7f06b4016640 1 --2- 172.21.15.49:0/3392660220 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06ac0b0790 0x7f06ac157670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:03.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.509+0000 7f06b4016640 1 -- 172.21.15.49:0/3392660220 >> 172.21.15.49:0/3392660220 conn(0x7f06ac06c0a0 msgr2=0x7f06ac0c9bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:03.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.509+0000 7f06b4016640 1 -- 172.21.15.49:0/3392660220 shutdown_connections 2023-11-06T02:09:03.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.509+0000 7f06b4016640 1 -- 172.21.15.49:0/3392660220 wait complete. 2023-11-06T02:09:03.522 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T02:08:19.252222+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T02:08:24.776211+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T02:08:28.936178+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T02:08:29.563468+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T02:09:03.523 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T02:09:03.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.657+0000 7f6cab92f640 1 Processor -- start 2023-11-06T02:09:03.657 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.657+0000 7f6cab92f640 1 -- start start 2023-11-06T02:09:03.659 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.661+0000 7f6cab92f640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ca41595a0 0x7f6ca415dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:03.659 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.661+0000 7f6cab92f640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f6ca415a990 0x7f6ca40c8e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:03.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.661+0000 7f6cab92f640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ca406c5a0 con 0x7f6ca406d640 2023-11-06T02:09:03.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.661+0000 7f6cab92f640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ca415e250 con 0x7f6ca415a990 2023-11-06T02:09:03.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.661+0000 7f6cab92f640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca406c880 con 0x7f6ca41595a0 2023-11-06T02:09:03.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.661+0000 7f6ca8eca640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f6ca406d640 0x7f6ca40b0220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:48408/0 (socket says 172.21.15.49:48408) 2023-11-06T02:09:03.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.661+0000 7f6ca8eca640 1 -- 172.21.15.49:0/3209894751 learned_addr learned my addr 172.21.15.49:0/3209894751 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:09:03.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.661+0000 7f6ca9ecc640 1 --2- 172.21.15.49:0/3209894751 >> v2:172.21.15.150:3300/0 conn(0x7f6ca415a990 0x7f6ca40c8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:03.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.661+0000 7f6ca96cb640 1 --2- 172.21.15.49:0/3209894751 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ca41595a0 0x7f6ca415dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:03.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.661+0000 7f6ca3fff640 1 -- 172.21.15.49:0/3209894751 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4140882474 0 0) 0x7f6ca406c5a0 con 0x7f6ca406d640 2023-11-06T02:09:03.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.661+0000 7f6ca3fff640 1 -- 172.21.15.49:0/3209894751 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c8c003530 con 0x7f6ca406d640 2023-11-06T02:09:03.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.661+0000 7f6ca3fff640 1 -- 172.21.15.49:0/3209894751 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 682396750 0 0) 0x7f6c8c003530 con 0x7f6ca406d640 2023-11-06T02:09:03.662 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.661+0000 7f6ca3fff640 1 -- 172.21.15.49:0/3209894751 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ca41595a0 msgr2=0x7f6ca415dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:03.662 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.661+0000 7f6ca3fff640 1 --2- 172.21.15.49:0/3209894751 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ca41595a0 0x7f6ca415dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:03.662 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.661+0000 7f6ca3fff640 1 -- 172.21.15.49:0/3209894751 >> v2:172.21.15.150:3300/0 conn(0x7f6ca415a990 msgr2=0x7f6ca40c8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:03.662 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.661+0000 7f6ca3fff640 1 --2- 172.21.15.49:0/3209894751 >> v2:172.21.15.150:3300/0 conn(0x7f6ca415a990 0x7f6ca40c8e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:03.662 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.661+0000 7f6ca3fff640 1 -- 172.21.15.49:0/3209894751 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ca406cb60 con 0x7f6ca406d640 2023-11-06T02:09:03.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.661+0000 7f6ca3fff640 1 -- 172.21.15.49:0/3209894751 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7f6c98001680 con 0x7f6ca406d640 2023-11-06T02:09:03.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.661+0000 7f6ca3fff640 1 -- 172.21.15.49:0/3209894751 >> v1:172.21.15.49:6789/0 conn(0x7f6ca406d640 legacy=0x7f6ca40b0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:03.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.661+0000 7f6ca3fff640 1 --2- 172.21.15.49:0/3209894751 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ca41595a0 0x7f6c8c003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:03.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.661+0000 7f6ca3fff640 1 --2- 172.21.15.49:0/3209894751 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c8c003f00 0x7f6c8c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:03.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.661+0000 7f6ca3fff640 1 --2- 172.21.15.49:0/3209894751 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6c8c004d10 0x7f6c8c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:03.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.661+0000 7f6ca3fff640 1 -- 172.21.15.49:0/3209894751 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca415e250 con 0x7f6c8c003f00 2023-11-06T02:09:03.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.661+0000 7f6ca3fff640 1 -- 172.21.15.49:0/3209894751 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca406c880 con 0x7f6c8c004d10 2023-11-06T02:09:03.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.661+0000 7f6ca3fff640 1 -- 172.21.15.49:0/3209894751 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8c00b990 con 0x7f6ca41595a0 2023-11-06T02:09:03.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.661+0000 7f6ca8eca640 1 --2- 172.21.15.49:0/3209894751 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c8c003f00 0x7f6c8c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:03.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.661+0000 7f6ca96cb640 1 --2- 172.21.15.49:0/3209894751 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ca41595a0 0x7f6c8c003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:03.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.661+0000 7f6ca9ecc640 1 --2- 172.21.15.49:0/3209894751 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6c8c004d10 0x7f6c8c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:03.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.661+0000 7f6ca8eca640 1 -- 172.21.15.49:0/3209894751 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ca41595a0 msgr2=0x7f6c8c003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:03.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.661+0000 7f6ca8eca640 1 --2- 172.21.15.49:0/3209894751 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ca41595a0 0x7f6c8c003980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:03.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.661+0000 7f6ca8eca640 1 -- 172.21.15.49:0/3209894751 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6c8c004d10 msgr2=0x7f6c8c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:03.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.661+0000 7f6ca8eca640 1 --2- 172.21.15.49:0/3209894751 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6c8c004d10 0x7f6c8c005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:03.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.661+0000 7f6ca8eca640 1 -- 172.21.15.49:0/3209894751 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c8c003530 con 0x7f6c8c003f00 2023-11-06T02:09:03.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.661+0000 7f6ca9ecc640 1 --2- 172.21.15.49:0/3209894751 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6c8c004d10 0x7f6c8c005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:09:03.669 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.665+0000 7f6ca8eca640 1 --2- 172.21.15.49:0/3209894751 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c8c003f00 0x7f6c8c0047d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6c9800c290 tx=0x7f6c9801f3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b84d966efa6b1353 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:03.670 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.665+0000 7f6ca3fff640 1 -- 172.21.15.49:0/3209894751 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c98005ae0 con 0x7f6c8c003f00 2023-11-06T02:09:03.670 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.665+0000 7f6ca3fff640 1 -- 172.21.15.49:0/3209894751 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c980ab070 con 0x7f6c8c003f00 2023-11-06T02:09:03.670 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.665+0000 7f6ca3fff640 1 -- 172.21.15.49:0/3209894751 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c98005e80 con 0x7f6c8c003f00 2023-11-06T02:09:03.670 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.665+0000 7f6cab92f640 1 -- 172.21.15.49:0/3209894751 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c8c003f00 msgr2=0x7f6c8c0047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:03.670 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.665+0000 7f6cab92f640 1 --2- 172.21.15.49:0/3209894751 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c8c003f00 0x7f6c8c0047d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6c9800c290 tx=0x7f6c9801f3d0 comp rx=0 tx=0).stop 2023-11-06T02:09:03.671 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.665+0000 7f6ca9ecc640 1 -- 172.21.15.49:0/3209894751 reap_dead start 2023-11-06T02:09:03.671 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.665+0000 7f6cab92f640 1 -- 172.21.15.49:0/3209894751 shutdown_connections 2023-11-06T02:09:03.671 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.665+0000 7f6cab92f640 1 -- 172.21.15.49:0/3209894751 >> 172.21.15.49:0/3209894751 conn(0x7f6ca406a370 msgr2=0x7f6ca40ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:03.671 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.665+0000 7f6cab92f640 1 -- 172.21.15.49:0/3209894751 shutdown_connections 2023-11-06T02:09:03.672 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.665+0000 7f6cab92f640 1 -- 172.21.15.49:0/3209894751 wait complete. 2023-11-06T02:09:03.672 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.665+0000 7f6cab92f640 1 Processor -- start 2023-11-06T02:09:03.672 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.665+0000 7f6cab92f640 1 -- start start 2023-11-06T02:09:03.672 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.665+0000 7f6cab92f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c8c003f00 0x7f6ca414c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:03.673 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.665+0000 7f6cab92f640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ca41f3220 0x7f6ca41f2060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:03.673 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.665+0000 7f6cab92f640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ca41f25a0 0x7f6ca40c7940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:03.673 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.665+0000 7f6cab92f640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca4155f40 con 0x7f6c8c003f00 2023-11-06T02:09:03.673 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.665+0000 7f6cab92f640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca41f2b20 con 0x7f6ca41f3220 2023-11-06T02:09:03.673 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.665+0000 7f6cab92f640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca41f2e00 con 0x7f6ca41f25a0 2023-11-06T02:09:03.673 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.665+0000 7f6ca96cb640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c8c003f00 0x7f6ca414c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:03.674 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.665+0000 7f6ca96cb640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c8c003f00 0x7f6ca414c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47654/0 (socket says 172.21.15.49:47654) 2023-11-06T02:09:03.674 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.665+0000 7f6ca96cb640 1 -- 172.21.15.49:0/3461646815 learned_addr learned my addr 172.21.15.49:0/3461646815 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:03.674 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.665+0000 7f6ca9ecc640 1 --2- 172.21.15.49:0/3461646815 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ca41f25a0 0x7f6ca40c7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:03.674 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.665+0000 7f6ca8eca640 1 --2- 172.21.15.49:0/3461646815 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ca41f3220 0x7f6ca41f2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:03.675 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.665+0000 7f6ca96cb640 1 -- 172.21.15.49:0/3461646815 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ca41f25a0 msgr2=0x7f6ca40c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:03.675 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.665+0000 7f6ca96cb640 1 --2- 172.21.15.49:0/3461646815 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ca41f25a0 0x7f6ca40c7940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:03.675 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.665+0000 7f6ca96cb640 1 -- 172.21.15.49:0/3461646815 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ca41f3220 msgr2=0x7f6ca41f2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:03.675 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.665+0000 7f6ca96cb640 1 --2- 172.21.15.49:0/3461646815 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ca41f3220 0x7f6ca41f2060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:03.676 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.665+0000 7f6ca96cb640 1 -- 172.21.15.49:0/3461646815 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ca4155b60 con 0x7f6c8c003f00 2023-11-06T02:09:03.676 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.665+0000 7f6ca8eca640 1 --2- 172.21.15.49:0/3461646815 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ca41f3220 0x7f6ca41f2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:09:03.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.665+0000 7f6ca96cb640 1 --2- 172.21.15.49:0/3461646815 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c8c003f00 0x7f6ca414c240 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6c94010890 tx=0x7f6c94010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:03.679 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.665+0000 7f6ca1ffb640 1 -- 172.21.15.49:0/3461646815 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c94009630 con 0x7f6c8c003f00 2023-11-06T02:09:03.679 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.665+0000 7f6ca1ffb640 1 -- 172.21.15.49:0/3461646815 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c9400eb60 con 0x7f6c8c003f00 2023-11-06T02:09:03.679 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.665+0000 7f6cab92f640 1 -- 172.21.15.49:0/3461646815 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ca41f36a0 con 0x7f6c8c003f00 2023-11-06T02:09:03.679 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.669+0000 7f6ca1ffb640 1 -- 172.21.15.49:0/3461646815 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c9401c400 con 0x7f6c8c003f00 2023-11-06T02:09:03.679 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.669+0000 7f6cab92f640 1 -- 172.21.15.49:0/3461646815 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ca40c7e80 con 0x7f6c8c003f00 2023-11-06T02:09:03.680 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.669+0000 7f6cab92f640 1 -- 172.21.15.49:0/3461646815 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ca41561b0 con 0x7f6c8c003f00 2023-11-06T02:09:03.680 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.669+0000 7f6ca1ffb640 1 -- 172.21.15.49:0/3461646815 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f6c9401c560 con 0x7f6c8c003f00 2023-11-06T02:09:03.680 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.669+0000 7f6ca1ffb640 1 --2- 172.21.15.49:0/3461646815 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6c90065a90 0x7f6c90067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:03.680 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.669+0000 7f6ca1ffb640 1 -- 172.21.15.49:0/3461646815 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6c94083780 con 0x7f6c8c003f00 2023-11-06T02:09:03.680 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.669+0000 7f6ca8eca640 1 --2- 172.21.15.49:0/3461646815 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6c90065a90 0x7f6c90067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:03.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.669+0000 7f6ca8eca640 1 --2- 172.21.15.49:0/3461646815 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6c90065a90 0x7f6c90067f50 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6c980ae000 tx=0x7f6c98001a10 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:03.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.673+0000 7f6ca1ffb640 1 -- 172.21.15.49:0/3461646815 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f6c94050fe0 con 0x7f6c8c003f00 2023-11-06T02:09:03.825 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.829+0000 7f6cab92f640 1 -- 172.21.15.49:0/3461646815 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6ca406c210 con 0x7f6c8c003f00 2023-11-06T02:09:03.826 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.829+0000 7f6ca1ffb640 1 -- 172.21.15.49:0/3461646815 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6ca406c210 con 0x7f6c8c003f00 2023-11-06T02:09:03.826 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2023-11-06T02:09:03.828 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.829+0000 7f6cab92f640 1 -- 172.21.15.49:0/3461646815 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6c90065a90 msgr2=0x7f6c90067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:03.828 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.829+0000 7f6cab92f640 1 --2- 172.21.15.49:0/3461646815 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6c90065a90 0x7f6c90067f50 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6c980ae000 tx=0x7f6c98001a10 comp rx=0 tx=0).stop 2023-11-06T02:09:03.829 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.829+0000 7f6cab92f640 1 -- 172.21.15.49:0/3461646815 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c8c003f00 msgr2=0x7f6ca414c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:03.829 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.829+0000 7f6cab92f640 1 --2- 172.21.15.49:0/3461646815 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c8c003f00 0x7f6ca414c240 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6c94010890 tx=0x7f6c94010d50 comp rx=0 tx=0).stop 2023-11-06T02:09:03.829 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.829+0000 7f6cab92f640 1 -- 172.21.15.49:0/3461646815 shutdown_connections 2023-11-06T02:09:03.829 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.829+0000 7f6cab92f640 1 --2- 172.21.15.49:0/3461646815 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6c90065a90 0x7f6c90067f50 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:03.830 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.829+0000 7f6cab92f640 1 --2- 172.21.15.49:0/3461646815 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ca41f25a0 0x7f6ca40c7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:03.830 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.829+0000 7f6cab92f640 1 --2- 172.21.15.49:0/3461646815 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ca41f3220 0x7f6ca41f2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:03.830 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.829+0000 7f6cab92f640 1 --2- 172.21.15.49:0/3461646815 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c8c003f00 0x7f6ca414c240 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:03.830 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.829+0000 7f6cab92f640 1 -- 172.21.15.49:0/3461646815 >> 172.21.15.49:0/3461646815 conn(0x7f6ca406a370 msgr2=0x7f6ca40cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:03.830 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.829+0000 7f6cab92f640 1 -- 172.21.15.49:0/3461646815 shutdown_connections 2023-11-06T02:09:03.831 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.829+0000 7f6cab92f640 1 -- 172.21.15.49:0/3461646815 wait complete. 2023-11-06T02:09:03.845 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T02:09:03.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.981+0000 7f8e061e8640 1 Processor -- start 2023-11-06T02:09:03.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.981+0000 7f8e061e8640 1 -- start start 2023-11-06T02:09:03.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.981+0000 7f8e061e8640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8e0015cbb0 0x7f8e000d1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:03.980 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.981+0000 7f8e061e8640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f8e000d24e0 0x7f8e000d2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:03.980 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.981+0000 7f8e061e8640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e000d3400 con 0x7f8e000d2fa0 2023-11-06T02:09:03.980 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.981+0000 7f8e061e8640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e0015cf90 con 0x7f8e000d24e0 2023-11-06T02:09:03.980 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.981+0000 7f8e061e8640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e000d0b80 con 0x7f8e0015cbb0 2023-11-06T02:09:03.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.981+0000 7f8dfeffd640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f8e000d24e0 0x7f8e000d2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:03.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.981+0000 7f8dfeffd640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f8e000d24e0 0x7f8e000d2a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47666/0 (socket says 172.21.15.49:47666) 2023-11-06T02:09:03.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.981+0000 7f8dfeffd640 1 -- 172.21.15.49:0/1362949351 learned_addr learned my addr 172.21.15.49:0/1362949351 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:03.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.981+0000 7f8dff7fe640 1 --2- 172.21.15.49:0/1362949351 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8e0015cbb0 0x7f8e000d1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:03.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.981+0000 7f8dfeffd640 1 -- 172.21.15.49:0/1362949351 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8e0015cbb0 msgr2=0x7f8e000d1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:03.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.981+0000 7f8dfeffd640 1 --2- 172.21.15.49:0/1362949351 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8e0015cbb0 0x7f8e000d1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:03.982 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.981+0000 7f8dfeffd640 1 -- 172.21.15.49:0/1362949351 >> v1:172.21.15.150:6789/0 conn(0x7f8e000d2fa0 legacy=0x7f8e000b4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:03.982 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.981+0000 7f8dfeffd640 1 -- 172.21.15.49:0/1362949351 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e000d0e60 con 0x7f8e000d24e0 2023-11-06T02:09:03.983 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.985+0000 7f8dfeffd640 1 --2- 172.21.15.49:0/1362949351 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e000d24e0 0x7f8e000d2a60 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8de8009870 tx=0x7f8de801ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f9b14d9be816027 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:03.983 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.985+0000 7f8dfe7fc640 1 -- 172.21.15.49:0/1362949351 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de80a8a30 con 0x7f8e000d24e0 2023-11-06T02:09:03.983 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.985+0000 7f8dfe7fc640 1 -- 172.21.15.49:0/1362949351 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8de80a8b90 con 0x7f8e000d24e0 2023-11-06T02:09:03.984 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.985+0000 7f8dfe7fc640 1 -- 172.21.15.49:0/1362949351 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de80b0440 con 0x7f8e000d24e0 2023-11-06T02:09:03.984 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.985+0000 7f8e061e8640 1 -- 172.21.15.49:0/1362949351 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e000d24e0 msgr2=0x7f8e000d2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:03.984 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.985+0000 7f8e061e8640 1 --2- 172.21.15.49:0/1362949351 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e000d24e0 0x7f8e000d2a60 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8de8009870 tx=0x7f8de801ed30 comp rx=0 tx=0).stop 2023-11-06T02:09:03.984 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.985+0000 7f8e061e8640 1 -- 172.21.15.49:0/1362949351 shutdown_connections 2023-11-06T02:09:03.984 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.985+0000 7f8e061e8640 1 --2- 172.21.15.49:0/1362949351 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e000d24e0 0x7f8e000d2a60 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:03.985 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.985+0000 7f8e061e8640 1 --2- 172.21.15.49:0/1362949351 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8e0015cbb0 0x7f8e000d1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:03.985 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.985+0000 7f8e061e8640 1 -- 172.21.15.49:0/1362949351 >> 172.21.15.49:0/1362949351 conn(0x7f8e000718a0 msgr2=0x7f8e00071cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:03.985 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.985+0000 7f8e061e8640 1 -- 172.21.15.49:0/1362949351 shutdown_connections 2023-11-06T02:09:03.985 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.985+0000 7f8e061e8640 1 -- 172.21.15.49:0/1362949351 wait complete. 2023-11-06T02:09:03.988 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.985+0000 7f8e061e8640 1 Processor -- start 2023-11-06T02:09:03.988 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.985+0000 7f8e061e8640 1 -- start start 2023-11-06T02:09:03.988 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.989+0000 7f8e061e8640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e000d24e0 0x7f8e00152f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:03.988 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.989+0000 7f8e061e8640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8e000d2fa0 0x7f8e00159850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:03.988 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.989+0000 7f8e061e8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e0015cbb0 0x7f8e0015f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:03.989 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.989+0000 7f8e061e8640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e001f6720 con 0x7f8e0015cbb0 2023-11-06T02:09:03.989 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.989+0000 7f8e061e8640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e00153480 con 0x7f8e000d24e0 2023-11-06T02:09:03.989 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.989+0000 7f8e061e8640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e00159d90 con 0x7f8e000d2fa0 2023-11-06T02:09:03.989 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.989+0000 7f8dfffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e0015cbb0 0x7f8e0015f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:03.990 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.989+0000 7f8dfffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e0015cbb0 0x7f8e0015f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47680/0 (socket says 172.21.15.49:47680) 2023-11-06T02:09:03.990 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.989+0000 7f8dfffff640 1 -- 172.21.15.49:0/3445722013 learned_addr learned my addr 172.21.15.49:0/3445722013 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:03.990 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.989+0000 7f8dfeffd640 1 --2- 172.21.15.49:0/3445722013 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8e000d2fa0 0x7f8e00159850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:03.990 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.989+0000 7f8dff7fe640 1 --2- 172.21.15.49:0/3445722013 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e000d24e0 0x7f8e00152f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:03.990 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.989+0000 7f8dfffff640 1 -- 172.21.15.49:0/3445722013 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8e000d2fa0 msgr2=0x7f8e00159850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:03.990 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.989+0000 7f8dfffff640 1 --2- 172.21.15.49:0/3445722013 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8e000d2fa0 0x7f8e00159850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:03.991 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.989+0000 7f8dfffff640 1 -- 172.21.15.49:0/3445722013 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e000d24e0 msgr2=0x7f8e00152f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:03.991 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.989+0000 7f8dfffff640 1 --2- 172.21.15.49:0/3445722013 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e000d24e0 0x7f8e00152f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:03.991 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.989+0000 7f8dfffff640 1 -- 172.21.15.49:0/3445722013 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8de80094d0 con 0x7f8e0015cbb0 2023-11-06T02:09:03.991 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.989+0000 7f8dfffff640 1 --2- 172.21.15.49:0/3445722013 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e0015cbb0 0x7f8e0015f710 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8df800c7a0 tx=0x7f8df800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:03.992 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.989+0000 7f8df7fff640 1 -- 172.21.15.49:0/3445722013 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df8003ba0 con 0x7f8e0015cbb0 2023-11-06T02:09:03.992 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.989+0000 7f8df7fff640 1 -- 172.21.15.49:0/3445722013 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8df8003d00 con 0x7f8e0015cbb0 2023-11-06T02:09:03.992 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.989+0000 7f8e061e8640 1 -- 172.21.15.49:0/3445722013 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e001f6440 con 0x7f8e0015cbb0 2023-11-06T02:09:03.992 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.989+0000 7f8df7fff640 1 -- 172.21.15.49:0/3445722013 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df800aa30 con 0x7f8e0015cbb0 2023-11-06T02:09:03.992 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.989+0000 7f8e061e8640 1 -- 172.21.15.49:0/3445722013 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e00159050 con 0x7f8e0015cbb0 2023-11-06T02:09:03.992 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.989+0000 7f8e061e8640 1 -- 172.21.15.49:0/3445722013 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e00158bd0 con 0x7f8e0015cbb0 2023-11-06T02:09:03.993 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.989+0000 7f8df7fff640 1 -- 172.21.15.49:0/3445722013 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f8df800ab90 con 0x7f8e0015cbb0 2023-11-06T02:09:03.993 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.989+0000 7f8df7fff640 1 --2- 172.21.15.49:0/3445722013 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8de0065a60 0x7f8de0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:03.993 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.989+0000 7f8df7fff640 1 -- 172.21.15.49:0/3445722013 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8df8083070 con 0x7f8e0015cbb0 2023-11-06T02:09:03.993 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.989+0000 7f8dff7fe640 1 --2- 172.21.15.49:0/3445722013 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8de0065a60 0x7f8de0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:03.994 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.993+0000 7f8dff7fe640 1 --2- 172.21.15.49:0/3445722013 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8de0065a60 0x7f8de0067f20 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8df000b930 tx=0x7f8df0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:03.994 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:03.993+0000 7f8df7fff640 1 -- 172.21.15.49:0/3445722013 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f8e00158bd0 con 0x7f8e0015cbb0 2023-11-06T02:09:04.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.153+0000 7f8e061e8640 1 -- 172.21.15.49:0/3445722013 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f8e00158bd0 con 0x7f8e0015cbb0 2023-11-06T02:09:04.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.153+0000 7f8df7fff640 1 -- 172.21.15.49:0/3445722013 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f8e00158bd0 con 0x7f8e0015cbb0 2023-11-06T02:09:04.151 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 8 2023-11-06T02:09:04.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.157+0000 7f8e061e8640 1 -- 172.21.15.49:0/3445722013 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8de0065a60 msgr2=0x7f8de0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:04.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.157+0000 7f8e061e8640 1 --2- 172.21.15.49:0/3445722013 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8de0065a60 0x7f8de0067f20 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8df000b930 tx=0x7f8df0008040 comp rx=0 tx=0).stop 2023-11-06T02:09:04.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.157+0000 7f8e061e8640 1 -- 172.21.15.49:0/3445722013 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e0015cbb0 msgr2=0x7f8e0015f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:04.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.157+0000 7f8e061e8640 1 --2- 172.21.15.49:0/3445722013 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e0015cbb0 0x7f8e0015f710 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8df800c7a0 tx=0x7f8df800cc60 comp rx=0 tx=0).stop 2023-11-06T02:09:04.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.157+0000 7f8e061e8640 1 -- 172.21.15.49:0/3445722013 shutdown_connections 2023-11-06T02:09:04.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.157+0000 7f8e061e8640 1 --2- 172.21.15.49:0/3445722013 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8de0065a60 0x7f8de0067f20 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:04.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.157+0000 7f8e061e8640 1 --2- 172.21.15.49:0/3445722013 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e0015cbb0 0x7f8e0015f710 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:04.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.157+0000 7f8e061e8640 1 --2- 172.21.15.49:0/3445722013 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8e000d2fa0 0x7f8e00159850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:04.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.157+0000 7f8e061e8640 1 --2- 172.21.15.49:0/3445722013 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e000d24e0 0x7f8e00152f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:04.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.157+0000 7f8e061e8640 1 -- 172.21.15.49:0/3445722013 >> 172.21.15.49:0/3445722013 conn(0x7f8e000718a0 msgr2=0x7f8e000cae30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:04.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.157+0000 7f8e061e8640 1 -- 172.21.15.49:0/3445722013 shutdown_connections 2023-11-06T02:09:04.156 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.157+0000 7f8e061e8640 1 -- 172.21.15.49:0/3445722013 wait complete. 2023-11-06T02:09:04.169 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T02:09:04.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.289+0000 7f7e1a002640 1 Processor -- start 2023-11-06T02:09:04.286 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.289+0000 7f7e1a002640 1 -- start start 2023-11-06T02:09:04.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.289+0000 7f7e1a002640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f7e1415a880 0x7f7e140b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:04.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.289+0000 7f7e1a002640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e140ae360 con 0x7f7e1406d860 2023-11-06T02:09:04.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.289+0000 7f7e1a002640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e140ae640 con 0x7f7e14070160 2023-11-06T02:09:04.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.289+0000 7f7e1a002640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e140ae920 con 0x7f7e1415a880 2023-11-06T02:09:04.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.289+0000 7f7e137fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f7e1415a880 0x7f7e140b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:04.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.289+0000 7f7e13fff640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f7e1406d860 0x7f7e140af120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:48424/0 (socket says 172.21.15.49:48424) 2023-11-06T02:09:04.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.289+0000 7f7e13fff640 1 -- 172.21.15.49:0/3170020642 learned_addr learned my addr 172.21.15.49:0/3170020642 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:09:04.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.289+0000 7f7e12ffd640 1 -- 172.21.15.49:0/3170020642 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 116546442 0 0) 0x7f7e140ae360 con 0x7f7e1406d860 2023-11-06T02:09:04.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.289+0000 7f7e12ffd640 1 -- 172.21.15.49:0/3170020642 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e00003530 con 0x7f7e1406d860 2023-11-06T02:09:04.289 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.289+0000 7f7e137fe640 1 -- 172.21.15.49:0/3170020642 >> v1:172.21.15.150:6789/0 conn(0x7f7e14070160 legacy=0x7f7e1406d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:04.289 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.289+0000 7f7e137fe640 1 -- 172.21.15.49:0/3170020642 >> v1:172.21.15.49:6789/0 conn(0x7f7e1406d860 legacy=0x7f7e140af120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:04.289 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.289+0000 7f7e137fe640 1 -- 172.21.15.49:0/3170020642 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e140aec00 con 0x7f7e1415a880 2023-11-06T02:09:04.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.293+0000 7f7e137fe640 1 --2- 172.21.15.49:0/3170020642 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e1415a880 0x7f7e140b15c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7e040097c0 tx=0x7f7e0401c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=766445fb08cd6d3d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:04.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.293+0000 7f7e12ffd640 1 -- 172.21.15.49:0/3170020642 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e04004230 con 0x7f7e1415a880 2023-11-06T02:09:04.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.293+0000 7f7e12ffd640 1 -- 172.21.15.49:0/3170020642 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e040a7050 con 0x7f7e1415a880 2023-11-06T02:09:04.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.293+0000 7f7e12ffd640 1 -- 172.21.15.49:0/3170020642 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e04003620 con 0x7f7e1415a880 2023-11-06T02:09:04.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.293+0000 7f7e1a002640 1 -- 172.21.15.49:0/3170020642 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e1415a880 msgr2=0x7f7e140b15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:04.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.293+0000 7f7e1a002640 1 --2- 172.21.15.49:0/3170020642 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e1415a880 0x7f7e140b15c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7e040097c0 tx=0x7f7e0401c440 comp rx=0 tx=0).stop 2023-11-06T02:09:04.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.293+0000 7f7e1a002640 1 -- 172.21.15.49:0/3170020642 shutdown_connections 2023-11-06T02:09:04.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.293+0000 7f7e1a002640 1 --2- 172.21.15.49:0/3170020642 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e1415a880 0x7f7e140b15c0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:04.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.293+0000 7f7e1a002640 1 -- 172.21.15.49:0/3170020642 >> 172.21.15.49:0/3170020642 conn(0x7f7e1406a370 msgr2=0x7f7e140ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:04.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.293+0000 7f7e1a002640 1 -- 172.21.15.49:0/3170020642 shutdown_connections 2023-11-06T02:09:04.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.293+0000 7f7e1a002640 1 -- 172.21.15.49:0/3170020642 wait complete. 2023-11-06T02:09:04.294 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.293+0000 7f7e1a002640 1 Processor -- start 2023-11-06T02:09:04.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.293+0000 7f7e1a002640 1 -- start start 2023-11-06T02:09:04.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.293+0000 7f7e1a002640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e1406d860 0x7f7e140cc870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:04.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.293+0000 7f7e1a002640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7e14070160 0x7f7e14141e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:04.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.293+0000 7f7e1a002640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7e1415a880 0x7f7e14150930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:04.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.293+0000 7f7e1a002640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e140ccdb0 con 0x7f7e1406d860 2023-11-06T02:09:04.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.293+0000 7f7e1a002640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e140cb7d0 con 0x7f7e14070160 2023-11-06T02:09:04.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.293+0000 7f7e1a002640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e140cbab0 con 0x7f7e1415a880 2023-11-06T02:09:04.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.293+0000 7f7e137fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e1406d860 0x7f7e140cc870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:04.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.293+0000 7f7e137fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e1406d860 0x7f7e140cc870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47686/0 (socket says 172.21.15.49:47686) 2023-11-06T02:09:04.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.293+0000 7f7e137fe640 1 -- 172.21.15.49:0/540430457 learned_addr learned my addr 172.21.15.49:0/540430457 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:04.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.293+0000 7f7e13fff640 1 --2- 172.21.15.49:0/540430457 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7e1415a880 0x7f7e14150930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:04.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.293+0000 7f7e137fe640 1 -- 172.21.15.49:0/540430457 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7e1415a880 msgr2=0x7f7e14150930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:04.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.293+0000 7f7e0bfff640 1 --2- 172.21.15.49:0/540430457 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7e14070160 0x7f7e14141e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:04.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.293+0000 7f7e137fe640 1 --2- 172.21.15.49:0/540430457 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7e1415a880 0x7f7e14150930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:04.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.293+0000 7f7e137fe640 1 -- 172.21.15.49:0/540430457 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7e14070160 msgr2=0x7f7e14141e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:04.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.293+0000 7f7e137fe640 1 --2- 172.21.15.49:0/540430457 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7e14070160 0x7f7e14141e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:04.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.293+0000 7f7e137fe640 1 -- 172.21.15.49:0/540430457 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e04009420 con 0x7f7e1406d860 2023-11-06T02:09:04.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.293+0000 7f7e13fff640 1 --2- 172.21.15.49:0/540430457 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7e1415a880 0x7f7e14150930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:09:04.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.293+0000 7f7e137fe640 1 --2- 172.21.15.49:0/540430457 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e1406d860 0x7f7e140cc870 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7e04019050 tx=0x7f7e040191e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:04.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.293+0000 7f7e10ff9640 1 -- 172.21.15.49:0/540430457 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e040bc070 con 0x7f7e1406d860 2023-11-06T02:09:04.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.293+0000 7f7e10ff9640 1 -- 172.21.15.49:0/540430457 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e04003830 con 0x7f7e1406d860 2023-11-06T02:09:04.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.293+0000 7f7e10ff9640 1 -- 172.21.15.49:0/540430457 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e04003ba0 con 0x7f7e1406d860 2023-11-06T02:09:04.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.293+0000 7f7e1a002640 1 -- 172.21.15.49:0/540430457 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e140cbd90 con 0x7f7e1406d860 2023-11-06T02:09:04.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.293+0000 7f7e1a002640 1 -- 172.21.15.49:0/540430457 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e140cc350 con 0x7f7e1406d860 2023-11-06T02:09:04.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.297+0000 7f7e1a002640 1 -- 172.21.15.49:0/540430457 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e140cc540 con 0x7f7e1406d860 2023-11-06T02:09:04.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.297+0000 7f7e10ff9640 1 -- 172.21.15.49:0/540430457 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f7e040b2050 con 0x7f7e1406d860 2023-11-06T02:09:04.300 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.297+0000 7f7e10ff9640 1 --2- 172.21.15.49:0/540430457 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7df4065a60 0x7f7df4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:04.300 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.297+0000 7f7e10ff9640 1 -- 172.21.15.49:0/540430457 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7e0411cae0 con 0x7f7e1406d860 2023-11-06T02:09:04.300 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.297+0000 7f7e0bfff640 1 --2- 172.21.15.49:0/540430457 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7df4065a60 0x7f7df4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:04.301 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.297+0000 7f7e0bfff640 1 --2- 172.21.15.49:0/540430457 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7df4065a60 0x7f7df4067f20 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7e140cd330 tx=0x7f7dfc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:04.301 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.301+0000 7f7e10ff9640 1 -- 172.21.15.49:0/540430457 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f7e040ea3d0 con 0x7f7e1406d860 2023-11-06T02:09:04.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.453+0000 7f7e1a002640 1 -- 172.21.15.49:0/540430457 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7e14074040 con 0x7f7e1406d860 2023-11-06T02:09:04.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.453+0000 7f7e10ff9640 1 -- 172.21.15.49:0/540430457 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7e040ed9f0 con 0x7f7e1406d860 2023-11-06T02:09:04.451 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2023-11-06T02:09:04.453 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.453+0000 7f7e1a002640 1 -- 172.21.15.49:0/540430457 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7df4065a60 msgr2=0x7f7df4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:04.453 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.453+0000 7f7e1a002640 1 --2- 172.21.15.49:0/540430457 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7df4065a60 0x7f7df4067f20 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7e140cd330 tx=0x7f7dfc009040 comp rx=0 tx=0).stop 2023-11-06T02:09:04.453 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.453+0000 7f7e1a002640 1 -- 172.21.15.49:0/540430457 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e1406d860 msgr2=0x7f7e140cc870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:04.453 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.453+0000 7f7e1a002640 1 --2- 172.21.15.49:0/540430457 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e1406d860 0x7f7e140cc870 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7e04019050 tx=0x7f7e040191e0 comp rx=0 tx=0).stop 2023-11-06T02:09:04.454 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.453+0000 7f7e1a002640 1 -- 172.21.15.49:0/540430457 shutdown_connections 2023-11-06T02:09:04.454 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.453+0000 7f7e1a002640 1 --2- 172.21.15.49:0/540430457 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7df4065a60 0x7f7df4067f20 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:04.454 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.453+0000 7f7e1a002640 1 --2- 172.21.15.49:0/540430457 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7e1415a880 0x7f7e14150930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:04.454 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.453+0000 7f7e1a002640 1 --2- 172.21.15.49:0/540430457 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7e14070160 0x7f7e14141e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:04.455 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.453+0000 7f7e1a002640 1 --2- 172.21.15.49:0/540430457 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e1406d860 0x7f7e140cc870 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:04.455 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.453+0000 7f7e1a002640 1 -- 172.21.15.49:0/540430457 >> 172.21.15.49:0/540430457 conn(0x7f7e1406a370 msgr2=0x7f7e1414eb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:04.455 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.453+0000 7f7e1a002640 1 -- 172.21.15.49:0/540430457 shutdown_connections 2023-11-06T02:09:04.455 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.453+0000 7f7e1a002640 1 -- 172.21.15.49:0/540430457 wait complete. 2023-11-06T02:09:04.468 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T02:09:04.588 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.589+0000 7f098c803640 1 Processor -- start 2023-11-06T02:09:04.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.589+0000 7f098c803640 1 -- start start 2023-11-06T02:09:04.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.593+0000 7f098c803640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f09881595a0 0x7f098815dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:04.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.593+0000 7f098c803640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f098806d640 0x7f09880b0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:04.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.593+0000 7f098c803640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f098806c5a0 con 0x7f09880b0760 2023-11-06T02:09:04.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.593+0000 7f098c803640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f098815e250 con 0x7f09881595a0 2023-11-06T02:09:04.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.593+0000 7f098c803640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f098806c880 con 0x7f098806d640 2023-11-06T02:09:04.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.593+0000 7f0986d9d640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f09880b0760 0x7f09880c8e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:48438/0 (socket says 172.21.15.49:48438) 2023-11-06T02:09:04.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.593+0000 7f0986d9d640 1 -- 172.21.15.49:0/2478323573 learned_addr learned my addr 172.21.15.49:0/2478323573 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:09:04.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.593+0000 7f098559a640 1 -- 172.21.15.49:0/2478323573 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1488633197 0 0) 0x7f098806c5a0 con 0x7f09880b0760 2023-11-06T02:09:04.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.593+0000 7f098559a640 1 -- 172.21.15.49:0/2478323573 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0970005740 con 0x7f09880b0760 2023-11-06T02:09:04.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.593+0000 7f098659c640 1 --2- 172.21.15.49:0/2478323573 >> v2:172.21.15.150:3300/0 conn(0x7f09881595a0 0x7f098815dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:04.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.593+0000 7f0985d9b640 1 --2- 172.21.15.49:0/2478323573 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f098806d640 0x7f09880b0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:04.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.593+0000 7f098559a640 1 -- 172.21.15.49:0/2478323573 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3097433109 0 0) 0x7f0970005740 con 0x7f09880b0760 2023-11-06T02:09:04.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.593+0000 7f098559a640 1 -- 172.21.15.49:0/2478323573 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f098806d640 msgr2=0x7f09880b0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:04.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.593+0000 7f098559a640 1 --2- 172.21.15.49:0/2478323573 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f098806d640 0x7f09880b0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:04.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.593+0000 7f098559a640 1 -- 172.21.15.49:0/2478323573 >> v2:172.21.15.150:3300/0 conn(0x7f09881595a0 msgr2=0x7f098815dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:04.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.593+0000 7f098559a640 1 --2- 172.21.15.49:0/2478323573 >> v2:172.21.15.150:3300/0 conn(0x7f09881595a0 0x7f098815dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:04.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.593+0000 7f098559a640 1 -- 172.21.15.49:0/2478323573 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f098806cb60 con 0x7f09880b0760 2023-11-06T02:09:04.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.593+0000 7f098559a640 1 -- 172.21.15.49:0/2478323573 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7f0980001a00 con 0x7f09880b0760 2023-11-06T02:09:04.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.593+0000 7f098559a640 1 -- 172.21.15.49:0/2478323573 >> v1:172.21.15.49:6789/0 conn(0x7f09880b0760 legacy=0x7f09880c8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:04.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.593+0000 7f098559a640 1 --2- 172.21.15.49:0/2478323573 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f098806d640 0x7f0970005b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:04.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.593+0000 7f098559a640 1 --2- 172.21.15.49:0/2478323573 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0970006110 0x7f09700069e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:04.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.593+0000 7f098559a640 1 --2- 172.21.15.49:0/2478323573 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0970006f20 0x7f09700073a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:04.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.593+0000 7f098559a640 1 -- 172.21.15.49:0/2478323573 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f098815e250 con 0x7f0970006f20 2023-11-06T02:09:04.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.593+0000 7f098559a640 1 -- 172.21.15.49:0/2478323573 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f098806c880 con 0x7f0970006110 2023-11-06T02:09:04.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.593+0000 7f098559a640 1 -- 172.21.15.49:0/2478323573 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f097000dba0 con 0x7f098806d640 2023-11-06T02:09:04.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.593+0000 7f0986d9d640 1 --2- 172.21.15.49:0/2478323573 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0970006f20 0x7f09700073a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:04.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.593+0000 7f098659c640 1 --2- 172.21.15.49:0/2478323573 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f098806d640 0x7f0970005b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:04.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.593+0000 7f0985d9b640 1 --2- 172.21.15.49:0/2478323573 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0970006110 0x7f09700069e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:04.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.593+0000 7f0986d9d640 1 -- 172.21.15.49:0/2478323573 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f098806d640 msgr2=0x7f0970005b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:04.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.593+0000 7f0986d9d640 1 --2- 172.21.15.49:0/2478323573 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f098806d640 0x7f0970005b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:04.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.593+0000 7f0986d9d640 1 -- 172.21.15.49:0/2478323573 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0970006110 msgr2=0x7f09700069e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:04.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.593+0000 7f0986d9d640 1 --2- 172.21.15.49:0/2478323573 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0970006110 0x7f09700069e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:04.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.593+0000 7f0986d9d640 1 -- 172.21.15.49:0/2478323573 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0970005740 con 0x7f0970006f20 2023-11-06T02:09:04.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.597+0000 7f0986d9d640 1 --2- 172.21.15.49:0/2478323573 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0970006f20 0x7f09700073a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f098000a8d0 tx=0x7f098001d380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a128bbee6e17b62 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:04.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.601+0000 7f098559a640 1 -- 172.21.15.49:0/2478323573 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09800ad630 con 0x7f0970006f20 2023-11-06T02:09:04.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.601+0000 7f098559a640 1 -- 172.21.15.49:0/2478323573 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09800a8070 con 0x7f0970006f20 2023-11-06T02:09:04.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.601+0000 7f098559a640 1 -- 172.21.15.49:0/2478323573 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09800ad9d0 con 0x7f0970006f20 2023-11-06T02:09:04.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.601+0000 7f098c803640 1 -- 172.21.15.49:0/2478323573 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0970006f20 msgr2=0x7f09700073a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:04.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.601+0000 7f098c803640 1 --2- 172.21.15.49:0/2478323573 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0970006f20 0x7f09700073a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f098000a8d0 tx=0x7f098001d380 comp rx=0 tx=0).stop 2023-11-06T02:09:04.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.601+0000 7f0986d9d640 1 -- 172.21.15.49:0/2478323573 reap_dead start 2023-11-06T02:09:04.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.601+0000 7f098c803640 1 -- 172.21.15.49:0/2478323573 shutdown_connections 2023-11-06T02:09:04.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.601+0000 7f098c803640 1 -- 172.21.15.49:0/2478323573 >> 172.21.15.49:0/2478323573 conn(0x7f098806a370 msgr2=0x7f09880ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:04.604 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.601+0000 7f098c803640 1 -- 172.21.15.49:0/2478323573 shutdown_connections 2023-11-06T02:09:04.604 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.601+0000 7f098c803640 1 -- 172.21.15.49:0/2478323573 wait complete. 2023-11-06T02:09:04.604 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.601+0000 7f098c803640 1 Processor -- start 2023-11-06T02:09:04.604 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.601+0000 7f098c803640 1 -- start start 2023-11-06T02:09:04.604 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.601+0000 7f098c803640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09881f59f0 0x7f09880b1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:04.604 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.601+0000 7f098c803640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09881f5dd0 0x7f09881f2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:04.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.601+0000 7f098c803640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f09881f2850 0x7f09881f2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:04.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.601+0000 7f098c803640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09880b2e50 con 0x7f09881f59f0 2023-11-06T02:09:04.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.601+0000 7f098c803640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09880b1db0 con 0x7f09881f5dd0 2023-11-06T02:09:04.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.601+0000 7f098c803640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09880b2090 con 0x7f09881f2850 2023-11-06T02:09:04.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.601+0000 7f098659c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09881f59f0 0x7f09880b1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:04.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.601+0000 7f098659c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09881f59f0 0x7f09880b1460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47704/0 (socket says 172.21.15.49:47704) 2023-11-06T02:09:04.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.601+0000 7f098659c640 1 -- 172.21.15.49:0/1220957001 learned_addr learned my addr 172.21.15.49:0/1220957001 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:04.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.601+0000 7f0986d9d640 1 --2- 172.21.15.49:0/1220957001 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f09881f2850 0x7f09881f2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:04.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.601+0000 7f0985d9b640 1 --2- 172.21.15.49:0/1220957001 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09881f5dd0 0x7f09881f2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:04.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.601+0000 7f098659c640 1 -- 172.21.15.49:0/1220957001 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f09881f2850 msgr2=0x7f09881f2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:04.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.601+0000 7f098659c640 1 --2- 172.21.15.49:0/1220957001 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f09881f2850 0x7f09881f2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:04.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.601+0000 7f098659c640 1 -- 172.21.15.49:0/1220957001 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09881f5dd0 msgr2=0x7f09881f2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:04.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.601+0000 7f098659c640 1 --2- 172.21.15.49:0/1220957001 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09881f5dd0 0x7f09881f2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:04.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.601+0000 7f098659c640 1 -- 172.21.15.49:0/1220957001 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09880b2370 con 0x7f09881f59f0 2023-11-06T02:09:04.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.601+0000 7f0986d9d640 1 --2- 172.21.15.49:0/1220957001 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f09881f2850 0x7f09881f2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:09:04.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.601+0000 7f0985d9b640 1 --2- 172.21.15.49:0/1220957001 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09881f5dd0 0x7f09881f2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:09:04.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.601+0000 7f098659c640 1 --2- 172.21.15.49:0/1220957001 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09881f59f0 0x7f09880b1460 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0978011aa0 tx=0x7f0978011f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:04.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.601+0000 7f0976ffd640 1 -- 172.21.15.49:0/1220957001 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f097800fb80 con 0x7f09881f59f0 2023-11-06T02:09:04.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.601+0000 7f0976ffd640 1 -- 172.21.15.49:0/1220957001 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f097800fce0 con 0x7f09881f59f0 2023-11-06T02:09:04.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.601+0000 7f098c803640 1 -- 172.21.15.49:0/1220957001 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09880b2650 con 0x7f09881f59f0 2023-11-06T02:09:04.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.601+0000 7f0976ffd640 1 -- 172.21.15.49:0/1220957001 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0978014680 con 0x7f09881f59f0 2023-11-06T02:09:04.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.601+0000 7f098c803640 1 -- 172.21.15.49:0/1220957001 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09880b19a0 con 0x7f09881f59f0 2023-11-06T02:09:04.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.605+0000 7f098c803640 1 -- 172.21.15.49:0/1220957001 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0988074040 con 0x7f09881f59f0 2023-11-06T02:09:04.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.605+0000 7f0976ffd640 1 -- 172.21.15.49:0/1220957001 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f09780147e0 con 0x7f09881f59f0 2023-11-06T02:09:04.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.605+0000 7f0976ffd640 1 --2- 172.21.15.49:0/1220957001 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0968065a60 0x7f0968067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:04.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.605+0000 7f0976ffd640 1 -- 172.21.15.49:0/1220957001 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0978083c90 con 0x7f09881f59f0 2023-11-06T02:09:04.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.605+0000 7f0985d9b640 1 --2- 172.21.15.49:0/1220957001 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0968065a60 0x7f0968067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:04.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.605+0000 7f0985d9b640 1 --2- 172.21.15.49:0/1220957001 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0968065a60 0x7f0968067f20 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f09880b31a0 tx=0x7f097c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:04.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.609+0000 7f0976ffd640 1 -- 172.21.15.49:0/1220957001 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f0978051470 con 0x7f09881f59f0 2023-11-06T02:09:04.764 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.765+0000 7f098c803640 1 -- 172.21.15.49:0/1220957001 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f09880cdad0 con 0x7f09881f59f0 2023-11-06T02:09:04.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.769+0000 7f0976ffd640 1 -- 172.21.15.49:0/1220957001 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0978054a90 con 0x7f09881f59f0 2023-11-06T02:09:04.766 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2023-11-06T02:09:04.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.769+0000 7f098c803640 1 -- 172.21.15.49:0/1220957001 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0968065a60 msgr2=0x7f0968067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:04.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.769+0000 7f098c803640 1 --2- 172.21.15.49:0/1220957001 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0968065a60 0x7f0968067f20 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f09880b31a0 tx=0x7f097c008040 comp rx=0 tx=0).stop 2023-11-06T02:09:04.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.769+0000 7f098c803640 1 -- 172.21.15.49:0/1220957001 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09881f59f0 msgr2=0x7f09880b1460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:04.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.769+0000 7f098c803640 1 --2- 172.21.15.49:0/1220957001 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09881f59f0 0x7f09880b1460 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0978011aa0 tx=0x7f0978011f60 comp rx=0 tx=0).stop 2023-11-06T02:09:04.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.769+0000 7f098c803640 1 -- 172.21.15.49:0/1220957001 shutdown_connections 2023-11-06T02:09:04.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.769+0000 7f098c803640 1 --2- 172.21.15.49:0/1220957001 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0968065a60 0x7f0968067f20 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:04.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.769+0000 7f098c803640 1 --2- 172.21.15.49:0/1220957001 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f09881f2850 0x7f09881f2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:04.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.769+0000 7f098c803640 1 --2- 172.21.15.49:0/1220957001 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09881f5dd0 0x7f09881f2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:04.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.769+0000 7f098c803640 1 --2- 172.21.15.49:0/1220957001 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09881f59f0 0x7f09880b1460 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:04.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.769+0000 7f098c803640 1 -- 172.21.15.49:0/1220957001 >> 172.21.15.49:0/1220957001 conn(0x7f098806a370 msgr2=0x7f09880cc550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:04.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.769+0000 7f098c803640 1 -- 172.21.15.49:0/1220957001 shutdown_connections 2023-11-06T02:09:04.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.769+0000 7f098c803640 1 -- 172.21.15.49:0/1220957001 wait complete. 2023-11-06T02:09:04.784 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T02:09:04.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.905+0000 7f3edbc1d640 1 Processor -- start 2023-11-06T02:09:04.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.905+0000 7f3edbc1d640 1 -- start start 2023-11-06T02:09:04.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.905+0000 7f3edbc1d640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f3ed40cda70 0x7f3ed4161650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:04.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.905+0000 7f3edbc1d640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ed40d17c0 con 0x7f3ed40b1f90 2023-11-06T02:09:04.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.905+0000 7f3edbc1d640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ed40ce6c0 con 0x7f3ed40714a0 2023-11-06T02:09:04.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.905+0000 7f3edbc1d640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ed40cdf20 con 0x7f3ed40cda70 2023-11-06T02:09:04.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.905+0000 7f3ed99b9640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f3ed40b1f90 0x7f3ed40d1c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:48448/0 (socket says 172.21.15.49:48448) 2023-11-06T02:09:04.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.905+0000 7f3ed99b9640 1 -- 172.21.15.49:0/3792470979 learned_addr learned my addr 172.21.15.49:0/3792470979 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:09:04.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.905+0000 7f3ed91b8640 1 --2- 172.21.15.49:0/3792470979 >> v2:172.21.15.150:3300/0 conn(0x7f3ed40cda70 0x7f3ed4161650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:04.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.905+0000 7f3ed89b7640 1 -- 172.21.15.49:0/3792470979 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2539070844 0 0) 0x7f3ed40d17c0 con 0x7f3ed40b1f90 2023-11-06T02:09:04.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.905+0000 7f3ed89b7640 1 -- 172.21.15.49:0/3792470979 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ec0003530 con 0x7f3ed40b1f90 2023-11-06T02:09:04.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.905+0000 7f3ed89b7640 1 -- 172.21.15.49:0/3792470979 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 668520780 0 0) 0x7f3ec0003530 con 0x7f3ed40b1f90 2023-11-06T02:09:04.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.905+0000 7f3ed89b7640 1 -- 172.21.15.49:0/3792470979 >> v2:172.21.15.150:3300/0 conn(0x7f3ed40cda70 msgr2=0x7f3ed4161650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:04.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.905+0000 7f3ed89b7640 1 --2- 172.21.15.49:0/3792470979 >> v2:172.21.15.150:3300/0 conn(0x7f3ed40cda70 0x7f3ed4161650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:04.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.905+0000 7f3ed89b7640 1 -- 172.21.15.49:0/3792470979 >> v1:172.21.15.150:6789/0 conn(0x7f3ed40714a0 legacy=0x7f3ed40718b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:04.908 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.905+0000 7f3ed89b7640 1 -- 172.21.15.49:0/3792470979 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ed40d2360 con 0x7f3ed40b1f90 2023-11-06T02:09:04.908 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.905+0000 7f3ed89b7640 1 -- 172.21.15.49:0/3792470979 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7f3ec4001680 con 0x7f3ed40b1f90 2023-11-06T02:09:04.908 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.905+0000 7f3ed89b7640 1 -- 172.21.15.49:0/3792470979 >> v1:172.21.15.49:6789/0 conn(0x7f3ed40b1f90 legacy=0x7f3ed40d1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:04.908 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.905+0000 7f3ed89b7640 1 --2- 172.21.15.49:0/3792470979 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3ec0003980 0x7f3ec0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:04.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.905+0000 7f3ed89b7640 1 --2- 172.21.15.49:0/3792470979 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ec00047d0 0x7f3ec0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:04.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.905+0000 7f3ed89b7640 1 --2- 172.21.15.49:0/3792470979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ec0005190 0x7f3ec000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:04.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.905+0000 7f3ed89b7640 1 -- 172.21.15.49:0/3792470979 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed40cdf20 con 0x7f3ec00047d0 2023-11-06T02:09:04.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.905+0000 7f3ed89b7640 1 -- 172.21.15.49:0/3792470979 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec000c030 con 0x7f3ec0005190 2023-11-06T02:09:04.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.905+0000 7f3ed89b7640 1 -- 172.21.15.49:0/3792470979 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec000c1f0 con 0x7f3ec0003980 2023-11-06T02:09:04.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.905+0000 7f3ed91b8640 1 --2- 172.21.15.49:0/3792470979 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ec00047d0 0x7f3ec0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:04.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.905+0000 7f3eda1ba640 1 --2- 172.21.15.49:0/3792470979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ec0005190 0x7f3ec000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:04.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.905+0000 7f3ed99b9640 1 --2- 172.21.15.49:0/3792470979 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3ec0003980 0x7f3ec0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:04.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.905+0000 7f3ed91b8640 1 -- 172.21.15.49:0/3792470979 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3ec0003980 msgr2=0x7f3ec0003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:04.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.905+0000 7f3ed91b8640 1 --2- 172.21.15.49:0/3792470979 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3ec0003980 0x7f3ec0003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:04.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.905+0000 7f3ed91b8640 1 -- 172.21.15.49:0/3792470979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ec0005190 msgr2=0x7f3ec000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:04.911 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.905+0000 7f3ed91b8640 1 --2- 172.21.15.49:0/3792470979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ec0005190 0x7f3ec000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:04.911 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.905+0000 7f3eda1ba640 1 -- 172.21.15.49:0/3792470979 reap_dead start 2023-11-06T02:09:04.911 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.905+0000 7f3ed91b8640 1 -- 172.21.15.49:0/3792470979 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ec0003530 con 0x7f3ec00047d0 2023-11-06T02:09:04.911 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.905+0000 7f3ed99b9640 1 --2- 172.21.15.49:0/3792470979 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3ec0003980 0x7f3ec0003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:09:04.911 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.909+0000 7f3ed91b8640 1 --2- 172.21.15.49:0/3792470979 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ec00047d0 0x7f3ec0004c50 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f3ed0009ae0 tx=0x7f3ed001ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbaab6311a98567 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:04.912 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.909+0000 7f3ed89b7640 1 -- 172.21.15.49:0/3792470979 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ed00a5640 con 0x7f3ec00047d0 2023-11-06T02:09:04.912 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.909+0000 7f3ed89b7640 1 -- 172.21.15.49:0/3792470979 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ec4001d80 con 0x7f3ec00047d0 2023-11-06T02:09:04.912 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.909+0000 7f3ed89b7640 1 -- 172.21.15.49:0/3792470979 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ed00a59e0 con 0x7f3ec00047d0 2023-11-06T02:09:04.912 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.909+0000 7f3edbc1d640 1 -- 172.21.15.49:0/3792470979 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ec00047d0 msgr2=0x7f3ec0004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:04.912 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.909+0000 7f3edbc1d640 1 --2- 172.21.15.49:0/3792470979 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ec00047d0 0x7f3ec0004c50 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f3ed0009ae0 tx=0x7f3ed001ca50 comp rx=0 tx=0).stop 2023-11-06T02:09:04.913 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.909+0000 7f3edbc1d640 1 -- 172.21.15.49:0/3792470979 shutdown_connections 2023-11-06T02:09:04.913 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.909+0000 7f3edbc1d640 1 --2- 172.21.15.49:0/3792470979 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ec00047d0 0x7f3ec0004c50 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:04.913 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.909+0000 7f3edbc1d640 1 -- 172.21.15.49:0/3792470979 >> 172.21.15.49:0/3792470979 conn(0x7f3ed40d2aa0 msgr2=0x7f3ed40d2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:04.913 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.909+0000 7f3edbc1d640 1 -- 172.21.15.49:0/3792470979 shutdown_connections 2023-11-06T02:09:04.913 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.909+0000 7f3edbc1d640 1 -- 172.21.15.49:0/3792470979 wait complete. 2023-11-06T02:09:04.919 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.909+0000 7f3edbc1d640 1 Processor -- start 2023-11-06T02:09:04.919 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.909+0000 7f3edbc1d640 1 -- start start 2023-11-06T02:09:04.919 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.913+0000 7f3edbc1d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ec00047d0 0x7f3ed41f2230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:04.919 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.913+0000 7f3edbc1d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ed41f3390 0x7f3ed40cbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:04.919 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.913+0000 7f3edbc1d640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3ed40cc1f0 0x7f3ed415d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:04.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.913+0000 7f3edbc1d640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed41f2f70 con 0x7f3ed41f3390 2023-11-06T02:09:04.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.913+0000 7f3edbc1d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed40cc670 con 0x7f3ec00047d0 2023-11-06T02:09:04.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.913+0000 7f3edbc1d640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed415e730 con 0x7f3ed40cc1f0 2023-11-06T02:09:04.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.913+0000 7f3ed91b8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ed41f3390 0x7f3ed40cbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:04.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.913+0000 7f3ed91b8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ed41f3390 0x7f3ed40cbcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47716/0 (socket says 172.21.15.49:47716) 2023-11-06T02:09:04.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.913+0000 7f3ed91b8640 1 -- 172.21.15.49:0/4130425034 learned_addr learned my addr 172.21.15.49:0/4130425034 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:04.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.913+0000 7f3ed91b8640 1 -- 172.21.15.49:0/4130425034 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3ed40cc1f0 msgr2=0x7f3ed415d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:04.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.913+0000 7f3ed91b8640 1 --2- 172.21.15.49:0/4130425034 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3ed40cc1f0 0x7f3ed415d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:04.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.913+0000 7f3ed91b8640 1 -- 172.21.15.49:0/4130425034 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ec00047d0 msgr2=0x7f3ed41f2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:04.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.913+0000 7f3ed91b8640 1 --2- 172.21.15.49:0/4130425034 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ec00047d0 0x7f3ed41f2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:04.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.913+0000 7f3ed91b8640 1 -- 172.21.15.49:0/4130425034 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ed415e450 con 0x7f3ed41f3390 2023-11-06T02:09:04.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.913+0000 7f3ed91b8640 1 --2- 172.21.15.49:0/4130425034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ed41f3390 0x7f3ed40cbcb0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f3ed001c880 tx=0x7f3ed00197f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:04.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.913+0000 7f3eca7fc640 1 -- 172.21.15.49:0/4130425034 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ed000f0c0 con 0x7f3ed41f3390 2023-11-06T02:09:04.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.913+0000 7f3eca7fc640 1 -- 172.21.15.49:0/4130425034 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ed0004020 con 0x7f3ed41f3390 2023-11-06T02:09:04.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.913+0000 7f3edbc1d640 1 -- 172.21.15.49:0/4130425034 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ed41f28b0 con 0x7f3ed41f3390 2023-11-06T02:09:04.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.913+0000 7f3eca7fc640 1 -- 172.21.15.49:0/4130425034 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ed00043b0 con 0x7f3ed41f3390 2023-11-06T02:09:04.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.913+0000 7f3edbc1d640 1 -- 172.21.15.49:0/4130425034 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ed41f3810 con 0x7f3ed41f3390 2023-11-06T02:09:04.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.913+0000 7f3edbc1d640 1 -- 172.21.15.49:0/4130425034 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ed4074040 con 0x7f3ed41f3390 2023-11-06T02:09:04.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.913+0000 7f3eca7fc640 1 -- 172.21.15.49:0/4130425034 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f3ed00b4070 con 0x7f3ed41f3390 2023-11-06T02:09:04.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.913+0000 7f3eca7fc640 1 --2- 172.21.15.49:0/4130425034 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3eb8065a90 0x7f3eb8067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:04.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.913+0000 7f3eca7fc640 1 -- 172.21.15.49:0/4130425034 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3ed011d540 con 0x7f3ed41f3390 2023-11-06T02:09:04.924 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.913+0000 7f3ed99b9640 1 --2- 172.21.15.49:0/4130425034 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3eb8065a90 0x7f3eb8067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:04.924 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.917+0000 7f3ed99b9640 1 --2- 172.21.15.49:0/4130425034 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3eb8065a90 0x7f3eb8067f50 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3ec4004880 tx=0x7f3ec4001160 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:04.924 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:04.917+0000 7f3eca7fc640 1 -- 172.21.15.49:0/4130425034 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f3ed00eae30 con 0x7f3ed41f3390 2023-11-06T02:09:05.070 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.073+0000 7f3edbc1d640 1 -- 172.21.15.49:0/4130425034 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3ed415c9c0 con 0x7f3ed41f3390 2023-11-06T02:09:05.073 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.077+0000 7f3eca7fc640 1 -- 172.21.15.49:0/4130425034 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+17811 (secure 0 0 0) 0x7f3ed00ee450 con 0x7f3ed41f3390 2023-11-06T02:09:05.073 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:09:05.073 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":23,"fsid":"1a8b035c-6ba4-499a-b876-aea47fd9c0d4","created":"2023-11-06T02:08:06.440857+0000","modified":"2023-11-06T02:08:31.594964+0000","last_up_change":"2023-11-06T02:08:17.982372+0000","last_in_change":"2023-11-06T02:08:08.906631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T02:08:19.252222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T02:08:24.776211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-06T02:08:28.936178+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-06T02:08:29.563468+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ad55af8-0571-4e6a-8722-b5d429b7496d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6809","nonce":38116},{"type":"v1","addr":"172.21.15.49:6811","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6813","nonce":38116},{"type":"v1","addr":"172.21.15.49:6815","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6821","nonce":38116},{"type":"v1","addr":"172.21.15.49:6823","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6817","nonce":38116},{"type":"v1","addr":"172.21.15.49:6819","nonce":38116}]},"public_addr":"172.21.15.49:6811/38116","cluster_addr":"172.21.15.49:6815/38116","heartbeat_back_addr":"172.21.15.49:6823/38116","heartbeat_front_addr":"172.21.15.49:6819/38116","state":["exists","up"]},{"osd":1,"uuid":"c8b31f85-a757-43ed-9419-351acc46611b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":38113},{"type":"v1","addr":"172.21.15.49:6810","nonce":38113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":38113},{"type":"v1","addr":"172.21.15.49:6814","nonce":38113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":38113},{"type":"v1","addr":"172.21.15.49:6822","nonce":38113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":38113},{"type":"v1","addr":"172.21.15.49:6818","nonce":38113}]},"public_addr":"172.21.15.49:6810/38113","cluster_addr":"172.21.15.49:6814/38113","heartbeat_back_addr":"172.21.15.49:6822/38113","heartbeat_front_addr":"172.21.15.49:6818/38113","state":["exists","up"]},{"osd":2,"uuid":"555b3bf3-0a70-4b91-8efd-e4204b5fc26e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":38114},{"type":"v1","addr":"172.21.15.49:6801","nonce":38114}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":38114},{"type":"v1","addr":"172.21.15.49:6803","nonce":38114}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":38114},{"type":"v1","addr":"172.21.15.49:6807","nonce":38114}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":38114},{"type":"v1","addr":"172.21.15.49:6805","nonce":38114}]},"public_addr":"172.21.15.49:6801/38114","cluster_addr":"172.21.15.49:6803/38114","heartbeat_back_addr":"172.21.15.49:6807/38114","heartbeat_front_addr":"172.21.15.49:6805/38114","state":["exists","up"]},{"osd":3,"uuid":"8d4e5d77-03ef-4b9b-bfcb-5ae93a46795c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":38115},{"type":"v1","addr":"172.21.15.49:6825","nonce":38115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":38115},{"type":"v1","addr":"172.21.15.49:6827","nonce":38115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":38115},{"type":"v1","addr":"172.21.15.49:6831","nonce":38115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":38115},{"type":"v1","addr":"172.21.15.49:6829","nonce":38115}]},"public_addr":"172.21.15.49:6825/38115","cluster_addr":"172.21.15.49:6827/38115","heartbeat_back_addr":"172.21.15.49:6831/38115","heartbeat_front_addr":"172.21.15.49:6829/38115","state":["exists","up"]},{"osd":4,"uuid":"dba03ad4-68a3-4cf3-9218-ac611f02eacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":38417},{"type":"v1","addr":"172.21.15.150:6809","nonce":38417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":38417},{"type":"v1","addr":"172.21.15.150:6811","nonce":38417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":38417},{"type":"v1","addr":"172.21.15.150:6815","nonce":38417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":38417},{"type":"v1","addr":"172.21.15.150:6813","nonce":38417}]},"public_addr":"172.21.15.150:6809/38417","cluster_addr":"172.21.15.150:6811/38417","heartbeat_back_addr":"172.21.15.150:6815/38417","heartbeat_front_addr":"172.21.15.150:6813/38417","state":["exists","up"]},{"osd":5,"uuid":"0df9c303-c255-4640-8e91-01f1c3efe18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":38418},{"type":"v1","addr":"172.21.15.150:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":38418},{"type":"v1","addr":"172.21.15.150:6803","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":38418},{"type":"v1","addr":"172.21.15.150:6807","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":38418},{"type":"v1","addr":"172.21.15.150:6805","nonce":38418}]},"public_addr":"172.21.15.150:6801/38418","cluster_addr":"172.21.15.150:6803/38418","heartbeat_back_addr":"172.21.15.150:6807/38418","heartbeat_front_addr":"172.21.15.150:6805/38418","state":["exists","up"]},{"osd":6,"uuid":"e59fa189-4b70-4c06-9722-c02b90828bf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":38428},{"type":"v1","addr":"172.21.15.150:6825","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":38428},{"type":"v1","addr":"172.21.15.150:6827","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":38428},{"type":"v1","addr":"172.21.15.150:6831","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":38428},{"type":"v1","addr":"172.21.15.150:6829","nonce":38428}]},"public_addr":"172.21.15.150:6825/38428","cluster_addr":"172.21.15.150:6827/38428","heartbeat_back_addr":"172.21.15.150:6831/38428","heartbeat_front_addr":"172.21.15.150:6829/38428","state":["exists","up"]},{"osd":7,"uuid":"a64f57e1-ef98-46bd-b304-385759e95898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":38421},{"type":"v1","addr":"172.21.15.150:6817","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":38421},{"type":"v1","addr":"172.21.15.150:6819","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":38421},{"type":"v1","addr":"172.21.15.150:6823","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":38421},{"type":"v1","addr":"172.21.15.150:6821","nonce":38421}]},"public_addr":"172.21.15.150:6817/38421","cluster_addr":"172.21.15.150:6819/38421","heartbeat_back_addr":"172.21.15.150:6823/38421","heartbeat_front_addr":"172.21.15.150:6821/38421","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.438552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.413280+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.296466+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.424966+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.276953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.230984+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.410235+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.480084+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T02:09:05.075 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.077+0000 7f3edbc1d640 1 -- 172.21.15.49:0/4130425034 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3eb8065a90 msgr2=0x7f3eb8067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:05.075 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.077+0000 7f3edbc1d640 1 --2- 172.21.15.49:0/4130425034 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3eb8065a90 0x7f3eb8067f50 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3ec4004880 tx=0x7f3ec4001160 comp rx=0 tx=0).stop 2023-11-06T02:09:05.075 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.077+0000 7f3edbc1d640 1 -- 172.21.15.49:0/4130425034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ed41f3390 msgr2=0x7f3ed40cbcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:05.075 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.077+0000 7f3edbc1d640 1 --2- 172.21.15.49:0/4130425034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ed41f3390 0x7f3ed40cbcb0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f3ed001c880 tx=0x7f3ed00197f0 comp rx=0 tx=0).stop 2023-11-06T02:09:05.076 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.077+0000 7f3edbc1d640 1 -- 172.21.15.49:0/4130425034 shutdown_connections 2023-11-06T02:09:05.076 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.077+0000 7f3edbc1d640 1 --2- 172.21.15.49:0/4130425034 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3eb8065a90 0x7f3eb8067f50 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:05.076 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.077+0000 7f3edbc1d640 1 --2- 172.21.15.49:0/4130425034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ed41f3390 0x7f3ed40cbcb0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:05.076 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.077+0000 7f3edbc1d640 1 --2- 172.21.15.49:0/4130425034 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3ed40cc1f0 0x7f3ed415d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:05.076 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.077+0000 7f3edbc1d640 1 --2- 172.21.15.49:0/4130425034 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ec00047d0 0x7f3ed41f2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:05.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.077+0000 7f3edbc1d640 1 -- 172.21.15.49:0/4130425034 >> 172.21.15.49:0/4130425034 conn(0x7f3ed40d2aa0 msgr2=0x7f3ed40c6ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:05.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.077+0000 7f3edbc1d640 1 -- 172.21.15.49:0/4130425034 shutdown_connections 2023-11-06T02:09:05.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.077+0000 7f3edbc1d640 1 -- 172.21.15.49:0/4130425034 wait complete. 2023-11-06T02:09:05.091 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T02:08:19.252222+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T02:08:24.776211+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T02:08:28.936178+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T02:08:29.563468+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T02:09:05.092 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T02:09:05.208 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.209+0000 7f7c45359640 1 Processor -- start 2023-11-06T02:09:05.208 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.209+0000 7f7c45359640 1 -- start start 2023-11-06T02:09:05.209 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.213+0000 7f7c45359640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7c4015a990 0x7f7c4015dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:05.210 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.213+0000 7f7c45359640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f7c400b0220 0x7f7c400b0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:05.210 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.213+0000 7f7c45359640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c4006c5a0 con 0x7f7c401595a0 2023-11-06T02:09:05.210 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.213+0000 7f7c45359640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c4015e250 con 0x7f7c400b0220 2023-11-06T02:09:05.210 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.213+0000 7f7c45359640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c4006c880 con 0x7f7c4015a990 2023-11-06T02:09:05.210 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.213+0000 7f7c3f7fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f7c400b0220 0x7f7c400b0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:05.211 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.213+0000 7f7c3f7fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f7c400b0220 0x7f7c400b0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47724/0 (socket says 172.21.15.49:47724) 2023-11-06T02:09:05.211 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.213+0000 7f7c3f7fe640 1 -- 172.21.15.49:0/3486062206 learned_addr learned my addr 172.21.15.49:0/3486062206 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:05.211 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.213+0000 7f7c3e7fc640 1 --2- 172.21.15.49:0/3486062206 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7c4015a990 0x7f7c4015dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:05.211 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.213+0000 7f7c3f7fe640 1 -- 172.21.15.49:0/3486062206 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7c4015a990 msgr2=0x7f7c4015dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:05.211 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.213+0000 7f7c3f7fe640 1 --2- 172.21.15.49:0/3486062206 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7c4015a990 0x7f7c4015dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:05.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.213+0000 7f7c3f7fe640 1 -- 172.21.15.49:0/3486062206 >> v1:172.21.15.150:6789/0 conn(0x7f7c401595a0 legacy=0x7f7c4006d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:05.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.213+0000 7f7c3f7fe640 1 -- 172.21.15.49:0/3486062206 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c4006cb60 con 0x7f7c400b0220 2023-11-06T02:09:05.213 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.213+0000 7f7c3f7fe640 1 --2- 172.21.15.49:0/3486062206 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c400b0220 0x7f7c400b0670 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7c38009e50 tx=0x7f7c3801c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=463732040a8dcfc7 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:05.213 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.213+0000 7f7c3dffb640 1 -- 172.21.15.49:0/3486062206 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c380a58a0 con 0x7f7c400b0220 2023-11-06T02:09:05.214 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.213+0000 7f7c3dffb640 1 -- 172.21.15.49:0/3486062206 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c380a5a00 con 0x7f7c400b0220 2023-11-06T02:09:05.214 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.213+0000 7f7c3dffb640 1 -- 172.21.15.49:0/3486062206 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c380a5d90 con 0x7f7c400b0220 2023-11-06T02:09:05.214 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.213+0000 7f7c45359640 1 -- 172.21.15.49:0/3486062206 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c400b0220 msgr2=0x7f7c400b0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:05.214 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.213+0000 7f7c45359640 1 --2- 172.21.15.49:0/3486062206 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c400b0220 0x7f7c400b0670 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7c38009e50 tx=0x7f7c3801c9e0 comp rx=0 tx=0).stop 2023-11-06T02:09:05.214 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.213+0000 7f7c45359640 1 -- 172.21.15.49:0/3486062206 shutdown_connections 2023-11-06T02:09:05.215 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.213+0000 7f7c45359640 1 --2- 172.21.15.49:0/3486062206 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7c4015a990 0x7f7c4015dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:05.215 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.213+0000 7f7c45359640 1 --2- 172.21.15.49:0/3486062206 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c400b0220 0x7f7c400b0670 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:05.215 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.213+0000 7f7c45359640 1 -- 172.21.15.49:0/3486062206 >> 172.21.15.49:0/3486062206 conn(0x7f7c4006a370 msgr2=0x7f7c400ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:05.215 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.213+0000 7f7c45359640 1 -- 172.21.15.49:0/3486062206 shutdown_connections 2023-11-06T02:09:05.215 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.213+0000 7f7c45359640 1 -- 172.21.15.49:0/3486062206 wait complete. 2023-11-06T02:09:05.216 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.217+0000 7f7c45359640 1 Processor -- start 2023-11-06T02:09:05.216 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.217+0000 7f7c45359640 1 -- start start 2023-11-06T02:09:05.218 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.217+0000 7f7c45359640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c400b0220 0x7f7c4015bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:05.219 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.217+0000 7f7c45359640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7c401595a0 0x7f7c4014a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:05.219 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.217+0000 7f7c45359640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7c4015a990 0x7f7c400b27c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:05.219 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.217+0000 7f7c45359640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c400b1720 con 0x7f7c400b0220 2023-11-06T02:09:05.219 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.217+0000 7f7c45359640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c400b1a00 con 0x7f7c401595a0 2023-11-06T02:09:05.220 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.217+0000 7f7c45359640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c400b1ce0 con 0x7f7c4015a990 2023-11-06T02:09:05.220 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.217+0000 7f7c3effd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c400b0220 0x7f7c4015bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:05.220 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.217+0000 7f7c3effd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c400b0220 0x7f7c4015bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47736/0 (socket says 172.21.15.49:47736) 2023-11-06T02:09:05.220 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.217+0000 7f7c3effd640 1 -- 172.21.15.49:0/2865037405 learned_addr learned my addr 172.21.15.49:0/2865037405 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:05.220 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.217+0000 7f7c3f7fe640 1 --2- 172.21.15.49:0/2865037405 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7c4015a990 0x7f7c400b27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:05.220 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.217+0000 7f7c3e7fc640 1 --2- 172.21.15.49:0/2865037405 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7c401595a0 0x7f7c4014a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:05.221 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.217+0000 7f7c3effd640 1 -- 172.21.15.49:0/2865037405 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7c4015a990 msgr2=0x7f7c400b27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:05.221 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.217+0000 7f7c3effd640 1 --2- 172.21.15.49:0/2865037405 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7c4015a990 0x7f7c400b27c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:05.221 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.217+0000 7f7c3effd640 1 -- 172.21.15.49:0/2865037405 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7c401595a0 msgr2=0x7f7c4014a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:05.221 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.217+0000 7f7c3effd640 1 --2- 172.21.15.49:0/2865037405 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7c401595a0 0x7f7c4014a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:05.221 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.217+0000 7f7c3effd640 1 -- 172.21.15.49:0/2865037405 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c38009b30 con 0x7f7c400b0220 2023-11-06T02:09:05.222 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.217+0000 7f7c3e7fc640 1 --2- 172.21.15.49:0/2865037405 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7c401595a0 0x7f7c4014a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:09:05.222 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.217+0000 7f7c3effd640 1 --2- 172.21.15.49:0/2865037405 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c400b0220 0x7f7c4015bcd0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7c3000c500 tx=0x7f7c3000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:05.222 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.217+0000 7f7c277fe640 1 -- 172.21.15.49:0/2865037405 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c30005220 con 0x7f7c400b0220 2023-11-06T02:09:05.222 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.217+0000 7f7c277fe640 1 -- 172.21.15.49:0/2865037405 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c30005380 con 0x7f7c400b0220 2023-11-06T02:09:05.222 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.217+0000 7f7c45359640 1 -- 172.21.15.49:0/2865037405 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c400b1fc0 con 0x7f7c400b0220 2023-11-06T02:09:05.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.217+0000 7f7c277fe640 1 -- 172.21.15.49:0/2865037405 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c30007270 con 0x7f7c400b0220 2023-11-06T02:09:05.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.217+0000 7f7c45359640 1 -- 172.21.15.49:0/2865037405 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c4014a7b0 con 0x7f7c400b0220 2023-11-06T02:09:05.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.217+0000 7f7c45359640 1 -- 172.21.15.49:0/2865037405 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c4006c210 con 0x7f7c400b0220 2023-11-06T02:09:05.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.217+0000 7f7c277fe640 1 -- 172.21.15.49:0/2865037405 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f7c300073d0 con 0x7f7c400b0220 2023-11-06T02:09:05.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.221+0000 7f7c277fe640 1 --2- 172.21.15.49:0/2865037405 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7c2c065a60 0x7f7c2c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:05.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.221+0000 7f7c277fe640 1 -- 172.21.15.49:0/2865037405 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7c30082c40 con 0x7f7c400b0220 2023-11-06T02:09:05.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.221+0000 7f7c3e7fc640 1 --2- 172.21.15.49:0/2865037405 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7c2c065a60 0x7f7c2c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:05.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.221+0000 7f7c3e7fc640 1 --2- 172.21.15.49:0/2865037405 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7c2c065a60 0x7f7c2c067f20 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7c400af770 tx=0x7f7c34008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:05.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.225+0000 7f7c277fe640 1 -- 172.21.15.49:0/2865037405 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f7c4006c210 con 0x7f7c400b0220 2023-11-06T02:09:05.370 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.373+0000 7f7c45359640 1 -- 172.21.15.49:0/2865037405 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7c4006c210 con 0x7f7c400b0220 2023-11-06T02:09:05.370 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.373+0000 7f7c277fe640 1 -- 172.21.15.49:0/2865037405 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7c4006c210 con 0x7f7c400b0220 2023-11-06T02:09:05.371 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2023-11-06T02:09:05.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.373+0000 7f7c45359640 1 -- 172.21.15.49:0/2865037405 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7c2c065a60 msgr2=0x7f7c2c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:05.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.373+0000 7f7c45359640 1 --2- 172.21.15.49:0/2865037405 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7c2c065a60 0x7f7c2c067f20 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7c400af770 tx=0x7f7c34008040 comp rx=0 tx=0).stop 2023-11-06T02:09:05.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.373+0000 7f7c45359640 1 -- 172.21.15.49:0/2865037405 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c400b0220 msgr2=0x7f7c4015bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:05.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.373+0000 7f7c45359640 1 --2- 172.21.15.49:0/2865037405 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c400b0220 0x7f7c4015bcd0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7c3000c500 tx=0x7f7c3000c9c0 comp rx=0 tx=0).stop 2023-11-06T02:09:05.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.377+0000 7f7c45359640 1 -- 172.21.15.49:0/2865037405 shutdown_connections 2023-11-06T02:09:05.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.377+0000 7f7c45359640 1 --2- 172.21.15.49:0/2865037405 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7c2c065a60 0x7f7c2c067f20 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:05.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.377+0000 7f7c45359640 1 --2- 172.21.15.49:0/2865037405 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7c4015a990 0x7f7c400b27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:05.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.377+0000 7f7c45359640 1 --2- 172.21.15.49:0/2865037405 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7c401595a0 0x7f7c4014a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:05.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.377+0000 7f7c45359640 1 --2- 172.21.15.49:0/2865037405 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c400b0220 0x7f7c4015bcd0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:05.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.377+0000 7f7c45359640 1 -- 172.21.15.49:0/2865037405 >> 172.21.15.49:0/2865037405 conn(0x7f7c4006a370 msgr2=0x7f7c40072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:05.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.377+0000 7f7c45359640 1 -- 172.21.15.49:0/2865037405 shutdown_connections 2023-11-06T02:09:05.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.377+0000 7f7c45359640 1 -- 172.21.15.49:0/2865037405 wait complete. 2023-11-06T02:09:05.389 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T02:09:05.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.501+0000 7f6602091640 1 Processor -- start 2023-11-06T02:09:05.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.501+0000 7f6602091640 1 -- start start 2023-11-06T02:09:05.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.505+0000 7f6602091640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f65fc15a9e0 0x7f65fc0cb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:05.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.505+0000 7f6602091640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f65fc0afbb0 0x7f65fc0cca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:05.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.505+0000 7f6602091640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65fc0cc5c0 con 0x7f65fc0b04e0 2023-11-06T02:09:05.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.505+0000 7f6602091640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65fc0cb9b0 con 0x7f65fc15a9e0 2023-11-06T02:09:05.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.505+0000 7f6602091640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65fc0cbc90 con 0x7f65fc0afbb0 2023-11-06T02:09:05.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.505+0000 7f65fbfff640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f65fc0b04e0 0x7f65fc0ac4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:48458/0 (socket says 172.21.15.49:48458) 2023-11-06T02:09:05.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.505+0000 7f65fbfff640 1 -- 172.21.15.49:0/2961481277 learned_addr learned my addr 172.21.15.49:0/2961481277 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:09:05.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.505+0000 7f65fb7fe640 1 --2- 172.21.15.49:0/2961481277 >> v2:172.21.15.49:3300/0 conn(0x7f65fc15a9e0 0x7f65fc0cb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:05.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.505+0000 7f65f2ffd640 1 --2- 172.21.15.49:0/2961481277 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f65fc0afbb0 0x7f65fc0cca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:05.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.505+0000 7f65faffd640 1 -- 172.21.15.49:0/2961481277 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1363439422 0 0) 0x7f65fc0cc5c0 con 0x7f65fc0b04e0 2023-11-06T02:09:05.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.505+0000 7f65faffd640 1 -- 172.21.15.49:0/2961481277 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65e8003530 con 0x7f65fc0b04e0 2023-11-06T02:09:05.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.505+0000 7f65fb7fe640 1 -- 172.21.15.49:0/2961481277 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f65fc0afbb0 msgr2=0x7f65fc0cca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:05.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.505+0000 7f65fb7fe640 1 --2- 172.21.15.49:0/2961481277 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f65fc0afbb0 0x7f65fc0cca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:05.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.505+0000 7f65fb7fe640 1 -- 172.21.15.49:0/2961481277 >> v1:172.21.15.49:6789/0 conn(0x7f65fc0b04e0 legacy=0x7f65fc0ac4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:05.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.505+0000 7f65fb7fe640 1 -- 172.21.15.49:0/2961481277 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65fc0cbf70 con 0x7f65fc15a9e0 2023-11-06T02:09:05.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.505+0000 7f65fb7fe640 1 --2- 172.21.15.49:0/2961481277 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65fc15a9e0 0x7f65fc0cb2d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f65ec0097c0 tx=0x7f65ec01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7668980ec00d630a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:05.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.505+0000 7f65faffd640 1 -- 172.21.15.49:0/2961481277 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65ec0ad660 con 0x7f65fc15a9e0 2023-11-06T02:09:05.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.505+0000 7f65faffd640 1 -- 172.21.15.49:0/2961481277 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65ec0ad7c0 con 0x7f65fc15a9e0 2023-11-06T02:09:05.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.505+0000 7f65faffd640 1 -- 172.21.15.49:0/2961481277 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65ec0adb50 con 0x7f65fc15a9e0 2023-11-06T02:09:05.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.505+0000 7f6602091640 1 -- 172.21.15.49:0/2961481277 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65fc15a9e0 msgr2=0x7f65fc0cb2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:05.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.505+0000 7f6602091640 1 --2- 172.21.15.49:0/2961481277 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65fc15a9e0 0x7f65fc0cb2d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f65ec0097c0 tx=0x7f65ec01c450 comp rx=0 tx=0).stop 2023-11-06T02:09:05.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.509+0000 7f6602091640 1 -- 172.21.15.49:0/2961481277 shutdown_connections 2023-11-06T02:09:05.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.509+0000 7f6602091640 1 --2- 172.21.15.49:0/2961481277 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f65fc0afbb0 0x7f65fc0cca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:05.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.509+0000 7f6602091640 1 --2- 172.21.15.49:0/2961481277 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65fc15a9e0 0x7f65fc0cb2d0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:05.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.509+0000 7f6602091640 1 -- 172.21.15.49:0/2961481277 >> 172.21.15.49:0/2961481277 conn(0x7f65fc098a40 msgr2=0x7f65fc0cf860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:05.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.509+0000 7f6602091640 1 -- 172.21.15.49:0/2961481277 shutdown_connections 2023-11-06T02:09:05.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.509+0000 7f6602091640 1 -- 172.21.15.49:0/2961481277 wait complete. 2023-11-06T02:09:05.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.509+0000 7f6602091640 1 Processor -- start 2023-11-06T02:09:05.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.509+0000 7f6602091640 1 -- start start 2023-11-06T02:09:05.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.509+0000 7f6602091640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f65fc0afbb0 0x7f65fc1571a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:05.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.509+0000 7f6602091640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65fc0b04e0 0x7f65fc0b5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:05.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.509+0000 7f6602091640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f65fc15a9e0 0x7f65fc0cdd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:05.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.509+0000 7f6602091640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65fc0c6ad0 con 0x7f65fc0b04e0 2023-11-06T02:09:05.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.509+0000 7f6602091640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65fc0b56c0 con 0x7f65fc0afbb0 2023-11-06T02:09:05.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.509+0000 7f6602091640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65fc1577f0 con 0x7f65fc15a9e0 2023-11-06T02:09:05.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.509+0000 7f65f2ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65fc0b04e0 0x7f65fc0b5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:05.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.509+0000 7f65f2ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65fc0b04e0 0x7f65fc0b5180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47756/0 (socket says 172.21.15.49:47756) 2023-11-06T02:09:05.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.509+0000 7f65f2ffd640 1 -- 172.21.15.49:0/4184396610 learned_addr learned my addr 172.21.15.49:0/4184396610 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:05.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.509+0000 7f65fb7fe640 1 --2- 172.21.15.49:0/4184396610 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f65fc0afbb0 0x7f65fc1571a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:05.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.509+0000 7f65fbfff640 1 --2- 172.21.15.49:0/4184396610 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f65fc15a9e0 0x7f65fc0cdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:05.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.509+0000 7f65f2ffd640 1 -- 172.21.15.49:0/4184396610 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f65fc15a9e0 msgr2=0x7f65fc0cdd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:05.514 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.509+0000 7f65f2ffd640 1 --2- 172.21.15.49:0/4184396610 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f65fc15a9e0 0x7f65fc0cdd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:05.514 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.509+0000 7f65f2ffd640 1 -- 172.21.15.49:0/4184396610 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f65fc0afbb0 msgr2=0x7f65fc1571a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:05.514 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.509+0000 7f65f2ffd640 1 --2- 172.21.15.49:0/4184396610 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f65fc0afbb0 0x7f65fc1571a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:05.514 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.509+0000 7f65f2ffd640 1 -- 172.21.15.49:0/4184396610 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65ec009420 con 0x7f65fc0b04e0 2023-11-06T02:09:05.514 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.509+0000 7f65fbfff640 1 --2- 172.21.15.49:0/4184396610 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f65fc15a9e0 0x7f65fc0cdd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:09:05.515 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.509+0000 7f65fb7fe640 1 --2- 172.21.15.49:0/4184396610 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f65fc0afbb0 0x7f65fc1571a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:09:05.515 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.509+0000 7f65f2ffd640 1 --2- 172.21.15.49:0/4184396610 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65fc0b04e0 0x7f65fc0b5180 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f65e400d800 tx=0x7f65e400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:05.515 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.509+0000 7f65f8ff9640 1 -- 172.21.15.49:0/4184396610 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e400b6c0 con 0x7f65fc0b04e0 2023-11-06T02:09:05.515 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.509+0000 7f65f8ff9640 1 -- 172.21.15.49:0/4184396610 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65e400b820 con 0x7f65fc0b04e0 2023-11-06T02:09:05.515 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.509+0000 7f6602091640 1 -- 172.21.15.49:0/4184396610 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65fc0cd110 con 0x7f65fc0b04e0 2023-11-06T02:09:05.516 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.509+0000 7f65f8ff9640 1 -- 172.21.15.49:0/4184396610 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e400bb90 con 0x7f65fc0b04e0 2023-11-06T02:09:05.516 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.509+0000 7f6602091640 1 -- 172.21.15.49:0/4184396610 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65fc157db0 con 0x7f65fc0b04e0 2023-11-06T02:09:05.516 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.509+0000 7f6602091640 1 -- 172.21.15.49:0/4184396610 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65fc0ac5c0 con 0x7f65fc0b04e0 2023-11-06T02:09:05.517 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.513+0000 7f65f8ff9640 1 -- 172.21.15.49:0/4184396610 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f65e400bcf0 con 0x7f65fc0b04e0 2023-11-06T02:09:05.517 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.513+0000 7f65f8ff9640 1 --2- 172.21.15.49:0/4184396610 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f65dc065a60 0x7f65dc067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:05.518 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.513+0000 7f65f8ff9640 1 -- 172.21.15.49:0/4184396610 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f65e40842d0 con 0x7f65fc0b04e0 2023-11-06T02:09:05.518 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.513+0000 7f65fb7fe640 1 --2- 172.21.15.49:0/4184396610 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f65dc065a60 0x7f65dc067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:05.518 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.513+0000 7f65fb7fe640 1 --2- 172.21.15.49:0/4184396610 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f65dc065a60 0x7f65dc067f20 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f65ec0af000 tx=0x7f65ec008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:05.518 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.517+0000 7f65f8ff9640 1 -- 172.21.15.49:0/4184396610 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f65e4051c10 con 0x7f65fc0b04e0 2023-11-06T02:09:05.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.665+0000 7f6602091640 1 -- 172.21.15.49:0/4184396610 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f65fc159510 con 0x7f65fc0b04e0 2023-11-06T02:09:05.662 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.665+0000 7f65f8ff9640 1 -- 172.21.15.49:0/4184396610 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f65e4055230 con 0x7f65fc0b04e0 2023-11-06T02:09:05.662 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 8 2023-11-06T02:09:05.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.665+0000 7f6602091640 1 -- 172.21.15.49:0/4184396610 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f65dc065a60 msgr2=0x7f65dc067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:05.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.665+0000 7f6602091640 1 --2- 172.21.15.49:0/4184396610 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f65dc065a60 0x7f65dc067f20 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f65ec0af000 tx=0x7f65ec008040 comp rx=0 tx=0).stop 2023-11-06T02:09:05.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.665+0000 7f6602091640 1 -- 172.21.15.49:0/4184396610 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65fc0b04e0 msgr2=0x7f65fc0b5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:05.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.665+0000 7f6602091640 1 --2- 172.21.15.49:0/4184396610 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65fc0b04e0 0x7f65fc0b5180 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f65e400d800 tx=0x7f65e400dcc0 comp rx=0 tx=0).stop 2023-11-06T02:09:05.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.665+0000 7f6602091640 1 -- 172.21.15.49:0/4184396610 shutdown_connections 2023-11-06T02:09:05.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.665+0000 7f6602091640 1 --2- 172.21.15.49:0/4184396610 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f65dc065a60 0x7f65dc067f20 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:05.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.665+0000 7f6602091640 1 --2- 172.21.15.49:0/4184396610 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65fc0b04e0 0x7f65fc0b5180 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:05.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.665+0000 7f6602091640 1 --2- 172.21.15.49:0/4184396610 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f65fc15a9e0 0x7f65fc0cdd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:05.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.665+0000 7f6602091640 1 --2- 172.21.15.49:0/4184396610 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f65fc0afbb0 0x7f65fc1571a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:05.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.665+0000 7f6602091640 1 -- 172.21.15.49:0/4184396610 >> 172.21.15.49:0/4184396610 conn(0x7f65fc098a40 msgr2=0x7f65fc0aefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:05.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.665+0000 7f6602091640 1 -- 172.21.15.49:0/4184396610 shutdown_connections 2023-11-06T02:09:05.667 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.665+0000 7f6602091640 1 -- 172.21.15.49:0/4184396610 wait complete. 2023-11-06T02:09:05.679 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T02:09:05.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.797+0000 7f6b22df8640 1 Processor -- start 2023-11-06T02:09:05.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.797+0000 7f6b22df8640 1 -- start start 2023-11-06T02:09:05.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.797+0000 7f6b22df8640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6b1c1595a0 0x7f6b1c15dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:05.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.797+0000 7f6b22df8640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b1c06c5a0 con 0x7f6b1c06d640 2023-11-06T02:09:05.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.797+0000 7f6b22df8640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b1c06c880 con 0x7f6b1c15a990 2023-11-06T02:09:05.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.797+0000 7f6b22df8640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1c15e250 con 0x7f6b1c1595a0 2023-11-06T02:09:05.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.797+0000 7f6b1bfff640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f6b1c06d640 0x7f6b1c0b0220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:48472/0 (socket says 172.21.15.49:48472) 2023-11-06T02:09:05.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.797+0000 7f6b1bfff640 1 -- 172.21.15.49:0/2997768166 learned_addr learned my addr 172.21.15.49:0/2997768166 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:09:05.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.797+0000 7f6b20b94640 1 --2- 172.21.15.49:0/2997768166 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6b1c1595a0 0x7f6b1c15dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:05.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.797+0000 7f6b1b7fe640 1 -- 172.21.15.49:0/2997768166 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2914656877 0 0) 0x7f6b1c06c5a0 con 0x7f6b1c06d640 2023-11-06T02:09:05.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.797+0000 7f6b1b7fe640 1 -- 172.21.15.49:0/2997768166 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b04005740 con 0x7f6b1c06d640 2023-11-06T02:09:05.798 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.797+0000 7f6b1b7fe640 1 -- 172.21.15.49:0/2997768166 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 499977772 0 0) 0x7f6b04005740 con 0x7f6b1c06d640 2023-11-06T02:09:05.798 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.797+0000 7f6b1b7fe640 1 -- 172.21.15.49:0/2997768166 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6b1c1595a0 msgr2=0x7f6b1c15dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:05.798 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.797+0000 7f6b1b7fe640 1 --2- 172.21.15.49:0/2997768166 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6b1c1595a0 0x7f6b1c15dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:05.798 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.797+0000 7f6b1b7fe640 1 -- 172.21.15.49:0/2997768166 >> v1:172.21.15.150:6789/0 conn(0x7f6b1c15a990 legacy=0x7f6b1c0c8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:05.798 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.797+0000 7f6b1b7fe640 1 -- 172.21.15.49:0/2997768166 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b1c06cb60 con 0x7f6b1c06d640 2023-11-06T02:09:05.799 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.797+0000 7f6b1b7fe640 1 -- 172.21.15.49:0/2997768166 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7f6b10001a00 con 0x7f6b1c06d640 2023-11-06T02:09:05.799 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.797+0000 7f6b1b7fe640 1 -- 172.21.15.49:0/2997768166 >> v1:172.21.15.49:6789/0 conn(0x7f6b1c06d640 legacy=0x7f6b1c0b0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:05.799 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.797+0000 7f6b1b7fe640 1 --2- 172.21.15.49:0/2997768166 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b04005bc0 0x7f6b04005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:05.799 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.797+0000 7f6b1b7fe640 1 --2- 172.21.15.49:0/2997768166 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b04006570 0x7f6b04006a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:05.799 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.797+0000 7f6b1b7fe640 1 --2- 172.21.15.49:0/2997768166 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6b1c1595a0 0x7f6b040072a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:05.800 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.797+0000 7f6b1b7fe640 1 -- 172.21.15.49:0/2997768166 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1c15e250 con 0x7f6b04006570 2023-11-06T02:09:05.800 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.797+0000 7f6b1b7fe640 1 -- 172.21.15.49:0/2997768166 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b0400dba0 con 0x7f6b04005bc0 2023-11-06T02:09:05.800 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.797+0000 7f6b1b7fe640 1 -- 172.21.15.49:0/2997768166 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b0400dd60 con 0x7f6b1c1595a0 2023-11-06T02:09:05.800 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.797+0000 7f6b1bfff640 1 --2- 172.21.15.49:0/2997768166 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b04006570 0x7f6b04006a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:05.800 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.797+0000 7f6b21395640 1 --2- 172.21.15.49:0/2997768166 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6b1c1595a0 0x7f6b040072a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:05.801 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.797+0000 7f6b20b94640 1 --2- 172.21.15.49:0/2997768166 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b04005bc0 0x7f6b04005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:05.801 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.797+0000 7f6b1bfff640 1 -- 172.21.15.49:0/2997768166 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6b1c1595a0 msgr2=0x7f6b040072a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:05.801 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.797+0000 7f6b1bfff640 1 --2- 172.21.15.49:0/2997768166 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6b1c1595a0 0x7f6b040072a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:05.801 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.797+0000 7f6b1bfff640 1 -- 172.21.15.49:0/2997768166 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b04005bc0 msgr2=0x7f6b04005fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:05.801 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.797+0000 7f6b1bfff640 1 --2- 172.21.15.49:0/2997768166 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b04005bc0 0x7f6b04005fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:05.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.797+0000 7f6b1bfff640 1 -- 172.21.15.49:0/2997768166 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b04005740 con 0x7f6b04006570 2023-11-06T02:09:05.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.801+0000 7f6b1bfff640 1 --2- 172.21.15.49:0/2997768166 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b04006570 0x7f6b04006a40 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6b1000a830 tx=0x7f6b1001d310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=793beb04b497430c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:05.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.801+0000 7f6b1b7fe640 1 -- 172.21.15.49:0/2997768166 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b100ad630 con 0x7f6b04006570 2023-11-06T02:09:05.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.801+0000 7f6b1b7fe640 1 -- 172.21.15.49:0/2997768166 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b100a8070 con 0x7f6b04006570 2023-11-06T02:09:05.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.801+0000 7f6b1b7fe640 1 -- 172.21.15.49:0/2997768166 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b100ad9f0 con 0x7f6b04006570 2023-11-06T02:09:05.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.801+0000 7f6b22df8640 1 -- 172.21.15.49:0/2997768166 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b04006570 msgr2=0x7f6b04006a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:05.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.801+0000 7f6b22df8640 1 --2- 172.21.15.49:0/2997768166 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b04006570 0x7f6b04006a40 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6b1000a830 tx=0x7f6b1001d310 comp rx=0 tx=0).stop 2023-11-06T02:09:05.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.801+0000 7f6b21395640 1 -- 172.21.15.49:0/2997768166 reap_dead start 2023-11-06T02:09:05.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.801+0000 7f6b22df8640 1 -- 172.21.15.49:0/2997768166 shutdown_connections 2023-11-06T02:09:05.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.801+0000 7f6b22df8640 1 -- 172.21.15.49:0/2997768166 >> 172.21.15.49:0/2997768166 conn(0x7f6b1c06a370 msgr2=0x7f6b1c0ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:05.807 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.801+0000 7f6b22df8640 1 -- 172.21.15.49:0/2997768166 shutdown_connections 2023-11-06T02:09:05.807 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.801+0000 7f6b22df8640 1 -- 172.21.15.49:0/2997768166 wait complete. 2023-11-06T02:09:05.807 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.801+0000 7f6b22df8640 1 Processor -- start 2023-11-06T02:09:05.807 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.801+0000 7f6b22df8640 1 -- start start 2023-11-06T02:09:05.807 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.801+0000 7f6b22df8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b04006570 0x7f6b1c1467f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:05.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.801+0000 7f6b22df8640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6b1c0c7940 0x7f6b1c0c7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:05.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.801+0000 7f6b22df8640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b1c0b2530 0x7f6b1c0b29b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:05.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.801+0000 7f6b22df8640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1c0c8300 con 0x7f6b04006570 2023-11-06T02:09:05.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.801+0000 7f6b22df8640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1c147930 con 0x7f6b1c0b2530 2023-11-06T02:09:05.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.801+0000 7f6b22df8640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1c146d30 con 0x7f6b1c0c7940 2023-11-06T02:09:05.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.801+0000 7f6b20b94640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b04006570 0x7f6b1c1467f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:05.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.801+0000 7f6b20b94640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b04006570 0x7f6b1c1467f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47774/0 (socket says 172.21.15.49:47774) 2023-11-06T02:09:05.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.801+0000 7f6b20b94640 1 -- 172.21.15.49:0/1341716237 learned_addr learned my addr 172.21.15.49:0/1341716237 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:05.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.805+0000 7f6b21395640 1 --2- 172.21.15.49:0/1341716237 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b1c0b2530 0x7f6b1c0b29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:05.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.805+0000 7f6b1bfff640 1 --2- 172.21.15.49:0/1341716237 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6b1c0c7940 0x7f6b1c0c7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:05.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.805+0000 7f6b20b94640 1 -- 172.21.15.49:0/1341716237 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6b1c0c7940 msgr2=0x7f6b1c0c7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:05.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.805+0000 7f6b20b94640 1 --2- 172.21.15.49:0/1341716237 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6b1c0c7940 0x7f6b1c0c7dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:05.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.805+0000 7f6b20b94640 1 -- 172.21.15.49:0/1341716237 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b1c0b2530 msgr2=0x7f6b1c0b29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:05.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.805+0000 7f6b20b94640 1 --2- 172.21.15.49:0/1341716237 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b1c0b2530 0x7f6b1c0b29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:05.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.805+0000 7f6b20b94640 1 -- 172.21.15.49:0/1341716237 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b1c0b32a0 con 0x7f6b04006570 2023-11-06T02:09:05.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.805+0000 7f6b20b94640 1 --2- 172.21.15.49:0/1341716237 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b04006570 0x7f6b1c1467f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c011640 tx=0x7f6b0c011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:05.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.805+0000 7f6b197fa640 1 -- 172.21.15.49:0/1341716237 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c00f8c0 con 0x7f6b04006570 2023-11-06T02:09:05.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.805+0000 7f6b197fa640 1 -- 172.21.15.49:0/1341716237 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b0c00fa20 con 0x7f6b04006570 2023-11-06T02:09:05.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.805+0000 7f6b22df8640 1 -- 172.21.15.49:0/1341716237 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b1c147000 con 0x7f6b04006570 2023-11-06T02:09:05.812 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.805+0000 7f6b197fa640 1 -- 172.21.15.49:0/1341716237 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c00fd90 con 0x7f6b04006570 2023-11-06T02:09:05.812 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.805+0000 7f6b22df8640 1 -- 172.21.15.49:0/1341716237 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b1c1475c0 con 0x7f6b04006570 2023-11-06T02:09:05.812 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.805+0000 7f6b22df8640 1 -- 172.21.15.49:0/1341716237 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b1c147780 con 0x7f6b04006570 2023-11-06T02:09:05.812 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.805+0000 7f6b197fa640 1 -- 172.21.15.49:0/1341716237 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f6b0c015460 con 0x7f6b04006570 2023-11-06T02:09:05.812 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.805+0000 7f6b197fa640 1 --2- 172.21.15.49:0/1341716237 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6b08065a90 0x7f6b08067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:05.812 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.805+0000 7f6b197fa640 1 -- 172.21.15.49:0/1341716237 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6b0c083760 con 0x7f6b04006570 2023-11-06T02:09:05.813 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.805+0000 7f6b1bfff640 1 --2- 172.21.15.49:0/1341716237 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6b08065a90 0x7f6b08067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:05.813 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.805+0000 7f6b1bfff640 1 --2- 172.21.15.49:0/1341716237 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6b08065a90 0x7f6b08067f50 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6b1000a830 tx=0x7f6b100a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:05.813 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.809+0000 7f6b197fa640 1 -- 172.21.15.49:0/1341716237 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f6b0c050fc0 con 0x7f6b04006570 2023-11-06T02:09:05.957 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.957+0000 7f6b22df8640 1 -- 172.21.15.49:0/1341716237 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6b1c06c210 con 0x7f6b04006570 2023-11-06T02:09:05.957 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.961+0000 7f6b197fa640 1 -- 172.21.15.49:0/1341716237 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6b1c06c210 con 0x7f6b04006570 2023-11-06T02:09:05.958 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2023-11-06T02:09:05.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.961+0000 7f6b22df8640 1 -- 172.21.15.49:0/1341716237 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6b08065a90 msgr2=0x7f6b08067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:05.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.961+0000 7f6b22df8640 1 --2- 172.21.15.49:0/1341716237 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6b08065a90 0x7f6b08067f50 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6b1000a830 tx=0x7f6b100a7000 comp rx=0 tx=0).stop 2023-11-06T02:09:05.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.961+0000 7f6b22df8640 1 -- 172.21.15.49:0/1341716237 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b04006570 msgr2=0x7f6b1c1467f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:05.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.961+0000 7f6b22df8640 1 --2- 172.21.15.49:0/1341716237 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b04006570 0x7f6b1c1467f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c011640 tx=0x7f6b0c011b00 comp rx=0 tx=0).stop 2023-11-06T02:09:05.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.961+0000 7f6b22df8640 1 -- 172.21.15.49:0/1341716237 shutdown_connections 2023-11-06T02:09:05.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.961+0000 7f6b22df8640 1 --2- 172.21.15.49:0/1341716237 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6b08065a90 0x7f6b08067f50 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:05.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.961+0000 7f6b22df8640 1 --2- 172.21.15.49:0/1341716237 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b1c0b2530 0x7f6b1c0b29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:05.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.961+0000 7f6b22df8640 1 --2- 172.21.15.49:0/1341716237 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6b1c0c7940 0x7f6b1c0c7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:05.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.961+0000 7f6b22df8640 1 --2- 172.21.15.49:0/1341716237 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b04006570 0x7f6b1c1467f0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:05.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.961+0000 7f6b22df8640 1 -- 172.21.15.49:0/1341716237 >> 172.21.15.49:0/1341716237 conn(0x7f6b1c06a370 msgr2=0x7f6b1c0b0220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:05.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.961+0000 7f6b22df8640 1 -- 172.21.15.49:0/1341716237 shutdown_connections 2023-11-06T02:09:05.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:05.961+0000 7f6b22df8640 1 -- 172.21.15.49:0/1341716237 wait complete. 2023-11-06T02:09:05.975 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T02:09:06.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.113+0000 7f7cf39b4640 1 Processor -- start 2023-11-06T02:09:06.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.113+0000 7f7cf39b4640 1 -- start start 2023-11-06T02:09:06.113 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.117+0000 7f7cf39b4640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f7cec15a920 0x7f7cec15ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:06.113 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.117+0000 7f7cf39b4640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cec06ced0 con 0x7f7cec06dc90 2023-11-06T02:09:06.114 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.117+0000 7f7cf39b4640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cec06d1b0 con 0x7f7cec06ea60 2023-11-06T02:09:06.114 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.117+0000 7f7cf39b4640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cec06d870 con 0x7f7cec15a920 2023-11-06T02:09:06.114 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.117+0000 7f7cf0f4f640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f7cec06dc90 0x7f7cec06e350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:48488/0 (socket says 172.21.15.49:48488) 2023-11-06T02:09:06.114 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.117+0000 7f7cf0f4f640 1 -- 172.21.15.49:0/3773617412 learned_addr learned my addr 172.21.15.49:0/3773617412 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:09:06.114 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.117+0000 7f7cf1750640 1 --2- 172.21.15.49:0/3773617412 >> v2:172.21.15.150:3300/0 conn(0x7f7cec15a920 0x7f7cec15ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:06.115 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.117+0000 7f7cf1750640 1 -- 172.21.15.49:0/3773617412 >> v1:172.21.15.150:6789/0 conn(0x7f7cec06ea60 legacy=0x7f7cec0cb3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:06.115 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.117+0000 7f7cf1750640 1 -- 172.21.15.49:0/3773617412 >> v1:172.21.15.49:6789/0 conn(0x7f7cec06dc90 legacy=0x7f7cec06e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:06.116 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.117+0000 7f7cf1750640 1 -- 172.21.15.49:0/3773617412 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cec06d490 con 0x7f7cec15a920 2023-11-06T02:09:06.116 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.117+0000 7f7cf1750640 1 --2- 172.21.15.49:0/3773617412 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7cec15a920 0x7f7cec15ed50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7ce80097a0 tx=0x7f7ce801c460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb215934284d7a69 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:06.119 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.121+0000 7f7cdffff640 1 -- 172.21.15.49:0/3773617412 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce80ab600 con 0x7f7cec15a920 2023-11-06T02:09:06.119 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.121+0000 7f7cdffff640 1 -- 172.21.15.49:0/3773617412 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ce80a7050 con 0x7f7cec15a920 2023-11-06T02:09:06.119 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.121+0000 7f7cdffff640 1 -- 172.21.15.49:0/3773617412 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce80ab980 con 0x7f7cec15a920 2023-11-06T02:09:06.120 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.121+0000 7f7cf39b4640 1 -- 172.21.15.49:0/3773617412 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7cec15a920 msgr2=0x7f7cec15ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:06.120 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.121+0000 7f7cf39b4640 1 --2- 172.21.15.49:0/3773617412 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7cec15a920 0x7f7cec15ed50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7ce80097a0 tx=0x7f7ce801c460 comp rx=0 tx=0).stop 2023-11-06T02:09:06.120 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.121+0000 7f7cf39b4640 1 -- 172.21.15.49:0/3773617412 shutdown_connections 2023-11-06T02:09:06.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.121+0000 7f7cf39b4640 1 --2- 172.21.15.49:0/3773617412 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7cec15a920 0x7f7cec15ed50 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:06.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.121+0000 7f7cf39b4640 1 -- 172.21.15.49:0/3773617412 >> 172.21.15.49:0/3773617412 conn(0x7f7cec098af0 msgr2=0x7f7cec0cf740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:06.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.121+0000 7f7cf39b4640 1 -- 172.21.15.49:0/3773617412 shutdown_connections 2023-11-06T02:09:06.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.121+0000 7f7cf39b4640 1 -- 172.21.15.49:0/3773617412 wait complete. 2023-11-06T02:09:06.123 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.125+0000 7f7cf39b4640 1 Processor -- start 2023-11-06T02:09:06.124 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.125+0000 7f7cf39b4640 1 -- start start 2023-11-06T02:09:06.124 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.125+0000 7f7cf39b4640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7cec06dc90 0x7f7cec0ce270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:06.124 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.125+0000 7f7cf39b4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7cec06ea60 0x7f7cec15d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:06.124 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.125+0000 7f7cf39b4640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7cec15a920 0x7f7cec147b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:06.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.125+0000 7f7cf39b4640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cec0c53a0 con 0x7f7cec06ea60 2023-11-06T02:09:06.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.125+0000 7f7cf39b4640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cec0cd4b0 con 0x7f7cec06dc90 2023-11-06T02:09:06.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.125+0000 7f7cf39b4640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cec0cd790 con 0x7f7cec15a920 2023-11-06T02:09:06.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.125+0000 7f7cf0f4f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7cec06ea60 0x7f7cec15d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:06.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.125+0000 7f7cf0f4f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7cec06ea60 0x7f7cec15d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47778/0 (socket says 172.21.15.49:47778) 2023-11-06T02:09:06.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.125+0000 7f7cf0f4f640 1 -- 172.21.15.49:0/2460608649 learned_addr learned my addr 172.21.15.49:0/2460608649 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:06.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.125+0000 7f7cf1750640 1 --2- 172.21.15.49:0/2460608649 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7cec06dc90 0x7f7cec0ce270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:06.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.125+0000 7f7cf1f51640 1 --2- 172.21.15.49:0/2460608649 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7cec15a920 0x7f7cec147b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:06.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.125+0000 7f7cf0f4f640 1 -- 172.21.15.49:0/2460608649 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7cec15a920 msgr2=0x7f7cec147b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:06.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.125+0000 7f7cf0f4f640 1 --2- 172.21.15.49:0/2460608649 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7cec15a920 0x7f7cec147b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:06.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.125+0000 7f7cf0f4f640 1 -- 172.21.15.49:0/2460608649 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7cec06dc90 msgr2=0x7f7cec0ce270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:06.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.125+0000 7f7cf0f4f640 1 --2- 172.21.15.49:0/2460608649 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7cec06dc90 0x7f7cec0ce270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:06.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.125+0000 7f7cf0f4f640 1 -- 172.21.15.49:0/2460608649 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ce80093b0 con 0x7f7cec06ea60 2023-11-06T02:09:06.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.125+0000 7f7cf1750640 1 --2- 172.21.15.49:0/2460608649 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7cec06dc90 0x7f7cec0ce270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:09:06.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.125+0000 7f7cf0f4f640 1 --2- 172.21.15.49:0/2460608649 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7cec06ea60 0x7f7cec15d1c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7ce000c400 tx=0x7f7ce000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:06.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.125+0000 7f7cddffb640 1 -- 172.21.15.49:0/2460608649 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce0005cf0 con 0x7f7cec06ea60 2023-11-06T02:09:06.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.125+0000 7f7cf39b4640 1 -- 172.21.15.49:0/2460608649 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cec0cda70 con 0x7f7cec06ea60 2023-11-06T02:09:06.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.125+0000 7f7cddffb640 1 -- 172.21.15.49:0/2460608649 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ce0005e50 con 0x7f7cec06ea60 2023-11-06T02:09:06.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.125+0000 7f7cddffb640 1 -- 172.21.15.49:0/2460608649 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce00072d0 con 0x7f7cec06ea60 2023-11-06T02:09:06.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.125+0000 7f7cf39b4640 1 -- 172.21.15.49:0/2460608649 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cec0ce7b0 con 0x7f7cec06ea60 2023-11-06T02:09:06.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.125+0000 7f7cf39b4640 1 -- 172.21.15.49:0/2460608649 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cec074040 con 0x7f7cec06ea60 2023-11-06T02:09:06.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.125+0000 7f7cddffb640 1 -- 172.21.15.49:0/2460608649 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f7ce0007430 con 0x7f7cec06ea60 2023-11-06T02:09:06.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.125+0000 7f7cddffb640 1 --2- 172.21.15.49:0/2460608649 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7cd0065a60 0x7f7cd0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:06.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.125+0000 7f7cddffb640 1 -- 172.21.15.49:0/2460608649 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7ce00827d0 con 0x7f7cec06ea60 2023-11-06T02:09:06.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.129+0000 7f7cf1750640 1 --2- 172.21.15.49:0/2460608649 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7cd0065a60 0x7f7cd0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:06.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.129+0000 7f7cf1750640 1 --2- 172.21.15.49:0/2460608649 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7cd0065a60 0x7f7cd0067f20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7ce80ad000 tx=0x7f7ce80a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:06.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.129+0000 7f7cddffb640 1 -- 172.21.15.49:0/2460608649 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f7ce00500e0 con 0x7f7cec06ea60 2023-11-06T02:09:06.279 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.281+0000 7f7cf39b4640 1 -- 172.21.15.49:0/2460608649 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7cec0cf840 con 0x7f7cec06ea60 2023-11-06T02:09:06.279 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.281+0000 7f7cddffb640 1 -- 172.21.15.49:0/2460608649 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7ce0017020 con 0x7f7cec06ea60 2023-11-06T02:09:06.280 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2023-11-06T02:09:06.282 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.285+0000 7f7cf39b4640 1 -- 172.21.15.49:0/2460608649 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7cd0065a60 msgr2=0x7f7cd0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:06.282 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.285+0000 7f7cf39b4640 1 --2- 172.21.15.49:0/2460608649 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7cd0065a60 0x7f7cd0067f20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7ce80ad000 tx=0x7f7ce80a6000 comp rx=0 tx=0).stop 2023-11-06T02:09:06.282 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.285+0000 7f7cf39b4640 1 -- 172.21.15.49:0/2460608649 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7cec06ea60 msgr2=0x7f7cec15d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:06.283 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.285+0000 7f7cf39b4640 1 --2- 172.21.15.49:0/2460608649 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7cec06ea60 0x7f7cec15d1c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7ce000c400 tx=0x7f7ce000c8c0 comp rx=0 tx=0).stop 2023-11-06T02:09:06.283 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.285+0000 7f7cf39b4640 1 -- 172.21.15.49:0/2460608649 shutdown_connections 2023-11-06T02:09:06.283 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.285+0000 7f7cf39b4640 1 --2- 172.21.15.49:0/2460608649 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7cd0065a60 0x7f7cd0067f20 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:06.283 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.285+0000 7f7cf39b4640 1 --2- 172.21.15.49:0/2460608649 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7cec06ea60 0x7f7cec15d1c0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:06.283 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.285+0000 7f7cf39b4640 1 --2- 172.21.15.49:0/2460608649 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7cec15a920 0x7f7cec147b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:06.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.285+0000 7f7cf39b4640 1 --2- 172.21.15.49:0/2460608649 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7cec06dc90 0x7f7cec0ce270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:06.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.285+0000 7f7cf39b4640 1 -- 172.21.15.49:0/2460608649 >> 172.21.15.49:0/2460608649 conn(0x7f7cec098af0 msgr2=0x7f7cec0ae360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:06.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.285+0000 7f7cf39b4640 1 -- 172.21.15.49:0/2460608649 shutdown_connections 2023-11-06T02:09:06.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.285+0000 7f7cf39b4640 1 -- 172.21.15.49:0/2460608649 wait complete. 2023-11-06T02:09:06.297 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T02:09:06.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.409+0000 7fb45084c640 1 Processor -- start 2023-11-06T02:09:06.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.409+0000 7fb45084c640 1 -- start start 2023-11-06T02:09:06.419 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.409+0000 7fb45084c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb44c15ab50 0x7fb44c0c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:06.419 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.409+0000 7fb45084c640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fb44c0b0790 0x7fb44c0c2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:06.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.409+0000 7fb45084c640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb44c0cab80 con 0x7fb44c0cb010 2023-11-06T02:09:06.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.409+0000 7fb45084c640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb44c15af30 con 0x7fb44c0b0790 2023-11-06T02:09:06.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.409+0000 7fb45084c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb44c0cb5b0 con 0x7fb44c15ab50 2023-11-06T02:09:06.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.409+0000 7fb449d9b640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fb44c0cb010 0x7fb44c0b0080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:48494/0 (socket says 172.21.15.49:48494) 2023-11-06T02:09:06.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.409+0000 7fb449d9b640 1 -- 172.21.15.49:0/3791526819 learned_addr learned my addr 172.21.15.49:0/3791526819 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:09:06.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.413+0000 7fb44ad9d640 1 --2- 172.21.15.49:0/3791526819 >> v2:172.21.15.150:3300/0 conn(0x7fb44c0b0790 0x7fb44c0c2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:06.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.413+0000 7fb44a59c640 1 --2- 172.21.15.49:0/3791526819 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb44c15ab50 0x7fb44c0c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:06.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.413+0000 7fb44959a640 1 -- 172.21.15.49:0/3791526819 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 36246279 0 0) 0x7fb44c0cab80 con 0x7fb44c0cb010 2023-11-06T02:09:06.422 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.413+0000 7fb44959a640 1 -- 172.21.15.49:0/3791526819 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb438003530 con 0x7fb44c0cb010 2023-11-06T02:09:06.422 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.413+0000 7fb44959a640 1 -- 172.21.15.49:0/3791526819 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1357085043 0 0) 0x7fb438003530 con 0x7fb44c0cb010 2023-11-06T02:09:06.422 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.413+0000 7fb44959a640 1 -- 172.21.15.49:0/3791526819 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb44c15ab50 msgr2=0x7fb44c0c9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:06.422 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.413+0000 7fb44959a640 1 --2- 172.21.15.49:0/3791526819 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb44c15ab50 0x7fb44c0c9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:06.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.413+0000 7fb44959a640 1 -- 172.21.15.49:0/3791526819 >> v2:172.21.15.150:3300/0 conn(0x7fb44c0b0790 msgr2=0x7fb44c0c2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:06.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.413+0000 7fb44959a640 1 --2- 172.21.15.49:0/3791526819 >> v2:172.21.15.150:3300/0 conn(0x7fb44c0b0790 0x7fb44c0c2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:06.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.413+0000 7fb44959a640 1 -- 172.21.15.49:0/3791526819 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb44c0cb890 con 0x7fb44c0cb010 2023-11-06T02:09:06.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.413+0000 7fb44959a640 1 -- 172.21.15.49:0/3791526819 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7fb434001680 con 0x7fb44c0cb010 2023-11-06T02:09:06.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.413+0000 7fb44959a640 1 -- 172.21.15.49:0/3791526819 >> v1:172.21.15.49:6789/0 conn(0x7fb44c0cb010 legacy=0x7fb44c0b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:06.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.413+0000 7fb44959a640 1 --2- 172.21.15.49:0/3791526819 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb438003980 0x7fb438003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:06.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.413+0000 7fb44959a640 1 --2- 172.21.15.49:0/3791526819 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb44c15ab50 0x7fb4380047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:06.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.413+0000 7fb44959a640 1 --2- 172.21.15.49:0/3791526819 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb438004d10 0x7fb438005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:06.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.413+0000 7fb44959a640 1 -- 172.21.15.49:0/3791526819 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb44c15af30 con 0x7fb438003980 2023-11-06T02:09:06.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.413+0000 7fb44959a640 1 -- 172.21.15.49:0/3791526819 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb44c0cb5b0 con 0x7fb438004d10 2023-11-06T02:09:06.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.413+0000 7fb44959a640 1 -- 172.21.15.49:0/3791526819 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb43800b990 con 0x7fb44c15ab50 2023-11-06T02:09:06.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.413+0000 7fb44a59c640 1 --2- 172.21.15.49:0/3791526819 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb438003980 0x7fb438003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:06.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.413+0000 7fb449d9b640 1 --2- 172.21.15.49:0/3791526819 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb44c15ab50 0x7fb4380047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:06.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.413+0000 7fb44a59c640 1 -- 172.21.15.49:0/3791526819 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb44c15ab50 msgr2=0x7fb4380047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:06.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.413+0000 7fb44a59c640 1 --2- 172.21.15.49:0/3791526819 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb44c15ab50 0x7fb4380047d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:06.427 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.413+0000 7fb44a59c640 1 -- 172.21.15.49:0/3791526819 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb438004d10 msgr2=0x7fb438005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:06.427 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.413+0000 7fb44a59c640 1 --2- 172.21.15.49:0/3791526819 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb438004d10 0x7fb438005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:06.427 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.413+0000 7fb44a59c640 1 -- 172.21.15.49:0/3791526819 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb438003530 con 0x7fb438003980 2023-11-06T02:09:06.427 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.413+0000 7fb44a59c640 1 --2- 172.21.15.49:0/3791526819 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb438003980 0x7fb438003da0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fb44400e1a0 tx=0x7fb44401cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b71f2824e08f52b9 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:06.427 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.413+0000 7fb44959a640 1 -- 172.21.15.49:0/3791526819 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4440a5c40 con 0x7fb438003980 2023-11-06T02:09:06.428 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.413+0000 7fb44959a640 1 -- 172.21.15.49:0/3791526819 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb44400f040 con 0x7fb438003980 2023-11-06T02:09:06.428 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.413+0000 7fb44959a640 1 -- 172.21.15.49:0/3791526819 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4440ad440 con 0x7fb438003980 2023-11-06T02:09:06.428 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.413+0000 7fb45084c640 1 -- 172.21.15.49:0/3791526819 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb438003980 msgr2=0x7fb438003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:06.428 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.413+0000 7fb45084c640 1 --2- 172.21.15.49:0/3791526819 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb438003980 0x7fb438003da0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fb44400e1a0 tx=0x7fb44401cc50 comp rx=0 tx=0).stop 2023-11-06T02:09:06.428 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.413+0000 7fb44ad9d640 1 -- 172.21.15.49:0/3791526819 reap_dead start 2023-11-06T02:09:06.429 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.413+0000 7fb45084c640 1 -- 172.21.15.49:0/3791526819 shutdown_connections 2023-11-06T02:09:06.429 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.413+0000 7fb45084c640 1 -- 172.21.15.49:0/3791526819 >> 172.21.15.49:0/3791526819 conn(0x7fb44c06c0a0 msgr2=0x7fb44c06c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:06.429 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.413+0000 7fb45084c640 1 -- 172.21.15.49:0/3791526819 shutdown_connections 2023-11-06T02:09:06.429 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.413+0000 7fb45084c640 1 -- 172.21.15.49:0/3791526819 wait complete. 2023-11-06T02:09:06.429 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.417+0000 7fb45084c640 1 Processor -- start 2023-11-06T02:09:06.430 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.417+0000 7fb45084c640 1 -- start start 2023-11-06T02:09:06.430 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.417+0000 7fb45084c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb438003980 0x7fb44c155e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:06.430 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.417+0000 7fb45084c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb44c150940 0x7fb44c1f3100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:06.430 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.417+0000 7fb45084c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb44c1f3640 0x7fb44c0afaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:06.431 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.417+0000 7fb45084c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb44c150e20 con 0x7fb44c150940 2023-11-06T02:09:06.431 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.417+0000 7fb45084c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb44c151100 con 0x7fb438003980 2023-11-06T02:09:06.431 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.417+0000 7fb45084c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb44c1513e0 con 0x7fb44c1f3640 2023-11-06T02:09:06.431 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.417+0000 7fb449d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb44c150940 0x7fb44c1f3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:06.432 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.417+0000 7fb449d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb44c150940 0x7fb44c1f3100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47794/0 (socket says 172.21.15.49:47794) 2023-11-06T02:09:06.432 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.417+0000 7fb449d9b640 1 -- 172.21.15.49:0/2893787051 learned_addr learned my addr 172.21.15.49:0/2893787051 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:06.432 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.417+0000 7fb44a59c640 1 --2- 172.21.15.49:0/2893787051 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb438003980 0x7fb44c155e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:06.432 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.417+0000 7fb44ad9d640 1 --2- 172.21.15.49:0/2893787051 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb44c1f3640 0x7fb44c0afaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:06.433 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.417+0000 7fb449d9b640 1 -- 172.21.15.49:0/2893787051 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb44c1f3640 msgr2=0x7fb44c0afaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:06.433 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.417+0000 7fb449d9b640 1 --2- 172.21.15.49:0/2893787051 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb44c1f3640 0x7fb44c0afaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:06.434 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.417+0000 7fb449d9b640 1 -- 172.21.15.49:0/2893787051 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb438003980 msgr2=0x7fb44c155e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:06.434 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.417+0000 7fb449d9b640 1 --2- 172.21.15.49:0/2893787051 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb438003980 0x7fb44c155e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:06.434 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.417+0000 7fb449d9b640 1 -- 172.21.15.49:0/2893787051 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb44c1516c0 con 0x7fb44c150940 2023-11-06T02:09:06.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.417+0000 7fb44a59c640 1 --2- 172.21.15.49:0/2893787051 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb438003980 0x7fb44c155e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:09:06.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.417+0000 7fb449d9b640 1 --2- 172.21.15.49:0/2893787051 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb44c150940 0x7fb44c1f3100 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb43400d810 tx=0x7fb43400dcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:06.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.417+0000 7fb442ffd640 1 -- 172.21.15.49:0/2893787051 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4340074e0 con 0x7fb44c150940 2023-11-06T02:09:06.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.417+0000 7fb442ffd640 1 -- 172.21.15.49:0/2893787051 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb434007640 con 0x7fb44c150940 2023-11-06T02:09:06.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.417+0000 7fb45084c640 1 -- 172.21.15.49:0/2893787051 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb44c1519a0 con 0x7fb44c150940 2023-11-06T02:09:06.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.417+0000 7fb442ffd640 1 -- 172.21.15.49:0/2893787051 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb434004a10 con 0x7fb44c150940 2023-11-06T02:09:06.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.417+0000 7fb45084c640 1 -- 172.21.15.49:0/2893787051 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb44c156550 con 0x7fb44c150940 2023-11-06T02:09:06.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.417+0000 7fb45084c640 1 -- 172.21.15.49:0/2893787051 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb44c0b0de0 con 0x7fb44c150940 2023-11-06T02:09:06.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.417+0000 7fb442ffd640 1 -- 172.21.15.49:0/2893787051 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fb434004b70 con 0x7fb44c150940 2023-11-06T02:09:06.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.421+0000 7fb442ffd640 1 --2- 172.21.15.49:0/2893787051 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb42c065a90 0x7fb42c067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:06.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.421+0000 7fb442ffd640 1 -- 172.21.15.49:0/2893787051 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb434082e40 con 0x7fb44c150940 2023-11-06T02:09:06.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.421+0000 7fb44a59c640 1 --2- 172.21.15.49:0/2893787051 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb42c065a90 0x7fb42c067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:06.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.421+0000 7fb44a59c640 1 --2- 172.21.15.49:0/2893787051 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb42c065a90 0x7fb42c067f50 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb4440af000 tx=0x7fb4440a7040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:06.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.425+0000 7fb442ffd640 1 -- 172.21.15.49:0/2893787051 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fb4340506a0 con 0x7fb44c150940 2023-11-06T02:09:06.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.577+0000 7fb45084c640 1 -- 172.21.15.49:0/2893787051 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb44c151b90 con 0x7fb44c150940 2023-11-06T02:09:06.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.581+0000 7fb442ffd640 1 -- 172.21.15.49:0/2893787051 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+17811 (secure 0 0 0) 0x7fb434019090 con 0x7fb44c150940 2023-11-06T02:09:06.579 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:09:06.579 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":23,"fsid":"1a8b035c-6ba4-499a-b876-aea47fd9c0d4","created":"2023-11-06T02:08:06.440857+0000","modified":"2023-11-06T02:08:31.594964+0000","last_up_change":"2023-11-06T02:08:17.982372+0000","last_in_change":"2023-11-06T02:08:08.906631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T02:08:19.252222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T02:08:24.776211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-06T02:08:28.936178+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-06T02:08:29.563468+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ad55af8-0571-4e6a-8722-b5d429b7496d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6809","nonce":38116},{"type":"v1","addr":"172.21.15.49:6811","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6813","nonce":38116},{"type":"v1","addr":"172.21.15.49:6815","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6821","nonce":38116},{"type":"v1","addr":"172.21.15.49:6823","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6817","nonce":38116},{"type":"v1","addr":"172.21.15.49:6819","nonce":38116}]},"public_addr":"172.21.15.49:6811/38116","cluster_addr":"172.21.15.49:6815/38116","heartbeat_back_addr":"172.21.15.49:6823/38116","heartbeat_front_addr":"172.21.15.49:6819/38116","state":["exists","up"]},{"osd":1,"uuid":"c8b31f85-a757-43ed-9419-351acc46611b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":38113},{"type":"v1","addr":"172.21.15.49:6810","nonce":38113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":38113},{"type":"v1","addr":"172.21.15.49:6814","nonce":38113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":38113},{"type":"v1","addr":"172.21.15.49:6822","nonce":38113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":38113},{"type":"v1","addr":"172.21.15.49:6818","nonce":38113}]},"public_addr":"172.21.15.49:6810/38113","cluster_addr":"172.21.15.49:6814/38113","heartbeat_back_addr":"172.21.15.49:6822/38113","heartbeat_front_addr":"172.21.15.49:6818/38113","state":["exists","up"]},{"osd":2,"uuid":"555b3bf3-0a70-4b91-8efd-e4204b5fc26e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":38114},{"type":"v1","addr":"172.21.15.49:6801","nonce":38114}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":38114},{"type":"v1","addr":"172.21.15.49:6803","nonce":38114}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":38114},{"type":"v1","addr":"172.21.15.49:6807","nonce":38114}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":38114},{"type":"v1","addr":"172.21.15.49:6805","nonce":38114}]},"public_addr":"172.21.15.49:6801/38114","cluster_addr":"172.21.15.49:6803/38114","heartbeat_back_addr":"172.21.15.49:6807/38114","heartbeat_front_addr":"172.21.15.49:6805/38114","state":["exists","up"]},{"osd":3,"uuid":"8d4e5d77-03ef-4b9b-bfcb-5ae93a46795c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":38115},{"type":"v1","addr":"172.21.15.49:6825","nonce":38115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":38115},{"type":"v1","addr":"172.21.15.49:6827","nonce":38115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":38115},{"type":"v1","addr":"172.21.15.49:6831","nonce":38115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":38115},{"type":"v1","addr":"172.21.15.49:6829","nonce":38115}]},"public_addr":"172.21.15.49:6825/38115","cluster_addr":"172.21.15.49:6827/38115","heartbeat_back_addr":"172.21.15.49:6831/38115","heartbeat_front_addr":"172.21.15.49:6829/38115","state":["exists","up"]},{"osd":4,"uuid":"dba03ad4-68a3-4cf3-9218-ac611f02eacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":38417},{"type":"v1","addr":"172.21.15.150:6809","nonce":38417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":38417},{"type":"v1","addr":"172.21.15.150:6811","nonce":38417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":38417},{"type":"v1","addr":"172.21.15.150:6815","nonce":38417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":38417},{"type":"v1","addr":"172.21.15.150:6813","nonce":38417}]},"public_addr":"172.21.15.150:6809/38417","cluster_addr":"172.21.15.150:6811/38417","heartbeat_back_addr":"172.21.15.150:6815/38417","heartbeat_front_addr":"172.21.15.150:6813/38417","state":["exists","up"]},{"osd":5,"uuid":"0df9c303-c255-4640-8e91-01f1c3efe18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":38418},{"type":"v1","addr":"172.21.15.150:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":38418},{"type":"v1","addr":"172.21.15.150:6803","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":38418},{"type":"v1","addr":"172.21.15.150:6807","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":38418},{"type":"v1","addr":"172.21.15.150:6805","nonce":38418}]},"public_addr":"172.21.15.150:6801/38418","cluster_addr":"172.21.15.150:6803/38418","heartbeat_back_addr":"172.21.15.150:6807/38418","heartbeat_front_addr":"172.21.15.150:6805/38418","state":["exists","up"]},{"osd":6,"uuid":"e59fa189-4b70-4c06-9722-c02b90828bf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":38428},{"type":"v1","addr":"172.21.15.150:6825","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":38428},{"type":"v1","addr":"172.21.15.150:6827","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":38428},{"type":"v1","addr":"172.21.15.150:6831","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":38428},{"type":"v1","addr":"172.21.15.150:6829","nonce":38428}]},"public_addr":"172.21.15.150:6825/38428","cluster_addr":"172.21.15.150:6827/38428","heartbeat_back_addr":"172.21.15.150:6831/38428","heartbeat_front_addr":"172.21.15.150:6829/38428","state":["exists","up"]},{"osd":7,"uuid":"a64f57e1-ef98-46bd-b304-385759e95898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":38421},{"type":"v1","addr":"172.21.15.150:6817","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":38421},{"type":"v1","addr":"172.21.15.150:6819","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":38421},{"type":"v1","addr":"172.21.15.150:6823","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":38421},{"type":"v1","addr":"172.21.15.150:6821","nonce":38421}]},"public_addr":"172.21.15.150:6817/38421","cluster_addr":"172.21.15.150:6819/38421","heartbeat_back_addr":"172.21.15.150:6823/38421","heartbeat_front_addr":"172.21.15.150:6821/38421","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.438552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.413280+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.296466+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.424966+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.276953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.230984+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.410235+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.480084+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T02:09:06.582 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.585+0000 7fb45084c640 1 -- 172.21.15.49:0/2893787051 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb42c065a90 msgr2=0x7fb42c067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:06.582 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.585+0000 7fb45084c640 1 --2- 172.21.15.49:0/2893787051 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb42c065a90 0x7fb42c067f50 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb4440af000 tx=0x7fb4440a7040 comp rx=0 tx=0).stop 2023-11-06T02:09:06.583 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.585+0000 7fb45084c640 1 -- 172.21.15.49:0/2893787051 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb44c150940 msgr2=0x7fb44c1f3100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:06.583 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.585+0000 7fb45084c640 1 --2- 172.21.15.49:0/2893787051 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb44c150940 0x7fb44c1f3100 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb43400d810 tx=0x7fb43400dcd0 comp rx=0 tx=0).stop 2023-11-06T02:09:06.583 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.585+0000 7fb45084c640 1 -- 172.21.15.49:0/2893787051 shutdown_connections 2023-11-06T02:09:06.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.585+0000 7fb45084c640 1 --2- 172.21.15.49:0/2893787051 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb42c065a90 0x7fb42c067f50 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:06.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.585+0000 7fb45084c640 1 --2- 172.21.15.49:0/2893787051 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb44c150940 0x7fb44c1f3100 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:06.585 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.585+0000 7fb45084c640 1 --2- 172.21.15.49:0/2893787051 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb44c1f3640 0x7fb44c0afaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:06.585 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.585+0000 7fb45084c640 1 --2- 172.21.15.49:0/2893787051 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb438003980 0x7fb44c155e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:06.585 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.585+0000 7fb45084c640 1 -- 172.21.15.49:0/2893787051 >> 172.21.15.49:0/2893787051 conn(0x7fb44c06c0a0 msgr2=0x7fb44c159b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:06.585 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.585+0000 7fb45084c640 1 -- 172.21.15.49:0/2893787051 shutdown_connections 2023-11-06T02:09:06.585 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.585+0000 7fb45084c640 1 -- 172.21.15.49:0/2893787051 wait complete. 2023-11-06T02:09:06.598 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T02:08:19.252222+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T02:08:24.776211+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T02:08:28.936178+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T02:08:29.563468+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T02:09:06.600 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T02:09:06.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.721+0000 7fe025cb6640 1 Processor -- start 2023-11-06T02:09:06.718 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.721+0000 7fe025cb6640 1 -- start start 2023-11-06T02:09:06.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.721+0000 7fe025cb6640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fe02015a880 0x7fe0200b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:06.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.721+0000 7fe025cb6640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe020070160 0x7fe020070610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:06.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.721+0000 7fe025cb6640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0200ae360 con 0x7fe02006d150 2023-11-06T02:09:06.721 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.721+0000 7fe025cb6640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe020070b50 con 0x7fe020070160 2023-11-06T02:09:06.721 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.721+0000 7fe025cb6640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0200ae640 con 0x7fe02015a880 2023-11-06T02:09:06.721 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.721+0000 7fe01effd640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe020070160 0x7fe020070610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:06.721 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.721+0000 7fe01effd640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe020070160 0x7fe020070610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47806/0 (socket says 172.21.15.49:47806) 2023-11-06T02:09:06.722 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.721+0000 7fe01effd640 1 -- 172.21.15.49:0/3380753859 learned_addr learned my addr 172.21.15.49:0/3380753859 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:06.722 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.721+0000 7fe01f7fe640 1 --2- 172.21.15.49:0/3380753859 >> v2:172.21.15.150:3300/0 conn(0x7fe02015a880 0x7fe0200b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:06.722 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.721+0000 7fe01effd640 1 -- 172.21.15.49:0/3380753859 >> v2:172.21.15.150:3300/0 conn(0x7fe02015a880 msgr2=0x7fe0200b15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:06.722 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.721+0000 7fe01effd640 1 --2- 172.21.15.49:0/3380753859 >> v2:172.21.15.150:3300/0 conn(0x7fe02015a880 0x7fe0200b15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:06.723 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.721+0000 7fe01effd640 1 -- 172.21.15.49:0/3380753859 >> v1:172.21.15.150:6789/0 conn(0x7fe02006d150 legacy=0x7fe02006d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:06.723 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.721+0000 7fe01effd640 1 -- 172.21.15.49:0/3380753859 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0200ae920 con 0x7fe020070160 2023-11-06T02:09:06.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.725+0000 7fe01effd640 1 --2- 172.21.15.49:0/3380753859 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe020070160 0x7fe020070610 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe0200cafc0 tx=0x7fe01401c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6820cc4562044fa5 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:06.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.725+0000 7fe01e7fc640 1 -- 172.21.15.49:0/3380753859 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0140ac630 con 0x7fe020070160 2023-11-06T02:09:06.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.725+0000 7fe01e7fc640 1 -- 172.21.15.49:0/3380753859 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0140ac790 con 0x7fe020070160 2023-11-06T02:09:06.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.725+0000 7fe01e7fc640 1 -- 172.21.15.49:0/3380753859 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0140acb20 con 0x7fe020070160 2023-11-06T02:09:06.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.725+0000 7fe025cb6640 1 -- 172.21.15.49:0/3380753859 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe020070160 msgr2=0x7fe020070610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:06.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.725+0000 7fe025cb6640 1 --2- 172.21.15.49:0/3380753859 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe020070160 0x7fe020070610 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe0200cafc0 tx=0x7fe01401c4c0 comp rx=0 tx=0).stop 2023-11-06T02:09:06.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.725+0000 7fe025cb6640 1 -- 172.21.15.49:0/3380753859 shutdown_connections 2023-11-06T02:09:06.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.725+0000 7fe025cb6640 1 --2- 172.21.15.49:0/3380753859 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe020070160 0x7fe020070610 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:06.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.725+0000 7fe025cb6640 1 --2- 172.21.15.49:0/3380753859 >> v2:172.21.15.150:3300/0 conn(0x7fe02015a880 0x7fe0200b15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:06.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.725+0000 7fe025cb6640 1 -- 172.21.15.49:0/3380753859 >> 172.21.15.49:0/3380753859 conn(0x7fe02006a370 msgr2=0x7fe0200ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:06.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.725+0000 7fe025cb6640 1 -- 172.21.15.49:0/3380753859 shutdown_connections 2023-11-06T02:09:06.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.725+0000 7fe025cb6640 1 -- 172.21.15.49:0/3380753859 wait complete. 2023-11-06T02:09:06.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.725+0000 7fe025cb6640 1 Processor -- start 2023-11-06T02:09:06.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.725+0000 7fe025cb6640 1 -- start start 2023-11-06T02:09:06.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.725+0000 7fe025cb6640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe02006d150 0x7fe0201f3340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:06.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.725+0000 7fe025cb6640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe020070160 0x7fe0200cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:06.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.725+0000 7fe025cb6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe02015a880 0x7fe0200c4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:06.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.725+0000 7fe025cb6640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0201f44a0 con 0x7fe02015a880 2023-11-06T02:09:06.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.725+0000 7fe025cb6640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0201f3920 con 0x7fe020070160 2023-11-06T02:09:06.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.725+0000 7fe025cb6640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0201f3c00 con 0x7fe02006d150 2023-11-06T02:09:06.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.725+0000 7fe01ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe02015a880 0x7fe0200c4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:06.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.725+0000 7fe01ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe02015a880 0x7fe0200c4ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47816/0 (socket says 172.21.15.49:47816) 2023-11-06T02:09:06.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.725+0000 7fe01ffff640 1 -- 172.21.15.49:0/721004145 learned_addr learned my addr 172.21.15.49:0/721004145 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:06.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.725+0000 7fe01f7fe640 1 --2- 172.21.15.49:0/721004145 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe02006d150 0x7fe0201f3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:06.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.725+0000 7fe01effd640 1 --2- 172.21.15.49:0/721004145 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe020070160 0x7fe0200cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:06.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.725+0000 7fe01ffff640 1 -- 172.21.15.49:0/721004145 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe02006d150 msgr2=0x7fe0201f3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:06.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.725+0000 7fe01ffff640 1 --2- 172.21.15.49:0/721004145 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe02006d150 0x7fe0201f3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:06.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.725+0000 7fe01ffff640 1 -- 172.21.15.49:0/721004145 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe020070160 msgr2=0x7fe0200cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:06.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.725+0000 7fe01ffff640 1 --2- 172.21.15.49:0/721004145 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe020070160 0x7fe0200cdc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:06.736 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.725+0000 7fe01ffff640 1 -- 172.21.15.49:0/721004145 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0140094d0 con 0x7fe02015a880 2023-11-06T02:09:06.736 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.729+0000 7fe01ffff640 1 --2- 172.21.15.49:0/721004145 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe02015a880 0x7fe0200c4ee0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe01800c780 tx=0x7fe01800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:06.736 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.729+0000 7fe007fff640 1 -- 172.21.15.49:0/721004145 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe018003d90 con 0x7fe02015a880 2023-11-06T02:09:06.736 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.729+0000 7fe007fff640 1 -- 172.21.15.49:0/721004145 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe01800ce90 con 0x7fe02015a880 2023-11-06T02:09:06.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.729+0000 7fe025cb6640 1 -- 172.21.15.49:0/721004145 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0201f3ee0 con 0x7fe02015a880 2023-11-06T02:09:06.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.729+0000 7fe007fff640 1 -- 172.21.15.49:0/721004145 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0180052b0 con 0x7fe02015a880 2023-11-06T02:09:06.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.729+0000 7fe025cb6640 1 -- 172.21.15.49:0/721004145 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe02014e3d0 con 0x7fe02015a880 2023-11-06T02:09:06.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.729+0000 7fe025cb6640 1 -- 172.21.15.49:0/721004145 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe02014cc70 con 0x7fe02015a880 2023-11-06T02:09:06.738 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.729+0000 7fe007fff640 1 -- 172.21.15.49:0/721004145 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fe018005410 con 0x7fe02015a880 2023-11-06T02:09:06.738 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.729+0000 7fe007fff640 1 --2- 172.21.15.49:0/721004145 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe00c065a60 0x7fe00c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:06.738 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.729+0000 7fe007fff640 1 -- 172.21.15.49:0/721004145 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe018082770 con 0x7fe02015a880 2023-11-06T02:09:06.738 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.729+0000 7fe01f7fe640 1 --2- 172.21.15.49:0/721004145 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe00c065a60 0x7fe00c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:06.738 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.729+0000 7fe01f7fe640 1 --2- 172.21.15.49:0/721004145 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe00c065a60 0x7fe00c067f20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe01000a8d0 tx=0x7fe010008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:06.739 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.733+0000 7fe007fff640 1 -- 172.21.15.49:0/721004145 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fe02014cc70 con 0x7fe02015a880 2023-11-06T02:09:06.881 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.881+0000 7fe025cb6640 1 -- 172.21.15.49:0/721004145 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe02014cc70 con 0x7fe02015a880 2023-11-06T02:09:06.881 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.885+0000 7fe007fff640 1 -- 172.21.15.49:0/721004145 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe018017020 con 0x7fe02015a880 2023-11-06T02:09:06.882 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2023-11-06T02:09:06.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.885+0000 7fe025cb6640 1 -- 172.21.15.49:0/721004145 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe00c065a60 msgr2=0x7fe00c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:06.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.885+0000 7fe025cb6640 1 --2- 172.21.15.49:0/721004145 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe00c065a60 0x7fe00c067f20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe01000a8d0 tx=0x7fe010008040 comp rx=0 tx=0).stop 2023-11-06T02:09:06.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.885+0000 7fe025cb6640 1 -- 172.21.15.49:0/721004145 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe02015a880 msgr2=0x7fe0200c4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:06.885 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.885+0000 7fe025cb6640 1 --2- 172.21.15.49:0/721004145 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe02015a880 0x7fe0200c4ee0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe01800c780 tx=0x7fe01800cc40 comp rx=0 tx=0).stop 2023-11-06T02:09:06.885 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.885+0000 7fe025cb6640 1 -- 172.21.15.49:0/721004145 shutdown_connections 2023-11-06T02:09:06.885 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.885+0000 7fe025cb6640 1 --2- 172.21.15.49:0/721004145 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe00c065a60 0x7fe00c067f20 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:06.885 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.885+0000 7fe025cb6640 1 --2- 172.21.15.49:0/721004145 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe02015a880 0x7fe0200c4ee0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:06.886 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.885+0000 7fe025cb6640 1 --2- 172.21.15.49:0/721004145 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe020070160 0x7fe0200cdc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:06.886 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.885+0000 7fe025cb6640 1 --2- 172.21.15.49:0/721004145 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe02006d150 0x7fe0201f3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:06.886 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.885+0000 7fe025cb6640 1 -- 172.21.15.49:0/721004145 >> 172.21.15.49:0/721004145 conn(0x7fe02006a370 msgr2=0x7fe02015b880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:06.886 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.885+0000 7fe025cb6640 1 -- 172.21.15.49:0/721004145 shutdown_connections 2023-11-06T02:09:06.886 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:06.885+0000 7fe025cb6640 1 -- 172.21.15.49:0/721004145 wait complete. 2023-11-06T02:09:06.899 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T02:09:07.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.025+0000 7f7f3178d640 1 Processor -- start 2023-11-06T02:09:07.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.025+0000 7f7f3178d640 1 -- start start 2023-11-06T02:09:07.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.025+0000 7f7f3178d640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f7f2c15dd10 0x7f7f2c0cd190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:07.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.025+0000 7f7f3178d640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f2c15ad50 con 0x7f7f2c0caec0 2023-11-06T02:09:07.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.025+0000 7f7f3178d640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f2c0cecf0 con 0x7f7f2c0c9600 2023-11-06T02:09:07.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.025+0000 7f7f3178d640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f2c15e280 con 0x7f7f2c15dd10 2023-11-06T02:09:07.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.025+0000 7f7f2affd640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f7f2c0caec0 0x7f7f2c0cb2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:48500/0 (socket says 172.21.15.49:48500) 2023-11-06T02:09:07.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.025+0000 7f7f2affd640 1 -- 172.21.15.49:0/2579404397 learned_addr learned my addr 172.21.15.49:0/2579404397 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:09:07.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.025+0000 7f7f2b7fe640 1 --2- 172.21.15.49:0/2579404397 >> v2:172.21.15.150:3300/0 conn(0x7f7f2c15dd10 0x7f7f2c0cd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:07.026 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.025+0000 7f7f2a7fc640 1 -- 172.21.15.49:0/2579404397 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1595211935 0 0) 0x7f7f1c001120 con 0x7f7f2c0caec0 2023-11-06T02:09:07.026 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.025+0000 7f7f2a7fc640 1 -- 172.21.15.49:0/2579404397 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f18003530 con 0x7f7f2c0caec0 2023-11-06T02:09:07.026 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.025+0000 7f7f2a7fc640 1 -- 172.21.15.49:0/2579404397 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3199105207 0 0) 0x7f7f18003530 con 0x7f7f2c0caec0 2023-11-06T02:09:07.026 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.025+0000 7f7f2a7fc640 1 -- 172.21.15.49:0/2579404397 >> v2:172.21.15.150:3300/0 conn(0x7f7f2c15dd10 msgr2=0x7f7f2c0cd190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:07.026 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.025+0000 7f7f2a7fc640 1 --2- 172.21.15.49:0/2579404397 >> v2:172.21.15.150:3300/0 conn(0x7f7f2c15dd10 0x7f7f2c0cd190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:07.027 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.025+0000 7f7f2a7fc640 1 -- 172.21.15.49:0/2579404397 >> v1:172.21.15.150:6789/0 conn(0x7f7f2c0c9600 legacy=0x7f7f2c069d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:07.027 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.025+0000 7f7f2a7fc640 1 -- 172.21.15.49:0/2579404397 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f2c0ce730 con 0x7f7f2c0caec0 2023-11-06T02:09:07.027 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.025+0000 7f7f2a7fc640 1 -- 172.21.15.49:0/2579404397 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7f7f1c001810 con 0x7f7f2c0caec0 2023-11-06T02:09:07.027 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.025+0000 7f7f2a7fc640 1 -- 172.21.15.49:0/2579404397 >> v1:172.21.15.49:6789/0 conn(0x7f7f2c0caec0 legacy=0x7f7f2c0cb2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:07.028 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.025+0000 7f7f2a7fc640 1 --2- 172.21.15.49:0/2579404397 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f18003980 0x7f7f18003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:07.028 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.025+0000 7f7f2a7fc640 1 --2- 172.21.15.49:0/2579404397 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7f180047d0 0x7f7f18004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:07.028 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.025+0000 7f7f2a7fc640 1 --2- 172.21.15.49:0/2579404397 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f18005190 0x7f7f1800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:07.028 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.025+0000 7f7f2a7fc640 1 -- 172.21.15.49:0/2579404397 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f2c15e280 con 0x7f7f18005190 2023-11-06T02:09:07.029 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.025+0000 7f7f2a7fc640 1 -- 172.21.15.49:0/2579404397 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f1800c130 con 0x7f7f18003980 2023-11-06T02:09:07.029 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.025+0000 7f7f2a7fc640 1 -- 172.21.15.49:0/2579404397 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f1800c2f0 con 0x7f7f180047d0 2023-11-06T02:09:07.029 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.025+0000 7f7f2b7fe640 1 --2- 172.21.15.49:0/2579404397 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f18005190 0x7f7f1800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:07.029 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.025+0000 7f7f2affd640 1 --2- 172.21.15.49:0/2579404397 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f18003980 0x7f7f18003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:07.029 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.025+0000 7f7f227fc640 1 --2- 172.21.15.49:0/2579404397 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7f180047d0 0x7f7f18004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:07.029 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.025+0000 7f7f2b7fe640 1 -- 172.21.15.49:0/2579404397 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7f180047d0 msgr2=0x7f7f18004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:07.030 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.025+0000 7f7f2b7fe640 1 --2- 172.21.15.49:0/2579404397 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7f180047d0 0x7f7f18004c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:07.030 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.025+0000 7f7f2b7fe640 1 -- 172.21.15.49:0/2579404397 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f18003980 msgr2=0x7f7f18003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:07.030 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.025+0000 7f7f2b7fe640 1 --2- 172.21.15.49:0/2579404397 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f18003980 0x7f7f18003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:07.030 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.025+0000 7f7f2b7fe640 1 -- 172.21.15.49:0/2579404397 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f18003530 con 0x7f7f18005190 2023-11-06T02:09:07.034 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.029+0000 7f7f2b7fe640 1 -- 172.21.15.49:0/2579404397 reap_dead start 2023-11-06T02:09:07.034 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.029+0000 7f7f2b7fe640 1 --2- 172.21.15.49:0/2579404397 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f18005190 0x7f7f1800b990 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f7f24009fc0 tx=0x7f7f2401cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0753cd1e0057ad6 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:07.035 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.029+0000 7f7f2a7fc640 1 -- 172.21.15.49:0/2579404397 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f240a5cb0 con 0x7f7f18005190 2023-11-06T02:09:07.035 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.029+0000 7f7f2a7fc640 1 -- 172.21.15.49:0/2579404397 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f2400f040 con 0x7f7f18005190 2023-11-06T02:09:07.035 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.029+0000 7f7f2a7fc640 1 -- 172.21.15.49:0/2579404397 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f24003be0 con 0x7f7f18005190 2023-11-06T02:09:07.035 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.029+0000 7f7f3178d640 1 -- 172.21.15.49:0/2579404397 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f18005190 msgr2=0x7f7f1800b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:07.036 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.029+0000 7f7f3178d640 1 --2- 172.21.15.49:0/2579404397 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f18005190 0x7f7f1800b990 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f7f24009fc0 tx=0x7f7f2401cca0 comp rx=0 tx=0).stop 2023-11-06T02:09:07.036 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.029+0000 7f7f3178d640 1 -- 172.21.15.49:0/2579404397 shutdown_connections 2023-11-06T02:09:07.036 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.029+0000 7f7f3178d640 1 --2- 172.21.15.49:0/2579404397 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f18005190 0x7f7f1800b990 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:07.036 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.029+0000 7f7f3178d640 1 -- 172.21.15.49:0/2579404397 >> 172.21.15.49:0/2579404397 conn(0x7f7f2c0b0aa0 msgr2=0x7f7f2c0b0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:07.036 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.029+0000 7f7f3178d640 1 -- 172.21.15.49:0/2579404397 shutdown_connections 2023-11-06T02:09:07.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.029+0000 7f7f3178d640 1 -- 172.21.15.49:0/2579404397 wait complete. 2023-11-06T02:09:07.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.029+0000 7f7f3178d640 1 Processor -- start 2023-11-06T02:09:07.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.029+0000 7f7f3178d640 1 -- start start 2023-11-06T02:09:07.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.029+0000 7f7f3178d640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7f18005190 0x7f7f2c1f59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:07.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.029+0000 7f7f3178d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f2c15f410 0x7f7f2c15f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:07.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.029+0000 7f7f3178d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f2c0c49e0 0x7f7f2c0c4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:07.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.029+0000 7f7f3178d640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f2c15fdd0 con 0x7f7f2c0c49e0 2023-11-06T02:09:07.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.029+0000 7f7f3178d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f2c0b2a00 con 0x7f7f2c15f410 2023-11-06T02:09:07.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.029+0000 7f7f3178d640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f2c0c56d0 con 0x7f7f18005190 2023-11-06T02:09:07.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.029+0000 7f7f2b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f2c0c49e0 0x7f7f2c0c4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:07.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.029+0000 7f7f2b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f2c0c49e0 0x7f7f2c0c4e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47828/0 (socket says 172.21.15.49:47828) 2023-11-06T02:09:07.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.029+0000 7f7f2b7fe640 1 -- 172.21.15.49:0/2576238280 learned_addr learned my addr 172.21.15.49:0/2576238280 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:07.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.029+0000 7f7f227fc640 1 --2- 172.21.15.49:0/2576238280 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f2c15f410 0x7f7f2c15f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:07.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.029+0000 7f7f2affd640 1 --2- 172.21.15.49:0/2576238280 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7f18005190 0x7f7f2c1f59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:07.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.033+0000 7f7f2b7fe640 1 -- 172.21.15.49:0/2576238280 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7f18005190 msgr2=0x7f7f2c1f59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:07.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.033+0000 7f7f2b7fe640 1 --2- 172.21.15.49:0/2576238280 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7f18005190 0x7f7f2c1f59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:07.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.033+0000 7f7f2b7fe640 1 -- 172.21.15.49:0/2576238280 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f2c15f410 msgr2=0x7f7f2c15f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:07.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.033+0000 7f7f2b7fe640 1 --2- 172.21.15.49:0/2576238280 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f2c15f410 0x7f7f2c15f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:07.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.033+0000 7f7f2b7fe640 1 -- 172.21.15.49:0/2576238280 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f2c0b1c40 con 0x7f7f2c0c49e0 2023-11-06T02:09:07.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.033+0000 7f7f2b7fe640 1 --2- 172.21.15.49:0/2576238280 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f2c0c49e0 0x7f7f2c0c4e60 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7f2401c710 tx=0x7f7f24009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:07.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.033+0000 7f7f23fff640 1 -- 172.21.15.49:0/2576238280 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f24003db0 con 0x7f7f2c0c49e0 2023-11-06T02:09:07.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.033+0000 7f7f23fff640 1 -- 172.21.15.49:0/2576238280 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f24004030 con 0x7f7f2c0c49e0 2023-11-06T02:09:07.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.033+0000 7f7f3178d640 1 -- 172.21.15.49:0/2576238280 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f2c0b1f20 con 0x7f7f2c0c49e0 2023-11-06T02:09:07.042 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.033+0000 7f7f23fff640 1 -- 172.21.15.49:0/2576238280 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f240043a0 con 0x7f7f2c0c49e0 2023-11-06T02:09:07.042 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.033+0000 7f7f3178d640 1 -- 172.21.15.49:0/2576238280 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f2c0b24e0 con 0x7f7f2c0c49e0 2023-11-06T02:09:07.042 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.033+0000 7f7f3178d640 1 -- 172.21.15.49:0/2576238280 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f2c06f500 con 0x7f7f2c0c49e0 2023-11-06T02:09:07.042 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.033+0000 7f7f23fff640 1 -- 172.21.15.49:0/2576238280 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f7f24004500 con 0x7f7f2c0c49e0 2023-11-06T02:09:07.042 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.033+0000 7f7f23fff640 1 --2- 172.21.15.49:0/2576238280 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7f0c065a90 0x7f7f0c067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:07.043 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.033+0000 7f7f23fff640 1 -- 172.21.15.49:0/2576238280 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7f2411ed30 con 0x7f7f2c0c49e0 2023-11-06T02:09:07.043 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.033+0000 7f7f2affd640 1 --2- 172.21.15.49:0/2576238280 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7f0c065a90 0x7f7f0c067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:07.043 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.033+0000 7f7f2affd640 1 --2- 172.21.15.49:0/2576238280 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7f0c065a90 0x7f7f0c067f50 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c001280 tx=0x7f7f1c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:07.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.037+0000 7f7f23fff640 1 -- 172.21.15.49:0/2576238280 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f7f240ec570 con 0x7f7f2c0c49e0 2023-11-06T02:09:07.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.193+0000 7f7f3178d640 1 -- 172.21.15.49:0/2576238280 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f7f2c0b1620 con 0x7f7f2c0c49e0 2023-11-06T02:09:07.192 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.193+0000 7f7f23fff640 1 -- 172.21.15.49:0/2576238280 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f7f240efb90 con 0x7f7f2c0c49e0 2023-11-06T02:09:07.192 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 8 2023-11-06T02:09:07.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.197+0000 7f7f3178d640 1 -- 172.21.15.49:0/2576238280 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7f0c065a90 msgr2=0x7f7f0c067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:07.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.197+0000 7f7f3178d640 1 --2- 172.21.15.49:0/2576238280 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7f0c065a90 0x7f7f0c067f50 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c001280 tx=0x7f7f1c009040 comp rx=0 tx=0).stop 2023-11-06T02:09:07.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.197+0000 7f7f3178d640 1 -- 172.21.15.49:0/2576238280 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f2c0c49e0 msgr2=0x7f7f2c0c4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:07.195 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.197+0000 7f7f3178d640 1 --2- 172.21.15.49:0/2576238280 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f2c0c49e0 0x7f7f2c0c4e60 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7f2401c710 tx=0x7f7f24009750 comp rx=0 tx=0).stop 2023-11-06T02:09:07.195 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.197+0000 7f7f3178d640 1 -- 172.21.15.49:0/2576238280 shutdown_connections 2023-11-06T02:09:07.195 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.197+0000 7f7f3178d640 1 --2- 172.21.15.49:0/2576238280 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7f0c065a90 0x7f7f0c067f50 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:07.195 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.197+0000 7f7f3178d640 1 --2- 172.21.15.49:0/2576238280 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f2c0c49e0 0x7f7f2c0c4e60 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:07.195 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.197+0000 7f7f3178d640 1 --2- 172.21.15.49:0/2576238280 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f2c15f410 0x7f7f2c15f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:07.196 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.197+0000 7f7f3178d640 1 --2- 172.21.15.49:0/2576238280 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7f18005190 0x7f7f2c1f59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:07.196 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.197+0000 7f7f3178d640 1 -- 172.21.15.49:0/2576238280 >> 172.21.15.49:0/2576238280 conn(0x7f7f2c0b0aa0 msgr2=0x7f7f2c15aef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:07.196 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.197+0000 7f7f3178d640 1 -- 172.21.15.49:0/2576238280 shutdown_connections 2023-11-06T02:09:07.196 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.197+0000 7f7f3178d640 1 -- 172.21.15.49:0/2576238280 wait complete. 2023-11-06T02:09:07.210 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T02:09:07.326 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.329+0000 7f51b5cb5640 1 Processor -- start 2023-11-06T02:09:07.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.329+0000 7f51b5cb5640 1 -- start start 2023-11-06T02:09:07.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.329+0000 7f51b5cb5640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f51b015cbb0 0x7f51b00d1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:07.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.329+0000 7f51b5cb5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51b00d3170 0x7f51b00b5d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:07.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.329+0000 7f51b5cb5640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51b015cf90 con 0x7f51b00d24e0 2023-11-06T02:09:07.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.329+0000 7f51b5cb5640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51b00d0b80 con 0x7f51b015cbb0 2023-11-06T02:09:07.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.329+0000 7f51b5cb5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51b00d0e60 con 0x7f51b00d3170 2023-11-06T02:09:07.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.329+0000 7f51af7fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f51b015cbb0 0x7f51b00d1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:07.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.329+0000 7f51af7fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f51b015cbb0 0x7f51b00d1fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47844/0 (socket says 172.21.15.49:47844) 2023-11-06T02:09:07.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.329+0000 7f51af7fe640 1 -- 172.21.15.49:0/1595058356 learned_addr learned my addr 172.21.15.49:0/1595058356 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:07.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.329+0000 7f51affff640 1 --2- 172.21.15.49:0/1595058356 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51b00d3170 0x7f51b00b5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:07.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.329+0000 7f51af7fe640 1 -- 172.21.15.49:0/1595058356 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51b00d3170 msgr2=0x7f51b00b5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:07.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.329+0000 7f51af7fe640 1 --2- 172.21.15.49:0/1595058356 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51b00d3170 0x7f51b00b5d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:07.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.329+0000 7f51af7fe640 1 -- 172.21.15.49:0/1595058356 >> v1:172.21.15.150:6789/0 conn(0x7f51b00d24e0 legacy=0x7f51b00d2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:07.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.329+0000 7f51af7fe640 1 -- 172.21.15.49:0/1595058356 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51b00d1140 con 0x7f51b015cbb0 2023-11-06T02:09:07.335 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.333+0000 7f51af7fe640 1 --2- 172.21.15.49:0/1595058356 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51b015cbb0 0x7f51b00d1fa0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f51a0009870 tx=0x7f51a001ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55bcde306e4c6a10 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:07.335 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.333+0000 7f51ae7fc640 1 -- 172.21.15.49:0/1595058356 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a00a89b0 con 0x7f51b015cbb0 2023-11-06T02:09:07.335 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.333+0000 7f51ae7fc640 1 -- 172.21.15.49:0/1595058356 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51a00a8b10 con 0x7f51b015cbb0 2023-11-06T02:09:07.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.333+0000 7f51ae7fc640 1 -- 172.21.15.49:0/1595058356 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a00a8ea0 con 0x7f51b015cbb0 2023-11-06T02:09:07.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.333+0000 7f51b5cb5640 1 -- 172.21.15.49:0/1595058356 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51b015cbb0 msgr2=0x7f51b00d1fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:07.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.333+0000 7f51b5cb5640 1 --2- 172.21.15.49:0/1595058356 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51b015cbb0 0x7f51b00d1fa0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f51a0009870 tx=0x7f51a001ecf0 comp rx=0 tx=0).stop 2023-11-06T02:09:07.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.333+0000 7f51b5cb5640 1 -- 172.21.15.49:0/1595058356 shutdown_connections 2023-11-06T02:09:07.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.333+0000 7f51b5cb5640 1 --2- 172.21.15.49:0/1595058356 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51b00d3170 0x7f51b00b5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:07.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.333+0000 7f51b5cb5640 1 --2- 172.21.15.49:0/1595058356 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51b015cbb0 0x7f51b00d1fa0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:07.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.333+0000 7f51b5cb5640 1 -- 172.21.15.49:0/1595058356 >> 172.21.15.49:0/1595058356 conn(0x7f51b00718a0 msgr2=0x7f51b0071cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:07.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.333+0000 7f51b5cb5640 1 -- 172.21.15.49:0/1595058356 shutdown_connections 2023-11-06T02:09:07.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.333+0000 7f51b5cb5640 1 -- 172.21.15.49:0/1595058356 wait complete. 2023-11-06T02:09:07.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.333+0000 7f51b5cb5640 1 Processor -- start 2023-11-06T02:09:07.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.333+0000 7f51b5cb5640 1 -- start start 2023-11-06T02:09:07.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.337+0000 7f51b5cb5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51b00d24e0 0x7f51b015fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:07.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.337+0000 7f51b5cb5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51b00d3170 0x7f51b015a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:07.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.337+0000 7f51b5cb5640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51b015cbb0 0x7f51b015ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:07.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.337+0000 7f51b5cb5640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b015ea90 con 0x7f51b00d3170 2023-11-06T02:09:07.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.337+0000 7f51b5cb5640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b015ed70 con 0x7f51b015cbb0 2023-11-06T02:09:07.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.337+0000 7f51b5cb5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51b015f050 con 0x7f51b00d24e0 2023-11-06T02:09:07.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.337+0000 7f51aeffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51b00d3170 0x7f51b015a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:07.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.337+0000 7f51aeffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51b00d3170 0x7f51b015a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47850/0 (socket says 172.21.15.49:47850) 2023-11-06T02:09:07.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.337+0000 7f51aeffd640 1 -- 172.21.15.49:0/1537647160 learned_addr learned my addr 172.21.15.49:0/1537647160 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:07.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.337+0000 7f51affff640 1 --2- 172.21.15.49:0/1537647160 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51b015cbb0 0x7f51b015ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:07.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.337+0000 7f51af7fe640 1 --2- 172.21.15.49:0/1537647160 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51b00d24e0 0x7f51b015fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:07.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.337+0000 7f51aeffd640 1 -- 172.21.15.49:0/1537647160 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51b00d24e0 msgr2=0x7f51b015fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:07.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.337+0000 7f51aeffd640 1 --2- 172.21.15.49:0/1537647160 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51b00d24e0 0x7f51b015fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:07.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.337+0000 7f51aeffd640 1 -- 172.21.15.49:0/1537647160 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51b015cbb0 msgr2=0x7f51b015ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:07.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.337+0000 7f51aeffd640 1 --2- 172.21.15.49:0/1537647160 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51b015cbb0 0x7f51b015ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:07.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.337+0000 7f51aeffd640 1 -- 172.21.15.49:0/1537647160 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51a00094d0 con 0x7f51b00d3170 2023-11-06T02:09:07.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.337+0000 7f51aeffd640 1 --2- 172.21.15.49:0/1537647160 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51b00d3170 0x7f51b015a7d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f51a400c410 tx=0x7f51a400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:07.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.337+0000 7f5197fff640 1 -- 172.21.15.49:0/1537647160 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a4011790 con 0x7f51b00d3170 2023-11-06T02:09:07.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.337+0000 7f5197fff640 1 -- 172.21.15.49:0/1537647160 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51a40118f0 con 0x7f51b00d3170 2023-11-06T02:09:07.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.337+0000 7f5197fff640 1 -- 172.21.15.49:0/1537647160 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a4011ce0 con 0x7f51b00d3170 2023-11-06T02:09:07.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.337+0000 7f51b5cb5640 1 -- 172.21.15.49:0/1537647160 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51b015f330 con 0x7f51b00d3170 2023-11-06T02:09:07.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.337+0000 7f51b5cb5640 1 -- 172.21.15.49:0/1537647160 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51b0160070 con 0x7f51b00d3170 2023-11-06T02:09:07.344 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.337+0000 7f51b5cb5640 1 -- 172.21.15.49:0/1537647160 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51b00b7de0 con 0x7f51b00d3170 2023-11-06T02:09:07.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.337+0000 7f5197fff640 1 -- 172.21.15.49:0/1537647160 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f51a401c460 con 0x7f51b00d3170 2023-11-06T02:09:07.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.337+0000 7f5197fff640 1 --2- 172.21.15.49:0/1537647160 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f519c065a60 0x7f519c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:07.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.337+0000 7f5197fff640 1 -- 172.21.15.49:0/1537647160 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f51a4082fa0 con 0x7f51b00d3170 2023-11-06T02:09:07.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.337+0000 7f51af7fe640 1 --2- 172.21.15.49:0/1537647160 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f519c065a60 0x7f519c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:07.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.341+0000 7f51af7fe640 1 --2- 172.21.15.49:0/1537647160 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f519c065a60 0x7f519c067f20 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f51a00b2000 tx=0x7f51a00aa070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:07.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.341+0000 7f5197fff640 1 -- 172.21.15.49:0/1537647160 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f51a40b4070 con 0x7f51b00d3170 2023-11-06T02:09:07.488 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.489+0000 7f51b5cb5640 1 -- 172.21.15.49:0/1537647160 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f51b0074040 con 0x7f51b00d3170 2023-11-06T02:09:07.488 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.489+0000 7f5197fff640 1 -- 172.21.15.49:0/1537647160 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f51a4050890 con 0x7f51b00d3170 2023-11-06T02:09:07.489 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2023-11-06T02:09:07.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.493+0000 7f51b5cb5640 1 -- 172.21.15.49:0/1537647160 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f519c065a60 msgr2=0x7f519c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:07.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.493+0000 7f51b5cb5640 1 --2- 172.21.15.49:0/1537647160 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f519c065a60 0x7f519c067f20 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f51a00b2000 tx=0x7f51a00aa070 comp rx=0 tx=0).stop 2023-11-06T02:09:07.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.493+0000 7f51b5cb5640 1 -- 172.21.15.49:0/1537647160 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51b00d3170 msgr2=0x7f51b015a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:07.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.493+0000 7f51b5cb5640 1 --2- 172.21.15.49:0/1537647160 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51b00d3170 0x7f51b015a7d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f51a400c410 tx=0x7f51a400c8d0 comp rx=0 tx=0).stop 2023-11-06T02:09:07.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.493+0000 7f51b5cb5640 1 -- 172.21.15.49:0/1537647160 shutdown_connections 2023-11-06T02:09:07.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.493+0000 7f51b5cb5640 1 --2- 172.21.15.49:0/1537647160 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f519c065a60 0x7f519c067f20 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:07.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.493+0000 7f51b5cb5640 1 --2- 172.21.15.49:0/1537647160 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51b00d3170 0x7f51b015a7d0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:07.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.493+0000 7f51b5cb5640 1 --2- 172.21.15.49:0/1537647160 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51b015cbb0 0x7f51b015ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:07.493 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.493+0000 7f51b5cb5640 1 --2- 172.21.15.49:0/1537647160 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51b00d24e0 0x7f51b015fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:07.493 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.493+0000 7f51b5cb5640 1 -- 172.21.15.49:0/1537647160 >> 172.21.15.49:0/1537647160 conn(0x7f51b00718a0 msgr2=0x7f51b00cc1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:07.493 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.493+0000 7f51b5cb5640 1 -- 172.21.15.49:0/1537647160 shutdown_connections 2023-11-06T02:09:07.493 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.493+0000 7f51b5cb5640 1 -- 172.21.15.49:0/1537647160 wait complete. 2023-11-06T02:09:07.506 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T02:09:07.621 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.625+0000 7fdcfee78640 1 Processor -- start 2023-11-06T02:09:07.622 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.625+0000 7fdcfee78640 1 -- start start 2023-11-06T02:09:07.623 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.625+0000 7fdcfee78640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdcf815a880 0x7fdcf80b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:07.624 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.625+0000 7fdcfee78640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fdcf8070160 0x7fdcf80705e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:07.624 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.625+0000 7fdcfee78640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fdcf806d150 0x7fdcf806d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:07.624 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.625+0000 7fdcfee78640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcf8070b20 con 0x7fdcf8070160 2023-11-06T02:09:07.624 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.625+0000 7fdcfee78640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcf806db20 con 0x7fdcf806d150 2023-11-06T02:09:07.624 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.625+0000 7fdcfee78640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf80ae360 con 0x7fdcf815a880 2023-11-06T02:09:07.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.625+0000 7fdcf7fff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fdcf8070160 0x7fdcf80705e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:07.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.625+0000 7fdcf7fff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fdcf8070160 0x7fdcf80705e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47854/0 (socket says 172.21.15.49:47854) 2023-11-06T02:09:07.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.625+0000 7fdcf7fff640 1 -- 172.21.15.49:0/3925901299 learned_addr learned my addr 172.21.15.49:0/3925901299 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:07.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.625+0000 7fdcfcc14640 1 --2- 172.21.15.49:0/3925901299 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdcf815a880 0x7fdcf80b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:07.626 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.625+0000 7fdcfd415640 1 --2- 172.21.15.49:0/3925901299 >> v2:172.21.15.150:3300/0 conn(0x7fdcf806d150 0x7fdcf806d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:07.626 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.625+0000 7fdcf7fff640 1 -- 172.21.15.49:0/3925901299 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdcf815a880 msgr2=0x7fdcf80b15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:07.626 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.625+0000 7fdcf7fff640 1 --2- 172.21.15.49:0/3925901299 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdcf815a880 0x7fdcf80b15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:07.626 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.625+0000 7fdcf7fff640 1 -- 172.21.15.49:0/3925901299 >> v2:172.21.15.150:3300/0 conn(0x7fdcf806d150 msgr2=0x7fdcf806d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:07.626 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.625+0000 7fdcf7fff640 1 --2- 172.21.15.49:0/3925901299 >> v2:172.21.15.150:3300/0 conn(0x7fdcf806d150 0x7fdcf806d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:07.627 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.625+0000 7fdcf7fff640 1 -- 172.21.15.49:0/3925901299 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcf80ae640 con 0x7fdcf8070160 2023-11-06T02:09:07.627 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.625+0000 7fdcfd415640 1 --2- 172.21.15.49:0/3925901299 >> v2:172.21.15.150:3300/0 conn(0x7fdcf806d150 0x7fdcf806d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:09:07.630 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.629+0000 7fdcf7fff640 1 --2- 172.21.15.49:0/3925901299 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcf8070160 0x7fdcf80705e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fdcf80cf1b0 tx=0x7fdcec01c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c4562b2ad9185fd server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:07.630 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.629+0000 7fdcf77fe640 1 -- 172.21.15.49:0/3925901299 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcec0ac630 con 0x7fdcf8070160 2023-11-06T02:09:07.631 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.629+0000 7fdcf77fe640 1 -- 172.21.15.49:0/3925901299 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcec0ac790 con 0x7fdcf8070160 2023-11-06T02:09:07.631 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.629+0000 7fdcf77fe640 1 -- 172.21.15.49:0/3925901299 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcec0acb20 con 0x7fdcf8070160 2023-11-06T02:09:07.631 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.629+0000 7fdcfee78640 1 -- 172.21.15.49:0/3925901299 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcf8070160 msgr2=0x7fdcf80705e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:07.631 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.629+0000 7fdcfee78640 1 --2- 172.21.15.49:0/3925901299 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcf8070160 0x7fdcf80705e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fdcf80cf1b0 tx=0x7fdcec01c500 comp rx=0 tx=0).stop 2023-11-06T02:09:07.632 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.629+0000 7fdcfee78640 1 -- 172.21.15.49:0/3925901299 shutdown_connections 2023-11-06T02:09:07.632 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.629+0000 7fdcfee78640 1 --2- 172.21.15.49:0/3925901299 >> v2:172.21.15.150:3300/0 conn(0x7fdcf806d150 0x7fdcf806d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:07.632 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.629+0000 7fdcfee78640 1 --2- 172.21.15.49:0/3925901299 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcf8070160 0x7fdcf80705e0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:07.632 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.629+0000 7fdcfee78640 1 --2- 172.21.15.49:0/3925901299 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdcf815a880 0x7fdcf80b15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:07.633 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.629+0000 7fdcfee78640 1 -- 172.21.15.49:0/3925901299 >> 172.21.15.49:0/3925901299 conn(0x7fdcf806a370 msgr2=0x7fdcf80ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:07.633 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.629+0000 7fdcfee78640 1 -- 172.21.15.49:0/3925901299 shutdown_connections 2023-11-06T02:09:07.633 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.629+0000 7fdcfee78640 1 -- 172.21.15.49:0/3925901299 wait complete. 2023-11-06T02:09:07.633 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.629+0000 7fdcfee78640 1 Processor -- start 2023-11-06T02:09:07.633 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.629+0000 7fdcfee78640 1 -- start start 2023-11-06T02:09:07.634 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.629+0000 7fdcfee78640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdcf806d150 0x7fdcf8139720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:07.634 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.629+0000 7fdcfee78640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdcf8070160 0x7fdcf814fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:07.634 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.633+0000 7fdcfee78640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcf815a880 0x7fdcf814ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:07.634 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.633+0000 7fdcfee78640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf8139c60 con 0x7fdcf815a880 2023-11-06T02:09:07.634 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.633+0000 7fdcfee78640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf80cb020 con 0x7fdcf8070160 2023-11-06T02:09:07.635 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.633+0000 7fdcfee78640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf80cbff0 con 0x7fdcf806d150 2023-11-06T02:09:07.635 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.633+0000 7fdcfd415640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcf815a880 0x7fdcf814ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:07.635 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.633+0000 7fdcfd415640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcf815a880 0x7fdcf814ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47862/0 (socket says 172.21.15.49:47862) 2023-11-06T02:09:07.635 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.633+0000 7fdcfd415640 1 -- 172.21.15.49:0/1819153433 learned_addr learned my addr 172.21.15.49:0/1819153433 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:07.635 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.633+0000 7fdcfcc14640 1 --2- 172.21.15.49:0/1819153433 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdcf806d150 0x7fdcf8139720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:07.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.633+0000 7fdcf7fff640 1 --2- 172.21.15.49:0/1819153433 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdcf8070160 0x7fdcf814fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:07.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.633+0000 7fdcfd415640 1 -- 172.21.15.49:0/1819153433 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdcf806d150 msgr2=0x7fdcf8139720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:07.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.633+0000 7fdcfd415640 1 --2- 172.21.15.49:0/1819153433 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdcf806d150 0x7fdcf8139720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:07.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.633+0000 7fdcfd415640 1 -- 172.21.15.49:0/1819153433 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdcf8070160 msgr2=0x7fdcf814fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:07.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.633+0000 7fdcfd415640 1 --2- 172.21.15.49:0/1819153433 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdcf8070160 0x7fdcf814fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:07.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.633+0000 7fdcfd415640 1 -- 172.21.15.49:0/1819153433 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcec0094d0 con 0x7fdcf815a880 2023-11-06T02:09:07.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.633+0000 7fdcfd415640 1 --2- 172.21.15.49:0/1819153433 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcf815a880 0x7fdcf814ff90 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fdcf000cf20 tx=0x7fdcf000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:07.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.633+0000 7fdcf57fa640 1 -- 172.21.15.49:0/1819153433 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf000d840 con 0x7fdcf815a880 2023-11-06T02:09:07.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.633+0000 7fdcf57fa640 1 -- 172.21.15.49:0/1819153433 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcf000d9a0 con 0x7fdcf815a880 2023-11-06T02:09:07.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.633+0000 7fdcf57fa640 1 -- 172.21.15.49:0/1819153433 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf000dd50 con 0x7fdcf815a880 2023-11-06T02:09:07.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.633+0000 7fdcfee78640 1 -- 172.21.15.49:0/1819153433 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcf815ade0 con 0x7fdcf815a880 2023-11-06T02:09:07.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.633+0000 7fdcfee78640 1 -- 172.21.15.49:0/1819153433 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcf80cb510 con 0x7fdcf815a880 2023-11-06T02:09:07.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.633+0000 7fdcfee78640 1 -- 172.21.15.49:0/1819153433 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcf8074040 con 0x7fdcf815a880 2023-11-06T02:09:07.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.633+0000 7fdcf57fa640 1 -- 172.21.15.49:0/1819153433 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fdcf001e3b0 con 0x7fdcf815a880 2023-11-06T02:09:07.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.633+0000 7fdcf57fa640 1 --2- 172.21.15.49:0/1819153433 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fdce4065a60 0x7fdce4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:07.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.633+0000 7fdcf57fa640 1 -- 172.21.15.49:0/1819153433 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdcf0083cf0 con 0x7fdcf815a880 2023-11-06T02:09:07.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.633+0000 7fdcfcc14640 1 --2- 172.21.15.49:0/1819153433 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fdce4065a60 0x7fdce4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:07.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.637+0000 7fdcfcc14640 1 --2- 172.21.15.49:0/1819153433 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fdce4065a60 0x7fdce4067f20 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fdcf8151df0 tx=0x7fdce8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:07.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.637+0000 7fdcf57fa640 1 -- 172.21.15.49:0/1819153433 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fdcf0088070 con 0x7fdcf815a880 2023-11-06T02:09:07.790 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.793+0000 7fdcfee78640 1 -- 172.21.15.49:0/1819153433 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdcf80ce580 con 0x7fdcf815a880 2023-11-06T02:09:07.790 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.793+0000 7fdcf57fa640 1 -- 172.21.15.49:0/1819153433 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdcf001a0a0 con 0x7fdcf815a880 2023-11-06T02:09:07.791 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2023-11-06T02:09:07.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.793+0000 7fdcfee78640 1 -- 172.21.15.49:0/1819153433 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fdce4065a60 msgr2=0x7fdce4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:07.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.793+0000 7fdcfee78640 1 --2- 172.21.15.49:0/1819153433 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fdce4065a60 0x7fdce4067f20 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fdcf8151df0 tx=0x7fdce8008040 comp rx=0 tx=0).stop 2023-11-06T02:09:07.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.793+0000 7fdcfee78640 1 -- 172.21.15.49:0/1819153433 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcf815a880 msgr2=0x7fdcf814ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:07.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.793+0000 7fdcfee78640 1 --2- 172.21.15.49:0/1819153433 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcf815a880 0x7fdcf814ff90 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fdcf000cf20 tx=0x7fdcf000a710 comp rx=0 tx=0).stop 2023-11-06T02:09:07.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.793+0000 7fdcfee78640 1 -- 172.21.15.49:0/1819153433 shutdown_connections 2023-11-06T02:09:07.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.793+0000 7fdcfee78640 1 --2- 172.21.15.49:0/1819153433 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fdce4065a60 0x7fdce4067f20 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:07.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.793+0000 7fdcfee78640 1 --2- 172.21.15.49:0/1819153433 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcf815a880 0x7fdcf814ff90 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:07.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.793+0000 7fdcfee78640 1 --2- 172.21.15.49:0/1819153433 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdcf8070160 0x7fdcf814fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:07.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.793+0000 7fdcfee78640 1 --2- 172.21.15.49:0/1819153433 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdcf806d150 0x7fdcf8139720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:07.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.793+0000 7fdcfee78640 1 -- 172.21.15.49:0/1819153433 >> 172.21.15.49:0/1819153433 conn(0x7fdcf806a370 msgr2=0x7fdcf81543d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:07.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.793+0000 7fdcfee78640 1 -- 172.21.15.49:0/1819153433 shutdown_connections 2023-11-06T02:09:07.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.793+0000 7fdcfee78640 1 -- 172.21.15.49:0/1819153433 wait complete. 2023-11-06T02:09:07.838 INFO:tasks.cephfs_test_runner:Starting test: test_apply_tag (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2023-11-06T02:09:07.839 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag' 2023-11-06T02:09:07.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.965+0000 7fcdf7713640 1 Processor -- start 2023-11-06T02:09:07.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.965+0000 7fcdf7713640 1 -- start start 2023-11-06T02:09:07.965 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.965+0000 7fcdf7713640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcdf00cb010 0x7fcdf00c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:07.965 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.965+0000 7fcdf7713640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdf00cab80 con 0x7fcdf015ab50 2023-11-06T02:09:07.966 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.965+0000 7fcdf7713640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdf015af30 con 0x7fcdf00b0790 2023-11-06T02:09:07.966 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.965+0000 7fcdf7713640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf00cb5b0 con 0x7fcdf00cb010 2023-11-06T02:09:07.966 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.965+0000 7fcdf54af640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fcdf015ab50 0x7fcdf00b0080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:48508/0 (socket says 172.21.15.49:48508) 2023-11-06T02:09:07.966 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.965+0000 7fcdf54af640 1 -- 172.21.15.49:0/683457764 learned_addr learned my addr 172.21.15.49:0/683457764 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:09:07.966 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.965+0000 7fcdf5cb0640 1 --2- 172.21.15.49:0/683457764 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcdf00cb010 0x7fcdf00c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:07.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.965+0000 7fcde7fff640 1 -- 172.21.15.49:0/683457764 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3763540013 0 0) 0x7fcdf00cab80 con 0x7fcdf015ab50 2023-11-06T02:09:07.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.965+0000 7fcde7fff640 1 -- 172.21.15.49:0/683457764 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcddc003530 con 0x7fcdf015ab50 2023-11-06T02:09:07.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.965+0000 7fcde7fff640 1 -- 172.21.15.49:0/683457764 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3103787902 0 0) 0x7fcddc003530 con 0x7fcdf015ab50 2023-11-06T02:09:07.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.965+0000 7fcde7fff640 1 -- 172.21.15.49:0/683457764 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcdf00cb010 msgr2=0x7fcdf00c9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:07.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.965+0000 7fcde7fff640 1 --2- 172.21.15.49:0/683457764 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcdf00cb010 0x7fcdf00c9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:07.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.965+0000 7fcde7fff640 1 -- 172.21.15.49:0/683457764 >> v1:172.21.15.150:6789/0 conn(0x7fcdf00b0790 legacy=0x7fcdf00c2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:07.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.965+0000 7fcde7fff640 1 -- 172.21.15.49:0/683457764 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdf00cb890 con 0x7fcdf015ab50 2023-11-06T02:09:07.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.965+0000 7fcde7fff640 1 -- 172.21.15.49:0/683457764 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7fcde0001820 con 0x7fcdf015ab50 2023-11-06T02:09:07.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.965+0000 7fcdf5cb0640 1 --2- 172.21.15.49:0/683457764 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcdf00cb010 0x7fcdf00c9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:09:07.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.965+0000 7fcde7fff640 1 -- 172.21.15.49:0/683457764 >> v1:172.21.15.49:6789/0 conn(0x7fcdf015ab50 legacy=0x7fcdf00b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:07.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.965+0000 7fcde7fff640 1 --2- 172.21.15.49:0/683457764 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcddc0039b0 0x7fcddc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:07.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.965+0000 7fcde7fff640 1 --2- 172.21.15.49:0/683457764 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcdf00cb010 0x7fcddc0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:07.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.965+0000 7fcde7fff640 1 --2- 172.21.15.49:0/683457764 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcddc004d10 0x7fcddc005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:07.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.965+0000 7fcde7fff640 1 -- 172.21.15.49:0/683457764 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf00cb5b0 con 0x7fcddc0039b0 2023-11-06T02:09:07.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.965+0000 7fcde7fff640 1 -- 172.21.15.49:0/683457764 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcddc00baa0 con 0x7fcddc004d10 2023-11-06T02:09:07.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.965+0000 7fcde7fff640 1 -- 172.21.15.49:0/683457764 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcddc00bc60 con 0x7fcdf00cb010 2023-11-06T02:09:07.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.965+0000 7fcdf54af640 1 --2- 172.21.15.49:0/683457764 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcddc0039b0 0x7fcddc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:07.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.965+0000 7fcdf5cb0640 1 --2- 172.21.15.49:0/683457764 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcddc004d10 0x7fcddc005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:07.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.965+0000 7fcdf4cae640 1 --2- 172.21.15.49:0/683457764 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcdf00cb010 0x7fcddc0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:07.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.965+0000 7fcdf54af640 1 -- 172.21.15.49:0/683457764 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcdf00cb010 msgr2=0x7fcddc0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:07.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.965+0000 7fcdf54af640 1 --2- 172.21.15.49:0/683457764 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcdf00cb010 0x7fcddc0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:07.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.965+0000 7fcdf54af640 1 -- 172.21.15.49:0/683457764 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcddc004d10 msgr2=0x7fcddc005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:07.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.965+0000 7fcdf54af640 1 --2- 172.21.15.49:0/683457764 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcddc004d10 0x7fcddc005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:07.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.965+0000 7fcdf54af640 1 -- 172.21.15.49:0/683457764 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcddc003530 con 0x7fcddc0039b0 2023-11-06T02:09:07.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.965+0000 7fcdf4cae640 1 --2- 172.21.15.49:0/683457764 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcdf00cb010 0x7fcddc0047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:09:07.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.969+0000 7fcdf54af640 1 --2- 172.21.15.49:0/683457764 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcddc0039b0 0x7fcddc003dd0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fcde0001ec0 tx=0x7fcde001cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef8d72aca0953338 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:07.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.969+0000 7fcde7fff640 1 -- 172.21.15.49:0/683457764 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde0007270 con 0x7fcddc0039b0 2023-11-06T02:09:07.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.969+0000 7fcde7fff640 1 -- 172.21.15.49:0/683457764 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcde00073d0 con 0x7fcddc0039b0 2023-11-06T02:09:07.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.969+0000 7fcde7fff640 1 -- 172.21.15.49:0/683457764 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde00a5810 con 0x7fcddc0039b0 2023-11-06T02:09:07.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.969+0000 7fcdf7713640 1 -- 172.21.15.49:0/683457764 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcddc0039b0 msgr2=0x7fcddc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:07.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.969+0000 7fcdf7713640 1 --2- 172.21.15.49:0/683457764 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcddc0039b0 0x7fcddc003dd0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fcde0001ec0 tx=0x7fcde001cb30 comp rx=0 tx=0).stop 2023-11-06T02:09:07.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.969+0000 7fcdf5cb0640 1 -- 172.21.15.49:0/683457764 reap_dead start 2023-11-06T02:09:07.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.969+0000 7fcdf7713640 1 -- 172.21.15.49:0/683457764 shutdown_connections 2023-11-06T02:09:07.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.969+0000 7fcdf7713640 1 -- 172.21.15.49:0/683457764 >> 172.21.15.49:0/683457764 conn(0x7fcdf006c0a0 msgr2=0x7fcdf006c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:07.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.969+0000 7fcdf7713640 1 -- 172.21.15.49:0/683457764 shutdown_connections 2023-11-06T02:09:07.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.969+0000 7fcdf7713640 1 -- 172.21.15.49:0/683457764 wait complete. 2023-11-06T02:09:07.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.969+0000 7fcdf7713640 1 Processor -- start 2023-11-06T02:09:07.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.969+0000 7fcdf7713640 1 -- start start 2023-11-06T02:09:07.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.973+0000 7fcdf7713640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcddc0039b0 0x7fcdf00c0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:07.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.973+0000 7fcdf7713640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcdf0151de0 0x7fcdf0071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:07.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.973+0000 7fcdf7713640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcdf00716c0 0x7fcdf00c71c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:07.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.973+0000 7fcdf7713640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf0071b40 con 0x7fcddc0039b0 2023-11-06T02:09:07.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.973+0000 7fcdf7713640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf00c1160 con 0x7fcdf00716c0 2023-11-06T02:09:07.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.973+0000 7fcdf7713640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf0150d40 con 0x7fcdf0151de0 2023-11-06T02:09:07.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.973+0000 7fcdf54af640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcddc0039b0 0x7fcdf00c0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:07.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.973+0000 7fcdf54af640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcddc0039b0 0x7fcdf00c0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47882/0 (socket says 172.21.15.49:47882) 2023-11-06T02:09:07.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.973+0000 7fcdf54af640 1 -- 172.21.15.49:0/324372678 learned_addr learned my addr 172.21.15.49:0/324372678 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:07.980 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.973+0000 7fcdf54af640 1 -- 172.21.15.49:0/324372678 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcdf0151de0 msgr2=0x7fcdf0071180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:07.980 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.973+0000 7fcdf5cb0640 1 --2- 172.21.15.49:0/324372678 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcdf00716c0 0x7fcdf00c71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:07.980 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.973+0000 7fcdf54af640 1 --2- 172.21.15.49:0/324372678 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcdf0151de0 0x7fcdf0071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:07.980 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.973+0000 7fcdf54af640 1 -- 172.21.15.49:0/324372678 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcdf00716c0 msgr2=0x7fcdf00c71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:07.980 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.973+0000 7fcdf54af640 1 --2- 172.21.15.49:0/324372678 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcdf00716c0 0x7fcdf00c71c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:07.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.973+0000 7fcdf54af640 1 -- 172.21.15.49:0/324372678 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdf0151020 con 0x7fcddc0039b0 2023-11-06T02:09:07.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.973+0000 7fcdf54af640 1 --2- 172.21.15.49:0/324372678 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcddc0039b0 0x7fcdf00c0c20 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fcde001c960 tx=0x7fcddc005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:07.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.973+0000 7fcde5ffb640 1 -- 172.21.15.49:0/324372678 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde00b4070 con 0x7fcddc0039b0 2023-11-06T02:09:07.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.973+0000 7fcde5ffb640 1 -- 172.21.15.49:0/324372678 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcde0000f80 con 0x7fcddc0039b0 2023-11-06T02:09:07.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.973+0000 7fcde5ffb640 1 -- 172.21.15.49:0/324372678 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde00070a0 con 0x7fcddc0039b0 2023-11-06T02:09:07.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.973+0000 7fcdf7713640 1 -- 172.21.15.49:0/324372678 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdf0151300 con 0x7fcddc0039b0 2023-11-06T02:09:07.982 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.973+0000 7fcdf7713640 1 -- 172.21.15.49:0/324372678 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdf01518c0 con 0x7fcddc0039b0 2023-11-06T02:09:07.982 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.973+0000 7fcdf7713640 1 -- 172.21.15.49:0/324372678 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdf00b0de0 con 0x7fcddc0039b0 2023-11-06T02:09:07.982 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.973+0000 7fcde5ffb640 1 -- 172.21.15.49:0/324372678 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fcde00015e0 con 0x7fcddc0039b0 2023-11-06T02:09:07.982 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.973+0000 7fcde5ffb640 1 --2- 172.21.15.49:0/324372678 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fcdd4065a90 0x7fcdd4067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:07.982 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.973+0000 7fcde5ffb640 1 -- 172.21.15.49:0/324372678 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcde011c840 con 0x7fcddc0039b0 2023-11-06T02:09:07.983 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.973+0000 7fcdf4cae640 1 --2- 172.21.15.49:0/324372678 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fcdd4065a90 0x7fcdd4067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:07.983 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.977+0000 7fcdf4cae640 1 --2- 172.21.15.49:0/324372678 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fcdd4065a90 0x7fcdd4067f50 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fcdf00c4450 tx=0x7fcdec009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:07.983 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:07.977+0000 7fcde5ffb640 1 -- 172.21.15.49:0/324372678 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fcde00ea000 con 0x7fcddc0039b0 2023-11-06T02:09:08.182 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:08.185+0000 7fcdf7713640 1 -- 172.21.15.49:0/324372678 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]} v 0) v1 -- 0x7fcdf0074040 con 0x7fcddc0039b0 2023-11-06T02:09:09.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.169+0000 7fcde5ffb640 1 -- 172.21.15.49:0/324372678 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]}]=0 v57) v1 ==== 145+0+0 (secure 0 0 0) 0x7fcde00bd0d0 con 0x7fcddc0039b0 2023-11-06T02:09:09.171 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.173+0000 7fcdf7713640 1 -- 172.21.15.49:0/324372678 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fcdd4065a90 msgr2=0x7fcdd4067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:09.171 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.173+0000 7fcdf7713640 1 --2- 172.21.15.49:0/324372678 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fcdd4065a90 0x7fcdd4067f50 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fcdf00c4450 tx=0x7fcdec009040 comp rx=0 tx=0).stop 2023-11-06T02:09:09.171 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.173+0000 7fcdf7713640 1 -- 172.21.15.49:0/324372678 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcddc0039b0 msgr2=0x7fcdf00c0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:09.172 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.173+0000 7fcdf7713640 1 --2- 172.21.15.49:0/324372678 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcddc0039b0 0x7fcdf00c0c20 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fcde001c960 tx=0x7fcddc005800 comp rx=0 tx=0).stop 2023-11-06T02:09:09.172 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.173+0000 7fcdf7713640 1 -- 172.21.15.49:0/324372678 shutdown_connections 2023-11-06T02:09:09.172 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.173+0000 7fcdf7713640 1 --2- 172.21.15.49:0/324372678 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fcdd4065a90 0x7fcdd4067f50 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:09.172 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.173+0000 7fcdf7713640 1 --2- 172.21.15.49:0/324372678 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcdf00716c0 0x7fcdf00c71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:09.172 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.173+0000 7fcdf7713640 1 --2- 172.21.15.49:0/324372678 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcdf0151de0 0x7fcdf0071180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:09.173 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.173+0000 7fcdf7713640 1 --2- 172.21.15.49:0/324372678 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcddc0039b0 0x7fcdf00c0c20 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:09.173 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.173+0000 7fcdf7713640 1 -- 172.21.15.49:0/324372678 >> 172.21.15.49:0/324372678 conn(0x7fcdf006c0a0 msgr2=0x7fcdf00af080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:09.173 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.173+0000 7fcdf7713640 1 -- 172.21.15.49:0/324372678 shutdown_connections 2023-11-06T02:09:09.173 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.173+0000 7fcdf7713640 1 -- 172.21.15.49:0/324372678 wait complete. 2023-11-06T02:09:09.191 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-06T02:09:09.322 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.325+0000 7f0a3979e640 1 Processor -- start 2023-11-06T02:09:09.322 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.325+0000 7f0a3979e640 1 -- start start 2023-11-06T02:09:09.325 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.325+0000 7f0a3979e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0a340d08b0 0x7f0a340d0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:09.325 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.325+0000 7f0a3979e640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a340d1230 con 0x7f0a3415ab60 2023-11-06T02:09:09.325 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.325+0000 7f0a3979e640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a340adf50 con 0x7f0a340aeff0 2023-11-06T02:09:09.325 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.325+0000 7f0a3979e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a340cb340 con 0x7f0a340d08b0 2023-11-06T02:09:09.326 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.325+0000 7f0a32ffd640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f0a3415ab60 0x7f0a340ad520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:60762/0 (socket says 172.21.15.49:60762) 2023-11-06T02:09:09.326 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.325+0000 7f0a32ffd640 1 -- 172.21.15.49:0/409501324 learned_addr learned my addr 172.21.15.49:0/409501324 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:09:09.326 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.325+0000 7f0a31ffb640 1 -- 172.21.15.49:0/409501324 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 731124761 0 0) 0x7f0a340d1230 con 0x7f0a3415ab60 2023-11-06T02:09:09.326 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.325+0000 7f0a31ffb640 1 -- 172.21.15.49:0/409501324 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a1c003530 con 0x7f0a3415ab60 2023-11-06T02:09:09.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.325+0000 7f0a327fc640 1 --2- 172.21.15.49:0/409501324 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0a340d08b0 0x7f0a340d0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:09.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.325+0000 7f0a31ffb640 1 -- 172.21.15.49:0/409501324 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3049994421 0 0) 0x7f0a1c003530 con 0x7f0a3415ab60 2023-11-06T02:09:09.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.325+0000 7f0a31ffb640 1 -- 172.21.15.49:0/409501324 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0a340d08b0 msgr2=0x7f0a340d0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:09.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.325+0000 7f0a31ffb640 1 --2- 172.21.15.49:0/409501324 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0a340d08b0 0x7f0a340d0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:09.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.325+0000 7f0a31ffb640 1 -- 172.21.15.49:0/409501324 >> v1:172.21.15.150:6789/0 conn(0x7f0a340aeff0 legacy=0x7f0a340af480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:09.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.325+0000 7f0a31ffb640 1 -- 172.21.15.49:0/409501324 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a340ae230 con 0x7f0a3415ab60 2023-11-06T02:09:09.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.325+0000 7f0a31ffb640 1 -- 172.21.15.49:0/409501324 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7f0a24001680 con 0x7f0a3415ab60 2023-11-06T02:09:09.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.325+0000 7f0a31ffb640 1 -- 172.21.15.49:0/409501324 >> v1:172.21.15.49:6789/0 conn(0x7f0a3415ab60 legacy=0x7f0a340ad520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:09.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.325+0000 7f0a31ffb640 1 --2- 172.21.15.49:0/409501324 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0a340d08b0 0x7f0a1c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:09.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.325+0000 7f0a31ffb640 1 --2- 172.21.15.49:0/409501324 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0a1c003f30 0x7f0a1c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:09.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.325+0000 7f0a31ffb640 1 --2- 172.21.15.49:0/409501324 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a1c004d10 0x7f0a1c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:09.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.325+0000 7f0a31ffb640 1 -- 172.21.15.49:0/409501324 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a340cb340 con 0x7f0a1c003f30 2023-11-06T02:09:09.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.325+0000 7f0a31ffb640 1 -- 172.21.15.49:0/409501324 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a1c00ba70 con 0x7f0a1c004d10 2023-11-06T02:09:09.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.325+0000 7f0a31ffb640 1 -- 172.21.15.49:0/409501324 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a1c00bc30 con 0x7f0a340d08b0 2023-11-06T02:09:09.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.325+0000 7f0a327fc640 1 --2- 172.21.15.49:0/409501324 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0a1c003f30 0x7f0a1c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:09.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.325+0000 7f0a337fe640 1 --2- 172.21.15.49:0/409501324 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a1c004d10 0x7f0a1c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:09.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.325+0000 7f0a32ffd640 1 --2- 172.21.15.49:0/409501324 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0a340d08b0 0x7f0a1c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:09.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.325+0000 7f0a327fc640 1 -- 172.21.15.49:0/409501324 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0a340d08b0 msgr2=0x7f0a1c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:09.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.325+0000 7f0a327fc640 1 --2- 172.21.15.49:0/409501324 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0a340d08b0 0x7f0a1c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:09.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.325+0000 7f0a327fc640 1 -- 172.21.15.49:0/409501324 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a1c004d10 msgr2=0x7f0a1c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:09.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.325+0000 7f0a327fc640 1 --2- 172.21.15.49:0/409501324 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a1c004d10 0x7f0a1c005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:09.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.325+0000 7f0a327fc640 1 -- 172.21.15.49:0/409501324 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a1c003530 con 0x7f0a1c003f30 2023-11-06T02:09:09.335 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.329+0000 7f0a327fc640 1 --2- 172.21.15.49:0/409501324 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0a1c003f30 0x7f0a1c0047d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f0a28009ae0 tx=0x7f0a2801ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec5591821a355a22 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:09.335 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.329+0000 7f0a31ffb640 1 -- 172.21.15.49:0/409501324 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a280a5530 con 0x7f0a1c003f30 2023-11-06T02:09:09.335 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.329+0000 7f0a31ffb640 1 -- 172.21.15.49:0/409501324 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a280a5690 con 0x7f0a1c003f30 2023-11-06T02:09:09.335 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.329+0000 7f0a31ffb640 1 -- 172.21.15.49:0/409501324 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a280a5a40 con 0x7f0a1c003f30 2023-11-06T02:09:09.335 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.329+0000 7f0a3979e640 1 -- 172.21.15.49:0/409501324 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0a1c003f30 msgr2=0x7f0a1c0047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:09.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.329+0000 7f0a3979e640 1 --2- 172.21.15.49:0/409501324 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0a1c003f30 0x7f0a1c0047d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f0a28009ae0 tx=0x7f0a2801ca50 comp rx=0 tx=0).stop 2023-11-06T02:09:09.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.329+0000 7f0a337fe640 1 -- 172.21.15.49:0/409501324 reap_dead start 2023-11-06T02:09:09.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.329+0000 7f0a3979e640 1 -- 172.21.15.49:0/409501324 shutdown_connections 2023-11-06T02:09:09.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.329+0000 7f0a3979e640 1 -- 172.21.15.49:0/409501324 >> 172.21.15.49:0/409501324 conn(0x7f0a34089fd0 msgr2=0x7f0a34099c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:09.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.329+0000 7f0a3979e640 1 -- 172.21.15.49:0/409501324 shutdown_connections 2023-11-06T02:09:09.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.329+0000 7f0a3979e640 1 -- 172.21.15.49:0/409501324 wait complete. 2023-11-06T02:09:09.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.329+0000 7f0a3979e640 1 Processor -- start 2023-11-06T02:09:09.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.329+0000 7f0a3979e640 1 -- start start 2023-11-06T02:09:09.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.333+0000 7f0a3979e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a1c003f30 0x7f0a341f5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:09.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.333+0000 7f0a3979e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0a340c5120 0x7f0a3406c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:09.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.333+0000 7f0a3979e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0a340c4110 0x7f0a341593c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:09.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.333+0000 7f0a3979e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3406afb0 con 0x7f0a340c4110 2023-11-06T02:09:09.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.333+0000 7f0a3979e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a341f60b0 con 0x7f0a1c003f30 2023-11-06T02:09:09.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.333+0000 7f0a3979e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a341f6390 con 0x7f0a340c5120 2023-11-06T02:09:09.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.333+0000 7f0a337fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0a340c4110 0x7f0a341593c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:09.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.333+0000 7f0a337fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0a340c4110 0x7f0a341593c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48064/0 (socket says 172.21.15.49:48064) 2023-11-06T02:09:09.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.333+0000 7f0a337fe640 1 -- 172.21.15.49:0/2276042313 learned_addr learned my addr 172.21.15.49:0/2276042313 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:09.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.333+0000 7f0a32ffd640 1 --2- 172.21.15.49:0/2276042313 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a1c003f30 0x7f0a341f5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:09.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.333+0000 7f0a327fc640 1 --2- 172.21.15.49:0/2276042313 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0a340c5120 0x7f0a3406c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:09.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.333+0000 7f0a337fe640 1 -- 172.21.15.49:0/2276042313 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0a340c5120 msgr2=0x7f0a3406c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:09.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.333+0000 7f0a337fe640 1 --2- 172.21.15.49:0/2276042313 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0a340c5120 0x7f0a3406c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:09.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.333+0000 7f0a337fe640 1 -- 172.21.15.49:0/2276042313 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a1c003f30 msgr2=0x7f0a341f5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:09.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.333+0000 7f0a337fe640 1 --2- 172.21.15.49:0/2276042313 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a1c003f30 0x7f0a341f5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:09.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.333+0000 7f0a337fe640 1 -- 172.21.15.49:0/2276042313 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a341f6670 con 0x7f0a340c4110 2023-11-06T02:09:09.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.333+0000 7f0a327fc640 1 --2- 172.21.15.49:0/2276042313 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0a340c5120 0x7f0a3406c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:09:09.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.333+0000 7f0a337fe640 1 --2- 172.21.15.49:0/2276042313 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0a340c4110 0x7f0a341593c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f0a340ce3f0 tx=0x7f0a2c00ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:09.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.333+0000 7f0a1b7fe640 1 -- 172.21.15.49:0/2276042313 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2c003980 con 0x7f0a340c4110 2023-11-06T02:09:09.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.333+0000 7f0a1b7fe640 1 -- 172.21.15.49:0/2276042313 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a2c003b00 con 0x7f0a340c4110 2023-11-06T02:09:09.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.333+0000 7f0a3979e640 1 -- 172.21.15.49:0/2276042313 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a340c55a0 con 0x7f0a340c4110 2023-11-06T02:09:09.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.333+0000 7f0a1b7fe640 1 -- 172.21.15.49:0/2276042313 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2c0050d0 con 0x7f0a340c4110 2023-11-06T02:09:09.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.333+0000 7f0a3979e640 1 -- 172.21.15.49:0/2276042313 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a3406cd50 con 0x7f0a340c4110 2023-11-06T02:09:09.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.333+0000 7f0a3979e640 1 -- 172.21.15.49:0/2276042313 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a34067140 con 0x7f0a340c4110 2023-11-06T02:09:09.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.333+0000 7f0a1b7fe640 1 -- 172.21.15.49:0/2276042313 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f0a2c005230 con 0x7f0a340c4110 2023-11-06T02:09:09.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.333+0000 7f0a1b7fe640 1 --2- 172.21.15.49:0/2276042313 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0a20065a90 0x7f0a20067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:09.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.333+0000 7f0a1b7fe640 1 -- 172.21.15.49:0/2276042313 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0a2c0831d0 con 0x7f0a340c4110 2023-11-06T02:09:09.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.333+0000 7f0a32ffd640 1 --2- 172.21.15.49:0/2276042313 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0a20065a90 0x7f0a20067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:09.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.337+0000 7f0a32ffd640 1 --2- 172.21.15.49:0/2276042313 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0a20065a90 0x7f0a20067f50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0a240053a0 tx=0x7f0a24009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:09.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.341+0000 7f0a1b7fe640 1 -- 172.21.15.49:0/2276042313 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f0a2c050990 con 0x7f0a340c4110 2023-11-06T02:09:09.493 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.497+0000 7f0a3979e640 1 -- 172.21.15.49:0/2276042313 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f0a340d0670 con 0x7f0a340c4110 2023-11-06T02:09:09.498 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.501+0000 7f0a1b7fe640 1 -- 172.21.15.49:0/2276042313 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0a2c053fb0 con 0x7f0a340c4110 2023-11-06T02:09:09.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.501+0000 7f0a3979e640 1 -- 172.21.15.49:0/2276042313 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0a20065a90 msgr2=0x7f0a20067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:09.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.501+0000 7f0a3979e640 1 --2- 172.21.15.49:0/2276042313 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0a20065a90 0x7f0a20067f50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0a240053a0 tx=0x7f0a24009040 comp rx=0 tx=0).stop 2023-11-06T02:09:09.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.501+0000 7f0a3979e640 1 -- 172.21.15.49:0/2276042313 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0a340c4110 msgr2=0x7f0a341593c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:09.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.501+0000 7f0a3979e640 1 --2- 172.21.15.49:0/2276042313 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0a340c4110 0x7f0a341593c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f0a340ce3f0 tx=0x7f0a2c00ec10 comp rx=0 tx=0).stop 2023-11-06T02:09:09.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.501+0000 7f0a3979e640 1 -- 172.21.15.49:0/2276042313 shutdown_connections 2023-11-06T02:09:09.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.501+0000 7f0a3979e640 1 --2- 172.21.15.49:0/2276042313 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0a20065a90 0x7f0a20067f50 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:09.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.501+0000 7f0a3979e640 1 --2- 172.21.15.49:0/2276042313 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0a340c4110 0x7f0a341593c0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:09.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.501+0000 7f0a3979e640 1 --2- 172.21.15.49:0/2276042313 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0a340c5120 0x7f0a3406c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:09.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.501+0000 7f0a3979e640 1 --2- 172.21.15.49:0/2276042313 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a1c003f30 0x7f0a341f5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:09.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.501+0000 7f0a3979e640 1 -- 172.21.15.49:0/2276042313 >> 172.21.15.49:0/2276042313 conn(0x7f0a34089fd0 msgr2=0x7f0a340af3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:09.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.501+0000 7f0a3979e640 1 -- 172.21.15.49:0/2276042313 shutdown_connections 2023-11-06T02:09:09.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.505+0000 7f0a3979e640 1 -- 172.21.15.49:0/2276042313 wait complete. 2023-11-06T02:09:09.518 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-06T02:09:09.518 DEBUG:teuthology.orchestra.run.smithi049:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T02:09:09.562 DEBUG:teuthology.orchestra.run.smithi150:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T02:09:09.610 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-06T02:09:09.611 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-11-06T02:09:09.611 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T02:09:09.617 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-11-06T02:09:09.617 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T02:09:09.666 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T02:09:09.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.837+0000 7fefaa84f640 1 Processor -- start 2023-11-06T02:09:09.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.837+0000 7fefaa84f640 1 -- start start 2023-11-06T02:09:09.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.841+0000 7fefaa84f640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fefa40b0790 0x7fefa40c8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:09.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.841+0000 7fefaa84f640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fefa40adae0 0x7fefa4070b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:09.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.841+0000 7fefaa84f640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefa41594d0 con 0x7fefa41590f0 2023-11-06T02:09:09.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.841+0000 7fefaa84f640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefa415a140 con 0x7fefa40b0790 2023-11-06T02:09:09.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.841+0000 7fefaa84f640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefa40c92e0 con 0x7fefa40adae0 2023-11-06T02:09:09.839 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.841+0000 7fefa8dec640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fefa40adae0 0x7fefa4070b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:09.839 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.841+0000 7fef9b7fe640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fefa40b0790 0x7fefa40c8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:09.840 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.841+0000 7fefa8dec640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fefa40adae0 0x7fefa4070b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.49:50902/0 (socket says 172.21.15.49:50902) 2023-11-06T02:09:09.840 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.841+0000 7fefa8dec640 1 -- 172.21.15.49:0/883833906 learned_addr learned my addr 172.21.15.49:0/883833906 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:09.840 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.841+0000 7fefa8dec640 1 -- 172.21.15.49:0/883833906 >> v2:172.21.15.150:3300/0 conn(0x7fefa40b0790 msgr2=0x7fefa40c8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:09.840 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.841+0000 7fefa8dec640 1 --2- 172.21.15.49:0/883833906 >> v2:172.21.15.150:3300/0 conn(0x7fefa40b0790 0x7fefa40c8c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:09.840 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.841+0000 7fefa8dec640 1 -- 172.21.15.49:0/883833906 >> v1:172.21.15.150:6789/0 conn(0x7fefa41590f0 legacy=0x7fefa40b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:09.841 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.841+0000 7fefa37fe640 1 -- 172.21.15.49:0/883833906 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 415992054 0 0) 0x7fefa41594d0 con 0x7fefa41590f0 2023-11-06T02:09:09.841 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.841+0000 7fefa8dec640 1 -- 172.21.15.49:0/883833906 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefa41597a0 con 0x7fefa40adae0 2023-11-06T02:09:09.841 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.841+0000 7fef9b7fe640 1 --2- 172.21.15.49:0/883833906 >> v2:172.21.15.150:3300/0 conn(0x7fefa40b0790 0x7fefa40c8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T02:09:09.842 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.841+0000 7fefa8dec640 1 --2- 172.21.15.49:0/883833906 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fefa40adae0 0x7fefa4070b20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fef9c009e50 tx=0x7fef9c01c8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dfefac6ae0860e1f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:09.842 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.845+0000 7fefa37fe640 1 -- 172.21.15.49:0/883833906 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef9c0a5530 con 0x7fefa40adae0 2023-11-06T02:09:09.843 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.845+0000 7fefa37fe640 1 -- 172.21.15.49:0/883833906 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef9c0a5690 con 0x7fefa40adae0 2023-11-06T02:09:09.845 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.845+0000 7fefa37fe640 1 -- 172.21.15.49:0/883833906 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef9c0a5a20 con 0x7fefa40adae0 2023-11-06T02:09:09.845 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.845+0000 7fefaa84f640 1 -- 172.21.15.49:0/883833906 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fefa40adae0 msgr2=0x7fefa4070b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:09.845 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.845+0000 7fefaa84f640 1 --2- 172.21.15.49:0/883833906 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fefa40adae0 0x7fefa4070b20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fef9c009e50 tx=0x7fef9c01c8f0 comp rx=0 tx=0).stop 2023-11-06T02:09:09.846 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.845+0000 7fefaa84f640 1 -- 172.21.15.49:0/883833906 shutdown_connections 2023-11-06T02:09:09.846 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.845+0000 7fefaa84f640 1 --2- 172.21.15.49:0/883833906 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fefa40adae0 0x7fefa4070b20 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:09.846 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.845+0000 7fefaa84f640 1 --2- 172.21.15.49:0/883833906 >> v2:172.21.15.150:3300/0 conn(0x7fefa40b0790 0x7fefa40c8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:09.846 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.845+0000 7fefaa84f640 1 -- 172.21.15.49:0/883833906 >> 172.21.15.49:0/883833906 conn(0x7fefa406c0a0 msgr2=0x7fefa406c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:09.846 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.845+0000 7fefaa84f640 1 -- 172.21.15.49:0/883833906 shutdown_connections 2023-11-06T02:09:09.847 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.845+0000 7fefaa84f640 1 -- 172.21.15.49:0/883833906 wait complete. 2023-11-06T02:09:09.849 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.845+0000 7fefaa84f640 1 Processor -- start 2023-11-06T02:09:09.850 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.845+0000 7fefaa84f640 1 -- start start 2023-11-06T02:09:09.850 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.845+0000 7fefaa84f640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fefa40adae0 0x7fefa41539c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:09.850 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.845+0000 7fefaa84f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fefa40b0790 0x7fefa4152dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:09.850 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.845+0000 7fefaa84f640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fefa41590f0 0x7fefa40c7a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:09.850 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.845+0000 7fefaa84f640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefa40acb10 con 0x7fefa40b0790 2023-11-06T02:09:09.851 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.845+0000 7fefaa84f640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefa4153310 con 0x7fefa40adae0 2023-11-06T02:09:09.851 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.845+0000 7fefaa84f640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefa4156540 con 0x7fefa41590f0 2023-11-06T02:09:09.851 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.845+0000 7fef9b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fefa40b0790 0x7fefa4152dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:09.851 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.849+0000 7fefa3fff640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fefa40adae0 0x7fefa41539c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:09.852 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.849+0000 7fef9b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fefa40b0790 0x7fefa4152dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48078/0 (socket says 172.21.15.49:48078) 2023-11-06T02:09:09.852 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.849+0000 7fef9b7fe640 1 -- 172.21.15.49:0/1619141449 learned_addr learned my addr 172.21.15.49:0/1619141449 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:09.852 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.849+0000 7fefa8dec640 1 --2- 172.21.15.49:0/1619141449 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fefa41590f0 0x7fefa40c7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:09.852 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.849+0000 7fef9b7fe640 1 -- 172.21.15.49:0/1619141449 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fefa41590f0 msgr2=0x7fefa40c7a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:09.853 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.849+0000 7fef9b7fe640 1 --2- 172.21.15.49:0/1619141449 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fefa41590f0 0x7fefa40c7a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:09.853 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.849+0000 7fef9b7fe640 1 -- 172.21.15.49:0/1619141449 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fefa40adae0 msgr2=0x7fefa41539c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:09.853 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.849+0000 7fef9b7fe640 1 --2- 172.21.15.49:0/1619141449 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fefa40adae0 0x7fefa41539c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:09.853 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.849+0000 7fef9b7fe640 1 -- 172.21.15.49:0/1619141449 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef8c009420 con 0x7fefa40b0790 2023-11-06T02:09:09.853 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.849+0000 7fef9b7fe640 1 --2- 172.21.15.49:0/1619141449 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fefa40b0790 0x7fefa4152dd0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fef8c00ebe0 tx=0x7fef8c00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:09.854 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.849+0000 7fefa17fa640 1 -- 172.21.15.49:0/1619141449 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8c00eee0 con 0x7fefa40b0790 2023-11-06T02:09:09.854 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.849+0000 7fefa17fa640 1 -- 172.21.15.49:0/1619141449 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef8c00cc30 con 0x7fefa40b0790 2023-11-06T02:09:09.854 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.849+0000 7fefa17fa640 1 -- 172.21.15.49:0/1619141449 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8c013470 con 0x7fefa40b0790 2023-11-06T02:09:09.854 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.849+0000 7fefaa84f640 1 -- 172.21.15.49:0/1619141449 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef9c009b30 con 0x7fefa40b0790 2023-11-06T02:09:09.855 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.849+0000 7fefaa84f640 1 -- 172.21.15.49:0/1619141449 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefa40c60e0 con 0x7fefa40b0790 2023-11-06T02:09:09.855 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.849+0000 7fefaa84f640 1 -- 172.21.15.49:0/1619141449 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefa4074040 con 0x7fefa40b0790 2023-11-06T02:09:09.855 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.849+0000 7fefa17fa640 1 -- 172.21.15.49:0/1619141449 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fef8c0135d0 con 0x7fefa40b0790 2023-11-06T02:09:09.855 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.849+0000 7fefa17fa640 1 --2- 172.21.15.49:0/1619141449 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fef84065a60 0x7fef84067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:09.855 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.849+0000 7fefa17fa640 1 -- 172.21.15.49:0/1619141449 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fef8c083520 con 0x7fefa40b0790 2023-11-06T02:09:09.856 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.849+0000 7fefa3fff640 1 --2- 172.21.15.49:0/1619141449 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fef84065a60 0x7fef84067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:09.856 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.853+0000 7fefa3fff640 1 --2- 172.21.15.49:0/1619141449 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fef84065a60 0x7fef84067f20 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fefa41554f0 tx=0x7fef94009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:09.856 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:09.857+0000 7fefa17fa640 1 -- 172.21.15.49:0/1619141449 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fef8c050e20 con 0x7fefa40b0790 2023-11-06T02:09:10.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.009+0000 7fefaa84f640 1 -- 172.21.15.49:0/1619141449 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fefa40acd80 con 0x7fefa40b0790 2023-11-06T02:09:10.009 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.013+0000 7fefa17fa640 1 -- 172.21.15.49:0/1619141449 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+17811 (secure 0 0 0) 0x7fefa40acd80 con 0x7fefa40b0790 2023-11-06T02:09:10.009 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:09:10.009 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":23,"fsid":"1a8b035c-6ba4-499a-b876-aea47fd9c0d4","created":"2023-11-06T02:08:06.440857+0000","modified":"2023-11-06T02:08:31.594964+0000","last_up_change":"2023-11-06T02:08:17.982372+0000","last_in_change":"2023-11-06T02:08:08.906631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T02:08:19.252222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T02:08:24.776211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-06T02:08:28.936178+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-06T02:08:29.563468+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ad55af8-0571-4e6a-8722-b5d429b7496d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6809","nonce":38116},{"type":"v1","addr":"172.21.15.49:6811","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6813","nonce":38116},{"type":"v1","addr":"172.21.15.49:6815","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6821","nonce":38116},{"type":"v1","addr":"172.21.15.49:6823","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6817","nonce":38116},{"type":"v1","addr":"172.21.15.49:6819","nonce":38116}]},"public_addr":"172.21.15.49:6811/38116","cluster_addr":"172.21.15.49:6815/38116","heartbeat_back_addr":"172.21.15.49:6823/38116","heartbeat_front_addr":"172.21.15.49:6819/38116","state":["exists","up"]},{"osd":1,"uuid":"c8b31f85-a757-43ed-9419-351acc46611b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":38113},{"type":"v1","addr":"172.21.15.49:6810","nonce":38113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":38113},{"type":"v1","addr":"172.21.15.49:6814","nonce":38113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":38113},{"type":"v1","addr":"172.21.15.49:6822","nonce":38113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":38113},{"type":"v1","addr":"172.21.15.49:6818","nonce":38113}]},"public_addr":"172.21.15.49:6810/38113","cluster_addr":"172.21.15.49:6814/38113","heartbeat_back_addr":"172.21.15.49:6822/38113","heartbeat_front_addr":"172.21.15.49:6818/38113","state":["exists","up"]},{"osd":2,"uuid":"555b3bf3-0a70-4b91-8efd-e4204b5fc26e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":38114},{"type":"v1","addr":"172.21.15.49:6801","nonce":38114}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":38114},{"type":"v1","addr":"172.21.15.49:6803","nonce":38114}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":38114},{"type":"v1","addr":"172.21.15.49:6807","nonce":38114}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":38114},{"type":"v1","addr":"172.21.15.49:6805","nonce":38114}]},"public_addr":"172.21.15.49:6801/38114","cluster_addr":"172.21.15.49:6803/38114","heartbeat_back_addr":"172.21.15.49:6807/38114","heartbeat_front_addr":"172.21.15.49:6805/38114","state":["exists","up"]},{"osd":3,"uuid":"8d4e5d77-03ef-4b9b-bfcb-5ae93a46795c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":38115},{"type":"v1","addr":"172.21.15.49:6825","nonce":38115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":38115},{"type":"v1","addr":"172.21.15.49:6827","nonce":38115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":38115},{"type":"v1","addr":"172.21.15.49:6831","nonce":38115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":38115},{"type":"v1","addr":"172.21.15.49:6829","nonce":38115}]},"public_addr":"172.21.15.49:6825/38115","cluster_addr":"172.21.15.49:6827/38115","heartbeat_back_addr":"172.21.15.49:6831/38115","heartbeat_front_addr":"172.21.15.49:6829/38115","state":["exists","up"]},{"osd":4,"uuid":"dba03ad4-68a3-4cf3-9218-ac611f02eacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":38417},{"type":"v1","addr":"172.21.15.150:6809","nonce":38417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":38417},{"type":"v1","addr":"172.21.15.150:6811","nonce":38417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":38417},{"type":"v1","addr":"172.21.15.150:6815","nonce":38417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":38417},{"type":"v1","addr":"172.21.15.150:6813","nonce":38417}]},"public_addr":"172.21.15.150:6809/38417","cluster_addr":"172.21.15.150:6811/38417","heartbeat_back_addr":"172.21.15.150:6815/38417","heartbeat_front_addr":"172.21.15.150:6813/38417","state":["exists","up"]},{"osd":5,"uuid":"0df9c303-c255-4640-8e91-01f1c3efe18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":38418},{"type":"v1","addr":"172.21.15.150:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":38418},{"type":"v1","addr":"172.21.15.150:6803","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":38418},{"type":"v1","addr":"172.21.15.150:6807","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":38418},{"type":"v1","addr":"172.21.15.150:6805","nonce":38418}]},"public_addr":"172.21.15.150:6801/38418","cluster_addr":"172.21.15.150:6803/38418","heartbeat_back_addr":"172.21.15.150:6807/38418","heartbeat_front_addr":"172.21.15.150:6805/38418","state":["exists","up"]},{"osd":6,"uuid":"e59fa189-4b70-4c06-9722-c02b90828bf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":38428},{"type":"v1","addr":"172.21.15.150:6825","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":38428},{"type":"v1","addr":"172.21.15.150:6827","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":38428},{"type":"v1","addr":"172.21.15.150:6831","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":38428},{"type":"v1","addr":"172.21.15.150:6829","nonce":38428}]},"public_addr":"172.21.15.150:6825/38428","cluster_addr":"172.21.15.150:6827/38428","heartbeat_back_addr":"172.21.15.150:6831/38428","heartbeat_front_addr":"172.21.15.150:6829/38428","state":["exists","up"]},{"osd":7,"uuid":"a64f57e1-ef98-46bd-b304-385759e95898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":38421},{"type":"v1","addr":"172.21.15.150:6817","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":38421},{"type":"v1","addr":"172.21.15.150:6819","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":38421},{"type":"v1","addr":"172.21.15.150:6823","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":38421},{"type":"v1","addr":"172.21.15.150:6821","nonce":38421}]},"public_addr":"172.21.15.150:6817/38421","cluster_addr":"172.21.15.150:6819/38421","heartbeat_back_addr":"172.21.15.150:6823/38421","heartbeat_front_addr":"172.21.15.150:6821/38421","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.438552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.413280+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.296466+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.424966+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.276953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.230984+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.410235+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.480084+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T02:09:10.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.013+0000 7fefaa84f640 1 -- 172.21.15.49:0/1619141449 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fef84065a60 msgr2=0x7fef84067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:10.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.013+0000 7fefaa84f640 1 --2- 172.21.15.49:0/1619141449 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fef84065a60 0x7fef84067f20 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fefa41554f0 tx=0x7fef94009040 comp rx=0 tx=0).stop 2023-11-06T02:09:10.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.013+0000 7fefaa84f640 1 -- 172.21.15.49:0/1619141449 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fefa40b0790 msgr2=0x7fefa4152dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:10.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.013+0000 7fefaa84f640 1 --2- 172.21.15.49:0/1619141449 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fefa40b0790 0x7fefa4152dd0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fef8c00ebe0 tx=0x7fef8c00c520 comp rx=0 tx=0).stop 2023-11-06T02:09:10.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.013+0000 7fefaa84f640 1 -- 172.21.15.49:0/1619141449 shutdown_connections 2023-11-06T02:09:10.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.013+0000 7fefaa84f640 1 --2- 172.21.15.49:0/1619141449 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fef84065a60 0x7fef84067f20 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:10.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.013+0000 7fefaa84f640 1 --2- 172.21.15.49:0/1619141449 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fefa40b0790 0x7fefa4152dd0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:10.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.013+0000 7fefaa84f640 1 --2- 172.21.15.49:0/1619141449 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fefa41590f0 0x7fefa40c7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:10.013 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.013+0000 7fefaa84f640 1 --2- 172.21.15.49:0/1619141449 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fefa40adae0 0x7fefa41539c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:10.013 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.013+0000 7fefaa84f640 1 -- 172.21.15.49:0/1619141449 >> 172.21.15.49:0/1619141449 conn(0x7fefa406c0a0 msgr2=0x7fefa40cd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:10.013 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.013+0000 7fefaa84f640 1 -- 172.21.15.49:0/1619141449 shutdown_connections 2023-11-06T02:09:10.013 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.013+0000 7fefaa84f640 1 -- 172.21.15.49:0/1619141449 wait complete. 2023-11-06T02:09:10.026 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T02:08:19.252222+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T02:08:24.776211+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T02:08:28.936178+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T02:08:29.563468+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T02:09:10.027 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T02:09:10.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.157+0000 7f6571a44640 1 Processor -- start 2023-11-06T02:09:10.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.157+0000 7f6571a44640 1 -- start start 2023-11-06T02:09:10.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.157+0000 7f6571a44640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f656c1595a0 0x7f656c15dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:10.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.157+0000 7f6571a44640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f656c06c5a0 con 0x7f656c06d640 2023-11-06T02:09:10.156 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.157+0000 7f6571a44640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f656c06c880 con 0x7f656c15a990 2023-11-06T02:09:10.156 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.157+0000 7f6571a44640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f656c15e250 con 0x7f656c1595a0 2023-11-06T02:09:10.156 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.157+0000 7f656affd640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f656c1595a0 0x7f656c15dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:10.156 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.157+0000 7f656a7fc640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f656c06d640 0x7f656c0b0220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:60770/0 (socket says 172.21.15.49:60770) 2023-11-06T02:09:10.156 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.157+0000 7f656affd640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f656c1595a0 0x7f656c15dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48080/0 (socket says 172.21.15.49:48080) 2023-11-06T02:09:10.157 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.157+0000 7f656a7fc640 1 -- 172.21.15.49:0/3065134329 learned_addr learned my addr 172.21.15.49:0/3065134329 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:09:10.157 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.157+0000 7f656affd640 1 -- 172.21.15.49:0/3065134329 >> v1:172.21.15.150:6789/0 conn(0x7f656c15a990 legacy=0x7f656c0c8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:10.157 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.157+0000 7f656affd640 1 -- 172.21.15.49:0/3065134329 >> v1:172.21.15.49:6789/0 conn(0x7f656c06d640 legacy=0x7f656c0b0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:10.157 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.157+0000 7f656affd640 1 -- 172.21.15.49:0/3065134329 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f656c06cb60 con 0x7f656c1595a0 2023-11-06T02:09:10.157 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.157+0000 7f6569ffb640 1 -- 172.21.15.49:0/3065134329 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1910568572 0 0) 0x7f656c06c5a0 con 0x7f656c06d640 2023-11-06T02:09:10.158 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.161+0000 7f656affd640 1 --2- 172.21.15.49:0/3065134329 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f656c1595a0 0x7f656c15dd10 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f655c0097c0 tx=0x7f655c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f13e78ab33d22c05 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:10.159 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.161+0000 7f6569ffb640 1 -- 172.21.15.49:0/3065134329 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f655c0ad660 con 0x7f656c1595a0 2023-11-06T02:09:10.159 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.161+0000 7f6569ffb640 1 -- 172.21.15.49:0/3065134329 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f655c0ad7c0 con 0x7f656c1595a0 2023-11-06T02:09:10.159 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.161+0000 7f6569ffb640 1 -- 172.21.15.49:0/3065134329 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f655c0adb50 con 0x7f656c1595a0 2023-11-06T02:09:10.159 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.161+0000 7f6571a44640 1 -- 172.21.15.49:0/3065134329 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f656c1595a0 msgr2=0x7f656c15dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:10.160 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.161+0000 7f6571a44640 1 --2- 172.21.15.49:0/3065134329 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f656c1595a0 0x7f656c15dd10 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f655c0097c0 tx=0x7f655c01c450 comp rx=0 tx=0).stop 2023-11-06T02:09:10.160 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.161+0000 7f6571a44640 1 -- 172.21.15.49:0/3065134329 shutdown_connections 2023-11-06T02:09:10.160 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.161+0000 7f6571a44640 1 --2- 172.21.15.49:0/3065134329 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f656c1595a0 0x7f656c15dd10 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:10.160 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.161+0000 7f6571a44640 1 -- 172.21.15.49:0/3065134329 >> 172.21.15.49:0/3065134329 conn(0x7f656c06a370 msgr2=0x7f656c0ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:10.160 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.161+0000 7f6571a44640 1 -- 172.21.15.49:0/3065134329 shutdown_connections 2023-11-06T02:09:10.161 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.161+0000 7f6571a44640 1 -- 172.21.15.49:0/3065134329 wait complete. 2023-11-06T02:09:10.163 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.161+0000 7f6571a44640 1 Processor -- start 2023-11-06T02:09:10.163 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.161+0000 7f6571a44640 1 -- start start 2023-11-06T02:09:10.164 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.161+0000 7f6571a44640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f656c06d640 0x7f656c1326b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:10.164 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.161+0000 7f6571a44640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f656c15a990 0x7f656c0cd940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:10.164 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.161+0000 7f6571a44640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f656c15add0 0x7f656c15b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:10.164 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.161+0000 7f6571a44640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f656c15b7c0 con 0x7f656c15add0 2023-11-06T02:09:10.164 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.161+0000 7f6571a44640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f656c144d40 con 0x7f656c06d640 2023-11-06T02:09:10.165 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.161+0000 7f6571a44640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f656c132bf0 con 0x7f656c15a990 2023-11-06T02:09:10.165 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.161+0000 7f656b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f656c15add0 0x7f656c15b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:10.165 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.161+0000 7f656b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f656c15add0 0x7f656c15b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48092/0 (socket says 172.21.15.49:48092) 2023-11-06T02:09:10.165 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.161+0000 7f656b7fe640 1 -- 172.21.15.49:0/4082413981 learned_addr learned my addr 172.21.15.49:0/4082413981 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:10.165 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.161+0000 7f656affd640 1 --2- 172.21.15.49:0/4082413981 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f656c06d640 0x7f656c1326b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:10.165 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.161+0000 7f656a7fc640 1 --2- 172.21.15.49:0/4082413981 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f656c15a990 0x7f656c0cd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:10.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.161+0000 7f656b7fe640 1 -- 172.21.15.49:0/4082413981 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f656c15a990 msgr2=0x7f656c0cd940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:10.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.161+0000 7f656b7fe640 1 --2- 172.21.15.49:0/4082413981 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f656c15a990 0x7f656c0cd940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:10.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.161+0000 7f656b7fe640 1 -- 172.21.15.49:0/4082413981 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f656c06d640 msgr2=0x7f656c1326b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:10.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.161+0000 7f656b7fe640 1 --2- 172.21.15.49:0/4082413981 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f656c06d640 0x7f656c1326b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:10.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.161+0000 7f656b7fe640 1 -- 172.21.15.49:0/4082413981 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f655c009420 con 0x7f656c15add0 2023-11-06T02:09:10.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.161+0000 7f656b7fe640 1 --2- 172.21.15.49:0/4082413981 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f656c15add0 0x7f656c15b280 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f656400c760 tx=0x7f656400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:10.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.165+0000 7f6553fff640 1 -- 172.21.15.49:0/4082413981 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6564003b80 con 0x7f656c15add0 2023-11-06T02:09:10.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.165+0000 7f6553fff640 1 -- 172.21.15.49:0/4082413981 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6564003ce0 con 0x7f656c15add0 2023-11-06T02:09:10.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.165+0000 7f6571a44640 1 -- 172.21.15.49:0/4082413981 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f656c144130 con 0x7f656c15add0 2023-11-06T02:09:10.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.165+0000 7f6553fff640 1 -- 172.21.15.49:0/4082413981 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f656400aa90 con 0x7f656c15add0 2023-11-06T02:09:10.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.165+0000 7f6571a44640 1 -- 172.21.15.49:0/4082413981 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f656c1446f0 con 0x7f656c15add0 2023-11-06T02:09:10.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.165+0000 7f6571a44640 1 -- 172.21.15.49:0/4082413981 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f656c06da70 con 0x7f656c15add0 2023-11-06T02:09:10.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.165+0000 7f6553fff640 1 -- 172.21.15.49:0/4082413981 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f656400abf0 con 0x7f656c15add0 2023-11-06T02:09:10.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.165+0000 7f6553fff640 1 --2- 172.21.15.49:0/4082413981 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6558065a60 0x7f6558067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:10.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.165+0000 7f6553fff640 1 -- 172.21.15.49:0/4082413981 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6564082fe0 con 0x7f656c15add0 2023-11-06T02:09:10.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.165+0000 7f656affd640 1 --2- 172.21.15.49:0/4082413981 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6558065a60 0x7f6558067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:10.169 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.165+0000 7f656affd640 1 --2- 172.21.15.49:0/4082413981 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6558065a60 0x7f6558067f20 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f655c0af000 tx=0x7f655c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:10.169 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.169+0000 7f6553fff640 1 -- 172.21.15.49:0/4082413981 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f656c06da70 con 0x7f656c15add0 2023-11-06T02:09:10.323 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.325+0000 7f6571a44640 1 -- 172.21.15.49:0/4082413981 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f656c06da70 con 0x7f656c15add0 2023-11-06T02:09:10.324 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.325+0000 7f6553fff640 1 -- 172.21.15.49:0/4082413981 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f656c06da70 con 0x7f656c15add0 2023-11-06T02:09:10.324 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2023-11-06T02:09:10.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.329+0000 7f6571a44640 1 -- 172.21.15.49:0/4082413981 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6558065a60 msgr2=0x7f6558067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:10.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.329+0000 7f6571a44640 1 --2- 172.21.15.49:0/4082413981 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6558065a60 0x7f6558067f20 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f655c0af000 tx=0x7f655c008040 comp rx=0 tx=0).stop 2023-11-06T02:09:10.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.329+0000 7f6571a44640 1 -- 172.21.15.49:0/4082413981 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f656c15add0 msgr2=0x7f656c15b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:10.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.329+0000 7f6571a44640 1 --2- 172.21.15.49:0/4082413981 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f656c15add0 0x7f656c15b280 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f656400c760 tx=0x7f656400cc20 comp rx=0 tx=0).stop 2023-11-06T02:09:10.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.329+0000 7f6571a44640 1 -- 172.21.15.49:0/4082413981 shutdown_connections 2023-11-06T02:09:10.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.329+0000 7f6571a44640 1 --2- 172.21.15.49:0/4082413981 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6558065a60 0x7f6558067f20 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:10.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.329+0000 7f6571a44640 1 --2- 172.21.15.49:0/4082413981 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f656c15add0 0x7f656c15b280 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:10.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.329+0000 7f6571a44640 1 --2- 172.21.15.49:0/4082413981 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f656c15a990 0x7f656c0cd940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:10.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.329+0000 7f6571a44640 1 --2- 172.21.15.49:0/4082413981 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f656c06d640 0x7f656c1326b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:10.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.329+0000 7f6571a44640 1 -- 172.21.15.49:0/4082413981 >> 172.21.15.49:0/4082413981 conn(0x7f656c06a370 msgr2=0x7f656c0ae720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:10.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.329+0000 7f6571a44640 1 -- 172.21.15.49:0/4082413981 shutdown_connections 2023-11-06T02:09:10.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.329+0000 7f6571a44640 1 -- 172.21.15.49:0/4082413981 wait complete. 2023-11-06T02:09:10.344 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T02:09:10.469 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.473+0000 7fc7d85ba640 1 Processor -- start 2023-11-06T02:09:10.469 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.473+0000 7fc7d85ba640 1 -- start start 2023-11-06T02:09:10.470 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.473+0000 7fc7d85ba640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc7d00d2ed0 0x7fc7d00b6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:10.471 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.473+0000 7fc7d85ba640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7d015cf90 con 0x7fc7d015cbb0 2023-11-06T02:09:10.471 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.473+0000 7fc7d85ba640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7d00d3360 con 0x7fc7d00d2a60 2023-11-06T02:09:10.471 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.473+0000 7fc7d85ba640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7d00d0b80 con 0x7fc7d00d2ed0 2023-11-06T02:09:10.471 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.473+0000 7fc7d6b57640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc7d00d2ed0 0x7fc7d00b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:10.472 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.473+0000 7fc7d6b57640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc7d00d2ed0 0x7fc7d00b6450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48096/0 (socket says 172.21.15.49:48096) 2023-11-06T02:09:10.472 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.473+0000 7fc7d6b57640 1 -- 172.21.15.49:0/856345215 learned_addr learned my addr 172.21.15.49:0/856345215 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:10.472 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.473+0000 7fc7d6b57640 1 -- 172.21.15.49:0/856345215 >> v1:172.21.15.150:6789/0 conn(0x7fc7d00d2a60 legacy=0x7fc7d00b5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:10.472 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.473+0000 7fc7d6b57640 1 -- 172.21.15.49:0/856345215 >> v1:172.21.15.49:6789/0 conn(0x7fc7d015cbb0 legacy=0x7fc7d00d1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:10.472 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.473+0000 7fc7d6b57640 1 -- 172.21.15.49:0/856345215 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7d00d0e60 con 0x7fc7d00d2ed0 2023-11-06T02:09:10.473 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.473+0000 7fc7d5354640 1 -- 172.21.15.49:0/856345215 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1583125007 0 0) 0x7fc7d015cf90 con 0x7fc7d015cbb0 2023-11-06T02:09:10.474 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.473+0000 7fc7d6b57640 1 --2- 172.21.15.49:0/856345215 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7d00d2ed0 0x7fc7d00b6450 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc7c8009e50 tx=0x7fc7c801f1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b22c53edafc58f1a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:10.474 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.473+0000 7fc7d5354640 1 -- 172.21.15.49:0/856345215 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c80af400 con 0x7fc7d00d2ed0 2023-11-06T02:09:10.474 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.473+0000 7fc7d5354640 1 -- 172.21.15.49:0/856345215 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7c80ab050 con 0x7fc7d00d2ed0 2023-11-06T02:09:10.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.477+0000 7fc7d5354640 1 -- 172.21.15.49:0/856345215 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c80af780 con 0x7fc7d00d2ed0 2023-11-06T02:09:10.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.477+0000 7fc7d85ba640 1 -- 172.21.15.49:0/856345215 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7d00d2ed0 msgr2=0x7fc7d00b6450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:10.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.477+0000 7fc7d85ba640 1 --2- 172.21.15.49:0/856345215 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7d00d2ed0 0x7fc7d00b6450 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc7c8009e50 tx=0x7fc7c801f1c0 comp rx=0 tx=0).stop 2023-11-06T02:09:10.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.477+0000 7fc7d85ba640 1 -- 172.21.15.49:0/856345215 shutdown_connections 2023-11-06T02:09:10.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.477+0000 7fc7d85ba640 1 --2- 172.21.15.49:0/856345215 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7d00d2ed0 0x7fc7d00b6450 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:10.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.477+0000 7fc7d85ba640 1 -- 172.21.15.49:0/856345215 >> 172.21.15.49:0/856345215 conn(0x7fc7d00718a0 msgr2=0x7fc7d0071cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:10.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.477+0000 7fc7d85ba640 1 -- 172.21.15.49:0/856345215 shutdown_connections 2023-11-06T02:09:10.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.477+0000 7fc7d85ba640 1 -- 172.21.15.49:0/856345215 wait complete. 2023-11-06T02:09:10.478 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.477+0000 7fc7d85ba640 1 Processor -- start 2023-11-06T02:09:10.479 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.477+0000 7fc7d85ba640 1 -- start start 2023-11-06T02:09:10.479 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.477+0000 7fc7d85ba640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7d00d2a60 0x7fc7d00aa8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:10.479 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.477+0000 7fc7d85ba640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc7d00d2ed0 0x7fc7d0158e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:10.479 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.477+0000 7fc7d85ba640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc7d015cbb0 0x7fc7d015d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:10.479 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.477+0000 7fc7d85ba640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d00a9c00 con 0x7fc7d00d2a60 2023-11-06T02:09:10.480 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.477+0000 7fc7d85ba640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d01f6680 con 0x7fc7d015cbb0 2023-11-06T02:09:10.480 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.477+0000 7fc7d85ba640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d00a9e10 con 0x7fc7d00d2ed0 2023-11-06T02:09:10.480 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.477+0000 7fc7d6356640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7d00d2a60 0x7fc7d00aa8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:10.480 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.477+0000 7fc7d6356640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7d00d2a60 0x7fc7d00aa8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48102/0 (socket says 172.21.15.49:48102) 2023-11-06T02:09:10.480 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.477+0000 7fc7d6356640 1 -- 172.21.15.49:0/3449375178 learned_addr learned my addr 172.21.15.49:0/3449375178 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:10.481 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.477+0000 7fc7d6b57640 1 --2- 172.21.15.49:0/3449375178 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc7d015cbb0 0x7fc7d015d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:10.481 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.477+0000 7fc7d5b55640 1 --2- 172.21.15.49:0/3449375178 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc7d00d2ed0 0x7fc7d0158e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:10.481 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.477+0000 7fc7d6356640 1 -- 172.21.15.49:0/3449375178 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc7d00d2ed0 msgr2=0x7fc7d0158e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:10.481 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.477+0000 7fc7d6356640 1 --2- 172.21.15.49:0/3449375178 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc7d00d2ed0 0x7fc7d0158e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:10.481 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.477+0000 7fc7d6356640 1 -- 172.21.15.49:0/3449375178 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc7d015cbb0 msgr2=0x7fc7d015d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:10.482 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.477+0000 7fc7d6356640 1 --2- 172.21.15.49:0/3449375178 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc7d015cbb0 0x7fc7d015d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:10.482 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.477+0000 7fc7d6356640 1 -- 172.21.15.49:0/3449375178 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7c8009b30 con 0x7fc7d00d2a60 2023-11-06T02:09:10.482 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.477+0000 7fc7d6356640 1 --2- 172.21.15.49:0/3449375178 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7d00d2a60 0x7fc7d00aa8f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc00c4d0 tx=0x7fc7cc00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:10.482 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.477+0000 7fc7c6ffd640 1 -- 172.21.15.49:0/3449375178 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7cc005480 con 0x7fc7d00d2a60 2023-11-06T02:09:10.482 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.477+0000 7fc7c6ffd640 1 -- 172.21.15.49:0/3449375178 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7cc003600 con 0x7fc7d00d2a60 2023-11-06T02:09:10.483 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.477+0000 7fc7d85ba640 1 -- 172.21.15.49:0/3449375178 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7d00aa0f0 con 0x7fc7d00d2a60 2023-11-06T02:09:10.483 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.477+0000 7fc7c6ffd640 1 -- 172.21.15.49:0/3449375178 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7cc0039b0 con 0x7fc7d00d2a60 2023-11-06T02:09:10.483 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.477+0000 7fc7d85ba640 1 -- 172.21.15.49:0/3449375178 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7d00aae30 con 0x7fc7d00d2a60 2023-11-06T02:09:10.483 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.481+0000 7fc7d85ba640 1 -- 172.21.15.49:0/3449375178 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7d0074040 con 0x7fc7d00d2a60 2023-11-06T02:09:10.483 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.481+0000 7fc7c6ffd640 1 -- 172.21.15.49:0/3449375178 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fc7cc003b10 con 0x7fc7d00d2a60 2023-11-06T02:09:10.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.481+0000 7fc7c6ffd640 1 --2- 172.21.15.49:0/3449375178 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc7b4065a60 0x7fc7b4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:10.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.481+0000 7fc7c6ffd640 1 -- 172.21.15.49:0/3449375178 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc7cc082880 con 0x7fc7d00d2a60 2023-11-06T02:09:10.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.481+0000 7fc7d5b55640 1 --2- 172.21.15.49:0/3449375178 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc7b4065a60 0x7fc7b4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:10.485 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.481+0000 7fc7d5b55640 1 --2- 172.21.15.49:0/3449375178 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc7b4065a60 0x7fc7b4067f20 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc7d00d08a0 tx=0x7fc7c0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:10.485 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.485+0000 7fc7c6ffd640 1 -- 172.21.15.49:0/3449375178 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fc7cc050230 con 0x7fc7d00d2a60 2023-11-06T02:09:10.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.645+0000 7fc7d85ba640 1 -- 172.21.15.49:0/3449375178 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc7d00cc960 con 0x7fc7d00d2a60 2023-11-06T02:09:10.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.645+0000 7fc7c6ffd640 1 -- 172.21.15.49:0/3449375178 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc7cc0170a0 con 0x7fc7d00d2a60 2023-11-06T02:09:10.643 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 8 2023-11-06T02:09:10.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.645+0000 7fc7d85ba640 1 -- 172.21.15.49:0/3449375178 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc7b4065a60 msgr2=0x7fc7b4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:10.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.645+0000 7fc7d85ba640 1 --2- 172.21.15.49:0/3449375178 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc7b4065a60 0x7fc7b4067f20 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc7d00d08a0 tx=0x7fc7c0008040 comp rx=0 tx=0).stop 2023-11-06T02:09:10.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.645+0000 7fc7d85ba640 1 -- 172.21.15.49:0/3449375178 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7d00d2a60 msgr2=0x7fc7d00aa8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:10.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.645+0000 7fc7d85ba640 1 --2- 172.21.15.49:0/3449375178 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7d00d2a60 0x7fc7d00aa8f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc00c4d0 tx=0x7fc7cc00c990 comp rx=0 tx=0).stop 2023-11-06T02:09:10.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.645+0000 7fc7d85ba640 1 -- 172.21.15.49:0/3449375178 shutdown_connections 2023-11-06T02:09:10.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.645+0000 7fc7d85ba640 1 --2- 172.21.15.49:0/3449375178 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc7b4065a60 0x7fc7b4067f20 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:10.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.645+0000 7fc7d85ba640 1 --2- 172.21.15.49:0/3449375178 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc7d015cbb0 0x7fc7d015d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:10.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.645+0000 7fc7d85ba640 1 --2- 172.21.15.49:0/3449375178 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc7d00d2ed0 0x7fc7d0158e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:10.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.645+0000 7fc7d85ba640 1 --2- 172.21.15.49:0/3449375178 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7d00d2a60 0x7fc7d00aa8f0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:10.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.645+0000 7fc7d85ba640 1 -- 172.21.15.49:0/3449375178 >> 172.21.15.49:0/3449375178 conn(0x7fc7d00718a0 msgr2=0x7fc7d015af70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:10.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.645+0000 7fc7d85ba640 1 -- 172.21.15.49:0/3449375178 shutdown_connections 2023-11-06T02:09:10.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.645+0000 7fc7d85ba640 1 -- 172.21.15.49:0/3449375178 wait complete. 2023-11-06T02:09:10.661 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T02:09:10.786 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.789+0000 7f2fe520c640 1 Processor -- start 2023-11-06T02:09:10.787 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.789+0000 7f2fe520c640 1 -- start start 2023-11-06T02:09:10.789 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.789+0000 7f2fe520c640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f2fe00caa90 0x7f2fe00b0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:10.789 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.789+0000 7f2fe520c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2fe00b0ac0 0x7f2fe01631f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:10.790 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.789+0000 7f2fe520c640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fe006db60 con 0x7f2fe00b01a0 2023-11-06T02:09:10.790 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.789+0000 7f2fe520c640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fe00c8cc0 con 0x7f2fe00caa90 2023-11-06T02:09:10.790 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.789+0000 7f2fe520c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe00c8fa0 con 0x7f2fe00b0ac0 2023-11-06T02:09:10.790 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.789+0000 7f2fded9d640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f2fe00b01a0 0x7f2fe006d100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:60792/0 (socket says 172.21.15.49:60792) 2023-11-06T02:09:10.791 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.789+0000 7f2fded9d640 1 -- 172.21.15.49:0/2073819717 learned_addr learned my addr 172.21.15.49:0/2073819717 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:09:10.791 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.789+0000 7f2fdf59e640 1 --2- 172.21.15.49:0/2073819717 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2fe00b0ac0 0x7f2fe01631f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:10.791 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.789+0000 7f2fd659c640 1 --2- 172.21.15.49:0/2073819717 >> v2:172.21.15.150:3300/0 conn(0x7f2fe00caa90 0x7f2fe00b0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:10.791 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.789+0000 7f2fde59c640 1 -- 172.21.15.49:0/2073819717 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4155585860 0 0) 0x7f2fe006db60 con 0x7f2fe00b01a0 2023-11-06T02:09:10.791 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.789+0000 7f2fde59c640 1 -- 172.21.15.49:0/2073819717 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2fcc003530 con 0x7f2fe00b01a0 2023-11-06T02:09:10.792 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.789+0000 7f2fde59c640 1 -- 172.21.15.49:0/2073819717 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2867824457 0 0) 0x7f2fcc003530 con 0x7f2fe00b01a0 2023-11-06T02:09:10.792 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.789+0000 7f2fde59c640 1 -- 172.21.15.49:0/2073819717 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2fe00b0ac0 msgr2=0x7f2fe01631f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:10.792 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.789+0000 7f2fde59c640 1 --2- 172.21.15.49:0/2073819717 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2fe00b0ac0 0x7f2fe01631f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:10.792 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.789+0000 7f2fde59c640 1 -- 172.21.15.49:0/2073819717 >> v2:172.21.15.150:3300/0 conn(0x7f2fe00caa90 msgr2=0x7f2fe00b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:10.792 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.789+0000 7f2fde59c640 1 --2- 172.21.15.49:0/2073819717 >> v2:172.21.15.150:3300/0 conn(0x7f2fe00caa90 0x7f2fe00b0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:10.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.789+0000 7f2fde59c640 1 -- 172.21.15.49:0/2073819717 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fe00c9280 con 0x7f2fe00b01a0 2023-11-06T02:09:10.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.789+0000 7f2fde59c640 1 -- 172.21.15.49:0/2073819717 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7f2fd0001680 con 0x7f2fe00b01a0 2023-11-06T02:09:10.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.789+0000 7f2fde59c640 1 -- 172.21.15.49:0/2073819717 >> v1:172.21.15.49:6789/0 conn(0x7f2fe00b01a0 legacy=0x7f2fe006d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:10.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.789+0000 7f2fde59c640 1 --2- 172.21.15.49:0/2073819717 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2fcc003980 0x7f2fcc003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:10.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.789+0000 7f2fde59c640 1 --2- 172.21.15.49:0/2073819717 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2fcc0047d0 0x7f2fcc004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:10.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.789+0000 7f2fde59c640 1 --2- 172.21.15.49:0/2073819717 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2fe00b0ac0 0x7f2fcc005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:10.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.789+0000 7f2fde59c640 1 -- 172.21.15.49:0/2073819717 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe00c8cc0 con 0x7f2fcc0047d0 2023-11-06T02:09:10.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.789+0000 7f2fde59c640 1 -- 172.21.15.49:0/2073819717 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe00c8fa0 con 0x7f2fcc003980 2023-11-06T02:09:10.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.789+0000 7f2fde59c640 1 -- 172.21.15.49:0/2073819717 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fcc00b990 con 0x7f2fe00b0ac0 2023-11-06T02:09:10.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.789+0000 7f2fd659c640 1 --2- 172.21.15.49:0/2073819717 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2fcc0047d0 0x7f2fcc004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:10.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.789+0000 7f2fded9d640 1 --2- 172.21.15.49:0/2073819717 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2fcc003980 0x7f2fcc003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:10.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.789+0000 7f2fdf59e640 1 --2- 172.21.15.49:0/2073819717 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2fe00b0ac0 0x7f2fcc005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:10.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.789+0000 7f2fd659c640 1 -- 172.21.15.49:0/2073819717 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2fe00b0ac0 msgr2=0x7f2fcc005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:10.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.789+0000 7f2fd659c640 1 --2- 172.21.15.49:0/2073819717 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2fe00b0ac0 0x7f2fcc005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:10.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.789+0000 7f2fd659c640 1 -- 172.21.15.49:0/2073819717 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2fcc003980 msgr2=0x7f2fcc003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:10.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.789+0000 7f2fd659c640 1 --2- 172.21.15.49:0/2073819717 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2fcc003980 0x7f2fcc003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:10.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.789+0000 7f2fd659c640 1 -- 172.21.15.49:0/2073819717 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fcc003530 con 0x7f2fcc0047d0 2023-11-06T02:09:10.799 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.793+0000 7f2fd659c640 1 --2- 172.21.15.49:0/2073819717 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2fcc0047d0 0x7f2fcc004c50 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2fc8009b10 tx=0x7f2fc801ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=277e684999e86d8e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:10.799 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.793+0000 7f2fde59c640 1 -- 172.21.15.49:0/2073819717 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc80a6480 con 0x7f2fcc0047d0 2023-11-06T02:09:10.800 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.793+0000 7f2fde59c640 1 -- 172.21.15.49:0/2073819717 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fc80a65e0 con 0x7f2fcc0047d0 2023-11-06T02:09:10.800 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.793+0000 7f2fde59c640 1 -- 172.21.15.49:0/2073819717 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc80a6990 con 0x7f2fcc0047d0 2023-11-06T02:09:10.800 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.793+0000 7f2fe520c640 1 -- 172.21.15.49:0/2073819717 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2fcc0047d0 msgr2=0x7f2fcc004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:10.800 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.793+0000 7f2fe520c640 1 --2- 172.21.15.49:0/2073819717 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2fcc0047d0 0x7f2fcc004c50 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2fc8009b10 tx=0x7f2fc801ca60 comp rx=0 tx=0).stop 2023-11-06T02:09:10.800 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.793+0000 7f2fdf59e640 1 -- 172.21.15.49:0/2073819717 reap_dead start 2023-11-06T02:09:10.801 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.793+0000 7f2fe520c640 1 -- 172.21.15.49:0/2073819717 shutdown_connections 2023-11-06T02:09:10.801 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.793+0000 7f2fe520c640 1 -- 172.21.15.49:0/2073819717 >> 172.21.15.49:0/2073819717 conn(0x7f2fe00cec90 msgr2=0x7f2fe00cb0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:10.801 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.793+0000 7f2fe520c640 1 -- 172.21.15.49:0/2073819717 shutdown_connections 2023-11-06T02:09:10.801 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.793+0000 7f2fe520c640 1 -- 172.21.15.49:0/2073819717 wait complete. 2023-11-06T02:09:10.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.797+0000 7f2fe520c640 1 Processor -- start 2023-11-06T02:09:10.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.797+0000 7f2fe520c640 1 -- start start 2023-11-06T02:09:10.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.797+0000 7f2fe520c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2fcc0047d0 0x7f2fe013ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:10.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.797+0000 7f2fe520c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2fe0149ca0 0x7f2fe014d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:10.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.797+0000 7f2fe520c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2fe014b3e0 0x7f2fe014b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:10.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.797+0000 7f2fe520c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe014bda0 con 0x7f2fcc0047d0 2023-11-06T02:09:10.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.797+0000 7f2fe520c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe0146c40 con 0x7f2fe014b3e0 2023-11-06T02:09:10.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.797+0000 7f2fe520c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe014dd70 con 0x7f2fe0149ca0 2023-11-06T02:09:10.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.797+0000 7f2fded9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2fcc0047d0 0x7f2fe013ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:10.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.797+0000 7f2fded9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2fcc0047d0 0x7f2fe013ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48120/0 (socket says 172.21.15.49:48120) 2023-11-06T02:09:10.804 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.797+0000 7f2fded9d640 1 -- 172.21.15.49:0/2515522973 learned_addr learned my addr 172.21.15.49:0/2515522973 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:10.804 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.797+0000 7f2fd659c640 1 --2- 172.21.15.49:0/2515522973 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2fe0149ca0 0x7f2fe014d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:10.804 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.797+0000 7f2fdf59e640 1 --2- 172.21.15.49:0/2515522973 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2fe014b3e0 0x7f2fe014b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:10.804 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.797+0000 7f2fded9d640 1 -- 172.21.15.49:0/2515522973 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2fe0149ca0 msgr2=0x7f2fe014d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:10.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.797+0000 7f2fded9d640 1 --2- 172.21.15.49:0/2515522973 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2fe0149ca0 0x7f2fe014d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:10.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.797+0000 7f2fded9d640 1 -- 172.21.15.49:0/2515522973 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2fe014b3e0 msgr2=0x7f2fe014b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:10.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.797+0000 7f2fded9d640 1 --2- 172.21.15.49:0/2515522973 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2fe014b3e0 0x7f2fe014b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:10.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.797+0000 7f2fded9d640 1 -- 172.21.15.49:0/2515522973 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fe014e050 con 0x7f2fcc0047d0 2023-11-06T02:09:10.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.797+0000 7f2fded9d640 1 --2- 172.21.15.49:0/2515522973 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2fcc0047d0 0x7f2fe013ab60 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2fd000e780 tx=0x7f2fd000ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:10.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.797+0000 7f2fd7fff640 1 -- 172.21.15.49:0/2515522973 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd000ee90 con 0x7f2fcc0047d0 2023-11-06T02:09:10.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.797+0000 7f2fd7fff640 1 -- 172.21.15.49:0/2515522973 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fd000c8d0 con 0x7f2fcc0047d0 2023-11-06T02:09:10.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.797+0000 7f2fe520c640 1 -- 172.21.15.49:0/2515522973 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fe014e330 con 0x7f2fcc0047d0 2023-11-06T02:09:10.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.797+0000 7f2fd7fff640 1 -- 172.21.15.49:0/2515522973 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd000cc80 con 0x7f2fcc0047d0 2023-11-06T02:09:10.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.797+0000 7f2fe520c640 1 -- 172.21.15.49:0/2515522973 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fe013b0a0 con 0x7f2fcc0047d0 2023-11-06T02:09:10.807 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.797+0000 7f2fe520c640 1 -- 172.21.15.49:0/2515522973 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fe0074040 con 0x7f2fcc0047d0 2023-11-06T02:09:10.807 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.797+0000 7f2fd7fff640 1 -- 172.21.15.49:0/2515522973 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f2fd001d460 con 0x7f2fcc0047d0 2023-11-06T02:09:10.807 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.797+0000 7f2fd7fff640 1 --2- 172.21.15.49:0/2515522973 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2fc0065a90 0x7f2fc0067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:10.807 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.797+0000 7f2fd7fff640 1 -- 172.21.15.49:0/2515522973 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2fd00832a0 con 0x7f2fcc0047d0 2023-11-06T02:09:10.807 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.797+0000 7f2fd659c640 1 --2- 172.21.15.49:0/2515522973 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2fc0065a90 0x7f2fc0067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:10.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.801+0000 7f2fd659c640 1 --2- 172.21.15.49:0/2515522973 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2fc0065a90 0x7f2fc0067f50 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2fc80ad000 tx=0x7f2fc80b2040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:10.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.801+0000 7f2fd7fff640 1 -- 172.21.15.49:0/2515522973 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f2fd0050b00 con 0x7f2fcc0047d0 2023-11-06T02:09:10.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.957+0000 7f2fe520c640 1 -- 172.21.15.49:0/2515522973 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2fe00b4700 con 0x7f2fcc0047d0 2023-11-06T02:09:10.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.957+0000 7f2fd7fff640 1 -- 172.21.15.49:0/2515522973 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2fd0054120 con 0x7f2fcc0047d0 2023-11-06T02:09:10.956 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2023-11-06T02:09:10.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.961+0000 7f2fe520c640 1 -- 172.21.15.49:0/2515522973 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2fc0065a90 msgr2=0x7f2fc0067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:10.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.961+0000 7f2fe520c640 1 --2- 172.21.15.49:0/2515522973 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2fc0065a90 0x7f2fc0067f50 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2fc80ad000 tx=0x7f2fc80b2040 comp rx=0 tx=0).stop 2023-11-06T02:09:10.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.961+0000 7f2fe520c640 1 -- 172.21.15.49:0/2515522973 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2fcc0047d0 msgr2=0x7f2fe013ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:10.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.961+0000 7f2fe520c640 1 --2- 172.21.15.49:0/2515522973 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2fcc0047d0 0x7f2fe013ab60 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2fd000e780 tx=0x7f2fd000ec40 comp rx=0 tx=0).stop 2023-11-06T02:09:10.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.961+0000 7f2fe520c640 1 -- 172.21.15.49:0/2515522973 shutdown_connections 2023-11-06T02:09:10.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.961+0000 7f2fe520c640 1 --2- 172.21.15.49:0/2515522973 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2fc0065a90 0x7f2fc0067f50 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:10.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.961+0000 7f2fe520c640 1 --2- 172.21.15.49:0/2515522973 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2fe014b3e0 0x7f2fe014b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:10.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.961+0000 7f2fe520c640 1 --2- 172.21.15.49:0/2515522973 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2fe0149ca0 0x7f2fe014d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:10.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.961+0000 7f2fe520c640 1 --2- 172.21.15.49:0/2515522973 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2fcc0047d0 0x7f2fe013ab60 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:10.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.961+0000 7f2fe520c640 1 -- 172.21.15.49:0/2515522973 >> 172.21.15.49:0/2515522973 conn(0x7f2fe00cec90 msgr2=0x7f2fe00cda20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:10.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.961+0000 7f2fe520c640 1 -- 172.21.15.49:0/2515522973 shutdown_connections 2023-11-06T02:09:10.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:10.961+0000 7f2fe520c640 1 -- 172.21.15.49:0/2515522973 wait complete. 2023-11-06T02:09:10.976 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T02:09:11.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.109+0000 7f0b9a486640 1 Processor -- start 2023-11-06T02:09:11.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.109+0000 7f0b9a486640 1 -- start start 2023-11-06T02:09:11.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.109+0000 7f0b9a486640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0b940d2ed0 0x7f0b940b6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:11.109 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.109+0000 7f0b9a486640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b9415cf90 con 0x7f0b9415cbb0 2023-11-06T02:09:11.109 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.109+0000 7f0b9a486640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b940d3360 con 0x7f0b940d2a60 2023-11-06T02:09:11.109 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.109+0000 7f0b9a486640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b940d0b80 con 0x7f0b940d2ed0 2023-11-06T02:09:11.109 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.109+0000 7f0b93fff640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f0b9415cbb0 0x7f0b940d1fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:60794/0 (socket says 172.21.15.49:60794) 2023-11-06T02:09:11.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.109+0000 7f0b93fff640 1 -- 172.21.15.49:0/3486689306 learned_addr learned my addr 172.21.15.49:0/3486689306 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:09:11.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.109+0000 7f0b98a23640 1 --2- 172.21.15.49:0/3486689306 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0b940d2ed0 0x7f0b940b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:11.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.109+0000 7f0b92ffd640 1 -- 172.21.15.49:0/3486689306 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1667311898 0 0) 0x7f0b9415cf90 con 0x7f0b9415cbb0 2023-11-06T02:09:11.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.109+0000 7f0b92ffd640 1 -- 172.21.15.49:0/3486689306 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b7c003530 con 0x7f0b9415cbb0 2023-11-06T02:09:11.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.109+0000 7f0b92ffd640 1 -- 172.21.15.49:0/3486689306 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 694428526 0 0) 0x7f0b7c003530 con 0x7f0b9415cbb0 2023-11-06T02:09:11.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.109+0000 7f0b92ffd640 1 -- 172.21.15.49:0/3486689306 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0b940d2ed0 msgr2=0x7f0b940b6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:11.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.109+0000 7f0b92ffd640 1 --2- 172.21.15.49:0/3486689306 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0b940d2ed0 0x7f0b940b6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:11.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.109+0000 7f0b92ffd640 1 -- 172.21.15.49:0/3486689306 >> v1:172.21.15.150:6789/0 conn(0x7f0b940d2a60 legacy=0x7f0b940b5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:11.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.109+0000 7f0b92ffd640 1 -- 172.21.15.49:0/3486689306 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b940d0e60 con 0x7f0b9415cbb0 2023-11-06T02:09:11.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.109+0000 7f0b92ffd640 1 -- 172.21.15.49:0/3486689306 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7f0b840016a0 con 0x7f0b9415cbb0 2023-11-06T02:09:11.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.109+0000 7f0b92ffd640 1 -- 172.21.15.49:0/3486689306 >> v1:172.21.15.49:6789/0 conn(0x7f0b9415cbb0 legacy=0x7f0b940d1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:11.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.109+0000 7f0b92ffd640 1 --2- 172.21.15.49:0/3486689306 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0b940d2ed0 0x7f0b7c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:11.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.109+0000 7f0b92ffd640 1 --2- 172.21.15.49:0/3486689306 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0b7c003f30 0x7f0b7c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:11.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.109+0000 7f0b92ffd640 1 --2- 172.21.15.49:0/3486689306 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b7c004d10 0x7f0b7c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:11.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.109+0000 7f0b92ffd640 1 -- 172.21.15.49:0/3486689306 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b940d0b80 con 0x7f0b7c003f30 2023-11-06T02:09:11.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.109+0000 7f0b92ffd640 1 -- 172.21.15.49:0/3486689306 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b7c00bb00 con 0x7f0b7c004d10 2023-11-06T02:09:11.113 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.109+0000 7f0b92ffd640 1 -- 172.21.15.49:0/3486689306 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b7c00bcc0 con 0x7f0b940d2ed0 2023-11-06T02:09:11.113 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.109+0000 7f0b937fe640 1 --2- 172.21.15.49:0/3486689306 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0b7c003f30 0x7f0b7c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:11.113 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.109+0000 7f0b93fff640 1 --2- 172.21.15.49:0/3486689306 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0b940d2ed0 0x7f0b7c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:11.113 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.109+0000 7f0b98a23640 1 --2- 172.21.15.49:0/3486689306 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b7c004d10 0x7f0b7c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:11.113 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.109+0000 7f0b937fe640 1 -- 172.21.15.49:0/3486689306 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0b940d2ed0 msgr2=0x7f0b7c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:11.113 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.109+0000 7f0b937fe640 1 --2- 172.21.15.49:0/3486689306 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0b940d2ed0 0x7f0b7c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:11.114 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.109+0000 7f0b937fe640 1 -- 172.21.15.49:0/3486689306 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b7c004d10 msgr2=0x7f0b7c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:11.114 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.109+0000 7f0b937fe640 1 --2- 172.21.15.49:0/3486689306 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b7c004d10 0x7f0b7c005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:11.114 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.109+0000 7f0b937fe640 1 -- 172.21.15.49:0/3486689306 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b7c003530 con 0x7f0b7c003f30 2023-11-06T02:09:11.118 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.113+0000 7f0b937fe640 1 --2- 172.21.15.49:0/3486689306 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0b7c003f30 0x7f0b7c0047d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f0b88001780 tx=0x7f0b8801c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5991247d29a22740 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:11.118 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.113+0000 7f0b92ffd640 1 -- 172.21.15.49:0/3486689306 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b880052d0 con 0x7f0b7c003f30 2023-11-06T02:09:11.118 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.113+0000 7f0b92ffd640 1 -- 172.21.15.49:0/3486689306 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b880a7050 con 0x7f0b7c003f30 2023-11-06T02:09:11.118 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.113+0000 7f0b92ffd640 1 -- 172.21.15.49:0/3486689306 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b88003600 con 0x7f0b7c003f30 2023-11-06T02:09:11.119 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.113+0000 7f0b9a486640 1 -- 172.21.15.49:0/3486689306 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0b7c003f30 msgr2=0x7f0b7c0047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:11.119 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.113+0000 7f0b9a486640 1 --2- 172.21.15.49:0/3486689306 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0b7c003f30 0x7f0b7c0047d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f0b88001780 tx=0x7f0b8801c400 comp rx=0 tx=0).stop 2023-11-06T02:09:11.119 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.113+0000 7f0b98a23640 1 -- 172.21.15.49:0/3486689306 reap_dead start 2023-11-06T02:09:11.119 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.113+0000 7f0b9a486640 1 -- 172.21.15.49:0/3486689306 shutdown_connections 2023-11-06T02:09:11.119 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.113+0000 7f0b9a486640 1 -- 172.21.15.49:0/3486689306 >> 172.21.15.49:0/3486689306 conn(0x7f0b940718a0 msgr2=0x7f0b94071cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:11.120 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.113+0000 7f0b9a486640 1 -- 172.21.15.49:0/3486689306 shutdown_connections 2023-11-06T02:09:11.120 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.113+0000 7f0b9a486640 1 -- 172.21.15.49:0/3486689306 wait complete. 2023-11-06T02:09:11.120 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.113+0000 7f0b9a486640 1 Processor -- start 2023-11-06T02:09:11.120 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.113+0000 7f0b9a486640 1 -- start start 2023-11-06T02:09:11.120 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.117+0000 7f0b9a486640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0b94151ec0 0x7f0b941f2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:11.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.117+0000 7f0b9a486640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0b940c91a0 0x7f0b941f3230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:11.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.117+0000 7f0b9a486640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b941f3770 0x7f0b9415e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:11.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.117+0000 7f0b9a486640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b94152cf0 con 0x7f0b94151ec0 2023-11-06T02:09:11.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.117+0000 7f0b9a486640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b941529a0 con 0x7f0b941f3770 2023-11-06T02:09:11.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.117+0000 7f0b9a486640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b94158d60 con 0x7f0b940c91a0 2023-11-06T02:09:11.122 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.117+0000 7f0b93fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0b94151ec0 0x7f0b941f2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:11.122 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.117+0000 7f0b93fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0b94151ec0 0x7f0b941f2af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48146/0 (socket says 172.21.15.49:48146) 2023-11-06T02:09:11.122 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.117+0000 7f0b93fff640 1 -- 172.21.15.49:0/2930073176 learned_addr learned my addr 172.21.15.49:0/2930073176 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:11.122 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.117+0000 7f0b937fe640 1 --2- 172.21.15.49:0/2930073176 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0b940c91a0 0x7f0b941f3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:11.122 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.117+0000 7f0b98a23640 1 --2- 172.21.15.49:0/2930073176 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b941f3770 0x7f0b9415e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:11.123 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.117+0000 7f0b93fff640 1 -- 172.21.15.49:0/2930073176 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0b940c91a0 msgr2=0x7f0b941f3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:11.123 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.117+0000 7f0b93fff640 1 --2- 172.21.15.49:0/2930073176 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0b940c91a0 0x7f0b941f3230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:11.123 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.117+0000 7f0b93fff640 1 -- 172.21.15.49:0/2930073176 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b941f3770 msgr2=0x7f0b9415e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:11.123 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.117+0000 7f0b93fff640 1 --2- 172.21.15.49:0/2930073176 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b941f3770 0x7f0b9415e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:11.124 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.117+0000 7f0b93fff640 1 -- 172.21.15.49:0/2930073176 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b94155370 con 0x7f0b94151ec0 2023-11-06T02:09:11.124 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.117+0000 7f0b93fff640 1 --2- 172.21.15.49:0/2930073176 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0b94151ec0 0x7f0b941f2af0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f0b84004850 tx=0x7f0b8400ef80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:11.124 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.117+0000 7f0b90ff9640 1 -- 172.21.15.49:0/2930073176 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b8400c870 con 0x7f0b94151ec0 2023-11-06T02:09:11.124 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.117+0000 7f0b90ff9640 1 -- 172.21.15.49:0/2930073176 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b8400c9d0 con 0x7f0b94151ec0 2023-11-06T02:09:11.124 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.117+0000 7f0b9a486640 1 -- 172.21.15.49:0/2930073176 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b94155650 con 0x7f0b94151ec0 2023-11-06T02:09:11.124 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.117+0000 7f0b90ff9640 1 -- 172.21.15.49:0/2930073176 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b8400cd80 con 0x7f0b94151ec0 2023-11-06T02:09:11.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.117+0000 7f0b9a486640 1 -- 172.21.15.49:0/2930073176 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b941522a0 con 0x7f0b94151ec0 2023-11-06T02:09:11.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.117+0000 7f0b9a486640 1 -- 172.21.15.49:0/2930073176 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b94074040 con 0x7f0b94151ec0 2023-11-06T02:09:11.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.121+0000 7f0b90ff9640 1 -- 172.21.15.49:0/2930073176 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f0b8401d460 con 0x7f0b94151ec0 2023-11-06T02:09:11.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.121+0000 7f0b90ff9640 1 --2- 172.21.15.49:0/2930073176 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0b80065a90 0x7f0b80067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:11.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.121+0000 7f0b90ff9640 1 -- 172.21.15.49:0/2930073176 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0b84083340 con 0x7f0b94151ec0 2023-11-06T02:09:11.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.121+0000 7f0b937fe640 1 --2- 172.21.15.49:0/2930073176 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0b80065a90 0x7f0b80067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:11.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.121+0000 7f0b937fe640 1 --2- 172.21.15.49:0/2930073176 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0b80065a90 0x7f0b80067f50 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0b88004000 tx=0x7f0b880011a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:11.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.125+0000 7f0b90ff9640 1 -- 172.21.15.49:0/2930073176 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f0b84050b00 con 0x7f0b94151ec0 2023-11-06T02:09:11.279 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.281+0000 7f0b9a486640 1 -- 172.21.15.49:0/2930073176 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0b940c9930 con 0x7f0b94151ec0 2023-11-06T02:09:11.279 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.281+0000 7f0b90ff9640 1 -- 172.21.15.49:0/2930073176 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0b84054120 con 0x7f0b94151ec0 2023-11-06T02:09:11.280 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2023-11-06T02:09:11.282 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.285+0000 7f0b9a486640 1 -- 172.21.15.49:0/2930073176 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0b80065a90 msgr2=0x7f0b80067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:11.282 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.285+0000 7f0b9a486640 1 --2- 172.21.15.49:0/2930073176 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0b80065a90 0x7f0b80067f50 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0b88004000 tx=0x7f0b880011a0 comp rx=0 tx=0).stop 2023-11-06T02:09:11.282 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.285+0000 7f0b9a486640 1 -- 172.21.15.49:0/2930073176 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0b94151ec0 msgr2=0x7f0b941f2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:11.282 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.285+0000 7f0b9a486640 1 --2- 172.21.15.49:0/2930073176 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0b94151ec0 0x7f0b941f2af0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f0b84004850 tx=0x7f0b8400ef80 comp rx=0 tx=0).stop 2023-11-06T02:09:11.282 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.285+0000 7f0b9a486640 1 -- 172.21.15.49:0/2930073176 shutdown_connections 2023-11-06T02:09:11.283 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.285+0000 7f0b9a486640 1 --2- 172.21.15.49:0/2930073176 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0b80065a90 0x7f0b80067f50 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:11.283 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.285+0000 7f0b9a486640 1 --2- 172.21.15.49:0/2930073176 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b941f3770 0x7f0b9415e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:11.283 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.285+0000 7f0b9a486640 1 --2- 172.21.15.49:0/2930073176 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0b940c91a0 0x7f0b941f3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:11.283 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.285+0000 7f0b9a486640 1 --2- 172.21.15.49:0/2930073176 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0b94151ec0 0x7f0b941f2af0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:11.283 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.285+0000 7f0b9a486640 1 -- 172.21.15.49:0/2930073176 >> 172.21.15.49:0/2930073176 conn(0x7f0b940718a0 msgr2=0x7f0b940cb0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:11.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.285+0000 7f0b9a486640 1 -- 172.21.15.49:0/2930073176 shutdown_connections 2023-11-06T02:09:11.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.285+0000 7f0b9a486640 1 -- 172.21.15.49:0/2930073176 wait complete. 2023-11-06T02:09:11.299 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-06T02:09:11.418 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.421+0000 7f6aaca52640 1 Processor -- start 2023-11-06T02:09:11.418 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.421+0000 7f6aaca52640 1 -- start start 2023-11-06T02:09:11.419 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.421+0000 7f6aaca52640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f6aa815ab50 0x7f6aa80c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:11.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.421+0000 7f6aaca52640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6aa80b0790 0x7f6aa80c2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:11.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.421+0000 7f6aaca52640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6aa80cab80 con 0x7f6aa80cb010 2023-11-06T02:09:11.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.421+0000 7f6aaca52640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6aa815af30 con 0x7f6aa815ab50 2023-11-06T02:09:11.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.421+0000 7f6aaca52640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa80cb5b0 con 0x7f6aa80b0790 2023-11-06T02:09:11.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.421+0000 7f6aa5d9b640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f6aa80cb010 0x7f6aa80b0080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:60808/0 (socket says 172.21.15.49:60808) 2023-11-06T02:09:11.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.421+0000 7f6aa5d9b640 1 -- 172.21.15.49:0/1615029099 learned_addr learned my addr 172.21.15.49:0/1615029099 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:09:11.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.421+0000 7f6aa659c640 1 --2- 172.21.15.49:0/1615029099 >> v2:172.21.15.150:3300/0 conn(0x7f6aa815ab50 0x7f6aa80c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:11.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.421+0000 7f6aa6d9d640 1 --2- 172.21.15.49:0/1615029099 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6aa80b0790 0x7f6aa80c2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:11.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.421+0000 7f6aa559a640 1 -- 172.21.15.49:0/1615029099 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2821698105 0 0) 0x7f6aa80cab80 con 0x7f6aa80cb010 2023-11-06T02:09:11.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.421+0000 7f6aa559a640 1 -- 172.21.15.49:0/1615029099 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a90003530 con 0x7f6aa80cb010 2023-11-06T02:09:11.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.421+0000 7f6aa559a640 1 -- 172.21.15.49:0/1615029099 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1517024795 0 0) 0x7f6a90003530 con 0x7f6aa80cb010 2023-11-06T02:09:11.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.421+0000 7f6aa559a640 1 -- 172.21.15.49:0/1615029099 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6aa80b0790 msgr2=0x7f6aa80c2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:11.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.421+0000 7f6aa559a640 1 --2- 172.21.15.49:0/1615029099 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6aa80b0790 0x7f6aa80c2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:11.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.421+0000 7f6aa559a640 1 -- 172.21.15.49:0/1615029099 >> v2:172.21.15.150:3300/0 conn(0x7f6aa815ab50 msgr2=0x7f6aa80c9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:11.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.421+0000 7f6aa559a640 1 --2- 172.21.15.49:0/1615029099 >> v2:172.21.15.150:3300/0 conn(0x7f6aa815ab50 0x7f6aa80c9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:11.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.421+0000 7f6aa559a640 1 -- 172.21.15.49:0/1615029099 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aa80cb890 con 0x7f6aa80cb010 2023-11-06T02:09:11.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.421+0000 7f6aa559a640 1 -- 172.21.15.49:0/1615029099 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7f6a9c0016a0 con 0x7f6aa80cb010 2023-11-06T02:09:11.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.421+0000 7f6aa559a640 1 -- 172.21.15.49:0/1615029099 >> v1:172.21.15.49:6789/0 conn(0x7f6aa80cb010 legacy=0x7f6aa80b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:11.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.421+0000 7f6aa559a640 1 --2- 172.21.15.49:0/1615029099 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6aa80b0790 0x7f6a90003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:11.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.421+0000 7f6aa559a640 1 --2- 172.21.15.49:0/1615029099 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a90003f00 0x7f6a900047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:11.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.421+0000 7f6aa559a640 1 --2- 172.21.15.49:0/1615029099 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a90004d10 0x7f6a90005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:11.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.421+0000 7f6aa559a640 1 -- 172.21.15.49:0/1615029099 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa815af30 con 0x7f6a90004d10 2023-11-06T02:09:11.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.421+0000 7f6aa559a640 1 -- 172.21.15.49:0/1615029099 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa80cb5b0 con 0x7f6a90003f00 2023-11-06T02:09:11.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.421+0000 7f6aa559a640 1 -- 172.21.15.49:0/1615029099 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a9000b990 con 0x7f6aa80b0790 2023-11-06T02:09:11.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.421+0000 7f6aa6d9d640 1 --2- 172.21.15.49:0/1615029099 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a90004d10 0x7f6a90005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:11.427 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.421+0000 7f6aa5d9b640 1 --2- 172.21.15.49:0/1615029099 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a90003f00 0x7f6a900047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:11.427 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.421+0000 7f6aa659c640 1 --2- 172.21.15.49:0/1615029099 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6aa80b0790 0x7f6a90003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:11.427 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.421+0000 7f6aa6d9d640 1 -- 172.21.15.49:0/1615029099 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6aa80b0790 msgr2=0x7f6a90003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:11.427 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.421+0000 7f6aa6d9d640 1 --2- 172.21.15.49:0/1615029099 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6aa80b0790 0x7f6a90003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:11.427 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.421+0000 7f6aa6d9d640 1 -- 172.21.15.49:0/1615029099 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a90003f00 msgr2=0x7f6a900047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:11.428 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.421+0000 7f6aa6d9d640 1 --2- 172.21.15.49:0/1615029099 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a90003f00 0x7f6a900047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:11.428 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.421+0000 7f6aa6d9d640 1 -- 172.21.15.49:0/1615029099 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a90003530 con 0x7f6a90004d10 2023-11-06T02:09:11.428 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.421+0000 7f6aa5d9b640 1 --2- 172.21.15.49:0/1615029099 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a90003f00 0x7f6a900047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:09:11.428 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.421+0000 7f6aa659c640 1 --2- 172.21.15.49:0/1615029099 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6aa80b0790 0x7f6a90003980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:09:11.428 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.425+0000 7f6aa6d9d640 1 --2- 172.21.15.49:0/1615029099 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a90004d10 0x7f6a90005190 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f6aa8070050 tx=0x7f6aa001cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab3528abb13040e1 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:11.429 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.425+0000 7f6aa559a640 1 -- 172.21.15.49:0/1615029099 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa00a5c10 con 0x7f6a90004d10 2023-11-06T02:09:11.429 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.425+0000 7f6aa559a640 1 -- 172.21.15.49:0/1615029099 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6aa000f040 con 0x7f6a90004d10 2023-11-06T02:09:11.429 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.425+0000 7f6aa559a640 1 -- 172.21.15.49:0/1615029099 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa00ad440 con 0x7f6a90004d10 2023-11-06T02:09:11.429 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.425+0000 7f6aaca52640 1 -- 172.21.15.49:0/1615029099 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a90004d10 msgr2=0x7f6a90005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:11.429 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.425+0000 7f6aaca52640 1 --2- 172.21.15.49:0/1615029099 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a90004d10 0x7f6a90005190 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f6aa8070050 tx=0x7f6aa001cba0 comp rx=0 tx=0).stop 2023-11-06T02:09:11.429 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.425+0000 7f6aa6d9d640 1 -- 172.21.15.49:0/1615029099 reap_dead start 2023-11-06T02:09:11.430 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.425+0000 7f6aaca52640 1 -- 172.21.15.49:0/1615029099 shutdown_connections 2023-11-06T02:09:11.430 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.425+0000 7f6aaca52640 1 -- 172.21.15.49:0/1615029099 >> 172.21.15.49:0/1615029099 conn(0x7f6aa806c0a0 msgr2=0x7f6aa806c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:11.430 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.425+0000 7f6aaca52640 1 -- 172.21.15.49:0/1615029099 shutdown_connections 2023-11-06T02:09:11.430 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.425+0000 7f6aaca52640 1 -- 172.21.15.49:0/1615029099 wait complete. 2023-11-06T02:09:11.433 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.425+0000 7f6aaca52640 1 Processor -- start 2023-11-06T02:09:11.433 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.425+0000 7f6aaca52640 1 -- start start 2023-11-06T02:09:11.433 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.429+0000 7f6aaca52640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a90004d10 0x7f6aa81f4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:11.433 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.429+0000 7f6aaca52640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6aa80c5a90 0x7f6aa815d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:11.434 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.429+0000 7f6aaca52640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6aa80c2e10 0x7f6aa80c32c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:11.434 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.429+0000 7f6aaca52640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa80c3800 con 0x7f6aa80c5a90 2023-11-06T02:09:11.434 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.429+0000 7f6aaca52640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa80c82f0 con 0x7f6a90004d10 2023-11-06T02:09:11.434 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.429+0000 7f6aaca52640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa81f4750 con 0x7f6aa80c2e10 2023-11-06T02:09:11.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.429+0000 7f6aa5d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6aa80c5a90 0x7f6aa815d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:11.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.429+0000 7f6aa5d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6aa80c5a90 0x7f6aa815d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48170/0 (socket says 172.21.15.49:48170) 2023-11-06T02:09:11.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.429+0000 7f6aa5d9b640 1 -- 172.21.15.49:0/3900244846 learned_addr learned my addr 172.21.15.49:0/3900244846 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:11.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.429+0000 7f6aa659c640 1 --2- 172.21.15.49:0/3900244846 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a90004d10 0x7f6aa81f4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:11.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.429+0000 7f6aa6d9d640 1 --2- 172.21.15.49:0/3900244846 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6aa80c2e10 0x7f6aa80c32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:11.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.429+0000 7f6aa5d9b640 1 -- 172.21.15.49:0/3900244846 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6aa80c2e10 msgr2=0x7f6aa80c32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:11.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.429+0000 7f6aa5d9b640 1 --2- 172.21.15.49:0/3900244846 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6aa80c2e10 0x7f6aa80c32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:11.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.429+0000 7f6aa5d9b640 1 -- 172.21.15.49:0/3900244846 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a90004d10 msgr2=0x7f6aa81f4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:11.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.429+0000 7f6aa5d9b640 1 --2- 172.21.15.49:0/3900244846 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a90004d10 0x7f6aa81f4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:11.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.429+0000 7f6aa5d9b640 1 -- 172.21.15.49:0/3900244846 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aa815dcf0 con 0x7f6aa80c5a90 2023-11-06T02:09:11.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.429+0000 7f6aa659c640 1 --2- 172.21.15.49:0/3900244846 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a90004d10 0x7f6aa81f4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:09:11.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.429+0000 7f6aa6d9d640 1 --2- 172.21.15.49:0/3900244846 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6aa80c2e10 0x7f6aa80c32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:09:11.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.429+0000 7f6aa5d9b640 1 --2- 172.21.15.49:0/3900244846 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6aa80c5a90 0x7f6aa815d7b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c00d8b0 tx=0x7f6a9c00dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:11.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.429+0000 7f6a96ffd640 1 -- 172.21.15.49:0/3900244846 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c0074e0 con 0x7f6aa80c5a90 2023-11-06T02:09:11.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.429+0000 7f6a96ffd640 1 -- 172.21.15.49:0/3900244846 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a9c007640 con 0x7f6aa80c5a90 2023-11-06T02:09:11.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.429+0000 7f6aaca52640 1 -- 172.21.15.49:0/3900244846 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6aa80c6030 con 0x7f6aa80c5a90 2023-11-06T02:09:11.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.429+0000 7f6a96ffd640 1 -- 172.21.15.49:0/3900244846 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c0049f0 con 0x7f6aa80c5a90 2023-11-06T02:09:11.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.429+0000 7f6aaca52640 1 -- 172.21.15.49:0/3900244846 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6aa80c65f0 con 0x7f6aa80c5a90 2023-11-06T02:09:11.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.429+0000 7f6aaca52640 1 -- 172.21.15.49:0/3900244846 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6aa80b0de0 con 0x7f6aa80c5a90 2023-11-06T02:09:11.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.429+0000 7f6a96ffd640 1 -- 172.21.15.49:0/3900244846 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f6a9c004b50 con 0x7f6aa80c5a90 2023-11-06T02:09:11.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.429+0000 7f6a96ffd640 1 --2- 172.21.15.49:0/3900244846 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6a88065a60 0x7f6a88067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:11.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.429+0000 7f6a96ffd640 1 -- 172.21.15.49:0/3900244846 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6a9c082fe0 con 0x7f6aa80c5a90 2023-11-06T02:09:11.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.429+0000 7f6aa659c640 1 --2- 172.21.15.49:0/3900244846 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6a88065a60 0x7f6a88067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:11.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.433+0000 7f6aa659c640 1 --2- 172.21.15.49:0/3900244846 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6a88065a60 0x7f6a88067f20 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6aa80ad810 tx=0x7f6a98008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:11.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.433+0000 7f6a96ffd640 1 -- 172.21.15.49:0/3900244846 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f6a9c050840 con 0x7f6aa80c5a90 2023-11-06T02:09:11.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.597+0000 7f6aaca52640 1 -- 172.21.15.49:0/3900244846 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6aa8074040 con 0x7f6aa80c5a90 2023-11-06T02:09:11.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.597+0000 7f6a96ffd640 1 -- 172.21.15.49:0/3900244846 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) v1 ==== 80+0+0 (secure 0 0 0) 0x7f6a9c019080 con 0x7f6aa80c5a90 2023-11-06T02:09:11.595 INFO:teuthology.orchestra.run.smithi049.stderr:listed 0 entries 2023-11-06T02:09:11.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.597+0000 7f6aaca52640 1 -- 172.21.15.49:0/3900244846 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6a88065a60 msgr2=0x7f6a88067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:11.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.597+0000 7f6aaca52640 1 --2- 172.21.15.49:0/3900244846 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6a88065a60 0x7f6a88067f20 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6aa80ad810 tx=0x7f6a98008040 comp rx=0 tx=0).stop 2023-11-06T02:09:11.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.597+0000 7f6aaca52640 1 -- 172.21.15.49:0/3900244846 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6aa80c5a90 msgr2=0x7f6aa815d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:11.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.597+0000 7f6aaca52640 1 --2- 172.21.15.49:0/3900244846 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6aa80c5a90 0x7f6aa815d7b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c00d8b0 tx=0x7f6a9c00dd70 comp rx=0 tx=0).stop 2023-11-06T02:09:11.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.597+0000 7f6aaca52640 1 -- 172.21.15.49:0/3900244846 shutdown_connections 2023-11-06T02:09:11.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.597+0000 7f6aaca52640 1 --2- 172.21.15.49:0/3900244846 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6a88065a60 0x7f6a88067f20 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:11.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.597+0000 7f6aaca52640 1 --2- 172.21.15.49:0/3900244846 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6aa80c5a90 0x7f6aa815d7b0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:11.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.597+0000 7f6aaca52640 1 --2- 172.21.15.49:0/3900244846 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6aa80c2e10 0x7f6aa80c32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:11.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.597+0000 7f6aaca52640 1 --2- 172.21.15.49:0/3900244846 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a90004d10 0x7f6aa81f4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:11.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.597+0000 7f6aaca52640 1 -- 172.21.15.49:0/3900244846 >> 172.21.15.49:0/3900244846 conn(0x7f6aa806c0a0 msgr2=0x7f6aa8159c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:11.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.597+0000 7f6aaca52640 1 -- 172.21.15.49:0/3900244846 shutdown_connections 2023-11-06T02:09:11.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.597+0000 7f6aaca52640 1 -- 172.21.15.49:0/3900244846 wait complete. 2023-11-06T02:09:11.612 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-11-06T02:09:11.612 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T02:09:11.618 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T02:09:11.789 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.789+0000 7fc7b44d6640 1 Processor -- start 2023-11-06T02:09:11.789 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.789+0000 7fc7b44d6640 1 -- start start 2023-11-06T02:09:11.790 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.793+0000 7fc7b44d6640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc7ac0b0790 0x7fc7ac0c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:11.790 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.793+0000 7fc7b44d6640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7ac0cab80 con 0x7fc7ac0cb010 2023-11-06T02:09:11.790 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.793+0000 7fc7b44d6640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7ac15af30 con 0x7fc7ac15ab50 2023-11-06T02:09:11.791 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.793+0000 7fc7b44d6640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7ac0cb5b0 con 0x7fc7ac0b0790 2023-11-06T02:09:11.791 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.793+0000 7fc7b2a73640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fc7ac0cb010 0x7fc7ac0c2710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:60812/0 (socket says 172.21.15.49:60812) 2023-11-06T02:09:11.791 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.793+0000 7fc7b2a73640 1 -- 172.21.15.49:0/4227558122 learned_addr learned my addr 172.21.15.49:0/4227558122 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:09:11.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.793+0000 7fc7b1a71640 1 --2- 172.21.15.49:0/4227558122 >> v2:172.21.15.150:3300/0 conn(0x7fc7ac0b0790 0x7fc7ac0c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:11.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.793+0000 7fc7b1270640 1 -- 172.21.15.49:0/4227558122 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1609749689 0 0) 0x7fc7ac0cab80 con 0x7fc7ac0cb010 2023-11-06T02:09:11.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.793+0000 7fc7b1270640 1 -- 172.21.15.49:0/4227558122 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc798003530 con 0x7fc7ac0cb010 2023-11-06T02:09:11.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.793+0000 7fc7b1270640 1 -- 172.21.15.49:0/4227558122 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3778343778 0 0) 0x7fc798003530 con 0x7fc7ac0cb010 2023-11-06T02:09:11.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.793+0000 7fc7b1270640 1 -- 172.21.15.49:0/4227558122 >> v2:172.21.15.150:3300/0 conn(0x7fc7ac0b0790 msgr2=0x7fc7ac0c9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:11.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.793+0000 7fc7b1270640 1 --2- 172.21.15.49:0/4227558122 >> v2:172.21.15.150:3300/0 conn(0x7fc7ac0b0790 0x7fc7ac0c9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:11.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.793+0000 7fc7b1270640 1 -- 172.21.15.49:0/4227558122 >> v1:172.21.15.150:6789/0 conn(0x7fc7ac15ab50 legacy=0x7fc7ac0b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:11.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.793+0000 7fc7b1270640 1 -- 172.21.15.49:0/4227558122 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7ac0cb890 con 0x7fc7ac0cb010 2023-11-06T02:09:11.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.793+0000 7fc7b1270640 1 -- 172.21.15.49:0/4227558122 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7fc7a4001a00 con 0x7fc7ac0cb010 2023-11-06T02:09:11.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.793+0000 7fc7b1270640 1 -- 172.21.15.49:0/4227558122 >> v1:172.21.15.49:6789/0 conn(0x7fc7ac0cb010 legacy=0x7fc7ac0c2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:11.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.793+0000 7fc7b1270640 1 --2- 172.21.15.49:0/4227558122 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc798003980 0x7fc798003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:11.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.793+0000 7fc7b1270640 1 --2- 172.21.15.49:0/4227558122 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7980047d0 0x7fc798004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:11.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.793+0000 7fc7b1270640 1 --2- 172.21.15.49:0/4227558122 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc798005190 0x7fc79800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:11.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.793+0000 7fc7b1270640 1 -- 172.21.15.49:0/4227558122 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7ac0cb5b0 con 0x7fc7980047d0 2023-11-06T02:09:11.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.793+0000 7fc7b1270640 1 -- 172.21.15.49:0/4227558122 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc79800c030 con 0x7fc798005190 2023-11-06T02:09:11.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.793+0000 7fc7b1270640 1 -- 172.21.15.49:0/4227558122 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc79800c1f0 con 0x7fc798003980 2023-11-06T02:09:11.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.793+0000 7fc7b1a71640 1 --2- 172.21.15.49:0/4227558122 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7980047d0 0x7fc798004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:11.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.793+0000 7fc7b2272640 1 --2- 172.21.15.49:0/4227558122 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc798003980 0x7fc798003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:11.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.793+0000 7fc7b2a73640 1 --2- 172.21.15.49:0/4227558122 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc798005190 0x7fc79800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:11.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.793+0000 7fc7b1a71640 1 -- 172.21.15.49:0/4227558122 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc798003980 msgr2=0x7fc798003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:11.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.793+0000 7fc7b1a71640 1 --2- 172.21.15.49:0/4227558122 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc798003980 0x7fc798003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:11.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.793+0000 7fc7b1a71640 1 -- 172.21.15.49:0/4227558122 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc798005190 msgr2=0x7fc79800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:11.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.793+0000 7fc7b1a71640 1 --2- 172.21.15.49:0/4227558122 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc798005190 0x7fc79800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:11.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.793+0000 7fc7b1a71640 1 -- 172.21.15.49:0/4227558122 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc798003530 con 0x7fc7980047d0 2023-11-06T02:09:11.798 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.793+0000 7fc7b2a73640 1 -- 172.21.15.49:0/4227558122 reap_dead start 2023-11-06T02:09:11.798 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.793+0000 7fc7b2272640 1 --2- 172.21.15.49:0/4227558122 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc798003980 0x7fc798003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:09:11.801 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.797+0000 7fc7b1a71640 1 --2- 172.21.15.49:0/4227558122 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7980047d0 0x7fc798004c50 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc7a8009ae0 tx=0x7fc7a801ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52c92f38e1532cfe server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:11.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.797+0000 7fc7b1270640 1 -- 172.21.15.49:0/4227558122 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a80a5530 con 0x7fc7980047d0 2023-11-06T02:09:11.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.797+0000 7fc7b1270640 1 -- 172.21.15.49:0/4227558122 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7a80a5690 con 0x7fc7980047d0 2023-11-06T02:09:11.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.797+0000 7fc7b1270640 1 -- 172.21.15.49:0/4227558122 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a80a5a40 con 0x7fc7980047d0 2023-11-06T02:09:11.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.797+0000 7fc7b44d6640 1 -- 172.21.15.49:0/4227558122 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7980047d0 msgr2=0x7fc798004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:11.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.797+0000 7fc7b44d6640 1 --2- 172.21.15.49:0/4227558122 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7980047d0 0x7fc798004c50 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc7a8009ae0 tx=0x7fc7a801ca50 comp rx=0 tx=0).stop 2023-11-06T02:09:11.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.797+0000 7fc7b44d6640 1 -- 172.21.15.49:0/4227558122 shutdown_connections 2023-11-06T02:09:11.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.797+0000 7fc7b44d6640 1 --2- 172.21.15.49:0/4227558122 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7980047d0 0x7fc798004c50 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:11.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.797+0000 7fc7b44d6640 1 -- 172.21.15.49:0/4227558122 >> 172.21.15.49:0/4227558122 conn(0x7fc7ac06c0a0 msgr2=0x7fc7ac06c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:11.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.797+0000 7fc7b44d6640 1 -- 172.21.15.49:0/4227558122 shutdown_connections 2023-11-06T02:09:11.804 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.797+0000 7fc7b44d6640 1 -- 172.21.15.49:0/4227558122 wait complete. 2023-11-06T02:09:11.804 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.797+0000 7fc7b44d6640 1 Processor -- start 2023-11-06T02:09:11.804 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.797+0000 7fc7b44d6640 1 -- start start 2023-11-06T02:09:11.804 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.797+0000 7fc7b44d6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7980047d0 0x7fc7ac14de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:11.804 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.797+0000 7fc7b44d6640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc7ac0af310 0x7fc7ac14a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:11.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.797+0000 7fc7b44d6640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc7ac0af790 0x7fc7ac14aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:11.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.797+0000 7fc7b44d6640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7ac151870 con 0x7fc7980047d0 2023-11-06T02:09:11.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.797+0000 7fc7b44d6640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7ac14e350 con 0x7fc7ac0af790 2023-11-06T02:09:11.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.797+0000 7fc7b44d6640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7ac1f5b20 con 0x7fc7ac0af310 2023-11-06T02:09:11.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.797+0000 7fc7b2272640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7980047d0 0x7fc7ac14de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:11.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.797+0000 7fc7b2272640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7980047d0 0x7fc7ac14de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48188/0 (socket says 172.21.15.49:48188) 2023-11-06T02:09:11.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.797+0000 7fc7b2272640 1 -- 172.21.15.49:0/3923875715 learned_addr learned my addr 172.21.15.49:0/3923875715 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:11.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.797+0000 7fc7b2a73640 1 --2- 172.21.15.49:0/3923875715 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc7ac0af790 0x7fc7ac14aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:11.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.797+0000 7fc7b1a71640 1 --2- 172.21.15.49:0/3923875715 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc7ac0af310 0x7fc7ac14a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:11.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.801+0000 7fc7b2272640 1 -- 172.21.15.49:0/3923875715 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc7ac0af310 msgr2=0x7fc7ac14a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:11.807 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.801+0000 7fc7b2272640 1 --2- 172.21.15.49:0/3923875715 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc7ac0af310 0x7fc7ac14a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:11.807 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.801+0000 7fc7b2272640 1 -- 172.21.15.49:0/3923875715 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc7ac0af790 msgr2=0x7fc7ac14aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:11.807 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.801+0000 7fc7b2272640 1 --2- 172.21.15.49:0/3923875715 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc7ac0af790 0x7fc7ac14aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:11.807 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.801+0000 7fc7b2272640 1 -- 172.21.15.49:0/3923875715 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7ac1f5e00 con 0x7fc7980047d0 2023-11-06T02:09:11.807 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.801+0000 7fc7b2272640 1 --2- 172.21.15.49:0/3923875715 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7980047d0 0x7fc7ac14de10 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc79c00d470 tx=0x7fc79c00d930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:11.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.801+0000 7fc7a2ffd640 1 -- 172.21.15.49:0/3923875715 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc79c005480 con 0x7fc7980047d0 2023-11-06T02:09:11.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.801+0000 7fc7a2ffd640 1 -- 172.21.15.49:0/3923875715 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc79c003670 con 0x7fc7980047d0 2023-11-06T02:09:11.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.801+0000 7fc7a2ffd640 1 -- 172.21.15.49:0/3923875715 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc79c003a00 con 0x7fc7980047d0 2023-11-06T02:09:11.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.801+0000 7fc7b44d6640 1 -- 172.21.15.49:0/3923875715 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7ac1f60e0 con 0x7fc7980047d0 2023-11-06T02:09:11.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.801+0000 7fc7b44d6640 1 -- 172.21.15.49:0/3923875715 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7ac14b3b0 con 0x7fc7980047d0 2023-11-06T02:09:11.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.801+0000 7fc7b44d6640 1 -- 172.21.15.49:0/3923875715 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7ac0b0de0 con 0x7fc7980047d0 2023-11-06T02:09:11.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.801+0000 7fc7a2ffd640 1 -- 172.21.15.49:0/3923875715 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fc79c003b60 con 0x7fc7980047d0 2023-11-06T02:09:11.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.801+0000 7fc7a2ffd640 1 --2- 172.21.15.49:0/3923875715 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc790065a60 0x7fc790067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:11.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.801+0000 7fc7a2ffd640 1 -- 172.21.15.49:0/3923875715 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc79c083140 con 0x7fc7980047d0 2023-11-06T02:09:11.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.801+0000 7fc7b1a71640 1 --2- 172.21.15.49:0/3923875715 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc790065a60 0x7fc790067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:11.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.801+0000 7fc7b1a71640 1 --2- 172.21.15.49:0/3923875715 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc790065a60 0x7fc790067f20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc7a80af000 tx=0x7fc7a8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:11.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.805+0000 7fc7a2ffd640 1 -- 172.21.15.49:0/3923875715 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fc79c0509a0 con 0x7fc7980047d0 2023-11-06T02:09:11.957 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.957+0000 7fc7b44d6640 1 -- 172.21.15.49:0/3923875715 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc7ac1f6590 con 0x7fc7980047d0 2023-11-06T02:09:11.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.961+0000 7fc7a2ffd640 1 -- 172.21.15.49:0/3923875715 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+17811 (secure 0 0 0) 0x7fc79c053fc0 con 0x7fc7980047d0 2023-11-06T02:09:11.959 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:09:11.959 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":23,"fsid":"1a8b035c-6ba4-499a-b876-aea47fd9c0d4","created":"2023-11-06T02:08:06.440857+0000","modified":"2023-11-06T02:08:31.594964+0000","last_up_change":"2023-11-06T02:08:17.982372+0000","last_in_change":"2023-11-06T02:08:08.906631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T02:08:19.252222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T02:08:24.776211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-06T02:08:28.936178+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-06T02:08:29.563468+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ad55af8-0571-4e6a-8722-b5d429b7496d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6809","nonce":38116},{"type":"v1","addr":"172.21.15.49:6811","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6813","nonce":38116},{"type":"v1","addr":"172.21.15.49:6815","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6821","nonce":38116},{"type":"v1","addr":"172.21.15.49:6823","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6817","nonce":38116},{"type":"v1","addr":"172.21.15.49:6819","nonce":38116}]},"public_addr":"172.21.15.49:6811/38116","cluster_addr":"172.21.15.49:6815/38116","heartbeat_back_addr":"172.21.15.49:6823/38116","heartbeat_front_addr":"172.21.15.49:6819/38116","state":["exists","up"]},{"osd":1,"uuid":"c8b31f85-a757-43ed-9419-351acc46611b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":38113},{"type":"v1","addr":"172.21.15.49:6810","nonce":38113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":38113},{"type":"v1","addr":"172.21.15.49:6814","nonce":38113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":38113},{"type":"v1","addr":"172.21.15.49:6822","nonce":38113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":38113},{"type":"v1","addr":"172.21.15.49:6818","nonce":38113}]},"public_addr":"172.21.15.49:6810/38113","cluster_addr":"172.21.15.49:6814/38113","heartbeat_back_addr":"172.21.15.49:6822/38113","heartbeat_front_addr":"172.21.15.49:6818/38113","state":["exists","up"]},{"osd":2,"uuid":"555b3bf3-0a70-4b91-8efd-e4204b5fc26e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":38114},{"type":"v1","addr":"172.21.15.49:6801","nonce":38114}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":38114},{"type":"v1","addr":"172.21.15.49:6803","nonce":38114}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":38114},{"type":"v1","addr":"172.21.15.49:6807","nonce":38114}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":38114},{"type":"v1","addr":"172.21.15.49:6805","nonce":38114}]},"public_addr":"172.21.15.49:6801/38114","cluster_addr":"172.21.15.49:6803/38114","heartbeat_back_addr":"172.21.15.49:6807/38114","heartbeat_front_addr":"172.21.15.49:6805/38114","state":["exists","up"]},{"osd":3,"uuid":"8d4e5d77-03ef-4b9b-bfcb-5ae93a46795c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":38115},{"type":"v1","addr":"172.21.15.49:6825","nonce":38115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":38115},{"type":"v1","addr":"172.21.15.49:6827","nonce":38115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":38115},{"type":"v1","addr":"172.21.15.49:6831","nonce":38115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":38115},{"type":"v1","addr":"172.21.15.49:6829","nonce":38115}]},"public_addr":"172.21.15.49:6825/38115","cluster_addr":"172.21.15.49:6827/38115","heartbeat_back_addr":"172.21.15.49:6831/38115","heartbeat_front_addr":"172.21.15.49:6829/38115","state":["exists","up"]},{"osd":4,"uuid":"dba03ad4-68a3-4cf3-9218-ac611f02eacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":38417},{"type":"v1","addr":"172.21.15.150:6809","nonce":38417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":38417},{"type":"v1","addr":"172.21.15.150:6811","nonce":38417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":38417},{"type":"v1","addr":"172.21.15.150:6815","nonce":38417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":38417},{"type":"v1","addr":"172.21.15.150:6813","nonce":38417}]},"public_addr":"172.21.15.150:6809/38417","cluster_addr":"172.21.15.150:6811/38417","heartbeat_back_addr":"172.21.15.150:6815/38417","heartbeat_front_addr":"172.21.15.150:6813/38417","state":["exists","up"]},{"osd":5,"uuid":"0df9c303-c255-4640-8e91-01f1c3efe18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":38418},{"type":"v1","addr":"172.21.15.150:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":38418},{"type":"v1","addr":"172.21.15.150:6803","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":38418},{"type":"v1","addr":"172.21.15.150:6807","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":38418},{"type":"v1","addr":"172.21.15.150:6805","nonce":38418}]},"public_addr":"172.21.15.150:6801/38418","cluster_addr":"172.21.15.150:6803/38418","heartbeat_back_addr":"172.21.15.150:6807/38418","heartbeat_front_addr":"172.21.15.150:6805/38418","state":["exists","up"]},{"osd":6,"uuid":"e59fa189-4b70-4c06-9722-c02b90828bf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":38428},{"type":"v1","addr":"172.21.15.150:6825","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":38428},{"type":"v1","addr":"172.21.15.150:6827","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":38428},{"type":"v1","addr":"172.21.15.150:6831","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":38428},{"type":"v1","addr":"172.21.15.150:6829","nonce":38428}]},"public_addr":"172.21.15.150:6825/38428","cluster_addr":"172.21.15.150:6827/38428","heartbeat_back_addr":"172.21.15.150:6831/38428","heartbeat_front_addr":"172.21.15.150:6829/38428","state":["exists","up"]},{"osd":7,"uuid":"a64f57e1-ef98-46bd-b304-385759e95898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":38421},{"type":"v1","addr":"172.21.15.150:6817","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":38421},{"type":"v1","addr":"172.21.15.150:6819","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":38421},{"type":"v1","addr":"172.21.15.150:6823","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":38421},{"type":"v1","addr":"172.21.15.150:6821","nonce":38421}]},"public_addr":"172.21.15.150:6817/38421","cluster_addr":"172.21.15.150:6819/38421","heartbeat_back_addr":"172.21.15.150:6823/38421","heartbeat_front_addr":"172.21.15.150:6821/38421","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.438552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.413280+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.296466+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.424966+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.276953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.230984+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.410235+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.480084+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T02:09:11.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.961+0000 7fc7b44d6640 1 -- 172.21.15.49:0/3923875715 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc790065a60 msgr2=0x7fc790067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:11.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.961+0000 7fc7b44d6640 1 --2- 172.21.15.49:0/3923875715 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc790065a60 0x7fc790067f20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc7a80af000 tx=0x7fc7a8008040 comp rx=0 tx=0).stop 2023-11-06T02:09:11.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.961+0000 7fc7b44d6640 1 -- 172.21.15.49:0/3923875715 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7980047d0 msgr2=0x7fc7ac14de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:11.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.961+0000 7fc7b44d6640 1 --2- 172.21.15.49:0/3923875715 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7980047d0 0x7fc7ac14de10 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc79c00d470 tx=0x7fc79c00d930 comp rx=0 tx=0).stop 2023-11-06T02:09:11.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.961+0000 7fc7b44d6640 1 -- 172.21.15.49:0/3923875715 shutdown_connections 2023-11-06T02:09:11.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.961+0000 7fc7b44d6640 1 --2- 172.21.15.49:0/3923875715 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc790065a60 0x7fc790067f20 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:11.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.961+0000 7fc7b44d6640 1 --2- 172.21.15.49:0/3923875715 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc7ac0af790 0x7fc7ac14aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:11.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.961+0000 7fc7b44d6640 1 --2- 172.21.15.49:0/3923875715 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc7ac0af310 0x7fc7ac14a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:11.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.961+0000 7fc7b44d6640 1 --2- 172.21.15.49:0/3923875715 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7980047d0 0x7fc7ac14de10 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:11.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.961+0000 7fc7b44d6640 1 -- 172.21.15.49:0/3923875715 >> 172.21.15.49:0/3923875715 conn(0x7fc7ac06c0a0 msgr2=0x7fc7ac06c480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:11.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.965+0000 7fc7b44d6640 1 -- 172.21.15.49:0/3923875715 shutdown_connections 2023-11-06T02:09:11.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:11.965+0000 7fc7b44d6640 1 -- 172.21.15.49:0/3923875715 wait complete. 2023-11-06T02:09:11.977 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T02:08:19.252222+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T02:08:24.776211+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T02:08:28.936178+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T02:08:29.563468+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T02:09:11.978 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T02:09:12.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.097+0000 7f75e2add640 1 Processor -- start 2023-11-06T02:09:12.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.097+0000 7f75e2add640 1 -- start start 2023-11-06T02:09:12.096 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.097+0000 7f75e2add640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f75dc161260 0x7f75dc070160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:12.096 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.097+0000 7f75e2add640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75dc0c91f0 con 0x7f75dc06d9b0 2023-11-06T02:09:12.096 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.097+0000 7f75e2add640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75dc06c230 con 0x7f75dc0c8790 2023-11-06T02:09:12.096 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.097+0000 7f75e2add640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75dc06c580 con 0x7f75dc161260 2023-11-06T02:09:12.097 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.097+0000 7f75e107a640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f75dc06d9b0 0x7f75dc0af7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:60818/0 (socket says 172.21.15.49:60818) 2023-11-06T02:09:12.097 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.097+0000 7f75e107a640 1 -- 172.21.15.49:0/2970421320 learned_addr learned my addr 172.21.15.49:0/2970421320 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:09:12.097 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.097+0000 7f75e0879640 1 --2- 172.21.15.49:0/2970421320 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f75dc161260 0x7f75dc070160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:12.097 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.097+0000 7f75db7fe640 1 -- 172.21.15.49:0/2970421320 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 334575265 0 0) 0x7f75dc0c91f0 con 0x7f75dc06d9b0 2023-11-06T02:09:12.097 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.097+0000 7f75db7fe640 1 -- 172.21.15.49:0/2970421320 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75c8003530 con 0x7f75dc06d9b0 2023-11-06T02:09:12.098 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.097+0000 7f75db7fe640 1 -- 172.21.15.49:0/2970421320 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 891367777 0 0) 0x7f75c8003530 con 0x7f75dc06d9b0 2023-11-06T02:09:12.098 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.097+0000 7f75db7fe640 1 -- 172.21.15.49:0/2970421320 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f75dc161260 msgr2=0x7f75dc070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:12.098 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.097+0000 7f75db7fe640 1 --2- 172.21.15.49:0/2970421320 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f75dc161260 0x7f75dc070160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:12.099 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.097+0000 7f75db7fe640 1 -- 172.21.15.49:0/2970421320 >> v1:172.21.15.150:6789/0 conn(0x7f75dc0c8790 legacy=0x7f75dc0af090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:12.099 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.097+0000 7f75db7fe640 1 -- 172.21.15.49:0/2970421320 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75dc0b02d0 con 0x7f75dc06d9b0 2023-11-06T02:09:12.099 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.097+0000 7f75db7fe640 1 -- 172.21.15.49:0/2970421320 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7f75d4001a00 con 0x7f75dc06d9b0 2023-11-06T02:09:12.099 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.097+0000 7f75db7fe640 1 -- 172.21.15.49:0/2970421320 >> v1:172.21.15.49:6789/0 conn(0x7f75dc06d9b0 legacy=0x7f75dc0af7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:12.099 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.097+0000 7f75db7fe640 1 --2- 172.21.15.49:0/2970421320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75c80039b0 0x7f75c8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:12.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.097+0000 7f75db7fe640 1 --2- 172.21.15.49:0/2970421320 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f75c80047d0 0x7f75c8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:12.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.097+0000 7f75db7fe640 1 --2- 172.21.15.49:0/2970421320 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f75dc161260 0x7f75c800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:12.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.097+0000 7f75db7fe640 1 -- 172.21.15.49:0/2970421320 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75dc06c580 con 0x7f75c80039b0 2023-11-06T02:09:12.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.097+0000 7f75db7fe640 1 -- 172.21.15.49:0/2970421320 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75c8005510 con 0x7f75c80047d0 2023-11-06T02:09:12.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.097+0000 7f75db7fe640 1 -- 172.21.15.49:0/2970421320 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75c800bed0 con 0x7f75dc161260 2023-11-06T02:09:12.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.097+0000 7f75e0879640 1 --2- 172.21.15.49:0/2970421320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75c80039b0 0x7f75c8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:12.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.097+0000 7f75dbfff640 1 --2- 172.21.15.49:0/2970421320 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f75c80047d0 0x7f75c8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:12.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.097+0000 7f75e107a640 1 --2- 172.21.15.49:0/2970421320 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f75dc161260 0x7f75c800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:12.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.097+0000 7f75e0879640 1 -- 172.21.15.49:0/2970421320 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f75dc161260 msgr2=0x7f75c800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:12.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.097+0000 7f75e0879640 1 --2- 172.21.15.49:0/2970421320 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f75dc161260 0x7f75c800b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:12.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.097+0000 7f75e0879640 1 -- 172.21.15.49:0/2970421320 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f75c80047d0 msgr2=0x7f75c8004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:12.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.097+0000 7f75e0879640 1 --2- 172.21.15.49:0/2970421320 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f75c80047d0 0x7f75c8004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:12.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.097+0000 7f75e0879640 1 -- 172.21.15.49:0/2970421320 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75c8003530 con 0x7f75c80039b0 2023-11-06T02:09:12.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.097+0000 7f75dbfff640 1 --2- 172.21.15.49:0/2970421320 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f75c80047d0 0x7f75c8004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:09:12.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.101+0000 7f75e0879640 1 --2- 172.21.15.49:0/2970421320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75c80039b0 0x7f75c8003dd0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f75cc00ba30 tx=0x7f75cc01c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa0f1dd104a4640c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:12.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.101+0000 7f75db7fe640 1 -- 172.21.15.49:0/2970421320 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75cc0ac620 con 0x7f75c80039b0 2023-11-06T02:09:12.107 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.101+0000 7f75db7fe640 1 -- 172.21.15.49:0/2970421320 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75cc0a6040 con 0x7f75c80039b0 2023-11-06T02:09:12.107 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.101+0000 7f75db7fe640 1 -- 172.21.15.49:0/2970421320 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75cc0ac9c0 con 0x7f75c80039b0 2023-11-06T02:09:12.107 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.101+0000 7f75e2add640 1 -- 172.21.15.49:0/2970421320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75c80039b0 msgr2=0x7f75c8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:12.107 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.101+0000 7f75e2add640 1 --2- 172.21.15.49:0/2970421320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75c80039b0 0x7f75c8003dd0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f75cc00ba30 tx=0x7f75cc01c680 comp rx=0 tx=0).stop 2023-11-06T02:09:12.107 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.101+0000 7f75e107a640 1 -- 172.21.15.49:0/2970421320 reap_dead start 2023-11-06T02:09:12.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.101+0000 7f75e2add640 1 -- 172.21.15.49:0/2970421320 shutdown_connections 2023-11-06T02:09:12.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.101+0000 7f75e2add640 1 -- 172.21.15.49:0/2970421320 >> 172.21.15.49:0/2970421320 conn(0x7f75dc06a370 msgr2=0x7f75dc0ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:12.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.101+0000 7f75e2add640 1 -- 172.21.15.49:0/2970421320 shutdown_connections 2023-11-06T02:09:12.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.101+0000 7f75e2add640 1 -- 172.21.15.49:0/2970421320 wait complete. 2023-11-06T02:09:12.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.101+0000 7f75e2add640 1 Processor -- start 2023-11-06T02:09:12.109 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.101+0000 7f75e2add640 1 -- start start 2023-11-06T02:09:12.109 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.101+0000 7f75e2add640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f75c80039b0 0x7f75dc163b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:12.109 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.101+0000 7f75e2add640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f75dc13d330 0x7f75dc0b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:12.109 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.101+0000 7f75e2add640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75dc0b4200 0x7f75dc13f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:12.109 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.101+0000 7f75e2add640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75dc13dc10 con 0x7f75dc0b4200 2023-11-06T02:09:12.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.101+0000 7f75e2add640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75dc13fe80 con 0x7f75dc13d330 2023-11-06T02:09:12.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.101+0000 7f75e2add640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75dc140160 con 0x7f75c80039b0 2023-11-06T02:09:12.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.101+0000 7f75e107a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75dc0b4200 0x7f75dc13f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:12.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.101+0000 7f75e107a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75dc0b4200 0x7f75dc13f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48200/0 (socket says 172.21.15.49:48200) 2023-11-06T02:09:12.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.101+0000 7f75e107a640 1 -- 172.21.15.49:0/3143724392 learned_addr learned my addr 172.21.15.49:0/3143724392 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:12.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.101+0000 7f75e0879640 1 --2- 172.21.15.49:0/3143724392 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f75c80039b0 0x7f75dc163b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:12.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.101+0000 7f75dbfff640 1 --2- 172.21.15.49:0/3143724392 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f75dc13d330 0x7f75dc0b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:12.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.101+0000 7f75e107a640 1 -- 172.21.15.49:0/3143724392 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f75c80039b0 msgr2=0x7f75dc163b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:12.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.101+0000 7f75e107a640 1 --2- 172.21.15.49:0/3143724392 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f75c80039b0 0x7f75dc163b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:12.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.101+0000 7f75e107a640 1 -- 172.21.15.49:0/3143724392 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f75dc13d330 msgr2=0x7f75dc0b3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:12.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.101+0000 7f75e107a640 1 --2- 172.21.15.49:0/3143724392 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f75dc13d330 0x7f75dc0b3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:12.113 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.101+0000 7f75e107a640 1 -- 172.21.15.49:0/3143724392 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75dc0b34c0 con 0x7f75dc0b4200 2023-11-06T02:09:12.113 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.101+0000 7f75dbfff640 1 --2- 172.21.15.49:0/3143724392 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f75dc13d330 0x7f75dc0b3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:09:12.113 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.101+0000 7f75e107a640 1 --2- 172.21.15.49:0/3143724392 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75dc0b4200 0x7f75dc13f390 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f75dc0aeb20 tx=0x7f75d400df70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:12.113 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.105+0000 7f75d97fa640 1 -- 172.21.15.49:0/3143724392 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d400b800 con 0x7f75dc0b4200 2023-11-06T02:09:12.113 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.105+0000 7f75d97fa640 1 -- 172.21.15.49:0/3143724392 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75d400b960 con 0x7f75dc0b4200 2023-11-06T02:09:12.114 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.105+0000 7f75d97fa640 1 -- 172.21.15.49:0/3143724392 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d400bd10 con 0x7f75dc0b4200 2023-11-06T02:09:12.114 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.105+0000 7f75e2add640 1 -- 172.21.15.49:0/3143724392 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75dc0b37a0 con 0x7f75dc0b4200 2023-11-06T02:09:12.114 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.105+0000 7f75e2add640 1 -- 172.21.15.49:0/3143724392 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75dc151c40 con 0x7f75dc0b4200 2023-11-06T02:09:12.114 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.105+0000 7f75e2add640 1 -- 172.21.15.49:0/3143724392 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75dc140320 con 0x7f75dc0b4200 2023-11-06T02:09:12.115 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.105+0000 7f75d97fa640 1 -- 172.21.15.49:0/3143724392 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f75d4007070 con 0x7f75dc0b4200 2023-11-06T02:09:12.115 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.105+0000 7f75d97fa640 1 --2- 172.21.15.49:0/3143724392 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f75bc065a60 0x7f75bc067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:12.115 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.105+0000 7f75d97fa640 1 -- 172.21.15.49:0/3143724392 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f75d4083590 con 0x7f75dc0b4200 2023-11-06T02:09:12.115 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.105+0000 7f75e0879640 1 --2- 172.21.15.49:0/3143724392 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f75bc065a60 0x7f75bc067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:12.115 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.105+0000 7f75e0879640 1 --2- 172.21.15.49:0/3143724392 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f75bc065a60 0x7f75bc067f20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f75cc0b0000 tx=0x7f75cc0b7040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:12.116 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.109+0000 7f75d97fa640 1 -- 172.21.15.49:0/3143724392 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f75d4050d60 con 0x7f75dc0b4200 2023-11-06T02:09:12.272 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.273+0000 7f75e2add640 1 -- 172.21.15.49:0/3143724392 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f75dc074040 con 0x7f75dc0b4200 2023-11-06T02:09:12.272 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.273+0000 7f75d97fa640 1 -- 172.21.15.49:0/3143724392 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f75d4054380 con 0x7f75dc0b4200 2023-11-06T02:09:12.273 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2023-11-06T02:09:12.274 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.277+0000 7f75e2add640 1 -- 172.21.15.49:0/3143724392 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f75bc065a60 msgr2=0x7f75bc067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:12.275 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.277+0000 7f75e2add640 1 --2- 172.21.15.49:0/3143724392 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f75bc065a60 0x7f75bc067f20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f75cc0b0000 tx=0x7f75cc0b7040 comp rx=0 tx=0).stop 2023-11-06T02:09:12.275 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.277+0000 7f75e2add640 1 -- 172.21.15.49:0/3143724392 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75dc0b4200 msgr2=0x7f75dc13f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:12.275 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.277+0000 7f75e2add640 1 --2- 172.21.15.49:0/3143724392 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75dc0b4200 0x7f75dc13f390 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f75dc0aeb20 tx=0x7f75d400df70 comp rx=0 tx=0).stop 2023-11-06T02:09:12.275 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.277+0000 7f75e2add640 1 -- 172.21.15.49:0/3143724392 shutdown_connections 2023-11-06T02:09:12.275 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.277+0000 7f75e2add640 1 --2- 172.21.15.49:0/3143724392 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f75bc065a60 0x7f75bc067f20 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:12.275 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.277+0000 7f75e2add640 1 --2- 172.21.15.49:0/3143724392 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75dc0b4200 0x7f75dc13f390 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:12.276 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.277+0000 7f75e2add640 1 --2- 172.21.15.49:0/3143724392 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f75dc13d330 0x7f75dc0b3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:12.276 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.277+0000 7f75e2add640 1 --2- 172.21.15.49:0/3143724392 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f75c80039b0 0x7f75dc163b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:12.276 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.277+0000 7f75e2add640 1 -- 172.21.15.49:0/3143724392 >> 172.21.15.49:0/3143724392 conn(0x7f75dc06a370 msgr2=0x7f75dc0af090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:12.276 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.277+0000 7f75e2add640 1 -- 172.21.15.49:0/3143724392 shutdown_connections 2023-11-06T02:09:12.276 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.277+0000 7f75e2add640 1 -- 172.21.15.49:0/3143724392 wait complete. 2023-11-06T02:09:12.290 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T02:09:12.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.417+0000 7ff4cd7e8640 1 Processor -- start 2023-11-06T02:09:12.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.417+0000 7ff4cd7e8640 1 -- start start 2023-11-06T02:09:12.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.417+0000 7ff4cd7e8640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff4c80afe20 0x7ff4c80ce9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:12.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.417+0000 7ff4cd7e8640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4c80d0fe0 0x7ff4c80b0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:12.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.417+0000 7ff4cd7e8640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4c80cf340 con 0x7ff4c806bda0 2023-11-06T02:09:12.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.417+0000 7ff4cd7e8640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4c80cf620 con 0x7ff4c80afe20 2023-11-06T02:09:12.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.417+0000 7ff4cd7e8640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c80cf900 con 0x7ff4c80d0fe0 2023-11-06T02:09:12.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.417+0000 7ff4c6ffd640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff4c80afe20 0x7ff4c80ce9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:12.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.417+0000 7ff4c67fc640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7ff4c806bda0 0x7ff4c80d08d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:60828/0 (socket says 172.21.15.49:60828) 2023-11-06T02:09:12.417 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.417+0000 7ff4c67fc640 1 -- 172.21.15.49:0/2937991965 learned_addr learned my addr 172.21.15.49:0/2937991965 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:09:12.417 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.417+0000 7ff4c6ffd640 1 -- 172.21.15.49:0/2937991965 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4c80d0fe0 msgr2=0x7ff4c80b0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:12.417 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.417+0000 7ff4c6ffd640 1 --2- 172.21.15.49:0/2937991965 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4c80d0fe0 0x7ff4c80b0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:12.417 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.417+0000 7ff4c6ffd640 1 -- 172.21.15.49:0/2937991965 >> v1:172.21.15.49:6789/0 conn(0x7ff4c806bda0 legacy=0x7ff4c80d08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:12.417 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.417+0000 7ff4c5ffb640 1 -- 172.21.15.49:0/2937991965 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3508579989 0 0) 0x7ff4c80cf340 con 0x7ff4c806bda0 2023-11-06T02:09:12.417 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.417+0000 7ff4c6ffd640 1 -- 172.21.15.49:0/2937991965 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4c80cfbe0 con 0x7ff4c80afe20 2023-11-06T02:09:12.419 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.421+0000 7ff4c6ffd640 1 --2- 172.21.15.49:0/2937991965 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4c80afe20 0x7ff4c80ce9f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff4b8009d90 tx=0x7ff4b801c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8081eae2a0c21c55 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:12.419 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.421+0000 7ff4c5ffb640 1 -- 172.21.15.49:0/2937991965 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c80cf140 con 0x7ff4c80afe20 2023-11-06T02:09:12.419 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.421+0000 7ff4c5ffb640 1 -- 172.21.15.49:0/2937991965 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4b80a5920 con 0x7ff4c80afe20 2023-11-06T02:09:12.419 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.421+0000 7ff4c5ffb640 1 -- 172.21.15.49:0/2937991965 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b80a5cb0 con 0x7ff4c80afe20 2023-11-06T02:09:12.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.421+0000 7ff4cd7e8640 1 -- 172.21.15.49:0/2937991965 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4c80afe20 msgr2=0x7ff4c80ce9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:12.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.421+0000 7ff4cd7e8640 1 --2- 172.21.15.49:0/2937991965 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4c80afe20 0x7ff4c80ce9f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff4b8009d90 tx=0x7ff4b801c8c0 comp rx=0 tx=0).stop 2023-11-06T02:09:12.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.421+0000 7ff4cd7e8640 1 -- 172.21.15.49:0/2937991965 shutdown_connections 2023-11-06T02:09:12.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.421+0000 7ff4cd7e8640 1 --2- 172.21.15.49:0/2937991965 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4c80d0fe0 0x7ff4c80b0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:12.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.421+0000 7ff4cd7e8640 1 --2- 172.21.15.49:0/2937991965 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4c80afe20 0x7ff4c80ce9f0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:12.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.421+0000 7ff4cd7e8640 1 -- 172.21.15.49:0/2937991965 >> 172.21.15.49:0/2937991965 conn(0x7ff4c8089fd0 msgr2=0x7ff4c8099c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:12.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.421+0000 7ff4cd7e8640 1 -- 172.21.15.49:0/2937991965 shutdown_connections 2023-11-06T02:09:12.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.421+0000 7ff4cd7e8640 1 -- 172.21.15.49:0/2937991965 wait complete. 2023-11-06T02:09:12.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.421+0000 7ff4cd7e8640 1 Processor -- start 2023-11-06T02:09:12.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.421+0000 7ff4cd7e8640 1 -- start start 2023-11-06T02:09:12.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.421+0000 7ff4cd7e8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4c806bda0 0x7ff4c8157ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:12.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.421+0000 7ff4cd7e8640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4c80afe20 0x7ff4c8158010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:12.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.421+0000 7ff4cd7e8640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4c80d0fe0 0x7ff4c80ae650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:12.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.421+0000 7ff4cd7e8640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c8154270 con 0x7ff4c806bda0 2023-11-06T02:09:12.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.421+0000 7ff4cd7e8640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c80adf60 con 0x7ff4c80d0fe0 2023-11-06T02:09:12.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.421+0000 7ff4cd7e8640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c80af230 con 0x7ff4c80afe20 2023-11-06T02:09:12.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.421+0000 7ff4c6ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4c806bda0 0x7ff4c8157ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:12.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.421+0000 7ff4c6ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4c806bda0 0x7ff4c8157ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48214/0 (socket says 172.21.15.49:48214) 2023-11-06T02:09:12.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.421+0000 7ff4c6ffd640 1 -- 172.21.15.49:0/114735374 learned_addr learned my addr 172.21.15.49:0/114735374 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:12.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.421+0000 7ff4c67fc640 1 --2- 172.21.15.49:0/114735374 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4c80afe20 0x7ff4c8158010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:12.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.421+0000 7ff4c6ffd640 1 -- 172.21.15.49:0/114735374 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4c80afe20 msgr2=0x7ff4c8158010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:12.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.421+0000 7ff4c6ffd640 1 --2- 172.21.15.49:0/114735374 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4c80afe20 0x7ff4c8158010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:12.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.421+0000 7ff4c6ffd640 1 -- 172.21.15.49:0/114735374 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4c80d0fe0 msgr2=0x7ff4c80ae650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:12.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.421+0000 7ff4c77fe640 1 --2- 172.21.15.49:0/114735374 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4c80d0fe0 0x7ff4c80ae650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:12.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.421+0000 7ff4c6ffd640 1 --2- 172.21.15.49:0/114735374 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4c80d0fe0 0x7ff4c80ae650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:12.427 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.421+0000 7ff4c6ffd640 1 -- 172.21.15.49:0/114735374 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4b8009a70 con 0x7ff4c806bda0 2023-11-06T02:09:12.427 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.421+0000 7ff4c6ffd640 1 --2- 172.21.15.49:0/114735374 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4c806bda0 0x7ff4c8157ad0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff4b801a4f0 tx=0x7ff4b8019660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:12.427 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.421+0000 7ff4af7fe640 1 -- 172.21.15.49:0/114735374 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b80ad950 con 0x7ff4c806bda0 2023-11-06T02:09:12.427 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.421+0000 7ff4af7fe640 1 -- 172.21.15.49:0/114735374 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4b80adab0 con 0x7ff4c806bda0 2023-11-06T02:09:12.427 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.421+0000 7ff4af7fe640 1 -- 172.21.15.49:0/114735374 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b80ade40 con 0x7ff4c806bda0 2023-11-06T02:09:12.427 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.421+0000 7ff4cd7e8640 1 -- 172.21.15.49:0/114735374 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4c80ae3f0 con 0x7ff4c806bda0 2023-11-06T02:09:12.428 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.421+0000 7ff4cd7e8640 1 -- 172.21.15.49:0/114735374 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4c80ad910 con 0x7ff4c806bda0 2023-11-06T02:09:12.428 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.425+0000 7ff4cd7e8640 1 -- 172.21.15.49:0/114735374 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4c8067140 con 0x7ff4c806bda0 2023-11-06T02:09:12.428 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.425+0000 7ff4af7fe640 1 -- 172.21.15.49:0/114735374 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7ff4b80b4050 con 0x7ff4c806bda0 2023-11-06T02:09:12.428 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.425+0000 7ff4af7fe640 1 --2- 172.21.15.49:0/114735374 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff4b4065a60 0x7ff4b4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:12.428 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.425+0000 7ff4af7fe640 1 -- 172.21.15.49:0/114735374 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6120+0+0 (secure 0 0 0) 0x7ff4b811e3f0 con 0x7ff4c806bda0 2023-11-06T02:09:12.429 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.425+0000 7ff4c67fc640 1 --2- 172.21.15.49:0/114735374 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff4b4065a60 0x7ff4b4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:12.429 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.425+0000 7ff4c67fc640 1 --2- 172.21.15.49:0/114735374 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff4b4065a60 0x7ff4b4067f20 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc007fd0 tx=0x7ff4bc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:12.429 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.429+0000 7ff4af7fe640 1 -- 172.21.15.49:0/114735374 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7ff4b80ebd40 con 0x7ff4c806bda0 2023-11-06T02:09:12.582 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.585+0000 7ff4cd7e8640 1 -- 172.21.15.49:0/114735374 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff4c815c6c0 con 0x7ff4c806bda0 2023-11-06T02:09:12.583 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.585+0000 7ff4af7fe640 1 -- 172.21.15.49:0/114735374 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff4b80b2020 con 0x7ff4c806bda0 2023-11-06T02:09:12.584 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 8 2023-11-06T02:09:12.585 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.589+0000 7ff4cd7e8640 1 -- 172.21.15.49:0/114735374 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff4b4065a60 msgr2=0x7ff4b4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:12.586 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.589+0000 7ff4cd7e8640 1 --2- 172.21.15.49:0/114735374 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff4b4065a60 0x7ff4b4067f20 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc007fd0 tx=0x7ff4bc009040 comp rx=0 tx=0).stop 2023-11-06T02:09:12.586 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.589+0000 7ff4cd7e8640 1 -- 172.21.15.49:0/114735374 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4c806bda0 msgr2=0x7ff4c8157ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:12.586 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.589+0000 7ff4cd7e8640 1 --2- 172.21.15.49:0/114735374 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4c806bda0 0x7ff4c8157ad0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff4b801a4f0 tx=0x7ff4b8019660 comp rx=0 tx=0).stop 2023-11-06T02:09:12.586 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.589+0000 7ff4cd7e8640 1 -- 172.21.15.49:0/114735374 shutdown_connections 2023-11-06T02:09:12.587 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.589+0000 7ff4cd7e8640 1 --2- 172.21.15.49:0/114735374 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff4b4065a60 0x7ff4b4067f20 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:12.587 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.589+0000 7ff4cd7e8640 1 --2- 172.21.15.49:0/114735374 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4c80d0fe0 0x7ff4c80ae650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:12.587 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.589+0000 7ff4cd7e8640 1 --2- 172.21.15.49:0/114735374 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4c80afe20 0x7ff4c8158010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:12.587 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.589+0000 7ff4cd7e8640 1 --2- 172.21.15.49:0/114735374 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4c806bda0 0x7ff4c8157ad0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:12.587 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.589+0000 7ff4cd7e8640 1 -- 172.21.15.49:0/114735374 >> 172.21.15.49:0/114735374 conn(0x7ff4c8089fd0 msgr2=0x7ff4c815c950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:12.588 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.589+0000 7ff4cd7e8640 1 -- 172.21.15.49:0/114735374 shutdown_connections 2023-11-06T02:09:12.588 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.589+0000 7ff4cd7e8640 1 -- 172.21.15.49:0/114735374 wait complete. 2023-11-06T02:09:12.602 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T02:09:12.721 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.721+0000 7f24476f5640 1 Processor -- start 2023-11-06T02:09:12.721 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.721+0000 7f24476f5640 1 -- start start 2023-11-06T02:09:12.722 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.725+0000 7f24476f5640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f2440153d50 0x7f244015d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:12.722 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.725+0000 7f24476f5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f24400707b0 0x7f2440070bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:12.723 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.725+0000 7f24476f5640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f244015e100 con 0x7f24400c9100 2023-11-06T02:09:12.723 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.725+0000 7f24476f5640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f244006e0a0 con 0x7f2440153d50 2023-11-06T02:09:12.723 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.725+0000 7f24476f5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2440154130 con 0x7f24400707b0 2023-11-06T02:09:12.723 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.725+0000 7f2445491640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f2440153d50 0x7f244015d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:12.723 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.725+0000 7f2444c90640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f24400c9100 0x7f244015e320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:60830/0 (socket says 172.21.15.49:60830) 2023-11-06T02:09:12.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.725+0000 7f2444c90640 1 -- 172.21.15.49:0/1976129047 learned_addr learned my addr 172.21.15.49:0/1976129047 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:09:12.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.725+0000 7f2445c92640 1 --2- 172.21.15.49:0/1976129047 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f24400707b0 0x7f2440070bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:12.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.725+0000 7f2445491640 1 -- 172.21.15.49:0/1976129047 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f24400707b0 msgr2=0x7f2440070bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:12.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.725+0000 7f2445491640 1 --2- 172.21.15.49:0/1976129047 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f24400707b0 0x7f2440070bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:12.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.725+0000 7f2445491640 1 -- 172.21.15.49:0/1976129047 >> v1:172.21.15.49:6789/0 conn(0x7f24400c9100 legacy=0x7f244015e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:12.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.725+0000 7f2445491640 1 -- 172.21.15.49:0/1976129047 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2440071100 con 0x7f2440153d50 2023-11-06T02:09:12.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.725+0000 7f2445c92640 1 --2- 172.21.15.49:0/1976129047 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f24400707b0 0x7f2440070bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:09:12.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.725+0000 7f2445491640 1 --2- 172.21.15.49:0/1976129047 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2440153d50 0x7f244015d160 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f24300097c0 tx=0x7f243001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f29a922bb97e8fb5 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:12.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.725+0000 7f2437fff640 1 -- 172.21.15.49:0/1976129047 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24300ab5f0 con 0x7f2440153d50 2023-11-06T02:09:12.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.725+0000 7f2437fff640 1 -- 172.21.15.49:0/1976129047 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24300ab750 con 0x7f2440153d50 2023-11-06T02:09:12.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.725+0000 7f2437fff640 1 -- 172.21.15.49:0/1976129047 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24300abae0 con 0x7f2440153d50 2023-11-06T02:09:12.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.729+0000 7f24476f5640 1 -- 172.21.15.49:0/1976129047 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2440153d50 msgr2=0x7f244015d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:12.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.729+0000 7f24476f5640 1 --2- 172.21.15.49:0/1976129047 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2440153d50 0x7f244015d160 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f24300097c0 tx=0x7f243001c3f0 comp rx=0 tx=0).stop 2023-11-06T02:09:12.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.729+0000 7f24476f5640 1 -- 172.21.15.49:0/1976129047 shutdown_connections 2023-11-06T02:09:12.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.729+0000 7f24476f5640 1 --2- 172.21.15.49:0/1976129047 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f24400707b0 0x7f2440070bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:12.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.729+0000 7f24476f5640 1 --2- 172.21.15.49:0/1976129047 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2440153d50 0x7f244015d160 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:12.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.729+0000 7f24476f5640 1 -- 172.21.15.49:0/1976129047 >> 172.21.15.49:0/1976129047 conn(0x7f244006a370 msgr2=0x7f24400ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:12.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.729+0000 7f24476f5640 1 -- 172.21.15.49:0/1976129047 shutdown_connections 2023-11-06T02:09:12.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.729+0000 7f24476f5640 1 -- 172.21.15.49:0/1976129047 wait complete. 2023-11-06T02:09:12.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.729+0000 7f24476f5640 1 Processor -- start 2023-11-06T02:09:12.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.729+0000 7f24476f5640 1 -- start start 2023-11-06T02:09:12.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.729+0000 7f24476f5640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f24400707b0 0x7f2440145be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:12.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.729+0000 7f24476f5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f24400c9100 0x7f244014d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:12.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.729+0000 7f24476f5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2440153d50 0x7f24401464a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:12.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.729+0000 7f24476f5640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f244014c5a0 con 0x7f24400c9100 2023-11-06T02:09:12.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.729+0000 7f24476f5640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24400b18b0 con 0x7f24400707b0 2023-11-06T02:09:12.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.729+0000 7f24476f5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24400b1b90 con 0x7f2440153d50 2023-11-06T02:09:12.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.729+0000 7f2444c90640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f24400c9100 0x7f244014d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:12.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.729+0000 7f2444c90640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f24400c9100 0x7f244014d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48226/0 (socket says 172.21.15.49:48226) 2023-11-06T02:09:12.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.729+0000 7f2444c90640 1 -- 172.21.15.49:0/1463276858 learned_addr learned my addr 172.21.15.49:0/1463276858 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:12.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.729+0000 7f2445491640 1 --2- 172.21.15.49:0/1463276858 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f24400707b0 0x7f2440145be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:12.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.729+0000 7f2445c92640 1 --2- 172.21.15.49:0/1463276858 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2440153d50 0x7f24401464a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:12.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.729+0000 7f2444c90640 1 -- 172.21.15.49:0/1463276858 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2440153d50 msgr2=0x7f24401464a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:12.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.729+0000 7f2444c90640 1 --2- 172.21.15.49:0/1463276858 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2440153d50 0x7f24401464a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:12.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.729+0000 7f2444c90640 1 -- 172.21.15.49:0/1463276858 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f24400707b0 msgr2=0x7f2440145be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:12.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.729+0000 7f2444c90640 1 --2- 172.21.15.49:0/1463276858 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f24400707b0 0x7f2440145be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:12.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.729+0000 7f2444c90640 1 -- 172.21.15.49:0/1463276858 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2430009420 con 0x7f24400c9100 2023-11-06T02:09:12.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.729+0000 7f2444c90640 1 --2- 172.21.15.49:0/1463276858 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f24400c9100 0x7f244014d900 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f243c00c500 tx=0x7f243c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:12.736 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.729+0000 7f2435ffb640 1 -- 172.21.15.49:0/1463276858 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243c005cf0 con 0x7f24400c9100 2023-11-06T02:09:12.736 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.729+0000 7f2435ffb640 1 -- 172.21.15.49:0/1463276858 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f243c005e50 con 0x7f24400c9100 2023-11-06T02:09:12.736 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.729+0000 7f24476f5640 1 -- 172.21.15.49:0/1463276858 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24400b1e70 con 0x7f24400c9100 2023-11-06T02:09:12.736 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.729+0000 7f2435ffb640 1 -- 172.21.15.49:0/1463276858 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243c0072f0 con 0x7f24400c9100 2023-11-06T02:09:12.736 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.729+0000 7f24476f5640 1 -- 172.21.15.49:0/1463276858 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2440146120 con 0x7f24400c9100 2023-11-06T02:09:12.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.733+0000 7f24476f5640 1 -- 172.21.15.49:0/1463276858 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2440074040 con 0x7f24400c9100 2023-11-06T02:09:12.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.733+0000 7f2435ffb640 1 -- 172.21.15.49:0/1463276858 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f243c007450 con 0x7f24400c9100 2023-11-06T02:09:12.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.733+0000 7f2435ffb640 1 --2- 172.21.15.49:0/1463276858 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2424065a90 0x7f2424067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:12.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.733+0000 7f2435ffb640 1 -- 172.21.15.49:0/1463276858 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f243c0826e0 con 0x7f24400c9100 2023-11-06T02:09:12.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.733+0000 7f2445491640 1 --2- 172.21.15.49:0/1463276858 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2424065a90 0x7f2424067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:12.738 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.733+0000 7f2445491640 1 --2- 172.21.15.49:0/1463276858 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2424065a90 0x7f2424067f50 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f24300af000 tx=0x7f24300a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:12.738 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.737+0000 7f2435ffb640 1 -- 172.21.15.49:0/1463276858 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f243c050040 con 0x7f24400c9100 2023-11-06T02:09:12.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.885+0000 7f24476f5640 1 -- 172.21.15.49:0/1463276858 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f24400b0dd0 con 0x7f24400c9100 2023-11-06T02:09:12.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.885+0000 7f2435ffb640 1 -- 172.21.15.49:0/1463276858 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f243c053660 con 0x7f24400c9100 2023-11-06T02:09:12.884 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2023-11-06T02:09:12.885 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.885+0000 7f24476f5640 1 -- 172.21.15.49:0/1463276858 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2424065a90 msgr2=0x7f2424067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:12.886 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.885+0000 7f24476f5640 1 --2- 172.21.15.49:0/1463276858 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2424065a90 0x7f2424067f50 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f24300af000 tx=0x7f24300a7000 comp rx=0 tx=0).stop 2023-11-06T02:09:12.886 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.885+0000 7f24476f5640 1 -- 172.21.15.49:0/1463276858 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f24400c9100 msgr2=0x7f244014d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:12.886 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.885+0000 7f24476f5640 1 --2- 172.21.15.49:0/1463276858 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f24400c9100 0x7f244014d900 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f243c00c500 tx=0x7f243c00c9c0 comp rx=0 tx=0).stop 2023-11-06T02:09:12.886 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.889+0000 7f24476f5640 1 -- 172.21.15.49:0/1463276858 shutdown_connections 2023-11-06T02:09:12.886 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.889+0000 7f24476f5640 1 --2- 172.21.15.49:0/1463276858 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2424065a90 0x7f2424067f50 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:12.887 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.889+0000 7f24476f5640 1 --2- 172.21.15.49:0/1463276858 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f24400c9100 0x7f244014d900 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:12.887 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.889+0000 7f24476f5640 1 --2- 172.21.15.49:0/1463276858 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2440153d50 0x7f24401464a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:12.887 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.889+0000 7f24476f5640 1 --2- 172.21.15.49:0/1463276858 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f24400707b0 0x7f2440145be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:12.888 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.889+0000 7f24476f5640 1 -- 172.21.15.49:0/1463276858 >> 172.21.15.49:0/1463276858 conn(0x7f244006a370 msgr2=0x7f244015ad80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:12.888 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.889+0000 7f24476f5640 1 -- 172.21.15.49:0/1463276858 shutdown_connections 2023-11-06T02:09:12.888 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:12.889+0000 7f24476f5640 1 -- 172.21.15.49:0/1463276858 wait complete. 2023-11-06T02:09:12.901 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T02:09:13.026 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.029+0000 7f4abca70640 1 Processor -- start 2023-11-06T02:09:13.026 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.029+0000 7f4abca70640 1 -- start start 2023-11-06T02:09:13.027 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.029+0000 7f4abca70640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f4ab815e4a0 0x7f4ab80ce840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:13.028 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.029+0000 7f4abca70640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f4ab80b16e0 0x7f4ab80b1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:13.028 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.029+0000 7f4abca70640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ab80d1430 con 0x7f4ab815e0c0 2023-11-06T02:09:13.028 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.029+0000 7f4abca70640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ab80cd8d0 con 0x7f4ab80b16e0 2023-11-06T02:09:13.028 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.029+0000 7f4abca70640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ab80b2090 con 0x7f4ab815e4a0 2023-11-06T02:09:13.028 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.029+0000 7f4ab6d9d640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f4ab80b16e0 0x7f4ab80b1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:13.029 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.029+0000 7f4ab659c640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f4ab815e0c0 0x7f4ab80cdcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:60838/0 (socket says 172.21.15.49:60838) 2023-11-06T02:09:13.029 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.029+0000 7f4ab659c640 1 -- 172.21.15.49:0/2191820003 learned_addr learned my addr 172.21.15.49:0/2191820003 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:09:13.029 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.029+0000 7f4ab5d9b640 1 --2- 172.21.15.49:0/2191820003 >> v2:172.21.15.150:3300/0 conn(0x7f4ab815e4a0 0x7f4ab80ce840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:13.029 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.029+0000 7f4ab559a640 1 -- 172.21.15.49:0/2191820003 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 349246962 0 0) 0x7f4ab80d1430 con 0x7f4ab815e0c0 2023-11-06T02:09:13.029 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.029+0000 7f4ab559a640 1 -- 172.21.15.49:0/2191820003 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4aa0003530 con 0x7f4ab815e0c0 2023-11-06T02:09:13.029 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.029+0000 7f4ab6d9d640 1 -- 172.21.15.49:0/2191820003 >> v2:172.21.15.150:3300/0 conn(0x7f4ab815e4a0 msgr2=0x7f4ab80ce840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:13.030 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.029+0000 7f4ab6d9d640 1 --2- 172.21.15.49:0/2191820003 >> v2:172.21.15.150:3300/0 conn(0x7f4ab815e4a0 0x7f4ab80ce840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:13.030 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.029+0000 7f4ab6d9d640 1 -- 172.21.15.49:0/2191820003 >> v1:172.21.15.49:6789/0 conn(0x7f4ab815e0c0 legacy=0x7f4ab80cdcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:13.030 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.029+0000 7f4ab6d9d640 1 -- 172.21.15.49:0/2191820003 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ab80d1710 con 0x7f4ab80b16e0 2023-11-06T02:09:13.031 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.033+0000 7f4ab6d9d640 1 --2- 172.21.15.49:0/2191820003 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4ab80b16e0 0x7f4ab80b1b50 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4ab8073c70 tx=0x7f4aa801c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=889f0f5662d60a30 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:13.032 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.033+0000 7f4ab559a640 1 -- 172.21.15.49:0/2191820003 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa80ab600 con 0x7f4ab80b16e0 2023-11-06T02:09:13.032 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.033+0000 7f4ab559a640 1 -- 172.21.15.49:0/2191820003 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4aa80a7050 con 0x7f4ab80b16e0 2023-11-06T02:09:13.032 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.033+0000 7f4ab559a640 1 -- 172.21.15.49:0/2191820003 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa80ab980 con 0x7f4ab80b16e0 2023-11-06T02:09:13.032 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.033+0000 7f4abca70640 1 -- 172.21.15.49:0/2191820003 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4ab80b16e0 msgr2=0x7f4ab80b1b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:13.032 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.033+0000 7f4abca70640 1 --2- 172.21.15.49:0/2191820003 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4ab80b16e0 0x7f4ab80b1b50 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4ab8073c70 tx=0x7f4aa801c3d0 comp rx=0 tx=0).stop 2023-11-06T02:09:13.033 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.033+0000 7f4abca70640 1 -- 172.21.15.49:0/2191820003 shutdown_connections 2023-11-06T02:09:13.033 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.033+0000 7f4abca70640 1 --2- 172.21.15.49:0/2191820003 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4ab80b16e0 0x7f4ab80b1b50 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:13.033 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.033+0000 7f4abca70640 1 --2- 172.21.15.49:0/2191820003 >> v2:172.21.15.150:3300/0 conn(0x7f4ab815e4a0 0x7f4ab80ce840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:13.033 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.033+0000 7f4abca70640 1 -- 172.21.15.49:0/2191820003 >> 172.21.15.49:0/2191820003 conn(0x7f4ab8099c50 msgr2=0x7f4ab80d0940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:13.033 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.033+0000 7f4abca70640 1 -- 172.21.15.49:0/2191820003 shutdown_connections 2023-11-06T02:09:13.034 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.033+0000 7f4abca70640 1 -- 172.21.15.49:0/2191820003 wait complete. 2023-11-06T02:09:13.034 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.033+0000 7f4abca70640 1 Processor -- start 2023-11-06T02:09:13.034 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.033+0000 7f4abca70640 1 -- start start 2023-11-06T02:09:13.036 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.033+0000 7f4abca70640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4ab80b16e0 0x7f4ab815d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:13.036 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.033+0000 7f4abca70640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4ab815e0c0 0x7f4ab814cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:13.036 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.033+0000 7f4abca70640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4ab815e4a0 0x7f4ab80ad5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:13.036 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.033+0000 7f4abca70640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab815d990 con 0x7f4ab80b16e0 2023-11-06T02:09:13.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.033+0000 7f4abca70640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab80aef30 con 0x7f4ab815e4a0 2023-11-06T02:09:13.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.033+0000 7f4abca70640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab815ba80 con 0x7f4ab815e0c0 2023-11-06T02:09:13.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.033+0000 7f4ab659c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4ab80b16e0 0x7f4ab815d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:13.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.033+0000 7f4ab659c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4ab80b16e0 0x7f4ab815d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48242/0 (socket says 172.21.15.49:48242) 2023-11-06T02:09:13.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.033+0000 7f4ab659c640 1 -- 172.21.15.49:0/2777808179 learned_addr learned my addr 172.21.15.49:0/2777808179 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:13.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.033+0000 7f4ab659c640 1 -- 172.21.15.49:0/2777808179 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4ab815e0c0 msgr2=0x7f4ab814cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:13.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.033+0000 7f4ab659c640 1 --2- 172.21.15.49:0/2777808179 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4ab815e0c0 0x7f4ab814cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:13.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.033+0000 7f4ab659c640 1 -- 172.21.15.49:0/2777808179 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4ab815e4a0 msgr2=0x7f4ab80ad5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:13.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.033+0000 7f4ab659c640 1 --2- 172.21.15.49:0/2777808179 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4ab815e4a0 0x7f4ab80ad5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:13.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.033+0000 7f4ab659c640 1 -- 172.21.15.49:0/2777808179 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aa8009420 con 0x7f4ab80b16e0 2023-11-06T02:09:13.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.033+0000 7f4ab659c640 1 --2- 172.21.15.49:0/2777808179 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4ab80b16e0 0x7f4ab815d450 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f4ab000cbc0 tx=0x7f4ab000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:13.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.033+0000 7f4aa6ffd640 1 -- 172.21.15.49:0/2777808179 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab000ace0 con 0x7f4ab80b16e0 2023-11-06T02:09:13.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.033+0000 7f4aa6ffd640 1 -- 172.21.15.49:0/2777808179 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ab000ae40 con 0x7f4ab80b16e0 2023-11-06T02:09:13.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.033+0000 7f4abca70640 1 -- 172.21.15.49:0/2777808179 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ab80ae8e0 con 0x7f4ab80b16e0 2023-11-06T02:09:13.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.033+0000 7f4aa6ffd640 1 -- 172.21.15.49:0/2777808179 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab0012600 con 0x7f4ab80b16e0 2023-11-06T02:09:13.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.037+0000 7f4abca70640 1 -- 172.21.15.49:0/2777808179 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ab814d240 con 0x7f4ab80b16e0 2023-11-06T02:09:13.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.037+0000 7f4abca70640 1 -- 172.21.15.49:0/2777808179 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ab8067140 con 0x7f4ab80b16e0 2023-11-06T02:09:13.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.037+0000 7f4aa6ffd640 1 -- 172.21.15.49:0/2777808179 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f4ab0007070 con 0x7f4ab80b16e0 2023-11-06T02:09:13.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.037+0000 7f4aa6ffd640 1 --2- 172.21.15.49:0/2777808179 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4a98065a60 0x7f4a98067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:13.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.037+0000 7f4aa6ffd640 1 -- 172.21.15.49:0/2777808179 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f4ab0083250 con 0x7f4ab80b16e0 2023-11-06T02:09:13.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.037+0000 7f4ab5d9b640 1 --2- 172.21.15.49:0/2777808179 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4a98065a60 0x7f4a98067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:13.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.037+0000 7f4ab5d9b640 1 --2- 172.21.15.49:0/2777808179 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4a98065a60 0x7f4a98067f20 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4ab80d0f70 tx=0x7f4aac008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:13.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.041+0000 7f4aa6ffd640 1 -- 172.21.15.49:0/2777808179 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f4ab0050b20 con 0x7f4ab80b16e0 2023-11-06T02:09:13.199 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.201+0000 7f4abca70640 1 -- 172.21.15.49:0/2777808179 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4ab808a060 con 0x7f4ab80b16e0 2023-11-06T02:09:13.205 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.209+0000 7f4aa6ffd640 1 -- 172.21.15.49:0/2777808179 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4ab0054140 con 0x7f4ab80b16e0 2023-11-06T02:09:13.206 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2023-11-06T02:09:13.208 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.209+0000 7f4abca70640 1 -- 172.21.15.49:0/2777808179 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4a98065a60 msgr2=0x7f4a98067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:13.209 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.209+0000 7f4abca70640 1 --2- 172.21.15.49:0/2777808179 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4a98065a60 0x7f4a98067f20 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4ab80d0f70 tx=0x7f4aac008040 comp rx=0 tx=0).stop 2023-11-06T02:09:13.209 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.209+0000 7f4abca70640 1 -- 172.21.15.49:0/2777808179 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4ab80b16e0 msgr2=0x7f4ab815d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:13.209 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.209+0000 7f4abca70640 1 --2- 172.21.15.49:0/2777808179 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4ab80b16e0 0x7f4ab815d450 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f4ab000cbc0 tx=0x7f4ab000a430 comp rx=0 tx=0).stop 2023-11-06T02:09:13.209 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.209+0000 7f4abca70640 1 -- 172.21.15.49:0/2777808179 shutdown_connections 2023-11-06T02:09:13.210 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.209+0000 7f4abca70640 1 --2- 172.21.15.49:0/2777808179 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4a98065a60 0x7f4a98067f20 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:13.210 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.209+0000 7f4abca70640 1 --2- 172.21.15.49:0/2777808179 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4ab815e4a0 0x7f4ab80ad5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:13.210 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.209+0000 7f4abca70640 1 --2- 172.21.15.49:0/2777808179 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4ab815e0c0 0x7f4ab814cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:13.210 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.209+0000 7f4abca70640 1 --2- 172.21.15.49:0/2777808179 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4ab80b16e0 0x7f4ab815d450 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:13.210 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.209+0000 7f4abca70640 1 -- 172.21.15.49:0/2777808179 >> 172.21.15.49:0/2777808179 conn(0x7f4ab8099c50 msgr2=0x7f4ab80cbd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:13.211 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.209+0000 7f4abca70640 1 -- 172.21.15.49:0/2777808179 shutdown_connections 2023-11-06T02:09:13.211 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.209+0000 7f4abca70640 1 -- 172.21.15.49:0/2777808179 wait complete. 2023-11-06T02:09:13.226 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-06T02:09:13.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.345+0000 7f7b501cf640 1 Processor -- start 2023-11-06T02:09:13.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.345+0000 7f7b501cf640 1 -- start start 2023-11-06T02:09:13.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.345+0000 7f7b501cf640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f7b48159980 0x7f7b48071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:13.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.345+0000 7f7b501cf640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7b480cb010 0x7f7b480cb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:13.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.345+0000 7f7b501cf640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b48071890 con 0x7f7b480b1450 2023-11-06T02:09:13.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.345+0000 7f7b501cf640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b48072410 con 0x7f7b48159980 2023-11-06T02:09:13.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.345+0000 7f7b501cf640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b480ce440 con 0x7f7b480cb010 2023-11-06T02:09:13.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.345+0000 7f7b4d76a640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f7b480b1450 0x7f7b480cdd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:60840/0 (socket says 172.21.15.49:60840) 2023-11-06T02:09:13.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.345+0000 7f7b4d76a640 1 -- 172.21.15.49:0/240052946 learned_addr learned my addr 172.21.15.49:0/240052946 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:09:13.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.345+0000 7f7b4cf69640 1 -- 172.21.15.49:0/240052946 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1594577721 0 0) 0x7f7b48071890 con 0x7f7b480b1450 2023-11-06T02:09:13.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.345+0000 7f7b4e76c640 1 --2- 172.21.15.49:0/240052946 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7b480cb010 0x7f7b480cb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:13.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.345+0000 7f7b4df6b640 1 --2- 172.21.15.49:0/240052946 >> v2:172.21.15.150:3300/0 conn(0x7f7b48159980 0x7f7b48071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:13.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.345+0000 7f7b4cf69640 1 -- 172.21.15.49:0/240052946 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b34003530 con 0x7f7b480b1450 2023-11-06T02:09:13.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.345+0000 7f7b4cf69640 1 -- 172.21.15.49:0/240052946 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 21060084 0 0) 0x7f7b34003530 con 0x7f7b480b1450 2023-11-06T02:09:13.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.349+0000 7f7b4cf69640 1 -- 172.21.15.49:0/240052946 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7b480cb010 msgr2=0x7f7b480cb4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:13.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.349+0000 7f7b4cf69640 1 --2- 172.21.15.49:0/240052946 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7b480cb010 0x7f7b480cb4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:13.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.349+0000 7f7b4cf69640 1 -- 172.21.15.49:0/240052946 >> v2:172.21.15.150:3300/0 conn(0x7f7b48159980 msgr2=0x7f7b48071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:13.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.349+0000 7f7b4cf69640 1 --2- 172.21.15.49:0/240052946 >> v2:172.21.15.150:3300/0 conn(0x7f7b48159980 0x7f7b48071290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:13.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.349+0000 7f7b4cf69640 1 -- 172.21.15.49:0/240052946 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b48071b70 con 0x7f7b480b1450 2023-11-06T02:09:13.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.349+0000 7f7b4cf69640 1 -- 172.21.15.49:0/240052946 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7f7b38001680 con 0x7f7b480b1450 2023-11-06T02:09:13.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.349+0000 7f7b4cf69640 1 -- 172.21.15.49:0/240052946 >> v1:172.21.15.49:6789/0 conn(0x7f7b480b1450 legacy=0x7f7b480cdd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:13.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.349+0000 7f7b4cf69640 1 --2- 172.21.15.49:0/240052946 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7b480cb010 0x7f7b340039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:13.350 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.349+0000 7f7b4cf69640 1 --2- 172.21.15.49:0/240052946 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b34003f30 0x7f7b340047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:13.350 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.349+0000 7f7b4cf69640 1 --2- 172.21.15.49:0/240052946 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b34004d10 0x7f7b34005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:13.350 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.349+0000 7f7b4cf69640 1 -- 172.21.15.49:0/240052946 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b48072410 con 0x7f7b34004d10 2023-11-06T02:09:13.350 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.349+0000 7f7b4cf69640 1 -- 172.21.15.49:0/240052946 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b480ce440 con 0x7f7b34003f30 2023-11-06T02:09:13.350 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.349+0000 7f7b4cf69640 1 -- 172.21.15.49:0/240052946 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3400b990 con 0x7f7b480cb010 2023-11-06T02:09:13.351 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.349+0000 7f7b4e76c640 1 --2- 172.21.15.49:0/240052946 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b34004d10 0x7f7b34005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:13.351 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.349+0000 7f7b4e76c640 1 -- 172.21.15.49:0/240052946 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7b480cb010 msgr2=0x7f7b340039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:13.351 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.349+0000 7f7b4df6b640 1 --2- 172.21.15.49:0/240052946 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7b480cb010 0x7f7b340039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:13.351 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.349+0000 7f7b4e76c640 1 --2- 172.21.15.49:0/240052946 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7b480cb010 0x7f7b340039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:13.351 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.349+0000 7f7b4d76a640 1 --2- 172.21.15.49:0/240052946 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b34003f30 0x7f7b340047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:13.352 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.349+0000 7f7b4e76c640 1 -- 172.21.15.49:0/240052946 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b34003f30 msgr2=0x7f7b340047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:13.352 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.349+0000 7f7b4e76c640 1 --2- 172.21.15.49:0/240052946 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b34003f30 0x7f7b340047d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:13.352 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.349+0000 7f7b4e76c640 1 -- 172.21.15.49:0/240052946 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b34003530 con 0x7f7b34004d10 2023-11-06T02:09:13.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.349+0000 7f7b4e76c640 1 --2- 172.21.15.49:0/240052946 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b34004d10 0x7f7b34005190 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f7b40009fc0 tx=0x7f7b4001cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a49d677a2192c5f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:13.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.349+0000 7f7b4cf69640 1 -- 172.21.15.49:0/240052946 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b400a5c70 con 0x7f7b34004d10 2023-11-06T02:09:13.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.349+0000 7f7b4cf69640 1 -- 172.21.15.49:0/240052946 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b4000f040 con 0x7f7b34004d10 2023-11-06T02:09:13.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.349+0000 7f7b4cf69640 1 -- 172.21.15.49:0/240052946 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b400ad440 con 0x7f7b34004d10 2023-11-06T02:09:13.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.349+0000 7f7b501cf640 1 -- 172.21.15.49:0/240052946 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b34004d10 msgr2=0x7f7b34005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:13.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.349+0000 7f7b501cf640 1 --2- 172.21.15.49:0/240052946 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b34004d10 0x7f7b34005190 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f7b40009fc0 tx=0x7f7b4001cc60 comp rx=0 tx=0).stop 2023-11-06T02:09:13.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.349+0000 7f7b4e76c640 1 -- 172.21.15.49:0/240052946 reap_dead start 2023-11-06T02:09:13.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.349+0000 7f7b501cf640 1 -- 172.21.15.49:0/240052946 shutdown_connections 2023-11-06T02:09:13.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.349+0000 7f7b501cf640 1 -- 172.21.15.49:0/240052946 >> 172.21.15.49:0/240052946 conn(0x7f7b4806c0a0 msgr2=0x7f7b4806c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:13.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.349+0000 7f7b501cf640 1 -- 172.21.15.49:0/240052946 shutdown_connections 2023-11-06T02:09:13.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.349+0000 7f7b501cf640 1 -- 172.21.15.49:0/240052946 wait complete. 2023-11-06T02:09:13.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.353+0000 7f7b501cf640 1 Processor -- start 2023-11-06T02:09:13.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.353+0000 7f7b501cf640 1 -- start start 2023-11-06T02:09:13.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.353+0000 7f7b501cf640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b4814c6b0 0x7f7b4815c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:13.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.353+0000 7f7b501cf640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7b4815c850 0x7f7b480c70e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:13.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.353+0000 7f7b501cf640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b480c7620 0x7f7b480c3180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:13.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.353+0000 7f7b501cf640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b480b1890 con 0x7f7b480c7620 2023-11-06T02:09:13.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.353+0000 7f7b501cf640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b4815ccd0 con 0x7f7b4814c6b0 2023-11-06T02:09:13.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.353+0000 7f7b501cf640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b480c89d0 con 0x7f7b4815c850 2023-11-06T02:09:13.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.353+0000 7f7b4e76c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b480c7620 0x7f7b480c3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:13.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.353+0000 7f7b4e76c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b480c7620 0x7f7b480c3180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48262/0 (socket says 172.21.15.49:48262) 2023-11-06T02:09:13.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.353+0000 7f7b4e76c640 1 -- 172.21.15.49:0/107682765 learned_addr learned my addr 172.21.15.49:0/107682765 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:13.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.353+0000 7f7b4d76a640 1 --2- 172.21.15.49:0/107682765 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7b4815c850 0x7f7b480c70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:13.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.353+0000 7f7b4df6b640 1 --2- 172.21.15.49:0/107682765 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b4814c6b0 0x7f7b4815c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:13.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.353+0000 7f7b4e76c640 1 -- 172.21.15.49:0/107682765 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7b4815c850 msgr2=0x7f7b480c70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:13.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.353+0000 7f7b4e76c640 1 --2- 172.21.15.49:0/107682765 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7b4815c850 0x7f7b480c70e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:13.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.353+0000 7f7b4e76c640 1 -- 172.21.15.49:0/107682765 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b4814c6b0 msgr2=0x7f7b4815c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:13.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.353+0000 7f7b4e76c640 1 --2- 172.21.15.49:0/107682765 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b4814c6b0 0x7f7b4815c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:13.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.353+0000 7f7b4e76c640 1 -- 172.21.15.49:0/107682765 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b480c5a10 con 0x7f7b480c7620 2023-11-06T02:09:13.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.353+0000 7f7b4e76c640 1 --2- 172.21.15.49:0/107682765 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b480c7620 0x7f7b480c3180 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f7b400b0040 tx=0x7f7b40009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:13.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.353+0000 7f7b3e7fc640 1 -- 172.21.15.49:0/107682765 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b400039a0 con 0x7f7b480c7620 2023-11-06T02:09:13.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.353+0000 7f7b3e7fc640 1 -- 172.21.15.49:0/107682765 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b40003b00 con 0x7f7b480c7620 2023-11-06T02:09:13.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.353+0000 7f7b501cf640 1 -- 172.21.15.49:0/107682765 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b480c7dc0 con 0x7f7b480c7620 2023-11-06T02:09:13.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.353+0000 7f7b3e7fc640 1 -- 172.21.15.49:0/107682765 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b40003e90 con 0x7f7b480c7620 2023-11-06T02:09:13.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.353+0000 7f7b501cf640 1 -- 172.21.15.49:0/107682765 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b480c8380 con 0x7f7b480c7620 2023-11-06T02:09:13.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.353+0000 7f7b501cf640 1 -- 172.21.15.49:0/107682765 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b48159ef0 con 0x7f7b480c7620 2023-11-06T02:09:13.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.357+0000 7f7b3e7fc640 1 -- 172.21.15.49:0/107682765 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f7b400ae890 con 0x7f7b480c7620 2023-11-06T02:09:13.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.357+0000 7f7b3e7fc640 1 --2- 172.21.15.49:0/107682765 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7b2c065a60 0x7f7b2c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:13.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.357+0000 7f7b3e7fc640 1 -- 172.21.15.49:0/107682765 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f7b4011f290 con 0x7f7b480c7620 2023-11-06T02:09:13.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.357+0000 7f7b4df6b640 1 --2- 172.21.15.49:0/107682765 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7b2c065a60 0x7f7b2c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:13.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.357+0000 7f7b4df6b640 1 --2- 172.21.15.49:0/107682765 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7b2c065a60 0x7f7b2c067f20 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7b44009a90 tx=0x7f7b44008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:13.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.361+0000 7f7b3e7fc640 1 -- 172.21.15.49:0/107682765 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f7b48159ef0 con 0x7f7b480c7620 2023-11-06T02:09:13.526 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.529+0000 7f7b501cf640 1 -- 172.21.15.49:0/107682765 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f7b48159ef0 con 0x7f7b480c7620 2023-11-06T02:09:13.526 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.529+0000 7f7b3e7fc640 1 -- 172.21.15.49:0/107682765 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7f7b48159ef0 con 0x7f7b480c7620 2023-11-06T02:09:13.527 INFO:teuthology.orchestra.run.smithi049.stderr:listed 0 entries 2023-11-06T02:09:13.529 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.529+0000 7f7b501cf640 1 -- 172.21.15.49:0/107682765 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7b2c065a60 msgr2=0x7f7b2c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:13.529 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.529+0000 7f7b501cf640 1 --2- 172.21.15.49:0/107682765 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7b2c065a60 0x7f7b2c067f20 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7b44009a90 tx=0x7f7b44008040 comp rx=0 tx=0).stop 2023-11-06T02:09:13.529 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.529+0000 7f7b501cf640 1 -- 172.21.15.49:0/107682765 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b480c7620 msgr2=0x7f7b480c3180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:13.529 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.529+0000 7f7b501cf640 1 --2- 172.21.15.49:0/107682765 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b480c7620 0x7f7b480c3180 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f7b400b0040 tx=0x7f7b40009750 comp rx=0 tx=0).stop 2023-11-06T02:09:13.529 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.529+0000 7f7b501cf640 1 -- 172.21.15.49:0/107682765 shutdown_connections 2023-11-06T02:09:13.530 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.529+0000 7f7b501cf640 1 --2- 172.21.15.49:0/107682765 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7b2c065a60 0x7f7b2c067f20 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:13.530 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.529+0000 7f7b501cf640 1 --2- 172.21.15.49:0/107682765 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b480c7620 0x7f7b480c3180 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:13.530 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.529+0000 7f7b501cf640 1 --2- 172.21.15.49:0/107682765 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7b4815c850 0x7f7b480c70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:13.530 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.529+0000 7f7b501cf640 1 --2- 172.21.15.49:0/107682765 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b4814c6b0 0x7f7b4815c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:13.530 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.529+0000 7f7b501cf640 1 -- 172.21.15.49:0/107682765 >> 172.21.15.49:0/107682765 conn(0x7f7b4806c0a0 msgr2=0x7f7b480c8c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:13.531 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.529+0000 7f7b501cf640 1 -- 172.21.15.49:0/107682765 shutdown_connections 2023-11-06T02:09:13.531 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.529+0000 7f7b501cf640 1 -- 172.21.15.49:0/107682765 wait complete. 2023-11-06T02:09:13.545 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi049.front.sepia.ceph.com... 2023-11-06T02:09:13.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:09:13.545 DEBUG:teuthology.orchestra.run.smithi049:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-06T02:09:13.564 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T02:09:13.565 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi049.front.sepia.ceph.com 2023-11-06T02:09:13.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:09:13.565 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-06T02:09:13.614 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-11-06T02:09:13.614 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T02:09:13.621 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-11-06T02:09:13.621 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T02:09:13.670 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T02:09:13.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.801+0000 7f824748b640 1 Processor -- start 2023-11-06T02:09:13.798 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.801+0000 7f824748b640 1 -- start start 2023-11-06T02:09:13.798 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.801+0000 7f824748b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f82400b0790 0x7f82400c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:13.799 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.801+0000 7f824748b640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82400cab80 con 0x7f824015ab50 2023-11-06T02:09:13.799 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.801+0000 7f824748b640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f824015af30 con 0x7f82400cb010 2023-11-06T02:09:13.799 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.801+0000 7f824748b640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82400cb5b0 con 0x7f82400b0790 2023-11-06T02:09:13.799 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.801+0000 7f8245227640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f824015ab50 0x7f82400b0080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:60854/0 (socket says 172.21.15.49:60854) 2023-11-06T02:09:13.799 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.801+0000 7f8245227640 1 -- 172.21.15.49:0/851210931 learned_addr learned my addr 172.21.15.49:0/851210931 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:09:13.801 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.801+0000 7f8244a26640 1 --2- 172.21.15.49:0/851210931 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f82400b0790 0x7f82400c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:13.801 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.801+0000 7f8237fff640 1 -- 172.21.15.49:0/851210931 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3637468058 0 0) 0x7f82400cab80 con 0x7f824015ab50 2023-11-06T02:09:13.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.801+0000 7f8237fff640 1 -- 172.21.15.49:0/851210931 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f822c003530 con 0x7f824015ab50 2023-11-06T02:09:13.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.801+0000 7f8237fff640 1 -- 172.21.15.49:0/851210931 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 650989430 0 0) 0x7f822c003530 con 0x7f824015ab50 2023-11-06T02:09:13.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.801+0000 7f8237fff640 1 -- 172.21.15.49:0/851210931 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f82400b0790 msgr2=0x7f82400c9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:13.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.801+0000 7f8237fff640 1 --2- 172.21.15.49:0/851210931 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f82400b0790 0x7f82400c9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:13.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.801+0000 7f8237fff640 1 -- 172.21.15.49:0/851210931 >> v1:172.21.15.150:6789/0 conn(0x7f82400cb010 legacy=0x7f82400c2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:13.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.801+0000 7f8237fff640 1 -- 172.21.15.49:0/851210931 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82400cb890 con 0x7f824015ab50 2023-11-06T02:09:13.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.801+0000 7f8237fff640 1 -- 172.21.15.49:0/851210931 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7f823c0016d0 con 0x7f824015ab50 2023-11-06T02:09:13.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.801+0000 7f8237fff640 1 -- 172.21.15.49:0/851210931 >> v1:172.21.15.49:6789/0 conn(0x7f824015ab50 legacy=0x7f82400b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:13.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.801+0000 7f8237fff640 1 --2- 172.21.15.49:0/851210931 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f822c0039b0 0x7f822c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:13.804 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.801+0000 7f8237fff640 1 --2- 172.21.15.49:0/851210931 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f82400b0790 0x7f822c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:13.804 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.801+0000 7f8237fff640 1 --2- 172.21.15.49:0/851210931 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f822c004d10 0x7f822c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:13.804 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.801+0000 7f8237fff640 1 -- 172.21.15.49:0/851210931 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82400cb5b0 con 0x7f822c0039b0 2023-11-06T02:09:13.804 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.801+0000 7f8237fff640 1 -- 172.21.15.49:0/851210931 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f822c00ba40 con 0x7f822c004d10 2023-11-06T02:09:13.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.801+0000 7f8237fff640 1 -- 172.21.15.49:0/851210931 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f822c00bc00 con 0x7f82400b0790 2023-11-06T02:09:13.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.801+0000 7f8245227640 1 --2- 172.21.15.49:0/851210931 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f822c0039b0 0x7f822c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:13.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.801+0000 7f8244a26640 1 --2- 172.21.15.49:0/851210931 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f82400b0790 0x7f822c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:13.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.801+0000 7f8245a28640 1 --2- 172.21.15.49:0/851210931 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f822c004d10 0x7f822c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:13.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.801+0000 7f8245227640 1 -- 172.21.15.49:0/851210931 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f82400b0790 msgr2=0x7f822c0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:13.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.801+0000 7f8245227640 1 --2- 172.21.15.49:0/851210931 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f82400b0790 0x7f822c0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:13.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.801+0000 7f8245227640 1 -- 172.21.15.49:0/851210931 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f822c004d10 msgr2=0x7f822c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:13.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.801+0000 7f8245227640 1 --2- 172.21.15.49:0/851210931 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f822c004d10 0x7f822c005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:13.807 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.801+0000 7f8245227640 1 -- 172.21.15.49:0/851210931 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f822c003530 con 0x7f822c0039b0 2023-11-06T02:09:13.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.805+0000 7f8245227640 1 --2- 172.21.15.49:0/851210931 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f822c0039b0 0x7f822c003d90 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f823c00c1c0 tx=0x7f823c01cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3171acc1767ee74 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:13.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.805+0000 7f8237fff640 1 -- 172.21.15.49:0/851210931 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f823c0a5af0 con 0x7f822c0039b0 2023-11-06T02:09:13.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.805+0000 7f8237fff640 1 -- 172.21.15.49:0/851210931 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f823c0a5c50 con 0x7f822c0039b0 2023-11-06T02:09:13.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.805+0000 7f8237fff640 1 -- 172.21.15.49:0/851210931 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f823c0058e0 con 0x7f822c0039b0 2023-11-06T02:09:13.812 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.805+0000 7f824748b640 1 -- 172.21.15.49:0/851210931 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f822c0039b0 msgr2=0x7f822c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:13.812 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.805+0000 7f824748b640 1 --2- 172.21.15.49:0/851210931 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f822c0039b0 0x7f822c003d90 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f823c00c1c0 tx=0x7f823c01cc10 comp rx=0 tx=0).stop 2023-11-06T02:09:13.812 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.805+0000 7f8245a28640 1 -- 172.21.15.49:0/851210931 reap_dead start 2023-11-06T02:09:13.812 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.805+0000 7f824748b640 1 -- 172.21.15.49:0/851210931 shutdown_connections 2023-11-06T02:09:13.812 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.805+0000 7f824748b640 1 -- 172.21.15.49:0/851210931 >> 172.21.15.49:0/851210931 conn(0x7f824006c0a0 msgr2=0x7f824006c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:13.813 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.805+0000 7f824748b640 1 -- 172.21.15.49:0/851210931 shutdown_connections 2023-11-06T02:09:13.813 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.805+0000 7f824748b640 1 -- 172.21.15.49:0/851210931 wait complete. 2023-11-06T02:09:13.813 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.805+0000 7f824748b640 1 Processor -- start 2023-11-06T02:09:13.813 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.805+0000 7f824748b640 1 -- start start 2023-11-06T02:09:13.813 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.805+0000 7f824748b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82401f8ac0 0x7f82401f5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:13.814 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.805+0000 7f824748b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f82400ac700 0x7f824015b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:13.814 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.805+0000 7f824748b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8240158cb0 0x7f8240159130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:13.814 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.805+0000 7f824748b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82400cc1d0 con 0x7f82401f8ac0 2023-11-06T02:09:13.814 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.805+0000 7f824748b640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8240159670 con 0x7f8240158cb0 2023-11-06T02:09:13.815 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.805+0000 7f824748b640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82401f8ea0 con 0x7f82400ac700 2023-11-06T02:09:13.815 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.805+0000 7f8245227640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82401f8ac0 0x7f82401f5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:13.815 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.805+0000 7f8245227640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82401f8ac0 0x7f82401f5b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48272/0 (socket says 172.21.15.49:48272) 2023-11-06T02:09:13.815 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.805+0000 7f8245227640 1 -- 172.21.15.49:0/2077276340 learned_addr learned my addr 172.21.15.49:0/2077276340 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:13.815 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.805+0000 7f8245a28640 1 --2- 172.21.15.49:0/2077276340 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8240158cb0 0x7f8240159130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:13.816 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.805+0000 7f8244a26640 1 --2- 172.21.15.49:0/2077276340 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f82400ac700 0x7f824015b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:13.816 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.805+0000 7f8245227640 1 -- 172.21.15.49:0/2077276340 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f82400ac700 msgr2=0x7f824015b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:13.816 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.805+0000 7f8245227640 1 --2- 172.21.15.49:0/2077276340 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f82400ac700 0x7f824015b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:13.816 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.805+0000 7f8245227640 1 -- 172.21.15.49:0/2077276340 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8240158cb0 msgr2=0x7f8240159130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:13.816 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.805+0000 7f8245227640 1 --2- 172.21.15.49:0/2077276340 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8240158cb0 0x7f8240159130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:13.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.805+0000 7f8245227640 1 -- 172.21.15.49:0/2077276340 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82400cc470 con 0x7f82401f8ac0 2023-11-06T02:09:13.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.805+0000 7f8245a28640 1 --2- 172.21.15.49:0/2077276340 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8240158cb0 0x7f8240159130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:09:13.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.809+0000 7f8245227640 1 --2- 172.21.15.49:0/2077276340 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82401f8ac0 0x7f82401f5b10 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f823c01d4b0 tx=0x7f822c005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:13.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.809+0000 7f8235ffb640 1 -- 172.21.15.49:0/2077276340 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f823c0b3070 con 0x7f82401f8ac0 2023-11-06T02:09:13.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.809+0000 7f8235ffb640 1 -- 172.21.15.49:0/2077276340 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f823c004e70 con 0x7f82401f8ac0 2023-11-06T02:09:13.818 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.809+0000 7f824748b640 1 -- 172.21.15.49:0/2077276340 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82400acc30 con 0x7f82401f8ac0 2023-11-06T02:09:13.818 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.809+0000 7f8235ffb640 1 -- 172.21.15.49:0/2077276340 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f823c0a5af0 con 0x7f82401f8ac0 2023-11-06T02:09:13.818 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.809+0000 7f824748b640 1 -- 172.21.15.49:0/2077276340 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82400ad1f0 con 0x7f82401f8ac0 2023-11-06T02:09:13.818 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.809+0000 7f824748b640 1 -- 172.21.15.49:0/2077276340 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82400b0de0 con 0x7f82401f8ac0 2023-11-06T02:09:13.818 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.809+0000 7f8235ffb640 1 -- 172.21.15.49:0/2077276340 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f823c001490 con 0x7f82401f8ac0 2023-11-06T02:09:13.819 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.809+0000 7f8235ffb640 1 --2- 172.21.15.49:0/2077276340 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8224065a60 0x7f8224067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:13.819 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.809+0000 7f8235ffb640 1 -- 172.21.15.49:0/2077276340 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f823c11cfb0 con 0x7f82401f8ac0 2023-11-06T02:09:13.819 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.809+0000 7f8244a26640 1 --2- 172.21.15.49:0/2077276340 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8224065a60 0x7f8224067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:13.819 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.809+0000 7f8244a26640 1 --2- 172.21.15.49:0/2077276340 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8224065a60 0x7f8224067f20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f82400cdf40 tx=0x7f8230008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:13.820 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.813+0000 7f8235ffb640 1 -- 172.21.15.49:0/2077276340 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f823c0ea860 con 0x7f82401f8ac0 2023-11-06T02:09:13.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.973+0000 7f824748b640 1 -- 172.21.15.49:0/2077276340 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f82400ad3e0 con 0x7f82401f8ac0 2023-11-06T02:09:13.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.973+0000 7f8235ffb640 1 -- 172.21.15.49:0/2077276340 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17906 (secure 0 0 0) 0x7f823c0ede80 con 0x7f82401f8ac0 2023-11-06T02:09:13.973 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:09:13.973 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":25,"fsid":"1a8b035c-6ba4-499a-b876-aea47fd9c0d4","created":"2023-11-06T02:08:06.440857+0000","modified":"2023-11-06T02:09:13.195118+0000","last_up_change":"2023-11-06T02:08:17.982372+0000","last_in_change":"2023-11-06T02:08:08.906631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T02:08:19.252222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T02:08:24.776211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-06T02:08:28.936178+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-06T02:08:29.563468+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ad55af8-0571-4e6a-8722-b5d429b7496d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6809","nonce":38116},{"type":"v1","addr":"172.21.15.49:6811","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6813","nonce":38116},{"type":"v1","addr":"172.21.15.49:6815","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6821","nonce":38116},{"type":"v1","addr":"172.21.15.49:6823","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6817","nonce":38116},{"type":"v1","addr":"172.21.15.49:6819","nonce":38116}]},"public_addr":"172.21.15.49:6811/38116","cluster_addr":"172.21.15.49:6815/38116","heartbeat_back_addr":"172.21.15.49:6823/38116","heartbeat_front_addr":"172.21.15.49:6819/38116","state":["exists","up"]},{"osd":1,"uuid":"c8b31f85-a757-43ed-9419-351acc46611b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":38113},{"type":"v1","addr":"172.21.15.49:6810","nonce":38113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":38113},{"type":"v1","addr":"172.21.15.49:6814","nonce":38113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":38113},{"type":"v1","addr":"172.21.15.49:6822","nonce":38113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":38113},{"type":"v1","addr":"172.21.15.49:6818","nonce":38113}]},"public_addr":"172.21.15.49:6810/38113","cluster_addr":"172.21.15.49:6814/38113","heartbeat_back_addr":"172.21.15.49:6822/38113","heartbeat_front_addr":"172.21.15.49:6818/38113","state":["exists","up"]},{"osd":2,"uuid":"555b3bf3-0a70-4b91-8efd-e4204b5fc26e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":38114},{"type":"v1","addr":"172.21.15.49:6801","nonce":38114}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":38114},{"type":"v1","addr":"172.21.15.49:6803","nonce":38114}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":38114},{"type":"v1","addr":"172.21.15.49:6807","nonce":38114}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":38114},{"type":"v1","addr":"172.21.15.49:6805","nonce":38114}]},"public_addr":"172.21.15.49:6801/38114","cluster_addr":"172.21.15.49:6803/38114","heartbeat_back_addr":"172.21.15.49:6807/38114","heartbeat_front_addr":"172.21.15.49:6805/38114","state":["exists","up"]},{"osd":3,"uuid":"8d4e5d77-03ef-4b9b-bfcb-5ae93a46795c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":38115},{"type":"v1","addr":"172.21.15.49:6825","nonce":38115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":38115},{"type":"v1","addr":"172.21.15.49:6827","nonce":38115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":38115},{"type":"v1","addr":"172.21.15.49:6831","nonce":38115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":38115},{"type":"v1","addr":"172.21.15.49:6829","nonce":38115}]},"public_addr":"172.21.15.49:6825/38115","cluster_addr":"172.21.15.49:6827/38115","heartbeat_back_addr":"172.21.15.49:6831/38115","heartbeat_front_addr":"172.21.15.49:6829/38115","state":["exists","up"]},{"osd":4,"uuid":"dba03ad4-68a3-4cf3-9218-ac611f02eacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":38417},{"type":"v1","addr":"172.21.15.150:6809","nonce":38417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":38417},{"type":"v1","addr":"172.21.15.150:6811","nonce":38417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":38417},{"type":"v1","addr":"172.21.15.150:6815","nonce":38417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":38417},{"type":"v1","addr":"172.21.15.150:6813","nonce":38417}]},"public_addr":"172.21.15.150:6809/38417","cluster_addr":"172.21.15.150:6811/38417","heartbeat_back_addr":"172.21.15.150:6815/38417","heartbeat_front_addr":"172.21.15.150:6813/38417","state":["exists","up"]},{"osd":5,"uuid":"0df9c303-c255-4640-8e91-01f1c3efe18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":38418},{"type":"v1","addr":"172.21.15.150:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":38418},{"type":"v1","addr":"172.21.15.150:6803","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":38418},{"type":"v1","addr":"172.21.15.150:6807","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":38418},{"type":"v1","addr":"172.21.15.150:6805","nonce":38418}]},"public_addr":"172.21.15.150:6801/38418","cluster_addr":"172.21.15.150:6803/38418","heartbeat_back_addr":"172.21.15.150:6807/38418","heartbeat_front_addr":"172.21.15.150:6805/38418","state":["exists","up"]},{"osd":6,"uuid":"e59fa189-4b70-4c06-9722-c02b90828bf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":38428},{"type":"v1","addr":"172.21.15.150:6825","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":38428},{"type":"v1","addr":"172.21.15.150:6827","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":38428},{"type":"v1","addr":"172.21.15.150:6831","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":38428},{"type":"v1","addr":"172.21.15.150:6829","nonce":38428}]},"public_addr":"172.21.15.150:6825/38428","cluster_addr":"172.21.15.150:6827/38428","heartbeat_back_addr":"172.21.15.150:6831/38428","heartbeat_front_addr":"172.21.15.150:6829/38428","state":["exists","up"]},{"osd":7,"uuid":"a64f57e1-ef98-46bd-b304-385759e95898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":38421},{"type":"v1","addr":"172.21.15.150:6817","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":38421},{"type":"v1","addr":"172.21.15.150:6819","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":38421},{"type":"v1","addr":"172.21.15.150:6823","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":38421},{"type":"v1","addr":"172.21.15.150:6821","nonce":38421}]},"public_addr":"172.21.15.150:6817/38421","cluster_addr":"172.21.15.150:6819/38421","heartbeat_back_addr":"172.21.15.150:6823/38421","heartbeat_front_addr":"172.21.15.150:6821/38421","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.438552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.413280+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.296466+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.424966+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.276953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.230984+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.410235+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.480084+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.14","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T02:09:13.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.977+0000 7f824748b640 1 -- 172.21.15.49:0/2077276340 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8224065a60 msgr2=0x7f8224067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:13.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.977+0000 7f824748b640 1 --2- 172.21.15.49:0/2077276340 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8224065a60 0x7f8224067f20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f82400cdf40 tx=0x7f8230008040 comp rx=0 tx=0).stop 2023-11-06T02:09:13.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.977+0000 7f824748b640 1 -- 172.21.15.49:0/2077276340 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82401f8ac0 msgr2=0x7f82401f5b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:13.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.977+0000 7f824748b640 1 --2- 172.21.15.49:0/2077276340 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82401f8ac0 0x7f82401f5b10 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f823c01d4b0 tx=0x7f822c005800 comp rx=0 tx=0).stop 2023-11-06T02:09:13.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.977+0000 7f824748b640 1 -- 172.21.15.49:0/2077276340 shutdown_connections 2023-11-06T02:09:13.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.977+0000 7f824748b640 1 --2- 172.21.15.49:0/2077276340 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8224065a60 0x7f8224067f20 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:13.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.977+0000 7f824748b640 1 --2- 172.21.15.49:0/2077276340 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8240158cb0 0x7f8240159130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:13.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.977+0000 7f824748b640 1 --2- 172.21.15.49:0/2077276340 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f82400ac700 0x7f824015b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:13.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.977+0000 7f824748b640 1 --2- 172.21.15.49:0/2077276340 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82401f8ac0 0x7f82401f5b10 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:13.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.977+0000 7f824748b640 1 -- 172.21.15.49:0/2077276340 >> 172.21.15.49:0/2077276340 conn(0x7f824006c0a0 msgr2=0x7f82400aec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:13.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.977+0000 7f824748b640 1 -- 172.21.15.49:0/2077276340 shutdown_connections 2023-11-06T02:09:13.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:13.977+0000 7f824748b640 1 -- 172.21.15.49:0/2077276340 wait complete. 2023-11-06T02:09:13.992 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T02:08:19.252222+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T02:08:24.776211+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T02:08:28.936178+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T02:08:29.563468+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T02:09:13.993 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T02:09:14.118 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.121+0000 7f58cbcfc640 1 Processor -- start 2023-11-06T02:09:14.118 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.121+0000 7f58cbcfc640 1 -- start start 2023-11-06T02:09:14.119 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.121+0000 7f58cbcfc640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f58c415cbb0 0x7f58c40d1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:14.120 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.121+0000 7f58cbcfc640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f58c40d3170 0x7f58c40b5d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:14.120 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.121+0000 7f58cbcfc640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58c415cf90 con 0x7f58c40d24e0 2023-11-06T02:09:14.120 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.121+0000 7f58cbcfc640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58c40d0b80 con 0x7f58c415cbb0 2023-11-06T02:09:14.120 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.121+0000 7f58cbcfc640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58c40d0e60 con 0x7f58c40d3170 2023-11-06T02:09:14.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.121+0000 7f58c9a98640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f58c415cbb0 0x7f58c40d1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:14.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.121+0000 7f58c9a98640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f58c415cbb0 0x7f58c40d1fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48282/0 (socket says 172.21.15.49:48282) 2023-11-06T02:09:14.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.121+0000 7f58c9a98640 1 -- 172.21.15.49:0/3603578932 learned_addr learned my addr 172.21.15.49:0/3603578932 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:14.122 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.121+0000 7f58ca299640 1 --2- 172.21.15.49:0/3603578932 >> v2:172.21.15.150:3300/0 conn(0x7f58c40d3170 0x7f58c40b5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:14.122 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.121+0000 7f58c9a98640 1 -- 172.21.15.49:0/3603578932 >> v2:172.21.15.150:3300/0 conn(0x7f58c40d3170 msgr2=0x7f58c40b5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:14.122 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.121+0000 7f58c9a98640 1 --2- 172.21.15.49:0/3603578932 >> v2:172.21.15.150:3300/0 conn(0x7f58c40d3170 0x7f58c40b5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:14.122 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.121+0000 7f58c9a98640 1 -- 172.21.15.49:0/3603578932 >> v1:172.21.15.150:6789/0 conn(0x7f58c40d24e0 legacy=0x7f58c40d2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:14.123 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.121+0000 7f58c9a98640 1 -- 172.21.15.49:0/3603578932 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58c40d1140 con 0x7f58c415cbb0 2023-11-06T02:09:14.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.125+0000 7f58c9a98640 1 --2- 172.21.15.49:0/3603578932 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f58c415cbb0 0x7f58c40d1fa0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f58c40cfef0 tx=0x7f58c001ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61ec8013d3e8acd3 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:14.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.125+0000 7f58c8a96640 1 -- 172.21.15.49:0/3603578932 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c00a89b0 con 0x7f58c415cbb0 2023-11-06T02:09:14.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.125+0000 7f58c8a96640 1 -- 172.21.15.49:0/3603578932 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58c00a8b10 con 0x7f58c415cbb0 2023-11-06T02:09:14.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.125+0000 7f58c8a96640 1 -- 172.21.15.49:0/3603578932 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c00a8ea0 con 0x7f58c415cbb0 2023-11-06T02:09:14.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.125+0000 7f58cbcfc640 1 -- 172.21.15.49:0/3603578932 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f58c415cbb0 msgr2=0x7f58c40d1fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:14.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.125+0000 7f58cbcfc640 1 --2- 172.21.15.49:0/3603578932 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f58c415cbb0 0x7f58c40d1fa0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f58c40cfef0 tx=0x7f58c001ecb0 comp rx=0 tx=0).stop 2023-11-06T02:09:14.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.125+0000 7f58cbcfc640 1 -- 172.21.15.49:0/3603578932 shutdown_connections 2023-11-06T02:09:14.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.125+0000 7f58cbcfc640 1 --2- 172.21.15.49:0/3603578932 >> v2:172.21.15.150:3300/0 conn(0x7f58c40d3170 0x7f58c40b5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:14.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.125+0000 7f58cbcfc640 1 --2- 172.21.15.49:0/3603578932 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f58c415cbb0 0x7f58c40d1fa0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:14.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.125+0000 7f58cbcfc640 1 -- 172.21.15.49:0/3603578932 >> 172.21.15.49:0/3603578932 conn(0x7f58c40718a0 msgr2=0x7f58c4071cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:14.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.125+0000 7f58cbcfc640 1 -- 172.21.15.49:0/3603578932 shutdown_connections 2023-11-06T02:09:14.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.125+0000 7f58cbcfc640 1 -- 172.21.15.49:0/3603578932 wait complete. 2023-11-06T02:09:14.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.125+0000 7f58cbcfc640 1 Processor -- start 2023-11-06T02:09:14.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.125+0000 7f58cbcfc640 1 -- start start 2023-11-06T02:09:14.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.125+0000 7f58cbcfc640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f58c40d24e0 0x7f58c4159380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:14.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.125+0000 7f58cbcfc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f58c40d3170 0x7f58c4156750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:14.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.125+0000 7f58cbcfc640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58c415cbb0 0x7f58c4148ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:14.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.125+0000 7f58cbcfc640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c415e6e0 con 0x7f58c40d3170 2023-11-06T02:09:14.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.125+0000 7f58cbcfc640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c4149400 con 0x7f58c415cbb0 2023-11-06T02:09:14.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.125+0000 7f58cbcfc640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58c415eab0 con 0x7f58c40d24e0 2023-11-06T02:09:14.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.125+0000 7f58c9297640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f58c40d3170 0x7f58c4156750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:14.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.125+0000 7f58c9297640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f58c40d3170 0x7f58c4156750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48294/0 (socket says 172.21.15.49:48294) 2023-11-06T02:09:14.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.125+0000 7f58c9297640 1 -- 172.21.15.49:0/1411831326 learned_addr learned my addr 172.21.15.49:0/1411831326 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:14.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.125+0000 7f58ca299640 1 --2- 172.21.15.49:0/1411831326 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58c415cbb0 0x7f58c4148ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:14.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.125+0000 7f58c9a98640 1 --2- 172.21.15.49:0/1411831326 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f58c40d24e0 0x7f58c4159380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:14.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.129+0000 7f58c9297640 1 -- 172.21.15.49:0/1411831326 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f58c40d24e0 msgr2=0x7f58c4159380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:14.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.129+0000 7f58c9297640 1 --2- 172.21.15.49:0/1411831326 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f58c40d24e0 0x7f58c4159380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:14.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.129+0000 7f58c9297640 1 -- 172.21.15.49:0/1411831326 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58c415cbb0 msgr2=0x7f58c4148ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:14.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.129+0000 7f58c9297640 1 --2- 172.21.15.49:0/1411831326 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58c415cbb0 0x7f58c4148ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:14.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.129+0000 7f58c9297640 1 -- 172.21.15.49:0/1411831326 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58c00094d0 con 0x7f58c40d3170 2023-11-06T02:09:14.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.129+0000 7f58c9297640 1 --2- 172.21.15.49:0/1411831326 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f58c40d3170 0x7f58c4156750 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f58b400c410 tx=0x7f58b400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:14.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.129+0000 7f58ba7fc640 1 -- 172.21.15.49:0/1411831326 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b4011790 con 0x7f58c40d3170 2023-11-06T02:09:14.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.129+0000 7f58ba7fc640 1 -- 172.21.15.49:0/1411831326 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58b40118f0 con 0x7f58c40d3170 2023-11-06T02:09:14.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.129+0000 7f58ba7fc640 1 -- 172.21.15.49:0/1411831326 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b4011ce0 con 0x7f58c40d3170 2023-11-06T02:09:14.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.129+0000 7f58cbcfc640 1 -- 172.21.15.49:0/1411831326 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58c415ed90 con 0x7f58c40d3170 2023-11-06T02:09:14.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.129+0000 7f58cbcfc640 1 -- 172.21.15.49:0/1411831326 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58c415f350 con 0x7f58c40d3170 2023-11-06T02:09:14.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.129+0000 7f58cbcfc640 1 -- 172.21.15.49:0/1411831326 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58c4074040 con 0x7f58c40d3170 2023-11-06T02:09:14.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.129+0000 7f58ba7fc640 1 -- 172.21.15.49:0/1411831326 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f58b401c460 con 0x7f58c40d3170 2023-11-06T02:09:14.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.129+0000 7f58ba7fc640 1 --2- 172.21.15.49:0/1411831326 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f58a8065a60 0x7f58a8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:14.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.129+0000 7f58ba7fc640 1 -- 172.21.15.49:0/1411831326 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f58b40830a0 con 0x7f58c40d3170 2023-11-06T02:09:14.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.129+0000 7f58c9a98640 1 --2- 172.21.15.49:0/1411831326 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f58a8065a60 0x7f58a8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:14.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.129+0000 7f58c9a98640 1 --2- 172.21.15.49:0/1411831326 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f58a8065a60 0x7f58a8067f20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f58c00b2000 tx=0x7f58c0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:14.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.133+0000 7f58ba7fc640 1 -- 172.21.15.49:0/1411831326 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f58b40508c0 con 0x7f58c40d3170 2023-11-06T02:09:14.286 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.289+0000 7f58cbcfc640 1 -- 172.21.15.49:0/1411831326 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f58c40b6f80 con 0x7f58c40d3170 2023-11-06T02:09:14.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.289+0000 7f58ba7fc640 1 -- 172.21.15.49:0/1411831326 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f58b4053ee0 con 0x7f58c40d3170 2023-11-06T02:09:14.288 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2023-11-06T02:09:14.289 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.293+0000 7f58cbcfc640 1 -- 172.21.15.49:0/1411831326 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f58a8065a60 msgr2=0x7f58a8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:14.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.293+0000 7f58cbcfc640 1 --2- 172.21.15.49:0/1411831326 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f58a8065a60 0x7f58a8067f20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f58c00b2000 tx=0x7f58c0008040 comp rx=0 tx=0).stop 2023-11-06T02:09:14.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.293+0000 7f58cbcfc640 1 -- 172.21.15.49:0/1411831326 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f58c40d3170 msgr2=0x7f58c4156750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:14.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.293+0000 7f58cbcfc640 1 --2- 172.21.15.49:0/1411831326 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f58c40d3170 0x7f58c4156750 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f58b400c410 tx=0x7f58b400c8d0 comp rx=0 tx=0).stop 2023-11-06T02:09:14.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.293+0000 7f58cbcfc640 1 -- 172.21.15.49:0/1411831326 shutdown_connections 2023-11-06T02:09:14.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.293+0000 7f58cbcfc640 1 --2- 172.21.15.49:0/1411831326 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f58a8065a60 0x7f58a8067f20 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:14.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.293+0000 7f58cbcfc640 1 --2- 172.21.15.49:0/1411831326 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f58c40d3170 0x7f58c4156750 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:14.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.293+0000 7f58cbcfc640 1 --2- 172.21.15.49:0/1411831326 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58c415cbb0 0x7f58c4148ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:14.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.293+0000 7f58cbcfc640 1 --2- 172.21.15.49:0/1411831326 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f58c40d24e0 0x7f58c4159380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:14.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.293+0000 7f58cbcfc640 1 -- 172.21.15.49:0/1411831326 >> 172.21.15.49:0/1411831326 conn(0x7f58c40718a0 msgr2=0x7f58c40c9ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:14.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.293+0000 7f58cbcfc640 1 -- 172.21.15.49:0/1411831326 shutdown_connections 2023-11-06T02:09:14.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.293+0000 7f58cbcfc640 1 -- 172.21.15.49:0/1411831326 wait complete. 2023-11-06T02:09:14.307 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T02:09:14.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.437+0000 7f9a7e185640 1 Processor -- start 2023-11-06T02:09:14.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.437+0000 7f9a7e185640 1 -- start start 2023-11-06T02:09:14.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.441+0000 7f9a7e185640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f9a780ccab0 0x7f9a780d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:14.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.441+0000 7f9a7e185640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a780cf9c0 con 0x7f9a780cf190 2023-11-06T02:09:14.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.441+0000 7f9a7e185640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a78159dc0 con 0x7f9a781599e0 2023-11-06T02:09:14.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.441+0000 7f9a7e185640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a780aeeb0 con 0x7f9a780ccab0 2023-11-06T02:09:14.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.441+0000 7f9a76ffd640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f9a780ccab0 0x7f9a780d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:14.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.441+0000 7f9a77fff640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f9a780cf190 0x7f9a780cb120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:60870/0 (socket says 172.21.15.49:60870) 2023-11-06T02:09:14.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.441+0000 7f9a77fff640 1 -- 172.21.15.49:0/624620437 learned_addr learned my addr 172.21.15.49:0/624620437 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:09:14.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.441+0000 7f9a76ffd640 1 -- 172.21.15.49:0/624620437 >> v1:172.21.15.150:6789/0 conn(0x7f9a781599e0 legacy=0x7f9a780cc3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:14.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.441+0000 7f9a76ffd640 1 -- 172.21.15.49:0/624620437 >> v1:172.21.15.49:6789/0 conn(0x7f9a780cf190 legacy=0x7f9a780cb120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:14.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.441+0000 7f9a76ffd640 1 -- 172.21.15.49:0/624620437 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a780cfc60 con 0x7f9a780ccab0 2023-11-06T02:09:14.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.441+0000 7f9a767fc640 1 -- 172.21.15.49:0/624620437 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3838716525 0 0) 0x7f9a780cf9c0 con 0x7f9a780cf190 2023-11-06T02:09:14.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.441+0000 7f9a76ffd640 1 --2- 172.21.15.49:0/624620437 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a780ccab0 0x7f9a780d0460 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f9a600097c0 tx=0x7f9a6001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=989a728f2546f40f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:14.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.441+0000 7f9a767fc640 1 -- 172.21.15.49:0/624620437 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a600ab600 con 0x7f9a780ccab0 2023-11-06T02:09:14.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.441+0000 7f9a767fc640 1 -- 172.21.15.49:0/624620437 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a600a7050 con 0x7f9a780ccab0 2023-11-06T02:09:14.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.441+0000 7f9a767fc640 1 -- 172.21.15.49:0/624620437 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a600ab980 con 0x7f9a780ccab0 2023-11-06T02:09:14.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.441+0000 7f9a7e185640 1 -- 172.21.15.49:0/624620437 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a780ccab0 msgr2=0x7f9a780d0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:14.443 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.441+0000 7f9a7e185640 1 --2- 172.21.15.49:0/624620437 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a780ccab0 0x7f9a780d0460 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f9a600097c0 tx=0x7f9a6001c3c0 comp rx=0 tx=0).stop 2023-11-06T02:09:14.443 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.441+0000 7f9a7e185640 1 -- 172.21.15.49:0/624620437 shutdown_connections 2023-11-06T02:09:14.443 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.441+0000 7f9a7e185640 1 --2- 172.21.15.49:0/624620437 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a780ccab0 0x7f9a780d0460 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:14.443 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.441+0000 7f9a7e185640 1 -- 172.21.15.49:0/624620437 >> 172.21.15.49:0/624620437 conn(0x7f9a7806d4b0 msgr2=0x7f9a7806d890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:14.443 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.441+0000 7f9a7e185640 1 -- 172.21.15.49:0/624620437 shutdown_connections 2023-11-06T02:09:14.444 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.441+0000 7f9a7e185640 1 -- 172.21.15.49:0/624620437 wait complete. 2023-11-06T02:09:14.444 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.445+0000 7f9a7e185640 1 Processor -- start 2023-11-06T02:09:14.444 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.445+0000 7f9a7e185640 1 -- start start 2023-11-06T02:09:14.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.445+0000 7f9a7e185640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9a780ccab0 0x7f9a7814eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:14.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.445+0000 7f9a7e185640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a780cf190 0x7f9a780c7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:14.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.445+0000 7f9a7e185640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a781599e0 0x7f9a78153590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:14.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.445+0000 7f9a7e185640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a780acd00 con 0x7f9a780cf190 2023-11-06T02:09:14.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.445+0000 7f9a7e185640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7814fc30 con 0x7f9a781599e0 2023-11-06T02:09:14.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.445+0000 7f9a7e185640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a780ac4a0 con 0x7f9a780ccab0 2023-11-06T02:09:14.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.445+0000 7f9a76ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a780cf190 0x7f9a780c7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:14.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.445+0000 7f9a76ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a780cf190 0x7f9a780c7a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48312/0 (socket says 172.21.15.49:48312) 2023-11-06T02:09:14.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.445+0000 7f9a76ffd640 1 -- 172.21.15.49:0/1607476458 learned_addr learned my addr 172.21.15.49:0/1607476458 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:14.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.445+0000 7f9a77fff640 1 --2- 172.21.15.49:0/1607476458 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a781599e0 0x7f9a78153590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:14.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.445+0000 7f9a777fe640 1 --2- 172.21.15.49:0/1607476458 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9a780ccab0 0x7f9a7814eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:14.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.445+0000 7f9a76ffd640 1 -- 172.21.15.49:0/1607476458 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9a780ccab0 msgr2=0x7f9a7814eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:14.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.445+0000 7f9a76ffd640 1 --2- 172.21.15.49:0/1607476458 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9a780ccab0 0x7f9a7814eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:14.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.445+0000 7f9a76ffd640 1 -- 172.21.15.49:0/1607476458 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a781599e0 msgr2=0x7f9a78153590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:14.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.445+0000 7f9a76ffd640 1 --2- 172.21.15.49:0/1607476458 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a781599e0 0x7f9a78153590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:14.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.445+0000 7f9a76ffd640 1 -- 172.21.15.49:0/1607476458 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a60009420 con 0x7f9a780cf190 2023-11-06T02:09:14.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.445+0000 7f9a77fff640 1 --2- 172.21.15.49:0/1607476458 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a781599e0 0x7f9a78153590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:09:14.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.445+0000 7f9a777fe640 1 --2- 172.21.15.49:0/1607476458 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9a780ccab0 0x7f9a7814eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:09:14.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.445+0000 7f9a76ffd640 1 --2- 172.21.15.49:0/1607476458 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a780cf190 0x7f9a780c7a60 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9a60018fd0 tx=0x7f9a60019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:14.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.445+0000 7f9a6ffff640 1 -- 172.21.15.49:0/1607476458 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a600b4840 con 0x7f9a780cf190 2023-11-06T02:09:14.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.445+0000 7f9a6ffff640 1 -- 172.21.15.49:0/1607476458 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a780cc040 con 0x7f9a780cf190 2023-11-06T02:09:14.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.445+0000 7f9a7e185640 1 -- 172.21.15.49:0/1607476458 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a780ac780 con 0x7f9a780cf190 2023-11-06T02:09:14.451 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.445+0000 7f9a6ffff640 1 -- 172.21.15.49:0/1607476458 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a600b4c00 con 0x7f9a780cf190 2023-11-06T02:09:14.451 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.445+0000 7f9a7e185640 1 -- 172.21.15.49:0/1607476458 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a780ad230 con 0x7f9a780cf190 2023-11-06T02:09:14.451 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.445+0000 7f9a7e185640 1 -- 172.21.15.49:0/1607476458 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a780caa90 con 0x7f9a780cf190 2023-11-06T02:09:14.451 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.445+0000 7f9a6ffff640 1 -- 172.21.15.49:0/1607476458 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f9a600b2040 con 0x7f9a780cf190 2023-11-06T02:09:14.451 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.449+0000 7f9a6ffff640 1 --2- 172.21.15.49:0/1607476458 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9a58065a90 0x7f9a58067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:14.452 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.449+0000 7f9a6ffff640 1 -- 172.21.15.49:0/1607476458 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f9a6011d500 con 0x7f9a780cf190 2023-11-06T02:09:14.452 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.449+0000 7f9a777fe640 1 --2- 172.21.15.49:0/1607476458 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9a58065a90 0x7f9a58067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:14.452 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.449+0000 7f9a777fe640 1 --2- 172.21.15.49:0/1607476458 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9a58065a90 0x7f9a58067f50 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9a68007fd0 tx=0x7f9a68009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:14.453 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.453+0000 7f9a6ffff640 1 -- 172.21.15.49:0/1607476458 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f9a600ead20 con 0x7f9a780cf190 2023-11-06T02:09:14.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.609+0000 7f9a7e185640 1 -- 172.21.15.49:0/1607476458 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f9a78067140 con 0x7f9a780cf190 2023-11-06T02:09:14.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.609+0000 7f9a6ffff640 1 -- 172.21.15.49:0/1607476458 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7f9a600ee340 con 0x7f9a780cf190 2023-11-06T02:09:14.607 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 8 2023-11-06T02:09:14.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.609+0000 7f9a7e185640 1 -- 172.21.15.49:0/1607476458 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9a58065a90 msgr2=0x7f9a58067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:14.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.609+0000 7f9a7e185640 1 --2- 172.21.15.49:0/1607476458 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9a58065a90 0x7f9a58067f50 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9a68007fd0 tx=0x7f9a68009040 comp rx=0 tx=0).stop 2023-11-06T02:09:14.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.609+0000 7f9a7e185640 1 -- 172.21.15.49:0/1607476458 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a780cf190 msgr2=0x7f9a780c7a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:14.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.609+0000 7f9a7e185640 1 --2- 172.21.15.49:0/1607476458 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a780cf190 0x7f9a780c7a60 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9a60018fd0 tx=0x7f9a60019160 comp rx=0 tx=0).stop 2023-11-06T02:09:14.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.609+0000 7f9a7e185640 1 -- 172.21.15.49:0/1607476458 shutdown_connections 2023-11-06T02:09:14.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.609+0000 7f9a7e185640 1 --2- 172.21.15.49:0/1607476458 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9a58065a90 0x7f9a58067f50 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:14.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.609+0000 7f9a7e185640 1 --2- 172.21.15.49:0/1607476458 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a780cf190 0x7f9a780c7a60 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:14.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.609+0000 7f9a7e185640 1 --2- 172.21.15.49:0/1607476458 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a781599e0 0x7f9a78153590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:14.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.609+0000 7f9a7e185640 1 --2- 172.21.15.49:0/1607476458 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9a780ccab0 0x7f9a7814eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:14.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.609+0000 7f9a7e185640 1 -- 172.21.15.49:0/1607476458 >> 172.21.15.49:0/1607476458 conn(0x7f9a7806d4b0 msgr2=0x7f9a780cc370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:14.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.609+0000 7f9a7e185640 1 -- 172.21.15.49:0/1607476458 shutdown_connections 2023-11-06T02:09:14.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.609+0000 7f9a7e185640 1 -- 172.21.15.49:0/1607476458 wait complete. 2023-11-06T02:09:14.626 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T02:09:14.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.761+0000 7fe7d2c0f640 1 Processor -- start 2023-11-06T02:09:14.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.761+0000 7fe7d2c0f640 1 -- start start 2023-11-06T02:09:14.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.761+0000 7fe7d2c0f640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fe7cc070870 0x7fe7cc0b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:14.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.761+0000 7fe7d2c0f640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7cc0ae360 con 0x7fe7cc15a880 2023-11-06T02:09:14.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.761+0000 7fe7d2c0f640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7cc0ae640 con 0x7fe7cc06d150 2023-11-06T02:09:14.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.761+0000 7fe7d2c0f640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7cc0ae920 con 0x7fe7cc070870 2023-11-06T02:09:14.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.761+0000 7fe7d09ab640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fe7cc15a880 0x7fe7cc070160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:60884/0 (socket says 172.21.15.49:60884) 2023-11-06T02:09:14.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.761+0000 7fe7d09ab640 1 -- 172.21.15.49:0/878911965 learned_addr learned my addr 172.21.15.49:0/878911965 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:09:14.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.761+0000 7fe7cbfff640 1 --2- 172.21.15.49:0/878911965 >> v2:172.21.15.150:3300/0 conn(0x7fe7cc070870 0x7fe7cc0b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:14.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.761+0000 7fe7cb7fe640 1 -- 172.21.15.49:0/878911965 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2183449421 0 0) 0x7fe7cc0ae360 con 0x7fe7cc15a880 2023-11-06T02:09:14.762 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.761+0000 7fe7cb7fe640 1 -- 172.21.15.49:0/878911965 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7b4003530 con 0x7fe7cc15a880 2023-11-06T02:09:14.762 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.761+0000 7fe7cb7fe640 1 -- 172.21.15.49:0/878911965 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 48135051 0 0) 0x7fe7b4003530 con 0x7fe7cc15a880 2023-11-06T02:09:14.762 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.761+0000 7fe7cb7fe640 1 -- 172.21.15.49:0/878911965 >> v2:172.21.15.150:3300/0 conn(0x7fe7cc070870 msgr2=0x7fe7cc0b15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:14.762 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.761+0000 7fe7cb7fe640 1 --2- 172.21.15.49:0/878911965 >> v2:172.21.15.150:3300/0 conn(0x7fe7cc070870 0x7fe7cc0b15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:14.763 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.761+0000 7fe7cb7fe640 1 -- 172.21.15.49:0/878911965 >> v1:172.21.15.150:6789/0 conn(0x7fe7cc06d150 legacy=0x7fe7cc06d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:14.763 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.761+0000 7fe7cb7fe640 1 -- 172.21.15.49:0/878911965 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7cc0aec00 con 0x7fe7cc15a880 2023-11-06T02:09:14.763 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.761+0000 7fe7cb7fe640 1 -- 172.21.15.49:0/878911965 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7fe7c0001680 con 0x7fe7cc15a880 2023-11-06T02:09:14.763 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.761+0000 7fe7cb7fe640 1 -- 172.21.15.49:0/878911965 >> v1:172.21.15.49:6789/0 conn(0x7fe7cc15a880 legacy=0x7fe7cc070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:14.763 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.761+0000 7fe7cb7fe640 1 --2- 172.21.15.49:0/878911965 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7b40039d0 0x7fe7b4003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:14.763 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.761+0000 7fe7cb7fe640 1 --2- 172.21.15.49:0/878911965 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7b40047d0 0x7fe7b4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:14.764 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.761+0000 7fe7cb7fe640 1 --2- 172.21.15.49:0/878911965 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe7b40051e0 0x7fe7b400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:14.764 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.761+0000 7fe7cb7fe640 1 -- 172.21.15.49:0/878911965 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7cc0ae920 con 0x7fe7b40047d0 2023-11-06T02:09:14.764 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.761+0000 7fe7cb7fe640 1 -- 172.21.15.49:0/878911965 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b400c130 con 0x7fe7b40039d0 2023-11-06T02:09:14.764 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.761+0000 7fe7cb7fe640 1 -- 172.21.15.49:0/878911965 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b400c2f0 con 0x7fe7b40051e0 2023-11-06T02:09:14.764 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.761+0000 7fe7cbfff640 1 --2- 172.21.15.49:0/878911965 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7b40047d0 0x7fe7b4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:14.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.761+0000 7fe7d09ab640 1 --2- 172.21.15.49:0/878911965 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7b40039d0 0x7fe7b4003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:14.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.761+0000 7fe7d11ac640 1 --2- 172.21.15.49:0/878911965 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe7b40051e0 0x7fe7b400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:14.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.761+0000 7fe7cbfff640 1 -- 172.21.15.49:0/878911965 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe7b40051e0 msgr2=0x7fe7b400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:14.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.761+0000 7fe7cbfff640 1 --2- 172.21.15.49:0/878911965 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe7b40051e0 0x7fe7b400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:14.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.761+0000 7fe7cbfff640 1 -- 172.21.15.49:0/878911965 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7b40039d0 msgr2=0x7fe7b4003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:14.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.761+0000 7fe7cbfff640 1 --2- 172.21.15.49:0/878911965 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7b40039d0 0x7fe7b4003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:14.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.761+0000 7fe7cbfff640 1 -- 172.21.15.49:0/878911965 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7b4003530 con 0x7fe7b40047d0 2023-11-06T02:09:14.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.761+0000 7fe7d11ac640 1 --2- 172.21.15.49:0/878911965 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe7b40051e0 0x7fe7b400b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:09:14.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.761+0000 7fe7d11ac640 1 -- 172.21.15.49:0/878911965 reap_dead start 2023-11-06T02:09:14.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.765+0000 7fe7cbfff640 1 --2- 172.21.15.49:0/878911965 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7b40047d0 0x7fe7b4004c50 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc009f90 tx=0x7fe7bc01cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e1bae5d1e3c0921 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:14.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.765+0000 7fe7cb7fe640 1 -- 172.21.15.49:0/878911965 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7bc0a5a10 con 0x7fe7b40047d0 2023-11-06T02:09:14.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.765+0000 7fe7cb7fe640 1 -- 172.21.15.49:0/878911965 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7bc00f040 con 0x7fe7b40047d0 2023-11-06T02:09:14.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.765+0000 7fe7cb7fe640 1 -- 172.21.15.49:0/878911965 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7bc0a5db0 con 0x7fe7b40047d0 2023-11-06T02:09:14.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.765+0000 7fe7d2c0f640 1 -- 172.21.15.49:0/878911965 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7b40047d0 msgr2=0x7fe7b4004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:14.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.765+0000 7fe7d2c0f640 1 --2- 172.21.15.49:0/878911965 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7b40047d0 0x7fe7b4004c50 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc009f90 tx=0x7fe7bc01cc20 comp rx=0 tx=0).stop 2023-11-06T02:09:14.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.765+0000 7fe7d2c0f640 1 -- 172.21.15.49:0/878911965 shutdown_connections 2023-11-06T02:09:14.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.765+0000 7fe7d2c0f640 1 --2- 172.21.15.49:0/878911965 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7b40047d0 0x7fe7b4004c50 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:14.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.765+0000 7fe7d2c0f640 1 -- 172.21.15.49:0/878911965 >> 172.21.15.49:0/878911965 conn(0x7fe7cc06a370 msgr2=0x7fe7cc0ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:14.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.765+0000 7fe7d2c0f640 1 -- 172.21.15.49:0/878911965 shutdown_connections 2023-11-06T02:09:14.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.765+0000 7fe7d2c0f640 1 -- 172.21.15.49:0/878911965 wait complete. 2023-11-06T02:09:14.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.765+0000 7fe7d2c0f640 1 Processor -- start 2023-11-06T02:09:14.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.765+0000 7fe7d2c0f640 1 -- start start 2023-11-06T02:09:14.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.765+0000 7fe7d2c0f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7b40047d0 0x7fe7cc152ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:14.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.765+0000 7fe7d2c0f640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe7cc154840 0x7fe7cc150c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:14.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.765+0000 7fe7d2c0f640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7cc13b740 0x7fe7cc13bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:14.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.765+0000 7fe7d2c0f640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7cc13c130 con 0x7fe7b40047d0 2023-11-06T02:09:14.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.765+0000 7fe7d2c0f640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7cc156110 con 0x7fe7cc13b740 2023-11-06T02:09:14.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.765+0000 7fe7d2c0f640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7cc155090 con 0x7fe7cc154840 2023-11-06T02:09:14.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.765+0000 7fe7d09ab640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7b40047d0 0x7fe7cc152ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:14.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.769+0000 7fe7d09ab640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7b40047d0 0x7fe7cc152ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48334/0 (socket says 172.21.15.49:48334) 2023-11-06T02:09:14.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.769+0000 7fe7d09ab640 1 -- 172.21.15.49:0/4115481164 learned_addr learned my addr 172.21.15.49:0/4115481164 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:14.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.769+0000 7fe7d11ac640 1 --2- 172.21.15.49:0/4115481164 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7cc13b740 0x7fe7cc13bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:14.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.769+0000 7fe7cbfff640 1 --2- 172.21.15.49:0/4115481164 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe7cc154840 0x7fe7cc150c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:14.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.769+0000 7fe7d09ab640 1 -- 172.21.15.49:0/4115481164 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe7cc154840 msgr2=0x7fe7cc150c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:14.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.769+0000 7fe7d09ab640 1 --2- 172.21.15.49:0/4115481164 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe7cc154840 0x7fe7cc150c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:14.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.769+0000 7fe7d09ab640 1 -- 172.21.15.49:0/4115481164 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7cc13b740 msgr2=0x7fe7cc13bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:14.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.769+0000 7fe7d09ab640 1 --2- 172.21.15.49:0/4115481164 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7cc13b740 0x7fe7cc13bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:14.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.769+0000 7fe7d09ab640 1 -- 172.21.15.49:0/4115481164 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7cc155370 con 0x7fe7b40047d0 2023-11-06T02:09:14.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.769+0000 7fe7d09ab640 1 --2- 172.21.15.49:0/4115481164 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7b40047d0 0x7fe7cc152ba0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe7c000e7f0 tx=0x7fe7c000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:14.776 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.769+0000 7fe7c97fa640 1 -- 172.21.15.49:0/4115481164 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7c000c8d0 con 0x7fe7b40047d0 2023-11-06T02:09:14.776 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.769+0000 7fe7c97fa640 1 -- 172.21.15.49:0/4115481164 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7c000ca30 con 0x7fe7b40047d0 2023-11-06T02:09:14.776 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.769+0000 7fe7d2c0f640 1 -- 172.21.15.49:0/4115481164 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7cc155650 con 0x7fe7b40047d0 2023-11-06T02:09:14.776 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.769+0000 7fe7c97fa640 1 -- 172.21.15.49:0/4115481164 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7c000cde0 con 0x7fe7b40047d0 2023-11-06T02:09:14.776 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.769+0000 7fe7d2c0f640 1 -- 172.21.15.49:0/4115481164 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7cc155c10 con 0x7fe7b40047d0 2023-11-06T02:09:14.777 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.769+0000 7fe7d2c0f640 1 -- 172.21.15.49:0/4115481164 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7cc074040 con 0x7fe7b40047d0 2023-11-06T02:09:14.777 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.769+0000 7fe7c97fa640 1 -- 172.21.15.49:0/4115481164 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fe7c001d460 con 0x7fe7b40047d0 2023-11-06T02:09:14.777 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.769+0000 7fe7c97fa640 1 --2- 172.21.15.49:0/4115481164 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe7b8065a60 0x7fe7b8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:14.777 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.769+0000 7fe7c97fa640 1 -- 172.21.15.49:0/4115481164 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fe7c0083410 con 0x7fe7b40047d0 2023-11-06T02:09:14.777 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.769+0000 7fe7cbfff640 1 --2- 172.21.15.49:0/4115481164 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe7b8065a60 0x7fe7b8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:14.778 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.769+0000 7fe7cbfff640 1 --2- 172.21.15.49:0/4115481164 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe7b8065a60 0x7fe7b8067f20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc0af000 tx=0x7fe7bc0a7040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:14.778 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.773+0000 7fe7c97fa640 1 -- 172.21.15.49:0/4115481164 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fe7c0050ba0 con 0x7fe7b40047d0 2023-11-06T02:09:14.925 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.925+0000 7fe7d2c0f640 1 -- 172.21.15.49:0/4115481164 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe7cc06c430 con 0x7fe7b40047d0 2023-11-06T02:09:14.925 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.929+0000 7fe7c97fa640 1 -- 172.21.15.49:0/4115481164 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe7c00541c0 con 0x7fe7b40047d0 2023-11-06T02:09:14.926 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2023-11-06T02:09:14.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.929+0000 7fe7d2c0f640 1 -- 172.21.15.49:0/4115481164 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe7b8065a60 msgr2=0x7fe7b8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:14.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.929+0000 7fe7d2c0f640 1 --2- 172.21.15.49:0/4115481164 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe7b8065a60 0x7fe7b8067f20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc0af000 tx=0x7fe7bc0a7040 comp rx=0 tx=0).stop 2023-11-06T02:09:14.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.929+0000 7fe7d2c0f640 1 -- 172.21.15.49:0/4115481164 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7b40047d0 msgr2=0x7fe7cc152ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:14.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.929+0000 7fe7d2c0f640 1 --2- 172.21.15.49:0/4115481164 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7b40047d0 0x7fe7cc152ba0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe7c000e7f0 tx=0x7fe7c000ecb0 comp rx=0 tx=0).stop 2023-11-06T02:09:14.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.929+0000 7fe7d2c0f640 1 -- 172.21.15.49:0/4115481164 shutdown_connections 2023-11-06T02:09:14.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.929+0000 7fe7d2c0f640 1 --2- 172.21.15.49:0/4115481164 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe7b8065a60 0x7fe7b8067f20 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:14.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.929+0000 7fe7d2c0f640 1 --2- 172.21.15.49:0/4115481164 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7cc13b740 0x7fe7cc13bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:14.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.929+0000 7fe7d2c0f640 1 --2- 172.21.15.49:0/4115481164 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe7cc154840 0x7fe7cc150c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:14.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.929+0000 7fe7d2c0f640 1 --2- 172.21.15.49:0/4115481164 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7b40047d0 0x7fe7cc152ba0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:14.930 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.929+0000 7fe7d2c0f640 1 -- 172.21.15.49:0/4115481164 >> 172.21.15.49:0/4115481164 conn(0x7fe7cc06a370 msgr2=0x7fe7cc150080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:14.930 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.929+0000 7fe7d2c0f640 1 -- 172.21.15.49:0/4115481164 shutdown_connections 2023-11-06T02:09:14.930 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:14.929+0000 7fe7d2c0f640 1 -- 172.21.15.49:0/4115481164 wait complete. 2023-11-06T02:09:14.943 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T02:09:15.069 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.073+0000 7f9ba9fc8640 1 Processor -- start 2023-11-06T02:09:15.069 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.073+0000 7f9ba9fc8640 1 -- start start 2023-11-06T02:09:15.071 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.073+0000 7f9ba9fc8640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f9ba40cf1f0 0x7f9ba415dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:15.071 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.073+0000 7f9ba9fc8640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f9ba40d1fa0 0x7f9ba415e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:15.071 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.073+0000 7f9ba9fc8640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ba40d2810 con 0x7f9ba40cda10 2023-11-06T02:09:15.071 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.073+0000 7f9ba9fc8640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ba40d1b10 con 0x7f9ba40cf1f0 2023-11-06T02:09:15.072 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.073+0000 7f9ba9fc8640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ba40d0f70 con 0x7f9ba40d1fa0 2023-11-06T02:09:15.072 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.073+0000 7f9ba37fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f9ba40cf1f0 0x7f9ba415dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:15.072 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.073+0000 7f9ba2ffd640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f9ba40cda10 0x7f9ba40cf5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:60892/0 (socket says 172.21.15.49:60892) 2023-11-06T02:09:15.072 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.073+0000 7f9ba37fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f9ba40cf1f0 0x7f9ba415dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48346/0 (socket says 172.21.15.49:48346) 2023-11-06T02:09:15.073 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.073+0000 7f9ba2ffd640 1 -- 172.21.15.49:0/1009148347 learned_addr learned my addr 172.21.15.49:0/1009148347 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:09:15.073 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.073+0000 7f9ba3fff640 1 --2- 172.21.15.49:0/1009148347 >> v2:172.21.15.150:3300/0 conn(0x7f9ba40d1fa0 0x7f9ba415e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:15.073 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.073+0000 7f9ba37fe640 1 -- 172.21.15.49:0/1009148347 >> v2:172.21.15.150:3300/0 conn(0x7f9ba40d1fa0 msgr2=0x7f9ba415e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:15.073 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.073+0000 7f9ba37fe640 1 --2- 172.21.15.49:0/1009148347 >> v2:172.21.15.150:3300/0 conn(0x7f9ba40d1fa0 0x7f9ba415e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:15.073 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.073+0000 7f9ba37fe640 1 -- 172.21.15.49:0/1009148347 >> v1:172.21.15.49:6789/0 conn(0x7f9ba40cda10 legacy=0x7f9ba40cf5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:15.074 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.073+0000 7f9ba27fc640 1 -- 172.21.15.49:0/1009148347 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3912463788 0 0) 0x7f9ba40d2810 con 0x7f9ba40cda10 2023-11-06T02:09:15.074 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.073+0000 7f9ba37fe640 1 -- 172.21.15.49:0/1009148347 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ba415e190 con 0x7f9ba40cf1f0 2023-11-06T02:09:15.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.073+0000 7f9ba37fe640 1 --2- 172.21.15.49:0/1009148347 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ba40cf1f0 0x7f9ba415dc50 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f9b940097c0 tx=0x7f9b9401c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8663b6235f08e482 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:15.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.073+0000 7f9ba27fc640 1 -- 172.21.15.49:0/1009148347 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b940ab640 con 0x7f9ba40cf1f0 2023-11-06T02:09:15.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.073+0000 7f9ba27fc640 1 -- 172.21.15.49:0/1009148347 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b940a7070 con 0x7f9ba40cf1f0 2023-11-06T02:09:15.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.073+0000 7f9ba27fc640 1 -- 172.21.15.49:0/1009148347 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b940ab9c0 con 0x7f9ba40cf1f0 2023-11-06T02:09:15.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.077+0000 7f9ba9fc8640 1 -- 172.21.15.49:0/1009148347 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ba40cf1f0 msgr2=0x7f9ba415dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:15.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.077+0000 7f9ba9fc8640 1 --2- 172.21.15.49:0/1009148347 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ba40cf1f0 0x7f9ba415dc50 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f9b940097c0 tx=0x7f9b9401c440 comp rx=0 tx=0).stop 2023-11-06T02:09:15.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.077+0000 7f9ba9fc8640 1 -- 172.21.15.49:0/1009148347 shutdown_connections 2023-11-06T02:09:15.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.077+0000 7f9ba9fc8640 1 --2- 172.21.15.49:0/1009148347 >> v2:172.21.15.150:3300/0 conn(0x7f9ba40d1fa0 0x7f9ba415e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:15.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.077+0000 7f9ba9fc8640 1 --2- 172.21.15.49:0/1009148347 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ba40cf1f0 0x7f9ba415dc50 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:15.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.077+0000 7f9ba9fc8640 1 -- 172.21.15.49:0/1009148347 >> 172.21.15.49:0/1009148347 conn(0x7f9ba4071aa0 msgr2=0x7f9ba406b110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:15.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.077+0000 7f9ba9fc8640 1 -- 172.21.15.49:0/1009148347 shutdown_connections 2023-11-06T02:09:15.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.077+0000 7f9ba9fc8640 1 -- 172.21.15.49:0/1009148347 wait complete. 2023-11-06T02:09:15.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.077+0000 7f9ba9fc8640 1 Processor -- start 2023-11-06T02:09:15.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.077+0000 7f9ba9fc8640 1 -- start start 2023-11-06T02:09:15.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.077+0000 7f9ba9fc8640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ba40cda10 0x7f9ba4152a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:15.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.077+0000 7f9ba9fc8640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9ba40cf1f0 0x7f9ba40c9b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:15.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.077+0000 7f9ba9fc8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ba40d1fa0 0x7f9ba4140500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:15.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.077+0000 7f9ba9fc8640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba4152620 con 0x7f9ba40d1fa0 2023-11-06T02:09:15.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.077+0000 7f9ba9fc8640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba4140a40 con 0x7f9ba40cda10 2023-11-06T02:09:15.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.077+0000 7f9ba9fc8640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba40ca260 con 0x7f9ba40cf1f0 2023-11-06T02:09:15.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.077+0000 7f9ba3fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ba40d1fa0 0x7f9ba4140500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:15.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.077+0000 7f9ba3fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ba40d1fa0 0x7f9ba4140500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48360/0 (socket says 172.21.15.49:48360) 2023-11-06T02:09:15.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.077+0000 7f9ba3fff640 1 -- 172.21.15.49:0/165511495 learned_addr learned my addr 172.21.15.49:0/165511495 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:15.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.077+0000 7f9ba3fff640 1 -- 172.21.15.49:0/165511495 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9ba40cf1f0 msgr2=0x7f9ba40c9b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:15.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.077+0000 7f9ba2ffd640 1 --2- 172.21.15.49:0/165511495 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9ba40cf1f0 0x7f9ba40c9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:15.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.077+0000 7f9ba37fe640 1 --2- 172.21.15.49:0/165511495 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ba40cda10 0x7f9ba4152a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:15.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.077+0000 7f9ba3fff640 1 --2- 172.21.15.49:0/165511495 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9ba40cf1f0 0x7f9ba40c9b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:15.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.077+0000 7f9ba3fff640 1 -- 172.21.15.49:0/165511495 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ba40cda10 msgr2=0x7f9ba4152a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:15.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.077+0000 7f9ba3fff640 1 --2- 172.21.15.49:0/165511495 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ba40cda10 0x7f9ba4152a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:15.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.077+0000 7f9ba3fff640 1 -- 172.21.15.49:0/165511495 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b94009420 con 0x7f9ba40d1fa0 2023-11-06T02:09:15.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.077+0000 7f9ba3fff640 1 --2- 172.21.15.49:0/165511495 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ba40d1fa0 0x7f9ba4140500 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c007e90 tx=0x7f9b9c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:15.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.077+0000 7f9b8bfff640 1 -- 172.21.15.49:0/165511495 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b9c014420 con 0x7f9ba40d1fa0 2023-11-06T02:09:15.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.077+0000 7f9b8bfff640 1 -- 172.21.15.49:0/165511495 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b9c014580 con 0x7f9ba40d1fa0 2023-11-06T02:09:15.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.077+0000 7f9ba9fc8640 1 -- 172.21.15.49:0/165511495 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ba4151c80 con 0x7f9ba40d1fa0 2023-11-06T02:09:15.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.077+0000 7f9b8bfff640 1 -- 172.21.15.49:0/165511495 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b9c014930 con 0x7f9ba40d1fa0 2023-11-06T02:09:15.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.077+0000 7f9ba9fc8640 1 -- 172.21.15.49:0/165511495 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ba4152240 con 0x7f9ba40d1fa0 2023-11-06T02:09:15.086 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.081+0000 7f9ba9fc8640 1 -- 172.21.15.49:0/165511495 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ba4152760 con 0x7f9ba40d1fa0 2023-11-06T02:09:15.086 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.081+0000 7f9b8bfff640 1 -- 172.21.15.49:0/165511495 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f9b9c014a90 con 0x7f9ba40d1fa0 2023-11-06T02:09:15.086 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.081+0000 7f9b8bfff640 1 --2- 172.21.15.49:0/165511495 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9b90065a60 0x7f9b90067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:15.086 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.081+0000 7f9b8bfff640 1 -- 172.21.15.49:0/165511495 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f9b9c083c40 con 0x7f9ba40d1fa0 2023-11-06T02:09:15.087 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.081+0000 7f9ba37fe640 1 --2- 172.21.15.49:0/165511495 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9b90065a60 0x7f9b90067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:15.087 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.081+0000 7f9ba37fe640 1 --2- 172.21.15.49:0/165511495 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9b90065a60 0x7f9b90067f20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9b940af000 tx=0x7f9b94008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:15.087 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.085+0000 7f9b8bfff640 1 -- 172.21.15.49:0/165511495 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f9ba4152760 con 0x7f9ba40d1fa0 2023-11-06T02:09:15.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.241+0000 7f9ba9fc8640 1 -- 172.21.15.49:0/165511495 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9ba4152760 con 0x7f9ba40d1fa0 2023-11-06T02:09:15.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.241+0000 7f9b8bfff640 1 -- 172.21.15.49:0/165511495 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9ba4152760 con 0x7f9ba40d1fa0 2023-11-06T02:09:15.238 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2023-11-06T02:09:15.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.241+0000 7f9ba9fc8640 1 -- 172.21.15.49:0/165511495 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9b90065a60 msgr2=0x7f9b90067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:15.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.241+0000 7f9ba9fc8640 1 --2- 172.21.15.49:0/165511495 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9b90065a60 0x7f9b90067f20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9b940af000 tx=0x7f9b94008040 comp rx=0 tx=0).stop 2023-11-06T02:09:15.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.241+0000 7f9ba9fc8640 1 -- 172.21.15.49:0/165511495 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ba40d1fa0 msgr2=0x7f9ba4140500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:15.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.241+0000 7f9ba9fc8640 1 --2- 172.21.15.49:0/165511495 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ba40d1fa0 0x7f9ba4140500 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c007e90 tx=0x7f9b9c00e4b0 comp rx=0 tx=0).stop 2023-11-06T02:09:15.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.241+0000 7f9ba9fc8640 1 -- 172.21.15.49:0/165511495 shutdown_connections 2023-11-06T02:09:15.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.241+0000 7f9ba9fc8640 1 --2- 172.21.15.49:0/165511495 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9b90065a60 0x7f9b90067f20 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:15.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.241+0000 7f9ba9fc8640 1 --2- 172.21.15.49:0/165511495 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ba40d1fa0 0x7f9ba4140500 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:15.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.241+0000 7f9ba9fc8640 1 --2- 172.21.15.49:0/165511495 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9ba40cf1f0 0x7f9ba40c9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:15.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.241+0000 7f9ba9fc8640 1 --2- 172.21.15.49:0/165511495 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ba40cda10 0x7f9ba4152a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:15.242 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.241+0000 7f9ba9fc8640 1 -- 172.21.15.49:0/165511495 >> 172.21.15.49:0/165511495 conn(0x7f9ba4071aa0 msgr2=0x7f9ba415c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:15.242 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.241+0000 7f9ba9fc8640 1 -- 172.21.15.49:0/165511495 shutdown_connections 2023-11-06T02:09:15.242 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.241+0000 7f9ba9fc8640 1 -- 172.21.15.49:0/165511495 wait complete. 2023-11-06T02:09:15.257 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-06T02:09:15.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.381+0000 7f24c5382640 1 Processor -- start 2023-11-06T02:09:15.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.381+0000 7f24c5382640 1 -- start start 2023-11-06T02:09:15.384 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.385+0000 7f24c5382640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f24c015aab0 0x7f24c00cb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:15.384 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.385+0000 7f24c5382640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f24c00cb970 0x7f24c00cd280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:15.384 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.385+0000 7f24c5382640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24c00ce3f0 con 0x7f24c00b2250 2023-11-06T02:09:15.384 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.385+0000 7f24c5382640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24c015ae90 con 0x7f24c00cb970 2023-11-06T02:09:15.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.385+0000 7f24c5382640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24c00cbdf0 con 0x7f24c015aab0 2023-11-06T02:09:15.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.385+0000 7f24bf7fe640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f24c00b2250 0x7f24c015e510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:60902/0 (socket says 172.21.15.49:60902) 2023-11-06T02:09:15.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.385+0000 7f24bf7fe640 1 -- 172.21.15.49:0/2615223506 learned_addr learned my addr 172.21.15.49:0/2615223506 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:09:15.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.385+0000 7f24b7fff640 1 --2- 172.21.15.49:0/2615223506 >> v2:172.21.15.150:3300/0 conn(0x7f24c00cb970 0x7f24c00cd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:15.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.385+0000 7f24be7fc640 1 -- 172.21.15.49:0/2615223506 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4260599486 0 0) 0x7f24c00ce3f0 con 0x7f24c00b2250 2023-11-06T02:09:15.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.385+0000 7f24be7fc640 1 -- 172.21.15.49:0/2615223506 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24ac003530 con 0x7f24c00b2250 2023-11-06T02:09:15.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.385+0000 7f24beffd640 1 --2- 172.21.15.49:0/2615223506 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f24c015aab0 0x7f24c00cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:15.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.385+0000 7f24be7fc640 1 -- 172.21.15.49:0/2615223506 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3546646730 0 0) 0x7f24ac003530 con 0x7f24c00b2250 2023-11-06T02:09:15.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.385+0000 7f24be7fc640 1 -- 172.21.15.49:0/2615223506 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f24c015aab0 msgr2=0x7f24c00cb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:15.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.385+0000 7f24be7fc640 1 --2- 172.21.15.49:0/2615223506 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f24c015aab0 0x7f24c00cb430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:15.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.385+0000 7f24be7fc640 1 -- 172.21.15.49:0/2615223506 >> v2:172.21.15.150:3300/0 conn(0x7f24c00cb970 msgr2=0x7f24c00cd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:15.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.385+0000 7f24be7fc640 1 --2- 172.21.15.49:0/2615223506 >> v2:172.21.15.150:3300/0 conn(0x7f24c00cb970 0x7f24c00cd280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:15.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.385+0000 7f24be7fc640 1 -- 172.21.15.49:0/2615223506 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24c00cc0d0 con 0x7f24c00b2250 2023-11-06T02:09:15.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.385+0000 7f24be7fc640 1 -- 172.21.15.49:0/2615223506 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7f24b8001a00 con 0x7f24c00b2250 2023-11-06T02:09:15.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.385+0000 7f24beffd640 1 --2- 172.21.15.49:0/2615223506 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f24c015aab0 0x7f24c00cb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:09:15.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.385+0000 7f24be7fc640 1 -- 172.21.15.49:0/2615223506 >> v1:172.21.15.49:6789/0 conn(0x7f24c00b2250 legacy=0x7f24c015e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:15.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.385+0000 7f24be7fc640 1 --2- 172.21.15.49:0/2615223506 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f24ac003980 0x7f24ac003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:15.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.385+0000 7f24be7fc640 1 --2- 172.21.15.49:0/2615223506 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f24c015aab0 0x7f24ac0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:15.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.385+0000 7f24be7fc640 1 --2- 172.21.15.49:0/2615223506 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f24ac004d10 0x7f24ac005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:15.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.385+0000 7f24be7fc640 1 -- 172.21.15.49:0/2615223506 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24c015ae90 con 0x7f24ac003980 2023-11-06T02:09:15.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.385+0000 7f24be7fc640 1 -- 172.21.15.49:0/2615223506 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24c00cbdf0 con 0x7f24ac004d10 2023-11-06T02:09:15.389 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.385+0000 7f24beffd640 1 --2- 172.21.15.49:0/2615223506 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f24ac003980 0x7f24ac003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:15.389 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.385+0000 7f24be7fc640 1 -- 172.21.15.49:0/2615223506 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24ac00b9e0 con 0x7f24c015aab0 2023-11-06T02:09:15.389 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.385+0000 7f24bf7fe640 1 --2- 172.21.15.49:0/2615223506 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f24ac004d10 0x7f24ac005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:15.389 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.385+0000 7f24b7fff640 1 --2- 172.21.15.49:0/2615223506 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f24c015aab0 0x7f24ac0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:15.389 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.385+0000 7f24beffd640 1 -- 172.21.15.49:0/2615223506 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f24c015aab0 msgr2=0x7f24ac0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:15.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.385+0000 7f24beffd640 1 --2- 172.21.15.49:0/2615223506 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f24c015aab0 0x7f24ac0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:15.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.385+0000 7f24beffd640 1 -- 172.21.15.49:0/2615223506 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f24ac004d10 msgr2=0x7f24ac005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:15.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.385+0000 7f24beffd640 1 --2- 172.21.15.49:0/2615223506 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f24ac004d10 0x7f24ac005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:15.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.385+0000 7f24beffd640 1 -- 172.21.15.49:0/2615223506 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24ac003530 con 0x7f24ac003980 2023-11-06T02:09:15.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.385+0000 7f24bf7fe640 1 --2- 172.21.15.49:0/2615223506 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f24ac004d10 0x7f24ac005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:09:15.391 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.385+0000 7f24b7fff640 1 --2- 172.21.15.49:0/2615223506 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f24c015aab0 0x7f24ac0047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:09:15.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.385+0000 7f24beffd640 1 --2- 172.21.15.49:0/2615223506 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f24ac003980 0x7f24ac003da0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f24b00098a0 tx=0x7f24b001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25155e82278a80ea server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:15.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.385+0000 7f24be7fc640 1 -- 172.21.15.49:0/2615223506 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b00a6040 con 0x7f24ac003980 2023-11-06T02:09:15.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.385+0000 7f24be7fc640 1 -- 172.21.15.49:0/2615223506 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24b0004230 con 0x7f24ac003980 2023-11-06T02:09:15.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.385+0000 7f24be7fc640 1 -- 172.21.15.49:0/2615223506 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b0003620 con 0x7f24ac003980 2023-11-06T02:09:15.395 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.389+0000 7f24c5382640 1 -- 172.21.15.49:0/2615223506 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f24ac003980 msgr2=0x7f24ac003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:15.395 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.389+0000 7f24c5382640 1 --2- 172.21.15.49:0/2615223506 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f24ac003980 0x7f24ac003da0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f24b00098a0 tx=0x7f24b001c560 comp rx=0 tx=0).stop 2023-11-06T02:09:15.395 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.389+0000 7f24bf7fe640 1 -- 172.21.15.49:0/2615223506 reap_dead start 2023-11-06T02:09:15.395 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.389+0000 7f24c5382640 1 -- 172.21.15.49:0/2615223506 shutdown_connections 2023-11-06T02:09:15.395 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.389+0000 7f24c5382640 1 -- 172.21.15.49:0/2615223506 >> 172.21.15.49:0/2615223506 conn(0x7f24c0089fd0 msgr2=0x7f24c0099c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:15.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.389+0000 7f24c5382640 1 -- 172.21.15.49:0/2615223506 shutdown_connections 2023-11-06T02:09:15.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.389+0000 7f24c5382640 1 -- 172.21.15.49:0/2615223506 wait complete. 2023-11-06T02:09:15.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.389+0000 7f24c5382640 1 Processor -- start 2023-11-06T02:09:15.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.389+0000 7f24c5382640 1 -- start start 2023-11-06T02:09:15.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.389+0000 7f24c5382640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f24ac003980 0x7f24c01f4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:15.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.389+0000 7f24c5382640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f24c015c060 0x7f24c00aea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:15.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.389+0000 7f24c5382640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f24c015c4e0 0x7f24c00aefd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:15.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.389+0000 7f24c5382640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24c006b320 con 0x7f24c015c4e0 2023-11-06T02:09:15.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.389+0000 7f24c5382640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24c015d070 con 0x7f24c015c060 2023-11-06T02:09:15.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.389+0000 7f24c5382640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24c015cab0 con 0x7f24ac003980 2023-11-06T02:09:15.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.389+0000 7f24bf7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f24c015c4e0 0x7f24c00aefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:15.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.389+0000 7f24bf7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f24c015c4e0 0x7f24c00aefd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48388/0 (socket says 172.21.15.49:48388) 2023-11-06T02:09:15.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.389+0000 7f24bf7fe640 1 -- 172.21.15.49:0/3379961362 learned_addr learned my addr 172.21.15.49:0/3379961362 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:15.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.389+0000 7f24beffd640 1 --2- 172.21.15.49:0/3379961362 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f24ac003980 0x7f24c01f4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:15.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.389+0000 7f24b7fff640 1 --2- 172.21.15.49:0/3379961362 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f24c015c060 0x7f24c00aea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:15.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.389+0000 7f24bf7fe640 1 -- 172.21.15.49:0/3379961362 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f24ac003980 msgr2=0x7f24c01f4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:15.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.389+0000 7f24bf7fe640 1 --2- 172.21.15.49:0/3379961362 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f24ac003980 0x7f24c01f4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:15.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.389+0000 7f24bf7fe640 1 -- 172.21.15.49:0/3379961362 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f24c015c060 msgr2=0x7f24c00aea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:15.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.389+0000 7f24bf7fe640 1 --2- 172.21.15.49:0/3379961362 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f24c015c060 0x7f24c00aea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:15.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.389+0000 7f24bf7fe640 1 -- 172.21.15.49:0/3379961362 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24c015cd90 con 0x7f24c015c4e0 2023-11-06T02:09:15.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.389+0000 7f24bf7fe640 1 --2- 172.21.15.49:0/3379961362 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f24c015c4e0 0x7f24c00aefd0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f24c00d18c0 tx=0x7f24b800df90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:15.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.389+0000 7f24b77fe640 1 -- 172.21.15.49:0/3379961362 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b80019a0 con 0x7f24c015c4e0 2023-11-06T02:09:15.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.389+0000 7f24b77fe640 1 -- 172.21.15.49:0/3379961362 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24b800bb10 con 0x7f24c015c4e0 2023-11-06T02:09:15.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.389+0000 7f24b77fe640 1 -- 172.21.15.49:0/3379961362 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b800bec0 con 0x7f24c015c4e0 2023-11-06T02:09:15.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.389+0000 7f24c5382640 1 -- 172.21.15.49:0/3379961362 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24c006afb0 con 0x7f24c015c4e0 2023-11-06T02:09:15.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.389+0000 7f24c5382640 1 -- 172.21.15.49:0/3379961362 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24c00c78f0 con 0x7f24c015c4e0 2023-11-06T02:09:15.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.393+0000 7f24c5382640 1 -- 172.21.15.49:0/3379961362 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24c00b6730 con 0x7f24c015c4e0 2023-11-06T02:09:15.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.393+0000 7f24b77fe640 1 -- 172.21.15.49:0/3379961362 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f24b8007110 con 0x7f24c015c4e0 2023-11-06T02:09:15.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.393+0000 7f24b77fe640 1 --2- 172.21.15.49:0/3379961362 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f24a0065a60 0x7f24a0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:15.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.393+0000 7f24b77fe640 1 -- 172.21.15.49:0/3379961362 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f24b80836e0 con 0x7f24c015c4e0 2023-11-06T02:09:15.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.393+0000 7f24beffd640 1 --2- 172.21.15.49:0/3379961362 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f24a0065a60 0x7f24a0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:15.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.393+0000 7f24beffd640 1 --2- 172.21.15.49:0/3379961362 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f24a0065a60 0x7f24a0067f20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f24b00ac000 tx=0x7f24b00046d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:15.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.397+0000 7f24b77fe640 1 -- 172.21.15.49:0/3379961362 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f24b8050f90 con 0x7f24c015c4e0 2023-11-06T02:09:15.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.565+0000 7f24c5382640 1 -- 172.21.15.49:0/3379961362 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f24c00b03e0 con 0x7f24c015c4e0 2023-11-06T02:09:15.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.565+0000 7f24b77fe640 1 -- 172.21.15.49:0/3379961362 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7f24b80545b0 con 0x7f24c015c4e0 2023-11-06T02:09:15.563 INFO:teuthology.orchestra.run.smithi049.stderr:listed 0 entries 2023-11-06T02:09:15.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.565+0000 7f24c5382640 1 -- 172.21.15.49:0/3379961362 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f24a0065a60 msgr2=0x7f24a0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:15.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.565+0000 7f24c5382640 1 --2- 172.21.15.49:0/3379961362 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f24a0065a60 0x7f24a0067f20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f24b00ac000 tx=0x7f24b00046d0 comp rx=0 tx=0).stop 2023-11-06T02:09:15.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.565+0000 7f24c5382640 1 -- 172.21.15.49:0/3379961362 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f24c015c4e0 msgr2=0x7f24c00aefd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:15.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.565+0000 7f24c5382640 1 --2- 172.21.15.49:0/3379961362 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f24c015c4e0 0x7f24c00aefd0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f24c00d18c0 tx=0x7f24b800df90 comp rx=0 tx=0).stop 2023-11-06T02:09:15.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.565+0000 7f24c5382640 1 -- 172.21.15.49:0/3379961362 shutdown_connections 2023-11-06T02:09:15.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.565+0000 7f24c5382640 1 --2- 172.21.15.49:0/3379961362 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f24a0065a60 0x7f24a0067f20 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:15.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.565+0000 7f24c5382640 1 --2- 172.21.15.49:0/3379961362 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f24c015c4e0 0x7f24c00aefd0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:15.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.565+0000 7f24c5382640 1 --2- 172.21.15.49:0/3379961362 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f24c015c060 0x7f24c00aea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:15.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.565+0000 7f24c5382640 1 --2- 172.21.15.49:0/3379961362 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f24ac003980 0x7f24c01f4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:15.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.565+0000 7f24c5382640 1 -- 172.21.15.49:0/3379961362 >> 172.21.15.49:0/3379961362 conn(0x7f24c0089fd0 msgr2=0x7f24c00cca00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:15.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.565+0000 7f24c5382640 1 -- 172.21.15.49:0/3379961362 shutdown_connections 2023-11-06T02:09:15.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.565+0000 7f24c5382640 1 -- 172.21.15.49:0/3379961362 wait complete. 2023-11-06T02:09:15.581 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-11-06T02:09:15.582 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T02:09:15.589 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T02:09:15.701 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.705+0000 7f0940e9f640 1 Processor -- start 2023-11-06T02:09:15.702 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.705+0000 7f0940e9f640 1 -- start start 2023-11-06T02:09:15.703 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.705+0000 7f0940e9f640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f093c1590f0 0x7f093c0c8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:15.703 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.705+0000 7f0940e9f640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f093c1594d0 con 0x7f093c0adae0 2023-11-06T02:09:15.703 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.705+0000 7f0940e9f640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f093c1597a0 con 0x7f093c0b0790 2023-11-06T02:09:15.704 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.705+0000 7f0940e9f640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f093c15a140 con 0x7f093c1590f0 2023-11-06T02:09:15.704 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.705+0000 7f093a59c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f093c1590f0 0x7f093c0c8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:15.704 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.705+0000 7f0939d9b640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f093c0adae0 0x7f093c0b0080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:60912/0 (socket says 172.21.15.49:60912) 2023-11-06T02:09:15.704 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.705+0000 7f0939d9b640 1 -- 172.21.15.49:0/2887434532 learned_addr learned my addr 172.21.15.49:0/2887434532 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:09:15.704 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.705+0000 7f093a59c640 1 -- 172.21.15.49:0/2887434532 >> v1:172.21.15.150:6789/0 conn(0x7f093c0b0790 legacy=0x7f093c070b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:15.705 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.705+0000 7f093a59c640 1 -- 172.21.15.49:0/2887434532 >> v1:172.21.15.49:6789/0 conn(0x7f093c0adae0 legacy=0x7f093c0b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:15.705 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.705+0000 7f093959a640 1 -- 172.21.15.49:0/2887434532 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 338802824 0 0) 0x7f093c1594d0 con 0x7f093c0adae0 2023-11-06T02:09:15.705 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.705+0000 7f093a59c640 1 -- 172.21.15.49:0/2887434532 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f093c159a80 con 0x7f093c1590f0 2023-11-06T02:09:15.706 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.709+0000 7f093a59c640 1 --2- 172.21.15.49:0/2887434532 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f093c1590f0 0x7f093c0c8c50 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f092c009de0 tx=0x7f092c01c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=131cb7af384c3c7d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:15.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.709+0000 7f093959a640 1 -- 172.21.15.49:0/2887434532 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f092c0a5780 con 0x7f093c1590f0 2023-11-06T02:09:15.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.709+0000 7f093959a640 1 -- 172.21.15.49:0/2887434532 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f092c0a58e0 con 0x7f093c1590f0 2023-11-06T02:09:15.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.709+0000 7f093959a640 1 -- 172.21.15.49:0/2887434532 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f092c0a5c70 con 0x7f093c1590f0 2023-11-06T02:09:15.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.709+0000 7f0940e9f640 1 -- 172.21.15.49:0/2887434532 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f093c1590f0 msgr2=0x7f093c0c8c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:15.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.709+0000 7f0940e9f640 1 --2- 172.21.15.49:0/2887434532 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f093c1590f0 0x7f093c0c8c50 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f092c009de0 tx=0x7f092c01c8c0 comp rx=0 tx=0).stop 2023-11-06T02:09:15.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.709+0000 7f0940e9f640 1 -- 172.21.15.49:0/2887434532 shutdown_connections 2023-11-06T02:09:15.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.709+0000 7f0940e9f640 1 --2- 172.21.15.49:0/2887434532 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f093c1590f0 0x7f093c0c8c50 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:15.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.709+0000 7f0940e9f640 1 -- 172.21.15.49:0/2887434532 >> 172.21.15.49:0/2887434532 conn(0x7f093c06c0a0 msgr2=0x7f093c06c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:15.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.709+0000 7f0940e9f640 1 -- 172.21.15.49:0/2887434532 shutdown_connections 2023-11-06T02:09:15.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.709+0000 7f0940e9f640 1 -- 172.21.15.49:0/2887434532 wait complete. 2023-11-06T02:09:15.711 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.709+0000 7f0940e9f640 1 Processor -- start 2023-11-06T02:09:15.711 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.709+0000 7f0940e9f640 1 -- start start 2023-11-06T02:09:15.711 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.709+0000 7f0940e9f640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f093c0adae0 0x7f093c141860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:15.712 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.709+0000 7f0940e9f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f093c0b0790 0x7f093c0b0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:15.712 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.709+0000 7f0940e9f640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f093c1590f0 0x7f093c0cc560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:15.712 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.709+0000 7f0940e9f640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f093c0b37b0 con 0x7f093c0b0790 2023-11-06T02:09:15.712 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.709+0000 7f0940e9f640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f093c0b3a90 con 0x7f093c0adae0 2023-11-06T02:09:15.712 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.709+0000 7f0940e9f640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f093c0b3d70 con 0x7f093c1590f0 2023-11-06T02:09:15.712 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.709+0000 7f0939d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f093c0b0790 0x7f093c0b0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:15.713 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.709+0000 7f0939d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f093c0b0790 0x7f093c0b0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48396/0 (socket says 172.21.15.49:48396) 2023-11-06T02:09:15.713 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.709+0000 7f0939d9b640 1 -- 172.21.15.49:0/3772281063 learned_addr learned my addr 172.21.15.49:0/3772281063 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:15.713 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.709+0000 7f093a59c640 1 --2- 172.21.15.49:0/3772281063 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f093c0adae0 0x7f093c141860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:15.713 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.709+0000 7f093ad9d640 1 --2- 172.21.15.49:0/3772281063 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f093c1590f0 0x7f093c0cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:15.713 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.709+0000 7f0939d9b640 1 -- 172.21.15.49:0/3772281063 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f093c1590f0 msgr2=0x7f093c0cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:15.714 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.709+0000 7f0939d9b640 1 --2- 172.21.15.49:0/3772281063 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f093c1590f0 0x7f093c0cc560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:15.714 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.709+0000 7f0939d9b640 1 -- 172.21.15.49:0/3772281063 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f093c0adae0 msgr2=0x7f093c141860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:15.714 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.709+0000 7f0939d9b640 1 --2- 172.21.15.49:0/3772281063 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f093c0adae0 0x7f093c141860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:15.714 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.709+0000 7f0939d9b640 1 -- 172.21.15.49:0/3772281063 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f092c009a70 con 0x7f093c0b0790 2023-11-06T02:09:15.715 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.709+0000 7f0939d9b640 1 --2- 172.21.15.49:0/3772281063 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f093c0b0790 0x7f093c0b0c00 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f092400c450 tx=0x7f092400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:15.715 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.709+0000 7f0932ffd640 1 -- 172.21.15.49:0/3772281063 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09240074e0 con 0x7f093c0b0790 2023-11-06T02:09:15.715 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.709+0000 7f0932ffd640 1 -- 172.21.15.49:0/3772281063 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0924007640 con 0x7f093c0b0790 2023-11-06T02:09:15.715 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.709+0000 7f0932ffd640 1 -- 172.21.15.49:0/3772281063 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09240049f0 con 0x7f093c0b0790 2023-11-06T02:09:15.715 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.709+0000 7f0940e9f640 1 -- 172.21.15.49:0/3772281063 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f093c0b4050 con 0x7f093c0b0790 2023-11-06T02:09:15.716 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.709+0000 7f0940e9f640 1 -- 172.21.15.49:0/3772281063 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f093c0b1140 con 0x7f093c0b0790 2023-11-06T02:09:15.716 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.713+0000 7f0940e9f640 1 -- 172.21.15.49:0/3772281063 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f093c074040 con 0x7f093c0b0790 2023-11-06T02:09:15.716 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.713+0000 7f0932ffd640 1 -- 172.21.15.49:0/3772281063 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f0924004b50 con 0x7f093c0b0790 2023-11-06T02:09:15.716 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.713+0000 7f0932ffd640 1 --2- 172.21.15.49:0/3772281063 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f091c065a60 0x7f091c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:15.716 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.713+0000 7f0932ffd640 1 -- 172.21.15.49:0/3772281063 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f09240825a0 con 0x7f093c0b0790 2023-11-06T02:09:15.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.713+0000 7f093a59c640 1 --2- 172.21.15.49:0/3772281063 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f091c065a60 0x7f091c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:15.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.713+0000 7f093a59c640 1 --2- 172.21.15.49:0/3772281063 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f091c065a60 0x7f091c067f20 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f092c0af000 tx=0x7f092c0a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:15.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.717+0000 7f0932ffd640 1 -- 172.21.15.49:0/3772281063 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f09240537c0 con 0x7f093c0b0790 2023-11-06T02:09:15.872 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.873+0000 7f0940e9f640 1 -- 172.21.15.49:0/3772281063 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f093c0b41e0 con 0x7f093c0b0790 2023-11-06T02:09:15.874 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.877+0000 7f0932ffd640 1 -- 172.21.15.49:0/3772281063 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17906 (secure 0 0 0) 0x7f0924018020 con 0x7f093c0b0790 2023-11-06T02:09:15.874 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:09:15.875 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":25,"fsid":"1a8b035c-6ba4-499a-b876-aea47fd9c0d4","created":"2023-11-06T02:08:06.440857+0000","modified":"2023-11-06T02:09:13.195118+0000","last_up_change":"2023-11-06T02:08:17.982372+0000","last_in_change":"2023-11-06T02:08:08.906631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T02:08:19.252222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T02:08:24.776211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-06T02:08:28.936178+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-06T02:08:29.563468+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ad55af8-0571-4e6a-8722-b5d429b7496d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6809","nonce":38116},{"type":"v1","addr":"172.21.15.49:6811","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6813","nonce":38116},{"type":"v1","addr":"172.21.15.49:6815","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6821","nonce":38116},{"type":"v1","addr":"172.21.15.49:6823","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6817","nonce":38116},{"type":"v1","addr":"172.21.15.49:6819","nonce":38116}]},"public_addr":"172.21.15.49:6811/38116","cluster_addr":"172.21.15.49:6815/38116","heartbeat_back_addr":"172.21.15.49:6823/38116","heartbeat_front_addr":"172.21.15.49:6819/38116","state":["exists","up"]},{"osd":1,"uuid":"c8b31f85-a757-43ed-9419-351acc46611b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":38113},{"type":"v1","addr":"172.21.15.49:6810","nonce":38113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":38113},{"type":"v1","addr":"172.21.15.49:6814","nonce":38113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":38113},{"type":"v1","addr":"172.21.15.49:6822","nonce":38113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":38113},{"type":"v1","addr":"172.21.15.49:6818","nonce":38113}]},"public_addr":"172.21.15.49:6810/38113","cluster_addr":"172.21.15.49:6814/38113","heartbeat_back_addr":"172.21.15.49:6822/38113","heartbeat_front_addr":"172.21.15.49:6818/38113","state":["exists","up"]},{"osd":2,"uuid":"555b3bf3-0a70-4b91-8efd-e4204b5fc26e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":38114},{"type":"v1","addr":"172.21.15.49:6801","nonce":38114}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":38114},{"type":"v1","addr":"172.21.15.49:6803","nonce":38114}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":38114},{"type":"v1","addr":"172.21.15.49:6807","nonce":38114}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":38114},{"type":"v1","addr":"172.21.15.49:6805","nonce":38114}]},"public_addr":"172.21.15.49:6801/38114","cluster_addr":"172.21.15.49:6803/38114","heartbeat_back_addr":"172.21.15.49:6807/38114","heartbeat_front_addr":"172.21.15.49:6805/38114","state":["exists","up"]},{"osd":3,"uuid":"8d4e5d77-03ef-4b9b-bfcb-5ae93a46795c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":38115},{"type":"v1","addr":"172.21.15.49:6825","nonce":38115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":38115},{"type":"v1","addr":"172.21.15.49:6827","nonce":38115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":38115},{"type":"v1","addr":"172.21.15.49:6831","nonce":38115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":38115},{"type":"v1","addr":"172.21.15.49:6829","nonce":38115}]},"public_addr":"172.21.15.49:6825/38115","cluster_addr":"172.21.15.49:6827/38115","heartbeat_back_addr":"172.21.15.49:6831/38115","heartbeat_front_addr":"172.21.15.49:6829/38115","state":["exists","up"]},{"osd":4,"uuid":"dba03ad4-68a3-4cf3-9218-ac611f02eacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":38417},{"type":"v1","addr":"172.21.15.150:6809","nonce":38417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":38417},{"type":"v1","addr":"172.21.15.150:6811","nonce":38417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":38417},{"type":"v1","addr":"172.21.15.150:6815","nonce":38417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":38417},{"type":"v1","addr":"172.21.15.150:6813","nonce":38417}]},"public_addr":"172.21.15.150:6809/38417","cluster_addr":"172.21.15.150:6811/38417","heartbeat_back_addr":"172.21.15.150:6815/38417","heartbeat_front_addr":"172.21.15.150:6813/38417","state":["exists","up"]},{"osd":5,"uuid":"0df9c303-c255-4640-8e91-01f1c3efe18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":38418},{"type":"v1","addr":"172.21.15.150:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":38418},{"type":"v1","addr":"172.21.15.150:6803","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":38418},{"type":"v1","addr":"172.21.15.150:6807","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":38418},{"type":"v1","addr":"172.21.15.150:6805","nonce":38418}]},"public_addr":"172.21.15.150:6801/38418","cluster_addr":"172.21.15.150:6803/38418","heartbeat_back_addr":"172.21.15.150:6807/38418","heartbeat_front_addr":"172.21.15.150:6805/38418","state":["exists","up"]},{"osd":6,"uuid":"e59fa189-4b70-4c06-9722-c02b90828bf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":38428},{"type":"v1","addr":"172.21.15.150:6825","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":38428},{"type":"v1","addr":"172.21.15.150:6827","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":38428},{"type":"v1","addr":"172.21.15.150:6831","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":38428},{"type":"v1","addr":"172.21.15.150:6829","nonce":38428}]},"public_addr":"172.21.15.150:6825/38428","cluster_addr":"172.21.15.150:6827/38428","heartbeat_back_addr":"172.21.15.150:6831/38428","heartbeat_front_addr":"172.21.15.150:6829/38428","state":["exists","up"]},{"osd":7,"uuid":"a64f57e1-ef98-46bd-b304-385759e95898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":38421},{"type":"v1","addr":"172.21.15.150:6817","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":38421},{"type":"v1","addr":"172.21.15.150:6819","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":38421},{"type":"v1","addr":"172.21.15.150:6823","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":38421},{"type":"v1","addr":"172.21.15.150:6821","nonce":38421}]},"public_addr":"172.21.15.150:6817/38421","cluster_addr":"172.21.15.150:6819/38421","heartbeat_back_addr":"172.21.15.150:6823/38421","heartbeat_front_addr":"172.21.15.150:6821/38421","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.438552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.413280+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.296466+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.424966+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.276953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.230984+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.410235+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.480084+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.14","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T02:09:15.876 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.877+0000 7f0940e9f640 1 -- 172.21.15.49:0/3772281063 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f091c065a60 msgr2=0x7f091c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:15.876 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.877+0000 7f0940e9f640 1 --2- 172.21.15.49:0/3772281063 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f091c065a60 0x7f091c067f20 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f092c0af000 tx=0x7f092c0a7070 comp rx=0 tx=0).stop 2023-11-06T02:09:15.876 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.877+0000 7f0940e9f640 1 -- 172.21.15.49:0/3772281063 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f093c0b0790 msgr2=0x7f093c0b0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:15.877 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.877+0000 7f0940e9f640 1 --2- 172.21.15.49:0/3772281063 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f093c0b0790 0x7f093c0b0c00 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f092400c450 tx=0x7f092400c910 comp rx=0 tx=0).stop 2023-11-06T02:09:15.877 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.877+0000 7f0940e9f640 1 -- 172.21.15.49:0/3772281063 shutdown_connections 2023-11-06T02:09:15.877 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.877+0000 7f0940e9f640 1 --2- 172.21.15.49:0/3772281063 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f091c065a60 0x7f091c067f20 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:15.877 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.877+0000 7f0940e9f640 1 --2- 172.21.15.49:0/3772281063 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f093c0b0790 0x7f093c0b0c00 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:15.878 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.877+0000 7f0940e9f640 1 --2- 172.21.15.49:0/3772281063 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f093c1590f0 0x7f093c0cc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:15.878 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.877+0000 7f0940e9f640 1 --2- 172.21.15.49:0/3772281063 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f093c0adae0 0x7f093c141860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:15.878 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.877+0000 7f0940e9f640 1 -- 172.21.15.49:0/3772281063 >> 172.21.15.49:0/3772281063 conn(0x7f093c06c0a0 msgr2=0x7f093c06c510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:15.878 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.877+0000 7f0940e9f640 1 -- 172.21.15.49:0/3772281063 shutdown_connections 2023-11-06T02:09:15.878 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:15.877+0000 7f0940e9f640 1 -- 172.21.15.49:0/3772281063 wait complete. 2023-11-06T02:09:15.893 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T02:08:19.252222+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T02:08:24.776211+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T02:08:28.936178+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T02:08:29.563468+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T02:09:15.894 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T02:09:16.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.009+0000 7f124246a640 1 Processor -- start 2023-11-06T02:09:16.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.009+0000 7f124246a640 1 -- start start 2023-11-06T02:09:16.009 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.009+0000 7f124246a640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f123c070870 0x7f123c0b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:16.009 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.009+0000 7f124246a640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f123c06d150 0x7f123c06d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:16.009 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.009+0000 7f124246a640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f123c06db40 con 0x7f123c15a880 2023-11-06T02:09:16.009 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.009+0000 7f124246a640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f123c0ae360 con 0x7f123c070870 2023-11-06T02:09:16.010 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.009+0000 7f124246a640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f123c0ae640 con 0x7f123c06d150 2023-11-06T02:09:16.010 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.009+0000 7f123b7fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f123c070870 0x7f123c0b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:16.010 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.009+0000 7f123b7fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f123c070870 0x7f123c0b15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48410/0 (socket says 172.21.15.49:48410) 2023-11-06T02:09:16.010 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.009+0000 7f123b7fe640 1 -- 172.21.15.49:0/1484696097 learned_addr learned my addr 172.21.15.49:0/1484696097 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:16.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.009+0000 7f1240a07640 1 --2- 172.21.15.49:0/1484696097 >> v2:172.21.15.150:3300/0 conn(0x7f123c06d150 0x7f123c06d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:16.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.009+0000 7f123b7fe640 1 -- 172.21.15.49:0/1484696097 >> v2:172.21.15.150:3300/0 conn(0x7f123c06d150 msgr2=0x7f123c06d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:16.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.009+0000 7f123b7fe640 1 --2- 172.21.15.49:0/1484696097 >> v2:172.21.15.150:3300/0 conn(0x7f123c06d150 0x7f123c06d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:16.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.009+0000 7f123b7fe640 1 -- 172.21.15.49:0/1484696097 >> v1:172.21.15.150:6789/0 conn(0x7f123c15a880 legacy=0x7f123c070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:16.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.009+0000 7f123b7fe640 1 -- 172.21.15.49:0/1484696097 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f123c0ae920 con 0x7f123c070870 2023-11-06T02:09:16.013 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.013+0000 7f123b7fe640 1 --2- 172.21.15.49:0/1484696097 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f123c070870 0x7f123c0b15c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f123c0b02a0 tx=0x7f123001c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=637ac32f135f26df server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:16.013 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.013+0000 7f123affd640 1 -- 172.21.15.49:0/1484696097 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12300ab600 con 0x7f123c070870 2023-11-06T02:09:16.013 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.013+0000 7f123affd640 1 -- 172.21.15.49:0/1484696097 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12300a7050 con 0x7f123c070870 2023-11-06T02:09:16.013 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.013+0000 7f123affd640 1 -- 172.21.15.49:0/1484696097 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12300ab980 con 0x7f123c070870 2023-11-06T02:09:16.014 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.013+0000 7f124246a640 1 -- 172.21.15.49:0/1484696097 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f123c070870 msgr2=0x7f123c0b15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:16.014 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.013+0000 7f124246a640 1 --2- 172.21.15.49:0/1484696097 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f123c070870 0x7f123c0b15c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f123c0b02a0 tx=0x7f123001c430 comp rx=0 tx=0).stop 2023-11-06T02:09:16.014 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.013+0000 7f124246a640 1 -- 172.21.15.49:0/1484696097 shutdown_connections 2023-11-06T02:09:16.014 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.013+0000 7f124246a640 1 --2- 172.21.15.49:0/1484696097 >> v2:172.21.15.150:3300/0 conn(0x7f123c06d150 0x7f123c06d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:16.014 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.013+0000 7f124246a640 1 --2- 172.21.15.49:0/1484696097 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f123c070870 0x7f123c0b15c0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:16.015 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.013+0000 7f124246a640 1 -- 172.21.15.49:0/1484696097 >> 172.21.15.49:0/1484696097 conn(0x7f123c06a370 msgr2=0x7f123c0ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:16.015 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.013+0000 7f124246a640 1 -- 172.21.15.49:0/1484696097 shutdown_connections 2023-11-06T02:09:16.015 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.013+0000 7f124246a640 1 -- 172.21.15.49:0/1484696097 wait complete. 2023-11-06T02:09:16.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.017+0000 7f124246a640 1 Processor -- start 2023-11-06T02:09:16.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.017+0000 7f124246a640 1 -- start start 2023-11-06T02:09:16.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.017+0000 7f124246a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f123c06d150 0x7f123c0cce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:16.019 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.017+0000 7f124246a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f123c070870 0x7f123c14fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:16.019 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.017+0000 7f124246a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f123c15a880 0x7f123c150410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:16.019 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.017+0000 7f124246a640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f123c0cca40 con 0x7f123c070870 2023-11-06T02:09:16.019 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.017+0000 7f124246a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f123c0cd3a0 con 0x7f123c06d150 2023-11-06T02:09:16.019 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.017+0000 7f124246a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f123c0cc0e0 con 0x7f123c15a880 2023-11-06T02:09:16.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.017+0000 7f123b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f123c070870 0x7f123c14fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:16.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.017+0000 7f123b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f123c070870 0x7f123c14fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48426/0 (socket says 172.21.15.49:48426) 2023-11-06T02:09:16.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.017+0000 7f123b7fe640 1 -- 172.21.15.49:0/571121909 learned_addr learned my addr 172.21.15.49:0/571121909 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:16.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.017+0000 7f123bfff640 1 --2- 172.21.15.49:0/571121909 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f123c06d150 0x7f123c0cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:16.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.017+0000 7f1240a07640 1 --2- 172.21.15.49:0/571121909 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f123c15a880 0x7f123c150410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:16.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.017+0000 7f123b7fe640 1 -- 172.21.15.49:0/571121909 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f123c15a880 msgr2=0x7f123c150410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:16.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.017+0000 7f123b7fe640 1 --2- 172.21.15.49:0/571121909 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f123c15a880 0x7f123c150410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:16.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.017+0000 7f123b7fe640 1 -- 172.21.15.49:0/571121909 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f123c06d150 msgr2=0x7f123c0cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:16.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.017+0000 7f123b7fe640 1 --2- 172.21.15.49:0/571121909 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f123c06d150 0x7f123c0cce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:16.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.017+0000 7f123b7fe640 1 -- 172.21.15.49:0/571121909 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12300094d0 con 0x7f123c070870 2023-11-06T02:09:16.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.017+0000 7f123b7fe640 1 --2- 172.21.15.49:0/571121909 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f123c070870 0x7f123c14fed0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f123001aae0 tx=0x7f12300191d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:16.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.017+0000 7f1238ff9640 1 -- 172.21.15.49:0/571121909 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12300b6640 con 0x7f123c070870 2023-11-06T02:09:16.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.017+0000 7f1238ff9640 1 -- 172.21.15.49:0/571121909 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12300b67a0 con 0x7f123c070870 2023-11-06T02:09:16.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.017+0000 7f124246a640 1 -- 172.21.15.49:0/571121909 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f123c0cc380 con 0x7f123c070870 2023-11-06T02:09:16.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.017+0000 7f1238ff9640 1 -- 172.21.15.49:0/571121909 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12300b6b30 con 0x7f123c070870 2023-11-06T02:09:16.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.017+0000 7f124246a640 1 -- 172.21.15.49:0/571121909 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f123c15c1b0 con 0x7f123c070870 2023-11-06T02:09:16.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.017+0000 7f124246a640 1 -- 172.21.15.49:0/571121909 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f123c0ccb80 con 0x7f123c070870 2023-11-06T02:09:16.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.017+0000 7f1238ff9640 1 -- 172.21.15.49:0/571121909 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f12300b2040 con 0x7f123c070870 2023-11-06T02:09:16.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.021+0000 7f1238ff9640 1 --2- 172.21.15.49:0/571121909 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1228065a60 0x7f1228067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:16.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.021+0000 7f1238ff9640 1 -- 172.21.15.49:0/571121909 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f123011cff0 con 0x7f123c070870 2023-11-06T02:09:16.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.021+0000 7f123bfff640 1 --2- 172.21.15.49:0/571121909 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1228065a60 0x7f1228067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:16.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.021+0000 7f123bfff640 1 --2- 172.21.15.49:0/571121909 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1228065a60 0x7f1228067f20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f123c14d330 tx=0x7f122c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:16.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.025+0000 7f1238ff9640 1 -- 172.21.15.49:0/571121909 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f123c0ccb80 con 0x7f123c070870 2023-11-06T02:09:16.173 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.177+0000 7f124246a640 1 -- 172.21.15.49:0/571121909 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f123c0ccb80 con 0x7f123c070870 2023-11-06T02:09:16.174 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.177+0000 7f1238ff9640 1 -- 172.21.15.49:0/571121909 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f123c0ccb80 con 0x7f123c070870 2023-11-06T02:09:16.174 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2023-11-06T02:09:16.176 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.177+0000 7f124246a640 1 -- 172.21.15.49:0/571121909 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1228065a60 msgr2=0x7f1228067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:16.177 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.177+0000 7f124246a640 1 --2- 172.21.15.49:0/571121909 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1228065a60 0x7f1228067f20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f123c14d330 tx=0x7f122c009040 comp rx=0 tx=0).stop 2023-11-06T02:09:16.177 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.177+0000 7f124246a640 1 -- 172.21.15.49:0/571121909 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f123c070870 msgr2=0x7f123c14fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:16.177 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.177+0000 7f124246a640 1 --2- 172.21.15.49:0/571121909 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f123c070870 0x7f123c14fed0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f123001aae0 tx=0x7f12300191d0 comp rx=0 tx=0).stop 2023-11-06T02:09:16.177 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.177+0000 7f124246a640 1 -- 172.21.15.49:0/571121909 shutdown_connections 2023-11-06T02:09:16.177 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.177+0000 7f124246a640 1 --2- 172.21.15.49:0/571121909 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1228065a60 0x7f1228067f20 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:16.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.177+0000 7f124246a640 1 --2- 172.21.15.49:0/571121909 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f123c070870 0x7f123c14fed0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:16.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.177+0000 7f124246a640 1 --2- 172.21.15.49:0/571121909 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f123c15a880 0x7f123c150410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:16.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.177+0000 7f124246a640 1 --2- 172.21.15.49:0/571121909 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f123c06d150 0x7f123c0cce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:16.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.177+0000 7f124246a640 1 -- 172.21.15.49:0/571121909 >> 172.21.15.49:0/571121909 conn(0x7f123c06a370 msgr2=0x7f123c15d720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:16.179 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.177+0000 7f124246a640 1 -- 172.21.15.49:0/571121909 shutdown_connections 2023-11-06T02:09:16.179 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.177+0000 7f124246a640 1 -- 172.21.15.49:0/571121909 wait complete. 2023-11-06T02:09:16.194 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T02:09:16.321 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.325+0000 7f5a60ca6640 1 Processor -- start 2023-11-06T02:09:16.322 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.325+0000 7f5a60ca6640 1 -- start start 2023-11-06T02:09:16.325 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.325+0000 7f5a60ca6640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5a5c1599e0 0x7f5a5c0d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:16.325 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.325+0000 7f5a60ca6640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a5c0cf9c0 con 0x7f5a5c0cf190 2023-11-06T02:09:16.325 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.325+0000 7f5a60ca6640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a5c159dc0 con 0x7f5a5c0ccab0 2023-11-06T02:09:16.326 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.325+0000 7f5a60ca6640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5c0aeeb0 con 0x7f5a5c1599e0 2023-11-06T02:09:16.326 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.325+0000 7f5a59d9b640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f5a5c0cf190 0x7f5a5c0cc3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:60918/0 (socket says 172.21.15.49:60918) 2023-11-06T02:09:16.326 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.325+0000 7f5a59d9b640 1 -- 172.21.15.49:0/2223856012 learned_addr learned my addr 172.21.15.49:0/2223856012 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:09:16.326 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.325+0000 7f5a5a59c640 1 --2- 172.21.15.49:0/2223856012 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5a5c1599e0 0x7f5a5c0d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:16.326 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.325+0000 7f5a5959a640 1 -- 172.21.15.49:0/2223856012 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4077199080 0 0) 0x7f5a5c0cf9c0 con 0x7f5a5c0cf190 2023-11-06T02:09:16.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.325+0000 7f5a5959a640 1 -- 172.21.15.49:0/2223856012 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a44003530 con 0x7f5a5c0cf190 2023-11-06T02:09:16.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.325+0000 7f5a5959a640 1 -- 172.21.15.49:0/2223856012 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1029751856 0 0) 0x7f5a44003530 con 0x7f5a5c0cf190 2023-11-06T02:09:16.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.325+0000 7f5a5959a640 1 -- 172.21.15.49:0/2223856012 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5a5c1599e0 msgr2=0x7f5a5c0d0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:16.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.325+0000 7f5a5959a640 1 --2- 172.21.15.49:0/2223856012 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5a5c1599e0 0x7f5a5c0d0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:16.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.325+0000 7f5a5959a640 1 -- 172.21.15.49:0/2223856012 >> v1:172.21.15.150:6789/0 conn(0x7f5a5c0ccab0 legacy=0x7f5a5c0cb120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:16.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.325+0000 7f5a5959a640 1 -- 172.21.15.49:0/2223856012 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a5c0cfc60 con 0x7f5a5c0cf190 2023-11-06T02:09:16.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.325+0000 7f5a5959a640 1 -- 172.21.15.49:0/2223856012 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7f5a50001680 con 0x7f5a5c0cf190 2023-11-06T02:09:16.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.325+0000 7f5a5959a640 1 -- 172.21.15.49:0/2223856012 >> v1:172.21.15.49:6789/0 conn(0x7f5a5c0cf190 legacy=0x7f5a5c0cc3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:16.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.325+0000 7f5a5959a640 1 --2- 172.21.15.49:0/2223856012 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5a5c1599e0 0x7f5a440039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:16.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.325+0000 7f5a5959a640 1 --2- 172.21.15.49:0/2223856012 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5a44003f30 0x7f5a440047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:16.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.325+0000 7f5a5959a640 1 --2- 172.21.15.49:0/2223856012 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a44004d10 0x7f5a44005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:16.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.325+0000 7f5a5959a640 1 -- 172.21.15.49:0/2223856012 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5c0aeeb0 con 0x7f5a44004d10 2023-11-06T02:09:16.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.325+0000 7f5a5959a640 1 -- 172.21.15.49:0/2223856012 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a4400bb20 con 0x7f5a44003f30 2023-11-06T02:09:16.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.325+0000 7f5a5959a640 1 -- 172.21.15.49:0/2223856012 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a4400bce0 con 0x7f5a5c1599e0 2023-11-06T02:09:16.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.325+0000 7f5a5ad9d640 1 --2- 172.21.15.49:0/2223856012 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a44004d10 0x7f5a44005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:16.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.325+0000 7f5a59d9b640 1 --2- 172.21.15.49:0/2223856012 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5a44003f30 0x7f5a440047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:16.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.325+0000 7f5a5a59c640 1 --2- 172.21.15.49:0/2223856012 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5a5c1599e0 0x7f5a440039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:16.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.325+0000 7f5a5ad9d640 1 -- 172.21.15.49:0/2223856012 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5a5c1599e0 msgr2=0x7f5a440039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:16.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.325+0000 7f5a5ad9d640 1 --2- 172.21.15.49:0/2223856012 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5a5c1599e0 0x7f5a440039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:16.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.325+0000 7f5a5ad9d640 1 -- 172.21.15.49:0/2223856012 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5a44003f30 msgr2=0x7f5a440047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:16.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.325+0000 7f5a5ad9d640 1 --2- 172.21.15.49:0/2223856012 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5a44003f30 0x7f5a440047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:16.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.325+0000 7f5a5ad9d640 1 -- 172.21.15.49:0/2223856012 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a44003530 con 0x7f5a44004d10 2023-11-06T02:09:16.335 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.329+0000 7f5a5ad9d640 1 --2- 172.21.15.49:0/2223856012 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a44004d10 0x7f5a44005190 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c070a90 tx=0x7f5a5401c720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16e46c7001850182 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:16.335 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.329+0000 7f5a5959a640 1 -- 172.21.15.49:0/2223856012 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a54003b80 con 0x7f5a44004d10 2023-11-06T02:09:16.335 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.329+0000 7f5a5959a640 1 -- 172.21.15.49:0/2223856012 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a54003ce0 con 0x7f5a44004d10 2023-11-06T02:09:16.335 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.329+0000 7f5a5959a640 1 -- 172.21.15.49:0/2223856012 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a540052c0 con 0x7f5a44004d10 2023-11-06T02:09:16.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.329+0000 7f5a60ca6640 1 -- 172.21.15.49:0/2223856012 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a44004d10 msgr2=0x7f5a44005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:16.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.329+0000 7f5a60ca6640 1 --2- 172.21.15.49:0/2223856012 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a44004d10 0x7f5a44005190 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c070a90 tx=0x7f5a5401c720 comp rx=0 tx=0).stop 2023-11-06T02:09:16.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.329+0000 7f5a5ad9d640 1 -- 172.21.15.49:0/2223856012 reap_dead start 2023-11-06T02:09:16.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.329+0000 7f5a60ca6640 1 -- 172.21.15.49:0/2223856012 shutdown_connections 2023-11-06T02:09:16.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.329+0000 7f5a60ca6640 1 -- 172.21.15.49:0/2223856012 >> 172.21.15.49:0/2223856012 conn(0x7f5a5c06d4b0 msgr2=0x7f5a5c06d890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:16.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.329+0000 7f5a60ca6640 1 -- 172.21.15.49:0/2223856012 shutdown_connections 2023-11-06T02:09:16.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.329+0000 7f5a60ca6640 1 -- 172.21.15.49:0/2223856012 wait complete. 2023-11-06T02:09:16.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.329+0000 7f5a60ca6640 1 Processor -- start 2023-11-06T02:09:16.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.329+0000 7f5a60ca6640 1 -- start start 2023-11-06T02:09:16.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.329+0000 7f5a60ca6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a44004d10 0x7f5a5c158970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:16.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.329+0000 7f5a60ca6640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5a5c14cf50 0x7f5a5c1486f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:16.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.329+0000 7f5a60ca6640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5a5c148c30 0x7f5a5c14bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:16.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.329+0000 7f5a60ca6640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5c1490b0 con 0x7f5a44004d10 2023-11-06T02:09:16.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.329+0000 7f5a60ca6640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5c157680 con 0x7f5a5c148c30 2023-11-06T02:09:16.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.329+0000 7f5a60ca6640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5c1584e0 con 0x7f5a5c14cf50 2023-11-06T02:09:16.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.329+0000 7f5a5a59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a44004d10 0x7f5a5c158970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:16.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.333+0000 7f5a5a59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a44004d10 0x7f5a5c158970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48448/0 (socket says 172.21.15.49:48448) 2023-11-06T02:09:16.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.333+0000 7f5a5a59c640 1 -- 172.21.15.49:0/2567973215 learned_addr learned my addr 172.21.15.49:0/2567973215 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:16.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.333+0000 7f5a5ad9d640 1 --2- 172.21.15.49:0/2567973215 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5a5c148c30 0x7f5a5c14bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:16.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.333+0000 7f5a59d9b640 1 --2- 172.21.15.49:0/2567973215 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5a5c14cf50 0x7f5a5c1486f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:16.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.333+0000 7f5a5a59c640 1 -- 172.21.15.49:0/2567973215 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5a5c14cf50 msgr2=0x7f5a5c1486f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:16.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.333+0000 7f5a5a59c640 1 --2- 172.21.15.49:0/2567973215 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5a5c14cf50 0x7f5a5c1486f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:16.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.333+0000 7f5a5a59c640 1 -- 172.21.15.49:0/2567973215 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5a5c148c30 msgr2=0x7f5a5c14bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:16.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.333+0000 7f5a5a59c640 1 --2- 172.21.15.49:0/2567973215 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5a5c148c30 0x7f5a5c14bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:16.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.333+0000 7f5a5a59c640 1 -- 172.21.15.49:0/2567973215 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a5c1578d0 con 0x7f5a44004d10 2023-11-06T02:09:16.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.333+0000 7f5a5ad9d640 1 --2- 172.21.15.49:0/2567973215 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5a5c148c30 0x7f5a5c14bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:09:16.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.333+0000 7f5a5a59c640 1 --2- 172.21.15.49:0/2567973215 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a44004d10 0x7f5a5c158970 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c0115e0 tx=0x7f5a4c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:16.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.333+0000 7f5a4affd640 1 -- 172.21.15.49:0/2567973215 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a4c015870 con 0x7f5a44004d10 2023-11-06T02:09:16.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.333+0000 7f5a4affd640 1 -- 172.21.15.49:0/2567973215 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a4c0159d0 con 0x7f5a44004d10 2023-11-06T02:09:16.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.333+0000 7f5a60ca6640 1 -- 172.21.15.49:0/2567973215 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a5c157bb0 con 0x7f5a44004d10 2023-11-06T02:09:16.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.333+0000 7f5a4affd640 1 -- 172.21.15.49:0/2567973215 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a4c015d40 con 0x7f5a44004d10 2023-11-06T02:09:16.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.333+0000 7f5a60ca6640 1 -- 172.21.15.49:0/2567973215 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a5c158170 con 0x7f5a44004d10 2023-11-06T02:09:16.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.333+0000 7f5a60ca6640 1 -- 172.21.15.49:0/2567973215 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a5c067140 con 0x7f5a44004d10 2023-11-06T02:09:16.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.333+0000 7f5a4affd640 1 -- 172.21.15.49:0/2567973215 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f5a4c0143e0 con 0x7f5a44004d10 2023-11-06T02:09:16.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.333+0000 7f5a4affd640 1 --2- 172.21.15.49:0/2567973215 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5a3c065a60 0x7f5a3c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:16.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.333+0000 7f5a4affd640 1 -- 172.21.15.49:0/2567973215 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f5a4c083bc0 con 0x7f5a44004d10 2023-11-06T02:09:16.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.333+0000 7f5a59d9b640 1 --2- 172.21.15.49:0/2567973215 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5a3c065a60 0x7f5a3c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:16.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.333+0000 7f5a59d9b640 1 --2- 172.21.15.49:0/2567973215 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5a3c065a60 0x7f5a3c067f20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5a50004880 tx=0x7f5a500047f0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:16.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.337+0000 7f5a4affd640 1 -- 172.21.15.49:0/2567973215 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f5a4c0513e0 con 0x7f5a44004d10 2023-11-06T02:09:16.488 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.489+0000 7f5a60ca6640 1 -- 172.21.15.49:0/2567973215 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f5a5c1565b0 con 0x7f5a44004d10 2023-11-06T02:09:16.489 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.489+0000 7f5a4affd640 1 -- 172.21.15.49:0/2567973215 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5a4c054a00 con 0x7f5a44004d10 2023-11-06T02:09:16.489 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 8 2023-11-06T02:09:16.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.493+0000 7f5a60ca6640 1 -- 172.21.15.49:0/2567973215 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5a3c065a60 msgr2=0x7f5a3c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:16.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.493+0000 7f5a60ca6640 1 --2- 172.21.15.49:0/2567973215 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5a3c065a60 0x7f5a3c067f20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5a50004880 tx=0x7f5a500047f0 comp rx=0 tx=0).stop 2023-11-06T02:09:16.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.493+0000 7f5a60ca6640 1 -- 172.21.15.49:0/2567973215 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a44004d10 msgr2=0x7f5a5c158970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:16.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.493+0000 7f5a60ca6640 1 --2- 172.21.15.49:0/2567973215 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a44004d10 0x7f5a5c158970 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c0115e0 tx=0x7f5a4c011aa0 comp rx=0 tx=0).stop 2023-11-06T02:09:16.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.493+0000 7f5a60ca6640 1 -- 172.21.15.49:0/2567973215 shutdown_connections 2023-11-06T02:09:16.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.493+0000 7f5a60ca6640 1 --2- 172.21.15.49:0/2567973215 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5a3c065a60 0x7f5a3c067f20 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:16.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.493+0000 7f5a60ca6640 1 --2- 172.21.15.49:0/2567973215 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5a5c148c30 0x7f5a5c14bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:16.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.493+0000 7f5a60ca6640 1 --2- 172.21.15.49:0/2567973215 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5a5c14cf50 0x7f5a5c1486f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:16.493 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.493+0000 7f5a60ca6640 1 --2- 172.21.15.49:0/2567973215 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a44004d10 0x7f5a5c158970 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:16.493 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.493+0000 7f5a60ca6640 1 -- 172.21.15.49:0/2567973215 >> 172.21.15.49:0/2567973215 conn(0x7f5a5c06d4b0 msgr2=0x7f5a5c1540a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:16.493 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.493+0000 7f5a60ca6640 1 -- 172.21.15.49:0/2567973215 shutdown_connections 2023-11-06T02:09:16.493 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.493+0000 7f5a60ca6640 1 -- 172.21.15.49:0/2567973215 wait complete. 2023-11-06T02:09:16.506 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T02:09:16.633 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.637+0000 7f356c16e640 1 Processor -- start 2023-11-06T02:09:16.634 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.637+0000 7f356c16e640 1 -- start start 2023-11-06T02:09:16.635 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.637+0000 7f356c16e640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f356415a9e0 0x7f35640cb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:16.635 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.637+0000 7f356c16e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f35640afbb0 0x7f35640cca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:16.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.637+0000 7f356c16e640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35640cc5c0 con 0x7f35640b04e0 2023-11-06T02:09:16.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.637+0000 7f356c16e640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35640cb9b0 con 0x7f356415a9e0 2023-11-06T02:09:16.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.637+0000 7f356c16e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35640cbc90 con 0x7f35640afbb0 2023-11-06T02:09:16.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.637+0000 7f3569f0a640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f356415a9e0 0x7f35640cb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:16.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.637+0000 7f356a70b640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f35640b04e0 0x7f35640ac4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:60926/0 (socket says 172.21.15.49:60926) 2023-11-06T02:09:16.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.637+0000 7f3569f0a640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f356415a9e0 0x7f35640cb2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48462/0 (socket says 172.21.15.49:48462) 2023-11-06T02:09:16.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.637+0000 7f356a70b640 1 -- 172.21.15.49:0/673837627 learned_addr learned my addr 172.21.15.49:0/673837627 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:09:16.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.637+0000 7f3569709640 1 --2- 172.21.15.49:0/673837627 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f35640afbb0 0x7f35640cca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:16.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.637+0000 7f3569f0a640 1 -- 172.21.15.49:0/673837627 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f35640afbb0 msgr2=0x7f35640cca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:16.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.637+0000 7f3569f0a640 1 --2- 172.21.15.49:0/673837627 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f35640afbb0 0x7f35640cca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:16.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.637+0000 7f3569f0a640 1 -- 172.21.15.49:0/673837627 >> v1:172.21.15.49:6789/0 conn(0x7f35640b04e0 legacy=0x7f35640ac4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:16.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.637+0000 7f3569f0a640 1 -- 172.21.15.49:0/673837627 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35640cbf70 con 0x7f356415a9e0 2023-11-06T02:09:16.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.637+0000 7f3568f08640 1 -- 172.21.15.49:0/673837627 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3870704808 0 0) 0x7f35640cc5c0 con 0x7f35640b04e0 2023-11-06T02:09:16.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.637+0000 7f3569f0a640 1 --2- 172.21.15.49:0/673837627 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f356415a9e0 0x7f35640cb2d0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f355c009d90 tx=0x7f355c01c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c590e76a4252012 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:16.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.641+0000 7f3568f08640 1 -- 172.21.15.49:0/673837627 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f355c0a58f0 con 0x7f356415a9e0 2023-11-06T02:09:16.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.641+0000 7f3568f08640 1 -- 172.21.15.49:0/673837627 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f355c0a5a50 con 0x7f356415a9e0 2023-11-06T02:09:16.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.641+0000 7f3568f08640 1 -- 172.21.15.49:0/673837627 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f355c0a5de0 con 0x7f356415a9e0 2023-11-06T02:09:16.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.641+0000 7f356c16e640 1 -- 172.21.15.49:0/673837627 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f356415a9e0 msgr2=0x7f35640cb2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:16.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.641+0000 7f356c16e640 1 --2- 172.21.15.49:0/673837627 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f356415a9e0 0x7f35640cb2d0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f355c009d90 tx=0x7f355c01c920 comp rx=0 tx=0).stop 2023-11-06T02:09:16.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.641+0000 7f356c16e640 1 -- 172.21.15.49:0/673837627 shutdown_connections 2023-11-06T02:09:16.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.641+0000 7f356c16e640 1 --2- 172.21.15.49:0/673837627 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f35640afbb0 0x7f35640cca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:16.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.641+0000 7f356c16e640 1 --2- 172.21.15.49:0/673837627 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f356415a9e0 0x7f35640cb2d0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:16.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.641+0000 7f356c16e640 1 -- 172.21.15.49:0/673837627 >> 172.21.15.49:0/673837627 conn(0x7f3564098a40 msgr2=0x7f35640cf860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:16.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.641+0000 7f356c16e640 1 -- 172.21.15.49:0/673837627 shutdown_connections 2023-11-06T02:09:16.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.641+0000 7f356c16e640 1 -- 172.21.15.49:0/673837627 wait complete. 2023-11-06T02:09:16.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.641+0000 7f356c16e640 1 Processor -- start 2023-11-06T02:09:16.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.641+0000 7f356c16e640 1 -- start start 2023-11-06T02:09:16.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.641+0000 7f356c16e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35640afbb0 0x7f3564153970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:16.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.641+0000 7f356c16e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f35640b04e0 0x7f3564164a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:16.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.641+0000 7f356c16e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f356415a9e0 0x7f35640b3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:16.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.641+0000 7f356c16e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3564153eb0 con 0x7f35640afbb0 2023-11-06T02:09:16.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.641+0000 7f356c16e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35640ce2d0 con 0x7f356415a9e0 2023-11-06T02:09:16.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.641+0000 7f356c16e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35640ce5b0 con 0x7f35640b04e0 2023-11-06T02:09:16.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.641+0000 7f3569f0a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35640afbb0 0x7f3564153970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:16.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.641+0000 7f3569f0a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35640afbb0 0x7f3564153970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48472/0 (socket says 172.21.15.49:48472) 2023-11-06T02:09:16.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.641+0000 7f3569f0a640 1 -- 172.21.15.49:0/3223085840 learned_addr learned my addr 172.21.15.49:0/3223085840 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:16.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.641+0000 7f3569709640 1 --2- 172.21.15.49:0/3223085840 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f35640b04e0 0x7f3564164a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:16.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.641+0000 7f356a70b640 1 --2- 172.21.15.49:0/3223085840 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f356415a9e0 0x7f35640b3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:16.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.641+0000 7f3569f0a640 1 -- 172.21.15.49:0/3223085840 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f35640b04e0 msgr2=0x7f3564164a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:16.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.641+0000 7f3569f0a640 1 --2- 172.21.15.49:0/3223085840 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f35640b04e0 0x7f3564164a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:16.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.641+0000 7f3569f0a640 1 -- 172.21.15.49:0/3223085840 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f356415a9e0 msgr2=0x7f35640b3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:16.648 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.641+0000 7f3569f0a640 1 --2- 172.21.15.49:0/3223085840 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f356415a9e0 0x7f35640b3bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:16.648 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.641+0000 7f3569f0a640 1 -- 172.21.15.49:0/3223085840 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f355c009a70 con 0x7f35640afbb0 2023-11-06T02:09:16.648 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.641+0000 7f3569709640 1 --2- 172.21.15.49:0/3223085840 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f35640b04e0 0x7f3564164a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:09:16.648 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.641+0000 7f3569f0a640 1 --2- 172.21.15.49:0/3223085840 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35640afbb0 0x7f3564153970 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f355c0b0040 tx=0x7f355c0196c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:16.648 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.641+0000 7f35567fc640 1 -- 172.21.15.49:0/3223085840 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f355c0a58f0 con 0x7f35640afbb0 2023-11-06T02:09:16.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.641+0000 7f35567fc640 1 -- 172.21.15.49:0/3223085840 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f355c0b9860 con 0x7f35640afbb0 2023-11-06T02:09:16.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.641+0000 7f356c16e640 1 -- 172.21.15.49:0/3223085840 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35640cd110 con 0x7f35640afbb0 2023-11-06T02:09:16.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.641+0000 7f35567fc640 1 -- 172.21.15.49:0/3223085840 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f355c0b9bd0 con 0x7f35640afbb0 2023-11-06T02:09:16.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.641+0000 7f356c16e640 1 -- 172.21.15.49:0/3223085840 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35640b36b0 con 0x7f35640afbb0 2023-11-06T02:09:16.650 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.645+0000 7f356c16e640 1 -- 172.21.15.49:0/3223085840 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35640cb430 con 0x7f35640afbb0 2023-11-06T02:09:16.651 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.645+0000 7f35567fc640 1 -- 172.21.15.49:0/3223085840 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f355c0b9d30 con 0x7f35640afbb0 2023-11-06T02:09:16.651 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.645+0000 7f35567fc640 1 --2- 172.21.15.49:0/3223085840 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3548065a60 0x7f3548067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:16.651 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.645+0000 7f35567fc640 1 -- 172.21.15.49:0/3223085840 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f355c11e6c0 con 0x7f35640afbb0 2023-11-06T02:09:16.651 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.645+0000 7f3569709640 1 --2- 172.21.15.49:0/3223085840 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3548065a60 0x7f3548067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:16.652 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.645+0000 7f3569709640 1 --2- 172.21.15.49:0/3223085840 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3548065a60 0x7f3548067f20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f35641f89f0 tx=0x7f3558008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:16.652 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.649+0000 7f35567fc640 1 -- 172.21.15.49:0/3223085840 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f355c0ebf70 con 0x7f35640afbb0 2023-11-06T02:09:16.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.797+0000 7f356c16e640 1 -- 172.21.15.49:0/3223085840 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f35640670c0 con 0x7f35640afbb0 2023-11-06T02:09:16.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.797+0000 7f35567fc640 1 -- 172.21.15.49:0/3223085840 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f355c0ef590 con 0x7f35640afbb0 2023-11-06T02:09:16.796 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2023-11-06T02:09:16.798 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.801+0000 7f356c16e640 1 -- 172.21.15.49:0/3223085840 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3548065a60 msgr2=0x7f3548067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:16.798 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.801+0000 7f356c16e640 1 --2- 172.21.15.49:0/3223085840 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3548065a60 0x7f3548067f20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f35641f89f0 tx=0x7f3558008040 comp rx=0 tx=0).stop 2023-11-06T02:09:16.799 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.801+0000 7f356c16e640 1 -- 172.21.15.49:0/3223085840 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35640afbb0 msgr2=0x7f3564153970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:16.799 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.801+0000 7f356c16e640 1 --2- 172.21.15.49:0/3223085840 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35640afbb0 0x7f3564153970 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f355c0b0040 tx=0x7f355c0196c0 comp rx=0 tx=0).stop 2023-11-06T02:09:16.799 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.801+0000 7f356c16e640 1 -- 172.21.15.49:0/3223085840 shutdown_connections 2023-11-06T02:09:16.799 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.801+0000 7f356c16e640 1 --2- 172.21.15.49:0/3223085840 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3548065a60 0x7f3548067f20 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:16.799 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.801+0000 7f356c16e640 1 --2- 172.21.15.49:0/3223085840 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f356415a9e0 0x7f35640b3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:16.800 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.801+0000 7f356c16e640 1 --2- 172.21.15.49:0/3223085840 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f35640b04e0 0x7f3564164a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:16.800 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.801+0000 7f356c16e640 1 --2- 172.21.15.49:0/3223085840 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35640afbb0 0x7f3564153970 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:16.800 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.801+0000 7f356c16e640 1 -- 172.21.15.49:0/3223085840 >> 172.21.15.49:0/3223085840 conn(0x7f3564098a40 msgr2=0x7f35640ac4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:16.800 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.801+0000 7f356c16e640 1 -- 172.21.15.49:0/3223085840 shutdown_connections 2023-11-06T02:09:16.800 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.801+0000 7f356c16e640 1 -- 172.21.15.49:0/3223085840 wait complete. 2023-11-06T02:09:16.813 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T02:09:16.930 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.933+0000 7fc4df943640 1 Processor -- start 2023-11-06T02:09:16.930 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.933+0000 7fc4df943640 1 -- start start 2023-11-06T02:09:16.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.933+0000 7fc4df943640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc4d815e0c0 0x7fc4d80ce840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:16.932 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.933+0000 7fc4df943640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc4d80b16e0 0x7fc4d80b1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:16.932 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.933+0000 7fc4df943640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4d80d1430 con 0x7fc4d815e4a0 2023-11-06T02:09:16.932 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.933+0000 7fc4df943640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4d80cd8d0 con 0x7fc4d815e0c0 2023-11-06T02:09:16.932 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.933+0000 7fc4df943640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4d80b20d0 con 0x7fc4d80b16e0 2023-11-06T02:09:16.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.933+0000 7fc4dd6df640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc4d815e0c0 0x7fc4d80ce840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:16.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.933+0000 7fc4dcede640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fc4d815e4a0 0x7fc4d80cdcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:60942/0 (socket says 172.21.15.49:60942) 2023-11-06T02:09:16.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.933+0000 7fc4dcede640 1 -- 172.21.15.49:0/3576815841 learned_addr learned my addr 172.21.15.49:0/3576815841 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:09:16.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.933+0000 7fc4ddee0640 1 --2- 172.21.15.49:0/3576815841 >> v2:172.21.15.150:3300/0 conn(0x7fc4d80b16e0 0x7fc4d80b1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:16.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.933+0000 7fc4dd6df640 1 -- 172.21.15.49:0/3576815841 >> v2:172.21.15.150:3300/0 conn(0x7fc4d80b16e0 msgr2=0x7fc4d80b1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:16.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.933+0000 7fc4dd6df640 1 --2- 172.21.15.49:0/3576815841 >> v2:172.21.15.150:3300/0 conn(0x7fc4d80b16e0 0x7fc4d80b1b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:16.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.933+0000 7fc4dd6df640 1 -- 172.21.15.49:0/3576815841 >> v1:172.21.15.49:6789/0 conn(0x7fc4d815e4a0 legacy=0x7fc4d80cdcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:16.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.933+0000 7fc4dd6df640 1 -- 172.21.15.49:0/3576815841 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4d80d1710 con 0x7fc4d815e0c0 2023-11-06T02:09:16.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.933+0000 7fc4cffff640 1 -- 172.21.15.49:0/3576815841 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 220817694 0 0) 0x7fc4d80d1430 con 0x7fc4d815e4a0 2023-11-06T02:09:16.939 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.937+0000 7fc4dd6df640 1 --2- 172.21.15.49:0/3576815841 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4d815e0c0 0x7fc4d80ce840 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fc4d80b22f0 tx=0x7fc4d001c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eee885640b7c821a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:16.939 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.937+0000 7fc4cffff640 1 -- 172.21.15.49:0/3576815841 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d00a5830 con 0x7fc4d815e0c0 2023-11-06T02:09:16.939 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.937+0000 7fc4cffff640 1 -- 172.21.15.49:0/3576815841 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4d00a5990 con 0x7fc4d815e0c0 2023-11-06T02:09:16.940 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.937+0000 7fc4cffff640 1 -- 172.21.15.49:0/3576815841 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d00a5d20 con 0x7fc4d815e0c0 2023-11-06T02:09:16.940 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.937+0000 7fc4df943640 1 -- 172.21.15.49:0/3576815841 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4d815e0c0 msgr2=0x7fc4d80ce840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:16.940 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.937+0000 7fc4df943640 1 --2- 172.21.15.49:0/3576815841 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4d815e0c0 0x7fc4d80ce840 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fc4d80b22f0 tx=0x7fc4d001c880 comp rx=0 tx=0).stop 2023-11-06T02:09:16.940 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.937+0000 7fc4df943640 1 -- 172.21.15.49:0/3576815841 shutdown_connections 2023-11-06T02:09:16.941 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.937+0000 7fc4df943640 1 --2- 172.21.15.49:0/3576815841 >> v2:172.21.15.150:3300/0 conn(0x7fc4d80b16e0 0x7fc4d80b1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:16.941 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.937+0000 7fc4df943640 1 --2- 172.21.15.49:0/3576815841 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4d815e0c0 0x7fc4d80ce840 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:16.941 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.937+0000 7fc4df943640 1 -- 172.21.15.49:0/3576815841 >> 172.21.15.49:0/3576815841 conn(0x7fc4d8099c50 msgr2=0x7fc4d80d0940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:16.941 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.937+0000 7fc4df943640 1 -- 172.21.15.49:0/3576815841 shutdown_connections 2023-11-06T02:09:16.941 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.937+0000 7fc4df943640 1 -- 172.21.15.49:0/3576815841 wait complete. 2023-11-06T02:09:16.942 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.937+0000 7fc4df943640 1 Processor -- start 2023-11-06T02:09:16.942 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.937+0000 7fc4df943640 1 -- start start 2023-11-06T02:09:16.942 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.937+0000 7fc4df943640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4d80b16e0 0x7fc4d815b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:16.942 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.937+0000 7fc4df943640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4d815e0c0 0x7fc4d815cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:16.942 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.937+0000 7fc4df943640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4d815e4a0 0x7fc4d815d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:16.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.937+0000 7fc4df943640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d806ce40 con 0x7fc4d80b16e0 2023-11-06T02:09:16.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.937+0000 7fc4df943640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d806c230 con 0x7fc4d815e4a0 2023-11-06T02:09:16.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.937+0000 7fc4df943640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d806c510 con 0x7fc4d815e0c0 2023-11-06T02:09:16.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.937+0000 7fc4dd6df640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4d80b16e0 0x7fc4d815b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:16.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.937+0000 7fc4dd6df640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4d80b16e0 0x7fc4d815b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48488/0 (socket says 172.21.15.49:48488) 2023-11-06T02:09:16.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.937+0000 7fc4dd6df640 1 -- 172.21.15.49:0/4136614354 learned_addr learned my addr 172.21.15.49:0/4136614354 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:16.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.937+0000 7fc4dd6df640 1 -- 172.21.15.49:0/4136614354 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4d815e0c0 msgr2=0x7fc4d815cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:16.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.937+0000 7fc4dd6df640 1 --2- 172.21.15.49:0/4136614354 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4d815e0c0 0x7fc4d815cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:16.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.937+0000 7fc4dd6df640 1 -- 172.21.15.49:0/4136614354 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4d815e4a0 msgr2=0x7fc4d815d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:16.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.937+0000 7fc4dd6df640 1 --2- 172.21.15.49:0/4136614354 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4d815e4a0 0x7fc4d815d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:16.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.937+0000 7fc4dd6df640 1 -- 172.21.15.49:0/4136614354 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4d0009a70 con 0x7fc4d80b16e0 2023-11-06T02:09:16.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.937+0000 7fc4dd6df640 1 --2- 172.21.15.49:0/4136614354 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4d80b16e0 0x7fc4d815b1f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc4d0019490 tx=0x7fc4d0019620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:16.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.941+0000 7fc4cdffb640 1 -- 172.21.15.49:0/4136614354 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d00ad850 con 0x7fc4d80b16e0 2023-11-06T02:09:16.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.941+0000 7fc4cdffb640 1 -- 172.21.15.49:0/4136614354 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4d00ad9b0 con 0x7fc4d80b16e0 2023-11-06T02:09:16.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.941+0000 7fc4df943640 1 -- 172.21.15.49:0/4136614354 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4d806c7f0 con 0x7fc4d80b16e0 2023-11-06T02:09:16.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.941+0000 7fc4cdffb640 1 -- 172.21.15.49:0/4136614354 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d00add20 con 0x7fc4d80b16e0 2023-11-06T02:09:16.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.941+0000 7fc4df943640 1 -- 172.21.15.49:0/4136614354 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4d815b730 con 0x7fc4d80b16e0 2023-11-06T02:09:16.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.941+0000 7fc4df943640 1 -- 172.21.15.49:0/4136614354 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4d8067140 con 0x7fc4d80b16e0 2023-11-06T02:09:16.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.941+0000 7fc4cdffb640 1 -- 172.21.15.49:0/4136614354 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fc4d00b4050 con 0x7fc4d80b16e0 2023-11-06T02:09:16.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.941+0000 7fc4cdffb640 1 --2- 172.21.15.49:0/4136614354 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc4bc065a90 0x7fc4bc067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:16.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.941+0000 7fc4cdffb640 1 -- 172.21.15.49:0/4136614354 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fc4d011df10 con 0x7fc4d80b16e0 2023-11-06T02:09:16.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.941+0000 7fc4dcede640 1 --2- 172.21.15.49:0/4136614354 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc4bc065a90 0x7fc4bc067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:16.948 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.941+0000 7fc4dcede640 1 --2- 172.21.15.49:0/4136614354 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc4bc065a90 0x7fc4bc067f50 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc4c80011f0 tx=0x7fc4c8007930 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:16.948 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:16.945+0000 7fc4cdffb640 1 -- 172.21.15.49:0/4136614354 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fc4d00eb730 con 0x7fc4d80b16e0 2023-11-06T02:09:17.096 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.097+0000 7fc4df943640 1 -- 172.21.15.49:0/4136614354 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc4d80cc9b0 con 0x7fc4d80b16e0 2023-11-06T02:09:17.097 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.101+0000 7fc4cdffb640 1 -- 172.21.15.49:0/4136614354 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc4d00eed50 con 0x7fc4d80b16e0 2023-11-06T02:09:17.098 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2023-11-06T02:09:17.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.101+0000 7fc4df943640 1 -- 172.21.15.49:0/4136614354 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc4bc065a90 msgr2=0x7fc4bc067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:17.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.101+0000 7fc4df943640 1 --2- 172.21.15.49:0/4136614354 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc4bc065a90 0x7fc4bc067f50 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc4c80011f0 tx=0x7fc4c8007930 comp rx=0 tx=0).stop 2023-11-06T02:09:17.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.101+0000 7fc4df943640 1 -- 172.21.15.49:0/4136614354 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4d80b16e0 msgr2=0x7fc4d815b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:17.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.101+0000 7fc4df943640 1 --2- 172.21.15.49:0/4136614354 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4d80b16e0 0x7fc4d815b1f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc4d0019490 tx=0x7fc4d0019620 comp rx=0 tx=0).stop 2023-11-06T02:09:17.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.101+0000 7fc4df943640 1 -- 172.21.15.49:0/4136614354 shutdown_connections 2023-11-06T02:09:17.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.101+0000 7fc4df943640 1 --2- 172.21.15.49:0/4136614354 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc4bc065a90 0x7fc4bc067f50 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:17.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.101+0000 7fc4df943640 1 --2- 172.21.15.49:0/4136614354 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4d815e4a0 0x7fc4d815d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:17.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.101+0000 7fc4df943640 1 --2- 172.21.15.49:0/4136614354 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4d815e0c0 0x7fc4d815cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:17.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.101+0000 7fc4df943640 1 --2- 172.21.15.49:0/4136614354 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4d80b16e0 0x7fc4d815b1f0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:17.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.101+0000 7fc4df943640 1 -- 172.21.15.49:0/4136614354 >> 172.21.15.49:0/4136614354 conn(0x7fc4d8099c50 msgr2=0x7fc4d806fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:17.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.101+0000 7fc4df943640 1 -- 172.21.15.49:0/4136614354 shutdown_connections 2023-11-06T02:09:17.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.101+0000 7fc4df943640 1 -- 172.21.15.49:0/4136614354 wait complete. 2023-11-06T02:09:17.117 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-06T02:09:17.247 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.249+0000 7f21a39ec640 1 Processor -- start 2023-11-06T02:09:17.247 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.249+0000 7f21a39ec640 1 -- start start 2023-11-06T02:09:17.248 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.249+0000 7f21a39ec640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f219c0b6100 0x7f219c0cb960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:17.248 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.249+0000 7f21a39ec640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f219c15e440 0x7f219c15b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:17.249 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.249+0000 7f21a39ec640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f219c15f3c0 con 0x7f219c0cb580 2023-11-06T02:09:17.249 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.249+0000 7f21a39ec640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f219c15e8d0 con 0x7f219c0b6100 2023-11-06T02:09:17.249 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.249+0000 7f21a39ec640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f219c15eb20 con 0x7f219c15e440 2023-11-06T02:09:17.249 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.249+0000 7f21a0f87640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f219c0b6100 0x7f219c0cb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:17.249 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.249+0000 7f21a0f87640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f219c0b6100 0x7f219c0cb960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48502/0 (socket says 172.21.15.49:48502) 2023-11-06T02:09:17.249 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.249+0000 7f21a0f87640 1 -- 172.21.15.49:0/1819569694 learned_addr learned my addr 172.21.15.49:0/1819569694 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:17.250 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.249+0000 7f21a1f89640 1 --2- 172.21.15.49:0/1819569694 >> v2:172.21.15.150:3300/0 conn(0x7f219c15e440 0x7f219c15b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:17.250 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.249+0000 7f21a0f87640 1 -- 172.21.15.49:0/1819569694 >> v2:172.21.15.150:3300/0 conn(0x7f219c15e440 msgr2=0x7f219c15b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:17.250 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.249+0000 7f21a0f87640 1 --2- 172.21.15.49:0/1819569694 >> v2:172.21.15.150:3300/0 conn(0x7f219c15e440 0x7f219c15b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:17.250 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.249+0000 7f21a0f87640 1 -- 172.21.15.49:0/1819569694 >> v1:172.21.15.150:6789/0 conn(0x7f219c0cb580 legacy=0x7f219c0ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:17.250 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.249+0000 7f21a0f87640 1 -- 172.21.15.49:0/1819569694 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f219c15ee00 con 0x7f219c0b6100 2023-11-06T02:09:17.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.253+0000 7f21a0f87640 1 --2- 172.21.15.49:0/1819569694 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f219c0b6100 0x7f219c0cb960 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f219c158a40 tx=0x7f219001c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6e36e4e1f338b84 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:17.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.253+0000 7f219bfff640 1 -- 172.21.15.49:0/1819569694 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21900ac640 con 0x7f219c0b6100 2023-11-06T02:09:17.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.253+0000 7f219bfff640 1 -- 172.21.15.49:0/1819569694 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21900ac7a0 con 0x7f219c0b6100 2023-11-06T02:09:17.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.253+0000 7f219bfff640 1 -- 172.21.15.49:0/1819569694 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21900acb30 con 0x7f219c0b6100 2023-11-06T02:09:17.253 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.253+0000 7f21a39ec640 1 -- 172.21.15.49:0/1819569694 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f219c0b6100 msgr2=0x7f219c0cb960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:17.253 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.253+0000 7f21a39ec640 1 --2- 172.21.15.49:0/1819569694 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f219c0b6100 0x7f219c0cb960 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f219c158a40 tx=0x7f219001c400 comp rx=0 tx=0).stop 2023-11-06T02:09:17.253 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.253+0000 7f21a39ec640 1 -- 172.21.15.49:0/1819569694 shutdown_connections 2023-11-06T02:09:17.253 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.253+0000 7f21a39ec640 1 --2- 172.21.15.49:0/1819569694 >> v2:172.21.15.150:3300/0 conn(0x7f219c15e440 0x7f219c15b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:17.253 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.253+0000 7f21a39ec640 1 --2- 172.21.15.49:0/1819569694 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f219c0b6100 0x7f219c0cb960 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:17.253 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.253+0000 7f21a39ec640 1 -- 172.21.15.49:0/1819569694 >> 172.21.15.49:0/1819569694 conn(0x7f219c0cd2d0 msgr2=0x7f219c0cd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:17.254 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.253+0000 7f21a39ec640 1 -- 172.21.15.49:0/1819569694 shutdown_connections 2023-11-06T02:09:17.254 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.253+0000 7f21a39ec640 1 -- 172.21.15.49:0/1819569694 wait complete. 2023-11-06T02:09:17.256 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.253+0000 7f21a39ec640 1 Processor -- start 2023-11-06T02:09:17.256 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.253+0000 7f21a39ec640 1 -- start start 2023-11-06T02:09:17.256 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.257+0000 7f21a39ec640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f219c0b6100 0x7f219c15f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:17.257 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.257+0000 7f21a39ec640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f219c0cb580 0x7f219c1549e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:17.257 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.257+0000 7f21a39ec640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f219c15e440 0x7f219c154f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:17.257 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.257+0000 7f21a39ec640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f219c0b1c20 con 0x7f219c0cb580 2023-11-06T02:09:17.257 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.257+0000 7f21a39ec640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f219c0b1360 con 0x7f219c0b6100 2023-11-06T02:09:17.257 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.257+0000 7f21a39ec640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f219c0b15d0 con 0x7f219c15e440 2023-11-06T02:09:17.257 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.257+0000 7f21a0f87640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f219c0cb580 0x7f219c1549e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:17.258 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.257+0000 7f21a0f87640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f219c0cb580 0x7f219c1549e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48516/0 (socket says 172.21.15.49:48516) 2023-11-06T02:09:17.258 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.257+0000 7f21a0f87640 1 -- 172.21.15.49:0/767845853 learned_addr learned my addr 172.21.15.49:0/767845853 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:17.258 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.257+0000 7f21a1788640 1 --2- 172.21.15.49:0/767845853 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f219c0b6100 0x7f219c15f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:17.258 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.257+0000 7f21a1f89640 1 --2- 172.21.15.49:0/767845853 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f219c15e440 0x7f219c154f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:17.258 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.257+0000 7f21a0f87640 1 -- 172.21.15.49:0/767845853 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f219c15e440 msgr2=0x7f219c154f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:17.259 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.257+0000 7f21a0f87640 1 --2- 172.21.15.49:0/767845853 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f219c15e440 0x7f219c154f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:17.259 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.257+0000 7f21a0f87640 1 -- 172.21.15.49:0/767845853 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f219c0b6100 msgr2=0x7f219c15f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:17.259 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.257+0000 7f21a0f87640 1 --2- 172.21.15.49:0/767845853 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f219c0b6100 0x7f219c15f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:17.259 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.257+0000 7f21a0f87640 1 -- 172.21.15.49:0/767845853 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21900094d0 con 0x7f219c0cb580 2023-11-06T02:09:17.259 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.257+0000 7f21a1788640 1 --2- 172.21.15.49:0/767845853 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f219c0b6100 0x7f219c15f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:09:17.260 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.257+0000 7f21a0f87640 1 --2- 172.21.15.49:0/767845853 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f219c0cb580 0x7f219c1549e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f219001aab0 tx=0x7f21900191a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:17.260 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.257+0000 7f2199ffb640 1 -- 172.21.15.49:0/767845853 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21900b7640 con 0x7f219c0cb580 2023-11-06T02:09:17.260 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.257+0000 7f2199ffb640 1 -- 172.21.15.49:0/767845853 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21900b77a0 con 0x7f219c0cb580 2023-11-06T02:09:17.260 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.257+0000 7f21a39ec640 1 -- 172.21.15.49:0/767845853 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f219c0cecb0 con 0x7f219c0cb580 2023-11-06T02:09:17.260 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.257+0000 7f2199ffb640 1 -- 172.21.15.49:0/767845853 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21900b7b30 con 0x7f219c0cb580 2023-11-06T02:09:17.260 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.257+0000 7f21a39ec640 1 -- 172.21.15.49:0/767845853 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f219c15fa90 con 0x7f219c0cb580 2023-11-06T02:09:17.261 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.257+0000 7f21a39ec640 1 -- 172.21.15.49:0/767845853 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f219c0cc530 con 0x7f219c0cb580 2023-11-06T02:09:17.261 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.257+0000 7f2199ffb640 1 -- 172.21.15.49:0/767845853 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f21900b3040 con 0x7f219c0cb580 2023-11-06T02:09:17.261 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.257+0000 7f2199ffb640 1 --2- 172.21.15.49:0/767845853 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2188065a60 0x7f2188067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:17.261 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.257+0000 7f2199ffb640 1 -- 172.21.15.49:0/767845853 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f219011dff0 con 0x7f219c0cb580 2023-11-06T02:09:17.262 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.257+0000 7f21a1788640 1 --2- 172.21.15.49:0/767845853 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2188065a60 0x7f2188067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:17.262 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.261+0000 7f21a1788640 1 --2- 172.21.15.49:0/767845853 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2188065a60 0x7f2188067f20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2194007e80 tx=0x7f2194007390 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:17.262 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.261+0000 7f2199ffb640 1 -- 172.21.15.49:0/767845853 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f21900eb8f0 con 0x7f219c0cb580 2023-11-06T02:09:17.422 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.425+0000 7f21a39ec640 1 -- 172.21.15.49:0/767845853 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f219c0cc530 con 0x7f219c0cb580 2023-11-06T02:09:17.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.425+0000 7f2199ffb640 1 -- 172.21.15.49:0/767845853 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7f21900b1080 con 0x7f219c0cb580 2023-11-06T02:09:17.423 INFO:teuthology.orchestra.run.smithi049.stderr:listed 0 entries 2023-11-06T02:09:17.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.425+0000 7f21a39ec640 1 -- 172.21.15.49:0/767845853 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2188065a60 msgr2=0x7f2188067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:17.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.425+0000 7f21a39ec640 1 --2- 172.21.15.49:0/767845853 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2188065a60 0x7f2188067f20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2194007e80 tx=0x7f2194007390 comp rx=0 tx=0).stop 2023-11-06T02:09:17.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.425+0000 7f21a39ec640 1 -- 172.21.15.49:0/767845853 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f219c0cb580 msgr2=0x7f219c1549e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:17.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.425+0000 7f21a39ec640 1 --2- 172.21.15.49:0/767845853 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f219c0cb580 0x7f219c1549e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f219001aab0 tx=0x7f21900191a0 comp rx=0 tx=0).stop 2023-11-06T02:09:17.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.429+0000 7f21a39ec640 1 -- 172.21.15.49:0/767845853 shutdown_connections 2023-11-06T02:09:17.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.429+0000 7f21a39ec640 1 --2- 172.21.15.49:0/767845853 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2188065a60 0x7f2188067f20 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:17.427 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.429+0000 7f21a39ec640 1 --2- 172.21.15.49:0/767845853 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f219c0cb580 0x7f219c1549e0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:17.427 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.429+0000 7f21a39ec640 1 --2- 172.21.15.49:0/767845853 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f219c15e440 0x7f219c154f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:17.427 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.429+0000 7f21a39ec640 1 --2- 172.21.15.49:0/767845853 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f219c0b6100 0x7f219c15f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:17.427 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.429+0000 7f21a39ec640 1 -- 172.21.15.49:0/767845853 >> 172.21.15.49:0/767845853 conn(0x7f219c0cd2d0 msgr2=0x7f219c157850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:17.427 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.429+0000 7f21a39ec640 1 -- 172.21.15.49:0/767845853 shutdown_connections 2023-11-06T02:09:17.428 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.429+0000 7f21a39ec640 1 -- 172.21.15.49:0/767845853 wait complete. 2023-11-06T02:09:17.440 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi150.front.sepia.ceph.com... 2023-11-06T02:09:17.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:09:17.441 DEBUG:teuthology.orchestra.run.smithi150:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-11-06T02:09:17.471 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T02:09:17.471 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi150.front.sepia.ceph.com 2023-11-06T02:09:17.472 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:09:17.472 DEBUG:teuthology.orchestra.run.smithi150:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-06T02:09:17.527 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-11-06T02:09:17.528 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T02:09:17.533 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-11-06T02:09:17.534 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T02:09:17.582 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T02:09:17.750 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.753+0000 7f94071ce640 1 Processor -- start 2023-11-06T02:09:17.750 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.753+0000 7f94071ce640 1 -- start start 2023-11-06T02:09:17.752 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.753+0000 7f94071ce640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f940015ab50 0x7f94000c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:17.752 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.753+0000 7f94071ce640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f94000b0790 0x7f94000c2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:17.752 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.753+0000 7f94071ce640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94000cab80 con 0x7f94000cb010 2023-11-06T02:09:17.752 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.753+0000 7f94071ce640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f940015af30 con 0x7f940015ab50 2023-11-06T02:09:17.753 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.753+0000 7f94071ce640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94000cb5b0 con 0x7f94000b0790 2023-11-06T02:09:17.753 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.753+0000 7f9404f6a640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f940015ab50 0x7f94000c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:17.753 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.753+0000 7f9404f6a640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f940015ab50 0x7f94000c9bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48524/0 (socket says 172.21.15.49:48524) 2023-11-06T02:09:17.753 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.753+0000 7f9404f6a640 1 -- 172.21.15.49:0/2092957641 learned_addr learned my addr 172.21.15.49:0/2092957641 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:17.753 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.753+0000 7f940576b640 1 --2- 172.21.15.49:0/2092957641 >> v2:172.21.15.150:3300/0 conn(0x7f94000b0790 0x7f94000c2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:17.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.753+0000 7f9404f6a640 1 -- 172.21.15.49:0/2092957641 >> v2:172.21.15.150:3300/0 conn(0x7f94000b0790 msgr2=0x7f94000c2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:17.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.753+0000 7f9404f6a640 1 --2- 172.21.15.49:0/2092957641 >> v2:172.21.15.150:3300/0 conn(0x7f94000b0790 0x7f94000c2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:17.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.753+0000 7f9404f6a640 1 -- 172.21.15.49:0/2092957641 >> v1:172.21.15.49:6789/0 conn(0x7f94000cb010 legacy=0x7f94000b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:17.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.753+0000 7f9404f6a640 1 -- 172.21.15.49:0/2092957641 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94000cb890 con 0x7f940015ab50 2023-11-06T02:09:17.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.757+0000 7f9404f6a640 1 --2- 172.21.15.49:0/2092957641 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f940015ab50 0x7f94000c9bb0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f94000ca5d0 tx=0x7f93f401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c637448e1411d6db server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:17.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.757+0000 7f93ff7fe640 1 -- 172.21.15.49:0/2092957641 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f40aa430 con 0x7f940015ab50 2023-11-06T02:09:17.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.757+0000 7f93ff7fe640 1 -- 172.21.15.49:0/2092957641 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93f40aa590 con 0x7f940015ab50 2023-11-06T02:09:17.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.757+0000 7f93ff7fe640 1 -- 172.21.15.49:0/2092957641 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f40aa920 con 0x7f940015ab50 2023-11-06T02:09:17.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.757+0000 7f94071ce640 1 -- 172.21.15.49:0/2092957641 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f940015ab50 msgr2=0x7f94000c9bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:17.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.757+0000 7f94071ce640 1 --2- 172.21.15.49:0/2092957641 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f940015ab50 0x7f94000c9bb0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f94000ca5d0 tx=0x7f93f401c4d0 comp rx=0 tx=0).stop 2023-11-06T02:09:17.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.757+0000 7f94071ce640 1 -- 172.21.15.49:0/2092957641 shutdown_connections 2023-11-06T02:09:17.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.757+0000 7f94071ce640 1 --2- 172.21.15.49:0/2092957641 >> v2:172.21.15.150:3300/0 conn(0x7f94000b0790 0x7f94000c2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:17.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.757+0000 7f94071ce640 1 --2- 172.21.15.49:0/2092957641 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f940015ab50 0x7f94000c9bb0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:17.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.757+0000 7f94071ce640 1 -- 172.21.15.49:0/2092957641 >> 172.21.15.49:0/2092957641 conn(0x7f940006c0a0 msgr2=0x7f940006c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:17.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.757+0000 7f94071ce640 1 -- 172.21.15.49:0/2092957641 shutdown_connections 2023-11-06T02:09:17.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.757+0000 7f94071ce640 1 -- 172.21.15.49:0/2092957641 wait complete. 2023-11-06T02:09:17.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.757+0000 7f94071ce640 1 Processor -- start 2023-11-06T02:09:17.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.757+0000 7f94071ce640 1 -- start start 2023-11-06T02:09:17.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.757+0000 7f94071ce640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f94000b0790 0x7f94000c6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:17.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.757+0000 7f94071ce640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f94000cb010 0x7f940015c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:17.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.757+0000 7f94071ce640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f940015ab50 0x7f94000c0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:17.762 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.757+0000 7f94071ce640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f940015cc10 con 0x7f94000b0790 2023-11-06T02:09:17.762 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.757+0000 7f94071ce640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94000c6b20 con 0x7f940015ab50 2023-11-06T02:09:17.762 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.757+0000 7f94071ce640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94000c6e00 con 0x7f94000cb010 2023-11-06T02:09:17.762 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.757+0000 7f9404f6a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f94000b0790 0x7f94000c6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:17.763 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.757+0000 7f9404f6a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f94000b0790 0x7f94000c6490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48528/0 (socket says 172.21.15.49:48528) 2023-11-06T02:09:17.763 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.757+0000 7f9404f6a640 1 -- 172.21.15.49:0/862312866 learned_addr learned my addr 172.21.15.49:0/862312866 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:17.763 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.757+0000 7f940576b640 1 --2- 172.21.15.49:0/862312866 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f940015ab50 0x7f94000c0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:17.763 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.757+0000 7f93fffff640 1 --2- 172.21.15.49:0/862312866 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f94000cb010 0x7f940015c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:17.763 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.761+0000 7f9404f6a640 1 -- 172.21.15.49:0/862312866 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f94000cb010 msgr2=0x7f940015c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:17.764 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.761+0000 7f9404f6a640 1 --2- 172.21.15.49:0/862312866 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f94000cb010 0x7f940015c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:17.764 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.761+0000 7f9404f6a640 1 -- 172.21.15.49:0/862312866 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f940015ab50 msgr2=0x7f94000c0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:17.764 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.761+0000 7f9404f6a640 1 --2- 172.21.15.49:0/862312866 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f940015ab50 0x7f94000c0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:17.764 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.761+0000 7f9404f6a640 1 -- 172.21.15.49:0/862312866 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93f4009480 con 0x7f94000b0790 2023-11-06T02:09:17.764 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.761+0000 7f9404f6a640 1 --2- 172.21.15.49:0/862312866 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f94000b0790 0x7f94000c6490 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f93f401a100 tx=0x7f93f4019270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:17.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.761+0000 7f93fd7fa640 1 -- 172.21.15.49:0/862312866 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f40aabd0 con 0x7f94000b0790 2023-11-06T02:09:17.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.761+0000 7f93fd7fa640 1 -- 172.21.15.49:0/862312866 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93f40aad30 con 0x7f94000b0790 2023-11-06T02:09:17.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.761+0000 7f94071ce640 1 -- 172.21.15.49:0/862312866 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94001415d0 con 0x7f94000b0790 2023-11-06T02:09:17.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.761+0000 7f93fd7fa640 1 -- 172.21.15.49:0/862312866 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f40aaea0 con 0x7f94000b0790 2023-11-06T02:09:17.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.761+0000 7f94071ce640 1 -- 172.21.15.49:0/862312866 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94000c70e0 con 0x7f94000b0790 2023-11-06T02:09:17.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.761+0000 7f94071ce640 1 -- 172.21.15.49:0/862312866 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94000b0de0 con 0x7f94000b0790 2023-11-06T02:09:17.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.761+0000 7f93fd7fa640 1 -- 172.21.15.49:0/862312866 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f93f4003620 con 0x7f94000b0790 2023-11-06T02:09:17.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.761+0000 7f93fd7fa640 1 --2- 172.21.15.49:0/862312866 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f93ec065a60 0x7f93ec067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:17.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.761+0000 7f93fd7fa640 1 -- 172.21.15.49:0/862312866 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f93f411dae0 con 0x7f94000b0790 2023-11-06T02:09:17.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.761+0000 7f93fffff640 1 --2- 172.21.15.49:0/862312866 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f93ec065a60 0x7f93ec067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:17.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.761+0000 7f93fffff640 1 --2- 172.21.15.49:0/862312866 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f93ec065a60 0x7f93ec067f20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f94000af190 tx=0x7f93f0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:17.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.765+0000 7f93fd7fa640 1 -- 172.21.15.49:0/862312866 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f93f40eb390 con 0x7f94000b0790 2023-11-06T02:09:17.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.925+0000 7f94071ce640 1 -- 172.21.15.49:0/862312866 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9400141af0 con 0x7f94000b0790 2023-11-06T02:09:17.924 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.925+0000 7f93fd7fa640 1 -- 172.21.15.49:0/862312866 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17906 (secure 0 0 0) 0x7f93f40c1090 con 0x7f94000b0790 2023-11-06T02:09:17.925 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:09:17.925 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":25,"fsid":"1a8b035c-6ba4-499a-b876-aea47fd9c0d4","created":"2023-11-06T02:08:06.440857+0000","modified":"2023-11-06T02:09:13.195118+0000","last_up_change":"2023-11-06T02:08:17.982372+0000","last_in_change":"2023-11-06T02:08:08.906631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T02:08:19.252222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T02:08:24.776211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-06T02:08:28.936178+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-06T02:08:29.563468+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ad55af8-0571-4e6a-8722-b5d429b7496d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6809","nonce":38116},{"type":"v1","addr":"172.21.15.49:6811","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6813","nonce":38116},{"type":"v1","addr":"172.21.15.49:6815","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6821","nonce":38116},{"type":"v1","addr":"172.21.15.49:6823","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6817","nonce":38116},{"type":"v1","addr":"172.21.15.49:6819","nonce":38116}]},"public_addr":"172.21.15.49:6811/38116","cluster_addr":"172.21.15.49:6815/38116","heartbeat_back_addr":"172.21.15.49:6823/38116","heartbeat_front_addr":"172.21.15.49:6819/38116","state":["exists","up"]},{"osd":1,"uuid":"c8b31f85-a757-43ed-9419-351acc46611b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":38113},{"type":"v1","addr":"172.21.15.49:6810","nonce":38113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":38113},{"type":"v1","addr":"172.21.15.49:6814","nonce":38113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":38113},{"type":"v1","addr":"172.21.15.49:6822","nonce":38113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":38113},{"type":"v1","addr":"172.21.15.49:6818","nonce":38113}]},"public_addr":"172.21.15.49:6810/38113","cluster_addr":"172.21.15.49:6814/38113","heartbeat_back_addr":"172.21.15.49:6822/38113","heartbeat_front_addr":"172.21.15.49:6818/38113","state":["exists","up"]},{"osd":2,"uuid":"555b3bf3-0a70-4b91-8efd-e4204b5fc26e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":38114},{"type":"v1","addr":"172.21.15.49:6801","nonce":38114}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":38114},{"type":"v1","addr":"172.21.15.49:6803","nonce":38114}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":38114},{"type":"v1","addr":"172.21.15.49:6807","nonce":38114}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":38114},{"type":"v1","addr":"172.21.15.49:6805","nonce":38114}]},"public_addr":"172.21.15.49:6801/38114","cluster_addr":"172.21.15.49:6803/38114","heartbeat_back_addr":"172.21.15.49:6807/38114","heartbeat_front_addr":"172.21.15.49:6805/38114","state":["exists","up"]},{"osd":3,"uuid":"8d4e5d77-03ef-4b9b-bfcb-5ae93a46795c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":38115},{"type":"v1","addr":"172.21.15.49:6825","nonce":38115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":38115},{"type":"v1","addr":"172.21.15.49:6827","nonce":38115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":38115},{"type":"v1","addr":"172.21.15.49:6831","nonce":38115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":38115},{"type":"v1","addr":"172.21.15.49:6829","nonce":38115}]},"public_addr":"172.21.15.49:6825/38115","cluster_addr":"172.21.15.49:6827/38115","heartbeat_back_addr":"172.21.15.49:6831/38115","heartbeat_front_addr":"172.21.15.49:6829/38115","state":["exists","up"]},{"osd":4,"uuid":"dba03ad4-68a3-4cf3-9218-ac611f02eacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":38417},{"type":"v1","addr":"172.21.15.150:6809","nonce":38417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":38417},{"type":"v1","addr":"172.21.15.150:6811","nonce":38417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":38417},{"type":"v1","addr":"172.21.15.150:6815","nonce":38417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":38417},{"type":"v1","addr":"172.21.15.150:6813","nonce":38417}]},"public_addr":"172.21.15.150:6809/38417","cluster_addr":"172.21.15.150:6811/38417","heartbeat_back_addr":"172.21.15.150:6815/38417","heartbeat_front_addr":"172.21.15.150:6813/38417","state":["exists","up"]},{"osd":5,"uuid":"0df9c303-c255-4640-8e91-01f1c3efe18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":38418},{"type":"v1","addr":"172.21.15.150:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":38418},{"type":"v1","addr":"172.21.15.150:6803","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":38418},{"type":"v1","addr":"172.21.15.150:6807","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":38418},{"type":"v1","addr":"172.21.15.150:6805","nonce":38418}]},"public_addr":"172.21.15.150:6801/38418","cluster_addr":"172.21.15.150:6803/38418","heartbeat_back_addr":"172.21.15.150:6807/38418","heartbeat_front_addr":"172.21.15.150:6805/38418","state":["exists","up"]},{"osd":6,"uuid":"e59fa189-4b70-4c06-9722-c02b90828bf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":38428},{"type":"v1","addr":"172.21.15.150:6825","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":38428},{"type":"v1","addr":"172.21.15.150:6827","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":38428},{"type":"v1","addr":"172.21.15.150:6831","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":38428},{"type":"v1","addr":"172.21.15.150:6829","nonce":38428}]},"public_addr":"172.21.15.150:6825/38428","cluster_addr":"172.21.15.150:6827/38428","heartbeat_back_addr":"172.21.15.150:6831/38428","heartbeat_front_addr":"172.21.15.150:6829/38428","state":["exists","up"]},{"osd":7,"uuid":"a64f57e1-ef98-46bd-b304-385759e95898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":38421},{"type":"v1","addr":"172.21.15.150:6817","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":38421},{"type":"v1","addr":"172.21.15.150:6819","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":38421},{"type":"v1","addr":"172.21.15.150:6823","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":38421},{"type":"v1","addr":"172.21.15.150:6821","nonce":38421}]},"public_addr":"172.21.15.150:6817/38421","cluster_addr":"172.21.15.150:6819/38421","heartbeat_back_addr":"172.21.15.150:6823/38421","heartbeat_front_addr":"172.21.15.150:6821/38421","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.438552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.413280+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.296466+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.424966+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.276953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.230984+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.410235+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.480084+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.14","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T02:09:17.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.929+0000 7f94071ce640 1 -- 172.21.15.49:0/862312866 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f93ec065a60 msgr2=0x7f93ec067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:17.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.929+0000 7f94071ce640 1 --2- 172.21.15.49:0/862312866 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f93ec065a60 0x7f93ec067f20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f94000af190 tx=0x7f93f0009040 comp rx=0 tx=0).stop 2023-11-06T02:09:17.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.929+0000 7f94071ce640 1 -- 172.21.15.49:0/862312866 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f94000b0790 msgr2=0x7f94000c6490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:17.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.929+0000 7f94071ce640 1 --2- 172.21.15.49:0/862312866 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f94000b0790 0x7f94000c6490 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f93f401a100 tx=0x7f93f4019270 comp rx=0 tx=0).stop 2023-11-06T02:09:17.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.929+0000 7f94071ce640 1 -- 172.21.15.49:0/862312866 shutdown_connections 2023-11-06T02:09:17.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.929+0000 7f94071ce640 1 --2- 172.21.15.49:0/862312866 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f93ec065a60 0x7f93ec067f20 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:17.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.929+0000 7f94071ce640 1 --2- 172.21.15.49:0/862312866 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f940015ab50 0x7f94000c0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:17.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.929+0000 7f94071ce640 1 --2- 172.21.15.49:0/862312866 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f94000cb010 0x7f940015c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:17.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.929+0000 7f94071ce640 1 --2- 172.21.15.49:0/862312866 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f94000b0790 0x7f94000c6490 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:17.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.929+0000 7f94071ce640 1 -- 172.21.15.49:0/862312866 >> 172.21.15.49:0/862312866 conn(0x7f940006c0a0 msgr2=0x7f94000c39a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:17.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.929+0000 7f94071ce640 1 -- 172.21.15.49:0/862312866 shutdown_connections 2023-11-06T02:09:17.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:17.929+0000 7f94071ce640 1 -- 172.21.15.49:0/862312866 wait complete. 2023-11-06T02:09:17.944 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T02:08:19.252222+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T02:08:24.776211+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T02:08:28.936178+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T02:08:29.563468+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T02:09:17.945 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T02:09:18.069 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.073+0000 7efd3ea00640 1 Processor -- start 2023-11-06T02:09:18.070 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.073+0000 7efd3ea00640 1 -- start start 2023-11-06T02:09:18.071 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.073+0000 7efd3ea00640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7efd38070870 0x7efd380b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:18.071 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.073+0000 7efd3ea00640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7efd3806d150 0x7efd3806d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:18.071 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.073+0000 7efd3ea00640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd3806db40 con 0x7efd3815a880 2023-11-06T02:09:18.072 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.073+0000 7efd3ea00640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd380ae360 con 0x7efd3806d150 2023-11-06T02:09:18.072 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.073+0000 7efd3ea00640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd380ae640 con 0x7efd38070870 2023-11-06T02:09:18.072 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.073+0000 7efd3cf9d640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7efd3806d150 0x7efd3806d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:18.072 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.073+0000 7efd3cf9d640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7efd3806d150 0x7efd3806d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48530/0 (socket says 172.21.15.49:48530) 2023-11-06T02:09:18.072 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.073+0000 7efd3cf9d640 1 -- 172.21.15.49:0/2527453117 learned_addr learned my addr 172.21.15.49:0/2527453117 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:18.073 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.073+0000 7efd2f7fe640 1 --2- 172.21.15.49:0/2527453117 >> v2:172.21.15.150:3300/0 conn(0x7efd38070870 0x7efd380b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:18.073 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.073+0000 7efd3cf9d640 1 -- 172.21.15.49:0/2527453117 >> v2:172.21.15.150:3300/0 conn(0x7efd38070870 msgr2=0x7efd380b15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:18.073 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.073+0000 7efd3cf9d640 1 --2- 172.21.15.49:0/2527453117 >> v2:172.21.15.150:3300/0 conn(0x7efd38070870 0x7efd380b15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:18.073 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.073+0000 7efd3cf9d640 1 -- 172.21.15.49:0/2527453117 >> v1:172.21.15.150:6789/0 conn(0x7efd3815a880 legacy=0x7efd38070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:18.073 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.073+0000 7efd3cf9d640 1 -- 172.21.15.49:0/2527453117 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd380ae920 con 0x7efd3806d150 2023-11-06T02:09:18.075 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.077+0000 7efd3cf9d640 1 --2- 172.21.15.49:0/2527453117 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd3806d150 0x7efd3806d600 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7efd38070ec0 tx=0x7efd3001c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=166b6d3273eec0bf server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:18.075 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.077+0000 7efd377fe640 1 -- 172.21.15.49:0/2527453117 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd300a56d0 con 0x7efd3806d150 2023-11-06T02:09:18.075 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.077+0000 7efd377fe640 1 -- 172.21.15.49:0/2527453117 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd300a5830 con 0x7efd3806d150 2023-11-06T02:09:18.075 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.077+0000 7efd377fe640 1 -- 172.21.15.49:0/2527453117 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd300a5bc0 con 0x7efd3806d150 2023-11-06T02:09:18.076 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.077+0000 7efd3ea00640 1 -- 172.21.15.49:0/2527453117 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd3806d150 msgr2=0x7efd3806d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:18.076 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.077+0000 7efd3ea00640 1 --2- 172.21.15.49:0/2527453117 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd3806d150 0x7efd3806d600 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7efd38070ec0 tx=0x7efd3001c900 comp rx=0 tx=0).stop 2023-11-06T02:09:18.076 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.077+0000 7efd3ea00640 1 -- 172.21.15.49:0/2527453117 shutdown_connections 2023-11-06T02:09:18.076 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.077+0000 7efd3ea00640 1 --2- 172.21.15.49:0/2527453117 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd3806d150 0x7efd3806d600 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:18.076 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.077+0000 7efd3ea00640 1 --2- 172.21.15.49:0/2527453117 >> v2:172.21.15.150:3300/0 conn(0x7efd38070870 0x7efd380b15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:18.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.077+0000 7efd3ea00640 1 -- 172.21.15.49:0/2527453117 >> 172.21.15.49:0/2527453117 conn(0x7efd3806a370 msgr2=0x7efd380ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:18.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.077+0000 7efd3ea00640 1 -- 172.21.15.49:0/2527453117 shutdown_connections 2023-11-06T02:09:18.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.077+0000 7efd3ea00640 1 -- 172.21.15.49:0/2527453117 wait complete. 2023-11-06T02:09:18.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.077+0000 7efd3ea00640 1 Processor -- start 2023-11-06T02:09:18.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.077+0000 7efd3ea00640 1 -- start start 2023-11-06T02:09:18.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.077+0000 7efd3ea00640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efd3806d150 0x7efd38150350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:18.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.077+0000 7efd3ea00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd38070870 0x7efd381523b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:18.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.077+0000 7efd3ea00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd3815a880 0x7efd38154e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:18.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.077+0000 7efd3ea00640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd38150890 con 0x7efd38070870 2023-11-06T02:09:18.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.077+0000 7efd3ea00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd3814f320 con 0x7efd3815a880 2023-11-06T02:09:18.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.077+0000 7efd3ea00640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd3814f600 con 0x7efd3806d150 2023-11-06T02:09:18.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.077+0000 7efd2f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd38070870 0x7efd381523b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:18.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.077+0000 7efd2f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd38070870 0x7efd381523b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48544/0 (socket says 172.21.15.49:48544) 2023-11-06T02:09:18.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.077+0000 7efd2f7fe640 1 -- 172.21.15.49:0/3300274456 learned_addr learned my addr 172.21.15.49:0/3300274456 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:18.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.077+0000 7efd3cf9d640 1 --2- 172.21.15.49:0/3300274456 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd3815a880 0x7efd38154e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:18.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.077+0000 7efd37fff640 1 --2- 172.21.15.49:0/3300274456 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efd3806d150 0x7efd38150350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:18.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.077+0000 7efd2f7fe640 1 -- 172.21.15.49:0/3300274456 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efd3806d150 msgr2=0x7efd38150350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:18.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.077+0000 7efd2f7fe640 1 --2- 172.21.15.49:0/3300274456 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efd3806d150 0x7efd38150350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:18.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.077+0000 7efd2f7fe640 1 -- 172.21.15.49:0/3300274456 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd3815a880 msgr2=0x7efd38154e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:18.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.077+0000 7efd2f7fe640 1 --2- 172.21.15.49:0/3300274456 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd3815a880 0x7efd38154e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:18.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.077+0000 7efd2f7fe640 1 -- 172.21.15.49:0/3300274456 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd30009b30 con 0x7efd38070870 2023-11-06T02:09:18.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.077+0000 7efd2f7fe640 1 --2- 172.21.15.49:0/3300274456 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd38070870 0x7efd381523b0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7efd2000d800 tx=0x7efd2000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:18.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.077+0000 7efd357fa640 1 -- 172.21.15.49:0/3300274456 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd2000b6c0 con 0x7efd38070870 2023-11-06T02:09:18.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.077+0000 7efd357fa640 1 -- 172.21.15.49:0/3300274456 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd2000b820 con 0x7efd38070870 2023-11-06T02:09:18.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.077+0000 7efd3ea00640 1 -- 172.21.15.49:0/3300274456 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd3814f8e0 con 0x7efd38070870 2023-11-06T02:09:18.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.077+0000 7efd357fa640 1 -- 172.21.15.49:0/3300274456 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd2000bb90 con 0x7efd38070870 2023-11-06T02:09:18.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.081+0000 7efd3ea00640 1 -- 172.21.15.49:0/3300274456 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd38154950 con 0x7efd38070870 2023-11-06T02:09:18.085 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.081+0000 7efd3ea00640 1 -- 172.21.15.49:0/3300274456 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd3814fad0 con 0x7efd38070870 2023-11-06T02:09:18.085 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.081+0000 7efd357fa640 1 -- 172.21.15.49:0/3300274456 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7efd2000bcf0 con 0x7efd38070870 2023-11-06T02:09:18.085 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.081+0000 7efd357fa640 1 --2- 172.21.15.49:0/3300274456 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7efd18065a60 0x7efd18067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:18.085 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.081+0000 7efd357fa640 1 -- 172.21.15.49:0/3300274456 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7efd200838c0 con 0x7efd38070870 2023-11-06T02:09:18.085 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.081+0000 7efd37fff640 1 --2- 172.21.15.49:0/3300274456 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7efd18065a60 0x7efd18067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:18.085 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.081+0000 7efd37fff640 1 --2- 172.21.15.49:0/3300274456 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7efd18065a60 0x7efd18067f20 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7efd380ce8d0 tx=0x7efd28009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:18.086 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.085+0000 7efd357fa640 1 -- 172.21.15.49:0/3300274456 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7efd200510e0 con 0x7efd38070870 2023-11-06T02:09:18.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.241+0000 7efd3ea00640 1 -- 172.21.15.49:0/3300274456 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efd38074040 con 0x7efd38070870 2023-11-06T02:09:18.242 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.245+0000 7efd357fa640 1 -- 172.21.15.49:0/3300274456 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7efd20011340 con 0x7efd38070870 2023-11-06T02:09:18.242 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2023-11-06T02:09:18.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.245+0000 7efd3ea00640 1 -- 172.21.15.49:0/3300274456 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7efd18065a60 msgr2=0x7efd18067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:18.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.245+0000 7efd3ea00640 1 --2- 172.21.15.49:0/3300274456 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7efd18065a60 0x7efd18067f20 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7efd380ce8d0 tx=0x7efd28009040 comp rx=0 tx=0).stop 2023-11-06T02:09:18.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.245+0000 7efd3ea00640 1 -- 172.21.15.49:0/3300274456 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd38070870 msgr2=0x7efd381523b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:18.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.245+0000 7efd3ea00640 1 --2- 172.21.15.49:0/3300274456 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd38070870 0x7efd381523b0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7efd2000d800 tx=0x7efd2000dcc0 comp rx=0 tx=0).stop 2023-11-06T02:09:18.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.245+0000 7efd3ea00640 1 -- 172.21.15.49:0/3300274456 shutdown_connections 2023-11-06T02:09:18.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.245+0000 7efd3ea00640 1 --2- 172.21.15.49:0/3300274456 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7efd18065a60 0x7efd18067f20 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:18.246 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.245+0000 7efd3ea00640 1 --2- 172.21.15.49:0/3300274456 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd38070870 0x7efd381523b0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:18.246 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.245+0000 7efd3ea00640 1 --2- 172.21.15.49:0/3300274456 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd3815a880 0x7efd38154e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:18.246 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.245+0000 7efd3ea00640 1 --2- 172.21.15.49:0/3300274456 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efd3806d150 0x7efd38150350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:18.246 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.245+0000 7efd3ea00640 1 -- 172.21.15.49:0/3300274456 >> 172.21.15.49:0/3300274456 conn(0x7efd3806a370 msgr2=0x7efd380b3700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:18.246 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.245+0000 7efd3ea00640 1 -- 172.21.15.49:0/3300274456 shutdown_connections 2023-11-06T02:09:18.247 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.245+0000 7efd3ea00640 1 -- 172.21.15.49:0/3300274456 wait complete. 2023-11-06T02:09:18.260 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T02:09:18.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.381+0000 7f6d13fcc640 1 Processor -- start 2023-11-06T02:09:18.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.381+0000 7f6d13fcc640 1 -- start start 2023-11-06T02:09:18.382 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.385+0000 7f6d13fcc640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f6d0c1595a0 0x7f6d0c15dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:18.382 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.385+0000 7f6d13fcc640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6d0c06d640 0x7f6d0c0b0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:18.382 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.385+0000 7f6d13fcc640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d0c06c5a0 con 0x7f6d0c0b0760 2023-11-06T02:09:18.382 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.385+0000 7f6d13fcc640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d0c15e250 con 0x7f6d0c1595a0 2023-11-06T02:09:18.383 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.385+0000 7f6d13fcc640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d0c06c880 con 0x7f6d0c06d640 2023-11-06T02:09:18.383 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.385+0000 7f6d11d68640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f6d0c1595a0 0x7f6d0c15dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:18.383 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.385+0000 7f6d11d68640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f6d0c1595a0 0x7f6d0c15dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48554/0 (socket says 172.21.15.49:48554) 2023-11-06T02:09:18.383 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.385+0000 7f6d11d68640 1 -- 172.21.15.49:0/71341067 learned_addr learned my addr 172.21.15.49:0/71341067 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:18.383 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.385+0000 7f6d11d68640 1 -- 172.21.15.49:0/71341067 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6d0c06d640 msgr2=0x7f6d0c0b0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:18.384 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.385+0000 7f6d11d68640 1 --2- 172.21.15.49:0/71341067 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6d0c06d640 0x7f6d0c0b0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:18.384 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.385+0000 7f6d11d68640 1 -- 172.21.15.49:0/71341067 >> v1:172.21.15.150:6789/0 conn(0x7f6d0c0b0760 legacy=0x7f6d0c0c8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:18.384 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.385+0000 7f6d11d68640 1 -- 172.21.15.49:0/71341067 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d0c06cb60 con 0x7f6d0c1595a0 2023-11-06T02:09:18.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.385+0000 7f6d11d68640 1 --2- 172.21.15.49:0/71341067 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d0c1595a0 0x7f6d0c15dd10 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f6d0400bb10 tx=0x7f6d0401eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76bd952bd62464c3 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:18.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.385+0000 7f6d10d66640 1 -- 172.21.15.49:0/71341067 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d040a7b30 con 0x7f6d0c1595a0 2023-11-06T02:09:18.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.385+0000 7f6d10d66640 1 -- 172.21.15.49:0/71341067 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d040a7c90 con 0x7f6d0c1595a0 2023-11-06T02:09:18.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.385+0000 7f6d10d66640 1 -- 172.21.15.49:0/71341067 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d040af440 con 0x7f6d0c1595a0 2023-11-06T02:09:18.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.385+0000 7f6d13fcc640 1 -- 172.21.15.49:0/71341067 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d0c1595a0 msgr2=0x7f6d0c15dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:18.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.385+0000 7f6d13fcc640 1 --2- 172.21.15.49:0/71341067 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d0c1595a0 0x7f6d0c15dd10 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f6d0400bb10 tx=0x7f6d0401eba0 comp rx=0 tx=0).stop 2023-11-06T02:09:18.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.385+0000 7f6d13fcc640 1 -- 172.21.15.49:0/71341067 shutdown_connections 2023-11-06T02:09:18.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.385+0000 7f6d13fcc640 1 --2- 172.21.15.49:0/71341067 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6d0c06d640 0x7f6d0c0b0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:18.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.385+0000 7f6d13fcc640 1 --2- 172.21.15.49:0/71341067 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d0c1595a0 0x7f6d0c15dd10 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:18.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.385+0000 7f6d13fcc640 1 -- 172.21.15.49:0/71341067 >> 172.21.15.49:0/71341067 conn(0x7f6d0c06a370 msgr2=0x7f6d0c0ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:18.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.385+0000 7f6d13fcc640 1 -- 172.21.15.49:0/71341067 shutdown_connections 2023-11-06T02:09:18.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.385+0000 7f6d13fcc640 1 -- 172.21.15.49:0/71341067 wait complete. 2023-11-06T02:09:18.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.389+0000 7f6d13fcc640 1 Processor -- start 2023-11-06T02:09:18.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.389+0000 7f6d13fcc640 1 -- start start 2023-11-06T02:09:18.391 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.389+0000 7f6d13fcc640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6d0c06d640 0x7f6d0c0c7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:18.391 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.389+0000 7f6d13fcc640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d0c0b0760 0x7f6d0c13bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:18.391 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.389+0000 7f6d13fcc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d0c1595a0 0x7f6d0c13c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:18.391 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.389+0000 7f6d13fcc640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d0c1526f0 con 0x7f6d0c1595a0 2023-11-06T02:09:18.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.389+0000 7f6d13fcc640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d0c13c920 con 0x7f6d0c0b0760 2023-11-06T02:09:18.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.389+0000 7f6d13fcc640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d0c13cc00 con 0x7f6d0c06d640 2023-11-06T02:09:18.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.389+0000 7f6d12569640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d0c1595a0 0x7f6d0c13c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:18.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.389+0000 7f6d12569640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d0c1595a0 0x7f6d0c13c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48558/0 (socket says 172.21.15.49:48558) 2023-11-06T02:09:18.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.389+0000 7f6d12569640 1 -- 172.21.15.49:0/248731334 learned_addr learned my addr 172.21.15.49:0/248731334 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:18.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.389+0000 7f6d11d68640 1 --2- 172.21.15.49:0/248731334 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6d0c06d640 0x7f6d0c0c7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:18.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.389+0000 7f6d11567640 1 --2- 172.21.15.49:0/248731334 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d0c0b0760 0x7f6d0c13bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:18.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.389+0000 7f6d12569640 1 -- 172.21.15.49:0/248731334 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6d0c06d640 msgr2=0x7f6d0c0c7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:18.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.389+0000 7f6d12569640 1 --2- 172.21.15.49:0/248731334 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6d0c06d640 0x7f6d0c0c7dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:18.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.389+0000 7f6d12569640 1 -- 172.21.15.49:0/248731334 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d0c0b0760 msgr2=0x7f6d0c13bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:18.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.389+0000 7f6d12569640 1 --2- 172.21.15.49:0/248731334 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d0c0b0760 0x7f6d0c13bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:18.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.389+0000 7f6d12569640 1 -- 172.21.15.49:0/248731334 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d0400b770 con 0x7f6d0c1595a0 2023-11-06T02:09:18.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.389+0000 7f6d11567640 1 --2- 172.21.15.49:0/248731334 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d0c0b0760 0x7f6d0c13bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:09:18.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.389+0000 7f6d12569640 1 --2- 172.21.15.49:0/248731334 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d0c1595a0 0x7f6d0c13c280 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc00c780 tx=0x7f6cfc00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:18.395 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.389+0000 7f6d0a7fc640 1 -- 172.21.15.49:0/248731334 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cfc004b70 con 0x7f6d0c1595a0 2023-11-06T02:09:18.395 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.389+0000 7f6d0a7fc640 1 -- 172.21.15.49:0/248731334 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cfc00ce90 con 0x7f6d0c1595a0 2023-11-06T02:09:18.395 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.389+0000 7f6d13fcc640 1 -- 172.21.15.49:0/248731334 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d0c13cee0 con 0x7f6d0c1595a0 2023-11-06T02:09:18.395 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.389+0000 7f6d0a7fc640 1 -- 172.21.15.49:0/248731334 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cfc00a850 con 0x7f6d0c1595a0 2023-11-06T02:09:18.395 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.389+0000 7f6d13fcc640 1 -- 172.21.15.49:0/248731334 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d0c13d4a0 con 0x7f6d0c1595a0 2023-11-06T02:09:18.395 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.389+0000 7f6d13fcc640 1 -- 172.21.15.49:0/248731334 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d0c13d6c0 con 0x7f6d0c1595a0 2023-11-06T02:09:18.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.389+0000 7f6d0a7fc640 1 -- 172.21.15.49:0/248731334 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f6cfc00a9b0 con 0x7f6d0c1595a0 2023-11-06T02:09:18.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.389+0000 7f6d0a7fc640 1 --2- 172.21.15.49:0/248731334 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6cf8065a60 0x7f6cf8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:18.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.389+0000 7f6d0a7fc640 1 -- 172.21.15.49:0/248731334 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f6cfc0834a0 con 0x7f6d0c1595a0 2023-11-06T02:09:18.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.393+0000 7f6d11d68640 1 --2- 172.21.15.49:0/248731334 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6cf8065a60 0x7f6cf8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:18.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.393+0000 7f6d11d68640 1 --2- 172.21.15.49:0/248731334 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6cf8065a60 0x7f6cf8067f20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6d040b1000 tx=0x7f6d040a9070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:18.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.393+0000 7f6d0a7fc640 1 -- 172.21.15.49:0/248731334 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f6cfc050d50 con 0x7f6d0c1595a0 2023-11-06T02:09:18.542 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.545+0000 7f6d13fcc640 1 -- 172.21.15.49:0/248731334 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f6d0c074040 con 0x7f6d0c1595a0 2023-11-06T02:09:18.543 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.545+0000 7f6d0a7fc640 1 -- 172.21.15.49:0/248731334 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7f6cfc054370 con 0x7f6d0c1595a0 2023-11-06T02:09:18.543 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 8 2023-11-06T02:09:18.545 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.545+0000 7f6d13fcc640 1 -- 172.21.15.49:0/248731334 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6cf8065a60 msgr2=0x7f6cf8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:18.545 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.545+0000 7f6d13fcc640 1 --2- 172.21.15.49:0/248731334 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6cf8065a60 0x7f6cf8067f20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6d040b1000 tx=0x7f6d040a9070 comp rx=0 tx=0).stop 2023-11-06T02:09:18.546 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.545+0000 7f6d13fcc640 1 -- 172.21.15.49:0/248731334 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d0c1595a0 msgr2=0x7f6d0c13c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:18.546 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.545+0000 7f6d13fcc640 1 --2- 172.21.15.49:0/248731334 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d0c1595a0 0x7f6d0c13c280 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc00c780 tx=0x7f6cfc00cc40 comp rx=0 tx=0).stop 2023-11-06T02:09:18.546 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.545+0000 7f6d13fcc640 1 -- 172.21.15.49:0/248731334 shutdown_connections 2023-11-06T02:09:18.546 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.545+0000 7f6d13fcc640 1 --2- 172.21.15.49:0/248731334 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6cf8065a60 0x7f6cf8067f20 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:18.546 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.545+0000 7f6d13fcc640 1 --2- 172.21.15.49:0/248731334 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d0c1595a0 0x7f6d0c13c280 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:18.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.545+0000 7f6d13fcc640 1 --2- 172.21.15.49:0/248731334 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d0c0b0760 0x7f6d0c13bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:18.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.545+0000 7f6d13fcc640 1 --2- 172.21.15.49:0/248731334 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6d0c06d640 0x7f6d0c0c7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:18.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.545+0000 7f6d13fcc640 1 -- 172.21.15.49:0/248731334 >> 172.21.15.49:0/248731334 conn(0x7f6d0c06a370 msgr2=0x7f6d0c0cd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:18.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.545+0000 7f6d13fcc640 1 -- 172.21.15.49:0/248731334 shutdown_connections 2023-11-06T02:09:18.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.545+0000 7f6d13fcc640 1 -- 172.21.15.49:0/248731334 wait complete. 2023-11-06T02:09:18.560 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T02:09:18.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.689+0000 7f8924b96640 1 Processor -- start 2023-11-06T02:09:18.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.693+0000 7f8924b96640 1 -- start start 2023-11-06T02:09:18.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.693+0000 7f8924b96640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8920070870 0x7f892006c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:18.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.693+0000 7f8924b96640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f89200c8790 0x7f892006cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:18.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.693+0000 7f8924b96640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89200b00a0 con 0x7f8920161260 2023-11-06T02:09:18.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.693+0000 7f8924b96640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f892006c580 con 0x7f89200c8790 2023-11-06T02:09:18.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.693+0000 7f8924b96640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89200c91f0 con 0x7f8920070870 2023-11-06T02:09:18.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.693+0000 7f891e59c640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f8920161260 0x7f8920070160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:35440/0 (socket says 172.21.15.49:35440) 2023-11-06T02:09:18.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.693+0000 7f891e59c640 1 -- 172.21.15.49:0/3685867963 learned_addr learned my addr 172.21.15.49:0/3685867963 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:09:18.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.693+0000 7f891ed9d640 1 --2- 172.21.15.49:0/3685867963 >> v2:172.21.15.150:3300/0 conn(0x7f89200c8790 0x7f892006cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:18.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.693+0000 7f8915d9b640 1 --2- 172.21.15.49:0/3685867963 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8920070870 0x7f892006c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:18.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.693+0000 7f891dd9b640 1 -- 172.21.15.49:0/3685867963 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 318144777 0 0) 0x7f89200b00a0 con 0x7f8920161260 2023-11-06T02:09:18.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.693+0000 7f891dd9b640 1 -- 172.21.15.49:0/3685867963 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f890c003530 con 0x7f8920161260 2023-11-06T02:09:18.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.693+0000 7f891dd9b640 1 -- 172.21.15.49:0/3685867963 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3975607256 0 0) 0x7f890c003530 con 0x7f8920161260 2023-11-06T02:09:18.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.693+0000 7f891dd9b640 1 -- 172.21.15.49:0/3685867963 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8920070870 msgr2=0x7f892006c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:18.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.693+0000 7f891dd9b640 1 --2- 172.21.15.49:0/3685867963 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8920070870 0x7f892006c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:18.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.693+0000 7f891dd9b640 1 -- 172.21.15.49:0/3685867963 >> v2:172.21.15.150:3300/0 conn(0x7f89200c8790 msgr2=0x7f892006cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:18.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.693+0000 7f891dd9b640 1 --2- 172.21.15.49:0/3685867963 >> v2:172.21.15.150:3300/0 conn(0x7f89200c8790 0x7f892006cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:18.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.693+0000 7f891dd9b640 1 -- 172.21.15.49:0/3685867963 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f892006c230 con 0x7f8920161260 2023-11-06T02:09:18.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.693+0000 7f891dd9b640 1 -- 172.21.15.49:0/3685867963 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7f8910001680 con 0x7f8920161260 2023-11-06T02:09:18.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.693+0000 7f891dd9b640 1 -- 172.21.15.49:0/3685867963 >> v1:172.21.15.49:6789/0 conn(0x7f8920161260 legacy=0x7f8920070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:18.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.693+0000 7f891dd9b640 1 --2- 172.21.15.49:0/3685867963 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f890c003980 0x7f890c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:18.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.693+0000 7f891dd9b640 1 --2- 172.21.15.49:0/3685867963 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f890c0047d0 0x7f890c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:18.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.693+0000 7f891dd9b640 1 --2- 172.21.15.49:0/3685867963 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8920070870 0x7f890c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:18.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.693+0000 7f891dd9b640 1 -- 172.21.15.49:0/3685867963 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f892006c580 con 0x7f890c0047d0 2023-11-06T02:09:18.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.693+0000 7f891dd9b640 1 -- 172.21.15.49:0/3685867963 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89200c91f0 con 0x7f890c003980 2023-11-06T02:09:18.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.693+0000 7f891dd9b640 1 -- 172.21.15.49:0/3685867963 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f890c00ba10 con 0x7f8920070870 2023-11-06T02:09:18.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.693+0000 7f8915d9b640 1 --2- 172.21.15.49:0/3685867963 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f890c0047d0 0x7f890c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:18.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.693+0000 7f891e59c640 1 --2- 172.21.15.49:0/3685867963 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f890c003980 0x7f890c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:18.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.693+0000 7f891ed9d640 1 --2- 172.21.15.49:0/3685867963 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8920070870 0x7f890c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:18.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.693+0000 7f8915d9b640 1 -- 172.21.15.49:0/3685867963 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8920070870 msgr2=0x7f890c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:18.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.693+0000 7f8915d9b640 1 --2- 172.21.15.49:0/3685867963 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8920070870 0x7f890c005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:18.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.693+0000 7f8915d9b640 1 -- 172.21.15.49:0/3685867963 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f890c003980 msgr2=0x7f890c003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:18.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.693+0000 7f8915d9b640 1 --2- 172.21.15.49:0/3685867963 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f890c003980 0x7f890c003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:18.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.693+0000 7f8915d9b640 1 -- 172.21.15.49:0/3685867963 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f890c003530 con 0x7f890c0047d0 2023-11-06T02:09:18.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.693+0000 7f891ed9d640 1 --2- 172.21.15.49:0/3685867963 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8920070870 0x7f890c005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:09:18.702 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.697+0000 7f8915d9b640 1 --2- 172.21.15.49:0/3685867963 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f890c0047d0 0x7f890c004c50 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f8908009ae0 tx=0x7f890801ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eb459ae2a079faf server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:18.702 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.697+0000 7f891dd9b640 1 -- 172.21.15.49:0/3685867963 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89080a5530 con 0x7f890c0047d0 2023-11-06T02:09:18.703 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.697+0000 7f891dd9b640 1 -- 172.21.15.49:0/3685867963 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89080a5690 con 0x7f890c0047d0 2023-11-06T02:09:18.703 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.697+0000 7f891dd9b640 1 -- 172.21.15.49:0/3685867963 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89080a5a40 con 0x7f890c0047d0 2023-11-06T02:09:18.703 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.697+0000 7f8924b96640 1 -- 172.21.15.49:0/3685867963 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f890c0047d0 msgr2=0x7f890c004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:18.703 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.697+0000 7f8924b96640 1 --2- 172.21.15.49:0/3685867963 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f890c0047d0 0x7f890c004c50 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f8908009ae0 tx=0x7f890801ca50 comp rx=0 tx=0).stop 2023-11-06T02:09:18.703 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.697+0000 7f891ed9d640 1 -- 172.21.15.49:0/3685867963 reap_dead start 2023-11-06T02:09:18.704 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.697+0000 7f8924b96640 1 -- 172.21.15.49:0/3685867963 shutdown_connections 2023-11-06T02:09:18.704 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.697+0000 7f8924b96640 1 -- 172.21.15.49:0/3685867963 >> 172.21.15.49:0/3685867963 conn(0x7f892006a370 msgr2=0x7f89200ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:18.704 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.697+0000 7f8924b96640 1 -- 172.21.15.49:0/3685867963 shutdown_connections 2023-11-06T02:09:18.704 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.697+0000 7f8924b96640 1 -- 172.21.15.49:0/3685867963 wait complete. 2023-11-06T02:09:18.704 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.697+0000 7f8924b96640 1 Processor -- start 2023-11-06T02:09:18.705 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.697+0000 7f8924b96640 1 -- start start 2023-11-06T02:09:18.705 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.701+0000 7f8924b96640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f890c0047d0 0x7f892014a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:18.705 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.701+0000 7f8924b96640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89200b2030 0x7f8920159be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:18.705 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.701+0000 7f8924b96640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f892015a120 0x7f89200b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:18.705 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.701+0000 7f8924b96640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f892015a8a0 con 0x7f890c0047d0 2023-11-06T02:09:18.706 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.701+0000 7f8924b96640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f892014ae40 con 0x7f89200b2030 2023-11-06T02:09:18.706 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.701+0000 7f8924b96640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89200b1ba0 con 0x7f892015a120 2023-11-06T02:09:18.706 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.701+0000 7f891e59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f890c0047d0 0x7f892014a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:18.706 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.701+0000 7f891e59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f890c0047d0 0x7f892014a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41698/0 (socket says 172.21.15.49:41698) 2023-11-06T02:09:18.706 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.701+0000 7f891e59c640 1 -- 172.21.15.49:0/3042739468 learned_addr learned my addr 172.21.15.49:0/3042739468 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:18.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.701+0000 7f891e59c640 1 -- 172.21.15.49:0/3042739468 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f892015a120 msgr2=0x7f89200b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:18.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.701+0000 7f8915d9b640 1 --2- 172.21.15.49:0/3042739468 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89200b2030 0x7f8920159be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:18.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.701+0000 7f891e59c640 1 --2- 172.21.15.49:0/3042739468 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f892015a120 0x7f89200b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:18.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.701+0000 7f891e59c640 1 -- 172.21.15.49:0/3042739468 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89200b2030 msgr2=0x7f8920159be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:18.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.701+0000 7f891e59c640 1 --2- 172.21.15.49:0/3042739468 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89200b2030 0x7f8920159be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:18.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.701+0000 7f891e59c640 1 -- 172.21.15.49:0/3042739468 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f892015a5c0 con 0x7f890c0047d0 2023-11-06T02:09:18.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.701+0000 7f891e59c640 1 --2- 172.21.15.49:0/3042739468 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f890c0047d0 0x7f892014a900 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f891000e700 tx=0x7f891000ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:18.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.701+0000 7f89177fe640 1 -- 172.21.15.49:0/3042739468 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f891000c6c0 con 0x7f890c0047d0 2023-11-06T02:09:18.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.701+0000 7f89177fe640 1 -- 172.21.15.49:0/3042739468 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f891000c820 con 0x7f890c0047d0 2023-11-06T02:09:18.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.701+0000 7f8924b96640 1 -- 172.21.15.49:0/3042739468 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89200b1550 con 0x7f890c0047d0 2023-11-06T02:09:18.709 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.701+0000 7f89177fe640 1 -- 172.21.15.49:0/3042739468 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f891000cbd0 con 0x7f890c0047d0 2023-11-06T02:09:18.709 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.701+0000 7f8924b96640 1 -- 172.21.15.49:0/3042739468 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8920157cd0 con 0x7f890c0047d0 2023-11-06T02:09:18.709 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.701+0000 7f8924b96640 1 -- 172.21.15.49:0/3042739468 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8920074040 con 0x7f890c0047d0 2023-11-06T02:09:18.709 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.701+0000 7f89177fe640 1 -- 172.21.15.49:0/3042739468 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f891000cd30 con 0x7f890c0047d0 2023-11-06T02:09:18.709 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.701+0000 7f89177fe640 1 --2- 172.21.15.49:0/3042739468 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8900065a60 0x7f8900067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:18.709 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.701+0000 7f89177fe640 1 -- 172.21.15.49:0/3042739468 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f8910083260 con 0x7f890c0047d0 2023-11-06T02:09:18.710 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.701+0000 7f8915d9b640 1 --2- 172.21.15.49:0/3042739468 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8900065a60 0x7f8900067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:18.710 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.705+0000 7f8915d9b640 1 --2- 172.21.15.49:0/3042739468 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8900065a60 0x7f8900067f20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f89080af000 tx=0x7f89080046d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:18.710 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.705+0000 7f89177fe640 1 -- 172.21.15.49:0/3042739468 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f8910050970 con 0x7f890c0047d0 2023-11-06T02:09:18.858 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.861+0000 7f8924b96640 1 -- 172.21.15.49:0/3042739468 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f892016b380 con 0x7f890c0047d0 2023-11-06T02:09:18.859 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.861+0000 7f89177fe640 1 -- 172.21.15.49:0/3042739468 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8910053f90 con 0x7f890c0047d0 2023-11-06T02:09:18.860 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2023-11-06T02:09:18.861 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.861+0000 7f8924b96640 1 -- 172.21.15.49:0/3042739468 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8900065a60 msgr2=0x7f8900067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:18.861 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.861+0000 7f8924b96640 1 --2- 172.21.15.49:0/3042739468 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8900065a60 0x7f8900067f20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f89080af000 tx=0x7f89080046d0 comp rx=0 tx=0).stop 2023-11-06T02:09:18.861 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.861+0000 7f8924b96640 1 -- 172.21.15.49:0/3042739468 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f890c0047d0 msgr2=0x7f892014a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:18.862 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.861+0000 7f8924b96640 1 --2- 172.21.15.49:0/3042739468 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f890c0047d0 0x7f892014a900 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f891000e700 tx=0x7f891000ebc0 comp rx=0 tx=0).stop 2023-11-06T02:09:18.862 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.861+0000 7f8924b96640 1 -- 172.21.15.49:0/3042739468 shutdown_connections 2023-11-06T02:09:18.862 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.861+0000 7f8924b96640 1 --2- 172.21.15.49:0/3042739468 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8900065a60 0x7f8900067f20 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:18.863 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.861+0000 7f8924b96640 1 --2- 172.21.15.49:0/3042739468 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f892015a120 0x7f89200b2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:18.863 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.861+0000 7f8924b96640 1 --2- 172.21.15.49:0/3042739468 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89200b2030 0x7f8920159be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:18.863 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.861+0000 7f8924b96640 1 --2- 172.21.15.49:0/3042739468 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f890c0047d0 0x7f892014a900 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:18.864 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.861+0000 7f8924b96640 1 -- 172.21.15.49:0/3042739468 >> 172.21.15.49:0/3042739468 conn(0x7f892006a370 msgr2=0x7f892016ab90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:18.864 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.861+0000 7f8924b96640 1 -- 172.21.15.49:0/3042739468 shutdown_connections 2023-11-06T02:09:18.864 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:18.861+0000 7f8924b96640 1 -- 172.21.15.49:0/3042739468 wait complete. 2023-11-06T02:09:18.878 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T02:09:19.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.009+0000 7f1ea0270640 1 Processor -- start 2023-11-06T02:09:19.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.009+0000 7f1ea0270640 1 -- start start 2023-11-06T02:09:19.009 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.009+0000 7f1ea0270640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e9815a880 0x7f1e980b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:19.009 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.009+0000 7f1ea0270640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f1e98070160 0x7f1e980705e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:19.009 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.009+0000 7f1ea0270640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f1e9806d150 0x7f1e9806d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:19.009 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.009+0000 7f1ea0270640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e98070b20 con 0x7f1e98070160 2023-11-06T02:09:19.009 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.009+0000 7f1ea0270640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e9806db20 con 0x7f1e9806d150 2023-11-06T02:09:19.010 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.009+0000 7f1ea0270640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e980ae360 con 0x7f1e9815a880 2023-11-06T02:09:19.010 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.009+0000 7f1e9d80b640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f1e98070160 0x7f1e980705e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:19.010 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.009+0000 7f1e9d80b640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f1e98070160 0x7f1e980705e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41704/0 (socket says 172.21.15.49:41704) 2023-11-06T02:09:19.010 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.009+0000 7f1e9d80b640 1 -- 172.21.15.49:0/2401585300 learned_addr learned my addr 172.21.15.49:0/2401585300 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:19.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.009+0000 7f1e9e80d640 1 --2- 172.21.15.49:0/2401585300 >> v2:172.21.15.150:3300/0 conn(0x7f1e9806d150 0x7f1e9806d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:19.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.009+0000 7f1e9e00c640 1 --2- 172.21.15.49:0/2401585300 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e9815a880 0x7f1e980b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:19.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.009+0000 7f1e9d80b640 1 -- 172.21.15.49:0/2401585300 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e9815a880 msgr2=0x7f1e980b15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:19.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.009+0000 7f1e9d80b640 1 --2- 172.21.15.49:0/2401585300 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e9815a880 0x7f1e980b15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:19.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.009+0000 7f1e9d80b640 1 -- 172.21.15.49:0/2401585300 >> v2:172.21.15.150:3300/0 conn(0x7f1e9806d150 msgr2=0x7f1e9806d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:19.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.009+0000 7f1e9d80b640 1 --2- 172.21.15.49:0/2401585300 >> v2:172.21.15.150:3300/0 conn(0x7f1e9806d150 0x7f1e9806d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:19.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.009+0000 7f1e9d80b640 1 -- 172.21.15.49:0/2401585300 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e980ae640 con 0x7f1e98070160 2023-11-06T02:09:19.015 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.013+0000 7f1e9d80b640 1 --2- 172.21.15.49:0/2401585300 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e98070160 0x7f1e980705e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f1e980cf1b0 tx=0x7f1e8801c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f764bb7e0f137c52 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:19.015 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.013+0000 7f1e9d00a640 1 -- 172.21.15.49:0/2401585300 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e880ac630 con 0x7f1e98070160 2023-11-06T02:09:19.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.013+0000 7f1e9d00a640 1 -- 172.21.15.49:0/2401585300 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e880ac790 con 0x7f1e98070160 2023-11-06T02:09:19.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.013+0000 7f1e9d00a640 1 -- 172.21.15.49:0/2401585300 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e880acb20 con 0x7f1e98070160 2023-11-06T02:09:19.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.013+0000 7f1ea0270640 1 -- 172.21.15.49:0/2401585300 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e98070160 msgr2=0x7f1e980705e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:19.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.013+0000 7f1ea0270640 1 --2- 172.21.15.49:0/2401585300 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e98070160 0x7f1e980705e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f1e980cf1b0 tx=0x7f1e8801c500 comp rx=0 tx=0).stop 2023-11-06T02:09:19.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.013+0000 7f1ea0270640 1 -- 172.21.15.49:0/2401585300 shutdown_connections 2023-11-06T02:09:19.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.013+0000 7f1ea0270640 1 --2- 172.21.15.49:0/2401585300 >> v2:172.21.15.150:3300/0 conn(0x7f1e9806d150 0x7f1e9806d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:19.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.013+0000 7f1ea0270640 1 --2- 172.21.15.49:0/2401585300 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e98070160 0x7f1e980705e0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:19.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.013+0000 7f1ea0270640 1 --2- 172.21.15.49:0/2401585300 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e9815a880 0x7f1e980b15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:19.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.013+0000 7f1ea0270640 1 -- 172.21.15.49:0/2401585300 >> 172.21.15.49:0/2401585300 conn(0x7f1e9806a370 msgr2=0x7f1e980ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:19.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.013+0000 7f1ea0270640 1 -- 172.21.15.49:0/2401585300 shutdown_connections 2023-11-06T02:09:19.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.013+0000 7f1ea0270640 1 -- 172.21.15.49:0/2401585300 wait complete. 2023-11-06T02:09:19.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.013+0000 7f1ea0270640 1 Processor -- start 2023-11-06T02:09:19.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.013+0000 7f1ea0270640 1 -- start start 2023-11-06T02:09:19.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.017+0000 7f1ea0270640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e9806d150 0x7f1e980cd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:19.019 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.017+0000 7f1ea0270640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1e98070160 0x7f1e9814e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:19.019 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.017+0000 7f1ea0270640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e9815a880 0x7f1e98155490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:19.019 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.017+0000 7f1ea0270640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e980cda20 con 0x7f1e9806d150 2023-11-06T02:09:19.019 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.017+0000 7f1ea0270640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e980cc440 con 0x7f1e98070160 2023-11-06T02:09:19.019 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.017+0000 7f1ea0270640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e980cc720 con 0x7f1e9815a880 2023-11-06T02:09:19.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.017+0000 7f1e9e00c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e9806d150 0x7f1e980cd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:19.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.017+0000 7f1e9e00c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e9806d150 0x7f1e980cd4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41712/0 (socket says 172.21.15.49:41712) 2023-11-06T02:09:19.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.017+0000 7f1e9e00c640 1 -- 172.21.15.49:0/1397501397 learned_addr learned my addr 172.21.15.49:0/1397501397 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:19.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.017+0000 7f1e9d80b640 1 --2- 172.21.15.49:0/1397501397 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1e98070160 0x7f1e9814e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:19.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.017+0000 7f1e9e80d640 1 --2- 172.21.15.49:0/1397501397 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e9815a880 0x7f1e98155490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:19.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.017+0000 7f1e9e00c640 1 -- 172.21.15.49:0/1397501397 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e9815a880 msgr2=0x7f1e98155490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:19.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.017+0000 7f1e9e00c640 1 --2- 172.21.15.49:0/1397501397 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e9815a880 0x7f1e98155490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:19.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.017+0000 7f1e9e00c640 1 -- 172.21.15.49:0/1397501397 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1e98070160 msgr2=0x7f1e9814e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:19.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.017+0000 7f1e9e00c640 1 --2- 172.21.15.49:0/1397501397 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1e98070160 0x7f1e9814e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:19.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.017+0000 7f1e9e00c640 1 -- 172.21.15.49:0/1397501397 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e880094d0 con 0x7f1e9806d150 2023-11-06T02:09:19.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.017+0000 7f1e9e00c640 1 --2- 172.21.15.49:0/1397501397 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e9806d150 0x7f1e980cd4e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f1e9400d5f0 tx=0x7f1e9400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:19.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.017+0000 7f1e8effd640 1 -- 172.21.15.49:0/1397501397 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e940125f0 con 0x7f1e9806d150 2023-11-06T02:09:19.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.017+0000 7f1e8effd640 1 -- 172.21.15.49:0/1397501397 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e94012750 con 0x7f1e9806d150 2023-11-06T02:09:19.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.017+0000 7f1e8effd640 1 -- 172.21.15.49:0/1397501397 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e94012ae0 con 0x7f1e9806d150 2023-11-06T02:09:19.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.017+0000 7f1ea0270640 1 -- 172.21.15.49:0/1397501397 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e9815ade0 con 0x7f1e9806d150 2023-11-06T02:09:19.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.017+0000 7f1ea0270640 1 -- 172.21.15.49:0/1397501397 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e980ccce0 con 0x7f1e9806d150 2023-11-06T02:09:19.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.017+0000 7f1ea0270640 1 -- 172.21.15.49:0/1397501397 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e98074040 con 0x7f1e9806d150 2023-11-06T02:09:19.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.017+0000 7f1e8effd640 1 -- 172.21.15.49:0/1397501397 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f1e94012c40 con 0x7f1e9806d150 2023-11-06T02:09:19.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.017+0000 7f1e8effd640 1 --2- 172.21.15.49:0/1397501397 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1e7c065a60 0x7f1e7c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:19.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.017+0000 7f1e8effd640 1 -- 172.21.15.49:0/1397501397 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f1e940837a0 con 0x7f1e9806d150 2023-11-06T02:09:19.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.017+0000 7f1e9d80b640 1 --2- 172.21.15.49:0/1397501397 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1e7c065a60 0x7f1e7c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:19.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.021+0000 7f1e9d80b640 1 --2- 172.21.15.49:0/1397501397 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1e7c065a60 0x7f1e7c067f20 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1e880ae000 tx=0x7f1e88008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:19.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.021+0000 7f1e8effd640 1 -- 172.21.15.49:0/1397501397 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f1e94051050 con 0x7f1e9806d150 2023-11-06T02:09:19.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.181+0000 7f1ea0270640 1 -- 172.21.15.49:0/1397501397 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1e9806c430 con 0x7f1e9806d150 2023-11-06T02:09:19.179 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.181+0000 7f1e8effd640 1 -- 172.21.15.49:0/1397501397 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1e94054670 con 0x7f1e9806d150 2023-11-06T02:09:19.179 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2023-11-06T02:09:19.181 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.181+0000 7f1ea0270640 1 -- 172.21.15.49:0/1397501397 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1e7c065a60 msgr2=0x7f1e7c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:19.181 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.181+0000 7f1ea0270640 1 --2- 172.21.15.49:0/1397501397 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1e7c065a60 0x7f1e7c067f20 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1e880ae000 tx=0x7f1e88008040 comp rx=0 tx=0).stop 2023-11-06T02:09:19.182 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.181+0000 7f1ea0270640 1 -- 172.21.15.49:0/1397501397 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e9806d150 msgr2=0x7f1e980cd4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:19.182 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.181+0000 7f1ea0270640 1 --2- 172.21.15.49:0/1397501397 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e9806d150 0x7f1e980cd4e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f1e9400d5f0 tx=0x7f1e9400dab0 comp rx=0 tx=0).stop 2023-11-06T02:09:19.182 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.185+0000 7f1ea0270640 1 -- 172.21.15.49:0/1397501397 shutdown_connections 2023-11-06T02:09:19.182 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.185+0000 7f1ea0270640 1 --2- 172.21.15.49:0/1397501397 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1e7c065a60 0x7f1e7c067f20 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:19.183 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.185+0000 7f1ea0270640 1 --2- 172.21.15.49:0/1397501397 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e9815a880 0x7f1e98155490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:19.183 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.185+0000 7f1ea0270640 1 --2- 172.21.15.49:0/1397501397 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1e98070160 0x7f1e9814e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:19.183 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.185+0000 7f1ea0270640 1 --2- 172.21.15.49:0/1397501397 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e9806d150 0x7f1e980cd4e0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:19.183 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.185+0000 7f1ea0270640 1 -- 172.21.15.49:0/1397501397 >> 172.21.15.49:0/1397501397 conn(0x7f1e9806a370 msgr2=0x7f1e980b3ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:19.183 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.185+0000 7f1ea0270640 1 -- 172.21.15.49:0/1397501397 shutdown_connections 2023-11-06T02:09:19.183 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.185+0000 7f1ea0270640 1 -- 172.21.15.49:0/1397501397 wait complete. 2023-11-06T02:09:19.199 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-06T02:09:19.326 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.329+0000 7fcf5e4c9640 1 Processor -- start 2023-11-06T02:09:19.326 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.329+0000 7fcf5e4c9640 1 -- start start 2023-11-06T02:09:19.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.329+0000 7fcf5e4c9640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcf580b0d80 0x7fcf5806b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:19.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.329+0000 7fcf5e4c9640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fcf580b1420 0x7fcf5806b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:19.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.329+0000 7fcf5e4c9640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fcf580b1970 0x7fcf580cd280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:19.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.329+0000 7fcf5e4c9640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf580cffb0 con 0x7fcf580b1970 2023-11-06T02:09:19.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.329+0000 7fcf5e4c9640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf580cf710 con 0x7fcf580b1420 2023-11-06T02:09:19.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.329+0000 7fcf5e4c9640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf580cf9f0 con 0x7fcf580b0d80 2023-11-06T02:09:19.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.329+0000 7fcf5ca66640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fcf580b1970 0x7fcf580cd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:19.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.329+0000 7fcf5ca66640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fcf580b1970 0x7fcf580cd280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41718/0 (socket says 172.21.15.49:41718) 2023-11-06T02:09:19.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.329+0000 7fcf5ca66640 1 -- 172.21.15.49:0/3514448413 learned_addr learned my addr 172.21.15.49:0/3514448413 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:19.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.329+0000 7fcf577fe640 1 --2- 172.21.15.49:0/3514448413 >> v2:172.21.15.150:3300/0 conn(0x7fcf580b1420 0x7fcf5806b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:19.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.329+0000 7fcf57fff640 1 --2- 172.21.15.49:0/3514448413 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcf580b0d80 0x7fcf5806b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:19.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.329+0000 7fcf5ca66640 1 -- 172.21.15.49:0/3514448413 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcf580b0d80 msgr2=0x7fcf5806b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:19.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.329+0000 7fcf5ca66640 1 --2- 172.21.15.49:0/3514448413 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcf580b0d80 0x7fcf5806b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:19.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.329+0000 7fcf5ca66640 1 -- 172.21.15.49:0/3514448413 >> v2:172.21.15.150:3300/0 conn(0x7fcf580b1420 msgr2=0x7fcf5806b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:19.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.329+0000 7fcf5ca66640 1 --2- 172.21.15.49:0/3514448413 >> v2:172.21.15.150:3300/0 conn(0x7fcf580b1420 0x7fcf5806b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:19.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.329+0000 7fcf5ca66640 1 -- 172.21.15.49:0/3514448413 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf580cfcd0 con 0x7fcf580b1970 2023-11-06T02:09:19.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.333+0000 7fcf5ca66640 1 --2- 172.21.15.49:0/3514448413 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcf580b1970 0x7fcf580cd280 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fcf5808ae00 tx=0x7fcf5001c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7266f1043f18b665 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:19.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.333+0000 7fcf56ffd640 1 -- 172.21.15.49:0/3514448413 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf500a5520 con 0x7fcf580b1970 2023-11-06T02:09:19.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.333+0000 7fcf56ffd640 1 -- 172.21.15.49:0/3514448413 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf500a5680 con 0x7fcf580b1970 2023-11-06T02:09:19.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.333+0000 7fcf56ffd640 1 -- 172.21.15.49:0/3514448413 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf500a5a10 con 0x7fcf580b1970 2023-11-06T02:09:19.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.333+0000 7fcf5e4c9640 1 -- 172.21.15.49:0/3514448413 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcf580b1970 msgr2=0x7fcf580cd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:19.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.333+0000 7fcf5e4c9640 1 --2- 172.21.15.49:0/3514448413 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcf580b1970 0x7fcf580cd280 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fcf5808ae00 tx=0x7fcf5001c680 comp rx=0 tx=0).stop 2023-11-06T02:09:19.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.333+0000 7fcf5e4c9640 1 -- 172.21.15.49:0/3514448413 shutdown_connections 2023-11-06T02:09:19.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.333+0000 7fcf5e4c9640 1 --2- 172.21.15.49:0/3514448413 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcf580b1970 0x7fcf580cd280 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:19.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.333+0000 7fcf5e4c9640 1 --2- 172.21.15.49:0/3514448413 >> v2:172.21.15.150:3300/0 conn(0x7fcf580b1420 0x7fcf5806b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:19.334 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.333+0000 7fcf5e4c9640 1 --2- 172.21.15.49:0/3514448413 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcf580b0d80 0x7fcf5806b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:19.334 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.333+0000 7fcf5e4c9640 1 -- 172.21.15.49:0/3514448413 >> 172.21.15.49:0/3514448413 conn(0x7fcf580d0890 msgr2=0x7fcf580d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:19.334 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.333+0000 7fcf5e4c9640 1 -- 172.21.15.49:0/3514448413 shutdown_connections 2023-11-06T02:09:19.334 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.333+0000 7fcf5e4c9640 1 -- 172.21.15.49:0/3514448413 wait complete. 2023-11-06T02:09:19.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.333+0000 7fcf5e4c9640 1 Processor -- start 2023-11-06T02:09:19.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.333+0000 7fcf5e4c9640 1 -- start start 2023-11-06T02:09:19.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.337+0000 7fcf5e4c9640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcf580b0d80 0x7fcf580c5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:19.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.337+0000 7fcf5e4c9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcf580b1420 0x7fcf5814ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:19.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.337+0000 7fcf5e4c9640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf580b1970 0x7fcf5815a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:19.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.337+0000 7fcf5e4c9640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5814f410 con 0x7fcf580b1420 2023-11-06T02:09:19.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.337+0000 7fcf5e4c9640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5814f9d0 con 0x7fcf580b1970 2023-11-06T02:09:19.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.337+0000 7fcf5e4c9640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf580c5540 con 0x7fcf580b0d80 2023-11-06T02:09:19.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.337+0000 7fcf577fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcf580b1420 0x7fcf5814ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:19.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.337+0000 7fcf577fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcf580b1420 0x7fcf5814ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41726/0 (socket says 172.21.15.49:41726) 2023-11-06T02:09:19.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.337+0000 7fcf577fe640 1 -- 172.21.15.49:0/2428867704 learned_addr learned my addr 172.21.15.49:0/2428867704 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:19.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.337+0000 7fcf5ca66640 1 --2- 172.21.15.49:0/2428867704 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf580b1970 0x7fcf5815a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:19.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.337+0000 7fcf57fff640 1 --2- 172.21.15.49:0/2428867704 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcf580b0d80 0x7fcf580c5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:19.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.337+0000 7fcf577fe640 1 -- 172.21.15.49:0/2428867704 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcf580b0d80 msgr2=0x7fcf580c5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:19.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.337+0000 7fcf577fe640 1 --2- 172.21.15.49:0/2428867704 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcf580b0d80 0x7fcf580c5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:19.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.337+0000 7fcf577fe640 1 -- 172.21.15.49:0/2428867704 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf580b1970 msgr2=0x7fcf5815a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:19.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.337+0000 7fcf577fe640 1 --2- 172.21.15.49:0/2428867704 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf580b1970 0x7fcf5815a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:19.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.337+0000 7fcf577fe640 1 -- 172.21.15.49:0/2428867704 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf50009b30 con 0x7fcf580b1420 2023-11-06T02:09:19.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.337+0000 7fcf57fff640 1 --2- 172.21.15.49:0/2428867704 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcf580b0d80 0x7fcf580c5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:09:19.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.337+0000 7fcf5ca66640 1 --2- 172.21.15.49:0/2428867704 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf580b1970 0x7fcf5815a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:09:19.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.337+0000 7fcf577fe640 1 --2- 172.21.15.49:0/2428867704 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcf580b1420 0x7fcf5814ec70 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c00e960 tx=0x7fcf4c00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:19.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.337+0000 7fcf54ff9640 1 -- 172.21.15.49:0/2428867704 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf4c009690 con 0x7fcf580b1420 2023-11-06T02:09:19.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.337+0000 7fcf54ff9640 1 -- 172.21.15.49:0/2428867704 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf4c00ca00 con 0x7fcf580b1420 2023-11-06T02:09:19.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.337+0000 7fcf5e4c9640 1 -- 172.21.15.49:0/2428867704 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf580c6c80 con 0x7fcf580b1420 2023-11-06T02:09:19.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.337+0000 7fcf54ff9640 1 -- 172.21.15.49:0/2428867704 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf4c00cd70 con 0x7fcf580b1420 2023-11-06T02:09:19.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.337+0000 7fcf5e4c9640 1 -- 172.21.15.49:0/2428867704 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf5814f660 con 0x7fcf580b1420 2023-11-06T02:09:19.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.337+0000 7fcf5e4c9640 1 -- 172.21.15.49:0/2428867704 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf5814f210 con 0x7fcf580b1420 2023-11-06T02:09:19.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.337+0000 7fcf54ff9640 1 -- 172.21.15.49:0/2428867704 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fcf4c013470 con 0x7fcf580b1420 2023-11-06T02:09:19.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.337+0000 7fcf54ff9640 1 --2- 172.21.15.49:0/2428867704 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fcf44065a60 0x7fcf44067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:19.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.337+0000 7fcf54ff9640 1 -- 172.21.15.49:0/2428867704 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fcf4c0834f0 con 0x7fcf580b1420 2023-11-06T02:09:19.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.337+0000 7fcf57fff640 1 --2- 172.21.15.49:0/2428867704 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fcf44065a60 0x7fcf44067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:19.344 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.341+0000 7fcf57fff640 1 --2- 172.21.15.49:0/2428867704 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fcf44065a60 0x7fcf44067f20 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fcf581537f0 tx=0x7fcf48008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:19.344 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.341+0000 7fcf54ff9640 1 -- 172.21.15.49:0/2428867704 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fcf4c050da0 con 0x7fcf580b1420 2023-11-06T02:09:19.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.509+0000 7fcf5e4c9640 1 -- 172.21.15.49:0/2428867704 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fcf5814fb40 con 0x7fcf580b1420 2023-11-06T02:09:19.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.513+0000 7fcf54ff9640 1 -- 172.21.15.49:0/2428867704 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7fcf4c0543c0 con 0x7fcf580b1420 2023-11-06T02:09:19.510 INFO:teuthology.orchestra.run.smithi049.stderr:listed 0 entries 2023-11-06T02:09:19.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.513+0000 7fcf5e4c9640 1 -- 172.21.15.49:0/2428867704 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fcf44065a60 msgr2=0x7fcf44067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:19.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.513+0000 7fcf5e4c9640 1 --2- 172.21.15.49:0/2428867704 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fcf44065a60 0x7fcf44067f20 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fcf581537f0 tx=0x7fcf48008040 comp rx=0 tx=0).stop 2023-11-06T02:09:19.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.513+0000 7fcf5e4c9640 1 -- 172.21.15.49:0/2428867704 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcf580b1420 msgr2=0x7fcf5814ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:19.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.513+0000 7fcf5e4c9640 1 --2- 172.21.15.49:0/2428867704 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcf580b1420 0x7fcf5814ec70 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c00e960 tx=0x7fcf4c00ee20 comp rx=0 tx=0).stop 2023-11-06T02:09:19.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.513+0000 7fcf5e4c9640 1 -- 172.21.15.49:0/2428867704 shutdown_connections 2023-11-06T02:09:19.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.513+0000 7fcf5e4c9640 1 --2- 172.21.15.49:0/2428867704 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fcf44065a60 0x7fcf44067f20 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:19.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.513+0000 7fcf5e4c9640 1 --2- 172.21.15.49:0/2428867704 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcf580b1420 0x7fcf5814ec70 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:19.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.513+0000 7fcf5e4c9640 1 --2- 172.21.15.49:0/2428867704 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf580b1970 0x7fcf5815a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:19.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.513+0000 7fcf5e4c9640 1 --2- 172.21.15.49:0/2428867704 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcf580b0d80 0x7fcf580c5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:19.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.513+0000 7fcf5e4c9640 1 -- 172.21.15.49:0/2428867704 >> 172.21.15.49:0/2428867704 conn(0x7fcf580d0890 msgr2=0x7fcf580cc5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:19.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.513+0000 7fcf5e4c9640 1 -- 172.21.15.49:0/2428867704 shutdown_connections 2023-11-06T02:09:19.514 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.513+0000 7fcf5e4c9640 1 -- 172.21.15.49:0/2428867704 wait complete. 2023-11-06T02:09:19.528 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-11-06T02:09:19.529 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T02:09:19.535 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T02:09:19.710 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.713+0000 7feb2ee9b640 1 Processor -- start 2023-11-06T02:09:19.711 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.713+0000 7feb2ee9b640 1 -- start start 2023-11-06T02:09:19.713 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.713+0000 7feb2ee9b640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7feb280ca980 0x7feb281644c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:19.714 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.713+0000 7feb2ee9b640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb280cad90 con 0x7feb28159b40 2023-11-06T02:09:19.714 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.713+0000 7feb2ee9b640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb280723a0 con 0x7feb280cb010 2023-11-06T02:09:19.714 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.713+0000 7feb2ee9b640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb280715e0 con 0x7feb280ca980 2023-11-06T02:09:19.714 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.713+0000 7feb27fff640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7feb28159b40 0x7feb280cdd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:35448/0 (socket says 172.21.15.49:35448) 2023-11-06T02:09:19.714 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.713+0000 7feb27fff640 1 -- 172.21.15.49:0/154715473 learned_addr learned my addr 172.21.15.49:0/154715473 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:09:19.715 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.713+0000 7feb2d438640 1 --2- 172.21.15.49:0/154715473 >> v2:172.21.15.150:3300/0 conn(0x7feb280ca980 0x7feb281644c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:19.715 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.713+0000 7feb277fe640 1 -- 172.21.15.49:0/154715473 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2080382987 0 0) 0x7feb280cad90 con 0x7feb28159b40 2023-11-06T02:09:19.715 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.713+0000 7feb277fe640 1 -- 172.21.15.49:0/154715473 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb10003530 con 0x7feb28159b40 2023-11-06T02:09:19.715 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.713+0000 7feb277fe640 1 -- 172.21.15.49:0/154715473 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 827423360 0 0) 0x7feb10003530 con 0x7feb28159b40 2023-11-06T02:09:19.716 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.713+0000 7feb277fe640 1 -- 172.21.15.49:0/154715473 >> v2:172.21.15.150:3300/0 conn(0x7feb280ca980 msgr2=0x7feb281644c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:19.716 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.713+0000 7feb277fe640 1 --2- 172.21.15.49:0/154715473 >> v2:172.21.15.150:3300/0 conn(0x7feb280ca980 0x7feb281644c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:19.716 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.713+0000 7feb277fe640 1 -- 172.21.15.49:0/154715473 >> v1:172.21.15.150:6789/0 conn(0x7feb280cb010 legacy=0x7feb280cb3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:19.716 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.713+0000 7feb277fe640 1 -- 172.21.15.49:0/154715473 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb280718c0 con 0x7feb28159b40 2023-11-06T02:09:19.716 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.713+0000 7feb277fe640 1 -- 172.21.15.49:0/154715473 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7feb1c001680 con 0x7feb28159b40 2023-11-06T02:09:19.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.713+0000 7feb277fe640 1 -- 172.21.15.49:0/154715473 >> v1:172.21.15.49:6789/0 conn(0x7feb28159b40 legacy=0x7feb280cdd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:19.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.713+0000 7feb277fe640 1 --2- 172.21.15.49:0/154715473 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feb10003980 0x7feb10003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:19.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.713+0000 7feb277fe640 1 --2- 172.21.15.49:0/154715473 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb100047d0 0x7feb10004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:19.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.713+0000 7feb277fe640 1 --2- 172.21.15.49:0/154715473 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb10005190 0x7feb1000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:19.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.713+0000 7feb277fe640 1 -- 172.21.15.49:0/154715473 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb280715e0 con 0x7feb10005190 2023-11-06T02:09:19.718 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.713+0000 7feb277fe640 1 -- 172.21.15.49:0/154715473 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb1000c030 con 0x7feb100047d0 2023-11-06T02:09:19.718 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.713+0000 7feb277fe640 1 -- 172.21.15.49:0/154715473 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb1000c1f0 con 0x7feb10003980 2023-11-06T02:09:19.718 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.713+0000 7feb2d438640 1 --2- 172.21.15.49:0/154715473 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb10005190 0x7feb1000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:19.718 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.713+0000 7feb2cc37640 1 --2- 172.21.15.49:0/154715473 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feb10003980 0x7feb10003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:19.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.713+0000 7feb2d438640 1 -- 172.21.15.49:0/154715473 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feb10003980 msgr2=0x7feb10003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:19.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.713+0000 7feb2d438640 1 --2- 172.21.15.49:0/154715473 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feb10003980 0x7feb10003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:19.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.713+0000 7feb2d438640 1 -- 172.21.15.49:0/154715473 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb100047d0 msgr2=0x7feb10004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:19.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.713+0000 7feb27fff640 1 --2- 172.21.15.49:0/154715473 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb100047d0 0x7feb10004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:19.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.713+0000 7feb2d438640 1 --2- 172.21.15.49:0/154715473 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb100047d0 0x7feb10004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:19.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.713+0000 7feb2d438640 1 -- 172.21.15.49:0/154715473 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb10003530 con 0x7feb10005190 2023-11-06T02:09:19.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.713+0000 7feb27fff640 1 --2- 172.21.15.49:0/154715473 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb100047d0 0x7feb10004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:09:19.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.717+0000 7feb2d438640 1 -- 172.21.15.49:0/154715473 reap_dead start 2023-11-06T02:09:19.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.717+0000 7feb2d438640 1 --2- 172.21.15.49:0/154715473 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb10005190 0x7feb1000b990 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7feb20009ae0 tx=0x7feb2001ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe61b87d09ab63d9 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:19.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.717+0000 7feb277fe640 1 -- 172.21.15.49:0/154715473 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb20004020 con 0x7feb10005190 2023-11-06T02:09:19.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.717+0000 7feb277fe640 1 -- 172.21.15.49:0/154715473 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb200041c0 con 0x7feb10005190 2023-11-06T02:09:19.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.717+0000 7feb277fe640 1 -- 172.21.15.49:0/154715473 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb200a5680 con 0x7feb10005190 2023-11-06T02:09:19.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.717+0000 7feb2ee9b640 1 -- 172.21.15.49:0/154715473 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb10005190 msgr2=0x7feb1000b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:19.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.717+0000 7feb2ee9b640 1 --2- 172.21.15.49:0/154715473 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb10005190 0x7feb1000b990 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7feb20009ae0 tx=0x7feb2001ca90 comp rx=0 tx=0).stop 2023-11-06T02:09:19.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.717+0000 7feb2ee9b640 1 -- 172.21.15.49:0/154715473 shutdown_connections 2023-11-06T02:09:19.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.717+0000 7feb2ee9b640 1 --2- 172.21.15.49:0/154715473 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb10005190 0x7feb1000b990 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:19.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.717+0000 7feb2ee9b640 1 -- 172.21.15.49:0/154715473 >> 172.21.15.49:0/154715473 conn(0x7feb2806c0a0 msgr2=0x7feb2806c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:19.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.717+0000 7feb2ee9b640 1 -- 172.21.15.49:0/154715473 shutdown_connections 2023-11-06T02:09:19.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.717+0000 7feb2ee9b640 1 -- 172.21.15.49:0/154715473 wait complete. 2023-11-06T02:09:19.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.721+0000 7feb2ee9b640 1 Processor -- start 2023-11-06T02:09:19.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.721+0000 7feb2ee9b640 1 -- start start 2023-11-06T02:09:19.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.721+0000 7feb2ee9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb10005190 0x7feb281f23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:19.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.721+0000 7feb2ee9b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feb280acf20 0x7feb280c3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:19.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.721+0000 7feb2ee9b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb280ab8c0 0x7feb280c6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:19.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.721+0000 7feb2ee9b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb280b01e0 con 0x7feb10005190 2023-11-06T02:09:19.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.721+0000 7feb2ee9b640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb280abe80 con 0x7feb280ab8c0 2023-11-06T02:09:19.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.721+0000 7feb2ee9b640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb280ac160 con 0x7feb280acf20 2023-11-06T02:09:19.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.721+0000 7feb2cc37640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb10005190 0x7feb281f23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:19.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.721+0000 7feb2cc37640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb10005190 0x7feb281f23e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41742/0 (socket says 172.21.15.49:41742) 2023-11-06T02:09:19.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.721+0000 7feb2cc37640 1 -- 172.21.15.49:0/576961735 learned_addr learned my addr 172.21.15.49:0/576961735 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:19.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.721+0000 7feb2d438640 1 --2- 172.21.15.49:0/576961735 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb280ab8c0 0x7feb280c6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:19.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.721+0000 7feb27fff640 1 --2- 172.21.15.49:0/576961735 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feb280acf20 0x7feb280c3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:19.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.721+0000 7feb2cc37640 1 -- 172.21.15.49:0/576961735 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feb280acf20 msgr2=0x7feb280c3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:19.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.721+0000 7feb2cc37640 1 --2- 172.21.15.49:0/576961735 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feb280acf20 0x7feb280c3b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:19.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.721+0000 7feb2cc37640 1 -- 172.21.15.49:0/576961735 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb280ab8c0 msgr2=0x7feb280c6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:19.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.721+0000 7feb2cc37640 1 --2- 172.21.15.49:0/576961735 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb280ab8c0 0x7feb280c6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:19.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.721+0000 7feb2cc37640 1 -- 172.21.15.49:0/576961735 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb280ac440 con 0x7feb10005190 2023-11-06T02:09:19.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.721+0000 7feb2d438640 1 --2- 172.21.15.49:0/576961735 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb280ab8c0 0x7feb280c6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:09:19.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.721+0000 7feb2cc37640 1 --2- 172.21.15.49:0/576961735 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb10005190 0x7feb281f23e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7feb1800e7b0 tx=0x7feb1800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:19.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.721+0000 7feb257fa640 1 -- 172.21.15.49:0/576961735 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb1800ee60 con 0x7feb10005190 2023-11-06T02:09:19.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.721+0000 7feb257fa640 1 -- 172.21.15.49:0/576961735 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb18003be0 con 0x7feb10005190 2023-11-06T02:09:19.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.721+0000 7feb2ee9b640 1 -- 172.21.15.49:0/576961735 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb280ac720 con 0x7feb10005190 2023-11-06T02:09:19.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.721+0000 7feb257fa640 1 -- 172.21.15.49:0/576961735 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb180050d0 con 0x7feb10005190 2023-11-06T02:09:19.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.721+0000 7feb2ee9b640 1 -- 172.21.15.49:0/576961735 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb281f2920 con 0x7feb10005190 2023-11-06T02:09:19.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.721+0000 7feb2ee9b640 1 -- 172.21.15.49:0/576961735 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb28074040 con 0x7feb10005190 2023-11-06T02:09:19.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.721+0000 7feb257fa640 1 -- 172.21.15.49:0/576961735 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7feb18005230 con 0x7feb10005190 2023-11-06T02:09:19.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.725+0000 7feb257fa640 1 --2- 172.21.15.49:0/576961735 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7feb14065a90 0x7feb14067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:19.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.725+0000 7feb257fa640 1 -- 172.21.15.49:0/576961735 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7feb18083410 con 0x7feb10005190 2023-11-06T02:09:19.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.725+0000 7feb27fff640 1 --2- 172.21.15.49:0/576961735 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7feb14065a90 0x7feb14067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:19.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.725+0000 7feb27fff640 1 --2- 172.21.15.49:0/576961735 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7feb14065a90 0x7feb14067f50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7feb280c5630 tx=0x7feb1c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:19.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.729+0000 7feb257fa640 1 -- 172.21.15.49:0/576961735 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7feb18050ba0 con 0x7feb10005190 2023-11-06T02:09:19.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.885+0000 7feb2ee9b640 1 -- 172.21.15.49:0/576961735 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feb2806b110 con 0x7feb10005190 2023-11-06T02:09:19.886 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.889+0000 7feb257fa640 1 -- 172.21.15.49:0/576961735 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17906 (secure 0 0 0) 0x7feb180541c0 con 0x7feb10005190 2023-11-06T02:09:19.887 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:09:19.887 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":25,"fsid":"1a8b035c-6ba4-499a-b876-aea47fd9c0d4","created":"2023-11-06T02:08:06.440857+0000","modified":"2023-11-06T02:09:13.195118+0000","last_up_change":"2023-11-06T02:08:17.982372+0000","last_in_change":"2023-11-06T02:08:08.906631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T02:08:19.252222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T02:08:24.776211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-06T02:08:28.936178+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-06T02:08:29.563468+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ad55af8-0571-4e6a-8722-b5d429b7496d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6809","nonce":38116},{"type":"v1","addr":"172.21.15.49:6811","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6813","nonce":38116},{"type":"v1","addr":"172.21.15.49:6815","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6821","nonce":38116},{"type":"v1","addr":"172.21.15.49:6823","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6817","nonce":38116},{"type":"v1","addr":"172.21.15.49:6819","nonce":38116}]},"public_addr":"172.21.15.49:6811/38116","cluster_addr":"172.21.15.49:6815/38116","heartbeat_back_addr":"172.21.15.49:6823/38116","heartbeat_front_addr":"172.21.15.49:6819/38116","state":["exists","up"]},{"osd":1,"uuid":"c8b31f85-a757-43ed-9419-351acc46611b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":38113},{"type":"v1","addr":"172.21.15.49:6810","nonce":38113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":38113},{"type":"v1","addr":"172.21.15.49:6814","nonce":38113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":38113},{"type":"v1","addr":"172.21.15.49:6822","nonce":38113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":38113},{"type":"v1","addr":"172.21.15.49:6818","nonce":38113}]},"public_addr":"172.21.15.49:6810/38113","cluster_addr":"172.21.15.49:6814/38113","heartbeat_back_addr":"172.21.15.49:6822/38113","heartbeat_front_addr":"172.21.15.49:6818/38113","state":["exists","up"]},{"osd":2,"uuid":"555b3bf3-0a70-4b91-8efd-e4204b5fc26e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":38114},{"type":"v1","addr":"172.21.15.49:6801","nonce":38114}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":38114},{"type":"v1","addr":"172.21.15.49:6803","nonce":38114}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":38114},{"type":"v1","addr":"172.21.15.49:6807","nonce":38114}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":38114},{"type":"v1","addr":"172.21.15.49:6805","nonce":38114}]},"public_addr":"172.21.15.49:6801/38114","cluster_addr":"172.21.15.49:6803/38114","heartbeat_back_addr":"172.21.15.49:6807/38114","heartbeat_front_addr":"172.21.15.49:6805/38114","state":["exists","up"]},{"osd":3,"uuid":"8d4e5d77-03ef-4b9b-bfcb-5ae93a46795c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":38115},{"type":"v1","addr":"172.21.15.49:6825","nonce":38115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":38115},{"type":"v1","addr":"172.21.15.49:6827","nonce":38115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":38115},{"type":"v1","addr":"172.21.15.49:6831","nonce":38115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":38115},{"type":"v1","addr":"172.21.15.49:6829","nonce":38115}]},"public_addr":"172.21.15.49:6825/38115","cluster_addr":"172.21.15.49:6827/38115","heartbeat_back_addr":"172.21.15.49:6831/38115","heartbeat_front_addr":"172.21.15.49:6829/38115","state":["exists","up"]},{"osd":4,"uuid":"dba03ad4-68a3-4cf3-9218-ac611f02eacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":38417},{"type":"v1","addr":"172.21.15.150:6809","nonce":38417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":38417},{"type":"v1","addr":"172.21.15.150:6811","nonce":38417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":38417},{"type":"v1","addr":"172.21.15.150:6815","nonce":38417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":38417},{"type":"v1","addr":"172.21.15.150:6813","nonce":38417}]},"public_addr":"172.21.15.150:6809/38417","cluster_addr":"172.21.15.150:6811/38417","heartbeat_back_addr":"172.21.15.150:6815/38417","heartbeat_front_addr":"172.21.15.150:6813/38417","state":["exists","up"]},{"osd":5,"uuid":"0df9c303-c255-4640-8e91-01f1c3efe18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":38418},{"type":"v1","addr":"172.21.15.150:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":38418},{"type":"v1","addr":"172.21.15.150:6803","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":38418},{"type":"v1","addr":"172.21.15.150:6807","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":38418},{"type":"v1","addr":"172.21.15.150:6805","nonce":38418}]},"public_addr":"172.21.15.150:6801/38418","cluster_addr":"172.21.15.150:6803/38418","heartbeat_back_addr":"172.21.15.150:6807/38418","heartbeat_front_addr":"172.21.15.150:6805/38418","state":["exists","up"]},{"osd":6,"uuid":"e59fa189-4b70-4c06-9722-c02b90828bf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":38428},{"type":"v1","addr":"172.21.15.150:6825","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":38428},{"type":"v1","addr":"172.21.15.150:6827","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":38428},{"type":"v1","addr":"172.21.15.150:6831","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":38428},{"type":"v1","addr":"172.21.15.150:6829","nonce":38428}]},"public_addr":"172.21.15.150:6825/38428","cluster_addr":"172.21.15.150:6827/38428","heartbeat_back_addr":"172.21.15.150:6831/38428","heartbeat_front_addr":"172.21.15.150:6829/38428","state":["exists","up"]},{"osd":7,"uuid":"a64f57e1-ef98-46bd-b304-385759e95898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":38421},{"type":"v1","addr":"172.21.15.150:6817","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":38421},{"type":"v1","addr":"172.21.15.150:6819","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":38421},{"type":"v1","addr":"172.21.15.150:6823","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":38421},{"type":"v1","addr":"172.21.15.150:6821","nonce":38421}]},"public_addr":"172.21.15.150:6817/38421","cluster_addr":"172.21.15.150:6819/38421","heartbeat_back_addr":"172.21.15.150:6823/38421","heartbeat_front_addr":"172.21.15.150:6821/38421","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.438552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.413280+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.296466+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.424966+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.276953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.230984+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.410235+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.480084+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.14","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T02:09:19.888 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.889+0000 7feb2ee9b640 1 -- 172.21.15.49:0/576961735 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7feb14065a90 msgr2=0x7feb14067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:19.889 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.889+0000 7feb2ee9b640 1 --2- 172.21.15.49:0/576961735 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7feb14065a90 0x7feb14067f50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7feb280c5630 tx=0x7feb1c009040 comp rx=0 tx=0).stop 2023-11-06T02:09:19.889 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.889+0000 7feb2ee9b640 1 -- 172.21.15.49:0/576961735 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb10005190 msgr2=0x7feb281f23e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:19.889 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.889+0000 7feb2ee9b640 1 --2- 172.21.15.49:0/576961735 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb10005190 0x7feb281f23e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7feb1800e7b0 tx=0x7feb1800ec70 comp rx=0 tx=0).stop 2023-11-06T02:09:19.889 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.889+0000 7feb2ee9b640 1 -- 172.21.15.49:0/576961735 shutdown_connections 2023-11-06T02:09:19.890 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.889+0000 7feb2ee9b640 1 --2- 172.21.15.49:0/576961735 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7feb14065a90 0x7feb14067f50 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:19.890 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.889+0000 7feb2ee9b640 1 --2- 172.21.15.49:0/576961735 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb280ab8c0 0x7feb280c6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:19.890 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.889+0000 7feb2ee9b640 1 --2- 172.21.15.49:0/576961735 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feb280acf20 0x7feb280c3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:19.890 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.889+0000 7feb2ee9b640 1 --2- 172.21.15.49:0/576961735 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb10005190 0x7feb281f23e0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:19.891 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.889+0000 7feb2ee9b640 1 -- 172.21.15.49:0/576961735 >> 172.21.15.49:0/576961735 conn(0x7feb2806c0a0 msgr2=0x7feb28158ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:19.891 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.889+0000 7feb2ee9b640 1 -- 172.21.15.49:0/576961735 shutdown_connections 2023-11-06T02:09:19.891 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:19.889+0000 7feb2ee9b640 1 -- 172.21.15.49:0/576961735 wait complete. 2023-11-06T02:09:19.906 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T02:08:19.252222+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T02:08:24.776211+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T02:08:28.936178+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T02:08:29.563468+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T02:09:19.907 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T02:09:20.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.037+0000 7fd3989f9640 1 Processor -- start 2023-11-06T02:09:20.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.037+0000 7fd3989f9640 1 -- start start 2023-11-06T02:09:20.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.041+0000 7fd3989f9640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd394070870 0x7fd3940b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:20.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.041+0000 7fd3989f9640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3940ae360 con 0x7fd39406d150 2023-11-06T02:09:20.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.041+0000 7fd3989f9640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3940ae640 con 0x7fd39415a880 2023-11-06T02:09:20.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.041+0000 7fd3989f9640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3940ae920 con 0x7fd394070870 2023-11-06T02:09:20.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.041+0000 7fd391d9b640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd394070870 0x7fd3940b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:20.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.041+0000 7fd391d9b640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd394070870 0x7fd3940b15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41744/0 (socket says 172.21.15.49:41744) 2023-11-06T02:09:20.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.041+0000 7fd391d9b640 1 -- 172.21.15.49:0/2535398412 learned_addr learned my addr 172.21.15.49:0/2535398412 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:20.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.041+0000 7fd391d9b640 1 -- 172.21.15.49:0/2535398412 >> v1:172.21.15.150:6789/0 conn(0x7fd39415a880 legacy=0x7fd394070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:20.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.041+0000 7fd391d9b640 1 -- 172.21.15.49:0/2535398412 >> v1:172.21.15.49:6789/0 conn(0x7fd39406d150 legacy=0x7fd39406d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:20.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.041+0000 7fd391d9b640 1 -- 172.21.15.49:0/2535398412 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3940aec00 con 0x7fd394070870 2023-11-06T02:09:20.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.041+0000 7fd391d9b640 1 --2- 172.21.15.49:0/2535398412 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd394070870 0x7fd3940b15c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fd384009870 tx=0x7fd38401c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a541ff688aeba30 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:20.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.041+0000 7fd39159a640 1 -- 172.21.15.49:0/2535398412 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3840ac620 con 0x7fd394070870 2023-11-06T02:09:20.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.045+0000 7fd39159a640 1 -- 172.21.15.49:0/2535398412 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3840ac780 con 0x7fd394070870 2023-11-06T02:09:20.042 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.045+0000 7fd39159a640 1 -- 172.21.15.49:0/2535398412 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3840acb10 con 0x7fd394070870 2023-11-06T02:09:20.042 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.045+0000 7fd3989f9640 1 -- 172.21.15.49:0/2535398412 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd394070870 msgr2=0x7fd3940b15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:20.043 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.045+0000 7fd3989f9640 1 --2- 172.21.15.49:0/2535398412 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd394070870 0x7fd3940b15c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fd384009870 tx=0x7fd38401c4b0 comp rx=0 tx=0).stop 2023-11-06T02:09:20.043 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.045+0000 7fd3989f9640 1 -- 172.21.15.49:0/2535398412 shutdown_connections 2023-11-06T02:09:20.043 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.045+0000 7fd3989f9640 1 --2- 172.21.15.49:0/2535398412 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd394070870 0x7fd3940b15c0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:20.043 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.045+0000 7fd3989f9640 1 -- 172.21.15.49:0/2535398412 >> 172.21.15.49:0/2535398412 conn(0x7fd39406a370 msgr2=0x7fd3940ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:20.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.045+0000 7fd3989f9640 1 -- 172.21.15.49:0/2535398412 shutdown_connections 2023-11-06T02:09:20.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.045+0000 7fd3989f9640 1 -- 172.21.15.49:0/2535398412 wait complete. 2023-11-06T02:09:20.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.045+0000 7fd3989f9640 1 Processor -- start 2023-11-06T02:09:20.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.045+0000 7fd3989f9640 1 -- start start 2023-11-06T02:09:20.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.045+0000 7fd3989f9640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd39406d150 0x7fd3940c3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:20.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.045+0000 7fd3989f9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd39415a880 0x7fd3940cb520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:20.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.045+0000 7fd3989f9640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd39414b970 0x7fd39414be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:20.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.045+0000 7fd3989f9640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd39414a960 con 0x7fd39415a880 2023-11-06T02:09:20.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.045+0000 7fd3989f9640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd39414abd0 con 0x7fd39414b970 2023-11-06T02:09:20.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.045+0000 7fd3989f9640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd39414aeb0 con 0x7fd39406d150 2023-11-06T02:09:20.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.045+0000 7fd391d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd39415a880 0x7fd3940cb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:20.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.045+0000 7fd391d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd39415a880 0x7fd3940cb520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41754/0 (socket says 172.21.15.49:41754) 2023-11-06T02:09:20.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.045+0000 7fd391d9b640 1 -- 172.21.15.49:0/1461098612 learned_addr learned my addr 172.21.15.49:0/1461098612 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:20.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.045+0000 7fd39259c640 1 --2- 172.21.15.49:0/1461098612 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd39406d150 0x7fd3940c3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:20.049 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.045+0000 7fd392d9d640 1 --2- 172.21.15.49:0/1461098612 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd39414b970 0x7fd39414be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:20.049 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.045+0000 7fd391d9b640 1 -- 172.21.15.49:0/1461098612 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd39406d150 msgr2=0x7fd3940c3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:20.049 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.045+0000 7fd391d9b640 1 --2- 172.21.15.49:0/1461098612 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd39406d150 0x7fd3940c3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:20.049 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.045+0000 7fd391d9b640 1 -- 172.21.15.49:0/1461098612 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd39414b970 msgr2=0x7fd39414be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:20.049 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.045+0000 7fd391d9b640 1 --2- 172.21.15.49:0/1461098612 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd39414b970 0x7fd39414be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:20.050 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.045+0000 7fd391d9b640 1 -- 172.21.15.49:0/1461098612 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3840094d0 con 0x7fd39415a880 2023-11-06T02:09:20.050 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.045+0000 7fd391d9b640 1 --2- 172.21.15.49:0/1461098612 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd39415a880 0x7fd3940cb520 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fd3840af040 tx=0x7fd384019250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:20.050 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.045+0000 7fd38affd640 1 -- 172.21.15.49:0/1461098612 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd384004220 con 0x7fd39415a880 2023-11-06T02:09:20.050 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.045+0000 7fd38affd640 1 -- 172.21.15.49:0/1461098612 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd384004380 con 0x7fd39415a880 2023-11-06T02:09:20.050 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.045+0000 7fd38affd640 1 -- 172.21.15.49:0/1461098612 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd384003840 con 0x7fd39415a880 2023-11-06T02:09:20.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.045+0000 7fd3989f9640 1 -- 172.21.15.49:0/1461098612 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd39414b190 con 0x7fd39415a880 2023-11-06T02:09:20.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.049+0000 7fd3989f9640 1 -- 172.21.15.49:0/1461098612 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3940c3ab0 con 0x7fd39415a880 2023-11-06T02:09:20.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.049+0000 7fd3989f9640 1 -- 172.21.15.49:0/1461098612 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd394074040 con 0x7fd39415a880 2023-11-06T02:09:20.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.049+0000 7fd38affd640 1 -- 172.21.15.49:0/1461098612 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fd3840b3070 con 0x7fd39415a880 2023-11-06T02:09:20.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.049+0000 7fd38affd640 1 --2- 172.21.15.49:0/1461098612 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd374065a60 0x7fd374067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:20.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.049+0000 7fd38affd640 1 -- 172.21.15.49:0/1461098612 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fd38411cfe0 con 0x7fd39415a880 2023-11-06T02:09:20.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.049+0000 7fd39259c640 1 --2- 172.21.15.49:0/1461098612 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd374065a60 0x7fd374067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:20.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.049+0000 7fd39259c640 1 --2- 172.21.15.49:0/1461098612 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd374065a60 0x7fd374067f20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd39415c020 tx=0x7fd37c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:20.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.053+0000 7fd38affd640 1 -- 172.21.15.49:0/1461098612 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fd3840ea890 con 0x7fd39415a880 2023-11-06T02:09:20.205 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.209+0000 7fd3989f9640 1 -- 172.21.15.49:0/1461098612 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd39406c430 con 0x7fd39415a880 2023-11-06T02:09:20.206 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.209+0000 7fd38affd640 1 -- 172.21.15.49:0/1461098612 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd3840b10a0 con 0x7fd39415a880 2023-11-06T02:09:20.206 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2023-11-06T02:09:20.209 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.209+0000 7fd3989f9640 1 -- 172.21.15.49:0/1461098612 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd374065a60 msgr2=0x7fd374067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:20.209 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.209+0000 7fd3989f9640 1 --2- 172.21.15.49:0/1461098612 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd374065a60 0x7fd374067f20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd39415c020 tx=0x7fd37c009040 comp rx=0 tx=0).stop 2023-11-06T02:09:20.209 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.209+0000 7fd3989f9640 1 -- 172.21.15.49:0/1461098612 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd39415a880 msgr2=0x7fd3940cb520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:20.209 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.209+0000 7fd3989f9640 1 --2- 172.21.15.49:0/1461098612 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd39415a880 0x7fd3940cb520 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fd3840af040 tx=0x7fd384019250 comp rx=0 tx=0).stop 2023-11-06T02:09:20.209 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.209+0000 7fd3989f9640 1 -- 172.21.15.49:0/1461098612 shutdown_connections 2023-11-06T02:09:20.210 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.209+0000 7fd3989f9640 1 --2- 172.21.15.49:0/1461098612 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd374065a60 0x7fd374067f20 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:20.210 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.209+0000 7fd3989f9640 1 --2- 172.21.15.49:0/1461098612 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd39415a880 0x7fd3940cb520 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:20.210 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.209+0000 7fd3989f9640 1 --2- 172.21.15.49:0/1461098612 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd39414b970 0x7fd39414be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:20.210 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.209+0000 7fd3989f9640 1 --2- 172.21.15.49:0/1461098612 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd39406d150 0x7fd3940c3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:20.211 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.209+0000 7fd3989f9640 1 -- 172.21.15.49:0/1461098612 >> 172.21.15.49:0/1461098612 conn(0x7fd39406a370 msgr2=0x7fd39406c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:20.211 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.209+0000 7fd3989f9640 1 -- 172.21.15.49:0/1461098612 shutdown_connections 2023-11-06T02:09:20.211 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.209+0000 7fd3989f9640 1 -- 172.21.15.49:0/1461098612 wait complete. 2023-11-06T02:09:20.226 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T02:09:20.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.349+0000 7f594db56640 1 Processor -- start 2023-11-06T02:09:20.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.349+0000 7f594db56640 1 -- start start 2023-11-06T02:09:20.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.349+0000 7f594db56640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f59480cb2d0 0x7f59480cca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:20.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.349+0000 7f594db56640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f59480b04e0 0x7f59480b08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:20.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.349+0000 7f594db56640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59480cc5c0 con 0x7f594815a9e0 2023-11-06T02:09:20.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.349+0000 7f594db56640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59480cb760 con 0x7f59480cb2d0 2023-11-06T02:09:20.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.349+0000 7f594db56640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59480b0e30 con 0x7f59480b04e0 2023-11-06T02:09:20.350 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.349+0000 7f5946ffd640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f59480cb2d0 0x7f59480cca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:20.350 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.349+0000 7f5946ffd640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f59480cb2d0 0x7f59480cca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41756/0 (socket says 172.21.15.49:41756) 2023-11-06T02:09:20.350 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.349+0000 7f5946ffd640 1 -- 172.21.15.49:0/2105161580 learned_addr learned my addr 172.21.15.49:0/2105161580 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:20.350 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.349+0000 7f5947fff640 1 --2- 172.21.15.49:0/2105161580 >> v2:172.21.15.150:3300/0 conn(0x7f59480b04e0 0x7f59480b08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:20.351 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.349+0000 7f5946ffd640 1 -- 172.21.15.49:0/2105161580 >> v2:172.21.15.150:3300/0 conn(0x7f59480b04e0 msgr2=0x7f59480b08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:20.351 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.349+0000 7f5946ffd640 1 --2- 172.21.15.49:0/2105161580 >> v2:172.21.15.150:3300/0 conn(0x7f59480b04e0 0x7f59480b08f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:20.351 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.349+0000 7f5946ffd640 1 -- 172.21.15.49:0/2105161580 >> v1:172.21.15.150:6789/0 conn(0x7f594815a9e0 legacy=0x7f59480afbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:20.351 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.349+0000 7f5946ffd640 1 -- 172.21.15.49:0/2105161580 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59480cb9b0 con 0x7f59480cb2d0 2023-11-06T02:09:20.353 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.353+0000 7f5946ffd640 1 --2- 172.21.15.49:0/2105161580 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f59480cb2d0 0x7f59480cca50 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f59480d05c0 tx=0x7f593c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12f3dfbfc5fe4298 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:20.353 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.353+0000 7f59467fc640 1 -- 172.21.15.49:0/2105161580 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f593c0aa5f0 con 0x7f59480cb2d0 2023-11-06T02:09:20.353 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.353+0000 7f59467fc640 1 -- 172.21.15.49:0/2105161580 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f593c0aa750 con 0x7f59480cb2d0 2023-11-06T02:09:20.354 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.353+0000 7f59467fc640 1 -- 172.21.15.49:0/2105161580 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f593c0aaae0 con 0x7f59480cb2d0 2023-11-06T02:09:20.354 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.353+0000 7f594db56640 1 -- 172.21.15.49:0/2105161580 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f59480cb2d0 msgr2=0x7f59480cca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:20.354 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.353+0000 7f594db56640 1 --2- 172.21.15.49:0/2105161580 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f59480cb2d0 0x7f59480cca50 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f59480d05c0 tx=0x7f593c01c490 comp rx=0 tx=0).stop 2023-11-06T02:09:20.354 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.353+0000 7f594db56640 1 -- 172.21.15.49:0/2105161580 shutdown_connections 2023-11-06T02:09:20.354 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.353+0000 7f594db56640 1 --2- 172.21.15.49:0/2105161580 >> v2:172.21.15.150:3300/0 conn(0x7f59480b04e0 0x7f59480b08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:20.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.353+0000 7f594db56640 1 --2- 172.21.15.49:0/2105161580 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f59480cb2d0 0x7f59480cca50 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:20.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.353+0000 7f594db56640 1 -- 172.21.15.49:0/2105161580 >> 172.21.15.49:0/2105161580 conn(0x7f5948098a40 msgr2=0x7f59480cf860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:20.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.353+0000 7f594db56640 1 -- 172.21.15.49:0/2105161580 shutdown_connections 2023-11-06T02:09:20.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.353+0000 7f594db56640 1 -- 172.21.15.49:0/2105161580 wait complete. 2023-11-06T02:09:20.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.353+0000 7f594db56640 1 Processor -- start 2023-11-06T02:09:20.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.353+0000 7f594db56640 1 -- start start 2023-11-06T02:09:20.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.357+0000 7f594db56640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f59480b04e0 0x7f59480cde40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:20.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.357+0000 7f594db56640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f59480cb2d0 0x7f594815db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:20.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.357+0000 7f594db56640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f594815a9e0 0x7f5948151430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:20.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.357+0000 7f594db56640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f594815e090 con 0x7f594815a9e0 2023-11-06T02:09:20.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.357+0000 7f594db56640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59480ce490 con 0x7f59480cb2d0 2023-11-06T02:09:20.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.357+0000 7f594db56640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59480ce770 con 0x7f59480b04e0 2023-11-06T02:09:20.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.357+0000 7f5947fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f594815a9e0 0x7f5948151430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:20.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.357+0000 7f5947fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f594815a9e0 0x7f5948151430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41772/0 (socket says 172.21.15.49:41772) 2023-11-06T02:09:20.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.357+0000 7f5947fff640 1 -- 172.21.15.49:0/751685226 learned_addr learned my addr 172.21.15.49:0/751685226 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:20.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.357+0000 7f5946ffd640 1 --2- 172.21.15.49:0/751685226 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f59480cb2d0 0x7f594815db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:20.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.357+0000 7f59477fe640 1 --2- 172.21.15.49:0/751685226 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f59480b04e0 0x7f59480cde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:20.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.357+0000 7f5947fff640 1 -- 172.21.15.49:0/751685226 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f59480b04e0 msgr2=0x7f59480cde40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:20.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.357+0000 7f5947fff640 1 --2- 172.21.15.49:0/751685226 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f59480b04e0 0x7f59480cde40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:20.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.357+0000 7f5947fff640 1 -- 172.21.15.49:0/751685226 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f59480cb2d0 msgr2=0x7f594815db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:20.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.357+0000 7f5947fff640 1 --2- 172.21.15.49:0/751685226 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f59480cb2d0 0x7f594815db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:20.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.357+0000 7f5947fff640 1 -- 172.21.15.49:0/751685226 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f593c0094d0 con 0x7f594815a9e0 2023-11-06T02:09:20.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.357+0000 7f5946ffd640 1 --2- 172.21.15.49:0/751685226 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f59480cb2d0 0x7f594815db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:09:20.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.357+0000 7f5947fff640 1 --2- 172.21.15.49:0/751685226 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f594815a9e0 0x7f5948151430 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f593800d560 tx=0x7f593800da20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:20.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.357+0000 7f592ffff640 1 -- 172.21.15.49:0/751685226 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59380125f0 con 0x7f594815a9e0 2023-11-06T02:09:20.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.357+0000 7f592ffff640 1 -- 172.21.15.49:0/751685226 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5938012770 con 0x7f594815a9e0 2023-11-06T02:09:20.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.357+0000 7f592ffff640 1 -- 172.21.15.49:0/751685226 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5938012b20 con 0x7f594815a9e0 2023-11-06T02:09:20.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.357+0000 7f594db56640 1 -- 172.21.15.49:0/751685226 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59480cea50 con 0x7f594815a9e0 2023-11-06T02:09:20.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.357+0000 7f594db56640 1 -- 172.21.15.49:0/751685226 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5948162e10 con 0x7f594815a9e0 2023-11-06T02:09:20.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.357+0000 7f594db56640 1 -- 172.21.15.49:0/751685226 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5948163000 con 0x7f594815a9e0 2023-11-06T02:09:20.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.357+0000 7f592ffff640 1 -- 172.21.15.49:0/751685226 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f5938012c80 con 0x7f594815a9e0 2023-11-06T02:09:20.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.357+0000 7f592ffff640 1 --2- 172.21.15.49:0/751685226 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5934065a60 0x7f5934067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:20.365 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.357+0000 7f592ffff640 1 -- 172.21.15.49:0/751685226 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f5938083760 con 0x7f594815a9e0 2023-11-06T02:09:20.365 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.357+0000 7f59477fe640 1 --2- 172.21.15.49:0/751685226 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5934065a60 0x7f5934067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:20.365 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.361+0000 7f59477fe640 1 --2- 172.21.15.49:0/751685226 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5934065a60 0x7f5934067f20 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5940005530 tx=0x7f5940009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:20.366 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.361+0000 7f592ffff640 1 -- 172.21.15.49:0/751685226 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f5938051010 con 0x7f594815a9e0 2023-11-06T02:09:20.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.513+0000 7f594db56640 1 -- 172.21.15.49:0/751685226 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f594806e430 con 0x7f594815a9e0 2023-11-06T02:09:20.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.513+0000 7f592ffff640 1 -- 172.21.15.49:0/751685226 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5938054630 con 0x7f594815a9e0 2023-11-06T02:09:20.512 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 8 2023-11-06T02:09:20.514 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.517+0000 7f594db56640 1 -- 172.21.15.49:0/751685226 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5934065a60 msgr2=0x7f5934067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:20.514 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.517+0000 7f594db56640 1 --2- 172.21.15.49:0/751685226 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5934065a60 0x7f5934067f20 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5940005530 tx=0x7f5940009040 comp rx=0 tx=0).stop 2023-11-06T02:09:20.514 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.517+0000 7f594db56640 1 -- 172.21.15.49:0/751685226 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f594815a9e0 msgr2=0x7f5948151430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:20.515 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.517+0000 7f594db56640 1 --2- 172.21.15.49:0/751685226 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f594815a9e0 0x7f5948151430 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f593800d560 tx=0x7f593800da20 comp rx=0 tx=0).stop 2023-11-06T02:09:20.515 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.517+0000 7f594db56640 1 -- 172.21.15.49:0/751685226 shutdown_connections 2023-11-06T02:09:20.515 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.517+0000 7f594db56640 1 --2- 172.21.15.49:0/751685226 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5934065a60 0x7f5934067f20 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:20.515 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.517+0000 7f594db56640 1 --2- 172.21.15.49:0/751685226 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f594815a9e0 0x7f5948151430 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:20.515 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.517+0000 7f594db56640 1 --2- 172.21.15.49:0/751685226 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f59480cb2d0 0x7f594815db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:20.516 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.517+0000 7f594db56640 1 --2- 172.21.15.49:0/751685226 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f59480b04e0 0x7f59480cde40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:20.516 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.517+0000 7f594db56640 1 -- 172.21.15.49:0/751685226 >> 172.21.15.49:0/751685226 conn(0x7f5948098a40 msgr2=0x7f59480ae370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:20.516 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.517+0000 7f594db56640 1 -- 172.21.15.49:0/751685226 shutdown_connections 2023-11-06T02:09:20.516 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.517+0000 7f594db56640 1 -- 172.21.15.49:0/751685226 wait complete. 2023-11-06T02:09:20.530 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T02:09:20.650 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.653+0000 7f323c6bf640 1 Processor -- start 2023-11-06T02:09:20.651 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.653+0000 7f323c6bf640 1 -- start start 2023-11-06T02:09:20.652 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.653+0000 7f323c6bf640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f323415a880 0x7f32340b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:20.653 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.653+0000 7f323c6bf640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f323406d860 0x7f3234070610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:20.653 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.653+0000 7f323c6bf640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32340ae360 con 0x7f3234070160 2023-11-06T02:09:20.653 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.653+0000 7f323c6bf640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3234070b50 con 0x7f323406d860 2023-11-06T02:09:20.653 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.653+0000 7f323c6bf640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32340ae640 con 0x7f323415a880 2023-11-06T02:09:20.654 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.653+0000 7f3239c5a640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f3234070160 0x7f323406d150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:35466/0 (socket says 172.21.15.49:35466) 2023-11-06T02:09:20.654 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.653+0000 7f3239c5a640 1 -- 172.21.15.49:0/3510594821 learned_addr learned my addr 172.21.15.49:0/3510594821 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:09:20.654 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.653+0000 7f323ac5c640 1 --2- 172.21.15.49:0/3510594821 >> v2:172.21.15.49:3300/0 conn(0x7f323406d860 0x7f3234070610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:20.654 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.653+0000 7f323a45b640 1 --2- 172.21.15.49:0/3510594821 >> v2:172.21.15.150:3300/0 conn(0x7f323415a880 0x7f32340b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:20.654 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.653+0000 7f3239459640 1 -- 172.21.15.49:0/3510594821 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2926054756 0 0) 0x7f32340ae360 con 0x7f3234070160 2023-11-06T02:09:20.655 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.653+0000 7f3239459640 1 -- 172.21.15.49:0/3510594821 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3220003530 con 0x7f3234070160 2023-11-06T02:09:20.655 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.653+0000 7f323ac5c640 1 -- 172.21.15.49:0/3510594821 >> v2:172.21.15.150:3300/0 conn(0x7f323415a880 msgr2=0x7f32340b15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:20.655 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.653+0000 7f323ac5c640 1 --2- 172.21.15.49:0/3510594821 >> v2:172.21.15.150:3300/0 conn(0x7f323415a880 0x7f32340b15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:20.655 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.653+0000 7f323ac5c640 1 -- 172.21.15.49:0/3510594821 >> v1:172.21.15.49:6789/0 conn(0x7f3234070160 legacy=0x7f323406d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:20.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.653+0000 7f323ac5c640 1 -- 172.21.15.49:0/3510594821 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32340ae920 con 0x7f323406d860 2023-11-06T02:09:20.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.653+0000 7f323a45b640 1 --2- 172.21.15.49:0/3510594821 >> v2:172.21.15.150:3300/0 conn(0x7f323415a880 0x7f32340b15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:09:20.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.657+0000 7f323ac5c640 1 --2- 172.21.15.49:0/3510594821 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f323406d860 0x7f3234070610 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f32340cafc0 tx=0x7f322c01c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e641c93838ff9082 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:20.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.657+0000 7f3239459640 1 -- 172.21.15.49:0/3510594821 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f322c0a58e0 con 0x7f323406d860 2023-11-06T02:09:20.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.657+0000 7f3239459640 1 -- 172.21.15.49:0/3510594821 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f322c0a5a40 con 0x7f323406d860 2023-11-06T02:09:20.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.657+0000 7f3239459640 1 -- 172.21.15.49:0/3510594821 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f322c0a5dd0 con 0x7f323406d860 2023-11-06T02:09:20.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.657+0000 7f323c6bf640 1 -- 172.21.15.49:0/3510594821 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f323406d860 msgr2=0x7f3234070610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:20.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.657+0000 7f323c6bf640 1 --2- 172.21.15.49:0/3510594821 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f323406d860 0x7f3234070610 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f32340cafc0 tx=0x7f322c01c910 comp rx=0 tx=0).stop 2023-11-06T02:09:20.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.657+0000 7f323c6bf640 1 -- 172.21.15.49:0/3510594821 shutdown_connections 2023-11-06T02:09:20.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.657+0000 7f323c6bf640 1 --2- 172.21.15.49:0/3510594821 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f323406d860 0x7f3234070610 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:20.662 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.657+0000 7f323c6bf640 1 --2- 172.21.15.49:0/3510594821 >> v2:172.21.15.150:3300/0 conn(0x7f323415a880 0x7f32340b15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:20.662 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.657+0000 7f323c6bf640 1 -- 172.21.15.49:0/3510594821 >> 172.21.15.49:0/3510594821 conn(0x7f323406a370 msgr2=0x7f32340ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:20.662 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.657+0000 7f323c6bf640 1 -- 172.21.15.49:0/3510594821 shutdown_connections 2023-11-06T02:09:20.662 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.657+0000 7f323c6bf640 1 -- 172.21.15.49:0/3510594821 wait complete. 2023-11-06T02:09:20.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.657+0000 7f323c6bf640 1 Processor -- start 2023-11-06T02:09:20.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.657+0000 7f323c6bf640 1 -- start start 2023-11-06T02:09:20.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.657+0000 7f323c6bf640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f323406d860 0x7f3234147be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:20.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.657+0000 7f323c6bf640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3234070160 0x7f323414aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:20.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.657+0000 7f323c6bf640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f323415a880 0x7f3234149540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:20.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.657+0000 7f323c6bf640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f323414b6e0 con 0x7f3234070160 2023-11-06T02:09:20.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.657+0000 7f323c6bf640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32341511d0 con 0x7f323406d860 2023-11-06T02:09:20.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.657+0000 7f323c6bf640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3234149a80 con 0x7f323415a880 2023-11-06T02:09:20.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.657+0000 7f3239c5a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3234070160 0x7f323414aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:20.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.657+0000 7f3239c5a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3234070160 0x7f323414aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41786/0 (socket says 172.21.15.49:41786) 2023-11-06T02:09:20.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.657+0000 7f3239c5a640 1 -- 172.21.15.49:0/3330970561 learned_addr learned my addr 172.21.15.49:0/3330970561 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:20.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.661+0000 7f323a45b640 1 --2- 172.21.15.49:0/3330970561 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f323406d860 0x7f3234147be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:20.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.661+0000 7f323ac5c640 1 --2- 172.21.15.49:0/3330970561 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f323415a880 0x7f3234149540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:20.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.661+0000 7f3239c5a640 1 -- 172.21.15.49:0/3330970561 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f323415a880 msgr2=0x7f3234149540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:20.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.661+0000 7f3239c5a640 1 --2- 172.21.15.49:0/3330970561 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f323415a880 0x7f3234149540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:20.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.661+0000 7f3239c5a640 1 -- 172.21.15.49:0/3330970561 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f323406d860 msgr2=0x7f3234147be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:20.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.661+0000 7f3239c5a640 1 --2- 172.21.15.49:0/3330970561 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f323406d860 0x7f3234147be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:20.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.661+0000 7f3239c5a640 1 -- 172.21.15.49:0/3330970561 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f322c009a70 con 0x7f3234070160 2023-11-06T02:09:20.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.661+0000 7f3239c5a640 1 --2- 172.21.15.49:0/3330970561 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3234070160 0x7f323414aa40 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f322400c450 tx=0x7f322400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:20.667 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.661+0000 7f322affd640 1 -- 172.21.15.49:0/3330970561 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3224005cf0 con 0x7f3234070160 2023-11-06T02:09:20.667 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.661+0000 7f322affd640 1 -- 172.21.15.49:0/3330970561 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3224005e50 con 0x7f3234070160 2023-11-06T02:09:20.667 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.661+0000 7f323c6bf640 1 -- 172.21.15.49:0/3330970561 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f323414b120 con 0x7f3234070160 2023-11-06T02:09:20.667 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.661+0000 7f322affd640 1 -- 172.21.15.49:0/3330970561 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3224007290 con 0x7f3234070160 2023-11-06T02:09:20.667 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.661+0000 7f323c6bf640 1 -- 172.21.15.49:0/3330970561 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3234151420 con 0x7f3234070160 2023-11-06T02:09:20.668 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.661+0000 7f323c6bf640 1 -- 172.21.15.49:0/3330970561 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3234074040 con 0x7f3234070160 2023-11-06T02:09:20.668 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.661+0000 7f322affd640 1 -- 172.21.15.49:0/3330970561 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f32240073f0 con 0x7f3234070160 2023-11-06T02:09:20.668 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.661+0000 7f322affd640 1 --2- 172.21.15.49:0/3330970561 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3218065a60 0x7f3218067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:20.668 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.661+0000 7f322affd640 1 -- 172.21.15.49:0/3330970561 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f3224082860 con 0x7f3234070160 2023-11-06T02:09:20.668 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.661+0000 7f323a45b640 1 --2- 172.21.15.49:0/3330970561 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3218065a60 0x7f3218067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:20.669 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.661+0000 7f323a45b640 1 --2- 172.21.15.49:0/3330970561 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3218065a60 0x7f3218067f20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f323415dd70 tx=0x7f3230008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:20.669 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.665+0000 7f322affd640 1 -- 172.21.15.49:0/3330970561 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f3224050110 con 0x7f3234070160 2023-11-06T02:09:20.818 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.820+0000 7f323c6bf640 1 -- 172.21.15.49:0/3330970561 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f32340cdff0 con 0x7f3234070160 2023-11-06T02:09:20.818 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.820+0000 7f322affd640 1 -- 172.21.15.49:0/3330970561 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3224017020 con 0x7f3234070160 2023-11-06T02:09:20.819 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2023-11-06T02:09:20.820 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.820+0000 7f323c6bf640 1 -- 172.21.15.49:0/3330970561 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3218065a60 msgr2=0x7f3218067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:20.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.820+0000 7f323c6bf640 1 --2- 172.21.15.49:0/3330970561 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3218065a60 0x7f3218067f20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f323415dd70 tx=0x7f3230008040 comp rx=0 tx=0).stop 2023-11-06T02:09:20.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.820+0000 7f323c6bf640 1 -- 172.21.15.49:0/3330970561 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3234070160 msgr2=0x7f323414aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:20.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.820+0000 7f323c6bf640 1 --2- 172.21.15.49:0/3330970561 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3234070160 0x7f323414aa40 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f322400c450 tx=0x7f322400c910 comp rx=0 tx=0).stop 2023-11-06T02:09:20.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.820+0000 7f323c6bf640 1 -- 172.21.15.49:0/3330970561 shutdown_connections 2023-11-06T02:09:20.822 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.820+0000 7f323c6bf640 1 --2- 172.21.15.49:0/3330970561 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3218065a60 0x7f3218067f20 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:20.822 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.820+0000 7f323c6bf640 1 --2- 172.21.15.49:0/3330970561 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3234070160 0x7f323414aa40 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:20.822 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.820+0000 7f323c6bf640 1 --2- 172.21.15.49:0/3330970561 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f323415a880 0x7f3234149540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:20.822 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.820+0000 7f323c6bf640 1 --2- 172.21.15.49:0/3330970561 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f323406d860 0x7f3234147be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:20.822 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.820+0000 7f323c6bf640 1 -- 172.21.15.49:0/3330970561 >> 172.21.15.49:0/3330970561 conn(0x7f323406a370 msgr2=0x7f32341531c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:20.823 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.820+0000 7f323c6bf640 1 -- 172.21.15.49:0/3330970561 shutdown_connections 2023-11-06T02:09:20.823 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.820+0000 7f323c6bf640 1 -- 172.21.15.49:0/3330970561 wait complete. 2023-11-06T02:09:20.838 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T02:09:20.966 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.968+0000 7fb219387640 1 Processor -- start 2023-11-06T02:09:20.966 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.968+0000 7fb219387640 1 -- start start 2023-11-06T02:09:20.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.968+0000 7fb219387640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2140b0930 0x7fb21415dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:20.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.968+0000 7fb219387640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb21406c5a0 con 0x7fb21415a990 2023-11-06T02:09:20.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.968+0000 7fb219387640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb21406c880 con 0x7fb2141595a0 2023-11-06T02:09:20.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.968+0000 7fb219387640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb21415e250 con 0x7fb2140b0930 2023-11-06T02:09:20.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.968+0000 7fb2127fc640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fb21415a990 0x7fb2140b0220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:35482/0 (socket says 172.21.15.49:35482) 2023-11-06T02:09:20.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.968+0000 7fb2127fc640 1 -- 172.21.15.49:0/3386680784 learned_addr learned my addr 172.21.15.49:0/3386680784 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:09:20.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.968+0000 7fb211ffb640 1 -- 172.21.15.49:0/3386680784 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2686711328 0 0) 0x7fb21406c5a0 con 0x7fb21415a990 2023-11-06T02:09:20.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.968+0000 7fb2137fe640 1 --2- 172.21.15.49:0/3386680784 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2140b0930 0x7fb21415dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:20.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.968+0000 7fb211ffb640 1 -- 172.21.15.49:0/3386680784 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1fc003530 con 0x7fb21415a990 2023-11-06T02:09:20.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.968+0000 7fb211ffb640 1 -- 172.21.15.49:0/3386680784 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 704530403 0 0) 0x7fb1fc003530 con 0x7fb21415a990 2023-11-06T02:09:20.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.968+0000 7fb211ffb640 1 -- 172.21.15.49:0/3386680784 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2140b0930 msgr2=0x7fb21415dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:20.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.968+0000 7fb211ffb640 1 --2- 172.21.15.49:0/3386680784 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2140b0930 0x7fb21415dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:20.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.968+0000 7fb211ffb640 1 -- 172.21.15.49:0/3386680784 >> v1:172.21.15.150:6789/0 conn(0x7fb2141595a0 legacy=0x7fb21406d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:20.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.968+0000 7fb211ffb640 1 -- 172.21.15.49:0/3386680784 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb21406cb60 con 0x7fb21415a990 2023-11-06T02:09:20.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.968+0000 7fb211ffb640 1 -- 172.21.15.49:0/3386680784 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7fb208001680 con 0x7fb21415a990 2023-11-06T02:09:20.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.968+0000 7fb211ffb640 1 -- 172.21.15.49:0/3386680784 >> v1:172.21.15.49:6789/0 conn(0x7fb21415a990 legacy=0x7fb2140b0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:20.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.968+0000 7fb211ffb640 1 --2- 172.21.15.49:0/3386680784 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2140b0930 0x7fb1fc0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:20.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.968+0000 7fb211ffb640 1 --2- 172.21.15.49:0/3386680784 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1fc003fe0 0x7fb1fc0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:20.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.968+0000 7fb211ffb640 1 --2- 172.21.15.49:0/3386680784 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb1fc004d10 0x7fb1fc005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:20.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.968+0000 7fb211ffb640 1 -- 172.21.15.49:0/3386680784 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb21415e250 con 0x7fb1fc003fe0 2023-11-06T02:09:20.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.968+0000 7fb211ffb640 1 -- 172.21.15.49:0/3386680784 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1fc00bb50 con 0x7fb1fc004d10 2023-11-06T02:09:20.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.968+0000 7fb211ffb640 1 -- 172.21.15.49:0/3386680784 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1fc00bd10 con 0x7fb2140b0930 2023-11-06T02:09:20.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.968+0000 7fb2127fc640 1 --2- 172.21.15.49:0/3386680784 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1fc003fe0 0x7fb1fc0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:20.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.968+0000 7fb2137fe640 1 --2- 172.21.15.49:0/3386680784 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb1fc004d10 0x7fb1fc005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:20.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.968+0000 7fb212ffd640 1 --2- 172.21.15.49:0/3386680784 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2140b0930 0x7fb1fc0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:20.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.968+0000 7fb2127fc640 1 -- 172.21.15.49:0/3386680784 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2140b0930 msgr2=0x7fb1fc0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:20.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.968+0000 7fb2127fc640 1 --2- 172.21.15.49:0/3386680784 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2140b0930 0x7fb1fc0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:20.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.968+0000 7fb2127fc640 1 -- 172.21.15.49:0/3386680784 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb1fc004d10 msgr2=0x7fb1fc005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:20.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.968+0000 7fb2127fc640 1 --2- 172.21.15.49:0/3386680784 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb1fc004d10 0x7fb1fc005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:20.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.968+0000 7fb2127fc640 1 -- 172.21.15.49:0/3386680784 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1fc003530 con 0x7fb1fc003fe0 2023-11-06T02:09:20.985 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.972+0000 7fb2127fc640 1 --2- 172.21.15.49:0/3386680784 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1fc003fe0 0x7fb1fc0047d0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fb20800c210 tx=0x7fb20801f370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24367676565d488d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:20.985 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.972+0000 7fb211ffb640 1 -- 172.21.15.49:0/3386680784 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb208005ae0 con 0x7fb1fc003fe0 2023-11-06T02:09:20.985 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.972+0000 7fb211ffb640 1 -- 172.21.15.49:0/3386680784 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2080ab070 con 0x7fb1fc003fe0 2023-11-06T02:09:20.986 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.972+0000 7fb211ffb640 1 -- 172.21.15.49:0/3386680784 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb208007070 con 0x7fb1fc003fe0 2023-11-06T02:09:20.986 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.972+0000 7fb219387640 1 -- 172.21.15.49:0/3386680784 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1fc003fe0 msgr2=0x7fb1fc0047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:20.987 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.972+0000 7fb219387640 1 --2- 172.21.15.49:0/3386680784 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1fc003fe0 0x7fb1fc0047d0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fb20800c210 tx=0x7fb20801f370 comp rx=0 tx=0).stop 2023-11-06T02:09:20.987 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.972+0000 7fb2137fe640 1 -- 172.21.15.49:0/3386680784 reap_dead start 2023-11-06T02:09:20.987 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.972+0000 7fb219387640 1 -- 172.21.15.49:0/3386680784 shutdown_connections 2023-11-06T02:09:20.988 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.972+0000 7fb219387640 1 -- 172.21.15.49:0/3386680784 >> 172.21.15.49:0/3386680784 conn(0x7fb21406a370 msgr2=0x7fb2140ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:20.988 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.972+0000 7fb219387640 1 -- 172.21.15.49:0/3386680784 shutdown_connections 2023-11-06T02:09:20.989 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.972+0000 7fb219387640 1 -- 172.21.15.49:0/3386680784 wait complete. 2023-11-06T02:09:20.989 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.972+0000 7fb219387640 1 Processor -- start 2023-11-06T02:09:20.989 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.972+0000 7fb219387640 1 -- start start 2023-11-06T02:09:20.990 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.972+0000 7fb219387640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb214146010 0x7fb2140cd130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:20.990 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.972+0000 7fb219387640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb2140cd670 0x7fb214157da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:20.990 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.972+0000 7fb219387640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2141582e0 0x7fb2140b2e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:20.991 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.972+0000 7fb219387640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2140cdaf0 con 0x7fb2140cd670 2023-11-06T02:09:20.991 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.972+0000 7fb219387640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2141463f0 con 0x7fb214146010 2023-11-06T02:09:20.992 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.972+0000 7fb219387640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb214150f30 con 0x7fb2141582e0 2023-11-06T02:09:20.992 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.972+0000 7fb2127fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb2140cd670 0x7fb214157da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:20.992 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.972+0000 7fb2127fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb2140cd670 0x7fb214157da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41798/0 (socket says 172.21.15.49:41798) 2023-11-06T02:09:20.993 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.972+0000 7fb2127fc640 1 -- 172.21.15.49:0/3600923078 learned_addr learned my addr 172.21.15.49:0/3600923078 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:20.993 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.976+0000 7fb212ffd640 1 --2- 172.21.15.49:0/3600923078 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb214146010 0x7fb2140cd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:20.993 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.976+0000 7fb2137fe640 1 --2- 172.21.15.49:0/3600923078 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2141582e0 0x7fb2140b2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:20.994 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.976+0000 7fb2127fc640 1 -- 172.21.15.49:0/3600923078 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2141582e0 msgr2=0x7fb2140b2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:20.994 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.976+0000 7fb2127fc640 1 --2- 172.21.15.49:0/3600923078 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2141582e0 0x7fb2140b2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:20.995 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.976+0000 7fb2127fc640 1 -- 172.21.15.49:0/3600923078 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb214146010 msgr2=0x7fb2140cd130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:20.995 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.976+0000 7fb2127fc640 1 --2- 172.21.15.49:0/3600923078 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb214146010 0x7fb2140cd130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:20.995 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.976+0000 7fb2127fc640 1 -- 172.21.15.49:0/3600923078 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb214151210 con 0x7fb2140cd670 2023-11-06T02:09:20.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.976+0000 7fb212ffd640 1 --2- 172.21.15.49:0/3600923078 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb214146010 0x7fb2140cd130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:09:20.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.976+0000 7fb2127fc640 1 --2- 172.21.15.49:0/3600923078 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb2140cd670 0x7fb214157da0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb208025fe0 tx=0x7fb1fc005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:20.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.976+0000 7fb1fb7fe640 1 -- 172.21.15.49:0/3600923078 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb208004ab0 con 0x7fb2140cd670 2023-11-06T02:09:20.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.976+0000 7fb1fb7fe640 1 -- 172.21.15.49:0/3600923078 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb208004c10 con 0x7fb2140cd670 2023-11-06T02:09:20.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.976+0000 7fb219387640 1 -- 172.21.15.49:0/3600923078 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2141514f0 con 0x7fb2140cd670 2023-11-06T02:09:20.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.976+0000 7fb1fb7fe640 1 -- 172.21.15.49:0/3600923078 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb208007070 con 0x7fb2140cd670 2023-11-06T02:09:20.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.976+0000 7fb219387640 1 -- 172.21.15.49:0/3600923078 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb214151ab0 con 0x7fb2140cd670 2023-11-06T02:09:20.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.976+0000 7fb219387640 1 -- 172.21.15.49:0/3600923078 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb21406c210 con 0x7fb2140cd670 2023-11-06T02:09:20.999 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.976+0000 7fb1fb7fe640 1 -- 172.21.15.49:0/3600923078 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fb2080071d0 con 0x7fb2140cd670 2023-11-06T02:09:20.999 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.976+0000 7fb1fb7fe640 1 --2- 172.21.15.49:0/3600923078 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb200065a90 0x7fb200067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:20.999 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.976+0000 7fb1fb7fe640 1 -- 172.21.15.49:0/3600923078 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fb2080b7070 con 0x7fb2140cd670 2023-11-06T02:09:21.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.976+0000 7fb212ffd640 1 --2- 172.21.15.49:0/3600923078 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb200065a90 0x7fb200067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:21.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.976+0000 7fb212ffd640 1 --2- 172.21.15.49:0/3600923078 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb200065a90 0x7fb200067f50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb204000c10 tx=0x7fb204009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:21.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:20.980+0000 7fb1fb7fe640 1 -- 172.21.15.49:0/3600923078 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fb21406c210 con 0x7fb2140cd670 2023-11-06T02:09:21.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.128+0000 7fb219387640 1 -- 172.21.15.49:0/3600923078 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb21406c210 con 0x7fb2140cd670 2023-11-06T02:09:21.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.132+0000 7fb1fb7fe640 1 -- 172.21.15.49:0/3600923078 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb21406c210 con 0x7fb2140cd670 2023-11-06T02:09:21.130 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2023-11-06T02:09:21.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.132+0000 7fb219387640 1 -- 172.21.15.49:0/3600923078 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb200065a90 msgr2=0x7fb200067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:21.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.132+0000 7fb219387640 1 --2- 172.21.15.49:0/3600923078 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb200065a90 0x7fb200067f50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb204000c10 tx=0x7fb204009040 comp rx=0 tx=0).stop 2023-11-06T02:09:21.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.132+0000 7fb219387640 1 -- 172.21.15.49:0/3600923078 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb2140cd670 msgr2=0x7fb214157da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:21.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.132+0000 7fb219387640 1 --2- 172.21.15.49:0/3600923078 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb2140cd670 0x7fb214157da0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb208025fe0 tx=0x7fb1fc005800 comp rx=0 tx=0).stop 2023-11-06T02:09:21.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.132+0000 7fb219387640 1 -- 172.21.15.49:0/3600923078 shutdown_connections 2023-11-06T02:09:21.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.132+0000 7fb219387640 1 --2- 172.21.15.49:0/3600923078 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb200065a90 0x7fb200067f50 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:21.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.132+0000 7fb219387640 1 --2- 172.21.15.49:0/3600923078 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb2140cd670 0x7fb214157da0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:21.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.132+0000 7fb219387640 1 --2- 172.21.15.49:0/3600923078 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2141582e0 0x7fb2140b2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:21.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.132+0000 7fb219387640 1 --2- 172.21.15.49:0/3600923078 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb214146010 0x7fb2140cd130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:21.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.132+0000 7fb219387640 1 -- 172.21.15.49:0/3600923078 >> 172.21.15.49:0/3600923078 conn(0x7fb21406a370 msgr2=0x7fb214072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:21.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.132+0000 7fb219387640 1 -- 172.21.15.49:0/3600923078 shutdown_connections 2023-11-06T02:09:21.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.132+0000 7fb219387640 1 -- 172.21.15.49:0/3600923078 wait complete. 2023-11-06T02:09:21.148 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-06T02:09:21.263 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.264+0000 7f0d9941e640 1 Processor -- start 2023-11-06T02:09:21.263 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.264+0000 7f0d9941e640 1 -- start start 2023-11-06T02:09:21.264 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.264+0000 7f0d9941e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0d940b0790 0x7f0d940c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:21.265 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.264+0000 7f0d9941e640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f0d940cb010 0x7f0d940c2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:21.265 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.264+0000 7f0d9941e640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d940cab80 con 0x7f0d9415ab50 2023-11-06T02:09:21.265 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.264+0000 7f0d9941e640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d9415af30 con 0x7f0d940cb010 2023-11-06T02:09:21.265 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.264+0000 7f0d9941e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d940cb5b0 con 0x7f0d940b0790 2023-11-06T02:09:21.266 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.264+0000 7f0d937fe640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f0d940cb010 0x7f0d940c2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:21.266 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.264+0000 7f0d927fc640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0d940b0790 0x7f0d940c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:21.266 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.264+0000 7f0d92ffd640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f0d9415ab50 0x7f0d940b0080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.49:45972/0 (socket says 172.21.15.49:45972) 2023-11-06T02:09:21.266 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.264+0000 7f0d937fe640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f0d940cb010 0x7f0d940c2710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.49:58834/0 (socket says 172.21.15.49:58834) 2023-11-06T02:09:21.266 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.264+0000 7f0d92ffd640 1 -- 172.21.15.49:0/3419283048 learned_addr learned my addr 172.21.15.49:0/3419283048 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:09:21.267 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.264+0000 7f0d937fe640 1 -- 172.21.15.49:0/3419283048 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0d940b0790 msgr2=0x7f0d940c9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:21.267 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.264+0000 7f0d937fe640 1 --2- 172.21.15.49:0/3419283048 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0d940b0790 0x7f0d940c9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:21.267 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.264+0000 7f0d937fe640 1 -- 172.21.15.49:0/3419283048 >> v1:172.21.15.150:6789/0 conn(0x7f0d9415ab50 legacy=0x7f0d940b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:21.267 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.264+0000 7f0d937fe640 1 -- 172.21.15.49:0/3419283048 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d940cb890 con 0x7f0d940cb010 2023-11-06T02:09:21.267 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.264+0000 7f0d927fc640 1 --2- 172.21.15.49:0/3419283048 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0d940b0790 0x7f0d940c9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T02:09:21.269 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.268+0000 7f0d937fe640 1 --2- 172.21.15.49:0/3419283048 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d940cb010 0x7f0d940c2710 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c0097c0 tx=0x7f0d8c01c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f3686aaacad3292 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:21.269 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.268+0000 7f0d91ffb640 1 -- 172.21.15.49:0/3419283048 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c0ab600 con 0x7f0d940cb010 2023-11-06T02:09:21.270 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.268+0000 7f0d91ffb640 1 -- 172.21.15.49:0/3419283048 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d8c0a7050 con 0x7f0d940cb010 2023-11-06T02:09:21.270 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.268+0000 7f0d91ffb640 1 -- 172.21.15.49:0/3419283048 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c0ab980 con 0x7f0d940cb010 2023-11-06T02:09:21.270 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.268+0000 7f0d9941e640 1 -- 172.21.15.49:0/3419283048 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d940cb010 msgr2=0x7f0d940c2710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:21.270 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.268+0000 7f0d9941e640 1 --2- 172.21.15.49:0/3419283048 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d940cb010 0x7f0d940c2710 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c0097c0 tx=0x7f0d8c01c3c0 comp rx=0 tx=0).stop 2023-11-06T02:09:21.270 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.268+0000 7f0d9941e640 1 -- 172.21.15.49:0/3419283048 shutdown_connections 2023-11-06T02:09:21.271 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.268+0000 7f0d9941e640 1 --2- 172.21.15.49:0/3419283048 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0d940b0790 0x7f0d940c9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:21.271 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.268+0000 7f0d9941e640 1 --2- 172.21.15.49:0/3419283048 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d940cb010 0x7f0d940c2710 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:21.271 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.268+0000 7f0d9941e640 1 -- 172.21.15.49:0/3419283048 >> 172.21.15.49:0/3419283048 conn(0x7f0d9406c0a0 msgr2=0x7f0d9406c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:21.271 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.268+0000 7f0d9941e640 1 -- 172.21.15.49:0/3419283048 shutdown_connections 2023-11-06T02:09:21.271 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.268+0000 7f0d9941e640 1 -- 172.21.15.49:0/3419283048 wait complete. 2023-11-06T02:09:21.272 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.272+0000 7f0d9941e640 1 Processor -- start 2023-11-06T02:09:21.272 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.272+0000 7f0d9941e640 1 -- start start 2023-11-06T02:09:21.274 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.272+0000 7f0d9941e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d940b0790 0x7f0d940c8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:21.275 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.272+0000 7f0d9941e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0d940cb010 0x7f0d940c67a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:21.275 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.272+0000 7f0d9941e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0d9415ab50 0x7f0d9415ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:21.275 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.272+0000 7f0d9941e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d940c7c00 con 0x7f0d9415ab50 2023-11-06T02:09:21.275 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.272+0000 7f0d9941e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d940c85d0 con 0x7f0d940b0790 2023-11-06T02:09:21.276 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.272+0000 7f0d9941e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d940c6ff0 con 0x7f0d940cb010 2023-11-06T02:09:21.276 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.272+0000 7f0d937fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0d9415ab50 0x7f0d9415ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:21.276 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.272+0000 7f0d937fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0d9415ab50 0x7f0d9415ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41800/0 (socket says 172.21.15.49:41800) 2023-11-06T02:09:21.276 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.272+0000 7f0d937fe640 1 -- 172.21.15.49:0/183715579 learned_addr learned my addr 172.21.15.49:0/183715579 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:21.276 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.272+0000 7f0d92ffd640 1 --2- 172.21.15.49:0/183715579 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d940b0790 0x7f0d940c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:21.277 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.272+0000 7f0d927fc640 1 --2- 172.21.15.49:0/183715579 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0d940cb010 0x7f0d940c67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:21.277 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.272+0000 7f0d937fe640 1 -- 172.21.15.49:0/183715579 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0d940cb010 msgr2=0x7f0d940c67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:21.277 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.272+0000 7f0d937fe640 1 --2- 172.21.15.49:0/183715579 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0d940cb010 0x7f0d940c67a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:21.277 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.272+0000 7f0d937fe640 1 -- 172.21.15.49:0/183715579 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d940b0790 msgr2=0x7f0d940c8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:21.277 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.272+0000 7f0d937fe640 1 --2- 172.21.15.49:0/183715579 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d940b0790 0x7f0d940c8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:21.277 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.272+0000 7f0d937fe640 1 -- 172.21.15.49:0/183715579 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d7c009420 con 0x7f0d9415ab50 2023-11-06T02:09:21.278 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.272+0000 7f0d937fe640 1 --2- 172.21.15.49:0/183715579 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0d9415ab50 0x7f0d9415ba30 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c0ae040 tx=0x7f0d8c019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:21.278 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.272+0000 7f0d8b7fe640 1 -- 172.21.15.49:0/183715579 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c0b4410 con 0x7f0d9415ab50 2023-11-06T02:09:21.278 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.272+0000 7f0d8b7fe640 1 -- 172.21.15.49:0/183715579 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d8c0b4570 con 0x7f0d9415ab50 2023-11-06T02:09:21.278 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.272+0000 7f0d9941e640 1 -- 172.21.15.49:0/183715579 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d8c009420 con 0x7f0d9415ab50 2023-11-06T02:09:21.278 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.272+0000 7f0d8b7fe640 1 -- 172.21.15.49:0/183715579 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c0b4900 con 0x7f0d9415ab50 2023-11-06T02:09:21.279 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.272+0000 7f0d9941e640 1 -- 172.21.15.49:0/183715579 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d940c75b0 con 0x7f0d9415ab50 2023-11-06T02:09:21.279 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.272+0000 7f0d9941e640 1 -- 172.21.15.49:0/183715579 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d940b0de0 con 0x7f0d9415ab50 2023-11-06T02:09:21.279 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.272+0000 7f0d8b7fe640 1 -- 172.21.15.49:0/183715579 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f0d8c0b2040 con 0x7f0d9415ab50 2023-11-06T02:09:21.279 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.272+0000 7f0d8b7fe640 1 --2- 172.21.15.49:0/183715579 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0d74065a60 0x7f0d74067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:21.279 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.272+0000 7f0d8b7fe640 1 -- 172.21.15.49:0/183715579 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f0d8c11d290 con 0x7f0d9415ab50 2023-11-06T02:09:21.280 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.272+0000 7f0d92ffd640 1 --2- 172.21.15.49:0/183715579 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0d74065a60 0x7f0d74067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:21.280 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.276+0000 7f0d92ffd640 1 --2- 172.21.15.49:0/183715579 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0d74065a60 0x7f0d74067f20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0d84005c60 tx=0x7f0d84009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:21.280 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.276+0000 7f0d8b7fe640 1 -- 172.21.15.49:0/183715579 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f0d8c0ead10 con 0x7f0d9415ab50 2023-11-06T02:09:21.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.436+0000 7f0d9941e640 1 -- 172.21.15.49:0/183715579 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f0d94074040 con 0x7f0d9415ab50 2023-11-06T02:09:21.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.440+0000 7f0d8b7fe640 1 -- 172.21.15.49:0/183715579 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7f0d8c0ee330 con 0x7f0d9415ab50 2023-11-06T02:09:21.438 INFO:teuthology.orchestra.run.smithi049.stderr:listed 0 entries 2023-11-06T02:09:21.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.440+0000 7f0d9941e640 1 -- 172.21.15.49:0/183715579 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0d74065a60 msgr2=0x7f0d74067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:21.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.440+0000 7f0d9941e640 1 --2- 172.21.15.49:0/183715579 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0d74065a60 0x7f0d74067f20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0d84005c60 tx=0x7f0d84009040 comp rx=0 tx=0).stop 2023-11-06T02:09:21.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.440+0000 7f0d9941e640 1 -- 172.21.15.49:0/183715579 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0d9415ab50 msgr2=0x7f0d9415ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:21.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.440+0000 7f0d9941e640 1 --2- 172.21.15.49:0/183715579 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0d9415ab50 0x7f0d9415ba30 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c0ae040 tx=0x7f0d8c019160 comp rx=0 tx=0).stop 2023-11-06T02:09:21.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.440+0000 7f0d9941e640 1 -- 172.21.15.49:0/183715579 shutdown_connections 2023-11-06T02:09:21.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.440+0000 7f0d9941e640 1 --2- 172.21.15.49:0/183715579 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0d74065a60 0x7f0d74067f20 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:21.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.440+0000 7f0d9941e640 1 --2- 172.21.15.49:0/183715579 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0d9415ab50 0x7f0d9415ba30 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:21.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.440+0000 7f0d9941e640 1 --2- 172.21.15.49:0/183715579 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0d940cb010 0x7f0d940c67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:21.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.440+0000 7f0d9941e640 1 --2- 172.21.15.49:0/183715579 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d940b0790 0x7f0d940c8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:21.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.440+0000 7f0d9941e640 1 -- 172.21.15.49:0/183715579 >> 172.21.15.49:0/183715579 conn(0x7f0d9406c0a0 msgr2=0x7f0d94071180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:21.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.440+0000 7f0d9941e640 1 -- 172.21.15.49:0/183715579 shutdown_connections 2023-11-06T02:09:21.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.440+0000 7f0d9941e640 1 -- 172.21.15.49:0/183715579 wait complete. 2023-11-06T02:09:21.453 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T02:09:21.455 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi049.front.sepia.ceph.com... 2023-11-06T02:09:21.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:09:21.456 DEBUG:teuthology.orchestra.run.smithi049:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2023-11-06T02:09:21.502 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T02:09:21.514 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T02:09:21.515 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi049.front.sepia.ceph.com 2023-11-06T02:09:21.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:09:21.516 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-06T02:09:21.598 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-11-06T02:09:21.598 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T02:09:21.605 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-11-06T02:09:21.605 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T02:09:21.654 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T02:09:21.780 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.780+0000 7fe2e438a640 1 Processor -- start 2023-11-06T02:09:21.781 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.780+0000 7fe2e438a640 1 -- start start 2023-11-06T02:09:21.783 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.784+0000 7fe2e438a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe2dc0b1f90 0x7fe2dc0cda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:21.784 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.784+0000 7fe2e438a640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fe2dc161650 0x7fe2dc0d1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:21.784 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.784+0000 7fe2e438a640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe2dc0d2190 0x7fe2dc0714a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:21.784 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.784+0000 7fe2e438a640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2dc0d17c0 con 0x7fe2dc0d2190 2023-11-06T02:09:21.784 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.784+0000 7fe2e438a640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2dc0ce6c0 con 0x7fe2dc161650 2023-11-06T02:09:21.785 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.784+0000 7fe2e438a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2dc161ad0 con 0x7fe2dc0b1f90 2023-11-06T02:09:21.785 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.784+0000 7fe2e2927640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe2dc0d2190 0x7fe2dc0714a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:21.785 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.784+0000 7fe2e2927640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe2dc0d2190 0x7fe2dc0714a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41816/0 (socket says 172.21.15.49:41816) 2023-11-06T02:09:21.786 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.784+0000 7fe2e2927640 1 -- 172.21.15.49:0/3641442902 learned_addr learned my addr 172.21.15.49:0/3641442902 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:21.786 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.784+0000 7fe2e2126640 1 --2- 172.21.15.49:0/3641442902 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe2dc0b1f90 0x7fe2dc0cda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:21.786 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.784+0000 7fe2e2927640 1 -- 172.21.15.49:0/3641442902 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe2dc0b1f90 msgr2=0x7fe2dc0cda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:21.786 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.784+0000 7fe2e2927640 1 --2- 172.21.15.49:0/3641442902 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe2dc0b1f90 0x7fe2dc0cda70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:21.787 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.784+0000 7fe2e2927640 1 -- 172.21.15.49:0/3641442902 >> v2:172.21.15.150:3300/0 conn(0x7fe2dc161650 msgr2=0x7fe2dc0d1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:21.787 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.784+0000 7fe2e2927640 1 --2- 172.21.15.49:0/3641442902 >> v2:172.21.15.150:3300/0 conn(0x7fe2dc161650 0x7fe2dc0d1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:21.787 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.784+0000 7fe2e2927640 1 -- 172.21.15.49:0/3641442902 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2dc0ce100 con 0x7fe2dc0d2190 2023-11-06T02:09:21.791 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.784+0000 7fe2e2927640 1 --2- 172.21.15.49:0/3641442902 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2dc0d2190 0x7fe2dc0714a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc0b5660 tx=0x7fe2d401c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b77ff1558381a8b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:21.792 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.784+0000 7fe2e1124640 1 -- 172.21.15.49:0/3641442902 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2d40a5730 con 0x7fe2dc0d2190 2023-11-06T02:09:21.792 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.784+0000 7fe2e1124640 1 -- 172.21.15.49:0/3641442902 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2d40a5890 con 0x7fe2dc0d2190 2023-11-06T02:09:21.792 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.784+0000 7fe2e1124640 1 -- 172.21.15.49:0/3641442902 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2d40a5c20 con 0x7fe2dc0d2190 2023-11-06T02:09:21.792 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.784+0000 7fe2e438a640 1 -- 172.21.15.49:0/3641442902 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2dc0d2190 msgr2=0x7fe2dc0714a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:21.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.784+0000 7fe2e438a640 1 --2- 172.21.15.49:0/3641442902 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2dc0d2190 0x7fe2dc0714a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc0b5660 tx=0x7fe2d401c9a0 comp rx=0 tx=0).stop 2023-11-06T02:09:21.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.784+0000 7fe2e438a640 1 -- 172.21.15.49:0/3641442902 shutdown_connections 2023-11-06T02:09:21.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.784+0000 7fe2e438a640 1 --2- 172.21.15.49:0/3641442902 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2dc0d2190 0x7fe2dc0714a0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:21.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.784+0000 7fe2e438a640 1 --2- 172.21.15.49:0/3641442902 >> v2:172.21.15.150:3300/0 conn(0x7fe2dc161650 0x7fe2dc0d1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:21.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.784+0000 7fe2e438a640 1 --2- 172.21.15.49:0/3641442902 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe2dc0b1f90 0x7fe2dc0cda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:21.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.784+0000 7fe2e438a640 1 -- 172.21.15.49:0/3641442902 >> 172.21.15.49:0/3641442902 conn(0x7fe2dc0d2aa0 msgr2=0x7fe2dc0d2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:21.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.784+0000 7fe2e438a640 1 -- 172.21.15.49:0/3641442902 shutdown_connections 2023-11-06T02:09:21.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.784+0000 7fe2e438a640 1 -- 172.21.15.49:0/3641442902 wait complete. 2023-11-06T02:09:21.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.788+0000 7fe2e438a640 1 Processor -- start 2023-11-06T02:09:21.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.788+0000 7fe2e438a640 1 -- start start 2023-11-06T02:09:21.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.788+0000 7fe2e438a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2dc0b1f90 0x7fe2dc15d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:21.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.788+0000 7fe2e438a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe2dc0d2190 0x7fe2dc1591c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:21.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.788+0000 7fe2e438a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe2dc161650 0x7fe2dc15ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:21.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.788+0000 7fe2e438a640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2dc15e700 con 0x7fe2dc0b1f90 2023-11-06T02:09:21.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.788+0000 7fe2e438a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2dc15daf0 con 0x7fe2dc0d2190 2023-11-06T02:09:21.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.788+0000 7fe2e438a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2dc15ddd0 con 0x7fe2dc161650 2023-11-06T02:09:21.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.788+0000 7fe2e2126640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2dc0b1f90 0x7fe2dc15d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:21.798 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.788+0000 7fe2e2126640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2dc0b1f90 0x7fe2dc15d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41820/0 (socket says 172.21.15.49:41820) 2023-11-06T02:09:21.798 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.788+0000 7fe2e2126640 1 -- 172.21.15.49:0/184437437 learned_addr learned my addr 172.21.15.49:0/184437437 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:21.798 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.788+0000 7fe2e2927640 1 --2- 172.21.15.49:0/184437437 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe2dc161650 0x7fe2dc15ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:21.799 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.788+0000 7fe2e1925640 1 --2- 172.21.15.49:0/184437437 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe2dc0d2190 0x7fe2dc1591c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:21.799 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.788+0000 7fe2e2126640 1 -- 172.21.15.49:0/184437437 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe2dc161650 msgr2=0x7fe2dc15ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:21.799 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.788+0000 7fe2e2126640 1 --2- 172.21.15.49:0/184437437 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe2dc161650 0x7fe2dc15ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:21.799 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.788+0000 7fe2e2126640 1 -- 172.21.15.49:0/184437437 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe2dc0d2190 msgr2=0x7fe2dc1591c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:21.800 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.788+0000 7fe2e2126640 1 --2- 172.21.15.49:0/184437437 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe2dc0d2190 0x7fe2dc1591c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:21.800 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.788+0000 7fe2e2126640 1 -- 172.21.15.49:0/184437437 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2d4009b30 con 0x7fe2dc0b1f90 2023-11-06T02:09:21.800 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.788+0000 7fe2e2927640 1 --2- 172.21.15.49:0/184437437 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe2dc161650 0x7fe2dc15ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:09:21.800 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.788+0000 7fe2e1925640 1 --2- 172.21.15.49:0/184437437 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe2dc0d2190 0x7fe2dc1591c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:09:21.801 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.788+0000 7fe2e2126640 1 --2- 172.21.15.49:0/184437437 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2dc0b1f90 0x7fe2dc15d2a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fe2d800c7f0 tx=0x7fe2d800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:21.801 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.788+0000 7fe2d2ffd640 1 -- 172.21.15.49:0/184437437 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2d8004220 con 0x7fe2dc0b1f90 2023-11-06T02:09:21.801 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.788+0000 7fe2d2ffd640 1 -- 172.21.15.49:0/184437437 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2d80043a0 con 0x7fe2dc0b1f90 2023-11-06T02:09:21.801 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.788+0000 7fe2e438a640 1 -- 172.21.15.49:0/184437437 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2dc15e0b0 con 0x7fe2dc0b1f90 2023-11-06T02:09:21.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.788+0000 7fe2d2ffd640 1 -- 172.21.15.49:0/184437437 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2d8004510 con 0x7fe2dc0b1f90 2023-11-06T02:09:21.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.788+0000 7fe2e438a640 1 -- 172.21.15.49:0/184437437 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2dc15d7e0 con 0x7fe2dc0b1f90 2023-11-06T02:09:21.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.788+0000 7fe2e438a640 1 -- 172.21.15.49:0/184437437 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2dc074040 con 0x7fe2dc0b1f90 2023-11-06T02:09:21.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.792+0000 7fe2d2ffd640 1 -- 172.21.15.49:0/184437437 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fe2d8003a70 con 0x7fe2dc0b1f90 2023-11-06T02:09:21.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.792+0000 7fe2d2ffd640 1 --2- 172.21.15.49:0/184437437 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe2c0065a60 0x7fe2c0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:21.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.792+0000 7fe2d2ffd640 1 -- 172.21.15.49:0/184437437 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fe2d8083270 con 0x7fe2dc0b1f90 2023-11-06T02:09:21.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.792+0000 7fe2e1925640 1 --2- 172.21.15.49:0/184437437 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe2c0065a60 0x7fe2c0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:21.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.792+0000 7fe2e1925640 1 --2- 172.21.15.49:0/184437437 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe2c0065a60 0x7fe2c0067f20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc1607d0 tx=0x7fe2cc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:21.804 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.796+0000 7fe2d2ffd640 1 -- 172.21.15.49:0/184437437 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fe2d8050b20 con 0x7fe2dc0b1f90 2023-11-06T02:09:21.950 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.952+0000 7fe2e438a640 1 -- 172.21.15.49:0/184437437 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe2dc0d0c30 con 0x7fe2dc0b1f90 2023-11-06T02:09:21.953 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.956+0000 7fe2d2ffd640 1 -- 172.21.15.49:0/184437437 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17906 (secure 0 0 0) 0x7fe2d8054140 con 0x7fe2dc0b1f90 2023-11-06T02:09:21.954 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:09:21.954 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":25,"fsid":"1a8b035c-6ba4-499a-b876-aea47fd9c0d4","created":"2023-11-06T02:08:06.440857+0000","modified":"2023-11-06T02:09:13.195118+0000","last_up_change":"2023-11-06T02:08:17.982372+0000","last_in_change":"2023-11-06T02:08:08.906631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T02:08:19.252222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T02:08:24.776211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-06T02:08:28.936178+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-06T02:08:29.563468+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ad55af8-0571-4e6a-8722-b5d429b7496d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6809","nonce":38116},{"type":"v1","addr":"172.21.15.49:6811","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6813","nonce":38116},{"type":"v1","addr":"172.21.15.49:6815","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6821","nonce":38116},{"type":"v1","addr":"172.21.15.49:6823","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6817","nonce":38116},{"type":"v1","addr":"172.21.15.49:6819","nonce":38116}]},"public_addr":"172.21.15.49:6811/38116","cluster_addr":"172.21.15.49:6815/38116","heartbeat_back_addr":"172.21.15.49:6823/38116","heartbeat_front_addr":"172.21.15.49:6819/38116","state":["exists","up"]},{"osd":1,"uuid":"c8b31f85-a757-43ed-9419-351acc46611b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":38113},{"type":"v1","addr":"172.21.15.49:6810","nonce":38113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":38113},{"type":"v1","addr":"172.21.15.49:6814","nonce":38113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":38113},{"type":"v1","addr":"172.21.15.49:6822","nonce":38113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":38113},{"type":"v1","addr":"172.21.15.49:6818","nonce":38113}]},"public_addr":"172.21.15.49:6810/38113","cluster_addr":"172.21.15.49:6814/38113","heartbeat_back_addr":"172.21.15.49:6822/38113","heartbeat_front_addr":"172.21.15.49:6818/38113","state":["exists","up"]},{"osd":2,"uuid":"555b3bf3-0a70-4b91-8efd-e4204b5fc26e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":38114},{"type":"v1","addr":"172.21.15.49:6801","nonce":38114}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":38114},{"type":"v1","addr":"172.21.15.49:6803","nonce":38114}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":38114},{"type":"v1","addr":"172.21.15.49:6807","nonce":38114}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":38114},{"type":"v1","addr":"172.21.15.49:6805","nonce":38114}]},"public_addr":"172.21.15.49:6801/38114","cluster_addr":"172.21.15.49:6803/38114","heartbeat_back_addr":"172.21.15.49:6807/38114","heartbeat_front_addr":"172.21.15.49:6805/38114","state":["exists","up"]},{"osd":3,"uuid":"8d4e5d77-03ef-4b9b-bfcb-5ae93a46795c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":38115},{"type":"v1","addr":"172.21.15.49:6825","nonce":38115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":38115},{"type":"v1","addr":"172.21.15.49:6827","nonce":38115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":38115},{"type":"v1","addr":"172.21.15.49:6831","nonce":38115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":38115},{"type":"v1","addr":"172.21.15.49:6829","nonce":38115}]},"public_addr":"172.21.15.49:6825/38115","cluster_addr":"172.21.15.49:6827/38115","heartbeat_back_addr":"172.21.15.49:6831/38115","heartbeat_front_addr":"172.21.15.49:6829/38115","state":["exists","up"]},{"osd":4,"uuid":"dba03ad4-68a3-4cf3-9218-ac611f02eacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":38417},{"type":"v1","addr":"172.21.15.150:6809","nonce":38417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":38417},{"type":"v1","addr":"172.21.15.150:6811","nonce":38417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":38417},{"type":"v1","addr":"172.21.15.150:6815","nonce":38417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":38417},{"type":"v1","addr":"172.21.15.150:6813","nonce":38417}]},"public_addr":"172.21.15.150:6809/38417","cluster_addr":"172.21.15.150:6811/38417","heartbeat_back_addr":"172.21.15.150:6815/38417","heartbeat_front_addr":"172.21.15.150:6813/38417","state":["exists","up"]},{"osd":5,"uuid":"0df9c303-c255-4640-8e91-01f1c3efe18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":38418},{"type":"v1","addr":"172.21.15.150:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":38418},{"type":"v1","addr":"172.21.15.150:6803","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":38418},{"type":"v1","addr":"172.21.15.150:6807","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":38418},{"type":"v1","addr":"172.21.15.150:6805","nonce":38418}]},"public_addr":"172.21.15.150:6801/38418","cluster_addr":"172.21.15.150:6803/38418","heartbeat_back_addr":"172.21.15.150:6807/38418","heartbeat_front_addr":"172.21.15.150:6805/38418","state":["exists","up"]},{"osd":6,"uuid":"e59fa189-4b70-4c06-9722-c02b90828bf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":38428},{"type":"v1","addr":"172.21.15.150:6825","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":38428},{"type":"v1","addr":"172.21.15.150:6827","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":38428},{"type":"v1","addr":"172.21.15.150:6831","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":38428},{"type":"v1","addr":"172.21.15.150:6829","nonce":38428}]},"public_addr":"172.21.15.150:6825/38428","cluster_addr":"172.21.15.150:6827/38428","heartbeat_back_addr":"172.21.15.150:6831/38428","heartbeat_front_addr":"172.21.15.150:6829/38428","state":["exists","up"]},{"osd":7,"uuid":"a64f57e1-ef98-46bd-b304-385759e95898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":38421},{"type":"v1","addr":"172.21.15.150:6817","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":38421},{"type":"v1","addr":"172.21.15.150:6819","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":38421},{"type":"v1","addr":"172.21.15.150:6823","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":38421},{"type":"v1","addr":"172.21.15.150:6821","nonce":38421}]},"public_addr":"172.21.15.150:6817/38421","cluster_addr":"172.21.15.150:6819/38421","heartbeat_back_addr":"172.21.15.150:6823/38421","heartbeat_front_addr":"172.21.15.150:6821/38421","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.438552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.413280+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.296466+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.424966+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.276953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.230984+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.410235+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.480084+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.14","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T02:09:21.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.956+0000 7fe2e438a640 1 -- 172.21.15.49:0/184437437 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe2c0065a60 msgr2=0x7fe2c0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:21.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.956+0000 7fe2e438a640 1 --2- 172.21.15.49:0/184437437 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe2c0065a60 0x7fe2c0067f20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc1607d0 tx=0x7fe2cc008040 comp rx=0 tx=0).stop 2023-11-06T02:09:21.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.956+0000 7fe2e438a640 1 -- 172.21.15.49:0/184437437 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2dc0b1f90 msgr2=0x7fe2dc15d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:21.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.956+0000 7fe2e438a640 1 --2- 172.21.15.49:0/184437437 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2dc0b1f90 0x7fe2dc15d2a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fe2d800c7f0 tx=0x7fe2d800ccb0 comp rx=0 tx=0).stop 2023-11-06T02:09:21.957 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.956+0000 7fe2e438a640 1 -- 172.21.15.49:0/184437437 shutdown_connections 2023-11-06T02:09:21.957 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.956+0000 7fe2e438a640 1 --2- 172.21.15.49:0/184437437 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe2c0065a60 0x7fe2c0067f20 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:21.957 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.956+0000 7fe2e438a640 1 --2- 172.21.15.49:0/184437437 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe2dc161650 0x7fe2dc15ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:21.957 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.956+0000 7fe2e438a640 1 --2- 172.21.15.49:0/184437437 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe2dc0d2190 0x7fe2dc1591c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:21.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.956+0000 7fe2e438a640 1 --2- 172.21.15.49:0/184437437 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2dc0b1f90 0x7fe2dc15d2a0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:21.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.956+0000 7fe2e438a640 1 -- 172.21.15.49:0/184437437 >> 172.21.15.49:0/184437437 conn(0x7fe2dc0d2aa0 msgr2=0x7fe2dc0b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:21.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.956+0000 7fe2e438a640 1 -- 172.21.15.49:0/184437437 shutdown_connections 2023-11-06T02:09:21.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:21.956+0000 7fe2e438a640 1 -- 172.21.15.49:0/184437437 wait complete. 2023-11-06T02:09:21.973 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T02:08:19.252222+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T02:08:24.776211+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T02:08:28.936178+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T02:08:29.563468+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T02:09:21.975 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T02:09:22.098 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.100+0000 7fc7f6323640 1 Processor -- start 2023-11-06T02:09:22.098 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.100+0000 7fc7f6323640 1 -- start start 2023-11-06T02:09:22.099 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.100+0000 7fc7f6323640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc7f015a880 0x7fc7f00b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:22.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.100+0000 7fc7f6323640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc7f0070160 0x7fc7f0070610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:22.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.100+0000 7fc7f6323640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc7f006d150 0x7fc7f006d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:22.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.100+0000 7fc7f6323640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7f0070b50 con 0x7fc7f006d150 2023-11-06T02:09:22.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.100+0000 7fc7f6323640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7f00ae360 con 0x7fc7f015a880 2023-11-06T02:09:22.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.100+0000 7fc7f6323640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f00ae640 con 0x7fc7f0070160 2023-11-06T02:09:22.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.100+0000 7fc7f48c0640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc7f006d150 0x7fc7f006d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:22.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.100+0000 7fc7f48c0640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc7f006d150 0x7fc7f006d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41830/0 (socket says 172.21.15.49:41830) 2023-11-06T02:09:22.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.100+0000 7fc7f48c0640 1 -- 172.21.15.49:0/570611669 learned_addr learned my addr 172.21.15.49:0/570611669 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:22.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.100+0000 7fc7ef7fe640 1 --2- 172.21.15.49:0/570611669 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc7f0070160 0x7fc7f0070610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:22.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.100+0000 7fc7effff640 1 --2- 172.21.15.49:0/570611669 >> v2:172.21.15.150:3300/0 conn(0x7fc7f015a880 0x7fc7f00b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:22.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.100+0000 7fc7f48c0640 1 -- 172.21.15.49:0/570611669 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc7f0070160 msgr2=0x7fc7f0070610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:22.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.100+0000 7fc7f48c0640 1 --2- 172.21.15.49:0/570611669 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc7f0070160 0x7fc7f0070610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:22.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.100+0000 7fc7f48c0640 1 -- 172.21.15.49:0/570611669 >> v2:172.21.15.150:3300/0 conn(0x7fc7f015a880 msgr2=0x7fc7f00b15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:22.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.100+0000 7fc7f48c0640 1 --2- 172.21.15.49:0/570611669 >> v2:172.21.15.150:3300/0 conn(0x7fc7f015a880 0x7fc7f00b15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:22.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.100+0000 7fc7f48c0640 1 -- 172.21.15.49:0/570611669 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7f00ae920 con 0x7fc7f006d150 2023-11-06T02:09:22.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.100+0000 7fc7effff640 1 --2- 172.21.15.49:0/570611669 >> v2:172.21.15.150:3300/0 conn(0x7fc7f015a880 0x7fc7f00b15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:09:22.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.104+0000 7fc7f48c0640 1 --2- 172.21.15.49:0/570611669 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7f006d150 0x7fc7f006d5b0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fc7f00cafc0 tx=0x7fc7e801c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca0cf0dc07019356 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:22.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.104+0000 7fc7eeffd640 1 -- 172.21.15.49:0/570611669 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e80a5760 con 0x7fc7f006d150 2023-11-06T02:09:22.107 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.104+0000 7fc7eeffd640 1 -- 172.21.15.49:0/570611669 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7e80a58c0 con 0x7fc7f006d150 2023-11-06T02:09:22.107 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.104+0000 7fc7eeffd640 1 -- 172.21.15.49:0/570611669 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e80a5c50 con 0x7fc7f006d150 2023-11-06T02:09:22.107 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.104+0000 7fc7f6323640 1 -- 172.21.15.49:0/570611669 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7f006d150 msgr2=0x7fc7f006d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:22.107 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.104+0000 7fc7f6323640 1 --2- 172.21.15.49:0/570611669 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7f006d150 0x7fc7f006d5b0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fc7f00cafc0 tx=0x7fc7e801c9d0 comp rx=0 tx=0).stop 2023-11-06T02:09:22.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.104+0000 7fc7f6323640 1 -- 172.21.15.49:0/570611669 shutdown_connections 2023-11-06T02:09:22.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.104+0000 7fc7f6323640 1 --2- 172.21.15.49:0/570611669 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc7f0070160 0x7fc7f0070610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:22.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.104+0000 7fc7f6323640 1 --2- 172.21.15.49:0/570611669 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7f006d150 0x7fc7f006d5b0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:22.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.104+0000 7fc7f6323640 1 --2- 172.21.15.49:0/570611669 >> v2:172.21.15.150:3300/0 conn(0x7fc7f015a880 0x7fc7f00b15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:22.109 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.104+0000 7fc7f6323640 1 -- 172.21.15.49:0/570611669 >> 172.21.15.49:0/570611669 conn(0x7fc7f006a370 msgr2=0x7fc7f00ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:22.109 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.104+0000 7fc7f6323640 1 -- 172.21.15.49:0/570611669 shutdown_connections 2023-11-06T02:09:22.109 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.104+0000 7fc7f6323640 1 -- 172.21.15.49:0/570611669 wait complete. 2023-11-06T02:09:22.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.104+0000 7fc7f6323640 1 Processor -- start 2023-11-06T02:09:22.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.104+0000 7fc7f6323640 1 -- start start 2023-11-06T02:09:22.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.104+0000 7fc7f6323640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc7f006d150 0x7fc7f014f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:22.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.104+0000 7fc7f6323640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7f0070160 0x7fc7f0156f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:22.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.104+0000 7fc7f6323640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc7f015a880 0x7fc7f00cc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:22.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.104+0000 7fc7f6323640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f014fa90 con 0x7fc7f0070160 2023-11-06T02:09:22.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.104+0000 7fc7f6323640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f0155450 con 0x7fc7f015a880 2023-11-06T02:09:22.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.104+0000 7fc7f6323640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f0154630 con 0x7fc7f006d150 2023-11-06T02:09:22.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.104+0000 7fc7ef7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7f0070160 0x7fc7f0156f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:22.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.104+0000 7fc7ef7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7f0070160 0x7fc7f0156f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41842/0 (socket says 172.21.15.49:41842) 2023-11-06T02:09:22.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.104+0000 7fc7ef7fe640 1 -- 172.21.15.49:0/2678739611 learned_addr learned my addr 172.21.15.49:0/2678739611 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:22.113 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.104+0000 7fc7effff640 1 --2- 172.21.15.49:0/2678739611 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc7f006d150 0x7fc7f014f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:22.113 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.104+0000 7fc7f48c0640 1 --2- 172.21.15.49:0/2678739611 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc7f015a880 0x7fc7f00cc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:22.114 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.108+0000 7fc7ef7fe640 1 -- 172.21.15.49:0/2678739611 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc7f006d150 msgr2=0x7fc7f014f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:22.115 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.108+0000 7fc7ef7fe640 1 --2- 172.21.15.49:0/2678739611 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc7f006d150 0x7fc7f014f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:22.115 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.108+0000 7fc7ef7fe640 1 -- 172.21.15.49:0/2678739611 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc7f015a880 msgr2=0x7fc7f00cc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:22.116 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.108+0000 7fc7ef7fe640 1 --2- 172.21.15.49:0/2678739611 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc7f015a880 0x7fc7f00cc5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:22.116 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.108+0000 7fc7ef7fe640 1 -- 172.21.15.49:0/2678739611 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7e8009b30 con 0x7fc7f0070160 2023-11-06T02:09:22.116 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.108+0000 7fc7ef7fe640 1 --2- 172.21.15.49:0/2678739611 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7f0070160 0x7fc7f0156f40 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fc7e400e8b0 tx=0x7fc7e400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:22.117 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.108+0000 7fc7ecff9640 1 -- 172.21.15.49:0/2678739611 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e400c940 con 0x7fc7f0070160 2023-11-06T02:09:22.117 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.108+0000 7fc7ecff9640 1 -- 172.21.15.49:0/2678739611 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7e400caa0 con 0x7fc7f0070160 2023-11-06T02:09:22.117 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.108+0000 7fc7ecff9640 1 -- 172.21.15.49:0/2678739611 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e400ce30 con 0x7fc7f0070160 2023-11-06T02:09:22.118 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.108+0000 7fc7f6323640 1 -- 172.21.15.49:0/2678739611 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7f015ade0 con 0x7fc7f0070160 2023-11-06T02:09:22.118 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.108+0000 7fc7f6323640 1 -- 172.21.15.49:0/2678739611 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7f0154bf0 con 0x7fc7f0070160 2023-11-06T02:09:22.120 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.108+0000 7fc7f6323640 1 -- 172.21.15.49:0/2678739611 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7f0155600 con 0x7fc7f0070160 2023-11-06T02:09:22.120 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.108+0000 7fc7ecff9640 1 -- 172.21.15.49:0/2678739611 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fc7e4013470 con 0x7fc7f0070160 2023-11-06T02:09:22.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.108+0000 7fc7ecff9640 1 --2- 172.21.15.49:0/2678739611 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc7dc065a60 0x7fc7dc067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:22.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.108+0000 7fc7ecff9640 1 -- 172.21.15.49:0/2678739611 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fc7e40834f0 con 0x7fc7f0070160 2023-11-06T02:09:22.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.108+0000 7fc7effff640 1 --2- 172.21.15.49:0/2678739611 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc7dc065a60 0x7fc7dc067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:22.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.112+0000 7fc7effff640 1 --2- 172.21.15.49:0/2678739611 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc7dc065a60 0x7fc7dc067f20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc7e000ba10 tx=0x7fc7e0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:22.122 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.112+0000 7fc7ecff9640 1 -- 172.21.15.49:0/2678739611 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fc7f0155600 con 0x7fc7f0070160 2023-11-06T02:09:22.265 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.264+0000 7fc7f6323640 1 -- 172.21.15.49:0/2678739611 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc7f01555c0 con 0x7fc7f0070160 2023-11-06T02:09:22.266 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.268+0000 7fc7ecff9640 1 -- 172.21.15.49:0/2678739611 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc7e4050d10 con 0x7fc7f0070160 2023-11-06T02:09:22.266 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2023-11-06T02:09:22.268 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.268+0000 7fc7f6323640 1 -- 172.21.15.49:0/2678739611 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc7dc065a60 msgr2=0x7fc7dc067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:22.269 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.268+0000 7fc7f6323640 1 --2- 172.21.15.49:0/2678739611 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc7dc065a60 0x7fc7dc067f20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc7e000ba10 tx=0x7fc7e0008040 comp rx=0 tx=0).stop 2023-11-06T02:09:22.269 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.268+0000 7fc7f6323640 1 -- 172.21.15.49:0/2678739611 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7f0070160 msgr2=0x7fc7f0156f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:22.269 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.268+0000 7fc7f6323640 1 --2- 172.21.15.49:0/2678739611 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7f0070160 0x7fc7f0156f40 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fc7e400e8b0 tx=0x7fc7e400ed70 comp rx=0 tx=0).stop 2023-11-06T02:09:22.269 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.268+0000 7fc7f6323640 1 -- 172.21.15.49:0/2678739611 shutdown_connections 2023-11-06T02:09:22.269 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.268+0000 7fc7f6323640 1 --2- 172.21.15.49:0/2678739611 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc7dc065a60 0x7fc7dc067f20 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:22.270 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.268+0000 7fc7f6323640 1 --2- 172.21.15.49:0/2678739611 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7f0070160 0x7fc7f0156f40 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:22.270 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.268+0000 7fc7f6323640 1 --2- 172.21.15.49:0/2678739611 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc7f015a880 0x7fc7f00cc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:22.270 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.268+0000 7fc7f6323640 1 --2- 172.21.15.49:0/2678739611 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc7f006d150 0x7fc7f014f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:22.271 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.268+0000 7fc7f6323640 1 -- 172.21.15.49:0/2678739611 >> 172.21.15.49:0/2678739611 conn(0x7fc7f006a370 msgr2=0x7fc7f015af70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:22.271 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.268+0000 7fc7f6323640 1 -- 172.21.15.49:0/2678739611 shutdown_connections 2023-11-06T02:09:22.272 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.268+0000 7fc7f6323640 1 -- 172.21.15.49:0/2678739611 wait complete. 2023-11-06T02:09:22.285 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T02:09:22.411 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.412+0000 7fdfbc2a3640 1 Processor -- start 2023-11-06T02:09:22.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.412+0000 7fdfbc2a3640 1 -- start start 2023-11-06T02:09:22.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.412+0000 7fdfbc2a3640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fdfb415ab60 0x7fdfb40ad520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:22.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.412+0000 7fdfbc2a3640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fdfb40ada60 0x7fdfb40d08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:22.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.412+0000 7fdfbc2a3640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfb40d1240 con 0x7fdfb40d0df0 2023-11-06T02:09:22.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.412+0000 7fdfbc2a3640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfb40cb340 con 0x7fdfb415ab60 2023-11-06T02:09:22.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.412+0000 7fdfbc2a3640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfb40adf50 con 0x7fdfb40ada60 2023-11-06T02:09:22.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.412+0000 7fdfba03f640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fdfb415ab60 0x7fdfb40ad520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:22.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.412+0000 7fdfba03f640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fdfb415ab60 0x7fdfb40ad520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41844/0 (socket says 172.21.15.49:41844) 2023-11-06T02:09:22.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.412+0000 7fdfba03f640 1 -- 172.21.15.49:0/2777485934 learned_addr learned my addr 172.21.15.49:0/2777485934 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:22.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.412+0000 7fdfb983e640 1 --2- 172.21.15.49:0/2777485934 >> v2:172.21.15.150:3300/0 conn(0x7fdfb40ada60 0x7fdfb40d08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:22.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.412+0000 7fdfba03f640 1 -- 172.21.15.49:0/2777485934 >> v2:172.21.15.150:3300/0 conn(0x7fdfb40ada60 msgr2=0x7fdfb40d08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:22.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.412+0000 7fdfba03f640 1 --2- 172.21.15.49:0/2777485934 >> v2:172.21.15.150:3300/0 conn(0x7fdfb40ada60 0x7fdfb40d08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:22.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.412+0000 7fdfba03f640 1 -- 172.21.15.49:0/2777485934 >> v1:172.21.15.150:6789/0 conn(0x7fdfb40d0df0 legacy=0x7fdfb40b08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:22.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.412+0000 7fdfba03f640 1 -- 172.21.15.49:0/2777485934 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfb40ae230 con 0x7fdfb415ab60 2023-11-06T02:09:22.417 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.416+0000 7fdfba03f640 1 --2- 172.21.15.49:0/2777485934 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdfb415ab60 0x7fdfb40ad520 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fdfb40719c0 tx=0x7fdfb001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c13975e841f6adb server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:22.417 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.416+0000 7fdfb903d640 1 -- 172.21.15.49:0/2777485934 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb00aa430 con 0x7fdfb415ab60 2023-11-06T02:09:22.417 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.416+0000 7fdfb903d640 1 -- 172.21.15.49:0/2777485934 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfb00aa590 con 0x7fdfb415ab60 2023-11-06T02:09:22.418 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.416+0000 7fdfb903d640 1 -- 172.21.15.49:0/2777485934 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb00aa920 con 0x7fdfb415ab60 2023-11-06T02:09:22.418 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.416+0000 7fdfbc2a3640 1 -- 172.21.15.49:0/2777485934 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdfb415ab60 msgr2=0x7fdfb40ad520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:22.418 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.416+0000 7fdfbc2a3640 1 --2- 172.21.15.49:0/2777485934 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdfb415ab60 0x7fdfb40ad520 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fdfb40719c0 tx=0x7fdfb001c4d0 comp rx=0 tx=0).stop 2023-11-06T02:09:22.418 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.416+0000 7fdfbc2a3640 1 -- 172.21.15.49:0/2777485934 shutdown_connections 2023-11-06T02:09:22.419 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.416+0000 7fdfbc2a3640 1 --2- 172.21.15.49:0/2777485934 >> v2:172.21.15.150:3300/0 conn(0x7fdfb40ada60 0x7fdfb40d08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:22.419 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.416+0000 7fdfbc2a3640 1 --2- 172.21.15.49:0/2777485934 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdfb415ab60 0x7fdfb40ad520 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:22.419 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.416+0000 7fdfbc2a3640 1 -- 172.21.15.49:0/2777485934 >> 172.21.15.49:0/2777485934 conn(0x7fdfb4089fd0 msgr2=0x7fdfb4099c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:22.419 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.416+0000 7fdfbc2a3640 1 -- 172.21.15.49:0/2777485934 shutdown_connections 2023-11-06T02:09:22.419 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.416+0000 7fdfbc2a3640 1 -- 172.21.15.49:0/2777485934 wait complete. 2023-11-06T02:09:22.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.416+0000 7fdfbc2a3640 1 Processor -- start 2023-11-06T02:09:22.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.416+0000 7fdfbc2a3640 1 -- start start 2023-11-06T02:09:22.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.420+0000 7fdfbc2a3640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdfb40ada60 0x7fdfb414a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:22.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.420+0000 7fdfbc2a3640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdfb40d0df0 0x7fdfb406f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:22.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.420+0000 7fdfbc2a3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdfb406fb60 0x7fdfb406af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:22.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.420+0000 7fdfbc2a3640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb406c950 con 0x7fdfb406fb60 2023-11-06T02:09:22.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.420+0000 7fdfbc2a3640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb406fff0 con 0x7fdfb40ada60 2023-11-06T02:09:22.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.420+0000 7fdfbc2a3640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb406bcd0 con 0x7fdfb40d0df0 2023-11-06T02:09:22.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.420+0000 7fdfba840640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdfb406fb60 0x7fdfb406af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:22.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.420+0000 7fdfba840640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdfb406fb60 0x7fdfb406af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41854/0 (socket says 172.21.15.49:41854) 2023-11-06T02:09:22.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.420+0000 7fdfba840640 1 -- 172.21.15.49:0/1542243220 learned_addr learned my addr 172.21.15.49:0/1542243220 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:22.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.420+0000 7fdfb983e640 1 --2- 172.21.15.49:0/1542243220 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdfb40d0df0 0x7fdfb406f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:22.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.420+0000 7fdfba03f640 1 --2- 172.21.15.49:0/1542243220 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdfb40ada60 0x7fdfb414a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:22.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.420+0000 7fdfba840640 1 -- 172.21.15.49:0/1542243220 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdfb40d0df0 msgr2=0x7fdfb406f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:22.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.420+0000 7fdfba840640 1 --2- 172.21.15.49:0/1542243220 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdfb40d0df0 0x7fdfb406f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:22.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.420+0000 7fdfba840640 1 -- 172.21.15.49:0/1542243220 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdfb40ada60 msgr2=0x7fdfb414a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:22.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.420+0000 7fdfba840640 1 --2- 172.21.15.49:0/1542243220 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdfb40ada60 0x7fdfb414a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:22.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.420+0000 7fdfba840640 1 -- 172.21.15.49:0/1542243220 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfb00094d0 con 0x7fdfb406fb60 2023-11-06T02:09:22.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.420+0000 7fdfba840640 1 --2- 172.21.15.49:0/1542243220 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdfb406fb60 0x7fdfb406af10 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fdfac00c770 tx=0x7fdfac00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:22.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.420+0000 7fdfa6ffd640 1 -- 172.21.15.49:0/1542243220 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfac003b80 con 0x7fdfb406fb60 2023-11-06T02:09:22.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.420+0000 7fdfa6ffd640 1 -- 172.21.15.49:0/1542243220 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfac003ce0 con 0x7fdfb406fb60 2023-11-06T02:09:22.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.420+0000 7fdfa6ffd640 1 -- 172.21.15.49:0/1542243220 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfac00aa90 con 0x7fdfb406fb60 2023-11-06T02:09:22.427 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.420+0000 7fdfbc2a3640 1 -- 172.21.15.49:0/1542243220 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfb406bfb0 con 0x7fdfb406fb60 2023-11-06T02:09:22.427 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.420+0000 7fdfbc2a3640 1 -- 172.21.15.49:0/1542243220 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfb406c570 con 0x7fdfb406fb60 2023-11-06T02:09:22.427 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.420+0000 7fdfbc2a3640 1 -- 172.21.15.49:0/1542243220 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfb406ca90 con 0x7fdfb406fb60 2023-11-06T02:09:22.427 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.420+0000 7fdfa6ffd640 1 -- 172.21.15.49:0/1542243220 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fdfac00abf0 con 0x7fdfb406fb60 2023-11-06T02:09:22.427 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.420+0000 7fdfa6ffd640 1 --2- 172.21.15.49:0/1542243220 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fdf98065a60 0x7fdf98067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:22.428 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.420+0000 7fdfa6ffd640 1 -- 172.21.15.49:0/1542243220 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fdfac082ff0 con 0x7fdfb406fb60 2023-11-06T02:09:22.428 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.420+0000 7fdfba03f640 1 --2- 172.21.15.49:0/1542243220 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fdf98065a60 0x7fdf98067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:22.428 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.424+0000 7fdfba03f640 1 --2- 172.21.15.49:0/1542243220 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fdf98065a60 0x7fdf98067f20 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fdfb00ae000 tx=0x7fdfb00a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:22.428 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.424+0000 7fdfa6ffd640 1 -- 172.21.15.49:0/1542243220 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fdfb406ca90 con 0x7fdfb406fb60 2023-11-06T02:09:22.580 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.580+0000 7fdfbc2a3640 1 -- 172.21.15.49:0/1542243220 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fdfb40d1240 con 0x7fdfb406fb60 2023-11-06T02:09:22.581 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.580+0000 7fdfa6ffd640 1 -- 172.21.15.49:0/1542243220 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7fdfb40d1240 con 0x7fdfb406fb60 2023-11-06T02:09:22.581 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 8 2023-11-06T02:09:22.583 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.584+0000 7fdfbc2a3640 1 -- 172.21.15.49:0/1542243220 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fdf98065a60 msgr2=0x7fdf98067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:22.583 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.584+0000 7fdfbc2a3640 1 --2- 172.21.15.49:0/1542243220 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fdf98065a60 0x7fdf98067f20 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fdfb00ae000 tx=0x7fdfb00a7000 comp rx=0 tx=0).stop 2023-11-06T02:09:22.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.584+0000 7fdfbc2a3640 1 -- 172.21.15.49:0/1542243220 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdfb406fb60 msgr2=0x7fdfb406af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:22.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.584+0000 7fdfbc2a3640 1 --2- 172.21.15.49:0/1542243220 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdfb406fb60 0x7fdfb406af10 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fdfac00c770 tx=0x7fdfac00cc30 comp rx=0 tx=0).stop 2023-11-06T02:09:22.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.584+0000 7fdfbc2a3640 1 -- 172.21.15.49:0/1542243220 shutdown_connections 2023-11-06T02:09:22.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.584+0000 7fdfbc2a3640 1 --2- 172.21.15.49:0/1542243220 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fdf98065a60 0x7fdf98067f20 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:22.585 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.584+0000 7fdfbc2a3640 1 --2- 172.21.15.49:0/1542243220 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdfb406fb60 0x7fdfb406af10 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:22.585 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.584+0000 7fdfbc2a3640 1 --2- 172.21.15.49:0/1542243220 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdfb40d0df0 0x7fdfb406f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:22.585 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.584+0000 7fdfbc2a3640 1 --2- 172.21.15.49:0/1542243220 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdfb40ada60 0x7fdfb414a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:22.585 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.584+0000 7fdfbc2a3640 1 -- 172.21.15.49:0/1542243220 >> 172.21.15.49:0/1542243220 conn(0x7fdfb4089fd0 msgr2=0x7fdfb40b18e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:22.585 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.584+0000 7fdfbc2a3640 1 -- 172.21.15.49:0/1542243220 shutdown_connections 2023-11-06T02:09:22.586 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.584+0000 7fdfbc2a3640 1 -- 172.21.15.49:0/1542243220 wait complete. 2023-11-06T02:09:22.601 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T02:09:22.722 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.724+0000 7f50022ad640 1 Processor -- start 2023-11-06T02:09:22.722 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.724+0000 7f50022ad640 1 -- start start 2023-11-06T02:09:22.723 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.724+0000 7f50022ad640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f4ffc0af7b0 0x7f4ffc0b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:22.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.724+0000 7f50022ad640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ffc06cf00 con 0x7f4ffc15a710 2023-11-06T02:09:22.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.724+0000 7f50022ad640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ffc06d1e0 con 0x7f4ffc071090 2023-11-06T02:09:22.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.724+0000 7f50022ad640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ffc06d7a0 con 0x7f4ffc0af7b0 2023-11-06T02:09:22.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.724+0000 7f4ffb7fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f4ffc0af7b0 0x7f4ffc0b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:22.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.724+0000 7f4ffb7fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f4ffc0af7b0 0x7f4ffc0b1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41866/0 (socket says 172.21.15.49:41866) 2023-11-06T02:09:22.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.724+0000 7f4ffb7fe640 1 -- 172.21.15.49:0/1229430708 learned_addr learned my addr 172.21.15.49:0/1229430708 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:22.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.724+0000 7f4ffb7fe640 1 -- 172.21.15.49:0/1229430708 >> v1:172.21.15.150:6789/0 conn(0x7f4ffc071090 legacy=0x7f4ffc0b1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:22.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.724+0000 7f4ffb7fe640 1 -- 172.21.15.49:0/1229430708 >> v1:172.21.15.49:6789/0 conn(0x7f4ffc15a710 legacy=0x7f4ffc0af0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:22.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.724+0000 7f4ffb7fe640 1 -- 172.21.15.49:0/1229430708 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ffc06d4c0 con 0x7f4ffc0af7b0 2023-11-06T02:09:22.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.728+0000 7f4ffb7fe640 1 --2- 172.21.15.49:0/1229430708 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4ffc0af7b0 0x7f4ffc0b1a10 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f4fec009870 tx=0x7f4ffc0b2e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e9615e42496a294 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:22.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.728+0000 7f4ffaffd640 1 -- 172.21.15.49:0/1229430708 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fec0ab610 con 0x7f4ffc0af7b0 2023-11-06T02:09:22.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.728+0000 7f4ffaffd640 1 -- 172.21.15.49:0/1229430708 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fec0ab770 con 0x7f4ffc0af7b0 2023-11-06T02:09:22.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.728+0000 7f4ffaffd640 1 -- 172.21.15.49:0/1229430708 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fec0abb00 con 0x7f4ffc0af7b0 2023-11-06T02:09:22.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.728+0000 7f50022ad640 1 -- 172.21.15.49:0/1229430708 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4ffc0af7b0 msgr2=0x7f4ffc0b1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:22.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.728+0000 7f50022ad640 1 --2- 172.21.15.49:0/1229430708 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4ffc0af7b0 0x7f4ffc0b1a10 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f4fec009870 tx=0x7f4ffc0b2e70 comp rx=0 tx=0).stop 2023-11-06T02:09:22.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.728+0000 7f50022ad640 1 -- 172.21.15.49:0/1229430708 shutdown_connections 2023-11-06T02:09:22.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.728+0000 7f50022ad640 1 --2- 172.21.15.49:0/1229430708 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4ffc0af7b0 0x7f4ffc0b1a10 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:22.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.728+0000 7f50022ad640 1 -- 172.21.15.49:0/1229430708 >> 172.21.15.49:0/1229430708 conn(0x7f4ffc0cec90 msgr2=0x7f4ffc0cb0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:22.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.728+0000 7f50022ad640 1 -- 172.21.15.49:0/1229430708 shutdown_connections 2023-11-06T02:09:22.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.728+0000 7f50022ad640 1 -- 172.21.15.49:0/1229430708 wait complete. 2023-11-06T02:09:22.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.728+0000 7f50022ad640 1 Processor -- start 2023-11-06T02:09:22.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.728+0000 7f50022ad640 1 -- start start 2023-11-06T02:09:22.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.728+0000 7f50022ad640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4ffc071090 0x7f4ffc149eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:22.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.728+0000 7f50022ad640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4ffc0af7b0 0x7f4ffc155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:22.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.728+0000 7f50022ad640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4ffc15a710 0x7f4ffc156520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:22.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.728+0000 7f50022ad640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ffc1531c0 con 0x7f4ffc0af7b0 2023-11-06T02:09:22.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.728+0000 7f50022ad640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ffc14a3f0 con 0x7f4ffc15a710 2023-11-06T02:09:22.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.728+0000 7f50022ad640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ffc14ffa0 con 0x7f4ffc071090 2023-11-06T02:09:22.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.728+0000 7f4ffb7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4ffc0af7b0 0x7f4ffc155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:22.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.728+0000 7f4ffb7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4ffc0af7b0 0x7f4ffc155fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41872/0 (socket says 172.21.15.49:41872) 2023-11-06T02:09:22.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.728+0000 7f4ffb7fe640 1 -- 172.21.15.49:0/3556948695 learned_addr learned my addr 172.21.15.49:0/3556948695 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:22.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.728+0000 7f4ffb7fe640 1 -- 172.21.15.49:0/3556948695 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4ffc071090 msgr2=0x7f4ffc149eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:22.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.728+0000 7f4ffb7fe640 1 --2- 172.21.15.49:0/3556948695 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4ffc071090 0x7f4ffc149eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:22.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.728+0000 7f4ffb7fe640 1 -- 172.21.15.49:0/3556948695 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4ffc15a710 msgr2=0x7f4ffc156520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:22.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.728+0000 7f4ffb7fe640 1 --2- 172.21.15.49:0/3556948695 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4ffc15a710 0x7f4ffc156520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:22.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.728+0000 7f4ffb7fe640 1 -- 172.21.15.49:0/3556948695 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fec0094d0 con 0x7f4ffc0af7b0 2023-11-06T02:09:22.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.728+0000 7f4ffb7fe640 1 --2- 172.21.15.49:0/3556948695 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4ffc0af7b0 0x7f4ffc155fe0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f4fec0ad040 tx=0x7f4fec01acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:22.736 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.728+0000 7f4ff8ff9640 1 -- 172.21.15.49:0/3556948695 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fec0b4800 con 0x7f4ffc0af7b0 2023-11-06T02:09:22.736 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.728+0000 7f4ff8ff9640 1 -- 172.21.15.49:0/3556948695 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fec0b4980 con 0x7f4ffc0af7b0 2023-11-06T02:09:22.736 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.728+0000 7f4ff8ff9640 1 -- 172.21.15.49:0/3556948695 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fec0b4d30 con 0x7f4ffc0af7b0 2023-11-06T02:09:22.736 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.728+0000 7f50022ad640 1 -- 172.21.15.49:0/3556948695 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ffc150280 con 0x7f4ffc0af7b0 2023-11-06T02:09:22.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.728+0000 7f50022ad640 1 -- 172.21.15.49:0/3556948695 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ffc150840 con 0x7f4ffc0af7b0 2023-11-06T02:09:22.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.732+0000 7f50022ad640 1 -- 172.21.15.49:0/3556948695 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ffc150a30 con 0x7f4ffc0af7b0 2023-11-06T02:09:22.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.732+0000 7f4ff8ff9640 1 -- 172.21.15.49:0/3556948695 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f4fec0b3380 con 0x7f4ffc0af7b0 2023-11-06T02:09:22.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.732+0000 7f4ff8ff9640 1 --2- 172.21.15.49:0/3556948695 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4fe8065a60 0x7f4fe8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:22.738 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.732+0000 7f4ff8ff9640 1 -- 172.21.15.49:0/3556948695 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f4fec11dff0 con 0x7f4ffc0af7b0 2023-11-06T02:09:22.738 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.732+0000 7f4ffbfff640 1 --2- 172.21.15.49:0/3556948695 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4fe8065a60 0x7f4fe8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:22.738 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.732+0000 7f4ffbfff640 1 --2- 172.21.15.49:0/3556948695 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4fe8065a60 0x7f4fe8067f20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4ff0000f70 tx=0x7f4ff00079c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:22.738 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.736+0000 7f4ff8ff9640 1 -- 172.21.15.49:0/3556948695 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f4fec0eb820 con 0x7f4ffc0af7b0 2023-11-06T02:09:22.886 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.888+0000 7f50022ad640 1 -- 172.21.15.49:0/3556948695 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4ffc074040 con 0x7f4ffc0af7b0 2023-11-06T02:09:22.887 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.888+0000 7f4ff8ff9640 1 -- 172.21.15.49:0/3556948695 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4fec0eee40 con 0x7f4ffc0af7b0 2023-11-06T02:09:22.888 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2023-11-06T02:09:22.890 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.888+0000 7f50022ad640 1 -- 172.21.15.49:0/3556948695 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4fe8065a60 msgr2=0x7f4fe8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:22.890 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.888+0000 7f50022ad640 1 --2- 172.21.15.49:0/3556948695 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4fe8065a60 0x7f4fe8067f20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4ff0000f70 tx=0x7f4ff00079c0 comp rx=0 tx=0).stop 2023-11-06T02:09:22.890 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.888+0000 7f50022ad640 1 -- 172.21.15.49:0/3556948695 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4ffc0af7b0 msgr2=0x7f4ffc155fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:22.890 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.888+0000 7f50022ad640 1 --2- 172.21.15.49:0/3556948695 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4ffc0af7b0 0x7f4ffc155fe0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f4fec0ad040 tx=0x7f4fec01acb0 comp rx=0 tx=0).stop 2023-11-06T02:09:22.891 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.892+0000 7f50022ad640 1 -- 172.21.15.49:0/3556948695 shutdown_connections 2023-11-06T02:09:22.891 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.892+0000 7f50022ad640 1 --2- 172.21.15.49:0/3556948695 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4fe8065a60 0x7f4fe8067f20 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:22.891 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.892+0000 7f50022ad640 1 --2- 172.21.15.49:0/3556948695 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4ffc0af7b0 0x7f4ffc155fe0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:22.891 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.892+0000 7f50022ad640 1 --2- 172.21.15.49:0/3556948695 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4ffc15a710 0x7f4ffc156520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:22.891 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.892+0000 7f50022ad640 1 --2- 172.21.15.49:0/3556948695 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4ffc071090 0x7f4ffc149eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:22.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.892+0000 7f50022ad640 1 -- 172.21.15.49:0/3556948695 >> 172.21.15.49:0/3556948695 conn(0x7f4ffc0cec90 msgr2=0x7f4ffc1596c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:22.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.892+0000 7f50022ad640 1 -- 172.21.15.49:0/3556948695 shutdown_connections 2023-11-06T02:09:22.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:22.892+0000 7f50022ad640 1 -- 172.21.15.49:0/3556948695 wait complete. 2023-11-06T02:09:22.905 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T02:09:23.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.020+0000 7f8501d1e640 1 Processor -- start 2023-11-06T02:09:23.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.020+0000 7f8501d1e640 1 -- start start 2023-11-06T02:09:23.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.020+0000 7f8501d1e640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f84fc15a880 0x7f84fc0b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:23.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.020+0000 7f8501d1e640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84fc0ae360 con 0x7f84fc070160 2023-11-06T02:09:23.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.020+0000 7f8501d1e640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84fc0ae640 con 0x7f84fc06d860 2023-11-06T02:09:23.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.020+0000 7f8501d1e640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84fc0ae920 con 0x7f84fc15a880 2023-11-06T02:09:23.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.020+0000 7f84faffd640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f84fc070160 0x7f84fc06d150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:35506/0 (socket says 172.21.15.49:35506) 2023-11-06T02:09:23.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.020+0000 7f84faffd640 1 -- 172.21.15.49:0/1046770270 learned_addr learned my addr 172.21.15.49:0/1046770270 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:09:23.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.020+0000 7f84fb7fe640 1 --2- 172.21.15.49:0/1046770270 >> v2:172.21.15.150:3300/0 conn(0x7f84fc15a880 0x7f84fc0b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:23.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.020+0000 7f84fa7fc640 1 -- 172.21.15.49:0/1046770270 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1040694271 0 0) 0x7f84fc0ae360 con 0x7f84fc070160 2023-11-06T02:09:23.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.020+0000 7f84fa7fc640 1 -- 172.21.15.49:0/1046770270 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84e4003530 con 0x7f84fc070160 2023-11-06T02:09:23.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.020+0000 7f84fa7fc640 1 -- 172.21.15.49:0/1046770270 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 18736099 0 0) 0x7f84e4003530 con 0x7f84fc070160 2023-11-06T02:09:23.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.020+0000 7f84fa7fc640 1 -- 172.21.15.49:0/1046770270 >> v2:172.21.15.150:3300/0 conn(0x7f84fc15a880 msgr2=0x7f84fc0b15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:23.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.020+0000 7f84fa7fc640 1 --2- 172.21.15.49:0/1046770270 >> v2:172.21.15.150:3300/0 conn(0x7f84fc15a880 0x7f84fc0b15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:23.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.020+0000 7f84fa7fc640 1 -- 172.21.15.49:0/1046770270 >> v1:172.21.15.150:6789/0 conn(0x7f84fc06d860 legacy=0x7f84fc0af120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:23.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.020+0000 7f84fa7fc640 1 -- 172.21.15.49:0/1046770270 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84fc0aec00 con 0x7f84fc070160 2023-11-06T02:09:23.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.020+0000 7f84fa7fc640 1 -- 172.21.15.49:0/1046770270 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7f84f0001680 con 0x7f84fc070160 2023-11-06T02:09:23.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.020+0000 7f84fa7fc640 1 -- 172.21.15.49:0/1046770270 >> v1:172.21.15.49:6789/0 conn(0x7f84fc070160 legacy=0x7f84fc06d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:23.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.020+0000 7f84fa7fc640 1 --2- 172.21.15.49:0/1046770270 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84e4003980 0x7f84e4003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:23.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.020+0000 7f84fa7fc640 1 --2- 172.21.15.49:0/1046770270 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f84e40047d0 0x7f84e4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:23.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.020+0000 7f84fa7fc640 1 --2- 172.21.15.49:0/1046770270 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84e4005190 0x7f84e400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:23.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.020+0000 7f84fa7fc640 1 -- 172.21.15.49:0/1046770270 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84fc0ae920 con 0x7f84e4005190 2023-11-06T02:09:23.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.020+0000 7f84fa7fc640 1 -- 172.21.15.49:0/1046770270 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84e400c030 con 0x7f84e4003980 2023-11-06T02:09:23.026 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.020+0000 7f84fa7fc640 1 -- 172.21.15.49:0/1046770270 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84e400c1f0 con 0x7f84e40047d0 2023-11-06T02:09:23.026 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.020+0000 7f84fbfff640 1 --2- 172.21.15.49:0/1046770270 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84e4005190 0x7f84e400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:23.026 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.020+0000 7f84faffd640 1 --2- 172.21.15.49:0/1046770270 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f84e40047d0 0x7f84e4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:23.026 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.020+0000 7f84fbfff640 1 -- 172.21.15.49:0/1046770270 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f84e40047d0 msgr2=0x7f84e4004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:23.026 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.020+0000 7f84fbfff640 1 --2- 172.21.15.49:0/1046770270 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f84e40047d0 0x7f84e4004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:23.026 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.020+0000 7f84fbfff640 1 -- 172.21.15.49:0/1046770270 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84e4003980 msgr2=0x7f84e4003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:23.027 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.020+0000 7f84fbfff640 1 --2- 172.21.15.49:0/1046770270 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84e4003980 0x7f84e4003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:23.027 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.020+0000 7f84fbfff640 1 -- 172.21.15.49:0/1046770270 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84e4003530 con 0x7f84e4005190 2023-11-06T02:09:23.027 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.020+0000 7f84faffd640 1 --2- 172.21.15.49:0/1046770270 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f84e40047d0 0x7f84e4004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:09:23.031 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.024+0000 7f84fbfff640 1 -- 172.21.15.49:0/1046770270 reap_dead start 2023-11-06T02:09:23.031 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.024+0000 7f84fbfff640 1 --2- 172.21.15.49:0/1046770270 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84e4005190 0x7f84e400b990 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f84f4001e70 tx=0x7f84f401c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b32ef46d52a97524 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:23.031 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.024+0000 7f84fa7fc640 1 -- 172.21.15.49:0/1046770270 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f4003b80 con 0x7f84e4005190 2023-11-06T02:09:23.031 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.024+0000 7f84fa7fc640 1 -- 172.21.15.49:0/1046770270 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84f4003d20 con 0x7f84e4005190 2023-11-06T02:09:23.032 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.024+0000 7f84fa7fc640 1 -- 172.21.15.49:0/1046770270 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f4005260 con 0x7f84e4005190 2023-11-06T02:09:23.032 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.024+0000 7f8501d1e640 1 -- 172.21.15.49:0/1046770270 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84e4005190 msgr2=0x7f84e400b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:23.032 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.024+0000 7f8501d1e640 1 --2- 172.21.15.49:0/1046770270 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84e4005190 0x7f84e400b990 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f84f4001e70 tx=0x7f84f401c760 comp rx=0 tx=0).stop 2023-11-06T02:09:23.032 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.024+0000 7f8501d1e640 1 -- 172.21.15.49:0/1046770270 shutdown_connections 2023-11-06T02:09:23.032 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.024+0000 7f8501d1e640 1 --2- 172.21.15.49:0/1046770270 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84e4005190 0x7f84e400b990 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:23.033 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.024+0000 7f8501d1e640 1 -- 172.21.15.49:0/1046770270 >> 172.21.15.49:0/1046770270 conn(0x7f84fc06a370 msgr2=0x7f84fc0ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:23.033 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.024+0000 7f8501d1e640 1 -- 172.21.15.49:0/1046770270 shutdown_connections 2023-11-06T02:09:23.033 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.024+0000 7f8501d1e640 1 -- 172.21.15.49:0/1046770270 wait complete. 2023-11-06T02:09:23.033 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.024+0000 7f8501d1e640 1 Processor -- start 2023-11-06T02:09:23.034 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.024+0000 7f8501d1e640 1 -- start start 2023-11-06T02:09:23.034 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.024+0000 7f8501d1e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84e4005190 0x7f84fc14d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:23.034 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.024+0000 7f8501d1e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f84fc14f320 0x7f84fc1ed330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:23.034 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.024+0000 7f8501d1e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84fc0b3790 0x7f84fc0b3c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:23.034 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.024+0000 7f8501d1e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84fc0b4150 con 0x7f84fc0b3790 2023-11-06T02:09:23.035 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.024+0000 7f8501d1e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84fc1f4530 con 0x7f84e4005190 2023-11-06T02:09:23.035 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.024+0000 7f8501d1e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84fc14de30 con 0x7f84fc14f320 2023-11-06T02:09:23.035 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.024+0000 7f84fbfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84fc0b3790 0x7f84fc0b3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:23.035 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.024+0000 7f84fbfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84fc0b3790 0x7f84fc0b3c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41882/0 (socket says 172.21.15.49:41882) 2023-11-06T02:09:23.035 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.024+0000 7f84fbfff640 1 -- 172.21.15.49:0/2801741892 learned_addr learned my addr 172.21.15.49:0/2801741892 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:23.036 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.024+0000 7f84faffd640 1 --2- 172.21.15.49:0/2801741892 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f84fc14f320 0x7f84fc1ed330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:23.036 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.024+0000 7f84fb7fe640 1 --2- 172.21.15.49:0/2801741892 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84e4005190 0x7f84fc14d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:23.036 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.024+0000 7f84fbfff640 1 -- 172.21.15.49:0/2801741892 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f84fc14f320 msgr2=0x7f84fc1ed330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:23.036 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.024+0000 7f84fbfff640 1 --2- 172.21.15.49:0/2801741892 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f84fc14f320 0x7f84fc1ed330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:23.036 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.028+0000 7f84fbfff640 1 -- 172.21.15.49:0/2801741892 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84e4005190 msgr2=0x7f84fc14d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:23.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.028+0000 7f84fbfff640 1 --2- 172.21.15.49:0/2801741892 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84e4005190 0x7f84fc14d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:23.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.028+0000 7f84fbfff640 1 -- 172.21.15.49:0/2801741892 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84fc15ade0 con 0x7f84fc0b3790 2023-11-06T02:09:23.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.028+0000 7f84faffd640 1 --2- 172.21.15.49:0/2801741892 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f84fc14f320 0x7f84fc1ed330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:09:23.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.028+0000 7f84fbfff640 1 --2- 172.21.15.49:0/2801741892 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84fc0b3790 0x7f84fc0b3c10 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f84f401cc40 tx=0x7f84e4005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:23.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.028+0000 7f84e3fff640 1 -- 172.21.15.49:0/2801741892 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f40ac070 con 0x7f84fc0b3790 2023-11-06T02:09:23.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.028+0000 7f84e3fff640 1 -- 172.21.15.49:0/2801741892 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84f4003e90 con 0x7f84fc0b3790 2023-11-06T02:09:23.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.028+0000 7f8501d1e640 1 -- 172.21.15.49:0/2801741892 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84fc1f39b0 con 0x7f84fc0b3790 2023-11-06T02:09:23.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.028+0000 7f84e3fff640 1 -- 172.21.15.49:0/2801741892 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f4006da0 con 0x7f84fc0b3790 2023-11-06T02:09:23.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.028+0000 7f8501d1e640 1 -- 172.21.15.49:0/2801741892 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84fc1f3f70 con 0x7f84fc0b3790 2023-11-06T02:09:23.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.028+0000 7f8501d1e640 1 -- 172.21.15.49:0/2801741892 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84fc074040 con 0x7f84fc0b3790 2023-11-06T02:09:23.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.028+0000 7f84e3fff640 1 -- 172.21.15.49:0/2801741892 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f84f4007030 con 0x7f84fc0b3790 2023-11-06T02:09:23.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.028+0000 7f84e3fff640 1 --2- 172.21.15.49:0/2801741892 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f84e8065a90 0x7f84e8067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:23.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.028+0000 7f84e3fff640 1 -- 172.21.15.49:0/2801741892 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f84f411bf80 con 0x7f84fc0b3790 2023-11-06T02:09:23.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.028+0000 7f84fb7fe640 1 --2- 172.21.15.49:0/2801741892 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f84e8065a90 0x7f84e8067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:23.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.028+0000 7f84fb7fe640 1 --2- 172.21.15.49:0/2801741892 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f84e8065a90 0x7f84e8067f50 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f84ec0047c0 tx=0x7f84ec008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:23.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.032+0000 7f84e3fff640 1 -- 172.21.15.49:0/2801741892 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f84f40e9850 con 0x7f84fc0b3790 2023-11-06T02:09:23.184 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.184+0000 7f8501d1e640 1 -- 172.21.15.49:0/2801741892 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f84fc13eec0 con 0x7f84fc0b3790 2023-11-06T02:09:23.185 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.184+0000 7f84e3fff640 1 -- 172.21.15.49:0/2801741892 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f84f40b6020 con 0x7f84fc0b3790 2023-11-06T02:09:23.185 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2023-11-06T02:09:23.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.188+0000 7f8501d1e640 1 -- 172.21.15.49:0/2801741892 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f84e8065a90 msgr2=0x7f84e8067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:23.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.188+0000 7f8501d1e640 1 --2- 172.21.15.49:0/2801741892 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f84e8065a90 0x7f84e8067f50 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f84ec0047c0 tx=0x7f84ec008040 comp rx=0 tx=0).stop 2023-11-06T02:09:23.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.188+0000 7f8501d1e640 1 -- 172.21.15.49:0/2801741892 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84fc0b3790 msgr2=0x7f84fc0b3c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:23.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.188+0000 7f8501d1e640 1 --2- 172.21.15.49:0/2801741892 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84fc0b3790 0x7f84fc0b3c10 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f84f401cc40 tx=0x7f84e4005800 comp rx=0 tx=0).stop 2023-11-06T02:09:23.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.188+0000 7f8501d1e640 1 -- 172.21.15.49:0/2801741892 shutdown_connections 2023-11-06T02:09:23.188 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.188+0000 7f8501d1e640 1 --2- 172.21.15.49:0/2801741892 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f84e8065a90 0x7f84e8067f50 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:23.188 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.188+0000 7f8501d1e640 1 --2- 172.21.15.49:0/2801741892 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84fc0b3790 0x7f84fc0b3c10 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:23.188 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.188+0000 7f8501d1e640 1 --2- 172.21.15.49:0/2801741892 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f84fc14f320 0x7f84fc1ed330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:23.188 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.188+0000 7f8501d1e640 1 --2- 172.21.15.49:0/2801741892 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84e4005190 0x7f84fc14d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:23.188 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.188+0000 7f8501d1e640 1 -- 172.21.15.49:0/2801741892 >> 172.21.15.49:0/2801741892 conn(0x7f84fc06a370 msgr2=0x7f84fc15c220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:23.189 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.188+0000 7f8501d1e640 1 -- 172.21.15.49:0/2801741892 shutdown_connections 2023-11-06T02:09:23.189 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.188+0000 7f8501d1e640 1 -- 172.21.15.49:0/2801741892 wait complete. 2023-11-06T02:09:23.203 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-06T02:09:23.326 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.328+0000 7fa4fb95c640 1 Processor -- start 2023-11-06T02:09:23.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.328+0000 7fa4fb95c640 1 -- start start 2023-11-06T02:09:23.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.328+0000 7fa4fb95c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa4f4071bb0 0x7fa4f4161650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:23.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.328+0000 7fa4fb95c640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4f40d17c0 con 0x7fa4f40cda70 2023-11-06T02:09:23.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.328+0000 7fa4fb95c640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4f40ce6c0 con 0x7fa4f40b1f90 2023-11-06T02:09:23.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.328+0000 7fa4fb95c640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4f40cdf20 con 0x7fa4f4071bb0 2023-11-06T02:09:23.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.328+0000 7fa4f9ef9640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa4f4071bb0 0x7fa4f4161650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:23.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.328+0000 7fa4f9ef9640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa4f4071bb0 0x7fa4f4161650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41892/0 (socket says 172.21.15.49:41892) 2023-11-06T02:09:23.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.328+0000 7fa4f9ef9640 1 -- 172.21.15.49:0/208452115 learned_addr learned my addr 172.21.15.49:0/208452115 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:23.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.328+0000 7fa4f9ef9640 1 -- 172.21.15.49:0/208452115 >> v1:172.21.15.150:6789/0 conn(0x7fa4f40b1f90 legacy=0x7fa4f40d1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:23.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.328+0000 7fa4f9ef9640 1 -- 172.21.15.49:0/208452115 >> v1:172.21.15.49:6789/0 conn(0x7fa4f40cda70 legacy=0x7fa4f40714a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:23.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.328+0000 7fa4f9ef9640 1 -- 172.21.15.49:0/208452115 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4f40d2360 con 0x7fa4f4071bb0 2023-11-06T02:09:23.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.332+0000 7fa4f9ef9640 1 --2- 172.21.15.49:0/208452115 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4f4071bb0 0x7fa4f4161650 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec009e50 tx=0x7fa4ec01ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66f41bcbaccadd2c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:23.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.332+0000 7fa4ebfff640 1 -- 172.21.15.49:0/208452115 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4ec0a57d0 con 0x7fa4f4071bb0 2023-11-06T02:09:23.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.332+0000 7fa4ebfff640 1 -- 172.21.15.49:0/208452115 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4ec0a5930 con 0x7fa4f4071bb0 2023-11-06T02:09:23.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.332+0000 7fa4ebfff640 1 -- 172.21.15.49:0/208452115 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4ec0a5cc0 con 0x7fa4f4071bb0 2023-11-06T02:09:23.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.332+0000 7fa4fb95c640 1 -- 172.21.15.49:0/208452115 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4f4071bb0 msgr2=0x7fa4f4161650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:23.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.332+0000 7fa4fb95c640 1 --2- 172.21.15.49:0/208452115 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4f4071bb0 0x7fa4f4161650 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec009e50 tx=0x7fa4ec01ca10 comp rx=0 tx=0).stop 2023-11-06T02:09:23.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.332+0000 7fa4fb95c640 1 -- 172.21.15.49:0/208452115 shutdown_connections 2023-11-06T02:09:23.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.332+0000 7fa4fb95c640 1 --2- 172.21.15.49:0/208452115 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4f4071bb0 0x7fa4f4161650 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:23.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.332+0000 7fa4fb95c640 1 -- 172.21.15.49:0/208452115 >> 172.21.15.49:0/208452115 conn(0x7fa4f40d2aa0 msgr2=0x7fa4f40d2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:23.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.332+0000 7fa4fb95c640 1 -- 172.21.15.49:0/208452115 shutdown_connections 2023-11-06T02:09:23.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.332+0000 7fa4fb95c640 1 -- 172.21.15.49:0/208452115 wait complete. 2023-11-06T02:09:23.334 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.332+0000 7fa4fb95c640 1 Processor -- start 2023-11-06T02:09:23.335 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.332+0000 7fa4fb95c640 1 -- start start 2023-11-06T02:09:23.335 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.332+0000 7fa4fb95c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4f4071bb0 0x7fa4f4158f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:23.335 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.332+0000 7fa4fb95c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa4f40b1f90 0x7fa4f415f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:23.335 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.332+0000 7fa4fb95c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4f40cda70 0x7fa4f40b7b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:23.335 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.332+0000 7fa4fb95c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f415fcd0 con 0x7fa4f4071bb0 2023-11-06T02:09:23.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.332+0000 7fa4fb95c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f40c9b80 con 0x7fa4f40cda70 2023-11-06T02:09:23.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.332+0000 7fa4fb95c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f40c9e60 con 0x7fa4f40b1f90 2023-11-06T02:09:23.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.332+0000 7fa4f96f8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4f4071bb0 0x7fa4f4158f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:23.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.332+0000 7fa4f96f8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4f4071bb0 0x7fa4f4158f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41906/0 (socket says 172.21.15.49:41906) 2023-11-06T02:09:23.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.332+0000 7fa4f96f8640 1 -- 172.21.15.49:0/3782160655 learned_addr learned my addr 172.21.15.49:0/3782160655 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:23.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.332+0000 7fa4f9ef9640 1 --2- 172.21.15.49:0/3782160655 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4f40cda70 0x7fa4f40b7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:23.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.332+0000 7fa4f8ef7640 1 --2- 172.21.15.49:0/3782160655 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa4f40b1f90 0x7fa4f415f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:23.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.336+0000 7fa4f96f8640 1 -- 172.21.15.49:0/3782160655 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa4f40b1f90 msgr2=0x7fa4f415f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:23.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.336+0000 7fa4f96f8640 1 --2- 172.21.15.49:0/3782160655 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa4f40b1f90 0x7fa4f415f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:23.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.336+0000 7fa4f96f8640 1 -- 172.21.15.49:0/3782160655 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4f40cda70 msgr2=0x7fa4f40b7b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:23.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.336+0000 7fa4f96f8640 1 --2- 172.21.15.49:0/3782160655 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4f40cda70 0x7fa4f40b7b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:23.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.336+0000 7fa4f96f8640 1 -- 172.21.15.49:0/3782160655 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4ec009b30 con 0x7fa4f4071bb0 2023-11-06T02:09:23.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.336+0000 7fa4f8ef7640 1 --2- 172.21.15.49:0/3782160655 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa4f40b1f90 0x7fa4f415f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:09:23.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.336+0000 7fa4f96f8640 1 --2- 172.21.15.49:0/3782160655 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4f4071bb0 0x7fa4f4158f90 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa4f000c500 tx=0x7fa4f000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:23.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.336+0000 7fa4e9ffb640 1 -- 172.21.15.49:0/3782160655 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f0005220 con 0x7fa4f4071bb0 2023-11-06T02:09:23.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.336+0000 7fa4e9ffb640 1 -- 172.21.15.49:0/3782160655 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4f0005380 con 0x7fa4f4071bb0 2023-11-06T02:09:23.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.336+0000 7fa4fb95c640 1 -- 172.21.15.49:0/3782160655 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4f40ca140 con 0x7fa4f4071bb0 2023-11-06T02:09:23.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.336+0000 7fa4e9ffb640 1 -- 172.21.15.49:0/3782160655 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f0007270 con 0x7fa4f4071bb0 2023-11-06T02:09:23.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.336+0000 7fa4fb95c640 1 -- 172.21.15.49:0/3782160655 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4f41594d0 con 0x7fa4f4071bb0 2023-11-06T02:09:23.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.336+0000 7fa4fb95c640 1 -- 172.21.15.49:0/3782160655 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4f4074040 con 0x7fa4f4071bb0 2023-11-06T02:09:23.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.336+0000 7fa4e9ffb640 1 -- 172.21.15.49:0/3782160655 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fa4f00073d0 con 0x7fa4f4071bb0 2023-11-06T02:09:23.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.336+0000 7fa4e9ffb640 1 --2- 172.21.15.49:0/3782160655 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa4d8065a90 0x7fa4d8067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:23.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.336+0000 7fa4e9ffb640 1 -- 172.21.15.49:0/3782160655 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fa4f0082b50 con 0x7fa4f4071bb0 2023-11-06T02:09:23.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.336+0000 7fa4f8ef7640 1 --2- 172.21.15.49:0/3782160655 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa4d8065a90 0x7fa4d8067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:23.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.336+0000 7fa4f8ef7640 1 --2- 172.21.15.49:0/3782160655 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa4d8065a90 0x7fa4d8067f50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa4f40b1990 tx=0x7fa4e4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:23.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.340+0000 7fa4e9ffb640 1 -- 172.21.15.49:0/3782160655 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fa4f0050380 con 0x7fa4f4071bb0 2023-11-06T02:09:23.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.504+0000 7fa4fb95c640 1 -- 172.21.15.49:0/3782160655 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa4f40d0c30 con 0x7fa4f4071bb0 2023-11-06T02:09:23.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.504+0000 7fa4e9ffb640 1 -- 172.21.15.49:0/3782160655 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7fa4f00539a0 con 0x7fa4f4071bb0 2023-11-06T02:09:23.505 INFO:teuthology.orchestra.run.smithi049.stderr:listed 0 entries 2023-11-06T02:09:23.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.508+0000 7fa4fb95c640 1 -- 172.21.15.49:0/3782160655 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa4d8065a90 msgr2=0x7fa4d8067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:23.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.508+0000 7fa4fb95c640 1 --2- 172.21.15.49:0/3782160655 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa4d8065a90 0x7fa4d8067f50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa4f40b1990 tx=0x7fa4e4009040 comp rx=0 tx=0).stop 2023-11-06T02:09:23.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.508+0000 7fa4fb95c640 1 -- 172.21.15.49:0/3782160655 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4f4071bb0 msgr2=0x7fa4f4158f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:23.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.508+0000 7fa4fb95c640 1 --2- 172.21.15.49:0/3782160655 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4f4071bb0 0x7fa4f4158f90 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa4f000c500 tx=0x7fa4f000c9c0 comp rx=0 tx=0).stop 2023-11-06T02:09:23.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.508+0000 7fa4fb95c640 1 -- 172.21.15.49:0/3782160655 shutdown_connections 2023-11-06T02:09:23.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.508+0000 7fa4fb95c640 1 --2- 172.21.15.49:0/3782160655 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa4d8065a90 0x7fa4d8067f50 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:23.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.508+0000 7fa4fb95c640 1 --2- 172.21.15.49:0/3782160655 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4f40cda70 0x7fa4f40b7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:23.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.508+0000 7fa4fb95c640 1 --2- 172.21.15.49:0/3782160655 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa4f40b1f90 0x7fa4f415f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:23.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.508+0000 7fa4fb95c640 1 --2- 172.21.15.49:0/3782160655 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4f4071bb0 0x7fa4f4158f90 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:23.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.508+0000 7fa4fb95c640 1 -- 172.21.15.49:0/3782160655 >> 172.21.15.49:0/3782160655 conn(0x7fa4f40d2aa0 msgr2=0x7fa4f415bc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:23.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.508+0000 7fa4fb95c640 1 -- 172.21.15.49:0/3782160655 shutdown_connections 2023-11-06T02:09:23.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.508+0000 7fa4fb95c640 1 -- 172.21.15.49:0/3782160655 wait complete. 2023-11-06T02:09:23.522 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-11-06T02:09:23.523 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T02:09:23.530 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T02:09:23.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.644+0000 7f6ba64c7640 1 Processor -- start 2023-11-06T02:09:23.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.644+0000 7f6ba64c7640 1 -- start start 2023-11-06T02:09:23.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.644+0000 7f6ba64c7640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f6ba0159b40 0x7f6ba00ca980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:23.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.644+0000 7f6ba64c7640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f6ba00cdd50 0x7f6ba00cc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:23.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.644+0000 7f6ba64c7640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ba00723a0 con 0x7f6ba00cb010 2023-11-06T02:09:23.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.644+0000 7f6ba64c7640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ba00715e0 con 0x7f6ba00cdd50 2023-11-06T02:09:23.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.644+0000 7f6ba64c7640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ba00718c0 con 0x7f6ba0159b40 2023-11-06T02:09:23.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.644+0000 7f6ba4a64640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f6ba00cdd50 0x7f6ba00cc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:23.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.644+0000 7f6ba4a64640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f6ba00cdd50 0x7f6ba00cc1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41918/0 (socket says 172.21.15.49:41918) 2023-11-06T02:09:23.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.644+0000 7f6ba4a64640 1 -- 172.21.15.49:0/1697722417 learned_addr learned my addr 172.21.15.49:0/1697722417 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:23.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.644+0000 7f6b97fff640 1 --2- 172.21.15.49:0/1697722417 >> v2:172.21.15.150:3300/0 conn(0x7f6ba0159b40 0x7f6ba00ca980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:23.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.644+0000 7f6ba4a64640 1 -- 172.21.15.49:0/1697722417 >> v2:172.21.15.150:3300/0 conn(0x7f6ba0159b40 msgr2=0x7f6ba00ca980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:23.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.644+0000 7f6ba4a64640 1 --2- 172.21.15.49:0/1697722417 >> v2:172.21.15.150:3300/0 conn(0x7f6ba0159b40 0x7f6ba00ca980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:23.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.644+0000 7f6ba4a64640 1 -- 172.21.15.49:0/1697722417 >> v1:172.21.15.49:6789/0 conn(0x7f6ba00cb010 legacy=0x7f6ba00cb3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:23.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.644+0000 7f6ba4a64640 1 -- 172.21.15.49:0/1697722417 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ba0071ba0 con 0x7f6ba00cdd50 2023-11-06T02:09:23.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.644+0000 7f6ba4a64640 1 --2- 172.21.15.49:0/1697722417 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ba00cdd50 0x7f6ba00cc1d0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f6ba006fef0 tx=0x7f6b9801c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9bd490dcd1cb7e2 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:23.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.648+0000 7f6b9f7fe640 1 -- 172.21.15.49:0/1697722417 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b980a5880 con 0x7f6ba00cdd50 2023-11-06T02:09:23.650 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.648+0000 7f6b9f7fe640 1 -- 172.21.15.49:0/1697722417 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b980a59e0 con 0x7f6ba00cdd50 2023-11-06T02:09:23.650 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.648+0000 7f6b9f7fe640 1 -- 172.21.15.49:0/1697722417 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b980a5d70 con 0x7f6ba00cdd50 2023-11-06T02:09:23.650 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.648+0000 7f6ba64c7640 1 -- 172.21.15.49:0/1697722417 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ba00cdd50 msgr2=0x7f6ba00cc1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:23.650 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.648+0000 7f6ba64c7640 1 --2- 172.21.15.49:0/1697722417 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ba00cdd50 0x7f6ba00cc1d0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f6ba006fef0 tx=0x7f6b9801c9a0 comp rx=0 tx=0).stop 2023-11-06T02:09:23.651 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.648+0000 7f6ba64c7640 1 -- 172.21.15.49:0/1697722417 shutdown_connections 2023-11-06T02:09:23.651 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.648+0000 7f6ba64c7640 1 --2- 172.21.15.49:0/1697722417 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ba00cdd50 0x7f6ba00cc1d0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:23.651 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.648+0000 7f6ba64c7640 1 --2- 172.21.15.49:0/1697722417 >> v2:172.21.15.150:3300/0 conn(0x7f6ba0159b40 0x7f6ba00ca980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:23.651 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.648+0000 7f6ba64c7640 1 -- 172.21.15.49:0/1697722417 >> 172.21.15.49:0/1697722417 conn(0x7f6ba006c0a0 msgr2=0x7f6ba006c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:23.651 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.648+0000 7f6ba64c7640 1 -- 172.21.15.49:0/1697722417 shutdown_connections 2023-11-06T02:09:23.652 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.648+0000 7f6ba64c7640 1 -- 172.21.15.49:0/1697722417 wait complete. 2023-11-06T02:09:23.652 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.648+0000 7f6ba64c7640 1 Processor -- start 2023-11-06T02:09:23.652 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.648+0000 7f6ba64c7640 1 -- start start 2023-11-06T02:09:23.652 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.648+0000 7f6ba64c7640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ba00cb010 0x7f6ba015bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:23.652 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.648+0000 7f6ba64c7640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ba00cdd50 0x7f6ba01f6700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:23.653 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.648+0000 7f6ba64c7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ba0159b40 0x7f6ba01f6c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:23.653 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.648+0000 7f6ba64c7640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba00c32c0 con 0x7f6ba0159b40 2023-11-06T02:09:23.653 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.648+0000 7f6ba64c7640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba00c2e30 con 0x7f6ba00cdd50 2023-11-06T02:09:23.653 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.648+0000 7f6ba64c7640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba00c88c0 con 0x7f6ba00cb010 2023-11-06T02:09:23.654 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.648+0000 7f6ba4a64640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ba0159b40 0x7f6ba01f6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:23.654 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.648+0000 7f6ba4a64640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ba0159b40 0x7f6ba01f6c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41926/0 (socket says 172.21.15.49:41926) 2023-11-06T02:09:23.654 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.648+0000 7f6ba4a64640 1 -- 172.21.15.49:0/2923549082 learned_addr learned my addr 172.21.15.49:0/2923549082 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:23.654 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.648+0000 7f6b97fff640 1 --2- 172.21.15.49:0/2923549082 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ba00cdd50 0x7f6ba01f6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:23.654 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.648+0000 7f6b9ffff640 1 --2- 172.21.15.49:0/2923549082 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ba00cb010 0x7f6ba015bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:23.655 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.648+0000 7f6ba4a64640 1 -- 172.21.15.49:0/2923549082 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ba00cb010 msgr2=0x7f6ba015bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:23.655 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.648+0000 7f6ba4a64640 1 --2- 172.21.15.49:0/2923549082 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ba00cb010 0x7f6ba015bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:23.655 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.648+0000 7f6ba4a64640 1 -- 172.21.15.49:0/2923549082 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ba00cdd50 msgr2=0x7f6ba01f6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:23.655 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.648+0000 7f6ba4a64640 1 --2- 172.21.15.49:0/2923549082 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ba00cdd50 0x7f6ba01f6700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:23.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.648+0000 7f6ba4a64640 1 -- 172.21.15.49:0/2923549082 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b98009b30 con 0x7f6ba0159b40 2023-11-06T02:09:23.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.648+0000 7f6b9ffff640 1 --2- 172.21.15.49:0/2923549082 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ba00cb010 0x7f6ba015bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:09:23.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.648+0000 7f6ba4a64640 1 --2- 172.21.15.49:0/2923549082 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ba0159b40 0x7f6ba01f6c40 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f6b9801a5d0 tx=0x7f6b98019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:23.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.648+0000 7f6b9d7fa640 1 -- 172.21.15.49:0/2923549082 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b980b6680 con 0x7f6ba0159b40 2023-11-06T02:09:23.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.648+0000 7f6b9d7fa640 1 -- 172.21.15.49:0/2923549082 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b980b6800 con 0x7f6ba0159b40 2023-11-06T02:09:23.657 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.648+0000 7f6b9d7fa640 1 -- 172.21.15.49:0/2923549082 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b980b6bb0 con 0x7f6ba0159b40 2023-11-06T02:09:23.657 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.648+0000 7f6ba64c7640 1 -- 172.21.15.49:0/2923549082 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ba00c8300 con 0x7f6ba0159b40 2023-11-06T02:09:23.657 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.648+0000 7f6ba64c7640 1 -- 172.21.15.49:0/2923549082 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ba00c2ac0 con 0x7f6ba0159b40 2023-11-06T02:09:23.658 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.652+0000 7f6ba64c7640 1 -- 172.21.15.49:0/2923549082 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ba00c2fc0 con 0x7f6ba0159b40 2023-11-06T02:09:23.658 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.652+0000 7f6b9d7fa640 1 -- 172.21.15.49:0/2923549082 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f6b980b6d10 con 0x7f6ba0159b40 2023-11-06T02:09:23.659 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.652+0000 7f6b9d7fa640 1 --2- 172.21.15.49:0/2923549082 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6b80065a60 0x7f6b80067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:23.659 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.652+0000 7f6b9d7fa640 1 -- 172.21.15.49:0/2923549082 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f6b9811e5d0 con 0x7f6ba0159b40 2023-11-06T02:09:23.659 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.652+0000 7f6b9ffff640 1 --2- 172.21.15.49:0/2923549082 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6b80065a60 0x7f6b80067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:23.659 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.652+0000 7f6b9ffff640 1 --2- 172.21.15.49:0/2923549082 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6b80065a60 0x7f6b80067f20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6b90000fe0 tx=0x7f6b90009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:23.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.656+0000 7f6b9d7fa640 1 -- 172.21.15.49:0/2923549082 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f6b980ebf20 con 0x7f6ba0159b40 2023-11-06T02:09:23.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.812+0000 7f6ba64c7640 1 -- 172.21.15.49:0/2923549082 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6ba0074040 con 0x7f6ba0159b40 2023-11-06T02:09:23.812 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.812+0000 7f6b9d7fa640 1 -- 172.21.15.49:0/2923549082 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17906 (secure 0 0 0) 0x7f6b980ef540 con 0x7f6ba0159b40 2023-11-06T02:09:23.812 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:09:23.812 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":25,"fsid":"1a8b035c-6ba4-499a-b876-aea47fd9c0d4","created":"2023-11-06T02:08:06.440857+0000","modified":"2023-11-06T02:09:13.195118+0000","last_up_change":"2023-11-06T02:08:17.982372+0000","last_in_change":"2023-11-06T02:08:08.906631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T02:08:19.252222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T02:08:24.776211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-06T02:08:28.936178+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-06T02:08:29.563468+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ad55af8-0571-4e6a-8722-b5d429b7496d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6809","nonce":38116},{"type":"v1","addr":"172.21.15.49:6811","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6813","nonce":38116},{"type":"v1","addr":"172.21.15.49:6815","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6821","nonce":38116},{"type":"v1","addr":"172.21.15.49:6823","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6817","nonce":38116},{"type":"v1","addr":"172.21.15.49:6819","nonce":38116}]},"public_addr":"172.21.15.49:6811/38116","cluster_addr":"172.21.15.49:6815/38116","heartbeat_back_addr":"172.21.15.49:6823/38116","heartbeat_front_addr":"172.21.15.49:6819/38116","state":["exists","up"]},{"osd":1,"uuid":"c8b31f85-a757-43ed-9419-351acc46611b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":38113},{"type":"v1","addr":"172.21.15.49:6810","nonce":38113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":38113},{"type":"v1","addr":"172.21.15.49:6814","nonce":38113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":38113},{"type":"v1","addr":"172.21.15.49:6822","nonce":38113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":38113},{"type":"v1","addr":"172.21.15.49:6818","nonce":38113}]},"public_addr":"172.21.15.49:6810/38113","cluster_addr":"172.21.15.49:6814/38113","heartbeat_back_addr":"172.21.15.49:6822/38113","heartbeat_front_addr":"172.21.15.49:6818/38113","state":["exists","up"]},{"osd":2,"uuid":"555b3bf3-0a70-4b91-8efd-e4204b5fc26e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":38114},{"type":"v1","addr":"172.21.15.49:6801","nonce":38114}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":38114},{"type":"v1","addr":"172.21.15.49:6803","nonce":38114}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":38114},{"type":"v1","addr":"172.21.15.49:6807","nonce":38114}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":38114},{"type":"v1","addr":"172.21.15.49:6805","nonce":38114}]},"public_addr":"172.21.15.49:6801/38114","cluster_addr":"172.21.15.49:6803/38114","heartbeat_back_addr":"172.21.15.49:6807/38114","heartbeat_front_addr":"172.21.15.49:6805/38114","state":["exists","up"]},{"osd":3,"uuid":"8d4e5d77-03ef-4b9b-bfcb-5ae93a46795c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":38115},{"type":"v1","addr":"172.21.15.49:6825","nonce":38115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":38115},{"type":"v1","addr":"172.21.15.49:6827","nonce":38115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":38115},{"type":"v1","addr":"172.21.15.49:6831","nonce":38115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":38115},{"type":"v1","addr":"172.21.15.49:6829","nonce":38115}]},"public_addr":"172.21.15.49:6825/38115","cluster_addr":"172.21.15.49:6827/38115","heartbeat_back_addr":"172.21.15.49:6831/38115","heartbeat_front_addr":"172.21.15.49:6829/38115","state":["exists","up"]},{"osd":4,"uuid":"dba03ad4-68a3-4cf3-9218-ac611f02eacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":38417},{"type":"v1","addr":"172.21.15.150:6809","nonce":38417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":38417},{"type":"v1","addr":"172.21.15.150:6811","nonce":38417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":38417},{"type":"v1","addr":"172.21.15.150:6815","nonce":38417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":38417},{"type":"v1","addr":"172.21.15.150:6813","nonce":38417}]},"public_addr":"172.21.15.150:6809/38417","cluster_addr":"172.21.15.150:6811/38417","heartbeat_back_addr":"172.21.15.150:6815/38417","heartbeat_front_addr":"172.21.15.150:6813/38417","state":["exists","up"]},{"osd":5,"uuid":"0df9c303-c255-4640-8e91-01f1c3efe18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":38418},{"type":"v1","addr":"172.21.15.150:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":38418},{"type":"v1","addr":"172.21.15.150:6803","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":38418},{"type":"v1","addr":"172.21.15.150:6807","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":38418},{"type":"v1","addr":"172.21.15.150:6805","nonce":38418}]},"public_addr":"172.21.15.150:6801/38418","cluster_addr":"172.21.15.150:6803/38418","heartbeat_back_addr":"172.21.15.150:6807/38418","heartbeat_front_addr":"172.21.15.150:6805/38418","state":["exists","up"]},{"osd":6,"uuid":"e59fa189-4b70-4c06-9722-c02b90828bf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":38428},{"type":"v1","addr":"172.21.15.150:6825","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":38428},{"type":"v1","addr":"172.21.15.150:6827","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":38428},{"type":"v1","addr":"172.21.15.150:6831","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":38428},{"type":"v1","addr":"172.21.15.150:6829","nonce":38428}]},"public_addr":"172.21.15.150:6825/38428","cluster_addr":"172.21.15.150:6827/38428","heartbeat_back_addr":"172.21.15.150:6831/38428","heartbeat_front_addr":"172.21.15.150:6829/38428","state":["exists","up"]},{"osd":7,"uuid":"a64f57e1-ef98-46bd-b304-385759e95898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":38421},{"type":"v1","addr":"172.21.15.150:6817","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":38421},{"type":"v1","addr":"172.21.15.150:6819","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":38421},{"type":"v1","addr":"172.21.15.150:6823","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":38421},{"type":"v1","addr":"172.21.15.150:6821","nonce":38421}]},"public_addr":"172.21.15.150:6817/38421","cluster_addr":"172.21.15.150:6819/38421","heartbeat_back_addr":"172.21.15.150:6823/38421","heartbeat_front_addr":"172.21.15.150:6821/38421","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.438552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.413280+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.296466+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.424966+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.276953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.230984+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.410235+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.480084+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.14","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T02:09:23.814 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.816+0000 7f6ba64c7640 1 -- 172.21.15.49:0/2923549082 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6b80065a60 msgr2=0x7f6b80067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:23.814 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.816+0000 7f6ba64c7640 1 --2- 172.21.15.49:0/2923549082 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6b80065a60 0x7f6b80067f20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6b90000fe0 tx=0x7f6b90009040 comp rx=0 tx=0).stop 2023-11-06T02:09:23.814 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.816+0000 7f6ba64c7640 1 -- 172.21.15.49:0/2923549082 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ba0159b40 msgr2=0x7f6ba01f6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:23.814 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.816+0000 7f6ba64c7640 1 --2- 172.21.15.49:0/2923549082 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ba0159b40 0x7f6ba01f6c40 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f6b9801a5d0 tx=0x7f6b98019740 comp rx=0 tx=0).stop 2023-11-06T02:09:23.815 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.816+0000 7f6ba64c7640 1 -- 172.21.15.49:0/2923549082 shutdown_connections 2023-11-06T02:09:23.815 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.816+0000 7f6ba64c7640 1 --2- 172.21.15.49:0/2923549082 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6b80065a60 0x7f6b80067f20 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:23.815 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.816+0000 7f6ba64c7640 1 --2- 172.21.15.49:0/2923549082 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ba0159b40 0x7f6ba01f6c40 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:23.815 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.816+0000 7f6ba64c7640 1 --2- 172.21.15.49:0/2923549082 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ba00cdd50 0x7f6ba01f6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:23.815 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.816+0000 7f6ba64c7640 1 --2- 172.21.15.49:0/2923549082 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ba00cb010 0x7f6ba015bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:23.815 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.816+0000 7f6ba64c7640 1 -- 172.21.15.49:0/2923549082 >> 172.21.15.49:0/2923549082 conn(0x7f6ba006c0a0 msgr2=0x7f6ba006ffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:23.816 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.816+0000 7f6ba64c7640 1 -- 172.21.15.49:0/2923549082 shutdown_connections 2023-11-06T02:09:23.816 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.816+0000 7f6ba64c7640 1 -- 172.21.15.49:0/2923549082 wait complete. 2023-11-06T02:09:23.831 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T02:08:19.252222+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T02:08:24.776211+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T02:08:28.936178+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T02:08:29.563468+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T02:09:23.832 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T02:09:23.957 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.956+0000 7fda3cbd1640 1 Processor -- start 2023-11-06T02:09:23.957 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.956+0000 7fda3cbd1640 1 -- start start 2023-11-06T02:09:23.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.960+0000 7fda3cbd1640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fda3815a990 0x7fda3815dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:23.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.960+0000 7fda3cbd1640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fda380b0220 0x7fda380b0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:23.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.960+0000 7fda3cbd1640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda3806c5a0 con 0x7fda381595a0 2023-11-06T02:09:23.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.960+0000 7fda3cbd1640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda3815e250 con 0x7fda3815a990 2023-11-06T02:09:23.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.960+0000 7fda3cbd1640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda3806c880 con 0x7fda380b0220 2023-11-06T02:09:23.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.960+0000 7fda35d9b640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fda3815a990 0x7fda3815dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:23.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.960+0000 7fda35d9b640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fda3815a990 0x7fda3815dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41932/0 (socket says 172.21.15.49:41932) 2023-11-06T02:09:23.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.960+0000 7fda35d9b640 1 -- 172.21.15.49:0/2388907132 learned_addr learned my addr 172.21.15.49:0/2388907132 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:23.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.960+0000 7fda36d9d640 1 --2- 172.21.15.49:0/2388907132 >> v2:172.21.15.150:3300/0 conn(0x7fda380b0220 0x7fda380b0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:23.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.960+0000 7fda35d9b640 1 -- 172.21.15.49:0/2388907132 >> v2:172.21.15.150:3300/0 conn(0x7fda380b0220 msgr2=0x7fda380b0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:23.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.960+0000 7fda35d9b640 1 --2- 172.21.15.49:0/2388907132 >> v2:172.21.15.150:3300/0 conn(0x7fda380b0220 0x7fda380b0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:23.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.960+0000 7fda35d9b640 1 -- 172.21.15.49:0/2388907132 >> v1:172.21.15.49:6789/0 conn(0x7fda381595a0 legacy=0x7fda3806d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:23.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.960+0000 7fda35d9b640 1 -- 172.21.15.49:0/2388907132 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda3806cb60 con 0x7fda3815a990 2023-11-06T02:09:23.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.964+0000 7fda35d9b640 1 --2- 172.21.15.49:0/2388907132 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda3815a990 0x7fda3815dd10 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fda380afa80 tx=0x7fda2c01eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4f7434056e638f6 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:23.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.964+0000 7fda3559a640 1 -- 172.21.15.49:0/2388907132 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2c0a7c70 con 0x7fda3815a990 2023-11-06T02:09:23.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.964+0000 7fda3559a640 1 -- 172.21.15.49:0/2388907132 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda2c0a7dd0 con 0x7fda3815a990 2023-11-06T02:09:23.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.964+0000 7fda3559a640 1 -- 172.21.15.49:0/2388907132 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2c0af660 con 0x7fda3815a990 2023-11-06T02:09:23.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.964+0000 7fda3cbd1640 1 -- 172.21.15.49:0/2388907132 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda3815a990 msgr2=0x7fda3815dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:23.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.964+0000 7fda3cbd1640 1 --2- 172.21.15.49:0/2388907132 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda3815a990 0x7fda3815dd10 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fda380afa80 tx=0x7fda2c01eba0 comp rx=0 tx=0).stop 2023-11-06T02:09:23.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.964+0000 7fda3cbd1640 1 -- 172.21.15.49:0/2388907132 shutdown_connections 2023-11-06T02:09:23.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.964+0000 7fda3cbd1640 1 --2- 172.21.15.49:0/2388907132 >> v2:172.21.15.150:3300/0 conn(0x7fda380b0220 0x7fda380b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:23.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.964+0000 7fda3cbd1640 1 --2- 172.21.15.49:0/2388907132 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda3815a990 0x7fda3815dd10 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:23.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.964+0000 7fda3cbd1640 1 -- 172.21.15.49:0/2388907132 >> 172.21.15.49:0/2388907132 conn(0x7fda3806a370 msgr2=0x7fda380ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:23.964 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.964+0000 7fda3cbd1640 1 -- 172.21.15.49:0/2388907132 shutdown_connections 2023-11-06T02:09:23.964 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.964+0000 7fda3cbd1640 1 -- 172.21.15.49:0/2388907132 wait complete. 2023-11-06T02:09:23.966 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.964+0000 7fda3cbd1640 1 Processor -- start 2023-11-06T02:09:23.966 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.964+0000 7fda3cbd1640 1 -- start start 2023-11-06T02:09:23.966 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.964+0000 7fda3cbd1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda380b0220 0x7fda38147260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:23.966 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.964+0000 7fda3cbd1640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fda381595a0 0x7fda38157550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:23.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.964+0000 7fda3cbd1640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fda3815a990 0x7fda381f85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:23.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.964+0000 7fda3cbd1640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda3814b370 con 0x7fda380b0220 2023-11-06T02:09:23.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.964+0000 7fda3cbd1640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda3814b610 con 0x7fda381595a0 2023-11-06T02:09:23.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.964+0000 7fda3cbd1640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda3814b8f0 con 0x7fda3815a990 2023-11-06T02:09:23.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.964+0000 7fda3659c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda380b0220 0x7fda38147260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:23.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.964+0000 7fda3659c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda380b0220 0x7fda38147260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41944/0 (socket says 172.21.15.49:41944) 2023-11-06T02:09:23.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.964+0000 7fda3659c640 1 -- 172.21.15.49:0/1364169322 learned_addr learned my addr 172.21.15.49:0/1364169322 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:23.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.964+0000 7fda36d9d640 1 --2- 172.21.15.49:0/1364169322 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fda3815a990 0x7fda381f85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:23.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.964+0000 7fda35d9b640 1 --2- 172.21.15.49:0/1364169322 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fda381595a0 0x7fda38157550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:23.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.964+0000 7fda3659c640 1 -- 172.21.15.49:0/1364169322 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fda3815a990 msgr2=0x7fda381f85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:23.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.964+0000 7fda3659c640 1 --2- 172.21.15.49:0/1364169322 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fda3815a990 0x7fda381f85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:23.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.964+0000 7fda3659c640 1 -- 172.21.15.49:0/1364169322 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fda381595a0 msgr2=0x7fda38157550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:23.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.964+0000 7fda3659c640 1 --2- 172.21.15.49:0/1364169322 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fda381595a0 0x7fda38157550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:23.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.964+0000 7fda3659c640 1 -- 172.21.15.49:0/1364169322 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda2c00b770 con 0x7fda380b0220 2023-11-06T02:09:23.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.964+0000 7fda35d9b640 1 --2- 172.21.15.49:0/1364169322 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fda381595a0 0x7fda38157550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:09:23.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.964+0000 7fda36d9d640 1 --2- 172.21.15.49:0/1364169322 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fda3815a990 0x7fda381f85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:09:23.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.964+0000 7fda3659c640 1 --2- 172.21.15.49:0/1364169322 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda380b0220 0x7fda38147260 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fda2800c500 tx=0x7fda2800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:23.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.964+0000 7fda26ffd640 1 -- 172.21.15.49:0/1364169322 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda28005220 con 0x7fda380b0220 2023-11-06T02:09:23.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.964+0000 7fda26ffd640 1 -- 172.21.15.49:0/1364169322 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda28005380 con 0x7fda380b0220 2023-11-06T02:09:23.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.964+0000 7fda3cbd1640 1 -- 172.21.15.49:0/1364169322 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda3814bbd0 con 0x7fda380b0220 2023-11-06T02:09:23.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.968+0000 7fda26ffd640 1 -- 172.21.15.49:0/1364169322 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda28007270 con 0x7fda380b0220 2023-11-06T02:09:23.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.968+0000 7fda3cbd1640 1 -- 172.21.15.49:0/1364169322 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda381477a0 con 0x7fda380b0220 2023-11-06T02:09:23.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.968+0000 7fda3cbd1640 1 -- 172.21.15.49:0/1364169322 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda3806c210 con 0x7fda380b0220 2023-11-06T02:09:23.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.968+0000 7fda26ffd640 1 -- 172.21.15.49:0/1364169322 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fda280073d0 con 0x7fda380b0220 2023-11-06T02:09:23.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.968+0000 7fda26ffd640 1 --2- 172.21.15.49:0/1364169322 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fda18065a60 0x7fda18067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:23.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.968+0000 7fda26ffd640 1 -- 172.21.15.49:0/1364169322 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fda28082cb0 con 0x7fda380b0220 2023-11-06T02:09:23.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.968+0000 7fda35d9b640 1 --2- 172.21.15.49:0/1364169322 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fda18065a60 0x7fda18067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:23.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.968+0000 7fda35d9b640 1 --2- 172.21.15.49:0/1364169322 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fda18065a60 0x7fda18067f20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fda2c0b1000 tx=0x7fda2c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:23.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:23.972+0000 7fda26ffd640 1 -- 172.21.15.49:0/1364169322 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fda3806c210 con 0x7fda380b0220 2023-11-06T02:09:24.120 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.120+0000 7fda3cbd1640 1 -- 172.21.15.49:0/1364169322 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fda3806c210 con 0x7fda380b0220 2023-11-06T02:09:24.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.124+0000 7fda26ffd640 1 -- 172.21.15.49:0/1364169322 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7fda3806c210 con 0x7fda380b0220 2023-11-06T02:09:24.121 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2023-11-06T02:09:24.123 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.124+0000 7fda3cbd1640 1 -- 172.21.15.49:0/1364169322 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fda18065a60 msgr2=0x7fda18067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:24.124 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.124+0000 7fda3cbd1640 1 --2- 172.21.15.49:0/1364169322 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fda18065a60 0x7fda18067f20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fda2c0b1000 tx=0x7fda2c008040 comp rx=0 tx=0).stop 2023-11-06T02:09:24.124 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.124+0000 7fda3cbd1640 1 -- 172.21.15.49:0/1364169322 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda380b0220 msgr2=0x7fda38147260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:24.124 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.124+0000 7fda3cbd1640 1 --2- 172.21.15.49:0/1364169322 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda380b0220 0x7fda38147260 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fda2800c500 tx=0x7fda2800c9c0 comp rx=0 tx=0).stop 2023-11-06T02:09:24.124 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.124+0000 7fda3cbd1640 1 -- 172.21.15.49:0/1364169322 shutdown_connections 2023-11-06T02:09:24.124 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.124+0000 7fda3cbd1640 1 --2- 172.21.15.49:0/1364169322 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fda18065a60 0x7fda18067f20 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:24.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.124+0000 7fda3cbd1640 1 --2- 172.21.15.49:0/1364169322 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fda3815a990 0x7fda381f85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:24.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.124+0000 7fda3cbd1640 1 --2- 172.21.15.49:0/1364169322 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fda381595a0 0x7fda38157550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:24.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.124+0000 7fda3cbd1640 1 --2- 172.21.15.49:0/1364169322 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda380b0220 0x7fda38147260 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:24.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.124+0000 7fda3cbd1640 1 -- 172.21.15.49:0/1364169322 >> 172.21.15.49:0/1364169322 conn(0x7fda3806a370 msgr2=0x7fda380ca490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:24.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.124+0000 7fda3cbd1640 1 -- 172.21.15.49:0/1364169322 shutdown_connections 2023-11-06T02:09:24.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.124+0000 7fda3cbd1640 1 -- 172.21.15.49:0/1364169322 wait complete. 2023-11-06T02:09:24.139 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T02:09:24.254 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.256+0000 7f824cf6e640 1 Processor -- start 2023-11-06T02:09:24.254 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.256+0000 7f824cf6e640 1 -- start start 2023-11-06T02:09:24.255 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.256+0000 7f824cf6e640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f82481595a0 0x7f824815dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:24.256 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.256+0000 7f824cf6e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f824806d640 0x7f82480b0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:24.256 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.256+0000 7f824cf6e640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f824806c5a0 con 0x7f82480b0760 2023-11-06T02:09:24.256 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.256+0000 7f824cf6e640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f824815e250 con 0x7f82481595a0 2023-11-06T02:09:24.256 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.256+0000 7f824cf6e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f824806c880 con 0x7f824806d640 2023-11-06T02:09:24.256 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.256+0000 7f824659c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f82481595a0 0x7f824815dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:24.257 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.256+0000 7f824659c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f82481595a0 0x7f824815dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41958/0 (socket says 172.21.15.49:41958) 2023-11-06T02:09:24.257 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.256+0000 7f824659c640 1 -- 172.21.15.49:0/2303545622 learned_addr learned my addr 172.21.15.49:0/2303545622 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:24.257 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.256+0000 7f823dd9b640 1 --2- 172.21.15.49:0/2303545622 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f824806d640 0x7f82480b0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:24.257 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.256+0000 7f824659c640 1 -- 172.21.15.49:0/2303545622 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f824806d640 msgr2=0x7f82480b0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:24.258 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.256+0000 7f824659c640 1 --2- 172.21.15.49:0/2303545622 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f824806d640 0x7f82480b0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:24.258 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.256+0000 7f824659c640 1 -- 172.21.15.49:0/2303545622 >> v1:172.21.15.150:6789/0 conn(0x7f82480b0760 legacy=0x7f82480c8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:24.258 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.256+0000 7f824659c640 1 -- 172.21.15.49:0/2303545622 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f824806cb60 con 0x7f82481595a0 2023-11-06T02:09:24.260 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.260+0000 7f824659c640 1 --2- 172.21.15.49:0/2303545622 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82481595a0 0x7f824815dd10 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f8238009870 tx=0x7f823801c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf5edd33fca0f23b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:24.260 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.260+0000 7f8245d9b640 1 -- 172.21.15.49:0/2303545622 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82380aa670 con 0x7f82481595a0 2023-11-06T02:09:24.260 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.260+0000 7f8245d9b640 1 -- 172.21.15.49:0/2303545622 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82380aa7d0 con 0x7f82481595a0 2023-11-06T02:09:24.260 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.260+0000 7f8245d9b640 1 -- 172.21.15.49:0/2303545622 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82380aab60 con 0x7f82481595a0 2023-11-06T02:09:24.260 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.260+0000 7f824cf6e640 1 -- 172.21.15.49:0/2303545622 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82481595a0 msgr2=0x7f824815dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:24.261 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.260+0000 7f824cf6e640 1 --2- 172.21.15.49:0/2303545622 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82481595a0 0x7f824815dd10 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f8238009870 tx=0x7f823801c510 comp rx=0 tx=0).stop 2023-11-06T02:09:24.261 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.260+0000 7f824cf6e640 1 -- 172.21.15.49:0/2303545622 shutdown_connections 2023-11-06T02:09:24.261 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.260+0000 7f824cf6e640 1 --2- 172.21.15.49:0/2303545622 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f824806d640 0x7f82480b0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:24.261 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.260+0000 7f824cf6e640 1 --2- 172.21.15.49:0/2303545622 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82481595a0 0x7f824815dd10 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:24.261 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.260+0000 7f824cf6e640 1 -- 172.21.15.49:0/2303545622 >> 172.21.15.49:0/2303545622 conn(0x7f824806a370 msgr2=0x7f82480ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:24.262 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.260+0000 7f824cf6e640 1 -- 172.21.15.49:0/2303545622 shutdown_connections 2023-11-06T02:09:24.262 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.260+0000 7f824cf6e640 1 -- 172.21.15.49:0/2303545622 wait complete. 2023-11-06T02:09:24.262 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.260+0000 7f824cf6e640 1 Processor -- start 2023-11-06T02:09:24.262 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.260+0000 7f824cf6e640 1 -- start start 2023-11-06T02:09:24.265 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.260+0000 7f824cf6e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f824806d640 0x7f82480b18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:24.265 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.260+0000 7f824cf6e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82480b0760 0x7f824814eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:24.265 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.260+0000 7f824cf6e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f82481595a0 0x7f8248148270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:24.265 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.260+0000 7f824cf6e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82481492b0 con 0x7f82480b0760 2023-11-06T02:09:24.266 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.260+0000 7f824cf6e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f824814f410 con 0x7f824806d640 2023-11-06T02:09:24.266 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.260+0000 7f824cf6e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82481487b0 con 0x7f82481595a0 2023-11-06T02:09:24.266 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.260+0000 7f823dd9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82480b0760 0x7f824814eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:24.266 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.260+0000 7f823dd9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82480b0760 0x7f824814eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41964/0 (socket says 172.21.15.49:41964) 2023-11-06T02:09:24.267 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.260+0000 7f823dd9b640 1 -- 172.21.15.49:0/791640406 learned_addr learned my addr 172.21.15.49:0/791640406 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:24.267 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.260+0000 7f824659c640 1 --2- 172.21.15.49:0/791640406 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f824806d640 0x7f82480b18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:24.267 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.260+0000 7f823dd9b640 1 -- 172.21.15.49:0/791640406 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f82481595a0 msgr2=0x7f8248148270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:24.267 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.260+0000 7f823dd9b640 1 --2- 172.21.15.49:0/791640406 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f82481595a0 0x7f8248148270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:24.267 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.260+0000 7f823dd9b640 1 -- 172.21.15.49:0/791640406 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f824806d640 msgr2=0x7f82480b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:24.268 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.260+0000 7f823dd9b640 1 --2- 172.21.15.49:0/791640406 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f824806d640 0x7f82480b18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:24.268 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.260+0000 7f823dd9b640 1 -- 172.21.15.49:0/791640406 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82380094d0 con 0x7f82480b0760 2023-11-06T02:09:24.268 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.264+0000 7f823dd9b640 1 --2- 172.21.15.49:0/791640406 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82480b0760 0x7f824814eed0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f823000d5f0 tx=0x7f823000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:24.268 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.264+0000 7f823f7fe640 1 -- 172.21.15.49:0/791640406 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82300125f0 con 0x7f82480b0760 2023-11-06T02:09:24.268 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.264+0000 7f823f7fe640 1 -- 172.21.15.49:0/791640406 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8230012750 con 0x7f82480b0760 2023-11-06T02:09:24.269 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.264+0000 7f824cf6e640 1 -- 172.21.15.49:0/791640406 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8248148a10 con 0x7f82480b0760 2023-11-06T02:09:24.269 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.264+0000 7f823f7fe640 1 -- 172.21.15.49:0/791640406 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8230012ae0 con 0x7f82480b0760 2023-11-06T02:09:24.269 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.264+0000 7f824cf6e640 1 -- 172.21.15.49:0/791640406 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8248148fd0 con 0x7f82480b0760 2023-11-06T02:09:24.269 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.264+0000 7f824cf6e640 1 -- 172.21.15.49:0/791640406 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f824806c210 con 0x7f82480b0760 2023-11-06T02:09:24.269 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.264+0000 7f823f7fe640 1 -- 172.21.15.49:0/791640406 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f8230012c40 con 0x7f82480b0760 2023-11-06T02:09:24.270 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.264+0000 7f823f7fe640 1 --2- 172.21.15.49:0/791640406 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8228065a60 0x7f8228067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:24.270 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.264+0000 7f823f7fe640 1 -- 172.21.15.49:0/791640406 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f8230083720 con 0x7f82480b0760 2023-11-06T02:09:24.270 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.264+0000 7f824659c640 1 --2- 172.21.15.49:0/791640406 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8228065a60 0x7f8228067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:24.270 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.264+0000 7f824659c640 1 --2- 172.21.15.49:0/791640406 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8228065a60 0x7f8228067f20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f82380ae000 tx=0x7f8238008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:24.271 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.268+0000 7f823f7fe640 1 -- 172.21.15.49:0/791640406 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f824806c210 con 0x7f82480b0760 2023-11-06T02:09:24.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.416+0000 7f824cf6e640 1 -- 172.21.15.49:0/791640406 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f824806c210 con 0x7f82480b0760 2023-11-06T02:09:24.417 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.416+0000 7f823f7fe640 1 -- 172.21.15.49:0/791640406 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7f824806c210 con 0x7f82480b0760 2023-11-06T02:09:24.417 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 8 2023-11-06T02:09:24.419 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.420+0000 7f824cf6e640 1 -- 172.21.15.49:0/791640406 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8228065a60 msgr2=0x7f8228067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:24.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.420+0000 7f824cf6e640 1 --2- 172.21.15.49:0/791640406 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8228065a60 0x7f8228067f20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f82380ae000 tx=0x7f8238008040 comp rx=0 tx=0).stop 2023-11-06T02:09:24.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.420+0000 7f824cf6e640 1 -- 172.21.15.49:0/791640406 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82480b0760 msgr2=0x7f824814eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:24.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.420+0000 7f824cf6e640 1 --2- 172.21.15.49:0/791640406 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82480b0760 0x7f824814eed0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f823000d5f0 tx=0x7f823000dab0 comp rx=0 tx=0).stop 2023-11-06T02:09:24.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.420+0000 7f824cf6e640 1 -- 172.21.15.49:0/791640406 shutdown_connections 2023-11-06T02:09:24.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.420+0000 7f824cf6e640 1 --2- 172.21.15.49:0/791640406 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8228065a60 0x7f8228067f20 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:24.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.420+0000 7f824cf6e640 1 --2- 172.21.15.49:0/791640406 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82480b0760 0x7f824814eed0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:24.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.420+0000 7f824cf6e640 1 --2- 172.21.15.49:0/791640406 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f82481595a0 0x7f8248148270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:24.422 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.420+0000 7f824cf6e640 1 --2- 172.21.15.49:0/791640406 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f824806d640 0x7f82480b18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:24.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.420+0000 7f824cf6e640 1 -- 172.21.15.49:0/791640406 >> 172.21.15.49:0/791640406 conn(0x7f824806a370 msgr2=0x7f82480c8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:24.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.420+0000 7f824cf6e640 1 -- 172.21.15.49:0/791640406 shutdown_connections 2023-11-06T02:09:24.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.420+0000 7f824cf6e640 1 -- 172.21.15.49:0/791640406 wait complete. 2023-11-06T02:09:24.434 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T02:09:24.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.564+0000 7ff80acb2640 1 Processor -- start 2023-11-06T02:09:24.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.564+0000 7ff80acb2640 1 -- start start 2023-11-06T02:09:24.563 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.564+0000 7ff80acb2640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7ff80415cb10 0x7ff8040cdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:24.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.564+0000 7ff80acb2640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff80415cef0 0x7ff8040ce030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:24.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.564+0000 7ff80acb2640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8040cf900 con 0x7ff804069d70 2023-11-06T02:09:24.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.564+0000 7ff80acb2640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8040cfb70 con 0x7ff80415cb10 2023-11-06T02:09:24.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.564+0000 7ff80acb2640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8040cfe50 con 0x7ff80415cef0 2023-11-06T02:09:24.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.564+0000 7ff803fff640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff80415cef0 0x7ff8040ce030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:24.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.564+0000 7ff808a4e640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7ff80415cb10 0x7ff8040cdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:24.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.564+0000 7ff803fff640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff80415cef0 0x7ff8040ce030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.49:52668/0 (socket says 172.21.15.49:52668) 2023-11-06T02:09:24.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.564+0000 7ff803fff640 1 -- 172.21.15.49:0/2396939630 learned_addr learned my addr 172.21.15.49:0/2396939630 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:24.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.564+0000 7ff803fff640 1 -- 172.21.15.49:0/2396939630 >> v2:172.21.15.150:3300/0 conn(0x7ff80415cb10 msgr2=0x7ff8040cdaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:24.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.564+0000 7ff803fff640 1 --2- 172.21.15.49:0/2396939630 >> v2:172.21.15.150:3300/0 conn(0x7ff80415cb10 0x7ff8040cdaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:24.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.564+0000 7ff803fff640 1 -- 172.21.15.49:0/2396939630 >> v1:172.21.15.150:6789/0 conn(0x7ff804069d70 legacy=0x7ff804160850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:24.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.564+0000 7ff803fff640 1 -- 172.21.15.49:0/2396939630 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8040d0130 con 0x7ff80415cef0 2023-11-06T02:09:24.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.568+0000 7ff803fff640 1 --2- 172.21.15.49:0/2396939630 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff80415cef0 0x7ff8040ce030 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff7f8009870 tx=0x7ff7f801c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d3f852b35828333 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:24.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.568+0000 7ff8037fe640 1 -- 172.21.15.49:0/2396939630 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f80ad640 con 0x7ff80415cef0 2023-11-06T02:09:24.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.568+0000 7ff8037fe640 1 -- 172.21.15.49:0/2396939630 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7f80ad7a0 con 0x7ff80415cef0 2023-11-06T02:09:24.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.568+0000 7ff8037fe640 1 -- 172.21.15.49:0/2396939630 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f80adb30 con 0x7ff80415cef0 2023-11-06T02:09:24.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.568+0000 7ff80acb2640 1 -- 172.21.15.49:0/2396939630 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff80415cef0 msgr2=0x7ff8040ce030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:24.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.568+0000 7ff80acb2640 1 --2- 172.21.15.49:0/2396939630 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff80415cef0 0x7ff8040ce030 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff7f8009870 tx=0x7ff7f801c480 comp rx=0 tx=0).stop 2023-11-06T02:09:24.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.568+0000 7ff80acb2640 1 -- 172.21.15.49:0/2396939630 shutdown_connections 2023-11-06T02:09:24.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.568+0000 7ff80acb2640 1 --2- 172.21.15.49:0/2396939630 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff80415cef0 0x7ff8040ce030 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:24.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.568+0000 7ff80acb2640 1 --2- 172.21.15.49:0/2396939630 >> v2:172.21.15.150:3300/0 conn(0x7ff80415cb10 0x7ff8040cdaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:24.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.568+0000 7ff80acb2640 1 -- 172.21.15.49:0/2396939630 >> 172.21.15.49:0/2396939630 conn(0x7ff80406d4b0 msgr2=0x7ff80406d890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:24.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.568+0000 7ff80acb2640 1 -- 172.21.15.49:0/2396939630 shutdown_connections 2023-11-06T02:09:24.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.568+0000 7ff80acb2640 1 -- 172.21.15.49:0/2396939630 wait complete. 2023-11-06T02:09:24.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.568+0000 7ff80acb2640 1 Processor -- start 2023-11-06T02:09:24.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.568+0000 7ff80acb2640 1 -- start start 2023-11-06T02:09:24.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.572+0000 7ff80acb2640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff804069d70 0x7ff80414b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:24.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.572+0000 7ff80acb2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff80415cb10 0x7ff8040cc640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:24.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.572+0000 7ff80acb2640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff80415cef0 0x7ff80415a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:24.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.572+0000 7ff80acb2640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8040ccb80 con 0x7ff80415cb10 2023-11-06T02:09:24.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.572+0000 7ff80acb2640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff804151100 con 0x7ff80415cef0 2023-11-06T02:09:24.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.572+0000 7ff80acb2640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8041513a0 con 0x7ff804069d70 2023-11-06T02:09:24.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.572+0000 7ff803fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff80415cb10 0x7ff8040cc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:24.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.572+0000 7ff803fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff80415cb10 0x7ff8040cc640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41980/0 (socket says 172.21.15.49:41980) 2023-11-06T02:09:24.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.572+0000 7ff803fff640 1 -- 172.21.15.49:0/1829957811 learned_addr learned my addr 172.21.15.49:0/1829957811 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:24.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.572+0000 7ff808a4e640 1 --2- 172.21.15.49:0/1829957811 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff804069d70 0x7ff80414b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:24.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.572+0000 7ff80924f640 1 --2- 172.21.15.49:0/1829957811 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff80415cef0 0x7ff80415a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:24.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.572+0000 7ff803fff640 1 -- 172.21.15.49:0/1829957811 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff804069d70 msgr2=0x7ff80414b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:24.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.572+0000 7ff803fff640 1 --2- 172.21.15.49:0/1829957811 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff804069d70 0x7ff80414b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:24.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.572+0000 7ff803fff640 1 -- 172.21.15.49:0/1829957811 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff80415cef0 msgr2=0x7ff80415a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:24.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.572+0000 7ff803fff640 1 --2- 172.21.15.49:0/1829957811 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff80415cef0 0x7ff80415a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:24.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.572+0000 7ff803fff640 1 -- 172.21.15.49:0/1829957811 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7f80094d0 con 0x7ff80415cb10 2023-11-06T02:09:24.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.572+0000 7ff803fff640 1 --2- 172.21.15.49:0/1829957811 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff80415cb10 0x7ff8040cc640 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff7f8000c10 tx=0x7ff7f8019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:24.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.572+0000 7ff8017fa640 1 -- 172.21.15.49:0/1829957811 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f80bb070 con 0x7ff80415cb10 2023-11-06T02:09:24.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.572+0000 7ff8017fa640 1 -- 172.21.15.49:0/1829957811 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7f80a6050 con 0x7ff80415cb10 2023-11-06T02:09:24.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.572+0000 7ff80acb2640 1 -- 172.21.15.49:0/1829957811 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff804151680 con 0x7ff80415cb10 2023-11-06T02:09:24.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.572+0000 7ff8017fa640 1 -- 172.21.15.49:0/1829957811 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f80b6670 con 0x7ff80415cb10 2023-11-06T02:09:24.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.572+0000 7ff80acb2640 1 -- 172.21.15.49:0/1829957811 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff804151c40 con 0x7ff80415cb10 2023-11-06T02:09:24.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.572+0000 7ff80acb2640 1 -- 172.21.15.49:0/1829957811 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff804151e30 con 0x7ff80415cb10 2023-11-06T02:09:24.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.572+0000 7ff8017fa640 1 -- 172.21.15.49:0/1829957811 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7ff7f80b4040 con 0x7ff80415cb10 2023-11-06T02:09:24.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.572+0000 7ff8017fa640 1 --2- 172.21.15.49:0/1829957811 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff7f0065a60 0x7ff7f0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:24.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.572+0000 7ff8017fa640 1 -- 172.21.15.49:0/1829957811 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7ff7f811cbb0 con 0x7ff80415cb10 2023-11-06T02:09:24.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.572+0000 7ff808a4e640 1 --2- 172.21.15.49:0/1829957811 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff7f0065a60 0x7ff7f0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:24.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.576+0000 7ff808a4e640 1 --2- 172.21.15.49:0/1829957811 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff7f0065a60 0x7ff7f0067f20 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff7f40113e0 tx=0x7ff7f400b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:24.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.576+0000 7ff8017fa640 1 -- 172.21.15.49:0/1829957811 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7ff7f80ea2c0 con 0x7ff80415cb10 2023-11-06T02:09:24.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.728+0000 7ff80acb2640 1 -- 172.21.15.49:0/1829957811 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff804067140 con 0x7ff80415cb10 2023-11-06T02:09:24.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.728+0000 7ff8017fa640 1 -- 172.21.15.49:0/1829957811 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff7f80b2020 con 0x7ff80415cb10 2023-11-06T02:09:24.727 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2023-11-06T02:09:24.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.728+0000 7ff80acb2640 1 -- 172.21.15.49:0/1829957811 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff7f0065a60 msgr2=0x7ff7f0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:24.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.728+0000 7ff80acb2640 1 --2- 172.21.15.49:0/1829957811 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff7f0065a60 0x7ff7f0067f20 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff7f40113e0 tx=0x7ff7f400b040 comp rx=0 tx=0).stop 2023-11-06T02:09:24.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.728+0000 7ff80acb2640 1 -- 172.21.15.49:0/1829957811 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff80415cb10 msgr2=0x7ff8040cc640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:24.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.728+0000 7ff80acb2640 1 --2- 172.21.15.49:0/1829957811 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff80415cb10 0x7ff8040cc640 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff7f8000c10 tx=0x7ff7f8019220 comp rx=0 tx=0).stop 2023-11-06T02:09:24.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.728+0000 7ff80acb2640 1 -- 172.21.15.49:0/1829957811 shutdown_connections 2023-11-06T02:09:24.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.728+0000 7ff80acb2640 1 --2- 172.21.15.49:0/1829957811 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff7f0065a60 0x7ff7f0067f20 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:24.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.728+0000 7ff80acb2640 1 --2- 172.21.15.49:0/1829957811 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff80415cb10 0x7ff8040cc640 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:24.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.728+0000 7ff80acb2640 1 --2- 172.21.15.49:0/1829957811 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff80415cef0 0x7ff80415a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:24.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.728+0000 7ff80acb2640 1 --2- 172.21.15.49:0/1829957811 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff804069d70 0x7ff80414b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:24.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.728+0000 7ff80acb2640 1 -- 172.21.15.49:0/1829957811 >> 172.21.15.49:0/1829957811 conn(0x7ff80406d4b0 msgr2=0x7ff804158ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:24.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.728+0000 7ff80acb2640 1 -- 172.21.15.49:0/1829957811 shutdown_connections 2023-11-06T02:09:24.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.728+0000 7ff80acb2640 1 -- 172.21.15.49:0/1829957811 wait complete. 2023-11-06T02:09:24.744 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T02:09:24.871 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.872+0000 7f93ea354640 1 Processor -- start 2023-11-06T02:09:24.871 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.872+0000 7f93ea354640 1 -- start start 2023-11-06T02:09:24.873 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.872+0000 7f93ea354640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f93e41595a0 0x7f93e415dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:24.873 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.872+0000 7f93ea354640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f93e406d640 0x7f93e40b0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:24.873 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.872+0000 7f93ea354640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93e406c5a0 con 0x7f93e40b0760 2023-11-06T02:09:24.873 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.872+0000 7f93ea354640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93e415e250 con 0x7f93e406d640 2023-11-06T02:09:24.874 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.872+0000 7f93ea354640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93e406c880 con 0x7f93e41595a0 2023-11-06T02:09:24.874 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.872+0000 7f93e37fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f93e406d640 0x7f93e40b0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:24.874 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.872+0000 7f93e37fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f93e406d640 0x7f93e40b0220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41984/0 (socket says 172.21.15.49:41984) 2023-11-06T02:09:24.874 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.872+0000 7f93e37fe640 1 -- 172.21.15.49:0/3472966550 learned_addr learned my addr 172.21.15.49:0/3472966550 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:24.874 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.872+0000 7f93e3fff640 1 --2- 172.21.15.49:0/3472966550 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f93e41595a0 0x7f93e415dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:24.875 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.872+0000 7f93e37fe640 1 -- 172.21.15.49:0/3472966550 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f93e41595a0 msgr2=0x7f93e415dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:24.875 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.872+0000 7f93e37fe640 1 --2- 172.21.15.49:0/3472966550 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f93e41595a0 0x7f93e415dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:24.875 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.872+0000 7f93e37fe640 1 -- 172.21.15.49:0/3472966550 >> v1:172.21.15.49:6789/0 conn(0x7f93e40b0760 legacy=0x7f93e40c8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:24.875 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.872+0000 7f93e37fe640 1 -- 172.21.15.49:0/3472966550 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93e406cb60 con 0x7f93e406d640 2023-11-06T02:09:24.877 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.876+0000 7f93e37fe640 1 --2- 172.21.15.49:0/3472966550 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93e406d640 0x7f93e40b0220 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f93d8009870 tx=0x7f93d801c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be8fa7b19859601d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:24.877 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.876+0000 7f93e2ffd640 1 -- 172.21.15.49:0/3472966550 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d80aa670 con 0x7f93e406d640 2023-11-06T02:09:24.877 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.876+0000 7f93e2ffd640 1 -- 172.21.15.49:0/3472966550 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93d80aa7d0 con 0x7f93e406d640 2023-11-06T02:09:24.878 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.876+0000 7f93e2ffd640 1 -- 172.21.15.49:0/3472966550 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d80aab60 con 0x7f93e406d640 2023-11-06T02:09:24.878 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.876+0000 7f93ea354640 1 -- 172.21.15.49:0/3472966550 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93e406d640 msgr2=0x7f93e40b0220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:24.878 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.876+0000 7f93ea354640 1 --2- 172.21.15.49:0/3472966550 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93e406d640 0x7f93e40b0220 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f93d8009870 tx=0x7f93d801c510 comp rx=0 tx=0).stop 2023-11-06T02:09:24.878 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.876+0000 7f93ea354640 1 -- 172.21.15.49:0/3472966550 shutdown_connections 2023-11-06T02:09:24.879 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.876+0000 7f93ea354640 1 --2- 172.21.15.49:0/3472966550 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93e406d640 0x7f93e40b0220 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:24.879 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.876+0000 7f93ea354640 1 --2- 172.21.15.49:0/3472966550 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f93e41595a0 0x7f93e415dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:24.879 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.876+0000 7f93ea354640 1 -- 172.21.15.49:0/3472966550 >> 172.21.15.49:0/3472966550 conn(0x7f93e406a370 msgr2=0x7f93e40ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:24.879 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.876+0000 7f93ea354640 1 -- 172.21.15.49:0/3472966550 shutdown_connections 2023-11-06T02:09:24.879 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.876+0000 7f93ea354640 1 -- 172.21.15.49:0/3472966550 wait complete. 2023-11-06T02:09:24.880 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.876+0000 7f93ea354640 1 Processor -- start 2023-11-06T02:09:24.880 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.876+0000 7f93ea354640 1 -- start start 2023-11-06T02:09:24.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.880+0000 7f93ea354640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93e406d640 0x7f93e40b1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:24.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.880+0000 7f93ea354640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f93e40b0760 0x7f93e4145f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:24.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.880+0000 7f93ea354640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93e41595a0 0x7f93e40cb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:24.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.880+0000 7f93ea354640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e41656a0 con 0x7f93e406d640 2023-11-06T02:09:24.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.880+0000 7f93ea354640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e40b21b0 con 0x7f93e41595a0 2023-11-06T02:09:24.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.880+0000 7f93ea354640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93e40b2490 con 0x7f93e40b0760 2023-11-06T02:09:24.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.880+0000 7f93e3fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93e406d640 0x7f93e40b1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:24.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.880+0000 7f93e3fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93e406d640 0x7f93e40b1980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41996/0 (socket says 172.21.15.49:41996) 2023-11-06T02:09:24.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.880+0000 7f93e3fff640 1 -- 172.21.15.49:0/1971516932 learned_addr learned my addr 172.21.15.49:0/1971516932 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:24.885 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.880+0000 7f93e37fe640 1 --2- 172.21.15.49:0/1971516932 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f93e40b0760 0x7f93e4145f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:24.885 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.880+0000 7f93e88f1640 1 --2- 172.21.15.49:0/1971516932 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93e41595a0 0x7f93e40cb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:24.885 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.880+0000 7f93e3fff640 1 -- 172.21.15.49:0/1971516932 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f93e40b0760 msgr2=0x7f93e4145f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:24.885 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.880+0000 7f93e3fff640 1 --2- 172.21.15.49:0/1971516932 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f93e40b0760 0x7f93e4145f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:24.886 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.880+0000 7f93e3fff640 1 -- 172.21.15.49:0/1971516932 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93e41595a0 msgr2=0x7f93e40cb5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:24.886 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.880+0000 7f93e3fff640 1 --2- 172.21.15.49:0/1971516932 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93e41595a0 0x7f93e40cb5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:24.886 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.880+0000 7f93e3fff640 1 -- 172.21.15.49:0/1971516932 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93d80094d0 con 0x7f93e406d640 2023-11-06T02:09:24.887 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.880+0000 7f93e3fff640 1 --2- 172.21.15.49:0/1971516932 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93e406d640 0x7f93e40b1980 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f93d400d5f0 tx=0x7f93d400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:24.887 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.880+0000 7f93e0ff9640 1 -- 172.21.15.49:0/1971516932 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d40125f0 con 0x7f93e406d640 2023-11-06T02:09:24.887 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.880+0000 7f93e0ff9640 1 -- 172.21.15.49:0/1971516932 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93d4012750 con 0x7f93e406d640 2023-11-06T02:09:24.888 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.880+0000 7f93e0ff9640 1 -- 172.21.15.49:0/1971516932 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d4012ae0 con 0x7f93e406d640 2023-11-06T02:09:24.888 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.880+0000 7f93ea354640 1 -- 172.21.15.49:0/1971516932 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93e41658f0 con 0x7f93e406d640 2023-11-06T02:09:24.888 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.880+0000 7f93ea354640 1 -- 172.21.15.49:0/1971516932 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93e40b1ec0 con 0x7f93e406d640 2023-11-06T02:09:24.889 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.880+0000 7f93ea354640 1 -- 172.21.15.49:0/1971516932 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93e406c210 con 0x7f93e406d640 2023-11-06T02:09:24.889 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.880+0000 7f93e0ff9640 1 -- 172.21.15.49:0/1971516932 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f93d4012c40 con 0x7f93e406d640 2023-11-06T02:09:24.890 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.880+0000 7f93e0ff9640 1 --2- 172.21.15.49:0/1971516932 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f93d0065a60 0x7f93d0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:24.890 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.880+0000 7f93e0ff9640 1 -- 172.21.15.49:0/1971516932 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f93d40838c0 con 0x7f93e406d640 2023-11-06T02:09:24.890 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.880+0000 7f93e37fe640 1 --2- 172.21.15.49:0/1971516932 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f93d0065a60 0x7f93d0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:24.891 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.884+0000 7f93e37fe640 1 --2- 172.21.15.49:0/1971516932 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f93d0065a60 0x7f93d0067f20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f93d80ae000 tx=0x7f93d80a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:24.891 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:24.884+0000 7f93e0ff9640 1 -- 172.21.15.49:0/1971516932 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f93e406c210 con 0x7f93e406d640 2023-11-06T02:09:25.042 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.044+0000 7f93ea354640 1 -- 172.21.15.49:0/1971516932 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f93e406c210 con 0x7f93e406d640 2023-11-06T02:09:25.043 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.044+0000 7f93e0ff9640 1 -- 172.21.15.49:0/1971516932 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f93e406c210 con 0x7f93e406d640 2023-11-06T02:09:25.044 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2023-11-06T02:09:25.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.048+0000 7f93ea354640 1 -- 172.21.15.49:0/1971516932 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f93d0065a60 msgr2=0x7f93d0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:25.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.048+0000 7f93ea354640 1 --2- 172.21.15.49:0/1971516932 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f93d0065a60 0x7f93d0067f20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f93d80ae000 tx=0x7f93d80a7000 comp rx=0 tx=0).stop 2023-11-06T02:09:25.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.048+0000 7f93ea354640 1 -- 172.21.15.49:0/1971516932 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93e406d640 msgr2=0x7f93e40b1980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:25.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.048+0000 7f93ea354640 1 --2- 172.21.15.49:0/1971516932 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93e406d640 0x7f93e40b1980 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f93d400d5f0 tx=0x7f93d400dab0 comp rx=0 tx=0).stop 2023-11-06T02:09:25.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.048+0000 7f93ea354640 1 -- 172.21.15.49:0/1971516932 shutdown_connections 2023-11-06T02:09:25.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.048+0000 7f93ea354640 1 --2- 172.21.15.49:0/1971516932 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f93d0065a60 0x7f93d0067f20 secure :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f93d80ae000 tx=0x7f93d80a7000 comp rx=0 tx=0).stop 2023-11-06T02:09:25.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.048+0000 7f93ea354640 1 --2- 172.21.15.49:0/1971516932 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93e41595a0 0x7f93e40cb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:25.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.048+0000 7f93ea354640 1 --2- 172.21.15.49:0/1971516932 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f93e40b0760 0x7f93e4145f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:25.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.048+0000 7f93ea354640 1 --2- 172.21.15.49:0/1971516932 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93e406d640 0x7f93e40b1980 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:25.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.048+0000 7f93ea354640 1 -- 172.21.15.49:0/1971516932 >> 172.21.15.49:0/1971516932 conn(0x7f93e406a370 msgr2=0x7f93e4070340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:25.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.048+0000 7f93ea354640 1 -- 172.21.15.49:0/1971516932 shutdown_connections 2023-11-06T02:09:25.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.048+0000 7f93ea354640 1 -- 172.21.15.49:0/1971516932 wait complete. 2023-11-06T02:09:25.063 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-06T02:09:25.180 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.180+0000 7f2b8650e640 1 Processor -- start 2023-11-06T02:09:25.180 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.180+0000 7f2b8650e640 1 -- start start 2023-11-06T02:09:25.182 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.180+0000 7f2b8650e640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f2b80159980 0x7f2b80071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:25.182 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.180+0000 7f2b8650e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b800b1450 0x7f2b800cdd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:25.182 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.180+0000 7f2b8650e640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b80071890 con 0x7f2b800cb010 2023-11-06T02:09:25.183 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.184+0000 7f2b8650e640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b80072410 con 0x7f2b80159980 2023-11-06T02:09:25.183 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.184+0000 7f2b8650e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b800b1890 con 0x7f2b800b1450 2023-11-06T02:09:25.183 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.184+0000 7f2b7ffff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f2b80159980 0x7f2b80071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:25.183 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.184+0000 7f2b7ffff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f2b80159980 0x7f2b80071290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42010/0 (socket says 172.21.15.49:42010) 2023-11-06T02:09:25.183 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.184+0000 7f2b7ffff640 1 -- 172.21.15.49:0/3718093864 learned_addr learned my addr 172.21.15.49:0/3718093864 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:25.184 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.184+0000 7f2b7f7fe640 1 --2- 172.21.15.49:0/3718093864 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b800b1450 0x7f2b800cdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:25.184 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.184+0000 7f2b7ffff640 1 -- 172.21.15.49:0/3718093864 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b800b1450 msgr2=0x7f2b800cdd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:25.184 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.184+0000 7f2b7ffff640 1 --2- 172.21.15.49:0/3718093864 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b800b1450 0x7f2b800cdd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:25.184 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.184+0000 7f2b7ffff640 1 -- 172.21.15.49:0/3718093864 >> v1:172.21.15.150:6789/0 conn(0x7f2b800cb010 legacy=0x7f2b800cc540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:25.185 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.184+0000 7f2b7ffff640 1 -- 172.21.15.49:0/3718093864 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b80071b70 con 0x7f2b80159980 2023-11-06T02:09:25.185 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.184+0000 7f2b7f7fe640 1 --2- 172.21.15.49:0/3718093864 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b800b1450 0x7f2b800cdd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:09:25.188 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.184+0000 7f2b7ffff640 1 --2- 172.21.15.49:0/3718093864 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b80159980 0x7f2b80071290 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f2b70009ea0 tx=0x7f2b7001ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71a622865db28b33 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:25.188 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.184+0000 7f2b7effd640 1 -- 172.21.15.49:0/3718093864 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b700a5890 con 0x7f2b80159980 2023-11-06T02:09:25.188 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.184+0000 7f2b7effd640 1 -- 172.21.15.49:0/3718093864 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b700a59f0 con 0x7f2b80159980 2023-11-06T02:09:25.189 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.184+0000 7f2b7effd640 1 -- 172.21.15.49:0/3718093864 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b700a5d80 con 0x7f2b80159980 2023-11-06T02:09:25.189 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.184+0000 7f2b8650e640 1 -- 172.21.15.49:0/3718093864 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b80159980 msgr2=0x7f2b80071290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:25.189 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.184+0000 7f2b8650e640 1 --2- 172.21.15.49:0/3718093864 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b80159980 0x7f2b80071290 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f2b70009ea0 tx=0x7f2b7001ca50 comp rx=0 tx=0).stop 2023-11-06T02:09:25.189 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.184+0000 7f2b8650e640 1 -- 172.21.15.49:0/3718093864 shutdown_connections 2023-11-06T02:09:25.190 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.184+0000 7f2b8650e640 1 --2- 172.21.15.49:0/3718093864 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b800b1450 0x7f2b800cdd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:25.190 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.184+0000 7f2b8650e640 1 --2- 172.21.15.49:0/3718093864 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b80159980 0x7f2b80071290 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:25.190 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.184+0000 7f2b8650e640 1 -- 172.21.15.49:0/3718093864 >> 172.21.15.49:0/3718093864 conn(0x7f2b8006c0a0 msgr2=0x7f2b8006c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:25.190 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.184+0000 7f2b8650e640 1 -- 172.21.15.49:0/3718093864 shutdown_connections 2023-11-06T02:09:25.190 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.184+0000 7f2b8650e640 1 -- 172.21.15.49:0/3718093864 wait complete. 2023-11-06T02:09:25.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.188+0000 7f2b8650e640 1 Processor -- start 2023-11-06T02:09:25.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.188+0000 7f2b8650e640 1 -- start start 2023-11-06T02:09:25.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.188+0000 7f2b8650e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b800b1450 0x7f2b801484c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:25.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.188+0000 7f2b8650e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b800cb010 0x7f2b8014e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:25.192 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.188+0000 7f2b8650e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b80159980 0x7f2b80147dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:25.192 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.188+0000 7f2b8650e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b80148dd0 con 0x7f2b80159980 2023-11-06T02:09:25.192 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.188+0000 7f2b8650e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b8014eb70 con 0x7f2b800cb010 2023-11-06T02:09:25.192 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.188+0000 7f2b8650e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b80148310 con 0x7f2b800b1450 2023-11-06T02:09:25.192 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.188+0000 7f2b84aab640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b80159980 0x7f2b80147dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:25.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.188+0000 7f2b84aab640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b80159980 0x7f2b80147dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42014/0 (socket says 172.21.15.49:42014) 2023-11-06T02:09:25.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.188+0000 7f2b84aab640 1 -- 172.21.15.49:0/3704444135 learned_addr learned my addr 172.21.15.49:0/3704444135 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:25.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.188+0000 7f2b7ffff640 1 --2- 172.21.15.49:0/3704444135 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b800b1450 0x7f2b801484c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:25.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.188+0000 7f2b7f7fe640 1 --2- 172.21.15.49:0/3704444135 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b800cb010 0x7f2b8014e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:25.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.188+0000 7f2b84aab640 1 -- 172.21.15.49:0/3704444135 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b800b1450 msgr2=0x7f2b801484c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:25.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.188+0000 7f2b84aab640 1 --2- 172.21.15.49:0/3704444135 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b800b1450 0x7f2b801484c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:25.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.188+0000 7f2b84aab640 1 -- 172.21.15.49:0/3704444135 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b800cb010 msgr2=0x7f2b8014e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:25.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.188+0000 7f2b84aab640 1 --2- 172.21.15.49:0/3704444135 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b800cb010 0x7f2b8014e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:25.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.188+0000 7f2b84aab640 1 -- 172.21.15.49:0/3704444135 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b70009b30 con 0x7f2b80159980 2023-11-06T02:09:25.195 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.188+0000 7f2b7f7fe640 1 --2- 172.21.15.49:0/3704444135 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b800cb010 0x7f2b8014e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:09:25.195 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.188+0000 7f2b7ffff640 1 --2- 172.21.15.49:0/3704444135 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b800b1450 0x7f2b801484c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:09:25.195 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.188+0000 7f2b84aab640 1 --2- 172.21.15.49:0/3704444135 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b80159980 0x7f2b80147dd0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f2b7800c430 tx=0x7f2b7800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:25.195 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.188+0000 7f2b7cff9640 1 -- 172.21.15.49:0/3704444135 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b78005220 con 0x7f2b80159980 2023-11-06T02:09:25.196 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.188+0000 7f2b7cff9640 1 -- 172.21.15.49:0/3704444135 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b78005380 con 0x7f2b80159980 2023-11-06T02:09:25.196 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.188+0000 7f2b8650e640 1 -- 172.21.15.49:0/3704444135 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b800ce3f0 con 0x7f2b80159980 2023-11-06T02:09:25.196 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.188+0000 7f2b7cff9640 1 -- 172.21.15.49:0/3704444135 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b78007270 con 0x7f2b80159980 2023-11-06T02:09:25.196 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.188+0000 7f2b8650e640 1 -- 172.21.15.49:0/3704444135 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b8015b1b0 con 0x7f2b80159980 2023-11-06T02:09:25.197 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.188+0000 7f2b8650e640 1 -- 172.21.15.49:0/3704444135 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b8015b340 con 0x7f2b80159980 2023-11-06T02:09:25.198 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.192+0000 7f2b7cff9640 1 -- 172.21.15.49:0/3704444135 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f2b780073d0 con 0x7f2b80159980 2023-11-06T02:09:25.198 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.192+0000 7f2b7cff9640 1 --2- 172.21.15.49:0/3704444135 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2b6c065a60 0x7f2b6c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:25.198 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.192+0000 7f2b7cff9640 1 -- 172.21.15.49:0/3704444135 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f2b78082890 con 0x7f2b80159980 2023-11-06T02:09:25.198 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.192+0000 7f2b7ffff640 1 --2- 172.21.15.49:0/3704444135 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2b6c065a60 0x7f2b6c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:25.199 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.192+0000 7f2b7ffff640 1 --2- 172.21.15.49:0/3704444135 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2b6c065a60 0x7f2b6c067f20 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2b700ae000 tx=0x7f2b700a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:25.199 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.196+0000 7f2b7cff9640 1 -- 172.21.15.49:0/3704444135 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f2b78050140 con 0x7f2b80159980 2023-11-06T02:09:25.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.360+0000 7f2b8650e640 1 -- 172.21.15.49:0/3704444135 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f2b80074040 con 0x7f2b80159980 2023-11-06T02:09:25.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.360+0000 7f2b7cff9640 1 -- 172.21.15.49:0/3704444135 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7f2b78053760 con 0x7f2b80159980 2023-11-06T02:09:25.359 INFO:teuthology.orchestra.run.smithi049.stderr:listed 0 entries 2023-11-06T02:09:25.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.360+0000 7f2b8650e640 1 -- 172.21.15.49:0/3704444135 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2b6c065a60 msgr2=0x7f2b6c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:25.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.360+0000 7f2b8650e640 1 --2- 172.21.15.49:0/3704444135 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2b6c065a60 0x7f2b6c067f20 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2b700ae000 tx=0x7f2b700a7070 comp rx=0 tx=0).stop 2023-11-06T02:09:25.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.360+0000 7f2b8650e640 1 -- 172.21.15.49:0/3704444135 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b80159980 msgr2=0x7f2b80147dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:25.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.360+0000 7f2b8650e640 1 --2- 172.21.15.49:0/3704444135 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b80159980 0x7f2b80147dd0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f2b7800c430 tx=0x7f2b7800c8f0 comp rx=0 tx=0).stop 2023-11-06T02:09:25.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.364+0000 7f2b8650e640 1 -- 172.21.15.49:0/3704444135 shutdown_connections 2023-11-06T02:09:25.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.364+0000 7f2b8650e640 1 --2- 172.21.15.49:0/3704444135 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2b6c065a60 0x7f2b6c067f20 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:25.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.364+0000 7f2b8650e640 1 --2- 172.21.15.49:0/3704444135 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b80159980 0x7f2b80147dd0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:25.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.364+0000 7f2b8650e640 1 --2- 172.21.15.49:0/3704444135 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b800cb010 0x7f2b8014e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:25.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.364+0000 7f2b8650e640 1 --2- 172.21.15.49:0/3704444135 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b800b1450 0x7f2b801484c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:25.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.364+0000 7f2b8650e640 1 -- 172.21.15.49:0/3704444135 >> 172.21.15.49:0/3704444135 conn(0x7f2b8006c0a0 msgr2=0x7f2b800b19d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:25.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.364+0000 7f2b8650e640 1 -- 172.21.15.49:0/3704444135 shutdown_connections 2023-11-06T02:09:25.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.364+0000 7f2b8650e640 1 -- 172.21.15.49:0/3704444135 wait complete. 2023-11-06T02:09:25.377 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi150.front.sepia.ceph.com... 2023-11-06T02:09:25.377 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:09:25.378 DEBUG:teuthology.orchestra.run.smithi150:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2023-11-06T02:09:25.392 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T02:09:25.393 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi150.front.sepia.ceph.com 2023-11-06T02:09:25.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:09:25.394 DEBUG:teuthology.orchestra.run.smithi150:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-06T02:09:25.443 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-06T02:09:25.443 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T02:09:25.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.568+0000 7f2b71f3d640 1 Processor -- start 2023-11-06T02:09:25.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.568+0000 7f2b71f3d640 1 -- start start 2023-11-06T02:09:25.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.568+0000 7f2b71f3d640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b6c15ab50 0x7f2b6c0c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:25.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.568+0000 7f2b71f3d640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f2b6c0cb010 0x7f2b6c0b0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:25.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.568+0000 7f2b71f3d640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b6c0cab80 con 0x7f2b6c0b05c0 2023-11-06T02:09:25.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.568+0000 7f2b71f3d640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b6c0b0a50 con 0x7f2b6c0cb010 2023-11-06T02:09:25.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.568+0000 7f2b71f3d640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b6c15af30 con 0x7f2b6c15ab50 2023-11-06T02:09:25.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.568+0000 7f2b6bfff640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f2b6c0b05c0 0x7f2b6c0700a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:35530/0 (socket says 172.21.15.49:35530) 2023-11-06T02:09:25.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.568+0000 7f2b6bfff640 1 -- 172.21.15.49:0/4119573011 learned_addr learned my addr 172.21.15.49:0/4119573011 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:09:25.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.568+0000 7f2b6affd640 1 --2- 172.21.15.49:0/4119573011 >> v2:172.21.15.150:3300/0 conn(0x7f2b6c0cb010 0x7f2b6c0b0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:25.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.568+0000 7f2b6b7fe640 1 --2- 172.21.15.49:0/4119573011 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b6c15ab50 0x7f2b6c0c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:25.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.568+0000 7f2b6a7fc640 1 -- 172.21.15.49:0/4119573011 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1819888386 0 0) 0x7f2b6c0cab80 con 0x7f2b6c0b05c0 2023-11-06T02:09:25.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.568+0000 7f2b6a7fc640 1 -- 172.21.15.49:0/4119573011 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b54003530 con 0x7f2b6c0b05c0 2023-11-06T02:09:25.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.568+0000 7f2b6a7fc640 1 -- 172.21.15.49:0/4119573011 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2737010334 0 0) 0x7f2b54003530 con 0x7f2b6c0b05c0 2023-11-06T02:09:25.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.568+0000 7f2b6a7fc640 1 -- 172.21.15.49:0/4119573011 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b6c15ab50 msgr2=0x7f2b6c0c9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:25.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.568+0000 7f2b6a7fc640 1 --2- 172.21.15.49:0/4119573011 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b6c15ab50 0x7f2b6c0c9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:25.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.568+0000 7f2b6a7fc640 1 -- 172.21.15.49:0/4119573011 >> v2:172.21.15.150:3300/0 conn(0x7f2b6c0cb010 msgr2=0x7f2b6c0b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:25.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.568+0000 7f2b6a7fc640 1 --2- 172.21.15.49:0/4119573011 >> v2:172.21.15.150:3300/0 conn(0x7f2b6c0cb010 0x7f2b6c0b0080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:25.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.568+0000 7f2b6a7fc640 1 -- 172.21.15.49:0/4119573011 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b6c0cb5b0 con 0x7f2b6c0b05c0 2023-11-06T02:09:25.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.568+0000 7f2b6a7fc640 1 -- 172.21.15.49:0/4119573011 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7f2b64001a00 con 0x7f2b6c0b05c0 2023-11-06T02:09:25.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.568+0000 7f2b6a7fc640 1 -- 172.21.15.49:0/4119573011 >> v1:172.21.15.49:6789/0 conn(0x7f2b6c0b05c0 legacy=0x7f2b6c0700a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:25.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.568+0000 7f2b6a7fc640 1 --2- 172.21.15.49:0/4119573011 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b6c15ab50 0x7f2b54003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:25.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.568+0000 7f2b6a7fc640 1 --2- 172.21.15.49:0/4119573011 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b54003f00 0x7f2b540047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:25.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.568+0000 7f2b6a7fc640 1 --2- 172.21.15.49:0/4119573011 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b54004d10 0x7f2b54005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:25.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.568+0000 7f2b6a7fc640 1 -- 172.21.15.49:0/4119573011 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b6c0b0a50 con 0x7f2b54004d10 2023-11-06T02:09:25.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.568+0000 7f2b6a7fc640 1 -- 172.21.15.49:0/4119573011 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b6c15af30 con 0x7f2b54003f00 2023-11-06T02:09:25.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.568+0000 7f2b6a7fc640 1 -- 172.21.15.49:0/4119573011 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5400b990 con 0x7f2b6c15ab50 2023-11-06T02:09:25.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.568+0000 7f2b6bfff640 1 --2- 172.21.15.49:0/4119573011 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b54004d10 0x7f2b54005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:25.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.568+0000 7f2b6b7fe640 1 --2- 172.21.15.49:0/4119573011 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b6c15ab50 0x7f2b54003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:25.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.568+0000 7f2b6affd640 1 --2- 172.21.15.49:0/4119573011 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b54003f00 0x7f2b540047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:25.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.568+0000 7f2b6bfff640 1 -- 172.21.15.49:0/4119573011 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b6c15ab50 msgr2=0x7f2b54003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:25.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.568+0000 7f2b6bfff640 1 --2- 172.21.15.49:0/4119573011 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b6c15ab50 0x7f2b54003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:25.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.568+0000 7f2b6bfff640 1 -- 172.21.15.49:0/4119573011 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b54003f00 msgr2=0x7f2b540047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:25.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.568+0000 7f2b6bfff640 1 --2- 172.21.15.49:0/4119573011 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b54003f00 0x7f2b540047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:25.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.568+0000 7f2b6bfff640 1 -- 172.21.15.49:0/4119573011 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b54003530 con 0x7f2b54004d10 2023-11-06T02:09:25.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.568+0000 7f2b6b7fe640 1 --2- 172.21.15.49:0/4119573011 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b6c15ab50 0x7f2b54003980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:09:25.580 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.572+0000 7f2b6bfff640 1 --2- 172.21.15.49:0/4119573011 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b54004d10 0x7f2b54005190 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f2b6400a8d0 tx=0x7f2b6401d380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3975370b9736503 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:25.580 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.572+0000 7f2b6a7fc640 1 -- 172.21.15.49:0/4119573011 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b640ad630 con 0x7f2b54004d10 2023-11-06T02:09:25.581 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.572+0000 7f2b6a7fc640 1 -- 172.21.15.49:0/4119573011 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b640a8070 con 0x7f2b54004d10 2023-11-06T02:09:25.581 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.572+0000 7f2b6a7fc640 1 -- 172.21.15.49:0/4119573011 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b640ad9d0 con 0x7f2b54004d10 2023-11-06T02:09:25.581 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.572+0000 7f2b71f3d640 1 -- 172.21.15.49:0/4119573011 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b54004d10 msgr2=0x7f2b54005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:25.582 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.572+0000 7f2b71f3d640 1 --2- 172.21.15.49:0/4119573011 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b54004d10 0x7f2b54005190 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f2b6400a8d0 tx=0x7f2b6401d380 comp rx=0 tx=0).stop 2023-11-06T02:09:25.582 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.572+0000 7f2b6bfff640 1 -- 172.21.15.49:0/4119573011 reap_dead start 2023-11-06T02:09:25.582 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.572+0000 7f2b71f3d640 1 -- 172.21.15.49:0/4119573011 shutdown_connections 2023-11-06T02:09:25.583 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.572+0000 7f2b71f3d640 1 -- 172.21.15.49:0/4119573011 >> 172.21.15.49:0/4119573011 conn(0x7f2b6c06c0a0 msgr2=0x7f2b6c06c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:25.583 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.572+0000 7f2b71f3d640 1 -- 172.21.15.49:0/4119573011 shutdown_connections 2023-11-06T02:09:25.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.572+0000 7f2b71f3d640 1 -- 172.21.15.49:0/4119573011 wait complete. 2023-11-06T02:09:25.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.572+0000 7f2b71f3d640 1 Processor -- start 2023-11-06T02:09:25.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.572+0000 7f2b71f3d640 1 -- start start 2023-11-06T02:09:25.585 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.576+0000 7f2b71f3d640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b54004d10 0x7f2b6c14eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:25.585 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.576+0000 7f2b71f3d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b6c152ef0 0x7f2b6c150700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:25.585 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.576+0000 7f2b71f3d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b6c0c56d0 0x7f2b6c0af0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:25.586 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.576+0000 7f2b71f3d640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b6c15c980 con 0x7f2b6c152ef0 2023-11-06T02:09:25.586 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.576+0000 7f2b71f3d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b6c15c370 con 0x7f2b6c0c56d0 2023-11-06T02:09:25.587 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.576+0000 7f2b71f3d640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b6c150c40 con 0x7f2b54004d10 2023-11-06T02:09:25.587 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.576+0000 7f2b6affd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b6c152ef0 0x7f2b6c150700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:25.587 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.576+0000 7f2b6affd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b6c152ef0 0x7f2b6c150700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42040/0 (socket says 172.21.15.49:42040) 2023-11-06T02:09:25.588 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.576+0000 7f2b6affd640 1 -- 172.21.15.49:0/1560286331 learned_addr learned my addr 172.21.15.49:0/1560286331 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:25.588 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.576+0000 7f2b6b7fe640 1 --2- 172.21.15.49:0/1560286331 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b54004d10 0x7f2b6c14eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:25.588 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.576+0000 7f2b6bfff640 1 --2- 172.21.15.49:0/1560286331 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b6c0c56d0 0x7f2b6c0af0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:25.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.576+0000 7f2b6affd640 1 -- 172.21.15.49:0/1560286331 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b54004d10 msgr2=0x7f2b6c14eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:25.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.576+0000 7f2b6affd640 1 --2- 172.21.15.49:0/1560286331 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b54004d10 0x7f2b6c14eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:25.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.576+0000 7f2b6affd640 1 -- 172.21.15.49:0/1560286331 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b6c0c56d0 msgr2=0x7f2b6c0af0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:25.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.576+0000 7f2b6affd640 1 --2- 172.21.15.49:0/1560286331 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b6c0c56d0 0x7f2b6c0af0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:25.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.576+0000 7f2b6affd640 1 -- 172.21.15.49:0/1560286331 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b6c1533b0 con 0x7f2b6c152ef0 2023-11-06T02:09:25.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.576+0000 7f2b6affd640 1 --2- 172.21.15.49:0/1560286331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b6c152ef0 0x7f2b6c150700 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f2b60011590 tx=0x7f2b60011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:25.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.576+0000 7f2b53fff640 1 -- 172.21.15.49:0/1560286331 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b60015870 con 0x7f2b6c152ef0 2023-11-06T02:09:25.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.576+0000 7f2b53fff640 1 -- 172.21.15.49:0/1560286331 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b600159d0 con 0x7f2b6c152ef0 2023-11-06T02:09:25.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.576+0000 7f2b71f3d640 1 -- 172.21.15.49:0/1560286331 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b6c15c610 con 0x7f2b6c152ef0 2023-11-06T02:09:25.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.576+0000 7f2b53fff640 1 -- 172.21.15.49:0/1560286331 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b60015d40 con 0x7f2b6c152ef0 2023-11-06T02:09:25.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.576+0000 7f2b71f3d640 1 -- 172.21.15.49:0/1560286331 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b6c14f3c0 con 0x7f2b6c152ef0 2023-11-06T02:09:25.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.576+0000 7f2b71f3d640 1 -- 172.21.15.49:0/1560286331 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b6c0b0de0 con 0x7f2b6c152ef0 2023-11-06T02:09:25.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.576+0000 7f2b53fff640 1 -- 172.21.15.49:0/1560286331 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f2b60013400 con 0x7f2b6c152ef0 2023-11-06T02:09:25.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.576+0000 7f2b53fff640 1 --2- 172.21.15.49:0/1560286331 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2b58065a90 0x7f2b58067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:25.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.576+0000 7f2b53fff640 1 -- 172.21.15.49:0/1560286331 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f2b60083a30 con 0x7f2b6c152ef0 2023-11-06T02:09:25.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.576+0000 7f2b6b7fe640 1 --2- 172.21.15.49:0/1560286331 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2b58065a90 0x7f2b58067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:25.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.580+0000 7f2b6b7fe640 1 --2- 172.21.15.49:0/1560286331 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2b58065a90 0x7f2b58067f50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c009a90 tx=0x7f2b5c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:25.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.580+0000 7f2b53fff640 1 -- 172.21.15.49:0/1560286331 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f2b60085e60 con 0x7f2b6c152ef0 2023-11-06T02:09:25.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.772+0000 7f2b71f3d640 1 -- 172.21.15.49:0/1560286331 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2b6c074040 con 0x7f2b6c152ef0 2023-11-06T02:09:25.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.776+0000 7f2b53fff640 1 -- 172.21.15.49:0/1560286331 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4648 (secure 0 0 0) 0x7f2b60051390 con 0x7f2b6c152ef0 2023-11-06T02:09:25.774 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:09:25.775 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6834/1377586971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1377586971},{"type":"v1","addr":"172.21.15.49:6834","nonce":1377586971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4288,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2620584959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2620584959},{"type":"v1","addr":"172.21.15.150:6835","nonce":2620584959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4291,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/4272191418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":4272191418},{"type":"v1","addr":"172.21.15.150:6837","nonce":4272191418}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T02:08:30.592185+0000","modified":"2023-11-06T02:08:33.245446+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4293},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4293":{"gid":4293,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.49:6835/4280448297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6833","nonce":4280448297},{"type":"v1","addr":"172.21.15.49:6835","nonce":4280448297}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-06T02:09:25.775 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 7 2023-11-06T02:09:25.777 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.776+0000 7f2b71f3d640 1 -- 172.21.15.49:0/1560286331 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2b58065a90 msgr2=0x7f2b58067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:25.777 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.776+0000 7f2b71f3d640 1 --2- 172.21.15.49:0/1560286331 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2b58065a90 0x7f2b58067f50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c009a90 tx=0x7f2b5c008040 comp rx=0 tx=0).stop 2023-11-06T02:09:25.777 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.776+0000 7f2b71f3d640 1 -- 172.21.15.49:0/1560286331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b6c152ef0 msgr2=0x7f2b6c150700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:25.778 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.776+0000 7f2b71f3d640 1 --2- 172.21.15.49:0/1560286331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b6c152ef0 0x7f2b6c150700 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f2b60011590 tx=0x7f2b60011a50 comp rx=0 tx=0).stop 2023-11-06T02:09:25.778 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.776+0000 7f2b71f3d640 1 -- 172.21.15.49:0/1560286331 shutdown_connections 2023-11-06T02:09:25.778 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.776+0000 7f2b71f3d640 1 --2- 172.21.15.49:0/1560286331 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2b58065a90 0x7f2b58067f50 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:25.778 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.776+0000 7f2b71f3d640 1 --2- 172.21.15.49:0/1560286331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b6c152ef0 0x7f2b6c150700 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:25.778 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.776+0000 7f2b71f3d640 1 --2- 172.21.15.49:0/1560286331 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b6c0c56d0 0x7f2b6c0af0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:25.779 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.776+0000 7f2b71f3d640 1 --2- 172.21.15.49:0/1560286331 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b54004d10 0x7f2b6c14eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:25.779 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.776+0000 7f2b71f3d640 1 -- 172.21.15.49:0/1560286331 >> 172.21.15.49:0/1560286331 conn(0x7f2b6c06c0a0 msgr2=0x7f2b6c0c2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:25.779 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.776+0000 7f2b71f3d640 1 -- 172.21.15.49:0/1560286331 shutdown_connections 2023-11-06T02:09:25.779 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.776+0000 7f2b71f3d640 1 -- 172.21.15.49:0/1560286331 wait complete. 2023-11-06T02:09:25.794 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T02:09:25.919 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.920+0000 7f96777a1640 1 Processor -- start 2023-11-06T02:09:25.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.920+0000 7f96777a1640 1 -- start start 2023-11-06T02:09:25.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.920+0000 7f96777a1640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f967015abd0 0x7f96700b43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:25.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.920+0000 7f96777a1640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f96700710c0 0x7f96700b1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:25.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.920+0000 7f96777a1640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9670069d70 con 0x7f96700b4900 2023-11-06T02:09:25.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.920+0000 7f96777a1640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96700b4d80 con 0x7f967015abd0 2023-11-06T02:09:25.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.920+0000 7f96777a1640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96700ca160 con 0x7f96700710c0 2023-11-06T02:09:25.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.920+0000 7f9675d3e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f96700710c0 0x7f96700b1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:25.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.920+0000 7f967553d640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f967015abd0 0x7f96700b43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:25.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.920+0000 7f9675d3e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f96700710c0 0x7f96700b1c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.49:52718/0 (socket says 172.21.15.49:52718) 2023-11-06T02:09:25.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.920+0000 7f9675d3e640 1 -- 172.21.15.49:0/2772048215 learned_addr learned my addr 172.21.15.49:0/2772048215 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:25.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.924+0000 7f967553d640 1 -- 172.21.15.49:0/2772048215 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f96700710c0 msgr2=0x7f96700b1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:25.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.924+0000 7f967553d640 1 --2- 172.21.15.49:0/2772048215 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f96700710c0 0x7f96700b1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:25.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.924+0000 7f967553d640 1 -- 172.21.15.49:0/2772048215 >> v1:172.21.15.150:6789/0 conn(0x7f96700b4900 legacy=0x7f96700709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:25.924 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.924+0000 7f967553d640 1 -- 172.21.15.49:0/2772048215 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f967006ad80 con 0x7f967015abd0 2023-11-06T02:09:25.925 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.924+0000 7f967553d640 1 --2- 172.21.15.49:0/2772048215 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f967015abd0 0x7f96700b43c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f966c0097c0 tx=0x7f966c01c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4313e9de146f7f93 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:25.926 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.924+0000 7f9667fff640 1 -- 172.21.15.49:0/2772048215 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f966c0ac620 con 0x7f967015abd0 2023-11-06T02:09:25.926 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.924+0000 7f9667fff640 1 -- 172.21.15.49:0/2772048215 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f966c0ac780 con 0x7f967015abd0 2023-11-06T02:09:25.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.924+0000 7f9667fff640 1 -- 172.21.15.49:0/2772048215 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f966c0acb10 con 0x7f967015abd0 2023-11-06T02:09:25.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.924+0000 7f96777a1640 1 -- 172.21.15.49:0/2772048215 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f967015abd0 msgr2=0x7f96700b43c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:25.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.924+0000 7f96777a1640 1 --2- 172.21.15.49:0/2772048215 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f967015abd0 0x7f96700b43c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f966c0097c0 tx=0x7f966c01c3c0 comp rx=0 tx=0).stop 2023-11-06T02:09:25.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.924+0000 7f96777a1640 1 -- 172.21.15.49:0/2772048215 shutdown_connections 2023-11-06T02:09:25.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.924+0000 7f96777a1640 1 --2- 172.21.15.49:0/2772048215 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f96700710c0 0x7f96700b1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:25.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.924+0000 7f96777a1640 1 --2- 172.21.15.49:0/2772048215 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f967015abd0 0x7f96700b43c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:25.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.924+0000 7f96777a1640 1 -- 172.21.15.49:0/2772048215 >> 172.21.15.49:0/2772048215 conn(0x7f96700cee20 msgr2=0x7f96700cf230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:25.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.924+0000 7f96777a1640 1 -- 172.21.15.49:0/2772048215 shutdown_connections 2023-11-06T02:09:25.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.924+0000 7f96777a1640 1 -- 172.21.15.49:0/2772048215 wait complete. 2023-11-06T02:09:25.930 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.928+0000 7f96777a1640 1 Processor -- start 2023-11-06T02:09:25.930 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.928+0000 7f96777a1640 1 -- start start 2023-11-06T02:09:25.932 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.928+0000 7f96777a1640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f96700710c0 0x7f967014d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:25.932 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.928+0000 7f96777a1640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f96700b4900 0x7f96700cd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:25.932 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.928+0000 7f96777a1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f967015abd0 0x7f96700c5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:25.932 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.928+0000 7f96777a1640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96700c3ff0 con 0x7f967015abd0 2023-11-06T02:09:25.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.928+0000 7f96777a1640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96700c2dc0 con 0x7f96700b4900 2023-11-06T02:09:25.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.928+0000 7f96777a1640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96700c36c0 con 0x7f96700710c0 2023-11-06T02:09:25.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.928+0000 7f9675d3e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f967015abd0 0x7f96700c5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:25.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.928+0000 7f9675d3e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f967015abd0 0x7f96700c5ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42056/0 (socket says 172.21.15.49:42056) 2023-11-06T02:09:25.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.928+0000 7f9675d3e640 1 -- 172.21.15.49:0/2365689054 learned_addr learned my addr 172.21.15.49:0/2365689054 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:25.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.928+0000 7f9674d3c640 1 --2- 172.21.15.49:0/2365689054 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f96700b4900 0x7f96700cd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:25.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.928+0000 7f967553d640 1 --2- 172.21.15.49:0/2365689054 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f96700710c0 0x7f967014d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:25.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.928+0000 7f9675d3e640 1 -- 172.21.15.49:0/2365689054 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f96700710c0 msgr2=0x7f967014d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:25.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.928+0000 7f9675d3e640 1 --2- 172.21.15.49:0/2365689054 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f96700710c0 0x7f967014d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:25.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.928+0000 7f9675d3e640 1 -- 172.21.15.49:0/2365689054 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f96700b4900 msgr2=0x7f96700cd1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:25.935 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.928+0000 7f9675d3e640 1 --2- 172.21.15.49:0/2365689054 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f96700b4900 0x7f96700cd1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:25.935 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.928+0000 7f9675d3e640 1 -- 172.21.15.49:0/2365689054 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f966c009420 con 0x7f967015abd0 2023-11-06T02:09:25.935 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.928+0000 7f9675d3e640 1 --2- 172.21.15.49:0/2365689054 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f967015abd0 0x7f96700c5ef0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f966800efc0 tx=0x7f966800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:25.935 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.928+0000 7f9665ffb640 1 -- 172.21.15.49:0/2365689054 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9668009bd0 con 0x7f967015abd0 2023-11-06T02:09:25.935 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.928+0000 7f9665ffb640 1 -- 172.21.15.49:0/2365689054 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f966800f040 con 0x7f967015abd0 2023-11-06T02:09:25.936 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.928+0000 7f9665ffb640 1 -- 172.21.15.49:0/2365689054 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96680039a0 con 0x7f967015abd0 2023-11-06T02:09:25.936 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.928+0000 7f96777a1640 1 -- 172.21.15.49:0/2365689054 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96700c39a0 con 0x7f967015abd0 2023-11-06T02:09:25.936 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.928+0000 7f96777a1640 1 -- 172.21.15.49:0/2365689054 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f967014e4d0 con 0x7f967015abd0 2023-11-06T02:09:25.936 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.928+0000 7f96777a1640 1 -- 172.21.15.49:0/2365689054 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9670074040 con 0x7f967015abd0 2023-11-06T02:09:25.936 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.928+0000 7f9665ffb640 1 -- 172.21.15.49:0/2365689054 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f9668003b00 con 0x7f967015abd0 2023-11-06T02:09:25.937 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.932+0000 7f9665ffb640 1 --2- 172.21.15.49:0/2365689054 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9654065a60 0x7f9654067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:25.937 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.932+0000 7f9665ffb640 1 -- 172.21.15.49:0/2365689054 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f9668083830 con 0x7f967015abd0 2023-11-06T02:09:25.937 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.932+0000 7f967553d640 1 --2- 172.21.15.49:0/2365689054 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9654065a60 0x7f9654067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:25.937 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.932+0000 7f967553d640 1 --2- 172.21.15.49:0/2365689054 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9654065a60 0x7f9654067f20 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f966c0ae000 tx=0x7f966c0a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:25.938 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:25.936+0000 7f9665ffb640 1 -- 172.21.15.49:0/2365689054 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f9668051190 con 0x7f967015abd0 2023-11-06T02:09:26.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.088+0000 7f96777a1640 1 -- 172.21.15.49:0/2365689054 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f96700c4130 con 0x7f967015abd0 2023-11-06T02:09:26.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.092+0000 7f9665ffb640 1 -- 172.21.15.49:0/2365689054 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17906 (secure 0 0 0) 0x7f96680547b0 con 0x7f967015abd0 2023-11-06T02:09:26.092 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:09:26.092 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":25,"fsid":"1a8b035c-6ba4-499a-b876-aea47fd9c0d4","created":"2023-11-06T02:08:06.440857+0000","modified":"2023-11-06T02:09:13.195118+0000","last_up_change":"2023-11-06T02:08:17.982372+0000","last_in_change":"2023-11-06T02:08:08.906631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T02:08:19.252222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T02:08:24.776211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-06T02:08:28.936178+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-06T02:08:29.563468+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ad55af8-0571-4e6a-8722-b5d429b7496d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6809","nonce":38116},{"type":"v1","addr":"172.21.15.49:6811","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6813","nonce":38116},{"type":"v1","addr":"172.21.15.49:6815","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6821","nonce":38116},{"type":"v1","addr":"172.21.15.49:6823","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6817","nonce":38116},{"type":"v1","addr":"172.21.15.49:6819","nonce":38116}]},"public_addr":"172.21.15.49:6811/38116","cluster_addr":"172.21.15.49:6815/38116","heartbeat_back_addr":"172.21.15.49:6823/38116","heartbeat_front_addr":"172.21.15.49:6819/38116","state":["exists","up"]},{"osd":1,"uuid":"c8b31f85-a757-43ed-9419-351acc46611b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":38113},{"type":"v1","addr":"172.21.15.49:6810","nonce":38113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":38113},{"type":"v1","addr":"172.21.15.49:6814","nonce":38113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":38113},{"type":"v1","addr":"172.21.15.49:6822","nonce":38113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":38113},{"type":"v1","addr":"172.21.15.49:6818","nonce":38113}]},"public_addr":"172.21.15.49:6810/38113","cluster_addr":"172.21.15.49:6814/38113","heartbeat_back_addr":"172.21.15.49:6822/38113","heartbeat_front_addr":"172.21.15.49:6818/38113","state":["exists","up"]},{"osd":2,"uuid":"555b3bf3-0a70-4b91-8efd-e4204b5fc26e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":38114},{"type":"v1","addr":"172.21.15.49:6801","nonce":38114}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":38114},{"type":"v1","addr":"172.21.15.49:6803","nonce":38114}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":38114},{"type":"v1","addr":"172.21.15.49:6807","nonce":38114}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":38114},{"type":"v1","addr":"172.21.15.49:6805","nonce":38114}]},"public_addr":"172.21.15.49:6801/38114","cluster_addr":"172.21.15.49:6803/38114","heartbeat_back_addr":"172.21.15.49:6807/38114","heartbeat_front_addr":"172.21.15.49:6805/38114","state":["exists","up"]},{"osd":3,"uuid":"8d4e5d77-03ef-4b9b-bfcb-5ae93a46795c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":38115},{"type":"v1","addr":"172.21.15.49:6825","nonce":38115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":38115},{"type":"v1","addr":"172.21.15.49:6827","nonce":38115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":38115},{"type":"v1","addr":"172.21.15.49:6831","nonce":38115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":38115},{"type":"v1","addr":"172.21.15.49:6829","nonce":38115}]},"public_addr":"172.21.15.49:6825/38115","cluster_addr":"172.21.15.49:6827/38115","heartbeat_back_addr":"172.21.15.49:6831/38115","heartbeat_front_addr":"172.21.15.49:6829/38115","state":["exists","up"]},{"osd":4,"uuid":"dba03ad4-68a3-4cf3-9218-ac611f02eacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":38417},{"type":"v1","addr":"172.21.15.150:6809","nonce":38417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":38417},{"type":"v1","addr":"172.21.15.150:6811","nonce":38417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":38417},{"type":"v1","addr":"172.21.15.150:6815","nonce":38417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":38417},{"type":"v1","addr":"172.21.15.150:6813","nonce":38417}]},"public_addr":"172.21.15.150:6809/38417","cluster_addr":"172.21.15.150:6811/38417","heartbeat_back_addr":"172.21.15.150:6815/38417","heartbeat_front_addr":"172.21.15.150:6813/38417","state":["exists","up"]},{"osd":5,"uuid":"0df9c303-c255-4640-8e91-01f1c3efe18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":38418},{"type":"v1","addr":"172.21.15.150:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":38418},{"type":"v1","addr":"172.21.15.150:6803","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":38418},{"type":"v1","addr":"172.21.15.150:6807","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":38418},{"type":"v1","addr":"172.21.15.150:6805","nonce":38418}]},"public_addr":"172.21.15.150:6801/38418","cluster_addr":"172.21.15.150:6803/38418","heartbeat_back_addr":"172.21.15.150:6807/38418","heartbeat_front_addr":"172.21.15.150:6805/38418","state":["exists","up"]},{"osd":6,"uuid":"e59fa189-4b70-4c06-9722-c02b90828bf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":38428},{"type":"v1","addr":"172.21.15.150:6825","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":38428},{"type":"v1","addr":"172.21.15.150:6827","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":38428},{"type":"v1","addr":"172.21.15.150:6831","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":38428},{"type":"v1","addr":"172.21.15.150:6829","nonce":38428}]},"public_addr":"172.21.15.150:6825/38428","cluster_addr":"172.21.15.150:6827/38428","heartbeat_back_addr":"172.21.15.150:6831/38428","heartbeat_front_addr":"172.21.15.150:6829/38428","state":["exists","up"]},{"osd":7,"uuid":"a64f57e1-ef98-46bd-b304-385759e95898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":38421},{"type":"v1","addr":"172.21.15.150:6817","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":38421},{"type":"v1","addr":"172.21.15.150:6819","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":38421},{"type":"v1","addr":"172.21.15.150:6823","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":38421},{"type":"v1","addr":"172.21.15.150:6821","nonce":38421}]},"public_addr":"172.21.15.150:6817/38421","cluster_addr":"172.21.15.150:6819/38421","heartbeat_back_addr":"172.21.15.150:6823/38421","heartbeat_front_addr":"172.21.15.150:6821/38421","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.438552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.413280+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.296466+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.424966+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.276953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.230984+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.410235+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.480084+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.14","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T02:09:26.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.096+0000 7f96777a1640 1 -- 172.21.15.49:0/2365689054 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9654065a60 msgr2=0x7f9654067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:26.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.096+0000 7f96777a1640 1 --2- 172.21.15.49:0/2365689054 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9654065a60 0x7f9654067f20 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f966c0ae000 tx=0x7f966c0a7000 comp rx=0 tx=0).stop 2023-11-06T02:09:26.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.096+0000 7f96777a1640 1 -- 172.21.15.49:0/2365689054 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f967015abd0 msgr2=0x7f96700c5ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:26.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.096+0000 7f96777a1640 1 --2- 172.21.15.49:0/2365689054 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f967015abd0 0x7f96700c5ef0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f966800efc0 tx=0x7f966800c490 comp rx=0 tx=0).stop 2023-11-06T02:09:26.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.096+0000 7f96777a1640 1 -- 172.21.15.49:0/2365689054 shutdown_connections 2023-11-06T02:09:26.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.096+0000 7f96777a1640 1 --2- 172.21.15.49:0/2365689054 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9654065a60 0x7f9654067f20 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:26.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.096+0000 7f96777a1640 1 --2- 172.21.15.49:0/2365689054 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f967015abd0 0x7f96700c5ef0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:26.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.096+0000 7f96777a1640 1 --2- 172.21.15.49:0/2365689054 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f96700b4900 0x7f96700cd1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:26.096 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.096+0000 7f96777a1640 1 --2- 172.21.15.49:0/2365689054 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f96700710c0 0x7f967014d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:26.096 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.096+0000 7f96777a1640 1 -- 172.21.15.49:0/2365689054 >> 172.21.15.49:0/2365689054 conn(0x7f96700cee20 msgr2=0x7f9670159510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:26.096 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.096+0000 7f96777a1640 1 -- 172.21.15.49:0/2365689054 shutdown_connections 2023-11-06T02:09:26.096 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.096+0000 7f96777a1640 1 -- 172.21.15.49:0/2365689054 wait complete. 2023-11-06T02:09:26.110 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T02:08:19.252222+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T02:08:24.776211+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T02:08:28.936178+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T02:08:29.563468+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T02:09:26.111 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T02:09:26.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.228+0000 7f68b7e8c640 1 Processor -- start 2023-11-06T02:09:26.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.228+0000 7f68b7e8c640 1 -- start start 2023-11-06T02:09:26.229 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.228+0000 7f68b7e8c640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f68b015a990 0x7f68b015dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:26.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.228+0000 7f68b7e8c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f68b00b0220 0x7f68b00b0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:26.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.228+0000 7f68b7e8c640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68b006c5a0 con 0x7f68b01595a0 2023-11-06T02:09:26.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.228+0000 7f68b7e8c640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68b015e250 con 0x7f68b015a990 2023-11-06T02:09:26.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.228+0000 7f68b7e8c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68b006c880 con 0x7f68b00b0220 2023-11-06T02:09:26.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.228+0000 7f68b6429640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f68b00b0220 0x7f68b00b0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:26.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.228+0000 7f68b6429640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f68b00b0220 0x7f68b00b0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.49:52742/0 (socket says 172.21.15.49:52742) 2023-11-06T02:09:26.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.228+0000 7f68b6429640 1 -- 172.21.15.49:0/2256221580 learned_addr learned my addr 172.21.15.49:0/2256221580 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:26.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.228+0000 7f68b5427640 1 --2- 172.21.15.49:0/2256221580 >> v2:172.21.15.150:3300/0 conn(0x7f68b015a990 0x7f68b015dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:26.234 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.228+0000 7f68b6429640 1 -- 172.21.15.49:0/2256221580 >> v2:172.21.15.150:3300/0 conn(0x7f68b015a990 msgr2=0x7f68b015dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:26.234 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.228+0000 7f68b6429640 1 --2- 172.21.15.49:0/2256221580 >> v2:172.21.15.150:3300/0 conn(0x7f68b015a990 0x7f68b015dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:26.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.228+0000 7f68b6429640 1 -- 172.21.15.49:0/2256221580 >> v1:172.21.15.150:6789/0 conn(0x7f68b01595a0 legacy=0x7f68b006d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:26.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.228+0000 7f68b6429640 1 -- 172.21.15.49:0/2256221580 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68b006cb60 con 0x7f68b00b0220 2023-11-06T02:09:26.242 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.232+0000 7f68b6429640 1 --2- 172.21.15.49:0/2256221580 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f68b00b0220 0x7f68b00b0620 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f68a800d150 tx=0x7f68a801ebe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b96ce91a54c6bc8d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:26.242 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.232+0000 7f68b4c26640 1 -- 172.21.15.49:0/2256221580 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a800b2d0 con 0x7f68b00b0220 2023-11-06T02:09:26.243 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.232+0000 7f68b4c26640 1 -- 172.21.15.49:0/2256221580 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68a800b430 con 0x7f68b00b0220 2023-11-06T02:09:26.243 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.232+0000 7f68b4c26640 1 -- 172.21.15.49:0/2256221580 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a80a7cd0 con 0x7f68b00b0220 2023-11-06T02:09:26.243 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.232+0000 7f68b7e8c640 1 -- 172.21.15.49:0/2256221580 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f68b00b0220 msgr2=0x7f68b00b0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:26.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.232+0000 7f68b7e8c640 1 --2- 172.21.15.49:0/2256221580 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f68b00b0220 0x7f68b00b0620 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f68a800d150 tx=0x7f68a801ebe0 comp rx=0 tx=0).stop 2023-11-06T02:09:26.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.232+0000 7f68b7e8c640 1 -- 172.21.15.49:0/2256221580 shutdown_connections 2023-11-06T02:09:26.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.232+0000 7f68b7e8c640 1 --2- 172.21.15.49:0/2256221580 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f68b00b0220 0x7f68b00b0620 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:26.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.232+0000 7f68b7e8c640 1 --2- 172.21.15.49:0/2256221580 >> v2:172.21.15.150:3300/0 conn(0x7f68b015a990 0x7f68b015dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:26.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.232+0000 7f68b7e8c640 1 -- 172.21.15.49:0/2256221580 >> 172.21.15.49:0/2256221580 conn(0x7f68b006a370 msgr2=0x7f68b00ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:26.246 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.232+0000 7f68b7e8c640 1 -- 172.21.15.49:0/2256221580 shutdown_connections 2023-11-06T02:09:26.246 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.232+0000 7f68b7e8c640 1 -- 172.21.15.49:0/2256221580 wait complete. 2023-11-06T02:09:26.246 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.232+0000 7f68b7e8c640 1 Processor -- start 2023-11-06T02:09:26.247 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.232+0000 7f68b7e8c640 1 -- start start 2023-11-06T02:09:26.247 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.236+0000 7f68b7e8c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f68b00b0220 0x7f68b014d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:26.247 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.236+0000 7f68b7e8c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f68b01595a0 0x7f68b0145b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:26.248 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.236+0000 7f68b7e8c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68b015a990 0x7f68b01460c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:26.248 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.236+0000 7f68b7e8c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68b00b1240 con 0x7f68b00b0220 2023-11-06T02:09:26.249 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.236+0000 7f68b7e8c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68b0146600 con 0x7f68b015a990 2023-11-06T02:09:26.249 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.236+0000 7f68b7e8c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68b01581e0 con 0x7f68b01595a0 2023-11-06T02:09:26.250 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.236+0000 7f68b5c28640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f68b00b0220 0x7f68b014d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:26.250 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.236+0000 7f68b5c28640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f68b00b0220 0x7f68b014d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42070/0 (socket says 172.21.15.49:42070) 2023-11-06T02:09:26.250 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.236+0000 7f68b5c28640 1 -- 172.21.15.49:0/737088799 learned_addr learned my addr 172.21.15.49:0/737088799 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:26.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.236+0000 7f68b5427640 1 --2- 172.21.15.49:0/737088799 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f68b01595a0 0x7f68b0145b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:26.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.236+0000 7f68b6429640 1 --2- 172.21.15.49:0/737088799 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68b015a990 0x7f68b01460c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:26.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.236+0000 7f68b5c28640 1 -- 172.21.15.49:0/737088799 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f68b01595a0 msgr2=0x7f68b0145b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:26.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.236+0000 7f68b5c28640 1 --2- 172.21.15.49:0/737088799 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f68b01595a0 0x7f68b0145b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:26.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.236+0000 7f68b5c28640 1 -- 172.21.15.49:0/737088799 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68b015a990 msgr2=0x7f68b01460c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:26.253 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.236+0000 7f68b5c28640 1 --2- 172.21.15.49:0/737088799 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68b015a990 0x7f68b01460c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:26.253 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.236+0000 7f68b5c28640 1 -- 172.21.15.49:0/737088799 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68a800bd40 con 0x7f68b00b0220 2023-11-06T02:09:26.254 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.236+0000 7f68b5c28640 1 --2- 172.21.15.49:0/737088799 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f68b00b0220 0x7f68b014d750 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f68ac00c500 tx=0x7f68ac00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:26.254 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.236+0000 7f68a67fc640 1 -- 172.21.15.49:0/737088799 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68ac005220 con 0x7f68b00b0220 2023-11-06T02:09:26.254 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.236+0000 7f68a67fc640 1 -- 172.21.15.49:0/737088799 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68ac005380 con 0x7f68b00b0220 2023-11-06T02:09:26.255 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.236+0000 7f68a67fc640 1 -- 172.21.15.49:0/737088799 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68ac007270 con 0x7f68b00b0220 2023-11-06T02:09:26.255 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.236+0000 7f68b7e8c640 1 -- 172.21.15.49:0/737088799 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68b00b1a70 con 0x7f68b00b0220 2023-11-06T02:09:26.255 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.236+0000 7f68b7e8c640 1 -- 172.21.15.49:0/737088799 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68b00b1700 con 0x7f68b00b0220 2023-11-06T02:09:26.256 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.236+0000 7f68b7e8c640 1 -- 172.21.15.49:0/737088799 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68b006c210 con 0x7f68b00b0220 2023-11-06T02:09:26.256 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.236+0000 7f68a67fc640 1 -- 172.21.15.49:0/737088799 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f68ac0073d0 con 0x7f68b00b0220 2023-11-06T02:09:26.256 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.236+0000 7f68a67fc640 1 --2- 172.21.15.49:0/737088799 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6894065a60 0x7f6894067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:26.257 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.236+0000 7f68a67fc640 1 -- 172.21.15.49:0/737088799 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f68ac082c90 con 0x7f68b00b0220 2023-11-06T02:09:26.257 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.236+0000 7f68b5427640 1 --2- 172.21.15.49:0/737088799 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6894065a60 0x7f6894067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:26.258 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.240+0000 7f68b5427640 1 --2- 172.21.15.49:0/737088799 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6894065a60 0x7f6894067f20 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f68b014d130 tx=0x7f68a0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:26.258 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.240+0000 7f68a67fc640 1 -- 172.21.15.49:0/737088799 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f68b006c210 con 0x7f68b00b0220 2023-11-06T02:09:26.391 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.392+0000 7f68b7e8c640 1 -- 172.21.15.49:0/737088799 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f68b006c210 con 0x7f68b00b0220 2023-11-06T02:09:26.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.392+0000 7f68a67fc640 1 -- 172.21.15.49:0/737088799 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f68b006c210 con 0x7f68b00b0220 2023-11-06T02:09:26.392 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2023-11-06T02:09:26.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.396+0000 7f68b7e8c640 1 -- 172.21.15.49:0/737088799 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6894065a60 msgr2=0x7f6894067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:26.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.396+0000 7f68b7e8c640 1 --2- 172.21.15.49:0/737088799 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6894065a60 0x7f6894067f20 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f68b014d130 tx=0x7f68a0008040 comp rx=0 tx=0).stop 2023-11-06T02:09:26.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.396+0000 7f68b7e8c640 1 -- 172.21.15.49:0/737088799 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f68b00b0220 msgr2=0x7f68b014d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:26.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.396+0000 7f68b7e8c640 1 --2- 172.21.15.49:0/737088799 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f68b00b0220 0x7f68b014d750 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f68ac00c500 tx=0x7f68ac00c9c0 comp rx=0 tx=0).stop 2023-11-06T02:09:26.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.396+0000 7f68b7e8c640 1 -- 172.21.15.49:0/737088799 shutdown_connections 2023-11-06T02:09:26.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.396+0000 7f68b7e8c640 1 --2- 172.21.15.49:0/737088799 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6894065a60 0x7f6894067f20 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:26.395 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.396+0000 7f68b7e8c640 1 --2- 172.21.15.49:0/737088799 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68b015a990 0x7f68b01460c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:26.395 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.396+0000 7f68b7e8c640 1 --2- 172.21.15.49:0/737088799 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f68b01595a0 0x7f68b0145b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:26.395 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.396+0000 7f68b7e8c640 1 --2- 172.21.15.49:0/737088799 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f68b00b0220 0x7f68b014d750 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:26.395 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.396+0000 7f68b7e8c640 1 -- 172.21.15.49:0/737088799 >> 172.21.15.49:0/737088799 conn(0x7f68b006a370 msgr2=0x7f68b0072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:26.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.396+0000 7f68b7e8c640 1 -- 172.21.15.49:0/737088799 shutdown_connections 2023-11-06T02:09:26.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.396+0000 7f68b7e8c640 1 -- 172.21.15.49:0/737088799 wait complete. 2023-11-06T02:09:26.409 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T02:09:26.533 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.536+0000 7f2e74590640 1 Processor -- start 2023-11-06T02:09:26.534 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.536+0000 7f2e74590640 1 -- start start 2023-11-06T02:09:26.537 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.536+0000 7f2e74590640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f2e6c0cda10 0x7f2e6c0cf5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:26.537 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.536+0000 7f2e74590640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e6c0d2810 con 0x7f2e6c0d1fa0 2023-11-06T02:09:26.537 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.536+0000 7f2e74590640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e6c0d1b10 con 0x7f2e6c0cf1f0 2023-11-06T02:09:26.537 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.536+0000 7f2e74590640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e6c0d0f70 con 0x7f2e6c0cda10 2023-11-06T02:09:26.537 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.536+0000 7f2e72b2d640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f2e6c0d1fa0 0x7f2e6c15e3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:35542/0 (socket says 172.21.15.49:35542) 2023-11-06T02:09:26.538 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.536+0000 7f2e72b2d640 1 -- 172.21.15.49:0/514303539 learned_addr learned my addr 172.21.15.49:0/514303539 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:09:26.538 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.536+0000 7f2e71b2b640 1 --2- 172.21.15.49:0/514303539 >> v2:172.21.15.150:3300/0 conn(0x7f2e6c0cda10 0x7f2e6c0cf5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:26.538 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.536+0000 7f2e7132a640 1 -- 172.21.15.49:0/514303539 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 388940600 0 0) 0x7f2e6c0d2810 con 0x7f2e6c0d1fa0 2023-11-06T02:09:26.538 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.536+0000 7f2e7132a640 1 -- 172.21.15.49:0/514303539 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e58003530 con 0x7f2e6c0d1fa0 2023-11-06T02:09:26.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.536+0000 7f2e7132a640 1 -- 172.21.15.49:0/514303539 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2355127925 0 0) 0x7f2e58003530 con 0x7f2e6c0d1fa0 2023-11-06T02:09:26.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.536+0000 7f2e7132a640 1 -- 172.21.15.49:0/514303539 >> v2:172.21.15.150:3300/0 conn(0x7f2e6c0cda10 msgr2=0x7f2e6c0cf5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:26.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.536+0000 7f2e7132a640 1 --2- 172.21.15.49:0/514303539 >> v2:172.21.15.150:3300/0 conn(0x7f2e6c0cda10 0x7f2e6c0cf5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:26.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.536+0000 7f2e7132a640 1 -- 172.21.15.49:0/514303539 >> v1:172.21.15.150:6789/0 conn(0x7f2e6c0cf1f0 legacy=0x7f2e6c15dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:26.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.536+0000 7f2e7132a640 1 -- 172.21.15.49:0/514303539 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e6c0d11e0 con 0x7f2e6c0d1fa0 2023-11-06T02:09:26.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.536+0000 7f2e7132a640 1 -- 172.21.15.49:0/514303539 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7f2e64001a00 con 0x7f2e6c0d1fa0 2023-11-06T02:09:26.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.536+0000 7f2e7132a640 1 -- 172.21.15.49:0/514303539 >> v1:172.21.15.49:6789/0 conn(0x7f2e6c0d1fa0 legacy=0x7f2e6c15e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:26.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.536+0000 7f2e7132a640 1 --2- 172.21.15.49:0/514303539 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2e58003a00 0x7f2e58003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:26.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.536+0000 7f2e7132a640 1 --2- 172.21.15.49:0/514303539 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e580047d0 0x7f2e58004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:26.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.536+0000 7f2e7132a640 1 --2- 172.21.15.49:0/514303539 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2e58005190 0x7f2e5800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:26.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.536+0000 7f2e7132a640 1 -- 172.21.15.49:0/514303539 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e6c0d0f70 con 0x7f2e580047d0 2023-11-06T02:09:26.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.536+0000 7f2e7132a640 1 -- 172.21.15.49:0/514303539 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e5800c0c0 con 0x7f2e58003a00 2023-11-06T02:09:26.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.536+0000 7f2e7132a640 1 -- 172.21.15.49:0/514303539 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e5800c280 con 0x7f2e58005190 2023-11-06T02:09:26.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.536+0000 7f2e71b2b640 1 --2- 172.21.15.49:0/514303539 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e580047d0 0x7f2e58004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:26.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.536+0000 7f2e7232c640 1 --2- 172.21.15.49:0/514303539 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2e58003a00 0x7f2e58003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:26.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.536+0000 7f2e72b2d640 1 --2- 172.21.15.49:0/514303539 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2e58005190 0x7f2e5800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:26.542 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.536+0000 7f2e71b2b640 1 -- 172.21.15.49:0/514303539 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2e58005190 msgr2=0x7f2e5800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:26.542 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.536+0000 7f2e71b2b640 1 --2- 172.21.15.49:0/514303539 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2e58005190 0x7f2e5800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:26.542 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.536+0000 7f2e71b2b640 1 -- 172.21.15.49:0/514303539 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2e58003a00 msgr2=0x7f2e58003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:26.542 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.536+0000 7f2e71b2b640 1 --2- 172.21.15.49:0/514303539 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2e58003a00 0x7f2e58003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:26.542 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.536+0000 7f2e71b2b640 1 -- 172.21.15.49:0/514303539 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e58003530 con 0x7f2e580047d0 2023-11-06T02:09:26.543 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.536+0000 7f2e72b2d640 1 --2- 172.21.15.49:0/514303539 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2e58005190 0x7f2e5800b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:09:26.543 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.536+0000 7f2e7232c640 1 --2- 172.21.15.49:0/514303539 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2e58003a00 0x7f2e58003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:09:26.543 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.536+0000 7f2e72b2d640 1 -- 172.21.15.49:0/514303539 reap_dead start 2023-11-06T02:09:26.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.540+0000 7f2e71b2b640 1 --2- 172.21.15.49:0/514303539 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e580047d0 0x7f2e58004c50 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c009ab0 tx=0x7f2e5c01ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f040c54097b2f65 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:26.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.540+0000 7f2e7132a640 1 -- 172.21.15.49:0/514303539 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e5c0a5620 con 0x7f2e580047d0 2023-11-06T02:09:26.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.540+0000 7f2e7132a640 1 -- 172.21.15.49:0/514303539 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e5c0a5780 con 0x7f2e580047d0 2023-11-06T02:09:26.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.540+0000 7f2e7132a640 1 -- 172.21.15.49:0/514303539 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e5c0a5b30 con 0x7f2e580047d0 2023-11-06T02:09:26.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.540+0000 7f2e74590640 1 -- 172.21.15.49:0/514303539 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e580047d0 msgr2=0x7f2e58004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:26.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.540+0000 7f2e74590640 1 --2- 172.21.15.49:0/514303539 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e580047d0 0x7f2e58004c50 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c009ab0 tx=0x7f2e5c01ca30 comp rx=0 tx=0).stop 2023-11-06T02:09:26.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.540+0000 7f2e74590640 1 -- 172.21.15.49:0/514303539 shutdown_connections 2023-11-06T02:09:26.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.540+0000 7f2e74590640 1 --2- 172.21.15.49:0/514303539 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e580047d0 0x7f2e58004c50 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:26.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.540+0000 7f2e74590640 1 -- 172.21.15.49:0/514303539 >> 172.21.15.49:0/514303539 conn(0x7f2e6c071aa0 msgr2=0x7f2e6c06b110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:26.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.540+0000 7f2e74590640 1 -- 172.21.15.49:0/514303539 shutdown_connections 2023-11-06T02:09:26.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.540+0000 7f2e74590640 1 -- 172.21.15.49:0/514303539 wait complete. 2023-11-06T02:09:26.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.540+0000 7f2e74590640 1 Processor -- start 2023-11-06T02:09:26.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.540+0000 7f2e74590640 1 -- start start 2023-11-06T02:09:26.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.540+0000 7f2e74590640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e580047d0 0x7f2e6c0c9b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:26.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.540+0000 7f2e74590640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2e6c0cc5a0 0x7f2e6c15c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:26.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.540+0000 7f2e74590640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2e6c14dad0 0x7f2e6c14df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:26.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.540+0000 7f2e74590640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e6c0cb070 con 0x7f2e580047d0 2023-11-06T02:09:26.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.540+0000 7f2e74590640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e6c14e4a0 con 0x7f2e6c14dad0 2023-11-06T02:09:26.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.540+0000 7f2e74590640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e6c0ca3f0 con 0x7f2e6c0cc5a0 2023-11-06T02:09:26.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.540+0000 7f2e7232c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e580047d0 0x7f2e6c0c9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:26.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.540+0000 7f2e7232c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e580047d0 0x7f2e6c0c9b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42088/0 (socket says 172.21.15.49:42088) 2023-11-06T02:09:26.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.540+0000 7f2e7232c640 1 -- 172.21.15.49:0/1677034829 learned_addr learned my addr 172.21.15.49:0/1677034829 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:26.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.540+0000 7f2e72b2d640 1 --2- 172.21.15.49:0/1677034829 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2e6c14dad0 0x7f2e6c14df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:26.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.544+0000 7f2e71b2b640 1 --2- 172.21.15.49:0/1677034829 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2e6c0cc5a0 0x7f2e6c15c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:26.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.544+0000 7f2e7232c640 1 -- 172.21.15.49:0/1677034829 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2e6c0cc5a0 msgr2=0x7f2e6c15c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:26.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.544+0000 7f2e7232c640 1 --2- 172.21.15.49:0/1677034829 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2e6c0cc5a0 0x7f2e6c15c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:26.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.544+0000 7f2e7232c640 1 -- 172.21.15.49:0/1677034829 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2e6c14dad0 msgr2=0x7f2e6c14df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:26.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.544+0000 7f2e7232c640 1 --2- 172.21.15.49:0/1677034829 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2e6c14dad0 0x7f2e6c14df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:26.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.544+0000 7f2e7232c640 1 -- 172.21.15.49:0/1677034829 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e6c0ca6d0 con 0x7f2e580047d0 2023-11-06T02:09:26.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.544+0000 7f2e71b2b640 1 --2- 172.21.15.49:0/1677034829 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2e6c0cc5a0 0x7f2e6c15c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:09:26.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.544+0000 7f2e7232c640 1 --2- 172.21.15.49:0/1677034829 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e580047d0 0x7f2e6c0c9b60 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f2e6800d7a0 tx=0x7f2e6800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:26.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.544+0000 7f2e62ffd640 1 -- 172.21.15.49:0/1677034829 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e68005270 con 0x7f2e580047d0 2023-11-06T02:09:26.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.544+0000 7f2e62ffd640 1 -- 172.21.15.49:0/1677034829 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e680053d0 con 0x7f2e580047d0 2023-11-06T02:09:26.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.544+0000 7f2e74590640 1 -- 172.21.15.49:0/1677034829 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e6c0ca9b0 con 0x7f2e580047d0 2023-11-06T02:09:26.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.544+0000 7f2e62ffd640 1 -- 172.21.15.49:0/1677034829 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e6800b8d0 con 0x7f2e580047d0 2023-11-06T02:09:26.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.544+0000 7f2e74590640 1 -- 172.21.15.49:0/1677034829 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e6c0ca0a0 con 0x7f2e580047d0 2023-11-06T02:09:26.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.544+0000 7f2e74590640 1 -- 172.21.15.49:0/1677034829 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e6c067140 con 0x7f2e580047d0 2023-11-06T02:09:26.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.544+0000 7f2e62ffd640 1 -- 172.21.15.49:0/1677034829 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f2e6800ba30 con 0x7f2e580047d0 2023-11-06T02:09:26.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.544+0000 7f2e62ffd640 1 --2- 172.21.15.49:0/1677034829 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2e50065a60 0x7f2e50067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:26.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.544+0000 7f2e62ffd640 1 -- 172.21.15.49:0/1677034829 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f2e68083240 con 0x7f2e580047d0 2023-11-06T02:09:26.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.544+0000 7f2e71b2b640 1 --2- 172.21.15.49:0/1677034829 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2e50065a60 0x7f2e50067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:26.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.544+0000 7f2e71b2b640 1 --2- 172.21.15.49:0/1677034829 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2e50065a60 0x7f2e50067f20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c0af000 tx=0x7f2e5c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:26.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.548+0000 7f2e62ffd640 1 -- 172.21.15.49:0/1677034829 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f2e68050af0 con 0x7f2e580047d0 2023-11-06T02:09:26.709 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.708+0000 7f2e74590640 1 -- 172.21.15.49:0/1677034829 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f2e6c15f730 con 0x7f2e580047d0 2023-11-06T02:09:26.710 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.712+0000 7f2e62ffd640 1 -- 172.21.15.49:0/1677034829 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7f2e68019310 con 0x7f2e580047d0 2023-11-06T02:09:26.710 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 8 2023-11-06T02:09:26.712 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.712+0000 7f2e74590640 1 -- 172.21.15.49:0/1677034829 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2e50065a60 msgr2=0x7f2e50067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:26.712 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.712+0000 7f2e74590640 1 --2- 172.21.15.49:0/1677034829 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2e50065a60 0x7f2e50067f20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c0af000 tx=0x7f2e5c008040 comp rx=0 tx=0).stop 2023-11-06T02:09:26.712 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.712+0000 7f2e74590640 1 -- 172.21.15.49:0/1677034829 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e580047d0 msgr2=0x7f2e6c0c9b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:26.712 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.712+0000 7f2e74590640 1 --2- 172.21.15.49:0/1677034829 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e580047d0 0x7f2e6c0c9b60 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f2e6800d7a0 tx=0x7f2e6800dc60 comp rx=0 tx=0).stop 2023-11-06T02:09:26.713 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.712+0000 7f2e74590640 1 -- 172.21.15.49:0/1677034829 shutdown_connections 2023-11-06T02:09:26.713 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.712+0000 7f2e74590640 1 --2- 172.21.15.49:0/1677034829 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2e50065a60 0x7f2e50067f20 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:26.713 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.712+0000 7f2e74590640 1 --2- 172.21.15.49:0/1677034829 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2e6c14dad0 0x7f2e6c14df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:26.713 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.712+0000 7f2e74590640 1 --2- 172.21.15.49:0/1677034829 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2e6c0cc5a0 0x7f2e6c15c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:26.714 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.712+0000 7f2e74590640 1 --2- 172.21.15.49:0/1677034829 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e580047d0 0x7f2e6c0c9b60 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:26.714 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.712+0000 7f2e74590640 1 -- 172.21.15.49:0/1677034829 >> 172.21.15.49:0/1677034829 conn(0x7f2e6c071aa0 msgr2=0x7f2e6c0b8000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:26.714 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.712+0000 7f2e74590640 1 -- 172.21.15.49:0/1677034829 shutdown_connections 2023-11-06T02:09:26.714 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.712+0000 7f2e74590640 1 -- 172.21.15.49:0/1677034829 wait complete. 2023-11-06T02:09:26.729 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T02:09:26.850 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.852+0000 7f832966a640 1 Processor -- start 2023-11-06T02:09:26.851 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.852+0000 7f832966a640 1 -- start start 2023-11-06T02:09:26.852 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.852+0000 7f832966a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83240b0930 0x7f832415dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:26.852 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.852+0000 7f832966a640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f832406c5a0 con 0x7f832415a990 2023-11-06T02:09:26.852 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.852+0000 7f832966a640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f832406c880 con 0x7f83241595a0 2023-11-06T02:09:26.852 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.852+0000 7f832966a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f832415e250 con 0x7f83240b0930 2023-11-06T02:09:26.852 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.852+0000 7f83227fc640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f832415a990 0x7f83240b0220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:35558/0 (socket says 172.21.15.49:35558) 2023-11-06T02:09:26.853 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.852+0000 7f83227fc640 1 -- 172.21.15.49:0/3158387782 learned_addr learned my addr 172.21.15.49:0/3158387782 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:09:26.853 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.852+0000 7f83237fe640 1 --2- 172.21.15.49:0/3158387782 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83240b0930 0x7f832415dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:26.855 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.852+0000 7f8321ffb640 1 -- 172.21.15.49:0/3158387782 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1698937931 0 0) 0x7f832406c5a0 con 0x7f832415a990 2023-11-06T02:09:26.855 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.852+0000 7f8321ffb640 1 -- 172.21.15.49:0/3158387782 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f830c003530 con 0x7f832415a990 2023-11-06T02:09:26.855 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.852+0000 7f8321ffb640 1 -- 172.21.15.49:0/3158387782 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3030200987 0 0) 0x7f830c003530 con 0x7f832415a990 2023-11-06T02:09:26.856 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.852+0000 7f8321ffb640 1 -- 172.21.15.49:0/3158387782 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83240b0930 msgr2=0x7f832415dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:26.856 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.852+0000 7f8321ffb640 1 --2- 172.21.15.49:0/3158387782 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83240b0930 0x7f832415dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:26.856 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.852+0000 7f8321ffb640 1 -- 172.21.15.49:0/3158387782 >> v1:172.21.15.150:6789/0 conn(0x7f83241595a0 legacy=0x7f832406d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:26.856 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.852+0000 7f8321ffb640 1 -- 172.21.15.49:0/3158387782 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f832406cb60 con 0x7f832415a990 2023-11-06T02:09:26.856 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.852+0000 7f8321ffb640 1 -- 172.21.15.49:0/3158387782 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7f8318001680 con 0x7f832415a990 2023-11-06T02:09:26.857 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.852+0000 7f8321ffb640 1 -- 172.21.15.49:0/3158387782 >> v1:172.21.15.49:6789/0 conn(0x7f832415a990 legacy=0x7f83240b0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:26.857 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.852+0000 7f8321ffb640 1 --2- 172.21.15.49:0/3158387782 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83240b0930 0x7f830c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:26.857 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.852+0000 7f8321ffb640 1 --2- 172.21.15.49:0/3158387782 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f830c003f30 0x7f830c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:26.857 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.852+0000 7f8321ffb640 1 --2- 172.21.15.49:0/3158387782 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f830c004d10 0x7f830c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:26.857 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.852+0000 7f8321ffb640 1 -- 172.21.15.49:0/3158387782 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f832415e250 con 0x7f830c004d10 2023-11-06T02:09:26.857 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.852+0000 7f8321ffb640 1 -- 172.21.15.49:0/3158387782 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f830c00ba70 con 0x7f830c003f30 2023-11-06T02:09:26.858 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.852+0000 7f8321ffb640 1 -- 172.21.15.49:0/3158387782 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f830c00bc30 con 0x7f83240b0930 2023-11-06T02:09:26.858 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.852+0000 7f83237fe640 1 --2- 172.21.15.49:0/3158387782 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f830c004d10 0x7f830c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:26.858 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.852+0000 7f83227fc640 1 --2- 172.21.15.49:0/3158387782 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f830c003f30 0x7f830c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:26.860 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.852+0000 7f8322ffd640 1 --2- 172.21.15.49:0/3158387782 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83240b0930 0x7f830c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:26.860 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.856+0000 7f83237fe640 1 -- 172.21.15.49:0/3158387782 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83240b0930 msgr2=0x7f830c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:26.861 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.856+0000 7f83237fe640 1 --2- 172.21.15.49:0/3158387782 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83240b0930 0x7f830c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:26.861 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.856+0000 7f83237fe640 1 -- 172.21.15.49:0/3158387782 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f830c003f30 msgr2=0x7f830c0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:26.861 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.856+0000 7f83237fe640 1 --2- 172.21.15.49:0/3158387782 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f830c003f30 0x7f830c0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:26.861 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.856+0000 7f83237fe640 1 -- 172.21.15.49:0/3158387782 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f830c003530 con 0x7f830c004d10 2023-11-06T02:09:26.861 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.856+0000 7f83237fe640 1 --2- 172.21.15.49:0/3158387782 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f830c004d10 0x7f830c005190 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f831c009fc0 tx=0x7f831c01cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80b06615773c909d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:26.862 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.856+0000 7f8321ffb640 1 -- 172.21.15.49:0/3158387782 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f831c0a5c70 con 0x7f830c004d10 2023-11-06T02:09:26.862 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.856+0000 7f8321ffb640 1 -- 172.21.15.49:0/3158387782 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f831c00f040 con 0x7f830c004d10 2023-11-06T02:09:26.862 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.856+0000 7f8321ffb640 1 -- 172.21.15.49:0/3158387782 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f831c0ad440 con 0x7f830c004d10 2023-11-06T02:09:26.865 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.856+0000 7f832966a640 1 -- 172.21.15.49:0/3158387782 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f830c004d10 msgr2=0x7f830c005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:26.866 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.856+0000 7f832966a640 1 --2- 172.21.15.49:0/3158387782 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f830c004d10 0x7f830c005190 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f831c009fc0 tx=0x7f831c01cc60 comp rx=0 tx=0).stop 2023-11-06T02:09:26.866 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.856+0000 7f83237fe640 1 -- 172.21.15.49:0/3158387782 reap_dead start 2023-11-06T02:09:26.866 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.856+0000 7f832966a640 1 -- 172.21.15.49:0/3158387782 shutdown_connections 2023-11-06T02:09:26.866 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.856+0000 7f832966a640 1 -- 172.21.15.49:0/3158387782 >> 172.21.15.49:0/3158387782 conn(0x7f832406a370 msgr2=0x7f83240ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:26.866 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.856+0000 7f832966a640 1 -- 172.21.15.49:0/3158387782 shutdown_connections 2023-11-06T02:09:26.867 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.856+0000 7f832966a640 1 -- 172.21.15.49:0/3158387782 wait complete. 2023-11-06T02:09:26.867 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.860+0000 7f832966a640 1 Processor -- start 2023-11-06T02:09:26.867 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.860+0000 7f832966a640 1 -- start start 2023-11-06T02:09:26.867 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.860+0000 7f832966a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f830c004d10 0x7f83241f1b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:26.867 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.860+0000 7f832966a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8324150c60 0x7f832414fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:26.868 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.860+0000 7f832966a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83240c9910 0x7f83240c9d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:26.868 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.860+0000 7f832966a640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83240ca2d0 con 0x7f83240c9910 2023-11-06T02:09:26.868 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.860+0000 7f832966a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f832414ff50 con 0x7f830c004d10 2023-11-06T02:09:26.868 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.860+0000 7f832966a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83241f1700 con 0x7f8324150c60 2023-11-06T02:09:26.868 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.860+0000 7f83237fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83240c9910 0x7f83240c9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:26.869 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.860+0000 7f83237fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83240c9910 0x7f83240c9d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42110/0 (socket says 172.21.15.49:42110) 2023-11-06T02:09:26.869 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.860+0000 7f83237fe640 1 -- 172.21.15.49:0/2909932318 learned_addr learned my addr 172.21.15.49:0/2909932318 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:26.869 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.860+0000 7f8322ffd640 1 --2- 172.21.15.49:0/2909932318 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f830c004d10 0x7f83241f1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:26.869 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.860+0000 7f83227fc640 1 --2- 172.21.15.49:0/2909932318 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8324150c60 0x7f832414fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:26.869 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.860+0000 7f83237fe640 1 -- 172.21.15.49:0/2909932318 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8324150c60 msgr2=0x7f832414fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:26.869 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.860+0000 7f83237fe640 1 --2- 172.21.15.49:0/2909932318 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8324150c60 0x7f832414fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:26.870 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.860+0000 7f83237fe640 1 -- 172.21.15.49:0/2909932318 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f830c004d10 msgr2=0x7f83241f1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:26.870 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.860+0000 7f83237fe640 1 --2- 172.21.15.49:0/2909932318 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f830c004d10 0x7f83241f1b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:26.870 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.860+0000 7f83237fe640 1 -- 172.21.15.49:0/2909932318 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83241510e0 con 0x7f83240c9910 2023-11-06T02:09:26.870 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.860+0000 7f83227fc640 1 --2- 172.21.15.49:0/2909932318 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8324150c60 0x7f832414fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:09:26.870 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.860+0000 7f83237fe640 1 --2- 172.21.15.49:0/2909932318 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83240c9910 0x7f83240c9d90 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f831c002240 tx=0x7f831c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:26.871 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.860+0000 7f830b7fe640 1 -- 172.21.15.49:0/2909932318 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f831c0bc070 con 0x7f83240c9910 2023-11-06T02:09:26.871 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.860+0000 7f830b7fe640 1 -- 172.21.15.49:0/2909932318 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f831c0039a0 con 0x7f83240c9910 2023-11-06T02:09:26.871 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.860+0000 7f832966a640 1 -- 172.21.15.49:0/2909932318 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f832415b4a0 con 0x7f83240c9910 2023-11-06T02:09:26.871 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.860+0000 7f830b7fe640 1 -- 172.21.15.49:0/2909932318 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f831c003b30 con 0x7f83240c9910 2023-11-06T02:09:26.871 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.860+0000 7f832966a640 1 -- 172.21.15.49:0/2909932318 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83241f10b0 con 0x7f83240c9910 2023-11-06T02:09:26.871 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.860+0000 7f832966a640 1 -- 172.21.15.49:0/2909932318 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8324074040 con 0x7f83240c9910 2023-11-06T02:09:26.872 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.860+0000 7f830b7fe640 1 -- 172.21.15.49:0/2909932318 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f831c0b4040 con 0x7f83240c9910 2023-11-06T02:09:26.872 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.860+0000 7f830b7fe640 1 --2- 172.21.15.49:0/2909932318 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8310065a60 0x7f8310067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:26.872 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.860+0000 7f830b7fe640 1 -- 172.21.15.49:0/2909932318 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f831c11d050 con 0x7f83240c9910 2023-11-06T02:09:26.872 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.864+0000 7f8322ffd640 1 --2- 172.21.15.49:0/2909932318 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8310065a60 0x7f8310067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:26.872 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.864+0000 7f8322ffd640 1 --2- 172.21.15.49:0/2909932318 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8310065a60 0x7f8310067f20 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8314007fd0 tx=0x7f8314009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:26.873 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:26.864+0000 7f830b7fe640 1 -- 172.21.15.49:0/2909932318 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f831c0ea9b0 con 0x7f83240c9910 2023-11-06T02:09:27.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.020+0000 7f832966a640 1 -- 172.21.15.49:0/2909932318 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f83241f1840 con 0x7f83240c9910 2023-11-06T02:09:27.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.020+0000 7f830b7fe640 1 -- 172.21.15.49:0/2909932318 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f831c0edfd0 con 0x7f83240c9910 2023-11-06T02:09:27.021 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2023-11-06T02:09:27.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.024+0000 7f832966a640 1 -- 172.21.15.49:0/2909932318 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8310065a60 msgr2=0x7f8310067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:27.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.024+0000 7f832966a640 1 --2- 172.21.15.49:0/2909932318 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8310065a60 0x7f8310067f20 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8314007fd0 tx=0x7f8314009040 comp rx=0 tx=0).stop 2023-11-06T02:09:27.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.024+0000 7f832966a640 1 -- 172.21.15.49:0/2909932318 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83240c9910 msgr2=0x7f83240c9d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:27.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.024+0000 7f832966a640 1 --2- 172.21.15.49:0/2909932318 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83240c9910 0x7f83240c9d90 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f831c002240 tx=0x7f831c009750 comp rx=0 tx=0).stop 2023-11-06T02:09:27.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.024+0000 7f832966a640 1 -- 172.21.15.49:0/2909932318 shutdown_connections 2023-11-06T02:09:27.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.024+0000 7f832966a640 1 --2- 172.21.15.49:0/2909932318 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8310065a60 0x7f8310067f20 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:27.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.024+0000 7f832966a640 1 --2- 172.21.15.49:0/2909932318 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83240c9910 0x7f83240c9d90 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:27.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.024+0000 7f832966a640 1 --2- 172.21.15.49:0/2909932318 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8324150c60 0x7f832414fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:27.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.024+0000 7f832966a640 1 --2- 172.21.15.49:0/2909932318 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f830c004d10 0x7f83241f1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:27.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.024+0000 7f832966a640 1 -- 172.21.15.49:0/2909932318 >> 172.21.15.49:0/2909932318 conn(0x7f832406a370 msgr2=0x7f8324070060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:27.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.024+0000 7f832966a640 1 -- 172.21.15.49:0/2909932318 shutdown_connections 2023-11-06T02:09:27.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.024+0000 7f832966a640 1 -- 172.21.15.49:0/2909932318 wait complete. 2023-11-06T02:09:27.040 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T02:09:27.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.168+0000 7f4083466640 1 Processor -- start 2023-11-06T02:09:27.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.168+0000 7f4083466640 1 -- start start 2023-11-06T02:09:27.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.168+0000 7f4083466640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f407c15a850 0x7f407c0caa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:27.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.168+0000 7f4083466640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f407c0d07b0 con 0x7f407c15e4a0 2023-11-06T02:09:27.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.168+0000 7f4083466640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f407c0cf8e0 con 0x7f407c15e900 2023-11-06T02:09:27.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.168+0000 7f4083466640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f407c15edd0 con 0x7f407c15a850 2023-11-06T02:09:27.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.168+0000 7f4081202640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f407c15a850 0x7f407c0caa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:27.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.168+0000 7f4081202640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f407c15a850 0x7f407c0caa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42122/0 (socket says 172.21.15.49:42122) 2023-11-06T02:09:27.169 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.168+0000 7f4081202640 1 -- 172.21.15.49:0/1283879770 learned_addr learned my addr 172.21.15.49:0/1283879770 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:27.169 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.168+0000 7f4081202640 1 -- 172.21.15.49:0/1283879770 >> v1:172.21.15.150:6789/0 conn(0x7f407c15e900 legacy=0x7f407c069d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:27.169 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.168+0000 7f4081202640 1 -- 172.21.15.49:0/1283879770 >> v1:172.21.15.49:6789/0 conn(0x7f407c15e4a0 legacy=0x7f407c0d00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:27.169 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.168+0000 7f4081202640 1 -- 172.21.15.49:0/1283879770 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f407c15f0b0 con 0x7f407c15a850 2023-11-06T02:09:27.170 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.172+0000 7f4081202640 1 --2- 172.21.15.49:0/1283879770 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f407c15a850 0x7f407c0caa10 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f4078009870 tx=0x7f407801c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36355c7579bd955e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:27.171 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.172+0000 7f4073fff640 1 -- 172.21.15.49:0/1283879770 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40780a95f0 con 0x7f407c15a850 2023-11-06T02:09:27.171 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.172+0000 7f4073fff640 1 -- 172.21.15.49:0/1283879770 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40780a9750 con 0x7f407c15a850 2023-11-06T02:09:27.171 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.172+0000 7f4073fff640 1 -- 172.21.15.49:0/1283879770 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40780a9ae0 con 0x7f407c15a850 2023-11-06T02:09:27.171 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.172+0000 7f4083466640 1 -- 172.21.15.49:0/1283879770 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f407c15a850 msgr2=0x7f407c0caa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:27.172 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.172+0000 7f4083466640 1 --2- 172.21.15.49:0/1283879770 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f407c15a850 0x7f407c0caa10 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f4078009870 tx=0x7f407801c510 comp rx=0 tx=0).stop 2023-11-06T02:09:27.172 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.172+0000 7f4083466640 1 -- 172.21.15.49:0/1283879770 shutdown_connections 2023-11-06T02:09:27.172 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.172+0000 7f4083466640 1 --2- 172.21.15.49:0/1283879770 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f407c15a850 0x7f407c0caa10 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:27.172 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.172+0000 7f4083466640 1 -- 172.21.15.49:0/1283879770 >> 172.21.15.49:0/1283879770 conn(0x7f407c06d4b0 msgr2=0x7f407c06d890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:27.172 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.172+0000 7f4083466640 1 -- 172.21.15.49:0/1283879770 shutdown_connections 2023-11-06T02:09:27.172 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.172+0000 7f4083466640 1 -- 172.21.15.49:0/1283879770 wait complete. 2023-11-06T02:09:27.175 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.172+0000 7f4083466640 1 Processor -- start 2023-11-06T02:09:27.175 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.172+0000 7f4083466640 1 -- start start 2023-11-06T02:09:27.175 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.172+0000 7f4083466640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f407c15a850 0x7f407c0c58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:27.176 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.172+0000 7f4083466640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f407c15e4a0 0x7f407c0c34c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:27.176 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.172+0000 7f4083466640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f407c15e900 0x7f407c158d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:27.176 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.172+0000 7f4083466640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f407c1639b0 con 0x7f407c15a850 2023-11-06T02:09:27.176 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.172+0000 7f4083466640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f407c1666e0 con 0x7f407c15e4a0 2023-11-06T02:09:27.176 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.172+0000 7f4083466640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f407c154f50 con 0x7f407c15e900 2023-11-06T02:09:27.177 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.172+0000 7f4081202640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f407c15a850 0x7f407c0c58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:27.177 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.172+0000 7f4081202640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f407c15a850 0x7f407c0c58c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42136/0 (socket says 172.21.15.49:42136) 2023-11-06T02:09:27.177 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.172+0000 7f4081202640 1 -- 172.21.15.49:0/2505548075 learned_addr learned my addr 172.21.15.49:0/2505548075 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:27.177 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.172+0000 7f4080a01640 1 --2- 172.21.15.49:0/2505548075 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f407c15e4a0 0x7f407c0c34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:27.177 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.172+0000 7f4081a03640 1 --2- 172.21.15.49:0/2505548075 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f407c15e900 0x7f407c158d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:27.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.172+0000 7f4081202640 1 -- 172.21.15.49:0/2505548075 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f407c15e900 msgr2=0x7f407c158d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:27.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.172+0000 7f4081202640 1 --2- 172.21.15.49:0/2505548075 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f407c15e900 0x7f407c158d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:27.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.172+0000 7f4081202640 1 -- 172.21.15.49:0/2505548075 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f407c15e4a0 msgr2=0x7f407c0c34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:27.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.172+0000 7f4081202640 1 --2- 172.21.15.49:0/2505548075 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f407c15e4a0 0x7f407c0c34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:27.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.172+0000 7f4081202640 1 -- 172.21.15.49:0/2505548075 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40780094d0 con 0x7f407c15a850 2023-11-06T02:09:27.179 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.172+0000 7f4080a01640 1 --2- 172.21.15.49:0/2505548075 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f407c15e4a0 0x7f407c0c34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:09:27.179 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.172+0000 7f4081202640 1 --2- 172.21.15.49:0/2505548075 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f407c15a850 0x7f407c0c58c0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f4078019120 tx=0x7f40780192b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:27.179 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.172+0000 7f4071ffb640 1 -- 172.21.15.49:0/2505548075 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40780b1800 con 0x7f407c15a850 2023-11-06T02:09:27.179 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.176+0000 7f4071ffb640 1 -- 172.21.15.49:0/2505548075 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40780b1960 con 0x7f407c15a850 2023-11-06T02:09:27.179 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.176+0000 7f4083466640 1 -- 172.21.15.49:0/2505548075 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f407c0c5ee0 con 0x7f407c15a850 2023-11-06T02:09:27.179 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.176+0000 7f4071ffb640 1 -- 172.21.15.49:0/2505548075 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40780b1cf0 con 0x7f407c15a850 2023-11-06T02:09:27.180 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.176+0000 7f4083466640 1 -- 172.21.15.49:0/2505548075 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f407c163b90 con 0x7f407c15a850 2023-11-06T02:09:27.180 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.176+0000 7f4083466640 1 -- 172.21.15.49:0/2505548075 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f407c067140 con 0x7f407c15a850 2023-11-06T02:09:27.180 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.176+0000 7f4071ffb640 1 -- 172.21.15.49:0/2505548075 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f40780c43f0 con 0x7f407c15a850 2023-11-06T02:09:27.180 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.176+0000 7f4071ffb640 1 --2- 172.21.15.49:0/2505548075 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4060065a60 0x7f4060067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:27.180 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.176+0000 7f4071ffb640 1 -- 172.21.15.49:0/2505548075 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f407811dda0 con 0x7f407c15a850 2023-11-06T02:09:27.181 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.176+0000 7f4080a01640 1 --2- 172.21.15.49:0/2505548075 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4060065a60 0x7f4060067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:27.181 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.176+0000 7f4080a01640 1 --2- 172.21.15.49:0/2505548075 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4060065a60 0x7f4060067f20 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f407c0cf740 tx=0x7f406c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:27.181 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.180+0000 7f4071ffb640 1 -- 172.21.15.49:0/2505548075 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f40780eb5c0 con 0x7f407c15a850 2023-11-06T02:09:27.335 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.336+0000 7f4083466640 1 -- 172.21.15.49:0/2505548075 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f407c06e1c0 con 0x7f407c15a850 2023-11-06T02:09:27.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.336+0000 7f4071ffb640 1 -- 172.21.15.49:0/2505548075 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f40780eebe0 con 0x7f407c15a850 2023-11-06T02:09:27.336 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2023-11-06T02:09:27.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.340+0000 7f4083466640 1 -- 172.21.15.49:0/2505548075 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4060065a60 msgr2=0x7f4060067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:27.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.340+0000 7f4083466640 1 --2- 172.21.15.49:0/2505548075 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4060065a60 0x7f4060067f20 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f407c0cf740 tx=0x7f406c009040 comp rx=0 tx=0).stop 2023-11-06T02:09:27.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.340+0000 7f4083466640 1 -- 172.21.15.49:0/2505548075 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f407c15a850 msgr2=0x7f407c0c58c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:27.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.340+0000 7f4083466640 1 --2- 172.21.15.49:0/2505548075 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f407c15a850 0x7f407c0c58c0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f4078019120 tx=0x7f40780192b0 comp rx=0 tx=0).stop 2023-11-06T02:09:27.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.340+0000 7f4083466640 1 -- 172.21.15.49:0/2505548075 shutdown_connections 2023-11-06T02:09:27.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.340+0000 7f4083466640 1 --2- 172.21.15.49:0/2505548075 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4060065a60 0x7f4060067f20 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:27.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.340+0000 7f4083466640 1 --2- 172.21.15.49:0/2505548075 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f407c15e900 0x7f407c158d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:27.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.340+0000 7f4083466640 1 --2- 172.21.15.49:0/2505548075 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f407c15e4a0 0x7f407c0c34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:27.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.340+0000 7f4083466640 1 --2- 172.21.15.49:0/2505548075 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f407c15a850 0x7f407c0c58c0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:27.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.340+0000 7f4083466640 1 -- 172.21.15.49:0/2505548075 >> 172.21.15.49:0/2505548075 conn(0x7f407c06d4b0 msgr2=0x7f407c0c7370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:27.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.340+0000 7f4083466640 1 -- 172.21.15.49:0/2505548075 shutdown_connections 2023-11-06T02:09:27.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.340+0000 7f4083466640 1 -- 172.21.15.49:0/2505548075 wait complete. 2023-11-06T02:09:27.355 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T02:09:27.477 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.480+0000 7fa01d2d3640 1 Processor -- start 2023-11-06T02:09:27.478 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.480+0000 7fa01d2d3640 1 -- start start 2023-11-06T02:09:27.479 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.480+0000 7fa01d2d3640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa01815ab50 0x7fa0180c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:27.479 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.480+0000 7fa01d2d3640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa0180b0790 0x7fa0180c2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:27.480 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.480+0000 7fa01d2d3640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0180cab80 con 0x7fa0180cb010 2023-11-06T02:09:27.480 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.480+0000 7fa01d2d3640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa01815af30 con 0x7fa0180b0790 2023-11-06T02:09:27.480 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.480+0000 7fa01d2d3640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0180cb5b0 con 0x7fa01815ab50 2023-11-06T02:09:27.480 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.480+0000 7fa0177fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa0180b0790 0x7fa0180c2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:27.480 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.480+0000 7fa0177fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa0180b0790 0x7fa0180c2710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42138/0 (socket says 172.21.15.49:42138) 2023-11-06T02:09:27.481 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.480+0000 7fa0177fe640 1 -- 172.21.15.49:0/3392871061 learned_addr learned my addr 172.21.15.49:0/3392871061 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:27.481 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.480+0000 7fa016ffd640 1 --2- 172.21.15.49:0/3392871061 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa01815ab50 0x7fa0180c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:27.481 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.480+0000 7fa0177fe640 1 -- 172.21.15.49:0/3392871061 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa01815ab50 msgr2=0x7fa0180c9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:27.481 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.480+0000 7fa0177fe640 1 --2- 172.21.15.49:0/3392871061 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa01815ab50 0x7fa0180c9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:27.481 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.480+0000 7fa0177fe640 1 -- 172.21.15.49:0/3392871061 >> v1:172.21.15.49:6789/0 conn(0x7fa0180cb010 legacy=0x7fa0180b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:27.482 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.480+0000 7fa0177fe640 1 -- 172.21.15.49:0/3392871061 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0180cb890 con 0x7fa0180b0790 2023-11-06T02:09:27.483 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.480+0000 7fa0177fe640 1 --2- 172.21.15.49:0/3392871061 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0180b0790 0x7fa0180c2710 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fa010009e50 tx=0x7fa01001c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=334c3972556eea84 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:27.483 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.484+0000 7fa015ffb640 1 -- 172.21.15.49:0/3392871061 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0100a58d0 con 0x7fa0180b0790 2023-11-06T02:09:27.483 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.484+0000 7fa015ffb640 1 -- 172.21.15.49:0/3392871061 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0100a5a30 con 0x7fa0180b0790 2023-11-06T02:09:27.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.484+0000 7fa015ffb640 1 -- 172.21.15.49:0/3392871061 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0100a5dc0 con 0x7fa0180b0790 2023-11-06T02:09:27.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.484+0000 7fa01d2d3640 1 -- 172.21.15.49:0/3392871061 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0180b0790 msgr2=0x7fa0180c2710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:27.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.484+0000 7fa01d2d3640 1 --2- 172.21.15.49:0/3392871061 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0180b0790 0x7fa0180c2710 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fa010009e50 tx=0x7fa01001c9e0 comp rx=0 tx=0).stop 2023-11-06T02:09:27.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.484+0000 7fa01d2d3640 1 -- 172.21.15.49:0/3392871061 shutdown_connections 2023-11-06T02:09:27.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.484+0000 7fa01d2d3640 1 --2- 172.21.15.49:0/3392871061 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0180b0790 0x7fa0180c2710 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:27.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.484+0000 7fa01d2d3640 1 --2- 172.21.15.49:0/3392871061 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa01815ab50 0x7fa0180c9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:27.485 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.484+0000 7fa01d2d3640 1 -- 172.21.15.49:0/3392871061 >> 172.21.15.49:0/3392871061 conn(0x7fa01806c0a0 msgr2=0x7fa01806c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:27.485 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.484+0000 7fa01d2d3640 1 -- 172.21.15.49:0/3392871061 shutdown_connections 2023-11-06T02:09:27.485 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.484+0000 7fa01d2d3640 1 -- 172.21.15.49:0/3392871061 wait complete. 2023-11-06T02:09:27.488 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.484+0000 7fa01d2d3640 1 Processor -- start 2023-11-06T02:09:27.488 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.484+0000 7fa01d2d3640 1 -- start start 2023-11-06T02:09:27.488 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.484+0000 7fa01d2d3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0180b0790 0x7fa0180c81b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:27.488 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.484+0000 7fa01d2d3640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa0180cb010 0x7fa0180c6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:27.488 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.484+0000 7fa01d2d3640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa01815ab50 0x7fa018149940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:27.489 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.484+0000 7fa01d2d3640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa018158a40 con 0x7fa0180b0790 2023-11-06T02:09:27.489 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.484+0000 7fa01d2d3640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0180c5870 con 0x7fa01815ab50 2023-11-06T02:09:27.489 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.484+0000 7fa01d2d3640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0180c5b50 con 0x7fa0180cb010 2023-11-06T02:09:27.489 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.484+0000 7fa016ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0180b0790 0x7fa0180c81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:27.489 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.484+0000 7fa016ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0180b0790 0x7fa0180c81b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42152/0 (socket says 172.21.15.49:42152) 2023-11-06T02:09:27.490 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.484+0000 7fa016ffd640 1 -- 172.21.15.49:0/4083334279 learned_addr learned my addr 172.21.15.49:0/4083334279 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:27.490 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.484+0000 7fa0167fc640 1 --2- 172.21.15.49:0/4083334279 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa0180cb010 0x7fa0180c6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:27.490 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.484+0000 7fa0177fe640 1 --2- 172.21.15.49:0/4083334279 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa01815ab50 0x7fa018149940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:27.490 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.484+0000 7fa016ffd640 1 -- 172.21.15.49:0/4083334279 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa0180cb010 msgr2=0x7fa0180c6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:27.490 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.484+0000 7fa016ffd640 1 --2- 172.21.15.49:0/4083334279 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa0180cb010 0x7fa0180c6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:27.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.484+0000 7fa016ffd640 1 -- 172.21.15.49:0/4083334279 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa01815ab50 msgr2=0x7fa018149940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:27.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.484+0000 7fa016ffd640 1 --2- 172.21.15.49:0/4083334279 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa01815ab50 0x7fa018149940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:27.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.484+0000 7fa016ffd640 1 -- 172.21.15.49:0/4083334279 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa010009b30 con 0x7fa0180b0790 2023-11-06T02:09:27.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.488+0000 7fa016ffd640 1 --2- 172.21.15.49:0/4083334279 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0180b0790 0x7fa0180c81b0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fa00800d7d0 tx=0x7fa00800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:27.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.488+0000 7f9fff7fe640 1 -- 172.21.15.49:0/4083334279 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa00800b6c0 con 0x7fa0180b0790 2023-11-06T02:09:27.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.488+0000 7f9fff7fe640 1 -- 172.21.15.49:0/4083334279 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa00800b820 con 0x7fa0180b0790 2023-11-06T02:09:27.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.488+0000 7fa01d2d3640 1 -- 172.21.15.49:0/4083334279 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0180c5e30 con 0x7fa0180b0790 2023-11-06T02:09:27.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.488+0000 7f9fff7fe640 1 -- 172.21.15.49:0/4083334279 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa00800bbd0 con 0x7fa0180b0790 2023-11-06T02:09:27.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.488+0000 7fa01d2d3640 1 -- 172.21.15.49:0/4083334279 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0180c6b70 con 0x7fa0180b0790 2023-11-06T02:09:27.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.488+0000 7fa01d2d3640 1 -- 172.21.15.49:0/4083334279 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0180b0de0 con 0x7fa0180b0790 2023-11-06T02:09:27.493 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.488+0000 7f9fff7fe640 1 -- 172.21.15.49:0/4083334279 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fa00800bd30 con 0x7fa0180b0790 2023-11-06T02:09:27.493 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.488+0000 7f9fff7fe640 1 --2- 172.21.15.49:0/4083334279 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa004065a60 0x7fa004067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:27.493 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.488+0000 7f9fff7fe640 1 -- 172.21.15.49:0/4083334279 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fa0080838a0 con 0x7fa0180b0790 2023-11-06T02:09:27.493 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.488+0000 7fa0167fc640 1 --2- 172.21.15.49:0/4083334279 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa004065a60 0x7fa004067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:27.494 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.488+0000 7fa0167fc640 1 --2- 172.21.15.49:0/4083334279 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa004065a60 0x7fa004067f20 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa018154090 tx=0x7fa00c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:27.494 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.492+0000 7f9fff7fe640 1 -- 172.21.15.49:0/4083334279 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fa008051030 con 0x7fa0180b0790 2023-11-06T02:09:27.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.680+0000 7fa01d2d3640 1 -- 172.21.15.49:0/4083334279 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa018074040 con 0x7fa0180b0790 2023-11-06T02:09:27.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.684+0000 7f9fff7fe640 1 -- 172.21.15.49:0/4083334279 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4648 (secure 0 0 0) 0x7fa008054650 con 0x7fa0180b0790 2023-11-06T02:09:27.682 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:09:27.683 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6834/1377586971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1377586971},{"type":"v1","addr":"172.21.15.49:6834","nonce":1377586971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4288,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2620584959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2620584959},{"type":"v1","addr":"172.21.15.150:6835","nonce":2620584959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4291,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/4272191418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":4272191418},{"type":"v1","addr":"172.21.15.150:6837","nonce":4272191418}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T02:08:30.592185+0000","modified":"2023-11-06T02:08:33.245446+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4293},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4293":{"gid":4293,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.49:6835/4280448297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6833","nonce":4280448297},{"type":"v1","addr":"172.21.15.49:6835","nonce":4280448297}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-06T02:09:27.683 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 7 2023-11-06T02:09:27.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.684+0000 7fa01d2d3640 1 -- 172.21.15.49:0/4083334279 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa004065a60 msgr2=0x7fa004067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:27.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.684+0000 7fa01d2d3640 1 --2- 172.21.15.49:0/4083334279 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa004065a60 0x7fa004067f20 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa018154090 tx=0x7fa00c009040 comp rx=0 tx=0).stop 2023-11-06T02:09:27.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.684+0000 7fa01d2d3640 1 -- 172.21.15.49:0/4083334279 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0180b0790 msgr2=0x7fa0180c81b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:27.686 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.684+0000 7fa01d2d3640 1 --2- 172.21.15.49:0/4083334279 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0180b0790 0x7fa0180c81b0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fa00800d7d0 tx=0x7fa00800dc90 comp rx=0 tx=0).stop 2023-11-06T02:09:27.686 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.684+0000 7fa01d2d3640 1 -- 172.21.15.49:0/4083334279 shutdown_connections 2023-11-06T02:09:27.686 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.684+0000 7fa01d2d3640 1 --2- 172.21.15.49:0/4083334279 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa004065a60 0x7fa004067f20 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:27.686 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.684+0000 7fa01d2d3640 1 --2- 172.21.15.49:0/4083334279 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa01815ab50 0x7fa018149940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:27.686 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.684+0000 7fa01d2d3640 1 --2- 172.21.15.49:0/4083334279 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa0180cb010 0x7fa0180c6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:27.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.684+0000 7fa01d2d3640 1 --2- 172.21.15.49:0/4083334279 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0180b0790 0x7fa0180c81b0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:27.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.684+0000 7fa01d2d3640 1 -- 172.21.15.49:0/4083334279 >> 172.21.15.49:0/4083334279 conn(0x7fa01806c0a0 msgr2=0x7fa018159ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:27.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.684+0000 7fa01d2d3640 1 -- 172.21.15.49:0/4083334279 shutdown_connections 2023-11-06T02:09:27.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.684+0000 7fa01d2d3640 1 -- 172.21.15.49:0/4083334279 wait complete. 2023-11-06T02:09:27.701 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T02:09:27.826 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.828+0000 7fb72c1ad640 1 Processor -- start 2023-11-06T02:09:27.826 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.828+0000 7fb72c1ad640 1 -- start start 2023-11-06T02:09:27.827 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.828+0000 7fb72c1ad640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb7240b0d80 0x7fb72406b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:27.827 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.828+0000 7fb72c1ad640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7240b1970 0x7fb7240cb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:27.828 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.828+0000 7fb72c1ad640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb72406bce0 con 0x7fb7240b1420 2023-11-06T02:09:27.828 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.828+0000 7fb72c1ad640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7240cffb0 con 0x7fb7240b0d80 2023-11-06T02:09:27.828 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.828+0000 7fb72c1ad640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7240cf710 con 0x7fb7240b1970 2023-11-06T02:09:27.828 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.828+0000 7fb729f49640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb7240b0d80 0x7fb72406b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:27.828 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.828+0000 7fb729f49640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb7240b0d80 0x7fb72406b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42158/0 (socket says 172.21.15.49:42158) 2023-11-06T02:09:27.829 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.828+0000 7fb729f49640 1 -- 172.21.15.49:0/1044527141 learned_addr learned my addr 172.21.15.49:0/1044527141 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:27.829 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.828+0000 7fb72a74a640 1 --2- 172.21.15.49:0/1044527141 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7240b1970 0x7fb7240cb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:27.829 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.828+0000 7fb729f49640 1 -- 172.21.15.49:0/1044527141 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7240b1970 msgr2=0x7fb7240cb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:27.829 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.828+0000 7fb729f49640 1 --2- 172.21.15.49:0/1044527141 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7240b1970 0x7fb7240cb280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:27.829 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.828+0000 7fb729f49640 1 -- 172.21.15.49:0/1044527141 >> v1:172.21.15.49:6789/0 conn(0x7fb7240b1420 legacy=0x7fb72406b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:27.830 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.828+0000 7fb729f49640 1 -- 172.21.15.49:0/1044527141 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7240cf9f0 con 0x7fb7240b0d80 2023-11-06T02:09:27.830 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.828+0000 7fb72a74a640 1 --2- 172.21.15.49:0/1044527141 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7240b1970 0x7fb7240cb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:09:27.831 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.832+0000 7fb729f49640 1 --2- 172.21.15.49:0/1044527141 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7240b0d80 0x7fb72406b090 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fb720006fd0 tx=0x7fb72001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a871b8d36276619c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:27.831 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.832+0000 7fb728f47640 1 -- 172.21.15.49:0/1044527141 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7200aa670 con 0x7fb7240b0d80 2023-11-06T02:09:27.832 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.832+0000 7fb728f47640 1 -- 172.21.15.49:0/1044527141 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7200aa7d0 con 0x7fb7240b0d80 2023-11-06T02:09:27.832 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.832+0000 7fb728f47640 1 -- 172.21.15.49:0/1044527141 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7200aab60 con 0x7fb7240b0d80 2023-11-06T02:09:27.832 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.832+0000 7fb72c1ad640 1 -- 172.21.15.49:0/1044527141 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7240b0d80 msgr2=0x7fb72406b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:27.832 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.832+0000 7fb72c1ad640 1 --2- 172.21.15.49:0/1044527141 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7240b0d80 0x7fb72406b090 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fb720006fd0 tx=0x7fb72001c4f0 comp rx=0 tx=0).stop 2023-11-06T02:09:27.832 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.832+0000 7fb72c1ad640 1 -- 172.21.15.49:0/1044527141 shutdown_connections 2023-11-06T02:09:27.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.832+0000 7fb72c1ad640 1 --2- 172.21.15.49:0/1044527141 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7240b1970 0x7fb7240cb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:27.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.832+0000 7fb72c1ad640 1 --2- 172.21.15.49:0/1044527141 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7240b0d80 0x7fb72406b090 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:27.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.832+0000 7fb72c1ad640 1 -- 172.21.15.49:0/1044527141 >> 172.21.15.49:0/1044527141 conn(0x7fb7240d0890 msgr2=0x7fb7240d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:27.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.832+0000 7fb72c1ad640 1 -- 172.21.15.49:0/1044527141 shutdown_connections 2023-11-06T02:09:27.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.832+0000 7fb72c1ad640 1 -- 172.21.15.49:0/1044527141 wait complete. 2023-11-06T02:09:27.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.832+0000 7fb72c1ad640 1 Processor -- start 2023-11-06T02:09:27.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.832+0000 7fb72c1ad640 1 -- start start 2023-11-06T02:09:27.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.832+0000 7fb72c1ad640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7240b0d80 0x7fb72415eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:27.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.832+0000 7fb72c1ad640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7240b1420 0x7fb7240c90a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:27.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.832+0000 7fb72c1ad640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb7240b1970 0x7fb7241566a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:27.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.832+0000 7fb72c1ad640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7240c7d80 con 0x7fb7240b0d80 2023-11-06T02:09:27.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.832+0000 7fb72c1ad640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7240c8060 con 0x7fb7240b1970 2023-11-06T02:09:27.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.832+0000 7fb72c1ad640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7240c8340 con 0x7fb7240b1420 2023-11-06T02:09:27.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.832+0000 7fb729f49640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7240b0d80 0x7fb72415eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:27.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.832+0000 7fb729f49640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7240b0d80 0x7fb72415eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42170/0 (socket says 172.21.15.49:42170) 2023-11-06T02:09:27.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.832+0000 7fb729f49640 1 -- 172.21.15.49:0/297931768 learned_addr learned my addr 172.21.15.49:0/297931768 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:27.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.832+0000 7fb72a74a640 1 --2- 172.21.15.49:0/297931768 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb7240b1970 0x7fb7241566a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:27.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.832+0000 7fb729748640 1 --2- 172.21.15.49:0/297931768 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7240b1420 0x7fb7240c90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:27.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.832+0000 7fb729f49640 1 -- 172.21.15.49:0/297931768 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7240b1420 msgr2=0x7fb7240c90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:27.839 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.832+0000 7fb729f49640 1 --2- 172.21.15.49:0/297931768 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7240b1420 0x7fb7240c90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:27.839 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.832+0000 7fb729f49640 1 -- 172.21.15.49:0/297931768 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb7240b1970 msgr2=0x7fb7241566a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:27.839 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.832+0000 7fb729f49640 1 --2- 172.21.15.49:0/297931768 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb7240b1970 0x7fb7241566a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:27.839 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.832+0000 7fb729f49640 1 -- 172.21.15.49:0/297931768 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb720009480 con 0x7fb7240b0d80 2023-11-06T02:09:27.839 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.832+0000 7fb729748640 1 --2- 172.21.15.49:0/297931768 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7240b1420 0x7fb7240c90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:09:27.840 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.832+0000 7fb729f49640 1 --2- 172.21.15.49:0/297931768 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7240b0d80 0x7fb72415eb10 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fb720000c10 tx=0x7fb720019290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:27.840 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.836+0000 7fb71a7fc640 1 -- 172.21.15.49:0/297931768 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7200aab40 con 0x7fb7240b0d80 2023-11-06T02:09:27.840 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.836+0000 7fb71a7fc640 1 -- 172.21.15.49:0/297931768 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7200aacc0 con 0x7fb7240b0d80 2023-11-06T02:09:27.840 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.836+0000 7fb72c1ad640 1 -- 172.21.15.49:0/297931768 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7240c8620 con 0x7fb7240b0d80 2023-11-06T02:09:27.840 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.836+0000 7fb71a7fc640 1 -- 172.21.15.49:0/297931768 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb720003860 con 0x7fb7240b0d80 2023-11-06T02:09:27.840 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.836+0000 7fb72c1ad640 1 -- 172.21.15.49:0/297931768 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7240c7960 con 0x7fb7240b0d80 2023-11-06T02:09:27.841 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.836+0000 7fb72c1ad640 1 -- 172.21.15.49:0/297931768 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7240c7b50 con 0x7fb7240b0d80 2023-11-06T02:09:27.841 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.836+0000 7fb71a7fc640 1 -- 172.21.15.49:0/297931768 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fb7200039c0 con 0x7fb7240b0d80 2023-11-06T02:09:27.841 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.836+0000 7fb71a7fc640 1 --2- 172.21.15.49:0/297931768 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb708065a90 0x7fb708067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:27.841 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.836+0000 7fb71a7fc640 1 -- 172.21.15.49:0/297931768 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fb72011dfd0 con 0x7fb7240b0d80 2023-11-06T02:09:27.841 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.836+0000 7fb729748640 1 --2- 172.21.15.49:0/297931768 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb708065a90 0x7fb708067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:27.842 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.836+0000 7fb729748640 1 --2- 172.21.15.49:0/297931768 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb708065a90 0x7fb708067f50 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb7240c72f0 tx=0x7fb714009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:27.842 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.840+0000 7fb71a7fc640 1 -- 172.21.15.49:0/297931768 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fb7200eba10 con 0x7fb7240b0d80 2023-11-06T02:09:27.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:27.996+0000 7fb72c1ad640 1 -- 172.21.15.49:0/297931768 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb724074040 con 0x7fb7240b0d80 2023-11-06T02:09:27.999 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.000+0000 7fb71a7fc640 1 -- 172.21.15.49:0/297931768 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17906 (secure 0 0 0) 0x7fb7200c3090 con 0x7fb7240b0d80 2023-11-06T02:09:28.000 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:09:28.000 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":25,"fsid":"1a8b035c-6ba4-499a-b876-aea47fd9c0d4","created":"2023-11-06T02:08:06.440857+0000","modified":"2023-11-06T02:09:13.195118+0000","last_up_change":"2023-11-06T02:08:17.982372+0000","last_in_change":"2023-11-06T02:08:08.906631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T02:08:19.252222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T02:08:24.776211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-06T02:08:28.936178+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-06T02:08:29.563468+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ad55af8-0571-4e6a-8722-b5d429b7496d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6809","nonce":38116},{"type":"v1","addr":"172.21.15.49:6811","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6813","nonce":38116},{"type":"v1","addr":"172.21.15.49:6815","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6821","nonce":38116},{"type":"v1","addr":"172.21.15.49:6823","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6817","nonce":38116},{"type":"v1","addr":"172.21.15.49:6819","nonce":38116}]},"public_addr":"172.21.15.49:6811/38116","cluster_addr":"172.21.15.49:6815/38116","heartbeat_back_addr":"172.21.15.49:6823/38116","heartbeat_front_addr":"172.21.15.49:6819/38116","state":["exists","up"]},{"osd":1,"uuid":"c8b31f85-a757-43ed-9419-351acc46611b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":38113},{"type":"v1","addr":"172.21.15.49:6810","nonce":38113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":38113},{"type":"v1","addr":"172.21.15.49:6814","nonce":38113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":38113},{"type":"v1","addr":"172.21.15.49:6822","nonce":38113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":38113},{"type":"v1","addr":"172.21.15.49:6818","nonce":38113}]},"public_addr":"172.21.15.49:6810/38113","cluster_addr":"172.21.15.49:6814/38113","heartbeat_back_addr":"172.21.15.49:6822/38113","heartbeat_front_addr":"172.21.15.49:6818/38113","state":["exists","up"]},{"osd":2,"uuid":"555b3bf3-0a70-4b91-8efd-e4204b5fc26e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":38114},{"type":"v1","addr":"172.21.15.49:6801","nonce":38114}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":38114},{"type":"v1","addr":"172.21.15.49:6803","nonce":38114}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":38114},{"type":"v1","addr":"172.21.15.49:6807","nonce":38114}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":38114},{"type":"v1","addr":"172.21.15.49:6805","nonce":38114}]},"public_addr":"172.21.15.49:6801/38114","cluster_addr":"172.21.15.49:6803/38114","heartbeat_back_addr":"172.21.15.49:6807/38114","heartbeat_front_addr":"172.21.15.49:6805/38114","state":["exists","up"]},{"osd":3,"uuid":"8d4e5d77-03ef-4b9b-bfcb-5ae93a46795c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":38115},{"type":"v1","addr":"172.21.15.49:6825","nonce":38115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":38115},{"type":"v1","addr":"172.21.15.49:6827","nonce":38115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":38115},{"type":"v1","addr":"172.21.15.49:6831","nonce":38115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":38115},{"type":"v1","addr":"172.21.15.49:6829","nonce":38115}]},"public_addr":"172.21.15.49:6825/38115","cluster_addr":"172.21.15.49:6827/38115","heartbeat_back_addr":"172.21.15.49:6831/38115","heartbeat_front_addr":"172.21.15.49:6829/38115","state":["exists","up"]},{"osd":4,"uuid":"dba03ad4-68a3-4cf3-9218-ac611f02eacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":38417},{"type":"v1","addr":"172.21.15.150:6809","nonce":38417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":38417},{"type":"v1","addr":"172.21.15.150:6811","nonce":38417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":38417},{"type":"v1","addr":"172.21.15.150:6815","nonce":38417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":38417},{"type":"v1","addr":"172.21.15.150:6813","nonce":38417}]},"public_addr":"172.21.15.150:6809/38417","cluster_addr":"172.21.15.150:6811/38417","heartbeat_back_addr":"172.21.15.150:6815/38417","heartbeat_front_addr":"172.21.15.150:6813/38417","state":["exists","up"]},{"osd":5,"uuid":"0df9c303-c255-4640-8e91-01f1c3efe18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":38418},{"type":"v1","addr":"172.21.15.150:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":38418},{"type":"v1","addr":"172.21.15.150:6803","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":38418},{"type":"v1","addr":"172.21.15.150:6807","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":38418},{"type":"v1","addr":"172.21.15.150:6805","nonce":38418}]},"public_addr":"172.21.15.150:6801/38418","cluster_addr":"172.21.15.150:6803/38418","heartbeat_back_addr":"172.21.15.150:6807/38418","heartbeat_front_addr":"172.21.15.150:6805/38418","state":["exists","up"]},{"osd":6,"uuid":"e59fa189-4b70-4c06-9722-c02b90828bf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":38428},{"type":"v1","addr":"172.21.15.150:6825","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":38428},{"type":"v1","addr":"172.21.15.150:6827","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":38428},{"type":"v1","addr":"172.21.15.150:6831","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":38428},{"type":"v1","addr":"172.21.15.150:6829","nonce":38428}]},"public_addr":"172.21.15.150:6825/38428","cluster_addr":"172.21.15.150:6827/38428","heartbeat_back_addr":"172.21.15.150:6831/38428","heartbeat_front_addr":"172.21.15.150:6829/38428","state":["exists","up"]},{"osd":7,"uuid":"a64f57e1-ef98-46bd-b304-385759e95898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":38421},{"type":"v1","addr":"172.21.15.150:6817","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":38421},{"type":"v1","addr":"172.21.15.150:6819","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":38421},{"type":"v1","addr":"172.21.15.150:6823","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":38421},{"type":"v1","addr":"172.21.15.150:6821","nonce":38421}]},"public_addr":"172.21.15.150:6817/38421","cluster_addr":"172.21.15.150:6819/38421","heartbeat_back_addr":"172.21.15.150:6823/38421","heartbeat_front_addr":"172.21.15.150:6821/38421","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.438552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.413280+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.296466+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.424966+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.276953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.230984+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.410235+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.480084+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.14","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T02:09:28.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.004+0000 7fb72c1ad640 1 -- 172.21.15.49:0/297931768 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb708065a90 msgr2=0x7fb708067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:28.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.004+0000 7fb72c1ad640 1 --2- 172.21.15.49:0/297931768 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb708065a90 0x7fb708067f50 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb7240c72f0 tx=0x7fb714009040 comp rx=0 tx=0).stop 2023-11-06T02:09:28.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.004+0000 7fb72c1ad640 1 -- 172.21.15.49:0/297931768 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7240b0d80 msgr2=0x7fb72415eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:28.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.004+0000 7fb72c1ad640 1 --2- 172.21.15.49:0/297931768 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7240b0d80 0x7fb72415eb10 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fb720000c10 tx=0x7fb720019290 comp rx=0 tx=0).stop 2023-11-06T02:09:28.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.004+0000 7fb72c1ad640 1 -- 172.21.15.49:0/297931768 shutdown_connections 2023-11-06T02:09:28.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.004+0000 7fb72c1ad640 1 --2- 172.21.15.49:0/297931768 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb708065a90 0x7fb708067f50 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:28.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.004+0000 7fb72c1ad640 1 --2- 172.21.15.49:0/297931768 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb7240b1970 0x7fb7241566a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:28.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.004+0000 7fb72c1ad640 1 --2- 172.21.15.49:0/297931768 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7240b1420 0x7fb7240c90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:28.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.004+0000 7fb72c1ad640 1 --2- 172.21.15.49:0/297931768 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7240b0d80 0x7fb72415eb10 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:28.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.004+0000 7fb72c1ad640 1 -- 172.21.15.49:0/297931768 >> 172.21.15.49:0/297931768 conn(0x7fb7240d0890 msgr2=0x7fb72406f710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:28.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.004+0000 7fb72c1ad640 1 -- 172.21.15.49:0/297931768 shutdown_connections 2023-11-06T02:09:28.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.004+0000 7fb72c1ad640 1 -- 172.21.15.49:0/297931768 wait complete. 2023-11-06T02:09:28.019 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-06T02:09:28.020 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T02:09:28.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.148+0000 7f5e33cd5640 1 Processor -- start 2023-11-06T02:09:28.146 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.148+0000 7f5e33cd5640 1 -- start start 2023-11-06T02:09:28.146 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.148+0000 7f5e33cd5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5e2c15ab50 0x7f5e2c0c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:28.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.148+0000 7f5e33cd5640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e2c0cab80 con 0x7f5e2c0cb010 2023-11-06T02:09:28.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.148+0000 7f5e33cd5640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e2c15af30 con 0x7f5e2c0b0790 2023-11-06T02:09:28.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.148+0000 7f5e33cd5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e2c0cb5b0 con 0x7f5e2c15ab50 2023-11-06T02:09:28.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.148+0000 7f5e31270640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f5e2c0cb010 0x7f5e2c0b0080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:35580/0 (socket says 172.21.15.49:35580) 2023-11-06T02:09:28.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.148+0000 7f5e31270640 1 -- 172.21.15.49:0/2342270781 learned_addr learned my addr 172.21.15.49:0/2342270781 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:09:28.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.148+0000 7f5e31a71640 1 --2- 172.21.15.49:0/2342270781 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5e2c15ab50 0x7f5e2c0c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:28.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.148+0000 7f5e30a6f640 1 -- 172.21.15.49:0/2342270781 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3775852239 0 0) 0x7f5e2c0cab80 con 0x7f5e2c0cb010 2023-11-06T02:09:28.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.148+0000 7f5e30a6f640 1 -- 172.21.15.49:0/2342270781 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e18003530 con 0x7f5e2c0cb010 2023-11-06T02:09:28.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.148+0000 7f5e30a6f640 1 -- 172.21.15.49:0/2342270781 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3016514030 0 0) 0x7f5e18003530 con 0x7f5e2c0cb010 2023-11-06T02:09:28.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.148+0000 7f5e30a6f640 1 -- 172.21.15.49:0/2342270781 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5e2c15ab50 msgr2=0x7f5e2c0c9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:28.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.148+0000 7f5e30a6f640 1 --2- 172.21.15.49:0/2342270781 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5e2c15ab50 0x7f5e2c0c9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:28.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.148+0000 7f5e30a6f640 1 -- 172.21.15.49:0/2342270781 >> v1:172.21.15.150:6789/0 conn(0x7f5e2c0b0790 legacy=0x7f5e2c0c2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:28.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.148+0000 7f5e30a6f640 1 -- 172.21.15.49:0/2342270781 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e2c0cb890 con 0x7f5e2c0cb010 2023-11-06T02:09:28.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.148+0000 7f5e30a6f640 1 -- 172.21.15.49:0/2342270781 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7f5e28001680 con 0x7f5e2c0cb010 2023-11-06T02:09:28.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.148+0000 7f5e30a6f640 1 -- 172.21.15.49:0/2342270781 >> v1:172.21.15.49:6789/0 conn(0x7f5e2c0cb010 legacy=0x7f5e2c0b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:28.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.148+0000 7f5e30a6f640 1 --2- 172.21.15.49:0/2342270781 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e180039b0 0x7f5e18003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:28.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.148+0000 7f5e30a6f640 1 --2- 172.21.15.49:0/2342270781 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e180047d0 0x7f5e18004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:28.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.148+0000 7f5e30a6f640 1 --2- 172.21.15.49:0/2342270781 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5e2c15ab50 0x7f5e18005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:28.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.148+0000 7f5e30a6f640 1 -- 172.21.15.49:0/2342270781 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e2c0cb5b0 con 0x7f5e180039b0 2023-11-06T02:09:28.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.148+0000 7f5e30a6f640 1 -- 172.21.15.49:0/2342270781 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e1800ba40 con 0x7f5e180047d0 2023-11-06T02:09:28.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.148+0000 7f5e30a6f640 1 -- 172.21.15.49:0/2342270781 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e1800bc00 con 0x7f5e2c15ab50 2023-11-06T02:09:28.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.148+0000 7f5e31a71640 1 --2- 172.21.15.49:0/2342270781 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e180039b0 0x7f5e18003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:28.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.148+0000 7f5e31270640 1 --2- 172.21.15.49:0/2342270781 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e180047d0 0x7f5e18004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:28.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.148+0000 7f5e32272640 1 --2- 172.21.15.49:0/2342270781 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5e2c15ab50 0x7f5e18005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:28.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.148+0000 7f5e31a71640 1 -- 172.21.15.49:0/2342270781 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5e2c15ab50 msgr2=0x7f5e18005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:28.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.148+0000 7f5e31a71640 1 --2- 172.21.15.49:0/2342270781 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5e2c15ab50 0x7f5e18005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:28.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.148+0000 7f5e31a71640 1 -- 172.21.15.49:0/2342270781 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e180047d0 msgr2=0x7f5e18004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:28.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.148+0000 7f5e31a71640 1 --2- 172.21.15.49:0/2342270781 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e180047d0 0x7f5e18004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:28.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.148+0000 7f5e31a71640 1 -- 172.21.15.49:0/2342270781 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e18003530 con 0x7f5e180039b0 2023-11-06T02:09:28.157 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.152+0000 7f5e31a71640 1 --2- 172.21.15.49:0/2342270781 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e180039b0 0x7f5e18003dd0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c009a90 tx=0x7f5e1c01c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eb3fb49599f9dab server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:28.157 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.152+0000 7f5e30a6f640 1 -- 172.21.15.49:0/2342270781 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e1c0059a0 con 0x7f5e180039b0 2023-11-06T02:09:28.158 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.152+0000 7f5e30a6f640 1 -- 172.21.15.49:0/2342270781 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e1c005b00 con 0x7f5e180039b0 2023-11-06T02:09:28.158 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.152+0000 7f5e30a6f640 1 -- 172.21.15.49:0/2342270781 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e1c006030 con 0x7f5e180039b0 2023-11-06T02:09:28.158 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.152+0000 7f5e33cd5640 1 -- 172.21.15.49:0/2342270781 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e180039b0 msgr2=0x7f5e18003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:28.158 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.152+0000 7f5e33cd5640 1 --2- 172.21.15.49:0/2342270781 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e180039b0 0x7f5e18003dd0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c009a90 tx=0x7f5e1c01c9c0 comp rx=0 tx=0).stop 2023-11-06T02:09:28.158 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.152+0000 7f5e32272640 1 -- 172.21.15.49:0/2342270781 reap_dead start 2023-11-06T02:09:28.159 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.152+0000 7f5e33cd5640 1 -- 172.21.15.49:0/2342270781 shutdown_connections 2023-11-06T02:09:28.159 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.152+0000 7f5e33cd5640 1 -- 172.21.15.49:0/2342270781 >> 172.21.15.49:0/2342270781 conn(0x7f5e2c06c0a0 msgr2=0x7f5e2c06c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:28.159 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.152+0000 7f5e33cd5640 1 -- 172.21.15.49:0/2342270781 shutdown_connections 2023-11-06T02:09:28.159 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.152+0000 7f5e33cd5640 1 -- 172.21.15.49:0/2342270781 wait complete. 2023-11-06T02:09:28.159 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.152+0000 7f5e33cd5640 1 Processor -- start 2023-11-06T02:09:28.160 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.152+0000 7f5e33cd5640 1 -- start start 2023-11-06T02:09:28.160 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.152+0000 7f5e33cd5640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e180039b0 0x7f5e2c14fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:28.160 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.152+0000 7f5e33cd5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5e2c1f5620 0x7f5e2c150a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:28.160 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.156+0000 7f5e33cd5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e2c152440 0x7f5e2c1528c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:28.160 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.156+0000 7f5e33cd5640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e2c0cc470 con 0x7f5e2c152440 2023-11-06T02:09:28.161 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.156+0000 7f5e33cd5640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e2c150100 con 0x7f5e180039b0 2023-11-06T02:09:28.161 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.156+0000 7f5e33cd5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e2c1f6660 con 0x7f5e2c1f5620 2023-11-06T02:09:28.161 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.156+0000 7f5e32272640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e2c152440 0x7f5e2c1528c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:28.161 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.156+0000 7f5e32272640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e2c152440 0x7f5e2c1528c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42192/0 (socket says 172.21.15.49:42192) 2023-11-06T02:09:28.161 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.156+0000 7f5e32272640 1 -- 172.21.15.49:0/1886996345 learned_addr learned my addr 172.21.15.49:0/1886996345 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:28.161 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.156+0000 7f5e31270640 1 --2- 172.21.15.49:0/1886996345 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5e2c1f5620 0x7f5e2c150a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:28.162 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.156+0000 7f5e31a71640 1 --2- 172.21.15.49:0/1886996345 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e180039b0 0x7f5e2c14fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:28.162 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.156+0000 7f5e32272640 1 -- 172.21.15.49:0/1886996345 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5e2c1f5620 msgr2=0x7f5e2c150a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:28.162 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.156+0000 7f5e32272640 1 --2- 172.21.15.49:0/1886996345 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5e2c1f5620 0x7f5e2c150a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:28.162 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.156+0000 7f5e32272640 1 -- 172.21.15.49:0/1886996345 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e180039b0 msgr2=0x7f5e2c14fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:28.162 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.156+0000 7f5e32272640 1 --2- 172.21.15.49:0/1886996345 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e180039b0 0x7f5e2c14fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:28.163 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.156+0000 7f5e32272640 1 -- 172.21.15.49:0/1886996345 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e2c150fb0 con 0x7f5e2c152440 2023-11-06T02:09:28.163 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.156+0000 7f5e31270640 1 --2- 172.21.15.49:0/1886996345 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5e2c1f5620 0x7f5e2c150a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:09:28.163 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.156+0000 7f5e32272640 1 --2- 172.21.15.49:0/1886996345 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e2c152440 0x7f5e2c1528c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c070050 tx=0x7f5e2400ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:28.163 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.156+0000 7f5e227fc640 1 -- 172.21.15.49:0/1886996345 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e24003980 con 0x7f5e2c152440 2023-11-06T02:09:28.163 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.156+0000 7f5e227fc640 1 -- 172.21.15.49:0/1886996345 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e24003ae0 con 0x7f5e2c152440 2023-11-06T02:09:28.164 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.156+0000 7f5e33cd5640 1 -- 172.21.15.49:0/1886996345 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e2c1f5dc0 con 0x7f5e2c152440 2023-11-06T02:09:28.164 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.156+0000 7f5e227fc640 1 -- 172.21.15.49:0/1886996345 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e24003e90 con 0x7f5e2c152440 2023-11-06T02:09:28.164 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.156+0000 7f5e33cd5640 1 -- 172.21.15.49:0/1886996345 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e2c1f6380 con 0x7f5e2c152440 2023-11-06T02:09:28.164 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.156+0000 7f5e33cd5640 1 -- 172.21.15.49:0/1886996345 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e2c0b0de0 con 0x7f5e2c152440 2023-11-06T02:09:28.164 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.156+0000 7f5e227fc640 1 -- 172.21.15.49:0/1886996345 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f5e240050d0 con 0x7f5e2c152440 2023-11-06T02:09:28.165 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.156+0000 7f5e227fc640 1 --2- 172.21.15.49:0/1886996345 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5e10065a60 0x7f5e10067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:28.165 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.156+0000 7f5e227fc640 1 -- 172.21.15.49:0/1886996345 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f5e24083110 con 0x7f5e2c152440 2023-11-06T02:09:28.165 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.156+0000 7f5e31a71640 1 --2- 172.21.15.49:0/1886996345 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5e10065a60 0x7f5e10067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:28.165 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.160+0000 7f5e31a71640 1 --2- 172.21.15.49:0/1886996345 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5e10065a60 0x7f5e10067f20 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c0ae000 tx=0x7f5e1c0046d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:28.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.160+0000 7f5e227fc640 1 -- 172.21.15.49:0/1886996345 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f5e24050930 con 0x7f5e2c152440 2023-11-06T02:09:28.350 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.352+0000 7f5e33cd5640 1 -- 172.21.15.49:0/1886996345 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5e2c0cc5b0 con 0x7f5e2c152440 2023-11-06T02:09:28.351 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.352+0000 7f5e227fc640 1 -- 172.21.15.49:0/1886996345 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4648 (secure 0 0 0) 0x7f5e2c0cc5b0 con 0x7f5e2c152440 2023-11-06T02:09:28.352 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:09:28.352 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6834/1377586971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1377586971},{"type":"v1","addr":"172.21.15.49:6834","nonce":1377586971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4288,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2620584959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2620584959},{"type":"v1","addr":"172.21.15.150:6835","nonce":2620584959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4291,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/4272191418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":4272191418},{"type":"v1","addr":"172.21.15.150:6837","nonce":4272191418}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T02:08:30.592185+0000","modified":"2023-11-06T02:08:33.245446+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4293},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4293":{"gid":4293,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.49:6835/4280448297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6833","nonce":4280448297},{"type":"v1","addr":"172.21.15.49:6835","nonce":4280448297}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-06T02:09:28.352 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 7 2023-11-06T02:09:28.354 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.356+0000 7f5e33cd5640 1 -- 172.21.15.49:0/1886996345 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5e10065a60 msgr2=0x7f5e10067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:28.354 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.356+0000 7f5e33cd5640 1 --2- 172.21.15.49:0/1886996345 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5e10065a60 0x7f5e10067f20 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c0ae000 tx=0x7f5e1c0046d0 comp rx=0 tx=0).stop 2023-11-06T02:09:28.354 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.356+0000 7f5e33cd5640 1 -- 172.21.15.49:0/1886996345 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e2c152440 msgr2=0x7f5e2c1528c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:28.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.356+0000 7f5e33cd5640 1 --2- 172.21.15.49:0/1886996345 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e2c152440 0x7f5e2c1528c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c070050 tx=0x7f5e2400ec10 comp rx=0 tx=0).stop 2023-11-06T02:09:28.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.356+0000 7f5e33cd5640 1 -- 172.21.15.49:0/1886996345 shutdown_connections 2023-11-06T02:09:28.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.356+0000 7f5e33cd5640 1 --2- 172.21.15.49:0/1886996345 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5e10065a60 0x7f5e10067f20 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:28.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.356+0000 7f5e33cd5640 1 --2- 172.21.15.49:0/1886996345 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e2c152440 0x7f5e2c1528c0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:28.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.356+0000 7f5e33cd5640 1 --2- 172.21.15.49:0/1886996345 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5e2c1f5620 0x7f5e2c150a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:28.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.356+0000 7f5e33cd5640 1 --2- 172.21.15.49:0/1886996345 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e180039b0 0x7f5e2c14fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:28.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.356+0000 7f5e33cd5640 1 -- 172.21.15.49:0/1886996345 >> 172.21.15.49:0/1886996345 conn(0x7f5e2c06c0a0 msgr2=0x7f5e2c159e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:28.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.356+0000 7f5e33cd5640 1 -- 172.21.15.49:0/1886996345 shutdown_connections 2023-11-06T02:09:28.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.356+0000 7f5e33cd5640 1 -- 172.21.15.49:0/1886996345 wait complete. 2023-11-06T02:09:28.372 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T02:09:28.498 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.500+0000 7fd39d585640 1 Processor -- start 2023-11-06T02:09:28.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.500+0000 7fd39d585640 1 -- start start 2023-11-06T02:09:28.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.500+0000 7fd39d585640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd3980c9c80 0x7fd3980b2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:28.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.500+0000 7fd39d585640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fd3980b5270 0x7fd3980ca640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:28.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.500+0000 7fd39d585640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3980ae6e0 con 0x7fd398071120 2023-11-06T02:09:28.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.500+0000 7fd39d585640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3980ae9c0 con 0x7fd3980b5270 2023-11-06T02:09:28.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.500+0000 7fd39d585640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3980af010 con 0x7fd3980c9c80 2023-11-06T02:09:28.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.500+0000 7fd396ffd640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fd398071120 0x7fd398071500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:51378/0 (socket says 172.21.15.49:51378) 2023-11-06T02:09:28.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.500+0000 7fd396ffd640 1 -- 172.21.15.49:0/580432681 learned_addr learned my addr 172.21.15.49:0/580432681 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:09:28.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.500+0000 7fd3977fe640 1 --2- 172.21.15.49:0/580432681 >> v2:172.21.15.150:3300/0 conn(0x7fd3980b5270 0x7fd3980ca640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:28.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.500+0000 7fd3967fc640 1 --2- 172.21.15.49:0/580432681 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd3980c9c80 0x7fd3980b2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:28.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.500+0000 7fd395ffb640 1 -- 172.21.15.49:0/580432681 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1899000802 0 0) 0x7fd3980ae6e0 con 0x7fd398071120 2023-11-06T02:09:28.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.500+0000 7fd395ffb640 1 -- 172.21.15.49:0/580432681 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd384003530 con 0x7fd398071120 2023-11-06T02:09:28.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.500+0000 7fd395ffb640 1 -- 172.21.15.49:0/580432681 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 569058740 0 0) 0x7fd384003530 con 0x7fd398071120 2023-11-06T02:09:28.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.500+0000 7fd395ffb640 1 -- 172.21.15.49:0/580432681 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd3980c9c80 msgr2=0x7fd3980b2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:28.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.500+0000 7fd395ffb640 1 --2- 172.21.15.49:0/580432681 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd3980c9c80 0x7fd3980b2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:28.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.500+0000 7fd395ffb640 1 -- 172.21.15.49:0/580432681 >> v2:172.21.15.150:3300/0 conn(0x7fd3980b5270 msgr2=0x7fd3980ca640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:28.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.500+0000 7fd395ffb640 1 --2- 172.21.15.49:0/580432681 >> v2:172.21.15.150:3300/0 conn(0x7fd3980b5270 0x7fd3980ca640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:28.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.500+0000 7fd395ffb640 1 -- 172.21.15.49:0/580432681 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3980aed30 con 0x7fd398071120 2023-11-06T02:09:28.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.500+0000 7fd395ffb640 1 -- 172.21.15.49:0/580432681 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7fd390001a00 con 0x7fd398071120 2023-11-06T02:09:28.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.500+0000 7fd395ffb640 1 -- 172.21.15.49:0/580432681 >> v1:172.21.15.49:6789/0 conn(0x7fd398071120 legacy=0x7fd398071500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:28.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.500+0000 7fd395ffb640 1 --2- 172.21.15.49:0/580432681 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd3980c9c80 0x7fd384003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:28.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.500+0000 7fd395ffb640 1 --2- 172.21.15.49:0/580432681 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd384003f00 0x7fd3840047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:28.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.500+0000 7fd395ffb640 1 --2- 172.21.15.49:0/580432681 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd384004d10 0x7fd384005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:28.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.500+0000 7fd395ffb640 1 -- 172.21.15.49:0/580432681 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3980ae9c0 con 0x7fd384003f00 2023-11-06T02:09:28.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.500+0000 7fd395ffb640 1 -- 172.21.15.49:0/580432681 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3980af010 con 0x7fd384004d10 2023-11-06T02:09:28.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.500+0000 7fd395ffb640 1 -- 172.21.15.49:0/580432681 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd38400fdd0 con 0x7fd3980c9c80 2023-11-06T02:09:28.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.500+0000 7fd3967fc640 1 --2- 172.21.15.49:0/580432681 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd384003f00 0x7fd3840047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:28.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.500+0000 7fd396ffd640 1 --2- 172.21.15.49:0/580432681 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd3980c9c80 0x7fd384003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:28.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.500+0000 7fd3977fe640 1 --2- 172.21.15.49:0/580432681 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd384004d10 0x7fd384005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:28.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.500+0000 7fd3967fc640 1 -- 172.21.15.49:0/580432681 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd3980c9c80 msgr2=0x7fd384003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:28.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.500+0000 7fd3967fc640 1 --2- 172.21.15.49:0/580432681 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd3980c9c80 0x7fd384003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:28.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.500+0000 7fd3967fc640 1 -- 172.21.15.49:0/580432681 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd384004d10 msgr2=0x7fd384005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:28.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.500+0000 7fd3967fc640 1 --2- 172.21.15.49:0/580432681 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd384004d10 0x7fd384005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:28.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.500+0000 7fd3967fc640 1 -- 172.21.15.49:0/580432681 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd384003530 con 0x7fd384003f00 2023-11-06T02:09:28.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.500+0000 7fd3977fe640 1 --2- 172.21.15.49:0/580432681 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd384004d10 0x7fd384005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:09:28.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.504+0000 7fd3967fc640 1 --2- 172.21.15.49:0/580432681 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd384003f00 0x7fd3840047d0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fd380009ae0 tx=0x7fd38001ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d390e773ce56ad09 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:28.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.504+0000 7fd395ffb640 1 -- 172.21.15.49:0/580432681 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3800a5640 con 0x7fd384003f00 2023-11-06T02:09:28.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.504+0000 7fd395ffb640 1 -- 172.21.15.49:0/580432681 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3800a57a0 con 0x7fd384003f00 2023-11-06T02:09:28.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.504+0000 7fd395ffb640 1 -- 172.21.15.49:0/580432681 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3800a5b50 con 0x7fd384003f00 2023-11-06T02:09:28.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.504+0000 7fd39d585640 1 -- 172.21.15.49:0/580432681 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd384003f00 msgr2=0x7fd3840047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:28.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.504+0000 7fd39d585640 1 --2- 172.21.15.49:0/580432681 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd384003f00 0x7fd3840047d0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fd380009ae0 tx=0x7fd38001ca50 comp rx=0 tx=0).stop 2023-11-06T02:09:28.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.504+0000 7fd3977fe640 1 -- 172.21.15.49:0/580432681 reap_dead start 2023-11-06T02:09:28.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.504+0000 7fd39d585640 1 -- 172.21.15.49:0/580432681 shutdown_connections 2023-11-06T02:09:28.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.504+0000 7fd39d585640 1 -- 172.21.15.49:0/580432681 >> 172.21.15.49:0/580432681 conn(0x7fd3980cc140 msgr2=0x7fd3980cc550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:28.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.504+0000 7fd39d585640 1 -- 172.21.15.49:0/580432681 shutdown_connections 2023-11-06T02:09:28.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.504+0000 7fd39d585640 1 -- 172.21.15.49:0/580432681 wait complete. 2023-11-06T02:09:28.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.508+0000 7fd39d585640 1 Processor -- start 2023-11-06T02:09:28.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.508+0000 7fd39d585640 1 -- start start 2023-11-06T02:09:28.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.508+0000 7fd39d585640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd384003f00 0x7fd398155e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:28.514 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.508+0000 7fd39d585640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd3981e23c0 0x7fd398146910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:28.514 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.508+0000 7fd39d585640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd3980c7750 0x7fd3980c7c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:28.514 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.508+0000 7fd39d585640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3980c8140 con 0x7fd3980c7750 2023-11-06T02:09:28.514 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.508+0000 7fd39d585640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd398156390 con 0x7fd384003f00 2023-11-06T02:09:28.514 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.508+0000 7fd39d585640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd398146e50 con 0x7fd3981e23c0 2023-11-06T02:09:28.515 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.508+0000 7fd3977fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd3980c7750 0x7fd3980c7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:28.515 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.508+0000 7fd3977fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd3980c7750 0x7fd3980c7c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40498/0 (socket says 172.21.15.49:40498) 2023-11-06T02:09:28.515 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.508+0000 7fd3977fe640 1 -- 172.21.15.49:0/394979407 learned_addr learned my addr 172.21.15.49:0/394979407 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:28.515 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.508+0000 7fd396ffd640 1 --2- 172.21.15.49:0/394979407 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd384003f00 0x7fd398155e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:28.515 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.508+0000 7fd3967fc640 1 --2- 172.21.15.49:0/394979407 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd3981e23c0 0x7fd398146910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:28.516 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.508+0000 7fd3977fe640 1 -- 172.21.15.49:0/394979407 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd3981e23c0 msgr2=0x7fd398146910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:28.516 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.508+0000 7fd3977fe640 1 --2- 172.21.15.49:0/394979407 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd3981e23c0 0x7fd398146910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:28.516 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.508+0000 7fd3977fe640 1 -- 172.21.15.49:0/394979407 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd384003f00 msgr2=0x7fd398155e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:28.516 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.508+0000 7fd3977fe640 1 --2- 172.21.15.49:0/394979407 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd384003f00 0x7fd398155e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:28.516 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.508+0000 7fd3977fe640 1 -- 172.21.15.49:0/394979407 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3981e2860 con 0x7fd3980c7750 2023-11-06T02:09:28.517 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.508+0000 7fd3977fe640 1 --2- 172.21.15.49:0/394979407 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd3980c7750 0x7fd3980c7c00 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fd3980b5030 tx=0x7fd388010b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:28.517 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.508+0000 7fd38f7fe640 1 -- 172.21.15.49:0/394979407 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd38800e870 con 0x7fd3980c7750 2023-11-06T02:09:28.517 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.508+0000 7fd38f7fe640 1 -- 172.21.15.49:0/394979407 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd38800e9d0 con 0x7fd3980c7750 2023-11-06T02:09:28.517 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.508+0000 7fd38f7fe640 1 -- 172.21.15.49:0/394979407 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd38800ed40 con 0x7fd3980c7750 2023-11-06T02:09:28.517 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.508+0000 7fd39d585640 1 -- 172.21.15.49:0/394979407 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3981e2b40 con 0x7fd3980c7750 2023-11-06T02:09:28.517 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.508+0000 7fd39d585640 1 -- 172.21.15.49:0/394979407 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3981e3100 con 0x7fd3980c7750 2023-11-06T02:09:28.518 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.508+0000 7fd39d585640 1 -- 172.21.15.49:0/394979407 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd398143ec0 con 0x7fd3980c7750 2023-11-06T02:09:28.518 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.508+0000 7fd38f7fe640 1 -- 172.21.15.49:0/394979407 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fd3880143f0 con 0x7fd3980c7750 2023-11-06T02:09:28.518 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.508+0000 7fd38f7fe640 1 --2- 172.21.15.49:0/394979407 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd378065a90 0x7fd378067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:28.518 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.508+0000 7fd38f7fe640 1 -- 172.21.15.49:0/394979407 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fd388083040 con 0x7fd3980c7750 2023-11-06T02:09:28.518 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.512+0000 7fd396ffd640 1 --2- 172.21.15.49:0/394979407 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd378065a90 0x7fd378067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:28.519 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.512+0000 7fd396ffd640 1 --2- 172.21.15.49:0/394979407 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd378065a90 0x7fd378067f50 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd390001bb0 tx=0x7fd3900017c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:28.519 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.516+0000 7fd38f7fe640 1 -- 172.21.15.49:0/394979407 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fd3880508f0 con 0x7fd3980c7750 2023-11-06T02:09:28.703 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.704+0000 7fd39d585640 1 -- 172.21.15.49:0/394979407 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd398067140 con 0x7fd3980c7750 2023-11-06T02:09:28.703 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.704+0000 7fd38f7fe640 1 -- 172.21.15.49:0/394979407 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4648 (secure 0 0 0) 0x7fd388053f10 con 0x7fd3980c7750 2023-11-06T02:09:28.704 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:09:28.704 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6834/1377586971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1377586971},{"type":"v1","addr":"172.21.15.49:6834","nonce":1377586971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4288,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2620584959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2620584959},{"type":"v1","addr":"172.21.15.150:6835","nonce":2620584959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4291,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/4272191418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":4272191418},{"type":"v1","addr":"172.21.15.150:6837","nonce":4272191418}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T02:08:30.592185+0000","modified":"2023-11-06T02:08:33.245446+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4293},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4293":{"gid":4293,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.49:6835/4280448297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6833","nonce":4280448297},{"type":"v1","addr":"172.21.15.49:6835","nonce":4280448297}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-06T02:09:28.704 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 7 2023-11-06T02:09:28.706 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.708+0000 7fd39d585640 1 -- 172.21.15.49:0/394979407 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd378065a90 msgr2=0x7fd378067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:28.706 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.708+0000 7fd39d585640 1 --2- 172.21.15.49:0/394979407 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd378065a90 0x7fd378067f50 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd390001bb0 tx=0x7fd3900017c0 comp rx=0 tx=0).stop 2023-11-06T02:09:28.706 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.708+0000 7fd39d585640 1 -- 172.21.15.49:0/394979407 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd3980c7750 msgr2=0x7fd3980c7c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:28.706 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.708+0000 7fd39d585640 1 --2- 172.21.15.49:0/394979407 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd3980c7750 0x7fd3980c7c00 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fd3980b5030 tx=0x7fd388010b70 comp rx=0 tx=0).stop 2023-11-06T02:09:28.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.708+0000 7fd39d585640 1 -- 172.21.15.49:0/394979407 shutdown_connections 2023-11-06T02:09:28.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.708+0000 7fd39d585640 1 --2- 172.21.15.49:0/394979407 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd378065a90 0x7fd378067f50 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:28.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.708+0000 7fd39d585640 1 --2- 172.21.15.49:0/394979407 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd3980c7750 0x7fd3980c7c00 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:28.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.708+0000 7fd39d585640 1 --2- 172.21.15.49:0/394979407 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd3981e23c0 0x7fd398146910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:28.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.708+0000 7fd39d585640 1 --2- 172.21.15.49:0/394979407 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd384003f00 0x7fd398155e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:28.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.708+0000 7fd39d585640 1 -- 172.21.15.49:0/394979407 >> 172.21.15.49:0/394979407 conn(0x7fd3980cc140 msgr2=0x7fd39815a0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:28.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.708+0000 7fd39d585640 1 -- 172.21.15.49:0/394979407 shutdown_connections 2023-11-06T02:09:28.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.708+0000 7fd39d585640 1 -- 172.21.15.49:0/394979407 wait complete. 2023-11-06T02:09:28.723 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T02:09:28.849 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.852+0000 7f3bb0ee4640 1 Processor -- start 2023-11-06T02:09:28.850 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.852+0000 7f3bb0ee4640 1 -- start start 2023-11-06T02:09:28.851 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.852+0000 7f3bb0ee4640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3bac0b0790 0x7f3bac070a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:28.851 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.852+0000 7f3bb0ee4640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f3bac0ca830 0x7f3bac0b1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:28.852 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.852+0000 7f3bb0ee4640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bac0cacf0 con 0x7f3bac15aae0 2023-11-06T02:09:28.852 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.852+0000 7f3bb0ee4640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bac070fc0 con 0x7f3bac0ca830 2023-11-06T02:09:28.852 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.852+0000 7f3bb0ee4640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bac15aec0 con 0x7f3bac0b0790 2023-11-06T02:09:28.852 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.852+0000 7f3baad9d640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f3bac0ca830 0x7f3bac0b1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:28.852 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.852+0000 7f3baad9d640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f3bac0ca830 0x7f3bac0b1450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40502/0 (socket says 172.21.15.49:40502) 2023-11-06T02:09:28.853 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.852+0000 7f3baad9d640 1 -- 172.21.15.49:0/4272789999 learned_addr learned my addr 172.21.15.49:0/4272789999 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:28.853 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.852+0000 7f3ba3fff640 1 --2- 172.21.15.49:0/4272789999 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3bac0b0790 0x7f3bac070a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:28.853 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.852+0000 7f3baad9d640 1 -- 172.21.15.49:0/4272789999 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3bac0b0790 msgr2=0x7f3bac070a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:28.853 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.852+0000 7f3baad9d640 1 --2- 172.21.15.49:0/4272789999 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3bac0b0790 0x7f3bac070a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:28.853 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.852+0000 7f3baad9d640 1 -- 172.21.15.49:0/4272789999 >> v1:172.21.15.150:6789/0 conn(0x7f3bac15aae0 legacy=0x7f3bac0b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:28.854 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.852+0000 7f3baad9d640 1 -- 172.21.15.49:0/4272789999 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bac0c8ff0 con 0x7f3bac0ca830 2023-11-06T02:09:28.855 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.852+0000 7f3baad9d640 1 --2- 172.21.15.49:0/4272789999 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bac0ca830 0x7f3bac0b1450 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f3ba4009e50 tx=0x7f3ba401c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5b1f9a5b835359d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:28.855 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.856+0000 7f3ba9d9b640 1 -- 172.21.15.49:0/4272789999 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba40a5790 con 0x7f3bac0ca830 2023-11-06T02:09:28.856 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.856+0000 7f3ba9d9b640 1 -- 172.21.15.49:0/4272789999 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ba40a58f0 con 0x7f3bac0ca830 2023-11-06T02:09:28.856 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.856+0000 7f3ba9d9b640 1 -- 172.21.15.49:0/4272789999 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba40a5c80 con 0x7f3bac0ca830 2023-11-06T02:09:28.856 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.856+0000 7f3bb0ee4640 1 -- 172.21.15.49:0/4272789999 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bac0ca830 msgr2=0x7f3bac0b1450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:28.856 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.856+0000 7f3bb0ee4640 1 --2- 172.21.15.49:0/4272789999 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bac0ca830 0x7f3bac0b1450 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f3ba4009e50 tx=0x7f3ba401c910 comp rx=0 tx=0).stop 2023-11-06T02:09:28.856 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.856+0000 7f3bb0ee4640 1 -- 172.21.15.49:0/4272789999 shutdown_connections 2023-11-06T02:09:28.857 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.856+0000 7f3bb0ee4640 1 --2- 172.21.15.49:0/4272789999 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3bac0b0790 0x7f3bac070a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:28.857 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.856+0000 7f3bb0ee4640 1 --2- 172.21.15.49:0/4272789999 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bac0ca830 0x7f3bac0b1450 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:28.857 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.856+0000 7f3bb0ee4640 1 -- 172.21.15.49:0/4272789999 >> 172.21.15.49:0/4272789999 conn(0x7f3bac06c0a0 msgr2=0x7f3bac06c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:28.857 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.856+0000 7f3bb0ee4640 1 -- 172.21.15.49:0/4272789999 shutdown_connections 2023-11-06T02:09:28.857 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.856+0000 7f3bb0ee4640 1 -- 172.21.15.49:0/4272789999 wait complete. 2023-11-06T02:09:28.858 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.856+0000 7f3bb0ee4640 1 Processor -- start 2023-11-06T02:09:28.858 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.856+0000 7f3bb0ee4640 1 -- start start 2023-11-06T02:09:28.860 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.856+0000 7f3bb0ee4640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3bac0b0790 0x7f3bac14db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:28.860 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.856+0000 7f3bb0ee4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bac0ca830 0x7f3bac1551c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:28.861 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.856+0000 7f3bb0ee4640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3bac15aae0 0x7f3bac155700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:28.861 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.856+0000 7f3bb0ee4640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bac0c46b0 con 0x7f3bac0ca830 2023-11-06T02:09:28.861 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.856+0000 7f3bb0ee4640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bac0cd940 con 0x7f3bac0b0790 2023-11-06T02:09:28.861 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.856+0000 7f3bb0ee4640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bac0cdbe0 con 0x7f3bac15aae0 2023-11-06T02:09:28.862 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.856+0000 7f3ba3fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bac0ca830 0x7f3bac1551c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:28.862 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.856+0000 7f3ba3fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bac0ca830 0x7f3bac1551c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40512/0 (socket says 172.21.15.49:40512) 2023-11-06T02:09:28.862 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.856+0000 7f3ba3fff640 1 -- 172.21.15.49:0/3814844813 learned_addr learned my addr 172.21.15.49:0/3814844813 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:28.862 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.856+0000 7f3baad9d640 1 --2- 172.21.15.49:0/3814844813 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3bac15aae0 0x7f3bac155700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:28.862 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.856+0000 7f3baa59c640 1 --2- 172.21.15.49:0/3814844813 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3bac0b0790 0x7f3bac14db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:28.863 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.856+0000 7f3ba3fff640 1 -- 172.21.15.49:0/3814844813 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3bac15aae0 msgr2=0x7f3bac155700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:28.863 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.856+0000 7f3ba3fff640 1 --2- 172.21.15.49:0/3814844813 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3bac15aae0 0x7f3bac155700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:28.863 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.856+0000 7f3ba3fff640 1 -- 172.21.15.49:0/3814844813 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3bac0b0790 msgr2=0x7f3bac14db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:28.863 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.856+0000 7f3ba3fff640 1 --2- 172.21.15.49:0/3814844813 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3bac0b0790 0x7f3bac14db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:28.863 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.856+0000 7f3ba3fff640 1 -- 172.21.15.49:0/3814844813 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ba4009b30 con 0x7f3bac0ca830 2023-11-06T02:09:28.864 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.856+0000 7f3ba3fff640 1 --2- 172.21.15.49:0/3814844813 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bac0ca830 0x7f3bac1551c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f3b9400d800 tx=0x7f3b9400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:28.864 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.860+0000 7f3ba2ffd640 1 -- 172.21.15.49:0/3814844813 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b9400b6c0 con 0x7f3bac0ca830 2023-11-06T02:09:28.864 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.860+0000 7f3ba2ffd640 1 -- 172.21.15.49:0/3814844813 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b9400b820 con 0x7f3bac0ca830 2023-11-06T02:09:28.864 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.860+0000 7f3bb0ee4640 1 -- 172.21.15.49:0/3814844813 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bac0cdec0 con 0x7f3bac0ca830 2023-11-06T02:09:28.864 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.860+0000 7f3ba2ffd640 1 -- 172.21.15.49:0/3814844813 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b9400bb90 con 0x7f3bac0ca830 2023-11-06T02:09:28.865 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.860+0000 7f3bb0ee4640 1 -- 172.21.15.49:0/3814844813 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bac0c33b0 con 0x7f3bac0ca830 2023-11-06T02:09:28.865 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.860+0000 7f3bb0ee4640 1 -- 172.21.15.49:0/3814844813 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bac074040 con 0x7f3bac0ca830 2023-11-06T02:09:28.865 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.860+0000 7f3ba2ffd640 1 -- 172.21.15.49:0/3814844813 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f3b9400bcf0 con 0x7f3bac0ca830 2023-11-06T02:09:28.865 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.860+0000 7f3ba2ffd640 1 --2- 172.21.15.49:0/3814844813 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3b8c065a60 0x7f3b8c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:28.865 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.860+0000 7f3ba2ffd640 1 -- 172.21.15.49:0/3814844813 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f3b94083830 con 0x7f3bac0ca830 2023-11-06T02:09:28.866 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.860+0000 7f3baa59c640 1 --2- 172.21.15.49:0/3814844813 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3b8c065a60 0x7f3b8c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:28.866 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.860+0000 7f3baa59c640 1 --2- 172.21.15.49:0/3814844813 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3b8c065a60 0x7f3b8c067f20 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c006fd0 tx=0x7f3b9c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:28.866 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:28.864+0000 7f3ba2ffd640 1 -- 172.21.15.49:0/3814844813 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f3b94051050 con 0x7f3bac0ca830 2023-11-06T02:09:29.019 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.020+0000 7f3bb0ee4640 1 -- 172.21.15.49:0/3814844813 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3bac14e0c0 con 0x7f3bac0ca830 2023-11-06T02:09:29.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.024+0000 7f3ba2ffd640 1 -- 172.21.15.49:0/3814844813 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17906 (secure 0 0 0) 0x7f3b94054670 con 0x7f3bac0ca830 2023-11-06T02:09:29.022 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:09:29.022 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":25,"fsid":"1a8b035c-6ba4-499a-b876-aea47fd9c0d4","created":"2023-11-06T02:08:06.440857+0000","modified":"2023-11-06T02:09:13.195118+0000","last_up_change":"2023-11-06T02:08:17.982372+0000","last_in_change":"2023-11-06T02:08:08.906631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T02:08:19.252222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T02:08:24.776211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-06T02:08:28.936178+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-06T02:08:29.563468+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ad55af8-0571-4e6a-8722-b5d429b7496d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6809","nonce":38116},{"type":"v1","addr":"172.21.15.49:6811","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6813","nonce":38116},{"type":"v1","addr":"172.21.15.49:6815","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6821","nonce":38116},{"type":"v1","addr":"172.21.15.49:6823","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6817","nonce":38116},{"type":"v1","addr":"172.21.15.49:6819","nonce":38116}]},"public_addr":"172.21.15.49:6811/38116","cluster_addr":"172.21.15.49:6815/38116","heartbeat_back_addr":"172.21.15.49:6823/38116","heartbeat_front_addr":"172.21.15.49:6819/38116","state":["exists","up"]},{"osd":1,"uuid":"c8b31f85-a757-43ed-9419-351acc46611b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":38113},{"type":"v1","addr":"172.21.15.49:6810","nonce":38113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":38113},{"type":"v1","addr":"172.21.15.49:6814","nonce":38113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":38113},{"type":"v1","addr":"172.21.15.49:6822","nonce":38113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":38113},{"type":"v1","addr":"172.21.15.49:6818","nonce":38113}]},"public_addr":"172.21.15.49:6810/38113","cluster_addr":"172.21.15.49:6814/38113","heartbeat_back_addr":"172.21.15.49:6822/38113","heartbeat_front_addr":"172.21.15.49:6818/38113","state":["exists","up"]},{"osd":2,"uuid":"555b3bf3-0a70-4b91-8efd-e4204b5fc26e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":38114},{"type":"v1","addr":"172.21.15.49:6801","nonce":38114}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":38114},{"type":"v1","addr":"172.21.15.49:6803","nonce":38114}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":38114},{"type":"v1","addr":"172.21.15.49:6807","nonce":38114}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":38114},{"type":"v1","addr":"172.21.15.49:6805","nonce":38114}]},"public_addr":"172.21.15.49:6801/38114","cluster_addr":"172.21.15.49:6803/38114","heartbeat_back_addr":"172.21.15.49:6807/38114","heartbeat_front_addr":"172.21.15.49:6805/38114","state":["exists","up"]},{"osd":3,"uuid":"8d4e5d77-03ef-4b9b-bfcb-5ae93a46795c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":38115},{"type":"v1","addr":"172.21.15.49:6825","nonce":38115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":38115},{"type":"v1","addr":"172.21.15.49:6827","nonce":38115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":38115},{"type":"v1","addr":"172.21.15.49:6831","nonce":38115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":38115},{"type":"v1","addr":"172.21.15.49:6829","nonce":38115}]},"public_addr":"172.21.15.49:6825/38115","cluster_addr":"172.21.15.49:6827/38115","heartbeat_back_addr":"172.21.15.49:6831/38115","heartbeat_front_addr":"172.21.15.49:6829/38115","state":["exists","up"]},{"osd":4,"uuid":"dba03ad4-68a3-4cf3-9218-ac611f02eacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":38417},{"type":"v1","addr":"172.21.15.150:6809","nonce":38417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":38417},{"type":"v1","addr":"172.21.15.150:6811","nonce":38417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":38417},{"type":"v1","addr":"172.21.15.150:6815","nonce":38417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":38417},{"type":"v1","addr":"172.21.15.150:6813","nonce":38417}]},"public_addr":"172.21.15.150:6809/38417","cluster_addr":"172.21.15.150:6811/38417","heartbeat_back_addr":"172.21.15.150:6815/38417","heartbeat_front_addr":"172.21.15.150:6813/38417","state":["exists","up"]},{"osd":5,"uuid":"0df9c303-c255-4640-8e91-01f1c3efe18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":38418},{"type":"v1","addr":"172.21.15.150:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":38418},{"type":"v1","addr":"172.21.15.150:6803","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":38418},{"type":"v1","addr":"172.21.15.150:6807","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":38418},{"type":"v1","addr":"172.21.15.150:6805","nonce":38418}]},"public_addr":"172.21.15.150:6801/38418","cluster_addr":"172.21.15.150:6803/38418","heartbeat_back_addr":"172.21.15.150:6807/38418","heartbeat_front_addr":"172.21.15.150:6805/38418","state":["exists","up"]},{"osd":6,"uuid":"e59fa189-4b70-4c06-9722-c02b90828bf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":38428},{"type":"v1","addr":"172.21.15.150:6825","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":38428},{"type":"v1","addr":"172.21.15.150:6827","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":38428},{"type":"v1","addr":"172.21.15.150:6831","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":38428},{"type":"v1","addr":"172.21.15.150:6829","nonce":38428}]},"public_addr":"172.21.15.150:6825/38428","cluster_addr":"172.21.15.150:6827/38428","heartbeat_back_addr":"172.21.15.150:6831/38428","heartbeat_front_addr":"172.21.15.150:6829/38428","state":["exists","up"]},{"osd":7,"uuid":"a64f57e1-ef98-46bd-b304-385759e95898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":38421},{"type":"v1","addr":"172.21.15.150:6817","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":38421},{"type":"v1","addr":"172.21.15.150:6819","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":38421},{"type":"v1","addr":"172.21.15.150:6823","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":38421},{"type":"v1","addr":"172.21.15.150:6821","nonce":38421}]},"public_addr":"172.21.15.150:6817/38421","cluster_addr":"172.21.15.150:6819/38421","heartbeat_back_addr":"172.21.15.150:6823/38421","heartbeat_front_addr":"172.21.15.150:6821/38421","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.438552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.413280+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.296466+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.424966+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.276953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.230984+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.410235+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.480084+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.14","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T02:09:29.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.024+0000 7f3bb0ee4640 1 -- 172.21.15.49:0/3814844813 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3b8c065a60 msgr2=0x7f3b8c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:29.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.024+0000 7f3bb0ee4640 1 --2- 172.21.15.49:0/3814844813 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3b8c065a60 0x7f3b8c067f20 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c006fd0 tx=0x7f3b9c008040 comp rx=0 tx=0).stop 2023-11-06T02:09:29.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.024+0000 7f3bb0ee4640 1 -- 172.21.15.49:0/3814844813 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bac0ca830 msgr2=0x7f3bac1551c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:29.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.024+0000 7f3bb0ee4640 1 --2- 172.21.15.49:0/3814844813 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bac0ca830 0x7f3bac1551c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f3b9400d800 tx=0x7f3b9400dcc0 comp rx=0 tx=0).stop 2023-11-06T02:09:29.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.024+0000 7f3bb0ee4640 1 -- 172.21.15.49:0/3814844813 shutdown_connections 2023-11-06T02:09:29.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.024+0000 7f3bb0ee4640 1 --2- 172.21.15.49:0/3814844813 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3b8c065a60 0x7f3b8c067f20 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:29.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.024+0000 7f3bb0ee4640 1 --2- 172.21.15.49:0/3814844813 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3bac0ca830 0x7f3bac1551c0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:29.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.024+0000 7f3bb0ee4640 1 --2- 172.21.15.49:0/3814844813 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3bac15aae0 0x7f3bac155700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:29.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.024+0000 7f3bb0ee4640 1 --2- 172.21.15.49:0/3814844813 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3bac0b0790 0x7f3bac14db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:29.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.024+0000 7f3bb0ee4640 1 -- 172.21.15.49:0/3814844813 >> 172.21.15.49:0/3814844813 conn(0x7f3bac06c0a0 msgr2=0x7f3bac0c2210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:29.026 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.024+0000 7f3bb0ee4640 1 -- 172.21.15.49:0/3814844813 shutdown_connections 2023-11-06T02:09:29.026 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.024+0000 7f3bb0ee4640 1 -- 172.21.15.49:0/3814844813 wait complete. 2023-11-06T02:09:29.041 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-06T02:09:29.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.168+0000 7fe9f51ee640 1 Processor -- start 2023-11-06T02:09:29.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.168+0000 7fe9f51ee640 1 -- start start 2023-11-06T02:09:29.170 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.168+0000 7fe9f51ee640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe9f015a9d0 0x7fe9f00d1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:29.171 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.168+0000 7fe9f51ee640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9f00cf8a0 con 0x7fe9f00cd250 2023-11-06T02:09:29.171 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.168+0000 7fe9f51ee640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9f00ce210 con 0x7fe9f00b0470 2023-11-06T02:09:29.171 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.168+0000 7fe9f51ee640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f00cf000 con 0x7fe9f015a9d0 2023-11-06T02:09:29.171 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.168+0000 7fe9ee59c640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fe9f00cd250 0x7fe9f00cd6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:51390/0 (socket says 172.21.15.49:51390) 2023-11-06T02:09:29.172 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.168+0000 7fe9ee59c640 1 -- 172.21.15.49:0/1774561158 learned_addr learned my addr 172.21.15.49:0/1774561158 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:09:29.172 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.168+0000 7fe9eed9d640 1 --2- 172.21.15.49:0/1774561158 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe9f015a9d0 0x7fe9f00d1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:29.172 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.168+0000 7fe9edd9b640 1 -- 172.21.15.49:0/1774561158 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4090895032 0 0) 0x7fe9f00cf8a0 con 0x7fe9f00cd250 2023-11-06T02:09:29.172 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.168+0000 7fe9edd9b640 1 -- 172.21.15.49:0/1774561158 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9d8003530 con 0x7fe9f00cd250 2023-11-06T02:09:29.172 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.172+0000 7fe9edd9b640 1 -- 172.21.15.49:0/1774561158 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2480846721 0 0) 0x7fe9d8003530 con 0x7fe9f00cd250 2023-11-06T02:09:29.173 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.172+0000 7fe9edd9b640 1 -- 172.21.15.49:0/1774561158 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe9f015a9d0 msgr2=0x7fe9f00d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:29.173 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.172+0000 7fe9edd9b640 1 --2- 172.21.15.49:0/1774561158 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe9f015a9d0 0x7fe9f00d1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:29.173 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.172+0000 7fe9edd9b640 1 -- 172.21.15.49:0/1774561158 >> v1:172.21.15.150:6789/0 conn(0x7fe9f00b0470 legacy=0x7fe9f00b0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:29.173 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.172+0000 7fe9edd9b640 1 -- 172.21.15.49:0/1774561158 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9f00cf2e0 con 0x7fe9f00cd250 2023-11-06T02:09:29.173 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.172+0000 7fe9edd9b640 1 -- 172.21.15.49:0/1774561158 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7fe9e4001680 con 0x7fe9f00cd250 2023-11-06T02:09:29.174 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.172+0000 7fe9edd9b640 1 -- 172.21.15.49:0/1774561158 >> v1:172.21.15.49:6789/0 conn(0x7fe9f00cd250 legacy=0x7fe9f00cd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:29.174 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.172+0000 7fe9edd9b640 1 --2- 172.21.15.49:0/1774561158 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe9d80039b0 0x7fe9d8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:29.174 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.172+0000 7fe9edd9b640 1 --2- 172.21.15.49:0/1774561158 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9d80047d0 0x7fe9d8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:29.174 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.172+0000 7fe9edd9b640 1 --2- 172.21.15.49:0/1774561158 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe9f015a9d0 0x7fe9d80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:29.174 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.172+0000 7fe9edd9b640 1 -- 172.21.15.49:0/1774561158 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f00cf000 con 0x7fe9d80047d0 2023-11-06T02:09:29.175 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.172+0000 7fe9edd9b640 1 -- 172.21.15.49:0/1774561158 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d800bac0 con 0x7fe9d80039b0 2023-11-06T02:09:29.175 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.172+0000 7fe9edd9b640 1 -- 172.21.15.49:0/1774561158 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d800bc80 con 0x7fe9f015a9d0 2023-11-06T02:09:29.175 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.172+0000 7fe9ee59c640 1 --2- 172.21.15.49:0/1774561158 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9d80047d0 0x7fe9d8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:29.175 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.172+0000 7fe9eed9d640 1 --2- 172.21.15.49:0/1774561158 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe9d80039b0 0x7fe9d8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:29.175 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.172+0000 7fe9ef59e640 1 --2- 172.21.15.49:0/1774561158 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe9f015a9d0 0x7fe9d80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:29.176 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.172+0000 7fe9ee59c640 1 -- 172.21.15.49:0/1774561158 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe9f015a9d0 msgr2=0x7fe9d80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:29.176 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.172+0000 7fe9ee59c640 1 --2- 172.21.15.49:0/1774561158 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe9f015a9d0 0x7fe9d80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:29.176 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.172+0000 7fe9ee59c640 1 -- 172.21.15.49:0/1774561158 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe9d80039b0 msgr2=0x7fe9d8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:29.176 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.172+0000 7fe9ee59c640 1 --2- 172.21.15.49:0/1774561158 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe9d80039b0 0x7fe9d8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:29.176 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.172+0000 7fe9ee59c640 1 -- 172.21.15.49:0/1774561158 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9d8003530 con 0x7fe9d80047d0 2023-11-06T02:09:29.177 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.172+0000 7fe9eed9d640 1 --2- 172.21.15.49:0/1774561158 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe9d80039b0 0x7fe9d8003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:09:29.180 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.172+0000 7fe9ee59c640 1 --2- 172.21.15.49:0/1774561158 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9d80047d0 0x7fe9d8004c50 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fe9e400c240 tx=0x7fe9e401cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7a8b668cc1ff377 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:29.180 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.172+0000 7fe9edd9b640 1 -- 172.21.15.49:0/1774561158 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e4005ae0 con 0x7fe9d80047d0 2023-11-06T02:09:29.181 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.172+0000 7fe9edd9b640 1 -- 172.21.15.49:0/1774561158 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9e4005c40 con 0x7fe9d80047d0 2023-11-06T02:09:29.181 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.172+0000 7fe9edd9b640 1 -- 172.21.15.49:0/1774561158 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e40a5800 con 0x7fe9d80047d0 2023-11-06T02:09:29.181 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.172+0000 7fe9f51ee640 1 -- 172.21.15.49:0/1774561158 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9d80047d0 msgr2=0x7fe9d8004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:29.181 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.172+0000 7fe9f51ee640 1 --2- 172.21.15.49:0/1774561158 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9d80047d0 0x7fe9d8004c50 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fe9e400c240 tx=0x7fe9e401cc10 comp rx=0 tx=0).stop 2023-11-06T02:09:29.181 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.172+0000 7fe9ef59e640 1 -- 172.21.15.49:0/1774561158 reap_dead start 2023-11-06T02:09:29.182 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.172+0000 7fe9f51ee640 1 -- 172.21.15.49:0/1774561158 shutdown_connections 2023-11-06T02:09:29.182 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.172+0000 7fe9f51ee640 1 -- 172.21.15.49:0/1774561158 >> 172.21.15.49:0/1774561158 conn(0x7fe9f00d08f0 msgr2=0x7fe9f00d0d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:29.182 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.176+0000 7fe9f51ee640 1 -- 172.21.15.49:0/1774561158 shutdown_connections 2023-11-06T02:09:29.182 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.176+0000 7fe9f51ee640 1 -- 172.21.15.49:0/1774561158 wait complete. 2023-11-06T02:09:29.182 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.176+0000 7fe9f51ee640 1 Processor -- start 2023-11-06T02:09:29.183 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.176+0000 7fe9f51ee640 1 -- start start 2023-11-06T02:09:29.183 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.176+0000 7fe9f51ee640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe9d80047d0 0x7fe9f01f4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:29.183 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.176+0000 7fe9f51ee640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe9f0152e60 0x7fe9f01f0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:29.183 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.176+0000 7fe9f51ee640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9f01547f0 0x7fe9f0154c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:29.183 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.176+0000 7fe9f51ee640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f01551b0 con 0x7fe9f01547f0 2023-11-06T02:09:29.184 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.176+0000 7fe9f51ee640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f01f4740 con 0x7fe9f0152e60 2023-11-06T02:09:29.184 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.176+0000 7fe9f51ee640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f0151dc0 con 0x7fe9d80047d0 2023-11-06T02:09:29.184 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.176+0000 7fe9ef59e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9f01547f0 0x7fe9f0154c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:29.184 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.176+0000 7fe9ef59e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9f01547f0 0x7fe9f0154c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40530/0 (socket says 172.21.15.49:40530) 2023-11-06T02:09:29.184 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.176+0000 7fe9ef59e640 1 -- 172.21.15.49:0/1505526614 learned_addr learned my addr 172.21.15.49:0/1505526614 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:29.185 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.176+0000 7fe9eed9d640 1 --2- 172.21.15.49:0/1505526614 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe9d80047d0 0x7fe9f01f4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:29.185 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.176+0000 7fe9ee59c640 1 --2- 172.21.15.49:0/1505526614 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe9f0152e60 0x7fe9f01f0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:29.185 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.176+0000 7fe9ef59e640 1 -- 172.21.15.49:0/1505526614 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe9d80047d0 msgr2=0x7fe9f01f4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:29.185 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.176+0000 7fe9ef59e640 1 --2- 172.21.15.49:0/1505526614 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe9d80047d0 0x7fe9f01f4200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:29.185 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.176+0000 7fe9ef59e640 1 -- 172.21.15.49:0/1505526614 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe9f0152e60 msgr2=0x7fe9f01f0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:29.186 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.176+0000 7fe9ef59e640 1 --2- 172.21.15.49:0/1505526614 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe9f0152e60 0x7fe9f01f0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:29.186 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.176+0000 7fe9ef59e640 1 -- 172.21.15.49:0/1505526614 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9f01520a0 con 0x7fe9f01547f0 2023-11-06T02:09:29.186 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.176+0000 7fe9ef59e640 1 --2- 172.21.15.49:0/1505526614 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9f01547f0 0x7fe9f0154c70 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8004750 tx=0x7fe9e800eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:29.186 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.176+0000 7fe9d77fe640 1 -- 172.21.15.49:0/1505526614 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e800eed0 con 0x7fe9f01547f0 2023-11-06T02:09:29.186 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.176+0000 7fe9d77fe640 1 -- 172.21.15.49:0/1505526614 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9e8003980 con 0x7fe9f01547f0 2023-11-06T02:09:29.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.176+0000 7fe9d77fe640 1 -- 172.21.15.49:0/1505526614 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e8003d30 con 0x7fe9f01547f0 2023-11-06T02:09:29.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.176+0000 7fe9f51ee640 1 -- 172.21.15.49:0/1505526614 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9f0152380 con 0x7fe9f01547f0 2023-11-06T02:09:29.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.176+0000 7fe9f51ee640 1 -- 172.21.15.49:0/1505526614 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9f0152940 con 0x7fe9f01547f0 2023-11-06T02:09:29.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.176+0000 7fe9f51ee640 1 -- 172.21.15.49:0/1505526614 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9f0074040 con 0x7fe9f01547f0 2023-11-06T02:09:29.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.180+0000 7fe9d77fe640 1 -- 172.21.15.49:0/1505526614 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fe9e80050d0 con 0x7fe9f01547f0 2023-11-06T02:09:29.188 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.180+0000 7fe9d77fe640 1 --2- 172.21.15.49:0/1505526614 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe9dc065a60 0x7fe9dc067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:29.188 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.180+0000 7fe9d77fe640 1 -- 172.21.15.49:0/1505526614 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fe9e80830b0 con 0x7fe9f01547f0 2023-11-06T02:09:29.188 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.180+0000 7fe9eed9d640 1 --2- 172.21.15.49:0/1505526614 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe9dc065a60 0x7fe9dc067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:29.188 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.180+0000 7fe9eed9d640 1 --2- 172.21.15.49:0/1505526614 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe9dc065a60 0x7fe9dc067f20 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe9e0009a90 tx=0x7fe9e0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:29.189 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.184+0000 7fe9d77fe640 1 -- 172.21.15.49:0/1505526614 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fe9e80508d0 con 0x7fe9f01547f0 2023-11-06T02:09:29.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.376+0000 7fe9f51ee640 1 -- 172.21.15.49:0/1505526614 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fe9f006ffb0 con 0x7fe9f01547f0 2023-11-06T02:09:29.378 INFO:tasks.ceph.mon.a.smithi049.stderr:2023-11-06T02:09:29.380+0000 7f0164e5a640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-06T02:09:29.395 INFO:tasks.ceph.mds.a.smithi049.stderr: -3289> 2023-11-06T02:08:33.183+0000 7f5e10569640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-06T02:09:29.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.396+0000 7fe9d77fe640 1 -- 172.21.15.49:0/1505526614 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v8) v1 ==== 161+0+0 (secure 0 0 0) 0x7fe9e8053ef0 con 0x7fe9f01547f0 2023-11-06T02:09:29.396 INFO:teuthology.orchestra.run.smithi049.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-06T02:09:29.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.400+0000 7fe9f51ee640 1 -- 172.21.15.49:0/1505526614 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe9dc065a60 msgr2=0x7fe9dc067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:29.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.400+0000 7fe9f51ee640 1 --2- 172.21.15.49:0/1505526614 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe9dc065a60 0x7fe9dc067f20 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe9e0009a90 tx=0x7fe9e0008040 comp rx=0 tx=0).stop 2023-11-06T02:09:29.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.400+0000 7fe9f51ee640 1 -- 172.21.15.49:0/1505526614 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9f01547f0 msgr2=0x7fe9f0154c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:29.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.400+0000 7fe9f51ee640 1 --2- 172.21.15.49:0/1505526614 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9f01547f0 0x7fe9f0154c70 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8004750 tx=0x7fe9e800eb90 comp rx=0 tx=0).stop 2023-11-06T02:09:29.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.400+0000 7fe9f51ee640 1 -- 172.21.15.49:0/1505526614 shutdown_connections 2023-11-06T02:09:29.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.400+0000 7fe9f51ee640 1 --2- 172.21.15.49:0/1505526614 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe9dc065a60 0x7fe9dc067f20 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:29.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.400+0000 7fe9f51ee640 1 --2- 172.21.15.49:0/1505526614 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9f01547f0 0x7fe9f0154c70 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:29.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.400+0000 7fe9f51ee640 1 --2- 172.21.15.49:0/1505526614 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe9f0152e60 0x7fe9f01f0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:29.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.400+0000 7fe9f51ee640 1 --2- 172.21.15.49:0/1505526614 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe9d80047d0 0x7fe9f01f4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:29.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.400+0000 7fe9f51ee640 1 -- 172.21.15.49:0/1505526614 >> 172.21.15.49:0/1505526614 conn(0x7fe9f00d08f0 msgr2=0x7fe9f00ae5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:29.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.400+0000 7fe9f51ee640 1 -- 172.21.15.49:0/1505526614 shutdown_connections 2023-11-06T02:09:29.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.400+0000 7fe9f51ee640 1 -- 172.21.15.49:0/1505526614 wait complete. 2023-11-06T02:09:29.419 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-06T02:09:29.427 INFO:tasks.ceph.mds.a.smithi049.stdout:starting mds.a at 2023-11-06T02:09:29.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.540+0000 7f6248197640 1 Processor -- start 2023-11-06T02:09:29.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.540+0000 7f6248197640 1 -- start start 2023-11-06T02:09:29.542 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.544+0000 7f6248197640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f62400c9600 0x7f62400cb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:29.542 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.544+0000 7f6248197640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f62400cb5c0 0x7f62400c99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:29.543 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.544+0000 7f6248197640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f6240065930 0x7f62400c9f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:29.543 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.544+0000 7f6248197640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f624006def0 con 0x7f6240065930 2023-11-06T02:09:29.543 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.544+0000 7f6248197640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f624015b970 con 0x7f62400c9600 2023-11-06T02:09:29.543 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.544+0000 7f6248197640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f624006cee0 con 0x7f62400cb5c0 2023-11-06T02:09:29.543 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.544+0000 7f6246734640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f6240065930 0x7f62400c9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:29.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.544+0000 7f6246734640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f6240065930 0x7f62400c9f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40560/0 (socket says 172.21.15.49:40560) 2023-11-06T02:09:29.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.544+0000 7f6246734640 1 -- 172.21.15.49:0/1906488906 learned_addr learned my addr 172.21.15.49:0/1906488906 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:29.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.544+0000 7f6245732640 1 --2- 172.21.15.49:0/1906488906 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f62400cb5c0 0x7f62400c99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:29.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.544+0000 7f6245f33640 1 --2- 172.21.15.49:0/1906488906 >> v2:172.21.15.150:3300/0 conn(0x7f62400c9600 0x7f62400cb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:29.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.544+0000 7f6246734640 1 -- 172.21.15.49:0/1906488906 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f62400cb5c0 msgr2=0x7f62400c99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:29.545 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.544+0000 7f6246734640 1 --2- 172.21.15.49:0/1906488906 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f62400cb5c0 0x7f62400c99e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:29.545 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.544+0000 7f6246734640 1 -- 172.21.15.49:0/1906488906 >> v2:172.21.15.150:3300/0 conn(0x7f62400c9600 msgr2=0x7f62400cb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:29.545 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.544+0000 7f6246734640 1 --2- 172.21.15.49:0/1906488906 >> v2:172.21.15.150:3300/0 conn(0x7f62400c9600 0x7f62400cb080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:29.545 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.544+0000 7f6246734640 1 -- 172.21.15.49:0/1906488906 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62400cabf0 con 0x7f6240065930 2023-11-06T02:09:29.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.544+0000 7f6246734640 1 --2- 172.21.15.49:0/1906488906 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6240065930 0x7f62400c9f20 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f62400cee60 tx=0x7f623801c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b84f8413f7b0e61 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:29.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.544+0000 7f6244f31640 1 -- 172.21.15.49:0/1906488906 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62380a5650 con 0x7f6240065930 2023-11-06T02:09:29.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.544+0000 7f6244f31640 1 -- 172.21.15.49:0/1906488906 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62380a57b0 con 0x7f6240065930 2023-11-06T02:09:29.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.544+0000 7f6244f31640 1 -- 172.21.15.49:0/1906488906 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62380a5b40 con 0x7f6240065930 2023-11-06T02:09:29.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.544+0000 7f6248197640 1 -- 172.21.15.49:0/1906488906 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6240065930 msgr2=0x7f62400c9f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:29.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.544+0000 7f6248197640 1 --2- 172.21.15.49:0/1906488906 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6240065930 0x7f62400c9f20 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f62400cee60 tx=0x7f623801c9d0 comp rx=0 tx=0).stop 2023-11-06T02:09:29.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.548+0000 7f6248197640 1 -- 172.21.15.49:0/1906488906 shutdown_connections 2023-11-06T02:09:29.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.548+0000 7f6248197640 1 --2- 172.21.15.49:0/1906488906 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f62400cb5c0 0x7f62400c99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:29.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.548+0000 7f6248197640 1 --2- 172.21.15.49:0/1906488906 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6240065930 0x7f62400c9f20 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:29.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.548+0000 7f6248197640 1 --2- 172.21.15.49:0/1906488906 >> v2:172.21.15.150:3300/0 conn(0x7f62400c9600 0x7f62400cb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:29.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.548+0000 7f6248197640 1 -- 172.21.15.49:0/1906488906 >> 172.21.15.49:0/1906488906 conn(0x7f62400b0aa0 msgr2=0x7f62400b0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:29.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.548+0000 7f6248197640 1 -- 172.21.15.49:0/1906488906 shutdown_connections 2023-11-06T02:09:29.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.548+0000 7f6248197640 1 -- 172.21.15.49:0/1906488906 wait complete. 2023-11-06T02:09:29.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.548+0000 7f6248197640 1 Processor -- start 2023-11-06T02:09:29.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.548+0000 7f6248197640 1 -- start start 2023-11-06T02:09:29.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.548+0000 7f6248197640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6240065930 0x7f624006ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:29.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.548+0000 7f6248197640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f62400c9600 0x7f62400ac8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:29.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.548+0000 7f6248197640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f62400cb5c0 0x7f62400af180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:29.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.548+0000 7f6248197640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62400ace30 con 0x7f6240065930 2023-11-06T02:09:29.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.548+0000 7f6248197640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62400c50c0 con 0x7f62400c9600 2023-11-06T02:09:29.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.548+0000 7f6248197640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62400c53a0 con 0x7f62400cb5c0 2023-11-06T02:09:29.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.548+0000 7f6245f33640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6240065930 0x7f624006ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:29.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.548+0000 7f6245f33640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6240065930 0x7f624006ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40570/0 (socket says 172.21.15.49:40570) 2023-11-06T02:09:29.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.548+0000 7f6245f33640 1 -- 172.21.15.49:0/3748017356 learned_addr learned my addr 172.21.15.49:0/3748017356 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:29.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.548+0000 7f6245732640 1 --2- 172.21.15.49:0/3748017356 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f62400c9600 0x7f62400ac8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:29.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.548+0000 7f6246734640 1 --2- 172.21.15.49:0/3748017356 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f62400cb5c0 0x7f62400af180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:29.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.548+0000 7f6245f33640 1 -- 172.21.15.49:0/3748017356 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f62400cb5c0 msgr2=0x7f62400af180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:29.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.548+0000 7f6245f33640 1 --2- 172.21.15.49:0/3748017356 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f62400cb5c0 0x7f62400af180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:29.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.548+0000 7f6245f33640 1 -- 172.21.15.49:0/3748017356 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f62400c9600 msgr2=0x7f62400ac8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:29.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.548+0000 7f6245f33640 1 --2- 172.21.15.49:0/3748017356 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f62400c9600 0x7f62400ac8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:29.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.548+0000 7f6245f33640 1 -- 172.21.15.49:0/3748017356 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6238009b30 con 0x7f6240065930 2023-11-06T02:09:29.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.548+0000 7f6246734640 1 --2- 172.21.15.49:0/3748017356 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f62400cb5c0 0x7f62400af180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:09:29.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.548+0000 7f6245f33640 1 --2- 172.21.15.49:0/3748017356 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6240065930 0x7f624006ab40 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f623c00d800 tx=0x7f623c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:29.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.548+0000 7f62367fc640 1 -- 172.21.15.49:0/3748017356 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f623c0096a0 con 0x7f6240065930 2023-11-06T02:09:29.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.548+0000 7f62367fc640 1 -- 172.21.15.49:0/3748017356 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f623c009800 con 0x7f6240065930 2023-11-06T02:09:29.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.548+0000 7f6248197640 1 -- 172.21.15.49:0/3748017356 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62400c5680 con 0x7f6240065930 2023-11-06T02:09:29.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.548+0000 7f62367fc640 1 -- 172.21.15.49:0/3748017356 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f623c009990 con 0x7f6240065930 2023-11-06T02:09:29.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.548+0000 7f6248197640 1 -- 172.21.15.49:0/3748017356 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6240157220 con 0x7f6240065930 2023-11-06T02:09:29.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.552+0000 7f6248197640 1 -- 172.21.15.49:0/3748017356 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62400b4c00 con 0x7f6240065930 2023-11-06T02:09:29.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.552+0000 7f62367fc640 1 -- 172.21.15.49:0/3748017356 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f623c009af0 con 0x7f6240065930 2023-11-06T02:09:29.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.552+0000 7f62367fc640 1 --2- 172.21.15.49:0/3748017356 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6224065a60 0x7f6224067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:29.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.552+0000 7f62367fc640 1 -- 172.21.15.49:0/3748017356 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6206+0+0 (secure 0 0 0) 0x7f623c083770 con 0x7f6240065930 2023-11-06T02:09:29.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.552+0000 7f6245732640 1 --2- 172.21.15.49:0/3748017356 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6224065a60 0x7f6224067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:29.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.552+0000 7f6245732640 1 --2- 172.21.15.49:0/3748017356 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6224065a60 0x7f6224067f20 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f62400c9180 tx=0x7f6230008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:29.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.556+0000 7f62367fc640 1 -- 172.21.15.49:0/3748017356 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f623c050fd0 con 0x7f6240065930 2023-11-06T02:09:29.743 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:29.744+0000 7f6248197640 1 -- 172.21.15.49:0/3748017356 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f624006d750 con 0x7f6240065930 2023-11-06T02:09:30.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:30.400+0000 7f62367fc640 1 -- 172.21.15.49:0/3748017356 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v9) v1 ==== 104+0+0 (secure 0 0 0) 0x7f623c0545f0 con 0x7f6240065930 2023-11-06T02:09:30.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:30.400+0000 7f6248197640 1 -- 172.21.15.49:0/3748017356 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6224065a60 msgr2=0x7f6224067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:30.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:30.400+0000 7f6248197640 1 --2- 172.21.15.49:0/3748017356 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6224065a60 0x7f6224067f20 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f62400c9180 tx=0x7f6230008040 comp rx=0 tx=0).stop 2023-11-06T02:09:30.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:30.400+0000 7f6248197640 1 -- 172.21.15.49:0/3748017356 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6240065930 msgr2=0x7f624006ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:30.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:30.400+0000 7f6248197640 1 --2- 172.21.15.49:0/3748017356 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6240065930 0x7f624006ab40 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f623c00d800 tx=0x7f623c00dcc0 comp rx=0 tx=0).stop 2023-11-06T02:09:30.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:30.404+0000 7f6248197640 1 -- 172.21.15.49:0/3748017356 shutdown_connections 2023-11-06T02:09:30.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:30.404+0000 7f6248197640 1 --2- 172.21.15.49:0/3748017356 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6224065a60 0x7f6224067f20 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:30.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:30.404+0000 7f6248197640 1 --2- 172.21.15.49:0/3748017356 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f62400cb5c0 0x7f62400af180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:30.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:30.404+0000 7f6248197640 1 --2- 172.21.15.49:0/3748017356 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f62400c9600 0x7f62400ac8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:30.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:30.404+0000 7f6248197640 1 --2- 172.21.15.49:0/3748017356 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6240065930 0x7f624006ab40 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:30.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:30.404+0000 7f6248197640 1 -- 172.21.15.49:0/3748017356 >> 172.21.15.49:0/3748017356 conn(0x7f62400b0aa0 msgr2=0x7f624009e200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:30.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:30.404+0000 7f6248197640 1 -- 172.21.15.49:0/3748017356 shutdown_connections 2023-11-06T02:09:30.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:30.404+0000 7f6248197640 1 -- 172.21.15.49:0/3748017356 wait complete. 2023-11-06T02:09:30.424 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-06T02:09:30.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:30.604+0000 7fe0d007c640 1 Processor -- start 2023-11-06T02:09:30.604 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:30.604+0000 7fe0d007c640 1 -- start start 2023-11-06T02:09:30.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:30.604+0000 7fe0d007c640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fe0c815abd0 0x7fe0c8069d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:30.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:30.604+0000 7fe0d007c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe0c815e780 0x7fe0c806a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:30.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:30.604+0000 7fe0d007c640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0c80c94b0 con 0x7fe0c80ad720 2023-11-06T02:09:30.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:30.604+0000 7fe0d007c640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0c815ecd0 con 0x7fe0c815abd0 2023-11-06T02:09:30.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:30.604+0000 7fe0d007c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c815efb0 con 0x7fe0c815e780 2023-11-06T02:09:30.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:30.604+0000 7fe0cd617640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe0c815e780 0x7fe0c806a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:30.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:30.604+0000 7fe0cde18640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fe0c815abd0 0x7fe0c8069d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:30.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:30.604+0000 7fe0cd617640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe0c815e780 0x7fe0c806a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.49:55190/0 (socket says 172.21.15.49:55190) 2023-11-06T02:09:30.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:30.604+0000 7fe0ce619640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7fe0c80ad720 0x7fe0c80b0090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.49:49094/0 (socket says 172.21.15.49:49094) 2023-11-06T02:09:30.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:30.604+0000 7fe0cd617640 1 -- 172.21.15.49:0/421570586 learned_addr learned my addr 172.21.15.49:0/421570586 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:30.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:30.608+0000 7fe0cd617640 1 -- 172.21.15.49:0/421570586 >> v2:172.21.15.150:3300/0 conn(0x7fe0c815abd0 msgr2=0x7fe0c8069d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:30.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:30.608+0000 7fe0cd617640 1 --2- 172.21.15.49:0/421570586 >> v2:172.21.15.150:3300/0 conn(0x7fe0c815abd0 0x7fe0c8069d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:30.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:30.608+0000 7fe0cd617640 1 -- 172.21.15.49:0/421570586 >> v1:172.21.15.150:6789/0 conn(0x7fe0c80ad720 legacy=0x7fe0c80b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:30.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:30.608+0000 7fe0cd617640 1 -- 172.21.15.49:0/421570586 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0c815f290 con 0x7fe0c815e780 2023-11-06T02:09:30.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:30.608+0000 7fe0cde18640 1 --2- 172.21.15.49:0/421570586 >> v2:172.21.15.150:3300/0 conn(0x7fe0c815abd0 0x7fe0c8069d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T02:09:30.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:30.608+0000 7fe0cd617640 1 --2- 172.21.15.49:0/421570586 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe0c815e780 0x7fe0c806a2b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe0c4009870 tx=0x7fe0c401c3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9aa90f63b7c10918 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:30.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:30.608+0000 7fe0cce16640 1 -- 172.21.15.49:0/421570586 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c40ab610 con 0x7fe0c815e780 2023-11-06T02:09:30.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:30.608+0000 7fe0cce16640 1 -- 172.21.15.49:0/421570586 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0c40ab770 con 0x7fe0c815e780 2023-11-06T02:09:30.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:30.608+0000 7fe0cce16640 1 -- 172.21.15.49:0/421570586 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c40abb00 con 0x7fe0c815e780 2023-11-06T02:09:30.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:30.608+0000 7fe0d007c640 1 -- 172.21.15.49:0/421570586 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe0c815e780 msgr2=0x7fe0c806a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:30.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:30.608+0000 7fe0d007c640 1 --2- 172.21.15.49:0/421570586 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe0c815e780 0x7fe0c806a2b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe0c4009870 tx=0x7fe0c401c3f0 comp rx=0 tx=0).stop 2023-11-06T02:09:30.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:30.608+0000 7fe0d007c640 1 -- 172.21.15.49:0/421570586 shutdown_connections 2023-11-06T02:09:30.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:30.608+0000 7fe0d007c640 1 --2- 172.21.15.49:0/421570586 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe0c815e780 0x7fe0c806a2b0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:30.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:30.608+0000 7fe0d007c640 1 --2- 172.21.15.49:0/421570586 >> v2:172.21.15.150:3300/0 conn(0x7fe0c815abd0 0x7fe0c8069d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:30.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:30.608+0000 7fe0d007c640 1 -- 172.21.15.49:0/421570586 >> 172.21.15.49:0/421570586 conn(0x7fe0c80ce610 msgr2=0x7fe0c80cea20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:30.612 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:30.608+0000 7fe0d007c640 1 -- 172.21.15.49:0/421570586 shutdown_connections 2023-11-06T02:09:30.612 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:30.608+0000 7fe0d007c640 1 -- 172.21.15.49:0/421570586 wait complete. 2023-11-06T02:09:30.612 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:30.612+0000 7fe0d007c640 1 Processor -- start 2023-11-06T02:09:30.612 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:30.612+0000 7fe0d007c640 1 -- start start 2023-11-06T02:09:30.615 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:30.612+0000 7fe0d007c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe0c80ad720 0x7fe0c81f9020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:30.615 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:30.612+0000 7fe0d007c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe0c815abd0 0x7fe0c81f9560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:30.615 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:30.612+0000 7fe0d007c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe0c815e780 0x7fe0c81f9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:30.615 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:30.612+0000 7fe0d007c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c80cd290 con 0x7fe0c80ad720 2023-11-06T02:09:30.616 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:30.612+0000 7fe0d007c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c80cd570 con 0x7fe0c815abd0 2023-11-06T02:09:30.616 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:30.612+0000 7fe0d007c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c80cd850 con 0x7fe0c815e780 2023-11-06T02:09:30.616 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:30.612+0000 7fe0cde18640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe0c80ad720 0x7fe0c81f9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:30.616 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:30.612+0000 7fe0cde18640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe0c80ad720 0x7fe0c81f9020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40580/0 (socket says 172.21.15.49:40580) 2023-11-06T02:09:30.616 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:30.612+0000 7fe0cde18640 1 -- 172.21.15.49:0/2042833162 learned_addr learned my addr 172.21.15.49:0/2042833162 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:30.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:30.612+0000 7fe0ce619640 1 --2- 172.21.15.49:0/2042833162 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe0c815e780 0x7fe0c81f9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:30.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:30.612+0000 7fe0cde18640 1 -- 172.21.15.49:0/2042833162 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe0c815e780 msgr2=0x7fe0c81f9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:30.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:30.612+0000 7fe0cde18640 1 --2- 172.21.15.49:0/2042833162 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe0c815e780 0x7fe0c81f9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:30.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:30.612+0000 7fe0cde18640 1 -- 172.21.15.49:0/2042833162 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe0c815abd0 msgr2=0x7fe0c81f9560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:30.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:30.612+0000 7fe0cd617640 1 --2- 172.21.15.49:0/2042833162 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe0c815abd0 0x7fe0c81f9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:30.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:30.612+0000 7fe0cde18640 1 --2- 172.21.15.49:0/2042833162 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe0c815abd0 0x7fe0c81f9560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:30.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:30.612+0000 7fe0cde18640 1 -- 172.21.15.49:0/2042833162 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0bc009420 con 0x7fe0c80ad720 2023-11-06T02:09:30.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:30.612+0000 7fe0cd617640 1 --2- 172.21.15.49:0/2042833162 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe0c815abd0 0x7fe0c81f9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:09:30.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:30.612+0000 7fe0cde18640 1 --2- 172.21.15.49:0/2042833162 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe0c80ad720 0x7fe0c81f9020 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc00ebe0 tx=0x7fe0bc00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:30.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:30.612+0000 7fe0ba7fc640 1 -- 172.21.15.49:0/2042833162 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0bc00eee0 con 0x7fe0c80ad720 2023-11-06T02:09:30.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:30.612+0000 7fe0ba7fc640 1 -- 172.21.15.49:0/2042833162 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0bc00cc30 con 0x7fe0c80ad720 2023-11-06T02:09:30.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:30.612+0000 7fe0d007c640 1 -- 172.21.15.49:0/2042833162 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0c40094d0 con 0x7fe0c80ad720 2023-11-06T02:09:30.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:30.612+0000 7fe0ba7fc640 1 -- 172.21.15.49:0/2042833162 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0bc013470 con 0x7fe0c80ad720 2023-11-06T02:09:30.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:30.612+0000 7fe0d007c640 1 -- 172.21.15.49:0/2042833162 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0c80cde10 con 0x7fe0c80ad720 2023-11-06T02:09:30.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:30.612+0000 7fe0d007c640 1 -- 172.21.15.49:0/2042833162 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0c8074040 con 0x7fe0c80ad720 2023-11-06T02:09:30.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:30.616+0000 7fe0ba7fc640 1 -- 172.21.15.49:0/2042833162 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fe0bc0135d0 con 0x7fe0c80ad720 2023-11-06T02:09:30.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:30.616+0000 7fe0ba7fc640 1 --2- 172.21.15.49:0/2042833162 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe0ac065a60 0x7fe0ac067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:30.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:30.616+0000 7fe0ba7fc640 1 -- 172.21.15.49:0/2042833162 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6206+0+0 (secure 0 0 0) 0x7fe0bc0836f0 con 0x7fe0c80ad720 2023-11-06T02:09:30.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:30.616+0000 7fe0cd617640 1 --2- 172.21.15.49:0/2042833162 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe0ac065a60 0x7fe0ac067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:30.621 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:30.616+0000 7fe0cd617640 1 --2- 172.21.15.49:0/2042833162 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe0ac065a60 0x7fe0ac067f20 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe0c40af000 tx=0x7fe0c40a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:30.621 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:30.620+0000 7fe0ba7fc640 1 -- 172.21.15.49:0/2042833162 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fe0bc050ec0 con 0x7fe0c80ad720 2023-11-06T02:09:30.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:30.776+0000 7fe0d007c640 1 -- 172.21.15.49:0/2042833162 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe0c80c8d80 con 0x7fe0c80ad720 2023-11-06T02:09:31.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:31.404+0000 7fe0ba7fc640 1 -- 172.21.15.49:0/2042833162 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v27) v1 ==== 181+0+0 (secure 0 0 0) 0x7fe0bc0544e0 con 0x7fe0c80ad720 2023-11-06T02:09:31.403 INFO:teuthology.orchestra.run.smithi049.stderr:pool 'cephfs_metadata' removed 2023-11-06T02:09:31.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:31.408+0000 7fe0d007c640 1 -- 172.21.15.49:0/2042833162 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe0ac065a60 msgr2=0x7fe0ac067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:31.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:31.408+0000 7fe0d007c640 1 --2- 172.21.15.49:0/2042833162 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe0ac065a60 0x7fe0ac067f20 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe0c40af000 tx=0x7fe0c40a7000 comp rx=0 tx=0).stop 2023-11-06T02:09:31.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:31.408+0000 7fe0d007c640 1 -- 172.21.15.49:0/2042833162 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe0c80ad720 msgr2=0x7fe0c81f9020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:31.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:31.408+0000 7fe0d007c640 1 --2- 172.21.15.49:0/2042833162 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe0c80ad720 0x7fe0c81f9020 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc00ebe0 tx=0x7fe0bc00c520 comp rx=0 tx=0).stop 2023-11-06T02:09:31.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:31.412+0000 7fe0d007c640 1 -- 172.21.15.49:0/2042833162 shutdown_connections 2023-11-06T02:09:31.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:31.412+0000 7fe0d007c640 1 --2- 172.21.15.49:0/2042833162 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe0ac065a60 0x7fe0ac067f20 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:31.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:31.412+0000 7fe0d007c640 1 --2- 172.21.15.49:0/2042833162 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe0c815e780 0x7fe0c81f9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:31.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:31.412+0000 7fe0d007c640 1 --2- 172.21.15.49:0/2042833162 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe0c815abd0 0x7fe0c81f9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:31.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:31.412+0000 7fe0d007c640 1 --2- 172.21.15.49:0/2042833162 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe0c80ad720 0x7fe0c81f9020 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:31.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:31.412+0000 7fe0d007c640 1 -- 172.21.15.49:0/2042833162 >> 172.21.15.49:0/2042833162 conn(0x7fe0c80ce610 msgr2=0x7fe0c80b1660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:31.411 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:31.412+0000 7fe0d007c640 1 -- 172.21.15.49:0/2042833162 shutdown_connections 2023-11-06T02:09:31.411 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:31.412+0000 7fe0d007c640 1 -- 172.21.15.49:0/2042833162 wait complete. 2023-11-06T02:09:31.432 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-06T02:09:31.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:31.564+0000 7f4a01298640 1 Processor -- start 2023-11-06T02:09:31.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:31.564+0000 7f4a01298640 1 -- start start 2023-11-06T02:09:31.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:31.568+0000 7f4a01298640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f49fc0cd060 0x7f49fc15d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:31.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:31.568+0000 7f4a01298640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49fc0cd4a0 con 0x7f49fc0714a0 2023-11-06T02:09:31.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:31.568+0000 7f4a01298640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49fc15da20 con 0x7f49fc0d25c0 2023-11-06T02:09:31.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:31.568+0000 7f4a01298640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49fc15d830 con 0x7f49fc0cd060 2023-11-06T02:09:31.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:31.568+0000 7f49faffd640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f49fc0714a0 0x7f49fc071880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:51418/0 (socket says 172.21.15.49:51418) 2023-11-06T02:09:31.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:31.568+0000 7f49faffd640 1 -- 172.21.15.49:0/3585550947 learned_addr learned my addr 172.21.15.49:0/3585550947 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:09:31.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:31.568+0000 7f49fb7fe640 1 --2- 172.21.15.49:0/3585550947 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f49fc0cd060 0x7f49fc15d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:31.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:31.568+0000 7f49f9ffb640 1 -- 172.21.15.49:0/3585550947 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2527707332 0 0) 0x7f49ec001120 con 0x7f49fc0714a0 2023-11-06T02:09:31.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:31.568+0000 7f49f9ffb640 1 -- 172.21.15.49:0/3585550947 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49e4003530 con 0x7f49fc0714a0 2023-11-06T02:09:31.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:31.568+0000 7f49f9ffb640 1 -- 172.21.15.49:0/3585550947 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 26758097 0 0) 0x7f49e4003530 con 0x7f49fc0714a0 2023-11-06T02:09:31.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:31.568+0000 7f49f9ffb640 1 -- 172.21.15.49:0/3585550947 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f49fc0cd060 msgr2=0x7f49fc15d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:31.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:31.568+0000 7f49f9ffb640 1 --2- 172.21.15.49:0/3585550947 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f49fc0cd060 0x7f49fc15d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:31.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:31.568+0000 7f49f9ffb640 1 -- 172.21.15.49:0/3585550947 >> v1:172.21.15.150:6789/0 conn(0x7f49fc0d25c0 legacy=0x7f49fc0d0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:31.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:31.568+0000 7f49f9ffb640 1 -- 172.21.15.49:0/3585550947 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49fc15dd00 con 0x7f49fc0714a0 2023-11-06T02:09:31.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:31.568+0000 7f49f9ffb640 1 -- 172.21.15.49:0/3585550947 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7f49ec001810 con 0x7f49fc0714a0 2023-11-06T02:09:31.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:31.568+0000 7f49f9ffb640 1 -- 172.21.15.49:0/3585550947 >> v1:172.21.15.49:6789/0 conn(0x7f49fc0714a0 legacy=0x7f49fc071880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:31.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:31.568+0000 7f49f9ffb640 1 --2- 172.21.15.49:0/3585550947 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49e40039b0 0x7f49e4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:31.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:31.568+0000 7f49f9ffb640 1 --2- 172.21.15.49:0/3585550947 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f49e40047d0 0x7f49e4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:31.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:31.568+0000 7f49f9ffb640 1 --2- 172.21.15.49:0/3585550947 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f49fc0cd060 0x7f49e40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:31.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:31.568+0000 7f49f9ffb640 1 -- 172.21.15.49:0/3585550947 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e400bac0 con 0x7f49e40039b0 2023-11-06T02:09:31.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:31.568+0000 7f49f9ffb640 1 -- 172.21.15.49:0/3585550947 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e400bc30 con 0x7f49e40047d0 2023-11-06T02:09:31.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:31.568+0000 7f49f9ffb640 1 -- 172.21.15.49:0/3585550947 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49e400bdf0 con 0x7f49fc0cd060 2023-11-06T02:09:31.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:31.568+0000 7f49faffd640 1 --2- 172.21.15.49:0/3585550947 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49e40039b0 0x7f49e4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:31.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:31.568+0000 7f49fb7fe640 1 --2- 172.21.15.49:0/3585550947 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f49fc0cd060 0x7f49e40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:31.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:31.568+0000 7f49fa7fc640 1 --2- 172.21.15.49:0/3585550947 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f49e40047d0 0x7f49e4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:31.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:31.568+0000 7f49faffd640 1 -- 172.21.15.49:0/3585550947 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f49fc0cd060 msgr2=0x7f49e40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:31.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:31.568+0000 7f49faffd640 1 --2- 172.21.15.49:0/3585550947 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f49fc0cd060 0x7f49e40051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:31.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:31.568+0000 7f49faffd640 1 -- 172.21.15.49:0/3585550947 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f49e40047d0 msgr2=0x7f49e4004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:31.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:31.568+0000 7f49faffd640 1 --2- 172.21.15.49:0/3585550947 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f49e40047d0 0x7f49e4004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:31.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:31.568+0000 7f49faffd640 1 -- 172.21.15.49:0/3585550947 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49e4003530 con 0x7f49e40039b0 2023-11-06T02:09:31.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:31.572+0000 7f49faffd640 1 --2- 172.21.15.49:0/3585550947 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49e40039b0 0x7f49e4003d90 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f49ec00c160 tx=0x7f49ec01cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2be6a938b9458429 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:31.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:31.572+0000 7f49f9ffb640 1 -- 172.21.15.49:0/3585550947 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ec005ae0 con 0x7f49e40039b0 2023-11-06T02:09:31.579 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:31.572+0000 7f49f9ffb640 1 -- 172.21.15.49:0/3585550947 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49ec005c60 con 0x7f49e40039b0 2023-11-06T02:09:31.579 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:31.572+0000 7f49f9ffb640 1 -- 172.21.15.49:0/3585550947 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ec0a57a0 con 0x7f49e40039b0 2023-11-06T02:09:31.579 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:31.572+0000 7f4a01298640 1 -- 172.21.15.49:0/3585550947 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49e40039b0 msgr2=0x7f49e4003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:31.579 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:31.572+0000 7f4a01298640 1 --2- 172.21.15.49:0/3585550947 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49e40039b0 0x7f49e4003d90 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f49ec00c160 tx=0x7f49ec01cbb0 comp rx=0 tx=0).stop 2023-11-06T02:09:31.580 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:31.572+0000 7f49fb7fe640 1 -- 172.21.15.49:0/3585550947 reap_dead start 2023-11-06T02:09:31.580 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:31.572+0000 7f4a01298640 1 -- 172.21.15.49:0/3585550947 shutdown_connections 2023-11-06T02:09:31.580 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:31.572+0000 7f4a01298640 1 -- 172.21.15.49:0/3585550947 >> 172.21.15.49:0/3585550947 conn(0x7f49fc0d2aa0 msgr2=0x7f49fc0d2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:31.580 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:31.572+0000 7f4a01298640 1 -- 172.21.15.49:0/3585550947 shutdown_connections 2023-11-06T02:09:31.580 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:31.572+0000 7f4a01298640 1 -- 172.21.15.49:0/3585550947 wait complete. 2023-11-06T02:09:31.581 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:31.572+0000 7f4a01298640 1 Processor -- start 2023-11-06T02:09:31.581 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:31.572+0000 7f4a01298640 1 -- start start 2023-11-06T02:09:31.581 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:31.572+0000 7f4a01298640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49fc151ad0 0x7f49fc0b7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:31.581 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:31.572+0000 7f4a01298640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f49fc0b60e0 0x7f49fc0b6560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:31.582 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:31.572+0000 7f4a01298640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f49fc159170 0x7f49fc1595f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:31.582 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:31.572+0000 7f4a01298640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49fc0b6aa0 con 0x7f49fc151ad0 2023-11-06T02:09:31.582 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:31.572+0000 7f4a01298640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49fc0b33d0 con 0x7f49fc159170 2023-11-06T02:09:31.582 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:31.572+0000 7f4a01298640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49fc0b7990 con 0x7f49fc0b60e0 2023-11-06T02:09:31.582 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:31.572+0000 7f49faffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49fc151ad0 0x7f49fc0b7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:31.583 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:31.572+0000 7f49faffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49fc151ad0 0x7f49fc0b7450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40586/0 (socket says 172.21.15.49:40586) 2023-11-06T02:09:31.583 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:31.572+0000 7f49faffd640 1 -- 172.21.15.49:0/618100642 learned_addr learned my addr 172.21.15.49:0/618100642 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:31.583 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:31.572+0000 7f49fb7fe640 1 --2- 172.21.15.49:0/618100642 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f49fc159170 0x7f49fc1595f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:31.583 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:31.572+0000 7f49fa7fc640 1 --2- 172.21.15.49:0/618100642 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f49fc0b60e0 0x7f49fc0b6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:31.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:31.572+0000 7f49faffd640 1 -- 172.21.15.49:0/618100642 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f49fc0b60e0 msgr2=0x7f49fc0b6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:31.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:31.572+0000 7f49faffd640 1 --2- 172.21.15.49:0/618100642 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f49fc0b60e0 0x7f49fc0b6560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:31.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:31.572+0000 7f49faffd640 1 -- 172.21.15.49:0/618100642 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f49fc159170 msgr2=0x7f49fc1595f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:31.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:31.572+0000 7f49faffd640 1 --2- 172.21.15.49:0/618100642 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f49fc159170 0x7f49fc1595f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:31.585 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:31.572+0000 7f49faffd640 1 -- 172.21.15.49:0/618100642 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49fc0b27c0 con 0x7f49fc151ad0 2023-11-06T02:09:31.585 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:31.572+0000 7f49fa7fc640 1 --2- 172.21.15.49:0/618100642 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f49fc0b60e0 0x7f49fc0b6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:09:31.585 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:31.576+0000 7f49faffd640 1 --2- 172.21.15.49:0/618100642 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49fc151ad0 0x7f49fc0b7450 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f49ec01c9e0 tx=0x7f49e4005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:31.585 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:31.576+0000 7f49e37fe640 1 -- 172.21.15.49:0/618100642 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ec0b4070 con 0x7f49fc151ad0 2023-11-06T02:09:31.585 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:31.576+0000 7f49e37fe640 1 -- 172.21.15.49:0/618100642 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49ec0a5e00 con 0x7f49fc151ad0 2023-11-06T02:09:31.586 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:31.576+0000 7f49e37fe640 1 -- 172.21.15.49:0/618100642 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ec0015d0 con 0x7f49fc151ad0 2023-11-06T02:09:31.586 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:31.576+0000 7f4a01298640 1 -- 172.21.15.49:0/618100642 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49fc0b2aa0 con 0x7f49fc151ad0 2023-11-06T02:09:31.586 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:31.576+0000 7f4a01298640 1 -- 172.21.15.49:0/618100642 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49fc0b3060 con 0x7f49fc151ad0 2023-11-06T02:09:31.586 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:31.576+0000 7f4a01298640 1 -- 172.21.15.49:0/618100642 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49fc074040 con 0x7f49fc151ad0 2023-11-06T02:09:31.587 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:31.576+0000 7f49e37fe640 1 -- 172.21.15.49:0/618100642 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f49ec0b7070 con 0x7f49fc151ad0 2023-11-06T02:09:31.587 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:31.576+0000 7f49e37fe640 1 --2- 172.21.15.49:0/618100642 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f49e8065a60 0x7f49e8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:31.587 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:31.576+0000 7f49e37fe640 1 -- 172.21.15.49:0/618100642 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 5768+0+0 (secure 0 0 0) 0x7f49ec11c430 con 0x7f49fc151ad0 2023-11-06T02:09:31.587 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:31.576+0000 7f49fa7fc640 1 --2- 172.21.15.49:0/618100642 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f49e8065a60 0x7f49e8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:31.587 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:31.576+0000 7f49fa7fc640 1 --2- 172.21.15.49:0/618100642 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f49e8065a60 0x7f49e8067f20 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f49f0007fd0 tx=0x7f49f0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:31.588 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:31.580+0000 7f49e37fe640 1 -- 172.21.15.49:0/618100642 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f49ec0e9db0 con 0x7f49fc151ad0 2023-11-06T02:09:31.738 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:31.740+0000 7f4a01298640 1 -- 172.21.15.49:0/618100642 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f49fc0cd4a0 con 0x7f49fc151ad0 2023-11-06T02:09:32.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:32.408+0000 7f49e37fe640 1 -- 172.21.15.49:0/618100642 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v28) v1 ==== 169+0+0 (secure 0 0 0) 0x7f49fc0cd4a0 con 0x7f49fc151ad0 2023-11-06T02:09:32.407 INFO:teuthology.orchestra.run.smithi049.stderr:pool 'cephfs_data' removed 2023-11-06T02:09:32.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:32.416+0000 7f4a01298640 1 -- 172.21.15.49:0/618100642 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f49e8065a60 msgr2=0x7f49e8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:32.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:32.416+0000 7f4a01298640 1 --2- 172.21.15.49:0/618100642 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f49e8065a60 0x7f49e8067f20 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f49f0007fd0 tx=0x7f49f0009040 comp rx=0 tx=0).stop 2023-11-06T02:09:32.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:32.416+0000 7f4a01298640 1 -- 172.21.15.49:0/618100642 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49fc151ad0 msgr2=0x7f49fc0b7450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:32.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:32.416+0000 7f4a01298640 1 --2- 172.21.15.49:0/618100642 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49fc151ad0 0x7f49fc0b7450 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f49ec01c9e0 tx=0x7f49e4005800 comp rx=0 tx=0).stop 2023-11-06T02:09:32.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:32.416+0000 7f4a01298640 1 -- 172.21.15.49:0/618100642 shutdown_connections 2023-11-06T02:09:32.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:32.416+0000 7f4a01298640 1 --2- 172.21.15.49:0/618100642 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f49e8065a60 0x7f49e8067f20 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:32.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:32.416+0000 7f4a01298640 1 --2- 172.21.15.49:0/618100642 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f49fc159170 0x7f49fc1595f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:32.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:32.416+0000 7f4a01298640 1 --2- 172.21.15.49:0/618100642 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f49fc0b60e0 0x7f49fc0b6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:32.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:32.416+0000 7f4a01298640 1 --2- 172.21.15.49:0/618100642 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49fc151ad0 0x7f49fc0b7450 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:32.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:32.416+0000 7f4a01298640 1 -- 172.21.15.49:0/618100642 >> 172.21.15.49:0/618100642 conn(0x7f49fc0d2aa0 msgr2=0x7f49fc0c9ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:32.417 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:32.416+0000 7f4a01298640 1 -- 172.21.15.49:0/618100642 shutdown_connections 2023-11-06T02:09:32.417 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:32.416+0000 7f4a01298640 1 -- 172.21.15.49:0/618100642 wait complete. 2023-11-06T02:09:32.440 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-06T02:09:32.441 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-06T02:09:32.441 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-06T02:09:32.441 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T02:09:32.442 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-06T02:09:32.442 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-06T02:09:32.442 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-06T02:09:32.442 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T02:09:32.443 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-06T02:09:32.443 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-06T02:09:32.443 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-06T02:09:32.443 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T02:09:32.444 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-06T02:09:32.444 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-06T02:09:32.444 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-06T02:09:32.445 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T02:09:32.452 INFO:tasks.ceph.mds.a:Stopped 2023-11-06T02:09:32.452 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-06T02:09:32.454 INFO:tasks.ceph.mds.c:Stopped 2023-11-06T02:09:32.454 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-06T02:09:32.463 INFO:tasks.ceph.mds.d:Stopped 2023-11-06T02:09:32.463 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-06T02:09:32.467 INFO:tasks.ceph.mds.b:Stopped 2023-11-06T02:09:32.467 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-06T02:09:32.498 INFO:tasks.ceph.mds.a:Started 2023-11-06T02:09:32.499 DEBUG:teuthology.parallel:result is None 2023-11-06T02:09:32.500 INFO:tasks.ceph.mds.c:Started 2023-11-06T02:09:32.500 DEBUG:teuthology.parallel:result is None 2023-11-06T02:09:32.514 INFO:tasks.ceph.mds.d:Started 2023-11-06T02:09:32.514 DEBUG:teuthology.parallel:result is None 2023-11-06T02:09:32.515 INFO:tasks.ceph.mds.b:Started 2023-11-06T02:09:32.516 DEBUG:teuthology.parallel:result is None 2023-11-06T02:09:32.516 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-06T02:09:32.588 INFO:tasks.ceph.mds.c.smithi049.stdout:starting mds.c at 2023-11-06T02:09:32.589 INFO:tasks.ceph.mds.d.smithi150.stdout:starting mds.d at 2023-11-06T02:09:32.589 INFO:tasks.ceph.mds.b.smithi150.stdout:starting mds.b at 2023-11-06T02:09:32.594 INFO:tasks.ceph.mds.a.smithi049.stdout:starting mds.a at 2023-11-06T02:09:32.658 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:32.660+0000 7ff46eb3d640 1 Processor -- start 2023-11-06T02:09:32.658 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:32.660+0000 7ff46eb3d640 1 -- start start 2023-11-06T02:09:32.659 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:32.660+0000 7ff46eb3d640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff4680cbc10 0x7ff468071120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:32.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:32.660+0000 7ff46eb3d640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff46806b1b0 con 0x7ff4680b5290 2023-11-06T02:09:32.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:32.660+0000 7ff46eb3d640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff46806b490 con 0x7ff4680cf2a0 2023-11-06T02:09:32.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:32.660+0000 7ff46eb3d640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4680cbff0 con 0x7ff4680cbc10 2023-11-06T02:09:32.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:32.660+0000 7ff46c8d9640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff4680cbc10 0x7ff468071120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:32.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:32.660+0000 7ff467fff640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7ff4680b5290 0x7ff4680ceb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:51426/0 (socket says 172.21.15.49:51426) 2023-11-06T02:09:32.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:32.660+0000 7ff467fff640 1 -- 172.21.15.49:0/3404445952 learned_addr learned my addr 172.21.15.49:0/3404445952 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:09:32.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:32.660+0000 7ff46c8d9640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff4680cbc10 0x7ff468071120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40606/0 (socket says 172.21.15.49:40606) 2023-11-06T02:09:32.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:32.660+0000 7ff46c8d9640 1 -- 172.21.15.49:0/3404445952 >> v1:172.21.15.150:6789/0 conn(0x7ff4680cf2a0 legacy=0x7ff46815d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:32.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:32.660+0000 7ff46c8d9640 1 -- 172.21.15.49:0/3404445952 >> v1:172.21.15.49:6789/0 conn(0x7ff4680b5290 legacy=0x7ff4680ceb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:32.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:32.660+0000 7ff46c8d9640 1 -- 172.21.15.49:0/3404445952 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff46806b770 con 0x7ff4680cbc10 2023-11-06T02:09:32.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:32.664+0000 7ff46c8d9640 1 --2- 172.21.15.49:0/3404445952 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4680cbc10 0x7ff468071120 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7ff4580097c0 tx=0x7ff45801c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffa6b00cb2496321 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:32.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:32.664+0000 7ff4677fe640 1 -- 172.21.15.49:0/3404445952 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff458004230 con 0x7ff4680cbc10 2023-11-06T02:09:32.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:32.664+0000 7ff4677fe640 1 -- 172.21.15.49:0/3404445952 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4580a7050 con 0x7ff4680cbc10 2023-11-06T02:09:32.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:32.664+0000 7ff4677fe640 1 -- 172.21.15.49:0/3404445952 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff458003620 con 0x7ff4680cbc10 2023-11-06T02:09:32.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:32.664+0000 7ff46eb3d640 1 -- 172.21.15.49:0/3404445952 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4680cbc10 msgr2=0x7ff468071120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:32.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:32.664+0000 7ff46eb3d640 1 --2- 172.21.15.49:0/3404445952 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4680cbc10 0x7ff468071120 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7ff4580097c0 tx=0x7ff45801c440 comp rx=0 tx=0).stop 2023-11-06T02:09:32.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:32.664+0000 7ff46eb3d640 1 -- 172.21.15.49:0/3404445952 shutdown_connections 2023-11-06T02:09:32.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:32.664+0000 7ff46eb3d640 1 --2- 172.21.15.49:0/3404445952 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4680cbc10 0x7ff468071120 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:32.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:32.664+0000 7ff46eb3d640 1 -- 172.21.15.49:0/3404445952 >> 172.21.15.49:0/3404445952 conn(0x7ff4680cc140 msgr2=0x7ff4680cc550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:32.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:32.664+0000 7ff46eb3d640 1 -- 172.21.15.49:0/3404445952 shutdown_connections 2023-11-06T02:09:32.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:32.664+0000 7ff46eb3d640 1 -- 172.21.15.49:0/3404445952 wait complete. 2023-11-06T02:09:32.667 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:32.664+0000 7ff46eb3d640 1 Processor -- start 2023-11-06T02:09:32.668 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:32.664+0000 7ff46eb3d640 1 -- start start 2023-11-06T02:09:32.668 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:32.664+0000 7ff46eb3d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4680b5290 0x7ff468157950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:32.668 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:32.664+0000 7ff46eb3d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4680cbc10 0x7ff4681459e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:32.668 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:32.664+0000 7ff46eb3d640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4680cf2a0 0x7ff468145f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:32.669 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:32.664+0000 7ff46eb3d640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4680c86e0 con 0x7ff4680b5290 2023-11-06T02:09:32.669 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:32.664+0000 7ff46eb3d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4680c89c0 con 0x7ff4680cbc10 2023-11-06T02:09:32.670 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:32.664+0000 7ff46eb3d640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4680c8ca0 con 0x7ff4680cf2a0 2023-11-06T02:09:32.670 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:32.664+0000 7ff46c8d9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4680b5290 0x7ff468157950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:32.670 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:32.664+0000 7ff46c8d9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4680b5290 0x7ff468157950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40616/0 (socket says 172.21.15.49:40616) 2023-11-06T02:09:32.670 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:32.664+0000 7ff46c8d9640 1 -- 172.21.15.49:0/3079739244 learned_addr learned my addr 172.21.15.49:0/3079739244 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:32.670 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:32.664+0000 7ff467fff640 1 --2- 172.21.15.49:0/3079739244 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4680cbc10 0x7ff4681459e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:32.671 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:32.664+0000 7ff46d0da640 1 --2- 172.21.15.49:0/3079739244 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4680cf2a0 0x7ff468145f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:32.671 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:32.664+0000 7ff46c8d9640 1 -- 172.21.15.49:0/3079739244 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4680cf2a0 msgr2=0x7ff468145f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:32.671 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:32.664+0000 7ff46c8d9640 1 --2- 172.21.15.49:0/3079739244 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4680cf2a0 0x7ff468145f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:32.671 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:32.664+0000 7ff46c8d9640 1 -- 172.21.15.49:0/3079739244 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4680cbc10 msgr2=0x7ff4681459e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:32.671 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:32.664+0000 7ff46c8d9640 1 --2- 172.21.15.49:0/3079739244 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4680cbc10 0x7ff4681459e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:32.672 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:32.664+0000 7ff46c8d9640 1 -- 172.21.15.49:0/3079739244 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff458009420 con 0x7ff4680b5290 2023-11-06T02:09:32.672 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:32.664+0000 7ff46d0da640 1 --2- 172.21.15.49:0/3079739244 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4680cf2a0 0x7ff468145f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:09:32.672 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:32.664+0000 7ff467fff640 1 --2- 172.21.15.49:0/3079739244 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4680cbc10 0x7ff4681459e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:09:32.672 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:32.664+0000 7ff46c8d9640 1 --2- 172.21.15.49:0/3079739244 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4680b5290 0x7ff468157950 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7ff458019050 tx=0x7ff4580191e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:32.672 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:32.664+0000 7ff4657fa640 1 -- 172.21.15.49:0/3079739244 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff458003d10 con 0x7ff4680b5290 2023-11-06T02:09:32.672 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:32.668+0000 7ff4657fa640 1 -- 172.21.15.49:0/3079739244 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff458003e70 con 0x7ff4680b5290 2023-11-06T02:09:32.673 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:32.668+0000 7ff4657fa640 1 -- 172.21.15.49:0/3079739244 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4580ab660 con 0x7ff4680b5290 2023-11-06T02:09:32.673 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:32.668+0000 7ff46eb3d640 1 -- 172.21.15.49:0/3079739244 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4680c8f80 con 0x7ff4680b5290 2023-11-06T02:09:32.673 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:32.668+0000 7ff46eb3d640 1 -- 172.21.15.49:0/3079739244 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff468157e90 con 0x7ff4680b5290 2023-11-06T02:09:32.673 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:32.668+0000 7ff4657fa640 1 -- 172.21.15.49:0/3079739244 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7ff4580b2040 con 0x7ff4680b5290 2023-11-06T02:09:32.673 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:32.668+0000 7ff46eb3d640 1 -- 172.21.15.49:0/3079739244 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff468158080 con 0x7ff4680b5290 2023-11-06T02:09:32.674 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:32.668+0000 7ff4657fa640 1 --2- 172.21.15.49:0/3079739244 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff454065a60 0x7ff454067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:32.674 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:32.668+0000 7ff4657fa640 1 -- 172.21.15.49:0/3079739244 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5296+0+0 (secure 0 0 0) 0x7ff45811cfe0 con 0x7ff4680b5290 2023-11-06T02:09:32.674 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:32.668+0000 7ff467fff640 1 --2- 172.21.15.49:0/3079739244 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff454065a60 0x7ff454067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:32.674 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:32.668+0000 7ff467fff640 1 --2- 172.21.15.49:0/3079739244 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff454065a60 0x7ff454067f20 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff4680cedc0 tx=0x7ff45c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:32.675 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:32.672+0000 7ff4657fa640 1 -- 172.21.15.49:0/3079739244 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7ff468158080 con 0x7ff4680b5290 2023-11-06T02:09:32.839 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:32.840+0000 7ff46eb3d640 1 -- 172.21.15.49:0/3079739244 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7ff468158080 con 0x7ff4680b5290 2023-11-06T02:09:33.635 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:33.636+0000 7ff4657fa640 1 -- 172.21.15.49:0/3079739244 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v33) v1 ==== 97+0+0 (secure 0 0 0) 0x7ff468158080 con 0x7ff4680b5290 2023-11-06T02:09:33.635 INFO:teuthology.orchestra.run.smithi049.stderr: removed all blocklist entries 2023-11-06T02:09:33.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:33.640+0000 7ff46eb3d640 1 -- 172.21.15.49:0/3079739244 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff454065a60 msgr2=0x7ff454067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:33.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:33.640+0000 7ff46eb3d640 1 --2- 172.21.15.49:0/3079739244 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff454065a60 0x7ff454067f20 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff4680cedc0 tx=0x7ff45c009040 comp rx=0 tx=0).stop 2023-11-06T02:09:33.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:33.640+0000 7ff46eb3d640 1 -- 172.21.15.49:0/3079739244 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4680b5290 msgr2=0x7ff468157950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:33.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:33.640+0000 7ff46eb3d640 1 --2- 172.21.15.49:0/3079739244 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4680b5290 0x7ff468157950 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7ff458019050 tx=0x7ff4580191e0 comp rx=0 tx=0).stop 2023-11-06T02:09:33.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:33.640+0000 7ff46eb3d640 1 -- 172.21.15.49:0/3079739244 shutdown_connections 2023-11-06T02:09:33.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:33.640+0000 7ff46eb3d640 1 --2- 172.21.15.49:0/3079739244 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff454065a60 0x7ff454067f20 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:33.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:33.640+0000 7ff46eb3d640 1 --2- 172.21.15.49:0/3079739244 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4680cf2a0 0x7ff468145f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:33.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:33.640+0000 7ff46eb3d640 1 --2- 172.21.15.49:0/3079739244 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4680cbc10 0x7ff4681459e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:33.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:33.640+0000 7ff46eb3d640 1 --2- 172.21.15.49:0/3079739244 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4680b5290 0x7ff468157950 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:33.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:33.640+0000 7ff46eb3d640 1 -- 172.21.15.49:0/3079739244 >> 172.21.15.49:0/3079739244 conn(0x7ff4680cc140 msgr2=0x7ff4680c6700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:33.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:33.640+0000 7ff46eb3d640 1 -- 172.21.15.49:0/3079739244 shutdown_connections 2023-11-06T02:09:33.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:33.640+0000 7ff46eb3d640 1 -- 172.21.15.49:0/3079739244 wait complete. 2023-11-06T02:09:33.664 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-06T02:09:33.798 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:33.800+0000 7f84faba6640 1 Processor -- start 2023-11-06T02:09:33.799 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:33.800+0000 7f84faba6640 1 -- start start 2023-11-06T02:09:33.800 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:33.800+0000 7f84faba6640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f84f415aab0 0x7f84f40cb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:33.800 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:33.800+0000 7f84faba6640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84f40ce3f0 con 0x7f84f40cd990 2023-11-06T02:09:33.800 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:33.800+0000 7f84faba6640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84f40cbdf0 con 0x7f84f40cb970 2023-11-06T02:09:33.801 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:33.800+0000 7f84faba6640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84f415ae90 con 0x7f84f415aab0 2023-11-06T02:09:33.801 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:33.800+0000 7f84f8942640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f84f415aab0 0x7f84f40cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:33.801 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:33.800+0000 7f84f9143640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f84f40cd990 0x7f84f40ce880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:51436/0 (socket says 172.21.15.49:51436) 2023-11-06T02:09:33.801 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:33.800+0000 7f84f9143640 1 -- 172.21.15.49:0/4271603030 learned_addr learned my addr 172.21.15.49:0/4271603030 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:09:33.801 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:33.800+0000 7f84f8942640 1 -- 172.21.15.49:0/4271603030 >> v1:172.21.15.150:6789/0 conn(0x7f84f40cb970 legacy=0x7f84f40cd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:33.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:33.800+0000 7f84f8942640 1 -- 172.21.15.49:0/4271603030 >> v1:172.21.15.49:6789/0 conn(0x7f84f40cd990 legacy=0x7f84f40ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:33.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:33.800+0000 7f84f8942640 1 -- 172.21.15.49:0/4271603030 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84f40cc0d0 con 0x7f84f415aab0 2023-11-06T02:09:33.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:33.800+0000 7f84f37fe640 1 -- 172.21.15.49:0/4271603030 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 100522788 0 0) 0x7f84f40ce3f0 con 0x7f84f40cd990 2023-11-06T02:09:33.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:33.804+0000 7f84f8942640 1 --2- 172.21.15.49:0/4271603030 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84f415aab0 0x7f84f40cb430 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f84e80097a0 tx=0x7f84e801c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38dd5a85354f38b8 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:33.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:33.804+0000 7f84f37fe640 1 -- 172.21.15.49:0/4271603030 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e80ab600 con 0x7f84f415aab0 2023-11-06T02:09:33.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:33.804+0000 7f84f37fe640 1 -- 172.21.15.49:0/4271603030 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84e80a7050 con 0x7f84f415aab0 2023-11-06T02:09:33.804 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:33.804+0000 7f84f37fe640 1 -- 172.21.15.49:0/4271603030 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e80ab980 con 0x7f84f415aab0 2023-11-06T02:09:33.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:33.804+0000 7f84faba6640 1 -- 172.21.15.49:0/4271603030 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84f415aab0 msgr2=0x7f84f40cb430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:33.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:33.804+0000 7f84faba6640 1 --2- 172.21.15.49:0/4271603030 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84f415aab0 0x7f84f40cb430 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f84e80097a0 tx=0x7f84e801c3a0 comp rx=0 tx=0).stop 2023-11-06T02:09:33.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:33.804+0000 7f84faba6640 1 -- 172.21.15.49:0/4271603030 shutdown_connections 2023-11-06T02:09:33.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:33.804+0000 7f84faba6640 1 --2- 172.21.15.49:0/4271603030 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84f415aab0 0x7f84f40cb430 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:33.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:33.804+0000 7f84faba6640 1 -- 172.21.15.49:0/4271603030 >> 172.21.15.49:0/4271603030 conn(0x7f84f4089fd0 msgr2=0x7f84f4099c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:33.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:33.804+0000 7f84faba6640 1 -- 172.21.15.49:0/4271603030 shutdown_connections 2023-11-06T02:09:33.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:33.804+0000 7f84faba6640 1 -- 172.21.15.49:0/4271603030 wait complete. 2023-11-06T02:09:33.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:33.808+0000 7f84faba6640 1 Processor -- start 2023-11-06T02:09:33.807 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:33.808+0000 7f84faba6640 1 -- start start 2023-11-06T02:09:33.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:33.808+0000 7f84faba6640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f84f40cb970 0x7f84f4151e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:33.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:33.808+0000 7f84faba6640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84f40cd990 0x7f84f40c4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:33.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:33.808+0000 7f84faba6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84f415aab0 0x7f84f414fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:33.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:33.808+0000 7f84faba6640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f406cf40 con 0x7f84f415aab0 2023-11-06T02:09:33.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:33.808+0000 7f84faba6640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f4150f70 con 0x7f84f40cd990 2023-11-06T02:09:33.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:33.808+0000 7f84faba6640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84f406cc60 con 0x7f84f40cb970 2023-11-06T02:09:33.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:33.808+0000 7f84f9143640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84f415aab0 0x7f84f414fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:33.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:33.808+0000 7f84f9143640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84f415aab0 0x7f84f414fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40634/0 (socket says 172.21.15.49:40634) 2023-11-06T02:09:33.812 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:33.808+0000 7f84f9143640 1 -- 172.21.15.49:0/3233281921 learned_addr learned my addr 172.21.15.49:0/3233281921 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:33.812 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:33.808+0000 7f84f8942640 1 --2- 172.21.15.49:0/3233281921 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f84f40cb970 0x7f84f4151e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:33.812 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:33.808+0000 7f84f3fff640 1 --2- 172.21.15.49:0/3233281921 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84f40cd990 0x7f84f40c4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:33.812 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:33.808+0000 7f84f9143640 1 -- 172.21.15.49:0/3233281921 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f84f40cb970 msgr2=0x7f84f4151e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:33.812 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:33.808+0000 7f84f9143640 1 --2- 172.21.15.49:0/3233281921 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f84f40cb970 0x7f84f4151e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:33.813 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:33.808+0000 7f84f9143640 1 -- 172.21.15.49:0/3233281921 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84f40cd990 msgr2=0x7f84f40c4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:33.813 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:33.808+0000 7f84f9143640 1 --2- 172.21.15.49:0/3233281921 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84f40cd990 0x7f84f40c4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:33.813 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:33.808+0000 7f84f9143640 1 -- 172.21.15.49:0/3233281921 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84e80093b0 con 0x7f84f415aab0 2023-11-06T02:09:33.813 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:33.808+0000 7f84f9143640 1 --2- 172.21.15.49:0/3233281921 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84f415aab0 0x7f84f414fbf0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f84ec002240 tx=0x7f84ec00cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:33.814 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:33.808+0000 7f84f17fa640 1 -- 172.21.15.49:0/3233281921 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84ec00cea0 con 0x7f84f415aab0 2023-11-06T02:09:33.814 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:33.808+0000 7f84f17fa640 1 -- 172.21.15.49:0/3233281921 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84ec00a640 con 0x7f84f415aab0 2023-11-06T02:09:33.814 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:33.808+0000 7f84f17fa640 1 -- 172.21.15.49:0/3233281921 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84ec00a9f0 con 0x7f84f415aab0 2023-11-06T02:09:33.814 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:33.808+0000 7f84faba6640 1 -- 172.21.15.49:0/3233281921 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84f4150920 con 0x7f84f415aab0 2023-11-06T02:09:33.814 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:33.808+0000 7f84faba6640 1 -- 172.21.15.49:0/3233281921 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84f414ee50 con 0x7f84f415aab0 2023-11-06T02:09:33.815 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:33.808+0000 7f84faba6640 1 -- 172.21.15.49:0/3233281921 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84f4150780 con 0x7f84f415aab0 2023-11-06T02:09:33.815 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:33.808+0000 7f84f17fa640 1 -- 172.21.15.49:0/3233281921 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f84ec00ab50 con 0x7f84f415aab0 2023-11-06T02:09:33.815 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:33.808+0000 7f84f17fa640 1 --2- 172.21.15.49:0/3233281921 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f84e0065a60 0x7f84e0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:33.815 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:33.808+0000 7f84f17fa640 1 -- 172.21.15.49:0/3233281921 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f84ec082490 con 0x7f84f415aab0 2023-11-06T02:09:33.816 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:33.812+0000 7f84f8942640 1 --2- 172.21.15.49:0/3233281921 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f84e0065a60 0x7f84e0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:33.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:33.812+0000 7f84f8942640 1 --2- 172.21.15.49:0/3233281921 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f84e0065a60 0x7f84e0067f20 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f84e80ad000 tx=0x7f84e80a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:33.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:33.812+0000 7f84f17fa640 1 -- 172.21.15.49:0/3233281921 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f84f4150780 con 0x7f84f415aab0 2023-11-06T02:09:34.031 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.032+0000 7f84faba6640 1 -- 172.21.15.49:0/3233281921 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f84f4150780 con 0x7f84f415aab0 2023-11-06T02:09:34.032 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.032+0000 7f84f17fa640 1 -- 172.21.15.49:0/3233281921 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f84f4150780 con 0x7f84f415aab0 2023-11-06T02:09:34.032 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:09:34.033 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-11-06T02:09:34.033 INFO:teuthology.orchestra.run.smithi049.stdout: "auth_dump": [ 2023-11-06T02:09:34.033 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:09:34.033 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "mds.a", 2023-11-06T02:09:34.033 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQBbSkhlbRFwIhAApA6xaqdIIeKDsOsNhyWCaA==", 2023-11-06T02:09:34.034 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-11-06T02:09:34.034 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": "allow", 2023-11-06T02:09:34.034 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow *", 2023-11-06T02:09:34.034 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow *", 2023-11-06T02:09:34.035 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-11-06T02:09:34.035 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:09:34.035 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:09:34.035 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:09:34.035 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "mds.b", 2023-11-06T02:09:34.036 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQBbSkhl6QhcMRAAgmOxZWghmuu11F0Gfut3sQ==", 2023-11-06T02:09:34.036 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-11-06T02:09:34.036 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": "allow", 2023-11-06T02:09:34.037 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow *", 2023-11-06T02:09:34.037 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow *", 2023-11-06T02:09:34.038 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-11-06T02:09:34.038 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:09:34.038 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:09:34.038 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:09:34.038 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "mds.c", 2023-11-06T02:09:34.039 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQBbSkhlMoZHKxAATV62bAItdXt3ShpsT4oVbA==", 2023-11-06T02:09:34.039 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-11-06T02:09:34.039 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": "allow", 2023-11-06T02:09:34.039 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow *", 2023-11-06T02:09:34.039 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow *", 2023-11-06T02:09:34.040 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-11-06T02:09:34.040 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:09:34.040 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:09:34.040 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:09:34.040 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "mds.d", 2023-11-06T02:09:34.040 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQBbSkhl376GORAAS2NIQJMzg6Es0JCYPak6fw==", 2023-11-06T02:09:34.041 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-11-06T02:09:34.041 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": "allow", 2023-11-06T02:09:34.041 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow *", 2023-11-06T02:09:34.041 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow *", 2023-11-06T02:09:34.041 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-11-06T02:09:34.042 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:09:34.042 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:09:34.042 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:09:34.042 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "osd.0", 2023-11-06T02:09:34.042 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQBdSkhlxuQtFRAAPttjVsYpkwexC5x2Pu2b+Q==", 2023-11-06T02:09:34.043 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-11-06T02:09:34.043 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow profile osd", 2023-11-06T02:09:34.043 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow profile osd", 2023-11-06T02:09:34.043 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-11-06T02:09:34.043 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:09:34.043 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:09:34.044 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:09:34.044 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "osd.1", 2023-11-06T02:09:34.044 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQBgSkhlCGeiHBAA8SofNV6C1+RQNajUsewVpw==", 2023-11-06T02:09:34.044 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-11-06T02:09:34.044 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow profile osd", 2023-11-06T02:09:34.044 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow profile osd", 2023-11-06T02:09:34.045 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-11-06T02:09:34.045 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:09:34.045 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:09:34.045 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:09:34.045 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "osd.2", 2023-11-06T02:09:34.045 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQBjSkhlBju+JBAAPJeIFngKDoamUvEjv1h8tw==", 2023-11-06T02:09:34.046 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-11-06T02:09:34.046 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow profile osd", 2023-11-06T02:09:34.046 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow profile osd", 2023-11-06T02:09:34.046 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-11-06T02:09:34.046 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:09:34.047 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:09:34.047 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:09:34.047 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "osd.3", 2023-11-06T02:09:34.047 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQBmSkhlMclTLRAABnbmQA9j3TLn40H1wYFx5g==", 2023-11-06T02:09:34.047 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-11-06T02:09:34.047 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow profile osd", 2023-11-06T02:09:34.048 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow profile osd", 2023-11-06T02:09:34.048 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-11-06T02:09:34.048 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:09:34.048 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:09:34.048 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:09:34.048 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "osd.4", 2023-11-06T02:09:34.049 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQBsSkhl7fmmLhAArv62qbJ+StgYMKhCftM5HA==", 2023-11-06T02:09:34.049 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-11-06T02:09:34.049 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow profile osd", 2023-11-06T02:09:34.049 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow profile osd", 2023-11-06T02:09:34.049 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-11-06T02:09:34.049 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:09:34.050 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:09:34.050 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:09:34.050 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "osd.5", 2023-11-06T02:09:34.050 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQBvSkhl5pAANhAArNFy9/YuEcXTdzM5XL46Xw==", 2023-11-06T02:09:34.050 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-11-06T02:09:34.051 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow profile osd", 2023-11-06T02:09:34.051 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow profile osd", 2023-11-06T02:09:34.051 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-11-06T02:09:34.051 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:09:34.051 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:09:34.052 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:09:34.052 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "osd.6", 2023-11-06T02:09:34.052 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQBzSkhlgfs2BBAAeHXVJ/tFiFagugkynGakFw==", 2023-11-06T02:09:34.052 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-11-06T02:09:34.052 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow profile osd", 2023-11-06T02:09:34.052 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow profile osd", 2023-11-06T02:09:34.053 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-11-06T02:09:34.053 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:09:34.053 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:09:34.053 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:09:34.053 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "osd.7", 2023-11-06T02:09:34.053 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQB2SkhlQbCYDRAAhE7vwbjJjqZDHfgDfdy1cA==", 2023-11-06T02:09:34.054 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-11-06T02:09:34.054 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow profile osd", 2023-11-06T02:09:34.054 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow profile osd", 2023-11-06T02:09:34.054 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-11-06T02:09:34.054 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:09:34.055 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:09:34.055 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:09:34.055 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "client.0", 2023-11-06T02:09:34.055 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQBcSkhlY46mAxAALClTH91L6USom3nVss6iiQ==", 2023-11-06T02:09:34.055 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-11-06T02:09:34.056 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": "allow", 2023-11-06T02:09:34.056 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow r", 2023-11-06T02:09:34.056 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow rw", 2023-11-06T02:09:34.056 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow rwx" 2023-11-06T02:09:34.056 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:09:34.056 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:09:34.057 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:09:34.057 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "client.1", 2023-11-06T02:09:34.057 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQBcSkhllvSyCxAAfcSiBhHHNgaIr9mhpY11fg==", 2023-11-06T02:09:34.057 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-11-06T02:09:34.057 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": "allow", 2023-11-06T02:09:34.057 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow r", 2023-11-06T02:09:34.058 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow rw", 2023-11-06T02:09:34.058 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow rwx" 2023-11-06T02:09:34.058 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:09:34.058 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:09:34.058 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:09:34.059 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "client.2", 2023-11-06T02:09:34.059 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQBcSkhlURf6CBAA9g2y/UMD4x42IT0fgSgIcg==", 2023-11-06T02:09:34.059 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-11-06T02:09:34.059 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": "allow", 2023-11-06T02:09:34.059 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow r", 2023-11-06T02:09:34.060 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow rw", 2023-11-06T02:09:34.060 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow rwx" 2023-11-06T02:09:34.060 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:09:34.060 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:09:34.060 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:09:34.060 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "client.3", 2023-11-06T02:09:34.061 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQBcSkhlbjXXEBAA0xQGCY/avAWL18GLQtIjgg==", 2023-11-06T02:09:34.061 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-11-06T02:09:34.061 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": "allow", 2023-11-06T02:09:34.061 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow r", 2023-11-06T02:09:34.061 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow rw", 2023-11-06T02:09:34.062 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow rwx" 2023-11-06T02:09:34.062 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:09:34.062 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:09:34.062 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:09:34.063 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "client.admin", 2023-11-06T02:09:34.063 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQBbSkhloSJXCxAA++ZHscsqHENk8soyqq/GPQ==", 2023-11-06T02:09:34.063 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-11-06T02:09:34.063 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": "allow *", 2023-11-06T02:09:34.063 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow *", 2023-11-06T02:09:34.063 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow *", 2023-11-06T02:09:34.064 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-11-06T02:09:34.064 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:09:34.064 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:09:34.064 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:09:34.064 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "client.bootstrap-mds", 2023-11-06T02:09:34.065 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQCGSkhlHRxiGhAA0KieHLUMUcQPhR34336/Bw==", 2023-11-06T02:09:34.065 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-11-06T02:09:34.065 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow profile bootstrap-mds" 2023-11-06T02:09:34.065 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:09:34.065 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:09:34.066 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:09:34.066 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "client.bootstrap-mgr", 2023-11-06T02:09:34.066 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQCGSkhlg05iGhAAqjGOwmkomnzW6A7upDJm6A==", 2023-11-06T02:09:34.066 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-11-06T02:09:34.066 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow profile bootstrap-mgr" 2023-11-06T02:09:34.067 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:09:34.067 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:09:34.067 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:09:34.067 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "client.bootstrap-osd", 2023-11-06T02:09:34.067 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQCGSkhlDGpiGhAAiiAjiVmUHL+rupYgax5LiQ==", 2023-11-06T02:09:34.067 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-11-06T02:09:34.068 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow profile bootstrap-osd" 2023-11-06T02:09:34.068 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:09:34.068 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:09:34.068 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:09:34.068 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "client.bootstrap-rbd", 2023-11-06T02:09:34.069 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQCGSkhlQYBiGhAA2GTiwQT2kILXDCC4CNpt/A==", 2023-11-06T02:09:34.069 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-11-06T02:09:34.069 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow profile bootstrap-rbd" 2023-11-06T02:09:34.069 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:09:34.069 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:09:34.070 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:09:34.070 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-11-06T02:09:34.070 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQCGSkhlXZZiGhAAShJDmZmMbVf+uJLbAdg5aQ==", 2023-11-06T02:09:34.070 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-11-06T02:09:34.070 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-11-06T02:09:34.071 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:09:34.071 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:09:34.071 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:09:34.071 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "client.bootstrap-rgw", 2023-11-06T02:09:34.071 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQCGSkhlW61iGhAAyhozZ8io+2m9GC+hc88wrQ==", 2023-11-06T02:09:34.071 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-11-06T02:09:34.072 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow profile bootstrap-rgw" 2023-11-06T02:09:34.072 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:09:34.072 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:09:34.072 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:09:34.072 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "mgr.x", 2023-11-06T02:09:34.073 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQBbSkhlpJ4iHxAAGEz4XS5fILy7/NZbAenPFw==", 2023-11-06T02:09:34.073 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-11-06T02:09:34.073 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": "allow *", 2023-11-06T02:09:34.073 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow profile mgr", 2023-11-06T02:09:34.073 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-11-06T02:09:34.073 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:09:34.074 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:09:34.074 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:09:34.074 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "mgr.y", 2023-11-06T02:09:34.074 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQBbSkhln1JuHBAAymHRkniPpsI6pf9L2C2y+g==", 2023-11-06T02:09:34.074 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-11-06T02:09:34.075 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": "allow *", 2023-11-06T02:09:34.075 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow profile mgr", 2023-11-06T02:09:34.075 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-11-06T02:09:34.075 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:09:34.075 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:09:34.075 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2023-11-06T02:09:34.076 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-11-06T02:09:34.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.036+0000 7f84faba6640 1 -- 172.21.15.49:0/3233281921 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f84e0065a60 msgr2=0x7f84e0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:34.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.036+0000 7f84faba6640 1 --2- 172.21.15.49:0/3233281921 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f84e0065a60 0x7f84e0067f20 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f84e80ad000 tx=0x7f84e80a6000 comp rx=0 tx=0).stop 2023-11-06T02:09:34.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.036+0000 7f84faba6640 1 -- 172.21.15.49:0/3233281921 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84f415aab0 msgr2=0x7f84f414fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:34.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.036+0000 7f84faba6640 1 --2- 172.21.15.49:0/3233281921 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84f415aab0 0x7f84f414fbf0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f84ec002240 tx=0x7f84ec00cc50 comp rx=0 tx=0).stop 2023-11-06T02:09:34.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.036+0000 7f84faba6640 1 -- 172.21.15.49:0/3233281921 shutdown_connections 2023-11-06T02:09:34.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.036+0000 7f84faba6640 1 --2- 172.21.15.49:0/3233281921 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f84e0065a60 0x7f84e0067f20 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:34.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.036+0000 7f84faba6640 1 --2- 172.21.15.49:0/3233281921 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84f415aab0 0x7f84f414fbf0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:34.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.036+0000 7f84faba6640 1 --2- 172.21.15.49:0/3233281921 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84f40cd990 0x7f84f40c4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:34.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.036+0000 7f84faba6640 1 --2- 172.21.15.49:0/3233281921 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f84f40cb970 0x7f84f4151e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:34.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.036+0000 7f84faba6640 1 -- 172.21.15.49:0/3233281921 >> 172.21.15.49:0/3233281921 conn(0x7f84f4089fd0 msgr2=0x7f84f40b09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:34.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.036+0000 7f84faba6640 1 -- 172.21.15.49:0/3233281921 shutdown_connections 2023-11-06T02:09:34.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.036+0000 7f84faba6640 1 -- 172.21.15.49:0/3233281921 wait complete. 2023-11-06T02:09:34.080 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-11-06T02:09:34.207 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.208+0000 7f5a4e947640 1 Processor -- start 2023-11-06T02:09:34.208 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.208+0000 7f5a4e947640 1 -- start start 2023-11-06T02:09:34.208 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.208+0000 7f5a4e947640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f5a480cda70 0x7f5a48161650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:34.209 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.208+0000 7f5a4e947640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5a480714a0 0x7f5a480718d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:34.209 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.208+0000 7f5a4e947640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a480d17c0 con 0x7f5a480b1f90 2023-11-06T02:09:34.209 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.208+0000 7f5a4e947640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a480cdf20 con 0x7f5a480cda70 2023-11-06T02:09:34.209 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.208+0000 7f5a4e947640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a48161b90 con 0x7f5a480714a0 2023-11-06T02:09:34.209 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.208+0000 7f5a47fff640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f5a480b1f90 0x7f5a480d1c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:51442/0 (socket says 172.21.15.49:51442) 2023-11-06T02:09:34.210 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.208+0000 7f5a47fff640 1 -- 172.21.15.49:0/296653707 learned_addr learned my addr 172.21.15.49:0/296653707 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:09:34.211 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.208+0000 7f5a477fe640 1 --2- 172.21.15.49:0/296653707 >> v2:172.21.15.150:3300/0 conn(0x7f5a480cda70 0x7f5a48161650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:34.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.208+0000 7f5a4cee4640 1 --2- 172.21.15.49:0/296653707 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5a480714a0 0x7f5a480718d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:34.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.208+0000 7f5a46ffd640 1 -- 172.21.15.49:0/296653707 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 829078661 0 0) 0x7f5a480d17c0 con 0x7f5a480b1f90 2023-11-06T02:09:34.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.208+0000 7f5a46ffd640 1 -- 172.21.15.49:0/296653707 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a34003530 con 0x7f5a480b1f90 2023-11-06T02:09:34.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.212+0000 7f5a46ffd640 1 -- 172.21.15.49:0/296653707 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3614295749 0 0) 0x7f5a34003530 con 0x7f5a480b1f90 2023-11-06T02:09:34.213 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.212+0000 7f5a46ffd640 1 -- 172.21.15.49:0/296653707 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5a480714a0 msgr2=0x7f5a480718d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:34.213 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.212+0000 7f5a46ffd640 1 --2- 172.21.15.49:0/296653707 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5a480714a0 0x7f5a480718d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:34.213 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.212+0000 7f5a46ffd640 1 -- 172.21.15.49:0/296653707 >> v2:172.21.15.150:3300/0 conn(0x7f5a480cda70 msgr2=0x7f5a48161650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:34.213 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.212+0000 7f5a46ffd640 1 --2- 172.21.15.49:0/296653707 >> v2:172.21.15.150:3300/0 conn(0x7f5a480cda70 0x7f5a48161650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:34.213 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.212+0000 7f5a46ffd640 1 -- 172.21.15.49:0/296653707 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a48071e10 con 0x7f5a480b1f90 2023-11-06T02:09:34.213 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.212+0000 7f5a46ffd640 1 -- 172.21.15.49:0/296653707 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7f5a40001a00 con 0x7f5a480b1f90 2023-11-06T02:09:34.214 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.212+0000 7f5a46ffd640 1 -- 172.21.15.49:0/296653707 >> v1:172.21.15.49:6789/0 conn(0x7f5a480b1f90 legacy=0x7f5a480d1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:34.214 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.212+0000 7f5a46ffd640 1 --2- 172.21.15.49:0/296653707 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5a480714a0 0x7f5a340039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:34.214 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.212+0000 7f5a46ffd640 1 --2- 172.21.15.49:0/296653707 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5a34003f30 0x7f5a340047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:34.214 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.212+0000 7f5a46ffd640 1 --2- 172.21.15.49:0/296653707 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a34004d10 0x7f5a34005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:34.215 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.212+0000 7f5a46ffd640 1 -- 172.21.15.49:0/296653707 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a480cdf20 con 0x7f5a34004d10 2023-11-06T02:09:34.215 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.212+0000 7f5a46ffd640 1 -- 172.21.15.49:0/296653707 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3400b990 con 0x7f5a34003f30 2023-11-06T02:09:34.215 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.212+0000 7f5a46ffd640 1 -- 172.21.15.49:0/296653707 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a340142e0 con 0x7f5a480714a0 2023-11-06T02:09:34.215 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.212+0000 7f5a4cee4640 1 --2- 172.21.15.49:0/296653707 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a34004d10 0x7f5a34005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:34.215 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.212+0000 7f5a477fe640 1 --2- 172.21.15.49:0/296653707 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5a34003f30 0x7f5a340047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:34.216 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.212+0000 7f5a47fff640 1 --2- 172.21.15.49:0/296653707 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5a480714a0 0x7f5a340039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:34.216 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.212+0000 7f5a4cee4640 1 -- 172.21.15.49:0/296653707 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5a480714a0 msgr2=0x7f5a340039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:34.216 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.212+0000 7f5a4cee4640 1 --2- 172.21.15.49:0/296653707 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5a480714a0 0x7f5a340039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:34.216 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.212+0000 7f5a4cee4640 1 -- 172.21.15.49:0/296653707 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5a34003f30 msgr2=0x7f5a340047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:34.216 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.212+0000 7f5a4cee4640 1 --2- 172.21.15.49:0/296653707 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5a34003f30 0x7f5a340047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:34.216 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.212+0000 7f5a4cee4640 1 -- 172.21.15.49:0/296653707 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a34003530 con 0x7f5a34004d10 2023-11-06T02:09:34.220 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.212+0000 7f5a4cee4640 1 --2- 172.21.15.49:0/296653707 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a34004d10 0x7f5a34005190 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f5a38009ae0 tx=0x7f5a3801ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3679a693ff689df5 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:34.220 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.212+0000 7f5a46ffd640 1 -- 172.21.15.49:0/296653707 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a380a5640 con 0x7f5a34004d10 2023-11-06T02:09:34.220 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.212+0000 7f5a46ffd640 1 -- 172.21.15.49:0/296653707 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a380a57a0 con 0x7f5a34004d10 2023-11-06T02:09:34.220 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.212+0000 7f5a46ffd640 1 -- 172.21.15.49:0/296653707 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a380a5b50 con 0x7f5a34004d10 2023-11-06T02:09:34.221 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.212+0000 7f5a4e947640 1 -- 172.21.15.49:0/296653707 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a34004d10 msgr2=0x7f5a34005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:34.221 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.212+0000 7f5a4e947640 1 --2- 172.21.15.49:0/296653707 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a34004d10 0x7f5a34005190 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f5a38009ae0 tx=0x7f5a3801ca50 comp rx=0 tx=0).stop 2023-11-06T02:09:34.221 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.212+0000 7f5a4cee4640 1 -- 172.21.15.49:0/296653707 reap_dead start 2023-11-06T02:09:34.221 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.212+0000 7f5a4e947640 1 -- 172.21.15.49:0/296653707 shutdown_connections 2023-11-06T02:09:34.221 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.212+0000 7f5a4e947640 1 -- 172.21.15.49:0/296653707 >> 172.21.15.49:0/296653707 conn(0x7f5a480d2aa0 msgr2=0x7f5a480d2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:34.222 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.212+0000 7f5a4e947640 1 -- 172.21.15.49:0/296653707 shutdown_connections 2023-11-06T02:09:34.222 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.212+0000 7f5a4e947640 1 -- 172.21.15.49:0/296653707 wait complete. 2023-11-06T02:09:34.222 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.216+0000 7f5a4e947640 1 Processor -- start 2023-11-06T02:09:34.222 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.216+0000 7f5a4e947640 1 -- start start 2023-11-06T02:09:34.222 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.216+0000 7f5a4e947640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5a480b4930 0x7f5a480cab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:34.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.216+0000 7f5a4e947640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5a480b4d10 0x7f5a4815bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:34.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.216+0000 7f5a4e947640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a48149a90 0x7f5a4815c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:34.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.216+0000 7f5a4e947640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a48146dc0 con 0x7f5a48149a90 2023-11-06T02:09:34.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.216+0000 7f5a4e947640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a481470a0 con 0x7f5a480b4d10 2023-11-06T02:09:34.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.216+0000 7f5a4e947640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a480c6330 con 0x7f5a480b4930 2023-11-06T02:09:34.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.216+0000 7f5a4cee4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a48149a90 0x7f5a4815c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:34.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.216+0000 7f5a4cee4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a48149a90 0x7f5a4815c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40656/0 (socket says 172.21.15.49:40656) 2023-11-06T02:09:34.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.216+0000 7f5a4cee4640 1 -- 172.21.15.49:0/2892392776 learned_addr learned my addr 172.21.15.49:0/2892392776 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:34.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.216+0000 7f5a47fff640 1 --2- 172.21.15.49:0/2892392776 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5a480b4930 0x7f5a480cab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:34.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.216+0000 7f5a477fe640 1 --2- 172.21.15.49:0/2892392776 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5a480b4d10 0x7f5a4815bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:34.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.216+0000 7f5a4cee4640 1 -- 172.21.15.49:0/2892392776 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5a480b4930 msgr2=0x7f5a480cab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:34.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.216+0000 7f5a4cee4640 1 --2- 172.21.15.49:0/2892392776 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5a480b4930 0x7f5a480cab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:34.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.216+0000 7f5a4cee4640 1 -- 172.21.15.49:0/2892392776 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5a480b4d10 msgr2=0x7f5a4815bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:34.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.216+0000 7f5a4cee4640 1 --2- 172.21.15.49:0/2892392776 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5a480b4d10 0x7f5a4815bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:34.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.216+0000 7f5a4cee4640 1 -- 172.21.15.49:0/2892392776 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a480cb3a0 con 0x7f5a48149a90 2023-11-06T02:09:34.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.216+0000 7f5a47fff640 1 --2- 172.21.15.49:0/2892392776 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5a480b4930 0x7f5a480cab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:09:34.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.216+0000 7f5a477fe640 1 --2- 172.21.15.49:0/2892392776 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5a480b4d10 0x7f5a4815bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:09:34.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.216+0000 7f5a4cee4640 1 --2- 172.21.15.49:0/2892392776 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a48149a90 0x7f5a4815c050 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f5a38009ae0 tx=0x7f5a34005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:34.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.216+0000 7f5a44ff9640 1 -- 172.21.15.49:0/2892392776 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a380bb070 con 0x7f5a48149a90 2023-11-06T02:09:34.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.216+0000 7f5a44ff9640 1 -- 172.21.15.49:0/2892392776 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a38004020 con 0x7f5a48149a90 2023-11-06T02:09:34.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.216+0000 7f5a44ff9640 1 -- 172.21.15.49:0/2892392776 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a380a5640 con 0x7f5a48149a90 2023-11-06T02:09:34.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.216+0000 7f5a4e947640 1 -- 172.21.15.49:0/2892392776 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a480cb680 con 0x7f5a48149a90 2023-11-06T02:09:34.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.216+0000 7f5a4e947640 1 -- 172.21.15.49:0/2892392776 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a480cb070 con 0x7f5a48149a90 2023-11-06T02:09:34.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.216+0000 7f5a4e947640 1 -- 172.21.15.49:0/2892392776 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a48147230 con 0x7f5a48149a90 2023-11-06T02:09:34.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.220+0000 7f5a44ff9640 1 -- 172.21.15.49:0/2892392776 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f5a380b4040 con 0x7f5a48149a90 2023-11-06T02:09:34.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.220+0000 7f5a44ff9640 1 --2- 172.21.15.49:0/2892392776 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5a28065a90 0x7f5a28067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:34.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.220+0000 7f5a44ff9640 1 -- 172.21.15.49:0/2892392776 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f5a3811c670 con 0x7f5a48149a90 2023-11-06T02:09:34.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.220+0000 7f5a47fff640 1 --2- 172.21.15.49:0/2892392776 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5a28065a90 0x7f5a28067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:34.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.220+0000 7f5a47fff640 1 --2- 172.21.15.49:0/2892392776 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5a28065a90 0x7f5a28067f50 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5a400013e0 tx=0x7f5a40001370 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:34.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.224+0000 7f5a44ff9640 1 -- 172.21.15.49:0/2892392776 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f5a380ea2b0 con 0x7f5a48149a90 2023-11-06T02:09:34.433 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.432+0000 7f5a4e947640 1 -- 172.21.15.49:0/2892392776 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f5a48074040 con 0x7f5a48149a90 2023-11-06T02:09:34.451 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.452+0000 7f5a44ff9640 1 -- 172.21.15.49:0/2892392776 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) v1 ==== 90+0+0 (secure 0 0 0) 0x7f5a380ed8d0 con 0x7f5a48149a90 2023-11-06T02:09:34.454 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.456+0000 7f5a4e947640 1 -- 172.21.15.49:0/2892392776 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5a28065a90 msgr2=0x7f5a28067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:34.454 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.456+0000 7f5a4e947640 1 --2- 172.21.15.49:0/2892392776 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5a28065a90 0x7f5a28067f50 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5a400013e0 tx=0x7f5a40001370 comp rx=0 tx=0).stop 2023-11-06T02:09:34.454 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.456+0000 7f5a4e947640 1 -- 172.21.15.49:0/2892392776 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a48149a90 msgr2=0x7f5a4815c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:34.455 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.456+0000 7f5a4e947640 1 --2- 172.21.15.49:0/2892392776 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a48149a90 0x7f5a4815c050 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f5a38009ae0 tx=0x7f5a34005800 comp rx=0 tx=0).stop 2023-11-06T02:09:34.455 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.456+0000 7f5a4e947640 1 -- 172.21.15.49:0/2892392776 shutdown_connections 2023-11-06T02:09:34.455 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.456+0000 7f5a4e947640 1 --2- 172.21.15.49:0/2892392776 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5a28065a90 0x7f5a28067f50 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:34.455 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.456+0000 7f5a4e947640 1 --2- 172.21.15.49:0/2892392776 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a48149a90 0x7f5a4815c050 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:34.455 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.456+0000 7f5a4e947640 1 --2- 172.21.15.49:0/2892392776 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5a480b4d10 0x7f5a4815bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:34.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.456+0000 7f5a4e947640 1 --2- 172.21.15.49:0/2892392776 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5a480b4930 0x7f5a480cab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:34.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.456+0000 7f5a4e947640 1 -- 172.21.15.49:0/2892392776 >> 172.21.15.49:0/2892392776 conn(0x7f5a480d2aa0 msgr2=0x7f5a480c8030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:34.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.456+0000 7f5a4e947640 1 -- 172.21.15.49:0/2892392776 shutdown_connections 2023-11-06T02:09:34.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.456+0000 7f5a4e947640 1 -- 172.21.15.49:0/2892392776 wait complete. 2023-11-06T02:09:34.471 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-11-06T02:09:34.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.600+0000 7f3241789640 1 Processor -- start 2023-11-06T02:09:34.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.600+0000 7f3241789640 1 -- start start 2023-11-06T02:09:34.600 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.600+0000 7f3241789640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f323c0cb010 0x7f323c0ca980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:34.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.600+0000 7f3241789640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f323c159b40 0x7f323c0cb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:34.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.600+0000 7f3241789640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f323c0cb930 con 0x7f323c0cdd50 2023-11-06T02:09:34.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.600+0000 7f3241789640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f323c0723a0 con 0x7f323c159b40 2023-11-06T02:09:34.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.600+0000 7f3241789640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f323c0715e0 con 0x7f323c0cb010 2023-11-06T02:09:34.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.600+0000 7f323a7fc640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f323c159b40 0x7f323c0cb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:34.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.600+0000 7f323a7fc640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f323c159b40 0x7f323c0cb3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40670/0 (socket says 172.21.15.49:40670) 2023-11-06T02:09:34.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.600+0000 7f323a7fc640 1 -- 172.21.15.49:0/101927012 learned_addr learned my addr 172.21.15.49:0/101927012 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:34.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.600+0000 7f323b7fe640 1 --1- 172.21.15.49:0/101927012 >> v1:172.21.15.49:6789/0 conn(0x7f323c0cdd50 0x7f323c0c8c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:51450/0 (socket says 172.21.15.49:51450) 2023-11-06T02:09:34.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.600+0000 7f323affd640 1 --2- 172.21.15.49:0/101927012 >> v2:172.21.15.150:3300/0 conn(0x7f323c0cb010 0x7f323c0ca980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:34.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.600+0000 7f323a7fc640 1 -- 172.21.15.49:0/101927012 >> v2:172.21.15.150:3300/0 conn(0x7f323c0cb010 msgr2=0x7f323c0ca980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:34.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.600+0000 7f323a7fc640 1 --2- 172.21.15.49:0/101927012 >> v2:172.21.15.150:3300/0 conn(0x7f323c0cb010 0x7f323c0ca980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:34.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.600+0000 7f323a7fc640 1 -- 172.21.15.49:0/101927012 >> v1:172.21.15.49:6789/0 conn(0x7f323c0cdd50 legacy=0x7f323c0c8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:34.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.600+0000 7f323a7fc640 1 -- 172.21.15.49:0/101927012 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f323c0718c0 con 0x7f323c159b40 2023-11-06T02:09:34.604 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.604+0000 7f323a7fc640 1 --2- 172.21.15.49:0/101927012 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f323c159b40 0x7f323c0cb3f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f323c06e5b0 tx=0x7f322401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a36eb596e4ef35d5 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:34.604 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.604+0000 7f3239ffb640 1 -- 172.21.15.49:0/101927012 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32240ab5f0 con 0x7f323c159b40 2023-11-06T02:09:34.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.604+0000 7f3239ffb640 1 -- 172.21.15.49:0/101927012 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32240ab750 con 0x7f323c159b40 2023-11-06T02:09:34.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.604+0000 7f3239ffb640 1 -- 172.21.15.49:0/101927012 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32240abae0 con 0x7f323c159b40 2023-11-06T02:09:34.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.604+0000 7f3241789640 1 -- 172.21.15.49:0/101927012 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f323c159b40 msgr2=0x7f323c0cb3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:34.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.604+0000 7f3241789640 1 --2- 172.21.15.49:0/101927012 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f323c159b40 0x7f323c0cb3f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f323c06e5b0 tx=0x7f322401c4d0 comp rx=0 tx=0).stop 2023-11-06T02:09:34.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.604+0000 7f3241789640 1 -- 172.21.15.49:0/101927012 shutdown_connections 2023-11-06T02:09:34.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.604+0000 7f3241789640 1 --2- 172.21.15.49:0/101927012 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f323c159b40 0x7f323c0cb3f0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:34.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.604+0000 7f3241789640 1 --2- 172.21.15.49:0/101927012 >> v2:172.21.15.150:3300/0 conn(0x7f323c0cb010 0x7f323c0ca980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:34.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.604+0000 7f3241789640 1 -- 172.21.15.49:0/101927012 >> 172.21.15.49:0/101927012 conn(0x7f323c06c0a0 msgr2=0x7f323c06c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:34.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.604+0000 7f3241789640 1 -- 172.21.15.49:0/101927012 shutdown_connections 2023-11-06T02:09:34.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.604+0000 7f3241789640 1 -- 172.21.15.49:0/101927012 wait complete. 2023-11-06T02:09:34.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.604+0000 7f3241789640 1 Processor -- start 2023-11-06T02:09:34.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.604+0000 7f3241789640 1 -- start start 2023-11-06T02:09:34.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.608+0000 7f3241789640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f323c0cb010 0x7f323c158880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:34.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.608+0000 7f3241789640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f323c0cdd50 0x7f323c1519d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:34.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.608+0000 7f3241789640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f323c159b40 0x7f323c151f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:34.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.608+0000 7f3241789640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f323c152450 con 0x7f323c159b40 2023-11-06T02:09:34.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.608+0000 7f3241789640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f323c1537a0 con 0x7f323c0cdd50 2023-11-06T02:09:34.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.608+0000 7f3241789640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f323c153a80 con 0x7f323c0cb010 2023-11-06T02:09:34.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.608+0000 7f323b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f323c159b40 0x7f323c151f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:34.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.608+0000 7f323b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f323c159b40 0x7f323c151f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40672/0 (socket says 172.21.15.49:40672) 2023-11-06T02:09:34.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.608+0000 7f323b7fe640 1 -- 172.21.15.49:0/4170819383 learned_addr learned my addr 172.21.15.49:0/4170819383 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:34.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.608+0000 7f323a7fc640 1 --2- 172.21.15.49:0/4170819383 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f323c0cdd50 0x7f323c1519d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:34.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.608+0000 7f323affd640 1 --2- 172.21.15.49:0/4170819383 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f323c0cb010 0x7f323c158880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:34.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.608+0000 7f323b7fe640 1 -- 172.21.15.49:0/4170819383 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f323c0cb010 msgr2=0x7f323c158880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:34.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.608+0000 7f323b7fe640 1 --2- 172.21.15.49:0/4170819383 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f323c0cb010 0x7f323c158880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:34.612 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.608+0000 7f323b7fe640 1 -- 172.21.15.49:0/4170819383 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f323c0cdd50 msgr2=0x7f323c1519d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:34.612 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.608+0000 7f323b7fe640 1 --2- 172.21.15.49:0/4170819383 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f323c0cdd50 0x7f323c1519d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:34.612 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.608+0000 7f323b7fe640 1 -- 172.21.15.49:0/4170819383 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32240094d0 con 0x7f323c159b40 2023-11-06T02:09:34.612 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.608+0000 7f323b7fe640 1 --2- 172.21.15.49:0/4170819383 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f323c159b40 0x7f323c151f10 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f322c00c3c0 tx=0x7f322c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:34.612 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.608+0000 7f32337fe640 1 -- 172.21.15.49:0/4170819383 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f322c0074e0 con 0x7f323c159b40 2023-11-06T02:09:34.613 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.608+0000 7f32337fe640 1 -- 172.21.15.49:0/4170819383 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f322c007640 con 0x7f323c159b40 2023-11-06T02:09:34.613 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.608+0000 7f3241789640 1 -- 172.21.15.49:0/4170819383 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f323c153d60 con 0x7f323c159b40 2023-11-06T02:09:34.613 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.608+0000 7f32337fe640 1 -- 172.21.15.49:0/4170819383 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f322c004a10 con 0x7f323c159b40 2023-11-06T02:09:34.613 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.608+0000 7f3241789640 1 -- 172.21.15.49:0/4170819383 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f323c154320 con 0x7f323c159b40 2023-11-06T02:09:34.613 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.608+0000 7f3241789640 1 -- 172.21.15.49:0/4170819383 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f323c074040 con 0x7f323c159b40 2023-11-06T02:09:34.614 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.608+0000 7f32337fe640 1 -- 172.21.15.49:0/4170819383 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f322c004b70 con 0x7f323c159b40 2023-11-06T02:09:34.614 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.608+0000 7f32337fe640 1 --2- 172.21.15.49:0/4170819383 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f321c065a90 0x7f321c067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:34.614 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.608+0000 7f32337fe640 1 -- 172.21.15.49:0/4170819383 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f322c082850 con 0x7f323c159b40 2023-11-06T02:09:34.614 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.608+0000 7f323affd640 1 --2- 172.21.15.49:0/4170819383 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f321c065a90 0x7f321c067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:34.615 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.612+0000 7f323affd640 1 --2- 172.21.15.49:0/4170819383 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f321c065a90 0x7f321c067f50 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f32340070b0 tx=0x7f3234007040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:34.615 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.612+0000 7f32337fe640 1 -- 172.21.15.49:0/4170819383 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f322c050490 con 0x7f323c159b40 2023-11-06T02:09:34.822 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.824+0000 7f3241789640 1 -- 172.21.15.49:0/4170819383 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f323c1544e0 con 0x7f323c159b40 2023-11-06T02:09:34.831 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.832+0000 7f32337fe640 1 -- 172.21.15.49:0/4170819383 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7f322c053ab0 con 0x7f323c159b40 2023-11-06T02:09:34.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.836+0000 7f3241789640 1 -- 172.21.15.49:0/4170819383 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f321c065a90 msgr2=0x7f321c067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:34.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.836+0000 7f3241789640 1 --2- 172.21.15.49:0/4170819383 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f321c065a90 0x7f321c067f50 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f32340070b0 tx=0x7f3234007040 comp rx=0 tx=0).stop 2023-11-06T02:09:34.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.836+0000 7f3241789640 1 -- 172.21.15.49:0/4170819383 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f323c159b40 msgr2=0x7f323c151f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:34.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.836+0000 7f3241789640 1 --2- 172.21.15.49:0/4170819383 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f323c159b40 0x7f323c151f10 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f322c00c3c0 tx=0x7f322c00c880 comp rx=0 tx=0).stop 2023-11-06T02:09:34.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.836+0000 7f3241789640 1 -- 172.21.15.49:0/4170819383 shutdown_connections 2023-11-06T02:09:34.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.836+0000 7f3241789640 1 --2- 172.21.15.49:0/4170819383 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f321c065a90 0x7f321c067f50 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:34.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.836+0000 7f3241789640 1 --2- 172.21.15.49:0/4170819383 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f323c159b40 0x7f323c151f10 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:34.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.836+0000 7f3241789640 1 --2- 172.21.15.49:0/4170819383 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f323c0cdd50 0x7f323c1519d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:34.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.836+0000 7f3241789640 1 --2- 172.21.15.49:0/4170819383 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f323c0cb010 0x7f323c158880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:34.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.836+0000 7f3241789640 1 -- 172.21.15.49:0/4170819383 >> 172.21.15.49:0/4170819383 conn(0x7f323c06c0a0 msgr2=0x7f323c15c970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:34.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.836+0000 7f3241789640 1 -- 172.21.15.49:0/4170819383 shutdown_connections 2023-11-06T02:09:34.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.836+0000 7f3241789640 1 -- 172.21.15.49:0/4170819383 wait complete. 2023-11-06T02:09:34.850 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-11-06T02:09:34.965 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.964+0000 7f25af466640 1 Processor -- start 2023-11-06T02:09:34.965 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.964+0000 7f25af466640 1 -- start start 2023-11-06T02:09:34.966 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.968+0000 7f25af466640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f25a80b1f90 0x7f25a80cda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:34.966 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.968+0000 7f25af466640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f25a8161650 0x7f25a80d1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:34.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.968+0000 7f25af466640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f25a80d2190 0x7f25a80714a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:34.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.968+0000 7f25af466640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25a80d17c0 con 0x7f25a8161650 2023-11-06T02:09:34.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.968+0000 7f25af466640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25a80ce6c0 con 0x7f25a80d2190 2023-11-06T02:09:34.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.968+0000 7f25af466640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25a8161ad0 con 0x7f25a80b1f90 2023-11-06T02:09:34.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.968+0000 7f25aca01640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f25a8161650 0x7f25a80d1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:34.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.968+0000 7f25aca01640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f25a8161650 0x7f25a80d1c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40686/0 (socket says 172.21.15.49:40686) 2023-11-06T02:09:34.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.968+0000 7f25aca01640 1 -- 172.21.15.49:0/1288775860 learned_addr learned my addr 172.21.15.49:0/1288775860 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:34.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.968+0000 7f25ada03640 1 --2- 172.21.15.49:0/1288775860 >> v2:172.21.15.150:3300/0 conn(0x7f25a80d2190 0x7f25a80714a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:34.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.968+0000 7f25ad202640 1 --2- 172.21.15.49:0/1288775860 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f25a80b1f90 0x7f25a80cda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:34.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.968+0000 7f25aca01640 1 -- 172.21.15.49:0/1288775860 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f25a80b1f90 msgr2=0x7f25a80cda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:34.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.968+0000 7f25aca01640 1 --2- 172.21.15.49:0/1288775860 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f25a80b1f90 0x7f25a80cda70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:34.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.968+0000 7f25aca01640 1 -- 172.21.15.49:0/1288775860 >> v2:172.21.15.150:3300/0 conn(0x7f25a80d2190 msgr2=0x7f25a80714a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:34.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.968+0000 7f25aca01640 1 --2- 172.21.15.49:0/1288775860 >> v2:172.21.15.150:3300/0 conn(0x7f25a80d2190 0x7f25a80714a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:34.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.968+0000 7f25aca01640 1 -- 172.21.15.49:0/1288775860 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25a80ce100 con 0x7f25a8161650 2023-11-06T02:09:34.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.968+0000 7f25aca01640 1 --2- 172.21.15.49:0/1288775860 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25a8161650 0x7f25a80d1c50 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f25a806c560 tx=0x7f259801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fa6f57f95e3d91f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:34.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.968+0000 7f259ffff640 1 -- 172.21.15.49:0/1288775860 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25980ac600 con 0x7f25a8161650 2023-11-06T02:09:34.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.968+0000 7f259ffff640 1 -- 172.21.15.49:0/1288775860 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25980a8050 con 0x7f25a8161650 2023-11-06T02:09:34.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.968+0000 7f259ffff640 1 -- 172.21.15.49:0/1288775860 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25980ac980 con 0x7f25a8161650 2023-11-06T02:09:34.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.968+0000 7f25af466640 1 -- 172.21.15.49:0/1288775860 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25a8161650 msgr2=0x7f25a80d1c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:34.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.968+0000 7f25af466640 1 --2- 172.21.15.49:0/1288775860 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25a8161650 0x7f25a80d1c50 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f25a806c560 tx=0x7f259801c4d0 comp rx=0 tx=0).stop 2023-11-06T02:09:34.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.968+0000 7f25af466640 1 -- 172.21.15.49:0/1288775860 shutdown_connections 2023-11-06T02:09:34.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.968+0000 7f25af466640 1 --2- 172.21.15.49:0/1288775860 >> v2:172.21.15.150:3300/0 conn(0x7f25a80d2190 0x7f25a80714a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:34.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.968+0000 7f25af466640 1 --2- 172.21.15.49:0/1288775860 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25a8161650 0x7f25a80d1c50 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:34.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.968+0000 7f25af466640 1 --2- 172.21.15.49:0/1288775860 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f25a80b1f90 0x7f25a80cda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:34.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.968+0000 7f25af466640 1 -- 172.21.15.49:0/1288775860 >> 172.21.15.49:0/1288775860 conn(0x7f25a80d2aa0 msgr2=0x7f25a80d2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:34.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.968+0000 7f25af466640 1 -- 172.21.15.49:0/1288775860 shutdown_connections 2023-11-06T02:09:34.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.968+0000 7f25af466640 1 -- 172.21.15.49:0/1288775860 wait complete. 2023-11-06T02:09:34.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.972+0000 7f25af466640 1 Processor -- start 2023-11-06T02:09:34.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.972+0000 7f25af466640 1 -- start start 2023-11-06T02:09:34.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.972+0000 7f25af466640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f25a80b1f90 0x7f25a8154a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:34.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.972+0000 7f25af466640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25a80d2190 0x7f25a8151ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:34.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.972+0000 7f25af466640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25a8161650 0x7f25a8150870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:34.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.972+0000 7f25af466640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a815a760 con 0x7f25a80d2190 2023-11-06T02:09:34.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.972+0000 7f25af466640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a8151350 con 0x7f25a8161650 2023-11-06T02:09:34.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.972+0000 7f25af466640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25a8151ff0 con 0x7f25a80b1f90 2023-11-06T02:09:34.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.972+0000 7f25aca01640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25a80d2190 0x7f25a8151ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:34.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.972+0000 7f25aca01640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25a80d2190 0x7f25a8151ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40696/0 (socket says 172.21.15.49:40696) 2023-11-06T02:09:34.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.972+0000 7f25aca01640 1 -- 172.21.15.49:0/808372574 learned_addr learned my addr 172.21.15.49:0/808372574 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:34.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.972+0000 7f25aca01640 1 -- 172.21.15.49:0/808372574 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f25a80b1f90 msgr2=0x7f25a8154a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:34.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.972+0000 7f25aca01640 1 --2- 172.21.15.49:0/808372574 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f25a80b1f90 0x7f25a8154a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:34.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.972+0000 7f25aca01640 1 -- 172.21.15.49:0/808372574 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25a8161650 msgr2=0x7f25a8150870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:34.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.972+0000 7f25ada03640 1 --2- 172.21.15.49:0/808372574 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25a8161650 0x7f25a8150870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:34.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.972+0000 7f25aca01640 1 --2- 172.21.15.49:0/808372574 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25a8161650 0x7f25a8150870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:34.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.972+0000 7f25aca01640 1 -- 172.21.15.49:0/808372574 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25980094d0 con 0x7f25a80d2190 2023-11-06T02:09:34.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.972+0000 7f25aca01640 1 --2- 172.21.15.49:0/808372574 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25a80d2190 0x7f25a8151ab0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f259801ab80 tx=0x7f2598019270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:34.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.972+0000 7f259dffb640 1 -- 172.21.15.49:0/808372574 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25980bb070 con 0x7f25a80d2190 2023-11-06T02:09:34.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.972+0000 7f259dffb640 1 -- 172.21.15.49:0/808372574 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25980b56f0 con 0x7f25a80d2190 2023-11-06T02:09:34.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.972+0000 7f25af466640 1 -- 172.21.15.49:0/808372574 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25a815a1a0 con 0x7f25a80d2190 2023-11-06T02:09:34.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.972+0000 7f259dffb640 1 -- 172.21.15.49:0/808372574 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25980b5aa0 con 0x7f25a80d2190 2023-11-06T02:09:34.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.972+0000 7f25af466640 1 -- 172.21.15.49:0/808372574 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25a8154220 con 0x7f25a80d2190 2023-11-06T02:09:34.980 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.972+0000 7f25af466640 1 -- 172.21.15.49:0/808372574 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25a8074040 con 0x7f25a80d2190 2023-11-06T02:09:34.980 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.976+0000 7f259dffb640 1 -- 172.21.15.49:0/808372574 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f25980b3070 con 0x7f25a80d2190 2023-11-06T02:09:34.980 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.976+0000 7f259dffb640 1 --2- 172.21.15.49:0/808372574 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f258c065a90 0x7f258c067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:34.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.976+0000 7f259dffb640 1 -- 172.21.15.49:0/808372574 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f259811d030 con 0x7f25a80d2190 2023-11-06T02:09:34.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.976+0000 7f25ad202640 1 --2- 172.21.15.49:0/808372574 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f258c065a90 0x7f258c067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:34.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.976+0000 7f25ad202640 1 --2- 172.21.15.49:0/808372574 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f258c065a90 0x7f258c067f50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f25a4006b20 tx=0x7f25a4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:34.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:34.980+0000 7f259dffb640 1 -- 172.21.15.49:0/808372574 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f25980eac70 con 0x7f25a80d2190 2023-11-06T02:09:35.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.188+0000 7f25af466640 1 -- 172.21.15.49:0/808372574 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f25a80d0c30 con 0x7f25a80d2190 2023-11-06T02:09:35.196 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.196+0000 7f259dffb640 1 -- 172.21.15.49:0/808372574 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7f25980ee290 con 0x7f25a80d2190 2023-11-06T02:09:35.198 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.200+0000 7f25af466640 1 -- 172.21.15.49:0/808372574 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f258c065a90 msgr2=0x7f258c067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:35.199 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.200+0000 7f25af466640 1 --2- 172.21.15.49:0/808372574 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f258c065a90 0x7f258c067f50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f25a4006b20 tx=0x7f25a4008040 comp rx=0 tx=0).stop 2023-11-06T02:09:35.199 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.200+0000 7f25af466640 1 -- 172.21.15.49:0/808372574 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25a80d2190 msgr2=0x7f25a8151ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:35.199 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.200+0000 7f25af466640 1 --2- 172.21.15.49:0/808372574 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25a80d2190 0x7f25a8151ab0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f259801ab80 tx=0x7f2598019270 comp rx=0 tx=0).stop 2023-11-06T02:09:35.199 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.200+0000 7f25af466640 1 -- 172.21.15.49:0/808372574 shutdown_connections 2023-11-06T02:09:35.200 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.200+0000 7f25af466640 1 --2- 172.21.15.49:0/808372574 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f258c065a90 0x7f258c067f50 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:35.200 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.200+0000 7f25af466640 1 --2- 172.21.15.49:0/808372574 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25a80d2190 0x7f25a8151ab0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:35.200 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.200+0000 7f25af466640 1 --2- 172.21.15.49:0/808372574 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25a8161650 0x7f25a8150870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:35.200 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.200+0000 7f25af466640 1 --2- 172.21.15.49:0/808372574 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f25a80b1f90 0x7f25a8154a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:35.200 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.200+0000 7f25af466640 1 -- 172.21.15.49:0/808372574 >> 172.21.15.49:0/808372574 conn(0x7f25a80d2aa0 msgr2=0x7f25a80d2e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:35.201 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.200+0000 7f25af466640 1 -- 172.21.15.49:0/808372574 shutdown_connections 2023-11-06T02:09:35.201 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.200+0000 7f25af466640 1 -- 172.21.15.49:0/808372574 wait complete. 2023-11-06T02:09:35.215 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-11-06T02:09:35.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.332+0000 7f1edb279640 1 Processor -- start 2023-11-06T02:09:35.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.332+0000 7f1edb279640 1 -- start start 2023-11-06T02:09:35.335 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.332+0000 7f1edb279640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f1ed40b0790 0x7f1ed40cb110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:35.335 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.332+0000 7f1edb279640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ed40cb840 con 0x7f1ed415aae0 2023-11-06T02:09:35.335 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.332+0000 7f1edb279640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ed40cbb20 con 0x7f1ed4070a00 2023-11-06T02:09:35.335 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.332+0000 7f1edb279640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ed415aec0 con 0x7f1ed40b0790 2023-11-06T02:09:35.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.332+0000 7f1ed9015640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f1ed415aae0 0x7f1ed40b0080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:51454/0 (socket says 172.21.15.49:51454) 2023-11-06T02:09:35.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.332+0000 7f1ed9015640 1 -- 172.21.15.49:0/2754329973 learned_addr learned my addr 172.21.15.49:0/2754329973 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:09:35.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.336+0000 7f1ed8814640 1 --2- 172.21.15.49:0/2754329973 >> v2:172.21.15.150:3300/0 conn(0x7f1ed40b0790 0x7f1ed40cb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:35.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.336+0000 7f1ecbfff640 1 -- 172.21.15.49:0/2754329973 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3577218478 0 0) 0x7f1ed40cb840 con 0x7f1ed415aae0 2023-11-06T02:09:35.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.336+0000 7f1ecbfff640 1 -- 172.21.15.49:0/2754329973 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ec0003530 con 0x7f1ed415aae0 2023-11-06T02:09:35.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.336+0000 7f1ecbfff640 1 -- 172.21.15.49:0/2754329973 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 551157519 0 0) 0x7f1ed40cbb20 con 0x7f1ed4070a00 2023-11-06T02:09:35.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.336+0000 7f1ecbfff640 1 -- 172.21.15.49:0/2754329973 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ed40cb840 con 0x7f1ed4070a00 2023-11-06T02:09:35.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.336+0000 7f1ecbfff640 1 -- 172.21.15.49:0/2754329973 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3284206556 0 0) 0x7f1ec0003530 con 0x7f1ed415aae0 2023-11-06T02:09:35.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.336+0000 7f1ecbfff640 1 -- 172.21.15.49:0/2754329973 >> v2:172.21.15.150:3300/0 conn(0x7f1ed40b0790 msgr2=0x7f1ed40cb110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:35.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.336+0000 7f1ecbfff640 1 --2- 172.21.15.49:0/2754329973 >> v2:172.21.15.150:3300/0 conn(0x7f1ed40b0790 0x7f1ed40cb110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:35.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.336+0000 7f1ecbfff640 1 -- 172.21.15.49:0/2754329973 >> v1:172.21.15.150:6789/0 conn(0x7f1ed4070a00 legacy=0x7f1ed40b1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:35.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.336+0000 7f1ecbfff640 1 -- 172.21.15.49:0/2754329973 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ed40cbe00 con 0x7f1ed415aae0 2023-11-06T02:09:35.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.336+0000 7f1ecbfff640 1 -- 172.21.15.49:0/2754329973 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7f1ed0001800 con 0x7f1ed415aae0 2023-11-06T02:09:35.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.336+0000 7f1ecbfff640 1 -- 172.21.15.49:0/2754329973 >> v1:172.21.15.49:6789/0 conn(0x7f1ed415aae0 legacy=0x7f1ed40b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:35.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.336+0000 7f1ecbfff640 1 --2- 172.21.15.49:0/2754329973 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ec0003b60 0x7f1ec0003f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:35.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.336+0000 7f1ecbfff640 1 --2- 172.21.15.49:0/2754329973 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ec00047d0 0x7f1ec0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:35.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.336+0000 7f1ecbfff640 1 --2- 172.21.15.49:0/2754329973 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1ec0005190 0x7f1ec000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:35.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.336+0000 7f1ecbfff640 1 -- 172.21.15.49:0/2754329973 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed415aec0 con 0x7f1ec0003b60 2023-11-06T02:09:35.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.336+0000 7f1ecbfff640 1 -- 172.21.15.49:0/2754329973 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec000c060 con 0x7f1ec00047d0 2023-11-06T02:09:35.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.336+0000 7f1ecbfff640 1 -- 172.21.15.49:0/2754329973 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec000c220 con 0x7f1ec0005190 2023-11-06T02:09:35.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.336+0000 7f1ed9015640 1 --2- 172.21.15.49:0/2754329973 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ec0003b60 0x7f1ec0003f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:35.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.336+0000 7f1ed8814640 1 --2- 172.21.15.49:0/2754329973 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ec00047d0 0x7f1ec0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:35.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.336+0000 7f1ed9015640 1 -- 172.21.15.49:0/2754329973 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1ec0005190 msgr2=0x7f1ec000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:35.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.336+0000 7f1ed9015640 1 --2- 172.21.15.49:0/2754329973 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1ec0005190 0x7f1ec000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:35.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.336+0000 7f1ed9015640 1 -- 172.21.15.49:0/2754329973 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ec00047d0 msgr2=0x7f1ec0004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:35.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.336+0000 7f1ed9015640 1 --2- 172.21.15.49:0/2754329973 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ec00047d0 0x7f1ec0004c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:35.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.336+0000 7f1ed9015640 1 -- 172.21.15.49:0/2754329973 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ec0003530 con 0x7f1ec0003b60 2023-11-06T02:09:35.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.336+0000 7f1ed9816640 1 -- 172.21.15.49:0/2754329973 reap_dead start 2023-11-06T02:09:35.344 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.336+0000 7f1ed9015640 1 --2- 172.21.15.49:0/2754329973 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ec0003b60 0x7f1ec0003f60 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f1ed0007fb0 tx=0x7f1ed001caf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aa07d8ad2a2c825 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:35.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.336+0000 7f1ecbfff640 1 -- 172.21.15.49:0/2754329973 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed0007270 con 0x7f1ec0003b60 2023-11-06T02:09:35.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.336+0000 7f1ecbfff640 1 -- 172.21.15.49:0/2754329973 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ed00073f0 con 0x7f1ec0003b60 2023-11-06T02:09:35.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.336+0000 7f1ecbfff640 1 -- 172.21.15.49:0/2754329973 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed00057b0 con 0x7f1ec0003b60 2023-11-06T02:09:35.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.340+0000 7f1edb279640 1 -- 172.21.15.49:0/2754329973 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ec0003b60 msgr2=0x7f1ec0003f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:35.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.340+0000 7f1edb279640 1 --2- 172.21.15.49:0/2754329973 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ec0003b60 0x7f1ec0003f60 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f1ed0007fb0 tx=0x7f1ed001caf0 comp rx=0 tx=0).stop 2023-11-06T02:09:35.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.340+0000 7f1edb279640 1 -- 172.21.15.49:0/2754329973 shutdown_connections 2023-11-06T02:09:35.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.340+0000 7f1edb279640 1 --2- 172.21.15.49:0/2754329973 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ec0003b60 0x7f1ec0003f60 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:35.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.340+0000 7f1edb279640 1 -- 172.21.15.49:0/2754329973 >> 172.21.15.49:0/2754329973 conn(0x7f1ed406c0a0 msgr2=0x7f1ed406c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:35.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.340+0000 7f1edb279640 1 -- 172.21.15.49:0/2754329973 shutdown_connections 2023-11-06T02:09:35.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.340+0000 7f1edb279640 1 -- 172.21.15.49:0/2754329973 wait complete. 2023-11-06T02:09:35.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.340+0000 7f1edb279640 1 Processor -- start 2023-11-06T02:09:35.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.340+0000 7f1edb279640 1 -- start start 2023-11-06T02:09:35.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.340+0000 7f1edb279640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1ec0003b60 0x7f1ed415b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:35.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.340+0000 7f1edb279640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ed40c3420 0x7f1ed40c3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:35.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.340+0000 7f1edb279640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ed40a8410 0x7f1ed40a8890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:35.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.340+0000 7f1edb279640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed40a7650 con 0x7f1ed40a8410 2023-11-06T02:09:35.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.340+0000 7f1edb279640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed40a7930 con 0x7f1ed40c3420 2023-11-06T02:09:35.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.340+0000 7f1edb279640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed40a7c10 con 0x7f1ec0003b60 2023-11-06T02:09:35.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.340+0000 7f1ed9816640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ed40a8410 0x7f1ed40a8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:35.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.340+0000 7f1ed9816640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ed40a8410 0x7f1ed40a8890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40712/0 (socket says 172.21.15.49:40712) 2023-11-06T02:09:35.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.340+0000 7f1ed9816640 1 -- 172.21.15.49:0/522993435 learned_addr learned my addr 172.21.15.49:0/522993435 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:35.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.340+0000 7f1ed8814640 1 --2- 172.21.15.49:0/522993435 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ed40c3420 0x7f1ed40c3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:35.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.340+0000 7f1ed9015640 1 --2- 172.21.15.49:0/522993435 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1ec0003b60 0x7f1ed415b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:35.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.340+0000 7f1ed9816640 1 -- 172.21.15.49:0/522993435 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1ec0003b60 msgr2=0x7f1ed415b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:35.350 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.340+0000 7f1ed9816640 1 --2- 172.21.15.49:0/522993435 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1ec0003b60 0x7f1ed415b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:35.350 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.340+0000 7f1ed9816640 1 -- 172.21.15.49:0/522993435 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ed40c3420 msgr2=0x7f1ed40c3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:35.350 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.340+0000 7f1ed9816640 1 --2- 172.21.15.49:0/522993435 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ed40c3420 0x7f1ed40c3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:35.350 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.340+0000 7f1ed9816640 1 -- 172.21.15.49:0/522993435 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ed40a7ef0 con 0x7f1ed40a8410 2023-11-06T02:09:35.351 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.340+0000 7f1ed9816640 1 --2- 172.21.15.49:0/522993435 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ed40a8410 0x7f1ed40a8890 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f1ed40b0cf0 tx=0x7f1ecc00caa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:35.351 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.340+0000 7f1ec9ffb640 1 -- 172.21.15.49:0/522993435 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ecc007070 con 0x7f1ed40a8410 2023-11-06T02:09:35.351 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.340+0000 7f1ec9ffb640 1 -- 172.21.15.49:0/522993435 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ecc0071f0 con 0x7f1ed40a8410 2023-11-06T02:09:35.351 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.340+0000 7f1edb279640 1 -- 172.21.15.49:0/522993435 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ed40c3dc0 con 0x7f1ed40a8410 2023-11-06T02:09:35.351 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.340+0000 7f1ec9ffb640 1 -- 172.21.15.49:0/522993435 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ecc0075a0 con 0x7f1ed40a8410 2023-11-06T02:09:35.351 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.340+0000 7f1edb279640 1 -- 172.21.15.49:0/522993435 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ed415b9e0 con 0x7f1ed40a8410 2023-11-06T02:09:35.352 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.340+0000 7f1edb279640 1 -- 172.21.15.49:0/522993435 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ed40a8130 con 0x7f1ed40a8410 2023-11-06T02:09:35.352 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.344+0000 7f1ec9ffb640 1 -- 172.21.15.49:0/522993435 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f1ecc007700 con 0x7f1ed40a8410 2023-11-06T02:09:35.352 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.344+0000 7f1ec9ffb640 1 --2- 172.21.15.49:0/522993435 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1eb8065a60 0x7f1eb8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:35.352 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.344+0000 7f1ec9ffb640 1 -- 172.21.15.49:0/522993435 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f1ecc0824e0 con 0x7f1ed40a8410 2023-11-06T02:09:35.352 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.344+0000 7f1ed9015640 1 --2- 172.21.15.49:0/522993435 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1eb8065a60 0x7f1eb8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:35.353 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.344+0000 7f1ed9015640 1 --2- 172.21.15.49:0/522993435 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1eb8065a60 0x7f1eb8067f20 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1ed00ab000 tx=0x7f1ed00a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:35.353 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.348+0000 7f1ec9ffb640 1 -- 172.21.15.49:0/522993435 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f1ed40a8130 con 0x7f1ed40a8410 2023-11-06T02:09:35.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.556+0000 7f1edb279640 1 -- 172.21.15.49:0/522993435 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f1ed40a8130 con 0x7f1ed40a8410 2023-11-06T02:09:35.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.564+0000 7f1ec9ffb640 1 -- 172.21.15.49:0/522993435 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f1ed40a8130 con 0x7f1ed40a8410 2023-11-06T02:09:35.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.564+0000 7f1edb279640 1 -- 172.21.15.49:0/522993435 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1eb8065a60 msgr2=0x7f1eb8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:35.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.564+0000 7f1edb279640 1 --2- 172.21.15.49:0/522993435 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1eb8065a60 0x7f1eb8067f20 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1ed00ab000 tx=0x7f1ed00a7070 comp rx=0 tx=0).stop 2023-11-06T02:09:35.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.564+0000 7f1edb279640 1 -- 172.21.15.49:0/522993435 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ed40a8410 msgr2=0x7f1ed40a8890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:35.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.564+0000 7f1edb279640 1 --2- 172.21.15.49:0/522993435 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ed40a8410 0x7f1ed40a8890 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f1ed40b0cf0 tx=0x7f1ecc00caa0 comp rx=0 tx=0).stop 2023-11-06T02:09:35.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.564+0000 7f1edb279640 1 -- 172.21.15.49:0/522993435 shutdown_connections 2023-11-06T02:09:35.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.564+0000 7f1edb279640 1 --2- 172.21.15.49:0/522993435 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1eb8065a60 0x7f1eb8067f20 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:35.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.564+0000 7f1edb279640 1 --2- 172.21.15.49:0/522993435 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ed40a8410 0x7f1ed40a8890 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:35.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.564+0000 7f1edb279640 1 --2- 172.21.15.49:0/522993435 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ed40c3420 0x7f1ed40c3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:35.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.564+0000 7f1edb279640 1 --2- 172.21.15.49:0/522993435 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1ec0003b60 0x7f1ed415b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:35.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.564+0000 7f1edb279640 1 -- 172.21.15.49:0/522993435 >> 172.21.15.49:0/522993435 conn(0x7f1ed406c0a0 msgr2=0x7f1ed40ca500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:35.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.564+0000 7f1edb279640 1 -- 172.21.15.49:0/522993435 shutdown_connections 2023-11-06T02:09:35.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.564+0000 7f1edb279640 1 -- 172.21.15.49:0/522993435 wait complete. 2023-11-06T02:09:35.580 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-11-06T02:09:35.703 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.704+0000 7f7b1a8a5640 1 Processor -- start 2023-11-06T02:09:35.703 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.704+0000 7f7b1a8a5640 1 -- start start 2023-11-06T02:09:35.706 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.704+0000 7f7b1a8a5640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f7b140b0790 0x7f7b140cc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:35.706 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.704+0000 7f7b1a8a5640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b14165b80 con 0x7f7b141646f0 2023-11-06T02:09:35.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.704+0000 7f7b1a8a5640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b14165600 con 0x7f7b140c8c50 2023-11-06T02:09:35.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.704+0000 7f7b1a8a5640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b141658a0 con 0x7f7b140b0790 2023-11-06T02:09:35.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.704+0000 7f7b13fff640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f7b141646f0 0x7f7b140b0080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:51460/0 (socket says 172.21.15.49:51460) 2023-11-06T02:09:35.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.704+0000 7f7b13fff640 1 -- 172.21.15.49:0/1676225975 learned_addr learned my addr 172.21.15.49:0/1676225975 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:09:35.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.704+0000 7f7b0bfff640 1 --2- 172.21.15.49:0/1676225975 >> v2:172.21.15.150:3300/0 conn(0x7f7b140b0790 0x7f7b140cc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:35.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.704+0000 7f7b137fe640 1 -- 172.21.15.49:0/1676225975 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1687944232 0 0) 0x7f7b14165b80 con 0x7f7b141646f0 2023-11-06T02:09:35.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.704+0000 7f7b137fe640 1 -- 172.21.15.49:0/1676225975 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b00003530 con 0x7f7b141646f0 2023-11-06T02:09:35.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.704+0000 7f7b137fe640 1 -- 172.21.15.49:0/1676225975 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 319363400 0 0) 0x7f7b00003530 con 0x7f7b141646f0 2023-11-06T02:09:35.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.704+0000 7f7b137fe640 1 -- 172.21.15.49:0/1676225975 >> v2:172.21.15.150:3300/0 conn(0x7f7b140b0790 msgr2=0x7f7b140cc990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:35.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.704+0000 7f7b137fe640 1 --2- 172.21.15.49:0/1676225975 >> v2:172.21.15.150:3300/0 conn(0x7f7b140b0790 0x7f7b140cc990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:35.709 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.704+0000 7f7b137fe640 1 -- 172.21.15.49:0/1676225975 >> v1:172.21.15.150:6789/0 conn(0x7f7b140c8c50 legacy=0x7f7b140cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:35.709 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.704+0000 7f7b137fe640 1 -- 172.21.15.49:0/1676225975 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b140c96f0 con 0x7f7b141646f0 2023-11-06T02:09:35.709 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.704+0000 7f7b137fe640 1 -- 172.21.15.49:0/1676225975 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7f7b04001800 con 0x7f7b141646f0 2023-11-06T02:09:35.709 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.704+0000 7f7b137fe640 1 -- 172.21.15.49:0/1676225975 >> v1:172.21.15.49:6789/0 conn(0x7f7b141646f0 legacy=0x7f7b140b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:35.709 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.704+0000 7f7b137fe640 1 --2- 172.21.15.49:0/1676225975 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b00003980 0x7f7b00003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:35.710 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.704+0000 7f7b137fe640 1 --2- 172.21.15.49:0/1676225975 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7b000047d0 0x7f7b00004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:35.710 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.704+0000 7f7b137fe640 1 --2- 172.21.15.49:0/1676225975 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b00005190 0x7f7b0000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:35.710 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.704+0000 7f7b137fe640 1 -- 172.21.15.49:0/1676225975 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b141658a0 con 0x7f7b00005190 2023-11-06T02:09:35.710 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.704+0000 7f7b137fe640 1 -- 172.21.15.49:0/1676225975 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0000c030 con 0x7f7b00003980 2023-11-06T02:09:35.710 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.704+0000 7f7b137fe640 1 -- 172.21.15.49:0/1676225975 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0000c1f0 con 0x7f7b000047d0 2023-11-06T02:09:35.711 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.708+0000 7f7b18e42640 1 --2- 172.21.15.49:0/1676225975 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b00005190 0x7f7b0000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:35.711 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.708+0000 7f7b13fff640 1 --2- 172.21.15.49:0/1676225975 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b00003980 0x7f7b00003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:35.711 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.708+0000 7f7b0bfff640 1 --2- 172.21.15.49:0/1676225975 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7b000047d0 0x7f7b00004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:35.711 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.708+0000 7f7b18e42640 1 -- 172.21.15.49:0/1676225975 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7b000047d0 msgr2=0x7f7b00004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:35.711 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.708+0000 7f7b18e42640 1 --2- 172.21.15.49:0/1676225975 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7b000047d0 0x7f7b00004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:35.712 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.708+0000 7f7b18e42640 1 -- 172.21.15.49:0/1676225975 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b00003980 msgr2=0x7f7b00003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:35.712 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.708+0000 7f7b18e42640 1 --2- 172.21.15.49:0/1676225975 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b00003980 0x7f7b00003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:35.712 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.708+0000 7f7b18e42640 1 -- 172.21.15.49:0/1676225975 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b00003530 con 0x7f7b00005190 2023-11-06T02:09:35.715 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.708+0000 7f7b18e42640 1 -- 172.21.15.49:0/1676225975 reap_dead start 2023-11-06T02:09:35.715 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.708+0000 7f7b18e42640 1 --2- 172.21.15.49:0/1676225975 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b00005190 0x7f7b0000b990 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c001e40 tx=0x7f7b0c01c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fefc215739f5417b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:35.716 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.708+0000 7f7b137fe640 1 -- 172.21.15.49:0/1676225975 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b0c003b80 con 0x7f7b00005190 2023-11-06T02:09:35.716 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.708+0000 7f7b137fe640 1 -- 172.21.15.49:0/1676225975 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b0c003ce0 con 0x7f7b00005190 2023-11-06T02:09:35.716 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.708+0000 7f7b137fe640 1 -- 172.21.15.49:0/1676225975 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b0c005260 con 0x7f7b00005190 2023-11-06T02:09:35.716 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.708+0000 7f7b1a8a5640 1 -- 172.21.15.49:0/1676225975 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b00005190 msgr2=0x7f7b0000b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:35.716 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.708+0000 7f7b1a8a5640 1 --2- 172.21.15.49:0/1676225975 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b00005190 0x7f7b0000b990 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c001e40 tx=0x7f7b0c01c740 comp rx=0 tx=0).stop 2023-11-06T02:09:35.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.708+0000 7f7b1a8a5640 1 -- 172.21.15.49:0/1676225975 shutdown_connections 2023-11-06T02:09:35.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.708+0000 7f7b1a8a5640 1 --2- 172.21.15.49:0/1676225975 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b00005190 0x7f7b0000b990 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:35.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.708+0000 7f7b1a8a5640 1 -- 172.21.15.49:0/1676225975 >> 172.21.15.49:0/1676225975 conn(0x7f7b1406c0a0 msgr2=0x7f7b1406c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:35.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.708+0000 7f7b1a8a5640 1 -- 172.21.15.49:0/1676225975 shutdown_connections 2023-11-06T02:09:35.718 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.708+0000 7f7b1a8a5640 1 -- 172.21.15.49:0/1676225975 wait complete. 2023-11-06T02:09:35.718 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.712+0000 7f7b1a8a5640 1 Processor -- start 2023-11-06T02:09:35.718 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.712+0000 7f7b1a8a5640 1 -- start start 2023-11-06T02:09:35.718 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.712+0000 7f7b1a8a5640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b00005190 0x7f7b1406ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:35.718 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.712+0000 7f7b1a8a5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b1415b160 0x7f7b1415aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:35.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.712+0000 7f7b1a8a5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7b14071cd0 0x7f7b14160080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:35.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.712+0000 7f7b1a8a5640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1415b000 con 0x7f7b1415b160 2023-11-06T02:09:35.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.712+0000 7f7b1a8a5640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b140b1b10 con 0x7f7b00005190 2023-11-06T02:09:35.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.712+0000 7f7b1a8a5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b140b0f00 con 0x7f7b14071cd0 2023-11-06T02:09:35.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.712+0000 7f7b0bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b1415b160 0x7f7b1415aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:35.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.712+0000 7f7b0bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b1415b160 0x7f7b1415aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40734/0 (socket says 172.21.15.49:40734) 2023-11-06T02:09:35.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.712+0000 7f7b0bfff640 1 -- 172.21.15.49:0/2434241660 learned_addr learned my addr 172.21.15.49:0/2434241660 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:35.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.712+0000 7f7b18e42640 1 --2- 172.21.15.49:0/2434241660 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7b14071cd0 0x7f7b14160080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:35.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.712+0000 7f7b0bfff640 1 -- 172.21.15.49:0/2434241660 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7b14071cd0 msgr2=0x7f7b14160080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:35.721 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.712+0000 7f7b0bfff640 1 --2- 172.21.15.49:0/2434241660 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7b14071cd0 0x7f7b14160080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:35.721 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.712+0000 7f7b0bfff640 1 -- 172.21.15.49:0/2434241660 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b00005190 msgr2=0x7f7b1406ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:35.721 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.712+0000 7f7b0bfff640 1 --2- 172.21.15.49:0/2434241660 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b00005190 0x7f7b1406ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:35.721 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.712+0000 7f7b0bfff640 1 -- 172.21.15.49:0/2434241660 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b140b11e0 con 0x7f7b1415b160 2023-11-06T02:09:35.721 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.712+0000 7f7b0bfff640 1 --2- 172.21.15.49:0/2434241660 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b1415b160 0x7f7b1415aac0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f7afc011590 tx=0x7f7afc011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:35.722 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.712+0000 7f7b117fa640 1 -- 172.21.15.49:0/2434241660 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7afc015870 con 0x7f7b1415b160 2023-11-06T02:09:35.722 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.712+0000 7f7b117fa640 1 -- 172.21.15.49:0/2434241660 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7afc0159d0 con 0x7f7b1415b160 2023-11-06T02:09:35.722 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.712+0000 7f7b1a8a5640 1 -- 172.21.15.49:0/2434241660 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b140b14c0 con 0x7f7b1415b160 2023-11-06T02:09:35.722 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.712+0000 7f7b117fa640 1 -- 172.21.15.49:0/2434241660 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7afc015d40 con 0x7f7b1415b160 2023-11-06T02:09:35.723 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.712+0000 7f7b1a8a5640 1 -- 172.21.15.49:0/2434241660 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b14072150 con 0x7f7b1415b160 2023-11-06T02:09:35.723 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.712+0000 7f7b1a8a5640 1 -- 172.21.15.49:0/2434241660 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b14074040 con 0x7f7b1415b160 2023-11-06T02:09:35.723 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.712+0000 7f7b117fa640 1 -- 172.21.15.49:0/2434241660 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f7afc013400 con 0x7f7b1415b160 2023-11-06T02:09:35.723 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.712+0000 7f7b117fa640 1 --2- 172.21.15.49:0/2434241660 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7af4065a60 0x7f7af4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:35.723 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.712+0000 7f7b117fa640 1 -- 172.21.15.49:0/2434241660 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f7afc083280 con 0x7f7b1415b160 2023-11-06T02:09:35.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.716+0000 7f7b13fff640 1 --2- 172.21.15.49:0/2434241660 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7af4065a60 0x7f7af4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:35.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.716+0000 7f7b13fff640 1 --2- 172.21.15.49:0/2434241660 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7af4065a60 0x7f7af4067f20 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7b040011e0 tx=0x7f7b04009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:35.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.716+0000 7f7b117fa640 1 -- 172.21.15.49:0/2434241660 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f7afc050ec0 con 0x7f7b1415b160 2023-11-06T02:09:35.930 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.932+0000 7f7b1a8a5640 1 -- 172.21.15.49:0/2434241660 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f7b140c9080 con 0x7f7b1415b160 2023-11-06T02:09:35.939 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.940+0000 7f7b117fa640 1 -- 172.21.15.49:0/2434241660 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) v1 ==== 97+0+0 (secure 0 0 0) 0x7f7afc0544e0 con 0x7f7b1415b160 2023-11-06T02:09:35.942 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.944+0000 7f7b1a8a5640 1 -- 172.21.15.49:0/2434241660 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7af4065a60 msgr2=0x7f7af4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:35.942 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.944+0000 7f7b1a8a5640 1 --2- 172.21.15.49:0/2434241660 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7af4065a60 0x7f7af4067f20 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7b040011e0 tx=0x7f7b04009040 comp rx=0 tx=0).stop 2023-11-06T02:09:35.942 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.944+0000 7f7b1a8a5640 1 -- 172.21.15.49:0/2434241660 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b1415b160 msgr2=0x7f7b1415aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:35.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.944+0000 7f7b1a8a5640 1 --2- 172.21.15.49:0/2434241660 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b1415b160 0x7f7b1415aac0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f7afc011590 tx=0x7f7afc011a50 comp rx=0 tx=0).stop 2023-11-06T02:09:35.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.944+0000 7f7b1a8a5640 1 -- 172.21.15.49:0/2434241660 shutdown_connections 2023-11-06T02:09:35.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.944+0000 7f7b1a8a5640 1 --2- 172.21.15.49:0/2434241660 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7af4065a60 0x7f7af4067f20 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:35.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.944+0000 7f7b1a8a5640 1 --2- 172.21.15.49:0/2434241660 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b1415b160 0x7f7b1415aac0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:35.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.944+0000 7f7b1a8a5640 1 --2- 172.21.15.49:0/2434241660 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7b14071cd0 0x7f7b14160080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:35.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.944+0000 7f7b1a8a5640 1 --2- 172.21.15.49:0/2434241660 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b00005190 0x7f7b1406ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:35.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.944+0000 7f7b1a8a5640 1 -- 172.21.15.49:0/2434241660 >> 172.21.15.49:0/2434241660 conn(0x7f7b1406c0a0 msgr2=0x7f7b14162710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:35.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.944+0000 7f7b1a8a5640 1 -- 172.21.15.49:0/2434241660 shutdown_connections 2023-11-06T02:09:35.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:35.944+0000 7f7b1a8a5640 1 -- 172.21.15.49:0/2434241660 wait complete. 2023-11-06T02:09:35.959 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-11-06T02:09:36.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.084+0000 7f71f0de3640 1 Processor -- start 2023-11-06T02:09:36.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.084+0000 7f71f0de3640 1 -- start start 2023-11-06T02:09:36.086 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.084+0000 7f71f0de3640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71ec0ca980 0x7f71ec1644c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:36.086 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.084+0000 7f71f0de3640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71ec0cadd0 con 0x7f71ec159b40 2023-11-06T02:09:36.087 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.084+0000 7f71f0de3640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71ec0723a0 con 0x7f71ec0cb010 2023-11-06T02:09:36.087 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.084+0000 7f71f0de3640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71ec0715e0 con 0x7f71ec0ca980 2023-11-06T02:09:36.087 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.084+0000 7f71e9d9b640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f71ec159b40 0x7f71ec0cdd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:51468/0 (socket says 172.21.15.49:51468) 2023-11-06T02:09:36.087 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.084+0000 7f71e9d9b640 1 -- 172.21.15.49:0/1194995650 learned_addr learned my addr 172.21.15.49:0/1194995650 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:09:36.087 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.084+0000 7f71ead9d640 1 --2- 172.21.15.49:0/1194995650 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71ec0ca980 0x7f71ec1644c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:36.088 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.084+0000 7f71e959a640 1 -- 172.21.15.49:0/1194995650 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4076858923 0 0) 0x7f71e0001120 con 0x7f71ec159b40 2023-11-06T02:09:36.088 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.084+0000 7f71e959a640 1 -- 172.21.15.49:0/1194995650 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71d4003530 con 0x7f71ec159b40 2023-11-06T02:09:36.088 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.084+0000 7f71e959a640 1 -- 172.21.15.49:0/1194995650 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3794852465 0 0) 0x7f71d4003530 con 0x7f71ec159b40 2023-11-06T02:09:36.088 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.084+0000 7f71e959a640 1 -- 172.21.15.49:0/1194995650 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71ec0ca980 msgr2=0x7f71ec1644c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:36.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.084+0000 7f71e959a640 1 --2- 172.21.15.49:0/1194995650 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71ec0ca980 0x7f71ec1644c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:36.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.084+0000 7f71e959a640 1 -- 172.21.15.49:0/1194995650 >> v1:172.21.15.150:6789/0 conn(0x7f71ec0cb010 legacy=0x7f71ec0cb3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:36.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.084+0000 7f71e959a640 1 -- 172.21.15.49:0/1194995650 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71ec0718c0 con 0x7f71ec159b40 2023-11-06T02:09:36.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.084+0000 7f71e959a640 1 -- 172.21.15.49:0/1194995650 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7f71e0001810 con 0x7f71ec159b40 2023-11-06T02:09:36.090 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.088+0000 7f71e959a640 1 -- 172.21.15.49:0/1194995650 >> v1:172.21.15.49:6789/0 conn(0x7f71ec159b40 legacy=0x7f71ec0cdd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:36.090 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.088+0000 7f71e959a640 1 --2- 172.21.15.49:0/1194995650 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71d40039b0 0x7f71d4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:36.090 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.088+0000 7f71e959a640 1 --2- 172.21.15.49:0/1194995650 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71d40047d0 0x7f71d4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:36.090 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.088+0000 7f71e959a640 1 --2- 172.21.15.49:0/1194995650 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71ec0ca980 0x7f71d4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:36.091 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.088+0000 7f71e959a640 1 -- 172.21.15.49:0/1194995650 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71ec0715e0 con 0x7f71d40047d0 2023-11-06T02:09:36.091 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.088+0000 7f71e959a640 1 -- 172.21.15.49:0/1194995650 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71d400ba40 con 0x7f71d40039b0 2023-11-06T02:09:36.091 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.088+0000 7f71e959a640 1 -- 172.21.15.49:0/1194995650 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71d400de10 con 0x7f71ec0ca980 2023-11-06T02:09:36.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.088+0000 7f71e9d9b640 1 --2- 172.21.15.49:0/1194995650 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71d40047d0 0x7f71d4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:36.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.088+0000 7f71ea59c640 1 --2- 172.21.15.49:0/1194995650 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71d40039b0 0x7f71d4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:36.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.088+0000 7f71ead9d640 1 --2- 172.21.15.49:0/1194995650 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71ec0ca980 0x7f71d4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:36.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.088+0000 7f71e9d9b640 1 -- 172.21.15.49:0/1194995650 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71ec0ca980 msgr2=0x7f71d4005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:36.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.088+0000 7f71e9d9b640 1 --2- 172.21.15.49:0/1194995650 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71ec0ca980 0x7f71d4005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:36.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.088+0000 7f71e9d9b640 1 -- 172.21.15.49:0/1194995650 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71d40039b0 msgr2=0x7f71d4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:36.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.088+0000 7f71e9d9b640 1 --2- 172.21.15.49:0/1194995650 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71d40039b0 0x7f71d4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:36.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.088+0000 7f71e9d9b640 1 -- 172.21.15.49:0/1194995650 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71d4003530 con 0x7f71d40047d0 2023-11-06T02:09:36.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.088+0000 7f71ead9d640 1 --2- 172.21.15.49:0/1194995650 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71ec0ca980 0x7f71d4005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:09:36.097 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.088+0000 7f71e9d9b640 1 --2- 172.21.15.49:0/1194995650 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71d40047d0 0x7f71d4004c50 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f71ec159f50 tx=0x7f71e001f330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5ab8b4a2bdfd884 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:36.097 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.088+0000 7f71e959a640 1 -- 172.21.15.49:0/1194995650 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e0005ae0 con 0x7f71d40047d0 2023-11-06T02:09:36.097 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.088+0000 7f71e959a640 1 -- 172.21.15.49:0/1194995650 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71e00ab070 con 0x7f71d40047d0 2023-11-06T02:09:36.097 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.088+0000 7f71e959a640 1 -- 172.21.15.49:0/1194995650 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e0005ea0 con 0x7f71d40047d0 2023-11-06T02:09:36.098 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.088+0000 7f71f0de3640 1 -- 172.21.15.49:0/1194995650 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71d40047d0 msgr2=0x7f71d4004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:36.098 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.088+0000 7f71f0de3640 1 --2- 172.21.15.49:0/1194995650 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71d40047d0 0x7f71d4004c50 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f71ec159f50 tx=0x7f71e001f330 comp rx=0 tx=0).stop 2023-11-06T02:09:36.098 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.088+0000 7f71ead9d640 1 -- 172.21.15.49:0/1194995650 reap_dead start 2023-11-06T02:09:36.098 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.088+0000 7f71f0de3640 1 -- 172.21.15.49:0/1194995650 shutdown_connections 2023-11-06T02:09:36.099 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.088+0000 7f71f0de3640 1 -- 172.21.15.49:0/1194995650 >> 172.21.15.49:0/1194995650 conn(0x7f71ec06c0a0 msgr2=0x7f71ec06c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:36.099 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.088+0000 7f71f0de3640 1 -- 172.21.15.49:0/1194995650 shutdown_connections 2023-11-06T02:09:36.099 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.088+0000 7f71f0de3640 1 -- 172.21.15.49:0/1194995650 wait complete. 2023-11-06T02:09:36.099 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.092+0000 7f71f0de3640 1 Processor -- start 2023-11-06T02:09:36.099 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.092+0000 7f71f0de3640 1 -- start start 2023-11-06T02:09:36.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.092+0000 7f71f0de3640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71d40047d0 0x7f71ec0ace60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:36.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.092+0000 7f71f0de3640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71ec0c57f0 0x7f71ec152b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:36.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.092+0000 7f71f0de3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71ec0ab8c0 0x7f71ec0c7e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:36.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.092+0000 7f71f0de3640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71ec0ac9d0 con 0x7f71ec0ab8c0 2023-11-06T02:09:36.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.092+0000 7f71f0de3640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71ec0abdc0 con 0x7f71d40047d0 2023-11-06T02:09:36.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.092+0000 7f71f0de3640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71ec0ac0a0 con 0x7f71ec0c57f0 2023-11-06T02:09:36.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.092+0000 7f71ead9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71ec0ab8c0 0x7f71ec0c7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:36.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.092+0000 7f71ead9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71ec0ab8c0 0x7f71ec0c7e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40748/0 (socket says 172.21.15.49:40748) 2023-11-06T02:09:36.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.092+0000 7f71ead9d640 1 -- 172.21.15.49:0/3877997547 learned_addr learned my addr 172.21.15.49:0/3877997547 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:36.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.092+0000 7f71ea59c640 1 --2- 172.21.15.49:0/3877997547 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71d40047d0 0x7f71ec0ace60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:36.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.092+0000 7f71e9d9b640 1 --2- 172.21.15.49:0/3877997547 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71ec0c57f0 0x7f71ec152b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:36.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.092+0000 7f71ead9d640 1 -- 172.21.15.49:0/3877997547 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71ec0c57f0 msgr2=0x7f71ec152b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:36.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.092+0000 7f71ead9d640 1 --2- 172.21.15.49:0/3877997547 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71ec0c57f0 0x7f71ec152b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:36.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.092+0000 7f71ead9d640 1 -- 172.21.15.49:0/3877997547 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71d40047d0 msgr2=0x7f71ec0ace60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:36.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.092+0000 7f71ead9d640 1 --2- 172.21.15.49:0/3877997547 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71d40047d0 0x7f71ec0ace60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:36.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.092+0000 7f71ead9d640 1 -- 172.21.15.49:0/3877997547 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71ec0ac380 con 0x7f71ec0ab8c0 2023-11-06T02:09:36.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.092+0000 7f71ead9d640 1 --2- 172.21.15.49:0/3877997547 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71ec0ab8c0 0x7f71ec0c7e70 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f71ec06add0 tx=0x7f71e4010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:36.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.092+0000 7f71daffd640 1 -- 172.21.15.49:0/3877997547 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e400e8a0 con 0x7f71ec0ab8c0 2023-11-06T02:09:36.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.092+0000 7f71daffd640 1 -- 172.21.15.49:0/3877997547 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71e400ea00 con 0x7f71ec0ab8c0 2023-11-06T02:09:36.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.092+0000 7f71f0de3640 1 -- 172.21.15.49:0/3877997547 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71ec0ac660 con 0x7f71ec0ab8c0 2023-11-06T02:09:36.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.092+0000 7f71daffd640 1 -- 172.21.15.49:0/3877997547 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e400ed90 con 0x7f71ec0ab8c0 2023-11-06T02:09:36.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.092+0000 7f71f0de3640 1 -- 172.21.15.49:0/3877997547 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71ec1562b0 con 0x7f71ec0ab8c0 2023-11-06T02:09:36.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.092+0000 7f71f0de3640 1 -- 172.21.15.49:0/3877997547 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71ec0acb40 con 0x7f71ec0ab8c0 2023-11-06T02:09:36.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.096+0000 7f71daffd640 1 -- 172.21.15.49:0/3877997547 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f71e4020410 con 0x7f71ec0ab8c0 2023-11-06T02:09:36.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.096+0000 7f71daffd640 1 --2- 172.21.15.49:0/3877997547 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f71cc065a90 0x7f71cc067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:36.105 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.096+0000 7f71daffd640 1 -- 172.21.15.49:0/3877997547 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f71e4082970 con 0x7f71ec0ab8c0 2023-11-06T02:09:36.105 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.096+0000 7f71ea59c640 1 --2- 172.21.15.49:0/3877997547 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f71cc065a90 0x7f71cc067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:36.105 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.096+0000 7f71ea59c640 1 --2- 172.21.15.49:0/3877997547 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f71cc065a90 0x7f71cc067f50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f71dc003fd0 tx=0x7f71dc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:36.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.100+0000 7f71daffd640 1 -- 172.21.15.49:0/3877997547 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f71e4050520 con 0x7f71ec0ab8c0 2023-11-06T02:09:36.303 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.304+0000 7f71f0de3640 1 -- 172.21.15.49:0/3877997547 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f71ec074040 con 0x7f71ec0ab8c0 2023-11-06T02:09:36.311 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.312+0000 7f71daffd640 1 -- 172.21.15.49:0/3877997547 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7f71e4053b40 con 0x7f71ec0ab8c0 2023-11-06T02:09:36.313 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.312+0000 7f71f0de3640 1 -- 172.21.15.49:0/3877997547 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f71cc065a90 msgr2=0x7f71cc067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:36.314 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.312+0000 7f71f0de3640 1 --2- 172.21.15.49:0/3877997547 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f71cc065a90 0x7f71cc067f50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f71dc003fd0 tx=0x7f71dc009040 comp rx=0 tx=0).stop 2023-11-06T02:09:36.314 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.312+0000 7f71f0de3640 1 -- 172.21.15.49:0/3877997547 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71ec0ab8c0 msgr2=0x7f71ec0c7e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:36.314 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.312+0000 7f71f0de3640 1 --2- 172.21.15.49:0/3877997547 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71ec0ab8c0 0x7f71ec0c7e70 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f71ec06add0 tx=0x7f71e4010c20 comp rx=0 tx=0).stop 2023-11-06T02:09:36.314 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.316+0000 7f71f0de3640 1 -- 172.21.15.49:0/3877997547 shutdown_connections 2023-11-06T02:09:36.315 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.316+0000 7f71f0de3640 1 --2- 172.21.15.49:0/3877997547 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f71cc065a90 0x7f71cc067f50 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:36.315 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.316+0000 7f71f0de3640 1 --2- 172.21.15.49:0/3877997547 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71ec0ab8c0 0x7f71ec0c7e70 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:36.315 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.316+0000 7f71f0de3640 1 --2- 172.21.15.49:0/3877997547 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71ec0c57f0 0x7f71ec152b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:36.315 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.316+0000 7f71f0de3640 1 --2- 172.21.15.49:0/3877997547 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71d40047d0 0x7f71ec0ace60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:36.315 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.316+0000 7f71f0de3640 1 -- 172.21.15.49:0/3877997547 >> 172.21.15.49:0/3877997547 conn(0x7f71ec06c0a0 msgr2=0x7f71ec0cdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:36.316 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.316+0000 7f71f0de3640 1 -- 172.21.15.49:0/3877997547 shutdown_connections 2023-11-06T02:09:36.316 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.316+0000 7f71f0de3640 1 -- 172.21.15.49:0/3877997547 wait complete. 2023-11-06T02:09:36.328 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T02:09:36.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.448+0000 7f83a6781640 1 Processor -- start 2023-11-06T02:09:36.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.448+0000 7f83a6781640 1 -- start start 2023-11-06T02:09:36.452 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.452+0000 7f83a6781640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83a00cb010 0x7f83a00ca980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:36.452 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.452+0000 7f83a6781640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83a00723a0 con 0x7f83a0159b40 2023-11-06T02:09:36.453 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.452+0000 7f83a6781640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83a00715e0 con 0x7f83a00cdd50 2023-11-06T02:09:36.453 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.452+0000 7f83a6781640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83a00718c0 con 0x7f83a00cb010 2023-11-06T02:09:36.453 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.452+0000 7f83977fe640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f83a0159b40 0x7f83a00cb3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:51484/0 (socket says 172.21.15.49:51484) 2023-11-06T02:09:36.453 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.452+0000 7f83977fe640 1 -- 172.21.15.49:0/2463999523 learned_addr learned my addr 172.21.15.49:0/2463999523 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:09:36.453 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.452+0000 7f839ffff640 1 --2- 172.21.15.49:0/2463999523 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83a00cb010 0x7f83a00ca980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:36.454 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.452+0000 7f839f7fe640 1 -- 172.21.15.49:0/2463999523 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 341837706 0 0) 0x7f83a00723a0 con 0x7f83a0159b40 2023-11-06T02:09:36.454 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.452+0000 7f839f7fe640 1 -- 172.21.15.49:0/2463999523 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f838c003530 con 0x7f83a0159b40 2023-11-06T02:09:36.454 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.452+0000 7f839f7fe640 1 -- 172.21.15.49:0/2463999523 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3022366505 0 0) 0x7f838c003530 con 0x7f83a0159b40 2023-11-06T02:09:36.454 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.452+0000 7f839f7fe640 1 -- 172.21.15.49:0/2463999523 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83a00cb010 msgr2=0x7f83a00ca980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:36.454 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.452+0000 7f839f7fe640 1 --2- 172.21.15.49:0/2463999523 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83a00cb010 0x7f83a00ca980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:36.455 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.452+0000 7f839f7fe640 1 -- 172.21.15.49:0/2463999523 >> v1:172.21.15.150:6789/0 conn(0x7f83a00cdd50 legacy=0x7f83a00cc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:36.455 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.452+0000 7f839f7fe640 1 -- 172.21.15.49:0/2463999523 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83a0071ba0 con 0x7f83a0159b40 2023-11-06T02:09:36.455 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.452+0000 7f839f7fe640 1 -- 172.21.15.49:0/2463999523 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7f8388001680 con 0x7f83a0159b40 2023-11-06T02:09:36.455 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.452+0000 7f839f7fe640 1 -- 172.21.15.49:0/2463999523 >> v1:172.21.15.49:6789/0 conn(0x7f83a0159b40 legacy=0x7f83a00cb3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:36.455 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.452+0000 7f839f7fe640 1 --2- 172.21.15.49:0/2463999523 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f838c0039b0 0x7f838c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:36.455 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.452+0000 7f839f7fe640 1 --2- 172.21.15.49:0/2463999523 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f838c0043a0 0x7f838c004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:36.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.452+0000 7f839f7fe640 1 --2- 172.21.15.49:0/2463999523 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f838c005070 0x7f838c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:36.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.452+0000 7f839f7fe640 1 -- 172.21.15.49:0/2463999523 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83a00718c0 con 0x7f838c005070 2023-11-06T02:09:36.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.452+0000 7f839f7fe640 1 -- 172.21.15.49:0/2463999523 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f838c00bf80 con 0x7f838c0043a0 2023-11-06T02:09:36.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.452+0000 7f839f7fe640 1 -- 172.21.15.49:0/2463999523 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f838c00c140 con 0x7f838c0039b0 2023-11-06T02:09:36.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.452+0000 7f839ffff640 1 --2- 172.21.15.49:0/2463999523 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83a00cb010 0x7f83a00ca980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T02:09:36.457 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.452+0000 7f83a4d1e640 1 --2- 172.21.15.49:0/2463999523 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f838c005070 0x7f838c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:36.457 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.452+0000 7f83977fe640 1 --2- 172.21.15.49:0/2463999523 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f838c0043a0 0x7f838c004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:36.457 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.452+0000 7f83a4d1e640 1 -- 172.21.15.49:0/2463999523 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f838c0039b0 msgr2=0x7f838c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:36.457 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.452+0000 7f83a4d1e640 1 --2- 172.21.15.49:0/2463999523 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f838c0039b0 0x7f838c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:36.457 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.452+0000 7f83a4d1e640 1 -- 172.21.15.49:0/2463999523 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f838c0043a0 msgr2=0x7f838c004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:36.458 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.452+0000 7f83a4d1e640 1 --2- 172.21.15.49:0/2463999523 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f838c0043a0 0x7f838c004870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:36.458 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.452+0000 7f83a4d1e640 1 -- 172.21.15.49:0/2463999523 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f838c003530 con 0x7f838c005070 2023-11-06T02:09:36.461 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.456+0000 7f83a4d1e640 1 --2- 172.21.15.49:0/2463999523 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f838c005070 0x7f838c00b990 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f8398001a50 tx=0x7f839801e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6029c5b22c3ee656 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:36.461 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.456+0000 7f839f7fe640 1 -- 172.21.15.49:0/2463999523 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8398003b80 con 0x7f838c005070 2023-11-06T02:09:36.462 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.456+0000 7f839f7fe640 1 -- 172.21.15.49:0/2463999523 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8398003ce0 con 0x7f838c005070 2023-11-06T02:09:36.462 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.456+0000 7f839f7fe640 1 -- 172.21.15.49:0/2463999523 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8398005310 con 0x7f838c005070 2023-11-06T02:09:36.462 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.456+0000 7f83a6781640 1 -- 172.21.15.49:0/2463999523 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f838c005070 msgr2=0x7f838c00b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:36.462 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.456+0000 7f83a6781640 1 --2- 172.21.15.49:0/2463999523 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f838c005070 0x7f838c00b990 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f8398001a50 tx=0x7f839801e990 comp rx=0 tx=0).stop 2023-11-06T02:09:36.462 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.456+0000 7f83a4d1e640 1 -- 172.21.15.49:0/2463999523 reap_dead start 2023-11-06T02:09:36.463 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.456+0000 7f83a6781640 1 -- 172.21.15.49:0/2463999523 shutdown_connections 2023-11-06T02:09:36.463 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.456+0000 7f83a6781640 1 -- 172.21.15.49:0/2463999523 >> 172.21.15.49:0/2463999523 conn(0x7f83a006c0a0 msgr2=0x7f83a006c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:36.463 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.456+0000 7f83a6781640 1 -- 172.21.15.49:0/2463999523 shutdown_connections 2023-11-06T02:09:36.463 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.456+0000 7f83a6781640 1 -- 172.21.15.49:0/2463999523 wait complete. 2023-11-06T02:09:36.463 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.456+0000 7f83a6781640 1 Processor -- start 2023-11-06T02:09:36.464 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.456+0000 7f83a6781640 1 -- start start 2023-11-06T02:09:36.464 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.456+0000 7f83a6781640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f838c005070 0x7f83a014ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:36.464 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.456+0000 7f83a6781640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83a00c3b90 0x7f83a00c4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:36.464 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.456+0000 7f83a6781640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f83a0159ff0 0x7f83a015a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:36.464 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.456+0000 7f83a6781640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83a00c4550 con 0x7f838c005070 2023-11-06T02:09:36.465 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.456+0000 7f83a6781640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83a014efc0 con 0x7f83a0159ff0 2023-11-06T02:09:36.465 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.456+0000 7f83a6781640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83a015ad60 con 0x7f83a00c3b90 2023-11-06T02:09:36.465 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.456+0000 7f839ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f838c005070 0x7f83a014ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:36.465 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.456+0000 7f839ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f838c005070 0x7f83a014ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40764/0 (socket says 172.21.15.49:40764) 2023-11-06T02:09:36.465 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.456+0000 7f839ffff640 1 -- 172.21.15.49:0/3558400368 learned_addr learned my addr 172.21.15.49:0/3558400368 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:36.466 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.456+0000 7f83977fe640 1 --2- 172.21.15.49:0/3558400368 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83a00c3b90 0x7f83a00c4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:36.466 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.456+0000 7f83a4d1e640 1 --2- 172.21.15.49:0/3558400368 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f83a0159ff0 0x7f83a015a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:36.466 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.456+0000 7f839ffff640 1 -- 172.21.15.49:0/3558400368 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83a00c3b90 msgr2=0x7f83a00c4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:36.466 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.456+0000 7f839ffff640 1 --2- 172.21.15.49:0/3558400368 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83a00c3b90 0x7f83a00c4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:36.466 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.456+0000 7f839ffff640 1 -- 172.21.15.49:0/3558400368 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f83a0159ff0 msgr2=0x7f83a015a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:36.467 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.456+0000 7f839ffff640 1 --2- 172.21.15.49:0/3558400368 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f83a0159ff0 0x7f83a015a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:36.467 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.456+0000 7f839ffff640 1 -- 172.21.15.49:0/3558400368 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83a015b040 con 0x7f838c005070 2023-11-06T02:09:36.467 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.456+0000 7f83a4d1e640 1 --2- 172.21.15.49:0/3558400368 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f83a0159ff0 0x7f83a015a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:09:36.467 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.456+0000 7f839ffff640 1 --2- 172.21.15.49:0/3558400368 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f838c005070 0x7f83a014ea80 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f83900047a0 tx=0x7f839000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:36.467 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.460+0000 7f839d7fa640 1 -- 172.21.15.49:0/3558400368 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f839000eec0 con 0x7f838c005070 2023-11-06T02:09:36.468 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.460+0000 7f839d7fa640 1 -- 172.21.15.49:0/3558400368 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f839000c8c0 con 0x7f838c005070 2023-11-06T02:09:36.468 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.460+0000 7f83a6781640 1 -- 172.21.15.49:0/3558400368 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83a015b320 con 0x7f838c005070 2023-11-06T02:09:36.468 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.460+0000 7f839d7fa640 1 -- 172.21.15.49:0/3558400368 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f839000cc50 con 0x7f838c005070 2023-11-06T02:09:36.468 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.460+0000 7f83a6781640 1 -- 172.21.15.49:0/3558400368 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83a015b8e0 con 0x7f838c005070 2023-11-06T02:09:36.468 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.460+0000 7f83a6781640 1 -- 172.21.15.49:0/3558400368 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83a015baa0 con 0x7f838c005070 2023-11-06T02:09:36.469 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.460+0000 7f839d7fa640 1 -- 172.21.15.49:0/3558400368 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f8390013470 con 0x7f838c005070 2023-11-06T02:09:36.469 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.460+0000 7f839d7fa640 1 --2- 172.21.15.49:0/3558400368 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8380065a60 0x7f8380067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:36.469 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.460+0000 7f839d7fa640 1 -- 172.21.15.49:0/3558400368 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f8390083410 con 0x7f838c005070 2023-11-06T02:09:36.469 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.460+0000 7f83977fe640 1 --2- 172.21.15.49:0/3558400368 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8380065a60 0x7f8380067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:36.469 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.460+0000 7f83977fe640 1 --2- 172.21.15.49:0/3558400368 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8380065a60 0x7f8380067f20 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8388000c10 tx=0x7f8388009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:36.470 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.464+0000 7f839d7fa640 1 -- 172.21.15.49:0/3558400368 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f8390054a50 con 0x7f838c005070 2023-11-06T02:09:36.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.620+0000 7f83a6781640 1 -- 172.21.15.49:0/3558400368 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f83a00c1a80 con 0x7f838c005070 2023-11-06T02:09:36.622 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.624+0000 7f839d7fa640 1 -- 172.21.15.49:0/3558400368 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+14171 (secure 0 0 0) 0x7f83900545e0 con 0x7f838c005070 2023-11-06T02:09:36.622 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:09:36.623 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":33,"fsid":"1a8b035c-6ba4-499a-b876-aea47fd9c0d4","created":"2023-11-06T02:08:06.440857+0000","modified":"2023-11-06T02:09:33.630470+0000","last_up_change":"2023-11-06T02:08:17.982372+0000","last_in_change":"2023-11-06T02:08:08.906631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T02:08:19.252222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T02:08:24.776211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ad55af8-0571-4e6a-8722-b5d429b7496d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6809","nonce":38116},{"type":"v1","addr":"172.21.15.49:6811","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6813","nonce":38116},{"type":"v1","addr":"172.21.15.49:6815","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6821","nonce":38116},{"type":"v1","addr":"172.21.15.49:6823","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6817","nonce":38116},{"type":"v1","addr":"172.21.15.49:6819","nonce":38116}]},"public_addr":"172.21.15.49:6811/38116","cluster_addr":"172.21.15.49:6815/38116","heartbeat_back_addr":"172.21.15.49:6823/38116","heartbeat_front_addr":"172.21.15.49:6819/38116","state":["exists","up"]},{"osd":1,"uuid":"c8b31f85-a757-43ed-9419-351acc46611b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":38113},{"type":"v1","addr":"172.21.15.49:6810","nonce":38113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":38113},{"type":"v1","addr":"172.21.15.49:6814","nonce":38113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":38113},{"type":"v1","addr":"172.21.15.49:6822","nonce":38113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":38113},{"type":"v1","addr":"172.21.15.49:6818","nonce":38113}]},"public_addr":"172.21.15.49:6810/38113","cluster_addr":"172.21.15.49:6814/38113","heartbeat_back_addr":"172.21.15.49:6822/38113","heartbeat_front_addr":"172.21.15.49:6818/38113","state":["exists","up"]},{"osd":2,"uuid":"555b3bf3-0a70-4b91-8efd-e4204b5fc26e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":38114},{"type":"v1","addr":"172.21.15.49:6801","nonce":38114}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":38114},{"type":"v1","addr":"172.21.15.49:6803","nonce":38114}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":38114},{"type":"v1","addr":"172.21.15.49:6807","nonce":38114}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":38114},{"type":"v1","addr":"172.21.15.49:6805","nonce":38114}]},"public_addr":"172.21.15.49:6801/38114","cluster_addr":"172.21.15.49:6803/38114","heartbeat_back_addr":"172.21.15.49:6807/38114","heartbeat_front_addr":"172.21.15.49:6805/38114","state":["exists","up"]},{"osd":3,"uuid":"8d4e5d77-03ef-4b9b-bfcb-5ae93a46795c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":38115},{"type":"v1","addr":"172.21.15.49:6825","nonce":38115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":38115},{"type":"v1","addr":"172.21.15.49:6827","nonce":38115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":38115},{"type":"v1","addr":"172.21.15.49:6831","nonce":38115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":38115},{"type":"v1","addr":"172.21.15.49:6829","nonce":38115}]},"public_addr":"172.21.15.49:6825/38115","cluster_addr":"172.21.15.49:6827/38115","heartbeat_back_addr":"172.21.15.49:6831/38115","heartbeat_front_addr":"172.21.15.49:6829/38115","state":["exists","up"]},{"osd":4,"uuid":"dba03ad4-68a3-4cf3-9218-ac611f02eacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":38417},{"type":"v1","addr":"172.21.15.150:6809","nonce":38417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":38417},{"type":"v1","addr":"172.21.15.150:6811","nonce":38417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":38417},{"type":"v1","addr":"172.21.15.150:6815","nonce":38417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":38417},{"type":"v1","addr":"172.21.15.150:6813","nonce":38417}]},"public_addr":"172.21.15.150:6809/38417","cluster_addr":"172.21.15.150:6811/38417","heartbeat_back_addr":"172.21.15.150:6815/38417","heartbeat_front_addr":"172.21.15.150:6813/38417","state":["exists","up"]},{"osd":5,"uuid":"0df9c303-c255-4640-8e91-01f1c3efe18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":38418},{"type":"v1","addr":"172.21.15.150:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":38418},{"type":"v1","addr":"172.21.15.150:6803","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":38418},{"type":"v1","addr":"172.21.15.150:6807","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":38418},{"type":"v1","addr":"172.21.15.150:6805","nonce":38418}]},"public_addr":"172.21.15.150:6801/38418","cluster_addr":"172.21.15.150:6803/38418","heartbeat_back_addr":"172.21.15.150:6807/38418","heartbeat_front_addr":"172.21.15.150:6805/38418","state":["exists","up"]},{"osd":6,"uuid":"e59fa189-4b70-4c06-9722-c02b90828bf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":38428},{"type":"v1","addr":"172.21.15.150:6825","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":38428},{"type":"v1","addr":"172.21.15.150:6827","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":38428},{"type":"v1","addr":"172.21.15.150:6831","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":38428},{"type":"v1","addr":"172.21.15.150:6829","nonce":38428}]},"public_addr":"172.21.15.150:6825/38428","cluster_addr":"172.21.15.150:6827/38428","heartbeat_back_addr":"172.21.15.150:6831/38428","heartbeat_front_addr":"172.21.15.150:6829/38428","state":["exists","up"]},{"osd":7,"uuid":"a64f57e1-ef98-46bd-b304-385759e95898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":38421},{"type":"v1","addr":"172.21.15.150:6817","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":38421},{"type":"v1","addr":"172.21.15.150:6819","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":38421},{"type":"v1","addr":"172.21.15.150:6823","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":38421},{"type":"v1","addr":"172.21.15.150:6821","nonce":38421}]},"public_addr":"172.21.15.150:6817/38421","cluster_addr":"172.21.15.150:6819/38421","heartbeat_back_addr":"172.21.15.150:6823/38421","heartbeat_front_addr":"172.21.15.150:6821/38421","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.438552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.413280+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.296466+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.424966+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.276953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.230984+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.410235+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.480084+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T02:09:36.624 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.624+0000 7f83a6781640 1 -- 172.21.15.49:0/3558400368 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8380065a60 msgr2=0x7f8380067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:36.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.624+0000 7f83a6781640 1 --2- 172.21.15.49:0/3558400368 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8380065a60 0x7f8380067f20 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8388000c10 tx=0x7f8388009040 comp rx=0 tx=0).stop 2023-11-06T02:09:36.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.624+0000 7f83a6781640 1 -- 172.21.15.49:0/3558400368 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f838c005070 msgr2=0x7f83a014ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:36.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.624+0000 7f83a6781640 1 --2- 172.21.15.49:0/3558400368 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f838c005070 0x7f83a014ea80 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f83900047a0 tx=0x7f839000ec70 comp rx=0 tx=0).stop 2023-11-06T02:09:36.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.624+0000 7f83a6781640 1 -- 172.21.15.49:0/3558400368 shutdown_connections 2023-11-06T02:09:36.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.624+0000 7f83a6781640 1 --2- 172.21.15.49:0/3558400368 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8380065a60 0x7f8380067f20 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:36.626 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.624+0000 7f83a6781640 1 --2- 172.21.15.49:0/3558400368 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f83a0159ff0 0x7f83a015a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:36.626 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.624+0000 7f83a6781640 1 --2- 172.21.15.49:0/3558400368 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83a00c3b90 0x7f83a00c4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:36.626 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.624+0000 7f83a6781640 1 --2- 172.21.15.49:0/3558400368 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f838c005070 0x7f83a014ea80 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:36.626 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.624+0000 7f83a6781640 1 -- 172.21.15.49:0/3558400368 >> 172.21.15.49:0/3558400368 conn(0x7f83a006c0a0 msgr2=0x7f83a00af1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:36.626 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.624+0000 7f83a6781640 1 -- 172.21.15.49:0/3558400368 shutdown_connections 2023-11-06T02:09:36.627 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.624+0000 7f83a6781640 1 -- 172.21.15.49:0/3558400368 wait complete. 2023-11-06T02:09:36.641 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T02:08:19.252222+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T02:08:24.776211+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T02:09:36.642 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T02:09:36.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.768+0000 7f381c6f2640 1 Processor -- start 2023-11-06T02:09:36.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.768+0000 7f381c6f2640 1 -- start start 2023-11-06T02:09:36.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.768+0000 7f381c6f2640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f381415a880 0x7f38140b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:36.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.768+0000 7f381c6f2640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f381406d860 0x7f3814070610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:36.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.768+0000 7f381c6f2640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38140ae360 con 0x7f3814070160 2023-11-06T02:09:36.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.768+0000 7f381c6f2640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3814070b50 con 0x7f381406d860 2023-11-06T02:09:36.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.768+0000 7f381c6f2640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38140ae640 con 0x7f381415a880 2023-11-06T02:09:36.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.768+0000 7f381ac8f640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f381406d860 0x7f3814070610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:36.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.768+0000 7f381ac8f640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f381406d860 0x7f3814070610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40776/0 (socket says 172.21.15.49:40776) 2023-11-06T02:09:36.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.768+0000 7f381ac8f640 1 -- 172.21.15.49:0/2453510983 learned_addr learned my addr 172.21.15.49:0/2453510983 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:36.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.768+0000 7f381a48e640 1 --2- 172.21.15.49:0/2453510983 >> v2:172.21.15.150:3300/0 conn(0x7f381415a880 0x7f38140b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:36.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.768+0000 7f381ac8f640 1 -- 172.21.15.49:0/2453510983 >> v2:172.21.15.150:3300/0 conn(0x7f381415a880 msgr2=0x7f38140b15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:36.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.768+0000 7f381ac8f640 1 --2- 172.21.15.49:0/2453510983 >> v2:172.21.15.150:3300/0 conn(0x7f381415a880 0x7f38140b15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:36.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.768+0000 7f381ac8f640 1 -- 172.21.15.49:0/2453510983 >> v1:172.21.15.150:6789/0 conn(0x7f3814070160 legacy=0x7f381406d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:36.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.768+0000 7f381ac8f640 1 -- 172.21.15.49:0/2453510983 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38140ae920 con 0x7f381406d860 2023-11-06T02:09:36.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.772+0000 7f381ac8f640 1 --2- 172.21.15.49:0/2453510983 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f381406d860 0x7f3814070610 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f38140cafc0 tx=0x7f380c01c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a0e200c427e12ae server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:36.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.772+0000 7f381948c640 1 -- 172.21.15.49:0/2453510983 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f380c0a5760 con 0x7f381406d860 2023-11-06T02:09:36.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.772+0000 7f381948c640 1 -- 172.21.15.49:0/2453510983 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f380c0a58c0 con 0x7f381406d860 2023-11-06T02:09:36.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.772+0000 7f381948c640 1 -- 172.21.15.49:0/2453510983 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f380c0a5c50 con 0x7f381406d860 2023-11-06T02:09:36.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.772+0000 7f381c6f2640 1 -- 172.21.15.49:0/2453510983 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f381406d860 msgr2=0x7f3814070610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:36.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.772+0000 7f381c6f2640 1 --2- 172.21.15.49:0/2453510983 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f381406d860 0x7f3814070610 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f38140cafc0 tx=0x7f380c01c990 comp rx=0 tx=0).stop 2023-11-06T02:09:36.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.772+0000 7f381c6f2640 1 -- 172.21.15.49:0/2453510983 shutdown_connections 2023-11-06T02:09:36.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.772+0000 7f381c6f2640 1 --2- 172.21.15.49:0/2453510983 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f381406d860 0x7f3814070610 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:36.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.772+0000 7f381c6f2640 1 --2- 172.21.15.49:0/2453510983 >> v2:172.21.15.150:3300/0 conn(0x7f381415a880 0x7f38140b15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:36.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.772+0000 7f381c6f2640 1 -- 172.21.15.49:0/2453510983 >> 172.21.15.49:0/2453510983 conn(0x7f381406a370 msgr2=0x7f38140ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:36.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.772+0000 7f381c6f2640 1 -- 172.21.15.49:0/2453510983 shutdown_connections 2023-11-06T02:09:36.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.772+0000 7f381c6f2640 1 -- 172.21.15.49:0/2453510983 wait complete. 2023-11-06T02:09:36.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.772+0000 7f381c6f2640 1 Processor -- start 2023-11-06T02:09:36.776 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.772+0000 7f381c6f2640 1 -- start start 2023-11-06T02:09:36.778 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.776+0000 7f381c6f2640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f381406d860 0x7f38140ca110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:36.778 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.776+0000 7f381c6f2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3814070160 0x7f3814158fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:36.778 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.776+0000 7f381c6f2640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f381415a880 0x7f3814150cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:36.778 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.776+0000 7f381c6f2640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38141f5da0 con 0x7f3814070160 2023-11-06T02:09:36.779 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.776+0000 7f381c6f2640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38140ca650 con 0x7f381406d860 2023-11-06T02:09:36.779 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.776+0000 7f381c6f2640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38141f5190 con 0x7f381415a880 2023-11-06T02:09:36.779 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.776+0000 7f3819c8d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3814070160 0x7f3814158fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:36.779 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.776+0000 7f3819c8d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3814070160 0x7f3814158fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40788/0 (socket says 172.21.15.49:40788) 2023-11-06T02:09:36.779 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.776+0000 7f3819c8d640 1 -- 172.21.15.49:0/2663509332 learned_addr learned my addr 172.21.15.49:0/2663509332 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:36.780 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.776+0000 7f381a48e640 1 --2- 172.21.15.49:0/2663509332 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f381406d860 0x7f38140ca110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:36.780 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.776+0000 7f381ac8f640 1 --2- 172.21.15.49:0/2663509332 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f381415a880 0x7f3814150cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:36.780 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.776+0000 7f3819c8d640 1 -- 172.21.15.49:0/2663509332 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f381415a880 msgr2=0x7f3814150cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:36.780 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.776+0000 7f3819c8d640 1 --2- 172.21.15.49:0/2663509332 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f381415a880 0x7f3814150cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:36.780 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.776+0000 7f3819c8d640 1 -- 172.21.15.49:0/2663509332 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f381406d860 msgr2=0x7f38140ca110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:36.781 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.776+0000 7f3819c8d640 1 --2- 172.21.15.49:0/2663509332 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f381406d860 0x7f38140ca110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:36.781 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.776+0000 7f3819c8d640 1 -- 172.21.15.49:0/2663509332 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f380c009b30 con 0x7f3814070160 2023-11-06T02:09:36.781 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.776+0000 7f3819c8d640 1 --2- 172.21.15.49:0/2663509332 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3814070160 0x7f3814158fe0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f380400c450 tx=0x7f380400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:36.781 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.776+0000 7f380affd640 1 -- 172.21.15.49:0/2663509332 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3804012830 con 0x7f3814070160 2023-11-06T02:09:36.781 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.776+0000 7f380affd640 1 -- 172.21.15.49:0/2663509332 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38040129b0 con 0x7f3814070160 2023-11-06T02:09:36.782 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.776+0000 7f381c6f2640 1 -- 172.21.15.49:0/2663509332 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38141f5470 con 0x7f3814070160 2023-11-06T02:09:36.782 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.776+0000 7f380affd640 1 -- 172.21.15.49:0/2663509332 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3804012d60 con 0x7f3814070160 2023-11-06T02:09:36.782 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.776+0000 7f381c6f2640 1 -- 172.21.15.49:0/2663509332 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38141f5a30 con 0x7f3814070160 2023-11-06T02:09:36.782 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.776+0000 7f381c6f2640 1 -- 172.21.15.49:0/2663509332 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3814074040 con 0x7f3814070160 2023-11-06T02:09:36.782 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.776+0000 7f380affd640 1 -- 172.21.15.49:0/2663509332 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f380401c460 con 0x7f3814070160 2023-11-06T02:09:36.783 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.776+0000 7f380affd640 1 --2- 172.21.15.49:0/2663509332 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f37f8065a60 0x7f37f8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:36.783 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.776+0000 7f380affd640 1 -- 172.21.15.49:0/2663509332 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f3804082d80 con 0x7f3814070160 2023-11-06T02:09:36.783 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.776+0000 7f381a48e640 1 --2- 172.21.15.49:0/2663509332 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f37f8065a60 0x7f37f8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:36.783 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.780+0000 7f381a48e640 1 --2- 172.21.15.49:0/2663509332 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f37f8065a60 0x7f37f8067f20 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f381000b930 tx=0x7f3810008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:36.784 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.780+0000 7f380affd640 1 -- 172.21.15.49:0/2663509332 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f38040509c0 con 0x7f3814070160 2023-11-06T02:09:36.937 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.936+0000 7f381c6f2640 1 -- 172.21.15.49:0/2663509332 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f381415bee0 con 0x7f3814070160 2023-11-06T02:09:36.938 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.940+0000 7f380affd640 1 -- 172.21.15.49:0/2663509332 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v33) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3804053fe0 con 0x7f3814070160 2023-11-06T02:09:36.938 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2023-11-06T02:09:36.940 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.940+0000 7f381c6f2640 1 -- 172.21.15.49:0/2663509332 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f37f8065a60 msgr2=0x7f37f8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:36.940 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.940+0000 7f381c6f2640 1 --2- 172.21.15.49:0/2663509332 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f37f8065a60 0x7f37f8067f20 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f381000b930 tx=0x7f3810008040 comp rx=0 tx=0).stop 2023-11-06T02:09:36.941 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.940+0000 7f381c6f2640 1 -- 172.21.15.49:0/2663509332 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3814070160 msgr2=0x7f3814158fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:36.941 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.940+0000 7f381c6f2640 1 --2- 172.21.15.49:0/2663509332 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3814070160 0x7f3814158fe0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f380400c450 tx=0x7f380400c910 comp rx=0 tx=0).stop 2023-11-06T02:09:36.941 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.940+0000 7f381c6f2640 1 -- 172.21.15.49:0/2663509332 shutdown_connections 2023-11-06T02:09:36.942 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.940+0000 7f381c6f2640 1 --2- 172.21.15.49:0/2663509332 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f37f8065a60 0x7f37f8067f20 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:36.942 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.940+0000 7f381c6f2640 1 --2- 172.21.15.49:0/2663509332 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3814070160 0x7f3814158fe0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:36.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.940+0000 7f381c6f2640 1 --2- 172.21.15.49:0/2663509332 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f381415a880 0x7f3814150cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:36.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.940+0000 7f381c6f2640 1 --2- 172.21.15.49:0/2663509332 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f381406d860 0x7f38140ca110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:36.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.940+0000 7f381c6f2640 1 -- 172.21.15.49:0/2663509332 >> 172.21.15.49:0/2663509332 conn(0x7f381406a370 msgr2=0x7f38141520b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:36.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.940+0000 7f381c6f2640 1 -- 172.21.15.49:0/2663509332 shutdown_connections 2023-11-06T02:09:36.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:36.940+0000 7f381c6f2640 1 -- 172.21.15.49:0/2663509332 wait complete. 2023-11-06T02:09:36.957 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T02:09:37.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.084+0000 7f2da0120640 1 Processor -- start 2023-11-06T02:09:37.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.084+0000 7f2da0120640 1 -- start start 2023-11-06T02:09:37.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.084+0000 7f2da0120640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f2d98070160 0x7f2d980705d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:37.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.084+0000 7f2da0120640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d980c8c80 con 0x7f2d9814ffc0 2023-11-06T02:09:37.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.084+0000 7f2da0120640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d98159700 con 0x7f2d9806c230 2023-11-06T02:09:37.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.084+0000 7f2da0120640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d980ae4d0 con 0x7f2d98070160 2023-11-06T02:09:37.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.084+0000 7f2d9d6bb640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f2d98070160 0x7f2d980705d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:37.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.084+0000 7f2d9d6bb640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f2d98070160 0x7f2d980705d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40798/0 (socket says 172.21.15.49:40798) 2023-11-06T02:09:37.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.084+0000 7f2d9d6bb640 1 -- 172.21.15.49:0/882940707 learned_addr learned my addr 172.21.15.49:0/882940707 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:37.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.084+0000 7f2d9debc640 1 --1- 172.21.15.49:0/882940707 >> v1:172.21.15.49:6789/0 conn(0x7f2d9814ffc0 0x7f2d980ada30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:51492/0 (socket says 172.21.15.49:51492) 2023-11-06T02:09:37.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.084+0000 7f2d9d6bb640 1 -- 172.21.15.49:0/882940707 >> v1:172.21.15.150:6789/0 conn(0x7f2d9806c230 legacy=0x7f2d9806c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:37.085 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.084+0000 7f2d9d6bb640 1 -- 172.21.15.49:0/882940707 >> v1:172.21.15.49:6789/0 conn(0x7f2d9814ffc0 legacy=0x7f2d980ada30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:37.085 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.084+0000 7f2d9d6bb640 1 -- 172.21.15.49:0/882940707 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d981599e0 con 0x7f2d98070160 2023-11-06T02:09:37.086 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.088+0000 7f2d9d6bb640 1 --2- 172.21.15.49:0/882940707 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d98070160 0x7f2d980705d0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f2d88006fd0 tx=0x7f2d8801e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db0d55c0a4bffdf9 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:37.086 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.088+0000 7f2d9ceba640 1 -- 172.21.15.49:0/882940707 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d88004230 con 0x7f2d98070160 2023-11-06T02:09:37.086 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.088+0000 7f2d9ceba640 1 -- 172.21.15.49:0/882940707 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d880ab070 con 0x7f2d98070160 2023-11-06T02:09:37.087 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.088+0000 7f2d9ceba640 1 -- 172.21.15.49:0/882940707 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d88003620 con 0x7f2d98070160 2023-11-06T02:09:37.087 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.088+0000 7f2da0120640 1 -- 172.21.15.49:0/882940707 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d98070160 msgr2=0x7f2d980705d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:37.087 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.088+0000 7f2da0120640 1 --2- 172.21.15.49:0/882940707 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d98070160 0x7f2d980705d0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f2d88006fd0 tx=0x7f2d8801e990 comp rx=0 tx=0).stop 2023-11-06T02:09:37.087 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.088+0000 7f2da0120640 1 -- 172.21.15.49:0/882940707 shutdown_connections 2023-11-06T02:09:37.087 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.088+0000 7f2da0120640 1 --2- 172.21.15.49:0/882940707 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d98070160 0x7f2d980705d0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:37.088 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.088+0000 7f2da0120640 1 -- 172.21.15.49:0/882940707 >> 172.21.15.49:0/882940707 conn(0x7f2d9806a370 msgr2=0x7f2d980ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:37.088 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.088+0000 7f2da0120640 1 -- 172.21.15.49:0/882940707 shutdown_connections 2023-11-06T02:09:37.088 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.088+0000 7f2da0120640 1 -- 172.21.15.49:0/882940707 wait complete. 2023-11-06T02:09:37.090 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.088+0000 7f2da0120640 1 Processor -- start 2023-11-06T02:09:37.090 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.088+0000 7f2da0120640 1 -- start start 2023-11-06T02:09:37.091 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.088+0000 7f2da0120640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d9806c230 0x7f2d981515a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:37.091 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.088+0000 7f2da0120640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d98070160 0x7f2d98153190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:37.091 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.088+0000 7f2da0120640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2d9814ffc0 0x7f2d98149870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:37.091 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.088+0000 7f2da0120640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d980b3080 con 0x7f2d98070160 2023-11-06T02:09:37.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.088+0000 7f2da0120640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d9813e900 con 0x7f2d9806c230 2023-11-06T02:09:37.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.088+0000 7f2da0120640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d980b26e0 con 0x7f2d9814ffc0 2023-11-06T02:09:37.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.088+0000 7f2d9d6bb640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d98070160 0x7f2d98153190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:37.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.088+0000 7f2d9d6bb640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d98070160 0x7f2d98153190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40808/0 (socket says 172.21.15.49:40808) 2023-11-06T02:09:37.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.088+0000 7f2d9d6bb640 1 -- 172.21.15.49:0/4038632454 learned_addr learned my addr 172.21.15.49:0/4038632454 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:37.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.088+0000 7f2d9debc640 1 --2- 172.21.15.49:0/4038632454 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d9806c230 0x7f2d981515a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:37.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.088+0000 7f2d9e6bd640 1 --2- 172.21.15.49:0/4038632454 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2d9814ffc0 0x7f2d98149870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:37.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.088+0000 7f2d9d6bb640 1 -- 172.21.15.49:0/4038632454 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2d9814ffc0 msgr2=0x7f2d98149870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:37.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.088+0000 7f2d9d6bb640 1 --2- 172.21.15.49:0/4038632454 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2d9814ffc0 0x7f2d98149870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:37.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.088+0000 7f2d9d6bb640 1 -- 172.21.15.49:0/4038632454 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d9806c230 msgr2=0x7f2d981515a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:37.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.088+0000 7f2d9d6bb640 1 --2- 172.21.15.49:0/4038632454 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d9806c230 0x7f2d981515a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:37.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.088+0000 7f2d9d6bb640 1 -- 172.21.15.49:0/4038632454 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d8800b780 con 0x7f2d98070160 2023-11-06T02:09:37.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.088+0000 7f2d9d6bb640 1 --2- 172.21.15.49:0/4038632454 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d98070160 0x7f2d98153190 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f2d88000c10 tx=0x7f2d8801b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:37.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.088+0000 7f2d8e7fc640 1 -- 172.21.15.49:0/4038632454 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d88003b20 con 0x7f2d98070160 2023-11-06T02:09:37.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.088+0000 7f2d8e7fc640 1 -- 172.21.15.49:0/4038632454 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d88003ca0 con 0x7f2d98070160 2023-11-06T02:09:37.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.088+0000 7f2da0120640 1 -- 172.21.15.49:0/4038632454 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d980b29c0 con 0x7f2d98070160 2023-11-06T02:09:37.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.088+0000 7f2d8e7fc640 1 -- 172.21.15.49:0/4038632454 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d880a7820 con 0x7f2d98070160 2023-11-06T02:09:37.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.088+0000 7f2da0120640 1 -- 172.21.15.49:0/4038632454 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d981536d0 con 0x7f2d98070160 2023-11-06T02:09:37.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.092+0000 7f2da0120640 1 -- 172.21.15.49:0/4038632454 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d980b31c0 con 0x7f2d98070160 2023-11-06T02:09:37.096 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.092+0000 7f2d8e7fc640 1 -- 172.21.15.49:0/4038632454 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f2d880a7980 con 0x7f2d98070160 2023-11-06T02:09:37.096 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.092+0000 7f2d8e7fc640 1 --2- 172.21.15.49:0/4038632454 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2d7c065a60 0x7f2d7c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:37.097 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.092+0000 7f2d8e7fc640 1 -- 172.21.15.49:0/4038632454 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f2d881203c0 con 0x7f2d98070160 2023-11-06T02:09:37.097 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.092+0000 7f2d9debc640 1 --2- 172.21.15.49:0/4038632454 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2d7c065a60 0x7f2d7c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:37.097 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.092+0000 7f2d9debc640 1 --2- 172.21.15.49:0/4038632454 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2d7c065a60 0x7f2d7c067f20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2d94000f70 tx=0x7f2d94009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:37.097 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.096+0000 7f2d8e7fc640 1 -- 172.21.15.49:0/4038632454 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f2d880edfe0 con 0x7f2d98070160 2023-11-06T02:09:37.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.252+0000 7f2da0120640 1 -- 172.21.15.49:0/4038632454 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f2d980b31c0 con 0x7f2d98070160 2023-11-06T02:09:37.253 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.256+0000 7f2d8e7fc640 1 -- 172.21.15.49:0/4038632454 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v33) v1 ==== 92+0+10 (secure 0 0 0) 0x7f2d980b31c0 con 0x7f2d98070160 2023-11-06T02:09:37.253 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 8 2023-11-06T02:09:37.255 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.256+0000 7f2da0120640 1 -- 172.21.15.49:0/4038632454 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2d7c065a60 msgr2=0x7f2d7c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:37.256 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.256+0000 7f2da0120640 1 --2- 172.21.15.49:0/4038632454 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2d7c065a60 0x7f2d7c067f20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2d94000f70 tx=0x7f2d94009040 comp rx=0 tx=0).stop 2023-11-06T02:09:37.256 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.256+0000 7f2da0120640 1 -- 172.21.15.49:0/4038632454 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d98070160 msgr2=0x7f2d98153190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:37.256 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.256+0000 7f2da0120640 1 --2- 172.21.15.49:0/4038632454 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d98070160 0x7f2d98153190 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f2d88000c10 tx=0x7f2d8801b730 comp rx=0 tx=0).stop 2023-11-06T02:09:37.256 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.256+0000 7f2da0120640 1 -- 172.21.15.49:0/4038632454 shutdown_connections 2023-11-06T02:09:37.256 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.256+0000 7f2da0120640 1 --2- 172.21.15.49:0/4038632454 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2d7c065a60 0x7f2d7c067f20 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:37.257 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.256+0000 7f2da0120640 1 --2- 172.21.15.49:0/4038632454 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2d98070160 0x7f2d98153190 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:37.257 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.256+0000 7f2da0120640 1 --2- 172.21.15.49:0/4038632454 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2d9814ffc0 0x7f2d98149870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:37.257 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.256+0000 7f2da0120640 1 --2- 172.21.15.49:0/4038632454 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d9806c230 0x7f2d981515a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:37.257 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.256+0000 7f2da0120640 1 -- 172.21.15.49:0/4038632454 >> 172.21.15.49:0/4038632454 conn(0x7f2d9806a370 msgr2=0x7f2d980ad820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:37.257 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.256+0000 7f2da0120640 1 -- 172.21.15.49:0/4038632454 shutdown_connections 2023-11-06T02:09:37.258 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.256+0000 7f2da0120640 1 -- 172.21.15.49:0/4038632454 wait complete. 2023-11-06T02:09:37.272 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-06T02:09:37.389 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.392+0000 7ff7ba756640 1 Processor -- start 2023-11-06T02:09:37.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.392+0000 7ff7ba756640 1 -- start start 2023-11-06T02:09:37.391 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.392+0000 7ff7ba756640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7ff7b41590f0 0x7ff7b40c8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:37.391 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.392+0000 7ff7ba756640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff7b40adae0 0x7ff7b40b0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:37.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.392+0000 7ff7ba756640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7b41594d0 con 0x7ff7b40b05c0 2023-11-06T02:09:37.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.392+0000 7ff7ba756640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7b415a140 con 0x7ff7b40adae0 2023-11-06T02:09:37.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.392+0000 7ff7ba756640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7b40c92e0 con 0x7ff7b41590f0 2023-11-06T02:09:37.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.392+0000 7ff7b37fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff7b40adae0 0x7ff7b40b0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:37.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.392+0000 7ff7b37fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff7b40adae0 0x7ff7b40b0080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40820/0 (socket says 172.21.15.49:40820) 2023-11-06T02:09:37.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.392+0000 7ff7b37fe640 1 -- 172.21.15.49:0/2375939637 learned_addr learned my addr 172.21.15.49:0/2375939637 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:37.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.392+0000 7ff7b3fff640 1 --2- 172.21.15.49:0/2375939637 >> v2:172.21.15.150:3300/0 conn(0x7ff7b41590f0 0x7ff7b40c8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:37.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.392+0000 7ff7b37fe640 1 -- 172.21.15.49:0/2375939637 >> v2:172.21.15.150:3300/0 conn(0x7ff7b41590f0 msgr2=0x7ff7b40c8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:37.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.392+0000 7ff7b37fe640 1 --2- 172.21.15.49:0/2375939637 >> v2:172.21.15.150:3300/0 conn(0x7ff7b41590f0 0x7ff7b40c8c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:37.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.392+0000 7ff7b37fe640 1 -- 172.21.15.49:0/2375939637 >> v1:172.21.15.49:6789/0 conn(0x7ff7b40b05c0 legacy=0x7ff7b415cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:37.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.392+0000 7ff7b2ffd640 1 -- 172.21.15.49:0/2375939637 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 532915811 0 0) 0x7ff7b41594d0 con 0x7ff7b40b05c0 2023-11-06T02:09:37.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.392+0000 7ff7b37fe640 1 -- 172.21.15.49:0/2375939637 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7b41597a0 con 0x7ff7b40adae0 2023-11-06T02:09:37.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.392+0000 7ff7b3fff640 1 --2- 172.21.15.49:0/2375939637 >> v2:172.21.15.150:3300/0 conn(0x7ff7b41590f0 0x7ff7b40c8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:09:37.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.396+0000 7ff7b37fe640 1 --2- 172.21.15.49:0/2375939637 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7b40adae0 0x7ff7b40b0080 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7ff7b406dde0 tx=0x7ff7a801c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a12688ebcd36c910 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:37.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.396+0000 7ff7b2ffd640 1 -- 172.21.15.49:0/2375939637 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a80ab600 con 0x7ff7b40adae0 2023-11-06T02:09:37.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.396+0000 7ff7b2ffd640 1 -- 172.21.15.49:0/2375939637 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7a80a7050 con 0x7ff7b40adae0 2023-11-06T02:09:37.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.396+0000 7ff7b2ffd640 1 -- 172.21.15.49:0/2375939637 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a80ab980 con 0x7ff7b40adae0 2023-11-06T02:09:37.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.396+0000 7ff7ba756640 1 -- 172.21.15.49:0/2375939637 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7b40adae0 msgr2=0x7ff7b40b0080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:37.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.396+0000 7ff7ba756640 1 --2- 172.21.15.49:0/2375939637 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7b40adae0 0x7ff7b40b0080 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7ff7b406dde0 tx=0x7ff7a801c470 comp rx=0 tx=0).stop 2023-11-06T02:09:37.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.396+0000 7ff7ba756640 1 -- 172.21.15.49:0/2375939637 shutdown_connections 2023-11-06T02:09:37.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.396+0000 7ff7ba756640 1 --2- 172.21.15.49:0/2375939637 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7b40adae0 0x7ff7b40b0080 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:37.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.396+0000 7ff7ba756640 1 --2- 172.21.15.49:0/2375939637 >> v2:172.21.15.150:3300/0 conn(0x7ff7b41590f0 0x7ff7b40c8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:37.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.396+0000 7ff7ba756640 1 -- 172.21.15.49:0/2375939637 >> 172.21.15.49:0/2375939637 conn(0x7ff7b406c0a0 msgr2=0x7ff7b406c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:37.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.396+0000 7ff7ba756640 1 -- 172.21.15.49:0/2375939637 shutdown_connections 2023-11-06T02:09:37.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.396+0000 7ff7ba756640 1 -- 172.21.15.49:0/2375939637 wait complete. 2023-11-06T02:09:37.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.396+0000 7ff7ba756640 1 Processor -- start 2023-11-06T02:09:37.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.396+0000 7ff7ba756640 1 -- start start 2023-11-06T02:09:37.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.396+0000 7ff7ba756640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff7b40adae0 0x7ff7b414c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:37.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.396+0000 7ff7ba756640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7b40b05c0 0x7ff7b40c6d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:37.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.396+0000 7ff7ba756640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7b41590f0 0x7ff7b40c7280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:37.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.396+0000 7ff7ba756640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b415ae00 con 0x7ff7b41590f0 2023-11-06T02:09:37.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.396+0000 7ff7ba756640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b4151320 con 0x7ff7b40b05c0 2023-11-06T02:09:37.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.396+0000 7ff7ba756640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b415aa60 con 0x7ff7b40adae0 2023-11-06T02:09:37.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.396+0000 7ff7b8cf3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7b41590f0 0x7ff7b40c7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:37.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.396+0000 7ff7b8cf3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7b41590f0 0x7ff7b40c7280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40826/0 (socket says 172.21.15.49:40826) 2023-11-06T02:09:37.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.396+0000 7ff7b8cf3640 1 -- 172.21.15.49:0/1855579252 learned_addr learned my addr 172.21.15.49:0/1855579252 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:37.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.396+0000 7ff7b37fe640 1 --2- 172.21.15.49:0/1855579252 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7b40b05c0 0x7ff7b40c6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:37.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.396+0000 7ff7b3fff640 1 --2- 172.21.15.49:0/1855579252 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff7b40adae0 0x7ff7b414c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:37.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.396+0000 7ff7b8cf3640 1 -- 172.21.15.49:0/1855579252 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff7b40adae0 msgr2=0x7ff7b414c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:37.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.396+0000 7ff7b8cf3640 1 --2- 172.21.15.49:0/1855579252 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff7b40adae0 0x7ff7b414c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:37.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.396+0000 7ff7b8cf3640 1 -- 172.21.15.49:0/1855579252 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7b40b05c0 msgr2=0x7ff7b40c6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:37.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.396+0000 7ff7b8cf3640 1 --2- 172.21.15.49:0/1855579252 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7b40b05c0 0x7ff7b40c6d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:37.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.396+0000 7ff7b8cf3640 1 -- 172.21.15.49:0/1855579252 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7a80094d0 con 0x7ff7b41590f0 2023-11-06T02:09:37.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.396+0000 7ff7b3fff640 1 --2- 172.21.15.49:0/1855579252 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff7b40adae0 0x7ff7b414c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:09:37.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.396+0000 7ff7b8cf3640 1 --2- 172.21.15.49:0/1855579252 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7b41590f0 0x7ff7b40c7280 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac002240 tx=0x7ff7ac00cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:37.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.400+0000 7ff7b0ff9640 1 -- 172.21.15.49:0/1855579252 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ac003db0 con 0x7ff7b41590f0 2023-11-06T02:09:37.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.400+0000 7ff7b0ff9640 1 -- 172.21.15.49:0/1855579252 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7ac00ce30 con 0x7ff7b41590f0 2023-11-06T02:09:37.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.400+0000 7ff7b0ff9640 1 -- 172.21.15.49:0/1855579252 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ac005270 con 0x7ff7b41590f0 2023-11-06T02:09:37.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.400+0000 7ff7ba756640 1 -- 172.21.15.49:0/1855579252 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7b4150d60 con 0x7ff7b41590f0 2023-11-06T02:09:37.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.400+0000 7ff7ba756640 1 -- 172.21.15.49:0/1855579252 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7b40b12a0 con 0x7ff7b41590f0 2023-11-06T02:09:37.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.400+0000 7ff7ba756640 1 -- 172.21.15.49:0/1855579252 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7b415af40 con 0x7ff7b41590f0 2023-11-06T02:09:37.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.400+0000 7ff7b0ff9640 1 -- 172.21.15.49:0/1855579252 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7ff7ac0053d0 con 0x7ff7b41590f0 2023-11-06T02:09:37.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.400+0000 7ff7b0ff9640 1 --2- 172.21.15.49:0/1855579252 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff7a0065a60 0x7ff7a0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:37.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.400+0000 7ff7b0ff9640 1 -- 172.21.15.49:0/1855579252 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5210+0+0 (secure 0 0 0) 0x7ff7ac082650 con 0x7ff7b41590f0 2023-11-06T02:09:37.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.400+0000 7ff7b3fff640 1 --2- 172.21.15.49:0/1855579252 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff7a0065a60 0x7ff7a0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:37.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.400+0000 7ff7b3fff640 1 --2- 172.21.15.49:0/1855579252 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff7a0065a60 0x7ff7a0067f20 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff7a4006fd0 tx=0x7ff7a4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:37.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.404+0000 7ff7b0ff9640 1 -- 172.21.15.49:0/1855579252 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7ff7b415af40 con 0x7ff7b41590f0 2023-11-06T02:09:37.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.560+0000 7ff7ba756640 1 -- 172.21.15.49:0/1855579252 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7ff7b415af40 con 0x7ff7b41590f0 2023-11-06T02:09:37.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.560+0000 7ff7b0ff9640 1 -- 172.21.15.49:0/1855579252 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v33) v1 ==== 84+0+126 (secure 0 0 0) 0x7ff7b415af40 con 0x7ff7b41590f0 2023-11-06T02:09:37.561 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:09:37.561 INFO:teuthology.orchestra.run.smithi049.stdout:[ 2023-11-06T02:09:37.562 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:09:37.562 INFO:teuthology.orchestra.run.smithi049.stdout: "poolnum": 1, 2023-11-06T02:09:37.562 INFO:teuthology.orchestra.run.smithi049.stdout: "poolname": ".mgr" 2023-11-06T02:09:37.562 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:09:37.563 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:09:37.563 INFO:teuthology.orchestra.run.smithi049.stdout: "poolnum": 2, 2023-11-06T02:09:37.563 INFO:teuthology.orchestra.run.smithi049.stdout: "poolname": "rbd" 2023-11-06T02:09:37.563 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:09:37.563 INFO:teuthology.orchestra.run.smithi049.stdout:] 2023-11-06T02:09:37.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.564+0000 7ff7ba756640 1 -- 172.21.15.49:0/1855579252 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff7a0065a60 msgr2=0x7ff7a0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:37.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.564+0000 7ff7ba756640 1 --2- 172.21.15.49:0/1855579252 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff7a0065a60 0x7ff7a0067f20 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff7a4006fd0 tx=0x7ff7a4008040 comp rx=0 tx=0).stop 2023-11-06T02:09:37.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.564+0000 7ff7ba756640 1 -- 172.21.15.49:0/1855579252 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7b41590f0 msgr2=0x7ff7b40c7280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:37.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.564+0000 7ff7ba756640 1 --2- 172.21.15.49:0/1855579252 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7b41590f0 0x7ff7b40c7280 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac002240 tx=0x7ff7ac00cbe0 comp rx=0 tx=0).stop 2023-11-06T02:09:37.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.564+0000 7ff7ba756640 1 -- 172.21.15.49:0/1855579252 shutdown_connections 2023-11-06T02:09:37.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.564+0000 7ff7ba756640 1 --2- 172.21.15.49:0/1855579252 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff7a0065a60 0x7ff7a0067f20 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:37.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.564+0000 7ff7ba756640 1 --2- 172.21.15.49:0/1855579252 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff7b41590f0 0x7ff7b40c7280 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:37.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.564+0000 7ff7ba756640 1 --2- 172.21.15.49:0/1855579252 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7b40b05c0 0x7ff7b40c6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:37.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.564+0000 7ff7ba756640 1 --2- 172.21.15.49:0/1855579252 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff7b40adae0 0x7ff7b414c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:37.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.564+0000 7ff7ba756640 1 -- 172.21.15.49:0/1855579252 >> 172.21.15.49:0/1855579252 conn(0x7ff7b406c0a0 msgr2=0x7ff7b40cae00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:37.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.564+0000 7ff7ba756640 1 -- 172.21.15.49:0/1855579252 shutdown_connections 2023-11-06T02:09:37.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.564+0000 7ff7ba756640 1 -- 172.21.15.49:0/1855579252 wait complete. 2023-11-06T02:09:37.580 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-06T02:09:37.581 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-06T02:09:37.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.696+0000 7f563424b640 1 Processor -- start 2023-11-06T02:09:37.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.696+0000 7f563424b640 1 -- start start 2023-11-06T02:09:37.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.700+0000 7f563424b640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f562c06a480 0x7f562c0c9940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:37.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.700+0000 7f563424b640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f562c0c94b0 con 0x7f562c15abd0 2023-11-06T02:09:37.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.700+0000 7f563424b640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f562c15ecd0 con 0x7f562c15e780 2023-11-06T02:09:37.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.700+0000 7f563424b640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f562c15efb0 con 0x7f562c06a480 2023-11-06T02:09:37.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.700+0000 7f56317e6640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f562c06a480 0x7f562c0c9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:37.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.700+0000 7f56317e6640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f562c06a480 0x7f562c0c9940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40838/0 (socket says 172.21.15.49:40838) 2023-11-06T02:09:37.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.700+0000 7f5631fe7640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f562c15abd0 0x7f562c069d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:51518/0 (socket says 172.21.15.49:51518) 2023-11-06T02:09:37.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.700+0000 7f56317e6640 1 -- 172.21.15.49:0/3045316989 learned_addr learned my addr 172.21.15.49:0/3045316989 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:37.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.700+0000 7f56317e6640 1 -- 172.21.15.49:0/3045316989 >> v1:172.21.15.150:6789/0 conn(0x7f562c15e780 legacy=0x7f562c0c9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:37.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.700+0000 7f56317e6640 1 -- 172.21.15.49:0/3045316989 >> v1:172.21.15.49:6789/0 conn(0x7f562c15abd0 legacy=0x7f562c069d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:37.701 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.700+0000 7f5630fe5640 1 -- 172.21.15.49:0/3045316989 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 176204965 0 0) 0x7f562c0c94b0 con 0x7f562c15abd0 2023-11-06T02:09:37.701 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.700+0000 7f56317e6640 1 -- 172.21.15.49:0/3045316989 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f562c15f290 con 0x7f562c06a480 2023-11-06T02:09:37.702 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.704+0000 7f56317e6640 1 --2- 172.21.15.49:0/3045316989 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f562c06a480 0x7f562c0c9940 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f5620009870 tx=0x7f562c0cada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7bc5b481f2b982e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:37.702 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.704+0000 7f5630fe5640 1 -- 172.21.15.49:0/3045316989 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56200ab660 con 0x7f562c06a480 2023-11-06T02:09:37.703 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.704+0000 7f5630fe5640 1 -- 172.21.15.49:0/3045316989 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56200ab7c0 con 0x7f562c06a480 2023-11-06T02:09:37.703 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.704+0000 7f5630fe5640 1 -- 172.21.15.49:0/3045316989 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56200abb50 con 0x7f562c06a480 2023-11-06T02:09:37.703 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.704+0000 7f563424b640 1 -- 172.21.15.49:0/3045316989 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f562c06a480 msgr2=0x7f562c0c9940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:37.703 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.704+0000 7f563424b640 1 --2- 172.21.15.49:0/3045316989 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f562c06a480 0x7f562c0c9940 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f5620009870 tx=0x7f562c0cada0 comp rx=0 tx=0).stop 2023-11-06T02:09:37.704 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.704+0000 7f563424b640 1 -- 172.21.15.49:0/3045316989 shutdown_connections 2023-11-06T02:09:37.704 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.704+0000 7f563424b640 1 --2- 172.21.15.49:0/3045316989 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f562c06a480 0x7f562c0c9940 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:37.704 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.704+0000 7f563424b640 1 -- 172.21.15.49:0/3045316989 >> 172.21.15.49:0/3045316989 conn(0x7f562c0ce610 msgr2=0x7f562c0cea20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:37.704 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.704+0000 7f563424b640 1 -- 172.21.15.49:0/3045316989 shutdown_connections 2023-11-06T02:09:37.704 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.704+0000 7f563424b640 1 -- 172.21.15.49:0/3045316989 wait complete. 2023-11-06T02:09:37.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.704+0000 7f563424b640 1 Processor -- start 2023-11-06T02:09:37.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.704+0000 7f563424b640 1 -- start start 2023-11-06T02:09:37.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.704+0000 7f563424b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f562c06a480 0x7f562c15c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:37.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.704+0000 7f563424b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f562c15abd0 0x7f562c0c4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:37.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.704+0000 7f563424b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f562c15e780 0x7f562c15d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:37.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.704+0000 7f563424b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f562c15ca90 con 0x7f562c06a480 2023-11-06T02:09:37.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.704+0000 7f563424b640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f562c0c3ac0 con 0x7f562c15e780 2023-11-06T02:09:37.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.704+0000 7f563424b640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f562c0c3da0 con 0x7f562c15abd0 2023-11-06T02:09:37.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.704+0000 7f5631fe7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f562c06a480 0x7f562c15c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:37.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.704+0000 7f5631fe7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f562c06a480 0x7f562c15c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40850/0 (socket says 172.21.15.49:40850) 2023-11-06T02:09:37.709 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.704+0000 7f5631fe7640 1 -- 172.21.15.49:0/3757236383 learned_addr learned my addr 172.21.15.49:0/3757236383 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:37.709 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.704+0000 7f56317e6640 1 --2- 172.21.15.49:0/3757236383 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f562c15abd0 0x7f562c0c4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:37.709 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.704+0000 7f56327e8640 1 --2- 172.21.15.49:0/3757236383 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f562c15e780 0x7f562c15d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:37.709 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.704+0000 7f5631fe7640 1 -- 172.21.15.49:0/3757236383 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f562c15abd0 msgr2=0x7f562c0c4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:37.709 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.704+0000 7f5631fe7640 1 --2- 172.21.15.49:0/3757236383 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f562c15abd0 0x7f562c0c4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:37.710 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.704+0000 7f5631fe7640 1 -- 172.21.15.49:0/3757236383 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f562c15e780 msgr2=0x7f562c15d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:37.710 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.704+0000 7f5631fe7640 1 --2- 172.21.15.49:0/3757236383 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f562c15e780 0x7f562c15d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:37.710 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.704+0000 7f5631fe7640 1 -- 172.21.15.49:0/3757236383 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56200094d0 con 0x7f562c06a480 2023-11-06T02:09:37.710 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.704+0000 7f56317e6640 1 --2- 172.21.15.49:0/3757236383 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f562c15abd0 0x7f562c0c4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:09:37.710 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.704+0000 7f5631fe7640 1 --2- 172.21.15.49:0/3757236383 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f562c06a480 0x7f562c15c550 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f5624001570 tx=0x7f562400cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:37.711 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.704+0000 7f562a7fc640 1 -- 172.21.15.49:0/3757236383 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f562400a640 con 0x7f562c06a480 2023-11-06T02:09:37.711 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.704+0000 7f562a7fc640 1 -- 172.21.15.49:0/3757236383 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f562400a7c0 con 0x7f562c06a480 2023-11-06T02:09:37.711 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.704+0000 7f563424b640 1 -- 172.21.15.49:0/3757236383 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f562c0c4080 con 0x7f562c06a480 2023-11-06T02:09:37.711 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.704+0000 7f562a7fc640 1 -- 172.21.15.49:0/3757236383 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f562400ab70 con 0x7f562c06a480 2023-11-06T02:09:37.711 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.704+0000 7f563424b640 1 -- 172.21.15.49:0/3757236383 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f562c1f3f20 con 0x7f562c06a480 2023-11-06T02:09:37.712 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.708+0000 7f563424b640 1 -- 172.21.15.49:0/3757236383 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f562c074040 con 0x7f562c06a480 2023-11-06T02:09:37.712 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.708+0000 7f562a7fc640 1 -- 172.21.15.49:0/3757236383 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f562400acd0 con 0x7f562c06a480 2023-11-06T02:09:37.712 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.708+0000 7f562a7fc640 1 --2- 172.21.15.49:0/3757236383 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5618065a90 0x7f5618067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:37.712 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.708+0000 7f562a7fc640 1 -- 172.21.15.49:0/3757236383 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f56240824d0 con 0x7f562c06a480 2023-11-06T02:09:37.713 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.708+0000 7f56317e6640 1 --2- 172.21.15.49:0/3757236383 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5618065a90 0x7f5618067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:37.713 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.708+0000 7f56317e6640 1 --2- 172.21.15.49:0/3757236383 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5618065a90 0x7f5618067f50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f56200af000 tx=0x7f5620008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:37.713 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.712+0000 7f562a7fc640 1 -- 172.21.15.49:0/3757236383 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f5624050070 con 0x7f562c06a480 2023-11-06T02:09:37.860 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:37.860+0000 7f563424b640 1 -- 172.21.15.49:0/3757236383 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f562c0c8d80 con 0x7f562c06a480 2023-11-06T02:09:38.468 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:38.468+0000 7f562a7fc640 1 -- 172.21.15.49:0/3757236383 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v34) v1 ==== 138+0+0 (secure 0 0 0) 0x7f5624053690 con 0x7f562c06a480 2023-11-06T02:09:38.468 INFO:teuthology.orchestra.run.smithi049.stderr:pool 'cephfs_metadata' created 2023-11-06T02:09:38.472 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:38.472+0000 7f563424b640 1 -- 172.21.15.49:0/3757236383 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5618065a90 msgr2=0x7f5618067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:38.473 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:38.472+0000 7f563424b640 1 --2- 172.21.15.49:0/3757236383 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5618065a90 0x7f5618067f50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f56200af000 tx=0x7f5620008040 comp rx=0 tx=0).stop 2023-11-06T02:09:38.473 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:38.472+0000 7f563424b640 1 -- 172.21.15.49:0/3757236383 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f562c06a480 msgr2=0x7f562c15c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:38.473 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:38.472+0000 7f563424b640 1 --2- 172.21.15.49:0/3757236383 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f562c06a480 0x7f562c15c550 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f5624001570 tx=0x7f562400cd20 comp rx=0 tx=0).stop 2023-11-06T02:09:38.473 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:38.472+0000 7f563424b640 1 -- 172.21.15.49:0/3757236383 shutdown_connections 2023-11-06T02:09:38.474 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:38.472+0000 7f563424b640 1 --2- 172.21.15.49:0/3757236383 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5618065a90 0x7f5618067f50 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:38.474 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:38.472+0000 7f563424b640 1 --2- 172.21.15.49:0/3757236383 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f562c15e780 0x7f562c15d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:38.474 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:38.472+0000 7f563424b640 1 --2- 172.21.15.49:0/3757236383 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f562c15abd0 0x7f562c0c4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:38.474 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:38.472+0000 7f563424b640 1 --2- 172.21.15.49:0/3757236383 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f562c06a480 0x7f562c15c550 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:38.474 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:38.472+0000 7f563424b640 1 -- 172.21.15.49:0/3757236383 >> 172.21.15.49:0/3757236383 conn(0x7f562c0ce610 msgr2=0x7f562c0afc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:38.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:38.472+0000 7f563424b640 1 -- 172.21.15.49:0/3757236383 shutdown_connections 2023-11-06T02:09:38.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:38.472+0000 7f563424b640 1 -- 172.21.15.49:0/3757236383 wait complete. 2023-11-06T02:09:38.496 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-06T02:09:38.633 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:38.636+0000 7fd9cefeb640 1 Processor -- start 2023-11-06T02:09:38.634 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:38.636+0000 7fd9cefeb640 1 -- start start 2023-11-06T02:09:38.635 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:38.636+0000 7fd9cefeb640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd9c814ffc0 0x7fd9c80ada30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:38.635 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:38.636+0000 7fd9cefeb640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fd9c80adf70 0x7fd9c8070160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:38.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:38.636+0000 7fd9cefeb640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9c80c8c80 con 0x7fd9c80706a0 2023-11-06T02:09:38.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:38.636+0000 7fd9cefeb640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9c80ae4d0 con 0x7fd9c814ffc0 2023-11-06T02:09:38.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:38.636+0000 7fd9cefeb640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9c81594f0 con 0x7fd9c80adf70 2023-11-06T02:09:38.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:38.636+0000 7fd9ccd87640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd9c814ffc0 0x7fd9c80ada30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:38.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:38.636+0000 7fd9ccd87640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd9c814ffc0 0x7fd9c80ada30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36416/0 (socket says 172.21.15.49:36416) 2023-11-06T02:09:38.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:38.636+0000 7fd9ccd87640 1 -- 172.21.15.49:0/3584645902 learned_addr learned my addr 172.21.15.49:0/3584645902 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:38.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:38.636+0000 7fd9c7fff640 1 --2- 172.21.15.49:0/3584645902 >> v2:172.21.15.150:3300/0 conn(0x7fd9c80adf70 0x7fd9c8070160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:38.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:38.636+0000 7fd9ccd87640 1 -- 172.21.15.49:0/3584645902 >> v2:172.21.15.150:3300/0 conn(0x7fd9c80adf70 msgr2=0x7fd9c8070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:38.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:38.636+0000 7fd9ccd87640 1 --2- 172.21.15.49:0/3584645902 >> v2:172.21.15.150:3300/0 conn(0x7fd9c80adf70 0x7fd9c8070160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:38.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:38.636+0000 7fd9ccd87640 1 -- 172.21.15.49:0/3584645902 >> v1:172.21.15.150:6789/0 conn(0x7fd9c80706a0 legacy=0x7fd9c815cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:38.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:38.636+0000 7fd9ccd87640 1 -- 172.21.15.49:0/3584645902 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9c8070af0 con 0x7fd9c814ffc0 2023-11-06T02:09:38.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:38.636+0000 7fd9c7fff640 1 --2- 172.21.15.49:0/3584645902 >> v2:172.21.15.150:3300/0 conn(0x7fd9c80adf70 0x7fd9c8070160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:09:38.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:38.640+0000 7fd9ccd87640 1 --2- 172.21.15.49:0/3584645902 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9c814ffc0 0x7fd9c80ada30 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fd9c80cb010 tx=0x7fd9b801c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=844506d80e76d5a8 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:38.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:38.640+0000 7fd9c77fe640 1 -- 172.21.15.49:0/3584645902 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b80a55b0 con 0x7fd9c814ffc0 2023-11-06T02:09:38.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:38.640+0000 7fd9c77fe640 1 -- 172.21.15.49:0/3584645902 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9b80a5710 con 0x7fd9c814ffc0 2023-11-06T02:09:38.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:38.640+0000 7fd9c77fe640 1 -- 172.21.15.49:0/3584645902 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b80a5aa0 con 0x7fd9c814ffc0 2023-11-06T02:09:38.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:38.640+0000 7fd9cefeb640 1 -- 172.21.15.49:0/3584645902 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9c814ffc0 msgr2=0x7fd9c80ada30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:38.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:38.640+0000 7fd9cefeb640 1 --2- 172.21.15.49:0/3584645902 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9c814ffc0 0x7fd9c80ada30 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fd9c80cb010 tx=0x7fd9b801c990 comp rx=0 tx=0).stop 2023-11-06T02:09:38.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:38.640+0000 7fd9cefeb640 1 -- 172.21.15.49:0/3584645902 shutdown_connections 2023-11-06T02:09:38.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:38.640+0000 7fd9cefeb640 1 --2- 172.21.15.49:0/3584645902 >> v2:172.21.15.150:3300/0 conn(0x7fd9c80adf70 0x7fd9c8070160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:38.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:38.640+0000 7fd9cefeb640 1 --2- 172.21.15.49:0/3584645902 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9c814ffc0 0x7fd9c80ada30 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:38.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:38.640+0000 7fd9cefeb640 1 -- 172.21.15.49:0/3584645902 >> 172.21.15.49:0/3584645902 conn(0x7fd9c806a370 msgr2=0x7fd9c80ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:38.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:38.640+0000 7fd9cefeb640 1 -- 172.21.15.49:0/3584645902 shutdown_connections 2023-11-06T02:09:38.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:38.640+0000 7fd9cefeb640 1 -- 172.21.15.49:0/3584645902 wait complete. 2023-11-06T02:09:38.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:38.640+0000 7fd9cefeb640 1 Processor -- start 2023-11-06T02:09:38.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:38.640+0000 7fd9cefeb640 1 -- start start 2023-11-06T02:09:38.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:38.640+0000 7fd9cefeb640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd9c80706a0 0x7fd9c8143300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:38.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:38.640+0000 7fd9cefeb640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9c80adf70 0x7fd9c81483e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:38.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:38.640+0000 7fd9cefeb640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9c814ffc0 0x7fd9c80b1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:38.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:38.640+0000 7fd9cefeb640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c8147f50 con 0x7fd9c80adf70 2023-11-06T02:09:38.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:38.640+0000 7fd9cefeb640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c8147060 con 0x7fd9c814ffc0 2023-11-06T02:09:38.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:38.640+0000 7fd9cefeb640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c8147340 con 0x7fd9c80706a0 2023-11-06T02:09:38.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:38.640+0000 7fd9c7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9c80adf70 0x7fd9c81483e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:38.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:38.640+0000 7fd9c7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9c80adf70 0x7fd9c81483e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36428/0 (socket says 172.21.15.49:36428) 2023-11-06T02:09:38.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:38.640+0000 7fd9c7fff640 1 -- 172.21.15.49:0/3106503140 learned_addr learned my addr 172.21.15.49:0/3106503140 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:38.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:38.640+0000 7fd9ccd87640 1 --2- 172.21.15.49:0/3106503140 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd9c80706a0 0x7fd9c8143300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:38.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:38.640+0000 7fd9cd588640 1 --2- 172.21.15.49:0/3106503140 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9c814ffc0 0x7fd9c80b1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:38.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:38.640+0000 7fd9c7fff640 1 -- 172.21.15.49:0/3106503140 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd9c80706a0 msgr2=0x7fd9c8143300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:38.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:38.640+0000 7fd9c7fff640 1 --2- 172.21.15.49:0/3106503140 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd9c80706a0 0x7fd9c8143300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:38.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:38.640+0000 7fd9c7fff640 1 -- 172.21.15.49:0/3106503140 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9c814ffc0 msgr2=0x7fd9c80b1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:38.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:38.640+0000 7fd9c7fff640 1 --2- 172.21.15.49:0/3106503140 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9c814ffc0 0x7fd9c80b1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:38.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:38.640+0000 7fd9c7fff640 1 -- 172.21.15.49:0/3106503140 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9b8009b30 con 0x7fd9c80adf70 2023-11-06T02:09:38.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:38.640+0000 7fd9cd588640 1 --2- 172.21.15.49:0/3106503140 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9c814ffc0 0x7fd9c80b1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:09:38.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:38.644+0000 7fd9c7fff640 1 --2- 172.21.15.49:0/3106503140 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9c80adf70 0x7fd9c81483e0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc00c7c0 tx=0x7fd9bc00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:38.648 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:38.644+0000 7fd9c57fa640 1 -- 172.21.15.49:0/3106503140 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9bc007a30 con 0x7fd9c80adf70 2023-11-06T02:09:38.648 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:38.644+0000 7fd9c57fa640 1 -- 172.21.15.49:0/3106503140 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9bc007b90 con 0x7fd9c80adf70 2023-11-06T02:09:38.648 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:38.644+0000 7fd9cefeb640 1 -- 172.21.15.49:0/3106503140 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9c8147620 con 0x7fd9c80adf70 2023-11-06T02:09:38.648 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:38.644+0000 7fd9c57fa640 1 -- 172.21.15.49:0/3106503140 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9bc00ce70 con 0x7fd9c80adf70 2023-11-06T02:09:38.648 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:38.644+0000 7fd9cefeb640 1 -- 172.21.15.49:0/3106503140 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9c8147be0 con 0x7fd9c80adf70 2023-11-06T02:09:38.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:38.644+0000 7fd9cefeb640 1 -- 172.21.15.49:0/3106503140 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9c8074040 con 0x7fd9c80adf70 2023-11-06T02:09:38.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:38.644+0000 7fd9c57fa640 1 -- 172.21.15.49:0/3106503140 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fd9bc0113f0 con 0x7fd9c80adf70 2023-11-06T02:09:38.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:38.644+0000 7fd9c57fa640 1 --2- 172.21.15.49:0/3106503140 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd9b4065a60 0x7fd9b4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:38.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:38.644+0000 7fd9c57fa640 1 -- 172.21.15.49:0/3106503140 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5580+0+0 (secure 0 0 0) 0x7fd9bc0835a0 con 0x7fd9c80adf70 2023-11-06T02:09:38.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:38.644+0000 7fd9ccd87640 1 --2- 172.21.15.49:0/3106503140 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd9b4065a60 0x7fd9b4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:38.650 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:38.644+0000 7fd9ccd87640 1 --2- 172.21.15.49:0/3106503140 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd9b4065a60 0x7fd9b4067f20 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd9b80ae000 tx=0x7fd9b80a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:38.650 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:38.648+0000 7fd9c57fa640 1 -- 172.21.15.49:0/3106503140 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fd9bc051070 con 0x7fd9c80adf70 2023-11-06T02:09:38.804 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:38.804+0000 7fd9cefeb640 1 -- 172.21.15.49:0/3106503140 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fd9c8158fa0 con 0x7fd9c80adf70 2023-11-06T02:09:39.485 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:39.484+0000 7fd9c57fa640 1 -- 172.21.15.49:0/3106503140 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v35) v1 ==== 170+0+0 (secure 0 0 0) 0x7fd9bc054690 con 0x7fd9c80adf70 2023-11-06T02:09:39.485 INFO:teuthology.orchestra.run.smithi049.stderr:pool 'cephfs_data' created 2023-11-06T02:09:39.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:39.492+0000 7fd9cefeb640 1 -- 172.21.15.49:0/3106503140 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd9b4065a60 msgr2=0x7fd9b4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:39.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:39.492+0000 7fd9cefeb640 1 --2- 172.21.15.49:0/3106503140 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd9b4065a60 0x7fd9b4067f20 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd9b80ae000 tx=0x7fd9b80a7070 comp rx=0 tx=0).stop 2023-11-06T02:09:39.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:39.492+0000 7fd9cefeb640 1 -- 172.21.15.49:0/3106503140 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9c80adf70 msgr2=0x7fd9c81483e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:39.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:39.492+0000 7fd9cefeb640 1 --2- 172.21.15.49:0/3106503140 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9c80adf70 0x7fd9c81483e0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc00c7c0 tx=0x7fd9bc00cc80 comp rx=0 tx=0).stop 2023-11-06T02:09:39.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:39.512+0000 7fd9cefeb640 1 -- 172.21.15.49:0/3106503140 shutdown_connections 2023-11-06T02:09:39.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:39.512+0000 7fd9cefeb640 1 --2- 172.21.15.49:0/3106503140 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd9b4065a60 0x7fd9b4067f20 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:39.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:39.512+0000 7fd9cefeb640 1 --2- 172.21.15.49:0/3106503140 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9c80adf70 0x7fd9c81483e0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:39.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:39.512+0000 7fd9cefeb640 1 --2- 172.21.15.49:0/3106503140 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9c814ffc0 0x7fd9c80b1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:39.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:39.512+0000 7fd9cefeb640 1 --2- 172.21.15.49:0/3106503140 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd9c80706a0 0x7fd9c8143300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:39.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:39.512+0000 7fd9cefeb640 1 -- 172.21.15.49:0/3106503140 >> 172.21.15.49:0/3106503140 conn(0x7fd9c806a370 msgr2=0x7fd9c80af920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:39.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:39.512+0000 7fd9cefeb640 1 -- 172.21.15.49:0/3106503140 shutdown_connections 2023-11-06T02:09:39.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:39.512+0000 7fd9cefeb640 1 -- 172.21.15.49:0/3106503140 wait complete. 2023-11-06T02:09:39.536 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-06T02:09:39.662 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:39.664+0000 7f5753c6d640 1 Processor -- start 2023-11-06T02:09:39.662 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:39.664+0000 7f5753c6d640 1 -- start start 2023-11-06T02:09:39.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:39.664+0000 7f5753c6d640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f574c0c9600 0x7f574c0cb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:39.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:39.664+0000 7f5753c6d640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f574c0cb5c0 0x7f574c0c99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:39.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:39.664+0000 7f5753c6d640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f574c065930 0x7f574c0c9f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:39.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:39.664+0000 7f5753c6d640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f574c06def0 con 0x7f574c065930 2023-11-06T02:09:39.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:39.664+0000 7f5753c6d640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f574c15b970 con 0x7f574c0c9600 2023-11-06T02:09:39.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:39.664+0000 7f5753c6d640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f574c06cee0 con 0x7f574c0cb5c0 2023-11-06T02:09:39.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:39.664+0000 7f575220a640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f574c065930 0x7f574c0c9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:39.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:39.664+0000 7f575220a640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f574c065930 0x7f574c0c9f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36442/0 (socket says 172.21.15.49:36442) 2023-11-06T02:09:39.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:39.664+0000 7f575220a640 1 -- 172.21.15.49:0/3632738810 learned_addr learned my addr 172.21.15.49:0/3632738810 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:39.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:39.664+0000 7f5751208640 1 --2- 172.21.15.49:0/3632738810 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f574c0cb5c0 0x7f574c0c99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:39.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:39.664+0000 7f5751a09640 1 --2- 172.21.15.49:0/3632738810 >> v2:172.21.15.150:3300/0 conn(0x7f574c0c9600 0x7f574c0cb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:39.667 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:39.664+0000 7f575220a640 1 -- 172.21.15.49:0/3632738810 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f574c0cb5c0 msgr2=0x7f574c0c99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:39.667 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:39.664+0000 7f575220a640 1 --2- 172.21.15.49:0/3632738810 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f574c0cb5c0 0x7f574c0c99e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:39.667 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:39.664+0000 7f575220a640 1 -- 172.21.15.49:0/3632738810 >> v2:172.21.15.150:3300/0 conn(0x7f574c0c9600 msgr2=0x7f574c0cb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:39.667 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:39.664+0000 7f575220a640 1 --2- 172.21.15.49:0/3632738810 >> v2:172.21.15.150:3300/0 conn(0x7f574c0c9600 0x7f574c0cb080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:39.667 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:39.664+0000 7f575220a640 1 -- 172.21.15.49:0/3632738810 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f574c0cabf0 con 0x7f574c065930 2023-11-06T02:09:39.670 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:39.668+0000 7f575220a640 1 --2- 172.21.15.49:0/3632738810 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f574c065930 0x7f574c0c9f20 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f574c0cee60 tx=0x7f574401c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eea1bebd3f5fa08b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:39.670 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:39.668+0000 7f5750a07640 1 -- 172.21.15.49:0/3632738810 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57440a5650 con 0x7f574c065930 2023-11-06T02:09:39.671 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:39.668+0000 7f5750a07640 1 -- 172.21.15.49:0/3632738810 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57440a57b0 con 0x7f574c065930 2023-11-06T02:09:39.671 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:39.668+0000 7f5750a07640 1 -- 172.21.15.49:0/3632738810 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57440a5b40 con 0x7f574c065930 2023-11-06T02:09:39.671 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:39.668+0000 7f5753c6d640 1 -- 172.21.15.49:0/3632738810 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f574c065930 msgr2=0x7f574c0c9f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:39.671 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:39.668+0000 7f5753c6d640 1 --2- 172.21.15.49:0/3632738810 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f574c065930 0x7f574c0c9f20 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f574c0cee60 tx=0x7f574401c9d0 comp rx=0 tx=0).stop 2023-11-06T02:09:39.671 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:39.668+0000 7f5753c6d640 1 -- 172.21.15.49:0/3632738810 shutdown_connections 2023-11-06T02:09:39.672 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:39.668+0000 7f5753c6d640 1 --2- 172.21.15.49:0/3632738810 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f574c0cb5c0 0x7f574c0c99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:39.672 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:39.668+0000 7f5753c6d640 1 --2- 172.21.15.49:0/3632738810 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f574c065930 0x7f574c0c9f20 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:39.672 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:39.668+0000 7f5753c6d640 1 --2- 172.21.15.49:0/3632738810 >> v2:172.21.15.150:3300/0 conn(0x7f574c0c9600 0x7f574c0cb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:39.672 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:39.668+0000 7f5753c6d640 1 -- 172.21.15.49:0/3632738810 >> 172.21.15.49:0/3632738810 conn(0x7f574c0b0aa0 msgr2=0x7f574c0b0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:39.672 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:39.668+0000 7f5753c6d640 1 -- 172.21.15.49:0/3632738810 shutdown_connections 2023-11-06T02:09:39.673 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:39.668+0000 7f5753c6d640 1 -- 172.21.15.49:0/3632738810 wait complete. 2023-11-06T02:09:39.673 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:39.668+0000 7f5753c6d640 1 Processor -- start 2023-11-06T02:09:39.673 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:39.668+0000 7f5753c6d640 1 -- start start 2023-11-06T02:09:39.673 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:39.672+0000 7f5753c6d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f574c065930 0x7f574c06ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:39.673 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:39.672+0000 7f5753c6d640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f574c0c9600 0x7f574c0ac8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:39.674 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:39.672+0000 7f5753c6d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f574c0cb5c0 0x7f574c0af180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:39.674 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:39.672+0000 7f5753c6d640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f574c0ace30 con 0x7f574c065930 2023-11-06T02:09:39.674 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:39.672+0000 7f5753c6d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f574c0c50c0 con 0x7f574c0cb5c0 2023-11-06T02:09:39.674 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:39.672+0000 7f5753c6d640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f574c0c53a0 con 0x7f574c0c9600 2023-11-06T02:09:39.674 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:39.672+0000 7f5751a09640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f574c065930 0x7f574c06ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:39.675 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:39.672+0000 7f5751a09640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f574c065930 0x7f574c06ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36456/0 (socket says 172.21.15.49:36456) 2023-11-06T02:09:39.675 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:39.672+0000 7f5751a09640 1 -- 172.21.15.49:0/4108539369 learned_addr learned my addr 172.21.15.49:0/4108539369 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:39.675 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:39.672+0000 7f5751208640 1 --2- 172.21.15.49:0/4108539369 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f574c0c9600 0x7f574c0ac8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:39.675 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:39.672+0000 7f5751a09640 1 -- 172.21.15.49:0/4108539369 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f574c0c9600 msgr2=0x7f574c0ac8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:39.675 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:39.672+0000 7f575220a640 1 --2- 172.21.15.49:0/4108539369 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f574c0cb5c0 0x7f574c0af180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:39.676 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:39.672+0000 7f5751a09640 1 --2- 172.21.15.49:0/4108539369 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f574c0c9600 0x7f574c0ac8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:39.676 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:39.672+0000 7f5751a09640 1 -- 172.21.15.49:0/4108539369 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f574c0cb5c0 msgr2=0x7f574c0af180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:39.676 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:39.672+0000 7f5751a09640 1 --2- 172.21.15.49:0/4108539369 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f574c0cb5c0 0x7f574c0af180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:39.676 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:39.672+0000 7f5751a09640 1 -- 172.21.15.49:0/4108539369 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5744009b30 con 0x7f574c065930 2023-11-06T02:09:39.677 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:39.672+0000 7f5751a09640 1 --2- 172.21.15.49:0/4108539369 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f574c065930 0x7f574c06ab40 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f574800d800 tx=0x7f574800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:39.677 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:39.672+0000 7f57427fc640 1 -- 172.21.15.49:0/4108539369 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57480096a0 con 0x7f574c065930 2023-11-06T02:09:39.677 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:39.672+0000 7f57427fc640 1 -- 172.21.15.49:0/4108539369 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5748009800 con 0x7f574c065930 2023-11-06T02:09:39.677 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:39.672+0000 7f57427fc640 1 -- 172.21.15.49:0/4108539369 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5748009990 con 0x7f574c065930 2023-11-06T02:09:39.677 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:39.672+0000 7f5753c6d640 1 -- 172.21.15.49:0/4108539369 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f574c0c5680 con 0x7f574c065930 2023-11-06T02:09:39.677 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:39.672+0000 7f5753c6d640 1 -- 172.21.15.49:0/4108539369 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f574c157220 con 0x7f574c065930 2023-11-06T02:09:39.679 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:39.672+0000 7f5753c6d640 1 -- 172.21.15.49:0/4108539369 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f574c0b4c00 con 0x7f574c065930 2023-11-06T02:09:39.679 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:39.672+0000 7f57427fc640 1 -- 172.21.15.49:0/4108539369 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f5748009af0 con 0x7f574c065930 2023-11-06T02:09:39.679 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:39.672+0000 7f57427fc640 1 --2- 172.21.15.49:0/4108539369 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5730065a60 0x7f5730067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:39.679 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:39.672+0000 7f57427fc640 1 -- 172.21.15.49:0/4108539369 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5962+0+0 (secure 0 0 0) 0x7f5748083680 con 0x7f574c065930 2023-11-06T02:09:39.679 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:39.672+0000 7f5751208640 1 --2- 172.21.15.49:0/4108539369 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5730065a60 0x7f5730067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:39.680 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:39.676+0000 7f5751208640 1 --2- 172.21.15.49:0/4108539369 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5730065a60 0x7f5730067f20 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f574c0c9180 tx=0x7f573c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:39.680 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:39.676+0000 7f57427fc640 1 -- 172.21.15.49:0/4108539369 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f5748050fd0 con 0x7f574c065930 2023-11-06T02:09:39.867 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:39.868+0000 7f5753c6d640 1 -- 172.21.15.49:0/4108539369 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f574c06d750 con 0x7f574c065930 2023-11-06T02:09:39.873 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:39.876+0000 7f57427fc640 1 -- 172.21.15.49:0/4108539369 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v11) v1 ==== 172+0+0 (secure 0 0 0) 0x7f574800b3a0 con 0x7f574c065930 2023-11-06T02:09:39.874 INFO:teuthology.orchestra.run.smithi049.stderr:new fs with metadata pool 5 and data pool 6 2023-11-06T02:09:39.881 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:39.880+0000 7f5753c6d640 1 -- 172.21.15.49:0/4108539369 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5730065a60 msgr2=0x7f5730067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:39.881 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:39.880+0000 7f5753c6d640 1 --2- 172.21.15.49:0/4108539369 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5730065a60 0x7f5730067f20 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f574c0c9180 tx=0x7f573c008040 comp rx=0 tx=0).stop 2023-11-06T02:09:39.881 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:39.880+0000 7f5753c6d640 1 -- 172.21.15.49:0/4108539369 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f574c065930 msgr2=0x7f574c06ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:39.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:39.880+0000 7f5753c6d640 1 --2- 172.21.15.49:0/4108539369 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f574c065930 0x7f574c06ab40 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f574800d800 tx=0x7f574800dcc0 comp rx=0 tx=0).stop 2023-11-06T02:09:39.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:39.884+0000 7f5753c6d640 1 -- 172.21.15.49:0/4108539369 shutdown_connections 2023-11-06T02:09:39.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:39.884+0000 7f5753c6d640 1 --2- 172.21.15.49:0/4108539369 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5730065a60 0x7f5730067f20 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:39.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:39.884+0000 7f5753c6d640 1 --2- 172.21.15.49:0/4108539369 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f574c0cb5c0 0x7f574c0af180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:39.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:39.884+0000 7f5753c6d640 1 --2- 172.21.15.49:0/4108539369 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f574c0c9600 0x7f574c0ac8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:39.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:39.884+0000 7f5753c6d640 1 --2- 172.21.15.49:0/4108539369 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f574c065930 0x7f574c06ab40 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:39.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:39.884+0000 7f5753c6d640 1 -- 172.21.15.49:0/4108539369 >> 172.21.15.49:0/4108539369 conn(0x7f574c0b0aa0 msgr2=0x7f574c09e200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:39.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:39.884+0000 7f5753c6d640 1 -- 172.21.15.49:0/4108539369 shutdown_connections 2023-11-06T02:09:39.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:39.884+0000 7f5753c6d640 1 -- 172.21.15.49:0/4108539369 wait complete. 2023-11-06T02:09:39.913 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T02:09:40.033 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.032+0000 7f0caa7a1640 1 Processor -- start 2023-11-06T02:09:40.033 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.032+0000 7f0caa7a1640 1 -- start start 2023-11-06T02:09:40.034 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.036+0000 7f0caa7a1640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f0ca415ceb0 0x7f0ca40cb970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:40.035 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.036+0000 7f0caa7a1640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f0ca40cb370 0x7f0ca4069d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:40.035 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.036+0000 7f0caa7a1640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ca40cbeb0 con 0x7f0ca40caf30 2023-11-06T02:09:40.035 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.036+0000 7f0caa7a1640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ca40caaa0 con 0x7f0ca40cb370 2023-11-06T02:09:40.035 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.036+0000 7f0caa7a1640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ca40ca1e0 con 0x7f0ca415ceb0 2023-11-06T02:09:40.035 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.036+0000 7f0ca8d3e640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f0ca40cb370 0x7f0ca4069d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:40.036 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.036+0000 7f0ca8d3e640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f0ca40cb370 0x7f0ca4069d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36470/0 (socket says 172.21.15.49:36470) 2023-11-06T02:09:40.036 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.036+0000 7f0ca8d3e640 1 -- 172.21.15.49:0/1581949878 learned_addr learned my addr 172.21.15.49:0/1581949878 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:40.036 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.036+0000 7f0ca3fff640 1 --2- 172.21.15.49:0/1581949878 >> v2:172.21.15.150:3300/0 conn(0x7f0ca415ceb0 0x7f0ca40cb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:40.036 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.036+0000 7f0ca8d3e640 1 -- 172.21.15.49:0/1581949878 >> v2:172.21.15.150:3300/0 conn(0x7f0ca415ceb0 msgr2=0x7f0ca40cb970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:40.036 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.036+0000 7f0ca8d3e640 1 --2- 172.21.15.49:0/1581949878 >> v2:172.21.15.150:3300/0 conn(0x7f0ca415ceb0 0x7f0ca40cb970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:40.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.036+0000 7f0ca8d3e640 1 -- 172.21.15.49:0/1581949878 >> v1:172.21.15.150:6789/0 conn(0x7f0ca40caf30 legacy=0x7f0ca4071120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:40.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.036+0000 7f0ca8d3e640 1 -- 172.21.15.49:0/1581949878 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ca40ce490 con 0x7f0ca40cb370 2023-11-06T02:09:40.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.036+0000 7f0ca3fff640 1 --2- 172.21.15.49:0/1581949878 >> v2:172.21.15.150:3300/0 conn(0x7f0ca415ceb0 0x7f0ca40cb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:09:40.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.036+0000 7f0ca8d3e640 1 --2- 172.21.15.49:0/1581949878 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ca40cb370 0x7f0ca4069d70 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c009e50 tx=0x7f0c9c01c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72603d6ecf434ca7 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:40.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.036+0000 7f0ca2ffd640 1 -- 172.21.15.49:0/1581949878 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c9c0a5740 con 0x7f0ca40cb370 2023-11-06T02:09:40.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.036+0000 7f0ca2ffd640 1 -- 172.21.15.49:0/1581949878 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c9c0a58a0 con 0x7f0ca40cb370 2023-11-06T02:09:40.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.036+0000 7f0ca2ffd640 1 -- 172.21.15.49:0/1581949878 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c9c0a5c30 con 0x7f0ca40cb370 2023-11-06T02:09:40.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.036+0000 7f0caa7a1640 1 -- 172.21.15.49:0/1581949878 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ca40cb370 msgr2=0x7f0ca4069d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:40.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.036+0000 7f0caa7a1640 1 --2- 172.21.15.49:0/1581949878 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ca40cb370 0x7f0ca4069d70 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c009e50 tx=0x7f0c9c01c990 comp rx=0 tx=0).stop 2023-11-06T02:09:40.042 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.040+0000 7f0caa7a1640 1 -- 172.21.15.49:0/1581949878 shutdown_connections 2023-11-06T02:09:40.042 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.040+0000 7f0caa7a1640 1 --2- 172.21.15.49:0/1581949878 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ca40cb370 0x7f0ca4069d70 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:40.042 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.040+0000 7f0caa7a1640 1 --2- 172.21.15.49:0/1581949878 >> v2:172.21.15.150:3300/0 conn(0x7f0ca415ceb0 0x7f0ca40cb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:40.042 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.040+0000 7f0caa7a1640 1 -- 172.21.15.49:0/1581949878 >> 172.21.15.49:0/1581949878 conn(0x7f0ca40cc140 msgr2=0x7f0ca40cc550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:40.043 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.040+0000 7f0caa7a1640 1 -- 172.21.15.49:0/1581949878 shutdown_connections 2023-11-06T02:09:40.043 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.040+0000 7f0caa7a1640 1 -- 172.21.15.49:0/1581949878 wait complete. 2023-11-06T02:09:40.043 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.040+0000 7f0caa7a1640 1 Processor -- start 2023-11-06T02:09:40.043 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.040+0000 7f0caa7a1640 1 -- start start 2023-11-06T02:09:40.043 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.040+0000 7f0caa7a1640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0ca40caf30 0x7f0ca4150e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:40.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.040+0000 7f0caa7a1640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ca415ceb0 0x7f0ca40c32f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:40.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.040+0000 7f0caa7a1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ca40c6700 0x7f0ca40c3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:40.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.040+0000 7f0caa7a1640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca40c3d70 con 0x7f0ca40c6700 2023-11-06T02:09:40.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.040+0000 7f0caa7a1640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca4156f40 con 0x7f0ca415ceb0 2023-11-06T02:09:40.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.040+0000 7f0caa7a1640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca4157220 con 0x7f0ca40caf30 2023-11-06T02:09:40.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.040+0000 7f0ca8d3e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ca40c6700 0x7f0ca40c3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:40.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.040+0000 7f0ca8d3e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ca40c6700 0x7f0ca40c3830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36480/0 (socket says 172.21.15.49:36480) 2023-11-06T02:09:40.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.040+0000 7f0ca8d3e640 1 -- 172.21.15.49:0/2493322278 learned_addr learned my addr 172.21.15.49:0/2493322278 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:40.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.040+0000 7f0ca3fff640 1 --2- 172.21.15.49:0/2493322278 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0ca40caf30 0x7f0ca4150e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:40.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.040+0000 7f0ca37fe640 1 --2- 172.21.15.49:0/2493322278 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ca415ceb0 0x7f0ca40c32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:40.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.040+0000 7f0ca8d3e640 1 -- 172.21.15.49:0/2493322278 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0ca40caf30 msgr2=0x7f0ca4150e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:40.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.040+0000 7f0ca8d3e640 1 --2- 172.21.15.49:0/2493322278 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0ca40caf30 0x7f0ca4150e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:40.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.040+0000 7f0ca8d3e640 1 -- 172.21.15.49:0/2493322278 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ca415ceb0 msgr2=0x7f0ca40c32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:40.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.040+0000 7f0ca8d3e640 1 --2- 172.21.15.49:0/2493322278 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ca415ceb0 0x7f0ca40c32f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:40.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.040+0000 7f0ca8d3e640 1 -- 172.21.15.49:0/2493322278 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c9c009b30 con 0x7f0ca40c6700 2023-11-06T02:09:40.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.040+0000 7f0ca8d3e640 1 --2- 172.21.15.49:0/2493322278 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ca40c6700 0x7f0ca40c3830 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c01a5c0 tx=0x7f0c9c019730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:40.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.040+0000 7f0ca0ff9640 1 -- 172.21.15.49:0/2493322278 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c9c0b8670 con 0x7f0ca40c6700 2023-11-06T02:09:40.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.040+0000 7f0ca0ff9640 1 -- 172.21.15.49:0/2493322278 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c9c0b87d0 con 0x7f0ca40c6700 2023-11-06T02:09:40.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.040+0000 7f0caa7a1640 1 -- 172.21.15.49:0/2493322278 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ca4157500 con 0x7f0ca40c6700 2023-11-06T02:09:40.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.040+0000 7f0ca0ff9640 1 -- 172.21.15.49:0/2493322278 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c9c0b8b60 con 0x7f0ca40c6700 2023-11-06T02:09:40.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.040+0000 7f0caa7a1640 1 -- 172.21.15.49:0/2493322278 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ca41513a0 con 0x7f0ca40c6700 2023-11-06T02:09:40.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.044+0000 7f0caa7a1640 1 -- 172.21.15.49:0/2493322278 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ca4067140 con 0x7f0ca40c6700 2023-11-06T02:09:40.049 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.044+0000 7f0ca0ff9640 1 -- 172.21.15.49:0/2493322278 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f0c9c0b8cc0 con 0x7f0ca40c6700 2023-11-06T02:09:40.049 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.044+0000 7f0ca0ff9640 1 --2- 172.21.15.49:0/2493322278 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0c90065a60 0x7f0c90067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:40.049 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.044+0000 7f0ca0ff9640 1 -- 172.21.15.49:0/2493322278 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0c9c11eb10 con 0x7f0ca40c6700 2023-11-06T02:09:40.049 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.044+0000 7f0ca3fff640 1 --2- 172.21.15.49:0/2493322278 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0c90065a60 0x7f0c90067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:40.050 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.044+0000 7f0ca3fff640 1 --2- 172.21.15.49:0/2493322278 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0c90065a60 0x7f0c90067f20 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0c94006fd0 tx=0x7f0c94008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:40.050 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.048+0000 7f0ca0ff9640 1 -- 172.21.15.49:0/2493322278 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f0c9c0ec5a0 con 0x7f0ca40c6700 2023-11-06T02:09:40.203 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.204+0000 7f0caa7a1640 1 -- 172.21.15.49:0/2493322278 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0ca4071e20 con 0x7f0ca40c6700 2023-11-06T02:09:40.206 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.208+0000 7f0ca0ff9640 1 -- 172.21.15.49:0/2493322278 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+17896 (secure 0 0 0) 0x7f0c9c0efbc0 con 0x7f0ca40c6700 2023-11-06T02:09:40.207 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:09:40.207 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":36,"fsid":"1a8b035c-6ba4-499a-b876-aea47fd9c0d4","created":"2023-11-06T02:08:06.440857+0000","modified":"2023-11-06T02:09:39.873188+0000","last_up_change":"2023-11-06T02:08:17.982372+0000","last_in_change":"2023-11-06T02:08:08.906631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T02:08:19.252222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T02:08:24.776211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-06T02:09:37.906848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-06T02:09:38.849380+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ad55af8-0571-4e6a-8722-b5d429b7496d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6809","nonce":38116},{"type":"v1","addr":"172.21.15.49:6811","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6813","nonce":38116},{"type":"v1","addr":"172.21.15.49:6815","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6821","nonce":38116},{"type":"v1","addr":"172.21.15.49:6823","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6817","nonce":38116},{"type":"v1","addr":"172.21.15.49:6819","nonce":38116}]},"public_addr":"172.21.15.49:6811/38116","cluster_addr":"172.21.15.49:6815/38116","heartbeat_back_addr":"172.21.15.49:6823/38116","heartbeat_front_addr":"172.21.15.49:6819/38116","state":["exists","up"]},{"osd":1,"uuid":"c8b31f85-a757-43ed-9419-351acc46611b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":38113},{"type":"v1","addr":"172.21.15.49:6810","nonce":38113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":38113},{"type":"v1","addr":"172.21.15.49:6814","nonce":38113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":38113},{"type":"v1","addr":"172.21.15.49:6822","nonce":38113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":38113},{"type":"v1","addr":"172.21.15.49:6818","nonce":38113}]},"public_addr":"172.21.15.49:6810/38113","cluster_addr":"172.21.15.49:6814/38113","heartbeat_back_addr":"172.21.15.49:6822/38113","heartbeat_front_addr":"172.21.15.49:6818/38113","state":["exists","up"]},{"osd":2,"uuid":"555b3bf3-0a70-4b91-8efd-e4204b5fc26e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":38114},{"type":"v1","addr":"172.21.15.49:6801","nonce":38114}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":38114},{"type":"v1","addr":"172.21.15.49:6803","nonce":38114}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":38114},{"type":"v1","addr":"172.21.15.49:6807","nonce":38114}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":38114},{"type":"v1","addr":"172.21.15.49:6805","nonce":38114}]},"public_addr":"172.21.15.49:6801/38114","cluster_addr":"172.21.15.49:6803/38114","heartbeat_back_addr":"172.21.15.49:6807/38114","heartbeat_front_addr":"172.21.15.49:6805/38114","state":["exists","up"]},{"osd":3,"uuid":"8d4e5d77-03ef-4b9b-bfcb-5ae93a46795c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":38115},{"type":"v1","addr":"172.21.15.49:6825","nonce":38115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":38115},{"type":"v1","addr":"172.21.15.49:6827","nonce":38115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":38115},{"type":"v1","addr":"172.21.15.49:6831","nonce":38115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":38115},{"type":"v1","addr":"172.21.15.49:6829","nonce":38115}]},"public_addr":"172.21.15.49:6825/38115","cluster_addr":"172.21.15.49:6827/38115","heartbeat_back_addr":"172.21.15.49:6831/38115","heartbeat_front_addr":"172.21.15.49:6829/38115","state":["exists","up"]},{"osd":4,"uuid":"dba03ad4-68a3-4cf3-9218-ac611f02eacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":38417},{"type":"v1","addr":"172.21.15.150:6809","nonce":38417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":38417},{"type":"v1","addr":"172.21.15.150:6811","nonce":38417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":38417},{"type":"v1","addr":"172.21.15.150:6815","nonce":38417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":38417},{"type":"v1","addr":"172.21.15.150:6813","nonce":38417}]},"public_addr":"172.21.15.150:6809/38417","cluster_addr":"172.21.15.150:6811/38417","heartbeat_back_addr":"172.21.15.150:6815/38417","heartbeat_front_addr":"172.21.15.150:6813/38417","state":["exists","up"]},{"osd":5,"uuid":"0df9c303-c255-4640-8e91-01f1c3efe18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":38418},{"type":"v1","addr":"172.21.15.150:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":38418},{"type":"v1","addr":"172.21.15.150:6803","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":38418},{"type":"v1","addr":"172.21.15.150:6807","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":38418},{"type":"v1","addr":"172.21.15.150:6805","nonce":38418}]},"public_addr":"172.21.15.150:6801/38418","cluster_addr":"172.21.15.150:6803/38418","heartbeat_back_addr":"172.21.15.150:6807/38418","heartbeat_front_addr":"172.21.15.150:6805/38418","state":["exists","up"]},{"osd":6,"uuid":"e59fa189-4b70-4c06-9722-c02b90828bf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":38428},{"type":"v1","addr":"172.21.15.150:6825","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":38428},{"type":"v1","addr":"172.21.15.150:6827","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":38428},{"type":"v1","addr":"172.21.15.150:6831","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":38428},{"type":"v1","addr":"172.21.15.150:6829","nonce":38428}]},"public_addr":"172.21.15.150:6825/38428","cluster_addr":"172.21.15.150:6827/38428","heartbeat_back_addr":"172.21.15.150:6831/38428","heartbeat_front_addr":"172.21.15.150:6829/38428","state":["exists","up"]},{"osd":7,"uuid":"a64f57e1-ef98-46bd-b304-385759e95898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":38421},{"type":"v1","addr":"172.21.15.150:6817","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":38421},{"type":"v1","addr":"172.21.15.150:6819","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":38421},{"type":"v1","addr":"172.21.15.150:6823","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":38421},{"type":"v1","addr":"172.21.15.150:6821","nonce":38421}]},"public_addr":"172.21.15.150:6817/38421","cluster_addr":"172.21.15.150:6819/38421","heartbeat_back_addr":"172.21.15.150:6823/38421","heartbeat_front_addr":"172.21.15.150:6821/38421","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.438552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.413280+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.296466+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.424966+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.276953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.230984+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.410235+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.480084+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T02:09:40.209 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.208+0000 7f0caa7a1640 1 -- 172.21.15.49:0/2493322278 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0c90065a60 msgr2=0x7f0c90067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:40.209 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.208+0000 7f0caa7a1640 1 --2- 172.21.15.49:0/2493322278 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0c90065a60 0x7f0c90067f20 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0c94006fd0 tx=0x7f0c94008040 comp rx=0 tx=0).stop 2023-11-06T02:09:40.209 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.208+0000 7f0caa7a1640 1 -- 172.21.15.49:0/2493322278 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ca40c6700 msgr2=0x7f0ca40c3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:40.209 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.208+0000 7f0caa7a1640 1 --2- 172.21.15.49:0/2493322278 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ca40c6700 0x7f0ca40c3830 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c01a5c0 tx=0x7f0c9c019730 comp rx=0 tx=0).stop 2023-11-06T02:09:40.210 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.208+0000 7f0caa7a1640 1 -- 172.21.15.49:0/2493322278 shutdown_connections 2023-11-06T02:09:40.210 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.208+0000 7f0caa7a1640 1 --2- 172.21.15.49:0/2493322278 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0c90065a60 0x7f0c90067f20 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:40.210 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.208+0000 7f0caa7a1640 1 --2- 172.21.15.49:0/2493322278 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ca40c6700 0x7f0ca40c3830 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:40.210 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.208+0000 7f0caa7a1640 1 --2- 172.21.15.49:0/2493322278 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ca415ceb0 0x7f0ca40c32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:40.210 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.208+0000 7f0caa7a1640 1 --2- 172.21.15.49:0/2493322278 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0ca40caf30 0x7f0ca4150e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:40.211 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.208+0000 7f0caa7a1640 1 -- 172.21.15.49:0/2493322278 >> 172.21.15.49:0/2493322278 conn(0x7f0ca40cc140 msgr2=0x7f0ca4154c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:40.211 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.208+0000 7f0caa7a1640 1 -- 172.21.15.49:0/2493322278 shutdown_connections 2023-11-06T02:09:40.211 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.208+0000 7f0caa7a1640 1 -- 172.21.15.49:0/2493322278 wait complete. 2023-11-06T02:09:40.226 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T02:09:40.353 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.352+0000 7fc37ae87640 1 Processor -- start 2023-11-06T02:09:40.353 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.352+0000 7fc37ae87640 1 -- start start 2023-11-06T02:09:40.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.356+0000 7fc37ae87640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc37415ab50 0x7fc3740c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:40.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.356+0000 7fc37ae87640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc3740b0790 0x7fc3740c2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:40.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.356+0000 7fc37ae87640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3740cab80 con 0x7fc3740cb010 2023-11-06T02:09:40.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.356+0000 7fc37ae87640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc37415af30 con 0x7fc3740b0790 2023-11-06T02:09:40.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.356+0000 7fc37ae87640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3740cb5b0 con 0x7fc37415ab50 2023-11-06T02:09:40.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.356+0000 7fc379424640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc3740b0790 0x7fc3740c2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:40.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.356+0000 7fc379424640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc3740b0790 0x7fc3740c2710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36482/0 (socket says 172.21.15.49:36482) 2023-11-06T02:09:40.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.356+0000 7fc379424640 1 -- 172.21.15.49:0/3276123957 learned_addr learned my addr 172.21.15.49:0/3276123957 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:40.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.356+0000 7fc378c23640 1 --2- 172.21.15.49:0/3276123957 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc37415ab50 0x7fc3740c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:40.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.356+0000 7fc379424640 1 -- 172.21.15.49:0/3276123957 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc37415ab50 msgr2=0x7fc3740c9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:40.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.356+0000 7fc379424640 1 --2- 172.21.15.49:0/3276123957 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc37415ab50 0x7fc3740c9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:40.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.356+0000 7fc379424640 1 -- 172.21.15.49:0/3276123957 >> v1:172.21.15.150:6789/0 conn(0x7fc3740cb010 legacy=0x7fc3740b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:40.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.356+0000 7fc379424640 1 -- 172.21.15.49:0/3276123957 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3740cb890 con 0x7fc3740b0790 2023-11-06T02:09:40.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.356+0000 7fc379424640 1 --2- 172.21.15.49:0/3276123957 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3740b0790 0x7fc3740c2710 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fc36c009e50 tx=0x7fc36c01c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d94b291c45cc49c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:40.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.356+0000 7fc3737fe640 1 -- 172.21.15.49:0/3276123957 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc36c0a58d0 con 0x7fc3740b0790 2023-11-06T02:09:40.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.360+0000 7fc3737fe640 1 -- 172.21.15.49:0/3276123957 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc36c0a5a30 con 0x7fc3740b0790 2023-11-06T02:09:40.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.360+0000 7fc3737fe640 1 -- 172.21.15.49:0/3276123957 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc36c0a5dc0 con 0x7fc3740b0790 2023-11-06T02:09:40.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.360+0000 7fc37ae87640 1 -- 172.21.15.49:0/3276123957 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3740b0790 msgr2=0x7fc3740c2710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:40.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.360+0000 7fc37ae87640 1 --2- 172.21.15.49:0/3276123957 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3740b0790 0x7fc3740c2710 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fc36c009e50 tx=0x7fc36c01c9e0 comp rx=0 tx=0).stop 2023-11-06T02:09:40.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.360+0000 7fc37ae87640 1 -- 172.21.15.49:0/3276123957 shutdown_connections 2023-11-06T02:09:40.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.360+0000 7fc37ae87640 1 --2- 172.21.15.49:0/3276123957 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3740b0790 0x7fc3740c2710 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:40.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.360+0000 7fc37ae87640 1 --2- 172.21.15.49:0/3276123957 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc37415ab50 0x7fc3740c9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:40.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.360+0000 7fc37ae87640 1 -- 172.21.15.49:0/3276123957 >> 172.21.15.49:0/3276123957 conn(0x7fc37406c0a0 msgr2=0x7fc37406c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:40.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.360+0000 7fc37ae87640 1 -- 172.21.15.49:0/3276123957 shutdown_connections 2023-11-06T02:09:40.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.360+0000 7fc37ae87640 1 -- 172.21.15.49:0/3276123957 wait complete. 2023-11-06T02:09:40.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.360+0000 7fc37ae87640 1 Processor -- start 2023-11-06T02:09:40.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.360+0000 7fc37ae87640 1 -- start start 2023-11-06T02:09:40.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.360+0000 7fc37ae87640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3740b0790 0x7fc374149440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:40.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.360+0000 7fc37ae87640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc3740cb010 0x7fc3741535c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:40.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.360+0000 7fc37ae87640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc37415ab50 0x7fc374153b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:40.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.360+0000 7fc37ae87640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc374149e30 con 0x7fc3740b0790 2023-11-06T02:09:40.365 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.360+0000 7fc37ae87640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3741f5a40 con 0x7fc3740cb010 2023-11-06T02:09:40.365 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.360+0000 7fc37ae87640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3741f5760 con 0x7fc37415ab50 2023-11-06T02:09:40.365 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.360+0000 7fc378c23640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3740b0790 0x7fc374149440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:40.365 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.360+0000 7fc378c23640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3740b0790 0x7fc374149440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36490/0 (socket says 172.21.15.49:36490) 2023-11-06T02:09:40.365 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.360+0000 7fc378c23640 1 -- 172.21.15.49:0/3852823699 learned_addr learned my addr 172.21.15.49:0/3852823699 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:40.366 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.360+0000 7fc379424640 1 --2- 172.21.15.49:0/3852823699 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc37415ab50 0x7fc374153b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:40.366 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.360+0000 7fc378c23640 1 -- 172.21.15.49:0/3852823699 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc37415ab50 msgr2=0x7fc374153b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:40.366 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.360+0000 7fc378c23640 1 --2- 172.21.15.49:0/3852823699 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc37415ab50 0x7fc374153b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:40.366 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.360+0000 7fc378c23640 1 -- 172.21.15.49:0/3852823699 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc3740cb010 msgr2=0x7fc3741535c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:40.367 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.360+0000 7fc373fff640 1 --2- 172.21.15.49:0/3852823699 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc3740cb010 0x7fc3741535c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:40.367 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.360+0000 7fc378c23640 1 --2- 172.21.15.49:0/3852823699 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc3740cb010 0x7fc3741535c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:40.367 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.360+0000 7fc378c23640 1 -- 172.21.15.49:0/3852823699 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc36c009b30 con 0x7fc3740b0790 2023-11-06T02:09:40.367 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.360+0000 7fc378c23640 1 --2- 172.21.15.49:0/3852823699 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3740b0790 0x7fc374149440 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fc36400e870 tx=0x7fc36400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:40.367 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.360+0000 7fc3717fa640 1 -- 172.21.15.49:0/3852823699 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3640095d0 con 0x7fc3740b0790 2023-11-06T02:09:40.368 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.360+0000 7fc3717fa640 1 -- 172.21.15.49:0/3852823699 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc36400c940 con 0x7fc3740b0790 2023-11-06T02:09:40.368 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.360+0000 7fc3717fa640 1 -- 172.21.15.49:0/3852823699 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc36400ccb0 con 0x7fc3740b0790 2023-11-06T02:09:40.368 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.360+0000 7fc37ae87640 1 -- 172.21.15.49:0/3852823699 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3741544b0 con 0x7fc3740b0790 2023-11-06T02:09:40.368 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.360+0000 7fc37ae87640 1 -- 172.21.15.49:0/3852823699 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc374154a70 con 0x7fc3740b0790 2023-11-06T02:09:40.368 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.364+0000 7fc37ae87640 1 -- 172.21.15.49:0/3852823699 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc374154c30 con 0x7fc3740b0790 2023-11-06T02:09:40.369 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.364+0000 7fc3717fa640 1 -- 172.21.15.49:0/3852823699 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fc364013470 con 0x7fc3740b0790 2023-11-06T02:09:40.369 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.364+0000 7fc3717fa640 1 --2- 172.21.15.49:0/3852823699 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc360065a60 0x7fc360067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:40.369 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.364+0000 7fc3717fa640 1 -- 172.21.15.49:0/3852823699 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc3640838d0 con 0x7fc3740b0790 2023-11-06T02:09:40.369 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.364+0000 7fc373fff640 1 --2- 172.21.15.49:0/3852823699 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc360065a60 0x7fc360067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:40.370 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.364+0000 7fc373fff640 1 --2- 172.21.15.49:0/3852823699 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc360065a60 0x7fc360067f20 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc3741541b0 tx=0x7fc368009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:40.370 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.368+0000 7fc3717fa640 1 -- 172.21.15.49:0/3852823699 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fc364051150 con 0x7fc3740b0790 2023-11-06T02:09:40.527 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.528+0000 7fc37ae87640 1 -- 172.21.15.49:0/3852823699 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc3740b0de0 con 0x7fc3740b0790 2023-11-06T02:09:40.530 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.532+0000 7fc3717fa640 1 -- 172.21.15.49:0/3852823699 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+17896 (secure 0 0 0) 0x7fc364021090 con 0x7fc3740b0790 2023-11-06T02:09:40.530 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:09:40.531 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":36,"fsid":"1a8b035c-6ba4-499a-b876-aea47fd9c0d4","created":"2023-11-06T02:08:06.440857+0000","modified":"2023-11-06T02:09:39.873188+0000","last_up_change":"2023-11-06T02:08:17.982372+0000","last_in_change":"2023-11-06T02:08:08.906631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T02:08:19.252222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T02:08:24.776211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-06T02:09:37.906848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-06T02:09:38.849380+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ad55af8-0571-4e6a-8722-b5d429b7496d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6809","nonce":38116},{"type":"v1","addr":"172.21.15.49:6811","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6813","nonce":38116},{"type":"v1","addr":"172.21.15.49:6815","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6821","nonce":38116},{"type":"v1","addr":"172.21.15.49:6823","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6817","nonce":38116},{"type":"v1","addr":"172.21.15.49:6819","nonce":38116}]},"public_addr":"172.21.15.49:6811/38116","cluster_addr":"172.21.15.49:6815/38116","heartbeat_back_addr":"172.21.15.49:6823/38116","heartbeat_front_addr":"172.21.15.49:6819/38116","state":["exists","up"]},{"osd":1,"uuid":"c8b31f85-a757-43ed-9419-351acc46611b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":38113},{"type":"v1","addr":"172.21.15.49:6810","nonce":38113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":38113},{"type":"v1","addr":"172.21.15.49:6814","nonce":38113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":38113},{"type":"v1","addr":"172.21.15.49:6822","nonce":38113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":38113},{"type":"v1","addr":"172.21.15.49:6818","nonce":38113}]},"public_addr":"172.21.15.49:6810/38113","cluster_addr":"172.21.15.49:6814/38113","heartbeat_back_addr":"172.21.15.49:6822/38113","heartbeat_front_addr":"172.21.15.49:6818/38113","state":["exists","up"]},{"osd":2,"uuid":"555b3bf3-0a70-4b91-8efd-e4204b5fc26e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":38114},{"type":"v1","addr":"172.21.15.49:6801","nonce":38114}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":38114},{"type":"v1","addr":"172.21.15.49:6803","nonce":38114}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":38114},{"type":"v1","addr":"172.21.15.49:6807","nonce":38114}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":38114},{"type":"v1","addr":"172.21.15.49:6805","nonce":38114}]},"public_addr":"172.21.15.49:6801/38114","cluster_addr":"172.21.15.49:6803/38114","heartbeat_back_addr":"172.21.15.49:6807/38114","heartbeat_front_addr":"172.21.15.49:6805/38114","state":["exists","up"]},{"osd":3,"uuid":"8d4e5d77-03ef-4b9b-bfcb-5ae93a46795c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":38115},{"type":"v1","addr":"172.21.15.49:6825","nonce":38115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":38115},{"type":"v1","addr":"172.21.15.49:6827","nonce":38115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":38115},{"type":"v1","addr":"172.21.15.49:6831","nonce":38115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":38115},{"type":"v1","addr":"172.21.15.49:6829","nonce":38115}]},"public_addr":"172.21.15.49:6825/38115","cluster_addr":"172.21.15.49:6827/38115","heartbeat_back_addr":"172.21.15.49:6831/38115","heartbeat_front_addr":"172.21.15.49:6829/38115","state":["exists","up"]},{"osd":4,"uuid":"dba03ad4-68a3-4cf3-9218-ac611f02eacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":38417},{"type":"v1","addr":"172.21.15.150:6809","nonce":38417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":38417},{"type":"v1","addr":"172.21.15.150:6811","nonce":38417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":38417},{"type":"v1","addr":"172.21.15.150:6815","nonce":38417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":38417},{"type":"v1","addr":"172.21.15.150:6813","nonce":38417}]},"public_addr":"172.21.15.150:6809/38417","cluster_addr":"172.21.15.150:6811/38417","heartbeat_back_addr":"172.21.15.150:6815/38417","heartbeat_front_addr":"172.21.15.150:6813/38417","state":["exists","up"]},{"osd":5,"uuid":"0df9c303-c255-4640-8e91-01f1c3efe18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":38418},{"type":"v1","addr":"172.21.15.150:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":38418},{"type":"v1","addr":"172.21.15.150:6803","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":38418},{"type":"v1","addr":"172.21.15.150:6807","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":38418},{"type":"v1","addr":"172.21.15.150:6805","nonce":38418}]},"public_addr":"172.21.15.150:6801/38418","cluster_addr":"172.21.15.150:6803/38418","heartbeat_back_addr":"172.21.15.150:6807/38418","heartbeat_front_addr":"172.21.15.150:6805/38418","state":["exists","up"]},{"osd":6,"uuid":"e59fa189-4b70-4c06-9722-c02b90828bf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":38428},{"type":"v1","addr":"172.21.15.150:6825","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":38428},{"type":"v1","addr":"172.21.15.150:6827","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":38428},{"type":"v1","addr":"172.21.15.150:6831","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":38428},{"type":"v1","addr":"172.21.15.150:6829","nonce":38428}]},"public_addr":"172.21.15.150:6825/38428","cluster_addr":"172.21.15.150:6827/38428","heartbeat_back_addr":"172.21.15.150:6831/38428","heartbeat_front_addr":"172.21.15.150:6829/38428","state":["exists","up"]},{"osd":7,"uuid":"a64f57e1-ef98-46bd-b304-385759e95898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":38421},{"type":"v1","addr":"172.21.15.150:6817","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":38421},{"type":"v1","addr":"172.21.15.150:6819","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":38421},{"type":"v1","addr":"172.21.15.150:6823","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":38421},{"type":"v1","addr":"172.21.15.150:6821","nonce":38421}]},"public_addr":"172.21.15.150:6817/38421","cluster_addr":"172.21.15.150:6819/38421","heartbeat_back_addr":"172.21.15.150:6823/38421","heartbeat_front_addr":"172.21.15.150:6821/38421","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.438552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.413280+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.296466+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.424966+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.276953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.230984+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.410235+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.480084+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T02:09:40.532 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.532+0000 7fc37ae87640 1 -- 172.21.15.49:0/3852823699 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc360065a60 msgr2=0x7fc360067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:40.533 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.532+0000 7fc37ae87640 1 --2- 172.21.15.49:0/3852823699 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc360065a60 0x7fc360067f20 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc3741541b0 tx=0x7fc368009040 comp rx=0 tx=0).stop 2023-11-06T02:09:40.533 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.532+0000 7fc37ae87640 1 -- 172.21.15.49:0/3852823699 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3740b0790 msgr2=0x7fc374149440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:40.533 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.532+0000 7fc37ae87640 1 --2- 172.21.15.49:0/3852823699 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3740b0790 0x7fc374149440 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fc36400e870 tx=0x7fc36400ed30 comp rx=0 tx=0).stop 2023-11-06T02:09:40.533 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.532+0000 7fc37ae87640 1 -- 172.21.15.49:0/3852823699 shutdown_connections 2023-11-06T02:09:40.533 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.532+0000 7fc37ae87640 1 --2- 172.21.15.49:0/3852823699 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc360065a60 0x7fc360067f20 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:40.534 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.532+0000 7fc37ae87640 1 --2- 172.21.15.49:0/3852823699 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc37415ab50 0x7fc374153b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:40.534 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.532+0000 7fc37ae87640 1 --2- 172.21.15.49:0/3852823699 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc3740cb010 0x7fc3741535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:40.534 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.532+0000 7fc37ae87640 1 --2- 172.21.15.49:0/3852823699 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3740b0790 0x7fc374149440 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:40.534 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.532+0000 7fc37ae87640 1 -- 172.21.15.49:0/3852823699 >> 172.21.15.49:0/3852823699 conn(0x7fc37406c0a0 msgr2=0x7fc3740acc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:40.534 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.532+0000 7fc37ae87640 1 -- 172.21.15.49:0/3852823699 shutdown_connections 2023-11-06T02:09:40.535 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.532+0000 7fc37ae87640 1 -- 172.21.15.49:0/3852823699 wait complete. 2023-11-06T02:09:40.550 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-06T02:09:40.674 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.676+0000 7ff1ef07c640 1 Processor -- start 2023-11-06T02:09:40.675 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.676+0000 7ff1ef07c640 1 -- start start 2023-11-06T02:09:40.676 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.676+0000 7ff1ef07c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff1e8070870 0x7ff1e80b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:40.676 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.676+0000 7ff1ef07c640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1e80ae360 con 0x7ff1e806d150 2023-11-06T02:09:40.676 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.676+0000 7ff1ef07c640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1e80ae640 con 0x7ff1e815a880 2023-11-06T02:09:40.677 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.676+0000 7ff1ef07c640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1e80ae920 con 0x7ff1e8070870 2023-11-06T02:09:40.677 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.676+0000 7ff1e7fff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff1e8070870 0x7ff1e80b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:40.677 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.676+0000 7ff1e7fff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff1e8070870 0x7ff1e80b15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36504/0 (socket says 172.21.15.49:36504) 2023-11-06T02:09:40.677 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.676+0000 7ff1e7fff640 1 -- 172.21.15.49:0/3250225418 learned_addr learned my addr 172.21.15.49:0/3250225418 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:40.677 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.676+0000 7ff1ed619640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7ff1e806d150 0x7ff1e806d600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:48164/0 (socket says 172.21.15.49:48164) 2023-11-06T02:09:40.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.676+0000 7ff1e7fff640 1 -- 172.21.15.49:0/3250225418 >> v1:172.21.15.150:6789/0 conn(0x7ff1e815a880 legacy=0x7ff1e8070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:40.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.676+0000 7ff1e7fff640 1 -- 172.21.15.49:0/3250225418 >> v1:172.21.15.49:6789/0 conn(0x7ff1e806d150 legacy=0x7ff1e806d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:40.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.676+0000 7ff1e7fff640 1 -- 172.21.15.49:0/3250225418 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1e80aec00 con 0x7ff1e8070870 2023-11-06T02:09:40.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.676+0000 7ff1e77fe640 1 -- 172.21.15.49:0/3250225418 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1823881501 0 0) 0x7ff1e80ae360 con 0x7ff1e806d150 2023-11-06T02:09:40.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.680+0000 7ff1e7fff640 1 --2- 172.21.15.49:0/3250225418 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff1e8070870 0x7ff1e80b15c0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7ff1d8009870 tx=0x7ff1d801c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceff8fda1c7f87d5 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:40.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.680+0000 7ff1e77fe640 1 -- 172.21.15.49:0/3250225418 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d80ad680 con 0x7ff1e8070870 2023-11-06T02:09:40.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.680+0000 7ff1e77fe640 1 -- 172.21.15.49:0/3250225418 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1d80ad7e0 con 0x7ff1e8070870 2023-11-06T02:09:40.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.680+0000 7ff1e77fe640 1 -- 172.21.15.49:0/3250225418 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d80adb70 con 0x7ff1e8070870 2023-11-06T02:09:40.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.680+0000 7ff1ef07c640 1 -- 172.21.15.49:0/3250225418 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff1e8070870 msgr2=0x7ff1e80b15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:40.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.680+0000 7ff1ef07c640 1 --2- 172.21.15.49:0/3250225418 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff1e8070870 0x7ff1e80b15c0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7ff1d8009870 tx=0x7ff1d801c470 comp rx=0 tx=0).stop 2023-11-06T02:09:40.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.680+0000 7ff1ef07c640 1 -- 172.21.15.49:0/3250225418 shutdown_connections 2023-11-06T02:09:40.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.680+0000 7ff1ef07c640 1 --2- 172.21.15.49:0/3250225418 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff1e8070870 0x7ff1e80b15c0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:40.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.680+0000 7ff1ef07c640 1 -- 172.21.15.49:0/3250225418 >> 172.21.15.49:0/3250225418 conn(0x7ff1e806a370 msgr2=0x7ff1e80ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:40.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.680+0000 7ff1ef07c640 1 -- 172.21.15.49:0/3250225418 shutdown_connections 2023-11-06T02:09:40.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.680+0000 7ff1ef07c640 1 -- 172.21.15.49:0/3250225418 wait complete. 2023-11-06T02:09:40.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.680+0000 7ff1ef07c640 1 Processor -- start 2023-11-06T02:09:40.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.680+0000 7ff1ef07c640 1 -- start start 2023-11-06T02:09:40.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.680+0000 7ff1ef07c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff1e806d150 0x7ff1e80cb020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:40.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.680+0000 7ff1ef07c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1e8070870 0x7ff1e814b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:40.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.680+0000 7ff1ef07c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1e815a880 0x7ff1e80c6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:40.686 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.680+0000 7ff1ef07c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e80c7430 con 0x7ff1e806d150 2023-11-06T02:09:40.686 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.680+0000 7ff1ef07c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e80cb6b0 con 0x7ff1e815a880 2023-11-06T02:09:40.686 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.680+0000 7ff1ef07c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e80cb990 con 0x7ff1e8070870 2023-11-06T02:09:40.686 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.680+0000 7ff1ece18640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff1e806d150 0x7ff1e80cb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:40.686 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.684+0000 7ff1ece18640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff1e806d150 0x7ff1e80cb020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36520/0 (socket says 172.21.15.49:36520) 2023-11-06T02:09:40.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.684+0000 7ff1ece18640 1 -- 172.21.15.49:0/2263948846 learned_addr learned my addr 172.21.15.49:0/2263948846 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:40.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.684+0000 7ff1e7fff640 1 --2- 172.21.15.49:0/2263948846 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1e8070870 0x7ff1e814b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:40.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.684+0000 7ff1ed619640 1 --2- 172.21.15.49:0/2263948846 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1e815a880 0x7ff1e80c6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:40.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.684+0000 7ff1ece18640 1 -- 172.21.15.49:0/2263948846 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1e8070870 msgr2=0x7ff1e814b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:40.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.684+0000 7ff1ece18640 1 --2- 172.21.15.49:0/2263948846 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1e8070870 0x7ff1e814b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:40.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.684+0000 7ff1ece18640 1 -- 172.21.15.49:0/2263948846 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1e815a880 msgr2=0x7ff1e80c6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:40.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.684+0000 7ff1ece18640 1 --2- 172.21.15.49:0/2263948846 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1e815a880 0x7ff1e80c6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:40.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.684+0000 7ff1ece18640 1 -- 172.21.15.49:0/2263948846 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1d80094d0 con 0x7ff1e806d150 2023-11-06T02:09:40.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.684+0000 7ff1ed619640 1 --2- 172.21.15.49:0/2263948846 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1e815a880 0x7ff1e80c6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:09:40.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.684+0000 7ff1ece18640 1 --2- 172.21.15.49:0/2263948846 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff1e806d150 0x7ff1e80cb020 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc00b410 tx=0x7ff1dc00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:40.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.684+0000 7ff1e57fa640 1 -- 172.21.15.49:0/2263948846 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc004020 con 0x7ff1e806d150 2023-11-06T02:09:40.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.684+0000 7ff1ef07c640 1 -- 172.21.15.49:0/2263948846 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1e80cbc70 con 0x7ff1e806d150 2023-11-06T02:09:40.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.684+0000 7ff1e57fa640 1 -- 172.21.15.49:0/2263948846 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1dc004180 con 0x7ff1e806d150 2023-11-06T02:09:40.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.684+0000 7ff1e57fa640 1 -- 172.21.15.49:0/2263948846 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc011630 con 0x7ff1e806d150 2023-11-06T02:09:40.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.684+0000 7ff1ef07c640 1 -- 172.21.15.49:0/2263948846 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1e80cc230 con 0x7ff1e806d150 2023-11-06T02:09:40.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.684+0000 7ff1ef07c640 1 -- 172.21.15.49:0/2263948846 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1e8074040 con 0x7ff1e806d150 2023-11-06T02:09:40.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.684+0000 7ff1e57fa640 1 -- 172.21.15.49:0/2263948846 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7ff1dc011790 con 0x7ff1e806d150 2023-11-06T02:09:40.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.684+0000 7ff1e57fa640 1 --2- 172.21.15.49:0/2263948846 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff1d4065a60 0x7ff1d4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:40.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.684+0000 7ff1e57fa640 1 -- 172.21.15.49:0/2263948846 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff1dc083030 con 0x7ff1e806d150 2023-11-06T02:09:40.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.684+0000 7ff1e7fff640 1 --2- 172.21.15.49:0/2263948846 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff1d4065a60 0x7ff1d4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:40.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.688+0000 7ff1e7fff640 1 --2- 172.21.15.49:0/2263948846 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff1d4065a60 0x7ff1d4067f20 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff1d80af000 tx=0x7ff1d80a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:40.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.688+0000 7ff1e57fa640 1 -- 172.21.15.49:0/2263948846 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7ff1dc050810 con 0x7ff1e806d150 2023-11-06T02:09:40.873 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:40.876+0000 7ff1ef07c640 1 -- 172.21.15.49:0/2263948846 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7ff1e806c430 con 0x7ff1e806d150 2023-11-06T02:09:41.889 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:41.888+0000 7ff1e57fa640 1 -- 172.21.15.49:0/2263948846 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v14) v1 ==== 118+0+0 (secure 0 0 0) 0x7ff1dc053e30 con 0x7ff1e806d150 2023-11-06T02:09:41.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:41.892+0000 7ff1ef07c640 1 -- 172.21.15.49:0/2263948846 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff1d4065a60 msgr2=0x7ff1d4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:41.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:41.892+0000 7ff1ef07c640 1 --2- 172.21.15.49:0/2263948846 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff1d4065a60 0x7ff1d4067f20 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff1d80af000 tx=0x7ff1d80a7000 comp rx=0 tx=0).stop 2023-11-06T02:09:41.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:41.892+0000 7ff1ef07c640 1 -- 172.21.15.49:0/2263948846 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff1e806d150 msgr2=0x7ff1e80cb020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:41.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:41.892+0000 7ff1ef07c640 1 --2- 172.21.15.49:0/2263948846 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff1e806d150 0x7ff1e80cb020 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc00b410 tx=0x7ff1dc00b8d0 comp rx=0 tx=0).stop 2023-11-06T02:09:41.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:41.892+0000 7ff1ef07c640 1 -- 172.21.15.49:0/2263948846 shutdown_connections 2023-11-06T02:09:41.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:41.892+0000 7ff1ef07c640 1 --2- 172.21.15.49:0/2263948846 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff1d4065a60 0x7ff1d4067f20 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:41.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:41.892+0000 7ff1ef07c640 1 --2- 172.21.15.49:0/2263948846 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1e815a880 0x7ff1e80c6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:41.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:41.892+0000 7ff1ef07c640 1 --2- 172.21.15.49:0/2263948846 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1e8070870 0x7ff1e814b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:41.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:41.892+0000 7ff1ef07c640 1 --2- 172.21.15.49:0/2263948846 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff1e806d150 0x7ff1e80cb020 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:41.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:41.892+0000 7ff1ef07c640 1 -- 172.21.15.49:0/2263948846 >> 172.21.15.49:0/2263948846 conn(0x7ff1e806a370 msgr2=0x7ff1e806c650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:41.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:41.892+0000 7ff1ef07c640 1 -- 172.21.15.49:0/2263948846 shutdown_connections 2023-11-06T02:09:41.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:41.892+0000 7ff1ef07c640 1 -- 172.21.15.49:0/2263948846 wait complete. 2023-11-06T02:09:41.914 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-06T02:09:41.914 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-06T02:09:42.049 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:42.048+0000 7f413c0a4640 1 Processor -- start 2023-11-06T02:09:42.049 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:42.048+0000 7f413c0a4640 1 -- start start 2023-11-06T02:09:42.050 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:42.052+0000 7f413c0a4640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f41341595a0 0x7f413415dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:42.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:42.052+0000 7f413c0a4640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f413406d640 0x7f41340b0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:42.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:42.052+0000 7f413c0a4640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f41340b0760 0x7f41340c8e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:42.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:42.052+0000 7f413c0a4640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f413415e250 con 0x7f41341595a0 2023-11-06T02:09:42.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:42.052+0000 7f413c0a4640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f413406c5a0 con 0x7f41340b0760 2023-11-06T02:09:42.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:42.052+0000 7f413c0a4640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f413406c880 con 0x7f413406d640 2023-11-06T02:09:42.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:42.052+0000 7f4139e40640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f41341595a0 0x7f413415dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:42.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:42.052+0000 7f4139e40640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f41341595a0 0x7f413415dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36536/0 (socket says 172.21.15.49:36536) 2023-11-06T02:09:42.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:42.052+0000 7f4139e40640 1 -- 172.21.15.49:0/1454647400 learned_addr learned my addr 172.21.15.49:0/1454647400 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:42.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:42.052+0000 7f4139e40640 1 -- 172.21.15.49:0/1454647400 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f413406d640 msgr2=0x7f41340b0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:42.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:42.052+0000 7f4139e40640 1 --2- 172.21.15.49:0/1454647400 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f413406d640 0x7f41340b0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:42.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:42.052+0000 7f4139e40640 1 -- 172.21.15.49:0/1454647400 >> v2:172.21.15.150:3300/0 conn(0x7f41340b0760 msgr2=0x7f41340c8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:42.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:42.052+0000 7f4139e40640 1 --2- 172.21.15.49:0/1454647400 >> v2:172.21.15.150:3300/0 conn(0x7f41340b0760 0x7f41340c8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:42.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:42.052+0000 7f4139e40640 1 -- 172.21.15.49:0/1454647400 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f413406cb60 con 0x7f41341595a0 2023-11-06T02:09:42.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:42.052+0000 7f4139e40640 1 --2- 172.21.15.49:0/1454647400 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41341595a0 0x7f413415dd10 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f413406e400 tx=0x7f413001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=222e4e2793654d1 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:42.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:42.052+0000 7f4138e3e640 1 -- 172.21.15.49:0/1454647400 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41300ab650 con 0x7f41341595a0 2023-11-06T02:09:42.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:42.052+0000 7f4138e3e640 1 -- 172.21.15.49:0/1454647400 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41300ab7b0 con 0x7f41341595a0 2023-11-06T02:09:42.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:42.052+0000 7f4138e3e640 1 -- 172.21.15.49:0/1454647400 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41300abb40 con 0x7f41341595a0 2023-11-06T02:09:42.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:42.056+0000 7f413c0a4640 1 -- 172.21.15.49:0/1454647400 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41341595a0 msgr2=0x7f413415dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:42.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:42.056+0000 7f413c0a4640 1 --2- 172.21.15.49:0/1454647400 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41341595a0 0x7f413415dd10 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f413406e400 tx=0x7f413001c4d0 comp rx=0 tx=0).stop 2023-11-06T02:09:42.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:42.056+0000 7f413c0a4640 1 -- 172.21.15.49:0/1454647400 shutdown_connections 2023-11-06T02:09:42.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:42.056+0000 7f413c0a4640 1 --2- 172.21.15.49:0/1454647400 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f413406d640 0x7f41340b0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:42.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:42.056+0000 7f413c0a4640 1 --2- 172.21.15.49:0/1454647400 >> v2:172.21.15.150:3300/0 conn(0x7f41340b0760 0x7f41340c8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:42.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:42.056+0000 7f413c0a4640 1 --2- 172.21.15.49:0/1454647400 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41341595a0 0x7f413415dd10 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:42.057 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:42.056+0000 7f413c0a4640 1 -- 172.21.15.49:0/1454647400 >> 172.21.15.49:0/1454647400 conn(0x7f413406a370 msgr2=0x7f41340ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:42.057 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:42.056+0000 7f413c0a4640 1 -- 172.21.15.49:0/1454647400 shutdown_connections 2023-11-06T02:09:42.057 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:42.056+0000 7f413c0a4640 1 -- 172.21.15.49:0/1454647400 wait complete. 2023-11-06T02:09:42.057 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:42.056+0000 7f413c0a4640 1 Processor -- start 2023-11-06T02:09:42.057 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:42.056+0000 7f413c0a4640 1 -- start start 2023-11-06T02:09:42.060 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:42.056+0000 7f413c0a4640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f413406d640 0x7f413414b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:42.060 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:42.056+0000 7f413c0a4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41340b0760 0x7f413414c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:42.060 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:42.056+0000 7f413c0a4640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f41341595a0 0x7f413414e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:42.061 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:42.056+0000 7f413c0a4640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f413414d6c0 con 0x7f41340b0760 2023-11-06T02:09:42.061 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:42.056+0000 7f413c0a4640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f413414baf0 con 0x7f413406d640 2023-11-06T02:09:42.061 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:42.056+0000 7f413c0a4640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f413414cab0 con 0x7f41341595a0 2023-11-06T02:09:42.061 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:42.056+0000 7f413963f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41340b0760 0x7f413414c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:42.061 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:42.056+0000 7f413963f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41340b0760 0x7f413414c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36538/0 (socket says 172.21.15.49:36538) 2023-11-06T02:09:42.061 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:42.056+0000 7f413963f640 1 -- 172.21.15.49:0/3294880276 learned_addr learned my addr 172.21.15.49:0/3294880276 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:42.062 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:42.056+0000 7f4139e40640 1 --2- 172.21.15.49:0/3294880276 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f413406d640 0x7f413414b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:42.062 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:42.056+0000 7f413a641640 1 --2- 172.21.15.49:0/3294880276 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f41341595a0 0x7f413414e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:42.062 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:42.056+0000 7f413963f640 1 -- 172.21.15.49:0/3294880276 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f41341595a0 msgr2=0x7f413414e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:42.062 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:42.056+0000 7f413963f640 1 --2- 172.21.15.49:0/3294880276 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f41341595a0 0x7f413414e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:42.062 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:42.056+0000 7f413963f640 1 -- 172.21.15.49:0/3294880276 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f413406d640 msgr2=0x7f413414b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:42.063 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:42.056+0000 7f413963f640 1 --2- 172.21.15.49:0/3294880276 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f413406d640 0x7f413414b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:42.063 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:42.056+0000 7f413963f640 1 -- 172.21.15.49:0/3294880276 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41300094d0 con 0x7f41340b0760 2023-11-06T02:09:42.063 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:42.056+0000 7f4139e40640 1 --2- 172.21.15.49:0/3294880276 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f413406d640 0x7f413414b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:09:42.063 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:42.056+0000 7f413963f640 1 --2- 172.21.15.49:0/3294880276 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41340b0760 0x7f413414c060 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f412400b550 tx=0x7f412400ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:42.063 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:42.056+0000 7f412a7fc640 1 -- 172.21.15.49:0/3294880276 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4124010870 con 0x7f41340b0760 2023-11-06T02:09:42.064 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:42.056+0000 7f412a7fc640 1 -- 172.21.15.49:0/3294880276 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41240109f0 con 0x7f41340b0760 2023-11-06T02:09:42.064 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:42.056+0000 7f413c0a4640 1 -- 172.21.15.49:0/3294880276 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f413414cd90 con 0x7f41340b0760 2023-11-06T02:09:42.064 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:42.056+0000 7f412a7fc640 1 -- 172.21.15.49:0/3294880276 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4124010da0 con 0x7f41340b0760 2023-11-06T02:09:42.064 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:42.056+0000 7f413c0a4640 1 -- 172.21.15.49:0/3294880276 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f413414d350 con 0x7f41340b0760 2023-11-06T02:09:42.064 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:42.060+0000 7f413c0a4640 1 -- 172.21.15.49:0/3294880276 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f413406c210 con 0x7f41340b0760 2023-11-06T02:09:42.065 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:42.060+0000 7f412a7fc640 1 -- 172.21.15.49:0/3294880276 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f412400f3c0 con 0x7f41340b0760 2023-11-06T02:09:42.065 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:42.060+0000 7f412a7fc640 1 --2- 172.21.15.49:0/3294880276 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4118065a60 0x7f4118067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:42.065 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:42.060+0000 7f412a7fc640 1 -- 172.21.15.49:0/3294880276 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4124083380 con 0x7f41340b0760 2023-11-06T02:09:42.065 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:42.060+0000 7f4139e40640 1 --2- 172.21.15.49:0/3294880276 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4118065a60 0x7f4118067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:42.065 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:42.060+0000 7f4139e40640 1 --2- 172.21.15.49:0/3294880276 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4118065a60 0x7f4118067f20 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f41300af000 tx=0x7f4130008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:42.066 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:42.064+0000 7f412a7fc640 1 -- 172.21.15.49:0/3294880276 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f413406c210 con 0x7f41340b0760 2023-11-06T02:09:42.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:42.252+0000 7f413c0a4640 1 -- 172.21.15.49:0/3294880276 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f413406c210 con 0x7f41340b0760 2023-11-06T02:09:43.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.136+0000 7f412a7fc640 1 -- 172.21.15.49:0/3294880276 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v15) v1 ==== 122+0+0 (secure 0 0 0) 0x7f413406c210 con 0x7f41340b0760 2023-11-06T02:09:43.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.140+0000 7f413c0a4640 1 -- 172.21.15.49:0/3294880276 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4118065a60 msgr2=0x7f4118067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:43.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.140+0000 7f413c0a4640 1 --2- 172.21.15.49:0/3294880276 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4118065a60 0x7f4118067f20 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f41300af000 tx=0x7f4130008040 comp rx=0 tx=0).stop 2023-11-06T02:09:43.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.140+0000 7f413c0a4640 1 -- 172.21.15.49:0/3294880276 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41340b0760 msgr2=0x7f413414c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:43.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.140+0000 7f413c0a4640 1 --2- 172.21.15.49:0/3294880276 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41340b0760 0x7f413414c060 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f412400b550 tx=0x7f412400ba10 comp rx=0 tx=0).stop 2023-11-06T02:09:43.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.140+0000 7f413c0a4640 1 -- 172.21.15.49:0/3294880276 shutdown_connections 2023-11-06T02:09:43.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.140+0000 7f413c0a4640 1 --2- 172.21.15.49:0/3294880276 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4118065a60 0x7f4118067f20 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:43.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.140+0000 7f413c0a4640 1 --2- 172.21.15.49:0/3294880276 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41340b0760 0x7f413414c060 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:43.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.140+0000 7f413c0a4640 1 --2- 172.21.15.49:0/3294880276 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f41341595a0 0x7f413414e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:43.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.140+0000 7f413c0a4640 1 --2- 172.21.15.49:0/3294880276 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f413406d640 0x7f413414b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:43.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.140+0000 7f413c0a4640 1 -- 172.21.15.49:0/3294880276 >> 172.21.15.49:0/3294880276 conn(0x7f413406a370 msgr2=0x7f41340cc7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:43.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.140+0000 7f413c0a4640 1 -- 172.21.15.49:0/3294880276 shutdown_connections 2023-11-06T02:09:43.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.140+0000 7f413c0a4640 1 -- 172.21.15.49:0/3294880276 wait complete. 2023-11-06T02:09:43.158 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-06T02:09:43.158 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T02:09:43.280 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.280+0000 7efed3102640 1 Processor -- start 2023-11-06T02:09:43.281 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.280+0000 7efed3102640 1 -- start start 2023-11-06T02:09:43.282 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.284+0000 7efed3102640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7efecc0b0790 0x7efecc0cc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:43.282 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.284+0000 7efed3102640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efecc0c8c50 0x7efecc0cced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:43.282 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.284+0000 7efed3102640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efecc165b80 con 0x7efecc1646f0 2023-11-06T02:09:43.282 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.284+0000 7efed3102640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efecc165600 con 0x7efecc0b0790 2023-11-06T02:09:43.283 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.284+0000 7efed3102640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efecc1658a0 con 0x7efecc0c8c50 2023-11-06T02:09:43.283 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.284+0000 7efecbfff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7efecc0b0790 0x7efecc0cc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:43.283 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.284+0000 7efecbfff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7efecc0b0790 0x7efecc0cc990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36552/0 (socket says 172.21.15.49:36552) 2023-11-06T02:09:43.283 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.284+0000 7efecbfff640 1 -- 172.21.15.49:0/1042032724 learned_addr learned my addr 172.21.15.49:0/1042032724 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:43.283 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.284+0000 7efed169f640 1 --2- 172.21.15.49:0/1042032724 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efecc0c8c50 0x7efecc0cced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:43.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.284+0000 7efecbfff640 1 -- 172.21.15.49:0/1042032724 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efecc0c8c50 msgr2=0x7efecc0cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:43.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.284+0000 7efecbfff640 1 --2- 172.21.15.49:0/1042032724 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efecc0c8c50 0x7efecc0cced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:43.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.284+0000 7efecbfff640 1 -- 172.21.15.49:0/1042032724 >> v1:172.21.15.150:6789/0 conn(0x7efecc1646f0 legacy=0x7efecc0b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:43.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.284+0000 7efecbfff640 1 -- 172.21.15.49:0/1042032724 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efecc0c96f0 con 0x7efecc0b0790 2023-11-06T02:09:43.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.284+0000 7efecbfff640 1 --2- 172.21.15.49:0/1042032724 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efecc0b0790 0x7efecc0cc990 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7efebc009870 tx=0x7efebc01c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b599910413c1b2a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:43.286 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.284+0000 7efecb7fe640 1 -- 172.21.15.49:0/1042032724 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efebc0aa440 con 0x7efecc0b0790 2023-11-06T02:09:43.286 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.284+0000 7efecb7fe640 1 -- 172.21.15.49:0/1042032724 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efebc0aa5a0 con 0x7efecc0b0790 2023-11-06T02:09:43.286 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.284+0000 7efecb7fe640 1 -- 172.21.15.49:0/1042032724 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efebc0aa930 con 0x7efecc0b0790 2023-11-06T02:09:43.286 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.284+0000 7efed3102640 1 -- 172.21.15.49:0/1042032724 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efecc0b0790 msgr2=0x7efecc0cc990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:43.286 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.284+0000 7efed3102640 1 --2- 172.21.15.49:0/1042032724 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efecc0b0790 0x7efecc0cc990 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7efebc009870 tx=0x7efebc01c500 comp rx=0 tx=0).stop 2023-11-06T02:09:43.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.284+0000 7efed3102640 1 -- 172.21.15.49:0/1042032724 shutdown_connections 2023-11-06T02:09:43.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.284+0000 7efed3102640 1 --2- 172.21.15.49:0/1042032724 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efecc0c8c50 0x7efecc0cced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:43.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.284+0000 7efed3102640 1 --2- 172.21.15.49:0/1042032724 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efecc0b0790 0x7efecc0cc990 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:43.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.284+0000 7efed3102640 1 -- 172.21.15.49:0/1042032724 >> 172.21.15.49:0/1042032724 conn(0x7efecc06c0a0 msgr2=0x7efecc06c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:43.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.284+0000 7efed3102640 1 -- 172.21.15.49:0/1042032724 shutdown_connections 2023-11-06T02:09:43.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.284+0000 7efed3102640 1 -- 172.21.15.49:0/1042032724 wait complete. 2023-11-06T02:09:43.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.288+0000 7efed3102640 1 Processor -- start 2023-11-06T02:09:43.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.288+0000 7efed3102640 1 -- start start 2023-11-06T02:09:43.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.288+0000 7efed3102640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efecc0b0790 0x7efecc15fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:43.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.288+0000 7efed3102640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efecc0c8c50 0x7efecc1586a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:43.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.288+0000 7efed3102640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efecc1646f0 0x7efecc1577d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:43.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.288+0000 7efed3102640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efecc158ed0 con 0x7efecc0b0790 2023-11-06T02:09:43.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.288+0000 7efed3102640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efecc1591b0 con 0x7efecc1646f0 2023-11-06T02:09:43.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.288+0000 7efed3102640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efecc159490 con 0x7efecc0c8c50 2023-11-06T02:09:43.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.288+0000 7efed0e9e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efecc0b0790 0x7efecc15fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:43.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.288+0000 7efed0e9e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efecc0b0790 0x7efecc15fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36556/0 (socket says 172.21.15.49:36556) 2023-11-06T02:09:43.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.288+0000 7efed0e9e640 1 -- 172.21.15.49:0/859386044 learned_addr learned my addr 172.21.15.49:0/859386044 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:43.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.288+0000 7efecbfff640 1 --2- 172.21.15.49:0/859386044 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efecc0c8c50 0x7efecc1586a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:43.293 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.288+0000 7efed169f640 1 --2- 172.21.15.49:0/859386044 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efecc1646f0 0x7efecc1577d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:43.293 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.288+0000 7efed0e9e640 1 -- 172.21.15.49:0/859386044 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efecc0c8c50 msgr2=0x7efecc1586a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:43.293 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.288+0000 7efed0e9e640 1 --2- 172.21.15.49:0/859386044 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efecc0c8c50 0x7efecc1586a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:43.293 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.288+0000 7efed0e9e640 1 -- 172.21.15.49:0/859386044 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efecc1646f0 msgr2=0x7efecc1577d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:43.293 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.288+0000 7efed0e9e640 1 --2- 172.21.15.49:0/859386044 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efecc1646f0 0x7efecc1577d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:43.294 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.288+0000 7efed0e9e640 1 -- 172.21.15.49:0/859386044 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efebc0094d0 con 0x7efecc0b0790 2023-11-06T02:09:43.294 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.288+0000 7efed169f640 1 --2- 172.21.15.49:0/859386044 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efecc1646f0 0x7efecc1577d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:09:43.294 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.288+0000 7efecbfff640 1 --2- 172.21.15.49:0/859386044 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efecc0c8c50 0x7efecc1586a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:09:43.294 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.288+0000 7efed0e9e640 1 --2- 172.21.15.49:0/859386044 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efecc0b0790 0x7efecc15fc10 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7efec000b3f0 tx=0x7efec000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:43.294 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.288+0000 7efec97fa640 1 -- 172.21.15.49:0/859386044 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec0011830 con 0x7efecc0b0790 2023-11-06T02:09:43.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.288+0000 7efec97fa640 1 -- 172.21.15.49:0/859386044 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efec00119b0 con 0x7efecc0b0790 2023-11-06T02:09:43.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.288+0000 7efed3102640 1 -- 172.21.15.49:0/859386044 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efecc159770 con 0x7efecc0b0790 2023-11-06T02:09:43.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.288+0000 7efec97fa640 1 -- 172.21.15.49:0/859386044 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec0011d60 con 0x7efecc0b0790 2023-11-06T02:09:43.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.288+0000 7efed3102640 1 -- 172.21.15.49:0/859386044 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efecc158be0 con 0x7efecc0b0790 2023-11-06T02:09:43.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.288+0000 7efed3102640 1 -- 172.21.15.49:0/859386044 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efecc074040 con 0x7efecc0b0790 2023-11-06T02:09:43.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.292+0000 7efec97fa640 1 -- 172.21.15.49:0/859386044 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7efec001a410 con 0x7efecc0b0790 2023-11-06T02:09:43.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.292+0000 7efec97fa640 1 --2- 172.21.15.49:0/859386044 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7efeb8065a60 0x7efeb8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:43.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.292+0000 7efec97fa640 1 -- 172.21.15.49:0/859386044 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7efec0083160 con 0x7efecc0b0790 2023-11-06T02:09:43.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.292+0000 7efecbfff640 1 --2- 172.21.15.49:0/859386044 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7efeb8065a60 0x7efeb8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:43.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.292+0000 7efecbfff640 1 --2- 172.21.15.49:0/859386044 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7efeb8065a60 0x7efeb8067f20 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7efebc0ae000 tx=0x7efebc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:43.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.296+0000 7efec97fa640 1 -- 172.21.15.49:0/859386044 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7efec00509c0 con 0x7efecc0b0790 2023-11-06T02:09:43.448 INFO:tasks.ceph.mds.a.smithi049.stderr:2023-11-06T02:09:43.448+0000 7f5fe4f76640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-06T02:09:43.487 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.488+0000 7efed3102640 1 -- 172.21.15.49:0/859386044 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efecc159c20 con 0x7efecc0b0790 2023-11-06T02:09:43.488 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.488+0000 7efec97fa640 1 -- 172.21.15.49:0/859386044 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4651 (secure 0 0 0) 0x7efec0053fe0 con 0x7efecc0b0790 2023-11-06T02:09:43.488 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:09:43.489 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5032,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/280428805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":280428805},{"type":"v1","addr":"172.21.15.150:6835","nonce":280428805}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":5035,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/2734589844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2734589844},{"type":"v1","addr":"172.21.15.49:6833","nonce":2734589844}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":5038,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/1319221246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":1319221246},{"type":"v1","addr":"172.21.15.150:6837","nonce":1319221246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T02:09:39.873514+0000","modified":"2023-11-06T02:09:43.132890+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5041},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5041":{"gid":5041,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.49:6835/1547192091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":1547192091},{"type":"v1","addr":"172.21.15.49:6835","nonce":1547192091}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-06T02:09:43.489 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 15 2023-11-06T02:09:43.490 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.492+0000 7efed3102640 1 -- 172.21.15.49:0/859386044 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7efeb8065a60 msgr2=0x7efeb8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:43.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.492+0000 7efed3102640 1 --2- 172.21.15.49:0/859386044 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7efeb8065a60 0x7efeb8067f20 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7efebc0ae000 tx=0x7efebc008040 comp rx=0 tx=0).stop 2023-11-06T02:09:43.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.492+0000 7efed3102640 1 -- 172.21.15.49:0/859386044 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efecc0b0790 msgr2=0x7efecc15fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:43.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.492+0000 7efed3102640 1 --2- 172.21.15.49:0/859386044 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efecc0b0790 0x7efecc15fc10 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7efec000b3f0 tx=0x7efec000b8b0 comp rx=0 tx=0).stop 2023-11-06T02:09:43.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.492+0000 7efed3102640 1 -- 172.21.15.49:0/859386044 shutdown_connections 2023-11-06T02:09:43.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.492+0000 7efed3102640 1 --2- 172.21.15.49:0/859386044 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7efeb8065a60 0x7efeb8067f20 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:43.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.492+0000 7efed3102640 1 --2- 172.21.15.49:0/859386044 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efecc1646f0 0x7efecc1577d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:43.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.492+0000 7efed3102640 1 --2- 172.21.15.49:0/859386044 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efecc0c8c50 0x7efecc1586a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:43.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.492+0000 7efed3102640 1 --2- 172.21.15.49:0/859386044 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efecc0b0790 0x7efecc15fc10 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:43.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.492+0000 7efed3102640 1 -- 172.21.15.49:0/859386044 >> 172.21.15.49:0/859386044 conn(0x7efecc06c0a0 msgr2=0x7efecc06c480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:43.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.492+0000 7efed3102640 1 -- 172.21.15.49:0/859386044 shutdown_connections 2023-11-06T02:09:43.493 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.492+0000 7efed3102640 1 -- 172.21.15.49:0/859386044 wait complete. 2023-11-06T02:09:43.507 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T02:09:43.630 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.632+0000 7fcebf41c640 1 Processor -- start 2023-11-06T02:09:43.630 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.632+0000 7fcebf41c640 1 -- start start 2023-11-06T02:09:43.632 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.632+0000 7fcebf41c640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fceb80cb010 0x7fceb80c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:43.633 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.632+0000 7fcebf41c640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fceb80cab80 con 0x7fceb815ab50 2023-11-06T02:09:43.633 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.632+0000 7fcebf41c640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fceb815af30 con 0x7fceb80b0790 2023-11-06T02:09:43.633 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.632+0000 7fcebf41c640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fceb80cb5b0 con 0x7fceb80cb010 2023-11-06T02:09:43.633 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.632+0000 7fcebd1b8640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fceb815ab50 0x7fceb80b0080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:48166/0 (socket says 172.21.15.49:48166) 2023-11-06T02:09:43.633 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.632+0000 7fcebd1b8640 1 -- 172.21.15.49:0/287782736 learned_addr learned my addr 172.21.15.49:0/287782736 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:09:43.634 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.632+0000 7fcebd9b9640 1 --2- 172.21.15.49:0/287782736 >> v2:172.21.15.150:3300/0 conn(0x7fceb80cb010 0x7fceb80c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:43.634 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.632+0000 7fceaffff640 1 -- 172.21.15.49:0/287782736 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3382063396 0 0) 0x7fceb80cab80 con 0x7fceb815ab50 2023-11-06T02:09:43.634 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.632+0000 7fceaffff640 1 -- 172.21.15.49:0/287782736 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcea4003530 con 0x7fceb815ab50 2023-11-06T02:09:43.634 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.632+0000 7fceaffff640 1 -- 172.21.15.49:0/287782736 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3075823690 0 0) 0x7fcea4003530 con 0x7fceb815ab50 2023-11-06T02:09:43.634 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.632+0000 7fceaffff640 1 -- 172.21.15.49:0/287782736 >> v2:172.21.15.150:3300/0 conn(0x7fceb80cb010 msgr2=0x7fceb80c9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:43.635 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.632+0000 7fceaffff640 1 --2- 172.21.15.49:0/287782736 >> v2:172.21.15.150:3300/0 conn(0x7fceb80cb010 0x7fceb80c9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:43.635 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.632+0000 7fceaffff640 1 -- 172.21.15.49:0/287782736 >> v1:172.21.15.150:6789/0 conn(0x7fceb80b0790 legacy=0x7fceb80c2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:43.635 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.632+0000 7fceaffff640 1 -- 172.21.15.49:0/287782736 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fceb80cb890 con 0x7fceb815ab50 2023-11-06T02:09:43.635 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.632+0000 7fceaffff640 1 -- 172.21.15.49:0/287782736 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7fceb4001800 con 0x7fceb815ab50 2023-11-06T02:09:43.635 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.632+0000 7fceaffff640 1 -- 172.21.15.49:0/287782736 >> v1:172.21.15.49:6789/0 conn(0x7fceb815ab50 legacy=0x7fceb80b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:43.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.632+0000 7fceaffff640 1 --2- 172.21.15.49:0/287782736 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcea4003980 0x7fcea4003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:43.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.632+0000 7fceaffff640 1 --2- 172.21.15.49:0/287782736 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcea4004380 0x7fcea4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:43.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.632+0000 7fceaffff640 1 --2- 172.21.15.49:0/287782736 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcea4005050 0x7fcea400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:43.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.632+0000 7fceaffff640 1 -- 172.21.15.49:0/287782736 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fceb80cb5b0 con 0x7fcea4003980 2023-11-06T02:09:43.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.632+0000 7fceaffff640 1 -- 172.21.15.49:0/287782736 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcea400bfb0 con 0x7fcea4005050 2023-11-06T02:09:43.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.632+0000 7fceaffff640 1 -- 172.21.15.49:0/287782736 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcea400c170 con 0x7fcea4004380 2023-11-06T02:09:43.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.632+0000 7fcebd1b8640 1 --2- 172.21.15.49:0/287782736 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcea4003980 0x7fcea4003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:43.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.632+0000 7fcebd9b9640 1 --2- 172.21.15.49:0/287782736 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcea4005050 0x7fcea400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:43.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.632+0000 7fcebc9b7640 1 --2- 172.21.15.49:0/287782736 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcea4004380 0x7fcea4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:43.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.632+0000 7fcebd1b8640 1 -- 172.21.15.49:0/287782736 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcea4004380 msgr2=0x7fcea4004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:43.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.632+0000 7fcebd1b8640 1 --2- 172.21.15.49:0/287782736 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcea4004380 0x7fcea4004850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:43.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.632+0000 7fcebd1b8640 1 -- 172.21.15.49:0/287782736 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcea4005050 msgr2=0x7fcea400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:43.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.632+0000 7fcebd1b8640 1 --2- 172.21.15.49:0/287782736 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcea4005050 0x7fcea400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:43.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.632+0000 7fcebd1b8640 1 -- 172.21.15.49:0/287782736 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcea4003530 con 0x7fcea4003980 2023-11-06T02:09:43.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.632+0000 7fcebd9b9640 1 --2- 172.21.15.49:0/287782736 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcea4005050 0x7fcea400b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:09:43.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.632+0000 7fcebd9b9640 1 -- 172.21.15.49:0/287782736 reap_dead start 2023-11-06T02:09:43.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.636+0000 7fcebd1b8640 1 --2- 172.21.15.49:0/287782736 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcea4003980 0x7fcea4003d60 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fceb400c1c0 tx=0x7fceb401cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5e027d5a7307b8d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:43.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.636+0000 7fceaffff640 1 -- 172.21.15.49:0/287782736 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceb4007270 con 0x7fcea4003980 2023-11-06T02:09:43.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.636+0000 7fceaffff640 1 -- 172.21.15.49:0/287782736 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fceb40073d0 con 0x7fcea4003980 2023-11-06T02:09:43.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.636+0000 7fceaffff640 1 -- 172.21.15.49:0/287782736 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceb40057b0 con 0x7fcea4003980 2023-11-06T02:09:43.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.636+0000 7fcebf41c640 1 -- 172.21.15.49:0/287782736 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcea4003980 msgr2=0x7fcea4003d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:43.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.636+0000 7fcebf41c640 1 --2- 172.21.15.49:0/287782736 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcea4003980 0x7fcea4003d60 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fceb400c1c0 tx=0x7fceb401cb60 comp rx=0 tx=0).stop 2023-11-06T02:09:43.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.636+0000 7fcebf41c640 1 -- 172.21.15.49:0/287782736 shutdown_connections 2023-11-06T02:09:43.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.636+0000 7fcebf41c640 1 --2- 172.21.15.49:0/287782736 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcea4003980 0x7fcea4003d60 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:43.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.636+0000 7fcebf41c640 1 -- 172.21.15.49:0/287782736 >> 172.21.15.49:0/287782736 conn(0x7fceb806c0a0 msgr2=0x7fceb806c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:43.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.636+0000 7fcebf41c640 1 -- 172.21.15.49:0/287782736 shutdown_connections 2023-11-06T02:09:43.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.636+0000 7fcebf41c640 1 -- 172.21.15.49:0/287782736 wait complete. 2023-11-06T02:09:43.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.636+0000 7fcebf41c640 1 Processor -- start 2023-11-06T02:09:43.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.636+0000 7fcebf41c640 1 -- start start 2023-11-06T02:09:43.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.636+0000 7fcebf41c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcea4003980 0x7fceb81566c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:43.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.636+0000 7fcebf41c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fceb8156ed0 0x7fceb80c41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:43.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.636+0000 7fcebf41c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fceb80c1530 0x7fceb80c4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:43.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.636+0000 7fcebf41c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fceb80a8530 con 0x7fceb80c1530 2023-11-06T02:09:43.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.636+0000 7fcebf41c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fceb80a7bb0 con 0x7fceb8156ed0 2023-11-06T02:09:43.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.636+0000 7fcebf41c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fceb80a78d0 con 0x7fcea4003980 2023-11-06T02:09:43.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.640+0000 7fcebd9b9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fceb80c1530 0x7fceb80c4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:43.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.640+0000 7fcebd9b9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fceb80c1530 0x7fceb80c4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36576/0 (socket says 172.21.15.49:36576) 2023-11-06T02:09:43.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.640+0000 7fcebd9b9640 1 -- 172.21.15.49:0/1338673358 learned_addr learned my addr 172.21.15.49:0/1338673358 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:43.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.640+0000 7fcebd1b8640 1 --2- 172.21.15.49:0/1338673358 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcea4003980 0x7fceb81566c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:43.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.640+0000 7fcebc9b7640 1 --2- 172.21.15.49:0/1338673358 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fceb8156ed0 0x7fceb80c41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:43.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.640+0000 7fcebd9b9640 1 -- 172.21.15.49:0/1338673358 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcea4003980 msgr2=0x7fceb81566c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:43.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.640+0000 7fcebd9b9640 1 --2- 172.21.15.49:0/1338673358 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcea4003980 0x7fceb81566c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:43.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.640+0000 7fcebd9b9640 1 -- 172.21.15.49:0/1338673358 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fceb8156ed0 msgr2=0x7fceb80c41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:43.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.640+0000 7fcebd9b9640 1 --2- 172.21.15.49:0/1338673358 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fceb8156ed0 0x7fceb80c41d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:43.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.640+0000 7fcebd9b9640 1 -- 172.21.15.49:0/1338673358 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fceb80a7ee0 con 0x7fceb80c1530 2023-11-06T02:09:43.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.640+0000 7fcebc9b7640 1 --2- 172.21.15.49:0/1338673358 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fceb8156ed0 0x7fceb80c41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:09:43.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.640+0000 7fcebd9b9640 1 --2- 172.21.15.49:0/1338673358 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fceb80c1530 0x7fceb80c4710 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fceb80c5150 tx=0x7fceb000f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:43.648 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.640+0000 7fceadffb640 1 -- 172.21.15.49:0/1338673358 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceb0016620 con 0x7fceb80c1530 2023-11-06T02:09:43.648 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.640+0000 7fceadffb640 1 -- 172.21.15.49:0/1338673358 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fceb0016780 con 0x7fceb80c1530 2023-11-06T02:09:43.648 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.640+0000 7fcebf41c640 1 -- 172.21.15.49:0/1338673358 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fceb80a81c0 con 0x7fceb80c1530 2023-11-06T02:09:43.648 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.640+0000 7fceadffb640 1 -- 172.21.15.49:0/1338673358 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceb0016af0 con 0x7fceb80c1530 2023-11-06T02:09:43.648 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.640+0000 7fcebf41c640 1 -- 172.21.15.49:0/1338673358 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fceb80a74b0 con 0x7fceb80c1530 2023-11-06T02:09:43.648 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.640+0000 7fcebf41c640 1 -- 172.21.15.49:0/1338673358 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fceb80c60e0 con 0x7fceb80c1530 2023-11-06T02:09:43.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.640+0000 7fceadffb640 1 -- 172.21.15.49:0/1338673358 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fceb0016c50 con 0x7fceb80c1530 2023-11-06T02:09:43.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.640+0000 7fceadffb640 1 --2- 172.21.15.49:0/1338673358 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fce9c065a60 0x7fce9c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:43.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.640+0000 7fceadffb640 1 -- 172.21.15.49:0/1338673358 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fceb0083fe0 con 0x7fceb80c1530 2023-11-06T02:09:43.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.640+0000 7fcebd1b8640 1 --2- 172.21.15.49:0/1338673358 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fce9c065a60 0x7fce9c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:43.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.640+0000 7fcebd1b8640 1 --2- 172.21.15.49:0/1338673358 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fce9c065a60 0x7fce9c067f20 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fceb40ab000 tx=0x7fceb40a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:43.650 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.644+0000 7fceadffb640 1 -- 172.21.15.49:0/1338673358 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fceb00518d0 con 0x7fceb80c1530 2023-11-06T02:09:43.800 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.800+0000 7fcebf41c640 1 -- 172.21.15.49:0/1338673358 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fceb80b0de0 con 0x7fceb80c1530 2023-11-06T02:09:43.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.804+0000 7fceadffb640 1 -- 172.21.15.49:0/1338673358 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+17883 (secure 0 0 0) 0x7fceb0054ef0 con 0x7fceb80c1530 2023-11-06T02:09:43.802 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:09:43.803 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":37,"fsid":"1a8b035c-6ba4-499a-b876-aea47fd9c0d4","created":"2023-11-06T02:08:06.440857+0000","modified":"2023-11-06T02:09:40.876751+0000","last_up_change":"2023-11-06T02:08:17.982372+0000","last_in_change":"2023-11-06T02:08:08.906631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T02:08:19.252222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T02:08:24.776211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-06T02:09:37.906848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-06T02:09:38.849380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ad55af8-0571-4e6a-8722-b5d429b7496d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6809","nonce":38116},{"type":"v1","addr":"172.21.15.49:6811","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6813","nonce":38116},{"type":"v1","addr":"172.21.15.49:6815","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6821","nonce":38116},{"type":"v1","addr":"172.21.15.49:6823","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6817","nonce":38116},{"type":"v1","addr":"172.21.15.49:6819","nonce":38116}]},"public_addr":"172.21.15.49:6811/38116","cluster_addr":"172.21.15.49:6815/38116","heartbeat_back_addr":"172.21.15.49:6823/38116","heartbeat_front_addr":"172.21.15.49:6819/38116","state":["exists","up"]},{"osd":1,"uuid":"c8b31f85-a757-43ed-9419-351acc46611b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":38113},{"type":"v1","addr":"172.21.15.49:6810","nonce":38113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":38113},{"type":"v1","addr":"172.21.15.49:6814","nonce":38113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":38113},{"type":"v1","addr":"172.21.15.49:6822","nonce":38113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":38113},{"type":"v1","addr":"172.21.15.49:6818","nonce":38113}]},"public_addr":"172.21.15.49:6810/38113","cluster_addr":"172.21.15.49:6814/38113","heartbeat_back_addr":"172.21.15.49:6822/38113","heartbeat_front_addr":"172.21.15.49:6818/38113","state":["exists","up"]},{"osd":2,"uuid":"555b3bf3-0a70-4b91-8efd-e4204b5fc26e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":38114},{"type":"v1","addr":"172.21.15.49:6801","nonce":38114}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":38114},{"type":"v1","addr":"172.21.15.49:6803","nonce":38114}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":38114},{"type":"v1","addr":"172.21.15.49:6807","nonce":38114}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":38114},{"type":"v1","addr":"172.21.15.49:6805","nonce":38114}]},"public_addr":"172.21.15.49:6801/38114","cluster_addr":"172.21.15.49:6803/38114","heartbeat_back_addr":"172.21.15.49:6807/38114","heartbeat_front_addr":"172.21.15.49:6805/38114","state":["exists","up"]},{"osd":3,"uuid":"8d4e5d77-03ef-4b9b-bfcb-5ae93a46795c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":38115},{"type":"v1","addr":"172.21.15.49:6825","nonce":38115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":38115},{"type":"v1","addr":"172.21.15.49:6827","nonce":38115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":38115},{"type":"v1","addr":"172.21.15.49:6831","nonce":38115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":38115},{"type":"v1","addr":"172.21.15.49:6829","nonce":38115}]},"public_addr":"172.21.15.49:6825/38115","cluster_addr":"172.21.15.49:6827/38115","heartbeat_back_addr":"172.21.15.49:6831/38115","heartbeat_front_addr":"172.21.15.49:6829/38115","state":["exists","up"]},{"osd":4,"uuid":"dba03ad4-68a3-4cf3-9218-ac611f02eacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":38417},{"type":"v1","addr":"172.21.15.150:6809","nonce":38417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":38417},{"type":"v1","addr":"172.21.15.150:6811","nonce":38417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":38417},{"type":"v1","addr":"172.21.15.150:6815","nonce":38417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":38417},{"type":"v1","addr":"172.21.15.150:6813","nonce":38417}]},"public_addr":"172.21.15.150:6809/38417","cluster_addr":"172.21.15.150:6811/38417","heartbeat_back_addr":"172.21.15.150:6815/38417","heartbeat_front_addr":"172.21.15.150:6813/38417","state":["exists","up"]},{"osd":5,"uuid":"0df9c303-c255-4640-8e91-01f1c3efe18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":38418},{"type":"v1","addr":"172.21.15.150:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":38418},{"type":"v1","addr":"172.21.15.150:6803","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":38418},{"type":"v1","addr":"172.21.15.150:6807","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":38418},{"type":"v1","addr":"172.21.15.150:6805","nonce":38418}]},"public_addr":"172.21.15.150:6801/38418","cluster_addr":"172.21.15.150:6803/38418","heartbeat_back_addr":"172.21.15.150:6807/38418","heartbeat_front_addr":"172.21.15.150:6805/38418","state":["exists","up"]},{"osd":6,"uuid":"e59fa189-4b70-4c06-9722-c02b90828bf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":38428},{"type":"v1","addr":"172.21.15.150:6825","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":38428},{"type":"v1","addr":"172.21.15.150:6827","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":38428},{"type":"v1","addr":"172.21.15.150:6831","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":38428},{"type":"v1","addr":"172.21.15.150:6829","nonce":38428}]},"public_addr":"172.21.15.150:6825/38428","cluster_addr":"172.21.15.150:6827/38428","heartbeat_back_addr":"172.21.15.150:6831/38428","heartbeat_front_addr":"172.21.15.150:6829/38428","state":["exists","up"]},{"osd":7,"uuid":"a64f57e1-ef98-46bd-b304-385759e95898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":38421},{"type":"v1","addr":"172.21.15.150:6817","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":38421},{"type":"v1","addr":"172.21.15.150:6819","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":38421},{"type":"v1","addr":"172.21.15.150:6823","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":38421},{"type":"v1","addr":"172.21.15.150:6821","nonce":38421}]},"public_addr":"172.21.15.150:6817/38421","cluster_addr":"172.21.15.150:6819/38421","heartbeat_back_addr":"172.21.15.150:6823/38421","heartbeat_front_addr":"172.21.15.150:6821/38421","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.438552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.413280+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.296466+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.424966+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.276953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.230984+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.410235+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.480084+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T02:09:43.804 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.804+0000 7fcebf41c640 1 -- 172.21.15.49:0/1338673358 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fce9c065a60 msgr2=0x7fce9c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:43.804 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.804+0000 7fcebf41c640 1 --2- 172.21.15.49:0/1338673358 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fce9c065a60 0x7fce9c067f20 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fceb40ab000 tx=0x7fceb40a7070 comp rx=0 tx=0).stop 2023-11-06T02:09:43.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.804+0000 7fcebf41c640 1 -- 172.21.15.49:0/1338673358 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fceb80c1530 msgr2=0x7fceb80c4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:43.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.804+0000 7fcebf41c640 1 --2- 172.21.15.49:0/1338673358 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fceb80c1530 0x7fceb80c4710 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fceb80c5150 tx=0x7fceb000f520 comp rx=0 tx=0).stop 2023-11-06T02:09:43.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.804+0000 7fcebf41c640 1 -- 172.21.15.49:0/1338673358 shutdown_connections 2023-11-06T02:09:43.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.804+0000 7fcebf41c640 1 --2- 172.21.15.49:0/1338673358 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fce9c065a60 0x7fce9c067f20 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:43.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.804+0000 7fcebf41c640 1 --2- 172.21.15.49:0/1338673358 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fceb80c1530 0x7fceb80c4710 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:43.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.804+0000 7fcebf41c640 1 --2- 172.21.15.49:0/1338673358 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fceb8156ed0 0x7fceb80c41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:43.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.804+0000 7fcebf41c640 1 --2- 172.21.15.49:0/1338673358 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcea4003980 0x7fceb81566c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:43.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.804+0000 7fcebf41c640 1 -- 172.21.15.49:0/1338673358 >> 172.21.15.49:0/1338673358 conn(0x7fceb806c0a0 msgr2=0x7fceb80708b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:43.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.804+0000 7fcebf41c640 1 -- 172.21.15.49:0/1338673358 shutdown_connections 2023-11-06T02:09:43.807 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.804+0000 7fcebf41c640 1 -- 172.21.15.49:0/1338673358 wait complete. 2023-11-06T02:09:43.821 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-06T02:09:43.821 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T02:09:43.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.944+0000 7faab0f49640 1 Processor -- start 2023-11-06T02:09:43.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.944+0000 7faab0f49640 1 -- start start 2023-11-06T02:09:43.948 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.948+0000 7faab0f49640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faaac15ab80 0x7faaac06f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:43.948 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.948+0000 7faab0f49640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faaac15f4d0 con 0x7faaac0cd5d0 2023-11-06T02:09:43.948 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.948+0000 7faab0f49640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faaac0719e0 con 0x7faaac071240 2023-11-06T02:09:43.948 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.948+0000 7faab0f49640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faaac15af60 con 0x7faaac15ab80 2023-11-06T02:09:43.949 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.948+0000 7faaaad9d640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7faaac0cd5d0 0x7faaac0b2520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:48180/0 (socket says 172.21.15.49:48180) 2023-11-06T02:09:43.949 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.948+0000 7faaaad9d640 1 -- 172.21.15.49:0/2026959410 learned_addr learned my addr 172.21.15.49:0/2026959410 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:09:43.949 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.948+0000 7faaaa59c640 1 --2- 172.21.15.49:0/2026959410 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faaac15ab80 0x7faaac06f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:43.949 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.948+0000 7faaa959a640 1 -- 172.21.15.49:0/2026959410 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3912904801 0 0) 0x7faaac15f4d0 con 0x7faaac0cd5d0 2023-11-06T02:09:43.949 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.948+0000 7faaa959a640 1 -- 172.21.15.49:0/2026959410 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa98003530 con 0x7faaac0cd5d0 2023-11-06T02:09:43.950 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.948+0000 7faaa959a640 1 -- 172.21.15.49:0/2026959410 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1114935252 0 0) 0x7faa98003530 con 0x7faaac0cd5d0 2023-11-06T02:09:43.950 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.948+0000 7faaa959a640 1 -- 172.21.15.49:0/2026959410 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faaac15ab80 msgr2=0x7faaac06f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:43.950 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.948+0000 7faaa959a640 1 --2- 172.21.15.49:0/2026959410 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faaac15ab80 0x7faaac06f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:43.950 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.948+0000 7faaa959a640 1 -- 172.21.15.49:0/2026959410 >> v1:172.21.15.150:6789/0 conn(0x7faaac071240 legacy=0x7faaac15edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:43.950 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.948+0000 7faaa959a640 1 -- 172.21.15.49:0/2026959410 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaac071cc0 con 0x7faaac0cd5d0 2023-11-06T02:09:43.951 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.948+0000 7faaa959a640 1 -- 172.21.15.49:0/2026959410 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7faaa4001680 con 0x7faaac0cd5d0 2023-11-06T02:09:43.951 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.948+0000 7faaa959a640 1 -- 172.21.15.49:0/2026959410 >> v1:172.21.15.49:6789/0 conn(0x7faaac0cd5d0 legacy=0x7faaac0b2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:43.951 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.948+0000 7faaa959a640 1 --2- 172.21.15.49:0/2026959410 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa980039b0 0x7faa98003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:43.951 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.948+0000 7faaa959a640 1 --2- 172.21.15.49:0/2026959410 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faaac15ab80 0x7faa980047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:43.951 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.948+0000 7faaa959a640 1 --2- 172.21.15.49:0/2026959410 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa98004d10 0x7faa98005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:43.952 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.948+0000 7faaa959a640 1 -- 172.21.15.49:0/2026959410 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaac15af60 con 0x7faa98004d10 2023-11-06T02:09:43.952 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.948+0000 7faaa959a640 1 -- 172.21.15.49:0/2026959410 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa9800bac0 con 0x7faa980039b0 2023-11-06T02:09:43.952 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.948+0000 7faaa959a640 1 -- 172.21.15.49:0/2026959410 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa9800bc80 con 0x7faaac15ab80 2023-11-06T02:09:43.952 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.948+0000 7faaaad9d640 1 --2- 172.21.15.49:0/2026959410 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa98004d10 0x7faa98005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:43.952 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.948+0000 7faaaa59c640 1 --2- 172.21.15.49:0/2026959410 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa980039b0 0x7faa98003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:43.952 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.948+0000 7faaa9d9b640 1 --2- 172.21.15.49:0/2026959410 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faaac15ab80 0x7faa980047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:43.953 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.948+0000 7faaaad9d640 1 -- 172.21.15.49:0/2026959410 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faaac15ab80 msgr2=0x7faa980047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:43.953 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.948+0000 7faaaad9d640 1 --2- 172.21.15.49:0/2026959410 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faaac15ab80 0x7faa980047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:43.953 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.948+0000 7faaaad9d640 1 -- 172.21.15.49:0/2026959410 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa980039b0 msgr2=0x7faa98003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:43.953 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.948+0000 7faaaad9d640 1 --2- 172.21.15.49:0/2026959410 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa980039b0 0x7faa98003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:43.953 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.948+0000 7faaaad9d640 1 -- 172.21.15.49:0/2026959410 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa98003530 con 0x7faa98004d10 2023-11-06T02:09:43.954 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.948+0000 7faaa9d9b640 1 --2- 172.21.15.49:0/2026959410 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faaac15ab80 0x7faa980047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:09:43.957 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.952+0000 7faaaad9d640 1 --2- 172.21.15.49:0/2026959410 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa98004d10 0x7faa98005190 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7faaa400c240 tx=0x7faaa401f3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81c1c1a5a81bfafb server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:43.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.952+0000 7faaa959a640 1 -- 172.21.15.49:0/2026959410 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa4005ae0 con 0x7faa98004d10 2023-11-06T02:09:43.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.952+0000 7faaa959a640 1 -- 172.21.15.49:0/2026959410 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaa40ab070 con 0x7faa98004d10 2023-11-06T02:09:43.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.952+0000 7faaa959a640 1 -- 172.21.15.49:0/2026959410 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa4005ea0 con 0x7faa98004d10 2023-11-06T02:09:43.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.952+0000 7faab0f49640 1 -- 172.21.15.49:0/2026959410 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa98004d10 msgr2=0x7faa98005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:43.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.952+0000 7faab0f49640 1 --2- 172.21.15.49:0/2026959410 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa98004d10 0x7faa98005190 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7faaa400c240 tx=0x7faaa401f3b0 comp rx=0 tx=0).stop 2023-11-06T02:09:43.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.952+0000 7faaaad9d640 1 -- 172.21.15.49:0/2026959410 reap_dead start 2023-11-06T02:09:43.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.952+0000 7faab0f49640 1 -- 172.21.15.49:0/2026959410 shutdown_connections 2023-11-06T02:09:43.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.952+0000 7faab0f49640 1 -- 172.21.15.49:0/2026959410 >> 172.21.15.49:0/2026959410 conn(0x7faaac0cc150 msgr2=0x7faaac0cc560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:43.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.952+0000 7faab0f49640 1 -- 172.21.15.49:0/2026959410 shutdown_connections 2023-11-06T02:09:43.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.952+0000 7faab0f49640 1 -- 172.21.15.49:0/2026959410 wait complete. 2023-11-06T02:09:43.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.952+0000 7faab0f49640 1 Processor -- start 2023-11-06T02:09:43.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.952+0000 7faab0f49640 1 -- start start 2023-11-06T02:09:43.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.952+0000 7faab0f49640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faaac0c5e00 0x7faaac0c6b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:43.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.952+0000 7faab0f49640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faaac0c83a0 0x7faaac149800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:43.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.952+0000 7faab0f49640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faaac0c8820 0x7faaac149d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:43.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.952+0000 7faab0f49640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaac0ae370 con 0x7faaac0c5e00 2023-11-06T02:09:43.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.952+0000 7faab0f49640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaac0c61e0 con 0x7faaac0c83a0 2023-11-06T02:09:43.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.952+0000 7faab0f49640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faaac0c8cd0 con 0x7faaac0c8820 2023-11-06T02:09:43.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.952+0000 7faaaa59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faaac0c5e00 0x7faaac0c6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:43.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.952+0000 7faaaa59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faaac0c5e00 0x7faaac0c6b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36592/0 (socket says 172.21.15.49:36592) 2023-11-06T02:09:43.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.952+0000 7faaaa59c640 1 -- 172.21.15.49:0/3726366504 learned_addr learned my addr 172.21.15.49:0/3726366504 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:43.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.952+0000 7faaa9d9b640 1 --2- 172.21.15.49:0/3726366504 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faaac0c83a0 0x7faaac149800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:43.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.956+0000 7faaaad9d640 1 --2- 172.21.15.49:0/3726366504 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faaac0c8820 0x7faaac149d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:43.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.956+0000 7faaaa59c640 1 -- 172.21.15.49:0/3726366504 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faaac0c8820 msgr2=0x7faaac149d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:43.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.956+0000 7faaaa59c640 1 --2- 172.21.15.49:0/3726366504 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faaac0c8820 0x7faaac149d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:43.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.956+0000 7faaaa59c640 1 -- 172.21.15.49:0/3726366504 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faaac0c83a0 msgr2=0x7faaac149800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:43.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.956+0000 7faaaa59c640 1 --2- 172.21.15.49:0/3726366504 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faaac0c83a0 0x7faaac149800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:43.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.956+0000 7faaaa59c640 1 -- 172.21.15.49:0/3726366504 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaac0c8fb0 con 0x7faaac0c5e00 2023-11-06T02:09:43.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.956+0000 7faaaad9d640 1 --2- 172.21.15.49:0/3726366504 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faaac0c8820 0x7faaac149d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:09:43.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.956+0000 7faaaa59c640 1 --2- 172.21.15.49:0/3726366504 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faaac0c5e00 0x7faaac0c6b80 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7faa9c0119d0 tx=0x7faa9c011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:43.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.956+0000 7faaa2ffd640 1 -- 172.21.15.49:0/3726366504 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa9c00fb50 con 0x7faaac0c5e00 2023-11-06T02:09:43.964 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.956+0000 7faaa2ffd640 1 -- 172.21.15.49:0/3726366504 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa9c00fcb0 con 0x7faaac0c5e00 2023-11-06T02:09:43.964 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.956+0000 7faab0f49640 1 -- 172.21.15.49:0/3726366504 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faaac0c9290 con 0x7faaac0c5e00 2023-11-06T02:09:43.964 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.956+0000 7faaa2ffd640 1 -- 172.21.15.49:0/3726366504 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa9c014440 con 0x7faaac0c5e00 2023-11-06T02:09:43.964 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.956+0000 7faab0f49640 1 -- 172.21.15.49:0/3726366504 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faaac1f53d0 con 0x7faaac0c5e00 2023-11-06T02:09:43.964 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.956+0000 7faab0f49640 1 -- 172.21.15.49:0/3726366504 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faaac0c9c10 con 0x7faaac0c5e00 2023-11-06T02:09:43.964 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.956+0000 7faaa2ffd640 1 -- 172.21.15.49:0/3726366504 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7faa9c0145a0 con 0x7faaac0c5e00 2023-11-06T02:09:43.965 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.956+0000 7faaa2ffd640 1 --2- 172.21.15.49:0/3726366504 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7faa8c065a60 0x7faa8c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:43.965 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.956+0000 7faaa2ffd640 1 -- 172.21.15.49:0/3726366504 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7faa9c083b20 con 0x7faaac0c5e00 2023-11-06T02:09:43.965 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.956+0000 7faaa9d9b640 1 --2- 172.21.15.49:0/3726366504 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7faa8c065a60 0x7faa8c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:43.965 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.956+0000 7faaa9d9b640 1 --2- 172.21.15.49:0/3726366504 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7faa8c065a60 0x7faa8c067f20 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7faaac1e50e0 tx=0x7faa94009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:43.966 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:43.960+0000 7faaa2ffd640 1 -- 172.21.15.49:0/3726366504 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7faa9c0514d0 con 0x7faaac0c5e00 2023-11-06T02:09:44.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.112+0000 7faab0f49640 1 -- 172.21.15.49:0/3726366504 --> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7faaac15e5c0 con 0x7faa8c065a60 2023-11-06T02:09:44.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.136+0000 7faaa2ffd640 1 -- 172.21.15.49:0/3726366504 <== mgr.4100 v2:172.21.15.150:6832/38104 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+294851 (secure 0 0 0) 0x7faa940057a0 con 0x7faa8c065a60 2023-11-06T02:09:44.137 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:09:44.140 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2023-11-06T02:09:44.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.140+0000 7faab0f49640 1 -- 172.21.15.49:0/3726366504 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7faa8c065a60 msgr2=0x7faa8c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:44.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.140+0000 7faab0f49640 1 --2- 172.21.15.49:0/3726366504 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7faa8c065a60 0x7faa8c067f20 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7faaac1e50e0 tx=0x7faa94009040 comp rx=0 tx=0).stop 2023-11-06T02:09:44.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.140+0000 7faab0f49640 1 -- 172.21.15.49:0/3726366504 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faaac0c5e00 msgr2=0x7faaac0c6b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:44.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.140+0000 7faab0f49640 1 --2- 172.21.15.49:0/3726366504 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faaac0c5e00 0x7faaac0c6b80 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7faa9c0119d0 tx=0x7faa9c011e90 comp rx=0 tx=0).stop 2023-11-06T02:09:44.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.140+0000 7faab0f49640 1 -- 172.21.15.49:0/3726366504 shutdown_connections 2023-11-06T02:09:44.143 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.140+0000 7faab0f49640 1 --2- 172.21.15.49:0/3726366504 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7faa8c065a60 0x7faa8c067f20 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:44.143 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.140+0000 7faab0f49640 1 --2- 172.21.15.49:0/3726366504 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faaac0c8820 0x7faaac149d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:44.143 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.140+0000 7faab0f49640 1 --2- 172.21.15.49:0/3726366504 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faaac0c83a0 0x7faaac149800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:44.143 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.140+0000 7faab0f49640 1 --2- 172.21.15.49:0/3726366504 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faaac0c5e00 0x7faaac0c6b80 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:44.143 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.140+0000 7faab0f49640 1 -- 172.21.15.49:0/3726366504 >> 172.21.15.49:0/3726366504 conn(0x7faaac0cc150 msgr2=0x7faaac0af670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:44.144 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.140+0000 7faab0f49640 1 -- 172.21.15.49:0/3726366504 shutdown_connections 2023-11-06T02:09:44.144 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.140+0000 7faab0f49640 1 -- 172.21.15.49:0/3726366504 wait complete. 2023-11-06T02:09:44.158 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":73,"stamp":"2023-11-06T02:09:43.200949+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":236068,"kb_used_data":4068,"kb_used_omap":0,"kb_used_meta":231936,"kb_avail":754738652,"statfs":{"total":773094113280,"available":772852379648,"internally_reserved":0,"allocated":4165632,"data_stored":2368733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":237502464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.729934"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":175,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-06T02:09:33.265052+0000","last_change":"2023-11-06T02:08:20.523427+0000","last_active":"2023-11-06T02:09:33.265052+0000","last_peered":"2023-11-06T02:09:33.265052+0000","last_clean":"2023-11-06T02:09:33.265052+0000","last_became_active":"2023-11-06T02:08:20.521089+0000","last_became_peered":"2023-11-06T02:08:20.521089+0000","last_unstale":"2023-11-06T02:09:33.265052+0000","last_undegraded":"2023-11-06T02:09:33.265052+0000","last_fullsized":"2023-11-06T02:09:33.265052+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:19.252348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:19.252348+0000","last_clean_scrub_stamp":"2023-11-06T02:08:19.252348+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:30:15.566397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-06T02:09:34.944240+0000","last_change":"2023-11-06T02:08:26.059502+0000","last_active":"2023-11-06T02:09:34.944240+0000","last_peered":"2023-11-06T02:09:34.944240+0000","last_clean":"2023-11-06T02:09:34.944240+0000","last_became_active":"2023-11-06T02:08:26.058612+0000","last_became_peered":"2023-11-06T02:08:26.058612+0000","last_unstale":"2023-11-06T02:09:34.944240+0000","last_undegraded":"2023-11-06T02:09:34.944240+0000","last_fullsized":"2023-11-06T02:09:34.944240+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:10.626021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-06T02:09:33.264953+0000","last_change":"2023-11-06T02:08:26.060406+0000","last_active":"2023-11-06T02:09:33.264953+0000","last_peered":"2023-11-06T02:09:33.264953+0000","last_clean":"2023-11-06T02:09:33.264953+0000","last_became_active":"2023-11-06T02:08:26.059262+0000","last_became_peered":"2023-11-06T02:08:26.059262+0000","last_unstale":"2023-11-06T02:09:33.264953+0000","last_undegraded":"2023-11-06T02:09:33.264953+0000","last_fullsized":"2023-11-06T02:09:33.264953+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:10.626021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-06T02:09:34.567262+0000","last_change":"2023-11-06T02:08:26.559477+0000","last_active":"2023-11-06T02:09:34.567262+0000","last_peered":"2023-11-06T02:09:34.567262+0000","last_clean":"2023-11-06T02:09:34.567262+0000","last_became_active":"2023-11-06T02:08:26.559202+0000","last_became_peered":"2023-11-06T02:08:26.559202+0000","last_unstale":"2023-11-06T02:09:34.567262+0000","last_undegraded":"2023-11-06T02:09:34.567262+0000","last_fullsized":"2023-11-06T02:09:34.567262+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:55:04.552404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-06T02:09:34.944356+0000","last_change":"2023-11-06T02:08:26.059546+0000","last_active":"2023-11-06T02:09:34.944356+0000","last_peered":"2023-11-06T02:09:34.944356+0000","last_clean":"2023-11-06T02:09:34.944356+0000","last_became_active":"2023-11-06T02:08:26.058806+0000","last_became_peered":"2023-11-06T02:08:26.058806+0000","last_unstale":"2023-11-06T02:09:34.944356+0000","last_undegraded":"2023-11-06T02:09:34.944356+0000","last_fullsized":"2023-11-06T02:09:34.944356+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:23.468510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-11-06T02:09:33.866940+0000","last_change":"2023-11-06T02:08:26.436463+0000","last_active":"2023-11-06T02:09:33.866940+0000","last_peered":"2023-11-06T02:09:33.866940+0000","last_clean":"2023-11-06T02:09:33.866940+0000","last_became_active":"2023-11-06T02:08:26.436075+0000","last_became_peered":"2023-11-06T02:08:26.436075+0000","last_unstale":"2023-11-06T02:09:33.866940+0000","last_undegraded":"2023-11-06T02:09:33.866940+0000","last_fullsized":"2023-11-06T02:09:33.866940+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:23.468510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-06T02:09:33.265056+0000","last_change":"2023-11-06T02:08:26.436607+0000","last_active":"2023-11-06T02:09:33.265056+0000","last_peered":"2023-11-06T02:09:33.265056+0000","last_clean":"2023-11-06T02:09:33.265056+0000","last_became_active":"2023-11-06T02:08:26.435924+0000","last_became_peered":"2023-11-06T02:08:26.435924+0000","last_unstale":"2023-11-06T02:09:33.265056+0000","last_undegraded":"2023-11-06T02:09:33.265056+0000","last_fullsized":"2023-11-06T02:09:33.265056+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:23.468510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-11-06T02:09:33.866882+0000","last_change":"2023-11-06T02:08:26.057076+0000","last_active":"2023-11-06T02:09:33.866882+0000","last_peered":"2023-11-06T02:09:33.866882+0000","last_clean":"2023-11-06T02:09:33.866882+0000","last_became_active":"2023-11-06T02:08:26.056419+0000","last_became_peered":"2023-11-06T02:08:26.056419+0000","last_unstale":"2023-11-06T02:09:33.866882+0000","last_undegraded":"2023-11-06T02:09:33.866882+0000","last_fullsized":"2023-11-06T02:09:33.866882+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:10.626021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":24,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-06T02:09:32.628412+0000","last_change":"2023-11-06T02:08:26.058596+0000","last_active":"2023-11-06T02:09:32.628412+0000","last_peered":"2023-11-06T02:09:32.628412+0000","last_clean":"2023-11-06T02:09:32.628412+0000","last_became_active":"2023-11-06T02:08:26.057015+0000","last_became_peered":"2023-11-06T02:08:26.057015+0000","last_unstale":"2023-11-06T02:09:32.628412+0000","last_undegraded":"2023-11-06T02:09:32.628412+0000","last_fullsized":"2023-11-06T02:09:32.628412+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:55:04.552404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574867,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343944,"statfs":{"total":96636764160,"available":96608198656,"internally_reserved":0,"allocated":966656,"data_stored":738995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":1.4650000000000001},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":1.3959999999999999}]},{"osd":1,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.94899999999999995},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.76300000000000001}]},{"osd":2,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":1.367},{"interface":"front","average":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":1.3420000000000001}]},{"osd":3,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":1.2969999999999999},{"interface":"front","average":{"1min":1.077,"5min":1.077,"15min":1.077},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":1.526}]},{"osd":4,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":1.163},{"interface":"front","average":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":1.115}]},{"osd":5,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.079,"15min":1.079},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":1.2450000000000001},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.7809999999999999,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":1.0760000000000001}]},{"osd":6,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.202,"15min":1.202},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.6679999999999999,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":1.4930000000000001},{"interface":"front","average":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.74,"5min":1.74,"15min":1.74},"last":1.421}]}]},{"osd":6,"up_from":13,"seq":55834574866,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":372736,"data_stored":148627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.65200000000000002}]},{"osd":1,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.68799999999999994}]},{"osd":2,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.28,"5min":1.28,"15min":1.28},"last":0.89300000000000002}]},{"osd":3,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.84099999999999997},{"interface":"front","average":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.79800000000000004}]},{"osd":4,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.474,"5min":1.474,"15min":1.474},"last":0.76900000000000002},{"interface":"front","average":{"1min":1.04,"5min":1.04,"15min":1.04},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":1.034}]},{"osd":5,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.365,"5min":1.365,"15min":1.365},"last":1.006},{"interface":"front","average":{"1min":1.115,"5min":1.115,"15min":1.115},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.97599999999999998}]},{"osd":7,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.93300000000000005},{"interface":"front","average":{"1min":1.165,"5min":1.165,"15min":1.165},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":1.0629999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574867,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":372736,"data_stored":148627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":1.038},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.6579999999999999,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":1.1739999999999999}]},{"osd":1,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":1.109}]},{"osd":2,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.377,"5min":1.377,"15min":1.377},"last":1.2909999999999999}]},{"osd":3,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":1.081,"5min":1.081,"15min":1.081},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.7250000000000001,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":0.67400000000000004}]},{"osd":4,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":1.214},{"interface":"front","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":1.0800000000000001}]},{"osd":6,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.97799999999999998},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.84199999999999997}]},{"osd":7,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":1.0089999999999999},{"interface":"front","average":{"1min":1.155,"5min":1.155,"15min":1.155},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.8089999999999999,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":1.2549999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27316,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":372736,"data_stored":148627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":1.0920000000000001},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.755}]},{"osd":1,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":1.077},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.94599999999999995}]},{"osd":2,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":1.02,"15min":1.02},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.615,"5min":1.615,"15min":1.615},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.83599999999999997}]},{"osd":3,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.52,"5min":1.52,"15min":1.52},"last":0.97899999999999998},{"interface":"front","average":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.4970000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.91800000000000004}]},{"osd":5,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"min":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":1.377,"5min":1.377,"15min":1.377},"last":1.1339999999999999},{"interface":"front","average":{"1min":1.056,"5min":1.056,"15min":1.056},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.7869999999999999,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":0.67000000000000004}]},{"osd":6,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":1.1180000000000001},{"interface":"front","average":{"1min":1.198,"5min":1.198,"15min":1.198},"min":{"1min":0.87,"5min":0.87,"15min":0.87},"max":{"1min":2.1190000000000002,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":1.0429999999999999}]},{"osd":7,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.7110000000000001,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.89000000000000001},{"interface":"front","average":{"1min":1.24,"5min":1.24,"15min":1.24},"min":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":1.7410000000000001,"5min":1.7410000000000001,"15min":1.7410000000000001},"last":1.014}]}]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":372736,"data_stored":148627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:09:19 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.45700000000000002}]},{"osd":1,"last update":"Mon Nov 6 02:09:19 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.78600000000000003}]},{"osd":2,"last update":"Mon Nov 6 02:09:19 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.61399999999999999}]},{"osd":4,"last update":"Mon Nov 6 02:09:19 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.405,"5min":1.405,"15min":1.405},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.85199999999999998}]},{"osd":5,"last update":"Mon Nov 6 02:09:19 2023","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.276,"5min":1.276,"15min":1.276},"last":0.97299999999999998},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":2.21,"5min":2.21,"15min":2.21},"last":0.88}]},{"osd":6,"last update":"Mon Nov 6 02:09:19 2023","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":1.026,"15min":1.026},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.67600000000000005},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":1.0609999999999999}]},{"osd":7,"last update":"Mon Nov 6 02:09:19 2023","interfaces":[{"interface":"back","average":{"1min":1.133,"5min":1.133,"15min":1.133},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":2.2829999999999999,"5min":2.2829999999999999,"15min":2.2829999999999999},"last":0.93999999999999995},{"interface":"front","average":{"1min":1.107,"5min":1.107,"15min":1.107},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":0.82199999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":372736,"data_stored":148627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.53000000000000003}]},{"osd":1,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.48599999999999999}]},{"osd":3,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.83099999999999996}]},{"osd":4,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.753},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.98299999999999998}]},{"osd":5,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.5369999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":1.1850000000000001},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.7110000000000001,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.94599999999999995}]},{"osd":6,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":1.0489999999999999},{"interface":"front","average":{"1min":0.997,"5min":0.997,"15min":0.997},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.677,"5min":1.677,"15min":1.677},"last":1.296}]},{"osd":7,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.633,"5min":1.633,"15min":1.633},"last":1.0880000000000001},{"interface":"front","average":{"1min":1.042,"5min":1.042,"15min":1.042},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":1.2509999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":372736,"data_stored":148627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:09:19 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.44500000000000001}]},{"osd":2,"last update":"Mon Nov 6 02:09:19 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.56699999999999995}]},{"osd":3,"last update":"Mon Nov 6 02:09:19 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.8149999999999999,"5min":1.8149999999999999,"15min":1.8149999999999999},"last":0.59899999999999998}]},{"osd":4,"last update":"Mon Nov 6 02:09:19 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.82999999999999996}]},{"osd":5,"last update":"Mon Nov 6 02:09:19 2023","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.90000000000000002},{"interface":"front","average":{"1min":1.079,"5min":1.079,"15min":1.079},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.6759999999999999,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":1.1120000000000001}]},{"osd":6,"last update":"Mon Nov 6 02:09:19 2023","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.103,"15min":1.103},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.742,"5min":1.742,"15min":1.742},"last":1.077},{"interface":"front","average":{"1min":1.077,"5min":1.077,"15min":1.077},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.393,"5min":1.393,"15min":1.393},"last":1.159}]},{"osd":7,"last update":"Mon Nov 6 02:09:19 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.486,"5min":1.486,"15min":1.486},"last":1.3620000000000001},{"interface":"front","average":{"1min":1.161,"5min":1.161,"15min":1.161},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.893,"5min":1.893,"15min":1.893},"last":1.226}]}]},{"osd":0,"up_from":13,"seq":55834574867,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31988,"kb_used_data":940,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339852,"statfs":{"total":96636764160,"available":96604008448,"internally_reserved":0,"allocated":962560,"data_stored":737976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.872}]},{"osd":2,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.64500000000000002}]},{"osd":3,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":1.0469999999999999}]},{"osd":4,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":1.085},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.97599999999999998}]},{"osd":5,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":1.137},{"interface":"front","average":{"1min":0.872,"5min":0.872,"15min":0.872},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.448,"5min":1.448,"15min":1.448},"last":1.0109999999999999}]},{"osd":6,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":1.3089999999999999},{"interface":"front","average":{"1min":1,"5min":1,"15min":1},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":1.234}]},{"osd":7,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":1.294},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.556,"5min":1.556,"15min":1.556},"last":1.173}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T02:09:44.166 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T02:09:44.325 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.324+0000 7f2773eba640 1 Processor -- start 2023-11-06T02:09:44.325 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.324+0000 7f2773eba640 1 -- start start 2023-11-06T02:09:44.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.328+0000 7f2773eba640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f276c0cb010 0x7f276c0c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:44.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.328+0000 7f2773eba640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f276c0cab80 con 0x7f276c15ab50 2023-11-06T02:09:44.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.328+0000 7f2773eba640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f276c15af30 con 0x7f276c0b0790 2023-11-06T02:09:44.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.328+0000 7f2773eba640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f276c0cb5b0 con 0x7f276c0cb010 2023-11-06T02:09:44.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.328+0000 7f2771c56640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f276c15ab50 0x7f276c0b0080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:48182/0 (socket says 172.21.15.49:48182) 2023-11-06T02:09:44.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.328+0000 7f2771c56640 1 -- 172.21.15.49:0/3856883265 learned_addr learned my addr 172.21.15.49:0/3856883265 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:09:44.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.328+0000 7f2770c54640 1 -- 172.21.15.49:0/3856883265 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4207392079 0 0) 0x7f276c0cab80 con 0x7f276c15ab50 2023-11-06T02:09:44.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.328+0000 7f2772457640 1 --2- 172.21.15.49:0/3856883265 >> v2:172.21.15.150:3300/0 conn(0x7f276c0cb010 0x7f276c0c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:44.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.328+0000 7f2770c54640 1 -- 172.21.15.49:0/3856883265 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2758003530 con 0x7f276c15ab50 2023-11-06T02:09:44.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.328+0000 7f2770c54640 1 -- 172.21.15.49:0/3856883265 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1374076455 0 0) 0x7f2758003530 con 0x7f276c15ab50 2023-11-06T02:09:44.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.328+0000 7f2770c54640 1 -- 172.21.15.49:0/3856883265 >> v2:172.21.15.150:3300/0 conn(0x7f276c0cb010 msgr2=0x7f276c0c9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:44.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.328+0000 7f2770c54640 1 --2- 172.21.15.49:0/3856883265 >> v2:172.21.15.150:3300/0 conn(0x7f276c0cb010 0x7f276c0c9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:44.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.328+0000 7f2770c54640 1 -- 172.21.15.49:0/3856883265 >> v1:172.21.15.150:6789/0 conn(0x7f276c0b0790 legacy=0x7f276c0c2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:44.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.328+0000 7f2770c54640 1 -- 172.21.15.49:0/3856883265 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f276c0cb890 con 0x7f276c15ab50 2023-11-06T02:09:44.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.328+0000 7f2770c54640 1 -- 172.21.15.49:0/3856883265 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7f2768001800 con 0x7f276c15ab50 2023-11-06T02:09:44.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.328+0000 7f2770c54640 1 -- 172.21.15.49:0/3856883265 >> v1:172.21.15.49:6789/0 conn(0x7f276c15ab50 legacy=0x7f276c0b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:44.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.328+0000 7f2770c54640 1 --2- 172.21.15.49:0/3856883265 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2758003980 0x7f2758003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:44.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.328+0000 7f2770c54640 1 --2- 172.21.15.49:0/3856883265 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f27580047d0 0x7f2758004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:44.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.328+0000 7f2770c54640 1 --2- 172.21.15.49:0/3856883265 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2758005190 0x7f275800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:44.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.328+0000 7f2770c54640 1 -- 172.21.15.49:0/3856883265 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f276c0cb5b0 con 0x7f27580047d0 2023-11-06T02:09:44.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.328+0000 7f2770c54640 1 -- 172.21.15.49:0/3856883265 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f275800c030 con 0x7f2758005190 2023-11-06T02:09:44.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.328+0000 7f2770c54640 1 -- 172.21.15.49:0/3856883265 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f275800c1f0 con 0x7f2758003980 2023-11-06T02:09:44.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.328+0000 7f2771455640 1 --2- 172.21.15.49:0/3856883265 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f27580047d0 0x7f2758004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:44.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.328+0000 7f2771c56640 1 --2- 172.21.15.49:0/3856883265 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2758003980 0x7f2758003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:44.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.328+0000 7f2772457640 1 --2- 172.21.15.49:0/3856883265 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2758005190 0x7f275800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:44.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.328+0000 7f2771455640 1 -- 172.21.15.49:0/3856883265 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2758003980 msgr2=0x7f2758003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:44.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.328+0000 7f2771455640 1 --2- 172.21.15.49:0/3856883265 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2758003980 0x7f2758003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:44.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.328+0000 7f2771455640 1 -- 172.21.15.49:0/3856883265 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2758005190 msgr2=0x7f275800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:44.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.328+0000 7f2771455640 1 --2- 172.21.15.49:0/3856883265 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2758005190 0x7f275800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:44.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.328+0000 7f2772457640 1 -- 172.21.15.49:0/3856883265 reap_dead start 2023-11-06T02:09:44.334 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.328+0000 7f2771455640 1 -- 172.21.15.49:0/3856883265 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2758003530 con 0x7f27580047d0 2023-11-06T02:09:44.334 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.328+0000 7f2771c56640 1 --2- 172.21.15.49:0/3856883265 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2758003980 0x7f2758003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:09:44.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.332+0000 7f2771455640 1 --2- 172.21.15.49:0/3856883265 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f27580047d0 0x7f2758004c50 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f275c001780 tx=0x7f275c01c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7db373fe2a3c8c40 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:44.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.332+0000 7f2770c54640 1 -- 172.21.15.49:0/3856883265 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f275c0052d0 con 0x7f27580047d0 2023-11-06T02:09:44.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.332+0000 7f2770c54640 1 -- 172.21.15.49:0/3856883265 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f275c0a7050 con 0x7f27580047d0 2023-11-06T02:09:44.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.332+0000 7f2770c54640 1 -- 172.21.15.49:0/3856883265 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f275c003600 con 0x7f27580047d0 2023-11-06T02:09:44.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.332+0000 7f2773eba640 1 -- 172.21.15.49:0/3856883265 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f27580047d0 msgr2=0x7f2758004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:44.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.332+0000 7f2773eba640 1 --2- 172.21.15.49:0/3856883265 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f27580047d0 0x7f2758004c50 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f275c001780 tx=0x7f275c01c3e0 comp rx=0 tx=0).stop 2023-11-06T02:09:44.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.332+0000 7f2773eba640 1 -- 172.21.15.49:0/3856883265 shutdown_connections 2023-11-06T02:09:44.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.332+0000 7f2773eba640 1 --2- 172.21.15.49:0/3856883265 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f27580047d0 0x7f2758004c50 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:44.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.332+0000 7f2773eba640 1 -- 172.21.15.49:0/3856883265 >> 172.21.15.49:0/3856883265 conn(0x7f276c06c0a0 msgr2=0x7f276c06c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:44.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.332+0000 7f2773eba640 1 -- 172.21.15.49:0/3856883265 shutdown_connections 2023-11-06T02:09:44.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.332+0000 7f2773eba640 1 -- 172.21.15.49:0/3856883265 wait complete. 2023-11-06T02:09:44.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.332+0000 7f2773eba640 1 Processor -- start 2023-11-06T02:09:44.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.332+0000 7f2773eba640 1 -- start start 2023-11-06T02:09:44.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.332+0000 7f2773eba640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f27580047d0 0x7f276c158650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:44.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.332+0000 7f2773eba640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f276c1efc20 0x7f276c0c8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:44.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.332+0000 7f2773eba640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f276c0c91f0 0x7f276c15b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:44.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.332+0000 7f2773eba640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f276c0c9640 con 0x7f276c1efc20 2023-11-06T02:09:44.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.332+0000 7f2773eba640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f276c158b90 con 0x7f27580047d0 2023-11-06T02:09:44.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.332+0000 7f2773eba640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f276c0c5850 con 0x7f276c0c91f0 2023-11-06T02:09:44.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.332+0000 7f2771455640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f276c1efc20 0x7f276c0c8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:44.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.332+0000 7f2771455640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f276c1efc20 0x7f276c0c8cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36598/0 (socket says 172.21.15.49:36598) 2023-11-06T02:09:44.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.332+0000 7f2771455640 1 -- 172.21.15.49:0/1829958339 learned_addr learned my addr 172.21.15.49:0/1829958339 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:44.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.332+0000 7f2771c56640 1 --2- 172.21.15.49:0/1829958339 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f27580047d0 0x7f276c158650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:44.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.332+0000 7f2772457640 1 --2- 172.21.15.49:0/1829958339 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f276c0c91f0 0x7f276c15b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:44.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.332+0000 7f2771455640 1 -- 172.21.15.49:0/1829958339 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f276c0c91f0 msgr2=0x7f276c15b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:44.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.332+0000 7f2771455640 1 --2- 172.21.15.49:0/1829958339 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f276c0c91f0 0x7f276c15b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:44.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.332+0000 7f2771455640 1 -- 172.21.15.49:0/1829958339 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f27580047d0 msgr2=0x7f276c158650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:44.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.332+0000 7f2771455640 1 --2- 172.21.15.49:0/1829958339 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f27580047d0 0x7f276c158650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:44.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.332+0000 7f2771455640 1 -- 172.21.15.49:0/1829958339 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f276c0c5b30 con 0x7f276c1efc20 2023-11-06T02:09:44.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.336+0000 7f2771455640 1 --2- 172.21.15.49:0/1829958339 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f276c1efc20 0x7f276c0c8cb0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f275c01c210 tx=0x7f2758005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:44.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.336+0000 7f27627fc640 1 -- 172.21.15.49:0/1829958339 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f275c003e40 con 0x7f276c1efc20 2023-11-06T02:09:44.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.336+0000 7f27627fc640 1 -- 172.21.15.49:0/1829958339 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f275c007a40 con 0x7f276c1efc20 2023-11-06T02:09:44.344 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.336+0000 7f2773eba640 1 -- 172.21.15.49:0/1829958339 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f276c0c5e10 con 0x7f276c1efc20 2023-11-06T02:09:44.344 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.336+0000 7f27627fc640 1 -- 172.21.15.49:0/1829958339 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f275c007270 con 0x7f276c1efc20 2023-11-06T02:09:44.344 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.336+0000 7f2773eba640 1 -- 172.21.15.49:0/1829958339 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f276c15b940 con 0x7f276c1efc20 2023-11-06T02:09:44.344 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.336+0000 7f2773eba640 1 -- 172.21.15.49:0/1829958339 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f276c0b0de0 con 0x7f276c1efc20 2023-11-06T02:09:44.344 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.336+0000 7f27627fc640 1 -- 172.21.15.49:0/1829958339 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f275c0073d0 con 0x7f276c1efc20 2023-11-06T02:09:44.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.336+0000 7f27627fc640 1 --2- 172.21.15.49:0/1829958339 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2750065a60 0x7f2750067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:44.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.336+0000 7f27627fc640 1 -- 172.21.15.49:0/1829958339 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f275c11d790 con 0x7f276c1efc20 2023-11-06T02:09:44.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.336+0000 7f2771c56640 1 --2- 172.21.15.49:0/1829958339 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2750065a60 0x7f2750067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:44.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.336+0000 7f2771c56640 1 --2- 172.21.15.49:0/1829958339 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2750065a60 0x7f2750067f20 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2768000c10 tx=0x7f2768009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:44.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.340+0000 7f27627fc640 1 -- 172.21.15.49:0/1829958339 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f275c0eb140 con 0x7f276c1efc20 2023-11-06T02:09:44.493 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.492+0000 7f2773eba640 1 -- 172.21.15.49:0/1829958339 --> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f276c0c59c0 con 0x7f2750065a60 2023-11-06T02:09:44.515 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.516+0000 7f27627fc640 1 -- 172.21.15.49:0/1829958339 <== mgr.4100 v2:172.21.15.150:6832/38104 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+294851 (secure 0 0 0) 0x7f276c0c59c0 con 0x7f2750065a60 2023-11-06T02:09:44.518 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:09:44.520 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2023-11-06T02:09:44.521 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.520+0000 7f2773eba640 1 -- 172.21.15.49:0/1829958339 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2750065a60 msgr2=0x7f2750067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:44.521 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.520+0000 7f2773eba640 1 --2- 172.21.15.49:0/1829958339 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2750065a60 0x7f2750067f20 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2768000c10 tx=0x7f2768009040 comp rx=0 tx=0).stop 2023-11-06T02:09:44.522 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.520+0000 7f2773eba640 1 -- 172.21.15.49:0/1829958339 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f276c1efc20 msgr2=0x7f276c0c8cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:44.522 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.520+0000 7f2773eba640 1 --2- 172.21.15.49:0/1829958339 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f276c1efc20 0x7f276c0c8cb0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f275c01c210 tx=0x7f2758005800 comp rx=0 tx=0).stop 2023-11-06T02:09:44.522 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.520+0000 7f2773eba640 1 -- 172.21.15.49:0/1829958339 shutdown_connections 2023-11-06T02:09:44.522 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.520+0000 7f2773eba640 1 --2- 172.21.15.49:0/1829958339 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2750065a60 0x7f2750067f20 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:44.523 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.520+0000 7f2773eba640 1 --2- 172.21.15.49:0/1829958339 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f276c1efc20 0x7f276c0c8cb0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:44.523 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.520+0000 7f2773eba640 1 --2- 172.21.15.49:0/1829958339 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f276c0c91f0 0x7f276c15b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:44.523 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.520+0000 7f2773eba640 1 --2- 172.21.15.49:0/1829958339 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f27580047d0 0x7f276c158650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:44.523 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.520+0000 7f2773eba640 1 -- 172.21.15.49:0/1829958339 >> 172.21.15.49:0/1829958339 conn(0x7f276c06c0a0 msgr2=0x7f276c0c15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:44.524 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.520+0000 7f2773eba640 1 -- 172.21.15.49:0/1829958339 shutdown_connections 2023-11-06T02:09:44.524 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.520+0000 7f2773eba640 1 -- 172.21.15.49:0/1829958339 wait complete. 2023-11-06T02:09:44.538 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":73,"stamp":"2023-11-06T02:09:43.200949+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":236068,"kb_used_data":4068,"kb_used_omap":0,"kb_used_meta":231936,"kb_avail":754738652,"statfs":{"total":773094113280,"available":772852379648,"internally_reserved":0,"allocated":4165632,"data_stored":2368733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":237502464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.729934"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":175,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-06T02:09:33.265052+0000","last_change":"2023-11-06T02:08:20.523427+0000","last_active":"2023-11-06T02:09:33.265052+0000","last_peered":"2023-11-06T02:09:33.265052+0000","last_clean":"2023-11-06T02:09:33.265052+0000","last_became_active":"2023-11-06T02:08:20.521089+0000","last_became_peered":"2023-11-06T02:08:20.521089+0000","last_unstale":"2023-11-06T02:09:33.265052+0000","last_undegraded":"2023-11-06T02:09:33.265052+0000","last_fullsized":"2023-11-06T02:09:33.265052+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:19.252348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:19.252348+0000","last_clean_scrub_stamp":"2023-11-06T02:08:19.252348+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:30:15.566397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-06T02:09:34.944240+0000","last_change":"2023-11-06T02:08:26.059502+0000","last_active":"2023-11-06T02:09:34.944240+0000","last_peered":"2023-11-06T02:09:34.944240+0000","last_clean":"2023-11-06T02:09:34.944240+0000","last_became_active":"2023-11-06T02:08:26.058612+0000","last_became_peered":"2023-11-06T02:08:26.058612+0000","last_unstale":"2023-11-06T02:09:34.944240+0000","last_undegraded":"2023-11-06T02:09:34.944240+0000","last_fullsized":"2023-11-06T02:09:34.944240+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:10.626021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-06T02:09:33.264953+0000","last_change":"2023-11-06T02:08:26.060406+0000","last_active":"2023-11-06T02:09:33.264953+0000","last_peered":"2023-11-06T02:09:33.264953+0000","last_clean":"2023-11-06T02:09:33.264953+0000","last_became_active":"2023-11-06T02:08:26.059262+0000","last_became_peered":"2023-11-06T02:08:26.059262+0000","last_unstale":"2023-11-06T02:09:33.264953+0000","last_undegraded":"2023-11-06T02:09:33.264953+0000","last_fullsized":"2023-11-06T02:09:33.264953+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:10.626021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-06T02:09:34.567262+0000","last_change":"2023-11-06T02:08:26.559477+0000","last_active":"2023-11-06T02:09:34.567262+0000","last_peered":"2023-11-06T02:09:34.567262+0000","last_clean":"2023-11-06T02:09:34.567262+0000","last_became_active":"2023-11-06T02:08:26.559202+0000","last_became_peered":"2023-11-06T02:08:26.559202+0000","last_unstale":"2023-11-06T02:09:34.567262+0000","last_undegraded":"2023-11-06T02:09:34.567262+0000","last_fullsized":"2023-11-06T02:09:34.567262+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:55:04.552404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-06T02:09:34.944356+0000","last_change":"2023-11-06T02:08:26.059546+0000","last_active":"2023-11-06T02:09:34.944356+0000","last_peered":"2023-11-06T02:09:34.944356+0000","last_clean":"2023-11-06T02:09:34.944356+0000","last_became_active":"2023-11-06T02:08:26.058806+0000","last_became_peered":"2023-11-06T02:08:26.058806+0000","last_unstale":"2023-11-06T02:09:34.944356+0000","last_undegraded":"2023-11-06T02:09:34.944356+0000","last_fullsized":"2023-11-06T02:09:34.944356+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:23.468510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-11-06T02:09:33.866940+0000","last_change":"2023-11-06T02:08:26.436463+0000","last_active":"2023-11-06T02:09:33.866940+0000","last_peered":"2023-11-06T02:09:33.866940+0000","last_clean":"2023-11-06T02:09:33.866940+0000","last_became_active":"2023-11-06T02:08:26.436075+0000","last_became_peered":"2023-11-06T02:08:26.436075+0000","last_unstale":"2023-11-06T02:09:33.866940+0000","last_undegraded":"2023-11-06T02:09:33.866940+0000","last_fullsized":"2023-11-06T02:09:33.866940+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:23.468510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-06T02:09:33.265056+0000","last_change":"2023-11-06T02:08:26.436607+0000","last_active":"2023-11-06T02:09:33.265056+0000","last_peered":"2023-11-06T02:09:33.265056+0000","last_clean":"2023-11-06T02:09:33.265056+0000","last_became_active":"2023-11-06T02:08:26.435924+0000","last_became_peered":"2023-11-06T02:08:26.435924+0000","last_unstale":"2023-11-06T02:09:33.265056+0000","last_undegraded":"2023-11-06T02:09:33.265056+0000","last_fullsized":"2023-11-06T02:09:33.265056+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:23.468510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-11-06T02:09:33.866882+0000","last_change":"2023-11-06T02:08:26.057076+0000","last_active":"2023-11-06T02:09:33.866882+0000","last_peered":"2023-11-06T02:09:33.866882+0000","last_clean":"2023-11-06T02:09:33.866882+0000","last_became_active":"2023-11-06T02:08:26.056419+0000","last_became_peered":"2023-11-06T02:08:26.056419+0000","last_unstale":"2023-11-06T02:09:33.866882+0000","last_undegraded":"2023-11-06T02:09:33.866882+0000","last_fullsized":"2023-11-06T02:09:33.866882+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:10.626021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":24,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-06T02:09:32.628412+0000","last_change":"2023-11-06T02:08:26.058596+0000","last_active":"2023-11-06T02:09:32.628412+0000","last_peered":"2023-11-06T02:09:32.628412+0000","last_clean":"2023-11-06T02:09:32.628412+0000","last_became_active":"2023-11-06T02:08:26.057015+0000","last_became_peered":"2023-11-06T02:08:26.057015+0000","last_unstale":"2023-11-06T02:09:32.628412+0000","last_undegraded":"2023-11-06T02:09:32.628412+0000","last_fullsized":"2023-11-06T02:09:32.628412+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:55:04.552404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574867,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343944,"statfs":{"total":96636764160,"available":96608198656,"internally_reserved":0,"allocated":966656,"data_stored":738995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":1.4650000000000001},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":1.3959999999999999}]},{"osd":1,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.94899999999999995},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.76300000000000001}]},{"osd":2,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":1.367},{"interface":"front","average":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":1.3420000000000001}]},{"osd":3,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":1.2969999999999999},{"interface":"front","average":{"1min":1.077,"5min":1.077,"15min":1.077},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":1.526}]},{"osd":4,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":1.163},{"interface":"front","average":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":1.115}]},{"osd":5,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.079,"15min":1.079},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":1.2450000000000001},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.7809999999999999,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":1.0760000000000001}]},{"osd":6,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.202,"15min":1.202},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.6679999999999999,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":1.4930000000000001},{"interface":"front","average":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.74,"5min":1.74,"15min":1.74},"last":1.421}]}]},{"osd":6,"up_from":13,"seq":55834574866,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":372736,"data_stored":148627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.65200000000000002}]},{"osd":1,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.68799999999999994}]},{"osd":2,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.28,"5min":1.28,"15min":1.28},"last":0.89300000000000002}]},{"osd":3,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.84099999999999997},{"interface":"front","average":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.79800000000000004}]},{"osd":4,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.474,"5min":1.474,"15min":1.474},"last":0.76900000000000002},{"interface":"front","average":{"1min":1.04,"5min":1.04,"15min":1.04},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":1.034}]},{"osd":5,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.365,"5min":1.365,"15min":1.365},"last":1.006},{"interface":"front","average":{"1min":1.115,"5min":1.115,"15min":1.115},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.97599999999999998}]},{"osd":7,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.93300000000000005},{"interface":"front","average":{"1min":1.165,"5min":1.165,"15min":1.165},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":1.0629999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574867,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":372736,"data_stored":148627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":1.038},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.6579999999999999,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":1.1739999999999999}]},{"osd":1,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":1.109}]},{"osd":2,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.377,"5min":1.377,"15min":1.377},"last":1.2909999999999999}]},{"osd":3,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":1.081,"5min":1.081,"15min":1.081},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.7250000000000001,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":0.67400000000000004}]},{"osd":4,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":1.214},{"interface":"front","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":1.0800000000000001}]},{"osd":6,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.97799999999999998},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.84199999999999997}]},{"osd":7,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":1.0089999999999999},{"interface":"front","average":{"1min":1.155,"5min":1.155,"15min":1.155},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.8089999999999999,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":1.2549999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27316,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":372736,"data_stored":148627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":1.0920000000000001},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.755}]},{"osd":1,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":1.077},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.94599999999999995}]},{"osd":2,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":1.02,"15min":1.02},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.615,"5min":1.615,"15min":1.615},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.83599999999999997}]},{"osd":3,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.52,"5min":1.52,"15min":1.52},"last":0.97899999999999998},{"interface":"front","average":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.4970000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.91800000000000004}]},{"osd":5,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"min":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":1.377,"5min":1.377,"15min":1.377},"last":1.1339999999999999},{"interface":"front","average":{"1min":1.056,"5min":1.056,"15min":1.056},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.7869999999999999,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":0.67000000000000004}]},{"osd":6,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":1.1180000000000001},{"interface":"front","average":{"1min":1.198,"5min":1.198,"15min":1.198},"min":{"1min":0.87,"5min":0.87,"15min":0.87},"max":{"1min":2.1190000000000002,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":1.0429999999999999}]},{"osd":7,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.7110000000000001,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.89000000000000001},{"interface":"front","average":{"1min":1.24,"5min":1.24,"15min":1.24},"min":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":1.7410000000000001,"5min":1.7410000000000001,"15min":1.7410000000000001},"last":1.014}]}]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":372736,"data_stored":148627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:09:19 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.45700000000000002}]},{"osd":1,"last update":"Mon Nov 6 02:09:19 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.78600000000000003}]},{"osd":2,"last update":"Mon Nov 6 02:09:19 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.61399999999999999}]},{"osd":4,"last update":"Mon Nov 6 02:09:19 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.405,"5min":1.405,"15min":1.405},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.85199999999999998}]},{"osd":5,"last update":"Mon Nov 6 02:09:19 2023","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.276,"5min":1.276,"15min":1.276},"last":0.97299999999999998},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":2.21,"5min":2.21,"15min":2.21},"last":0.88}]},{"osd":6,"last update":"Mon Nov 6 02:09:19 2023","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":1.026,"15min":1.026},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.67600000000000005},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":1.0609999999999999}]},{"osd":7,"last update":"Mon Nov 6 02:09:19 2023","interfaces":[{"interface":"back","average":{"1min":1.133,"5min":1.133,"15min":1.133},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":2.2829999999999999,"5min":2.2829999999999999,"15min":2.2829999999999999},"last":0.93999999999999995},{"interface":"front","average":{"1min":1.107,"5min":1.107,"15min":1.107},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":0.82199999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":372736,"data_stored":148627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.53000000000000003}]},{"osd":1,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.48599999999999999}]},{"osd":3,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.83099999999999996}]},{"osd":4,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.753},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.98299999999999998}]},{"osd":5,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.5369999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":1.1850000000000001},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.7110000000000001,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.94599999999999995}]},{"osd":6,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":1.0489999999999999},{"interface":"front","average":{"1min":0.997,"5min":0.997,"15min":0.997},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.677,"5min":1.677,"15min":1.677},"last":1.296}]},{"osd":7,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.633,"5min":1.633,"15min":1.633},"last":1.0880000000000001},{"interface":"front","average":{"1min":1.042,"5min":1.042,"15min":1.042},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":1.2509999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":372736,"data_stored":148627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:09:19 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.44500000000000001}]},{"osd":2,"last update":"Mon Nov 6 02:09:19 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.56699999999999995}]},{"osd":3,"last update":"Mon Nov 6 02:09:19 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.8149999999999999,"5min":1.8149999999999999,"15min":1.8149999999999999},"last":0.59899999999999998}]},{"osd":4,"last update":"Mon Nov 6 02:09:19 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.82999999999999996}]},{"osd":5,"last update":"Mon Nov 6 02:09:19 2023","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.90000000000000002},{"interface":"front","average":{"1min":1.079,"5min":1.079,"15min":1.079},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.6759999999999999,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":1.1120000000000001}]},{"osd":6,"last update":"Mon Nov 6 02:09:19 2023","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.103,"15min":1.103},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.742,"5min":1.742,"15min":1.742},"last":1.077},{"interface":"front","average":{"1min":1.077,"5min":1.077,"15min":1.077},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.393,"5min":1.393,"15min":1.393},"last":1.159}]},{"osd":7,"last update":"Mon Nov 6 02:09:19 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.486,"5min":1.486,"15min":1.486},"last":1.3620000000000001},{"interface":"front","average":{"1min":1.161,"5min":1.161,"15min":1.161},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.893,"5min":1.893,"15min":1.893},"last":1.226}]}]},{"osd":0,"up_from":13,"seq":55834574867,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31988,"kb_used_data":940,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339852,"statfs":{"total":96636764160,"available":96604008448,"internally_reserved":0,"allocated":962560,"data_stored":737976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.872}]},{"osd":2,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.64500000000000002}]},{"osd":3,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":1.0469999999999999}]},{"osd":4,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":1.085},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.97599999999999998}]},{"osd":5,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":1.137},{"interface":"front","average":{"1min":0.872,"5min":0.872,"15min":0.872},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.448,"5min":1.448,"15min":1.448},"last":1.0109999999999999}]},{"osd":6,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":1.3089999999999999},{"interface":"front","average":{"1min":1,"5min":1,"15min":1},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":1.234}]},{"osd":7,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":1.294},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.556,"5min":1.556,"15min":1.556},"last":1.173}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T02:09:44.544 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T02:09:44.704 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.704+0000 7fb37e830640 1 Processor -- start 2023-11-06T02:09:44.705 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.704+0000 7fb37e830640 1 -- start start 2023-11-06T02:09:44.706 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.708+0000 7fb37e830640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb37815cd70 0x7fb37806b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:44.706 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.708+0000 7fb37e830640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb37806bfa0 0x7fb3780b2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:44.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.708+0000 7fb37e830640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3780aff70 con 0x7fb3780b0830 2023-11-06T02:09:44.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.708+0000 7fb37e830640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3780b0dd0 con 0x7fb37815cd70 2023-11-06T02:09:44.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.708+0000 7fb37e830640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3780b10b0 con 0x7fb37806bfa0 2023-11-06T02:09:44.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.708+0000 7fb377fff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb37815cd70 0x7fb37806b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:44.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.708+0000 7fb377fff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb37815cd70 0x7fb37806b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36606/0 (socket says 172.21.15.49:36606) 2023-11-06T02:09:44.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.708+0000 7fb377fff640 1 -- 172.21.15.49:0/1225938074 learned_addr learned my addr 172.21.15.49:0/1225938074 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:44.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.708+0000 7fb37cdcd640 1 --2- 172.21.15.49:0/1225938074 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb37806bfa0 0x7fb3780b2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:44.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.708+0000 7fb377fff640 1 -- 172.21.15.49:0/1225938074 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb37806bfa0 msgr2=0x7fb3780b2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:44.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.708+0000 7fb377fff640 1 --2- 172.21.15.49:0/1225938074 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb37806bfa0 0x7fb3780b2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:44.709 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.708+0000 7fb377fff640 1 -- 172.21.15.49:0/1225938074 >> v1:172.21.15.150:6789/0 conn(0x7fb3780b0830 legacy=0x7fb37806b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:44.709 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.708+0000 7fb377fff640 1 -- 172.21.15.49:0/1225938074 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3780b1390 con 0x7fb37815cd70 2023-11-06T02:09:44.710 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.712+0000 7fb377fff640 1 --2- 172.21.15.49:0/1225938074 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb37815cd70 0x7fb37806b220 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fb368006fd0 tx=0x7fb36801c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66d4141e3f9adc64 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:44.710 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.712+0000 7fb3777fe640 1 -- 172.21.15.49:0/1225938074 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3680ab620 con 0x7fb37815cd70 2023-11-06T02:09:44.710 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.712+0000 7fb3777fe640 1 -- 172.21.15.49:0/1225938074 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3680ab780 con 0x7fb37815cd70 2023-11-06T02:09:44.710 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.712+0000 7fb3777fe640 1 -- 172.21.15.49:0/1225938074 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3680abb10 con 0x7fb37815cd70 2023-11-06T02:09:44.712 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.712+0000 7fb37e830640 1 -- 172.21.15.49:0/1225938074 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb37815cd70 msgr2=0x7fb37806b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:44.712 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.712+0000 7fb37e830640 1 --2- 172.21.15.49:0/1225938074 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb37815cd70 0x7fb37806b220 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fb368006fd0 tx=0x7fb36801c400 comp rx=0 tx=0).stop 2023-11-06T02:09:44.712 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.712+0000 7fb37e830640 1 -- 172.21.15.49:0/1225938074 shutdown_connections 2023-11-06T02:09:44.712 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.712+0000 7fb37e830640 1 --2- 172.21.15.49:0/1225938074 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb37806bfa0 0x7fb3780b2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:44.713 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.712+0000 7fb37e830640 1 --2- 172.21.15.49:0/1225938074 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb37815cd70 0x7fb37806b220 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:44.713 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.712+0000 7fb37e830640 1 -- 172.21.15.49:0/1225938074 >> 172.21.15.49:0/1225938074 conn(0x7fb378089fd0 msgr2=0x7fb378099c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:44.713 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.712+0000 7fb37e830640 1 -- 172.21.15.49:0/1225938074 shutdown_connections 2023-11-06T02:09:44.713 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.712+0000 7fb37e830640 1 -- 172.21.15.49:0/1225938074 wait complete. 2023-11-06T02:09:44.714 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.712+0000 7fb37e830640 1 Processor -- start 2023-11-06T02:09:44.714 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.712+0000 7fb37e830640 1 -- start start 2023-11-06T02:09:44.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.712+0000 7fb37e830640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb37806bfa0 0x7fb37814bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:44.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.712+0000 7fb37e830640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb3780b0830 0x7fb37815d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:44.718 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.712+0000 7fb37e830640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb37815cd70 0x7fb37815f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:44.718 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.712+0000 7fb37e830640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3780ad400 con 0x7fb37806bfa0 2023-11-06T02:09:44.718 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.712+0000 7fb37e830640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb37814c070 con 0x7fb37815cd70 2023-11-06T02:09:44.718 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.712+0000 7fb37e830640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb37815dbd0 con 0x7fb3780b0830 2023-11-06T02:09:44.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.712+0000 7fb377fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb37806bfa0 0x7fb37814bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:44.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.712+0000 7fb377fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb37806bfa0 0x7fb37814bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36614/0 (socket says 172.21.15.49:36614) 2023-11-06T02:09:44.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.712+0000 7fb377fff640 1 -- 172.21.15.49:0/1755469057 learned_addr learned my addr 172.21.15.49:0/1755469057 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:44.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.716+0000 7fb37cdcd640 1 --2- 172.21.15.49:0/1755469057 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb37815cd70 0x7fb37815f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:44.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.716+0000 7fb36ffff640 1 --2- 172.21.15.49:0/1755469057 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb3780b0830 0x7fb37815d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:44.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.716+0000 7fb377fff640 1 -- 172.21.15.49:0/1755469057 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb3780b0830 msgr2=0x7fb37815d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:44.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.716+0000 7fb377fff640 1 --2- 172.21.15.49:0/1755469057 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb3780b0830 0x7fb37815d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:44.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.716+0000 7fb377fff640 1 -- 172.21.15.49:0/1755469057 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb37815cd70 msgr2=0x7fb37815f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:44.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.716+0000 7fb377fff640 1 --2- 172.21.15.49:0/1755469057 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb37815cd70 0x7fb37815f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:44.721 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.716+0000 7fb377fff640 1 -- 172.21.15.49:0/1755469057 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb368009480 con 0x7fb37806bfa0 2023-11-06T02:09:44.721 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.716+0000 7fb377fff640 1 --2- 172.21.15.49:0/1755469057 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb37806bfa0 0x7fb37814bb30 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fb36801a030 tx=0x7fb3680191a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:44.721 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.716+0000 7fb3757fa640 1 -- 172.21.15.49:0/1755469057 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3680abaf0 con 0x7fb37806bfa0 2023-11-06T02:09:44.721 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.716+0000 7fb3757fa640 1 -- 172.21.15.49:0/1755469057 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3680abc70 con 0x7fb37806bfa0 2023-11-06T02:09:44.721 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.716+0000 7fb37e830640 1 -- 172.21.15.49:0/1755469057 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb37815deb0 con 0x7fb37806bfa0 2023-11-06T02:09:44.722 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.716+0000 7fb3757fa640 1 -- 172.21.15.49:0/1755469057 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb368003620 con 0x7fb37806bfa0 2023-11-06T02:09:44.722 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.716+0000 7fb37e830640 1 -- 172.21.15.49:0/1755469057 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3780ad930 con 0x7fb37806bfa0 2023-11-06T02:09:44.722 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.716+0000 7fb37e830640 1 -- 172.21.15.49:0/1755469057 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb37806ae80 con 0x7fb37806bfa0 2023-11-06T02:09:44.723 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.716+0000 7fb3757fa640 1 -- 172.21.15.49:0/1755469057 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fb3680039b0 con 0x7fb37806bfa0 2023-11-06T02:09:44.723 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.716+0000 7fb3757fa640 1 --2- 172.21.15.49:0/1755469057 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb358065a60 0x7fb358067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:44.723 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.716+0000 7fb3757fa640 1 -- 172.21.15.49:0/1755469057 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb36811dc30 con 0x7fb37806bfa0 2023-11-06T02:09:44.723 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.716+0000 7fb36ffff640 1 --2- 172.21.15.49:0/1755469057 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb358065a60 0x7fb358067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:44.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.716+0000 7fb36ffff640 1 --2- 172.21.15.49:0/1755469057 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb358065a60 0x7fb358067f20 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb3780cce20 tx=0x7fb360009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:44.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.720+0000 7fb3757fa640 1 -- 172.21.15.49:0/1755469057 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fb3680eb3f0 con 0x7fb37806bfa0 2023-11-06T02:09:44.871 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.872+0000 7fb37e830640 1 -- 172.21.15.49:0/1755469057 --> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb3780c7740 con 0x7fb358065a60 2023-11-06T02:09:44.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.892+0000 7fb3757fa640 1 -- 172.21.15.49:0/1755469057 <== mgr.4100 v2:172.21.15.150:6832/38104 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+294851 (secure 0 0 0) 0x7fb3780c7740 con 0x7fb358065a60 2023-11-06T02:09:44.896 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:09:44.897 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2023-11-06T02:09:44.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.900+0000 7fb37e830640 1 -- 172.21.15.49:0/1755469057 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb358065a60 msgr2=0x7fb358067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:44.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.900+0000 7fb37e830640 1 --2- 172.21.15.49:0/1755469057 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb358065a60 0x7fb358067f20 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb3780cce20 tx=0x7fb360009040 comp rx=0 tx=0).stop 2023-11-06T02:09:44.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.900+0000 7fb37e830640 1 -- 172.21.15.49:0/1755469057 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb37806bfa0 msgr2=0x7fb37814bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:44.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.900+0000 7fb37e830640 1 --2- 172.21.15.49:0/1755469057 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb37806bfa0 0x7fb37814bb30 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fb36801a030 tx=0x7fb3680191a0 comp rx=0 tx=0).stop 2023-11-06T02:09:44.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.900+0000 7fb37e830640 1 -- 172.21.15.49:0/1755469057 shutdown_connections 2023-11-06T02:09:44.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.900+0000 7fb37e830640 1 --2- 172.21.15.49:0/1755469057 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb358065a60 0x7fb358067f20 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:44.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.900+0000 7fb37e830640 1 --2- 172.21.15.49:0/1755469057 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb37815cd70 0x7fb37815f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:44.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.900+0000 7fb37e830640 1 --2- 172.21.15.49:0/1755469057 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb3780b0830 0x7fb37815d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:44.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.900+0000 7fb37e830640 1 --2- 172.21.15.49:0/1755469057 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb37806bfa0 0x7fb37814bb30 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:44.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.900+0000 7fb37e830640 1 -- 172.21.15.49:0/1755469057 >> 172.21.15.49:0/1755469057 conn(0x7fb378089fd0 msgr2=0x7fb3780cf3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:44.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.900+0000 7fb37e830640 1 -- 172.21.15.49:0/1755469057 shutdown_connections 2023-11-06T02:09:44.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:44.900+0000 7fb37e830640 1 -- 172.21.15.49:0/1755469057 wait complete. 2023-11-06T02:09:44.915 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":73,"stamp":"2023-11-06T02:09:43.200949+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":236068,"kb_used_data":4068,"kb_used_omap":0,"kb_used_meta":231936,"kb_avail":754738652,"statfs":{"total":773094113280,"available":772852379648,"internally_reserved":0,"allocated":4165632,"data_stored":2368733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":237502464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.729934"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":175,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-06T02:09:33.265052+0000","last_change":"2023-11-06T02:08:20.523427+0000","last_active":"2023-11-06T02:09:33.265052+0000","last_peered":"2023-11-06T02:09:33.265052+0000","last_clean":"2023-11-06T02:09:33.265052+0000","last_became_active":"2023-11-06T02:08:20.521089+0000","last_became_peered":"2023-11-06T02:08:20.521089+0000","last_unstale":"2023-11-06T02:09:33.265052+0000","last_undegraded":"2023-11-06T02:09:33.265052+0000","last_fullsized":"2023-11-06T02:09:33.265052+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:19.252348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:19.252348+0000","last_clean_scrub_stamp":"2023-11-06T02:08:19.252348+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:30:15.566397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-06T02:09:34.944240+0000","last_change":"2023-11-06T02:08:26.059502+0000","last_active":"2023-11-06T02:09:34.944240+0000","last_peered":"2023-11-06T02:09:34.944240+0000","last_clean":"2023-11-06T02:09:34.944240+0000","last_became_active":"2023-11-06T02:08:26.058612+0000","last_became_peered":"2023-11-06T02:08:26.058612+0000","last_unstale":"2023-11-06T02:09:34.944240+0000","last_undegraded":"2023-11-06T02:09:34.944240+0000","last_fullsized":"2023-11-06T02:09:34.944240+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:10.626021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-06T02:09:33.264953+0000","last_change":"2023-11-06T02:08:26.060406+0000","last_active":"2023-11-06T02:09:33.264953+0000","last_peered":"2023-11-06T02:09:33.264953+0000","last_clean":"2023-11-06T02:09:33.264953+0000","last_became_active":"2023-11-06T02:08:26.059262+0000","last_became_peered":"2023-11-06T02:08:26.059262+0000","last_unstale":"2023-11-06T02:09:33.264953+0000","last_undegraded":"2023-11-06T02:09:33.264953+0000","last_fullsized":"2023-11-06T02:09:33.264953+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:10.626021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-06T02:09:34.567262+0000","last_change":"2023-11-06T02:08:26.559477+0000","last_active":"2023-11-06T02:09:34.567262+0000","last_peered":"2023-11-06T02:09:34.567262+0000","last_clean":"2023-11-06T02:09:34.567262+0000","last_became_active":"2023-11-06T02:08:26.559202+0000","last_became_peered":"2023-11-06T02:08:26.559202+0000","last_unstale":"2023-11-06T02:09:34.567262+0000","last_undegraded":"2023-11-06T02:09:34.567262+0000","last_fullsized":"2023-11-06T02:09:34.567262+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:55:04.552404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-06T02:09:34.944356+0000","last_change":"2023-11-06T02:08:26.059546+0000","last_active":"2023-11-06T02:09:34.944356+0000","last_peered":"2023-11-06T02:09:34.944356+0000","last_clean":"2023-11-06T02:09:34.944356+0000","last_became_active":"2023-11-06T02:08:26.058806+0000","last_became_peered":"2023-11-06T02:08:26.058806+0000","last_unstale":"2023-11-06T02:09:34.944356+0000","last_undegraded":"2023-11-06T02:09:34.944356+0000","last_fullsized":"2023-11-06T02:09:34.944356+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:23.468510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-11-06T02:09:33.866940+0000","last_change":"2023-11-06T02:08:26.436463+0000","last_active":"2023-11-06T02:09:33.866940+0000","last_peered":"2023-11-06T02:09:33.866940+0000","last_clean":"2023-11-06T02:09:33.866940+0000","last_became_active":"2023-11-06T02:08:26.436075+0000","last_became_peered":"2023-11-06T02:08:26.436075+0000","last_unstale":"2023-11-06T02:09:33.866940+0000","last_undegraded":"2023-11-06T02:09:33.866940+0000","last_fullsized":"2023-11-06T02:09:33.866940+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:23.468510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-06T02:09:33.265056+0000","last_change":"2023-11-06T02:08:26.436607+0000","last_active":"2023-11-06T02:09:33.265056+0000","last_peered":"2023-11-06T02:09:33.265056+0000","last_clean":"2023-11-06T02:09:33.265056+0000","last_became_active":"2023-11-06T02:08:26.435924+0000","last_became_peered":"2023-11-06T02:08:26.435924+0000","last_unstale":"2023-11-06T02:09:33.265056+0000","last_undegraded":"2023-11-06T02:09:33.265056+0000","last_fullsized":"2023-11-06T02:09:33.265056+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:23.468510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-11-06T02:09:33.866882+0000","last_change":"2023-11-06T02:08:26.057076+0000","last_active":"2023-11-06T02:09:33.866882+0000","last_peered":"2023-11-06T02:09:33.866882+0000","last_clean":"2023-11-06T02:09:33.866882+0000","last_became_active":"2023-11-06T02:08:26.056419+0000","last_became_peered":"2023-11-06T02:08:26.056419+0000","last_unstale":"2023-11-06T02:09:33.866882+0000","last_undegraded":"2023-11-06T02:09:33.866882+0000","last_fullsized":"2023-11-06T02:09:33.866882+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:10.626021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:39.469569+0000","last_change":"2023-11-06T02:09:39.469569+0000","last_active":"2023-11-06T02:09:39.469569+0000","last_peered":"2023-11-06T02:09:39.469569+0000","last_clean":"2023-11-06T02:09:39.469569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:39.469569+0000","last_undegraded":"2023-11-06T02:09:39.469569+0000","last_fullsized":"2023-11-06T02:09:39.469569+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:09:38.457583+0000","last_change":"2023-11-06T02:09:38.457583+0000","last_active":"2023-11-06T02:09:38.457583+0000","last_peered":"2023-11-06T02:09:38.457583+0000","last_clean":"2023-11-06T02:09:38.457583+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:09:38.457583+0000","last_undegraded":"2023-11-06T02:09:38.457583+0000","last_fullsized":"2023-11-06T02:09:38.457583+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":24,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-06T02:09:32.628412+0000","last_change":"2023-11-06T02:08:26.058596+0000","last_active":"2023-11-06T02:09:32.628412+0000","last_peered":"2023-11-06T02:09:32.628412+0000","last_clean":"2023-11-06T02:09:32.628412+0000","last_became_active":"2023-11-06T02:08:26.057015+0000","last_became_peered":"2023-11-06T02:08:26.057015+0000","last_unstale":"2023-11-06T02:09:32.628412+0000","last_undegraded":"2023-11-06T02:09:32.628412+0000","last_fullsized":"2023-11-06T02:09:32.628412+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:55:04.552404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574867,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343944,"statfs":{"total":96636764160,"available":96608198656,"internally_reserved":0,"allocated":966656,"data_stored":738995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":1.4650000000000001},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":1.3959999999999999}]},{"osd":1,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.94899999999999995},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.76300000000000001}]},{"osd":2,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":1.367},{"interface":"front","average":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":1.3420000000000001}]},{"osd":3,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":1.2969999999999999},{"interface":"front","average":{"1min":1.077,"5min":1.077,"15min":1.077},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":1.526}]},{"osd":4,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":1.163},{"interface":"front","average":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":1.115}]},{"osd":5,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.079,"15min":1.079},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":1.2450000000000001},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.7809999999999999,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":1.0760000000000001}]},{"osd":6,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.202,"15min":1.202},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.6679999999999999,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":1.4930000000000001},{"interface":"front","average":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.74,"5min":1.74,"15min":1.74},"last":1.421}]}]},{"osd":6,"up_from":13,"seq":55834574866,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":372736,"data_stored":148627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.65200000000000002}]},{"osd":1,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.68799999999999994}]},{"osd":2,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.28,"5min":1.28,"15min":1.28},"last":0.89300000000000002}]},{"osd":3,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.84099999999999997},{"interface":"front","average":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.79800000000000004}]},{"osd":4,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.474,"5min":1.474,"15min":1.474},"last":0.76900000000000002},{"interface":"front","average":{"1min":1.04,"5min":1.04,"15min":1.04},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":1.034}]},{"osd":5,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.365,"5min":1.365,"15min":1.365},"last":1.006},{"interface":"front","average":{"1min":1.115,"5min":1.115,"15min":1.115},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.97599999999999998}]},{"osd":7,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.93300000000000005},{"interface":"front","average":{"1min":1.165,"5min":1.165,"15min":1.165},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":1.0629999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574867,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":372736,"data_stored":148627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":1.038},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.6579999999999999,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":1.1739999999999999}]},{"osd":1,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":1.109}]},{"osd":2,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.377,"5min":1.377,"15min":1.377},"last":1.2909999999999999}]},{"osd":3,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":1.081,"5min":1.081,"15min":1.081},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.7250000000000001,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":0.67400000000000004}]},{"osd":4,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":1.214},{"interface":"front","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":1.0800000000000001}]},{"osd":6,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.97799999999999998},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.84199999999999997}]},{"osd":7,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":1.0089999999999999},{"interface":"front","average":{"1min":1.155,"5min":1.155,"15min":1.155},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.8089999999999999,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":1.2549999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27316,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":372736,"data_stored":148627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":1.0920000000000001},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.755}]},{"osd":1,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":1.077},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.94599999999999995}]},{"osd":2,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":1.02,"15min":1.02},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.615,"5min":1.615,"15min":1.615},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.83599999999999997}]},{"osd":3,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.52,"5min":1.52,"15min":1.52},"last":0.97899999999999998},{"interface":"front","average":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.4970000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.91800000000000004}]},{"osd":5,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"min":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":1.377,"5min":1.377,"15min":1.377},"last":1.1339999999999999},{"interface":"front","average":{"1min":1.056,"5min":1.056,"15min":1.056},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.7869999999999999,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":0.67000000000000004}]},{"osd":6,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":1.1180000000000001},{"interface":"front","average":{"1min":1.198,"5min":1.198,"15min":1.198},"min":{"1min":0.87,"5min":0.87,"15min":0.87},"max":{"1min":2.1190000000000002,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":1.0429999999999999}]},{"osd":7,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.7110000000000001,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.89000000000000001},{"interface":"front","average":{"1min":1.24,"5min":1.24,"15min":1.24},"min":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":1.7410000000000001,"5min":1.7410000000000001,"15min":1.7410000000000001},"last":1.014}]}]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":372736,"data_stored":148627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:09:19 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.45700000000000002}]},{"osd":1,"last update":"Mon Nov 6 02:09:19 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.78600000000000003}]},{"osd":2,"last update":"Mon Nov 6 02:09:19 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.61399999999999999}]},{"osd":4,"last update":"Mon Nov 6 02:09:19 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.405,"5min":1.405,"15min":1.405},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.85199999999999998}]},{"osd":5,"last update":"Mon Nov 6 02:09:19 2023","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.276,"5min":1.276,"15min":1.276},"last":0.97299999999999998},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":2.21,"5min":2.21,"15min":2.21},"last":0.88}]},{"osd":6,"last update":"Mon Nov 6 02:09:19 2023","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":1.026,"15min":1.026},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.67600000000000005},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":1.0609999999999999}]},{"osd":7,"last update":"Mon Nov 6 02:09:19 2023","interfaces":[{"interface":"back","average":{"1min":1.133,"5min":1.133,"15min":1.133},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":2.2829999999999999,"5min":2.2829999999999999,"15min":2.2829999999999999},"last":0.93999999999999995},{"interface":"front","average":{"1min":1.107,"5min":1.107,"15min":1.107},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":0.82199999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":372736,"data_stored":148627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.53000000000000003}]},{"osd":1,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.48599999999999999}]},{"osd":3,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.83099999999999996}]},{"osd":4,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.753},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.98299999999999998}]},{"osd":5,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.5369999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":1.1850000000000001},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.7110000000000001,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.94599999999999995}]},{"osd":6,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":1.0489999999999999},{"interface":"front","average":{"1min":0.997,"5min":0.997,"15min":0.997},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.677,"5min":1.677,"15min":1.677},"last":1.296}]},{"osd":7,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.633,"5min":1.633,"15min":1.633},"last":1.0880000000000001},{"interface":"front","average":{"1min":1.042,"5min":1.042,"15min":1.042},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":1.2509999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":372736,"data_stored":148627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:09:19 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.44500000000000001}]},{"osd":2,"last update":"Mon Nov 6 02:09:19 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.56699999999999995}]},{"osd":3,"last update":"Mon Nov 6 02:09:19 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.8149999999999999,"5min":1.8149999999999999,"15min":1.8149999999999999},"last":0.59899999999999998}]},{"osd":4,"last update":"Mon Nov 6 02:09:19 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.82999999999999996}]},{"osd":5,"last update":"Mon Nov 6 02:09:19 2023","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.90000000000000002},{"interface":"front","average":{"1min":1.079,"5min":1.079,"15min":1.079},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.6759999999999999,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":1.1120000000000001}]},{"osd":6,"last update":"Mon Nov 6 02:09:19 2023","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.103,"15min":1.103},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.742,"5min":1.742,"15min":1.742},"last":1.077},{"interface":"front","average":{"1min":1.077,"5min":1.077,"15min":1.077},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.393,"5min":1.393,"15min":1.393},"last":1.159}]},{"osd":7,"last update":"Mon Nov 6 02:09:19 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.486,"5min":1.486,"15min":1.486},"last":1.3620000000000001},{"interface":"front","average":{"1min":1.161,"5min":1.161,"15min":1.161},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.893,"5min":1.893,"15min":1.893},"last":1.226}]}]},{"osd":0,"up_from":13,"seq":55834574867,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31988,"kb_used_data":940,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339852,"statfs":{"total":96636764160,"available":96604008448,"internally_reserved":0,"allocated":962560,"data_stored":737976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.872}]},{"osd":2,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.64500000000000002}]},{"osd":3,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":1.0469999999999999}]},{"osd":4,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":1.085},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.97599999999999998}]},{"osd":5,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":1.137},{"interface":"front","average":{"1min":0.872,"5min":0.872,"15min":0.872},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.448,"5min":1.448,"15min":1.448},"last":1.0109999999999999}]},{"osd":6,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":1.3089999999999999},{"interface":"front","average":{"1min":1,"5min":1,"15min":1},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":1.234}]},{"osd":7,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":1.294},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.556,"5min":1.556,"15min":1.556},"last":1.173}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T02:09:44.921 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3f is not active+clean 2023-11-06T02:09:44.922 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:39.469569+0000', 'last_change': '2023-11-06T02:09:39.469569+0000', 'last_active': '2023-11-06T02:09:39.469569+0000', 'last_peered': '2023-11-06T02:09:39.469569+0000', 'last_clean': '2023-11-06T02:09:39.469569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:39.469569+0000', 'last_undegraded': '2023-11-06T02:09:39.469569+0000', 'last_fullsized': '2023-11-06T02:09:39.469569+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.922 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3e is not active+clean 2023-11-06T02:09:44.923 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:39.469569+0000', 'last_change': '2023-11-06T02:09:39.469569+0000', 'last_active': '2023-11-06T02:09:39.469569+0000', 'last_peered': '2023-11-06T02:09:39.469569+0000', 'last_clean': '2023-11-06T02:09:39.469569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:39.469569+0000', 'last_undegraded': '2023-11-06T02:09:39.469569+0000', 'last_fullsized': '2023-11-06T02:09:39.469569+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.923 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3d is not active+clean 2023-11-06T02:09:44.923 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:39.469569+0000', 'last_change': '2023-11-06T02:09:39.469569+0000', 'last_active': '2023-11-06T02:09:39.469569+0000', 'last_peered': '2023-11-06T02:09:39.469569+0000', 'last_clean': '2023-11-06T02:09:39.469569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:39.469569+0000', 'last_undegraded': '2023-11-06T02:09:39.469569+0000', 'last_fullsized': '2023-11-06T02:09:39.469569+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.924 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3c is not active+clean 2023-11-06T02:09:44.924 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:39.469569+0000', 'last_change': '2023-11-06T02:09:39.469569+0000', 'last_active': '2023-11-06T02:09:39.469569+0000', 'last_peered': '2023-11-06T02:09:39.469569+0000', 'last_clean': '2023-11-06T02:09:39.469569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:39.469569+0000', 'last_undegraded': '2023-11-06T02:09:39.469569+0000', 'last_fullsized': '2023-11-06T02:09:39.469569+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.925 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3b is not active+clean 2023-11-06T02:09:44.925 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:39.469569+0000', 'last_change': '2023-11-06T02:09:39.469569+0000', 'last_active': '2023-11-06T02:09:39.469569+0000', 'last_peered': '2023-11-06T02:09:39.469569+0000', 'last_clean': '2023-11-06T02:09:39.469569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:39.469569+0000', 'last_undegraded': '2023-11-06T02:09:39.469569+0000', 'last_fullsized': '2023-11-06T02:09:39.469569+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.925 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3a is not active+clean 2023-11-06T02:09:44.926 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:39.469569+0000', 'last_change': '2023-11-06T02:09:39.469569+0000', 'last_active': '2023-11-06T02:09:39.469569+0000', 'last_peered': '2023-11-06T02:09:39.469569+0000', 'last_clean': '2023-11-06T02:09:39.469569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:39.469569+0000', 'last_undegraded': '2023-11-06T02:09:39.469569+0000', 'last_fullsized': '2023-11-06T02:09:39.469569+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.926 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.39 is not active+clean 2023-11-06T02:09:44.927 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:39.469569+0000', 'last_change': '2023-11-06T02:09:39.469569+0000', 'last_active': '2023-11-06T02:09:39.469569+0000', 'last_peered': '2023-11-06T02:09:39.469569+0000', 'last_clean': '2023-11-06T02:09:39.469569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:39.469569+0000', 'last_undegraded': '2023-11-06T02:09:39.469569+0000', 'last_fullsized': '2023-11-06T02:09:39.469569+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.927 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.38 is not active+clean 2023-11-06T02:09:44.927 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:39.469569+0000', 'last_change': '2023-11-06T02:09:39.469569+0000', 'last_active': '2023-11-06T02:09:39.469569+0000', 'last_peered': '2023-11-06T02:09:39.469569+0000', 'last_clean': '2023-11-06T02:09:39.469569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:39.469569+0000', 'last_undegraded': '2023-11-06T02:09:39.469569+0000', 'last_fullsized': '2023-11-06T02:09:39.469569+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.928 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.37 is not active+clean 2023-11-06T02:09:44.928 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:39.469569+0000', 'last_change': '2023-11-06T02:09:39.469569+0000', 'last_active': '2023-11-06T02:09:39.469569+0000', 'last_peered': '2023-11-06T02:09:39.469569+0000', 'last_clean': '2023-11-06T02:09:39.469569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:39.469569+0000', 'last_undegraded': '2023-11-06T02:09:39.469569+0000', 'last_fullsized': '2023-11-06T02:09:39.469569+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.929 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.36 is not active+clean 2023-11-06T02:09:44.929 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:39.469569+0000', 'last_change': '2023-11-06T02:09:39.469569+0000', 'last_active': '2023-11-06T02:09:39.469569+0000', 'last_peered': '2023-11-06T02:09:39.469569+0000', 'last_clean': '2023-11-06T02:09:39.469569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:39.469569+0000', 'last_undegraded': '2023-11-06T02:09:39.469569+0000', 'last_fullsized': '2023-11-06T02:09:39.469569+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.929 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.35 is not active+clean 2023-11-06T02:09:44.930 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:39.469569+0000', 'last_change': '2023-11-06T02:09:39.469569+0000', 'last_active': '2023-11-06T02:09:39.469569+0000', 'last_peered': '2023-11-06T02:09:39.469569+0000', 'last_clean': '2023-11-06T02:09:39.469569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:39.469569+0000', 'last_undegraded': '2023-11-06T02:09:39.469569+0000', 'last_fullsized': '2023-11-06T02:09:39.469569+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.930 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.34 is not active+clean 2023-11-06T02:09:44.930 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:39.469569+0000', 'last_change': '2023-11-06T02:09:39.469569+0000', 'last_active': '2023-11-06T02:09:39.469569+0000', 'last_peered': '2023-11-06T02:09:39.469569+0000', 'last_clean': '2023-11-06T02:09:39.469569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:39.469569+0000', 'last_undegraded': '2023-11-06T02:09:39.469569+0000', 'last_fullsized': '2023-11-06T02:09:39.469569+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.931 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.33 is not active+clean 2023-11-06T02:09:44.931 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:39.469569+0000', 'last_change': '2023-11-06T02:09:39.469569+0000', 'last_active': '2023-11-06T02:09:39.469569+0000', 'last_peered': '2023-11-06T02:09:39.469569+0000', 'last_clean': '2023-11-06T02:09:39.469569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:39.469569+0000', 'last_undegraded': '2023-11-06T02:09:39.469569+0000', 'last_fullsized': '2023-11-06T02:09:39.469569+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.932 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.32 is not active+clean 2023-11-06T02:09:44.932 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:39.469569+0000', 'last_change': '2023-11-06T02:09:39.469569+0000', 'last_active': '2023-11-06T02:09:39.469569+0000', 'last_peered': '2023-11-06T02:09:39.469569+0000', 'last_clean': '2023-11-06T02:09:39.469569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:39.469569+0000', 'last_undegraded': '2023-11-06T02:09:39.469569+0000', 'last_fullsized': '2023-11-06T02:09:39.469569+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.933 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.31 is not active+clean 2023-11-06T02:09:44.933 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:39.469569+0000', 'last_change': '2023-11-06T02:09:39.469569+0000', 'last_active': '2023-11-06T02:09:39.469569+0000', 'last_peered': '2023-11-06T02:09:39.469569+0000', 'last_clean': '2023-11-06T02:09:39.469569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:39.469569+0000', 'last_undegraded': '2023-11-06T02:09:39.469569+0000', 'last_fullsized': '2023-11-06T02:09:39.469569+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.933 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.30 is not active+clean 2023-11-06T02:09:44.934 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:39.469569+0000', 'last_change': '2023-11-06T02:09:39.469569+0000', 'last_active': '2023-11-06T02:09:39.469569+0000', 'last_peered': '2023-11-06T02:09:39.469569+0000', 'last_clean': '2023-11-06T02:09:39.469569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:39.469569+0000', 'last_undegraded': '2023-11-06T02:09:39.469569+0000', 'last_fullsized': '2023-11-06T02:09:39.469569+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.934 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2f is not active+clean 2023-11-06T02:09:44.934 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:39.469569+0000', 'last_change': '2023-11-06T02:09:39.469569+0000', 'last_active': '2023-11-06T02:09:39.469569+0000', 'last_peered': '2023-11-06T02:09:39.469569+0000', 'last_clean': '2023-11-06T02:09:39.469569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:39.469569+0000', 'last_undegraded': '2023-11-06T02:09:39.469569+0000', 'last_fullsized': '2023-11-06T02:09:39.469569+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.935 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2e is not active+clean 2023-11-06T02:09:44.935 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:39.469569+0000', 'last_change': '2023-11-06T02:09:39.469569+0000', 'last_active': '2023-11-06T02:09:39.469569+0000', 'last_peered': '2023-11-06T02:09:39.469569+0000', 'last_clean': '2023-11-06T02:09:39.469569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:39.469569+0000', 'last_undegraded': '2023-11-06T02:09:39.469569+0000', 'last_fullsized': '2023-11-06T02:09:39.469569+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.936 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2d is not active+clean 2023-11-06T02:09:44.936 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:39.469569+0000', 'last_change': '2023-11-06T02:09:39.469569+0000', 'last_active': '2023-11-06T02:09:39.469569+0000', 'last_peered': '2023-11-06T02:09:39.469569+0000', 'last_clean': '2023-11-06T02:09:39.469569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:39.469569+0000', 'last_undegraded': '2023-11-06T02:09:39.469569+0000', 'last_fullsized': '2023-11-06T02:09:39.469569+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.936 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2c is not active+clean 2023-11-06T02:09:44.937 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:39.469569+0000', 'last_change': '2023-11-06T02:09:39.469569+0000', 'last_active': '2023-11-06T02:09:39.469569+0000', 'last_peered': '2023-11-06T02:09:39.469569+0000', 'last_clean': '2023-11-06T02:09:39.469569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:39.469569+0000', 'last_undegraded': '2023-11-06T02:09:39.469569+0000', 'last_fullsized': '2023-11-06T02:09:39.469569+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.937 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2b is not active+clean 2023-11-06T02:09:44.938 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:39.469569+0000', 'last_change': '2023-11-06T02:09:39.469569+0000', 'last_active': '2023-11-06T02:09:39.469569+0000', 'last_peered': '2023-11-06T02:09:39.469569+0000', 'last_clean': '2023-11-06T02:09:39.469569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:39.469569+0000', 'last_undegraded': '2023-11-06T02:09:39.469569+0000', 'last_fullsized': '2023-11-06T02:09:39.469569+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.938 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2a is not active+clean 2023-11-06T02:09:44.938 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:39.469569+0000', 'last_change': '2023-11-06T02:09:39.469569+0000', 'last_active': '2023-11-06T02:09:39.469569+0000', 'last_peered': '2023-11-06T02:09:39.469569+0000', 'last_clean': '2023-11-06T02:09:39.469569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:39.469569+0000', 'last_undegraded': '2023-11-06T02:09:39.469569+0000', 'last_fullsized': '2023-11-06T02:09:39.469569+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.939 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.29 is not active+clean 2023-11-06T02:09:44.939 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:39.469569+0000', 'last_change': '2023-11-06T02:09:39.469569+0000', 'last_active': '2023-11-06T02:09:39.469569+0000', 'last_peered': '2023-11-06T02:09:39.469569+0000', 'last_clean': '2023-11-06T02:09:39.469569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:39.469569+0000', 'last_undegraded': '2023-11-06T02:09:39.469569+0000', 'last_fullsized': '2023-11-06T02:09:39.469569+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.940 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.28 is not active+clean 2023-11-06T02:09:44.940 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:39.469569+0000', 'last_change': '2023-11-06T02:09:39.469569+0000', 'last_active': '2023-11-06T02:09:39.469569+0000', 'last_peered': '2023-11-06T02:09:39.469569+0000', 'last_clean': '2023-11-06T02:09:39.469569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:39.469569+0000', 'last_undegraded': '2023-11-06T02:09:39.469569+0000', 'last_fullsized': '2023-11-06T02:09:39.469569+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.940 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.27 is not active+clean 2023-11-06T02:09:44.941 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:39.469569+0000', 'last_change': '2023-11-06T02:09:39.469569+0000', 'last_active': '2023-11-06T02:09:39.469569+0000', 'last_peered': '2023-11-06T02:09:39.469569+0000', 'last_clean': '2023-11-06T02:09:39.469569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:39.469569+0000', 'last_undegraded': '2023-11-06T02:09:39.469569+0000', 'last_fullsized': '2023-11-06T02:09:39.469569+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.941 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.26 is not active+clean 2023-11-06T02:09:44.941 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:39.469569+0000', 'last_change': '2023-11-06T02:09:39.469569+0000', 'last_active': '2023-11-06T02:09:39.469569+0000', 'last_peered': '2023-11-06T02:09:39.469569+0000', 'last_clean': '2023-11-06T02:09:39.469569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:39.469569+0000', 'last_undegraded': '2023-11-06T02:09:39.469569+0000', 'last_fullsized': '2023-11-06T02:09:39.469569+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.942 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.25 is not active+clean 2023-11-06T02:09:44.942 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:39.469569+0000', 'last_change': '2023-11-06T02:09:39.469569+0000', 'last_active': '2023-11-06T02:09:39.469569+0000', 'last_peered': '2023-11-06T02:09:39.469569+0000', 'last_clean': '2023-11-06T02:09:39.469569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:39.469569+0000', 'last_undegraded': '2023-11-06T02:09:39.469569+0000', 'last_fullsized': '2023-11-06T02:09:39.469569+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.943 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.24 is not active+clean 2023-11-06T02:09:44.943 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:39.469569+0000', 'last_change': '2023-11-06T02:09:39.469569+0000', 'last_active': '2023-11-06T02:09:39.469569+0000', 'last_peered': '2023-11-06T02:09:39.469569+0000', 'last_clean': '2023-11-06T02:09:39.469569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:39.469569+0000', 'last_undegraded': '2023-11-06T02:09:39.469569+0000', 'last_fullsized': '2023-11-06T02:09:39.469569+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.944 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.23 is not active+clean 2023-11-06T02:09:44.944 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:39.469569+0000', 'last_change': '2023-11-06T02:09:39.469569+0000', 'last_active': '2023-11-06T02:09:39.469569+0000', 'last_peered': '2023-11-06T02:09:39.469569+0000', 'last_clean': '2023-11-06T02:09:39.469569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:39.469569+0000', 'last_undegraded': '2023-11-06T02:09:39.469569+0000', 'last_fullsized': '2023-11-06T02:09:39.469569+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.944 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.22 is not active+clean 2023-11-06T02:09:44.945 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:39.469569+0000', 'last_change': '2023-11-06T02:09:39.469569+0000', 'last_active': '2023-11-06T02:09:39.469569+0000', 'last_peered': '2023-11-06T02:09:39.469569+0000', 'last_clean': '2023-11-06T02:09:39.469569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:39.469569+0000', 'last_undegraded': '2023-11-06T02:09:39.469569+0000', 'last_fullsized': '2023-11-06T02:09:39.469569+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.945 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.21 is not active+clean 2023-11-06T02:09:44.945 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:39.469569+0000', 'last_change': '2023-11-06T02:09:39.469569+0000', 'last_active': '2023-11-06T02:09:39.469569+0000', 'last_peered': '2023-11-06T02:09:39.469569+0000', 'last_clean': '2023-11-06T02:09:39.469569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:39.469569+0000', 'last_undegraded': '2023-11-06T02:09:39.469569+0000', 'last_fullsized': '2023-11-06T02:09:39.469569+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.946 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.20 is not active+clean 2023-11-06T02:09:44.946 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:39.469569+0000', 'last_change': '2023-11-06T02:09:39.469569+0000', 'last_active': '2023-11-06T02:09:39.469569+0000', 'last_peered': '2023-11-06T02:09:39.469569+0000', 'last_clean': '2023-11-06T02:09:39.469569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:39.469569+0000', 'last_undegraded': '2023-11-06T02:09:39.469569+0000', 'last_fullsized': '2023-11-06T02:09:39.469569+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.947 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1c is not active+clean 2023-11-06T02:09:44.947 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:39.469569+0000', 'last_change': '2023-11-06T02:09:39.469569+0000', 'last_active': '2023-11-06T02:09:39.469569+0000', 'last_peered': '2023-11-06T02:09:39.469569+0000', 'last_clean': '2023-11-06T02:09:39.469569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:39.469569+0000', 'last_undegraded': '2023-11-06T02:09:39.469569+0000', 'last_fullsized': '2023-11-06T02:09:39.469569+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.947 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1f is not active+clean 2023-11-06T02:09:44.948 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:38.457583+0000', 'last_change': '2023-11-06T02:09:38.457583+0000', 'last_active': '2023-11-06T02:09:38.457583+0000', 'last_peered': '2023-11-06T02:09:38.457583+0000', 'last_clean': '2023-11-06T02:09:38.457583+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:38.457583+0000', 'last_undegraded': '2023-11-06T02:09:38.457583+0000', 'last_fullsized': '2023-11-06T02:09:38.457583+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.948 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1d is not active+clean 2023-11-06T02:09:44.948 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:39.469569+0000', 'last_change': '2023-11-06T02:09:39.469569+0000', 'last_active': '2023-11-06T02:09:39.469569+0000', 'last_peered': '2023-11-06T02:09:39.469569+0000', 'last_clean': '2023-11-06T02:09:39.469569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:39.469569+0000', 'last_undegraded': '2023-11-06T02:09:39.469569+0000', 'last_fullsized': '2023-11-06T02:09:39.469569+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.949 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1e is not active+clean 2023-11-06T02:09:44.949 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:38.457583+0000', 'last_change': '2023-11-06T02:09:38.457583+0000', 'last_active': '2023-11-06T02:09:38.457583+0000', 'last_peered': '2023-11-06T02:09:38.457583+0000', 'last_clean': '2023-11-06T02:09:38.457583+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:38.457583+0000', 'last_undegraded': '2023-11-06T02:09:38.457583+0000', 'last_fullsized': '2023-11-06T02:09:38.457583+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.950 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1e is not active+clean 2023-11-06T02:09:44.950 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:39.469569+0000', 'last_change': '2023-11-06T02:09:39.469569+0000', 'last_active': '2023-11-06T02:09:39.469569+0000', 'last_peered': '2023-11-06T02:09:39.469569+0000', 'last_clean': '2023-11-06T02:09:39.469569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:39.469569+0000', 'last_undegraded': '2023-11-06T02:09:39.469569+0000', 'last_fullsized': '2023-11-06T02:09:39.469569+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.951 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1d is not active+clean 2023-11-06T02:09:44.951 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:38.457583+0000', 'last_change': '2023-11-06T02:09:38.457583+0000', 'last_active': '2023-11-06T02:09:38.457583+0000', 'last_peered': '2023-11-06T02:09:38.457583+0000', 'last_clean': '2023-11-06T02:09:38.457583+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:38.457583+0000', 'last_undegraded': '2023-11-06T02:09:38.457583+0000', 'last_fullsized': '2023-11-06T02:09:38.457583+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.951 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1f is not active+clean 2023-11-06T02:09:44.952 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:39.469569+0000', 'last_change': '2023-11-06T02:09:39.469569+0000', 'last_active': '2023-11-06T02:09:39.469569+0000', 'last_peered': '2023-11-06T02:09:39.469569+0000', 'last_clean': '2023-11-06T02:09:39.469569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:39.469569+0000', 'last_undegraded': '2023-11-06T02:09:39.469569+0000', 'last_fullsized': '2023-11-06T02:09:39.469569+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.952 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1c is not active+clean 2023-11-06T02:09:44.952 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:38.457583+0000', 'last_change': '2023-11-06T02:09:38.457583+0000', 'last_active': '2023-11-06T02:09:38.457583+0000', 'last_peered': '2023-11-06T02:09:38.457583+0000', 'last_clean': '2023-11-06T02:09:38.457583+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:38.457583+0000', 'last_undegraded': '2023-11-06T02:09:38.457583+0000', 'last_fullsized': '2023-11-06T02:09:38.457583+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.953 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.18 is not active+clean 2023-11-06T02:09:44.953 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:39.469569+0000', 'last_change': '2023-11-06T02:09:39.469569+0000', 'last_active': '2023-11-06T02:09:39.469569+0000', 'last_peered': '2023-11-06T02:09:39.469569+0000', 'last_clean': '2023-11-06T02:09:39.469569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:39.469569+0000', 'last_undegraded': '2023-11-06T02:09:39.469569+0000', 'last_fullsized': '2023-11-06T02:09:39.469569+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.954 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1b is not active+clean 2023-11-06T02:09:44.954 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:38.457583+0000', 'last_change': '2023-11-06T02:09:38.457583+0000', 'last_active': '2023-11-06T02:09:38.457583+0000', 'last_peered': '2023-11-06T02:09:38.457583+0000', 'last_clean': '2023-11-06T02:09:38.457583+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:38.457583+0000', 'last_undegraded': '2023-11-06T02:09:38.457583+0000', 'last_fullsized': '2023-11-06T02:09:38.457583+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.954 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.19 is not active+clean 2023-11-06T02:09:44.955 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:39.469569+0000', 'last_change': '2023-11-06T02:09:39.469569+0000', 'last_active': '2023-11-06T02:09:39.469569+0000', 'last_peered': '2023-11-06T02:09:39.469569+0000', 'last_clean': '2023-11-06T02:09:39.469569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:39.469569+0000', 'last_undegraded': '2023-11-06T02:09:39.469569+0000', 'last_fullsized': '2023-11-06T02:09:39.469569+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.955 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1a is not active+clean 2023-11-06T02:09:44.955 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:38.457583+0000', 'last_change': '2023-11-06T02:09:38.457583+0000', 'last_active': '2023-11-06T02:09:38.457583+0000', 'last_peered': '2023-11-06T02:09:38.457583+0000', 'last_clean': '2023-11-06T02:09:38.457583+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:38.457583+0000', 'last_undegraded': '2023-11-06T02:09:38.457583+0000', 'last_fullsized': '2023-11-06T02:09:38.457583+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.956 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1a is not active+clean 2023-11-06T02:09:44.956 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:39.469569+0000', 'last_change': '2023-11-06T02:09:39.469569+0000', 'last_active': '2023-11-06T02:09:39.469569+0000', 'last_peered': '2023-11-06T02:09:39.469569+0000', 'last_clean': '2023-11-06T02:09:39.469569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:39.469569+0000', 'last_undegraded': '2023-11-06T02:09:39.469569+0000', 'last_fullsized': '2023-11-06T02:09:39.469569+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.957 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.19 is not active+clean 2023-11-06T02:09:44.957 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:38.457583+0000', 'last_change': '2023-11-06T02:09:38.457583+0000', 'last_active': '2023-11-06T02:09:38.457583+0000', 'last_peered': '2023-11-06T02:09:38.457583+0000', 'last_clean': '2023-11-06T02:09:38.457583+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:38.457583+0000', 'last_undegraded': '2023-11-06T02:09:38.457583+0000', 'last_fullsized': '2023-11-06T02:09:38.457583+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.957 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1b is not active+clean 2023-11-06T02:09:44.958 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:39.469569+0000', 'last_change': '2023-11-06T02:09:39.469569+0000', 'last_active': '2023-11-06T02:09:39.469569+0000', 'last_peered': '2023-11-06T02:09:39.469569+0000', 'last_clean': '2023-11-06T02:09:39.469569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:39.469569+0000', 'last_undegraded': '2023-11-06T02:09:39.469569+0000', 'last_fullsized': '2023-11-06T02:09:39.469569+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.958 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.18 is not active+clean 2023-11-06T02:09:44.959 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:38.457583+0000', 'last_change': '2023-11-06T02:09:38.457583+0000', 'last_active': '2023-11-06T02:09:38.457583+0000', 'last_peered': '2023-11-06T02:09:38.457583+0000', 'last_clean': '2023-11-06T02:09:38.457583+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:38.457583+0000', 'last_undegraded': '2023-11-06T02:09:38.457583+0000', 'last_fullsized': '2023-11-06T02:09:38.457583+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.959 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.14 is not active+clean 2023-11-06T02:09:44.959 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:39.469569+0000', 'last_change': '2023-11-06T02:09:39.469569+0000', 'last_active': '2023-11-06T02:09:39.469569+0000', 'last_peered': '2023-11-06T02:09:39.469569+0000', 'last_clean': '2023-11-06T02:09:39.469569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:39.469569+0000', 'last_undegraded': '2023-11-06T02:09:39.469569+0000', 'last_fullsized': '2023-11-06T02:09:39.469569+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.960 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.17 is not active+clean 2023-11-06T02:09:44.960 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:38.457583+0000', 'last_change': '2023-11-06T02:09:38.457583+0000', 'last_active': '2023-11-06T02:09:38.457583+0000', 'last_peered': '2023-11-06T02:09:38.457583+0000', 'last_clean': '2023-11-06T02:09:38.457583+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:38.457583+0000', 'last_undegraded': '2023-11-06T02:09:38.457583+0000', 'last_fullsized': '2023-11-06T02:09:38.457583+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.960 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.15 is not active+clean 2023-11-06T02:09:44.961 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:39.469569+0000', 'last_change': '2023-11-06T02:09:39.469569+0000', 'last_active': '2023-11-06T02:09:39.469569+0000', 'last_peered': '2023-11-06T02:09:39.469569+0000', 'last_clean': '2023-11-06T02:09:39.469569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:39.469569+0000', 'last_undegraded': '2023-11-06T02:09:39.469569+0000', 'last_fullsized': '2023-11-06T02:09:39.469569+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.961 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.16 is not active+clean 2023-11-06T02:09:44.962 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:38.457583+0000', 'last_change': '2023-11-06T02:09:38.457583+0000', 'last_active': '2023-11-06T02:09:38.457583+0000', 'last_peered': '2023-11-06T02:09:38.457583+0000', 'last_clean': '2023-11-06T02:09:38.457583+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:38.457583+0000', 'last_undegraded': '2023-11-06T02:09:38.457583+0000', 'last_fullsized': '2023-11-06T02:09:38.457583+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.962 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.16 is not active+clean 2023-11-06T02:09:44.962 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:39.469569+0000', 'last_change': '2023-11-06T02:09:39.469569+0000', 'last_active': '2023-11-06T02:09:39.469569+0000', 'last_peered': '2023-11-06T02:09:39.469569+0000', 'last_clean': '2023-11-06T02:09:39.469569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:39.469569+0000', 'last_undegraded': '2023-11-06T02:09:39.469569+0000', 'last_fullsized': '2023-11-06T02:09:39.469569+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.963 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.15 is not active+clean 2023-11-06T02:09:44.963 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:38.457583+0000', 'last_change': '2023-11-06T02:09:38.457583+0000', 'last_active': '2023-11-06T02:09:38.457583+0000', 'last_peered': '2023-11-06T02:09:38.457583+0000', 'last_clean': '2023-11-06T02:09:38.457583+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:38.457583+0000', 'last_undegraded': '2023-11-06T02:09:38.457583+0000', 'last_fullsized': '2023-11-06T02:09:38.457583+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.964 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.17 is not active+clean 2023-11-06T02:09:44.964 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:39.469569+0000', 'last_change': '2023-11-06T02:09:39.469569+0000', 'last_active': '2023-11-06T02:09:39.469569+0000', 'last_peered': '2023-11-06T02:09:39.469569+0000', 'last_clean': '2023-11-06T02:09:39.469569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:39.469569+0000', 'last_undegraded': '2023-11-06T02:09:39.469569+0000', 'last_fullsized': '2023-11-06T02:09:39.469569+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.964 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.14 is not active+clean 2023-11-06T02:09:44.965 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:38.457583+0000', 'last_change': '2023-11-06T02:09:38.457583+0000', 'last_active': '2023-11-06T02:09:38.457583+0000', 'last_peered': '2023-11-06T02:09:38.457583+0000', 'last_clean': '2023-11-06T02:09:38.457583+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:38.457583+0000', 'last_undegraded': '2023-11-06T02:09:38.457583+0000', 'last_fullsized': '2023-11-06T02:09:38.457583+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.965 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.10 is not active+clean 2023-11-06T02:09:44.965 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:39.469569+0000', 'last_change': '2023-11-06T02:09:39.469569+0000', 'last_active': '2023-11-06T02:09:39.469569+0000', 'last_peered': '2023-11-06T02:09:39.469569+0000', 'last_clean': '2023-11-06T02:09:39.469569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:39.469569+0000', 'last_undegraded': '2023-11-06T02:09:39.469569+0000', 'last_fullsized': '2023-11-06T02:09:39.469569+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.966 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.13 is not active+clean 2023-11-06T02:09:44.966 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:38.457583+0000', 'last_change': '2023-11-06T02:09:38.457583+0000', 'last_active': '2023-11-06T02:09:38.457583+0000', 'last_peered': '2023-11-06T02:09:38.457583+0000', 'last_clean': '2023-11-06T02:09:38.457583+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:38.457583+0000', 'last_undegraded': '2023-11-06T02:09:38.457583+0000', 'last_fullsized': '2023-11-06T02:09:38.457583+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.967 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.11 is not active+clean 2023-11-06T02:09:44.967 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:39.469569+0000', 'last_change': '2023-11-06T02:09:39.469569+0000', 'last_active': '2023-11-06T02:09:39.469569+0000', 'last_peered': '2023-11-06T02:09:39.469569+0000', 'last_clean': '2023-11-06T02:09:39.469569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:39.469569+0000', 'last_undegraded': '2023-11-06T02:09:39.469569+0000', 'last_fullsized': '2023-11-06T02:09:39.469569+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.967 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.12 is not active+clean 2023-11-06T02:09:44.968 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:38.457583+0000', 'last_change': '2023-11-06T02:09:38.457583+0000', 'last_active': '2023-11-06T02:09:38.457583+0000', 'last_peered': '2023-11-06T02:09:38.457583+0000', 'last_clean': '2023-11-06T02:09:38.457583+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:38.457583+0000', 'last_undegraded': '2023-11-06T02:09:38.457583+0000', 'last_fullsized': '2023-11-06T02:09:38.457583+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.968 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.12 is not active+clean 2023-11-06T02:09:44.969 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:39.469569+0000', 'last_change': '2023-11-06T02:09:39.469569+0000', 'last_active': '2023-11-06T02:09:39.469569+0000', 'last_peered': '2023-11-06T02:09:39.469569+0000', 'last_clean': '2023-11-06T02:09:39.469569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:39.469569+0000', 'last_undegraded': '2023-11-06T02:09:39.469569+0000', 'last_fullsized': '2023-11-06T02:09:39.469569+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.969 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.11 is not active+clean 2023-11-06T02:09:44.969 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:38.457583+0000', 'last_change': '2023-11-06T02:09:38.457583+0000', 'last_active': '2023-11-06T02:09:38.457583+0000', 'last_peered': '2023-11-06T02:09:38.457583+0000', 'last_clean': '2023-11-06T02:09:38.457583+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:38.457583+0000', 'last_undegraded': '2023-11-06T02:09:38.457583+0000', 'last_fullsized': '2023-11-06T02:09:38.457583+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.970 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.13 is not active+clean 2023-11-06T02:09:44.970 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:39.469569+0000', 'last_change': '2023-11-06T02:09:39.469569+0000', 'last_active': '2023-11-06T02:09:39.469569+0000', 'last_peered': '2023-11-06T02:09:39.469569+0000', 'last_clean': '2023-11-06T02:09:39.469569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:39.469569+0000', 'last_undegraded': '2023-11-06T02:09:39.469569+0000', 'last_fullsized': '2023-11-06T02:09:39.469569+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.971 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.10 is not active+clean 2023-11-06T02:09:44.971 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:38.457583+0000', 'last_change': '2023-11-06T02:09:38.457583+0000', 'last_active': '2023-11-06T02:09:38.457583+0000', 'last_peered': '2023-11-06T02:09:38.457583+0000', 'last_clean': '2023-11-06T02:09:38.457583+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:38.457583+0000', 'last_undegraded': '2023-11-06T02:09:38.457583+0000', 'last_fullsized': '2023-11-06T02:09:38.457583+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.971 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.c is not active+clean 2023-11-06T02:09:44.972 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:39.469569+0000', 'last_change': '2023-11-06T02:09:39.469569+0000', 'last_active': '2023-11-06T02:09:39.469569+0000', 'last_peered': '2023-11-06T02:09:39.469569+0000', 'last_clean': '2023-11-06T02:09:39.469569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:39.469569+0000', 'last_undegraded': '2023-11-06T02:09:39.469569+0000', 'last_fullsized': '2023-11-06T02:09:39.469569+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.972 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.f is not active+clean 2023-11-06T02:09:44.972 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:38.457583+0000', 'last_change': '2023-11-06T02:09:38.457583+0000', 'last_active': '2023-11-06T02:09:38.457583+0000', 'last_peered': '2023-11-06T02:09:38.457583+0000', 'last_clean': '2023-11-06T02:09:38.457583+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:38.457583+0000', 'last_undegraded': '2023-11-06T02:09:38.457583+0000', 'last_fullsized': '2023-11-06T02:09:38.457583+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.973 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.d is not active+clean 2023-11-06T02:09:44.973 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:39.469569+0000', 'last_change': '2023-11-06T02:09:39.469569+0000', 'last_active': '2023-11-06T02:09:39.469569+0000', 'last_peered': '2023-11-06T02:09:39.469569+0000', 'last_clean': '2023-11-06T02:09:39.469569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:39.469569+0000', 'last_undegraded': '2023-11-06T02:09:39.469569+0000', 'last_fullsized': '2023-11-06T02:09:39.469569+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.974 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.e is not active+clean 2023-11-06T02:09:44.974 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:38.457583+0000', 'last_change': '2023-11-06T02:09:38.457583+0000', 'last_active': '2023-11-06T02:09:38.457583+0000', 'last_peered': '2023-11-06T02:09:38.457583+0000', 'last_clean': '2023-11-06T02:09:38.457583+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:38.457583+0000', 'last_undegraded': '2023-11-06T02:09:38.457583+0000', 'last_fullsized': '2023-11-06T02:09:38.457583+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.974 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.e is not active+clean 2023-11-06T02:09:44.975 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:39.469569+0000', 'last_change': '2023-11-06T02:09:39.469569+0000', 'last_active': '2023-11-06T02:09:39.469569+0000', 'last_peered': '2023-11-06T02:09:39.469569+0000', 'last_clean': '2023-11-06T02:09:39.469569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:39.469569+0000', 'last_undegraded': '2023-11-06T02:09:39.469569+0000', 'last_fullsized': '2023-11-06T02:09:39.469569+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.975 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.d is not active+clean 2023-11-06T02:09:44.975 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:38.457583+0000', 'last_change': '2023-11-06T02:09:38.457583+0000', 'last_active': '2023-11-06T02:09:38.457583+0000', 'last_peered': '2023-11-06T02:09:38.457583+0000', 'last_clean': '2023-11-06T02:09:38.457583+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:38.457583+0000', 'last_undegraded': '2023-11-06T02:09:38.457583+0000', 'last_fullsized': '2023-11-06T02:09:38.457583+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.976 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.f is not active+clean 2023-11-06T02:09:44.976 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:39.469569+0000', 'last_change': '2023-11-06T02:09:39.469569+0000', 'last_active': '2023-11-06T02:09:39.469569+0000', 'last_peered': '2023-11-06T02:09:39.469569+0000', 'last_clean': '2023-11-06T02:09:39.469569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:39.469569+0000', 'last_undegraded': '2023-11-06T02:09:39.469569+0000', 'last_fullsized': '2023-11-06T02:09:39.469569+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.977 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.c is not active+clean 2023-11-06T02:09:44.977 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:38.457583+0000', 'last_change': '2023-11-06T02:09:38.457583+0000', 'last_active': '2023-11-06T02:09:38.457583+0000', 'last_peered': '2023-11-06T02:09:38.457583+0000', 'last_clean': '2023-11-06T02:09:38.457583+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:38.457583+0000', 'last_undegraded': '2023-11-06T02:09:38.457583+0000', 'last_fullsized': '2023-11-06T02:09:38.457583+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.977 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.8 is not active+clean 2023-11-06T02:09:44.978 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:39.469569+0000', 'last_change': '2023-11-06T02:09:39.469569+0000', 'last_active': '2023-11-06T02:09:39.469569+0000', 'last_peered': '2023-11-06T02:09:39.469569+0000', 'last_clean': '2023-11-06T02:09:39.469569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:39.469569+0000', 'last_undegraded': '2023-11-06T02:09:39.469569+0000', 'last_fullsized': '2023-11-06T02:09:39.469569+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.978 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.b is not active+clean 2023-11-06T02:09:44.979 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:38.457583+0000', 'last_change': '2023-11-06T02:09:38.457583+0000', 'last_active': '2023-11-06T02:09:38.457583+0000', 'last_peered': '2023-11-06T02:09:38.457583+0000', 'last_clean': '2023-11-06T02:09:38.457583+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:38.457583+0000', 'last_undegraded': '2023-11-06T02:09:38.457583+0000', 'last_fullsized': '2023-11-06T02:09:38.457583+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.979 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.9 is not active+clean 2023-11-06T02:09:44.979 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:39.469569+0000', 'last_change': '2023-11-06T02:09:39.469569+0000', 'last_active': '2023-11-06T02:09:39.469569+0000', 'last_peered': '2023-11-06T02:09:39.469569+0000', 'last_clean': '2023-11-06T02:09:39.469569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:39.469569+0000', 'last_undegraded': '2023-11-06T02:09:39.469569+0000', 'last_fullsized': '2023-11-06T02:09:39.469569+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.980 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.a is not active+clean 2023-11-06T02:09:44.980 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:38.457583+0000', 'last_change': '2023-11-06T02:09:38.457583+0000', 'last_active': '2023-11-06T02:09:38.457583+0000', 'last_peered': '2023-11-06T02:09:38.457583+0000', 'last_clean': '2023-11-06T02:09:38.457583+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:38.457583+0000', 'last_undegraded': '2023-11-06T02:09:38.457583+0000', 'last_fullsized': '2023-11-06T02:09:38.457583+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.981 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.a is not active+clean 2023-11-06T02:09:44.981 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:39.469569+0000', 'last_change': '2023-11-06T02:09:39.469569+0000', 'last_active': '2023-11-06T02:09:39.469569+0000', 'last_peered': '2023-11-06T02:09:39.469569+0000', 'last_clean': '2023-11-06T02:09:39.469569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:39.469569+0000', 'last_undegraded': '2023-11-06T02:09:39.469569+0000', 'last_fullsized': '2023-11-06T02:09:39.469569+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.981 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.9 is not active+clean 2023-11-06T02:09:44.982 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:38.457583+0000', 'last_change': '2023-11-06T02:09:38.457583+0000', 'last_active': '2023-11-06T02:09:38.457583+0000', 'last_peered': '2023-11-06T02:09:38.457583+0000', 'last_clean': '2023-11-06T02:09:38.457583+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:38.457583+0000', 'last_undegraded': '2023-11-06T02:09:38.457583+0000', 'last_fullsized': '2023-11-06T02:09:38.457583+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.982 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.b is not active+clean 2023-11-06T02:09:44.982 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:39.469569+0000', 'last_change': '2023-11-06T02:09:39.469569+0000', 'last_active': '2023-11-06T02:09:39.469569+0000', 'last_peered': '2023-11-06T02:09:39.469569+0000', 'last_clean': '2023-11-06T02:09:39.469569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:39.469569+0000', 'last_undegraded': '2023-11-06T02:09:39.469569+0000', 'last_fullsized': '2023-11-06T02:09:39.469569+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.983 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.8 is not active+clean 2023-11-06T02:09:44.983 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:38.457583+0000', 'last_change': '2023-11-06T02:09:38.457583+0000', 'last_active': '2023-11-06T02:09:38.457583+0000', 'last_peered': '2023-11-06T02:09:38.457583+0000', 'last_clean': '2023-11-06T02:09:38.457583+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:38.457583+0000', 'last_undegraded': '2023-11-06T02:09:38.457583+0000', 'last_fullsized': '2023-11-06T02:09:38.457583+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.984 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.7 is not active+clean 2023-11-06T02:09:44.984 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:39.469569+0000', 'last_change': '2023-11-06T02:09:39.469569+0000', 'last_active': '2023-11-06T02:09:39.469569+0000', 'last_peered': '2023-11-06T02:09:39.469569+0000', 'last_clean': '2023-11-06T02:09:39.469569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:39.469569+0000', 'last_undegraded': '2023-11-06T02:09:39.469569+0000', 'last_fullsized': '2023-11-06T02:09:39.469569+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.984 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.4 is not active+clean 2023-11-06T02:09:44.985 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:38.457583+0000', 'last_change': '2023-11-06T02:09:38.457583+0000', 'last_active': '2023-11-06T02:09:38.457583+0000', 'last_peered': '2023-11-06T02:09:38.457583+0000', 'last_clean': '2023-11-06T02:09:38.457583+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:38.457583+0000', 'last_undegraded': '2023-11-06T02:09:38.457583+0000', 'last_fullsized': '2023-11-06T02:09:38.457583+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.985 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.4 is not active+clean 2023-11-06T02:09:44.986 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:39.469569+0000', 'last_change': '2023-11-06T02:09:39.469569+0000', 'last_active': '2023-11-06T02:09:39.469569+0000', 'last_peered': '2023-11-06T02:09:39.469569+0000', 'last_clean': '2023-11-06T02:09:39.469569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:39.469569+0000', 'last_undegraded': '2023-11-06T02:09:39.469569+0000', 'last_fullsized': '2023-11-06T02:09:39.469569+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.986 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.7 is not active+clean 2023-11-06T02:09:44.986 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:38.457583+0000', 'last_change': '2023-11-06T02:09:38.457583+0000', 'last_active': '2023-11-06T02:09:38.457583+0000', 'last_peered': '2023-11-06T02:09:38.457583+0000', 'last_clean': '2023-11-06T02:09:38.457583+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:38.457583+0000', 'last_undegraded': '2023-11-06T02:09:38.457583+0000', 'last_fullsized': '2023-11-06T02:09:38.457583+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.987 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.5 is not active+clean 2023-11-06T02:09:44.987 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:39.469569+0000', 'last_change': '2023-11-06T02:09:39.469569+0000', 'last_active': '2023-11-06T02:09:39.469569+0000', 'last_peered': '2023-11-06T02:09:39.469569+0000', 'last_clean': '2023-11-06T02:09:39.469569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:39.469569+0000', 'last_undegraded': '2023-11-06T02:09:39.469569+0000', 'last_fullsized': '2023-11-06T02:09:39.469569+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.988 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.6 is not active+clean 2023-11-06T02:09:44.988 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:38.457583+0000', 'last_change': '2023-11-06T02:09:38.457583+0000', 'last_active': '2023-11-06T02:09:38.457583+0000', 'last_peered': '2023-11-06T02:09:38.457583+0000', 'last_clean': '2023-11-06T02:09:38.457583+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:38.457583+0000', 'last_undegraded': '2023-11-06T02:09:38.457583+0000', 'last_fullsized': '2023-11-06T02:09:38.457583+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.988 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.6 is not active+clean 2023-11-06T02:09:44.989 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:39.469569+0000', 'last_change': '2023-11-06T02:09:39.469569+0000', 'last_active': '2023-11-06T02:09:39.469569+0000', 'last_peered': '2023-11-06T02:09:39.469569+0000', 'last_clean': '2023-11-06T02:09:39.469569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:39.469569+0000', 'last_undegraded': '2023-11-06T02:09:39.469569+0000', 'last_fullsized': '2023-11-06T02:09:39.469569+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.989 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.5 is not active+clean 2023-11-06T02:09:44.990 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:38.457583+0000', 'last_change': '2023-11-06T02:09:38.457583+0000', 'last_active': '2023-11-06T02:09:38.457583+0000', 'last_peered': '2023-11-06T02:09:38.457583+0000', 'last_clean': '2023-11-06T02:09:38.457583+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:38.457583+0000', 'last_undegraded': '2023-11-06T02:09:38.457583+0000', 'last_fullsized': '2023-11-06T02:09:38.457583+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.990 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.0 is not active+clean 2023-11-06T02:09:44.990 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:39.469569+0000', 'last_change': '2023-11-06T02:09:39.469569+0000', 'last_active': '2023-11-06T02:09:39.469569+0000', 'last_peered': '2023-11-06T02:09:39.469569+0000', 'last_clean': '2023-11-06T02:09:39.469569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:39.469569+0000', 'last_undegraded': '2023-11-06T02:09:39.469569+0000', 'last_fullsized': '2023-11-06T02:09:39.469569+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.991 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3 is not active+clean 2023-11-06T02:09:44.991 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:38.457583+0000', 'last_change': '2023-11-06T02:09:38.457583+0000', 'last_active': '2023-11-06T02:09:38.457583+0000', 'last_peered': '2023-11-06T02:09:38.457583+0000', 'last_clean': '2023-11-06T02:09:38.457583+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:38.457583+0000', 'last_undegraded': '2023-11-06T02:09:38.457583+0000', 'last_fullsized': '2023-11-06T02:09:38.457583+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.992 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1 is not active+clean 2023-11-06T02:09:44.992 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:39.469569+0000', 'last_change': '2023-11-06T02:09:39.469569+0000', 'last_active': '2023-11-06T02:09:39.469569+0000', 'last_peered': '2023-11-06T02:09:39.469569+0000', 'last_clean': '2023-11-06T02:09:39.469569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:39.469569+0000', 'last_undegraded': '2023-11-06T02:09:39.469569+0000', 'last_fullsized': '2023-11-06T02:09:39.469569+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.992 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2 is not active+clean 2023-11-06T02:09:44.993 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:38.457583+0000', 'last_change': '2023-11-06T02:09:38.457583+0000', 'last_active': '2023-11-06T02:09:38.457583+0000', 'last_peered': '2023-11-06T02:09:38.457583+0000', 'last_clean': '2023-11-06T02:09:38.457583+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:38.457583+0000', 'last_undegraded': '2023-11-06T02:09:38.457583+0000', 'last_fullsized': '2023-11-06T02:09:38.457583+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.993 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2 is not active+clean 2023-11-06T02:09:44.994 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:39.469569+0000', 'last_change': '2023-11-06T02:09:39.469569+0000', 'last_active': '2023-11-06T02:09:39.469569+0000', 'last_peered': '2023-11-06T02:09:39.469569+0000', 'last_clean': '2023-11-06T02:09:39.469569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:39.469569+0000', 'last_undegraded': '2023-11-06T02:09:39.469569+0000', 'last_fullsized': '2023-11-06T02:09:39.469569+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.994 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1 is not active+clean 2023-11-06T02:09:44.994 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:38.457583+0000', 'last_change': '2023-11-06T02:09:38.457583+0000', 'last_active': '2023-11-06T02:09:38.457583+0000', 'last_peered': '2023-11-06T02:09:38.457583+0000', 'last_clean': '2023-11-06T02:09:38.457583+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:38.457583+0000', 'last_undegraded': '2023-11-06T02:09:38.457583+0000', 'last_fullsized': '2023-11-06T02:09:38.457583+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.995 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3 is not active+clean 2023-11-06T02:09:44.995 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:39.469569+0000', 'last_change': '2023-11-06T02:09:39.469569+0000', 'last_active': '2023-11-06T02:09:39.469569+0000', 'last_peered': '2023-11-06T02:09:39.469569+0000', 'last_clean': '2023-11-06T02:09:39.469569+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:39.469569+0000', 'last_undegraded': '2023-11-06T02:09:39.469569+0000', 'last_fullsized': '2023-11-06T02:09:39.469569+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:39.469569+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.995 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.0 is not active+clean 2023-11-06T02:09:44.996 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:09:38.457583+0000', 'last_change': '2023-11-06T02:09:38.457583+0000', 'last_active': '2023-11-06T02:09:38.457583+0000', 'last_peered': '2023-11-06T02:09:38.457583+0000', 'last_clean': '2023-11-06T02:09:38.457583+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:09:38.457583+0000', 'last_undegraded': '2023-11-06T02:09:38.457583+0000', 'last_fullsized': '2023-11-06T02:09:38.457583+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'last_clean_scrub_stamp': '2023-11-06T02:09:38.457583+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:09:44.997 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-11-06T02:09:45.120 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.120+0000 7f4d2d38a640 1 Processor -- start 2023-11-06T02:09:45.120 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.120+0000 7f4d2d38a640 1 -- start start 2023-11-06T02:09:45.123 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.120+0000 7f4d2d38a640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f4d280b0790 0x7f4d280cb110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:45.123 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.120+0000 7f4d2d38a640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d280cb840 con 0x7f4d2815aae0 2023-11-06T02:09:45.123 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.120+0000 7f4d2d38a640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d280cbb20 con 0x7f4d28070a00 2023-11-06T02:09:45.123 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.120+0000 7f4d2d38a640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d2815aec0 con 0x7f4d280b0790 2023-11-06T02:09:45.124 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.120+0000 7f4d26ffd640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f4d2815aae0 0x7f4d280b0080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:48194/0 (socket says 172.21.15.49:48194) 2023-11-06T02:09:45.124 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.120+0000 7f4d26ffd640 1 -- 172.21.15.49:0/2676838087 learned_addr learned my addr 172.21.15.49:0/2676838087 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:09:45.124 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.124+0000 7f4d267fc640 1 --2- 172.21.15.49:0/2676838087 >> v2:172.21.15.150:3300/0 conn(0x7f4d280b0790 0x7f4d280cb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:45.124 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.124+0000 7f4d25ffb640 1 -- 172.21.15.49:0/2676838087 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3848480166 0 0) 0x7f4d280cb840 con 0x7f4d2815aae0 2023-11-06T02:09:45.124 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.124+0000 7f4d25ffb640 1 -- 172.21.15.49:0/2676838087 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d14003530 con 0x7f4d2815aae0 2023-11-06T02:09:45.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.124+0000 7f4d25ffb640 1 -- 172.21.15.49:0/2676838087 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3285546249 0 0) 0x7f4d14003530 con 0x7f4d2815aae0 2023-11-06T02:09:45.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.124+0000 7f4d25ffb640 1 -- 172.21.15.49:0/2676838087 >> v2:172.21.15.150:3300/0 conn(0x7f4d280b0790 msgr2=0x7f4d280cb110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:45.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.124+0000 7f4d25ffb640 1 --2- 172.21.15.49:0/2676838087 >> v2:172.21.15.150:3300/0 conn(0x7f4d280b0790 0x7f4d280cb110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:45.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.124+0000 7f4d25ffb640 1 -- 172.21.15.49:0/2676838087 >> v1:172.21.15.150:6789/0 conn(0x7f4d28070a00 legacy=0x7f4d280b1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:45.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.124+0000 7f4d25ffb640 1 -- 172.21.15.49:0/2676838087 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d280cbe00 con 0x7f4d2815aae0 2023-11-06T02:09:45.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.124+0000 7f4d25ffb640 1 -- 172.21.15.49:0/2676838087 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7f4d18001800 con 0x7f4d2815aae0 2023-11-06T02:09:45.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.124+0000 7f4d25ffb640 1 -- 172.21.15.49:0/2676838087 >> v1:172.21.15.49:6789/0 conn(0x7f4d2815aae0 legacy=0x7f4d280b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:45.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.124+0000 7f4d25ffb640 1 --2- 172.21.15.49:0/2676838087 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d14003980 0x7f4d14003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:45.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.124+0000 7f4d25ffb640 1 --2- 172.21.15.49:0/2676838087 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d140047d0 0x7f4d14004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:45.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.124+0000 7f4d25ffb640 1 --2- 172.21.15.49:0/2676838087 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4d14005190 0x7f4d1400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:45.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.124+0000 7f4d25ffb640 1 -- 172.21.15.49:0/2676838087 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d2815aec0 con 0x7f4d140047d0 2023-11-06T02:09:45.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.124+0000 7f4d25ffb640 1 -- 172.21.15.49:0/2676838087 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1400c030 con 0x7f4d14003980 2023-11-06T02:09:45.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.124+0000 7f4d25ffb640 1 -- 172.21.15.49:0/2676838087 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1400c1f0 con 0x7f4d14005190 2023-11-06T02:09:45.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.124+0000 7f4d267fc640 1 --2- 172.21.15.49:0/2676838087 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d140047d0 0x7f4d14004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:45.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.124+0000 7f4d26ffd640 1 --2- 172.21.15.49:0/2676838087 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d14003980 0x7f4d14003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:45.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.124+0000 7f4d277fe640 1 --2- 172.21.15.49:0/2676838087 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4d14005190 0x7f4d1400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:45.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.124+0000 7f4d267fc640 1 -- 172.21.15.49:0/2676838087 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4d14005190 msgr2=0x7f4d1400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:45.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.124+0000 7f4d267fc640 1 --2- 172.21.15.49:0/2676838087 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4d14005190 0x7f4d1400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:45.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.124+0000 7f4d267fc640 1 -- 172.21.15.49:0/2676838087 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d14003980 msgr2=0x7f4d14003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:45.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.124+0000 7f4d267fc640 1 --2- 172.21.15.49:0/2676838087 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d14003980 0x7f4d14003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:45.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.124+0000 7f4d267fc640 1 -- 172.21.15.49:0/2676838087 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d14003530 con 0x7f4d140047d0 2023-11-06T02:09:45.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.124+0000 7f4d277fe640 1 --2- 172.21.15.49:0/2676838087 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4d14005190 0x7f4d1400b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:09:45.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.124+0000 7f4d277fe640 1 -- 172.21.15.49:0/2676838087 reap_dead start 2023-11-06T02:09:45.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.124+0000 7f4d267fc640 1 --2- 172.21.15.49:0/2676838087 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d140047d0 0x7f4d14004c50 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f4d10009ae0 tx=0x7f4d1001ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3cdf0d87bc9c16b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:45.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.124+0000 7f4d25ffb640 1 -- 172.21.15.49:0/2676838087 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d100a5610 con 0x7f4d140047d0 2023-11-06T02:09:45.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.124+0000 7f4d25ffb640 1 -- 172.21.15.49:0/2676838087 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d100a5770 con 0x7f4d140047d0 2023-11-06T02:09:45.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.124+0000 7f4d25ffb640 1 -- 172.21.15.49:0/2676838087 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d100a5b20 con 0x7f4d140047d0 2023-11-06T02:09:45.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.124+0000 7f4d2d38a640 1 -- 172.21.15.49:0/2676838087 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d140047d0 msgr2=0x7f4d14004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:45.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.124+0000 7f4d2d38a640 1 --2- 172.21.15.49:0/2676838087 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d140047d0 0x7f4d14004c50 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f4d10009ae0 tx=0x7f4d1001ca50 comp rx=0 tx=0).stop 2023-11-06T02:09:45.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.128+0000 7f4d2d38a640 1 -- 172.21.15.49:0/2676838087 shutdown_connections 2023-11-06T02:09:45.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.128+0000 7f4d2d38a640 1 --2- 172.21.15.49:0/2676838087 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d140047d0 0x7f4d14004c50 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:45.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.128+0000 7f4d2d38a640 1 -- 172.21.15.49:0/2676838087 >> 172.21.15.49:0/2676838087 conn(0x7f4d2806c0a0 msgr2=0x7f4d2806c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:45.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.128+0000 7f4d2d38a640 1 -- 172.21.15.49:0/2676838087 shutdown_connections 2023-11-06T02:09:45.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.128+0000 7f4d2d38a640 1 -- 172.21.15.49:0/2676838087 wait complete. 2023-11-06T02:09:45.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.128+0000 7f4d2d38a640 1 Processor -- start 2023-11-06T02:09:45.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.128+0000 7f4d2d38a640 1 -- start start 2023-11-06T02:09:45.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.128+0000 7f4d2d38a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d140047d0 0x7f4d28150d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:45.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.128+0000 7f4d2d38a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4d280c60e0 0x7f4d280a7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:45.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.128+0000 7f4d2d38a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d280a8230 0x7f4d280b2700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:45.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.128+0000 7f4d2d38a640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d280c65c0 con 0x7f4d280a8230 2023-11-06T02:09:45.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.128+0000 7f4d2d38a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d280c68a0 con 0x7f4d140047d0 2023-11-06T02:09:45.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.128+0000 7f4d2d38a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d280c6b80 con 0x7f4d280c60e0 2023-11-06T02:09:45.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.128+0000 7f4d277fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d280a8230 0x7f4d280b2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:45.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.128+0000 7f4d277fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d280a8230 0x7f4d280b2700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36634/0 (socket says 172.21.15.49:36634) 2023-11-06T02:09:45.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.128+0000 7f4d277fe640 1 -- 172.21.15.49:0/4085231445 learned_addr learned my addr 172.21.15.49:0/4085231445 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:45.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.128+0000 7f4d26ffd640 1 --2- 172.21.15.49:0/4085231445 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d140047d0 0x7f4d28150d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:45.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.128+0000 7f4d267fc640 1 --2- 172.21.15.49:0/4085231445 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4d280c60e0 0x7f4d280a7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:45.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.128+0000 7f4d277fe640 1 -- 172.21.15.49:0/4085231445 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4d280c60e0 msgr2=0x7f4d280a7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:45.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.128+0000 7f4d277fe640 1 --2- 172.21.15.49:0/4085231445 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4d280c60e0 0x7f4d280a7cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:45.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.128+0000 7f4d277fe640 1 -- 172.21.15.49:0/4085231445 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d140047d0 msgr2=0x7f4d28150d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:45.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.128+0000 7f4d277fe640 1 --2- 172.21.15.49:0/4085231445 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d140047d0 0x7f4d28150d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:45.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.128+0000 7f4d277fe640 1 -- 172.21.15.49:0/4085231445 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d280c6e60 con 0x7f4d280a8230 2023-11-06T02:09:45.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.128+0000 7f4d277fe640 1 --2- 172.21.15.49:0/4085231445 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d280a8230 0x7f4d280b2700 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f4d280b0cf0 tx=0x7f4d2000d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:45.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.128+0000 7f4d1f7fe640 1 -- 172.21.15.49:0/4085231445 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d20003d90 con 0x7f4d280a8230 2023-11-06T02:09:45.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.128+0000 7f4d1f7fe640 1 -- 172.21.15.49:0/4085231445 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d20005070 con 0x7f4d280a8230 2023-11-06T02:09:45.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.128+0000 7f4d2d38a640 1 -- 172.21.15.49:0/4085231445 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d280c7140 con 0x7f4d280a8230 2023-11-06T02:09:45.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.128+0000 7f4d1f7fe640 1 -- 172.21.15.49:0/4085231445 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d20005420 con 0x7f4d280a8230 2023-11-06T02:09:45.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.128+0000 7f4d2d38a640 1 -- 172.21.15.49:0/4085231445 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d281512a0 con 0x7f4d280a8230 2023-11-06T02:09:45.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.128+0000 7f4d2d38a640 1 -- 172.21.15.49:0/4085231445 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d280c7330 con 0x7f4d280a8230 2023-11-06T02:09:45.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.132+0000 7f4d1f7fe640 1 -- 172.21.15.49:0/4085231445 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f4d20005580 con 0x7f4d280a8230 2023-11-06T02:09:45.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.132+0000 7f4d1f7fe640 1 --2- 172.21.15.49:0/4085231445 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4d08065a60 0x7f4d08067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:45.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.132+0000 7f4d1f7fe640 1 -- 172.21.15.49:0/4085231445 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4d20082b20 con 0x7f4d280a8230 2023-11-06T02:09:45.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.132+0000 7f4d26ffd640 1 --2- 172.21.15.49:0/4085231445 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4d08065a60 0x7f4d08067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:45.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.132+0000 7f4d26ffd640 1 --2- 172.21.15.49:0/4085231445 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4d08065a60 0x7f4d08067f20 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4d18001e20 tx=0x7f4d18009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:45.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.136+0000 7f4d1f7fe640 1 -- 172.21.15.49:0/4085231445 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f4d20050410 con 0x7f4d280a8230 2023-11-06T02:09:45.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.348+0000 7f4d2d38a640 1 -- 172.21.15.49:0/4085231445 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f4d28074040 con 0x7f4d280a8230 2023-11-06T02:09:45.350 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.352+0000 7f4d1f7fe640 1 -- 172.21.15.49:0/4085231445 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1183 (secure 0 0 0) 0x7f4d20018080 con 0x7f4d280a8230 2023-11-06T02:09:45.350 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:09:45.350 INFO:teuthology.orchestra.run.smithi049.stdout:{"fsid":"1a8b035c-6ba4-499a-b876-aea47fd9c0d4","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":98,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":37,"num_osds":8,"num_up_osds":8,"osd_up_since":1699236497,"num_in_osds":8,"osd_in_since":1699236488,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":96},{"state_name":"active+clean","count":9}],"num_pgs":105,"num_pools":4,"num_objects":2,"data_bytes":590368,"bytes_used":241733632,"bytes_avail":772852379648,"bytes_total":773094113280,"unknown_pgs_ratio":0.91428571939468384},"fsmap":{"epoch":15,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"a","status":"up:active","gid":5041}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":4,"modified":"2023-11-06T02:09:27.196149+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-11-06T02:09:45.352 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.352+0000 7f4d2d38a640 1 -- 172.21.15.49:0/4085231445 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4d08065a60 msgr2=0x7f4d08067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:45.352 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.352+0000 7f4d2d38a640 1 --2- 172.21.15.49:0/4085231445 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4d08065a60 0x7f4d08067f20 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4d18001e20 tx=0x7f4d18009040 comp rx=0 tx=0).stop 2023-11-06T02:09:45.353 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.352+0000 7f4d2d38a640 1 -- 172.21.15.49:0/4085231445 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d280a8230 msgr2=0x7f4d280b2700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:45.353 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.352+0000 7f4d2d38a640 1 --2- 172.21.15.49:0/4085231445 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d280a8230 0x7f4d280b2700 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f4d280b0cf0 tx=0x7f4d2000d9d0 comp rx=0 tx=0).stop 2023-11-06T02:09:45.353 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.352+0000 7f4d2d38a640 1 -- 172.21.15.49:0/4085231445 shutdown_connections 2023-11-06T02:09:45.353 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.352+0000 7f4d2d38a640 1 --2- 172.21.15.49:0/4085231445 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4d08065a60 0x7f4d08067f20 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:45.354 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.352+0000 7f4d2d38a640 1 --2- 172.21.15.49:0/4085231445 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d280a8230 0x7f4d280b2700 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:45.354 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.352+0000 7f4d2d38a640 1 --2- 172.21.15.49:0/4085231445 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4d280c60e0 0x7f4d280a7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:45.354 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.352+0000 7f4d2d38a640 1 --2- 172.21.15.49:0/4085231445 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d140047d0 0x7f4d28150d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:45.354 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.352+0000 7f4d2d38a640 1 -- 172.21.15.49:0/4085231445 >> 172.21.15.49:0/4085231445 conn(0x7f4d2806c0a0 msgr2=0x7f4d280b1a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:45.354 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.352+0000 7f4d2d38a640 1 -- 172.21.15.49:0/4085231445 shutdown_connections 2023-11-06T02:09:45.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.352+0000 7f4d2d38a640 1 -- 172.21.15.49:0/4085231445 wait complete. 2023-11-06T02:09:45.368 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-11-06T02:09:45.369 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T02:09:45.485 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.488+0000 7fb0b39a8640 1 Processor -- start 2023-11-06T02:09:45.486 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.488+0000 7fb0b39a8640 1 -- start start 2023-11-06T02:09:45.486 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.488+0000 7fb0b39a8640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb0ac159980 0x7fb0ac071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:45.487 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.488+0000 7fb0b39a8640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fb0ac0cb010 0x7fb0ac0cb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:45.487 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.488+0000 7fb0b39a8640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0ac071890 con 0x7fb0ac0b1450 2023-11-06T02:09:45.487 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.488+0000 7fb0b39a8640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0ac072410 con 0x7fb0ac0cb010 2023-11-06T02:09:45.487 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.488+0000 7fb0b39a8640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ac0ce440 con 0x7fb0ac159980 2023-11-06T02:09:45.488 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.488+0000 7fb0b0f43640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fb0ac0b1450 0x7fb0ac0cdd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:48210/0 (socket says 172.21.15.49:48210) 2023-11-06T02:09:45.488 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.488+0000 7fb0b0f43640 1 -- 172.21.15.49:0/1212533139 learned_addr learned my addr 172.21.15.49:0/1212533139 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:09:45.489 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.488+0000 7fb0a3fff640 1 -- 172.21.15.49:0/1212533139 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3182100388 0 0) 0x7fb0ac071890 con 0x7fb0ac0b1450 2023-11-06T02:09:45.490 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.488+0000 7fb0b1f45640 1 --2- 172.21.15.49:0/1212533139 >> v2:172.21.15.150:3300/0 conn(0x7fb0ac0cb010 0x7fb0ac0cb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:45.490 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.488+0000 7fb0a3fff640 1 -- 172.21.15.49:0/1212533139 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb098005740 con 0x7fb0ac0b1450 2023-11-06T02:09:45.490 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.488+0000 7fb0b1744640 1 --2- 172.21.15.49:0/1212533139 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb0ac159980 0x7fb0ac071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:45.490 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.488+0000 7fb0a3fff640 1 -- 172.21.15.49:0/1212533139 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2202081226 0 0) 0x7fb098005740 con 0x7fb0ac0b1450 2023-11-06T02:09:45.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.488+0000 7fb0a3fff640 1 -- 172.21.15.49:0/1212533139 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb0ac159980 msgr2=0x7fb0ac071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:45.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.488+0000 7fb0a3fff640 1 --2- 172.21.15.49:0/1212533139 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb0ac159980 0x7fb0ac071290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:45.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.488+0000 7fb0a3fff640 1 -- 172.21.15.49:0/1212533139 >> v2:172.21.15.150:3300/0 conn(0x7fb0ac0cb010 msgr2=0x7fb0ac0cb4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:45.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.488+0000 7fb0a3fff640 1 --2- 172.21.15.49:0/1212533139 >> v2:172.21.15.150:3300/0 conn(0x7fb0ac0cb010 0x7fb0ac0cb4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:45.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.488+0000 7fb0a3fff640 1 -- 172.21.15.49:0/1212533139 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0ac071b70 con 0x7fb0ac0b1450 2023-11-06T02:09:45.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.488+0000 7fb0a3fff640 1 -- 172.21.15.49:0/1212533139 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7fb09c001680 con 0x7fb0ac0b1450 2023-11-06T02:09:45.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.488+0000 7fb0b1f45640 1 --2- 172.21.15.49:0/1212533139 >> v2:172.21.15.150:3300/0 conn(0x7fb0ac0cb010 0x7fb0ac0cb4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:09:45.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.488+0000 7fb0a3fff640 1 -- 172.21.15.49:0/1212533139 >> v1:172.21.15.49:6789/0 conn(0x7fb0ac0b1450 legacy=0x7fb0ac0cdd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:45.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.488+0000 7fb0a3fff640 1 --2- 172.21.15.49:0/1212533139 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb0ac159980 0x7fb098005b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:45.493 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.488+0000 7fb0a3fff640 1 --2- 172.21.15.49:0/1212533139 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb098006110 0x7fb0980069e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:45.493 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.488+0000 7fb0a3fff640 1 --2- 172.21.15.49:0/1212533139 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb098006f20 0x7fb0980073a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:45.493 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.488+0000 7fb0a3fff640 1 -- 172.21.15.49:0/1212533139 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ac072410 con 0x7fb098006110 2023-11-06T02:09:45.493 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.488+0000 7fb0a3fff640 1 -- 172.21.15.49:0/1212533139 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ac0ce440 con 0x7fb098006f20 2023-11-06T02:09:45.493 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.488+0000 7fb0a3fff640 1 -- 172.21.15.49:0/1212533139 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb09800dbf0 con 0x7fb0ac159980 2023-11-06T02:09:45.493 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.488+0000 7fb0b0f43640 1 --2- 172.21.15.49:0/1212533139 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb098006110 0x7fb0980069e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:45.494 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.488+0000 7fb0b1f45640 1 --2- 172.21.15.49:0/1212533139 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb098006f20 0x7fb0980073a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:45.494 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.488+0000 7fb0b1744640 1 --2- 172.21.15.49:0/1212533139 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb0ac159980 0x7fb098005b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:45.494 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.488+0000 7fb0b0f43640 1 -- 172.21.15.49:0/1212533139 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb0ac159980 msgr2=0x7fb098005b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:45.494 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.488+0000 7fb0b0f43640 1 --2- 172.21.15.49:0/1212533139 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb0ac159980 0x7fb098005b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:45.494 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.488+0000 7fb0b0f43640 1 -- 172.21.15.49:0/1212533139 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb098006f20 msgr2=0x7fb0980073a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:45.495 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.488+0000 7fb0b0f43640 1 --2- 172.21.15.49:0/1212533139 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb098006f20 0x7fb0980073a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:45.495 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.488+0000 7fb0b0f43640 1 -- 172.21.15.49:0/1212533139 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb098005740 con 0x7fb098006110 2023-11-06T02:09:45.495 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.488+0000 7fb0b1744640 1 --2- 172.21.15.49:0/1212533139 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb0ac159980 0x7fb098005b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:09:45.498 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.492+0000 7fb0b0f43640 1 --2- 172.21.15.49:0/1212533139 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb098006110 0x7fb0980069e0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fb09c00c290 tx=0x7fb09c01f430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce1302924bd319b8 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:45.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.492+0000 7fb0a3fff640 1 -- 172.21.15.49:0/1212533139 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb09c005ae0 con 0x7fb098006110 2023-11-06T02:09:45.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.492+0000 7fb0a3fff640 1 -- 172.21.15.49:0/1212533139 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb09c0ab070 con 0x7fb098006110 2023-11-06T02:09:45.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.492+0000 7fb0a3fff640 1 -- 172.21.15.49:0/1212533139 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb09c005e80 con 0x7fb098006110 2023-11-06T02:09:45.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.492+0000 7fb0b39a8640 1 -- 172.21.15.49:0/1212533139 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb098006110 msgr2=0x7fb0980069e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:45.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.492+0000 7fb0b39a8640 1 --2- 172.21.15.49:0/1212533139 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb098006110 0x7fb0980069e0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fb09c00c290 tx=0x7fb09c01f430 comp rx=0 tx=0).stop 2023-11-06T02:09:45.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.492+0000 7fb0b1f45640 1 -- 172.21.15.49:0/1212533139 reap_dead start 2023-11-06T02:09:45.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.492+0000 7fb0b39a8640 1 -- 172.21.15.49:0/1212533139 shutdown_connections 2023-11-06T02:09:45.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.492+0000 7fb0b39a8640 1 -- 172.21.15.49:0/1212533139 >> 172.21.15.49:0/1212533139 conn(0x7fb0ac06c0a0 msgr2=0x7fb0ac06c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:45.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.492+0000 7fb0b39a8640 1 -- 172.21.15.49:0/1212533139 shutdown_connections 2023-11-06T02:09:45.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.492+0000 7fb0b39a8640 1 -- 172.21.15.49:0/1212533139 wait complete. 2023-11-06T02:09:45.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.492+0000 7fb0b39a8640 1 Processor -- start 2023-11-06T02:09:45.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.492+0000 7fb0b39a8640 1 -- start start 2023-11-06T02:09:45.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.492+0000 7fb0b39a8640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb098006110 0x7fb0ac1e43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:45.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.492+0000 7fb0b39a8640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0ac14eb70 0x7fb0ac0c6530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:45.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.492+0000 7fb0b39a8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb0ac0acb30 0x7fb0ac0acfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:45.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.492+0000 7fb0b39a8640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ac0b1890 con 0x7fb0ac0acb30 2023-11-06T02:09:45.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.492+0000 7fb0b39a8640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ac0ad520 con 0x7fb0ac14eb70 2023-11-06T02:09:45.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.492+0000 7fb0b39a8640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ac0c7ee0 con 0x7fb098006110 2023-11-06T02:09:45.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.492+0000 7fb0b1f45640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb0ac0acb30 0x7fb0ac0acfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:45.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.492+0000 7fb0b1f45640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb0ac0acb30 0x7fb0ac0acfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36660/0 (socket says 172.21.15.49:36660) 2023-11-06T02:09:45.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.492+0000 7fb0b1f45640 1 -- 172.21.15.49:0/2884888347 learned_addr learned my addr 172.21.15.49:0/2884888347 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:45.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.496+0000 7fb0b1744640 1 --2- 172.21.15.49:0/2884888347 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb098006110 0x7fb0ac1e43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:45.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.496+0000 7fb0b0f43640 1 --2- 172.21.15.49:0/2884888347 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0ac14eb70 0x7fb0ac0c6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:45.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.496+0000 7fb0b1f45640 1 -- 172.21.15.49:0/2884888347 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb098006110 msgr2=0x7fb0ac1e43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:45.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.496+0000 7fb0b1f45640 1 --2- 172.21.15.49:0/2884888347 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb098006110 0x7fb0ac1e43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:45.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.496+0000 7fb0b1f45640 1 -- 172.21.15.49:0/2884888347 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0ac14eb70 msgr2=0x7fb0ac0c6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:45.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.496+0000 7fb0b1f45640 1 --2- 172.21.15.49:0/2884888347 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0ac14eb70 0x7fb0ac0c6530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:45.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.496+0000 7fb0b1f45640 1 -- 172.21.15.49:0/2884888347 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0ac0c7a50 con 0x7fb0ac0acb30 2023-11-06T02:09:45.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.496+0000 7fb0b1f45640 1 --2- 172.21.15.49:0/2884888347 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb0ac0acb30 0x7fb0ac0acfe0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac0c5650 tx=0x7fb0a4007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:45.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.496+0000 7fb0a1ffb640 1 -- 172.21.15.49:0/2884888347 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a40048a0 con 0x7fb0ac0acb30 2023-11-06T02:09:45.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.496+0000 7fb0a1ffb640 1 -- 172.21.15.49:0/2884888347 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0a400f050 con 0x7fb0ac0acb30 2023-11-06T02:09:45.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.496+0000 7fb0a1ffb640 1 -- 172.21.15.49:0/2884888347 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a4014630 con 0x7fb0ac0acb30 2023-11-06T02:09:45.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.496+0000 7fb0b39a8640 1 -- 172.21.15.49:0/2884888347 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0ac0c6e40 con 0x7fb0ac0acb30 2023-11-06T02:09:45.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.496+0000 7fb0b39a8640 1 -- 172.21.15.49:0/2884888347 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0ac0c7400 con 0x7fb0ac0acb30 2023-11-06T02:09:45.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.496+0000 7fb0b39a8640 1 -- 172.21.15.49:0/2884888347 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0ac0c7be0 con 0x7fb0ac0acb30 2023-11-06T02:09:45.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.496+0000 7fb0a1ffb640 1 -- 172.21.15.49:0/2884888347 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fb0a4014790 con 0x7fb0ac0acb30 2023-11-06T02:09:45.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.496+0000 7fb0a1ffb640 1 --2- 172.21.15.49:0/2884888347 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb090065a90 0x7fb090067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:45.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.496+0000 7fb0a1ffb640 1 -- 172.21.15.49:0/2884888347 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb0a40837f0 con 0x7fb0ac0acb30 2023-11-06T02:09:45.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.496+0000 7fb0b1744640 1 --2- 172.21.15.49:0/2884888347 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb090065a90 0x7fb090067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:45.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.496+0000 7fb0b1744640 1 --2- 172.21.15.49:0/2884888347 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb090065a90 0x7fb090067f50 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac157bd0 tx=0x7fb0a8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:45.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.500+0000 7fb0a1ffb640 1 -- 172.21.15.49:0/2884888347 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fb0a4051040 con 0x7fb0ac0acb30 2023-11-06T02:09:45.648 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.648+0000 7fb0b39a8640 1 -- 172.21.15.49:0/2884888347 --> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb0ac072130 con 0x7fb090065a90 2023-11-06T02:09:45.672 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.676+0000 7fb0a1ffb640 1 -- 172.21.15.49:0/2884888347 <== mgr.4100 v2:172.21.15.150:6832/38104 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+309511 (secure 0 0 0) 0x7fb0a8013960 con 0x7fb090065a90 2023-11-06T02:09:45.674 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:09:45.676 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2023-11-06T02:09:45.677 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.676+0000 7fb0b39a8640 1 -- 172.21.15.49:0/2884888347 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb090065a90 msgr2=0x7fb090067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:45.677 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.676+0000 7fb0b39a8640 1 --2- 172.21.15.49:0/2884888347 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb090065a90 0x7fb090067f50 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac157bd0 tx=0x7fb0a8008040 comp rx=0 tx=0).stop 2023-11-06T02:09:45.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.676+0000 7fb0b39a8640 1 -- 172.21.15.49:0/2884888347 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb0ac0acb30 msgr2=0x7fb0ac0acfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:45.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.676+0000 7fb0b39a8640 1 --2- 172.21.15.49:0/2884888347 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb0ac0acb30 0x7fb0ac0acfe0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac0c5650 tx=0x7fb0a4007700 comp rx=0 tx=0).stop 2023-11-06T02:09:45.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.676+0000 7fb0b39a8640 1 -- 172.21.15.49:0/2884888347 shutdown_connections 2023-11-06T02:09:45.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.676+0000 7fb0b39a8640 1 --2- 172.21.15.49:0/2884888347 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb090065a90 0x7fb090067f50 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:45.679 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.676+0000 7fb0b39a8640 1 --2- 172.21.15.49:0/2884888347 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb0ac0acb30 0x7fb0ac0acfe0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:45.679 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.676+0000 7fb0b39a8640 1 --2- 172.21.15.49:0/2884888347 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0ac14eb70 0x7fb0ac0c6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:45.679 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.676+0000 7fb0b39a8640 1 --2- 172.21.15.49:0/2884888347 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb098006110 0x7fb0ac1e43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:45.679 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.676+0000 7fb0b39a8640 1 -- 172.21.15.49:0/2884888347 >> 172.21.15.49:0/2884888347 conn(0x7fb0ac06c0a0 msgr2=0x7fb0ac157d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:45.679 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.676+0000 7fb0b39a8640 1 -- 172.21.15.49:0/2884888347 shutdown_connections 2023-11-06T02:09:45.679 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:45.676+0000 7fb0b39a8640 1 -- 172.21.15.49:0/2884888347 wait complete. 2023-11-06T02:09:45.692 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":74,"stamp":"2023-11-06T02:09:45.201549+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":181,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236504,"kb_used_data":4504,"kb_used_omap":0,"kb_used_meta":231936,"kb_avail":754738216,"statfs":{"total":773094113280,"available":772851933184,"internally_reserved":0,"allocated":4612096,"data_stored":2579436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":237502464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":54,"apply_latency_ms":54,"commit_latency_ns":54000000,"apply_latency_ns":54000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001762"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.666671+0000","last_change":"2023-11-06T02:09:39.894697+0000","last_active":"2023-11-06T02:09:42.666671+0000","last_peered":"2023-11-06T02:09:42.666671+0000","last_clean":"2023-11-06T02:09:42.666671+0000","last_became_active":"2023-11-06T02:09:39.894432+0000","last_became_peered":"2023-11-06T02:09:39.894432+0000","last_unstale":"2023-11-06T02:09:42.666671+0000","last_undegraded":"2023-11-06T02:09:42.666671+0000","last_fullsized":"2023-11-06T02:09:42.666671+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:18:48.057417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.666162+0000","last_change":"2023-11-06T02:09:39.893106+0000","last_active":"2023-11-06T02:09:42.666162+0000","last_peered":"2023-11-06T02:09:42.666162+0000","last_clean":"2023-11-06T02:09:42.666162+0000","last_became_active":"2023-11-06T02:09:39.892854+0000","last_became_peered":"2023-11-06T02:09:39.892854+0000","last_unstale":"2023-11-06T02:09:42.666162+0000","last_undegraded":"2023-11-06T02:09:42.666162+0000","last_fullsized":"2023-11-06T02:09:42.666162+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:18:26.908517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:41.493454+0000","last_change":"2023-11-06T02:09:39.950175+0000","last_active":"2023-11-06T02:09:41.493454+0000","last_peered":"2023-11-06T02:09:41.493454+0000","last_clean":"2023-11-06T02:09:41.493454+0000","last_became_active":"2023-11-06T02:09:39.949828+0000","last_became_peered":"2023-11-06T02:09:39.949828+0000","last_unstale":"2023-11-06T02:09:41.493454+0000","last_undegraded":"2023-11-06T02:09:41.493454+0000","last_fullsized":"2023-11-06T02:09:41.493454+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:24:44.156346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:40.889850+0000","last_change":"2023-11-06T02:09:39.896077+0000","last_active":"2023-11-06T02:09:40.889850+0000","last_peered":"2023-11-06T02:09:40.889850+0000","last_clean":"2023-11-06T02:09:40.889850+0000","last_became_active":"2023-11-06T02:09:39.895853+0000","last_became_peered":"2023-11-06T02:09:39.895853+0000","last_unstale":"2023-11-06T02:09:40.889850+0000","last_undegraded":"2023-11-06T02:09:40.889850+0000","last_fullsized":"2023-11-06T02:09:40.889850+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:30:00.565422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:40.889686+0000","last_change":"2023-11-06T02:09:39.892525+0000","last_active":"2023-11-06T02:09:40.889686+0000","last_peered":"2023-11-06T02:09:40.889686+0000","last_clean":"2023-11-06T02:09:40.889686+0000","last_became_active":"2023-11-06T02:09:39.892227+0000","last_became_peered":"2023-11-06T02:09:39.892227+0000","last_unstale":"2023-11-06T02:09:40.889686+0000","last_undegraded":"2023-11-06T02:09:40.889686+0000","last_fullsized":"2023-11-06T02:09:40.889686+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:17:06.187075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.666612+0000","last_change":"2023-11-06T02:09:39.892888+0000","last_active":"2023-11-06T02:09:42.666612+0000","last_peered":"2023-11-06T02:09:42.666612+0000","last_clean":"2023-11-06T02:09:42.666612+0000","last_became_active":"2023-11-06T02:09:39.892324+0000","last_became_peered":"2023-11-06T02:09:39.892324+0000","last_unstale":"2023-11-06T02:09:42.666612+0000","last_undegraded":"2023-11-06T02:09:42.666612+0000","last_fullsized":"2023-11-06T02:09:42.666612+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:43:55.630807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.616213+0000","last_change":"2023-11-06T02:09:39.935211+0000","last_active":"2023-11-06T02:09:42.616213+0000","last_peered":"2023-11-06T02:09:42.616213+0000","last_clean":"2023-11-06T02:09:42.616213+0000","last_became_active":"2023-11-06T02:09:39.934429+0000","last_became_peered":"2023-11-06T02:09:39.934429+0000","last_unstale":"2023-11-06T02:09:42.616213+0000","last_undegraded":"2023-11-06T02:09:42.616213+0000","last_fullsized":"2023-11-06T02:09:42.616213+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:14:47.981503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.665585+0000","last_change":"2023-11-06T02:09:39.936779+0000","last_active":"2023-11-06T02:09:42.665585+0000","last_peered":"2023-11-06T02:09:42.665585+0000","last_clean":"2023-11-06T02:09:42.665585+0000","last_became_active":"2023-11-06T02:09:39.936501+0000","last_became_peered":"2023-11-06T02:09:39.936501+0000","last_unstale":"2023-11-06T02:09:42.665585+0000","last_undegraded":"2023-11-06T02:09:42.665585+0000","last_fullsized":"2023-11-06T02:09:42.665585+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:37:29.456658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:41.493698+0000","last_change":"2023-11-06T02:09:39.948773+0000","last_active":"2023-11-06T02:09:41.493698+0000","last_peered":"2023-11-06T02:09:41.493698+0000","last_clean":"2023-11-06T02:09:41.493698+0000","last_became_active":"2023-11-06T02:09:39.948499+0000","last_became_peered":"2023-11-06T02:09:39.948499+0000","last_unstale":"2023-11-06T02:09:41.493698+0000","last_undegraded":"2023-11-06T02:09:41.493698+0000","last_fullsized":"2023-11-06T02:09:41.493698+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:59:20.637813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.617038+0000","last_change":"2023-11-06T02:09:39.944022+0000","last_active":"2023-11-06T02:09:42.617038+0000","last_peered":"2023-11-06T02:09:42.617038+0000","last_clean":"2023-11-06T02:09:42.617038+0000","last_became_active":"2023-11-06T02:09:39.943075+0000","last_became_peered":"2023-11-06T02:09:39.943075+0000","last_unstale":"2023-11-06T02:09:42.617038+0000","last_undegraded":"2023-11-06T02:09:42.617038+0000","last_fullsized":"2023-11-06T02:09:42.617038+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:04:26.233460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.617072+0000","last_change":"2023-11-06T02:09:39.950021+0000","last_active":"2023-11-06T02:09:42.617072+0000","last_peered":"2023-11-06T02:09:42.617072+0000","last_clean":"2023-11-06T02:09:42.617072+0000","last_became_active":"2023-11-06T02:09:39.945579+0000","last_became_peered":"2023-11-06T02:09:39.945579+0000","last_unstale":"2023-11-06T02:09:42.617072+0000","last_undegraded":"2023-11-06T02:09:42.617072+0000","last_fullsized":"2023-11-06T02:09:42.617072+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:03:27.079692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:41.493328+0000","last_change":"2023-11-06T02:09:39.894500+0000","last_active":"2023-11-06T02:09:41.493328+0000","last_peered":"2023-11-06T02:09:41.493328+0000","last_clean":"2023-11-06T02:09:41.493328+0000","last_became_active":"2023-11-06T02:09:39.893806+0000","last_became_peered":"2023-11-06T02:09:39.893806+0000","last_unstale":"2023-11-06T02:09:41.493328+0000","last_undegraded":"2023-11-06T02:09:41.493328+0000","last_fullsized":"2023-11-06T02:09:41.493328+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:39:49.506672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.616771+0000","last_change":"2023-11-06T02:09:39.936301+0000","last_active":"2023-11-06T02:09:42.616771+0000","last_peered":"2023-11-06T02:09:42.616771+0000","last_clean":"2023-11-06T02:09:42.616771+0000","last_became_active":"2023-11-06T02:09:39.934917+0000","last_became_peered":"2023-11-06T02:09:39.934917+0000","last_unstale":"2023-11-06T02:09:42.616771+0000","last_undegraded":"2023-11-06T02:09:42.616771+0000","last_fullsized":"2023-11-06T02:09:42.616771+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:18:48.057417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.444971+0000","last_change":"2023-11-06T02:09:39.920273+0000","last_active":"2023-11-06T02:09:42.444971+0000","last_peered":"2023-11-06T02:09:42.444971+0000","last_clean":"2023-11-06T02:09:42.444971+0000","last_became_active":"2023-11-06T02:09:39.917769+0000","last_became_peered":"2023-11-06T02:09:39.917769+0000","last_unstale":"2023-11-06T02:09:42.444971+0000","last_undegraded":"2023-11-06T02:09:42.444971+0000","last_fullsized":"2023-11-06T02:09:42.444971+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:30:28.443414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.616264+0000","last_change":"2023-11-06T02:09:39.935981+0000","last_active":"2023-11-06T02:09:42.616264+0000","last_peered":"2023-11-06T02:09:42.616264+0000","last_clean":"2023-11-06T02:09:42.616264+0000","last_became_active":"2023-11-06T02:09:39.933574+0000","last_became_peered":"2023-11-06T02:09:39.933574+0000","last_unstale":"2023-11-06T02:09:42.616264+0000","last_undegraded":"2023-11-06T02:09:42.616264+0000","last_fullsized":"2023-11-06T02:09:42.616264+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:01:17.009165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:40.892687+0000","last_change":"2023-11-06T02:09:39.946799+0000","last_active":"2023-11-06T02:09:40.892687+0000","last_peered":"2023-11-06T02:09:40.892687+0000","last_clean":"2023-11-06T02:09:40.892687+0000","last_became_active":"2023-11-06T02:09:39.946390+0000","last_became_peered":"2023-11-06T02:09:39.946390+0000","last_unstale":"2023-11-06T02:09:40.892687+0000","last_undegraded":"2023-11-06T02:09:40.892687+0000","last_fullsized":"2023-11-06T02:09:40.892687+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:38:43.083280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:40.894513+0000","last_change":"2023-11-06T02:09:39.949727+0000","last_active":"2023-11-06T02:09:40.894513+0000","last_peered":"2023-11-06T02:09:40.894513+0000","last_clean":"2023-11-06T02:09:40.894513+0000","last_became_active":"2023-11-06T02:09:39.949138+0000","last_became_peered":"2023-11-06T02:09:39.949138+0000","last_unstale":"2023-11-06T02:09:40.894513+0000","last_undegraded":"2023-11-06T02:09:40.894513+0000","last_fullsized":"2023-11-06T02:09:40.894513+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:59:20.637813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:40.894336+0000","last_change":"2023-11-06T02:09:39.913899+0000","last_active":"2023-11-06T02:09:40.894336+0000","last_peered":"2023-11-06T02:09:40.894336+0000","last_clean":"2023-11-06T02:09:40.894336+0000","last_became_active":"2023-11-06T02:09:39.913107+0000","last_became_peered":"2023-11-06T02:09:39.913107+0000","last_unstale":"2023-11-06T02:09:40.894336+0000","last_undegraded":"2023-11-06T02:09:40.894336+0000","last_fullsized":"2023-11-06T02:09:40.894336+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:45:01.120311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:40.889838+0000","last_change":"2023-11-06T02:09:39.890295+0000","last_active":"2023-11-06T02:09:40.889838+0000","last_peered":"2023-11-06T02:09:40.889838+0000","last_clean":"2023-11-06T02:09:40.889838+0000","last_became_active":"2023-11-06T02:09:39.888442+0000","last_became_peered":"2023-11-06T02:09:39.888442+0000","last_unstale":"2023-11-06T02:09:40.889838+0000","last_undegraded":"2023-11-06T02:09:40.889838+0000","last_fullsized":"2023-11-06T02:09:40.889838+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:06:38.586189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:40.895042+0000","last_change":"2023-11-06T02:09:39.913639+0000","last_active":"2023-11-06T02:09:40.895042+0000","last_peered":"2023-11-06T02:09:40.895042+0000","last_clean":"2023-11-06T02:09:40.895042+0000","last_became_active":"2023-11-06T02:09:39.912758+0000","last_became_peered":"2023-11-06T02:09:39.912758+0000","last_unstale":"2023-11-06T02:09:40.895042+0000","last_undegraded":"2023-11-06T02:09:40.895042+0000","last_fullsized":"2023-11-06T02:09:40.895042+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:19:54.348389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.616361+0000","last_change":"2023-11-06T02:09:39.936823+0000","last_active":"2023-11-06T02:09:42.616361+0000","last_peered":"2023-11-06T02:09:42.616361+0000","last_clean":"2023-11-06T02:09:42.616361+0000","last_became_active":"2023-11-06T02:09:39.935648+0000","last_became_peered":"2023-11-06T02:09:39.935648+0000","last_unstale":"2023-11-06T02:09:42.616361+0000","last_undegraded":"2023-11-06T02:09:42.616361+0000","last_fullsized":"2023-11-06T02:09:42.616361+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:28:05.481974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.445677+0000","last_change":"2023-11-06T02:09:39.898194+0000","last_active":"2023-11-06T02:09:42.445677+0000","last_peered":"2023-11-06T02:09:42.445677+0000","last_clean":"2023-11-06T02:09:42.445677+0000","last_became_active":"2023-11-06T02:09:39.897460+0000","last_became_peered":"2023-11-06T02:09:39.897460+0000","last_unstale":"2023-11-06T02:09:42.445677+0000","last_undegraded":"2023-11-06T02:09:42.445677+0000","last_fullsized":"2023-11-06T02:09:42.445677+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:01:17.009165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:40.893463+0000","last_change":"2023-11-06T02:09:39.928101+0000","last_active":"2023-11-06T02:09:40.893463+0000","last_peered":"2023-11-06T02:09:40.893463+0000","last_clean":"2023-11-06T02:09:40.893463+0000","last_became_active":"2023-11-06T02:09:39.927170+0000","last_became_peered":"2023-11-06T02:09:39.927170+0000","last_unstale":"2023-11-06T02:09:40.893463+0000","last_undegraded":"2023-11-06T02:09:40.893463+0000","last_fullsized":"2023-11-06T02:09:40.893463+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:12:15.009411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:40.889446+0000","last_change":"2023-11-06T02:09:39.924861+0000","last_active":"2023-11-06T02:09:40.889446+0000","last_peered":"2023-11-06T02:09:40.889446+0000","last_clean":"2023-11-06T02:09:40.889446+0000","last_became_active":"2023-11-06T02:09:39.924654+0000","last_became_peered":"2023-11-06T02:09:39.924654+0000","last_unstale":"2023-11-06T02:09:40.889446+0000","last_undegraded":"2023-11-06T02:09:40.889446+0000","last_fullsized":"2023-11-06T02:09:40.889446+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:23:31.744003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:40.893650+0000","last_change":"2023-11-06T02:09:39.917240+0000","last_active":"2023-11-06T02:09:40.893650+0000","last_peered":"2023-11-06T02:09:40.893650+0000","last_clean":"2023-11-06T02:09:40.893650+0000","last_became_active":"2023-11-06T02:09:39.916284+0000","last_became_peered":"2023-11-06T02:09:39.916284+0000","last_unstale":"2023-11-06T02:09:40.893650+0000","last_undegraded":"2023-11-06T02:09:40.893650+0000","last_fullsized":"2023-11-06T02:09:40.893650+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:18:07.867946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.667212+0000","last_change":"2023-11-06T02:09:39.934780+0000","last_active":"2023-11-06T02:09:42.667212+0000","last_peered":"2023-11-06T02:09:42.667212+0000","last_clean":"2023-11-06T02:09:42.667212+0000","last_became_active":"2023-11-06T02:09:39.934264+0000","last_became_peered":"2023-11-06T02:09:39.934264+0000","last_unstale":"2023-11-06T02:09:42.667212+0000","last_undegraded":"2023-11-06T02:09:42.667212+0000","last_fullsized":"2023-11-06T02:09:42.667212+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:38:43.083280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:40.894599+0000","last_change":"2023-11-06T02:09:39.914882+0000","last_active":"2023-11-06T02:09:40.894599+0000","last_peered":"2023-11-06T02:09:40.894599+0000","last_clean":"2023-11-06T02:09:40.894599+0000","last_became_active":"2023-11-06T02:09:39.914057+0000","last_became_peered":"2023-11-06T02:09:39.914057+0000","last_unstale":"2023-11-06T02:09:40.894599+0000","last_undegraded":"2023-11-06T02:09:40.894599+0000","last_fullsized":"2023-11-06T02:09:40.894599+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:10:50.292864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.616845+0000","last_change":"2023-11-06T02:09:39.944891+0000","last_active":"2023-11-06T02:09:42.616845+0000","last_peered":"2023-11-06T02:09:42.616845+0000","last_clean":"2023-11-06T02:09:42.616845+0000","last_became_active":"2023-11-06T02:09:39.944414+0000","last_became_peered":"2023-11-06T02:09:39.944414+0000","last_unstale":"2023-11-06T02:09:42.616845+0000","last_undegraded":"2023-11-06T02:09:42.616845+0000","last_fullsized":"2023-11-06T02:09:42.616845+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:51:33.759418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:40.893934+0000","last_change":"2023-11-06T02:09:39.916632+0000","last_active":"2023-11-06T02:09:40.893934+0000","last_peered":"2023-11-06T02:09:40.893934+0000","last_clean":"2023-11-06T02:09:40.893934+0000","last_became_active":"2023-11-06T02:09:39.914132+0000","last_became_peered":"2023-11-06T02:09:39.914132+0000","last_unstale":"2023-11-06T02:09:40.893934+0000","last_undegraded":"2023-11-06T02:09:40.893934+0000","last_fullsized":"2023-11-06T02:09:40.893934+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:41:08.024502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:40.893821+0000","last_change":"2023-11-06T02:09:39.913252+0000","last_active":"2023-11-06T02:09:40.893821+0000","last_peered":"2023-11-06T02:09:40.893821+0000","last_clean":"2023-11-06T02:09:40.893821+0000","last_became_active":"2023-11-06T02:09:39.912390+0000","last_became_peered":"2023-11-06T02:09:39.912390+0000","last_unstale":"2023-11-06T02:09:40.893821+0000","last_undegraded":"2023-11-06T02:09:40.893821+0000","last_fullsized":"2023-11-06T02:09:40.893821+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:02:38.456326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:40.893731+0000","last_change":"2023-11-06T02:09:39.928747+0000","last_active":"2023-11-06T02:09:40.893731+0000","last_peered":"2023-11-06T02:09:40.893731+0000","last_clean":"2023-11-06T02:09:40.893731+0000","last_became_active":"2023-11-06T02:09:39.927707+0000","last_became_peered":"2023-11-06T02:09:39.927707+0000","last_unstale":"2023-11-06T02:09:40.893731+0000","last_undegraded":"2023-11-06T02:09:40.893731+0000","last_fullsized":"2023-11-06T02:09:40.893731+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:51:33.759418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.667463+0000","last_change":"2023-11-06T02:09:39.893420+0000","last_active":"2023-11-06T02:09:42.667463+0000","last_peered":"2023-11-06T02:09:42.667463+0000","last_clean":"2023-11-06T02:09:42.667463+0000","last_became_active":"2023-11-06T02:09:39.893152+0000","last_became_peered":"2023-11-06T02:09:39.893152+0000","last_unstale":"2023-11-06T02:09:42.667463+0000","last_undegraded":"2023-11-06T02:09:42.667463+0000","last_fullsized":"2023-11-06T02:09:42.667463+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:23:31.744003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:40.894952+0000","last_change":"2023-11-06T02:09:39.946680+0000","last_active":"2023-11-06T02:09:40.894952+0000","last_peered":"2023-11-06T02:09:40.894952+0000","last_clean":"2023-11-06T02:09:40.894952+0000","last_became_active":"2023-11-06T02:09:39.945969+0000","last_became_peered":"2023-11-06T02:09:39.945969+0000","last_unstale":"2023-11-06T02:09:40.894952+0000","last_undegraded":"2023-11-06T02:09:40.894952+0000","last_fullsized":"2023-11-06T02:09:40.894952+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:03:27.079692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"36'4","reported_seq":15,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:40.895403+0000","last_change":"2023-11-06T02:09:39.522170+0000","last_active":"2023-11-06T02:09:40.895403+0000","last_peered":"2023-11-06T02:09:40.895403+0000","last_clean":"2023-11-06T02:09:40.895403+0000","last_became_active":"2023-11-06T02:09:39.521622+0000","last_became_peered":"2023-11-06T02:09:39.521622+0000","last_unstale":"2023-11-06T02:09:40.895403+0000","last_undegraded":"2023-11-06T02:09:40.895403+0000","last_fullsized":"2023-11-06T02:09:40.895403+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:45:00.010604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.667549+0000","last_change":"2023-11-06T02:09:39.949469+0000","last_active":"2023-11-06T02:09:42.667549+0000","last_peered":"2023-11-06T02:09:42.667549+0000","last_clean":"2023-11-06T02:09:42.667549+0000","last_became_active":"2023-11-06T02:09:39.949119+0000","last_became_peered":"2023-11-06T02:09:39.949119+0000","last_unstale":"2023-11-06T02:09:42.667549+0000","last_undegraded":"2023-11-06T02:09:42.667549+0000","last_fullsized":"2023-11-06T02:09:42.667549+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:20:03.661315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"36'1","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:41.493536+0000","last_change":"2023-11-06T02:09:39.530442+0000","last_active":"2023-11-06T02:09:41.493536+0000","last_peered":"2023-11-06T02:09:41.493536+0000","last_clean":"2023-11-06T02:09:41.493536+0000","last_became_active":"2023-11-06T02:09:39.530124+0000","last_became_peered":"2023-11-06T02:09:39.530124+0000","last_unstale":"2023-11-06T02:09:41.493536+0000","last_undegraded":"2023-11-06T02:09:41.493536+0000","last_fullsized":"2023-11-06T02:09:41.493536+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:18:06.855960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.445640+0000","last_change":"2023-11-06T02:09:39.923492+0000","last_active":"2023-11-06T02:09:42.445640+0000","last_peered":"2023-11-06T02:09:42.445640+0000","last_clean":"2023-11-06T02:09:42.445640+0000","last_became_active":"2023-11-06T02:09:39.922628+0000","last_became_peered":"2023-11-06T02:09:39.922628+0000","last_unstale":"2023-11-06T02:09:42.445640+0000","last_undegraded":"2023-11-06T02:09:42.445640+0000","last_fullsized":"2023-11-06T02:09:42.445640+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:18:48.057417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"36'2","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.667680+0000","last_change":"2023-11-06T02:09:39.516455+0000","last_active":"2023-11-06T02:09:42.667680+0000","last_peered":"2023-11-06T02:09:42.667680+0000","last_clean":"2023-11-06T02:09:42.667680+0000","last_became_active":"2023-11-06T02:09:39.516057+0000","last_became_peered":"2023-11-06T02:09:39.516057+0000","last_unstale":"2023-11-06T02:09:42.667680+0000","last_undegraded":"2023-11-06T02:09:42.667680+0000","last_fullsized":"2023-11-06T02:09:42.667680+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:02:37.444340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.666415+0000","last_change":"2023-11-06T02:09:39.920710+0000","last_active":"2023-11-06T02:09:42.666415+0000","last_peered":"2023-11-06T02:09:42.666415+0000","last_clean":"2023-11-06T02:09:42.666415+0000","last_became_active":"2023-11-06T02:09:39.920415+0000","last_became_peered":"2023-11-06T02:09:39.920415+0000","last_unstale":"2023-11-06T02:09:42.666415+0000","last_undegraded":"2023-11-06T02:09:42.666415+0000","last_fullsized":"2023-11-06T02:09:42.666415+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:16:12.206417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"36'2","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.445520+0000","last_change":"2023-11-06T02:09:39.523237+0000","last_active":"2023-11-06T02:09:42.445520+0000","last_peered":"2023-11-06T02:09:42.445520+0000","last_clean":"2023-11-06T02:09:42.445520+0000","last_became_active":"2023-11-06T02:09:39.522434+0000","last_became_peered":"2023-11-06T02:09:39.522434+0000","last_unstale":"2023-11-06T02:09:42.445520+0000","last_undegraded":"2023-11-06T02:09:42.445520+0000","last_fullsized":"2023-11-06T02:09:42.445520+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:59:19.625827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:41.492608+0000","last_change":"2023-11-06T02:09:39.895217+0000","last_active":"2023-11-06T02:09:41.492608+0000","last_peered":"2023-11-06T02:09:41.492608+0000","last_clean":"2023-11-06T02:09:41.492608+0000","last_became_active":"2023-11-06T02:09:39.894437+0000","last_became_peered":"2023-11-06T02:09:39.894437+0000","last_unstale":"2023-11-06T02:09:41.492608+0000","last_undegraded":"2023-11-06T02:09:41.492608+0000","last_fullsized":"2023-11-06T02:09:41.492608+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:16:12.206417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"36'1","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.616450+0000","last_change":"2023-11-06T02:09:39.530213+0000","last_active":"2023-11-06T02:09:42.616450+0000","last_peered":"2023-11-06T02:09:42.616450+0000","last_clean":"2023-11-06T02:09:42.616450+0000","last_became_active":"2023-11-06T02:09:39.529428+0000","last_became_peered":"2023-11-06T02:09:39.529428+0000","last_unstale":"2023-11-06T02:09:42.616450+0000","last_undegraded":"2023-11-06T02:09:42.616450+0000","last_fullsized":"2023-11-06T02:09:42.616450+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:17:05.175089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.616695+0000","last_change":"2023-11-06T02:09:39.944083+0000","last_active":"2023-11-06T02:09:42.616695+0000","last_peered":"2023-11-06T02:09:42.616695+0000","last_clean":"2023-11-06T02:09:42.616695+0000","last_became_active":"2023-11-06T02:09:39.943344+0000","last_became_peered":"2023-11-06T02:09:39.943344+0000","last_unstale":"2023-11-06T02:09:42.616695+0000","last_undegraded":"2023-11-06T02:09:42.616695+0000","last_fullsized":"2023-11-06T02:09:42.616695+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:30:28.443414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"36'1","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:40.893347+0000","last_change":"2023-11-06T02:09:39.529686+0000","last_active":"2023-11-06T02:09:40.893347+0000","last_peered":"2023-11-06T02:09:40.893347+0000","last_clean":"2023-11-06T02:09:40.893347+0000","last_became_active":"2023-11-06T02:09:39.528866+0000","last_became_peered":"2023-11-06T02:09:39.528866+0000","last_unstale":"2023-11-06T02:09:40.893347+0000","last_undegraded":"2023-11-06T02:09:40.893347+0000","last_fullsized":"2023-11-06T02:09:40.893347+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:41:07.012516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.445266+0000","last_change":"2023-11-06T02:09:39.945345+0000","last_active":"2023-11-06T02:09:42.445266+0000","last_peered":"2023-11-06T02:09:42.445266+0000","last_clean":"2023-11-06T02:09:42.445266+0000","last_became_active":"2023-11-06T02:09:39.944852+0000","last_became_peered":"2023-11-06T02:09:39.944852+0000","last_unstale":"2023-11-06T02:09:42.445266+0000","last_undegraded":"2023-11-06T02:09:42.445266+0000","last_fullsized":"2023-11-06T02:09:42.445266+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:46:16.536437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.666771+0000","last_change":"2023-11-06T02:09:39.534802+0000","last_active":"2023-11-06T02:09:42.666771+0000","last_peered":"2023-11-06T02:09:42.666771+0000","last_clean":"2023-11-06T02:09:42.666771+0000","last_became_active":"2023-11-06T02:09:39.534382+0000","last_became_peered":"2023-11-06T02:09:39.534382+0000","last_unstale":"2023-11-06T02:09:42.666771+0000","last_undegraded":"2023-11-06T02:09:42.666771+0000","last_fullsized":"2023-11-06T02:09:42.666771+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:05:38.381689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.665954+0000","last_change":"2023-11-06T02:09:39.932625+0000","last_active":"2023-11-06T02:09:42.665954+0000","last_peered":"2023-11-06T02:09:42.665954+0000","last_clean":"2023-11-06T02:09:42.665954+0000","last_became_active":"2023-11-06T02:09:39.932419+0000","last_became_peered":"2023-11-06T02:09:39.932419+0000","last_unstale":"2023-11-06T02:09:42.665954+0000","last_undegraded":"2023-11-06T02:09:42.665954+0000","last_fullsized":"2023-11-06T02:09:42.665954+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:01:17.009165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.444854+0000","last_change":"2023-11-06T02:09:39.520842+0000","last_active":"2023-11-06T02:09:42.444854+0000","last_peered":"2023-11-06T02:09:42.444854+0000","last_clean":"2023-11-06T02:09:42.444854+0000","last_became_active":"2023-11-06T02:09:39.520372+0000","last_became_peered":"2023-11-06T02:09:39.520372+0000","last_unstale":"2023-11-06T02:09:42.444854+0000","last_undegraded":"2023-11-06T02:09:42.444854+0000","last_fullsized":"2023-11-06T02:09:42.444854+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:39:48.494686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:40.889764+0000","last_change":"2023-11-06T02:09:39.895957+0000","last_active":"2023-11-06T02:09:40.889764+0000","last_peered":"2023-11-06T02:09:40.889764+0000","last_clean":"2023-11-06T02:09:40.889764+0000","last_became_active":"2023-11-06T02:09:39.895360+0000","last_became_peered":"2023-11-06T02:09:39.895360+0000","last_unstale":"2023-11-06T02:09:40.889764+0000","last_undegraded":"2023-11-06T02:09:40.889764+0000","last_fullsized":"2023-11-06T02:09:40.889764+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:45:01.120311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.666885+0000","last_change":"2023-11-06T02:09:39.507720+0000","last_active":"2023-11-06T02:09:42.666885+0000","last_peered":"2023-11-06T02:09:42.666885+0000","last_clean":"2023-11-06T02:09:42.666885+0000","last_became_active":"2023-11-06T02:09:39.507153+0000","last_became_peered":"2023-11-06T02:09:39.507153+0000","last_unstale":"2023-11-06T02:09:42.666885+0000","last_undegraded":"2023-11-06T02:09:42.666885+0000","last_fullsized":"2023-11-06T02:09:42.666885+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:06:37.574203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:40.894035+0000","last_change":"2023-11-06T02:09:39.934807+0000","last_active":"2023-11-06T02:09:40.894035+0000","last_peered":"2023-11-06T02:09:40.894035+0000","last_clean":"2023-11-06T02:09:40.894035+0000","last_became_active":"2023-11-06T02:09:39.934323+0000","last_became_peered":"2023-11-06T02:09:39.934323+0000","last_unstale":"2023-11-06T02:09:40.894035+0000","last_undegraded":"2023-11-06T02:09:40.894035+0000","last_fullsized":"2023-11-06T02:09:40.894035+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:16:12.206417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.617153+0000","last_change":"2023-11-06T02:09:39.536553+0000","last_active":"2023-11-06T02:09:42.617153+0000","last_peered":"2023-11-06T02:09:42.617153+0000","last_clean":"2023-11-06T02:09:42.617153+0000","last_became_active":"2023-11-06T02:09:39.533822+0000","last_became_peered":"2023-11-06T02:09:39.533822+0000","last_unstale":"2023-11-06T02:09:42.617153+0000","last_undegraded":"2023-11-06T02:09:42.617153+0000","last_fullsized":"2023-11-06T02:09:42.617153+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:45:00.108325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:41.493505+0000","last_change":"2023-11-06T02:09:39.925758+0000","last_active":"2023-11-06T02:09:41.493505+0000","last_peered":"2023-11-06T02:09:41.493505+0000","last_clean":"2023-11-06T02:09:41.493505+0000","last_became_active":"2023-11-06T02:09:39.925443+0000","last_became_peered":"2023-11-06T02:09:39.925443+0000","last_unstale":"2023-11-06T02:09:41.493505+0000","last_undegraded":"2023-11-06T02:09:41.493505+0000","last_fullsized":"2023-11-06T02:09:41.493505+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:49:03.209827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"36'3","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.617311+0000","last_change":"2023-11-06T02:09:39.527840+0000","last_active":"2023-11-06T02:09:42.617311+0000","last_peered":"2023-11-06T02:09:42.617311+0000","last_clean":"2023-11-06T02:09:42.617311+0000","last_became_active":"2023-11-06T02:09:39.527192+0000","last_became_peered":"2023-11-06T02:09:39.527192+0000","last_unstale":"2023-11-06T02:09:42.617311+0000","last_undegraded":"2023-11-06T02:09:42.617311+0000","last_fullsized":"2023-11-06T02:09:42.617311+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:06:37.574203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.445359+0000","last_change":"2023-11-06T02:09:39.897369+0000","last_active":"2023-11-06T02:09:42.445359+0000","last_peered":"2023-11-06T02:09:42.445359+0000","last_clean":"2023-11-06T02:09:42.445359+0000","last_became_active":"2023-11-06T02:09:39.896531+0000","last_became_peered":"2023-11-06T02:09:39.896531+0000","last_unstale":"2023-11-06T02:09:42.445359+0000","last_undegraded":"2023-11-06T02:09:42.445359+0000","last_fullsized":"2023-11-06T02:09:42.445359+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:03:27.079692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"36'6","reported_seq":17,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.666444+0000","last_change":"2023-11-06T02:09:39.530827+0000","last_active":"2023-11-06T02:09:42.666444+0000","last_peered":"2023-11-06T02:09:42.666444+0000","last_clean":"2023-11-06T02:09:42.666444+0000","last_became_active":"2023-11-06T02:09:39.530374+0000","last_became_peered":"2023-11-06T02:09:39.530374+0000","last_unstale":"2023-11-06T02:09:42.666444+0000","last_undegraded":"2023-11-06T02:09:42.666444+0000","last_fullsized":"2023-11-06T02:09:42.666444+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:21:41.865490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:41.492786+0000","last_change":"2023-11-06T02:09:39.944160+0000","last_active":"2023-11-06T02:09:41.492786+0000","last_peered":"2023-11-06T02:09:41.492786+0000","last_clean":"2023-11-06T02:09:41.492786+0000","last_became_active":"2023-11-06T02:09:39.943858+0000","last_became_peered":"2023-11-06T02:09:39.943858+0000","last_unstale":"2023-11-06T02:09:41.492786+0000","last_undegraded":"2023-11-06T02:09:41.492786+0000","last_fullsized":"2023-11-06T02:09:41.492786+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:18:26.908517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"36'2","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.666774+0000","last_change":"2023-11-06T02:09:39.514824+0000","last_active":"2023-11-06T02:09:42.666774+0000","last_peered":"2023-11-06T02:09:42.666774+0000","last_clean":"2023-11-06T02:09:42.666774+0000","last_became_active":"2023-11-06T02:09:39.514231+0000","last_became_peered":"2023-11-06T02:09:39.514231+0000","last_unstale":"2023-11-06T02:09:42.666774+0000","last_undegraded":"2023-11-06T02:09:42.666774+0000","last_fullsized":"2023-11-06T02:09:42.666774+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:23:30.732017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.666097+0000","last_change":"2023-11-06T02:09:39.895501+0000","last_active":"2023-11-06T02:09:42.666097+0000","last_peered":"2023-11-06T02:09:42.666097+0000","last_clean":"2023-11-06T02:09:42.666097+0000","last_became_active":"2023-11-06T02:09:39.895254+0000","last_became_peered":"2023-11-06T02:09:39.895254+0000","last_unstale":"2023-11-06T02:09:42.666097+0000","last_undegraded":"2023-11-06T02:09:42.666097+0000","last_fullsized":"2023-11-06T02:09:42.666097+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:28:05.481974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.666856+0000","last_change":"2023-11-06T02:09:39.505888+0000","last_active":"2023-11-06T02:09:42.666856+0000","last_peered":"2023-11-06T02:09:42.666856+0000","last_clean":"2023-11-06T02:09:42.666856+0000","last_became_active":"2023-11-06T02:09:39.505514+0000","last_became_peered":"2023-11-06T02:09:39.505514+0000","last_unstale":"2023-11-06T02:09:42.666856+0000","last_undegraded":"2023-11-06T02:09:42.666856+0000","last_fullsized":"2023-11-06T02:09:42.666856+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:06:29.233287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:40.893108+0000","last_change":"2023-11-06T02:09:39.926232+0000","last_active":"2023-11-06T02:09:40.893108+0000","last_peered":"2023-11-06T02:09:40.893108+0000","last_clean":"2023-11-06T02:09:40.893108+0000","last_became_active":"2023-11-06T02:09:39.924958+0000","last_became_peered":"2023-11-06T02:09:39.924958+0000","last_unstale":"2023-11-06T02:09:40.893108+0000","last_undegraded":"2023-11-06T02:09:40.893108+0000","last_fullsized":"2023-11-06T02:09:40.893108+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:18:26.908517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:40.894773+0000","last_change":"2023-11-06T02:09:39.525068+0000","last_active":"2023-11-06T02:09:40.894773+0000","last_peered":"2023-11-06T02:09:40.894773+0000","last_clean":"2023-11-06T02:09:40.894773+0000","last_became_active":"2023-11-06T02:09:39.524560+0000","last_became_peered":"2023-11-06T02:09:39.524560+0000","last_unstale":"2023-11-06T02:09:40.894773+0000","last_undegraded":"2023-11-06T02:09:40.894773+0000","last_fullsized":"2023-11-06T02:09:40.894773+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:03:26.558497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.666243+0000","last_change":"2023-11-06T02:09:39.895998+0000","last_active":"2023-11-06T02:09:42.666243+0000","last_peered":"2023-11-06T02:09:42.666243+0000","last_clean":"2023-11-06T02:09:42.666243+0000","last_became_active":"2023-11-06T02:09:39.895762+0000","last_became_peered":"2023-11-06T02:09:39.895762+0000","last_unstale":"2023-11-06T02:09:42.666243+0000","last_undegraded":"2023-11-06T02:09:42.666243+0000","last_fullsized":"2023-11-06T02:09:42.666243+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:14:47.981503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"36'8","reported_seq":19,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:40.892504+0000","last_change":"2023-11-06T02:09:39.530857+0000","last_active":"2023-11-06T02:09:40.892504+0000","last_peered":"2023-11-06T02:09:40.892504+0000","last_clean":"2023-11-06T02:09:40.892504+0000","last_became_active":"2023-11-06T02:09:39.530243+0000","last_became_peered":"2023-11-06T02:09:39.530243+0000","last_unstale":"2023-11-06T02:09:40.892504+0000","last_undegraded":"2023-11-06T02:09:40.892504+0000","last_fullsized":"2023-11-06T02:09:40.892504+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:10:49.280878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.666195+0000","last_change":"2023-11-06T02:09:39.916393+0000","last_active":"2023-11-06T02:09:42.666195+0000","last_peered":"2023-11-06T02:09:42.666195+0000","last_clean":"2023-11-06T02:09:42.666195+0000","last_became_active":"2023-11-06T02:09:39.916048+0000","last_became_peered":"2023-11-06T02:09:39.916048+0000","last_unstale":"2023-11-06T02:09:42.666195+0000","last_undegraded":"2023-11-06T02:09:42.666195+0000","last_fullsized":"2023-11-06T02:09:42.666195+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:03:27.079692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"36'1","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.617139+0000","last_change":"2023-11-06T02:09:39.528406+0000","last_active":"2023-11-06T02:09:42.617139+0000","last_peered":"2023-11-06T02:09:42.617139+0000","last_clean":"2023-11-06T02:09:42.617139+0000","last_became_active":"2023-11-06T02:09:39.527718+0000","last_became_peered":"2023-11-06T02:09:39.527718+0000","last_unstale":"2023-11-06T02:09:42.617139+0000","last_undegraded":"2023-11-06T02:09:42.617139+0000","last_fullsized":"2023-11-06T02:09:42.617139+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:23:30.732017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.617319+0000","last_change":"2023-11-06T02:09:39.951145+0000","last_active":"2023-11-06T02:09:42.617319+0000","last_peered":"2023-11-06T02:09:42.617319+0000","last_clean":"2023-11-06T02:09:42.617319+0000","last_became_active":"2023-11-06T02:09:39.948408+0000","last_became_peered":"2023-11-06T02:09:39.948408+0000","last_unstale":"2023-11-06T02:09:42.617319+0000","last_undegraded":"2023-11-06T02:09:42.617319+0000","last_fullsized":"2023-11-06T02:09:42.617319+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:46:16.536437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"36'2","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.445803+0000","last_change":"2023-11-06T02:09:39.537804+0000","last_active":"2023-11-06T02:09:42.445803+0000","last_peered":"2023-11-06T02:09:42.445803+0000","last_clean":"2023-11-06T02:09:42.445803+0000","last_became_active":"2023-11-06T02:09:39.537228+0000","last_became_peered":"2023-11-06T02:09:39.537228+0000","last_unstale":"2023-11-06T02:09:42.445803+0000","last_undegraded":"2023-11-06T02:09:42.445803+0000","last_fullsized":"2023-11-06T02:09:42.445803+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:49:02.197841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.445634+0000","last_change":"2023-11-06T02:09:39.899930+0000","last_active":"2023-11-06T02:09:42.445634+0000","last_peered":"2023-11-06T02:09:42.445634+0000","last_clean":"2023-11-06T02:09:42.445634+0000","last_became_active":"2023-11-06T02:09:39.898977+0000","last_became_peered":"2023-11-06T02:09:39.898977+0000","last_unstale":"2023-11-06T02:09:42.445634+0000","last_undegraded":"2023-11-06T02:09:42.445634+0000","last_fullsized":"2023-11-06T02:09:42.445634+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:28:05.481974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"36'1","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:40.890057+0000","last_change":"2023-11-06T02:09:39.531338+0000","last_active":"2023-11-06T02:09:40.890057+0000","last_peered":"2023-11-06T02:09:40.890057+0000","last_clean":"2023-11-06T02:09:40.890057+0000","last_became_active":"2023-11-06T02:09:39.530227+0000","last_became_peered":"2023-11-06T02:09:39.530227+0000","last_unstale":"2023-11-06T02:09:40.890057+0000","last_undegraded":"2023-11-06T02:09:40.890057+0000","last_fullsized":"2023-11-06T02:09:40.890057+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:16:11.505589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:40.889974+0000","last_change":"2023-11-06T02:09:39.891558+0000","last_active":"2023-11-06T02:09:40.889974+0000","last_peered":"2023-11-06T02:09:40.889974+0000","last_clean":"2023-11-06T02:09:40.889974+0000","last_became_active":"2023-11-06T02:09:39.891176+0000","last_became_peered":"2023-11-06T02:09:39.891176+0000","last_unstale":"2023-11-06T02:09:40.889974+0000","last_undegraded":"2023-11-06T02:09:40.889974+0000","last_fullsized":"2023-11-06T02:09:40.889974+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:06:30.245273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.666918+0000","last_change":"2023-11-06T02:09:39.516262+0000","last_active":"2023-11-06T02:09:42.666918+0000","last_peered":"2023-11-06T02:09:42.666918+0000","last_clean":"2023-11-06T02:09:42.666918+0000","last_became_active":"2023-11-06T02:09:39.515910+0000","last_became_peered":"2023-11-06T02:09:39.515910+0000","last_unstale":"2023-11-06T02:09:42.666918+0000","last_undegraded":"2023-11-06T02:09:42.666918+0000","last_fullsized":"2023-11-06T02:09:42.666918+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:41:07.012516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:40.893013+0000","last_change":"2023-11-06T02:09:39.925305+0000","last_active":"2023-11-06T02:09:40.893013+0000","last_peered":"2023-11-06T02:09:40.893013+0000","last_clean":"2023-11-06T02:09:40.893013+0000","last_became_active":"2023-11-06T02:09:39.924545+0000","last_became_peered":"2023-11-06T02:09:39.924545+0000","last_unstale":"2023-11-06T02:09:40.893013+0000","last_undegraded":"2023-11-06T02:09:40.893013+0000","last_fullsized":"2023-11-06T02:09:40.893013+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:24:44.156346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"36'2","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:40.889975+0000","last_change":"2023-11-06T02:09:39.521591+0000","last_active":"2023-11-06T02:09:40.889975+0000","last_peered":"2023-11-06T02:09:40.889975+0000","last_clean":"2023-11-06T02:09:40.889975+0000","last_became_active":"2023-11-06T02:09:39.521127+0000","last_became_peered":"2023-11-06T02:09:39.521127+0000","last_unstale":"2023-11-06T02:09:40.889975+0000","last_undegraded":"2023-11-06T02:09:40.889975+0000","last_fullsized":"2023-11-06T02:09:40.889975+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:03:26.558497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:41.492794+0000","last_change":"2023-11-06T02:09:39.927086+0000","last_active":"2023-11-06T02:09:41.492794+0000","last_peered":"2023-11-06T02:09:41.492794+0000","last_clean":"2023-11-06T02:09:41.492794+0000","last_became_active":"2023-11-06T02:09:39.926791+0000","last_became_peered":"2023-11-06T02:09:39.926791+0000","last_unstale":"2023-11-06T02:09:41.492794+0000","last_undegraded":"2023-11-06T02:09:41.492794+0000","last_fullsized":"2023-11-06T02:09:41.492794+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:21:42.877476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:40.889754+0000","last_change":"2023-11-06T02:09:39.528293+0000","last_active":"2023-11-06T02:09:40.889754+0000","last_peered":"2023-11-06T02:09:40.889754+0000","last_clean":"2023-11-06T02:09:40.889754+0000","last_became_active":"2023-11-06T02:09:39.527957+0000","last_became_peered":"2023-11-06T02:09:39.527957+0000","last_unstale":"2023-11-06T02:09:40.889754+0000","last_undegraded":"2023-11-06T02:09:40.889754+0000","last_fullsized":"2023-11-06T02:09:40.889754+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:48:21.380950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.616240+0000","last_change":"2023-11-06T02:09:39.945627+0000","last_active":"2023-11-06T02:09:42.616240+0000","last_peered":"2023-11-06T02:09:42.616240+0000","last_clean":"2023-11-06T02:09:42.616240+0000","last_became_active":"2023-11-06T02:09:39.943242+0000","last_became_peered":"2023-11-06T02:09:39.943242+0000","last_unstale":"2023-11-06T02:09:42.616240+0000","last_undegraded":"2023-11-06T02:09:42.616240+0000","last_fullsized":"2023-11-06T02:09:42.616240+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:37:29.456658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:40.893963+0000","last_change":"2023-11-06T02:09:39.528907+0000","last_active":"2023-11-06T02:09:40.893963+0000","last_peered":"2023-11-06T02:09:40.893963+0000","last_clean":"2023-11-06T02:09:40.893963+0000","last_became_active":"2023-11-06T02:09:39.528318+0000","last_became_peered":"2023-11-06T02:09:39.528318+0000","last_unstale":"2023-11-06T02:09:40.893963+0000","last_undegraded":"2023-11-06T02:09:40.893963+0000","last_fullsized":"2023-11-06T02:09:40.893963+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:06:29.233287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.666520+0000","last_change":"2023-11-06T02:09:39.925619+0000","last_active":"2023-11-06T02:09:42.666520+0000","last_peered":"2023-11-06T02:09:42.666520+0000","last_clean":"2023-11-06T02:09:42.666520+0000","last_became_active":"2023-11-06T02:09:39.925306+0000","last_became_peered":"2023-11-06T02:09:39.925306+0000","last_unstale":"2023-11-06T02:09:42.666520+0000","last_undegraded":"2023-11-06T02:09:42.666520+0000","last_fullsized":"2023-11-06T02:09:42.666520+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:04:26.233460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:40.889523+0000","last_change":"2023-11-06T02:09:39.516004+0000","last_active":"2023-11-06T02:09:40.889523+0000","last_peered":"2023-11-06T02:09:40.889523+0000","last_clean":"2023-11-06T02:09:40.889523+0000","last_became_active":"2023-11-06T02:09:39.514485+0000","last_became_peered":"2023-11-06T02:09:39.514485+0000","last_unstale":"2023-11-06T02:09:40.889523+0000","last_undegraded":"2023-11-06T02:09:40.889523+0000","last_fullsized":"2023-11-06T02:09:40.889523+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:10:37.611351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.616984+0000","last_change":"2023-11-06T02:09:39.948726+0000","last_active":"2023-11-06T02:09:42.616984+0000","last_peered":"2023-11-06T02:09:42.616984+0000","last_clean":"2023-11-06T02:09:42.616984+0000","last_became_active":"2023-11-06T02:09:39.948138+0000","last_became_peered":"2023-11-06T02:09:39.948138+0000","last_unstale":"2023-11-06T02:09:42.616984+0000","last_undegraded":"2023-11-06T02:09:42.616984+0000","last_fullsized":"2023-11-06T02:09:42.616984+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:24:44.156346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:40.895071+0000","last_change":"2023-11-06T02:09:39.528889+0000","last_active":"2023-11-06T02:09:40.895071+0000","last_peered":"2023-11-06T02:09:40.895071+0000","last_clean":"2023-11-06T02:09:40.895071+0000","last_became_active":"2023-11-06T02:09:39.528371+0000","last_became_peered":"2023-11-06T02:09:39.528371+0000","last_unstale":"2023-11-06T02:09:40.895071+0000","last_undegraded":"2023-11-06T02:09:40.895071+0000","last_fullsized":"2023-11-06T02:09:40.895071+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:02:37.444340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":179,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-06T02:09:39.909075+0000","last_change":"2023-11-06T02:08:20.523427+0000","last_active":"2023-11-06T02:09:39.909075+0000","last_peered":"2023-11-06T02:09:39.909075+0000","last_clean":"2023-11-06T02:09:39.909075+0000","last_became_active":"2023-11-06T02:08:20.521089+0000","last_became_peered":"2023-11-06T02:08:20.521089+0000","last_unstale":"2023-11-06T02:09:39.909075+0000","last_undegraded":"2023-11-06T02:09:39.909075+0000","last_fullsized":"2023-11-06T02:09:39.909075+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:19.252348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:19.252348+0000","last_clean_scrub_stamp":"2023-11-06T02:08:19.252348+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:30:15.566397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":26,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-06T02:09:39.933443+0000","last_change":"2023-11-06T02:08:26.059502+0000","last_active":"2023-11-06T02:09:39.933443+0000","last_peered":"2023-11-06T02:09:39.933443+0000","last_clean":"2023-11-06T02:09:39.933443+0000","last_became_active":"2023-11-06T02:08:26.058612+0000","last_became_peered":"2023-11-06T02:08:26.058612+0000","last_unstale":"2023-11-06T02:09:39.933443+0000","last_undegraded":"2023-11-06T02:09:39.933443+0000","last_fullsized":"2023-11-06T02:09:39.933443+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:10.626021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.666793+0000","last_change":"2023-11-06T02:09:39.945906+0000","last_active":"2023-11-06T02:09:42.666793+0000","last_peered":"2023-11-06T02:09:42.666793+0000","last_clean":"2023-11-06T02:09:42.666793+0000","last_became_active":"2023-11-06T02:09:39.945686+0000","last_became_peered":"2023-11-06T02:09:39.945686+0000","last_unstale":"2023-11-06T02:09:42.666793+0000","last_undegraded":"2023-11-06T02:09:42.666793+0000","last_fullsized":"2023-11-06T02:09:42.666793+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:36:18.647776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"36'2","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.617264+0000","last_change":"2023-11-06T02:09:39.518660+0000","last_active":"2023-11-06T02:09:42.617264+0000","last_peered":"2023-11-06T02:09:42.617264+0000","last_clean":"2023-11-06T02:09:42.617264+0000","last_became_active":"2023-11-06T02:09:39.517929+0000","last_became_peered":"2023-11-06T02:09:39.517929+0000","last_unstale":"2023-11-06T02:09:42.617264+0000","last_undegraded":"2023-11-06T02:09:42.617264+0000","last_fullsized":"2023-11-06T02:09:42.617264+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:29:59.553436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":26,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-06T02:09:39.908271+0000","last_change":"2023-11-06T02:08:26.060406+0000","last_active":"2023-11-06T02:09:39.908271+0000","last_peered":"2023-11-06T02:09:39.908271+0000","last_clean":"2023-11-06T02:09:39.908271+0000","last_became_active":"2023-11-06T02:08:26.059262+0000","last_became_peered":"2023-11-06T02:08:26.059262+0000","last_unstale":"2023-11-06T02:09:39.908271+0000","last_undegraded":"2023-11-06T02:09:39.908271+0000","last_fullsized":"2023-11-06T02:09:39.908271+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:10.626021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:40.894343+0000","last_change":"2023-11-06T02:09:39.935264+0000","last_active":"2023-11-06T02:09:40.894343+0000","last_peered":"2023-11-06T02:09:40.894343+0000","last_clean":"2023-11-06T02:09:40.894343+0000","last_became_active":"2023-11-06T02:09:39.934501+0000","last_became_peered":"2023-11-06T02:09:39.934501+0000","last_unstale":"2023-11-06T02:09:40.894343+0000","last_undegraded":"2023-11-06T02:09:40.894343+0000","last_fullsized":"2023-11-06T02:09:40.894343+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:49:03.209827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"37'1","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.614843+0000","last_change":"2023-11-06T02:09:39.538233+0000","last_active":"2023-11-06T02:09:42.614843+0000","last_peered":"2023-11-06T02:09:42.614843+0000","last_clean":"2023-11-06T02:09:42.614843+0000","last_became_active":"2023-11-06T02:09:39.537707+0000","last_became_peered":"2023-11-06T02:09:39.537707+0000","last_unstale":"2023-11-06T02:09:42.614843+0000","last_undegraded":"2023-11-06T02:09:42.614843+0000","last_fullsized":"2023-11-06T02:09:42.614843+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:46:10.180159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":26,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-06T02:09:39.880659+0000","last_change":"2023-11-06T02:08:26.559477+0000","last_active":"2023-11-06T02:09:39.880659+0000","last_peered":"2023-11-06T02:09:39.880659+0000","last_clean":"2023-11-06T02:09:39.880659+0000","last_became_active":"2023-11-06T02:08:26.559202+0000","last_became_peered":"2023-11-06T02:08:26.559202+0000","last_unstale":"2023-11-06T02:09:39.880659+0000","last_undegraded":"2023-11-06T02:09:39.880659+0000","last_fullsized":"2023-11-06T02:09:39.880659+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:55:04.552404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.666254+0000","last_change":"2023-11-06T02:09:39.904655+0000","last_active":"2023-11-06T02:09:42.666254+0000","last_peered":"2023-11-06T02:09:42.666254+0000","last_clean":"2023-11-06T02:09:42.666254+0000","last_became_active":"2023-11-06T02:09:39.904332+0000","last_became_peered":"2023-11-06T02:09:39.904332+0000","last_unstale":"2023-11-06T02:09:42.666254+0000","last_undegraded":"2023-11-06T02:09:42.666254+0000","last_fullsized":"2023-11-06T02:09:42.666254+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:30:28.443414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:41.493308+0000","last_change":"2023-11-06T02:09:39.511832+0000","last_active":"2023-11-06T02:09:41.493308+0000","last_peered":"2023-11-06T02:09:41.493308+0000","last_clean":"2023-11-06T02:09:41.493308+0000","last_became_active":"2023-11-06T02:09:39.511211+0000","last_became_peered":"2023-11-06T02:09:39.511211+0000","last_unstale":"2023-11-06T02:09:41.493308+0000","last_undegraded":"2023-11-06T02:09:41.493308+0000","last_fullsized":"2023-11-06T02:09:41.493308+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:10:49.280878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":26,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-06T02:09:39.932835+0000","last_change":"2023-11-06T02:08:26.059546+0000","last_active":"2023-11-06T02:09:39.932835+0000","last_peered":"2023-11-06T02:09:39.932835+0000","last_clean":"2023-11-06T02:09:39.932835+0000","last_became_active":"2023-11-06T02:08:26.058806+0000","last_became_peered":"2023-11-06T02:08:26.058806+0000","last_unstale":"2023-11-06T02:09:39.932835+0000","last_undegraded":"2023-11-06T02:09:39.932835+0000","last_fullsized":"2023-11-06T02:09:39.932835+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:23.468510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:41.492874+0000","last_change":"2023-11-06T02:09:39.891191+0000","last_active":"2023-11-06T02:09:41.492874+0000","last_peered":"2023-11-06T02:09:41.492874+0000","last_clean":"2023-11-06T02:09:41.492874+0000","last_became_active":"2023-11-06T02:09:39.890890+0000","last_became_peered":"2023-11-06T02:09:39.890890+0000","last_unstale":"2023-11-06T02:09:41.492874+0000","last_undegraded":"2023-11-06T02:09:41.492874+0000","last_fullsized":"2023-11-06T02:09:41.492874+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:36:18.647776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:41.493313+0000","last_change":"2023-11-06T02:09:39.524444+0000","last_active":"2023-11-06T02:09:41.493313+0000","last_peered":"2023-11-06T02:09:41.493313+0000","last_clean":"2023-11-06T02:09:41.493313+0000","last_became_active":"2023-11-06T02:09:39.523926+0000","last_became_peered":"2023-11-06T02:09:39.523926+0000","last_unstale":"2023-11-06T02:09:41.493313+0000","last_undegraded":"2023-11-06T02:09:41.493313+0000","last_fullsized":"2023-11-06T02:09:41.493313+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:19:53.336403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":25,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-06T02:09:39.885610+0000","last_change":"2023-11-06T02:08:26.436463+0000","last_active":"2023-11-06T02:09:39.885610+0000","last_peered":"2023-11-06T02:09:39.885610+0000","last_clean":"2023-11-06T02:09:39.885610+0000","last_became_active":"2023-11-06T02:08:26.436075+0000","last_became_peered":"2023-11-06T02:08:26.436075+0000","last_unstale":"2023-11-06T02:09:39.885610+0000","last_undegraded":"2023-11-06T02:09:39.885610+0000","last_fullsized":"2023-11-06T02:09:39.885610+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:23.468510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.666644+0000","last_change":"2023-11-06T02:09:39.913449+0000","last_active":"2023-11-06T02:09:42.666644+0000","last_peered":"2023-11-06T02:09:42.666644+0000","last_clean":"2023-11-06T02:09:42.666644+0000","last_became_active":"2023-11-06T02:09:39.913198+0000","last_became_peered":"2023-11-06T02:09:39.913198+0000","last_unstale":"2023-11-06T02:09:42.666644+0000","last_undegraded":"2023-11-06T02:09:42.666644+0000","last_fullsized":"2023-11-06T02:09:42.666644+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:21:42.877476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:40.893941+0000","last_change":"2023-11-06T02:09:39.512989+0000","last_active":"2023-11-06T02:09:40.893941+0000","last_peered":"2023-11-06T02:09:40.893941+0000","last_clean":"2023-11-06T02:09:40.893941+0000","last_became_active":"2023-11-06T02:09:39.512373+0000","last_became_peered":"2023-11-06T02:09:39.512373+0000","last_unstale":"2023-11-06T02:09:40.893941+0000","last_undegraded":"2023-11-06T02:09:40.893941+0000","last_fullsized":"2023-11-06T02:09:40.893941+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:54:33.107256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":26,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-06T02:09:39.916699+0000","last_change":"2023-11-06T02:08:26.436607+0000","last_active":"2023-11-06T02:09:39.916699+0000","last_peered":"2023-11-06T02:09:39.916699+0000","last_clean":"2023-11-06T02:09:39.916699+0000","last_became_active":"2023-11-06T02:08:26.435924+0000","last_became_peered":"2023-11-06T02:08:26.435924+0000","last_unstale":"2023-11-06T02:09:39.916699+0000","last_undegraded":"2023-11-06T02:09:39.916699+0000","last_fullsized":"2023-11-06T02:09:39.916699+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:23.468510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.445511+0000","last_change":"2023-11-06T02:09:39.901835+0000","last_active":"2023-11-06T02:09:42.445511+0000","last_peered":"2023-11-06T02:09:42.445511+0000","last_clean":"2023-11-06T02:09:42.445511+0000","last_became_active":"2023-11-06T02:09:39.901031+0000","last_became_peered":"2023-11-06T02:09:39.901031+0000","last_unstale":"2023-11-06T02:09:42.445511+0000","last_undegraded":"2023-11-06T02:09:42.445511+0000","last_fullsized":"2023-11-06T02:09:42.445511+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:02:27.832459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.444856+0000","last_change":"2023-11-06T02:09:39.523134+0000","last_active":"2023-11-06T02:09:42.444856+0000","last_peered":"2023-11-06T02:09:42.444856+0000","last_clean":"2023-11-06T02:09:42.444856+0000","last_became_active":"2023-11-06T02:09:39.522556+0000","last_became_peered":"2023-11-06T02:09:39.522556+0000","last_unstale":"2023-11-06T02:09:42.444856+0000","last_undegraded":"2023-11-06T02:09:42.444856+0000","last_fullsized":"2023-11-06T02:09:42.444856+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:38:42.071294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":25,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-06T02:09:39.885813+0000","last_change":"2023-11-06T02:08:26.057076+0000","last_active":"2023-11-06T02:09:39.885813+0000","last_peered":"2023-11-06T02:09:39.885813+0000","last_clean":"2023-11-06T02:09:39.885813+0000","last_became_active":"2023-11-06T02:08:26.056419+0000","last_became_peered":"2023-11-06T02:08:26.056419+0000","last_unstale":"2023-11-06T02:09:39.885813+0000","last_undegraded":"2023-11-06T02:09:39.885813+0000","last_fullsized":"2023-11-06T02:09:39.885813+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:10.626021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:40.894004+0000","last_change":"2023-11-06T02:09:39.939164+0000","last_active":"2023-11-06T02:09:40.894004+0000","last_peered":"2023-11-06T02:09:40.894004+0000","last_clean":"2023-11-06T02:09:40.894004+0000","last_became_active":"2023-11-06T02:09:39.938583+0000","last_became_peered":"2023-11-06T02:09:39.938583+0000","last_unstale":"2023-11-06T02:09:40.894004+0000","last_undegraded":"2023-11-06T02:09:40.894004+0000","last_fullsized":"2023-11-06T02:09:40.894004+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:39:49.506672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.667623+0000","last_change":"2023-11-06T02:09:39.507309+0000","last_active":"2023-11-06T02:09:42.667623+0000","last_peered":"2023-11-06T02:09:42.667623+0000","last_clean":"2023-11-06T02:09:42.667623+0000","last_became_active":"2023-11-06T02:09:39.506868+0000","last_became_peered":"2023-11-06T02:09:39.506868+0000","last_unstale":"2023-11-06T02:09:42.667623+0000","last_undegraded":"2023-11-06T02:09:42.667623+0000","last_fullsized":"2023-11-06T02:09:42.667623+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:29:59.553436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":28,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-06T02:09:39.890458+0000","last_change":"2023-11-06T02:08:26.058596+0000","last_active":"2023-11-06T02:09:39.890458+0000","last_peered":"2023-11-06T02:09:39.890458+0000","last_clean":"2023-11-06T02:09:39.890458+0000","last_became_active":"2023-11-06T02:08:26.057015+0000","last_became_peered":"2023-11-06T02:08:26.057015+0000","last_unstale":"2023-11-06T02:09:39.890458+0000","last_undegraded":"2023-11-06T02:09:39.890458+0000","last_fullsized":"2023-11-06T02:09:39.890458+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:55:04.552404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574868,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27960,"kb_used_data":1008,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343880,"statfs":{"total":96636764160,"available":96608133120,"internally_reserved":0,"allocated":1032192,"data_stored":767208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":1.03}]},{"osd":1,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":1.1399999999999999},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.879}]},{"osd":2,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":1.077},{"interface":"front","average":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.81799999999999995}]},{"osd":3,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":1.2849999999999999},{"interface":"front","average":{"1min":1.077,"5min":1.077,"15min":1.077},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":1.1779999999999999}]},{"osd":4,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":1.232},{"interface":"front","average":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.98299999999999998}]},{"osd":5,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.079,"15min":1.079},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":1.2589999999999999},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.7809999999999999,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":1.113}]},{"osd":6,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.202,"15min":1.202},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.6679999999999999,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":1.319},{"interface":"front","average":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.74,"5min":1.74,"15min":1.74},"last":1.363}]}]},{"osd":6,"up_from":13,"seq":55834574867,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27368,"kb_used_data":416,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":425984,"data_stored":175788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.621}]},{"osd":1,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.98599999999999999}]},{"osd":2,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.28,"5min":1.28,"15min":1.28},"last":0.90600000000000003}]},{"osd":3,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.71699999999999997}]},{"osd":4,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.474,"5min":1.474,"15min":1.474},"last":1.0900000000000001},{"interface":"front","average":{"1min":1.04,"5min":1.04,"15min":1.04},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.94299999999999995}]},{"osd":5,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.365,"5min":1.365,"15min":1.365},"last":0.96599999999999997},{"interface":"front","average":{"1min":1.115,"5min":1.115,"15min":1.115},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":1.1439999999999999}]},{"osd":7,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":1.117},{"interface":"front","average":{"1min":1.165,"5min":1.165,"15min":1.165},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":1.034}]}]},{"osd":5,"up_from":13,"seq":55834574868,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":408,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":417792,"data_stored":168665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.6579999999999999,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":0.64300000000000002}]},{"osd":1,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.84399999999999997}]},{"osd":2,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.377,"5min":1.377,"15min":1.377},"last":1.036}]},{"osd":3,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":1.0589999999999999},{"interface":"front","average":{"1min":1.081,"5min":1.081,"15min":1.081},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.7250000000000001,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":0.93300000000000005}]},{"osd":4,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.72899999999999998},{"interface":"front","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":0.90000000000000002}]},{"osd":6,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.68500000000000005},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.95199999999999996}]},{"osd":7,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.155,"5min":1.155,"15min":1.155},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.8089999999999999,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":1.014}]}]},{"osd":4,"up_from":13,"seq":55834574868,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27372,"kb_used_data":420,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344468,"statfs":{"total":96636764160,"available":96608735232,"internally_reserved":0,"allocated":430080,"data_stored":175810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":1.014},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.98499999999999999}]},{"osd":1,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":1.462},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.82599999999999996}]},{"osd":2,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":1.02,"15min":1.02},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.615,"5min":1.615,"15min":1.615},"last":1.319},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.93700000000000006}]},{"osd":3,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.52,"5min":1.52,"15min":1.52},"last":0.87},{"interface":"front","average":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.4970000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":1.194}]},{"osd":5,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"min":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":1.377,"5min":1.377,"15min":1.377},"last":0.72799999999999998},{"interface":"front","average":{"1min":1.056,"5min":1.056,"15min":1.056},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.7869999999999999,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":0.76200000000000001}]},{"osd":6,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":0.89100000000000001},{"interface":"front","average":{"1min":1.198,"5min":1.198,"15min":1.198},"min":{"1min":0.87,"5min":0.87,"15min":0.87},"max":{"1min":2.1190000000000002,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":1.1599999999999999}]},{"osd":7,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.7110000000000001,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":1.111},{"interface":"front","average":{"1min":1.24,"5min":1.24,"15min":1.24},"min":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":1.7410000000000001,"5min":1.7410000000000001,"15min":1.7410000000000001},"last":1.048}]}]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31476,"kb_used_data":428,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340364,"statfs":{"total":96636764160,"available":96604532736,"internally_reserved":0,"allocated":438272,"data_stored":176804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:09:19 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.96899999999999997}]},{"osd":1,"last update":"Mon Nov 6 02:09:19 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.79900000000000004}]},{"osd":2,"last update":"Mon Nov 6 02:09:19 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.88100000000000001}]},{"osd":4,"last update":"Mon Nov 6 02:09:19 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.405,"5min":1.405,"15min":1.405},"last":1.234},{"interface":"front","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.92800000000000005}]},{"osd":5,"last update":"Mon Nov 6 02:09:19 2023","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.276,"5min":1.276,"15min":1.276},"last":1.0369999999999999},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":2.21,"5min":2.21,"15min":2.21},"last":1.0629999999999999}]},{"osd":6,"last update":"Mon Nov 6 02:09:19 2023","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":1.026,"15min":1.026},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":1.1479999999999999},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":1.21}]},{"osd":7,"last update":"Mon Nov 6 02:09:19 2023","interfaces":[{"interface":"back","average":{"1min":1.133,"5min":1.133,"15min":1.133},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":2.2829999999999999,"5min":2.2829999999999999,"15min":2.2829999999999999},"last":1.1279999999999999},{"interface":"front","average":{"1min":1.107,"5min":1.107,"15min":1.107},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":1.3169999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574867,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31464,"kb_used_data":416,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340376,"statfs":{"total":96636764160,"available":96604545024,"internally_reserved":0,"allocated":425984,"data_stored":175276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.40799999999999997}]},{"osd":1,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.38500000000000001}]},{"osd":3,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.438}]},{"osd":4,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.873}]},{"osd":5,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.5369999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.746},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.7110000000000001,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.65000000000000002}]},{"osd":6,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.997,"5min":0.997,"15min":0.997},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.677,"5min":1.677,"15min":1.677},"last":0.66100000000000003}]},{"osd":7,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.633,"5min":1.633,"15min":1.633},"last":0.84399999999999997},{"interface":"front","average":{"1min":1.042,"5min":1.042,"15min":1.042},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":0.81399999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574868,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31464,"kb_used_data":416,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340376,"statfs":{"total":96636764160,"available":96604545024,"internally_reserved":0,"allocated":425984,"data_stored":175296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:09:19 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.90700000000000003}]},{"osd":2,"last update":"Mon Nov 6 02:09:19 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.76600000000000001}]},{"osd":3,"last update":"Mon Nov 6 02:09:19 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.8149999999999999,"5min":1.8149999999999999,"15min":1.8149999999999999},"last":0.79700000000000004}]},{"osd":4,"last update":"Mon Nov 6 02:09:19 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":1.1519999999999999}]},{"osd":5,"last update":"Mon Nov 6 02:09:19 2023","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.98799999999999999},{"interface":"front","average":{"1min":1.079,"5min":1.079,"15min":1.079},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.6759999999999999,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.86199999999999999}]},{"osd":6,"last update":"Mon Nov 6 02:09:19 2023","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.103,"15min":1.103},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.742,"5min":1.742,"15min":1.742},"last":1.052},{"interface":"front","average":{"1min":1.077,"5min":1.077,"15min":1.077},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.393,"5min":1.393,"15min":1.393},"last":1.079}]},{"osd":7,"last update":"Mon Nov 6 02:09:19 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.486,"5min":1.486,"15min":1.486},"last":1.02},{"interface":"front","average":{"1min":1.161,"5min":1.161,"15min":1.161},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.893,"5min":1.893,"15min":1.893},"last":1.115}]}]},{"osd":0,"up_from":13,"seq":55834574868,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32040,"kb_used_data":992,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339800,"statfs":{"total":96636764160,"available":96603955200,"internally_reserved":0,"allocated":1015808,"data_stored":764589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.71999999999999997}]},{"osd":2,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":1.0129999999999999},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":1.0589999999999999}]},{"osd":3,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":1.25}]},{"osd":4,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":1.163},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":1.3839999999999999}]},{"osd":5,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":1.3140000000000001},{"interface":"front","average":{"1min":0.872,"5min":0.872,"15min":0.872},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.448,"5min":1.448,"15min":1.448},"last":1.351}]},{"osd":6,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":1.581},{"interface":"front","average":{"1min":1,"5min":1,"15min":1},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":1.5529999999999999}]},{"osd":7,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":1.421},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.556,"5min":1.556,"15min":1.556},"last":1.4950000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T02:09:48.699 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T02:09:48.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:48.836+0000 7f72ad73d640 1 Processor -- start 2023-11-06T02:09:48.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:48.836+0000 7f72ad73d640 1 -- start start 2023-11-06T02:09:48.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:48.836+0000 7f72ad73d640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f72a815ab50 0x7f72a80c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:48.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:48.836+0000 7f72ad73d640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f72a80cb010 0x7f72a80b0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:48.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:48.836+0000 7f72ad73d640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72a80cab80 con 0x7f72a80b05c0 2023-11-06T02:09:48.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:48.836+0000 7f72ad73d640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72a80b0a50 con 0x7f72a80cb010 2023-11-06T02:09:48.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:48.836+0000 7f72ad73d640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72a815af30 con 0x7f72a815ab50 2023-11-06T02:09:48.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:48.836+0000 7f72a67fc640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f72a80cb010 0x7f72a80b0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:48.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:48.836+0000 7f72a77fe640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f72a80b05c0 0x7f72a80700a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:49362/0 (socket says 172.21.15.49:49362) 2023-11-06T02:09:48.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:48.836+0000 7f72a77fe640 1 -- 172.21.15.49:0/2009684065 learned_addr learned my addr 172.21.15.49:0/2009684065 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:09:48.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:48.836+0000 7f72a6ffd640 1 --2- 172.21.15.49:0/2009684065 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f72a815ab50 0x7f72a80c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:48.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:48.836+0000 7f72a67fc640 1 -- 172.21.15.49:0/2009684065 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f72a815ab50 msgr2=0x7f72a80c9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:48.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:48.836+0000 7f72a67fc640 1 --2- 172.21.15.49:0/2009684065 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f72a815ab50 0x7f72a80c9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:48.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:48.836+0000 7f72a67fc640 1 -- 172.21.15.49:0/2009684065 >> v1:172.21.15.49:6789/0 conn(0x7f72a80b05c0 legacy=0x7f72a80700a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:48.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:48.836+0000 7f72a67fc640 1 -- 172.21.15.49:0/2009684065 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72a80cb5b0 con 0x7f72a80cb010 2023-11-06T02:09:48.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:48.836+0000 7f72a5ffb640 1 -- 172.21.15.49:0/2009684065 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3341996010 0 0) 0x7f72a80cab80 con 0x7f72a80b05c0 2023-11-06T02:09:48.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:48.836+0000 7f72a6ffd640 1 --2- 172.21.15.49:0/2009684065 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f72a815ab50 0x7f72a80c9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:09:48.841 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:48.840+0000 7f72a67fc640 1 --2- 172.21.15.49:0/2009684065 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f72a80cb010 0x7f72a80b0080 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f729c0097c0 tx=0x7f729c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dceece5a10ba3b8 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:48.842 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:48.840+0000 7f72a5ffb640 1 -- 172.21.15.49:0/2009684065 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f729c0ab650 con 0x7f72a80cb010 2023-11-06T02:09:48.842 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:48.840+0000 7f72a5ffb640 1 -- 172.21.15.49:0/2009684065 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f729c0ab7b0 con 0x7f72a80cb010 2023-11-06T02:09:48.842 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:48.840+0000 7f72a5ffb640 1 -- 172.21.15.49:0/2009684065 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f729c0abb40 con 0x7f72a80cb010 2023-11-06T02:09:48.842 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:48.840+0000 7f72ad73d640 1 -- 172.21.15.49:0/2009684065 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f72a80cb010 msgr2=0x7f72a80b0080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:48.842 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:48.840+0000 7f72ad73d640 1 --2- 172.21.15.49:0/2009684065 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f72a80cb010 0x7f72a80b0080 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f729c0097c0 tx=0x7f729c01c450 comp rx=0 tx=0).stop 2023-11-06T02:09:48.843 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:48.840+0000 7f72ad73d640 1 -- 172.21.15.49:0/2009684065 shutdown_connections 2023-11-06T02:09:48.843 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:48.840+0000 7f72ad73d640 1 --2- 172.21.15.49:0/2009684065 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f72a80cb010 0x7f72a80b0080 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:48.843 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:48.840+0000 7f72ad73d640 1 --2- 172.21.15.49:0/2009684065 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f72a815ab50 0x7f72a80c9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:48.843 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:48.840+0000 7f72ad73d640 1 -- 172.21.15.49:0/2009684065 >> 172.21.15.49:0/2009684065 conn(0x7f72a806c0a0 msgr2=0x7f72a806c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:48.843 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:48.840+0000 7f72ad73d640 1 -- 172.21.15.49:0/2009684065 shutdown_connections 2023-11-06T02:09:48.844 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:48.840+0000 7f72ad73d640 1 -- 172.21.15.49:0/2009684065 wait complete. 2023-11-06T02:09:48.844 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:48.840+0000 7f72ad73d640 1 Processor -- start 2023-11-06T02:09:48.844 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:48.840+0000 7f72ad73d640 1 -- start start 2023-11-06T02:09:48.844 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:48.840+0000 7f72ad73d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f72a80b05c0 0x7f72a815b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:48.845 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:48.840+0000 7f72ad73d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72a80cb010 0x7f72a814c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:48.845 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:48.840+0000 7f72ad73d640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f72a815ab50 0x7f72a81512e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:48.845 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:48.840+0000 7f72ad73d640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a814cdc0 con 0x7f72a80b05c0 2023-11-06T02:09:48.845 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:48.840+0000 7f72ad73d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a8159760 con 0x7f72a80cb010 2023-11-06T02:09:48.845 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:48.840+0000 7f72ad73d640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72a814d1c0 con 0x7f72a815ab50 2023-11-06T02:09:48.846 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:48.840+0000 7f72a6ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f72a80b05c0 0x7f72a815b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:48.846 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:48.840+0000 7f72a6ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f72a80b05c0 0x7f72a815b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44566/0 (socket says 172.21.15.49:44566) 2023-11-06T02:09:48.846 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:48.840+0000 7f72a6ffd640 1 -- 172.21.15.49:0/381679004 learned_addr learned my addr 172.21.15.49:0/381679004 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:48.846 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:48.840+0000 7f72a77fe640 1 --2- 172.21.15.49:0/381679004 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f72a815ab50 0x7f72a81512e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:48.846 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:48.840+0000 7f72a67fc640 1 --2- 172.21.15.49:0/381679004 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72a80cb010 0x7f72a814c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:48.847 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:48.840+0000 7f72a6ffd640 1 -- 172.21.15.49:0/381679004 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f72a815ab50 msgr2=0x7f72a81512e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:48.847 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:48.840+0000 7f72a6ffd640 1 --2- 172.21.15.49:0/381679004 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f72a815ab50 0x7f72a81512e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:48.847 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:48.840+0000 7f72a6ffd640 1 -- 172.21.15.49:0/381679004 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72a80cb010 msgr2=0x7f72a814c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:48.847 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:48.840+0000 7f72a6ffd640 1 --2- 172.21.15.49:0/381679004 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72a80cb010 0x7f72a814c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:48.848 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:48.840+0000 7f72a6ffd640 1 -- 172.21.15.49:0/381679004 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f729c009420 con 0x7f72a80b05c0 2023-11-06T02:09:48.848 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:48.840+0000 7f72a67fc640 1 --2- 172.21.15.49:0/381679004 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72a80cb010 0x7f72a814c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:09:48.848 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:48.840+0000 7f72a6ffd640 1 --2- 172.21.15.49:0/381679004 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f72a80b05c0 0x7f72a815b130 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f729800c7f0 tx=0x7f729800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:48.848 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:48.844+0000 7f728f7fe640 1 -- 172.21.15.49:0/381679004 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7298005ba0 con 0x7f72a80b05c0 2023-11-06T02:09:48.849 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:48.844+0000 7f728f7fe640 1 -- 172.21.15.49:0/381679004 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7298005d00 con 0x7f72a80b05c0 2023-11-06T02:09:48.849 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:48.844+0000 7f72ad73d640 1 -- 172.21.15.49:0/381679004 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72a815a320 con 0x7f72a80b05c0 2023-11-06T02:09:48.849 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:48.844+0000 7f728f7fe640 1 -- 172.21.15.49:0/381679004 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72980079f0 con 0x7f72a80b05c0 2023-11-06T02:09:48.849 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:48.844+0000 7f72ad73d640 1 -- 172.21.15.49:0/381679004 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72a8159fb0 con 0x7f72a80b05c0 2023-11-06T02:09:48.851 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:48.844+0000 7f72ad73d640 1 -- 172.21.15.49:0/381679004 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72a80b0de0 con 0x7f72a80b05c0 2023-11-06T02:09:48.851 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:48.844+0000 7f728f7fe640 1 -- 172.21.15.49:0/381679004 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f7298007b50 con 0x7f72a80b05c0 2023-11-06T02:09:48.851 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:48.844+0000 7f728f7fe640 1 --2- 172.21.15.49:0/381679004 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7294065a60 0x7f7294067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:48.851 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:48.844+0000 7f728f7fe640 1 -- 172.21.15.49:0/381679004 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f72980835f0 con 0x7f72a80b05c0 2023-11-06T02:09:48.852 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:48.844+0000 7f72a67fc640 1 --2- 172.21.15.49:0/381679004 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7294065a60 0x7f7294067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:48.852 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:48.848+0000 7f72a67fc640 1 --2- 172.21.15.49:0/381679004 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7294065a60 0x7f7294067f20 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f729c0af000 tx=0x7f729c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:48.852 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:48.848+0000 7f728f7fe640 1 -- 172.21.15.49:0/381679004 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f7298050fc0 con 0x7f72a80b05c0 2023-11-06T02:09:48.999 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.000+0000 7f72ad73d640 1 -- 172.21.15.49:0/381679004 --> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f72a80cbe50 con 0x7f7294065a60 2023-11-06T02:09:49.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.024+0000 7f728f7fe640 1 -- 172.21.15.49:0/381679004 <== mgr.4100 v2:172.21.15.150:6832/38104 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+309511 (secure 0 0 0) 0x7f729c0bc090 con 0x7f7294065a60 2023-11-06T02:09:49.025 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:09:49.028 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2023-11-06T02:09:49.029 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.028+0000 7f72ad73d640 1 -- 172.21.15.49:0/381679004 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7294065a60 msgr2=0x7f7294067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:49.029 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.028+0000 7f72ad73d640 1 --2- 172.21.15.49:0/381679004 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7294065a60 0x7f7294067f20 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f729c0af000 tx=0x7f729c008040 comp rx=0 tx=0).stop 2023-11-06T02:09:49.029 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.028+0000 7f72ad73d640 1 -- 172.21.15.49:0/381679004 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f72a80b05c0 msgr2=0x7f72a815b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:49.030 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.028+0000 7f72ad73d640 1 --2- 172.21.15.49:0/381679004 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f72a80b05c0 0x7f72a815b130 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f729800c7f0 tx=0x7f729800cbf0 comp rx=0 tx=0).stop 2023-11-06T02:09:49.030 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.028+0000 7f72ad73d640 1 -- 172.21.15.49:0/381679004 shutdown_connections 2023-11-06T02:09:49.030 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.028+0000 7f72ad73d640 1 --2- 172.21.15.49:0/381679004 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7294065a60 0x7f7294067f20 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:49.030 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.028+0000 7f72ad73d640 1 --2- 172.21.15.49:0/381679004 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f72a815ab50 0x7f72a81512e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:49.030 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.028+0000 7f72ad73d640 1 --2- 172.21.15.49:0/381679004 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72a80cb010 0x7f72a814c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:49.031 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.028+0000 7f72ad73d640 1 --2- 172.21.15.49:0/381679004 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f72a80b05c0 0x7f72a815b130 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:49.031 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.028+0000 7f72ad73d640 1 -- 172.21.15.49:0/381679004 >> 172.21.15.49:0/381679004 conn(0x7f72a806c0a0 msgr2=0x7f72a80c9bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:49.031 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.028+0000 7f72ad73d640 1 -- 172.21.15.49:0/381679004 shutdown_connections 2023-11-06T02:09:49.031 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.028+0000 7f72ad73d640 1 -- 172.21.15.49:0/381679004 wait complete. 2023-11-06T02:09:49.047 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":75,"stamp":"2023-11-06T02:09:47.202443+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":181,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236504,"kb_used_data":4504,"kb_used_omap":0,"kb_used_meta":231936,"kb_avail":754738216,"statfs":{"total":773094113280,"available":772851933184,"internally_reserved":0,"allocated":4612096,"data_stored":2579436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":237502464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":54,"apply_latency_ms":54,"commit_latency_ns":54000000,"apply_latency_ns":54000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.713906"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.666671+0000","last_change":"2023-11-06T02:09:39.894697+0000","last_active":"2023-11-06T02:09:42.666671+0000","last_peered":"2023-11-06T02:09:42.666671+0000","last_clean":"2023-11-06T02:09:42.666671+0000","last_became_active":"2023-11-06T02:09:39.894432+0000","last_became_peered":"2023-11-06T02:09:39.894432+0000","last_unstale":"2023-11-06T02:09:42.666671+0000","last_undegraded":"2023-11-06T02:09:42.666671+0000","last_fullsized":"2023-11-06T02:09:42.666671+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:18:48.057417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.666162+0000","last_change":"2023-11-06T02:09:39.893106+0000","last_active":"2023-11-06T02:09:42.666162+0000","last_peered":"2023-11-06T02:09:42.666162+0000","last_clean":"2023-11-06T02:09:42.666162+0000","last_became_active":"2023-11-06T02:09:39.892854+0000","last_became_peered":"2023-11-06T02:09:39.892854+0000","last_unstale":"2023-11-06T02:09:42.666162+0000","last_undegraded":"2023-11-06T02:09:42.666162+0000","last_fullsized":"2023-11-06T02:09:42.666162+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:18:26.908517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:41.493454+0000","last_change":"2023-11-06T02:09:39.950175+0000","last_active":"2023-11-06T02:09:41.493454+0000","last_peered":"2023-11-06T02:09:41.493454+0000","last_clean":"2023-11-06T02:09:41.493454+0000","last_became_active":"2023-11-06T02:09:39.949828+0000","last_became_peered":"2023-11-06T02:09:39.949828+0000","last_unstale":"2023-11-06T02:09:41.493454+0000","last_undegraded":"2023-11-06T02:09:41.493454+0000","last_fullsized":"2023-11-06T02:09:41.493454+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:24:44.156346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:40.889850+0000","last_change":"2023-11-06T02:09:39.896077+0000","last_active":"2023-11-06T02:09:40.889850+0000","last_peered":"2023-11-06T02:09:40.889850+0000","last_clean":"2023-11-06T02:09:40.889850+0000","last_became_active":"2023-11-06T02:09:39.895853+0000","last_became_peered":"2023-11-06T02:09:39.895853+0000","last_unstale":"2023-11-06T02:09:40.889850+0000","last_undegraded":"2023-11-06T02:09:40.889850+0000","last_fullsized":"2023-11-06T02:09:40.889850+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:30:00.565422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:40.889686+0000","last_change":"2023-11-06T02:09:39.892525+0000","last_active":"2023-11-06T02:09:40.889686+0000","last_peered":"2023-11-06T02:09:40.889686+0000","last_clean":"2023-11-06T02:09:40.889686+0000","last_became_active":"2023-11-06T02:09:39.892227+0000","last_became_peered":"2023-11-06T02:09:39.892227+0000","last_unstale":"2023-11-06T02:09:40.889686+0000","last_undegraded":"2023-11-06T02:09:40.889686+0000","last_fullsized":"2023-11-06T02:09:40.889686+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:17:06.187075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.666612+0000","last_change":"2023-11-06T02:09:39.892888+0000","last_active":"2023-11-06T02:09:42.666612+0000","last_peered":"2023-11-06T02:09:42.666612+0000","last_clean":"2023-11-06T02:09:42.666612+0000","last_became_active":"2023-11-06T02:09:39.892324+0000","last_became_peered":"2023-11-06T02:09:39.892324+0000","last_unstale":"2023-11-06T02:09:42.666612+0000","last_undegraded":"2023-11-06T02:09:42.666612+0000","last_fullsized":"2023-11-06T02:09:42.666612+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:43:55.630807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.616213+0000","last_change":"2023-11-06T02:09:39.935211+0000","last_active":"2023-11-06T02:09:42.616213+0000","last_peered":"2023-11-06T02:09:42.616213+0000","last_clean":"2023-11-06T02:09:42.616213+0000","last_became_active":"2023-11-06T02:09:39.934429+0000","last_became_peered":"2023-11-06T02:09:39.934429+0000","last_unstale":"2023-11-06T02:09:42.616213+0000","last_undegraded":"2023-11-06T02:09:42.616213+0000","last_fullsized":"2023-11-06T02:09:42.616213+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:14:47.981503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.665585+0000","last_change":"2023-11-06T02:09:39.936779+0000","last_active":"2023-11-06T02:09:42.665585+0000","last_peered":"2023-11-06T02:09:42.665585+0000","last_clean":"2023-11-06T02:09:42.665585+0000","last_became_active":"2023-11-06T02:09:39.936501+0000","last_became_peered":"2023-11-06T02:09:39.936501+0000","last_unstale":"2023-11-06T02:09:42.665585+0000","last_undegraded":"2023-11-06T02:09:42.665585+0000","last_fullsized":"2023-11-06T02:09:42.665585+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:37:29.456658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:41.493698+0000","last_change":"2023-11-06T02:09:39.948773+0000","last_active":"2023-11-06T02:09:41.493698+0000","last_peered":"2023-11-06T02:09:41.493698+0000","last_clean":"2023-11-06T02:09:41.493698+0000","last_became_active":"2023-11-06T02:09:39.948499+0000","last_became_peered":"2023-11-06T02:09:39.948499+0000","last_unstale":"2023-11-06T02:09:41.493698+0000","last_undegraded":"2023-11-06T02:09:41.493698+0000","last_fullsized":"2023-11-06T02:09:41.493698+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:59:20.637813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.617038+0000","last_change":"2023-11-06T02:09:39.944022+0000","last_active":"2023-11-06T02:09:42.617038+0000","last_peered":"2023-11-06T02:09:42.617038+0000","last_clean":"2023-11-06T02:09:42.617038+0000","last_became_active":"2023-11-06T02:09:39.943075+0000","last_became_peered":"2023-11-06T02:09:39.943075+0000","last_unstale":"2023-11-06T02:09:42.617038+0000","last_undegraded":"2023-11-06T02:09:42.617038+0000","last_fullsized":"2023-11-06T02:09:42.617038+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:04:26.233460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.617072+0000","last_change":"2023-11-06T02:09:39.950021+0000","last_active":"2023-11-06T02:09:42.617072+0000","last_peered":"2023-11-06T02:09:42.617072+0000","last_clean":"2023-11-06T02:09:42.617072+0000","last_became_active":"2023-11-06T02:09:39.945579+0000","last_became_peered":"2023-11-06T02:09:39.945579+0000","last_unstale":"2023-11-06T02:09:42.617072+0000","last_undegraded":"2023-11-06T02:09:42.617072+0000","last_fullsized":"2023-11-06T02:09:42.617072+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:03:27.079692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:41.493328+0000","last_change":"2023-11-06T02:09:39.894500+0000","last_active":"2023-11-06T02:09:41.493328+0000","last_peered":"2023-11-06T02:09:41.493328+0000","last_clean":"2023-11-06T02:09:41.493328+0000","last_became_active":"2023-11-06T02:09:39.893806+0000","last_became_peered":"2023-11-06T02:09:39.893806+0000","last_unstale":"2023-11-06T02:09:41.493328+0000","last_undegraded":"2023-11-06T02:09:41.493328+0000","last_fullsized":"2023-11-06T02:09:41.493328+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:39:49.506672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.616771+0000","last_change":"2023-11-06T02:09:39.936301+0000","last_active":"2023-11-06T02:09:42.616771+0000","last_peered":"2023-11-06T02:09:42.616771+0000","last_clean":"2023-11-06T02:09:42.616771+0000","last_became_active":"2023-11-06T02:09:39.934917+0000","last_became_peered":"2023-11-06T02:09:39.934917+0000","last_unstale":"2023-11-06T02:09:42.616771+0000","last_undegraded":"2023-11-06T02:09:42.616771+0000","last_fullsized":"2023-11-06T02:09:42.616771+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:18:48.057417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.444971+0000","last_change":"2023-11-06T02:09:39.920273+0000","last_active":"2023-11-06T02:09:42.444971+0000","last_peered":"2023-11-06T02:09:42.444971+0000","last_clean":"2023-11-06T02:09:42.444971+0000","last_became_active":"2023-11-06T02:09:39.917769+0000","last_became_peered":"2023-11-06T02:09:39.917769+0000","last_unstale":"2023-11-06T02:09:42.444971+0000","last_undegraded":"2023-11-06T02:09:42.444971+0000","last_fullsized":"2023-11-06T02:09:42.444971+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:30:28.443414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.616264+0000","last_change":"2023-11-06T02:09:39.935981+0000","last_active":"2023-11-06T02:09:42.616264+0000","last_peered":"2023-11-06T02:09:42.616264+0000","last_clean":"2023-11-06T02:09:42.616264+0000","last_became_active":"2023-11-06T02:09:39.933574+0000","last_became_peered":"2023-11-06T02:09:39.933574+0000","last_unstale":"2023-11-06T02:09:42.616264+0000","last_undegraded":"2023-11-06T02:09:42.616264+0000","last_fullsized":"2023-11-06T02:09:42.616264+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:01:17.009165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:40.892687+0000","last_change":"2023-11-06T02:09:39.946799+0000","last_active":"2023-11-06T02:09:40.892687+0000","last_peered":"2023-11-06T02:09:40.892687+0000","last_clean":"2023-11-06T02:09:40.892687+0000","last_became_active":"2023-11-06T02:09:39.946390+0000","last_became_peered":"2023-11-06T02:09:39.946390+0000","last_unstale":"2023-11-06T02:09:40.892687+0000","last_undegraded":"2023-11-06T02:09:40.892687+0000","last_fullsized":"2023-11-06T02:09:40.892687+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:38:43.083280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:40.894513+0000","last_change":"2023-11-06T02:09:39.949727+0000","last_active":"2023-11-06T02:09:40.894513+0000","last_peered":"2023-11-06T02:09:40.894513+0000","last_clean":"2023-11-06T02:09:40.894513+0000","last_became_active":"2023-11-06T02:09:39.949138+0000","last_became_peered":"2023-11-06T02:09:39.949138+0000","last_unstale":"2023-11-06T02:09:40.894513+0000","last_undegraded":"2023-11-06T02:09:40.894513+0000","last_fullsized":"2023-11-06T02:09:40.894513+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:59:20.637813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:40.894336+0000","last_change":"2023-11-06T02:09:39.913899+0000","last_active":"2023-11-06T02:09:40.894336+0000","last_peered":"2023-11-06T02:09:40.894336+0000","last_clean":"2023-11-06T02:09:40.894336+0000","last_became_active":"2023-11-06T02:09:39.913107+0000","last_became_peered":"2023-11-06T02:09:39.913107+0000","last_unstale":"2023-11-06T02:09:40.894336+0000","last_undegraded":"2023-11-06T02:09:40.894336+0000","last_fullsized":"2023-11-06T02:09:40.894336+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:45:01.120311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:40.889838+0000","last_change":"2023-11-06T02:09:39.890295+0000","last_active":"2023-11-06T02:09:40.889838+0000","last_peered":"2023-11-06T02:09:40.889838+0000","last_clean":"2023-11-06T02:09:40.889838+0000","last_became_active":"2023-11-06T02:09:39.888442+0000","last_became_peered":"2023-11-06T02:09:39.888442+0000","last_unstale":"2023-11-06T02:09:40.889838+0000","last_undegraded":"2023-11-06T02:09:40.889838+0000","last_fullsized":"2023-11-06T02:09:40.889838+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:06:38.586189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:40.895042+0000","last_change":"2023-11-06T02:09:39.913639+0000","last_active":"2023-11-06T02:09:40.895042+0000","last_peered":"2023-11-06T02:09:40.895042+0000","last_clean":"2023-11-06T02:09:40.895042+0000","last_became_active":"2023-11-06T02:09:39.912758+0000","last_became_peered":"2023-11-06T02:09:39.912758+0000","last_unstale":"2023-11-06T02:09:40.895042+0000","last_undegraded":"2023-11-06T02:09:40.895042+0000","last_fullsized":"2023-11-06T02:09:40.895042+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:19:54.348389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.616361+0000","last_change":"2023-11-06T02:09:39.936823+0000","last_active":"2023-11-06T02:09:42.616361+0000","last_peered":"2023-11-06T02:09:42.616361+0000","last_clean":"2023-11-06T02:09:42.616361+0000","last_became_active":"2023-11-06T02:09:39.935648+0000","last_became_peered":"2023-11-06T02:09:39.935648+0000","last_unstale":"2023-11-06T02:09:42.616361+0000","last_undegraded":"2023-11-06T02:09:42.616361+0000","last_fullsized":"2023-11-06T02:09:42.616361+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:28:05.481974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.445677+0000","last_change":"2023-11-06T02:09:39.898194+0000","last_active":"2023-11-06T02:09:42.445677+0000","last_peered":"2023-11-06T02:09:42.445677+0000","last_clean":"2023-11-06T02:09:42.445677+0000","last_became_active":"2023-11-06T02:09:39.897460+0000","last_became_peered":"2023-11-06T02:09:39.897460+0000","last_unstale":"2023-11-06T02:09:42.445677+0000","last_undegraded":"2023-11-06T02:09:42.445677+0000","last_fullsized":"2023-11-06T02:09:42.445677+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:01:17.009165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:40.893463+0000","last_change":"2023-11-06T02:09:39.928101+0000","last_active":"2023-11-06T02:09:40.893463+0000","last_peered":"2023-11-06T02:09:40.893463+0000","last_clean":"2023-11-06T02:09:40.893463+0000","last_became_active":"2023-11-06T02:09:39.927170+0000","last_became_peered":"2023-11-06T02:09:39.927170+0000","last_unstale":"2023-11-06T02:09:40.893463+0000","last_undegraded":"2023-11-06T02:09:40.893463+0000","last_fullsized":"2023-11-06T02:09:40.893463+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:12:15.009411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:40.889446+0000","last_change":"2023-11-06T02:09:39.924861+0000","last_active":"2023-11-06T02:09:40.889446+0000","last_peered":"2023-11-06T02:09:40.889446+0000","last_clean":"2023-11-06T02:09:40.889446+0000","last_became_active":"2023-11-06T02:09:39.924654+0000","last_became_peered":"2023-11-06T02:09:39.924654+0000","last_unstale":"2023-11-06T02:09:40.889446+0000","last_undegraded":"2023-11-06T02:09:40.889446+0000","last_fullsized":"2023-11-06T02:09:40.889446+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:23:31.744003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:40.893650+0000","last_change":"2023-11-06T02:09:39.917240+0000","last_active":"2023-11-06T02:09:40.893650+0000","last_peered":"2023-11-06T02:09:40.893650+0000","last_clean":"2023-11-06T02:09:40.893650+0000","last_became_active":"2023-11-06T02:09:39.916284+0000","last_became_peered":"2023-11-06T02:09:39.916284+0000","last_unstale":"2023-11-06T02:09:40.893650+0000","last_undegraded":"2023-11-06T02:09:40.893650+0000","last_fullsized":"2023-11-06T02:09:40.893650+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:18:07.867946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.667212+0000","last_change":"2023-11-06T02:09:39.934780+0000","last_active":"2023-11-06T02:09:42.667212+0000","last_peered":"2023-11-06T02:09:42.667212+0000","last_clean":"2023-11-06T02:09:42.667212+0000","last_became_active":"2023-11-06T02:09:39.934264+0000","last_became_peered":"2023-11-06T02:09:39.934264+0000","last_unstale":"2023-11-06T02:09:42.667212+0000","last_undegraded":"2023-11-06T02:09:42.667212+0000","last_fullsized":"2023-11-06T02:09:42.667212+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:38:43.083280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:40.894599+0000","last_change":"2023-11-06T02:09:39.914882+0000","last_active":"2023-11-06T02:09:40.894599+0000","last_peered":"2023-11-06T02:09:40.894599+0000","last_clean":"2023-11-06T02:09:40.894599+0000","last_became_active":"2023-11-06T02:09:39.914057+0000","last_became_peered":"2023-11-06T02:09:39.914057+0000","last_unstale":"2023-11-06T02:09:40.894599+0000","last_undegraded":"2023-11-06T02:09:40.894599+0000","last_fullsized":"2023-11-06T02:09:40.894599+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:10:50.292864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.616845+0000","last_change":"2023-11-06T02:09:39.944891+0000","last_active":"2023-11-06T02:09:42.616845+0000","last_peered":"2023-11-06T02:09:42.616845+0000","last_clean":"2023-11-06T02:09:42.616845+0000","last_became_active":"2023-11-06T02:09:39.944414+0000","last_became_peered":"2023-11-06T02:09:39.944414+0000","last_unstale":"2023-11-06T02:09:42.616845+0000","last_undegraded":"2023-11-06T02:09:42.616845+0000","last_fullsized":"2023-11-06T02:09:42.616845+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:51:33.759418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:40.893934+0000","last_change":"2023-11-06T02:09:39.916632+0000","last_active":"2023-11-06T02:09:40.893934+0000","last_peered":"2023-11-06T02:09:40.893934+0000","last_clean":"2023-11-06T02:09:40.893934+0000","last_became_active":"2023-11-06T02:09:39.914132+0000","last_became_peered":"2023-11-06T02:09:39.914132+0000","last_unstale":"2023-11-06T02:09:40.893934+0000","last_undegraded":"2023-11-06T02:09:40.893934+0000","last_fullsized":"2023-11-06T02:09:40.893934+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:41:08.024502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:40.893821+0000","last_change":"2023-11-06T02:09:39.913252+0000","last_active":"2023-11-06T02:09:40.893821+0000","last_peered":"2023-11-06T02:09:40.893821+0000","last_clean":"2023-11-06T02:09:40.893821+0000","last_became_active":"2023-11-06T02:09:39.912390+0000","last_became_peered":"2023-11-06T02:09:39.912390+0000","last_unstale":"2023-11-06T02:09:40.893821+0000","last_undegraded":"2023-11-06T02:09:40.893821+0000","last_fullsized":"2023-11-06T02:09:40.893821+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:02:38.456326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:40.893731+0000","last_change":"2023-11-06T02:09:39.928747+0000","last_active":"2023-11-06T02:09:40.893731+0000","last_peered":"2023-11-06T02:09:40.893731+0000","last_clean":"2023-11-06T02:09:40.893731+0000","last_became_active":"2023-11-06T02:09:39.927707+0000","last_became_peered":"2023-11-06T02:09:39.927707+0000","last_unstale":"2023-11-06T02:09:40.893731+0000","last_undegraded":"2023-11-06T02:09:40.893731+0000","last_fullsized":"2023-11-06T02:09:40.893731+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:51:33.759418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.667463+0000","last_change":"2023-11-06T02:09:39.893420+0000","last_active":"2023-11-06T02:09:42.667463+0000","last_peered":"2023-11-06T02:09:42.667463+0000","last_clean":"2023-11-06T02:09:42.667463+0000","last_became_active":"2023-11-06T02:09:39.893152+0000","last_became_peered":"2023-11-06T02:09:39.893152+0000","last_unstale":"2023-11-06T02:09:42.667463+0000","last_undegraded":"2023-11-06T02:09:42.667463+0000","last_fullsized":"2023-11-06T02:09:42.667463+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:23:31.744003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:40.894952+0000","last_change":"2023-11-06T02:09:39.946680+0000","last_active":"2023-11-06T02:09:40.894952+0000","last_peered":"2023-11-06T02:09:40.894952+0000","last_clean":"2023-11-06T02:09:40.894952+0000","last_became_active":"2023-11-06T02:09:39.945969+0000","last_became_peered":"2023-11-06T02:09:39.945969+0000","last_unstale":"2023-11-06T02:09:40.894952+0000","last_undegraded":"2023-11-06T02:09:40.894952+0000","last_fullsized":"2023-11-06T02:09:40.894952+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:03:27.079692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"36'4","reported_seq":15,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:40.895403+0000","last_change":"2023-11-06T02:09:39.522170+0000","last_active":"2023-11-06T02:09:40.895403+0000","last_peered":"2023-11-06T02:09:40.895403+0000","last_clean":"2023-11-06T02:09:40.895403+0000","last_became_active":"2023-11-06T02:09:39.521622+0000","last_became_peered":"2023-11-06T02:09:39.521622+0000","last_unstale":"2023-11-06T02:09:40.895403+0000","last_undegraded":"2023-11-06T02:09:40.895403+0000","last_fullsized":"2023-11-06T02:09:40.895403+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:45:00.010604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.667549+0000","last_change":"2023-11-06T02:09:39.949469+0000","last_active":"2023-11-06T02:09:42.667549+0000","last_peered":"2023-11-06T02:09:42.667549+0000","last_clean":"2023-11-06T02:09:42.667549+0000","last_became_active":"2023-11-06T02:09:39.949119+0000","last_became_peered":"2023-11-06T02:09:39.949119+0000","last_unstale":"2023-11-06T02:09:42.667549+0000","last_undegraded":"2023-11-06T02:09:42.667549+0000","last_fullsized":"2023-11-06T02:09:42.667549+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:20:03.661315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"36'1","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:41.493536+0000","last_change":"2023-11-06T02:09:39.530442+0000","last_active":"2023-11-06T02:09:41.493536+0000","last_peered":"2023-11-06T02:09:41.493536+0000","last_clean":"2023-11-06T02:09:41.493536+0000","last_became_active":"2023-11-06T02:09:39.530124+0000","last_became_peered":"2023-11-06T02:09:39.530124+0000","last_unstale":"2023-11-06T02:09:41.493536+0000","last_undegraded":"2023-11-06T02:09:41.493536+0000","last_fullsized":"2023-11-06T02:09:41.493536+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:18:06.855960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.445640+0000","last_change":"2023-11-06T02:09:39.923492+0000","last_active":"2023-11-06T02:09:42.445640+0000","last_peered":"2023-11-06T02:09:42.445640+0000","last_clean":"2023-11-06T02:09:42.445640+0000","last_became_active":"2023-11-06T02:09:39.922628+0000","last_became_peered":"2023-11-06T02:09:39.922628+0000","last_unstale":"2023-11-06T02:09:42.445640+0000","last_undegraded":"2023-11-06T02:09:42.445640+0000","last_fullsized":"2023-11-06T02:09:42.445640+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:18:48.057417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"36'2","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.667680+0000","last_change":"2023-11-06T02:09:39.516455+0000","last_active":"2023-11-06T02:09:42.667680+0000","last_peered":"2023-11-06T02:09:42.667680+0000","last_clean":"2023-11-06T02:09:42.667680+0000","last_became_active":"2023-11-06T02:09:39.516057+0000","last_became_peered":"2023-11-06T02:09:39.516057+0000","last_unstale":"2023-11-06T02:09:42.667680+0000","last_undegraded":"2023-11-06T02:09:42.667680+0000","last_fullsized":"2023-11-06T02:09:42.667680+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:02:37.444340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.666415+0000","last_change":"2023-11-06T02:09:39.920710+0000","last_active":"2023-11-06T02:09:42.666415+0000","last_peered":"2023-11-06T02:09:42.666415+0000","last_clean":"2023-11-06T02:09:42.666415+0000","last_became_active":"2023-11-06T02:09:39.920415+0000","last_became_peered":"2023-11-06T02:09:39.920415+0000","last_unstale":"2023-11-06T02:09:42.666415+0000","last_undegraded":"2023-11-06T02:09:42.666415+0000","last_fullsized":"2023-11-06T02:09:42.666415+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:16:12.206417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"36'2","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.445520+0000","last_change":"2023-11-06T02:09:39.523237+0000","last_active":"2023-11-06T02:09:42.445520+0000","last_peered":"2023-11-06T02:09:42.445520+0000","last_clean":"2023-11-06T02:09:42.445520+0000","last_became_active":"2023-11-06T02:09:39.522434+0000","last_became_peered":"2023-11-06T02:09:39.522434+0000","last_unstale":"2023-11-06T02:09:42.445520+0000","last_undegraded":"2023-11-06T02:09:42.445520+0000","last_fullsized":"2023-11-06T02:09:42.445520+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:59:19.625827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:41.492608+0000","last_change":"2023-11-06T02:09:39.895217+0000","last_active":"2023-11-06T02:09:41.492608+0000","last_peered":"2023-11-06T02:09:41.492608+0000","last_clean":"2023-11-06T02:09:41.492608+0000","last_became_active":"2023-11-06T02:09:39.894437+0000","last_became_peered":"2023-11-06T02:09:39.894437+0000","last_unstale":"2023-11-06T02:09:41.492608+0000","last_undegraded":"2023-11-06T02:09:41.492608+0000","last_fullsized":"2023-11-06T02:09:41.492608+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:16:12.206417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"36'1","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.616450+0000","last_change":"2023-11-06T02:09:39.530213+0000","last_active":"2023-11-06T02:09:42.616450+0000","last_peered":"2023-11-06T02:09:42.616450+0000","last_clean":"2023-11-06T02:09:42.616450+0000","last_became_active":"2023-11-06T02:09:39.529428+0000","last_became_peered":"2023-11-06T02:09:39.529428+0000","last_unstale":"2023-11-06T02:09:42.616450+0000","last_undegraded":"2023-11-06T02:09:42.616450+0000","last_fullsized":"2023-11-06T02:09:42.616450+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:17:05.175089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.616695+0000","last_change":"2023-11-06T02:09:39.944083+0000","last_active":"2023-11-06T02:09:42.616695+0000","last_peered":"2023-11-06T02:09:42.616695+0000","last_clean":"2023-11-06T02:09:42.616695+0000","last_became_active":"2023-11-06T02:09:39.943344+0000","last_became_peered":"2023-11-06T02:09:39.943344+0000","last_unstale":"2023-11-06T02:09:42.616695+0000","last_undegraded":"2023-11-06T02:09:42.616695+0000","last_fullsized":"2023-11-06T02:09:42.616695+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:30:28.443414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"36'1","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:40.893347+0000","last_change":"2023-11-06T02:09:39.529686+0000","last_active":"2023-11-06T02:09:40.893347+0000","last_peered":"2023-11-06T02:09:40.893347+0000","last_clean":"2023-11-06T02:09:40.893347+0000","last_became_active":"2023-11-06T02:09:39.528866+0000","last_became_peered":"2023-11-06T02:09:39.528866+0000","last_unstale":"2023-11-06T02:09:40.893347+0000","last_undegraded":"2023-11-06T02:09:40.893347+0000","last_fullsized":"2023-11-06T02:09:40.893347+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:41:07.012516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.445266+0000","last_change":"2023-11-06T02:09:39.945345+0000","last_active":"2023-11-06T02:09:42.445266+0000","last_peered":"2023-11-06T02:09:42.445266+0000","last_clean":"2023-11-06T02:09:42.445266+0000","last_became_active":"2023-11-06T02:09:39.944852+0000","last_became_peered":"2023-11-06T02:09:39.944852+0000","last_unstale":"2023-11-06T02:09:42.445266+0000","last_undegraded":"2023-11-06T02:09:42.445266+0000","last_fullsized":"2023-11-06T02:09:42.445266+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:46:16.536437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.666771+0000","last_change":"2023-11-06T02:09:39.534802+0000","last_active":"2023-11-06T02:09:42.666771+0000","last_peered":"2023-11-06T02:09:42.666771+0000","last_clean":"2023-11-06T02:09:42.666771+0000","last_became_active":"2023-11-06T02:09:39.534382+0000","last_became_peered":"2023-11-06T02:09:39.534382+0000","last_unstale":"2023-11-06T02:09:42.666771+0000","last_undegraded":"2023-11-06T02:09:42.666771+0000","last_fullsized":"2023-11-06T02:09:42.666771+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:05:38.381689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.665954+0000","last_change":"2023-11-06T02:09:39.932625+0000","last_active":"2023-11-06T02:09:42.665954+0000","last_peered":"2023-11-06T02:09:42.665954+0000","last_clean":"2023-11-06T02:09:42.665954+0000","last_became_active":"2023-11-06T02:09:39.932419+0000","last_became_peered":"2023-11-06T02:09:39.932419+0000","last_unstale":"2023-11-06T02:09:42.665954+0000","last_undegraded":"2023-11-06T02:09:42.665954+0000","last_fullsized":"2023-11-06T02:09:42.665954+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:01:17.009165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.444854+0000","last_change":"2023-11-06T02:09:39.520842+0000","last_active":"2023-11-06T02:09:42.444854+0000","last_peered":"2023-11-06T02:09:42.444854+0000","last_clean":"2023-11-06T02:09:42.444854+0000","last_became_active":"2023-11-06T02:09:39.520372+0000","last_became_peered":"2023-11-06T02:09:39.520372+0000","last_unstale":"2023-11-06T02:09:42.444854+0000","last_undegraded":"2023-11-06T02:09:42.444854+0000","last_fullsized":"2023-11-06T02:09:42.444854+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:39:48.494686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:40.889764+0000","last_change":"2023-11-06T02:09:39.895957+0000","last_active":"2023-11-06T02:09:40.889764+0000","last_peered":"2023-11-06T02:09:40.889764+0000","last_clean":"2023-11-06T02:09:40.889764+0000","last_became_active":"2023-11-06T02:09:39.895360+0000","last_became_peered":"2023-11-06T02:09:39.895360+0000","last_unstale":"2023-11-06T02:09:40.889764+0000","last_undegraded":"2023-11-06T02:09:40.889764+0000","last_fullsized":"2023-11-06T02:09:40.889764+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:45:01.120311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.666885+0000","last_change":"2023-11-06T02:09:39.507720+0000","last_active":"2023-11-06T02:09:42.666885+0000","last_peered":"2023-11-06T02:09:42.666885+0000","last_clean":"2023-11-06T02:09:42.666885+0000","last_became_active":"2023-11-06T02:09:39.507153+0000","last_became_peered":"2023-11-06T02:09:39.507153+0000","last_unstale":"2023-11-06T02:09:42.666885+0000","last_undegraded":"2023-11-06T02:09:42.666885+0000","last_fullsized":"2023-11-06T02:09:42.666885+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:06:37.574203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:40.894035+0000","last_change":"2023-11-06T02:09:39.934807+0000","last_active":"2023-11-06T02:09:40.894035+0000","last_peered":"2023-11-06T02:09:40.894035+0000","last_clean":"2023-11-06T02:09:40.894035+0000","last_became_active":"2023-11-06T02:09:39.934323+0000","last_became_peered":"2023-11-06T02:09:39.934323+0000","last_unstale":"2023-11-06T02:09:40.894035+0000","last_undegraded":"2023-11-06T02:09:40.894035+0000","last_fullsized":"2023-11-06T02:09:40.894035+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:16:12.206417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.617153+0000","last_change":"2023-11-06T02:09:39.536553+0000","last_active":"2023-11-06T02:09:42.617153+0000","last_peered":"2023-11-06T02:09:42.617153+0000","last_clean":"2023-11-06T02:09:42.617153+0000","last_became_active":"2023-11-06T02:09:39.533822+0000","last_became_peered":"2023-11-06T02:09:39.533822+0000","last_unstale":"2023-11-06T02:09:42.617153+0000","last_undegraded":"2023-11-06T02:09:42.617153+0000","last_fullsized":"2023-11-06T02:09:42.617153+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:45:00.108325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:41.493505+0000","last_change":"2023-11-06T02:09:39.925758+0000","last_active":"2023-11-06T02:09:41.493505+0000","last_peered":"2023-11-06T02:09:41.493505+0000","last_clean":"2023-11-06T02:09:41.493505+0000","last_became_active":"2023-11-06T02:09:39.925443+0000","last_became_peered":"2023-11-06T02:09:39.925443+0000","last_unstale":"2023-11-06T02:09:41.493505+0000","last_undegraded":"2023-11-06T02:09:41.493505+0000","last_fullsized":"2023-11-06T02:09:41.493505+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:49:03.209827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"36'3","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.617311+0000","last_change":"2023-11-06T02:09:39.527840+0000","last_active":"2023-11-06T02:09:42.617311+0000","last_peered":"2023-11-06T02:09:42.617311+0000","last_clean":"2023-11-06T02:09:42.617311+0000","last_became_active":"2023-11-06T02:09:39.527192+0000","last_became_peered":"2023-11-06T02:09:39.527192+0000","last_unstale":"2023-11-06T02:09:42.617311+0000","last_undegraded":"2023-11-06T02:09:42.617311+0000","last_fullsized":"2023-11-06T02:09:42.617311+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:06:37.574203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.445359+0000","last_change":"2023-11-06T02:09:39.897369+0000","last_active":"2023-11-06T02:09:42.445359+0000","last_peered":"2023-11-06T02:09:42.445359+0000","last_clean":"2023-11-06T02:09:42.445359+0000","last_became_active":"2023-11-06T02:09:39.896531+0000","last_became_peered":"2023-11-06T02:09:39.896531+0000","last_unstale":"2023-11-06T02:09:42.445359+0000","last_undegraded":"2023-11-06T02:09:42.445359+0000","last_fullsized":"2023-11-06T02:09:42.445359+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:03:27.079692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"36'6","reported_seq":17,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.666444+0000","last_change":"2023-11-06T02:09:39.530827+0000","last_active":"2023-11-06T02:09:42.666444+0000","last_peered":"2023-11-06T02:09:42.666444+0000","last_clean":"2023-11-06T02:09:42.666444+0000","last_became_active":"2023-11-06T02:09:39.530374+0000","last_became_peered":"2023-11-06T02:09:39.530374+0000","last_unstale":"2023-11-06T02:09:42.666444+0000","last_undegraded":"2023-11-06T02:09:42.666444+0000","last_fullsized":"2023-11-06T02:09:42.666444+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:21:41.865490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:41.492786+0000","last_change":"2023-11-06T02:09:39.944160+0000","last_active":"2023-11-06T02:09:41.492786+0000","last_peered":"2023-11-06T02:09:41.492786+0000","last_clean":"2023-11-06T02:09:41.492786+0000","last_became_active":"2023-11-06T02:09:39.943858+0000","last_became_peered":"2023-11-06T02:09:39.943858+0000","last_unstale":"2023-11-06T02:09:41.492786+0000","last_undegraded":"2023-11-06T02:09:41.492786+0000","last_fullsized":"2023-11-06T02:09:41.492786+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:18:26.908517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"36'2","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.666774+0000","last_change":"2023-11-06T02:09:39.514824+0000","last_active":"2023-11-06T02:09:42.666774+0000","last_peered":"2023-11-06T02:09:42.666774+0000","last_clean":"2023-11-06T02:09:42.666774+0000","last_became_active":"2023-11-06T02:09:39.514231+0000","last_became_peered":"2023-11-06T02:09:39.514231+0000","last_unstale":"2023-11-06T02:09:42.666774+0000","last_undegraded":"2023-11-06T02:09:42.666774+0000","last_fullsized":"2023-11-06T02:09:42.666774+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:23:30.732017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.666097+0000","last_change":"2023-11-06T02:09:39.895501+0000","last_active":"2023-11-06T02:09:42.666097+0000","last_peered":"2023-11-06T02:09:42.666097+0000","last_clean":"2023-11-06T02:09:42.666097+0000","last_became_active":"2023-11-06T02:09:39.895254+0000","last_became_peered":"2023-11-06T02:09:39.895254+0000","last_unstale":"2023-11-06T02:09:42.666097+0000","last_undegraded":"2023-11-06T02:09:42.666097+0000","last_fullsized":"2023-11-06T02:09:42.666097+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:28:05.481974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.666856+0000","last_change":"2023-11-06T02:09:39.505888+0000","last_active":"2023-11-06T02:09:42.666856+0000","last_peered":"2023-11-06T02:09:42.666856+0000","last_clean":"2023-11-06T02:09:42.666856+0000","last_became_active":"2023-11-06T02:09:39.505514+0000","last_became_peered":"2023-11-06T02:09:39.505514+0000","last_unstale":"2023-11-06T02:09:42.666856+0000","last_undegraded":"2023-11-06T02:09:42.666856+0000","last_fullsized":"2023-11-06T02:09:42.666856+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:06:29.233287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:40.893108+0000","last_change":"2023-11-06T02:09:39.926232+0000","last_active":"2023-11-06T02:09:40.893108+0000","last_peered":"2023-11-06T02:09:40.893108+0000","last_clean":"2023-11-06T02:09:40.893108+0000","last_became_active":"2023-11-06T02:09:39.924958+0000","last_became_peered":"2023-11-06T02:09:39.924958+0000","last_unstale":"2023-11-06T02:09:40.893108+0000","last_undegraded":"2023-11-06T02:09:40.893108+0000","last_fullsized":"2023-11-06T02:09:40.893108+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:18:26.908517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:40.894773+0000","last_change":"2023-11-06T02:09:39.525068+0000","last_active":"2023-11-06T02:09:40.894773+0000","last_peered":"2023-11-06T02:09:40.894773+0000","last_clean":"2023-11-06T02:09:40.894773+0000","last_became_active":"2023-11-06T02:09:39.524560+0000","last_became_peered":"2023-11-06T02:09:39.524560+0000","last_unstale":"2023-11-06T02:09:40.894773+0000","last_undegraded":"2023-11-06T02:09:40.894773+0000","last_fullsized":"2023-11-06T02:09:40.894773+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:03:26.558497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.666243+0000","last_change":"2023-11-06T02:09:39.895998+0000","last_active":"2023-11-06T02:09:42.666243+0000","last_peered":"2023-11-06T02:09:42.666243+0000","last_clean":"2023-11-06T02:09:42.666243+0000","last_became_active":"2023-11-06T02:09:39.895762+0000","last_became_peered":"2023-11-06T02:09:39.895762+0000","last_unstale":"2023-11-06T02:09:42.666243+0000","last_undegraded":"2023-11-06T02:09:42.666243+0000","last_fullsized":"2023-11-06T02:09:42.666243+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:14:47.981503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"36'8","reported_seq":19,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:40.892504+0000","last_change":"2023-11-06T02:09:39.530857+0000","last_active":"2023-11-06T02:09:40.892504+0000","last_peered":"2023-11-06T02:09:40.892504+0000","last_clean":"2023-11-06T02:09:40.892504+0000","last_became_active":"2023-11-06T02:09:39.530243+0000","last_became_peered":"2023-11-06T02:09:39.530243+0000","last_unstale":"2023-11-06T02:09:40.892504+0000","last_undegraded":"2023-11-06T02:09:40.892504+0000","last_fullsized":"2023-11-06T02:09:40.892504+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:10:49.280878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.666195+0000","last_change":"2023-11-06T02:09:39.916393+0000","last_active":"2023-11-06T02:09:42.666195+0000","last_peered":"2023-11-06T02:09:42.666195+0000","last_clean":"2023-11-06T02:09:42.666195+0000","last_became_active":"2023-11-06T02:09:39.916048+0000","last_became_peered":"2023-11-06T02:09:39.916048+0000","last_unstale":"2023-11-06T02:09:42.666195+0000","last_undegraded":"2023-11-06T02:09:42.666195+0000","last_fullsized":"2023-11-06T02:09:42.666195+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:03:27.079692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"36'1","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.617139+0000","last_change":"2023-11-06T02:09:39.528406+0000","last_active":"2023-11-06T02:09:42.617139+0000","last_peered":"2023-11-06T02:09:42.617139+0000","last_clean":"2023-11-06T02:09:42.617139+0000","last_became_active":"2023-11-06T02:09:39.527718+0000","last_became_peered":"2023-11-06T02:09:39.527718+0000","last_unstale":"2023-11-06T02:09:42.617139+0000","last_undegraded":"2023-11-06T02:09:42.617139+0000","last_fullsized":"2023-11-06T02:09:42.617139+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:23:30.732017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.617319+0000","last_change":"2023-11-06T02:09:39.951145+0000","last_active":"2023-11-06T02:09:42.617319+0000","last_peered":"2023-11-06T02:09:42.617319+0000","last_clean":"2023-11-06T02:09:42.617319+0000","last_became_active":"2023-11-06T02:09:39.948408+0000","last_became_peered":"2023-11-06T02:09:39.948408+0000","last_unstale":"2023-11-06T02:09:42.617319+0000","last_undegraded":"2023-11-06T02:09:42.617319+0000","last_fullsized":"2023-11-06T02:09:42.617319+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:46:16.536437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"36'2","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.445803+0000","last_change":"2023-11-06T02:09:39.537804+0000","last_active":"2023-11-06T02:09:42.445803+0000","last_peered":"2023-11-06T02:09:42.445803+0000","last_clean":"2023-11-06T02:09:42.445803+0000","last_became_active":"2023-11-06T02:09:39.537228+0000","last_became_peered":"2023-11-06T02:09:39.537228+0000","last_unstale":"2023-11-06T02:09:42.445803+0000","last_undegraded":"2023-11-06T02:09:42.445803+0000","last_fullsized":"2023-11-06T02:09:42.445803+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:49:02.197841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.445634+0000","last_change":"2023-11-06T02:09:39.899930+0000","last_active":"2023-11-06T02:09:42.445634+0000","last_peered":"2023-11-06T02:09:42.445634+0000","last_clean":"2023-11-06T02:09:42.445634+0000","last_became_active":"2023-11-06T02:09:39.898977+0000","last_became_peered":"2023-11-06T02:09:39.898977+0000","last_unstale":"2023-11-06T02:09:42.445634+0000","last_undegraded":"2023-11-06T02:09:42.445634+0000","last_fullsized":"2023-11-06T02:09:42.445634+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:28:05.481974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"36'1","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:40.890057+0000","last_change":"2023-11-06T02:09:39.531338+0000","last_active":"2023-11-06T02:09:40.890057+0000","last_peered":"2023-11-06T02:09:40.890057+0000","last_clean":"2023-11-06T02:09:40.890057+0000","last_became_active":"2023-11-06T02:09:39.530227+0000","last_became_peered":"2023-11-06T02:09:39.530227+0000","last_unstale":"2023-11-06T02:09:40.890057+0000","last_undegraded":"2023-11-06T02:09:40.890057+0000","last_fullsized":"2023-11-06T02:09:40.890057+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:16:11.505589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:40.889974+0000","last_change":"2023-11-06T02:09:39.891558+0000","last_active":"2023-11-06T02:09:40.889974+0000","last_peered":"2023-11-06T02:09:40.889974+0000","last_clean":"2023-11-06T02:09:40.889974+0000","last_became_active":"2023-11-06T02:09:39.891176+0000","last_became_peered":"2023-11-06T02:09:39.891176+0000","last_unstale":"2023-11-06T02:09:40.889974+0000","last_undegraded":"2023-11-06T02:09:40.889974+0000","last_fullsized":"2023-11-06T02:09:40.889974+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:06:30.245273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.666918+0000","last_change":"2023-11-06T02:09:39.516262+0000","last_active":"2023-11-06T02:09:42.666918+0000","last_peered":"2023-11-06T02:09:42.666918+0000","last_clean":"2023-11-06T02:09:42.666918+0000","last_became_active":"2023-11-06T02:09:39.515910+0000","last_became_peered":"2023-11-06T02:09:39.515910+0000","last_unstale":"2023-11-06T02:09:42.666918+0000","last_undegraded":"2023-11-06T02:09:42.666918+0000","last_fullsized":"2023-11-06T02:09:42.666918+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:41:07.012516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:40.893013+0000","last_change":"2023-11-06T02:09:39.925305+0000","last_active":"2023-11-06T02:09:40.893013+0000","last_peered":"2023-11-06T02:09:40.893013+0000","last_clean":"2023-11-06T02:09:40.893013+0000","last_became_active":"2023-11-06T02:09:39.924545+0000","last_became_peered":"2023-11-06T02:09:39.924545+0000","last_unstale":"2023-11-06T02:09:40.893013+0000","last_undegraded":"2023-11-06T02:09:40.893013+0000","last_fullsized":"2023-11-06T02:09:40.893013+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:24:44.156346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"36'2","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:40.889975+0000","last_change":"2023-11-06T02:09:39.521591+0000","last_active":"2023-11-06T02:09:40.889975+0000","last_peered":"2023-11-06T02:09:40.889975+0000","last_clean":"2023-11-06T02:09:40.889975+0000","last_became_active":"2023-11-06T02:09:39.521127+0000","last_became_peered":"2023-11-06T02:09:39.521127+0000","last_unstale":"2023-11-06T02:09:40.889975+0000","last_undegraded":"2023-11-06T02:09:40.889975+0000","last_fullsized":"2023-11-06T02:09:40.889975+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:03:26.558497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:41.492794+0000","last_change":"2023-11-06T02:09:39.927086+0000","last_active":"2023-11-06T02:09:41.492794+0000","last_peered":"2023-11-06T02:09:41.492794+0000","last_clean":"2023-11-06T02:09:41.492794+0000","last_became_active":"2023-11-06T02:09:39.926791+0000","last_became_peered":"2023-11-06T02:09:39.926791+0000","last_unstale":"2023-11-06T02:09:41.492794+0000","last_undegraded":"2023-11-06T02:09:41.492794+0000","last_fullsized":"2023-11-06T02:09:41.492794+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:21:42.877476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:40.889754+0000","last_change":"2023-11-06T02:09:39.528293+0000","last_active":"2023-11-06T02:09:40.889754+0000","last_peered":"2023-11-06T02:09:40.889754+0000","last_clean":"2023-11-06T02:09:40.889754+0000","last_became_active":"2023-11-06T02:09:39.527957+0000","last_became_peered":"2023-11-06T02:09:39.527957+0000","last_unstale":"2023-11-06T02:09:40.889754+0000","last_undegraded":"2023-11-06T02:09:40.889754+0000","last_fullsized":"2023-11-06T02:09:40.889754+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:48:21.380950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.616240+0000","last_change":"2023-11-06T02:09:39.945627+0000","last_active":"2023-11-06T02:09:42.616240+0000","last_peered":"2023-11-06T02:09:42.616240+0000","last_clean":"2023-11-06T02:09:42.616240+0000","last_became_active":"2023-11-06T02:09:39.943242+0000","last_became_peered":"2023-11-06T02:09:39.943242+0000","last_unstale":"2023-11-06T02:09:42.616240+0000","last_undegraded":"2023-11-06T02:09:42.616240+0000","last_fullsized":"2023-11-06T02:09:42.616240+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:37:29.456658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:40.893963+0000","last_change":"2023-11-06T02:09:39.528907+0000","last_active":"2023-11-06T02:09:40.893963+0000","last_peered":"2023-11-06T02:09:40.893963+0000","last_clean":"2023-11-06T02:09:40.893963+0000","last_became_active":"2023-11-06T02:09:39.528318+0000","last_became_peered":"2023-11-06T02:09:39.528318+0000","last_unstale":"2023-11-06T02:09:40.893963+0000","last_undegraded":"2023-11-06T02:09:40.893963+0000","last_fullsized":"2023-11-06T02:09:40.893963+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:06:29.233287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.666520+0000","last_change":"2023-11-06T02:09:39.925619+0000","last_active":"2023-11-06T02:09:42.666520+0000","last_peered":"2023-11-06T02:09:42.666520+0000","last_clean":"2023-11-06T02:09:42.666520+0000","last_became_active":"2023-11-06T02:09:39.925306+0000","last_became_peered":"2023-11-06T02:09:39.925306+0000","last_unstale":"2023-11-06T02:09:42.666520+0000","last_undegraded":"2023-11-06T02:09:42.666520+0000","last_fullsized":"2023-11-06T02:09:42.666520+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:04:26.233460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:40.889523+0000","last_change":"2023-11-06T02:09:39.516004+0000","last_active":"2023-11-06T02:09:40.889523+0000","last_peered":"2023-11-06T02:09:40.889523+0000","last_clean":"2023-11-06T02:09:40.889523+0000","last_became_active":"2023-11-06T02:09:39.514485+0000","last_became_peered":"2023-11-06T02:09:39.514485+0000","last_unstale":"2023-11-06T02:09:40.889523+0000","last_undegraded":"2023-11-06T02:09:40.889523+0000","last_fullsized":"2023-11-06T02:09:40.889523+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:10:37.611351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.616984+0000","last_change":"2023-11-06T02:09:39.948726+0000","last_active":"2023-11-06T02:09:42.616984+0000","last_peered":"2023-11-06T02:09:42.616984+0000","last_clean":"2023-11-06T02:09:42.616984+0000","last_became_active":"2023-11-06T02:09:39.948138+0000","last_became_peered":"2023-11-06T02:09:39.948138+0000","last_unstale":"2023-11-06T02:09:42.616984+0000","last_undegraded":"2023-11-06T02:09:42.616984+0000","last_fullsized":"2023-11-06T02:09:42.616984+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:24:44.156346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:40.895071+0000","last_change":"2023-11-06T02:09:39.528889+0000","last_active":"2023-11-06T02:09:40.895071+0000","last_peered":"2023-11-06T02:09:40.895071+0000","last_clean":"2023-11-06T02:09:40.895071+0000","last_became_active":"2023-11-06T02:09:39.528371+0000","last_became_peered":"2023-11-06T02:09:39.528371+0000","last_unstale":"2023-11-06T02:09:40.895071+0000","last_undegraded":"2023-11-06T02:09:40.895071+0000","last_fullsized":"2023-11-06T02:09:40.895071+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:02:37.444340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":179,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-06T02:09:39.909075+0000","last_change":"2023-11-06T02:08:20.523427+0000","last_active":"2023-11-06T02:09:39.909075+0000","last_peered":"2023-11-06T02:09:39.909075+0000","last_clean":"2023-11-06T02:09:39.909075+0000","last_became_active":"2023-11-06T02:08:20.521089+0000","last_became_peered":"2023-11-06T02:08:20.521089+0000","last_unstale":"2023-11-06T02:09:39.909075+0000","last_undegraded":"2023-11-06T02:09:39.909075+0000","last_fullsized":"2023-11-06T02:09:39.909075+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:19.252348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:19.252348+0000","last_clean_scrub_stamp":"2023-11-06T02:08:19.252348+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:30:15.566397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":26,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-06T02:09:39.933443+0000","last_change":"2023-11-06T02:08:26.059502+0000","last_active":"2023-11-06T02:09:39.933443+0000","last_peered":"2023-11-06T02:09:39.933443+0000","last_clean":"2023-11-06T02:09:39.933443+0000","last_became_active":"2023-11-06T02:08:26.058612+0000","last_became_peered":"2023-11-06T02:08:26.058612+0000","last_unstale":"2023-11-06T02:09:39.933443+0000","last_undegraded":"2023-11-06T02:09:39.933443+0000","last_fullsized":"2023-11-06T02:09:39.933443+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:10.626021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.666793+0000","last_change":"2023-11-06T02:09:39.945906+0000","last_active":"2023-11-06T02:09:42.666793+0000","last_peered":"2023-11-06T02:09:42.666793+0000","last_clean":"2023-11-06T02:09:42.666793+0000","last_became_active":"2023-11-06T02:09:39.945686+0000","last_became_peered":"2023-11-06T02:09:39.945686+0000","last_unstale":"2023-11-06T02:09:42.666793+0000","last_undegraded":"2023-11-06T02:09:42.666793+0000","last_fullsized":"2023-11-06T02:09:42.666793+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:36:18.647776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"36'2","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.617264+0000","last_change":"2023-11-06T02:09:39.518660+0000","last_active":"2023-11-06T02:09:42.617264+0000","last_peered":"2023-11-06T02:09:42.617264+0000","last_clean":"2023-11-06T02:09:42.617264+0000","last_became_active":"2023-11-06T02:09:39.517929+0000","last_became_peered":"2023-11-06T02:09:39.517929+0000","last_unstale":"2023-11-06T02:09:42.617264+0000","last_undegraded":"2023-11-06T02:09:42.617264+0000","last_fullsized":"2023-11-06T02:09:42.617264+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:29:59.553436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":26,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-06T02:09:39.908271+0000","last_change":"2023-11-06T02:08:26.060406+0000","last_active":"2023-11-06T02:09:39.908271+0000","last_peered":"2023-11-06T02:09:39.908271+0000","last_clean":"2023-11-06T02:09:39.908271+0000","last_became_active":"2023-11-06T02:08:26.059262+0000","last_became_peered":"2023-11-06T02:08:26.059262+0000","last_unstale":"2023-11-06T02:09:39.908271+0000","last_undegraded":"2023-11-06T02:09:39.908271+0000","last_fullsized":"2023-11-06T02:09:39.908271+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:10.626021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:40.894343+0000","last_change":"2023-11-06T02:09:39.935264+0000","last_active":"2023-11-06T02:09:40.894343+0000","last_peered":"2023-11-06T02:09:40.894343+0000","last_clean":"2023-11-06T02:09:40.894343+0000","last_became_active":"2023-11-06T02:09:39.934501+0000","last_became_peered":"2023-11-06T02:09:39.934501+0000","last_unstale":"2023-11-06T02:09:40.894343+0000","last_undegraded":"2023-11-06T02:09:40.894343+0000","last_fullsized":"2023-11-06T02:09:40.894343+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:49:03.209827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"37'1","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.614843+0000","last_change":"2023-11-06T02:09:39.538233+0000","last_active":"2023-11-06T02:09:42.614843+0000","last_peered":"2023-11-06T02:09:42.614843+0000","last_clean":"2023-11-06T02:09:42.614843+0000","last_became_active":"2023-11-06T02:09:39.537707+0000","last_became_peered":"2023-11-06T02:09:39.537707+0000","last_unstale":"2023-11-06T02:09:42.614843+0000","last_undegraded":"2023-11-06T02:09:42.614843+0000","last_fullsized":"2023-11-06T02:09:42.614843+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:46:10.180159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":26,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-06T02:09:39.880659+0000","last_change":"2023-11-06T02:08:26.559477+0000","last_active":"2023-11-06T02:09:39.880659+0000","last_peered":"2023-11-06T02:09:39.880659+0000","last_clean":"2023-11-06T02:09:39.880659+0000","last_became_active":"2023-11-06T02:08:26.559202+0000","last_became_peered":"2023-11-06T02:08:26.559202+0000","last_unstale":"2023-11-06T02:09:39.880659+0000","last_undegraded":"2023-11-06T02:09:39.880659+0000","last_fullsized":"2023-11-06T02:09:39.880659+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:55:04.552404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.666254+0000","last_change":"2023-11-06T02:09:39.904655+0000","last_active":"2023-11-06T02:09:42.666254+0000","last_peered":"2023-11-06T02:09:42.666254+0000","last_clean":"2023-11-06T02:09:42.666254+0000","last_became_active":"2023-11-06T02:09:39.904332+0000","last_became_peered":"2023-11-06T02:09:39.904332+0000","last_unstale":"2023-11-06T02:09:42.666254+0000","last_undegraded":"2023-11-06T02:09:42.666254+0000","last_fullsized":"2023-11-06T02:09:42.666254+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:30:28.443414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:41.493308+0000","last_change":"2023-11-06T02:09:39.511832+0000","last_active":"2023-11-06T02:09:41.493308+0000","last_peered":"2023-11-06T02:09:41.493308+0000","last_clean":"2023-11-06T02:09:41.493308+0000","last_became_active":"2023-11-06T02:09:39.511211+0000","last_became_peered":"2023-11-06T02:09:39.511211+0000","last_unstale":"2023-11-06T02:09:41.493308+0000","last_undegraded":"2023-11-06T02:09:41.493308+0000","last_fullsized":"2023-11-06T02:09:41.493308+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:10:49.280878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":26,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-06T02:09:39.932835+0000","last_change":"2023-11-06T02:08:26.059546+0000","last_active":"2023-11-06T02:09:39.932835+0000","last_peered":"2023-11-06T02:09:39.932835+0000","last_clean":"2023-11-06T02:09:39.932835+0000","last_became_active":"2023-11-06T02:08:26.058806+0000","last_became_peered":"2023-11-06T02:08:26.058806+0000","last_unstale":"2023-11-06T02:09:39.932835+0000","last_undegraded":"2023-11-06T02:09:39.932835+0000","last_fullsized":"2023-11-06T02:09:39.932835+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:23.468510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:41.492874+0000","last_change":"2023-11-06T02:09:39.891191+0000","last_active":"2023-11-06T02:09:41.492874+0000","last_peered":"2023-11-06T02:09:41.492874+0000","last_clean":"2023-11-06T02:09:41.492874+0000","last_became_active":"2023-11-06T02:09:39.890890+0000","last_became_peered":"2023-11-06T02:09:39.890890+0000","last_unstale":"2023-11-06T02:09:41.492874+0000","last_undegraded":"2023-11-06T02:09:41.492874+0000","last_fullsized":"2023-11-06T02:09:41.492874+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:36:18.647776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:41.493313+0000","last_change":"2023-11-06T02:09:39.524444+0000","last_active":"2023-11-06T02:09:41.493313+0000","last_peered":"2023-11-06T02:09:41.493313+0000","last_clean":"2023-11-06T02:09:41.493313+0000","last_became_active":"2023-11-06T02:09:39.523926+0000","last_became_peered":"2023-11-06T02:09:39.523926+0000","last_unstale":"2023-11-06T02:09:41.493313+0000","last_undegraded":"2023-11-06T02:09:41.493313+0000","last_fullsized":"2023-11-06T02:09:41.493313+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:19:53.336403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":25,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-06T02:09:39.885610+0000","last_change":"2023-11-06T02:08:26.436463+0000","last_active":"2023-11-06T02:09:39.885610+0000","last_peered":"2023-11-06T02:09:39.885610+0000","last_clean":"2023-11-06T02:09:39.885610+0000","last_became_active":"2023-11-06T02:08:26.436075+0000","last_became_peered":"2023-11-06T02:08:26.436075+0000","last_unstale":"2023-11-06T02:09:39.885610+0000","last_undegraded":"2023-11-06T02:09:39.885610+0000","last_fullsized":"2023-11-06T02:09:39.885610+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:23.468510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.666644+0000","last_change":"2023-11-06T02:09:39.913449+0000","last_active":"2023-11-06T02:09:42.666644+0000","last_peered":"2023-11-06T02:09:42.666644+0000","last_clean":"2023-11-06T02:09:42.666644+0000","last_became_active":"2023-11-06T02:09:39.913198+0000","last_became_peered":"2023-11-06T02:09:39.913198+0000","last_unstale":"2023-11-06T02:09:42.666644+0000","last_undegraded":"2023-11-06T02:09:42.666644+0000","last_fullsized":"2023-11-06T02:09:42.666644+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:21:42.877476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:40.893941+0000","last_change":"2023-11-06T02:09:39.512989+0000","last_active":"2023-11-06T02:09:40.893941+0000","last_peered":"2023-11-06T02:09:40.893941+0000","last_clean":"2023-11-06T02:09:40.893941+0000","last_became_active":"2023-11-06T02:09:39.512373+0000","last_became_peered":"2023-11-06T02:09:39.512373+0000","last_unstale":"2023-11-06T02:09:40.893941+0000","last_undegraded":"2023-11-06T02:09:40.893941+0000","last_fullsized":"2023-11-06T02:09:40.893941+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:54:33.107256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":26,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-06T02:09:39.916699+0000","last_change":"2023-11-06T02:08:26.436607+0000","last_active":"2023-11-06T02:09:39.916699+0000","last_peered":"2023-11-06T02:09:39.916699+0000","last_clean":"2023-11-06T02:09:39.916699+0000","last_became_active":"2023-11-06T02:08:26.435924+0000","last_became_peered":"2023-11-06T02:08:26.435924+0000","last_unstale":"2023-11-06T02:09:39.916699+0000","last_undegraded":"2023-11-06T02:09:39.916699+0000","last_fullsized":"2023-11-06T02:09:39.916699+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:23.468510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.445511+0000","last_change":"2023-11-06T02:09:39.901835+0000","last_active":"2023-11-06T02:09:42.445511+0000","last_peered":"2023-11-06T02:09:42.445511+0000","last_clean":"2023-11-06T02:09:42.445511+0000","last_became_active":"2023-11-06T02:09:39.901031+0000","last_became_peered":"2023-11-06T02:09:39.901031+0000","last_unstale":"2023-11-06T02:09:42.445511+0000","last_undegraded":"2023-11-06T02:09:42.445511+0000","last_fullsized":"2023-11-06T02:09:42.445511+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:02:27.832459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.444856+0000","last_change":"2023-11-06T02:09:39.523134+0000","last_active":"2023-11-06T02:09:42.444856+0000","last_peered":"2023-11-06T02:09:42.444856+0000","last_clean":"2023-11-06T02:09:42.444856+0000","last_became_active":"2023-11-06T02:09:39.522556+0000","last_became_peered":"2023-11-06T02:09:39.522556+0000","last_unstale":"2023-11-06T02:09:42.444856+0000","last_undegraded":"2023-11-06T02:09:42.444856+0000","last_fullsized":"2023-11-06T02:09:42.444856+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:38:42.071294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":25,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-06T02:09:39.885813+0000","last_change":"2023-11-06T02:08:26.057076+0000","last_active":"2023-11-06T02:09:39.885813+0000","last_peered":"2023-11-06T02:09:39.885813+0000","last_clean":"2023-11-06T02:09:39.885813+0000","last_became_active":"2023-11-06T02:08:26.056419+0000","last_became_peered":"2023-11-06T02:08:26.056419+0000","last_unstale":"2023-11-06T02:09:39.885813+0000","last_undegraded":"2023-11-06T02:09:39.885813+0000","last_fullsized":"2023-11-06T02:09:39.885813+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:10.626021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:40.894004+0000","last_change":"2023-11-06T02:09:39.939164+0000","last_active":"2023-11-06T02:09:40.894004+0000","last_peered":"2023-11-06T02:09:40.894004+0000","last_clean":"2023-11-06T02:09:40.894004+0000","last_became_active":"2023-11-06T02:09:39.938583+0000","last_became_peered":"2023-11-06T02:09:39.938583+0000","last_unstale":"2023-11-06T02:09:40.894004+0000","last_undegraded":"2023-11-06T02:09:40.894004+0000","last_fullsized":"2023-11-06T02:09:40.894004+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:39.469569+0000","last_clean_scrub_stamp":"2023-11-06T02:09:39.469569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:39:49.506672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T02:09:42.667623+0000","last_change":"2023-11-06T02:09:39.507309+0000","last_active":"2023-11-06T02:09:42.667623+0000","last_peered":"2023-11-06T02:09:42.667623+0000","last_clean":"2023-11-06T02:09:42.667623+0000","last_became_active":"2023-11-06T02:09:39.506868+0000","last_became_peered":"2023-11-06T02:09:39.506868+0000","last_unstale":"2023-11-06T02:09:42.667623+0000","last_undegraded":"2023-11-06T02:09:42.667623+0000","last_fullsized":"2023-11-06T02:09:42.667623+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:09:38.457583+0000","last_clean_scrub_stamp":"2023-11-06T02:09:38.457583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:29:59.553436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":28,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-06T02:09:39.890458+0000","last_change":"2023-11-06T02:08:26.058596+0000","last_active":"2023-11-06T02:09:39.890458+0000","last_peered":"2023-11-06T02:09:39.890458+0000","last_clean":"2023-11-06T02:09:39.890458+0000","last_became_active":"2023-11-06T02:08:26.057015+0000","last_became_peered":"2023-11-06T02:08:26.057015+0000","last_unstale":"2023-11-06T02:09:39.890458+0000","last_undegraded":"2023-11-06T02:09:39.890458+0000","last_fullsized":"2023-11-06T02:09:39.890458+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:55:04.552404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574868,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27960,"kb_used_data":1008,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343880,"statfs":{"total":96636764160,"available":96608133120,"internally_reserved":0,"allocated":1032192,"data_stored":767208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":1.03}]},{"osd":1,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":1.1399999999999999},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.879}]},{"osd":2,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":1.077},{"interface":"front","average":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.81799999999999995}]},{"osd":3,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":1.2849999999999999},{"interface":"front","average":{"1min":1.077,"5min":1.077,"15min":1.077},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":1.1779999999999999}]},{"osd":4,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":1.232},{"interface":"front","average":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.98299999999999998}]},{"osd":5,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.079,"15min":1.079},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":1.2589999999999999},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.7809999999999999,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":1.113}]},{"osd":6,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.202,"15min":1.202},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.6679999999999999,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":1.319},{"interface":"front","average":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.74,"5min":1.74,"15min":1.74},"last":1.363}]}]},{"osd":6,"up_from":13,"seq":55834574867,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27368,"kb_used_data":416,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":425984,"data_stored":175788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.621}]},{"osd":1,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.98599999999999999}]},{"osd":2,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.28,"5min":1.28,"15min":1.28},"last":0.90600000000000003}]},{"osd":3,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.71699999999999997}]},{"osd":4,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.474,"5min":1.474,"15min":1.474},"last":1.0900000000000001},{"interface":"front","average":{"1min":1.04,"5min":1.04,"15min":1.04},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.94299999999999995}]},{"osd":5,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.365,"5min":1.365,"15min":1.365},"last":0.96599999999999997},{"interface":"front","average":{"1min":1.115,"5min":1.115,"15min":1.115},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":1.1439999999999999}]},{"osd":7,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":1.117},{"interface":"front","average":{"1min":1.165,"5min":1.165,"15min":1.165},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":1.034}]}]},{"osd":5,"up_from":13,"seq":55834574868,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":408,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":417792,"data_stored":168665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.6579999999999999,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":0.64300000000000002}]},{"osd":1,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.84399999999999997}]},{"osd":2,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.377,"5min":1.377,"15min":1.377},"last":1.036}]},{"osd":3,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":1.0589999999999999},{"interface":"front","average":{"1min":1.081,"5min":1.081,"15min":1.081},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.7250000000000001,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":0.93300000000000005}]},{"osd":4,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.72899999999999998},{"interface":"front","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":0.90000000000000002}]},{"osd":6,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.68500000000000005},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.95199999999999996}]},{"osd":7,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.155,"5min":1.155,"15min":1.155},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.8089999999999999,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":1.014}]}]},{"osd":4,"up_from":13,"seq":55834574868,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27372,"kb_used_data":420,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344468,"statfs":{"total":96636764160,"available":96608735232,"internally_reserved":0,"allocated":430080,"data_stored":175810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":1.014},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.98499999999999999}]},{"osd":1,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":1.462},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.82599999999999996}]},{"osd":2,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":1.02,"15min":1.02},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.615,"5min":1.615,"15min":1.615},"last":1.319},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.93700000000000006}]},{"osd":3,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.52,"5min":1.52,"15min":1.52},"last":0.87},{"interface":"front","average":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.4970000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":1.194}]},{"osd":5,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"min":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":1.377,"5min":1.377,"15min":1.377},"last":0.72799999999999998},{"interface":"front","average":{"1min":1.056,"5min":1.056,"15min":1.056},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.7869999999999999,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":0.76200000000000001}]},{"osd":6,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":0.89100000000000001},{"interface":"front","average":{"1min":1.198,"5min":1.198,"15min":1.198},"min":{"1min":0.87,"5min":0.87,"15min":0.87},"max":{"1min":2.1190000000000002,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":1.1599999999999999}]},{"osd":7,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.7110000000000001,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":1.111},{"interface":"front","average":{"1min":1.24,"5min":1.24,"15min":1.24},"min":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":1.7410000000000001,"5min":1.7410000000000001,"15min":1.7410000000000001},"last":1.048}]}]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31476,"kb_used_data":428,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340364,"statfs":{"total":96636764160,"available":96604532736,"internally_reserved":0,"allocated":438272,"data_stored":176804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:09:19 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.96899999999999997}]},{"osd":1,"last update":"Mon Nov 6 02:09:19 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.79900000000000004}]},{"osd":2,"last update":"Mon Nov 6 02:09:19 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.88100000000000001}]},{"osd":4,"last update":"Mon Nov 6 02:09:19 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.405,"5min":1.405,"15min":1.405},"last":1.234},{"interface":"front","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.92800000000000005}]},{"osd":5,"last update":"Mon Nov 6 02:09:19 2023","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.276,"5min":1.276,"15min":1.276},"last":1.0369999999999999},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":2.21,"5min":2.21,"15min":2.21},"last":1.0629999999999999}]},{"osd":6,"last update":"Mon Nov 6 02:09:19 2023","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":1.026,"15min":1.026},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":1.1479999999999999},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":1.21}]},{"osd":7,"last update":"Mon Nov 6 02:09:19 2023","interfaces":[{"interface":"back","average":{"1min":1.133,"5min":1.133,"15min":1.133},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":2.2829999999999999,"5min":2.2829999999999999,"15min":2.2829999999999999},"last":1.1279999999999999},{"interface":"front","average":{"1min":1.107,"5min":1.107,"15min":1.107},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":1.3169999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574867,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31464,"kb_used_data":416,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340376,"statfs":{"total":96636764160,"available":96604545024,"internally_reserved":0,"allocated":425984,"data_stored":175276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.40799999999999997}]},{"osd":1,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.38500000000000001}]},{"osd":3,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.438}]},{"osd":4,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.873}]},{"osd":5,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.5369999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.746},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.7110000000000001,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.65000000000000002}]},{"osd":6,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.997,"5min":0.997,"15min":0.997},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.677,"5min":1.677,"15min":1.677},"last":0.66100000000000003}]},{"osd":7,"last update":"Mon Nov 6 02:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.633,"5min":1.633,"15min":1.633},"last":0.84399999999999997},{"interface":"front","average":{"1min":1.042,"5min":1.042,"15min":1.042},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":0.81399999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574868,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31464,"kb_used_data":416,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340376,"statfs":{"total":96636764160,"available":96604545024,"internally_reserved":0,"allocated":425984,"data_stored":175296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:09:19 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.90700000000000003}]},{"osd":2,"last update":"Mon Nov 6 02:09:19 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.76600000000000001}]},{"osd":3,"last update":"Mon Nov 6 02:09:19 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.8149999999999999,"5min":1.8149999999999999,"15min":1.8149999999999999},"last":0.79700000000000004}]},{"osd":4,"last update":"Mon Nov 6 02:09:19 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":1.1519999999999999}]},{"osd":5,"last update":"Mon Nov 6 02:09:19 2023","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.98799999999999999},{"interface":"front","average":{"1min":1.079,"5min":1.079,"15min":1.079},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.6759999999999999,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.86199999999999999}]},{"osd":6,"last update":"Mon Nov 6 02:09:19 2023","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.103,"15min":1.103},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.742,"5min":1.742,"15min":1.742},"last":1.052},{"interface":"front","average":{"1min":1.077,"5min":1.077,"15min":1.077},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.393,"5min":1.393,"15min":1.393},"last":1.079}]},{"osd":7,"last update":"Mon Nov 6 02:09:19 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.486,"5min":1.486,"15min":1.486},"last":1.02},{"interface":"front","average":{"1min":1.161,"5min":1.161,"15min":1.161},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.893,"5min":1.893,"15min":1.893},"last":1.115}]}]},{"osd":0,"up_from":13,"seq":55834574868,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32040,"kb_used_data":992,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339800,"statfs":{"total":96636764160,"available":96603955200,"internally_reserved":0,"allocated":1015808,"data_stored":764589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.71999999999999997}]},{"osd":2,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":1.0129999999999999},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":1.0589999999999999}]},{"osd":3,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":1.25}]},{"osd":4,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":1.163},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":1.3839999999999999}]},{"osd":5,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":1.3140000000000001},{"interface":"front","average":{"1min":0.872,"5min":0.872,"15min":0.872},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.448,"5min":1.448,"15min":1.448},"last":1.351}]},{"osd":6,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":1.581},{"interface":"front","average":{"1min":1,"5min":1,"15min":1},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":1.5529999999999999}]},{"osd":7,"last update":"Mon Nov 6 02:09:21 2023","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":1.421},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.556,"5min":1.556,"15min":1.556},"last":1.4950000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T02:09:49.054 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-06T02:09:49.054 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-06T02:09:49.229 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.228+0000 7fbbbea40640 1 Processor -- start 2023-11-06T02:09:49.229 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.228+0000 7fbbbea40640 1 -- start start 2023-11-06T02:09:49.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.232+0000 7fbbbea40640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fbbb81595a0 0x7fbbb815dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:49.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.232+0000 7fbbbea40640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbbb806d640 0x7fbbb80b0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:49.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.232+0000 7fbbbea40640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fbbb80b0760 0x7fbbb80c8e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:49.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.232+0000 7fbbbea40640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbb815e250 con 0x7fbbb81595a0 2023-11-06T02:09:49.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.232+0000 7fbbbea40640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbb806c5a0 con 0x7fbbb80b0760 2023-11-06T02:09:49.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.232+0000 7fbbbea40640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb806c880 con 0x7fbbb806d640 2023-11-06T02:09:49.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.232+0000 7fbbb7fff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fbbb81595a0 0x7fbbb815dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:49.232 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.232+0000 7fbbb7fff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fbbb81595a0 0x7fbbb815dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44576/0 (socket says 172.21.15.49:44576) 2023-11-06T02:09:49.232 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.232+0000 7fbbb7fff640 1 -- 172.21.15.49:0/2873591334 learned_addr learned my addr 172.21.15.49:0/2873591334 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:49.232 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.232+0000 7fbbaffff640 1 --2- 172.21.15.49:0/2873591334 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbbb806d640 0x7fbbb80b0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:49.232 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.232+0000 7fbbb7fff640 1 -- 172.21.15.49:0/2873591334 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbbb806d640 msgr2=0x7fbbb80b0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:49.232 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.232+0000 7fbbb7fff640 1 --2- 172.21.15.49:0/2873591334 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbbb806d640 0x7fbbb80b0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:49.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.232+0000 7fbbb7fff640 1 -- 172.21.15.49:0/2873591334 >> v2:172.21.15.150:3300/0 conn(0x7fbbb80b0760 msgr2=0x7fbbb80c8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:49.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.232+0000 7fbbb7fff640 1 --2- 172.21.15.49:0/2873591334 >> v2:172.21.15.150:3300/0 conn(0x7fbbb80b0760 0x7fbbb80c8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:49.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.232+0000 7fbbb7fff640 1 -- 172.21.15.49:0/2873591334 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbb806cb60 con 0x7fbbb81595a0 2023-11-06T02:09:49.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.232+0000 7fbbaffff640 1 --2- 172.21.15.49:0/2873591334 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbbb806d640 0x7fbbb80b0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:09:49.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.232+0000 7fbbb7fff640 1 --2- 172.21.15.49:0/2873591334 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbbb81595a0 0x7fbbb815dd10 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fbbb806e400 tx=0x7fbba801eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f3495f5edf09cc5 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:49.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.232+0000 7fbbb77fe640 1 -- 172.21.15.49:0/2873591334 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba80a7bf0 con 0x7fbbb81595a0 2023-11-06T02:09:49.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.232+0000 7fbbb77fe640 1 -- 172.21.15.49:0/2873591334 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbba80a7d50 con 0x7fbbb81595a0 2023-11-06T02:09:49.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.232+0000 7fbbb77fe640 1 -- 172.21.15.49:0/2873591334 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba80af660 con 0x7fbbb81595a0 2023-11-06T02:09:49.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.236+0000 7fbbbea40640 1 -- 172.21.15.49:0/2873591334 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbbb81595a0 msgr2=0x7fbbb815dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:49.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.236+0000 7fbbbea40640 1 --2- 172.21.15.49:0/2873591334 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbbb81595a0 0x7fbbb815dd10 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fbbb806e400 tx=0x7fbba801eba0 comp rx=0 tx=0).stop 2023-11-06T02:09:49.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.236+0000 7fbbbea40640 1 -- 172.21.15.49:0/2873591334 shutdown_connections 2023-11-06T02:09:49.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.236+0000 7fbbbea40640 1 --2- 172.21.15.49:0/2873591334 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbbb806d640 0x7fbbb80b0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:49.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.236+0000 7fbbbea40640 1 --2- 172.21.15.49:0/2873591334 >> v2:172.21.15.150:3300/0 conn(0x7fbbb80b0760 0x7fbbb80c8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:49.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.236+0000 7fbbbea40640 1 --2- 172.21.15.49:0/2873591334 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbbb81595a0 0x7fbbb815dd10 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:49.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.236+0000 7fbbbea40640 1 -- 172.21.15.49:0/2873591334 >> 172.21.15.49:0/2873591334 conn(0x7fbbb806a370 msgr2=0x7fbbb80ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:49.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.236+0000 7fbbbea40640 1 -- 172.21.15.49:0/2873591334 shutdown_connections 2023-11-06T02:09:49.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.236+0000 7fbbbea40640 1 -- 172.21.15.49:0/2873591334 wait complete. 2023-11-06T02:09:49.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.236+0000 7fbbbea40640 1 Processor -- start 2023-11-06T02:09:49.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.236+0000 7fbbbea40640 1 -- start start 2023-11-06T02:09:49.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.236+0000 7fbbbea40640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbbb806d640 0x7fbbb80b3150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:49.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.236+0000 7fbbbea40640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbbb80b0760 0x7fbbb80b1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:49.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.236+0000 7fbbbea40640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbbb81595a0 0x7fbbb813b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:49.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.236+0000 7fbbbea40640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb80b20b0 con 0x7fbbb80b0760 2023-11-06T02:09:49.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.236+0000 7fbbbea40640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb80b2390 con 0x7fbbb81595a0 2023-11-06T02:09:49.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.236+0000 7fbbbea40640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb80b2670 con 0x7fbbb806d640 2023-11-06T02:09:49.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.236+0000 7fbbaffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbbb80b0760 0x7fbbb80b1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:49.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.236+0000 7fbbaffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbbb80b0760 0x7fbbb80b1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44588/0 (socket says 172.21.15.49:44588) 2023-11-06T02:09:49.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.236+0000 7fbbaffff640 1 -- 172.21.15.49:0/1062334580 learned_addr learned my addr 172.21.15.49:0/1062334580 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:49.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.236+0000 7fbbb7fff640 1 --2- 172.21.15.49:0/1062334580 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbbb806d640 0x7fbbb80b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:49.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.236+0000 7fbbbcfdd640 1 --2- 172.21.15.49:0/1062334580 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbbb81595a0 0x7fbbb813b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:49.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.236+0000 7fbbaffff640 1 -- 172.21.15.49:0/1062334580 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbbb806d640 msgr2=0x7fbbb80b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:49.242 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.236+0000 7fbbaffff640 1 --2- 172.21.15.49:0/1062334580 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbbb806d640 0x7fbbb80b3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:49.242 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.236+0000 7fbbaffff640 1 -- 172.21.15.49:0/1062334580 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbbb81595a0 msgr2=0x7fbbb813b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:49.242 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.236+0000 7fbbaffff640 1 --2- 172.21.15.49:0/1062334580 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbbb81595a0 0x7fbbb813b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:49.242 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.236+0000 7fbbaffff640 1 -- 172.21.15.49:0/1062334580 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbba800b770 con 0x7fbbb80b0760 2023-11-06T02:09:49.242 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.236+0000 7fbbaffff640 1 --2- 172.21.15.49:0/1062334580 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbbb80b0760 0x7fbbb80b1a20 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fbba000c7f0 tx=0x7fbba000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:49.243 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.236+0000 7fbbb57fa640 1 -- 172.21.15.49:0/1062334580 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba0007820 con 0x7fbbb80b0760 2023-11-06T02:09:49.243 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.236+0000 7fbbb57fa640 1 -- 172.21.15.49:0/1062334580 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbba0007980 con 0x7fbbb80b0760 2023-11-06T02:09:49.243 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.236+0000 7fbbbea40640 1 -- 172.21.15.49:0/1062334580 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbb80b2950 con 0x7fbbb80b0760 2023-11-06T02:09:49.243 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.236+0000 7fbbb57fa640 1 -- 172.21.15.49:0/1062334580 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba0007d30 con 0x7fbbb80b0760 2023-11-06T02:09:49.243 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.236+0000 7fbbbea40640 1 -- 172.21.15.49:0/1062334580 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbb815c990 con 0x7fbbb80b0760 2023-11-06T02:09:49.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.240+0000 7fbbbea40640 1 -- 172.21.15.49:0/1062334580 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbb806c210 con 0x7fbbb80b0760 2023-11-06T02:09:49.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.240+0000 7fbbb57fa640 1 -- 172.21.15.49:0/1062334580 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fbba00113f0 con 0x7fbbb80b0760 2023-11-06T02:09:49.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.240+0000 7fbbb57fa640 1 --2- 172.21.15.49:0/1062334580 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fbb98065a60 0x7fbb98067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:49.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.240+0000 7fbbb57fa640 1 -- 172.21.15.49:0/1062334580 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbba0083360 con 0x7fbbb80b0760 2023-11-06T02:09:49.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.240+0000 7fbbb7fff640 1 --2- 172.21.15.49:0/1062334580 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fbb98065a60 0x7fbb98067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:49.246 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.240+0000 7fbbb7fff640 1 --2- 172.21.15.49:0/1062334580 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fbb98065a60 0x7fbb98067f20 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fbba80b1000 tx=0x7fbba8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:49.246 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.244+0000 7fbbb57fa640 1 -- 172.21.15.49:0/1062334580 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fbbb806c210 con 0x7fbbb80b0760 2023-11-06T02:09:49.454 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.456+0000 7fbbbea40640 1 -- 172.21.15.49:0/1062334580 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fbbb806c210 con 0x7fbbb80b0760 2023-11-06T02:09:49.463 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.464+0000 7fbbb57fa640 1 -- 172.21.15.49:0/1062334580 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) v1 ==== 190+0+0 (secure 0 0 0) 0x7fbbb806c210 con 0x7fbbb80b0760 2023-11-06T02:09:49.463 INFO:teuthology.orchestra.run.smithi049.stderr:updated caps for client.0 2023-11-06T02:09:49.465 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.464+0000 7fbbbea40640 1 -- 172.21.15.49:0/1062334580 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fbb98065a60 msgr2=0x7fbb98067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:49.465 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.464+0000 7fbbbea40640 1 --2- 172.21.15.49:0/1062334580 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fbb98065a60 0x7fbb98067f20 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fbba80b1000 tx=0x7fbba8008040 comp rx=0 tx=0).stop 2023-11-06T02:09:49.466 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.464+0000 7fbbbea40640 1 -- 172.21.15.49:0/1062334580 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbbb80b0760 msgr2=0x7fbbb80b1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:49.466 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.464+0000 7fbbbea40640 1 --2- 172.21.15.49:0/1062334580 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbbb80b0760 0x7fbbb80b1a20 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fbba000c7f0 tx=0x7fbba000ccb0 comp rx=0 tx=0).stop 2023-11-06T02:09:49.466 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.464+0000 7fbbbea40640 1 -- 172.21.15.49:0/1062334580 shutdown_connections 2023-11-06T02:09:49.466 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.464+0000 7fbbbea40640 1 --2- 172.21.15.49:0/1062334580 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fbb98065a60 0x7fbb98067f20 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:49.467 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.464+0000 7fbbbea40640 1 --2- 172.21.15.49:0/1062334580 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbbb80b0760 0x7fbbb80b1a20 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:49.467 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.464+0000 7fbbbea40640 1 --2- 172.21.15.49:0/1062334580 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbbb81595a0 0x7fbbb813b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:49.467 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.464+0000 7fbbbea40640 1 --2- 172.21.15.49:0/1062334580 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbbb806d640 0x7fbbb80b3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:49.467 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.464+0000 7fbbbea40640 1 -- 172.21.15.49:0/1062334580 >> 172.21.15.49:0/1062334580 conn(0x7fbbb806a370 msgr2=0x7fbbb8150c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:49.467 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.464+0000 7fbbbea40640 1 -- 172.21.15.49:0/1062334580 shutdown_connections 2023-11-06T02:09:49.468 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.468+0000 7fbbbea40640 1 -- 172.21.15.49:0/1062334580 wait complete. 2023-11-06T02:09:49.481 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T02:09:49.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.592+0000 7f8348438640 1 Processor -- start 2023-11-06T02:09:49.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.596+0000 7f8348438640 1 -- start start 2023-11-06T02:09:49.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.596+0000 7f8348438640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83400cda70 0x7f8340161650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:49.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.596+0000 7f8348438640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83400d17c0 con 0x7f83400b1f90 2023-11-06T02:09:49.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.596+0000 7f8348438640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83400ce6c0 con 0x7f83400714a0 2023-11-06T02:09:49.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.596+0000 7f8348438640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8340161b90 con 0x7f83400cda70 2023-11-06T02:09:49.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.596+0000 7f83461d4640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f83400b1f90 0x7f83400d1c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:49378/0 (socket says 172.21.15.49:49378) 2023-11-06T02:09:49.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.596+0000 7f83461d4640 1 -- 172.21.15.49:0/2266552831 learned_addr learned my addr 172.21.15.49:0/2266552831 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:09:49.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.596+0000 7f83451d2640 1 -- 172.21.15.49:0/2266552831 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3464839666 0 0) 0x7f83400d17c0 con 0x7f83400b1f90 2023-11-06T02:09:49.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.596+0000 7f83451d2640 1 -- 172.21.15.49:0/2266552831 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f832c003530 con 0x7f83400b1f90 2023-11-06T02:09:49.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.596+0000 7f83459d3640 1 --2- 172.21.15.49:0/2266552831 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83400cda70 0x7f8340161650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:49.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.596+0000 7f83451d2640 1 -- 172.21.15.49:0/2266552831 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1042164280 0 0) 0x7f832c003530 con 0x7f83400b1f90 2023-11-06T02:09:49.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.596+0000 7f83451d2640 1 -- 172.21.15.49:0/2266552831 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83400cda70 msgr2=0x7f8340161650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:49.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.596+0000 7f83451d2640 1 --2- 172.21.15.49:0/2266552831 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83400cda70 0x7f8340161650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:49.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.596+0000 7f83451d2640 1 -- 172.21.15.49:0/2266552831 >> v1:172.21.15.150:6789/0 conn(0x7f83400714a0 legacy=0x7f83400b6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:49.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.596+0000 7f83451d2640 1 -- 172.21.15.49:0/2266552831 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83400d2360 con 0x7f83400b1f90 2023-11-06T02:09:49.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.596+0000 7f83451d2640 1 -- 172.21.15.49:0/2266552831 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7f833c001680 con 0x7f83400b1f90 2023-11-06T02:09:49.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.596+0000 7f83451d2640 1 -- 172.21.15.49:0/2266552831 >> v1:172.21.15.49:6789/0 conn(0x7f83400b1f90 legacy=0x7f83400d1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:49.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.596+0000 7f83451d2640 1 --2- 172.21.15.49:0/2266552831 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f832c0039b0 0x7f832c003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:49.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.596+0000 7f83451d2640 1 --2- 172.21.15.49:0/2266552831 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83400cda70 0x7f832c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:49.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.596+0000 7f83451d2640 1 --2- 172.21.15.49:0/2266552831 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f832c004d10 0x7f832c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:49.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.596+0000 7f83451d2640 1 -- 172.21.15.49:0/2266552831 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f832c00bac0 con 0x7f832c0039b0 2023-11-06T02:09:49.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.596+0000 7f83451d2640 1 -- 172.21.15.49:0/2266552831 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f832c00bc30 con 0x7f832c004d10 2023-11-06T02:09:49.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.596+0000 7f83451d2640 1 -- 172.21.15.49:0/2266552831 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f832c00bdf0 con 0x7f83400cda70 2023-11-06T02:09:49.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.596+0000 7f83461d4640 1 --2- 172.21.15.49:0/2266552831 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f832c0039b0 0x7f832c003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:49.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.596+0000 7f83459d3640 1 --2- 172.21.15.49:0/2266552831 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83400cda70 0x7f832c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:49.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.596+0000 7f83469d5640 1 --2- 172.21.15.49:0/2266552831 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f832c004d10 0x7f832c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:49.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.596+0000 7f83461d4640 1 -- 172.21.15.49:0/2266552831 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83400cda70 msgr2=0x7f832c0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:49.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.596+0000 7f83461d4640 1 --2- 172.21.15.49:0/2266552831 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83400cda70 0x7f832c0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:49.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.596+0000 7f83461d4640 1 -- 172.21.15.49:0/2266552831 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f832c004d10 msgr2=0x7f832c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:49.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.596+0000 7f83461d4640 1 --2- 172.21.15.49:0/2266552831 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f832c004d10 0x7f832c005190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:49.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.596+0000 7f83461d4640 1 -- 172.21.15.49:0/2266552831 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f832c003530 con 0x7f832c0039b0 2023-11-06T02:09:49.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.600+0000 7f83461d4640 1 --2- 172.21.15.49:0/2266552831 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f832c0039b0 0x7f832c003db0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f833c00c210 tx=0x7f833c01cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c488f592e554f92 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:49.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.600+0000 7f83451d2640 1 -- 172.21.15.49:0/2266552831 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f833c005ae0 con 0x7f832c0039b0 2023-11-06T02:09:49.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.600+0000 7f83451d2640 1 -- 172.21.15.49:0/2266552831 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f833c005c40 con 0x7f832c0039b0 2023-11-06T02:09:49.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.600+0000 7f83451d2640 1 -- 172.21.15.49:0/2266552831 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f833c007070 con 0x7f832c0039b0 2023-11-06T02:09:49.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.600+0000 7f8348438640 1 -- 172.21.15.49:0/2266552831 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f832c0039b0 msgr2=0x7f832c003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:49.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.600+0000 7f8348438640 1 --2- 172.21.15.49:0/2266552831 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f832c0039b0 0x7f832c003db0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f833c00c210 tx=0x7f833c01cba0 comp rx=0 tx=0).stop 2023-11-06T02:09:49.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.600+0000 7f83469d5640 1 -- 172.21.15.49:0/2266552831 reap_dead start 2023-11-06T02:09:49.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.600+0000 7f8348438640 1 -- 172.21.15.49:0/2266552831 shutdown_connections 2023-11-06T02:09:49.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.600+0000 7f8348438640 1 -- 172.21.15.49:0/2266552831 >> 172.21.15.49:0/2266552831 conn(0x7f83400d2aa0 msgr2=0x7f83400d2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:49.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.600+0000 7f8348438640 1 -- 172.21.15.49:0/2266552831 shutdown_connections 2023-11-06T02:09:49.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.600+0000 7f8348438640 1 -- 172.21.15.49:0/2266552831 wait complete. 2023-11-06T02:09:49.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.600+0000 7f8348438640 1 Processor -- start 2023-11-06T02:09:49.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.600+0000 7f8348438640 1 -- start start 2023-11-06T02:09:49.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.600+0000 7f8348438640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f832c0039b0 0x7f83401534b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:49.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.600+0000 7f8348438640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83400b3490 0x7f834015cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:49.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.600+0000 7f8348438640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f83400b6de0 0x7f834015b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:49.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.600+0000 7f8348438640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83400cb530 con 0x7f832c0039b0 2023-11-06T02:09:49.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.600+0000 7f8348438640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8340153d20 con 0x7f83400b6de0 2023-11-06T02:09:49.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.600+0000 7f8348438640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83400ca670 con 0x7f83400b3490 2023-11-06T02:09:49.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.600+0000 7f83461d4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f832c0039b0 0x7f83401534b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:49.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.600+0000 7f83461d4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f832c0039b0 0x7f83401534b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44610/0 (socket says 172.21.15.49:44610) 2023-11-06T02:09:49.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.600+0000 7f83461d4640 1 -- 172.21.15.49:0/1826611894 learned_addr learned my addr 172.21.15.49:0/1826611894 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:49.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.600+0000 7f83459d3640 1 --2- 172.21.15.49:0/1826611894 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83400b3490 0x7f834015cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:49.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.600+0000 7f83469d5640 1 --2- 172.21.15.49:0/1826611894 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f83400b6de0 0x7f834015b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:49.612 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.600+0000 7f83461d4640 1 -- 172.21.15.49:0/1826611894 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83400b3490 msgr2=0x7f834015cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:49.612 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.600+0000 7f83461d4640 1 --2- 172.21.15.49:0/1826611894 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83400b3490 0x7f834015cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:49.612 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.600+0000 7f83461d4640 1 -- 172.21.15.49:0/1826611894 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f83400b6de0 msgr2=0x7f834015b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:49.612 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.600+0000 7f83461d4640 1 --2- 172.21.15.49:0/1826611894 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f83400b6de0 0x7f834015b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:49.612 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.600+0000 7f83461d4640 1 -- 172.21.15.49:0/1826611894 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83400ca950 con 0x7f832c0039b0 2023-11-06T02:09:49.613 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.600+0000 7f83461d4640 1 --2- 172.21.15.49:0/1826611894 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f832c0039b0 0x7f83401534b0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f833c00c210 tx=0x7f832c005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:49.613 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.604+0000 7f8336ffd640 1 -- 172.21.15.49:0/1826611894 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f833c0b4070 con 0x7f832c0039b0 2023-11-06T02:09:49.613 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.604+0000 7f8336ffd640 1 -- 172.21.15.49:0/1826611894 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f833c001500 con 0x7f832c0039b0 2023-11-06T02:09:49.613 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.604+0000 7f8348438640 1 -- 172.21.15.49:0/1826611894 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83400cac30 con 0x7f832c0039b0 2023-11-06T02:09:49.613 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.604+0000 7f8336ffd640 1 -- 172.21.15.49:0/1826611894 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f833c0070a0 con 0x7f832c0039b0 2023-11-06T02:09:49.614 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.604+0000 7f8348438640 1 -- 172.21.15.49:0/1826611894 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83400b7440 con 0x7f832c0039b0 2023-11-06T02:09:49.614 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.604+0000 7f8348438640 1 -- 172.21.15.49:0/1826611894 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8340074040 con 0x7f832c0039b0 2023-11-06T02:09:49.614 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.604+0000 7f8336ffd640 1 -- 172.21.15.49:0/1826611894 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f833c007200 con 0x7f832c0039b0 2023-11-06T02:09:49.614 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.604+0000 7f8336ffd640 1 --2- 172.21.15.49:0/1826611894 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8324065a60 0x7f8324067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:49.614 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.604+0000 7f8336ffd640 1 -- 172.21.15.49:0/1826611894 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f833c11bc80 con 0x7f832c0039b0 2023-11-06T02:09:49.615 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.604+0000 7f83459d3640 1 --2- 172.21.15.49:0/1826611894 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8324065a60 0x7f8324067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:49.615 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.604+0000 7f83459d3640 1 --2- 172.21.15.49:0/1826611894 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8324065a60 0x7f8324067f20 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f83400b50d0 tx=0x7f8330008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:49.615 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.608+0000 7f8336ffd640 1 -- 172.21.15.49:0/1826611894 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f833c0ecf70 con 0x7f832c0039b0 2023-11-06T02:09:49.790 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.792+0000 7f8348438640 1 -- 172.21.15.49:0/1826611894 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f83400d0c30 con 0x7f832c0039b0 2023-11-06T02:09:49.791 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.792+0000 7f8336ffd640 1 -- 172.21.15.49:0/1826611894 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4651 (secure 0 0 0) 0x7f833c0ecb00 con 0x7f832c0039b0 2023-11-06T02:09:49.791 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:09:49.792 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5032,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/280428805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":280428805},{"type":"v1","addr":"172.21.15.150:6835","nonce":280428805}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":5035,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/2734589844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2734589844},{"type":"v1","addr":"172.21.15.49:6833","nonce":2734589844}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":5038,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/1319221246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":1319221246},{"type":"v1","addr":"172.21.15.150:6837","nonce":1319221246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T02:09:39.873514+0000","modified":"2023-11-06T02:09:43.132890+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5041},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5041":{"gid":5041,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.49:6835/1547192091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":1547192091},{"type":"v1","addr":"172.21.15.49:6835","nonce":1547192091}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-06T02:09:49.792 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 15 2023-11-06T02:09:49.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.792+0000 7f8348438640 1 -- 172.21.15.49:0/1826611894 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8324065a60 msgr2=0x7f8324067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:49.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.792+0000 7f8348438640 1 --2- 172.21.15.49:0/1826611894 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8324065a60 0x7f8324067f20 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f83400b50d0 tx=0x7f8330008040 comp rx=0 tx=0).stop 2023-11-06T02:09:49.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.796+0000 7f8348438640 1 -- 172.21.15.49:0/1826611894 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f832c0039b0 msgr2=0x7f83401534b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:49.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.796+0000 7f8348438640 1 --2- 172.21.15.49:0/1826611894 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f832c0039b0 0x7f83401534b0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f833c00c210 tx=0x7f832c005800 comp rx=0 tx=0).stop 2023-11-06T02:09:49.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.796+0000 7f8348438640 1 -- 172.21.15.49:0/1826611894 shutdown_connections 2023-11-06T02:09:49.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.796+0000 7f8348438640 1 --2- 172.21.15.49:0/1826611894 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8324065a60 0x7f8324067f20 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:49.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.796+0000 7f8348438640 1 --2- 172.21.15.49:0/1826611894 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f83400b6de0 0x7f834015b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:49.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.796+0000 7f8348438640 1 --2- 172.21.15.49:0/1826611894 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83400b3490 0x7f834015cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:49.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.796+0000 7f8348438640 1 --2- 172.21.15.49:0/1826611894 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f832c0039b0 0x7f83401534b0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:49.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.796+0000 7f8348438640 1 -- 172.21.15.49:0/1826611894 >> 172.21.15.49:0/1826611894 conn(0x7f83400d2aa0 msgr2=0x7f83401687b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:49.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.796+0000 7f8348438640 1 -- 172.21.15.49:0/1826611894 shutdown_connections 2023-11-06T02:09:49.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:49.796+0000 7f8348438640 1 -- 172.21.15.49:0/1826611894 wait complete. 2023-11-06T02:09:50.213 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 15, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-06T02:09:39.873514+0000', 'modified': '2023-11-06T02:09:43.132890+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5041}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5041': {'gid': 5041, 'name': 'a', 'rank': 0, 'incarnation': 11, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.49:6835/1547192091', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6834', 'nonce': 1547192091}, {'type': 'v1', 'addr': '172.21.15.49:6835', 'nonce': 1547192091}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-06T02:09:50.214 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-06T02:09:50.214 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-06T02:09:50.351 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.352+0000 7f6c502c4640 1 Processor -- start 2023-11-06T02:09:50.352 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.352+0000 7f6c502c4640 1 -- start start 2023-11-06T02:09:50.353 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.352+0000 7f6c502c4640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6c4815ab50 0x7f6c480c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:50.353 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.352+0000 7f6c502c4640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f6c480b0790 0x7f6c480c2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:50.353 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.352+0000 7f6c502c4640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c480cab80 con 0x7f6c480cb010 2023-11-06T02:09:50.353 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.352+0000 7f6c502c4640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c4815af30 con 0x7f6c480b0790 2023-11-06T02:09:50.353 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.352+0000 7f6c502c4640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c480cb5b0 con 0x7f6c4815ab50 2023-11-06T02:09:50.354 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.352+0000 7f6c4e861640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f6c480b0790 0x7f6c480c2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:50.354 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.352+0000 7f6c4e861640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f6c480b0790 0x7f6c480c2710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.49:37764/0 (socket says 172.21.15.49:37764) 2023-11-06T02:09:50.354 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.352+0000 7f6c4e861640 1 -- 172.21.15.49:0/529626037 learned_addr learned my addr 172.21.15.49:0/529626037 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:50.354 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.352+0000 7f6c4e060640 1 --2- 172.21.15.49:0/529626037 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6c4815ab50 0x7f6c480c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:50.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.356+0000 7f6c4e861640 1 -- 172.21.15.49:0/529626037 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6c4815ab50 msgr2=0x7f6c480c9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:50.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.356+0000 7f6c4e861640 1 --2- 172.21.15.49:0/529626037 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6c4815ab50 0x7f6c480c9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:50.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.356+0000 7f6c4e861640 1 -- 172.21.15.49:0/529626037 >> v1:172.21.15.150:6789/0 conn(0x7f6c480cb010 legacy=0x7f6c480b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:50.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.356+0000 7f6c4e861640 1 -- 172.21.15.49:0/529626037 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c480cb890 con 0x7f6c480b0790 2023-11-06T02:09:50.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.356+0000 7f6c4e861640 1 --2- 172.21.15.49:0/529626037 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6c480b0790 0x7f6c480c2710 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6c38009870 tx=0x7f6c3801c510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42264d63b8ad93b6 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:50.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.356+0000 7f6c4d05e640 1 -- 172.21.15.49:0/529626037 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c380a95f0 con 0x7f6c480b0790 2023-11-06T02:09:50.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.356+0000 7f6c4d05e640 1 -- 172.21.15.49:0/529626037 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c380a9750 con 0x7f6c480b0790 2023-11-06T02:09:50.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.356+0000 7f6c4d05e640 1 -- 172.21.15.49:0/529626037 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c380a9ae0 con 0x7f6c480b0790 2023-11-06T02:09:50.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.356+0000 7f6c502c4640 1 -- 172.21.15.49:0/529626037 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6c480b0790 msgr2=0x7f6c480c2710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:50.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.356+0000 7f6c502c4640 1 --2- 172.21.15.49:0/529626037 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6c480b0790 0x7f6c480c2710 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6c38009870 tx=0x7f6c3801c510 comp rx=0 tx=0).stop 2023-11-06T02:09:50.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.356+0000 7f6c502c4640 1 -- 172.21.15.49:0/529626037 shutdown_connections 2023-11-06T02:09:50.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.356+0000 7f6c502c4640 1 --2- 172.21.15.49:0/529626037 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6c480b0790 0x7f6c480c2710 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:50.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.356+0000 7f6c502c4640 1 --2- 172.21.15.49:0/529626037 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6c4815ab50 0x7f6c480c9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:50.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.356+0000 7f6c502c4640 1 -- 172.21.15.49:0/529626037 >> 172.21.15.49:0/529626037 conn(0x7f6c4806c0a0 msgr2=0x7f6c4806c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:50.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.356+0000 7f6c502c4640 1 -- 172.21.15.49:0/529626037 shutdown_connections 2023-11-06T02:09:50.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.356+0000 7f6c502c4640 1 -- 172.21.15.49:0/529626037 wait complete. 2023-11-06T02:09:50.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.360+0000 7f6c502c4640 1 Processor -- start 2023-11-06T02:09:50.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.360+0000 7f6c502c4640 1 -- start start 2023-11-06T02:09:50.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.360+0000 7f6c502c4640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6c480b0790 0x7f6c48149b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:50.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.360+0000 7f6c502c4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c480cb010 0x7f6c48151ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:50.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.360+0000 7f6c502c4640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6c4815ab50 0x7f6c48152420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:50.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.360+0000 7f6c502c4640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c4814a040 con 0x7f6c480cb010 2023-11-06T02:09:50.365 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.360+0000 7f6c502c4640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c480cda00 con 0x7f6c480b0790 2023-11-06T02:09:50.365 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.360+0000 7f6c502c4640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c480acd90 con 0x7f6c4815ab50 2023-11-06T02:09:50.365 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.360+0000 7f6c4d85f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c480cb010 0x7f6c48151ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:50.365 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.360+0000 7f6c4d85f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c480cb010 0x7f6c48151ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44624/0 (socket says 172.21.15.49:44624) 2023-11-06T02:09:50.365 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.360+0000 7f6c4d85f640 1 -- 172.21.15.49:0/1234069245 learned_addr learned my addr 172.21.15.49:0/1234069245 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:50.365 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.360+0000 7f6c4e060640 1 --2- 172.21.15.49:0/1234069245 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6c480b0790 0x7f6c48149b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:50.366 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.360+0000 7f6c4e861640 1 --2- 172.21.15.49:0/1234069245 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6c4815ab50 0x7f6c48152420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:50.366 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.360+0000 7f6c4d85f640 1 -- 172.21.15.49:0/1234069245 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6c4815ab50 msgr2=0x7f6c48152420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:50.366 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.360+0000 7f6c4d85f640 1 --2- 172.21.15.49:0/1234069245 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6c4815ab50 0x7f6c48152420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:50.366 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.360+0000 7f6c4d85f640 1 -- 172.21.15.49:0/1234069245 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6c480b0790 msgr2=0x7f6c48149b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:50.366 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.360+0000 7f6c4d85f640 1 --2- 172.21.15.49:0/1234069245 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6c480b0790 0x7f6c48149b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:50.367 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.360+0000 7f6c4d85f640 1 -- 172.21.15.49:0/1234069245 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c380094d0 con 0x7f6c480cb010 2023-11-06T02:09:50.367 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.360+0000 7f6c4d85f640 1 --2- 172.21.15.49:0/1234069245 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c480cb010 0x7f6c48151ee0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c00c420 tx=0x7f6c3c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:50.367 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.360+0000 7f6c46ffd640 1 -- 172.21.15.49:0/1234069245 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c0052d0 con 0x7f6c480cb010 2023-11-06T02:09:50.367 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.360+0000 7f6c46ffd640 1 -- 172.21.15.49:0/1234069245 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c3c005430 con 0x7f6c480cb010 2023-11-06T02:09:50.367 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.360+0000 7f6c502c4640 1 -- 172.21.15.49:0/1234069245 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c480ad070 con 0x7f6c480cb010 2023-11-06T02:09:50.368 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.360+0000 7f6c46ffd640 1 -- 172.21.15.49:0/1234069245 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c007260 con 0x7f6c480cb010 2023-11-06T02:09:50.368 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.360+0000 7f6c502c4640 1 -- 172.21.15.49:0/1234069245 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c48152ac0 con 0x7f6c480cb010 2023-11-06T02:09:50.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.360+0000 7f6c46ffd640 1 -- 172.21.15.49:0/1234069245 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f6c3c0073c0 con 0x7f6c480cb010 2023-11-06T02:09:50.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.364+0000 7f6c46ffd640 1 --2- 172.21.15.49:0/1234069245 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6c34065a90 0x7f6c34067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:50.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.364+0000 7f6c46ffd640 1 -- 172.21.15.49:0/1234069245 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6c3c082670 con 0x7f6c480cb010 2023-11-06T02:09:50.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.364+0000 7f6c4e060640 1 --2- 172.21.15.49:0/1234069245 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6c34065a90 0x7f6c34067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:50.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.364+0000 7f6c533ec000 1 Processor -- start 2023-11-06T02:09:50.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.364+0000 7f6c4e060640 1 --2- 172.21.15.49:0/1234069245 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6c34065a90 0x7f6c34067f50 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6c480c90c0 tx=0x7f6c4000b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:50.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.364+0000 7f6c533ec000 1 -- start start 2023-11-06T02:09:50.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.364+0000 7f6c533ec000 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55965c21b6e0 0x55965c21bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:50.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.364+0000 7f6c533ec000 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55965c21c000 0x55965c1d1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:50.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.364+0000 7f6c533ec000 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55965c1d1a90 0x55965c1d1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:50.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.364+0000 7f6c533ec000 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55965c099fc0 con 0x55965c21c000 2023-11-06T02:09:50.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.364+0000 7f6c533ec000 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55965c0281a0 con 0x55965c21b6e0 2023-11-06T02:09:50.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.364+0000 7f6c533ec000 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55965c025770 con 0x55965c1d1a90 2023-11-06T02:09:50.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.364+0000 7f6c4d85f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55965c21c000 0x55965c1d1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:50.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.364+0000 7f6c4d85f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55965c21c000 0x55965c1d1550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44636/0 (socket says 172.21.15.49:44636) 2023-11-06T02:09:50.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.364+0000 7f6c4d85f640 1 -- 172.21.15.49:0/505150161 learned_addr learned my addr 172.21.15.49:0/505150161 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:50.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.364+0000 7f6c4e060640 1 --2- 172.21.15.49:0/505150161 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55965c21b6e0 0x55965c21bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:50.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.364+0000 7f6c4e861640 1 --2- 172.21.15.49:0/505150161 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55965c1d1a90 0x55965c1d1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:50.378 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.364+0000 7f6c4d85f640 1 -- 172.21.15.49:0/505150161 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55965c1d1a90 msgr2=0x55965c1d1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:50.378 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.364+0000 7f6c4d85f640 1 --2- 172.21.15.49:0/505150161 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55965c1d1a90 0x55965c1d1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:50.378 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.364+0000 7f6c4d85f640 1 -- 172.21.15.49:0/505150161 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55965c21b6e0 msgr2=0x55965c21bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:50.378 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.364+0000 7f6c4d85f640 1 --2- 172.21.15.49:0/505150161 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55965c21b6e0 0x55965c21bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:50.378 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.364+0000 7f6c4d85f640 1 -- 172.21.15.49:0/505150161 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55965bd1d890 con 0x55965c21c000 2023-11-06T02:09:50.378 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.364+0000 7f6c4e060640 1 --2- 172.21.15.49:0/505150161 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55965c21b6e0 0x55965c21bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:09:50.379 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.364+0000 7f6c4d85f640 1 --2- 172.21.15.49:0/505150161 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55965c21c000 0x55965c1d1550 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c053b30 tx=0x7f6c3c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9929e7b0b7d7baf server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:50.379 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.364+0000 7f6c2ffff640 1 -- 172.21.15.49:0/505150161 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c05bad0 con 0x55965c21c000 2023-11-06T02:09:50.379 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.364+0000 7f6c2ffff640 1 -- 172.21.15.49:0/505150161 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c3c051ce0 con 0x55965c21c000 2023-11-06T02:09:50.379 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.364+0000 7f6c2ffff640 1 -- 172.21.15.49:0/505150161 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c0502b0 con 0x55965c21c000 2023-11-06T02:09:50.379 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.364+0000 7f6c533ec000 1 -- 172.21.15.49:0/505150161 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55965c21c000 msgr2=0x55965c1d1550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:50.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.364+0000 7f6c533ec000 1 --2- 172.21.15.49:0/505150161 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55965c21c000 0x55965c1d1550 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c053b30 tx=0x7f6c3c00cf90 comp rx=0 tx=0).stop 2023-11-06T02:09:50.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.364+0000 7f6c533ec000 1 -- 172.21.15.49:0/505150161 shutdown_connections 2023-11-06T02:09:50.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.364+0000 7f6c533ec000 1 --2- 172.21.15.49:0/505150161 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55965c21c000 0x55965c1d1550 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:50.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.364+0000 7f6c533ec000 1 --2- 172.21.15.49:0/505150161 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55965c1d1a90 0x55965c1d1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:50.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.364+0000 7f6c533ec000 1 --2- 172.21.15.49:0/505150161 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55965c21b6e0 0x55965c21bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:50.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.364+0000 7f6c533ec000 1 -- 172.21.15.49:0/505150161 >> 172.21.15.49:0/505150161 conn(0x55965c1d7bf0 msgr2=0x55965c1cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:50.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.364+0000 7f6c533ec000 1 -- 172.21.15.49:0/505150161 shutdown_connections 2023-11-06T02:09:50.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.364+0000 7f6c533ec000 1 -- 172.21.15.49:0/505150161 wait complete. 2023-11-06T02:09:50.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.368+0000 7f6c533ec000 1 -- start start 2023-11-06T02:09:50.382 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.368+0000 7f6c533ec000 1 Processor -- start 2023-11-06T02:09:50.382 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.368+0000 7f6c533ec000 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55965c1d1a90 0x55965c1dc880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:50.382 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.368+0000 7f6c533ec000 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55965c21b6e0 0x55965c1dcdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:50.382 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.368+0000 7f6c533ec000 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55965c21c000 0x55965c1dd300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:50.383 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.368+0000 7f6c533ec000 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55965c021060 con 0x55965c1d1a90 2023-11-06T02:09:50.383 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.368+0000 7f6c533ec000 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55965bfc0290 con 0x55965c21b6e0 2023-11-06T02:09:50.383 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.368+0000 7f6c533ec000 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55965bfde6f0 con 0x55965c21c000 2023-11-06T02:09:50.383 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.368+0000 7f6c533ec000 10 client.0 _ll_register_callbacks cb 0x55965c1bd1d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T02:09:50.383 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.368+0000 7f6c4e060640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55965c1d1a90 0x55965c1dc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:50.383 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.368+0000 7f6c4e060640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55965c1d1a90 0x55965c1dc880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44638/0 (socket says 172.21.15.49:44638) 2023-11-06T02:09:50.384 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.368+0000 7f6c4e060640 1 -- 172.21.15.49:0/722432225 learned_addr learned my addr 172.21.15.49:0/722432225 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:50.384 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.368+0000 7f6c4d85f640 1 --2- 172.21.15.49:0/722432225 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55965c21b6e0 0x55965c1dcdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:50.384 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.368+0000 7f6c4e861640 1 --2- 172.21.15.49:0/722432225 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55965c21c000 0x55965c1dd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:50.384 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.368+0000 7f6c4e060640 1 -- 172.21.15.49:0/722432225 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55965c21c000 msgr2=0x55965c1dd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:50.384 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.368+0000 7f6c4e060640 1 --2- 172.21.15.49:0/722432225 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55965c21c000 0x55965c1dd300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:50.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.368+0000 7f6c4e060640 1 -- 172.21.15.49:0/722432225 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55965c21b6e0 msgr2=0x55965c1dcdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:50.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.368+0000 7f6c4e060640 1 --2- 172.21.15.49:0/722432225 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55965c21b6e0 0x55965c1dcdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:50.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.368+0000 7f6c4e060640 1 -- 172.21.15.49:0/722432225 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c48152ac0 con 0x55965c1d1a90 2023-11-06T02:09:50.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.368+0000 7f6c4e861640 1 --2- 172.21.15.49:0/722432225 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55965c21c000 0x55965c1dd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:09:50.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.368+0000 7f6c4e060640 1 --2- 172.21.15.49:0/722432225 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55965c1d1a90 0x55965c1dc880 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f6c40004470 tx=0x7f6c40004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ebcd74064be981a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:50.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.368+0000 7f6c2d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-11-06T02:09:50.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.368+0000 7f6c2d7fa640 1 -- 172.21.15.49:0/722432225 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4000e040 con 0x55965c1d1a90 2023-11-06T02:09:50.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.368+0000 7f6c2d7fa640 1 -- 172.21.15.49:0/722432225 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6c40015de0 con 0x55965c1d1a90 2023-11-06T02:09:50.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.368+0000 7f6c533ec000 1 -- 172.21.15.49:0/722432225 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55965c01f9b0 con 0x55965c1d1a90 2023-11-06T02:09:50.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.368+0000 7f6c2d7fa640 1 -- 172.21.15.49:0/722432225 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c40017440 con 0x55965c1d1a90 2023-11-06T02:09:50.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.368+0000 7f6c533ec000 10 client.5559 fetch_fsmap learned FSMap version 15 2023-11-06T02:09:50.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.368+0000 7f6c533ec000 1 -- 172.21.15.49:0/722432225 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) v3 -- 0x55965bfe61e0 con 0x55965c1d1a90 2023-11-06T02:09:50.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.368+0000 7f6c2d7fa640 1 -- 172.21.15.49:0/722432225 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55965c01f9b0 con 0x55965c1d1a90 2023-11-06T02:09:50.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.368+0000 7f6c2d7fa640 1 -- 172.21.15.49:0/722432225 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 15) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f6c400175a0 con 0x55965c1d1a90 2023-11-06T02:09:50.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.368+0000 7f6c533ec000 10 client.5559 fetch_fsmap finished waiting for FSMap version 15 2023-11-06T02:09:50.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.368+0000 7f6c533ec000 10 client.5559 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-06T02:09:50.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.368+0000 7f6c533ec000 20 client.5559 populate_metadata read hostname 'smithi049' 2023-11-06T02:09:50.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.368+0000 7f6c533ec000 1 --2- 172.21.15.49:0/722432225 >> [v2:172.21.15.49:6834/1547192091,v1:172.21.15.49:6835/1547192091] conn(0x55965c22a770 0x55965c22cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:50.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.368+0000 7f6c533ec000 4 client.5559 mds_command: new command op to 5041 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T02:09:50.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.368+0000 7f6c533ec000 1 -- 172.21.15.49:0/722432225 --> [v2:172.21.15.49:6834/1547192091,v1:172.21.15.49:6835/1547192091] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55965c0504d0 con 0x55965c22a770 2023-11-06T02:09:50.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.368+0000 7f6c4d85f640 1 --2- 172.21.15.49:0/722432225 >> [v2:172.21.15.49:6834/1547192091,v1:172.21.15.49:6835/1547192091] conn(0x55965c22a770 0x55965c22cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:50.389 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.368+0000 7f6c2d7fa640 10 client.5559 ms_handle_connect on v2:172.21.15.49:6834/1547192091 2023-11-06T02:09:50.389 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.368+0000 7f6c4d85f640 1 --2- 172.21.15.49:0/722432225 >> [v2:172.21.15.49:6834/1547192091,v1:172.21.15.49:6835/1547192091] conn(0x55965c22a770 0x55965c22cb60 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acaff91cf75a97b5 server_cookie=2e2575c2021702b6 in_seq=0 out_seq=0 2023-11-06T02:09:50.389 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.368+0000 7f6c2d7fa640 1 -- 172.21.15.49:0/722432225 <== mds.0 v2:172.21.15.49:6834/1547192091 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55965c0504d0 con 0x55965c22a770 2023-11-06T02:09:50.389 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.368+0000 7f6c2d7fa640 10 client.5559 handle_command_reply: tid=0 2023-11-06T02:09:50.389 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.368+0000 7f6c533ec000 1 client.5559 shutdown 2023-11-06T02:09:50.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.368+0000 7f6c533ec000 1 -- 172.21.15.49:0/722432225 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55965c1d1a90 msgr2=0x55965c1dc880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:50.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.368+0000 7f6c533ec000 1 --2- 172.21.15.49:0/722432225 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55965c1d1a90 0x55965c1dc880 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f6c40004470 tx=0x7f6c40004ac0 comp rx=0 tx=0).stop 2023-11-06T02:09:50.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.368+0000 7f6c533ec000 1 -- 172.21.15.49:0/722432225 shutdown_connections 2023-11-06T02:09:50.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.368+0000 7f6c533ec000 1 --2- 172.21.15.49:0/722432225 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55965c21c000 0x55965c1dd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:50.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.368+0000 7f6c533ec000 1 --2- 172.21.15.49:0/722432225 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55965c21b6e0 0x55965c1dcdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:50.391 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.368+0000 7f6c533ec000 1 --2- 172.21.15.49:0/722432225 >> [v2:172.21.15.49:6834/1547192091,v1:172.21.15.49:6835/1547192091] conn(0x55965c22a770 0x55965c22cb60 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:50.391 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.368+0000 7f6c533ec000 1 --2- 172.21.15.49:0/722432225 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55965c1d1a90 0x55965c1dc880 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:50.391 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.368+0000 7f6c2d7fa640 0 client.5559 ms_handle_reset on v2:172.21.15.49:6834/1547192091 2023-11-06T02:09:50.391 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.368+0000 7f6c533ec000 1 -- 172.21.15.49:0/722432225 >> 172.21.15.49:0/722432225 conn(0x55965c1d7bf0 msgr2=0x55965c1d0530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:50.391 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.368+0000 7f6c533ec000 1 -- 172.21.15.49:0/722432225 shutdown_connections 2023-11-06T02:09:50.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.368+0000 7f6c533ec000 1 -- 172.21.15.49:0/722432225 wait complete. 2023-11-06T02:09:50.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.372+0000 7f6c533ec000 20 client.5559 trim_cache size 0 max 16384 2023-11-06T02:09:50.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.384+0000 7f6c533ec000 1 Processor -- start 2023-11-06T02:09:50.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.384+0000 7f6c533ec000 1 -- start start 2023-11-06T02:09:50.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.384+0000 7f6c533ec000 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55965c1d1a90 0x55965c22e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:50.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.384+0000 7f6c533ec000 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55965c21b6e0 0x55965c22e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:50.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.384+0000 7f6c533ec000 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55965c21c000 0x55965c214250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:50.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.384+0000 7f6c533ec000 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55965bcde940 con 0x55965c21b6e0 2023-11-06T02:09:50.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.384+0000 7f6c533ec000 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55965c01c9d0 con 0x55965c21c000 2023-11-06T02:09:50.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.384+0000 7f6c533ec000 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55965c0225f0 con 0x55965c1d1a90 2023-11-06T02:09:50.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.384+0000 7f6c4d85f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55965c21b6e0 0x55965c22e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:50.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.384+0000 7f6c4d85f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55965c21b6e0 0x55965c22e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44646/0 (socket says 172.21.15.49:44646) 2023-11-06T02:09:50.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.384+0000 7f6c4d85f640 1 -- 172.21.15.49:0/2962656462 learned_addr learned my addr 172.21.15.49:0/2962656462 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:50.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.384+0000 7f6c4e060640 1 --2- 172.21.15.49:0/2962656462 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55965c1d1a90 0x55965c22e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:50.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.384+0000 7f6c4e861640 1 --2- 172.21.15.49:0/2962656462 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55965c21c000 0x55965c214250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:50.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.388+0000 7f6c4d85f640 1 -- 172.21.15.49:0/2962656462 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55965c1d1a90 msgr2=0x55965c22e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:50.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.388+0000 7f6c4d85f640 1 --2- 172.21.15.49:0/2962656462 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55965c1d1a90 0x55965c22e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:50.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.388+0000 7f6c4d85f640 1 -- 172.21.15.49:0/2962656462 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55965c21c000 msgr2=0x55965c214250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:50.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.388+0000 7f6c4d85f640 1 --2- 172.21.15.49:0/2962656462 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55965c21c000 0x55965c214250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:50.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.388+0000 7f6c4d85f640 1 -- 172.21.15.49:0/2962656462 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55965bfcc330 con 0x55965c21b6e0 2023-11-06T02:09:50.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.388+0000 7f6c4d85f640 1 --2- 172.21.15.49:0/2962656462 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55965c21b6e0 0x55965c22e7c0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c053b30 tx=0x7f6c3c082070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c3679b008049b77 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:50.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.388+0000 7f6c44ff9640 1 -- 172.21.15.49:0/2962656462 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c051ce0 con 0x55965c21b6e0 2023-11-06T02:09:50.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.388+0000 7f6c44ff9640 1 -- 172.21.15.49:0/2962656462 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c3c07e070 con 0x55965c21b6e0 2023-11-06T02:09:50.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.388+0000 7f6c44ff9640 1 -- 172.21.15.49:0/2962656462 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c051f40 con 0x55965c21b6e0 2023-11-06T02:09:50.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.388+0000 7f6c533ec000 1 -- 172.21.15.49:0/2962656462 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55965c21b6e0 msgr2=0x55965c22e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:50.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.388+0000 7f6c533ec000 1 --2- 172.21.15.49:0/2962656462 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55965c21b6e0 0x55965c22e7c0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c053b30 tx=0x7f6c3c082070 comp rx=0 tx=0).stop 2023-11-06T02:09:50.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.388+0000 7f6c533ec000 1 -- 172.21.15.49:0/2962656462 shutdown_connections 2023-11-06T02:09:50.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.388+0000 7f6c533ec000 1 --2- 172.21.15.49:0/2962656462 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55965c21b6e0 0x55965c22e7c0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:50.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.388+0000 7f6c533ec000 1 --2- 172.21.15.49:0/2962656462 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55965c21c000 0x55965c214250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:50.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.388+0000 7f6c533ec000 1 --2- 172.21.15.49:0/2962656462 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55965c1d1a90 0x55965c22e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:50.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.388+0000 7f6c533ec000 1 -- 172.21.15.49:0/2962656462 >> 172.21.15.49:0/2962656462 conn(0x55965c1d7bf0 msgr2=0x55965c22cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:50.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.388+0000 7f6c533ec000 1 -- 172.21.15.49:0/2962656462 shutdown_connections 2023-11-06T02:09:50.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.388+0000 7f6c533ec000 1 -- 172.21.15.49:0/2962656462 wait complete. 2023-11-06T02:09:50.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.388+0000 7f6c533ec000 1 -- start start 2023-11-06T02:09:50.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.388+0000 7f6c533ec000 1 Processor -- start 2023-11-06T02:09:50.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.388+0000 7f6c533ec000 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55965c1d1a90 0x55965c277ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:50.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.388+0000 7f6c533ec000 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55965c21b6e0 0x55965c2781e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:50.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.388+0000 7f6c533ec000 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55965c21c000 0x55965c2742f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:50.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.388+0000 7f6c533ec000 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55965c021910 con 0x55965c1d1a90 2023-11-06T02:09:50.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.388+0000 7f6c533ec000 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55965bff0090 con 0x55965c21b6e0 2023-11-06T02:09:50.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.388+0000 7f6c533ec000 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55965bfe37f0 con 0x55965c21c000 2023-11-06T02:09:50.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.388+0000 7f6c533ec000 10 client.0 _ll_register_callbacks cb 0x55965c19d9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T02:09:50.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.388+0000 7f6c4e060640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55965c1d1a90 0x55965c277ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:50.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.388+0000 7f6c4e060640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55965c1d1a90 0x55965c277ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44648/0 (socket says 172.21.15.49:44648) 2023-11-06T02:09:50.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.388+0000 7f6c4e060640 1 -- 172.21.15.49:0/4175207946 learned_addr learned my addr 172.21.15.49:0/4175207946 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:50.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.388+0000 7f6c4d85f640 1 --2- 172.21.15.49:0/4175207946 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55965c21b6e0 0x55965c2781e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:50.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.388+0000 7f6c4e861640 1 --2- 172.21.15.49:0/4175207946 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55965c21c000 0x55965c2742f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:50.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.388+0000 7f6c4e060640 1 -- 172.21.15.49:0/4175207946 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55965c21c000 msgr2=0x55965c2742f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:50.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.388+0000 7f6c4e060640 1 --2- 172.21.15.49:0/4175207946 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55965c21c000 0x55965c2742f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:50.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.388+0000 7f6c4e060640 1 -- 172.21.15.49:0/4175207946 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55965c21b6e0 msgr2=0x55965c2781e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:50.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.388+0000 7f6c4e060640 1 --2- 172.21.15.49:0/4175207946 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55965c21b6e0 0x55965c2781e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:50.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.388+0000 7f6c4e060640 1 -- 172.21.15.49:0/4175207946 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c3c06b040 con 0x55965c1d1a90 2023-11-06T02:09:50.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.388+0000 7f6c4e060640 1 --2- 172.21.15.49:0/4175207946 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55965c1d1a90 0x55965c277ca0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f6c40003dc0 tx=0x7f6c40004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2db109084ccca820 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:50.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.388+0000 7f6c2d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-11-06T02:09:50.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.388+0000 7f6c2d7fa640 1 -- 172.21.15.49:0/4175207946 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4000e030 con 0x55965c1d1a90 2023-11-06T02:09:50.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.388+0000 7f6c2d7fa640 1 -- 172.21.15.49:0/4175207946 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6c4003ec50 con 0x55965c1d1a90 2023-11-06T02:09:50.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.388+0000 7f6c533ec000 1 -- 172.21.15.49:0/4175207946 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55965bff0b90 con 0x55965c1d1a90 2023-11-06T02:09:50.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.388+0000 7f6c2d7fa640 1 -- 172.21.15.49:0/4175207946 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c40015820 con 0x55965c1d1a90 2023-11-06T02:09:50.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.388+0000 7f6c533ec000 10 client.5565 fetch_fsmap learned FSMap version 15 2023-11-06T02:09:50.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.388+0000 7f6c533ec000 1 -- 172.21.15.49:0/4175207946 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) v3 -- 0x55965bd1dcc0 con 0x55965c1d1a90 2023-11-06T02:09:50.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.388+0000 7f6c2d7fa640 1 -- 172.21.15.49:0/4175207946 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55965bff0b90 con 0x55965c1d1a90 2023-11-06T02:09:50.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.388+0000 7f6c2d7fa640 1 -- 172.21.15.49:0/4175207946 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 15) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f6c40017a10 con 0x55965c1d1a90 2023-11-06T02:09:50.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.388+0000 7f6c533ec000 10 client.5565 fetch_fsmap finished waiting for FSMap version 15 2023-11-06T02:09:50.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.388+0000 7f6c533ec000 10 client.5565 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-06T02:09:50.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.388+0000 7f6c533ec000 20 client.5565 populate_metadata read hostname 'smithi049' 2023-11-06T02:09:50.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.388+0000 7f6c533ec000 1 --2- 172.21.15.49:0/4175207946 >> [v2:172.21.15.49:6834/1547192091,v1:172.21.15.49:6835/1547192091] conn(0x55965c274830 0x55965c274c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:50.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.388+0000 7f6c533ec000 4 client.5565 mds_command: new command op to 5041 tid=0[{"prefix": "status"}] 2023-11-06T02:09:50.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.388+0000 7f6c533ec000 1 -- 172.21.15.49:0/4175207946 --> [v2:172.21.15.49:6834/1547192091,v1:172.21.15.49:6835/1547192091] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55965c01a0e0 con 0x55965c274830 2023-11-06T02:09:50.411 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.388+0000 7f6c4d85f640 1 --2- 172.21.15.49:0/4175207946 >> [v2:172.21.15.49:6834/1547192091,v1:172.21.15.49:6835/1547192091] conn(0x55965c274830 0x55965c274c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:50.411 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.392+0000 7f6c4d85f640 1 --2- 172.21.15.49:0/4175207946 >> [v2:172.21.15.49:6834/1547192091,v1:172.21.15.49:6835/1547192091] conn(0x55965c274830 0x55965c274c10 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c7fcc48227a8558 server_cookie=7a232678cadacf81 in_seq=0 out_seq=0 2023-11-06T02:09:50.411 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.392+0000 7f6c2d7fa640 10 client.5565 ms_handle_connect on v2:172.21.15.49:6834/1547192091 2023-11-06T02:09:50.411 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.392+0000 7f6c2d7fa640 1 -- 172.21.15.49:0/4175207946 <== mds.0 v2:172.21.15.49:6834/1547192091 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55965c01a0e0 con 0x55965c274830 2023-11-06T02:09:50.411 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.392+0000 7f6c2d7fa640 10 client.5565 handle_command_reply: tid=0 2023-11-06T02:09:50.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.392+0000 7f6c533ec000 1 client.5565 shutdown 2023-11-06T02:09:50.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.392+0000 7f6c533ec000 1 -- 172.21.15.49:0/4175207946 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55965c1d1a90 msgr2=0x55965c277ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:50.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.392+0000 7f6c533ec000 1 --2- 172.21.15.49:0/4175207946 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55965c1d1a90 0x55965c277ca0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f6c40003dc0 tx=0x7f6c40004a50 comp rx=0 tx=0).stop 2023-11-06T02:09:50.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.392+0000 7f6c533ec000 1 -- 172.21.15.49:0/4175207946 shutdown_connections 2023-11-06T02:09:50.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.392+0000 7f6c533ec000 1 --2- 172.21.15.49:0/4175207946 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55965c21c000 0x55965c2742f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:50.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.392+0000 7f6c533ec000 1 --2- 172.21.15.49:0/4175207946 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55965c21b6e0 0x55965c2781e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:50.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.392+0000 7f6c533ec000 1 --2- 172.21.15.49:0/4175207946 >> [v2:172.21.15.49:6834/1547192091,v1:172.21.15.49:6835/1547192091] conn(0x55965c274830 0x55965c274c10 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:50.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.392+0000 7f6c533ec000 1 --2- 172.21.15.49:0/4175207946 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55965c1d1a90 0x55965c277ca0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:50.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.392+0000 7f6c533ec000 1 -- 172.21.15.49:0/4175207946 >> 172.21.15.49:0/4175207946 conn(0x55965c1d7bf0 msgr2=0x55965c22cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:50.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.392+0000 7f6c2d7fa640 0 client.5565 ms_handle_reset on v2:172.21.15.49:6834/1547192091 2023-11-06T02:09:50.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.392+0000 7f6c533ec000 1 -- 172.21.15.49:0/4175207946 shutdown_connections 2023-11-06T02:09:50.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.392+0000 7f6c533ec000 1 -- 172.21.15.49:0/4175207946 wait complete. 2023-11-06T02:09:50.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.392+0000 7f6c533ec000 20 client.5565 trim_cache size 0 max 16384 2023-11-06T02:09:50.415 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-11-06T02:09:50.415 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_fsid": "1a8b035c-6ba4-499a-b876-aea47fd9c0d4", 2023-11-06T02:09:50.415 INFO:teuthology.orchestra.run.smithi049.stdout: "whoami": 0, 2023-11-06T02:09:50.417 INFO:teuthology.orchestra.run.smithi049.stdout: "id": 5041, 2023-11-06T02:09:50.417 INFO:teuthology.orchestra.run.smithi049.stdout: "want_state": "up:active", 2023-11-06T02:09:50.417 INFO:teuthology.orchestra.run.smithi049.stdout: "state": "up:active", 2023-11-06T02:09:50.418 INFO:teuthology.orchestra.run.smithi049.stdout: "fs_name": "cephfs", 2023-11-06T02:09:50.418 INFO:teuthology.orchestra.run.smithi049.stdout: "rank_uptime": 10.516906737999999, 2023-11-06T02:09:50.418 INFO:teuthology.orchestra.run.smithi049.stdout: "mdsmap_epoch": 15, 2023-11-06T02:09:50.418 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch": 37, 2023-11-06T02:09:50.418 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch_barrier": 37, 2023-11-06T02:09:50.419 INFO:teuthology.orchestra.run.smithi049.stdout: "uptime": 17.790784460000001 2023-11-06T02:09:50.419 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-11-06T02:09:50.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.392+0000 7f6c502c4640 1 -- 172.21.15.49:0/1234069245 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6c34065a90 msgr2=0x7f6c34067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:50.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.392+0000 7f6c502c4640 1 --2- 172.21.15.49:0/1234069245 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6c34065a90 0x7f6c34067f50 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6c480c90c0 tx=0x7f6c4000b040 comp rx=0 tx=0).stop 2023-11-06T02:09:50.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.392+0000 7f6c502c4640 1 -- 172.21.15.49:0/1234069245 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c480cb010 msgr2=0x7f6c48151ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:50.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.392+0000 7f6c502c4640 1 --2- 172.21.15.49:0/1234069245 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c480cb010 0x7f6c48151ee0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c00c420 tx=0x7f6c3c00c8e0 comp rx=0 tx=0).stop 2023-11-06T02:09:50.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.392+0000 7f6c502c4640 1 -- 172.21.15.49:0/1234069245 shutdown_connections 2023-11-06T02:09:50.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.392+0000 7f6c502c4640 1 --2- 172.21.15.49:0/1234069245 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6c34065a90 0x7f6c34067f50 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:50.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.392+0000 7f6c502c4640 1 --2- 172.21.15.49:0/1234069245 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c480cb010 0x7f6c48151ee0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:50.422 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.392+0000 7f6c502c4640 1 --2- 172.21.15.49:0/1234069245 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6c4815ab50 0x7f6c48152420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:50.422 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.392+0000 7f6c502c4640 1 --2- 172.21.15.49:0/1234069245 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6c480b0790 0x7f6c48149b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:50.422 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.392+0000 7f6c502c4640 1 -- 172.21.15.49:0/1234069245 >> 172.21.15.49:0/1234069245 conn(0x7f6c4806c0a0 msgr2=0x7f6c480c9bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:50.422 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.392+0000 7f6c502c4640 1 -- 172.21.15.49:0/1234069245 shutdown_connections 2023-11-06T02:09:50.422 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.392+0000 7f6c502c4640 1 -- 172.21.15.49:0/1234069245 wait complete. 2023-11-06T02:09:50.423 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-06T02:09:50.537 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.536+0000 7f40c0c7c640 1 Processor -- start 2023-11-06T02:09:50.537 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.536+0000 7f40c0c7c640 1 -- start start 2023-11-06T02:09:50.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.540+0000 7f40c0c7c640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f40bc0cb010 0x7f40bc0c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:50.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.540+0000 7f40c0c7c640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40bc0cab80 con 0x7f40bc15ab50 2023-11-06T02:09:50.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.540+0000 7f40c0c7c640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40bc15af30 con 0x7f40bc0b0790 2023-11-06T02:09:50.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.540+0000 7f40c0c7c640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40bc0cb5b0 con 0x7f40bc0cb010 2023-11-06T02:09:50.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.540+0000 7f40ba59c640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f40bc15ab50 0x7f40bc0b0080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:49380/0 (socket says 172.21.15.49:49380) 2023-11-06T02:09:50.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.540+0000 7f40ba59c640 1 -- 172.21.15.49:0/379884842 learned_addr learned my addr 172.21.15.49:0/379884842 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:09:50.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.540+0000 7f40bad9d640 1 --2- 172.21.15.49:0/379884842 >> v2:172.21.15.150:3300/0 conn(0x7f40bc0cb010 0x7f40bc0c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:50.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.540+0000 7f40b959a640 1 -- 172.21.15.49:0/379884842 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2546131743 0 0) 0x7f40bc0cab80 con 0x7f40bc15ab50 2023-11-06T02:09:50.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.540+0000 7f40b959a640 1 -- 172.21.15.49:0/379884842 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40a4003530 con 0x7f40bc15ab50 2023-11-06T02:09:50.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.540+0000 7f40b959a640 1 -- 172.21.15.49:0/379884842 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3115036284 0 0) 0x7f40a4003530 con 0x7f40bc15ab50 2023-11-06T02:09:50.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.540+0000 7f40b959a640 1 -- 172.21.15.49:0/379884842 >> v2:172.21.15.150:3300/0 conn(0x7f40bc0cb010 msgr2=0x7f40bc0c9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:50.542 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.540+0000 7f40b959a640 1 --2- 172.21.15.49:0/379884842 >> v2:172.21.15.150:3300/0 conn(0x7f40bc0cb010 0x7f40bc0c9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:50.542 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.540+0000 7f40b959a640 1 -- 172.21.15.49:0/379884842 >> v1:172.21.15.150:6789/0 conn(0x7f40bc0b0790 legacy=0x7f40bc0c2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:50.542 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.540+0000 7f40b959a640 1 -- 172.21.15.49:0/379884842 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40bc0cb890 con 0x7f40bc15ab50 2023-11-06T02:09:50.542 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.540+0000 7f40b959a640 1 -- 172.21.15.49:0/379884842 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7f40ac001800 con 0x7f40bc15ab50 2023-11-06T02:09:50.542 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.540+0000 7f40b959a640 1 -- 172.21.15.49:0/379884842 >> v1:172.21.15.49:6789/0 conn(0x7f40bc15ab50 legacy=0x7f40bc0b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:50.543 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.540+0000 7f40b959a640 1 --2- 172.21.15.49:0/379884842 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f40a4003980 0x7f40a4003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:50.543 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.540+0000 7f40b959a640 1 --2- 172.21.15.49:0/379884842 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f40a40047d0 0x7f40a4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:50.543 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.540+0000 7f40b959a640 1 --2- 172.21.15.49:0/379884842 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f40a4005190 0x7f40a400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:50.543 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.540+0000 7f40b959a640 1 -- 172.21.15.49:0/379884842 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40bc0cb5b0 con 0x7f40a4005190 2023-11-06T02:09:50.543 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.540+0000 7f40b959a640 1 -- 172.21.15.49:0/379884842 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a400c030 con 0x7f40a4003980 2023-11-06T02:09:50.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.540+0000 7f40b959a640 1 -- 172.21.15.49:0/379884842 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40a400c1f0 con 0x7f40a40047d0 2023-11-06T02:09:50.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.540+0000 7f40bad9d640 1 --2- 172.21.15.49:0/379884842 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f40a4005190 0x7f40a400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:50.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.540+0000 7f40ba59c640 1 --2- 172.21.15.49:0/379884842 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f40a4003980 0x7f40a4003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:50.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.540+0000 7f40b9d9b640 1 --2- 172.21.15.49:0/379884842 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f40a40047d0 0x7f40a4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:50.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.540+0000 7f40bad9d640 1 -- 172.21.15.49:0/379884842 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f40a40047d0 msgr2=0x7f40a4004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:50.545 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.540+0000 7f40bad9d640 1 --2- 172.21.15.49:0/379884842 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f40a40047d0 0x7f40a4004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:50.545 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.540+0000 7f40bad9d640 1 -- 172.21.15.49:0/379884842 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f40a4003980 msgr2=0x7f40a4003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:50.545 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.540+0000 7f40bad9d640 1 --2- 172.21.15.49:0/379884842 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f40a4003980 0x7f40a4003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:50.545 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.540+0000 7f40bad9d640 1 -- 172.21.15.49:0/379884842 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40a4003530 con 0x7f40a4005190 2023-11-06T02:09:50.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.544+0000 7f40bad9d640 1 -- 172.21.15.49:0/379884842 reap_dead start 2023-11-06T02:09:50.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.544+0000 7f40bad9d640 1 --2- 172.21.15.49:0/379884842 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f40a4005190 0x7f40a400b990 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f40b4009ae0 tx=0x7f40b401ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cef02908b6804c8 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:50.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.544+0000 7f40b959a640 1 -- 172.21.15.49:0/379884842 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b4004020 con 0x7f40a4005190 2023-11-06T02:09:50.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.544+0000 7f40b959a640 1 -- 172.21.15.49:0/379884842 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40b40041c0 con 0x7f40a4005190 2023-11-06T02:09:50.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.544+0000 7f40b959a640 1 -- 172.21.15.49:0/379884842 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b40a5680 con 0x7f40a4005190 2023-11-06T02:09:50.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.544+0000 7f40c0c7c640 1 -- 172.21.15.49:0/379884842 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f40a4005190 msgr2=0x7f40a400b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:50.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.544+0000 7f40c0c7c640 1 --2- 172.21.15.49:0/379884842 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f40a4005190 0x7f40a400b990 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f40b4009ae0 tx=0x7f40b401ca90 comp rx=0 tx=0).stop 2023-11-06T02:09:50.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.544+0000 7f40c0c7c640 1 -- 172.21.15.49:0/379884842 shutdown_connections 2023-11-06T02:09:50.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.544+0000 7f40c0c7c640 1 --2- 172.21.15.49:0/379884842 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f40a4005190 0x7f40a400b990 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:50.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.544+0000 7f40c0c7c640 1 -- 172.21.15.49:0/379884842 >> 172.21.15.49:0/379884842 conn(0x7f40bc06c0a0 msgr2=0x7f40bc06c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:50.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.544+0000 7f40c0c7c640 1 -- 172.21.15.49:0/379884842 shutdown_connections 2023-11-06T02:09:50.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.544+0000 7f40c0c7c640 1 -- 172.21.15.49:0/379884842 wait complete. 2023-11-06T02:09:50.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.544+0000 7f40c0c7c640 1 Processor -- start 2023-11-06T02:09:50.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.544+0000 7f40c0c7c640 1 -- start start 2023-11-06T02:09:50.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.544+0000 7f40c0c7c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f40a4005190 0x7f40bc155530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:50.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.544+0000 7f40c0c7c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f40bc155a70 0x7f40bc0c79f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:50.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.544+0000 7f40c0c7c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f40bc0c7f30 0x7f40bc0c6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:50.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.544+0000 7f40c0c7c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40bc0c6730 con 0x7f40bc0c7f30 2023-11-06T02:09:50.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.544+0000 7f40c0c7c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40bc0c5b20 con 0x7f40bc155a70 2023-11-06T02:09:50.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.544+0000 7f40c0c7c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40bc0c5e00 con 0x7f40a4005190 2023-11-06T02:09:50.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.544+0000 7f40bad9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f40bc0c7f30 0x7f40bc0c6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:50.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.544+0000 7f40bad9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f40bc0c7f30 0x7f40bc0c6bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44674/0 (socket says 172.21.15.49:44674) 2023-11-06T02:09:50.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.544+0000 7f40bad9d640 1 -- 172.21.15.49:0/4086204741 learned_addr learned my addr 172.21.15.49:0/4086204741 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:50.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.544+0000 7f40ba59c640 1 --2- 172.21.15.49:0/4086204741 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f40a4005190 0x7f40bc155530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:50.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.544+0000 7f40b9d9b640 1 --2- 172.21.15.49:0/4086204741 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f40bc155a70 0x7f40bc0c79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:50.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.544+0000 7f40bad9d640 1 -- 172.21.15.49:0/4086204741 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f40a4005190 msgr2=0x7f40bc155530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:50.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.544+0000 7f40bad9d640 1 --2- 172.21.15.49:0/4086204741 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f40a4005190 0x7f40bc155530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:50.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.544+0000 7f40bad9d640 1 -- 172.21.15.49:0/4086204741 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f40bc155a70 msgr2=0x7f40bc0c79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:50.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.544+0000 7f40bad9d640 1 --2- 172.21.15.49:0/4086204741 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f40bc155a70 0x7f40bc0c79f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:50.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.544+0000 7f40bad9d640 1 -- 172.21.15.49:0/4086204741 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40bc0c60e0 con 0x7f40bc0c7f30 2023-11-06T02:09:50.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.544+0000 7f40b9d9b640 1 --2- 172.21.15.49:0/4086204741 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f40bc155a70 0x7f40bc0c79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:09:50.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.544+0000 7f40bad9d640 1 --2- 172.21.15.49:0/4086204741 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f40bc0c7f30 0x7f40bc0c6bc0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f40b401c8c0 tx=0x7f40a4005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:50.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.548+0000 7f40aaffd640 1 -- 172.21.15.49:0/4086204741 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b4004330 con 0x7f40bc0c7f30 2023-11-06T02:09:50.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.548+0000 7f40aaffd640 1 -- 172.21.15.49:0/4086204741 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40b4004490 con 0x7f40bc0c7f30 2023-11-06T02:09:50.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.548+0000 7f40c0c7c640 1 -- 172.21.15.49:0/4086204741 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40bc0c63c0 con 0x7f40bc0c7f30 2023-11-06T02:09:50.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.548+0000 7f40aaffd640 1 -- 172.21.15.49:0/4086204741 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b4003840 con 0x7f40bc0c7f30 2023-11-06T02:09:50.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.548+0000 7f40c0c7c640 1 -- 172.21.15.49:0/4086204741 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40bc0c58a0 con 0x7f40bc0c7f30 2023-11-06T02:09:50.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.548+0000 7f40c0c7c640 1 -- 172.21.15.49:0/4086204741 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40bc0b0de0 con 0x7f40bc0c7f30 2023-11-06T02:09:50.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.548+0000 7f40aaffd640 1 -- 172.21.15.49:0/4086204741 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f40b40039a0 con 0x7f40bc0c7f30 2023-11-06T02:09:50.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.548+0000 7f40aaffd640 1 --2- 172.21.15.49:0/4086204741 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f409c065a90 0x7f409c067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:50.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.548+0000 7f40aaffd640 1 -- 172.21.15.49:0/4086204741 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f40b411cbf0 con 0x7f40bc0c7f30 2023-11-06T02:09:50.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.548+0000 7f40ba59c640 1 --2- 172.21.15.49:0/4086204741 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f409c065a90 0x7f409c067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:50.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.548+0000 7f40ba59c640 1 --2- 172.21.15.49:0/4086204741 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f409c065a90 0x7f409c067f50 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f40ac0011e0 tx=0x7f40ac009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:50.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.552+0000 7f40aaffd640 1 -- 172.21.15.49:0/4086204741 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f40b40ea470 con 0x7f40bc0c7f30 2023-11-06T02:09:50.744 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.744+0000 7f40c0c7c640 1 -- 172.21.15.49:0/4086204741 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f40bc074040 con 0x7f40bc0c7f30 2023-11-06T02:09:50.745 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.744+0000 7f40aaffd640 1 -- 172.21.15.49:0/4086204741 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v15) v1 ==== 53+0+73 (secure 0 0 0) 0x7f40b40b2020 con 0x7f40bc0c7f30 2023-11-06T02:09:50.745 INFO:teuthology.orchestra.run.smithi049.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-06T02:09:50.747 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.748+0000 7f40c0c7c640 1 -- 172.21.15.49:0/4086204741 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f409c065a90 msgr2=0x7f409c067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:50.747 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.748+0000 7f40c0c7c640 1 --2- 172.21.15.49:0/4086204741 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f409c065a90 0x7f409c067f50 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f40ac0011e0 tx=0x7f40ac009040 comp rx=0 tx=0).stop 2023-11-06T02:09:50.748 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.748+0000 7f40c0c7c640 1 -- 172.21.15.49:0/4086204741 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f40bc0c7f30 msgr2=0x7f40bc0c6bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:50.748 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.748+0000 7f40c0c7c640 1 --2- 172.21.15.49:0/4086204741 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f40bc0c7f30 0x7f40bc0c6bc0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f40b401c8c0 tx=0x7f40a4005800 comp rx=0 tx=0).stop 2023-11-06T02:09:50.748 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.748+0000 7f40c0c7c640 1 -- 172.21.15.49:0/4086204741 shutdown_connections 2023-11-06T02:09:50.748 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.748+0000 7f40c0c7c640 1 --2- 172.21.15.49:0/4086204741 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f409c065a90 0x7f409c067f50 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:50.748 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.748+0000 7f40c0c7c640 1 --2- 172.21.15.49:0/4086204741 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f40bc0c7f30 0x7f40bc0c6bc0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:50.749 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.748+0000 7f40c0c7c640 1 --2- 172.21.15.49:0/4086204741 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f40bc155a70 0x7f40bc0c79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:50.749 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.748+0000 7f40c0c7c640 1 --2- 172.21.15.49:0/4086204741 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f40a4005190 0x7f40bc155530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:50.749 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.748+0000 7f40c0c7c640 1 -- 172.21.15.49:0/4086204741 >> 172.21.15.49:0/4086204741 conn(0x7f40bc06c0a0 msgr2=0x7f40bc1569a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:50.749 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.748+0000 7f40c0c7c640 1 -- 172.21.15.49:0/4086204741 shutdown_connections 2023-11-06T02:09:50.749 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:50.748+0000 7f40c0c7c640 1 -- 172.21.15.49:0/4086204741 wait complete. 2023-11-06T02:09:50.764 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-06T02:09:50.764 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-06T02:09:50.764 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi049.front.sepia.ceph.com 2023-11-06T02:09:50.765 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-06T02:09:50.765 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-06T02:09:50.765 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-06T02:09:50.766 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-06T02:09:50.767 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-06T02:09:50.767 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-06T02:09:50.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:09:50.768 DEBUG:teuthology.orchestra.run.smithi049:> ip addr 2023-11-06T02:09:50.774 INFO:teuthology.orchestra.run.smithi049.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-06T02:09:50.775 INFO:teuthology.orchestra.run.smithi049.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-06T02:09:50.775 INFO:teuthology.orchestra.run.smithi049.stdout: inet 127.0.0.1/8 scope host lo 2023-11-06T02:09:50.775 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:09:50.775 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 ::1/128 scope host 2023-11-06T02:09:50.775 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:09:50.776 INFO:teuthology.orchestra.run.smithi049.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-06T02:09:50.776 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:6b:ff:84 brd ff:ff:ff:ff:ff:ff 2023-11-06T02:09:50.776 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp7s0f0 2023-11-06T02:09:50.776 INFO:teuthology.orchestra.run.smithi049.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-06T02:09:50.776 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:6b:ff:85 brd ff:ff:ff:ff:ff:ff 2023-11-06T02:09:50.777 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp7s0f1 2023-11-06T02:09:50.777 INFO:teuthology.orchestra.run.smithi049.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-06T02:09:50.777 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:bd:15:66 brd ff:ff:ff:ff:ff:ff 2023-11-06T02:09:50.777 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp1s0f0 2023-11-06T02:09:50.778 INFO:teuthology.orchestra.run.smithi049.stdout: inet 172.21.15.49/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-11-06T02:09:50.778 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft 42406sec preferred_lft 42406sec 2023-11-06T02:09:50.778 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 fe80::ec4:7aff:febd:1566/64 scope link 2023-11-06T02:09:50.778 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:09:50.778 INFO:teuthology.orchestra.run.smithi049.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-06T02:09:50.779 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:bd:15:67 brd ff:ff:ff:ff:ff:ff 2023-11-06T02:09:50.779 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp1s0f1 2023-11-06T02:09:50.779 INFO:teuthology.orchestra.run.smithi049.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-06T02:09:50.779 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 02:42:47:49:33:69 brd ff:ff:ff:ff:ff:ff 2023-11-06T02:09:50.779 INFO:teuthology.orchestra.run.smithi049.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-06T02:09:50.780 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:09:50.780 INFO:teuthology.orchestra.run.smithi049.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-06T02:09:50.780 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-11-06T02:09:50.780 INFO:teuthology.orchestra.run.smithi049.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-06T02:09:50.780 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:09:50.781 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-11-06T02:09:50.781 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:09:50.781 INFO:teuthology.orchestra.run.smithi049.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-06T02:09:50.781 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-06T02:09:50.781 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-11-06T02:09:50.782 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:09:50.782 INFO:teuthology.orchestra.run.smithi049.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-06T02:09:50.782 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-06T02:09:50.782 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-11-06T02:09:50.782 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:09:50.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:09:50.783 DEBUG:teuthology.orchestra.run.smithi049:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-06T02:09:50.827 INFO:teuthology.orchestra.run.smithi049.stdout:1 2023-11-06T02:09:50.830 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:09:50.830 DEBUG:teuthology.orchestra.run.smithi049:> route 2023-11-06T02:09:50.879 INFO:teuthology.orchestra.run.smithi049.stdout:Kernel IP routing table 2023-11-06T02:09:50.879 INFO:teuthology.orchestra.run.smithi049.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-06T02:09:50.880 INFO:teuthology.orchestra.run.smithi049.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-11-06T02:09:50.880 INFO:teuthology.orchestra.run.smithi049.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-06T02:09:50.880 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-11-06T02:09:50.880 INFO:teuthology.orchestra.run.smithi049.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-06T02:09:50.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:09:50.881 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec bash -c ' 2023-11-06T02:09:50.882 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2023-11-06T02:09:50.882 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-06T02:09:50.882 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-06T02:09:50.883 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-06T02:09:50.883 DEBUG:teuthology.orchestra.run.smithi049:> ') 2023-11-06T02:09:50.954 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:09:50.955 DEBUG:teuthology.orchestra.run.smithi049:> ip netns list 2023-11-06T02:09:51.003 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-06T02:09:51.003 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-06T02:09:51.003 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-06T02:09:51.004 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:09:51.004 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link set brx.0 up 2023-11-06T02:09:51.059 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-06T02:09:51.059 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-06T02:09:51.059 DEBUG:teuthology.orchestra.run.smithi049:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-06T02:09:51.110 INFO:teuthology.orchestra.run.smithi049.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-06T02:09:51.111 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-06T02:09:51.111 DEBUG:teuthology.orchestra.run.smithi049:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-06T02:09:51.158 DEBUG:teuthology.orchestra.run.smithi049:> sudo modprobe fuse 2023-11-06T02:09:51.215 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-06T02:09:51.216 DEBUG:teuthology.orchestra.run.smithi049:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-06T02:09:51.268 INFO:teuthology.orchestra.run.smithi049.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-06T02:09:51.269 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-06T02:09:51.270 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:09:51.270 DEBUG:teuthology.orchestra.run.smithi049:> ls /sys/fs/fuse/connections 2023-11-06T02:09:51.318 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-06T02:09:51.318 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-06T02:09:51.367 DEBUG:teuthology.orchestra.run.smithi049:> sudo modprobe fuse 2023-11-06T02:09:51.382 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-06T02:09:51.383 DEBUG:teuthology.orchestra.run.smithi049:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-06T02:09:51.439 INFO:teuthology.orchestra.run.smithi049.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-06T02:09:51.440 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-06T02:09:51.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:09:51.441 DEBUG:teuthology.orchestra.run.smithi049:> ls /sys/fs/fuse/connections 2023-11-06T02:09:51.452 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi049.stderr:2023-11-06T02:09:51.452+0000 7efc3124a580 -1 init, newargv = 0x55def9ee23b0 newargc=15 2023-11-06T02:09:51.452 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi049.stderr:ceph-fuse[48364]: starting ceph client 2023-11-06T02:09:51.458 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi049.stderr:ceph-fuse[48364]: starting fuse 2023-11-06T02:09:51.566 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T02:09:51.568 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T02:09:52.558 DEBUG:teuthology.orchestra.run.smithi049:> sudo modprobe fuse 2023-11-06T02:09:52.575 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-06T02:09:52.576 DEBUG:teuthology.orchestra.run.smithi049:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-06T02:09:52.630 INFO:teuthology.orchestra.run.smithi049.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-06T02:09:52.631 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-06T02:09:52.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:09:52.632 DEBUG:teuthology.orchestra.run.smithi049:> ls /sys/fs/fuse/connections 2023-11-06T02:09:52.678 INFO:teuthology.orchestra.run.smithi049.stdout:45 2023-11-06T02:09:52.679 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2023-11-06T02:09:52.679 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-11-06T02:09:52.680 DEBUG:teuthology.orchestra.run.smithi049:> import glob 2023-11-06T02:09:52.680 DEBUG:teuthology.orchestra.run.smithi049:> import re 2023-11-06T02:09:52.680 DEBUG:teuthology.orchestra.run.smithi049:> import os 2023-11-06T02:09:52.681 DEBUG:teuthology.orchestra.run.smithi049:> import subprocess 2023-11-06T02:09:52.681 DEBUG:teuthology.orchestra.run.smithi049:> 2023-11-06T02:09:52.681 DEBUG:teuthology.orchestra.run.smithi049:> def _find_admin_socket(client_name): 2023-11-06T02:09:52.682 DEBUG:teuthology.orchestra.run.smithi049:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-06T02:09:52.682 DEBUG:teuthology.orchestra.run.smithi049:> files = glob.glob(asok_path) 2023-11-06T02:09:52.683 DEBUG:teuthology.orchestra.run.smithi049:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-06T02:09:52.683 DEBUG:teuthology.orchestra.run.smithi049:> 2023-11-06T02:09:52.684 DEBUG:teuthology.orchestra.run.smithi049:> # Given a non-glob path, it better be there 2023-11-06T02:09:52.684 DEBUG:teuthology.orchestra.run.smithi049:> if "*" not in asok_path: 2023-11-06T02:09:52.684 DEBUG:teuthology.orchestra.run.smithi049:> assert(len(files) == 1) 2023-11-06T02:09:52.685 DEBUG:teuthology.orchestra.run.smithi049:> return files[0] 2023-11-06T02:09:52.685 DEBUG:teuthology.orchestra.run.smithi049:> 2023-11-06T02:09:52.685 DEBUG:teuthology.orchestra.run.smithi049:> for f in files: 2023-11-06T02:09:52.686 DEBUG:teuthology.orchestra.run.smithi049:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-06T02:09:52.686 DEBUG:teuthology.orchestra.run.smithi049:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-06T02:09:52.686 DEBUG:teuthology.orchestra.run.smithi049:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-06T02:09:52.687 DEBUG:teuthology.orchestra.run.smithi049:> contents = proc_f.read() 2023-11-06T02:09:52.687 DEBUG:teuthology.orchestra.run.smithi049:> if mountpoint in contents: 2023-11-06T02:09:52.688 DEBUG:teuthology.orchestra.run.smithi049:> return f 2023-11-06T02:09:52.688 DEBUG:teuthology.orchestra.run.smithi049:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-06T02:09:52.688 DEBUG:teuthology.orchestra.run.smithi049:> 2023-11-06T02:09:52.689 DEBUG:teuthology.orchestra.run.smithi049:> print(_find_admin_socket("client.0")) 2023-11-06T02:09:52.689 DEBUG:teuthology.orchestra.run.smithi049:> ' 2023-11-06T02:09:52.802 INFO:teuthology.orchestra.run.smithi049.stdout:/var/run/ceph/ceph-client.0.48364.asok 2023-11-06T02:09:52.983 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.48364.asok 2023-11-06T02:09:52.984 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:09:52.984 DEBUG:teuthology.orchestra.run.smithi049:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.48364.asok status 2023-11-06T02:09:53.141 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-11-06T02:09:53.141 INFO:teuthology.orchestra.run.smithi049.stdout: "metadata": { 2023-11-06T02:09:53.141 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_sha1": "f7e9f9af51acf98e0ccc3037487018e4ae40caa5", 2023-11-06T02:09:53.142 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_version": "ceph version 18.2.0-1181-gf7e9f9af (f7e9f9af51acf98e0ccc3037487018e4ae40caa5) reef (stable)", 2023-11-06T02:09:53.142 INFO:teuthology.orchestra.run.smithi049.stdout: "entity_id": "0", 2023-11-06T02:09:53.142 INFO:teuthology.orchestra.run.smithi049.stdout: "hostname": "smithi049", 2023-11-06T02:09:53.142 INFO:teuthology.orchestra.run.smithi049.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-06T02:09:53.143 INFO:teuthology.orchestra.run.smithi049.stdout: "pid": "48364", 2023-11-06T02:09:53.143 INFO:teuthology.orchestra.run.smithi049.stdout: "root": "/" 2023-11-06T02:09:53.143 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:09:53.143 INFO:teuthology.orchestra.run.smithi049.stdout: "dentry_count": 0, 2023-11-06T02:09:53.144 INFO:teuthology.orchestra.run.smithi049.stdout: "dentry_pinned_count": 0, 2023-11-06T02:09:53.144 INFO:teuthology.orchestra.run.smithi049.stdout: "id": 5577, 2023-11-06T02:09:53.144 INFO:teuthology.orchestra.run.smithi049.stdout: "inst": { 2023-11-06T02:09:53.144 INFO:teuthology.orchestra.run.smithi049.stdout: "name": { 2023-11-06T02:09:53.144 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "client", 2023-11-06T02:09:53.144 INFO:teuthology.orchestra.run.smithi049.stdout: "num": 5577 2023-11-06T02:09:53.145 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:09:53.145 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": { 2023-11-06T02:09:53.145 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2023-11-06T02:09:53.145 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "192.168.0.1:0", 2023-11-06T02:09:53.145 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 4210553337 2023-11-06T02:09:53.146 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:09:53.146 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:09:53.146 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": { 2023-11-06T02:09:53.146 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2023-11-06T02:09:53.147 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "192.168.0.1:0", 2023-11-06T02:09:53.147 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 4210553337 2023-11-06T02:09:53.147 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:09:53.147 INFO:teuthology.orchestra.run.smithi049.stdout: "inst_str": "client.5577 192.168.0.1:0/4210553337", 2023-11-06T02:09:53.147 INFO:teuthology.orchestra.run.smithi049.stdout: "addr_str": "192.168.0.1:0/4210553337", 2023-11-06T02:09:53.148 INFO:teuthology.orchestra.run.smithi049.stdout: "inode_count": 1, 2023-11-06T02:09:53.148 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_epoch": 15, 2023-11-06T02:09:53.148 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_epoch": 37, 2023-11-06T02:09:53.148 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_epoch_barrier": 0, 2023-11-06T02:09:53.149 INFO:teuthology.orchestra.run.smithi049.stdout: "blocklisted": false, 2023-11-06T02:09:53.149 INFO:teuthology.orchestra.run.smithi049.stdout: "fs_name": "cephfs" 2023-11-06T02:09:53.149 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-11-06T02:09:53.155 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:09:53.155 DEBUG:teuthology.orchestra.run.smithi049:> stat --file-system '--printf=%T 2023-11-06T02:09:53.156 DEBUG:teuthology.orchestra.run.smithi049:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-06T02:09:53.162 INFO:teuthology.orchestra.run.smithi049.stdout:fuseblk 2023-11-06T02:09:53.163 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-06T02:09:53.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:09:53.164 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-06T02:09:53.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:09:53.220 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parentdir) 2023-11-06T02:09:53.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:09:53.276 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parentdir/childdir) 2023-11-06T02:09:53.328 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:09:53.328 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch rfile) 2023-11-06T02:09:53.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:09:53.380 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parentdir/pfile) 2023-11-06T02:09:53.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:09:53.441 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parentdir/childdir/cfile) 2023-11-06T02:09:53.490 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:09:53.491 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2023-11-06T02:09:53.554 INFO:teuthology.orchestra.run.smithi049.stdout:./ 2023-11-06T02:09:53.555 INFO:teuthology.orchestra.run.smithi049.stdout:./rfile 2023-11-06T02:09:53.555 INFO:teuthology.orchestra.run.smithi049.stdout:./parentdir 2023-11-06T02:09:53.555 INFO:teuthology.orchestra.run.smithi049.stdout:./parentdir/pfile 2023-11-06T02:09:53.555 INFO:teuthology.orchestra.run.smithi049.stdout:./parentdir/childdir 2023-11-06T02:09:53.556 INFO:teuthology.orchestra.run.smithi049.stdout:./parentdir/childdir/cfile 2023-11-06T02:09:53.557 DEBUG:teuthology.orchestra.run.smithi049:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-06T02:09:53.557 DEBUG:teuthology.orchestra.run.smithi049:> import os 2023-11-06T02:09:53.557 DEBUG:teuthology.orchestra.run.smithi049:> import stat 2023-11-06T02:09:53.558 DEBUG:teuthology.orchestra.run.smithi049:> 2023-11-06T02:09:53.558 DEBUG:teuthology.orchestra.run.smithi049:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2023-11-06T02:09:53.559 DEBUG:teuthology.orchestra.run.smithi049:> ' 2023-11-06T02:09:53.666 INFO:teuthology.orchestra.run.smithi049.stdout:1 2023-11-06T02:09:53.855 DEBUG:teuthology.orchestra.run.smithi049:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-06T02:09:53.855 DEBUG:teuthology.orchestra.run.smithi049:> import os 2023-11-06T02:09:53.855 DEBUG:teuthology.orchestra.run.smithi049:> import stat 2023-11-06T02:09:53.856 DEBUG:teuthology.orchestra.run.smithi049:> 2023-11-06T02:09:53.856 DEBUG:teuthology.orchestra.run.smithi049:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./rfile").st_ino) 2023-11-06T02:09:53.856 DEBUG:teuthology.orchestra.run.smithi049:> ' 2023-11-06T02:09:53.964 INFO:teuthology.orchestra.run.smithi049.stdout:1099511627778 2023-11-06T02:09:54.151 DEBUG:teuthology.orchestra.run.smithi049:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-06T02:09:54.152 DEBUG:teuthology.orchestra.run.smithi049:> import os 2023-11-06T02:09:54.152 DEBUG:teuthology.orchestra.run.smithi049:> import stat 2023-11-06T02:09:54.152 DEBUG:teuthology.orchestra.run.smithi049:> 2023-11-06T02:09:54.153 DEBUG:teuthology.orchestra.run.smithi049:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir").st_ino) 2023-11-06T02:09:54.153 DEBUG:teuthology.orchestra.run.smithi049:> ' 2023-11-06T02:09:54.265 INFO:teuthology.orchestra.run.smithi049.stdout:1099511627776 2023-11-06T02:09:54.453 DEBUG:teuthology.orchestra.run.smithi049:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-06T02:09:54.453 DEBUG:teuthology.orchestra.run.smithi049:> import os 2023-11-06T02:09:54.454 DEBUG:teuthology.orchestra.run.smithi049:> import stat 2023-11-06T02:09:54.454 DEBUG:teuthology.orchestra.run.smithi049:> 2023-11-06T02:09:54.454 DEBUG:teuthology.orchestra.run.smithi049:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/pfile").st_ino) 2023-11-06T02:09:54.455 DEBUG:teuthology.orchestra.run.smithi049:> ' 2023-11-06T02:09:54.561 INFO:teuthology.orchestra.run.smithi049.stdout:1099511628279 2023-11-06T02:09:54.748 DEBUG:teuthology.orchestra.run.smithi049:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-06T02:09:54.749 DEBUG:teuthology.orchestra.run.smithi049:> import os 2023-11-06T02:09:54.749 DEBUG:teuthology.orchestra.run.smithi049:> import stat 2023-11-06T02:09:54.750 DEBUG:teuthology.orchestra.run.smithi049:> 2023-11-06T02:09:54.750 DEBUG:teuthology.orchestra.run.smithi049:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/childdir").st_ino) 2023-11-06T02:09:54.750 DEBUG:teuthology.orchestra.run.smithi049:> ' 2023-11-06T02:09:54.853 INFO:teuthology.orchestra.run.smithi049.stdout:1099511627777 2023-11-06T02:09:55.041 DEBUG:teuthology.orchestra.run.smithi049:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-06T02:09:55.042 DEBUG:teuthology.orchestra.run.smithi049:> import os 2023-11-06T02:09:55.042 DEBUG:teuthology.orchestra.run.smithi049:> import stat 2023-11-06T02:09:55.042 DEBUG:teuthology.orchestra.run.smithi049:> 2023-11-06T02:09:55.043 DEBUG:teuthology.orchestra.run.smithi049:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/childdir/cfile").st_ino) 2023-11-06T02:09:55.043 DEBUG:teuthology.orchestra.run.smithi049:> ' 2023-11-06T02:09:55.152 INFO:teuthology.orchestra.run.smithi049.stdout:1099511628280 2023-11-06T02:09:55.339 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-11-06T02:09:55.339 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T02:09:55.387 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T02:09:55.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.568+0000 7f84a3b7e640 1 Processor -- start 2023-11-06T02:09:55.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.568+0000 7f84a3b7e640 1 -- start start 2023-11-06T02:09:55.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.568+0000 7f84a3b7e640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f849c15ab50 0x7f849c0c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:55.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.568+0000 7f84a3b7e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f849c0b0790 0x7f849c0c2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:55.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.568+0000 7f84a3b7e640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f849c0cab80 con 0x7f849c0cb010 2023-11-06T02:09:55.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.568+0000 7f84a3b7e640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f849c15af30 con 0x7f849c15ab50 2023-11-06T02:09:55.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.568+0000 7f84a3b7e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f849c0cb5b0 con 0x7f849c0b0790 2023-11-06T02:09:55.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.568+0000 7f84a1119640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f849c0cb010 0x7f849c0b0080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:49394/0 (socket says 172.21.15.49:49394) 2023-11-06T02:09:55.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.568+0000 7f84a1119640 1 -- 172.21.15.49:0/3718249338 learned_addr learned my addr 172.21.15.49:0/3718249338 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:09:55.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.568+0000 7f84a191a640 1 --2- 172.21.15.49:0/3718249338 >> v2:172.21.15.150:3300/0 conn(0x7f849c15ab50 0x7f849c0c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:55.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.568+0000 7f84a211b640 1 --2- 172.21.15.49:0/3718249338 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f849c0b0790 0x7f849c0c2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:55.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.568+0000 7f84a0918640 1 -- 172.21.15.49:0/3718249338 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2158991621 0 0) 0x7f849c0cab80 con 0x7f849c0cb010 2023-11-06T02:09:55.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.568+0000 7f84a0918640 1 -- 172.21.15.49:0/3718249338 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8488003530 con 0x7f849c0cb010 2023-11-06T02:09:55.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.568+0000 7f84a0918640 1 -- 172.21.15.49:0/3718249338 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1583425430 0 0) 0x7f8488003530 con 0x7f849c0cb010 2023-11-06T02:09:55.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.568+0000 7f84a0918640 1 -- 172.21.15.49:0/3718249338 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f849c0b0790 msgr2=0x7f849c0c2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:55.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.568+0000 7f84a0918640 1 --2- 172.21.15.49:0/3718249338 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f849c0b0790 0x7f849c0c2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:55.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.568+0000 7f84a0918640 1 -- 172.21.15.49:0/3718249338 >> v2:172.21.15.150:3300/0 conn(0x7f849c15ab50 msgr2=0x7f849c0c9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:55.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.568+0000 7f84a0918640 1 --2- 172.21.15.49:0/3718249338 >> v2:172.21.15.150:3300/0 conn(0x7f849c15ab50 0x7f849c0c9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:55.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.568+0000 7f84a0918640 1 -- 172.21.15.49:0/3718249338 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f849c0cb890 con 0x7f849c0cb010 2023-11-06T02:09:55.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.568+0000 7f84a0918640 1 -- 172.21.15.49:0/3718249338 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7f848c0016a0 con 0x7f849c0cb010 2023-11-06T02:09:55.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.568+0000 7f84a0918640 1 -- 172.21.15.49:0/3718249338 >> v1:172.21.15.49:6789/0 conn(0x7f849c0cb010 legacy=0x7f849c0b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:55.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.568+0000 7f84a0918640 1 --2- 172.21.15.49:0/3718249338 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8488003980 0x7f8488003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:55.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.568+0000 7f84a0918640 1 --2- 172.21.15.49:0/3718249338 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f849c0b0790 0x7f84880047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:55.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.568+0000 7f84a0918640 1 --2- 172.21.15.49:0/3718249338 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8488004d10 0x7f8488005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:55.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.568+0000 7f84a0918640 1 -- 172.21.15.49:0/3718249338 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f849c15af30 con 0x7f8488004d10 2023-11-06T02:09:55.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.568+0000 7f84a0918640 1 -- 172.21.15.49:0/3718249338 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f849c0cb5b0 con 0x7f8488003980 2023-11-06T02:09:55.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.568+0000 7f84a0918640 1 -- 172.21.15.49:0/3718249338 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f848800b990 con 0x7f849c0b0790 2023-11-06T02:09:55.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.568+0000 7f84a211b640 1 --2- 172.21.15.49:0/3718249338 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8488004d10 0x7f8488005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:55.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.568+0000 7f84a191a640 1 --2- 172.21.15.49:0/3718249338 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8488003980 0x7f8488003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:55.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.568+0000 7f84a1119640 1 --2- 172.21.15.49:0/3718249338 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f849c0b0790 0x7f84880047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:55.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.568+0000 7f84a211b640 1 -- 172.21.15.49:0/3718249338 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f849c0b0790 msgr2=0x7f84880047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:55.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.568+0000 7f84a211b640 1 --2- 172.21.15.49:0/3718249338 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f849c0b0790 0x7f84880047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:55.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.568+0000 7f84a211b640 1 -- 172.21.15.49:0/3718249338 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8488003980 msgr2=0x7f8488003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:55.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.568+0000 7f84a211b640 1 --2- 172.21.15.49:0/3718249338 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8488003980 0x7f8488003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:55.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.568+0000 7f84a211b640 1 -- 172.21.15.49:0/3718249338 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8488003530 con 0x7f8488004d10 2023-11-06T02:09:55.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.568+0000 7f84a191a640 1 --2- 172.21.15.49:0/3718249338 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8488003980 0x7f8488003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:09:55.581 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.572+0000 7f84a211b640 1 --2- 172.21.15.49:0/3718249338 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8488004d10 0x7f8488005190 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f8494009f90 tx=0x7f849401cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13dee3abccef29c1 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:55.581 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.572+0000 7f84a0918640 1 -- 172.21.15.49:0/3718249338 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84940a5bf0 con 0x7f8488004d10 2023-11-06T02:09:55.581 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.572+0000 7f84a0918640 1 -- 172.21.15.49:0/3718249338 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f849400f040 con 0x7f8488004d10 2023-11-06T02:09:55.582 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.572+0000 7f84a0918640 1 -- 172.21.15.49:0/3718249338 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84940ad440 con 0x7f8488004d10 2023-11-06T02:09:55.582 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.572+0000 7f84a3b7e640 1 -- 172.21.15.49:0/3718249338 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8488004d10 msgr2=0x7f8488005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:55.582 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.572+0000 7f84a3b7e640 1 --2- 172.21.15.49:0/3718249338 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8488004d10 0x7f8488005190 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f8494009f90 tx=0x7f849401cbe0 comp rx=0 tx=0).stop 2023-11-06T02:09:55.582 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.572+0000 7f84a211b640 1 -- 172.21.15.49:0/3718249338 reap_dead start 2023-11-06T02:09:55.582 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.572+0000 7f84a3b7e640 1 -- 172.21.15.49:0/3718249338 shutdown_connections 2023-11-06T02:09:55.583 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.572+0000 7f84a3b7e640 1 -- 172.21.15.49:0/3718249338 >> 172.21.15.49:0/3718249338 conn(0x7f849c06c0a0 msgr2=0x7f849c06c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:55.583 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.572+0000 7f84a3b7e640 1 -- 172.21.15.49:0/3718249338 shutdown_connections 2023-11-06T02:09:55.583 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.572+0000 7f84a3b7e640 1 -- 172.21.15.49:0/3718249338 wait complete. 2023-11-06T02:09:55.583 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.572+0000 7f84a3b7e640 1 Processor -- start 2023-11-06T02:09:55.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.572+0000 7f84a3b7e640 1 -- start start 2023-11-06T02:09:55.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.576+0000 7f84a3b7e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8488004d10 0x7f849c14fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:55.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.576+0000 7f84a3b7e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f849c150be0 0x7f849c0af0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:55.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.576+0000 7f84a3b7e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f849c151060 0x7f849c0af5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:55.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.576+0000 7f84a3b7e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f849c151610 con 0x7f849c150be0 2023-11-06T02:09:55.585 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.576+0000 7f84a3b7e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f849c15c5a0 con 0x7f849c151060 2023-11-06T02:09:55.585 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.576+0000 7f84a3b7e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f849c15c880 con 0x7f8488004d10 2023-11-06T02:09:55.585 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.576+0000 7f84a1119640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f849c150be0 0x7f849c0af0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:55.585 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.576+0000 7f84a1119640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f849c150be0 0x7f849c0af0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44690/0 (socket says 172.21.15.49:44690) 2023-11-06T02:09:55.585 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.576+0000 7f84a1119640 1 -- 172.21.15.49:0/1614310165 learned_addr learned my addr 172.21.15.49:0/1614310165 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:55.586 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.576+0000 7f84a211b640 1 --2- 172.21.15.49:0/1614310165 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f849c151060 0x7f849c0af5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:55.586 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.576+0000 7f84a191a640 1 --2- 172.21.15.49:0/1614310165 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8488004d10 0x7f849c14fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:55.586 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.576+0000 7f84a1119640 1 -- 172.21.15.49:0/1614310165 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8488004d10 msgr2=0x7f849c14fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:55.586 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.576+0000 7f84a1119640 1 --2- 172.21.15.49:0/1614310165 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8488004d10 0x7f849c14fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:55.587 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.576+0000 7f84a1119640 1 -- 172.21.15.49:0/1614310165 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f849c151060 msgr2=0x7f849c0af5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:55.587 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.576+0000 7f84a1119640 1 --2- 172.21.15.49:0/1614310165 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f849c151060 0x7f849c0af5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:55.587 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.576+0000 7f84a1119640 1 -- 172.21.15.49:0/1614310165 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f849c15cb60 con 0x7f849c150be0 2023-11-06T02:09:55.587 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.576+0000 7f84a191a640 1 --2- 172.21.15.49:0/1614310165 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8488004d10 0x7f849c14fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:09:55.587 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.576+0000 7f84a1119640 1 --2- 172.21.15.49:0/1614310165 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f849c150be0 0x7f849c0af0b0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f848c001ec0 tx=0x7f848c00ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:55.588 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.576+0000 7f84927fc640 1 -- 172.21.15.49:0/1614310165 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f848c001bc0 con 0x7f849c150be0 2023-11-06T02:09:55.588 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.576+0000 7f84927fc640 1 -- 172.21.15.49:0/1614310165 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f848c00c8d0 con 0x7f849c150be0 2023-11-06T02:09:55.588 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.576+0000 7f84a3b7e640 1 -- 172.21.15.49:0/1614310165 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f849c15ce40 con 0x7f849c150be0 2023-11-06T02:09:55.588 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.576+0000 7f84927fc640 1 -- 172.21.15.49:0/1614310165 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f848c00cc60 con 0x7f849c150be0 2023-11-06T02:09:55.588 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.576+0000 7f84a3b7e640 1 -- 172.21.15.49:0/1614310165 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f849c156ab0 con 0x7f849c150be0 2023-11-06T02:09:55.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.576+0000 7f84a3b7e640 1 -- 172.21.15.49:0/1614310165 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f849c0b0de0 con 0x7f849c150be0 2023-11-06T02:09:55.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.576+0000 7f84927fc640 1 -- 172.21.15.49:0/1614310165 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f848c01d460 con 0x7f849c150be0 2023-11-06T02:09:55.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.576+0000 7f84927fc640 1 --2- 172.21.15.49:0/1614310165 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8480065a90 0x7f8480067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:55.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.576+0000 7f84927fc640 1 -- 172.21.15.49:0/1614310165 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f848c0833b0 con 0x7f849c150be0 2023-11-06T02:09:55.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.576+0000 7f84a191a640 1 --2- 172.21.15.49:0/1614310165 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8480065a90 0x7f8480067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:55.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.580+0000 7f84a191a640 1 --2- 172.21.15.49:0/1614310165 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8480065a90 0x7f8480067f50 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f849c1f35e0 tx=0x7f8498008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:55.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.584+0000 7f84927fc640 1 -- 172.21.15.49:0/1614310165 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f848c050d60 con 0x7f849c150be0 2023-11-06T02:09:55.738 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.740+0000 7f84a3b7e640 1 -- 172.21.15.49:0/1614310165 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f849c0cbe20 con 0x7f849c150be0 2023-11-06T02:09:55.741 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.744+0000 7f84927fc640 1 -- 172.21.15.49:0/1614310165 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+17883 (secure 0 0 0) 0x7f848c001660 con 0x7f849c150be0 2023-11-06T02:09:55.742 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:09:55.742 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":37,"fsid":"1a8b035c-6ba4-499a-b876-aea47fd9c0d4","created":"2023-11-06T02:08:06.440857+0000","modified":"2023-11-06T02:09:40.876751+0000","last_up_change":"2023-11-06T02:08:17.982372+0000","last_in_change":"2023-11-06T02:08:08.906631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T02:08:19.252222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T02:08:24.776211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-06T02:09:37.906848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-06T02:09:38.849380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ad55af8-0571-4e6a-8722-b5d429b7496d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6809","nonce":38116},{"type":"v1","addr":"172.21.15.49:6811","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6813","nonce":38116},{"type":"v1","addr":"172.21.15.49:6815","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6821","nonce":38116},{"type":"v1","addr":"172.21.15.49:6823","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6817","nonce":38116},{"type":"v1","addr":"172.21.15.49:6819","nonce":38116}]},"public_addr":"172.21.15.49:6811/38116","cluster_addr":"172.21.15.49:6815/38116","heartbeat_back_addr":"172.21.15.49:6823/38116","heartbeat_front_addr":"172.21.15.49:6819/38116","state":["exists","up"]},{"osd":1,"uuid":"c8b31f85-a757-43ed-9419-351acc46611b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":38113},{"type":"v1","addr":"172.21.15.49:6810","nonce":38113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":38113},{"type":"v1","addr":"172.21.15.49:6814","nonce":38113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":38113},{"type":"v1","addr":"172.21.15.49:6822","nonce":38113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":38113},{"type":"v1","addr":"172.21.15.49:6818","nonce":38113}]},"public_addr":"172.21.15.49:6810/38113","cluster_addr":"172.21.15.49:6814/38113","heartbeat_back_addr":"172.21.15.49:6822/38113","heartbeat_front_addr":"172.21.15.49:6818/38113","state":["exists","up"]},{"osd":2,"uuid":"555b3bf3-0a70-4b91-8efd-e4204b5fc26e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":38114},{"type":"v1","addr":"172.21.15.49:6801","nonce":38114}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":38114},{"type":"v1","addr":"172.21.15.49:6803","nonce":38114}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":38114},{"type":"v1","addr":"172.21.15.49:6807","nonce":38114}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":38114},{"type":"v1","addr":"172.21.15.49:6805","nonce":38114}]},"public_addr":"172.21.15.49:6801/38114","cluster_addr":"172.21.15.49:6803/38114","heartbeat_back_addr":"172.21.15.49:6807/38114","heartbeat_front_addr":"172.21.15.49:6805/38114","state":["exists","up"]},{"osd":3,"uuid":"8d4e5d77-03ef-4b9b-bfcb-5ae93a46795c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":38115},{"type":"v1","addr":"172.21.15.49:6825","nonce":38115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":38115},{"type":"v1","addr":"172.21.15.49:6827","nonce":38115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":38115},{"type":"v1","addr":"172.21.15.49:6831","nonce":38115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":38115},{"type":"v1","addr":"172.21.15.49:6829","nonce":38115}]},"public_addr":"172.21.15.49:6825/38115","cluster_addr":"172.21.15.49:6827/38115","heartbeat_back_addr":"172.21.15.49:6831/38115","heartbeat_front_addr":"172.21.15.49:6829/38115","state":["exists","up"]},{"osd":4,"uuid":"dba03ad4-68a3-4cf3-9218-ac611f02eacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":38417},{"type":"v1","addr":"172.21.15.150:6809","nonce":38417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":38417},{"type":"v1","addr":"172.21.15.150:6811","nonce":38417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":38417},{"type":"v1","addr":"172.21.15.150:6815","nonce":38417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":38417},{"type":"v1","addr":"172.21.15.150:6813","nonce":38417}]},"public_addr":"172.21.15.150:6809/38417","cluster_addr":"172.21.15.150:6811/38417","heartbeat_back_addr":"172.21.15.150:6815/38417","heartbeat_front_addr":"172.21.15.150:6813/38417","state":["exists","up"]},{"osd":5,"uuid":"0df9c303-c255-4640-8e91-01f1c3efe18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":38418},{"type":"v1","addr":"172.21.15.150:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":38418},{"type":"v1","addr":"172.21.15.150:6803","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":38418},{"type":"v1","addr":"172.21.15.150:6807","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":38418},{"type":"v1","addr":"172.21.15.150:6805","nonce":38418}]},"public_addr":"172.21.15.150:6801/38418","cluster_addr":"172.21.15.150:6803/38418","heartbeat_back_addr":"172.21.15.150:6807/38418","heartbeat_front_addr":"172.21.15.150:6805/38418","state":["exists","up"]},{"osd":6,"uuid":"e59fa189-4b70-4c06-9722-c02b90828bf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":38428},{"type":"v1","addr":"172.21.15.150:6825","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":38428},{"type":"v1","addr":"172.21.15.150:6827","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":38428},{"type":"v1","addr":"172.21.15.150:6831","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":38428},{"type":"v1","addr":"172.21.15.150:6829","nonce":38428}]},"public_addr":"172.21.15.150:6825/38428","cluster_addr":"172.21.15.150:6827/38428","heartbeat_back_addr":"172.21.15.150:6831/38428","heartbeat_front_addr":"172.21.15.150:6829/38428","state":["exists","up"]},{"osd":7,"uuid":"a64f57e1-ef98-46bd-b304-385759e95898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":38421},{"type":"v1","addr":"172.21.15.150:6817","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":38421},{"type":"v1","addr":"172.21.15.150:6819","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":38421},{"type":"v1","addr":"172.21.15.150:6823","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":38421},{"type":"v1","addr":"172.21.15.150:6821","nonce":38421}]},"public_addr":"172.21.15.150:6817/38421","cluster_addr":"172.21.15.150:6819/38421","heartbeat_back_addr":"172.21.15.150:6823/38421","heartbeat_front_addr":"172.21.15.150:6821/38421","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.438552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.413280+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.296466+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.424966+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.276953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.230984+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.410235+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.480084+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T02:09:55.744 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.744+0000 7f84a3b7e640 1 -- 172.21.15.49:0/1614310165 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8480065a90 msgr2=0x7f8480067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:55.744 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.744+0000 7f84a3b7e640 1 --2- 172.21.15.49:0/1614310165 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8480065a90 0x7f8480067f50 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f849c1f35e0 tx=0x7f8498008040 comp rx=0 tx=0).stop 2023-11-06T02:09:55.744 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.744+0000 7f84a3b7e640 1 -- 172.21.15.49:0/1614310165 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f849c150be0 msgr2=0x7f849c0af0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:55.744 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.744+0000 7f84a3b7e640 1 --2- 172.21.15.49:0/1614310165 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f849c150be0 0x7f849c0af0b0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f848c001ec0 tx=0x7f848c00ece0 comp rx=0 tx=0).stop 2023-11-06T02:09:55.744 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.744+0000 7f84a3b7e640 1 -- 172.21.15.49:0/1614310165 shutdown_connections 2023-11-06T02:09:55.745 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.744+0000 7f84a3b7e640 1 --2- 172.21.15.49:0/1614310165 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8480065a90 0x7f8480067f50 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:55.745 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.744+0000 7f84a3b7e640 1 --2- 172.21.15.49:0/1614310165 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f849c150be0 0x7f849c0af0b0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:55.745 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.744+0000 7f84a3b7e640 1 --2- 172.21.15.49:0/1614310165 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f849c151060 0x7f849c0af5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:55.745 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.744+0000 7f84a3b7e640 1 --2- 172.21.15.49:0/1614310165 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8488004d10 0x7f849c14fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:55.745 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.744+0000 7f84a3b7e640 1 -- 172.21.15.49:0/1614310165 >> 172.21.15.49:0/1614310165 conn(0x7f849c06c0a0 msgr2=0x7f849c0c3dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:55.746 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.744+0000 7f84a3b7e640 1 -- 172.21.15.49:0/1614310165 shutdown_connections 2023-11-06T02:09:55.746 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.744+0000 7f84a3b7e640 1 -- 172.21.15.49:0/1614310165 wait complete. 2023-11-06T02:09:55.760 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T02:08:19.252222+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T02:08:24.776211+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T02:09:37.906848+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T02:09:38.849380+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T02:09:55.761 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T02:09:55.891 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.892+0000 7f1a87686640 1 Processor -- start 2023-11-06T02:09:55.891 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.892+0000 7f1a87686640 1 -- start start 2023-11-06T02:09:55.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.892+0000 7f1a87686640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f1a8014ffc0 0x7f1a800ada30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:55.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.892+0000 7f1a87686640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f1a800adf70 0x7f1a80070160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:55.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.892+0000 7f1a87686640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a800c8c80 con 0x7f1a800706a0 2023-11-06T02:09:55.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.892+0000 7f1a87686640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a800ae4d0 con 0x7f1a800adf70 2023-11-06T02:09:55.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.892+0000 7f1a87686640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a801594f0 con 0x7f1a8014ffc0 2023-11-06T02:09:55.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.892+0000 7f1a84c21640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f1a800adf70 0x7f1a80070160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:55.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.892+0000 7f1a85422640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f1a8014ffc0 0x7f1a800ada30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:55.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.892+0000 7f1a84c21640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f1a800adf70 0x7f1a80070160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44704/0 (socket says 172.21.15.49:44704) 2023-11-06T02:09:55.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.892+0000 7f1a84c21640 1 -- 172.21.15.49:0/2833371605 learned_addr learned my addr 172.21.15.49:0/2833371605 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:55.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.892+0000 7f1a84c21640 1 -- 172.21.15.49:0/2833371605 >> v2:172.21.15.150:3300/0 conn(0x7f1a8014ffc0 msgr2=0x7f1a800ada30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:55.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.892+0000 7f1a84c21640 1 --2- 172.21.15.49:0/2833371605 >> v2:172.21.15.150:3300/0 conn(0x7f1a8014ffc0 0x7f1a800ada30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:55.895 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.892+0000 7f1a84c21640 1 -- 172.21.15.49:0/2833371605 >> v1:172.21.15.150:6789/0 conn(0x7f1a800706a0 legacy=0x7f1a8015cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:55.895 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.892+0000 7f1a84c21640 1 -- 172.21.15.49:0/2833371605 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a80070af0 con 0x7f1a800adf70 2023-11-06T02:09:55.896 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.896+0000 7f1a84c21640 1 --2- 172.21.15.49:0/2833371605 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1a800adf70 0x7f1a80070160 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f1a8006e390 tx=0x7f1a7401c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ec48cf6d8f47bcb server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:55.896 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.896+0000 7f1a73fff640 1 -- 172.21.15.49:0/2833371605 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a740ab600 con 0x7f1a800adf70 2023-11-06T02:09:55.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.896+0000 7f1a73fff640 1 -- 172.21.15.49:0/2833371605 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a740a7050 con 0x7f1a800adf70 2023-11-06T02:09:55.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.896+0000 7f1a73fff640 1 -- 172.21.15.49:0/2833371605 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a740ab980 con 0x7f1a800adf70 2023-11-06T02:09:55.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.896+0000 7f1a87686640 1 -- 172.21.15.49:0/2833371605 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1a800adf70 msgr2=0x7f1a80070160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:55.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.896+0000 7f1a87686640 1 --2- 172.21.15.49:0/2833371605 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1a800adf70 0x7f1a80070160 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f1a8006e390 tx=0x7f1a7401c500 comp rx=0 tx=0).stop 2023-11-06T02:09:55.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.896+0000 7f1a87686640 1 -- 172.21.15.49:0/2833371605 shutdown_connections 2023-11-06T02:09:55.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.896+0000 7f1a87686640 1 --2- 172.21.15.49:0/2833371605 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1a800adf70 0x7f1a80070160 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:55.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.896+0000 7f1a87686640 1 --2- 172.21.15.49:0/2833371605 >> v2:172.21.15.150:3300/0 conn(0x7f1a8014ffc0 0x7f1a800ada30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:55.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.896+0000 7f1a87686640 1 -- 172.21.15.49:0/2833371605 >> 172.21.15.49:0/2833371605 conn(0x7f1a8006a370 msgr2=0x7f1a800ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:55.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.896+0000 7f1a87686640 1 -- 172.21.15.49:0/2833371605 shutdown_connections 2023-11-06T02:09:55.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.896+0000 7f1a87686640 1 -- 172.21.15.49:0/2833371605 wait complete. 2023-11-06T02:09:55.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.900+0000 7f1a87686640 1 Processor -- start 2023-11-06T02:09:55.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.900+0000 7f1a87686640 1 -- start start 2023-11-06T02:09:55.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.900+0000 7f1a87686640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1a800706a0 0x7f1a8014af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:55.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.900+0000 7f1a87686640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1a800adf70 0x7f1a80144c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:55.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.900+0000 7f1a87686640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1a8014ffc0 0x7f1a800c9290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:55.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.900+0000 7f1a87686640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a80145b90 con 0x7f1a800706a0 2023-11-06T02:09:55.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.900+0000 7f1a87686640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a8014b4d0 con 0x7f1a8014ffc0 2023-11-06T02:09:55.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.900+0000 7f1a87686640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a801451a0 con 0x7f1a800adf70 2023-11-06T02:09:55.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.900+0000 7f1a85422640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1a800706a0 0x7f1a8014af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:55.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.900+0000 7f1a85422640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1a800706a0 0x7f1a8014af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44706/0 (socket says 172.21.15.49:44706) 2023-11-06T02:09:55.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.900+0000 7f1a85422640 1 -- 172.21.15.49:0/2587816938 learned_addr learned my addr 172.21.15.49:0/2587816938 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:55.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.900+0000 7f1a84c21640 1 --2- 172.21.15.49:0/2587816938 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1a800adf70 0x7f1a80144c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:55.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.900+0000 7f1a85c23640 1 --2- 172.21.15.49:0/2587816938 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1a8014ffc0 0x7f1a800c9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:55.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.900+0000 7f1a85422640 1 -- 172.21.15.49:0/2587816938 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1a800adf70 msgr2=0x7f1a80144c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:55.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.900+0000 7f1a85422640 1 --2- 172.21.15.49:0/2587816938 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1a800adf70 0x7f1a80144c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:55.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.900+0000 7f1a85422640 1 -- 172.21.15.49:0/2587816938 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1a8014ffc0 msgr2=0x7f1a800c9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:55.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.900+0000 7f1a85422640 1 --2- 172.21.15.49:0/2587816938 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1a8014ffc0 0x7f1a800c9290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:55.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.900+0000 7f1a85422640 1 -- 172.21.15.49:0/2587816938 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a740094d0 con 0x7f1a800706a0 2023-11-06T02:09:55.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.900+0000 7f1a85422640 1 --2- 172.21.15.49:0/2587816938 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1a800706a0 0x7f1a8014af90 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f1a78009d10 tx=0x7f1a7800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:55.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.900+0000 7f1a71ffb640 1 -- 172.21.15.49:0/2587816938 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a78009b80 con 0x7f1a800706a0 2023-11-06T02:09:55.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.900+0000 7f1a71ffb640 1 -- 172.21.15.49:0/2587816938 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a7800f040 con 0x7f1a800706a0 2023-11-06T02:09:55.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.900+0000 7f1a87686640 1 -- 172.21.15.49:0/2587816938 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a801506f0 con 0x7f1a800706a0 2023-11-06T02:09:55.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.900+0000 7f1a71ffb640 1 -- 172.21.15.49:0/2587816938 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a78014680 con 0x7f1a800706a0 2023-11-06T02:09:55.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.900+0000 7f1a87686640 1 -- 172.21.15.49:0/2587816938 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a80150cb0 con 0x7f1a800706a0 2023-11-06T02:09:55.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.900+0000 7f1a87686640 1 -- 172.21.15.49:0/2587816938 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a80074040 con 0x7f1a800706a0 2023-11-06T02:09:55.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.900+0000 7f1a71ffb640 1 -- 172.21.15.49:0/2587816938 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f1a780147e0 con 0x7f1a800706a0 2023-11-06T02:09:55.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.904+0000 7f1a71ffb640 1 --2- 172.21.15.49:0/2587816938 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1a64065a60 0x7f1a64067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:55.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.904+0000 7f1a71ffb640 1 -- 172.21.15.49:0/2587816938 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1a780840a0 con 0x7f1a800706a0 2023-11-06T02:09:55.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.904+0000 7f1a84c21640 1 --2- 172.21.15.49:0/2587816938 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1a64065a60 0x7f1a64067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:55.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.904+0000 7f1a84c21640 1 --2- 172.21.15.49:0/2587816938 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1a64065a60 0x7f1a64067f20 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1a740ad000 tx=0x7f1a74008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:55.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:55.908+0000 7f1a71ffb640 1 -- 172.21.15.49:0/2587816938 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f1a78051990 con 0x7f1a800706a0 2023-11-06T02:09:56.059 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.060+0000 7f1a87686640 1 -- 172.21.15.49:0/2587816938 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1a800c8fb0 con 0x7f1a800706a0 2023-11-06T02:09:56.060 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.060+0000 7f1a71ffb640 1 -- 172.21.15.49:0/2587816938 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v37) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1a78054fb0 con 0x7f1a800706a0 2023-11-06T02:09:56.060 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2023-11-06T02:09:56.062 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.064+0000 7f1a87686640 1 -- 172.21.15.49:0/2587816938 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1a64065a60 msgr2=0x7f1a64067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:56.063 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.064+0000 7f1a87686640 1 --2- 172.21.15.49:0/2587816938 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1a64065a60 0x7f1a64067f20 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1a740ad000 tx=0x7f1a74008040 comp rx=0 tx=0).stop 2023-11-06T02:09:56.063 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.064+0000 7f1a87686640 1 -- 172.21.15.49:0/2587816938 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1a800706a0 msgr2=0x7f1a8014af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:56.063 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.064+0000 7f1a87686640 1 --2- 172.21.15.49:0/2587816938 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1a800706a0 0x7f1a8014af90 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f1a78009d10 tx=0x7f1a7800c490 comp rx=0 tx=0).stop 2023-11-06T02:09:56.063 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.064+0000 7f1a87686640 1 -- 172.21.15.49:0/2587816938 shutdown_connections 2023-11-06T02:09:56.064 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.064+0000 7f1a87686640 1 --2- 172.21.15.49:0/2587816938 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1a64065a60 0x7f1a64067f20 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:56.064 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.064+0000 7f1a87686640 1 --2- 172.21.15.49:0/2587816938 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1a8014ffc0 0x7f1a800c9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:56.064 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.064+0000 7f1a87686640 1 --2- 172.21.15.49:0/2587816938 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1a800adf70 0x7f1a80144c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:56.064 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.064+0000 7f1a87686640 1 --2- 172.21.15.49:0/2587816938 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1a800706a0 0x7f1a8014af90 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:56.064 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.064+0000 7f1a87686640 1 -- 172.21.15.49:0/2587816938 >> 172.21.15.49:0/2587816938 conn(0x7f1a8006a370 msgr2=0x7f1a800af920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:56.064 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.064+0000 7f1a87686640 1 -- 172.21.15.49:0/2587816938 shutdown_connections 2023-11-06T02:09:56.065 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.064+0000 7f1a87686640 1 -- 172.21.15.49:0/2587816938 wait complete. 2023-11-06T02:09:56.078 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T02:09:56.196 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.196+0000 7fa853560640 1 Processor -- start 2023-11-06T02:09:56.197 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.196+0000 7fa853560640 1 -- start start 2023-11-06T02:09:56.197 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.200+0000 7fa853560640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fa84c070160 0x7fa84c0705d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:56.197 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.200+0000 7fa853560640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa84c06c230 0x7fa84c06c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:56.198 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.200+0000 7fa853560640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa84c0c8c80 con 0x7fa84c14ffc0 2023-11-06T02:09:56.198 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.200+0000 7fa853560640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa84c0ae4d0 con 0x7fa84c06c230 2023-11-06T02:09:56.198 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.200+0000 7fa853560640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa84c1594f0 con 0x7fa84c070160 2023-11-06T02:09:56.199 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.200+0000 7fa851afd640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa84c06c230 0x7fa84c06c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:56.199 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.200+0000 7fa850afb640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fa84c070160 0x7fa84c0705d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:56.199 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.200+0000 7fa850afb640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fa84c070160 0x7fa84c0705d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.49:37880/0 (socket says 172.21.15.49:37880) 2023-11-06T02:09:56.200 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.200+0000 7fa850afb640 1 -- 172.21.15.49:0/1701136900 learned_addr learned my addr 172.21.15.49:0/1701136900 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:56.200 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.200+0000 7fa851afd640 1 -- 172.21.15.49:0/1701136900 >> v2:172.21.15.150:3300/0 conn(0x7fa84c070160 msgr2=0x7fa84c0705d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:56.200 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.200+0000 7fa851afd640 1 --2- 172.21.15.49:0/1701136900 >> v2:172.21.15.150:3300/0 conn(0x7fa84c070160 0x7fa84c0705d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:56.201 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.200+0000 7fa851afd640 1 -- 172.21.15.49:0/1701136900 >> v1:172.21.15.150:6789/0 conn(0x7fa84c14ffc0 legacy=0x7fa84c0ada30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:56.201 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.200+0000 7fa851afd640 1 -- 172.21.15.49:0/1701136900 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa84c159700 con 0x7fa84c06c230 2023-11-06T02:09:56.202 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.200+0000 7fa851afd640 1 --2- 172.21.15.49:0/1701136900 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa84c06c230 0x7fa84c06c680 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fa84c0cb010 tx=0x7fa84401da30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2774f34f40506d9b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:56.203 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.200+0000 7fa843fff640 1 -- 172.21.15.49:0/1701136900 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa84400a2d0 con 0x7fa84c06c230 2023-11-06T02:09:56.203 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.200+0000 7fa843fff640 1 -- 172.21.15.49:0/1701136900 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8440a6690 con 0x7fa84c06c230 2023-11-06T02:09:56.203 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.200+0000 7fa843fff640 1 -- 172.21.15.49:0/1701136900 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8440a6a20 con 0x7fa84c06c230 2023-11-06T02:09:56.204 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.204+0000 7fa853560640 1 -- 172.21.15.49:0/1701136900 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa84c06c230 msgr2=0x7fa84c06c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:56.204 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.204+0000 7fa853560640 1 --2- 172.21.15.49:0/1701136900 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa84c06c230 0x7fa84c06c680 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fa84c0cb010 tx=0x7fa84401da30 comp rx=0 tx=0).stop 2023-11-06T02:09:56.204 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.204+0000 7fa853560640 1 -- 172.21.15.49:0/1701136900 shutdown_connections 2023-11-06T02:09:56.204 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.204+0000 7fa853560640 1 --2- 172.21.15.49:0/1701136900 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa84c06c230 0x7fa84c06c680 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:56.204 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.204+0000 7fa853560640 1 --2- 172.21.15.49:0/1701136900 >> v2:172.21.15.150:3300/0 conn(0x7fa84c070160 0x7fa84c0705d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:56.205 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.204+0000 7fa853560640 1 -- 172.21.15.49:0/1701136900 >> 172.21.15.49:0/1701136900 conn(0x7fa84c06a370 msgr2=0x7fa84c0ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:56.205 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.204+0000 7fa853560640 1 -- 172.21.15.49:0/1701136900 shutdown_connections 2023-11-06T02:09:56.205 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.204+0000 7fa853560640 1 -- 172.21.15.49:0/1701136900 wait complete. 2023-11-06T02:09:56.205 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.204+0000 7fa853560640 1 Processor -- start 2023-11-06T02:09:56.205 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.204+0000 7fa853560640 1 -- start start 2023-11-06T02:09:56.208 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.204+0000 7fa853560640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa84c06c230 0x7fa84c0c7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:56.208 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.204+0000 7fa853560640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa84c070160 0x7fa84c1f4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:56.209 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.204+0000 7fa853560640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa84c14ffc0 0x7fa84c0b0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:56.209 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.204+0000 7fa853560640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa84c14bb10 con 0x7fa84c06c230 2023-11-06T02:09:56.209 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.204+0000 7fa853560640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa84c14d1e0 con 0x7fa84c14ffc0 2023-11-06T02:09:56.209 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.204+0000 7fa853560640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa84c14b450 con 0x7fa84c070160 2023-11-06T02:09:56.209 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.204+0000 7fa8512fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa84c06c230 0x7fa84c0c7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:56.210 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.204+0000 7fa8512fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa84c06c230 0x7fa84c0c7ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44724/0 (socket says 172.21.15.49:44724) 2023-11-06T02:09:56.210 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.204+0000 7fa8512fc640 1 -- 172.21.15.49:0/2385213266 learned_addr learned my addr 172.21.15.49:0/2385213266 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:56.210 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.204+0000 7fa850afb640 1 --2- 172.21.15.49:0/2385213266 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa84c070160 0x7fa84c1f4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:56.210 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.204+0000 7fa851afd640 1 --2- 172.21.15.49:0/2385213266 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa84c14ffc0 0x7fa84c0b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:56.211 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.204+0000 7fa8512fc640 1 -- 172.21.15.49:0/2385213266 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa84c070160 msgr2=0x7fa84c1f4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:56.211 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.204+0000 7fa8512fc640 1 --2- 172.21.15.49:0/2385213266 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa84c070160 0x7fa84c1f4ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:56.211 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.204+0000 7fa8512fc640 1 -- 172.21.15.49:0/2385213266 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa84c14ffc0 msgr2=0x7fa84c0b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:56.211 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.204+0000 7fa8512fc640 1 --2- 172.21.15.49:0/2385213266 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa84c14ffc0 0x7fa84c0b0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:56.211 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.204+0000 7fa8512fc640 1 -- 172.21.15.49:0/2385213266 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa84400ab80 con 0x7fa84c06c230 2023-11-06T02:09:56.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.204+0000 7fa8512fc640 1 --2- 172.21.15.49:0/2385213266 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa84c06c230 0x7fa84c0c7ba0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fa84800c500 tx=0x7fa84800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:56.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.204+0000 7fa841ffb640 1 -- 172.21.15.49:0/2385213266 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa848005220 con 0x7fa84c06c230 2023-11-06T02:09:56.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.204+0000 7fa841ffb640 1 -- 172.21.15.49:0/2385213266 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa848005380 con 0x7fa84c06c230 2023-11-06T02:09:56.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.204+0000 7fa853560640 1 -- 172.21.15.49:0/2385213266 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa84c14b730 con 0x7fa84c06c230 2023-11-06T02:09:56.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.204+0000 7fa841ffb640 1 -- 172.21.15.49:0/2385213266 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa848007270 con 0x7fa84c06c230 2023-11-06T02:09:56.213 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.204+0000 7fa853560640 1 -- 172.21.15.49:0/2385213266 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa84c0c80e0 con 0x7fa84c06c230 2023-11-06T02:09:56.213 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.208+0000 7fa853560640 1 -- 172.21.15.49:0/2385213266 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa84c14b970 con 0x7fa84c06c230 2023-11-06T02:09:56.213 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.208+0000 7fa841ffb640 1 -- 172.21.15.49:0/2385213266 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fa8480073d0 con 0x7fa84c06c230 2023-11-06T02:09:56.213 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.208+0000 7fa841ffb640 1 --2- 172.21.15.49:0/2385213266 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa830065a90 0x7fa830067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:56.213 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.208+0000 7fa841ffb640 1 -- 172.21.15.49:0/2385213266 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa8480826a0 con 0x7fa84c06c230 2023-11-06T02:09:56.214 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.208+0000 7fa850afb640 1 --2- 172.21.15.49:0/2385213266 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa830065a90 0x7fa830067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:56.214 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.208+0000 7fa850afb640 1 --2- 172.21.15.49:0/2385213266 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa830065a90 0x7fa830067f50 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa84c149e20 tx=0x7fa83c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:56.214 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.212+0000 7fa841ffb640 1 -- 172.21.15.49:0/2385213266 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fa84c14b970 con 0x7fa84c06c230 2023-11-06T02:09:56.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.360+0000 7fa853560640 1 -- 172.21.15.49:0/2385213266 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa84c074040 con 0x7fa84c06c230 2023-11-06T02:09:56.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.360+0000 7fa841ffb640 1 -- 172.21.15.49:0/2385213266 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v37) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa84804fe70 con 0x7fa84c06c230 2023-11-06T02:09:56.361 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 8 2023-11-06T02:09:56.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.364+0000 7fa853560640 1 -- 172.21.15.49:0/2385213266 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa830065a90 msgr2=0x7fa830067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:56.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.364+0000 7fa853560640 1 --2- 172.21.15.49:0/2385213266 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa830065a90 0x7fa830067f50 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa84c149e20 tx=0x7fa83c008040 comp rx=0 tx=0).stop 2023-11-06T02:09:56.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.364+0000 7fa853560640 1 -- 172.21.15.49:0/2385213266 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa84c06c230 msgr2=0x7fa84c0c7ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:56.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.364+0000 7fa853560640 1 --2- 172.21.15.49:0/2385213266 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa84c06c230 0x7fa84c0c7ba0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fa84800c500 tx=0x7fa84800c9c0 comp rx=0 tx=0).stop 2023-11-06T02:09:56.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.364+0000 7fa853560640 1 -- 172.21.15.49:0/2385213266 shutdown_connections 2023-11-06T02:09:56.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.364+0000 7fa853560640 1 --2- 172.21.15.49:0/2385213266 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa830065a90 0x7fa830067f50 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:56.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.364+0000 7fa853560640 1 --2- 172.21.15.49:0/2385213266 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa84c14ffc0 0x7fa84c0b0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:56.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.364+0000 7fa853560640 1 --2- 172.21.15.49:0/2385213266 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa84c070160 0x7fa84c1f4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:56.365 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.364+0000 7fa853560640 1 --2- 172.21.15.49:0/2385213266 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa84c06c230 0x7fa84c0c7ba0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:56.365 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.364+0000 7fa853560640 1 -- 172.21.15.49:0/2385213266 >> 172.21.15.49:0/2385213266 conn(0x7fa84c06a370 msgr2=0x7fa84c0c9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:56.365 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.364+0000 7fa853560640 1 -- 172.21.15.49:0/2385213266 shutdown_connections 2023-11-06T02:09:56.365 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.364+0000 7fa853560640 1 -- 172.21.15.49:0/2385213266 wait complete. 2023-11-06T02:09:56.378 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T02:09:56.494 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.496+0000 7fa233b6d640 1 Processor -- start 2023-11-06T02:09:56.495 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.496+0000 7fa233b6d640 1 -- start start 2023-11-06T02:09:56.496 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.496+0000 7fa233b6d640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa22c15a880 0x7fa22c0b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:56.496 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.496+0000 7fa233b6d640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fa22c070160 0x7fa22c070610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:56.497 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.496+0000 7fa233b6d640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa22c06d150 0x7fa22c06d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:56.497 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.496+0000 7fa233b6d640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa22c070b50 con 0x7fa22c15a880 2023-11-06T02:09:56.497 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.496+0000 7fa233b6d640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa22c06db40 con 0x7fa22c070160 2023-11-06T02:09:56.497 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.496+0000 7fa233b6d640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa22c0ae360 con 0x7fa22c06d150 2023-11-06T02:09:56.497 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.496+0000 7fa231909640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa22c15a880 0x7fa22c0b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:56.498 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.496+0000 7fa231909640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa22c15a880 0x7fa22c0b15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44728/0 (socket says 172.21.15.49:44728) 2023-11-06T02:09:56.498 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.496+0000 7fa231909640 1 -- 172.21.15.49:0/3728412123 learned_addr learned my addr 172.21.15.49:0/3728412123 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:56.498 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.496+0000 7fa23210a640 1 --2- 172.21.15.49:0/3728412123 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa22c06d150 0x7fa22c06d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:56.498 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.496+0000 7fa231108640 1 --2- 172.21.15.49:0/3728412123 >> v2:172.21.15.150:3300/0 conn(0x7fa22c070160 0x7fa22c070610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:56.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.496+0000 7fa231909640 1 -- 172.21.15.49:0/3728412123 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa22c06d150 msgr2=0x7fa22c06d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:56.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.496+0000 7fa231909640 1 --2- 172.21.15.49:0/3728412123 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa22c06d150 0x7fa22c06d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:56.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.496+0000 7fa231909640 1 -- 172.21.15.49:0/3728412123 >> v2:172.21.15.150:3300/0 conn(0x7fa22c070160 msgr2=0x7fa22c070610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:56.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.496+0000 7fa231909640 1 --2- 172.21.15.49:0/3728412123 >> v2:172.21.15.150:3300/0 conn(0x7fa22c070160 0x7fa22c070610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:56.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.496+0000 7fa231909640 1 -- 172.21.15.49:0/3728412123 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa22c0ae640 con 0x7fa22c15a880 2023-11-06T02:09:56.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.496+0000 7fa231108640 1 --2- 172.21.15.49:0/3728412123 >> v2:172.21.15.150:3300/0 conn(0x7fa22c070160 0x7fa22c070610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:09:56.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.500+0000 7fa231909640 1 --2- 172.21.15.49:0/3728412123 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa22c15a880 0x7fa22c0b15c0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fa22c15e3a0 tx=0x7fa22801c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bade76d7db24c32 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:56.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.500+0000 7fa230907640 1 -- 172.21.15.49:0/3728412123 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2280ac630 con 0x7fa22c15a880 2023-11-06T02:09:56.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.500+0000 7fa230907640 1 -- 172.21.15.49:0/3728412123 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2280ac790 con 0x7fa22c15a880 2023-11-06T02:09:56.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.500+0000 7fa230907640 1 -- 172.21.15.49:0/3728412123 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2280acb20 con 0x7fa22c15a880 2023-11-06T02:09:56.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.500+0000 7fa233b6d640 1 -- 172.21.15.49:0/3728412123 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa22c15a880 msgr2=0x7fa22c0b15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:56.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.500+0000 7fa233b6d640 1 --2- 172.21.15.49:0/3728412123 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa22c15a880 0x7fa22c0b15c0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fa22c15e3a0 tx=0x7fa22801c500 comp rx=0 tx=0).stop 2023-11-06T02:09:56.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.500+0000 7fa233b6d640 1 -- 172.21.15.49:0/3728412123 shutdown_connections 2023-11-06T02:09:56.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.500+0000 7fa233b6d640 1 --2- 172.21.15.49:0/3728412123 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa22c06d150 0x7fa22c06d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:56.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.500+0000 7fa233b6d640 1 --2- 172.21.15.49:0/3728412123 >> v2:172.21.15.150:3300/0 conn(0x7fa22c070160 0x7fa22c070610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:56.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.500+0000 7fa233b6d640 1 --2- 172.21.15.49:0/3728412123 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa22c15a880 0x7fa22c0b15c0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:56.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.500+0000 7fa233b6d640 1 -- 172.21.15.49:0/3728412123 >> 172.21.15.49:0/3728412123 conn(0x7fa22c06a370 msgr2=0x7fa22c0ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:56.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.500+0000 7fa233b6d640 1 -- 172.21.15.49:0/3728412123 shutdown_connections 2023-11-06T02:09:56.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.500+0000 7fa233b6d640 1 -- 172.21.15.49:0/3728412123 wait complete. 2023-11-06T02:09:56.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.500+0000 7fa233b6d640 1 Processor -- start 2023-11-06T02:09:56.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.500+0000 7fa233b6d640 1 -- start start 2023-11-06T02:09:56.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.504+0000 7fa233b6d640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa22c06d150 0x7fa22c152070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:56.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.504+0000 7fa233b6d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa22c070160 0x7fa22c1f5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:56.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.504+0000 7fa233b6d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa22c15a880 0x7fa22c159000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:56.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.504+0000 7fa233b6d640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa22c0c9aa0 con 0x7fa22c070160 2023-11-06T02:09:56.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.504+0000 7fa233b6d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa22c159540 con 0x7fa22c15a880 2023-11-06T02:09:56.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.504+0000 7fa233b6d640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa22c0c8e20 con 0x7fa22c06d150 2023-11-06T02:09:56.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.504+0000 7fa231108640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa22c070160 0x7fa22c1f5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:56.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.504+0000 7fa231108640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa22c070160 0x7fa22c1f5490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44744/0 (socket says 172.21.15.49:44744) 2023-11-06T02:09:56.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.504+0000 7fa231108640 1 -- 172.21.15.49:0/2498864193 learned_addr learned my addr 172.21.15.49:0/2498864193 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:56.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.504+0000 7fa231909640 1 --2- 172.21.15.49:0/2498864193 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa22c06d150 0x7fa22c152070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:56.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.504+0000 7fa231108640 1 -- 172.21.15.49:0/2498864193 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa22c06d150 msgr2=0x7fa22c152070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:56.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.504+0000 7fa23210a640 1 --2- 172.21.15.49:0/2498864193 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa22c15a880 0x7fa22c159000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:56.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.504+0000 7fa231108640 1 --2- 172.21.15.49:0/2498864193 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa22c06d150 0x7fa22c152070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:56.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.504+0000 7fa231108640 1 -- 172.21.15.49:0/2498864193 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa22c15a880 msgr2=0x7fa22c159000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:56.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.504+0000 7fa231108640 1 --2- 172.21.15.49:0/2498864193 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa22c15a880 0x7fa22c159000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:56.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.504+0000 7fa231108640 1 -- 172.21.15.49:0/2498864193 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2280094d0 con 0x7fa22c070160 2023-11-06T02:09:56.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.504+0000 7fa231909640 1 --2- 172.21.15.49:0/2498864193 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa22c06d150 0x7fa22c152070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:09:56.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.504+0000 7fa231108640 1 --2- 172.21.15.49:0/2498864193 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa22c070160 0x7fa22c1f5490 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fa21c00c7c0 tx=0x7fa21c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:56.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.504+0000 7fa2227fc640 1 -- 172.21.15.49:0/2498864193 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa21c007a30 con 0x7fa22c070160 2023-11-06T02:09:56.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.504+0000 7fa2227fc640 1 -- 172.21.15.49:0/2498864193 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa21c007b90 con 0x7fa22c070160 2023-11-06T02:09:56.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.504+0000 7fa233b6d640 1 -- 172.21.15.49:0/2498864193 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa22c15ade0 con 0x7fa22c070160 2023-11-06T02:09:56.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.504+0000 7fa2227fc640 1 -- 172.21.15.49:0/2498864193 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa21c00ce70 con 0x7fa22c070160 2023-11-06T02:09:56.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.504+0000 7fa233b6d640 1 -- 172.21.15.49:0/2498864193 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa22c0c93e0 con 0x7fa22c070160 2023-11-06T02:09:56.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.504+0000 7fa233b6d640 1 -- 172.21.15.49:0/2498864193 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa22c067190 con 0x7fa22c070160 2023-11-06T02:09:56.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.504+0000 7fa2227fc640 1 -- 172.21.15.49:0/2498864193 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fa21c0113f0 con 0x7fa22c070160 2023-11-06T02:09:56.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.504+0000 7fa2227fc640 1 --2- 172.21.15.49:0/2498864193 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa210065a60 0x7fa210067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:56.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.504+0000 7fa2227fc640 1 -- 172.21.15.49:0/2498864193 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa21c083800 con 0x7fa22c070160 2023-11-06T02:09:56.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.504+0000 7fa231909640 1 --2- 172.21.15.49:0/2498864193 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa210065a60 0x7fa210067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:56.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.508+0000 7fa231909640 1 --2- 172.21.15.49:0/2498864193 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa210065a60 0x7fa210067f20 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa2280ae000 tx=0x7fa228008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:56.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.508+0000 7fa2227fc640 1 -- 172.21.15.49:0/2498864193 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fa21c051060 con 0x7fa22c070160 2023-11-06T02:09:56.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.660+0000 7fa233b6d640 1 -- 172.21.15.49:0/2498864193 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa22c074040 con 0x7fa22c070160 2023-11-06T02:09:56.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.664+0000 7fa2227fc640 1 -- 172.21.15.49:0/2498864193 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v37) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa21c01e020 con 0x7fa22c070160 2023-11-06T02:09:56.662 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2023-11-06T02:09:56.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.668+0000 7fa233b6d640 1 -- 172.21.15.49:0/2498864193 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa210065a60 msgr2=0x7fa210067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:56.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.668+0000 7fa233b6d640 1 --2- 172.21.15.49:0/2498864193 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa210065a60 0x7fa210067f20 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa2280ae000 tx=0x7fa228008040 comp rx=0 tx=0).stop 2023-11-06T02:09:56.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.668+0000 7fa233b6d640 1 -- 172.21.15.49:0/2498864193 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa22c070160 msgr2=0x7fa22c1f5490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:56.667 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.668+0000 7fa233b6d640 1 --2- 172.21.15.49:0/2498864193 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa22c070160 0x7fa22c1f5490 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fa21c00c7c0 tx=0x7fa21c00cc80 comp rx=0 tx=0).stop 2023-11-06T02:09:56.667 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.668+0000 7fa233b6d640 1 -- 172.21.15.49:0/2498864193 shutdown_connections 2023-11-06T02:09:56.667 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.668+0000 7fa233b6d640 1 --2- 172.21.15.49:0/2498864193 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa210065a60 0x7fa210067f20 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:56.667 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.668+0000 7fa233b6d640 1 --2- 172.21.15.49:0/2498864193 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa22c070160 0x7fa22c1f5490 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:56.667 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.668+0000 7fa233b6d640 1 --2- 172.21.15.49:0/2498864193 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa22c15a880 0x7fa22c159000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:56.668 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.668+0000 7fa233b6d640 1 --2- 172.21.15.49:0/2498864193 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa22c06d150 0x7fa22c152070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:56.668 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.668+0000 7fa233b6d640 1 -- 172.21.15.49:0/2498864193 >> 172.21.15.49:0/2498864193 conn(0x7fa22c06a370 msgr2=0x7fa22c15af70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:56.668 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.668+0000 7fa233b6d640 1 -- 172.21.15.49:0/2498864193 shutdown_connections 2023-11-06T02:09:56.668 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.668+0000 7fa233b6d640 1 -- 172.21.15.49:0/2498864193 wait complete. 2023-11-06T02:09:56.682 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T02:09:56.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.808+0000 7f78b62b0640 1 Processor -- start 2023-11-06T02:09:56.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.808+0000 7f78b62b0640 1 -- start start 2023-11-06T02:09:56.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.808+0000 7f78b62b0640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f78b00cd380 0x7f78b00b5b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:56.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.808+0000 7f78b62b0640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f78b00b6060 0x7f78b00d1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:56.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.808+0000 7f78b62b0640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78b00d0d00 con 0x7f78b00cf530 2023-11-06T02:09:56.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.808+0000 7f78b62b0640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78b00ce120 con 0x7f78b00b6060 2023-11-06T02:09:56.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.808+0000 7f78b62b0640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78b00cd880 con 0x7f78b00cd380 2023-11-06T02:09:56.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.808+0000 7f78affff640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f78b00cf530 0x7f78b00cf910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:49396/0 (socket says 172.21.15.49:49396) 2023-11-06T02:09:56.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.808+0000 7f78affff640 1 -- 172.21.15.49:0/3909972244 learned_addr learned my addr 172.21.15.49:0/3909972244 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:09:56.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.808+0000 7f78af7fe640 1 --2- 172.21.15.49:0/3909972244 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f78b00cd380 0x7f78b00b5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:56.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.808+0000 7f78b484d640 1 --2- 172.21.15.49:0/3909972244 >> v2:172.21.15.150:3300/0 conn(0x7f78b00b6060 0x7f78b00d1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:56.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.808+0000 7f78aeffd640 1 -- 172.21.15.49:0/3909972244 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1809922713 0 0) 0x7f78b00d0d00 con 0x7f78b00cf530 2023-11-06T02:09:56.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.808+0000 7f78aeffd640 1 -- 172.21.15.49:0/3909972244 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7898003530 con 0x7f78b00cf530 2023-11-06T02:09:56.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.808+0000 7f78aeffd640 1 -- 172.21.15.49:0/3909972244 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2262547352 0 0) 0x7f7898003530 con 0x7f78b00cf530 2023-11-06T02:09:56.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.808+0000 7f78aeffd640 1 -- 172.21.15.49:0/3909972244 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f78b00cd380 msgr2=0x7f78b00b5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:56.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.808+0000 7f78aeffd640 1 --2- 172.21.15.49:0/3909972244 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f78b00cd380 0x7f78b00b5b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:56.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.808+0000 7f78aeffd640 1 -- 172.21.15.49:0/3909972244 >> v2:172.21.15.150:3300/0 conn(0x7f78b00b6060 msgr2=0x7f78b00d1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:56.813 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.808+0000 7f78aeffd640 1 --2- 172.21.15.49:0/3909972244 >> v2:172.21.15.150:3300/0 conn(0x7f78b00b6060 0x7f78b00d1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:56.813 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.808+0000 7f78aeffd640 1 -- 172.21.15.49:0/3909972244 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78b00cdb60 con 0x7f78b00cf530 2023-11-06T02:09:56.814 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.808+0000 7f78aeffd640 1 -- 172.21.15.49:0/3909972244 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7f78a0001680 con 0x7f78b00cf530 2023-11-06T02:09:56.814 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.808+0000 7f78aeffd640 1 -- 172.21.15.49:0/3909972244 >> v1:172.21.15.49:6789/0 conn(0x7f78b00cf530 legacy=0x7f78b00cf910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:56.814 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.808+0000 7f78aeffd640 1 --2- 172.21.15.49:0/3909972244 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7898003980 0x7f7898003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:56.814 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.808+0000 7f78aeffd640 1 --2- 172.21.15.49:0/3909972244 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78980047d0 0x7f7898004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:56.815 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.808+0000 7f78aeffd640 1 --2- 172.21.15.49:0/3909972244 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f78b00cd380 0x7f7898005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:56.815 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.808+0000 7f78aeffd640 1 -- 172.21.15.49:0/3909972244 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78b00ce120 con 0x7f78980047d0 2023-11-06T02:09:56.815 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.808+0000 7f78aeffd640 1 -- 172.21.15.49:0/3909972244 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78b00cd880 con 0x7f7898003980 2023-11-06T02:09:56.815 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.808+0000 7f78aeffd640 1 -- 172.21.15.49:0/3909972244 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f789800b990 con 0x7f78b00cd380 2023-11-06T02:09:56.815 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.808+0000 7f78af7fe640 1 --2- 172.21.15.49:0/3909972244 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78980047d0 0x7f7898004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:56.816 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.808+0000 7f78b484d640 1 --2- 172.21.15.49:0/3909972244 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f78b00cd380 0x7f7898005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:56.816 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.808+0000 7f78affff640 1 --2- 172.21.15.49:0/3909972244 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7898003980 0x7f7898003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:56.816 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.808+0000 7f78af7fe640 1 -- 172.21.15.49:0/3909972244 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f78b00cd380 msgr2=0x7f7898005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:56.816 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.808+0000 7f78af7fe640 1 --2- 172.21.15.49:0/3909972244 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f78b00cd380 0x7f7898005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:56.816 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.808+0000 7f78af7fe640 1 -- 172.21.15.49:0/3909972244 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7898003980 msgr2=0x7f7898003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:56.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.808+0000 7f78af7fe640 1 --2- 172.21.15.49:0/3909972244 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7898003980 0x7f7898003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:56.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.808+0000 7f78af7fe640 1 -- 172.21.15.49:0/3909972244 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7898003530 con 0x7f78980047d0 2023-11-06T02:09:56.820 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.812+0000 7f78af7fe640 1 --2- 172.21.15.49:0/3909972244 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78980047d0 0x7f7898004c50 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f78a4009ae0 tx=0x7f78a401ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1bc45fa383ef25c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:56.820 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.812+0000 7f78aeffd640 1 -- 172.21.15.49:0/3909972244 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a40a5560 con 0x7f78980047d0 2023-11-06T02:09:56.820 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.812+0000 7f78aeffd640 1 -- 172.21.15.49:0/3909972244 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78a40a56c0 con 0x7f78980047d0 2023-11-06T02:09:56.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.812+0000 7f78aeffd640 1 -- 172.21.15.49:0/3909972244 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a40a5a70 con 0x7f78980047d0 2023-11-06T02:09:56.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.812+0000 7f78b62b0640 1 -- 172.21.15.49:0/3909972244 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78980047d0 msgr2=0x7f7898004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:56.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.812+0000 7f78b62b0640 1 --2- 172.21.15.49:0/3909972244 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78980047d0 0x7f7898004c50 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f78a4009ae0 tx=0x7f78a401ca50 comp rx=0 tx=0).stop 2023-11-06T02:09:56.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.812+0000 7f78b484d640 1 -- 172.21.15.49:0/3909972244 reap_dead start 2023-11-06T02:09:56.822 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.812+0000 7f78b62b0640 1 -- 172.21.15.49:0/3909972244 shutdown_connections 2023-11-06T02:09:56.822 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.812+0000 7f78b62b0640 1 -- 172.21.15.49:0/3909972244 >> 172.21.15.49:0/3909972244 conn(0x7f78b006b090 msgr2=0x7f78b006b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:56.822 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.812+0000 7f78b62b0640 1 -- 172.21.15.49:0/3909972244 shutdown_connections 2023-11-06T02:09:56.822 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.812+0000 7f78b62b0640 1 -- 172.21.15.49:0/3909972244 wait complete. 2023-11-06T02:09:56.822 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.812+0000 7f78b62b0640 1 Processor -- start 2023-11-06T02:09:56.823 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.812+0000 7f78b62b0640 1 -- start start 2023-11-06T02:09:56.823 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.816+0000 7f78b62b0640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f78980047d0 0x7f78b00a9470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:56.823 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.816+0000 7f78b62b0640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f78b00b2000 0x7f78b00d13f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:56.823 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.816+0000 7f78b62b0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78b015a2a0 0x7f78b015a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:56.824 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.816+0000 7f78b62b0640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78b00a99b0 con 0x7f78b015a2a0 2023-11-06T02:09:56.824 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.816+0000 7f78b62b0640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78b00a9c10 con 0x7f78b00b2000 2023-11-06T02:09:56.824 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.816+0000 7f78b62b0640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78b00a9ef0 con 0x7f78980047d0 2023-11-06T02:09:56.824 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.816+0000 7f78b484d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78b015a2a0 0x7f78b015a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:56.824 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.816+0000 7f78b484d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78b015a2a0 0x7f78b015a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44768/0 (socket says 172.21.15.49:44768) 2023-11-06T02:09:56.825 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.816+0000 7f78b484d640 1 -- 172.21.15.49:0/4154180658 learned_addr learned my addr 172.21.15.49:0/4154180658 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:56.825 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.816+0000 7f78affff640 1 --2- 172.21.15.49:0/4154180658 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f78980047d0 0x7f78b00a9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:56.825 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.816+0000 7f78af7fe640 1 --2- 172.21.15.49:0/4154180658 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f78b00b2000 0x7f78b00d13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:56.825 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.816+0000 7f78b484d640 1 -- 172.21.15.49:0/4154180658 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f78980047d0 msgr2=0x7f78b00a9470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:56.825 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.816+0000 7f78b484d640 1 --2- 172.21.15.49:0/4154180658 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f78980047d0 0x7f78b00a9470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:56.826 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.816+0000 7f78b484d640 1 -- 172.21.15.49:0/4154180658 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f78b00b2000 msgr2=0x7f78b00d13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:56.826 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.816+0000 7f78b484d640 1 --2- 172.21.15.49:0/4154180658 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f78b00b2000 0x7f78b00d13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:56.826 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.816+0000 7f78b484d640 1 -- 172.21.15.49:0/4154180658 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78b00aa1d0 con 0x7f78b015a2a0 2023-11-06T02:09:56.826 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.816+0000 7f78b484d640 1 --2- 172.21.15.49:0/4154180658 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78b015a2a0 0x7f78b015a720 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f78b0160590 tx=0x7f78a80115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:56.826 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.816+0000 7f78acff9640 1 -- 172.21.15.49:0/4154180658 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a800e070 con 0x7f78b015a2a0 2023-11-06T02:09:56.827 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.816+0000 7f78acff9640 1 -- 172.21.15.49:0/4154180658 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78a8011ed0 con 0x7f78b015a2a0 2023-11-06T02:09:56.827 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.816+0000 7f78b62b0640 1 -- 172.21.15.49:0/4154180658 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78b00aa4b0 con 0x7f78b015a2a0 2023-11-06T02:09:56.827 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.816+0000 7f78acff9640 1 -- 172.21.15.49:0/4154180658 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a80039a0 con 0x7f78b015a2a0 2023-11-06T02:09:56.827 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.816+0000 7f78b62b0640 1 -- 172.21.15.49:0/4154180658 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78b015ac60 con 0x7f78b015a2a0 2023-11-06T02:09:56.827 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.816+0000 7f78b62b0640 1 -- 172.21.15.49:0/4154180658 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78b00aa9d0 con 0x7f78b015a2a0 2023-11-06T02:09:56.828 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.816+0000 7f78acff9640 1 -- 172.21.15.49:0/4154180658 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f78a8003b00 con 0x7f78b015a2a0 2023-11-06T02:09:56.828 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.816+0000 7f78acff9640 1 --2- 172.21.15.49:0/4154180658 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f789c065a90 0x7f789c067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:56.828 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.816+0000 7f78acff9640 1 -- 172.21.15.49:0/4154180658 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f78a8083920 con 0x7f78b015a2a0 2023-11-06T02:09:56.828 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.816+0000 7f78affff640 1 --2- 172.21.15.49:0/4154180658 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f789c065a90 0x7f789c067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:56.828 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.820+0000 7f78affff640 1 --2- 172.21.15.49:0/4154180658 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f789c065a90 0x7f789c067f50 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f78a0001060 tx=0x7f78a0007040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:56.829 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.820+0000 7f78acff9640 1 -- 172.21.15.49:0/4154180658 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f78b00aa9d0 con 0x7f78b015a2a0 2023-11-06T02:09:56.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.976+0000 7f78b62b0640 1 -- 172.21.15.49:0/4154180658 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f78b00aa9d0 con 0x7f78b015a2a0 2023-11-06T02:09:56.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.976+0000 7f78acff9640 1 -- 172.21.15.49:0/4154180658 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v37) v1 ==== 100+0+11 (secure 0 0 0) 0x7f78b00aa9d0 con 0x7f78b015a2a0 2023-11-06T02:09:56.978 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2023-11-06T02:09:56.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.976+0000 7f78b62b0640 1 -- 172.21.15.49:0/4154180658 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f789c065a90 msgr2=0x7f789c067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:56.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.976+0000 7f78b62b0640 1 --2- 172.21.15.49:0/4154180658 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f789c065a90 0x7f789c067f50 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f78a0001060 tx=0x7f78a0007040 comp rx=0 tx=0).stop 2023-11-06T02:09:56.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.976+0000 7f78b62b0640 1 -- 172.21.15.49:0/4154180658 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78b015a2a0 msgr2=0x7f78b015a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:56.980 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.976+0000 7f78b62b0640 1 --2- 172.21.15.49:0/4154180658 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78b015a2a0 0x7f78b015a720 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f78b0160590 tx=0x7f78a80115a0 comp rx=0 tx=0).stop 2023-11-06T02:09:56.980 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.976+0000 7f78b62b0640 1 -- 172.21.15.49:0/4154180658 shutdown_connections 2023-11-06T02:09:56.980 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.976+0000 7f78b62b0640 1 --2- 172.21.15.49:0/4154180658 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f789c065a90 0x7f789c067f50 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:56.980 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.980+0000 7f78b62b0640 1 --2- 172.21.15.49:0/4154180658 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78b015a2a0 0x7f78b015a720 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:56.980 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.980+0000 7f78b62b0640 1 --2- 172.21.15.49:0/4154180658 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f78b00b2000 0x7f78b00d13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:56.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.980+0000 7f78b62b0640 1 --2- 172.21.15.49:0/4154180658 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f78980047d0 0x7f78b00a9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:56.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.980+0000 7f78b62b0640 1 -- 172.21.15.49:0/4154180658 >> 172.21.15.49:0/4154180658 conn(0x7f78b006b090 msgr2=0x7f78b015d940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:56.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.980+0000 7f78b62b0640 1 -- 172.21.15.49:0/4154180658 shutdown_connections 2023-11-06T02:09:56.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:56.980+0000 7f78b62b0640 1 -- 172.21.15.49:0/4154180658 wait complete. 2023-11-06T02:09:56.994 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-06T02:09:57.114 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.116+0000 7f3dfb6e6640 1 Processor -- start 2023-11-06T02:09:57.115 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.116+0000 7f3dfb6e6640 1 -- start start 2023-11-06T02:09:57.116 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.116+0000 7f3dfb6e6640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f3df40b1f90 0x7f3df40cda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:57.116 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.116+0000 7f3dfb6e6640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f3df4161650 0x7f3df40d1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:57.116 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.116+0000 7f3dfb6e6640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3df40d17c0 con 0x7f3df40d2190 2023-11-06T02:09:57.117 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.116+0000 7f3dfb6e6640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3df4161b00 con 0x7f3df4161650 2023-11-06T02:09:57.117 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.116+0000 7f3dfb6e6640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3df40ce6c0 con 0x7f3df40b1f90 2023-11-06T02:09:57.117 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.116+0000 7f3df8c81640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f3df4161650 0x7f3df40d1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:57.117 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.116+0000 7f3df8c81640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f3df4161650 0x7f3df40d1c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44776/0 (socket says 172.21.15.49:44776) 2023-11-06T02:09:57.118 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.116+0000 7f3df8c81640 1 -- 172.21.15.49:0/1469478472 learned_addr learned my addr 172.21.15.49:0/1469478472 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:57.118 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.116+0000 7f3df9482640 1 --2- 172.21.15.49:0/1469478472 >> v2:172.21.15.150:3300/0 conn(0x7f3df40b1f90 0x7f3df40cda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:57.118 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.116+0000 7f3df8c81640 1 -- 172.21.15.49:0/1469478472 >> v2:172.21.15.150:3300/0 conn(0x7f3df40b1f90 msgr2=0x7f3df40cda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:57.118 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.116+0000 7f3df8c81640 1 --2- 172.21.15.49:0/1469478472 >> v2:172.21.15.150:3300/0 conn(0x7f3df40b1f90 0x7f3df40cda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:57.118 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.116+0000 7f3df8c81640 1 -- 172.21.15.49:0/1469478472 >> v1:172.21.15.150:6789/0 conn(0x7f3df40d2190 legacy=0x7f3df40714a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:57.119 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.116+0000 7f3df8c81640 1 -- 172.21.15.49:0/1469478472 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3df40ce100 con 0x7f3df4161650 2023-11-06T02:09:57.120 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.120+0000 7f3df8c81640 1 --2- 172.21.15.49:0/1469478472 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3df4161650 0x7f3df40d1c50 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f3df4072180 tx=0x7f3de401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f06ec8c530c6a0b4 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:57.120 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.120+0000 7f3debfff640 1 -- 172.21.15.49:0/1469478472 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de40aa430 con 0x7f3df4161650 2023-11-06T02:09:57.120 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.120+0000 7f3debfff640 1 -- 172.21.15.49:0/1469478472 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3de40aa590 con 0x7f3df4161650 2023-11-06T02:09:57.120 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.120+0000 7f3debfff640 1 -- 172.21.15.49:0/1469478472 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de40aa920 con 0x7f3df4161650 2023-11-06T02:09:57.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.120+0000 7f3dfb6e6640 1 -- 172.21.15.49:0/1469478472 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3df4161650 msgr2=0x7f3df40d1c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:57.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.120+0000 7f3dfb6e6640 1 --2- 172.21.15.49:0/1469478472 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3df4161650 0x7f3df40d1c50 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f3df4072180 tx=0x7f3de401c4d0 comp rx=0 tx=0).stop 2023-11-06T02:09:57.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.120+0000 7f3dfb6e6640 1 -- 172.21.15.49:0/1469478472 shutdown_connections 2023-11-06T02:09:57.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.120+0000 7f3dfb6e6640 1 --2- 172.21.15.49:0/1469478472 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3df4161650 0x7f3df40d1c50 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:57.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.120+0000 7f3dfb6e6640 1 --2- 172.21.15.49:0/1469478472 >> v2:172.21.15.150:3300/0 conn(0x7f3df40b1f90 0x7f3df40cda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:57.122 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.120+0000 7f3dfb6e6640 1 -- 172.21.15.49:0/1469478472 >> 172.21.15.49:0/1469478472 conn(0x7f3df40d2aa0 msgr2=0x7f3df40d2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:57.122 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.120+0000 7f3dfb6e6640 1 -- 172.21.15.49:0/1469478472 shutdown_connections 2023-11-06T02:09:57.122 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.120+0000 7f3dfb6e6640 1 -- 172.21.15.49:0/1469478472 wait complete. 2023-11-06T02:09:57.122 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.120+0000 7f3dfb6e6640 1 Processor -- start 2023-11-06T02:09:57.123 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.120+0000 7f3dfb6e6640 1 -- start start 2023-11-06T02:09:57.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.124+0000 7f3dfb6e6640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3df40b1f90 0x7f3df415d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:57.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.124+0000 7f3dfb6e6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3df40d2190 0x7f3df4155eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:57.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.124+0000 7f3dfb6e6640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3df4161650 0x7f3df415f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:57.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.124+0000 7f3dfb6e6640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df40cac10 con 0x7f3df40d2190 2023-11-06T02:09:57.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.124+0000 7f3dfb6e6640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df415d970 con 0x7f3df4161650 2023-11-06T02:09:57.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.124+0000 7f3dfb6e6640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3df415dc50 con 0x7f3df40b1f90 2023-11-06T02:09:57.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.124+0000 7f3df8c81640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3df40d2190 0x7f3df4155eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:57.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.124+0000 7f3df8c81640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3df40d2190 0x7f3df4155eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44786/0 (socket says 172.21.15.49:44786) 2023-11-06T02:09:57.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.124+0000 7f3df8c81640 1 -- 172.21.15.49:0/3514192258 learned_addr learned my addr 172.21.15.49:0/3514192258 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:57.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.124+0000 7f3df9482640 1 --2- 172.21.15.49:0/3514192258 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3df40b1f90 0x7f3df415d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:57.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.124+0000 7f3df8c81640 1 -- 172.21.15.49:0/3514192258 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3df40b1f90 msgr2=0x7f3df415d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:57.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.124+0000 7f3df8c81640 1 --2- 172.21.15.49:0/3514192258 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3df40b1f90 0x7f3df415d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:57.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.124+0000 7f3df9c83640 1 --2- 172.21.15.49:0/3514192258 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3df4161650 0x7f3df415f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:57.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.124+0000 7f3df8c81640 1 -- 172.21.15.49:0/3514192258 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3df4161650 msgr2=0x7f3df415f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:57.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.124+0000 7f3df8c81640 1 --2- 172.21.15.49:0/3514192258 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3df4161650 0x7f3df415f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:57.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.124+0000 7f3df8c81640 1 -- 172.21.15.49:0/3514192258 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3de40094d0 con 0x7f3df40d2190 2023-11-06T02:09:57.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.124+0000 7f3df9482640 1 --2- 172.21.15.49:0/3514192258 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3df40b1f90 0x7f3df415d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:09:57.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.124+0000 7f3df9c83640 1 --2- 172.21.15.49:0/3514192258 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3df4161650 0x7f3df415f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:09:57.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.124+0000 7f3df8c81640 1 --2- 172.21.15.49:0/3514192258 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3df40d2190 0x7f3df4155eb0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f3de40190e0 tx=0x7f3de401ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:57.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.124+0000 7f3de9ffb640 1 -- 172.21.15.49:0/3514192258 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de40aa430 con 0x7f3df40d2190 2023-11-06T02:09:57.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.124+0000 7f3de9ffb640 1 -- 172.21.15.49:0/3514192258 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3de40b3800 con 0x7f3df40d2190 2023-11-06T02:09:57.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.124+0000 7f3de9ffb640 1 -- 172.21.15.49:0/3514192258 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de40b3bb0 con 0x7f3df40d2190 2023-11-06T02:09:57.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.124+0000 7f3dfb6e6640 1 -- 172.21.15.49:0/3514192258 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3df40cae60 con 0x7f3df40d2190 2023-11-06T02:09:57.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.124+0000 7f3dfb6e6640 1 -- 172.21.15.49:0/3514192258 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3df415d5b0 con 0x7f3df40d2190 2023-11-06T02:09:57.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.124+0000 7f3dfb6e6640 1 -- 172.21.15.49:0/3514192258 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3df4074040 con 0x7f3df40d2190 2023-11-06T02:09:57.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.124+0000 7f3de9ffb640 1 -- 172.21.15.49:0/3514192258 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f3de40b3d10 con 0x7f3df40d2190 2023-11-06T02:09:57.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.124+0000 7f3de9ffb640 1 --2- 172.21.15.49:0/3514192258 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3dd8065a60 0x7f3dd8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:57.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.124+0000 7f3de9ffb640 1 -- 172.21.15.49:0/3514192258 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3de411d6a0 con 0x7f3df40d2190 2023-11-06T02:09:57.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.124+0000 7f3df9482640 1 --2- 172.21.15.49:0/3514192258 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3dd8065a60 0x7f3dd8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:57.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.124+0000 7f3df9482640 1 --2- 172.21.15.49:0/3514192258 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3dd8065a60 0x7f3dd8067f20 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f3df415b5d0 tx=0x7f3df0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:57.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.128+0000 7f3de9ffb640 1 -- 172.21.15.49:0/3514192258 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f3de40eafb0 con 0x7f3df40d2190 2023-11-06T02:09:57.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.300+0000 7f3dfb6e6640 1 -- 172.21.15.49:0/3514192258 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f3df40cb310 con 0x7f3df40d2190 2023-11-06T02:09:57.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.300+0000 7f3de9ffb640 1 -- 172.21.15.49:0/3514192258 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v37) v1 ==== 80+0+0 (secure 0 0 0) 0x7f3de40ee5d0 con 0x7f3df40d2190 2023-11-06T02:09:57.299 INFO:teuthology.orchestra.run.smithi049.stderr:listed 0 entries 2023-11-06T02:09:57.301 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.300+0000 7f3dfb6e6640 1 -- 172.21.15.49:0/3514192258 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3dd8065a60 msgr2=0x7f3dd8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:57.301 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.300+0000 7f3dfb6e6640 1 --2- 172.21.15.49:0/3514192258 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3dd8065a60 0x7f3dd8067f20 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f3df415b5d0 tx=0x7f3df0008040 comp rx=0 tx=0).stop 2023-11-06T02:09:57.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.300+0000 7f3dfb6e6640 1 -- 172.21.15.49:0/3514192258 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3df40d2190 msgr2=0x7f3df4155eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:57.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.300+0000 7f3dfb6e6640 1 --2- 172.21.15.49:0/3514192258 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3df40d2190 0x7f3df4155eb0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f3de40190e0 tx=0x7f3de401ad10 comp rx=0 tx=0).stop 2023-11-06T02:09:57.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.300+0000 7f3dfb6e6640 1 -- 172.21.15.49:0/3514192258 shutdown_connections 2023-11-06T02:09:57.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.300+0000 7f3dfb6e6640 1 --2- 172.21.15.49:0/3514192258 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3dd8065a60 0x7f3dd8067f20 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:57.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.300+0000 7f3dfb6e6640 1 --2- 172.21.15.49:0/3514192258 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3df40d2190 0x7f3df4155eb0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:57.303 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.300+0000 7f3dfb6e6640 1 --2- 172.21.15.49:0/3514192258 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3df4161650 0x7f3df415f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:57.303 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.300+0000 7f3dfb6e6640 1 --2- 172.21.15.49:0/3514192258 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3df40b1f90 0x7f3df415d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:57.303 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.300+0000 7f3dfb6e6640 1 -- 172.21.15.49:0/3514192258 >> 172.21.15.49:0/3514192258 conn(0x7f3df40d2aa0 msgr2=0x7f3df40b6c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:57.303 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.304+0000 7f3dfb6e6640 1 -- 172.21.15.49:0/3514192258 shutdown_connections 2023-11-06T02:09:57.303 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.304+0000 7f3dfb6e6640 1 -- 172.21.15.49:0/3514192258 wait complete. 2023-11-06T02:09:57.318 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-11-06T02:09:57.318 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T02:09:57.323 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T02:09:57.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.492+0000 7f32946a3640 1 Processor -- start 2023-11-06T02:09:57.493 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.492+0000 7f32946a3640 1 -- start start 2023-11-06T02:09:57.496 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.496+0000 7f32946a3640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f328c0cb010 0x7f328c0c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:57.496 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.496+0000 7f32946a3640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f328c0cab80 con 0x7f328c0b0790 2023-11-06T02:09:57.496 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.496+0000 7f32946a3640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f328c15af30 con 0x7f328c15ab50 2023-11-06T02:09:57.497 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.496+0000 7f32946a3640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f328c0cb5b0 con 0x7f328c0cb010 2023-11-06T02:09:57.497 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.496+0000 7f3291c3e640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f328c0b0790 0x7f328c0c2710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:49400/0 (socket says 172.21.15.49:49400) 2023-11-06T02:09:57.497 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.496+0000 7f3291c3e640 1 -- 172.21.15.49:0/3113190552 learned_addr learned my addr 172.21.15.49:0/3113190552 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:09:57.497 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.496+0000 7f3292c40640 1 --2- 172.21.15.49:0/3113190552 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f328c0cb010 0x7f328c0c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:57.497 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.496+0000 7f329143d640 1 -- 172.21.15.49:0/3113190552 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 656112020 0 0) 0x7f328c0cab80 con 0x7f328c0b0790 2023-11-06T02:09:57.498 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.496+0000 7f329143d640 1 -- 172.21.15.49:0/3113190552 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3278003530 con 0x7f328c0b0790 2023-11-06T02:09:57.498 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.496+0000 7f329143d640 1 -- 172.21.15.49:0/3113190552 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4247805247 0 0) 0x7f3278003530 con 0x7f328c0b0790 2023-11-06T02:09:57.498 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.496+0000 7f329143d640 1 -- 172.21.15.49:0/3113190552 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f328c0cb010 msgr2=0x7f328c0c9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:57.498 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.496+0000 7f329143d640 1 --2- 172.21.15.49:0/3113190552 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f328c0cb010 0x7f328c0c9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:57.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.496+0000 7f329143d640 1 -- 172.21.15.49:0/3113190552 >> v1:172.21.15.150:6789/0 conn(0x7f328c15ab50 legacy=0x7f328c0b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:57.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.496+0000 7f329143d640 1 -- 172.21.15.49:0/3113190552 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f328c0cb890 con 0x7f328c0b0790 2023-11-06T02:09:57.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.496+0000 7f329143d640 1 -- 172.21.15.49:0/3113190552 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7f327c0016a0 con 0x7f328c0b0790 2023-11-06T02:09:57.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.496+0000 7f329143d640 1 -- 172.21.15.49:0/3113190552 >> v1:172.21.15.49:6789/0 conn(0x7f328c0b0790 legacy=0x7f328c0c2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:57.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.496+0000 7f329143d640 1 --2- 172.21.15.49:0/3113190552 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32780039b0 0x7f3278003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:57.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.496+0000 7f329143d640 1 --2- 172.21.15.49:0/3113190552 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f328c0cb010 0x7f32780047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:57.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.496+0000 7f329143d640 1 --2- 172.21.15.49:0/3113190552 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3278004d10 0x7f3278005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:57.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.496+0000 7f329143d640 1 -- 172.21.15.49:0/3113190552 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f328c0cb5b0 con 0x7f32780039b0 2023-11-06T02:09:57.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.496+0000 7f329143d640 1 -- 172.21.15.49:0/3113190552 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f327800ba40 con 0x7f3278004d10 2023-11-06T02:09:57.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.496+0000 7f329143d640 1 -- 172.21.15.49:0/3113190552 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f327800bc00 con 0x7f328c0cb010 2023-11-06T02:09:57.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.496+0000 7f329243f640 1 --2- 172.21.15.49:0/3113190552 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32780039b0 0x7f3278003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:57.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.496+0000 7f3291c3e640 1 --2- 172.21.15.49:0/3113190552 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f328c0cb010 0x7f32780047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:57.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.496+0000 7f3292c40640 1 --2- 172.21.15.49:0/3113190552 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3278004d10 0x7f3278005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:57.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.496+0000 7f329243f640 1 -- 172.21.15.49:0/3113190552 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f328c0cb010 msgr2=0x7f32780047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:57.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.496+0000 7f329243f640 1 --2- 172.21.15.49:0/3113190552 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f328c0cb010 0x7f32780047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:57.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.496+0000 7f329243f640 1 -- 172.21.15.49:0/3113190552 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3278004d10 msgr2=0x7f3278005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:57.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.496+0000 7f329243f640 1 --2- 172.21.15.49:0/3113190552 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3278004d10 0x7f3278005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:57.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.496+0000 7f329243f640 1 -- 172.21.15.49:0/3113190552 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3278003530 con 0x7f32780039b0 2023-11-06T02:09:57.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.496+0000 7f3291c3e640 1 --2- 172.21.15.49:0/3113190552 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f328c0cb010 0x7f32780047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:09:57.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.496+0000 7f3292c40640 1 --2- 172.21.15.49:0/3113190552 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3278004d10 0x7f3278005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:09:57.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.500+0000 7f329243f640 1 --2- 172.21.15.49:0/3113190552 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32780039b0 0x7f3278003dd0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f3288001780 tx=0x7f328801c370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d961c121dc19a94c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:57.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.500+0000 7f329143d640 1 -- 172.21.15.49:0/3113190552 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32880052d0 con 0x7f32780039b0 2023-11-06T02:09:57.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.500+0000 7f329143d640 1 -- 172.21.15.49:0/3113190552 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32880a7050 con 0x7f32780039b0 2023-11-06T02:09:57.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.500+0000 7f329143d640 1 -- 172.21.15.49:0/3113190552 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3288003650 con 0x7f32780039b0 2023-11-06T02:09:57.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.500+0000 7f32946a3640 1 -- 172.21.15.49:0/3113190552 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32780039b0 msgr2=0x7f3278003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:57.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.500+0000 7f32946a3640 1 --2- 172.21.15.49:0/3113190552 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32780039b0 0x7f3278003dd0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f3288001780 tx=0x7f328801c370 comp rx=0 tx=0).stop 2023-11-06T02:09:57.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.500+0000 7f3292c40640 1 -- 172.21.15.49:0/3113190552 reap_dead start 2023-11-06T02:09:57.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.500+0000 7f32946a3640 1 -- 172.21.15.49:0/3113190552 shutdown_connections 2023-11-06T02:09:57.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.500+0000 7f32946a3640 1 -- 172.21.15.49:0/3113190552 >> 172.21.15.49:0/3113190552 conn(0x7f328c06c0a0 msgr2=0x7f328c06c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:57.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.500+0000 7f32946a3640 1 -- 172.21.15.49:0/3113190552 shutdown_connections 2023-11-06T02:09:57.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.500+0000 7f32946a3640 1 -- 172.21.15.49:0/3113190552 wait complete. 2023-11-06T02:09:57.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.500+0000 7f32946a3640 1 Processor -- start 2023-11-06T02:09:57.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.500+0000 7f32946a3640 1 -- start start 2023-11-06T02:09:57.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.500+0000 7f32946a3640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32780039b0 0x7f328c14aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:57.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.500+0000 7f32946a3640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f328c15b2a0 0x7f328c152260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:57.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.500+0000 7f32946a3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f328c1527a0 0x7f328c152c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:57.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.500+0000 7f32946a3640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f328c14af90 con 0x7f328c1527a0 2023-11-06T02:09:57.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.500+0000 7f32946a3640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f328c0c5c50 con 0x7f32780039b0 2023-11-06T02:09:57.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.500+0000 7f32946a3640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f328c15df40 con 0x7f328c15b2a0 2023-11-06T02:09:57.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.504+0000 7f3292c40640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f328c1527a0 0x7f328c152c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:57.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.504+0000 7f3292c40640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f328c1527a0 0x7f328c152c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44816/0 (socket says 172.21.15.49:44816) 2023-11-06T02:09:57.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.504+0000 7f3292c40640 1 -- 172.21.15.49:0/1786013577 learned_addr learned my addr 172.21.15.49:0/1786013577 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:57.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.504+0000 7f3291c3e640 1 --2- 172.21.15.49:0/1786013577 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f328c15b2a0 0x7f328c152260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:57.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.504+0000 7f329243f640 1 --2- 172.21.15.49:0/1786013577 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32780039b0 0x7f328c14aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:57.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.504+0000 7f3292c40640 1 -- 172.21.15.49:0/1786013577 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f328c15b2a0 msgr2=0x7f328c152260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:57.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.504+0000 7f3292c40640 1 --2- 172.21.15.49:0/1786013577 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f328c15b2a0 0x7f328c152260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:57.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.504+0000 7f3292c40640 1 -- 172.21.15.49:0/1786013577 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32780039b0 msgr2=0x7f328c14aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:57.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.504+0000 7f3292c40640 1 --2- 172.21.15.49:0/1786013577 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32780039b0 0x7f328c14aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:57.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.504+0000 7f3292c40640 1 -- 172.21.15.49:0/1786013577 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f328c15e220 con 0x7f328c1527a0 2023-11-06T02:09:57.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.504+0000 7f3292c40640 1 --2- 172.21.15.49:0/1786013577 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f328c1527a0 0x7f328c152c20 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f328c0cf670 tx=0x7f3284010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:57.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.504+0000 7f3282ffd640 1 -- 172.21.15.49:0/1786013577 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f328400e050 con 0x7f328c1527a0 2023-11-06T02:09:57.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.504+0000 7f3282ffd640 1 -- 172.21.15.49:0/1786013577 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f328400f040 con 0x7f328c1527a0 2023-11-06T02:09:57.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.504+0000 7f32946a3640 1 -- 172.21.15.49:0/1786013577 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f328c0c5ea0 con 0x7f328c1527a0 2023-11-06T02:09:57.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.504+0000 7f3282ffd640 1 -- 172.21.15.49:0/1786013577 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32840039a0 con 0x7f328c1527a0 2023-11-06T02:09:57.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.504+0000 7f32946a3640 1 -- 172.21.15.49:0/1786013577 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f328c15b720 con 0x7f328c1527a0 2023-11-06T02:09:57.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.504+0000 7f32946a3640 1 -- 172.21.15.49:0/1786013577 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f328c0b0de0 con 0x7f328c1527a0 2023-11-06T02:09:57.514 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.504+0000 7f3282ffd640 1 -- 172.21.15.49:0/1786013577 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f3284015440 con 0x7f328c1527a0 2023-11-06T02:09:57.514 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.504+0000 7f3282ffd640 1 --2- 172.21.15.49:0/1786013577 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3270065a60 0x7f3270067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:57.514 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.504+0000 7f3282ffd640 1 -- 172.21.15.49:0/1786013577 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3284083770 con 0x7f328c1527a0 2023-11-06T02:09:57.514 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.504+0000 7f329243f640 1 --2- 172.21.15.49:0/1786013577 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3270065a60 0x7f3270067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:57.514 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.508+0000 7f329243f640 1 --2- 172.21.15.49:0/1786013577 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3270065a60 0x7f3270067f20 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f32880ac000 tx=0x7f32880011f0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:57.515 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.508+0000 7f3282ffd640 1 -- 172.21.15.49:0/1786013577 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f3284051060 con 0x7f328c1527a0 2023-11-06T02:09:57.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.668+0000 7f32946a3640 1 -- 172.21.15.49:0/1786013577 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f328c074040 con 0x7f328c1527a0 2023-11-06T02:09:57.669 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.668+0000 7f3282ffd640 1 -- 172.21.15.49:0/1786013577 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+17883 (secure 0 0 0) 0x7f3284054680 con 0x7f328c1527a0 2023-11-06T02:09:57.669 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:09:57.669 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":37,"fsid":"1a8b035c-6ba4-499a-b876-aea47fd9c0d4","created":"2023-11-06T02:08:06.440857+0000","modified":"2023-11-06T02:09:40.876751+0000","last_up_change":"2023-11-06T02:08:17.982372+0000","last_in_change":"2023-11-06T02:08:08.906631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T02:08:19.252222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T02:08:24.776211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-06T02:09:37.906848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-06T02:09:38.849380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ad55af8-0571-4e6a-8722-b5d429b7496d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6809","nonce":38116},{"type":"v1","addr":"172.21.15.49:6811","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6813","nonce":38116},{"type":"v1","addr":"172.21.15.49:6815","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6821","nonce":38116},{"type":"v1","addr":"172.21.15.49:6823","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6817","nonce":38116},{"type":"v1","addr":"172.21.15.49:6819","nonce":38116}]},"public_addr":"172.21.15.49:6811/38116","cluster_addr":"172.21.15.49:6815/38116","heartbeat_back_addr":"172.21.15.49:6823/38116","heartbeat_front_addr":"172.21.15.49:6819/38116","state":["exists","up"]},{"osd":1,"uuid":"c8b31f85-a757-43ed-9419-351acc46611b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":38113},{"type":"v1","addr":"172.21.15.49:6810","nonce":38113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":38113},{"type":"v1","addr":"172.21.15.49:6814","nonce":38113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":38113},{"type":"v1","addr":"172.21.15.49:6822","nonce":38113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":38113},{"type":"v1","addr":"172.21.15.49:6818","nonce":38113}]},"public_addr":"172.21.15.49:6810/38113","cluster_addr":"172.21.15.49:6814/38113","heartbeat_back_addr":"172.21.15.49:6822/38113","heartbeat_front_addr":"172.21.15.49:6818/38113","state":["exists","up"]},{"osd":2,"uuid":"555b3bf3-0a70-4b91-8efd-e4204b5fc26e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":38114},{"type":"v1","addr":"172.21.15.49:6801","nonce":38114}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":38114},{"type":"v1","addr":"172.21.15.49:6803","nonce":38114}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":38114},{"type":"v1","addr":"172.21.15.49:6807","nonce":38114}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":38114},{"type":"v1","addr":"172.21.15.49:6805","nonce":38114}]},"public_addr":"172.21.15.49:6801/38114","cluster_addr":"172.21.15.49:6803/38114","heartbeat_back_addr":"172.21.15.49:6807/38114","heartbeat_front_addr":"172.21.15.49:6805/38114","state":["exists","up"]},{"osd":3,"uuid":"8d4e5d77-03ef-4b9b-bfcb-5ae93a46795c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":38115},{"type":"v1","addr":"172.21.15.49:6825","nonce":38115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":38115},{"type":"v1","addr":"172.21.15.49:6827","nonce":38115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":38115},{"type":"v1","addr":"172.21.15.49:6831","nonce":38115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":38115},{"type":"v1","addr":"172.21.15.49:6829","nonce":38115}]},"public_addr":"172.21.15.49:6825/38115","cluster_addr":"172.21.15.49:6827/38115","heartbeat_back_addr":"172.21.15.49:6831/38115","heartbeat_front_addr":"172.21.15.49:6829/38115","state":["exists","up"]},{"osd":4,"uuid":"dba03ad4-68a3-4cf3-9218-ac611f02eacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":38417},{"type":"v1","addr":"172.21.15.150:6809","nonce":38417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":38417},{"type":"v1","addr":"172.21.15.150:6811","nonce":38417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":38417},{"type":"v1","addr":"172.21.15.150:6815","nonce":38417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":38417},{"type":"v1","addr":"172.21.15.150:6813","nonce":38417}]},"public_addr":"172.21.15.150:6809/38417","cluster_addr":"172.21.15.150:6811/38417","heartbeat_back_addr":"172.21.15.150:6815/38417","heartbeat_front_addr":"172.21.15.150:6813/38417","state":["exists","up"]},{"osd":5,"uuid":"0df9c303-c255-4640-8e91-01f1c3efe18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":38418},{"type":"v1","addr":"172.21.15.150:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":38418},{"type":"v1","addr":"172.21.15.150:6803","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":38418},{"type":"v1","addr":"172.21.15.150:6807","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":38418},{"type":"v1","addr":"172.21.15.150:6805","nonce":38418}]},"public_addr":"172.21.15.150:6801/38418","cluster_addr":"172.21.15.150:6803/38418","heartbeat_back_addr":"172.21.15.150:6807/38418","heartbeat_front_addr":"172.21.15.150:6805/38418","state":["exists","up"]},{"osd":6,"uuid":"e59fa189-4b70-4c06-9722-c02b90828bf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":38428},{"type":"v1","addr":"172.21.15.150:6825","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":38428},{"type":"v1","addr":"172.21.15.150:6827","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":38428},{"type":"v1","addr":"172.21.15.150:6831","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":38428},{"type":"v1","addr":"172.21.15.150:6829","nonce":38428}]},"public_addr":"172.21.15.150:6825/38428","cluster_addr":"172.21.15.150:6827/38428","heartbeat_back_addr":"172.21.15.150:6831/38428","heartbeat_front_addr":"172.21.15.150:6829/38428","state":["exists","up"]},{"osd":7,"uuid":"a64f57e1-ef98-46bd-b304-385759e95898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":38421},{"type":"v1","addr":"172.21.15.150:6817","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":38421},{"type":"v1","addr":"172.21.15.150:6819","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":38421},{"type":"v1","addr":"172.21.15.150:6823","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":38421},{"type":"v1","addr":"172.21.15.150:6821","nonce":38421}]},"public_addr":"172.21.15.150:6817/38421","cluster_addr":"172.21.15.150:6819/38421","heartbeat_back_addr":"172.21.15.150:6823/38421","heartbeat_front_addr":"172.21.15.150:6821/38421","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.438552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.413280+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.296466+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.424966+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.276953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.230984+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.410235+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.480084+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T02:09:57.672 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.672+0000 7f32946a3640 1 -- 172.21.15.49:0/1786013577 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3270065a60 msgr2=0x7f3270067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:57.672 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.672+0000 7f32946a3640 1 --2- 172.21.15.49:0/1786013577 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3270065a60 0x7f3270067f20 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f32880ac000 tx=0x7f32880011f0 comp rx=0 tx=0).stop 2023-11-06T02:09:57.673 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.672+0000 7f32946a3640 1 -- 172.21.15.49:0/1786013577 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f328c1527a0 msgr2=0x7f328c152c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:57.673 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.672+0000 7f32946a3640 1 --2- 172.21.15.49:0/1786013577 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f328c1527a0 0x7f328c152c20 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f328c0cf670 tx=0x7f3284010590 comp rx=0 tx=0).stop 2023-11-06T02:09:57.673 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.672+0000 7f32946a3640 1 -- 172.21.15.49:0/1786013577 shutdown_connections 2023-11-06T02:09:57.673 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.672+0000 7f32946a3640 1 --2- 172.21.15.49:0/1786013577 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3270065a60 0x7f3270067f20 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:57.673 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.672+0000 7f32946a3640 1 --2- 172.21.15.49:0/1786013577 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f328c1527a0 0x7f328c152c20 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:57.674 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.672+0000 7f32946a3640 1 --2- 172.21.15.49:0/1786013577 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f328c15b2a0 0x7f328c152260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:57.674 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.672+0000 7f32946a3640 1 --2- 172.21.15.49:0/1786013577 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32780039b0 0x7f328c14aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:57.674 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.672+0000 7f32946a3640 1 -- 172.21.15.49:0/1786013577 >> 172.21.15.49:0/1786013577 conn(0x7f328c06c0a0 msgr2=0x7f328c0aea70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:57.674 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.672+0000 7f32946a3640 1 -- 172.21.15.49:0/1786013577 shutdown_connections 2023-11-06T02:09:57.674 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.672+0000 7f32946a3640 1 -- 172.21.15.49:0/1786013577 wait complete. 2023-11-06T02:09:57.689 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T02:08:19.252222+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T02:08:24.776211+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T02:09:37.906848+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T02:09:38.849380+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T02:09:57.690 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T02:09:57.813 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.812+0000 7fc45d5f8640 1 Processor -- start 2023-11-06T02:09:57.813 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.812+0000 7fc45d5f8640 1 -- start start 2023-11-06T02:09:57.814 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.816+0000 7fc45d5f8640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc45815a880 0x7fc4580b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:57.814 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.816+0000 7fc45d5f8640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc45806d860 0x7fc458070610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:57.815 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.816+0000 7fc45d5f8640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4580ae360 con 0x7fc458070160 2023-11-06T02:09:57.815 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.816+0000 7fc45d5f8640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc458070b50 con 0x7fc45806d860 2023-11-06T02:09:57.815 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.816+0000 7fc45d5f8640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4580ae640 con 0x7fc45815a880 2023-11-06T02:09:57.815 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.816+0000 7fc44e7fc640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fc458070160 0x7fc45806d150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:49402/0 (socket says 172.21.15.49:49402) 2023-11-06T02:09:57.815 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.816+0000 7fc44e7fc640 1 -- 172.21.15.49:0/933080649 learned_addr learned my addr 172.21.15.49:0/933080649 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:09:57.815 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.816+0000 7fc4577fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc45806d860 0x7fc458070610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:57.816 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.816+0000 7fc4567fc640 1 -- 172.21.15.49:0/933080649 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 558599497 0 0) 0x7fc4580ae360 con 0x7fc458070160 2023-11-06T02:09:57.816 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.816+0000 7fc456ffd640 1 --2- 172.21.15.49:0/933080649 >> v2:172.21.15.150:3300/0 conn(0x7fc45815a880 0x7fc4580b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:57.816 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.816+0000 7fc4567fc640 1 -- 172.21.15.49:0/933080649 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc444003530 con 0x7fc458070160 2023-11-06T02:09:57.816 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.816+0000 7fc4577fe640 1 -- 172.21.15.49:0/933080649 >> v2:172.21.15.150:3300/0 conn(0x7fc45815a880 msgr2=0x7fc4580b15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:57.816 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.816+0000 7fc4577fe640 1 --2- 172.21.15.49:0/933080649 >> v2:172.21.15.150:3300/0 conn(0x7fc45815a880 0x7fc4580b15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:57.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.816+0000 7fc4577fe640 1 -- 172.21.15.49:0/933080649 >> v1:172.21.15.49:6789/0 conn(0x7fc458070160 legacy=0x7fc45806d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:57.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.816+0000 7fc4577fe640 1 -- 172.21.15.49:0/933080649 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4580ae920 con 0x7fc45806d860 2023-11-06T02:09:57.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.820+0000 7fc4577fe640 1 --2- 172.21.15.49:0/933080649 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc45806d860 0x7fc458070610 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fc4580cafc0 tx=0x7fc45001c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7d422eff1365740 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:57.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.820+0000 7fc4567fc640 1 -- 172.21.15.49:0/933080649 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4500a58e0 con 0x7fc45806d860 2023-11-06T02:09:57.818 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.820+0000 7fc4567fc640 1 -- 172.21.15.49:0/933080649 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4500a5a40 con 0x7fc45806d860 2023-11-06T02:09:57.818 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.820+0000 7fc4567fc640 1 -- 172.21.15.49:0/933080649 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4500a5db0 con 0x7fc45806d860 2023-11-06T02:09:57.819 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.820+0000 7fc45d5f8640 1 -- 172.21.15.49:0/933080649 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc45806d860 msgr2=0x7fc458070610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:57.819 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.820+0000 7fc45d5f8640 1 --2- 172.21.15.49:0/933080649 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc45806d860 0x7fc458070610 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fc4580cafc0 tx=0x7fc45001c910 comp rx=0 tx=0).stop 2023-11-06T02:09:57.819 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.820+0000 7fc45d5f8640 1 -- 172.21.15.49:0/933080649 shutdown_connections 2023-11-06T02:09:57.819 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.820+0000 7fc45d5f8640 1 --2- 172.21.15.49:0/933080649 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc45806d860 0x7fc458070610 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:57.819 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.820+0000 7fc45d5f8640 1 --2- 172.21.15.49:0/933080649 >> v2:172.21.15.150:3300/0 conn(0x7fc45815a880 0x7fc4580b15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:57.820 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.820+0000 7fc45d5f8640 1 -- 172.21.15.49:0/933080649 >> 172.21.15.49:0/933080649 conn(0x7fc45806a370 msgr2=0x7fc4580ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:57.820 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.820+0000 7fc45d5f8640 1 -- 172.21.15.49:0/933080649 shutdown_connections 2023-11-06T02:09:57.820 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.820+0000 7fc45d5f8640 1 -- 172.21.15.49:0/933080649 wait complete. 2023-11-06T02:09:57.822 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.820+0000 7fc45d5f8640 1 Processor -- start 2023-11-06T02:09:57.822 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.820+0000 7fc45d5f8640 1 -- start start 2023-11-06T02:09:57.822 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.820+0000 7fc45d5f8640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc45806d860 0x7fc45814d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:57.823 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.820+0000 7fc45d5f8640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc458070160 0x7fc458143f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:57.823 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.820+0000 7fc45d5f8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc45815a880 0x7fc458147ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:57.823 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.820+0000 7fc45d5f8640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc45814d6b0 con 0x7fc45815a880 2023-11-06T02:09:57.823 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.820+0000 7fc45d5f8640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc45814b600 con 0x7fc45806d860 2023-11-06T02:09:57.823 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.820+0000 7fc45d5f8640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc45814b8e0 con 0x7fc458070160 2023-11-06T02:09:57.824 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.820+0000 7fc4577fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc45815a880 0x7fc458147ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:57.824 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.820+0000 7fc4577fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc45815a880 0x7fc458147ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44832/0 (socket says 172.21.15.49:44832) 2023-11-06T02:09:57.824 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.820+0000 7fc4577fe640 1 -- 172.21.15.49:0/1906887693 learned_addr learned my addr 172.21.15.49:0/1906887693 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:57.824 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.820+0000 7fc44e7fc640 1 --2- 172.21.15.49:0/1906887693 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc458070160 0x7fc458143f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:57.824 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.820+0000 7fc456ffd640 1 --2- 172.21.15.49:0/1906887693 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc45806d860 0x7fc45814d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:57.824 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.820+0000 7fc4577fe640 1 -- 172.21.15.49:0/1906887693 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc458070160 msgr2=0x7fc458143f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:57.825 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.820+0000 7fc4577fe640 1 --2- 172.21.15.49:0/1906887693 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc458070160 0x7fc458143f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:57.825 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.820+0000 7fc4577fe640 1 -- 172.21.15.49:0/1906887693 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc45806d860 msgr2=0x7fc45814d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:57.825 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.820+0000 7fc4577fe640 1 --2- 172.21.15.49:0/1906887693 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc45806d860 0x7fc45814d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:57.825 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.820+0000 7fc4577fe640 1 -- 172.21.15.49:0/1906887693 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc450009a70 con 0x7fc45815a880 2023-11-06T02:09:57.825 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.820+0000 7fc44e7fc640 1 --2- 172.21.15.49:0/1906887693 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc458070160 0x7fc458143f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:09:57.826 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.820+0000 7fc4577fe640 1 --2- 172.21.15.49:0/1906887693 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc45815a880 0x7fc458147ee0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fc450019520 tx=0x7fc4500196b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:57.826 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.824+0000 7fc44ffff640 1 -- 172.21.15.49:0/1906887693 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4500ad850 con 0x7fc45815a880 2023-11-06T02:09:57.826 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.824+0000 7fc44ffff640 1 -- 172.21.15.49:0/1906887693 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4500ad9b0 con 0x7fc45815a880 2023-11-06T02:09:57.826 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.824+0000 7fc45d5f8640 1 -- 172.21.15.49:0/1906887693 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc45814bbc0 con 0x7fc45815a880 2023-11-06T02:09:57.826 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.824+0000 7fc44ffff640 1 -- 172.21.15.49:0/1906887693 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4500add20 con 0x7fc45815a880 2023-11-06T02:09:57.826 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.824+0000 7fc45d5f8640 1 -- 172.21.15.49:0/1906887693 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc45814c180 con 0x7fc45815a880 2023-11-06T02:09:57.827 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.824+0000 7fc45d5f8640 1 -- 172.21.15.49:0/1906887693 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4580b05f0 con 0x7fc45815a880 2023-11-06T02:09:57.828 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.824+0000 7fc44ffff640 1 -- 172.21.15.49:0/1906887693 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fc4500b4070 con 0x7fc45815a880 2023-11-06T02:09:57.828 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.824+0000 7fc44ffff640 1 --2- 172.21.15.49:0/1906887693 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc438065a60 0x7fc438067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:57.828 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.824+0000 7fc44ffff640 1 -- 172.21.15.49:0/1906887693 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc45011e050 con 0x7fc45815a880 2023-11-06T02:09:57.828 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.824+0000 7fc456ffd640 1 --2- 172.21.15.49:0/1906887693 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc438065a60 0x7fc438067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:57.828 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.824+0000 7fc456ffd640 1 --2- 172.21.15.49:0/1906887693 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc438065a60 0x7fc438067f20 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc4480096f0 tx=0x7fc448008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:57.829 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.828+0000 7fc44ffff640 1 -- 172.21.15.49:0/1906887693 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fc4500eb820 con 0x7fc45815a880 2023-11-06T02:09:57.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.976+0000 7fc45d5f8640 1 -- 172.21.15.49:0/1906887693 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc45814c370 con 0x7fc45815a880 2023-11-06T02:09:57.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.976+0000 7fc44ffff640 1 -- 172.21.15.49:0/1906887693 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v37) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc4500b2020 con 0x7fc45815a880 2023-11-06T02:09:57.976 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2023-11-06T02:09:57.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.980+0000 7fc45d5f8640 1 -- 172.21.15.49:0/1906887693 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc438065a60 msgr2=0x7fc438067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:57.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.980+0000 7fc45d5f8640 1 --2- 172.21.15.49:0/1906887693 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc438065a60 0x7fc438067f20 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc4480096f0 tx=0x7fc448008040 comp rx=0 tx=0).stop 2023-11-06T02:09:57.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.980+0000 7fc45d5f8640 1 -- 172.21.15.49:0/1906887693 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc45815a880 msgr2=0x7fc458147ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:57.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.980+0000 7fc45d5f8640 1 --2- 172.21.15.49:0/1906887693 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc45815a880 0x7fc458147ee0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fc450019520 tx=0x7fc4500196b0 comp rx=0 tx=0).stop 2023-11-06T02:09:57.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.980+0000 7fc45d5f8640 1 -- 172.21.15.49:0/1906887693 shutdown_connections 2023-11-06T02:09:57.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.980+0000 7fc45d5f8640 1 --2- 172.21.15.49:0/1906887693 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc438065a60 0x7fc438067f20 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:57.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.980+0000 7fc45d5f8640 1 --2- 172.21.15.49:0/1906887693 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc45815a880 0x7fc458147ee0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:57.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.980+0000 7fc45d5f8640 1 --2- 172.21.15.49:0/1906887693 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc458070160 0x7fc458143f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:57.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.980+0000 7fc45d5f8640 1 --2- 172.21.15.49:0/1906887693 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc45806d860 0x7fc45814d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:57.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.980+0000 7fc45d5f8640 1 -- 172.21.15.49:0/1906887693 >> 172.21.15.49:0/1906887693 conn(0x7fc45806a370 msgr2=0x7fc45815ae70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:57.980 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.980+0000 7fc45d5f8640 1 -- 172.21.15.49:0/1906887693 shutdown_connections 2023-11-06T02:09:57.980 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:57.980+0000 7fc45d5f8640 1 -- 172.21.15.49:0/1906887693 wait complete. 2023-11-06T02:09:57.993 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T02:09:58.118 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.120+0000 7ff49ba7a640 1 Processor -- start 2023-11-06T02:09:58.118 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.120+0000 7ff49ba7a640 1 -- start start 2023-11-06T02:09:58.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.120+0000 7ff49ba7a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4940d2ed0 0x7ff4940b6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:58.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.120+0000 7ff49ba7a640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff49415cf90 con 0x7ff4940d2a60 2023-11-06T02:09:58.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.120+0000 7ff49ba7a640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4940d3360 con 0x7ff49415cbb0 2023-11-06T02:09:58.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.120+0000 7ff49ba7a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4940d0b80 con 0x7ff4940d2ed0 2023-11-06T02:09:58.122 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.120+0000 7ff499015640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7ff4940d2a60 0x7ff4940b5d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:49412/0 (socket says 172.21.15.49:49412) 2023-11-06T02:09:58.122 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.120+0000 7ff499015640 1 -- 172.21.15.49:0/1002986886 learned_addr learned my addr 172.21.15.49:0/1002986886 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:09:58.122 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.120+0000 7ff49a017640 1 --2- 172.21.15.49:0/1002986886 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4940d2ed0 0x7ff4940b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:58.122 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.120+0000 7ff498814640 1 -- 172.21.15.49:0/1002986886 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 328394266 0 0) 0x7ff49415cf90 con 0x7ff4940d2a60 2023-11-06T02:09:58.122 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.120+0000 7ff498814640 1 -- 172.21.15.49:0/1002986886 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff480003530 con 0x7ff4940d2a60 2023-11-06T02:09:58.123 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.120+0000 7ff498814640 1 -- 172.21.15.49:0/1002986886 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3538730782 0 0) 0x7ff480003530 con 0x7ff4940d2a60 2023-11-06T02:09:58.123 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.120+0000 7ff498814640 1 -- 172.21.15.49:0/1002986886 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4940d2ed0 msgr2=0x7ff4940b6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:58.123 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.120+0000 7ff498814640 1 --2- 172.21.15.49:0/1002986886 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4940d2ed0 0x7ff4940b6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:58.123 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.120+0000 7ff498814640 1 -- 172.21.15.49:0/1002986886 >> v1:172.21.15.150:6789/0 conn(0x7ff49415cbb0 legacy=0x7ff4940d1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:58.123 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.120+0000 7ff498814640 1 -- 172.21.15.49:0/1002986886 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4940d0e60 con 0x7ff4940d2a60 2023-11-06T02:09:58.123 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.120+0000 7ff498814640 1 -- 172.21.15.49:0/1002986886 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7ff4840016a0 con 0x7ff4940d2a60 2023-11-06T02:09:58.124 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.120+0000 7ff498814640 1 -- 172.21.15.49:0/1002986886 >> v1:172.21.15.49:6789/0 conn(0x7ff4940d2a60 legacy=0x7ff4940b5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:58.124 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.120+0000 7ff498814640 1 --2- 172.21.15.49:0/1002986886 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4800039b0 0x7ff480003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:58.124 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.120+0000 7ff498814640 1 --2- 172.21.15.49:0/1002986886 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4800047d0 0x7ff480004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:58.124 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.120+0000 7ff498814640 1 --2- 172.21.15.49:0/1002986886 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4940d2ed0 0x7ff480005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:58.124 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.120+0000 7ff498814640 1 -- 172.21.15.49:0/1002986886 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4940d0b80 con 0x7ff4800047d0 2023-11-06T02:09:58.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.120+0000 7ff498814640 1 -- 172.21.15.49:0/1002986886 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff48000bb00 con 0x7ff4800039b0 2023-11-06T02:09:58.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.120+0000 7ff498814640 1 -- 172.21.15.49:0/1002986886 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff48000bcc0 con 0x7ff4940d2ed0 2023-11-06T02:09:58.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.120+0000 7ff499015640 1 --2- 172.21.15.49:0/1002986886 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4800047d0 0x7ff480004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:58.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.120+0000 7ff49a017640 1 --2- 172.21.15.49:0/1002986886 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4940d2ed0 0x7ff480005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:58.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.120+0000 7ff499816640 1 --2- 172.21.15.49:0/1002986886 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4800039b0 0x7ff480003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:58.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.120+0000 7ff499015640 1 -- 172.21.15.49:0/1002986886 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4940d2ed0 msgr2=0x7ff480005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:58.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.120+0000 7ff499015640 1 --2- 172.21.15.49:0/1002986886 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4940d2ed0 0x7ff480005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:58.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.120+0000 7ff499015640 1 -- 172.21.15.49:0/1002986886 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4800039b0 msgr2=0x7ff480003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:58.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.120+0000 7ff499015640 1 --2- 172.21.15.49:0/1002986886 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4800039b0 0x7ff480003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:58.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.120+0000 7ff499015640 1 -- 172.21.15.49:0/1002986886 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff480003530 con 0x7ff4800047d0 2023-11-06T02:09:58.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.120+0000 7ff499816640 1 --2- 172.21.15.49:0/1002986886 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4800039b0 0x7ff480003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:09:58.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.124+0000 7ff499015640 1 --2- 172.21.15.49:0/1002986886 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4800047d0 0x7ff480004c50 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7ff4940cd0d0 tx=0x7ff48401cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40d9ba04a9215beb server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:58.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.124+0000 7ff498814640 1 -- 172.21.15.49:0/1002986886 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff484005ae0 con 0x7ff4800047d0 2023-11-06T02:09:58.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.124+0000 7ff498814640 1 -- 172.21.15.49:0/1002986886 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff484005c40 con 0x7ff4800047d0 2023-11-06T02:09:58.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.124+0000 7ff498814640 1 -- 172.21.15.49:0/1002986886 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4840a58c0 con 0x7ff4800047d0 2023-11-06T02:09:58.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.124+0000 7ff49ba7a640 1 -- 172.21.15.49:0/1002986886 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4800047d0 msgr2=0x7ff480004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:58.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.124+0000 7ff49ba7a640 1 --2- 172.21.15.49:0/1002986886 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4800047d0 0x7ff480004c50 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7ff4940cd0d0 tx=0x7ff48401cbc0 comp rx=0 tx=0).stop 2023-11-06T02:09:58.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.124+0000 7ff49a017640 1 -- 172.21.15.49:0/1002986886 reap_dead start 2023-11-06T02:09:58.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.124+0000 7ff49ba7a640 1 -- 172.21.15.49:0/1002986886 shutdown_connections 2023-11-06T02:09:58.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.124+0000 7ff49ba7a640 1 -- 172.21.15.49:0/1002986886 >> 172.21.15.49:0/1002986886 conn(0x7ff4940718a0 msgr2=0x7ff494071cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:58.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.124+0000 7ff49ba7a640 1 -- 172.21.15.49:0/1002986886 shutdown_connections 2023-11-06T02:09:58.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.124+0000 7ff49ba7a640 1 -- 172.21.15.49:0/1002986886 wait complete. 2023-11-06T02:09:58.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.124+0000 7ff49ba7a640 1 Processor -- start 2023-11-06T02:09:58.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.124+0000 7ff49ba7a640 1 -- start start 2023-11-06T02:09:58.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.128+0000 7ff49ba7a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4800047d0 0x7ff4940c8590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:58.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.128+0000 7ff49ba7a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4941f2380 0x7ff494153e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:58.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.128+0000 7ff49ba7a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4940c9ba0 0x7ff4940ca050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:58.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.128+0000 7ff49ba7a640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4940ca590 con 0x7ff4941f2380 2023-11-06T02:09:58.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.128+0000 7ff49ba7a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4941f3390 con 0x7ff4940c9ba0 2023-11-06T02:09:58.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.128+0000 7ff49ba7a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff494154380 con 0x7ff4800047d0 2023-11-06T02:09:58.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.128+0000 7ff499015640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4941f2380 0x7ff494153e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:58.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.128+0000 7ff499015640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4941f2380 0x7ff494153e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44838/0 (socket says 172.21.15.49:44838) 2023-11-06T02:09:58.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.128+0000 7ff499015640 1 -- 172.21.15.49:0/1049862278 learned_addr learned my addr 172.21.15.49:0/1049862278 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:58.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.128+0000 7ff499816640 1 --2- 172.21.15.49:0/1049862278 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4800047d0 0x7ff4940c8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:58.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.128+0000 7ff49a017640 1 --2- 172.21.15.49:0/1049862278 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4940c9ba0 0x7ff4940ca050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:58.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.128+0000 7ff499015640 1 -- 172.21.15.49:0/1049862278 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4800047d0 msgr2=0x7ff4940c8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:58.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.128+0000 7ff499015640 1 --2- 172.21.15.49:0/1049862278 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4800047d0 0x7ff4940c8590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:58.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.128+0000 7ff499015640 1 -- 172.21.15.49:0/1049862278 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4940c9ba0 msgr2=0x7ff4940ca050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:58.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.128+0000 7ff499015640 1 --2- 172.21.15.49:0/1049862278 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4940c9ba0 0x7ff4940ca050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:58.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.128+0000 7ff499015640 1 -- 172.21.15.49:0/1049862278 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4941f2810 con 0x7ff4941f2380 2023-11-06T02:09:58.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.128+0000 7ff499816640 1 --2- 172.21.15.49:0/1049862278 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4800047d0 0x7ff4940c8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:09:58.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.128+0000 7ff499015640 1 --2- 172.21.15.49:0/1049862278 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4941f2380 0x7ff494153e40 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7ff48401c9f0 tx=0x7ff480005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:58.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.128+0000 7ff48a7fc640 1 -- 172.21.15.49:0/1049862278 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4840b4070 con 0x7ff4941f2380 2023-11-06T02:09:58.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.128+0000 7ff48a7fc640 1 -- 172.21.15.49:0/1049862278 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4840017b0 con 0x7ff4941f2380 2023-11-06T02:09:58.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.128+0000 7ff48a7fc640 1 -- 172.21.15.49:0/1049862278 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4840a5ac0 con 0x7ff4941f2380 2023-11-06T02:09:58.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.128+0000 7ff49ba7a640 1 -- 172.21.15.49:0/1049862278 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4941f2af0 con 0x7ff4941f2380 2023-11-06T02:09:58.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.128+0000 7ff49ba7a640 1 -- 172.21.15.49:0/1049862278 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4941f30b0 con 0x7ff4941f2380 2023-11-06T02:09:58.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.128+0000 7ff49ba7a640 1 -- 172.21.15.49:0/1049862278 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff494074040 con 0x7ff4941f2380 2023-11-06T02:09:58.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.128+0000 7ff48a7fc640 1 -- 172.21.15.49:0/1049862278 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7ff4840a5c20 con 0x7ff4941f2380 2023-11-06T02:09:58.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.128+0000 7ff48a7fc640 1 --2- 172.21.15.49:0/1049862278 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff478065a60 0x7ff478067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:58.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.128+0000 7ff48a7fc640 1 -- 172.21.15.49:0/1049862278 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff48411d390 con 0x7ff4941f2380 2023-11-06T02:09:58.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.128+0000 7ff499816640 1 --2- 172.21.15.49:0/1049862278 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff478065a60 0x7ff478067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:58.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.132+0000 7ff499816640 1 --2- 172.21.15.49:0/1049862278 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff478065a60 0x7ff478067f20 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff490007fd0 tx=0x7ff490009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:58.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.132+0000 7ff48a7fc640 1 -- 172.21.15.49:0/1049862278 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7ff4840eac80 con 0x7ff4941f2380 2023-11-06T02:09:58.289 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.292+0000 7ff49ba7a640 1 -- 172.21.15.49:0/1049862278 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff49415b3b0 con 0x7ff4941f2380 2023-11-06T02:09:58.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.292+0000 7ff48a7fc640 1 -- 172.21.15.49:0/1049862278 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v37) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff4840ee2a0 con 0x7ff4941f2380 2023-11-06T02:09:58.290 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 8 2023-11-06T02:09:58.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.292+0000 7ff49ba7a640 1 -- 172.21.15.49:0/1049862278 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff478065a60 msgr2=0x7ff478067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:58.293 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.292+0000 7ff49ba7a640 1 --2- 172.21.15.49:0/1049862278 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff478065a60 0x7ff478067f20 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff490007fd0 tx=0x7ff490009040 comp rx=0 tx=0).stop 2023-11-06T02:09:58.293 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.292+0000 7ff49ba7a640 1 -- 172.21.15.49:0/1049862278 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4941f2380 msgr2=0x7ff494153e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:58.293 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.292+0000 7ff49ba7a640 1 --2- 172.21.15.49:0/1049862278 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4941f2380 0x7ff494153e40 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7ff48401c9f0 tx=0x7ff480005800 comp rx=0 tx=0).stop 2023-11-06T02:09:58.293 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.292+0000 7ff49ba7a640 1 -- 172.21.15.49:0/1049862278 shutdown_connections 2023-11-06T02:09:58.293 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.292+0000 7ff49ba7a640 1 --2- 172.21.15.49:0/1049862278 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff478065a60 0x7ff478067f20 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:58.293 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.292+0000 7ff49ba7a640 1 --2- 172.21.15.49:0/1049862278 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff4941f2380 0x7ff494153e40 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:58.294 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.292+0000 7ff49ba7a640 1 --2- 172.21.15.49:0/1049862278 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4940c9ba0 0x7ff4940ca050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:58.294 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.292+0000 7ff49ba7a640 1 --2- 172.21.15.49:0/1049862278 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4800047d0 0x7ff4940c8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:58.294 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.292+0000 7ff49ba7a640 1 -- 172.21.15.49:0/1049862278 >> 172.21.15.49:0/1049862278 conn(0x7ff4940718a0 msgr2=0x7ff4940c9250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:58.294 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.292+0000 7ff49ba7a640 1 -- 172.21.15.49:0/1049862278 shutdown_connections 2023-11-06T02:09:58.294 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.292+0000 7ff49ba7a640 1 -- 172.21.15.49:0/1049862278 wait complete. 2023-11-06T02:09:58.309 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T02:09:58.433 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.436+0000 7f449839d640 1 Processor -- start 2023-11-06T02:09:58.434 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.436+0000 7f449839d640 1 -- start start 2023-11-06T02:09:58.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.436+0000 7f449839d640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f449015a880 0x7f44900b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:58.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.436+0000 7f449839d640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4490070160 0x7f4490070610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:58.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.436+0000 7f449839d640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44900ae360 con 0x7f449006d150 2023-11-06T02:09:58.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.436+0000 7f449839d640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4490070b50 con 0x7f449015a880 2023-11-06T02:09:58.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.436+0000 7f449839d640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44900ae640 con 0x7f4490070160 2023-11-06T02:09:58.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.436+0000 7f4496139640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f449015a880 0x7f44900b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:58.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.436+0000 7f449693a640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f449006d150 0x7f449006d5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:49428/0 (socket says 172.21.15.49:49428) 2023-11-06T02:09:58.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.436+0000 7f4496139640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f449015a880 0x7f44900b15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44846/0 (socket says 172.21.15.49:44846) 2023-11-06T02:09:58.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.436+0000 7f449693a640 1 -- 172.21.15.49:0/682065948 learned_addr learned my addr 172.21.15.49:0/682065948 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:09:58.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.436+0000 7f4495938640 1 --2- 172.21.15.49:0/682065948 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4490070160 0x7f4490070610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:58.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.436+0000 7f4496139640 1 -- 172.21.15.49:0/682065948 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4490070160 msgr2=0x7f4490070610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:58.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.436+0000 7f4496139640 1 --2- 172.21.15.49:0/682065948 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4490070160 0x7f4490070610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:58.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.436+0000 7f4496139640 1 -- 172.21.15.49:0/682065948 >> v1:172.21.15.49:6789/0 conn(0x7f449006d150 legacy=0x7f449006d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:58.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.436+0000 7f4495137640 1 -- 172.21.15.49:0/682065948 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 87130788 0 0) 0x7f44900ae360 con 0x7f449006d150 2023-11-06T02:09:58.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.436+0000 7f4496139640 1 -- 172.21.15.49:0/682065948 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44900ae920 con 0x7f449015a880 2023-11-06T02:09:58.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.440+0000 7f4496139640 1 --2- 172.21.15.49:0/682065948 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f449015a880 0x7f44900b15c0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f44800097c0 tx=0x7f448001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a92b43ed8a46e5de server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:58.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.440+0000 7f4495137640 1 -- 172.21.15.49:0/682065948 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44800ad640 con 0x7f449015a880 2023-11-06T02:09:58.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.440+0000 7f4495137640 1 -- 172.21.15.49:0/682065948 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44800ad7a0 con 0x7f449015a880 2023-11-06T02:09:58.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.440+0000 7f4495137640 1 -- 172.21.15.49:0/682065948 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44800adb30 con 0x7f449015a880 2023-11-06T02:09:58.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.440+0000 7f449839d640 1 -- 172.21.15.49:0/682065948 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f449015a880 msgr2=0x7f44900b15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:58.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.440+0000 7f449839d640 1 --2- 172.21.15.49:0/682065948 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f449015a880 0x7f44900b15c0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f44800097c0 tx=0x7f448001c480 comp rx=0 tx=0).stop 2023-11-06T02:09:58.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.440+0000 7f449839d640 1 -- 172.21.15.49:0/682065948 shutdown_connections 2023-11-06T02:09:58.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.440+0000 7f449839d640 1 --2- 172.21.15.49:0/682065948 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4490070160 0x7f4490070610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:58.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.440+0000 7f449839d640 1 --2- 172.21.15.49:0/682065948 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f449015a880 0x7f44900b15c0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:58.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.440+0000 7f449839d640 1 -- 172.21.15.49:0/682065948 >> 172.21.15.49:0/682065948 conn(0x7f449006a370 msgr2=0x7f44900ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:58.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.440+0000 7f449839d640 1 -- 172.21.15.49:0/682065948 shutdown_connections 2023-11-06T02:09:58.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.440+0000 7f449839d640 1 -- 172.21.15.49:0/682065948 wait complete. 2023-11-06T02:09:58.444 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.440+0000 7f449839d640 1 Processor -- start 2023-11-06T02:09:58.444 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.440+0000 7f449839d640 1 -- start start 2023-11-06T02:09:58.444 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.440+0000 7f449839d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f449006d150 0x7f449014dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:58.445 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.440+0000 7f449839d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4490070160 0x7f44900c67d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:58.445 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.440+0000 7f449839d640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f449015a880 0x7f44900c6d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:58.445 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.440+0000 7f449839d640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44900c7c10 con 0x7f449006d150 2023-11-06T02:09:58.445 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.440+0000 7f449839d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44900c7370 con 0x7f4490070160 2023-11-06T02:09:58.445 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.440+0000 7f449839d640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44900c7650 con 0x7f449015a880 2023-11-06T02:09:58.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.440+0000 7f4496139640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f449006d150 0x7f449014dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:58.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.440+0000 7f4496139640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f449006d150 0x7f449014dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44854/0 (socket says 172.21.15.49:44854) 2023-11-06T02:09:58.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.440+0000 7f4496139640 1 -- 172.21.15.49:0/2426623917 learned_addr learned my addr 172.21.15.49:0/2426623917 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:58.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.440+0000 7f4496139640 1 -- 172.21.15.49:0/2426623917 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f449015a880 msgr2=0x7f44900c6d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:58.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.440+0000 7f4496139640 1 --2- 172.21.15.49:0/2426623917 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f449015a880 0x7f44900c6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:58.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.440+0000 7f4496139640 1 -- 172.21.15.49:0/2426623917 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4490070160 msgr2=0x7f44900c67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:58.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.440+0000 7f4496139640 1 --2- 172.21.15.49:0/2426623917 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4490070160 0x7f44900c67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:58.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.440+0000 7f4496139640 1 -- 172.21.15.49:0/2426623917 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4480009420 con 0x7f449006d150 2023-11-06T02:09:58.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.444+0000 7f4496139640 1 --2- 172.21.15.49:0/2426623917 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f449006d150 0x7f449014dff0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f44800b0040 tx=0x7f4480019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:58.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.444+0000 7f4486ffd640 1 -- 172.21.15.49:0/2426623917 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44800b6820 con 0x7f449006d150 2023-11-06T02:09:58.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.444+0000 7f4486ffd640 1 -- 172.21.15.49:0/2426623917 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44800a6040 con 0x7f449006d150 2023-11-06T02:09:58.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.444+0000 7f449839d640 1 -- 172.21.15.49:0/2426623917 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44900c7930 con 0x7f449006d150 2023-11-06T02:09:58.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.444+0000 7f4486ffd640 1 -- 172.21.15.49:0/2426623917 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44800b6bc0 con 0x7f449006d150 2023-11-06T02:09:58.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.444+0000 7f449839d640 1 -- 172.21.15.49:0/2426623917 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44900ccab0 con 0x7f449006d150 2023-11-06T02:09:58.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.444+0000 7f449839d640 1 -- 172.21.15.49:0/2426623917 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44900c3530 con 0x7f449006d150 2023-11-06T02:09:58.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.444+0000 7f4486ffd640 1 -- 172.21.15.49:0/2426623917 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f44800b4040 con 0x7f449006d150 2023-11-06T02:09:58.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.444+0000 7f4486ffd640 1 --2- 172.21.15.49:0/2426623917 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4474065a60 0x7f4474067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:58.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.444+0000 7f4486ffd640 1 -- 172.21.15.49:0/2426623917 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f448011d520 con 0x7f449006d150 2023-11-06T02:09:58.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.444+0000 7f4495938640 1 --2- 172.21.15.49:0/2426623917 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4474065a60 0x7f4474067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:58.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.444+0000 7f4495938640 1 --2- 172.21.15.49:0/2426623917 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4474065a60 0x7f4474067f20 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f44900cd1e0 tx=0x7f448c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:58.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.448+0000 7f4486ffd640 1 -- 172.21.15.49:0/2426623917 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f44900c3530 con 0x7f449006d150 2023-11-06T02:09:58.604 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.604+0000 7f449839d640 1 -- 172.21.15.49:0/2426623917 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f44900c3530 con 0x7f449006d150 2023-11-06T02:09:58.604 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.604+0000 7f4486ffd640 1 -- 172.21.15.49:0/2426623917 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v37) v1 ==== 104+0+11 (secure 0 0 0) 0x7f44900c3530 con 0x7f449006d150 2023-11-06T02:09:58.605 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2023-11-06T02:09:58.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.608+0000 7f449839d640 1 -- 172.21.15.49:0/2426623917 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4474065a60 msgr2=0x7f4474067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:58.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.608+0000 7f449839d640 1 --2- 172.21.15.49:0/2426623917 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4474065a60 0x7f4474067f20 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f44900cd1e0 tx=0x7f448c008040 comp rx=0 tx=0).stop 2023-11-06T02:09:58.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.608+0000 7f449839d640 1 -- 172.21.15.49:0/2426623917 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f449006d150 msgr2=0x7f449014dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:58.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.608+0000 7f449839d640 1 --2- 172.21.15.49:0/2426623917 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f449006d150 0x7f449014dff0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f44800b0040 tx=0x7f4480019220 comp rx=0 tx=0).stop 2023-11-06T02:09:58.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.608+0000 7f449839d640 1 -- 172.21.15.49:0/2426623917 shutdown_connections 2023-11-06T02:09:58.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.608+0000 7f449839d640 1 --2- 172.21.15.49:0/2426623917 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4474065a60 0x7f4474067f20 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:58.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.608+0000 7f449839d640 1 --2- 172.21.15.49:0/2426623917 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f449015a880 0x7f44900c6d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:58.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.608+0000 7f449839d640 1 --2- 172.21.15.49:0/2426623917 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4490070160 0x7f44900c67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:58.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.608+0000 7f449839d640 1 --2- 172.21.15.49:0/2426623917 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f449006d150 0x7f449014dff0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:58.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.608+0000 7f449839d640 1 -- 172.21.15.49:0/2426623917 >> 172.21.15.49:0/2426623917 conn(0x7f449006a370 msgr2=0x7f44901533c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:58.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.608+0000 7f449839d640 1 -- 172.21.15.49:0/2426623917 shutdown_connections 2023-11-06T02:09:58.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.608+0000 7f449839d640 1 -- 172.21.15.49:0/2426623917 wait complete. 2023-11-06T02:09:58.623 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T02:09:58.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.756+0000 7f337d8ed640 1 Processor -- start 2023-11-06T02:09:58.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.756+0000 7f337d8ed640 1 -- start start 2023-11-06T02:09:58.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.756+0000 7f337d8ed640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f337815d160 0x7f337815e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:58.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.756+0000 7f337d8ed640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f337815e860 0x7f33780707b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:58.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.756+0000 7f337d8ed640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f337815e100 con 0x7f3378153d50 2023-11-06T02:09:58.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.756+0000 7f337d8ed640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f337806e0a0 con 0x7f337815e860 2023-11-06T02:09:58.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.756+0000 7f337d8ed640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3378154130 con 0x7f337815d160 2023-11-06T02:09:58.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.756+0000 7f3376ffd640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f3378153d50 0x7f33780c9100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:44404/0 (socket says 172.21.15.49:44404) 2023-11-06T02:09:58.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.756+0000 7f3376ffd640 1 -- 172.21.15.49:0/3669262726 learned_addr learned my addr 172.21.15.49:0/3669262726 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:09:58.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.756+0000 7f33767fc640 1 --2- 172.21.15.49:0/3669262726 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f337815d160 0x7f337815e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:58.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.756+0000 7f33777fe640 1 --2- 172.21.15.49:0/3669262726 >> v2:172.21.15.150:3300/0 conn(0x7f337815e860 0x7f33780707b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:58.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.756+0000 7f3375ffb640 1 -- 172.21.15.49:0/3669262726 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 902002287 0 0) 0x7f337815e100 con 0x7f3378153d50 2023-11-06T02:09:58.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.756+0000 7f3375ffb640 1 -- 172.21.15.49:0/3669262726 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3360003530 con 0x7f3378153d50 2023-11-06T02:09:58.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.756+0000 7f3375ffb640 1 -- 172.21.15.49:0/3669262726 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 239190411 0 0) 0x7f3360003530 con 0x7f3378153d50 2023-11-06T02:09:58.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.756+0000 7f3375ffb640 1 -- 172.21.15.49:0/3669262726 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f337815d160 msgr2=0x7f337815e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:58.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.756+0000 7f3375ffb640 1 --2- 172.21.15.49:0/3669262726 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f337815d160 0x7f337815e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:58.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.756+0000 7f3375ffb640 1 -- 172.21.15.49:0/3669262726 >> v2:172.21.15.150:3300/0 conn(0x7f337815e860 msgr2=0x7f33780707b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:58.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.756+0000 7f3375ffb640 1 --2- 172.21.15.49:0/3669262726 >> v2:172.21.15.150:3300/0 conn(0x7f337815e860 0x7f33780707b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:58.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.756+0000 7f3375ffb640 1 -- 172.21.15.49:0/3669262726 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f337815d5c0 con 0x7f3378153d50 2023-11-06T02:09:58.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.756+0000 7f3375ffb640 1 -- 172.21.15.49:0/3669262726 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7f3368001680 con 0x7f3378153d50 2023-11-06T02:09:58.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.756+0000 7f3375ffb640 1 -- 172.21.15.49:0/3669262726 >> v1:172.21.15.49:6789/0 conn(0x7f3378153d50 legacy=0x7f33780c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:58.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.756+0000 7f3375ffb640 1 --2- 172.21.15.49:0/3669262726 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3360003980 0x7f3360003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:58.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.756+0000 7f3375ffb640 1 --2- 172.21.15.49:0/3669262726 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f337815d160 0x7f33600047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:58.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.756+0000 7f3375ffb640 1 --2- 172.21.15.49:0/3669262726 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3360004d10 0x7f3360005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:58.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.756+0000 7f3375ffb640 1 -- 172.21.15.49:0/3669262726 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f337806e0a0 con 0x7f3360003980 2023-11-06T02:09:58.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.756+0000 7f3375ffb640 1 -- 172.21.15.49:0/3669262726 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f336000b990 con 0x7f3360004d10 2023-11-06T02:09:58.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.756+0000 7f3375ffb640 1 -- 172.21.15.49:0/3669262726 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f336000bb00 con 0x7f337815d160 2023-11-06T02:09:58.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.756+0000 7f3376ffd640 1 --2- 172.21.15.49:0/3669262726 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3360003980 0x7f3360003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:58.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.756+0000 7f33767fc640 1 --2- 172.21.15.49:0/3669262726 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f337815d160 0x7f33600047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:58.762 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.756+0000 7f33777fe640 1 --2- 172.21.15.49:0/3669262726 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3360004d10 0x7f3360005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:58.762 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.756+0000 7f3376ffd640 1 -- 172.21.15.49:0/3669262726 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f337815d160 msgr2=0x7f33600047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:58.762 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.756+0000 7f3376ffd640 1 --2- 172.21.15.49:0/3669262726 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f337815d160 0x7f33600047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:58.762 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.756+0000 7f3376ffd640 1 -- 172.21.15.49:0/3669262726 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3360004d10 msgr2=0x7f3360005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:58.762 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.756+0000 7f3376ffd640 1 --2- 172.21.15.49:0/3669262726 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3360004d10 0x7f3360005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:58.763 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.756+0000 7f3376ffd640 1 -- 172.21.15.49:0/3669262726 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3360003530 con 0x7f3360003980 2023-11-06T02:09:58.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.760+0000 7f3376ffd640 1 --2- 172.21.15.49:0/3669262726 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3360003980 0x7f3360003da0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f336800c210 tx=0x7f336801cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ad4f777dc329fec server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:58.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.760+0000 7f3375ffb640 1 -- 172.21.15.49:0/3669262726 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33680058e0 con 0x7f3360003980 2023-11-06T02:09:58.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.760+0000 7f3375ffb640 1 -- 172.21.15.49:0/3669262726 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3368005a40 con 0x7f3360003980 2023-11-06T02:09:58.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.760+0000 7f3375ffb640 1 -- 172.21.15.49:0/3669262726 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3368005e10 con 0x7f3360003980 2023-11-06T02:09:58.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.760+0000 7f337d8ed640 1 -- 172.21.15.49:0/3669262726 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3360003980 msgr2=0x7f3360003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:58.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.760+0000 7f337d8ed640 1 --2- 172.21.15.49:0/3669262726 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3360003980 0x7f3360003da0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f336800c210 tx=0x7f336801cc30 comp rx=0 tx=0).stop 2023-11-06T02:09:58.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.760+0000 7f33777fe640 1 -- 172.21.15.49:0/3669262726 reap_dead start 2023-11-06T02:09:58.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.760+0000 7f337d8ed640 1 -- 172.21.15.49:0/3669262726 shutdown_connections 2023-11-06T02:09:58.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.760+0000 7f337d8ed640 1 -- 172.21.15.49:0/3669262726 >> 172.21.15.49:0/3669262726 conn(0x7f337806a370 msgr2=0x7f33780ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:58.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.760+0000 7f337d8ed640 1 -- 172.21.15.49:0/3669262726 shutdown_connections 2023-11-06T02:09:58.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.760+0000 7f337d8ed640 1 -- 172.21.15.49:0/3669262726 wait complete. 2023-11-06T02:09:58.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.760+0000 7f337d8ed640 1 Processor -- start 2023-11-06T02:09:58.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.760+0000 7f337d8ed640 1 -- start start 2023-11-06T02:09:58.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.764+0000 7f337d8ed640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3360003980 0x7f337814e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:58.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.764+0000 7f337d8ed640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f33780b45f0 0x7f337814cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:58.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.764+0000 7f337d8ed640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f337814c290 0x7f33780cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:58.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.764+0000 7f337d8ed640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f337814de90 con 0x7f337814c290 2023-11-06T02:09:58.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.764+0000 7f337d8ed640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f337814c710 con 0x7f3360003980 2023-11-06T02:09:58.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.764+0000 7f337d8ed640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f337814d560 con 0x7f33780b45f0 2023-11-06T02:09:58.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.764+0000 7f33777fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f337814c290 0x7f33780cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:58.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.764+0000 7f33777fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f337814c290 0x7f33780cd5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43472/0 (socket says 172.21.15.49:43472) 2023-11-06T02:09:58.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.764+0000 7f33777fe640 1 -- 172.21.15.49:0/542483034 learned_addr learned my addr 172.21.15.49:0/542483034 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:58.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.764+0000 7f3376ffd640 1 --2- 172.21.15.49:0/542483034 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3360003980 0x7f337814e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:58.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.764+0000 7f33767fc640 1 --2- 172.21.15.49:0/542483034 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f33780b45f0 0x7f337814cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:58.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.764+0000 7f33777fe640 1 -- 172.21.15.49:0/542483034 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f33780b45f0 msgr2=0x7f337814cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:58.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.764+0000 7f33777fe640 1 --2- 172.21.15.49:0/542483034 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f33780b45f0 0x7f337814cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:58.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.764+0000 7f33777fe640 1 -- 172.21.15.49:0/542483034 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3360003980 msgr2=0x7f337814e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:58.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.764+0000 7f33777fe640 1 --2- 172.21.15.49:0/542483034 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3360003980 0x7f337814e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:58.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.764+0000 7f33777fe640 1 -- 172.21.15.49:0/542483034 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f337814d840 con 0x7f337814c290 2023-11-06T02:09:58.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.764+0000 7f33777fe640 1 --2- 172.21.15.49:0/542483034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f337814c290 0x7f33780cd5f0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f337815ec40 tx=0x7f33700115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:58.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.764+0000 7f335ffff640 1 -- 172.21.15.49:0/542483034 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f337000e070 con 0x7f337814c290 2023-11-06T02:09:58.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.764+0000 7f337d8ed640 1 -- 172.21.15.49:0/542483034 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f337814db20 con 0x7f337814c290 2023-11-06T02:09:58.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.764+0000 7f335ffff640 1 -- 172.21.15.49:0/542483034 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3370011e10 con 0x7f337814c290 2023-11-06T02:09:58.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.764+0000 7f335ffff640 1 -- 172.21.15.49:0/542483034 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33700039a0 con 0x7f337814c290 2023-11-06T02:09:58.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.764+0000 7f337d8ed640 1 -- 172.21.15.49:0/542483034 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33780cdb30 con 0x7f337814c290 2023-11-06T02:09:58.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.764+0000 7f337d8ed640 1 -- 172.21.15.49:0/542483034 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f337814dcb0 con 0x7f337814c290 2023-11-06T02:09:58.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.764+0000 7f335ffff640 1 -- 172.21.15.49:0/542483034 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f3370003b00 con 0x7f337814c290 2023-11-06T02:09:58.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.764+0000 7f335ffff640 1 --2- 172.21.15.49:0/542483034 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3364065a60 0x7f3364067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:58.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.764+0000 7f335ffff640 1 -- 172.21.15.49:0/542483034 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3370083960 con 0x7f337814c290 2023-11-06T02:09:58.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.764+0000 7f3376ffd640 1 --2- 172.21.15.49:0/542483034 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3364065a60 0x7f3364067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:58.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.768+0000 7f3376ffd640 1 --2- 172.21.15.49:0/542483034 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3364065a60 0x7f3364067f20 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f33680aa000 tx=0x7f3368001a10 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:58.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.768+0000 7f335ffff640 1 -- 172.21.15.49:0/542483034 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f33700511c0 con 0x7f337814c290 2023-11-06T02:09:58.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.916+0000 7f337d8ed640 1 -- 172.21.15.49:0/542483034 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3378074040 con 0x7f337814c290 2023-11-06T02:09:58.917 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.916+0000 7f335ffff640 1 -- 172.21.15.49:0/542483034 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v37) v1 ==== 100+0+11 (secure 0 0 0) 0x7f33700547e0 con 0x7f337814c290 2023-11-06T02:09:58.917 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2023-11-06T02:09:58.919 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.920+0000 7f337d8ed640 1 -- 172.21.15.49:0/542483034 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3364065a60 msgr2=0x7f3364067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:58.919 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.920+0000 7f337d8ed640 1 --2- 172.21.15.49:0/542483034 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3364065a60 0x7f3364067f20 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f33680aa000 tx=0x7f3368001a10 comp rx=0 tx=0).stop 2023-11-06T02:09:58.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.920+0000 7f337d8ed640 1 -- 172.21.15.49:0/542483034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f337814c290 msgr2=0x7f33780cd5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:58.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.920+0000 7f337d8ed640 1 --2- 172.21.15.49:0/542483034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f337814c290 0x7f33780cd5f0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f337815ec40 tx=0x7f33700115a0 comp rx=0 tx=0).stop 2023-11-06T02:09:58.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.920+0000 7f337d8ed640 1 -- 172.21.15.49:0/542483034 shutdown_connections 2023-11-06T02:09:58.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.920+0000 7f337d8ed640 1 --2- 172.21.15.49:0/542483034 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3364065a60 0x7f3364067f20 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:58.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.920+0000 7f337d8ed640 1 --2- 172.21.15.49:0/542483034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f337814c290 0x7f33780cd5f0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:58.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.920+0000 7f337d8ed640 1 --2- 172.21.15.49:0/542483034 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f33780b45f0 0x7f337814cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:58.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.920+0000 7f337d8ed640 1 --2- 172.21.15.49:0/542483034 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3360003980 0x7f337814e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:58.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.920+0000 7f337d8ed640 1 -- 172.21.15.49:0/542483034 >> 172.21.15.49:0/542483034 conn(0x7f337806a370 msgr2=0x7f337806c230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:58.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.920+0000 7f337d8ed640 1 -- 172.21.15.49:0/542483034 shutdown_connections 2023-11-06T02:09:58.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:58.920+0000 7f337d8ed640 1 -- 172.21.15.49:0/542483034 wait complete. 2023-11-06T02:09:58.935 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-06T02:09:59.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.052+0000 7f84556c2640 1 Processor -- start 2023-11-06T02:09:59.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.052+0000 7f84556c2640 1 -- start start 2023-11-06T02:09:59.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.056+0000 7f84556c2640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f845015ab50 0x7f84500c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:59.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.056+0000 7f84556c2640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f84500b0790 0x7f84500c2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:59.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.056+0000 7f84556c2640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84500cab80 con 0x7f84500cb010 2023-11-06T02:09:59.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.056+0000 7f84556c2640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f845015af30 con 0x7f84500b0790 2023-11-06T02:09:59.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.056+0000 7f84556c2640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84500cb5b0 con 0x7f845015ab50 2023-11-06T02:09:59.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.056+0000 7f844e7fc640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f84500cb010 0x7f84500b0080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:44408/0 (socket says 172.21.15.49:44408) 2023-11-06T02:09:59.057 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.056+0000 7f844e7fc640 1 -- 172.21.15.49:0/2365368862 learned_addr learned my addr 172.21.15.49:0/2365368862 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:09:59.057 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.056+0000 7f844effd640 1 --2- 172.21.15.49:0/2365368862 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f845015ab50 0x7f84500c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:59.057 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.056+0000 7f844f7fe640 1 --2- 172.21.15.49:0/2365368862 >> v2:172.21.15.150:3300/0 conn(0x7f84500b0790 0x7f84500c2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:59.057 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.056+0000 7f844dffb640 1 -- 172.21.15.49:0/2365368862 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1230310313 0 0) 0x7f84500cab80 con 0x7f84500cb010 2023-11-06T02:09:59.057 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.056+0000 7f844dffb640 1 -- 172.21.15.49:0/2365368862 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8438003530 con 0x7f84500cb010 2023-11-06T02:09:59.057 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.056+0000 7f844dffb640 1 -- 172.21.15.49:0/2365368862 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1226804371 0 0) 0x7f8438003530 con 0x7f84500cb010 2023-11-06T02:09:59.058 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.056+0000 7f844dffb640 1 -- 172.21.15.49:0/2365368862 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f845015ab50 msgr2=0x7f84500c9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:59.058 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.056+0000 7f844dffb640 1 --2- 172.21.15.49:0/2365368862 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f845015ab50 0x7f84500c9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:59.058 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.056+0000 7f844dffb640 1 -- 172.21.15.49:0/2365368862 >> v2:172.21.15.150:3300/0 conn(0x7f84500b0790 msgr2=0x7f84500c2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:59.058 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.056+0000 7f844dffb640 1 --2- 172.21.15.49:0/2365368862 >> v2:172.21.15.150:3300/0 conn(0x7f84500b0790 0x7f84500c2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:59.058 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.056+0000 7f844dffb640 1 -- 172.21.15.49:0/2365368862 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84500cb890 con 0x7f84500cb010 2023-11-06T02:09:59.059 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.056+0000 7f844dffb640 1 -- 172.21.15.49:0/2365368862 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7f8444001680 con 0x7f84500cb010 2023-11-06T02:09:59.059 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.056+0000 7f844dffb640 1 -- 172.21.15.49:0/2365368862 >> v1:172.21.15.49:6789/0 conn(0x7f84500cb010 legacy=0x7f84500b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:59.059 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.056+0000 7f844dffb640 1 --2- 172.21.15.49:0/2365368862 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8438003980 0x7f8438003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:59.059 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.056+0000 7f844dffb640 1 --2- 172.21.15.49:0/2365368862 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84380047d0 0x7f8438004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:59.059 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.056+0000 7f844dffb640 1 --2- 172.21.15.49:0/2365368862 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f845015ab50 0x7f8438005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:59.060 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.056+0000 7f844dffb640 1 -- 172.21.15.49:0/2365368862 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f845015af30 con 0x7f8438003980 2023-11-06T02:09:59.060 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.056+0000 7f844dffb640 1 -- 172.21.15.49:0/2365368862 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84500cb5b0 con 0x7f84380047d0 2023-11-06T02:09:59.060 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.056+0000 7f844dffb640 1 -- 172.21.15.49:0/2365368862 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f843800b990 con 0x7f845015ab50 2023-11-06T02:09:59.060 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.056+0000 7f844effd640 1 --2- 172.21.15.49:0/2365368862 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8438003980 0x7f8438003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:59.060 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.056+0000 7f844e7fc640 1 --2- 172.21.15.49:0/2365368862 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84380047d0 0x7f8438004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:59.061 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.056+0000 7f844effd640 1 -- 172.21.15.49:0/2365368862 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f845015ab50 msgr2=0x7f8438005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:59.061 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.056+0000 7f844effd640 1 --2- 172.21.15.49:0/2365368862 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f845015ab50 0x7f8438005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:59.061 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.056+0000 7f844effd640 1 -- 172.21.15.49:0/2365368862 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84380047d0 msgr2=0x7f8438004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:09:59.061 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.056+0000 7f844effd640 1 --2- 172.21.15.49:0/2365368862 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84380047d0 0x7f8438004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:59.061 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.056+0000 7f844effd640 1 -- 172.21.15.49:0/2365368862 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8438003530 con 0x7f8438003980 2023-11-06T02:09:59.065 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.060+0000 7f844effd640 1 --2- 172.21.15.49:0/2365368862 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8438003980 0x7f8438003da0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f8440009ac0 tx=0x7f844001ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2321dafcfad96b78 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:59.065 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.060+0000 7f844dffb640 1 -- 172.21.15.49:0/2365368862 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84400059a0 con 0x7f8438003980 2023-11-06T02:09:59.065 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.060+0000 7f844dffb640 1 -- 172.21.15.49:0/2365368862 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8440005b00 con 0x7f8438003980 2023-11-06T02:09:59.065 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.060+0000 7f844dffb640 1 -- 172.21.15.49:0/2365368862 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8440006030 con 0x7f8438003980 2023-11-06T02:09:59.066 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.060+0000 7f84556c2640 1 -- 172.21.15.49:0/2365368862 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8438003980 msgr2=0x7f8438003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:59.066 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.060+0000 7f84556c2640 1 --2- 172.21.15.49:0/2365368862 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8438003980 0x7f8438003da0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f8440009ac0 tx=0x7f844001ca40 comp rx=0 tx=0).stop 2023-11-06T02:09:59.066 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.060+0000 7f844f7fe640 1 -- 172.21.15.49:0/2365368862 reap_dead start 2023-11-06T02:09:59.066 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.060+0000 7f84556c2640 1 -- 172.21.15.49:0/2365368862 shutdown_connections 2023-11-06T02:09:59.067 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.060+0000 7f84556c2640 1 -- 172.21.15.49:0/2365368862 >> 172.21.15.49:0/2365368862 conn(0x7f845006c0a0 msgr2=0x7f845006c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:59.067 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.060+0000 7f84556c2640 1 -- 172.21.15.49:0/2365368862 shutdown_connections 2023-11-06T02:09:59.067 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.060+0000 7f84556c2640 1 -- 172.21.15.49:0/2365368862 wait complete. 2023-11-06T02:09:59.067 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.060+0000 7f84556c2640 1 Processor -- start 2023-11-06T02:09:59.067 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.060+0000 7f84556c2640 1 -- start start 2023-11-06T02:09:59.068 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.060+0000 7f84556c2640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8438003980 0x7f84501f80e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:59.068 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.060+0000 7f84556c2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8450158be0 0x7f84501566d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:59.068 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.060+0000 7f84556c2640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8450154910 0x7f8450154dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:59.068 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.060+0000 7f84556c2640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8450155300 con 0x7f8450158be0 2023-11-06T02:09:59.068 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.060+0000 7f84556c2640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84500c8770 con 0x7f8450154910 2023-11-06T02:09:59.069 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.060+0000 7f84556c2640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8450158750 con 0x7f8438003980 2023-11-06T02:09:59.069 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.060+0000 7f844e7fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8450158be0 0x7f84501566d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:59.069 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.060+0000 7f844e7fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8450158be0 0x7f84501566d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43482/0 (socket says 172.21.15.49:43482) 2023-11-06T02:09:59.069 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.060+0000 7f844e7fc640 1 -- 172.21.15.49:0/1749754915 learned_addr learned my addr 172.21.15.49:0/1749754915 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:09:59.069 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.060+0000 7f844effd640 1 --2- 172.21.15.49:0/1749754915 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8438003980 0x7f84501f80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:59.069 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.060+0000 7f844f7fe640 1 --2- 172.21.15.49:0/1749754915 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8450154910 0x7f8450154dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:59.070 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.060+0000 7f844e7fc640 1 -- 172.21.15.49:0/1749754915 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8438003980 msgr2=0x7f84501f80e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:59.070 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.060+0000 7f844e7fc640 1 --2- 172.21.15.49:0/1749754915 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8438003980 0x7f84501f80e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:59.070 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.060+0000 7f844e7fc640 1 -- 172.21.15.49:0/1749754915 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8450154910 msgr2=0x7f8450154dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:59.070 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.060+0000 7f844e7fc640 1 --2- 172.21.15.49:0/1749754915 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8450154910 0x7f8450154dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:59.070 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.060+0000 7f844e7fc640 1 -- 172.21.15.49:0/1749754915 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84501f8620 con 0x7f8450158be0 2023-11-06T02:09:59.071 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.060+0000 7f844e7fc640 1 --2- 172.21.15.49:0/1749754915 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8450158be0 0x7f84501566d0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f844400e780 tx=0x7f844400ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:59.071 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.064+0000 7f84377fe640 1 -- 172.21.15.49:0/1749754915 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f844400ee90 con 0x7f8450158be0 2023-11-06T02:09:59.071 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.064+0000 7f84377fe640 1 -- 172.21.15.49:0/1749754915 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f844400c8d0 con 0x7f8450158be0 2023-11-06T02:09:59.071 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.064+0000 7f84377fe640 1 -- 172.21.15.49:0/1749754915 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f844400cc80 con 0x7f8450158be0 2023-11-06T02:09:59.071 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.064+0000 7f84556c2640 1 -- 172.21.15.49:0/1749754915 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8450157860 con 0x7f8450158be0 2023-11-06T02:09:59.072 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.064+0000 7f84556c2640 1 -- 172.21.15.49:0/1749754915 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8450157e20 con 0x7f8450158be0 2023-11-06T02:09:59.072 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.064+0000 7f84556c2640 1 -- 172.21.15.49:0/1749754915 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84500b0de0 con 0x7f8450158be0 2023-11-06T02:09:59.072 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.064+0000 7f84377fe640 1 -- 172.21.15.49:0/1749754915 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f844401d460 con 0x7f8450158be0 2023-11-06T02:09:59.072 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.064+0000 7f84377fe640 1 --2- 172.21.15.49:0/1749754915 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f843c065a60 0x7f843c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:09:59.072 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.064+0000 7f84377fe640 1 -- 172.21.15.49:0/1749754915 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8444083260 con 0x7f8450158be0 2023-11-06T02:09:59.073 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.064+0000 7f844effd640 1 --2- 172.21.15.49:0/1749754915 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f843c065a60 0x7f843c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:09:59.073 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.064+0000 7f844effd640 1 --2- 172.21.15.49:0/1749754915 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f843c065a60 0x7f843c067f20 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f84400ae000 tx=0x7f84400046d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:09:59.073 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.068+0000 7f84377fe640 1 -- 172.21.15.49:0/1749754915 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f84440545e0 con 0x7f8450158be0 2023-11-06T02:09:59.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.236+0000 7f84556c2640 1 -- 172.21.15.49:0/1749754915 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f8450074040 con 0x7f8450158be0 2023-11-06T02:09:59.234 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.236+0000 7f84377fe640 1 -- 172.21.15.49:0/1749754915 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v37) v1 ==== 80+0+0 (secure 0 0 0) 0x7f8444054170 con 0x7f8450158be0 2023-11-06T02:09:59.234 INFO:teuthology.orchestra.run.smithi049.stderr:listed 0 entries 2023-11-06T02:09:59.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.236+0000 7f84556c2640 1 -- 172.21.15.49:0/1749754915 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f843c065a60 msgr2=0x7f843c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:59.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.236+0000 7f84556c2640 1 --2- 172.21.15.49:0/1749754915 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f843c065a60 0x7f843c067f20 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f84400ae000 tx=0x7f84400046d0 comp rx=0 tx=0).stop 2023-11-06T02:09:59.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.236+0000 7f84556c2640 1 -- 172.21.15.49:0/1749754915 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8450158be0 msgr2=0x7f84501566d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:09:59.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.236+0000 7f84556c2640 1 --2- 172.21.15.49:0/1749754915 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8450158be0 0x7f84501566d0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f844400e780 tx=0x7f844400ec40 comp rx=0 tx=0).stop 2023-11-06T02:09:59.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.236+0000 7f84556c2640 1 -- 172.21.15.49:0/1749754915 shutdown_connections 2023-11-06T02:09:59.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.236+0000 7f84556c2640 1 --2- 172.21.15.49:0/1749754915 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f843c065a60 0x7f843c067f20 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:59.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.236+0000 7f84556c2640 1 --2- 172.21.15.49:0/1749754915 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8450158be0 0x7f84501566d0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:59.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.236+0000 7f84556c2640 1 --2- 172.21.15.49:0/1749754915 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8450154910 0x7f8450154dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:59.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.236+0000 7f84556c2640 1 --2- 172.21.15.49:0/1749754915 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8438003980 0x7f84501f80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:09:59.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.236+0000 7f84556c2640 1 -- 172.21.15.49:0/1749754915 >> 172.21.15.49:0/1749754915 conn(0x7f845006c0a0 msgr2=0x7f8450071180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:09:59.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.236+0000 7f84556c2640 1 -- 172.21.15.49:0/1749754915 shutdown_connections 2023-11-06T02:09:59.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:09:59.236+0000 7f84556c2640 1 -- 172.21.15.49:0/1749754915 wait complete. 2023-11-06T02:09:59.252 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi049.front.sepia.ceph.com... 2023-11-06T02:09:59.252 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:09:59.252 DEBUG:teuthology.orchestra.run.smithi049:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-06T02:09:59.263 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi049.stderr:ceph-fuse[48364]: fuse finished with error 0 and tester_r 0 2023-11-06T02:09:59.264 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T02:10:05.366 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi049.front.sepia.ceph.com 2023-11-06T02:10:05.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:10:05.367 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-06T02:10:05.375 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T02:10:05.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:05.547+0000 7f6eb7376640 1 Processor -- start 2023-11-06T02:10:05.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:05.551+0000 7f6eb7376640 1 -- start start 2023-11-06T02:10:05.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:05.551+0000 7f6eb7376640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f6eb015ab50 0x7f6eb00c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:05.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:05.551+0000 7f6eb7376640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6eb00cb010 0x7f6eb00b0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:05.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:05.551+0000 7f6eb7376640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6eb00cab80 con 0x7f6eb00b05c0 2023-11-06T02:10:05.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:05.551+0000 7f6eb7376640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6eb00b0a50 con 0x7f6eb015ab50 2023-11-06T02:10:05.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:05.551+0000 7f6eb7376640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb015af30 con 0x7f6eb00cb010 2023-11-06T02:10:05.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:05.551+0000 7f6eb5112640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f6eb015ab50 0x7f6eb00c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:05.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:05.551+0000 7f6eb4911640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6eb00cb010 0x7f6eb00b0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:05.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:05.551+0000 7f6eb5913640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f6eb00b05c0 0x7f6eb00700a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.49:55364/0 (socket says 172.21.15.49:55364) 2023-11-06T02:10:05.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:05.551+0000 7f6eb5913640 1 -- 172.21.15.49:0/3415907661 learned_addr learned my addr 172.21.15.49:0/3415907661 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:10:05.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:05.551+0000 7f6eb5112640 1 -- 172.21.15.49:0/3415907661 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6eb00cb010 msgr2=0x7f6eb00b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:05.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:05.551+0000 7f6eb5112640 1 --2- 172.21.15.49:0/3415907661 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6eb00cb010 0x7f6eb00b0080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:05.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:05.551+0000 7f6eb5112640 1 -- 172.21.15.49:0/3415907661 >> v1:172.21.15.150:6789/0 conn(0x7f6eb00b05c0 legacy=0x7f6eb00700a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:05.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:05.551+0000 7f6eb5112640 1 -- 172.21.15.49:0/3415907661 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eb00cb5b0 con 0x7f6eb015ab50 2023-11-06T02:10:05.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:05.551+0000 7f6eb4911640 1 --2- 172.21.15.49:0/3415907661 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6eb00cb010 0x7f6eb00b0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T02:10:05.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:05.555+0000 7f6eb5112640 1 --2- 172.21.15.49:0/3415907661 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6eb015ab50 0x7f6eb00c9bb0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6eac0097a0 tx=0x7f6eac01c430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54490ce5beb8c98a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:05.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:05.555+0000 7f6ea3fff640 1 -- 172.21.15.49:0/3415907661 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eac0ab650 con 0x7f6eb015ab50 2023-11-06T02:10:05.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:05.555+0000 7f6ea3fff640 1 -- 172.21.15.49:0/3415907661 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6eac0ab7b0 con 0x7f6eb015ab50 2023-11-06T02:10:05.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:05.555+0000 7f6ea3fff640 1 -- 172.21.15.49:0/3415907661 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eac0abb40 con 0x7f6eb015ab50 2023-11-06T02:10:05.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:05.555+0000 7f6eb7376640 1 -- 172.21.15.49:0/3415907661 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6eb015ab50 msgr2=0x7f6eb00c9bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:05.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:05.555+0000 7f6eb7376640 1 --2- 172.21.15.49:0/3415907661 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6eb015ab50 0x7f6eb00c9bb0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6eac0097a0 tx=0x7f6eac01c430 comp rx=0 tx=0).stop 2023-11-06T02:10:05.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:05.555+0000 7f6eb7376640 1 -- 172.21.15.49:0/3415907661 shutdown_connections 2023-11-06T02:10:05.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:05.555+0000 7f6eb7376640 1 --2- 172.21.15.49:0/3415907661 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6eb00cb010 0x7f6eb00b0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:05.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:05.555+0000 7f6eb7376640 1 --2- 172.21.15.49:0/3415907661 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6eb015ab50 0x7f6eb00c9bb0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:05.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:05.555+0000 7f6eb7376640 1 -- 172.21.15.49:0/3415907661 >> 172.21.15.49:0/3415907661 conn(0x7f6eb006c0a0 msgr2=0x7f6eb006c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:05.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:05.555+0000 7f6eb7376640 1 -- 172.21.15.49:0/3415907661 shutdown_connections 2023-11-06T02:10:05.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:05.555+0000 7f6eb7376640 1 -- 172.21.15.49:0/3415907661 wait complete. 2023-11-06T02:10:05.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:05.555+0000 7f6eb7376640 1 Processor -- start 2023-11-06T02:10:05.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:05.555+0000 7f6eb7376640 1 -- start start 2023-11-06T02:10:05.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:05.559+0000 7f6eb7376640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6eb00b05c0 0x7f6eb014b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:05.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:05.559+0000 7f6eb7376640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6eb00cb010 0x7f6eb0157770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:05.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:05.559+0000 7f6eb7376640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6eb015ab50 0x7f6eb00af1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:05.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:05.559+0000 7f6eb7376640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb0156a60 con 0x7f6eb00cb010 2023-11-06T02:10:05.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:05.559+0000 7f6eb7376640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb00ce250 con 0x7f6eb015ab50 2023-11-06T02:10:05.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:05.559+0000 7f6eb7376640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb014b920 con 0x7f6eb00b05c0 2023-11-06T02:10:05.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:05.559+0000 7f6eb4911640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6eb00cb010 0x7f6eb0157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:05.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:05.559+0000 7f6eb4911640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6eb00cb010 0x7f6eb0157770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43484/0 (socket says 172.21.15.49:43484) 2023-11-06T02:10:05.563 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:05.559+0000 7f6eb4911640 1 -- 172.21.15.49:0/3770908387 learned_addr learned my addr 172.21.15.49:0/3770908387 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:05.563 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:05.559+0000 7f6eb5913640 1 --2- 172.21.15.49:0/3770908387 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6eb015ab50 0x7f6eb00af1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:05.563 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:05.559+0000 7f6eb5112640 1 --2- 172.21.15.49:0/3770908387 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6eb00b05c0 0x7f6eb014b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:05.563 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:05.559+0000 7f6eb4911640 1 -- 172.21.15.49:0/3770908387 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6eb00b05c0 msgr2=0x7f6eb014b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:05.563 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:05.559+0000 7f6eb4911640 1 --2- 172.21.15.49:0/3770908387 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6eb00b05c0 0x7f6eb014b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:05.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:05.559+0000 7f6eb4911640 1 -- 172.21.15.49:0/3770908387 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6eb015ab50 msgr2=0x7f6eb00af1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:05.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:05.559+0000 7f6eb4911640 1 --2- 172.21.15.49:0/3770908387 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6eb015ab50 0x7f6eb00af1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:05.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:05.559+0000 7f6eb4911640 1 -- 172.21.15.49:0/3770908387 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ea4009420 con 0x7f6eb00cb010 2023-11-06T02:10:05.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:05.559+0000 7f6eb4911640 1 --2- 172.21.15.49:0/3770908387 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6eb00cb010 0x7f6eb0157770 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f6ea400e980 tx=0x7f6ea400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:05.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:05.559+0000 7f6ea1ffb640 1 -- 172.21.15.49:0/3770908387 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ea400ca00 con 0x7f6eb00cb010 2023-11-06T02:10:05.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:05.559+0000 7f6ea1ffb640 1 -- 172.21.15.49:0/3770908387 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ea400cb60 con 0x7f6eb00cb010 2023-11-06T02:10:05.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:05.559+0000 7f6eb7376640 1 -- 172.21.15.49:0/3770908387 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6eac0093b0 con 0x7f6eb00cb010 2023-11-06T02:10:05.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:05.559+0000 7f6ea1ffb640 1 -- 172.21.15.49:0/3770908387 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ea4013470 con 0x7f6eb00cb010 2023-11-06T02:10:05.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:05.559+0000 7f6eb7376640 1 -- 172.21.15.49:0/3770908387 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6eb01527c0 con 0x7f6eb00cb010 2023-11-06T02:10:05.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:05.559+0000 7f6eb7376640 1 -- 172.21.15.49:0/3770908387 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6eb00b0de0 con 0x7f6eb00cb010 2023-11-06T02:10:05.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:05.559+0000 7f6ea1ffb640 1 -- 172.21.15.49:0/3770908387 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f6ea40135d0 con 0x7f6eb00cb010 2023-11-06T02:10:05.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:05.559+0000 7f6ea1ffb640 1 --2- 172.21.15.49:0/3770908387 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6e94065a60 0x7f6e94067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:05.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:05.559+0000 7f6ea1ffb640 1 -- 172.21.15.49:0/3770908387 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6ea4083cd0 con 0x7f6eb00cb010 2023-11-06T02:10:05.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:05.559+0000 7f6eb5112640 1 --2- 172.21.15.49:0/3770908387 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6e94065a60 0x7f6e94067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:05.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:05.563+0000 7f6eb5112640 1 --2- 172.21.15.49:0/3770908387 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6e94065a60 0x7f6e94067f20 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6eac0af000 tx=0x7f6eac0a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:05.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:05.563+0000 7f6ea1ffb640 1 -- 172.21.15.49:0/3770908387 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f6ea40515c0 con 0x7f6eb00cb010 2023-11-06T02:10:05.753 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:05.751+0000 7f6eb7376640 1 -- 172.21.15.49:0/3770908387 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6eb0074040 con 0x7f6eb00cb010 2023-11-06T02:10:05.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:05.755+0000 7f6ea1ffb640 1 -- 172.21.15.49:0/3770908387 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4651 (secure 0 0 0) 0x7f6ea4054be0 con 0x7f6eb00cb010 2023-11-06T02:10:05.754 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:10:05.755 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5032,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/280428805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":280428805},{"type":"v1","addr":"172.21.15.150:6835","nonce":280428805}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":5035,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/2734589844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2734589844},{"type":"v1","addr":"172.21.15.49:6833","nonce":2734589844}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":5038,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/1319221246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":1319221246},{"type":"v1","addr":"172.21.15.150:6837","nonce":1319221246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T02:09:39.873514+0000","modified":"2023-11-06T02:09:43.132890+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5041},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5041":{"gid":5041,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.49:6835/1547192091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":1547192091},{"type":"v1","addr":"172.21.15.49:6835","nonce":1547192091}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-06T02:10:05.755 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 15 2023-11-06T02:10:05.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:05.755+0000 7f6eb7376640 1 -- 172.21.15.49:0/3770908387 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6e94065a60 msgr2=0x7f6e94067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:05.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:05.755+0000 7f6eb7376640 1 --2- 172.21.15.49:0/3770908387 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6e94065a60 0x7f6e94067f20 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6eac0af000 tx=0x7f6eac0a7000 comp rx=0 tx=0).stop 2023-11-06T02:10:05.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:05.755+0000 7f6eb7376640 1 -- 172.21.15.49:0/3770908387 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6eb00cb010 msgr2=0x7f6eb0157770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:05.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:05.755+0000 7f6eb7376640 1 --2- 172.21.15.49:0/3770908387 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6eb00cb010 0x7f6eb0157770 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f6ea400e980 tx=0x7f6ea400ee40 comp rx=0 tx=0).stop 2023-11-06T02:10:05.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:05.755+0000 7f6eb7376640 1 -- 172.21.15.49:0/3770908387 shutdown_connections 2023-11-06T02:10:05.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:05.755+0000 7f6eb7376640 1 --2- 172.21.15.49:0/3770908387 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6e94065a60 0x7f6e94067f20 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:05.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:05.755+0000 7f6eb7376640 1 --2- 172.21.15.49:0/3770908387 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6eb00cb010 0x7f6eb0157770 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:05.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:05.755+0000 7f6eb7376640 1 --2- 172.21.15.49:0/3770908387 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6eb015ab50 0x7f6eb00af1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:05.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:05.755+0000 7f6eb7376640 1 --2- 172.21.15.49:0/3770908387 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6eb00b05c0 0x7f6eb014b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:05.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:05.755+0000 7f6eb7376640 1 -- 172.21.15.49:0/3770908387 >> 172.21.15.49:0/3770908387 conn(0x7f6eb006c0a0 msgr2=0x7f6eb00c48b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:05.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:05.755+0000 7f6eb7376640 1 -- 172.21.15.49:0/3770908387 shutdown_connections 2023-11-06T02:10:05.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:05.755+0000 7f6eb7376640 1 -- 172.21.15.49:0/3770908387 wait complete. 2023-11-06T02:10:05.774 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2023-11-06T02:10:05.931 INFO:teuthology.orchestra.run.smithi049.stdout:{"message":"","return_code":0} 2023-11-06T02:10:05.932 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2023-11-06T02:10:05.932 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T02:10:06.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:06.055+0000 7f3612a8d640 1 Processor -- start 2023-11-06T02:10:06.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:06.055+0000 7f3612a8d640 1 -- start start 2023-11-06T02:10:06.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:06.055+0000 7f3612a8d640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f360c153ea0 0x7f360c0cb170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:06.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:06.055+0000 7f3612a8d640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f360c0ce420 0x7f360c15a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:06.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:06.055+0000 7f3612a8d640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f360c0cace0 con 0x7f360c159e60 2023-11-06T02:10:06.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:06.055+0000 7f3612a8d640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f360c0ca6f0 con 0x7f360c153ea0 2023-11-06T02:10:06.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:06.055+0000 7f3612a8d640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f360c15a920 con 0x7f360c0ce420 2023-11-06T02:10:06.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:06.055+0000 7f3610829640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f360c153ea0 0x7f360c0cb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:06.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:06.055+0000 7f3610829640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f360c153ea0 0x7f360c0cb170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43486/0 (socket says 172.21.15.49:43486) 2023-11-06T02:10:06.057 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:06.055+0000 7f3610829640 1 -- 172.21.15.49:0/1711135327 learned_addr learned my addr 172.21.15.49:0/1711135327 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:06.057 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:06.055+0000 7f361102a640 1 --2- 172.21.15.49:0/1711135327 >> v2:172.21.15.150:3300/0 conn(0x7f360c0ce420 0x7f360c15a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:06.057 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:06.055+0000 7f3610829640 1 -- 172.21.15.49:0/1711135327 >> v2:172.21.15.150:3300/0 conn(0x7f360c0ce420 msgr2=0x7f360c15a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:06.057 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:06.055+0000 7f3610829640 1 --2- 172.21.15.49:0/1711135327 >> v2:172.21.15.150:3300/0 conn(0x7f360c0ce420 0x7f360c15a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:06.058 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:06.055+0000 7f3610829640 1 -- 172.21.15.49:0/1711135327 >> v1:172.21.15.150:6789/0 conn(0x7f360c159e60 legacy=0x7f360c0cdd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:06.058 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:06.055+0000 7f3610829640 1 -- 172.21.15.49:0/1711135327 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f360c15ac00 con 0x7f360c153ea0 2023-11-06T02:10:06.061 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:06.059+0000 7f3610829640 1 --2- 172.21.15.49:0/1711135327 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f360c153ea0 0x7f360c0cb170 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f360c0aeaf0 tx=0x7f360001c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f96d6b7f17bf3e5 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:06.061 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:06.059+0000 7f360b7fe640 1 -- 172.21.15.49:0/1711135327 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36000aa5f0 con 0x7f360c153ea0 2023-11-06T02:10:06.062 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:06.059+0000 7f360b7fe640 1 -- 172.21.15.49:0/1711135327 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36000aa750 con 0x7f360c153ea0 2023-11-06T02:10:06.062 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:06.059+0000 7f360b7fe640 1 -- 172.21.15.49:0/1711135327 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36000aaae0 con 0x7f360c153ea0 2023-11-06T02:10:06.062 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:06.059+0000 7f3612a8d640 1 -- 172.21.15.49:0/1711135327 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f360c153ea0 msgr2=0x7f360c0cb170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:06.063 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:06.059+0000 7f3612a8d640 1 --2- 172.21.15.49:0/1711135327 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f360c153ea0 0x7f360c0cb170 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f360c0aeaf0 tx=0x7f360001c500 comp rx=0 tx=0).stop 2023-11-06T02:10:06.063 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:06.059+0000 7f3612a8d640 1 -- 172.21.15.49:0/1711135327 shutdown_connections 2023-11-06T02:10:06.063 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:06.059+0000 7f3612a8d640 1 --2- 172.21.15.49:0/1711135327 >> v2:172.21.15.150:3300/0 conn(0x7f360c0ce420 0x7f360c15a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:06.063 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:06.059+0000 7f3612a8d640 1 --2- 172.21.15.49:0/1711135327 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f360c153ea0 0x7f360c0cb170 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:06.064 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:06.059+0000 7f3612a8d640 1 -- 172.21.15.49:0/1711135327 >> 172.21.15.49:0/1711135327 conn(0x7f360c0b0bd0 msgr2=0x7f360c0b0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:06.064 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:06.059+0000 7f3612a8d640 1 -- 172.21.15.49:0/1711135327 shutdown_connections 2023-11-06T02:10:06.064 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:06.059+0000 7f3612a8d640 1 -- 172.21.15.49:0/1711135327 wait complete. 2023-11-06T02:10:06.064 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:06.059+0000 7f3612a8d640 1 Processor -- start 2023-11-06T02:10:06.064 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:06.059+0000 7f3612a8d640 1 -- start start 2023-11-06T02:10:06.064 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:06.059+0000 7f3612a8d640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f360c0ce420 0x7f360c14dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:06.065 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:06.059+0000 7f3612a8d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f360c153ea0 0x7f360c1499c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:06.065 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:06.059+0000 7f3612a8d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f360c159e60 0x7f360c151bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:06.065 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:06.059+0000 7f3612a8d640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f360c0c8720 con 0x7f360c159e60 2023-11-06T02:10:06.065 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:06.059+0000 7f3612a8d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f360c14f960 con 0x7f360c153ea0 2023-11-06T02:10:06.065 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:06.059+0000 7f3612a8d640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f360c14fc40 con 0x7f360c0ce420 2023-11-06T02:10:06.066 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:06.059+0000 7f361102a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f360c159e60 0x7f360c151bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:06.066 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:06.059+0000 7f361102a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f360c159e60 0x7f360c151bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43502/0 (socket says 172.21.15.49:43502) 2023-11-06T02:10:06.066 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:06.059+0000 7f361102a640 1 -- 172.21.15.49:0/3839488772 learned_addr learned my addr 172.21.15.49:0/3839488772 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:06.066 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:06.059+0000 7f3610829640 1 --2- 172.21.15.49:0/3839488772 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f360c0ce420 0x7f360c14dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:06.066 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:06.059+0000 7f360bfff640 1 --2- 172.21.15.49:0/3839488772 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f360c153ea0 0x7f360c1499c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:06.067 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:06.059+0000 7f361102a640 1 -- 172.21.15.49:0/3839488772 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f360c0ce420 msgr2=0x7f360c14dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:06.067 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:06.059+0000 7f361102a640 1 --2- 172.21.15.49:0/3839488772 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f360c0ce420 0x7f360c14dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:06.067 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:06.059+0000 7f361102a640 1 -- 172.21.15.49:0/3839488772 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f360c153ea0 msgr2=0x7f360c1499c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:06.067 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:06.059+0000 7f361102a640 1 --2- 172.21.15.49:0/3839488772 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f360c153ea0 0x7f360c1499c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:06.067 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:06.059+0000 7f361102a640 1 -- 172.21.15.49:0/3839488772 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36000094d0 con 0x7f360c159e60 2023-11-06T02:10:06.068 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:06.059+0000 7f361102a640 1 --2- 172.21.15.49:0/3839488772 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f360c159e60 0x7f360c151bd0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f3604007e60 tx=0x7f360400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:06.068 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:06.059+0000 7f36097fa640 1 -- 172.21.15.49:0/3839488772 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3604014620 con 0x7f360c159e60 2023-11-06T02:10:06.068 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:06.059+0000 7f36097fa640 1 -- 172.21.15.49:0/3839488772 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3604014780 con 0x7f360c159e60 2023-11-06T02:10:06.068 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:06.059+0000 7f3612a8d640 1 -- 172.21.15.49:0/3839488772 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f360c0c8970 con 0x7f360c159e60 2023-11-06T02:10:06.068 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:06.059+0000 7f36097fa640 1 -- 172.21.15.49:0/3839488772 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3604014b30 con 0x7f360c159e60 2023-11-06T02:10:06.068 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:06.059+0000 7f3612a8d640 1 -- 172.21.15.49:0/3839488772 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f360c14a680 con 0x7f360c159e60 2023-11-06T02:10:06.069 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:06.063+0000 7f3612a8d640 1 -- 172.21.15.49:0/3839488772 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f360c067140 con 0x7f360c159e60 2023-11-06T02:10:06.070 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:06.063+0000 7f36097fa640 1 -- 172.21.15.49:0/3839488772 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f3604014c90 con 0x7f360c159e60 2023-11-06T02:10:06.070 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:06.063+0000 7f36097fa640 1 --2- 172.21.15.49:0/3839488772 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f35f0065a60 0x7f35f0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:06.070 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:06.063+0000 7f36097fa640 1 -- 172.21.15.49:0/3839488772 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3604083d30 con 0x7f360c159e60 2023-11-06T02:10:06.070 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:06.063+0000 7f3610829640 1 --2- 172.21.15.49:0/3839488772 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f35f0065a60 0x7f35f0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:06.071 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:06.063+0000 7f3610829640 1 --2- 172.21.15.49:0/3839488772 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f35f0065a60 0x7f35f0067f20 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f36000ae000 tx=0x7f3600008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:06.071 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:06.067+0000 7f36097fa640 1 -- 172.21.15.49:0/3839488772 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f3604051500 con 0x7f360c159e60 2023-11-06T02:10:06.258 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:06.259+0000 7f3612a8d640 1 -- 172.21.15.49:0/3839488772 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f360c0c8e20 con 0x7f360c159e60 2023-11-06T02:10:06.259 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:06.259+0000 7f36097fa640 1 -- 172.21.15.49:0/3839488772 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4651 (secure 0 0 0) 0x7f3604054b20 con 0x7f360c159e60 2023-11-06T02:10:06.259 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:10:06.259 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5032,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/280428805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":280428805},{"type":"v1","addr":"172.21.15.150:6835","nonce":280428805}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":5035,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/2734589844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2734589844},{"type":"v1","addr":"172.21.15.49:6833","nonce":2734589844}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":5038,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/1319221246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":1319221246},{"type":"v1","addr":"172.21.15.150:6837","nonce":1319221246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T02:09:39.873514+0000","modified":"2023-11-06T02:09:43.132890+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5041},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5041":{"gid":5041,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.49:6835/1547192091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":1547192091},{"type":"v1","addr":"172.21.15.49:6835","nonce":1547192091}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-06T02:10:06.260 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 15 2023-11-06T02:10:06.261 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:06.259+0000 7f3612a8d640 1 -- 172.21.15.49:0/3839488772 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f35f0065a60 msgr2=0x7f35f0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:06.261 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:06.259+0000 7f3612a8d640 1 --2- 172.21.15.49:0/3839488772 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f35f0065a60 0x7f35f0067f20 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f36000ae000 tx=0x7f3600008040 comp rx=0 tx=0).stop 2023-11-06T02:10:06.262 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:06.259+0000 7f3612a8d640 1 -- 172.21.15.49:0/3839488772 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f360c159e60 msgr2=0x7f360c151bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:06.262 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:06.259+0000 7f3612a8d640 1 --2- 172.21.15.49:0/3839488772 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f360c159e60 0x7f360c151bd0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f3604007e60 tx=0x7f360400e4b0 comp rx=0 tx=0).stop 2023-11-06T02:10:06.262 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:06.263+0000 7f3612a8d640 1 -- 172.21.15.49:0/3839488772 shutdown_connections 2023-11-06T02:10:06.262 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:06.263+0000 7f3612a8d640 1 --2- 172.21.15.49:0/3839488772 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f35f0065a60 0x7f35f0067f20 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:06.262 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:06.263+0000 7f3612a8d640 1 --2- 172.21.15.49:0/3839488772 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f360c159e60 0x7f360c151bd0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:06.263 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:06.263+0000 7f3612a8d640 1 --2- 172.21.15.49:0/3839488772 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f360c153ea0 0x7f360c1499c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:06.263 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:06.263+0000 7f3612a8d640 1 --2- 172.21.15.49:0/3839488772 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f360c0ce420 0x7f360c14dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:06.263 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:06.263+0000 7f3612a8d640 1 -- 172.21.15.49:0/3839488772 >> 172.21.15.49:0/3839488772 conn(0x7f360c0b0bd0 msgr2=0x7f360c14b290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:06.263 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:06.263+0000 7f3612a8d640 1 -- 172.21.15.49:0/3839488772 shutdown_connections 2023-11-06T02:10:06.263 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:06.263+0000 7f3612a8d640 1 -- 172.21.15.49:0/3839488772 wait complete. 2023-11-06T02:10:06.279 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json tag path /parentdir mytag 2023-11-06T02:10:06.419 INFO:teuthology.orchestra.run.smithi049.stdout:{"return_code":0,"scrub_tag":"mytag","mode":"asynchronous"} 2023-11-06T02:10:06.419 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mode": "asynchronous", "return_code": 0, "scrub_tag": "mytag" } 2023-11-06T02:10:16.420 DEBUG:teuthology.orchestra.run.smithi049:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000000.00000000 scrub_tag 2023-11-06T02:10:16.464 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.463+0000 7fac23481080 1 Processor -- start 2023-11-06T02:10:16.464 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.463+0000 7fac23481080 1 -- start start 2023-11-06T02:10:16.467 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.463+0000 7fac23481080 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559a8cdfbaf0 0x559a8ce840d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:16.467 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.463+0000 7fac23481080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a8cdfbed0 0x559a8ce851d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:16.467 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.463+0000 7fac23481080 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x559a8ce88120 0x559a8ce7f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:16.467 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.463+0000 7fac23481080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a8ccf7450 con 0x559a8cdfbed0 2023-11-06T02:10:16.468 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.463+0000 7fac23481080 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a8ccf7150 con 0x559a8cdfbaf0 2023-11-06T02:10:16.468 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.463+0000 7fac23481080 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x559a8cceddc0 con 0x559a8ce88120 2023-11-06T02:10:16.468 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.463+0000 7fac2079a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a8cdfbed0 0x559a8ce851d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:16.468 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.463+0000 7fac2079a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a8cdfbed0 0x559a8ce851d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33246/0 (socket says 172.21.15.49:33246) 2023-11-06T02:10:16.468 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.463+0000 7fac2079a640 1 -- 172.21.15.49:0/1899619868 learned_addr learned my addr 172.21.15.49:0/1899619868 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:16.469 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.467+0000 7fac20f9b640 1 --2- 172.21.15.49:0/1899619868 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559a8cdfbaf0 0x559a8ce840d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:16.469 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.467+0000 7fac2179c640 1 --2- 172.21.15.49:0/1899619868 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x559a8ce88120 0x559a8ce7f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:16.469 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.467+0000 7fac2079a640 1 -- 172.21.15.49:0/1899619868 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x559a8ce88120 msgr2=0x559a8ce7f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:16.469 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.467+0000 7fac2079a640 1 --2- 172.21.15.49:0/1899619868 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x559a8ce88120 0x559a8ce7f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:16.469 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.467+0000 7fac2079a640 1 -- 172.21.15.49:0/1899619868 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559a8cdfbaf0 msgr2=0x559a8ce840d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:16.470 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.467+0000 7fac2079a640 1 --2- 172.21.15.49:0/1899619868 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559a8cdfbaf0 0x559a8ce840d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:16.470 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.467+0000 7fac2079a640 1 -- 172.21.15.49:0/1899619868 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac14005df0 con 0x559a8cdfbed0 2023-11-06T02:10:16.470 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.467+0000 7fac2079a640 1 --2- 172.21.15.49:0/1899619868 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a8cdfbed0 0x559a8ce851d0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fac1800b7c0 tx=0x7fac1800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94c324405bf7e048 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:16.470 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.467+0000 7fac1f798640 1 -- 172.21.15.49:0/1899619868 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac1800bed0 con 0x559a8cdfbed0 2023-11-06T02:10:16.470 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.467+0000 7fac1f798640 1 -- 172.21.15.49:0/1899619868 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac1800d9c0 con 0x559a8cdfbed0 2023-11-06T02:10:16.471 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.467+0000 7fac1f798640 1 -- 172.21.15.49:0/1899619868 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac1800dd50 con 0x559a8cdfbed0 2023-11-06T02:10:16.471 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.467+0000 7fac23481080 1 -- 172.21.15.49:0/1899619868 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a8cdfbed0 msgr2=0x559a8ce851d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:16.471 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.467+0000 7fac23481080 1 --2- 172.21.15.49:0/1899619868 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a8cdfbed0 0x559a8ce851d0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fac1800b7c0 tx=0x7fac1800bc80 comp rx=0 tx=0).stop 2023-11-06T02:10:16.471 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.467+0000 7fac23481080 1 -- 172.21.15.49:0/1899619868 shutdown_connections 2023-11-06T02:10:16.471 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.467+0000 7fac23481080 1 --2- 172.21.15.49:0/1899619868 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a8cdfbed0 0x559a8ce851d0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:16.472 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.467+0000 7fac23481080 1 --2- 172.21.15.49:0/1899619868 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x559a8ce88120 0x559a8ce7f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:16.472 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.467+0000 7fac23481080 1 --2- 172.21.15.49:0/1899619868 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559a8cdfbaf0 0x559a8ce840d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:16.472 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.467+0000 7fac23481080 1 -- 172.21.15.49:0/1899619868 >> 172.21.15.49:0/1899619868 conn(0x559a8cddf0e0 msgr2=0x559a8ce8a520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:16.472 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.467+0000 7fac23481080 1 -- 172.21.15.49:0/1899619868 shutdown_connections 2023-11-06T02:10:16.472 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.467+0000 7fac23481080 1 -- 172.21.15.49:0/1899619868 wait complete. 2023-11-06T02:10:16.473 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.467+0000 7fac23481080 1 Processor -- start 2023-11-06T02:10:16.473 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.467+0000 7fac23481080 1 -- start start 2023-11-06T02:10:16.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.467+0000 7fac23481080 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559a8cdfbaf0 0x559a8ce7fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:16.477 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.467+0000 7fac23481080 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x559a8cdfbed0 0x559a8ce68310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:16.477 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.467+0000 7fac23481080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a8ce88120 0x559a8ce68850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:16.477 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.467+0000 7fac23481080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a8ce7c0c0 con 0x559a8ce88120 2023-11-06T02:10:16.477 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.467+0000 7fac23481080 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a8ce68d90 con 0x559a8cdfbaf0 2023-11-06T02:10:16.477 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.467+0000 7fac23481080 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x559a8ce7b1a0 con 0x559a8cdfbed0 2023-11-06T02:10:16.478 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.467+0000 7fac2179c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a8ce88120 0x559a8ce68850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:16.478 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.467+0000 7fac2179c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a8ce88120 0x559a8ce68850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33248/0 (socket says 172.21.15.49:33248) 2023-11-06T02:10:16.478 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.467+0000 7fac2179c640 1 -- 172.21.15.49:0/305146873 learned_addr learned my addr 172.21.15.49:0/305146873 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:16.478 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.467+0000 7fac20f9b640 1 --2- 172.21.15.49:0/305146873 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559a8cdfbaf0 0x559a8ce7fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:16.478 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.467+0000 7fac2079a640 1 --2- 172.21.15.49:0/305146873 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x559a8cdfbed0 0x559a8ce68310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:16.479 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.467+0000 7fac2179c640 1 -- 172.21.15.49:0/305146873 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x559a8cdfbed0 msgr2=0x559a8ce68310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:16.479 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.467+0000 7fac2179c640 1 --2- 172.21.15.49:0/305146873 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x559a8cdfbed0 0x559a8ce68310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:16.479 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.467+0000 7fac2179c640 1 -- 172.21.15.49:0/305146873 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559a8cdfbaf0 msgr2=0x559a8ce7fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:16.479 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.467+0000 7fac2179c640 1 --2- 172.21.15.49:0/305146873 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559a8cdfbaf0 0x559a8ce7fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:16.479 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.467+0000 7fac2179c640 1 -- 172.21.15.49:0/305146873 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac1800b3c0 con 0x559a8ce88120 2023-11-06T02:10:16.480 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.467+0000 7fac2179c640 1 --2- 172.21.15.49:0/305146873 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a8ce88120 0x559a8ce68850 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fac140ad040 tx=0x7fac14018160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:16.480 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.467+0000 7fac1e796640 1 -- 172.21.15.49:0/305146873 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac140b5680 con 0x559a8ce88120 2023-11-06T02:10:16.480 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.467+0000 7fac1e796640 1 -- 172.21.15.49:0/305146873 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac140b57e0 con 0x559a8ce88120 2023-11-06T02:10:16.480 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.467+0000 7fac23481080 1 -- 172.21.15.49:0/305146873 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x559a8cd05ff0 con 0x559a8ce88120 2023-11-06T02:10:16.480 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.467+0000 7fac1e796640 1 -- 172.21.15.49:0/305146873 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac140b5b90 con 0x559a8ce88120 2023-11-06T02:10:16.481 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.467+0000 7fac23481080 1 -- 172.21.15.49:0/305146873 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559a8ccf9430 con 0x559a8ce88120 2023-11-06T02:10:16.481 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.471+0000 7fac1e796640 1 -- 172.21.15.49:0/305146873 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fac140b1040 con 0x559a8ce88120 2023-11-06T02:10:16.481 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.471+0000 7fac1e796640 1 --2- 172.21.15.49:0/305146873 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fac0c064440 0x7fac0c066900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:16.481 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.471+0000 7fac1e796640 1 -- 172.21.15.49:0/305146873 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6091+0+0 (secure 0 0 0) 0x7fac1411ce30 con 0x559a8ce88120 2023-11-06T02:10:16.481 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.471+0000 7fac23481080 1 --2- 172.21.15.49:0/305146873 >> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] conn(0x559a8ce80140 0x559a8ce7cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:16.481 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.471+0000 7fac23481080 1 -- 172.21.15.49:0/305146873 --> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] -- osd_op(unknown.0.0:1 5.10 5:0cf6ad0f:::10000000000.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39) v8 -- 0x559a8ce82f00 con 0x559a8ce80140 2023-11-06T02:10:16.482 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.471+0000 7fac20f9b640 1 --2- 172.21.15.49:0/305146873 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fac0c064440 0x7fac0c066900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:16.482 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.471+0000 7fac2079a640 1 --2- 172.21.15.49:0/305146873 >> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] conn(0x559a8ce80140 0x559a8ce7cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:16.482 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.471+0000 7fac20f9b640 1 --2- 172.21.15.49:0/305146873 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fac0c064440 0x7fac0c066900 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fac10003f20 tx=0x7fac10003880 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:16.482 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.471+0000 7fac2079a640 1 --2- 172.21.15.49:0/305146873 >> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] conn(0x559a8ce80140 0x559a8ce7cc70 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:16.482 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.475+0000 7fac2079a640 1 -- 172.21.15.49:0/305146873 <== osd.7 v2:172.21.15.150:6816/38421 1 ==== osd_op_reply(1 10000000000.00000000 [getxattr (9) out=9b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7fac18018070 con 0x559a8ce80140 2023-11-06T02:10:16.483 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.475+0000 7fac23481080 1 -- 172.21.15.49:0/305146873 >> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] conn(0x559a8ce80140 msgr2=0x559a8ce7cc70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:16.483 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.475+0000 7fac23481080 1 --2- 172.21.15.49:0/305146873 >> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] conn(0x559a8ce80140 0x559a8ce7cc70 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:16.483 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.475+0000 7fac23481080 1 -- 172.21.15.49:0/305146873 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fac0c064440 msgr2=0x7fac0c066900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:16.483 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.475+0000 7fac23481080 1 --2- 172.21.15.49:0/305146873 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fac0c064440 0x7fac0c066900 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fac10003f20 tx=0x7fac10003880 comp rx=0 tx=0).stop 2023-11-06T02:10:16.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.475+0000 7fac23481080 1 -- 172.21.15.49:0/305146873 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a8ce88120 msgr2=0x559a8ce68850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:16.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.475+0000 7fac23481080 1 --2- 172.21.15.49:0/305146873 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a8ce88120 0x559a8ce68850 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fac140ad040 tx=0x7fac14018160 comp rx=0 tx=0).stop 2023-11-06T02:10:16.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.475+0000 7fac2179c640 1 -- 172.21.15.49:0/305146873 reap_dead start 2023-11-06T02:10:16.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.475+0000 7fac23481080 1 -- 172.21.15.49:0/305146873 shutdown_connections 2023-11-06T02:10:16.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.475+0000 7fac23481080 1 -- 172.21.15.49:0/305146873 >> 172.21.15.49:0/305146873 conn(0x559a8cddf0e0 msgr2=0x559a8ce85160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:16.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.475+0000 7fac23481080 1 -- 172.21.15.49:0/305146873 shutdown_connections 2023-11-06T02:10:16.485 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.475+0000 7fac23481080 1 -- 172.21.15.49:0/305146873 wait complete. 2023-11-06T02:10:16.486 INFO:teuthology.orchestra.run.smithi049.stdout:mytag 2023-11-06T02:10:16.486 DEBUG:teuthology.orchestra.run.smithi049:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000001.00000000 scrub_tag 2023-11-06T02:10:16.563 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.563+0000 7f8cb343b080 1 Processor -- start 2023-11-06T02:10:16.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.563+0000 7f8cb343b080 1 -- start start 2023-11-06T02:10:16.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.563+0000 7f8cb343b080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558f1743aa80 0x558f174e1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:16.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.563+0000 7f8cb343b080 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558f17458830 0x558f17410400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:16.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.563+0000 7f8cb343b080 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558f174c7780 0x558f174c7b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:16.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.563+0000 7f8cb343b080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f17356450 con 0x558f1743aa80 2023-11-06T02:10:16.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.563+0000 7f8cb343b080 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f17356150 con 0x558f17458830 2023-11-06T02:10:16.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.563+0000 7f8cb343b080 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x558f1734cdc0 con 0x558f174c7780 2023-11-06T02:10:16.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.563+0000 7f8cb0f55640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558f1743aa80 0x558f174e1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:16.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.563+0000 7f8cb0f55640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558f1743aa80 0x558f174e1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33262/0 (socket says 172.21.15.49:33262) 2023-11-06T02:10:16.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.563+0000 7f8cb0f55640 1 -- 172.21.15.49:0/67470822 learned_addr learned my addr 172.21.15.49:0/67470822 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:16.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.563+0000 7f8cb0754640 1 --2- 172.21.15.49:0/67470822 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558f17458830 0x558f17410400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:16.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.563+0000 7f8cb1756640 1 --2- 172.21.15.49:0/67470822 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558f174c7780 0x558f174c7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:16.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.567+0000 7f8cb0f55640 1 -- 172.21.15.49:0/67470822 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558f174c7780 msgr2=0x558f174c7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:16.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.567+0000 7f8cb0f55640 1 --2- 172.21.15.49:0/67470822 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558f174c7780 0x558f174c7b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:16.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.567+0000 7f8cb0f55640 1 -- 172.21.15.49:0/67470822 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558f17458830 msgr2=0x558f17410400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:16.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.567+0000 7f8cb0f55640 1 --2- 172.21.15.49:0/67470822 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558f17458830 0x558f17410400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:16.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.567+0000 7f8cb0f55640 1 -- 172.21.15.49:0/67470822 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ca800a160 con 0x558f1743aa80 2023-11-06T02:10:16.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.567+0000 7f8cb0754640 1 --2- 172.21.15.49:0/67470822 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558f17458830 0x558f17410400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:10:16.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.567+0000 7f8cb1756640 1 --2- 172.21.15.49:0/67470822 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558f174c7780 0x558f174c7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:10:16.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.567+0000 7f8cb0f55640 1 --2- 172.21.15.49:0/67470822 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558f1743aa80 0x558f174e1820 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f8ca8019700 tx=0x7f8ca80033b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcd3491d7380ae07 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:16.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.567+0000 7f8caf752640 1 -- 172.21.15.49:0/67470822 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca80ab8c0 con 0x558f1743aa80 2023-11-06T02:10:16.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.567+0000 7f8caf752640 1 -- 172.21.15.49:0/67470822 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ca80aba20 con 0x558f1743aa80 2023-11-06T02:10:16.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.567+0000 7f8caf752640 1 -- 172.21.15.49:0/67470822 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca80abdb0 con 0x558f1743aa80 2023-11-06T02:10:16.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.567+0000 7f8cb343b080 1 -- 172.21.15.49:0/67470822 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558f1743aa80 msgr2=0x558f174e1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:16.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.567+0000 7f8cb343b080 1 --2- 172.21.15.49:0/67470822 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558f1743aa80 0x558f174e1820 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f8ca8019700 tx=0x7f8ca80033b0 comp rx=0 tx=0).stop 2023-11-06T02:10:16.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.567+0000 7f8cb343b080 1 -- 172.21.15.49:0/67470822 shutdown_connections 2023-11-06T02:10:16.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.567+0000 7f8cb343b080 1 --2- 172.21.15.49:0/67470822 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558f174c7780 0x558f174c7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:16.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.567+0000 7f8cb343b080 1 --2- 172.21.15.49:0/67470822 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558f17458830 0x558f17410400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:16.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.567+0000 7f8cb343b080 1 --2- 172.21.15.49:0/67470822 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558f1743aa80 0x558f174e1820 secure :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f8ca8019700 tx=0x7f8ca80033b0 comp rx=0 tx=0).stop 2023-11-06T02:10:16.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.567+0000 7f8cb343b080 1 -- 172.21.15.49:0/67470822 >> 172.21.15.49:0/67470822 conn(0x558f17439660 msgr2=0x558f17450200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:16.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.567+0000 7f8cb343b080 1 -- 172.21.15.49:0/67470822 shutdown_connections 2023-11-06T02:10:16.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.567+0000 7f8cb343b080 1 -- 172.21.15.49:0/67470822 wait complete. 2023-11-06T02:10:16.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.567+0000 7f8cb343b080 1 Processor -- start 2023-11-06T02:10:16.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.567+0000 7f8cb343b080 1 -- start start 2023-11-06T02:10:16.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.567+0000 7f8cb343b080 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558f17458830 0x558f174e2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:16.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.567+0000 7f8cb343b080 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558f174c7780 0x558f174ee660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:16.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.567+0000 7f8cb343b080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558f174c6e50 0x558f174c7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:16.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.567+0000 7f8cb343b080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f174eee50 con 0x558f174c6e50 2023-11-06T02:10:16.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.567+0000 7f8cb343b080 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f174ef0f0 con 0x558f17458830 2023-11-06T02:10:16.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.567+0000 7f8cb343b080 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x558f1740fd10 con 0x558f174c7780 2023-11-06T02:10:16.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.567+0000 7f8cb1756640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558f174c6e50 0x558f174c7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:16.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.567+0000 7f8cb1756640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558f174c6e50 0x558f174c7230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33268/0 (socket says 172.21.15.49:33268) 2023-11-06T02:10:16.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.567+0000 7f8cb1756640 1 -- 172.21.15.49:0/849600019 learned_addr learned my addr 172.21.15.49:0/849600019 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:16.579 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.567+0000 7f8cb0f55640 1 --2- 172.21.15.49:0/849600019 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558f17458830 0x558f174e2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:16.579 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.567+0000 7f8cb0754640 1 --2- 172.21.15.49:0/849600019 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558f174c7780 0x558f174ee660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:16.579 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.567+0000 7f8cb1756640 1 -- 172.21.15.49:0/849600019 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558f174c7780 msgr2=0x558f174ee660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:16.579 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.567+0000 7f8cb1756640 1 --2- 172.21.15.49:0/849600019 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558f174c7780 0x558f174ee660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:16.579 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.567+0000 7f8cb1756640 1 -- 172.21.15.49:0/849600019 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558f17458830 msgr2=0x558f174e2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:16.580 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.567+0000 7f8cb1756640 1 --2- 172.21.15.49:0/849600019 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558f17458830 0x558f174e2bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:16.580 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.567+0000 7f8cb1756640 1 -- 172.21.15.49:0/849600019 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f1737f190 con 0x558f174c6e50 2023-11-06T02:10:16.580 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.567+0000 7f8cb1756640 1 --2- 172.21.15.49:0/849600019 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558f174c6e50 0x558f174c7230 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f8ca0001ed0 tx=0x7f8ca000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:16.580 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.567+0000 7f8cae750640 1 -- 172.21.15.49:0/849600019 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca000f410 con 0x558f174c6e50 2023-11-06T02:10:16.580 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.567+0000 7f8cae750640 1 -- 172.21.15.49:0/849600019 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ca000f570 con 0x558f174c6e50 2023-11-06T02:10:16.581 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.567+0000 7f8cb343b080 1 -- 172.21.15.49:0/849600019 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x558f17364ff0 con 0x558f174c6e50 2023-11-06T02:10:16.581 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.567+0000 7f8cae750640 1 -- 172.21.15.49:0/849600019 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca000f920 con 0x558f174c6e50 2023-11-06T02:10:16.581 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.571+0000 7f8cb343b080 1 -- 172.21.15.49:0/849600019 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558f17358430 con 0x558f174c6e50 2023-11-06T02:10:16.581 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.571+0000 7f8cae750640 1 -- 172.21.15.49:0/849600019 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f8ca000fa80 con 0x558f174c6e50 2023-11-06T02:10:16.581 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.571+0000 7f8cae750640 1 --2- 172.21.15.49:0/849600019 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8c9c064410 0x7f8c9c0668d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:16.582 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.571+0000 7f8cae750640 1 -- 172.21.15.49:0/849600019 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f8ca0082360 con 0x558f174c6e50 2023-11-06T02:10:16.582 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.571+0000 7f8cb343b080 1 --2- 172.21.15.49:0/849600019 >> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] conn(0x558f174106f0 0x558f17410b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:16.582 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.571+0000 7f8cb0f55640 1 --2- 172.21.15.49:0/849600019 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8c9c064410 0x7f8c9c0668d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:16.582 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.571+0000 7f8cb343b080 1 -- 172.21.15.49:0/849600019 --> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] -- osd_op(unknown.0.0:1 5.7 5:e2e07930:::10000000001.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39) v8 -- 0x558f17411040 con 0x558f174106f0 2023-11-06T02:10:16.582 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.571+0000 7f8cb0754640 1 --2- 172.21.15.49:0/849600019 >> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] conn(0x558f174106f0 0x558f17410b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:16.583 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.571+0000 7f8cb0f55640 1 --2- 172.21.15.49:0/849600019 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8c9c064410 0x7f8c9c0668d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8ca80b5000 tx=0x7f8ca80a5000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:16.583 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.571+0000 7f8cb0754640 1 --2- 172.21.15.49:0/849600019 >> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] conn(0x558f174106f0 0x558f17410b00 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:16.583 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.575+0000 7f8cb0754640 1 -- 172.21.15.49:0/849600019 <== osd.5 v2:172.21.15.150:6800/38418 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (9) out=9b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7f8ca4009730 con 0x558f174106f0 2023-11-06T02:10:16.583 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.575+0000 7f8cb343b080 1 -- 172.21.15.49:0/849600019 >> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] conn(0x558f174106f0 msgr2=0x558f17410b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:16.583 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.575+0000 7f8cb343b080 1 --2- 172.21.15.49:0/849600019 >> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] conn(0x558f174106f0 0x558f17410b00 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:16.583 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.575+0000 7f8cb343b080 1 -- 172.21.15.49:0/849600019 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8c9c064410 msgr2=0x7f8c9c0668d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:16.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.575+0000 7f8cb343b080 1 --2- 172.21.15.49:0/849600019 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8c9c064410 0x7f8c9c0668d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8ca80b5000 tx=0x7f8ca80a5000 comp rx=0 tx=0).stop 2023-11-06T02:10:16.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.575+0000 7f8cb343b080 1 -- 172.21.15.49:0/849600019 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558f174c6e50 msgr2=0x558f174c7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:16.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.575+0000 7f8cb343b080 1 --2- 172.21.15.49:0/849600019 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558f174c6e50 0x558f174c7230 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f8ca0001ed0 tx=0x7f8ca000cf90 comp rx=0 tx=0).stop 2023-11-06T02:10:16.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.575+0000 7f8cb1756640 1 -- 172.21.15.49:0/849600019 reap_dead start 2023-11-06T02:10:16.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.575+0000 7f8cb343b080 1 -- 172.21.15.49:0/849600019 shutdown_connections 2023-11-06T02:10:16.585 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.575+0000 7f8cb343b080 1 -- 172.21.15.49:0/849600019 >> 172.21.15.49:0/849600019 conn(0x558f17439660 msgr2=0x558f1744feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:16.585 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.575+0000 7f8cb343b080 1 -- 172.21.15.49:0/849600019 shutdown_connections 2023-11-06T02:10:16.585 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.575+0000 7f8cb343b080 1 -- 172.21.15.49:0/849600019 wait complete. 2023-11-06T02:10:16.586 INFO:teuthology.orchestra.run.smithi049.stdout:mytag 2023-11-06T02:10:16.586 DEBUG:teuthology.orchestra.run.smithi049:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f7.00000000 scrub_tag 2023-11-06T02:10:16.668 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.667+0000 7f98b2a5c080 1 Processor -- start 2023-11-06T02:10:16.669 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.667+0000 7f98b2a5c080 1 -- start start 2023-11-06T02:10:16.672 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.671+0000 7f98b2a5c080 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555e03d71e20 0x555e03d70190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:16.672 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.671+0000 7f98b2a5c080 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555e03d72200 0x555e03edd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:16.672 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.671+0000 7f98b2a5c080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555e03edd860 0x555e03eddc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:16.673 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.671+0000 7f98b2a5c080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e03cb8450 con 0x555e03edd860 2023-11-06T02:10:16.673 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.671+0000 7f98b2a5c080 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e03cb8150 con 0x555e03d72200 2023-11-06T02:10:16.673 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.671+0000 7f98b2a5c080 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x555e03caedc0 con 0x555e03d71e20 2023-11-06T02:10:16.673 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.671+0000 7f98b0d77640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555e03edd860 0x555e03eddc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:16.674 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.671+0000 7f98b0d77640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555e03edd860 0x555e03eddc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33276/0 (socket says 172.21.15.49:33276) 2023-11-06T02:10:16.674 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.671+0000 7f98b0d77640 1 -- 172.21.15.49:0/2700184747 learned_addr learned my addr 172.21.15.49:0/2700184747 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:16.674 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.671+0000 7f98b0576640 1 --2- 172.21.15.49:0/2700184747 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555e03d71e20 0x555e03d70190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:16.674 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.671+0000 7f98afd75640 1 --2- 172.21.15.49:0/2700184747 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555e03d72200 0x555e03edd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:16.674 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.671+0000 7f98b0d77640 1 -- 172.21.15.49:0/2700184747 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555e03d71e20 msgr2=0x555e03d70190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:16.675 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.671+0000 7f98b0d77640 1 --2- 172.21.15.49:0/2700184747 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555e03d71e20 0x555e03d70190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:16.675 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.671+0000 7f98b0d77640 1 -- 172.21.15.49:0/2700184747 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555e03d72200 msgr2=0x555e03edd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:16.675 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.671+0000 7f98b0d77640 1 --2- 172.21.15.49:0/2700184747 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555e03d72200 0x555e03edd320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:16.675 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.671+0000 7f98b0d77640 1 -- 172.21.15.49:0/2700184747 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9898005df0 con 0x555e03edd860 2023-11-06T02:10:16.676 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.671+0000 7f98b0d77640 1 --2- 172.21.15.49:0/2700184747 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555e03edd860 0x555e03eddc40 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f98a800a650 tx=0x7f98a800ab10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4895bf8e1c505a19 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:16.680 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.671+0000 7f98aed73640 1 -- 172.21.15.49:0/2700184747 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a800f070 con 0x555e03edd860 2023-11-06T02:10:16.680 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.671+0000 7f98aed73640 1 -- 172.21.15.49:0/2700184747 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98a8008c00 con 0x555e03edd860 2023-11-06T02:10:16.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.671+0000 7f98aed73640 1 -- 172.21.15.49:0/2700184747 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a8013430 con 0x555e03edd860 2023-11-06T02:10:16.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.671+0000 7f98b2a5c080 1 -- 172.21.15.49:0/2700184747 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555e03edd860 msgr2=0x555e03eddc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:16.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.671+0000 7f98b2a5c080 1 --2- 172.21.15.49:0/2700184747 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555e03edd860 0x555e03eddc40 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f98a800a650 tx=0x7f98a800ab10 comp rx=0 tx=0).stop 2023-11-06T02:10:16.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.671+0000 7f98b2a5c080 1 -- 172.21.15.49:0/2700184747 shutdown_connections 2023-11-06T02:10:16.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.671+0000 7f98b2a5c080 1 --2- 172.21.15.49:0/2700184747 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555e03edd860 0x555e03eddc40 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:16.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.671+0000 7f98b2a5c080 1 --2- 172.21.15.49:0/2700184747 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555e03d72200 0x555e03edd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:16.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.671+0000 7f98b2a5c080 1 --2- 172.21.15.49:0/2700184747 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555e03d71e20 0x555e03d70190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:16.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.671+0000 7f98b2a5c080 1 -- 172.21.15.49:0/2700184747 >> 172.21.15.49:0/2700184747 conn(0x555e03d9ccf0 msgr2=0x555e03dbafb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:16.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.671+0000 7f98b2a5c080 1 -- 172.21.15.49:0/2700184747 shutdown_connections 2023-11-06T02:10:16.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.671+0000 7f98b2a5c080 1 -- 172.21.15.49:0/2700184747 wait complete. 2023-11-06T02:10:16.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.671+0000 7f98b2a5c080 1 Processor -- start 2023-11-06T02:10:16.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.671+0000 7f98b2a5c080 1 -- start start 2023-11-06T02:10:16.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.675+0000 7f98b2a5c080 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555e03d71e20 0x555e03e4ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:16.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.675+0000 7f98b2a5c080 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555e03d72200 0x555e03ed72c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:16.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.675+0000 7f98b2a5c080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555e03edd860 0x555e03ed8960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:16.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.675+0000 7f98b2a5c080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e03ed7800 con 0x555e03edd860 2023-11-06T02:10:16.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.675+0000 7f98b2a5c080 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e03ed7a40 con 0x555e03d72200 2023-11-06T02:10:16.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.675+0000 7f98b2a5c080 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x555e03ed7ce0 con 0x555e03d71e20 2023-11-06T02:10:16.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.675+0000 7f98b0d77640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555e03edd860 0x555e03ed8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:16.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.675+0000 7f98b0d77640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555e03edd860 0x555e03ed8960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33278/0 (socket says 172.21.15.49:33278) 2023-11-06T02:10:16.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.675+0000 7f98b0d77640 1 -- 172.21.15.49:0/3170186905 learned_addr learned my addr 172.21.15.49:0/3170186905 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:16.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.675+0000 7f98b0576640 1 --2- 172.21.15.49:0/3170186905 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555e03d71e20 0x555e03e4ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:16.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.675+0000 7f98afd75640 1 --2- 172.21.15.49:0/3170186905 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555e03d72200 0x555e03ed72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:16.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.675+0000 7f98b0d77640 1 -- 172.21.15.49:0/3170186905 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555e03d71e20 msgr2=0x555e03e4ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:16.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.675+0000 7f98b0d77640 1 --2- 172.21.15.49:0/3170186905 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555e03d71e20 0x555e03e4ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:16.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.675+0000 7f98b0d77640 1 -- 172.21.15.49:0/3170186905 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555e03d72200 msgr2=0x555e03ed72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:16.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.675+0000 7f98b0d77640 1 --2- 172.21.15.49:0/3170186905 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555e03d72200 0x555e03ed72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:16.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.675+0000 7f98b0d77640 1 -- 172.21.15.49:0/3170186905 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98a8005e30 con 0x555e03edd860 2023-11-06T02:10:16.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.675+0000 7f98b0576640 1 --2- 172.21.15.49:0/3170186905 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555e03d71e20 0x555e03e4ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:10:16.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.675+0000 7f98b0d77640 1 --2- 172.21.15.49:0/3170186905 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555e03edd860 0x555e03ed8960 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x555e03e46c40 tx=0x7f98a8003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:16.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.675+0000 7f98add71640 1 -- 172.21.15.49:0/3170186905 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a80056d0 con 0x555e03edd860 2023-11-06T02:10:16.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.675+0000 7f98add71640 1 -- 172.21.15.49:0/3170186905 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98a801d630 con 0x555e03edd860 2023-11-06T02:10:16.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.675+0000 7f98b2a5c080 1 -- 172.21.15.49:0/3170186905 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x555e03cc6ff0 con 0x555e03edd860 2023-11-06T02:10:16.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.675+0000 7f98add71640 1 -- 172.21.15.49:0/3170186905 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a801d9e0 con 0x555e03edd860 2023-11-06T02:10:16.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.675+0000 7f98b2a5c080 1 -- 172.21.15.49:0/3170186905 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555e03cba430 con 0x555e03edd860 2023-11-06T02:10:16.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.675+0000 7f98add71640 1 -- 172.21.15.49:0/3170186905 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f98a801a050 con 0x555e03edd860 2023-11-06T02:10:16.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.675+0000 7f98add71640 1 --2- 172.21.15.49:0/3170186905 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9890064410 0x7f98900668d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:16.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.675+0000 7f98add71640 1 -- 172.21.15.49:0/3170186905 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f98a8083ef0 con 0x555e03edd860 2023-11-06T02:10:16.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.675+0000 7f98b2a5c080 1 --2- 172.21.15.49:0/3170186905 >> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] conn(0x555e03edd300 0x555e03ed93b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:16.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.675+0000 7f98b2a5c080 1 -- 172.21.15.49:0/3170186905 --> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] -- osd_op(unknown.0.0:1 6.36 6:6fe38dc2:::100000001f7.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39) v8 -- 0x555e03ed98f0 con 0x555e03edd300 2023-11-06T02:10:16.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.675+0000 7f98b0576640 1 --2- 172.21.15.49:0/3170186905 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9890064410 0x7f98900668d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:16.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.675+0000 7f98afd75640 1 --2- 172.21.15.49:0/3170186905 >> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] conn(0x555e03edd300 0x555e03ed93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:16.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.679+0000 7f98b0576640 1 --2- 172.21.15.49:0/3170186905 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9890064410 0x7f98900668d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f98a0006fd0 tx=0x7f98a0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:16.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.679+0000 7f98afd75640 1 --2- 172.21.15.49:0/3170186905 >> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] conn(0x555e03edd300 0x555e03ed93b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:16.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.679+0000 7f98afd75640 1 -- 172.21.15.49:0/3170186905 <== osd.5 v2:172.21.15.150:6800/38418 1 ==== osd_op_reply(1 100000001f7.00000000 [getxattr (9) out=9b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7f98980aa7b0 con 0x555e03edd300 2023-11-06T02:10:16.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.679+0000 7f98b2a5c080 1 -- 172.21.15.49:0/3170186905 >> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] conn(0x555e03edd300 msgr2=0x555e03ed93b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:16.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.679+0000 7f98b2a5c080 1 --2- 172.21.15.49:0/3170186905 >> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] conn(0x555e03edd300 0x555e03ed93b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:16.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.679+0000 7f98b2a5c080 1 -- 172.21.15.49:0/3170186905 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9890064410 msgr2=0x7f98900668d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:16.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.679+0000 7f98b2a5c080 1 --2- 172.21.15.49:0/3170186905 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9890064410 0x7f98900668d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f98a0006fd0 tx=0x7f98a0008040 comp rx=0 tx=0).stop 2023-11-06T02:10:16.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.679+0000 7f98b2a5c080 1 -- 172.21.15.49:0/3170186905 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555e03edd860 msgr2=0x555e03ed8960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:16.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.679+0000 7f98b2a5c080 1 --2- 172.21.15.49:0/3170186905 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555e03edd860 0x555e03ed8960 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x555e03e46c40 tx=0x7f98a8003030 comp rx=0 tx=0).stop 2023-11-06T02:10:16.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.679+0000 7f98b0d77640 1 -- 172.21.15.49:0/3170186905 reap_dead start 2023-11-06T02:10:16.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.679+0000 7f98b2a5c080 1 -- 172.21.15.49:0/3170186905 shutdown_connections 2023-11-06T02:10:16.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.679+0000 7f98b2a5c080 1 -- 172.21.15.49:0/3170186905 >> 172.21.15.49:0/3170186905 conn(0x555e03d9ccf0 msgr2=0x555e03da04c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:16.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.679+0000 7f98b2a5c080 1 -- 172.21.15.49:0/3170186905 shutdown_connections 2023-11-06T02:10:16.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.679+0000 7f98b2a5c080 1 -- 172.21.15.49:0/3170186905 wait complete. 2023-11-06T02:10:16.697 INFO:teuthology.orchestra.run.smithi049.stdout:mytag 2023-11-06T02:10:16.697 DEBUG:teuthology.orchestra.run.smithi049:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f8.00000000 scrub_tag 2023-11-06T02:10:16.777 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.779+0000 7f09409d1080 1 Processor -- start 2023-11-06T02:10:16.778 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.779+0000 7f09409d1080 1 -- start start 2023-11-06T02:10:16.780 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.779+0000 7f09409d1080 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56197103bcd0 0x5619710db5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:16.780 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.779+0000 7f09409d1080 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56197103e480 0x56197103a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:16.781 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.779+0000 7f09409d1080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5619710e38a0 0x56197103ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:16.781 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.779+0000 7f09409d1080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x561970f53450 con 0x5619710e38a0 2023-11-06T02:10:16.781 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.779+0000 7f09409d1080 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x561970f53150 con 0x56197103e480 2023-11-06T02:10:16.781 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.779+0000 7f09409d1080 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x561970f49dc0 con 0x56197103bcd0 2023-11-06T02:10:16.781 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.779+0000 7f093ecec640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5619710e38a0 0x56197103ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:16.782 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.779+0000 7f093ecec640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5619710e38a0 0x56197103ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33294/0 (socket says 172.21.15.49:33294) 2023-11-06T02:10:16.782 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.779+0000 7f093ecec640 1 -- 172.21.15.49:0/3197262130 learned_addr learned my addr 172.21.15.49:0/3197262130 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:16.782 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.779+0000 7f093e4eb640 1 --2- 172.21.15.49:0/3197262130 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56197103bcd0 0x5619710db5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:16.782 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.779+0000 7f093dcea640 1 --2- 172.21.15.49:0/3197262130 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56197103e480 0x56197103a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:16.782 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.779+0000 7f093ecec640 1 -- 172.21.15.49:0/3197262130 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56197103bcd0 msgr2=0x5619710db5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:16.783 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.779+0000 7f093ecec640 1 --2- 172.21.15.49:0/3197262130 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56197103bcd0 0x5619710db5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:16.783 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.779+0000 7f093ecec640 1 -- 172.21.15.49:0/3197262130 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56197103e480 msgr2=0x56197103a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:16.783 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.779+0000 7f093ecec640 1 --2- 172.21.15.49:0/3197262130 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56197103e480 0x56197103a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:16.783 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.779+0000 7f093ecec640 1 -- 172.21.15.49:0/3197262130 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0938005df0 con 0x5619710e38a0 2023-11-06T02:10:16.783 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.779+0000 7f093dcea640 1 --2- 172.21.15.49:0/3197262130 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56197103e480 0x56197103a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:10:16.783 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.779+0000 7f093ecec640 1 --2- 172.21.15.49:0/3197262130 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5619710e38a0 0x56197103ad30 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f093801b8a0 tx=0x7f0938003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f14f1290375ed748 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:16.784 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.779+0000 7f093cce8640 1 -- 172.21.15.49:0/3197262130 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09380aa680 con 0x5619710e38a0 2023-11-06T02:10:16.784 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.779+0000 7f093cce8640 1 -- 172.21.15.49:0/3197262130 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09380aa7e0 con 0x5619710e38a0 2023-11-06T02:10:16.784 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.779+0000 7f093cce8640 1 -- 172.21.15.49:0/3197262130 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09380aab90 con 0x5619710e38a0 2023-11-06T02:10:16.784 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.779+0000 7f09409d1080 1 -- 172.21.15.49:0/3197262130 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5619710e38a0 msgr2=0x56197103ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:16.784 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.779+0000 7f09409d1080 1 --2- 172.21.15.49:0/3197262130 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5619710e38a0 0x56197103ad30 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f093801b8a0 tx=0x7f0938003030 comp rx=0 tx=0).stop 2023-11-06T02:10:16.785 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.779+0000 7f09409d1080 1 -- 172.21.15.49:0/3197262130 shutdown_connections 2023-11-06T02:10:16.785 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.779+0000 7f09409d1080 1 --2- 172.21.15.49:0/3197262130 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5619710e38a0 0x56197103ad30 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:16.785 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.779+0000 7f09409d1080 1 --2- 172.21.15.49:0/3197262130 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56197103e480 0x56197103a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:16.785 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.779+0000 7f09409d1080 1 --2- 172.21.15.49:0/3197262130 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56197103bcd0 0x5619710db5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:16.785 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.779+0000 7f09409d1080 1 -- 172.21.15.49:0/3197262130 >> 172.21.15.49:0/3197262130 conn(0x56197103a1a0 msgr2=0x5619710e0e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:16.786 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.779+0000 7f09409d1080 1 -- 172.21.15.49:0/3197262130 shutdown_connections 2023-11-06T02:10:16.786 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.779+0000 7f09409d1080 1 -- 172.21.15.49:0/3197262130 wait complete. 2023-11-06T02:10:16.786 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.779+0000 7f09409d1080 1 Processor -- start 2023-11-06T02:10:16.786 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.779+0000 7f09409d1080 1 -- start start 2023-11-06T02:10:16.789 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.783+0000 7f09409d1080 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56197103bcd0 0x561971054410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:16.789 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.783+0000 7f09409d1080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56197103e480 0x5619710d9e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:16.790 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.783+0000 7f09409d1080 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5619710e38a0 0x5619710da350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:16.790 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.783+0000 7f09409d1080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x561971050f60 con 0x56197103e480 2023-11-06T02:10:16.790 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.783+0000 7f09409d1080 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x561971051200 con 0x56197103bcd0 2023-11-06T02:10:16.790 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.783+0000 7f09409d1080 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x5619710514a0 con 0x5619710e38a0 2023-11-06T02:10:16.791 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.783+0000 7f093dcea640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56197103e480 0x5619710d9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:16.791 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.783+0000 7f093dcea640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56197103e480 0x5619710d9e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33310/0 (socket says 172.21.15.49:33310) 2023-11-06T02:10:16.791 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.783+0000 7f093dcea640 1 -- 172.21.15.49:0/1221853058 learned_addr learned my addr 172.21.15.49:0/1221853058 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:16.791 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.783+0000 7f093e4eb640 1 --2- 172.21.15.49:0/1221853058 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56197103bcd0 0x561971054410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:16.791 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.783+0000 7f093ecec640 1 --2- 172.21.15.49:0/1221853058 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5619710e38a0 0x5619710da350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:16.791 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.783+0000 7f093dcea640 1 -- 172.21.15.49:0/1221853058 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5619710e38a0 msgr2=0x5619710da350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:16.792 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.783+0000 7f093dcea640 1 --2- 172.21.15.49:0/1221853058 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5619710e38a0 0x5619710da350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:16.792 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.783+0000 7f093dcea640 1 -- 172.21.15.49:0/1221853058 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56197103bcd0 msgr2=0x561971054410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:16.792 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.783+0000 7f093dcea640 1 --2- 172.21.15.49:0/1221853058 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56197103bcd0 0x561971054410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:16.792 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.783+0000 7f093dcea640 1 -- 172.21.15.49:0/1221853058 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561970f7c190 con 0x56197103e480 2023-11-06T02:10:16.792 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.783+0000 7f093dcea640 1 --2- 172.21.15.49:0/1221853058 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56197103e480 0x5619710d9e10 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f093400c270 tx=0x7f093400c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:16.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.783+0000 7f092f7fe640 1 -- 172.21.15.49:0/1221853058 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0934003660 con 0x56197103e480 2023-11-06T02:10:16.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.783+0000 7f092f7fe640 1 -- 172.21.15.49:0/1221853058 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09340037c0 con 0x56197103e480 2023-11-06T02:10:16.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.783+0000 7f092f7fe640 1 -- 172.21.15.49:0/1221853058 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0934003b70 con 0x56197103e480 2023-11-06T02:10:16.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.783+0000 7f09409d1080 1 -- 172.21.15.49:0/1221853058 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x561970f61ff0 con 0x56197103e480 2023-11-06T02:10:16.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.783+0000 7f09409d1080 1 -- 172.21.15.49:0/1221853058 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561970f55430 con 0x56197103e480 2023-11-06T02:10:16.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.783+0000 7f092f7fe640 1 -- 172.21.15.49:0/1221853058 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f0934003cd0 con 0x56197103e480 2023-11-06T02:10:16.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.783+0000 7f092f7fe640 1 --2- 172.21.15.49:0/1221853058 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0920064410 0x7f09200668d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:16.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.783+0000 7f092f7fe640 1 -- 172.21.15.49:0/1221853058 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f0934081eb0 con 0x56197103e480 2023-11-06T02:10:16.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.787+0000 7f09409d1080 1 --2- 172.21.15.49:0/1221853058 >> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] conn(0x561971054950 0x56197104fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:16.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.787+0000 7f093e4eb640 1 --2- 172.21.15.49:0/1221853058 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0920064410 0x7f09200668d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:16.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.787+0000 7f09409d1080 1 -- 172.21.15.49:0/1221853058 --> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] -- osd_op(unknown.0.0:1 6.19 6:997b7129:::100000001f8.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39) v8 -- 0x561971050430 con 0x561971054950 2023-11-06T02:10:16.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.787+0000 7f093ecec640 1 --2- 172.21.15.49:0/1221853058 >> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] conn(0x561971054950 0x56197104fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:16.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.787+0000 7f093e4eb640 1 --2- 172.21.15.49:0/1221853058 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0920064410 0x7f09200668d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0930009650 tx=0x7f09300059c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:16.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.787+0000 7f093ecec640 1 --2- 172.21.15.49:0/1221853058 >> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] conn(0x561971054950 0x56197104fef0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:16.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.787+0000 7f093ecec640 1 -- 172.21.15.49:0/1221853058 <== osd.5 v2:172.21.15.150:6800/38418 1 ==== osd_op_reply(1 100000001f8.00000000 [getxattr (9) out=9b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7f09380aa950 con 0x561971054950 2023-11-06T02:10:16.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.787+0000 7f09409d1080 1 -- 172.21.15.49:0/1221853058 >> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] conn(0x561971054950 msgr2=0x56197104fef0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:16.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.787+0000 7f09409d1080 1 --2- 172.21.15.49:0/1221853058 >> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] conn(0x561971054950 0x56197104fef0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:16.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.787+0000 7f09409d1080 1 -- 172.21.15.49:0/1221853058 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0920064410 msgr2=0x7f09200668d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:16.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.787+0000 7f09409d1080 1 --2- 172.21.15.49:0/1221853058 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0920064410 0x7f09200668d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0930009650 tx=0x7f09300059c0 comp rx=0 tx=0).stop 2023-11-06T02:10:16.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.787+0000 7f09409d1080 1 -- 172.21.15.49:0/1221853058 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56197103e480 msgr2=0x5619710d9e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:16.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.787+0000 7f09409d1080 1 --2- 172.21.15.49:0/1221853058 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56197103e480 0x5619710d9e10 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f093400c270 tx=0x7f093400c730 comp rx=0 tx=0).stop 2023-11-06T02:10:16.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.787+0000 7f093ecec640 1 -- 172.21.15.49:0/1221853058 reap_dead start 2023-11-06T02:10:16.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.787+0000 7f09409d1080 1 -- 172.21.15.49:0/1221853058 shutdown_connections 2023-11-06T02:10:16.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.787+0000 7f09409d1080 1 -- 172.21.15.49:0/1221853058 >> 172.21.15.49:0/1221853058 conn(0x56197103a1a0 msgr2=0x5619710e0e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:16.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.787+0000 7f09409d1080 1 -- 172.21.15.49:0/1221853058 shutdown_connections 2023-11-06T02:10:16.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.787+0000 7f09409d1080 1 -- 172.21.15.49:0/1221853058 wait complete. 2023-11-06T02:10:16.798 INFO:teuthology.orchestra.run.smithi049.stdout:mytag 2023-11-06T02:10:16.799 DEBUG:teuthology.orchestra.run.smithi049:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000002.00000000 scrub_tag 2023-11-06T02:10:16.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.883+0000 7f28d0bae080 1 Processor -- start 2023-11-06T02:10:16.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.883+0000 7f28d0bae080 1 -- start start 2023-11-06T02:10:16.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.883+0000 7f28d0bae080 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f892f416b0 0x55f892f14030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:16.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.883+0000 7f28d0bae080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f892f41a90 0x55f892fca8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:16.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.883+0000 7f28d0bae080 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f892f5f740 0x55f893080200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:16.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.883+0000 7f28d0bae080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f892e5b450 con 0x55f892f41a90 2023-11-06T02:10:16.885 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.883+0000 7f28d0bae080 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f892e5b150 con 0x55f892f416b0 2023-11-06T02:10:16.885 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.883+0000 7f28d0bae080 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f892e51dc0 con 0x55f892f5f740 2023-11-06T02:10:16.885 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.883+0000 7f28cdec7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f892f41a90 0x55f892fca8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:16.885 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.883+0000 7f28cdec7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f892f41a90 0x55f892fca8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33320/0 (socket says 172.21.15.49:33320) 2023-11-06T02:10:16.885 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.883+0000 7f28cdec7640 1 -- 172.21.15.49:0/1415745573 learned_addr learned my addr 172.21.15.49:0/1415745573 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:16.885 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.883+0000 7f28ce6c8640 1 --2- 172.21.15.49:0/1415745573 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f892f416b0 0x55f892f14030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:16.886 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.883+0000 7f28ceec9640 1 --2- 172.21.15.49:0/1415745573 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f892f5f740 0x55f893080200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:16.886 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.883+0000 7f28cdec7640 1 -- 172.21.15.49:0/1415745573 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f892f5f740 msgr2=0x55f893080200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:16.886 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.883+0000 7f28cdec7640 1 --2- 172.21.15.49:0/1415745573 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f892f5f740 0x55f893080200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:16.886 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.883+0000 7f28cdec7640 1 -- 172.21.15.49:0/1415745573 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f892f416b0 msgr2=0x55f892f14030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:16.886 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.883+0000 7f28cdec7640 1 --2- 172.21.15.49:0/1415745573 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f892f416b0 0x55f892f14030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:16.887 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.883+0000 7f28cdec7640 1 -- 172.21.15.49:0/1415745573 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28c8005df0 con 0x55f892f41a90 2023-11-06T02:10:16.887 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.883+0000 7f28ce6c8640 1 --2- 172.21.15.49:0/1415745573 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f892f416b0 0x55f892f14030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:10:16.887 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.883+0000 7f28cdec7640 1 --2- 172.21.15.49:0/1415745573 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f892f41a90 0x55f892fca8e0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f28b8001d70 tx=0x7f28b800b5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bf9c66b06bccc56 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:16.887 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.883+0000 7f28ccec5640 1 -- 172.21.15.49:0/1415745573 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b8014070 con 0x55f892f41a90 2023-11-06T02:10:16.887 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.883+0000 7f28ccec5640 1 -- 172.21.15.49:0/1415745573 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28b800f410 con 0x55f892f41a90 2023-11-06T02:10:16.887 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.883+0000 7f28ccec5640 1 -- 172.21.15.49:0/1415745573 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b800f7c0 con 0x55f892f41a90 2023-11-06T02:10:16.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.883+0000 7f28d0bae080 1 -- 172.21.15.49:0/1415745573 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f892f41a90 msgr2=0x55f892fca8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:16.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.883+0000 7f28d0bae080 1 --2- 172.21.15.49:0/1415745573 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f892f41a90 0x55f892fca8e0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f28b8001d70 tx=0x7f28b800b5c0 comp rx=0 tx=0).stop 2023-11-06T02:10:16.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.883+0000 7f28d0bae080 1 -- 172.21.15.49:0/1415745573 shutdown_connections 2023-11-06T02:10:16.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.883+0000 7f28d0bae080 1 --2- 172.21.15.49:0/1415745573 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f892f41a90 0x55f892fca8e0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:16.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.883+0000 7f28d0bae080 1 --2- 172.21.15.49:0/1415745573 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f892f5f740 0x55f893080200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:16.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.883+0000 7f28d0bae080 1 --2- 172.21.15.49:0/1415745573 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f892f416b0 0x55f892f14030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:16.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.883+0000 7f28d0bae080 1 -- 172.21.15.49:0/1415745573 >> 172.21.15.49:0/1415745573 conn(0x55f892f40020 msgr2=0x55f892f43150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:16.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.883+0000 7f28d0bae080 1 -- 172.21.15.49:0/1415745573 shutdown_connections 2023-11-06T02:10:16.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.883+0000 7f28d0bae080 1 -- 172.21.15.49:0/1415745573 wait complete. 2023-11-06T02:10:16.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.887+0000 7f28d0bae080 1 Processor -- start 2023-11-06T02:10:16.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.887+0000 7f28d0bae080 1 -- start start 2023-11-06T02:10:16.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.887+0000 7f28d0bae080 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f892f416b0 0x55f892f15060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:16.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.887+0000 7f28d0bae080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f892f41a90 0x55f892f15860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:16.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.887+0000 7f28d0bae080 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f892f5f740 0x55f892f175b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:16.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.887+0000 7f28d0bae080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f892f14140 con 0x55f892f41a90 2023-11-06T02:10:16.895 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.887+0000 7f28d0bae080 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f892f143e0 con 0x55f892f416b0 2023-11-06T02:10:16.895 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.887+0000 7f28d0bae080 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f892f14680 con 0x55f892f5f740 2023-11-06T02:10:16.895 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.887+0000 7f28cdec7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f892f41a90 0x55f892f15860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:16.895 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.887+0000 7f28cdec7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f892f41a90 0x55f892f15860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33330/0 (socket says 172.21.15.49:33330) 2023-11-06T02:10:16.895 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.887+0000 7f28cdec7640 1 -- 172.21.15.49:0/3266748678 learned_addr learned my addr 172.21.15.49:0/3266748678 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:16.896 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.887+0000 7f28ceec9640 1 --2- 172.21.15.49:0/3266748678 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f892f5f740 0x55f892f175b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:16.896 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.887+0000 7f28ce6c8640 1 --2- 172.21.15.49:0/3266748678 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f892f416b0 0x55f892f15060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:16.896 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.887+0000 7f28cdec7640 1 -- 172.21.15.49:0/3266748678 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f892f5f740 msgr2=0x55f892f175b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:16.896 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.887+0000 7f28cdec7640 1 --2- 172.21.15.49:0/3266748678 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f892f5f740 0x55f892f175b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:16.896 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.887+0000 7f28cdec7640 1 -- 172.21.15.49:0/3266748678 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f892f416b0 msgr2=0x55f892f15060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:16.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.887+0000 7f28cdec7640 1 --2- 172.21.15.49:0/3266748678 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f892f416b0 0x55f892f15060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:16.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.887+0000 7f28cdec7640 1 -- 172.21.15.49:0/3266748678 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f892e5d0a0 con 0x55f892f41a90 2023-11-06T02:10:16.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.887+0000 7f28cdec7640 1 --2- 172.21.15.49:0/3266748678 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f892f41a90 0x55f892f15860 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f28b800b600 tx=0x7f28b8000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:16.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.887+0000 7f28c77fe640 1 -- 172.21.15.49:0/3266748678 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b8014040 con 0x55f892f41a90 2023-11-06T02:10:16.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.887+0000 7f28c77fe640 1 -- 172.21.15.49:0/3266748678 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28b801b860 con 0x55f892f41a90 2023-11-06T02:10:16.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.887+0000 7f28d0bae080 1 -- 172.21.15.49:0/3266748678 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f892e69ff0 con 0x55f892f41a90 2023-11-06T02:10:16.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.887+0000 7f28c77fe640 1 -- 172.21.15.49:0/3266748678 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b801bc10 con 0x55f892f41a90 2023-11-06T02:10:16.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.887+0000 7f28d0bae080 1 -- 172.21.15.49:0/3266748678 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f892e5d430 con 0x55f892f41a90 2023-11-06T02:10:16.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.891+0000 7f28c77fe640 1 -- 172.21.15.49:0/3266748678 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f28b80193c0 con 0x55f892f41a90 2023-11-06T02:10:16.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.891+0000 7f28c77fe640 1 --2- 172.21.15.49:0/3266748678 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f28b0064410 0x7f28b00668d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:16.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.891+0000 7f28c77fe640 1 -- 172.21.15.49:0/3266748678 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f28b8083840 con 0x55f892f41a90 2023-11-06T02:10:16.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.891+0000 7f28d0bae080 1 --2- 172.21.15.49:0/3266748678 >> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] conn(0x55f892f15da0 0x55f892fe2570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:16.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.891+0000 7f28d0bae080 1 -- 172.21.15.49:0/3266748678 --> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] -- osd_op(unknown.0.0:1 6.12 6:4bec3886:::10000000002.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39) v8 -- 0x55f8930872e0 con 0x55f892f15da0 2023-11-06T02:10:16.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.891+0000 7f28ce6c8640 1 --2- 172.21.15.49:0/3266748678 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f28b0064410 0x7f28b00668d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:16.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.891+0000 7f28ceec9640 1 --2- 172.21.15.49:0/3266748678 >> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] conn(0x55f892f15da0 0x55f892fe2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:16.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.891+0000 7f28ceec9640 1 --2- 172.21.15.49:0/3266748678 >> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] conn(0x55f892f15da0 0x55f892fe2570 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:16.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.891+0000 7f28ce6c8640 1 --2- 172.21.15.49:0/3266748678 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f28b0064410 0x7f28b00668d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f28c000e7b0 tx=0x7f28c0007040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:16.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.891+0000 7f28ceec9640 1 -- 172.21.15.49:0/3266748678 <== osd.7 v2:172.21.15.150:6816/38421 1 ==== osd_op_reply(1 10000000002.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x7f28c80af020 con 0x55f892f15da0 2023-11-06T02:10:16.900 INFO:teuthology.orchestra.run.smithi049.stderr:error getting xattr cephfs_data/10000000002.00000000/scrub_tag: (61) No data available 2023-11-06T02:10:16.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.891+0000 7f28d0bae080 1 -- 172.21.15.49:0/3266748678 >> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] conn(0x55f892f15da0 msgr2=0x55f892fe2570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:16.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.891+0000 7f28d0bae080 1 --2- 172.21.15.49:0/3266748678 >> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] conn(0x55f892f15da0 0x55f892fe2570 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:16.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.891+0000 7f28d0bae080 1 -- 172.21.15.49:0/3266748678 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f28b0064410 msgr2=0x7f28b00668d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:16.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.891+0000 7f28d0bae080 1 --2- 172.21.15.49:0/3266748678 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f28b0064410 0x7f28b00668d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f28c000e7b0 tx=0x7f28c0007040 comp rx=0 tx=0).stop 2023-11-06T02:10:16.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.891+0000 7f28d0bae080 1 -- 172.21.15.49:0/3266748678 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f892f41a90 msgr2=0x55f892f15860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:16.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.891+0000 7f28d0bae080 1 --2- 172.21.15.49:0/3266748678 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f892f41a90 0x55f892f15860 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f28b800b600 tx=0x7f28b8000ee0 comp rx=0 tx=0).stop 2023-11-06T02:10:16.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.891+0000 7f28ceec9640 1 -- 172.21.15.49:0/3266748678 reap_dead start 2023-11-06T02:10:16.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.891+0000 7f28d0bae080 1 -- 172.21.15.49:0/3266748678 shutdown_connections 2023-11-06T02:10:16.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.891+0000 7f28d0bae080 1 -- 172.21.15.49:0/3266748678 >> 172.21.15.49:0/3266748678 conn(0x55f892f40020 msgr2=0x55f892f5b190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:16.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.891+0000 7f28d0bae080 1 -- 172.21.15.49:0/3266748678 shutdown_connections 2023-11-06T02:10:16.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:16.891+0000 7f28d0bae080 1 -- 172.21.15.49:0/3266748678 wait complete. 2023-11-06T02:10:16.903 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T02:10:16.903 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-06T02:10:16.904 DEBUG:teuthology.orchestra.run.smithi049:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T02:10:16.999 DEBUG:teuthology.orchestra.run.smithi150:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T02:10:17.051 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-06T02:10:17.051 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-11-06T02:10:17.052 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T02:10:17.058 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336476k,nr_inodes=4084119,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16105 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-06T02:10:17.058 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi049.front.sepia.ceph.com 2023-11-06T02:10:17.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:10:17.059 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-06T02:10:17.105 INFO:teuthology.orchestra.run.smithi049.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-11-06T02:10:17.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T02:10:17.107 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-11-06T02:10:17.107 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T02:10:17.114 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336552k,nr_inodes=4084138,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19862 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274692k,nr_inodes=818673,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-06T02:10:17.115 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi150.front.sepia.ceph.com 2023-11-06T02:10:17.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:10:17.115 DEBUG:teuthology.orchestra.run.smithi150:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-06T02:10:17.161 INFO:teuthology.orchestra.run.smithi150.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-11-06T02:10:17.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T02:10:17.163 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-11-06T02:10:17.164 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T02:10:17.170 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336476k,nr_inodes=4084119,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16105 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-06T02:10:17.170 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi049.front.sepia.ceph.com 2023-11-06T02:10:17.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:10:17.171 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-06T02:10:17.217 INFO:teuthology.orchestra.run.smithi049.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-11-06T02:10:17.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T02:10:17.219 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-11-06T02:10:17.219 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T02:10:17.226 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336552k,nr_inodes=4084138,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19862 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274692k,nr_inodes=818673,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-06T02:10:17.226 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi150.front.sepia.ceph.com 2023-11-06T02:10:17.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:10:17.227 DEBUG:teuthology.orchestra.run.smithi150:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-06T02:10:17.274 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T02:10:17.275 INFO:teuthology.orchestra.run.smithi150.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-11-06T02:10:17.276 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T02:10:17.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.411+0000 7fdeb5f9f640 1 Processor -- start 2023-11-06T02:10:17.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.411+0000 7fdeb5f9f640 1 -- start start 2023-11-06T02:10:17.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.411+0000 7fdeb5f9f640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fdeb00b0790 0x7fdeb00c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:17.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.411+0000 7fdeb5f9f640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fdeb00cb010 0x7fdeb00c2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:17.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.411+0000 7fdeb5f9f640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdeb00cab80 con 0x7fdeb015ab50 2023-11-06T02:10:17.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.411+0000 7fdeb5f9f640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdeb015af30 con 0x7fdeb00cb010 2023-11-06T02:10:17.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.411+0000 7fdeb5f9f640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdeb00cb5b0 con 0x7fdeb00b0790 2023-11-06T02:10:17.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.411+0000 7fdeaffff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fdeb00cb010 0x7fdeb00c2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:17.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.411+0000 7fdeaffff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fdeb00cb010 0x7fdeb00c2710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33332/0 (socket says 172.21.15.49:33332) 2023-11-06T02:10:17.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.411+0000 7fdeaf7fe640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fdeb015ab50 0x7fdeb00b0080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:42478/0 (socket says 172.21.15.49:42478) 2023-11-06T02:10:17.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.411+0000 7fdeaffff640 1 -- 172.21.15.49:0/4131411955 learned_addr learned my addr 172.21.15.49:0/4131411955 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:17.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.411+0000 7fdeaeffd640 1 --2- 172.21.15.49:0/4131411955 >> v2:172.21.15.150:3300/0 conn(0x7fdeb00b0790 0x7fdeb00c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:17.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.411+0000 7fdeaffff640 1 -- 172.21.15.49:0/4131411955 >> v2:172.21.15.150:3300/0 conn(0x7fdeb00b0790 msgr2=0x7fdeb00c9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:17.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.411+0000 7fdeaffff640 1 --2- 172.21.15.49:0/4131411955 >> v2:172.21.15.150:3300/0 conn(0x7fdeb00b0790 0x7fdeb00c9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:17.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.411+0000 7fdeaffff640 1 -- 172.21.15.49:0/4131411955 >> v1:172.21.15.49:6789/0 conn(0x7fdeb015ab50 legacy=0x7fdeb00b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:17.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.411+0000 7fdeae7fc640 1 -- 172.21.15.49:0/4131411955 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2532067358 0 0) 0x7fdeb00cab80 con 0x7fdeb015ab50 2023-11-06T02:10:17.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.411+0000 7fdeaffff640 1 -- 172.21.15.49:0/4131411955 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeb00cb890 con 0x7fdeb00cb010 2023-11-06T02:10:17.417 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.415+0000 7fdeaffff640 1 --2- 172.21.15.49:0/4131411955 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdeb00cb010 0x7fdeb00c2710 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fdeb006dde0 tx=0x7fdea801c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4576d235e1eb71d0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:17.417 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.415+0000 7fdeae7fc640 1 -- 172.21.15.49:0/4131411955 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea80a5860 con 0x7fdeb00cb010 2023-11-06T02:10:17.418 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.415+0000 7fdeae7fc640 1 -- 172.21.15.49:0/4131411955 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdea80a59c0 con 0x7fdeb00cb010 2023-11-06T02:10:17.418 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.415+0000 7fdeae7fc640 1 -- 172.21.15.49:0/4131411955 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea80a5d50 con 0x7fdeb00cb010 2023-11-06T02:10:17.418 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.415+0000 7fdeb5f9f640 1 -- 172.21.15.49:0/4131411955 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdeb00cb010 msgr2=0x7fdeb00c2710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:17.418 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.415+0000 7fdeb5f9f640 1 --2- 172.21.15.49:0/4131411955 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdeb00cb010 0x7fdeb00c2710 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fdeb006dde0 tx=0x7fdea801c9a0 comp rx=0 tx=0).stop 2023-11-06T02:10:17.419 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.415+0000 7fdeb5f9f640 1 -- 172.21.15.49:0/4131411955 shutdown_connections 2023-11-06T02:10:17.419 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.415+0000 7fdeb5f9f640 1 --2- 172.21.15.49:0/4131411955 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdeb00cb010 0x7fdeb00c2710 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:17.419 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.415+0000 7fdeb5f9f640 1 --2- 172.21.15.49:0/4131411955 >> v2:172.21.15.150:3300/0 conn(0x7fdeb00b0790 0x7fdeb00c9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:17.419 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.415+0000 7fdeb5f9f640 1 -- 172.21.15.49:0/4131411955 >> 172.21.15.49:0/4131411955 conn(0x7fdeb006c0a0 msgr2=0x7fdeb006c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:17.419 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.415+0000 7fdeb5f9f640 1 -- 172.21.15.49:0/4131411955 shutdown_connections 2023-11-06T02:10:17.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.415+0000 7fdeb5f9f640 1 -- 172.21.15.49:0/4131411955 wait complete. 2023-11-06T02:10:17.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.415+0000 7fdeb5f9f640 1 Processor -- start 2023-11-06T02:10:17.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.415+0000 7fdeb5f9f640 1 -- start start 2023-11-06T02:10:17.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.415+0000 7fdeb5f9f640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdeb00b0790 0x7fdeb0155be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:17.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.415+0000 7fdeb5f9f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdeb00cb010 0x7fdeb015ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:17.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.415+0000 7fdeb5f9f640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdeb015ab50 0x7fdeb015cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:17.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.415+0000 7fdeb5f9f640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb00c71f0 con 0x7fdeb00cb010 2023-11-06T02:10:17.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.415+0000 7fdeb5f9f640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb00c6670 con 0x7fdeb015ab50 2023-11-06T02:10:17.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.415+0000 7fdeb5f9f640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb00c6950 con 0x7fdeb00b0790 2023-11-06T02:10:17.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.415+0000 7fdeaeffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdeb00cb010 0x7fdeb015ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:17.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.415+0000 7fdeaeffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdeb00cb010 0x7fdeb015ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33334/0 (socket says 172.21.15.49:33334) 2023-11-06T02:10:17.422 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.415+0000 7fdeaeffd640 1 -- 172.21.15.49:0/1601787088 learned_addr learned my addr 172.21.15.49:0/1601787088 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:17.422 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.419+0000 7fdeaffff640 1 --2- 172.21.15.49:0/1601787088 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdeb015ab50 0x7fdeb015cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:17.422 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.419+0000 7fdeaf7fe640 1 --2- 172.21.15.49:0/1601787088 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdeb00b0790 0x7fdeb0155be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:17.422 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.419+0000 7fdeaeffd640 1 -- 172.21.15.49:0/1601787088 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdeb00b0790 msgr2=0x7fdeb0155be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:17.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.419+0000 7fdeaeffd640 1 --2- 172.21.15.49:0/1601787088 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdeb00b0790 0x7fdeb0155be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:17.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.419+0000 7fdeaeffd640 1 -- 172.21.15.49:0/1601787088 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdeb015ab50 msgr2=0x7fdeb015cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:17.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.419+0000 7fdeaeffd640 1 --2- 172.21.15.49:0/1601787088 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdeb015ab50 0x7fdeb015cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:17.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.419+0000 7fdeaeffd640 1 -- 172.21.15.49:0/1601787088 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdea8009b30 con 0x7fdeb00cb010 2023-11-06T02:10:17.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.419+0000 7fdeaf7fe640 1 --2- 172.21.15.49:0/1601787088 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdeb00b0790 0x7fdeb0155be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:10:17.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.419+0000 7fdeaeffd640 1 --2- 172.21.15.49:0/1601787088 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdeb00cb010 0x7fdeb015ca90 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fdea400d800 tx=0x7fdea400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:17.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.419+0000 7fde97fff640 1 -- 172.21.15.49:0/1601787088 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea400b6c0 con 0x7fdeb00cb010 2023-11-06T02:10:17.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.419+0000 7fde97fff640 1 -- 172.21.15.49:0/1601787088 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdea400b820 con 0x7fdeb00cb010 2023-11-06T02:10:17.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.419+0000 7fde97fff640 1 -- 172.21.15.49:0/1601787088 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea400bb90 con 0x7fdeb00cb010 2023-11-06T02:10:17.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.419+0000 7fdeb5f9f640 1 -- 172.21.15.49:0/1601787088 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdeb00c6c30 con 0x7fdeb00cb010 2023-11-06T02:10:17.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.419+0000 7fdeb5f9f640 1 -- 172.21.15.49:0/1601787088 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdeb0156120 con 0x7fdeb00cb010 2023-11-06T02:10:17.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.419+0000 7fdeb5f9f640 1 -- 172.21.15.49:0/1601787088 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdeb00b0de0 con 0x7fdeb00cb010 2023-11-06T02:10:17.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.419+0000 7fde97fff640 1 -- 172.21.15.49:0/1601787088 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fdea400bcf0 con 0x7fdeb00cb010 2023-11-06T02:10:17.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.419+0000 7fde97fff640 1 --2- 172.21.15.49:0/1601787088 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fde9c065a60 0x7fde9c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:17.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.419+0000 7fde97fff640 1 -- 172.21.15.49:0/1601787088 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6091+0+0 (secure 0 0 0) 0x7fdea4083940 con 0x7fdeb00cb010 2023-11-06T02:10:17.427 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.419+0000 7fdeaf7fe640 1 --2- 172.21.15.49:0/1601787088 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fde9c065a60 0x7fde9c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:17.427 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.423+0000 7fdeaf7fe640 1 --2- 172.21.15.49:0/1601787088 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fde9c065a60 0x7fde9c067f20 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fdea0001030 tx=0x7fdea0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:17.427 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.423+0000 7fde97fff640 1 -- 172.21.15.49:0/1601787088 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fdea40510e0 con 0x7fdeb00cb010 2023-11-06T02:10:17.615 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.615+0000 7fdeb5f9f640 1 -- 172.21.15.49:0/1601787088 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdeb0074040 con 0x7fdeb00cb010 2023-11-06T02:10:17.615 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.615+0000 7fde97fff640 1 -- 172.21.15.49:0/1601787088 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4651 (secure 0 0 0) 0x7fdea4011340 con 0x7fdeb00cb010 2023-11-06T02:10:17.616 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:10:17.616 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5032,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/280428805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":280428805},{"type":"v1","addr":"172.21.15.150:6835","nonce":280428805}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":5035,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/2734589844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2734589844},{"type":"v1","addr":"172.21.15.49:6833","nonce":2734589844}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":5038,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/1319221246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":1319221246},{"type":"v1","addr":"172.21.15.150:6837","nonce":1319221246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T02:09:39.873514+0000","modified":"2023-11-06T02:09:43.132890+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5041},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5041":{"gid":5041,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.49:6835/1547192091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":1547192091},{"type":"v1","addr":"172.21.15.49:6835","nonce":1547192091}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-06T02:10:17.616 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 15 2023-11-06T02:10:17.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.619+0000 7fdeb5f9f640 1 -- 172.21.15.49:0/1601787088 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fde9c065a60 msgr2=0x7fde9c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:17.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.619+0000 7fdeb5f9f640 1 --2- 172.21.15.49:0/1601787088 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fde9c065a60 0x7fde9c067f20 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fdea0001030 tx=0x7fdea0009040 comp rx=0 tx=0).stop 2023-11-06T02:10:17.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.619+0000 7fdeb5f9f640 1 -- 172.21.15.49:0/1601787088 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdeb00cb010 msgr2=0x7fdeb015ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:17.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.619+0000 7fdeb5f9f640 1 --2- 172.21.15.49:0/1601787088 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdeb00cb010 0x7fdeb015ca90 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fdea400d800 tx=0x7fdea400dcc0 comp rx=0 tx=0).stop 2023-11-06T02:10:17.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.619+0000 7fdeb5f9f640 1 -- 172.21.15.49:0/1601787088 shutdown_connections 2023-11-06T02:10:17.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.619+0000 7fdeb5f9f640 1 --2- 172.21.15.49:0/1601787088 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fde9c065a60 0x7fde9c067f20 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:17.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.619+0000 7fdeb5f9f640 1 --2- 172.21.15.49:0/1601787088 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdeb00cb010 0x7fdeb015ca90 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:17.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.619+0000 7fdeb5f9f640 1 --2- 172.21.15.49:0/1601787088 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdeb015ab50 0x7fdeb015cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:17.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.619+0000 7fdeb5f9f640 1 --2- 172.21.15.49:0/1601787088 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdeb00b0790 0x7fdeb0155be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:17.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.619+0000 7fdeb5f9f640 1 -- 172.21.15.49:0/1601787088 >> 172.21.15.49:0/1601787088 conn(0x7fdeb006c0a0 msgr2=0x7fdeb00b0050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:17.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.619+0000 7fdeb5f9f640 1 -- 172.21.15.49:0/1601787088 shutdown_connections 2023-11-06T02:10:17.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.619+0000 7fdeb5f9f640 1 -- 172.21.15.49:0/1601787088 wait complete. 2023-11-06T02:10:17.634 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T02:10:17.745 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.747+0000 7f87d068e640 1 Processor -- start 2023-11-06T02:10:17.746 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.747+0000 7f87d068e640 1 -- start start 2023-11-06T02:10:17.747 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.747+0000 7f87d068e640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f87c8071bb0 0x7f87c8161650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:17.747 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.747+0000 7f87d068e640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87c80d17c0 con 0x7f87c80b1f90 2023-11-06T02:10:17.747 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.747+0000 7f87d068e640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87c80ce6c0 con 0x7f87c80cda70 2023-11-06T02:10:17.748 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.747+0000 7f87d068e640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87c80cdf20 con 0x7f87c8071bb0 2023-11-06T02:10:17.748 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.747+0000 7f87cec2b640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f87c8071bb0 0x7f87c8161650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:17.748 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.747+0000 7f87cec2b640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f87c8071bb0 0x7f87c8161650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33344/0 (socket says 172.21.15.49:33344) 2023-11-06T02:10:17.748 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.747+0000 7f87cec2b640 1 -- 172.21.15.49:0/859637968 learned_addr learned my addr 172.21.15.49:0/859637968 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:17.748 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.747+0000 7f87ce42a640 1 --1- 172.21.15.49:0/859637968 >> v1:172.21.15.49:6789/0 conn(0x7f87c80b1f90 0x7f87c80d1c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:42484/0 (socket says 172.21.15.49:42484) 2023-11-06T02:10:17.748 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.747+0000 7f87cec2b640 1 -- 172.21.15.49:0/859637968 >> v1:172.21.15.150:6789/0 conn(0x7f87c80cda70 legacy=0x7f87c80714a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:17.749 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.747+0000 7f87cec2b640 1 -- 172.21.15.49:0/859637968 >> v1:172.21.15.49:6789/0 conn(0x7f87c80b1f90 legacy=0x7f87c80d1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:17.749 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.747+0000 7f87cec2b640 1 -- 172.21.15.49:0/859637968 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87c80d2360 con 0x7f87c8071bb0 2023-11-06T02:10:17.750 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.747+0000 7f87cec2b640 1 --2- 172.21.15.49:0/859637968 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87c8071bb0 0x7f87c8161650 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f87c0009e50 tx=0x7f87c001ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1798ea59121fa295 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:17.750 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.751+0000 7f87cd428640 1 -- 172.21.15.49:0/859637968 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c00a57d0 con 0x7f87c8071bb0 2023-11-06T02:10:17.750 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.751+0000 7f87cd428640 1 -- 172.21.15.49:0/859637968 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87c00a5930 con 0x7f87c8071bb0 2023-11-06T02:10:17.751 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.751+0000 7f87cd428640 1 -- 172.21.15.49:0/859637968 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c00a5cc0 con 0x7f87c8071bb0 2023-11-06T02:10:17.751 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.751+0000 7f87d068e640 1 -- 172.21.15.49:0/859637968 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87c8071bb0 msgr2=0x7f87c8161650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:17.751 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.751+0000 7f87d068e640 1 --2- 172.21.15.49:0/859637968 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87c8071bb0 0x7f87c8161650 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f87c0009e50 tx=0x7f87c001ca10 comp rx=0 tx=0).stop 2023-11-06T02:10:17.751 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.751+0000 7f87d068e640 1 -- 172.21.15.49:0/859637968 shutdown_connections 2023-11-06T02:10:17.752 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.751+0000 7f87d068e640 1 --2- 172.21.15.49:0/859637968 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87c8071bb0 0x7f87c8161650 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:17.752 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.751+0000 7f87d068e640 1 -- 172.21.15.49:0/859637968 >> 172.21.15.49:0/859637968 conn(0x7f87c80d2aa0 msgr2=0x7f87c80d2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:17.752 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.751+0000 7f87d068e640 1 -- 172.21.15.49:0/859637968 shutdown_connections 2023-11-06T02:10:17.752 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.751+0000 7f87d068e640 1 -- 172.21.15.49:0/859637968 wait complete. 2023-11-06T02:10:17.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.751+0000 7f87d068e640 1 Processor -- start 2023-11-06T02:10:17.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.751+0000 7f87d068e640 1 -- start start 2023-11-06T02:10:17.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.751+0000 7f87d068e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f87c8071bb0 0x7f87c8157790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:17.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.751+0000 7f87d068e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f87c80b1f90 0x7f87c8154ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:17.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.751+0000 7f87d068e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87c80cda70 0x7f87c80cadf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:17.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.751+0000 7f87d068e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87c8158540 con 0x7f87c80cda70 2023-11-06T02:10:17.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.751+0000 7f87d068e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87c80ca6c0 con 0x7f87c8071bb0 2023-11-06T02:10:17.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.751+0000 7f87d068e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87c80ca960 con 0x7f87c80b1f90 2023-11-06T02:10:17.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.751+0000 7f87cec2b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87c80cda70 0x7f87c80cadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:17.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.751+0000 7f87cec2b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87c80cda70 0x7f87c80cadf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33354/0 (socket says 172.21.15.49:33354) 2023-11-06T02:10:17.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.751+0000 7f87cec2b640 1 -- 172.21.15.49:0/2680256592 learned_addr learned my addr 172.21.15.49:0/2680256592 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:17.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.751+0000 7f87cec2b640 1 -- 172.21.15.49:0/2680256592 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f87c80b1f90 msgr2=0x7f87c8154ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:17.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.751+0000 7f87cec2b640 1 --2- 172.21.15.49:0/2680256592 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f87c80b1f90 0x7f87c8154ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:17.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.751+0000 7f87cec2b640 1 -- 172.21.15.49:0/2680256592 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f87c8071bb0 msgr2=0x7f87c8157790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:17.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.751+0000 7f87cec2b640 1 --2- 172.21.15.49:0/2680256592 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f87c8071bb0 0x7f87c8157790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:17.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.751+0000 7f87cec2b640 1 -- 172.21.15.49:0/2680256592 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87c0009b30 con 0x7f87c80cda70 2023-11-06T02:10:17.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.751+0000 7f87cec2b640 1 --2- 172.21.15.49:0/2680256592 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87c80cda70 0x7f87c80cadf0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f87c00af040 tx=0x7f87c00197b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:17.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.751+0000 7f87beffd640 1 -- 172.21.15.49:0/2680256592 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c00b8670 con 0x7f87c80cda70 2023-11-06T02:10:17.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.751+0000 7f87beffd640 1 -- 172.21.15.49:0/2680256592 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87c00b87d0 con 0x7f87c80cda70 2023-11-06T02:10:17.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.751+0000 7f87beffd640 1 -- 172.21.15.49:0/2680256592 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c00b8b60 con 0x7f87c80cda70 2023-11-06T02:10:17.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.751+0000 7f87d068e640 1 -- 172.21.15.49:0/2680256592 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87c80cc940 con 0x7f87c80cda70 2023-11-06T02:10:17.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.751+0000 7f87d068e640 1 -- 172.21.15.49:0/2680256592 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87c8155020 con 0x7f87c80cda70 2023-11-06T02:10:17.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.755+0000 7f87d068e640 1 -- 172.21.15.49:0/2680256592 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87c80ccdf0 con 0x7f87c80cda70 2023-11-06T02:10:17.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.755+0000 7f87beffd640 1 -- 172.21.15.49:0/2680256592 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f87c00b8cc0 con 0x7f87c80cda70 2023-11-06T02:10:17.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.755+0000 7f87beffd640 1 --2- 172.21.15.49:0/2680256592 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f87ac065a60 0x7f87ac067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:17.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.755+0000 7f87beffd640 1 -- 172.21.15.49:0/2680256592 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f87c011e860 con 0x7f87c80cda70 2023-11-06T02:10:17.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.755+0000 7f87ce42a640 1 --2- 172.21.15.49:0/2680256592 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f87ac065a60 0x7f87ac067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:17.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.755+0000 7f87ce42a640 1 --2- 172.21.15.49:0/2680256592 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f87ac065a60 0x7f87ac067f20 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f87c40015f0 tx=0x7f87c40079d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:17.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.759+0000 7f87beffd640 1 -- 172.21.15.49:0/2680256592 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f87c00ec250 con 0x7f87c80cda70 2023-11-06T02:10:17.912 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.911+0000 7f87d068e640 1 -- 172.21.15.49:0/2680256592 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f87c80caaf0 con 0x7f87c80cda70 2023-11-06T02:10:17.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.915+0000 7f87beffd640 1 -- 172.21.15.49:0/2680256592 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+17932 (secure 0 0 0) 0x7f87c00bc090 con 0x7f87c80cda70 2023-11-06T02:10:17.914 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:10:17.915 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":39,"fsid":"1a8b035c-6ba4-499a-b876-aea47fd9c0d4","created":"2023-11-06T02:08:06.440857+0000","modified":"2023-11-06T02:10:14.167395+0000","last_up_change":"2023-11-06T02:08:17.982372+0000","last_in_change":"2023-11-06T02:08:08.906631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T02:08:19.252222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T02:08:24.776211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-06T02:09:37.906848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-06T02:09:38.849380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ad55af8-0571-4e6a-8722-b5d429b7496d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6809","nonce":38116},{"type":"v1","addr":"172.21.15.49:6811","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6813","nonce":38116},{"type":"v1","addr":"172.21.15.49:6815","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6821","nonce":38116},{"type":"v1","addr":"172.21.15.49:6823","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6817","nonce":38116},{"type":"v1","addr":"172.21.15.49:6819","nonce":38116}]},"public_addr":"172.21.15.49:6811/38116","cluster_addr":"172.21.15.49:6815/38116","heartbeat_back_addr":"172.21.15.49:6823/38116","heartbeat_front_addr":"172.21.15.49:6819/38116","state":["exists","up"]},{"osd":1,"uuid":"c8b31f85-a757-43ed-9419-351acc46611b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":38113},{"type":"v1","addr":"172.21.15.49:6810","nonce":38113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":38113},{"type":"v1","addr":"172.21.15.49:6814","nonce":38113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":38113},{"type":"v1","addr":"172.21.15.49:6822","nonce":38113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":38113},{"type":"v1","addr":"172.21.15.49:6818","nonce":38113}]},"public_addr":"172.21.15.49:6810/38113","cluster_addr":"172.21.15.49:6814/38113","heartbeat_back_addr":"172.21.15.49:6822/38113","heartbeat_front_addr":"172.21.15.49:6818/38113","state":["exists","up"]},{"osd":2,"uuid":"555b3bf3-0a70-4b91-8efd-e4204b5fc26e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":38114},{"type":"v1","addr":"172.21.15.49:6801","nonce":38114}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":38114},{"type":"v1","addr":"172.21.15.49:6803","nonce":38114}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":38114},{"type":"v1","addr":"172.21.15.49:6807","nonce":38114}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":38114},{"type":"v1","addr":"172.21.15.49:6805","nonce":38114}]},"public_addr":"172.21.15.49:6801/38114","cluster_addr":"172.21.15.49:6803/38114","heartbeat_back_addr":"172.21.15.49:6807/38114","heartbeat_front_addr":"172.21.15.49:6805/38114","state":["exists","up"]},{"osd":3,"uuid":"8d4e5d77-03ef-4b9b-bfcb-5ae93a46795c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":38115},{"type":"v1","addr":"172.21.15.49:6825","nonce":38115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":38115},{"type":"v1","addr":"172.21.15.49:6827","nonce":38115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":38115},{"type":"v1","addr":"172.21.15.49:6831","nonce":38115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":38115},{"type":"v1","addr":"172.21.15.49:6829","nonce":38115}]},"public_addr":"172.21.15.49:6825/38115","cluster_addr":"172.21.15.49:6827/38115","heartbeat_back_addr":"172.21.15.49:6831/38115","heartbeat_front_addr":"172.21.15.49:6829/38115","state":["exists","up"]},{"osd":4,"uuid":"dba03ad4-68a3-4cf3-9218-ac611f02eacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":38417},{"type":"v1","addr":"172.21.15.150:6809","nonce":38417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":38417},{"type":"v1","addr":"172.21.15.150:6811","nonce":38417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":38417},{"type":"v1","addr":"172.21.15.150:6815","nonce":38417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":38417},{"type":"v1","addr":"172.21.15.150:6813","nonce":38417}]},"public_addr":"172.21.15.150:6809/38417","cluster_addr":"172.21.15.150:6811/38417","heartbeat_back_addr":"172.21.15.150:6815/38417","heartbeat_front_addr":"172.21.15.150:6813/38417","state":["exists","up"]},{"osd":5,"uuid":"0df9c303-c255-4640-8e91-01f1c3efe18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":38418},{"type":"v1","addr":"172.21.15.150:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":38418},{"type":"v1","addr":"172.21.15.150:6803","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":38418},{"type":"v1","addr":"172.21.15.150:6807","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":38418},{"type":"v1","addr":"172.21.15.150:6805","nonce":38418}]},"public_addr":"172.21.15.150:6801/38418","cluster_addr":"172.21.15.150:6803/38418","heartbeat_back_addr":"172.21.15.150:6807/38418","heartbeat_front_addr":"172.21.15.150:6805/38418","state":["exists","up"]},{"osd":6,"uuid":"e59fa189-4b70-4c06-9722-c02b90828bf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":38428},{"type":"v1","addr":"172.21.15.150:6825","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":38428},{"type":"v1","addr":"172.21.15.150:6827","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":38428},{"type":"v1","addr":"172.21.15.150:6831","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":38428},{"type":"v1","addr":"172.21.15.150:6829","nonce":38428}]},"public_addr":"172.21.15.150:6825/38428","cluster_addr":"172.21.15.150:6827/38428","heartbeat_back_addr":"172.21.15.150:6831/38428","heartbeat_front_addr":"172.21.15.150:6829/38428","state":["exists","up"]},{"osd":7,"uuid":"a64f57e1-ef98-46bd-b304-385759e95898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":38421},{"type":"v1","addr":"172.21.15.150:6817","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":38421},{"type":"v1","addr":"172.21.15.150:6819","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":38421},{"type":"v1","addr":"172.21.15.150:6823","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":38421},{"type":"v1","addr":"172.21.15.150:6821","nonce":38421}]},"public_addr":"172.21.15.150:6817/38421","cluster_addr":"172.21.15.150:6819/38421","heartbeat_back_addr":"172.21.15.150:6823/38421","heartbeat_front_addr":"172.21.15.150:6821/38421","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.438552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.413280+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.296466+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.424966+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.276953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.230984+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.410235+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.480084+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.3d","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T02:10:17.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.915+0000 7f87d068e640 1 -- 172.21.15.49:0/2680256592 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f87ac065a60 msgr2=0x7f87ac067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:17.917 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.915+0000 7f87d068e640 1 --2- 172.21.15.49:0/2680256592 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f87ac065a60 0x7f87ac067f20 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f87c40015f0 tx=0x7f87c40079d0 comp rx=0 tx=0).stop 2023-11-06T02:10:17.917 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.915+0000 7f87d068e640 1 -- 172.21.15.49:0/2680256592 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87c80cda70 msgr2=0x7f87c80cadf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:17.917 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.915+0000 7f87d068e640 1 --2- 172.21.15.49:0/2680256592 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87c80cda70 0x7f87c80cadf0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f87c00af040 tx=0x7f87c00197b0 comp rx=0 tx=0).stop 2023-11-06T02:10:17.917 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.915+0000 7f87d068e640 1 -- 172.21.15.49:0/2680256592 shutdown_connections 2023-11-06T02:10:17.918 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.915+0000 7f87d068e640 1 --2- 172.21.15.49:0/2680256592 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f87ac065a60 0x7f87ac067f20 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:17.918 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.915+0000 7f87d068e640 1 --2- 172.21.15.49:0/2680256592 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87c80cda70 0x7f87c80cadf0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:17.918 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.915+0000 7f87d068e640 1 --2- 172.21.15.49:0/2680256592 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f87c80b1f90 0x7f87c8154ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:17.918 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.915+0000 7f87d068e640 1 --2- 172.21.15.49:0/2680256592 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f87c8071bb0 0x7f87c8157790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:17.919 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.915+0000 7f87d068e640 1 -- 172.21.15.49:0/2680256592 >> 172.21.15.49:0/2680256592 conn(0x7f87c80d2aa0 msgr2=0x7f87c80d2e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:17.919 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.915+0000 7f87d068e640 1 -- 172.21.15.49:0/2680256592 shutdown_connections 2023-11-06T02:10:17.919 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:17.915+0000 7f87d068e640 1 -- 172.21.15.49:0/2680256592 wait complete. 2023-11-06T02:10:17.932 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T02:08:19.252222+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T02:08:24.776211+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T02:09:37.906848+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T02:09:38.849380+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T02:10:17.933 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T02:10:18.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.055+0000 7f45b0462640 1 Processor -- start 2023-11-06T02:10:18.057 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.055+0000 7f45b0462640 1 -- start start 2023-11-06T02:10:18.060 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.059+0000 7f45b0462640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f45a80ce280 0x7f45a80cd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:18.060 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.059+0000 7f45b0462640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45a815e7e0 con 0x7f45a80ae760 2023-11-06T02:10:18.060 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.059+0000 7f45b0462640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45a8099080 con 0x7f45a80cdea0 2023-11-06T02:10:18.060 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.059+0000 7f45b0462640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45a80b1100 con 0x7f45a80ce280 2023-11-06T02:10:18.061 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.059+0000 7f45ae9ff640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f45a80ae760 0x7f45a815e0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:42492/0 (socket says 172.21.15.49:42492) 2023-11-06T02:10:18.061 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.059+0000 7f45ae9ff640 1 -- 172.21.15.49:0/2080056064 learned_addr learned my addr 172.21.15.49:0/2080056064 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:10:18.061 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.059+0000 7f45ad9fd640 1 --2- 172.21.15.49:0/2080056064 >> v2:172.21.15.150:3300/0 conn(0x7f45a80ce280 0x7f45a80cd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:18.061 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.059+0000 7f45ad1fc640 1 -- 172.21.15.49:0/2080056064 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 948947386 0 0) 0x7f45a815e7e0 con 0x7f45a80ae760 2023-11-06T02:10:18.061 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.059+0000 7f45ad1fc640 1 -- 172.21.15.49:0/2080056064 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4590003530 con 0x7f45a80ae760 2023-11-06T02:10:18.062 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.059+0000 7f45ad1fc640 1 -- 172.21.15.49:0/2080056064 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2984697986 0 0) 0x7f4590003530 con 0x7f45a80ae760 2023-11-06T02:10:18.062 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.059+0000 7f45ad1fc640 1 -- 172.21.15.49:0/2080056064 >> v2:172.21.15.150:3300/0 conn(0x7f45a80ce280 msgr2=0x7f45a80cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:18.062 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.059+0000 7f45ad1fc640 1 --2- 172.21.15.49:0/2080056064 >> v2:172.21.15.150:3300/0 conn(0x7f45a80ce280 0x7f45a80cd430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:18.062 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.059+0000 7f45ad1fc640 1 -- 172.21.15.49:0/2080056064 >> v1:172.21.15.150:6789/0 conn(0x7f45a80cdea0 legacy=0x7f45a815d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:18.063 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.059+0000 7f45ad1fc640 1 -- 172.21.15.49:0/2080056064 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45a80b0b40 con 0x7f45a80ae760 2023-11-06T02:10:18.063 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.059+0000 7f45ad1fc640 1 -- 172.21.15.49:0/2080056064 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7f4598001680 con 0x7f45a80ae760 2023-11-06T02:10:18.063 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.059+0000 7f45ad1fc640 1 -- 172.21.15.49:0/2080056064 >> v1:172.21.15.49:6789/0 conn(0x7f45a80ae760 legacy=0x7f45a815e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:18.064 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.059+0000 7f45ad1fc640 1 --2- 172.21.15.49:0/2080056064 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4590003980 0x7f4590003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:18.064 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.059+0000 7f45ad1fc640 1 --2- 172.21.15.49:0/2080056064 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f45900047d0 0x7f4590004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:18.064 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.059+0000 7f45ad1fc640 1 --2- 172.21.15.49:0/2080056064 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4590005190 0x7f459000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:18.064 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.059+0000 7f45ad1fc640 1 -- 172.21.15.49:0/2080056064 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45a80b1100 con 0x7f4590005190 2023-11-06T02:10:18.065 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.059+0000 7f45ad1fc640 1 -- 172.21.15.49:0/2080056064 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f459000c030 con 0x7f4590003980 2023-11-06T02:10:18.065 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.059+0000 7f45ad1fc640 1 -- 172.21.15.49:0/2080056064 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f459000c1f0 con 0x7f45900047d0 2023-11-06T02:10:18.065 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.059+0000 7f45ae9ff640 1 --2- 172.21.15.49:0/2080056064 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4590005190 0x7f459000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:18.065 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.059+0000 7f45ae1fe640 1 --2- 172.21.15.49:0/2080056064 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4590003980 0x7f4590003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:18.065 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.059+0000 7f45ae9ff640 1 -- 172.21.15.49:0/2080056064 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f45900047d0 msgr2=0x7f4590004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:18.066 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.059+0000 7f45ad9fd640 1 --2- 172.21.15.49:0/2080056064 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f45900047d0 0x7f4590004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:18.066 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.059+0000 7f45ae9ff640 1 --2- 172.21.15.49:0/2080056064 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f45900047d0 0x7f4590004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:18.066 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.059+0000 7f45ae9ff640 1 -- 172.21.15.49:0/2080056064 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4590003980 msgr2=0x7f4590003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:18.066 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.059+0000 7f45ae9ff640 1 --2- 172.21.15.49:0/2080056064 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4590003980 0x7f4590003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:18.066 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.059+0000 7f45ae9ff640 1 -- 172.21.15.49:0/2080056064 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4590003530 con 0x7f4590005190 2023-11-06T02:10:18.067 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.059+0000 7f45ae1fe640 1 --2- 172.21.15.49:0/2080056064 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4590003980 0x7f4590003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:10:18.067 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.059+0000 7f45ad9fd640 1 --2- 172.21.15.49:0/2080056064 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f45900047d0 0x7f4590004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:10:18.070 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.059+0000 7f45ae9ff640 1 -- 172.21.15.49:0/2080056064 reap_dead start 2023-11-06T02:10:18.070 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.063+0000 7f45ae9ff640 1 --2- 172.21.15.49:0/2080056064 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4590005190 0x7f459000b990 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f459800c240 tx=0x7f459801cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9fb2d714ae009dd server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:18.070 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.063+0000 7f45ad1fc640 1 -- 172.21.15.49:0/2080056064 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45980a5b40 con 0x7f4590005190 2023-11-06T02:10:18.071 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.063+0000 7f45ad1fc640 1 -- 172.21.15.49:0/2080056064 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45980a5ce0 con 0x7f4590005190 2023-11-06T02:10:18.071 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.063+0000 7f45ad1fc640 1 -- 172.21.15.49:0/2080056064 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4598005b20 con 0x7f4590005190 2023-11-06T02:10:18.071 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.063+0000 7f45b0462640 1 -- 172.21.15.49:0/2080056064 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4590005190 msgr2=0x7f459000b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:18.071 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.063+0000 7f45b0462640 1 --2- 172.21.15.49:0/2080056064 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4590005190 0x7f459000b990 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f459800c240 tx=0x7f459801cc40 comp rx=0 tx=0).stop 2023-11-06T02:10:18.071 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.063+0000 7f45b0462640 1 -- 172.21.15.49:0/2080056064 shutdown_connections 2023-11-06T02:10:18.072 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.063+0000 7f45b0462640 1 --2- 172.21.15.49:0/2080056064 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4590005190 0x7f459000b990 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:18.072 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.063+0000 7f45b0462640 1 -- 172.21.15.49:0/2080056064 >> 172.21.15.49:0/2080056064 conn(0x7f45a8098a40 msgr2=0x7f45a80cf860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:18.072 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.063+0000 7f45b0462640 1 -- 172.21.15.49:0/2080056064 shutdown_connections 2023-11-06T02:10:18.072 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.063+0000 7f45b0462640 1 -- 172.21.15.49:0/2080056064 wait complete. 2023-11-06T02:10:18.072 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.063+0000 7f45b0462640 1 Processor -- start 2023-11-06T02:10:18.073 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.063+0000 7f45b0462640 1 -- start start 2023-11-06T02:10:18.073 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.063+0000 7f45b0462640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4590005190 0x7f45a806ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:18.073 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.063+0000 7f45b0462640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f45a80cb1b0 0x7f45a8152360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:18.073 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.063+0000 7f45b0462640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f45a814b1b0 0x7f45a814b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:18.073 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.063+0000 7f45b0462640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45a80b4020 con 0x7f4590005190 2023-11-06T02:10:18.074 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.063+0000 7f45b0462640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45a814bba0 con 0x7f45a814b1b0 2023-11-06T02:10:18.074 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.063+0000 7f45b0462640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45a80b4200 con 0x7f45a80cb1b0 2023-11-06T02:10:18.074 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.063+0000 7f45ae1fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4590005190 0x7f45a806ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:18.074 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.063+0000 7f45ae1fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4590005190 0x7f45a806ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33374/0 (socket says 172.21.15.49:33374) 2023-11-06T02:10:18.074 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.063+0000 7f45ae1fe640 1 -- 172.21.15.49:0/1956877322 learned_addr learned my addr 172.21.15.49:0/1956877322 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:18.075 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.063+0000 7f45ad9fd640 1 --2- 172.21.15.49:0/1956877322 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f45a80cb1b0 0x7f45a8152360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:18.075 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.063+0000 7f45ae9ff640 1 --2- 172.21.15.49:0/1956877322 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f45a814b1b0 0x7f45a814b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:18.075 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.063+0000 7f45ae1fe640 1 -- 172.21.15.49:0/1956877322 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f45a80cb1b0 msgr2=0x7f45a8152360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:18.075 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.063+0000 7f45ae1fe640 1 --2- 172.21.15.49:0/1956877322 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f45a80cb1b0 0x7f45a8152360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:18.075 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.063+0000 7f45ae1fe640 1 -- 172.21.15.49:0/1956877322 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f45a814b1b0 msgr2=0x7f45a814b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:18.076 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.063+0000 7f45ae1fe640 1 --2- 172.21.15.49:0/1956877322 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f45a814b1b0 0x7f45a814b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:18.076 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.063+0000 7f45ae1fe640 1 -- 172.21.15.49:0/1956877322 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45a80b44e0 con 0x7f4590005190 2023-11-06T02:10:18.076 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.063+0000 7f45ae1fe640 1 --2- 172.21.15.49:0/1956877322 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4590005190 0x7f45a806ec30 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f45a400e7e0 tx=0x7f45a400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:18.076 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.063+0000 7f459effd640 1 -- 172.21.15.49:0/1956877322 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a4003b90 con 0x7f4590005190 2023-11-06T02:10:18.076 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.063+0000 7f459effd640 1 -- 172.21.15.49:0/1956877322 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45a4003cf0 con 0x7f4590005190 2023-11-06T02:10:18.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.063+0000 7f45b0462640 1 -- 172.21.15.49:0/1956877322 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45a80b47c0 con 0x7f4590005190 2023-11-06T02:10:18.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.063+0000 7f459effd640 1 -- 172.21.15.49:0/1956877322 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a4005310 con 0x7f4590005190 2023-11-06T02:10:18.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.063+0000 7f45b0462640 1 -- 172.21.15.49:0/1956877322 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45a80b4d80 con 0x7f4590005190 2023-11-06T02:10:18.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.067+0000 7f45b0462640 1 -- 172.21.15.49:0/1956877322 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45a80b4f40 con 0x7f4590005190 2023-11-06T02:10:18.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.067+0000 7f459effd640 1 -- 172.21.15.49:0/1956877322 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f45a4005470 con 0x7f4590005190 2023-11-06T02:10:18.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.067+0000 7f459effd640 1 --2- 172.21.15.49:0/1956877322 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4594065a60 0x7f4594067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:18.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.067+0000 7f459effd640 1 -- 172.21.15.49:0/1956877322 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f45a4083320 con 0x7f4590005190 2023-11-06T02:10:18.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.067+0000 7f45ad9fd640 1 --2- 172.21.15.49:0/1956877322 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4594065a60 0x7f4594067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:18.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.067+0000 7f45ad9fd640 1 --2- 172.21.15.49:0/1956877322 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4594065a60 0x7f4594067f20 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f45a80cb6b0 tx=0x7f45a0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:18.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.071+0000 7f459effd640 1 -- 172.21.15.49:0/1956877322 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f45a4050b60 con 0x7f4590005190 2023-11-06T02:10:18.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.227+0000 7f45b0462640 1 -- 172.21.15.49:0/1956877322 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f45a8155cb0 con 0x7f4590005190 2023-11-06T02:10:18.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.227+0000 7f459effd640 1 -- 172.21.15.49:0/1956877322 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v39) v1 ==== 93+0+10 (secure 0 0 0) 0x7f45a4003e60 con 0x7f4590005190 2023-11-06T02:10:18.228 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2023-11-06T02:10:18.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.231+0000 7f45b0462640 1 -- 172.21.15.49:0/1956877322 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4594065a60 msgr2=0x7f4594067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:18.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.231+0000 7f45b0462640 1 --2- 172.21.15.49:0/1956877322 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4594065a60 0x7f4594067f20 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f45a80cb6b0 tx=0x7f45a0008040 comp rx=0 tx=0).stop 2023-11-06T02:10:18.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.231+0000 7f45b0462640 1 -- 172.21.15.49:0/1956877322 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4590005190 msgr2=0x7f45a806ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:18.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.231+0000 7f45b0462640 1 --2- 172.21.15.49:0/1956877322 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4590005190 0x7f45a806ec30 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f45a400e7e0 tx=0x7f45a400eca0 comp rx=0 tx=0).stop 2023-11-06T02:10:18.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.231+0000 7f45b0462640 1 -- 172.21.15.49:0/1956877322 shutdown_connections 2023-11-06T02:10:18.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.231+0000 7f45b0462640 1 --2- 172.21.15.49:0/1956877322 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4594065a60 0x7f4594067f20 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:18.232 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.231+0000 7f45b0462640 1 --2- 172.21.15.49:0/1956877322 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f45a814b1b0 0x7f45a814b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:18.232 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.231+0000 7f45b0462640 1 --2- 172.21.15.49:0/1956877322 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f45a80cb1b0 0x7f45a8152360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:18.232 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.231+0000 7f45b0462640 1 --2- 172.21.15.49:0/1956877322 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4590005190 0x7f45a806ec30 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:18.232 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.231+0000 7f45b0462640 1 -- 172.21.15.49:0/1956877322 >> 172.21.15.49:0/1956877322 conn(0x7f45a8098a40 msgr2=0x7f45a806daf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:18.232 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.231+0000 7f45b0462640 1 -- 172.21.15.49:0/1956877322 shutdown_connections 2023-11-06T02:10:18.232 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.231+0000 7f45b0462640 1 -- 172.21.15.49:0/1956877322 wait complete. 2023-11-06T02:10:18.247 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T02:10:18.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.375+0000 7f581b2db640 1 Processor -- start 2023-11-06T02:10:18.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.375+0000 7f581b2db640 1 -- start start 2023-11-06T02:10:18.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.375+0000 7f581b2db640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f581415cbb0 0x7f58140d1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:18.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.375+0000 7f581b2db640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f58140d24e0 0x7f58140d2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:18.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.375+0000 7f581b2db640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58140d3400 con 0x7f58140d2fa0 2023-11-06T02:10:18.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.375+0000 7f581b2db640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f581415cf90 con 0x7f58140d24e0 2023-11-06T02:10:18.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.375+0000 7f581b2db640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58140d0b80 con 0x7f581415cbb0 2023-11-06T02:10:18.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.375+0000 7f5818876640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f58140d24e0 0x7f58140d2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:18.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.375+0000 7f5818876640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f58140d24e0 0x7f58140d2a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33388/0 (socket says 172.21.15.49:33388) 2023-11-06T02:10:18.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.375+0000 7f5818876640 1 -- 172.21.15.49:0/3699042551 learned_addr learned my addr 172.21.15.49:0/3699042551 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:18.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.375+0000 7f5819077640 1 --2- 172.21.15.49:0/3699042551 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f581415cbb0 0x7f58140d1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:18.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.375+0000 7f5818876640 1 -- 172.21.15.49:0/3699042551 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f581415cbb0 msgr2=0x7f58140d1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:18.378 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.375+0000 7f5818876640 1 --2- 172.21.15.49:0/3699042551 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f581415cbb0 0x7f58140d1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:18.378 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.375+0000 7f5818876640 1 -- 172.21.15.49:0/3699042551 >> v1:172.21.15.150:6789/0 conn(0x7f58140d2fa0 legacy=0x7f58140b4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:18.378 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.375+0000 7f5818876640 1 -- 172.21.15.49:0/3699042551 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58140d0e60 con 0x7f58140d24e0 2023-11-06T02:10:18.379 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.379+0000 7f5818876640 1 --2- 172.21.15.49:0/3699042551 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f58140d24e0 0x7f58140d2a60 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f58140b55d0 tx=0x7f581001ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60531c40df7cd731 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:18.379 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.379+0000 7f5807fff640 1 -- 172.21.15.49:0/3699042551 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58100a8970 con 0x7f58140d24e0 2023-11-06T02:10:18.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.379+0000 7f5807fff640 1 -- 172.21.15.49:0/3699042551 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58100a8ad0 con 0x7f58140d24e0 2023-11-06T02:10:18.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.379+0000 7f5807fff640 1 -- 172.21.15.49:0/3699042551 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58100a8e60 con 0x7f58140d24e0 2023-11-06T02:10:18.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.379+0000 7f581b2db640 1 -- 172.21.15.49:0/3699042551 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f58140d24e0 msgr2=0x7f58140d2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:18.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.379+0000 7f581b2db640 1 --2- 172.21.15.49:0/3699042551 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f58140d24e0 0x7f58140d2a60 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f58140b55d0 tx=0x7f581001ecb0 comp rx=0 tx=0).stop 2023-11-06T02:10:18.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.379+0000 7f581b2db640 1 -- 172.21.15.49:0/3699042551 shutdown_connections 2023-11-06T02:10:18.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.379+0000 7f581b2db640 1 --2- 172.21.15.49:0/3699042551 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f58140d24e0 0x7f58140d2a60 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:18.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.379+0000 7f581b2db640 1 --2- 172.21.15.49:0/3699042551 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f581415cbb0 0x7f58140d1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:18.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.379+0000 7f581b2db640 1 -- 172.21.15.49:0/3699042551 >> 172.21.15.49:0/3699042551 conn(0x7f58140718a0 msgr2=0x7f5814071cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:18.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.379+0000 7f581b2db640 1 -- 172.21.15.49:0/3699042551 shutdown_connections 2023-11-06T02:10:18.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.379+0000 7f581b2db640 1 -- 172.21.15.49:0/3699042551 wait complete. 2023-11-06T02:10:18.384 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.379+0000 7f581b2db640 1 Processor -- start 2023-11-06T02:10:18.384 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.379+0000 7f581b2db640 1 -- start start 2023-11-06T02:10:18.384 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.379+0000 7f581b2db640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f58140d24e0 0x7f58141560c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:18.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.379+0000 7f581b2db640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58140d2fa0 0x7f58141596e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:18.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.379+0000 7f581b2db640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f581415cbb0 0x7f58140cbe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:18.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.379+0000 7f581b2db640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58140caa10 con 0x7f581415cbb0 2023-11-06T02:10:18.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.379+0000 7f581b2db640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58140aac50 con 0x7f58140d2fa0 2023-11-06T02:10:18.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.379+0000 7f581b2db640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5814156600 con 0x7f58140d24e0 2023-11-06T02:10:18.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.379+0000 7f5819878640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f581415cbb0 0x7f58140cbe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:18.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.383+0000 7f5819878640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f581415cbb0 0x7f58140cbe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33400/0 (socket says 172.21.15.49:33400) 2023-11-06T02:10:18.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.383+0000 7f5819878640 1 -- 172.21.15.49:0/2255463400 learned_addr learned my addr 172.21.15.49:0/2255463400 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:18.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.383+0000 7f5819077640 1 --2- 172.21.15.49:0/2255463400 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f58140d24e0 0x7f58141560c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:18.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.383+0000 7f5818876640 1 --2- 172.21.15.49:0/2255463400 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58140d2fa0 0x7f58141596e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:18.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.383+0000 7f5819878640 1 -- 172.21.15.49:0/2255463400 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f58140d24e0 msgr2=0x7f58141560c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:18.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.383+0000 7f5819878640 1 --2- 172.21.15.49:0/2255463400 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f58140d24e0 0x7f58141560c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:18.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.383+0000 7f5819878640 1 -- 172.21.15.49:0/2255463400 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58140d2fa0 msgr2=0x7f58141596e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:18.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.383+0000 7f5819878640 1 --2- 172.21.15.49:0/2255463400 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58140d2fa0 0x7f58141596e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:18.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.383+0000 7f5819878640 1 -- 172.21.15.49:0/2255463400 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58100094d0 con 0x7f581415cbb0 2023-11-06T02:10:18.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.383+0000 7f5819878640 1 --2- 172.21.15.49:0/2255463400 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f581415cbb0 0x7f58140cbe20 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f580c00c780 tx=0x7f580c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:18.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.383+0000 7f5805ffb640 1 -- 172.21.15.49:0/2255463400 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f580c004b70 con 0x7f581415cbb0 2023-11-06T02:10:18.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.383+0000 7f5805ffb640 1 -- 172.21.15.49:0/2255463400 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f580c00ce90 con 0x7f581415cbb0 2023-11-06T02:10:18.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.383+0000 7f581b2db640 1 -- 172.21.15.49:0/2255463400 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5814159430 con 0x7f581415cbb0 2023-11-06T02:10:18.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.383+0000 7f5805ffb640 1 -- 172.21.15.49:0/2255463400 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f580c00a850 con 0x7f581415cbb0 2023-11-06T02:10:18.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.383+0000 7f581b2db640 1 -- 172.21.15.49:0/2255463400 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58140aa190 con 0x7f581415cbb0 2023-11-06T02:10:18.389 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.383+0000 7f581b2db640 1 -- 172.21.15.49:0/2255463400 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5814074040 con 0x7f581415cbb0 2023-11-06T02:10:18.389 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.383+0000 7f5805ffb640 1 -- 172.21.15.49:0/2255463400 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f580c00a9b0 con 0x7f581415cbb0 2023-11-06T02:10:18.389 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.383+0000 7f5805ffb640 1 --2- 172.21.15.49:0/2255463400 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f57f8065a60 0x7f57f8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:18.389 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.383+0000 7f5805ffb640 1 -- 172.21.15.49:0/2255463400 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f580c083400 con 0x7f581415cbb0 2023-11-06T02:10:18.389 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.383+0000 7f5819077640 1 --2- 172.21.15.49:0/2255463400 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f57f8065a60 0x7f57f8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:18.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.387+0000 7f5819077640 1 --2- 172.21.15.49:0/2255463400 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f57f8065a60 0x7f57f8067f20 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f58080079c0 tx=0x7f5808008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:18.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.387+0000 7f5805ffb640 1 -- 172.21.15.49:0/2255463400 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f580c050cd0 con 0x7f581415cbb0 2023-11-06T02:10:18.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.543+0000 7f581b2db640 1 -- 172.21.15.49:0/2255463400 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f58140d2c70 con 0x7f581415cbb0 2023-11-06T02:10:18.545 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.547+0000 7f5805ffb640 1 -- 172.21.15.49:0/2255463400 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v39) v1 ==== 92+0+10 (secure 0 0 0) 0x7f580c0542f0 con 0x7f581415cbb0 2023-11-06T02:10:18.545 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 8 2023-11-06T02:10:18.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.547+0000 7f581b2db640 1 -- 172.21.15.49:0/2255463400 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f57f8065a60 msgr2=0x7f57f8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:18.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.547+0000 7f581b2db640 1 --2- 172.21.15.49:0/2255463400 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f57f8065a60 0x7f57f8067f20 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f58080079c0 tx=0x7f5808008040 comp rx=0 tx=0).stop 2023-11-06T02:10:18.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.547+0000 7f581b2db640 1 -- 172.21.15.49:0/2255463400 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f581415cbb0 msgr2=0x7f58140cbe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:18.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.547+0000 7f581b2db640 1 --2- 172.21.15.49:0/2255463400 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f581415cbb0 0x7f58140cbe20 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f580c00c780 tx=0x7f580c00cc40 comp rx=0 tx=0).stop 2023-11-06T02:10:18.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.547+0000 7f581b2db640 1 -- 172.21.15.49:0/2255463400 shutdown_connections 2023-11-06T02:10:18.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.547+0000 7f581b2db640 1 --2- 172.21.15.49:0/2255463400 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f57f8065a60 0x7f57f8067f20 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:18.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.547+0000 7f581b2db640 1 --2- 172.21.15.49:0/2255463400 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f581415cbb0 0x7f58140cbe20 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:18.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.547+0000 7f581b2db640 1 --2- 172.21.15.49:0/2255463400 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58140d2fa0 0x7f58141596e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:18.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.547+0000 7f581b2db640 1 --2- 172.21.15.49:0/2255463400 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f58140d24e0 0x7f58141560c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:18.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.547+0000 7f581b2db640 1 -- 172.21.15.49:0/2255463400 >> 172.21.15.49:0/2255463400 conn(0x7f58140718a0 msgr2=0x7f581415c220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:18.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.547+0000 7f581b2db640 1 -- 172.21.15.49:0/2255463400 shutdown_connections 2023-11-06T02:10:18.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.547+0000 7f581b2db640 1 -- 172.21.15.49:0/2255463400 wait complete. 2023-11-06T02:10:18.564 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T02:10:18.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.695+0000 7f25d300c640 1 Processor -- start 2023-11-06T02:10:18.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.695+0000 7f25d300c640 1 -- start start 2023-11-06T02:10:18.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.695+0000 7f25d300c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f25cc14ffc0 0x7f25cc0ced40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:18.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.695+0000 7f25d300c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f25cc0ada30 0x7f25cc0ade70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:18.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.695+0000 7f25d300c640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25cc0c8c80 con 0x7f25cc070160 2023-11-06T02:10:18.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.695+0000 7f25d300c640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25cc0ae4d0 con 0x7f25cc14ffc0 2023-11-06T02:10:18.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.695+0000 7f25d300c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25cc1594f0 con 0x7f25cc0ada30 2023-11-06T02:10:18.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.695+0000 7f25d0da8640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f25cc14ffc0 0x7f25cc0ced40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:18.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.695+0000 7f25d0da8640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f25cc14ffc0 0x7f25cc0ced40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46112/0 (socket says 172.21.15.49:46112) 2023-11-06T02:10:18.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.695+0000 7f25d15a9640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f25cc070160 0x7f25cc15cae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:59700/0 (socket says 172.21.15.49:59700) 2023-11-06T02:10:18.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.695+0000 7f25d0da8640 1 -- 172.21.15.49:0/478355189 learned_addr learned my addr 172.21.15.49:0/478355189 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:18.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.695+0000 7f25cbfff640 1 --2- 172.21.15.49:0/478355189 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f25cc0ada30 0x7f25cc0ade70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:18.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.695+0000 7f25d0da8640 1 -- 172.21.15.49:0/478355189 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f25cc0ada30 msgr2=0x7f25cc0ade70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:18.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.695+0000 7f25d0da8640 1 --2- 172.21.15.49:0/478355189 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f25cc0ada30 0x7f25cc0ade70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:18.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.695+0000 7f25d0da8640 1 -- 172.21.15.49:0/478355189 >> v1:172.21.15.49:6789/0 conn(0x7f25cc070160 legacy=0x7f25cc15cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:18.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.695+0000 7f25cb7fe640 1 -- 172.21.15.49:0/478355189 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 708625839 0 0) 0x7f25cc0c8c80 con 0x7f25cc070160 2023-11-06T02:10:18.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.695+0000 7f25d0da8640 1 -- 172.21.15.49:0/478355189 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25cc159700 con 0x7f25cc14ffc0 2023-11-06T02:10:18.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.699+0000 7f25d0da8640 1 --2- 172.21.15.49:0/478355189 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25cc14ffc0 0x7f25cc0ced40 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f25c0009870 tx=0x7f25c001c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26c35994ab4f8d7c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:18.701 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.699+0000 7f25cb7fe640 1 -- 172.21.15.49:0/478355189 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c00ab600 con 0x7f25cc14ffc0 2023-11-06T02:10:18.701 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.699+0000 7f25cb7fe640 1 -- 172.21.15.49:0/478355189 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25c00a7050 con 0x7f25cc14ffc0 2023-11-06T02:10:18.701 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.699+0000 7f25cb7fe640 1 -- 172.21.15.49:0/478355189 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c00ab980 con 0x7f25cc14ffc0 2023-11-06T02:10:18.701 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.699+0000 7f25d300c640 1 -- 172.21.15.49:0/478355189 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25cc14ffc0 msgr2=0x7f25cc0ced40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:18.702 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.699+0000 7f25d300c640 1 --2- 172.21.15.49:0/478355189 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25cc14ffc0 0x7f25cc0ced40 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f25c0009870 tx=0x7f25c001c540 comp rx=0 tx=0).stop 2023-11-06T02:10:18.702 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.699+0000 7f25d300c640 1 -- 172.21.15.49:0/478355189 shutdown_connections 2023-11-06T02:10:18.702 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.699+0000 7f25d300c640 1 --2- 172.21.15.49:0/478355189 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f25cc0ada30 0x7f25cc0ade70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:18.702 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.699+0000 7f25d300c640 1 --2- 172.21.15.49:0/478355189 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25cc14ffc0 0x7f25cc0ced40 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:18.702 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.699+0000 7f25d300c640 1 -- 172.21.15.49:0/478355189 >> 172.21.15.49:0/478355189 conn(0x7f25cc06a370 msgr2=0x7f25cc0ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:18.703 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.699+0000 7f25d300c640 1 -- 172.21.15.49:0/478355189 shutdown_connections 2023-11-06T02:10:18.703 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.699+0000 7f25d300c640 1 -- 172.21.15.49:0/478355189 wait complete. 2023-11-06T02:10:18.703 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.699+0000 7f25d300c640 1 Processor -- start 2023-11-06T02:10:18.703 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.703+0000 7f25d300c640 1 -- start start 2023-11-06T02:10:18.705 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.703+0000 7f25d300c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f25cc070160 0x7f25cc0b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:18.705 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.703+0000 7f25d300c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25cc0ada30 0x7f25cc06c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:18.706 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.703+0000 7f25d300c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25cc14ffc0 0x7f25cc142050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:18.706 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.703+0000 7f25d300c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25cc0b3b40 con 0x7f25cc0ada30 2023-11-06T02:10:18.706 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.703+0000 7f25d300c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25cc133900 con 0x7f25cc14ffc0 2023-11-06T02:10:18.706 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.703+0000 7f25d300c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25cc0b2f50 con 0x7f25cc070160 2023-11-06T02:10:18.706 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.703+0000 7f25cbfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25cc0ada30 0x7f25cc06c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:18.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.703+0000 7f25cbfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25cc0ada30 0x7f25cc06c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46122/0 (socket says 172.21.15.49:46122) 2023-11-06T02:10:18.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.703+0000 7f25cbfff640 1 -- 172.21.15.49:0/422625585 learned_addr learned my addr 172.21.15.49:0/422625585 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:18.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.703+0000 7f25d0da8640 1 --2- 172.21.15.49:0/422625585 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f25cc070160 0x7f25cc0b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:18.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.703+0000 7f25d15a9640 1 --2- 172.21.15.49:0/422625585 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25cc14ffc0 0x7f25cc142050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:18.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.703+0000 7f25cbfff640 1 -- 172.21.15.49:0/422625585 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f25cc070160 msgr2=0x7f25cc0b3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:18.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.703+0000 7f25cbfff640 1 --2- 172.21.15.49:0/422625585 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f25cc070160 0x7f25cc0b3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:18.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.703+0000 7f25cbfff640 1 -- 172.21.15.49:0/422625585 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25cc14ffc0 msgr2=0x7f25cc142050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:18.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.703+0000 7f25cbfff640 1 --2- 172.21.15.49:0/422625585 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25cc14ffc0 0x7f25cc142050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:18.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.703+0000 7f25cbfff640 1 -- 172.21.15.49:0/422625585 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25c00094d0 con 0x7f25cc0ada30 2023-11-06T02:10:18.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.703+0000 7f25d0da8640 1 --2- 172.21.15.49:0/422625585 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f25cc070160 0x7f25cc0b3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:10:18.709 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.703+0000 7f25cbfff640 1 --2- 172.21.15.49:0/422625585 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25cc0ada30 0x7f25cc06c490 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f25bc00d5f0 tx=0x7f25bc00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:18.709 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.703+0000 7f25c97fa640 1 -- 172.21.15.49:0/422625585 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25bc0125f0 con 0x7f25cc0ada30 2023-11-06T02:10:18.709 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.703+0000 7f25c97fa640 1 -- 172.21.15.49:0/422625585 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25bc012750 con 0x7f25cc0ada30 2023-11-06T02:10:18.709 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.703+0000 7f25c97fa640 1 -- 172.21.15.49:0/422625585 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25bc012ae0 con 0x7f25cc0ada30 2023-11-06T02:10:18.709 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.703+0000 7f25d300c640 1 -- 172.21.15.49:0/422625585 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25cc0b3210 con 0x7f25cc0ada30 2023-11-06T02:10:18.710 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.703+0000 7f25d300c640 1 -- 172.21.15.49:0/422625585 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25cc0b37d0 con 0x7f25cc0ada30 2023-11-06T02:10:18.710 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.703+0000 7f25d300c640 1 -- 172.21.15.49:0/422625585 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25cc0b3cf0 con 0x7f25cc0ada30 2023-11-06T02:10:18.710 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.703+0000 7f25c97fa640 1 -- 172.21.15.49:0/422625585 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f25bc012c40 con 0x7f25cc0ada30 2023-11-06T02:10:18.710 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.703+0000 7f25c97fa640 1 --2- 172.21.15.49:0/422625585 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f25b8065a60 0x7f25b8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:18.711 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.703+0000 7f25c97fa640 1 -- 172.21.15.49:0/422625585 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f25bc083740 con 0x7f25cc0ada30 2023-11-06T02:10:18.711 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.707+0000 7f25d0da8640 1 --2- 172.21.15.49:0/422625585 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f25b8065a60 0x7f25b8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:18.711 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.707+0000 7f25d0da8640 1 --2- 172.21.15.49:0/422625585 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f25b8065a60 0x7f25b8067f20 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f25cc06e480 tx=0x7f25c00a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:18.712 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.711+0000 7f25c97fa640 1 -- 172.21.15.49:0/422625585 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f25cc0b3cf0 con 0x7f25cc0ada30 2023-11-06T02:10:18.862 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.863+0000 7f25d300c640 1 -- 172.21.15.49:0/422625585 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f25cc0b3960 con 0x7f25cc0ada30 2023-11-06T02:10:18.863 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.863+0000 7f25c97fa640 1 -- 172.21.15.49:0/422625585 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v39) v1 ==== 104+0+11 (secure 0 0 0) 0x7f25bc050ff0 con 0x7f25cc0ada30 2023-11-06T02:10:18.863 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2023-11-06T02:10:18.865 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.867+0000 7f25d300c640 1 -- 172.21.15.49:0/422625585 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f25b8065a60 msgr2=0x7f25b8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:18.866 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.867+0000 7f25d300c640 1 --2- 172.21.15.49:0/422625585 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f25b8065a60 0x7f25b8067f20 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f25cc06e480 tx=0x7f25c00a6000 comp rx=0 tx=0).stop 2023-11-06T02:10:18.866 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.867+0000 7f25d300c640 1 -- 172.21.15.49:0/422625585 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25cc0ada30 msgr2=0x7f25cc06c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:18.866 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.867+0000 7f25d300c640 1 --2- 172.21.15.49:0/422625585 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25cc0ada30 0x7f25cc06c490 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f25bc00d5f0 tx=0x7f25bc00dab0 comp rx=0 tx=0).stop 2023-11-06T02:10:18.866 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.867+0000 7f25d300c640 1 -- 172.21.15.49:0/422625585 shutdown_connections 2023-11-06T02:10:18.866 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.867+0000 7f25d300c640 1 --2- 172.21.15.49:0/422625585 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f25b8065a60 0x7f25b8067f20 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:18.870 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.867+0000 7f25d300c640 1 --2- 172.21.15.49:0/422625585 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25cc0ada30 0x7f25cc06c490 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:18.870 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.867+0000 7f25d300c640 1 --2- 172.21.15.49:0/422625585 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25cc14ffc0 0x7f25cc142050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:18.870 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.867+0000 7f25d300c640 1 --2- 172.21.15.49:0/422625585 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f25cc070160 0x7f25cc0b3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:18.871 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.867+0000 7f25d300c640 1 -- 172.21.15.49:0/422625585 >> 172.21.15.49:0/422625585 conn(0x7f25cc06a370 msgr2=0x7f25cc0af960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:18.871 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.867+0000 7f25d300c640 1 -- 172.21.15.49:0/422625585 shutdown_connections 2023-11-06T02:10:18.871 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:18.867+0000 7f25d300c640 1 -- 172.21.15.49:0/422625585 wait complete. 2023-11-06T02:10:18.883 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T02:10:19.010 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.011+0000 7f53cc7cb640 1 Processor -- start 2023-11-06T02:10:19.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.011+0000 7f53cc7cb640 1 -- start start 2023-11-06T02:10:19.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.011+0000 7f53cc7cb640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f53c40b0930 0x7f53c415dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:19.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.011+0000 7f53cc7cb640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53c406c5a0 con 0x7f53c415a990 2023-11-06T02:10:19.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.011+0000 7f53cc7cb640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53c406c880 con 0x7f53c41595a0 2023-11-06T02:10:19.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.011+0000 7f53cc7cb640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53c415e250 con 0x7f53c40b0930 2023-11-06T02:10:19.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.011+0000 7f53c9d66640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f53c415a990 0x7f53c40b0220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:59716/0 (socket says 172.21.15.49:59716) 2023-11-06T02:10:19.013 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.011+0000 7f53c9d66640 1 -- 172.21.15.49:0/1170729579 learned_addr learned my addr 172.21.15.49:0/1170729579 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:10:19.013 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.011+0000 7f53cad68640 1 --2- 172.21.15.49:0/1170729579 >> v2:172.21.15.49:3300/0 conn(0x7f53c40b0930 0x7f53c415dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:19.014 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.011+0000 7f53c9565640 1 -- 172.21.15.49:0/1170729579 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2401838909 0 0) 0x7f53c406c5a0 con 0x7f53c415a990 2023-11-06T02:10:19.014 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.011+0000 7f53c9565640 1 -- 172.21.15.49:0/1170729579 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53b0003530 con 0x7f53c415a990 2023-11-06T02:10:19.014 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.011+0000 7f53cad68640 1 -- 172.21.15.49:0/1170729579 >> v1:172.21.15.150:6789/0 conn(0x7f53c41595a0 legacy=0x7f53c406d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:19.014 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.011+0000 7f53cad68640 1 -- 172.21.15.49:0/1170729579 >> v1:172.21.15.49:6789/0 conn(0x7f53c415a990 legacy=0x7f53c40b0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:19.014 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.011+0000 7f53cad68640 1 -- 172.21.15.49:0/1170729579 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53c406cb60 con 0x7f53c40b0930 2023-11-06T02:10:19.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.015+0000 7f53cad68640 1 --2- 172.21.15.49:0/1170729579 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53c40b0930 0x7f53c415dd10 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f53bc0097c0 tx=0x7f53bc01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b569d26d174761c8 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:19.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.015+0000 7f53c9565640 1 -- 172.21.15.49:0/1170729579 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53bc0ab650 con 0x7f53c40b0930 2023-11-06T02:10:19.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.015+0000 7f53c9565640 1 -- 172.21.15.49:0/1170729579 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53bc0ab7b0 con 0x7f53c40b0930 2023-11-06T02:10:19.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.015+0000 7f53c9565640 1 -- 172.21.15.49:0/1170729579 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53bc0abb40 con 0x7f53c40b0930 2023-11-06T02:10:19.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.015+0000 7f53cc7cb640 1 -- 172.21.15.49:0/1170729579 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53c40b0930 msgr2=0x7f53c415dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:19.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.015+0000 7f53cc7cb640 1 --2- 172.21.15.49:0/1170729579 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53c40b0930 0x7f53c415dd10 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f53bc0097c0 tx=0x7f53bc01c450 comp rx=0 tx=0).stop 2023-11-06T02:10:19.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.015+0000 7f53cc7cb640 1 -- 172.21.15.49:0/1170729579 shutdown_connections 2023-11-06T02:10:19.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.015+0000 7f53cc7cb640 1 --2- 172.21.15.49:0/1170729579 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53c40b0930 0x7f53c415dd10 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:19.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.015+0000 7f53cc7cb640 1 -- 172.21.15.49:0/1170729579 >> 172.21.15.49:0/1170729579 conn(0x7f53c406a370 msgr2=0x7f53c40ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:19.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.015+0000 7f53cc7cb640 1 -- 172.21.15.49:0/1170729579 shutdown_connections 2023-11-06T02:10:19.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.015+0000 7f53cc7cb640 1 -- 172.21.15.49:0/1170729579 wait complete. 2023-11-06T02:10:19.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.015+0000 7f53cc7cb640 1 Processor -- start 2023-11-06T02:10:19.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.015+0000 7f53cc7cb640 1 -- start start 2023-11-06T02:10:19.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.019+0000 7f53cc7cb640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f53c40b0930 0x7f53c40cd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:19.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.019+0000 7f53cc7cb640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f53c41595a0 0x7f53c40cba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:19.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.019+0000 7f53cc7cb640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53c415a990 0x7f53c41451b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:19.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.019+0000 7f53cc7cb640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c4146240 con 0x7f53c415a990 2023-11-06T02:10:19.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.019+0000 7f53cc7cb640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c4147320 con 0x7f53c41595a0 2023-11-06T02:10:19.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.019+0000 7f53cc7cb640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53c40cddb0 con 0x7f53c40b0930 2023-11-06T02:10:19.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.019+0000 7f53cad68640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53c415a990 0x7f53c41451b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:19.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.019+0000 7f53cad68640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53c415a990 0x7f53c41451b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46142/0 (socket says 172.21.15.49:46142) 2023-11-06T02:10:19.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.019+0000 7f53cad68640 1 -- 172.21.15.49:0/879331803 learned_addr learned my addr 172.21.15.49:0/879331803 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:19.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.019+0000 7f53ca567640 1 --2- 172.21.15.49:0/879331803 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f53c40b0930 0x7f53c40cd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:19.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.019+0000 7f53c9d66640 1 --2- 172.21.15.49:0/879331803 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f53c41595a0 0x7f53c40cba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:19.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.019+0000 7f53cad68640 1 -- 172.21.15.49:0/879331803 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f53c40b0930 msgr2=0x7f53c40cd4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:19.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.019+0000 7f53cad68640 1 --2- 172.21.15.49:0/879331803 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f53c40b0930 0x7f53c40cd4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:19.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.019+0000 7f53cad68640 1 -- 172.21.15.49:0/879331803 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f53c41595a0 msgr2=0x7f53c40cba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:19.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.019+0000 7f53cad68640 1 --2- 172.21.15.49:0/879331803 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f53c41595a0 0x7f53c40cba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:19.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.019+0000 7f53cad68640 1 -- 172.21.15.49:0/879331803 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53bc009420 con 0x7f53c415a990 2023-11-06T02:10:19.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.019+0000 7f53c9d66640 1 --2- 172.21.15.49:0/879331803 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f53c41595a0 0x7f53c40cba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:10:19.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.019+0000 7f53cad68640 1 --2- 172.21.15.49:0/879331803 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53c415a990 0x7f53c41451b0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f53bc0ad040 tx=0x7f53bc01ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:19.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.019+0000 7f53baffd640 1 -- 172.21.15.49:0/879331803 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53bc0bf610 con 0x7f53c415a990 2023-11-06T02:10:19.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.019+0000 7f53baffd640 1 -- 172.21.15.49:0/879331803 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53bc0bf770 con 0x7f53c415a990 2023-11-06T02:10:19.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.019+0000 7f53cc7cb640 1 -- 172.21.15.49:0/879331803 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53c4145bc0 con 0x7f53c415a990 2023-11-06T02:10:19.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.019+0000 7f53baffd640 1 -- 172.21.15.49:0/879331803 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53bc0bfb20 con 0x7f53c415a990 2023-11-06T02:10:19.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.019+0000 7f53cc7cb640 1 -- 172.21.15.49:0/879331803 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53c4145e60 con 0x7f53c415a990 2023-11-06T02:10:19.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.019+0000 7f53cc7cb640 1 -- 172.21.15.49:0/879331803 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53c4146380 con 0x7f53c415a990 2023-11-06T02:10:19.026 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.019+0000 7f53baffd640 1 -- 172.21.15.49:0/879331803 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f53bc0bfc80 con 0x7f53c415a990 2023-11-06T02:10:19.026 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.019+0000 7f53baffd640 1 --2- 172.21.15.49:0/879331803 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f53a8065a60 0x7f53a8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:19.026 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.019+0000 7f53baffd640 1 -- 172.21.15.49:0/879331803 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f53bc11e170 con 0x7f53c415a990 2023-11-06T02:10:19.026 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.023+0000 7f53ca567640 1 --2- 172.21.15.49:0/879331803 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f53a8065a60 0x7f53a8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:19.027 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.023+0000 7f53ca567640 1 --2- 172.21.15.49:0/879331803 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f53a8065a60 0x7f53a8067f20 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f53c00012f0 tx=0x7f53c0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:19.027 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.027+0000 7f53baffd640 1 -- 172.21.15.49:0/879331803 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f53bc0ebb60 con 0x7f53c415a990 2023-11-06T02:10:19.179 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.179+0000 7f53cc7cb640 1 -- 172.21.15.49:0/879331803 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f53c40aee90 con 0x7f53c415a990 2023-11-06T02:10:19.180 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.179+0000 7f53baffd640 1 -- 172.21.15.49:0/879331803 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v39) v1 ==== 100+0+11 (secure 0 0 0) 0x7f53bc0ef180 con 0x7f53c415a990 2023-11-06T02:10:19.180 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2023-11-06T02:10:19.182 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.183+0000 7f53cc7cb640 1 -- 172.21.15.49:0/879331803 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f53a8065a60 msgr2=0x7f53a8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:19.183 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.183+0000 7f53cc7cb640 1 --2- 172.21.15.49:0/879331803 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f53a8065a60 0x7f53a8067f20 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f53c00012f0 tx=0x7f53c0009040 comp rx=0 tx=0).stop 2023-11-06T02:10:19.183 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.183+0000 7f53cc7cb640 1 -- 172.21.15.49:0/879331803 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53c415a990 msgr2=0x7f53c41451b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:19.183 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.183+0000 7f53cc7cb640 1 --2- 172.21.15.49:0/879331803 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53c415a990 0x7f53c41451b0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f53bc0ad040 tx=0x7f53bc01ac90 comp rx=0 tx=0).stop 2023-11-06T02:10:19.183 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.183+0000 7f53cc7cb640 1 -- 172.21.15.49:0/879331803 shutdown_connections 2023-11-06T02:10:19.183 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.183+0000 7f53cc7cb640 1 --2- 172.21.15.49:0/879331803 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f53a8065a60 0x7f53a8067f20 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:19.184 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.183+0000 7f53cc7cb640 1 --2- 172.21.15.49:0/879331803 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53c415a990 0x7f53c41451b0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:19.184 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.183+0000 7f53cc7cb640 1 --2- 172.21.15.49:0/879331803 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f53c41595a0 0x7f53c40cba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:19.184 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.183+0000 7f53cc7cb640 1 --2- 172.21.15.49:0/879331803 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f53c40b0930 0x7f53c40cd4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:19.184 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.183+0000 7f53cc7cb640 1 -- 172.21.15.49:0/879331803 >> 172.21.15.49:0/879331803 conn(0x7f53c406a370 msgr2=0x7f53c40b0220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:19.184 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.183+0000 7f53cc7cb640 1 -- 172.21.15.49:0/879331803 shutdown_connections 2023-11-06T02:10:19.184 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.183+0000 7f53cc7cb640 1 -- 172.21.15.49:0/879331803 wait complete. 2023-11-06T02:10:19.199 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T02:10:19.317 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.319+0000 7f9d4e495640 1 Processor -- start 2023-11-06T02:10:19.318 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.319+0000 7f9d4e495640 1 -- start start 2023-11-06T02:10:19.319 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.319+0000 7f9d4e495640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9d48159980 0x7f9d48071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:19.319 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.319+0000 7f9d4e495640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f9d480b1450 0x7f9d480cdd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:19.319 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.319+0000 7f9d4e495640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f9d480cb010 0x7f9d480cb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:19.320 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.319+0000 7f9d4e495640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d48072410 con 0x7f9d480cb010 2023-11-06T02:10:19.320 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.319+0000 7f9d4e495640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d480b1890 con 0x7f9d480b1450 2023-11-06T02:10:19.320 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.319+0000 7f9d4e495640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d480cb9e0 con 0x7f9d48159980 2023-11-06T02:10:19.320 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.319+0000 7f9d4ca32640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f9d480cb010 0x7f9d480cb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:19.320 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.319+0000 7f9d4ca32640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f9d480cb010 0x7f9d480cb4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46152/0 (socket says 172.21.15.49:46152) 2023-11-06T02:10:19.321 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.319+0000 7f9d4ca32640 1 -- 172.21.15.49:0/1543625668 learned_addr learned my addr 172.21.15.49:0/1543625668 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:19.321 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.319+0000 7f9d47fff640 1 --2- 172.21.15.49:0/1543625668 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9d48159980 0x7f9d48071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:19.321 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.319+0000 7f9d3ffff640 1 --2- 172.21.15.49:0/1543625668 >> v2:172.21.15.150:3300/0 conn(0x7f9d480b1450 0x7f9d480cdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:19.321 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.319+0000 7f9d4ca32640 1 -- 172.21.15.49:0/1543625668 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9d48159980 msgr2=0x7f9d48071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:19.321 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.319+0000 7f9d4ca32640 1 --2- 172.21.15.49:0/1543625668 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9d48159980 0x7f9d48071290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:19.322 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.319+0000 7f9d4ca32640 1 -- 172.21.15.49:0/1543625668 >> v2:172.21.15.150:3300/0 conn(0x7f9d480b1450 msgr2=0x7f9d480cdd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:19.322 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.319+0000 7f9d4ca32640 1 --2- 172.21.15.49:0/1543625668 >> v2:172.21.15.150:3300/0 conn(0x7f9d480b1450 0x7f9d480cdd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:19.322 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.319+0000 7f9d4ca32640 1 -- 172.21.15.49:0/1543625668 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d48071890 con 0x7f9d480cb010 2023-11-06T02:10:19.325 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.323+0000 7f9d4ca32640 1 --2- 172.21.15.49:0/1543625668 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d480cb010 0x7f9d480cb4a0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f9d4806dda0 tx=0x7f9d4001c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a69b7affb8ee817 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:19.325 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.323+0000 7f9d477fe640 1 -- 172.21.15.49:0/1543625668 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d400a5850 con 0x7f9d480cb010 2023-11-06T02:10:19.325 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.323+0000 7f9d477fe640 1 -- 172.21.15.49:0/1543625668 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d400a59b0 con 0x7f9d480cb010 2023-11-06T02:10:19.325 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.323+0000 7f9d477fe640 1 -- 172.21.15.49:0/1543625668 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d400a5d40 con 0x7f9d480cb010 2023-11-06T02:10:19.326 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.323+0000 7f9d4e495640 1 -- 172.21.15.49:0/1543625668 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d480cb010 msgr2=0x7f9d480cb4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:19.326 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.323+0000 7f9d4e495640 1 --2- 172.21.15.49:0/1543625668 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d480cb010 0x7f9d480cb4a0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f9d4806dda0 tx=0x7f9d4001c9d0 comp rx=0 tx=0).stop 2023-11-06T02:10:19.326 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.323+0000 7f9d4e495640 1 -- 172.21.15.49:0/1543625668 shutdown_connections 2023-11-06T02:10:19.326 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.323+0000 7f9d4e495640 1 --2- 172.21.15.49:0/1543625668 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d480cb010 0x7f9d480cb4a0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:19.326 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.323+0000 7f9d4e495640 1 --2- 172.21.15.49:0/1543625668 >> v2:172.21.15.150:3300/0 conn(0x7f9d480b1450 0x7f9d480cdd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:19.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.323+0000 7f9d4e495640 1 --2- 172.21.15.49:0/1543625668 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9d48159980 0x7f9d48071290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:19.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.323+0000 7f9d4e495640 1 -- 172.21.15.49:0/1543625668 >> 172.21.15.49:0/1543625668 conn(0x7f9d4806c0a0 msgr2=0x7f9d4806c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:19.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.323+0000 7f9d4e495640 1 -- 172.21.15.49:0/1543625668 shutdown_connections 2023-11-06T02:10:19.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.323+0000 7f9d4e495640 1 -- 172.21.15.49:0/1543625668 wait complete. 2023-11-06T02:10:19.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.323+0000 7f9d4e495640 1 Processor -- start 2023-11-06T02:10:19.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.323+0000 7f9d4e495640 1 -- start start 2023-11-06T02:10:19.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.323+0000 7f9d4e495640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d480b1450 0x7f9d4815aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:19.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.323+0000 7f9d4e495640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9d480cb010 0x7f9d480ac0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:19.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.323+0000 7f9d4e495640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d48159980 0x7f9d48157f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:19.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.323+0000 7f9d4e495640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d48154e60 con 0x7f9d48159980 2023-11-06T02:10:19.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.323+0000 7f9d4e495640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d48154290 con 0x7f9d480b1450 2023-11-06T02:10:19.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.323+0000 7f9d4e495640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d48154530 con 0x7f9d480cb010 2023-11-06T02:10:19.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.323+0000 7f9d4ca32640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d48159980 0x7f9d48157f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:19.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.323+0000 7f9d4ca32640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d48159980 0x7f9d48157f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46164/0 (socket says 172.21.15.49:46164) 2023-11-06T02:10:19.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.323+0000 7f9d4ca32640 1 -- 172.21.15.49:0/4189412950 learned_addr learned my addr 172.21.15.49:0/4189412950 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:19.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.323+0000 7f9d3ffff640 1 --2- 172.21.15.49:0/4189412950 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9d480cb010 0x7f9d480ac0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:19.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.323+0000 7f9d47fff640 1 --2- 172.21.15.49:0/4189412950 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d480b1450 0x7f9d4815aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:19.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.323+0000 7f9d4ca32640 1 -- 172.21.15.49:0/4189412950 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9d480cb010 msgr2=0x7f9d480ac0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:19.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.323+0000 7f9d4ca32640 1 --2- 172.21.15.49:0/4189412950 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9d480cb010 0x7f9d480ac0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:19.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.323+0000 7f9d4ca32640 1 -- 172.21.15.49:0/4189412950 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d480b1450 msgr2=0x7f9d4815aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:19.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.323+0000 7f9d4ca32640 1 --2- 172.21.15.49:0/4189412950 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d480b1450 0x7f9d4815aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:19.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.323+0000 7f9d4ca32640 1 -- 172.21.15.49:0/4189412950 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d40009b30 con 0x7f9d48159980 2023-11-06T02:10:19.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.323+0000 7f9d4ca32640 1 --2- 172.21.15.49:0/4189412950 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d48159980 0x7f9d48157f90 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f9d4001b080 tx=0x7f9d40019770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:19.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.323+0000 7f9d457fa640 1 -- 172.21.15.49:0/4189412950 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d400b8670 con 0x7f9d48159980 2023-11-06T02:10:19.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.323+0000 7f9d457fa640 1 -- 172.21.15.49:0/4189412950 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d400b87f0 con 0x7f9d48159980 2023-11-06T02:10:19.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.327+0000 7f9d457fa640 1 -- 172.21.15.49:0/4189412950 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d400b8ba0 con 0x7f9d48159980 2023-11-06T02:10:19.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.327+0000 7f9d4e495640 1 -- 172.21.15.49:0/4189412950 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d480ce3f0 con 0x7f9d48159980 2023-11-06T02:10:19.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.327+0000 7f9d4e495640 1 -- 172.21.15.49:0/4189412950 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d48154af0 con 0x7f9d48159980 2023-11-06T02:10:19.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.327+0000 7f9d4e495640 1 -- 172.21.15.49:0/4189412950 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d48074040 con 0x7f9d48159980 2023-11-06T02:10:19.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.327+0000 7f9d457fa640 1 -- 172.21.15.49:0/4189412950 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f9d400b8d00 con 0x7f9d48159980 2023-11-06T02:10:19.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.327+0000 7f9d457fa640 1 --2- 172.21.15.49:0/4189412950 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9d2c065a60 0x7f9d2c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:19.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.327+0000 7f9d457fa640 1 -- 172.21.15.49:0/4189412950 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f9d4011e830 con 0x7f9d48159980 2023-11-06T02:10:19.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.327+0000 7f9d47fff640 1 --2- 172.21.15.49:0/4189412950 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9d2c065a60 0x7f9d2c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:19.334 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.327+0000 7f9d47fff640 1 --2- 172.21.15.49:0/4189412950 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9d2c065a60 0x7f9d2c067f20 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9d480abf20 tx=0x7f9d38008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:19.334 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.331+0000 7f9d457fa640 1 -- 172.21.15.49:0/4189412950 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f9d400ec1b0 con 0x7f9d48159980 2023-11-06T02:10:19.516 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.515+0000 7f9d4e495640 1 -- 172.21.15.49:0/4189412950 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9d48154fa0 con 0x7f9d48159980 2023-11-06T02:10:19.517 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.519+0000 7f9d457fa640 1 -- 172.21.15.49:0/4189412950 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4651 (secure 0 0 0) 0x7f9d400ef7d0 con 0x7f9d48159980 2023-11-06T02:10:19.517 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:10:19.517 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5032,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/280428805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":280428805},{"type":"v1","addr":"172.21.15.150:6835","nonce":280428805}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":5035,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/2734589844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2734589844},{"type":"v1","addr":"172.21.15.49:6833","nonce":2734589844}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":5038,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/1319221246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":1319221246},{"type":"v1","addr":"172.21.15.150:6837","nonce":1319221246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T02:09:39.873514+0000","modified":"2023-11-06T02:09:43.132890+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5041},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5041":{"gid":5041,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.49:6835/1547192091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":1547192091},{"type":"v1","addr":"172.21.15.49:6835","nonce":1547192091}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-06T02:10:19.518 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 15 2023-11-06T02:10:19.519 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.519+0000 7f9d4e495640 1 -- 172.21.15.49:0/4189412950 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9d2c065a60 msgr2=0x7f9d2c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:19.519 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.519+0000 7f9d4e495640 1 --2- 172.21.15.49:0/4189412950 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9d2c065a60 0x7f9d2c067f20 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9d480abf20 tx=0x7f9d38008040 comp rx=0 tx=0).stop 2023-11-06T02:10:19.519 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.519+0000 7f9d4e495640 1 -- 172.21.15.49:0/4189412950 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d48159980 msgr2=0x7f9d48157f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:19.520 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.519+0000 7f9d4e495640 1 --2- 172.21.15.49:0/4189412950 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d48159980 0x7f9d48157f90 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f9d4001b080 tx=0x7f9d40019770 comp rx=0 tx=0).stop 2023-11-06T02:10:19.520 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.519+0000 7f9d4e495640 1 -- 172.21.15.49:0/4189412950 shutdown_connections 2023-11-06T02:10:19.520 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.519+0000 7f9d4e495640 1 --2- 172.21.15.49:0/4189412950 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9d2c065a60 0x7f9d2c067f20 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:19.520 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.519+0000 7f9d4e495640 1 --2- 172.21.15.49:0/4189412950 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d48159980 0x7f9d48157f90 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:19.520 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.519+0000 7f9d4e495640 1 --2- 172.21.15.49:0/4189412950 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9d480cb010 0x7f9d480ac0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:19.521 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.519+0000 7f9d4e495640 1 --2- 172.21.15.49:0/4189412950 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d480b1450 0x7f9d4815aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:19.521 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.519+0000 7f9d4e495640 1 -- 172.21.15.49:0/4189412950 >> 172.21.15.49:0/4189412950 conn(0x7f9d4806c0a0 msgr2=0x7f9d480ca030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:19.521 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.519+0000 7f9d4e495640 1 -- 172.21.15.49:0/4189412950 shutdown_connections 2023-11-06T02:10:19.521 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.519+0000 7f9d4e495640 1 -- 172.21.15.49:0/4189412950 wait complete. 2023-11-06T02:10:19.535 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T02:10:19.648 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.647+0000 7fa8729da640 1 Processor -- start 2023-11-06T02:10:19.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.647+0000 7fa8729da640 1 -- start start 2023-11-06T02:10:19.650 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.651+0000 7fa8729da640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa86c0b1f90 0x7fa86c0cda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:19.650 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.651+0000 7fa8729da640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa86c161650 0x7fa86c0d1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:19.650 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.651+0000 7fa8729da640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa86c0d17c0 con 0x7fa86c0d2190 2023-11-06T02:10:19.651 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.651+0000 7fa8729da640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa86c0ce6c0 con 0x7fa86c161650 2023-11-06T02:10:19.651 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.651+0000 7fa8729da640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa86c161ad0 con 0x7fa86c0b1f90 2023-11-06T02:10:19.651 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.651+0000 7fa86b7fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa86c161650 0x7fa86c0d1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:19.651 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.651+0000 7fa86b7fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa86c161650 0x7fa86c0d1c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46174/0 (socket says 172.21.15.49:46174) 2023-11-06T02:10:19.651 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.651+0000 7fa86b7fe640 1 -- 172.21.15.49:0/402521389 learned_addr learned my addr 172.21.15.49:0/402521389 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:19.652 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.651+0000 7fa86bfff640 1 --2- 172.21.15.49:0/402521389 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa86c0b1f90 0x7fa86c0cda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:19.652 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.651+0000 7fa86b7fe640 1 -- 172.21.15.49:0/402521389 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa86c0b1f90 msgr2=0x7fa86c0cda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:19.652 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.651+0000 7fa86b7fe640 1 --2- 172.21.15.49:0/402521389 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa86c0b1f90 0x7fa86c0cda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:19.652 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.651+0000 7fa86b7fe640 1 -- 172.21.15.49:0/402521389 >> v1:172.21.15.150:6789/0 conn(0x7fa86c0d2190 legacy=0x7fa86c0714a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:19.652 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.651+0000 7fa86b7fe640 1 -- 172.21.15.49:0/402521389 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa86c0ce100 con 0x7fa86c161650 2023-11-06T02:10:19.654 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.651+0000 7fa86b7fe640 1 --2- 172.21.15.49:0/402521389 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa86c161650 0x7fa86c0d1c50 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fa860009870 tx=0x7fa86001c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a84b926d51d3999 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:19.654 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.651+0000 7fa86affd640 1 -- 172.21.15.49:0/402521389 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8600aa670 con 0x7fa86c161650 2023-11-06T02:10:19.654 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.651+0000 7fa86affd640 1 -- 172.21.15.49:0/402521389 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8600aa7d0 con 0x7fa86c161650 2023-11-06T02:10:19.654 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.651+0000 7fa86affd640 1 -- 172.21.15.49:0/402521389 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8600aab60 con 0x7fa86c161650 2023-11-06T02:10:19.654 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.655+0000 7fa8729da640 1 -- 172.21.15.49:0/402521389 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa86c161650 msgr2=0x7fa86c0d1c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:19.655 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.655+0000 7fa8729da640 1 --2- 172.21.15.49:0/402521389 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa86c161650 0x7fa86c0d1c50 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fa860009870 tx=0x7fa86001c510 comp rx=0 tx=0).stop 2023-11-06T02:10:19.655 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.655+0000 7fa8729da640 1 -- 172.21.15.49:0/402521389 shutdown_connections 2023-11-06T02:10:19.655 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.655+0000 7fa8729da640 1 --2- 172.21.15.49:0/402521389 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa86c161650 0x7fa86c0d1c50 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:19.655 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.655+0000 7fa8729da640 1 --2- 172.21.15.49:0/402521389 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa86c0b1f90 0x7fa86c0cda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:19.655 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.655+0000 7fa8729da640 1 -- 172.21.15.49:0/402521389 >> 172.21.15.49:0/402521389 conn(0x7fa86c0d2aa0 msgr2=0x7fa86c0d2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:19.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.655+0000 7fa8729da640 1 -- 172.21.15.49:0/402521389 shutdown_connections 2023-11-06T02:10:19.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.655+0000 7fa8729da640 1 -- 172.21.15.49:0/402521389 wait complete. 2023-11-06T02:10:19.658 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.655+0000 7fa8729da640 1 Processor -- start 2023-11-06T02:10:19.658 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.655+0000 7fa8729da640 1 -- start start 2023-11-06T02:10:19.659 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.655+0000 7fa8729da640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa86c0b1f90 0x7fa86c0cba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:19.659 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.655+0000 7fa8729da640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa86c0d2190 0x7fa86c0c8d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:19.659 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.655+0000 7fa8729da640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa86c161650 0x7fa86c13afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:19.659 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.655+0000 7fa8729da640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa86c0ccc60 con 0x7fa86c0d2190 2023-11-06T02:10:19.659 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.655+0000 7fa8729da640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa86c0cc050 con 0x7fa86c161650 2023-11-06T02:10:19.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.655+0000 7fa8729da640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa86c0cc330 con 0x7fa86c0b1f90 2023-11-06T02:10:19.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.655+0000 7fa86b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa86c0d2190 0x7fa86c0c8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:19.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.655+0000 7fa86b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa86c0d2190 0x7fa86c0c8d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46178/0 (socket says 172.21.15.49:46178) 2023-11-06T02:10:19.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.655+0000 7fa86b7fe640 1 -- 172.21.15.49:0/102457937 learned_addr learned my addr 172.21.15.49:0/102457937 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:19.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.655+0000 7fa870f77640 1 --2- 172.21.15.49:0/102457937 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa86c161650 0x7fa86c13afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:19.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.655+0000 7fa86bfff640 1 --2- 172.21.15.49:0/102457937 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa86c0b1f90 0x7fa86c0cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:19.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.655+0000 7fa86b7fe640 1 -- 172.21.15.49:0/102457937 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa86c0b1f90 msgr2=0x7fa86c0cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:19.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.655+0000 7fa86b7fe640 1 --2- 172.21.15.49:0/102457937 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa86c0b1f90 0x7fa86c0cba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:19.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.655+0000 7fa86b7fe640 1 -- 172.21.15.49:0/102457937 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa86c161650 msgr2=0x7fa86c13afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:19.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.655+0000 7fa86b7fe640 1 --2- 172.21.15.49:0/102457937 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa86c161650 0x7fa86c13afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:19.662 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.655+0000 7fa86b7fe640 1 -- 172.21.15.49:0/102457937 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8600094d0 con 0x7fa86c0d2190 2023-11-06T02:10:19.662 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.655+0000 7fa870f77640 1 --2- 172.21.15.49:0/102457937 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa86c161650 0x7fa86c13afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:10:19.662 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.655+0000 7fa86b7fe640 1 --2- 172.21.15.49:0/102457937 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa86c0d2190 0x7fa86c0c8d40 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fa8600ac040 tx=0x7fa8600192b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:19.662 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.655+0000 7fa868ff9640 1 -- 172.21.15.49:0/102457937 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8600b4810 con 0x7fa86c0d2190 2023-11-06T02:10:19.662 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.655+0000 7fa868ff9640 1 -- 172.21.15.49:0/102457937 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8600b4990 con 0x7fa86c0d2190 2023-11-06T02:10:19.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.655+0000 7fa8729da640 1 -- 172.21.15.49:0/102457937 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa86c0cc610 con 0x7fa86c0d2190 2023-11-06T02:10:19.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.655+0000 7fa868ff9640 1 -- 172.21.15.49:0/102457937 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8600b4d40 con 0x7fa86c0d2190 2023-11-06T02:10:19.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.655+0000 7fa8729da640 1 -- 172.21.15.49:0/102457937 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa86c0c9280 con 0x7fa86c0d2190 2023-11-06T02:10:19.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.655+0000 7fa8729da640 1 -- 172.21.15.49:0/102457937 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa86c074040 con 0x7fa86c0d2190 2023-11-06T02:10:19.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.659+0000 7fa868ff9640 1 -- 172.21.15.49:0/102457937 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fa8600be410 con 0x7fa86c0d2190 2023-11-06T02:10:19.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.659+0000 7fa868ff9640 1 --2- 172.21.15.49:0/102457937 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa858065a60 0x7fa858067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:19.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.659+0000 7fa868ff9640 1 -- 172.21.15.49:0/102457937 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6091+0+0 (secure 0 0 0) 0x7fa86011ddf0 con 0x7fa86c0d2190 2023-11-06T02:10:19.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.659+0000 7fa86bfff640 1 --2- 172.21.15.49:0/102457937 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa858065a60 0x7fa858067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:19.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.659+0000 7fa86bfff640 1 --2- 172.21.15.49:0/102457937 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa858065a60 0x7fa858067f20 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa85c00b9d0 tx=0x7fa85c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:19.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.663+0000 7fa868ff9640 1 -- 172.21.15.49:0/102457937 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fa8600eb630 con 0x7fa86c0d2190 2023-11-06T02:10:19.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.811+0000 7fa8729da640 1 -- 172.21.15.49:0/102457937 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa86c168da0 con 0x7fa86c0d2190 2023-11-06T02:10:19.814 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.815+0000 7fa868ff9640 1 -- 172.21.15.49:0/102457937 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+17932 (secure 0 0 0) 0x7fa8600f3e60 con 0x7fa86c0d2190 2023-11-06T02:10:19.814 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:10:19.815 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":39,"fsid":"1a8b035c-6ba4-499a-b876-aea47fd9c0d4","created":"2023-11-06T02:08:06.440857+0000","modified":"2023-11-06T02:10:14.167395+0000","last_up_change":"2023-11-06T02:08:17.982372+0000","last_in_change":"2023-11-06T02:08:08.906631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T02:08:19.252222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T02:08:24.776211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-06T02:09:37.906848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-06T02:09:38.849380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ad55af8-0571-4e6a-8722-b5d429b7496d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6809","nonce":38116},{"type":"v1","addr":"172.21.15.49:6811","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6813","nonce":38116},{"type":"v1","addr":"172.21.15.49:6815","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6821","nonce":38116},{"type":"v1","addr":"172.21.15.49:6823","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6817","nonce":38116},{"type":"v1","addr":"172.21.15.49:6819","nonce":38116}]},"public_addr":"172.21.15.49:6811/38116","cluster_addr":"172.21.15.49:6815/38116","heartbeat_back_addr":"172.21.15.49:6823/38116","heartbeat_front_addr":"172.21.15.49:6819/38116","state":["exists","up"]},{"osd":1,"uuid":"c8b31f85-a757-43ed-9419-351acc46611b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":38113},{"type":"v1","addr":"172.21.15.49:6810","nonce":38113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":38113},{"type":"v1","addr":"172.21.15.49:6814","nonce":38113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":38113},{"type":"v1","addr":"172.21.15.49:6822","nonce":38113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":38113},{"type":"v1","addr":"172.21.15.49:6818","nonce":38113}]},"public_addr":"172.21.15.49:6810/38113","cluster_addr":"172.21.15.49:6814/38113","heartbeat_back_addr":"172.21.15.49:6822/38113","heartbeat_front_addr":"172.21.15.49:6818/38113","state":["exists","up"]},{"osd":2,"uuid":"555b3bf3-0a70-4b91-8efd-e4204b5fc26e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":38114},{"type":"v1","addr":"172.21.15.49:6801","nonce":38114}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":38114},{"type":"v1","addr":"172.21.15.49:6803","nonce":38114}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":38114},{"type":"v1","addr":"172.21.15.49:6807","nonce":38114}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":38114},{"type":"v1","addr":"172.21.15.49:6805","nonce":38114}]},"public_addr":"172.21.15.49:6801/38114","cluster_addr":"172.21.15.49:6803/38114","heartbeat_back_addr":"172.21.15.49:6807/38114","heartbeat_front_addr":"172.21.15.49:6805/38114","state":["exists","up"]},{"osd":3,"uuid":"8d4e5d77-03ef-4b9b-bfcb-5ae93a46795c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":38115},{"type":"v1","addr":"172.21.15.49:6825","nonce":38115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":38115},{"type":"v1","addr":"172.21.15.49:6827","nonce":38115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":38115},{"type":"v1","addr":"172.21.15.49:6831","nonce":38115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":38115},{"type":"v1","addr":"172.21.15.49:6829","nonce":38115}]},"public_addr":"172.21.15.49:6825/38115","cluster_addr":"172.21.15.49:6827/38115","heartbeat_back_addr":"172.21.15.49:6831/38115","heartbeat_front_addr":"172.21.15.49:6829/38115","state":["exists","up"]},{"osd":4,"uuid":"dba03ad4-68a3-4cf3-9218-ac611f02eacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":38417},{"type":"v1","addr":"172.21.15.150:6809","nonce":38417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":38417},{"type":"v1","addr":"172.21.15.150:6811","nonce":38417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":38417},{"type":"v1","addr":"172.21.15.150:6815","nonce":38417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":38417},{"type":"v1","addr":"172.21.15.150:6813","nonce":38417}]},"public_addr":"172.21.15.150:6809/38417","cluster_addr":"172.21.15.150:6811/38417","heartbeat_back_addr":"172.21.15.150:6815/38417","heartbeat_front_addr":"172.21.15.150:6813/38417","state":["exists","up"]},{"osd":5,"uuid":"0df9c303-c255-4640-8e91-01f1c3efe18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":38418},{"type":"v1","addr":"172.21.15.150:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":38418},{"type":"v1","addr":"172.21.15.150:6803","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":38418},{"type":"v1","addr":"172.21.15.150:6807","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":38418},{"type":"v1","addr":"172.21.15.150:6805","nonce":38418}]},"public_addr":"172.21.15.150:6801/38418","cluster_addr":"172.21.15.150:6803/38418","heartbeat_back_addr":"172.21.15.150:6807/38418","heartbeat_front_addr":"172.21.15.150:6805/38418","state":["exists","up"]},{"osd":6,"uuid":"e59fa189-4b70-4c06-9722-c02b90828bf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":38428},{"type":"v1","addr":"172.21.15.150:6825","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":38428},{"type":"v1","addr":"172.21.15.150:6827","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":38428},{"type":"v1","addr":"172.21.15.150:6831","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":38428},{"type":"v1","addr":"172.21.15.150:6829","nonce":38428}]},"public_addr":"172.21.15.150:6825/38428","cluster_addr":"172.21.15.150:6827/38428","heartbeat_back_addr":"172.21.15.150:6831/38428","heartbeat_front_addr":"172.21.15.150:6829/38428","state":["exists","up"]},{"osd":7,"uuid":"a64f57e1-ef98-46bd-b304-385759e95898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":38421},{"type":"v1","addr":"172.21.15.150:6817","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":38421},{"type":"v1","addr":"172.21.15.150:6819","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":38421},{"type":"v1","addr":"172.21.15.150:6823","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":38421},{"type":"v1","addr":"172.21.15.150:6821","nonce":38421}]},"public_addr":"172.21.15.150:6817/38421","cluster_addr":"172.21.15.150:6819/38421","heartbeat_back_addr":"172.21.15.150:6823/38421","heartbeat_front_addr":"172.21.15.150:6821/38421","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.438552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.413280+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.296466+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.424966+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.276953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.230984+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.410235+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.480084+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.3d","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T02:10:19.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.815+0000 7fa8729da640 1 -- 172.21.15.49:0/102457937 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa858065a60 msgr2=0x7fa858067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:19.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.815+0000 7fa8729da640 1 --2- 172.21.15.49:0/102457937 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa858065a60 0x7fa858067f20 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa85c00b9d0 tx=0x7fa85c008040 comp rx=0 tx=0).stop 2023-11-06T02:10:19.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.815+0000 7fa8729da640 1 -- 172.21.15.49:0/102457937 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa86c0d2190 msgr2=0x7fa86c0c8d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:19.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.815+0000 7fa8729da640 1 --2- 172.21.15.49:0/102457937 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa86c0d2190 0x7fa86c0c8d40 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fa8600ac040 tx=0x7fa8600192b0 comp rx=0 tx=0).stop 2023-11-06T02:10:19.818 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.815+0000 7fa8729da640 1 -- 172.21.15.49:0/102457937 shutdown_connections 2023-11-06T02:10:19.818 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.815+0000 7fa8729da640 1 --2- 172.21.15.49:0/102457937 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa858065a60 0x7fa858067f20 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:19.818 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.815+0000 7fa8729da640 1 --2- 172.21.15.49:0/102457937 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa86c0d2190 0x7fa86c0c8d40 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:19.818 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.815+0000 7fa8729da640 1 --2- 172.21.15.49:0/102457937 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa86c161650 0x7fa86c13afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:19.818 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.815+0000 7fa8729da640 1 --2- 172.21.15.49:0/102457937 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa86c0b1f90 0x7fa86c0cba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:19.819 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.815+0000 7fa8729da640 1 -- 172.21.15.49:0/102457937 >> 172.21.15.49:0/102457937 conn(0x7fa86c0d2aa0 msgr2=0x7fa86c0b41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:19.819 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.815+0000 7fa8729da640 1 -- 172.21.15.49:0/102457937 shutdown_connections 2023-11-06T02:10:19.819 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.815+0000 7fa8729da640 1 -- 172.21.15.49:0/102457937 wait complete. 2023-11-06T02:10:19.832 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-06T02:10:19.833 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T02:10:19.957 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.959+0000 7f83695da640 1 Processor -- start 2023-11-06T02:10:19.957 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.959+0000 7f83695da640 1 -- start start 2023-11-06T02:10:19.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.959+0000 7f83695da640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f836415ab50 0x7f83640c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:19.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.959+0000 7f83695da640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83640cab80 con 0x7f83640b0790 2023-11-06T02:10:19.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.959+0000 7f83695da640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f836415af30 con 0x7f83640cb010 2023-11-06T02:10:19.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.959+0000 7f83695da640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83640cb5b0 con 0x7f836415ab50 2023-11-06T02:10:19.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.959+0000 7f8362ffd640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f836415ab50 0x7f83640c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:19.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.959+0000 7f83637fe640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f83640b0790 0x7f83640c2710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:59722/0 (socket says 172.21.15.49:59722) 2023-11-06T02:10:19.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.959+0000 7f83637fe640 1 -- 172.21.15.49:0/378769676 learned_addr learned my addr 172.21.15.49:0/378769676 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:10:19.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.959+0000 7f8361ffb640 1 -- 172.21.15.49:0/378769676 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1749366888 0 0) 0x7f83640cab80 con 0x7f83640b0790 2023-11-06T02:10:19.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.959+0000 7f8361ffb640 1 -- 172.21.15.49:0/378769676 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8350003530 con 0x7f83640b0790 2023-11-06T02:10:19.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.959+0000 7f8362ffd640 1 -- 172.21.15.49:0/378769676 >> v1:172.21.15.150:6789/0 conn(0x7f83640cb010 legacy=0x7f83640b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:19.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.959+0000 7f8362ffd640 1 -- 172.21.15.49:0/378769676 >> v1:172.21.15.49:6789/0 conn(0x7f83640b0790 legacy=0x7f83640c2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:19.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.959+0000 7f8362ffd640 1 -- 172.21.15.49:0/378769676 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83640cb890 con 0x7f836415ab50 2023-11-06T02:10:19.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.959+0000 7f8362ffd640 1 --2- 172.21.15.49:0/378769676 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f836415ab50 0x7f83640c9bb0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f83540097a0 tx=0x7f835401c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5664f1d6a0efc92 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:19.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.963+0000 7f8361ffb640 1 -- 172.21.15.49:0/378769676 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83540a7070 con 0x7f836415ab50 2023-11-06T02:10:19.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.963+0000 7f8361ffb640 1 -- 172.21.15.49:0/378769676 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83540ab430 con 0x7f836415ab50 2023-11-06T02:10:19.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.963+0000 7f8361ffb640 1 -- 172.21.15.49:0/378769676 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83540ab7c0 con 0x7f836415ab50 2023-11-06T02:10:19.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.963+0000 7f83695da640 1 -- 172.21.15.49:0/378769676 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f836415ab50 msgr2=0x7f83640c9bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:19.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.963+0000 7f83695da640 1 --2- 172.21.15.49:0/378769676 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f836415ab50 0x7f83640c9bb0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f83540097a0 tx=0x7f835401c430 comp rx=0 tx=0).stop 2023-11-06T02:10:19.964 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.963+0000 7f83695da640 1 -- 172.21.15.49:0/378769676 shutdown_connections 2023-11-06T02:10:19.964 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.963+0000 7f83695da640 1 --2- 172.21.15.49:0/378769676 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f836415ab50 0x7f83640c9bb0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:19.964 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.963+0000 7f83695da640 1 -- 172.21.15.49:0/378769676 >> 172.21.15.49:0/378769676 conn(0x7f836406c0a0 msgr2=0x7f836406c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:19.964 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.963+0000 7f83695da640 1 -- 172.21.15.49:0/378769676 shutdown_connections 2023-11-06T02:10:19.964 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.963+0000 7f83695da640 1 -- 172.21.15.49:0/378769676 wait complete. 2023-11-06T02:10:19.965 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.963+0000 7f83695da640 1 Processor -- start 2023-11-06T02:10:19.965 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.963+0000 7f83695da640 1 -- start start 2023-11-06T02:10:19.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.963+0000 7f83695da640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f83640b0790 0x7f8364150940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:19.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.963+0000 7f83695da640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83640cb010 0x7f836414f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:19.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.963+0000 7f83695da640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f836415ab50 0x7f836414c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:19.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.963+0000 7f83695da640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8364150e80 con 0x7f836415ab50 2023-11-06T02:10:19.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.963+0000 7f83695da640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f836414f6a0 con 0x7f83640b0790 2023-11-06T02:10:19.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.963+0000 7f83695da640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83640c7410 con 0x7f83640cb010 2023-11-06T02:10:19.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.963+0000 7f83637fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f836415ab50 0x7f836414c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:19.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.963+0000 7f83637fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f836415ab50 0x7f836414c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46198/0 (socket says 172.21.15.49:46198) 2023-11-06T02:10:19.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.963+0000 7f83637fe640 1 -- 172.21.15.49:0/2650394743 learned_addr learned my addr 172.21.15.49:0/2650394743 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:19.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.963+0000 7f8362ffd640 1 --2- 172.21.15.49:0/2650394743 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f83640b0790 0x7f8364150940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:19.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.963+0000 7f83627fc640 1 --2- 172.21.15.49:0/2650394743 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83640cb010 0x7f836414f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:19.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.963+0000 7f83637fe640 1 -- 172.21.15.49:0/2650394743 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83640cb010 msgr2=0x7f836414f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:19.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.963+0000 7f83637fe640 1 --2- 172.21.15.49:0/2650394743 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83640cb010 0x7f836414f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:19.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.963+0000 7f83637fe640 1 -- 172.21.15.49:0/2650394743 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f83640b0790 msgr2=0x7f8364150940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:19.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.963+0000 7f83637fe640 1 --2- 172.21.15.49:0/2650394743 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f83640b0790 0x7f8364150940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:19.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.963+0000 7f83637fe640 1 -- 172.21.15.49:0/2650394743 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83540093b0 con 0x7f836415ab50 2023-11-06T02:10:19.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.963+0000 7f8362ffd640 1 --2- 172.21.15.49:0/2650394743 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f83640b0790 0x7f8364150940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:10:19.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.963+0000 7f83637fe640 1 --2- 172.21.15.49:0/2650394743 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f836415ab50 0x7f836414c440 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f835c00c420 tx=0x7f835c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:19.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.963+0000 7f835b7fe640 1 -- 172.21.15.49:0/2650394743 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f835c005cf0 con 0x7f836415ab50 2023-11-06T02:10:19.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.963+0000 7f835b7fe640 1 -- 172.21.15.49:0/2650394743 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f835c005e50 con 0x7f836415ab50 2023-11-06T02:10:19.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.963+0000 7f835b7fe640 1 -- 172.21.15.49:0/2650394743 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f835c0072d0 con 0x7f836415ab50 2023-11-06T02:10:19.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.963+0000 7f83695da640 1 -- 172.21.15.49:0/2650394743 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83640c5a70 con 0x7f836415ab50 2023-11-06T02:10:19.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.967+0000 7f83695da640 1 -- 172.21.15.49:0/2650394743 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8364152e80 con 0x7f836415ab50 2023-11-06T02:10:19.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.967+0000 7f83695da640 1 -- 172.21.15.49:0/2650394743 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83640c75a0 con 0x7f836415ab50 2023-11-06T02:10:19.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.967+0000 7f835b7fe640 1 -- 172.21.15.49:0/2650394743 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f835c007430 con 0x7f836415ab50 2023-11-06T02:10:19.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.967+0000 7f835b7fe640 1 --2- 172.21.15.49:0/2650394743 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8344065a90 0x7f8344067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:19.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.967+0000 7f835b7fe640 1 -- 172.21.15.49:0/2650394743 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f835c082430 con 0x7f836415ab50 2023-11-06T02:10:19.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.967+0000 7f8362ffd640 1 --2- 172.21.15.49:0/2650394743 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8344065a90 0x7f8344067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:19.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.967+0000 7f8362ffd640 1 --2- 172.21.15.49:0/2650394743 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8344065a90 0x7f8344067f50 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f83540af000 tx=0x7f8354008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:19.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:19.971+0000 7f835b7fe640 1 -- 172.21.15.49:0/2650394743 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f835c04fde0 con 0x7f836415ab50 2023-11-06T02:10:20.160 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.159+0000 7f83695da640 1 -- 172.21.15.49:0/2650394743 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f83640b0da0 con 0x7f836415ab50 2023-11-06T02:10:20.161 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.163+0000 7f835b7fe640 1 -- 172.21.15.49:0/2650394743 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4651 (secure 0 0 0) 0x7f835c053400 con 0x7f836415ab50 2023-11-06T02:10:20.162 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:10:20.162 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5032,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/280428805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":280428805},{"type":"v1","addr":"172.21.15.150:6835","nonce":280428805}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":5035,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/2734589844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2734589844},{"type":"v1","addr":"172.21.15.49:6833","nonce":2734589844}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":5038,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/1319221246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":1319221246},{"type":"v1","addr":"172.21.15.150:6837","nonce":1319221246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T02:09:39.873514+0000","modified":"2023-11-06T02:09:43.132890+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5041},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5041":{"gid":5041,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.49:6835/1547192091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":1547192091},{"type":"v1","addr":"172.21.15.49:6835","nonce":1547192091}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-06T02:10:20.163 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 15 2023-11-06T02:10:20.164 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.163+0000 7f83695da640 1 -- 172.21.15.49:0/2650394743 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8344065a90 msgr2=0x7f8344067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:20.165 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.163+0000 7f83695da640 1 --2- 172.21.15.49:0/2650394743 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8344065a90 0x7f8344067f50 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f83540af000 tx=0x7f8354008040 comp rx=0 tx=0).stop 2023-11-06T02:10:20.165 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.163+0000 7f83695da640 1 -- 172.21.15.49:0/2650394743 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f836415ab50 msgr2=0x7f836414c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:20.165 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.163+0000 7f83695da640 1 --2- 172.21.15.49:0/2650394743 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f836415ab50 0x7f836414c440 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f835c00c420 tx=0x7f835c00c8e0 comp rx=0 tx=0).stop 2023-11-06T02:10:20.165 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.163+0000 7f83695da640 1 -- 172.21.15.49:0/2650394743 shutdown_connections 2023-11-06T02:10:20.165 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.163+0000 7f83695da640 1 --2- 172.21.15.49:0/2650394743 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8344065a90 0x7f8344067f50 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:20.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.163+0000 7f83695da640 1 --2- 172.21.15.49:0/2650394743 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f836415ab50 0x7f836414c440 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:20.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.163+0000 7f83695da640 1 --2- 172.21.15.49:0/2650394743 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83640cb010 0x7f836414f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:20.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.163+0000 7f83695da640 1 --2- 172.21.15.49:0/2650394743 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f83640b0790 0x7f8364150940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:20.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.163+0000 7f83695da640 1 -- 172.21.15.49:0/2650394743 >> 172.21.15.49:0/2650394743 conn(0x7f836406c0a0 msgr2=0x7f83640af890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:20.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.163+0000 7f83695da640 1 -- 172.21.15.49:0/2650394743 shutdown_connections 2023-11-06T02:10:20.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.163+0000 7f83695da640 1 -- 172.21.15.49:0/2650394743 wait complete. 2023-11-06T02:10:20.182 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T02:10:20.309 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.311+0000 7f550cafe640 1 Processor -- start 2023-11-06T02:10:20.309 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.311+0000 7f550cafe640 1 -- start start 2023-11-06T02:10:20.311 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.311+0000 7f550cafe640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f55080b1f90 0x7f55080cda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:20.311 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.311+0000 7f550cafe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f55080714a0 0x7f55080718b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:20.311 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.311+0000 7f550cafe640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55080d17c0 con 0x7f5508161650 2023-11-06T02:10:20.311 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.311+0000 7f550cafe640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5508161b00 con 0x7f55080714a0 2023-11-06T02:10:20.311 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.311+0000 7f550cafe640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5508071df0 con 0x7f55080b1f90 2023-11-06T02:10:20.311 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.311+0000 7f5506d9d640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f55080714a0 0x7f55080718b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:20.312 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.311+0000 7f5506d9d640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f55080714a0 0x7f55080718b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46214/0 (socket says 172.21.15.49:46214) 2023-11-06T02:10:20.312 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.311+0000 7f5506d9d640 1 -- 172.21.15.49:0/1049838563 learned_addr learned my addr 172.21.15.49:0/1049838563 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:20.312 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.311+0000 7f550659c640 1 --2- 172.21.15.49:0/1049838563 >> v2:172.21.15.150:3300/0 conn(0x7f55080b1f90 0x7f55080cda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:20.313 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.311+0000 7f5506d9d640 1 -- 172.21.15.49:0/1049838563 >> v2:172.21.15.150:3300/0 conn(0x7f55080b1f90 msgr2=0x7f55080cda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:20.313 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.311+0000 7f5506d9d640 1 --2- 172.21.15.49:0/1049838563 >> v2:172.21.15.150:3300/0 conn(0x7f55080b1f90 0x7f55080cda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:20.314 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.311+0000 7f5506d9d640 1 -- 172.21.15.49:0/1049838563 >> v1:172.21.15.150:6789/0 conn(0x7f5508161650 legacy=0x7f55080d1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:20.314 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.311+0000 7f5506d9d640 1 -- 172.21.15.49:0/1049838563 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55080ce6c0 con 0x7f55080714a0 2023-11-06T02:10:20.314 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.311+0000 7f550659c640 1 --2- 172.21.15.49:0/1049838563 >> v2:172.21.15.150:3300/0 conn(0x7f55080b1f90 0x7f55080cda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:10:20.317 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.311+0000 7f5506d9d640 1 --2- 172.21.15.49:0/1049838563 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55080714a0 0x7f55080718b0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f5508072180 tx=0x7f54f801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=249c0bae6f16968 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:20.317 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.311+0000 7f550559a640 1 -- 172.21.15.49:0/1049838563 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f80a6040 con 0x7f55080714a0 2023-11-06T02:10:20.317 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.315+0000 7f550559a640 1 -- 172.21.15.49:0/1049838563 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54f80aa600 con 0x7f55080714a0 2023-11-06T02:10:20.318 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.315+0000 7f550559a640 1 -- 172.21.15.49:0/1049838563 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f80aa990 con 0x7f55080714a0 2023-11-06T02:10:20.318 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.315+0000 7f550cafe640 1 -- 172.21.15.49:0/1049838563 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55080714a0 msgr2=0x7f55080718b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:20.318 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.315+0000 7f550cafe640 1 --2- 172.21.15.49:0/1049838563 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55080714a0 0x7f55080718b0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f5508072180 tx=0x7f54f801c4a0 comp rx=0 tx=0).stop 2023-11-06T02:10:20.318 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.315+0000 7f550cafe640 1 -- 172.21.15.49:0/1049838563 shutdown_connections 2023-11-06T02:10:20.319 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.315+0000 7f550cafe640 1 --2- 172.21.15.49:0/1049838563 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55080714a0 0x7f55080718b0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:20.319 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.315+0000 7f550cafe640 1 --2- 172.21.15.49:0/1049838563 >> v2:172.21.15.150:3300/0 conn(0x7f55080b1f90 0x7f55080cda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:20.319 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.315+0000 7f550cafe640 1 -- 172.21.15.49:0/1049838563 >> 172.21.15.49:0/1049838563 conn(0x7f55080d2aa0 msgr2=0x7f55080d2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:20.319 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.315+0000 7f550cafe640 1 -- 172.21.15.49:0/1049838563 shutdown_connections 2023-11-06T02:10:20.319 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.315+0000 7f550cafe640 1 -- 172.21.15.49:0/1049838563 wait complete. 2023-11-06T02:10:20.320 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.315+0000 7f550cafe640 1 Processor -- start 2023-11-06T02:10:20.320 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.315+0000 7f550cafe640 1 -- start start 2023-11-06T02:10:20.320 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.315+0000 7f550cafe640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f55080714a0 0x7f5508158560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:20.320 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.315+0000 7f550cafe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55080b1f90 0x7f55080cc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:20.320 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.315+0000 7f550cafe640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5508161650 0x7f5508158d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:20.321 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.315+0000 7f550cafe640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55080ca700 con 0x7f55080b1f90 2023-11-06T02:10:20.321 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.315+0000 7f550cafe640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55080ca950 con 0x7f5508161650 2023-11-06T02:10:20.321 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.315+0000 7f550cafe640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55080cac30 con 0x7f55080714a0 2023-11-06T02:10:20.321 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.315+0000 7f5505d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55080b1f90 0x7f55080cc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:20.321 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.315+0000 7f5505d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55080b1f90 0x7f55080cc6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46218/0 (socket says 172.21.15.49:46218) 2023-11-06T02:10:20.322 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.315+0000 7f5505d9b640 1 -- 172.21.15.49:0/26408665 learned_addr learned my addr 172.21.15.49:0/26408665 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:20.322 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.315+0000 7f5506d9d640 1 --2- 172.21.15.49:0/26408665 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5508161650 0x7f5508158d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:20.322 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.315+0000 7f550659c640 1 --2- 172.21.15.49:0/26408665 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f55080714a0 0x7f5508158560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:20.322 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.315+0000 7f5505d9b640 1 -- 172.21.15.49:0/26408665 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f55080714a0 msgr2=0x7f5508158560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:20.322 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.315+0000 7f5505d9b640 1 --2- 172.21.15.49:0/26408665 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f55080714a0 0x7f5508158560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:20.323 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.315+0000 7f5505d9b640 1 -- 172.21.15.49:0/26408665 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5508161650 msgr2=0x7f5508158d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:20.323 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.315+0000 7f5505d9b640 1 --2- 172.21.15.49:0/26408665 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5508161650 0x7f5508158d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:20.323 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.315+0000 7f5505d9b640 1 -- 172.21.15.49:0/26408665 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54f80094d0 con 0x7f55080b1f90 2023-11-06T02:10:20.323 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.315+0000 7f550659c640 1 --2- 172.21.15.49:0/26408665 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f55080714a0 0x7f5508158560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:10:20.323 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.315+0000 7f5505d9b640 1 --2- 172.21.15.49:0/26408665 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55080b1f90 0x7f55080cc6a0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f54fc00b410 tx=0x7f54fc00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:20.324 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.315+0000 7f54f6ffd640 1 -- 172.21.15.49:0/26408665 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54fc004020 con 0x7f55080b1f90 2023-11-06T02:10:20.324 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.315+0000 7f54f6ffd640 1 -- 172.21.15.49:0/26408665 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54fc004180 con 0x7f55080b1f90 2023-11-06T02:10:20.324 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.315+0000 7f54f6ffd640 1 -- 172.21.15.49:0/26408665 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54fc011630 con 0x7f55080b1f90 2023-11-06T02:10:20.324 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.315+0000 7f550cafe640 1 -- 172.21.15.49:0/26408665 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55080caf10 con 0x7f55080b1f90 2023-11-06T02:10:20.324 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.315+0000 7f550cafe640 1 -- 172.21.15.49:0/26408665 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55080cb4d0 con 0x7f55080b1f90 2023-11-06T02:10:20.324 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.319+0000 7f550cafe640 1 -- 172.21.15.49:0/26408665 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55080cb6f0 con 0x7f55080b1f90 2023-11-06T02:10:20.325 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.319+0000 7f54f6ffd640 1 -- 172.21.15.49:0/26408665 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f54fc011790 con 0x7f55080b1f90 2023-11-06T02:10:20.326 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.319+0000 7f54f6ffd640 1 --2- 172.21.15.49:0/26408665 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f54e8065a90 0x7f54e8067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:20.326 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.319+0000 7f54f6ffd640 1 -- 172.21.15.49:0/26408665 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f54fc0830c0 con 0x7f55080b1f90 2023-11-06T02:10:20.326 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.319+0000 7f550659c640 1 --2- 172.21.15.49:0/26408665 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f54e8065a90 0x7f54e8067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:20.326 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.319+0000 7f550659c640 1 --2- 172.21.15.49:0/26408665 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f54e8065a90 0x7f54e8067f50 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f550000d430 tx=0x7f5500007530 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:20.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.323+0000 7f54f6ffd640 1 -- 172.21.15.49:0/26408665 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f54fc050870 con 0x7f55080b1f90 2023-11-06T02:10:20.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.511+0000 7f550cafe640 1 -- 172.21.15.49:0/26408665 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f55080cd610 con 0x7f55080b1f90 2023-11-06T02:10:20.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.515+0000 7f54f6ffd640 1 -- 172.21.15.49:0/26408665 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4651 (secure 0 0 0) 0x7f54fc053e90 con 0x7f55080b1f90 2023-11-06T02:10:20.513 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:10:20.514 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5032,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/280428805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":280428805},{"type":"v1","addr":"172.21.15.150:6835","nonce":280428805}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":5035,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/2734589844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2734589844},{"type":"v1","addr":"172.21.15.49:6833","nonce":2734589844}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":5038,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/1319221246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":1319221246},{"type":"v1","addr":"172.21.15.150:6837","nonce":1319221246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T02:09:39.873514+0000","modified":"2023-11-06T02:09:43.132890+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5041},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5041":{"gid":5041,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.49:6835/1547192091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":1547192091},{"type":"v1","addr":"172.21.15.49:6835","nonce":1547192091}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-06T02:10:20.514 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 15 2023-11-06T02:10:20.516 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.515+0000 7f550cafe640 1 -- 172.21.15.49:0/26408665 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f54e8065a90 msgr2=0x7f54e8067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:20.516 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.515+0000 7f550cafe640 1 --2- 172.21.15.49:0/26408665 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f54e8065a90 0x7f54e8067f50 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f550000d430 tx=0x7f5500007530 comp rx=0 tx=0).stop 2023-11-06T02:10:20.517 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.515+0000 7f550cafe640 1 -- 172.21.15.49:0/26408665 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55080b1f90 msgr2=0x7f55080cc6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:20.517 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.515+0000 7f550cafe640 1 --2- 172.21.15.49:0/26408665 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55080b1f90 0x7f55080cc6a0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f54fc00b410 tx=0x7f54fc00b8d0 comp rx=0 tx=0).stop 2023-11-06T02:10:20.517 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.515+0000 7f550cafe640 1 -- 172.21.15.49:0/26408665 shutdown_connections 2023-11-06T02:10:20.517 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.515+0000 7f550cafe640 1 --2- 172.21.15.49:0/26408665 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f54e8065a90 0x7f54e8067f50 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:20.517 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.515+0000 7f550cafe640 1 --2- 172.21.15.49:0/26408665 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55080b1f90 0x7f55080cc6a0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:20.518 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.515+0000 7f550cafe640 1 --2- 172.21.15.49:0/26408665 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5508161650 0x7f5508158d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:20.518 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.515+0000 7f550cafe640 1 --2- 172.21.15.49:0/26408665 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f55080714a0 0x7f5508158560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:20.518 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.515+0000 7f550cafe640 1 -- 172.21.15.49:0/26408665 >> 172.21.15.49:0/26408665 conn(0x7f55080d2aa0 msgr2=0x7f55080b7e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:20.518 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.515+0000 7f550cafe640 1 -- 172.21.15.49:0/26408665 shutdown_connections 2023-11-06T02:10:20.519 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.515+0000 7f550cafe640 1 -- 172.21.15.49:0/26408665 wait complete. 2023-11-06T02:10:20.533 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T02:10:20.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.651+0000 7fa6cb521640 1 Processor -- start 2023-11-06T02:10:20.650 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.651+0000 7fa6cb521640 1 -- start start 2023-11-06T02:10:20.650 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.651+0000 7fa6cb521640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fa6c40b0790 0x7fa6c40c8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:20.651 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.651+0000 7fa6cb521640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6c40adae0 0x7fa6c4070b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:20.651 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.651+0000 7fa6cb521640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6c41594d0 con 0x7fa6c41590f0 2023-11-06T02:10:20.651 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.651+0000 7fa6cb521640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6c415a140 con 0x7fa6c40b0790 2023-11-06T02:10:20.651 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.651+0000 7fa6cb521640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c40c92e0 con 0x7fa6c40adae0 2023-11-06T02:10:20.652 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.651+0000 7fa6c9abe640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6c40adae0 0x7fa6c4070b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:20.652 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.651+0000 7fa6c8abc640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fa6c40b0790 0x7fa6c40c8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:20.653 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.651+0000 7fa6c9abe640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6c40adae0 0x7fa6c4070b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.49:54654/0 (socket says 172.21.15.49:54654) 2023-11-06T02:10:20.653 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.651+0000 7fa6c9abe640 1 -- 172.21.15.49:0/389750178 learned_addr learned my addr 172.21.15.49:0/389750178 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:20.653 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.651+0000 7fa6c9abe640 1 -- 172.21.15.49:0/389750178 >> v2:172.21.15.150:3300/0 conn(0x7fa6c40b0790 msgr2=0x7fa6c40c8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:20.653 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.651+0000 7fa6c9abe640 1 --2- 172.21.15.49:0/389750178 >> v2:172.21.15.150:3300/0 conn(0x7fa6c40b0790 0x7fa6c40c8c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:20.653 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.651+0000 7fa6c9abe640 1 -- 172.21.15.49:0/389750178 >> v1:172.21.15.150:6789/0 conn(0x7fa6c41590f0 legacy=0x7fa6c40b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:20.653 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.651+0000 7fa6c9abe640 1 -- 172.21.15.49:0/389750178 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6c41597a0 con 0x7fa6c40adae0 2023-11-06T02:10:20.654 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.655+0000 7fa6c9abe640 1 --2- 172.21.15.49:0/389750178 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6c40adae0 0x7fa6c4070b20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc00af60 tx=0x7fa6bc01d9f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=249d7b77abb47fe6 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:20.654 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.655+0000 7fa6bbfff640 1 -- 172.21.15.49:0/389750178 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6bc00a2d0 con 0x7fa6c40adae0 2023-11-06T02:10:20.655 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.655+0000 7fa6bbfff640 1 -- 172.21.15.49:0/389750178 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6bc0a6650 con 0x7fa6c40adae0 2023-11-06T02:10:20.655 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.655+0000 7fa6bbfff640 1 -- 172.21.15.49:0/389750178 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6bc0a69e0 con 0x7fa6c40adae0 2023-11-06T02:10:20.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.655+0000 7fa6cb521640 1 -- 172.21.15.49:0/389750178 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6c40adae0 msgr2=0x7fa6c4070b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:20.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.655+0000 7fa6cb521640 1 --2- 172.21.15.49:0/389750178 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6c40adae0 0x7fa6c4070b20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc00af60 tx=0x7fa6bc01d9f0 comp rx=0 tx=0).stop 2023-11-06T02:10:20.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.655+0000 7fa6cb521640 1 -- 172.21.15.49:0/389750178 shutdown_connections 2023-11-06T02:10:20.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.655+0000 7fa6cb521640 1 --2- 172.21.15.49:0/389750178 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6c40adae0 0x7fa6c4070b20 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:20.657 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.655+0000 7fa6cb521640 1 --2- 172.21.15.49:0/389750178 >> v2:172.21.15.150:3300/0 conn(0x7fa6c40b0790 0x7fa6c40c8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:20.657 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.655+0000 7fa6cb521640 1 -- 172.21.15.49:0/389750178 >> 172.21.15.49:0/389750178 conn(0x7fa6c406c0a0 msgr2=0x7fa6c406c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:20.657 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.655+0000 7fa6cb521640 1 -- 172.21.15.49:0/389750178 shutdown_connections 2023-11-06T02:10:20.657 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.655+0000 7fa6cb521640 1 -- 172.21.15.49:0/389750178 wait complete. 2023-11-06T02:10:20.659 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.655+0000 7fa6cb521640 1 Processor -- start 2023-11-06T02:10:20.659 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.655+0000 7fa6cb521640 1 -- start start 2023-11-06T02:10:20.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.659+0000 7fa6cb521640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6c40adae0 0x7fa6c40adfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:20.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.659+0000 7fa6cb521640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa6c40b0790 0x7fa6c4143590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:20.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.659+0000 7fa6cb521640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6c41590f0 0x7fa6c40b0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:20.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.659+0000 7fa6cb521640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c4149e30 con 0x7fa6c41590f0 2023-11-06T02:10:20.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.659+0000 7fa6cb521640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c40ae6a0 con 0x7fa6c40b0790 2023-11-06T02:10:20.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.659+0000 7fa6cb521640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c40ae980 con 0x7fa6c40adae0 2023-11-06T02:10:20.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.659+0000 7fa6c9abe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6c41590f0 0x7fa6c40b0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:20.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.659+0000 7fa6c9abe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6c41590f0 0x7fa6c40b0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46224/0 (socket says 172.21.15.49:46224) 2023-11-06T02:10:20.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.659+0000 7fa6c9abe640 1 -- 172.21.15.49:0/1050509750 learned_addr learned my addr 172.21.15.49:0/1050509750 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:20.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.659+0000 7fa6c8abc640 1 --2- 172.21.15.49:0/1050509750 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa6c40b0790 0x7fa6c4143590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:20.662 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.659+0000 7fa6c92bd640 1 --2- 172.21.15.49:0/1050509750 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6c40adae0 0x7fa6c40adfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:20.662 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.659+0000 7fa6c9abe640 1 -- 172.21.15.49:0/1050509750 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6c40adae0 msgr2=0x7fa6c40adfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:20.662 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.659+0000 7fa6c9abe640 1 --2- 172.21.15.49:0/1050509750 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6c40adae0 0x7fa6c40adfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:20.662 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.659+0000 7fa6c9abe640 1 -- 172.21.15.49:0/1050509750 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa6c40b0790 msgr2=0x7fa6c4143590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:20.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.659+0000 7fa6c9abe640 1 --2- 172.21.15.49:0/1050509750 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa6c40b0790 0x7fa6c4143590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:20.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.659+0000 7fa6c9abe640 1 -- 172.21.15.49:0/1050509750 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6bc00ac40 con 0x7fa6c41590f0 2023-11-06T02:10:20.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.659+0000 7fa6c9abe640 1 --2- 172.21.15.49:0/1050509750 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6c41590f0 0x7fa6c40b0bb0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc0b0040 tx=0x7fa6bc01a790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:20.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.659+0000 7fa6b9ffb640 1 -- 172.21.15.49:0/1050509750 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6bc0a6d50 con 0x7fa6c41590f0 2023-11-06T02:10:20.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.659+0000 7fa6cb521640 1 -- 172.21.15.49:0/1050509750 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6c40aec60 con 0x7fa6c41590f0 2023-11-06T02:10:20.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.659+0000 7fa6b9ffb640 1 -- 172.21.15.49:0/1050509750 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6bc008500 con 0x7fa6c41590f0 2023-11-06T02:10:20.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.659+0000 7fa6b9ffb640 1 -- 172.21.15.49:0/1050509750 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6bc0088f0 con 0x7fa6c41590f0 2023-11-06T02:10:20.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.659+0000 7fa6cb521640 1 -- 172.21.15.49:0/1050509750 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6c40af220 con 0x7fa6c41590f0 2023-11-06T02:10:20.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.659+0000 7fa6cb521640 1 -- 172.21.15.49:0/1050509750 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6c40af410 con 0x7fa6c41590f0 2023-11-06T02:10:20.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.659+0000 7fa6b9ffb640 1 -- 172.21.15.49:0/1050509750 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fa6bc008a50 con 0x7fa6c41590f0 2023-11-06T02:10:20.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.659+0000 7fa6b9ffb640 1 --2- 172.21.15.49:0/1050509750 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa6a8065a60 0x7fa6a8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:20.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.659+0000 7fa6b9ffb640 1 -- 172.21.15.49:0/1050509750 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6091+0+0 (secure 0 0 0) 0x7fa6bc11f040 con 0x7fa6c41590f0 2023-11-06T02:10:20.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.663+0000 7fa6c92bd640 1 --2- 172.21.15.49:0/1050509750 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa6a8065a60 0x7fa6a8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:20.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.663+0000 7fa6c92bd640 1 --2- 172.21.15.49:0/1050509750 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa6a8065a60 0x7fa6a8067f20 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa6b4001600 tx=0x7fa6b4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:20.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.663+0000 7fa6b9ffb640 1 -- 172.21.15.49:0/1050509750 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fa6bc0ec910 con 0x7fa6c41590f0 2023-11-06T02:10:20.819 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.819+0000 7fa6cb521640 1 -- 172.21.15.49:0/1050509750 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa6c40aeac0 con 0x7fa6c41590f0 2023-11-06T02:10:20.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.823+0000 7fa6b9ffb640 1 -- 172.21.15.49:0/1050509750 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+17932 (secure 0 0 0) 0x7fa6c40aeac0 con 0x7fa6c41590f0 2023-11-06T02:10:20.822 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:10:20.822 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":39,"fsid":"1a8b035c-6ba4-499a-b876-aea47fd9c0d4","created":"2023-11-06T02:08:06.440857+0000","modified":"2023-11-06T02:10:14.167395+0000","last_up_change":"2023-11-06T02:08:17.982372+0000","last_in_change":"2023-11-06T02:08:08.906631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T02:08:19.252222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T02:08:24.776211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-06T02:09:37.906848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-06T02:09:38.849380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ad55af8-0571-4e6a-8722-b5d429b7496d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6809","nonce":38116},{"type":"v1","addr":"172.21.15.49:6811","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6813","nonce":38116},{"type":"v1","addr":"172.21.15.49:6815","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6821","nonce":38116},{"type":"v1","addr":"172.21.15.49:6823","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6817","nonce":38116},{"type":"v1","addr":"172.21.15.49:6819","nonce":38116}]},"public_addr":"172.21.15.49:6811/38116","cluster_addr":"172.21.15.49:6815/38116","heartbeat_back_addr":"172.21.15.49:6823/38116","heartbeat_front_addr":"172.21.15.49:6819/38116","state":["exists","up"]},{"osd":1,"uuid":"c8b31f85-a757-43ed-9419-351acc46611b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":38113},{"type":"v1","addr":"172.21.15.49:6810","nonce":38113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":38113},{"type":"v1","addr":"172.21.15.49:6814","nonce":38113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":38113},{"type":"v1","addr":"172.21.15.49:6822","nonce":38113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":38113},{"type":"v1","addr":"172.21.15.49:6818","nonce":38113}]},"public_addr":"172.21.15.49:6810/38113","cluster_addr":"172.21.15.49:6814/38113","heartbeat_back_addr":"172.21.15.49:6822/38113","heartbeat_front_addr":"172.21.15.49:6818/38113","state":["exists","up"]},{"osd":2,"uuid":"555b3bf3-0a70-4b91-8efd-e4204b5fc26e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":38114},{"type":"v1","addr":"172.21.15.49:6801","nonce":38114}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":38114},{"type":"v1","addr":"172.21.15.49:6803","nonce":38114}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":38114},{"type":"v1","addr":"172.21.15.49:6807","nonce":38114}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":38114},{"type":"v1","addr":"172.21.15.49:6805","nonce":38114}]},"public_addr":"172.21.15.49:6801/38114","cluster_addr":"172.21.15.49:6803/38114","heartbeat_back_addr":"172.21.15.49:6807/38114","heartbeat_front_addr":"172.21.15.49:6805/38114","state":["exists","up"]},{"osd":3,"uuid":"8d4e5d77-03ef-4b9b-bfcb-5ae93a46795c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":38115},{"type":"v1","addr":"172.21.15.49:6825","nonce":38115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":38115},{"type":"v1","addr":"172.21.15.49:6827","nonce":38115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":38115},{"type":"v1","addr":"172.21.15.49:6831","nonce":38115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":38115},{"type":"v1","addr":"172.21.15.49:6829","nonce":38115}]},"public_addr":"172.21.15.49:6825/38115","cluster_addr":"172.21.15.49:6827/38115","heartbeat_back_addr":"172.21.15.49:6831/38115","heartbeat_front_addr":"172.21.15.49:6829/38115","state":["exists","up"]},{"osd":4,"uuid":"dba03ad4-68a3-4cf3-9218-ac611f02eacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":38417},{"type":"v1","addr":"172.21.15.150:6809","nonce":38417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":38417},{"type":"v1","addr":"172.21.15.150:6811","nonce":38417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":38417},{"type":"v1","addr":"172.21.15.150:6815","nonce":38417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":38417},{"type":"v1","addr":"172.21.15.150:6813","nonce":38417}]},"public_addr":"172.21.15.150:6809/38417","cluster_addr":"172.21.15.150:6811/38417","heartbeat_back_addr":"172.21.15.150:6815/38417","heartbeat_front_addr":"172.21.15.150:6813/38417","state":["exists","up"]},{"osd":5,"uuid":"0df9c303-c255-4640-8e91-01f1c3efe18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":38418},{"type":"v1","addr":"172.21.15.150:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":38418},{"type":"v1","addr":"172.21.15.150:6803","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":38418},{"type":"v1","addr":"172.21.15.150:6807","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":38418},{"type":"v1","addr":"172.21.15.150:6805","nonce":38418}]},"public_addr":"172.21.15.150:6801/38418","cluster_addr":"172.21.15.150:6803/38418","heartbeat_back_addr":"172.21.15.150:6807/38418","heartbeat_front_addr":"172.21.15.150:6805/38418","state":["exists","up"]},{"osd":6,"uuid":"e59fa189-4b70-4c06-9722-c02b90828bf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":38428},{"type":"v1","addr":"172.21.15.150:6825","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":38428},{"type":"v1","addr":"172.21.15.150:6827","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":38428},{"type":"v1","addr":"172.21.15.150:6831","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":38428},{"type":"v1","addr":"172.21.15.150:6829","nonce":38428}]},"public_addr":"172.21.15.150:6825/38428","cluster_addr":"172.21.15.150:6827/38428","heartbeat_back_addr":"172.21.15.150:6831/38428","heartbeat_front_addr":"172.21.15.150:6829/38428","state":["exists","up"]},{"osd":7,"uuid":"a64f57e1-ef98-46bd-b304-385759e95898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":38421},{"type":"v1","addr":"172.21.15.150:6817","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":38421},{"type":"v1","addr":"172.21.15.150:6819","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":38421},{"type":"v1","addr":"172.21.15.150:6823","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":38421},{"type":"v1","addr":"172.21.15.150:6821","nonce":38421}]},"public_addr":"172.21.15.150:6817/38421","cluster_addr":"172.21.15.150:6819/38421","heartbeat_back_addr":"172.21.15.150:6823/38421","heartbeat_front_addr":"172.21.15.150:6821/38421","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.438552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.413280+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.296466+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.424966+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.276953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.230984+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.410235+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.480084+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.3d","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T02:10:20.824 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.823+0000 7fa6cb521640 1 -- 172.21.15.49:0/1050509750 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa6a8065a60 msgr2=0x7fa6a8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:20.824 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.823+0000 7fa6cb521640 1 --2- 172.21.15.49:0/1050509750 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa6a8065a60 0x7fa6a8067f20 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa6b4001600 tx=0x7fa6b4009040 comp rx=0 tx=0).stop 2023-11-06T02:10:20.824 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.823+0000 7fa6cb521640 1 -- 172.21.15.49:0/1050509750 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6c41590f0 msgr2=0x7fa6c40b0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:20.825 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.823+0000 7fa6cb521640 1 --2- 172.21.15.49:0/1050509750 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6c41590f0 0x7fa6c40b0bb0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc0b0040 tx=0x7fa6bc01a790 comp rx=0 tx=0).stop 2023-11-06T02:10:20.825 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.823+0000 7fa6cb521640 1 -- 172.21.15.49:0/1050509750 shutdown_connections 2023-11-06T02:10:20.825 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.823+0000 7fa6cb521640 1 --2- 172.21.15.49:0/1050509750 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa6a8065a60 0x7fa6a8067f20 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:20.825 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.823+0000 7fa6cb521640 1 --2- 172.21.15.49:0/1050509750 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6c41590f0 0x7fa6c40b0bb0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:20.825 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.823+0000 7fa6cb521640 1 --2- 172.21.15.49:0/1050509750 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa6c40b0790 0x7fa6c4143590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:20.826 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.823+0000 7fa6cb521640 1 --2- 172.21.15.49:0/1050509750 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6c40adae0 0x7fa6c40adfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:20.826 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.823+0000 7fa6cb521640 1 -- 172.21.15.49:0/1050509750 >> 172.21.15.49:0/1050509750 conn(0x7fa6c406c0a0 msgr2=0x7fa6c40b3c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:20.826 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.823+0000 7fa6cb521640 1 -- 172.21.15.49:0/1050509750 shutdown_connections 2023-11-06T02:10:20.826 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.823+0000 7fa6cb521640 1 -- 172.21.15.49:0/1050509750 wait complete. 2023-11-06T02:10:20.841 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-06T02:10:20.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.955+0000 7fe23de0b640 1 Processor -- start 2023-11-06T02:10:20.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.955+0000 7fe23de0b640 1 -- start start 2023-11-06T02:10:20.957 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.955+0000 7fe23de0b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe23815ab50 0x7fe2380c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:20.957 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.955+0000 7fe23de0b640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe2380b0790 0x7fe2380c2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:20.957 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.955+0000 7fe23de0b640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2380cab80 con 0x7fe2380cb010 2023-11-06T02:10:20.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.955+0000 7fe23de0b640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe23815af30 con 0x7fe2380b0790 2023-11-06T02:10:20.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.955+0000 7fe23de0b640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2380cb5b0 con 0x7fe23815ab50 2023-11-06T02:10:20.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.955+0000 7fe237fff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe2380b0790 0x7fe2380c2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:20.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.955+0000 7fe236ffd640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fe2380cb010 0x7fe2380b0080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:59738/0 (socket says 172.21.15.49:59738) 2023-11-06T02:10:20.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.955+0000 7fe236ffd640 1 -- 172.21.15.49:0/3640323232 learned_addr learned my addr 172.21.15.49:0/3640323232 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:10:20.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.959+0000 7fe2377fe640 1 --2- 172.21.15.49:0/3640323232 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe23815ab50 0x7fe2380c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:20.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.959+0000 7fe237fff640 1 -- 172.21.15.49:0/3640323232 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe23815ab50 msgr2=0x7fe2380c9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:20.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.959+0000 7fe237fff640 1 --2- 172.21.15.49:0/3640323232 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe23815ab50 0x7fe2380c9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:20.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.959+0000 7fe237fff640 1 -- 172.21.15.49:0/3640323232 >> v1:172.21.15.49:6789/0 conn(0x7fe2380cb010 legacy=0x7fe2380b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:20.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.959+0000 7fe237fff640 1 -- 172.21.15.49:0/3640323232 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2380cb890 con 0x7fe2380b0790 2023-11-06T02:10:20.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.959+0000 7fe2367fc640 1 -- 172.21.15.49:0/3640323232 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4089465347 0 0) 0x7fe2380cab80 con 0x7fe2380cb010 2023-11-06T02:10:20.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.959+0000 7fe237fff640 1 --2- 172.21.15.49:0/3640323232 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2380b0790 0x7fe2380c2710 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fe230009d90 tx=0x7fe23001c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6814abbd29dd0ea3 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:20.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.959+0000 7fe2367fc640 1 -- 172.21.15.49:0/3640323232 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2300a59e0 con 0x7fe2380b0790 2023-11-06T02:10:20.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.959+0000 7fe2367fc640 1 -- 172.21.15.49:0/3640323232 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2300a5b40 con 0x7fe2380b0790 2023-11-06T02:10:20.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.959+0000 7fe2367fc640 1 -- 172.21.15.49:0/3640323232 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2300ad440 con 0x7fe2380b0790 2023-11-06T02:10:20.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.959+0000 7fe23de0b640 1 -- 172.21.15.49:0/3640323232 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2380b0790 msgr2=0x7fe2380c2710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:20.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.959+0000 7fe23de0b640 1 --2- 172.21.15.49:0/3640323232 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2380b0790 0x7fe2380c2710 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fe230009d90 tx=0x7fe23001c920 comp rx=0 tx=0).stop 2023-11-06T02:10:20.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.959+0000 7fe23de0b640 1 -- 172.21.15.49:0/3640323232 shutdown_connections 2023-11-06T02:10:20.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.959+0000 7fe23de0b640 1 --2- 172.21.15.49:0/3640323232 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2380b0790 0x7fe2380c2710 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:20.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.959+0000 7fe23de0b640 1 --2- 172.21.15.49:0/3640323232 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe23815ab50 0x7fe2380c9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:20.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.959+0000 7fe23de0b640 1 -- 172.21.15.49:0/3640323232 >> 172.21.15.49:0/3640323232 conn(0x7fe23806c0a0 msgr2=0x7fe23806c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:20.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.959+0000 7fe23de0b640 1 -- 172.21.15.49:0/3640323232 shutdown_connections 2023-11-06T02:10:20.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.959+0000 7fe23de0b640 1 -- 172.21.15.49:0/3640323232 wait complete. 2023-11-06T02:10:20.966 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.963+0000 7fe23de0b640 1 Processor -- start 2023-11-06T02:10:20.966 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.963+0000 7fe23de0b640 1 -- start start 2023-11-06T02:10:20.966 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.963+0000 7fe23de0b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe2380b0790 0x7fe2381495f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:20.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.963+0000 7fe23de0b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2380cb010 0x7fe2380ae060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:20.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.963+0000 7fe23de0b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe23815ab50 0x7fe2380ae5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:20.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.963+0000 7fe23de0b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2380c6dd0 con 0x7fe2380cb010 2023-11-06T02:10:20.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.963+0000 7fe23de0b640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2380c70b0 con 0x7fe23815ab50 2023-11-06T02:10:20.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.963+0000 7fe23de0b640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2381578a0 con 0x7fe2380b0790 2023-11-06T02:10:20.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.963+0000 7fe236ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2380cb010 0x7fe2380ae060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:20.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.963+0000 7fe236ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2380cb010 0x7fe2380ae060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46246/0 (socket says 172.21.15.49:46246) 2023-11-06T02:10:20.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.963+0000 7fe236ffd640 1 -- 172.21.15.49:0/3408063872 learned_addr learned my addr 172.21.15.49:0/3408063872 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:20.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.963+0000 7fe2377fe640 1 --2- 172.21.15.49:0/3408063872 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe2380b0790 0x7fe2381495f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:20.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.963+0000 7fe237fff640 1 --2- 172.21.15.49:0/3408063872 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe23815ab50 0x7fe2380ae5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:20.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.963+0000 7fe236ffd640 1 -- 172.21.15.49:0/3408063872 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe2380b0790 msgr2=0x7fe2381495f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:20.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.963+0000 7fe236ffd640 1 --2- 172.21.15.49:0/3408063872 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe2380b0790 0x7fe2381495f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:20.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.963+0000 7fe236ffd640 1 -- 172.21.15.49:0/3408063872 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe23815ab50 msgr2=0x7fe2380ae5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:20.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.963+0000 7fe236ffd640 1 --2- 172.21.15.49:0/3408063872 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe23815ab50 0x7fe2380ae5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:20.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.963+0000 7fe236ffd640 1 -- 172.21.15.49:0/3408063872 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe230009a70 con 0x7fe2380cb010 2023-11-06T02:10:20.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.963+0000 7fe236ffd640 1 --2- 172.21.15.49:0/3408063872 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2380cb010 0x7fe2380ae060 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fe22c0011f0 tx=0x7fe22c00c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:20.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.963+0000 7fe21ffff640 1 -- 172.21.15.49:0/3408063872 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe22c0074e0 con 0x7fe2380cb010 2023-11-06T02:10:20.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.963+0000 7fe21ffff640 1 -- 172.21.15.49:0/3408063872 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe22c007640 con 0x7fe2380cb010 2023-11-06T02:10:20.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.963+0000 7fe23de0b640 1 -- 172.21.15.49:0/3408063872 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe238157b80 con 0x7fe2380cb010 2023-11-06T02:10:20.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.963+0000 7fe21ffff640 1 -- 172.21.15.49:0/3408063872 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe22c004a10 con 0x7fe2380cb010 2023-11-06T02:10:20.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.963+0000 7fe23de0b640 1 -- 172.21.15.49:0/3408063872 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe238149b30 con 0x7fe2380cb010 2023-11-06T02:10:20.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.963+0000 7fe23de0b640 1 -- 172.21.15.49:0/3408063872 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2380b0de0 con 0x7fe2380cb010 2023-11-06T02:10:20.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.967+0000 7fe21ffff640 1 -- 172.21.15.49:0/3408063872 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fe22c004b70 con 0x7fe2380cb010 2023-11-06T02:10:20.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.967+0000 7fe21ffff640 1 --2- 172.21.15.49:0/3408063872 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe224065a60 0x7fe224067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:20.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.967+0000 7fe21ffff640 1 -- 172.21.15.49:0/3408063872 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6091+0+0 (secure 0 0 0) 0x7fe22c082830 con 0x7fe2380cb010 2023-11-06T02:10:20.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.967+0000 7fe2377fe640 1 --2- 172.21.15.49:0/3408063872 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe224065a60 0x7fe224067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:20.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.967+0000 7fe2377fe640 1 --2- 172.21.15.49:0/3408063872 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe224065a60 0x7fe224067f20 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fe228006fd0 tx=0x7fe228008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:20.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:20.971+0000 7fe21ffff640 1 -- 172.21.15.49:0/3408063872 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fe22c050070 con 0x7fe2380cb010 2023-11-06T02:10:21.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:21.151+0000 7fe23de0b640 1 -- 172.21.15.49:0/3408063872 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fe238074040 con 0x7fe2380cb010 2023-11-06T02:10:21.152 INFO:tasks.ceph.mon.a.smithi049.stderr:2023-11-06T02:10:21.151+0000 7f0164e5a640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-06T02:10:21.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:21.167+0000 7fe21ffff640 1 -- 172.21.15.49:0/3408063872 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v16) v1 ==== 161+0+0 (secure 0 0 0) 0x7fe22c018090 con 0x7fe2380cb010 2023-11-06T02:10:21.168 INFO:teuthology.orchestra.run.smithi049.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-06T02:10:21.170 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:21.171+0000 7fe23de0b640 1 -- 172.21.15.49:0/3408063872 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe224065a60 msgr2=0x7fe224067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:21.170 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:21.171+0000 7fe23de0b640 1 --2- 172.21.15.49:0/3408063872 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe224065a60 0x7fe224067f20 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fe228006fd0 tx=0x7fe228008040 comp rx=0 tx=0).stop 2023-11-06T02:10:21.171 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:21.171+0000 7fe23de0b640 1 -- 172.21.15.49:0/3408063872 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2380cb010 msgr2=0x7fe2380ae060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:21.171 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:21.171+0000 7fe23de0b640 1 --2- 172.21.15.49:0/3408063872 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2380cb010 0x7fe2380ae060 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fe22c0011f0 tx=0x7fe22c00c9b0 comp rx=0 tx=0).stop 2023-11-06T02:10:21.171 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:21.171+0000 7fe23de0b640 1 -- 172.21.15.49:0/3408063872 shutdown_connections 2023-11-06T02:10:21.171 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:21.171+0000 7fe23de0b640 1 --2- 172.21.15.49:0/3408063872 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe224065a60 0x7fe224067f20 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:21.171 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:21.171+0000 7fe23de0b640 1 --2- 172.21.15.49:0/3408063872 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2380cb010 0x7fe2380ae060 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:21.172 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:21.171+0000 7fe23de0b640 1 --2- 172.21.15.49:0/3408063872 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe23815ab50 0x7fe2380ae5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:21.172 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:21.171+0000 7fe23de0b640 1 --2- 172.21.15.49:0/3408063872 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe2380b0790 0x7fe2381495f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:21.172 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:21.171+0000 7fe23de0b640 1 -- 172.21.15.49:0/3408063872 >> 172.21.15.49:0/3408063872 conn(0x7fe23806c0a0 msgr2=0x7fe2381566d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:21.172 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:21.171+0000 7fe23de0b640 1 -- 172.21.15.49:0/3408063872 shutdown_connections 2023-11-06T02:10:21.172 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:21.171+0000 7fe23de0b640 1 -- 172.21.15.49:0/3408063872 wait complete. 2023-11-06T02:10:21.178 INFO:tasks.ceph.mds.a.smithi049.stderr: -5875> 2023-11-06T02:09:43.448+0000 7f5fe4f76640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-06T02:10:21.186 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-06T02:10:21.241 INFO:tasks.ceph.mds.a.smithi049.stdout:starting mds.a at 2023-11-06T02:10:21.311 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:21.311+0000 7f3abc3e8640 1 Processor -- start 2023-11-06T02:10:21.312 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:21.311+0000 7f3abc3e8640 1 -- start start 2023-11-06T02:10:21.313 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:21.311+0000 7f3abc3e8640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f3ab415ab60 0x7f3ab40ad520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:21.313 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:21.311+0000 7f3abc3e8640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f3ab40ada60 0x7f3ab40d08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:21.314 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:21.311+0000 7f3abc3e8640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ab40d1240 con 0x7f3ab40d0df0 2023-11-06T02:10:21.314 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:21.311+0000 7f3abc3e8640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ab40cb340 con 0x7f3ab40ada60 2023-11-06T02:10:21.314 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:21.311+0000 7f3abc3e8640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ab40adf50 con 0x7f3ab415ab60 2023-11-06T02:10:21.314 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:21.311+0000 7f3ab9983640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f3ab40ada60 0x7f3ab40d08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:21.315 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:21.311+0000 7f3ab9983640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f3ab40ada60 0x7f3ab40d08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46288/0 (socket says 172.21.15.49:46288) 2023-11-06T02:10:21.315 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:21.311+0000 7f3aba985640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f3ab40d0df0 0x7f3ab40b08d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:59748/0 (socket says 172.21.15.49:59748) 2023-11-06T02:10:21.315 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:21.315+0000 7f3ab9983640 1 -- 172.21.15.49:0/3534586711 learned_addr learned my addr 172.21.15.49:0/3534586711 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:21.315 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:21.315+0000 7f3aba184640 1 --2- 172.21.15.49:0/3534586711 >> v2:172.21.15.150:3300/0 conn(0x7f3ab415ab60 0x7f3ab40ad520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:21.315 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:21.315+0000 7f3ab9983640 1 -- 172.21.15.49:0/3534586711 >> v2:172.21.15.150:3300/0 conn(0x7f3ab415ab60 msgr2=0x7f3ab40ad520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:21.316 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:21.315+0000 7f3ab9983640 1 --2- 172.21.15.49:0/3534586711 >> v2:172.21.15.150:3300/0 conn(0x7f3ab415ab60 0x7f3ab40ad520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:21.316 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:21.315+0000 7f3ab9983640 1 -- 172.21.15.49:0/3534586711 >> v1:172.21.15.49:6789/0 conn(0x7f3ab40d0df0 legacy=0x7f3ab40b08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:21.316 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:21.315+0000 7f3ab9983640 1 -- 172.21.15.49:0/3534586711 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ab40ae230 con 0x7f3ab40ada60 2023-11-06T02:10:21.316 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:21.315+0000 7f3ab9182640 1 -- 172.21.15.49:0/3534586711 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1846281534 0 0) 0x7f3aac0014a0 con 0x7f3ab40d0df0 2023-11-06T02:10:21.319 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:21.315+0000 7f3ab9983640 1 --2- 172.21.15.49:0/3534586711 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ab40ada60 0x7f3ab40d08b0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f3ab40ce420 tx=0x7f3aa401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=960481852b972684 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:21.319 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:21.315+0000 7f3ab9182640 1 -- 172.21.15.49:0/3534586711 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa40aa430 con 0x7f3ab40ada60 2023-11-06T02:10:21.320 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:21.315+0000 7f3ab9182640 1 -- 172.21.15.49:0/3534586711 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3aa40aa590 con 0x7f3ab40ada60 2023-11-06T02:10:21.320 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:21.315+0000 7f3ab9182640 1 -- 172.21.15.49:0/3534586711 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa40aa920 con 0x7f3ab40ada60 2023-11-06T02:10:21.320 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:21.315+0000 7f3abc3e8640 1 -- 172.21.15.49:0/3534586711 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ab40ada60 msgr2=0x7f3ab40d08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:21.320 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:21.315+0000 7f3abc3e8640 1 --2- 172.21.15.49:0/3534586711 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ab40ada60 0x7f3ab40d08b0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f3ab40ce420 tx=0x7f3aa401c4d0 comp rx=0 tx=0).stop 2023-11-06T02:10:21.320 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:21.315+0000 7f3abc3e8640 1 -- 172.21.15.49:0/3534586711 shutdown_connections 2023-11-06T02:10:21.321 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:21.315+0000 7f3abc3e8640 1 --2- 172.21.15.49:0/3534586711 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ab40ada60 0x7f3ab40d08b0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:21.321 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:21.315+0000 7f3abc3e8640 1 --2- 172.21.15.49:0/3534586711 >> v2:172.21.15.150:3300/0 conn(0x7f3ab415ab60 0x7f3ab40ad520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:21.321 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:21.315+0000 7f3abc3e8640 1 -- 172.21.15.49:0/3534586711 >> 172.21.15.49:0/3534586711 conn(0x7f3ab4089fd0 msgr2=0x7f3ab4099c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:21.321 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:21.315+0000 7f3abc3e8640 1 -- 172.21.15.49:0/3534586711 shutdown_connections 2023-11-06T02:10:21.322 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:21.315+0000 7f3abc3e8640 1 -- 172.21.15.49:0/3534586711 wait complete. 2023-11-06T02:10:21.322 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:21.319+0000 7f3abc3e8640 1 Processor -- start 2023-11-06T02:10:21.322 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:21.319+0000 7f3abc3e8640 1 -- start start 2023-11-06T02:10:21.322 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:21.319+0000 7f3abc3e8640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3ab40ada60 0x7f3ab414db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:21.322 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:21.319+0000 7f3abc3e8640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ab40d0df0 0x7f3ab414cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:21.323 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:21.319+0000 7f3abc3e8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ab415da70 0x7f3ab414d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:21.323 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:21.319+0000 7f3abc3e8640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab41580b0 con 0x7f3ab415da70 2023-11-06T02:10:21.323 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:21.319+0000 7f3abc3e8640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab414d760 con 0x7f3ab40d0df0 2023-11-06T02:10:21.323 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:21.319+0000 7f3abc3e8640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab4158300 con 0x7f3ab40ada60 2023-11-06T02:10:21.323 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:21.319+0000 7f3aba985640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ab415da70 0x7f3ab414d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:21.324 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:21.319+0000 7f3aba985640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ab415da70 0x7f3ab414d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46292/0 (socket says 172.21.15.49:46292) 2023-11-06T02:10:21.324 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:21.319+0000 7f3aba985640 1 -- 172.21.15.49:0/648970731 learned_addr learned my addr 172.21.15.49:0/648970731 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:21.324 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:21.319+0000 7f3ab9983640 1 --2- 172.21.15.49:0/648970731 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ab40d0df0 0x7f3ab414cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:21.324 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:21.319+0000 7f3aba184640 1 --2- 172.21.15.49:0/648970731 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3ab40ada60 0x7f3ab414db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:21.324 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:21.319+0000 7f3aba985640 1 -- 172.21.15.49:0/648970731 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3ab40ada60 msgr2=0x7f3ab414db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:21.325 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:21.319+0000 7f3aba985640 1 --2- 172.21.15.49:0/648970731 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3ab40ada60 0x7f3ab414db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:21.325 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:21.319+0000 7f3aba985640 1 -- 172.21.15.49:0/648970731 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ab40d0df0 msgr2=0x7f3ab414cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:21.325 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:21.319+0000 7f3aba985640 1 --2- 172.21.15.49:0/648970731 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ab40d0df0 0x7f3ab414cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:21.325 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:21.319+0000 7f3aba985640 1 -- 172.21.15.49:0/648970731 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aa40094d0 con 0x7f3ab415da70 2023-11-06T02:10:21.326 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:21.319+0000 7f3aba184640 1 --2- 172.21.15.49:0/648970731 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3ab40ada60 0x7f3ab414db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:10:21.326 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:21.319+0000 7f3aba985640 1 --2- 172.21.15.49:0/648970731 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ab415da70 0x7f3ab414d220 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f3aac00c810 tx=0x7f3aac00ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:21.326 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:21.319+0000 7f3aaaffd640 1 -- 172.21.15.49:0/648970731 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aac00a640 con 0x7f3ab415da70 2023-11-06T02:10:21.326 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:21.319+0000 7f3aaaffd640 1 -- 172.21.15.49:0/648970731 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3aac00a7a0 con 0x7f3ab415da70 2023-11-06T02:10:21.326 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:21.319+0000 7f3abc3e8640 1 -- 172.21.15.49:0/648970731 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ab41585e0 con 0x7f3ab415da70 2023-11-06T02:10:21.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:21.319+0000 7f3aaaffd640 1 -- 172.21.15.49:0/648970731 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aac00ab50 con 0x7f3ab415da70 2023-11-06T02:10:21.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:21.319+0000 7f3abc3e8640 1 -- 172.21.15.49:0/648970731 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ab4158ba0 con 0x7f3ab415da70 2023-11-06T02:10:21.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:21.319+0000 7f3abc3e8640 1 -- 172.21.15.49:0/648970731 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ab4158d90 con 0x7f3ab415da70 2023-11-06T02:10:21.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:21.319+0000 7f3aaaffd640 1 -- 172.21.15.49:0/648970731 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f3aac00acb0 con 0x7f3ab415da70 2023-11-06T02:10:21.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:21.319+0000 7f3aaaffd640 1 --2- 172.21.15.49:0/648970731 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3a98065a60 0x7f3a98067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:21.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:21.323+0000 7f3aaaffd640 1 -- 172.21.15.49:0/648970731 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6177+0+0 (secure 0 0 0) 0x7f3aac0829d0 con 0x7f3ab415da70 2023-11-06T02:10:21.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:21.323+0000 7f3aba184640 1 --2- 172.21.15.49:0/648970731 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3a98065a60 0x7f3a98067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:21.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:21.323+0000 7f3aba184640 1 --2- 172.21.15.49:0/648970731 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3a98065a60 0x7f3a98067f20 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3ab00097a0 tx=0x7f3ab0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:21.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:21.323+0000 7f3aaaffd640 1 -- 172.21.15.49:0/648970731 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f3aac0501b0 con 0x7f3ab415da70 2023-11-06T02:10:21.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:21.515+0000 7f3abc3e8640 1 -- 172.21.15.49:0/648970731 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f3ab40d1240 con 0x7f3ab415da70 2023-11-06T02:10:21.613 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T02:10:21.616 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T02:10:22.183 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:22.183+0000 7f3aaaffd640 1 -- 172.21.15.49:0/648970731 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3ab40d1240 con 0x7f3ab415da70 2023-11-06T02:10:22.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:22.187+0000 7f3abc3e8640 1 -- 172.21.15.49:0/648970731 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3a98065a60 msgr2=0x7f3a98067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:22.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:22.187+0000 7f3abc3e8640 1 --2- 172.21.15.49:0/648970731 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3a98065a60 0x7f3a98067f20 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3ab00097a0 tx=0x7f3ab0008040 comp rx=0 tx=0).stop 2023-11-06T02:10:22.188 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:22.187+0000 7f3abc3e8640 1 -- 172.21.15.49:0/648970731 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ab415da70 msgr2=0x7f3ab414d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:22.188 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:22.187+0000 7f3abc3e8640 1 --2- 172.21.15.49:0/648970731 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ab415da70 0x7f3ab414d220 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f3aac00c810 tx=0x7f3aac00ccd0 comp rx=0 tx=0).stop 2023-11-06T02:10:22.188 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:22.187+0000 7f3abc3e8640 1 -- 172.21.15.49:0/648970731 shutdown_connections 2023-11-06T02:10:22.188 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:22.187+0000 7f3abc3e8640 1 --2- 172.21.15.49:0/648970731 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3a98065a60 0x7f3a98067f20 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:22.188 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:22.187+0000 7f3abc3e8640 1 --2- 172.21.15.49:0/648970731 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ab415da70 0x7f3ab414d220 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:22.189 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:22.187+0000 7f3abc3e8640 1 --2- 172.21.15.49:0/648970731 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ab40d0df0 0x7f3ab414cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:22.189 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:22.187+0000 7f3abc3e8640 1 --2- 172.21.15.49:0/648970731 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3ab40ada60 0x7f3ab414db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:22.189 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:22.187+0000 7f3abc3e8640 1 -- 172.21.15.49:0/648970731 >> 172.21.15.49:0/648970731 conn(0x7f3ab4089fd0 msgr2=0x7f3ab40cb480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:22.189 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:22.187+0000 7f3abc3e8640 1 -- 172.21.15.49:0/648970731 shutdown_connections 2023-11-06T02:10:22.189 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:22.187+0000 7f3abc3e8640 1 -- 172.21.15.49:0/648970731 wait complete. 2023-11-06T02:10:22.210 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-06T02:10:22.334 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:22.335+0000 7f98d8311640 1 Processor -- start 2023-11-06T02:10:22.335 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:22.335+0000 7f98d8311640 1 -- start start 2023-11-06T02:10:22.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:22.335+0000 7f98d8311640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f98d015cec0 0x7f98d00713d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:22.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:22.335+0000 7f98d8311640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f98d00cdf30 0x7f98d00c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:22.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:22.335+0000 7f98d8311640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98d00ccce0 con 0x7f98d0069d70 2023-11-06T02:10:22.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:22.335+0000 7f98d8311640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98d00ccfc0 con 0x7f98d00cdf30 2023-11-06T02:10:22.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:22.335+0000 7f98d8311640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98d0071e70 con 0x7f98d015cec0 2023-11-06T02:10:22.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:22.335+0000 7f98d58ac640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f98d00cdf30 0x7f98d00c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:22.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:22.335+0000 7f98d58ac640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f98d00cdf30 0x7f98d00c9ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46300/0 (socket says 172.21.15.49:46300) 2023-11-06T02:10:22.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:22.335+0000 7f98d58ac640 1 -- 172.21.15.49:0/3042868454 learned_addr learned my addr 172.21.15.49:0/3042868454 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:22.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:22.335+0000 7f98d60ad640 1 --2- 172.21.15.49:0/3042868454 >> v2:172.21.15.150:3300/0 conn(0x7f98d015cec0 0x7f98d00713d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:22.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:22.335+0000 7f98d58ac640 1 -- 172.21.15.49:0/3042868454 >> v2:172.21.15.150:3300/0 conn(0x7f98d015cec0 msgr2=0x7f98d00713d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:22.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:22.335+0000 7f98d58ac640 1 --2- 172.21.15.49:0/3042868454 >> v2:172.21.15.150:3300/0 conn(0x7f98d015cec0 0x7f98d00713d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:22.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:22.335+0000 7f98d58ac640 1 -- 172.21.15.49:0/3042868454 >> v1:172.21.15.150:6789/0 conn(0x7f98d0069d70 legacy=0x7f98d0160f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:22.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:22.335+0000 7f98d58ac640 1 -- 172.21.15.49:0/3042868454 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98d0072150 con 0x7f98d00cdf30 2023-11-06T02:10:22.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:22.339+0000 7f98d58ac640 1 --2- 172.21.15.49:0/3042868454 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98d00cdf30 0x7f98d00c9ed0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f98d0070860 tx=0x7f98c001c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c149aba692252862 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:22.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:22.339+0000 7f98d50ab640 1 -- 172.21.15.49:0/3042868454 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d0071cf0 con 0x7f98d00cdf30 2023-11-06T02:10:22.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:22.339+0000 7f98d50ab640 1 -- 172.21.15.49:0/3042868454 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98c00a7050 con 0x7f98d00cdf30 2023-11-06T02:10:22.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:22.339+0000 7f98d50ab640 1 -- 172.21.15.49:0/3042868454 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c00ab820 con 0x7f98d00cdf30 2023-11-06T02:10:22.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:22.339+0000 7f98d8311640 1 -- 172.21.15.49:0/3042868454 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98d00cdf30 msgr2=0x7f98d00c9ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:22.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:22.339+0000 7f98d8311640 1 --2- 172.21.15.49:0/3042868454 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98d00cdf30 0x7f98d00c9ed0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f98d0070860 tx=0x7f98c001c430 comp rx=0 tx=0).stop 2023-11-06T02:10:22.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:22.339+0000 7f98d8311640 1 -- 172.21.15.49:0/3042868454 shutdown_connections 2023-11-06T02:10:22.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:22.339+0000 7f98d8311640 1 --2- 172.21.15.49:0/3042868454 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98d00cdf30 0x7f98d00c9ed0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:22.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:22.339+0000 7f98d8311640 1 --2- 172.21.15.49:0/3042868454 >> v2:172.21.15.150:3300/0 conn(0x7f98d015cec0 0x7f98d00713d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:22.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:22.339+0000 7f98d8311640 1 -- 172.21.15.49:0/3042868454 >> 172.21.15.49:0/3042868454 conn(0x7f98d00cc1c0 msgr2=0x7f98d00cc5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:22.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:22.339+0000 7f98d8311640 1 -- 172.21.15.49:0/3042868454 shutdown_connections 2023-11-06T02:10:22.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:22.339+0000 7f98d8311640 1 -- 172.21.15.49:0/3042868454 wait complete. 2023-11-06T02:10:22.344 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:22.339+0000 7f98d8311640 1 Processor -- start 2023-11-06T02:10:22.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:22.339+0000 7f98d8311640 1 -- start start 2023-11-06T02:10:22.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:22.339+0000 7f98d8311640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f98d0069d70 0x7f98d00c5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:22.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:22.339+0000 7f98d8311640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98d00cdf30 0x7f98d015ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:22.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:22.339+0000 7f98d8311640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98d015cec0 0x7f98d00af2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:22.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:22.339+0000 7f98d8311640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98d0155a00 con 0x7f98d00cdf30 2023-11-06T02:10:22.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:22.339+0000 7f98d8311640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98d00c5f90 con 0x7f98d015cec0 2023-11-06T02:10:22.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:22.339+0000 7f98d8311640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98d0154d80 con 0x7f98d0069d70 2023-11-06T02:10:22.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:22.343+0000 7f98d58ac640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98d00cdf30 0x7f98d015ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:22.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:22.343+0000 7f98d58ac640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98d00cdf30 0x7f98d015ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46312/0 (socket says 172.21.15.49:46312) 2023-11-06T02:10:22.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:22.343+0000 7f98d58ac640 1 -- 172.21.15.49:0/531916259 learned_addr learned my addr 172.21.15.49:0/531916259 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:22.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:22.343+0000 7f98d68ae640 1 --2- 172.21.15.49:0/531916259 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98d015cec0 0x7f98d00af2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:22.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:22.343+0000 7f98d60ad640 1 --2- 172.21.15.49:0/531916259 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f98d0069d70 0x7f98d00c5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:22.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:22.343+0000 7f98d58ac640 1 -- 172.21.15.49:0/531916259 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f98d0069d70 msgr2=0x7f98d00c5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:22.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:22.343+0000 7f98d58ac640 1 --2- 172.21.15.49:0/531916259 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f98d0069d70 0x7f98d00c5a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:22.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:22.343+0000 7f98d58ac640 1 -- 172.21.15.49:0/531916259 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98d015cec0 msgr2=0x7f98d00af2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:22.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:22.343+0000 7f98d58ac640 1 --2- 172.21.15.49:0/531916259 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98d015cec0 0x7f98d00af2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:22.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:22.343+0000 7f98d58ac640 1 -- 172.21.15.49:0/531916259 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98c00094d0 con 0x7f98d00cdf30 2023-11-06T02:10:22.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:22.343+0000 7f98d68ae640 1 --2- 172.21.15.49:0/531916259 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98d015cec0 0x7f98d00af2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:10:22.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:22.343+0000 7f98d58ac640 1 --2- 172.21.15.49:0/531916259 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98d00cdf30 0x7f98d015ac20 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f98c0000c10 tx=0x7f98c001ac70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:22.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:22.343+0000 7f98c6ffd640 1 -- 172.21.15.49:0/531916259 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c00b6420 con 0x7f98d00cdf30 2023-11-06T02:10:22.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:22.343+0000 7f98c6ffd640 1 -- 172.21.15.49:0/531916259 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98c00b6580 con 0x7f98d00cdf30 2023-11-06T02:10:22.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:22.343+0000 7f98d8311640 1 -- 172.21.15.49:0/531916259 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98d0155060 con 0x7f98d00cdf30 2023-11-06T02:10:22.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:22.343+0000 7f98c6ffd640 1 -- 172.21.15.49:0/531916259 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c00b6910 con 0x7f98d00cdf30 2023-11-06T02:10:22.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:22.343+0000 7f98d8311640 1 -- 172.21.15.49:0/531916259 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98d0155620 con 0x7f98d00cdf30 2023-11-06T02:10:22.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:22.343+0000 7f98d8311640 1 -- 172.21.15.49:0/531916259 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98d0155b40 con 0x7f98d00cdf30 2023-11-06T02:10:22.350 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:22.343+0000 7f98c6ffd640 1 -- 172.21.15.49:0/531916259 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f98c00b2040 con 0x7f98d00cdf30 2023-11-06T02:10:22.350 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:22.343+0000 7f98c6ffd640 1 --2- 172.21.15.49:0/531916259 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f98b4065a60 0x7f98b4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:22.350 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:22.343+0000 7f98c6ffd640 1 -- 172.21.15.49:0/531916259 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6177+0+0 (secure 0 0 0) 0x7f98c011dac0 con 0x7f98d00cdf30 2023-11-06T02:10:22.350 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:22.343+0000 7f98d60ad640 1 --2- 172.21.15.49:0/531916259 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f98b4065a60 0x7f98b4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:22.350 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:22.343+0000 7f98d60ad640 1 --2- 172.21.15.49:0/531916259 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f98b4065a60 0x7f98b4067f20 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f98d01f8b60 tx=0x7f98c8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:22.351 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:22.347+0000 7f98c6ffd640 1 -- 172.21.15.49:0/531916259 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f98d0155b40 con 0x7f98d00cdf30 2023-11-06T02:10:22.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:22.507+0000 7f98d8311640 1 -- 172.21.15.49:0/531916259 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f98d0155b40 con 0x7f98d00cdf30 2023-11-06T02:10:23.300 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:23.299+0000 7f98c6ffd640 1 -- 172.21.15.49:0/531916259 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v41) v1 ==== 181+0+0 (secure 0 0 0) 0x7f98d0155b40 con 0x7f98d00cdf30 2023-11-06T02:10:23.301 INFO:teuthology.orchestra.run.smithi049.stderr:pool 'cephfs_metadata' removed 2023-11-06T02:10:23.305 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:23.303+0000 7f98d8311640 1 -- 172.21.15.49:0/531916259 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f98b4065a60 msgr2=0x7f98b4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:23.306 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:23.303+0000 7f98d8311640 1 --2- 172.21.15.49:0/531916259 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f98b4065a60 0x7f98b4067f20 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f98d01f8b60 tx=0x7f98c8008040 comp rx=0 tx=0).stop 2023-11-06T02:10:23.306 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:23.303+0000 7f98d8311640 1 -- 172.21.15.49:0/531916259 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98d00cdf30 msgr2=0x7f98d015ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:23.306 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:23.303+0000 7f98d8311640 1 --2- 172.21.15.49:0/531916259 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98d00cdf30 0x7f98d015ac20 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f98c0000c10 tx=0x7f98c001ac70 comp rx=0 tx=0).stop 2023-11-06T02:10:23.306 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:23.307+0000 7f98d8311640 1 -- 172.21.15.49:0/531916259 shutdown_connections 2023-11-06T02:10:23.306 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:23.307+0000 7f98d8311640 1 --2- 172.21.15.49:0/531916259 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f98b4065a60 0x7f98b4067f20 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:23.307 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:23.307+0000 7f98d8311640 1 --2- 172.21.15.49:0/531916259 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98d00cdf30 0x7f98d015ac20 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:23.307 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:23.307+0000 7f98d8311640 1 --2- 172.21.15.49:0/531916259 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98d015cec0 0x7f98d00af2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:23.307 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:23.307+0000 7f98d8311640 1 --2- 172.21.15.49:0/531916259 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f98d0069d70 0x7f98d00c5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:23.307 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:23.307+0000 7f98d8311640 1 -- 172.21.15.49:0/531916259 >> 172.21.15.49:0/531916259 conn(0x7f98d00cc1c0 msgr2=0x7f98d015fe70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:23.307 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:23.307+0000 7f98d8311640 1 -- 172.21.15.49:0/531916259 shutdown_connections 2023-11-06T02:10:23.308 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:23.307+0000 7f98d8311640 1 -- 172.21.15.49:0/531916259 wait complete. 2023-11-06T02:10:23.326 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-06T02:10:23.458 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:23.459+0000 7f54d1c03640 1 Processor -- start 2023-11-06T02:10:23.459 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:23.459+0000 7f54d1c03640 1 -- start start 2023-11-06T02:10:23.460 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:23.459+0000 7f54d1c03640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f54cc0714a0 0x7f54cc0cd060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:23.460 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:23.459+0000 7f54d1c03640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f54cc0d25c0 0x7f54cc15d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:23.461 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:23.459+0000 7f54d1c03640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f54cc071880 0x7f54cc0b4520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:23.461 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:23.459+0000 7f54d1c03640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54cc15d830 con 0x7f54cc0714a0 2023-11-06T02:10:23.461 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:23.459+0000 7f54d1c03640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54cc15da20 con 0x7f54cc071880 2023-11-06T02:10:23.461 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:23.459+0000 7f54d1c03640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54cc15dd00 con 0x7f54cc0d25c0 2023-11-06T02:10:23.461 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:23.459+0000 7f54cb7fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f54cc0714a0 0x7f54cc0cd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:23.462 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:23.459+0000 7f54cb7fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f54cc0714a0 0x7f54cc0cd060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46326/0 (socket says 172.21.15.49:46326) 2023-11-06T02:10:23.462 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:23.459+0000 7f54cb7fe640 1 -- 172.21.15.49:0/2071323412 learned_addr learned my addr 172.21.15.49:0/2071323412 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:23.462 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:23.459+0000 7f54caffd640 1 --2- 172.21.15.49:0/2071323412 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f54cc0d25c0 0x7f54cc15d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:23.462 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:23.459+0000 7f54cbfff640 1 --2- 172.21.15.49:0/2071323412 >> v2:172.21.15.150:3300/0 conn(0x7f54cc071880 0x7f54cc0b4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:23.462 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:23.459+0000 7f54cb7fe640 1 -- 172.21.15.49:0/2071323412 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f54cc0d25c0 msgr2=0x7f54cc15d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:23.463 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:23.459+0000 7f54cb7fe640 1 --2- 172.21.15.49:0/2071323412 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f54cc0d25c0 0x7f54cc15d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:23.463 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:23.459+0000 7f54cb7fe640 1 -- 172.21.15.49:0/2071323412 >> v2:172.21.15.150:3300/0 conn(0x7f54cc071880 msgr2=0x7f54cc0b4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:23.463 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:23.459+0000 7f54cb7fe640 1 --2- 172.21.15.49:0/2071323412 >> v2:172.21.15.150:3300/0 conn(0x7f54cc071880 0x7f54cc0b4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:23.463 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:23.459+0000 7f54cb7fe640 1 -- 172.21.15.49:0/2071323412 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54cc15dfe0 con 0x7f54cc0714a0 2023-11-06T02:10:23.463 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:23.459+0000 7f54cbfff640 1 --2- 172.21.15.49:0/2071323412 >> v2:172.21.15.150:3300/0 conn(0x7f54cc071880 0x7f54cc0b4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:10:23.464 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:23.459+0000 7f54caffd640 1 --2- 172.21.15.49:0/2071323412 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f54cc0d25c0 0x7f54cc15d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:10:23.467 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:23.463+0000 7f54cb7fe640 1 --2- 172.21.15.49:0/2071323412 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54cc0714a0 0x7f54cc0cd060 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f54cc08bca0 tx=0x7f54c401c5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f25f663ae102f93 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:23.467 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:23.463+0000 7f54ca7fc640 1 -- 172.21.15.49:0/2071323412 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54c40a6470 con 0x7f54cc0714a0 2023-11-06T02:10:23.467 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:23.463+0000 7f54ca7fc640 1 -- 172.21.15.49:0/2071323412 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54c40a65d0 con 0x7f54cc0714a0 2023-11-06T02:10:23.467 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:23.463+0000 7f54ca7fc640 1 -- 172.21.15.49:0/2071323412 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54c40a6960 con 0x7f54cc0714a0 2023-11-06T02:10:23.468 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:23.463+0000 7f54d1c03640 1 -- 172.21.15.49:0/2071323412 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54cc0714a0 msgr2=0x7f54cc0cd060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:23.468 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:23.463+0000 7f54d1c03640 1 --2- 172.21.15.49:0/2071323412 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54cc0714a0 0x7f54cc0cd060 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f54cc08bca0 tx=0x7f54c401c5c0 comp rx=0 tx=0).stop 2023-11-06T02:10:23.468 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:23.463+0000 7f54d1c03640 1 -- 172.21.15.49:0/2071323412 shutdown_connections 2023-11-06T02:10:23.468 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:23.463+0000 7f54d1c03640 1 --2- 172.21.15.49:0/2071323412 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f54cc0d25c0 0x7f54cc15d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:23.468 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:23.463+0000 7f54d1c03640 1 --2- 172.21.15.49:0/2071323412 >> v2:172.21.15.150:3300/0 conn(0x7f54cc071880 0x7f54cc0b4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:23.469 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:23.463+0000 7f54d1c03640 1 --2- 172.21.15.49:0/2071323412 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54cc0714a0 0x7f54cc0cd060 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:23.469 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:23.463+0000 7f54d1c03640 1 -- 172.21.15.49:0/2071323412 >> 172.21.15.49:0/2071323412 conn(0x7f54cc0d2aa0 msgr2=0x7f54cc0d2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:23.469 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:23.463+0000 7f54d1c03640 1 -- 172.21.15.49:0/2071323412 shutdown_connections 2023-11-06T02:10:23.469 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:23.463+0000 7f54d1c03640 1 -- 172.21.15.49:0/2071323412 wait complete. 2023-11-06T02:10:23.469 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:23.463+0000 7f54d1c03640 1 Processor -- start 2023-11-06T02:10:23.470 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:23.463+0000 7f54d1c03640 1 -- start start 2023-11-06T02:10:23.470 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:23.467+0000 7f54d1c03640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54cc0714a0 0x7f54cc153860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:23.470 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:23.467+0000 7f54d1c03640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f54cc071880 0x7f54cc14b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:23.470 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:23.467+0000 7f54d1c03640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f54cc0d25c0 0x7f54cc14bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:23.470 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:23.467+0000 7f54d1c03640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54cc14c4c0 con 0x7f54cc0714a0 2023-11-06T02:10:23.471 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:23.467+0000 7f54d1c03640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54cc153ed0 con 0x7f54cc071880 2023-11-06T02:10:23.471 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:23.467+0000 7f54d1c03640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54cc1541b0 con 0x7f54cc0d25c0 2023-11-06T02:10:23.471 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:23.467+0000 7f54cb7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54cc0714a0 0x7f54cc153860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:23.471 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:23.467+0000 7f54cb7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54cc0714a0 0x7f54cc153860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46330/0 (socket says 172.21.15.49:46330) 2023-11-06T02:10:23.471 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:23.467+0000 7f54cb7fe640 1 -- 172.21.15.49:0/3255372711 learned_addr learned my addr 172.21.15.49:0/3255372711 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:23.472 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:23.467+0000 7f54cb7fe640 1 -- 172.21.15.49:0/3255372711 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f54cc0d25c0 msgr2=0x7f54cc14bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:23.472 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:23.467+0000 7f54cb7fe640 1 --2- 172.21.15.49:0/3255372711 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f54cc0d25c0 0x7f54cc14bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:23.472 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:23.467+0000 7f54cb7fe640 1 -- 172.21.15.49:0/3255372711 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f54cc071880 msgr2=0x7f54cc14b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:23.472 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:23.467+0000 7f54cb7fe640 1 --2- 172.21.15.49:0/3255372711 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f54cc071880 0x7f54cc14b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:23.472 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:23.467+0000 7f54cb7fe640 1 -- 172.21.15.49:0/3255372711 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54c4009b30 con 0x7f54cc0714a0 2023-11-06T02:10:23.472 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:23.467+0000 7f54cb7fe640 1 --2- 172.21.15.49:0/3255372711 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54cc0714a0 0x7f54cc153860 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f54c401ac70 tx=0x7f54c4019360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:23.473 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:23.467+0000 7f54b3fff640 1 -- 172.21.15.49:0/3255372711 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54c40a7650 con 0x7f54cc0714a0 2023-11-06T02:10:23.473 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:23.467+0000 7f54b3fff640 1 -- 172.21.15.49:0/3255372711 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54c40a77d0 con 0x7f54cc0714a0 2023-11-06T02:10:23.473 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:23.467+0000 7f54b3fff640 1 -- 172.21.15.49:0/3255372711 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54c40a7b80 con 0x7f54cc0714a0 2023-11-06T02:10:23.473 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:23.467+0000 7f54d1c03640 1 -- 172.21.15.49:0/3255372711 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54cc154490 con 0x7f54cc0714a0 2023-11-06T02:10:23.473 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:23.467+0000 7f54d1c03640 1 -- 172.21.15.49:0/3255372711 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54cc0cd9f0 con 0x7f54cc0714a0 2023-11-06T02:10:23.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:23.467+0000 7f54d1c03640 1 -- 172.21.15.49:0/3255372711 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54cc074040 con 0x7f54cc0714a0 2023-11-06T02:10:23.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:23.467+0000 7f54b3fff640 1 -- 172.21.15.49:0/3255372711 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f54c40a7ce0 con 0x7f54cc0714a0 2023-11-06T02:10:23.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:23.467+0000 7f54b3fff640 1 --2- 172.21.15.49:0/3255372711 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f54b8065a60 0x7f54b8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:23.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:23.471+0000 7f54b3fff640 1 -- 172.21.15.49:0/3255372711 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5739+0+0 (secure 0 0 0) 0x7f54c411dd70 con 0x7f54cc0714a0 2023-11-06T02:10:23.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:23.471+0000 7f54caffd640 1 --2- 172.21.15.49:0/3255372711 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f54b8065a60 0x7f54b8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:23.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:23.471+0000 7f54caffd640 1 --2- 172.21.15.49:0/3255372711 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f54b8065a60 0x7f54b8067f20 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f54cc1f8f80 tx=0x7f54c0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:23.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:23.475+0000 7f54b3fff640 1 -- 172.21.15.49:0/3255372711 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f54c40eb7a0 con 0x7f54cc0714a0 2023-11-06T02:10:23.632 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:23.631+0000 7f54d1c03640 1 -- 172.21.15.49:0/3255372711 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f54cc15a530 con 0x7f54cc0714a0 2023-11-06T02:10:24.304 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.303+0000 7f54b3fff640 1 -- 172.21.15.49:0/3255372711 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v42) v1 ==== 169+0+0 (secure 0 0 0) 0x7f54c40eedc0 con 0x7f54cc0714a0 2023-11-06T02:10:24.304 INFO:teuthology.orchestra.run.smithi049.stderr:pool 'cephfs_data' removed 2023-11-06T02:10:24.310 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.311+0000 7f54d1c03640 1 -- 172.21.15.49:0/3255372711 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f54b8065a60 msgr2=0x7f54b8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:24.310 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.311+0000 7f54d1c03640 1 --2- 172.21.15.49:0/3255372711 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f54b8065a60 0x7f54b8067f20 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f54cc1f8f80 tx=0x7f54c0008040 comp rx=0 tx=0).stop 2023-11-06T02:10:24.310 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.311+0000 7f54d1c03640 1 -- 172.21.15.49:0/3255372711 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54cc0714a0 msgr2=0x7f54cc153860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:24.311 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.311+0000 7f54d1c03640 1 --2- 172.21.15.49:0/3255372711 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54cc0714a0 0x7f54cc153860 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f54c401ac70 tx=0x7f54c4019360 comp rx=0 tx=0).stop 2023-11-06T02:10:24.311 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.311+0000 7f54d1c03640 1 -- 172.21.15.49:0/3255372711 shutdown_connections 2023-11-06T02:10:24.311 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.311+0000 7f54d1c03640 1 --2- 172.21.15.49:0/3255372711 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f54b8065a60 0x7f54b8067f20 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:24.311 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.311+0000 7f54d1c03640 1 --2- 172.21.15.49:0/3255372711 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f54cc0d25c0 0x7f54cc14bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:24.311 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.311+0000 7f54d1c03640 1 --2- 172.21.15.49:0/3255372711 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f54cc071880 0x7f54cc14b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:24.312 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.311+0000 7f54d1c03640 1 --2- 172.21.15.49:0/3255372711 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54cc0714a0 0x7f54cc153860 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:24.312 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.311+0000 7f54d1c03640 1 -- 172.21.15.49:0/3255372711 >> 172.21.15.49:0/3255372711 conn(0x7f54cc0d2aa0 msgr2=0x7f54cc0b3460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:24.312 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.311+0000 7f54d1c03640 1 -- 172.21.15.49:0/3255372711 shutdown_connections 2023-11-06T02:10:24.312 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.311+0000 7f54d1c03640 1 -- 172.21.15.49:0/3255372711 wait complete. 2023-11-06T02:10:24.337 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-06T02:10:24.469 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.467+0000 7f57fba27640 1 Processor -- start 2023-11-06T02:10:24.469 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.467+0000 7f57fba27640 1 -- start start 2023-11-06T02:10:24.470 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.471+0000 7f57fba27640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f57f415a880 0x7f57f40b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:24.470 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.471+0000 7f57fba27640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f57f406d860 0x7f57f4070610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:24.470 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.471+0000 7f57fba27640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57f40ae360 con 0x7f57f4070160 2023-11-06T02:10:24.471 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.471+0000 7f57fba27640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57f4070b50 con 0x7f57f415a880 2023-11-06T02:10:24.471 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.471+0000 7f57fba27640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57f40ae640 con 0x7f57f406d860 2023-11-06T02:10:24.471 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.471+0000 7f57f97c3640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f57f415a880 0x7f57f40b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:24.471 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.471+0000 7f57f8fc2640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f57f4070160 0x7f57f406d150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:59752/0 (socket says 172.21.15.49:59752) 2023-11-06T02:10:24.471 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.471+0000 7f57f8fc2640 1 -- 172.21.15.49:0/2198247221 learned_addr learned my addr 172.21.15.49:0/2198247221 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:10:24.472 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.471+0000 7f57f97c3640 1 -- 172.21.15.49:0/2198247221 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f57f406d860 msgr2=0x7f57f4070610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:24.472 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.471+0000 7f57f97c3640 1 --2- 172.21.15.49:0/2198247221 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f57f406d860 0x7f57f4070610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:24.472 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.471+0000 7f57f97c3640 1 -- 172.21.15.49:0/2198247221 >> v1:172.21.15.49:6789/0 conn(0x7f57f4070160 legacy=0x7f57f406d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:24.473 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.471+0000 7f57ebfff640 1 -- 172.21.15.49:0/2198247221 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1012627946 0 0) 0x7f57f40ae360 con 0x7f57f4070160 2023-11-06T02:10:24.473 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.471+0000 7f57f97c3640 1 -- 172.21.15.49:0/2198247221 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57f40ae920 con 0x7f57f415a880 2023-11-06T02:10:24.474 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.471+0000 7f57f97c3640 1 --2- 172.21.15.49:0/2198247221 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f57f415a880 0x7f57f40b15c0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f57f00097c0 tx=0x7f57f001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5066ec27e8d48c8 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:24.474 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.471+0000 7f57ebfff640 1 -- 172.21.15.49:0/2198247221 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f00ad640 con 0x7f57f415a880 2023-11-06T02:10:24.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.471+0000 7f57ebfff640 1 -- 172.21.15.49:0/2198247221 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57f00ad7a0 con 0x7f57f415a880 2023-11-06T02:10:24.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.471+0000 7f57ebfff640 1 -- 172.21.15.49:0/2198247221 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f00adb30 con 0x7f57f415a880 2023-11-06T02:10:24.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.475+0000 7f57fba27640 1 -- 172.21.15.49:0/2198247221 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f57f415a880 msgr2=0x7f57f40b15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:24.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.475+0000 7f57fba27640 1 --2- 172.21.15.49:0/2198247221 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f57f415a880 0x7f57f40b15c0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f57f00097c0 tx=0x7f57f001c480 comp rx=0 tx=0).stop 2023-11-06T02:10:24.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.475+0000 7f57fba27640 1 -- 172.21.15.49:0/2198247221 shutdown_connections 2023-11-06T02:10:24.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.475+0000 7f57fba27640 1 --2- 172.21.15.49:0/2198247221 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f57f406d860 0x7f57f4070610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:24.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.475+0000 7f57fba27640 1 --2- 172.21.15.49:0/2198247221 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f57f415a880 0x7f57f40b15c0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:24.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.475+0000 7f57fba27640 1 -- 172.21.15.49:0/2198247221 >> 172.21.15.49:0/2198247221 conn(0x7f57f406a370 msgr2=0x7f57f40ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:24.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.475+0000 7f57fba27640 1 -- 172.21.15.49:0/2198247221 shutdown_connections 2023-11-06T02:10:24.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.475+0000 7f57fba27640 1 -- 172.21.15.49:0/2198247221 wait complete. 2023-11-06T02:10:24.477 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.475+0000 7f57fba27640 1 Processor -- start 2023-11-06T02:10:24.477 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.475+0000 7f57fba27640 1 -- start start 2023-11-06T02:10:24.479 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.475+0000 7f57fba27640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f57f406d860 0x7f57f4145590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:24.479 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.475+0000 7f57fba27640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f57f4070160 0x7f57f40cba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:24.480 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.475+0000 7f57fba27640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57f415a880 0x7f57f40cbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:24.480 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.475+0000 7f57fba27640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f40cdca0 con 0x7f57f406d860 2023-11-06T02:10:24.480 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.475+0000 7f57fba27640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f4145ad0 con 0x7f57f415a880 2023-11-06T02:10:24.480 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.475+0000 7f57fba27640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57f414c300 con 0x7f57f4070160 2023-11-06T02:10:24.480 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.475+0000 7f57f97c3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f57f406d860 0x7f57f4145590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:24.481 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.475+0000 7f57f97c3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f57f406d860 0x7f57f4145590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46334/0 (socket says 172.21.15.49:46334) 2023-11-06T02:10:24.481 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.475+0000 7f57f97c3640 1 -- 172.21.15.49:0/379741194 learned_addr learned my addr 172.21.15.49:0/379741194 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:24.481 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.475+0000 7f57f97c3640 1 -- 172.21.15.49:0/379741194 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f57f4070160 msgr2=0x7f57f40cba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:24.481 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.475+0000 7f57f97c3640 1 --2- 172.21.15.49:0/379741194 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f57f4070160 0x7f57f40cba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:24.482 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.475+0000 7f57f97c3640 1 -- 172.21.15.49:0/379741194 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57f415a880 msgr2=0x7f57f40cbfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:24.482 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.475+0000 7f57f97c3640 1 --2- 172.21.15.49:0/379741194 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57f415a880 0x7f57f40cbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:24.482 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.475+0000 7f57f97c3640 1 -- 172.21.15.49:0/379741194 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57f0009420 con 0x7f57f406d860 2023-11-06T02:10:24.482 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.475+0000 7f57f97c3640 1 --2- 172.21.15.49:0/379741194 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f57f406d860 0x7f57f4145590 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f57f001a0b0 tx=0x7f57f0019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:24.482 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.475+0000 7f57e9ffb640 1 -- 172.21.15.49:0/379741194 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f0004220 con 0x7f57f406d860 2023-11-06T02:10:24.483 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.475+0000 7f57e9ffb640 1 -- 172.21.15.49:0/379741194 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57f00a6040 con 0x7f57f406d860 2023-11-06T02:10:24.483 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.475+0000 7f57fba27640 1 -- 172.21.15.49:0/379741194 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57f414c5e0 con 0x7f57f406d860 2023-11-06T02:10:24.483 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.475+0000 7f57e9ffb640 1 -- 172.21.15.49:0/379741194 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f0003620 con 0x7f57f406d860 2023-11-06T02:10:24.483 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.475+0000 7f57fba27640 1 -- 172.21.15.49:0/379741194 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57f414cba0 con 0x7f57f406d860 2023-11-06T02:10:24.483 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.479+0000 7f57fba27640 1 -- 172.21.15.49:0/379741194 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57f4074040 con 0x7f57f406d860 2023-11-06T02:10:24.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.479+0000 7f57e9ffb640 1 -- 172.21.15.49:0/379741194 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f57f00b4040 con 0x7f57f406d860 2023-11-06T02:10:24.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.479+0000 7f57e9ffb640 1 --2- 172.21.15.49:0/379741194 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f57d8065a60 0x7f57d8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:24.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.479+0000 7f57e9ffb640 1 -- 172.21.15.49:0/379741194 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f57f011c700 con 0x7f57f406d860 2023-11-06T02:10:24.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.479+0000 7f57f8fc2640 1 --2- 172.21.15.49:0/379741194 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f57d8065a60 0x7f57d8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:24.485 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.479+0000 7f57f8fc2640 1 --2- 172.21.15.49:0/379741194 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f57d8065a60 0x7f57d8067f20 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f57e40011f0 tx=0x7f57e4007930 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:24.486 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.483+0000 7f57e9ffb640 1 -- 172.21.15.49:0/379741194 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f57f00ea260 con 0x7f57f406d860 2023-11-06T02:10:24.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.643+0000 7f57fba27640 1 -- 172.21.15.49:0/379741194 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f57f40cdff0 con 0x7f57f406d860 2023-11-06T02:10:24.651 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.651+0000 7f57e9ffb640 1 -- 172.21.15.49:0/379741194 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f57f00b20a0 con 0x7f57f406d860 2023-11-06T02:10:24.653 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.655+0000 7f57fba27640 1 -- 172.21.15.49:0/379741194 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f57d8065a60 msgr2=0x7f57d8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:24.653 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.655+0000 7f57fba27640 1 --2- 172.21.15.49:0/379741194 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f57d8065a60 0x7f57d8067f20 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f57e40011f0 tx=0x7f57e4007930 comp rx=0 tx=0).stop 2023-11-06T02:10:24.654 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.655+0000 7f57fba27640 1 -- 172.21.15.49:0/379741194 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f57f406d860 msgr2=0x7f57f4145590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:24.654 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.655+0000 7f57fba27640 1 --2- 172.21.15.49:0/379741194 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f57f406d860 0x7f57f4145590 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f57f001a0b0 tx=0x7f57f0019220 comp rx=0 tx=0).stop 2023-11-06T02:10:24.654 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.655+0000 7f57fba27640 1 -- 172.21.15.49:0/379741194 shutdown_connections 2023-11-06T02:10:24.654 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.655+0000 7f57fba27640 1 --2- 172.21.15.49:0/379741194 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f57d8065a60 0x7f57d8067f20 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:24.655 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.655+0000 7f57fba27640 1 --2- 172.21.15.49:0/379741194 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57f415a880 0x7f57f40cbfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:24.655 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.655+0000 7f57fba27640 1 --2- 172.21.15.49:0/379741194 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f57f4070160 0x7f57f40cba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:24.655 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.655+0000 7f57fba27640 1 --2- 172.21.15.49:0/379741194 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f57f406d860 0x7f57f4145590 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:24.655 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.655+0000 7f57fba27640 1 -- 172.21.15.49:0/379741194 >> 172.21.15.49:0/379741194 conn(0x7f57f406a370 msgr2=0x7f57f41503e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:24.655 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.655+0000 7f57fba27640 1 -- 172.21.15.49:0/379741194 shutdown_connections 2023-11-06T02:10:24.655 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.655+0000 7f57fba27640 1 -- 172.21.15.49:0/379741194 wait complete. 2023-11-06T02:10:24.671 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag' 2023-11-06T02:10:24.800 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.795+0000 7f4b032b6640 1 Processor -- start 2023-11-06T02:10:24.800 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.795+0000 7f4b032b6640 1 -- start start 2023-11-06T02:10:24.801 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.799+0000 7f4b032b6640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f4afc0cb580 0x7f4afc0cb960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:24.801 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.799+0000 7f4b032b6640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f4afc0b6100 0x7f4afc15e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:24.801 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.799+0000 7f4b032b6640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4afc15f3c0 con 0x7f4afc0ce420 2023-11-06T02:10:24.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.799+0000 7f4b032b6640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4afc15eb20 con 0x7f4afc0cb580 2023-11-06T02:10:24.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.799+0000 7f4b032b6640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4afc15ee00 con 0x7f4afc0b6100 2023-11-06T02:10:24.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.799+0000 7f4b01052640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f4afc0cb580 0x7f4afc0cb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:24.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.799+0000 7f4b01052640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f4afc0cb580 0x7f4afc0cb960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46350/0 (socket says 172.21.15.49:46350) 2023-11-06T02:10:24.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.799+0000 7f4b01052640 1 -- 172.21.15.49:0/1691211528 learned_addr learned my addr 172.21.15.49:0/1691211528 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:24.804 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.799+0000 7f4b00851640 1 --2- 172.21.15.49:0/1691211528 >> v2:172.21.15.150:3300/0 conn(0x7f4afc0b6100 0x7f4afc15e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:24.804 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.799+0000 7f4b01052640 1 -- 172.21.15.49:0/1691211528 >> v2:172.21.15.150:3300/0 conn(0x7f4afc0b6100 msgr2=0x7f4afc15e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:24.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.799+0000 7f4b01052640 1 --2- 172.21.15.49:0/1691211528 >> v2:172.21.15.150:3300/0 conn(0x7f4afc0b6100 0x7f4afc15e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:24.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.799+0000 7f4b01052640 1 -- 172.21.15.49:0/1691211528 >> v1:172.21.15.150:6789/0 conn(0x7f4afc0ce420 legacy=0x7f4afc15b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:24.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.799+0000 7f4b01052640 1 -- 172.21.15.49:0/1691211528 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4afc15f0e0 con 0x7f4afc0cb580 2023-11-06T02:10:24.813 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.799+0000 7f4b01052640 1 --2- 172.21.15.49:0/1691211528 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4afc0cb580 0x7f4afc0cb960 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f4afc0c78d0 tx=0x7f4af801c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39b6310469c0374c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:24.813 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.803+0000 7f4af3fff640 1 -- 172.21.15.49:0/1691211528 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af80ac640 con 0x7f4afc0cb580 2023-11-06T02:10:24.814 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.803+0000 7f4af3fff640 1 -- 172.21.15.49:0/1691211528 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4af80ac7a0 con 0x7f4afc0cb580 2023-11-06T02:10:24.814 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.803+0000 7f4af3fff640 1 -- 172.21.15.49:0/1691211528 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af80acb30 con 0x7f4afc0cb580 2023-11-06T02:10:24.815 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.803+0000 7f4b032b6640 1 -- 172.21.15.49:0/1691211528 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4afc0cb580 msgr2=0x7f4afc0cb960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:24.815 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.803+0000 7f4b032b6640 1 --2- 172.21.15.49:0/1691211528 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4afc0cb580 0x7f4afc0cb960 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f4afc0c78d0 tx=0x7f4af801c400 comp rx=0 tx=0).stop 2023-11-06T02:10:24.815 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.803+0000 7f4b032b6640 1 -- 172.21.15.49:0/1691211528 shutdown_connections 2023-11-06T02:10:24.816 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.803+0000 7f4b032b6640 1 --2- 172.21.15.49:0/1691211528 >> v2:172.21.15.150:3300/0 conn(0x7f4afc0b6100 0x7f4afc15e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:24.816 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.803+0000 7f4b032b6640 1 --2- 172.21.15.49:0/1691211528 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4afc0cb580 0x7f4afc0cb960 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:24.816 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.803+0000 7f4b032b6640 1 -- 172.21.15.49:0/1691211528 >> 172.21.15.49:0/1691211528 conn(0x7f4afc0cd2d0 msgr2=0x7f4afc0cd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:24.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.803+0000 7f4b032b6640 1 -- 172.21.15.49:0/1691211528 shutdown_connections 2023-11-06T02:10:24.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.803+0000 7f4b032b6640 1 -- 172.21.15.49:0/1691211528 wait complete. 2023-11-06T02:10:24.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.803+0000 7f4b032b6640 1 Processor -- start 2023-11-06T02:10:24.818 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.803+0000 7f4b032b6640 1 -- start start 2023-11-06T02:10:24.818 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.803+0000 7f4b032b6640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4afc0b6100 0x7f4afc0c8710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:24.818 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.803+0000 7f4b032b6640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4afc0cb580 0x7f4afc155c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:24.819 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.803+0000 7f4b032b6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4afc0ce420 0x7f4afc163960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:24.819 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.803+0000 7f4b032b6640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4afc0c9b70 con 0x7f4afc0ce420 2023-11-06T02:10:24.820 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.803+0000 7f4b032b6640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4afc0c8ff0 con 0x7f4afc0cb580 2023-11-06T02:10:24.820 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.803+0000 7f4b032b6640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4afc0c92d0 con 0x7f4afc0b6100 2023-11-06T02:10:24.820 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.803+0000 7f4b01853640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4afc0ce420 0x7f4afc163960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:24.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.803+0000 7f4b01853640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4afc0ce420 0x7f4afc163960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46362/0 (socket says 172.21.15.49:46362) 2023-11-06T02:10:24.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.803+0000 7f4b01853640 1 -- 172.21.15.49:0/531670768 learned_addr learned my addr 172.21.15.49:0/531670768 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:24.822 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.803+0000 7f4b01052640 1 --2- 172.21.15.49:0/531670768 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4afc0b6100 0x7f4afc0c8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:24.822 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.803+0000 7f4b00851640 1 --2- 172.21.15.49:0/531670768 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4afc0cb580 0x7f4afc155c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:24.822 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.803+0000 7f4b01853640 1 -- 172.21.15.49:0/531670768 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4afc0b6100 msgr2=0x7f4afc0c8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:24.823 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.803+0000 7f4b01853640 1 --2- 172.21.15.49:0/531670768 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4afc0b6100 0x7f4afc0c8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:24.823 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.803+0000 7f4b01853640 1 -- 172.21.15.49:0/531670768 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4afc0cb580 msgr2=0x7f4afc155c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:24.824 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.803+0000 7f4b01853640 1 --2- 172.21.15.49:0/531670768 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4afc0cb580 0x7f4afc155c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:24.824 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.803+0000 7f4b01853640 1 -- 172.21.15.49:0/531670768 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4af80094d0 con 0x7f4afc0ce420 2023-11-06T02:10:24.824 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.803+0000 7f4b00851640 1 --2- 172.21.15.49:0/531670768 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4afc0cb580 0x7f4afc155c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:10:24.825 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.803+0000 7f4b01853640 1 --2- 172.21.15.49:0/531670768 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4afc0ce420 0x7f4afc163960 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f4af400c780 tx=0x7f4af400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:24.825 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.803+0000 7f4af1ffb640 1 -- 172.21.15.49:0/531670768 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af4004b70 con 0x7f4afc0ce420 2023-11-06T02:10:24.825 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.803+0000 7f4af1ffb640 1 -- 172.21.15.49:0/531670768 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4af400ce90 con 0x7f4afc0ce420 2023-11-06T02:10:24.826 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.803+0000 7f4b032b6640 1 -- 172.21.15.49:0/531670768 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4afc15be30 con 0x7f4afc0ce420 2023-11-06T02:10:24.826 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.803+0000 7f4af1ffb640 1 -- 172.21.15.49:0/531670768 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af400a850 con 0x7f4afc0ce420 2023-11-06T02:10:24.827 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.803+0000 7f4b032b6640 1 -- 172.21.15.49:0/531670768 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4afc0c9890 con 0x7f4afc0ce420 2023-11-06T02:10:24.827 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.807+0000 7f4b032b6640 1 -- 172.21.15.49:0/531670768 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4afc15dfd0 con 0x7f4afc0ce420 2023-11-06T02:10:24.827 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.807+0000 7f4af1ffb640 1 -- 172.21.15.49:0/531670768 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f4af400a9b0 con 0x7f4afc0ce420 2023-11-06T02:10:24.828 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.807+0000 7f4af1ffb640 1 --2- 172.21.15.49:0/531670768 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4ae0065a60 0x7f4ae0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:24.828 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.807+0000 7f4af1ffb640 1 -- 172.21.15.49:0/531670768 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f4af40831f0 con 0x7f4afc0ce420 2023-11-06T02:10:24.828 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.807+0000 7f4b01052640 1 --2- 172.21.15.49:0/531670768 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4ae0065a60 0x7f4ae0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:24.829 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.807+0000 7f4b01052640 1 --2- 172.21.15.49:0/531670768 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4ae0065a60 0x7f4ae0067f20 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f4af80ae000 tx=0x7f4af80a6070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:24.829 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:24.811+0000 7f4af1ffb640 1 -- 172.21.15.49:0/531670768 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f4af4050d50 con 0x7f4afc0ce420 2023-11-06T02:10:25.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:25.023+0000 7f4b032b6640 1 -- 172.21.15.49:0/531670768 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]} v 0) v1 -- 0x7f4afc0ce170 con 0x7f4afc0ce420 2023-11-06T02:10:25.322 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:25.323+0000 7f4af1ffb640 1 -- 172.21.15.49:0/531670768 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]}]=0 v124) v1 ==== 142+0+0 (secure 0 0 0) 0x7f4af4054370 con 0x7f4afc0ce420 2023-11-06T02:10:25.326 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:25.327+0000 7f4b032b6640 1 -- 172.21.15.49:0/531670768 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4ae0065a60 msgr2=0x7f4ae0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:25.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:25.327+0000 7f4b032b6640 1 --2- 172.21.15.49:0/531670768 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4ae0065a60 0x7f4ae0067f20 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f4af80ae000 tx=0x7f4af80a6070 comp rx=0 tx=0).stop 2023-11-06T02:10:25.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:25.327+0000 7f4b032b6640 1 -- 172.21.15.49:0/531670768 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4afc0ce420 msgr2=0x7f4afc163960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:25.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:25.327+0000 7f4b032b6640 1 --2- 172.21.15.49:0/531670768 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4afc0ce420 0x7f4afc163960 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f4af400c780 tx=0x7f4af400cc40 comp rx=0 tx=0).stop 2023-11-06T02:10:25.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:25.327+0000 7f4b032b6640 1 -- 172.21.15.49:0/531670768 shutdown_connections 2023-11-06T02:10:25.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:25.327+0000 7f4b032b6640 1 --2- 172.21.15.49:0/531670768 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4ae0065a60 0x7f4ae0067f20 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:25.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:25.327+0000 7f4b032b6640 1 --2- 172.21.15.49:0/531670768 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4afc0ce420 0x7f4afc163960 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:25.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:25.327+0000 7f4b032b6640 1 --2- 172.21.15.49:0/531670768 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4afc0cb580 0x7f4afc155c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:25.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:25.327+0000 7f4b032b6640 1 --2- 172.21.15.49:0/531670768 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4afc0b6100 0x7f4afc0c8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:25.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:25.327+0000 7f4b032b6640 1 -- 172.21.15.49:0/531670768 >> 172.21.15.49:0/531670768 conn(0x7f4afc0cd2d0 msgr2=0x7f4afc0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:25.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:25.327+0000 7f4b032b6640 1 -- 172.21.15.49:0/531670768 shutdown_connections 2023-11-06T02:10:25.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:25.327+0000 7f4b032b6640 1 -- 172.21.15.49:0/531670768 wait complete. 2023-11-06T02:10:25.348 INFO:tasks.cephfs_test_runner:test_apply_tag (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2023-11-06T02:10:25.348 INFO:tasks.cephfs_test_runner:Starting test: test_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) That the MDS can repair an inodes backtrace in the data pool 2023-11-06T02:10:25.349 INFO:tasks.cephfs_test_runner:test_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2023-11-06T02:10:25.349 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair' 2023-11-06T02:10:25.474 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:25.475+0000 7f9b8fdbf640 1 Processor -- start 2023-11-06T02:10:25.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:25.475+0000 7f9b8fdbf640 1 -- start start 2023-11-06T02:10:25.477 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:25.475+0000 7f9b8fdbf640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f9b8806a9b0 0x7f9b880ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:25.477 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:25.475+0000 7f9b8fdbf640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b8815cfd0 con 0x7f9b8815cbf0 2023-11-06T02:10:25.477 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:25.475+0000 7f9b8fdbf640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b880ce0d0 con 0x7f9b88069d70 2023-11-06T02:10:25.478 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:25.475+0000 7f9b8fdbf640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b880cddd0 con 0x7f9b8806a9b0 2023-11-06T02:10:25.478 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:25.475+0000 7f9b8d35a640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f9b8806a9b0 0x7f9b880ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:25.478 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:25.475+0000 7f9b8d35a640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f9b8806a9b0 0x7f9b880ce560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46368/0 (socket says 172.21.15.49:46368) 2023-11-06T02:10:25.479 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:25.475+0000 7f9b8d35a640 1 -- 172.21.15.49:0/3673770731 learned_addr learned my addr 172.21.15.49:0/3673770731 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:25.480 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:25.475+0000 7f9b8d35a640 1 -- 172.21.15.49:0/3673770731 >> v1:172.21.15.150:6789/0 conn(0x7f9b88069d70 legacy=0x7f9b880c9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:25.480 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:25.475+0000 7f9b8d35a640 1 -- 172.21.15.49:0/3673770731 >> v1:172.21.15.49:6789/0 conn(0x7f9b8815cbf0 legacy=0x7f9b88161100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:25.481 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:25.475+0000 7f9b8d35a640 1 -- 172.21.15.49:0/3673770731 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b8806af90 con 0x7f9b8806a9b0 2023-11-06T02:10:25.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:25.479+0000 7f9b8d35a640 1 --2- 172.21.15.49:0/3673770731 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b8806a9b0 0x7f9b880ce560 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f9b84009870 tx=0x7f9b8401c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9fc927061f4a193 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:25.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:25.479+0000 7f9b8cb59640 1 -- 172.21.15.49:0/3673770731 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b840ab660 con 0x7f9b8806a9b0 2023-11-06T02:10:25.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:25.479+0000 7f9b8cb59640 1 -- 172.21.15.49:0/3673770731 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b840ab7c0 con 0x7f9b8806a9b0 2023-11-06T02:10:25.485 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:25.479+0000 7f9b8cb59640 1 -- 172.21.15.49:0/3673770731 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b840abb50 con 0x7f9b8806a9b0 2023-11-06T02:10:25.485 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:25.479+0000 7f9b8fdbf640 1 -- 172.21.15.49:0/3673770731 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b8806a9b0 msgr2=0x7f9b880ce560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:25.485 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:25.479+0000 7f9b8fdbf640 1 --2- 172.21.15.49:0/3673770731 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b8806a9b0 0x7f9b880ce560 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f9b84009870 tx=0x7f9b8401c440 comp rx=0 tx=0).stop 2023-11-06T02:10:25.485 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:25.479+0000 7f9b8fdbf640 1 -- 172.21.15.49:0/3673770731 shutdown_connections 2023-11-06T02:10:25.485 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:25.479+0000 7f9b8fdbf640 1 --2- 172.21.15.49:0/3673770731 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b8806a9b0 0x7f9b880ce560 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:25.486 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:25.479+0000 7f9b8fdbf640 1 -- 172.21.15.49:0/3673770731 >> 172.21.15.49:0/3673770731 conn(0x7f9b880b0bd0 msgr2=0x7f9b880b0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:25.486 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:25.479+0000 7f9b8fdbf640 1 -- 172.21.15.49:0/3673770731 shutdown_connections 2023-11-06T02:10:25.486 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:25.479+0000 7f9b8fdbf640 1 -- 172.21.15.49:0/3673770731 wait complete. 2023-11-06T02:10:25.486 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:25.479+0000 7f9b8fdbf640 1 Processor -- start 2023-11-06T02:10:25.486 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:25.479+0000 7f9b8fdbf640 1 -- start start 2023-11-06T02:10:25.487 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:25.479+0000 7f9b8fdbf640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b88069d70 0x7f9b88160670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:25.487 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:25.479+0000 7f9b8fdbf640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9b8806a9b0 0x7f9b881499e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:25.487 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:25.479+0000 7f9b8fdbf640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b8815cbf0 0x7f9b880af040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:25.487 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:25.479+0000 7f9b8fdbf640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8815f1c0 con 0x7f9b8815cbf0 2023-11-06T02:10:25.487 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:25.479+0000 7f9b8fdbf640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b88153ff0 con 0x7f9b88069d70 2023-11-06T02:10:25.488 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:25.479+0000 7f9b8fdbf640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8815f410 con 0x7f9b8806a9b0 2023-11-06T02:10:25.488 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:25.483+0000 7f9b8e35c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b8815cbf0 0x7f9b880af040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:25.488 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:25.483+0000 7f9b8e35c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b8815cbf0 0x7f9b880af040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46384/0 (socket says 172.21.15.49:46384) 2023-11-06T02:10:25.488 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:25.483+0000 7f9b8e35c640 1 -- 172.21.15.49:0/2769101209 learned_addr learned my addr 172.21.15.49:0/2769101209 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:25.488 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:25.483+0000 7f9b8db5b640 1 --2- 172.21.15.49:0/2769101209 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b88069d70 0x7f9b88160670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:25.489 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:25.483+0000 7f9b8d35a640 1 --2- 172.21.15.49:0/2769101209 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9b8806a9b0 0x7f9b881499e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:25.489 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:25.483+0000 7f9b8e35c640 1 -- 172.21.15.49:0/2769101209 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9b8806a9b0 msgr2=0x7f9b881499e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:25.489 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:25.483+0000 7f9b8e35c640 1 --2- 172.21.15.49:0/2769101209 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9b8806a9b0 0x7f9b881499e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:25.489 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:25.483+0000 7f9b8e35c640 1 -- 172.21.15.49:0/2769101209 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b88069d70 msgr2=0x7f9b88160670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:25.489 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:25.483+0000 7f9b8e35c640 1 --2- 172.21.15.49:0/2769101209 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b88069d70 0x7f9b88160670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:25.490 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:25.483+0000 7f9b8e35c640 1 -- 172.21.15.49:0/2769101209 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b840094d0 con 0x7f9b8815cbf0 2023-11-06T02:10:25.490 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:25.483+0000 7f9b8db5b640 1 --2- 172.21.15.49:0/2769101209 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b88069d70 0x7f9b88160670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:10:25.490 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:25.483+0000 7f9b8e35c640 1 --2- 172.21.15.49:0/2769101209 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b8815cbf0 0x7f9b880af040 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f9b8000c780 tx=0x7f9b8000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:25.490 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:25.483+0000 7f9b7a7fc640 1 -- 172.21.15.49:0/2769101209 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b80003d90 con 0x7f9b8815cbf0 2023-11-06T02:10:25.490 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:25.483+0000 7f9b7a7fc640 1 -- 172.21.15.49:0/2769101209 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b8000ce90 con 0x7f9b8815cbf0 2023-11-06T02:10:25.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:25.483+0000 7f9b8fdbf640 1 -- 172.21.15.49:0/2769101209 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b8815f6f0 con 0x7f9b8815cbf0 2023-11-06T02:10:25.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:25.483+0000 7f9b7a7fc640 1 -- 172.21.15.49:0/2769101209 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b800052b0 con 0x7f9b8815cbf0 2023-11-06T02:10:25.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:25.483+0000 7f9b8fdbf640 1 -- 172.21.15.49:0/2769101209 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b8814c9d0 con 0x7f9b8815cbf0 2023-11-06T02:10:25.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:25.483+0000 7f9b8fdbf640 1 -- 172.21.15.49:0/2769101209 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b88067140 con 0x7f9b8815cbf0 2023-11-06T02:10:25.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:25.483+0000 7f9b7a7fc640 1 -- 172.21.15.49:0/2769101209 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f9b80005410 con 0x7f9b8815cbf0 2023-11-06T02:10:25.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:25.483+0000 7f9b7a7fc640 1 --2- 172.21.15.49:0/2769101209 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9b6c065a60 0x7f9b6c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:25.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:25.483+0000 7f9b7a7fc640 1 -- 172.21.15.49:0/2769101209 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f9b80082850 con 0x7f9b8815cbf0 2023-11-06T02:10:25.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:25.483+0000 7f9b8db5b640 1 --2- 172.21.15.49:0/2769101209 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9b6c065a60 0x7f9b6c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:25.493 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:25.487+0000 7f9b8db5b640 1 --2- 172.21.15.49:0/2769101209 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9b6c065a60 0x7f9b6c067f20 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f9b880afe10 tx=0x7f9b7c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:25.493 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:25.487+0000 7f9b7a7fc640 1 -- 172.21.15.49:0/2769101209 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f9b800503b0 con 0x7f9b8815cbf0 2023-11-06T02:10:25.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:25.727+0000 7f9b8fdbf640 1 -- 172.21.15.49:0/2769101209 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]} v 0) v1 -- 0x7f9b880b4d40 con 0x7f9b8815cbf0 2023-11-06T02:10:26.323 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:26.323+0000 7f9b7a7fc640 1 -- 172.21.15.49:0/2769101209 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]}]=0 v125) v1 ==== 152+0+0 (secure 0 0 0) 0x7f9b800170d0 con 0x7f9b8815cbf0 2023-11-06T02:10:26.326 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:26.327+0000 7f9b8fdbf640 1 -- 172.21.15.49:0/2769101209 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9b6c065a60 msgr2=0x7f9b6c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:26.326 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:26.327+0000 7f9b8fdbf640 1 --2- 172.21.15.49:0/2769101209 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9b6c065a60 0x7f9b6c067f20 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f9b880afe10 tx=0x7f9b7c009040 comp rx=0 tx=0).stop 2023-11-06T02:10:26.326 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:26.327+0000 7f9b8fdbf640 1 -- 172.21.15.49:0/2769101209 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b8815cbf0 msgr2=0x7f9b880af040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:26.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:26.327+0000 7f9b8fdbf640 1 --2- 172.21.15.49:0/2769101209 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b8815cbf0 0x7f9b880af040 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f9b8000c780 tx=0x7f9b8000cc40 comp rx=0 tx=0).stop 2023-11-06T02:10:26.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:26.327+0000 7f9b8fdbf640 1 -- 172.21.15.49:0/2769101209 shutdown_connections 2023-11-06T02:10:26.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:26.327+0000 7f9b8fdbf640 1 --2- 172.21.15.49:0/2769101209 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9b6c065a60 0x7f9b6c067f20 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:26.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:26.327+0000 7f9b8fdbf640 1 --2- 172.21.15.49:0/2769101209 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b8815cbf0 0x7f9b880af040 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:26.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:26.327+0000 7f9b8fdbf640 1 --2- 172.21.15.49:0/2769101209 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9b8806a9b0 0x7f9b881499e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:26.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:26.327+0000 7f9b8fdbf640 1 --2- 172.21.15.49:0/2769101209 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b88069d70 0x7f9b88160670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:26.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:26.327+0000 7f9b8fdbf640 1 -- 172.21.15.49:0/2769101209 >> 172.21.15.49:0/2769101209 conn(0x7f9b880b0bd0 msgr2=0x7f9b880c8250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:26.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:26.327+0000 7f9b8fdbf640 1 -- 172.21.15.49:0/2769101209 shutdown_connections 2023-11-06T02:10:26.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:26.327+0000 7f9b8fdbf640 1 -- 172.21.15.49:0/2769101209 wait complete. 2023-11-06T02:10:26.346 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-06T02:10:26.477 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:26.479+0000 7fe322a9e640 1 Processor -- start 2023-11-06T02:10:26.478 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:26.479+0000 7fe322a9e640 1 -- start start 2023-11-06T02:10:26.479 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:26.479+0000 7fe322a9e640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe31c070870 0x7fe31c0b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:26.479 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:26.479+0000 7fe322a9e640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe31c0ae360 con 0x7fe31c06d150 2023-11-06T02:10:26.479 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:26.479+0000 7fe322a9e640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe31c0ae640 con 0x7fe31c15a880 2023-11-06T02:10:26.479 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:26.479+0000 7fe322a9e640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe31c0ae920 con 0x7fe31c070870 2023-11-06T02:10:26.479 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:26.479+0000 7fe32103b640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fe31c06d150 0x7fe31c06d600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:59774/0 (socket says 172.21.15.49:59774) 2023-11-06T02:10:26.480 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:26.479+0000 7fe32103b640 1 -- 172.21.15.49:0/1124470246 learned_addr learned my addr 172.21.15.49:0/1124470246 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:10:26.480 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:26.479+0000 7fe31bfff640 1 --2- 172.21.15.49:0/1124470246 >> v2:172.21.15.49:3300/0 conn(0x7fe31c070870 0x7fe31c0b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:26.481 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:26.479+0000 7fe31b7fe640 1 -- 172.21.15.49:0/1124470246 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3513423180 0 0) 0x7fe31c0ae360 con 0x7fe31c06d150 2023-11-06T02:10:26.481 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:26.479+0000 7fe31b7fe640 1 -- 172.21.15.49:0/1124470246 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe304003530 con 0x7fe31c06d150 2023-11-06T02:10:26.481 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:26.479+0000 7fe31bfff640 1 -- 172.21.15.49:0/1124470246 >> v1:172.21.15.150:6789/0 conn(0x7fe31c15a880 legacy=0x7fe31c070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:26.481 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:26.479+0000 7fe31bfff640 1 -- 172.21.15.49:0/1124470246 >> v1:172.21.15.49:6789/0 conn(0x7fe31c06d150 legacy=0x7fe31c06d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:26.481 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:26.479+0000 7fe31bfff640 1 -- 172.21.15.49:0/1124470246 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe31c0aec00 con 0x7fe31c070870 2023-11-06T02:10:26.483 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:26.483+0000 7fe31bfff640 1 --2- 172.21.15.49:0/1124470246 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe31c070870 0x7fe31c0b15c0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fe3100097c0 tx=0x7fe31001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dce816c09ecb190 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:26.483 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:26.483+0000 7fe31b7fe640 1 -- 172.21.15.49:0/1124470246 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3100ab600 con 0x7fe31c070870 2023-11-06T02:10:26.483 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:26.483+0000 7fe31b7fe640 1 -- 172.21.15.49:0/1124470246 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3100a7050 con 0x7fe31c070870 2023-11-06T02:10:26.483 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:26.483+0000 7fe31b7fe640 1 -- 172.21.15.49:0/1124470246 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3100ab980 con 0x7fe31c070870 2023-11-06T02:10:26.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:26.483+0000 7fe322a9e640 1 -- 172.21.15.49:0/1124470246 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe31c070870 msgr2=0x7fe31c0b15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:26.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:26.483+0000 7fe322a9e640 1 --2- 172.21.15.49:0/1124470246 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe31c070870 0x7fe31c0b15c0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fe3100097c0 tx=0x7fe31001c3f0 comp rx=0 tx=0).stop 2023-11-06T02:10:26.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:26.483+0000 7fe322a9e640 1 -- 172.21.15.49:0/1124470246 shutdown_connections 2023-11-06T02:10:26.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:26.483+0000 7fe322a9e640 1 --2- 172.21.15.49:0/1124470246 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe31c070870 0x7fe31c0b15c0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:26.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:26.483+0000 7fe322a9e640 1 -- 172.21.15.49:0/1124470246 >> 172.21.15.49:0/1124470246 conn(0x7fe31c06a370 msgr2=0x7fe31c0ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:26.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:26.483+0000 7fe322a9e640 1 -- 172.21.15.49:0/1124470246 shutdown_connections 2023-11-06T02:10:26.485 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:26.483+0000 7fe322a9e640 1 -- 172.21.15.49:0/1124470246 wait complete. 2023-11-06T02:10:26.487 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:26.483+0000 7fe322a9e640 1 Processor -- start 2023-11-06T02:10:26.487 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:26.483+0000 7fe322a9e640 1 -- start start 2023-11-06T02:10:26.487 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:26.483+0000 7fe322a9e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe31c06d150 0x7fe31c0cd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:26.488 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:26.483+0000 7fe322a9e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe31c070870 0x7fe31c15c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:26.488 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:26.483+0000 7fe322a9e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe31c15a880 0x7fe31c14dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:26.488 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:26.483+0000 7fe322a9e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe31c0cce90 con 0x7fe31c15a880 2023-11-06T02:10:26.488 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:26.483+0000 7fe322a9e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe31c0cd7f0 con 0x7fe31c070870 2023-11-06T02:10:26.488 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:26.483+0000 7fe322a9e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe31c0ad470 con 0x7fe31c06d150 2023-11-06T02:10:26.489 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:26.483+0000 7fe32103b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe31c15a880 0x7fe31c14dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:26.489 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:26.483+0000 7fe32103b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe31c15a880 0x7fe31c14dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46406/0 (socket says 172.21.15.49:46406) 2023-11-06T02:10:26.489 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:26.483+0000 7fe32103b640 1 -- 172.21.15.49:0/1657756344 learned_addr learned my addr 172.21.15.49:0/1657756344 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:26.489 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:26.483+0000 7fe32083a640 1 --2- 172.21.15.49:0/1657756344 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe31c06d150 0x7fe31c0cd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:26.489 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:26.483+0000 7fe32103b640 1 -- 172.21.15.49:0/1657756344 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe31c06d150 msgr2=0x7fe31c0cd2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:26.489 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:26.483+0000 7fe32103b640 1 --2- 172.21.15.49:0/1657756344 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe31c06d150 0x7fe31c0cd2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:26.490 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:26.483+0000 7fe32103b640 1 -- 172.21.15.49:0/1657756344 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe31c070870 msgr2=0x7fe31c15c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:26.490 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:26.483+0000 7fe31bfff640 1 --2- 172.21.15.49:0/1657756344 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe31c070870 0x7fe31c15c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:26.490 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:26.483+0000 7fe32103b640 1 --2- 172.21.15.49:0/1657756344 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe31c070870 0x7fe31c15c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:26.490 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:26.487+0000 7fe32103b640 1 -- 172.21.15.49:0/1657756344 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe310009420 con 0x7fe31c15a880 2023-11-06T02:10:26.490 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:26.487+0000 7fe32083a640 1 --2- 172.21.15.49:0/1657756344 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe31c06d150 0x7fe31c0cd2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:10:26.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:26.487+0000 7fe32103b640 1 --2- 172.21.15.49:0/1657756344 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe31c15a880 0x7fe31c14dd00 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fe31400c480 tx=0x7fe31400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:26.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:26.487+0000 7fe3197fa640 1 -- 172.21.15.49:0/1657756344 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe314005cf0 con 0x7fe31c15a880 2023-11-06T02:10:26.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:26.487+0000 7fe3197fa640 1 -- 172.21.15.49:0/1657756344 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe314005e50 con 0x7fe31c15a880 2023-11-06T02:10:26.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:26.487+0000 7fe322a9e640 1 -- 172.21.15.49:0/1657756344 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe31c0cc210 con 0x7fe31c15a880 2023-11-06T02:10:26.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:26.487+0000 7fe3197fa640 1 -- 172.21.15.49:0/1657756344 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3140072b0 con 0x7fe31c15a880 2023-11-06T02:10:26.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:26.487+0000 7fe322a9e640 1 -- 172.21.15.49:0/1657756344 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe31c0cc7d0 con 0x7fe31c15a880 2023-11-06T02:10:26.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:26.487+0000 7fe322a9e640 1 -- 172.21.15.49:0/1657756344 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe31c0ccfd0 con 0x7fe31c15a880 2023-11-06T02:10:26.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:26.487+0000 7fe3197fa640 1 -- 172.21.15.49:0/1657756344 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fe314007410 con 0x7fe31c15a880 2023-11-06T02:10:26.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:26.487+0000 7fe3197fa640 1 --2- 172.21.15.49:0/1657756344 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe308065a60 0x7fe308067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:26.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:26.487+0000 7fe3197fa640 1 -- 172.21.15.49:0/1657756344 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fe314082480 con 0x7fe31c15a880 2023-11-06T02:10:26.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:26.487+0000 7fe32083a640 1 --2- 172.21.15.49:0/1657756344 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe308065a60 0x7fe308067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:26.493 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:26.487+0000 7fe32083a640 1 --2- 172.21.15.49:0/1657756344 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe308065a60 0x7fe308067f20 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe30c007fd0 tx=0x7fe30c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:26.493 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:26.491+0000 7fe3197fa640 1 -- 172.21.15.49:0/1657756344 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fe31c0ccfd0 con 0x7fe31c15a880 2023-11-06T02:10:26.648 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:26.647+0000 7fe322a9e640 1 -- 172.21.15.49:0/1657756344 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fe31c0ccfd0 con 0x7fe31c15a880 2023-11-06T02:10:26.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:26.655+0000 7fe3197fa640 1 -- 172.21.15.49:0/1657756344 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe31c0ccfd0 con 0x7fe31c15a880 2023-11-06T02:10:26.658 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:26.659+0000 7fe322a9e640 1 -- 172.21.15.49:0/1657756344 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe308065a60 msgr2=0x7fe308067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:26.659 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:26.659+0000 7fe322a9e640 1 --2- 172.21.15.49:0/1657756344 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe308065a60 0x7fe308067f20 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe30c007fd0 tx=0x7fe30c009040 comp rx=0 tx=0).stop 2023-11-06T02:10:26.659 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:26.659+0000 7fe322a9e640 1 -- 172.21.15.49:0/1657756344 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe31c15a880 msgr2=0x7fe31c14dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:26.659 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:26.659+0000 7fe322a9e640 1 --2- 172.21.15.49:0/1657756344 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe31c15a880 0x7fe31c14dd00 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fe31400c480 tx=0x7fe31400c940 comp rx=0 tx=0).stop 2023-11-06T02:10:26.659 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:26.659+0000 7fe322a9e640 1 -- 172.21.15.49:0/1657756344 shutdown_connections 2023-11-06T02:10:26.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:26.659+0000 7fe322a9e640 1 --2- 172.21.15.49:0/1657756344 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe308065a60 0x7fe308067f20 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:26.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:26.659+0000 7fe322a9e640 1 --2- 172.21.15.49:0/1657756344 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe31c15a880 0x7fe31c14dd00 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:26.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:26.659+0000 7fe322a9e640 1 --2- 172.21.15.49:0/1657756344 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe31c070870 0x7fe31c15c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:26.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:26.659+0000 7fe322a9e640 1 --2- 172.21.15.49:0/1657756344 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe31c06d150 0x7fe31c0cd2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:26.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:26.659+0000 7fe322a9e640 1 -- 172.21.15.49:0/1657756344 >> 172.21.15.49:0/1657756344 conn(0x7fe31c06a370 msgr2=0x7fe31c152220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:26.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:26.659+0000 7fe322a9e640 1 -- 172.21.15.49:0/1657756344 shutdown_connections 2023-11-06T02:10:26.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:26.659+0000 7fe322a9e640 1 -- 172.21.15.49:0/1657756344 wait complete. 2023-11-06T02:10:26.676 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-06T02:10:26.676 DEBUG:teuthology.orchestra.run.smithi049:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T02:10:26.735 DEBUG:teuthology.orchestra.run.smithi150:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T02:10:26.782 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-06T02:10:26.783 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-11-06T02:10:26.783 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T02:10:26.790 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336476k,nr_inodes=4084119,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16105 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-06T02:10:26.790 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-11-06T02:10:26.790 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T02:10:26.797 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336552k,nr_inodes=4084138,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19862 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274692k,nr_inodes=818673,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-06T02:10:26.797 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-11-06T02:10:26.797 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T02:10:26.837 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336476k,nr_inodes=4084119,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16105 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-06T02:10:26.838 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-11-06T02:10:26.838 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T02:10:26.845 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336552k,nr_inodes=4084138,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19862 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274692k,nr_inodes=818673,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-06T02:10:26.845 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-06T02:10:26.846 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T02:10:27.013 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.011+0000 7f33aa95d640 1 Processor -- start 2023-11-06T02:10:27.013 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.011+0000 7f33aa95d640 1 -- start start 2023-11-06T02:10:27.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.015+0000 7f33aa95d640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f33a40ca980 0x7f33a41644c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:27.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.015+0000 7f33aa95d640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33a40cad90 con 0x7f33a40cb010 2023-11-06T02:10:27.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.015+0000 7f33aa95d640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33a40723a0 con 0x7f33a4159b40 2023-11-06T02:10:27.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.015+0000 7f33aa95d640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33a40715e0 con 0x7f33a40ca980 2023-11-06T02:10:27.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.015+0000 7f33a3fff640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f33a40cb010 0x7f33a40cb3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:59784/0 (socket says 172.21.15.49:59784) 2023-11-06T02:10:27.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.015+0000 7f33a3fff640 1 -- 172.21.15.49:0/3627225778 learned_addr learned my addr 172.21.15.49:0/3627225778 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:10:27.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.015+0000 7f33a8efa640 1 --2- 172.21.15.49:0/3627225778 >> v2:172.21.15.150:3300/0 conn(0x7f33a40ca980 0x7f33a41644c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:27.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.015+0000 7f33a2ffd640 1 -- 172.21.15.49:0/3627225778 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2239851432 0 0) 0x7f33a40cad90 con 0x7f33a40cb010 2023-11-06T02:10:27.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.015+0000 7f33a2ffd640 1 -- 172.21.15.49:0/3627225778 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f338c003530 con 0x7f33a40cb010 2023-11-06T02:10:27.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.015+0000 7f33a2ffd640 1 -- 172.21.15.49:0/3627225778 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2727521899 0 0) 0x7f338c003530 con 0x7f33a40cb010 2023-11-06T02:10:27.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.015+0000 7f33a2ffd640 1 -- 172.21.15.49:0/3627225778 >> v2:172.21.15.150:3300/0 conn(0x7f33a40ca980 msgr2=0x7f33a41644c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:27.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.015+0000 7f33a2ffd640 1 --2- 172.21.15.49:0/3627225778 >> v2:172.21.15.150:3300/0 conn(0x7f33a40ca980 0x7f33a41644c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:27.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.015+0000 7f33a2ffd640 1 -- 172.21.15.49:0/3627225778 >> v1:172.21.15.150:6789/0 conn(0x7f33a4159b40 legacy=0x7f33a40cdd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:27.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.015+0000 7f33a2ffd640 1 -- 172.21.15.49:0/3627225778 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33a40718c0 con 0x7f33a40cb010 2023-11-06T02:10:27.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.015+0000 7f33a2ffd640 1 -- 172.21.15.49:0/3627225778 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7f3394001b80 con 0x7f33a40cb010 2023-11-06T02:10:27.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.015+0000 7f33a2ffd640 1 -- 172.21.15.49:0/3627225778 >> v1:172.21.15.49:6789/0 conn(0x7f33a40cb010 legacy=0x7f33a40cb3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:27.019 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.015+0000 7f33a2ffd640 1 --2- 172.21.15.49:0/3627225778 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f338c003980 0x7f338c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:27.019 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.015+0000 7f33a2ffd640 1 --2- 172.21.15.49:0/3627225778 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f338c0047d0 0x7f338c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:27.019 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.015+0000 7f33a2ffd640 1 --2- 172.21.15.49:0/3627225778 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f338c005190 0x7f338c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:27.019 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.015+0000 7f33a2ffd640 1 -- 172.21.15.49:0/3627225778 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a40715e0 con 0x7f338c0047d0 2023-11-06T02:10:27.019 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.015+0000 7f33a2ffd640 1 -- 172.21.15.49:0/3627225778 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f338c00c0e0 con 0x7f338c005190 2023-11-06T02:10:27.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.015+0000 7f33a2ffd640 1 -- 172.21.15.49:0/3627225778 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f338c00c2a0 con 0x7f338c003980 2023-11-06T02:10:27.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.015+0000 7f33a8efa640 1 --2- 172.21.15.49:0/3627225778 >> v2:172.21.15.150:3300/0 conn(0x7f33a40ca980 0x7f33a41644c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:10:27.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.015+0000 7f33a37fe640 1 --2- 172.21.15.49:0/3627225778 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f338c0047d0 0x7f338c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:27.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.015+0000 7f33a3fff640 1 --2- 172.21.15.49:0/3627225778 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f338c003980 0x7f338c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:27.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.015+0000 7f33a37fe640 1 -- 172.21.15.49:0/3627225778 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f338c003980 msgr2=0x7f338c003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:27.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.015+0000 7f33a8efa640 1 --2- 172.21.15.49:0/3627225778 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f338c005190 0x7f338c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:27.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.015+0000 7f33a37fe640 1 --2- 172.21.15.49:0/3627225778 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f338c003980 0x7f338c003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:27.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.015+0000 7f33a37fe640 1 -- 172.21.15.49:0/3627225778 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f338c005190 msgr2=0x7f338c00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:27.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.015+0000 7f33a37fe640 1 --2- 172.21.15.49:0/3627225778 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f338c005190 0x7f338c00b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:27.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.015+0000 7f33a37fe640 1 -- 172.21.15.49:0/3627225778 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f338c003530 con 0x7f338c0047d0 2023-11-06T02:10:27.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.015+0000 7f33a8efa640 1 -- 172.21.15.49:0/3627225778 reap_dead start 2023-11-06T02:10:27.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.015+0000 7f33a3fff640 1 --2- 172.21.15.49:0/3627225778 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f338c003980 0x7f338c003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:10:27.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.019+0000 7f33a37fe640 1 --2- 172.21.15.49:0/3627225778 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f338c0047d0 0x7f338c004c50 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f3398001750 tx=0x7f339801c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bc6663e8e924 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:27.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.019+0000 7f33a2ffd640 1 -- 172.21.15.49:0/3627225778 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33980052d0 con 0x7f338c0047d0 2023-11-06T02:10:27.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.019+0000 7f33a2ffd640 1 -- 172.21.15.49:0/3627225778 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33980a7050 con 0x7f338c0047d0 2023-11-06T02:10:27.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.019+0000 7f33a2ffd640 1 -- 172.21.15.49:0/3627225778 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3398003600 con 0x7f338c0047d0 2023-11-06T02:10:27.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.019+0000 7f33aa95d640 1 -- 172.21.15.49:0/3627225778 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f338c0047d0 msgr2=0x7f338c004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:27.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.019+0000 7f33aa95d640 1 --2- 172.21.15.49:0/3627225778 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f338c0047d0 0x7f338c004c50 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f3398001750 tx=0x7f339801c3a0 comp rx=0 tx=0).stop 2023-11-06T02:10:27.026 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.019+0000 7f33aa95d640 1 -- 172.21.15.49:0/3627225778 shutdown_connections 2023-11-06T02:10:27.026 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.019+0000 7f33aa95d640 1 --2- 172.21.15.49:0/3627225778 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f338c0047d0 0x7f338c004c50 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:27.026 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.019+0000 7f33aa95d640 1 -- 172.21.15.49:0/3627225778 >> 172.21.15.49:0/3627225778 conn(0x7f33a406c0a0 msgr2=0x7f33a406c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:27.026 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.019+0000 7f33aa95d640 1 -- 172.21.15.49:0/3627225778 shutdown_connections 2023-11-06T02:10:27.026 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.019+0000 7f33aa95d640 1 -- 172.21.15.49:0/3627225778 wait complete. 2023-11-06T02:10:27.027 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.019+0000 7f33aa95d640 1 Processor -- start 2023-11-06T02:10:27.027 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.019+0000 7f33aa95d640 1 -- start start 2023-11-06T02:10:27.027 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.019+0000 7f33aa95d640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f338c0047d0 0x7f33a41f19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:27.027 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.019+0000 7f33aa95d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33a40c56d0 0x7f33a415a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:27.027 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.019+0000 7f33aa95d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f33a40b1200 0x7f33a40cc600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:27.028 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.019+0000 7f33aa95d640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a40b17c0 con 0x7f33a40b1200 2023-11-06T02:10:27.028 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.019+0000 7f33aa95d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a40b1aa0 con 0x7f33a40c56d0 2023-11-06T02:10:27.028 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.019+0000 7f33aa95d640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33a40b1d80 con 0x7f338c0047d0 2023-11-06T02:10:27.028 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.019+0000 7f33a8efa640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f33a40b1200 0x7f33a40cc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:27.028 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.019+0000 7f33a8efa640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f33a40b1200 0x7f33a40cc600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46430/0 (socket says 172.21.15.49:46430) 2023-11-06T02:10:27.028 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.019+0000 7f33a8efa640 1 -- 172.21.15.49:0/1368803459 learned_addr learned my addr 172.21.15.49:0/1368803459 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:27.029 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.019+0000 7f33a3fff640 1 --2- 172.21.15.49:0/1368803459 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f338c0047d0 0x7f33a41f19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:27.029 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.019+0000 7f33a37fe640 1 --2- 172.21.15.49:0/1368803459 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33a40c56d0 0x7f33a415a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:27.029 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.019+0000 7f33a8efa640 1 -- 172.21.15.49:0/1368803459 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f338c0047d0 msgr2=0x7f33a41f19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:27.029 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.019+0000 7f33a8efa640 1 --2- 172.21.15.49:0/1368803459 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f338c0047d0 0x7f33a41f19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:27.029 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.019+0000 7f33a8efa640 1 -- 172.21.15.49:0/1368803459 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33a40c56d0 msgr2=0x7f33a415a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:27.030 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.019+0000 7f33a8efa640 1 --2- 172.21.15.49:0/1368803459 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33a40c56d0 0x7f33a415a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:27.030 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.019+0000 7f33a8efa640 1 -- 172.21.15.49:0/1368803459 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33a40b2060 con 0x7f33a40b1200 2023-11-06T02:10:27.030 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.023+0000 7f33a8efa640 1 --2- 172.21.15.49:0/1368803459 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f33a40b1200 0x7f33a40cc600 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f33a406add0 tx=0x7f339c00f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:27.030 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.023+0000 7f33a0ff9640 1 -- 172.21.15.49:0/1368803459 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f339c014680 con 0x7f33a40b1200 2023-11-06T02:10:27.030 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.023+0000 7f33aa95d640 1 -- 172.21.15.49:0/1368803459 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33a415bb80 con 0x7f33a40b1200 2023-11-06T02:10:27.031 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.023+0000 7f33a0ff9640 1 -- 172.21.15.49:0/1368803459 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f339c00c030 con 0x7f33a40b1200 2023-11-06T02:10:27.031 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.023+0000 7f33a0ff9640 1 -- 172.21.15.49:0/1368803459 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f339c014a00 con 0x7f33a40b1200 2023-11-06T02:10:27.031 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.023+0000 7f33aa95d640 1 -- 172.21.15.49:0/1368803459 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33a415a610 con 0x7f33a40b1200 2023-11-06T02:10:27.031 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.023+0000 7f33aa95d640 1 -- 172.21.15.49:0/1368803459 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33a40b22a0 con 0x7f33a40b1200 2023-11-06T02:10:27.031 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.023+0000 7f33a0ff9640 1 -- 172.21.15.49:0/1368803459 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f339c014b60 con 0x7f33a40b1200 2023-11-06T02:10:27.032 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.023+0000 7f33a0ff9640 1 --2- 172.21.15.49:0/1368803459 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3390065a90 0x7f3390067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:27.032 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.023+0000 7f33a0ff9640 1 -- 172.21.15.49:0/1368803459 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f339c082a30 con 0x7f33a40b1200 2023-11-06T02:10:27.032 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.023+0000 7f33a3fff640 1 --2- 172.21.15.49:0/1368803459 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3390065a90 0x7f3390067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:27.032 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.023+0000 7f33a3fff640 1 --2- 172.21.15.49:0/1368803459 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3390065a90 0x7f3390067f50 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3394001500 tx=0x7f3394009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:27.033 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.027+0000 7f33a0ff9640 1 -- 172.21.15.49:0/1368803459 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f33a40b22a0 con 0x7f33a40b1200 2023-11-06T02:10:27.216 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.215+0000 7f33aa95d640 1 -- 172.21.15.49:0/1368803459 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f33a40b22a0 con 0x7f33a40b1200 2023-11-06T02:10:27.217 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.219+0000 7f33a0ff9640 1 -- 172.21.15.49:0/1368803459 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+3529 (secure 0 0 0) 0x7f33a40b22a0 con 0x7f33a40b1200 2023-11-06T02:10:27.217 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:10:27.217 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":17,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5032,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/280428805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":280428805},{"type":"v1","addr":"172.21.15.150:6835","nonce":280428805}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":5035,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/2734589844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2734589844},{"type":"v1","addr":"172.21.15.49:6833","nonce":2734589844}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":5038,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/1319221246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":1319221246},{"type":"v1","addr":"172.21.15.150:6837","nonce":1319221246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":5808,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/586586695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":586586695},{"type":"v1","addr":"172.21.15.49:6835","nonce":586586695}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17}],"filesystems":[]} 2023-11-06T02:10:27.218 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 17 2023-11-06T02:10:27.219 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.219+0000 7f33aa95d640 1 -- 172.21.15.49:0/1368803459 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3390065a90 msgr2=0x7f3390067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:27.220 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.219+0000 7f33aa95d640 1 --2- 172.21.15.49:0/1368803459 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3390065a90 0x7f3390067f50 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3394001500 tx=0x7f3394009040 comp rx=0 tx=0).stop 2023-11-06T02:10:27.220 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.219+0000 7f33aa95d640 1 -- 172.21.15.49:0/1368803459 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f33a40b1200 msgr2=0x7f33a40cc600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:27.220 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.219+0000 7f33aa95d640 1 --2- 172.21.15.49:0/1368803459 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f33a40b1200 0x7f33a40cc600 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f33a406add0 tx=0x7f339c00f670 comp rx=0 tx=0).stop 2023-11-06T02:10:27.220 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.219+0000 7f33aa95d640 1 -- 172.21.15.49:0/1368803459 shutdown_connections 2023-11-06T02:10:27.220 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.219+0000 7f33aa95d640 1 --2- 172.21.15.49:0/1368803459 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3390065a90 0x7f3390067f50 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:27.220 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.219+0000 7f33aa95d640 1 --2- 172.21.15.49:0/1368803459 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f33a40b1200 0x7f33a40cc600 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:27.221 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.219+0000 7f33aa95d640 1 --2- 172.21.15.49:0/1368803459 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33a40c56d0 0x7f33a415a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:27.221 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.219+0000 7f33aa95d640 1 --2- 172.21.15.49:0/1368803459 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f338c0047d0 0x7f33a41f19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:27.221 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.219+0000 7f33aa95d640 1 -- 172.21.15.49:0/1368803459 >> 172.21.15.49:0/1368803459 conn(0x7f33a406c0a0 msgr2=0x7f33a40b3410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:27.221 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.219+0000 7f33aa95d640 1 -- 172.21.15.49:0/1368803459 shutdown_connections 2023-11-06T02:10:27.221 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.219+0000 7f33aa95d640 1 -- 172.21.15.49:0/1368803459 wait complete. 2023-11-06T02:10:27.236 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-06T02:10:27.236 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-06T02:10:27.236 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-06T02:10:27.236 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T02:10:27.237 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-06T02:10:27.237 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-06T02:10:27.237 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-06T02:10:27.238 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T02:10:27.238 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-06T02:10:27.238 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-06T02:10:27.238 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-06T02:10:27.239 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T02:10:27.239 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-06T02:10:27.239 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-06T02:10:27.239 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-06T02:10:27.240 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T02:10:27.247 INFO:tasks.ceph.mds.c:Stopped 2023-11-06T02:10:27.247 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-06T02:10:27.249 INFO:tasks.ceph.mds.a:Stopped 2023-11-06T02:10:27.249 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-06T02:10:27.257 INFO:tasks.ceph.mds.d:Stopped 2023-11-06T02:10:27.257 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-06T02:10:27.258 INFO:tasks.ceph.mds.b:Stopped 2023-11-06T02:10:27.259 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-06T02:10:27.293 INFO:tasks.ceph.mds.c:Started 2023-11-06T02:10:27.294 DEBUG:teuthology.parallel:result is None 2023-11-06T02:10:27.295 INFO:tasks.ceph.mds.a:Started 2023-11-06T02:10:27.295 DEBUG:teuthology.parallel:result is None 2023-11-06T02:10:27.302 INFO:tasks.ceph.mds.d:Started 2023-11-06T02:10:27.302 DEBUG:teuthology.parallel:result is None 2023-11-06T02:10:27.303 INFO:tasks.ceph.mds.b:Started 2023-11-06T02:10:27.304 DEBUG:teuthology.parallel:result is None 2023-11-06T02:10:27.304 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-06T02:10:27.354 INFO:tasks.ceph.mds.c.smithi049.stdout:starting mds.c at 2023-11-06T02:10:27.361 INFO:tasks.ceph.mds.a.smithi049.stdout:starting mds.a at 2023-11-06T02:10:27.376 INFO:tasks.ceph.mds.d.smithi150.stdout:starting mds.d at 2023-11-06T02:10:27.387 INFO:tasks.ceph.mds.b.smithi150.stdout:starting mds.b at 2023-11-06T02:10:27.422 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.423+0000 7f0e490fa640 1 Processor -- start 2023-11-06T02:10:27.422 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.423+0000 7f0e490fa640 1 -- start start 2023-11-06T02:10:27.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.423+0000 7f0e490fa640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f0e440b0d80 0x7f0e4406b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:27.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.423+0000 7f0e490fa640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f0e440b1420 0x7f0e4406b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:27.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.423+0000 7f0e490fa640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e440cf710 con 0x7f0e440b1970 2023-11-06T02:10:27.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.423+0000 7f0e490fa640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e440cffb0 con 0x7f0e440b1420 2023-11-06T02:10:27.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.423+0000 7f0e490fa640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e440cf9f0 con 0x7f0e440b0d80 2023-11-06T02:10:27.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.423+0000 7f0e4259c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f0e440b1420 0x7f0e4406b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:27.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.423+0000 7f0e4359e640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f0e440b1970 0x7f0e440cd280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:59790/0 (socket says 172.21.15.49:59790) 2023-11-06T02:10:27.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.423+0000 7f0e4259c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f0e440b1420 0x7f0e4406b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46470/0 (socket says 172.21.15.49:46470) 2023-11-06T02:10:27.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.423+0000 7f0e4359e640 1 -- 172.21.15.49:0/4078685396 learned_addr learned my addr 172.21.15.49:0/4078685396 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:10:27.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.423+0000 7f0e42d9d640 1 --2- 172.21.15.49:0/4078685396 >> v2:172.21.15.150:3300/0 conn(0x7f0e440b0d80 0x7f0e4406b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:27.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.423+0000 7f0e4259c640 1 -- 172.21.15.49:0/4078685396 >> v2:172.21.15.150:3300/0 conn(0x7f0e440b0d80 msgr2=0x7f0e4406b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:27.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.423+0000 7f0e4259c640 1 --2- 172.21.15.49:0/4078685396 >> v2:172.21.15.150:3300/0 conn(0x7f0e440b0d80 0x7f0e4406b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:27.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.423+0000 7f0e4259c640 1 -- 172.21.15.49:0/4078685396 >> v1:172.21.15.49:6789/0 conn(0x7f0e440b1970 legacy=0x7f0e440cd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:27.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.423+0000 7f0e41d9b640 1 -- 172.21.15.49:0/4078685396 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2384674098 0 0) 0x7f0e440cf710 con 0x7f0e440b1970 2023-11-06T02:10:27.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.423+0000 7f0e4259c640 1 -- 172.21.15.49:0/4078685396 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e440cfcd0 con 0x7f0e440b1420 2023-11-06T02:10:27.429 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.423+0000 7f0e4259c640 1 --2- 172.21.15.49:0/4078685396 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e440b1420 0x7f0e4406b5d0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f0e44071fd0 tx=0x7f0e3401c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c62816c1cb44da2 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:27.429 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.427+0000 7f0e41d9b640 1 -- 172.21.15.49:0/4078685396 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e340ab5f0 con 0x7f0e440b1420 2023-11-06T02:10:27.429 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.427+0000 7f0e41d9b640 1 -- 172.21.15.49:0/4078685396 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e340ab750 con 0x7f0e440b1420 2023-11-06T02:10:27.430 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.427+0000 7f0e41d9b640 1 -- 172.21.15.49:0/4078685396 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e340abae0 con 0x7f0e440b1420 2023-11-06T02:10:27.430 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.427+0000 7f0e490fa640 1 -- 172.21.15.49:0/4078685396 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e440b1420 msgr2=0x7f0e4406b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:27.430 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.427+0000 7f0e490fa640 1 --2- 172.21.15.49:0/4078685396 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e440b1420 0x7f0e4406b5d0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f0e44071fd0 tx=0x7f0e3401c3d0 comp rx=0 tx=0).stop 2023-11-06T02:10:27.430 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.427+0000 7f0e490fa640 1 -- 172.21.15.49:0/4078685396 shutdown_connections 2023-11-06T02:10:27.430 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.427+0000 7f0e490fa640 1 --2- 172.21.15.49:0/4078685396 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e440b1420 0x7f0e4406b5d0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:27.431 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.427+0000 7f0e490fa640 1 --2- 172.21.15.49:0/4078685396 >> v2:172.21.15.150:3300/0 conn(0x7f0e440b0d80 0x7f0e4406b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:27.431 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.427+0000 7f0e490fa640 1 -- 172.21.15.49:0/4078685396 >> 172.21.15.49:0/4078685396 conn(0x7f0e440d0890 msgr2=0x7f0e440d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:27.431 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.427+0000 7f0e490fa640 1 -- 172.21.15.49:0/4078685396 shutdown_connections 2023-11-06T02:10:27.431 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.427+0000 7f0e490fa640 1 -- 172.21.15.49:0/4078685396 wait complete. 2023-11-06T02:10:27.431 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.427+0000 7f0e490fa640 1 Processor -- start 2023-11-06T02:10:27.432 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.427+0000 7f0e490fa640 1 -- start start 2023-11-06T02:10:27.432 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.427+0000 7f0e490fa640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e440b0d80 0x7f0e440aef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:27.432 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.427+0000 7f0e490fa640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e440b1420 0x7f0e440c98d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:27.432 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.427+0000 7f0e490fa640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0e440b1970 0x7f0e440ad400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:27.432 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.427+0000 7f0e490fa640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e440adec0 con 0x7f0e440b0d80 2023-11-06T02:10:27.433 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.427+0000 7f0e490fa640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e440ae1a0 con 0x7f0e440b1420 2023-11-06T02:10:27.433 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.427+0000 7f0e490fa640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e440ae480 con 0x7f0e440b1970 2023-11-06T02:10:27.433 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.427+0000 7f0e42d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e440b0d80 0x7f0e440aef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:27.433 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.427+0000 7f0e42d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e440b0d80 0x7f0e440aef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46474/0 (socket says 172.21.15.49:46474) 2023-11-06T02:10:27.433 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.427+0000 7f0e42d9d640 1 -- 172.21.15.49:0/3671107893 learned_addr learned my addr 172.21.15.49:0/3671107893 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:27.434 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.427+0000 7f0e4259c640 1 --2- 172.21.15.49:0/3671107893 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e440b1420 0x7f0e440c98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:27.434 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.427+0000 7f0e4359e640 1 --2- 172.21.15.49:0/3671107893 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0e440b1970 0x7f0e440ad400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:27.434 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.427+0000 7f0e42d9d640 1 -- 172.21.15.49:0/3671107893 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0e440b1970 msgr2=0x7f0e440ad400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:27.434 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.427+0000 7f0e42d9d640 1 --2- 172.21.15.49:0/3671107893 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0e440b1970 0x7f0e440ad400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:27.434 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.427+0000 7f0e42d9d640 1 -- 172.21.15.49:0/3671107893 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e440b1420 msgr2=0x7f0e440c98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:27.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.427+0000 7f0e42d9d640 1 --2- 172.21.15.49:0/3671107893 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e440b1420 0x7f0e440c98d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:27.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.427+0000 7f0e42d9d640 1 -- 172.21.15.49:0/3671107893 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e34009420 con 0x7f0e440b0d80 2023-11-06T02:10:27.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.427+0000 7f0e4259c640 1 --2- 172.21.15.49:0/3671107893 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e440b1420 0x7f0e440c98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:10:27.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.427+0000 7f0e42d9d640 1 --2- 172.21.15.49:0/3671107893 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e440b0d80 0x7f0e440aef60 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c00c7f0 tx=0x7f0e2c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:27.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.427+0000 7f0e3b7fe640 1 -- 172.21.15.49:0/3671107893 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e2c004220 con 0x7f0e440b0d80 2023-11-06T02:10:27.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.427+0000 7f0e3b7fe640 1 -- 172.21.15.49:0/3671107893 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e2c0043a0 con 0x7f0e440b0d80 2023-11-06T02:10:27.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.427+0000 7f0e490fa640 1 -- 172.21.15.49:0/3671107893 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e440ae760 con 0x7f0e440b0d80 2023-11-06T02:10:27.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.427+0000 7f0e3b7fe640 1 -- 172.21.15.49:0/3671107893 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e2c004510 con 0x7f0e440b0d80 2023-11-06T02:10:27.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.427+0000 7f0e490fa640 1 -- 172.21.15.49:0/3671107893 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e440ca410 con 0x7f0e440b0d80 2023-11-06T02:10:27.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.431+0000 7f0e490fa640 1 -- 172.21.15.49:0/3671107893 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e440aec80 con 0x7f0e440b0d80 2023-11-06T02:10:27.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.431+0000 7f0e3b7fe640 1 -- 172.21.15.49:0/3671107893 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f0e2c003a70 con 0x7f0e440b0d80 2023-11-06T02:10:27.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.431+0000 7f0e3b7fe640 1 --2- 172.21.15.49:0/3671107893 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0e24065a60 0x7f0e24067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:27.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.431+0000 7f0e3b7fe640 1 -- 172.21.15.49:0/3671107893 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f0e2c082fe0 con 0x7f0e440b0d80 2023-11-06T02:10:27.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.431+0000 7f0e4259c640 1 --2- 172.21.15.49:0/3671107893 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0e24065a60 0x7f0e24067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:27.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.431+0000 7f0e4259c640 1 --2- 172.21.15.49:0/3671107893 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0e24065a60 0x7f0e24067f20 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0e340af000 tx=0x7f0e340a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:27.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.435+0000 7f0e3b7fe640 1 -- 172.21.15.49:0/3671107893 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f0e2c050b40 con 0x7f0e440b0d80 2023-11-06T02:10:27.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:27.603+0000 7f0e490fa640 1 -- 172.21.15.49:0/3671107893 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f0e44074040 con 0x7f0e440b0d80 2023-11-06T02:10:28.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.415+0000 7f0e3b7fe640 1 -- 172.21.15.49:0/3671107893 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v47) v1 ==== 97+0+0 (secure 0 0 0) 0x7f0e2c01e080 con 0x7f0e440b0d80 2023-11-06T02:10:28.415 INFO:teuthology.orchestra.run.smithi049.stderr: removed all blocklist entries 2023-11-06T02:10:28.419 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.419+0000 7f0e490fa640 1 -- 172.21.15.49:0/3671107893 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0e24065a60 msgr2=0x7f0e24067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:28.419 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.419+0000 7f0e490fa640 1 --2- 172.21.15.49:0/3671107893 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0e24065a60 0x7f0e24067f20 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0e340af000 tx=0x7f0e340a7000 comp rx=0 tx=0).stop 2023-11-06T02:10:28.419 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.419+0000 7f0e490fa640 1 -- 172.21.15.49:0/3671107893 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e440b0d80 msgr2=0x7f0e440aef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:28.419 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.419+0000 7f0e490fa640 1 --2- 172.21.15.49:0/3671107893 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e440b0d80 0x7f0e440aef60 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c00c7f0 tx=0x7f0e2c00ccb0 comp rx=0 tx=0).stop 2023-11-06T02:10:28.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.419+0000 7f0e490fa640 1 -- 172.21.15.49:0/3671107893 shutdown_connections 2023-11-06T02:10:28.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.419+0000 7f0e490fa640 1 --2- 172.21.15.49:0/3671107893 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0e24065a60 0x7f0e24067f20 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:28.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.419+0000 7f0e490fa640 1 --2- 172.21.15.49:0/3671107893 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0e440b1970 0x7f0e440ad400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:28.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.419+0000 7f0e490fa640 1 --2- 172.21.15.49:0/3671107893 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e440b1420 0x7f0e440c98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:28.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.419+0000 7f0e490fa640 1 --2- 172.21.15.49:0/3671107893 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e440b0d80 0x7f0e440aef60 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:28.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.419+0000 7f0e490fa640 1 -- 172.21.15.49:0/3671107893 >> 172.21.15.49:0/3671107893 conn(0x7f0e440d0890 msgr2=0x7f0e4406f970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:28.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.419+0000 7f0e490fa640 1 -- 172.21.15.49:0/3671107893 shutdown_connections 2023-11-06T02:10:28.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.419+0000 7f0e490fa640 1 -- 172.21.15.49:0/3671107893 wait complete. 2023-11-06T02:10:28.442 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-06T02:10:28.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.567+0000 7f6bf9949640 1 Processor -- start 2023-11-06T02:10:28.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.567+0000 7f6bf9949640 1 -- start start 2023-11-06T02:10:28.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.567+0000 7f6bf9949640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f6bf40cda70 0x7f6bf4161650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:28.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.567+0000 7f6bf9949640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bf40d17c0 con 0x7f6bf40714a0 2023-11-06T02:10:28.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.567+0000 7f6bf9949640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bf40ce6c0 con 0x7f6bf40b1f90 2023-11-06T02:10:28.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.567+0000 7f6bf9949640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bf40cdf20 con 0x7f6bf40cda70 2023-11-06T02:10:28.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.567+0000 7f6bf27fc640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f6bf40cda70 0x7f6bf4161650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:28.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.567+0000 7f6bf27fc640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f6bf40cda70 0x7f6bf4161650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43898/0 (socket says 172.21.15.49:43898) 2023-11-06T02:10:28.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.567+0000 7f6bf27fc640 1 -- 172.21.15.49:0/922138168 learned_addr learned my addr 172.21.15.49:0/922138168 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:28.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.567+0000 7f6bf37fe640 1 --1- 172.21.15.49:0/922138168 >> v1:172.21.15.49:6789/0 conn(0x7f6bf40714a0 0x7f6bf40718b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:41506/0 (socket says 172.21.15.49:41506) 2023-11-06T02:10:28.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.567+0000 7f6bf27fc640 1 -- 172.21.15.49:0/922138168 >> v1:172.21.15.150:6789/0 conn(0x7f6bf40b1f90 legacy=0x7f6bf40d1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:28.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.567+0000 7f6bf27fc640 1 -- 172.21.15.49:0/922138168 >> v1:172.21.15.49:6789/0 conn(0x7f6bf40714a0 legacy=0x7f6bf40718b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:28.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.567+0000 7f6bf27fc640 1 -- 172.21.15.49:0/922138168 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bf40d2360 con 0x7f6bf40cda70 2023-11-06T02:10:28.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.571+0000 7f6bf27fc640 1 --2- 172.21.15.49:0/922138168 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6bf40cda70 0x7f6bf4161650 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc009e50 tx=0x7f6bdc01c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17c9de062cf7db6f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:28.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.571+0000 7f6bf1ffb640 1 -- 172.21.15.49:0/922138168 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bdc0a5780 con 0x7f6bf40cda70 2023-11-06T02:10:28.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.571+0000 7f6bf1ffb640 1 -- 172.21.15.49:0/922138168 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bdc0a58e0 con 0x7f6bf40cda70 2023-11-06T02:10:28.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.571+0000 7f6bf1ffb640 1 -- 172.21.15.49:0/922138168 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bdc0a5c70 con 0x7f6bf40cda70 2023-11-06T02:10:28.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.571+0000 7f6bf9949640 1 -- 172.21.15.49:0/922138168 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6bf40cda70 msgr2=0x7f6bf4161650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:28.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.571+0000 7f6bf9949640 1 --2- 172.21.15.49:0/922138168 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6bf40cda70 0x7f6bf4161650 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc009e50 tx=0x7f6bdc01c9d0 comp rx=0 tx=0).stop 2023-11-06T02:10:28.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.571+0000 7f6bf9949640 1 -- 172.21.15.49:0/922138168 shutdown_connections 2023-11-06T02:10:28.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.571+0000 7f6bf9949640 1 --2- 172.21.15.49:0/922138168 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6bf40cda70 0x7f6bf4161650 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:28.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.571+0000 7f6bf9949640 1 -- 172.21.15.49:0/922138168 >> 172.21.15.49:0/922138168 conn(0x7f6bf40d2aa0 msgr2=0x7f6bf40d2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:28.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.571+0000 7f6bf9949640 1 -- 172.21.15.49:0/922138168 shutdown_connections 2023-11-06T02:10:28.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.571+0000 7f6bf9949640 1 -- 172.21.15.49:0/922138168 wait complete. 2023-11-06T02:10:28.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.571+0000 7f6bf9949640 1 Processor -- start 2023-11-06T02:10:28.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.571+0000 7f6bf9949640 1 -- start start 2023-11-06T02:10:28.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.571+0000 7f6bf9949640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6bf40714a0 0x7f6bf4144a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:28.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.571+0000 7f6bf9949640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6bf40b1f90 0x7f6bf415b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:28.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.571+0000 7f6bf9949640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6bf40cda70 0x7f6bf415a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:28.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.571+0000 7f6bf9949640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bf415b860 con 0x7f6bf40b1f90 2023-11-06T02:10:28.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.571+0000 7f6bf9949640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bf4145100 con 0x7f6bf40714a0 2023-11-06T02:10:28.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.571+0000 7f6bf9949640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bf41453e0 con 0x7f6bf40cda70 2023-11-06T02:10:28.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.571+0000 7f6bf27fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6bf40b1f90 0x7f6bf415b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:28.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.571+0000 7f6bf27fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6bf40b1f90 0x7f6bf415b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43900/0 (socket says 172.21.15.49:43900) 2023-11-06T02:10:28.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.571+0000 7f6bf27fc640 1 -- 172.21.15.49:0/3735611761 learned_addr learned my addr 172.21.15.49:0/3735611761 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:28.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.571+0000 7f6bf27fc640 1 -- 172.21.15.49:0/3735611761 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6bf40cda70 msgr2=0x7f6bf415a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:28.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.571+0000 7f6bf27fc640 1 --2- 172.21.15.49:0/3735611761 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6bf40cda70 0x7f6bf415a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:28.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.571+0000 7f6bf27fc640 1 -- 172.21.15.49:0/3735611761 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6bf40714a0 msgr2=0x7f6bf4144a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:28.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.571+0000 7f6bf27fc640 1 --2- 172.21.15.49:0/3735611761 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6bf40714a0 0x7f6bf4144a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:28.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.571+0000 7f6bf27fc640 1 -- 172.21.15.49:0/3735611761 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bdc009b30 con 0x7f6bf40b1f90 2023-11-06T02:10:28.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.575+0000 7f6bf27fc640 1 --2- 172.21.15.49:0/3735611761 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6bf40b1f90 0x7f6bf415b320 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc0195e0 tx=0x7f6bdc019770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:28.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.575+0000 7f6bebfff640 1 -- 172.21.15.49:0/3735611761 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bdc0bd070 con 0x7f6bf40b1f90 2023-11-06T02:10:28.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.575+0000 7f6bebfff640 1 -- 172.21.15.49:0/3735611761 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bdc0b8470 con 0x7f6bf40b1f90 2023-11-06T02:10:28.579 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.575+0000 7f6bf9949640 1 -- 172.21.15.49:0/3735611761 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bf41456c0 con 0x7f6bf40b1f90 2023-11-06T02:10:28.579 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.575+0000 7f6bebfff640 1 -- 172.21.15.49:0/3735611761 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bdc0b8820 con 0x7f6bf40b1f90 2023-11-06T02:10:28.579 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.575+0000 7f6bf9949640 1 -- 172.21.15.49:0/3735611761 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bf4145c80 con 0x7f6bf40b1f90 2023-11-06T02:10:28.579 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.575+0000 7f6bf9949640 1 -- 172.21.15.49:0/3735611761 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bf4074040 con 0x7f6bf40b1f90 2023-11-06T02:10:28.580 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.575+0000 7f6bebfff640 1 -- 172.21.15.49:0/3735611761 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f6bdc0b8980 con 0x7f6bf40b1f90 2023-11-06T02:10:28.580 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.575+0000 7f6bebfff640 1 --2- 172.21.15.49:0/3735611761 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6bd4065a60 0x7f6bd4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:28.580 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.575+0000 7f6bebfff640 1 -- 172.21.15.49:0/3735611761 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f6bdc11e6e0 con 0x7f6bf40b1f90 2023-11-06T02:10:28.581 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.575+0000 7f6bf2ffd640 1 --2- 172.21.15.49:0/3735611761 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6bd4065a60 0x7f6bd4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:28.581 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.579+0000 7f6bf2ffd640 1 --2- 172.21.15.49:0/3735611761 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6bd4065a60 0x7f6bd4067f20 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6be4007fd0 tx=0x7f6be40079a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:28.581 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.579+0000 7f6bebfff640 1 -- 172.21.15.49:0/3735611761 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f6bdc0ec400 con 0x7f6bf40b1f90 2023-11-06T02:10:28.791 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.791+0000 7f6bf9949640 1 -- 172.21.15.49:0/3735611761 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f6bf4168da0 con 0x7f6bf40b1f90 2023-11-06T02:10:28.792 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.791+0000 7f6bebfff640 1 -- 172.21.15.49:0/3735611761 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v11) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f6bdc0cae60 con 0x7f6bf40b1f90 2023-11-06T02:10:28.792 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:10:28.792 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-11-06T02:10:28.793 INFO:teuthology.orchestra.run.smithi049.stdout: "auth_dump": [ 2023-11-06T02:10:28.793 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:10:28.793 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "mds.a", 2023-11-06T02:10:28.793 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQBbSkhlbRFwIhAApA6xaqdIIeKDsOsNhyWCaA==", 2023-11-06T02:10:28.793 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-11-06T02:10:28.794 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": "allow", 2023-11-06T02:10:28.794 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow *", 2023-11-06T02:10:28.794 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow *", 2023-11-06T02:10:28.794 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-11-06T02:10:28.794 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:10:28.795 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:10:28.795 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:10:28.795 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "mds.b", 2023-11-06T02:10:28.795 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQBbSkhl6QhcMRAAgmOxZWghmuu11F0Gfut3sQ==", 2023-11-06T02:10:28.795 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-11-06T02:10:28.795 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": "allow", 2023-11-06T02:10:28.796 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow *", 2023-11-06T02:10:28.796 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow *", 2023-11-06T02:10:28.796 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-11-06T02:10:28.796 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:10:28.796 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:10:28.797 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:10:28.797 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "mds.c", 2023-11-06T02:10:28.797 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQBbSkhlMoZHKxAATV62bAItdXt3ShpsT4oVbA==", 2023-11-06T02:10:28.797 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-11-06T02:10:28.797 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": "allow", 2023-11-06T02:10:28.798 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow *", 2023-11-06T02:10:28.798 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow *", 2023-11-06T02:10:28.798 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-11-06T02:10:28.798 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:10:28.798 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:10:28.799 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:10:28.799 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "mds.d", 2023-11-06T02:10:28.799 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQBbSkhl376GORAAS2NIQJMzg6Es0JCYPak6fw==", 2023-11-06T02:10:28.799 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-11-06T02:10:28.799 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": "allow", 2023-11-06T02:10:28.799 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow *", 2023-11-06T02:10:28.800 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow *", 2023-11-06T02:10:28.800 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-11-06T02:10:28.800 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:10:28.800 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:10:28.800 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:10:28.800 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "osd.0", 2023-11-06T02:10:28.801 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQBdSkhlxuQtFRAAPttjVsYpkwexC5x2Pu2b+Q==", 2023-11-06T02:10:28.801 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-11-06T02:10:28.801 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow profile osd", 2023-11-06T02:10:28.801 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow profile osd", 2023-11-06T02:10:28.801 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-11-06T02:10:28.801 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:10:28.802 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:10:28.802 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:10:28.802 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "osd.1", 2023-11-06T02:10:28.802 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQBgSkhlCGeiHBAA8SofNV6C1+RQNajUsewVpw==", 2023-11-06T02:10:28.802 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-11-06T02:10:28.803 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow profile osd", 2023-11-06T02:10:28.803 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow profile osd", 2023-11-06T02:10:28.803 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-11-06T02:10:28.803 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:10:28.803 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:10:28.803 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:10:28.804 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "osd.2", 2023-11-06T02:10:28.804 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQBjSkhlBju+JBAAPJeIFngKDoamUvEjv1h8tw==", 2023-11-06T02:10:28.804 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-11-06T02:10:28.804 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow profile osd", 2023-11-06T02:10:28.804 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow profile osd", 2023-11-06T02:10:28.805 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-11-06T02:10:28.805 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:10:28.805 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:10:28.805 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:10:28.805 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "osd.3", 2023-11-06T02:10:28.805 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQBmSkhlMclTLRAABnbmQA9j3TLn40H1wYFx5g==", 2023-11-06T02:10:28.806 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-11-06T02:10:28.806 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow profile osd", 2023-11-06T02:10:28.806 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow profile osd", 2023-11-06T02:10:28.806 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-11-06T02:10:28.806 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:10:28.807 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:10:28.807 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:10:28.807 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "osd.4", 2023-11-06T02:10:28.807 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQBsSkhl7fmmLhAArv62qbJ+StgYMKhCftM5HA==", 2023-11-06T02:10:28.807 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-11-06T02:10:28.808 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow profile osd", 2023-11-06T02:10:28.808 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow profile osd", 2023-11-06T02:10:28.808 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-11-06T02:10:28.808 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:10:28.808 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:10:28.809 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:10:28.809 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "osd.5", 2023-11-06T02:10:28.809 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQBvSkhl5pAANhAArNFy9/YuEcXTdzM5XL46Xw==", 2023-11-06T02:10:28.809 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-11-06T02:10:28.809 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow profile osd", 2023-11-06T02:10:28.810 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow profile osd", 2023-11-06T02:10:28.810 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-11-06T02:10:28.810 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:10:28.810 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:10:28.810 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:10:28.810 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "osd.6", 2023-11-06T02:10:28.811 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQBzSkhlgfs2BBAAeHXVJ/tFiFagugkynGakFw==", 2023-11-06T02:10:28.811 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-11-06T02:10:28.811 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow profile osd", 2023-11-06T02:10:28.811 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow profile osd", 2023-11-06T02:10:28.811 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-11-06T02:10:28.812 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:10:28.812 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:10:28.812 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:10:28.812 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "osd.7", 2023-11-06T02:10:28.812 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQB2SkhlQbCYDRAAhE7vwbjJjqZDHfgDfdy1cA==", 2023-11-06T02:10:28.813 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-11-06T02:10:28.813 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow profile osd", 2023-11-06T02:10:28.813 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow profile osd", 2023-11-06T02:10:28.813 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-11-06T02:10:28.813 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:10:28.814 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:10:28.814 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:10:28.814 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "client.0", 2023-11-06T02:10:28.814 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQBcSkhlY46mAxAALClTH91L6USom3nVss6iiQ==", 2023-11-06T02:10:28.814 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-11-06T02:10:28.815 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": "allow", 2023-11-06T02:10:28.815 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow r", 2023-11-06T02:10:28.815 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-06T02:10:28.815 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:10:28.815 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:10:28.815 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:10:28.816 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "client.1", 2023-11-06T02:10:28.816 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQBcSkhllvSyCxAAfcSiBhHHNgaIr9mhpY11fg==", 2023-11-06T02:10:28.816 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-11-06T02:10:28.816 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": "allow", 2023-11-06T02:10:28.816 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow r", 2023-11-06T02:10:28.817 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow rw", 2023-11-06T02:10:28.817 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow rwx" 2023-11-06T02:10:28.817 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:10:28.817 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:10:28.817 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:10:28.817 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "client.2", 2023-11-06T02:10:28.818 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQBcSkhlURf6CBAA9g2y/UMD4x42IT0fgSgIcg==", 2023-11-06T02:10:28.818 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-11-06T02:10:28.818 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": "allow", 2023-11-06T02:10:28.818 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow r", 2023-11-06T02:10:28.818 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow rw", 2023-11-06T02:10:28.819 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow rwx" 2023-11-06T02:10:28.819 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:10:28.819 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:10:28.819 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:10:28.819 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "client.3", 2023-11-06T02:10:28.819 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQBcSkhlbjXXEBAA0xQGCY/avAWL18GLQtIjgg==", 2023-11-06T02:10:28.820 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-11-06T02:10:28.820 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": "allow", 2023-11-06T02:10:28.820 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow r", 2023-11-06T02:10:28.820 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow rw", 2023-11-06T02:10:28.820 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow rwx" 2023-11-06T02:10:28.821 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:10:28.821 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:10:28.821 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:10:28.821 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "client.admin", 2023-11-06T02:10:28.821 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQBbSkhloSJXCxAA++ZHscsqHENk8soyqq/GPQ==", 2023-11-06T02:10:28.821 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-11-06T02:10:28.822 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": "allow *", 2023-11-06T02:10:28.822 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow *", 2023-11-06T02:10:28.822 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow *", 2023-11-06T02:10:28.822 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-11-06T02:10:28.822 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:10:28.823 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:10:28.823 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:10:28.823 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "mgr.x", 2023-11-06T02:10:28.823 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQBbSkhlpJ4iHxAAGEz4XS5fILy7/NZbAenPFw==", 2023-11-06T02:10:28.823 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-11-06T02:10:28.824 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": "allow *", 2023-11-06T02:10:28.824 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow profile mgr", 2023-11-06T02:10:28.824 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-11-06T02:10:28.824 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:10:28.824 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:10:28.825 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:10:28.825 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "mgr.y", 2023-11-06T02:10:28.825 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQBbSkhln1JuHBAAymHRkniPpsI6pf9L2C2y+g==", 2023-11-06T02:10:28.825 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-11-06T02:10:28.825 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": "allow *", 2023-11-06T02:10:28.825 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow profile mgr", 2023-11-06T02:10:28.826 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-11-06T02:10:28.826 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:10:28.826 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:10:28.826 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2023-11-06T02:10:28.826 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-11-06T02:10:28.830 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.795+0000 7f6bf9949640 1 -- 172.21.15.49:0/3735611761 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6bd4065a60 msgr2=0x7f6bd4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:28.830 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.795+0000 7f6bf9949640 1 --2- 172.21.15.49:0/3735611761 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6bd4065a60 0x7f6bd4067f20 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6be4007fd0 tx=0x7f6be40079a0 comp rx=0 tx=0).stop 2023-11-06T02:10:28.830 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.795+0000 7f6bf9949640 1 -- 172.21.15.49:0/3735611761 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6bf40b1f90 msgr2=0x7f6bf415b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:28.831 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.795+0000 7f6bf9949640 1 --2- 172.21.15.49:0/3735611761 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6bf40b1f90 0x7f6bf415b320 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc0195e0 tx=0x7f6bdc019770 comp rx=0 tx=0).stop 2023-11-06T02:10:28.831 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.795+0000 7f6bf9949640 1 -- 172.21.15.49:0/3735611761 shutdown_connections 2023-11-06T02:10:28.831 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.795+0000 7f6bf9949640 1 --2- 172.21.15.49:0/3735611761 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6bd4065a60 0x7f6bd4067f20 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:28.831 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.795+0000 7f6bf9949640 1 --2- 172.21.15.49:0/3735611761 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6bf40b1f90 0x7f6bf415b320 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:28.831 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.795+0000 7f6bf9949640 1 --2- 172.21.15.49:0/3735611761 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6bf40cda70 0x7f6bf415a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:28.832 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.795+0000 7f6bf9949640 1 --2- 172.21.15.49:0/3735611761 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6bf40714a0 0x7f6bf4144a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:28.832 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.795+0000 7f6bf9949640 1 -- 172.21.15.49:0/3735611761 >> 172.21.15.49:0/3735611761 conn(0x7f6bf40d2aa0 msgr2=0x7f6bf40c6520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:28.832 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.795+0000 7f6bf9949640 1 -- 172.21.15.49:0/3735611761 shutdown_connections 2023-11-06T02:10:28.832 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.795+0000 7f6bf9949640 1 -- 172.21.15.49:0/3735611761 wait complete. 2023-11-06T02:10:28.833 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T02:10:28.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.959+0000 7fc9ceb0d640 1 Processor -- start 2023-11-06T02:10:28.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.959+0000 7fc9ceb0d640 1 -- start start 2023-11-06T02:10:28.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.963+0000 7fc9ceb0d640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc9c80b0790 0x7fc9c80c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:28.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.963+0000 7fc9ceb0d640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc9c80cb010 0x7fc9c80c2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:28.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.963+0000 7fc9ceb0d640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9c80cab80 con 0x7fc9c815ab50 2023-11-06T02:10:28.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.963+0000 7fc9ceb0d640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9c815af30 con 0x7fc9c80cb010 2023-11-06T02:10:28.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.963+0000 7fc9ceb0d640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c80cb5b0 con 0x7fc9c80b0790 2023-11-06T02:10:28.964 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.963+0000 7fc9cc8a9640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fc9c815ab50 0x7fc9c80b0080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:41522/0 (socket says 172.21.15.49:41522) 2023-11-06T02:10:28.964 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.963+0000 7fc9cc8a9640 1 -- 172.21.15.49:0/1712953352 learned_addr learned my addr 172.21.15.49:0/1712953352 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:10:28.964 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.963+0000 7fc9cd0aa640 1 --2- 172.21.15.49:0/1712953352 >> v2:172.21.15.49:3300/0 conn(0x7fc9c80cb010 0x7fc9c80c2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:28.964 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.963+0000 7fc9c7fff640 1 --2- 172.21.15.49:0/1712953352 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc9c80b0790 0x7fc9c80c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:28.964 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.963+0000 7fc9c77fe640 1 -- 172.21.15.49:0/1712953352 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1804934842 0 0) 0x7fc9c80cab80 con 0x7fc9c815ab50 2023-11-06T02:10:28.965 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.963+0000 7fc9c77fe640 1 -- 172.21.15.49:0/1712953352 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9b4003530 con 0x7fc9c815ab50 2023-11-06T02:10:28.965 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.963+0000 7fc9cd0aa640 1 -- 172.21.15.49:0/1712953352 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc9c80b0790 msgr2=0x7fc9c80c9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:28.965 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.963+0000 7fc9cd0aa640 1 --2- 172.21.15.49:0/1712953352 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc9c80b0790 0x7fc9c80c9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:28.965 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.963+0000 7fc9cd0aa640 1 -- 172.21.15.49:0/1712953352 >> v1:172.21.15.49:6789/0 conn(0x7fc9c815ab50 legacy=0x7fc9c80b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:28.965 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.963+0000 7fc9cd0aa640 1 -- 172.21.15.49:0/1712953352 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9c80cb890 con 0x7fc9c80cb010 2023-11-06T02:10:28.966 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.967+0000 7fc9cd0aa640 1 --2- 172.21.15.49:0/1712953352 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc9c80cb010 0x7fc9c80c2710 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fc9c00097c0 tx=0x7fc9c001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45bb344af3e0ccce server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:28.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.967+0000 7fc9c77fe640 1 -- 172.21.15.49:0/1712953352 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c00ac620 con 0x7fc9c80cb010 2023-11-06T02:10:28.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.967+0000 7fc9c77fe640 1 -- 172.21.15.49:0/1712953352 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9c00ac780 con 0x7fc9c80cb010 2023-11-06T02:10:28.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.967+0000 7fc9c77fe640 1 -- 172.21.15.49:0/1712953352 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c00acaf0 con 0x7fc9c80cb010 2023-11-06T02:10:28.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.967+0000 7fc9ceb0d640 1 -- 172.21.15.49:0/1712953352 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc9c80cb010 msgr2=0x7fc9c80c2710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:28.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.967+0000 7fc9ceb0d640 1 --2- 172.21.15.49:0/1712953352 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc9c80cb010 0x7fc9c80c2710 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fc9c00097c0 tx=0x7fc9c001c3c0 comp rx=0 tx=0).stop 2023-11-06T02:10:28.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.967+0000 7fc9ceb0d640 1 -- 172.21.15.49:0/1712953352 shutdown_connections 2023-11-06T02:10:28.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.967+0000 7fc9ceb0d640 1 --2- 172.21.15.49:0/1712953352 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc9c80cb010 0x7fc9c80c2710 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:28.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.967+0000 7fc9ceb0d640 1 --2- 172.21.15.49:0/1712953352 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc9c80b0790 0x7fc9c80c9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:28.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.967+0000 7fc9ceb0d640 1 -- 172.21.15.49:0/1712953352 >> 172.21.15.49:0/1712953352 conn(0x7fc9c806c0a0 msgr2=0x7fc9c806c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:28.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.967+0000 7fc9ceb0d640 1 -- 172.21.15.49:0/1712953352 shutdown_connections 2023-11-06T02:10:28.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.967+0000 7fc9ceb0d640 1 -- 172.21.15.49:0/1712953352 wait complete. 2023-11-06T02:10:28.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.967+0000 7fc9ceb0d640 1 Processor -- start 2023-11-06T02:10:28.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.967+0000 7fc9ceb0d640 1 -- start start 2023-11-06T02:10:28.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.967+0000 7fc9ceb0d640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc9c80b0790 0x7fc9c814cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:28.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.967+0000 7fc9ceb0d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc9c80cb010 0x7fc9c815d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:28.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.967+0000 7fc9ceb0d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc9c815ab50 0x7fc9c81533a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:28.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.967+0000 7fc9ceb0d640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c814d0d0 con 0x7fc9c80cb010 2023-11-06T02:10:28.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.967+0000 7fc9ceb0d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c81f37c0 con 0x7fc9c815ab50 2023-11-06T02:10:28.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.967+0000 7fc9ceb0d640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c81f2e90 con 0x7fc9c80b0790 2023-11-06T02:10:28.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.967+0000 7fc9c7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc9c80cb010 0x7fc9c815d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:28.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.967+0000 7fc9c7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc9c80cb010 0x7fc9c815d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43914/0 (socket says 172.21.15.49:43914) 2023-11-06T02:10:28.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.967+0000 7fc9c7fff640 1 -- 172.21.15.49:0/2429584884 learned_addr learned my addr 172.21.15.49:0/2429584884 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:28.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.967+0000 7fc9cc8a9640 1 --2- 172.21.15.49:0/2429584884 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc9c80b0790 0x7fc9c814cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:28.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.967+0000 7fc9c7fff640 1 -- 172.21.15.49:0/2429584884 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc9c80b0790 msgr2=0x7fc9c814cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:28.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.967+0000 7fc9c7fff640 1 --2- 172.21.15.49:0/2429584884 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc9c80b0790 0x7fc9c814cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:28.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.967+0000 7fc9c7fff640 1 -- 172.21.15.49:0/2429584884 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc9c815ab50 msgr2=0x7fc9c81533a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:28.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.967+0000 7fc9cd0aa640 1 --2- 172.21.15.49:0/2429584884 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc9c815ab50 0x7fc9c81533a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:28.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.967+0000 7fc9c7fff640 1 --2- 172.21.15.49:0/2429584884 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc9c815ab50 0x7fc9c81533a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:28.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.967+0000 7fc9c7fff640 1 -- 172.21.15.49:0/2429584884 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9c0009420 con 0x7fc9c80cb010 2023-11-06T02:10:28.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.967+0000 7fc9c7fff640 1 --2- 172.21.15.49:0/2429584884 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc9c80cb010 0x7fc9c815d810 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fc9b8007ef0 tx=0x7fc9b800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:28.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.971+0000 7fc9c57fa640 1 -- 172.21.15.49:0/2429584884 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b8014620 con 0x7fc9c80cb010 2023-11-06T02:10:28.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.971+0000 7fc9c57fa640 1 -- 172.21.15.49:0/2429584884 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9b8014780 con 0x7fc9c80cb010 2023-11-06T02:10:28.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.971+0000 7fc9ceb0d640 1 -- 172.21.15.49:0/2429584884 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9c81f3170 con 0x7fc9c80cb010 2023-11-06T02:10:28.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.971+0000 7fc9c57fa640 1 -- 172.21.15.49:0/2429584884 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b8014b30 con 0x7fc9c80cb010 2023-11-06T02:10:28.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.971+0000 7fc9ceb0d640 1 -- 172.21.15.49:0/2429584884 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9c81f3c50 con 0x7fc9c80cb010 2023-11-06T02:10:28.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.971+0000 7fc9ceb0d640 1 -- 172.21.15.49:0/2429584884 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9c81f3930 con 0x7fc9c80cb010 2023-11-06T02:10:28.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.971+0000 7fc9c57fa640 1 -- 172.21.15.49:0/2429584884 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fc9b8014c90 con 0x7fc9c80cb010 2023-11-06T02:10:28.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.971+0000 7fc9c57fa640 1 --2- 172.21.15.49:0/2429584884 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc9a8065a90 0x7fc9a8067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:28.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.971+0000 7fc9c57fa640 1 -- 172.21.15.49:0/2429584884 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fc9b8083960 con 0x7fc9c80cb010 2023-11-06T02:10:28.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.971+0000 7fc9cc8a9640 1 --2- 172.21.15.49:0/2429584884 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc9a8065a90 0x7fc9a8067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:28.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.971+0000 7fc9cc8a9640 1 --2- 172.21.15.49:0/2429584884 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc9a8065a90 0x7fc9a8067f50 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fc9b0001370 tx=0x7fc9b0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:28.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:28.975+0000 7fc9c57fa640 1 -- 172.21.15.49:0/2429584884 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fc9b8051510 con 0x7fc9c80cb010 2023-11-06T02:10:29.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.131+0000 7fc9ceb0d640 1 -- 172.21.15.49:0/2429584884 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc9c8074040 con 0x7fc9c80cb010 2023-11-06T02:10:29.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.135+0000 7fc9c57fa640 1 -- 172.21.15.49:0/2429584884 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+14171 (secure 0 0 0) 0x7fc9b8054b30 con 0x7fc9c80cb010 2023-11-06T02:10:29.135 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:10:29.135 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":47,"fsid":"1a8b035c-6ba4-499a-b876-aea47fd9c0d4","created":"2023-11-06T02:08:06.440857+0000","modified":"2023-11-06T02:10:28.412018+0000","last_up_change":"2023-11-06T02:08:17.982372+0000","last_in_change":"2023-11-06T02:08:08.906631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T02:08:19.252222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T02:08:24.776211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ad55af8-0571-4e6a-8722-b5d429b7496d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6809","nonce":38116},{"type":"v1","addr":"172.21.15.49:6811","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6813","nonce":38116},{"type":"v1","addr":"172.21.15.49:6815","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6821","nonce":38116},{"type":"v1","addr":"172.21.15.49:6823","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6817","nonce":38116},{"type":"v1","addr":"172.21.15.49:6819","nonce":38116}]},"public_addr":"172.21.15.49:6811/38116","cluster_addr":"172.21.15.49:6815/38116","heartbeat_back_addr":"172.21.15.49:6823/38116","heartbeat_front_addr":"172.21.15.49:6819/38116","state":["exists","up"]},{"osd":1,"uuid":"c8b31f85-a757-43ed-9419-351acc46611b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":38113},{"type":"v1","addr":"172.21.15.49:6810","nonce":38113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":38113},{"type":"v1","addr":"172.21.15.49:6814","nonce":38113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":38113},{"type":"v1","addr":"172.21.15.49:6822","nonce":38113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":38113},{"type":"v1","addr":"172.21.15.49:6818","nonce":38113}]},"public_addr":"172.21.15.49:6810/38113","cluster_addr":"172.21.15.49:6814/38113","heartbeat_back_addr":"172.21.15.49:6822/38113","heartbeat_front_addr":"172.21.15.49:6818/38113","state":["exists","up"]},{"osd":2,"uuid":"555b3bf3-0a70-4b91-8efd-e4204b5fc26e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":38114},{"type":"v1","addr":"172.21.15.49:6801","nonce":38114}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":38114},{"type":"v1","addr":"172.21.15.49:6803","nonce":38114}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":38114},{"type":"v1","addr":"172.21.15.49:6807","nonce":38114}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":38114},{"type":"v1","addr":"172.21.15.49:6805","nonce":38114}]},"public_addr":"172.21.15.49:6801/38114","cluster_addr":"172.21.15.49:6803/38114","heartbeat_back_addr":"172.21.15.49:6807/38114","heartbeat_front_addr":"172.21.15.49:6805/38114","state":["exists","up"]},{"osd":3,"uuid":"8d4e5d77-03ef-4b9b-bfcb-5ae93a46795c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":38115},{"type":"v1","addr":"172.21.15.49:6825","nonce":38115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":38115},{"type":"v1","addr":"172.21.15.49:6827","nonce":38115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":38115},{"type":"v1","addr":"172.21.15.49:6831","nonce":38115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":38115},{"type":"v1","addr":"172.21.15.49:6829","nonce":38115}]},"public_addr":"172.21.15.49:6825/38115","cluster_addr":"172.21.15.49:6827/38115","heartbeat_back_addr":"172.21.15.49:6831/38115","heartbeat_front_addr":"172.21.15.49:6829/38115","state":["exists","up"]},{"osd":4,"uuid":"dba03ad4-68a3-4cf3-9218-ac611f02eacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":38417},{"type":"v1","addr":"172.21.15.150:6809","nonce":38417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":38417},{"type":"v1","addr":"172.21.15.150:6811","nonce":38417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":38417},{"type":"v1","addr":"172.21.15.150:6815","nonce":38417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":38417},{"type":"v1","addr":"172.21.15.150:6813","nonce":38417}]},"public_addr":"172.21.15.150:6809/38417","cluster_addr":"172.21.15.150:6811/38417","heartbeat_back_addr":"172.21.15.150:6815/38417","heartbeat_front_addr":"172.21.15.150:6813/38417","state":["exists","up"]},{"osd":5,"uuid":"0df9c303-c255-4640-8e91-01f1c3efe18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":38418},{"type":"v1","addr":"172.21.15.150:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":38418},{"type":"v1","addr":"172.21.15.150:6803","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":38418},{"type":"v1","addr":"172.21.15.150:6807","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":38418},{"type":"v1","addr":"172.21.15.150:6805","nonce":38418}]},"public_addr":"172.21.15.150:6801/38418","cluster_addr":"172.21.15.150:6803/38418","heartbeat_back_addr":"172.21.15.150:6807/38418","heartbeat_front_addr":"172.21.15.150:6805/38418","state":["exists","up"]},{"osd":6,"uuid":"e59fa189-4b70-4c06-9722-c02b90828bf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":38428},{"type":"v1","addr":"172.21.15.150:6825","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":38428},{"type":"v1","addr":"172.21.15.150:6827","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":38428},{"type":"v1","addr":"172.21.15.150:6831","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":38428},{"type":"v1","addr":"172.21.15.150:6829","nonce":38428}]},"public_addr":"172.21.15.150:6825/38428","cluster_addr":"172.21.15.150:6827/38428","heartbeat_back_addr":"172.21.15.150:6831/38428","heartbeat_front_addr":"172.21.15.150:6829/38428","state":["exists","up"]},{"osd":7,"uuid":"a64f57e1-ef98-46bd-b304-385759e95898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":38421},{"type":"v1","addr":"172.21.15.150:6817","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":38421},{"type":"v1","addr":"172.21.15.150:6819","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":38421},{"type":"v1","addr":"172.21.15.150:6823","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":38421},{"type":"v1","addr":"172.21.15.150:6821","nonce":38421}]},"public_addr":"172.21.15.150:6817/38421","cluster_addr":"172.21.15.150:6819/38421","heartbeat_back_addr":"172.21.15.150:6823/38421","heartbeat_front_addr":"172.21.15.150:6821/38421","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.438552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.413280+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.296466+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.424966+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.276953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.230984+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.410235+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.480084+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T02:10:29.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.135+0000 7fc9ceb0d640 1 -- 172.21.15.49:0/2429584884 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc9a8065a90 msgr2=0x7fc9a8067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:29.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.135+0000 7fc9ceb0d640 1 --2- 172.21.15.49:0/2429584884 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc9a8065a90 0x7fc9a8067f50 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fc9b0001370 tx=0x7fc9b0009040 comp rx=0 tx=0).stop 2023-11-06T02:10:29.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.139+0000 7fc9ceb0d640 1 -- 172.21.15.49:0/2429584884 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc9c80cb010 msgr2=0x7fc9c815d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:29.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.139+0000 7fc9ceb0d640 1 --2- 172.21.15.49:0/2429584884 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc9c80cb010 0x7fc9c815d810 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fc9b8007ef0 tx=0x7fc9b800e4b0 comp rx=0 tx=0).stop 2023-11-06T02:10:29.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.139+0000 7fc9ceb0d640 1 -- 172.21.15.49:0/2429584884 shutdown_connections 2023-11-06T02:10:29.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.139+0000 7fc9ceb0d640 1 --2- 172.21.15.49:0/2429584884 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc9a8065a90 0x7fc9a8067f50 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:29.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.139+0000 7fc9ceb0d640 1 --2- 172.21.15.49:0/2429584884 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc9c80cb010 0x7fc9c815d810 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:29.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.139+0000 7fc9ceb0d640 1 --2- 172.21.15.49:0/2429584884 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc9c815ab50 0x7fc9c81533a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:29.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.139+0000 7fc9ceb0d640 1 --2- 172.21.15.49:0/2429584884 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc9c80b0790 0x7fc9c814cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:29.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.139+0000 7fc9ceb0d640 1 -- 172.21.15.49:0/2429584884 >> 172.21.15.49:0/2429584884 conn(0x7fc9c806c0a0 msgr2=0x7fc9c80c4860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:29.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.139+0000 7fc9ceb0d640 1 -- 172.21.15.49:0/2429584884 shutdown_connections 2023-11-06T02:10:29.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.139+0000 7fc9ceb0d640 1 -- 172.21.15.49:0/2429584884 wait complete. 2023-11-06T02:10:29.154 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T02:08:19.252222+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T02:08:24.776211+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T02:10:29.155 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T02:10:29.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.283+0000 7f2b74e03640 1 Processor -- start 2023-11-06T02:10:29.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.283+0000 7f2b74e03640 1 -- start start 2023-11-06T02:10:29.286 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.287+0000 7f2b74e03640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f2b700705d0 0x7f2b7006c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:29.286 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.287+0000 7f2b74e03640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b700c8c80 con 0x7f2b7014ffc0 2023-11-06T02:10:29.286 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.287+0000 7f2b74e03640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b70159700 con 0x7f2b70070160 2023-11-06T02:10:29.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.287+0000 7f2b74e03640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b700ae4d0 con 0x7f2b700705d0 2023-11-06T02:10:29.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.287+0000 7f2b6ed9d640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f2b700705d0 0x7f2b7006c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:29.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.287+0000 7f2b6ed9d640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f2b700705d0 0x7f2b7006c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43928/0 (socket says 172.21.15.49:43928) 2023-11-06T02:10:29.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.287+0000 7f2b6e59c640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f2b7014ffc0 0x7f2b700ada30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:41524/0 (socket says 172.21.15.49:41524) 2023-11-06T02:10:29.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.287+0000 7f2b6ed9d640 1 -- 172.21.15.49:0/2100477450 learned_addr learned my addr 172.21.15.49:0/2100477450 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:29.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.287+0000 7f2b6ed9d640 1 -- 172.21.15.49:0/2100477450 >> v1:172.21.15.150:6789/0 conn(0x7f2b70070160 legacy=0x7f2b7006c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:29.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.287+0000 7f2b6ed9d640 1 -- 172.21.15.49:0/2100477450 >> v1:172.21.15.49:6789/0 conn(0x7f2b7014ffc0 legacy=0x7f2b700ada30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:29.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.287+0000 7f2b6d59a640 1 -- 172.21.15.49:0/2100477450 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3964909571 0 0) 0x7f2b700c8c80 con 0x7f2b7014ffc0 2023-11-06T02:10:29.289 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.287+0000 7f2b6ed9d640 1 -- 172.21.15.49:0/2100477450 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b701599e0 con 0x7f2b700705d0 2023-11-06T02:10:29.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.291+0000 7f2b6ed9d640 1 --2- 172.21.15.49:0/2100477450 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b700705d0 0x7f2b7006c940 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f2b68009e50 tx=0x7f2b6801ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab1d2d2ee490d94c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:29.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.291+0000 7f2b6d59a640 1 -- 172.21.15.49:0/2100477450 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b680a56e0 con 0x7f2b700705d0 2023-11-06T02:10:29.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.291+0000 7f2b6d59a640 1 -- 172.21.15.49:0/2100477450 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b680a5840 con 0x7f2b700705d0 2023-11-06T02:10:29.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.291+0000 7f2b6d59a640 1 -- 172.21.15.49:0/2100477450 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b680a5bd0 con 0x7f2b700705d0 2023-11-06T02:10:29.293 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.291+0000 7f2b74e03640 1 -- 172.21.15.49:0/2100477450 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b700705d0 msgr2=0x7f2b7006c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:29.293 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.291+0000 7f2b74e03640 1 --2- 172.21.15.49:0/2100477450 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b700705d0 0x7f2b7006c940 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f2b68009e50 tx=0x7f2b6801ca10 comp rx=0 tx=0).stop 2023-11-06T02:10:29.293 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.291+0000 7f2b74e03640 1 -- 172.21.15.49:0/2100477450 shutdown_connections 2023-11-06T02:10:29.294 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.291+0000 7f2b74e03640 1 --2- 172.21.15.49:0/2100477450 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b700705d0 0x7f2b7006c940 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:29.294 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.291+0000 7f2b74e03640 1 -- 172.21.15.49:0/2100477450 >> 172.21.15.49:0/2100477450 conn(0x7f2b7006a370 msgr2=0x7f2b700ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:29.294 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.291+0000 7f2b74e03640 1 -- 172.21.15.49:0/2100477450 shutdown_connections 2023-11-06T02:10:29.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.291+0000 7f2b74e03640 1 -- 172.21.15.49:0/2100477450 wait complete. 2023-11-06T02:10:29.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.295+0000 7f2b74e03640 1 Processor -- start 2023-11-06T02:10:29.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.295+0000 7f2b74e03640 1 -- start start 2023-11-06T02:10:29.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.295+0000 7f2b74e03640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b70070160 0x7f2b701333c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:29.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.295+0000 7f2b74e03640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b700705d0 0x7f2b70146620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:29.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.295+0000 7f2b74e03640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b7014ffc0 0x7f2b7014dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:29.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.295+0000 7f2b74e03640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b7014f860 con 0x7f2b700705d0 2023-11-06T02:10:29.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.295+0000 7f2b74e03640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b7014e7c0 con 0x7f2b7014ffc0 2023-11-06T02:10:29.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.295+0000 7f2b74e03640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b7014eaa0 con 0x7f2b70070160 2023-11-06T02:10:29.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.295+0000 7f2b6dd9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b700705d0 0x7f2b70146620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:29.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.295+0000 7f2b6dd9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b700705d0 0x7f2b70146620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43936/0 (socket says 172.21.15.49:43936) 2023-11-06T02:10:29.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.295+0000 7f2b6dd9b640 1 -- 172.21.15.49:0/174836404 learned_addr learned my addr 172.21.15.49:0/174836404 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:29.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.295+0000 7f2b6e59c640 1 --2- 172.21.15.49:0/174836404 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b70070160 0x7f2b701333c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:29.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.295+0000 7f2b6ed9d640 1 --2- 172.21.15.49:0/174836404 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b7014ffc0 0x7f2b7014dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:29.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.295+0000 7f2b6dd9b640 1 -- 172.21.15.49:0/174836404 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b70070160 msgr2=0x7f2b701333c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:29.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.295+0000 7f2b6dd9b640 1 --2- 172.21.15.49:0/174836404 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b70070160 0x7f2b701333c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:29.300 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.295+0000 7f2b6dd9b640 1 -- 172.21.15.49:0/174836404 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b7014ffc0 msgr2=0x7f2b7014dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:29.300 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.295+0000 7f2b6dd9b640 1 --2- 172.21.15.49:0/174836404 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b7014ffc0 0x7f2b7014dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:29.300 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.295+0000 7f2b6dd9b640 1 -- 172.21.15.49:0/174836404 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b68009b30 con 0x7f2b700705d0 2023-11-06T02:10:29.300 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.295+0000 7f2b6ed9d640 1 --2- 172.21.15.49:0/174836404 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b7014ffc0 0x7f2b7014dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:10:29.300 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.295+0000 7f2b6e59c640 1 --2- 172.21.15.49:0/174836404 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b70070160 0x7f2b701333c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:10:29.301 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.295+0000 7f2b6dd9b640 1 --2- 172.21.15.49:0/174836404 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b700705d0 0x7f2b70146620 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f2b5800c500 tx=0x7f2b5800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:29.301 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.295+0000 7f2b66ffd640 1 -- 172.21.15.49:0/174836404 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b58005220 con 0x7f2b700705d0 2023-11-06T02:10:29.301 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.295+0000 7f2b66ffd640 1 -- 172.21.15.49:0/174836404 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b58005380 con 0x7f2b700705d0 2023-11-06T02:10:29.301 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.295+0000 7f2b66ffd640 1 -- 172.21.15.49:0/174836404 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b58007270 con 0x7f2b700705d0 2023-11-06T02:10:29.301 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.295+0000 7f2b74e03640 1 -- 172.21.15.49:0/174836404 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b7014ed80 con 0x7f2b700705d0 2023-11-06T02:10:29.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.295+0000 7f2b74e03640 1 -- 172.21.15.49:0/174836404 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b7014f340 con 0x7f2b700705d0 2023-11-06T02:10:29.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.295+0000 7f2b74e03640 1 -- 172.21.15.49:0/174836404 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b70074040 con 0x7f2b700705d0 2023-11-06T02:10:29.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.299+0000 7f2b66ffd640 1 -- 172.21.15.49:0/174836404 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f2b580073d0 con 0x7f2b700705d0 2023-11-06T02:10:29.303 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.299+0000 7f2b66ffd640 1 --2- 172.21.15.49:0/174836404 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2b50065a60 0x7f2b50067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:29.303 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.299+0000 7f2b66ffd640 1 -- 172.21.15.49:0/174836404 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f2b58082900 con 0x7f2b700705d0 2023-11-06T02:10:29.303 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.299+0000 7f2b6e59c640 1 --2- 172.21.15.49:0/174836404 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2b50065a60 0x7f2b50067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:29.304 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.299+0000 7f2b6e59c640 1 --2- 172.21.15.49:0/174836404 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2b50065a60 0x7f2b50067f20 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f2b60000f70 tx=0x7f2b60009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:29.304 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.303+0000 7f2b66ffd640 1 -- 172.21.15.49:0/174836404 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f2b580504c0 con 0x7f2b700705d0 2023-11-06T02:10:29.460 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.459+0000 7f2b74e03640 1 -- 172.21.15.49:0/174836404 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2b700b3ab0 con 0x7f2b700705d0 2023-11-06T02:10:29.461 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.459+0000 7f2b66ffd640 1 -- 172.21.15.49:0/174836404 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v47) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2b58053ae0 con 0x7f2b700705d0 2023-11-06T02:10:29.461 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2023-11-06T02:10:29.463 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.463+0000 7f2b74e03640 1 -- 172.21.15.49:0/174836404 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2b50065a60 msgr2=0x7f2b50067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:29.463 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.463+0000 7f2b74e03640 1 --2- 172.21.15.49:0/174836404 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2b50065a60 0x7f2b50067f20 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f2b60000f70 tx=0x7f2b60009040 comp rx=0 tx=0).stop 2023-11-06T02:10:29.463 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.463+0000 7f2b74e03640 1 -- 172.21.15.49:0/174836404 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b700705d0 msgr2=0x7f2b70146620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:29.464 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.463+0000 7f2b74e03640 1 --2- 172.21.15.49:0/174836404 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b700705d0 0x7f2b70146620 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f2b5800c500 tx=0x7f2b5800c9c0 comp rx=0 tx=0).stop 2023-11-06T02:10:29.464 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.463+0000 7f2b74e03640 1 -- 172.21.15.49:0/174836404 shutdown_connections 2023-11-06T02:10:29.464 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.463+0000 7f2b74e03640 1 --2- 172.21.15.49:0/174836404 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2b50065a60 0x7f2b50067f20 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:29.464 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.463+0000 7f2b74e03640 1 --2- 172.21.15.49:0/174836404 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b700705d0 0x7f2b70146620 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:29.464 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.463+0000 7f2b74e03640 1 --2- 172.21.15.49:0/174836404 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b7014ffc0 0x7f2b7014dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:29.465 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.463+0000 7f2b74e03640 1 --2- 172.21.15.49:0/174836404 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b70070160 0x7f2b701333c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:29.465 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.463+0000 7f2b74e03640 1 -- 172.21.15.49:0/174836404 >> 172.21.15.49:0/174836404 conn(0x7f2b7006a370 msgr2=0x7f2b700c8a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:29.465 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.463+0000 7f2b74e03640 1 -- 172.21.15.49:0/174836404 shutdown_connections 2023-11-06T02:10:29.465 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.463+0000 7f2b74e03640 1 -- 172.21.15.49:0/174836404 wait complete. 2023-11-06T02:10:29.480 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T02:10:29.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.603+0000 7f306efcd640 1 Processor -- start 2023-11-06T02:10:29.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.603+0000 7f306efcd640 1 -- start start 2023-11-06T02:10:29.604 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.603+0000 7f306efcd640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f306815cbb0 0x7f30680d1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:29.604 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.603+0000 7f306efcd640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f30680d24e0 0x7f30680d2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:29.604 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.603+0000 7f306efcd640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f30680d2fa0 0x7f30680b4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:29.604 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.603+0000 7f306efcd640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30680d3430 con 0x7f30680d24e0 2023-11-06T02:10:29.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.603+0000 7f306efcd640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f306815cf90 con 0x7f306815cbb0 2023-11-06T02:10:29.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.603+0000 7f306efcd640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30680d0b80 con 0x7f30680d2fa0 2023-11-06T02:10:29.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.603+0000 7f3067fff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f30680d24e0 0x7f30680d2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:29.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.603+0000 7f3067fff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f30680d24e0 0x7f30680d2a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43942/0 (socket says 172.21.15.49:43942) 2023-11-06T02:10:29.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.603+0000 7f3067fff640 1 -- 172.21.15.49:0/1398201973 learned_addr learned my addr 172.21.15.49:0/1398201973 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:29.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.603+0000 7f306d56a640 1 --2- 172.21.15.49:0/1398201973 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f30680d2fa0 0x7f30680b4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:29.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.603+0000 7f3067fff640 1 -- 172.21.15.49:0/1398201973 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f30680d2fa0 msgr2=0x7f30680b4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:29.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.603+0000 7f306cd69640 1 --2- 172.21.15.49:0/1398201973 >> v2:172.21.15.150:3300/0 conn(0x7f306815cbb0 0x7f30680d1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:29.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.603+0000 7f3067fff640 1 --2- 172.21.15.49:0/1398201973 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f30680d2fa0 0x7f30680b4550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:29.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.603+0000 7f3067fff640 1 -- 172.21.15.49:0/1398201973 >> v2:172.21.15.150:3300/0 conn(0x7f306815cbb0 msgr2=0x7f30680d1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:29.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.603+0000 7f3067fff640 1 --2- 172.21.15.49:0/1398201973 >> v2:172.21.15.150:3300/0 conn(0x7f306815cbb0 0x7f30680d1fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:29.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.603+0000 7f3067fff640 1 -- 172.21.15.49:0/1398201973 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30680d0e60 con 0x7f30680d24e0 2023-11-06T02:10:29.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.607+0000 7f3067fff640 1 --2- 172.21.15.49:0/1398201973 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30680d24e0 0x7f30680d2a60 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f30680cf4b0 tx=0x7f305c01ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44a380da82345baa server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:29.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.607+0000 7f30677fe640 1 -- 172.21.15.49:0/1398201973 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f305c0a89b0 con 0x7f30680d24e0 2023-11-06T02:10:29.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.607+0000 7f30677fe640 1 -- 172.21.15.49:0/1398201973 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f305c0a8b10 con 0x7f30680d24e0 2023-11-06T02:10:29.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.607+0000 7f30677fe640 1 -- 172.21.15.49:0/1398201973 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f305c0a8ea0 con 0x7f30680d24e0 2023-11-06T02:10:29.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.607+0000 7f306efcd640 1 -- 172.21.15.49:0/1398201973 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30680d24e0 msgr2=0x7f30680d2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:29.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.607+0000 7f306efcd640 1 --2- 172.21.15.49:0/1398201973 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30680d24e0 0x7f30680d2a60 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f30680cf4b0 tx=0x7f305c01ecf0 comp rx=0 tx=0).stop 2023-11-06T02:10:29.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.607+0000 7f306efcd640 1 -- 172.21.15.49:0/1398201973 shutdown_connections 2023-11-06T02:10:29.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.607+0000 7f306efcd640 1 --2- 172.21.15.49:0/1398201973 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f30680d2fa0 0x7f30680b4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:29.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.607+0000 7f306efcd640 1 --2- 172.21.15.49:0/1398201973 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30680d24e0 0x7f30680d2a60 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:29.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.607+0000 7f306efcd640 1 --2- 172.21.15.49:0/1398201973 >> v2:172.21.15.150:3300/0 conn(0x7f306815cbb0 0x7f30680d1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:29.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.607+0000 7f306efcd640 1 -- 172.21.15.49:0/1398201973 >> 172.21.15.49:0/1398201973 conn(0x7f30680718a0 msgr2=0x7f3068071cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:29.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.607+0000 7f306efcd640 1 -- 172.21.15.49:0/1398201973 shutdown_connections 2023-11-06T02:10:29.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.607+0000 7f306efcd640 1 -- 172.21.15.49:0/1398201973 wait complete. 2023-11-06T02:10:29.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.607+0000 7f306efcd640 1 Processor -- start 2023-11-06T02:10:29.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.607+0000 7f306efcd640 1 -- start start 2023-11-06T02:10:29.613 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.611+0000 7f306efcd640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f30680d24e0 0x7f3068156530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:29.614 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.611+0000 7f306efcd640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f30680d2fa0 0x7f306815f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:29.614 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.611+0000 7f306efcd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f306815cbb0 0x7f30681f1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:29.614 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.611+0000 7f306efcd640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f306815fe20 con 0x7f306815cbb0 2023-11-06T02:10:29.614 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.611+0000 7f306efcd640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3068156b50 con 0x7f30680d2fa0 2023-11-06T02:10:29.614 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.611+0000 7f306efcd640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3068156e30 con 0x7f30680d24e0 2023-11-06T02:10:29.615 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.611+0000 7f306d56a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f306815cbb0 0x7f30681f1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:29.615 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.611+0000 7f306d56a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f306815cbb0 0x7f30681f1f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43956/0 (socket says 172.21.15.49:43956) 2023-11-06T02:10:29.615 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.611+0000 7f306d56a640 1 -- 172.21.15.49:0/2175742158 learned_addr learned my addr 172.21.15.49:0/2175742158 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:29.615 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.611+0000 7f3067fff640 1 --2- 172.21.15.49:0/2175742158 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f30680d2fa0 0x7f306815f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:29.615 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.611+0000 7f306cd69640 1 --2- 172.21.15.49:0/2175742158 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f30680d24e0 0x7f3068156530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:29.616 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.611+0000 7f306d56a640 1 -- 172.21.15.49:0/2175742158 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f30680d24e0 msgr2=0x7f3068156530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:29.616 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.611+0000 7f306d56a640 1 --2- 172.21.15.49:0/2175742158 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f30680d24e0 0x7f3068156530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:29.616 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.611+0000 7f306d56a640 1 -- 172.21.15.49:0/2175742158 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f30680d2fa0 msgr2=0x7f306815f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:29.616 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.611+0000 7f306d56a640 1 --2- 172.21.15.49:0/2175742158 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f30680d2fa0 0x7f306815f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:29.616 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.611+0000 7f306d56a640 1 -- 172.21.15.49:0/2175742158 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f305c0094d0 con 0x7f306815cbb0 2023-11-06T02:10:29.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.611+0000 7f306d56a640 1 --2- 172.21.15.49:0/2175742158 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f306815cbb0 0x7f30681f1f40 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f306000ccf0 tx=0x7f306000a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:29.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.611+0000 7f30657fa640 1 -- 172.21.15.49:0/2175742158 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3060003ba0 con 0x7f306815cbb0 2023-11-06T02:10:29.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.611+0000 7f30657fa640 1 -- 172.21.15.49:0/2175742158 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3060003d00 con 0x7f306815cbb0 2023-11-06T02:10:29.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.611+0000 7f30657fa640 1 -- 172.21.15.49:0/2175742158 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3060004240 con 0x7f306815cbb0 2023-11-06T02:10:29.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.611+0000 7f306efcd640 1 -- 172.21.15.49:0/2175742158 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3068157110 con 0x7f306815cbb0 2023-11-06T02:10:29.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.611+0000 7f306efcd640 1 -- 172.21.15.49:0/2175742158 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30681576d0 con 0x7f306815cbb0 2023-11-06T02:10:29.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.611+0000 7f306efcd640 1 -- 172.21.15.49:0/2175742158 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30680b7860 con 0x7f306815cbb0 2023-11-06T02:10:29.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.611+0000 7f30657fa640 1 -- 172.21.15.49:0/2175742158 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f30600043a0 con 0x7f306815cbb0 2023-11-06T02:10:29.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.611+0000 7f30657fa640 1 --2- 172.21.15.49:0/2175742158 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3054065a60 0x7f3054067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:29.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.611+0000 7f30657fa640 1 -- 172.21.15.49:0/2175742158 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f3060012070 con 0x7f306815cbb0 2023-11-06T02:10:29.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.611+0000 7f306cd69640 1 --2- 172.21.15.49:0/2175742158 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3054065a60 0x7f3054067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:29.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.615+0000 7f306cd69640 1 --2- 172.21.15.49:0/2175742158 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3054065a60 0x7f3054067f20 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f305800a8d0 tx=0x7f3058008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:29.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.615+0000 7f30657fa640 1 -- 172.21.15.49:0/2175742158 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f3060051400 con 0x7f306815cbb0 2023-11-06T02:10:29.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.775+0000 7f306efcd640 1 -- 172.21.15.49:0/2175742158 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f3068071fb0 con 0x7f306815cbb0 2023-11-06T02:10:29.776 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.775+0000 7f30657fa640 1 -- 172.21.15.49:0/2175742158 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v47) v1 ==== 92+0+10 (secure 0 0 0) 0x7f3060054a20 con 0x7f306815cbb0 2023-11-06T02:10:29.776 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 8 2023-11-06T02:10:29.778 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.779+0000 7f306efcd640 1 -- 172.21.15.49:0/2175742158 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3054065a60 msgr2=0x7f3054067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:29.779 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.779+0000 7f306efcd640 1 --2- 172.21.15.49:0/2175742158 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3054065a60 0x7f3054067f20 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f305800a8d0 tx=0x7f3058008040 comp rx=0 tx=0).stop 2023-11-06T02:10:29.779 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.779+0000 7f306efcd640 1 -- 172.21.15.49:0/2175742158 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f306815cbb0 msgr2=0x7f30681f1f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:29.779 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.779+0000 7f306efcd640 1 --2- 172.21.15.49:0/2175742158 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f306815cbb0 0x7f30681f1f40 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f306000ccf0 tx=0x7f306000a660 comp rx=0 tx=0).stop 2023-11-06T02:10:29.779 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.779+0000 7f306efcd640 1 -- 172.21.15.49:0/2175742158 shutdown_connections 2023-11-06T02:10:29.780 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.779+0000 7f306efcd640 1 --2- 172.21.15.49:0/2175742158 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3054065a60 0x7f3054067f20 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:29.780 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.779+0000 7f306efcd640 1 --2- 172.21.15.49:0/2175742158 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f306815cbb0 0x7f30681f1f40 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:29.780 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.779+0000 7f306efcd640 1 --2- 172.21.15.49:0/2175742158 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f30680d2fa0 0x7f306815f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:29.780 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.779+0000 7f306efcd640 1 --2- 172.21.15.49:0/2175742158 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f30680d24e0 0x7f3068156530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:29.780 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.779+0000 7f306efcd640 1 -- 172.21.15.49:0/2175742158 >> 172.21.15.49:0/2175742158 conn(0x7f30680718a0 msgr2=0x7f30680b6f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:29.780 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.779+0000 7f306efcd640 1 -- 172.21.15.49:0/2175742158 shutdown_connections 2023-11-06T02:10:29.781 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.779+0000 7f306efcd640 1 -- 172.21.15.49:0/2175742158 wait complete. 2023-11-06T02:10:29.795 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-06T02:10:29.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.919+0000 7fdb130e6640 1 Processor -- start 2023-11-06T02:10:29.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.919+0000 7fdb130e6640 1 -- start start 2023-11-06T02:10:29.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.923+0000 7fdb130e6640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdb0c15ab50 0x7fdb0c0c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:29.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.923+0000 7fdb130e6640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fdb0c0b0790 0x7fdb0c0c2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:29.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.923+0000 7fdb130e6640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb0c0cab80 con 0x7fdb0c0cb010 2023-11-06T02:10:29.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.923+0000 7fdb130e6640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb0c15af30 con 0x7fdb0c0b0790 2023-11-06T02:10:29.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.923+0000 7fdb130e6640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0c0cb5b0 con 0x7fdb0c15ab50 2023-11-06T02:10:29.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.923+0000 7fdb11683640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fdb0c0b0790 0x7fdb0c0c2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:29.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.923+0000 7fdb11683640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fdb0c0b0790 0x7fdb0c0c2710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43958/0 (socket says 172.21.15.49:43958) 2023-11-06T02:10:29.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.923+0000 7fdb11683640 1 -- 172.21.15.49:0/2972530727 learned_addr learned my addr 172.21.15.49:0/2972530727 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:29.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.923+0000 7fdb10e82640 1 --2- 172.21.15.49:0/2972530727 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdb0c15ab50 0x7fdb0c0c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:29.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.923+0000 7fdb11683640 1 -- 172.21.15.49:0/2972530727 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdb0c15ab50 msgr2=0x7fdb0c0c9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:29.924 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.923+0000 7fdb11683640 1 --2- 172.21.15.49:0/2972530727 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdb0c15ab50 0x7fdb0c0c9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:29.924 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.923+0000 7fdb11683640 1 -- 172.21.15.49:0/2972530727 >> v1:172.21.15.150:6789/0 conn(0x7fdb0c0cb010 legacy=0x7fdb0c0b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:29.924 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.923+0000 7fdb11683640 1 -- 172.21.15.49:0/2972530727 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb0c0cb890 con 0x7fdb0c0b0790 2023-11-06T02:10:29.925 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.923+0000 7fdb11683640 1 --2- 172.21.15.49:0/2972530727 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb0c0b0790 0x7fdb0c0c2710 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fdb04009e50 tx=0x7fdb0401c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80219517b9026e44 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:29.926 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.923+0000 7fdb0b7fe640 1 -- 172.21.15.49:0/2972530727 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb040a5820 con 0x7fdb0c0b0790 2023-11-06T02:10:29.926 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.923+0000 7fdb0b7fe640 1 -- 172.21.15.49:0/2972530727 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb040a5980 con 0x7fdb0c0b0790 2023-11-06T02:10:29.926 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.923+0000 7fdb0b7fe640 1 -- 172.21.15.49:0/2972530727 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb040a5d10 con 0x7fdb0c0b0790 2023-11-06T02:10:29.926 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.923+0000 7fdb130e6640 1 -- 172.21.15.49:0/2972530727 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb0c0b0790 msgr2=0x7fdb0c0c2710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:29.926 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.923+0000 7fdb130e6640 1 --2- 172.21.15.49:0/2972530727 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb0c0b0790 0x7fdb0c0c2710 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fdb04009e50 tx=0x7fdb0401c9a0 comp rx=0 tx=0).stop 2023-11-06T02:10:29.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.923+0000 7fdb130e6640 1 -- 172.21.15.49:0/2972530727 shutdown_connections 2023-11-06T02:10:29.930 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.923+0000 7fdb130e6640 1 --2- 172.21.15.49:0/2972530727 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb0c0b0790 0x7fdb0c0c2710 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:29.930 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.923+0000 7fdb130e6640 1 --2- 172.21.15.49:0/2972530727 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdb0c15ab50 0x7fdb0c0c9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:29.930 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.923+0000 7fdb130e6640 1 -- 172.21.15.49:0/2972530727 >> 172.21.15.49:0/2972530727 conn(0x7fdb0c06c0a0 msgr2=0x7fdb0c06c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:29.930 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.923+0000 7fdb130e6640 1 -- 172.21.15.49:0/2972530727 shutdown_connections 2023-11-06T02:10:29.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.923+0000 7fdb130e6640 1 -- 172.21.15.49:0/2972530727 wait complete. 2023-11-06T02:10:29.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.927+0000 7fdb130e6640 1 Processor -- start 2023-11-06T02:10:29.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.927+0000 7fdb130e6640 1 -- start start 2023-11-06T02:10:29.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.927+0000 7fdb130e6640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdb0c0b0790 0x7fdb0c0acfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:29.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.927+0000 7fdb130e6640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb0c0cb010 0x7fdb0c154b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:29.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.927+0000 7fdb130e6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb0c15ab50 0x7fdb0c1550a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:29.935 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.927+0000 7fdb130e6640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0c0ad500 con 0x7fdb0c15ab50 2023-11-06T02:10:29.935 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.927+0000 7fdb130e6640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0c0cc270 con 0x7fdb0c0cb010 2023-11-06T02:10:29.936 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.927+0000 7fdb130e6640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0c159a50 con 0x7fdb0c0b0790 2023-11-06T02:10:29.936 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.927+0000 7fdb11683640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb0c15ab50 0x7fdb0c1550a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:29.936 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.927+0000 7fdb11683640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb0c15ab50 0x7fdb0c1550a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43962/0 (socket says 172.21.15.49:43962) 2023-11-06T02:10:29.937 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.927+0000 7fdb11683640 1 -- 172.21.15.49:0/2036405624 learned_addr learned my addr 172.21.15.49:0/2036405624 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:29.937 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.927+0000 7fdb10e82640 1 --2- 172.21.15.49:0/2036405624 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdb0c0b0790 0x7fdb0c0acfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:29.938 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.927+0000 7fdb11683640 1 -- 172.21.15.49:0/2036405624 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdb0c0b0790 msgr2=0x7fdb0c0acfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:29.938 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.927+0000 7fdb0bfff640 1 --2- 172.21.15.49:0/2036405624 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb0c0cb010 0x7fdb0c154b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:29.938 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.927+0000 7fdb11683640 1 --2- 172.21.15.49:0/2036405624 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdb0c0b0790 0x7fdb0c0acfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:29.938 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.927+0000 7fdb11683640 1 -- 172.21.15.49:0/2036405624 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb0c0cb010 msgr2=0x7fdb0c154b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:29.939 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.927+0000 7fdb11683640 1 --2- 172.21.15.49:0/2036405624 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb0c0cb010 0x7fdb0c154b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:29.939 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.927+0000 7fdb11683640 1 -- 172.21.15.49:0/2036405624 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb04009b30 con 0x7fdb0c15ab50 2023-11-06T02:10:29.940 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.927+0000 7fdb10e82640 1 --2- 172.21.15.49:0/2036405624 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdb0c0b0790 0x7fdb0c0acfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:10:29.940 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.927+0000 7fdb11683640 1 --2- 172.21.15.49:0/2036405624 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb0c15ab50 0x7fdb0c1550a0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fdb0401b050 tx=0x7fdb04019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:29.940 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.927+0000 7fdb097fa640 1 -- 172.21.15.49:0/2036405624 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb040b8670 con 0x7fdb0c15ab50 2023-11-06T02:10:29.941 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.927+0000 7fdb097fa640 1 -- 172.21.15.49:0/2036405624 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb040b87d0 con 0x7fdb0c15ab50 2023-11-06T02:10:29.941 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.927+0000 7fdb130e6640 1 -- 172.21.15.49:0/2036405624 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb0c159d30 con 0x7fdb0c15ab50 2023-11-06T02:10:29.941 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.927+0000 7fdb097fa640 1 -- 172.21.15.49:0/2036405624 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb040b8b60 con 0x7fdb0c15ab50 2023-11-06T02:10:29.942 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.927+0000 7fdb130e6640 1 -- 172.21.15.49:0/2036405624 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb0c15a2f0 con 0x7fdb0c15ab50 2023-11-06T02:10:29.942 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.927+0000 7fdb130e6640 1 -- 172.21.15.49:0/2036405624 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb0c0b0de0 con 0x7fdb0c15ab50 2023-11-06T02:10:29.942 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.927+0000 7fdb097fa640 1 -- 172.21.15.49:0/2036405624 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fdb040b8cc0 con 0x7fdb0c15ab50 2023-11-06T02:10:29.942 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.931+0000 7fdb097fa640 1 --2- 172.21.15.49:0/2036405624 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fdaf8065a60 0x7fdaf8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:29.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.931+0000 7fdb097fa640 1 -- 172.21.15.49:0/2036405624 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fdb0411e590 con 0x7fdb0c15ab50 2023-11-06T02:10:29.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.931+0000 7fdb10e82640 1 --2- 172.21.15.49:0/2036405624 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fdaf8065a60 0x7fdaf8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:29.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.931+0000 7fdb10e82640 1 --2- 172.21.15.49:0/2036405624 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fdaf8065a60 0x7fdaf8067f20 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fdafc006fd0 tx=0x7fdafc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:29.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:29.935+0000 7fdb097fa640 1 -- 172.21.15.49:0/2036405624 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fdb040ec460 con 0x7fdb0c15ab50 2023-11-06T02:10:30.096 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.095+0000 7fdb130e6640 1 -- 172.21.15.49:0/2036405624 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fdb0c0cc3e0 con 0x7fdb0c15ab50 2023-11-06T02:10:30.097 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.095+0000 7fdb097fa640 1 -- 172.21.15.49:0/2036405624 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v47) v1 ==== 84+0+126 (secure 0 0 0) 0x7fdb040efa80 con 0x7fdb0c15ab50 2023-11-06T02:10:30.097 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:10:30.097 INFO:teuthology.orchestra.run.smithi049.stdout:[ 2023-11-06T02:10:30.098 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:10:30.098 INFO:teuthology.orchestra.run.smithi049.stdout: "poolnum": 1, 2023-11-06T02:10:30.098 INFO:teuthology.orchestra.run.smithi049.stdout: "poolname": ".mgr" 2023-11-06T02:10:30.098 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:10:30.098 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:10:30.099 INFO:teuthology.orchestra.run.smithi049.stdout: "poolnum": 2, 2023-11-06T02:10:30.099 INFO:teuthology.orchestra.run.smithi049.stdout: "poolname": "rbd" 2023-11-06T02:10:30.099 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:10:30.099 INFO:teuthology.orchestra.run.smithi049.stdout:] 2023-11-06T02:10:30.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.099+0000 7fdb130e6640 1 -- 172.21.15.49:0/2036405624 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fdaf8065a60 msgr2=0x7fdaf8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:30.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.099+0000 7fdb130e6640 1 --2- 172.21.15.49:0/2036405624 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fdaf8065a60 0x7fdaf8067f20 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fdafc006fd0 tx=0x7fdafc008040 comp rx=0 tx=0).stop 2023-11-06T02:10:30.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.099+0000 7fdb130e6640 1 -- 172.21.15.49:0/2036405624 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb0c15ab50 msgr2=0x7fdb0c1550a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:30.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.099+0000 7fdb130e6640 1 --2- 172.21.15.49:0/2036405624 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb0c15ab50 0x7fdb0c1550a0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fdb0401b050 tx=0x7fdb04019740 comp rx=0 tx=0).stop 2023-11-06T02:10:30.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.099+0000 7fdb130e6640 1 -- 172.21.15.49:0/2036405624 shutdown_connections 2023-11-06T02:10:30.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.099+0000 7fdb130e6640 1 --2- 172.21.15.49:0/2036405624 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fdaf8065a60 0x7fdaf8067f20 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:30.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.099+0000 7fdb130e6640 1 --2- 172.21.15.49:0/2036405624 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb0c15ab50 0x7fdb0c1550a0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:30.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.099+0000 7fdb130e6640 1 --2- 172.21.15.49:0/2036405624 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb0c0cb010 0x7fdb0c154b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:30.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.099+0000 7fdb130e6640 1 --2- 172.21.15.49:0/2036405624 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdb0c0b0790 0x7fdb0c0acfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:30.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.099+0000 7fdb130e6640 1 -- 172.21.15.49:0/2036405624 >> 172.21.15.49:0/2036405624 conn(0x7fdb0c06c0a0 msgr2=0x7fdb0c0c9bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:30.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.099+0000 7fdb130e6640 1 -- 172.21.15.49:0/2036405624 shutdown_connections 2023-11-06T02:10:30.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.099+0000 7fdb130e6640 1 -- 172.21.15.49:0/2036405624 wait complete. 2023-11-06T02:10:30.116 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-06T02:10:30.117 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-06T02:10:30.242 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.243+0000 7fed09ce0640 1 Processor -- start 2023-11-06T02:10:30.242 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.243+0000 7fed09ce0640 1 -- start start 2023-11-06T02:10:30.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.243+0000 7fed09ce0640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fed0415ac60 0x7fed040c9140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:30.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.243+0000 7fed09ce0640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fed040c9680 0x7fed04069d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:30.246 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.243+0000 7fed09ce0640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed040b26b0 con 0x7fed0406a2b0 2023-11-06T02:10:30.246 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.243+0000 7fed09ce0640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed0406a740 con 0x7fed0415ac60 2023-11-06T02:10:30.246 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.243+0000 7fed09ce0640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed040c9bc0 con 0x7fed040c9680 2023-11-06T02:10:30.247 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.243+0000 7fed037fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fed0415ac60 0x7fed040c9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:30.247 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.243+0000 7fed037fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fed0415ac60 0x7fed040c9140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43966/0 (socket says 172.21.15.49:43966) 2023-11-06T02:10:30.247 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.243+0000 7fed037fe640 1 -- 172.21.15.49:0/2693911647 learned_addr learned my addr 172.21.15.49:0/2693911647 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:30.248 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.243+0000 7fed02ffd640 1 --2- 172.21.15.49:0/2693911647 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fed040c9680 0x7fed04069d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:30.248 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.243+0000 7fed037fe640 1 -- 172.21.15.49:0/2693911647 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fed040c9680 msgr2=0x7fed04069d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:30.249 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.243+0000 7fed037fe640 1 --2- 172.21.15.49:0/2693911647 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fed040c9680 0x7fed04069d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:30.249 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.243+0000 7fed037fe640 1 -- 172.21.15.49:0/2693911647 >> v1:172.21.15.150:6789/0 conn(0x7fed0406a2b0 legacy=0x7fed040b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:30.249 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.243+0000 7fed037fe640 1 -- 172.21.15.49:0/2693911647 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed040c9ea0 con 0x7fed0415ac60 2023-11-06T02:10:30.256 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.247+0000 7fed037fe640 1 --2- 172.21.15.49:0/2693911647 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed0415ac60 0x7fed040c9140 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fecf4009870 tx=0x7fecf401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=561eeaaab706b55e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:30.256 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.247+0000 7fed027fc640 1 -- 172.21.15.49:0/2693911647 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf40ac410 con 0x7fed0415ac60 2023-11-06T02:10:30.257 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.247+0000 7fed027fc640 1 -- 172.21.15.49:0/2693911647 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecf40ac570 con 0x7fed0415ac60 2023-11-06T02:10:30.257 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.247+0000 7fed027fc640 1 -- 172.21.15.49:0/2693911647 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf40ac900 con 0x7fed0415ac60 2023-11-06T02:10:30.257 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.247+0000 7fed09ce0640 1 -- 172.21.15.49:0/2693911647 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed0415ac60 msgr2=0x7fed040c9140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:30.258 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.247+0000 7fed09ce0640 1 --2- 172.21.15.49:0/2693911647 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed0415ac60 0x7fed040c9140 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fecf4009870 tx=0x7fecf401c4d0 comp rx=0 tx=0).stop 2023-11-06T02:10:30.258 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.247+0000 7fed09ce0640 1 -- 172.21.15.49:0/2693911647 shutdown_connections 2023-11-06T02:10:30.259 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.247+0000 7fed09ce0640 1 --2- 172.21.15.49:0/2693911647 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fed040c9680 0x7fed04069d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:30.259 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.247+0000 7fed09ce0640 1 --2- 172.21.15.49:0/2693911647 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed0415ac60 0x7fed040c9140 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:30.259 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.247+0000 7fed09ce0640 1 -- 172.21.15.49:0/2693911647 >> 172.21.15.49:0/2693911647 conn(0x7fed040ce610 msgr2=0x7fed040cea20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:30.260 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.247+0000 7fed09ce0640 1 -- 172.21.15.49:0/2693911647 shutdown_connections 2023-11-06T02:10:30.260 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.247+0000 7fed09ce0640 1 -- 172.21.15.49:0/2693911647 wait complete. 2023-11-06T02:10:30.260 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.247+0000 7fed09ce0640 1 Processor -- start 2023-11-06T02:10:30.263 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.247+0000 7fed09ce0640 1 -- start start 2023-11-06T02:10:30.264 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.247+0000 7fed09ce0640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fed0406a2b0 0x7fed04148f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:30.264 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.247+0000 7fed09ce0640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fed040c9680 0x7fed040c5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:30.264 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.247+0000 7fed09ce0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed0415ac60 0x7fed040c2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:30.264 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.247+0000 7fed09ce0640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed04151ee0 con 0x7fed0415ac60 2023-11-06T02:10:30.264 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.247+0000 7fed09ce0640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed04151330 con 0x7fed0406a2b0 2023-11-06T02:10:30.265 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.247+0000 7fed09ce0640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed04149440 con 0x7fed040c9680 2023-11-06T02:10:30.265 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.247+0000 7fed03fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed0415ac60 0x7fed040c2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:30.265 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.247+0000 7fed03fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed0415ac60 0x7fed040c2090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43970/0 (socket says 172.21.15.49:43970) 2023-11-06T02:10:30.265 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.247+0000 7fed03fff640 1 -- 172.21.15.49:0/3179094005 learned_addr learned my addr 172.21.15.49:0/3179094005 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:30.265 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.251+0000 7fed037fe640 1 --2- 172.21.15.49:0/3179094005 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fed0406a2b0 0x7fed04148f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:30.266 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.251+0000 7fed02ffd640 1 --2- 172.21.15.49:0/3179094005 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fed040c9680 0x7fed040c5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:30.266 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.251+0000 7fed03fff640 1 -- 172.21.15.49:0/3179094005 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fed040c9680 msgr2=0x7fed040c5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:30.266 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.251+0000 7fed03fff640 1 --2- 172.21.15.49:0/3179094005 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fed040c9680 0x7fed040c5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:30.266 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.251+0000 7fed03fff640 1 -- 172.21.15.49:0/3179094005 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fed0406a2b0 msgr2=0x7fed04148f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:30.266 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.251+0000 7fed03fff640 1 --2- 172.21.15.49:0/3179094005 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fed0406a2b0 0x7fed04148f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:30.267 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.251+0000 7fed03fff640 1 -- 172.21.15.49:0/3179094005 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecf40094d0 con 0x7fed0415ac60 2023-11-06T02:10:30.267 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.251+0000 7fed037fe640 1 --2- 172.21.15.49:0/3179094005 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fed0406a2b0 0x7fed04148f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:10:30.267 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.251+0000 7fed03fff640 1 --2- 172.21.15.49:0/3179094005 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed0415ac60 0x7fed040c2090 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fecfc00c780 tx=0x7fecfc00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:30.267 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.251+0000 7fecebfff640 1 -- 172.21.15.49:0/3179094005 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecfc004b70 con 0x7fed0415ac60 2023-11-06T02:10:30.267 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.251+0000 7fecebfff640 1 -- 172.21.15.49:0/3179094005 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecfc00ce90 con 0x7fed0415ac60 2023-11-06T02:10:30.268 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.251+0000 7fed09ce0640 1 -- 172.21.15.49:0/3179094005 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed04151540 con 0x7fed0415ac60 2023-11-06T02:10:30.268 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.251+0000 7fecebfff640 1 -- 172.21.15.49:0/3179094005 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecfc00a850 con 0x7fed0415ac60 2023-11-06T02:10:30.268 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.251+0000 7fed09ce0640 1 -- 172.21.15.49:0/3179094005 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed04151b00 con 0x7fed0415ac60 2023-11-06T02:10:30.268 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.251+0000 7fed09ce0640 1 -- 172.21.15.49:0/3179094005 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed04152020 con 0x7fed0415ac60 2023-11-06T02:10:30.268 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.251+0000 7fecebfff640 1 -- 172.21.15.49:0/3179094005 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fecfc006b60 con 0x7fed0415ac60 2023-11-06T02:10:30.269 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.251+0000 7fecebfff640 1 --2- 172.21.15.49:0/3179094005 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fecf0065a60 0x7fecf0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:30.269 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.251+0000 7fecebfff640 1 -- 172.21.15.49:0/3179094005 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fecfc083940 con 0x7fed0415ac60 2023-11-06T02:10:30.269 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.251+0000 7fed037fe640 1 --2- 172.21.15.49:0/3179094005 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fecf0065a60 0x7fecf0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:30.269 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.251+0000 7fed037fe640 1 --2- 172.21.15.49:0/3179094005 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fecf0065a60 0x7fecf0067f20 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fecf40ae000 tx=0x7fecf4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:30.269 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.255+0000 7fecebfff640 1 -- 172.21.15.49:0/3179094005 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fecfc051660 con 0x7fed0415ac60 2023-11-06T02:10:30.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.411+0000 7fed09ce0640 1 -- 172.21.15.49:0/3179094005 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fed04152020 con 0x7fed0415ac60 2023-11-06T02:10:30.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.511+0000 7fecebfff640 1 -- 172.21.15.49:0/3179094005 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v48) v1 ==== 138+0+0 (secure 0 0 0) 0x7fed04152020 con 0x7fed0415ac60 2023-11-06T02:10:30.512 INFO:teuthology.orchestra.run.smithi049.stderr:pool 'cephfs_metadata' created 2023-11-06T02:10:30.516 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.515+0000 7fed09ce0640 1 -- 172.21.15.49:0/3179094005 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fecf0065a60 msgr2=0x7fecf0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:30.516 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.515+0000 7fed09ce0640 1 --2- 172.21.15.49:0/3179094005 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fecf0065a60 0x7fecf0067f20 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fecf40ae000 tx=0x7fecf4008040 comp rx=0 tx=0).stop 2023-11-06T02:10:30.516 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.515+0000 7fed09ce0640 1 -- 172.21.15.49:0/3179094005 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed0415ac60 msgr2=0x7fed040c2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:30.516 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.515+0000 7fed09ce0640 1 --2- 172.21.15.49:0/3179094005 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed0415ac60 0x7fed040c2090 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fecfc00c780 tx=0x7fecfc00cc40 comp rx=0 tx=0).stop 2023-11-06T02:10:30.516 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.515+0000 7fed09ce0640 1 -- 172.21.15.49:0/3179094005 shutdown_connections 2023-11-06T02:10:30.517 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.515+0000 7fed09ce0640 1 --2- 172.21.15.49:0/3179094005 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fecf0065a60 0x7fecf0067f20 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:30.517 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.515+0000 7fed09ce0640 1 --2- 172.21.15.49:0/3179094005 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed0415ac60 0x7fed040c2090 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:30.517 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.515+0000 7fed09ce0640 1 --2- 172.21.15.49:0/3179094005 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fed040c9680 0x7fed040c5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:30.517 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.515+0000 7fed09ce0640 1 --2- 172.21.15.49:0/3179094005 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fed0406a2b0 0x7fed04148f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:30.517 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.515+0000 7fed09ce0640 1 -- 172.21.15.49:0/3179094005 >> 172.21.15.49:0/3179094005 conn(0x7fed040ce610 msgr2=0x7fed040b3e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:30.518 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.515+0000 7fed09ce0640 1 -- 172.21.15.49:0/3179094005 shutdown_connections 2023-11-06T02:10:30.518 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.515+0000 7fed09ce0640 1 -- 172.21.15.49:0/3179094005 wait complete. 2023-11-06T02:10:30.543 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-06T02:10:30.679 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.679+0000 7f4ff91b2640 1 Processor -- start 2023-11-06T02:10:30.680 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.679+0000 7f4ff91b2640 1 -- start start 2023-11-06T02:10:30.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.679+0000 7f4ff91b2640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f4ff406a480 0x7f4ff40cb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:30.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.679+0000 7f4ff91b2640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ff40cf900 con 0x7f4ff415cb10 2023-11-06T02:10:30.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.679+0000 7f4ff91b2640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ff40cfb70 con 0x7f4ff415cef0 2023-11-06T02:10:30.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.679+0000 7f4ff91b2640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ff40cfe50 con 0x7f4ff406a480 2023-11-06T02:10:30.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.683+0000 7f4ff2d9d640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f4ff415cb10 0x7f4ff40cdaf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:41532/0 (socket says 172.21.15.49:41532) 2023-11-06T02:10:30.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.683+0000 7f4ff2d9d640 1 -- 172.21.15.49:0/1258917248 learned_addr learned my addr 172.21.15.49:0/1258917248 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:10:30.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.683+0000 7f4ff359e640 1 --2- 172.21.15.49:0/1258917248 >> v2:172.21.15.150:3300/0 conn(0x7f4ff406a480 0x7f4ff40cb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:30.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.683+0000 7f4ff1d9b640 1 -- 172.21.15.49:0/1258917248 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1249994434 0 0) 0x7f4ff40cf900 con 0x7f4ff415cb10 2023-11-06T02:10:30.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.683+0000 7f4ff1d9b640 1 -- 172.21.15.49:0/1258917248 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fdc003530 con 0x7f4ff415cb10 2023-11-06T02:10:30.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.683+0000 7f4ff1d9b640 1 -- 172.21.15.49:0/1258917248 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 160121501 0 0) 0x7f4fdc003530 con 0x7f4ff415cb10 2023-11-06T02:10:30.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.683+0000 7f4ff1d9b640 1 -- 172.21.15.49:0/1258917248 >> v2:172.21.15.150:3300/0 conn(0x7f4ff406a480 msgr2=0x7f4ff40cb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:30.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.683+0000 7f4ff1d9b640 1 --2- 172.21.15.49:0/1258917248 >> v2:172.21.15.150:3300/0 conn(0x7f4ff406a480 0x7f4ff40cb0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:30.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.683+0000 7f4ff1d9b640 1 -- 172.21.15.49:0/1258917248 >> v1:172.21.15.150:6789/0 conn(0x7f4ff415cef0 legacy=0x7f4ff4069d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:30.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.683+0000 7f4ff1d9b640 1 -- 172.21.15.49:0/1258917248 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ff40d0130 con 0x7f4ff415cb10 2023-11-06T02:10:30.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.683+0000 7f4ff1d9b640 1 -- 172.21.15.49:0/1258917248 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7f4fe4001680 con 0x7f4ff415cb10 2023-11-06T02:10:30.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.683+0000 7f4ff1d9b640 1 -- 172.21.15.49:0/1258917248 >> v1:172.21.15.49:6789/0 conn(0x7f4ff415cb10 legacy=0x7f4ff40cdaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:30.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.683+0000 7f4ff1d9b640 1 --2- 172.21.15.49:0/1258917248 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4fdc003980 0x7f4fdc003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:30.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.683+0000 7f4ff1d9b640 1 --2- 172.21.15.49:0/1258917248 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4fdc0047d0 0x7f4fdc004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:30.686 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.683+0000 7f4ff1d9b640 1 --2- 172.21.15.49:0/1258917248 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4fdc005190 0x7f4fdc00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:30.686 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.683+0000 7f4ff1d9b640 1 -- 172.21.15.49:0/1258917248 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff40cfe50 con 0x7f4fdc0047d0 2023-11-06T02:10:30.686 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.683+0000 7f4ff1d9b640 1 -- 172.21.15.49:0/1258917248 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fdc00c130 con 0x7f4fdc005190 2023-11-06T02:10:30.686 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.683+0000 7f4ff1d9b640 1 -- 172.21.15.49:0/1258917248 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fdc00c2f0 con 0x7f4fdc003980 2023-11-06T02:10:30.686 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.683+0000 7f4ff259c640 1 --2- 172.21.15.49:0/1258917248 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4fdc0047d0 0x7f4fdc004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:30.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.683+0000 7f4ff2d9d640 1 --2- 172.21.15.49:0/1258917248 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4fdc003980 0x7f4fdc003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:30.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.683+0000 7f4ff359e640 1 --2- 172.21.15.49:0/1258917248 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4fdc005190 0x7f4fdc00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:30.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.683+0000 7f4ff259c640 1 -- 172.21.15.49:0/1258917248 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4fdc003980 msgr2=0x7f4fdc003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:30.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.683+0000 7f4ff259c640 1 --2- 172.21.15.49:0/1258917248 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4fdc003980 0x7f4fdc003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:30.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.683+0000 7f4ff259c640 1 -- 172.21.15.49:0/1258917248 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4fdc005190 msgr2=0x7f4fdc00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:30.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.683+0000 7f4ff259c640 1 --2- 172.21.15.49:0/1258917248 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4fdc005190 0x7f4fdc00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:30.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.683+0000 7f4ff259c640 1 -- 172.21.15.49:0/1258917248 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fdc003530 con 0x7f4fdc0047d0 2023-11-06T02:10:30.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.683+0000 7f4ff359e640 1 -- 172.21.15.49:0/1258917248 reap_dead start 2023-11-06T02:10:30.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.687+0000 7f4ff259c640 1 --2- 172.21.15.49:0/1258917248 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4fdc0047d0 0x7f4fdc004c50 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f4fe8001750 tx=0x7f4fe801c370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d7e4734c4f26cf server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:30.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.687+0000 7f4ff1d9b640 1 -- 172.21.15.49:0/1258917248 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe80052d0 con 0x7f4fdc0047d0 2023-11-06T02:10:30.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.687+0000 7f4ff1d9b640 1 -- 172.21.15.49:0/1258917248 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fe80a7050 con 0x7f4fdc0047d0 2023-11-06T02:10:30.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.687+0000 7f4ff1d9b640 1 -- 172.21.15.49:0/1258917248 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe8003600 con 0x7f4fdc0047d0 2023-11-06T02:10:30.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.687+0000 7f4ff91b2640 1 -- 172.21.15.49:0/1258917248 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4fdc0047d0 msgr2=0x7f4fdc004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:30.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.687+0000 7f4ff91b2640 1 --2- 172.21.15.49:0/1258917248 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4fdc0047d0 0x7f4fdc004c50 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f4fe8001750 tx=0x7f4fe801c370 comp rx=0 tx=0).stop 2023-11-06T02:10:30.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.687+0000 7f4ff91b2640 1 -- 172.21.15.49:0/1258917248 shutdown_connections 2023-11-06T02:10:30.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.687+0000 7f4ff91b2640 1 --2- 172.21.15.49:0/1258917248 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4fdc0047d0 0x7f4fdc004c50 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:30.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.687+0000 7f4ff91b2640 1 -- 172.21.15.49:0/1258917248 >> 172.21.15.49:0/1258917248 conn(0x7f4ff406d4b0 msgr2=0x7f4ff406d890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:30.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.687+0000 7f4ff91b2640 1 -- 172.21.15.49:0/1258917248 shutdown_connections 2023-11-06T02:10:30.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.687+0000 7f4ff91b2640 1 -- 172.21.15.49:0/1258917248 wait complete. 2023-11-06T02:10:30.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.687+0000 7f4ff91b2640 1 Processor -- start 2023-11-06T02:10:30.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.687+0000 7f4ff91b2640 1 -- start start 2023-11-06T02:10:30.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.687+0000 7f4ff91b2640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4fdc0047d0 0x7f4ff41f5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:30.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.687+0000 7f4ff91b2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4ff40af3f0 0x7f4ff40b0f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:30.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.687+0000 7f4ff91b2640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4ff4155250 0x7f4ff41556d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:30.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.687+0000 7f4ff91b2640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff4155c10 con 0x7f4ff40af3f0 2023-11-06T02:10:30.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.687+0000 7f4ff91b2640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff40aba60 con 0x7f4ff4155250 2023-11-06T02:10:30.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.687+0000 7f4ff91b2640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff40b2310 con 0x7f4fdc0047d0 2023-11-06T02:10:30.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.687+0000 7f4ff259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4ff40af3f0 0x7f4ff40b0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:30.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.687+0000 7f4ff259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4ff40af3f0 0x7f4ff40b0f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43980/0 (socket says 172.21.15.49:43980) 2023-11-06T02:10:30.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.687+0000 7f4ff259c640 1 -- 172.21.15.49:0/324028007 learned_addr learned my addr 172.21.15.49:0/324028007 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:30.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.687+0000 7f4ff2d9d640 1 --2- 172.21.15.49:0/324028007 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4fdc0047d0 0x7f4ff41f5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:30.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.687+0000 7f4ff359e640 1 --2- 172.21.15.49:0/324028007 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4ff4155250 0x7f4ff41556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:30.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.691+0000 7f4ff259c640 1 -- 172.21.15.49:0/324028007 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4fdc0047d0 msgr2=0x7f4ff41f5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:30.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.691+0000 7f4ff259c640 1 --2- 172.21.15.49:0/324028007 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4fdc0047d0 0x7f4ff41f5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:30.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.691+0000 7f4ff259c640 1 -- 172.21.15.49:0/324028007 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4ff4155250 msgr2=0x7f4ff41556d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:30.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.691+0000 7f4ff259c640 1 --2- 172.21.15.49:0/324028007 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4ff4155250 0x7f4ff41556d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:30.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.691+0000 7f4ff259c640 1 -- 172.21.15.49:0/324028007 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ff40b1790 con 0x7f4ff40af3f0 2023-11-06T02:10:30.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.691+0000 7f4ff359e640 1 --2- 172.21.15.49:0/324028007 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4ff4155250 0x7f4ff41556d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:10:30.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.691+0000 7f4ff259c640 1 --2- 172.21.15.49:0/324028007 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4ff40af3f0 0x7f4ff40b0f40 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f4fe8001750 tx=0x7f4fdc005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:30.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.691+0000 7f4fdb7fe640 1 -- 172.21.15.49:0/324028007 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe8007180 con 0x7f4ff40af3f0 2023-11-06T02:10:30.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.691+0000 7f4fdb7fe640 1 -- 172.21.15.49:0/324028007 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fe80072e0 con 0x7f4ff40af3f0 2023-11-06T02:10:30.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.691+0000 7f4ff91b2640 1 -- 172.21.15.49:0/324028007 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ff40b1a70 con 0x7f4ff40af3f0 2023-11-06T02:10:30.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.691+0000 7f4fdb7fe640 1 -- 172.21.15.49:0/324028007 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe8007650 con 0x7f4ff40af3f0 2023-11-06T02:10:30.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.691+0000 7f4ff91b2640 1 -- 172.21.15.49:0/324028007 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ff40b2030 con 0x7f4ff40af3f0 2023-11-06T02:10:30.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.691+0000 7f4ff91b2640 1 -- 172.21.15.49:0/324028007 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ff40b24e0 con 0x7f4ff40af3f0 2023-11-06T02:10:30.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.691+0000 7f4fdb7fe640 1 -- 172.21.15.49:0/324028007 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f4fe80067e0 con 0x7f4ff40af3f0 2023-11-06T02:10:30.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.691+0000 7f4fdb7fe640 1 --2- 172.21.15.49:0/324028007 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4fe0065a90 0x7f4fe0067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:30.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.691+0000 7f4fdb7fe640 1 -- 172.21.15.49:0/324028007 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5580+0+0 (secure 0 0 0) 0x7f4fe811d4d0 con 0x7f4ff40af3f0 2023-11-06T02:10:30.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.691+0000 7f4ff2d9d640 1 --2- 172.21.15.49:0/324028007 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4fe0065a90 0x7f4fe0067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:30.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.691+0000 7f4ff2d9d640 1 --2- 172.21.15.49:0/324028007 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4fe0065a90 0x7f4fe0067f50 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f4fe4000fc0 tx=0x7f4fe4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:30.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.695+0000 7f4fdb7fe640 1 -- 172.21.15.49:0/324028007 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f4fe80eae80 con 0x7f4ff40af3f0 2023-11-06T02:10:30.853 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:30.851+0000 7f4ff91b2640 1 -- 172.21.15.49:0/324028007 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f4ff4067140 con 0x7f4ff40af3f0 2023-11-06T02:10:31.522 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:31.523+0000 7f4fdb7fe640 1 -- 172.21.15.49:0/324028007 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v49) v1 ==== 170+0+0 (secure 0 0 0) 0x7f4fe80ee4a0 con 0x7f4ff40af3f0 2023-11-06T02:10:31.524 INFO:teuthology.orchestra.run.smithi049.stderr:pool 'cephfs_data' created 2023-11-06T02:10:31.537 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:31.535+0000 7f4ff91b2640 1 -- 172.21.15.49:0/324028007 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4fe0065a90 msgr2=0x7f4fe0067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:31.537 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:31.535+0000 7f4ff91b2640 1 --2- 172.21.15.49:0/324028007 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4fe0065a90 0x7f4fe0067f50 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f4fe4000fc0 tx=0x7f4fe4009040 comp rx=0 tx=0).stop 2023-11-06T02:10:31.537 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:31.535+0000 7f4ff91b2640 1 -- 172.21.15.49:0/324028007 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4ff40af3f0 msgr2=0x7f4ff40b0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:31.538 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:31.535+0000 7f4ff91b2640 1 --2- 172.21.15.49:0/324028007 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4ff40af3f0 0x7f4ff40b0f40 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f4fe8001750 tx=0x7f4fdc005800 comp rx=0 tx=0).stop 2023-11-06T02:10:31.538 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:31.535+0000 7f4ff91b2640 1 -- 172.21.15.49:0/324028007 shutdown_connections 2023-11-06T02:10:31.538 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:31.535+0000 7f4ff91b2640 1 --2- 172.21.15.49:0/324028007 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4fe0065a90 0x7f4fe0067f50 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:31.538 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:31.535+0000 7f4ff91b2640 1 --2- 172.21.15.49:0/324028007 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4ff40af3f0 0x7f4ff40b0f40 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:31.538 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:31.535+0000 7f4ff91b2640 1 --2- 172.21.15.49:0/324028007 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4ff4155250 0x7f4ff41556d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:31.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:31.535+0000 7f4ff91b2640 1 --2- 172.21.15.49:0/324028007 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4fdc0047d0 0x7f4ff41f5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:31.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:31.535+0000 7f4ff91b2640 1 -- 172.21.15.49:0/324028007 >> 172.21.15.49:0/324028007 conn(0x7f4ff406d4b0 msgr2=0x7f4ff4152de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:31.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:31.535+0000 7f4ff91b2640 1 -- 172.21.15.49:0/324028007 shutdown_connections 2023-11-06T02:10:31.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:31.535+0000 7f4ff91b2640 1 -- 172.21.15.49:0/324028007 wait complete. 2023-11-06T02:10:31.569 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-06T02:10:31.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:31.691+0000 7f0a5e781640 1 Processor -- start 2023-11-06T02:10:31.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:31.691+0000 7f0a5e781640 1 -- start start 2023-11-06T02:10:31.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:31.691+0000 7f0a5e781640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0a580caec0 0x7f0a580cb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:31.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:31.691+0000 7f0a5e781640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f0a5815dd10 0x7f0a580cd190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:31.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:31.691+0000 7f0a5e781640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a5815ad50 con 0x7f0a580c9600 2023-11-06T02:10:31.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:31.691+0000 7f0a5e781640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a5815e280 con 0x7f0a5815dd10 2023-11-06T02:10:31.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:31.691+0000 7f0a5e781640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a580cecf0 con 0x7f0a580caec0 2023-11-06T02:10:31.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:31.691+0000 7f0a5cd1e640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f0a5815dd10 0x7f0a580cd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:31.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:31.691+0000 7f0a57fff640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0a580caec0 0x7f0a580cb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:31.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:31.691+0000 7f0a577fe640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f0a580c9600 0x7f0a58069d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.49:36298/0 (socket says 172.21.15.49:36298) 2023-11-06T02:10:31.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:31.691+0000 7f0a577fe640 1 -- 172.21.15.49:0/3487857148 learned_addr learned my addr 172.21.15.49:0/3487857148 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:10:31.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:31.691+0000 7f0a5cd1e640 1 -- 172.21.15.49:0/3487857148 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0a580caec0 msgr2=0x7f0a580cb2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:31.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:31.691+0000 7f0a5cd1e640 1 --2- 172.21.15.49:0/3487857148 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0a580caec0 0x7f0a580cb2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:31.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:31.691+0000 7f0a5cd1e640 1 -- 172.21.15.49:0/3487857148 >> v1:172.21.15.150:6789/0 conn(0x7f0a580c9600 legacy=0x7f0a58069d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:31.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:31.691+0000 7f0a5cd1e640 1 -- 172.21.15.49:0/3487857148 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a580cb7e0 con 0x7f0a5815dd10 2023-11-06T02:10:31.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:31.695+0000 7f0a5cd1e640 1 --2- 172.21.15.49:0/3487857148 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a5815dd10 0x7f0a580cd190 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0a480097c0 tx=0x7f0a4801c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18386853ddb17ec7 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:31.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:31.695+0000 7f0a56ffd640 1 -- 172.21.15.49:0/3487857148 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a480ac620 con 0x7f0a5815dd10 2023-11-06T02:10:31.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:31.695+0000 7f0a56ffd640 1 -- 172.21.15.49:0/3487857148 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a480ac780 con 0x7f0a5815dd10 2023-11-06T02:10:31.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:31.695+0000 7f0a56ffd640 1 -- 172.21.15.49:0/3487857148 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a480acb10 con 0x7f0a5815dd10 2023-11-06T02:10:31.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:31.695+0000 7f0a5e781640 1 -- 172.21.15.49:0/3487857148 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a5815dd10 msgr2=0x7f0a580cd190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:31.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:31.695+0000 7f0a5e781640 1 --2- 172.21.15.49:0/3487857148 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a5815dd10 0x7f0a580cd190 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0a480097c0 tx=0x7f0a4801c3c0 comp rx=0 tx=0).stop 2023-11-06T02:10:31.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:31.695+0000 7f0a5e781640 1 -- 172.21.15.49:0/3487857148 shutdown_connections 2023-11-06T02:10:31.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:31.695+0000 7f0a5e781640 1 --2- 172.21.15.49:0/3487857148 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a5815dd10 0x7f0a580cd190 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:31.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:31.695+0000 7f0a5e781640 1 --2- 172.21.15.49:0/3487857148 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0a580caec0 0x7f0a580cb2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:31.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:31.695+0000 7f0a5e781640 1 -- 172.21.15.49:0/3487857148 >> 172.21.15.49:0/3487857148 conn(0x7f0a580b0aa0 msgr2=0x7f0a580b0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:31.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:31.695+0000 7f0a5e781640 1 -- 172.21.15.49:0/3487857148 shutdown_connections 2023-11-06T02:10:31.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:31.695+0000 7f0a5e781640 1 -- 172.21.15.49:0/3487857148 wait complete. 2023-11-06T02:10:31.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:31.695+0000 7f0a5e781640 1 Processor -- start 2023-11-06T02:10:31.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:31.699+0000 7f0a5e781640 1 -- start start 2023-11-06T02:10:31.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:31.699+0000 7f0a5e781640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0a580c9600 0x7f0a58155ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:31.701 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:31.699+0000 7f0a5e781640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0a580caec0 0x7f0a580c2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:31.701 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:31.699+0000 7f0a5e781640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a5815dd10 0x7f0a58157b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:31.701 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:31.699+0000 7f0a5e781640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a580c59a0 con 0x7f0a580c9600 2023-11-06T02:10:31.701 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:31.699+0000 7f0a5e781640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a5806ea60 con 0x7f0a5815dd10 2023-11-06T02:10:31.701 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:31.699+0000 7f0a5e781640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a5806db90 con 0x7f0a580caec0 2023-11-06T02:10:31.701 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:31.699+0000 7f0a57fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0a580c9600 0x7f0a58155ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:31.702 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:31.699+0000 7f0a57fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0a580c9600 0x7f0a58155ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:43996/0 (socket says 172.21.15.49:43996) 2023-11-06T02:10:31.702 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:31.699+0000 7f0a57fff640 1 -- 172.21.15.49:0/1352798158 learned_addr learned my addr 172.21.15.49:0/1352798158 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:31.702 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:31.699+0000 7f0a577fe640 1 --2- 172.21.15.49:0/1352798158 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0a580caec0 0x7f0a580c2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:31.702 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:31.699+0000 7f0a5cd1e640 1 --2- 172.21.15.49:0/1352798158 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a5815dd10 0x7f0a58157b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:31.702 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:31.699+0000 7f0a57fff640 1 -- 172.21.15.49:0/1352798158 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0a580caec0 msgr2=0x7f0a580c2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:31.703 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:31.699+0000 7f0a57fff640 1 --2- 172.21.15.49:0/1352798158 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0a580caec0 0x7f0a580c2ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:31.703 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:31.699+0000 7f0a57fff640 1 -- 172.21.15.49:0/1352798158 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a5815dd10 msgr2=0x7f0a58157b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:31.703 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:31.699+0000 7f0a57fff640 1 --2- 172.21.15.49:0/1352798158 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a5815dd10 0x7f0a58157b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:31.703 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:31.699+0000 7f0a57fff640 1 -- 172.21.15.49:0/1352798158 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a48009420 con 0x7f0a580c9600 2023-11-06T02:10:31.703 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:31.699+0000 7f0a57fff640 1 --2- 172.21.15.49:0/1352798158 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0a580c9600 0x7f0a58155ff0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f0a5000efc0 tx=0x7f0a5000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:31.704 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:31.699+0000 7f0a54ff9640 1 -- 172.21.15.49:0/1352798158 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a50009b80 con 0x7f0a580c9600 2023-11-06T02:10:31.704 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:31.699+0000 7f0a54ff9640 1 -- 172.21.15.49:0/1352798158 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a5000f040 con 0x7f0a580c9600 2023-11-06T02:10:31.704 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:31.699+0000 7f0a5e781640 1 -- 172.21.15.49:0/1352798158 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a5806dde0 con 0x7f0a580c9600 2023-11-06T02:10:31.704 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:31.699+0000 7f0a54ff9640 1 -- 172.21.15.49:0/1352798158 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a500146a0 con 0x7f0a580c9600 2023-11-06T02:10:31.704 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:31.699+0000 7f0a5e781640 1 -- 172.21.15.49:0/1352798158 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a5806e3a0 con 0x7f0a580c9600 2023-11-06T02:10:31.705 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:31.699+0000 7f0a5e781640 1 -- 172.21.15.49:0/1352798158 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a5806f500 con 0x7f0a580c9600 2023-11-06T02:10:31.705 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:31.699+0000 7f0a54ff9640 1 -- 172.21.15.49:0/1352798158 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f0a50014800 con 0x7f0a580c9600 2023-11-06T02:10:31.706 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:31.703+0000 7f0a54ff9640 1 --2- 172.21.15.49:0/1352798158 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0a38065a60 0x7f0a38067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:31.706 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:31.703+0000 7f0a54ff9640 1 -- 172.21.15.49:0/1352798158 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5962+0+0 (secure 0 0 0) 0x7f0a50084090 con 0x7f0a580c9600 2023-11-06T02:10:31.706 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:31.703+0000 7f0a577fe640 1 --2- 172.21.15.49:0/1352798158 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0a38065a60 0x7f0a38067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:31.706 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:31.703+0000 7f0a577fe640 1 --2- 172.21.15.49:0/1352798158 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0a38065a60 0x7f0a38067f20 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0a58152640 tx=0x7f0a40009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:31.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:31.707+0000 7f0a54ff9640 1 -- 172.21.15.49:0/1352798158 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f0a50051950 con 0x7f0a580c9600 2023-11-06T02:10:31.896 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:31.895+0000 7f0a5e781640 1 -- 172.21.15.49:0/1352798158 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f0a580b1620 con 0x7f0a580c9600 2023-11-06T02:10:31.913 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:31.911+0000 7f0a54ff9640 1 -- 172.21.15.49:0/1352798158 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 7 and data pool 8 v19) v1 ==== 172+0+0 (secure 0 0 0) 0x7f0a50054f70 con 0x7f0a580c9600 2023-11-06T02:10:31.913 INFO:teuthology.orchestra.run.smithi049.stderr:new fs with metadata pool 7 and data pool 8 2023-11-06T02:10:31.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:31.915+0000 7f0a5e781640 1 -- 172.21.15.49:0/1352798158 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0a38065a60 msgr2=0x7f0a38067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:31.917 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:31.915+0000 7f0a5e781640 1 --2- 172.21.15.49:0/1352798158 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0a38065a60 0x7f0a38067f20 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0a58152640 tx=0x7f0a40009040 comp rx=0 tx=0).stop 2023-11-06T02:10:31.917 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:31.915+0000 7f0a5e781640 1 -- 172.21.15.49:0/1352798158 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0a580c9600 msgr2=0x7f0a58155ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:31.917 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:31.915+0000 7f0a5e781640 1 --2- 172.21.15.49:0/1352798158 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0a580c9600 0x7f0a58155ff0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f0a5000efc0 tx=0x7f0a5000c490 comp rx=0 tx=0).stop 2023-11-06T02:10:31.918 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:31.915+0000 7f0a5e781640 1 -- 172.21.15.49:0/1352798158 shutdown_connections 2023-11-06T02:10:31.918 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:31.915+0000 7f0a5e781640 1 --2- 172.21.15.49:0/1352798158 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0a38065a60 0x7f0a38067f20 secure :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0a58152640 tx=0x7f0a40009040 comp rx=0 tx=0).stop 2023-11-06T02:10:31.918 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:31.915+0000 7f0a5e781640 1 --2- 172.21.15.49:0/1352798158 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a5815dd10 0x7f0a58157b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:31.918 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:31.915+0000 7f0a5e781640 1 --2- 172.21.15.49:0/1352798158 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0a580caec0 0x7f0a580c2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:31.919 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:31.915+0000 7f0a5e781640 1 --2- 172.21.15.49:0/1352798158 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0a580c9600 0x7f0a58155ff0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:31.919 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:31.915+0000 7f0a5e781640 1 -- 172.21.15.49:0/1352798158 >> 172.21.15.49:0/1352798158 conn(0x7f0a580b0aa0 msgr2=0x7f0a5815b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:31.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:31.919+0000 7f0a5e781640 1 -- 172.21.15.49:0/1352798158 shutdown_connections 2023-11-06T02:10:31.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:31.919+0000 7f0a5e781640 1 -- 172.21.15.49:0/1352798158 wait complete. 2023-11-06T02:10:31.949 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T02:10:32.065 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.067+0000 7f7e0ae9a640 1 Processor -- start 2023-11-06T02:10:32.066 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.067+0000 7f7e0ae9a640 1 -- start start 2023-11-06T02:10:32.067 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.067+0000 7f7e0ae9a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7e040cb580 0x7f7e040cb960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:32.067 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.067+0000 7f7e0ae9a640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f7e040b6100 0x7f7e0415e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:32.068 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.067+0000 7f7e0ae9a640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f7e040ce420 0x7f7e040ce870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:32.068 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.067+0000 7f7e0ae9a640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e0415f3c0 con 0x7f7e040b6100 2023-11-06T02:10:32.068 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.067+0000 7f7e0ae9a640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e0415eb20 con 0x7f7e040ce420 2023-11-06T02:10:32.068 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.067+0000 7f7e0ae9a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e0415ee00 con 0x7f7e040cb580 2023-11-06T02:10:32.068 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.067+0000 7f7e03fff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f7e040b6100 0x7f7e0415e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:32.069 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.067+0000 7f7e03fff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f7e040b6100 0x7f7e0415e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44004/0 (socket says 172.21.15.49:44004) 2023-11-06T02:10:32.069 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.067+0000 7f7e03fff640 1 -- 172.21.15.49:0/2298170462 learned_addr learned my addr 172.21.15.49:0/2298170462 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:32.069 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.067+0000 7f7e09437640 1 --2- 172.21.15.49:0/2298170462 >> v2:172.21.15.150:3300/0 conn(0x7f7e040ce420 0x7f7e040ce870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:32.069 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.067+0000 7f7e08c36640 1 --2- 172.21.15.49:0/2298170462 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7e040cb580 0x7f7e040cb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:32.069 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.067+0000 7f7e03fff640 1 -- 172.21.15.49:0/2298170462 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7e040cb580 msgr2=0x7f7e040cb960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:32.070 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.067+0000 7f7e03fff640 1 --2- 172.21.15.49:0/2298170462 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7e040cb580 0x7f7e040cb960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:32.070 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.067+0000 7f7e03fff640 1 -- 172.21.15.49:0/2298170462 >> v2:172.21.15.150:3300/0 conn(0x7f7e040ce420 msgr2=0x7f7e040ce870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:32.070 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.067+0000 7f7e03fff640 1 --2- 172.21.15.49:0/2298170462 >> v2:172.21.15.150:3300/0 conn(0x7f7e040ce420 0x7f7e040ce870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:32.070 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.067+0000 7f7e03fff640 1 -- 172.21.15.49:0/2298170462 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e0415f0e0 con 0x7f7e040b6100 2023-11-06T02:10:32.070 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.067+0000 7f7e09437640 1 --2- 172.21.15.49:0/2298170462 >> v2:172.21.15.150:3300/0 conn(0x7f7e040ce420 0x7f7e040ce870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:10:32.073 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.071+0000 7f7e03fff640 1 --2- 172.21.15.49:0/2298170462 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e040b6100 0x7f7e0415e440 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f7e040c78d0 tx=0x7f7df801c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=537e96c3e9bf3ce6 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:32.074 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.071+0000 7f7e037fe640 1 -- 172.21.15.49:0/2298170462 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7df80ad640 con 0x7f7e040b6100 2023-11-06T02:10:32.074 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.071+0000 7f7e037fe640 1 -- 172.21.15.49:0/2298170462 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7df80ad7a0 con 0x7f7e040b6100 2023-11-06T02:10:32.074 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.071+0000 7f7e037fe640 1 -- 172.21.15.49:0/2298170462 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7df80adb30 con 0x7f7e040b6100 2023-11-06T02:10:32.074 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.071+0000 7f7e0ae9a640 1 -- 172.21.15.49:0/2298170462 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e040b6100 msgr2=0x7f7e0415e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:32.074 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.071+0000 7f7e0ae9a640 1 --2- 172.21.15.49:0/2298170462 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e040b6100 0x7f7e0415e440 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f7e040c78d0 tx=0x7f7df801c3b0 comp rx=0 tx=0).stop 2023-11-06T02:10:32.075 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.071+0000 7f7e0ae9a640 1 -- 172.21.15.49:0/2298170462 shutdown_connections 2023-11-06T02:10:32.075 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.071+0000 7f7e0ae9a640 1 --2- 172.21.15.49:0/2298170462 >> v2:172.21.15.150:3300/0 conn(0x7f7e040ce420 0x7f7e040ce870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:32.075 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.071+0000 7f7e0ae9a640 1 --2- 172.21.15.49:0/2298170462 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e040b6100 0x7f7e0415e440 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:32.075 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.071+0000 7f7e0ae9a640 1 --2- 172.21.15.49:0/2298170462 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7e040cb580 0x7f7e040cb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:32.075 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.071+0000 7f7e0ae9a640 1 -- 172.21.15.49:0/2298170462 >> 172.21.15.49:0/2298170462 conn(0x7f7e040cd2d0 msgr2=0x7f7e040cd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:32.076 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.071+0000 7f7e0ae9a640 1 -- 172.21.15.49:0/2298170462 shutdown_connections 2023-11-06T02:10:32.076 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.071+0000 7f7e0ae9a640 1 -- 172.21.15.49:0/2298170462 wait complete. 2023-11-06T02:10:32.076 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.071+0000 7f7e0ae9a640 1 Processor -- start 2023-11-06T02:10:32.076 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.071+0000 7f7e0ae9a640 1 -- start start 2023-11-06T02:10:32.076 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.071+0000 7f7e0ae9a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7e040b6100 0x7f7e040c6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:32.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.071+0000 7f7e0ae9a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7e040cb580 0x7f7e04151270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:32.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.071+0000 7f7e0ae9a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e040ce420 0x7f7e0415cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:32.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.071+0000 7f7e0ae9a640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e040c7920 con 0x7f7e040ce420 2023-11-06T02:10:32.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.071+0000 7f7e0ae9a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e04151b50 con 0x7f7e040b6100 2023-11-06T02:10:32.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.071+0000 7f7e0ae9a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e04151e30 con 0x7f7e040cb580 2023-11-06T02:10:32.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.071+0000 7f7e09437640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e040ce420 0x7f7e0415cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:32.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.071+0000 7f7e09437640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e040ce420 0x7f7e0415cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44016/0 (socket says 172.21.15.49:44016) 2023-11-06T02:10:32.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.071+0000 7f7e09437640 1 -- 172.21.15.49:0/138595446 learned_addr learned my addr 172.21.15.49:0/138595446 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:32.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.071+0000 7f7e08c36640 1 --2- 172.21.15.49:0/138595446 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7e040b6100 0x7f7e040c6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:32.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.071+0000 7f7e03fff640 1 --2- 172.21.15.49:0/138595446 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7e040cb580 0x7f7e04151270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:32.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.071+0000 7f7e09437640 1 -- 172.21.15.49:0/138595446 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7e040cb580 msgr2=0x7f7e04151270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:32.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.071+0000 7f7e09437640 1 --2- 172.21.15.49:0/138595446 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7e040cb580 0x7f7e04151270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:32.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.071+0000 7f7e09437640 1 -- 172.21.15.49:0/138595446 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7e040b6100 msgr2=0x7f7e040c6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:32.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.071+0000 7f7e09437640 1 --2- 172.21.15.49:0/138595446 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7e040b6100 0x7f7e040c6330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:32.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.071+0000 7f7e09437640 1 -- 172.21.15.49:0/138595446 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7df80094d0 con 0x7f7e040ce420 2023-11-06T02:10:32.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.071+0000 7f7e09437640 1 --2- 172.21.15.49:0/138595446 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e040ce420 0x7f7e0415cc30 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc00cef0 tx=0x7f7dfc00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:32.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.075+0000 7f7e017fa640 1 -- 172.21.15.49:0/138595446 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dfc00d840 con 0x7f7e040ce420 2023-11-06T02:10:32.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.075+0000 7f7e017fa640 1 -- 172.21.15.49:0/138595446 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dfc00d9c0 con 0x7f7e040ce420 2023-11-06T02:10:32.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.075+0000 7f7e017fa640 1 -- 172.21.15.49:0/138595446 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dfc00dd70 con 0x7f7e040ce420 2023-11-06T02:10:32.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.075+0000 7f7e0ae9a640 1 -- 172.21.15.49:0/138595446 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e04152110 con 0x7f7e040ce420 2023-11-06T02:10:32.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.075+0000 7f7e0ae9a640 1 -- 172.21.15.49:0/138595446 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e040c6870 con 0x7f7e040ce420 2023-11-06T02:10:32.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.075+0000 7f7e0ae9a640 1 -- 172.21.15.49:0/138595446 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e040c7a90 con 0x7f7e040ce420 2023-11-06T02:10:32.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.075+0000 7f7e017fa640 1 -- 172.21.15.49:0/138595446 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f7dfc01e3b0 con 0x7f7e040ce420 2023-11-06T02:10:32.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.075+0000 7f7e017fa640 1 --2- 172.21.15.49:0/138595446 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7df0065a60 0x7f7df0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:32.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.075+0000 7f7e017fa640 1 -- 172.21.15.49:0/138595446 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7dfc083d20 con 0x7f7e040ce420 2023-11-06T02:10:32.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.075+0000 7f7e08c36640 1 --2- 172.21.15.49:0/138595446 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7df0065a60 0x7f7df0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:32.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.075+0000 7f7e08c36640 1 --2- 172.21.15.49:0/138595446 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7df0065a60 0x7f7df0067f20 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7df4006fd0 tx=0x7f7df4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:32.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.079+0000 7f7e017fa640 1 -- 172.21.15.49:0/138595446 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f7dfc0516d0 con 0x7f7e040ce420 2023-11-06T02:10:32.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.239+0000 7f7e0ae9a640 1 -- 172.21.15.49:0/138595446 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7e040ce170 con 0x7f7e040ce420 2023-11-06T02:10:32.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.243+0000 7f7e017fa640 1 -- 172.21.15.49:0/138595446 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+17896 (secure 0 0 0) 0x7f7dfc01a090 con 0x7f7e040ce420 2023-11-06T02:10:32.241 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:10:32.242 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":50,"fsid":"1a8b035c-6ba4-499a-b876-aea47fd9c0d4","created":"2023-11-06T02:08:06.440857+0000","modified":"2023-11-06T02:10:31.903101+0000","last_up_change":"2023-11-06T02:08:17.982372+0000","last_in_change":"2023-11-06T02:08:08.906631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T02:08:19.252222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T02:08:24.776211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-06T02:10:30.458849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-06T02:10:30.899578+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ad55af8-0571-4e6a-8722-b5d429b7496d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6809","nonce":38116},{"type":"v1","addr":"172.21.15.49:6811","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6813","nonce":38116},{"type":"v1","addr":"172.21.15.49:6815","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6821","nonce":38116},{"type":"v1","addr":"172.21.15.49:6823","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6817","nonce":38116},{"type":"v1","addr":"172.21.15.49:6819","nonce":38116}]},"public_addr":"172.21.15.49:6811/38116","cluster_addr":"172.21.15.49:6815/38116","heartbeat_back_addr":"172.21.15.49:6823/38116","heartbeat_front_addr":"172.21.15.49:6819/38116","state":["exists","up"]},{"osd":1,"uuid":"c8b31f85-a757-43ed-9419-351acc46611b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":38113},{"type":"v1","addr":"172.21.15.49:6810","nonce":38113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":38113},{"type":"v1","addr":"172.21.15.49:6814","nonce":38113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":38113},{"type":"v1","addr":"172.21.15.49:6822","nonce":38113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":38113},{"type":"v1","addr":"172.21.15.49:6818","nonce":38113}]},"public_addr":"172.21.15.49:6810/38113","cluster_addr":"172.21.15.49:6814/38113","heartbeat_back_addr":"172.21.15.49:6822/38113","heartbeat_front_addr":"172.21.15.49:6818/38113","state":["exists","up"]},{"osd":2,"uuid":"555b3bf3-0a70-4b91-8efd-e4204b5fc26e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":38114},{"type":"v1","addr":"172.21.15.49:6801","nonce":38114}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":38114},{"type":"v1","addr":"172.21.15.49:6803","nonce":38114}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":38114},{"type":"v1","addr":"172.21.15.49:6807","nonce":38114}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":38114},{"type":"v1","addr":"172.21.15.49:6805","nonce":38114}]},"public_addr":"172.21.15.49:6801/38114","cluster_addr":"172.21.15.49:6803/38114","heartbeat_back_addr":"172.21.15.49:6807/38114","heartbeat_front_addr":"172.21.15.49:6805/38114","state":["exists","up"]},{"osd":3,"uuid":"8d4e5d77-03ef-4b9b-bfcb-5ae93a46795c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":38115},{"type":"v1","addr":"172.21.15.49:6825","nonce":38115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":38115},{"type":"v1","addr":"172.21.15.49:6827","nonce":38115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":38115},{"type":"v1","addr":"172.21.15.49:6831","nonce":38115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":38115},{"type":"v1","addr":"172.21.15.49:6829","nonce":38115}]},"public_addr":"172.21.15.49:6825/38115","cluster_addr":"172.21.15.49:6827/38115","heartbeat_back_addr":"172.21.15.49:6831/38115","heartbeat_front_addr":"172.21.15.49:6829/38115","state":["exists","up"]},{"osd":4,"uuid":"dba03ad4-68a3-4cf3-9218-ac611f02eacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":38417},{"type":"v1","addr":"172.21.15.150:6809","nonce":38417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":38417},{"type":"v1","addr":"172.21.15.150:6811","nonce":38417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":38417},{"type":"v1","addr":"172.21.15.150:6815","nonce":38417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":38417},{"type":"v1","addr":"172.21.15.150:6813","nonce":38417}]},"public_addr":"172.21.15.150:6809/38417","cluster_addr":"172.21.15.150:6811/38417","heartbeat_back_addr":"172.21.15.150:6815/38417","heartbeat_front_addr":"172.21.15.150:6813/38417","state":["exists","up"]},{"osd":5,"uuid":"0df9c303-c255-4640-8e91-01f1c3efe18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":38418},{"type":"v1","addr":"172.21.15.150:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":38418},{"type":"v1","addr":"172.21.15.150:6803","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":38418},{"type":"v1","addr":"172.21.15.150:6807","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":38418},{"type":"v1","addr":"172.21.15.150:6805","nonce":38418}]},"public_addr":"172.21.15.150:6801/38418","cluster_addr":"172.21.15.150:6803/38418","heartbeat_back_addr":"172.21.15.150:6807/38418","heartbeat_front_addr":"172.21.15.150:6805/38418","state":["exists","up"]},{"osd":6,"uuid":"e59fa189-4b70-4c06-9722-c02b90828bf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":38428},{"type":"v1","addr":"172.21.15.150:6825","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":38428},{"type":"v1","addr":"172.21.15.150:6827","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":38428},{"type":"v1","addr":"172.21.15.150:6831","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":38428},{"type":"v1","addr":"172.21.15.150:6829","nonce":38428}]},"public_addr":"172.21.15.150:6825/38428","cluster_addr":"172.21.15.150:6827/38428","heartbeat_back_addr":"172.21.15.150:6831/38428","heartbeat_front_addr":"172.21.15.150:6829/38428","state":["exists","up"]},{"osd":7,"uuid":"a64f57e1-ef98-46bd-b304-385759e95898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":38421},{"type":"v1","addr":"172.21.15.150:6817","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":38421},{"type":"v1","addr":"172.21.15.150:6819","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":38421},{"type":"v1","addr":"172.21.15.150:6823","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":38421},{"type":"v1","addr":"172.21.15.150:6821","nonce":38421}]},"public_addr":"172.21.15.150:6817/38421","cluster_addr":"172.21.15.150:6819/38421","heartbeat_back_addr":"172.21.15.150:6823/38421","heartbeat_front_addr":"172.21.15.150:6821/38421","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.438552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.413280+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.296466+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.424966+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.276953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.230984+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.410235+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.480084+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T02:10:32.243 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.243+0000 7f7e0ae9a640 1 -- 172.21.15.49:0/138595446 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7df0065a60 msgr2=0x7f7df0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:32.243 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.243+0000 7f7e0ae9a640 1 --2- 172.21.15.49:0/138595446 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7df0065a60 0x7f7df0067f20 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7df4006fd0 tx=0x7f7df4008040 comp rx=0 tx=0).stop 2023-11-06T02:10:32.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.243+0000 7f7e0ae9a640 1 -- 172.21.15.49:0/138595446 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e040ce420 msgr2=0x7f7e0415cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:32.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.243+0000 7f7e0ae9a640 1 --2- 172.21.15.49:0/138595446 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e040ce420 0x7f7e0415cc30 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc00cef0 tx=0x7f7dfc00a710 comp rx=0 tx=0).stop 2023-11-06T02:10:32.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.243+0000 7f7e0ae9a640 1 -- 172.21.15.49:0/138595446 shutdown_connections 2023-11-06T02:10:32.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.243+0000 7f7e0ae9a640 1 --2- 172.21.15.49:0/138595446 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7df0065a60 0x7f7df0067f20 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:32.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.243+0000 7f7e0ae9a640 1 --2- 172.21.15.49:0/138595446 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e040ce420 0x7f7e0415cc30 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:32.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.243+0000 7f7e0ae9a640 1 --2- 172.21.15.49:0/138595446 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7e040cb580 0x7f7e04151270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:32.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.243+0000 7f7e0ae9a640 1 --2- 172.21.15.49:0/138595446 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7e040b6100 0x7f7e040c6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:32.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.243+0000 7f7e0ae9a640 1 -- 172.21.15.49:0/138595446 >> 172.21.15.49:0/138595446 conn(0x7f7e040cd2d0 msgr2=0x7f7e040cb960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:32.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.243+0000 7f7e0ae9a640 1 -- 172.21.15.49:0/138595446 shutdown_connections 2023-11-06T02:10:32.246 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.243+0000 7f7e0ae9a640 1 -- 172.21.15.49:0/138595446 wait complete. 2023-11-06T02:10:32.261 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T02:10:32.379 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.379+0000 7f9870a79640 1 Processor -- start 2023-11-06T02:10:32.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.379+0000 7f9870a79640 1 -- start start 2023-11-06T02:10:32.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.379+0000 7f9870a79640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f986c071120 0x7f986c071500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:32.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.379+0000 7f9870a79640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f986c0c9c80 0x7f986c0b2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:32.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.379+0000 7f9870a79640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f986c071a40 con 0x7f986c0b5270 2023-11-06T02:10:32.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.379+0000 7f9870a79640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f986c0ae9c0 con 0x7f986c071120 2023-11-06T02:10:32.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.379+0000 7f9870a79640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f986c0af010 con 0x7f986c0c9c80 2023-11-06T02:10:32.382 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.379+0000 7f9869d9b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f986c0c9c80 0x7f986c0b2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:32.382 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.383+0000 7f986a59c640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f986c071120 0x7f986c071500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:32.382 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.383+0000 7f986ad9d640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f986c0b5270 0x7f986c0ca640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.49:36308/0 (socket says 172.21.15.49:36308) 2023-11-06T02:10:32.383 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.383+0000 7f986ad9d640 1 -- 172.21.15.49:0/1006648072 learned_addr learned my addr 172.21.15.49:0/1006648072 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:10:32.383 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.383+0000 7f9869d9b640 1 -- 172.21.15.49:0/1006648072 >> v2:172.21.15.150:3300/0 conn(0x7f986c071120 msgr2=0x7f986c071500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:32.383 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.383+0000 7f9869d9b640 1 --2- 172.21.15.49:0/1006648072 >> v2:172.21.15.150:3300/0 conn(0x7f986c071120 0x7f986c071500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:32.383 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.383+0000 7f9869d9b640 1 -- 172.21.15.49:0/1006648072 >> v1:172.21.15.150:6789/0 conn(0x7f986c0b5270 legacy=0x7f986c0ca640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:32.383 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.383+0000 7f9869d9b640 1 -- 172.21.15.49:0/1006648072 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f986c0ae6e0 con 0x7f986c0c9c80 2023-11-06T02:10:32.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.383+0000 7f986a59c640 1 --2- 172.21.15.49:0/1006648072 >> v2:172.21.15.150:3300/0 conn(0x7f986c071120 0x7f986c071500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T02:10:32.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.383+0000 7f9869d9b640 1 --2- 172.21.15.49:0/1006648072 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f986c0c9c80 0x7f986c0b2940 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f98540097c0 tx=0x7f985401ec70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62118813d1b4fcf9 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:32.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.383+0000 7f986959a640 1 -- 172.21.15.49:0/1006648072 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98540a8970 con 0x7f986c0c9c80 2023-11-06T02:10:32.389 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.383+0000 7f986959a640 1 -- 172.21.15.49:0/1006648072 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98540a8ad0 con 0x7f986c0c9c80 2023-11-06T02:10:32.389 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.383+0000 7f986959a640 1 -- 172.21.15.49:0/1006648072 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98540a8e60 con 0x7f986c0c9c80 2023-11-06T02:10:32.389 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.383+0000 7f9870a79640 1 -- 172.21.15.49:0/1006648072 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f986c0c9c80 msgr2=0x7f986c0b2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:32.389 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.383+0000 7f9870a79640 1 --2- 172.21.15.49:0/1006648072 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f986c0c9c80 0x7f986c0b2940 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f98540097c0 tx=0x7f985401ec70 comp rx=0 tx=0).stop 2023-11-06T02:10:32.389 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.383+0000 7f9870a79640 1 -- 172.21.15.49:0/1006648072 shutdown_connections 2023-11-06T02:10:32.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.383+0000 7f9870a79640 1 --2- 172.21.15.49:0/1006648072 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f986c0c9c80 0x7f986c0b2940 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:32.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.383+0000 7f9870a79640 1 --2- 172.21.15.49:0/1006648072 >> v2:172.21.15.150:3300/0 conn(0x7f986c071120 0x7f986c071500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:32.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.383+0000 7f9870a79640 1 -- 172.21.15.49:0/1006648072 >> 172.21.15.49:0/1006648072 conn(0x7f986c0cc140 msgr2=0x7f986c0cc550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:32.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.383+0000 7f9870a79640 1 -- 172.21.15.49:0/1006648072 shutdown_connections 2023-11-06T02:10:32.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.387+0000 7f9870a79640 1 -- 172.21.15.49:0/1006648072 wait complete. 2023-11-06T02:10:32.391 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.387+0000 7f9870a79640 1 Processor -- start 2023-11-06T02:10:32.391 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.387+0000 7f9870a79640 1 -- start start 2023-11-06T02:10:32.391 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.387+0000 7f9870a79640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f986c071120 0x7f986c14baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:32.391 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.387+0000 7f9870a79640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f986c0b5270 0x7f986c1593a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:32.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.387+0000 7f9870a79640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f986c0c9c80 0x7f986c1598e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:32.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.387+0000 7f9870a79640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f986c0c41b0 con 0x7f986c071120 2023-11-06T02:10:32.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.387+0000 7f9870a79640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f986c0afd00 con 0x7f986c0b5270 2023-11-06T02:10:32.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.387+0000 7f9870a79640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f986c0affa0 con 0x7f986c0c9c80 2023-11-06T02:10:32.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.387+0000 7f986a59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f986c071120 0x7f986c14baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:32.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.387+0000 7f986a59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f986c071120 0x7f986c14baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44028/0 (socket says 172.21.15.49:44028) 2023-11-06T02:10:32.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.387+0000 7f986a59c640 1 -- 172.21.15.49:0/1055126901 learned_addr learned my addr 172.21.15.49:0/1055126901 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:32.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.387+0000 7f9869d9b640 1 --2- 172.21.15.49:0/1055126901 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f986c0b5270 0x7f986c1593a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:32.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.387+0000 7f986ad9d640 1 --2- 172.21.15.49:0/1055126901 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f986c0c9c80 0x7f986c1598e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:32.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.387+0000 7f986a59c640 1 -- 172.21.15.49:0/1055126901 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f986c0c9c80 msgr2=0x7f986c1598e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:32.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.387+0000 7f986a59c640 1 --2- 172.21.15.49:0/1055126901 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f986c0c9c80 0x7f986c1598e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:32.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.387+0000 7f986a59c640 1 -- 172.21.15.49:0/1055126901 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f986c0b5270 msgr2=0x7f986c1593a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:32.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.387+0000 7f986a59c640 1 --2- 172.21.15.49:0/1055126901 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f986c0b5270 0x7f986c1593a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:32.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.387+0000 7f986a59c640 1 -- 172.21.15.49:0/1055126901 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f985c009a70 con 0x7f986c071120 2023-11-06T02:10:32.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.387+0000 7f986a59c640 1 --2- 172.21.15.49:0/1055126901 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f986c071120 0x7f986c14baa0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f985c00efc0 tx=0x7f985c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:32.395 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.387+0000 7f9862ffd640 1 -- 172.21.15.49:0/1055126901 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f985c014680 con 0x7f986c071120 2023-11-06T02:10:32.395 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.387+0000 7f9862ffd640 1 -- 172.21.15.49:0/1055126901 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f985c00f040 con 0x7f986c071120 2023-11-06T02:10:32.395 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.387+0000 7f9870a79640 1 -- 172.21.15.49:0/1055126901 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9854009420 con 0x7f986c071120 2023-11-06T02:10:32.395 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.387+0000 7f9862ffd640 1 -- 172.21.15.49:0/1055126901 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f985c014a00 con 0x7f986c071120 2023-11-06T02:10:32.395 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.387+0000 7f9870a79640 1 -- 172.21.15.49:0/1055126901 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f986c0b0560 con 0x7f986c071120 2023-11-06T02:10:32.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.391+0000 7f9870a79640 1 -- 172.21.15.49:0/1055126901 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f986c067140 con 0x7f986c071120 2023-11-06T02:10:32.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.391+0000 7f9862ffd640 1 -- 172.21.15.49:0/1055126901 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f985c014b60 con 0x7f986c071120 2023-11-06T02:10:32.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.391+0000 7f9862ffd640 1 --2- 172.21.15.49:0/1055126901 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f984c065a60 0x7f984c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:32.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.391+0000 7f9862ffd640 1 -- 172.21.15.49:0/1055126901 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f985c0842f0 con 0x7f986c071120 2023-11-06T02:10:32.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.391+0000 7f9869d9b640 1 --2- 172.21.15.49:0/1055126901 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f984c065a60 0x7f984c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:32.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.391+0000 7f9869d9b640 1 --2- 172.21.15.49:0/1055126901 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f984c065a60 0x7f984c067f20 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f98540097c0 tx=0x7f98540aa070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:32.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.395+0000 7f9862ffd640 1 -- 172.21.15.49:0/1055126901 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f985c051b50 con 0x7f986c071120 2023-11-06T02:10:32.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.555+0000 7f9870a79640 1 -- 172.21.15.49:0/1055126901 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f986c14bfe0 con 0x7f986c071120 2023-11-06T02:10:32.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.555+0000 7f9862ffd640 1 -- 172.21.15.49:0/1055126901 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+17896 (secure 0 0 0) 0x7f985c055170 con 0x7f986c071120 2023-11-06T02:10:32.557 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:10:32.557 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":50,"fsid":"1a8b035c-6ba4-499a-b876-aea47fd9c0d4","created":"2023-11-06T02:08:06.440857+0000","modified":"2023-11-06T02:10:31.903101+0000","last_up_change":"2023-11-06T02:08:17.982372+0000","last_in_change":"2023-11-06T02:08:08.906631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T02:08:19.252222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T02:08:24.776211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-06T02:10:30.458849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-06T02:10:30.899578+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ad55af8-0571-4e6a-8722-b5d429b7496d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6809","nonce":38116},{"type":"v1","addr":"172.21.15.49:6811","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6813","nonce":38116},{"type":"v1","addr":"172.21.15.49:6815","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6821","nonce":38116},{"type":"v1","addr":"172.21.15.49:6823","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6817","nonce":38116},{"type":"v1","addr":"172.21.15.49:6819","nonce":38116}]},"public_addr":"172.21.15.49:6811/38116","cluster_addr":"172.21.15.49:6815/38116","heartbeat_back_addr":"172.21.15.49:6823/38116","heartbeat_front_addr":"172.21.15.49:6819/38116","state":["exists","up"]},{"osd":1,"uuid":"c8b31f85-a757-43ed-9419-351acc46611b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":38113},{"type":"v1","addr":"172.21.15.49:6810","nonce":38113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":38113},{"type":"v1","addr":"172.21.15.49:6814","nonce":38113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":38113},{"type":"v1","addr":"172.21.15.49:6822","nonce":38113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":38113},{"type":"v1","addr":"172.21.15.49:6818","nonce":38113}]},"public_addr":"172.21.15.49:6810/38113","cluster_addr":"172.21.15.49:6814/38113","heartbeat_back_addr":"172.21.15.49:6822/38113","heartbeat_front_addr":"172.21.15.49:6818/38113","state":["exists","up"]},{"osd":2,"uuid":"555b3bf3-0a70-4b91-8efd-e4204b5fc26e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":38114},{"type":"v1","addr":"172.21.15.49:6801","nonce":38114}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":38114},{"type":"v1","addr":"172.21.15.49:6803","nonce":38114}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":38114},{"type":"v1","addr":"172.21.15.49:6807","nonce":38114}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":38114},{"type":"v1","addr":"172.21.15.49:6805","nonce":38114}]},"public_addr":"172.21.15.49:6801/38114","cluster_addr":"172.21.15.49:6803/38114","heartbeat_back_addr":"172.21.15.49:6807/38114","heartbeat_front_addr":"172.21.15.49:6805/38114","state":["exists","up"]},{"osd":3,"uuid":"8d4e5d77-03ef-4b9b-bfcb-5ae93a46795c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":38115},{"type":"v1","addr":"172.21.15.49:6825","nonce":38115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":38115},{"type":"v1","addr":"172.21.15.49:6827","nonce":38115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":38115},{"type":"v1","addr":"172.21.15.49:6831","nonce":38115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":38115},{"type":"v1","addr":"172.21.15.49:6829","nonce":38115}]},"public_addr":"172.21.15.49:6825/38115","cluster_addr":"172.21.15.49:6827/38115","heartbeat_back_addr":"172.21.15.49:6831/38115","heartbeat_front_addr":"172.21.15.49:6829/38115","state":["exists","up"]},{"osd":4,"uuid":"dba03ad4-68a3-4cf3-9218-ac611f02eacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":38417},{"type":"v1","addr":"172.21.15.150:6809","nonce":38417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":38417},{"type":"v1","addr":"172.21.15.150:6811","nonce":38417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":38417},{"type":"v1","addr":"172.21.15.150:6815","nonce":38417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":38417},{"type":"v1","addr":"172.21.15.150:6813","nonce":38417}]},"public_addr":"172.21.15.150:6809/38417","cluster_addr":"172.21.15.150:6811/38417","heartbeat_back_addr":"172.21.15.150:6815/38417","heartbeat_front_addr":"172.21.15.150:6813/38417","state":["exists","up"]},{"osd":5,"uuid":"0df9c303-c255-4640-8e91-01f1c3efe18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":38418},{"type":"v1","addr":"172.21.15.150:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":38418},{"type":"v1","addr":"172.21.15.150:6803","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":38418},{"type":"v1","addr":"172.21.15.150:6807","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":38418},{"type":"v1","addr":"172.21.15.150:6805","nonce":38418}]},"public_addr":"172.21.15.150:6801/38418","cluster_addr":"172.21.15.150:6803/38418","heartbeat_back_addr":"172.21.15.150:6807/38418","heartbeat_front_addr":"172.21.15.150:6805/38418","state":["exists","up"]},{"osd":6,"uuid":"e59fa189-4b70-4c06-9722-c02b90828bf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":38428},{"type":"v1","addr":"172.21.15.150:6825","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":38428},{"type":"v1","addr":"172.21.15.150:6827","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":38428},{"type":"v1","addr":"172.21.15.150:6831","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":38428},{"type":"v1","addr":"172.21.15.150:6829","nonce":38428}]},"public_addr":"172.21.15.150:6825/38428","cluster_addr":"172.21.15.150:6827/38428","heartbeat_back_addr":"172.21.15.150:6831/38428","heartbeat_front_addr":"172.21.15.150:6829/38428","state":["exists","up"]},{"osd":7,"uuid":"a64f57e1-ef98-46bd-b304-385759e95898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":38421},{"type":"v1","addr":"172.21.15.150:6817","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":38421},{"type":"v1","addr":"172.21.15.150:6819","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":38421},{"type":"v1","addr":"172.21.15.150:6823","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":38421},{"type":"v1","addr":"172.21.15.150:6821","nonce":38421}]},"public_addr":"172.21.15.150:6817/38421","cluster_addr":"172.21.15.150:6819/38421","heartbeat_back_addr":"172.21.15.150:6823/38421","heartbeat_front_addr":"172.21.15.150:6821/38421","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.438552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.413280+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.296466+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.424966+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.276953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.230984+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.410235+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.480084+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T02:10:32.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.559+0000 7f9870a79640 1 -- 172.21.15.49:0/1055126901 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f984c065a60 msgr2=0x7f984c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:32.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.559+0000 7f9870a79640 1 --2- 172.21.15.49:0/1055126901 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f984c065a60 0x7f984c067f20 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f98540097c0 tx=0x7f98540aa070 comp rx=0 tx=0).stop 2023-11-06T02:10:32.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.559+0000 7f9870a79640 1 -- 172.21.15.49:0/1055126901 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f986c071120 msgr2=0x7f986c14baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:32.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.559+0000 7f9870a79640 1 --2- 172.21.15.49:0/1055126901 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f986c071120 0x7f986c14baa0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f985c00efc0 tx=0x7f985c00c490 comp rx=0 tx=0).stop 2023-11-06T02:10:32.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.559+0000 7f9870a79640 1 -- 172.21.15.49:0/1055126901 shutdown_connections 2023-11-06T02:10:32.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.559+0000 7f9870a79640 1 --2- 172.21.15.49:0/1055126901 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f984c065a60 0x7f984c067f20 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:32.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.559+0000 7f9870a79640 1 --2- 172.21.15.49:0/1055126901 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f986c0c9c80 0x7f986c1598e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:32.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.559+0000 7f9870a79640 1 --2- 172.21.15.49:0/1055126901 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f986c0b5270 0x7f986c1593a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:32.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.559+0000 7f9870a79640 1 --2- 172.21.15.49:0/1055126901 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f986c071120 0x7f986c14baa0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:32.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.559+0000 7f9870a79640 1 -- 172.21.15.49:0/1055126901 >> 172.21.15.49:0/1055126901 conn(0x7f986c0cc140 msgr2=0x7f986c0c2930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:32.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.559+0000 7f9870a79640 1 -- 172.21.15.49:0/1055126901 shutdown_connections 2023-11-06T02:10:32.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.559+0000 7f9870a79640 1 -- 172.21.15.49:0/1055126901 wait complete. 2023-11-06T02:10:32.576 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-06T02:10:32.706 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.707+0000 7f301f2e7640 1 Processor -- start 2023-11-06T02:10:32.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.707+0000 7f301f2e7640 1 -- start start 2023-11-06T02:10:32.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.707+0000 7f301f2e7640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f301806bda0 0x7f30180d08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:32.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.707+0000 7f301f2e7640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f30180d0e10 0x7f30180b0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:32.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.707+0000 7f301f2e7640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30180cf100 con 0x7f30180afe20 2023-11-06T02:10:32.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.707+0000 7f301f2e7640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30180d1220 con 0x7f30180d0e10 2023-11-06T02:10:32.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.707+0000 7f301f2e7640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30180cf340 con 0x7f301806bda0 2023-11-06T02:10:32.709 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.707+0000 7f301d884640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f30180d0e10 0x7f30180b0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:32.709 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.707+0000 7f301d884640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f30180d0e10 0x7f30180b0a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44036/0 (socket says 172.21.15.49:44036) 2023-11-06T02:10:32.709 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.707+0000 7f301d884640 1 -- 172.21.15.49:0/3906290543 learned_addr learned my addr 172.21.15.49:0/3906290543 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:32.709 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.707+0000 7f301c882640 1 --2- 172.21.15.49:0/3906290543 >> v2:172.21.15.150:3300/0 conn(0x7f301806bda0 0x7f30180d08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:32.709 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.707+0000 7f301d884640 1 -- 172.21.15.49:0/3906290543 >> v2:172.21.15.150:3300/0 conn(0x7f301806bda0 msgr2=0x7f30180d08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:32.710 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.707+0000 7f301d884640 1 --2- 172.21.15.49:0/3906290543 >> v2:172.21.15.150:3300/0 conn(0x7f301806bda0 0x7f30180d08d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:32.710 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.707+0000 7f301d884640 1 -- 172.21.15.49:0/3906290543 >> v1:172.21.15.150:6789/0 conn(0x7f30180afe20 legacy=0x7f30180ce9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:32.710 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.707+0000 7f301d884640 1 -- 172.21.15.49:0/3906290543 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30180cf620 con 0x7f30180d0e10 2023-11-06T02:10:32.711 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.711+0000 7f301d884640 1 --2- 172.21.15.49:0/3906290543 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30180d0e10 0x7f30180b0a80 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f30180d0650 tx=0x7f301401c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a87cb7fe8641256 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:32.712 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.711+0000 7f300ffff640 1 -- 172.21.15.49:0/3906290543 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30140ab600 con 0x7f30180d0e10 2023-11-06T02:10:32.712 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.711+0000 7f300ffff640 1 -- 172.21.15.49:0/3906290543 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30140ab760 con 0x7f30180d0e10 2023-11-06T02:10:32.712 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.711+0000 7f300ffff640 1 -- 172.21.15.49:0/3906290543 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30140abaf0 con 0x7f30180d0e10 2023-11-06T02:10:32.712 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.711+0000 7f301f2e7640 1 -- 172.21.15.49:0/3906290543 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30180d0e10 msgr2=0x7f30180b0a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:32.712 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.711+0000 7f301f2e7640 1 --2- 172.21.15.49:0/3906290543 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30180d0e10 0x7f30180b0a80 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f30180d0650 tx=0x7f301401c400 comp rx=0 tx=0).stop 2023-11-06T02:10:32.713 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.711+0000 7f301f2e7640 1 -- 172.21.15.49:0/3906290543 shutdown_connections 2023-11-06T02:10:32.713 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.711+0000 7f301f2e7640 1 --2- 172.21.15.49:0/3906290543 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30180d0e10 0x7f30180b0a80 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:32.713 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.711+0000 7f301f2e7640 1 --2- 172.21.15.49:0/3906290543 >> v2:172.21.15.150:3300/0 conn(0x7f301806bda0 0x7f30180d08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:32.713 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.711+0000 7f301f2e7640 1 -- 172.21.15.49:0/3906290543 >> 172.21.15.49:0/3906290543 conn(0x7f3018089fd0 msgr2=0x7f3018099c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:32.713 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.711+0000 7f301f2e7640 1 -- 172.21.15.49:0/3906290543 shutdown_connections 2023-11-06T02:10:32.714 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.711+0000 7f301f2e7640 1 -- 172.21.15.49:0/3906290543 wait complete. 2023-11-06T02:10:32.716 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.711+0000 7f301f2e7640 1 Processor -- start 2023-11-06T02:10:32.716 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.711+0000 7f301f2e7640 1 -- start start 2023-11-06T02:10:32.716 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.715+0000 7f301f2e7640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f301806bda0 0x7f3018153230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:32.716 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.715+0000 7f301f2e7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30180afe20 0x7f30180b3580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:32.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.715+0000 7f301f2e7640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f30180d0e10 0x7f30180c7d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:32.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.715+0000 7f301f2e7640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30180c87c0 con 0x7f30180afe20 2023-11-06T02:10:32.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.715+0000 7f301f2e7640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3018156df0 con 0x7f30180d0e10 2023-11-06T02:10:32.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.715+0000 7f301f2e7640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30181570d0 con 0x7f301806bda0 2023-11-06T02:10:32.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.715+0000 7f301c882640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30180afe20 0x7f30180b3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:32.718 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.715+0000 7f301c882640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30180afe20 0x7f30180b3580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44052/0 (socket says 172.21.15.49:44052) 2023-11-06T02:10:32.718 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.715+0000 7f301c882640 1 -- 172.21.15.49:0/1568061579 learned_addr learned my addr 172.21.15.49:0/1568061579 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:32.718 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.715+0000 7f301c882640 1 -- 172.21.15.49:0/1568061579 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f301806bda0 msgr2=0x7f3018153230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:32.718 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.715+0000 7f301d083640 1 --2- 172.21.15.49:0/1568061579 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f301806bda0 0x7f3018153230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:32.718 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.715+0000 7f301c882640 1 --2- 172.21.15.49:0/1568061579 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f301806bda0 0x7f3018153230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:32.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.715+0000 7f301c882640 1 -- 172.21.15.49:0/1568061579 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f30180d0e10 msgr2=0x7f30180c7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:32.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.715+0000 7f301c882640 1 --2- 172.21.15.49:0/1568061579 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f30180d0e10 0x7f30180c7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:32.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.715+0000 7f301c882640 1 -- 172.21.15.49:0/1568061579 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30140094d0 con 0x7f30180afe20 2023-11-06T02:10:32.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.715+0000 7f301c882640 1 --2- 172.21.15.49:0/1568061579 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30180afe20 0x7f30180b3580 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f300800c7f0 tx=0x7f300800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:32.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.715+0000 7f300dffb640 1 -- 172.21.15.49:0/1568061579 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3008004220 con 0x7f30180afe20 2023-11-06T02:10:32.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.715+0000 7f300dffb640 1 -- 172.21.15.49:0/1568061579 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30080043a0 con 0x7f30180afe20 2023-11-06T02:10:32.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.715+0000 7f300dffb640 1 -- 172.21.15.49:0/1568061579 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3008004510 con 0x7f30180afe20 2023-11-06T02:10:32.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.715+0000 7f301f2e7640 1 -- 172.21.15.49:0/1568061579 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30181573b0 con 0x7f30180afe20 2023-11-06T02:10:32.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.715+0000 7f301f2e7640 1 -- 172.21.15.49:0/1568061579 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30180b3ac0 con 0x7f30180afe20 2023-11-06T02:10:32.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.715+0000 7f301f2e7640 1 -- 172.21.15.49:0/1568061579 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30181578d0 con 0x7f30180afe20 2023-11-06T02:10:32.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.715+0000 7f300dffb640 1 -- 172.21.15.49:0/1568061579 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f3008003a70 con 0x7f30180afe20 2023-11-06T02:10:32.721 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.715+0000 7f300dffb640 1 --2- 172.21.15.49:0/1568061579 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2ffc065a60 0x7f2ffc067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:32.721 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.715+0000 7f300dffb640 1 -- 172.21.15.49:0/1568061579 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f30080832f0 con 0x7f30180afe20 2023-11-06T02:10:32.721 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.715+0000 7f301d083640 1 --2- 172.21.15.49:0/1568061579 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2ffc065a60 0x7f2ffc067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:32.722 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.715+0000 7f301d083640 1 --2- 172.21.15.49:0/1568061579 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2ffc065a60 0x7f2ffc067f20 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f3010004450 tx=0x7f3010009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:32.722 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.719+0000 7f300dffb640 1 -- 172.21.15.49:0/1568061579 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f30181578d0 con 0x7f30180afe20 2023-11-06T02:10:32.912 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:32.911+0000 7f301f2e7640 1 -- 172.21.15.49:0/1568061579 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f30181578d0 con 0x7f30180afe20 2023-11-06T02:10:33.344 INFO:tasks.ceph.mds.a.smithi049.stderr:2023-11-06T02:10:33.347+0000 7feb1deab640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-06T02:10:33.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:33.923+0000 7f300dffb640 1 -- 172.21.15.49:0/1568061579 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v22) v1 ==== 118+0+0 (secure 0 0 0) 0x7f30181578d0 con 0x7f30180afe20 2023-11-06T02:10:33.924 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:33.923+0000 7f301f2e7640 1 -- 172.21.15.49:0/1568061579 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2ffc065a60 msgr2=0x7f2ffc067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:33.924 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:33.923+0000 7f301f2e7640 1 --2- 172.21.15.49:0/1568061579 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2ffc065a60 0x7f2ffc067f20 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f3010004450 tx=0x7f3010009040 comp rx=0 tx=0).stop 2023-11-06T02:10:33.925 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:33.923+0000 7f301f2e7640 1 -- 172.21.15.49:0/1568061579 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30180afe20 msgr2=0x7f30180b3580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:33.925 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:33.923+0000 7f301f2e7640 1 --2- 172.21.15.49:0/1568061579 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30180afe20 0x7f30180b3580 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f300800c7f0 tx=0x7f300800ccb0 comp rx=0 tx=0).stop 2023-11-06T02:10:33.925 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:33.923+0000 7f301f2e7640 1 -- 172.21.15.49:0/1568061579 shutdown_connections 2023-11-06T02:10:33.925 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:33.923+0000 7f301f2e7640 1 --2- 172.21.15.49:0/1568061579 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2ffc065a60 0x7f2ffc067f20 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:33.925 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:33.923+0000 7f301f2e7640 1 --2- 172.21.15.49:0/1568061579 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30180afe20 0x7f30180b3580 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:33.926 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:33.923+0000 7f301f2e7640 1 --2- 172.21.15.49:0/1568061579 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f30180d0e10 0x7f30180c7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:33.926 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:33.923+0000 7f301f2e7640 1 --2- 172.21.15.49:0/1568061579 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f301806bda0 0x7f3018153230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:33.926 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:33.923+0000 7f301f2e7640 1 -- 172.21.15.49:0/1568061579 >> 172.21.15.49:0/1568061579 conn(0x7f3018089fd0 msgr2=0x7f301806fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:33.926 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:33.923+0000 7f301f2e7640 1 -- 172.21.15.49:0/1568061579 shutdown_connections 2023-11-06T02:10:33.926 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:33.923+0000 7f301f2e7640 1 -- 172.21.15.49:0/1568061579 wait complete. 2023-11-06T02:10:33.945 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-06T02:10:33.945 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-06T02:10:34.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:34.079+0000 7f1b71686640 1 Processor -- start 2023-11-06T02:10:34.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:34.079+0000 7f1b71686640 1 -- start start 2023-11-06T02:10:34.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:34.079+0000 7f1b71686640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f1b6c0d2a60 0x7f1b6c0d2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:34.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:34.079+0000 7f1b71686640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1b6c0b5d40 0x7f1b6c0b61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:34.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:34.079+0000 7f1b71686640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b6c15cf90 con 0x7f1b6c15cbb0 2023-11-06T02:10:34.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:34.079+0000 7f1b71686640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b6c0d3410 con 0x7f1b6c0d2a60 2023-11-06T02:10:34.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:34.079+0000 7f1b71686640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6c0d0b80 con 0x7f1b6c0b5d40 2023-11-06T02:10:34.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:34.083+0000 7f1b6b7fe640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1b6c0b5d40 0x7f1b6c0b61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:34.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:34.083+0000 7f1b6a7fc640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f1b6c0d2a60 0x7f1b6c0d2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:34.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:34.083+0000 7f1b6b7fe640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1b6c0b5d40 0x7f1b6c0b61d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.49:41024/0 (socket says 172.21.15.49:41024) 2023-11-06T02:10:34.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:34.083+0000 7f1b6a7fc640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f1b6c0d2a60 0x7f1b6c0d2ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.49:52574/0 (socket says 172.21.15.49:52574) 2023-11-06T02:10:34.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:34.083+0000 7f1b6b7fe640 1 -- 172.21.15.49:0/2237555875 learned_addr learned my addr 172.21.15.49:0/2237555875 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:34.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:34.083+0000 7f1b6b7fe640 1 -- 172.21.15.49:0/2237555875 >> v2:172.21.15.150:3300/0 conn(0x7f1b6c0d2a60 msgr2=0x7f1b6c0d2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:34.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:34.083+0000 7f1b6b7fe640 1 --2- 172.21.15.49:0/2237555875 >> v2:172.21.15.150:3300/0 conn(0x7f1b6c0d2a60 0x7f1b6c0d2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:34.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:34.083+0000 7f1b6b7fe640 1 -- 172.21.15.49:0/2237555875 >> v1:172.21.15.150:6789/0 conn(0x7f1b6c15cbb0 legacy=0x7f1b6c0d1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:34.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:34.083+0000 7f1b6b7fe640 1 -- 172.21.15.49:0/2237555875 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b6c0d0e60 con 0x7f1b6c0b5d40 2023-11-06T02:10:34.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:34.083+0000 7f1b6a7fc640 1 --2- 172.21.15.49:0/2237555875 >> v2:172.21.15.150:3300/0 conn(0x7f1b6c0d2a60 0x7f1b6c0d2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T02:10:34.085 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:34.083+0000 7f1b6b7fe640 1 --2- 172.21.15.49:0/2237555875 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1b6c0b5d40 0x7f1b6c0b61d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1b64009e50 tx=0x7f1b6401f1f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9727c3e90afa729 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:34.085 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:34.083+0000 7f1b69ffb640 1 -- 172.21.15.49:0/2237555875 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b640af400 con 0x7f1b6c0b5d40 2023-11-06T02:10:34.086 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:34.083+0000 7f1b69ffb640 1 -- 172.21.15.49:0/2237555875 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b640ab050 con 0x7f1b6c0b5d40 2023-11-06T02:10:34.086 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:34.083+0000 7f1b69ffb640 1 -- 172.21.15.49:0/2237555875 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b640af780 con 0x7f1b6c0b5d40 2023-11-06T02:10:34.086 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:34.087+0000 7f1b71686640 1 -- 172.21.15.49:0/2237555875 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1b6c0b5d40 msgr2=0x7f1b6c0b61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:34.086 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:34.087+0000 7f1b71686640 1 --2- 172.21.15.49:0/2237555875 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1b6c0b5d40 0x7f1b6c0b61d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1b64009e50 tx=0x7f1b6401f1f0 comp rx=0 tx=0).stop 2023-11-06T02:10:34.086 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:34.087+0000 7f1b71686640 1 -- 172.21.15.49:0/2237555875 shutdown_connections 2023-11-06T02:10:34.087 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:34.087+0000 7f1b71686640 1 --2- 172.21.15.49:0/2237555875 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1b6c0b5d40 0x7f1b6c0b61d0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:34.087 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:34.087+0000 7f1b71686640 1 --2- 172.21.15.49:0/2237555875 >> v2:172.21.15.150:3300/0 conn(0x7f1b6c0d2a60 0x7f1b6c0d2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:34.087 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:34.087+0000 7f1b71686640 1 -- 172.21.15.49:0/2237555875 >> 172.21.15.49:0/2237555875 conn(0x7f1b6c0718a0 msgr2=0x7f1b6c071cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:34.087 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:34.087+0000 7f1b71686640 1 -- 172.21.15.49:0/2237555875 shutdown_connections 2023-11-06T02:10:34.087 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:34.087+0000 7f1b71686640 1 -- 172.21.15.49:0/2237555875 wait complete. 2023-11-06T02:10:34.090 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:34.087+0000 7f1b71686640 1 Processor -- start 2023-11-06T02:10:34.090 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:34.087+0000 7f1b71686640 1 -- start start 2023-11-06T02:10:34.090 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:34.087+0000 7f1b71686640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b6c0b5d40 0x7f1b6c0b5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:34.090 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:34.087+0000 7f1b71686640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b6c0d2a60 0x7f1b6c14fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:34.091 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:34.087+0000 7f1b71686640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1b6c15cbb0 0x7f1b6c151b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:34.091 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:34.087+0000 7f1b71686640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6c156150 con 0x7f1b6c0b5d40 2023-11-06T02:10:34.091 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:34.087+0000 7f1b71686640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6c1520a0 con 0x7f1b6c0d2a60 2023-11-06T02:10:34.091 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:34.087+0000 7f1b71686640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6c155e70 con 0x7f1b6c15cbb0 2023-11-06T02:10:34.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:34.087+0000 7f1b6affd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b6c0b5d40 0x7f1b6c0b5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:34.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:34.087+0000 7f1b6affd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b6c0b5d40 0x7f1b6c0b5070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44062/0 (socket says 172.21.15.49:44062) 2023-11-06T02:10:34.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:34.087+0000 7f1b6affd640 1 -- 172.21.15.49:0/2893849860 learned_addr learned my addr 172.21.15.49:0/2893849860 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:34.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:34.087+0000 7f1b6a7fc640 1 --2- 172.21.15.49:0/2893849860 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b6c0d2a60 0x7f1b6c14fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:34.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:34.087+0000 7f1b6b7fe640 1 --2- 172.21.15.49:0/2893849860 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1b6c15cbb0 0x7f1b6c151b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:34.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:34.087+0000 7f1b6affd640 1 -- 172.21.15.49:0/2893849860 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1b6c15cbb0 msgr2=0x7f1b6c151b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:34.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:34.087+0000 7f1b6affd640 1 --2- 172.21.15.49:0/2893849860 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1b6c15cbb0 0x7f1b6c151b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:34.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:34.087+0000 7f1b6affd640 1 -- 172.21.15.49:0/2893849860 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b6c0d2a60 msgr2=0x7f1b6c14fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:34.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:34.087+0000 7f1b6affd640 1 --2- 172.21.15.49:0/2893849860 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b6c0d2a60 0x7f1b6c14fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:34.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:34.087+0000 7f1b6affd640 1 -- 172.21.15.49:0/2893849860 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b60009420 con 0x7f1b6c0b5d40 2023-11-06T02:10:34.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:34.087+0000 7f1b6affd640 1 --2- 172.21.15.49:0/2893849860 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b6c0b5d40 0x7f1b6c0b5070 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c007fd0 tx=0x7f1b5c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:34.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:34.087+0000 7f1b537fe640 1 -- 172.21.15.49:0/2893849860 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b5c0052d0 con 0x7f1b6c0b5d40 2023-11-06T02:10:34.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:34.087+0000 7f1b537fe640 1 -- 172.21.15.49:0/2893849860 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b5c005430 con 0x7f1b6c0b5d40 2023-11-06T02:10:34.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:34.087+0000 7f1b71686640 1 -- 172.21.15.49:0/2893849860 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b64009b30 con 0x7f1b6c0b5d40 2023-11-06T02:10:34.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:34.087+0000 7f1b537fe640 1 -- 172.21.15.49:0/2893849860 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b5c003840 con 0x7f1b6c0b5d40 2023-11-06T02:10:34.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:34.087+0000 7f1b71686640 1 -- 172.21.15.49:0/2893849860 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b6c0ca120 con 0x7f1b6c0b5d40 2023-11-06T02:10:34.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:34.091+0000 7f1b71686640 1 -- 172.21.15.49:0/2893849860 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b6c0c9ca0 con 0x7f1b6c0b5d40 2023-11-06T02:10:34.096 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:34.091+0000 7f1b537fe640 1 -- 172.21.15.49:0/2893849860 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f1b5c0039a0 con 0x7f1b6c0b5d40 2023-11-06T02:10:34.096 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:34.091+0000 7f1b537fe640 1 --2- 172.21.15.49:0/2893849860 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1b58065a60 0x7f1b58067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:34.096 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:34.091+0000 7f1b537fe640 1 -- 172.21.15.49:0/2893849860 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1b5c082870 con 0x7f1b6c0b5d40 2023-11-06T02:10:34.096 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:34.091+0000 7f1b6a7fc640 1 --2- 172.21.15.49:0/2893849860 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1b58065a60 0x7f1b58067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:34.097 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:34.095+0000 7f1b6a7fc640 1 --2- 172.21.15.49:0/2893849860 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1b58065a60 0x7f1b58067f20 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c15ad70 tx=0x7f1b60009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:34.097 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:34.095+0000 7f1b537fe640 1 -- 172.21.15.49:0/2893849860 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f1b6c0c9ca0 con 0x7f1b6c0b5d40 2023-11-06T02:10:34.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:34.287+0000 7f1b71686640 1 -- 172.21.15.49:0/2893849860 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f1b6c0c9ca0 con 0x7f1b6c0b5d40 2023-11-06T02:10:34.932 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:34.931+0000 7f1b537fe640 1 -- 172.21.15.49:0/2893849860 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v23) v1 ==== 122+0+0 (secure 0 0 0) 0x7f1b6c0c9ca0 con 0x7f1b6c0b5d40 2023-11-06T02:10:34.935 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:34.935+0000 7f1b71686640 1 -- 172.21.15.49:0/2893849860 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1b58065a60 msgr2=0x7f1b58067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:34.936 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:34.935+0000 7f1b71686640 1 --2- 172.21.15.49:0/2893849860 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1b58065a60 0x7f1b58067f20 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c15ad70 tx=0x7f1b60009190 comp rx=0 tx=0).stop 2023-11-06T02:10:34.936 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:34.935+0000 7f1b71686640 1 -- 172.21.15.49:0/2893849860 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b6c0b5d40 msgr2=0x7f1b6c0b5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:34.936 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:34.935+0000 7f1b71686640 1 --2- 172.21.15.49:0/2893849860 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b6c0b5d40 0x7f1b6c0b5070 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c007fd0 tx=0x7f1b5c00c890 comp rx=0 tx=0).stop 2023-11-06T02:10:34.936 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:34.935+0000 7f1b71686640 1 -- 172.21.15.49:0/2893849860 shutdown_connections 2023-11-06T02:10:34.936 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:34.935+0000 7f1b71686640 1 --2- 172.21.15.49:0/2893849860 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1b58065a60 0x7f1b58067f20 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:34.937 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:34.935+0000 7f1b71686640 1 --2- 172.21.15.49:0/2893849860 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1b6c15cbb0 0x7f1b6c151b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:34.937 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:34.935+0000 7f1b71686640 1 --2- 172.21.15.49:0/2893849860 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b6c0d2a60 0x7f1b6c14fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:34.937 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:34.935+0000 7f1b71686640 1 --2- 172.21.15.49:0/2893849860 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b6c0b5d40 0x7f1b6c0b5070 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:34.937 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:34.935+0000 7f1b71686640 1 -- 172.21.15.49:0/2893849860 >> 172.21.15.49:0/2893849860 conn(0x7f1b6c0718a0 msgr2=0x7f1b6c0cbc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:34.937 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:34.935+0000 7f1b71686640 1 -- 172.21.15.49:0/2893849860 shutdown_connections 2023-11-06T02:10:34.938 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:34.935+0000 7f1b71686640 1 -- 172.21.15.49:0/2893849860 wait complete. 2023-11-06T02:10:34.958 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-06T02:10:34.959 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T02:10:35.085 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.087+0000 7f9757d4b640 1 Processor -- start 2023-11-06T02:10:35.086 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.087+0000 7f9757d4b640 1 -- start start 2023-11-06T02:10:35.087 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.087+0000 7f9757d4b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f975015ca60 0x7f97500b0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:35.087 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.087+0000 7f9757d4b640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f97500c9ac0 0x7f97500b0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:35.087 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.087+0000 7f9757d4b640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9750161090 con 0x7f97500cdee0 2023-11-06T02:10:35.088 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.087+0000 7f9757d4b640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97501607a0 con 0x7f97500c9ac0 2023-11-06T02:10:35.088 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.087+0000 7f9757d4b640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97500cd680 con 0x7f975015ca60 2023-11-06T02:10:35.088 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.087+0000 7f97562e8640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f97500cdee0 0x7f97500cb0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:41544/0 (socket says 172.21.15.49:41544) 2023-11-06T02:10:35.088 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.087+0000 7f97562e8640 1 -- 172.21.15.49:0/3160886875 learned_addr learned my addr 172.21.15.49:0/3160886875 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:10:35.088 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.087+0000 7f97552e6640 1 --2- 172.21.15.49:0/3160886875 >> v2:172.21.15.150:3300/0 conn(0x7f97500c9ac0 0x7f97500b0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:35.088 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.087+0000 7f9755ae7640 1 --2- 172.21.15.49:0/3160886875 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f975015ca60 0x7f97500b0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:35.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.087+0000 7f9754ae5640 1 -- 172.21.15.49:0/3160886875 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 258968060 0 0) 0x7f9750161090 con 0x7f97500cdee0 2023-11-06T02:10:35.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.087+0000 7f9754ae5640 1 -- 172.21.15.49:0/3160886875 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f973c003530 con 0x7f97500cdee0 2023-11-06T02:10:35.091 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.087+0000 7f9754ae5640 1 -- 172.21.15.49:0/3160886875 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 605673933 0 0) 0x7f973c003530 con 0x7f97500cdee0 2023-11-06T02:10:35.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.087+0000 7f9754ae5640 1 -- 172.21.15.49:0/3160886875 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f975015ca60 msgr2=0x7f97500b0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:35.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.087+0000 7f9754ae5640 1 --2- 172.21.15.49:0/3160886875 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f975015ca60 0x7f97500b0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:35.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.087+0000 7f9754ae5640 1 -- 172.21.15.49:0/3160886875 >> v2:172.21.15.150:3300/0 conn(0x7f97500c9ac0 msgr2=0x7f97500b0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:35.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.087+0000 7f9754ae5640 1 --2- 172.21.15.49:0/3160886875 >> v2:172.21.15.150:3300/0 conn(0x7f97500c9ac0 0x7f97500b0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:35.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.087+0000 7f9754ae5640 1 -- 172.21.15.49:0/3160886875 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9750160a40 con 0x7f97500cdee0 2023-11-06T02:10:35.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.087+0000 7f9754ae5640 1 -- 172.21.15.49:0/3160886875 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7f9748001680 con 0x7f97500cdee0 2023-11-06T02:10:35.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.087+0000 7f9754ae5640 1 -- 172.21.15.49:0/3160886875 >> v1:172.21.15.49:6789/0 conn(0x7f97500cdee0 legacy=0x7f97500cb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:35.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.087+0000 7f9754ae5640 1 --2- 172.21.15.49:0/3160886875 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f973c003980 0x7f973c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:35.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.087+0000 7f9754ae5640 1 --2- 172.21.15.49:0/3160886875 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f973c0047d0 0x7f973c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:35.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.087+0000 7f9754ae5640 1 --2- 172.21.15.49:0/3160886875 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f975015ca60 0x7f973c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:35.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.087+0000 7f9754ae5640 1 -- 172.21.15.49:0/3160886875 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97501607a0 con 0x7f973c003980 2023-11-06T02:10:35.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.087+0000 7f9754ae5640 1 -- 172.21.15.49:0/3160886875 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97500cd680 con 0x7f973c0047d0 2023-11-06T02:10:35.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.087+0000 7f9754ae5640 1 -- 172.21.15.49:0/3160886875 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f973c00b990 con 0x7f975015ca60 2023-11-06T02:10:35.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.087+0000 7f9755ae7640 1 --2- 172.21.15.49:0/3160886875 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f973c003980 0x7f973c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:35.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.087+0000 7f97562e8640 1 --2- 172.21.15.49:0/3160886875 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f975015ca60 0x7f973c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:35.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.087+0000 7f97552e6640 1 --2- 172.21.15.49:0/3160886875 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f973c0047d0 0x7f973c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:35.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.087+0000 7f9755ae7640 1 -- 172.21.15.49:0/3160886875 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f975015ca60 msgr2=0x7f973c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:35.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.087+0000 7f9755ae7640 1 --2- 172.21.15.49:0/3160886875 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f975015ca60 0x7f973c005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:35.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.087+0000 7f9755ae7640 1 -- 172.21.15.49:0/3160886875 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f973c0047d0 msgr2=0x7f973c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:35.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.087+0000 7f9755ae7640 1 --2- 172.21.15.49:0/3160886875 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f973c0047d0 0x7f973c004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:35.096 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.087+0000 7f9755ae7640 1 -- 172.21.15.49:0/3160886875 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f973c003530 con 0x7f973c003980 2023-11-06T02:10:35.096 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.087+0000 7f97552e6640 1 --2- 172.21.15.49:0/3160886875 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f973c0047d0 0x7f973c004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:10:35.096 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.091+0000 7f9755ae7640 1 --2- 172.21.15.49:0/3160886875 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f973c003980 0x7f973c003da0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f974c009ae0 tx=0x7f974c01ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2bd713d9bb8ff67 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:35.096 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.091+0000 7f9754ae5640 1 -- 172.21.15.49:0/3160886875 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f974c0a6460 con 0x7f973c003980 2023-11-06T02:10:35.096 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.091+0000 7f9754ae5640 1 -- 172.21.15.49:0/3160886875 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f974c0a65c0 con 0x7f973c003980 2023-11-06T02:10:35.097 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.091+0000 7f9754ae5640 1 -- 172.21.15.49:0/3160886875 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f974c0a6970 con 0x7f973c003980 2023-11-06T02:10:35.097 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.091+0000 7f9757d4b640 1 -- 172.21.15.49:0/3160886875 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f973c003980 msgr2=0x7f973c003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:35.097 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.091+0000 7f9757d4b640 1 --2- 172.21.15.49:0/3160886875 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f973c003980 0x7f973c003da0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f974c009ae0 tx=0x7f974c01ca50 comp rx=0 tx=0).stop 2023-11-06T02:10:35.097 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.091+0000 7f97562e8640 1 -- 172.21.15.49:0/3160886875 reap_dead start 2023-11-06T02:10:35.097 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.091+0000 7f9757d4b640 1 -- 172.21.15.49:0/3160886875 shutdown_connections 2023-11-06T02:10:35.097 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.091+0000 7f9757d4b640 1 -- 172.21.15.49:0/3160886875 >> 172.21.15.49:0/3160886875 conn(0x7f975006c160 msgr2=0x7f975006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:35.098 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.091+0000 7f9757d4b640 1 -- 172.21.15.49:0/3160886875 shutdown_connections 2023-11-06T02:10:35.098 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.091+0000 7f9757d4b640 1 -- 172.21.15.49:0/3160886875 wait complete. 2023-11-06T02:10:35.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.091+0000 7f9757d4b640 1 Processor -- start 2023-11-06T02:10:35.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.091+0000 7f9757d4b640 1 -- start start 2023-11-06T02:10:35.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.095+0000 7f9757d4b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f973c003980 0x7f975015f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:35.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.095+0000 7f9757d4b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9750159a60 0x7f975015ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:35.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.095+0000 7f9757d4b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f97500acfe0 0x7f97500ad460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:35.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.095+0000 7f9757d4b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97500ad9a0 con 0x7f973c003980 2023-11-06T02:10:35.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.095+0000 7f9757d4b640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97500b25b0 con 0x7f9750159a60 2023-11-06T02:10:35.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.095+0000 7f9757d4b640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f975015d6e0 con 0x7f97500acfe0 2023-11-06T02:10:35.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.095+0000 7f9755ae7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f973c003980 0x7f975015f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:35.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.095+0000 7f9755ae7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f973c003980 0x7f975015f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44084/0 (socket says 172.21.15.49:44084) 2023-11-06T02:10:35.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.095+0000 7f9755ae7640 1 -- 172.21.15.49:0/2351958619 learned_addr learned my addr 172.21.15.49:0/2351958619 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:35.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.095+0000 7f97552e6640 1 --2- 172.21.15.49:0/2351958619 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9750159a60 0x7f975015ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:35.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.095+0000 7f97562e8640 1 --2- 172.21.15.49:0/2351958619 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f97500acfe0 0x7f97500ad460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:35.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.095+0000 7f9755ae7640 1 -- 172.21.15.49:0/2351958619 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f97500acfe0 msgr2=0x7f97500ad460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:35.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.095+0000 7f9755ae7640 1 --2- 172.21.15.49:0/2351958619 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f97500acfe0 0x7f97500ad460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:35.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.095+0000 7f9755ae7640 1 -- 172.21.15.49:0/2351958619 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9750159a60 msgr2=0x7f975015ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:35.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.095+0000 7f9755ae7640 1 --2- 172.21.15.49:0/2351958619 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9750159a60 0x7f975015ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:35.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.095+0000 7f9755ae7640 1 -- 172.21.15.49:0/2351958619 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f975015d9c0 con 0x7f973c003980 2023-11-06T02:10:35.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.095+0000 7f97552e6640 1 --2- 172.21.15.49:0/2351958619 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9750159a60 0x7f975015ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:10:35.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.095+0000 7f9755ae7640 1 --2- 172.21.15.49:0/2351958619 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f973c003980 0x7f975015f760 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f974c009ae0 tx=0x7f973c005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:35.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.095+0000 7f97427fc640 1 -- 172.21.15.49:0/2351958619 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f974c0a6460 con 0x7f973c003980 2023-11-06T02:10:35.105 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.095+0000 7f97427fc640 1 -- 172.21.15.49:0/2351958619 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f974c004220 con 0x7f973c003980 2023-11-06T02:10:35.105 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.095+0000 7f9757d4b640 1 -- 172.21.15.49:0/2351958619 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f975015dca0 con 0x7f973c003980 2023-11-06T02:10:35.105 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.095+0000 7f97427fc640 1 -- 172.21.15.49:0/2351958619 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f974c003620 con 0x7f973c003980 2023-11-06T02:10:35.105 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.095+0000 7f9757d4b640 1 -- 172.21.15.49:0/2351958619 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f975015d380 con 0x7f973c003980 2023-11-06T02:10:35.105 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.095+0000 7f9757d4b640 1 -- 172.21.15.49:0/2351958619 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f975015e1c0 con 0x7f973c003980 2023-11-06T02:10:35.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.095+0000 7f97427fc640 1 -- 172.21.15.49:0/2351958619 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f974c003780 con 0x7f973c003980 2023-11-06T02:10:35.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.095+0000 7f97427fc640 1 --2- 172.21.15.49:0/2351958619 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9734065a60 0x7f9734067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:35.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.095+0000 7f97427fc640 1 -- 172.21.15.49:0/2351958619 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f974c11d760 con 0x7f973c003980 2023-11-06T02:10:35.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.095+0000 7f97552e6640 1 --2- 172.21.15.49:0/2351958619 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9734065a60 0x7f9734067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:35.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.099+0000 7f97552e6640 1 --2- 172.21.15.49:0/2351958619 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9734065a60 0x7f9734067f20 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f97500adba0 tx=0x7f974400b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:35.107 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.099+0000 7f97427fc640 1 -- 172.21.15.49:0/2351958619 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f975015e1c0 con 0x7f973c003980 2023-11-06T02:10:35.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.291+0000 7f9757d4b640 1 -- 172.21.15.49:0/2351958619 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f975015e1c0 con 0x7f973c003980 2023-11-06T02:10:35.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.291+0000 7f97427fc640 1 -- 172.21.15.49:0/2351958619 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4648 (secure 0 0 0) 0x7f975015e1c0 con 0x7f973c003980 2023-11-06T02:10:35.292 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:10:35.292 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5096,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1396196190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1396196190},{"type":"v1","addr":"172.21.15.150:6835","nonce":1396196190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5102,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/2465209170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":2465209170},{"type":"v1","addr":"172.21.15.150:6837","nonce":2465209170}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5871,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/1006951804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1006951804},{"type":"v1","addr":"172.21.15.49:6833","nonce":1006951804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T02:10:31.903864+0000","modified":"2023-11-06T02:10:34.930237+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5877},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5877":{"gid":5877,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.49:6835/73939548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":73939548},{"type":"v1","addr":"172.21.15.49:6835","nonce":73939548}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-06T02:10:35.292 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 23 2023-11-06T02:10:35.293 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.295+0000 7f9757d4b640 1 -- 172.21.15.49:0/2351958619 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9734065a60 msgr2=0x7f9734067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:35.294 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.295+0000 7f9757d4b640 1 --2- 172.21.15.49:0/2351958619 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9734065a60 0x7f9734067f20 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f97500adba0 tx=0x7f974400b040 comp rx=0 tx=0).stop 2023-11-06T02:10:35.294 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.295+0000 7f9757d4b640 1 -- 172.21.15.49:0/2351958619 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f973c003980 msgr2=0x7f975015f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:35.294 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.295+0000 7f9757d4b640 1 --2- 172.21.15.49:0/2351958619 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f973c003980 0x7f975015f760 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f974c009ae0 tx=0x7f973c005800 comp rx=0 tx=0).stop 2023-11-06T02:10:35.294 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.295+0000 7f9757d4b640 1 -- 172.21.15.49:0/2351958619 shutdown_connections 2023-11-06T02:10:35.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.295+0000 7f9757d4b640 1 --2- 172.21.15.49:0/2351958619 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9734065a60 0x7f9734067f20 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:35.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.295+0000 7f9757d4b640 1 --2- 172.21.15.49:0/2351958619 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f97500acfe0 0x7f97500ad460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:35.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.295+0000 7f9757d4b640 1 --2- 172.21.15.49:0/2351958619 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9750159a60 0x7f975015ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:35.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.295+0000 7f9757d4b640 1 --2- 172.21.15.49:0/2351958619 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f973c003980 0x7f975015f760 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:35.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.295+0000 7f9757d4b640 1 -- 172.21.15.49:0/2351958619 >> 172.21.15.49:0/2351958619 conn(0x7f975006c160 msgr2=0x7f975015a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:35.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.295+0000 7f9757d4b640 1 -- 172.21.15.49:0/2351958619 shutdown_connections 2023-11-06T02:10:35.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.295+0000 7f9757d4b640 1 -- 172.21.15.49:0/2351958619 wait complete. 2023-11-06T02:10:35.311 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T02:10:35.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.439+0000 7f57db174640 1 Processor -- start 2023-11-06T02:10:35.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.439+0000 7f57db174640 1 -- start start 2023-11-06T02:10:35.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.439+0000 7f57db174640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f57d4159980 0x7f57d4071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:35.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.439+0000 7f57db174640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f57d40cb010 0x7f57d40cb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:35.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.439+0000 7f57db174640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57d4071890 con 0x7f57d40b1450 2023-11-06T02:10:35.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.439+0000 7f57db174640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57d4072410 con 0x7f57d40cb010 2023-11-06T02:10:35.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.439+0000 7f57db174640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57d40ce440 con 0x7f57d4159980 2023-11-06T02:10:35.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.439+0000 7f57d9711640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f57d40cb010 0x7f57d40cb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:35.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.439+0000 7f57d9711640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f57d40cb010 0x7f57d40cb4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44096/0 (socket says 172.21.15.49:44096) 2023-11-06T02:10:35.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.439+0000 7f57d9711640 1 -- 172.21.15.49:0/1545299576 learned_addr learned my addr 172.21.15.49:0/1545299576 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:35.443 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.439+0000 7f57d8f10640 1 --2- 172.21.15.49:0/1545299576 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f57d4159980 0x7f57d4071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:35.443 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.439+0000 7f57d9711640 1 -- 172.21.15.49:0/1545299576 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f57d4159980 msgr2=0x7f57d4071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:35.443 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.439+0000 7f57d9711640 1 --2- 172.21.15.49:0/1545299576 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f57d4159980 0x7f57d4071290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:35.443 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.439+0000 7f57d9711640 1 -- 172.21.15.49:0/1545299576 >> v1:172.21.15.150:6789/0 conn(0x7f57d40b1450 legacy=0x7f57d40cdd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:35.444 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.443+0000 7f57d9711640 1 -- 172.21.15.49:0/1545299576 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57d4071b70 con 0x7f57d40cb010 2023-11-06T02:10:35.444 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.443+0000 7f57d8f10640 1 --2- 172.21.15.49:0/1545299576 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f57d4159980 0x7f57d4071290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:10:35.445 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.443+0000 7f57d9711640 1 --2- 172.21.15.49:0/1545299576 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f57d40cb010 0x7f57d40cb4a0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f57c4009870 tx=0x7f57c401c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94ce9ea2bee9a84d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:35.445 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.443+0000 7f57d37fe640 1 -- 172.21.15.49:0/1545299576 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c40ac630 con 0x7f57d40cb010 2023-11-06T02:10:35.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.443+0000 7f57d37fe640 1 -- 172.21.15.49:0/1545299576 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57c40ac790 con 0x7f57d40cb010 2023-11-06T02:10:35.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.443+0000 7f57d37fe640 1 -- 172.21.15.49:0/1545299576 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c40acb20 con 0x7f57d40cb010 2023-11-06T02:10:35.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.443+0000 7f57db174640 1 -- 172.21.15.49:0/1545299576 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f57d40cb010 msgr2=0x7f57d40cb4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:35.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.443+0000 7f57db174640 1 --2- 172.21.15.49:0/1545299576 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f57d40cb010 0x7f57d40cb4a0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f57c4009870 tx=0x7f57c401c500 comp rx=0 tx=0).stop 2023-11-06T02:10:35.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.443+0000 7f57db174640 1 -- 172.21.15.49:0/1545299576 shutdown_connections 2023-11-06T02:10:35.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.443+0000 7f57db174640 1 --2- 172.21.15.49:0/1545299576 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f57d40cb010 0x7f57d40cb4a0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:35.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.443+0000 7f57db174640 1 --2- 172.21.15.49:0/1545299576 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f57d4159980 0x7f57d4071290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:35.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.443+0000 7f57db174640 1 -- 172.21.15.49:0/1545299576 >> 172.21.15.49:0/1545299576 conn(0x7f57d406c0a0 msgr2=0x7f57d406c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:35.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.443+0000 7f57db174640 1 -- 172.21.15.49:0/1545299576 shutdown_connections 2023-11-06T02:10:35.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.443+0000 7f57db174640 1 -- 172.21.15.49:0/1545299576 wait complete. 2023-11-06T02:10:35.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.447+0000 7f57db174640 1 Processor -- start 2023-11-06T02:10:35.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.447+0000 7f57db174640 1 -- start start 2023-11-06T02:10:35.453 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.447+0000 7f57db174640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57d40b1450 0x7f57d40c9b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:35.453 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.447+0000 7f57db174640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f57d40cb010 0x7f57d4147be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:35.453 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.447+0000 7f57db174640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f57d4159980 0x7f57d4148120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:35.453 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.447+0000 7f57db174640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57d40b1890 con 0x7f57d40cb010 2023-11-06T02:10:35.453 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.447+0000 7f57db174640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57d4148660 con 0x7f57d40b1450 2023-11-06T02:10:35.454 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.447+0000 7f57db174640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57d41541c0 con 0x7f57d4159980 2023-11-06T02:10:35.454 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.447+0000 7f57d3fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f57d40cb010 0x7f57d4147be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:35.454 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.447+0000 7f57d3fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f57d40cb010 0x7f57d4147be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44100/0 (socket says 172.21.15.49:44100) 2023-11-06T02:10:35.454 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.447+0000 7f57d3fff640 1 -- 172.21.15.49:0/747741551 learned_addr learned my addr 172.21.15.49:0/747741551 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:35.455 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.447+0000 7f57d8f10640 1 --2- 172.21.15.49:0/747741551 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57d40b1450 0x7f57d40c9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:35.455 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.447+0000 7f57d9711640 1 --2- 172.21.15.49:0/747741551 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f57d4159980 0x7f57d4148120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:35.455 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.447+0000 7f57d3fff640 1 -- 172.21.15.49:0/747741551 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f57d4159980 msgr2=0x7f57d4148120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:35.455 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.447+0000 7f57d3fff640 1 --2- 172.21.15.49:0/747741551 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f57d4159980 0x7f57d4148120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:35.455 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.447+0000 7f57d3fff640 1 -- 172.21.15.49:0/747741551 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57d40b1450 msgr2=0x7f57d40c9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:35.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.447+0000 7f57d3fff640 1 --2- 172.21.15.49:0/747741551 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57d40b1450 0x7f57d40c9b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:35.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.447+0000 7f57d3fff640 1 -- 172.21.15.49:0/747741551 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57c40094d0 con 0x7f57d40cb010 2023-11-06T02:10:35.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.447+0000 7f57d3fff640 1 --2- 172.21.15.49:0/747741551 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f57d40cb010 0x7f57d4147be0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f57c800c7f0 tx=0x7f57c800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:35.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.447+0000 7f57d17fa640 1 -- 172.21.15.49:0/747741551 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c800a9f0 con 0x7f57d40cb010 2023-11-06T02:10:35.457 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.447+0000 7f57d17fa640 1 -- 172.21.15.49:0/747741551 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57c800ab50 con 0x7f57d40cb010 2023-11-06T02:10:35.457 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.447+0000 7f57d17fa640 1 -- 172.21.15.49:0/747741551 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c8012420 con 0x7f57d40cb010 2023-11-06T02:10:35.457 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.447+0000 7f57db174640 1 -- 172.21.15.49:0/747741551 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57d41535b0 con 0x7f57d40cb010 2023-11-06T02:10:35.457 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.447+0000 7f57db174640 1 -- 172.21.15.49:0/747741551 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57d4153b70 con 0x7f57d40cb010 2023-11-06T02:10:35.457 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.447+0000 7f57db174640 1 -- 172.21.15.49:0/747741551 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57d4074040 con 0x7f57d40cb010 2023-11-06T02:10:35.458 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.451+0000 7f57d17fa640 1 -- 172.21.15.49:0/747741551 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f57c8012580 con 0x7f57d40cb010 2023-11-06T02:10:35.458 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.451+0000 7f57d17fa640 1 --2- 172.21.15.49:0/747741551 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f57c0065a60 0x7f57c0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:35.458 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.451+0000 7f57d17fa640 1 -- 172.21.15.49:0/747741551 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f57c8083430 con 0x7f57d40cb010 2023-11-06T02:10:35.458 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.451+0000 7f57d8f10640 1 --2- 172.21.15.49:0/747741551 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f57c0065a60 0x7f57c0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:35.459 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.451+0000 7f57d8f10640 1 --2- 172.21.15.49:0/747741551 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f57c0065a60 0x7f57c0067f20 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f57cc00a950 tx=0x7f57cc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:35.459 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.455+0000 7f57d17fa640 1 -- 172.21.15.49:0/747741551 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f57c8050d20 con 0x7f57d40cb010 2023-11-06T02:10:35.614 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.615+0000 7f57db174640 1 -- 172.21.15.49:0/747741551 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f57d4154300 con 0x7f57d40cb010 2023-11-06T02:10:35.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.619+0000 7f57d17fa640 1 -- 172.21.15.49:0/747741551 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+17883 (secure 0 0 0) 0x7f57c8054340 con 0x7f57d40cb010 2023-11-06T02:10:35.617 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:10:35.618 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":51,"fsid":"1a8b035c-6ba4-499a-b876-aea47fd9c0d4","created":"2023-11-06T02:08:06.440857+0000","modified":"2023-11-06T02:10:32.908334+0000","last_up_change":"2023-11-06T02:08:17.982372+0000","last_in_change":"2023-11-06T02:08:08.906631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T02:08:19.252222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T02:08:24.776211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-06T02:10:30.458849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-06T02:10:30.899578+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ad55af8-0571-4e6a-8722-b5d429b7496d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6809","nonce":38116},{"type":"v1","addr":"172.21.15.49:6811","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6813","nonce":38116},{"type":"v1","addr":"172.21.15.49:6815","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6821","nonce":38116},{"type":"v1","addr":"172.21.15.49:6823","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6817","nonce":38116},{"type":"v1","addr":"172.21.15.49:6819","nonce":38116}]},"public_addr":"172.21.15.49:6811/38116","cluster_addr":"172.21.15.49:6815/38116","heartbeat_back_addr":"172.21.15.49:6823/38116","heartbeat_front_addr":"172.21.15.49:6819/38116","state":["exists","up"]},{"osd":1,"uuid":"c8b31f85-a757-43ed-9419-351acc46611b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":38113},{"type":"v1","addr":"172.21.15.49:6810","nonce":38113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":38113},{"type":"v1","addr":"172.21.15.49:6814","nonce":38113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":38113},{"type":"v1","addr":"172.21.15.49:6822","nonce":38113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":38113},{"type":"v1","addr":"172.21.15.49:6818","nonce":38113}]},"public_addr":"172.21.15.49:6810/38113","cluster_addr":"172.21.15.49:6814/38113","heartbeat_back_addr":"172.21.15.49:6822/38113","heartbeat_front_addr":"172.21.15.49:6818/38113","state":["exists","up"]},{"osd":2,"uuid":"555b3bf3-0a70-4b91-8efd-e4204b5fc26e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":38114},{"type":"v1","addr":"172.21.15.49:6801","nonce":38114}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":38114},{"type":"v1","addr":"172.21.15.49:6803","nonce":38114}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":38114},{"type":"v1","addr":"172.21.15.49:6807","nonce":38114}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":38114},{"type":"v1","addr":"172.21.15.49:6805","nonce":38114}]},"public_addr":"172.21.15.49:6801/38114","cluster_addr":"172.21.15.49:6803/38114","heartbeat_back_addr":"172.21.15.49:6807/38114","heartbeat_front_addr":"172.21.15.49:6805/38114","state":["exists","up"]},{"osd":3,"uuid":"8d4e5d77-03ef-4b9b-bfcb-5ae93a46795c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":38115},{"type":"v1","addr":"172.21.15.49:6825","nonce":38115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":38115},{"type":"v1","addr":"172.21.15.49:6827","nonce":38115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":38115},{"type":"v1","addr":"172.21.15.49:6831","nonce":38115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":38115},{"type":"v1","addr":"172.21.15.49:6829","nonce":38115}]},"public_addr":"172.21.15.49:6825/38115","cluster_addr":"172.21.15.49:6827/38115","heartbeat_back_addr":"172.21.15.49:6831/38115","heartbeat_front_addr":"172.21.15.49:6829/38115","state":["exists","up"]},{"osd":4,"uuid":"dba03ad4-68a3-4cf3-9218-ac611f02eacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":38417},{"type":"v1","addr":"172.21.15.150:6809","nonce":38417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":38417},{"type":"v1","addr":"172.21.15.150:6811","nonce":38417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":38417},{"type":"v1","addr":"172.21.15.150:6815","nonce":38417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":38417},{"type":"v1","addr":"172.21.15.150:6813","nonce":38417}]},"public_addr":"172.21.15.150:6809/38417","cluster_addr":"172.21.15.150:6811/38417","heartbeat_back_addr":"172.21.15.150:6815/38417","heartbeat_front_addr":"172.21.15.150:6813/38417","state":["exists","up"]},{"osd":5,"uuid":"0df9c303-c255-4640-8e91-01f1c3efe18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":38418},{"type":"v1","addr":"172.21.15.150:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":38418},{"type":"v1","addr":"172.21.15.150:6803","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":38418},{"type":"v1","addr":"172.21.15.150:6807","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":38418},{"type":"v1","addr":"172.21.15.150:6805","nonce":38418}]},"public_addr":"172.21.15.150:6801/38418","cluster_addr":"172.21.15.150:6803/38418","heartbeat_back_addr":"172.21.15.150:6807/38418","heartbeat_front_addr":"172.21.15.150:6805/38418","state":["exists","up"]},{"osd":6,"uuid":"e59fa189-4b70-4c06-9722-c02b90828bf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":38428},{"type":"v1","addr":"172.21.15.150:6825","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":38428},{"type":"v1","addr":"172.21.15.150:6827","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":38428},{"type":"v1","addr":"172.21.15.150:6831","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":38428},{"type":"v1","addr":"172.21.15.150:6829","nonce":38428}]},"public_addr":"172.21.15.150:6825/38428","cluster_addr":"172.21.15.150:6827/38428","heartbeat_back_addr":"172.21.15.150:6831/38428","heartbeat_front_addr":"172.21.15.150:6829/38428","state":["exists","up"]},{"osd":7,"uuid":"a64f57e1-ef98-46bd-b304-385759e95898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":38421},{"type":"v1","addr":"172.21.15.150:6817","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":38421},{"type":"v1","addr":"172.21.15.150:6819","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":38421},{"type":"v1","addr":"172.21.15.150:6823","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":38421},{"type":"v1","addr":"172.21.15.150:6821","nonce":38421}]},"public_addr":"172.21.15.150:6817/38421","cluster_addr":"172.21.15.150:6819/38421","heartbeat_back_addr":"172.21.15.150:6823/38421","heartbeat_front_addr":"172.21.15.150:6821/38421","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.438552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.413280+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.296466+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.424966+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.276953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.230984+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.410235+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.480084+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T02:10:35.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.619+0000 7f57db174640 1 -- 172.21.15.49:0/747741551 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f57c0065a60 msgr2=0x7f57c0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:35.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.619+0000 7f57db174640 1 --2- 172.21.15.49:0/747741551 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f57c0065a60 0x7f57c0067f20 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f57cc00a950 tx=0x7f57cc008040 comp rx=0 tx=0).stop 2023-11-06T02:10:35.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.619+0000 7f57db174640 1 -- 172.21.15.49:0/747741551 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f57d40cb010 msgr2=0x7f57d4147be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:35.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.619+0000 7f57db174640 1 --2- 172.21.15.49:0/747741551 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f57d40cb010 0x7f57d4147be0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f57c800c7f0 tx=0x7f57c800ccb0 comp rx=0 tx=0).stop 2023-11-06T02:10:35.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.619+0000 7f57db174640 1 -- 172.21.15.49:0/747741551 shutdown_connections 2023-11-06T02:10:35.621 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.619+0000 7f57db174640 1 --2- 172.21.15.49:0/747741551 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f57c0065a60 0x7f57c0067f20 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:35.621 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.619+0000 7f57db174640 1 --2- 172.21.15.49:0/747741551 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f57d40cb010 0x7f57d4147be0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:35.621 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.619+0000 7f57db174640 1 --2- 172.21.15.49:0/747741551 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f57d4159980 0x7f57d4148120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:35.621 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.619+0000 7f57db174640 1 --2- 172.21.15.49:0/747741551 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57d40b1450 0x7f57d40c9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:35.621 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.619+0000 7f57db174640 1 -- 172.21.15.49:0/747741551 >> 172.21.15.49:0/747741551 conn(0x7f57d406c0a0 msgr2=0x7f57d40af950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:35.622 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.619+0000 7f57db174640 1 -- 172.21.15.49:0/747741551 shutdown_connections 2023-11-06T02:10:35.622 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.619+0000 7f57db174640 1 -- 172.21.15.49:0/747741551 wait complete. 2023-11-06T02:10:35.637 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-06T02:10:35.638 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T02:10:35.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.759+0000 7f71b285f640 1 Processor -- start 2023-11-06T02:10:35.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.759+0000 7f71b285f640 1 -- start start 2023-11-06T02:10:35.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.759+0000 7f71b285f640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71ac15ab50 0x7f71ac0c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:35.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.759+0000 7f71b285f640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f71ac0b0790 0x7f71ac0c2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:35.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.759+0000 7f71b285f640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71ac0cab80 con 0x7f71ac0cb010 2023-11-06T02:10:35.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.759+0000 7f71b285f640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71ac15af30 con 0x7f71ac0b0790 2023-11-06T02:10:35.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.759+0000 7f71b285f640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71ac0cb5b0 con 0x7f71ac15ab50 2023-11-06T02:10:35.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.759+0000 7f71b0dfc640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f71ac0b0790 0x7f71ac0c2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:35.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.759+0000 7f71ab7fe640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f71ac0cb010 0x7f71ac0b0080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:41558/0 (socket says 172.21.15.49:41558) 2023-11-06T02:10:35.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.759+0000 7f71ab7fe640 1 -- 172.21.15.49:0/368069239 learned_addr learned my addr 172.21.15.49:0/368069239 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:10:35.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.759+0000 7f71abfff640 1 --2- 172.21.15.49:0/368069239 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71ac15ab50 0x7f71ac0c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:35.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.759+0000 7f71b0dfc640 1 -- 172.21.15.49:0/368069239 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71ac15ab50 msgr2=0x7f71ac0c9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:35.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.759+0000 7f71b0dfc640 1 --2- 172.21.15.49:0/368069239 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71ac15ab50 0x7f71ac0c9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:35.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.759+0000 7f71b0dfc640 1 -- 172.21.15.49:0/368069239 >> v1:172.21.15.49:6789/0 conn(0x7f71ac0cb010 legacy=0x7f71ac0b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:35.762 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.759+0000 7f71b0dfc640 1 -- 172.21.15.49:0/368069239 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71ac0cb890 con 0x7f71ac0b0790 2023-11-06T02:10:35.762 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.759+0000 7f71aaffd640 1 -- 172.21.15.49:0/368069239 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2036270609 0 0) 0x7f71ac0cab80 con 0x7f71ac0cb010 2023-11-06T02:10:35.762 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.759+0000 7f71abfff640 1 --2- 172.21.15.49:0/368069239 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71ac15ab50 0x7f71ac0c9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:10:35.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.759+0000 7f71b0dfc640 1 --2- 172.21.15.49:0/368069239 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71ac0b0790 0x7f71ac0c2710 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f71a4009d90 tx=0x7f71a401c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1d7b925ac1d6f22 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:35.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.759+0000 7f71aaffd640 1 -- 172.21.15.49:0/368069239 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a40a59e0 con 0x7f71ac0b0790 2023-11-06T02:10:35.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.759+0000 7f71aaffd640 1 -- 172.21.15.49:0/368069239 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71a40a5b40 con 0x7f71ac0b0790 2023-11-06T02:10:35.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.759+0000 7f71aaffd640 1 -- 172.21.15.49:0/368069239 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a40ad440 con 0x7f71ac0b0790 2023-11-06T02:10:35.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.763+0000 7f71b285f640 1 -- 172.21.15.49:0/368069239 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71ac0b0790 msgr2=0x7f71ac0c2710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:35.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.763+0000 7f71b285f640 1 --2- 172.21.15.49:0/368069239 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71ac0b0790 0x7f71ac0c2710 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f71a4009d90 tx=0x7f71a401c920 comp rx=0 tx=0).stop 2023-11-06T02:10:35.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.763+0000 7f71b285f640 1 -- 172.21.15.49:0/368069239 shutdown_connections 2023-11-06T02:10:35.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.763+0000 7f71b285f640 1 --2- 172.21.15.49:0/368069239 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71ac0b0790 0x7f71ac0c2710 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:35.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.763+0000 7f71b285f640 1 --2- 172.21.15.49:0/368069239 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71ac15ab50 0x7f71ac0c9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:35.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.763+0000 7f71b285f640 1 -- 172.21.15.49:0/368069239 >> 172.21.15.49:0/368069239 conn(0x7f71ac06c0a0 msgr2=0x7f71ac06c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:35.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.763+0000 7f71b285f640 1 -- 172.21.15.49:0/368069239 shutdown_connections 2023-11-06T02:10:35.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.763+0000 7f71b285f640 1 -- 172.21.15.49:0/368069239 wait complete. 2023-11-06T02:10:35.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.763+0000 7f71b285f640 1 Processor -- start 2023-11-06T02:10:35.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.763+0000 7f71b285f640 1 -- start start 2023-11-06T02:10:35.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.763+0000 7f71b285f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71ac0b0790 0x7f71ac1523e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:35.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.763+0000 7f71b285f640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71ac0cb010 0x7f71ac0c16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:35.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.763+0000 7f71b285f640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71ac15ab50 0x7f71ac0af180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:35.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.763+0000 7f71b285f640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71ac158bc0 con 0x7f71ac0b0790 2023-11-06T02:10:35.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.763+0000 7f71b285f640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71ac14a200 con 0x7f71ac15ab50 2023-11-06T02:10:35.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.763+0000 7f71b285f640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71ac0c6570 con 0x7f71ac0cb010 2023-11-06T02:10:35.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.763+0000 7f71abfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71ac0b0790 0x7f71ac1523e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:35.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.763+0000 7f71abfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71ac0b0790 0x7f71ac1523e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44114/0 (socket says 172.21.15.49:44114) 2023-11-06T02:10:35.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.763+0000 7f71abfff640 1 -- 172.21.15.49:0/2462410152 learned_addr learned my addr 172.21.15.49:0/2462410152 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:35.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.763+0000 7f71abfff640 1 -- 172.21.15.49:0/2462410152 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71ac0cb010 msgr2=0x7f71ac0c16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:35.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.763+0000 7f71abfff640 1 --2- 172.21.15.49:0/2462410152 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71ac0cb010 0x7f71ac0c16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:35.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.763+0000 7f71abfff640 1 -- 172.21.15.49:0/2462410152 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71ac15ab50 msgr2=0x7f71ac0af180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:35.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.763+0000 7f71abfff640 1 --2- 172.21.15.49:0/2462410152 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71ac15ab50 0x7f71ac0af180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:35.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.763+0000 7f71abfff640 1 -- 172.21.15.49:0/2462410152 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71a4009a70 con 0x7f71ac0b0790 2023-11-06T02:10:35.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.763+0000 7f71abfff640 1 --2- 172.21.15.49:0/2462410152 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71ac0b0790 0x7f71ac1523e0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f719400c820 tx=0x7f719400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:35.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.763+0000 7f71a8ff9640 1 -- 172.21.15.49:0/2462410152 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7194005ba0 con 0x7f71ac0b0790 2023-11-06T02:10:35.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.763+0000 7f71a8ff9640 1 -- 172.21.15.49:0/2462410152 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7194005d00 con 0x7f71ac0b0790 2023-11-06T02:10:35.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.763+0000 7f71b285f640 1 -- 172.21.15.49:0/2462410152 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71ac0c6850 con 0x7f71ac0b0790 2023-11-06T02:10:35.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.763+0000 7f71a8ff9640 1 -- 172.21.15.49:0/2462410152 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7194007a50 con 0x7f71ac0b0790 2023-11-06T02:10:35.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.763+0000 7f71b285f640 1 -- 172.21.15.49:0/2462410152 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71ac0c6e10 con 0x7f71ac0b0790 2023-11-06T02:10:35.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.767+0000 7f71b285f640 1 -- 172.21.15.49:0/2462410152 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71ac0b0de0 con 0x7f71ac0b0790 2023-11-06T02:10:35.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.767+0000 7f71a8ff9640 1 -- 172.21.15.49:0/2462410152 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f7194007bb0 con 0x7f71ac0b0790 2023-11-06T02:10:35.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.767+0000 7f71a8ff9640 1 --2- 172.21.15.49:0/2462410152 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f718c065a60 0x7f718c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:35.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.767+0000 7f71a8ff9640 1 -- 172.21.15.49:0/2462410152 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7194083170 con 0x7f71ac0b0790 2023-11-06T02:10:35.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.767+0000 7f71ab7fe640 1 --2- 172.21.15.49:0/2462410152 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f718c065a60 0x7f718c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:35.776 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.767+0000 7f71ab7fe640 1 --2- 172.21.15.49:0/2462410152 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f718c065a60 0x7f718c067f20 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f719c0011f0 tx=0x7f719c007930 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:35.776 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.771+0000 7f71a8ff9640 1 -- 172.21.15.49:0/2462410152 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f7194050a60 con 0x7f71ac0b0790 2023-11-06T02:10:35.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.923+0000 7f71b285f640 1 -- 172.21.15.49:0/2462410152 --> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f71ac0cbe50 con 0x7f718c065a60 2023-11-06T02:10:35.948 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.947+0000 7f71a8ff9640 1 -- 172.21.15.49:0/2462410152 <== mgr.4100 v2:172.21.15.150:6832/38104 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+309433 (secure 0 0 0) 0x7f719c0070d0 con 0x7f718c065a60 2023-11-06T02:10:35.952 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:10:35.954 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2023-11-06T02:10:35.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.951+0000 7f71b285f640 1 -- 172.21.15.49:0/2462410152 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f718c065a60 msgr2=0x7f718c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:35.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.951+0000 7f71b285f640 1 --2- 172.21.15.49:0/2462410152 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f718c065a60 0x7f718c067f20 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f719c0011f0 tx=0x7f719c007930 comp rx=0 tx=0).stop 2023-11-06T02:10:35.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.951+0000 7f71b285f640 1 -- 172.21.15.49:0/2462410152 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71ac0b0790 msgr2=0x7f71ac1523e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:35.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.951+0000 7f71b285f640 1 --2- 172.21.15.49:0/2462410152 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71ac0b0790 0x7f71ac1523e0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f719400c820 tx=0x7f719400cce0 comp rx=0 tx=0).stop 2023-11-06T02:10:35.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.951+0000 7f71b285f640 1 -- 172.21.15.49:0/2462410152 shutdown_connections 2023-11-06T02:10:35.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.951+0000 7f71b285f640 1 --2- 172.21.15.49:0/2462410152 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f718c065a60 0x7f718c067f20 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:35.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.951+0000 7f71b285f640 1 --2- 172.21.15.49:0/2462410152 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71ac15ab50 0x7f71ac0af180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:35.957 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.951+0000 7f71b285f640 1 --2- 172.21.15.49:0/2462410152 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71ac0cb010 0x7f71ac0c16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:35.957 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.951+0000 7f71b285f640 1 --2- 172.21.15.49:0/2462410152 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71ac0b0790 0x7f71ac1523e0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:35.957 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.951+0000 7f71b285f640 1 -- 172.21.15.49:0/2462410152 >> 172.21.15.49:0/2462410152 conn(0x7f71ac06c0a0 msgr2=0x7f71ac0c3a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:35.957 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.951+0000 7f71b285f640 1 -- 172.21.15.49:0/2462410152 shutdown_connections 2023-11-06T02:10:35.957 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:35.951+0000 7f71b285f640 1 -- 172.21.15.49:0/2462410152 wait complete. 2023-11-06T02:10:35.968 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":113,"stamp":"2023-11-06T02:10:35.219326+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":68,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237828,"kb_used_data":5828,"kb_used_omap":0,"kb_used_meta":231936,"kb_avail":754736892,"statfs":{"total":773094113280,"available":772850577408,"internally_reserved":0,"allocated":5967872,"data_stored":3216425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":237502464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":54,"apply_latency_ms":54,"commit_latency_ns":54000000,"apply_latency_ns":54000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.702203"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.962872+0000","last_change":"2023-11-06T02:10:31.961881+0000","last_active":"2023-11-06T02:10:31.962872+0000","last_peered":"2023-11-06T02:10:31.962872+0000","last_clean":"2023-11-06T02:10:31.962872+0000","last_became_active":"2023-11-06T02:10:31.961620+0000","last_became_peered":"2023-11-06T02:10:31.961620+0000","last_unstale":"2023-11-06T02:10:31.962872+0000","last_undegraded":"2023-11-06T02:10:31.962872+0000","last_fullsized":"2023-11-06T02:10:31.962872+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:42:44.259204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.971259+0000","last_change":"2023-11-06T02:10:31.963288+0000","last_active":"2023-11-06T02:10:31.971259+0000","last_peered":"2023-11-06T02:10:31.971259+0000","last_clean":"2023-11-06T02:10:31.971259+0000","last_became_active":"2023-11-06T02:10:31.962651+0000","last_became_peered":"2023-11-06T02:10:31.962651+0000","last_unstale":"2023-11-06T02:10:31.971259+0000","last_undegraded":"2023-11-06T02:10:31.971259+0000","last_fullsized":"2023-11-06T02:10:31.971259+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:27:11.980648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.949927+0000","last_change":"2023-11-06T02:10:31.934064+0000","last_active":"2023-11-06T02:10:31.949927+0000","last_peered":"2023-11-06T02:10:31.949927+0000","last_clean":"2023-11-06T02:10:31.949927+0000","last_became_active":"2023-11-06T02:10:31.933796+0000","last_became_peered":"2023-11-06T02:10:31.933796+0000","last_unstale":"2023-11-06T02:10:31.949927+0000","last_undegraded":"2023-11-06T02:10:31.949927+0000","last_fullsized":"2023-11-06T02:10:31.949927+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:53:49.090895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.967053+0000","last_change":"2023-11-06T02:10:31.949000+0000","last_active":"2023-11-06T02:10:31.967053+0000","last_peered":"2023-11-06T02:10:31.967053+0000","last_clean":"2023-11-06T02:10:31.967053+0000","last_became_active":"2023-11-06T02:10:31.948327+0000","last_became_peered":"2023-11-06T02:10:31.948327+0000","last_unstale":"2023-11-06T02:10:31.967053+0000","last_undegraded":"2023-11-06T02:10:31.967053+0000","last_fullsized":"2023-11-06T02:10:31.967053+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:13:48.613748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.957089+0000","last_change":"2023-11-06T02:10:31.940047+0000","last_active":"2023-11-06T02:10:31.957089+0000","last_peered":"2023-11-06T02:10:31.957089+0000","last_clean":"2023-11-06T02:10:31.957089+0000","last_became_active":"2023-11-06T02:10:31.939315+0000","last_became_peered":"2023-11-06T02:10:31.939315+0000","last_unstale":"2023-11-06T02:10:31.957089+0000","last_undegraded":"2023-11-06T02:10:31.957089+0000","last_fullsized":"2023-11-06T02:10:31.957089+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:06:02.709833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.945207+0000","last_change":"2023-11-06T02:10:31.941044+0000","last_active":"2023-11-06T02:10:31.945207+0000","last_peered":"2023-11-06T02:10:31.945207+0000","last_clean":"2023-11-06T02:10:31.945207+0000","last_became_active":"2023-11-06T02:10:31.940771+0000","last_became_peered":"2023-11-06T02:10:31.940771+0000","last_unstale":"2023-11-06T02:10:31.945207+0000","last_undegraded":"2023-11-06T02:10:31.945207+0000","last_fullsized":"2023-11-06T02:10:31.945207+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:37:08.723213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.950136+0000","last_change":"2023-11-06T02:10:31.935759+0000","last_active":"2023-11-06T02:10:31.950136+0000","last_peered":"2023-11-06T02:10:31.950136+0000","last_clean":"2023-11-06T02:10:31.950136+0000","last_became_active":"2023-11-06T02:10:31.935167+0000","last_became_peered":"2023-11-06T02:10:31.935167+0000","last_unstale":"2023-11-06T02:10:31.950136+0000","last_undegraded":"2023-11-06T02:10:31.950136+0000","last_fullsized":"2023-11-06T02:10:31.950136+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:13:36.170486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.946704+0000","last_change":"2023-11-06T02:10:31.936273+0000","last_active":"2023-11-06T02:10:31.946704+0000","last_peered":"2023-11-06T02:10:31.946704+0000","last_clean":"2023-11-06T02:10:31.946704+0000","last_became_active":"2023-11-06T02:10:31.934790+0000","last_became_peered":"2023-11-06T02:10:31.934790+0000","last_unstale":"2023-11-06T02:10:31.946704+0000","last_undegraded":"2023-11-06T02:10:31.946704+0000","last_fullsized":"2023-11-06T02:10:31.946704+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:35:16.923535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.932018+0000","last_change":"2023-11-06T02:10:31.928872+0000","last_active":"2023-11-06T02:10:31.932018+0000","last_peered":"2023-11-06T02:10:31.932018+0000","last_clean":"2023-11-06T02:10:31.932018+0000","last_became_active":"2023-11-06T02:10:31.928595+0000","last_became_peered":"2023-11-06T02:10:31.928595+0000","last_unstale":"2023-11-06T02:10:31.932018+0000","last_undegraded":"2023-11-06T02:10:31.932018+0000","last_fullsized":"2023-11-06T02:10:31.932018+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:31:04.831175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.969509+0000","last_change":"2023-11-06T02:10:31.968307+0000","last_active":"2023-11-06T02:10:31.969509+0000","last_peered":"2023-11-06T02:10:31.969509+0000","last_clean":"2023-11-06T02:10:31.969509+0000","last_became_active":"2023-11-06T02:10:31.967962+0000","last_became_peered":"2023-11-06T02:10:31.967962+0000","last_unstale":"2023-11-06T02:10:31.969509+0000","last_undegraded":"2023-11-06T02:10:31.969509+0000","last_fullsized":"2023-11-06T02:10:31.969509+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:28:33.741295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-06T02:10:32.922318+0000","last_change":"2023-11-06T02:10:31.961301+0000","last_active":"2023-11-06T02:10:32.922318+0000","last_peered":"2023-11-06T02:10:32.922318+0000","last_clean":"2023-11-06T02:10:32.922318+0000","last_became_active":"2023-11-06T02:10:31.960991+0000","last_became_peered":"2023-11-06T02:10:31.960991+0000","last_unstale":"2023-11-06T02:10:32.922318+0000","last_undegraded":"2023-11-06T02:10:32.922318+0000","last_fullsized":"2023-11-06T02:10:32.922318+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:05:12.860701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.929838+0000","last_change":"2023-11-06T02:10:31.924870+0000","last_active":"2023-11-06T02:10:31.929838+0000","last_peered":"2023-11-06T02:10:31.929838+0000","last_clean":"2023-11-06T02:10:31.929838+0000","last_became_active":"2023-11-06T02:10:31.924577+0000","last_became_peered":"2023-11-06T02:10:31.924577+0000","last_unstale":"2023-11-06T02:10:31.929838+0000","last_undegraded":"2023-11-06T02:10:31.929838+0000","last_fullsized":"2023-11-06T02:10:31.929838+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:55:15.265490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.969490+0000","last_change":"2023-11-06T02:10:31.968305+0000","last_active":"2023-11-06T02:10:31.969490+0000","last_peered":"2023-11-06T02:10:31.969490+0000","last_clean":"2023-11-06T02:10:31.969490+0000","last_became_active":"2023-11-06T02:10:31.967985+0000","last_became_peered":"2023-11-06T02:10:31.967985+0000","last_unstale":"2023-11-06T02:10:31.969490+0000","last_undegraded":"2023-11-06T02:10:31.969490+0000","last_fullsized":"2023-11-06T02:10:31.969490+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:26:13.513716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.950001+0000","last_change":"2023-11-06T02:10:31.934794+0000","last_active":"2023-11-06T02:10:31.950001+0000","last_peered":"2023-11-06T02:10:31.950001+0000","last_clean":"2023-11-06T02:10:31.950001+0000","last_became_active":"2023-11-06T02:10:31.934512+0000","last_became_peered":"2023-11-06T02:10:31.934512+0000","last_unstale":"2023-11-06T02:10:31.950001+0000","last_undegraded":"2023-11-06T02:10:31.950001+0000","last_fullsized":"2023-11-06T02:10:31.950001+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:45:36.472356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.967791+0000","last_change":"2023-11-06T02:10:31.956262+0000","last_active":"2023-11-06T02:10:31.967791+0000","last_peered":"2023-11-06T02:10:31.967791+0000","last_clean":"2023-11-06T02:10:31.967791+0000","last_became_active":"2023-11-06T02:10:31.951419+0000","last_became_peered":"2023-11-06T02:10:31.951419+0000","last_unstale":"2023-11-06T02:10:31.967791+0000","last_undegraded":"2023-11-06T02:10:31.967791+0000","last_fullsized":"2023-11-06T02:10:31.967791+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:00:38.993498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.969788+0000","last_change":"2023-11-06T02:10:31.968492+0000","last_active":"2023-11-06T02:10:31.969788+0000","last_peered":"2023-11-06T02:10:31.969788+0000","last_clean":"2023-11-06T02:10:31.969788+0000","last_became_active":"2023-11-06T02:10:31.967960+0000","last_became_peered":"2023-11-06T02:10:31.967960+0000","last_unstale":"2023-11-06T02:10:31.969788+0000","last_undegraded":"2023-11-06T02:10:31.969788+0000","last_fullsized":"2023-11-06T02:10:31.969788+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:03:31.067962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.963205+0000","last_change":"2023-11-06T02:10:31.962386+0000","last_active":"2023-11-06T02:10:31.963205+0000","last_peered":"2023-11-06T02:10:31.963205+0000","last_clean":"2023-11-06T02:10:31.963205+0000","last_became_active":"2023-11-06T02:10:31.962192+0000","last_became_peered":"2023-11-06T02:10:31.962192+0000","last_unstale":"2023-11-06T02:10:31.963205+0000","last_undegraded":"2023-11-06T02:10:31.963205+0000","last_fullsized":"2023-11-06T02:10:31.963205+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:32:04.870127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.974757+0000","last_change":"2023-11-06T02:10:31.971319+0000","last_active":"2023-11-06T02:10:31.974757+0000","last_peered":"2023-11-06T02:10:31.974757+0000","last_clean":"2023-11-06T02:10:31.974757+0000","last_became_active":"2023-11-06T02:10:31.970637+0000","last_became_peered":"2023-11-06T02:10:31.970637+0000","last_unstale":"2023-11-06T02:10:31.974757+0000","last_undegraded":"2023-11-06T02:10:31.974757+0000","last_fullsized":"2023-11-06T02:10:31.974757+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:32.392045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.957430+0000","last_change":"2023-11-06T02:10:31.956621+0000","last_active":"2023-11-06T02:10:31.957430+0000","last_peered":"2023-11-06T02:10:31.957430+0000","last_clean":"2023-11-06T02:10:31.957430+0000","last_became_active":"2023-11-06T02:10:31.956361+0000","last_became_peered":"2023-11-06T02:10:31.956361+0000","last_unstale":"2023-11-06T02:10:31.957430+0000","last_undegraded":"2023-11-06T02:10:31.957430+0000","last_fullsized":"2023-11-06T02:10:31.957430+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:32:49.788949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-06T02:10:32.926256+0000","last_change":"2023-11-06T02:10:31.958138+0000","last_active":"2023-11-06T02:10:32.926256+0000","last_peered":"2023-11-06T02:10:32.926256+0000","last_clean":"2023-11-06T02:10:32.926256+0000","last_became_active":"2023-11-06T02:10:31.956588+0000","last_became_peered":"2023-11-06T02:10:31.956588+0000","last_unstale":"2023-11-06T02:10:32.926256+0000","last_undegraded":"2023-11-06T02:10:32.926256+0000","last_fullsized":"2023-11-06T02:10:32.926256+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:16:37.324576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-06T02:10:32.926625+0000","last_change":"2023-11-06T02:10:31.948873+0000","last_active":"2023-11-06T02:10:32.926625+0000","last_peered":"2023-11-06T02:10:32.926625+0000","last_clean":"2023-11-06T02:10:32.926625+0000","last_became_active":"2023-11-06T02:10:31.948082+0000","last_became_peered":"2023-11-06T02:10:31.948082+0000","last_unstale":"2023-11-06T02:10:32.926625+0000","last_undegraded":"2023-11-06T02:10:32.926625+0000","last_fullsized":"2023-11-06T02:10:32.926625+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:45:06.160032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.946154+0000","last_change":"2023-11-06T02:10:31.931027+0000","last_active":"2023-11-06T02:10:31.946154+0000","last_peered":"2023-11-06T02:10:31.946154+0000","last_clean":"2023-11-06T02:10:31.946154+0000","last_became_active":"2023-11-06T02:10:31.930716+0000","last_became_peered":"2023-11-06T02:10:31.930716+0000","last_unstale":"2023-11-06T02:10:31.946154+0000","last_undegraded":"2023-11-06T02:10:31.946154+0000","last_fullsized":"2023-11-06T02:10:31.946154+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:54:44.600666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-06T02:10:32.921731+0000","last_change":"2023-11-06T02:10:31.940652+0000","last_active":"2023-11-06T02:10:32.921731+0000","last_peered":"2023-11-06T02:10:32.921731+0000","last_clean":"2023-11-06T02:10:32.921731+0000","last_became_active":"2023-11-06T02:10:31.940375+0000","last_became_peered":"2023-11-06T02:10:31.940375+0000","last_unstale":"2023-11-06T02:10:32.921731+0000","last_undegraded":"2023-11-06T02:10:32.921731+0000","last_fullsized":"2023-11-06T02:10:32.921731+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:39:40.475876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.950319+0000","last_change":"2023-11-06T02:10:31.936913+0000","last_active":"2023-11-06T02:10:31.950319+0000","last_peered":"2023-11-06T02:10:31.950319+0000","last_clean":"2023-11-06T02:10:31.950319+0000","last_became_active":"2023-11-06T02:10:31.936613+0000","last_became_peered":"2023-11-06T02:10:31.936613+0000","last_unstale":"2023-11-06T02:10:31.950319+0000","last_undegraded":"2023-11-06T02:10:31.950319+0000","last_fullsized":"2023-11-06T02:10:31.950319+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:57:16.913819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.950586+0000","last_change":"2023-11-06T02:10:31.946840+0000","last_active":"2023-11-06T02:10:31.950586+0000","last_peered":"2023-11-06T02:10:31.950586+0000","last_clean":"2023-11-06T02:10:31.950586+0000","last_became_active":"2023-11-06T02:10:31.946546+0000","last_became_peered":"2023-11-06T02:10:31.946546+0000","last_unstale":"2023-11-06T02:10:31.950586+0000","last_undegraded":"2023-11-06T02:10:31.950586+0000","last_fullsized":"2023-11-06T02:10:31.950586+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:15:49.333136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.947005+0000","last_change":"2023-11-06T02:10:31.939200+0000","last_active":"2023-11-06T02:10:31.947005+0000","last_peered":"2023-11-06T02:10:31.947005+0000","last_clean":"2023-11-06T02:10:31.947005+0000","last_became_active":"2023-11-06T02:10:31.938898+0000","last_became_peered":"2023-11-06T02:10:31.938898+0000","last_unstale":"2023-11-06T02:10:31.947005+0000","last_undegraded":"2023-11-06T02:10:31.947005+0000","last_fullsized":"2023-11-06T02:10:31.947005+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:18:59.261936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.942772+0000","last_change":"2023-11-06T02:10:31.936997+0000","last_active":"2023-11-06T02:10:31.942772+0000","last_peered":"2023-11-06T02:10:31.942772+0000","last_clean":"2023-11-06T02:10:31.942772+0000","last_became_active":"2023-11-06T02:10:31.936748+0000","last_became_peered":"2023-11-06T02:10:31.936748+0000","last_unstale":"2023-11-06T02:10:31.942772+0000","last_undegraded":"2023-11-06T02:10:31.942772+0000","last_fullsized":"2023-11-06T02:10:31.942772+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:07:03.689829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-06T02:10:32.926540+0000","last_change":"2023-11-06T02:10:31.931782+0000","last_active":"2023-11-06T02:10:32.926540+0000","last_peered":"2023-11-06T02:10:32.926540+0000","last_clean":"2023-11-06T02:10:32.926540+0000","last_became_active":"2023-11-06T02:10:31.931084+0000","last_became_peered":"2023-11-06T02:10:31.931084+0000","last_unstale":"2023-11-06T02:10:32.926540+0000","last_undegraded":"2023-11-06T02:10:32.926540+0000","last_fullsized":"2023-11-06T02:10:32.926540+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:34:11.282284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.951829+0000","last_change":"2023-11-06T02:10:31.950559+0000","last_active":"2023-11-06T02:10:31.951829+0000","last_peered":"2023-11-06T02:10:31.951829+0000","last_clean":"2023-11-06T02:10:31.951829+0000","last_became_active":"2023-11-06T02:10:31.950253+0000","last_became_peered":"2023-11-06T02:10:31.950253+0000","last_unstale":"2023-11-06T02:10:31.951829+0000","last_undegraded":"2023-11-06T02:10:31.951829+0000","last_fullsized":"2023-11-06T02:10:31.951829+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:57:03.504241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.966941+0000","last_change":"2023-11-06T02:10:31.948384+0000","last_active":"2023-11-06T02:10:31.966941+0000","last_peered":"2023-11-06T02:10:31.966941+0000","last_clean":"2023-11-06T02:10:31.966941+0000","last_became_active":"2023-11-06T02:10:31.947450+0000","last_became_peered":"2023-11-06T02:10:31.947450+0000","last_unstale":"2023-11-06T02:10:31.966941+0000","last_undegraded":"2023-11-06T02:10:31.966941+0000","last_fullsized":"2023-11-06T02:10:31.966941+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:28:33.741295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.950161+0000","last_change":"2023-11-06T02:10:31.935920+0000","last_active":"2023-11-06T02:10:31.950161+0000","last_peered":"2023-11-06T02:10:31.950161+0000","last_clean":"2023-11-06T02:10:31.950161+0000","last_became_active":"2023-11-06T02:10:31.935509+0000","last_became_peered":"2023-11-06T02:10:31.935509+0000","last_unstale":"2023-11-06T02:10:31.950161+0000","last_undegraded":"2023-11-06T02:10:31.950161+0000","last_fullsized":"2023-11-06T02:10:31.950161+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:18:59.261936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-06T02:10:32.927060+0000","last_change":"2023-11-06T02:10:31.965611+0000","last_active":"2023-11-06T02:10:32.927060+0000","last_peered":"2023-11-06T02:10:32.927060+0000","last_clean":"2023-11-06T02:10:32.927060+0000","last_became_active":"2023-11-06T02:10:31.964913+0000","last_became_peered":"2023-11-06T02:10:31.964913+0000","last_unstale":"2023-11-06T02:10:32.927060+0000","last_undegraded":"2023-11-06T02:10:32.927060+0000","last_fullsized":"2023-11-06T02:10:32.927060+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:32:49.788949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.972773+0000","last_change":"2023-11-06T02:10:31.969631+0000","last_active":"2023-11-06T02:10:31.972773+0000","last_peered":"2023-11-06T02:10:31.972773+0000","last_clean":"2023-11-06T02:10:31.972773+0000","last_became_active":"2023-11-06T02:10:31.969086+0000","last_became_peered":"2023-11-06T02:10:31.969086+0000","last_unstale":"2023-11-06T02:10:31.972773+0000","last_undegraded":"2023-11-06T02:10:31.972773+0000","last_fullsized":"2023-11-06T02:10:31.972773+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:06:02.709833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"50'4","reported_seq":15,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-06T02:10:32.926572+0000","last_change":"2023-11-06T02:10:31.562064+0000","last_active":"2023-11-06T02:10:32.926572+0000","last_peered":"2023-11-06T02:10:32.926572+0000","last_clean":"2023-11-06T02:10:32.926572+0000","last_became_active":"2023-11-06T02:10:31.561463+0000","last_became_peered":"2023-11-06T02:10:31.561463+0000","last_unstale":"2023-11-06T02:10:32.926572+0000","last_undegraded":"2023-11-06T02:10:32.926572+0000","last_fullsized":"2023-11-06T02:10:32.926572+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_clean_scrub_stamp":"2023-11-06T02:10:30.509051+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:09:08.748403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-06T02:10:32.928178+0000","last_change":"2023-11-06T02:10:31.963633+0000","last_active":"2023-11-06T02:10:32.928178+0000","last_peered":"2023-11-06T02:10:32.928178+0000","last_clean":"2023-11-06T02:10:32.928178+0000","last_became_active":"2023-11-06T02:10:31.962849+0000","last_became_peered":"2023-11-06T02:10:31.962849+0000","last_unstale":"2023-11-06T02:10:32.928178+0000","last_undegraded":"2023-11-06T02:10:32.928178+0000","last_fullsized":"2023-11-06T02:10:32.928178+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:01:30.948009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"50'1","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.947228+0000","last_change":"2023-11-06T02:10:31.556249+0000","last_active":"2023-11-06T02:10:31.947228+0000","last_peered":"2023-11-06T02:10:31.947228+0000","last_clean":"2023-11-06T02:10:31.947228+0000","last_became_active":"2023-11-06T02:10:31.555864+0000","last_became_peered":"2023-11-06T02:10:31.555864+0000","last_unstale":"2023-11-06T02:10:31.947228+0000","last_undegraded":"2023-11-06T02:10:31.947228+0000","last_fullsized":"2023-11-06T02:10:31.947228+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_clean_scrub_stamp":"2023-11-06T02:10:30.509051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:29:31.684272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-06T02:10:32.926160+0000","last_change":"2023-11-06T02:10:31.947889+0000","last_active":"2023-11-06T02:10:32.926160+0000","last_peered":"2023-11-06T02:10:32.926160+0000","last_clean":"2023-11-06T02:10:32.926160+0000","last_became_active":"2023-11-06T02:10:31.947029+0000","last_became_peered":"2023-11-06T02:10:31.947029+0000","last_unstale":"2023-11-06T02:10:32.926160+0000","last_undegraded":"2023-11-06T02:10:32.926160+0000","last_fullsized":"2023-11-06T02:10:32.926160+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:50:33.170839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"50'2","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.954057+0000","last_change":"2023-11-06T02:10:31.564822+0000","last_active":"2023-11-06T02:10:31.954057+0000","last_peered":"2023-11-06T02:10:31.954057+0000","last_clean":"2023-11-06T02:10:31.954057+0000","last_became_active":"2023-11-06T02:10:31.564460+0000","last_became_peered":"2023-11-06T02:10:31.564460+0000","last_unstale":"2023-11-06T02:10:31.954057+0000","last_undegraded":"2023-11-06T02:10:31.954057+0000","last_fullsized":"2023-11-06T02:10:31.954057+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_clean_scrub_stamp":"2023-11-06T02:10:30.509051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:05:11.856979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.947413+0000","last_change":"2023-11-06T02:10:31.941264+0000","last_active":"2023-11-06T02:10:31.947413+0000","last_peered":"2023-11-06T02:10:31.947413+0000","last_clean":"2023-11-06T02:10:31.947413+0000","last_became_active":"2023-11-06T02:10:31.937191+0000","last_became_peered":"2023-11-06T02:10:31.937191+0000","last_unstale":"2023-11-06T02:10:31.947413+0000","last_undegraded":"2023-11-06T02:10:31.947413+0000","last_fullsized":"2023-11-06T02:10:31.947413+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:57:16.913819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"50'2","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-06T02:10:32.928359+0000","last_change":"2023-11-06T02:10:31.558181+0000","last_active":"2023-11-06T02:10:32.928359+0000","last_peered":"2023-11-06T02:10:32.928359+0000","last_clean":"2023-11-06T02:10:32.928359+0000","last_became_active":"2023-11-06T02:10:31.557290+0000","last_became_peered":"2023-11-06T02:10:31.557290+0000","last_unstale":"2023-11-06T02:10:32.928359+0000","last_undegraded":"2023-11-06T02:10:32.928359+0000","last_fullsized":"2023-11-06T02:10:32.928359+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_clean_scrub_stamp":"2023-11-06T02:10:30.509051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:23:34.675851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.966033+0000","last_change":"2023-11-06T02:10:31.955324+0000","last_active":"2023-11-06T02:10:31.966033+0000","last_peered":"2023-11-06T02:10:31.966033+0000","last_clean":"2023-11-06T02:10:31.966033+0000","last_became_active":"2023-11-06T02:10:31.954475+0000","last_became_peered":"2023-11-06T02:10:31.954475+0000","last_unstale":"2023-11-06T02:10:31.966033+0000","last_undegraded":"2023-11-06T02:10:31.966033+0000","last_fullsized":"2023-11-06T02:10:31.966033+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:27:11.980648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"50'1","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.950665+0000","last_change":"2023-11-06T02:10:31.547787+0000","last_active":"2023-11-06T02:10:31.950665+0000","last_peered":"2023-11-06T02:10:31.950665+0000","last_clean":"2023-11-06T02:10:31.950665+0000","last_became_active":"2023-11-06T02:10:31.547398+0000","last_became_peered":"2023-11-06T02:10:31.547398+0000","last_unstale":"2023-11-06T02:10:31.950665+0000","last_undegraded":"2023-11-06T02:10:31.950665+0000","last_fullsized":"2023-11-06T02:10:31.950665+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_clean_scrub_stamp":"2023-11-06T02:10:30.509051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:16:17.912070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.970614+0000","last_change":"2023-11-06T02:10:31.960703+0000","last_active":"2023-11-06T02:10:31.970614+0000","last_peered":"2023-11-06T02:10:31.970614+0000","last_clean":"2023-11-06T02:10:31.970614+0000","last_became_active":"2023-11-06T02:10:31.959512+0000","last_became_peered":"2023-11-06T02:10:31.959512+0000","last_unstale":"2023-11-06T02:10:31.970614+0000","last_undegraded":"2023-11-06T02:10:31.970614+0000","last_fullsized":"2023-11-06T02:10:31.970614+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:26:13.513716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"50'1","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.977215+0000","last_change":"2023-11-06T02:10:31.557235+0000","last_active":"2023-11-06T02:10:31.977215+0000","last_peered":"2023-11-06T02:10:31.977215+0000","last_clean":"2023-11-06T02:10:31.977215+0000","last_became_active":"2023-11-06T02:10:31.556387+0000","last_became_peered":"2023-11-06T02:10:31.556387+0000","last_unstale":"2023-11-06T02:10:31.977215+0000","last_undegraded":"2023-11-06T02:10:31.977215+0000","last_fullsized":"2023-11-06T02:10:31.977215+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_clean_scrub_stamp":"2023-11-06T02:10:30.509051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:37:07.719491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.946964+0000","last_change":"2023-11-06T02:10:31.937747+0000","last_active":"2023-11-06T02:10:31.946964+0000","last_peered":"2023-11-06T02:10:31.946964+0000","last_clean":"2023-11-06T02:10:31.946964+0000","last_became_active":"2023-11-06T02:10:31.937479+0000","last_became_peered":"2023-11-06T02:10:31.937479+0000","last_unstale":"2023-11-06T02:10:31.946964+0000","last_undegraded":"2023-11-06T02:10:31.946964+0000","last_fullsized":"2023-11-06T02:10:31.946964+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:15:49.333136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-06T02:10:32.926047+0000","last_change":"2023-11-06T02:10:31.554495+0000","last_active":"2023-11-06T02:10:32.926047+0000","last_peered":"2023-11-06T02:10:32.926047+0000","last_clean":"2023-11-06T02:10:32.926047+0000","last_became_active":"2023-11-06T02:10:31.553778+0000","last_became_peered":"2023-11-06T02:10:31.553778+0000","last_unstale":"2023-11-06T02:10:32.926047+0000","last_undegraded":"2023-11-06T02:10:32.926047+0000","last_fullsized":"2023-11-06T02:10:32.926047+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_clean_scrub_stamp":"2023-11-06T02:10:30.509051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:15:11.104076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.967855+0000","last_change":"2023-11-06T02:10:31.956320+0000","last_active":"2023-11-06T02:10:31.967855+0000","last_peered":"2023-11-06T02:10:31.967855+0000","last_clean":"2023-11-06T02:10:31.967855+0000","last_became_active":"2023-11-06T02:10:31.951550+0000","last_became_peered":"2023-11-06T02:10:31.951550+0000","last_unstale":"2023-11-06T02:10:31.967855+0000","last_undegraded":"2023-11-06T02:10:31.967855+0000","last_fullsized":"2023-11-06T02:10:31.967855+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:42:44.259204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.923073+0000","last_change":"2023-11-06T02:10:31.542797+0000","last_active":"2023-11-06T02:10:31.923073+0000","last_peered":"2023-11-06T02:10:31.923073+0000","last_clean":"2023-11-06T02:10:31.923073+0000","last_became_active":"2023-11-06T02:10:31.542455+0000","last_became_peered":"2023-11-06T02:10:31.542455+0000","last_unstale":"2023-11-06T02:10:31.923073+0000","last_undegraded":"2023-11-06T02:10:31.923073+0000","last_fullsized":"2023-11-06T02:10:31.923073+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_clean_scrub_stamp":"2023-11-06T02:10:30.509051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:23:34.675851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.942510+0000","last_change":"2023-11-06T02:10:31.927455+0000","last_active":"2023-11-06T02:10:31.942510+0000","last_peered":"2023-11-06T02:10:31.942510+0000","last_clean":"2023-11-06T02:10:31.942510+0000","last_became_active":"2023-11-06T02:10:31.926732+0000","last_became_peered":"2023-11-06T02:10:31.926732+0000","last_unstale":"2023-11-06T02:10:31.942510+0000","last_undegraded":"2023-11-06T02:10:31.942510+0000","last_fullsized":"2023-11-06T02:10:31.942510+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:31:52.575652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.935157+0000","last_change":"2023-11-06T02:10:31.562799+0000","last_active":"2023-11-06T02:10:31.935157+0000","last_peered":"2023-11-06T02:10:31.935157+0000","last_clean":"2023-11-06T02:10:31.935157+0000","last_became_active":"2023-11-06T02:10:31.562086+0000","last_became_peered":"2023-11-06T02:10:31.562086+0000","last_unstale":"2023-11-06T02:10:31.935157+0000","last_undegraded":"2023-11-06T02:10:31.935157+0000","last_fullsized":"2023-11-06T02:10:31.935157+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_clean_scrub_stamp":"2023-11-06T02:10:30.509051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:34:10.278562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-06T02:10:32.927913+0000","last_change":"2023-11-06T02:10:31.949670+0000","last_active":"2023-11-06T02:10:32.927913+0000","last_peered":"2023-11-06T02:10:32.927913+0000","last_clean":"2023-11-06T02:10:32.927913+0000","last_became_active":"2023-11-06T02:10:31.946195+0000","last_became_peered":"2023-11-06T02:10:31.946195+0000","last_unstale":"2023-11-06T02:10:32.927913+0000","last_undegraded":"2023-11-06T02:10:32.927913+0000","last_fullsized":"2023-11-06T02:10:32.927913+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:06:02.709833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.919435+0000","last_change":"2023-11-06T02:10:31.545722+0000","last_active":"2023-11-06T02:10:31.919435+0000","last_peered":"2023-11-06T02:10:31.919435+0000","last_clean":"2023-11-06T02:10:31.919435+0000","last_became_active":"2023-11-06T02:10:31.545023+0000","last_became_peered":"2023-11-06T02:10:31.545023+0000","last_unstale":"2023-11-06T02:10:31.919435+0000","last_undegraded":"2023-11-06T02:10:31.919435+0000","last_fullsized":"2023-11-06T02:10:31.919435+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_clean_scrub_stamp":"2023-11-06T02:10:30.509051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:03:30.064240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.948006+0000","last_change":"2023-11-06T02:10:31.931764+0000","last_active":"2023-11-06T02:10:31.948006+0000","last_peered":"2023-11-06T02:10:31.948006+0000","last_clean":"2023-11-06T02:10:31.948006+0000","last_became_active":"2023-11-06T02:10:31.931286+0000","last_became_peered":"2023-11-06T02:10:31.931286+0000","last_unstale":"2023-11-06T02:10:31.948006+0000","last_undegraded":"2023-11-06T02:10:31.948006+0000","last_fullsized":"2023-11-06T02:10:31.948006+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:31:04.831175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"50'3","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.968264+0000","last_change":"2023-11-06T02:10:31.558718+0000","last_active":"2023-11-06T02:10:31.968264+0000","last_peered":"2023-11-06T02:10:31.968264+0000","last_clean":"2023-11-06T02:10:31.968264+0000","last_became_active":"2023-11-06T02:10:31.558194+0000","last_became_peered":"2023-11-06T02:10:31.558194+0000","last_unstale":"2023-11-06T02:10:31.968264+0000","last_undegraded":"2023-11-06T02:10:31.968264+0000","last_fullsized":"2023-11-06T02:10:31.968264+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_clean_scrub_stamp":"2023-11-06T02:10:30.509051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:29:16.337946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.957138+0000","last_change":"2023-11-06T02:10:31.941420+0000","last_active":"2023-11-06T02:10:31.957138+0000","last_peered":"2023-11-06T02:10:31.957138+0000","last_clean":"2023-11-06T02:10:31.957138+0000","last_became_active":"2023-11-06T02:10:31.940665+0000","last_became_peered":"2023-11-06T02:10:31.940665+0000","last_unstale":"2023-11-06T02:10:31.957138+0000","last_undegraded":"2023-11-06T02:10:31.957138+0000","last_fullsized":"2023-11-06T02:10:31.957138+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:13:48.613748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"50'6","reported_seq":16,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.952574+0000","last_change":"2023-11-06T02:10:31.553451+0000","last_active":"2023-11-06T02:10:31.952574+0000","last_peered":"2023-11-06T02:10:31.952574+0000","last_clean":"2023-11-06T02:10:31.952574+0000","last_became_active":"2023-11-06T02:10:31.552888+0000","last_became_peered":"2023-11-06T02:10:31.552888+0000","last_unstale":"2023-11-06T02:10:31.952574+0000","last_undegraded":"2023-11-06T02:10:31.952574+0000","last_fullsized":"2023-11-06T02:10:31.952574+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_clean_scrub_stamp":"2023-11-06T02:10:30.509051+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:42:43.255482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-06T02:10:32.927970+0000","last_change":"2023-11-06T02:10:31.963637+0000","last_active":"2023-11-06T02:10:32.927970+0000","last_peered":"2023-11-06T02:10:32.927970+0000","last_clean":"2023-11-06T02:10:32.927970+0000","last_became_active":"2023-11-06T02:10:31.962812+0000","last_became_peered":"2023-11-06T02:10:31.962812+0000","last_unstale":"2023-11-06T02:10:32.927970+0000","last_undegraded":"2023-11-06T02:10:32.927970+0000","last_fullsized":"2023-11-06T02:10:32.927970+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:13:48.613748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"50'2","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.967511+0000","last_change":"2023-11-06T02:10:31.551770+0000","last_active":"2023-11-06T02:10:31.967511+0000","last_peered":"2023-11-06T02:10:31.967511+0000","last_clean":"2023-11-06T02:10:31.967511+0000","last_became_active":"2023-11-06T02:10:31.551322+0000","last_became_peered":"2023-11-06T02:10:31.551322+0000","last_unstale":"2023-11-06T02:10:31.967511+0000","last_undegraded":"2023-11-06T02:10:31.967511+0000","last_fullsized":"2023-11-06T02:10:31.967511+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_clean_scrub_stamp":"2023-11-06T02:10:30.509051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:46.288303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.950355+0000","last_change":"2023-11-06T02:10:31.938632+0000","last_active":"2023-11-06T02:10:31.950355+0000","last_peered":"2023-11-06T02:10:31.950355+0000","last_clean":"2023-11-06T02:10:31.950355+0000","last_became_active":"2023-11-06T02:10:31.938130+0000","last_became_peered":"2023-11-06T02:10:31.938130+0000","last_unstale":"2023-11-06T02:10:31.950355+0000","last_undegraded":"2023-11-06T02:10:31.950355+0000","last_fullsized":"2023-11-06T02:10:31.950355+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:54:44.600666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-06T02:10:32.926132+0000","last_change":"2023-11-06T02:10:31.546238+0000","last_active":"2023-11-06T02:10:32.926132+0000","last_peered":"2023-11-06T02:10:32.926132+0000","last_clean":"2023-11-06T02:10:32.926132+0000","last_became_active":"2023-11-06T02:10:31.545410+0000","last_became_peered":"2023-11-06T02:10:31.545410+0000","last_unstale":"2023-11-06T02:10:32.926132+0000","last_undegraded":"2023-11-06T02:10:32.926132+0000","last_fullsized":"2023-11-06T02:10:32.926132+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_clean_scrub_stamp":"2023-11-06T02:10:30.509051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:15:54.225385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.947030+0000","last_change":"2023-11-06T02:10:31.940225+0000","last_active":"2023-11-06T02:10:31.947030+0000","last_peered":"2023-11-06T02:10:31.947030+0000","last_clean":"2023-11-06T02:10:31.947030+0000","last_became_active":"2023-11-06T02:10:31.939963+0000","last_became_peered":"2023-11-06T02:10:31.939963+0000","last_unstale":"2023-11-06T02:10:31.947030+0000","last_undegraded":"2023-11-06T02:10:31.947030+0000","last_fullsized":"2023-11-06T02:10:31.947030+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:13:36.170486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.933014+0000","last_change":"2023-11-06T02:10:31.568369+0000","last_active":"2023-11-06T02:10:31.933014+0000","last_peered":"2023-11-06T02:10:31.933014+0000","last_clean":"2023-11-06T02:10:31.933014+0000","last_became_active":"2023-11-06T02:10:31.567749+0000","last_became_peered":"2023-11-06T02:10:31.567749+0000","last_unstale":"2023-11-06T02:10:31.933014+0000","last_undegraded":"2023-11-06T02:10:31.933014+0000","last_fullsized":"2023-11-06T02:10:31.933014+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_clean_scrub_stamp":"2023-11-06T02:10:30.509051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:45:05.156310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.961860+0000","last_change":"2023-11-06T02:10:31.961074+0000","last_active":"2023-11-06T02:10:31.961860+0000","last_peered":"2023-11-06T02:10:31.961860+0000","last_clean":"2023-11-06T02:10:31.961860+0000","last_became_active":"2023-11-06T02:10:31.960783+0000","last_became_peered":"2023-11-06T02:10:31.960783+0000","last_unstale":"2023-11-06T02:10:31.961860+0000","last_undegraded":"2023-11-06T02:10:31.961860+0000","last_fullsized":"2023-11-06T02:10:31.961860+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:06:35.725675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"50'8","reported_seq":18,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.946933+0000","last_change":"2023-11-06T02:10:31.552065+0000","last_active":"2023-11-06T02:10:31.946933+0000","last_peered":"2023-11-06T02:10:31.946933+0000","last_clean":"2023-11-06T02:10:31.946933+0000","last_became_active":"2023-11-06T02:10:31.551688+0000","last_became_peered":"2023-11-06T02:10:31.551688+0000","last_unstale":"2023-11-06T02:10:31.946933+0000","last_undegraded":"2023-11-06T02:10:31.946933+0000","last_fullsized":"2023-11-06T02:10:31.946933+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_clean_scrub_stamp":"2023-11-06T02:10:30.509051+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:28:32.737572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.969597+0000","last_change":"2023-11-06T02:10:31.968813+0000","last_active":"2023-11-06T02:10:31.969597+0000","last_peered":"2023-11-06T02:10:31.969597+0000","last_clean":"2023-11-06T02:10:31.969597+0000","last_became_active":"2023-11-06T02:10:31.968341+0000","last_became_peered":"2023-11-06T02:10:31.968341+0000","last_unstale":"2023-11-06T02:10:31.969597+0000","last_undegraded":"2023-11-06T02:10:31.969597+0000","last_fullsized":"2023-11-06T02:10:31.969597+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:00:32.003705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"50'1","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.950535+0000","last_change":"2023-11-06T02:10:31.554140+0000","last_active":"2023-11-06T02:10:31.950535+0000","last_peered":"2023-11-06T02:10:31.950535+0000","last_clean":"2023-11-06T02:10:31.950535+0000","last_became_active":"2023-11-06T02:10:31.553822+0000","last_became_peered":"2023-11-06T02:10:31.553822+0000","last_unstale":"2023-11-06T02:10:31.950535+0000","last_undegraded":"2023-11-06T02:10:31.950535+0000","last_fullsized":"2023-11-06T02:10:31.950535+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_clean_scrub_stamp":"2023-11-06T02:10:30.509051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:37:07.719491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.937435+0000","last_change":"2023-11-06T02:10:31.935200+0000","last_active":"2023-11-06T02:10:31.937435+0000","last_peered":"2023-11-06T02:10:31.937435+0000","last_clean":"2023-11-06T02:10:31.937435+0000","last_became_active":"2023-11-06T02:10:31.934933+0000","last_became_peered":"2023-11-06T02:10:31.934933+0000","last_unstale":"2023-11-06T02:10:31.937435+0000","last_undegraded":"2023-11-06T02:10:31.937435+0000","last_fullsized":"2023-11-06T02:10:31.937435+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:23:35.679574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"50'2","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-06T02:10:32.922423+0000","last_change":"2023-11-06T02:10:31.549129+0000","last_active":"2023-11-06T02:10:32.922423+0000","last_peered":"2023-11-06T02:10:32.922423+0000","last_clean":"2023-11-06T02:10:32.922423+0000","last_became_active":"2023-11-06T02:10:31.548717+0000","last_became_peered":"2023-11-06T02:10:31.548717+0000","last_unstale":"2023-11-06T02:10:32.922423+0000","last_undegraded":"2023-11-06T02:10:32.922423+0000","last_fullsized":"2023-11-06T02:10:32.922423+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_clean_scrub_stamp":"2023-11-06T02:10:30.509051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:09:08.748403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.966852+0000","last_change":"2023-11-06T02:10:31.957602+0000","last_active":"2023-11-06T02:10:31.966852+0000","last_peered":"2023-11-06T02:10:31.966852+0000","last_clean":"2023-11-06T02:10:31.966852+0000","last_became_active":"2023-11-06T02:10:31.955757+0000","last_became_peered":"2023-11-06T02:10:31.955757+0000","last_unstale":"2023-11-06T02:10:31.966852+0000","last_undegraded":"2023-11-06T02:10:31.966852+0000","last_fullsized":"2023-11-06T02:10:31.966852+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:32.392045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"50'1","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-06T02:10:32.926866+0000","last_change":"2023-11-06T02:10:31.550899+0000","last_active":"2023-11-06T02:10:32.926866+0000","last_peered":"2023-11-06T02:10:32.926866+0000","last_clean":"2023-11-06T02:10:32.926866+0000","last_became_active":"2023-11-06T02:10:31.550663+0000","last_became_peered":"2023-11-06T02:10:31.550663+0000","last_unstale":"2023-11-06T02:10:32.926866+0000","last_undegraded":"2023-11-06T02:10:32.926866+0000","last_fullsized":"2023-11-06T02:10:32.926866+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_clean_scrub_stamp":"2023-11-06T02:10:30.509051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:13:49.387379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.962486+0000","last_change":"2023-11-06T02:10:31.961329+0000","last_active":"2023-11-06T02:10:31.962486+0000","last_peered":"2023-11-06T02:10:31.962486+0000","last_clean":"2023-11-06T02:10:31.962486+0000","last_became_active":"2023-11-06T02:10:31.960981+0000","last_became_peered":"2023-11-06T02:10:31.960981+0000","last_unstale":"2023-11-06T02:10:31.962486+0000","last_undegraded":"2023-11-06T02:10:31.962486+0000","last_fullsized":"2023-11-06T02:10:31.962486+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:47.292026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-06T02:10:32.926624+0000","last_change":"2023-11-06T02:10:31.555335+0000","last_active":"2023-11-06T02:10:32.926624+0000","last_peered":"2023-11-06T02:10:32.926624+0000","last_clean":"2023-11-06T02:10:32.926624+0000","last_became_active":"2023-11-06T02:10:31.554525+0000","last_became_peered":"2023-11-06T02:10:31.554525+0000","last_unstale":"2023-11-06T02:10:32.926624+0000","last_undegraded":"2023-11-06T02:10:32.926624+0000","last_fullsized":"2023-11-06T02:10:32.926624+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_clean_scrub_stamp":"2023-11-06T02:10:30.509051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:28:25.243365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-06T02:10:32.928217+0000","last_change":"2023-11-06T02:10:31.950231+0000","last_active":"2023-11-06T02:10:32.928217+0000","last_peered":"2023-11-06T02:10:32.928217+0000","last_clean":"2023-11-06T02:10:32.928217+0000","last_became_active":"2023-11-06T02:10:31.946970+0000","last_became_peered":"2023-11-06T02:10:31.946970+0000","last_unstale":"2023-11-06T02:10:32.928217+0000","last_undegraded":"2023-11-06T02:10:32.928217+0000","last_fullsized":"2023-11-06T02:10:32.928217+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:28:33.741295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"50'2","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.969050+0000","last_change":"2023-11-06T02:10:31.551494+0000","last_active":"2023-11-06T02:10:31.969050+0000","last_peered":"2023-11-06T02:10:31.969050+0000","last_clean":"2023-11-06T02:10:31.969050+0000","last_became_active":"2023-11-06T02:10:31.551122+0000","last_became_peered":"2023-11-06T02:10:31.551122+0000","last_unstale":"2023-11-06T02:10:31.969050+0000","last_undegraded":"2023-11-06T02:10:31.969050+0000","last_fullsized":"2023-11-06T02:10:31.969050+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_clean_scrub_stamp":"2023-11-06T02:10:30.509051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:39:39.472153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-06T02:10:32.926624+0000","last_change":"2023-11-06T02:10:31.957394+0000","last_active":"2023-11-06T02:10:32.926624+0000","last_peered":"2023-11-06T02:10:32.926624+0000","last_clean":"2023-11-06T02:10:32.926624+0000","last_became_active":"2023-11-06T02:10:31.956320+0000","last_became_peered":"2023-11-06T02:10:31.956320+0000","last_unstale":"2023-11-06T02:10:32.926624+0000","last_undegraded":"2023-11-06T02:10:32.926624+0000","last_fullsized":"2023-11-06T02:10:32.926624+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:29:17.341669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.911597+0000","last_change":"2023-11-06T02:10:31.555740+0000","last_active":"2023-11-06T02:10:31.911597+0000","last_peered":"2023-11-06T02:10:31.911597+0000","last_clean":"2023-11-06T02:10:31.911597+0000","last_became_active":"2023-11-06T02:10:31.554890+0000","last_became_peered":"2023-11-06T02:10:31.554890+0000","last_unstale":"2023-11-06T02:10:31.911597+0000","last_undegraded":"2023-11-06T02:10:31.911597+0000","last_fullsized":"2023-11-06T02:10:31.911597+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_clean_scrub_stamp":"2023-11-06T02:10:30.509051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:26:12.509993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.947890+0000","last_change":"2023-11-06T02:10:31.931516+0000","last_active":"2023-11-06T02:10:31.947890+0000","last_peered":"2023-11-06T02:10:31.947890+0000","last_clean":"2023-11-06T02:10:31.947890+0000","last_became_active":"2023-11-06T02:10:31.931235+0000","last_became_peered":"2023-11-06T02:10:31.931235+0000","last_unstale":"2023-11-06T02:10:31.947890+0000","last_undegraded":"2023-11-06T02:10:31.947890+0000","last_fullsized":"2023-11-06T02:10:31.947890+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:35:16.923535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.921399+0000","last_change":"2023-11-06T02:10:31.546787+0000","last_active":"2023-11-06T02:10:31.921399+0000","last_peered":"2023-11-06T02:10:31.921399+0000","last_clean":"2023-11-06T02:10:31.921399+0000","last_became_active":"2023-11-06T02:10:31.546100+0000","last_became_peered":"2023-11-06T02:10:31.546100+0000","last_unstale":"2023-11-06T02:10:31.921399+0000","last_undegraded":"2023-11-06T02:10:31.921399+0000","last_fullsized":"2023-11-06T02:10:31.921399+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_clean_scrub_stamp":"2023-11-06T02:10:30.509051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:07:02.686107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.934832+0000","last_change":"2023-11-06T02:10:31.933155+0000","last_active":"2023-11-06T02:10:31.934832+0000","last_peered":"2023-11-06T02:10:31.934832+0000","last_clean":"2023-11-06T02:10:31.934832+0000","last_became_active":"2023-11-06T02:10:31.932760+0000","last_became_peered":"2023-11-06T02:10:31.932760+0000","last_unstale":"2023-11-06T02:10:31.934832+0000","last_undegraded":"2023-11-06T02:10:31.934832+0000","last_fullsized":"2023-11-06T02:10:31.934832+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:54:03.567766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-06T02:10:32.921702+0000","last_change":"2023-11-06T02:10:31.555022+0000","last_active":"2023-11-06T02:10:32.921702+0000","last_peered":"2023-11-06T02:10:32.921702+0000","last_clean":"2023-11-06T02:10:32.921702+0000","last_became_active":"2023-11-06T02:10:31.554474+0000","last_became_peered":"2023-11-06T02:10:31.554474+0000","last_unstale":"2023-11-06T02:10:32.921702+0000","last_undegraded":"2023-11-06T02:10:32.921702+0000","last_fullsized":"2023-11-06T02:10:32.921702+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_clean_scrub_stamp":"2023-11-06T02:10:30.509051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:15:54.225385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-06T02:10:32.926289+0000","last_change":"2023-11-06T02:10:31.948540+0000","last_active":"2023-11-06T02:10:32.926289+0000","last_peered":"2023-11-06T02:10:32.926289+0000","last_clean":"2023-11-06T02:10:32.926289+0000","last_became_active":"2023-11-06T02:10:31.947789+0000","last_became_peered":"2023-11-06T02:10:31.947789+0000","last_unstale":"2023-11-06T02:10:32.926289+0000","last_undegraded":"2023-11-06T02:10:32.926289+0000","last_fullsized":"2023-11-06T02:10:32.926289+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:16:18.915793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.921745+0000","last_change":"2023-11-06T02:10:31.553943+0000","last_active":"2023-11-06T02:10:31.921745+0000","last_peered":"2023-11-06T02:10:31.921745+0000","last_clean":"2023-11-06T02:10:31.921745+0000","last_became_active":"2023-11-06T02:10:31.553497+0000","last_became_peered":"2023-11-06T02:10:31.553497+0000","last_unstale":"2023-11-06T02:10:31.921745+0000","last_undegraded":"2023-11-06T02:10:31.921745+0000","last_fullsized":"2023-11-06T02:10:31.921745+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_clean_scrub_stamp":"2023-11-06T02:10:30.509051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:03:30.064240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":191,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.543938+0000","last_change":"2023-11-06T02:08:20.523427+0000","last_active":"2023-11-06T02:10:31.543938+0000","last_peered":"2023-11-06T02:10:31.543938+0000","last_clean":"2023-11-06T02:10:31.543938+0000","last_became_active":"2023-11-06T02:08:20.521089+0000","last_became_peered":"2023-11-06T02:08:20.521089+0000","last_unstale":"2023-11-06T02:10:31.543938+0000","last_undegraded":"2023-11-06T02:10:31.543938+0000","last_fullsized":"2023-11-06T02:10:31.543938+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:19.252348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:19.252348+0000","last_clean_scrub_stamp":"2023-11-06T02:08:19.252348+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:30:15.566397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":39,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-06T02:10:32.928056+0000","last_change":"2023-11-06T02:08:26.059502+0000","last_active":"2023-11-06T02:10:32.928056+0000","last_peered":"2023-11-06T02:10:32.928056+0000","last_clean":"2023-11-06T02:10:32.928056+0000","last_became_active":"2023-11-06T02:08:26.058612+0000","last_became_peered":"2023-11-06T02:08:26.058612+0000","last_unstale":"2023-11-06T02:10:32.928056+0000","last_undegraded":"2023-11-06T02:10:32.928056+0000","last_fullsized":"2023-11-06T02:10:32.928056+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:10.626021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.938119+0000","last_change":"2023-11-06T02:10:31.936110+0000","last_active":"2023-11-06T02:10:31.938119+0000","last_peered":"2023-11-06T02:10:31.938119+0000","last_clean":"2023-11-06T02:10:31.938119+0000","last_became_active":"2023-11-06T02:10:31.935818+0000","last_became_peered":"2023-11-06T02:10:31.935818+0000","last_unstale":"2023-11-06T02:10:31.938119+0000","last_undegraded":"2023-11-06T02:10:31.938119+0000","last_fullsized":"2023-11-06T02:10:31.938119+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:51:59.093456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.935168+0000","last_change":"2023-11-06T02:10:31.557657+0000","last_active":"2023-11-06T02:10:31.935168+0000","last_peered":"2023-11-06T02:10:31.935168+0000","last_clean":"2023-11-06T02:10:31.935168+0000","last_became_active":"2023-11-06T02:10:31.557039+0000","last_became_peered":"2023-11-06T02:10:31.557039+0000","last_unstale":"2023-11-06T02:10:31.935168+0000","last_undegraded":"2023-11-06T02:10:31.935168+0000","last_fullsized":"2023-11-06T02:10:31.935168+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_clean_scrub_stamp":"2023-11-06T02:10:30.509051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:50:32.167116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":38,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.543715+0000","last_change":"2023-11-06T02:08:26.060406+0000","last_active":"2023-11-06T02:10:31.543715+0000","last_peered":"2023-11-06T02:10:31.543715+0000","last_clean":"2023-11-06T02:10:31.543715+0000","last_became_active":"2023-11-06T02:08:26.059262+0000","last_became_peered":"2023-11-06T02:08:26.059262+0000","last_unstale":"2023-11-06T02:10:31.543715+0000","last_undegraded":"2023-11-06T02:10:31.543715+0000","last_fullsized":"2023-11-06T02:10:31.543715+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:10.626021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.936129+0000","last_change":"2023-11-06T02:10:31.927287+0000","last_active":"2023-11-06T02:10:31.936129+0000","last_peered":"2023-11-06T02:10:31.936129+0000","last_clean":"2023-11-06T02:10:31.936129+0000","last_became_active":"2023-11-06T02:10:31.927001+0000","last_became_peered":"2023-11-06T02:10:31.927001+0000","last_unstale":"2023-11-06T02:10:31.936129+0000","last_undegraded":"2023-11-06T02:10:31.936129+0000","last_fullsized":"2023-11-06T02:10:31.936129+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:57:03.504241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.922810+0000","last_change":"2023-11-06T02:10:31.532950+0000","last_active":"2023-11-06T02:10:31.922810+0000","last_peered":"2023-11-06T02:10:31.922810+0000","last_clean":"2023-11-06T02:10:31.922810+0000","last_became_active":"2023-11-06T02:10:31.532290+0000","last_became_peered":"2023-11-06T02:10:31.532290+0000","last_unstale":"2023-11-06T02:10:31.922810+0000","last_undegraded":"2023-11-06T02:10:31.922810+0000","last_fullsized":"2023-11-06T02:10:31.922810+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_clean_scrub_stamp":"2023-11-06T02:10:30.509051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:07.035669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":40,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-06T02:10:32.921724+0000","last_change":"2023-11-06T02:08:26.559477+0000","last_active":"2023-11-06T02:10:32.921724+0000","last_peered":"2023-11-06T02:10:32.921724+0000","last_clean":"2023-11-06T02:10:32.921724+0000","last_became_active":"2023-11-06T02:08:26.559202+0000","last_became_peered":"2023-11-06T02:08:26.559202+0000","last_unstale":"2023-11-06T02:10:32.921724+0000","last_undegraded":"2023-11-06T02:10:32.921724+0000","last_fullsized":"2023-11-06T02:10:32.921724+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:55:04.552404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-06T02:10:32.927313+0000","last_change":"2023-11-06T02:10:31.970063+0000","last_active":"2023-11-06T02:10:32.927313+0000","last_peered":"2023-11-06T02:10:32.927313+0000","last_clean":"2023-11-06T02:10:32.927313+0000","last_became_active":"2023-11-06T02:10:31.966656+0000","last_became_peered":"2023-11-06T02:10:31.966656+0000","last_unstale":"2023-11-06T02:10:32.927313+0000","last_undegraded":"2023-11-06T02:10:32.927313+0000","last_fullsized":"2023-11-06T02:10:32.927313+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:31:52.575652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"50'2","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-06T02:10:32.926861+0000","last_change":"2023-11-06T02:10:31.548283+0000","last_active":"2023-11-06T02:10:32.926861+0000","last_peered":"2023-11-06T02:10:32.926861+0000","last_clean":"2023-11-06T02:10:32.926861+0000","last_became_active":"2023-11-06T02:10:31.548058+0000","last_became_peered":"2023-11-06T02:10:31.548058+0000","last_unstale":"2023-11-06T02:10:32.926861+0000","last_undegraded":"2023-11-06T02:10:32.926861+0000","last_fullsized":"2023-11-06T02:10:32.926861+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_clean_scrub_stamp":"2023-11-06T02:10:30.509051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:54:46.730754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":39,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-06T02:10:32.927747+0000","last_change":"2023-11-06T02:08:26.059546+0000","last_active":"2023-11-06T02:10:32.927747+0000","last_peered":"2023-11-06T02:10:32.927747+0000","last_clean":"2023-11-06T02:10:32.927747+0000","last_became_active":"2023-11-06T02:08:26.058806+0000","last_became_peered":"2023-11-06T02:08:26.058806+0000","last_unstale":"2023-11-06T02:10:32.927747+0000","last_undegraded":"2023-11-06T02:10:32.927747+0000","last_fullsized":"2023-11-06T02:10:32.927747+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:23.468510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.950288+0000","last_change":"2023-11-06T02:10:31.942711+0000","last_active":"2023-11-06T02:10:31.950288+0000","last_peered":"2023-11-06T02:10:31.950288+0000","last_clean":"2023-11-06T02:10:31.950288+0000","last_became_active":"2023-11-06T02:10:31.942440+0000","last_became_peered":"2023-11-06T02:10:31.942440+0000","last_unstale":"2023-11-06T02:10:31.950288+0000","last_undegraded":"2023-11-06T02:10:31.950288+0000","last_fullsized":"2023-11-06T02:10:31.950288+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:41:22.735337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.918807+0000","last_change":"2023-11-06T02:10:31.556959+0000","last_active":"2023-11-06T02:10:31.918807+0000","last_peered":"2023-11-06T02:10:31.918807+0000","last_clean":"2023-11-06T02:10:31.918807+0000","last_became_active":"2023-11-06T02:10:31.556286+0000","last_became_peered":"2023-11-06T02:10:31.556286+0000","last_unstale":"2023-11-06T02:10:31.918807+0000","last_undegraded":"2023-11-06T02:10:31.918807+0000","last_fullsized":"2023-11-06T02:10:31.918807+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_clean_scrub_stamp":"2023-11-06T02:10:30.509051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:07:02.686107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":36,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.928429+0000","last_change":"2023-11-06T02:08:26.436463+0000","last_active":"2023-11-06T02:10:31.928429+0000","last_peered":"2023-11-06T02:10:31.928429+0000","last_clean":"2023-11-06T02:10:31.928429+0000","last_became_active":"2023-11-06T02:08:26.436075+0000","last_became_peered":"2023-11-06T02:08:26.436075+0000","last_unstale":"2023-11-06T02:10:31.928429+0000","last_undegraded":"2023-11-06T02:10:31.928429+0000","last_fullsized":"2023-11-06T02:10:31.928429+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:23.468510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.946216+0000","last_change":"2023-11-06T02:10:31.931904+0000","last_active":"2023-11-06T02:10:31.946216+0000","last_peered":"2023-11-06T02:10:31.946216+0000","last_clean":"2023-11-06T02:10:31.946216+0000","last_became_active":"2023-11-06T02:10:31.931454+0000","last_became_peered":"2023-11-06T02:10:31.931454+0000","last_unstale":"2023-11-06T02:10:31.946216+0000","last_undegraded":"2023-11-06T02:10:31.946216+0000","last_fullsized":"2023-11-06T02:10:31.946216+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:41:22.735337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-06T02:10:32.927633+0000","last_change":"2023-11-06T02:10:31.555697+0000","last_active":"2023-11-06T02:10:32.927633+0000","last_peered":"2023-11-06T02:10:32.927633+0000","last_clean":"2023-11-06T02:10:32.927633+0000","last_became_active":"2023-11-06T02:10:31.555004+0000","last_became_peered":"2023-11-06T02:10:31.555004+0000","last_unstale":"2023-11-06T02:10:32.927633+0000","last_undegraded":"2023-11-06T02:10:32.927633+0000","last_fullsized":"2023-11-06T02:10:32.927633+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_clean_scrub_stamp":"2023-11-06T02:10:30.509051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:07.035669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":38,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.935384+0000","last_change":"2023-11-06T02:08:26.436607+0000","last_active":"2023-11-06T02:10:31.935384+0000","last_peered":"2023-11-06T02:10:31.935384+0000","last_clean":"2023-11-06T02:10:31.935384+0000","last_became_active":"2023-11-06T02:08:26.435924+0000","last_became_peered":"2023-11-06T02:08:26.435924+0000","last_unstale":"2023-11-06T02:10:31.935384+0000","last_undegraded":"2023-11-06T02:10:31.935384+0000","last_fullsized":"2023-11-06T02:10:31.935384+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:23.468510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.947879+0000","last_change":"2023-11-06T02:10:31.931111+0000","last_active":"2023-11-06T02:10:31.947879+0000","last_peered":"2023-11-06T02:10:31.947879+0000","last_clean":"2023-11-06T02:10:31.947879+0000","last_became_active":"2023-11-06T02:10:31.930621+0000","last_became_peered":"2023-11-06T02:10:31.930621+0000","last_unstale":"2023-11-06T02:10:31.947879+0000","last_undegraded":"2023-11-06T02:10:31.947879+0000","last_fullsized":"2023-11-06T02:10:31.947879+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:00:32.003705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.912533+0000","last_change":"2023-11-06T02:10:31.554303+0000","last_active":"2023-11-06T02:10:31.912533+0000","last_peered":"2023-11-06T02:10:31.912533+0000","last_clean":"2023-11-06T02:10:31.912533+0000","last_became_active":"2023-11-06T02:10:31.553183+0000","last_became_peered":"2023-11-06T02:10:31.553183+0000","last_unstale":"2023-11-06T02:10:31.912533+0000","last_undegraded":"2023-11-06T02:10:31.912533+0000","last_fullsized":"2023-11-06T02:10:31.912533+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_clean_scrub_stamp":"2023-11-06T02:10:30.509051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:54:02.564043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":35,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.549282+0000","last_change":"2023-11-06T02:08:26.057076+0000","last_active":"2023-11-06T02:10:31.549282+0000","last_peered":"2023-11-06T02:10:31.549282+0000","last_clean":"2023-11-06T02:10:31.549282+0000","last_became_active":"2023-11-06T02:08:26.056419+0000","last_became_peered":"2023-11-06T02:08:26.056419+0000","last_unstale":"2023-11-06T02:10:31.549282+0000","last_undegraded":"2023-11-06T02:10:31.549282+0000","last_fullsized":"2023-11-06T02:10:31.549282+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:10.626021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-06T02:10:32.928078+0000","last_change":"2023-11-06T02:10:31.970623+0000","last_active":"2023-11-06T02:10:32.928078+0000","last_peered":"2023-11-06T02:10:32.928078+0000","last_clean":"2023-11-06T02:10:32.928078+0000","last_became_active":"2023-11-06T02:10:31.969840+0000","last_became_peered":"2023-11-06T02:10:31.969840+0000","last_unstale":"2023-11-06T02:10:32.928078+0000","last_undegraded":"2023-11-06T02:10:32.928078+0000","last_fullsized":"2023-11-06T02:10:32.928078+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:00:38.993498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.912300+0000","last_change":"2023-11-06T02:10:31.553319+0000","last_active":"2023-11-06T02:10:31.912300+0000","last_peered":"2023-11-06T02:10:31.912300+0000","last_clean":"2023-11-06T02:10:31.912300+0000","last_became_active":"2023-11-06T02:10:31.552876+0000","last_became_peered":"2023-11-06T02:10:31.552876+0000","last_unstale":"2023-11-06T02:10:31.912300+0000","last_undegraded":"2023-11-06T02:10:31.912300+0000","last_fullsized":"2023-11-06T02:10:31.912300+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_clean_scrub_stamp":"2023-11-06T02:10:30.509051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:46.288303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":42,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.917574+0000","last_change":"2023-11-06T02:08:26.058596+0000","last_active":"2023-11-06T02:10:31.917574+0000","last_peered":"2023-11-06T02:10:31.917574+0000","last_clean":"2023-11-06T02:10:31.917574+0000","last_became_active":"2023-11-06T02:08:26.057015+0000","last_became_peered":"2023-11-06T02:08:26.057015+0000","last_unstale":"2023-11-06T02:10:31.917574+0000","last_undegraded":"2023-11-06T02:10:31.917574+0000","last_fullsized":"2023-11-06T02:10:31.917574+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:55:04.552404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574878,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28116,"kb_used_data":1164,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343724,"statfs":{"total":96636764160,"available":96607973376,"internally_reserved":0,"allocated":1191936,"data_stored":844490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:10:20 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.86799999999999999,"15min":0.85499999999999998},"min":{"1min":0.60499999999999998,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.421,"5min":1.506,"15min":1.506},"last":0.90200000000000002}]},{"osd":1,"last update":"Mon Nov 6 02:10:20 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.84199999999999997,"15min":0.84799999999999998},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.1399999999999999,"5min":1.161,"15min":1.161},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.90400000000000003,"15min":0.89400000000000002},"min":{"1min":0.76300000000000001,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.67800000000000005}]},{"osd":2,"last update":"Mon Nov 6 02:10:20 2023","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":0.96699999999999997,"15min":0.95799999999999996},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.99099999999999999,"15min":0.997},"min":{"1min":0.74399999999999999,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.3420000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.622}]},{"osd":3,"last update":"Mon Nov 6 02:10:20 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.875,"15min":0.875},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.083,"15min":1.079},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.526,"5min":1.526,"15min":1.526},"last":0.84699999999999998}]},{"osd":4,"last update":"Mon Nov 6 02:10:20 2023","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":0.96499999999999997,"15min":0.95999999999999996},"min":{"1min":0.70699999999999996,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.97299999999999998,"5min":1.03,"15min":1.04},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.2170000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":1.115}]},{"osd":5,"last update":"Mon Nov 6 02:10:20 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.093,"15min":1.083},"min":{"1min":0.89300000000000002,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":0.94699999999999995},{"interface":"front","average":{"1min":1.101,"5min":1.129,"15min":1.133},"min":{"1min":0.86599999999999999,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.3180000000000001,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":0.98799999999999999}]},{"osd":6,"last update":"Mon Nov 6 02:10:20 2023","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":1.2030000000000001,"15min":1.202},"min":{"1min":0.87,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.4930000000000001,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":1.0660000000000001},{"interface":"front","average":{"1min":1.133,"5min":1.153,"15min":1.1559999999999999},"min":{"1min":0.79500000000000004,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.421,"5min":1.74,"15min":1.74},"last":1.1719999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574877,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27536,"kb_used_data":584,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344304,"statfs":{"total":96636764160,"available":96608567296,"internally_reserved":0,"allocated":598016,"data_stored":254650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:10:22 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.84699999999999998,"15min":0.85399999999999998},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.0880000000000001,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.82299999999999995,"15min":0.81100000000000005},"min":{"1min":0.55300000000000005,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.52000000000000002}]},{"osd":1,"last update":"Mon Nov 6 02:10:22 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.86299999999999999,"15min":0.871},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.1100000000000001,"5min":1.179,"15min":1.179},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.876,"15min":0.874},"min":{"1min":0.60999999999999999,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.3200000000000001,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.98999999999999999}]},{"osd":2,"last update":"Mon Nov 6 02:10:22 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.88600000000000001,"15min":0.89400000000000002},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.087,"5min":1.173,"15min":1.173},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.92100000000000004,"15min":0.92700000000000005},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.1339999999999999,"5min":1.28,"15min":1.28},"last":0.72299999999999998}]},{"osd":3,"last update":"Mon Nov 6 02:10:22 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.95499999999999996,"15min":0.96599999999999997},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":1.05},{"interface":"front","average":{"1min":0.96599999999999997,"5min":0.99399999999999999,"15min":0.998},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.2310000000000001,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.752}]},{"osd":4,"last update":"Mon Nov 6 02:10:22 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.96499999999999997,"15min":0.98399999999999999},"min":{"1min":0.56799999999999995,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.1459999999999999,"5min":1.474,"15min":1.474},"last":1.097},{"interface":"front","average":{"1min":0.95799999999999996,"5min":1.0229999999999999,"15min":1.034},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.2989999999999999,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.88600000000000001}]},{"osd":5,"last update":"Mon Nov 6 02:10:22 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":1.0069999999999999,"15min":1.0209999999999999},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.2769999999999999,"5min":1.365,"15min":1.365},"last":0.79900000000000004},{"interface":"front","average":{"1min":1.03,"5min":1.0980000000000001,"15min":1.109},"min":{"1min":0.66900000000000004,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.234,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":1.145}]},{"osd":7,"last update":"Mon Nov 6 02:10:22 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.1060000000000001,"15min":1.1140000000000001},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.3460000000000001,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.98699999999999999,"5min":1.129,"15min":1.153},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.2,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.91800000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574878,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27528,"kb_used_data":576,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":589824,"data_stored":253998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:10:23 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.84499999999999997,"15min":0.83399999999999996},"min":{"1min":0.57399999999999995,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.381,"5min":1.381,"15min":1.381},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.57899999999999996,"5min":0.503,"15min":0.503},"max":{"1min":1.1739999999999999,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":1.3200000000000001}]},{"osd":1,"last update":"Mon Nov 6 02:10:23 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.85599999999999998,"15min":0.85199999999999998},"min":{"1min":0.64100000000000001,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.2190000000000001,"5min":1.331,"15min":1.331},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.82599999999999996,"15min":0.81999999999999995},"min":{"1min":0.56899999999999995,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.177,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":1.0289999999999999}]},{"osd":2,"last update":"Mon Nov 6 02:10:23 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.93600000000000005,"15min":0.94199999999999995},"min":{"1min":0.58999999999999997,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.1639999999999999,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.81200000000000006,"15min":0.79700000000000004},"min":{"1min":0.58999999999999997,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.2909999999999999,"5min":1.377,"15min":1.377},"last":1.0940000000000001}]},{"osd":3,"last update":"Mon Nov 6 02:10:23 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.91500000000000004,"15min":0.91800000000000004},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.143,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":1.0649999999999999},{"interface":"front","average":{"1min":0.96299999999999997,"5min":1.0569999999999999,"15min":1.073},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.349,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":1.141}]},{"osd":4,"last update":"Mon Nov 6 02:10:23 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.92400000000000004,"15min":0.91800000000000004},"min":{"1min":0.625,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.276,"5min":1.276,"15min":1.276},"last":0.98999999999999999},{"interface":"front","average":{"1min":1.01,"5min":1.024,"15min":1.026},"min":{"1min":0.68899999999999995,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.2370000000000001,"5min":1.554,"15min":1.554},"last":1.25}]},{"osd":6,"last update":"Mon Nov 6 02:10:23 2023","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":1.052,"15min":1.0620000000000001},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.409,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.95299999999999996},{"interface":"front","average":{"1min":1.002,"5min":1.0329999999999999,"15min":1.038},"min":{"1min":0.68400000000000005,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":1.171}]},{"osd":7,"last update":"Mon Nov 6 02:10:23 2023","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.0780000000000001,"15min":1.0760000000000001},"min":{"1min":0.71099999999999997,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.3140000000000001,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.125,"5min":1.149,"15min":1.153},"min":{"1min":0.80600000000000005,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.472,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":1.2250000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574878,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27532,"kb_used_data":580,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":593920,"data_stored":254580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:10:19 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.83299999999999996,"15min":0.82699999999999996},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.1679999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.876,"5min":0.79100000000000004,"15min":0.77700000000000002},"min":{"1min":0.65900000000000003,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.28,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.50600000000000001}]},{"osd":1,"last update":"Mon Nov 6 02:10:19 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.85799999999999998,"15min":0.84799999999999998},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.462,"5min":1.462,"15min":1.462},"last":1.024},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.90900000000000003,"15min":0.89700000000000002},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.91800000000000004}]},{"osd":2,"last update":"Mon Nov 6 02:10:19 2023","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.996,"15min":1.012},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.389,"5min":1.615,"15min":1.615},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.95399999999999996,"5min":0.93100000000000005,"15min":0.92700000000000005},"min":{"1min":0.70999999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.4379999999999999,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":0.997}]},{"osd":3,"last update":"Mon Nov 6 02:10:19 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.22,"5min":1.52,"15min":1.52},"last":0.86599999999999999},{"interface":"front","average":{"1min":1.012,"5min":1.0580000000000001,"15min":1.0660000000000001},"min":{"1min":0.69099999999999995,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.595,"5min":1.595,"15min":1.595},"last":0.95099999999999996}]},{"osd":5,"last update":"Mon Nov 6 02:10:19 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.046,"15min":1.0489999999999999},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.2949999999999999,"5min":1.377,"15min":1.377},"last":1.115},{"interface":"front","average":{"1min":0.94699999999999995,"5min":1.034,"15min":1.048},"min":{"1min":0.64300000000000002,"5min":0.629,"15min":0.629},"max":{"1min":1.2310000000000001,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":1.135}]},{"osd":6,"last update":"Mon Nov 6 02:10:19 2023","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":1.0760000000000001,"15min":1.0880000000000001},"min":{"1min":0.71999999999999997,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.2390000000000001,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":1.085},{"interface":"front","average":{"1min":1.123,"5min":1.1830000000000001,"15min":1.1930000000000001},"min":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":1.425,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":1.2070000000000001}]},{"osd":7,"last update":"Mon Nov 6 02:10:19 2023","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.089,"15min":1.0920000000000001},"min":{"1min":0.88500000000000001,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.3029999999999999,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":1.054},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.21,"15min":1.23},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.3140000000000001,"5min":1.7410000000000001,"15min":1.7410000000000001},"last":1.1759999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574877,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31640,"kb_used_data":592,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340200,"statfs":{"total":96636764160,"available":96604364800,"internally_reserved":0,"allocated":606208,"data_stored":255584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:10:22 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.63500000000000001,"15min":0.64300000000000002},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.80600000000000005,"5min":1.079,"15min":1.079},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.625,"15min":0.625},"min":{"1min":0.35999999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.50700000000000001}]},{"osd":1,"last update":"Mon Nov 6 02:10:22 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.61599999999999999,"15min":0.626},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.82499999999999996,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.72099999999999997,"15min":0.72899999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.94199999999999995,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.435}]},{"osd":2,"last update":"Mon Nov 6 02:10:22 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.72599999999999998,"15min":0.73199999999999998},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.0389999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.374},{"interface":"front","average":{"1min":0.745,"5min":0.80800000000000005,"15min":0.81799999999999995},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":1.405,"5min":1.405,"15min":1.405},"last":0.47199999999999998}]},{"osd":4,"last update":"Mon Nov 6 02:10:22 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.92400000000000004,"15min":0.92700000000000005},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.873},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.97899999999999998,"15min":0.99199999999999999},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.238,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.67800000000000005}]},{"osd":5,"last update":"Mon Nov 6 02:10:22 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":1.0109999999999999,"15min":1.0229999999999999},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.95699999999999996,"5min":1.0569999999999999,"15min":1.073},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.3580000000000001,"5min":2.21,"15min":2.21},"last":0.88800000000000001}]},{"osd":6,"last update":"Mon Nov 6 02:10:22 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":1.016,"15min":1.022},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.395,"5min":1.395,"15min":1.395},"last":0.76900000000000002},{"interface":"front","average":{"1min":1,"5min":1.0569999999999999,"15min":1.0669999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.2669999999999999,"5min":1.554,"15min":1.554},"last":0.71699999999999997}]},{"osd":7,"last update":"Mon Nov 6 02:10:22 2023","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":1.107,"15min":1.1240000000000001},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.4379999999999999,"5min":2.2829999999999999,"15min":2.2829999999999999},"last":1.0720000000000001},{"interface":"front","average":{"1min":1.048,"5min":1.095,"15min":1.103},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.3169999999999999,"5min":1.498,"15min":1.498},"last":0.92200000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574877,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31636,"kb_used_data":588,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340204,"statfs":{"total":96636764160,"available":96604368896,"internally_reserved":0,"allocated":602112,"data_stored":255562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:10:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.51000000000000001,"15min":0.502},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":1.079},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.54700000000000004,"15min":0.54100000000000004},"min":{"1min":0.316,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.90900000000000003,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.59899999999999998}]},{"osd":1,"last update":"Mon Nov 6 02:10:19 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.57399999999999995,"15min":0.56399999999999995},"min":{"1min":0.34799999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.0800000000000001,"5min":1.409,"15min":1.409},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.59499999999999997,"15min":0.57699999999999996},"min":{"1min":0.38100000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.083,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.53300000000000003}]},{"osd":3,"last update":"Mon Nov 6 02:10:19 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.61799999999999999,"15min":0.60399999999999998},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.121,"5min":1.155,"15min":1.155},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.65800000000000003,"15min":0.64700000000000002},"min":{"1min":0.36099999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.628}]},{"osd":4,"last update":"Mon Nov 6 02:10:19 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.81799999999999995,"15min":0.80500000000000005},"min":{"1min":0.56899999999999995,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.91200000000000003,"15min":0.91400000000000003},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.204,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":1.018}]},{"osd":5,"last update":"Mon Nov 6 02:10:19 2023","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.94799999999999995,"15min":0.94399999999999995},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.2390000000000001,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.96799999999999997},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.93999999999999995,"15min":0.93600000000000005},"min":{"1min":0.60999999999999999,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.4790000000000001,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.80800000000000005}]},{"osd":6,"last update":"Mon Nov 6 02:10:19 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.202,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":1.123},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.995,"15min":0.996},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.4199999999999999,"5min":1.677,"15min":1.677},"last":1.105}]},{"osd":7,"last update":"Mon Nov 6 02:10:19 2023","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.97899999999999998,"15min":0.97599999999999998},"min":{"1min":0.75900000000000001,"5min":0.622,"15min":0.622},"max":{"1min":1.2749999999999999,"5min":1.633,"15min":1.633},"last":1.1759999999999999},{"interface":"front","average":{"1min":1.048,"5min":1.0429999999999999,"15min":1.042},"min":{"1min":0.60799999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.345,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":1.0640000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574878,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31632,"kb_used_data":584,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340208,"statfs":{"total":96636764160,"available":96604372992,"internally_reserved":0,"allocated":598016,"data_stored":254158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:10:21 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.621,"15min":0.624},"min":{"1min":0.35699999999999998,"5min":0.224,"15min":0.224},"max":{"1min":0.85999999999999999,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65300000000000002,"15min":0.65200000000000002},"min":{"1min":0.443,"5min":0.315,"15min":0.315},"max":{"1min":0.90700000000000003,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.29199999999999998}]},{"osd":2,"last update":"Mon Nov 6 02:10:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.61599999999999999,"15min":0.62},"min":{"1min":0.34200000000000003,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.0109999999999999,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.504},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.73499999999999999,"15min":0.74099999999999999},"min":{"1min":0.39300000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.27400000000000002}]},{"osd":3,"last update":"Mon Nov 6 02:10:21 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.79500000000000004,"15min":0.80000000000000004},"min":{"1min":0.42699999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.98299999999999998,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.78900000000000003,"15min":0.80300000000000005},"min":{"1min":0.379,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.98799999999999999,"5min":1.8149999999999999,"15min":1.8149999999999999},"last":0.42699999999999999}]},{"osd":4,"last update":"Mon Nov 6 02:10:21 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.97799999999999998,"15min":0.98799999999999999},"min":{"1min":0.63200000000000001,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.1619999999999999,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.98299999999999998,"15min":0.98999999999999999},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.254,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.623}]},{"osd":5,"last update":"Mon Nov 6 02:10:21 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.96199999999999997,"15min":0.96499999999999997},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.196,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.58599999999999997},{"interface":"front","average":{"1min":1.024,"5min":1.0680000000000001,"15min":1.075},"min":{"1min":0.64600000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.4199999999999999,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.66500000000000004}]},{"osd":6,"last update":"Mon Nov 6 02:10:21 2023","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.087,"15min":1.097},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.296,"5min":1.742,"15min":1.742},"last":0.61199999999999999},{"interface":"front","average":{"1min":1.034,"5min":1.0680000000000001,"15min":1.0740000000000001},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.248,"5min":1.393,"15min":1.393},"last":0.70399999999999996}]},{"osd":7,"last update":"Mon Nov 6 02:10:21 2023","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":1.111,"15min":1.1180000000000001},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":1.081,"5min":1.145,"15min":1.155},"min":{"1min":0.70399999999999996,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.403,"5min":1.893,"15min":1.893},"last":0.76800000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574878,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32208,"kb_used_data":1160,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339632,"statfs":{"total":96636764160,"available":96603783168,"internally_reserved":0,"allocated":1187840,"data_stored":843403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 02:10:24 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.58799999999999997,"15min":0.59399999999999997},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.90100000000000002,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.432},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.63900000000000001,"15min":0.63700000000000001},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.39300000000000002}]},{"osd":2,"last update":"Mon Nov 6 02:10:24 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.65200000000000002,"15min":0.66000000000000003},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.34499999999999997,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.47599999999999998}]},{"osd":3,"last update":"Mon Nov 6 02:10:24 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.625,"15min":0.627},"min":{"1min":0.33100000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.429,"5min":1.429,"15min":1.429},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.73299999999999998,"15min":0.73099999999999998},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.61799999999999999}]},{"osd":4,"last update":"Mon Nov 6 02:10:24 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.83899999999999997,"15min":0.83099999999999996},"min":{"1min":0.55400000000000005,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.341,"5min":1.341,"15min":1.341},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.93200000000000005,"15min":0.93500000000000005},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.3839999999999999,"5min":1.387,"15min":1.387},"last":0.69099999999999995}]},{"osd":5,"last update":"Mon Nov 6 02:10:24 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.94899999999999995,"15min":0.94799999999999995},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.3140000000000001,"5min":1.339,"15min":1.339},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.88100000000000001,"15min":0.875},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.351,"5min":1.448,"15min":1.448},"last":0.77400000000000002}]},{"osd":6,"last update":"Mon Nov 6 02:10:24 2023","interfaces":[{"interface":"back","average":{"1min":1,"5min":1.0289999999999999,"15min":1.034},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.581,"5min":1.589,"15min":1.589},"last":0.83899999999999997},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.0109999999999999,"15min":1.0029999999999999},"min":{"1min":0.68899999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.5529999999999999,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.67600000000000005}]},{"osd":7,"last update":"Mon Nov 6 02:10:24 2023","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":1.0069999999999999,"15min":1.0149999999999999},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.421,"5min":1.421,"15min":1.421},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.99399999999999999,"15min":1.004},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.4950000000000001,"5min":1.556,"15min":1.556},"last":0.70999999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T02:10:35.975 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T02:10:36.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.131+0000 7f6e44db8640 1 Processor -- start 2023-11-06T02:10:36.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.131+0000 7f6e44db8640 1 -- start start 2023-11-06T02:10:36.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.131+0000 7f6e44db8640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6e400b1f90 0x7f6e400cda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:36.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.131+0000 7f6e44db8640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f6e40161650 0x7f6e400d1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:36.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.131+0000 7f6e44db8640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e400d17c0 con 0x7f6e400d2190 2023-11-06T02:10:36.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.131+0000 7f6e44db8640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e400ce6c0 con 0x7f6e40161650 2023-11-06T02:10:36.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.131+0000 7f6e44db8640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e40161ad0 con 0x7f6e400b1f90 2023-11-06T02:10:36.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.131+0000 7f6e3dd9b640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f6e40161650 0x7f6e400d1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:36.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.131+0000 7f6e3e59c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6e400b1f90 0x7f6e400cda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:36.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.131+0000 7f6e3ed9d640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f6e400d2190 0x7f6e400714a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.49:36346/0 (socket says 172.21.15.49:36346) 2023-11-06T02:10:36.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.131+0000 7f6e3ed9d640 1 -- 172.21.15.49:0/1812334410 learned_addr learned my addr 172.21.15.49:0/1812334410 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:10:36.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.131+0000 7f6e3e59c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6e400b1f90 0x7f6e400cda70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.49:41098/0 (socket says 172.21.15.49:41098) 2023-11-06T02:10:36.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.131+0000 7f6e3dd9b640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f6e40161650 0x7f6e400d1c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.49:52624/0 (socket says 172.21.15.49:52624) 2023-11-06T02:10:36.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.131+0000 7f6e3e59c640 1 -- 172.21.15.49:0/1812334410 >> v2:172.21.15.150:3300/0 conn(0x7f6e40161650 msgr2=0x7f6e400d1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:36.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.131+0000 7f6e3e59c640 1 --2- 172.21.15.49:0/1812334410 >> v2:172.21.15.150:3300/0 conn(0x7f6e40161650 0x7f6e400d1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:36.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.131+0000 7f6e3e59c640 1 -- 172.21.15.49:0/1812334410 >> v1:172.21.15.150:6789/0 conn(0x7f6e400d2190 legacy=0x7f6e400714a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:36.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.131+0000 7f6e3e59c640 1 -- 172.21.15.49:0/1812334410 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e400ce100 con 0x7f6e400b1f90 2023-11-06T02:10:36.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.131+0000 7f6e3dd9b640 1 --2- 172.21.15.49:0/1812334410 >> v2:172.21.15.150:3300/0 conn(0x7f6e40161650 0x7f6e400d1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T02:10:36.146 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.131+0000 7f6e3e59c640 1 --2- 172.21.15.49:0/1812334410 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6e400b1f90 0x7f6e400cda70 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6e300097c0 tx=0x7f6e3001c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b213c7b798ed6565 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:36.146 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.135+0000 7f6e3d59a640 1 -- 172.21.15.49:0/1812334410 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e300ab650 con 0x7f6e400b1f90 2023-11-06T02:10:36.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.135+0000 7f6e3d59a640 1 -- 172.21.15.49:0/1812334410 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e300ab7b0 con 0x7f6e400b1f90 2023-11-06T02:10:36.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.135+0000 7f6e3d59a640 1 -- 172.21.15.49:0/1812334410 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e300abb40 con 0x7f6e400b1f90 2023-11-06T02:10:36.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.135+0000 7f6e44db8640 1 -- 172.21.15.49:0/1812334410 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6e400b1f90 msgr2=0x7f6e400cda70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:36.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.135+0000 7f6e44db8640 1 --2- 172.21.15.49:0/1812334410 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6e400b1f90 0x7f6e400cda70 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6e300097c0 tx=0x7f6e3001c450 comp rx=0 tx=0).stop 2023-11-06T02:10:36.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.135+0000 7f6e44db8640 1 -- 172.21.15.49:0/1812334410 shutdown_connections 2023-11-06T02:10:36.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.135+0000 7f6e44db8640 1 --2- 172.21.15.49:0/1812334410 >> v2:172.21.15.150:3300/0 conn(0x7f6e40161650 0x7f6e400d1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:36.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.135+0000 7f6e44db8640 1 --2- 172.21.15.49:0/1812334410 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6e400b1f90 0x7f6e400cda70 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:36.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.135+0000 7f6e44db8640 1 -- 172.21.15.49:0/1812334410 >> 172.21.15.49:0/1812334410 conn(0x7f6e400d2aa0 msgr2=0x7f6e400d2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:36.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.135+0000 7f6e44db8640 1 -- 172.21.15.49:0/1812334410 shutdown_connections 2023-11-06T02:10:36.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.135+0000 7f6e44db8640 1 -- 172.21.15.49:0/1812334410 wait complete. 2023-11-06T02:10:36.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.135+0000 7f6e44db8640 1 Processor -- start 2023-11-06T02:10:36.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.135+0000 7f6e44db8640 1 -- start start 2023-11-06T02:10:36.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.135+0000 7f6e44db8640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e400b1f90 0x7f6e400c8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:36.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.135+0000 7f6e44db8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e400d2190 0x7f6e400c9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:36.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.135+0000 7f6e44db8640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6e40161650 0x7f6e400ca020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:36.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.135+0000 7f6e44db8640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e400c9030 con 0x7f6e400d2190 2023-11-06T02:10:36.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.135+0000 7f6e44db8640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e40156f90 con 0x7f6e400b1f90 2023-11-06T02:10:36.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.135+0000 7f6e44db8640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e40157270 con 0x7f6e40161650 2023-11-06T02:10:36.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.135+0000 7f6e3dd9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e400d2190 0x7f6e400c9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:36.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.135+0000 7f6e3dd9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e400d2190 0x7f6e400c9ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44118/0 (socket says 172.21.15.49:44118) 2023-11-06T02:10:36.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.135+0000 7f6e3dd9b640 1 -- 172.21.15.49:0/32574844 learned_addr learned my addr 172.21.15.49:0/32574844 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:36.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.135+0000 7f6e3e59c640 1 --2- 172.21.15.49:0/32574844 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e400b1f90 0x7f6e400c8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:36.156 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.135+0000 7f6e3ed9d640 1 --2- 172.21.15.49:0/32574844 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6e40161650 0x7f6e400ca020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:36.156 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.135+0000 7f6e3dd9b640 1 -- 172.21.15.49:0/32574844 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6e40161650 msgr2=0x7f6e400ca020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:36.157 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.135+0000 7f6e3dd9b640 1 --2- 172.21.15.49:0/32574844 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6e40161650 0x7f6e400ca020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:36.157 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.135+0000 7f6e3dd9b640 1 -- 172.21.15.49:0/32574844 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e400b1f90 msgr2=0x7f6e400c8af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:36.157 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.135+0000 7f6e3dd9b640 1 --2- 172.21.15.49:0/32574844 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e400b1f90 0x7f6e400c8af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:36.158 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.135+0000 7f6e3dd9b640 1 -- 172.21.15.49:0/32574844 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e34009420 con 0x7f6e400d2190 2023-11-06T02:10:36.158 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.135+0000 7f6e3e59c640 1 --2- 172.21.15.49:0/32574844 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e400b1f90 0x7f6e400c8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:10:36.158 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.135+0000 7f6e3dd9b640 1 --2- 172.21.15.49:0/32574844 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e400d2190 0x7f6e400c9ae0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f6e3400e980 tx=0x7f6e3400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:36.159 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.135+0000 7f6e2effd640 1 -- 172.21.15.49:0/32574844 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e3400ca00 con 0x7f6e400d2190 2023-11-06T02:10:36.159 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.135+0000 7f6e2effd640 1 -- 172.21.15.49:0/32574844 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e3400cb60 con 0x7f6e400d2190 2023-11-06T02:10:36.159 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.135+0000 7f6e44db8640 1 -- 172.21.15.49:0/32574844 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e30009420 con 0x7f6e400d2190 2023-11-06T02:10:36.159 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.135+0000 7f6e2effd640 1 -- 172.21.15.49:0/32574844 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e34013470 con 0x7f6e400d2190 2023-11-06T02:10:36.160 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.135+0000 7f6e44db8640 1 -- 172.21.15.49:0/32574844 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e40157830 con 0x7f6e400d2190 2023-11-06T02:10:36.160 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.139+0000 7f6e44db8640 1 -- 172.21.15.49:0/32574844 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e400c70e0 con 0x7f6e400d2190 2023-11-06T02:10:36.160 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.139+0000 7f6e2effd640 1 -- 172.21.15.49:0/32574844 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f6e340135d0 con 0x7f6e400d2190 2023-11-06T02:10:36.161 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.139+0000 7f6e2effd640 1 --2- 172.21.15.49:0/32574844 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6e20065a60 0x7f6e20067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:36.161 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.139+0000 7f6e2effd640 1 -- 172.21.15.49:0/32574844 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6e340835f0 con 0x7f6e400d2190 2023-11-06T02:10:36.162 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.139+0000 7f6e3e59c640 1 --2- 172.21.15.49:0/32574844 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6e20065a60 0x7f6e20067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:36.162 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.139+0000 7f6e3e59c640 1 --2- 172.21.15.49:0/32574844 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6e20065a60 0x7f6e20067f20 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f6e300af000 tx=0x7f6e300a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:36.162 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.143+0000 7f6e2effd640 1 -- 172.21.15.49:0/32574844 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f6e34050e50 con 0x7f6e400d2190 2023-11-06T02:10:36.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.295+0000 7f6e44db8640 1 -- 172.21.15.49:0/32574844 --> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6e400b6870 con 0x7f6e20065a60 2023-11-06T02:10:36.313 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.319+0000 7f6e2effd640 1 -- 172.21.15.49:0/32574844 <== mgr.4100 v2:172.21.15.150:6832/38104 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+309433 (secure 0 0 0) 0x7f6e300ab9a0 con 0x7f6e20065a60 2023-11-06T02:10:36.317 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:10:36.319 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2023-11-06T02:10:36.320 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.323+0000 7f6e44db8640 1 -- 172.21.15.49:0/32574844 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6e20065a60 msgr2=0x7f6e20067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:36.321 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.323+0000 7f6e44db8640 1 --2- 172.21.15.49:0/32574844 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6e20065a60 0x7f6e20067f20 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f6e300af000 tx=0x7f6e300a7000 comp rx=0 tx=0).stop 2023-11-06T02:10:36.321 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.323+0000 7f6e44db8640 1 -- 172.21.15.49:0/32574844 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e400d2190 msgr2=0x7f6e400c9ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:36.321 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.323+0000 7f6e44db8640 1 --2- 172.21.15.49:0/32574844 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e400d2190 0x7f6e400c9ae0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f6e3400e980 tx=0x7f6e3400ee40 comp rx=0 tx=0).stop 2023-11-06T02:10:36.321 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.323+0000 7f6e44db8640 1 -- 172.21.15.49:0/32574844 shutdown_connections 2023-11-06T02:10:36.322 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.323+0000 7f6e44db8640 1 --2- 172.21.15.49:0/32574844 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6e20065a60 0x7f6e20067f20 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:36.322 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.323+0000 7f6e44db8640 1 --2- 172.21.15.49:0/32574844 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e400d2190 0x7f6e400c9ae0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:36.322 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.323+0000 7f6e44db8640 1 --2- 172.21.15.49:0/32574844 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6e40161650 0x7f6e400ca020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:36.322 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.323+0000 7f6e44db8640 1 --2- 172.21.15.49:0/32574844 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e400b1f90 0x7f6e400c8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:36.322 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.323+0000 7f6e44db8640 1 -- 172.21.15.49:0/32574844 >> 172.21.15.49:0/32574844 conn(0x7f6e400d2aa0 msgr2=0x7f6e401600c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:36.323 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.323+0000 7f6e44db8640 1 -- 172.21.15.49:0/32574844 shutdown_connections 2023-11-06T02:10:36.323 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.323+0000 7f6e44db8640 1 -- 172.21.15.49:0/32574844 wait complete. 2023-11-06T02:10:36.333 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":113,"stamp":"2023-11-06T02:10:35.219326+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":68,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237828,"kb_used_data":5828,"kb_used_omap":0,"kb_used_meta":231936,"kb_avail":754736892,"statfs":{"total":773094113280,"available":772850577408,"internally_reserved":0,"allocated":5967872,"data_stored":3216425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":237502464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":54,"apply_latency_ms":54,"commit_latency_ns":54000000,"apply_latency_ns":54000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.702203"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.962872+0000","last_change":"2023-11-06T02:10:31.961881+0000","last_active":"2023-11-06T02:10:31.962872+0000","last_peered":"2023-11-06T02:10:31.962872+0000","last_clean":"2023-11-06T02:10:31.962872+0000","last_became_active":"2023-11-06T02:10:31.961620+0000","last_became_peered":"2023-11-06T02:10:31.961620+0000","last_unstale":"2023-11-06T02:10:31.962872+0000","last_undegraded":"2023-11-06T02:10:31.962872+0000","last_fullsized":"2023-11-06T02:10:31.962872+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:42:44.259204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.971259+0000","last_change":"2023-11-06T02:10:31.963288+0000","last_active":"2023-11-06T02:10:31.971259+0000","last_peered":"2023-11-06T02:10:31.971259+0000","last_clean":"2023-11-06T02:10:31.971259+0000","last_became_active":"2023-11-06T02:10:31.962651+0000","last_became_peered":"2023-11-06T02:10:31.962651+0000","last_unstale":"2023-11-06T02:10:31.971259+0000","last_undegraded":"2023-11-06T02:10:31.971259+0000","last_fullsized":"2023-11-06T02:10:31.971259+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:27:11.980648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.949927+0000","last_change":"2023-11-06T02:10:31.934064+0000","last_active":"2023-11-06T02:10:31.949927+0000","last_peered":"2023-11-06T02:10:31.949927+0000","last_clean":"2023-11-06T02:10:31.949927+0000","last_became_active":"2023-11-06T02:10:31.933796+0000","last_became_peered":"2023-11-06T02:10:31.933796+0000","last_unstale":"2023-11-06T02:10:31.949927+0000","last_undegraded":"2023-11-06T02:10:31.949927+0000","last_fullsized":"2023-11-06T02:10:31.949927+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:53:49.090895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.967053+0000","last_change":"2023-11-06T02:10:31.949000+0000","last_active":"2023-11-06T02:10:31.967053+0000","last_peered":"2023-11-06T02:10:31.967053+0000","last_clean":"2023-11-06T02:10:31.967053+0000","last_became_active":"2023-11-06T02:10:31.948327+0000","last_became_peered":"2023-11-06T02:10:31.948327+0000","last_unstale":"2023-11-06T02:10:31.967053+0000","last_undegraded":"2023-11-06T02:10:31.967053+0000","last_fullsized":"2023-11-06T02:10:31.967053+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:13:48.613748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.957089+0000","last_change":"2023-11-06T02:10:31.940047+0000","last_active":"2023-11-06T02:10:31.957089+0000","last_peered":"2023-11-06T02:10:31.957089+0000","last_clean":"2023-11-06T02:10:31.957089+0000","last_became_active":"2023-11-06T02:10:31.939315+0000","last_became_peered":"2023-11-06T02:10:31.939315+0000","last_unstale":"2023-11-06T02:10:31.957089+0000","last_undegraded":"2023-11-06T02:10:31.957089+0000","last_fullsized":"2023-11-06T02:10:31.957089+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:06:02.709833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.945207+0000","last_change":"2023-11-06T02:10:31.941044+0000","last_active":"2023-11-06T02:10:31.945207+0000","last_peered":"2023-11-06T02:10:31.945207+0000","last_clean":"2023-11-06T02:10:31.945207+0000","last_became_active":"2023-11-06T02:10:31.940771+0000","last_became_peered":"2023-11-06T02:10:31.940771+0000","last_unstale":"2023-11-06T02:10:31.945207+0000","last_undegraded":"2023-11-06T02:10:31.945207+0000","last_fullsized":"2023-11-06T02:10:31.945207+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:37:08.723213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.950136+0000","last_change":"2023-11-06T02:10:31.935759+0000","last_active":"2023-11-06T02:10:31.950136+0000","last_peered":"2023-11-06T02:10:31.950136+0000","last_clean":"2023-11-06T02:10:31.950136+0000","last_became_active":"2023-11-06T02:10:31.935167+0000","last_became_peered":"2023-11-06T02:10:31.935167+0000","last_unstale":"2023-11-06T02:10:31.950136+0000","last_undegraded":"2023-11-06T02:10:31.950136+0000","last_fullsized":"2023-11-06T02:10:31.950136+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:13:36.170486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.946704+0000","last_change":"2023-11-06T02:10:31.936273+0000","last_active":"2023-11-06T02:10:31.946704+0000","last_peered":"2023-11-06T02:10:31.946704+0000","last_clean":"2023-11-06T02:10:31.946704+0000","last_became_active":"2023-11-06T02:10:31.934790+0000","last_became_peered":"2023-11-06T02:10:31.934790+0000","last_unstale":"2023-11-06T02:10:31.946704+0000","last_undegraded":"2023-11-06T02:10:31.946704+0000","last_fullsized":"2023-11-06T02:10:31.946704+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:35:16.923535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.932018+0000","last_change":"2023-11-06T02:10:31.928872+0000","last_active":"2023-11-06T02:10:31.932018+0000","last_peered":"2023-11-06T02:10:31.932018+0000","last_clean":"2023-11-06T02:10:31.932018+0000","last_became_active":"2023-11-06T02:10:31.928595+0000","last_became_peered":"2023-11-06T02:10:31.928595+0000","last_unstale":"2023-11-06T02:10:31.932018+0000","last_undegraded":"2023-11-06T02:10:31.932018+0000","last_fullsized":"2023-11-06T02:10:31.932018+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:31:04.831175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.969509+0000","last_change":"2023-11-06T02:10:31.968307+0000","last_active":"2023-11-06T02:10:31.969509+0000","last_peered":"2023-11-06T02:10:31.969509+0000","last_clean":"2023-11-06T02:10:31.969509+0000","last_became_active":"2023-11-06T02:10:31.967962+0000","last_became_peered":"2023-11-06T02:10:31.967962+0000","last_unstale":"2023-11-06T02:10:31.969509+0000","last_undegraded":"2023-11-06T02:10:31.969509+0000","last_fullsized":"2023-11-06T02:10:31.969509+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:28:33.741295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-06T02:10:32.922318+0000","last_change":"2023-11-06T02:10:31.961301+0000","last_active":"2023-11-06T02:10:32.922318+0000","last_peered":"2023-11-06T02:10:32.922318+0000","last_clean":"2023-11-06T02:10:32.922318+0000","last_became_active":"2023-11-06T02:10:31.960991+0000","last_became_peered":"2023-11-06T02:10:31.960991+0000","last_unstale":"2023-11-06T02:10:32.922318+0000","last_undegraded":"2023-11-06T02:10:32.922318+0000","last_fullsized":"2023-11-06T02:10:32.922318+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:05:12.860701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.929838+0000","last_change":"2023-11-06T02:10:31.924870+0000","last_active":"2023-11-06T02:10:31.929838+0000","last_peered":"2023-11-06T02:10:31.929838+0000","last_clean":"2023-11-06T02:10:31.929838+0000","last_became_active":"2023-11-06T02:10:31.924577+0000","last_became_peered":"2023-11-06T02:10:31.924577+0000","last_unstale":"2023-11-06T02:10:31.929838+0000","last_undegraded":"2023-11-06T02:10:31.929838+0000","last_fullsized":"2023-11-06T02:10:31.929838+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:55:15.265490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.969490+0000","last_change":"2023-11-06T02:10:31.968305+0000","last_active":"2023-11-06T02:10:31.969490+0000","last_peered":"2023-11-06T02:10:31.969490+0000","last_clean":"2023-11-06T02:10:31.969490+0000","last_became_active":"2023-11-06T02:10:31.967985+0000","last_became_peered":"2023-11-06T02:10:31.967985+0000","last_unstale":"2023-11-06T02:10:31.969490+0000","last_undegraded":"2023-11-06T02:10:31.969490+0000","last_fullsized":"2023-11-06T02:10:31.969490+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:26:13.513716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.950001+0000","last_change":"2023-11-06T02:10:31.934794+0000","last_active":"2023-11-06T02:10:31.950001+0000","last_peered":"2023-11-06T02:10:31.950001+0000","last_clean":"2023-11-06T02:10:31.950001+0000","last_became_active":"2023-11-06T02:10:31.934512+0000","last_became_peered":"2023-11-06T02:10:31.934512+0000","last_unstale":"2023-11-06T02:10:31.950001+0000","last_undegraded":"2023-11-06T02:10:31.950001+0000","last_fullsized":"2023-11-06T02:10:31.950001+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:45:36.472356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.967791+0000","last_change":"2023-11-06T02:10:31.956262+0000","last_active":"2023-11-06T02:10:31.967791+0000","last_peered":"2023-11-06T02:10:31.967791+0000","last_clean":"2023-11-06T02:10:31.967791+0000","last_became_active":"2023-11-06T02:10:31.951419+0000","last_became_peered":"2023-11-06T02:10:31.951419+0000","last_unstale":"2023-11-06T02:10:31.967791+0000","last_undegraded":"2023-11-06T02:10:31.967791+0000","last_fullsized":"2023-11-06T02:10:31.967791+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:00:38.993498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.969788+0000","last_change":"2023-11-06T02:10:31.968492+0000","last_active":"2023-11-06T02:10:31.969788+0000","last_peered":"2023-11-06T02:10:31.969788+0000","last_clean":"2023-11-06T02:10:31.969788+0000","last_became_active":"2023-11-06T02:10:31.967960+0000","last_became_peered":"2023-11-06T02:10:31.967960+0000","last_unstale":"2023-11-06T02:10:31.969788+0000","last_undegraded":"2023-11-06T02:10:31.969788+0000","last_fullsized":"2023-11-06T02:10:31.969788+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:03:31.067962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.963205+0000","last_change":"2023-11-06T02:10:31.962386+0000","last_active":"2023-11-06T02:10:31.963205+0000","last_peered":"2023-11-06T02:10:31.963205+0000","last_clean":"2023-11-06T02:10:31.963205+0000","last_became_active":"2023-11-06T02:10:31.962192+0000","last_became_peered":"2023-11-06T02:10:31.962192+0000","last_unstale":"2023-11-06T02:10:31.963205+0000","last_undegraded":"2023-11-06T02:10:31.963205+0000","last_fullsized":"2023-11-06T02:10:31.963205+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:32:04.870127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.974757+0000","last_change":"2023-11-06T02:10:31.971319+0000","last_active":"2023-11-06T02:10:31.974757+0000","last_peered":"2023-11-06T02:10:31.974757+0000","last_clean":"2023-11-06T02:10:31.974757+0000","last_became_active":"2023-11-06T02:10:31.970637+0000","last_became_peered":"2023-11-06T02:10:31.970637+0000","last_unstale":"2023-11-06T02:10:31.974757+0000","last_undegraded":"2023-11-06T02:10:31.974757+0000","last_fullsized":"2023-11-06T02:10:31.974757+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:32.392045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.957430+0000","last_change":"2023-11-06T02:10:31.956621+0000","last_active":"2023-11-06T02:10:31.957430+0000","last_peered":"2023-11-06T02:10:31.957430+0000","last_clean":"2023-11-06T02:10:31.957430+0000","last_became_active":"2023-11-06T02:10:31.956361+0000","last_became_peered":"2023-11-06T02:10:31.956361+0000","last_unstale":"2023-11-06T02:10:31.957430+0000","last_undegraded":"2023-11-06T02:10:31.957430+0000","last_fullsized":"2023-11-06T02:10:31.957430+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:32:49.788949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-06T02:10:32.926256+0000","last_change":"2023-11-06T02:10:31.958138+0000","last_active":"2023-11-06T02:10:32.926256+0000","last_peered":"2023-11-06T02:10:32.926256+0000","last_clean":"2023-11-06T02:10:32.926256+0000","last_became_active":"2023-11-06T02:10:31.956588+0000","last_became_peered":"2023-11-06T02:10:31.956588+0000","last_unstale":"2023-11-06T02:10:32.926256+0000","last_undegraded":"2023-11-06T02:10:32.926256+0000","last_fullsized":"2023-11-06T02:10:32.926256+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:16:37.324576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-06T02:10:32.926625+0000","last_change":"2023-11-06T02:10:31.948873+0000","last_active":"2023-11-06T02:10:32.926625+0000","last_peered":"2023-11-06T02:10:32.926625+0000","last_clean":"2023-11-06T02:10:32.926625+0000","last_became_active":"2023-11-06T02:10:31.948082+0000","last_became_peered":"2023-11-06T02:10:31.948082+0000","last_unstale":"2023-11-06T02:10:32.926625+0000","last_undegraded":"2023-11-06T02:10:32.926625+0000","last_fullsized":"2023-11-06T02:10:32.926625+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:45:06.160032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.946154+0000","last_change":"2023-11-06T02:10:31.931027+0000","last_active":"2023-11-06T02:10:31.946154+0000","last_peered":"2023-11-06T02:10:31.946154+0000","last_clean":"2023-11-06T02:10:31.946154+0000","last_became_active":"2023-11-06T02:10:31.930716+0000","last_became_peered":"2023-11-06T02:10:31.930716+0000","last_unstale":"2023-11-06T02:10:31.946154+0000","last_undegraded":"2023-11-06T02:10:31.946154+0000","last_fullsized":"2023-11-06T02:10:31.946154+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:54:44.600666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-06T02:10:32.921731+0000","last_change":"2023-11-06T02:10:31.940652+0000","last_active":"2023-11-06T02:10:32.921731+0000","last_peered":"2023-11-06T02:10:32.921731+0000","last_clean":"2023-11-06T02:10:32.921731+0000","last_became_active":"2023-11-06T02:10:31.940375+0000","last_became_peered":"2023-11-06T02:10:31.940375+0000","last_unstale":"2023-11-06T02:10:32.921731+0000","last_undegraded":"2023-11-06T02:10:32.921731+0000","last_fullsized":"2023-11-06T02:10:32.921731+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:39:40.475876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.950319+0000","last_change":"2023-11-06T02:10:31.936913+0000","last_active":"2023-11-06T02:10:31.950319+0000","last_peered":"2023-11-06T02:10:31.950319+0000","last_clean":"2023-11-06T02:10:31.950319+0000","last_became_active":"2023-11-06T02:10:31.936613+0000","last_became_peered":"2023-11-06T02:10:31.936613+0000","last_unstale":"2023-11-06T02:10:31.950319+0000","last_undegraded":"2023-11-06T02:10:31.950319+0000","last_fullsized":"2023-11-06T02:10:31.950319+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:57:16.913819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.950586+0000","last_change":"2023-11-06T02:10:31.946840+0000","last_active":"2023-11-06T02:10:31.950586+0000","last_peered":"2023-11-06T02:10:31.950586+0000","last_clean":"2023-11-06T02:10:31.950586+0000","last_became_active":"2023-11-06T02:10:31.946546+0000","last_became_peered":"2023-11-06T02:10:31.946546+0000","last_unstale":"2023-11-06T02:10:31.950586+0000","last_undegraded":"2023-11-06T02:10:31.950586+0000","last_fullsized":"2023-11-06T02:10:31.950586+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:15:49.333136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.947005+0000","last_change":"2023-11-06T02:10:31.939200+0000","last_active":"2023-11-06T02:10:31.947005+0000","last_peered":"2023-11-06T02:10:31.947005+0000","last_clean":"2023-11-06T02:10:31.947005+0000","last_became_active":"2023-11-06T02:10:31.938898+0000","last_became_peered":"2023-11-06T02:10:31.938898+0000","last_unstale":"2023-11-06T02:10:31.947005+0000","last_undegraded":"2023-11-06T02:10:31.947005+0000","last_fullsized":"2023-11-06T02:10:31.947005+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:18:59.261936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.942772+0000","last_change":"2023-11-06T02:10:31.936997+0000","last_active":"2023-11-06T02:10:31.942772+0000","last_peered":"2023-11-06T02:10:31.942772+0000","last_clean":"2023-11-06T02:10:31.942772+0000","last_became_active":"2023-11-06T02:10:31.936748+0000","last_became_peered":"2023-11-06T02:10:31.936748+0000","last_unstale":"2023-11-06T02:10:31.942772+0000","last_undegraded":"2023-11-06T02:10:31.942772+0000","last_fullsized":"2023-11-06T02:10:31.942772+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:07:03.689829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-06T02:10:32.926540+0000","last_change":"2023-11-06T02:10:31.931782+0000","last_active":"2023-11-06T02:10:32.926540+0000","last_peered":"2023-11-06T02:10:32.926540+0000","last_clean":"2023-11-06T02:10:32.926540+0000","last_became_active":"2023-11-06T02:10:31.931084+0000","last_became_peered":"2023-11-06T02:10:31.931084+0000","last_unstale":"2023-11-06T02:10:32.926540+0000","last_undegraded":"2023-11-06T02:10:32.926540+0000","last_fullsized":"2023-11-06T02:10:32.926540+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:34:11.282284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.951829+0000","last_change":"2023-11-06T02:10:31.950559+0000","last_active":"2023-11-06T02:10:31.951829+0000","last_peered":"2023-11-06T02:10:31.951829+0000","last_clean":"2023-11-06T02:10:31.951829+0000","last_became_active":"2023-11-06T02:10:31.950253+0000","last_became_peered":"2023-11-06T02:10:31.950253+0000","last_unstale":"2023-11-06T02:10:31.951829+0000","last_undegraded":"2023-11-06T02:10:31.951829+0000","last_fullsized":"2023-11-06T02:10:31.951829+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:57:03.504241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.966941+0000","last_change":"2023-11-06T02:10:31.948384+0000","last_active":"2023-11-06T02:10:31.966941+0000","last_peered":"2023-11-06T02:10:31.966941+0000","last_clean":"2023-11-06T02:10:31.966941+0000","last_became_active":"2023-11-06T02:10:31.947450+0000","last_became_peered":"2023-11-06T02:10:31.947450+0000","last_unstale":"2023-11-06T02:10:31.966941+0000","last_undegraded":"2023-11-06T02:10:31.966941+0000","last_fullsized":"2023-11-06T02:10:31.966941+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:28:33.741295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.950161+0000","last_change":"2023-11-06T02:10:31.935920+0000","last_active":"2023-11-06T02:10:31.950161+0000","last_peered":"2023-11-06T02:10:31.950161+0000","last_clean":"2023-11-06T02:10:31.950161+0000","last_became_active":"2023-11-06T02:10:31.935509+0000","last_became_peered":"2023-11-06T02:10:31.935509+0000","last_unstale":"2023-11-06T02:10:31.950161+0000","last_undegraded":"2023-11-06T02:10:31.950161+0000","last_fullsized":"2023-11-06T02:10:31.950161+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:18:59.261936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-06T02:10:32.927060+0000","last_change":"2023-11-06T02:10:31.965611+0000","last_active":"2023-11-06T02:10:32.927060+0000","last_peered":"2023-11-06T02:10:32.927060+0000","last_clean":"2023-11-06T02:10:32.927060+0000","last_became_active":"2023-11-06T02:10:31.964913+0000","last_became_peered":"2023-11-06T02:10:31.964913+0000","last_unstale":"2023-11-06T02:10:32.927060+0000","last_undegraded":"2023-11-06T02:10:32.927060+0000","last_fullsized":"2023-11-06T02:10:32.927060+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:32:49.788949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.972773+0000","last_change":"2023-11-06T02:10:31.969631+0000","last_active":"2023-11-06T02:10:31.972773+0000","last_peered":"2023-11-06T02:10:31.972773+0000","last_clean":"2023-11-06T02:10:31.972773+0000","last_became_active":"2023-11-06T02:10:31.969086+0000","last_became_peered":"2023-11-06T02:10:31.969086+0000","last_unstale":"2023-11-06T02:10:31.972773+0000","last_undegraded":"2023-11-06T02:10:31.972773+0000","last_fullsized":"2023-11-06T02:10:31.972773+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:06:02.709833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"50'4","reported_seq":15,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-06T02:10:32.926572+0000","last_change":"2023-11-06T02:10:31.562064+0000","last_active":"2023-11-06T02:10:32.926572+0000","last_peered":"2023-11-06T02:10:32.926572+0000","last_clean":"2023-11-06T02:10:32.926572+0000","last_became_active":"2023-11-06T02:10:31.561463+0000","last_became_peered":"2023-11-06T02:10:31.561463+0000","last_unstale":"2023-11-06T02:10:32.926572+0000","last_undegraded":"2023-11-06T02:10:32.926572+0000","last_fullsized":"2023-11-06T02:10:32.926572+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_clean_scrub_stamp":"2023-11-06T02:10:30.509051+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:09:08.748403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-06T02:10:32.928178+0000","last_change":"2023-11-06T02:10:31.963633+0000","last_active":"2023-11-06T02:10:32.928178+0000","last_peered":"2023-11-06T02:10:32.928178+0000","last_clean":"2023-11-06T02:10:32.928178+0000","last_became_active":"2023-11-06T02:10:31.962849+0000","last_became_peered":"2023-11-06T02:10:31.962849+0000","last_unstale":"2023-11-06T02:10:32.928178+0000","last_undegraded":"2023-11-06T02:10:32.928178+0000","last_fullsized":"2023-11-06T02:10:32.928178+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:01:30.948009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"50'1","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.947228+0000","last_change":"2023-11-06T02:10:31.556249+0000","last_active":"2023-11-06T02:10:31.947228+0000","last_peered":"2023-11-06T02:10:31.947228+0000","last_clean":"2023-11-06T02:10:31.947228+0000","last_became_active":"2023-11-06T02:10:31.555864+0000","last_became_peered":"2023-11-06T02:10:31.555864+0000","last_unstale":"2023-11-06T02:10:31.947228+0000","last_undegraded":"2023-11-06T02:10:31.947228+0000","last_fullsized":"2023-11-06T02:10:31.947228+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_clean_scrub_stamp":"2023-11-06T02:10:30.509051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:29:31.684272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-06T02:10:32.926160+0000","last_change":"2023-11-06T02:10:31.947889+0000","last_active":"2023-11-06T02:10:32.926160+0000","last_peered":"2023-11-06T02:10:32.926160+0000","last_clean":"2023-11-06T02:10:32.926160+0000","last_became_active":"2023-11-06T02:10:31.947029+0000","last_became_peered":"2023-11-06T02:10:31.947029+0000","last_unstale":"2023-11-06T02:10:32.926160+0000","last_undegraded":"2023-11-06T02:10:32.926160+0000","last_fullsized":"2023-11-06T02:10:32.926160+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:50:33.170839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"50'2","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.954057+0000","last_change":"2023-11-06T02:10:31.564822+0000","last_active":"2023-11-06T02:10:31.954057+0000","last_peered":"2023-11-06T02:10:31.954057+0000","last_clean":"2023-11-06T02:10:31.954057+0000","last_became_active":"2023-11-06T02:10:31.564460+0000","last_became_peered":"2023-11-06T02:10:31.564460+0000","last_unstale":"2023-11-06T02:10:31.954057+0000","last_undegraded":"2023-11-06T02:10:31.954057+0000","last_fullsized":"2023-11-06T02:10:31.954057+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_clean_scrub_stamp":"2023-11-06T02:10:30.509051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:05:11.856979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.947413+0000","last_change":"2023-11-06T02:10:31.941264+0000","last_active":"2023-11-06T02:10:31.947413+0000","last_peered":"2023-11-06T02:10:31.947413+0000","last_clean":"2023-11-06T02:10:31.947413+0000","last_became_active":"2023-11-06T02:10:31.937191+0000","last_became_peered":"2023-11-06T02:10:31.937191+0000","last_unstale":"2023-11-06T02:10:31.947413+0000","last_undegraded":"2023-11-06T02:10:31.947413+0000","last_fullsized":"2023-11-06T02:10:31.947413+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:57:16.913819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"50'2","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-06T02:10:32.928359+0000","last_change":"2023-11-06T02:10:31.558181+0000","last_active":"2023-11-06T02:10:32.928359+0000","last_peered":"2023-11-06T02:10:32.928359+0000","last_clean":"2023-11-06T02:10:32.928359+0000","last_became_active":"2023-11-06T02:10:31.557290+0000","last_became_peered":"2023-11-06T02:10:31.557290+0000","last_unstale":"2023-11-06T02:10:32.928359+0000","last_undegraded":"2023-11-06T02:10:32.928359+0000","last_fullsized":"2023-11-06T02:10:32.928359+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_clean_scrub_stamp":"2023-11-06T02:10:30.509051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:23:34.675851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.966033+0000","last_change":"2023-11-06T02:10:31.955324+0000","last_active":"2023-11-06T02:10:31.966033+0000","last_peered":"2023-11-06T02:10:31.966033+0000","last_clean":"2023-11-06T02:10:31.966033+0000","last_became_active":"2023-11-06T02:10:31.954475+0000","last_became_peered":"2023-11-06T02:10:31.954475+0000","last_unstale":"2023-11-06T02:10:31.966033+0000","last_undegraded":"2023-11-06T02:10:31.966033+0000","last_fullsized":"2023-11-06T02:10:31.966033+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:27:11.980648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"50'1","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.950665+0000","last_change":"2023-11-06T02:10:31.547787+0000","last_active":"2023-11-06T02:10:31.950665+0000","last_peered":"2023-11-06T02:10:31.950665+0000","last_clean":"2023-11-06T02:10:31.950665+0000","last_became_active":"2023-11-06T02:10:31.547398+0000","last_became_peered":"2023-11-06T02:10:31.547398+0000","last_unstale":"2023-11-06T02:10:31.950665+0000","last_undegraded":"2023-11-06T02:10:31.950665+0000","last_fullsized":"2023-11-06T02:10:31.950665+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_clean_scrub_stamp":"2023-11-06T02:10:30.509051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:16:17.912070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.970614+0000","last_change":"2023-11-06T02:10:31.960703+0000","last_active":"2023-11-06T02:10:31.970614+0000","last_peered":"2023-11-06T02:10:31.970614+0000","last_clean":"2023-11-06T02:10:31.970614+0000","last_became_active":"2023-11-06T02:10:31.959512+0000","last_became_peered":"2023-11-06T02:10:31.959512+0000","last_unstale":"2023-11-06T02:10:31.970614+0000","last_undegraded":"2023-11-06T02:10:31.970614+0000","last_fullsized":"2023-11-06T02:10:31.970614+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:26:13.513716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"50'1","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.977215+0000","last_change":"2023-11-06T02:10:31.557235+0000","last_active":"2023-11-06T02:10:31.977215+0000","last_peered":"2023-11-06T02:10:31.977215+0000","last_clean":"2023-11-06T02:10:31.977215+0000","last_became_active":"2023-11-06T02:10:31.556387+0000","last_became_peered":"2023-11-06T02:10:31.556387+0000","last_unstale":"2023-11-06T02:10:31.977215+0000","last_undegraded":"2023-11-06T02:10:31.977215+0000","last_fullsized":"2023-11-06T02:10:31.977215+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_clean_scrub_stamp":"2023-11-06T02:10:30.509051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:37:07.719491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.946964+0000","last_change":"2023-11-06T02:10:31.937747+0000","last_active":"2023-11-06T02:10:31.946964+0000","last_peered":"2023-11-06T02:10:31.946964+0000","last_clean":"2023-11-06T02:10:31.946964+0000","last_became_active":"2023-11-06T02:10:31.937479+0000","last_became_peered":"2023-11-06T02:10:31.937479+0000","last_unstale":"2023-11-06T02:10:31.946964+0000","last_undegraded":"2023-11-06T02:10:31.946964+0000","last_fullsized":"2023-11-06T02:10:31.946964+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:15:49.333136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-06T02:10:32.926047+0000","last_change":"2023-11-06T02:10:31.554495+0000","last_active":"2023-11-06T02:10:32.926047+0000","last_peered":"2023-11-06T02:10:32.926047+0000","last_clean":"2023-11-06T02:10:32.926047+0000","last_became_active":"2023-11-06T02:10:31.553778+0000","last_became_peered":"2023-11-06T02:10:31.553778+0000","last_unstale":"2023-11-06T02:10:32.926047+0000","last_undegraded":"2023-11-06T02:10:32.926047+0000","last_fullsized":"2023-11-06T02:10:32.926047+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_clean_scrub_stamp":"2023-11-06T02:10:30.509051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:15:11.104076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.967855+0000","last_change":"2023-11-06T02:10:31.956320+0000","last_active":"2023-11-06T02:10:31.967855+0000","last_peered":"2023-11-06T02:10:31.967855+0000","last_clean":"2023-11-06T02:10:31.967855+0000","last_became_active":"2023-11-06T02:10:31.951550+0000","last_became_peered":"2023-11-06T02:10:31.951550+0000","last_unstale":"2023-11-06T02:10:31.967855+0000","last_undegraded":"2023-11-06T02:10:31.967855+0000","last_fullsized":"2023-11-06T02:10:31.967855+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:42:44.259204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.923073+0000","last_change":"2023-11-06T02:10:31.542797+0000","last_active":"2023-11-06T02:10:31.923073+0000","last_peered":"2023-11-06T02:10:31.923073+0000","last_clean":"2023-11-06T02:10:31.923073+0000","last_became_active":"2023-11-06T02:10:31.542455+0000","last_became_peered":"2023-11-06T02:10:31.542455+0000","last_unstale":"2023-11-06T02:10:31.923073+0000","last_undegraded":"2023-11-06T02:10:31.923073+0000","last_fullsized":"2023-11-06T02:10:31.923073+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_clean_scrub_stamp":"2023-11-06T02:10:30.509051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:23:34.675851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.942510+0000","last_change":"2023-11-06T02:10:31.927455+0000","last_active":"2023-11-06T02:10:31.942510+0000","last_peered":"2023-11-06T02:10:31.942510+0000","last_clean":"2023-11-06T02:10:31.942510+0000","last_became_active":"2023-11-06T02:10:31.926732+0000","last_became_peered":"2023-11-06T02:10:31.926732+0000","last_unstale":"2023-11-06T02:10:31.942510+0000","last_undegraded":"2023-11-06T02:10:31.942510+0000","last_fullsized":"2023-11-06T02:10:31.942510+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:31:52.575652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.935157+0000","last_change":"2023-11-06T02:10:31.562799+0000","last_active":"2023-11-06T02:10:31.935157+0000","last_peered":"2023-11-06T02:10:31.935157+0000","last_clean":"2023-11-06T02:10:31.935157+0000","last_became_active":"2023-11-06T02:10:31.562086+0000","last_became_peered":"2023-11-06T02:10:31.562086+0000","last_unstale":"2023-11-06T02:10:31.935157+0000","last_undegraded":"2023-11-06T02:10:31.935157+0000","last_fullsized":"2023-11-06T02:10:31.935157+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_clean_scrub_stamp":"2023-11-06T02:10:30.509051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:34:10.278562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-06T02:10:32.927913+0000","last_change":"2023-11-06T02:10:31.949670+0000","last_active":"2023-11-06T02:10:32.927913+0000","last_peered":"2023-11-06T02:10:32.927913+0000","last_clean":"2023-11-06T02:10:32.927913+0000","last_became_active":"2023-11-06T02:10:31.946195+0000","last_became_peered":"2023-11-06T02:10:31.946195+0000","last_unstale":"2023-11-06T02:10:32.927913+0000","last_undegraded":"2023-11-06T02:10:32.927913+0000","last_fullsized":"2023-11-06T02:10:32.927913+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:06:02.709833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.919435+0000","last_change":"2023-11-06T02:10:31.545722+0000","last_active":"2023-11-06T02:10:31.919435+0000","last_peered":"2023-11-06T02:10:31.919435+0000","last_clean":"2023-11-06T02:10:31.919435+0000","last_became_active":"2023-11-06T02:10:31.545023+0000","last_became_peered":"2023-11-06T02:10:31.545023+0000","last_unstale":"2023-11-06T02:10:31.919435+0000","last_undegraded":"2023-11-06T02:10:31.919435+0000","last_fullsized":"2023-11-06T02:10:31.919435+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_clean_scrub_stamp":"2023-11-06T02:10:30.509051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:03:30.064240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.948006+0000","last_change":"2023-11-06T02:10:31.931764+0000","last_active":"2023-11-06T02:10:31.948006+0000","last_peered":"2023-11-06T02:10:31.948006+0000","last_clean":"2023-11-06T02:10:31.948006+0000","last_became_active":"2023-11-06T02:10:31.931286+0000","last_became_peered":"2023-11-06T02:10:31.931286+0000","last_unstale":"2023-11-06T02:10:31.948006+0000","last_undegraded":"2023-11-06T02:10:31.948006+0000","last_fullsized":"2023-11-06T02:10:31.948006+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:31:04.831175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"50'3","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.968264+0000","last_change":"2023-11-06T02:10:31.558718+0000","last_active":"2023-11-06T02:10:31.968264+0000","last_peered":"2023-11-06T02:10:31.968264+0000","last_clean":"2023-11-06T02:10:31.968264+0000","last_became_active":"2023-11-06T02:10:31.558194+0000","last_became_peered":"2023-11-06T02:10:31.558194+0000","last_unstale":"2023-11-06T02:10:31.968264+0000","last_undegraded":"2023-11-06T02:10:31.968264+0000","last_fullsized":"2023-11-06T02:10:31.968264+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_clean_scrub_stamp":"2023-11-06T02:10:30.509051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:29:16.337946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.957138+0000","last_change":"2023-11-06T02:10:31.941420+0000","last_active":"2023-11-06T02:10:31.957138+0000","last_peered":"2023-11-06T02:10:31.957138+0000","last_clean":"2023-11-06T02:10:31.957138+0000","last_became_active":"2023-11-06T02:10:31.940665+0000","last_became_peered":"2023-11-06T02:10:31.940665+0000","last_unstale":"2023-11-06T02:10:31.957138+0000","last_undegraded":"2023-11-06T02:10:31.957138+0000","last_fullsized":"2023-11-06T02:10:31.957138+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:13:48.613748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"50'6","reported_seq":16,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.952574+0000","last_change":"2023-11-06T02:10:31.553451+0000","last_active":"2023-11-06T02:10:31.952574+0000","last_peered":"2023-11-06T02:10:31.952574+0000","last_clean":"2023-11-06T02:10:31.952574+0000","last_became_active":"2023-11-06T02:10:31.552888+0000","last_became_peered":"2023-11-06T02:10:31.552888+0000","last_unstale":"2023-11-06T02:10:31.952574+0000","last_undegraded":"2023-11-06T02:10:31.952574+0000","last_fullsized":"2023-11-06T02:10:31.952574+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_clean_scrub_stamp":"2023-11-06T02:10:30.509051+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:42:43.255482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-06T02:10:32.927970+0000","last_change":"2023-11-06T02:10:31.963637+0000","last_active":"2023-11-06T02:10:32.927970+0000","last_peered":"2023-11-06T02:10:32.927970+0000","last_clean":"2023-11-06T02:10:32.927970+0000","last_became_active":"2023-11-06T02:10:31.962812+0000","last_became_peered":"2023-11-06T02:10:31.962812+0000","last_unstale":"2023-11-06T02:10:32.927970+0000","last_undegraded":"2023-11-06T02:10:32.927970+0000","last_fullsized":"2023-11-06T02:10:32.927970+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:13:48.613748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"50'2","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.967511+0000","last_change":"2023-11-06T02:10:31.551770+0000","last_active":"2023-11-06T02:10:31.967511+0000","last_peered":"2023-11-06T02:10:31.967511+0000","last_clean":"2023-11-06T02:10:31.967511+0000","last_became_active":"2023-11-06T02:10:31.551322+0000","last_became_peered":"2023-11-06T02:10:31.551322+0000","last_unstale":"2023-11-06T02:10:31.967511+0000","last_undegraded":"2023-11-06T02:10:31.967511+0000","last_fullsized":"2023-11-06T02:10:31.967511+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_clean_scrub_stamp":"2023-11-06T02:10:30.509051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:46.288303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.950355+0000","last_change":"2023-11-06T02:10:31.938632+0000","last_active":"2023-11-06T02:10:31.950355+0000","last_peered":"2023-11-06T02:10:31.950355+0000","last_clean":"2023-11-06T02:10:31.950355+0000","last_became_active":"2023-11-06T02:10:31.938130+0000","last_became_peered":"2023-11-06T02:10:31.938130+0000","last_unstale":"2023-11-06T02:10:31.950355+0000","last_undegraded":"2023-11-06T02:10:31.950355+0000","last_fullsized":"2023-11-06T02:10:31.950355+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:54:44.600666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-06T02:10:32.926132+0000","last_change":"2023-11-06T02:10:31.546238+0000","last_active":"2023-11-06T02:10:32.926132+0000","last_peered":"2023-11-06T02:10:32.926132+0000","last_clean":"2023-11-06T02:10:32.926132+0000","last_became_active":"2023-11-06T02:10:31.545410+0000","last_became_peered":"2023-11-06T02:10:31.545410+0000","last_unstale":"2023-11-06T02:10:32.926132+0000","last_undegraded":"2023-11-06T02:10:32.926132+0000","last_fullsized":"2023-11-06T02:10:32.926132+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_clean_scrub_stamp":"2023-11-06T02:10:30.509051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:15:54.225385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.947030+0000","last_change":"2023-11-06T02:10:31.940225+0000","last_active":"2023-11-06T02:10:31.947030+0000","last_peered":"2023-11-06T02:10:31.947030+0000","last_clean":"2023-11-06T02:10:31.947030+0000","last_became_active":"2023-11-06T02:10:31.939963+0000","last_became_peered":"2023-11-06T02:10:31.939963+0000","last_unstale":"2023-11-06T02:10:31.947030+0000","last_undegraded":"2023-11-06T02:10:31.947030+0000","last_fullsized":"2023-11-06T02:10:31.947030+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:13:36.170486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.933014+0000","last_change":"2023-11-06T02:10:31.568369+0000","last_active":"2023-11-06T02:10:31.933014+0000","last_peered":"2023-11-06T02:10:31.933014+0000","last_clean":"2023-11-06T02:10:31.933014+0000","last_became_active":"2023-11-06T02:10:31.567749+0000","last_became_peered":"2023-11-06T02:10:31.567749+0000","last_unstale":"2023-11-06T02:10:31.933014+0000","last_undegraded":"2023-11-06T02:10:31.933014+0000","last_fullsized":"2023-11-06T02:10:31.933014+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_clean_scrub_stamp":"2023-11-06T02:10:30.509051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:45:05.156310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.961860+0000","last_change":"2023-11-06T02:10:31.961074+0000","last_active":"2023-11-06T02:10:31.961860+0000","last_peered":"2023-11-06T02:10:31.961860+0000","last_clean":"2023-11-06T02:10:31.961860+0000","last_became_active":"2023-11-06T02:10:31.960783+0000","last_became_peered":"2023-11-06T02:10:31.960783+0000","last_unstale":"2023-11-06T02:10:31.961860+0000","last_undegraded":"2023-11-06T02:10:31.961860+0000","last_fullsized":"2023-11-06T02:10:31.961860+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:06:35.725675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"50'8","reported_seq":18,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.946933+0000","last_change":"2023-11-06T02:10:31.552065+0000","last_active":"2023-11-06T02:10:31.946933+0000","last_peered":"2023-11-06T02:10:31.946933+0000","last_clean":"2023-11-06T02:10:31.946933+0000","last_became_active":"2023-11-06T02:10:31.551688+0000","last_became_peered":"2023-11-06T02:10:31.551688+0000","last_unstale":"2023-11-06T02:10:31.946933+0000","last_undegraded":"2023-11-06T02:10:31.946933+0000","last_fullsized":"2023-11-06T02:10:31.946933+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_clean_scrub_stamp":"2023-11-06T02:10:30.509051+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:28:32.737572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.969597+0000","last_change":"2023-11-06T02:10:31.968813+0000","last_active":"2023-11-06T02:10:31.969597+0000","last_peered":"2023-11-06T02:10:31.969597+0000","last_clean":"2023-11-06T02:10:31.969597+0000","last_became_active":"2023-11-06T02:10:31.968341+0000","last_became_peered":"2023-11-06T02:10:31.968341+0000","last_unstale":"2023-11-06T02:10:31.969597+0000","last_undegraded":"2023-11-06T02:10:31.969597+0000","last_fullsized":"2023-11-06T02:10:31.969597+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:00:32.003705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"50'1","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.950535+0000","last_change":"2023-11-06T02:10:31.554140+0000","last_active":"2023-11-06T02:10:31.950535+0000","last_peered":"2023-11-06T02:10:31.950535+0000","last_clean":"2023-11-06T02:10:31.950535+0000","last_became_active":"2023-11-06T02:10:31.553822+0000","last_became_peered":"2023-11-06T02:10:31.553822+0000","last_unstale":"2023-11-06T02:10:31.950535+0000","last_undegraded":"2023-11-06T02:10:31.950535+0000","last_fullsized":"2023-11-06T02:10:31.950535+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_clean_scrub_stamp":"2023-11-06T02:10:30.509051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:37:07.719491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.937435+0000","last_change":"2023-11-06T02:10:31.935200+0000","last_active":"2023-11-06T02:10:31.937435+0000","last_peered":"2023-11-06T02:10:31.937435+0000","last_clean":"2023-11-06T02:10:31.937435+0000","last_became_active":"2023-11-06T02:10:31.934933+0000","last_became_peered":"2023-11-06T02:10:31.934933+0000","last_unstale":"2023-11-06T02:10:31.937435+0000","last_undegraded":"2023-11-06T02:10:31.937435+0000","last_fullsized":"2023-11-06T02:10:31.937435+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:23:35.679574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"50'2","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-06T02:10:32.922423+0000","last_change":"2023-11-06T02:10:31.549129+0000","last_active":"2023-11-06T02:10:32.922423+0000","last_peered":"2023-11-06T02:10:32.922423+0000","last_clean":"2023-11-06T02:10:32.922423+0000","last_became_active":"2023-11-06T02:10:31.548717+0000","last_became_peered":"2023-11-06T02:10:31.548717+0000","last_unstale":"2023-11-06T02:10:32.922423+0000","last_undegraded":"2023-11-06T02:10:32.922423+0000","last_fullsized":"2023-11-06T02:10:32.922423+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_clean_scrub_stamp":"2023-11-06T02:10:30.509051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:09:08.748403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.966852+0000","last_change":"2023-11-06T02:10:31.957602+0000","last_active":"2023-11-06T02:10:31.966852+0000","last_peered":"2023-11-06T02:10:31.966852+0000","last_clean":"2023-11-06T02:10:31.966852+0000","last_became_active":"2023-11-06T02:10:31.955757+0000","last_became_peered":"2023-11-06T02:10:31.955757+0000","last_unstale":"2023-11-06T02:10:31.966852+0000","last_undegraded":"2023-11-06T02:10:31.966852+0000","last_fullsized":"2023-11-06T02:10:31.966852+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:32.392045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"50'1","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-06T02:10:32.926866+0000","last_change":"2023-11-06T02:10:31.550899+0000","last_active":"2023-11-06T02:10:32.926866+0000","last_peered":"2023-11-06T02:10:32.926866+0000","last_clean":"2023-11-06T02:10:32.926866+0000","last_became_active":"2023-11-06T02:10:31.550663+0000","last_became_peered":"2023-11-06T02:10:31.550663+0000","last_unstale":"2023-11-06T02:10:32.926866+0000","last_undegraded":"2023-11-06T02:10:32.926866+0000","last_fullsized":"2023-11-06T02:10:32.926866+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_clean_scrub_stamp":"2023-11-06T02:10:30.509051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:13:49.387379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.962486+0000","last_change":"2023-11-06T02:10:31.961329+0000","last_active":"2023-11-06T02:10:31.962486+0000","last_peered":"2023-11-06T02:10:31.962486+0000","last_clean":"2023-11-06T02:10:31.962486+0000","last_became_active":"2023-11-06T02:10:31.960981+0000","last_became_peered":"2023-11-06T02:10:31.960981+0000","last_unstale":"2023-11-06T02:10:31.962486+0000","last_undegraded":"2023-11-06T02:10:31.962486+0000","last_fullsized":"2023-11-06T02:10:31.962486+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:47.292026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-06T02:10:32.926624+0000","last_change":"2023-11-06T02:10:31.555335+0000","last_active":"2023-11-06T02:10:32.926624+0000","last_peered":"2023-11-06T02:10:32.926624+0000","last_clean":"2023-11-06T02:10:32.926624+0000","last_became_active":"2023-11-06T02:10:31.554525+0000","last_became_peered":"2023-11-06T02:10:31.554525+0000","last_unstale":"2023-11-06T02:10:32.926624+0000","last_undegraded":"2023-11-06T02:10:32.926624+0000","last_fullsized":"2023-11-06T02:10:32.926624+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_clean_scrub_stamp":"2023-11-06T02:10:30.509051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:28:25.243365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-06T02:10:32.928217+0000","last_change":"2023-11-06T02:10:31.950231+0000","last_active":"2023-11-06T02:10:32.928217+0000","last_peered":"2023-11-06T02:10:32.928217+0000","last_clean":"2023-11-06T02:10:32.928217+0000","last_became_active":"2023-11-06T02:10:31.946970+0000","last_became_peered":"2023-11-06T02:10:31.946970+0000","last_unstale":"2023-11-06T02:10:32.928217+0000","last_undegraded":"2023-11-06T02:10:32.928217+0000","last_fullsized":"2023-11-06T02:10:32.928217+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:28:33.741295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"50'2","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.969050+0000","last_change":"2023-11-06T02:10:31.551494+0000","last_active":"2023-11-06T02:10:31.969050+0000","last_peered":"2023-11-06T02:10:31.969050+0000","last_clean":"2023-11-06T02:10:31.969050+0000","last_became_active":"2023-11-06T02:10:31.551122+0000","last_became_peered":"2023-11-06T02:10:31.551122+0000","last_unstale":"2023-11-06T02:10:31.969050+0000","last_undegraded":"2023-11-06T02:10:31.969050+0000","last_fullsized":"2023-11-06T02:10:31.969050+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_clean_scrub_stamp":"2023-11-06T02:10:30.509051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:39:39.472153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-06T02:10:32.926624+0000","last_change":"2023-11-06T02:10:31.957394+0000","last_active":"2023-11-06T02:10:32.926624+0000","last_peered":"2023-11-06T02:10:32.926624+0000","last_clean":"2023-11-06T02:10:32.926624+0000","last_became_active":"2023-11-06T02:10:31.956320+0000","last_became_peered":"2023-11-06T02:10:31.956320+0000","last_unstale":"2023-11-06T02:10:32.926624+0000","last_undegraded":"2023-11-06T02:10:32.926624+0000","last_fullsized":"2023-11-06T02:10:32.926624+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:29:17.341669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.911597+0000","last_change":"2023-11-06T02:10:31.555740+0000","last_active":"2023-11-06T02:10:31.911597+0000","last_peered":"2023-11-06T02:10:31.911597+0000","last_clean":"2023-11-06T02:10:31.911597+0000","last_became_active":"2023-11-06T02:10:31.554890+0000","last_became_peered":"2023-11-06T02:10:31.554890+0000","last_unstale":"2023-11-06T02:10:31.911597+0000","last_undegraded":"2023-11-06T02:10:31.911597+0000","last_fullsized":"2023-11-06T02:10:31.911597+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_clean_scrub_stamp":"2023-11-06T02:10:30.509051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:26:12.509993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.947890+0000","last_change":"2023-11-06T02:10:31.931516+0000","last_active":"2023-11-06T02:10:31.947890+0000","last_peered":"2023-11-06T02:10:31.947890+0000","last_clean":"2023-11-06T02:10:31.947890+0000","last_became_active":"2023-11-06T02:10:31.931235+0000","last_became_peered":"2023-11-06T02:10:31.931235+0000","last_unstale":"2023-11-06T02:10:31.947890+0000","last_undegraded":"2023-11-06T02:10:31.947890+0000","last_fullsized":"2023-11-06T02:10:31.947890+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:35:16.923535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.921399+0000","last_change":"2023-11-06T02:10:31.546787+0000","last_active":"2023-11-06T02:10:31.921399+0000","last_peered":"2023-11-06T02:10:31.921399+0000","last_clean":"2023-11-06T02:10:31.921399+0000","last_became_active":"2023-11-06T02:10:31.546100+0000","last_became_peered":"2023-11-06T02:10:31.546100+0000","last_unstale":"2023-11-06T02:10:31.921399+0000","last_undegraded":"2023-11-06T02:10:31.921399+0000","last_fullsized":"2023-11-06T02:10:31.921399+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_clean_scrub_stamp":"2023-11-06T02:10:30.509051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:07:02.686107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.934832+0000","last_change":"2023-11-06T02:10:31.933155+0000","last_active":"2023-11-06T02:10:31.934832+0000","last_peered":"2023-11-06T02:10:31.934832+0000","last_clean":"2023-11-06T02:10:31.934832+0000","last_became_active":"2023-11-06T02:10:31.932760+0000","last_became_peered":"2023-11-06T02:10:31.932760+0000","last_unstale":"2023-11-06T02:10:31.934832+0000","last_undegraded":"2023-11-06T02:10:31.934832+0000","last_fullsized":"2023-11-06T02:10:31.934832+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:54:03.567766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-06T02:10:32.921702+0000","last_change":"2023-11-06T02:10:31.555022+0000","last_active":"2023-11-06T02:10:32.921702+0000","last_peered":"2023-11-06T02:10:32.921702+0000","last_clean":"2023-11-06T02:10:32.921702+0000","last_became_active":"2023-11-06T02:10:31.554474+0000","last_became_peered":"2023-11-06T02:10:31.554474+0000","last_unstale":"2023-11-06T02:10:32.921702+0000","last_undegraded":"2023-11-06T02:10:32.921702+0000","last_fullsized":"2023-11-06T02:10:32.921702+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_clean_scrub_stamp":"2023-11-06T02:10:30.509051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:15:54.225385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-06T02:10:32.926289+0000","last_change":"2023-11-06T02:10:31.948540+0000","last_active":"2023-11-06T02:10:32.926289+0000","last_peered":"2023-11-06T02:10:32.926289+0000","last_clean":"2023-11-06T02:10:32.926289+0000","last_became_active":"2023-11-06T02:10:31.947789+0000","last_became_peered":"2023-11-06T02:10:31.947789+0000","last_unstale":"2023-11-06T02:10:32.926289+0000","last_undegraded":"2023-11-06T02:10:32.926289+0000","last_fullsized":"2023-11-06T02:10:32.926289+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:16:18.915793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.921745+0000","last_change":"2023-11-06T02:10:31.553943+0000","last_active":"2023-11-06T02:10:31.921745+0000","last_peered":"2023-11-06T02:10:31.921745+0000","last_clean":"2023-11-06T02:10:31.921745+0000","last_became_active":"2023-11-06T02:10:31.553497+0000","last_became_peered":"2023-11-06T02:10:31.553497+0000","last_unstale":"2023-11-06T02:10:31.921745+0000","last_undegraded":"2023-11-06T02:10:31.921745+0000","last_fullsized":"2023-11-06T02:10:31.921745+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_clean_scrub_stamp":"2023-11-06T02:10:30.509051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:03:30.064240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":191,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.543938+0000","last_change":"2023-11-06T02:08:20.523427+0000","last_active":"2023-11-06T02:10:31.543938+0000","last_peered":"2023-11-06T02:10:31.543938+0000","last_clean":"2023-11-06T02:10:31.543938+0000","last_became_active":"2023-11-06T02:08:20.521089+0000","last_became_peered":"2023-11-06T02:08:20.521089+0000","last_unstale":"2023-11-06T02:10:31.543938+0000","last_undegraded":"2023-11-06T02:10:31.543938+0000","last_fullsized":"2023-11-06T02:10:31.543938+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:19.252348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:19.252348+0000","last_clean_scrub_stamp":"2023-11-06T02:08:19.252348+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:30:15.566397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":39,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-06T02:10:32.928056+0000","last_change":"2023-11-06T02:08:26.059502+0000","last_active":"2023-11-06T02:10:32.928056+0000","last_peered":"2023-11-06T02:10:32.928056+0000","last_clean":"2023-11-06T02:10:32.928056+0000","last_became_active":"2023-11-06T02:08:26.058612+0000","last_became_peered":"2023-11-06T02:08:26.058612+0000","last_unstale":"2023-11-06T02:10:32.928056+0000","last_undegraded":"2023-11-06T02:10:32.928056+0000","last_fullsized":"2023-11-06T02:10:32.928056+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:10.626021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.938119+0000","last_change":"2023-11-06T02:10:31.936110+0000","last_active":"2023-11-06T02:10:31.938119+0000","last_peered":"2023-11-06T02:10:31.938119+0000","last_clean":"2023-11-06T02:10:31.938119+0000","last_became_active":"2023-11-06T02:10:31.935818+0000","last_became_peered":"2023-11-06T02:10:31.935818+0000","last_unstale":"2023-11-06T02:10:31.938119+0000","last_undegraded":"2023-11-06T02:10:31.938119+0000","last_fullsized":"2023-11-06T02:10:31.938119+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:51:59.093456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.935168+0000","last_change":"2023-11-06T02:10:31.557657+0000","last_active":"2023-11-06T02:10:31.935168+0000","last_peered":"2023-11-06T02:10:31.935168+0000","last_clean":"2023-11-06T02:10:31.935168+0000","last_became_active":"2023-11-06T02:10:31.557039+0000","last_became_peered":"2023-11-06T02:10:31.557039+0000","last_unstale":"2023-11-06T02:10:31.935168+0000","last_undegraded":"2023-11-06T02:10:31.935168+0000","last_fullsized":"2023-11-06T02:10:31.935168+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_clean_scrub_stamp":"2023-11-06T02:10:30.509051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:50:32.167116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":38,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.543715+0000","last_change":"2023-11-06T02:08:26.060406+0000","last_active":"2023-11-06T02:10:31.543715+0000","last_peered":"2023-11-06T02:10:31.543715+0000","last_clean":"2023-11-06T02:10:31.543715+0000","last_became_active":"2023-11-06T02:08:26.059262+0000","last_became_peered":"2023-11-06T02:08:26.059262+0000","last_unstale":"2023-11-06T02:10:31.543715+0000","last_undegraded":"2023-11-06T02:10:31.543715+0000","last_fullsized":"2023-11-06T02:10:31.543715+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:10.626021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.936129+0000","last_change":"2023-11-06T02:10:31.927287+0000","last_active":"2023-11-06T02:10:31.936129+0000","last_peered":"2023-11-06T02:10:31.936129+0000","last_clean":"2023-11-06T02:10:31.936129+0000","last_became_active":"2023-11-06T02:10:31.927001+0000","last_became_peered":"2023-11-06T02:10:31.927001+0000","last_unstale":"2023-11-06T02:10:31.936129+0000","last_undegraded":"2023-11-06T02:10:31.936129+0000","last_fullsized":"2023-11-06T02:10:31.936129+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:57:03.504241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.922810+0000","last_change":"2023-11-06T02:10:31.532950+0000","last_active":"2023-11-06T02:10:31.922810+0000","last_peered":"2023-11-06T02:10:31.922810+0000","last_clean":"2023-11-06T02:10:31.922810+0000","last_became_active":"2023-11-06T02:10:31.532290+0000","last_became_peered":"2023-11-06T02:10:31.532290+0000","last_unstale":"2023-11-06T02:10:31.922810+0000","last_undegraded":"2023-11-06T02:10:31.922810+0000","last_fullsized":"2023-11-06T02:10:31.922810+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_clean_scrub_stamp":"2023-11-06T02:10:30.509051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:07.035669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":40,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-06T02:10:32.921724+0000","last_change":"2023-11-06T02:08:26.559477+0000","last_active":"2023-11-06T02:10:32.921724+0000","last_peered":"2023-11-06T02:10:32.921724+0000","last_clean":"2023-11-06T02:10:32.921724+0000","last_became_active":"2023-11-06T02:08:26.559202+0000","last_became_peered":"2023-11-06T02:08:26.559202+0000","last_unstale":"2023-11-06T02:10:32.921724+0000","last_undegraded":"2023-11-06T02:10:32.921724+0000","last_fullsized":"2023-11-06T02:10:32.921724+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:55:04.552404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-06T02:10:32.927313+0000","last_change":"2023-11-06T02:10:31.970063+0000","last_active":"2023-11-06T02:10:32.927313+0000","last_peered":"2023-11-06T02:10:32.927313+0000","last_clean":"2023-11-06T02:10:32.927313+0000","last_became_active":"2023-11-06T02:10:31.966656+0000","last_became_peered":"2023-11-06T02:10:31.966656+0000","last_unstale":"2023-11-06T02:10:32.927313+0000","last_undegraded":"2023-11-06T02:10:32.927313+0000","last_fullsized":"2023-11-06T02:10:32.927313+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:31:52.575652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"50'2","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-06T02:10:32.926861+0000","last_change":"2023-11-06T02:10:31.548283+0000","last_active":"2023-11-06T02:10:32.926861+0000","last_peered":"2023-11-06T02:10:32.926861+0000","last_clean":"2023-11-06T02:10:32.926861+0000","last_became_active":"2023-11-06T02:10:31.548058+0000","last_became_peered":"2023-11-06T02:10:31.548058+0000","last_unstale":"2023-11-06T02:10:32.926861+0000","last_undegraded":"2023-11-06T02:10:32.926861+0000","last_fullsized":"2023-11-06T02:10:32.926861+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_clean_scrub_stamp":"2023-11-06T02:10:30.509051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:54:46.730754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":39,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-06T02:10:32.927747+0000","last_change":"2023-11-06T02:08:26.059546+0000","last_active":"2023-11-06T02:10:32.927747+0000","last_peered":"2023-11-06T02:10:32.927747+0000","last_clean":"2023-11-06T02:10:32.927747+0000","last_became_active":"2023-11-06T02:08:26.058806+0000","last_became_peered":"2023-11-06T02:08:26.058806+0000","last_unstale":"2023-11-06T02:10:32.927747+0000","last_undegraded":"2023-11-06T02:10:32.927747+0000","last_fullsized":"2023-11-06T02:10:32.927747+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:23.468510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.950288+0000","last_change":"2023-11-06T02:10:31.942711+0000","last_active":"2023-11-06T02:10:31.950288+0000","last_peered":"2023-11-06T02:10:31.950288+0000","last_clean":"2023-11-06T02:10:31.950288+0000","last_became_active":"2023-11-06T02:10:31.942440+0000","last_became_peered":"2023-11-06T02:10:31.942440+0000","last_unstale":"2023-11-06T02:10:31.950288+0000","last_undegraded":"2023-11-06T02:10:31.950288+0000","last_fullsized":"2023-11-06T02:10:31.950288+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:41:22.735337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.918807+0000","last_change":"2023-11-06T02:10:31.556959+0000","last_active":"2023-11-06T02:10:31.918807+0000","last_peered":"2023-11-06T02:10:31.918807+0000","last_clean":"2023-11-06T02:10:31.918807+0000","last_became_active":"2023-11-06T02:10:31.556286+0000","last_became_peered":"2023-11-06T02:10:31.556286+0000","last_unstale":"2023-11-06T02:10:31.918807+0000","last_undegraded":"2023-11-06T02:10:31.918807+0000","last_fullsized":"2023-11-06T02:10:31.918807+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_clean_scrub_stamp":"2023-11-06T02:10:30.509051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:07:02.686107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":36,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.928429+0000","last_change":"2023-11-06T02:08:26.436463+0000","last_active":"2023-11-06T02:10:31.928429+0000","last_peered":"2023-11-06T02:10:31.928429+0000","last_clean":"2023-11-06T02:10:31.928429+0000","last_became_active":"2023-11-06T02:08:26.436075+0000","last_became_peered":"2023-11-06T02:08:26.436075+0000","last_unstale":"2023-11-06T02:10:31.928429+0000","last_undegraded":"2023-11-06T02:10:31.928429+0000","last_fullsized":"2023-11-06T02:10:31.928429+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:23.468510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.946216+0000","last_change":"2023-11-06T02:10:31.931904+0000","last_active":"2023-11-06T02:10:31.946216+0000","last_peered":"2023-11-06T02:10:31.946216+0000","last_clean":"2023-11-06T02:10:31.946216+0000","last_became_active":"2023-11-06T02:10:31.931454+0000","last_became_peered":"2023-11-06T02:10:31.931454+0000","last_unstale":"2023-11-06T02:10:31.946216+0000","last_undegraded":"2023-11-06T02:10:31.946216+0000","last_fullsized":"2023-11-06T02:10:31.946216+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:41:22.735337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-06T02:10:32.927633+0000","last_change":"2023-11-06T02:10:31.555697+0000","last_active":"2023-11-06T02:10:32.927633+0000","last_peered":"2023-11-06T02:10:32.927633+0000","last_clean":"2023-11-06T02:10:32.927633+0000","last_became_active":"2023-11-06T02:10:31.555004+0000","last_became_peered":"2023-11-06T02:10:31.555004+0000","last_unstale":"2023-11-06T02:10:32.927633+0000","last_undegraded":"2023-11-06T02:10:32.927633+0000","last_fullsized":"2023-11-06T02:10:32.927633+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_clean_scrub_stamp":"2023-11-06T02:10:30.509051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:07.035669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":38,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.935384+0000","last_change":"2023-11-06T02:08:26.436607+0000","last_active":"2023-11-06T02:10:31.935384+0000","last_peered":"2023-11-06T02:10:31.935384+0000","last_clean":"2023-11-06T02:10:31.935384+0000","last_became_active":"2023-11-06T02:08:26.435924+0000","last_became_peered":"2023-11-06T02:08:26.435924+0000","last_unstale":"2023-11-06T02:10:31.935384+0000","last_undegraded":"2023-11-06T02:10:31.935384+0000","last_fullsized":"2023-11-06T02:10:31.935384+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:23.468510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.947879+0000","last_change":"2023-11-06T02:10:31.931111+0000","last_active":"2023-11-06T02:10:31.947879+0000","last_peered":"2023-11-06T02:10:31.947879+0000","last_clean":"2023-11-06T02:10:31.947879+0000","last_became_active":"2023-11-06T02:10:31.930621+0000","last_became_peered":"2023-11-06T02:10:31.930621+0000","last_unstale":"2023-11-06T02:10:31.947879+0000","last_undegraded":"2023-11-06T02:10:31.947879+0000","last_fullsized":"2023-11-06T02:10:31.947879+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:00:32.003705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.912533+0000","last_change":"2023-11-06T02:10:31.554303+0000","last_active":"2023-11-06T02:10:31.912533+0000","last_peered":"2023-11-06T02:10:31.912533+0000","last_clean":"2023-11-06T02:10:31.912533+0000","last_became_active":"2023-11-06T02:10:31.553183+0000","last_became_peered":"2023-11-06T02:10:31.553183+0000","last_unstale":"2023-11-06T02:10:31.912533+0000","last_undegraded":"2023-11-06T02:10:31.912533+0000","last_fullsized":"2023-11-06T02:10:31.912533+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_clean_scrub_stamp":"2023-11-06T02:10:30.509051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:54:02.564043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":35,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.549282+0000","last_change":"2023-11-06T02:08:26.057076+0000","last_active":"2023-11-06T02:10:31.549282+0000","last_peered":"2023-11-06T02:10:31.549282+0000","last_clean":"2023-11-06T02:10:31.549282+0000","last_became_active":"2023-11-06T02:08:26.056419+0000","last_became_peered":"2023-11-06T02:08:26.056419+0000","last_unstale":"2023-11-06T02:10:31.549282+0000","last_undegraded":"2023-11-06T02:10:31.549282+0000","last_fullsized":"2023-11-06T02:10:31.549282+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:10.626021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-06T02:10:32.928078+0000","last_change":"2023-11-06T02:10:31.970623+0000","last_active":"2023-11-06T02:10:32.928078+0000","last_peered":"2023-11-06T02:10:32.928078+0000","last_clean":"2023-11-06T02:10:32.928078+0000","last_became_active":"2023-11-06T02:10:31.969840+0000","last_became_peered":"2023-11-06T02:10:31.969840+0000","last_unstale":"2023-11-06T02:10:32.928078+0000","last_undegraded":"2023-11-06T02:10:32.928078+0000","last_fullsized":"2023-11-06T02:10:32.928078+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:31.512773+0000","last_clean_scrub_stamp":"2023-11-06T02:10:31.512773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:00:38.993498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.912300+0000","last_change":"2023-11-06T02:10:31.553319+0000","last_active":"2023-11-06T02:10:31.912300+0000","last_peered":"2023-11-06T02:10:31.912300+0000","last_clean":"2023-11-06T02:10:31.912300+0000","last_became_active":"2023-11-06T02:10:31.552876+0000","last_became_peered":"2023-11-06T02:10:31.552876+0000","last_unstale":"2023-11-06T02:10:31.912300+0000","last_undegraded":"2023-11-06T02:10:31.912300+0000","last_fullsized":"2023-11-06T02:10:31.912300+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:10:30.509051+0000","last_clean_scrub_stamp":"2023-11-06T02:10:30.509051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:46.288303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":42,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-06T02:10:31.917574+0000","last_change":"2023-11-06T02:08:26.058596+0000","last_active":"2023-11-06T02:10:31.917574+0000","last_peered":"2023-11-06T02:10:31.917574+0000","last_clean":"2023-11-06T02:10:31.917574+0000","last_became_active":"2023-11-06T02:08:26.057015+0000","last_became_peered":"2023-11-06T02:08:26.057015+0000","last_unstale":"2023-11-06T02:10:31.917574+0000","last_undegraded":"2023-11-06T02:10:31.917574+0000","last_fullsized":"2023-11-06T02:10:31.917574+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:55:04.552404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574878,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28116,"kb_used_data":1164,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343724,"statfs":{"total":96636764160,"available":96607973376,"internally_reserved":0,"allocated":1191936,"data_stored":844490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:10:20 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.86799999999999999,"15min":0.85499999999999998},"min":{"1min":0.60499999999999998,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.421,"5min":1.506,"15min":1.506},"last":0.90200000000000002}]},{"osd":1,"last update":"Mon Nov 6 02:10:20 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.84199999999999997,"15min":0.84799999999999998},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.1399999999999999,"5min":1.161,"15min":1.161},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.90400000000000003,"15min":0.89400000000000002},"min":{"1min":0.76300000000000001,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.67800000000000005}]},{"osd":2,"last update":"Mon Nov 6 02:10:20 2023","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":0.96699999999999997,"15min":0.95799999999999996},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.99099999999999999,"15min":0.997},"min":{"1min":0.74399999999999999,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.3420000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.622}]},{"osd":3,"last update":"Mon Nov 6 02:10:20 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.875,"15min":0.875},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.083,"15min":1.079},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.526,"5min":1.526,"15min":1.526},"last":0.84699999999999998}]},{"osd":4,"last update":"Mon Nov 6 02:10:20 2023","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":0.96499999999999997,"15min":0.95999999999999996},"min":{"1min":0.70699999999999996,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.97299999999999998,"5min":1.03,"15min":1.04},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.2170000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":1.115}]},{"osd":5,"last update":"Mon Nov 6 02:10:20 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.093,"15min":1.083},"min":{"1min":0.89300000000000002,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":0.94699999999999995},{"interface":"front","average":{"1min":1.101,"5min":1.129,"15min":1.133},"min":{"1min":0.86599999999999999,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.3180000000000001,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":0.98799999999999999}]},{"osd":6,"last update":"Mon Nov 6 02:10:20 2023","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":1.2030000000000001,"15min":1.202},"min":{"1min":0.87,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.4930000000000001,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":1.0660000000000001},{"interface":"front","average":{"1min":1.133,"5min":1.153,"15min":1.1559999999999999},"min":{"1min":0.79500000000000004,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.421,"5min":1.74,"15min":1.74},"last":1.1719999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574877,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27536,"kb_used_data":584,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344304,"statfs":{"total":96636764160,"available":96608567296,"internally_reserved":0,"allocated":598016,"data_stored":254650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:10:22 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.84699999999999998,"15min":0.85399999999999998},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.0880000000000001,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.82299999999999995,"15min":0.81100000000000005},"min":{"1min":0.55300000000000005,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.52000000000000002}]},{"osd":1,"last update":"Mon Nov 6 02:10:22 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.86299999999999999,"15min":0.871},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.1100000000000001,"5min":1.179,"15min":1.179},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.876,"15min":0.874},"min":{"1min":0.60999999999999999,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.3200000000000001,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.98999999999999999}]},{"osd":2,"last update":"Mon Nov 6 02:10:22 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.88600000000000001,"15min":0.89400000000000002},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.087,"5min":1.173,"15min":1.173},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.92100000000000004,"15min":0.92700000000000005},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.1339999999999999,"5min":1.28,"15min":1.28},"last":0.72299999999999998}]},{"osd":3,"last update":"Mon Nov 6 02:10:22 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.95499999999999996,"15min":0.96599999999999997},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":1.05},{"interface":"front","average":{"1min":0.96599999999999997,"5min":0.99399999999999999,"15min":0.998},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.2310000000000001,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.752}]},{"osd":4,"last update":"Mon Nov 6 02:10:22 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.96499999999999997,"15min":0.98399999999999999},"min":{"1min":0.56799999999999995,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.1459999999999999,"5min":1.474,"15min":1.474},"last":1.097},{"interface":"front","average":{"1min":0.95799999999999996,"5min":1.0229999999999999,"15min":1.034},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.2989999999999999,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.88600000000000001}]},{"osd":5,"last update":"Mon Nov 6 02:10:22 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":1.0069999999999999,"15min":1.0209999999999999},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.2769999999999999,"5min":1.365,"15min":1.365},"last":0.79900000000000004},{"interface":"front","average":{"1min":1.03,"5min":1.0980000000000001,"15min":1.109},"min":{"1min":0.66900000000000004,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.234,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":1.145}]},{"osd":7,"last update":"Mon Nov 6 02:10:22 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.1060000000000001,"15min":1.1140000000000001},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.3460000000000001,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.98699999999999999,"5min":1.129,"15min":1.153},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.2,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.91800000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574878,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27528,"kb_used_data":576,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":589824,"data_stored":253998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:10:23 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.84499999999999997,"15min":0.83399999999999996},"min":{"1min":0.57399999999999995,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.381,"5min":1.381,"15min":1.381},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.57899999999999996,"5min":0.503,"15min":0.503},"max":{"1min":1.1739999999999999,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":1.3200000000000001}]},{"osd":1,"last update":"Mon Nov 6 02:10:23 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.85599999999999998,"15min":0.85199999999999998},"min":{"1min":0.64100000000000001,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.2190000000000001,"5min":1.331,"15min":1.331},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.82599999999999996,"15min":0.81999999999999995},"min":{"1min":0.56899999999999995,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.177,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":1.0289999999999999}]},{"osd":2,"last update":"Mon Nov 6 02:10:23 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.93600000000000005,"15min":0.94199999999999995},"min":{"1min":0.58999999999999997,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.1639999999999999,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.81200000000000006,"15min":0.79700000000000004},"min":{"1min":0.58999999999999997,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.2909999999999999,"5min":1.377,"15min":1.377},"last":1.0940000000000001}]},{"osd":3,"last update":"Mon Nov 6 02:10:23 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.91500000000000004,"15min":0.91800000000000004},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.143,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":1.0649999999999999},{"interface":"front","average":{"1min":0.96299999999999997,"5min":1.0569999999999999,"15min":1.073},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.349,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":1.141}]},{"osd":4,"last update":"Mon Nov 6 02:10:23 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.92400000000000004,"15min":0.91800000000000004},"min":{"1min":0.625,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.276,"5min":1.276,"15min":1.276},"last":0.98999999999999999},{"interface":"front","average":{"1min":1.01,"5min":1.024,"15min":1.026},"min":{"1min":0.68899999999999995,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.2370000000000001,"5min":1.554,"15min":1.554},"last":1.25}]},{"osd":6,"last update":"Mon Nov 6 02:10:23 2023","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":1.052,"15min":1.0620000000000001},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.409,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.95299999999999996},{"interface":"front","average":{"1min":1.002,"5min":1.0329999999999999,"15min":1.038},"min":{"1min":0.68400000000000005,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":1.171}]},{"osd":7,"last update":"Mon Nov 6 02:10:23 2023","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.0780000000000001,"15min":1.0760000000000001},"min":{"1min":0.71099999999999997,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.3140000000000001,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.125,"5min":1.149,"15min":1.153},"min":{"1min":0.80600000000000005,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.472,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":1.2250000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574878,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27532,"kb_used_data":580,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":593920,"data_stored":254580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:10:19 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.83299999999999996,"15min":0.82699999999999996},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.1679999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.876,"5min":0.79100000000000004,"15min":0.77700000000000002},"min":{"1min":0.65900000000000003,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.28,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.50600000000000001}]},{"osd":1,"last update":"Mon Nov 6 02:10:19 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.85799999999999998,"15min":0.84799999999999998},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.462,"5min":1.462,"15min":1.462},"last":1.024},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.90900000000000003,"15min":0.89700000000000002},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.91800000000000004}]},{"osd":2,"last update":"Mon Nov 6 02:10:19 2023","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.996,"15min":1.012},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.389,"5min":1.615,"15min":1.615},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.95399999999999996,"5min":0.93100000000000005,"15min":0.92700000000000005},"min":{"1min":0.70999999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.4379999999999999,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":0.997}]},{"osd":3,"last update":"Mon Nov 6 02:10:19 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.22,"5min":1.52,"15min":1.52},"last":0.86599999999999999},{"interface":"front","average":{"1min":1.012,"5min":1.0580000000000001,"15min":1.0660000000000001},"min":{"1min":0.69099999999999995,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.595,"5min":1.595,"15min":1.595},"last":0.95099999999999996}]},{"osd":5,"last update":"Mon Nov 6 02:10:19 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.046,"15min":1.0489999999999999},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.2949999999999999,"5min":1.377,"15min":1.377},"last":1.115},{"interface":"front","average":{"1min":0.94699999999999995,"5min":1.034,"15min":1.048},"min":{"1min":0.64300000000000002,"5min":0.629,"15min":0.629},"max":{"1min":1.2310000000000001,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":1.135}]},{"osd":6,"last update":"Mon Nov 6 02:10:19 2023","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":1.0760000000000001,"15min":1.0880000000000001},"min":{"1min":0.71999999999999997,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.2390000000000001,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":1.085},{"interface":"front","average":{"1min":1.123,"5min":1.1830000000000001,"15min":1.1930000000000001},"min":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":1.425,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":1.2070000000000001}]},{"osd":7,"last update":"Mon Nov 6 02:10:19 2023","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.089,"15min":1.0920000000000001},"min":{"1min":0.88500000000000001,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.3029999999999999,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":1.054},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.21,"15min":1.23},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.3140000000000001,"5min":1.7410000000000001,"15min":1.7410000000000001},"last":1.1759999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574877,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31640,"kb_used_data":592,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340200,"statfs":{"total":96636764160,"available":96604364800,"internally_reserved":0,"allocated":606208,"data_stored":255584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:10:22 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.63500000000000001,"15min":0.64300000000000002},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.80600000000000005,"5min":1.079,"15min":1.079},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.625,"15min":0.625},"min":{"1min":0.35999999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.50700000000000001}]},{"osd":1,"last update":"Mon Nov 6 02:10:22 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.61599999999999999,"15min":0.626},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.82499999999999996,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.72099999999999997,"15min":0.72899999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.94199999999999995,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.435}]},{"osd":2,"last update":"Mon Nov 6 02:10:22 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.72599999999999998,"15min":0.73199999999999998},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.0389999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.374},{"interface":"front","average":{"1min":0.745,"5min":0.80800000000000005,"15min":0.81799999999999995},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":1.405,"5min":1.405,"15min":1.405},"last":0.47199999999999998}]},{"osd":4,"last update":"Mon Nov 6 02:10:22 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.92400000000000004,"15min":0.92700000000000005},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.873},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.97899999999999998,"15min":0.99199999999999999},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.238,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.67800000000000005}]},{"osd":5,"last update":"Mon Nov 6 02:10:22 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":1.0109999999999999,"15min":1.0229999999999999},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.95699999999999996,"5min":1.0569999999999999,"15min":1.073},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.3580000000000001,"5min":2.21,"15min":2.21},"last":0.88800000000000001}]},{"osd":6,"last update":"Mon Nov 6 02:10:22 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":1.016,"15min":1.022},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.395,"5min":1.395,"15min":1.395},"last":0.76900000000000002},{"interface":"front","average":{"1min":1,"5min":1.0569999999999999,"15min":1.0669999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.2669999999999999,"5min":1.554,"15min":1.554},"last":0.71699999999999997}]},{"osd":7,"last update":"Mon Nov 6 02:10:22 2023","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":1.107,"15min":1.1240000000000001},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.4379999999999999,"5min":2.2829999999999999,"15min":2.2829999999999999},"last":1.0720000000000001},{"interface":"front","average":{"1min":1.048,"5min":1.095,"15min":1.103},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.3169999999999999,"5min":1.498,"15min":1.498},"last":0.92200000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574877,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31636,"kb_used_data":588,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340204,"statfs":{"total":96636764160,"available":96604368896,"internally_reserved":0,"allocated":602112,"data_stored":255562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:10:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.51000000000000001,"15min":0.502},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":1.079},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.54700000000000004,"15min":0.54100000000000004},"min":{"1min":0.316,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.90900000000000003,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.59899999999999998}]},{"osd":1,"last update":"Mon Nov 6 02:10:19 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.57399999999999995,"15min":0.56399999999999995},"min":{"1min":0.34799999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.0800000000000001,"5min":1.409,"15min":1.409},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.59499999999999997,"15min":0.57699999999999996},"min":{"1min":0.38100000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.083,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.53300000000000003}]},{"osd":3,"last update":"Mon Nov 6 02:10:19 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.61799999999999999,"15min":0.60399999999999998},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.121,"5min":1.155,"15min":1.155},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.65800000000000003,"15min":0.64700000000000002},"min":{"1min":0.36099999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.628}]},{"osd":4,"last update":"Mon Nov 6 02:10:19 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.81799999999999995,"15min":0.80500000000000005},"min":{"1min":0.56899999999999995,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.91200000000000003,"15min":0.91400000000000003},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.204,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":1.018}]},{"osd":5,"last update":"Mon Nov 6 02:10:19 2023","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.94799999999999995,"15min":0.94399999999999995},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.2390000000000001,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.96799999999999997},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.93999999999999995,"15min":0.93600000000000005},"min":{"1min":0.60999999999999999,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.4790000000000001,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.80800000000000005}]},{"osd":6,"last update":"Mon Nov 6 02:10:19 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.202,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":1.123},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.995,"15min":0.996},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.4199999999999999,"5min":1.677,"15min":1.677},"last":1.105}]},{"osd":7,"last update":"Mon Nov 6 02:10:19 2023","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.97899999999999998,"15min":0.97599999999999998},"min":{"1min":0.75900000000000001,"5min":0.622,"15min":0.622},"max":{"1min":1.2749999999999999,"5min":1.633,"15min":1.633},"last":1.1759999999999999},{"interface":"front","average":{"1min":1.048,"5min":1.0429999999999999,"15min":1.042},"min":{"1min":0.60799999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.345,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":1.0640000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574878,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31632,"kb_used_data":584,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340208,"statfs":{"total":96636764160,"available":96604372992,"internally_reserved":0,"allocated":598016,"data_stored":254158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:10:21 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.621,"15min":0.624},"min":{"1min":0.35699999999999998,"5min":0.224,"15min":0.224},"max":{"1min":0.85999999999999999,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65300000000000002,"15min":0.65200000000000002},"min":{"1min":0.443,"5min":0.315,"15min":0.315},"max":{"1min":0.90700000000000003,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.29199999999999998}]},{"osd":2,"last update":"Mon Nov 6 02:10:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.61599999999999999,"15min":0.62},"min":{"1min":0.34200000000000003,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.0109999999999999,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.504},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.73499999999999999,"15min":0.74099999999999999},"min":{"1min":0.39300000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.27400000000000002}]},{"osd":3,"last update":"Mon Nov 6 02:10:21 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.79500000000000004,"15min":0.80000000000000004},"min":{"1min":0.42699999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.98299999999999998,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.78900000000000003,"15min":0.80300000000000005},"min":{"1min":0.379,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.98799999999999999,"5min":1.8149999999999999,"15min":1.8149999999999999},"last":0.42699999999999999}]},{"osd":4,"last update":"Mon Nov 6 02:10:21 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.97799999999999998,"15min":0.98799999999999999},"min":{"1min":0.63200000000000001,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.1619999999999999,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.98299999999999998,"15min":0.98999999999999999},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.254,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.623}]},{"osd":5,"last update":"Mon Nov 6 02:10:21 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.96199999999999997,"15min":0.96499999999999997},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.196,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.58599999999999997},{"interface":"front","average":{"1min":1.024,"5min":1.0680000000000001,"15min":1.075},"min":{"1min":0.64600000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.4199999999999999,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.66500000000000004}]},{"osd":6,"last update":"Mon Nov 6 02:10:21 2023","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.087,"15min":1.097},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.296,"5min":1.742,"15min":1.742},"last":0.61199999999999999},{"interface":"front","average":{"1min":1.034,"5min":1.0680000000000001,"15min":1.0740000000000001},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.248,"5min":1.393,"15min":1.393},"last":0.70399999999999996}]},{"osd":7,"last update":"Mon Nov 6 02:10:21 2023","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":1.111,"15min":1.1180000000000001},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":1.081,"5min":1.145,"15min":1.155},"min":{"1min":0.70399999999999996,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.403,"5min":1.893,"15min":1.893},"last":0.76800000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574878,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32208,"kb_used_data":1160,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339632,"statfs":{"total":96636764160,"available":96603783168,"internally_reserved":0,"allocated":1187840,"data_stored":843403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 02:10:24 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.58799999999999997,"15min":0.59399999999999997},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.90100000000000002,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.432},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.63900000000000001,"15min":0.63700000000000001},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.39300000000000002}]},{"osd":2,"last update":"Mon Nov 6 02:10:24 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.65200000000000002,"15min":0.66000000000000003},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.34499999999999997,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.47599999999999998}]},{"osd":3,"last update":"Mon Nov 6 02:10:24 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.625,"15min":0.627},"min":{"1min":0.33100000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.429,"5min":1.429,"15min":1.429},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.73299999999999998,"15min":0.73099999999999998},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.61799999999999999}]},{"osd":4,"last update":"Mon Nov 6 02:10:24 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.83899999999999997,"15min":0.83099999999999996},"min":{"1min":0.55400000000000005,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.341,"5min":1.341,"15min":1.341},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.93200000000000005,"15min":0.93500000000000005},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.3839999999999999,"5min":1.387,"15min":1.387},"last":0.69099999999999995}]},{"osd":5,"last update":"Mon Nov 6 02:10:24 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.94899999999999995,"15min":0.94799999999999995},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.3140000000000001,"5min":1.339,"15min":1.339},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.88100000000000001,"15min":0.875},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.351,"5min":1.448,"15min":1.448},"last":0.77400000000000002}]},{"osd":6,"last update":"Mon Nov 6 02:10:24 2023","interfaces":[{"interface":"back","average":{"1min":1,"5min":1.0289999999999999,"15min":1.034},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.581,"5min":1.589,"15min":1.589},"last":0.83899999999999997},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.0109999999999999,"15min":1.0029999999999999},"min":{"1min":0.68899999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.5529999999999999,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.67600000000000005}]},{"osd":7,"last update":"Mon Nov 6 02:10:24 2023","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":1.0069999999999999,"15min":1.0149999999999999},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.421,"5min":1.421,"15min":1.421},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.99399999999999999,"15min":1.004},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.4950000000000001,"5min":1.556,"15min":1.556},"last":0.70999999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T02:10:36.341 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-06T02:10:36.341 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-06T02:10:36.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.503+0000 7f5b3c512640 1 Processor -- start 2023-11-06T02:10:36.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.503+0000 7f5b3c512640 1 -- start start 2023-11-06T02:10:36.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.503+0000 7f5b3c512640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5b3415a880 0x7f5b340b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:36.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.507+0000 7f5b3c512640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f5b34070160 0x7f5b340705e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:36.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.507+0000 7f5b3c512640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b340ae360 con 0x7f5b3406d150 2023-11-06T02:10:36.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.507+0000 7f5b3c512640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b34070b20 con 0x7f5b34070160 2023-11-06T02:10:36.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.507+0000 7f5b3c512640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b340ae640 con 0x7f5b3415a880 2023-11-06T02:10:36.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.507+0000 7f5b39aad640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f5b34070160 0x7f5b340705e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:36.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.507+0000 7f5b3aaaf640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f5b3406d150 0x7f5b3406d5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:41564/0 (socket says 172.21.15.49:41564) 2023-11-06T02:10:36.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.507+0000 7f5b3aaaf640 1 -- 172.21.15.49:0/40223961 learned_addr learned my addr 172.21.15.49:0/40223961 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:10:36.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.507+0000 7f5b392ac640 1 -- 172.21.15.49:0/40223961 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 725510247 0 0) 0x7f5b340ae360 con 0x7f5b3406d150 2023-11-06T02:10:36.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.507+0000 7f5b3a2ae640 1 --2- 172.21.15.49:0/40223961 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5b3415a880 0x7f5b340b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:36.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.507+0000 7f5b392ac640 1 -- 172.21.15.49:0/40223961 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b20003530 con 0x7f5b3406d150 2023-11-06T02:10:36.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.507+0000 7f5b39aad640 1 -- 172.21.15.49:0/40223961 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5b3415a880 msgr2=0x7f5b340b15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:36.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.507+0000 7f5b39aad640 1 --2- 172.21.15.49:0/40223961 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5b3415a880 0x7f5b340b15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:36.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.507+0000 7f5b39aad640 1 -- 172.21.15.49:0/40223961 >> v1:172.21.15.49:6789/0 conn(0x7f5b3406d150 legacy=0x7f5b3406d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:36.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.507+0000 7f5b39aad640 1 -- 172.21.15.49:0/40223961 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b340ae920 con 0x7f5b34070160 2023-11-06T02:10:36.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.507+0000 7f5b39aad640 1 --2- 172.21.15.49:0/40223961 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b34070160 0x7f5b340705e0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f5b240097c0 tx=0x7f5b2401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67fef3ab8664e4fe server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:36.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.507+0000 7f5b392ac640 1 -- 172.21.15.49:0/40223961 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b240ad640 con 0x7f5b34070160 2023-11-06T02:10:36.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.507+0000 7f5b392ac640 1 -- 172.21.15.49:0/40223961 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b240ad7a0 con 0x7f5b34070160 2023-11-06T02:10:36.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.507+0000 7f5b392ac640 1 -- 172.21.15.49:0/40223961 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b240adb10 con 0x7f5b34070160 2023-11-06T02:10:36.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.507+0000 7f5b3c512640 1 -- 172.21.15.49:0/40223961 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b34070160 msgr2=0x7f5b340705e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:36.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.507+0000 7f5b3c512640 1 --2- 172.21.15.49:0/40223961 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b34070160 0x7f5b340705e0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f5b240097c0 tx=0x7f5b2401c480 comp rx=0 tx=0).stop 2023-11-06T02:10:36.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.507+0000 7f5b3c512640 1 -- 172.21.15.49:0/40223961 shutdown_connections 2023-11-06T02:10:36.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.507+0000 7f5b3c512640 1 --2- 172.21.15.49:0/40223961 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b34070160 0x7f5b340705e0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:36.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.507+0000 7f5b3c512640 1 --2- 172.21.15.49:0/40223961 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5b3415a880 0x7f5b340b15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:36.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.507+0000 7f5b3c512640 1 -- 172.21.15.49:0/40223961 >> 172.21.15.49:0/40223961 conn(0x7f5b3406a370 msgr2=0x7f5b340ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:36.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.507+0000 7f5b3c512640 1 -- 172.21.15.49:0/40223961 shutdown_connections 2023-11-06T02:10:36.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.507+0000 7f5b3c512640 1 -- 172.21.15.49:0/40223961 wait complete. 2023-11-06T02:10:36.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.511+0000 7f5b3c512640 1 Processor -- start 2023-11-06T02:10:36.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.511+0000 7f5b3c512640 1 -- start start 2023-11-06T02:10:36.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.511+0000 7f5b3c512640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5b3406d150 0x7f5b341502e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:36.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.511+0000 7f5b3c512640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b34070160 0x7f5b340c5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:36.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.511+0000 7f5b3c512640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b3415a880 0x7f5b3415b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:36.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.511+0000 7f5b3c512640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b340c5e50 con 0x7f5b34070160 2023-11-06T02:10:36.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.511+0000 7f5b3c512640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b3414ea70 con 0x7f5b3415a880 2023-11-06T02:10:36.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.511+0000 7f5b3c512640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b3414ed10 con 0x7f5b3406d150 2023-11-06T02:10:36.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.511+0000 7f5b39aad640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b34070160 0x7f5b340c5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:36.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.511+0000 7f5b39aad640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b34070160 0x7f5b340c5910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44134/0 (socket says 172.21.15.49:44134) 2023-11-06T02:10:36.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.511+0000 7f5b39aad640 1 -- 172.21.15.49:0/2696724355 learned_addr learned my addr 172.21.15.49:0/2696724355 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:36.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.511+0000 7f5b3a2ae640 1 --2- 172.21.15.49:0/2696724355 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5b3406d150 0x7f5b341502e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:36.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.511+0000 7f5b3aaaf640 1 --2- 172.21.15.49:0/2696724355 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b3415a880 0x7f5b3415b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:36.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.511+0000 7f5b39aad640 1 -- 172.21.15.49:0/2696724355 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5b3406d150 msgr2=0x7f5b341502e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:36.514 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.511+0000 7f5b39aad640 1 --2- 172.21.15.49:0/2696724355 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5b3406d150 0x7f5b341502e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:36.514 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.511+0000 7f5b39aad640 1 -- 172.21.15.49:0/2696724355 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b3415a880 msgr2=0x7f5b3415b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:36.514 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.511+0000 7f5b39aad640 1 --2- 172.21.15.49:0/2696724355 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b3415a880 0x7f5b3415b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:36.515 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.511+0000 7f5b39aad640 1 -- 172.21.15.49:0/2696724355 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b24009420 con 0x7f5b34070160 2023-11-06T02:10:36.515 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.511+0000 7f5b3aaaf640 1 --2- 172.21.15.49:0/2696724355 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b3415a880 0x7f5b3415b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:10:36.515 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.511+0000 7f5b39aad640 1 --2- 172.21.15.49:0/2696724355 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b34070160 0x7f5b340c5910 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f5b24000c10 tx=0x7f5b24019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:36.515 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.511+0000 7f5b2affd640 1 -- 172.21.15.49:0/2696724355 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b240add20 con 0x7f5b34070160 2023-11-06T02:10:36.515 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.511+0000 7f5b2affd640 1 -- 172.21.15.49:0/2696724355 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b240a6070 con 0x7f5b34070160 2023-11-06T02:10:36.516 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.511+0000 7f5b2affd640 1 -- 172.21.15.49:0/2696724355 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b24003820 con 0x7f5b34070160 2023-11-06T02:10:36.516 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.511+0000 7f5b3c512640 1 -- 172.21.15.49:0/2696724355 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b3414eff0 con 0x7f5b34070160 2023-11-06T02:10:36.516 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.511+0000 7f5b3c512640 1 -- 172.21.15.49:0/2696724355 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b3414d1a0 con 0x7f5b34070160 2023-11-06T02:10:36.516 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.515+0000 7f5b3c512640 1 -- 172.21.15.49:0/2696724355 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b3414f4a0 con 0x7f5b34070160 2023-11-06T02:10:36.517 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.515+0000 7f5b2affd640 1 -- 172.21.15.49:0/2696724355 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f5b240b4040 con 0x7f5b34070160 2023-11-06T02:10:36.518 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.515+0000 7f5b2affd640 1 --2- 172.21.15.49:0/2696724355 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5b18065a90 0x7f5b18067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:36.518 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.515+0000 7f5b2affd640 1 -- 172.21.15.49:0/2696724355 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5b2411d3a0 con 0x7f5b34070160 2023-11-06T02:10:36.518 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.515+0000 7f5b3a2ae640 1 --2- 172.21.15.49:0/2696724355 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5b18065a90 0x7f5b18067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:36.518 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.515+0000 7f5b3a2ae640 1 --2- 172.21.15.49:0/2696724355 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5b18065a90 0x7f5b18067f50 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f5b340c8c50 tx=0x7f5b30008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:36.518 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.519+0000 7f5b2affd640 1 -- 172.21.15.49:0/2696724355 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f5b3414f4a0 con 0x7f5b34070160 2023-11-06T02:10:36.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.731+0000 7f5b3c512640 1 -- 172.21.15.49:0/2696724355 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f5b3414f4a0 con 0x7f5b34070160 2023-11-06T02:10:36.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.735+0000 7f5b2affd640 1 -- 172.21.15.49:0/2696724355 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v12) v1 ==== 190+0+0 (secure 0 0 0) 0x7f5b3414f4a0 con 0x7f5b34070160 2023-11-06T02:10:36.732 INFO:teuthology.orchestra.run.smithi049.stderr:updated caps for client.0 2023-11-06T02:10:36.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.739+0000 7f5b3c512640 1 -- 172.21.15.49:0/2696724355 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5b18065a90 msgr2=0x7f5b18067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:36.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.739+0000 7f5b3c512640 1 --2- 172.21.15.49:0/2696724355 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5b18065a90 0x7f5b18067f50 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f5b340c8c50 tx=0x7f5b30008040 comp rx=0 tx=0).stop 2023-11-06T02:10:36.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.739+0000 7f5b3c512640 1 -- 172.21.15.49:0/2696724355 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b34070160 msgr2=0x7f5b340c5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:36.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.739+0000 7f5b3c512640 1 --2- 172.21.15.49:0/2696724355 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b34070160 0x7f5b340c5910 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f5b24000c10 tx=0x7f5b24019220 comp rx=0 tx=0).stop 2023-11-06T02:10:36.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.739+0000 7f5b3c512640 1 -- 172.21.15.49:0/2696724355 shutdown_connections 2023-11-06T02:10:36.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.739+0000 7f5b3c512640 1 --2- 172.21.15.49:0/2696724355 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5b18065a90 0x7f5b18067f50 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:36.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.739+0000 7f5b3c512640 1 --2- 172.21.15.49:0/2696724355 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b34070160 0x7f5b340c5910 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:36.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.739+0000 7f5b3c512640 1 --2- 172.21.15.49:0/2696724355 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b3415a880 0x7f5b3415b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:36.736 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.739+0000 7f5b3c512640 1 --2- 172.21.15.49:0/2696724355 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5b3406d150 0x7f5b341502e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:36.736 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.739+0000 7f5b3c512640 1 -- 172.21.15.49:0/2696724355 >> 172.21.15.49:0/2696724355 conn(0x7f5b3406a370 msgr2=0x7f5b34153b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:36.736 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.739+0000 7f5b3c512640 1 -- 172.21.15.49:0/2696724355 shutdown_connections 2023-11-06T02:10:36.736 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.739+0000 7f5b3c512640 1 -- 172.21.15.49:0/2696724355 wait complete. 2023-11-06T02:10:36.750 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T02:10:36.861 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.867+0000 7f4d7a28e640 1 Processor -- start 2023-11-06T02:10:36.862 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.867+0000 7f4d7a28e640 1 -- start start 2023-11-06T02:10:36.863 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.867+0000 7f4d7a28e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4d740b1f90 0x7f4d740cda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:36.863 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.867+0000 7f4d7a28e640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f4d740714a0 0x7f4d740718f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:36.863 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.867+0000 7f4d7a28e640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d74071e30 con 0x7f4d74161650 2023-11-06T02:10:36.864 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.867+0000 7f4d7a28e640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d740d17c0 con 0x7f4d740714a0 2023-11-06T02:10:36.864 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.867+0000 7f4d7a28e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d740ce6c0 con 0x7f4d740b1f90 2023-11-06T02:10:36.864 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.867+0000 7f4d7882b640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f4d740714a0 0x7f4d740718f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:36.864 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.867+0000 7f4d73fff640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4d740b1f90 0x7f4d740cda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:36.864 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.867+0000 7f4d7882b640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f4d740714a0 0x7f4d740718f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.49:52632/0 (socket says 172.21.15.49:52632) 2023-11-06T02:10:36.865 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.867+0000 7f4d6bfff640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f4d74161650 0x7f4d740d1c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.49:36352/0 (socket says 172.21.15.49:36352) 2023-11-06T02:10:36.865 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.867+0000 7f4d7882b640 1 -- 172.21.15.49:0/3261552640 learned_addr learned my addr 172.21.15.49:0/3261552640 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:36.865 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.867+0000 7f4d7882b640 1 -- 172.21.15.49:0/3261552640 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4d740b1f90 msgr2=0x7f4d740cda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:36.865 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.867+0000 7f4d7882b640 1 --2- 172.21.15.49:0/3261552640 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4d740b1f90 0x7f4d740cda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:36.865 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.867+0000 7f4d7882b640 1 -- 172.21.15.49:0/3261552640 >> v1:172.21.15.150:6789/0 conn(0x7f4d74161650 legacy=0x7f4d740d1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:36.866 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.867+0000 7f4d7882b640 1 -- 172.21.15.49:0/3261552640 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d74161ad0 con 0x7f4d740714a0 2023-11-06T02:10:36.867 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.871+0000 7f4d7882b640 1 --2- 172.21.15.49:0/3261552640 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d740714a0 0x7f4d740718f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c009e50 tx=0x7f4d6c01c9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c056dfae0daa08b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:36.867 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.871+0000 7f4d737fe640 1 -- 172.21.15.49:0/3261552640 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d6c0a58b0 con 0x7f4d740714a0 2023-11-06T02:10:36.867 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.871+0000 7f4d737fe640 1 -- 172.21.15.49:0/3261552640 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d6c0a5a10 con 0x7f4d740714a0 2023-11-06T02:10:36.867 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.871+0000 7f4d737fe640 1 -- 172.21.15.49:0/3261552640 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d6c0a5da0 con 0x7f4d740714a0 2023-11-06T02:10:36.868 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.871+0000 7f4d7a28e640 1 -- 172.21.15.49:0/3261552640 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d740714a0 msgr2=0x7f4d740718f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:36.868 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.871+0000 7f4d7a28e640 1 --2- 172.21.15.49:0/3261552640 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d740714a0 0x7f4d740718f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c009e50 tx=0x7f4d6c01c9f0 comp rx=0 tx=0).stop 2023-11-06T02:10:36.868 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.871+0000 7f4d7a28e640 1 -- 172.21.15.49:0/3261552640 shutdown_connections 2023-11-06T02:10:36.868 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.871+0000 7f4d7a28e640 1 --2- 172.21.15.49:0/3261552640 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d740714a0 0x7f4d740718f0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:36.868 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.871+0000 7f4d7a28e640 1 --2- 172.21.15.49:0/3261552640 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4d740b1f90 0x7f4d740cda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:36.869 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.871+0000 7f4d7a28e640 1 -- 172.21.15.49:0/3261552640 >> 172.21.15.49:0/3261552640 conn(0x7f4d740d2aa0 msgr2=0x7f4d740d2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:36.869 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.871+0000 7f4d7a28e640 1 -- 172.21.15.49:0/3261552640 shutdown_connections 2023-11-06T02:10:36.869 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.871+0000 7f4d7a28e640 1 -- 172.21.15.49:0/3261552640 wait complete. 2023-11-06T02:10:36.871 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.871+0000 7f4d7a28e640 1 Processor -- start 2023-11-06T02:10:36.872 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.871+0000 7f4d7a28e640 1 -- start start 2023-11-06T02:10:36.872 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.871+0000 7f4d7a28e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d740714a0 0x7f4d740cb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:36.872 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.871+0000 7f4d7a28e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4d740b1f90 0x7f4d7413e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:36.872 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.871+0000 7f4d7a28e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d74161650 0x7f4d7415f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:36.872 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.871+0000 7f4d7a28e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d7413e5c0 con 0x7f4d740714a0 2023-11-06T02:10:36.873 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.871+0000 7f4d7a28e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d740ccca0 con 0x7f4d74161650 2023-11-06T02:10:36.873 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.871+0000 7f4d7a28e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d7415fa30 con 0x7f4d740b1f90 2023-11-06T02:10:36.873 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.871+0000 7f4d73fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d740714a0 0x7f4d740cb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:36.873 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.871+0000 7f4d73fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d740714a0 0x7f4d740cb9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44140/0 (socket says 172.21.15.49:44140) 2023-11-06T02:10:36.874 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.871+0000 7f4d73fff640 1 -- 172.21.15.49:0/2440497602 learned_addr learned my addr 172.21.15.49:0/2440497602 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:36.874 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.875+0000 7f4d7882b640 1 --2- 172.21.15.49:0/2440497602 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d74161650 0x7f4d7415f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:36.874 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.875+0000 7f4d6bfff640 1 --2- 172.21.15.49:0/2440497602 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4d740b1f90 0x7f4d7413e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:36.874 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.875+0000 7f4d73fff640 1 -- 172.21.15.49:0/2440497602 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4d740b1f90 msgr2=0x7f4d7413e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:36.874 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.875+0000 7f4d73fff640 1 --2- 172.21.15.49:0/2440497602 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4d740b1f90 0x7f4d7413e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:36.875 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.875+0000 7f4d73fff640 1 -- 172.21.15.49:0/2440497602 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d74161650 msgr2=0x7f4d7415f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:36.875 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.875+0000 7f4d73fff640 1 --2- 172.21.15.49:0/2440497602 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d74161650 0x7f4d7415f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:36.875 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.875+0000 7f4d73fff640 1 -- 172.21.15.49:0/2440497602 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d6c009b30 con 0x7f4d740714a0 2023-11-06T02:10:36.875 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.875+0000 7f4d73fff640 1 --2- 172.21.15.49:0/2440497602 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d740714a0 0x7f4d740cb9b0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f4d6400e930 tx=0x7f4d6400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:36.875 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.875+0000 7f4d717fa640 1 -- 172.21.15.49:0/2440497602 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d64009690 con 0x7f4d740714a0 2023-11-06T02:10:36.876 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.875+0000 7f4d717fa640 1 -- 172.21.15.49:0/2440497602 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d6400ca00 con 0x7f4d740714a0 2023-11-06T02:10:36.876 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.875+0000 7f4d7a28e640 1 -- 172.21.15.49:0/2440497602 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d740cc090 con 0x7f4d740714a0 2023-11-06T02:10:36.876 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.875+0000 7f4d717fa640 1 -- 172.21.15.49:0/2440497602 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d6400cd70 con 0x7f4d740714a0 2023-11-06T02:10:36.876 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.875+0000 7f4d7a28e640 1 -- 172.21.15.49:0/2440497602 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d740cc650 con 0x7f4d740714a0 2023-11-06T02:10:36.877 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.875+0000 7f4d7a28e640 1 -- 172.21.15.49:0/2440497602 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d74074040 con 0x7f4d740714a0 2023-11-06T02:10:36.877 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.875+0000 7f4d717fa640 1 -- 172.21.15.49:0/2440497602 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f4d64013470 con 0x7f4d740714a0 2023-11-06T02:10:36.877 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.875+0000 7f4d717fa640 1 --2- 172.21.15.49:0/2440497602 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4d54065a60 0x7f4d54067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:36.877 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.875+0000 7f4d717fa640 1 -- 172.21.15.49:0/2440497602 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4d64083570 con 0x7f4d740714a0 2023-11-06T02:10:36.877 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.875+0000 7f4d6bfff640 1 --2- 172.21.15.49:0/2440497602 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4d54065a60 0x7f4d54067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:36.878 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.875+0000 7f4d6bfff640 1 --2- 172.21.15.49:0/2440497602 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4d54065a60 0x7f4d54067f20 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f4d740cd730 tx=0x7f4d5c007cf0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:36.878 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:36.879+0000 7f4d717fa640 1 -- 172.21.15.49:0/2440497602 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f4d64050e90 con 0x7f4d740714a0 2023-11-06T02:10:37.061 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.067+0000 7f4d7a28e640 1 -- 172.21.15.49:0/2440497602 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4d740c9370 con 0x7f4d740714a0 2023-11-06T02:10:37.062 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.067+0000 7f4d717fa640 1 -- 172.21.15.49:0/2440497602 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4648 (secure 0 0 0) 0x7f4d640544b0 con 0x7f4d740714a0 2023-11-06T02:10:37.062 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:10:37.063 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5096,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1396196190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1396196190},{"type":"v1","addr":"172.21.15.150:6835","nonce":1396196190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5102,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/2465209170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":2465209170},{"type":"v1","addr":"172.21.15.150:6837","nonce":2465209170}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5871,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/1006951804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1006951804},{"type":"v1","addr":"172.21.15.49:6833","nonce":1006951804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T02:10:31.903864+0000","modified":"2023-11-06T02:10:34.930237+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5877},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5877":{"gid":5877,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.49:6835/73939548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":73939548},{"type":"v1","addr":"172.21.15.49:6835","nonce":73939548}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-06T02:10:37.063 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 23 2023-11-06T02:10:37.067 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.067+0000 7f4d7a28e640 1 -- 172.21.15.49:0/2440497602 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4d54065a60 msgr2=0x7f4d54067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:37.067 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.067+0000 7f4d7a28e640 1 --2- 172.21.15.49:0/2440497602 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4d54065a60 0x7f4d54067f20 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f4d740cd730 tx=0x7f4d5c007cf0 comp rx=0 tx=0).stop 2023-11-06T02:10:37.067 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.067+0000 7f4d7a28e640 1 -- 172.21.15.49:0/2440497602 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d740714a0 msgr2=0x7f4d740cb9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:37.067 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.067+0000 7f4d7a28e640 1 --2- 172.21.15.49:0/2440497602 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d740714a0 0x7f4d740cb9b0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f4d6400e930 tx=0x7f4d6400edf0 comp rx=0 tx=0).stop 2023-11-06T02:10:37.068 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.067+0000 7f4d7a28e640 1 -- 172.21.15.49:0/2440497602 shutdown_connections 2023-11-06T02:10:37.068 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.067+0000 7f4d7a28e640 1 --2- 172.21.15.49:0/2440497602 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4d54065a60 0x7f4d54067f20 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:37.068 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.067+0000 7f4d7a28e640 1 --2- 172.21.15.49:0/2440497602 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d74161650 0x7f4d7415f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:37.068 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.067+0000 7f4d7a28e640 1 --2- 172.21.15.49:0/2440497602 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4d740b1f90 0x7f4d7413e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:37.068 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.067+0000 7f4d7a28e640 1 --2- 172.21.15.49:0/2440497602 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d740714a0 0x7f4d740cb9b0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:37.068 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.067+0000 7f4d7a28e640 1 -- 172.21.15.49:0/2440497602 >> 172.21.15.49:0/2440497602 conn(0x7f4d740d2aa0 msgr2=0x7f4d740b4110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:37.069 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.067+0000 7f4d7a28e640 1 -- 172.21.15.49:0/2440497602 shutdown_connections 2023-11-06T02:10:37.069 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.067+0000 7f4d7a28e640 1 -- 172.21.15.49:0/2440497602 wait complete. 2023-11-06T02:10:37.482 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-06T02:10:31.903864+0000', 'modified': '2023-11-06T02:10:34.930237+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5877}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5877': {'gid': 5877, 'name': 'a', 'rank': 0, 'incarnation': 19, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.49:6835/73939548', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6834', 'nonce': 73939548}, {'type': 'v1', 'addr': '172.21.15.49:6835', 'nonce': 73939548}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-06T02:10:37.483 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-06T02:10:37.483 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-06T02:10:37.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.622+0000 7f3f3bab0640 1 Processor -- start 2023-11-06T02:10:37.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.622+0000 7f3f3bab0640 1 -- start start 2023-11-06T02:10:37.629 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.622+0000 7f3f3bab0640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f3f34159b40 0x7f3f340ca980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:37.629 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.622+0000 7f3f3bab0640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3f340cdd50 0x7f3f340cc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:37.630 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.622+0000 7f3f3bab0640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f340723a0 con 0x7f3f340cb010 2023-11-06T02:10:37.630 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.622+0000 7f3f3bab0640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f340715e0 con 0x7f3f34159b40 2023-11-06T02:10:37.630 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.622+0000 7f3f3bab0640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f340718c0 con 0x7f3f340cdd50 2023-11-06T02:10:37.631 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.622+0000 7f3f3904b640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f3f34159b40 0x7f3f340ca980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:37.631 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.622+0000 7f3f3904b640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f3f34159b40 0x7f3f340ca980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44144/0 (socket says 172.21.15.49:44144) 2023-11-06T02:10:37.632 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.622+0000 7f3f3904b640 1 -- 172.21.15.49:0/1127305844 learned_addr learned my addr 172.21.15.49:0/1127305844 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:37.632 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.622+0000 7f3f3a04d640 1 --2- 172.21.15.49:0/1127305844 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3f340cdd50 0x7f3f340cc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:37.632 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.626+0000 7f3f3904b640 1 -- 172.21.15.49:0/1127305844 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3f340cdd50 msgr2=0x7f3f340cc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:37.632 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.626+0000 7f3f3904b640 1 --2- 172.21.15.49:0/1127305844 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3f340cdd50 0x7f3f340cc1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:37.633 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.626+0000 7f3f3904b640 1 -- 172.21.15.49:0/1127305844 >> v1:172.21.15.150:6789/0 conn(0x7f3f340cb010 legacy=0x7f3f340cb3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:37.633 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.626+0000 7f3f3904b640 1 -- 172.21.15.49:0/1127305844 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f34071ba0 con 0x7f3f34159b40 2023-11-06T02:10:37.633 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.626+0000 7f3f3904b640 1 --2- 172.21.15.49:0/1127305844 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f34159b40 0x7f3f340ca980 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f3f24009870 tx=0x7f3f2401c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4edd13ffed50347 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:37.634 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.626+0000 7f3f3884a640 1 -- 172.21.15.49:0/1127305844 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f240a95f0 con 0x7f3f34159b40 2023-11-06T02:10:37.634 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.626+0000 7f3f3884a640 1 -- 172.21.15.49:0/1127305844 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f240a9750 con 0x7f3f34159b40 2023-11-06T02:10:37.635 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.626+0000 7f3f3884a640 1 -- 172.21.15.49:0/1127305844 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f240a9ae0 con 0x7f3f34159b40 2023-11-06T02:10:37.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.626+0000 7f3f3bab0640 1 -- 172.21.15.49:0/1127305844 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f34159b40 msgr2=0x7f3f340ca980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:37.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.626+0000 7f3f3bab0640 1 --2- 172.21.15.49:0/1127305844 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f34159b40 0x7f3f340ca980 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f3f24009870 tx=0x7f3f2401c510 comp rx=0 tx=0).stop 2023-11-06T02:10:37.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.626+0000 7f3f3bab0640 1 -- 172.21.15.49:0/1127305844 shutdown_connections 2023-11-06T02:10:37.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.626+0000 7f3f3bab0640 1 --2- 172.21.15.49:0/1127305844 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3f340cdd50 0x7f3f340cc1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:37.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.626+0000 7f3f3bab0640 1 --2- 172.21.15.49:0/1127305844 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f34159b40 0x7f3f340ca980 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:37.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.626+0000 7f3f3bab0640 1 -- 172.21.15.49:0/1127305844 >> 172.21.15.49:0/1127305844 conn(0x7f3f3406c0a0 msgr2=0x7f3f3406c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:37.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.626+0000 7f3f3bab0640 1 -- 172.21.15.49:0/1127305844 shutdown_connections 2023-11-06T02:10:37.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.626+0000 7f3f3bab0640 1 -- 172.21.15.49:0/1127305844 wait complete. 2023-11-06T02:10:37.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.630+0000 7f3f3bab0640 1 Processor -- start 2023-11-06T02:10:37.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.630+0000 7f3f3bab0640 1 -- start start 2023-11-06T02:10:37.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.630+0000 7f3f3bab0640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3f340cb010 0x7f3f341f6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:37.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.630+0000 7f3f3bab0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f340cdd50 0x7f3f340c0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:37.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.630+0000 7f3f3bab0640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f34159b40 0x7f3f340b00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:37.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.630+0000 7f3f3bab0640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f341f6ff0 con 0x7f3f340cdd50 2023-11-06T02:10:37.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.630+0000 7f3f3bab0640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f34151100 con 0x7f3f34159b40 2023-11-06T02:10:37.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.630+0000 7f3f3bab0640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f341513e0 con 0x7f3f340cb010 2023-11-06T02:10:37.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.630+0000 7f3f3904b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f340cdd50 0x7f3f340c0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:37.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.630+0000 7f3f3904b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f340cdd50 0x7f3f340c0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44156/0 (socket says 172.21.15.49:44156) 2023-11-06T02:10:37.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.630+0000 7f3f3904b640 1 -- 172.21.15.49:0/1648692942 learned_addr learned my addr 172.21.15.49:0/1648692942 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:37.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.630+0000 7f3f3a04d640 1 --2- 172.21.15.49:0/1648692942 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f34159b40 0x7f3f340b00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:37.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.630+0000 7f3f3984c640 1 --2- 172.21.15.49:0/1648692942 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3f340cb010 0x7f3f341f6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:37.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.630+0000 7f3f3904b640 1 -- 172.21.15.49:0/1648692942 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3f340cb010 msgr2=0x7f3f341f6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:37.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.630+0000 7f3f3904b640 1 --2- 172.21.15.49:0/1648692942 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3f340cb010 0x7f3f341f6ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:37.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.630+0000 7f3f3904b640 1 -- 172.21.15.49:0/1648692942 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f34159b40 msgr2=0x7f3f340b00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:37.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.630+0000 7f3f3904b640 1 --2- 172.21.15.49:0/1648692942 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f34159b40 0x7f3f340b00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:37.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.630+0000 7f3f3904b640 1 -- 172.21.15.49:0/1648692942 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f240094d0 con 0x7f3f340cdd50 2023-11-06T02:10:37.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.630+0000 7f3f3984c640 1 --2- 172.21.15.49:0/1648692942 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3f340cb010 0x7f3f341f6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:10:37.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.630+0000 7f3f3a04d640 1 --2- 172.21.15.49:0/1648692942 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f34159b40 0x7f3f340b00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:10:37.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.630+0000 7f3f3904b640 1 --2- 172.21.15.49:0/1648692942 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f340cdd50 0x7f3f340c0b90 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f3f2401abc0 tx=0x7f3f240192b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:37.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.630+0000 7f3f2a7fc640 1 -- 172.21.15.49:0/1648692942 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f240b9620 con 0x7f3f340cdd50 2023-11-06T02:10:37.648 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.630+0000 7f3f2a7fc640 1 -- 172.21.15.49:0/1648692942 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f240b9780 con 0x7f3f340cdd50 2023-11-06T02:10:37.648 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.630+0000 7f3f2a7fc640 1 -- 172.21.15.49:0/1648692942 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f240b9b10 con 0x7f3f340cdd50 2023-11-06T02:10:37.648 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.630+0000 7f3f3bab0640 1 -- 172.21.15.49:0/1648692942 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f341516c0 con 0x7f3f340cdd50 2023-11-06T02:10:37.648 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.630+0000 7f3f3bab0640 1 -- 172.21.15.49:0/1648692942 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f34151c80 con 0x7f3f340cdd50 2023-11-06T02:10:37.670 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.634+0000 7f3f2a7fc640 1 -- 172.21.15.49:0/1648692942 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f3f240b13f0 con 0x7f3f340cdd50 2023-11-06T02:10:37.670 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.634+0000 7f3f2a7fc640 1 --2- 172.21.15.49:0/1648692942 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3f18065a60 0x7f3f18067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:37.670 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.634+0000 7f3f2a7fc640 1 -- 172.21.15.49:0/1648692942 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3f2411dbb0 con 0x7f3f340cdd50 2023-11-06T02:10:37.671 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.634+0000 7f3f3984c640 1 --2- 172.21.15.49:0/1648692942 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3f18065a60 0x7f3f18067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:37.671 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.634+0000 7f3f3ebd8000 1 Processor -- start 2023-11-06T02:10:37.671 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.634+0000 7f3f3ebd8000 1 -- start start 2023-11-06T02:10:37.671 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.634+0000 7f3f3984c640 1 --2- 172.21.15.49:0/1648692942 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3f18065a60 0x7f3f18067f20 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f3f30000fc0 tx=0x7f3f30009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:37.672 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.634+0000 7f3f3ebd8000 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562e625ca6e0 0x562e625caac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:37.672 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.634+0000 7f3f3ebd8000 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e625cb000 0x562e62580550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:37.672 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.634+0000 7f3f3ebd8000 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e62580a90 0x562e62580e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:37.672 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.634+0000 7f3f3ebd8000 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e62448fc0 con 0x562e625cb000 2023-11-06T02:10:37.672 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.634+0000 7f3f3ebd8000 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e623d71a0 con 0x562e62580a90 2023-11-06T02:10:37.673 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.634+0000 7f3f3ebd8000 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x562e623d4770 con 0x562e625ca6e0 2023-11-06T02:10:37.673 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.634+0000 7f3f3904b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e625cb000 0x562e62580550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:37.673 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.634+0000 7f3f3904b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e625cb000 0x562e62580550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44164/0 (socket says 172.21.15.49:44164) 2023-11-06T02:10:37.673 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.634+0000 7f3f3904b640 1 -- 172.21.15.49:0/2135303064 learned_addr learned my addr 172.21.15.49:0/2135303064 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:37.673 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.634+0000 7f3f3984c640 1 --2- 172.21.15.49:0/2135303064 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562e625ca6e0 0x562e625caac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:37.674 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.634+0000 7f3f3a04d640 1 --2- 172.21.15.49:0/2135303064 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e62580a90 0x562e62580e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:37.674 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.634+0000 7f3f3904b640 1 -- 172.21.15.49:0/2135303064 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562e625ca6e0 msgr2=0x562e625caac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:37.674 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.634+0000 7f3f3904b640 1 --2- 172.21.15.49:0/2135303064 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562e625ca6e0 0x562e625caac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:37.674 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.634+0000 7f3f3904b640 1 -- 172.21.15.49:0/2135303064 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e62580a90 msgr2=0x562e62580e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:37.674 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.634+0000 7f3f3904b640 1 --2- 172.21.15.49:0/2135303064 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e62580a90 0x562e62580e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:37.675 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.634+0000 7f3f3904b640 1 -- 172.21.15.49:0/2135303064 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e620cc890 con 0x562e625cb000 2023-11-06T02:10:37.675 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.634+0000 7f3f3984c640 1 --2- 172.21.15.49:0/2135303064 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562e625ca6e0 0x562e625caac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:10:37.675 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.634+0000 7f3f3904b640 1 --2- 172.21.15.49:0/2135303064 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e625cb000 0x562e62580550 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f3f2411bfa0 tx=0x7f3f240eb540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ed1b97b5a8b8a1e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:37.675 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.634+0000 7f3f177fe640 1 -- 172.21.15.49:0/2135303064 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f240f70b0 con 0x562e625cb000 2023-11-06T02:10:37.675 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.634+0000 7f3f177fe640 1 -- 172.21.15.49:0/2135303064 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f240ed470 con 0x562e625cb000 2023-11-06T02:10:37.676 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.634+0000 7f3f177fe640 1 -- 172.21.15.49:0/2135303064 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f240ee5c0 con 0x562e625cb000 2023-11-06T02:10:37.676 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.634+0000 7f3f3ebd8000 1 -- 172.21.15.49:0/2135303064 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e625cb000 msgr2=0x562e62580550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:37.676 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.634+0000 7f3f3ebd8000 1 --2- 172.21.15.49:0/2135303064 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e625cb000 0x562e62580550 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f3f2411bfa0 tx=0x7f3f240eb540 comp rx=0 tx=0).stop 2023-11-06T02:10:37.676 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.634+0000 7f3f3ebd8000 1 -- 172.21.15.49:0/2135303064 shutdown_connections 2023-11-06T02:10:37.676 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.634+0000 7f3f3ebd8000 1 --2- 172.21.15.49:0/2135303064 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e625cb000 0x562e62580550 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:37.677 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.634+0000 7f3f3ebd8000 1 --2- 172.21.15.49:0/2135303064 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e62580a90 0x562e62580e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:37.677 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.634+0000 7f3f3ebd8000 1 --2- 172.21.15.49:0/2135303064 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562e625ca6e0 0x562e625caac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:37.677 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.634+0000 7f3f3ebd8000 1 -- 172.21.15.49:0/2135303064 >> 172.21.15.49:0/2135303064 conn(0x562e62586bf0 msgr2=0x562e6257ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:37.677 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.634+0000 7f3f3ebd8000 1 -- 172.21.15.49:0/2135303064 shutdown_connections 2023-11-06T02:10:37.677 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.634+0000 7f3f3ebd8000 1 -- 172.21.15.49:0/2135303064 wait complete. 2023-11-06T02:10:37.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.638+0000 7f3f3ebd8000 1 -- start start 2023-11-06T02:10:37.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.638+0000 7f3f3ebd8000 1 Processor -- start 2023-11-06T02:10:37.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.638+0000 7f3f3ebd8000 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562e62580a90 0x562e625854a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:37.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.638+0000 7f3f3ebd8000 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e625ca6e0 0x562e625859e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:37.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.638+0000 7f3f3ebd8000 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e625cb000 0x562e625cb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:37.679 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.638+0000 7f3f3ebd8000 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e623d0060 con 0x562e625ca6e0 2023-11-06T02:10:37.679 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.638+0000 7f3f3ebd8000 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e6236f290 con 0x562e625cb000 2023-11-06T02:10:37.679 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.638+0000 7f3f3ebd8000 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x562e6238d6f0 con 0x562e62580a90 2023-11-06T02:10:37.679 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.638+0000 7f3f3ebd8000 10 client.0 _ll_register_callbacks cb 0x562e6256c1d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T02:10:37.679 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.638+0000 7f3f3904b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e625ca6e0 0x562e625859e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:37.680 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.638+0000 7f3f3904b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e625ca6e0 0x562e625859e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44168/0 (socket says 172.21.15.49:44168) 2023-11-06T02:10:37.680 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.638+0000 7f3f3904b640 1 -- 172.21.15.49:0/2938883785 learned_addr learned my addr 172.21.15.49:0/2938883785 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:37.680 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.638+0000 7f3f3904b640 1 -- 172.21.15.49:0/2938883785 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562e62580a90 msgr2=0x562e625854a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:37.680 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.638+0000 7f3f3904b640 1 --2- 172.21.15.49:0/2938883785 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562e62580a90 0x562e625854a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:37.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.638+0000 7f3f3904b640 1 -- 172.21.15.49:0/2938883785 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e625cb000 msgr2=0x562e625cb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:37.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.638+0000 7f3f3904b640 1 --2- 172.21.15.49:0/2938883785 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e625cb000 0x562e625cb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:37.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.638+0000 7f3f3904b640 1 -- 172.21.15.49:0/2938883785 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f34151c80 con 0x562e625ca6e0 2023-11-06T02:10:37.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.638+0000 7f3f3904b640 1 --2- 172.21.15.49:0/2938883785 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e625ca6e0 0x562e625859e0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f3f240ef060 tx=0x7f3f24000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a506063dc8f1fcbd server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:37.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.638+0000 7f3f14ff9640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-11-06T02:10:37.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.638+0000 7f3f14ff9640 1 -- 172.21.15.49:0/2938883785 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f240f70b0 con 0x562e625ca6e0 2023-11-06T02:10:37.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.638+0000 7f3f3ebd8000 1 -- 172.21.15.49:0/2938883785 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562e623ce9b0 con 0x562e625ca6e0 2023-11-06T02:10:37.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.638+0000 7f3f14ff9640 1 -- 172.21.15.49:0/2938883785 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3f240ee5c0 con 0x562e625ca6e0 2023-11-06T02:10:37.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.638+0000 7f3f14ff9640 1 -- 172.21.15.49:0/2938883785 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3f240ee780 con 0x562e625ca6e0 2023-11-06T02:10:37.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.638+0000 7f3f3ebd8000 10 client.6018 fetch_fsmap learned FSMap version 23 2023-11-06T02:10:37.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.638+0000 7f3f3ebd8000 1 -- 172.21.15.49:0/2938883785 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x562e623951e0 con 0x562e625ca6e0 2023-11-06T02:10:37.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.638+0000 7f3f14ff9640 1 -- 172.21.15.49:0/2938883785 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562e623ce9b0 con 0x562e625ca6e0 2023-11-06T02:10:37.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.638+0000 7f3f14ff9640 1 -- 172.21.15.49:0/2938883785 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 23) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f3f240f6c20 con 0x562e625ca6e0 2023-11-06T02:10:37.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.638+0000 7f3f3ebd8000 10 client.6018 fetch_fsmap finished waiting for FSMap version 23 2023-11-06T02:10:37.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.638+0000 7f3f3ebd8000 10 client.6018 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-06T02:10:37.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.638+0000 7f3f3ebd8000 20 client.6018 populate_metadata read hostname 'smithi049' 2023-11-06T02:10:37.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.638+0000 7f3f3ebd8000 1 --2- 172.21.15.49:0/2938883785 >> [v2:172.21.15.49:6834/73939548,v1:172.21.15.49:6835/73939548] conn(0x562e625d9770 0x562e625dbb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:37.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.638+0000 7f3f3ebd8000 4 client.6018 mds_command: new command op to 5877 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T02:10:37.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.638+0000 7f3f3ebd8000 1 -- 172.21.15.49:0/2938883785 --> [v2:172.21.15.49:6834/73939548,v1:172.21.15.49:6835/73939548] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562e623ff4d0 con 0x562e625d9770 2023-11-06T02:10:37.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.638+0000 7f3f3984c640 1 --2- 172.21.15.49:0/2938883785 >> [v2:172.21.15.49:6834/73939548,v1:172.21.15.49:6835/73939548] conn(0x562e625d9770 0x562e625dbb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:37.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.638+0000 7f3f3984c640 1 --2- 172.21.15.49:0/2938883785 >> [v2:172.21.15.49:6834/73939548,v1:172.21.15.49:6835/73939548] conn(0x562e625d9770 0x562e625dbb60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=155580fdebb99af5 server_cookie=de49a7e87239f735 in_seq=0 out_seq=0 2023-11-06T02:10:37.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.638+0000 7f3f14ff9640 10 client.6018 ms_handle_connect on v2:172.21.15.49:6834/73939548 2023-11-06T02:10:37.686 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.638+0000 7f3f14ff9640 1 -- 172.21.15.49:0/2938883785 <== mds.0 v2:172.21.15.49:6834/73939548 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x562e623ff4d0 con 0x562e625d9770 2023-11-06T02:10:37.686 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.638+0000 7f3f14ff9640 10 client.6018 handle_command_reply: tid=0 2023-11-06T02:10:37.686 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.638+0000 7f3f3ebd8000 1 client.6018 shutdown 2023-11-06T02:10:37.686 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.638+0000 7f3f3ebd8000 1 -- 172.21.15.49:0/2938883785 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e625ca6e0 msgr2=0x562e625859e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:37.686 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.638+0000 7f3f3ebd8000 1 --2- 172.21.15.49:0/2938883785 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e625ca6e0 0x562e625859e0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f3f240ef060 tx=0x7f3f24000bd0 comp rx=0 tx=0).stop 2023-11-06T02:10:37.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.638+0000 7f3f3ebd8000 1 -- 172.21.15.49:0/2938883785 shutdown_connections 2023-11-06T02:10:37.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.638+0000 7f3f3ebd8000 1 --2- 172.21.15.49:0/2938883785 >> [v2:172.21.15.49:6834/73939548,v1:172.21.15.49:6835/73939548] conn(0x562e625d9770 0x562e625dbb60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:37.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.638+0000 7f3f3ebd8000 1 --2- 172.21.15.49:0/2938883785 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e625ca6e0 0x562e625859e0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:37.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.638+0000 7f3f3ebd8000 1 --2- 172.21.15.49:0/2938883785 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e625cb000 0x562e625cb3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:37.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.638+0000 7f3f14ff9640 0 client.6018 ms_handle_reset on v2:172.21.15.49:6834/73939548 2023-11-06T02:10:37.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.638+0000 7f3f3ebd8000 1 --2- 172.21.15.49:0/2938883785 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562e62580a90 0x562e625854a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:37.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.638+0000 7f3f3ebd8000 1 -- 172.21.15.49:0/2938883785 >> 172.21.15.49:0/2938883785 conn(0x562e62586bf0 msgr2=0x562e6257f530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:37.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.642+0000 7f3f3ebd8000 1 -- 172.21.15.49:0/2938883785 shutdown_connections 2023-11-06T02:10:37.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.642+0000 7f3f3ebd8000 1 -- 172.21.15.49:0/2938883785 wait complete. 2023-11-06T02:10:37.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.642+0000 7f3f3ebd8000 20 client.6018 trim_cache size 0 max 16384 2023-11-06T02:10:37.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.654+0000 7f3f3ebd8000 1 Processor -- start 2023-11-06T02:10:37.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.654+0000 7f3f3ebd8000 1 -- start start 2023-11-06T02:10:37.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.654+0000 7f3f3ebd8000 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e62580a90 0x562e625ddac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:37.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.654+0000 7f3f3ebd8000 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e625ca6e0 0x562e625dcb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:37.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.654+0000 7f3f3ebd8000 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562e625cb000 0x562e625dd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:37.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.654+0000 7f3f3ebd8000 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e625868a0 con 0x562e62580a90 2023-11-06T02:10:37.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.654+0000 7f3f3ebd8000 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e6208d940 con 0x562e625ca6e0 2023-11-06T02:10:37.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.654+0000 7f3f3ebd8000 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x562e623d4060 con 0x562e625cb000 2023-11-06T02:10:37.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.654+0000 7f3f3984c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e62580a90 0x562e625ddac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:37.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.658+0000 7f3f3984c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e62580a90 0x562e625ddac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44174/0 (socket says 172.21.15.49:44174) 2023-11-06T02:10:37.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.658+0000 7f3f3984c640 1 -- 172.21.15.49:0/1985571812 learned_addr learned my addr 172.21.15.49:0/1985571812 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:37.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.658+0000 7f3f3904b640 1 --2- 172.21.15.49:0/1985571812 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e625ca6e0 0x562e625dcb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:37.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.658+0000 7f3f3a04d640 1 --2- 172.21.15.49:0/1985571812 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562e625cb000 0x562e625dd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:37.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.658+0000 7f3f3984c640 1 -- 172.21.15.49:0/1985571812 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562e625cb000 msgr2=0x562e625dd0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:37.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.658+0000 7f3f3984c640 1 --2- 172.21.15.49:0/1985571812 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562e625cb000 0x562e625dd0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:37.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.658+0000 7f3f3984c640 1 -- 172.21.15.49:0/1985571812 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e625ca6e0 msgr2=0x562e625dcb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:37.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.658+0000 7f3f3984c640 1 --2- 172.21.15.49:0/1985571812 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e625ca6e0 0x562e625dcb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:37.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.658+0000 7f3f3984c640 1 -- 172.21.15.49:0/1985571812 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e6237b330 con 0x562e62580a90 2023-11-06T02:10:37.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.658+0000 7f3f3984c640 1 --2- 172.21.15.49:0/1985571812 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e62580a90 0x562e625ddac0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f3f300239a0 tx=0x7f3f30010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd830e4a1835b4be server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:37.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.658+0000 7f3f17fff640 1 -- 172.21.15.49:0/1985571812 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f300230e0 con 0x562e62580a90 2023-11-06T02:10:37.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.658+0000 7f3f17fff640 1 -- 172.21.15.49:0/1985571812 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f3001a070 con 0x562e62580a90 2023-11-06T02:10:37.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.658+0000 7f3f17fff640 1 -- 172.21.15.49:0/1985571812 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f30014420 con 0x562e62580a90 2023-11-06T02:10:37.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.658+0000 7f3f3ebd8000 1 -- 172.21.15.49:0/1985571812 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e62580a90 msgr2=0x562e625ddac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:37.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.658+0000 7f3f3ebd8000 1 --2- 172.21.15.49:0/1985571812 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e62580a90 0x562e625ddac0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f3f300239a0 tx=0x7f3f30010000 comp rx=0 tx=0).stop 2023-11-06T02:10:37.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.658+0000 7f3f3ebd8000 1 -- 172.21.15.49:0/1985571812 shutdown_connections 2023-11-06T02:10:37.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.658+0000 7f3f3ebd8000 1 --2- 172.21.15.49:0/1985571812 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562e625cb000 0x562e625dd0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:37.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.658+0000 7f3f3ebd8000 1 --2- 172.21.15.49:0/1985571812 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e625ca6e0 0x562e625dcb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:37.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.658+0000 7f3f3ebd8000 1 --2- 172.21.15.49:0/1985571812 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e62580a90 0x562e625ddac0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:37.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.658+0000 7f3f3ebd8000 1 -- 172.21.15.49:0/1985571812 >> 172.21.15.49:0/1985571812 conn(0x562e62586bf0 msgr2=0x562e62584f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:37.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.658+0000 7f3f3ebd8000 1 -- 172.21.15.49:0/1985571812 shutdown_connections 2023-11-06T02:10:37.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.658+0000 7f3f3ebd8000 1 -- 172.21.15.49:0/1985571812 wait complete. 2023-11-06T02:10:37.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.658+0000 7f3f3ebd8000 1 -- start start 2023-11-06T02:10:37.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.658+0000 7f3f3ebd8000 1 Processor -- start 2023-11-06T02:10:37.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.658+0000 7f3f3ebd8000 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e62580a90 0x562e62626cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:37.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.658+0000 7f3f3ebd8000 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e625ca6e0 0x562e62627210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:37.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.658+0000 7f3f3ebd8000 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562e625cb000 0x562e625c6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:37.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.658+0000 7f3f3ebd8000 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e62586760 con 0x562e62580a90 2023-11-06T02:10:37.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.658+0000 7f3f3ebd8000 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e623ceef0 con 0x562e625ca6e0 2023-11-06T02:10:37.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.658+0000 7f3f3ebd8000 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x562e62413560 con 0x562e625cb000 2023-11-06T02:10:37.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.658+0000 7f3f3ebd8000 10 client.0 _ll_register_callbacks cb 0x562e6254c9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T02:10:37.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.658+0000 7f3f3984c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e62580a90 0x562e62626cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:37.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.658+0000 7f3f3984c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e62580a90 0x562e62626cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44182/0 (socket says 172.21.15.49:44182) 2023-11-06T02:10:37.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.658+0000 7f3f3984c640 1 -- 172.21.15.49:0/4277853197 learned_addr learned my addr 172.21.15.49:0/4277853197 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:37.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.658+0000 7f3f3984c640 1 -- 172.21.15.49:0/4277853197 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562e625cb000 msgr2=0x562e625c6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:37.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.658+0000 7f3f3a04d640 1 --2- 172.21.15.49:0/4277853197 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562e625cb000 0x562e625c6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:37.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.658+0000 7f3f3904b640 1 --2- 172.21.15.49:0/4277853197 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e625ca6e0 0x562e62627210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:37.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.658+0000 7f3f3984c640 1 --2- 172.21.15.49:0/4277853197 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562e625cb000 0x562e625c6040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:37.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.658+0000 7f3f3984c640 1 -- 172.21.15.49:0/4277853197 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e625ca6e0 msgr2=0x562e62627210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:37.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.658+0000 7f3f3984c640 1 --2- 172.21.15.49:0/4277853197 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e625ca6e0 0x562e62627210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:37.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.658+0000 7f3f3984c640 1 -- 172.21.15.49:0/4277853197 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f340723a0 con 0x562e62580a90 2023-11-06T02:10:37.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.658+0000 7f3f3984c640 1 --2- 172.21.15.49:0/4277853197 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e62580a90 0x562e62626cd0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f3f30009010 tx=0x7f3f30000f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c06e9c0639dd404 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:37.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.658+0000 7f3f14ff9640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-11-06T02:10:37.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.658+0000 7f3f14ff9640 1 -- 172.21.15.49:0/4277853197 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f30036640 con 0x562e62580a90 2023-11-06T02:10:37.701 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.658+0000 7f3f14ff9640 1 -- 172.21.15.49:0/4277853197 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3f30014420 con 0x562e62580a90 2023-11-06T02:10:37.701 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.658+0000 7f3f3ebd8000 1 -- 172.21.15.49:0/4277853197 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562e6239fb90 con 0x562e62580a90 2023-11-06T02:10:37.701 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.658+0000 7f3f14ff9640 1 -- 172.21.15.49:0/4277853197 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3f300145e0 con 0x562e62580a90 2023-11-06T02:10:37.701 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.658+0000 7f3f3ebd8000 10 client.6024 fetch_fsmap learned FSMap version 23 2023-11-06T02:10:37.701 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.658+0000 7f3f3ebd8000 1 -- 172.21.15.49:0/4277853197 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x562e620cccc0 con 0x562e62580a90 2023-11-06T02:10:37.702 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.658+0000 7f3f14ff9640 1 -- 172.21.15.49:0/4277853197 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562e6239fb90 con 0x562e62580a90 2023-11-06T02:10:37.702 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.662+0000 7f3f14ff9640 1 -- 172.21.15.49:0/4277853197 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 23) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f3f300367a0 con 0x562e62580a90 2023-11-06T02:10:37.702 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.662+0000 7f3f3ebd8000 10 client.6024 fetch_fsmap finished waiting for FSMap version 23 2023-11-06T02:10:37.702 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.662+0000 7f3f3ebd8000 10 client.6024 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-06T02:10:37.703 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.662+0000 7f3f3ebd8000 20 client.6024 populate_metadata read hostname 'smithi049' 2023-11-06T02:10:37.703 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.662+0000 7f3f3ebd8000 1 --2- 172.21.15.49:0/4277853197 >> [v2:172.21.15.49:6834/73939548,v1:172.21.15.49:6835/73939548] conn(0x562e625c6580 0x562e625c6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:37.703 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.662+0000 7f3f3ebd8000 4 client.6024 mds_command: new command op to 5877 tid=0[{"prefix": "status"}] 2023-11-06T02:10:37.703 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.662+0000 7f3f3ebd8000 1 -- 172.21.15.49:0/4277853197 --> [v2:172.21.15.49:6834/73939548,v1:172.21.15.49:6835/73939548] -- command(tid 0: {"prefix": "status"}) v1 -- 0x562e623c90e0 con 0x562e625c6580 2023-11-06T02:10:37.703 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.662+0000 7f3f3904b640 1 --2- 172.21.15.49:0/4277853197 >> [v2:172.21.15.49:6834/73939548,v1:172.21.15.49:6835/73939548] conn(0x562e625c6580 0x562e625c6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:37.704 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.662+0000 7f3f14ff9640 10 client.6024 ms_handle_connect on v2:172.21.15.49:6834/73939548 2023-11-06T02:10:37.704 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.662+0000 7f3f3904b640 1 --2- 172.21.15.49:0/4277853197 >> [v2:172.21.15.49:6834/73939548,v1:172.21.15.49:6835/73939548] conn(0x562e625c6580 0x562e625c6960 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3d321a70542f4eb server_cookie=2bffa1bd87a969b5 in_seq=0 out_seq=0 2023-11-06T02:10:37.704 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.662+0000 7f3f14ff9640 1 -- 172.21.15.49:0/4277853197 <== mds.0 v2:172.21.15.49:6834/73939548 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x562e623c90e0 con 0x562e625c6580 2023-11-06T02:10:37.704 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.662+0000 7f3f14ff9640 10 client.6024 handle_command_reply: tid=0 2023-11-06T02:10:37.704 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.662+0000 7f3f3ebd8000 1 client.6024 shutdown 2023-11-06T02:10:37.705 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.662+0000 7f3f3ebd8000 1 -- 172.21.15.49:0/4277853197 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e62580a90 msgr2=0x562e62626cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:37.705 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.662+0000 7f3f3ebd8000 1 --2- 172.21.15.49:0/4277853197 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e62580a90 0x562e62626cd0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f3f30009010 tx=0x7f3f30000f30 comp rx=0 tx=0).stop 2023-11-06T02:10:37.705 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.662+0000 7f3f3ebd8000 1 -- 172.21.15.49:0/4277853197 shutdown_connections 2023-11-06T02:10:37.705 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.662+0000 7f3f3ebd8000 1 --2- 172.21.15.49:0/4277853197 >> [v2:172.21.15.49:6834/73939548,v1:172.21.15.49:6835/73939548] conn(0x562e625c6580 0x562e625c6960 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:37.706 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.662+0000 7f3f3ebd8000 1 --2- 172.21.15.49:0/4277853197 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562e625cb000 0x562e625c6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:37.706 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.662+0000 7f3f14ff9640 0 client.6024 ms_handle_reset on v2:172.21.15.49:6834/73939548 2023-11-06T02:10:37.706 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.662+0000 7f3f3ebd8000 1 --2- 172.21.15.49:0/4277853197 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e625ca6e0 0x562e62627210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:37.706 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.662+0000 7f3f3ebd8000 1 --2- 172.21.15.49:0/4277853197 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e62580a90 0x562e62626cd0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:37.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.662+0000 7f3f3ebd8000 1 -- 172.21.15.49:0/4277853197 >> 172.21.15.49:0/4277853197 conn(0x562e62586bf0 msgr2=0x562e62585670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:37.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.662+0000 7f3f3ebd8000 1 -- 172.21.15.49:0/4277853197 shutdown_connections 2023-11-06T02:10:37.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.662+0000 7f3f3ebd8000 1 -- 172.21.15.49:0/4277853197 wait complete. 2023-11-06T02:10:37.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.662+0000 7f3f3ebd8000 20 client.6024 trim_cache size 0 max 16384 2023-11-06T02:10:37.708 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-11-06T02:10:37.708 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_fsid": "1a8b035c-6ba4-499a-b876-aea47fd9c0d4", 2023-11-06T02:10:37.709 INFO:teuthology.orchestra.run.smithi049.stdout: "whoami": 0, 2023-11-06T02:10:37.709 INFO:teuthology.orchestra.run.smithi049.stdout: "id": 5877, 2023-11-06T02:10:37.709 INFO:teuthology.orchestra.run.smithi049.stdout: "want_state": "up:active", 2023-11-06T02:10:37.709 INFO:teuthology.orchestra.run.smithi049.stdout: "state": "up:active", 2023-11-06T02:10:37.710 INFO:teuthology.orchestra.run.smithi049.stdout: "fs_name": "cephfs", 2023-11-06T02:10:37.710 INFO:teuthology.orchestra.run.smithi049.stdout: "rank_uptime": 5.7551431270000002, 2023-11-06T02:10:37.710 INFO:teuthology.orchestra.run.smithi049.stdout: "mdsmap_epoch": 23, 2023-11-06T02:10:37.714 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch": 51, 2023-11-06T02:10:37.714 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch_barrier": 51, 2023-11-06T02:10:37.714 INFO:teuthology.orchestra.run.smithi049.stdout: "uptime": 10.29466446 2023-11-06T02:10:37.715 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-11-06T02:10:37.716 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.662+0000 7f3f3bab0640 1 -- 172.21.15.49:0/1648692942 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3f18065a60 msgr2=0x7f3f18067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:37.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.662+0000 7f3f3bab0640 1 --2- 172.21.15.49:0/1648692942 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3f18065a60 0x7f3f18067f20 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f3f30000fc0 tx=0x7f3f30009040 comp rx=0 tx=0).stop 2023-11-06T02:10:37.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.662+0000 7f3f3bab0640 1 -- 172.21.15.49:0/1648692942 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f340cdd50 msgr2=0x7f3f340c0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:37.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.662+0000 7f3f3bab0640 1 --2- 172.21.15.49:0/1648692942 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f340cdd50 0x7f3f340c0b90 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f3f2401abc0 tx=0x7f3f240192b0 comp rx=0 tx=0).stop 2023-11-06T02:10:37.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.662+0000 7f3f3bab0640 1 -- 172.21.15.49:0/1648692942 shutdown_connections 2023-11-06T02:10:37.718 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.662+0000 7f3f3bab0640 1 --2- 172.21.15.49:0/1648692942 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3f18065a60 0x7f3f18067f20 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:37.718 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.662+0000 7f3f3bab0640 1 --2- 172.21.15.49:0/1648692942 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f340cdd50 0x7f3f340c0b90 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:37.718 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.662+0000 7f3f3bab0640 1 --2- 172.21.15.49:0/1648692942 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f34159b40 0x7f3f340b00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:37.718 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.662+0000 7f3f3bab0640 1 --2- 172.21.15.49:0/1648692942 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3f340cb010 0x7f3f341f6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:37.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.662+0000 7f3f3bab0640 1 -- 172.21.15.49:0/1648692942 >> 172.21.15.49:0/1648692942 conn(0x7f3f3406c0a0 msgr2=0x7f3f340cb3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:37.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.662+0000 7f3f3bab0640 1 -- 172.21.15.49:0/1648692942 shutdown_connections 2023-11-06T02:10:37.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.662+0000 7f3f3bab0640 1 -- 172.21.15.49:0/1648692942 wait complete. 2023-11-06T02:10:37.720 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-06T02:10:37.845 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.850+0000 7f3114187640 1 Processor -- start 2023-11-06T02:10:37.867 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.850+0000 7f3114187640 1 -- start start 2023-11-06T02:10:37.873 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.850+0000 7f3114187640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f310c0b1f90 0x7f310c0cda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:37.873 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.850+0000 7f3114187640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f310c0d17c0 con 0x7f310c161650 2023-11-06T02:10:37.873 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.850+0000 7f3114187640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f310c161b00 con 0x7f310c0714a0 2023-11-06T02:10:37.873 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.850+0000 7f3114187640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f310c0ce6c0 con 0x7f310c0b1f90 2023-11-06T02:10:37.874 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.850+0000 7f3111722640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f310c161650 0x7f310c0d1c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:41570/0 (socket says 172.21.15.49:41570) 2023-11-06T02:10:37.874 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.850+0000 7f3111722640 1 -- 172.21.15.49:0/4244535867 learned_addr learned my addr 172.21.15.49:0/4244535867 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:10:37.874 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.850+0000 7f3111f23640 1 --2- 172.21.15.49:0/4244535867 >> v2:172.21.15.150:3300/0 conn(0x7f310c0b1f90 0x7f310c0cda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:37.874 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.850+0000 7f3110f21640 1 -- 172.21.15.49:0/4244535867 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2501974559 0 0) 0x7f310c0d17c0 con 0x7f310c161650 2023-11-06T02:10:37.874 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.850+0000 7f3110f21640 1 -- 172.21.15.49:0/4244535867 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30f8003530 con 0x7f310c161650 2023-11-06T02:10:37.875 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.850+0000 7f3110f21640 1 -- 172.21.15.49:0/4244535867 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 51308272 0 0) 0x7f30f8003530 con 0x7f310c161650 2023-11-06T02:10:37.875 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.850+0000 7f3110f21640 1 -- 172.21.15.49:0/4244535867 >> v2:172.21.15.150:3300/0 conn(0x7f310c0b1f90 msgr2=0x7f310c0cda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:37.875 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.850+0000 7f3110f21640 1 --2- 172.21.15.49:0/4244535867 >> v2:172.21.15.150:3300/0 conn(0x7f310c0b1f90 0x7f310c0cda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:37.875 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.850+0000 7f3110f21640 1 -- 172.21.15.49:0/4244535867 >> v1:172.21.15.150:6789/0 conn(0x7f310c0714a0 legacy=0x7f310c0718b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:37.875 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.850+0000 7f3110f21640 1 -- 172.21.15.49:0/4244535867 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f310c0d2360 con 0x7f310c161650 2023-11-06T02:10:37.876 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.850+0000 7f3110f21640 1 -- 172.21.15.49:0/4244535867 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7f30fc001680 con 0x7f310c161650 2023-11-06T02:10:37.876 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.850+0000 7f3110f21640 1 -- 172.21.15.49:0/4244535867 >> v1:172.21.15.49:6789/0 conn(0x7f310c161650 legacy=0x7f310c0d1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:37.876 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.850+0000 7f3110f21640 1 --2- 172.21.15.49:0/4244535867 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f30f8003980 0x7f30f8003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:37.876 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.850+0000 7f3110f21640 1 --2- 172.21.15.49:0/4244535867 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30f80047d0 0x7f30f8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:37.876 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.850+0000 7f3110f21640 1 --2- 172.21.15.49:0/4244535867 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f30f8005190 0x7f30f800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:37.877 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.850+0000 7f3110f21640 1 -- 172.21.15.49:0/4244535867 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f310c0ce6c0 con 0x7f30f80047d0 2023-11-06T02:10:37.877 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.850+0000 7f3110f21640 1 -- 172.21.15.49:0/4244535867 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30f800c030 con 0x7f30f8005190 2023-11-06T02:10:37.877 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.850+0000 7f3110f21640 1 -- 172.21.15.49:0/4244535867 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30f800c1f0 con 0x7f30f8003980 2023-11-06T02:10:37.877 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.850+0000 7f3111722640 1 --2- 172.21.15.49:0/4244535867 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30f80047d0 0x7f30f8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:37.878 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.850+0000 7f3111f23640 1 --2- 172.21.15.49:0/4244535867 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f30f8003980 0x7f30f8003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:37.878 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.850+0000 7f3111722640 1 -- 172.21.15.49:0/4244535867 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f30f8003980 msgr2=0x7f30f8003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:37.878 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.850+0000 7f3111722640 1 --2- 172.21.15.49:0/4244535867 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f30f8003980 0x7f30f8003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:37.878 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.850+0000 7f3112724640 1 --2- 172.21.15.49:0/4244535867 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f30f8005190 0x7f30f800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:37.878 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.850+0000 7f3111722640 1 -- 172.21.15.49:0/4244535867 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f30f8005190 msgr2=0x7f30f800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:37.878 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.850+0000 7f3111722640 1 --2- 172.21.15.49:0/4244535867 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f30f8005190 0x7f30f800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:37.879 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.850+0000 7f3111722640 1 -- 172.21.15.49:0/4244535867 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30f8003530 con 0x7f30f80047d0 2023-11-06T02:10:37.879 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.850+0000 7f3112724640 1 --2- 172.21.15.49:0/4244535867 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f30f8005190 0x7f30f800b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:10:37.879 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.850+0000 7f3111f23640 1 --2- 172.21.15.49:0/4244535867 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f30f8003980 0x7f30f8003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:10:37.879 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.850+0000 7f3112724640 1 -- 172.21.15.49:0/4244535867 reap_dead start 2023-11-06T02:10:37.879 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.854+0000 7f3111722640 1 --2- 172.21.15.49:0/4244535867 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30f80047d0 0x7f30f8004c50 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f30fc00c240 tx=0x7f30fc01f3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=714d1b77b559d381 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:37.880 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.854+0000 7f3110f21640 1 -- 172.21.15.49:0/4244535867 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30fc005ae0 con 0x7f30f80047d0 2023-11-06T02:10:37.880 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.854+0000 7f3110f21640 1 -- 172.21.15.49:0/4244535867 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30fc0ab070 con 0x7f30f80047d0 2023-11-06T02:10:37.880 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.854+0000 7f3110f21640 1 -- 172.21.15.49:0/4244535867 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30fc007070 con 0x7f30f80047d0 2023-11-06T02:10:37.880 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.854+0000 7f3114187640 1 -- 172.21.15.49:0/4244535867 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30f80047d0 msgr2=0x7f30f8004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:37.880 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.854+0000 7f3114187640 1 --2- 172.21.15.49:0/4244535867 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30f80047d0 0x7f30f8004c50 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f30fc00c240 tx=0x7f30fc01f3b0 comp rx=0 tx=0).stop 2023-11-06T02:10:37.881 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.854+0000 7f3114187640 1 -- 172.21.15.49:0/4244535867 shutdown_connections 2023-11-06T02:10:37.881 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.854+0000 7f3114187640 1 --2- 172.21.15.49:0/4244535867 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30f80047d0 0x7f30f8004c50 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:37.881 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.854+0000 7f3114187640 1 -- 172.21.15.49:0/4244535867 >> 172.21.15.49:0/4244535867 conn(0x7f310c0d2aa0 msgr2=0x7f310c0d2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:37.881 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.854+0000 7f3114187640 1 -- 172.21.15.49:0/4244535867 shutdown_connections 2023-11-06T02:10:37.881 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.854+0000 7f3114187640 1 -- 172.21.15.49:0/4244535867 wait complete. 2023-11-06T02:10:37.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.854+0000 7f3114187640 1 Processor -- start 2023-11-06T02:10:37.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.854+0000 7f3114187640 1 -- start start 2023-11-06T02:10:37.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.854+0000 7f3114187640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f30f80047d0 0x7f310c0b7330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:37.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.854+0000 7f3114187640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f310c149e10 0x7f310c0ca950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:37.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.854+0000 7f3114187640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f310c0cae90 0x7f310c1f5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:37.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.854+0000 7f3114187640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f310c0cb310 con 0x7f310c149e10 2023-11-06T02:10:37.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.854+0000 7f3114187640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f310c14ac30 con 0x7f30f80047d0 2023-11-06T02:10:37.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.854+0000 7f3114187640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f310c14af10 con 0x7f310c0cae90 2023-11-06T02:10:37.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.854+0000 7f3111722640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f310c149e10 0x7f310c0ca950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:37.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.854+0000 7f3111722640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f310c149e10 0x7f310c0ca950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44204/0 (socket says 172.21.15.49:44204) 2023-11-06T02:10:37.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.854+0000 7f3111722640 1 -- 172.21.15.49:0/709718472 learned_addr learned my addr 172.21.15.49:0/709718472 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:37.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.854+0000 7f3111f23640 1 --2- 172.21.15.49:0/709718472 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f30f80047d0 0x7f310c0b7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:37.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.858+0000 7f3112724640 1 --2- 172.21.15.49:0/709718472 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f310c0cae90 0x7f310c1f5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:37.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.858+0000 7f3111722640 1 -- 172.21.15.49:0/709718472 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f310c0cae90 msgr2=0x7f310c1f5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:37.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.858+0000 7f3111722640 1 --2- 172.21.15.49:0/709718472 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f310c0cae90 0x7f310c1f5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:37.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.858+0000 7f3111722640 1 -- 172.21.15.49:0/709718472 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f30f80047d0 msgr2=0x7f310c0b7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:37.885 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.858+0000 7f3111722640 1 --2- 172.21.15.49:0/709718472 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f30f80047d0 0x7f310c0b7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:37.885 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.858+0000 7f3111722640 1 -- 172.21.15.49:0/709718472 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f310c14b1f0 con 0x7f310c149e10 2023-11-06T02:10:37.885 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.858+0000 7f3112724640 1 --2- 172.21.15.49:0/709718472 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f310c0cae90 0x7f310c1f5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:10:37.885 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.858+0000 7f3111722640 1 --2- 172.21.15.49:0/709718472 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f310c149e10 0x7f310c0ca950 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f30fc01f1e0 tx=0x7f30f8005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:37.886 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.858+0000 7f31027fc640 1 -- 172.21.15.49:0/709718472 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30fc001680 con 0x7f310c149e10 2023-11-06T02:10:37.886 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.858+0000 7f31027fc640 1 -- 172.21.15.49:0/709718472 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30fc007070 con 0x7f310c149e10 2023-11-06T02:10:37.886 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.858+0000 7f3114187640 1 -- 172.21.15.49:0/709718472 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f310c14b4d0 con 0x7f310c149e10 2023-11-06T02:10:37.886 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.858+0000 7f31027fc640 1 -- 172.21.15.49:0/709718472 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30fc007400 con 0x7f310c149e10 2023-11-06T02:10:37.886 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.858+0000 7f3114187640 1 -- 172.21.15.49:0/709718472 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f310c14a290 con 0x7f310c149e10 2023-11-06T02:10:37.887 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.858+0000 7f3114187640 1 -- 172.21.15.49:0/709718472 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f310c074040 con 0x7f310c149e10 2023-11-06T02:10:37.887 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.858+0000 7f31027fc640 1 -- 172.21.15.49:0/709718472 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f30fc0b9030 con 0x7f310c149e10 2023-11-06T02:10:37.887 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.858+0000 7f31027fc640 1 --2- 172.21.15.49:0/709718472 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f30f0065a60 0x7f30f0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:37.887 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.858+0000 7f31027fc640 1 -- 172.21.15.49:0/709718472 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f30fc0b7070 con 0x7f310c149e10 2023-11-06T02:10:37.887 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.858+0000 7f3111f23640 1 --2- 172.21.15.49:0/709718472 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f30f0065a60 0x7f30f0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:37.887 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.858+0000 7f3111f23640 1 --2- 172.21.15.49:0/709718472 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f30f0065a60 0x7f30f0067f20 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f310c063940 tx=0x7f3108008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:37.888 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:37.862+0000 7f31027fc640 1 -- 172.21.15.49:0/709718472 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f30fc0ed960 con 0x7f310c149e10 2023-11-06T02:10:38.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:38.050+0000 7f3114187640 1 -- 172.21.15.49:0/709718472 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f310c0d0c30 con 0x7f310c149e10 2023-11-06T02:10:38.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:38.050+0000 7f31027fc640 1 -- 172.21.15.49:0/709718472 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v23) v1 ==== 53+0+73 (secure 0 0 0) 0x7f30fc0c1020 con 0x7f310c149e10 2023-11-06T02:10:38.048 INFO:teuthology.orchestra.run.smithi049.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-06T02:10:38.050 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:38.054+0000 7f3114187640 1 -- 172.21.15.49:0/709718472 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f30f0065a60 msgr2=0x7f30f0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:38.050 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:38.054+0000 7f3114187640 1 --2- 172.21.15.49:0/709718472 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f30f0065a60 0x7f30f0067f20 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f310c063940 tx=0x7f3108008040 comp rx=0 tx=0).stop 2023-11-06T02:10:38.050 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:38.054+0000 7f3114187640 1 -- 172.21.15.49:0/709718472 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f310c149e10 msgr2=0x7f310c0ca950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:38.050 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:38.054+0000 7f3114187640 1 --2- 172.21.15.49:0/709718472 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f310c149e10 0x7f310c0ca950 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f30fc01f1e0 tx=0x7f30f8005800 comp rx=0 tx=0).stop 2023-11-06T02:10:38.050 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:38.054+0000 7f3114187640 1 -- 172.21.15.49:0/709718472 shutdown_connections 2023-11-06T02:10:38.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:38.054+0000 7f3114187640 1 --2- 172.21.15.49:0/709718472 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f30f0065a60 0x7f30f0067f20 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:38.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:38.054+0000 7f3114187640 1 --2- 172.21.15.49:0/709718472 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f310c149e10 0x7f310c0ca950 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:38.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:38.054+0000 7f3114187640 1 --2- 172.21.15.49:0/709718472 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f310c0cae90 0x7f310c1f5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:38.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:38.054+0000 7f3114187640 1 --2- 172.21.15.49:0/709718472 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f30f80047d0 0x7f310c0b7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:38.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:38.054+0000 7f3114187640 1 -- 172.21.15.49:0/709718472 >> 172.21.15.49:0/709718472 conn(0x7f310c0d2aa0 msgr2=0x7f310c0c6540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:38.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:38.054+0000 7f3114187640 1 -- 172.21.15.49:0/709718472 shutdown_connections 2023-11-06T02:10:38.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:38.054+0000 7f3114187640 1 -- 172.21.15.49:0/709718472 wait complete. 2023-11-06T02:10:38.065 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-06T02:10:38.065 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-06T02:10:38.066 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi049.front.sepia.ceph.com 2023-11-06T02:10:38.066 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-06T02:10:38.066 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-06T02:10:38.067 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-06T02:10:38.067 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-06T02:10:38.067 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-06T02:10:38.068 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-06T02:10:38.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:10:38.068 DEBUG:teuthology.orchestra.run.smithi049:> ip addr 2023-11-06T02:10:38.075 INFO:teuthology.orchestra.run.smithi049.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-06T02:10:38.075 INFO:teuthology.orchestra.run.smithi049.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-06T02:10:38.075 INFO:teuthology.orchestra.run.smithi049.stdout: inet 127.0.0.1/8 scope host lo 2023-11-06T02:10:38.076 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:10:38.076 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 ::1/128 scope host 2023-11-06T02:10:38.076 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:10:38.076 INFO:teuthology.orchestra.run.smithi049.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-06T02:10:38.076 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:6b:ff:84 brd ff:ff:ff:ff:ff:ff 2023-11-06T02:10:38.077 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp7s0f0 2023-11-06T02:10:38.077 INFO:teuthology.orchestra.run.smithi049.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-06T02:10:38.077 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:6b:ff:85 brd ff:ff:ff:ff:ff:ff 2023-11-06T02:10:38.077 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp7s0f1 2023-11-06T02:10:38.077 INFO:teuthology.orchestra.run.smithi049.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-06T02:10:38.077 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:bd:15:66 brd ff:ff:ff:ff:ff:ff 2023-11-06T02:10:38.078 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp1s0f0 2023-11-06T02:10:38.078 INFO:teuthology.orchestra.run.smithi049.stdout: inet 172.21.15.49/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-11-06T02:10:38.078 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft 42359sec preferred_lft 42359sec 2023-11-06T02:10:38.078 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 fe80::ec4:7aff:febd:1566/64 scope link 2023-11-06T02:10:38.078 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:10:38.079 INFO:teuthology.orchestra.run.smithi049.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-06T02:10:38.079 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:bd:15:67 brd ff:ff:ff:ff:ff:ff 2023-11-06T02:10:38.079 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp1s0f1 2023-11-06T02:10:38.079 INFO:teuthology.orchestra.run.smithi049.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-06T02:10:38.080 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 02:42:47:49:33:69 brd ff:ff:ff:ff:ff:ff 2023-11-06T02:10:38.080 INFO:teuthology.orchestra.run.smithi049.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-06T02:10:38.080 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:10:38.080 INFO:teuthology.orchestra.run.smithi049.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-06T02:10:38.080 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-11-06T02:10:38.081 INFO:teuthology.orchestra.run.smithi049.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-06T02:10:38.081 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:10:38.081 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-11-06T02:10:38.081 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:10:38.081 INFO:teuthology.orchestra.run.smithi049.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-06T02:10:38.082 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-06T02:10:38.082 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-11-06T02:10:38.082 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:10:38.082 INFO:teuthology.orchestra.run.smithi049.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-06T02:10:38.082 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-06T02:10:38.083 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-11-06T02:10:38.083 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:10:38.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:10:38.084 DEBUG:teuthology.orchestra.run.smithi049:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-06T02:10:38.128 INFO:teuthology.orchestra.run.smithi049.stdout:1 2023-11-06T02:10:38.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:10:38.130 DEBUG:teuthology.orchestra.run.smithi049:> route 2023-11-06T02:10:38.180 INFO:teuthology.orchestra.run.smithi049.stdout:Kernel IP routing table 2023-11-06T02:10:38.180 INFO:teuthology.orchestra.run.smithi049.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-06T02:10:38.181 INFO:teuthology.orchestra.run.smithi049.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-11-06T02:10:38.181 INFO:teuthology.orchestra.run.smithi049.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-06T02:10:38.181 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-11-06T02:10:38.182 INFO:teuthology.orchestra.run.smithi049.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-06T02:10:38.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:10:38.183 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec bash -c ' 2023-11-06T02:10:38.184 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2023-11-06T02:10:38.185 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-06T02:10:38.185 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-06T02:10:38.186 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-06T02:10:38.187 DEBUG:teuthology.orchestra.run.smithi049:> ') 2023-11-06T02:10:38.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:10:38.254 DEBUG:teuthology.orchestra.run.smithi049:> ip netns list 2023-11-06T02:10:38.301 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-06T02:10:38.302 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-06T02:10:38.302 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-06T02:10:38.302 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:10:38.303 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link set brx.0 up 2023-11-06T02:10:38.358 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-06T02:10:38.359 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-06T02:10:38.359 DEBUG:teuthology.orchestra.run.smithi049:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-06T02:10:38.418 INFO:teuthology.orchestra.run.smithi049.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-06T02:10:38.419 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-06T02:10:38.419 DEBUG:teuthology.orchestra.run.smithi049:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-06T02:10:38.470 DEBUG:teuthology.orchestra.run.smithi049:> sudo modprobe fuse 2023-11-06T02:10:38.527 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-06T02:10:38.528 DEBUG:teuthology.orchestra.run.smithi049:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-06T02:10:38.581 INFO:teuthology.orchestra.run.smithi049.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-06T02:10:38.583 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-06T02:10:38.583 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:10:38.584 DEBUG:teuthology.orchestra.run.smithi049:> ls /sys/fs/fuse/connections 2023-11-06T02:10:38.630 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-06T02:10:38.630 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-06T02:10:38.675 DEBUG:teuthology.orchestra.run.smithi049:> sudo modprobe fuse 2023-11-06T02:10:38.691 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-06T02:10:38.691 DEBUG:teuthology.orchestra.run.smithi049:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-06T02:10:38.744 INFO:teuthology.orchestra.run.smithi049.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-06T02:10:38.745 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-06T02:10:38.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:10:38.746 DEBUG:teuthology.orchestra.run.smithi049:> ls /sys/fs/fuse/connections 2023-11-06T02:10:38.757 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi049.stderr:2023-11-06T02:10:38.762+0000 7f530a9e9580 -1 init, newargv = 0x5615504d6e40 newargc=15 2023-11-06T02:10:38.758 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi049.stderr:ceph-fuse[50158]: starting ceph client 2023-11-06T02:10:38.762 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi049.stderr:ceph-fuse[50158]: starting fuse 2023-11-06T02:10:39.864 DEBUG:teuthology.orchestra.run.smithi049:> sudo modprobe fuse 2023-11-06T02:10:39.882 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-06T02:10:39.883 DEBUG:teuthology.orchestra.run.smithi049:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-06T02:10:39.938 INFO:teuthology.orchestra.run.smithi049.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-06T02:10:39.939 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-06T02:10:39.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:10:39.940 DEBUG:teuthology.orchestra.run.smithi049:> ls /sys/fs/fuse/connections 2023-11-06T02:10:39.986 INFO:teuthology.orchestra.run.smithi049.stdout:45 2023-11-06T02:10:39.987 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2023-11-06T02:10:39.987 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-11-06T02:10:39.988 DEBUG:teuthology.orchestra.run.smithi049:> import glob 2023-11-06T02:10:39.988 DEBUG:teuthology.orchestra.run.smithi049:> import re 2023-11-06T02:10:39.988 DEBUG:teuthology.orchestra.run.smithi049:> import os 2023-11-06T02:10:39.989 DEBUG:teuthology.orchestra.run.smithi049:> import subprocess 2023-11-06T02:10:39.989 DEBUG:teuthology.orchestra.run.smithi049:> 2023-11-06T02:10:39.990 DEBUG:teuthology.orchestra.run.smithi049:> def _find_admin_socket(client_name): 2023-11-06T02:10:39.990 DEBUG:teuthology.orchestra.run.smithi049:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-06T02:10:39.990 DEBUG:teuthology.orchestra.run.smithi049:> files = glob.glob(asok_path) 2023-11-06T02:10:39.991 DEBUG:teuthology.orchestra.run.smithi049:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-06T02:10:39.991 DEBUG:teuthology.orchestra.run.smithi049:> 2023-11-06T02:10:39.991 DEBUG:teuthology.orchestra.run.smithi049:> # Given a non-glob path, it better be there 2023-11-06T02:10:39.992 DEBUG:teuthology.orchestra.run.smithi049:> if "*" not in asok_path: 2023-11-06T02:10:39.992 DEBUG:teuthology.orchestra.run.smithi049:> assert(len(files) == 1) 2023-11-06T02:10:39.993 DEBUG:teuthology.orchestra.run.smithi049:> return files[0] 2023-11-06T02:10:39.993 DEBUG:teuthology.orchestra.run.smithi049:> 2023-11-06T02:10:39.993 DEBUG:teuthology.orchestra.run.smithi049:> for f in files: 2023-11-06T02:10:39.994 DEBUG:teuthology.orchestra.run.smithi049:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-06T02:10:39.994 DEBUG:teuthology.orchestra.run.smithi049:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-06T02:10:39.994 DEBUG:teuthology.orchestra.run.smithi049:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-06T02:10:39.995 DEBUG:teuthology.orchestra.run.smithi049:> contents = proc_f.read() 2023-11-06T02:10:39.995 DEBUG:teuthology.orchestra.run.smithi049:> if mountpoint in contents: 2023-11-06T02:10:39.995 DEBUG:teuthology.orchestra.run.smithi049:> return f 2023-11-06T02:10:39.996 DEBUG:teuthology.orchestra.run.smithi049:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-06T02:10:39.996 DEBUG:teuthology.orchestra.run.smithi049:> 2023-11-06T02:10:39.997 DEBUG:teuthology.orchestra.run.smithi049:> print(_find_admin_socket("client.0")) 2023-11-06T02:10:39.997 DEBUG:teuthology.orchestra.run.smithi049:> ' 2023-11-06T02:10:40.110 INFO:teuthology.orchestra.run.smithi049.stdout:/var/run/ceph/ceph-client.0.50158.asok 2023-11-06T02:10:40.293 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.50158.asok 2023-11-06T02:10:40.293 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:10:40.294 DEBUG:teuthology.orchestra.run.smithi049:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.50158.asok status 2023-11-06T02:10:40.453 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-11-06T02:10:40.453 INFO:teuthology.orchestra.run.smithi049.stdout: "metadata": { 2023-11-06T02:10:40.453 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_sha1": "f7e9f9af51acf98e0ccc3037487018e4ae40caa5", 2023-11-06T02:10:40.453 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_version": "ceph version 18.2.0-1181-gf7e9f9af (f7e9f9af51acf98e0ccc3037487018e4ae40caa5) reef (stable)", 2023-11-06T02:10:40.454 INFO:teuthology.orchestra.run.smithi049.stdout: "entity_id": "0", 2023-11-06T02:10:40.454 INFO:teuthology.orchestra.run.smithi049.stdout: "hostname": "smithi049", 2023-11-06T02:10:40.454 INFO:teuthology.orchestra.run.smithi049.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-06T02:10:40.454 INFO:teuthology.orchestra.run.smithi049.stdout: "pid": "50158", 2023-11-06T02:10:40.454 INFO:teuthology.orchestra.run.smithi049.stdout: "root": "/" 2023-11-06T02:10:40.455 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:10:40.455 INFO:teuthology.orchestra.run.smithi049.stdout: "dentry_count": 0, 2023-11-06T02:10:40.455 INFO:teuthology.orchestra.run.smithi049.stdout: "dentry_pinned_count": 0, 2023-11-06T02:10:40.455 INFO:teuthology.orchestra.run.smithi049.stdout: "id": 6036, 2023-11-06T02:10:40.455 INFO:teuthology.orchestra.run.smithi049.stdout: "inst": { 2023-11-06T02:10:40.456 INFO:teuthology.orchestra.run.smithi049.stdout: "name": { 2023-11-06T02:10:40.456 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "client", 2023-11-06T02:10:40.456 INFO:teuthology.orchestra.run.smithi049.stdout: "num": 6036 2023-11-06T02:10:40.456 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:10:40.457 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": { 2023-11-06T02:10:40.457 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2023-11-06T02:10:40.457 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "192.168.0.1:0", 2023-11-06T02:10:40.457 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 2384454810 2023-11-06T02:10:40.458 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:10:40.458 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:10:40.458 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": { 2023-11-06T02:10:40.458 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2023-11-06T02:10:40.459 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "192.168.0.1:0", 2023-11-06T02:10:40.459 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 2384454810 2023-11-06T02:10:40.459 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:10:40.460 INFO:teuthology.orchestra.run.smithi049.stdout: "inst_str": "client.6036 192.168.0.1:0/2384454810", 2023-11-06T02:10:40.460 INFO:teuthology.orchestra.run.smithi049.stdout: "addr_str": "192.168.0.1:0/2384454810", 2023-11-06T02:10:40.460 INFO:teuthology.orchestra.run.smithi049.stdout: "inode_count": 1, 2023-11-06T02:10:40.460 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_epoch": 23, 2023-11-06T02:10:40.460 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_epoch": 51, 2023-11-06T02:10:40.460 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_epoch_barrier": 0, 2023-11-06T02:10:40.461 INFO:teuthology.orchestra.run.smithi049.stdout: "blocklisted": false, 2023-11-06T02:10:40.461 INFO:teuthology.orchestra.run.smithi049.stdout: "fs_name": "cephfs" 2023-11-06T02:10:40.461 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-11-06T02:10:40.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:10:40.468 DEBUG:teuthology.orchestra.run.smithi049:> stat --file-system '--printf=%T 2023-11-06T02:10:40.468 DEBUG:teuthology.orchestra.run.smithi049:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-06T02:10:40.475 INFO:teuthology.orchestra.run.smithi049.stdout:fuseblk 2023-11-06T02:10:40.475 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-06T02:10:40.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:10:40.476 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-06T02:10:40.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:10:40.532 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent_a) 2023-11-06T02:10:40.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:10:40.589 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parent_a/alpha) 2023-11-06T02:10:40.640 DEBUG:teuthology.orchestra.run.smithi049:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-06T02:10:40.640 DEBUG:teuthology.orchestra.run.smithi049:> import os 2023-11-06T02:10:40.641 DEBUG:teuthology.orchestra.run.smithi049:> import stat 2023-11-06T02:10:40.641 DEBUG:teuthology.orchestra.run.smithi049:> 2023-11-06T02:10:40.641 DEBUG:teuthology.orchestra.run.smithi049:> print(os.stat("/home/ubuntu/cephtest/mnt.0/parent_a/alpha").st_ino) 2023-11-06T02:10:40.642 DEBUG:teuthology.orchestra.run.smithi049:> ' 2023-11-06T02:10:40.747 INFO:teuthology.orchestra.run.smithi049.stdout:1099511627777 2023-11-06T02:10:40.934 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T02:10:41.109 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.110+0000 7ff8ebbaf640 1 Processor -- start 2023-11-06T02:10:41.109 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.114+0000 7ff8ebbaf640 1 -- start start 2023-11-06T02:10:41.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.114+0000 7ff8ebbaf640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff8e40b1970 0x7ff8e406b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:41.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.114+0000 7ff8ebbaf640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8e406bce0 con 0x7ff8e40b1420 2023-11-06T02:10:41.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.114+0000 7ff8ebbaf640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8e40cf710 con 0x7ff8e40b0d80 2023-11-06T02:10:41.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.114+0000 7ff8ebbaf640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8e40cffb0 con 0x7ff8e40b1970 2023-11-06T02:10:41.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.114+0000 7ff8ea14c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff8e40b1970 0x7ff8e406b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:41.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.114+0000 7ff8ea14c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff8e40b1970 0x7ff8e406b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40220/0 (socket says 172.21.15.49:40220) 2023-11-06T02:10:41.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.114+0000 7ff8ea14c640 1 -- 172.21.15.49:0/1826751374 learned_addr learned my addr 172.21.15.49:0/1826751374 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:41.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.114+0000 7ff8ea14c640 1 -- 172.21.15.49:0/1826751374 >> v1:172.21.15.150:6789/0 conn(0x7ff8e40b0d80 legacy=0x7ff8e406b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:41.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.114+0000 7ff8ea14c640 1 -- 172.21.15.49:0/1826751374 >> v1:172.21.15.49:6789/0 conn(0x7ff8e40b1420 legacy=0x7ff8e40cb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:41.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.114+0000 7ff8ea14c640 1 -- 172.21.15.49:0/1826751374 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8e40cf9f0 con 0x7ff8e40b1970 2023-11-06T02:10:41.114 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.114+0000 7ff8ea14c640 1 --2- 172.21.15.49:0/1826751374 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8e40b1970 0x7ff8e406b7a0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc009e50 tx=0x7ff8dc01c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2564cd8b647c433 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:41.114 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.114+0000 7ff8e8949640 1 -- 172.21.15.49:0/1826751374 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc0a57f0 con 0x7ff8e40b1970 2023-11-06T02:10:41.114 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.118+0000 7ff8e8949640 1 -- 172.21.15.49:0/1826751374 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8dc0a5950 con 0x7ff8e40b1970 2023-11-06T02:10:41.114 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.118+0000 7ff8e8949640 1 -- 172.21.15.49:0/1826751374 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc0a5ce0 con 0x7ff8e40b1970 2023-11-06T02:10:41.114 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.118+0000 7ff8ebbaf640 1 -- 172.21.15.49:0/1826751374 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8e40b1970 msgr2=0x7ff8e406b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:41.115 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.118+0000 7ff8ebbaf640 1 --2- 172.21.15.49:0/1826751374 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8e40b1970 0x7ff8e406b7a0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc009e50 tx=0x7ff8dc01c9a0 comp rx=0 tx=0).stop 2023-11-06T02:10:41.115 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.118+0000 7ff8ebbaf640 1 -- 172.21.15.49:0/1826751374 shutdown_connections 2023-11-06T02:10:41.115 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.118+0000 7ff8ebbaf640 1 --2- 172.21.15.49:0/1826751374 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8e40b1970 0x7ff8e406b7a0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:41.115 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.118+0000 7ff8ebbaf640 1 -- 172.21.15.49:0/1826751374 >> 172.21.15.49:0/1826751374 conn(0x7ff8e40d0890 msgr2=0x7ff8e40d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:41.115 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.118+0000 7ff8ebbaf640 1 -- 172.21.15.49:0/1826751374 shutdown_connections 2023-11-06T02:10:41.116 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.118+0000 7ff8ebbaf640 1 -- 172.21.15.49:0/1826751374 wait complete. 2023-11-06T02:10:41.118 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.118+0000 7ff8ebbaf640 1 Processor -- start 2023-11-06T02:10:41.118 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.118+0000 7ff8ebbaf640 1 -- start start 2023-11-06T02:10:41.118 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.118+0000 7ff8ebbaf640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8e40b0d80 0x7ff8e40af540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:41.118 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.118+0000 7ff8ebbaf640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8e40b1420 0x7ff8e4157210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:41.119 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.118+0000 7ff8ebbaf640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8e40b1970 0x7ff8e40b3480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:41.119 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.118+0000 7ff8ebbaf640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e40c8ed0 con 0x7ff8e40b1970 2023-11-06T02:10:41.119 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.118+0000 7ff8ebbaf640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e40afe90 con 0x7ff8e40b1420 2023-11-06T02:10:41.119 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.118+0000 7ff8ebbaf640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e40b0170 con 0x7ff8e40b0d80 2023-11-06T02:10:41.119 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.118+0000 7ff8ea14c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8e40b1970 0x7ff8e40b3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:41.120 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.118+0000 7ff8ea14c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8e40b1970 0x7ff8e40b3480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40228/0 (socket says 172.21.15.49:40228) 2023-11-06T02:10:41.120 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.118+0000 7ff8ea14c640 1 -- 172.21.15.49:0/2646241856 learned_addr learned my addr 172.21.15.49:0/2646241856 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:41.120 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.118+0000 7ff8e914a640 1 --2- 172.21.15.49:0/2646241856 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8e40b1420 0x7ff8e4157210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:41.120 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.118+0000 7ff8e994b640 1 --2- 172.21.15.49:0/2646241856 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8e40b0d80 0x7ff8e40af540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:41.120 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.118+0000 7ff8ea14c640 1 -- 172.21.15.49:0/2646241856 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8e40b0d80 msgr2=0x7ff8e40af540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:41.120 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.118+0000 7ff8ea14c640 1 --2- 172.21.15.49:0/2646241856 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8e40b0d80 0x7ff8e40af540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:41.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.118+0000 7ff8ea14c640 1 -- 172.21.15.49:0/2646241856 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8e40b1420 msgr2=0x7ff8e4157210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:41.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.118+0000 7ff8ea14c640 1 --2- 172.21.15.49:0/2646241856 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8e40b1420 0x7ff8e4157210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:41.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.118+0000 7ff8ea14c640 1 -- 172.21.15.49:0/2646241856 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8dc009b30 con 0x7ff8e40b1970 2023-11-06T02:10:41.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.118+0000 7ff8e914a640 1 --2- 172.21.15.49:0/2646241856 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8e40b1420 0x7ff8e4157210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:10:41.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.118+0000 7ff8e994b640 1 --2- 172.21.15.49:0/2646241856 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8e40b0d80 0x7ff8e40af540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:10:41.122 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.118+0000 7ff8ea14c640 1 --2- 172.21.15.49:0/2646241856 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8e40b1970 0x7ff8e40b3480 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc0af040 tx=0x7ff8dc019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:41.122 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.118+0000 7ff8da7fc640 1 -- 172.21.15.49:0/2646241856 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc0b8670 con 0x7ff8e40b1970 2023-11-06T02:10:41.122 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.118+0000 7ff8da7fc640 1 -- 172.21.15.49:0/2646241856 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8dc0b87d0 con 0x7ff8e40b1970 2023-11-06T02:10:41.122 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.118+0000 7ff8ebbaf640 1 -- 172.21.15.49:0/2646241856 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8e40c8af0 con 0x7ff8e40b1970 2023-11-06T02:10:41.122 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.118+0000 7ff8da7fc640 1 -- 172.21.15.49:0/2646241856 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc0b8b40 con 0x7ff8e40b1970 2023-11-06T02:10:41.123 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.118+0000 7ff8ebbaf640 1 -- 172.21.15.49:0/2646241856 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8e40afc50 con 0x7ff8e40b1970 2023-11-06T02:10:41.123 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.122+0000 7ff8ebbaf640 1 -- 172.21.15.49:0/2646241856 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8e40c9010 con 0x7ff8e40b1970 2023-11-06T02:10:41.124 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.122+0000 7ff8da7fc640 1 -- 172.21.15.49:0/2646241856 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7ff8dc0b8ca0 con 0x7ff8e40b1970 2023-11-06T02:10:41.124 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.122+0000 7ff8da7fc640 1 --2- 172.21.15.49:0/2646241856 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff8c8065a60 0x7ff8c8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:41.124 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.122+0000 7ff8da7fc640 1 -- 172.21.15.49:0/2646241856 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff8dc11e880 con 0x7ff8e40b1970 2023-11-06T02:10:41.124 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.122+0000 7ff8e994b640 1 --2- 172.21.15.49:0/2646241856 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff8c8065a60 0x7ff8c8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:41.124 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.122+0000 7ff8e994b640 1 --2- 172.21.15.49:0/2646241856 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff8c8065a60 0x7ff8c8067f20 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4158490 tx=0x7ff8d4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:41.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.126+0000 7ff8da7fc640 1 -- 172.21.15.49:0/2646241856 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7ff8e40c9010 con 0x7ff8e40b1970 2023-11-06T02:10:41.318 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.322+0000 7ff8ebbaf640 1 -- 172.21.15.49:0/2646241856 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff8e40afc50 con 0x7ff8e40b1970 2023-11-06T02:10:41.319 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.322+0000 7ff8da7fc640 1 -- 172.21.15.49:0/2646241856 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4648 (secure 0 0 0) 0x7ff8e40afc50 con 0x7ff8e40b1970 2023-11-06T02:10:41.319 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:10:41.319 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5096,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1396196190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1396196190},{"type":"v1","addr":"172.21.15.150:6835","nonce":1396196190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5102,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/2465209170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":2465209170},{"type":"v1","addr":"172.21.15.150:6837","nonce":2465209170}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5871,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/1006951804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1006951804},{"type":"v1","addr":"172.21.15.49:6833","nonce":1006951804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T02:10:31.903864+0000","modified":"2023-11-06T02:10:34.930237+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5877},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5877":{"gid":5877,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.49:6835/73939548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":73939548},{"type":"v1","addr":"172.21.15.49:6835","nonce":73939548}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-06T02:10:41.320 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 23 2023-11-06T02:10:41.321 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.322+0000 7ff8ebbaf640 1 -- 172.21.15.49:0/2646241856 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff8c8065a60 msgr2=0x7ff8c8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:41.321 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.322+0000 7ff8ebbaf640 1 --2- 172.21.15.49:0/2646241856 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff8c8065a60 0x7ff8c8067f20 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4158490 tx=0x7ff8d4009040 comp rx=0 tx=0).stop 2023-11-06T02:10:41.322 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.322+0000 7ff8ebbaf640 1 -- 172.21.15.49:0/2646241856 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8e40b1970 msgr2=0x7ff8e40b3480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:41.322 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.322+0000 7ff8ebbaf640 1 --2- 172.21.15.49:0/2646241856 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8e40b1970 0x7ff8e40b3480 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc0af040 tx=0x7ff8dc019740 comp rx=0 tx=0).stop 2023-11-06T02:10:41.322 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.322+0000 7ff8ebbaf640 1 -- 172.21.15.49:0/2646241856 shutdown_connections 2023-11-06T02:10:41.322 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.326+0000 7ff8ebbaf640 1 --2- 172.21.15.49:0/2646241856 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff8c8065a60 0x7ff8c8067f20 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:41.322 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.326+0000 7ff8ebbaf640 1 --2- 172.21.15.49:0/2646241856 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8e40b1970 0x7ff8e40b3480 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:41.323 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.326+0000 7ff8ebbaf640 1 --2- 172.21.15.49:0/2646241856 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8e40b1420 0x7ff8e4157210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:41.323 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.326+0000 7ff8ebbaf640 1 --2- 172.21.15.49:0/2646241856 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8e40b0d80 0x7ff8e40af540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:41.323 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.326+0000 7ff8ebbaf640 1 -- 172.21.15.49:0/2646241856 >> 172.21.15.49:0/2646241856 conn(0x7ff8e40d0890 msgr2=0x7ff8e406f0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:41.323 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.326+0000 7ff8ebbaf640 1 -- 172.21.15.49:0/2646241856 shutdown_connections 2023-11-06T02:10:41.324 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.326+0000 7ff8ebbaf640 1 -- 172.21.15.49:0/2646241856 wait complete. 2023-11-06T02:10:41.339 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2023-11-06T02:10:41.486 INFO:teuthology.orchestra.run.smithi049.stdout:{"message":"","return_code":0} 2023-11-06T02:10:41.487 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2023-11-06T02:10:41.487 DEBUG:teuthology.orchestra.run.smithi049:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2023-11-06T02:10:41.526 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.530+0000 7f3a389ac080 1 Processor -- start 2023-11-06T02:10:41.526 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.530+0000 7f3a389ac080 1 -- start start 2023-11-06T02:10:41.529 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.530+0000 7f3a389ac080 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5632105f6c90 0x5632105245e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:41.529 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.530+0000 7f3a389ac080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563210522650 0x5632105f7070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:41.529 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.530+0000 7f3a389ac080 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56321056b000 0x56321056b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:41.529 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.530+0000 7f3a389ac080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x563210468450 con 0x563210522650 2023-11-06T02:10:41.529 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.530+0000 7f3a389ac080 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x563210468150 con 0x56321056b000 2023-11-06T02:10:41.530 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.530+0000 7f3a389ac080 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x56321045edc0 con 0x5632105f6c90 2023-11-06T02:10:41.530 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.530+0000 7f3a35cc5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563210522650 0x5632105f7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:41.530 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.530+0000 7f3a35cc5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563210522650 0x5632105f7070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40242/0 (socket says 172.21.15.49:40242) 2023-11-06T02:10:41.530 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.530+0000 7f3a35cc5640 1 -- 172.21.15.49:0/3014905237 learned_addr learned my addr 172.21.15.49:0/3014905237 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:41.530 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.530+0000 7f3a364c6640 1 --2- 172.21.15.49:0/3014905237 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5632105f6c90 0x5632105245e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:41.531 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.530+0000 7f3a36cc7640 1 --2- 172.21.15.49:0/3014905237 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56321056b000 0x56321056b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:41.531 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.530+0000 7f3a35cc5640 1 -- 172.21.15.49:0/3014905237 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5632105f6c90 msgr2=0x5632105245e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:41.531 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.530+0000 7f3a35cc5640 1 --2- 172.21.15.49:0/3014905237 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5632105f6c90 0x5632105245e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:41.531 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.530+0000 7f3a35cc5640 1 -- 172.21.15.49:0/3014905237 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56321056b000 msgr2=0x56321056b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:41.531 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.530+0000 7f3a35cc5640 1 --2- 172.21.15.49:0/3014905237 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56321056b000 0x56321056b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:41.532 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.530+0000 7f3a35cc5640 1 -- 172.21.15.49:0/3014905237 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563210476ff0 con 0x563210522650 2023-11-06T02:10:41.532 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.530+0000 7f3a36cc7640 1 --2- 172.21.15.49:0/3014905237 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56321056b000 0x56321056b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:10:41.532 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.530+0000 7f3a35cc5640 1 --2- 172.21.15.49:0/3014905237 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563210522650 0x5632105f7070 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f3a20009940 tx=0x7f3a20009e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=555e9a809669c8cf server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:41.532 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.530+0000 7f3a34cc3640 1 -- 172.21.15.49:0/3014905237 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a20018070 con 0x563210522650 2023-11-06T02:10:41.532 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.530+0000 7f3a34cc3640 1 -- 172.21.15.49:0/3014905237 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a20010040 con 0x563210522650 2023-11-06T02:10:41.532 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.530+0000 7f3a34cc3640 1 -- 172.21.15.49:0/3014905237 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a200143f0 con 0x563210522650 2023-11-06T02:10:41.533 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.530+0000 7f3a389ac080 1 -- 172.21.15.49:0/3014905237 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563210522650 msgr2=0x5632105f7070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:41.533 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.530+0000 7f3a389ac080 1 --2- 172.21.15.49:0/3014905237 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563210522650 0x5632105f7070 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f3a20009940 tx=0x7f3a20009e00 comp rx=0 tx=0).stop 2023-11-06T02:10:41.533 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.530+0000 7f3a389ac080 1 -- 172.21.15.49:0/3014905237 shutdown_connections 2023-11-06T02:10:41.533 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.530+0000 7f3a389ac080 1 --2- 172.21.15.49:0/3014905237 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563210522650 0x5632105f7070 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:41.533 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.530+0000 7f3a389ac080 1 --2- 172.21.15.49:0/3014905237 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56321056b000 0x56321056b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:41.534 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.530+0000 7f3a389ac080 1 --2- 172.21.15.49:0/3014905237 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5632105f6c90 0x5632105245e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:41.534 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.530+0000 7f3a389ac080 1 -- 172.21.15.49:0/3014905237 >> 172.21.15.49:0/3014905237 conn(0x56321054d020 msgr2=0x563210569510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:41.534 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.530+0000 7f3a389ac080 1 -- 172.21.15.49:0/3014905237 shutdown_connections 2023-11-06T02:10:41.534 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.530+0000 7f3a389ac080 1 -- 172.21.15.49:0/3014905237 wait complete. 2023-11-06T02:10:41.534 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.530+0000 7f3a389ac080 1 Processor -- start 2023-11-06T02:10:41.535 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.530+0000 7f3a389ac080 1 -- start start 2023-11-06T02:10:41.538 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.534+0000 7f3a389ac080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563210522650 0x563210544cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:41.538 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.534+0000 7f3a389ac080 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56321056b000 0x563210680400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:41.538 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.534+0000 7f3a389ac080 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5632105f6c90 0x5632105691f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:41.538 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.534+0000 7f3a389ac080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x563210680bf0 con 0x563210522650 2023-11-06T02:10:41.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.534+0000 7f3a389ac080 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x563210680e90 con 0x56321056b000 2023-11-06T02:10:41.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.534+0000 7f3a389ac080 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x563210681130 con 0x5632105f6c90 2023-11-06T02:10:41.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.534+0000 7f3a364c6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563210522650 0x563210544cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:41.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.534+0000 7f3a364c6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563210522650 0x563210544cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40248/0 (socket says 172.21.15.49:40248) 2023-11-06T02:10:41.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.534+0000 7f3a364c6640 1 -- 172.21.15.49:0/1410463351 learned_addr learned my addr 172.21.15.49:0/1410463351 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:41.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.534+0000 7f3a36cc7640 1 --2- 172.21.15.49:0/1410463351 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5632105f6c90 0x5632105691f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:41.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.534+0000 7f3a35cc5640 1 --2- 172.21.15.49:0/1410463351 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56321056b000 0x563210680400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:41.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.534+0000 7f3a364c6640 1 -- 172.21.15.49:0/1410463351 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5632105f6c90 msgr2=0x5632105691f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:41.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.534+0000 7f3a364c6640 1 --2- 172.21.15.49:0/1410463351 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5632105f6c90 0x5632105691f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:41.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.534+0000 7f3a364c6640 1 -- 172.21.15.49:0/1410463351 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56321056b000 msgr2=0x563210680400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:41.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.534+0000 7f3a364c6640 1 --2- 172.21.15.49:0/1410463351 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56321056b000 0x563210680400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:41.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.534+0000 7f3a364c6640 1 -- 172.21.15.49:0/1410463351 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a20009590 con 0x563210522650 2023-11-06T02:10:41.542 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.534+0000 7f3a36cc7640 1 --2- 172.21.15.49:0/1410463351 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5632105f6c90 0x5632105691f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:10:41.542 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.534+0000 7f3a35cc5640 1 --2- 172.21.15.49:0/1410463351 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56321056b000 0x563210680400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:10:41.542 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.534+0000 7f3a364c6640 1 --2- 172.21.15.49:0/1410463351 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563210522650 0x563210544cb0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f3a28011690 tx=0x7f3a28011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:41.542 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.534+0000 7f3a2f7fe640 1 -- 172.21.15.49:0/1410463351 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a28011e60 con 0x563210522650 2023-11-06T02:10:41.543 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.534+0000 7f3a2f7fe640 1 -- 172.21.15.49:0/1410463351 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a2800fa40 con 0x563210522650 2023-11-06T02:10:41.543 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.534+0000 7f3a2f7fe640 1 -- 172.21.15.49:0/1410463351 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a2800fbb0 con 0x563210522650 2023-11-06T02:10:41.543 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.534+0000 7f3a389ac080 1 -- 172.21.15.49:0/1410463351 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56321046a430 con 0x563210522650 2023-11-06T02:10:41.543 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.534+0000 7f3a389ac080 1 -- 172.21.15.49:0/1410463351 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563210469e90 con 0x563210522650 2023-11-06T02:10:41.543 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.534+0000 7f3a2f7fe640 1 -- 172.21.15.49:0/1410463351 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f3a2800fd10 con 0x563210522650 2023-11-06T02:10:41.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.534+0000 7f3a2f7fe640 1 --2- 172.21.15.49:0/1410463351 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3a18064410 0x7f3a180668d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:41.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.534+0000 7f3a2f7fe640 1 -- 172.21.15.49:0/1410463351 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3a28082500 con 0x563210522650 2023-11-06T02:10:41.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.534+0000 7f3a389ac080 1 --2- 172.21.15.49:0/1410463351 >> [v2:172.21.15.49:6808/38113,v1:172.21.15.49:6810/38113] conn(0x563210569730 0x5632105e7490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:41.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.534+0000 7f3a389ac080 1 -- 172.21.15.49:0/1410463351 --> [v2:172.21.15.49:6808/38113,v1:172.21.15.49:6810/38113] -- osd_op(unknown.0.0:1 8.7 8:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51) v8 -- 0x5632105e79d0 con 0x563210569730 2023-11-06T02:10:41.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.534+0000 7f3a35cc5640 1 --2- 172.21.15.49:0/1410463351 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3a18064410 0x7f3a180668d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:41.545 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.534+0000 7f3a36cc7640 1 --2- 172.21.15.49:0/1410463351 >> [v2:172.21.15.49:6808/38113,v1:172.21.15.49:6810/38113] conn(0x563210569730 0x5632105e7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:41.545 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.538+0000 7f3a36cc7640 1 --2- 172.21.15.49:0/1410463351 >> [v2:172.21.15.49:6808/38113,v1:172.21.15.49:6810/38113] conn(0x563210569730 0x5632105e7490 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:41.545 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.538+0000 7f3a36cc7640 1 -- 172.21.15.49:0/1410463351 <== osd.1 v2:172.21.15.49:6808/38113 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+95 (crc 0 0 0) 0x7f3a300014d0 con 0x563210569730 2023-11-06T02:10:41.545 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.538+0000 7f3a389ac080 1 -- 172.21.15.49:0/1410463351 >> [v2:172.21.15.49:6808/38113,v1:172.21.15.49:6810/38113] conn(0x563210569730 msgr2=0x5632105e7490 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:41.545 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.538+0000 7f3a389ac080 1 --2- 172.21.15.49:0/1410463351 >> [v2:172.21.15.49:6808/38113,v1:172.21.15.49:6810/38113] conn(0x563210569730 0x5632105e7490 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:41.545 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.538+0000 7f3a35cc5640 1 --2- 172.21.15.49:0/1410463351 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3a18064410 0x7f3a180668d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f3a200098e0 tx=0x7f3a20009f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:41.546 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.538+0000 7f3a389ac080 1 -- 172.21.15.49:0/1410463351 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3a18064410 msgr2=0x7f3a180668d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:41.546 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.538+0000 7f3a389ac080 1 --2- 172.21.15.49:0/1410463351 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3a18064410 0x7f3a180668d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f3a200098e0 tx=0x7f3a20009f90 comp rx=0 tx=0).stop 2023-11-06T02:10:41.546 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.538+0000 7f3a389ac080 1 -- 172.21.15.49:0/1410463351 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563210522650 msgr2=0x563210544cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:41.546 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.538+0000 7f3a389ac080 1 --2- 172.21.15.49:0/1410463351 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563210522650 0x563210544cb0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f3a28011690 tx=0x7f3a28011b50 comp rx=0 tx=0).stop 2023-11-06T02:10:41.546 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.538+0000 7f3a36cc7640 1 -- 172.21.15.49:0/1410463351 reap_dead start 2023-11-06T02:10:41.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.538+0000 7f3a389ac080 1 -- 172.21.15.49:0/1410463351 shutdown_connections 2023-11-06T02:10:41.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.538+0000 7f3a389ac080 1 -- 172.21.15.49:0/1410463351 >> 172.21.15.49:0/1410463351 conn(0x56321054d020 msgr2=0x563210524320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:41.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.538+0000 7f3a389ac080 1 -- 172.21.15.49:0/1410463351 shutdown_connections 2023-11-06T02:10:41.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.538+0000 7f3a389ac080 1 -- 172.21.15.49:0/1410463351 wait complete. 2023-11-06T02:10:41.548 INFO:teuthology.orchestra.run.smithi049.stdout:Yalphaparent_a 2023-11-06T02:10:41.548 DEBUG:teuthology.orchestra.run.smithi049:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2023-11-06T02:10:41.637 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-11-06T02:10:41.638 INFO:teuthology.orchestra.run.smithi049.stdout: "ino": 1099511627777, 2023-11-06T02:10:41.638 INFO:teuthology.orchestra.run.smithi049.stdout: "ancestors": [ 2023-11-06T02:10:41.638 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:10:41.638 INFO:teuthology.orchestra.run.smithi049.stdout: "dirino": 1099511627776, 2023-11-06T02:10:41.638 INFO:teuthology.orchestra.run.smithi049.stdout: "dname": "alpha", 2023-11-06T02:10:41.639 INFO:teuthology.orchestra.run.smithi049.stdout: "version": 2 2023-11-06T02:10:41.639 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:10:41.639 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:10:41.639 INFO:teuthology.orchestra.run.smithi049.stdout: "dirino": 1, 2023-11-06T02:10:41.639 INFO:teuthology.orchestra.run.smithi049.stdout: "dname": "parent_a", 2023-11-06T02:10:41.640 INFO:teuthology.orchestra.run.smithi049.stdout: "version": 4 2023-11-06T02:10:41.640 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:10:41.640 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-11-06T02:10:41.640 INFO:teuthology.orchestra.run.smithi049.stdout: "pool": 8, 2023-11-06T02:10:41.640 INFO:teuthology.orchestra.run.smithi049.stdout: "old_pools": [] 2023-11-06T02:10:41.641 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-11-06T02:10:41.641 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:10:41.642 DEBUG:teuthology.orchestra.run.smithi049:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent 'oh i'"'"'m sorry did i overwrite your xattr?' 2023-11-06T02:10:41.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.726+0000 7f870ecda080 1 Processor -- start 2023-11-06T02:10:41.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.726+0000 7f870ecda080 1 -- start start 2023-11-06T02:10:41.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.730+0000 7f870ecda080 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86f80030c0 0x55e96883e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:41.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.730+0000 7f870ecda080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e9687a84b0 0x55e968762770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:41.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.730+0000 7f870ecda080 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e968826dc0 0x55e968840750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:41.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.730+0000 7f870ecda080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e96870c940 con 0x55e9687a84b0 2023-11-06T02:10:41.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.730+0000 7f870ecda080 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9686a8450 con 0x7f86f80030c0 2023-11-06T02:10:41.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.730+0000 7f870ecda080 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e9686a8150 con 0x55e968826dc0 2023-11-06T02:10:41.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.730+0000 7f870bff3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e9687a84b0 0x55e968762770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:41.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.730+0000 7f870bff3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e9687a84b0 0x55e968762770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40258/0 (socket says 172.21.15.49:40258) 2023-11-06T02:10:41.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.730+0000 7f870bff3640 1 -- 172.21.15.49:0/655888447 learned_addr learned my addr 172.21.15.49:0/655888447 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:41.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.730+0000 7f870cff5640 1 --2- 172.21.15.49:0/655888447 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e968826dc0 0x55e968840750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:41.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.730+0000 7f870c7f4640 1 --2- 172.21.15.49:0/655888447 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86f80030c0 0x55e96883e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:41.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.730+0000 7f870bff3640 1 -- 172.21.15.49:0/655888447 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e968826dc0 msgr2=0x55e968840750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:41.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.730+0000 7f870bff3640 1 --2- 172.21.15.49:0/655888447 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e968826dc0 0x55e968840750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:41.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.730+0000 7f870bff3640 1 -- 172.21.15.49:0/655888447 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86f80030c0 msgr2=0x55e96883e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:41.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.730+0000 7f870bff3640 1 --2- 172.21.15.49:0/655888447 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86f80030c0 0x55e96883e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:41.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.730+0000 7f870bff3640 1 -- 172.21.15.49:0/655888447 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e9686aa430 con 0x55e9687a84b0 2023-11-06T02:10:41.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.730+0000 7f870bff3640 1 --2- 172.21.15.49:0/655888447 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e9687a84b0 0x55e968762770 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f86f401c100 tx=0x7f86f4001a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85d8da4ea8679b07 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:41.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.730+0000 7f870aff1640 1 -- 172.21.15.49:0/655888447 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f4000ee0 con 0x55e9687a84b0 2023-11-06T02:10:41.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.730+0000 7f870aff1640 1 -- 172.21.15.49:0/655888447 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86f4001440 con 0x55e9687a84b0 2023-11-06T02:10:41.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.730+0000 7f870aff1640 1 -- 172.21.15.49:0/655888447 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f40a4ad0 con 0x55e9687a84b0 2023-11-06T02:10:41.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.730+0000 7f870ecda080 1 -- 172.21.15.49:0/655888447 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e9687a84b0 msgr2=0x55e968762770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:41.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.730+0000 7f870ecda080 1 --2- 172.21.15.49:0/655888447 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e9687a84b0 0x55e968762770 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f86f401c100 tx=0x7f86f4001a10 comp rx=0 tx=0).stop 2023-11-06T02:10:41.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.730+0000 7f870ecda080 1 -- 172.21.15.49:0/655888447 shutdown_connections 2023-11-06T02:10:41.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.730+0000 7f870ecda080 1 --2- 172.21.15.49:0/655888447 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e9687a84b0 0x55e968762770 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:41.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.730+0000 7f870ecda080 1 --2- 172.21.15.49:0/655888447 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e968826dc0 0x55e968840750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:41.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.730+0000 7f870ecda080 1 --2- 172.21.15.49:0/655888447 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86f80030c0 0x55e96883e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:41.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.730+0000 7f870ecda080 1 -- 172.21.15.49:0/655888447 >> 172.21.15.49:0/655888447 conn(0x55e96878bef0 msgr2=0x55e968764510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:41.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.730+0000 7f870ecda080 1 -- 172.21.15.49:0/655888447 shutdown_connections 2023-11-06T02:10:41.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.730+0000 7f870ecda080 1 -- 172.21.15.49:0/655888447 wait complete. 2023-11-06T02:10:41.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.730+0000 7f870ecda080 1 Processor -- start 2023-11-06T02:10:41.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.730+0000 7f870ecda080 1 -- start start 2023-11-06T02:10:41.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.730+0000 7f870ecda080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e9687a84b0 0x55e96883d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:41.738 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.730+0000 7f870ecda080 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e968826dc0 0x55e96883ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:41.738 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.730+0000 7f870ecda080 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86f80030c0 0x55e96883b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:41.738 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.730+0000 7f870ecda080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e96869edc0 con 0x55e9687a84b0 2023-11-06T02:10:41.738 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.730+0000 7f870ecda080 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9688cddc0 con 0x7f86f80030c0 2023-11-06T02:10:41.738 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.730+0000 7f870ecda080 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e9688cd880 con 0x55e968826dc0 2023-11-06T02:10:41.739 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.730+0000 7f870c7f4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e9687a84b0 0x55e96883d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:41.739 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.730+0000 7f870c7f4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e9687a84b0 0x55e96883d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40274/0 (socket says 172.21.15.49:40274) 2023-11-06T02:10:41.739 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.730+0000 7f870c7f4640 1 -- 172.21.15.49:0/3786833487 learned_addr learned my addr 172.21.15.49:0/3786833487 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:41.739 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.734+0000 7f870bff3640 1 --2- 172.21.15.49:0/3786833487 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e968826dc0 0x55e96883ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:41.739 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.734+0000 7f870cff5640 1 --2- 172.21.15.49:0/3786833487 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86f80030c0 0x55e96883b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:41.740 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.734+0000 7f870c7f4640 1 -- 172.21.15.49:0/3786833487 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e968826dc0 msgr2=0x55e96883ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:41.740 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.734+0000 7f870c7f4640 1 --2- 172.21.15.49:0/3786833487 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e968826dc0 0x55e96883ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:41.740 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.734+0000 7f870c7f4640 1 -- 172.21.15.49:0/3786833487 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86f80030c0 msgr2=0x55e96883b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:41.740 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.734+0000 7f870c7f4640 1 --2- 172.21.15.49:0/3786833487 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86f80030c0 0x55e96883b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:41.740 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.734+0000 7f870c7f4640 1 -- 172.21.15.49:0/3786833487 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86f8001e20 con 0x55e9687a84b0 2023-11-06T02:10:41.741 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.734+0000 7f870cff5640 1 --2- 172.21.15.49:0/3786833487 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86f80030c0 0x55e96883b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:10:41.741 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.734+0000 7f870c7f4640 1 --2- 172.21.15.49:0/3786833487 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e9687a84b0 0x55e96883d5d0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f8704011740 tx=0x7f8704011c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:41.741 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.734+0000 7f8709fef640 1 -- 172.21.15.49:0/3786833487 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8704011e50 con 0x55e9687a84b0 2023-11-06T02:10:41.741 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.734+0000 7f8709fef640 1 -- 172.21.15.49:0/3786833487 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f870400fa40 con 0x55e9687a84b0 2023-11-06T02:10:41.741 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.734+0000 7f870ecda080 1 -- 172.21.15.49:0/3786833487 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55e9686a9e90 con 0x55e9687a84b0 2023-11-06T02:10:41.742 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.734+0000 7f8709fef640 1 -- 172.21.15.49:0/3786833487 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f870400fdb0 con 0x55e9687a84b0 2023-11-06T02:10:41.742 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.734+0000 7f870ecda080 1 -- 172.21.15.49:0/3786833487 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e96880c7d0 con 0x55e9687a84b0 2023-11-06T02:10:41.742 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.734+0000 7f8709fef640 1 -- 172.21.15.49:0/3786833487 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f870401f450 con 0x55e9687a84b0 2023-11-06T02:10:41.742 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.734+0000 7f8709fef640 1 --2- 172.21.15.49:0/3786833487 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f86ec064410 0x7f86ec0668d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:41.742 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.734+0000 7f8709fef640 1 -- 172.21.15.49:0/3786833487 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f87040828c0 con 0x55e9687a84b0 2023-11-06T02:10:41.743 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.734+0000 7f870ecda080 1 --2- 172.21.15.49:0/3786833487 >> [v2:172.21.15.49:6808/38113,v1:172.21.15.49:6810/38113] conn(0x55e96883db10 0x55e9688e1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:41.743 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.734+0000 7f870ecda080 1 -- 172.21.15.49:0/3786833487 --> [v2:172.21.15.49:6808/38113,v1:172.21.15.49:6810/38113] -- osd_op(unknown.0.0:1 8.7 8:e2e07930:::10000000001.00000000:head [setxattr parent (40) in=46b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51) v8 -- 0x55e96880df20 con 0x55e96883db10 2023-11-06T02:10:41.743 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.734+0000 7f870bff3640 1 --2- 172.21.15.49:0/3786833487 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f86ec064410 0x7f86ec0668d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:41.743 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.734+0000 7f870cff5640 1 --2- 172.21.15.49:0/3786833487 >> [v2:172.21.15.49:6808/38113,v1:172.21.15.49:6810/38113] conn(0x55e96883db10 0x55e9688e1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:41.743 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.734+0000 7f870cff5640 1 --2- 172.21.15.49:0/3786833487 >> [v2:172.21.15.49:6808/38113,v1:172.21.15.49:6810/38113] conn(0x55e96883db10 0x55e9688e1ee0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:41.744 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.734+0000 7f870bff3640 1 --2- 172.21.15.49:0/3786833487 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f86ec064410 0x7f86ec0668d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f86f40b9000 tx=0x7f86f40a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:41.744 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.738+0000 7f870cff5640 1 -- 172.21.15.49:0/3786833487 <== osd.1 v2:172.21.15.49:6808/38113 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (40)] v51'2 uv2 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f86fc0039d0 con 0x55e96883db10 2023-11-06T02:10:41.744 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.738+0000 7f870ecda080 1 -- 172.21.15.49:0/3786833487 >> [v2:172.21.15.49:6808/38113,v1:172.21.15.49:6810/38113] conn(0x55e96883db10 msgr2=0x55e9688e1ee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:41.744 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.738+0000 7f870ecda080 1 --2- 172.21.15.49:0/3786833487 >> [v2:172.21.15.49:6808/38113,v1:172.21.15.49:6810/38113] conn(0x55e96883db10 0x55e9688e1ee0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:41.744 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.738+0000 7f870ecda080 1 -- 172.21.15.49:0/3786833487 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f86ec064410 msgr2=0x7f86ec0668d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:41.744 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.738+0000 7f870ecda080 1 --2- 172.21.15.49:0/3786833487 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f86ec064410 0x7f86ec0668d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f86f40b9000 tx=0x7f86f40a6000 comp rx=0 tx=0).stop 2023-11-06T02:10:41.745 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.738+0000 7f870ecda080 1 -- 172.21.15.49:0/3786833487 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e9687a84b0 msgr2=0x55e96883d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:41.745 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.738+0000 7f870ecda080 1 --2- 172.21.15.49:0/3786833487 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e9687a84b0 0x55e96883d5d0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f8704011740 tx=0x7f8704011c00 comp rx=0 tx=0).stop 2023-11-06T02:10:41.745 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.738+0000 7f870cff5640 1 -- 172.21.15.49:0/3786833487 reap_dead start 2023-11-06T02:10:41.745 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.738+0000 7f870ecda080 1 -- 172.21.15.49:0/3786833487 shutdown_connections 2023-11-06T02:10:41.745 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.738+0000 7f870ecda080 1 -- 172.21.15.49:0/3786833487 >> 172.21.15.49:0/3786833487 conn(0x55e96878bef0 msgr2=0x55e968763a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:41.746 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.738+0000 7f870ecda080 1 -- 172.21.15.49:0/3786833487 shutdown_connections 2023-11-06T02:10:41.746 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.738+0000 7f870ecda080 1 -- 172.21.15.49:0/3786833487 wait complete. 2023-11-06T02:10:41.746 DEBUG:teuthology.orchestra.run.smithi049:> sudo daemon-helper kill ceph --cluster ceph -w 2023-11-06T02:10:41.783 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start / repair,recursive 2023-11-06T02:10:41.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.910+0000 7fd537bd9640 1 Processor -- start 2023-11-06T02:10:41.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.910+0000 7fd537bd9640 1 -- start start 2023-11-06T02:10:41.908 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.910+0000 7fd537bd9640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fd530071120 0x7fd530071500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:41.908 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.910+0000 7fd537bd9640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd5300c9c80 0x7fd5300b2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:41.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.910+0000 7fd537bd9640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd5300b5270 0x7fd5300ca640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:41.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.910+0000 7fd537bd9640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5300ae9c0 con 0x7fd5300b5270 2023-11-06T02:10:41.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.910+0000 7fd537bd9640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5300af010 con 0x7fd530071120 2023-11-06T02:10:41.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.910+0000 7fd537bd9640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd530072090 con 0x7fd5300c9c80 2023-11-06T02:10:41.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.910+0000 7fd536176640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd5300b5270 0x7fd5300ca640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:41.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.910+0000 7fd536176640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd5300b5270 0x7fd5300ca640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40278/0 (socket says 172.21.15.49:40278) 2023-11-06T02:10:41.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.910+0000 7fd536176640 1 -- 172.21.15.49:0/4110938841 learned_addr learned my addr 172.21.15.49:0/4110938841 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:41.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.910+0000 7fd535975640 1 --2- 172.21.15.49:0/4110938841 >> v2:172.21.15.150:3300/0 conn(0x7fd530071120 0x7fd530071500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:41.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.910+0000 7fd535174640 1 --2- 172.21.15.49:0/4110938841 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd5300c9c80 0x7fd5300b2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:41.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.910+0000 7fd536176640 1 -- 172.21.15.49:0/4110938841 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd5300c9c80 msgr2=0x7fd5300b2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:41.911 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.910+0000 7fd536176640 1 --2- 172.21.15.49:0/4110938841 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd5300c9c80 0x7fd5300b2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:41.911 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.910+0000 7fd536176640 1 -- 172.21.15.49:0/4110938841 >> v2:172.21.15.150:3300/0 conn(0x7fd530071120 msgr2=0x7fd530071500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:41.911 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.910+0000 7fd536176640 1 --2- 172.21.15.49:0/4110938841 >> v2:172.21.15.150:3300/0 conn(0x7fd530071120 0x7fd530071500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:41.911 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.910+0000 7fd536176640 1 -- 172.21.15.49:0/4110938841 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd530071a40 con 0x7fd5300b5270 2023-11-06T02:10:41.911 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.910+0000 7fd535174640 1 --2- 172.21.15.49:0/4110938841 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd5300c9c80 0x7fd5300b2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:10:41.912 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.910+0000 7fd535975640 1 --2- 172.21.15.49:0/4110938841 >> v2:172.21.15.150:3300/0 conn(0x7fd530071120 0x7fd530071500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:10:41.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.914+0000 7fd536176640 1 --2- 172.21.15.49:0/4110938841 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd5300b5270 0x7fd5300ca640 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fd530068490 tx=0x7fd52c01f1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49656faf9f7a691d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:41.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.914+0000 7fd534973640 1 -- 172.21.15.49:0/4110938841 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd52c0af400 con 0x7fd5300b5270 2023-11-06T02:10:41.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.914+0000 7fd534973640 1 -- 172.21.15.49:0/4110938841 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd52c0ab050 con 0x7fd5300b5270 2023-11-06T02:10:41.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.914+0000 7fd534973640 1 -- 172.21.15.49:0/4110938841 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd52c0af780 con 0x7fd5300b5270 2023-11-06T02:10:41.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.914+0000 7fd537bd9640 1 -- 172.21.15.49:0/4110938841 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd5300b5270 msgr2=0x7fd5300ca640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:41.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.914+0000 7fd537bd9640 1 --2- 172.21.15.49:0/4110938841 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd5300b5270 0x7fd5300ca640 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fd530068490 tx=0x7fd52c01f1c0 comp rx=0 tx=0).stop 2023-11-06T02:10:41.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.914+0000 7fd537bd9640 1 -- 172.21.15.49:0/4110938841 shutdown_connections 2023-11-06T02:10:41.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.914+0000 7fd537bd9640 1 --2- 172.21.15.49:0/4110938841 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd5300c9c80 0x7fd5300b2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:41.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.914+0000 7fd537bd9640 1 --2- 172.21.15.49:0/4110938841 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd5300b5270 0x7fd5300ca640 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:41.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.914+0000 7fd537bd9640 1 --2- 172.21.15.49:0/4110938841 >> v2:172.21.15.150:3300/0 conn(0x7fd530071120 0x7fd530071500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:41.917 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.914+0000 7fd537bd9640 1 -- 172.21.15.49:0/4110938841 >> 172.21.15.49:0/4110938841 conn(0x7fd5300cc140 msgr2=0x7fd5300cc550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:41.917 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.914+0000 7fd537bd9640 1 -- 172.21.15.49:0/4110938841 shutdown_connections 2023-11-06T02:10:41.917 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.914+0000 7fd537bd9640 1 -- 172.21.15.49:0/4110938841 wait complete. 2023-11-06T02:10:41.917 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.914+0000 7fd537bd9640 1 Processor -- start 2023-11-06T02:10:41.917 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.914+0000 7fd537bd9640 1 -- start start 2023-11-06T02:10:41.918 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.918+0000 7fd537bd9640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd530071120 0x7fd5300c4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:41.918 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.918+0000 7fd537bd9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd5300b5270 0x7fd5301e4870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:41.918 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.918+0000 7fd537bd9640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5300c9c80 0x7fd53014d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:41.918 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.918+0000 7fd537bd9640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5300cd2c0 con 0x7fd5300b5270 2023-11-06T02:10:41.919 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.918+0000 7fd537bd9640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5301f5fa0 con 0x7fd5300c9c80 2023-11-06T02:10:41.919 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.918+0000 7fd537bd9640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5301f6280 con 0x7fd530071120 2023-11-06T02:10:41.919 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.918+0000 7fd535174640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd5300b5270 0x7fd5301e4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:41.919 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.918+0000 7fd535174640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd5300b5270 0x7fd5301e4870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40284/0 (socket says 172.21.15.49:40284) 2023-11-06T02:10:41.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.918+0000 7fd535174640 1 -- 172.21.15.49:0/929943944 learned_addr learned my addr 172.21.15.49:0/929943944 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:41.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.918+0000 7fd535975640 1 --2- 172.21.15.49:0/929943944 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd530071120 0x7fd5300c4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:41.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.918+0000 7fd536176640 1 --2- 172.21.15.49:0/929943944 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5300c9c80 0x7fd53014d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:41.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.918+0000 7fd535174640 1 -- 172.21.15.49:0/929943944 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd530071120 msgr2=0x7fd5300c4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:41.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.918+0000 7fd535174640 1 --2- 172.21.15.49:0/929943944 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd530071120 0x7fd5300c4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:41.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.918+0000 7fd535174640 1 -- 172.21.15.49:0/929943944 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5300c9c80 msgr2=0x7fd53014d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:41.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.918+0000 7fd535174640 1 --2- 172.21.15.49:0/929943944 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5300c9c80 0x7fd53014d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:41.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.918+0000 7fd535174640 1 -- 172.21.15.49:0/929943944 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd52c009b30 con 0x7fd5300b5270 2023-11-06T02:10:41.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.918+0000 7fd535174640 1 --2- 172.21.15.49:0/929943944 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd5300b5270 0x7fd5301e4870 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fd52000c7f0 tx=0x7fd52000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:41.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.918+0000 7fd5267fc640 1 -- 172.21.15.49:0/929943944 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd520007820 con 0x7fd5300b5270 2023-11-06T02:10:41.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.918+0000 7fd5267fc640 1 -- 172.21.15.49:0/929943944 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd520007980 con 0x7fd5300b5270 2023-11-06T02:10:41.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.918+0000 7fd537bd9640 1 -- 172.21.15.49:0/929943944 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5301f6560 con 0x7fd5300b5270 2023-11-06T02:10:41.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.918+0000 7fd5267fc640 1 -- 172.21.15.49:0/929943944 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd520007d30 con 0x7fd5300b5270 2023-11-06T02:10:41.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.918+0000 7fd537bd9640 1 -- 172.21.15.49:0/929943944 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5300c5b00 con 0x7fd5300b5270 2023-11-06T02:10:41.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.918+0000 7fd5267fc640 1 -- 172.21.15.49:0/929943944 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fd5200113f0 con 0x7fd5300b5270 2023-11-06T02:10:41.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.918+0000 7fd5267fc640 1 --2- 172.21.15.49:0/929943944 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd514065a60 0x7fd514067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:41.930 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.918+0000 7fd5267fc640 1 -- 172.21.15.49:0/929943944 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd520083580 con 0x7fd5300b5270 2023-11-06T02:10:41.930 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.918+0000 7fd535975640 1 --2- 172.21.15.49:0/929943944 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd514065a60 0x7fd514067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:41.930 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.922+0000 7fd53ad01000 1 Processor -- start 2023-11-06T02:10:41.930 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.922+0000 7fd53ad01000 1 -- start start 2023-11-06T02:10:41.930 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.922+0000 7fd535975640 1 --2- 172.21.15.49:0/929943944 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd514065a60 0x7fd514067f20 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fd528006b20 tx=0x7fd528008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:41.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.922+0000 7fd53ad01000 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564fa7f576a0 0x564fa7f57a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:41.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.922+0000 7fd53ad01000 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564fa7f57fc0 0x564fa7f3bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:41.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.922+0000 7fd53ad01000 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564fa7f3c400 0x564fa7f3c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:41.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.922+0000 7fd53ad01000 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fa7dd5a90 con 0x564fa7f3c400 2023-11-06T02:10:41.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.922+0000 7fd53ad01000 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fa7d63490 con 0x564fa7f57fc0 2023-11-06T02:10:41.932 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.922+0000 7fd53ad01000 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x564fa7d60a60 con 0x564fa7f576a0 2023-11-06T02:10:41.932 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.922+0000 7fd536176640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564fa7f3c400 0x564fa7f3c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:41.932 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.922+0000 7fd536176640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564fa7f3c400 0x564fa7f3c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40290/0 (socket says 172.21.15.49:40290) 2023-11-06T02:10:41.932 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.922+0000 7fd536176640 1 -- 172.21.15.49:0/4201446710 learned_addr learned my addr 172.21.15.49:0/4201446710 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:41.932 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.922+0000 7fd535975640 1 --2- 172.21.15.49:0/4201446710 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564fa7f576a0 0x564fa7f57a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:41.932 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.922+0000 7fd535174640 1 --2- 172.21.15.49:0/4201446710 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564fa7f57fc0 0x564fa7f3bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:41.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.922+0000 7fd536176640 1 -- 172.21.15.49:0/4201446710 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564fa7f576a0 msgr2=0x564fa7f57a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:41.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.922+0000 7fd536176640 1 --2- 172.21.15.49:0/4201446710 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564fa7f576a0 0x564fa7f57a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:41.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.922+0000 7fd536176640 1 -- 172.21.15.49:0/4201446710 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564fa7f57fc0 msgr2=0x564fa7f3bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:41.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.922+0000 7fd536176640 1 --2- 172.21.15.49:0/4201446710 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564fa7f57fc0 0x564fa7f3bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:41.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.922+0000 7fd536176640 1 -- 172.21.15.49:0/4201446710 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564fa7a59070 con 0x564fa7f3c400 2023-11-06T02:10:41.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.922+0000 7fd535174640 1 --2- 172.21.15.49:0/4201446710 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564fa7f57fc0 0x564fa7f3bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:10:41.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.922+0000 7fd536176640 1 --2- 172.21.15.49:0/4201446710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564fa7f3c400 0x564fa7f3c7e0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fd52c01fa60 tx=0x7fd52c004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=202d0bdc5a17a5e5 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:41.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.922+0000 7fd5137fe640 1 -- 172.21.15.49:0/4201446710 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd52c0b9670 con 0x564fa7f3c400 2023-11-06T02:10:41.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.922+0000 7fd5137fe640 1 -- 172.21.15.49:0/4201446710 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd52c0b97d0 con 0x564fa7f3c400 2023-11-06T02:10:41.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.922+0000 7fd5137fe640 1 -- 172.21.15.49:0/4201446710 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd52c0b9b80 con 0x564fa7f3c400 2023-11-06T02:10:41.935 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.922+0000 7fd53ad01000 1 -- 172.21.15.49:0/4201446710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564fa7f3c400 msgr2=0x564fa7f3c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:41.935 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.922+0000 7fd53ad01000 1 --2- 172.21.15.49:0/4201446710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564fa7f3c400 0x564fa7f3c7e0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fd52c01fa60 tx=0x7fd52c004ac0 comp rx=0 tx=0).stop 2023-11-06T02:10:41.935 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.922+0000 7fd53ad01000 1 -- 172.21.15.49:0/4201446710 shutdown_connections 2023-11-06T02:10:41.935 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.922+0000 7fd53ad01000 1 --2- 172.21.15.49:0/4201446710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564fa7f3c400 0x564fa7f3c7e0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:41.935 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.922+0000 7fd53ad01000 1 --2- 172.21.15.49:0/4201446710 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564fa7f57fc0 0x564fa7f3bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:41.936 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.922+0000 7fd53ad01000 1 --2- 172.21.15.49:0/4201446710 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564fa7f576a0 0x564fa7f57a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:41.936 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.922+0000 7fd53ad01000 1 -- 172.21.15.49:0/4201446710 >> 172.21.15.49:0/4201446710 conn(0x564fa7f04bc0 msgr2=0x564fa7f13bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:41.936 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.922+0000 7fd53ad01000 1 -- 172.21.15.49:0/4201446710 shutdown_connections 2023-11-06T02:10:41.936 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.922+0000 7fd53ad01000 1 -- 172.21.15.49:0/4201446710 wait complete. 2023-11-06T02:10:41.936 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.922+0000 7fd53ad01000 1 -- start start 2023-11-06T02:10:41.936 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.922+0000 7fd53ad01000 1 Processor -- start 2023-11-06T02:10:41.937 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.926+0000 7fd53ad01000 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564fa7f3c400 0x564fa7f3bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:41.937 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.926+0000 7fd53ad01000 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564fa7f576a0 0x564fa7f11030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:41.937 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.926+0000 7fd53ad01000 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564fa7f57fc0 0x564fa7f11570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:41.937 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.926+0000 7fd53ad01000 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fa7d5c350 con 0x564fa7f3c400 2023-11-06T02:10:41.937 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.926+0000 7fd53ad01000 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fa7cfb610 con 0x564fa7f576a0 2023-11-06T02:10:41.938 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.926+0000 7fd53ad01000 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x564fa7d19800 con 0x564fa7f57fc0 2023-11-06T02:10:41.938 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.926+0000 7fd53ad01000 10 client.0 _ll_register_callbacks cb 0x564fa7d51960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T02:10:41.938 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.926+0000 7fd535975640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564fa7f3c400 0x564fa7f3bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:41.938 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.926+0000 7fd535975640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564fa7f3c400 0x564fa7f3bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40294/0 (socket says 172.21.15.49:40294) 2023-11-06T02:10:41.938 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.926+0000 7fd535975640 1 -- 172.21.15.49:0/1515210939 learned_addr learned my addr 172.21.15.49:0/1515210939 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:41.939 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.926+0000 7fd535174640 1 --2- 172.21.15.49:0/1515210939 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564fa7f576a0 0x564fa7f11030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:41.939 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.926+0000 7fd536176640 1 --2- 172.21.15.49:0/1515210939 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564fa7f57fc0 0x564fa7f11570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:41.939 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.926+0000 7fd535975640 1 -- 172.21.15.49:0/1515210939 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564fa7f57fc0 msgr2=0x564fa7f11570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:41.939 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.926+0000 7fd535975640 1 --2- 172.21.15.49:0/1515210939 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564fa7f57fc0 0x564fa7f11570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:41.939 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.926+0000 7fd535975640 1 -- 172.21.15.49:0/1515210939 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564fa7f576a0 msgr2=0x564fa7f11030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:41.940 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.926+0000 7fd535975640 1 --2- 172.21.15.49:0/1515210939 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564fa7f576a0 0x564fa7f11030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:41.940 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.926+0000 7fd535975640 1 -- 172.21.15.49:0/1515210939 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd530071a40 con 0x564fa7f3c400 2023-11-06T02:10:41.940 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.926+0000 7fd536176640 1 --2- 172.21.15.49:0/1515210939 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564fa7f57fc0 0x564fa7f11570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:10:41.940 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.926+0000 7fd535975640 1 --2- 172.21.15.49:0/1515210939 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564fa7f3c400 0x564fa7f3bcb0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fd528013700 tx=0x7fd528004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=172c7cb276ae4d1 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:41.940 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.926+0000 7fd510ff9640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-11-06T02:10:41.941 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.926+0000 7fd510ff9640 1 -- 172.21.15.49:0/1515210939 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd528014850 con 0x564fa7f3c400 2023-11-06T02:10:41.941 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.926+0000 7fd510ff9640 1 -- 172.21.15.49:0/1515210939 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd5280149b0 con 0x564fa7f3c400 2023-11-06T02:10:41.941 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.926+0000 7fd53ad01000 1 -- 172.21.15.49:0/1515210939 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564fa7d2bc80 con 0x564fa7f3c400 2023-11-06T02:10:41.941 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.926+0000 7fd510ff9640 1 -- 172.21.15.49:0/1515210939 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd528014b70 con 0x564fa7f3c400 2023-11-06T02:10:41.941 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.926+0000 7fd53ad01000 10 client.6075 fetch_fsmap learned FSMap version 23 2023-11-06T02:10:41.941 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.926+0000 7fd53ad01000 1 -- 172.21.15.49:0/1515210939 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x564fa7d212d0 con 0x564fa7f3c400 2023-11-06T02:10:41.942 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.926+0000 7fd510ff9640 1 -- 172.21.15.49:0/1515210939 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564fa7d2bc80 con 0x564fa7f3c400 2023-11-06T02:10:41.942 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.926+0000 7fd510ff9640 1 -- 172.21.15.49:0/1515210939 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 23) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fd528011070 con 0x564fa7f3c400 2023-11-06T02:10:41.942 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.926+0000 7fd53ad01000 10 client.6075 fetch_fsmap finished waiting for FSMap version 23 2023-11-06T02:10:41.942 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.926+0000 7fd53ad01000 10 client.6075 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2023-11-06T02:10:41.942 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.926+0000 7fd53ad01000 20 client.6075 populate_metadata read hostname 'smithi049' 2023-11-06T02:10:41.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.926+0000 7fd53ad01000 1 --2- 172.21.15.49:0/1515210939 >> [v2:172.21.15.49:6834/73939548,v1:172.21.15.49:6835/73939548] conn(0x564fa7f11f80 0x564fa7f12360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:41.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.926+0000 7fd53ad01000 4 client.6075 mds_command: new command op to 5877 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T02:10:41.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.926+0000 7fd53ad01000 1 -- 172.21.15.49:0/1515210939 --> [v2:172.21.15.49:6834/73939548,v1:172.21.15.49:6835/73939548] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564fa7d5aca0 con 0x564fa7f11f80 2023-11-06T02:10:41.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.926+0000 7fd535174640 1 --2- 172.21.15.49:0/1515210939 >> [v2:172.21.15.49:6834/73939548,v1:172.21.15.49:6835/73939548] conn(0x564fa7f11f80 0x564fa7f12360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:41.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.926+0000 7fd510ff9640 10 client.6075 ms_handle_connect on v2:172.21.15.49:6834/73939548 2023-11-06T02:10:41.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.926+0000 7fd535174640 1 --2- 172.21.15.49:0/1515210939 >> [v2:172.21.15.49:6834/73939548,v1:172.21.15.49:6835/73939548] conn(0x564fa7f11f80 0x564fa7f12360 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ac84b8c7c90824d server_cookie=312f49fd6a88d187 in_seq=0 out_seq=0 2023-11-06T02:10:41.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.926+0000 7f58c9ae0640 1 Processor -- start 2023-11-06T02:10:41.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.926+0000 7fd510ff9640 1 -- 172.21.15.49:0/1515210939 <== mds.0 v2:172.21.15.49:6834/73939548 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x564fa7d5aca0 con 0x564fa7f11f80 2023-11-06T02:10:41.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.926+0000 7fd510ff9640 10 client.6075 handle_command_reply: tid=0 2023-11-06T02:10:41.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.926+0000 7f58c9ae0640 1 -- start start 2023-11-06T02:10:41.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.926+0000 7fd53ad01000 1 client.6075 shutdown 2023-11-06T02:10:41.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.926+0000 7fd53ad01000 1 -- 172.21.15.49:0/1515210939 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564fa7f3c400 msgr2=0x564fa7f3bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:41.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.926+0000 7fd53ad01000 1 --2- 172.21.15.49:0/1515210939 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564fa7f3c400 0x564fa7f3bcb0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fd528013700 tx=0x7fd528004740 comp rx=0 tx=0).stop 2023-11-06T02:10:41.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.926+0000 7fd53ad01000 1 -- 172.21.15.49:0/1515210939 shutdown_connections 2023-11-06T02:10:41.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.926+0000 7fd53ad01000 1 --2- 172.21.15.49:0/1515210939 >> [v2:172.21.15.49:6834/73939548,v1:172.21.15.49:6835/73939548] conn(0x564fa7f11f80 0x564fa7f12360 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:41.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.926+0000 7fd53ad01000 1 --2- 172.21.15.49:0/1515210939 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564fa7f57fc0 0x564fa7f11570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:41.948 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.926+0000 7fd53ad01000 1 --2- 172.21.15.49:0/1515210939 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564fa7f576a0 0x564fa7f11030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:41.948 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.926+0000 7fd510ff9640 0 client.6075 ms_handle_reset on v2:172.21.15.49:6834/73939548 2023-11-06T02:10:41.948 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.926+0000 7fd53ad01000 1 --2- 172.21.15.49:0/1515210939 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564fa7f3c400 0x564fa7f3bcb0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:41.948 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.926+0000 7fd53ad01000 1 -- 172.21.15.49:0/1515210939 >> 172.21.15.49:0/1515210939 conn(0x564fa7f04bc0 msgr2=0x564fa7f3d280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:41.948 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.926+0000 7fd53ad01000 1 -- 172.21.15.49:0/1515210939 shutdown_connections 2023-11-06T02:10:41.949 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.926+0000 7fd53ad01000 1 -- 172.21.15.49:0/1515210939 wait complete. 2023-11-06T02:10:41.949 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.926+0000 7fd53ad01000 20 client.6075 trim_cache size 0 max 16384 2023-11-06T02:10:41.953 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.926+0000 7f58c9ae0640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f58c415cdf0 0x7f58c40b5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:41.953 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.926+0000 7f58c9ae0640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f58c41608a0 0x7f58c40cd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:41.954 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.926+0000 7f58c9ae0640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58c40cd810 con 0x7f58c40cf120 2023-11-06T02:10:41.954 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.926+0000 7f58c9ae0640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58c40b6310 con 0x7f58c415cdf0 2023-11-06T02:10:41.954 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.926+0000 7f58c9ae0640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58c415d1d0 con 0x7f58c41608a0 2023-11-06T02:10:41.954 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.926+0000 7f58c37fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f58c415cdf0 0x7f58c40b5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:41.954 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.930+0000 7f58c37fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f58c415cdf0 0x7f58c40b5dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40310/0 (socket says 172.21.15.49:40310) 2023-11-06T02:10:41.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.930+0000 7f58c37fe640 1 -- 172.21.15.49:0/649491016 learned_addr learned my addr 172.21.15.49:0/649491016 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:41.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.930+0000 7f58c2ffd640 1 --2- 172.21.15.49:0/649491016 >> v2:172.21.15.150:3300/0 conn(0x7f58c41608a0 0x7f58c40cd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:41.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.930+0000 7f58c37fe640 1 -- 172.21.15.49:0/649491016 >> v2:172.21.15.150:3300/0 conn(0x7f58c41608a0 msgr2=0x7f58c40cd2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:41.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.930+0000 7f58c37fe640 1 --2- 172.21.15.49:0/649491016 >> v2:172.21.15.150:3300/0 conn(0x7f58c41608a0 0x7f58c40cd2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:41.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.930+0000 7f58c37fe640 1 -- 172.21.15.49:0/649491016 >> v1:172.21.15.150:6789/0 conn(0x7f58c40cf120 legacy=0x7f58c40cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:41.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.930+0000 7f58c37fe640 1 -- 172.21.15.49:0/649491016 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58c40cda40 con 0x7f58c415cdf0 2023-11-06T02:10:41.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.930+0000 7f58c37fe640 1 --2- 172.21.15.49:0/649491016 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f58c415cdf0 0x7f58c40b5dd0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f58c40b54e0 tx=0x7f58b401c370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=418efaf51dc133aa server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:41.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.930+0000 7f58c27fc640 1 -- 172.21.15.49:0/649491016 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b40ab600 con 0x7f58c415cdf0 2023-11-06T02:10:41.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.930+0000 7f58c27fc640 1 -- 172.21.15.49:0/649491016 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58b40a7050 con 0x7f58c415cdf0 2023-11-06T02:10:41.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.930+0000 7f58c27fc640 1 -- 172.21.15.49:0/649491016 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b40ab980 con 0x7f58c415cdf0 2023-11-06T02:10:41.957 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.930+0000 7f58c9ae0640 1 -- 172.21.15.49:0/649491016 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f58c415cdf0 msgr2=0x7f58c40b5dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:41.957 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.930+0000 7f58c9ae0640 1 --2- 172.21.15.49:0/649491016 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f58c415cdf0 0x7f58c40b5dd0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f58c40b54e0 tx=0x7f58b401c370 comp rx=0 tx=0).stop 2023-11-06T02:10:41.957 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.930+0000 7f58c9ae0640 1 -- 172.21.15.49:0/649491016 shutdown_connections 2023-11-06T02:10:41.957 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.930+0000 7f58c9ae0640 1 --2- 172.21.15.49:0/649491016 >> v2:172.21.15.150:3300/0 conn(0x7f58c41608a0 0x7f58c40cd2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:41.957 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.930+0000 7f58c9ae0640 1 --2- 172.21.15.49:0/649491016 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f58c415cdf0 0x7f58c40b5dd0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:41.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.930+0000 7f58c9ae0640 1 -- 172.21.15.49:0/649491016 >> 172.21.15.49:0/649491016 conn(0x7f58c406afb0 msgr2=0x7f58c406b3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:41.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.930+0000 7f58c9ae0640 1 -- 172.21.15.49:0/649491016 shutdown_connections 2023-11-06T02:10:41.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.930+0000 7f58c9ae0640 1 -- 172.21.15.49:0/649491016 wait complete. 2023-11-06T02:10:41.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.934+0000 7f58c9ae0640 1 Processor -- start 2023-11-06T02:10:41.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.934+0000 7f58c9ae0640 1 -- start start 2023-11-06T02:10:41.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.934+0000 7f58c9ae0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f58c40cf120 0x7f58c40c5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:41.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.934+0000 7f58c9ae0640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f58c415cdf0 0x7f58c4156980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:41.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.934+0000 7f58c9ae0640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58c41608a0 0x7f58c41448e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:41.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.934+0000 7f58c9ae0640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c4157d30 con 0x7f58c40cf120 2023-11-06T02:10:41.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.934+0000 7f58c9ae0640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c41571b0 con 0x7f58c41608a0 2023-11-06T02:10:41.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.934+0000 7f58c9ae0640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58c4157490 con 0x7f58c415cdf0 2023-11-06T02:10:41.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.934+0000 7f58c37fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f58c40cf120 0x7f58c40c5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:41.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.934+0000 7f58c37fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f58c40cf120 0x7f58c40c5e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40316/0 (socket says 172.21.15.49:40316) 2023-11-06T02:10:41.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.934+0000 7f58c37fe640 1 -- 172.21.15.49:0/1924679726 learned_addr learned my addr 172.21.15.49:0/1924679726 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:41.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.934+0000 7f58c2ffd640 1 --2- 172.21.15.49:0/1924679726 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f58c415cdf0 0x7f58c4156980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:41.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.934+0000 7f58c3fff640 1 --2- 172.21.15.49:0/1924679726 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58c41608a0 0x7f58c41448e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:41.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.934+0000 7f58c37fe640 1 -- 172.21.15.49:0/1924679726 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f58c415cdf0 msgr2=0x7f58c4156980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:41.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.934+0000 7f58c37fe640 1 --2- 172.21.15.49:0/1924679726 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f58c415cdf0 0x7f58c4156980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:41.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.934+0000 7f58c37fe640 1 -- 172.21.15.49:0/1924679726 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58c41608a0 msgr2=0x7f58c41448e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:41.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.934+0000 7f58c37fe640 1 --2- 172.21.15.49:0/1924679726 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58c41608a0 0x7f58c41448e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:41.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.934+0000 7f58c37fe640 1 -- 172.21.15.49:0/1924679726 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58b40094d0 con 0x7f58c40cf120 2023-11-06T02:10:41.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.934+0000 7f58c37fe640 1 --2- 172.21.15.49:0/1924679726 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f58c40cf120 0x7f58c40c5e00 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f58b40ae040 tx=0x7f58b401abb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:41.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.934+0000 7f58abfff640 1 -- 172.21.15.49:0/1924679726 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b40b6620 con 0x7f58c40cf120 2023-11-06T02:10:41.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.934+0000 7f58abfff640 1 -- 172.21.15.49:0/1924679726 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58b40abe90 con 0x7f58c40cf120 2023-11-06T02:10:41.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.934+0000 7f58abfff640 1 -- 172.21.15.49:0/1924679726 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b40b69e0 con 0x7f58c40cf120 2023-11-06T02:10:41.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.938+0000 7f58c9ae0640 1 -- 172.21.15.49:0/1924679726 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58c4157770 con 0x7f58c40cf120 2023-11-06T02:10:41.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.938+0000 7f58c9ae0640 1 -- 172.21.15.49:0/1924679726 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58c4156ec0 con 0x7f58c40cf120 2023-11-06T02:10:41.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.938+0000 7f58c9ae0640 1 -- 172.21.15.49:0/1924679726 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f58c404e8a0 con 0x7f58c40cf120 2023-11-06T02:10:41.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.938+0000 7f58abfff640 1 -- 172.21.15.49:0/1924679726 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f58b40b2040 con 0x7f58c40cf120 2023-11-06T02:10:41.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.938+0000 7f58abfff640 1 --2- 172.21.15.49:0/1924679726 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f58b0065a90 0x7f58b0067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:41.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.938+0000 7f58abfff640 1 -- 172.21.15.49:0/1924679726 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f58b411dd60 con 0x7f58c40cf120 2023-11-06T02:10:41.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.938+0000 7f58abfff640 1 -- 172.21.15.49:0/1924679726 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+503 (secure 0 0 0) 0x7f58b411e1d0 con 0x7f58c40cf120 2023-11-06T02:10:41.964 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.938+0000 7f58c9ae0640 1 -- 172.21.15.49:0/1924679726 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({log-info=0+}) v3 -- 0x7f58c40c74b0 con 0x7f58c40cf120 2023-11-06T02:10:41.964 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.938+0000 7f58c2ffd640 1 --2- 172.21.15.49:0/1924679726 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f58b0065a90 0x7f58b0067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:41.964 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.938+0000 7f58c2ffd640 1 --2- 172.21.15.49:0/1924679726 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f58b0065a90 0x7f58b0067f50 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f58b8006fd0 tx=0x7f58b8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:41.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.946+0000 7fd53ad01000 1 Processor -- start 2023-11-06T02:10:41.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.946+0000 7fd53ad01000 1 -- start start 2023-11-06T02:10:41.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.946+0000 7fd53ad01000 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564fa7f3c400 0x564fa7f67740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:41.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.946+0000 7fd53ad01000 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564fa7f576a0 0x564fa7f67c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:41.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.946+0000 7fd53ad01000 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564fa7f57fc0 0x564fa7f681c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:41.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.946+0000 7fd53ad01000 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fa7a19cf0 con 0x564fa7f576a0 2023-11-06T02:10:41.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.946+0000 7fd53ad01000 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fa7d57cc0 con 0x564fa7f3c400 2023-11-06T02:10:41.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.946+0000 7fd53ad01000 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x564fa7d566f0 con 0x564fa7f57fc0 2023-11-06T02:10:41.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.946+0000 7fd535174640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564fa7f576a0 0x564fa7f67c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:41.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.946+0000 7fd535174640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564fa7f576a0 0x564fa7f67c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40320/0 (socket says 172.21.15.49:40320) 2023-11-06T02:10:41.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.946+0000 7fd535174640 1 -- 172.21.15.49:0/4226488991 learned_addr learned my addr 172.21.15.49:0/4226488991 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:41.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.946+0000 7fd535174640 1 -- 172.21.15.49:0/4226488991 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564fa7f57fc0 msgr2=0x564fa7f681c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:41.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.946+0000 7fd535174640 1 --2- 172.21.15.49:0/4226488991 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564fa7f57fc0 0x564fa7f681c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:41.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.946+0000 7fd535174640 1 -- 172.21.15.49:0/4226488991 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564fa7f3c400 msgr2=0x564fa7f67740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:41.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.946+0000 7fd535174640 1 --2- 172.21.15.49:0/4226488991 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564fa7f3c400 0x564fa7f67740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:41.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.946+0000 7fd535174640 1 -- 172.21.15.49:0/4226488991 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564fa7f0e980 con 0x564fa7f576a0 2023-11-06T02:10:41.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.946+0000 7fd535174640 1 --2- 172.21.15.49:0/4226488991 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564fa7f576a0 0x564fa7f67c80 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fd520054830 tx=0x7fd5200818b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11a84593363ea941 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:41.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.946+0000 7fd513fff640 1 -- 172.21.15.49:0/4226488991 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd520052c70 con 0x564fa7f576a0 2023-11-06T02:10:41.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.946+0000 7fd513fff640 1 -- 172.21.15.49:0/4226488991 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd520052ed0 con 0x564fa7f576a0 2023-11-06T02:10:41.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.946+0000 7fd513fff640 1 -- 172.21.15.49:0/4226488991 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd52005c680 con 0x564fa7f576a0 2023-11-06T02:10:41.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.946+0000 7fd53ad01000 1 -- 172.21.15.49:0/4226488991 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564fa7f576a0 msgr2=0x564fa7f67c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:41.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.946+0000 7fd53ad01000 1 --2- 172.21.15.49:0/4226488991 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564fa7f576a0 0x564fa7f67c80 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fd520054830 tx=0x7fd5200818b0 comp rx=0 tx=0).stop 2023-11-06T02:10:41.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.946+0000 7fd53ad01000 1 -- 172.21.15.49:0/4226488991 shutdown_connections 2023-11-06T02:10:41.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.946+0000 7fd53ad01000 1 --2- 172.21.15.49:0/4226488991 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564fa7f576a0 0x564fa7f67c80 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:41.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.946+0000 7fd53ad01000 1 --2- 172.21.15.49:0/4226488991 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564fa7f57fc0 0x564fa7f681c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:41.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.946+0000 7fd53ad01000 1 --2- 172.21.15.49:0/4226488991 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564fa7f3c400 0x564fa7f67740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:41.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.946+0000 7fd53ad01000 1 -- 172.21.15.49:0/4226488991 >> 172.21.15.49:0/4226488991 conn(0x564fa7f04bc0 msgr2=0x564fa7f10b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:41.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.946+0000 7fd53ad01000 1 -- 172.21.15.49:0/4226488991 shutdown_connections 2023-11-06T02:10:41.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.946+0000 7fd53ad01000 1 -- 172.21.15.49:0/4226488991 wait complete. 2023-11-06T02:10:41.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.946+0000 7fd53ad01000 1 -- start start 2023-11-06T02:10:41.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.946+0000 7fd53ad01000 1 Processor -- start 2023-11-06T02:10:41.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.950+0000 7fd53ad01000 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564fa7f3c400 0x564fa7fb4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:41.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.950+0000 7fd53ad01000 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564fa7f576a0 0x564fa7fb5100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:41.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.950+0000 7fd53ad01000 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564fa7f57fc0 0x564fa7f6aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:41.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.950+0000 7fd53ad01000 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fa7f0e840 con 0x564fa7f57fc0 2023-11-06T02:10:41.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.950+0000 7fd53ad01000 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fa7d2b8e0 con 0x564fa7f3c400 2023-11-06T02:10:41.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.950+0000 7fd53ad01000 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x564fa7d9fc30 con 0x564fa7f576a0 2023-11-06T02:10:41.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.950+0000 7fd53ad01000 10 client.0 _ll_register_callbacks cb 0x564fa7d620a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T02:10:41.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.950+0000 7fd536176640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564fa7f57fc0 0x564fa7f6aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:41.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.950+0000 7fd536176640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564fa7f57fc0 0x564fa7f6aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40324/0 (socket says 172.21.15.49:40324) 2023-11-06T02:10:41.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.950+0000 7fd536176640 1 -- 172.21.15.49:0/3717405919 learned_addr learned my addr 172.21.15.49:0/3717405919 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:41.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.950+0000 7fd535174640 1 --2- 172.21.15.49:0/3717405919 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564fa7f576a0 0x564fa7fb5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:41.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.950+0000 7fd535975640 1 --2- 172.21.15.49:0/3717405919 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564fa7f3c400 0x564fa7fb4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:41.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.950+0000 7fd536176640 1 -- 172.21.15.49:0/3717405919 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564fa7f576a0 msgr2=0x564fa7fb5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:41.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.950+0000 7fd536176640 1 --2- 172.21.15.49:0/3717405919 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564fa7f576a0 0x564fa7fb5100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:41.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.950+0000 7fd536176640 1 -- 172.21.15.49:0/3717405919 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564fa7f3c400 msgr2=0x564fa7fb4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:41.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.950+0000 7fd536176640 1 --2- 172.21.15.49:0/3717405919 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564fa7f3c400 0x564fa7fb4bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:41.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.950+0000 7fd536176640 1 -- 172.21.15.49:0/3717405919 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5300c5b00 con 0x564fa7f57fc0 2023-11-06T02:10:41.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.950+0000 7fd536176640 1 --2- 172.21.15.49:0/3717405919 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564fa7f57fc0 0x564fa7f6aec0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fd52c009f10 tx=0x7fd52c01da00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72a56155449ea10f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:41.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.950+0000 7fd510ff9640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-11-06T02:10:41.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.950+0000 7fd510ff9640 1 -- 172.21.15.49:0/3717405919 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd52c007000 con 0x564fa7f57fc0 2023-11-06T02:10:41.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.950+0000 7fd510ff9640 1 -- 172.21.15.49:0/3717405919 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd52c0b9c60 con 0x564fa7f57fc0 2023-11-06T02:10:41.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.950+0000 7fd53ad01000 1 -- 172.21.15.49:0/3717405919 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564fa7d0bd50 con 0x564fa7f57fc0 2023-11-06T02:10:41.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.950+0000 7fd510ff9640 1 -- 172.21.15.49:0/3717405919 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd52c0a8a70 con 0x564fa7f57fc0 2023-11-06T02:10:41.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.950+0000 7fd53ad01000 10 client.6087 fetch_fsmap learned FSMap version 23 2023-11-06T02:10:41.980 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.950+0000 7fd53ad01000 1 -- 172.21.15.49:0/3717405919 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x564fa7d07620 con 0x564fa7f57fc0 2023-11-06T02:10:41.980 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.950+0000 7fd510ff9640 1 -- 172.21.15.49:0/3717405919 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564fa7d0bd50 con 0x564fa7f57fc0 2023-11-06T02:10:41.980 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.950+0000 7fd510ff9640 1 -- 172.21.15.49:0/3717405919 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 23) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fd52c0c3040 con 0x564fa7f57fc0 2023-11-06T02:10:41.980 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.950+0000 7fd53ad01000 10 client.6087 fetch_fsmap finished waiting for FSMap version 23 2023-11-06T02:10:41.980 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.950+0000 7fd53ad01000 10 client.6087 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2023-11-06T02:10:41.980 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.950+0000 7fd53ad01000 20 client.6087 populate_metadata read hostname 'smithi049' 2023-11-06T02:10:41.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.950+0000 7fd53ad01000 1 --2- 172.21.15.49:0/3717405919 >> [v2:172.21.15.49:6834/73939548,v1:172.21.15.49:6835/73939548] conn(0x564fa7f6b8b0 0x564fa7f6bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:41.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.950+0000 7fd53ad01000 4 client.6087 mds_command: new command op to 5877 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2023-11-06T02:10:41.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.950+0000 7fd53ad01000 1 -- 172.21.15.49:0/3717405919 --> [v2:172.21.15.49:6834/73939548,v1:172.21.15.49:6835/73939548] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) v1 -- 0x564fa7d553d0 con 0x564fa7f6b8b0 2023-11-06T02:10:41.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.950+0000 7fd535975640 1 --2- 172.21.15.49:0/3717405919 >> [v2:172.21.15.49:6834/73939548,v1:172.21.15.49:6835/73939548] conn(0x564fa7f6b8b0 0x564fa7f6bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:41.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.950+0000 7fd510ff9640 10 client.6087 ms_handle_connect on v2:172.21.15.49:6834/73939548 2023-11-06T02:10:41.982 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.950+0000 7fd535975640 1 --2- 172.21.15.49:0/3717405919 >> [v2:172.21.15.49:6834/73939548,v1:172.21.15.49:6835/73939548] conn(0x564fa7f6b8b0 0x564fa7f6bcc0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc18bf610918c32b server_cookie=70e4423cefe218ad in_seq=0 out_seq=0 2023-11-06T02:10:41.982 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.950+0000 7fd510ff9640 1 -- 172.21.15.49:0/3717405919 <== mds.0 v2:172.21.15.49:6834/73939548 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x564fa7d553d0 con 0x564fa7f6b8b0 2023-11-06T02:10:41.982 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.950+0000 7fd510ff9640 10 client.6087 handle_command_reply: tid=0 2023-11-06T02:10:41.982 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.950+0000 7fd53ad01000 1 client.6087 shutdown 2023-11-06T02:10:41.982 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.950+0000 7fd53ad01000 1 -- 172.21.15.49:0/3717405919 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564fa7f57fc0 msgr2=0x564fa7f6aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:41.982 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.950+0000 7fd53ad01000 1 --2- 172.21.15.49:0/3717405919 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564fa7f57fc0 0x564fa7f6aec0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fd52c009f10 tx=0x7fd52c01da00 comp rx=0 tx=0).stop 2023-11-06T02:10:41.983 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.950+0000 7fd53ad01000 1 -- 172.21.15.49:0/3717405919 shutdown_connections 2023-11-06T02:10:41.983 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.950+0000 7fd53ad01000 1 --2- 172.21.15.49:0/3717405919 >> [v2:172.21.15.49:6834/73939548,v1:172.21.15.49:6835/73939548] conn(0x564fa7f6b8b0 0x564fa7f6bcc0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:41.983 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.950+0000 7fd53ad01000 1 --2- 172.21.15.49:0/3717405919 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564fa7f57fc0 0x564fa7f6aec0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:41.983 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.950+0000 7fd53ad01000 1 --2- 172.21.15.49:0/3717405919 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564fa7f576a0 0x564fa7fb5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:41.983 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.950+0000 7fd510ff9640 0 client.6087 ms_handle_reset on v2:172.21.15.49:6834/73939548 2023-11-06T02:10:41.984 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.950+0000 7fd53ad01000 1 --2- 172.21.15.49:0/3717405919 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564fa7f3c400 0x564fa7fb4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:41.984 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.950+0000 7fd53ad01000 1 -- 172.21.15.49:0/3717405919 >> 172.21.15.49:0/3717405919 conn(0x564fa7f04bc0 msgr2=0x564fa7f11200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:41.984 INFO:tasks.ceph.mds.a.smithi049.stderr:2023-11-06T02:10:41.950+0000 7feb1a6a4640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /parent_a/alpha auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={6036=0-4194304@1} caps={6036=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=6036 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x55b9e9309700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000001:[<0x10000000000/alpha v2>,<0x1/parent_a v4>]//[]","error_str":"failed to decode on-disk backtrace (40 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-06T02:10:41.985 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.950+0000 7fd53ad01000 1 -- 172.21.15.49:0/3717405919 shutdown_connections 2023-11-06T02:10:41.985 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.950+0000 7fd53ad01000 1 -- 172.21.15.49:0/3717405919 wait complete. 2023-11-06T02:10:41.985 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.950+0000 7fd53ad01000 20 client.6087 trim_cache size 0 max 16384 2023-11-06T02:10:41.986 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-11-06T02:10:41.986 INFO:teuthology.orchestra.run.smithi049.stdout: "return_code": 0, 2023-11-06T02:10:41.986 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_tag": "03224875-0c30-4c54-853c-d6d7bcbe1f1a", 2023-11-06T02:10:41.986 INFO:teuthology.orchestra.run.smithi049.stdout: "mode": "asynchronous" 2023-11-06T02:10:41.987 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-11-06T02:10:41.988 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.954+0000 7fd537bd9640 1 -- 172.21.15.49:0/929943944 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd514065a60 msgr2=0x7fd514067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:41.988 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.954+0000 7fd537bd9640 1 --2- 172.21.15.49:0/929943944 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd514065a60 0x7fd514067f20 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fd528006b20 tx=0x7fd528008040 comp rx=0 tx=0).stop 2023-11-06T02:10:41.988 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.954+0000 7fd537bd9640 1 -- 172.21.15.49:0/929943944 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd5300b5270 msgr2=0x7fd5301e4870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:41.989 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.954+0000 7fd537bd9640 1 --2- 172.21.15.49:0/929943944 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd5300b5270 0x7fd5301e4870 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fd52000c7f0 tx=0x7fd52000ccb0 comp rx=0 tx=0).stop 2023-11-06T02:10:41.989 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.954+0000 7fd537bd9640 1 -- 172.21.15.49:0/929943944 shutdown_connections 2023-11-06T02:10:41.989 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.954+0000 7fd537bd9640 1 --2- 172.21.15.49:0/929943944 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd514065a60 0x7fd514067f20 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:41.989 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.954+0000 7fd537bd9640 1 --2- 172.21.15.49:0/929943944 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd5300b5270 0x7fd5301e4870 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:41.989 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.954+0000 7fd537bd9640 1 --2- 172.21.15.49:0/929943944 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5300c9c80 0x7fd53014d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:41.990 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.954+0000 7fd537bd9640 1 --2- 172.21.15.49:0/929943944 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd530071120 0x7fd5300c4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:41.990 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.954+0000 7fd537bd9640 1 -- 172.21.15.49:0/929943944 >> 172.21.15.49:0/929943944 conn(0x7fd5300cc140 msgr2=0x7fd5301593a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:41.990 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.954+0000 7fd537bd9640 1 -- 172.21.15.49:0/929943944 shutdown_connections 2023-11-06T02:10:41.990 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:41.954+0000 7fd537bd9640 1 -- 172.21.15.49:0/929943944 wait complete. 2023-11-06T02:10:41.991 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub status 2023-11-06T02:10:42.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.110+0000 7f8098d5c640 1 Processor -- start 2023-11-06T02:10:42.109 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.110+0000 7f8098d5c640 1 -- start start 2023-11-06T02:10:42.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.114+0000 7f8098d5c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f8094070160 0x7f80940705d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:42.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.114+0000 7f8098d5c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f809406c230 0x7f809406c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:42.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.114+0000 7f8098d5c640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80940c8c80 con 0x7f809414ffc0 2023-11-06T02:10:42.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.114+0000 7f8098d5c640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80940ae4d0 con 0x7f8094070160 2023-11-06T02:10:42.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.114+0000 7f8098d5c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80941594f0 con 0x7f809406c230 2023-11-06T02:10:42.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.114+0000 7f809259c640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f809414ffc0 0x7f80940ada30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:35090/0 (socket says 172.21.15.49:35090) 2023-11-06T02:10:42.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.114+0000 7f8091d9b640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f8094070160 0x7f80940705d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:42.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.114+0000 7f809259c640 1 -- 172.21.15.49:0/3246048205 learned_addr learned my addr 172.21.15.49:0/3246048205 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:10:42.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.114+0000 7f8092d9d640 1 --2- 172.21.15.49:0/3246048205 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f809406c230 0x7f809406c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:42.113 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.114+0000 7f809159a640 1 -- 172.21.15.49:0/3246048205 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2537348473 0 0) 0x7f80940c8c80 con 0x7f809414ffc0 2023-11-06T02:10:42.113 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.114+0000 7f809159a640 1 -- 172.21.15.49:0/3246048205 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f807c003530 con 0x7f809414ffc0 2023-11-06T02:10:42.113 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.114+0000 7f8091d9b640 1 -- 172.21.15.49:0/3246048205 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f809406c230 msgr2=0x7f809406c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:42.113 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.114+0000 7f8091d9b640 1 --2- 172.21.15.49:0/3246048205 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f809406c230 0x7f809406c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:42.113 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.114+0000 7f8091d9b640 1 -- 172.21.15.49:0/3246048205 >> v1:172.21.15.49:6789/0 conn(0x7f809414ffc0 legacy=0x7f80940ada30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:42.114 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.114+0000 7f8091d9b640 1 -- 172.21.15.49:0/3246048205 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8094159700 con 0x7f8094070160 2023-11-06T02:10:42.115 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.114+0000 7f8091d9b640 1 --2- 172.21.15.49:0/3246048205 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8094070160 0x7f80940705d0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f80880097c0 tx=0x7f808801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4cb0b35697108ba server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:42.115 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.114+0000 7f809159a640 1 -- 172.21.15.49:0/3246048205 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80880ab600 con 0x7f8094070160 2023-11-06T02:10:42.116 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.114+0000 7f809159a640 1 -- 172.21.15.49:0/3246048205 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80880a7050 con 0x7f8094070160 2023-11-06T02:10:42.116 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.114+0000 7f809159a640 1 -- 172.21.15.49:0/3246048205 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80880ab980 con 0x7f8094070160 2023-11-06T02:10:42.116 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.118+0000 7f8098d5c640 1 -- 172.21.15.49:0/3246048205 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8094070160 msgr2=0x7f80940705d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:42.116 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.118+0000 7f8098d5c640 1 --2- 172.21.15.49:0/3246048205 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8094070160 0x7f80940705d0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f80880097c0 tx=0x7f808801c480 comp rx=0 tx=0).stop 2023-11-06T02:10:42.116 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.118+0000 7f8098d5c640 1 -- 172.21.15.49:0/3246048205 shutdown_connections 2023-11-06T02:10:42.117 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.118+0000 7f8098d5c640 1 --2- 172.21.15.49:0/3246048205 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f809406c230 0x7f809406c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:42.117 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.118+0000 7f8098d5c640 1 --2- 172.21.15.49:0/3246048205 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8094070160 0x7f80940705d0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:42.117 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.118+0000 7f8098d5c640 1 -- 172.21.15.49:0/3246048205 >> 172.21.15.49:0/3246048205 conn(0x7f809406a370 msgr2=0x7f80940ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:42.117 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.118+0000 7f8098d5c640 1 -- 172.21.15.49:0/3246048205 shutdown_connections 2023-11-06T02:10:42.117 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.118+0000 7f8098d5c640 1 -- 172.21.15.49:0/3246048205 wait complete. 2023-11-06T02:10:42.118 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.118+0000 7f8098d5c640 1 Processor -- start 2023-11-06T02:10:42.118 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.118+0000 7f8098d5c640 1 -- start start 2023-11-06T02:10:42.123 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.118+0000 7f8098d5c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f809406c230 0x7f8094152df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:42.123 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.118+0000 7f8098d5c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8094070160 0x7f80940b2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:42.123 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.118+0000 7f8098d5c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f809414ffc0 0x7f8094149f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:42.123 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.118+0000 7f8098d5c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80940b3950 con 0x7f809414ffc0 2023-11-06T02:10:42.124 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.118+0000 7f8098d5c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8094153330 con 0x7f8094070160 2023-11-06T02:10:42.124 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.118+0000 7f8098d5c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80940b3bf0 con 0x7f809406c230 2023-11-06T02:10:42.124 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.118+0000 7f8092d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f809414ffc0 0x7f8094149f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:42.124 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.118+0000 7f8092d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f809414ffc0 0x7f8094149f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40342/0 (socket says 172.21.15.49:40342) 2023-11-06T02:10:42.124 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.118+0000 7f8092d9d640 1 -- 172.21.15.49:0/3291702248 learned_addr learned my addr 172.21.15.49:0/3291702248 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:42.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.118+0000 7f809259c640 1 --2- 172.21.15.49:0/3291702248 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f809406c230 0x7f8094152df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:42.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.118+0000 7f8091d9b640 1 --2- 172.21.15.49:0/3291702248 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8094070160 0x7f80940b2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:42.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.118+0000 7f8092d9d640 1 -- 172.21.15.49:0/3291702248 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f809406c230 msgr2=0x7f8094152df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:42.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.118+0000 7f8092d9d640 1 --2- 172.21.15.49:0/3291702248 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f809406c230 0x7f8094152df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:42.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.118+0000 7f8092d9d640 1 -- 172.21.15.49:0/3291702248 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8094070160 msgr2=0x7f80940b2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:42.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.118+0000 7f8092d9d640 1 --2- 172.21.15.49:0/3291702248 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8094070160 0x7f80940b2fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:42.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.118+0000 7f8092d9d640 1 -- 172.21.15.49:0/3291702248 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8088009420 con 0x7f809414ffc0 2023-11-06T02:10:42.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.118+0000 7f8091d9b640 1 --2- 172.21.15.49:0/3291702248 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8094070160 0x7f80940b2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:10:42.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.118+0000 7f8092d9d640 1 --2- 172.21.15.49:0/3291702248 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f809414ffc0 0x7f8094149f20 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f808c007e60 tx=0x7f808c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:42.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.118+0000 7f8082ffd640 1 -- 172.21.15.49:0/3291702248 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f808c014420 con 0x7f809414ffc0 2023-11-06T02:10:42.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.118+0000 7f8082ffd640 1 -- 172.21.15.49:0/3291702248 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f808c0145a0 con 0x7f809414ffc0 2023-11-06T02:10:42.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.118+0000 7f8098d5c640 1 -- 172.21.15.49:0/3291702248 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80940b3ed0 con 0x7f809414ffc0 2023-11-06T02:10:42.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.118+0000 7f8082ffd640 1 -- 172.21.15.49:0/3291702248 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f808c014950 con 0x7f809414ffc0 2023-11-06T02:10:42.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.118+0000 7f8098d5c640 1 -- 172.21.15.49:0/3291702248 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80940b1700 con 0x7f809414ffc0 2023-11-06T02:10:42.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.122+0000 7f8082ffd640 1 -- 172.21.15.49:0/3291702248 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f808c014ab0 con 0x7f809414ffc0 2023-11-06T02:10:42.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.122+0000 7f8082ffd640 1 --2- 172.21.15.49:0/3291702248 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8074065a60 0x7f8074067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:42.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.122+0000 7f8082ffd640 1 -- 172.21.15.49:0/3291702248 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f808c083cc0 con 0x7f809414ffc0 2023-11-06T02:10:42.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.122+0000 7f809259c640 1 --2- 172.21.15.49:0/3291702248 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8074065a60 0x7f8074067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:42.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.122+0000 7f809be84000 1 Processor -- start 2023-11-06T02:10:42.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.122+0000 7f809be84000 1 -- start start 2023-11-06T02:10:42.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.122+0000 7f809259c640 1 --2- 172.21.15.49:0/3291702248 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8074065a60 0x7f8074067f20 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f80840011f0 tx=0x7f8084009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:42.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.122+0000 7f809be84000 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ad59b6f130 0x55ad59b6f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:42.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.122+0000 7f809be84000 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ad59b6fa50 0x55ad59b24d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:42.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.122+0000 7f809be84000 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad59b25260 0x55ad59b25640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:42.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.122+0000 7f809be84000 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad599ecc10 con 0x55ad59b25260 2023-11-06T02:10:42.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.122+0000 7f809be84000 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad5997b3e0 con 0x55ad59b6f130 2023-11-06T02:10:42.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.122+0000 7f809be84000 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ad599789b0 con 0x55ad59b6fa50 2023-11-06T02:10:42.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.122+0000 7f8092d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad59b25260 0x55ad59b25640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:42.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.122+0000 7f8092d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad59b25260 0x55ad59b25640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40358/0 (socket says 172.21.15.49:40358) 2023-11-06T02:10:42.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.122+0000 7f8092d9d640 1 -- 172.21.15.49:0/2078148149 learned_addr learned my addr 172.21.15.49:0/2078148149 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:42.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.126+0000 7f8091d9b640 1 --2- 172.21.15.49:0/2078148149 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ad59b6fa50 0x55ad59b24d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:42.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.126+0000 7f809259c640 1 --2- 172.21.15.49:0/2078148149 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ad59b6f130 0x55ad59b6f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:42.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.126+0000 7f8092d9d640 1 -- 172.21.15.49:0/2078148149 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ad59b6fa50 msgr2=0x55ad59b24d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:42.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.126+0000 7f8092d9d640 1 --2- 172.21.15.49:0/2078148149 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ad59b6fa50 0x55ad59b24d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:42.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.126+0000 7f8092d9d640 1 -- 172.21.15.49:0/2078148149 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ad59b6f130 msgr2=0x55ad59b6f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:42.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.126+0000 7f8092d9d640 1 --2- 172.21.15.49:0/2078148149 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ad59b6f130 0x55ad59b6f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:42.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.126+0000 7f8092d9d640 1 -- 172.21.15.49:0/2078148149 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ad59670e20 con 0x55ad59b25260 2023-11-06T02:10:42.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.126+0000 7f8092d9d640 1 --2- 172.21.15.49:0/2078148149 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad59b25260 0x55ad59b25640 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f808c00c040 tx=0x7f808c054bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e598fd8fa7ff019 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:42.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.126+0000 7f8073fff640 1 -- 172.21.15.49:0/2078148149 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f808c05b070 con 0x55ad59b25260 2023-11-06T02:10:42.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.126+0000 7f8073fff640 1 -- 172.21.15.49:0/2078148149 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f808c05d120 con 0x55ad59b25260 2023-11-06T02:10:42.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.126+0000 7f8073fff640 1 -- 172.21.15.49:0/2078148149 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f808c051850 con 0x55ad59b25260 2023-11-06T02:10:42.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.126+0000 7f809be84000 1 -- 172.21.15.49:0/2078148149 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad59b25260 msgr2=0x55ad59b25640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:42.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.126+0000 7f809be84000 1 --2- 172.21.15.49:0/2078148149 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad59b25260 0x55ad59b25640 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f808c00c040 tx=0x7f808c054bd0 comp rx=0 tx=0).stop 2023-11-06T02:10:42.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.126+0000 7f809be84000 1 -- 172.21.15.49:0/2078148149 shutdown_connections 2023-11-06T02:10:42.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.126+0000 7f809be84000 1 --2- 172.21.15.49:0/2078148149 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad59b25260 0x55ad59b25640 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:42.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.126+0000 7f809be84000 1 --2- 172.21.15.49:0/2078148149 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ad59b6fa50 0x55ad59b24d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:42.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.126+0000 7f809be84000 1 --2- 172.21.15.49:0/2078148149 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ad59b6f130 0x55ad59b6f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:42.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.126+0000 7f809be84000 1 -- 172.21.15.49:0/2078148149 >> 172.21.15.49:0/2078148149 conn(0x55ad59b2b9f0 msgr2=0x55ad59b23600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:42.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.126+0000 7f809be84000 1 -- 172.21.15.49:0/2078148149 shutdown_connections 2023-11-06T02:10:42.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.126+0000 7f809be84000 1 -- 172.21.15.49:0/2078148149 wait complete. 2023-11-06T02:10:42.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.126+0000 7f809be84000 1 -- start start 2023-11-06T02:10:42.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.126+0000 7f809be84000 1 Processor -- start 2023-11-06T02:10:42.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.126+0000 7f809be84000 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad59b25260 0x55ad59b6fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:42.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.126+0000 7f809be84000 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ad59b6f130 0x55ad59b70370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:42.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.126+0000 7f809be84000 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ad59b6fa50 0x55ad59b708b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:42.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.126+0000 7f809be84000 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad599742a0 con 0x55ad59b25260 2023-11-06T02:10:42.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.126+0000 7f809be84000 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad59913470 con 0x55ad59b6f130 2023-11-06T02:10:42.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.126+0000 7f809be84000 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ad59931660 con 0x55ad59b6fa50 2023-11-06T02:10:42.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.126+0000 7f809be84000 10 client.0 _ll_register_callbacks cb 0x55ad59b10c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T02:10:42.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.126+0000 7f809259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad59b25260 0x55ad59b6fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:42.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.126+0000 7f809259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad59b25260 0x55ad59b6fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40374/0 (socket says 172.21.15.49:40374) 2023-11-06T02:10:42.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.126+0000 7f809259c640 1 -- 172.21.15.49:0/4183879351 learned_addr learned my addr 172.21.15.49:0/4183879351 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:42.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.126+0000 7f8092d9d640 1 --2- 172.21.15.49:0/4183879351 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ad59b6fa50 0x55ad59b708b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:42.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.126+0000 7f8091d9b640 1 --2- 172.21.15.49:0/4183879351 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ad59b6f130 0x55ad59b70370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:42.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.126+0000 7f809259c640 1 -- 172.21.15.49:0/4183879351 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ad59b6fa50 msgr2=0x55ad59b708b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:42.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.126+0000 7f809259c640 1 --2- 172.21.15.49:0/4183879351 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ad59b6fa50 0x55ad59b708b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:42.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.126+0000 7f809259c640 1 -- 172.21.15.49:0/4183879351 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ad59b6f130 msgr2=0x55ad59b70370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:42.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.126+0000 7f809259c640 1 --2- 172.21.15.49:0/4183879351 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ad59b6f130 0x55ad59b70370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:42.156 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.126+0000 7f809259c640 1 -- 172.21.15.49:0/4183879351 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80940b1700 con 0x55ad59b25260 2023-11-06T02:10:42.156 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.126+0000 7f809259c640 1 --2- 172.21.15.49:0/4183879351 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad59b25260 0x55ad59b6fe30 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f8084005e60 tx=0x7f8084000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f9c765373c8b9c7 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:42.156 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.126+0000 7f80717fa640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-11-06T02:10:42.156 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.126+0000 7f80717fa640 1 -- 172.21.15.49:0/4183879351 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8084007560 con 0x55ad59b25260 2023-11-06T02:10:42.157 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.126+0000 7f80717fa640 1 -- 172.21.15.49:0/4183879351 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f80840076c0 con 0x55ad59b25260 2023-11-06T02:10:42.157 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.126+0000 7f809be84000 1 -- 172.21.15.49:0/4183879351 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ad599310a0 con 0x55ad59b25260 2023-11-06T02:10:42.157 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.126+0000 7f80717fa640 1 -- 172.21.15.49:0/4183879351 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8084013420 con 0x55ad59b25260 2023-11-06T02:10:42.158 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.126+0000 7f809be84000 10 client.6102 fetch_fsmap learned FSMap version 23 2023-11-06T02:10:42.158 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.126+0000 7f809be84000 1 -- 172.21.15.49:0/4183879351 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x55ad599391d0 con 0x55ad59b25260 2023-11-06T02:10:42.158 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.126+0000 7f80717fa640 1 -- 172.21.15.49:0/4183879351 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ad599310a0 con 0x55ad59b25260 2023-11-06T02:10:42.159 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.126+0000 7f80717fa640 1 -- 172.21.15.49:0/4183879351 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 23) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f8084013580 con 0x55ad59b25260 2023-11-06T02:10:42.159 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.130+0000 7f809be84000 10 client.6102 fetch_fsmap finished waiting for FSMap version 23 2023-11-06T02:10:42.159 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.130+0000 7f809be84000 10 client.6102 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2023-11-06T02:10:42.160 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.130+0000 7f809be84000 20 client.6102 populate_metadata read hostname 'smithi049' 2023-11-06T02:10:42.160 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.130+0000 7f809be84000 1 --2- 172.21.15.49:0/4183879351 >> [v2:172.21.15.49:6834/73939548,v1:172.21.15.49:6835/73939548] conn(0x55ad59b7ecf0 0x55ad59b810e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:42.160 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.130+0000 7f809be84000 4 client.6102 mds_command: new command op to 5877 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T02:10:42.160 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.130+0000 7f809be84000 1 -- 172.21.15.49:0/4183879351 --> [v2:172.21.15.49:6834/73939548,v1:172.21.15.49:6835/73939548] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ad59972bf0 con 0x55ad59b7ecf0 2023-11-06T02:10:42.161 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.130+0000 7f8091d9b640 1 --2- 172.21.15.49:0/4183879351 >> [v2:172.21.15.49:6834/73939548,v1:172.21.15.49:6835/73939548] conn(0x55ad59b7ecf0 0x55ad59b810e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:42.161 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.130+0000 7f8091d9b640 1 --2- 172.21.15.49:0/4183879351 >> [v2:172.21.15.49:6834/73939548,v1:172.21.15.49:6835/73939548] conn(0x55ad59b7ecf0 0x55ad59b810e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15b389f92580a4e3 server_cookie=8eb0824811b2deb8 in_seq=0 out_seq=0 2023-11-06T02:10:42.161 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.130+0000 7f80717fa640 10 client.6102 ms_handle_connect on v2:172.21.15.49:6834/73939548 2023-11-06T02:10:42.162 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.130+0000 7f80717fa640 1 -- 172.21.15.49:0/4183879351 <== mds.0 v2:172.21.15.49:6834/73939548 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55ad59972bf0 con 0x55ad59b7ecf0 2023-11-06T02:10:42.162 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.130+0000 7f80717fa640 10 client.6102 handle_command_reply: tid=0 2023-11-06T02:10:42.162 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.130+0000 7f809be84000 1 client.6102 shutdown 2023-11-06T02:10:42.163 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.130+0000 7f809be84000 1 -- 172.21.15.49:0/4183879351 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad59b25260 msgr2=0x55ad59b6fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:42.163 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.130+0000 7f809be84000 1 --2- 172.21.15.49:0/4183879351 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad59b25260 0x55ad59b6fe30 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f8084005e60 tx=0x7f8084000ee0 comp rx=0 tx=0).stop 2023-11-06T02:10:42.163 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.130+0000 7f809be84000 1 -- 172.21.15.49:0/4183879351 shutdown_connections 2023-11-06T02:10:42.163 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.130+0000 7f809be84000 1 --2- 172.21.15.49:0/4183879351 >> [v2:172.21.15.49:6834/73939548,v1:172.21.15.49:6835/73939548] conn(0x55ad59b7ecf0 0x55ad59b810e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:42.164 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.130+0000 7f809be84000 1 --2- 172.21.15.49:0/4183879351 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ad59b6fa50 0x55ad59b708b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:42.164 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.130+0000 7f80717fa640 0 client.6102 ms_handle_reset on v2:172.21.15.49:6834/73939548 2023-11-06T02:10:42.164 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.130+0000 7f809be84000 1 --2- 172.21.15.49:0/4183879351 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ad59b6f130 0x55ad59b70370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:42.165 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.130+0000 7f809be84000 1 --2- 172.21.15.49:0/4183879351 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad59b25260 0x55ad59b6fe30 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:42.165 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.130+0000 7f809be84000 1 -- 172.21.15.49:0/4183879351 >> 172.21.15.49:0/4183879351 conn(0x55ad59b2b9f0 msgr2=0x55ad59b23d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:42.165 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.130+0000 7f809be84000 1 -- 172.21.15.49:0/4183879351 shutdown_connections 2023-11-06T02:10:42.165 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.130+0000 7f809be84000 1 -- 172.21.15.49:0/4183879351 wait complete. 2023-11-06T02:10:42.165 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.130+0000 7f809be84000 20 client.6102 trim_cache size 0 max 16384 2023-11-06T02:10:42.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.142+0000 7f809be84000 1 Processor -- start 2023-11-06T02:10:42.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.142+0000 7f809be84000 1 -- start start 2023-11-06T02:10:42.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.146+0000 7f809be84000 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ad59b25260 0x55ad59b810e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:42.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.146+0000 7f809be84000 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad59b6f130 0x55ad59b81620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:42.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.146+0000 7f809be84000 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ad59b6fa50 0x55ad59b81b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:42.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.146+0000 7f809be84000 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad59b70a60 con 0x55ad59b6f130 2023-11-06T02:10:42.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.146+0000 7f809be84000 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad59631aa0 con 0x55ad59b25260 2023-11-06T02:10:42.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.146+0000 7f809be84000 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ad59971ab0 con 0x55ad59b6fa50 2023-11-06T02:10:42.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.146+0000 7f8091d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad59b6f130 0x55ad59b81620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:42.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.146+0000 7f8091d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad59b6f130 0x55ad59b81620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40386/0 (socket says 172.21.15.49:40386) 2023-11-06T02:10:42.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.146+0000 7f8091d9b640 1 -- 172.21.15.49:0/4279202496 learned_addr learned my addr 172.21.15.49:0/4279202496 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:42.169 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.146+0000 7f8091d9b640 1 -- 172.21.15.49:0/4279202496 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ad59b6fa50 msgr2=0x55ad59b81b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:42.169 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.146+0000 7f8091d9b640 1 --2- 172.21.15.49:0/4279202496 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ad59b6fa50 0x55ad59b81b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:42.169 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.146+0000 7f8091d9b640 1 -- 172.21.15.49:0/4279202496 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ad59b25260 msgr2=0x55ad59b810e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:42.169 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.146+0000 7f8091d9b640 1 --2- 172.21.15.49:0/4279202496 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ad59b25260 0x55ad59b810e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:42.169 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.146+0000 7f8091d9b640 1 -- 172.21.15.49:0/4279202496 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ad5991f520 con 0x55ad59b6f130 2023-11-06T02:10:42.170 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.146+0000 7f8091d9b640 1 --2- 172.21.15.49:0/4279202496 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad59b6f130 0x55ad59b81620 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f80880ae040 tx=0x7f80880a6000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=625bb199da649001 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:42.170 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.146+0000 7f8080ff9640 1 -- 172.21.15.49:0/4279202496 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80880ab700 con 0x55ad59b6f130 2023-11-06T02:10:42.170 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.146+0000 7f8080ff9640 1 -- 172.21.15.49:0/4279202496 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80880a4e90 con 0x55ad59b6f130 2023-11-06T02:10:42.170 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.146+0000 7f8080ff9640 1 -- 172.21.15.49:0/4279202496 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80880aba60 con 0x55ad59b6f130 2023-11-06T02:10:42.171 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.146+0000 7f809be84000 1 -- 172.21.15.49:0/4279202496 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad59b6f130 msgr2=0x55ad59b81620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:42.171 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.146+0000 7f809be84000 1 --2- 172.21.15.49:0/4279202496 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad59b6f130 0x55ad59b81620 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f80880ae040 tx=0x7f80880a6000 comp rx=0 tx=0).stop 2023-11-06T02:10:42.171 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.146+0000 7f809be84000 1 -- 172.21.15.49:0/4279202496 shutdown_connections 2023-11-06T02:10:42.171 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.146+0000 7f809be84000 1 --2- 172.21.15.49:0/4279202496 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad59b6f130 0x55ad59b81620 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:42.171 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.146+0000 7f809be84000 1 --2- 172.21.15.49:0/4279202496 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ad59b6fa50 0x55ad59b81b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:42.172 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.146+0000 7f809be84000 1 --2- 172.21.15.49:0/4279202496 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ad59b25260 0x55ad59b810e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:42.172 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.146+0000 7f809be84000 1 -- 172.21.15.49:0/4279202496 >> 172.21.15.49:0/4279202496 conn(0x55ad59b2b9f0 msgr2=0x55ad59b29b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:42.172 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.146+0000 7f809be84000 1 -- 172.21.15.49:0/4279202496 shutdown_connections 2023-11-06T02:10:42.173 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.146+0000 7f809be84000 1 -- 172.21.15.49:0/4279202496 wait complete. 2023-11-06T02:10:42.173 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.146+0000 7f809be84000 1 -- start start 2023-11-06T02:10:42.173 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.146+0000 7f809be84000 1 Processor -- start 2023-11-06T02:10:42.173 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.146+0000 7f809be84000 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ad59b25260 0x55ad59bcc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:42.174 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.146+0000 7f809be84000 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ad59b6f130 0x55ad59bcc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:42.174 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.146+0000 7f809be84000 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad59b6fa50 0x55ad59b69760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:42.174 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.146+0000 7f809be84000 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad599197a0 con 0x55ad59b6fa50 2023-11-06T02:10:42.175 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.146+0000 7f809be84000 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad59973130 con 0x55ad59b25260 2023-11-06T02:10:42.175 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.146+0000 7f809be84000 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ad59b70920 con 0x55ad59b6f130 2023-11-06T02:10:42.175 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.146+0000 7f809be84000 10 client.0 _ll_register_callbacks cb 0x55ad59938cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T02:10:42.175 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.146+0000 7f8092d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad59b6fa50 0x55ad59b69760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:42.176 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.146+0000 7f8092d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad59b6fa50 0x55ad59b69760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40392/0 (socket says 172.21.15.49:40392) 2023-11-06T02:10:42.176 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.146+0000 7f8092d9d640 1 -- 172.21.15.49:0/3517975994 learned_addr learned my addr 172.21.15.49:0/3517975994 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:10:42.176 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.146+0000 7f809259c640 1 --2- 172.21.15.49:0/3517975994 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ad59b25260 0x55ad59bcc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:42.177 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.146+0000 7f8091d9b640 1 --2- 172.21.15.49:0/3517975994 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ad59b6f130 0x55ad59bcc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:42.177 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.146+0000 7f8092d9d640 1 -- 172.21.15.49:0/3517975994 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ad59b6f130 msgr2=0x55ad59bcc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:42.177 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.146+0000 7f8092d9d640 1 --2- 172.21.15.49:0/3517975994 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ad59b6f130 0x55ad59bcc580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:42.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.146+0000 7f8092d9d640 1 -- 172.21.15.49:0/3517975994 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ad59b25260 msgr2=0x55ad59bcc040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:10:42.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.146+0000 7f8092d9d640 1 --2- 172.21.15.49:0/3517975994 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ad59b25260 0x55ad59bcc040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:42.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.146+0000 7f8092d9d640 1 -- 172.21.15.49:0/3517975994 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8094159700 con 0x55ad59b6fa50 2023-11-06T02:10:42.179 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.146+0000 7f8092d9d640 1 --2- 172.21.15.49:0/3517975994 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad59b6fa50 0x55ad59b69760 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f808c00efd0 tx=0x7f808c081ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6b4f51849bcd920 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:10:42.179 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.146+0000 7f80717fa640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-11-06T02:10:42.179 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.146+0000 7f80717fa640 1 -- 172.21.15.49:0/3517975994 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f808c07d040 con 0x55ad59b6fa50 2023-11-06T02:10:42.180 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.146+0000 7f80717fa640 1 -- 172.21.15.49:0/3517975994 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f808c05d120 con 0x55ad59b6fa50 2023-11-06T02:10:42.180 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.146+0000 7f809be84000 1 -- 172.21.15.49:0/3517975994 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ad5993d620 con 0x55ad59b6fa50 2023-11-06T02:10:42.180 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.150+0000 7f80717fa640 1 -- 172.21.15.49:0/3517975994 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f808c05b070 con 0x55ad59b6fa50 2023-11-06T02:10:42.180 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.150+0000 7f809be84000 10 client.6108 fetch_fsmap learned FSMap version 23 2023-11-06T02:10:42.181 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.150+0000 7f809be84000 1 -- 172.21.15.49:0/3517975994 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x55ad59913090 con 0x55ad59b6fa50 2023-11-06T02:10:42.181 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.150+0000 7f80717fa640 1 -- 172.21.15.49:0/3517975994 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ad5993d620 con 0x55ad59b6fa50 2023-11-06T02:10:42.181 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.150+0000 7f80717fa640 1 -- 172.21.15.49:0/3517975994 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 23) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f808c005d20 con 0x55ad59b6fa50 2023-11-06T02:10:42.182 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.150+0000 7f809be84000 10 client.6108 fetch_fsmap finished waiting for FSMap version 23 2023-11-06T02:10:42.182 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.150+0000 7f809be84000 10 client.6108 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2023-11-06T02:10:42.182 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.150+0000 7f809be84000 20 client.6108 populate_metadata read hostname 'smithi049' 2023-11-06T02:10:42.183 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.150+0000 7f809be84000 1 --2- 172.21.15.49:0/3517975994 >> [v2:172.21.15.49:6834/73939548,v1:172.21.15.49:6835/73939548] conn(0x55ad59b69ca0 0x55ad59b6a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:10:42.183 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.150+0000 7f809be84000 4 client.6108 mds_command: new command op to 5877 tid=0[{"prefix": "scrub status"}] 2023-11-06T02:10:42.183 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.150+0000 7f809be84000 1 -- 172.21.15.49:0/3517975994 --> [v2:172.21.15.49:6834/73939548,v1:172.21.15.49:6835/73939548] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ad5996d320 con 0x55ad59b69ca0 2023-11-06T02:10:42.183 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.150+0000 7f809259c640 1 --2- 172.21.15.49:0/3517975994 >> [v2:172.21.15.49:6834/73939548,v1:172.21.15.49:6835/73939548] conn(0x55ad59b69ca0 0x55ad59b6a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:10:42.184 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.150+0000 7f80717fa640 10 client.6108 ms_handle_connect on v2:172.21.15.49:6834/73939548 2023-11-06T02:10:42.184 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.150+0000 7f809259c640 1 --2- 172.21.15.49:0/3517975994 >> [v2:172.21.15.49:6834/73939548,v1:172.21.15.49:6835/73939548] conn(0x55ad59b69ca0 0x55ad59b6a0b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3088da4c0f77798 server_cookie=2af2cec129fd3624 in_seq=0 out_seq=0 2023-11-06T02:10:42.185 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.150+0000 7f80717fa640 1 -- 172.21.15.49:0/3517975994 <== mds.0 v2:172.21.15.49:6834/73939548 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+260 (crc 0 0 0) 0x55ad5996d320 con 0x55ad59b69ca0 2023-11-06T02:10:42.185 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.150+0000 7f80717fa640 10 client.6108 handle_command_reply: tid=0 2023-11-06T02:10:42.185 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.150+0000 7f809be84000 1 client.6108 shutdown 2023-11-06T02:10:42.186 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.150+0000 7f809be84000 1 -- 172.21.15.49:0/3517975994 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad59b6fa50 msgr2=0x55ad59b69760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:42.186 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.150+0000 7f809be84000 1 --2- 172.21.15.49:0/3517975994 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad59b6fa50 0x55ad59b69760 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f808c00efd0 tx=0x7f808c081ec0 comp rx=0 tx=0).stop 2023-11-06T02:10:42.186 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.150+0000 7f809be84000 1 -- 172.21.15.49:0/3517975994 shutdown_connections 2023-11-06T02:10:42.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.150+0000 7f809be84000 1 --2- 172.21.15.49:0/3517975994 >> [v2:172.21.15.49:6834/73939548,v1:172.21.15.49:6835/73939548] conn(0x55ad59b69ca0 0x55ad59b6a0b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:42.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.150+0000 7f809be84000 1 --2- 172.21.15.49:0/3517975994 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ad59b6fa50 0x55ad59b69760 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:42.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.150+0000 7f809be84000 1 --2- 172.21.15.49:0/3517975994 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ad59b6f130 0x55ad59bcc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:42.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.150+0000 7f809be84000 1 --2- 172.21.15.49:0/3517975994 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ad59b25260 0x55ad59bcc040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:42.188 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.150+0000 7f80717fa640 0 client.6108 ms_handle_reset on v2:172.21.15.49:6834/73939548 2023-11-06T02:10:42.188 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.150+0000 7f809be84000 1 -- 172.21.15.49:0/3517975994 >> 172.21.15.49:0/3517975994 conn(0x55ad59b2b9f0 msgr2=0x55ad59b2a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:42.188 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.150+0000 7f809be84000 1 -- 172.21.15.49:0/3517975994 shutdown_connections 2023-11-06T02:10:42.189 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.150+0000 7f809be84000 1 -- 172.21.15.49:0/3517975994 wait complete. 2023-11-06T02:10:42.189 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.150+0000 7f809be84000 20 client.6108 trim_cache size 0 max 16384 2023-11-06T02:10:42.190 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-11-06T02:10:42.190 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-06T02:10:42.190 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": { 2023-11-06T02:10:42.191 INFO:teuthology.orchestra.run.smithi049.stdout: "03224875-0c30-4c54-853c-d6d7bcbe1f1a": { 2023-11-06T02:10:42.191 INFO:teuthology.orchestra.run.smithi049.stdout: "path": "/", 2023-11-06T02:10:42.191 INFO:teuthology.orchestra.run.smithi049.stdout: "tag": "03224875-0c30-4c54-853c-d6d7bcbe1f1a", 2023-11-06T02:10:42.191 INFO:teuthology.orchestra.run.smithi049.stdout: "options": "recursive,repair" 2023-11-06T02:10:42.192 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:10:42.192 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:10:42.192 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-11-06T02:10:42.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.150+0000 7f8098d5c640 1 -- 172.21.15.49:0/3291702248 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8074065a60 msgr2=0x7f8074067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:42.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.150+0000 7f8098d5c640 1 --2- 172.21.15.49:0/3291702248 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8074065a60 0x7f8074067f20 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f80840011f0 tx=0x7f8084009040 comp rx=0 tx=0).stop 2023-11-06T02:10:42.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.150+0000 7f8098d5c640 1 -- 172.21.15.49:0/3291702248 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f809414ffc0 msgr2=0x7f8094149f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:10:42.195 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.150+0000 7f8098d5c640 1 --2- 172.21.15.49:0/3291702248 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f809414ffc0 0x7f8094149f20 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f808c007e60 tx=0x7f808c00e4b0 comp rx=0 tx=0).stop 2023-11-06T02:10:42.195 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.150+0000 7f8098d5c640 1 -- 172.21.15.49:0/3291702248 shutdown_connections 2023-11-06T02:10:42.195 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.150+0000 7f8098d5c640 1 --2- 172.21.15.49:0/3291702248 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8074065a60 0x7f8074067f20 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:42.195 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.150+0000 7f8098d5c640 1 --2- 172.21.15.49:0/3291702248 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f809414ffc0 0x7f8094149f20 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:42.196 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.150+0000 7f8098d5c640 1 --2- 172.21.15.49:0/3291702248 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8094070160 0x7f80940b2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:42.196 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.150+0000 7f8098d5c640 1 --2- 172.21.15.49:0/3291702248 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f809406c230 0x7f8094152df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:10:42.196 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.150+0000 7f8098d5c640 1 -- 172.21.15.49:0/3291702248 >> 172.21.15.49:0/3291702248 conn(0x7f809406a370 msgr2=0x7f80940c8c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:10:42.196 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.150+0000 7f8098d5c640 1 -- 172.21.15.49:0/3291702248 shutdown_connections 2023-11-06T02:10:42.197 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:42.150+0000 7f8098d5c640 1 -- 172.21.15.49:0/3291702248 wait complete. 2023-11-06T02:10:42.197 INFO:tasks.cephfs.filesystem:scrub status for tag:03224875-0c30-4c54-853c-d6d7bcbe1f1a - {'path': '/', 'tag': '03224875-0c30-4c54-853c-d6d7bcbe1f1a', 'options': 'recursive,repair'} 2023-11-06T02:10:43.159 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:43.162+0000 7f58abfff640 1 -- 172.21.15.49:0/1924679726 <== mon.0 v2:172.21.15.49:3300/0 7 ==== log(6 entries from seq 1 at 2023-11-06T02:10:41.953626+0000) v1 ==== 1078+0+0 (secure 0 0 0) 0x7f58b40ef980 con 0x7f58c40cf120 2023-11-06T02:10:43.160 INFO:teuthology.orchestra.run.smithi049.stdout: cluster: 2023-11-06T02:10:43.160 INFO:teuthology.orchestra.run.smithi049.stdout: id: 1a8b035c-6ba4-499a-b876-aea47fd9c0d4 2023-11-06T02:10:43.161 INFO:teuthology.orchestra.run.smithi049.stdout: health: HEALTH_OK 2023-11-06T02:10:43.161 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:10:43.161 INFO:teuthology.orchestra.run.smithi049.stdout: services: 2023-11-06T02:10:43.161 INFO:teuthology.orchestra.run.smithi049.stdout: mon: 3 daemons, quorum a,b,c (age 2m) 2023-11-06T02:10:43.161 INFO:teuthology.orchestra.run.smithi049.stdout: mgr: x(active, since 2m), standbys: y 2023-11-06T02:10:43.162 INFO:teuthology.orchestra.run.smithi049.stdout: mds: 1/1 daemons up, 3 standby 2023-11-06T02:10:43.162 INFO:teuthology.orchestra.run.smithi049.stdout: osd: 8 osds: 8 up (since 2m), 8 in (since 2m) 2023-11-06T02:10:43.162 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:10:43.162 INFO:teuthology.orchestra.run.smithi049.stdout: data: 2023-11-06T02:10:43.163 INFO:teuthology.orchestra.run.smithi049.stdout: volumes: 1/1 healthy 2023-11-06T02:10:43.163 INFO:teuthology.orchestra.run.smithi049.stdout: pools: 4 pools, 105 pgs 2023-11-06T02:10:43.163 INFO:teuthology.orchestra.run.smithi049.stdout: objects: 24 objects, 579 KiB 2023-11-06T02:10:43.163 INFO:teuthology.orchestra.run.smithi049.stdout: usage: 232 MiB used, 720 GiB / 720 GiB avail 2023-11-06T02:10:43.163 INFO:teuthology.orchestra.run.smithi049.stdout: pgs: 105 active+clean 2023-11-06T02:10:43.164 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:10:43.164 INFO:teuthology.orchestra.run.smithi049.stdout: io: 2023-11-06T02:10:43.164 INFO:teuthology.orchestra.run.smithi049.stdout: client: 1.8 KiB/s wr, 0 op/s rd, 5 op/s wr 2023-11-06T02:10:43.164 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:10:43.164 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:10:43.164 INFO:teuthology.orchestra.run.smithi049.stdout:2023-11-06T02:10:41.953626+0000 mds.a [INF] scrub queued for path: / 2023-11-06T02:10:43.165 INFO:teuthology.orchestra.run.smithi049.stdout:2023-11-06T02:10:41.953630+0000 mds.a [INF] scrub summary: idle+waiting paths [/] 2023-11-06T02:10:43.165 INFO:teuthology.orchestra.run.smithi049.stdout:2023-11-06T02:10:41.953634+0000 mds.a [INF] scrub summary: active paths [/] 2023-11-06T02:10:43.165 INFO:teuthology.orchestra.run.smithi049.stdout:2023-11-06T02:10:41.954387+0000 mds.a [WRN] bad backtrace on inode 0x10000000001(/parent_a/alpha), rewriting it 2023-11-06T02:10:43.165 INFO:teuthology.orchestra.run.smithi049.stdout:2023-11-06T02:10:41.954400+0000 mds.a [INF] Scrub repaired inode 0x10000000001 (/parent_a/alpha) 2023-11-06T02:10:43.165 INFO:teuthology.orchestra.run.smithi049.stdout:2023-11-06T02:10:41.955472+0000 mds.a [INF] scrub summary: idle+waiting paths [/] 2023-11-06T02:10:44.169 INFO:teuthology.orchestra.run.smithi049.stdout:2023-11-06T02:10:42.371098+0000 mds.a [INF] scrub summary: idle 2023-11-06T02:10:44.170 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:10:44.174+0000 7f58abfff640 1 -- 172.21.15.49:0/1924679726 <== mon.0 v2:172.21.15.49:3300/0 8 ==== log(1 entries from seq 7 at 2023-11-06T02:10:42.371098+0000) v1 ==== 189+0+0 (secure 0 0 0) 0x7f58b40ef280 con 0x7f58c40cf120 2023-11-06T02:10:51.745 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T02:10:51.748 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T02:11:12.198 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub status 2023-11-06T02:11:12.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.338+0000 7fc435b24640 1 Processor -- start 2023-11-06T02:11:12.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.338+0000 7fc435b24640 1 -- start start 2023-11-06T02:11:12.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.342+0000 7fc435b24640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc430153d50 0x7fc43015d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:12.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.342+0000 7fc435b24640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc4300707b0 0x7fc430070be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:12.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.342+0000 7fc435b24640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc43015e100 con 0x7fc4300c9100 2023-11-06T02:11:12.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.342+0000 7fc435b24640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc43006e0a0 con 0x7fc4300707b0 2023-11-06T02:11:12.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.342+0000 7fc435b24640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc430154130 con 0x7fc430153d50 2023-11-06T02:11:12.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.342+0000 7fc42ffff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc4300707b0 0x7fc430070be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:12.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.342+0000 7fc42effd640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fc4300c9100 0x7fc43015e320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:59958/0 (socket says 172.21.15.49:59958) 2023-11-06T02:11:12.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.342+0000 7fc42effd640 1 -- 172.21.15.49:0/4161613056 learned_addr learned my addr 172.21.15.49:0/4161613056 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:11:12.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.342+0000 7fc42ffff640 1 -- 172.21.15.49:0/4161613056 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc430153d50 msgr2=0x7fc43015d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:12.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.342+0000 7fc42f7fe640 1 --2- 172.21.15.49:0/4161613056 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc430153d50 0x7fc43015d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:12.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.342+0000 7fc42e7fc640 1 -- 172.21.15.49:0/4161613056 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2230728750 0 0) 0x7fc43015e100 con 0x7fc4300c9100 2023-11-06T02:11:12.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.342+0000 7fc42ffff640 1 --2- 172.21.15.49:0/4161613056 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc430153d50 0x7fc43015d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:12.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.342+0000 7fc42ffff640 1 -- 172.21.15.49:0/4161613056 >> v1:172.21.15.49:6789/0 conn(0x7fc4300c9100 legacy=0x7fc43015e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:12.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.342+0000 7fc42ffff640 1 -- 172.21.15.49:0/4161613056 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc430071120 con 0x7fc4300707b0 2023-11-06T02:11:12.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.342+0000 7fc42f7fe640 1 --2- 172.21.15.49:0/4161613056 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc430153d50 0x7fc43015d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:11:12.344 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.346+0000 7fc42ffff640 1 --2- 172.21.15.49:0/4161613056 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4300707b0 0x7fc430070be0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fc4280097c0 tx=0x7fc42801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75bbc40c9d9ad21f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:12.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.346+0000 7fc42e7fc640 1 -- 172.21.15.49:0/4161613056 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4280ab5f0 con 0x7fc4300707b0 2023-11-06T02:11:12.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.346+0000 7fc42e7fc640 1 -- 172.21.15.49:0/4161613056 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4280ab750 con 0x7fc4300707b0 2023-11-06T02:11:12.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.346+0000 7fc42e7fc640 1 -- 172.21.15.49:0/4161613056 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4280abae0 con 0x7fc4300707b0 2023-11-06T02:11:12.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.346+0000 7fc435b24640 1 -- 172.21.15.49:0/4161613056 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4300707b0 msgr2=0x7fc430070be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:12.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.346+0000 7fc435b24640 1 --2- 172.21.15.49:0/4161613056 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4300707b0 0x7fc430070be0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fc4280097c0 tx=0x7fc42801c3f0 comp rx=0 tx=0).stop 2023-11-06T02:11:12.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.346+0000 7fc435b24640 1 -- 172.21.15.49:0/4161613056 shutdown_connections 2023-11-06T02:11:12.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.346+0000 7fc435b24640 1 --2- 172.21.15.49:0/4161613056 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4300707b0 0x7fc430070be0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:12.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.346+0000 7fc435b24640 1 --2- 172.21.15.49:0/4161613056 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc430153d50 0x7fc43015d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:12.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.346+0000 7fc435b24640 1 -- 172.21.15.49:0/4161613056 >> 172.21.15.49:0/4161613056 conn(0x7fc43006a370 msgr2=0x7fc4300ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:12.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.346+0000 7fc435b24640 1 -- 172.21.15.49:0/4161613056 shutdown_connections 2023-11-06T02:11:12.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.346+0000 7fc435b24640 1 -- 172.21.15.49:0/4161613056 wait complete. 2023-11-06T02:11:12.350 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.350+0000 7fc435b24640 1 Processor -- start 2023-11-06T02:11:12.350 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.350+0000 7fc435b24640 1 -- start start 2023-11-06T02:11:12.351 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.350+0000 7fc435b24640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4300707b0 0x7fc43014c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:12.353 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.350+0000 7fc435b24640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4300c9100 0x7fc4300b30a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:12.354 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.350+0000 7fc435b24640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc430153d50 0x7fc4300b35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:12.354 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.350+0000 7fc435b24640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4301584e0 con 0x7fc4300c9100 2023-11-06T02:11:12.354 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.350+0000 7fc435b24640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4301587c0 con 0x7fc4300707b0 2023-11-06T02:11:12.354 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.350+0000 7fc435b24640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc430158aa0 con 0x7fc430153d50 2023-11-06T02:11:12.354 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.350+0000 7fc42effd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4300c9100 0x7fc4300b30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:12.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.350+0000 7fc42effd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4300c9100 0x7fc4300b30a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42464/0 (socket says 172.21.15.49:42464) 2023-11-06T02:11:12.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.350+0000 7fc42effd640 1 -- 172.21.15.49:0/1635088001 learned_addr learned my addr 172.21.15.49:0/1635088001 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:12.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.350+0000 7fc42effd640 1 -- 172.21.15.49:0/1635088001 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc430153d50 msgr2=0x7fc4300b35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:12.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.350+0000 7fc42effd640 1 --2- 172.21.15.49:0/1635088001 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc430153d50 0x7fc4300b35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:12.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.350+0000 7fc42effd640 1 -- 172.21.15.49:0/1635088001 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4300707b0 msgr2=0x7fc43014c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:12.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.350+0000 7fc42effd640 1 --2- 172.21.15.49:0/1635088001 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4300707b0 0x7fc43014c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:12.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.350+0000 7fc42effd640 1 -- 172.21.15.49:0/1635088001 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc428009420 con 0x7fc4300c9100 2023-11-06T02:11:12.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.350+0000 7fc42effd640 1 --2- 172.21.15.49:0/1635088001 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4300c9100 0x7fc4300b30a0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fc4180011f0 tx=0x7fc41800c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:12.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.350+0000 7fc427fff640 1 -- 172.21.15.49:0/1635088001 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4180074e0 con 0x7fc4300c9100 2023-11-06T02:11:12.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.350+0000 7fc427fff640 1 -- 172.21.15.49:0/1635088001 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc418007640 con 0x7fc4300c9100 2023-11-06T02:11:12.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.350+0000 7fc427fff640 1 -- 172.21.15.49:0/1635088001 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc418004a10 con 0x7fc4300c9100 2023-11-06T02:11:12.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.350+0000 7fc435b24640 1 -- 172.21.15.49:0/1635088001 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc430158d80 con 0x7fc4300c9100 2023-11-06T02:11:12.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.350+0000 7fc435b24640 1 -- 172.21.15.49:0/1635088001 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc43014c7f0 con 0x7fc4300c9100 2023-11-06T02:11:12.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.354+0000 7fc427fff640 1 -- 172.21.15.49:0/1635088001 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fc418004b70 con 0x7fc4300c9100 2023-11-06T02:11:12.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.354+0000 7fc427fff640 1 --2- 172.21.15.49:0/1635088001 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc410065a60 0x7fc410067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:12.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.354+0000 7fc427fff640 1 -- 172.21.15.49:0/1635088001 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc4180825c0 con 0x7fc4300c9100 2023-11-06T02:11:12.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.354+0000 7fc42f7fe640 1 --2- 172.21.15.49:0/1635088001 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc410065a60 0x7fc410067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:12.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.354+0000 7fc438c4c000 1 Processor -- start 2023-11-06T02:11:12.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.354+0000 7fc438c4c000 1 -- start start 2023-11-06T02:11:12.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.354+0000 7fc42f7fe640 1 --2- 172.21.15.49:0/1635088001 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc410065a60 0x7fc410067f20 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc43014fcf0 tx=0x7fc420008330 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:12.365 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.354+0000 7fc438c4c000 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f8eda2130 0x557f8eda2510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:12.365 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.354+0000 7fc438c4c000 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557f8eda2a50 0x557f8ed57d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:12.365 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.354+0000 7fc438c4c000 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f8ed58260 0x557f8ed58640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:12.366 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.354+0000 7fc438c4c000 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f8ec1fc10 con 0x557f8ed58260 2023-11-06T02:11:12.366 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.354+0000 7fc438c4c000 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f8ebae3e0 con 0x557f8eda2130 2023-11-06T02:11:12.366 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.354+0000 7fc438c4c000 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x557f8ebab9b0 con 0x557f8eda2a50 2023-11-06T02:11:12.366 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.354+0000 7fc42ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f8ed58260 0x557f8ed58640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:12.366 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.354+0000 7fc42ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f8ed58260 0x557f8ed58640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42466/0 (socket says 172.21.15.49:42466) 2023-11-06T02:11:12.367 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.354+0000 7fc42ffff640 1 -- 172.21.15.49:0/2031770217 learned_addr learned my addr 172.21.15.49:0/2031770217 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:12.367 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.354+0000 7fc42f7fe640 1 --2- 172.21.15.49:0/2031770217 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f8eda2130 0x557f8eda2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:12.367 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.354+0000 7fc42effd640 1 --2- 172.21.15.49:0/2031770217 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557f8eda2a50 0x557f8ed57d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:12.367 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.354+0000 7fc42ffff640 1 -- 172.21.15.49:0/2031770217 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557f8eda2a50 msgr2=0x557f8ed57d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:12.367 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.354+0000 7fc42ffff640 1 --2- 172.21.15.49:0/2031770217 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557f8eda2a50 0x557f8ed57d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:12.367 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.354+0000 7fc42ffff640 1 -- 172.21.15.49:0/2031770217 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f8eda2130 msgr2=0x557f8eda2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:12.368 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.354+0000 7fc42ffff640 1 --2- 172.21.15.49:0/2031770217 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f8eda2130 0x557f8eda2510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:12.368 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.354+0000 7fc42ffff640 1 -- 172.21.15.49:0/2031770217 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f8e8a3e20 con 0x557f8ed58260 2023-11-06T02:11:12.368 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.354+0000 7fc42effd640 1 --2- 172.21.15.49:0/2031770217 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557f8eda2a50 0x557f8ed57d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:11:12.368 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.354+0000 7fc42ffff640 1 --2- 172.21.15.49:0/2031770217 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f8ed58260 0x557f8ed58640 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fc4280ad040 tx=0x7fc428004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35c752a30ae80694 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:12.368 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.354+0000 7fc4257fa640 1 -- 172.21.15.49:0/2031770217 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4280ab8c0 con 0x557f8ed58260 2023-11-06T02:11:12.369 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.354+0000 7fc4257fa640 1 -- 172.21.15.49:0/2031770217 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4280aba20 con 0x557f8ed58260 2023-11-06T02:11:12.369 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.354+0000 7fc4257fa640 1 -- 172.21.15.49:0/2031770217 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4280abdd0 con 0x557f8ed58260 2023-11-06T02:11:12.369 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.354+0000 7fc438c4c000 1 -- 172.21.15.49:0/2031770217 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f8ed58260 msgr2=0x557f8ed58640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:12.369 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.354+0000 7fc438c4c000 1 --2- 172.21.15.49:0/2031770217 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f8ed58260 0x557f8ed58640 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fc4280ad040 tx=0x7fc428004740 comp rx=0 tx=0).stop 2023-11-06T02:11:12.369 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.354+0000 7fc438c4c000 1 -- 172.21.15.49:0/2031770217 shutdown_connections 2023-11-06T02:11:12.370 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.354+0000 7fc438c4c000 1 --2- 172.21.15.49:0/2031770217 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f8ed58260 0x557f8ed58640 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:12.370 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.354+0000 7fc438c4c000 1 --2- 172.21.15.49:0/2031770217 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557f8eda2a50 0x557f8ed57d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:12.370 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.354+0000 7fc438c4c000 1 --2- 172.21.15.49:0/2031770217 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f8eda2130 0x557f8eda2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:12.370 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.354+0000 7fc438c4c000 1 -- 172.21.15.49:0/2031770217 >> 172.21.15.49:0/2031770217 conn(0x557f8ed5e9f0 msgr2=0x557f8ed56600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:12.370 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.358+0000 7fc438c4c000 1 -- 172.21.15.49:0/2031770217 shutdown_connections 2023-11-06T02:11:12.371 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.358+0000 7fc438c4c000 1 -- 172.21.15.49:0/2031770217 wait complete. 2023-11-06T02:11:12.371 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.358+0000 7fc438c4c000 1 -- start start 2023-11-06T02:11:12.371 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.358+0000 7fc438c4c000 1 Processor -- start 2023-11-06T02:11:12.371 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.358+0000 7fc438c4c000 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f8ed58260 0x557f8eda3040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:12.371 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.358+0000 7fc438c4c000 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f8eda2130 0x557f8eda3580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:12.371 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.358+0000 7fc438c4c000 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557f8eda2a50 0x557f8eda3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:12.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.358+0000 7fc438c4c000 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f8eba72a0 con 0x557f8ed58260 2023-11-06T02:11:12.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.358+0000 7fc438c4c000 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f8eb46470 con 0x557f8eda2130 2023-11-06T02:11:12.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.358+0000 7fc438c4c000 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x557f8eb64660 con 0x557f8eda2a50 2023-11-06T02:11:12.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.358+0000 7fc438c4c000 10 client.0 _ll_register_callbacks cb 0x557f8ed43c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T02:11:12.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.358+0000 7fc42f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f8ed58260 0x557f8eda3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:12.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.358+0000 7fc42f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f8ed58260 0x557f8eda3040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42472/0 (socket says 172.21.15.49:42472) 2023-11-06T02:11:12.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.358+0000 7fc42f7fe640 1 -- 172.21.15.49:0/4147535956 learned_addr learned my addr 172.21.15.49:0/4147535956 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:12.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.358+0000 7fc42ffff640 1 --2- 172.21.15.49:0/4147535956 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557f8eda2a50 0x557f8eda3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:12.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.358+0000 7fc42effd640 1 --2- 172.21.15.49:0/4147535956 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f8eda2130 0x557f8eda3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:12.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.358+0000 7fc42f7fe640 1 -- 172.21.15.49:0/4147535956 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557f8eda2a50 msgr2=0x557f8eda3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:12.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.358+0000 7fc42f7fe640 1 --2- 172.21.15.49:0/4147535956 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557f8eda2a50 0x557f8eda3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:12.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.358+0000 7fc42f7fe640 1 -- 172.21.15.49:0/4147535956 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f8eda2130 msgr2=0x557f8eda3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:12.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.358+0000 7fc42f7fe640 1 --2- 172.21.15.49:0/4147535956 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f8eda2130 0x557f8eda3580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:12.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.358+0000 7fc42f7fe640 1 -- 172.21.15.49:0/4147535956 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc430071120 con 0x557f8ed58260 2023-11-06T02:11:12.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.358+0000 7fc42f7fe640 1 --2- 172.21.15.49:0/4147535956 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f8ed58260 0x557f8eda3040 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fc42000fda0 tx=0x7fc420004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff6e1f1aafc3a6f3 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:12.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.358+0000 7fc40e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-11-06T02:11:12.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.358+0000 7fc40e7fc640 1 -- 172.21.15.49:0/4147535956 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc420008e50 con 0x557f8ed58260 2023-11-06T02:11:12.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.358+0000 7fc40e7fc640 1 -- 172.21.15.49:0/4147535956 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc42001d430 con 0x557f8ed58260 2023-11-06T02:11:12.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.358+0000 7fc438c4c000 1 -- 172.21.15.49:0/4147535956 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557f8eb640a0 con 0x557f8ed58260 2023-11-06T02:11:12.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.358+0000 7fc40e7fc640 1 -- 172.21.15.49:0/4147535956 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc42001d5f0 con 0x557f8ed58260 2023-11-06T02:11:12.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.358+0000 7fc438c4c000 10 client.6123 fetch_fsmap learned FSMap version 23 2023-11-06T02:11:12.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.358+0000 7fc438c4c000 1 -- 172.21.15.49:0/4147535956 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x557f8eb6c1d0 con 0x557f8ed58260 2023-11-06T02:11:12.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.358+0000 7fc40e7fc640 1 -- 172.21.15.49:0/4147535956 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557f8eb640a0 con 0x557f8ed58260 2023-11-06T02:11:12.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.358+0000 7fc40e7fc640 1 -- 172.21.15.49:0/4147535956 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 23) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fc420023020 con 0x557f8ed58260 2023-11-06T02:11:12.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.358+0000 7fc438c4c000 10 client.6123 fetch_fsmap finished waiting for FSMap version 23 2023-11-06T02:11:12.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.358+0000 7fc438c4c000 10 client.6123 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2023-11-06T02:11:12.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.358+0000 7fc438c4c000 20 client.6123 populate_metadata read hostname 'smithi049' 2023-11-06T02:11:12.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.358+0000 7fc438c4c000 1 --2- 172.21.15.49:0/4147535956 >> [v2:172.21.15.49:6834/73939548,v1:172.21.15.49:6835/73939548] conn(0x557f8eda44d0 0x557f8edb3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:12.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.358+0000 7fc438c4c000 4 client.6123 mds_command: new command op to 5877 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T02:11:12.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.358+0000 7fc438c4c000 1 -- 172.21.15.49:0/4147535956 --> [v2:172.21.15.49:6834/73939548,v1:172.21.15.49:6835/73939548] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557f8eba5bf0 con 0x557f8eda44d0 2023-11-06T02:11:12.378 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.358+0000 7fc42effd640 1 --2- 172.21.15.49:0/4147535956 >> [v2:172.21.15.49:6834/73939548,v1:172.21.15.49:6835/73939548] conn(0x557f8eda44d0 0x557f8edb3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:12.378 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.358+0000 7fc42effd640 1 --2- 172.21.15.49:0/4147535956 >> [v2:172.21.15.49:6834/73939548,v1:172.21.15.49:6835/73939548] conn(0x557f8eda44d0 0x557f8edb3f10 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7a4cc9d6a5b8e2f server_cookie=f7dd569ee9523758 in_seq=0 out_seq=0 2023-11-06T02:11:12.378 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.358+0000 7fc40e7fc640 10 client.6123 ms_handle_connect on v2:172.21.15.49:6834/73939548 2023-11-06T02:11:12.378 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.358+0000 7fc40e7fc640 1 -- 172.21.15.49:0/4147535956 <== mds.0 v2:172.21.15.49:6834/73939548 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x557f8eba5bf0 con 0x557f8eda44d0 2023-11-06T02:11:12.378 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.358+0000 7fc40e7fc640 10 client.6123 handle_command_reply: tid=0 2023-11-06T02:11:12.379 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.358+0000 7fc438c4c000 1 client.6123 shutdown 2023-11-06T02:11:12.379 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.362+0000 7fc438c4c000 1 -- 172.21.15.49:0/4147535956 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f8ed58260 msgr2=0x557f8eda3040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:12.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.362+0000 7fc438c4c000 1 --2- 172.21.15.49:0/4147535956 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f8ed58260 0x557f8eda3040 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fc42000fda0 tx=0x7fc420004a50 comp rx=0 tx=0).stop 2023-11-06T02:11:12.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.362+0000 7fc438c4c000 1 -- 172.21.15.49:0/4147535956 shutdown_connections 2023-11-06T02:11:12.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.362+0000 7fc438c4c000 1 --2- 172.21.15.49:0/4147535956 >> [v2:172.21.15.49:6834/73939548,v1:172.21.15.49:6835/73939548] conn(0x557f8eda44d0 0x557f8edb3f10 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:12.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.362+0000 7fc438c4c000 1 --2- 172.21.15.49:0/4147535956 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557f8eda2a50 0x557f8eda3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:12.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.362+0000 7fc438c4c000 1 --2- 172.21.15.49:0/4147535956 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f8eda2130 0x557f8eda3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:12.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.362+0000 7fc438c4c000 1 --2- 172.21.15.49:0/4147535956 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f8ed58260 0x557f8eda3040 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:12.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.362+0000 7fc40e7fc640 0 client.6123 ms_handle_reset on v2:172.21.15.49:6834/73939548 2023-11-06T02:11:12.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.362+0000 7fc438c4c000 1 -- 172.21.15.49:0/4147535956 >> 172.21.15.49:0/4147535956 conn(0x557f8ed5e9f0 msgr2=0x557f8ed56d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:12.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.362+0000 7fc438c4c000 1 -- 172.21.15.49:0/4147535956 shutdown_connections 2023-11-06T02:11:12.382 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.362+0000 7fc438c4c000 1 -- 172.21.15.49:0/4147535956 wait complete. 2023-11-06T02:11:12.382 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.362+0000 7fc438c4c000 20 client.6123 trim_cache size 0 max 16384 2023-11-06T02:11:12.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.374+0000 7fc438c4c000 1 Processor -- start 2023-11-06T02:11:12.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.374+0000 7fc438c4c000 1 -- start start 2023-11-06T02:11:12.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.374+0000 7fc438c4c000 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f8ed58260 0x557f8edb5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:12.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.374+0000 7fc438c4c000 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557f8eda2130 0x557f8edb4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:12.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.374+0000 7fc438c4c000 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f8eda2a50 0x557f8edb5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:12.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.374+0000 7fc438c4c000 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f8e864aa0 con 0x557f8ed58260 2023-11-06T02:11:12.389 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.374+0000 7fc438c4c000 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f8eba2c10 con 0x557f8eda2a50 2023-11-06T02:11:12.389 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.374+0000 7fc438c4c000 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x557f8ec4bbc0 con 0x557f8eda2130 2023-11-06T02:11:12.389 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.374+0000 7fc42f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f8ed58260 0x557f8edb5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:12.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.374+0000 7fc42f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f8ed58260 0x557f8edb5c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42476/0 (socket says 172.21.15.49:42476) 2023-11-06T02:11:12.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.374+0000 7fc42f7fe640 1 -- 172.21.15.49:0/2323682951 learned_addr learned my addr 172.21.15.49:0/2323682951 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:12.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.378+0000 7fc42effd640 1 --2- 172.21.15.49:0/2323682951 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557f8eda2130 0x557f8edb4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:12.391 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.378+0000 7fc42ffff640 1 --2- 172.21.15.49:0/2323682951 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f8eda2a50 0x557f8edb5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:12.391 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.378+0000 7fc42f7fe640 1 -- 172.21.15.49:0/2323682951 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557f8eda2130 msgr2=0x557f8edb4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:12.391 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.378+0000 7fc42f7fe640 1 --2- 172.21.15.49:0/2323682951 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557f8eda2130 0x557f8edb4f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:12.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.378+0000 7fc42f7fe640 1 -- 172.21.15.49:0/2323682951 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f8eda2a50 msgr2=0x557f8edb5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:12.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.378+0000 7fc42f7fe640 1 --2- 172.21.15.49:0/2323682951 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f8eda2a50 0x557f8edb5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:12.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.378+0000 7fc42f7fe640 1 -- 172.21.15.49:0/2323682951 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f8eb52520 con 0x557f8ed58260 2023-11-06T02:11:12.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.378+0000 7fc42ffff640 1 --2- 172.21.15.49:0/2323682951 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f8eda2a50 0x557f8edb5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:11:12.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.378+0000 7fc42f7fe640 1 --2- 172.21.15.49:0/2323682951 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f8ed58260 0x557f8edb5c60 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fc42000f810 tx=0x7fc420017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=553ac2f0b4bf0ab0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:12.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.378+0000 7fc425ffb640 1 -- 172.21.15.49:0/2323682951 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc42001d7f0 con 0x557f8ed58260 2023-11-06T02:11:12.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.378+0000 7fc425ffb640 1 -- 172.21.15.49:0/2323682951 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc42001d9d0 con 0x557f8ed58260 2023-11-06T02:11:12.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.378+0000 7fc425ffb640 1 -- 172.21.15.49:0/2323682951 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc42001d630 con 0x557f8ed58260 2023-11-06T02:11:12.395 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.378+0000 7fc438c4c000 1 -- 172.21.15.49:0/2323682951 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f8ed58260 msgr2=0x557f8edb5c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:12.395 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.378+0000 7fc438c4c000 1 --2- 172.21.15.49:0/2323682951 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f8ed58260 0x557f8edb5c60 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fc42000f810 tx=0x7fc420017000 comp rx=0 tx=0).stop 2023-11-06T02:11:12.395 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.378+0000 7fc438c4c000 1 -- 172.21.15.49:0/2323682951 shutdown_connections 2023-11-06T02:11:12.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.378+0000 7fc438c4c000 1 --2- 172.21.15.49:0/2323682951 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f8eda2a50 0x557f8edb5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:12.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.378+0000 7fc438c4c000 1 --2- 172.21.15.49:0/2323682951 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557f8eda2130 0x557f8edb4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:12.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.378+0000 7fc438c4c000 1 --2- 172.21.15.49:0/2323682951 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f8ed58260 0x557f8edb5c60 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:12.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.378+0000 7fc438c4c000 1 -- 172.21.15.49:0/2323682951 >> 172.21.15.49:0/2323682951 conn(0x557f8ed5e9f0 msgr2=0x557f8ed5d7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:12.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.378+0000 7fc438c4c000 1 -- 172.21.15.49:0/2323682951 shutdown_connections 2023-11-06T02:11:12.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.378+0000 7fc438c4c000 1 -- 172.21.15.49:0/2323682951 wait complete. 2023-11-06T02:11:12.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.378+0000 7fc438c4c000 1 -- start start 2023-11-06T02:11:12.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.378+0000 7fc438c4c000 1 Processor -- start 2023-11-06T02:11:12.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.378+0000 7fc438c4c000 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f8ed58260 0x557f8edff7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:12.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.378+0000 7fc438c4c000 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557f8eda2130 0x557f8edffd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:12.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.378+0000 7fc438c4c000 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f8eda2a50 0x557f8ede3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:12.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.378+0000 7fc438c4c000 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f8eba6130 con 0x557f8ed58260 2023-11-06T02:11:12.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.378+0000 7fc438c4c000 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f8ebab2a0 con 0x557f8eda2a50 2023-11-06T02:11:12.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.378+0000 7fc438c4c000 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x557f8ec42330 con 0x557f8eda2130 2023-11-06T02:11:12.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.378+0000 7fc438c4c000 10 client.0 _ll_register_callbacks cb 0x557f8eb6bcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T02:11:12.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.378+0000 7fc42f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f8ed58260 0x557f8edff7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:12.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.378+0000 7fc42f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f8ed58260 0x557f8edff7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42482/0 (socket says 172.21.15.49:42482) 2023-11-06T02:11:12.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.378+0000 7fc42f7fe640 1 -- 172.21.15.49:0/3297205106 learned_addr learned my addr 172.21.15.49:0/3297205106 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:12.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.378+0000 7fc42ffff640 1 --2- 172.21.15.49:0/3297205106 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f8eda2a50 0x557f8ede3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:12.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.378+0000 7fc42effd640 1 --2- 172.21.15.49:0/3297205106 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557f8eda2130 0x557f8edffd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:12.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.378+0000 7fc42f7fe640 1 -- 172.21.15.49:0/3297205106 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557f8eda2130 msgr2=0x557f8edffd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:12.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.378+0000 7fc42f7fe640 1 --2- 172.21.15.49:0/3297205106 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557f8eda2130 0x557f8edffd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:12.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.378+0000 7fc42f7fe640 1 -- 172.21.15.49:0/3297205106 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f8eda2a50 msgr2=0x557f8ede3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:12.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.378+0000 7fc42f7fe640 1 --2- 172.21.15.49:0/3297205106 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f8eda2a50 0x557f8ede3860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:12.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.378+0000 7fc42f7fe640 1 -- 172.21.15.49:0/3297205106 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557f8eb6c1d0 con 0x557f8ed58260 2023-11-06T02:11:12.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.378+0000 7fc42ffff640 1 --2- 172.21.15.49:0/3297205106 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f8eda2a50 0x557f8ede3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:11:12.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.378+0000 7fc42effd640 1 --2- 172.21.15.49:0/3297205106 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557f8eda2130 0x557f8edffd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:11:12.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.378+0000 7fc42f7fe640 1 --2- 172.21.15.49:0/3297205106 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f8ed58260 0x557f8edff7d0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fc42000db70 tx=0x7fc42001e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2062f4dc7fe841c2 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:12.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.378+0000 7fc40e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-11-06T02:11:12.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.378+0000 7fc40e7fc640 1 -- 172.21.15.49:0/3297205106 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc42001dd40 con 0x557f8ed58260 2023-11-06T02:11:12.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.378+0000 7fc40e7fc640 1 -- 172.21.15.49:0/3297205106 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc42003a460 con 0x557f8ed58260 2023-11-06T02:11:12.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.378+0000 7fc438c4c000 1 -- 172.21.15.49:0/3297205106 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557f8eb70620 con 0x557f8ed58260 2023-11-06T02:11:12.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.378+0000 7fc40e7fc640 1 -- 172.21.15.49:0/3297205106 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc42001dea0 con 0x557f8ed58260 2023-11-06T02:11:12.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.378+0000 7fc438c4c000 10 client.6129 fetch_fsmap learned FSMap version 23 2023-11-06T02:11:12.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.378+0000 7fc438c4c000 1 -- 172.21.15.49:0/3297205106 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x557f8eb46090 con 0x557f8ed58260 2023-11-06T02:11:12.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.378+0000 7fc40e7fc640 1 -- 172.21.15.49:0/3297205106 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557f8eb70620 con 0x557f8ed58260 2023-11-06T02:11:12.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.382+0000 7fc40e7fc640 1 -- 172.21.15.49:0/3297205106 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 23) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fc420023040 con 0x557f8ed58260 2023-11-06T02:11:12.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.382+0000 7fc438c4c000 10 client.6129 fetch_fsmap finished waiting for FSMap version 23 2023-11-06T02:11:12.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.382+0000 7fc438c4c000 10 client.6129 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2023-11-06T02:11:12.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.382+0000 7fc438c4c000 20 client.6129 populate_metadata read hostname 'smithi049' 2023-11-06T02:11:12.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.382+0000 7fc438c4c000 1 --2- 172.21.15.49:0/3297205106 >> [v2:172.21.15.49:6834/73939548,v1:172.21.15.49:6835/73939548] conn(0x557f8ede3da0 0x557f8ede4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:12.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.382+0000 7fc438c4c000 4 client.6129 mds_command: new command op to 5877 tid=0[{"prefix": "scrub status"}] 2023-11-06T02:11:12.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.382+0000 7fc438c4c000 1 -- 172.21.15.49:0/3297205106 --> [v2:172.21.15.49:6834/73939548,v1:172.21.15.49:6835/73939548] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557f8eba0320 con 0x557f8ede3da0 2023-11-06T02:11:12.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.382+0000 7fc42effd640 1 --2- 172.21.15.49:0/3297205106 >> [v2:172.21.15.49:6834/73939548,v1:172.21.15.49:6835/73939548] conn(0x557f8ede3da0 0x557f8ede4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:12.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.382+0000 7fc40e7fc640 10 client.6129 ms_handle_connect on v2:172.21.15.49:6834/73939548 2023-11-06T02:11:12.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.382+0000 7fc42effd640 1 --2- 172.21.15.49:0/3297205106 >> [v2:172.21.15.49:6834/73939548,v1:172.21.15.49:6835/73939548] conn(0x557f8ede3da0 0x557f8ede4180 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47d2efaa339e9569 server_cookie=6bb6259157e2074a in_seq=0 out_seq=0 2023-11-06T02:11:12.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.382+0000 7fc40e7fc640 1 -- 172.21.15.49:0/3297205106 <== mds.0 v2:172.21.15.49:6834/73939548 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x557f8eba0320 con 0x557f8ede3da0 2023-11-06T02:11:12.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.382+0000 7fc40e7fc640 10 client.6129 handle_command_reply: tid=0 2023-11-06T02:11:12.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.382+0000 7fc438c4c000 1 client.6129 shutdown 2023-11-06T02:11:12.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.382+0000 7fc438c4c000 1 -- 172.21.15.49:0/3297205106 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f8ed58260 msgr2=0x557f8edff7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:12.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.382+0000 7fc438c4c000 1 --2- 172.21.15.49:0/3297205106 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f8ed58260 0x557f8edff7d0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fc42000db70 tx=0x7fc42001e000 comp rx=0 tx=0).stop 2023-11-06T02:11:12.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.382+0000 7fc438c4c000 1 -- 172.21.15.49:0/3297205106 shutdown_connections 2023-11-06T02:11:12.411 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.382+0000 7fc438c4c000 1 --2- 172.21.15.49:0/3297205106 >> [v2:172.21.15.49:6834/73939548,v1:172.21.15.49:6835/73939548] conn(0x557f8ede3da0 0x557f8ede4180 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:12.411 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.382+0000 7fc438c4c000 1 --2- 172.21.15.49:0/3297205106 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f8eda2a50 0x557f8ede3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:12.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.382+0000 7fc438c4c000 1 --2- 172.21.15.49:0/3297205106 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557f8eda2130 0x557f8edffd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:12.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.382+0000 7fc40e7fc640 0 client.6129 ms_handle_reset on v2:172.21.15.49:6834/73939548 2023-11-06T02:11:12.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.382+0000 7fc438c4c000 1 --2- 172.21.15.49:0/3297205106 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f8ed58260 0x557f8edff7d0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:12.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.382+0000 7fc438c4c000 1 -- 172.21.15.49:0/3297205106 >> 172.21.15.49:0/3297205106 conn(0x557f8ed5e9f0 msgr2=0x557f8ed5dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:12.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.382+0000 7fc438c4c000 1 -- 172.21.15.49:0/3297205106 shutdown_connections 2023-11-06T02:11:12.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.382+0000 7fc438c4c000 1 -- 172.21.15.49:0/3297205106 wait complete. 2023-11-06T02:11:12.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.382+0000 7fc438c4c000 20 client.6129 trim_cache size 0 max 16384 2023-11-06T02:11:12.414 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-11-06T02:11:12.415 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "no active scrubs running", 2023-11-06T02:11:12.415 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": {} 2023-11-06T02:11:12.415 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-11-06T02:11:12.417 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.382+0000 7fc435b24640 1 -- 172.21.15.49:0/1635088001 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc410065a60 msgr2=0x7fc410067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:12.418 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.382+0000 7fc435b24640 1 --2- 172.21.15.49:0/1635088001 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc410065a60 0x7fc410067f20 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc43014fcf0 tx=0x7fc420008330 comp rx=0 tx=0).stop 2023-11-06T02:11:12.418 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.382+0000 7fc435b24640 1 -- 172.21.15.49:0/1635088001 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4300c9100 msgr2=0x7fc4300b30a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:12.418 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.382+0000 7fc435b24640 1 --2- 172.21.15.49:0/1635088001 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4300c9100 0x7fc4300b30a0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fc4180011f0 tx=0x7fc41800c9b0 comp rx=0 tx=0).stop 2023-11-06T02:11:12.418 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.382+0000 7fc435b24640 1 -- 172.21.15.49:0/1635088001 shutdown_connections 2023-11-06T02:11:12.419 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.382+0000 7fc435b24640 1 --2- 172.21.15.49:0/1635088001 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc410065a60 0x7fc410067f20 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:12.419 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.382+0000 7fc435b24640 1 --2- 172.21.15.49:0/1635088001 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4300c9100 0x7fc4300b30a0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:12.419 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.382+0000 7fc435b24640 1 --2- 172.21.15.49:0/1635088001 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc430153d50 0x7fc4300b35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:12.419 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.382+0000 7fc435b24640 1 --2- 172.21.15.49:0/1635088001 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4300707b0 0x7fc43014c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:12.419 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.382+0000 7fc435b24640 1 -- 172.21.15.49:0/1635088001 >> 172.21.15.49:0/1635088001 conn(0x7fc43006a370 msgr2=0x7fc4300cc6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:12.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.382+0000 7fc435b24640 1 -- 172.21.15.49:0/1635088001 shutdown_connections 2023-11-06T02:11:12.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.382+0000 7fc435b24640 1 -- 172.21.15.49:0/1635088001 wait complete. 2023-11-06T02:11:12.420 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-06T02:11:12.521 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T02:11:12.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.654+0000 7f44d81b3640 1 Processor -- start 2023-11-06T02:11:12.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.654+0000 7f44d81b3640 1 -- start start 2023-11-06T02:11:12.651 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.654+0000 7f44d81b3640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f44d00c9d50 0x7f44d0071520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:12.651 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.654+0000 7f44d81b3640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f44d006e190 0x7f44d015c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:12.651 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.654+0000 7f44d81b3640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44d0071090 con 0x7f44d0158b30 2023-11-06T02:11:12.652 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.654+0000 7f44d81b3640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44d0071a60 con 0x7f44d00c9d50 2023-11-06T02:11:12.652 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.654+0000 7f44d81b3640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44d015d480 con 0x7f44d006e190 2023-11-06T02:11:12.652 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.654+0000 7f44d574e640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f44d00c9d50 0x7f44d0071520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:12.652 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.654+0000 7f44d574e640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f44d00c9d50 0x7f44d0071520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42496/0 (socket says 172.21.15.49:42496) 2023-11-06T02:11:12.652 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.654+0000 7f44d574e640 1 -- 172.21.15.49:0/2894823708 learned_addr learned my addr 172.21.15.49:0/2894823708 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:12.653 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.654+0000 7f44d5f4f640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f44d0158b30 0x7f44d00c9640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:59974/0 (socket says 172.21.15.49:59974) 2023-11-06T02:11:12.653 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.654+0000 7f44d6750640 1 --2- 172.21.15.49:0/2894823708 >> v2:172.21.15.150:3300/0 conn(0x7f44d006e190 0x7f44d015c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:12.653 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.654+0000 7f44d574e640 1 -- 172.21.15.49:0/2894823708 >> v2:172.21.15.150:3300/0 conn(0x7f44d006e190 msgr2=0x7f44d015c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:12.653 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.654+0000 7f44d574e640 1 --2- 172.21.15.49:0/2894823708 >> v2:172.21.15.150:3300/0 conn(0x7f44d006e190 0x7f44d015c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:12.653 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.654+0000 7f44d574e640 1 -- 172.21.15.49:0/2894823708 >> v1:172.21.15.49:6789/0 conn(0x7f44d0158b30 legacy=0x7f44d00c9640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:12.654 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.654+0000 7f44d574e640 1 -- 172.21.15.49:0/2894823708 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44d0070a40 con 0x7f44d00c9d50 2023-11-06T02:11:12.654 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.654+0000 7f44d4f4d640 1 -- 172.21.15.49:0/2894823708 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2899155885 0 0) 0x7f44d0071090 con 0x7f44d0158b30 2023-11-06T02:11:12.655 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.658+0000 7f44d574e640 1 --2- 172.21.15.49:0/2894823708 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44d00c9d50 0x7f44d0071520 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f44d006f1f0 tx=0x7f44c001ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cae4409b7318a6f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:12.655 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.658+0000 7f44d4f4d640 1 -- 172.21.15.49:0/2894823708 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44c00a89f0 con 0x7f44d00c9d50 2023-11-06T02:11:12.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.658+0000 7f44d4f4d640 1 -- 172.21.15.49:0/2894823708 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44c00a8b50 con 0x7f44d00c9d50 2023-11-06T02:11:12.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.658+0000 7f44d4f4d640 1 -- 172.21.15.49:0/2894823708 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44c00b0440 con 0x7f44d00c9d50 2023-11-06T02:11:12.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.658+0000 7f44d81b3640 1 -- 172.21.15.49:0/2894823708 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44d00c9d50 msgr2=0x7f44d0071520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:12.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.658+0000 7f44d81b3640 1 --2- 172.21.15.49:0/2894823708 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44d00c9d50 0x7f44d0071520 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f44d006f1f0 tx=0x7f44c001ecf0 comp rx=0 tx=0).stop 2023-11-06T02:11:12.657 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.658+0000 7f44d81b3640 1 -- 172.21.15.49:0/2894823708 shutdown_connections 2023-11-06T02:11:12.657 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.658+0000 7f44d81b3640 1 --2- 172.21.15.49:0/2894823708 >> v2:172.21.15.150:3300/0 conn(0x7f44d006e190 0x7f44d015c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:12.657 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.658+0000 7f44d81b3640 1 --2- 172.21.15.49:0/2894823708 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44d00c9d50 0x7f44d0071520 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:12.657 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.658+0000 7f44d81b3640 1 -- 172.21.15.49:0/2894823708 >> 172.21.15.49:0/2894823708 conn(0x7f44d006c0a0 msgr2=0x7f44d006c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:12.657 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.658+0000 7f44d81b3640 1 -- 172.21.15.49:0/2894823708 shutdown_connections 2023-11-06T02:11:12.658 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.658+0000 7f44d81b3640 1 -- 172.21.15.49:0/2894823708 wait complete. 2023-11-06T02:11:12.658 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.658+0000 7f44d81b3640 1 Processor -- start 2023-11-06T02:11:12.658 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.658+0000 7f44d81b3640 1 -- start start 2023-11-06T02:11:12.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.658+0000 7f44d81b3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44d006e190 0x7f44d00b20e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:12.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.658+0000 7f44d81b3640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f44d00c9d50 0x7f44d00b03a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:12.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.658+0000 7f44d81b3640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44d0158b30 0x7f44d00c5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:12.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.658+0000 7f44d81b3640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44d00af340 con 0x7f44d006e190 2023-11-06T02:11:12.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.658+0000 7f44d81b3640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44d00af5e0 con 0x7f44d0158b30 2023-11-06T02:11:12.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.658+0000 7f44d81b3640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44d00af8c0 con 0x7f44d00c9d50 2023-11-06T02:11:12.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.658+0000 7f44d5f4f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44d006e190 0x7f44d00b20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:12.662 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.658+0000 7f44d5f4f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44d006e190 0x7f44d00b20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42500/0 (socket says 172.21.15.49:42500) 2023-11-06T02:11:12.662 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.658+0000 7f44d5f4f640 1 -- 172.21.15.49:0/675050365 learned_addr learned my addr 172.21.15.49:0/675050365 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:12.662 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.658+0000 7f44d5f4f640 1 -- 172.21.15.49:0/675050365 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f44d00c9d50 msgr2=0x7f44d00b03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:12.662 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.658+0000 7f44d5f4f640 1 --2- 172.21.15.49:0/675050365 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f44d00c9d50 0x7f44d00b03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:12.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.658+0000 7f44d6750640 1 --2- 172.21.15.49:0/675050365 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44d0158b30 0x7f44d00c5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:12.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.658+0000 7f44d5f4f640 1 -- 172.21.15.49:0/675050365 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44d0158b30 msgr2=0x7f44d00c5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:12.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.658+0000 7f44d5f4f640 1 --2- 172.21.15.49:0/675050365 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44d0158b30 0x7f44d00c5bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:12.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.658+0000 7f44d5f4f640 1 -- 172.21.15.49:0/675050365 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44c00094d0 con 0x7f44d006e190 2023-11-06T02:11:12.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.658+0000 7f44d5f4f640 1 --2- 172.21.15.49:0/675050365 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44d006e190 0x7f44d00b20e0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f44cc00c8c0 tx=0x7f44cc00cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:12.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.662+0000 7f44c67fc640 1 -- 172.21.15.49:0/675050365 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44cc00a5e0 con 0x7f44d006e190 2023-11-06T02:11:12.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.662+0000 7f44c67fc640 1 -- 172.21.15.49:0/675050365 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44cc00a760 con 0x7f44d006e190 2023-11-06T02:11:12.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.662+0000 7f44d81b3640 1 -- 172.21.15.49:0/675050365 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44d00afba0 con 0x7f44d006e190 2023-11-06T02:11:12.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.662+0000 7f44c67fc640 1 -- 172.21.15.49:0/675050365 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44cc00ab50 con 0x7f44d006e190 2023-11-06T02:11:12.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.662+0000 7f44d81b3640 1 -- 172.21.15.49:0/675050365 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44d0148cd0 con 0x7f44d006e190 2023-11-06T02:11:12.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.662+0000 7f44d81b3640 1 -- 172.21.15.49:0/675050365 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44d00b00c0 con 0x7f44d006e190 2023-11-06T02:11:12.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.662+0000 7f44c67fc640 1 -- 172.21.15.49:0/675050365 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f44cc00acb0 con 0x7f44d006e190 2023-11-06T02:11:12.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.662+0000 7f44c67fc640 1 --2- 172.21.15.49:0/675050365 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f44b4065a60 0x7f44b4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:12.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.662+0000 7f44c67fc640 1 -- 172.21.15.49:0/675050365 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f44cc082f60 con 0x7f44d006e190 2023-11-06T02:11:12.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.662+0000 7f44d574e640 1 --2- 172.21.15.49:0/675050365 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f44b4065a60 0x7f44b4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:12.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.662+0000 7f44d574e640 1 --2- 172.21.15.49:0/675050365 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f44b4065a60 0x7f44b4067f20 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f44c00b2000 tx=0x7f44c0004740 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:12.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.666+0000 7f44c67fc640 1 -- 172.21.15.49:0/675050365 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f44cc050830 con 0x7f44d006e190 2023-11-06T02:11:12.852 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.854+0000 7f44d81b3640 1 -- 172.21.15.49:0/675050365 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f44d0074040 con 0x7f44d006e190 2023-11-06T02:11:12.852 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.858+0000 7f44c67fc640 1 -- 172.21.15.49:0/675050365 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4648 (secure 0 0 0) 0x7f44cc017020 con 0x7f44d006e190 2023-11-06T02:11:12.853 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:11:12.853 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5096,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1396196190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1396196190},{"type":"v1","addr":"172.21.15.150:6835","nonce":1396196190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5102,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/2465209170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":2465209170},{"type":"v1","addr":"172.21.15.150:6837","nonce":2465209170}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5871,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/1006951804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1006951804},{"type":"v1","addr":"172.21.15.49:6833","nonce":1006951804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T02:10:31.903864+0000","modified":"2023-11-06T02:10:34.930237+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5877},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5877":{"gid":5877,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.49:6835/73939548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":73939548},{"type":"v1","addr":"172.21.15.49:6835","nonce":73939548}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-06T02:11:12.853 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 23 2023-11-06T02:11:12.855 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.858+0000 7f44d81b3640 1 -- 172.21.15.49:0/675050365 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f44b4065a60 msgr2=0x7f44b4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:12.855 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.858+0000 7f44d81b3640 1 --2- 172.21.15.49:0/675050365 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f44b4065a60 0x7f44b4067f20 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f44c00b2000 tx=0x7f44c0004740 comp rx=0 tx=0).stop 2023-11-06T02:11:12.856 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.858+0000 7f44d81b3640 1 -- 172.21.15.49:0/675050365 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44d006e190 msgr2=0x7f44d00b20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:12.856 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.858+0000 7f44d81b3640 1 --2- 172.21.15.49:0/675050365 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44d006e190 0x7f44d00b20e0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f44cc00c8c0 tx=0x7f44cc00cd80 comp rx=0 tx=0).stop 2023-11-06T02:11:12.856 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.858+0000 7f44d81b3640 1 -- 172.21.15.49:0/675050365 shutdown_connections 2023-11-06T02:11:12.857 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.858+0000 7f44d81b3640 1 --2- 172.21.15.49:0/675050365 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f44b4065a60 0x7f44b4067f20 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:12.857 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.858+0000 7f44d81b3640 1 --2- 172.21.15.49:0/675050365 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44d0158b30 0x7f44d00c5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:12.857 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.858+0000 7f44d81b3640 1 --2- 172.21.15.49:0/675050365 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f44d00c9d50 0x7f44d00b03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:12.858 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.858+0000 7f44d81b3640 1 --2- 172.21.15.49:0/675050365 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44d006e190 0x7f44d00b20e0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:12.858 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.858+0000 7f44d81b3640 1 -- 172.21.15.49:0/675050365 >> 172.21.15.49:0/675050365 conn(0x7f44d006c0a0 msgr2=0x7f44d00cdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:12.859 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.858+0000 7f44d81b3640 1 -- 172.21.15.49:0/675050365 shutdown_connections 2023-11-06T02:11:12.859 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:12.858+0000 7f44d81b3640 1 -- 172.21.15.49:0/675050365 wait complete. 2023-11-06T02:11:12.873 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2023-11-06T02:11:13.017 INFO:teuthology.orchestra.run.smithi049.stdout:{"message":"","return_code":0} 2023-11-06T02:11:13.018 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2023-11-06T02:11:13.019 DEBUG:teuthology.orchestra.run.smithi049:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2023-11-06T02:11:13.057 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.062+0000 7fa8b879b080 1 Processor -- start 2023-11-06T02:11:13.057 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.062+0000 7fa8b879b080 1 -- start start 2023-11-06T02:11:13.060 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.062+0000 7fa8b879b080 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563949cbeaf0 0x563949ca56b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:13.060 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.062+0000 7fa8b879b080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563949cbeed0 0x563949d46d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:13.060 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.062+0000 7fa8b879b080 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563949d4b120 0x563949ca43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:13.061 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.062+0000 7fa8b879b080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x563949bba450 con 0x563949cbeed0 2023-11-06T02:11:13.061 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.062+0000 7fa8b879b080 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x563949bba150 con 0x563949cbeaf0 2023-11-06T02:11:13.061 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.062+0000 7fa8b879b080 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x563949bb0dc0 con 0x563949d4b120 2023-11-06T02:11:13.061 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.062+0000 7fa8b5ab4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563949cbeed0 0x563949d46d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:13.061 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.062+0000 7fa8b5ab4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563949cbeed0 0x563949d46d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42520/0 (socket says 172.21.15.49:42520) 2023-11-06T02:11:13.062 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.062+0000 7fa8b5ab4640 1 -- 172.21.15.49:0/144700335 learned_addr learned my addr 172.21.15.49:0/144700335 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:13.062 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.062+0000 7fa8b6ab6640 1 --2- 172.21.15.49:0/144700335 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563949d4b120 0x563949ca43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:13.062 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.062+0000 7fa8b62b5640 1 --2- 172.21.15.49:0/144700335 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563949cbeaf0 0x563949ca56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:13.062 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.062+0000 7fa8b5ab4640 1 -- 172.21.15.49:0/144700335 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563949d4b120 msgr2=0x563949ca43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:13.062 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.062+0000 7fa8b5ab4640 1 --2- 172.21.15.49:0/144700335 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563949d4b120 0x563949ca43b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:13.063 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.062+0000 7fa8b5ab4640 1 -- 172.21.15.49:0/144700335 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563949cbeaf0 msgr2=0x563949ca56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:13.063 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.062+0000 7fa8b5ab4640 1 --2- 172.21.15.49:0/144700335 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563949cbeaf0 0x563949ca56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:13.063 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.062+0000 7fa8b5ab4640 1 -- 172.21.15.49:0/144700335 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8b000a1c0 con 0x563949cbeed0 2023-11-06T02:11:13.063 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.062+0000 7fa8b62b5640 1 --2- 172.21.15.49:0/144700335 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563949cbeaf0 0x563949ca56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:11:13.063 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.062+0000 7fa8b5ab4640 1 --2- 172.21.15.49:0/144700335 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563949cbeed0 0x563949d46d00 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fa8a000a7e0 tx=0x7fa8a000aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fdc1830560031da server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:13.064 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.062+0000 7fa8b4ab2640 1 -- 172.21.15.49:0/144700335 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8a000f070 con 0x563949cbeed0 2023-11-06T02:11:13.064 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.062+0000 7fa8b4ab2640 1 -- 172.21.15.49:0/144700335 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8a0008ba0 con 0x563949cbeed0 2023-11-06T02:11:13.064 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.062+0000 7fa8b4ab2640 1 -- 172.21.15.49:0/144700335 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8a00133f0 con 0x563949cbeed0 2023-11-06T02:11:13.064 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.062+0000 7fa8b879b080 1 -- 172.21.15.49:0/144700335 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563949cbeed0 msgr2=0x563949d46d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:13.064 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.062+0000 7fa8b879b080 1 --2- 172.21.15.49:0/144700335 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563949cbeed0 0x563949d46d00 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fa8a000a7e0 tx=0x7fa8a000aca0 comp rx=0 tx=0).stop 2023-11-06T02:11:13.065 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.062+0000 7fa8b879b080 1 -- 172.21.15.49:0/144700335 shutdown_connections 2023-11-06T02:11:13.065 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.062+0000 7fa8b879b080 1 --2- 172.21.15.49:0/144700335 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563949cbeed0 0x563949d46d00 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:13.065 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.062+0000 7fa8b879b080 1 --2- 172.21.15.49:0/144700335 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563949d4b120 0x563949ca43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:13.065 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.062+0000 7fa8b879b080 1 --2- 172.21.15.49:0/144700335 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563949cbeaf0 0x563949ca56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:13.066 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.062+0000 7fa8b879b080 1 -- 172.21.15.49:0/144700335 >> 172.21.15.49:0/144700335 conn(0x563949ca20e0 msgr2=0x563949d27b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:13.066 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.062+0000 7fa8b879b080 1 -- 172.21.15.49:0/144700335 shutdown_connections 2023-11-06T02:11:13.066 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.062+0000 7fa8b879b080 1 -- 172.21.15.49:0/144700335 wait complete. 2023-11-06T02:11:13.066 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.062+0000 7fa8b879b080 1 Processor -- start 2023-11-06T02:11:13.066 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.062+0000 7fa8b879b080 1 -- start start 2023-11-06T02:11:13.070 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.062+0000 7fa8b879b080 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563949cbeaf0 0x563949d29d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:13.070 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.062+0000 7fa8b879b080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563949cbeed0 0x563949d45730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:13.071 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.062+0000 7fa8b879b080 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563949d4b120 0x563949ca4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:13.071 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.062+0000 7fa8b879b080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x563949d26b80 con 0x563949cbeed0 2023-11-06T02:11:13.071 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.062+0000 7fa8b879b080 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x563949d45f20 con 0x563949cbeaf0 2023-11-06T02:11:13.072 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.062+0000 7fa8b879b080 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x563949d461c0 con 0x563949d4b120 2023-11-06T02:11:13.072 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.066+0000 7fa8b5ab4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563949cbeed0 0x563949d45730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:13.072 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.066+0000 7fa8b5ab4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563949cbeed0 0x563949d45730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42528/0 (socket says 172.21.15.49:42528) 2023-11-06T02:11:13.072 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.066+0000 7fa8b5ab4640 1 -- 172.21.15.49:0/1488959986 learned_addr learned my addr 172.21.15.49:0/1488959986 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:13.073 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.066+0000 7fa8b62b5640 1 --2- 172.21.15.49:0/1488959986 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563949cbeaf0 0x563949d29d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:13.073 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.066+0000 7fa8b6ab6640 1 --2- 172.21.15.49:0/1488959986 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563949d4b120 0x563949ca4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:13.073 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.066+0000 7fa8b5ab4640 1 -- 172.21.15.49:0/1488959986 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563949d4b120 msgr2=0x563949ca4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:13.073 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.066+0000 7fa8b5ab4640 1 --2- 172.21.15.49:0/1488959986 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563949d4b120 0x563949ca4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:13.074 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.066+0000 7fa8b5ab4640 1 -- 172.21.15.49:0/1488959986 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563949cbeaf0 msgr2=0x563949d29d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:13.074 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.066+0000 7fa8b5ab4640 1 --2- 172.21.15.49:0/1488959986 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563949cbeaf0 0x563949d29d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:13.074 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.066+0000 7fa8b5ab4640 1 -- 172.21.15.49:0/1488959986 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8a000a430 con 0x563949cbeed0 2023-11-06T02:11:13.075 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.066+0000 7fa8b5ab4640 1 --2- 172.21.15.49:0/1488959986 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563949cbeed0 0x563949d45730 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fa8a00087e0 tx=0x7fa8a0003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:13.075 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.066+0000 7fa8af7fe640 1 -- 172.21.15.49:0/1488959986 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8a0026660 con 0x563949cbeed0 2023-11-06T02:11:13.075 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.066+0000 7fa8af7fe640 1 -- 172.21.15.49:0/1488959986 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8a00267e0 con 0x563949cbeed0 2023-11-06T02:11:13.075 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.066+0000 7fa8b879b080 1 -- 172.21.15.49:0/1488959986 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x563949bc8ff0 con 0x563949cbeed0 2023-11-06T02:11:13.076 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.066+0000 7fa8af7fe640 1 -- 172.21.15.49:0/1488959986 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8a0026b90 con 0x563949cbeed0 2023-11-06T02:11:13.076 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.066+0000 7fa8b879b080 1 -- 172.21.15.49:0/1488959986 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563949bbc430 con 0x563949cbeed0 2023-11-06T02:11:13.076 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.066+0000 7fa8af7fe640 1 -- 172.21.15.49:0/1488959986 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fa8a001d410 con 0x563949cbeed0 2023-11-06T02:11:13.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.066+0000 7fa8af7fe640 1 --2- 172.21.15.49:0/1488959986 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa898064440 0x7fa898066900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:13.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.066+0000 7fa8af7fe640 1 -- 172.21.15.49:0/1488959986 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa8a0084d10 con 0x563949cbeed0 2023-11-06T02:11:13.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.066+0000 7fa8b879b080 1 --2- 172.21.15.49:0/1488959986 >> [v2:172.21.15.49:6808/38113,v1:172.21.15.49:6810/38113] conn(0x563949ca4ad0 0x563949d27280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:13.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.066+0000 7fa8b879b080 1 -- 172.21.15.49:0/1488959986 --> [v2:172.21.15.49:6808/38113,v1:172.21.15.49:6810/38113] -- osd_op(unknown.0.0:1 8.7 8:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51) v8 -- 0x563949de2f80 con 0x563949ca4ad0 2023-11-06T02:11:13.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.066+0000 7fa8b62b5640 1 --2- 172.21.15.49:0/1488959986 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa898064440 0x7fa898066900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:13.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.066+0000 7fa8b6ab6640 1 --2- 172.21.15.49:0/1488959986 >> [v2:172.21.15.49:6808/38113,v1:172.21.15.49:6810/38113] conn(0x563949ca4ad0 0x563949d27280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:13.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.066+0000 7fa8b6ab6640 1 --2- 172.21.15.49:0/1488959986 >> [v2:172.21.15.49:6808/38113,v1:172.21.15.49:6810/38113] conn(0x563949ca4ad0 0x563949d27280 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:13.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.066+0000 7fa8b6ab6640 1 -- 172.21.15.49:0/1488959986 <== osd.1 v2:172.21.15.49:6808/38113 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+95 (crc 0 0 0) 0x7fa8b00a8070 con 0x563949ca4ad0 2023-11-06T02:11:13.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.066+0000 7fa8b62b5640 1 --2- 172.21.15.49:0/1488959986 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa898064440 0x7fa898066900 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa8a8000c10 tx=0x7fa8a80016b0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:13.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.066+0000 7fa8b879b080 1 -- 172.21.15.49:0/1488959986 >> [v2:172.21.15.49:6808/38113,v1:172.21.15.49:6810/38113] conn(0x563949ca4ad0 msgr2=0x563949d27280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:13.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.066+0000 7fa8b879b080 1 --2- 172.21.15.49:0/1488959986 >> [v2:172.21.15.49:6808/38113,v1:172.21.15.49:6810/38113] conn(0x563949ca4ad0 0x563949d27280 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:13.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.066+0000 7fa8b879b080 1 -- 172.21.15.49:0/1488959986 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa898064440 msgr2=0x7fa898066900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:13.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.066+0000 7fa8b879b080 1 --2- 172.21.15.49:0/1488959986 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa898064440 0x7fa898066900 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa8a8000c10 tx=0x7fa8a80016b0 comp rx=0 tx=0).stop 2023-11-06T02:11:13.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.066+0000 7fa8b879b080 1 -- 172.21.15.49:0/1488959986 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563949cbeed0 msgr2=0x563949d45730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:13.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.066+0000 7fa8b879b080 1 --2- 172.21.15.49:0/1488959986 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563949cbeed0 0x563949d45730 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fa8a00087e0 tx=0x7fa8a0003030 comp rx=0 tx=0).stop 2023-11-06T02:11:13.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.066+0000 7fa8b6ab6640 1 -- 172.21.15.49:0/1488959986 reap_dead start 2023-11-06T02:11:13.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.070+0000 7fa8b879b080 1 -- 172.21.15.49:0/1488959986 shutdown_connections 2023-11-06T02:11:13.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.070+0000 7fa8b879b080 1 -- 172.21.15.49:0/1488959986 >> 172.21.15.49:0/1488959986 conn(0x563949ca20e0 msgr2=0x563949d4bdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:13.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.070+0000 7fa8b879b080 1 -- 172.21.15.49:0/1488959986 shutdown_connections 2023-11-06T02:11:13.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.070+0000 7fa8b879b080 1 -- 172.21.15.49:0/1488959986 wait complete. 2023-11-06T02:11:13.082 INFO:teuthology.orchestra.run.smithi049.stdout:Yalphaparent_a 2023-11-06T02:11:13.082 DEBUG:teuthology.orchestra.run.smithi049:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2023-11-06T02:11:13.169 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-11-06T02:11:13.169 INFO:teuthology.orchestra.run.smithi049.stdout: "ino": 1099511627777, 2023-11-06T02:11:13.169 INFO:teuthology.orchestra.run.smithi049.stdout: "ancestors": [ 2023-11-06T02:11:13.170 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:11:13.170 INFO:teuthology.orchestra.run.smithi049.stdout: "dirino": 1099511627776, 2023-11-06T02:11:13.170 INFO:teuthology.orchestra.run.smithi049.stdout: "dname": "alpha", 2023-11-06T02:11:13.170 INFO:teuthology.orchestra.run.smithi049.stdout: "version": 4 2023-11-06T02:11:13.170 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:11:13.171 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:11:13.171 INFO:teuthology.orchestra.run.smithi049.stdout: "dirino": 1, 2023-11-06T02:11:13.171 INFO:teuthology.orchestra.run.smithi049.stdout: "dname": "parent_a", 2023-11-06T02:11:13.171 INFO:teuthology.orchestra.run.smithi049.stdout: "version": 7 2023-11-06T02:11:13.171 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:11:13.172 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-11-06T02:11:13.172 INFO:teuthology.orchestra.run.smithi049.stdout: "pool": 8, 2023-11-06T02:11:13.172 INFO:teuthology.orchestra.run.smithi049.stdout: "old_pools": [] 2023-11-06T02:11:13.172 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-11-06T02:11:13.173 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:11:13.174 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-06T02:11:13.174 DEBUG:teuthology.orchestra.run.smithi049:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T02:11:13.267 DEBUG:teuthology.orchestra.run.smithi150:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T02:11:13.322 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-06T02:11:13.323 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-11-06T02:11:13.323 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T02:11:13.330 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T02:11:13.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.510+0000 7fc85a33f640 1 Processor -- start 2023-11-06T02:11:13.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.510+0000 7fc85a33f640 1 -- start start 2023-11-06T02:11:13.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.510+0000 7fc85a33f640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc8541646f0 0x7fc8540b0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:13.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.510+0000 7fc85a33f640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc8540cd9a0 0x7fc8540b05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:13.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.510+0000 7fc85a33f640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc854165b80 con 0x7fc8540c8c50 2023-11-06T02:11:13.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.510+0000 7fc85a33f640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc854165600 con 0x7fc8540cd9a0 2023-11-06T02:11:13.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.510+0000 7fc85a33f640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8541658a0 con 0x7fc8541646f0 2023-11-06T02:11:13.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.510+0000 7fc8588dc640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc8540cd9a0 0x7fc8540b05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:13.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.510+0000 7fc8588dc640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc8540cd9a0 0x7fc8540b05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42534/0 (socket says 172.21.15.49:42534) 2023-11-06T02:11:13.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.510+0000 7fc8588dc640 1 -- 172.21.15.49:0/444750666 learned_addr learned my addr 172.21.15.49:0/444750666 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:13.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.510+0000 7fc853fff640 1 --2- 172.21.15.49:0/444750666 >> v2:172.21.15.150:3300/0 conn(0x7fc8541646f0 0x7fc8540b0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:13.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.510+0000 7fc8588dc640 1 -- 172.21.15.49:0/444750666 >> v2:172.21.15.150:3300/0 conn(0x7fc8541646f0 msgr2=0x7fc8540b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:13.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.510+0000 7fc8588dc640 1 --2- 172.21.15.49:0/444750666 >> v2:172.21.15.150:3300/0 conn(0x7fc8541646f0 0x7fc8540b0080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:13.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.510+0000 7fc8588dc640 1 -- 172.21.15.49:0/444750666 >> v1:172.21.15.150:6789/0 conn(0x7fc8540c8c50 legacy=0x7fc8540cb010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:13.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.510+0000 7fc8588dc640 1 -- 172.21.15.49:0/444750666 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8540c96f0 con 0x7fc8540cd9a0 2023-11-06T02:11:13.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.514+0000 7fc8588dc640 1 --2- 172.21.15.49:0/444750666 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc8540cd9a0 0x7fc8540b05c0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fc85406ceb0 tx=0x7fc84401c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e90a028b7c35e000 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:13.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.514+0000 7fc852ffd640 1 -- 172.21.15.49:0/444750666 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8440aa640 con 0x7fc8540cd9a0 2023-11-06T02:11:13.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.514+0000 7fc852ffd640 1 -- 172.21.15.49:0/444750666 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8440aa7a0 con 0x7fc8540cd9a0 2023-11-06T02:11:13.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.514+0000 7fc852ffd640 1 -- 172.21.15.49:0/444750666 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8440aab30 con 0x7fc8540cd9a0 2023-11-06T02:11:13.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.514+0000 7fc85a33f640 1 -- 172.21.15.49:0/444750666 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc8540cd9a0 msgr2=0x7fc8540b05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:13.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.514+0000 7fc85a33f640 1 --2- 172.21.15.49:0/444750666 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc8540cd9a0 0x7fc8540b05c0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fc85406ceb0 tx=0x7fc84401c4c0 comp rx=0 tx=0).stop 2023-11-06T02:11:13.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.514+0000 7fc85a33f640 1 -- 172.21.15.49:0/444750666 shutdown_connections 2023-11-06T02:11:13.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.514+0000 7fc85a33f640 1 --2- 172.21.15.49:0/444750666 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc8540cd9a0 0x7fc8540b05c0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:13.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.514+0000 7fc85a33f640 1 --2- 172.21.15.49:0/444750666 >> v2:172.21.15.150:3300/0 conn(0x7fc8541646f0 0x7fc8540b0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:13.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.514+0000 7fc85a33f640 1 -- 172.21.15.49:0/444750666 >> 172.21.15.49:0/444750666 conn(0x7fc85406c0a0 msgr2=0x7fc85406c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:13.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.514+0000 7fc85a33f640 1 -- 172.21.15.49:0/444750666 shutdown_connections 2023-11-06T02:11:13.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.514+0000 7fc85a33f640 1 -- 172.21.15.49:0/444750666 wait complete. 2023-11-06T02:11:13.515 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.514+0000 7fc85a33f640 1 Processor -- start 2023-11-06T02:11:13.515 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.514+0000 7fc85a33f640 1 -- start start 2023-11-06T02:11:13.515 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.514+0000 7fc85a33f640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8540c8c50 0x7fc854158150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:13.515 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.514+0000 7fc85a33f640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc8540cd9a0 0x7fc85415d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:13.515 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.514+0000 7fc85a33f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc8541646f0 0x7fc8541622f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:13.516 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.514+0000 7fc85a33f640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc854158690 con 0x7fc8541646f0 2023-11-06T02:11:13.516 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.514+0000 7fc85a33f640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc854159e60 con 0x7fc8540c8c50 2023-11-06T02:11:13.516 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.514+0000 7fc85a33f640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc85415a8d0 con 0x7fc8540cd9a0 2023-11-06T02:11:13.516 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.514+0000 7fc8588dc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc8541646f0 0x7fc8541622f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:13.516 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.518+0000 7fc8588dc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc8541646f0 0x7fc8541622f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42540/0 (socket says 172.21.15.49:42540) 2023-11-06T02:11:13.517 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.518+0000 7fc8588dc640 1 -- 172.21.15.49:0/3175994554 learned_addr learned my addr 172.21.15.49:0/3175994554 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:13.517 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.518+0000 7fc853fff640 1 --2- 172.21.15.49:0/3175994554 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8540c8c50 0x7fc854158150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:13.517 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.518+0000 7fc8537fe640 1 --2- 172.21.15.49:0/3175994554 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc8540cd9a0 0x7fc85415d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:13.517 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.518+0000 7fc8588dc640 1 -- 172.21.15.49:0/3175994554 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc8540cd9a0 msgr2=0x7fc85415d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:13.517 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.518+0000 7fc8588dc640 1 --2- 172.21.15.49:0/3175994554 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc8540cd9a0 0x7fc85415d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:13.518 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.518+0000 7fc8588dc640 1 -- 172.21.15.49:0/3175994554 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8540c8c50 msgr2=0x7fc854158150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:13.518 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.518+0000 7fc8588dc640 1 --2- 172.21.15.49:0/3175994554 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8540c8c50 0x7fc854158150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:13.518 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.518+0000 7fc8588dc640 1 -- 172.21.15.49:0/3175994554 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8440094d0 con 0x7fc8541646f0 2023-11-06T02:11:13.518 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.518+0000 7fc8588dc640 1 --2- 172.21.15.49:0/3175994554 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc8541646f0 0x7fc8541622f0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fc8440ac040 tx=0x7fc84401ad00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:13.518 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.518+0000 7fc850ff9640 1 -- 172.21.15.49:0/3175994554 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8440bd5f0 con 0x7fc8541646f0 2023-11-06T02:11:13.519 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.518+0000 7fc850ff9640 1 -- 172.21.15.49:0/3175994554 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8440bd770 con 0x7fc8541646f0 2023-11-06T02:11:13.519 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.518+0000 7fc85a33f640 1 -- 172.21.15.49:0/3175994554 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc854155b10 con 0x7fc8541646f0 2023-11-06T02:11:13.519 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.518+0000 7fc850ff9640 1 -- 172.21.15.49:0/3175994554 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8440bdb20 con 0x7fc8541646f0 2023-11-06T02:11:13.519 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.518+0000 7fc85a33f640 1 -- 172.21.15.49:0/3175994554 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc85415a5f0 con 0x7fc8541646f0 2023-11-06T02:11:13.519 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.518+0000 7fc85a33f640 1 -- 172.21.15.49:0/3175994554 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc854074040 con 0x7fc8541646f0 2023-11-06T02:11:13.520 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.518+0000 7fc850ff9640 1 -- 172.21.15.49:0/3175994554 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fc8440a8050 con 0x7fc8541646f0 2023-11-06T02:11:13.520 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.518+0000 7fc850ff9640 1 --2- 172.21.15.49:0/3175994554 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc840065a90 0x7fc840067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:13.521 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.518+0000 7fc850ff9640 1 -- 172.21.15.49:0/3175994554 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc84411dcc0 con 0x7fc8541646f0 2023-11-06T02:11:13.521 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.518+0000 7fc853fff640 1 --2- 172.21.15.49:0/3175994554 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc840065a90 0x7fc840067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:13.521 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.522+0000 7fc853fff640 1 --2- 172.21.15.49:0/3175994554 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc840065a90 0x7fc840067f50 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc84c003b90 tx=0x7fc84c009880 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:13.521 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.522+0000 7fc850ff9640 1 -- 172.21.15.49:0/3175994554 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fc8440eb780 con 0x7fc8541646f0 2023-11-06T02:11:13.677 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.682+0000 7fc85a33f640 1 -- 172.21.15.49:0/3175994554 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc8540b4580 con 0x7fc8541646f0 2023-11-06T02:11:13.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.682+0000 7fc850ff9640 1 -- 172.21.15.49:0/3175994554 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+17883 (secure 0 0 0) 0x7fc8440eeda0 con 0x7fc8541646f0 2023-11-06T02:11:13.679 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:11:13.679 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":51,"fsid":"1a8b035c-6ba4-499a-b876-aea47fd9c0d4","created":"2023-11-06T02:08:06.440857+0000","modified":"2023-11-06T02:10:32.908334+0000","last_up_change":"2023-11-06T02:08:17.982372+0000","last_in_change":"2023-11-06T02:08:08.906631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T02:08:19.252222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T02:08:24.776211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-06T02:10:30.458849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-06T02:10:30.899578+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ad55af8-0571-4e6a-8722-b5d429b7496d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6809","nonce":38116},{"type":"v1","addr":"172.21.15.49:6811","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6813","nonce":38116},{"type":"v1","addr":"172.21.15.49:6815","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6821","nonce":38116},{"type":"v1","addr":"172.21.15.49:6823","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6817","nonce":38116},{"type":"v1","addr":"172.21.15.49:6819","nonce":38116}]},"public_addr":"172.21.15.49:6811/38116","cluster_addr":"172.21.15.49:6815/38116","heartbeat_back_addr":"172.21.15.49:6823/38116","heartbeat_front_addr":"172.21.15.49:6819/38116","state":["exists","up"]},{"osd":1,"uuid":"c8b31f85-a757-43ed-9419-351acc46611b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":38113},{"type":"v1","addr":"172.21.15.49:6810","nonce":38113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":38113},{"type":"v1","addr":"172.21.15.49:6814","nonce":38113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":38113},{"type":"v1","addr":"172.21.15.49:6822","nonce":38113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":38113},{"type":"v1","addr":"172.21.15.49:6818","nonce":38113}]},"public_addr":"172.21.15.49:6810/38113","cluster_addr":"172.21.15.49:6814/38113","heartbeat_back_addr":"172.21.15.49:6822/38113","heartbeat_front_addr":"172.21.15.49:6818/38113","state":["exists","up"]},{"osd":2,"uuid":"555b3bf3-0a70-4b91-8efd-e4204b5fc26e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":38114},{"type":"v1","addr":"172.21.15.49:6801","nonce":38114}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":38114},{"type":"v1","addr":"172.21.15.49:6803","nonce":38114}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":38114},{"type":"v1","addr":"172.21.15.49:6807","nonce":38114}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":38114},{"type":"v1","addr":"172.21.15.49:6805","nonce":38114}]},"public_addr":"172.21.15.49:6801/38114","cluster_addr":"172.21.15.49:6803/38114","heartbeat_back_addr":"172.21.15.49:6807/38114","heartbeat_front_addr":"172.21.15.49:6805/38114","state":["exists","up"]},{"osd":3,"uuid":"8d4e5d77-03ef-4b9b-bfcb-5ae93a46795c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":38115},{"type":"v1","addr":"172.21.15.49:6825","nonce":38115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":38115},{"type":"v1","addr":"172.21.15.49:6827","nonce":38115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":38115},{"type":"v1","addr":"172.21.15.49:6831","nonce":38115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":38115},{"type":"v1","addr":"172.21.15.49:6829","nonce":38115}]},"public_addr":"172.21.15.49:6825/38115","cluster_addr":"172.21.15.49:6827/38115","heartbeat_back_addr":"172.21.15.49:6831/38115","heartbeat_front_addr":"172.21.15.49:6829/38115","state":["exists","up"]},{"osd":4,"uuid":"dba03ad4-68a3-4cf3-9218-ac611f02eacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":38417},{"type":"v1","addr":"172.21.15.150:6809","nonce":38417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":38417},{"type":"v1","addr":"172.21.15.150:6811","nonce":38417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":38417},{"type":"v1","addr":"172.21.15.150:6815","nonce":38417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":38417},{"type":"v1","addr":"172.21.15.150:6813","nonce":38417}]},"public_addr":"172.21.15.150:6809/38417","cluster_addr":"172.21.15.150:6811/38417","heartbeat_back_addr":"172.21.15.150:6815/38417","heartbeat_front_addr":"172.21.15.150:6813/38417","state":["exists","up"]},{"osd":5,"uuid":"0df9c303-c255-4640-8e91-01f1c3efe18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":38418},{"type":"v1","addr":"172.21.15.150:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":38418},{"type":"v1","addr":"172.21.15.150:6803","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":38418},{"type":"v1","addr":"172.21.15.150:6807","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":38418},{"type":"v1","addr":"172.21.15.150:6805","nonce":38418}]},"public_addr":"172.21.15.150:6801/38418","cluster_addr":"172.21.15.150:6803/38418","heartbeat_back_addr":"172.21.15.150:6807/38418","heartbeat_front_addr":"172.21.15.150:6805/38418","state":["exists","up"]},{"osd":6,"uuid":"e59fa189-4b70-4c06-9722-c02b90828bf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":38428},{"type":"v1","addr":"172.21.15.150:6825","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":38428},{"type":"v1","addr":"172.21.15.150:6827","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":38428},{"type":"v1","addr":"172.21.15.150:6831","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":38428},{"type":"v1","addr":"172.21.15.150:6829","nonce":38428}]},"public_addr":"172.21.15.150:6825/38428","cluster_addr":"172.21.15.150:6827/38428","heartbeat_back_addr":"172.21.15.150:6831/38428","heartbeat_front_addr":"172.21.15.150:6829/38428","state":["exists","up"]},{"osd":7,"uuid":"a64f57e1-ef98-46bd-b304-385759e95898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":38421},{"type":"v1","addr":"172.21.15.150:6817","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":38421},{"type":"v1","addr":"172.21.15.150:6819","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":38421},{"type":"v1","addr":"172.21.15.150:6823","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":38421},{"type":"v1","addr":"172.21.15.150:6821","nonce":38421}]},"public_addr":"172.21.15.150:6817/38421","cluster_addr":"172.21.15.150:6819/38421","heartbeat_back_addr":"172.21.15.150:6823/38421","heartbeat_front_addr":"172.21.15.150:6821/38421","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.438552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.413280+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.296466+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.424966+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.276953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.230984+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.410235+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.480084+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T02:11:13.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.686+0000 7fc85a33f640 1 -- 172.21.15.49:0/3175994554 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc840065a90 msgr2=0x7fc840067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:13.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.686+0000 7fc85a33f640 1 --2- 172.21.15.49:0/3175994554 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc840065a90 0x7fc840067f50 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc84c003b90 tx=0x7fc84c009880 comp rx=0 tx=0).stop 2023-11-06T02:11:13.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.686+0000 7fc85a33f640 1 -- 172.21.15.49:0/3175994554 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc8541646f0 msgr2=0x7fc8541622f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:13.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.686+0000 7fc85a33f640 1 --2- 172.21.15.49:0/3175994554 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc8541646f0 0x7fc8541622f0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fc8440ac040 tx=0x7fc84401ad00 comp rx=0 tx=0).stop 2023-11-06T02:11:13.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.686+0000 7fc85a33f640 1 -- 172.21.15.49:0/3175994554 shutdown_connections 2023-11-06T02:11:13.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.686+0000 7fc85a33f640 1 --2- 172.21.15.49:0/3175994554 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc840065a90 0x7fc840067f50 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:13.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.686+0000 7fc85a33f640 1 --2- 172.21.15.49:0/3175994554 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc8541646f0 0x7fc8541622f0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:13.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.686+0000 7fc85a33f640 1 --2- 172.21.15.49:0/3175994554 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc8540cd9a0 0x7fc85415d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:13.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.686+0000 7fc85a33f640 1 --2- 172.21.15.49:0/3175994554 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8540c8c50 0x7fc854158150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:13.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.686+0000 7fc85a33f640 1 -- 172.21.15.49:0/3175994554 >> 172.21.15.49:0/3175994554 conn(0x7fc85406c0a0 msgr2=0x7fc854160b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:13.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.686+0000 7fc85a33f640 1 -- 172.21.15.49:0/3175994554 shutdown_connections 2023-11-06T02:11:13.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.686+0000 7fc85a33f640 1 -- 172.21.15.49:0/3175994554 wait complete. 2023-11-06T02:11:13.698 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T02:08:19.252222+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T02:08:24.776211+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T02:10:30.458849+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '50', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T02:10:30.899578+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '51', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T02:11:13.699 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T02:11:13.830 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.834+0000 7f1091ba6640 1 Processor -- start 2023-11-06T02:11:13.830 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.834+0000 7f1091ba6640 1 -- start start 2023-11-06T02:11:13.832 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.834+0000 7f1091ba6640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f108c0cd9b0 0x7f108c0cde40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:13.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.834+0000 7f1091ba6640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f108c0b3720 con 0x7f108c15a9b0 2023-11-06T02:11:13.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.834+0000 7f1091ba6640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f108c0cf690 con 0x7f108c15ad90 2023-11-06T02:11:13.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.834+0000 7f1091ba6640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f108c0b3970 con 0x7f108c0cd9b0 2023-11-06T02:11:13.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.834+0000 7f108b7fe640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f108c15a9b0 0x7f108c0cee80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:59990/0 (socket says 172.21.15.49:59990) 2023-11-06T02:11:13.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.834+0000 7f108b7fe640 1 -- 172.21.15.49:0/536138808 learned_addr learned my addr 172.21.15.49:0/536138808 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:11:13.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.834+0000 7f108bfff640 1 --2- 172.21.15.49:0/536138808 >> v2:172.21.15.150:3300/0 conn(0x7f108c0cd9b0 0x7f108c0cde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:13.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.834+0000 7f108a7fc640 1 -- 172.21.15.49:0/536138808 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1791367397 0 0) 0x7f108c0b3720 con 0x7f108c15a9b0 2023-11-06T02:11:13.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.834+0000 7f108a7fc640 1 -- 172.21.15.49:0/536138808 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1074003530 con 0x7f108c15a9b0 2023-11-06T02:11:13.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.834+0000 7f108a7fc640 1 -- 172.21.15.49:0/536138808 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3974219185 0 0) 0x7f1074003530 con 0x7f108c15a9b0 2023-11-06T02:11:13.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.834+0000 7f108a7fc640 1 -- 172.21.15.49:0/536138808 >> v2:172.21.15.150:3300/0 conn(0x7f108c0cd9b0 msgr2=0x7f108c0cde40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:13.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.834+0000 7f108a7fc640 1 --2- 172.21.15.49:0/536138808 >> v2:172.21.15.150:3300/0 conn(0x7f108c0cd9b0 0x7f108c0cde40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:13.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.834+0000 7f108a7fc640 1 -- 172.21.15.49:0/536138808 >> v1:172.21.15.150:6789/0 conn(0x7f108c15ad90 legacy=0x7f108c0cd2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:13.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.834+0000 7f108a7fc640 1 -- 172.21.15.49:0/536138808 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f108c0b3c50 con 0x7f108c15a9b0 2023-11-06T02:11:13.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.834+0000 7f108a7fc640 1 -- 172.21.15.49:0/536138808 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7f107c001800 con 0x7f108c15a9b0 2023-11-06T02:11:13.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.834+0000 7f108a7fc640 1 -- 172.21.15.49:0/536138808 >> v1:172.21.15.49:6789/0 conn(0x7f108c15a9b0 legacy=0x7f108c0cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:13.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.834+0000 7f108a7fc640 1 --2- 172.21.15.49:0/536138808 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1074003980 0x7f1074003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:13.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.834+0000 7f108a7fc640 1 --2- 172.21.15.49:0/536138808 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10740047d0 0x7f1074004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:13.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.834+0000 7f108a7fc640 1 --2- 172.21.15.49:0/536138808 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1074005190 0x7f107400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:13.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.834+0000 7f108a7fc640 1 -- 172.21.15.49:0/536138808 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f108c0b3970 con 0x7f1074003980 2023-11-06T02:11:13.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.834+0000 7f108a7fc640 1 -- 172.21.15.49:0/536138808 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f107400c030 con 0x7f10740047d0 2023-11-06T02:11:13.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.834+0000 7f108a7fc640 1 -- 172.21.15.49:0/536138808 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f107400c1f0 con 0x7f1074005190 2023-11-06T02:11:13.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.834+0000 7f108b7fe640 1 --2- 172.21.15.49:0/536138808 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1074003980 0x7f1074003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:13.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.834+0000 7f108affd640 1 --2- 172.21.15.49:0/536138808 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10740047d0 0x7f1074004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:13.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.834+0000 7f108b7fe640 1 -- 172.21.15.49:0/536138808 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1074005190 msgr2=0x7f107400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:13.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.834+0000 7f108b7fe640 1 --2- 172.21.15.49:0/536138808 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1074005190 0x7f107400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:13.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.834+0000 7f108b7fe640 1 -- 172.21.15.49:0/536138808 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10740047d0 msgr2=0x7f1074004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:13.839 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.834+0000 7f108b7fe640 1 --2- 172.21.15.49:0/536138808 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10740047d0 0x7f1074004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:13.839 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.834+0000 7f108b7fe640 1 -- 172.21.15.49:0/536138808 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1074003530 con 0x7f1074003980 2023-11-06T02:11:13.839 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.834+0000 7f108affd640 1 --2- 172.21.15.49:0/536138808 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10740047d0 0x7f1074004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:11:13.839 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.834+0000 7f108bfff640 1 -- 172.21.15.49:0/536138808 reap_dead start 2023-11-06T02:11:13.843 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.838+0000 7f108b7fe640 1 --2- 172.21.15.49:0/536138808 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1074003980 0x7f1074003da0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f107c00c240 tx=0x7f107c01cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34cdf458279262e6 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:13.843 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.838+0000 7f108a7fc640 1 -- 172.21.15.49:0/536138808 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f107c007270 con 0x7f1074003980 2023-11-06T02:11:13.843 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.838+0000 7f108a7fc640 1 -- 172.21.15.49:0/536138808 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f107c0073f0 con 0x7f1074003980 2023-11-06T02:11:13.844 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.838+0000 7f108a7fc640 1 -- 172.21.15.49:0/536138808 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f107c0a5860 con 0x7f1074003980 2023-11-06T02:11:13.844 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.838+0000 7f1091ba6640 1 -- 172.21.15.49:0/536138808 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1074003980 msgr2=0x7f1074003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:13.844 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.838+0000 7f1091ba6640 1 --2- 172.21.15.49:0/536138808 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1074003980 0x7f1074003da0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f107c00c240 tx=0x7f107c01cbe0 comp rx=0 tx=0).stop 2023-11-06T02:11:13.844 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.838+0000 7f1091ba6640 1 -- 172.21.15.49:0/536138808 shutdown_connections 2023-11-06T02:11:13.844 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.838+0000 7f1091ba6640 1 --2- 172.21.15.49:0/536138808 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1074003980 0x7f1074003da0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:13.845 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.838+0000 7f1091ba6640 1 -- 172.21.15.49:0/536138808 >> 172.21.15.49:0/536138808 conn(0x7f108c0d08d0 msgr2=0x7f108c0d0cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:13.845 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.838+0000 7f1091ba6640 1 -- 172.21.15.49:0/536138808 shutdown_connections 2023-11-06T02:11:13.845 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.838+0000 7f1091ba6640 1 -- 172.21.15.49:0/536138808 wait complete. 2023-11-06T02:11:13.845 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.838+0000 7f1091ba6640 1 Processor -- start 2023-11-06T02:11:13.846 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.838+0000 7f1091ba6640 1 -- start start 2023-11-06T02:11:13.846 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.842+0000 7f1091ba6640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1074003980 0x7f108c1e3190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:13.846 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.842+0000 7f1091ba6640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f108c14f900 0x7f108c06f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:13.846 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.842+0000 7f1091ba6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f108c06fbe0 0x7f108c15d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:13.846 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.842+0000 7f1091ba6640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f108c070090 con 0x7f108c06fbe0 2023-11-06T02:11:13.847 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.842+0000 7f1091ba6640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f108c1e55e0 con 0x7f108c14f900 2023-11-06T02:11:13.847 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.842+0000 7f1091ba6640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f108c15e020 con 0x7f1074003980 2023-11-06T02:11:13.847 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.842+0000 7f108bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f108c06fbe0 0x7f108c15d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:13.847 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.842+0000 7f108bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f108c06fbe0 0x7f108c15d330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42562/0 (socket says 172.21.15.49:42562) 2023-11-06T02:11:13.848 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.842+0000 7f108bfff640 1 -- 172.21.15.49:0/160730935 learned_addr learned my addr 172.21.15.49:0/160730935 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:13.848 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.842+0000 7f108b7fe640 1 --2- 172.21.15.49:0/160730935 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1074003980 0x7f108c1e3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:13.848 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.842+0000 7f108affd640 1 --2- 172.21.15.49:0/160730935 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f108c14f900 0x7f108c06f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:13.848 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.842+0000 7f108bfff640 1 -- 172.21.15.49:0/160730935 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1074003980 msgr2=0x7f108c1e3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:13.848 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.842+0000 7f108bfff640 1 --2- 172.21.15.49:0/160730935 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1074003980 0x7f108c1e3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:13.849 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.842+0000 7f108bfff640 1 -- 172.21.15.49:0/160730935 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f108c14f900 msgr2=0x7f108c06f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:13.849 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.842+0000 7f108bfff640 1 --2- 172.21.15.49:0/160730935 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f108c14f900 0x7f108c06f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:13.849 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.842+0000 7f108bfff640 1 -- 172.21.15.49:0/160730935 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f108c1e36d0 con 0x7f108c06fbe0 2023-11-06T02:11:13.849 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.842+0000 7f108affd640 1 --2- 172.21.15.49:0/160730935 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f108c14f900 0x7f108c06f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:11:13.850 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.842+0000 7f108bfff640 1 --2- 172.21.15.49:0/160730935 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f108c06fbe0 0x7f108c15d330 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f108c06ab00 tx=0x7f108400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:13.850 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.842+0000 7f1073fff640 1 -- 172.21.15.49:0/160730935 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1084010040 con 0x7f108c06fbe0 2023-11-06T02:11:13.850 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.842+0000 7f1073fff640 1 -- 172.21.15.49:0/160730935 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f108400bb10 con 0x7f108c06fbe0 2023-11-06T02:11:13.850 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.842+0000 7f1091ba6640 1 -- 172.21.15.49:0/160730935 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f108c0c4460 con 0x7f108c06fbe0 2023-11-06T02:11:13.850 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.842+0000 7f1073fff640 1 -- 172.21.15.49:0/160730935 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1084014400 con 0x7f108c06fbe0 2023-11-06T02:11:13.851 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.842+0000 7f1091ba6640 1 -- 172.21.15.49:0/160730935 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f108c0c49e0 con 0x7f108c06fbe0 2023-11-06T02:11:13.851 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.842+0000 7f1091ba6640 1 -- 172.21.15.49:0/160730935 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f108c074040 con 0x7f108c06fbe0 2023-11-06T02:11:13.851 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.842+0000 7f1073fff640 1 -- 172.21.15.49:0/160730935 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f1084014560 con 0x7f108c06fbe0 2023-11-06T02:11:13.851 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.842+0000 7f1073fff640 1 --2- 172.21.15.49:0/160730935 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1078065a90 0x7f1078067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:13.852 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.842+0000 7f1073fff640 1 -- 172.21.15.49:0/160730935 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1084083940 con 0x7f108c06fbe0 2023-11-06T02:11:13.852 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.842+0000 7f108b7fe640 1 --2- 172.21.15.49:0/160730935 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1078065a90 0x7f1078067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:13.852 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.846+0000 7f108b7fe640 1 --2- 172.21.15.49:0/160730935 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1078065a90 0x7f1078067f50 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f107c0ab000 tx=0x7f107c0a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:13.852 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:13.846+0000 7f1073fff640 1 -- 172.21.15.49:0/160730935 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f1084051100 con 0x7f108c06fbe0 2023-11-06T02:11:14.001 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.002+0000 7f1091ba6640 1 -- 172.21.15.49:0/160730935 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f108c0c4000 con 0x7f108c06fbe0 2023-11-06T02:11:14.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.006+0000 7f1073fff640 1 -- 172.21.15.49:0/160730935 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v51) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1084054720 con 0x7f108c06fbe0 2023-11-06T02:11:14.004 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2023-11-06T02:11:14.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.006+0000 7f1091ba6640 1 -- 172.21.15.49:0/160730935 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1078065a90 msgr2=0x7f1078067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:14.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.006+0000 7f1091ba6640 1 --2- 172.21.15.49:0/160730935 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1078065a90 0x7f1078067f50 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f107c0ab000 tx=0x7f107c0a7070 comp rx=0 tx=0).stop 2023-11-06T02:11:14.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.006+0000 7f1091ba6640 1 -- 172.21.15.49:0/160730935 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f108c06fbe0 msgr2=0x7f108c15d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:14.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.006+0000 7f1091ba6640 1 --2- 172.21.15.49:0/160730935 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f108c06fbe0 0x7f108c15d330 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f108c06ab00 tx=0x7f108400ee80 comp rx=0 tx=0).stop 2023-11-06T02:11:14.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.006+0000 7f1091ba6640 1 -- 172.21.15.49:0/160730935 shutdown_connections 2023-11-06T02:11:14.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.006+0000 7f1091ba6640 1 --2- 172.21.15.49:0/160730935 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1078065a90 0x7f1078067f50 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:14.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.006+0000 7f1091ba6640 1 --2- 172.21.15.49:0/160730935 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f108c06fbe0 0x7f108c15d330 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:14.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.006+0000 7f1091ba6640 1 --2- 172.21.15.49:0/160730935 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f108c14f900 0x7f108c06f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:14.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.006+0000 7f1091ba6640 1 --2- 172.21.15.49:0/160730935 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1074003980 0x7f108c1e3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:14.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.006+0000 7f1091ba6640 1 -- 172.21.15.49:0/160730935 >> 172.21.15.49:0/160730935 conn(0x7f108c0d08d0 msgr2=0x7f108c0ce3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:14.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.006+0000 7f1091ba6640 1 -- 172.21.15.49:0/160730935 shutdown_connections 2023-11-06T02:11:14.009 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.006+0000 7f1091ba6640 1 -- 172.21.15.49:0/160730935 wait complete. 2023-11-06T02:11:14.020 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T02:11:14.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.146+0000 7f2016fa0640 1 Processor -- start 2023-11-06T02:11:14.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.146+0000 7f2016fa0640 1 -- start start 2023-11-06T02:11:14.143 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.146+0000 7f2016fa0640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f201015cbb0 0x7f20100d1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:14.144 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.146+0000 7f2016fa0640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f20100d24e0 0x7f20100d2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:14.144 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.146+0000 7f2016fa0640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20100d3400 con 0x7f20100d2fa0 2023-11-06T02:11:14.144 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.146+0000 7f2016fa0640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f201015cf90 con 0x7f201015cbb0 2023-11-06T02:11:14.144 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.146+0000 7f2016fa0640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20100d0b80 con 0x7f20100d24e0 2023-11-06T02:11:14.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.146+0000 7f2014d3c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f201015cbb0 0x7f20100d1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:14.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.146+0000 7f2014d3c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f201015cbb0 0x7f20100d1fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42574/0 (socket says 172.21.15.49:42574) 2023-11-06T02:11:14.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.146+0000 7f2014d3c640 1 -- 172.21.15.49:0/3616270412 learned_addr learned my addr 172.21.15.49:0/3616270412 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:14.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.146+0000 7f2014d3c640 1 -- 172.21.15.49:0/3616270412 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f20100d24e0 msgr2=0x7f20100d2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:14.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.146+0000 7f200ffff640 1 --2- 172.21.15.49:0/3616270412 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f20100d24e0 0x7f20100d2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:14.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.146+0000 7f2014d3c640 1 --2- 172.21.15.49:0/3616270412 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f20100d24e0 0x7f20100d2a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:14.146 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.146+0000 7f2014d3c640 1 -- 172.21.15.49:0/3616270412 >> v1:172.21.15.150:6789/0 conn(0x7f20100d2fa0 legacy=0x7f20100b4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:14.146 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.146+0000 7f2014d3c640 1 -- 172.21.15.49:0/3616270412 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20100d0e60 con 0x7f201015cbb0 2023-11-06T02:11:14.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.150+0000 7f2014d3c640 1 --2- 172.21.15.49:0/3616270412 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f201015cbb0 0x7f20100d1fa0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f2004009870 tx=0x7f200401ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11f8988a67eb4b9f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:14.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.150+0000 7f200f7fe640 1 -- 172.21.15.49:0/3616270412 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20040a8a30 con 0x7f201015cbb0 2023-11-06T02:11:14.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.150+0000 7f200f7fe640 1 -- 172.21.15.49:0/3616270412 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20040a8b90 con 0x7f201015cbb0 2023-11-06T02:11:14.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.150+0000 7f200f7fe640 1 -- 172.21.15.49:0/3616270412 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20040b0440 con 0x7f201015cbb0 2023-11-06T02:11:14.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.150+0000 7f2016fa0640 1 -- 172.21.15.49:0/3616270412 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f201015cbb0 msgr2=0x7f20100d1fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:14.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.150+0000 7f2016fa0640 1 --2- 172.21.15.49:0/3616270412 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f201015cbb0 0x7f20100d1fa0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f2004009870 tx=0x7f200401ed30 comp rx=0 tx=0).stop 2023-11-06T02:11:14.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.150+0000 7f2016fa0640 1 -- 172.21.15.49:0/3616270412 shutdown_connections 2023-11-06T02:11:14.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.150+0000 7f2016fa0640 1 --2- 172.21.15.49:0/3616270412 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f20100d24e0 0x7f20100d2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:14.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.150+0000 7f2016fa0640 1 --2- 172.21.15.49:0/3616270412 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f201015cbb0 0x7f20100d1fa0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:14.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.150+0000 7f2016fa0640 1 -- 172.21.15.49:0/3616270412 >> 172.21.15.49:0/3616270412 conn(0x7f20100718a0 msgr2=0x7f2010071cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:14.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.150+0000 7f2016fa0640 1 -- 172.21.15.49:0/3616270412 shutdown_connections 2023-11-06T02:11:14.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.150+0000 7f2016fa0640 1 -- 172.21.15.49:0/3616270412 wait complete. 2023-11-06T02:11:14.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.150+0000 7f2016fa0640 1 Processor -- start 2023-11-06T02:11:14.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.150+0000 7f2016fa0640 1 -- start start 2023-11-06T02:11:14.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.150+0000 7f2016fa0640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20100d24e0 0x7f20100a9410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:14.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.150+0000 7f2016fa0640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f20100d2fa0 0x7f20101561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:14.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.150+0000 7f2016fa0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f201015cbb0 0x7f2010158cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:14.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.150+0000 7f2016fa0640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20101566e0 con 0x7f201015cbb0 2023-11-06T02:11:14.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.150+0000 7f2016fa0640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2010159230 con 0x7f20100d24e0 2023-11-06T02:11:14.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.150+0000 7f2016fa0640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20100a99f0 con 0x7f20100d2fa0 2023-11-06T02:11:14.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.150+0000 7f201553d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f201015cbb0 0x7f2010158cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:14.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.150+0000 7f201553d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f201015cbb0 0x7f2010158cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42576/0 (socket says 172.21.15.49:42576) 2023-11-06T02:11:14.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.150+0000 7f201553d640 1 -- 172.21.15.49:0/3825107033 learned_addr learned my addr 172.21.15.49:0/3825107033 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:14.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.150+0000 7f200ffff640 1 --2- 172.21.15.49:0/3825107033 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f20100d2fa0 0x7f20101561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:14.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.150+0000 7f2014d3c640 1 --2- 172.21.15.49:0/3825107033 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20100d24e0 0x7f20100a9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:14.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.150+0000 7f201553d640 1 -- 172.21.15.49:0/3825107033 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f20100d2fa0 msgr2=0x7f20101561a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:14.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.150+0000 7f201553d640 1 --2- 172.21.15.49:0/3825107033 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f20100d2fa0 0x7f20101561a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:14.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.150+0000 7f201553d640 1 -- 172.21.15.49:0/3825107033 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20100d24e0 msgr2=0x7f20100a9410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:14.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.150+0000 7f201553d640 1 --2- 172.21.15.49:0/3825107033 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20100d24e0 0x7f20100a9410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:14.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.150+0000 7f201553d640 1 -- 172.21.15.49:0/3825107033 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20040094d0 con 0x7f201015cbb0 2023-11-06T02:11:14.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.154+0000 7f201553d640 1 --2- 172.21.15.49:0/3825107033 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f201015cbb0 0x7f2010158cf0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f200800c7a0 tx=0x7f200800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:14.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.154+0000 7f200d7fa640 1 -- 172.21.15.49:0/3825107033 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2008003ba0 con 0x7f201015cbb0 2023-11-06T02:11:14.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.154+0000 7f200d7fa640 1 -- 172.21.15.49:0/3825107033 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2008003d00 con 0x7f201015cbb0 2023-11-06T02:11:14.156 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.154+0000 7f2016fa0640 1 -- 172.21.15.49:0/3825107033 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20100a9cd0 con 0x7f201015cbb0 2023-11-06T02:11:14.156 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.154+0000 7f200d7fa640 1 -- 172.21.15.49:0/3825107033 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f200800aa30 con 0x7f201015cbb0 2023-11-06T02:11:14.156 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.154+0000 7f2016fa0640 1 -- 172.21.15.49:0/3825107033 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20100aa290 con 0x7f201015cbb0 2023-11-06T02:11:14.156 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.154+0000 7f2016fa0640 1 -- 172.21.15.49:0/3825107033 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20100aa420 con 0x7f201015cbb0 2023-11-06T02:11:14.156 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.154+0000 7f200d7fa640 1 -- 172.21.15.49:0/3825107033 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f200800ab90 con 0x7f201015cbb0 2023-11-06T02:11:14.157 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.154+0000 7f200d7fa640 1 --2- 172.21.15.49:0/3825107033 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1ffc065a60 0x7f1ffc067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:14.157 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.154+0000 7f200d7fa640 1 -- 172.21.15.49:0/3825107033 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2008083140 con 0x7f201015cbb0 2023-11-06T02:11:14.157 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.154+0000 7f2014d3c640 1 --2- 172.21.15.49:0/3825107033 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1ffc065a60 0x7f1ffc067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:14.157 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.154+0000 7f2014d3c640 1 --2- 172.21.15.49:0/3825107033 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1ffc065a60 0x7f1ffc067f20 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f20040b2000 tx=0x7f20040aa070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:14.158 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.158+0000 7f200d7fa640 1 -- 172.21.15.49:0/3825107033 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f20080509a0 con 0x7f201015cbb0 2023-11-06T02:11:14.313 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.318+0000 7f2016fa0640 1 -- 172.21.15.49:0/3825107033 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f2010074040 con 0x7f201015cbb0 2023-11-06T02:11:14.314 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.318+0000 7f200d7fa640 1 -- 172.21.15.49:0/3825107033 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v52) v1 ==== 92+0+10 (secure 0 0 0) 0x7f2008053fc0 con 0x7f201015cbb0 2023-11-06T02:11:14.314 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 8 2023-11-06T02:11:14.316 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.318+0000 7f2016fa0640 1 -- 172.21.15.49:0/3825107033 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1ffc065a60 msgr2=0x7f1ffc067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:14.316 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.318+0000 7f2016fa0640 1 --2- 172.21.15.49:0/3825107033 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1ffc065a60 0x7f1ffc067f20 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f20040b2000 tx=0x7f20040aa070 comp rx=0 tx=0).stop 2023-11-06T02:11:14.316 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.318+0000 7f2016fa0640 1 -- 172.21.15.49:0/3825107033 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f201015cbb0 msgr2=0x7f2010158cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:14.317 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.318+0000 7f2016fa0640 1 --2- 172.21.15.49:0/3825107033 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f201015cbb0 0x7f2010158cf0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f200800c7a0 tx=0x7f200800cc60 comp rx=0 tx=0).stop 2023-11-06T02:11:14.317 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.318+0000 7f2016fa0640 1 -- 172.21.15.49:0/3825107033 shutdown_connections 2023-11-06T02:11:14.317 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.318+0000 7f2016fa0640 1 --2- 172.21.15.49:0/3825107033 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1ffc065a60 0x7f1ffc067f20 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:14.317 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.318+0000 7f2016fa0640 1 --2- 172.21.15.49:0/3825107033 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f201015cbb0 0x7f2010158cf0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:14.317 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.318+0000 7f2016fa0640 1 --2- 172.21.15.49:0/3825107033 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f20100d2fa0 0x7f20101561a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:14.318 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.318+0000 7f2016fa0640 1 --2- 172.21.15.49:0/3825107033 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20100d24e0 0x7f20100a9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:14.318 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.318+0000 7f2016fa0640 1 -- 172.21.15.49:0/3825107033 >> 172.21.15.49:0/3825107033 conn(0x7f20100718a0 msgr2=0x7f201015c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:14.318 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.318+0000 7f2016fa0640 1 -- 172.21.15.49:0/3825107033 shutdown_connections 2023-11-06T02:11:14.318 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.318+0000 7f2016fa0640 1 -- 172.21.15.49:0/3825107033 wait complete. 2023-11-06T02:11:14.332 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T02:11:14.461 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.462+0000 7efdc1aec640 1 Processor -- start 2023-11-06T02:11:14.461 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.466+0000 7efdc1aec640 1 -- start start 2023-11-06T02:11:14.463 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.466+0000 7efdc1aec640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7efdbc1595a0 0x7efdbc15dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:14.463 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.466+0000 7efdc1aec640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7efdbc15a990 0x7efdbc0c8e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:14.463 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.466+0000 7efdc1aec640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdbc06c5a0 con 0x7efdbc06d640 2023-11-06T02:11:14.463 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.466+0000 7efdc1aec640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdbc15e250 con 0x7efdbc15a990 2023-11-06T02:11:14.463 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.466+0000 7efdc1aec640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdbc06c880 con 0x7efdbc1595a0 2023-11-06T02:11:14.464 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.466+0000 7efdbbfff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7efdbc15a990 0x7efdbc0c8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:14.464 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.466+0000 7efdbbfff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7efdbc15a990 0x7efdbc0c8e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42590/0 (socket says 172.21.15.49:42590) 2023-11-06T02:11:14.464 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.466+0000 7efdbbfff640 1 -- 172.21.15.49:0/3309651830 learned_addr learned my addr 172.21.15.49:0/3309651830 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:14.464 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.466+0000 7efdbb7fe640 1 --2- 172.21.15.49:0/3309651830 >> v2:172.21.15.150:3300/0 conn(0x7efdbc1595a0 0x7efdbc15dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:14.464 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.466+0000 7efdbbfff640 1 -- 172.21.15.49:0/3309651830 >> v2:172.21.15.150:3300/0 conn(0x7efdbc1595a0 msgr2=0x7efdbc15dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:14.465 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.466+0000 7efdbbfff640 1 --2- 172.21.15.49:0/3309651830 >> v2:172.21.15.150:3300/0 conn(0x7efdbc1595a0 0x7efdbc15dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:14.465 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.466+0000 7efdbbfff640 1 -- 172.21.15.49:0/3309651830 >> v1:172.21.15.150:6789/0 conn(0x7efdbc06d640 legacy=0x7efdbc0b0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:14.465 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.466+0000 7efdbbfff640 1 -- 172.21.15.49:0/3309651830 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdbc06cb60 con 0x7efdbc15a990 2023-11-06T02:11:14.465 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.466+0000 7efdbb7fe640 1 --2- 172.21.15.49:0/3309651830 >> v2:172.21.15.150:3300/0 conn(0x7efdbc1595a0 0x7efdbc15dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:11:14.467 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.466+0000 7efdbbfff640 1 --2- 172.21.15.49:0/3309651830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efdbc15a990 0x7efdbc0c8e90 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7efdbc071420 tx=0x7efdb401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0d186fe54a803a1 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:14.467 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.470+0000 7efdba7fc640 1 -- 172.21.15.49:0/3309651830 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb40ac600 con 0x7efdbc15a990 2023-11-06T02:11:14.467 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.470+0000 7efdba7fc640 1 -- 172.21.15.49:0/3309651830 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdb40a8050 con 0x7efdbc15a990 2023-11-06T02:11:14.467 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.470+0000 7efdba7fc640 1 -- 172.21.15.49:0/3309651830 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb40ac980 con 0x7efdbc15a990 2023-11-06T02:11:14.469 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.470+0000 7efdc1aec640 1 -- 172.21.15.49:0/3309651830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efdbc15a990 msgr2=0x7efdbc0c8e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:14.469 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.470+0000 7efdc1aec640 1 --2- 172.21.15.49:0/3309651830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efdbc15a990 0x7efdbc0c8e90 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7efdbc071420 tx=0x7efdb401c490 comp rx=0 tx=0).stop 2023-11-06T02:11:14.469 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.470+0000 7efdc1aec640 1 -- 172.21.15.49:0/3309651830 shutdown_connections 2023-11-06T02:11:14.469 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.470+0000 7efdc1aec640 1 --2- 172.21.15.49:0/3309651830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efdbc15a990 0x7efdbc0c8e90 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:14.469 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.470+0000 7efdc1aec640 1 --2- 172.21.15.49:0/3309651830 >> v2:172.21.15.150:3300/0 conn(0x7efdbc1595a0 0x7efdbc15dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:14.470 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.470+0000 7efdc1aec640 1 -- 172.21.15.49:0/3309651830 >> 172.21.15.49:0/3309651830 conn(0x7efdbc06a370 msgr2=0x7efdbc0ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:14.470 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.470+0000 7efdc1aec640 1 -- 172.21.15.49:0/3309651830 shutdown_connections 2023-11-06T02:11:14.470 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.470+0000 7efdc1aec640 1 -- 172.21.15.49:0/3309651830 wait complete. 2023-11-06T02:11:14.470 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.470+0000 7efdc1aec640 1 Processor -- start 2023-11-06T02:11:14.470 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.470+0000 7efdc1aec640 1 -- start start 2023-11-06T02:11:14.473 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.470+0000 7efdc1aec640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efdbc06d640 0x7efdbc14d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:14.473 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.470+0000 7efdc1aec640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efdbc1595a0 0x7efdbc070bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:14.473 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.470+0000 7efdc1aec640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efdbc15a990 0x7efdbc144cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:14.474 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.470+0000 7efdc1aec640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdbc145230 con 0x7efdbc1595a0 2023-11-06T02:11:14.474 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.470+0000 7efdc1aec640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdbc14e010 con 0x7efdbc15a990 2023-11-06T02:11:14.474 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.470+0000 7efdc1aec640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdbc14e2f0 con 0x7efdbc06d640 2023-11-06T02:11:14.474 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.470+0000 7efdbaffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efdbc1595a0 0x7efdbc070bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:14.474 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.470+0000 7efdbaffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efdbc1595a0 0x7efdbc070bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42596/0 (socket says 172.21.15.49:42596) 2023-11-06T02:11:14.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.470+0000 7efdbaffd640 1 -- 172.21.15.49:0/3298233514 learned_addr learned my addr 172.21.15.49:0/3298233514 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:14.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.470+0000 7efdbb7fe640 1 --2- 172.21.15.49:0/3298233514 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efdbc06d640 0x7efdbc14d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:14.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.470+0000 7efdbbfff640 1 --2- 172.21.15.49:0/3298233514 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efdbc15a990 0x7efdbc144cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:14.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.470+0000 7efdbaffd640 1 -- 172.21.15.49:0/3298233514 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efdbc06d640 msgr2=0x7efdbc14d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:14.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.470+0000 7efdbaffd640 1 --2- 172.21.15.49:0/3298233514 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efdbc06d640 0x7efdbc14d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:14.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.470+0000 7efdbaffd640 1 -- 172.21.15.49:0/3298233514 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efdbc15a990 msgr2=0x7efdbc144cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:14.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.470+0000 7efdbaffd640 1 --2- 172.21.15.49:0/3298233514 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efdbc15a990 0x7efdbc144cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:14.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.470+0000 7efdbaffd640 1 -- 172.21.15.49:0/3298233514 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdb40094d0 con 0x7efdbc1595a0 2023-11-06T02:11:14.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.470+0000 7efdbaffd640 1 --2- 172.21.15.49:0/3298233514 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efdbc1595a0 0x7efdbc070bd0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7efdb000c450 tx=0x7efdb000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:14.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.470+0000 7efda3fff640 1 -- 172.21.15.49:0/3298233514 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb0012830 con 0x7efdbc1595a0 2023-11-06T02:11:14.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.470+0000 7efda3fff640 1 -- 172.21.15.49:0/3298233514 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdb00129b0 con 0x7efdbc1595a0 2023-11-06T02:11:14.477 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.470+0000 7efda3fff640 1 -- 172.21.15.49:0/3298233514 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb0012d60 con 0x7efdbc1595a0 2023-11-06T02:11:14.477 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.470+0000 7efdc1aec640 1 -- 172.21.15.49:0/3298233514 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdbc14e5d0 con 0x7efdbc1595a0 2023-11-06T02:11:14.477 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.474+0000 7efdc1aec640 1 -- 172.21.15.49:0/3298233514 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdbc1583b0 con 0x7efdbc1595a0 2023-11-06T02:11:14.477 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.474+0000 7efdc1aec640 1 -- 172.21.15.49:0/3298233514 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdbc06c210 con 0x7efdbc1595a0 2023-11-06T02:11:14.477 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.474+0000 7efda3fff640 1 -- 172.21.15.49:0/3298233514 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7efdb001c460 con 0x7efdbc1595a0 2023-11-06T02:11:14.478 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.474+0000 7efda3fff640 1 --2- 172.21.15.49:0/3298233514 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7efda8065a90 0x7efda8067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:14.478 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.474+0000 7efda3fff640 1 -- 172.21.15.49:0/3298233514 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6120+0+0 (secure 0 0 0) 0x7efdb0083130 con 0x7efdbc1595a0 2023-11-06T02:11:14.478 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.474+0000 7efdbb7fe640 1 --2- 172.21.15.49:0/3298233514 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7efda8065a90 0x7efda8067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:14.478 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.474+0000 7efdbb7fe640 1 --2- 172.21.15.49:0/3298233514 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7efda8065a90 0x7efda8067f50 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7efdac0075b0 tx=0x7efdac007540 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:14.479 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.478+0000 7efda3fff640 1 -- 172.21.15.49:0/3298233514 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7efdbc06c210 con 0x7efdbc1595a0 2023-11-06T02:11:14.633 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.638+0000 7efdc1aec640 1 -- 172.21.15.49:0/3298233514 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7efdbc06c210 con 0x7efdbc1595a0 2023-11-06T02:11:14.634 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.638+0000 7efda3fff640 1 -- 172.21.15.49:0/3298233514 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v52) v1 ==== 104+0+11 (secure 0 0 0) 0x7efdbc06c210 con 0x7efdbc1595a0 2023-11-06T02:11:14.634 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2023-11-06T02:11:14.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.638+0000 7efdc1aec640 1 -- 172.21.15.49:0/3298233514 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7efda8065a90 msgr2=0x7efda8067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:14.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.638+0000 7efdc1aec640 1 --2- 172.21.15.49:0/3298233514 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7efda8065a90 0x7efda8067f50 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7efdac0075b0 tx=0x7efdac007540 comp rx=0 tx=0).stop 2023-11-06T02:11:14.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.638+0000 7efdc1aec640 1 -- 172.21.15.49:0/3298233514 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efdbc1595a0 msgr2=0x7efdbc070bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:14.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.638+0000 7efdc1aec640 1 --2- 172.21.15.49:0/3298233514 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efdbc1595a0 0x7efdbc070bd0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7efdb000c450 tx=0x7efdb000c910 comp rx=0 tx=0).stop 2023-11-06T02:11:14.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.638+0000 7efdc1aec640 1 -- 172.21.15.49:0/3298233514 shutdown_connections 2023-11-06T02:11:14.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.638+0000 7efdc1aec640 1 --2- 172.21.15.49:0/3298233514 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7efda8065a90 0x7efda8067f50 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:14.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.638+0000 7efdc1aec640 1 --2- 172.21.15.49:0/3298233514 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efdbc1595a0 0x7efdbc070bd0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:14.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.638+0000 7efdc1aec640 1 --2- 172.21.15.49:0/3298233514 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efdbc15a990 0x7efdbc144cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:14.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.638+0000 7efdc1aec640 1 --2- 172.21.15.49:0/3298233514 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efdbc06d640 0x7efdbc14d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:14.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.638+0000 7efdc1aec640 1 -- 172.21.15.49:0/3298233514 >> 172.21.15.49:0/3298233514 conn(0x7efdbc06a370 msgr2=0x7efdbc0cc170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:14.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.638+0000 7efdc1aec640 1 -- 172.21.15.49:0/3298233514 shutdown_connections 2023-11-06T02:11:14.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.638+0000 7efdc1aec640 1 -- 172.21.15.49:0/3298233514 wait complete. 2023-11-06T02:11:14.654 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T02:11:14.785 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.790+0000 7f440613c640 1 Processor -- start 2023-11-06T02:11:14.786 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.790+0000 7f440613c640 1 -- start start 2023-11-06T02:11:14.787 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.790+0000 7f440613c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f440015a880 0x7f44000b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:14.787 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.790+0000 7f440613c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4400070160 0x7f4400070610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:14.787 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.790+0000 7f440613c640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44000ae360 con 0x7f440006d150 2023-11-06T02:11:14.787 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.790+0000 7f440613c640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4400070b50 con 0x7f440015a880 2023-11-06T02:11:14.787 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.790+0000 7f440613c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44000ae640 con 0x7f4400070160 2023-11-06T02:11:14.788 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.790+0000 7f43ff7fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f440015a880 0x7f44000b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:14.788 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.790+0000 7f43fffff640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f440006d150 0x7f440006d5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:59998/0 (socket says 172.21.15.49:59998) 2023-11-06T02:11:14.788 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.790+0000 7f43fffff640 1 -- 172.21.15.49:0/335574530 learned_addr learned my addr 172.21.15.49:0/335574530 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:11:14.788 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.790+0000 7f43feffd640 1 --2- 172.21.15.49:0/335574530 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4400070160 0x7f4400070610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:14.789 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.790+0000 7f43ff7fe640 1 -- 172.21.15.49:0/335574530 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4400070160 msgr2=0x7f4400070610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:14.790 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.790+0000 7f43ff7fe640 1 --2- 172.21.15.49:0/335574530 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4400070160 0x7f4400070610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:14.790 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.790+0000 7f43ff7fe640 1 -- 172.21.15.49:0/335574530 >> v1:172.21.15.49:6789/0 conn(0x7f440006d150 legacy=0x7f440006d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:14.790 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.790+0000 7f43fe7fc640 1 -- 172.21.15.49:0/335574530 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1732789994 0 0) 0x7f44000ae360 con 0x7f440006d150 2023-11-06T02:11:14.790 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.790+0000 7f43ff7fe640 1 -- 172.21.15.49:0/335574530 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44000ae920 con 0x7f440015a880 2023-11-06T02:11:14.790 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.794+0000 7f43ff7fe640 1 --2- 172.21.15.49:0/335574530 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f440015a880 0x7f44000b15c0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f43f00097c0 tx=0x7f43f001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80288c204b6f3040 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:14.791 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.794+0000 7f43fe7fc640 1 -- 172.21.15.49:0/335574530 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f00ad640 con 0x7f440015a880 2023-11-06T02:11:14.791 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.794+0000 7f43fe7fc640 1 -- 172.21.15.49:0/335574530 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43f00ad7a0 con 0x7f440015a880 2023-11-06T02:11:14.791 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.794+0000 7f43fe7fc640 1 -- 172.21.15.49:0/335574530 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f00adb30 con 0x7f440015a880 2023-11-06T02:11:14.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.794+0000 7f440613c640 1 -- 172.21.15.49:0/335574530 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f440015a880 msgr2=0x7f44000b15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:14.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.794+0000 7f440613c640 1 --2- 172.21.15.49:0/335574530 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f440015a880 0x7f44000b15c0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f43f00097c0 tx=0x7f43f001c480 comp rx=0 tx=0).stop 2023-11-06T02:11:14.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.794+0000 7f440613c640 1 -- 172.21.15.49:0/335574530 shutdown_connections 2023-11-06T02:11:14.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.794+0000 7f440613c640 1 --2- 172.21.15.49:0/335574530 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4400070160 0x7f4400070610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:14.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.794+0000 7f440613c640 1 --2- 172.21.15.49:0/335574530 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f440015a880 0x7f44000b15c0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:14.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.794+0000 7f440613c640 1 -- 172.21.15.49:0/335574530 >> 172.21.15.49:0/335574530 conn(0x7f440006a370 msgr2=0x7f44000ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:14.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.794+0000 7f440613c640 1 -- 172.21.15.49:0/335574530 shutdown_connections 2023-11-06T02:11:14.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.794+0000 7f440613c640 1 -- 172.21.15.49:0/335574530 wait complete. 2023-11-06T02:11:14.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.794+0000 7f440613c640 1 Processor -- start 2023-11-06T02:11:14.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.794+0000 7f440613c640 1 -- start start 2023-11-06T02:11:14.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.794+0000 7f440613c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f440006d150 0x7f440014dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:14.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.794+0000 7f440613c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4400070160 0x7f44000c67d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:14.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.794+0000 7f440613c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f440015a880 0x7f44000c6d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:14.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.794+0000 7f440613c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44000c7c10 con 0x7f4400070160 2023-11-06T02:11:14.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.794+0000 7f440613c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44000c7370 con 0x7f440015a880 2023-11-06T02:11:14.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.794+0000 7f440613c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44000c7650 con 0x7f440006d150 2023-11-06T02:11:14.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.794+0000 7f43feffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4400070160 0x7f44000c67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:14.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.794+0000 7f43feffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4400070160 0x7f44000c67d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42610/0 (socket says 172.21.15.49:42610) 2023-11-06T02:11:14.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.794+0000 7f43feffd640 1 -- 172.21.15.49:0/2277726272 learned_addr learned my addr 172.21.15.49:0/2277726272 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:14.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.798+0000 7f43fffff640 1 --2- 172.21.15.49:0/2277726272 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f440015a880 0x7f44000c6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:14.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.798+0000 7f43ff7fe640 1 --2- 172.21.15.49:0/2277726272 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f440006d150 0x7f440014dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:14.798 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.798+0000 7f43feffd640 1 -- 172.21.15.49:0/2277726272 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f440006d150 msgr2=0x7f440014dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:14.798 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.798+0000 7f43feffd640 1 --2- 172.21.15.49:0/2277726272 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f440006d150 0x7f440014dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:14.798 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.798+0000 7f43feffd640 1 -- 172.21.15.49:0/2277726272 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f440015a880 msgr2=0x7f44000c6d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:14.798 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.798+0000 7f43feffd640 1 --2- 172.21.15.49:0/2277726272 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f440015a880 0x7f44000c6d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:14.798 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.798+0000 7f43feffd640 1 -- 172.21.15.49:0/2277726272 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43f0009420 con 0x7f4400070160 2023-11-06T02:11:14.799 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.798+0000 7f43feffd640 1 --2- 172.21.15.49:0/2277726272 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4400070160 0x7f44000c67d0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f43f400d800 tx=0x7f43f400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:14.799 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.798+0000 7f43e7fff640 1 -- 172.21.15.49:0/2277726272 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f400b6c0 con 0x7f4400070160 2023-11-06T02:11:14.799 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.798+0000 7f43e7fff640 1 -- 172.21.15.49:0/2277726272 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43f400b820 con 0x7f4400070160 2023-11-06T02:11:14.799 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.798+0000 7f43e7fff640 1 -- 172.21.15.49:0/2277726272 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f400bb90 con 0x7f4400070160 2023-11-06T02:11:14.799 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.798+0000 7f440613c640 1 -- 172.21.15.49:0/2277726272 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44000c7930 con 0x7f4400070160 2023-11-06T02:11:14.800 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.798+0000 7f440613c640 1 -- 172.21.15.49:0/2277726272 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44000ccab0 con 0x7f4400070160 2023-11-06T02:11:14.801 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.798+0000 7f440613c640 1 -- 172.21.15.49:0/2277726272 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44000c3530 con 0x7f4400070160 2023-11-06T02:11:14.801 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.798+0000 7f43e7fff640 1 -- 172.21.15.49:0/2277726272 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f43f400bcf0 con 0x7f4400070160 2023-11-06T02:11:14.801 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.798+0000 7f43e7fff640 1 --2- 172.21.15.49:0/2277726272 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f43ec065a60 0x7f43ec067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:14.801 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.798+0000 7f43e7fff640 1 -- 172.21.15.49:0/2277726272 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f43f40838c0 con 0x7f4400070160 2023-11-06T02:11:14.801 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.798+0000 7f43ff7fe640 1 --2- 172.21.15.49:0/2277726272 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f43ec065a60 0x7f43ec067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:14.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.798+0000 7f43ff7fe640 1 --2- 172.21.15.49:0/2277726272 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f43ec065a60 0x7f43ec067f20 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f43f00af000 tx=0x7f43f0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:14.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.802+0000 7f43e7fff640 1 -- 172.21.15.49:0/2277726272 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f44000c3530 con 0x7f4400070160 2023-11-06T02:11:14.953 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.958+0000 7f440613c640 1 -- 172.21.15.49:0/2277726272 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f44000c3530 con 0x7f4400070160 2023-11-06T02:11:14.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.958+0000 7f43e7fff640 1 -- 172.21.15.49:0/2277726272 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v52) v1 ==== 100+0+11 (secure 0 0 0) 0x7f44000c3530 con 0x7f4400070160 2023-11-06T02:11:14.956 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2023-11-06T02:11:14.957 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.958+0000 7f440613c640 1 -- 172.21.15.49:0/2277726272 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f43ec065a60 msgr2=0x7f43ec067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:14.957 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.958+0000 7f440613c640 1 --2- 172.21.15.49:0/2277726272 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f43ec065a60 0x7f43ec067f20 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f43f00af000 tx=0x7f43f0008040 comp rx=0 tx=0).stop 2023-11-06T02:11:14.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.958+0000 7f440613c640 1 -- 172.21.15.49:0/2277726272 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4400070160 msgr2=0x7f44000c67d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:14.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.958+0000 7f440613c640 1 --2- 172.21.15.49:0/2277726272 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4400070160 0x7f44000c67d0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f43f400d800 tx=0x7f43f400dcc0 comp rx=0 tx=0).stop 2023-11-06T02:11:14.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.958+0000 7f440613c640 1 -- 172.21.15.49:0/2277726272 shutdown_connections 2023-11-06T02:11:14.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.958+0000 7f440613c640 1 --2- 172.21.15.49:0/2277726272 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f43ec065a60 0x7f43ec067f20 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:14.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.958+0000 7f440613c640 1 --2- 172.21.15.49:0/2277726272 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4400070160 0x7f44000c67d0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:14.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.958+0000 7f440613c640 1 --2- 172.21.15.49:0/2277726272 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f440015a880 0x7f44000c6d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:14.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.958+0000 7f440613c640 1 --2- 172.21.15.49:0/2277726272 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f440006d150 0x7f440014dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:14.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.958+0000 7f440613c640 1 -- 172.21.15.49:0/2277726272 >> 172.21.15.49:0/2277726272 conn(0x7f440006a370 msgr2=0x7f44001533c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:14.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.958+0000 7f440613c640 1 -- 172.21.15.49:0/2277726272 shutdown_connections 2023-11-06T02:11:14.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:14.958+0000 7f440613c640 1 -- 172.21.15.49:0/2277726272 wait complete. 2023-11-06T02:11:14.973 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-06T02:11:15.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.098+0000 7f5aea5fd640 1 Processor -- start 2023-11-06T02:11:15.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.098+0000 7f5aea5fd640 1 -- start start 2023-11-06T02:11:15.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.098+0000 7f5aea5fd640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f5ae415aab0 0x7f5ae40cb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:15.096 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.098+0000 7f5aea5fd640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ae40cd990 0x7f5ae40ce880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:15.096 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.098+0000 7f5aea5fd640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ae40ce3f0 con 0x7f5ae40cb970 2023-11-06T02:11:15.096 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.098+0000 7f5aea5fd640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ae415ae90 con 0x7f5ae415aab0 2023-11-06T02:11:15.096 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.098+0000 7f5aea5fd640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae40cbdf0 con 0x7f5ae40cd990 2023-11-06T02:11:15.097 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.098+0000 7f5ae3fff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f5ae415aab0 0x7f5ae40cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:15.097 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.098+0000 7f5ae3fff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f5ae415aab0 0x7f5ae40cb430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42626/0 (socket says 172.21.15.49:42626) 2023-11-06T02:11:15.097 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.098+0000 7f5ae3fff640 1 -- 172.21.15.49:0/205154135 learned_addr learned my addr 172.21.15.49:0/205154135 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:15.097 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.098+0000 7f5ae8b9a640 1 --2- 172.21.15.49:0/205154135 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ae40cd990 0x7f5ae40ce880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:15.097 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.098+0000 7f5ae3fff640 1 -- 172.21.15.49:0/205154135 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ae40cd990 msgr2=0x7f5ae40ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:15.098 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.098+0000 7f5ae3fff640 1 --2- 172.21.15.49:0/205154135 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ae40cd990 0x7f5ae40ce880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:15.098 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.098+0000 7f5ae3fff640 1 -- 172.21.15.49:0/205154135 >> v1:172.21.15.150:6789/0 conn(0x7f5ae40cb970 legacy=0x7f5ae40cd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:15.098 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.098+0000 7f5ae3fff640 1 -- 172.21.15.49:0/205154135 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ae40cc0d0 con 0x7f5ae415aab0 2023-11-06T02:11:15.099 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.102+0000 7f5ae3fff640 1 --2- 172.21.15.49:0/205154135 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ae415aab0 0x7f5ae40cb430 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f5ad4006fd0 tx=0x7f5ad401c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff123bf28ff68505 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:15.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.102+0000 7f5ae2ffd640 1 -- 172.21.15.49:0/205154135 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad40ac640 con 0x7f5ae415aab0 2023-11-06T02:11:15.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.102+0000 7f5ae2ffd640 1 -- 172.21.15.49:0/205154135 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ad40ac7a0 con 0x7f5ae415aab0 2023-11-06T02:11:15.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.102+0000 7f5ae2ffd640 1 -- 172.21.15.49:0/205154135 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad40acb30 con 0x7f5ae415aab0 2023-11-06T02:11:15.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.102+0000 7f5aea5fd640 1 -- 172.21.15.49:0/205154135 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ae415aab0 msgr2=0x7f5ae40cb430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:15.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.102+0000 7f5aea5fd640 1 --2- 172.21.15.49:0/205154135 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ae415aab0 0x7f5ae40cb430 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f5ad4006fd0 tx=0x7f5ad401c420 comp rx=0 tx=0).stop 2023-11-06T02:11:15.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.102+0000 7f5aea5fd640 1 -- 172.21.15.49:0/205154135 shutdown_connections 2023-11-06T02:11:15.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.102+0000 7f5aea5fd640 1 --2- 172.21.15.49:0/205154135 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ae40cd990 0x7f5ae40ce880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:15.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.102+0000 7f5aea5fd640 1 --2- 172.21.15.49:0/205154135 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ae415aab0 0x7f5ae40cb430 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:15.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.102+0000 7f5aea5fd640 1 -- 172.21.15.49:0/205154135 >> 172.21.15.49:0/205154135 conn(0x7f5ae4089fd0 msgr2=0x7f5ae4099c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:15.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.102+0000 7f5aea5fd640 1 -- 172.21.15.49:0/205154135 shutdown_connections 2023-11-06T02:11:15.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.102+0000 7f5aea5fd640 1 -- 172.21.15.49:0/205154135 wait complete. 2023-11-06T02:11:15.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.102+0000 7f5aea5fd640 1 Processor -- start 2023-11-06T02:11:15.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.102+0000 7f5aea5fd640 1 -- start start 2023-11-06T02:11:15.105 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.106+0000 7f5aea5fd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ae40cb970 0x7f5ae406ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:15.105 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.106+0000 7f5aea5fd640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ae40cd990 0x7f5ae414f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:15.105 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.106+0000 7f5aea5fd640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ae415aab0 0x7f5ae40c4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:15.105 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.106+0000 7f5aea5fd640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae406b9c0 con 0x7f5ae40cb970 2023-11-06T02:11:15.105 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.106+0000 7f5aea5fd640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae406bca0 con 0x7f5ae40cd990 2023-11-06T02:11:15.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.106+0000 7f5aea5fd640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae406bf80 con 0x7f5ae415aab0 2023-11-06T02:11:15.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.106+0000 7f5ae3fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ae40cb970 0x7f5ae406ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:15.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.106+0000 7f5ae3fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ae40cb970 0x7f5ae406ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42634/0 (socket says 172.21.15.49:42634) 2023-11-06T02:11:15.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.106+0000 7f5ae3fff640 1 -- 172.21.15.49:0/3742390570 learned_addr learned my addr 172.21.15.49:0/3742390570 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:15.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.106+0000 7f5ae37fe640 1 --2- 172.21.15.49:0/3742390570 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ae40cd990 0x7f5ae414f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:15.107 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.106+0000 7f5ae8b9a640 1 --2- 172.21.15.49:0/3742390570 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ae415aab0 0x7f5ae40c4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:15.107 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.106+0000 7f5ae3fff640 1 -- 172.21.15.49:0/3742390570 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ae415aab0 msgr2=0x7f5ae40c4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:15.107 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.106+0000 7f5ae3fff640 1 --2- 172.21.15.49:0/3742390570 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ae415aab0 0x7f5ae40c4e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:15.107 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.106+0000 7f5ae3fff640 1 -- 172.21.15.49:0/3742390570 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ae40cd990 msgr2=0x7f5ae414f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:15.107 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.106+0000 7f5ae3fff640 1 --2- 172.21.15.49:0/3742390570 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ae40cd990 0x7f5ae414f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:15.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.106+0000 7f5ae3fff640 1 -- 172.21.15.49:0/3742390570 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ad4009480 con 0x7f5ae40cb970 2023-11-06T02:11:15.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.106+0000 7f5ae37fe640 1 --2- 172.21.15.49:0/3742390570 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ae40cd990 0x7f5ae414f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:11:15.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.106+0000 7f5ae8b9a640 1 --2- 172.21.15.49:0/3742390570 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ae415aab0 0x7f5ae40c4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:11:15.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.106+0000 7f5ae3fff640 1 --2- 172.21.15.49:0/3742390570 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ae40cb970 0x7f5ae406ca60 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f5ad401aad0 tx=0x7f5ad40191c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:15.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.106+0000 7f5ae0ff9640 1 -- 172.21.15.49:0/3742390570 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad40b9070 con 0x7f5ae40cb970 2023-11-06T02:11:15.109 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.106+0000 7f5ae0ff9640 1 -- 172.21.15.49:0/3742390570 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ad40ac5c0 con 0x7f5ae40cb970 2023-11-06T02:11:15.109 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.106+0000 7f5ae0ff9640 1 -- 172.21.15.49:0/3742390570 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad40acd40 con 0x7f5ae40cb970 2023-11-06T02:11:15.109 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.106+0000 7f5aea5fd640 1 -- 172.21.15.49:0/3742390570 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ae406c260 con 0x7f5ae40cb970 2023-11-06T02:11:15.109 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.106+0000 7f5aea5fd640 1 -- 172.21.15.49:0/3742390570 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ae414fd70 con 0x7f5ae40cb970 2023-11-06T02:11:15.109 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.106+0000 7f5aea5fd640 1 -- 172.21.15.49:0/3742390570 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ae40ceaa0 con 0x7f5ae40cb970 2023-11-06T02:11:15.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.106+0000 7f5ae0ff9640 1 -- 172.21.15.49:0/3742390570 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f5ad4003620 con 0x7f5ae40cb970 2023-11-06T02:11:15.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.106+0000 7f5ae0ff9640 1 --2- 172.21.15.49:0/3742390570 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5ad0065a60 0x7f5ad0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:15.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.106+0000 7f5ae0ff9640 1 -- 172.21.15.49:0/3742390570 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f5ad411cf10 con 0x7f5ae40cb970 2023-11-06T02:11:15.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.106+0000 7f5ae37fe640 1 --2- 172.21.15.49:0/3742390570 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5ad0065a60 0x7f5ad0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:15.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.110+0000 7f5ae37fe640 1 --2- 172.21.15.49:0/3742390570 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5ad0065a60 0x7f5ad0067f20 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f5ae40c6c90 tx=0x7f5ad8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:15.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.110+0000 7f5ae0ff9640 1 -- 172.21.15.49:0/3742390570 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f5ad40ea7c0 con 0x7f5ae40cb970 2023-11-06T02:11:15.282 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.286+0000 7f5aea5fd640 1 -- 172.21.15.49:0/3742390570 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f5ae40b6730 con 0x7f5ae40cb970 2023-11-06T02:11:15.283 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.286+0000 7f5ae0ff9640 1 -- 172.21.15.49:0/3742390570 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v53) v1 ==== 80+0+0 (secure 0 0 0) 0x7f5ad40edde0 con 0x7f5ae40cb970 2023-11-06T02:11:15.283 INFO:teuthology.orchestra.run.smithi049.stderr:listed 0 entries 2023-11-06T02:11:15.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.290+0000 7f5aea5fd640 1 -- 172.21.15.49:0/3742390570 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5ad0065a60 msgr2=0x7f5ad0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:15.286 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.290+0000 7f5aea5fd640 1 --2- 172.21.15.49:0/3742390570 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5ad0065a60 0x7f5ad0067f20 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f5ae40c6c90 tx=0x7f5ad8009040 comp rx=0 tx=0).stop 2023-11-06T02:11:15.286 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.290+0000 7f5aea5fd640 1 -- 172.21.15.49:0/3742390570 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ae40cb970 msgr2=0x7f5ae406ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:15.286 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.290+0000 7f5aea5fd640 1 --2- 172.21.15.49:0/3742390570 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ae40cb970 0x7f5ae406ca60 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f5ad401aad0 tx=0x7f5ad40191c0 comp rx=0 tx=0).stop 2023-11-06T02:11:15.286 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.290+0000 7f5aea5fd640 1 -- 172.21.15.49:0/3742390570 shutdown_connections 2023-11-06T02:11:15.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.290+0000 7f5aea5fd640 1 --2- 172.21.15.49:0/3742390570 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5ad0065a60 0x7f5ad0067f20 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:15.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.290+0000 7f5aea5fd640 1 --2- 172.21.15.49:0/3742390570 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ae415aab0 0x7f5ae40c4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:15.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.290+0000 7f5aea5fd640 1 --2- 172.21.15.49:0/3742390570 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ae40cd990 0x7f5ae414f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:15.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.290+0000 7f5aea5fd640 1 --2- 172.21.15.49:0/3742390570 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ae40cb970 0x7f5ae406ca60 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:15.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.290+0000 7f5aea5fd640 1 -- 172.21.15.49:0/3742390570 >> 172.21.15.49:0/3742390570 conn(0x7f5ae4089fd0 msgr2=0x7f5ae406fd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:15.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.290+0000 7f5aea5fd640 1 -- 172.21.15.49:0/3742390570 shutdown_connections 2023-11-06T02:11:15.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.290+0000 7f5aea5fd640 1 -- 172.21.15.49:0/3742390570 wait complete. 2023-11-06T02:11:15.303 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi049.front.sepia.ceph.com... 2023-11-06T02:11:15.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:11:15.304 DEBUG:teuthology.orchestra.run.smithi049:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-06T02:11:15.315 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi049.stderr:ceph-fuse[50158]: fuse finished with error 0 and tester_r 0 2023-11-06T02:11:15.316 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi049.front.sepia.ceph.com 2023-11-06T02:11:15.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:11:15.317 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-06T02:11:15.393 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-11-06T02:11:15.393 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T02:11:15.400 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336552k,nr_inodes=4084138,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19862 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274692k,nr_inodes=818673,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-06T02:11:15.400 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi150.front.sepia.ceph.com 2023-11-06T02:11:15.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:11:15.401 DEBUG:teuthology.orchestra.run.smithi150:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-06T02:11:15.445 INFO:teuthology.orchestra.run.smithi150.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-11-06T02:11:15.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T02:11:15.447 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-11-06T02:11:15.447 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T02:11:15.452 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336476k,nr_inodes=4084119,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16105 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-06T02:11:15.453 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi049.front.sepia.ceph.com 2023-11-06T02:11:15.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:11:15.453 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-06T02:11:15.500 INFO:teuthology.orchestra.run.smithi049.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-11-06T02:11:15.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T02:11:15.502 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-11-06T02:11:15.502 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T02:11:15.508 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336552k,nr_inodes=4084138,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19862 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274692k,nr_inodes=818673,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-06T02:11:15.509 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi150.front.sepia.ceph.com 2023-11-06T02:11:15.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:11:15.510 DEBUG:teuthology.orchestra.run.smithi150:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-06T02:11:15.557 INFO:teuthology.orchestra.run.smithi150.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-11-06T02:11:15.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T02:11:15.559 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T02:11:15.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.690+0000 7f1fc777b640 1 Processor -- start 2023-11-06T02:11:15.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.690+0000 7f1fc777b640 1 -- start start 2023-11-06T02:11:15.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.690+0000 7f1fc777b640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f1fc00cb010 0x7f1fc00ca980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:15.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.690+0000 7f1fc777b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1fc00cdd50 0x7f1fc00cc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:15.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.690+0000 7f1fc777b640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fc00723a0 con 0x7f1fc0159b40 2023-11-06T02:11:15.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.690+0000 7f1fc777b640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fc00715e0 con 0x7f1fc00cb010 2023-11-06T02:11:15.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.690+0000 7f1fc777b640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc00718c0 con 0x7f1fc00cdd50 2023-11-06T02:11:15.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.690+0000 7f1fc5517640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f1fc00cb010 0x7f1fc00ca980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:15.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.690+0000 7f1fc5517640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f1fc00cb010 0x7f1fc00ca980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42646/0 (socket says 172.21.15.49:42646) 2023-11-06T02:11:15.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.690+0000 7f1fc5517640 1 -- 172.21.15.49:0/2591872613 learned_addr learned my addr 172.21.15.49:0/2591872613 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:15.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.690+0000 7f1fc5d18640 1 --2- 172.21.15.49:0/2591872613 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1fc00cdd50 0x7f1fc00cc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:15.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.690+0000 7f1fc5517640 1 -- 172.21.15.49:0/2591872613 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1fc00cdd50 msgr2=0x7f1fc00cc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:15.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.690+0000 7f1fc5517640 1 --2- 172.21.15.49:0/2591872613 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1fc00cdd50 0x7f1fc00cc1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:15.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.690+0000 7f1fc5517640 1 -- 172.21.15.49:0/2591872613 >> v1:172.21.15.150:6789/0 conn(0x7f1fc0159b40 legacy=0x7f1fc00cb3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:15.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.690+0000 7f1fc5517640 1 -- 172.21.15.49:0/2591872613 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fc0071ba0 con 0x7f1fc00cb010 2023-11-06T02:11:15.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.694+0000 7f1fc5517640 1 --2- 172.21.15.49:0/2591872613 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1fc00cb010 0x7f1fc00ca980 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc006fd0 tx=0x7f1fbc01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eb96486c0098bc9 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:15.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.694+0000 7f1fb7fff640 1 -- 172.21.15.49:0/2591872613 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fbc0aa670 con 0x7f1fc00cb010 2023-11-06T02:11:15.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.694+0000 7f1fb7fff640 1 -- 172.21.15.49:0/2591872613 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fbc0aa7d0 con 0x7f1fc00cb010 2023-11-06T02:11:15.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.694+0000 7f1fb7fff640 1 -- 172.21.15.49:0/2591872613 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fbc0aab60 con 0x7f1fc00cb010 2023-11-06T02:11:15.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.694+0000 7f1fc777b640 1 -- 172.21.15.49:0/2591872613 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1fc00cb010 msgr2=0x7f1fc00ca980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:15.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.694+0000 7f1fc777b640 1 --2- 172.21.15.49:0/2591872613 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1fc00cb010 0x7f1fc00ca980 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc006fd0 tx=0x7f1fbc01c4f0 comp rx=0 tx=0).stop 2023-11-06T02:11:15.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.694+0000 7f1fc777b640 1 -- 172.21.15.49:0/2591872613 shutdown_connections 2023-11-06T02:11:15.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.694+0000 7f1fc777b640 1 --2- 172.21.15.49:0/2591872613 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1fc00cdd50 0x7f1fc00cc1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:15.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.694+0000 7f1fc777b640 1 --2- 172.21.15.49:0/2591872613 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1fc00cb010 0x7f1fc00ca980 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:15.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.694+0000 7f1fc777b640 1 -- 172.21.15.49:0/2591872613 >> 172.21.15.49:0/2591872613 conn(0x7f1fc006c0a0 msgr2=0x7f1fc006c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:15.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.694+0000 7f1fc777b640 1 -- 172.21.15.49:0/2591872613 shutdown_connections 2023-11-06T02:11:15.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.694+0000 7f1fc777b640 1 -- 172.21.15.49:0/2591872613 wait complete. 2023-11-06T02:11:15.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.694+0000 7f1fc777b640 1 Processor -- start 2023-11-06T02:11:15.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.694+0000 7f1fc777b640 1 -- start start 2023-11-06T02:11:15.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.698+0000 7f1fc777b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1fc00cb010 0x7f1fc01f9130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:15.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.698+0000 7f1fc777b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1fc00cdd50 0x7f1fc00acb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:15.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.698+0000 7f1fc777b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1fc0159b40 0x7f1fc00ad080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:15.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.698+0000 7f1fc777b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc00c7ac0 con 0x7f1fc0159b40 2023-11-06T02:11:15.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.698+0000 7f1fc777b640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc01f5f30 con 0x7f1fc00cdd50 2023-11-06T02:11:15.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.698+0000 7f1fc777b640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc01f5c50 con 0x7f1fc00cb010 2023-11-06T02:11:15.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.698+0000 7f1fc5d18640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1fc0159b40 0x7f1fc00ad080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:15.701 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.698+0000 7f1fc5d18640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1fc0159b40 0x7f1fc00ad080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42662/0 (socket says 172.21.15.49:42662) 2023-11-06T02:11:15.701 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.698+0000 7f1fc5d18640 1 -- 172.21.15.49:0/2841415127 learned_addr learned my addr 172.21.15.49:0/2841415127 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:15.701 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.698+0000 7f1fc5517640 1 --2- 172.21.15.49:0/2841415127 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1fc00cb010 0x7f1fc01f9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:15.701 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.698+0000 7f1fc4d16640 1 --2- 172.21.15.49:0/2841415127 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1fc00cdd50 0x7f1fc00acb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:15.702 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.698+0000 7f1fc5d18640 1 -- 172.21.15.49:0/2841415127 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1fc00cb010 msgr2=0x7f1fc01f9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:15.702 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.698+0000 7f1fc5d18640 1 --2- 172.21.15.49:0/2841415127 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1fc00cb010 0x7f1fc01f9130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:15.702 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.698+0000 7f1fc5d18640 1 -- 172.21.15.49:0/2841415127 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1fc00cdd50 msgr2=0x7f1fc00acb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:15.703 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.698+0000 7f1fc5d18640 1 --2- 172.21.15.49:0/2841415127 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1fc00cdd50 0x7f1fc00acb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:15.703 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.698+0000 7f1fc5d18640 1 -- 172.21.15.49:0/2841415127 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fbc009480 con 0x7f1fc0159b40 2023-11-06T02:11:15.703 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.698+0000 7f1fc5517640 1 --2- 172.21.15.49:0/2841415127 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1fc00cb010 0x7f1fc01f9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:11:15.704 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.698+0000 7f1fc4d16640 1 --2- 172.21.15.49:0/2841415127 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1fc00cdd50 0x7f1fc00acb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:11:15.704 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.698+0000 7f1fc5d18640 1 --2- 172.21.15.49:0/2841415127 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1fc0159b40 0x7f1fc00ad080 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f1fb8007e40 tx=0x7f1fb800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:15.704 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.698+0000 7f1fb5ffb640 1 -- 172.21.15.49:0/2841415127 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb8014620 con 0x7f1fc0159b40 2023-11-06T02:11:15.705 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.698+0000 7f1fb5ffb640 1 -- 172.21.15.49:0/2841415127 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fb800ee90 con 0x7f1fc0159b40 2023-11-06T02:11:15.705 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.698+0000 7f1fc777b640 1 -- 172.21.15.49:0/2841415127 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fc01f8650 con 0x7f1fc0159b40 2023-11-06T02:11:15.705 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.698+0000 7f1fb5ffb640 1 -- 172.21.15.49:0/2841415127 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb80149e0 con 0x7f1fc0159b40 2023-11-06T02:11:15.705 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.698+0000 7f1fc777b640 1 -- 172.21.15.49:0/2841415127 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fc01f8c10 con 0x7f1fc0159b40 2023-11-06T02:11:15.706 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.698+0000 7f1fc777b640 1 -- 172.21.15.49:0/2841415127 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fc0074040 con 0x7f1fc0159b40 2023-11-06T02:11:15.706 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.698+0000 7f1fb5ffb640 1 -- 172.21.15.49:0/2841415127 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f1fb8014b40 con 0x7f1fc0159b40 2023-11-06T02:11:15.706 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.698+0000 7f1fb5ffb640 1 --2- 172.21.15.49:0/2841415127 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1fa4065a90 0x7f1fa4067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:15.706 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.698+0000 7f1fb5ffb640 1 -- 172.21.15.49:0/2841415127 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f1fb8083ce0 con 0x7f1fc0159b40 2023-11-06T02:11:15.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.698+0000 7f1fc5517640 1 --2- 172.21.15.49:0/2841415127 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1fa4065a90 0x7f1fa4067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:15.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.702+0000 7f1fc5517640 1 --2- 172.21.15.49:0/2841415127 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1fa4065a90 0x7f1fa4067f50 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc0ae000 tx=0x7f1fbc0a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:15.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.702+0000 7f1fb5ffb640 1 -- 172.21.15.49:0/2841415127 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f1fb8051500 con 0x7f1fc0159b40 2023-11-06T02:11:15.890 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.894+0000 7f1fc777b640 1 -- 172.21.15.49:0/2841415127 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1fc01f8dd0 con 0x7f1fc0159b40 2023-11-06T02:11:15.891 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.894+0000 7f1fb5ffb640 1 -- 172.21.15.49:0/2841415127 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4648 (secure 0 0 0) 0x7f1fb8054b20 con 0x7f1fc0159b40 2023-11-06T02:11:15.891 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:11:15.892 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5096,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1396196190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1396196190},{"type":"v1","addr":"172.21.15.150:6835","nonce":1396196190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5102,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/2465209170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":2465209170},{"type":"v1","addr":"172.21.15.150:6837","nonce":2465209170}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5871,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/1006951804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1006951804},{"type":"v1","addr":"172.21.15.49:6833","nonce":1006951804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T02:10:31.903864+0000","modified":"2023-11-06T02:10:34.930237+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5877},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5877":{"gid":5877,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.49:6835/73939548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":73939548},{"type":"v1","addr":"172.21.15.49:6835","nonce":73939548}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-06T02:11:15.892 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 23 2023-11-06T02:11:15.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.898+0000 7f1fc777b640 1 -- 172.21.15.49:0/2841415127 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1fa4065a90 msgr2=0x7f1fa4067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:15.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.898+0000 7f1fc777b640 1 --2- 172.21.15.49:0/2841415127 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1fa4065a90 0x7f1fa4067f50 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc0ae000 tx=0x7f1fbc0a7000 comp rx=0 tx=0).stop 2023-11-06T02:11:15.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.898+0000 7f1fc777b640 1 -- 172.21.15.49:0/2841415127 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1fc0159b40 msgr2=0x7f1fc00ad080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:15.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.898+0000 7f1fc777b640 1 --2- 172.21.15.49:0/2841415127 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1fc0159b40 0x7f1fc00ad080 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f1fb8007e40 tx=0x7f1fb800e4b0 comp rx=0 tx=0).stop 2023-11-06T02:11:15.895 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.898+0000 7f1fc777b640 1 -- 172.21.15.49:0/2841415127 shutdown_connections 2023-11-06T02:11:15.895 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.898+0000 7f1fc777b640 1 --2- 172.21.15.49:0/2841415127 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1fa4065a90 0x7f1fa4067f50 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:15.895 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.898+0000 7f1fc777b640 1 --2- 172.21.15.49:0/2841415127 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1fc0159b40 0x7f1fc00ad080 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:15.895 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.898+0000 7f1fc777b640 1 --2- 172.21.15.49:0/2841415127 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1fc00cdd50 0x7f1fc00acb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:15.896 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.898+0000 7f1fc777b640 1 --2- 172.21.15.49:0/2841415127 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1fc00cb010 0x7f1fc01f9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:15.896 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.898+0000 7f1fc777b640 1 -- 172.21.15.49:0/2841415127 >> 172.21.15.49:0/2841415127 conn(0x7f1fc006c0a0 msgr2=0x7f1fc015a0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:15.896 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.898+0000 7f1fc777b640 1 -- 172.21.15.49:0/2841415127 shutdown_connections 2023-11-06T02:11:15.896 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:15.898+0000 7f1fc777b640 1 -- 172.21.15.49:0/2841415127 wait complete. 2023-11-06T02:11:15.912 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T02:11:16.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.038+0000 7f3d43ba0640 1 Processor -- start 2023-11-06T02:11:16.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.038+0000 7f3d43ba0640 1 -- start start 2023-11-06T02:11:16.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.042+0000 7f3d43ba0640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f3d3c159980 0x7f3d3c071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:16.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.042+0000 7f3d43ba0640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3d3c0b1450 0x7f3d3c0cdd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:16.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.042+0000 7f3d43ba0640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d3c071890 con 0x7f3d3c0cb010 2023-11-06T02:11:16.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.042+0000 7f3d43ba0640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d3c072410 con 0x7f3d3c159980 2023-11-06T02:11:16.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.042+0000 7f3d43ba0640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d3c0b1890 con 0x7f3d3c0b1450 2023-11-06T02:11:16.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.042+0000 7f3d4193c640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f3d3c159980 0x7f3d3c071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:16.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.042+0000 7f3d4193c640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f3d3c159980 0x7f3d3c071290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.49:52666/0 (socket says 172.21.15.49:52666) 2023-11-06T02:11:16.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.042+0000 7f3d4193c640 1 -- 172.21.15.49:0/1579306547 learned_addr learned my addr 172.21.15.49:0/1579306547 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:16.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.042+0000 7f3d4113b640 1 --2- 172.21.15.49:0/1579306547 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3d3c0b1450 0x7f3d3c0cdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:16.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.042+0000 7f3d4113b640 1 -- 172.21.15.49:0/1579306547 >> v2:172.21.15.150:3300/0 conn(0x7f3d3c159980 msgr2=0x7f3d3c071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:16.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.042+0000 7f3d4113b640 1 --2- 172.21.15.49:0/1579306547 >> v2:172.21.15.150:3300/0 conn(0x7f3d3c159980 0x7f3d3c071290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:16.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.042+0000 7f3d4113b640 1 -- 172.21.15.49:0/1579306547 >> v1:172.21.15.150:6789/0 conn(0x7f3d3c0cb010 legacy=0x7f3d3c0cc540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:16.042 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.042+0000 7f3d4113b640 1 -- 172.21.15.49:0/1579306547 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d3c071b70 con 0x7f3d3c0b1450 2023-11-06T02:11:16.042 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.042+0000 7f3d4193c640 1 --2- 172.21.15.49:0/1579306547 >> v2:172.21.15.150:3300/0 conn(0x7f3d3c159980 0x7f3d3c071290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T02:11:16.043 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.046+0000 7f3d4113b640 1 --2- 172.21.15.49:0/1579306547 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3d3c0b1450 0x7f3d3c0cdd30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3d300097c0 tx=0x7f3d3001c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50515fa45b782c09 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:16.043 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.046+0000 7f3d4093a640 1 -- 172.21.15.49:0/1579306547 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d300ab5f0 con 0x7f3d3c0b1450 2023-11-06T02:11:16.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.046+0000 7f3d4093a640 1 -- 172.21.15.49:0/1579306547 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d300ab750 con 0x7f3d3c0b1450 2023-11-06T02:11:16.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.046+0000 7f3d4093a640 1 -- 172.21.15.49:0/1579306547 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d300abae0 con 0x7f3d3c0b1450 2023-11-06T02:11:16.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.046+0000 7f3d43ba0640 1 -- 172.21.15.49:0/1579306547 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3d3c0b1450 msgr2=0x7f3d3c0cdd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:16.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.046+0000 7f3d43ba0640 1 --2- 172.21.15.49:0/1579306547 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3d3c0b1450 0x7f3d3c0cdd30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3d300097c0 tx=0x7f3d3001c480 comp rx=0 tx=0).stop 2023-11-06T02:11:16.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.046+0000 7f3d43ba0640 1 -- 172.21.15.49:0/1579306547 shutdown_connections 2023-11-06T02:11:16.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.046+0000 7f3d43ba0640 1 --2- 172.21.15.49:0/1579306547 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3d3c0b1450 0x7f3d3c0cdd30 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:16.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.046+0000 7f3d43ba0640 1 --2- 172.21.15.49:0/1579306547 >> v2:172.21.15.150:3300/0 conn(0x7f3d3c159980 0x7f3d3c071290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:16.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.046+0000 7f3d43ba0640 1 -- 172.21.15.49:0/1579306547 >> 172.21.15.49:0/1579306547 conn(0x7f3d3c06c0a0 msgr2=0x7f3d3c06c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:16.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.046+0000 7f3d43ba0640 1 -- 172.21.15.49:0/1579306547 shutdown_connections 2023-11-06T02:11:16.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.046+0000 7f3d43ba0640 1 -- 172.21.15.49:0/1579306547 wait complete. 2023-11-06T02:11:16.049 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.046+0000 7f3d43ba0640 1 Processor -- start 2023-11-06T02:11:16.049 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.046+0000 7f3d43ba0640 1 -- start start 2023-11-06T02:11:16.049 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.050+0000 7f3d43ba0640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d3c0b1450 0x7f3d3c1530b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:16.049 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.050+0000 7f3d43ba0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d3c0cb010 0x7f3d3c0c97d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:16.050 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.050+0000 7f3d43ba0640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3d3c159980 0x7f3d3c0c9d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:16.050 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.050+0000 7f3d43ba0640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d3c152610 con 0x7f3d3c0cb010 2023-11-06T02:11:16.050 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.050+0000 7f3d43ba0640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d3c0ad010 con 0x7f3d3c0b1450 2023-11-06T02:11:16.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.050+0000 7f3d43ba0640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d3c0ad2f0 con 0x7f3d3c159980 2023-11-06T02:11:16.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.050+0000 7f3d4113b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d3c0cb010 0x7f3d3c0c97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:16.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.050+0000 7f3d4113b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d3c0cb010 0x7f3d3c0c97d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42666/0 (socket says 172.21.15.49:42666) 2023-11-06T02:11:16.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.050+0000 7f3d4113b640 1 -- 172.21.15.49:0/2602111474 learned_addr learned my addr 172.21.15.49:0/2602111474 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:16.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.050+0000 7f3d4213d640 1 --2- 172.21.15.49:0/2602111474 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3d3c159980 0x7f3d3c0c9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:16.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.050+0000 7f3d4193c640 1 --2- 172.21.15.49:0/2602111474 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d3c0b1450 0x7f3d3c1530b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:16.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.050+0000 7f3d4113b640 1 -- 172.21.15.49:0/2602111474 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3d3c159980 msgr2=0x7f3d3c0c9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:16.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.050+0000 7f3d4113b640 1 --2- 172.21.15.49:0/2602111474 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3d3c159980 0x7f3d3c0c9d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:16.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.050+0000 7f3d4113b640 1 -- 172.21.15.49:0/2602111474 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d3c0b1450 msgr2=0x7f3d3c1530b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:16.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.050+0000 7f3d4113b640 1 --2- 172.21.15.49:0/2602111474 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d3c0b1450 0x7f3d3c1530b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:16.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.050+0000 7f3d4113b640 1 -- 172.21.15.49:0/2602111474 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d34009b30 con 0x7f3d3c0cb010 2023-11-06T02:11:16.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.050+0000 7f3d4213d640 1 --2- 172.21.15.49:0/2602111474 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3d3c159980 0x7f3d3c0c9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:11:16.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.050+0000 7f3d4193c640 1 --2- 172.21.15.49:0/2602111474 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d3c0b1450 0x7f3d3c1530b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:11:16.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.050+0000 7f3d4113b640 1 --2- 172.21.15.49:0/2602111474 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d3c0cb010 0x7f3d3c0c97d0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f3d300ad040 tx=0x7f3d3001acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:16.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.050+0000 7f3d3a7fc640 1 -- 172.21.15.49:0/2602111474 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d300bf610 con 0x7f3d3c0cb010 2023-11-06T02:11:16.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.050+0000 7f3d3a7fc640 1 -- 172.21.15.49:0/2602111474 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d300bf790 con 0x7f3d3c0cb010 2023-11-06T02:11:16.054 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.050+0000 7f3d43ba0640 1 -- 172.21.15.49:0/2602111474 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d30009420 con 0x7f3d3c0cb010 2023-11-06T02:11:16.054 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.050+0000 7f3d43ba0640 1 -- 172.21.15.49:0/2602111474 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d3c0ce3f0 con 0x7f3d3c0cb010 2023-11-06T02:11:16.054 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.050+0000 7f3d3a7fc640 1 -- 172.21.15.49:0/2602111474 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d300bfb40 con 0x7f3d3c0cb010 2023-11-06T02:11:16.054 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.050+0000 7f3d43ba0640 1 -- 172.21.15.49:0/2602111474 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d1c005260 con 0x7f3d3c0cb010 2023-11-06T02:11:16.054 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.050+0000 7f3d3a7fc640 1 -- 172.21.15.49:0/2602111474 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f3d300bfca0 con 0x7f3d3c0cb010 2023-11-06T02:11:16.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.050+0000 7f3d3a7fc640 1 --2- 172.21.15.49:0/2602111474 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3d28065a60 0x7f3d28067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:16.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.050+0000 7f3d3a7fc640 1 -- 172.21.15.49:0/2602111474 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f3d300a8050 con 0x7f3d3c0cb010 2023-11-06T02:11:16.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.050+0000 7f3d4193c640 1 --2- 172.21.15.49:0/2602111474 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3d28065a60 0x7f3d28067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:16.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.054+0000 7f3d4193c640 1 --2- 172.21.15.49:0/2602111474 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3d28065a60 0x7f3d28067f20 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f3d34009fd0 tx=0x7f3d3400b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:16.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.054+0000 7f3d3a7fc640 1 -- 172.21.15.49:0/2602111474 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f3d300eb5b0 con 0x7f3d3c0cb010 2023-11-06T02:11:16.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.214+0000 7f3d43ba0640 1 -- 172.21.15.49:0/2602111474 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3d1c0057e0 con 0x7f3d3c0cb010 2023-11-06T02:11:16.214 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.218+0000 7f3d3a7fc640 1 -- 172.21.15.49:0/2602111474 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+17979 (secure 0 0 0) 0x7f3d300eebd0 con 0x7f3d3c0cb010 2023-11-06T02:11:16.215 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:11:16.215 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":53,"fsid":"1a8b035c-6ba4-499a-b876-aea47fd9c0d4","created":"2023-11-06T02:08:06.440857+0000","modified":"2023-11-06T02:11:15.186441+0000","last_up_change":"2023-11-06T02:08:17.982372+0000","last_in_change":"2023-11-06T02:08:08.906631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T02:08:19.252222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T02:08:24.776211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-06T02:10:30.458849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-06T02:10:30.899578+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ad55af8-0571-4e6a-8722-b5d429b7496d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6809","nonce":38116},{"type":"v1","addr":"172.21.15.49:6811","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6813","nonce":38116},{"type":"v1","addr":"172.21.15.49:6815","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6821","nonce":38116},{"type":"v1","addr":"172.21.15.49:6823","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6817","nonce":38116},{"type":"v1","addr":"172.21.15.49:6819","nonce":38116}]},"public_addr":"172.21.15.49:6811/38116","cluster_addr":"172.21.15.49:6815/38116","heartbeat_back_addr":"172.21.15.49:6823/38116","heartbeat_front_addr":"172.21.15.49:6819/38116","state":["exists","up"]},{"osd":1,"uuid":"c8b31f85-a757-43ed-9419-351acc46611b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":38113},{"type":"v1","addr":"172.21.15.49:6810","nonce":38113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":38113},{"type":"v1","addr":"172.21.15.49:6814","nonce":38113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":38113},{"type":"v1","addr":"172.21.15.49:6822","nonce":38113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":38113},{"type":"v1","addr":"172.21.15.49:6818","nonce":38113}]},"public_addr":"172.21.15.49:6810/38113","cluster_addr":"172.21.15.49:6814/38113","heartbeat_back_addr":"172.21.15.49:6822/38113","heartbeat_front_addr":"172.21.15.49:6818/38113","state":["exists","up"]},{"osd":2,"uuid":"555b3bf3-0a70-4b91-8efd-e4204b5fc26e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":38114},{"type":"v1","addr":"172.21.15.49:6801","nonce":38114}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":38114},{"type":"v1","addr":"172.21.15.49:6803","nonce":38114}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":38114},{"type":"v1","addr":"172.21.15.49:6807","nonce":38114}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":38114},{"type":"v1","addr":"172.21.15.49:6805","nonce":38114}]},"public_addr":"172.21.15.49:6801/38114","cluster_addr":"172.21.15.49:6803/38114","heartbeat_back_addr":"172.21.15.49:6807/38114","heartbeat_front_addr":"172.21.15.49:6805/38114","state":["exists","up"]},{"osd":3,"uuid":"8d4e5d77-03ef-4b9b-bfcb-5ae93a46795c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":38115},{"type":"v1","addr":"172.21.15.49:6825","nonce":38115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":38115},{"type":"v1","addr":"172.21.15.49:6827","nonce":38115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":38115},{"type":"v1","addr":"172.21.15.49:6831","nonce":38115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":38115},{"type":"v1","addr":"172.21.15.49:6829","nonce":38115}]},"public_addr":"172.21.15.49:6825/38115","cluster_addr":"172.21.15.49:6827/38115","heartbeat_back_addr":"172.21.15.49:6831/38115","heartbeat_front_addr":"172.21.15.49:6829/38115","state":["exists","up"]},{"osd":4,"uuid":"dba03ad4-68a3-4cf3-9218-ac611f02eacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":38417},{"type":"v1","addr":"172.21.15.150:6809","nonce":38417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":38417},{"type":"v1","addr":"172.21.15.150:6811","nonce":38417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":38417},{"type":"v1","addr":"172.21.15.150:6815","nonce":38417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":38417},{"type":"v1","addr":"172.21.15.150:6813","nonce":38417}]},"public_addr":"172.21.15.150:6809/38417","cluster_addr":"172.21.15.150:6811/38417","heartbeat_back_addr":"172.21.15.150:6815/38417","heartbeat_front_addr":"172.21.15.150:6813/38417","state":["exists","up"]},{"osd":5,"uuid":"0df9c303-c255-4640-8e91-01f1c3efe18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":38418},{"type":"v1","addr":"172.21.15.150:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":38418},{"type":"v1","addr":"172.21.15.150:6803","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":38418},{"type":"v1","addr":"172.21.15.150:6807","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":38418},{"type":"v1","addr":"172.21.15.150:6805","nonce":38418}]},"public_addr":"172.21.15.150:6801/38418","cluster_addr":"172.21.15.150:6803/38418","heartbeat_back_addr":"172.21.15.150:6807/38418","heartbeat_front_addr":"172.21.15.150:6805/38418","state":["exists","up"]},{"osd":6,"uuid":"e59fa189-4b70-4c06-9722-c02b90828bf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":38428},{"type":"v1","addr":"172.21.15.150:6825","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":38428},{"type":"v1","addr":"172.21.15.150:6827","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":38428},{"type":"v1","addr":"172.21.15.150:6831","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":38428},{"type":"v1","addr":"172.21.15.150:6829","nonce":38428}]},"public_addr":"172.21.15.150:6825/38428","cluster_addr":"172.21.15.150:6827/38428","heartbeat_back_addr":"172.21.15.150:6831/38428","heartbeat_front_addr":"172.21.15.150:6829/38428","state":["exists","up"]},{"osd":7,"uuid":"a64f57e1-ef98-46bd-b304-385759e95898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":38421},{"type":"v1","addr":"172.21.15.150:6817","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":38421},{"type":"v1","addr":"172.21.15.150:6819","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":38421},{"type":"v1","addr":"172.21.15.150:6823","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":38421},{"type":"v1","addr":"172.21.15.150:6821","nonce":38421}]},"public_addr":"172.21.15.150:6817/38421","cluster_addr":"172.21.15.150:6819/38421","heartbeat_back_addr":"172.21.15.150:6823/38421","heartbeat_front_addr":"172.21.15.150:6821/38421","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.438552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.413280+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.296466+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.424966+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.276953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.230984+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.410235+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.480084+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.37","mappings":[{"from":0,"to":6}]},{"pgid":"8.38","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T02:11:16.217 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.218+0000 7f3d43ba0640 1 -- 172.21.15.49:0/2602111474 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3d28065a60 msgr2=0x7f3d28067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:16.217 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.218+0000 7f3d43ba0640 1 --2- 172.21.15.49:0/2602111474 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3d28065a60 0x7f3d28067f20 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f3d34009fd0 tx=0x7f3d3400b040 comp rx=0 tx=0).stop 2023-11-06T02:11:16.218 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.218+0000 7f3d43ba0640 1 -- 172.21.15.49:0/2602111474 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d3c0cb010 msgr2=0x7f3d3c0c97d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:16.218 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.222+0000 7f3d43ba0640 1 --2- 172.21.15.49:0/2602111474 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d3c0cb010 0x7f3d3c0c97d0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f3d300ad040 tx=0x7f3d3001acc0 comp rx=0 tx=0).stop 2023-11-06T02:11:16.218 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.222+0000 7f3d43ba0640 1 -- 172.21.15.49:0/2602111474 shutdown_connections 2023-11-06T02:11:16.218 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.222+0000 7f3d43ba0640 1 --2- 172.21.15.49:0/2602111474 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3d28065a60 0x7f3d28067f20 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:16.219 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.222+0000 7f3d43ba0640 1 --2- 172.21.15.49:0/2602111474 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d3c0cb010 0x7f3d3c0c97d0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:16.219 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.222+0000 7f3d43ba0640 1 --2- 172.21.15.49:0/2602111474 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3d3c159980 0x7f3d3c0c9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:16.219 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.222+0000 7f3d43ba0640 1 --2- 172.21.15.49:0/2602111474 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d3c0b1450 0x7f3d3c1530b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:16.219 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.222+0000 7f3d43ba0640 1 -- 172.21.15.49:0/2602111474 >> 172.21.15.49:0/2602111474 conn(0x7f3d3c06c0a0 msgr2=0x7f3d3c154aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:16.220 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.222+0000 7f3d43ba0640 1 -- 172.21.15.49:0/2602111474 shutdown_connections 2023-11-06T02:11:16.220 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.222+0000 7f3d43ba0640 1 -- 172.21.15.49:0/2602111474 wait complete. 2023-11-06T02:11:16.234 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T02:08:19.252222+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T02:08:24.776211+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T02:10:30.458849+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '50', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T02:10:30.899578+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '51', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T02:11:16.235 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T02:11:16.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.366+0000 7faf00069640 1 Processor -- start 2023-11-06T02:11:16.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.366+0000 7faf00069640 1 -- start start 2023-11-06T02:11:16.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.366+0000 7faf00069640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7faef806d860 0x7faef80b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:16.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.366+0000 7faf00069640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faef80ae360 con 0x7faef8070870 2023-11-06T02:11:16.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.366+0000 7faf00069640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faef80ae640 con 0x7faef815a880 2023-11-06T02:11:16.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.366+0000 7faf00069640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faef80ae920 con 0x7faef806d860 2023-11-06T02:11:16.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.366+0000 7faefd604640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7faef8070870 0x7faef806d150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:60010/0 (socket says 172.21.15.49:60010) 2023-11-06T02:11:16.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.366+0000 7faefd604640 1 -- 172.21.15.49:0/635444765 learned_addr learned my addr 172.21.15.49:0/635444765 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:11:16.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.366+0000 7faefe606640 1 --2- 172.21.15.49:0/635444765 >> v2:172.21.15.49:3300/0 conn(0x7faef806d860 0x7faef80b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:16.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.366+0000 7faefce03640 1 -- 172.21.15.49:0/635444765 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1674370871 0 0) 0x7faef80ae360 con 0x7faef8070870 2023-11-06T02:11:16.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.366+0000 7faefce03640 1 -- 172.21.15.49:0/635444765 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faee4003530 con 0x7faef8070870 2023-11-06T02:11:16.365 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.366+0000 7faefe606640 1 -- 172.21.15.49:0/635444765 >> v1:172.21.15.150:6789/0 conn(0x7faef815a880 legacy=0x7faef8070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:16.365 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.366+0000 7faefe606640 1 -- 172.21.15.49:0/635444765 >> v1:172.21.15.49:6789/0 conn(0x7faef8070870 legacy=0x7faef806d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:16.365 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.366+0000 7faefe606640 1 -- 172.21.15.49:0/635444765 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faef80aec00 con 0x7faef806d860 2023-11-06T02:11:16.366 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.370+0000 7faefe606640 1 --2- 172.21.15.49:0/635444765 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faef806d860 0x7faef80b15c0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7faef00097c0 tx=0x7faef001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78880005ef54d79d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:16.367 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.370+0000 7faefce03640 1 -- 172.21.15.49:0/635444765 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef00ab600 con 0x7faef806d860 2023-11-06T02:11:16.367 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.370+0000 7faefce03640 1 -- 172.21.15.49:0/635444765 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faef00a7050 con 0x7faef806d860 2023-11-06T02:11:16.367 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.370+0000 7faefce03640 1 -- 172.21.15.49:0/635444765 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef00ab980 con 0x7faef806d860 2023-11-06T02:11:16.367 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.370+0000 7faf00069640 1 -- 172.21.15.49:0/635444765 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faef806d860 msgr2=0x7faef80b15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:16.368 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.370+0000 7faf00069640 1 --2- 172.21.15.49:0/635444765 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faef806d860 0x7faef80b15c0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7faef00097c0 tx=0x7faef001c3f0 comp rx=0 tx=0).stop 2023-11-06T02:11:16.368 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.370+0000 7faf00069640 1 -- 172.21.15.49:0/635444765 shutdown_connections 2023-11-06T02:11:16.368 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.370+0000 7faf00069640 1 --2- 172.21.15.49:0/635444765 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faef806d860 0x7faef80b15c0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:16.368 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.370+0000 7faf00069640 1 -- 172.21.15.49:0/635444765 >> 172.21.15.49:0/635444765 conn(0x7faef806a370 msgr2=0x7faef80ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:16.369 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.370+0000 7faf00069640 1 -- 172.21.15.49:0/635444765 shutdown_connections 2023-11-06T02:11:16.369 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.370+0000 7faf00069640 1 -- 172.21.15.49:0/635444765 wait complete. 2023-11-06T02:11:16.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.370+0000 7faf00069640 1 Processor -- start 2023-11-06T02:11:16.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.370+0000 7faf00069640 1 -- start start 2023-11-06T02:11:16.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.370+0000 7faf00069640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faef806d860 0x7faef80c3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:16.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.370+0000 7faf00069640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faef8070870 0x7faef814e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:16.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.370+0000 7faf00069640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faef815a880 0x7faef80cc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:16.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.370+0000 7faf00069640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faef80c51c0 con 0x7faef8070870 2023-11-06T02:11:16.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.370+0000 7faf00069640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faef80c45b0 con 0x7faef815a880 2023-11-06T02:11:16.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.370+0000 7faf00069640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faef80c4890 con 0x7faef806d860 2023-11-06T02:11:16.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.370+0000 7faefd604640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faef8070870 0x7faef814e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:16.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.374+0000 7faefd604640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faef8070870 0x7faef814e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42690/0 (socket says 172.21.15.49:42690) 2023-11-06T02:11:16.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.374+0000 7faefd604640 1 -- 172.21.15.49:0/3424214808 learned_addr learned my addr 172.21.15.49:0/3424214808 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:16.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.374+0000 7faefe606640 1 --2- 172.21.15.49:0/3424214808 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faef815a880 0x7faef80cc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:16.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.374+0000 7faefde05640 1 --2- 172.21.15.49:0/3424214808 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faef806d860 0x7faef80c3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:16.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.374+0000 7faefd604640 1 -- 172.21.15.49:0/3424214808 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faef806d860 msgr2=0x7faef80c3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:16.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.374+0000 7faefd604640 1 --2- 172.21.15.49:0/3424214808 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faef806d860 0x7faef80c3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:16.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.374+0000 7faefd604640 1 -- 172.21.15.49:0/3424214808 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faef815a880 msgr2=0x7faef80cc2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:16.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.374+0000 7faefd604640 1 --2- 172.21.15.49:0/3424214808 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faef815a880 0x7faef80cc2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:16.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.374+0000 7faefd604640 1 -- 172.21.15.49:0/3424214808 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faef0009420 con 0x7faef8070870 2023-11-06T02:11:16.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.374+0000 7faefe606640 1 --2- 172.21.15.49:0/3424214808 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faef815a880 0x7faef80cc2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:11:16.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.374+0000 7faefde05640 1 --2- 172.21.15.49:0/3424214808 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faef806d860 0x7faef80c3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:11:16.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.374+0000 7faefd604640 1 --2- 172.21.15.49:0/3424214808 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faef8070870 0x7faef814e2b0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7faee800c4d0 tx=0x7faee800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:16.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.374+0000 7faeee7fc640 1 -- 172.21.15.49:0/3424214808 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee8005cf0 con 0x7faef8070870 2023-11-06T02:11:16.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.374+0000 7faeee7fc640 1 -- 172.21.15.49:0/3424214808 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faee8005e50 con 0x7faef8070870 2023-11-06T02:11:16.378 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.374+0000 7faf00069640 1 -- 172.21.15.49:0/3424214808 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faef80c4b70 con 0x7faef8070870 2023-11-06T02:11:16.378 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.374+0000 7faeee7fc640 1 -- 172.21.15.49:0/3424214808 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee8007270 con 0x7faef8070870 2023-11-06T02:11:16.378 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.374+0000 7faf00069640 1 -- 172.21.15.49:0/3424214808 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faef814e7f0 con 0x7faef8070870 2023-11-06T02:11:16.378 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.374+0000 7faedbfff640 1 -- 172.21.15.49:0/3424214808 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faed4005260 con 0x7faef8070870 2023-11-06T02:11:16.378 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.374+0000 7faeee7fc640 1 -- 172.21.15.49:0/3424214808 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7faee80073d0 con 0x7faef8070870 2023-11-06T02:11:16.379 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.374+0000 7faeee7fc640 1 --2- 172.21.15.49:0/3424214808 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7faedc065a90 0x7faedc067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:16.379 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.374+0000 7faeee7fc640 1 -- 172.21.15.49:0/3424214808 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6120+0+0 (secure 0 0 0) 0x7faee8082800 con 0x7faef8070870 2023-11-06T02:11:16.379 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.374+0000 7faefde05640 1 --2- 172.21.15.49:0/3424214808 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7faedc065a90 0x7faedc067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:16.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.374+0000 7faefde05640 1 --2- 172.21.15.49:0/3424214808 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7faedc065a90 0x7faedc067f50 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7faef4007fd0 tx=0x7faef4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:16.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.378+0000 7faeee7fc640 1 -- 172.21.15.49:0/3424214808 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7faee80500c0 con 0x7faef8070870 2023-11-06T02:11:16.533 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.538+0000 7faedbfff640 1 -- 172.21.15.49:0/3424214808 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7faed4005a90 con 0x7faef8070870 2023-11-06T02:11:16.534 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.538+0000 7faeee7fc640 1 -- 172.21.15.49:0/3424214808 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v53) v1 ==== 93+0+10 (secure 0 0 0) 0x7faee80536e0 con 0x7faef8070870 2023-11-06T02:11:16.534 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2023-11-06T02:11:16.536 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.538+0000 7faedbfff640 1 -- 172.21.15.49:0/3424214808 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7faedc065a90 msgr2=0x7faedc067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:16.536 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.538+0000 7faedbfff640 1 --2- 172.21.15.49:0/3424214808 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7faedc065a90 0x7faedc067f50 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7faef4007fd0 tx=0x7faef4009040 comp rx=0 tx=0).stop 2023-11-06T02:11:16.537 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.538+0000 7faedbfff640 1 -- 172.21.15.49:0/3424214808 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faef8070870 msgr2=0x7faef814e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:16.537 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.538+0000 7faedbfff640 1 --2- 172.21.15.49:0/3424214808 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faef8070870 0x7faef814e2b0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7faee800c4d0 tx=0x7faee800c990 comp rx=0 tx=0).stop 2023-11-06T02:11:16.537 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.538+0000 7faedbfff640 1 -- 172.21.15.49:0/3424214808 shutdown_connections 2023-11-06T02:11:16.537 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.538+0000 7faedbfff640 1 --2- 172.21.15.49:0/3424214808 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7faedc065a90 0x7faedc067f50 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:16.537 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.538+0000 7faedbfff640 1 --2- 172.21.15.49:0/3424214808 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faef8070870 0x7faef814e2b0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:16.538 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.538+0000 7faedbfff640 1 --2- 172.21.15.49:0/3424214808 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faef815a880 0x7faef80cc2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:16.538 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.538+0000 7faedbfff640 1 --2- 172.21.15.49:0/3424214808 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faef806d860 0x7faef80c3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:16.538 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.538+0000 7faedbfff640 1 -- 172.21.15.49:0/3424214808 >> 172.21.15.49:0/3424214808 conn(0x7faef806a370 msgr2=0x7faef81520b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:16.538 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.538+0000 7faedbfff640 1 -- 172.21.15.49:0/3424214808 shutdown_connections 2023-11-06T02:11:16.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.538+0000 7faedbfff640 1 -- 172.21.15.49:0/3424214808 wait complete. 2023-11-06T02:11:16.555 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T02:11:16.677 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.682+0000 7fd72caba640 1 Processor -- start 2023-11-06T02:11:16.677 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.682+0000 7fd72caba640 1 -- start start 2023-11-06T02:11:16.679 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.682+0000 7fd72caba640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd72815a880 0x7fd7280b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:16.679 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.682+0000 7fd72caba640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7280ae360 con 0x7fd72806d860 2023-11-06T02:11:16.679 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.682+0000 7fd72caba640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7280ae640 con 0x7fd728070160 2023-11-06T02:11:16.679 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.682+0000 7fd72caba640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7280ae920 con 0x7fd72815a880 2023-11-06T02:11:16.679 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.682+0000 7fd72659c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd72815a880 0x7fd7280b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:16.680 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.682+0000 7fd726d9d640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fd72806d860 0x7fd7280af120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:60026/0 (socket says 172.21.15.49:60026) 2023-11-06T02:11:16.680 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.682+0000 7fd726d9d640 1 -- 172.21.15.49:0/4114327430 learned_addr learned my addr 172.21.15.49:0/4114327430 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:11:16.680 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.682+0000 7fd72659c640 1 -- 172.21.15.49:0/4114327430 >> v1:172.21.15.150:6789/0 conn(0x7fd728070160 legacy=0x7fd72806d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:16.680 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.682+0000 7fd72659c640 1 -- 172.21.15.49:0/4114327430 >> v1:172.21.15.49:6789/0 conn(0x7fd72806d860 legacy=0x7fd7280af120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:16.680 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.682+0000 7fd72659c640 1 -- 172.21.15.49:0/4114327430 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7280aec00 con 0x7fd72815a880 2023-11-06T02:11:16.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.686+0000 7fd72659c640 1 --2- 172.21.15.49:0/4114327430 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd72815a880 0x7fd7280b15c0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fd7180097c0 tx=0x7fd71801c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaf80c4eda03befe server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:16.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.686+0000 7fd72559a640 1 -- 172.21.15.49:0/4114327430 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd718004230 con 0x7fd72815a880 2023-11-06T02:11:16.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.686+0000 7fd72559a640 1 -- 172.21.15.49:0/4114327430 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7180a7050 con 0x7fd72815a880 2023-11-06T02:11:16.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.686+0000 7fd72559a640 1 -- 172.21.15.49:0/4114327430 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd718003620 con 0x7fd72815a880 2023-11-06T02:11:16.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.686+0000 7fd72caba640 1 -- 172.21.15.49:0/4114327430 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd72815a880 msgr2=0x7fd7280b15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:16.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.686+0000 7fd72caba640 1 --2- 172.21.15.49:0/4114327430 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd72815a880 0x7fd7280b15c0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fd7180097c0 tx=0x7fd71801c440 comp rx=0 tx=0).stop 2023-11-06T02:11:16.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.686+0000 7fd72caba640 1 -- 172.21.15.49:0/4114327430 shutdown_connections 2023-11-06T02:11:16.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.686+0000 7fd72caba640 1 --2- 172.21.15.49:0/4114327430 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd72815a880 0x7fd7280b15c0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:16.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.686+0000 7fd72caba640 1 -- 172.21.15.49:0/4114327430 >> 172.21.15.49:0/4114327430 conn(0x7fd72806a370 msgr2=0x7fd7280ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:16.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.686+0000 7fd72caba640 1 -- 172.21.15.49:0/4114327430 shutdown_connections 2023-11-06T02:11:16.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.686+0000 7fd72caba640 1 -- 172.21.15.49:0/4114327430 wait complete. 2023-11-06T02:11:16.686 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.686+0000 7fd72caba640 1 Processor -- start 2023-11-06T02:11:16.686 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.686+0000 7fd72caba640 1 -- start start 2023-11-06T02:11:16.686 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.686+0000 7fd72caba640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd72806d860 0x7fd7280c5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:16.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.686+0000 7fd72caba640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd728070160 0x7fd7280c3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:16.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.686+0000 7fd72caba640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd72815a880 0x7fd7280c40a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:16.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.686+0000 7fd72caba640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7280cc120 con 0x7fd728070160 2023-11-06T02:11:16.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.686+0000 7fd72caba640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7280cc400 con 0x7fd72815a880 2023-11-06T02:11:16.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.686+0000 7fd72caba640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7280cc6e0 con 0x7fd72806d860 2023-11-06T02:11:16.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.686+0000 7fd725d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd728070160 0x7fd7280c3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:16.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.686+0000 7fd725d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd728070160 0x7fd7280c3530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42702/0 (socket says 172.21.15.49:42702) 2023-11-06T02:11:16.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.686+0000 7fd725d9b640 1 -- 172.21.15.49:0/105987932 learned_addr learned my addr 172.21.15.49:0/105987932 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:16.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.690+0000 7fd725d9b640 1 -- 172.21.15.49:0/105987932 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd72806d860 msgr2=0x7fd7280c5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:16.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.690+0000 7fd72659c640 1 --2- 172.21.15.49:0/105987932 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd72806d860 0x7fd7280c5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:16.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.690+0000 7fd726d9d640 1 --2- 172.21.15.49:0/105987932 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd72815a880 0x7fd7280c40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:16.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.690+0000 7fd725d9b640 1 --2- 172.21.15.49:0/105987932 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd72806d860 0x7fd7280c5db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:16.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.690+0000 7fd725d9b640 1 -- 172.21.15.49:0/105987932 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd72815a880 msgr2=0x7fd7280c40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:16.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.690+0000 7fd725d9b640 1 --2- 172.21.15.49:0/105987932 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd72815a880 0x7fd7280c40a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:16.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.690+0000 7fd725d9b640 1 -- 172.21.15.49:0/105987932 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd718009420 con 0x7fd728070160 2023-11-06T02:11:16.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.690+0000 7fd725d9b640 1 --2- 172.21.15.49:0/105987932 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd728070160 0x7fd7280c3530 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fd71c00c450 tx=0x7fd71c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:16.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.690+0000 7fd716ffd640 1 -- 172.21.15.49:0/105987932 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd71c012830 con 0x7fd728070160 2023-11-06T02:11:16.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.690+0000 7fd716ffd640 1 -- 172.21.15.49:0/105987932 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd71c0129b0 con 0x7fd728070160 2023-11-06T02:11:16.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.690+0000 7fd72caba640 1 -- 172.21.15.49:0/105987932 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7280cc9c0 con 0x7fd728070160 2023-11-06T02:11:16.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.690+0000 7fd716ffd640 1 -- 172.21.15.49:0/105987932 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd71c012d60 con 0x7fd728070160 2023-11-06T02:11:16.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.690+0000 7fd72caba640 1 -- 172.21.15.49:0/105987932 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7280c3a70 con 0x7fd728070160 2023-11-06T02:11:16.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.690+0000 7fd72caba640 1 -- 172.21.15.49:0/105987932 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7280ccee0 con 0x7fd728070160 2023-11-06T02:11:16.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.690+0000 7fd716ffd640 1 -- 172.21.15.49:0/105987932 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fd71c01c460 con 0x7fd728070160 2023-11-06T02:11:16.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.690+0000 7fd716ffd640 1 --2- 172.21.15.49:0/105987932 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd708065a60 0x7fd708067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:16.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.690+0000 7fd716ffd640 1 -- 172.21.15.49:0/105987932 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fd71c083110 con 0x7fd728070160 2023-11-06T02:11:16.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.690+0000 7fd72659c640 1 --2- 172.21.15.49:0/105987932 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd708065a60 0x7fd708067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:16.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.690+0000 7fd72659c640 1 --2- 172.21.15.49:0/105987932 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd708065a60 0x7fd708067f20 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fd7180ad000 tx=0x7fd718008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:16.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.694+0000 7fd716ffd640 1 -- 172.21.15.49:0/105987932 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fd7280ccee0 con 0x7fd728070160 2023-11-06T02:11:16.839 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.842+0000 7fd72caba640 1 -- 172.21.15.49:0/105987932 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd7280ccee0 con 0x7fd728070160 2023-11-06T02:11:16.840 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.842+0000 7fd716ffd640 1 -- 172.21.15.49:0/105987932 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v53) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd7280ccee0 con 0x7fd728070160 2023-11-06T02:11:16.840 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 8 2023-11-06T02:11:16.842 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.846+0000 7fd72caba640 1 -- 172.21.15.49:0/105987932 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd708065a60 msgr2=0x7fd708067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:16.843 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.846+0000 7fd72caba640 1 --2- 172.21.15.49:0/105987932 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd708065a60 0x7fd708067f20 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fd7180ad000 tx=0x7fd718008040 comp rx=0 tx=0).stop 2023-11-06T02:11:16.843 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.846+0000 7fd72caba640 1 -- 172.21.15.49:0/105987932 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd728070160 msgr2=0x7fd7280c3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:16.843 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.846+0000 7fd72caba640 1 --2- 172.21.15.49:0/105987932 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd728070160 0x7fd7280c3530 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fd71c00c450 tx=0x7fd71c00c910 comp rx=0 tx=0).stop 2023-11-06T02:11:16.843 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.846+0000 7fd72caba640 1 -- 172.21.15.49:0/105987932 shutdown_connections 2023-11-06T02:11:16.843 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.846+0000 7fd72caba640 1 --2- 172.21.15.49:0/105987932 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd708065a60 0x7fd708067f20 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:16.844 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.846+0000 7fd72caba640 1 --2- 172.21.15.49:0/105987932 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd728070160 0x7fd7280c3530 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:16.844 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.846+0000 7fd72caba640 1 --2- 172.21.15.49:0/105987932 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd72815a880 0x7fd7280c40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:16.844 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.846+0000 7fd72caba640 1 --2- 172.21.15.49:0/105987932 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd72806d860 0x7fd7280c5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:16.844 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.846+0000 7fd72caba640 1 -- 172.21.15.49:0/105987932 >> 172.21.15.49:0/105987932 conn(0x7fd72806a370 msgr2=0x7fd72815ae70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:16.844 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.846+0000 7fd72caba640 1 -- 172.21.15.49:0/105987932 shutdown_connections 2023-11-06T02:11:16.845 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.846+0000 7fd72caba640 1 -- 172.21.15.49:0/105987932 wait complete. 2023-11-06T02:11:16.859 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T02:11:16.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.982+0000 7fc5b5a70640 1 Processor -- start 2023-11-06T02:11:16.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.982+0000 7fc5b5a70640 1 -- start start 2023-11-06T02:11:16.980 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.982+0000 7fc5b5a70640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc5b014ffc0 0x7fc5b00ada30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:16.980 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.982+0000 7fc5b5a70640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc5b0070870 0x7fc5b006c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:16.980 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.982+0000 7fc5b5a70640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5b00c8c80 con 0x7fc5b00adf70 2023-11-06T02:11:16.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.982+0000 7fc5b5a70640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5b00ae4d0 con 0x7fc5b0070870 2023-11-06T02:11:16.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.982+0000 7fc5b5a70640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5b01594f0 con 0x7fc5b014ffc0 2023-11-06T02:11:16.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.982+0000 7fc5affff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc5b0070870 0x7fc5b006c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:16.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.982+0000 7fc5aeffd640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fc5b00adf70 0x7fc5b0070160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:60038/0 (socket says 172.21.15.49:60038) 2023-11-06T02:11:16.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.982+0000 7fc5aeffd640 1 -- 172.21.15.49:0/1123661313 learned_addr learned my addr 172.21.15.49:0/1123661313 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:11:16.983 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.982+0000 7fc5af7fe640 1 --2- 172.21.15.49:0/1123661313 >> v2:172.21.15.150:3300/0 conn(0x7fc5b014ffc0 0x7fc5b00ada30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:16.983 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.986+0000 7fc5affff640 1 -- 172.21.15.49:0/1123661313 >> v2:172.21.15.150:3300/0 conn(0x7fc5b014ffc0 msgr2=0x7fc5b00ada30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:16.983 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.986+0000 7fc5affff640 1 --2- 172.21.15.49:0/1123661313 >> v2:172.21.15.150:3300/0 conn(0x7fc5b014ffc0 0x7fc5b00ada30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:16.983 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.986+0000 7fc5affff640 1 -- 172.21.15.49:0/1123661313 >> v1:172.21.15.49:6789/0 conn(0x7fc5b00adf70 legacy=0x7fc5b0070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:16.984 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.986+0000 7fc5ae7fc640 1 -- 172.21.15.49:0/1123661313 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 900028456 0 0) 0x7fc5b00c8c80 con 0x7fc5b00adf70 2023-11-06T02:11:16.984 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.986+0000 7fc5affff640 1 -- 172.21.15.49:0/1123661313 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5b0159700 con 0x7fc5b0070870 2023-11-06T02:11:16.984 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.986+0000 7fc5af7fe640 1 --2- 172.21.15.49:0/1123661313 >> v2:172.21.15.150:3300/0 conn(0x7fc5b014ffc0 0x7fc5b00ada30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:11:16.984 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.986+0000 7fc5affff640 1 --2- 172.21.15.49:0/1123661313 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5b0070870 0x7fc5b006c230 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fc5b006e390 tx=0x7fc5a801eb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2acaa3060fffe0ed server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:16.984 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.986+0000 7fc5ae7fc640 1 -- 172.21.15.49:0/1123661313 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a80a7a50 con 0x7fc5b0070870 2023-11-06T02:11:16.985 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.986+0000 7fc5ae7fc640 1 -- 172.21.15.49:0/1123661313 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5a80ab070 con 0x7fc5b0070870 2023-11-06T02:11:16.985 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.986+0000 7fc5ae7fc640 1 -- 172.21.15.49:0/1123661313 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a80a7dd0 con 0x7fc5b0070870 2023-11-06T02:11:16.988 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.986+0000 7fc5b5a70640 1 -- 172.21.15.49:0/1123661313 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5b0070870 msgr2=0x7fc5b006c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:16.988 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.986+0000 7fc5b5a70640 1 --2- 172.21.15.49:0/1123661313 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5b0070870 0x7fc5b006c230 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fc5b006e390 tx=0x7fc5a801eb10 comp rx=0 tx=0).stop 2023-11-06T02:11:16.988 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.986+0000 7fc5b5a70640 1 -- 172.21.15.49:0/1123661313 shutdown_connections 2023-11-06T02:11:16.988 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.986+0000 7fc5b5a70640 1 --2- 172.21.15.49:0/1123661313 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5b0070870 0x7fc5b006c230 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:16.988 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.986+0000 7fc5b5a70640 1 --2- 172.21.15.49:0/1123661313 >> v2:172.21.15.150:3300/0 conn(0x7fc5b014ffc0 0x7fc5b00ada30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:16.989 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.986+0000 7fc5b5a70640 1 -- 172.21.15.49:0/1123661313 >> 172.21.15.49:0/1123661313 conn(0x7fc5b006a370 msgr2=0x7fc5b00ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:16.989 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.986+0000 7fc5b5a70640 1 -- 172.21.15.49:0/1123661313 shutdown_connections 2023-11-06T02:11:16.989 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.986+0000 7fc5b5a70640 1 -- 172.21.15.49:0/1123661313 wait complete. 2023-11-06T02:11:16.989 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.990+0000 7fc5b5a70640 1 Processor -- start 2023-11-06T02:11:16.989 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.990+0000 7fc5b5a70640 1 -- start start 2023-11-06T02:11:16.990 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.990+0000 7fc5b5a70640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc5b0070870 0x7fc5b0152f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:16.990 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.990+0000 7fc5b5a70640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5b00adf70 0x7fc5b014b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:16.990 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.990+0000 7fc5b5a70640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc5b014ffc0 0x7fc5b01495d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:16.990 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.990+0000 7fc5b5a70640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b014a140 con 0x7fc5b00adf70 2023-11-06T02:11:16.990 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.990+0000 7fc5b5a70640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b014a420 con 0x7fc5b014ffc0 2023-11-06T02:11:16.991 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.990+0000 7fc5b5a70640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b014a700 con 0x7fc5b0070870 2023-11-06T02:11:16.991 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.990+0000 7fc5aeffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5b00adf70 0x7fc5b014b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:16.991 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.990+0000 7fc5aeffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5b00adf70 0x7fc5b014b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42720/0 (socket says 172.21.15.49:42720) 2023-11-06T02:11:16.991 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.990+0000 7fc5aeffd640 1 -- 172.21.15.49:0/387928135 learned_addr learned my addr 172.21.15.49:0/387928135 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:16.991 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.990+0000 7fc5affff640 1 --2- 172.21.15.49:0/387928135 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc5b014ffc0 0x7fc5b01495d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:16.992 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.990+0000 7fc5aeffd640 1 -- 172.21.15.49:0/387928135 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc5b0070870 msgr2=0x7fc5b0152f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:16.992 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.990+0000 7fc5af7fe640 1 --2- 172.21.15.49:0/387928135 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc5b0070870 0x7fc5b0152f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:16.992 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.990+0000 7fc5aeffd640 1 --2- 172.21.15.49:0/387928135 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc5b0070870 0x7fc5b0152f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:16.992 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.990+0000 7fc5aeffd640 1 -- 172.21.15.49:0/387928135 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc5b014ffc0 msgr2=0x7fc5b01495d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:16.992 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.990+0000 7fc5aeffd640 1 --2- 172.21.15.49:0/387928135 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc5b014ffc0 0x7fc5b01495d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:16.993 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.990+0000 7fc5aeffd640 1 -- 172.21.15.49:0/387928135 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5a800bc80 con 0x7fc5b00adf70 2023-11-06T02:11:16.993 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.990+0000 7fc5affff640 1 --2- 172.21.15.49:0/387928135 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc5b014ffc0 0x7fc5b01495d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:11:16.993 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.990+0000 7fc5aeffd640 1 --2- 172.21.15.49:0/387928135 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5b00adf70 0x7fc5b014b1e0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fc5a40011f0 tx=0x7fc5a400c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:16.993 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.990+0000 7fc597fff640 1 -- 172.21.15.49:0/387928135 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a40074e0 con 0x7fc5b00adf70 2023-11-06T02:11:16.994 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.990+0000 7fc597fff640 1 -- 172.21.15.49:0/387928135 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5a4007640 con 0x7fc5b00adf70 2023-11-06T02:11:16.994 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.990+0000 7fc597fff640 1 -- 172.21.15.49:0/387928135 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a4004a10 con 0x7fc5b00adf70 2023-11-06T02:11:16.994 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.990+0000 7fc5b5a70640 1 -- 172.21.15.49:0/387928135 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5b014a9e0 con 0x7fc5b00adf70 2023-11-06T02:11:16.994 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.990+0000 7fc5b5a70640 1 -- 172.21.15.49:0/387928135 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5b014be00 con 0x7fc5b00adf70 2023-11-06T02:11:16.994 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.990+0000 7fc5b5a70640 1 -- 172.21.15.49:0/387928135 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5b014af00 con 0x7fc5b00adf70 2023-11-06T02:11:16.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.990+0000 7fc597fff640 1 -- 172.21.15.49:0/387928135 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fc5a4004b70 con 0x7fc5b00adf70 2023-11-06T02:11:16.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.994+0000 7fc597fff640 1 --2- 172.21.15.49:0/387928135 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc59c065a60 0x7fc59c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:16.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.994+0000 7fc597fff640 1 -- 172.21.15.49:0/387928135 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fc5a4082630 con 0x7fc5b00adf70 2023-11-06T02:11:16.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.994+0000 7fc5af7fe640 1 --2- 172.21.15.49:0/387928135 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc59c065a60 0x7fc59c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:16.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.994+0000 7fc5af7fe640 1 --2- 172.21.15.49:0/387928135 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc59c065a60 0x7fc59c067f20 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc5a0006fd0 tx=0x7fc5a0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:16.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:16.998+0000 7fc597fff640 1 -- 172.21.15.49:0/387928135 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fc5b014af00 con 0x7fc5b00adf70 2023-11-06T02:11:17.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.150+0000 7fc5b5a70640 1 -- 172.21.15.49:0/387928135 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc5b014af00 con 0x7fc5b00adf70 2023-11-06T02:11:17.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.150+0000 7fc597fff640 1 -- 172.21.15.49:0/387928135 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v53) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc5b014af00 con 0x7fc5b00adf70 2023-11-06T02:11:17.149 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2023-11-06T02:11:17.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.154+0000 7fc5b5a70640 1 -- 172.21.15.49:0/387928135 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc59c065a60 msgr2=0x7fc59c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:17.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.154+0000 7fc5b5a70640 1 --2- 172.21.15.49:0/387928135 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc59c065a60 0x7fc59c067f20 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc5a0006fd0 tx=0x7fc5a0008040 comp rx=0 tx=0).stop 2023-11-06T02:11:17.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.154+0000 7fc5b5a70640 1 -- 172.21.15.49:0/387928135 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5b00adf70 msgr2=0x7fc5b014b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:17.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.154+0000 7fc5b5a70640 1 --2- 172.21.15.49:0/387928135 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5b00adf70 0x7fc5b014b1e0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fc5a40011f0 tx=0x7fc5a400c9b0 comp rx=0 tx=0).stop 2023-11-06T02:11:17.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.154+0000 7fc5b5a70640 1 -- 172.21.15.49:0/387928135 shutdown_connections 2023-11-06T02:11:17.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.154+0000 7fc5b5a70640 1 --2- 172.21.15.49:0/387928135 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc59c065a60 0x7fc59c067f20 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:17.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.154+0000 7fc5b5a70640 1 --2- 172.21.15.49:0/387928135 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5b00adf70 0x7fc5b014b1e0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:17.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.154+0000 7fc5b5a70640 1 --2- 172.21.15.49:0/387928135 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc5b014ffc0 0x7fc5b01495d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:17.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.154+0000 7fc5b5a70640 1 --2- 172.21.15.49:0/387928135 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc5b0070870 0x7fc5b0152f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:17.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.154+0000 7fc5b5a70640 1 -- 172.21.15.49:0/387928135 >> 172.21.15.49:0/387928135 conn(0x7fc5b006a370 msgr2=0x7fc5b00cd150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:17.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.154+0000 7fc5b5a70640 1 -- 172.21.15.49:0/387928135 shutdown_connections 2023-11-06T02:11:17.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.154+0000 7fc5b5a70640 1 -- 172.21.15.49:0/387928135 wait complete. 2023-11-06T02:11:17.167 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T02:11:17.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.302+0000 7f3cc9ba2640 1 Processor -- start 2023-11-06T02:11:17.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.302+0000 7f3cc9ba2640 1 -- start start 2023-11-06T02:11:17.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.302+0000 7f3cc9ba2640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3cc4070870 0x7f3cc40b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:17.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.302+0000 7f3cc9ba2640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cc40ae360 con 0x7f3cc415a880 2023-11-06T02:11:17.303 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.302+0000 7f3cc9ba2640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cc40ae640 con 0x7f3cc406d150 2023-11-06T02:11:17.303 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.302+0000 7f3cc9ba2640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc40ae920 con 0x7f3cc4070870 2023-11-06T02:11:17.303 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.302+0000 7f3cc37fe640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f3cc415a880 0x7f3cc4070160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:60050/0 (socket says 172.21.15.49:60050) 2023-11-06T02:11:17.304 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.302+0000 7f3cc37fe640 1 -- 172.21.15.49:0/2457571881 learned_addr learned my addr 172.21.15.49:0/2457571881 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:11:17.304 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.302+0000 7f3cc2ffd640 1 --2- 172.21.15.49:0/2457571881 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3cc4070870 0x7f3cc40b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:17.304 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.302+0000 7f3cc27fc640 1 -- 172.21.15.49:0/2457571881 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1718197580 0 0) 0x7f3cc40ae360 con 0x7f3cc415a880 2023-11-06T02:11:17.305 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.302+0000 7f3cc27fc640 1 -- 172.21.15.49:0/2457571881 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3cac003530 con 0x7f3cc415a880 2023-11-06T02:11:17.305 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.302+0000 7f3cc27fc640 1 -- 172.21.15.49:0/2457571881 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 177518078 0 0) 0x7f3cac003530 con 0x7f3cc415a880 2023-11-06T02:11:17.305 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.302+0000 7f3cc27fc640 1 -- 172.21.15.49:0/2457571881 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3cc4070870 msgr2=0x7f3cc40b15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:17.305 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.302+0000 7f3cc27fc640 1 --2- 172.21.15.49:0/2457571881 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3cc4070870 0x7f3cc40b15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:17.305 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.302+0000 7f3cc27fc640 1 -- 172.21.15.49:0/2457571881 >> v1:172.21.15.150:6789/0 conn(0x7f3cc406d150 legacy=0x7f3cc406d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:17.306 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.302+0000 7f3cc27fc640 1 -- 172.21.15.49:0/2457571881 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cc40aec00 con 0x7f3cc415a880 2023-11-06T02:11:17.306 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.302+0000 7f3cc27fc640 1 -- 172.21.15.49:0/2457571881 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7f3cbc001680 con 0x7f3cc415a880 2023-11-06T02:11:17.306 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.302+0000 7f3cc27fc640 1 -- 172.21.15.49:0/2457571881 >> v1:172.21.15.49:6789/0 conn(0x7f3cc415a880 legacy=0x7f3cc4070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:17.306 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.302+0000 7f3cc27fc640 1 --2- 172.21.15.49:0/2457571881 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3cc4070870 0x7f3cac0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:17.307 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.302+0000 7f3cc27fc640 1 --2- 172.21.15.49:0/2457571881 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3cac003f80 0x7f3cac0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:17.307 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.302+0000 7f3cc27fc640 1 --2- 172.21.15.49:0/2457571881 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3cac004d10 0x7f3cac005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:17.307 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.302+0000 7f3cc27fc640 1 -- 172.21.15.49:0/2457571881 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc40ae920 con 0x7f3cac003f80 2023-11-06T02:11:17.307 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.302+0000 7f3cc27fc640 1 -- 172.21.15.49:0/2457571881 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cac00bad0 con 0x7f3cac004d10 2023-11-06T02:11:17.308 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.302+0000 7f3cc27fc640 1 -- 172.21.15.49:0/2457571881 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cac00bc90 con 0x7f3cc4070870 2023-11-06T02:11:17.308 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.302+0000 7f3cc2ffd640 1 --2- 172.21.15.49:0/2457571881 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3cac003f80 0x7f3cac0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:17.308 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.302+0000 7f3cc37fe640 1 --2- 172.21.15.49:0/2457571881 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3cc4070870 0x7f3cac0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:17.308 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.302+0000 7f3cc3fff640 1 --2- 172.21.15.49:0/2457571881 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3cac004d10 0x7f3cac005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:17.309 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.302+0000 7f3cc2ffd640 1 -- 172.21.15.49:0/2457571881 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3cc4070870 msgr2=0x7f3cac0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:17.309 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.302+0000 7f3cc2ffd640 1 --2- 172.21.15.49:0/2457571881 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3cc4070870 0x7f3cac0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:17.309 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.302+0000 7f3cc2ffd640 1 -- 172.21.15.49:0/2457571881 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3cac004d10 msgr2=0x7f3cac005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:17.309 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.302+0000 7f3cc2ffd640 1 --2- 172.21.15.49:0/2457571881 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3cac004d10 0x7f3cac005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:17.310 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.302+0000 7f3cc2ffd640 1 -- 172.21.15.49:0/2457571881 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cac003530 con 0x7f3cac003f80 2023-11-06T02:11:17.310 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.302+0000 7f3cc37fe640 1 --2- 172.21.15.49:0/2457571881 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3cc4070870 0x7f3cac0039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:11:17.310 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.306+0000 7f3cc2ffd640 1 --2- 172.21.15.49:0/2457571881 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3cac003f80 0x7f3cac0047d0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f3cb8009fc0 tx=0x7f3cb801cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2baa3770b993481b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:17.311 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.306+0000 7f3cc27fc640 1 -- 172.21.15.49:0/2457571881 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb80a5aa0 con 0x7f3cac003f80 2023-11-06T02:11:17.311 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.306+0000 7f3cc27fc640 1 -- 172.21.15.49:0/2457571881 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cb800f040 con 0x7f3cac003f80 2023-11-06T02:11:17.311 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.306+0000 7f3cc27fc640 1 -- 172.21.15.49:0/2457571881 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb80a5e40 con 0x7f3cac003f80 2023-11-06T02:11:17.315 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.306+0000 7f3cc9ba2640 1 -- 172.21.15.49:0/2457571881 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3cac003f80 msgr2=0x7f3cac0047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:17.315 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.306+0000 7f3cc9ba2640 1 --2- 172.21.15.49:0/2457571881 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3cac003f80 0x7f3cac0047d0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f3cb8009fc0 tx=0x7f3cb801cba0 comp rx=0 tx=0).stop 2023-11-06T02:11:17.315 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.306+0000 7f3cc3fff640 1 -- 172.21.15.49:0/2457571881 reap_dead start 2023-11-06T02:11:17.315 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.306+0000 7f3cc9ba2640 1 -- 172.21.15.49:0/2457571881 shutdown_connections 2023-11-06T02:11:17.316 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.306+0000 7f3cc9ba2640 1 -- 172.21.15.49:0/2457571881 >> 172.21.15.49:0/2457571881 conn(0x7f3cc406a370 msgr2=0x7f3cc40ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:17.316 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.306+0000 7f3cc9ba2640 1 -- 172.21.15.49:0/2457571881 shutdown_connections 2023-11-06T02:11:17.316 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.306+0000 7f3cc9ba2640 1 -- 172.21.15.49:0/2457571881 wait complete. 2023-11-06T02:11:17.316 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.306+0000 7f3cc9ba2640 1 Processor -- start 2023-11-06T02:11:17.316 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.306+0000 7f3cc9ba2640 1 -- start start 2023-11-06T02:11:17.317 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.310+0000 7f3cc9ba2640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3cac003f80 0x7f3cc414e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:17.317 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.310+0000 7f3cc9ba2640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3cc415d740 0x7f3cc414f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:17.317 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.310+0000 7f3cc9ba2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3cc414c4d0 0x7f3cc4150ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:17.317 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.310+0000 7f3cc9ba2640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc415bb80 con 0x7f3cc414c4d0 2023-11-06T02:11:17.318 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.310+0000 7f3cc9ba2640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc415af70 con 0x7f3cac003f80 2023-11-06T02:11:17.318 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.310+0000 7f3cc9ba2640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc415b250 con 0x7f3cc415d740 2023-11-06T02:11:17.318 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.310+0000 7f3cc3fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3cc414c4d0 0x7f3cc4150ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:17.318 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.310+0000 7f3cc3fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3cc414c4d0 0x7f3cc4150ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42730/0 (socket says 172.21.15.49:42730) 2023-11-06T02:11:17.318 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.310+0000 7f3cc3fff640 1 -- 172.21.15.49:0/868454502 learned_addr learned my addr 172.21.15.49:0/868454502 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:17.319 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.310+0000 7f3cc3fff640 1 -- 172.21.15.49:0/868454502 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3cc415d740 msgr2=0x7f3cc414f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:17.319 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.310+0000 7f3cc37fe640 1 --2- 172.21.15.49:0/868454502 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3cac003f80 0x7f3cc414e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:17.319 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.310+0000 7f3cc2ffd640 1 --2- 172.21.15.49:0/868454502 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3cc415d740 0x7f3cc414f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:17.319 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.310+0000 7f3cc3fff640 1 --2- 172.21.15.49:0/868454502 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3cc415d740 0x7f3cc414f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:17.319 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.310+0000 7f3cc3fff640 1 -- 172.21.15.49:0/868454502 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3cac003f80 msgr2=0x7f3cc414e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:17.319 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.310+0000 7f3cc3fff640 1 --2- 172.21.15.49:0/868454502 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3cac003f80 0x7f3cc414e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:17.320 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.310+0000 7f3cc3fff640 1 -- 172.21.15.49:0/868454502 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cc415ade0 con 0x7f3cc414c4d0 2023-11-06T02:11:17.320 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.310+0000 7f3cc3fff640 1 --2- 172.21.15.49:0/868454502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3cc414c4d0 0x7f3cc4150ae0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f3cb4000c10 tx=0x7f3cb400e820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:17.320 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.310+0000 7f3cabfff640 1 -- 172.21.15.49:0/868454502 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb40052e0 con 0x7f3cc414c4d0 2023-11-06T02:11:17.320 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.310+0000 7f3cabfff640 1 -- 172.21.15.49:0/868454502 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cb4005460 con 0x7f3cc414c4d0 2023-11-06T02:11:17.320 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.310+0000 7f3cc9ba2640 1 -- 172.21.15.49:0/868454502 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cc415b530 con 0x7f3cc414c4d0 2023-11-06T02:11:17.321 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.310+0000 7f3cabfff640 1 -- 172.21.15.49:0/868454502 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb4003840 con 0x7f3cc414c4d0 2023-11-06T02:11:17.321 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.310+0000 7f3cc9ba2640 1 -- 172.21.15.49:0/868454502 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cc414fe20 con 0x7f3cc414c4d0 2023-11-06T02:11:17.321 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.310+0000 7f3cc9ba2640 1 -- 172.21.15.49:0/868454502 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cc4074040 con 0x7f3cc414c4d0 2023-11-06T02:11:17.321 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.310+0000 7f3cabfff640 1 -- 172.21.15.49:0/868454502 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f3cb40039a0 con 0x7f3cc414c4d0 2023-11-06T02:11:17.321 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.310+0000 7f3cabfff640 1 --2- 172.21.15.49:0/868454502 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3cb0065a90 0x7f3cb0067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:17.321 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.310+0000 7f3cabfff640 1 -- 172.21.15.49:0/868454502 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f3cb40829f0 con 0x7f3cc414c4d0 2023-11-06T02:11:17.322 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.310+0000 7f3cc37fe640 1 --2- 172.21.15.49:0/868454502 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3cb0065a90 0x7f3cb0067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:17.322 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.314+0000 7f3cc37fe640 1 --2- 172.21.15.49:0/868454502 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3cb0065a90 0x7f3cb0067f50 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc001000 tx=0x7f3cbc001480 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:17.322 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.314+0000 7f3cabfff640 1 -- 172.21.15.49:0/868454502 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f3cb4050210 con 0x7f3cc414c4d0 2023-11-06T02:11:17.466 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.470+0000 7f3cc9ba2640 1 -- 172.21.15.49:0/868454502 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3cc40cdc20 con 0x7f3cc414c4d0 2023-11-06T02:11:17.466 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.470+0000 7f3cabfff640 1 -- 172.21.15.49:0/868454502 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v53) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3cb4019020 con 0x7f3cc414c4d0 2023-11-06T02:11:17.467 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2023-11-06T02:11:17.470 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.470+0000 7f3cc9ba2640 1 -- 172.21.15.49:0/868454502 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3cb0065a90 msgr2=0x7f3cb0067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:17.470 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.470+0000 7f3cc9ba2640 1 --2- 172.21.15.49:0/868454502 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3cb0065a90 0x7f3cb0067f50 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc001000 tx=0x7f3cbc001480 comp rx=0 tx=0).stop 2023-11-06T02:11:17.471 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.470+0000 7f3cc9ba2640 1 -- 172.21.15.49:0/868454502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3cc414c4d0 msgr2=0x7f3cc4150ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:17.471 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.470+0000 7f3cc9ba2640 1 --2- 172.21.15.49:0/868454502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3cc414c4d0 0x7f3cc4150ae0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f3cb4000c10 tx=0x7f3cb400e820 comp rx=0 tx=0).stop 2023-11-06T02:11:17.471 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.470+0000 7f3cc9ba2640 1 -- 172.21.15.49:0/868454502 shutdown_connections 2023-11-06T02:11:17.471 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.470+0000 7f3cc9ba2640 1 --2- 172.21.15.49:0/868454502 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3cb0065a90 0x7f3cb0067f50 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:17.471 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.470+0000 7f3cc9ba2640 1 --2- 172.21.15.49:0/868454502 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3cc414c4d0 0x7f3cc4150ae0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:17.472 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.470+0000 7f3cc9ba2640 1 --2- 172.21.15.49:0/868454502 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3cc415d740 0x7f3cc414f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:17.472 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.470+0000 7f3cc9ba2640 1 --2- 172.21.15.49:0/868454502 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3cac003f80 0x7f3cc414e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:17.472 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.470+0000 7f3cc9ba2640 1 -- 172.21.15.49:0/868454502 >> 172.21.15.49:0/868454502 conn(0x7f3cc406a370 msgr2=0x7f3cc406c650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:17.472 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.474+0000 7f3cc9ba2640 1 -- 172.21.15.49:0/868454502 shutdown_connections 2023-11-06T02:11:17.472 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.474+0000 7f3cc9ba2640 1 -- 172.21.15.49:0/868454502 wait complete. 2023-11-06T02:11:17.484 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T02:11:17.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.602+0000 7f2ae9462640 1 Processor -- start 2023-11-06T02:11:17.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.602+0000 7f2ae9462640 1 -- start start 2023-11-06T02:11:17.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.602+0000 7f2ae9462640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f2ae415cd70 0x7f2ae406b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:17.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.602+0000 7f2ae9462640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f2ae40b0830 0x7f2ae406b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:17.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.602+0000 7f2ae9462640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2ae40cebe0 0x7f2ae406bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:17.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.602+0000 7f2ae9462640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ae40aff70 con 0x7f2ae40b0830 2023-11-06T02:11:17.600 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.602+0000 7f2ae9462640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ae40b0dd0 con 0x7f2ae415cd70 2023-11-06T02:11:17.600 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.602+0000 7f2ae9462640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae40b10b0 con 0x7f2ae40cebe0 2023-11-06T02:11:17.600 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.602+0000 7f2adbfff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f2ae40b0830 0x7f2ae406b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:17.600 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.602+0000 7f2adbfff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f2ae40b0830 0x7f2ae406b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42744/0 (socket says 172.21.15.49:42744) 2023-11-06T02:11:17.600 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.602+0000 7f2adbfff640 1 -- 172.21.15.49:0/2317963627 learned_addr learned my addr 172.21.15.49:0/2317963627 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:17.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.602+0000 7f2ae2ffd640 1 --2- 172.21.15.49:0/2317963627 >> v2:172.21.15.150:3300/0 conn(0x7f2ae415cd70 0x7f2ae406b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:17.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.602+0000 7f2ae37fe640 1 --2- 172.21.15.49:0/2317963627 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2ae40cebe0 0x7f2ae406bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:17.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.602+0000 7f2adbfff640 1 -- 172.21.15.49:0/2317963627 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2ae40cebe0 msgr2=0x7f2ae406bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:17.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.602+0000 7f2adbfff640 1 --2- 172.21.15.49:0/2317963627 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2ae40cebe0 0x7f2ae406bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:17.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.602+0000 7f2adbfff640 1 -- 172.21.15.49:0/2317963627 >> v2:172.21.15.150:3300/0 conn(0x7f2ae415cd70 msgr2=0x7f2ae406b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:17.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.602+0000 7f2adbfff640 1 --2- 172.21.15.49:0/2317963627 >> v2:172.21.15.150:3300/0 conn(0x7f2ae415cd70 0x7f2ae406b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:17.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.602+0000 7f2adbfff640 1 -- 172.21.15.49:0/2317963627 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ae40b1390 con 0x7f2ae40b0830 2023-11-06T02:11:17.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.606+0000 7f2adbfff640 1 --2- 172.21.15.49:0/2317963627 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ae40b0830 0x7f2ae406b890 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f2ae408ad00 tx=0x7f2ad401c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=572bbb9b93a62d76 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:17.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.606+0000 7f2ae27fc640 1 -- 172.21.15.49:0/2317963627 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ad40ac620 con 0x7f2ae40b0830 2023-11-06T02:11:17.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.606+0000 7f2ae27fc640 1 -- 172.21.15.49:0/2317963627 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ad40ac780 con 0x7f2ae40b0830 2023-11-06T02:11:17.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.606+0000 7f2ae27fc640 1 -- 172.21.15.49:0/2317963627 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ad40acb10 con 0x7f2ae40b0830 2023-11-06T02:11:17.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.606+0000 7f2ae9462640 1 -- 172.21.15.49:0/2317963627 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ae40b0830 msgr2=0x7f2ae406b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:17.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.606+0000 7f2ae9462640 1 --2- 172.21.15.49:0/2317963627 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ae40b0830 0x7f2ae406b890 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f2ae408ad00 tx=0x7f2ad401c3e0 comp rx=0 tx=0).stop 2023-11-06T02:11:17.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.606+0000 7f2ae9462640 1 -- 172.21.15.49:0/2317963627 shutdown_connections 2023-11-06T02:11:17.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.606+0000 7f2ae9462640 1 --2- 172.21.15.49:0/2317963627 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2ae40cebe0 0x7f2ae406bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:17.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.606+0000 7f2ae9462640 1 --2- 172.21.15.49:0/2317963627 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ae40b0830 0x7f2ae406b890 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:17.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.606+0000 7f2ae9462640 1 --2- 172.21.15.49:0/2317963627 >> v2:172.21.15.150:3300/0 conn(0x7f2ae415cd70 0x7f2ae406b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:17.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.606+0000 7f2ae9462640 1 -- 172.21.15.49:0/2317963627 >> 172.21.15.49:0/2317963627 conn(0x7f2ae4089fd0 msgr2=0x7f2ae4099c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:17.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.606+0000 7f2ae9462640 1 -- 172.21.15.49:0/2317963627 shutdown_connections 2023-11-06T02:11:17.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.606+0000 7f2ae9462640 1 -- 172.21.15.49:0/2317963627 wait complete. 2023-11-06T02:11:17.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.606+0000 7f2ae9462640 1 Processor -- start 2023-11-06T02:11:17.612 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.606+0000 7f2ae9462640 1 -- start start 2023-11-06T02:11:17.612 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.606+0000 7f2ae9462640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2ae40b0830 0x7f2ae40d08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:17.612 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.606+0000 7f2ae9462640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ae40cebe0 0x7f2ae415f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:17.612 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.606+0000 7f2ae9462640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ae415cd70 0x7f2ae4157650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:17.612 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.606+0000 7f2ae9462640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae40d0020 con 0x7f2ae40cebe0 2023-11-06T02:11:17.613 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.606+0000 7f2ae9462640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae40d0de0 con 0x7f2ae415cd70 2023-11-06T02:11:17.613 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.606+0000 7f2ae9462640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae40cfd40 con 0x7f2ae40b0830 2023-11-06T02:11:17.613 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.606+0000 7f2adbfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ae40cebe0 0x7f2ae415f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:17.613 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.606+0000 7f2adbfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ae40cebe0 0x7f2ae415f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42754/0 (socket says 172.21.15.49:42754) 2023-11-06T02:11:17.614 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.606+0000 7f2adbfff640 1 -- 172.21.15.49:0/816860157 learned_addr learned my addr 172.21.15.49:0/816860157 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:17.614 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.606+0000 7f2ae2ffd640 1 --2- 172.21.15.49:0/816860157 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2ae40b0830 0x7f2ae40d08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:17.614 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.606+0000 7f2ae37fe640 1 --2- 172.21.15.49:0/816860157 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ae415cd70 0x7f2ae4157650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:17.614 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.606+0000 7f2adbfff640 1 -- 172.21.15.49:0/816860157 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2ae40b0830 msgr2=0x7f2ae40d08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:17.615 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.606+0000 7f2adbfff640 1 --2- 172.21.15.49:0/816860157 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2ae40b0830 0x7f2ae40d08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:17.615 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.606+0000 7f2adbfff640 1 -- 172.21.15.49:0/816860157 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ae415cd70 msgr2=0x7f2ae4157650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:17.615 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.606+0000 7f2adbfff640 1 --2- 172.21.15.49:0/816860157 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ae415cd70 0x7f2ae4157650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:17.615 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.606+0000 7f2adbfff640 1 -- 172.21.15.49:0/816860157 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ad40094d0 con 0x7f2ae40cebe0 2023-11-06T02:11:17.615 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.606+0000 7f2ae37fe640 1 --2- 172.21.15.49:0/816860157 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ae415cd70 0x7f2ae4157650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:11:17.616 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.606+0000 7f2adbfff640 1 --2- 172.21.15.49:0/816860157 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ae40cebe0 0x7f2ae415f3b0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f2ad401aa90 tx=0x7f2ad4019180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:17.616 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.606+0000 7f2adb7fe640 1 -- 172.21.15.49:0/816860157 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ad40b5640 con 0x7f2ae40cebe0 2023-11-06T02:11:17.616 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.606+0000 7f2adb7fe640 1 -- 172.21.15.49:0/816860157 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ad40b57a0 con 0x7f2ae40cebe0 2023-11-06T02:11:17.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.606+0000 7f2ae9462640 1 -- 172.21.15.49:0/816860157 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ae415e8d0 con 0x7f2ae40cebe0 2023-11-06T02:11:17.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.606+0000 7f2adb7fe640 1 -- 172.21.15.49:0/816860157 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ad40b5b30 con 0x7f2ae40cebe0 2023-11-06T02:11:17.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.606+0000 7f2ae9462640 1 -- 172.21.15.49:0/816860157 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ae415ee90 con 0x7f2ae40cebe0 2023-11-06T02:11:17.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.610+0000 7f2ae9462640 1 -- 172.21.15.49:0/816860157 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ae415e730 con 0x7f2ae40cebe0 2023-11-06T02:11:17.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.610+0000 7f2adb7fe640 1 -- 172.21.15.49:0/816860157 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f2ad40b3070 con 0x7f2ae40cebe0 2023-11-06T02:11:17.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.610+0000 7f2adb7fe640 1 --2- 172.21.15.49:0/816860157 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2ac4065a60 0x7f2ac4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:17.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.610+0000 7f2adb7fe640 1 -- 172.21.15.49:0/816860157 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f2ad411d480 con 0x7f2ae40cebe0 2023-11-06T02:11:17.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.610+0000 7f2ae2ffd640 1 --2- 172.21.15.49:0/816860157 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2ac4065a60 0x7f2ac4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:17.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.610+0000 7f2ae2ffd640 1 --2- 172.21.15.49:0/816860157 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2ac4065a60 0x7f2ac4067f20 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f2acc006fd0 tx=0x7f2acc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:17.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.614+0000 7f2adb7fe640 1 -- 172.21.15.49:0/816860157 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f2ae415e730 con 0x7f2ae40cebe0 2023-11-06T02:11:17.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.798+0000 7f2ae9462640 1 -- 172.21.15.49:0/816860157 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2ae415e730 con 0x7f2ae40cebe0 2023-11-06T02:11:17.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.798+0000 7f2adb7fe640 1 -- 172.21.15.49:0/816860157 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4648 (secure 0 0 0) 0x7f2ae415e730 con 0x7f2ae40cebe0 2023-11-06T02:11:17.796 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:11:17.796 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5096,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1396196190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1396196190},{"type":"v1","addr":"172.21.15.150:6835","nonce":1396196190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5102,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/2465209170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":2465209170},{"type":"v1","addr":"172.21.15.150:6837","nonce":2465209170}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5871,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/1006951804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1006951804},{"type":"v1","addr":"172.21.15.49:6833","nonce":1006951804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T02:10:31.903864+0000","modified":"2023-11-06T02:10:34.930237+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5877},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5877":{"gid":5877,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.49:6835/73939548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":73939548},{"type":"v1","addr":"172.21.15.49:6835","nonce":73939548}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-06T02:11:17.797 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 23 2023-11-06T02:11:17.798 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.802+0000 7f2ae9462640 1 -- 172.21.15.49:0/816860157 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2ac4065a60 msgr2=0x7f2ac4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:17.798 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.802+0000 7f2ae9462640 1 --2- 172.21.15.49:0/816860157 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2ac4065a60 0x7f2ac4067f20 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f2acc006fd0 tx=0x7f2acc008040 comp rx=0 tx=0).stop 2023-11-06T02:11:17.799 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.802+0000 7f2ae9462640 1 -- 172.21.15.49:0/816860157 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ae40cebe0 msgr2=0x7f2ae415f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:17.799 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.802+0000 7f2ae9462640 1 --2- 172.21.15.49:0/816860157 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ae40cebe0 0x7f2ae415f3b0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f2ad401aa90 tx=0x7f2ad4019180 comp rx=0 tx=0).stop 2023-11-06T02:11:17.799 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.802+0000 7f2ae9462640 1 -- 172.21.15.49:0/816860157 shutdown_connections 2023-11-06T02:11:17.799 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.802+0000 7f2ae9462640 1 --2- 172.21.15.49:0/816860157 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2ac4065a60 0x7f2ac4067f20 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:17.800 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.802+0000 7f2ae9462640 1 --2- 172.21.15.49:0/816860157 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ae40cebe0 0x7f2ae415f3b0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:17.800 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.802+0000 7f2ae9462640 1 --2- 172.21.15.49:0/816860157 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ae415cd70 0x7f2ae4157650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:17.800 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.802+0000 7f2ae9462640 1 --2- 172.21.15.49:0/816860157 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2ae40b0830 0x7f2ae40d08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:17.800 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.802+0000 7f2ae9462640 1 -- 172.21.15.49:0/816860157 >> 172.21.15.49:0/816860157 conn(0x7f2ae4089fd0 msgr2=0x7f2ae406baa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:17.800 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.802+0000 7f2ae9462640 1 -- 172.21.15.49:0/816860157 shutdown_connections 2023-11-06T02:11:17.800 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.802+0000 7f2ae9462640 1 -- 172.21.15.49:0/816860157 wait complete. 2023-11-06T02:11:17.814 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T02:11:17.924 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.926+0000 7fc160cd8640 1 Processor -- start 2023-11-06T02:11:17.925 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.926+0000 7fc160cd8640 1 -- start start 2023-11-06T02:11:17.926 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.930+0000 7fc160cd8640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc15c0b0790 0x7fc15c0c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:17.926 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.930+0000 7fc160cd8640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc15c0cb010 0x7fc15c0c2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:17.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.930+0000 7fc160cd8640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc15c0cab80 con 0x7fc15c15ab50 2023-11-06T02:11:17.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.930+0000 7fc160cd8640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc15c15af30 con 0x7fc15c0b0790 2023-11-06T02:11:17.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.930+0000 7fc160cd8640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc15c0cb5b0 con 0x7fc15c0cb010 2023-11-06T02:11:17.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.930+0000 7fc15a59c640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fc15c15ab50 0x7fc15c0b0080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:60052/0 (socket says 172.21.15.49:60052) 2023-11-06T02:11:17.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.930+0000 7fc15a59c640 1 -- 172.21.15.49:0/2074058542 learned_addr learned my addr 172.21.15.49:0/2074058542 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:11:17.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.930+0000 7fc159d9b640 1 --2- 172.21.15.49:0/2074058542 >> v2:172.21.15.150:3300/0 conn(0x7fc15c0b0790 0x7fc15c0c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:17.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.930+0000 7fc15ad9d640 1 --2- 172.21.15.49:0/2074058542 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc15c0cb010 0x7fc15c0c2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:17.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.930+0000 7fc15959a640 1 -- 172.21.15.49:0/2074058542 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1828492035 0 0) 0x7fc15c0cab80 con 0x7fc15c15ab50 2023-11-06T02:11:17.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.930+0000 7fc15959a640 1 -- 172.21.15.49:0/2074058542 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc144003530 con 0x7fc15c15ab50 2023-11-06T02:11:17.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.930+0000 7fc159d9b640 1 -- 172.21.15.49:0/2074058542 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc15c0cb010 msgr2=0x7fc15c0c2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:17.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.930+0000 7fc15959a640 1 -- 172.21.15.49:0/2074058542 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4281734833 0 0) 0x7fc144003530 con 0x7fc15c15ab50 2023-11-06T02:11:17.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.930+0000 7fc159d9b640 1 --2- 172.21.15.49:0/2074058542 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc15c0cb010 0x7fc15c0c2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:17.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.930+0000 7fc159d9b640 1 -- 172.21.15.49:0/2074058542 >> v1:172.21.15.49:6789/0 conn(0x7fc15c15ab50 legacy=0x7fc15c0b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:17.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.930+0000 7fc159d9b640 1 -- 172.21.15.49:0/2074058542 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc15c0cb890 con 0x7fc15c0b0790 2023-11-06T02:11:17.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.934+0000 7fc159d9b640 1 --2- 172.21.15.49:0/2074058542 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc15c0b0790 0x7fc15c0c9bb0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc1500097c0 tx=0x7fc15001c370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fbb151c47e1d049c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:17.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.934+0000 7fc15959a640 1 -- 172.21.15.49:0/2074058542 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc14c001820 con 0x7fc15c0b0790 2023-11-06T02:11:17.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.934+0000 7fc15959a640 1 -- 172.21.15.49:0/2074058542 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1500a7050 con 0x7fc15c0b0790 2023-11-06T02:11:17.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.934+0000 7fc15959a640 1 -- 172.21.15.49:0/2074058542 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1500ab600 con 0x7fc15c0b0790 2023-11-06T02:11:17.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.934+0000 7fc160cd8640 1 -- 172.21.15.49:0/2074058542 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc15c0b0790 msgr2=0x7fc15c0c9bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:17.932 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.934+0000 7fc160cd8640 1 --2- 172.21.15.49:0/2074058542 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc15c0b0790 0x7fc15c0c9bb0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc1500097c0 tx=0x7fc15001c370 comp rx=0 tx=0).stop 2023-11-06T02:11:17.932 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.934+0000 7fc160cd8640 1 -- 172.21.15.49:0/2074058542 shutdown_connections 2023-11-06T02:11:17.932 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.934+0000 7fc160cd8640 1 --2- 172.21.15.49:0/2074058542 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc15c0cb010 0x7fc15c0c2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:17.932 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.934+0000 7fc160cd8640 1 --2- 172.21.15.49:0/2074058542 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc15c0b0790 0x7fc15c0c9bb0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:17.932 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.934+0000 7fc160cd8640 1 -- 172.21.15.49:0/2074058542 >> 172.21.15.49:0/2074058542 conn(0x7fc15c06c0a0 msgr2=0x7fc15c06c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:17.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.934+0000 7fc160cd8640 1 -- 172.21.15.49:0/2074058542 shutdown_connections 2023-11-06T02:11:17.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.934+0000 7fc160cd8640 1 -- 172.21.15.49:0/2074058542 wait complete. 2023-11-06T02:11:17.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.934+0000 7fc160cd8640 1 Processor -- start 2023-11-06T02:11:17.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.934+0000 7fc160cd8640 1 -- start start 2023-11-06T02:11:17.936 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.934+0000 7fc160cd8640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc15c0b0790 0x7fc15c0c60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:17.937 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.934+0000 7fc160cd8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc15c0cb010 0x7fc15c1507a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:17.937 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.934+0000 7fc160cd8640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc15c15ab50 0x7fc15c14f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:17.937 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.934+0000 7fc160cd8640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc15c14e6e0 con 0x7fc15c0cb010 2023-11-06T02:11:17.937 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.934+0000 7fc160cd8640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc15c0c66c0 con 0x7fc15c0b0790 2023-11-06T02:11:17.938 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.934+0000 7fc160cd8640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc15c14e980 con 0x7fc15c15ab50 2023-11-06T02:11:17.938 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.934+0000 7fc159d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc15c0cb010 0x7fc15c1507a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:17.938 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.934+0000 7fc159d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc15c0cb010 0x7fc15c1507a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42770/0 (socket says 172.21.15.49:42770) 2023-11-06T02:11:17.938 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.934+0000 7fc159d9b640 1 -- 172.21.15.49:0/1315236867 learned_addr learned my addr 172.21.15.49:0/1315236867 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:17.938 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.934+0000 7fc15a59c640 1 --2- 172.21.15.49:0/1315236867 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc15c0b0790 0x7fc15c0c60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:17.939 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.934+0000 7fc15ad9d640 1 --2- 172.21.15.49:0/1315236867 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc15c15ab50 0x7fc15c14f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:17.939 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.934+0000 7fc159d9b640 1 -- 172.21.15.49:0/1315236867 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc15c15ab50 msgr2=0x7fc15c14f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:17.939 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.934+0000 7fc159d9b640 1 --2- 172.21.15.49:0/1315236867 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc15c15ab50 0x7fc15c14f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:17.939 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.934+0000 7fc159d9b640 1 -- 172.21.15.49:0/1315236867 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc15c0b0790 msgr2=0x7fc15c0c60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:17.939 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.934+0000 7fc159d9b640 1 --2- 172.21.15.49:0/1315236867 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc15c0b0790 0x7fc15c0c60e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:17.940 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.934+0000 7fc159d9b640 1 -- 172.21.15.49:0/1315236867 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc150009420 con 0x7fc15c0cb010 2023-11-06T02:11:17.940 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.934+0000 7fc159d9b640 1 --2- 172.21.15.49:0/1315236867 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc15c0cb010 0x7fc15c1507a0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fc15001cc10 tx=0x7fc150019110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:17.940 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.934+0000 7fc14affd640 1 -- 172.21.15.49:0/1315236867 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc150003ab0 con 0x7fc15c0cb010 2023-11-06T02:11:17.940 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.934+0000 7fc14affd640 1 -- 172.21.15.49:0/1315236867 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc150003c10 con 0x7fc15c0cb010 2023-11-06T02:11:17.941 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.934+0000 7fc160cd8640 1 -- 172.21.15.49:0/1315236867 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc15c14ec60 con 0x7fc15c0cb010 2023-11-06T02:11:17.941 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.934+0000 7fc14affd640 1 -- 172.21.15.49:0/1315236867 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1500b5410 con 0x7fc15c0cb010 2023-11-06T02:11:17.941 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.934+0000 7fc160cd8640 1 -- 172.21.15.49:0/1315236867 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc15c150140 con 0x7fc15c0cb010 2023-11-06T02:11:17.941 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.938+0000 7fc160cd8640 1 -- 172.21.15.49:0/1315236867 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc15c0b0de0 con 0x7fc15c0cb010 2023-11-06T02:11:17.941 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.938+0000 7fc14affd640 1 -- 172.21.15.49:0/1315236867 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fc1500b2050 con 0x7fc15c0cb010 2023-11-06T02:11:17.942 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.938+0000 7fc14affd640 1 --2- 172.21.15.49:0/1315236867 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc13c065a60 0x7fc13c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:17.942 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.938+0000 7fc14affd640 1 -- 172.21.15.49:0/1315236867 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fc15011d520 con 0x7fc15c0cb010 2023-11-06T02:11:17.942 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.938+0000 7fc15a59c640 1 --2- 172.21.15.49:0/1315236867 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc13c065a60 0x7fc13c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:17.942 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.938+0000 7fc15a59c640 1 --2- 172.21.15.49:0/1315236867 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc13c065a60 0x7fc13c067f20 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fc15c159be0 tx=0x7fc14c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:17.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:17.942+0000 7fc14affd640 1 -- 172.21.15.49:0/1315236867 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fc1500eadd0 con 0x7fc15c0cb010 2023-11-06T02:11:18.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.098+0000 7fc160cd8640 1 -- 172.21.15.49:0/1315236867 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc15c074040 con 0x7fc15c0cb010 2023-11-06T02:11:18.096 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.098+0000 7fc14affd640 1 -- 172.21.15.49:0/1315236867 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+17979 (secure 0 0 0) 0x7fc1500ee3f0 con 0x7fc15c0cb010 2023-11-06T02:11:18.096 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:11:18.097 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":53,"fsid":"1a8b035c-6ba4-499a-b876-aea47fd9c0d4","created":"2023-11-06T02:08:06.440857+0000","modified":"2023-11-06T02:11:15.186441+0000","last_up_change":"2023-11-06T02:08:17.982372+0000","last_in_change":"2023-11-06T02:08:08.906631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T02:08:19.252222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T02:08:24.776211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-06T02:10:30.458849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-06T02:10:30.899578+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ad55af8-0571-4e6a-8722-b5d429b7496d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6809","nonce":38116},{"type":"v1","addr":"172.21.15.49:6811","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6813","nonce":38116},{"type":"v1","addr":"172.21.15.49:6815","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6821","nonce":38116},{"type":"v1","addr":"172.21.15.49:6823","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6817","nonce":38116},{"type":"v1","addr":"172.21.15.49:6819","nonce":38116}]},"public_addr":"172.21.15.49:6811/38116","cluster_addr":"172.21.15.49:6815/38116","heartbeat_back_addr":"172.21.15.49:6823/38116","heartbeat_front_addr":"172.21.15.49:6819/38116","state":["exists","up"]},{"osd":1,"uuid":"c8b31f85-a757-43ed-9419-351acc46611b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":38113},{"type":"v1","addr":"172.21.15.49:6810","nonce":38113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":38113},{"type":"v1","addr":"172.21.15.49:6814","nonce":38113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":38113},{"type":"v1","addr":"172.21.15.49:6822","nonce":38113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":38113},{"type":"v1","addr":"172.21.15.49:6818","nonce":38113}]},"public_addr":"172.21.15.49:6810/38113","cluster_addr":"172.21.15.49:6814/38113","heartbeat_back_addr":"172.21.15.49:6822/38113","heartbeat_front_addr":"172.21.15.49:6818/38113","state":["exists","up"]},{"osd":2,"uuid":"555b3bf3-0a70-4b91-8efd-e4204b5fc26e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":38114},{"type":"v1","addr":"172.21.15.49:6801","nonce":38114}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":38114},{"type":"v1","addr":"172.21.15.49:6803","nonce":38114}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":38114},{"type":"v1","addr":"172.21.15.49:6807","nonce":38114}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":38114},{"type":"v1","addr":"172.21.15.49:6805","nonce":38114}]},"public_addr":"172.21.15.49:6801/38114","cluster_addr":"172.21.15.49:6803/38114","heartbeat_back_addr":"172.21.15.49:6807/38114","heartbeat_front_addr":"172.21.15.49:6805/38114","state":["exists","up"]},{"osd":3,"uuid":"8d4e5d77-03ef-4b9b-bfcb-5ae93a46795c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":38115},{"type":"v1","addr":"172.21.15.49:6825","nonce":38115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":38115},{"type":"v1","addr":"172.21.15.49:6827","nonce":38115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":38115},{"type":"v1","addr":"172.21.15.49:6831","nonce":38115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":38115},{"type":"v1","addr":"172.21.15.49:6829","nonce":38115}]},"public_addr":"172.21.15.49:6825/38115","cluster_addr":"172.21.15.49:6827/38115","heartbeat_back_addr":"172.21.15.49:6831/38115","heartbeat_front_addr":"172.21.15.49:6829/38115","state":["exists","up"]},{"osd":4,"uuid":"dba03ad4-68a3-4cf3-9218-ac611f02eacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":38417},{"type":"v1","addr":"172.21.15.150:6809","nonce":38417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":38417},{"type":"v1","addr":"172.21.15.150:6811","nonce":38417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":38417},{"type":"v1","addr":"172.21.15.150:6815","nonce":38417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":38417},{"type":"v1","addr":"172.21.15.150:6813","nonce":38417}]},"public_addr":"172.21.15.150:6809/38417","cluster_addr":"172.21.15.150:6811/38417","heartbeat_back_addr":"172.21.15.150:6815/38417","heartbeat_front_addr":"172.21.15.150:6813/38417","state":["exists","up"]},{"osd":5,"uuid":"0df9c303-c255-4640-8e91-01f1c3efe18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":38418},{"type":"v1","addr":"172.21.15.150:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":38418},{"type":"v1","addr":"172.21.15.150:6803","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":38418},{"type":"v1","addr":"172.21.15.150:6807","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":38418},{"type":"v1","addr":"172.21.15.150:6805","nonce":38418}]},"public_addr":"172.21.15.150:6801/38418","cluster_addr":"172.21.15.150:6803/38418","heartbeat_back_addr":"172.21.15.150:6807/38418","heartbeat_front_addr":"172.21.15.150:6805/38418","state":["exists","up"]},{"osd":6,"uuid":"e59fa189-4b70-4c06-9722-c02b90828bf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":38428},{"type":"v1","addr":"172.21.15.150:6825","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":38428},{"type":"v1","addr":"172.21.15.150:6827","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":38428},{"type":"v1","addr":"172.21.15.150:6831","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":38428},{"type":"v1","addr":"172.21.15.150:6829","nonce":38428}]},"public_addr":"172.21.15.150:6825/38428","cluster_addr":"172.21.15.150:6827/38428","heartbeat_back_addr":"172.21.15.150:6831/38428","heartbeat_front_addr":"172.21.15.150:6829/38428","state":["exists","up"]},{"osd":7,"uuid":"a64f57e1-ef98-46bd-b304-385759e95898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":38421},{"type":"v1","addr":"172.21.15.150:6817","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":38421},{"type":"v1","addr":"172.21.15.150:6819","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":38421},{"type":"v1","addr":"172.21.15.150:6823","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":38421},{"type":"v1","addr":"172.21.15.150:6821","nonce":38421}]},"public_addr":"172.21.15.150:6817/38421","cluster_addr":"172.21.15.150:6819/38421","heartbeat_back_addr":"172.21.15.150:6823/38421","heartbeat_front_addr":"172.21.15.150:6821/38421","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.438552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.413280+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.296466+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.424966+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.276953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.230984+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.410235+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.480084+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.37","mappings":[{"from":0,"to":6}]},{"pgid":"8.38","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T02:11:18.098 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.102+0000 7fc160cd8640 1 -- 172.21.15.49:0/1315236867 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc13c065a60 msgr2=0x7fc13c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:18.099 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.102+0000 7fc160cd8640 1 --2- 172.21.15.49:0/1315236867 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc13c065a60 0x7fc13c067f20 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fc15c159be0 tx=0x7fc14c009040 comp rx=0 tx=0).stop 2023-11-06T02:11:18.099 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.102+0000 7fc160cd8640 1 -- 172.21.15.49:0/1315236867 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc15c0cb010 msgr2=0x7fc15c1507a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:18.099 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.102+0000 7fc160cd8640 1 --2- 172.21.15.49:0/1315236867 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc15c0cb010 0x7fc15c1507a0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fc15001cc10 tx=0x7fc150019110 comp rx=0 tx=0).stop 2023-11-06T02:11:18.099 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.102+0000 7fc160cd8640 1 -- 172.21.15.49:0/1315236867 shutdown_connections 2023-11-06T02:11:18.099 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.102+0000 7fc160cd8640 1 --2- 172.21.15.49:0/1315236867 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc13c065a60 0x7fc13c067f20 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:18.099 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.102+0000 7fc160cd8640 1 --2- 172.21.15.49:0/1315236867 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc15c0cb010 0x7fc15c1507a0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:18.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.102+0000 7fc160cd8640 1 --2- 172.21.15.49:0/1315236867 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc15c15ab50 0x7fc15c14f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:18.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.102+0000 7fc160cd8640 1 --2- 172.21.15.49:0/1315236867 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc15c0b0790 0x7fc15c0c60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:18.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.102+0000 7fc160cd8640 1 -- 172.21.15.49:0/1315236867 >> 172.21.15.49:0/1315236867 conn(0x7fc15c06c0a0 msgr2=0x7fc15c159e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:18.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.102+0000 7fc160cd8640 1 -- 172.21.15.49:0/1315236867 shutdown_connections 2023-11-06T02:11:18.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.102+0000 7fc160cd8640 1 -- 172.21.15.49:0/1315236867 wait complete. 2023-11-06T02:11:18.115 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-06T02:11:18.116 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T02:11:18.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.238+0000 7fa66b4d5640 1 Processor -- start 2023-11-06T02:11:18.234 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.238+0000 7fa66b4d5640 1 -- start start 2023-11-06T02:11:18.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.238+0000 7fa66b4d5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa66406d180 0x7fa6640b2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:18.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.238+0000 7fa66b4d5640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fa66415ad00 0x7fa6640b28d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:18.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.238+0000 7fa66b4d5640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa66406de00 con 0x7fa66415a920 2023-11-06T02:11:18.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.238+0000 7fa66b4d5640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa66406b060 con 0x7fa66415ad00 2023-11-06T02:11:18.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.238+0000 7fa66b4d5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa66406b340 con 0x7fa66406d180 2023-11-06T02:11:18.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.238+0000 7fa669271640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fa66415a920 0x7fa66406ca70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:60062/0 (socket says 172.21.15.49:60062) 2023-11-06T02:11:18.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.238+0000 7fa669271640 1 -- 172.21.15.49:0/1757456389 learned_addr learned my addr 172.21.15.49:0/1757456389 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:11:18.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.238+0000 7fa668a70640 1 --2- 172.21.15.49:0/1757456389 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa66406d180 0x7fa6640b2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:18.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.238+0000 7fa669a72640 1 --2- 172.21.15.49:0/1757456389 >> v2:172.21.15.150:3300/0 conn(0x7fa66415ad00 0x7fa6640b28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:18.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.238+0000 7fa65bfff640 1 -- 172.21.15.49:0/1757456389 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3694413254 0 0) 0x7fa66406de00 con 0x7fa66415a920 2023-11-06T02:11:18.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.238+0000 7fa65bfff640 1 -- 172.21.15.49:0/1757456389 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa650003530 con 0x7fa66415a920 2023-11-06T02:11:18.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.238+0000 7fa65bfff640 1 -- 172.21.15.49:0/1757456389 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 40449607 0 0) 0x7fa650003530 con 0x7fa66415a920 2023-11-06T02:11:18.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.238+0000 7fa65bfff640 1 -- 172.21.15.49:0/1757456389 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa66406d180 msgr2=0x7fa6640b2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:18.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.238+0000 7fa65bfff640 1 --2- 172.21.15.49:0/1757456389 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa66406d180 0x7fa6640b2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:18.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.238+0000 7fa65bfff640 1 -- 172.21.15.49:0/1757456389 >> v2:172.21.15.150:3300/0 conn(0x7fa66415ad00 msgr2=0x7fa6640b28d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:18.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.238+0000 7fa65bfff640 1 --2- 172.21.15.49:0/1757456389 >> v2:172.21.15.150:3300/0 conn(0x7fa66415ad00 0x7fa6640b28d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:18.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.238+0000 7fa65bfff640 1 -- 172.21.15.49:0/1757456389 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa66406b620 con 0x7fa66415a920 2023-11-06T02:11:18.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.238+0000 7fa65bfff640 1 -- 172.21.15.49:0/1757456389 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7fa660001a00 con 0x7fa66415a920 2023-11-06T02:11:18.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.238+0000 7fa65bfff640 1 -- 172.21.15.49:0/1757456389 >> v1:172.21.15.49:6789/0 conn(0x7fa66415a920 legacy=0x7fa66406ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:18.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.238+0000 7fa65bfff640 1 --2- 172.21.15.49:0/1757456389 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa650003980 0x7fa650003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:18.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.238+0000 7fa65bfff640 1 --2- 172.21.15.49:0/1757456389 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa66406d180 0x7fa6500047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:18.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.238+0000 7fa65bfff640 1 --2- 172.21.15.49:0/1757456389 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa650004d10 0x7fa650005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:18.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.238+0000 7fa65bfff640 1 -- 172.21.15.49:0/1757456389 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa66406b060 con 0x7fa650003980 2023-11-06T02:11:18.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.238+0000 7fa65bfff640 1 -- 172.21.15.49:0/1757456389 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa66406b340 con 0x7fa650004d10 2023-11-06T02:11:18.242 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.238+0000 7fa65bfff640 1 -- 172.21.15.49:0/1757456389 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa65000ba10 con 0x7fa66406d180 2023-11-06T02:11:18.242 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.238+0000 7fa669271640 1 --2- 172.21.15.49:0/1757456389 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa650003980 0x7fa650003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:18.242 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.238+0000 7fa668a70640 1 --2- 172.21.15.49:0/1757456389 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa66406d180 0x7fa6500047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:18.242 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.238+0000 7fa669271640 1 -- 172.21.15.49:0/1757456389 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa66406d180 msgr2=0x7fa6500047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:18.242 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.238+0000 7fa669271640 1 --2- 172.21.15.49:0/1757456389 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa66406d180 0x7fa6500047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:18.243 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.238+0000 7fa669a72640 1 --2- 172.21.15.49:0/1757456389 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa650004d10 0x7fa650005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:18.243 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.238+0000 7fa669271640 1 -- 172.21.15.49:0/1757456389 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa650004d10 msgr2=0x7fa650005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:18.243 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.238+0000 7fa669271640 1 --2- 172.21.15.49:0/1757456389 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa650004d10 0x7fa650005190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:18.243 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.238+0000 7fa669271640 1 -- 172.21.15.49:0/1757456389 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa650003530 con 0x7fa650003980 2023-11-06T02:11:18.247 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.242+0000 7fa669271640 1 --2- 172.21.15.49:0/1757456389 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa650003980 0x7fa650003da0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fa66000a860 tx=0x7fa66001d330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bce5974f96ce7c14 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:18.248 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.242+0000 7fa65bfff640 1 -- 172.21.15.49:0/1757456389 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6600ad630 con 0x7fa650003980 2023-11-06T02:11:18.248 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.242+0000 7fa65bfff640 1 -- 172.21.15.49:0/1757456389 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6600a8070 con 0x7fa650003980 2023-11-06T02:11:18.248 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.242+0000 7fa65bfff640 1 -- 172.21.15.49:0/1757456389 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6600ad9f0 con 0x7fa650003980 2023-11-06T02:11:18.249 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.242+0000 7fa66b4d5640 1 -- 172.21.15.49:0/1757456389 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa650003980 msgr2=0x7fa650003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:18.249 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.242+0000 7fa66b4d5640 1 --2- 172.21.15.49:0/1757456389 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa650003980 0x7fa650003da0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fa66000a860 tx=0x7fa66001d330 comp rx=0 tx=0).stop 2023-11-06T02:11:18.249 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.242+0000 7fa669a72640 1 -- 172.21.15.49:0/1757456389 reap_dead start 2023-11-06T02:11:18.249 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.242+0000 7fa66b4d5640 1 -- 172.21.15.49:0/1757456389 shutdown_connections 2023-11-06T02:11:18.250 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.242+0000 7fa66b4d5640 1 -- 172.21.15.49:0/1757456389 >> 172.21.15.49:0/1757456389 conn(0x7fa6640cb8a0 msgr2=0x7fa6640cbcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:18.250 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.242+0000 7fa66b4d5640 1 -- 172.21.15.49:0/1757456389 shutdown_connections 2023-11-06T02:11:18.250 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.242+0000 7fa66b4d5640 1 -- 172.21.15.49:0/1757456389 wait complete. 2023-11-06T02:11:18.250 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.242+0000 7fa66b4d5640 1 Processor -- start 2023-11-06T02:11:18.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.242+0000 7fa66b4d5640 1 -- start start 2023-11-06T02:11:18.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.246+0000 7fa66b4d5640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa650003980 0x7fa6640adfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:18.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.246+0000 7fa66b4d5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6640ce4b0 0x7fa664154390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:18.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.246+0000 7fa66b4d5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6641f94d0 0x7fa6640af630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:18.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.246+0000 7fa66b4d5640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6640abfb0 con 0x7fa6640ce4b0 2023-11-06T02:11:18.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.246+0000 7fa66b4d5640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6640ab9f0 con 0x7fa650003980 2023-11-06T02:11:18.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.246+0000 7fa66b4d5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6640abcd0 con 0x7fa6641f94d0 2023-11-06T02:11:18.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.246+0000 7fa668a70640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6640ce4b0 0x7fa664154390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:18.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.246+0000 7fa668a70640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6640ce4b0 0x7fa664154390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42782/0 (socket says 172.21.15.49:42782) 2023-11-06T02:11:18.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.246+0000 7fa668a70640 1 -- 172.21.15.49:0/1165770109 learned_addr learned my addr 172.21.15.49:0/1165770109 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:18.253 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.246+0000 7fa669271640 1 --2- 172.21.15.49:0/1165770109 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa650003980 0x7fa6640adfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:18.253 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.246+0000 7fa669a72640 1 --2- 172.21.15.49:0/1165770109 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6641f94d0 0x7fa6640af630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:18.253 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.246+0000 7fa668a70640 1 -- 172.21.15.49:0/1165770109 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6641f94d0 msgr2=0x7fa6640af630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:18.253 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.246+0000 7fa668a70640 1 --2- 172.21.15.49:0/1165770109 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6641f94d0 0x7fa6640af630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:18.253 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.246+0000 7fa668a70640 1 -- 172.21.15.49:0/1165770109 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa650003980 msgr2=0x7fa6640adfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:18.254 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.246+0000 7fa668a70640 1 --2- 172.21.15.49:0/1165770109 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa650003980 0x7fa6640adfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:18.254 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.246+0000 7fa668a70640 1 -- 172.21.15.49:0/1165770109 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6641f8cd0 con 0x7fa6640ce4b0 2023-11-06T02:11:18.254 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.246+0000 7fa669a72640 1 --2- 172.21.15.49:0/1165770109 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6641f94d0 0x7fa6640af630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:11:18.254 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.246+0000 7fa668a70640 1 --2- 172.21.15.49:0/1165770109 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6640ce4b0 0x7fa664154390 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fa6540115e0 tx=0x7fa654011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:18.254 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.246+0000 7fa659ffb640 1 -- 172.21.15.49:0/1165770109 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa65400f8f0 con 0x7fa6640ce4b0 2023-11-06T02:11:18.255 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.246+0000 7fa659ffb640 1 -- 172.21.15.49:0/1165770109 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa65400fa50 con 0x7fa6640ce4b0 2023-11-06T02:11:18.255 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.246+0000 7fa66b4d5640 1 -- 172.21.15.49:0/1165770109 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6641f8fb0 con 0x7fa6640ce4b0 2023-11-06T02:11:18.255 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.246+0000 7fa659ffb640 1 -- 172.21.15.49:0/1165770109 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa65400fdc0 con 0x7fa6640ce4b0 2023-11-06T02:11:18.255 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.246+0000 7fa66b4d5640 1 -- 172.21.15.49:0/1165770109 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6640ce930 con 0x7fa6640ce4b0 2023-11-06T02:11:18.256 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.246+0000 7fa66b4d5640 1 -- 172.21.15.49:0/1165770109 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa664074040 con 0x7fa6640ce4b0 2023-11-06T02:11:18.256 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.246+0000 7fa659ffb640 1 -- 172.21.15.49:0/1165770109 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fa654015460 con 0x7fa6640ce4b0 2023-11-06T02:11:18.256 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.246+0000 7fa659ffb640 1 --2- 172.21.15.49:0/1165770109 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa648065a90 0x7fa648067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:18.256 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.246+0000 7fa659ffb640 1 -- 172.21.15.49:0/1165770109 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fa6540837e0 con 0x7fa6640ce4b0 2023-11-06T02:11:18.256 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.246+0000 7fa669271640 1 --2- 172.21.15.49:0/1165770109 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa648065a90 0x7fa648067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:18.257 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.250+0000 7fa669271640 1 --2- 172.21.15.49:0/1165770109 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa648065a90 0x7fa648067f50 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa6600ad000 tx=0x7fa660001e00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:18.257 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.250+0000 7fa659ffb640 1 -- 172.21.15.49:0/1165770109 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fa654050ff0 con 0x7fa6640ce4b0 2023-11-06T02:11:18.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.438+0000 7fa66b4d5640 1 -- 172.21.15.49:0/1165770109 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa6640af410 con 0x7fa6640ce4b0 2023-11-06T02:11:18.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.442+0000 7fa659ffb640 1 -- 172.21.15.49:0/1165770109 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4648 (secure 0 0 0) 0x7fa654054610 con 0x7fa6640ce4b0 2023-11-06T02:11:18.437 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:11:18.437 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5096,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1396196190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1396196190},{"type":"v1","addr":"172.21.15.150:6835","nonce":1396196190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5102,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/2465209170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":2465209170},{"type":"v1","addr":"172.21.15.150:6837","nonce":2465209170}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5871,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/1006951804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1006951804},{"type":"v1","addr":"172.21.15.49:6833","nonce":1006951804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T02:10:31.903864+0000","modified":"2023-11-06T02:10:34.930237+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5877},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5877":{"gid":5877,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.49:6835/73939548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":73939548},{"type":"v1","addr":"172.21.15.49:6835","nonce":73939548}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-06T02:11:18.438 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 23 2023-11-06T02:11:18.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.442+0000 7fa66b4d5640 1 -- 172.21.15.49:0/1165770109 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa648065a90 msgr2=0x7fa648067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:18.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.442+0000 7fa66b4d5640 1 --2- 172.21.15.49:0/1165770109 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa648065a90 0x7fa648067f50 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa6600ad000 tx=0x7fa660001e00 comp rx=0 tx=0).stop 2023-11-06T02:11:18.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.442+0000 7fa66b4d5640 1 -- 172.21.15.49:0/1165770109 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6640ce4b0 msgr2=0x7fa664154390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:18.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.442+0000 7fa66b4d5640 1 --2- 172.21.15.49:0/1165770109 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6640ce4b0 0x7fa664154390 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fa6540115e0 tx=0x7fa654011aa0 comp rx=0 tx=0).stop 2023-11-06T02:11:18.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.442+0000 7fa66b4d5640 1 -- 172.21.15.49:0/1165770109 shutdown_connections 2023-11-06T02:11:18.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.442+0000 7fa66b4d5640 1 --2- 172.21.15.49:0/1165770109 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa648065a90 0x7fa648067f50 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:18.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.442+0000 7fa66b4d5640 1 --2- 172.21.15.49:0/1165770109 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6640ce4b0 0x7fa664154390 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:18.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.442+0000 7fa66b4d5640 1 --2- 172.21.15.49:0/1165770109 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6641f94d0 0x7fa6640af630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:18.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.442+0000 7fa66b4d5640 1 --2- 172.21.15.49:0/1165770109 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa650003980 0x7fa6640adfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:18.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.442+0000 7fa66b4d5640 1 -- 172.21.15.49:0/1165770109 >> 172.21.15.49:0/1165770109 conn(0x7fa6640cb8a0 msgr2=0x7fa664156910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:18.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.442+0000 7fa66b4d5640 1 -- 172.21.15.49:0/1165770109 shutdown_connections 2023-11-06T02:11:18.443 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.442+0000 7fa66b4d5640 1 -- 172.21.15.49:0/1165770109 wait complete. 2023-11-06T02:11:18.457 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T02:11:18.586 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.590+0000 7f825bf2a640 1 Processor -- start 2023-11-06T02:11:18.586 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.590+0000 7f825bf2a640 1 -- start start 2023-11-06T02:11:18.587 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.590+0000 7f825bf2a640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f8254159980 0x7f8254071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:18.587 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.590+0000 7f825bf2a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f82540b1450 0x7f82540cdd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:18.588 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.590+0000 7f825bf2a640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8254071890 con 0x7f82540cb010 2023-11-06T02:11:18.588 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.590+0000 7f825bf2a640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8254072410 con 0x7f8254159980 2023-11-06T02:11:18.588 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.590+0000 7f825bf2a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82540b1890 con 0x7f82540b1450 2023-11-06T02:11:18.588 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.590+0000 7f8259cc6640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f8254159980 0x7f8254071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:18.588 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.590+0000 7f82594c5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f82540b1450 0x7f82540cdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:18.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.590+0000 7f825a4c7640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f82540cb010 0x7f82540cc540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.49:59946/0 (socket says 172.21.15.49:59946) 2023-11-06T02:11:18.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.590+0000 7f825a4c7640 1 -- 172.21.15.49:0/4138952600 learned_addr learned my addr 172.21.15.49:0/4138952600 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:11:18.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.590+0000 7f8259cc6640 1 -- 172.21.15.49:0/4138952600 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f82540b1450 msgr2=0x7f82540cdd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:18.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.590+0000 7f8259cc6640 1 --2- 172.21.15.49:0/4138952600 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f82540b1450 0x7f82540cdd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:18.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.590+0000 7f8259cc6640 1 -- 172.21.15.49:0/4138952600 >> v1:172.21.15.150:6789/0 conn(0x7f82540cb010 legacy=0x7f82540cc540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:18.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.590+0000 7f8259cc6640 1 -- 172.21.15.49:0/4138952600 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8254071b70 con 0x7f8254159980 2023-11-06T02:11:18.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.590+0000 7f82594c5640 1 --2- 172.21.15.49:0/4138952600 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f82540b1450 0x7f82540cdd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T02:11:18.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.594+0000 7f8259cc6640 1 --2- 172.21.15.49:0/4138952600 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8254159980 0x7f8254071290 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f82500097a0 tx=0x7f825001c460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5cb7e94169875723 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:18.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.594+0000 7f8258cc4640 1 -- 172.21.15.49:0/4138952600 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82500ac630 con 0x7f8254159980 2023-11-06T02:11:18.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.594+0000 7f8258cc4640 1 -- 172.21.15.49:0/4138952600 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82500ac790 con 0x7f8254159980 2023-11-06T02:11:18.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.594+0000 7f8258cc4640 1 -- 172.21.15.49:0/4138952600 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82500acb20 con 0x7f8254159980 2023-11-06T02:11:18.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.594+0000 7f825bf2a640 1 -- 172.21.15.49:0/4138952600 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8254159980 msgr2=0x7f8254071290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:18.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.594+0000 7f825bf2a640 1 --2- 172.21.15.49:0/4138952600 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8254159980 0x7f8254071290 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f82500097a0 tx=0x7f825001c460 comp rx=0 tx=0).stop 2023-11-06T02:11:18.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.594+0000 7f825bf2a640 1 -- 172.21.15.49:0/4138952600 shutdown_connections 2023-11-06T02:11:18.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.594+0000 7f825bf2a640 1 --2- 172.21.15.49:0/4138952600 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f82540b1450 0x7f82540cdd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:18.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.594+0000 7f825bf2a640 1 --2- 172.21.15.49:0/4138952600 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8254159980 0x7f8254071290 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:18.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.594+0000 7f825bf2a640 1 -- 172.21.15.49:0/4138952600 >> 172.21.15.49:0/4138952600 conn(0x7f825406c0a0 msgr2=0x7f825406c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:18.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.594+0000 7f825bf2a640 1 -- 172.21.15.49:0/4138952600 shutdown_connections 2023-11-06T02:11:18.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.594+0000 7f825bf2a640 1 -- 172.21.15.49:0/4138952600 wait complete. 2023-11-06T02:11:18.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.598+0000 7f825bf2a640 1 Processor -- start 2023-11-06T02:11:18.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.598+0000 7f825bf2a640 1 -- start start 2023-11-06T02:11:18.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.598+0000 7f825bf2a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82540b1450 0x7f825415c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:18.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.598+0000 7f825bf2a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82540cb010 0x7f82540c4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:18.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.598+0000 7f825bf2a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f825415a840 0x7f82540c0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:18.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.598+0000 7f825bf2a640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f825415bba0 con 0x7f82540cb010 2023-11-06T02:11:18.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.598+0000 7f825bf2a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82540c33f0 con 0x7f82540b1450 2023-11-06T02:11:18.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.598+0000 7f825bf2a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f825415acd0 con 0x7f825415a840 2023-11-06T02:11:18.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.598+0000 7f82594c5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82540cb010 0x7f82540c4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:18.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.598+0000 7f82594c5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82540cb010 0x7f82540c4940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49310/0 (socket says 172.21.15.49:49310) 2023-11-06T02:11:18.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.598+0000 7f82594c5640 1 -- 172.21.15.49:0/3401041222 learned_addr learned my addr 172.21.15.49:0/3401041222 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:18.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.598+0000 7f825a4c7640 1 --2- 172.21.15.49:0/3401041222 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f825415a840 0x7f82540c0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:18.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.598+0000 7f8259cc6640 1 --2- 172.21.15.49:0/3401041222 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82540b1450 0x7f825415c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:18.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.598+0000 7f82594c5640 1 -- 172.21.15.49:0/3401041222 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f825415a840 msgr2=0x7f82540c0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:18.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.598+0000 7f82594c5640 1 --2- 172.21.15.49:0/3401041222 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f825415a840 0x7f82540c0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:18.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.598+0000 7f82594c5640 1 -- 172.21.15.49:0/3401041222 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82540b1450 msgr2=0x7f825415c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:18.600 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.598+0000 7f82594c5640 1 --2- 172.21.15.49:0/3401041222 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82540b1450 0x7f825415c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:18.600 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.598+0000 7f82594c5640 1 -- 172.21.15.49:0/3401041222 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8244009420 con 0x7f82540cb010 2023-11-06T02:11:18.600 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.598+0000 7f82594c5640 1 --2- 172.21.15.49:0/3401041222 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82540cb010 0x7f82540c4940 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f824400ec10 tx=0x7f824400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:18.600 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.598+0000 7f824a7fc640 1 -- 172.21.15.49:0/3401041222 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f824400cc30 con 0x7f82540cb010 2023-11-06T02:11:18.600 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.598+0000 7f824a7fc640 1 -- 172.21.15.49:0/3401041222 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f824400cd90 con 0x7f82540cb010 2023-11-06T02:11:18.600 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.598+0000 7f825bf2a640 1 -- 172.21.15.49:0/3401041222 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82500093b0 con 0x7f82540cb010 2023-11-06T02:11:18.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.598+0000 7f824a7fc640 1 -- 172.21.15.49:0/3401041222 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8244013690 con 0x7f82540cb010 2023-11-06T02:11:18.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.598+0000 7f825bf2a640 1 -- 172.21.15.49:0/3401041222 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82540ce3f0 con 0x7f82540cb010 2023-11-06T02:11:18.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.598+0000 7f825bf2a640 1 -- 172.21.15.49:0/3401041222 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f825415bd50 con 0x7f82540cb010 2023-11-06T02:11:18.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.598+0000 7f824a7fc640 1 -- 172.21.15.49:0/3401041222 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f82440137f0 con 0x7f82540cb010 2023-11-06T02:11:18.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.598+0000 7f824a7fc640 1 --2- 172.21.15.49:0/3401041222 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8238065a90 0x7f8238067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:18.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.602+0000 7f824a7fc640 1 -- 172.21.15.49:0/3401041222 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f8244083610 con 0x7f82540cb010 2023-11-06T02:11:18.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.602+0000 7f8259cc6640 1 --2- 172.21.15.49:0/3401041222 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8238065a90 0x7f8238067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:18.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.602+0000 7f8259cc6640 1 --2- 172.21.15.49:0/3401041222 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8238065a90 0x7f8238067f50 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f82500ae000 tx=0x7f8250008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:18.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.606+0000 7f824a7fc640 1 -- 172.21.15.49:0/3401041222 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f825415bd50 con 0x7f82540cb010 2023-11-06T02:11:18.798 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.802+0000 7f825bf2a640 1 -- 172.21.15.49:0/3401041222 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f825415bd50 con 0x7f82540cb010 2023-11-06T02:11:18.799 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.802+0000 7f824a7fc640 1 -- 172.21.15.49:0/3401041222 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4648 (secure 0 0 0) 0x7f825415bd50 con 0x7f82540cb010 2023-11-06T02:11:18.799 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:11:18.799 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5096,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1396196190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1396196190},{"type":"v1","addr":"172.21.15.150:6835","nonce":1396196190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5102,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/2465209170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":2465209170},{"type":"v1","addr":"172.21.15.150:6837","nonce":2465209170}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5871,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/1006951804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1006951804},{"type":"v1","addr":"172.21.15.49:6833","nonce":1006951804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T02:10:31.903864+0000","modified":"2023-11-06T02:10:34.930237+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5877},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5877":{"gid":5877,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.49:6835/73939548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":73939548},{"type":"v1","addr":"172.21.15.49:6835","nonce":73939548}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-06T02:11:18.800 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 23 2023-11-06T02:11:18.801 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.806+0000 7f825bf2a640 1 -- 172.21.15.49:0/3401041222 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8238065a90 msgr2=0x7f8238067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:18.801 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.806+0000 7f825bf2a640 1 --2- 172.21.15.49:0/3401041222 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8238065a90 0x7f8238067f50 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f82500ae000 tx=0x7f8250008040 comp rx=0 tx=0).stop 2023-11-06T02:11:18.801 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.806+0000 7f825bf2a640 1 -- 172.21.15.49:0/3401041222 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82540cb010 msgr2=0x7f82540c4940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:18.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.806+0000 7f825bf2a640 1 --2- 172.21.15.49:0/3401041222 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82540cb010 0x7f82540c4940 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f824400ec10 tx=0x7f824400c520 comp rx=0 tx=0).stop 2023-11-06T02:11:18.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.806+0000 7f825bf2a640 1 -- 172.21.15.49:0/3401041222 shutdown_connections 2023-11-06T02:11:18.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.806+0000 7f825bf2a640 1 --2- 172.21.15.49:0/3401041222 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8238065a90 0x7f8238067f50 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:18.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.806+0000 7f825bf2a640 1 --2- 172.21.15.49:0/3401041222 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82540cb010 0x7f82540c4940 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:18.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.806+0000 7f825bf2a640 1 --2- 172.21.15.49:0/3401041222 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f825415a840 0x7f82540c0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:18.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.806+0000 7f825bf2a640 1 --2- 172.21.15.49:0/3401041222 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82540b1450 0x7f825415c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:18.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.806+0000 7f825bf2a640 1 -- 172.21.15.49:0/3401041222 >> 172.21.15.49:0/3401041222 conn(0x7f825406c0a0 msgr2=0x7f82540b19d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:18.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.806+0000 7f825bf2a640 1 -- 172.21.15.49:0/3401041222 shutdown_connections 2023-11-06T02:11:18.804 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.806+0000 7f825bf2a640 1 -- 172.21.15.49:0/3401041222 wait complete. 2023-11-06T02:11:18.819 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T02:11:18.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.950+0000 7fb27fee1640 1 Processor -- start 2023-11-06T02:11:18.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.950+0000 7fb27fee1640 1 -- start start 2023-11-06T02:11:18.949 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.950+0000 7fb27fee1640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb278071bb0 0x7fb278161650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:18.949 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.950+0000 7fb27fee1640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2780d17c0 con 0x7fb2780cda70 2023-11-06T02:11:18.949 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.950+0000 7fb27fee1640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2780ce6c0 con 0x7fb2780b1f90 2023-11-06T02:11:18.950 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.950+0000 7fb27fee1640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2780cdf20 con 0x7fb278071bb0 2023-11-06T02:11:18.950 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.950+0000 7fb27d47c640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fb2780cda70 0x7fb2780714a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:45934/0 (socket says 172.21.15.49:45934) 2023-11-06T02:11:18.950 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.950+0000 7fb27d47c640 1 -- 172.21.15.49:0/2687275559 learned_addr learned my addr 172.21.15.49:0/2687275559 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:11:18.951 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.950+0000 7fb27e47e640 1 --2- 172.21.15.49:0/2687275559 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb278071bb0 0x7fb278161650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:18.951 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.950+0000 7fb27cc7b640 1 -- 172.21.15.49:0/2687275559 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2783691678 0 0) 0x7fb2780d17c0 con 0x7fb2780cda70 2023-11-06T02:11:18.951 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.950+0000 7fb27cc7b640 1 -- 172.21.15.49:0/2687275559 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb264003530 con 0x7fb2780cda70 2023-11-06T02:11:18.952 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.950+0000 7fb27cc7b640 1 -- 172.21.15.49:0/2687275559 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1694011164 0 0) 0x7fb264003530 con 0x7fb2780cda70 2023-11-06T02:11:18.952 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.950+0000 7fb27cc7b640 1 -- 172.21.15.49:0/2687275559 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb278071bb0 msgr2=0x7fb278161650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:18.952 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.950+0000 7fb27cc7b640 1 --2- 172.21.15.49:0/2687275559 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb278071bb0 0x7fb278161650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:18.953 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.950+0000 7fb27cc7b640 1 -- 172.21.15.49:0/2687275559 >> v1:172.21.15.150:6789/0 conn(0x7fb2780b1f90 legacy=0x7fb2780d1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:18.953 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.950+0000 7fb27cc7b640 1 -- 172.21.15.49:0/2687275559 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2780d2360 con 0x7fb2780cda70 2023-11-06T02:11:18.953 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.950+0000 7fb27cc7b640 1 -- 172.21.15.49:0/2687275559 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7fb268001680 con 0x7fb2780cda70 2023-11-06T02:11:18.953 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.950+0000 7fb27cc7b640 1 -- 172.21.15.49:0/2687275559 >> v1:172.21.15.49:6789/0 conn(0x7fb2780cda70 legacy=0x7fb2780714a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:18.954 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.950+0000 7fb27cc7b640 1 --2- 172.21.15.49:0/2687275559 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb278071bb0 0x7fb2640039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:18.954 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.950+0000 7fb27cc7b640 1 --2- 172.21.15.49:0/2687275559 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb264003f30 0x7fb2640047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:18.954 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.950+0000 7fb27cc7b640 1 --2- 172.21.15.49:0/2687275559 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb264004d10 0x7fb264005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:18.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.950+0000 7fb27cc7b640 1 -- 172.21.15.49:0/2687275559 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2780cdf20 con 0x7fb264003f30 2023-11-06T02:11:18.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.950+0000 7fb27cc7b640 1 -- 172.21.15.49:0/2687275559 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb26400ba70 con 0x7fb264004d10 2023-11-06T02:11:18.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.950+0000 7fb27cc7b640 1 -- 172.21.15.49:0/2687275559 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb26400bc30 con 0x7fb278071bb0 2023-11-06T02:11:18.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.950+0000 7fb27d47c640 1 --2- 172.21.15.49:0/2687275559 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb264003f30 0x7fb2640047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:18.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.950+0000 7fb27e47e640 1 --2- 172.21.15.49:0/2687275559 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb264004d10 0x7fb264005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:18.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.950+0000 7fb27dc7d640 1 --2- 172.21.15.49:0/2687275559 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb278071bb0 0x7fb2640039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:18.957 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.950+0000 7fb27d47c640 1 -- 172.21.15.49:0/2687275559 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb278071bb0 msgr2=0x7fb2640039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:18.957 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.950+0000 7fb27d47c640 1 --2- 172.21.15.49:0/2687275559 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb278071bb0 0x7fb2640039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:18.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.950+0000 7fb27d47c640 1 -- 172.21.15.49:0/2687275559 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb264004d10 msgr2=0x7fb264005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:18.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.950+0000 7fb27d47c640 1 --2- 172.21.15.49:0/2687275559 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb264004d10 0x7fb264005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:18.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.950+0000 7fb27d47c640 1 -- 172.21.15.49:0/2687275559 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb264003530 con 0x7fb264003f30 2023-11-06T02:11:18.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.954+0000 7fb27d47c640 1 --2- 172.21.15.49:0/2687275559 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb264003f30 0x7fb2640047d0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fb26800c240 tx=0x7fb26801f3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ed6a2133efd429c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:18.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.954+0000 7fb27cc7b640 1 -- 172.21.15.49:0/2687275559 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb268005ae0 con 0x7fb264003f30 2023-11-06T02:11:18.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.954+0000 7fb27cc7b640 1 -- 172.21.15.49:0/2687275559 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2680ab070 con 0x7fb264003f30 2023-11-06T02:11:18.964 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.954+0000 7fb27cc7b640 1 -- 172.21.15.49:0/2687275559 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb268007070 con 0x7fb264003f30 2023-11-06T02:11:18.964 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.954+0000 7fb27fee1640 1 -- 172.21.15.49:0/2687275559 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb264003f30 msgr2=0x7fb2640047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:18.964 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.954+0000 7fb27fee1640 1 --2- 172.21.15.49:0/2687275559 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb264003f30 0x7fb2640047d0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fb26800c240 tx=0x7fb26801f3b0 comp rx=0 tx=0).stop 2023-11-06T02:11:18.965 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.954+0000 7fb27e47e640 1 -- 172.21.15.49:0/2687275559 reap_dead start 2023-11-06T02:11:18.965 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.954+0000 7fb27fee1640 1 -- 172.21.15.49:0/2687275559 shutdown_connections 2023-11-06T02:11:18.965 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.954+0000 7fb27fee1640 1 -- 172.21.15.49:0/2687275559 >> 172.21.15.49:0/2687275559 conn(0x7fb2780d2aa0 msgr2=0x7fb2780d2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:18.965 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.954+0000 7fb27fee1640 1 -- 172.21.15.49:0/2687275559 shutdown_connections 2023-11-06T02:11:18.966 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.954+0000 7fb27fee1640 1 -- 172.21.15.49:0/2687275559 wait complete. 2023-11-06T02:11:18.966 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.954+0000 7fb27fee1640 1 Processor -- start 2023-11-06T02:11:18.966 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.954+0000 7fb27fee1640 1 -- start start 2023-11-06T02:11:18.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.954+0000 7fb27fee1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb264003f30 0x7fb2780b3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:18.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.954+0000 7fb27fee1640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2781f3190 0x7fb2781f1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:18.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.954+0000 7fb27fee1640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2781f2260 0x7fb2780c9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:18.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.954+0000 7fb27fee1640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2780b43b0 con 0x7fb264003f30 2023-11-06T02:11:18.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.954+0000 7fb27fee1640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2781f3610 con 0x7fb2781f3190 2023-11-06T02:11:18.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.954+0000 7fb27fee1640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2780b3b10 con 0x7fb2781f2260 2023-11-06T02:11:18.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.954+0000 7fb27dc7d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb264003f30 0x7fb2780b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:18.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.954+0000 7fb27dc7d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb264003f30 0x7fb2780b3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49336/0 (socket says 172.21.15.49:49336) 2023-11-06T02:11:18.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.954+0000 7fb27dc7d640 1 -- 172.21.15.49:0/2782700228 learned_addr learned my addr 172.21.15.49:0/2782700228 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:18.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.954+0000 7fb27d47c640 1 --2- 172.21.15.49:0/2782700228 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2781f3190 0x7fb2781f1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:18.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.954+0000 7fb27e47e640 1 --2- 172.21.15.49:0/2782700228 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2781f2260 0x7fb2780c9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:18.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.958+0000 7fb27dc7d640 1 -- 172.21.15.49:0/2782700228 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2781f2260 msgr2=0x7fb2780c9ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:18.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.958+0000 7fb27dc7d640 1 --2- 172.21.15.49:0/2782700228 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2781f2260 0x7fb2780c9ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:18.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.958+0000 7fb27dc7d640 1 -- 172.21.15.49:0/2782700228 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2781f3190 msgr2=0x7fb2781f1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:18.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.958+0000 7fb27dc7d640 1 --2- 172.21.15.49:0/2782700228 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2781f3190 0x7fb2781f1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:18.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.958+0000 7fb27dc7d640 1 -- 172.21.15.49:0/2782700228 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2780b3df0 con 0x7fb264003f30 2023-11-06T02:11:18.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.958+0000 7fb27dc7d640 1 --2- 172.21.15.49:0/2782700228 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb264003f30 0x7fb2780b3410 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fb27400e550 tx=0x7fb27400ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:18.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.958+0000 7fb26e7fc640 1 -- 172.21.15.49:0/2782700228 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2740052e0 con 0x7fb264003f30 2023-11-06T02:11:18.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.958+0000 7fb26e7fc640 1 -- 172.21.15.49:0/2782700228 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb274005440 con 0x7fb264003f30 2023-11-06T02:11:18.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.958+0000 7fb27fee1640 1 -- 172.21.15.49:0/2782700228 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2780b40d0 con 0x7fb264003f30 2023-11-06T02:11:18.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.958+0000 7fb26e7fc640 1 -- 172.21.15.49:0/2782700228 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb274003820 con 0x7fb264003f30 2023-11-06T02:11:18.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.958+0000 7fb27fee1640 1 -- 172.21.15.49:0/2782700228 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2780ca020 con 0x7fb264003f30 2023-11-06T02:11:18.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.958+0000 7fb27fee1640 1 -- 172.21.15.49:0/2782700228 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2780cb7d0 con 0x7fb264003f30 2023-11-06T02:11:18.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.958+0000 7fb26e7fc640 1 -- 172.21.15.49:0/2782700228 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fb274003980 con 0x7fb264003f30 2023-11-06T02:11:18.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.958+0000 7fb26e7fc640 1 --2- 172.21.15.49:0/2782700228 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb25c065a90 0x7fb25c067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:18.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.958+0000 7fb26e7fc640 1 -- 172.21.15.49:0/2782700228 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fb274082fd0 con 0x7fb264003f30 2023-11-06T02:11:18.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.958+0000 7fb27d47c640 1 --2- 172.21.15.49:0/2782700228 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb25c065a90 0x7fb25c067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:18.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.958+0000 7fb27d47c640 1 --2- 172.21.15.49:0/2782700228 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb25c065a90 0x7fb25c067f50 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb2680af000 tx=0x7fb2680aa000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:18.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:18.962+0000 7fb26e7fc640 1 -- 172.21.15.49:0/2782700228 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fb2780cb7d0 con 0x7fb264003f30 2023-11-06T02:11:19.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.130+0000 7fb27fee1640 1 -- 172.21.15.49:0/2782700228 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb2780cb7d0 con 0x7fb264003f30 2023-11-06T02:11:19.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.134+0000 7fb26e7fc640 1 -- 172.21.15.49:0/2782700228 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+17979 (secure 0 0 0) 0x7fb2780cb7d0 con 0x7fb264003f30 2023-11-06T02:11:19.130 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:11:19.130 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":53,"fsid":"1a8b035c-6ba4-499a-b876-aea47fd9c0d4","created":"2023-11-06T02:08:06.440857+0000","modified":"2023-11-06T02:11:15.186441+0000","last_up_change":"2023-11-06T02:08:17.982372+0000","last_in_change":"2023-11-06T02:08:08.906631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T02:08:19.252222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T02:08:24.776211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-06T02:10:30.458849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-06T02:10:30.899578+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ad55af8-0571-4e6a-8722-b5d429b7496d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6809","nonce":38116},{"type":"v1","addr":"172.21.15.49:6811","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6813","nonce":38116},{"type":"v1","addr":"172.21.15.49:6815","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6821","nonce":38116},{"type":"v1","addr":"172.21.15.49:6823","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6817","nonce":38116},{"type":"v1","addr":"172.21.15.49:6819","nonce":38116}]},"public_addr":"172.21.15.49:6811/38116","cluster_addr":"172.21.15.49:6815/38116","heartbeat_back_addr":"172.21.15.49:6823/38116","heartbeat_front_addr":"172.21.15.49:6819/38116","state":["exists","up"]},{"osd":1,"uuid":"c8b31f85-a757-43ed-9419-351acc46611b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":38113},{"type":"v1","addr":"172.21.15.49:6810","nonce":38113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":38113},{"type":"v1","addr":"172.21.15.49:6814","nonce":38113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":38113},{"type":"v1","addr":"172.21.15.49:6822","nonce":38113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":38113},{"type":"v1","addr":"172.21.15.49:6818","nonce":38113}]},"public_addr":"172.21.15.49:6810/38113","cluster_addr":"172.21.15.49:6814/38113","heartbeat_back_addr":"172.21.15.49:6822/38113","heartbeat_front_addr":"172.21.15.49:6818/38113","state":["exists","up"]},{"osd":2,"uuid":"555b3bf3-0a70-4b91-8efd-e4204b5fc26e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":38114},{"type":"v1","addr":"172.21.15.49:6801","nonce":38114}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":38114},{"type":"v1","addr":"172.21.15.49:6803","nonce":38114}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":38114},{"type":"v1","addr":"172.21.15.49:6807","nonce":38114}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":38114},{"type":"v1","addr":"172.21.15.49:6805","nonce":38114}]},"public_addr":"172.21.15.49:6801/38114","cluster_addr":"172.21.15.49:6803/38114","heartbeat_back_addr":"172.21.15.49:6807/38114","heartbeat_front_addr":"172.21.15.49:6805/38114","state":["exists","up"]},{"osd":3,"uuid":"8d4e5d77-03ef-4b9b-bfcb-5ae93a46795c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":38115},{"type":"v1","addr":"172.21.15.49:6825","nonce":38115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":38115},{"type":"v1","addr":"172.21.15.49:6827","nonce":38115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":38115},{"type":"v1","addr":"172.21.15.49:6831","nonce":38115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":38115},{"type":"v1","addr":"172.21.15.49:6829","nonce":38115}]},"public_addr":"172.21.15.49:6825/38115","cluster_addr":"172.21.15.49:6827/38115","heartbeat_back_addr":"172.21.15.49:6831/38115","heartbeat_front_addr":"172.21.15.49:6829/38115","state":["exists","up"]},{"osd":4,"uuid":"dba03ad4-68a3-4cf3-9218-ac611f02eacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":38417},{"type":"v1","addr":"172.21.15.150:6809","nonce":38417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":38417},{"type":"v1","addr":"172.21.15.150:6811","nonce":38417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":38417},{"type":"v1","addr":"172.21.15.150:6815","nonce":38417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":38417},{"type":"v1","addr":"172.21.15.150:6813","nonce":38417}]},"public_addr":"172.21.15.150:6809/38417","cluster_addr":"172.21.15.150:6811/38417","heartbeat_back_addr":"172.21.15.150:6815/38417","heartbeat_front_addr":"172.21.15.150:6813/38417","state":["exists","up"]},{"osd":5,"uuid":"0df9c303-c255-4640-8e91-01f1c3efe18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":38418},{"type":"v1","addr":"172.21.15.150:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":38418},{"type":"v1","addr":"172.21.15.150:6803","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":38418},{"type":"v1","addr":"172.21.15.150:6807","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":38418},{"type":"v1","addr":"172.21.15.150:6805","nonce":38418}]},"public_addr":"172.21.15.150:6801/38418","cluster_addr":"172.21.15.150:6803/38418","heartbeat_back_addr":"172.21.15.150:6807/38418","heartbeat_front_addr":"172.21.15.150:6805/38418","state":["exists","up"]},{"osd":6,"uuid":"e59fa189-4b70-4c06-9722-c02b90828bf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":38428},{"type":"v1","addr":"172.21.15.150:6825","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":38428},{"type":"v1","addr":"172.21.15.150:6827","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":38428},{"type":"v1","addr":"172.21.15.150:6831","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":38428},{"type":"v1","addr":"172.21.15.150:6829","nonce":38428}]},"public_addr":"172.21.15.150:6825/38428","cluster_addr":"172.21.15.150:6827/38428","heartbeat_back_addr":"172.21.15.150:6831/38428","heartbeat_front_addr":"172.21.15.150:6829/38428","state":["exists","up"]},{"osd":7,"uuid":"a64f57e1-ef98-46bd-b304-385759e95898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":38421},{"type":"v1","addr":"172.21.15.150:6817","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":38421},{"type":"v1","addr":"172.21.15.150:6819","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":38421},{"type":"v1","addr":"172.21.15.150:6823","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":38421},{"type":"v1","addr":"172.21.15.150:6821","nonce":38421}]},"public_addr":"172.21.15.150:6817/38421","cluster_addr":"172.21.15.150:6819/38421","heartbeat_back_addr":"172.21.15.150:6823/38421","heartbeat_front_addr":"172.21.15.150:6821/38421","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.438552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.413280+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.296466+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.424966+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.276953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.230984+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.410235+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.480084+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.37","mappings":[{"from":0,"to":6}]},{"pgid":"8.38","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T02:11:19.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.134+0000 7fb27fee1640 1 -- 172.21.15.49:0/2782700228 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb25c065a90 msgr2=0x7fb25c067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:19.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.134+0000 7fb27fee1640 1 --2- 172.21.15.49:0/2782700228 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb25c065a90 0x7fb25c067f50 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb2680af000 tx=0x7fb2680aa000 comp rx=0 tx=0).stop 2023-11-06T02:11:19.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.134+0000 7fb27fee1640 1 -- 172.21.15.49:0/2782700228 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb264003f30 msgr2=0x7fb2780b3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:19.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.134+0000 7fb27fee1640 1 --2- 172.21.15.49:0/2782700228 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb264003f30 0x7fb2780b3410 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fb27400e550 tx=0x7fb27400ea10 comp rx=0 tx=0).stop 2023-11-06T02:11:19.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.134+0000 7fb27fee1640 1 -- 172.21.15.49:0/2782700228 shutdown_connections 2023-11-06T02:11:19.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.134+0000 7fb27fee1640 1 --2- 172.21.15.49:0/2782700228 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb25c065a90 0x7fb25c067f50 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:19.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.134+0000 7fb27fee1640 1 --2- 172.21.15.49:0/2782700228 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2781f2260 0x7fb2780c9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:19.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.134+0000 7fb27fee1640 1 --2- 172.21.15.49:0/2782700228 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2781f3190 0x7fb2781f1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:19.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.134+0000 7fb27fee1640 1 --2- 172.21.15.49:0/2782700228 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb264003f30 0x7fb2780b3410 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:19.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.134+0000 7fb27fee1640 1 -- 172.21.15.49:0/2782700228 >> 172.21.15.49:0/2782700228 conn(0x7fb2780d2aa0 msgr2=0x7fb2780b6020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:19.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.134+0000 7fb27fee1640 1 -- 172.21.15.49:0/2782700228 shutdown_connections 2023-11-06T02:11:19.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.134+0000 7fb27fee1640 1 -- 172.21.15.49:0/2782700228 wait complete. 2023-11-06T02:11:19.149 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-06T02:11:19.270 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.274+0000 7f0abf9a3640 1 Processor -- start 2023-11-06T02:11:19.270 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.274+0000 7f0abf9a3640 1 -- start start 2023-11-06T02:11:19.272 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.274+0000 7f0abf9a3640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f0ab80b0790 0x7f0ab80c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:19.272 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.274+0000 7f0abf9a3640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ab80cab80 con 0x7f0ab815ab50 2023-11-06T02:11:19.272 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.274+0000 7f0abf9a3640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ab815af30 con 0x7f0ab80cb010 2023-11-06T02:11:19.272 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.274+0000 7f0abf9a3640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ab80cb5b0 con 0x7f0ab80b0790 2023-11-06T02:11:19.273 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.274+0000 7f0abd73f640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f0ab815ab50 0x7f0ab80b0080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:45944/0 (socket says 172.21.15.49:45944) 2023-11-06T02:11:19.273 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.274+0000 7f0abd73f640 1 -- 172.21.15.49:0/2707505653 learned_addr learned my addr 172.21.15.49:0/2707505653 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:11:19.273 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.274+0000 7f0abcf3e640 1 --2- 172.21.15.49:0/2707505653 >> v2:172.21.15.49:3300/0 conn(0x7f0ab80b0790 0x7f0ab80c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:19.273 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.274+0000 7f0aaffff640 1 -- 172.21.15.49:0/2707505653 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1317789681 0 0) 0x7f0ab80cab80 con 0x7f0ab815ab50 2023-11-06T02:11:19.273 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.274+0000 7f0aaffff640 1 -- 172.21.15.49:0/2707505653 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0aa4003530 con 0x7f0ab815ab50 2023-11-06T02:11:19.274 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.274+0000 7f0abcf3e640 1 -- 172.21.15.49:0/2707505653 >> v1:172.21.15.150:6789/0 conn(0x7f0ab80cb010 legacy=0x7f0ab80c2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:19.274 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.274+0000 7f0abcf3e640 1 -- 172.21.15.49:0/2707505653 >> v1:172.21.15.49:6789/0 conn(0x7f0ab815ab50 legacy=0x7f0ab80b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:19.274 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.274+0000 7f0aaffff640 1 -- 172.21.15.49:0/2707505653 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3926390798 0 0) 0x7f0aa4003530 con 0x7f0ab815ab50 2023-11-06T02:11:19.274 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.274+0000 7f0abcf3e640 1 -- 172.21.15.49:0/2707505653 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab80cb890 con 0x7f0ab80b0790 2023-11-06T02:11:19.275 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.278+0000 7f0abcf3e640 1 --2- 172.21.15.49:0/2707505653 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ab80b0790 0x7f0ab80c9bb0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f0aa80097c0 tx=0x7f0aa801c370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a419a2e78dc842ef server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:19.276 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.278+0000 7f0aaffff640 1 -- 172.21.15.49:0/2707505653 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab0001b80 con 0x7f0ab80b0790 2023-11-06T02:11:19.276 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.278+0000 7f0aaffff640 1 -- 172.21.15.49:0/2707505653 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aa80ac440 con 0x7f0ab80b0790 2023-11-06T02:11:19.276 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.278+0000 7f0aaffff640 1 -- 172.21.15.49:0/2707505653 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa80ac7d0 con 0x7f0ab80b0790 2023-11-06T02:11:19.276 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.278+0000 7f0abf9a3640 1 -- 172.21.15.49:0/2707505653 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ab80b0790 msgr2=0x7f0ab80c9bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:19.276 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.278+0000 7f0abf9a3640 1 --2- 172.21.15.49:0/2707505653 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ab80b0790 0x7f0ab80c9bb0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f0aa80097c0 tx=0x7f0aa801c370 comp rx=0 tx=0).stop 2023-11-06T02:11:19.277 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.278+0000 7f0abf9a3640 1 -- 172.21.15.49:0/2707505653 shutdown_connections 2023-11-06T02:11:19.277 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.278+0000 7f0abf9a3640 1 --2- 172.21.15.49:0/2707505653 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ab80b0790 0x7f0ab80c9bb0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:19.277 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.278+0000 7f0abf9a3640 1 -- 172.21.15.49:0/2707505653 >> 172.21.15.49:0/2707505653 conn(0x7f0ab806c0a0 msgr2=0x7f0ab806c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:19.277 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.278+0000 7f0abf9a3640 1 -- 172.21.15.49:0/2707505653 shutdown_connections 2023-11-06T02:11:19.277 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.278+0000 7f0abf9a3640 1 -- 172.21.15.49:0/2707505653 wait complete. 2023-11-06T02:11:19.280 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.278+0000 7f0abf9a3640 1 Processor -- start 2023-11-06T02:11:19.280 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.278+0000 7f0abf9a3640 1 -- start start 2023-11-06T02:11:19.280 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.282+0000 7f0abf9a3640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0ab80b0790 0x7f0ab814bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:19.280 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.282+0000 7f0abf9a3640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ab80cb010 0x7f0ab8152550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:19.281 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.282+0000 7f0abf9a3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ab815ab50 0x7f0ab81f8840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:19.281 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.282+0000 7f0abf9a3640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab80ad3f0 con 0x7f0ab815ab50 2023-11-06T02:11:19.281 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.282+0000 7f0abf9a3640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab80acb50 con 0x7f0ab80cb010 2023-11-06T02:11:19.281 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.282+0000 7f0abf9a3640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab80ace30 con 0x7f0ab80b0790 2023-11-06T02:11:19.281 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.282+0000 7f0abdf40640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ab815ab50 0x7f0ab81f8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:19.282 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.282+0000 7f0abdf40640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ab815ab50 0x7f0ab81f8840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49350/0 (socket says 172.21.15.49:49350) 2023-11-06T02:11:19.282 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.282+0000 7f0abdf40640 1 -- 172.21.15.49:0/2031216742 learned_addr learned my addr 172.21.15.49:0/2031216742 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:19.282 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.282+0000 7f0abcf3e640 1 --2- 172.21.15.49:0/2031216742 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ab80cb010 0x7f0ab8152550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:19.282 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.282+0000 7f0abd73f640 1 --2- 172.21.15.49:0/2031216742 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0ab80b0790 0x7f0ab814bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:19.283 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.282+0000 7f0abdf40640 1 -- 172.21.15.49:0/2031216742 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0ab80b0790 msgr2=0x7f0ab814bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:19.283 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.282+0000 7f0abdf40640 1 --2- 172.21.15.49:0/2031216742 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0ab80b0790 0x7f0ab814bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:19.283 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.282+0000 7f0abdf40640 1 -- 172.21.15.49:0/2031216742 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ab80cb010 msgr2=0x7f0ab8152550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:19.283 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.282+0000 7f0abdf40640 1 --2- 172.21.15.49:0/2031216742 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ab80cb010 0x7f0ab8152550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:19.283 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.282+0000 7f0abdf40640 1 -- 172.21.15.49:0/2031216742 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aa8009420 con 0x7f0ab815ab50 2023-11-06T02:11:19.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.282+0000 7f0abd73f640 1 --2- 172.21.15.49:0/2031216742 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0ab80b0790 0x7f0ab814bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:11:19.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.282+0000 7f0abcf3e640 1 --2- 172.21.15.49:0/2031216742 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ab80cb010 0x7f0ab8152550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:11:19.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.282+0000 7f0abdf40640 1 --2- 172.21.15.49:0/2031216742 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ab815ab50 0x7f0ab81f8840 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f0ab400c430 tx=0x7f0ab400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:19.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.282+0000 7f0aadffb640 1 -- 172.21.15.49:0/2031216742 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab4005220 con 0x7f0ab815ab50 2023-11-06T02:11:19.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.282+0000 7f0aadffb640 1 -- 172.21.15.49:0/2031216742 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ab4005380 con 0x7f0ab815ab50 2023-11-06T02:11:19.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.282+0000 7f0abf9a3640 1 -- 172.21.15.49:0/2031216742 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ab80ad110 con 0x7f0ab815ab50 2023-11-06T02:11:19.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.282+0000 7f0aadffb640 1 -- 172.21.15.49:0/2031216742 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab4007270 con 0x7f0ab815ab50 2023-11-06T02:11:19.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.282+0000 7f0abf9a3640 1 -- 172.21.15.49:0/2031216742 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ab814da40 con 0x7f0ab815ab50 2023-11-06T02:11:19.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.282+0000 7f0abf9a3640 1 -- 172.21.15.49:0/2031216742 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ab80b0de0 con 0x7f0ab815ab50 2023-11-06T02:11:19.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.282+0000 7f0aadffb640 1 -- 172.21.15.49:0/2031216742 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f0ab40073d0 con 0x7f0ab815ab50 2023-11-06T02:11:19.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.282+0000 7f0aadffb640 1 --2- 172.21.15.49:0/2031216742 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0a9c065a60 0x7f0a9c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:19.286 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.282+0000 7f0aadffb640 1 -- 172.21.15.49:0/2031216742 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f0ab4082940 con 0x7f0ab815ab50 2023-11-06T02:11:19.286 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.282+0000 7f0abd73f640 1 --2- 172.21.15.49:0/2031216742 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0a9c065a60 0x7f0a9c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:19.286 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.286+0000 7f0abd73f640 1 --2- 172.21.15.49:0/2031216742 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0a9c065a60 0x7f0a9c067f20 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f0ab00016f0 tx=0x7f0ab0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:19.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.290+0000 7f0aadffb640 1 -- 172.21.15.49:0/2031216742 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f0ab4050160 con 0x7f0ab815ab50 2023-11-06T02:11:19.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.478+0000 7f0abf9a3640 1 -- 172.21.15.49:0/2031216742 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f0ab814d5c0 con 0x7f0ab815ab50 2023-11-06T02:11:19.477 INFO:tasks.ceph.mon.a.smithi049.stderr:2023-11-06T02:11:19.482+0000 7f0164e5a640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-06T02:11:19.490 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.494+0000 7f0aadffb640 1 -- 172.21.15.49:0/2031216742 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v24) v1 ==== 161+0+0 (secure 0 0 0) 0x7f0ab814d5c0 con 0x7f0ab815ab50 2023-11-06T02:11:19.490 INFO:teuthology.orchestra.run.smithi049.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-06T02:11:19.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.494+0000 7f0abf9a3640 1 -- 172.21.15.49:0/2031216742 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0a9c065a60 msgr2=0x7f0a9c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:19.493 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.494+0000 7f0abf9a3640 1 --2- 172.21.15.49:0/2031216742 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0a9c065a60 0x7f0a9c067f20 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f0ab00016f0 tx=0x7f0ab0009040 comp rx=0 tx=0).stop 2023-11-06T02:11:19.493 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.494+0000 7f0abf9a3640 1 -- 172.21.15.49:0/2031216742 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ab815ab50 msgr2=0x7f0ab81f8840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:19.493 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.494+0000 7f0abf9a3640 1 --2- 172.21.15.49:0/2031216742 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ab815ab50 0x7f0ab81f8840 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f0ab400c430 tx=0x7f0ab400c8f0 comp rx=0 tx=0).stop 2023-11-06T02:11:19.493 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.494+0000 7f0abf9a3640 1 -- 172.21.15.49:0/2031216742 shutdown_connections 2023-11-06T02:11:19.493 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.494+0000 7f0abf9a3640 1 --2- 172.21.15.49:0/2031216742 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0a9c065a60 0x7f0a9c067f20 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:19.494 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.494+0000 7f0abf9a3640 1 --2- 172.21.15.49:0/2031216742 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ab815ab50 0x7f0ab81f8840 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:19.494 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.498+0000 7f0abf9a3640 1 --2- 172.21.15.49:0/2031216742 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ab80cb010 0x7f0ab8152550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:19.494 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.498+0000 7f0abf9a3640 1 --2- 172.21.15.49:0/2031216742 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0ab80b0790 0x7f0ab814bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:19.494 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.498+0000 7f0abf9a3640 1 -- 172.21.15.49:0/2031216742 >> 172.21.15.49:0/2031216742 conn(0x7f0ab806c0a0 msgr2=0x7f0ab80ae900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:19.494 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.498+0000 7f0abf9a3640 1 -- 172.21.15.49:0/2031216742 shutdown_connections 2023-11-06T02:11:19.495 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.498+0000 7f0abf9a3640 1 -- 172.21.15.49:0/2031216742 wait complete. 2023-11-06T02:11:19.500 INFO:tasks.ceph.mds.a.smithi049.stderr: -4682> 2023-11-06T02:10:33.347+0000 7feb1deab640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-06T02:11:19.505 INFO:tasks.ceph.mds.a.smithi049.stderr: -2989> 2023-11-06T02:10:41.950+0000 7feb1a6a4640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /parent_a/alpha auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={6036=0-4194304@1} caps={6036=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=6036 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x55b9e9309700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000001:[<0x10000000000/alpha v2>,<0x1/parent_a v4>]//[]","error_str":"failed to decode on-disk backtrace (40 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-06T02:11:19.508 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-06T02:11:19.566 INFO:tasks.ceph.mds.a.smithi049.stdout:starting mds.a at 2023-11-06T02:11:19.623 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.626+0000 7fb4539a1640 1 Processor -- start 2023-11-06T02:11:19.623 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.626+0000 7fb4539a1640 1 -- start start 2023-11-06T02:11:19.624 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.626+0000 7fb4539a1640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fb44c15cbb0 0x7fb44c0d1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:19.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.626+0000 7fb4539a1640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb44c0d24e0 0x7fb44c0d2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:19.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.626+0000 7fb4539a1640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb44c15cf90 con 0x7fb44c0d2fa0 2023-11-06T02:11:19.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.626+0000 7fb4539a1640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb44c0d3430 con 0x7fb44c0d24e0 2023-11-06T02:11:19.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.626+0000 7fb4539a1640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb44c0d0b80 con 0x7fb44c15cbb0 2023-11-06T02:11:19.626 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.626+0000 7fb450f3c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb44c0d24e0 0x7fb44c0d2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:19.626 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.626+0000 7fb450f3c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb44c0d24e0 0x7fb44c0d2a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49374/0 (socket says 172.21.15.49:49374) 2023-11-06T02:11:19.626 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.626+0000 7fb450f3c640 1 -- 172.21.15.49:0/3687991054 learned_addr learned my addr 172.21.15.49:0/3687991054 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:19.626 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.626+0000 7fb450f3c640 1 -- 172.21.15.49:0/3687991054 >> v2:172.21.15.150:3300/0 conn(0x7fb44c15cbb0 msgr2=0x7fb44c0d1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:19.626 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.626+0000 7fb450f3c640 1 --2- 172.21.15.49:0/3687991054 >> v2:172.21.15.150:3300/0 conn(0x7fb44c15cbb0 0x7fb44c0d1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:19.627 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.626+0000 7fb450f3c640 1 -- 172.21.15.49:0/3687991054 >> v1:172.21.15.150:6789/0 conn(0x7fb44c0d2fa0 legacy=0x7fb44c0b4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:19.627 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.626+0000 7fb450f3c640 1 -- 172.21.15.49:0/3687991054 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb44c0d0e60 con 0x7fb44c0d24e0 2023-11-06T02:11:19.628 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.630+0000 7fb450f3c640 1 --2- 172.21.15.49:0/3687991054 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb44c0d24e0 0x7fb44c0d2a60 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fb44c0cf4b0 tx=0x7fb43c01ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59dd9f7562aa9b3e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:19.628 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.630+0000 7fb443fff640 1 -- 172.21.15.49:0/3687991054 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb43c0a89b0 con 0x7fb44c0d24e0 2023-11-06T02:11:19.629 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.630+0000 7fb443fff640 1 -- 172.21.15.49:0/3687991054 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb43c0a8b10 con 0x7fb44c0d24e0 2023-11-06T02:11:19.629 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.630+0000 7fb443fff640 1 -- 172.21.15.49:0/3687991054 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb43c0a8ea0 con 0x7fb44c0d24e0 2023-11-06T02:11:19.629 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.630+0000 7fb4539a1640 1 -- 172.21.15.49:0/3687991054 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb44c0d24e0 msgr2=0x7fb44c0d2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:19.629 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.630+0000 7fb4539a1640 1 --2- 172.21.15.49:0/3687991054 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb44c0d24e0 0x7fb44c0d2a60 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fb44c0cf4b0 tx=0x7fb43c01ecb0 comp rx=0 tx=0).stop 2023-11-06T02:11:19.629 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.630+0000 7fb4539a1640 1 -- 172.21.15.49:0/3687991054 shutdown_connections 2023-11-06T02:11:19.630 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.630+0000 7fb4539a1640 1 --2- 172.21.15.49:0/3687991054 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb44c0d24e0 0x7fb44c0d2a60 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:19.630 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.630+0000 7fb4539a1640 1 --2- 172.21.15.49:0/3687991054 >> v2:172.21.15.150:3300/0 conn(0x7fb44c15cbb0 0x7fb44c0d1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:19.630 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.630+0000 7fb4539a1640 1 -- 172.21.15.49:0/3687991054 >> 172.21.15.49:0/3687991054 conn(0x7fb44c0718a0 msgr2=0x7fb44c071cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:19.631 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.630+0000 7fb4539a1640 1 -- 172.21.15.49:0/3687991054 shutdown_connections 2023-11-06T02:11:19.631 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.630+0000 7fb4539a1640 1 -- 172.21.15.49:0/3687991054 wait complete. 2023-11-06T02:11:19.631 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.630+0000 7fb4539a1640 1 Processor -- start 2023-11-06T02:11:19.631 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.630+0000 7fb4539a1640 1 -- start start 2023-11-06T02:11:19.634 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.634+0000 7fb4539a1640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb44c0d24e0 0x7fb44c0cb790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:19.634 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.634+0000 7fb4539a1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb44c0d2fa0 0x7fb44c157570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:19.634 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.634+0000 7fb4539a1640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb44c15cbb0 0x7fb44c15e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:19.634 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.634+0000 7fb4539a1640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb44c147020 con 0x7fb44c0d2fa0 2023-11-06T02:11:19.634 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.634+0000 7fb4539a1640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb44c147300 con 0x7fb44c15cbb0 2023-11-06T02:11:19.635 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.634+0000 7fb4539a1640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb44c1475e0 con 0x7fb44c0d24e0 2023-11-06T02:11:19.635 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.634+0000 7fb450f3c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb44c0d2fa0 0x7fb44c157570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:19.635 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.634+0000 7fb450f3c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb44c0d2fa0 0x7fb44c157570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49376/0 (socket says 172.21.15.49:49376) 2023-11-06T02:11:19.635 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.634+0000 7fb450f3c640 1 -- 172.21.15.49:0/2811711608 learned_addr learned my addr 172.21.15.49:0/2811711608 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:19.635 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.634+0000 7fb450f3c640 1 -- 172.21.15.49:0/2811711608 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb44c0d24e0 msgr2=0x7fb44c0cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:19.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.634+0000 7fb450f3c640 1 --2- 172.21.15.49:0/2811711608 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb44c0d24e0 0x7fb44c0cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:19.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.634+0000 7fb451f3e640 1 --2- 172.21.15.49:0/2811711608 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb44c15cbb0 0x7fb44c15e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:19.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.634+0000 7fb450f3c640 1 -- 172.21.15.49:0/2811711608 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb44c15cbb0 msgr2=0x7fb44c15e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:19.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.634+0000 7fb450f3c640 1 --2- 172.21.15.49:0/2811711608 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb44c15cbb0 0x7fb44c15e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:19.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.634+0000 7fb450f3c640 1 -- 172.21.15.49:0/2811711608 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb43c0094d0 con 0x7fb44c0d2fa0 2023-11-06T02:11:19.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.634+0000 7fb450f3c640 1 --2- 172.21.15.49:0/2811711608 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb44c0d2fa0 0x7fb44c157570 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fb43c0b3040 tx=0x7fb43c01ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:19.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.634+0000 7fb441ffb640 1 -- 172.21.15.49:0/2811711608 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb43c0b0e00 con 0x7fb44c0d2fa0 2023-11-06T02:11:19.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.634+0000 7fb441ffb640 1 -- 172.21.15.49:0/2811711608 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb43c0bb470 con 0x7fb44c0d2fa0 2023-11-06T02:11:19.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.634+0000 7fb4539a1640 1 -- 172.21.15.49:0/2811711608 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb44c1478c0 con 0x7fb44c0d2fa0 2023-11-06T02:11:19.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.634+0000 7fb441ffb640 1 -- 172.21.15.49:0/2811711608 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb43c0bb820 con 0x7fb44c0d2fa0 2023-11-06T02:11:19.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.634+0000 7fb4539a1640 1 -- 172.21.15.49:0/2811711608 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb44c146c60 con 0x7fb44c0d2fa0 2023-11-06T02:11:19.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.634+0000 7fb4539a1640 1 -- 172.21.15.49:0/2811711608 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb44c074040 con 0x7fb44c0d2fa0 2023-11-06T02:11:19.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.634+0000 7fb441ffb640 1 -- 172.21.15.49:0/2811711608 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fb43c0b7070 con 0x7fb44c0d2fa0 2023-11-06T02:11:19.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.634+0000 7fb441ffb640 1 --2- 172.21.15.49:0/2811711608 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb430065a60 0x7fb430067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:19.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.634+0000 7fb441ffb640 1 -- 172.21.15.49:0/2811711608 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6206+0+0 (secure 0 0 0) 0x7fb43c1221b0 con 0x7fb44c0d2fa0 2023-11-06T02:11:19.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.634+0000 7fb45173d640 1 --2- 172.21.15.49:0/2811711608 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb430065a60 0x7fb430067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:19.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.638+0000 7fb45173d640 1 --2- 172.21.15.49:0/2811711608 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb430065a60 0x7fb430067f20 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb448009690 tx=0x7fb448006c00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:19.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.638+0000 7fb441ffb640 1 -- 172.21.15.49:0/2811711608 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fb43c0efa60 con 0x7fb44c0d2fa0 2023-11-06T02:11:19.822 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:19.826+0000 7fb4539a1640 1 -- 172.21.15.49:0/2811711608 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fb44c147160 con 0x7fb44c0d2fa0 2023-11-06T02:11:20.496 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:20.498+0000 7fb441ffb640 1 -- 172.21.15.49:0/2811711608 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v25) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb44c147160 con 0x7fb44c0d2fa0 2023-11-06T02:11:20.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:20.502+0000 7fb4539a1640 1 -- 172.21.15.49:0/2811711608 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb430065a60 msgr2=0x7fb430067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:20.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:20.502+0000 7fb4539a1640 1 --2- 172.21.15.49:0/2811711608 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb430065a60 0x7fb430067f20 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb448009690 tx=0x7fb448006c00 comp rx=0 tx=0).stop 2023-11-06T02:11:20.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:20.502+0000 7fb4539a1640 1 -- 172.21.15.49:0/2811711608 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb44c0d2fa0 msgr2=0x7fb44c157570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:20.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:20.502+0000 7fb4539a1640 1 --2- 172.21.15.49:0/2811711608 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb44c0d2fa0 0x7fb44c157570 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fb43c0b3040 tx=0x7fb43c01ba50 comp rx=0 tx=0).stop 2023-11-06T02:11:20.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:20.502+0000 7fb4539a1640 1 -- 172.21.15.49:0/2811711608 shutdown_connections 2023-11-06T02:11:20.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:20.502+0000 7fb4539a1640 1 --2- 172.21.15.49:0/2811711608 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb430065a60 0x7fb430067f20 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:20.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:20.502+0000 7fb4539a1640 1 --2- 172.21.15.49:0/2811711608 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb44c0d2fa0 0x7fb44c157570 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:20.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:20.502+0000 7fb4539a1640 1 --2- 172.21.15.49:0/2811711608 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb44c15cbb0 0x7fb44c15e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:20.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:20.502+0000 7fb4539a1640 1 --2- 172.21.15.49:0/2811711608 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb44c0d24e0 0x7fb44c0cb790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:20.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:20.502+0000 7fb4539a1640 1 -- 172.21.15.49:0/2811711608 >> 172.21.15.49:0/2811711608 conn(0x7fb44c0718a0 msgr2=0x7fb44c071c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:20.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:20.502+0000 7fb4539a1640 1 -- 172.21.15.49:0/2811711608 shutdown_connections 2023-11-06T02:11:20.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:20.502+0000 7fb4539a1640 1 -- 172.21.15.49:0/2811711608 wait complete. 2023-11-06T02:11:20.519 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-06T02:11:20.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:20.646+0000 7fce5ac7e640 1 Processor -- start 2023-11-06T02:11:20.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:20.646+0000 7fce5ac7e640 1 -- start start 2023-11-06T02:11:20.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:20.650+0000 7fce5ac7e640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fce5415abd0 0x7fce54069d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:20.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:20.650+0000 7fce5ac7e640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fce5415e780 0x7fce5406a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:20.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:20.650+0000 7fce5ac7e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fce540ad720 0x7fce540b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:20.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:20.650+0000 7fce5ac7e640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce540c94b0 con 0x7fce5415abd0 2023-11-06T02:11:20.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:20.650+0000 7fce5ac7e640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce5415ecd0 con 0x7fce5415e780 2023-11-06T02:11:20.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:20.650+0000 7fce5ac7e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce5415efb0 con 0x7fce540ad720 2023-11-06T02:11:20.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:20.650+0000 7fce58a1a640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fce5415abd0 0x7fce54069d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:20.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:20.650+0000 7fce58a1a640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fce5415abd0 0x7fce54069d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49392/0 (socket says 172.21.15.49:49392) 2023-11-06T02:11:20.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:20.650+0000 7fce58a1a640 1 -- 172.21.15.49:0/3794874662 learned_addr learned my addr 172.21.15.49:0/3794874662 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:20.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:20.650+0000 7fce53fff640 1 --2- 172.21.15.49:0/3794874662 >> v2:172.21.15.150:3300/0 conn(0x7fce5415e780 0x7fce5406a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:20.648 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:20.650+0000 7fce5921b640 1 --2- 172.21.15.49:0/3794874662 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fce540ad720 0x7fce540b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:20.648 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:20.650+0000 7fce58a1a640 1 -- 172.21.15.49:0/3794874662 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fce540ad720 msgr2=0x7fce540b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:20.648 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:20.650+0000 7fce58a1a640 1 --2- 172.21.15.49:0/3794874662 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fce540ad720 0x7fce540b0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:20.648 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:20.650+0000 7fce58a1a640 1 -- 172.21.15.49:0/3794874662 >> v2:172.21.15.150:3300/0 conn(0x7fce5415e780 msgr2=0x7fce5406a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:20.648 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:20.650+0000 7fce58a1a640 1 --2- 172.21.15.49:0/3794874662 >> v2:172.21.15.150:3300/0 conn(0x7fce5415e780 0x7fce5406a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:20.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:20.650+0000 7fce58a1a640 1 -- 172.21.15.49:0/3794874662 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce5415f290 con 0x7fce5415abd0 2023-11-06T02:11:20.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:20.650+0000 7fce53fff640 1 --2- 172.21.15.49:0/3794874662 >> v2:172.21.15.150:3300/0 conn(0x7fce5415e780 0x7fce5406a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:11:20.650 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:20.654+0000 7fce58a1a640 1 --2- 172.21.15.49:0/3794874662 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fce5415abd0 0x7fce54069d70 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fce540c8b60 tx=0x7fce4401c5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a5703f0242c9849 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:20.651 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:20.654+0000 7fce537fe640 1 -- 172.21.15.49:0/3794874662 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce440a6470 con 0x7fce5415abd0 2023-11-06T02:11:20.651 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:20.654+0000 7fce537fe640 1 -- 172.21.15.49:0/3794874662 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce440a65d0 con 0x7fce5415abd0 2023-11-06T02:11:20.651 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:20.654+0000 7fce537fe640 1 -- 172.21.15.49:0/3794874662 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce440a6960 con 0x7fce5415abd0 2023-11-06T02:11:20.651 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:20.654+0000 7fce5ac7e640 1 -- 172.21.15.49:0/3794874662 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fce5415abd0 msgr2=0x7fce54069d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:20.652 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:20.654+0000 7fce5ac7e640 1 --2- 172.21.15.49:0/3794874662 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fce5415abd0 0x7fce54069d70 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fce540c8b60 tx=0x7fce4401c5c0 comp rx=0 tx=0).stop 2023-11-06T02:11:20.652 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:20.654+0000 7fce5ac7e640 1 -- 172.21.15.49:0/3794874662 shutdown_connections 2023-11-06T02:11:20.652 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:20.654+0000 7fce5ac7e640 1 --2- 172.21.15.49:0/3794874662 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fce540ad720 0x7fce540b0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:20.652 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:20.654+0000 7fce5ac7e640 1 --2- 172.21.15.49:0/3794874662 >> v2:172.21.15.150:3300/0 conn(0x7fce5415e780 0x7fce5406a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:20.652 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:20.654+0000 7fce5ac7e640 1 --2- 172.21.15.49:0/3794874662 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fce5415abd0 0x7fce54069d70 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:20.653 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:20.654+0000 7fce5ac7e640 1 -- 172.21.15.49:0/3794874662 >> 172.21.15.49:0/3794874662 conn(0x7fce540ce610 msgr2=0x7fce540cea20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:20.653 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:20.654+0000 7fce5ac7e640 1 -- 172.21.15.49:0/3794874662 shutdown_connections 2023-11-06T02:11:20.653 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:20.654+0000 7fce5ac7e640 1 -- 172.21.15.49:0/3794874662 wait complete. 2023-11-06T02:11:20.653 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:20.654+0000 7fce5ac7e640 1 Processor -- start 2023-11-06T02:11:20.653 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:20.654+0000 7fce5ac7e640 1 -- start start 2023-11-06T02:11:20.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:20.654+0000 7fce5ac7e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fce540ad720 0x7fce54154d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:20.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:20.654+0000 7fce5ac7e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fce5415abd0 0x7fce540c7d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:20.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:20.654+0000 7fce5ac7e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce5415e780 0x7fce5415ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:20.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:20.654+0000 7fce5ac7e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce54155240 con 0x7fce540ad720 2023-11-06T02:11:20.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:20.654+0000 7fce5ac7e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce540b35b0 con 0x7fce5415e780 2023-11-06T02:11:20.657 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:20.654+0000 7fce5ac7e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce540b2510 con 0x7fce5415abd0 2023-11-06T02:11:20.657 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:20.654+0000 7fce58a1a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fce540ad720 0x7fce54154d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:20.657 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:20.654+0000 7fce58a1a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fce540ad720 0x7fce54154d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49402/0 (socket says 172.21.15.49:49402) 2023-11-06T02:11:20.657 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:20.654+0000 7fce58a1a640 1 -- 172.21.15.49:0/2965762030 learned_addr learned my addr 172.21.15.49:0/2965762030 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:20.657 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:20.654+0000 7fce5921b640 1 --2- 172.21.15.49:0/2965762030 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce5415e780 0x7fce5415ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:20.658 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:20.654+0000 7fce53fff640 1 --2- 172.21.15.49:0/2965762030 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fce5415abd0 0x7fce540c7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:20.658 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:20.654+0000 7fce58a1a640 1 -- 172.21.15.49:0/2965762030 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fce5415abd0 msgr2=0x7fce540c7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:20.658 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:20.654+0000 7fce58a1a640 1 --2- 172.21.15.49:0/2965762030 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fce5415abd0 0x7fce540c7d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:20.658 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:20.654+0000 7fce58a1a640 1 -- 172.21.15.49:0/2965762030 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce5415e780 msgr2=0x7fce5415ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:20.658 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:20.654+0000 7fce58a1a640 1 --2- 172.21.15.49:0/2965762030 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce5415e780 0x7fce5415ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:20.659 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:20.654+0000 7fce58a1a640 1 -- 172.21.15.49:0/2965762030 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce44009b30 con 0x7fce540ad720 2023-11-06T02:11:20.659 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:20.654+0000 7fce58a1a640 1 --2- 172.21.15.49:0/2965762030 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fce540ad720 0x7fce54154d00 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fce4401ac70 tx=0x7fce44019360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:20.659 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:20.658+0000 7fce517fa640 1 -- 172.21.15.49:0/2965762030 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce440a6470 con 0x7fce540ad720 2023-11-06T02:11:20.659 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:20.658+0000 7fce517fa640 1 -- 172.21.15.49:0/2965762030 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce440a7650 con 0x7fce540ad720 2023-11-06T02:11:20.659 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:20.658+0000 7fce5ac7e640 1 -- 172.21.15.49:0/2965762030 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce540b27f0 con 0x7fce540ad720 2023-11-06T02:11:20.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:20.658+0000 7fce517fa640 1 -- 172.21.15.49:0/2965762030 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce440a79e0 con 0x7fce540ad720 2023-11-06T02:11:20.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:20.658+0000 7fce5ac7e640 1 -- 172.21.15.49:0/2965762030 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce540b2db0 con 0x7fce540ad720 2023-11-06T02:11:20.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:20.658+0000 7fce5ac7e640 1 -- 172.21.15.49:0/2965762030 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce54074040 con 0x7fce540ad720 2023-11-06T02:11:20.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:20.658+0000 7fce517fa640 1 -- 172.21.15.49:0/2965762030 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fce440a7b40 con 0x7fce540ad720 2023-11-06T02:11:20.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:20.658+0000 7fce517fa640 1 --2- 172.21.15.49:0/2965762030 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fce40065a60 0x7fce40067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:20.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:20.658+0000 7fce517fa640 1 -- 172.21.15.49:0/2965762030 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6206+0+0 (secure 0 0 0) 0x7fce4411de80 con 0x7fce540ad720 2023-11-06T02:11:20.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:20.658+0000 7fce53fff640 1 --2- 172.21.15.49:0/2965762030 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fce40065a60 0x7fce40067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:20.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:20.658+0000 7fce53fff640 1 --2- 172.21.15.49:0/2965762030 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fce40065a60 0x7fce40067f20 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fce540b0090 tx=0x7fce48008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:20.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:20.662+0000 7fce517fa640 1 -- 172.21.15.49:0/2965762030 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fce440eb6e0 con 0x7fce540ad720 2023-11-06T02:11:20.819 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:20.822+0000 7fce5ac7e640 1 -- 172.21.15.49:0/2965762030 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fce540c8d80 con 0x7fce540ad720 2023-11-06T02:11:21.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:21.502+0000 7fce517fa640 1 -- 172.21.15.49:0/2965762030 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v55) v1 ==== 181+0+0 (secure 0 0 0) 0x7fce44009510 con 0x7fce540ad720 2023-11-06T02:11:21.500 INFO:teuthology.orchestra.run.smithi049.stderr:pool 'cephfs_metadata' removed 2023-11-06T02:11:21.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:21.510+0000 7fce5ac7e640 1 -- 172.21.15.49:0/2965762030 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fce40065a60 msgr2=0x7fce40067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:21.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:21.510+0000 7fce5ac7e640 1 --2- 172.21.15.49:0/2965762030 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fce40065a60 0x7fce40067f20 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fce540b0090 tx=0x7fce48008040 comp rx=0 tx=0).stop 2023-11-06T02:11:21.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:21.510+0000 7fce5ac7e640 1 -- 172.21.15.49:0/2965762030 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fce540ad720 msgr2=0x7fce54154d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:21.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:21.510+0000 7fce5ac7e640 1 --2- 172.21.15.49:0/2965762030 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fce540ad720 0x7fce54154d00 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fce4401ac70 tx=0x7fce44019360 comp rx=0 tx=0).stop 2023-11-06T02:11:21.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:21.510+0000 7fce5ac7e640 1 -- 172.21.15.49:0/2965762030 shutdown_connections 2023-11-06T02:11:21.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:21.510+0000 7fce5ac7e640 1 --2- 172.21.15.49:0/2965762030 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fce40065a60 0x7fce40067f20 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:21.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:21.510+0000 7fce5ac7e640 1 --2- 172.21.15.49:0/2965762030 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce5415e780 0x7fce5415ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:21.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:21.510+0000 7fce5ac7e640 1 --2- 172.21.15.49:0/2965762030 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fce5415abd0 0x7fce540c7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:21.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:21.510+0000 7fce5ac7e640 1 --2- 172.21.15.49:0/2965762030 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fce540ad720 0x7fce54154d00 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:21.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:21.510+0000 7fce5ac7e640 1 -- 172.21.15.49:0/2965762030 >> 172.21.15.49:0/2965762030 conn(0x7fce540ce610 msgr2=0x7fce54069d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:21.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:21.510+0000 7fce5ac7e640 1 -- 172.21.15.49:0/2965762030 shutdown_connections 2023-11-06T02:11:21.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:21.510+0000 7fce5ac7e640 1 -- 172.21.15.49:0/2965762030 wait complete. 2023-11-06T02:11:21.533 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-06T02:11:21.670 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:21.674+0000 7f029fcf3640 1 Processor -- start 2023-11-06T02:11:21.670 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:21.674+0000 7f029fcf3640 1 -- start start 2023-11-06T02:11:21.671 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:21.674+0000 7f029fcf3640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f029815ceb0 0x7f02980cb970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:21.671 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:21.674+0000 7f029fcf3640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f02980cb370 0x7f0298069d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:21.672 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:21.674+0000 7f029fcf3640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02980cbeb0 con 0x7f02980caf30 2023-11-06T02:11:21.672 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:21.674+0000 7f029fcf3640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02980caaa0 con 0x7f029815ceb0 2023-11-06T02:11:21.672 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:21.674+0000 7f029fcf3640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02980ca1e0 con 0x7f02980cb370 2023-11-06T02:11:21.672 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:21.674+0000 7f029d28e640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f02980caf30 0x7f0298071120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:45960/0 (socket says 172.21.15.49:45960) 2023-11-06T02:11:21.672 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:21.674+0000 7f029d28e640 1 -- 172.21.15.49:0/2319471461 learned_addr learned my addr 172.21.15.49:0/2319471461 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:11:21.673 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:21.674+0000 7f029da8f640 1 --2- 172.21.15.49:0/2319471461 >> v2:172.21.15.49:3300/0 conn(0x7f029815ceb0 0x7f02980cb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:21.673 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:21.674+0000 7f029e290640 1 --2- 172.21.15.49:0/2319471461 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f02980cb370 0x7f0298069d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:21.673 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:21.674+0000 7f029da8f640 1 -- 172.21.15.49:0/2319471461 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f02980cb370 msgr2=0x7f0298069d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:21.673 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:21.674+0000 7f029da8f640 1 --2- 172.21.15.49:0/2319471461 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f02980cb370 0x7f0298069d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:21.673 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:21.674+0000 7f029da8f640 1 -- 172.21.15.49:0/2319471461 >> v1:172.21.15.49:6789/0 conn(0x7f02980caf30 legacy=0x7f0298071120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:21.674 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:21.674+0000 7f029da8f640 1 -- 172.21.15.49:0/2319471461 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02980ce490 con 0x7f029815ceb0 2023-11-06T02:11:21.675 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:21.678+0000 7f029da8f640 1 --2- 172.21.15.49:0/2319471461 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f029815ceb0 0x7f02980cb970 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f02940097c0 tx=0x7f029401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9acf8387704cc602 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:21.675 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:21.678+0000 7f029ca8d640 1 -- 172.21.15.49:0/2319471461 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02940ab600 con 0x7f029815ceb0 2023-11-06T02:11:21.675 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:21.678+0000 7f029ca8d640 1 -- 172.21.15.49:0/2319471461 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02940a7050 con 0x7f029815ceb0 2023-11-06T02:11:21.675 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:21.678+0000 7f029ca8d640 1 -- 172.21.15.49:0/2319471461 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02940ab980 con 0x7f029815ceb0 2023-11-06T02:11:21.676 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:21.678+0000 7f029fcf3640 1 -- 172.21.15.49:0/2319471461 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f029815ceb0 msgr2=0x7f02980cb970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:21.676 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:21.678+0000 7f029fcf3640 1 --2- 172.21.15.49:0/2319471461 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f029815ceb0 0x7f02980cb970 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f02940097c0 tx=0x7f029401c480 comp rx=0 tx=0).stop 2023-11-06T02:11:21.676 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:21.678+0000 7f029fcf3640 1 -- 172.21.15.49:0/2319471461 shutdown_connections 2023-11-06T02:11:21.676 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:21.678+0000 7f029fcf3640 1 --2- 172.21.15.49:0/2319471461 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f02980cb370 0x7f0298069d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:21.676 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:21.678+0000 7f029fcf3640 1 --2- 172.21.15.49:0/2319471461 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f029815ceb0 0x7f02980cb970 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:21.677 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:21.678+0000 7f029fcf3640 1 -- 172.21.15.49:0/2319471461 >> 172.21.15.49:0/2319471461 conn(0x7f02980cc140 msgr2=0x7f02980cc550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:21.677 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:21.678+0000 7f029fcf3640 1 -- 172.21.15.49:0/2319471461 shutdown_connections 2023-11-06T02:11:21.677 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:21.678+0000 7f029fcf3640 1 -- 172.21.15.49:0/2319471461 wait complete. 2023-11-06T02:11:21.679 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:21.678+0000 7f029fcf3640 1 Processor -- start 2023-11-06T02:11:21.679 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:21.678+0000 7f029fcf3640 1 -- start start 2023-11-06T02:11:21.679 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:21.682+0000 7f029fcf3640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f02980caf30 0x7f0298152870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:21.679 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:21.682+0000 7f029fcf3640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f029815ceb0 0x7f029814d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:21.680 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:21.682+0000 7f029fcf3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02980c7b00 0x7f029814bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:21.680 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:21.682+0000 7f029fcf3640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02980c8c80 con 0x7f02980c7b00 2023-11-06T02:11:21.680 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:21.682+0000 7f029fcf3640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02980c8070 con 0x7f02980caf30 2023-11-06T02:11:21.680 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:21.682+0000 7f029fcf3640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02980c8350 con 0x7f029815ceb0 2023-11-06T02:11:21.680 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:21.682+0000 7f029e290640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02980c7b00 0x7f029814bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:21.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:21.682+0000 7f029e290640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02980c7b00 0x7f029814bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49428/0 (socket says 172.21.15.49:49428) 2023-11-06T02:11:21.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:21.682+0000 7f029e290640 1 -- 172.21.15.49:0/70040046 learned_addr learned my addr 172.21.15.49:0/70040046 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:21.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:21.682+0000 7f029d28e640 1 --2- 172.21.15.49:0/70040046 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f029815ceb0 0x7f029814d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:21.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:21.682+0000 7f029da8f640 1 --2- 172.21.15.49:0/70040046 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f02980caf30 0x7f0298152870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:21.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:21.682+0000 7f029e290640 1 -- 172.21.15.49:0/70040046 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f029815ceb0 msgr2=0x7f029814d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:21.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:21.682+0000 7f029e290640 1 --2- 172.21.15.49:0/70040046 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f029815ceb0 0x7f029814d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:21.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:21.682+0000 7f029e290640 1 -- 172.21.15.49:0/70040046 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f02980caf30 msgr2=0x7f0298152870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:21.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:21.682+0000 7f029e290640 1 --2- 172.21.15.49:0/70040046 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f02980caf30 0x7f0298152870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:21.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:21.682+0000 7f029e290640 1 -- 172.21.15.49:0/70040046 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0294009420 con 0x7f02980c7b00 2023-11-06T02:11:21.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:21.682+0000 7f029e290640 1 --2- 172.21.15.49:0/70040046 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02980c7b00 0x7f029814bbd0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f0290007e30 tx=0x7f029000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:21.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:21.682+0000 7f028e7fc640 1 -- 172.21.15.49:0/70040046 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0290014620 con 0x7f02980c7b00 2023-11-06T02:11:21.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:21.682+0000 7f028e7fc640 1 -- 172.21.15.49:0/70040046 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f029000ee90 con 0x7f02980c7b00 2023-11-06T02:11:21.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:21.682+0000 7f029fcf3640 1 -- 172.21.15.49:0/70040046 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02980c8630 con 0x7f02980c7b00 2023-11-06T02:11:21.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:21.682+0000 7f028e7fc640 1 -- 172.21.15.49:0/70040046 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02900149c0 con 0x7f02980c7b00 2023-11-06T02:11:21.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:21.682+0000 7f029fcf3640 1 -- 172.21.15.49:0/70040046 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f029814dea0 con 0x7f02980c7b00 2023-11-06T02:11:21.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:21.682+0000 7f029fcf3640 1 -- 172.21.15.49:0/70040046 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02980c8df0 con 0x7f02980c7b00 2023-11-06T02:11:21.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:21.682+0000 7f028e7fc640 1 -- 172.21.15.49:0/70040046 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f0290014b20 con 0x7f02980c7b00 2023-11-06T02:11:21.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:21.682+0000 7f028e7fc640 1 --2- 172.21.15.49:0/70040046 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f027c065a60 0x7f027c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:21.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:21.682+0000 7f028e7fc640 1 -- 172.21.15.49:0/70040046 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5768+0+0 (secure 0 0 0) 0x7f0290083be0 con 0x7f02980c7b00 2023-11-06T02:11:21.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:21.682+0000 7f029da8f640 1 --2- 172.21.15.49:0/70040046 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f027c065a60 0x7f027c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:21.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:21.686+0000 7f029da8f640 1 --2- 172.21.15.49:0/70040046 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f027c065a60 0x7f027c067f20 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f02940ad000 tx=0x7f0294008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:21.686 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:21.690+0000 7f028e7fc640 1 -- 172.21.15.49:0/70040046 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f02900514d0 con 0x7f02980c7b00 2023-11-06T02:11:21.810 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T02:11:21.813 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T02:11:21.840 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:21.846+0000 7f029fcf3640 1 -- 172.21.15.49:0/70040046 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0298067140 con 0x7f02980c7b00 2023-11-06T02:11:22.506 INFO:teuthology.orchestra.run.smithi049.stderr:pool 'cephfs_data' removed 2023-11-06T02:11:22.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.506+0000 7f028e7fc640 1 -- 172.21.15.49:0/70040046 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v56) v1 ==== 169+0+0 (secure 0 0 0) 0x7f0290054af0 con 0x7f02980c7b00 2023-11-06T02:11:22.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.510+0000 7f029fcf3640 1 -- 172.21.15.49:0/70040046 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f027c065a60 msgr2=0x7f027c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:22.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.510+0000 7f029fcf3640 1 --2- 172.21.15.49:0/70040046 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f027c065a60 0x7f027c067f20 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f02940ad000 tx=0x7f0294008040 comp rx=0 tx=0).stop 2023-11-06T02:11:22.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.514+0000 7f029fcf3640 1 -- 172.21.15.49:0/70040046 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02980c7b00 msgr2=0x7f029814bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:22.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.514+0000 7f029fcf3640 1 --2- 172.21.15.49:0/70040046 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02980c7b00 0x7f029814bbd0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f0290007e30 tx=0x7f029000e4b0 comp rx=0 tx=0).stop 2023-11-06T02:11:22.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.514+0000 7f029fcf3640 1 -- 172.21.15.49:0/70040046 shutdown_connections 2023-11-06T02:11:22.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.514+0000 7f029fcf3640 1 --2- 172.21.15.49:0/70040046 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f027c065a60 0x7f027c067f20 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:22.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.514+0000 7f029fcf3640 1 --2- 172.21.15.49:0/70040046 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02980c7b00 0x7f029814bbd0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:22.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.514+0000 7f029fcf3640 1 --2- 172.21.15.49:0/70040046 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f029815ceb0 0x7f029814d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:22.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.514+0000 7f029fcf3640 1 --2- 172.21.15.49:0/70040046 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f02980caf30 0x7f0298152870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:22.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.514+0000 7f029fcf3640 1 -- 172.21.15.49:0/70040046 >> 172.21.15.49:0/70040046 conn(0x7f02980cc140 msgr2=0x7f02980ac8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:22.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.514+0000 7f029fcf3640 1 -- 172.21.15.49:0/70040046 shutdown_connections 2023-11-06T02:11:22.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.514+0000 7f029fcf3640 1 -- 172.21.15.49:0/70040046 wait complete. 2023-11-06T02:11:22.536 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-06T02:11:22.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.666+0000 7fe312e39640 1 Processor -- start 2023-11-06T02:11:22.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.666+0000 7fe312e39640 1 -- start start 2023-11-06T02:11:22.662 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.666+0000 7fe312e39640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fe30c1595a0 0x7fe30c15dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:22.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.666+0000 7fe312e39640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe30c15a990 0x7fe30c0c8e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:22.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.666+0000 7fe312e39640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe30c06c5a0 con 0x7fe30c06d640 2023-11-06T02:11:22.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.666+0000 7fe312e39640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe30c15e250 con 0x7fe30c15a990 2023-11-06T02:11:22.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.666+0000 7fe312e39640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe30c06c880 con 0x7fe30c1595a0 2023-11-06T02:11:22.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.666+0000 7fe3113d6640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe30c15a990 0x7fe30c0c8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:22.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.666+0000 7fe3113d6640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe30c15a990 0x7fe30c0c8e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49438/0 (socket says 172.21.15.49:49438) 2023-11-06T02:11:22.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.666+0000 7fe3113d6640 1 -- 172.21.15.49:0/2201119776 learned_addr learned my addr 172.21.15.49:0/2201119776 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:22.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.666+0000 7fe310bd5640 1 --2- 172.21.15.49:0/2201119776 >> v2:172.21.15.150:3300/0 conn(0x7fe30c1595a0 0x7fe30c15dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:22.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.666+0000 7fe3113d6640 1 -- 172.21.15.49:0/2201119776 >> v2:172.21.15.150:3300/0 conn(0x7fe30c1595a0 msgr2=0x7fe30c15dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:22.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.666+0000 7fe3113d6640 1 --2- 172.21.15.49:0/2201119776 >> v2:172.21.15.150:3300/0 conn(0x7fe30c1595a0 0x7fe30c15dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:22.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.666+0000 7fe3113d6640 1 -- 172.21.15.49:0/2201119776 >> v1:172.21.15.150:6789/0 conn(0x7fe30c06d640 legacy=0x7fe30c0b0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:22.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.666+0000 7fe3113d6640 1 -- 172.21.15.49:0/2201119776 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe30c06cb60 con 0x7fe30c15a990 2023-11-06T02:11:22.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.666+0000 7fe3113d6640 1 --2- 172.21.15.49:0/2201119776 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe30c15a990 0x7fe30c0c8e90 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fe30c071420 tx=0x7fe30401c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ea52e49a271098b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:22.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.670+0000 7fe30b7fe640 1 -- 172.21.15.49:0/2201119776 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3040a5730 con 0x7fe30c15a990 2023-11-06T02:11:22.667 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.670+0000 7fe30b7fe640 1 -- 172.21.15.49:0/2201119776 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3040a5890 con 0x7fe30c15a990 2023-11-06T02:11:22.667 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.670+0000 7fe30b7fe640 1 -- 172.21.15.49:0/2201119776 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3040a5c20 con 0x7fe30c15a990 2023-11-06T02:11:22.667 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.670+0000 7fe312e39640 1 -- 172.21.15.49:0/2201119776 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe30c15a990 msgr2=0x7fe30c0c8e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:22.667 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.670+0000 7fe312e39640 1 --2- 172.21.15.49:0/2201119776 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe30c15a990 0x7fe30c0c8e90 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fe30c071420 tx=0x7fe30401c960 comp rx=0 tx=0).stop 2023-11-06T02:11:22.667 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.670+0000 7fe312e39640 1 -- 172.21.15.49:0/2201119776 shutdown_connections 2023-11-06T02:11:22.668 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.670+0000 7fe312e39640 1 --2- 172.21.15.49:0/2201119776 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe30c15a990 0x7fe30c0c8e90 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:22.668 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.670+0000 7fe312e39640 1 --2- 172.21.15.49:0/2201119776 >> v2:172.21.15.150:3300/0 conn(0x7fe30c1595a0 0x7fe30c15dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:22.668 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.670+0000 7fe312e39640 1 -- 172.21.15.49:0/2201119776 >> 172.21.15.49:0/2201119776 conn(0x7fe30c06a370 msgr2=0x7fe30c0ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:22.668 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.670+0000 7fe312e39640 1 -- 172.21.15.49:0/2201119776 shutdown_connections 2023-11-06T02:11:22.668 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.670+0000 7fe312e39640 1 -- 172.21.15.49:0/2201119776 wait complete. 2023-11-06T02:11:22.671 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.670+0000 7fe312e39640 1 Processor -- start 2023-11-06T02:11:22.671 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.670+0000 7fe312e39640 1 -- start start 2023-11-06T02:11:22.672 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.670+0000 7fe312e39640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe30c06d640 0x7fe30c14afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:22.672 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.670+0000 7fe312e39640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe30c1595a0 0x7fe30c0cdba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:22.672 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.670+0000 7fe312e39640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe30c15a990 0x7fe30c142a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:22.672 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.670+0000 7fe312e39640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe30c1660a0 con 0x7fe30c06d640 2023-11-06T02:11:22.672 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.670+0000 7fe312e39640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe30c165740 con 0x7fe30c1595a0 2023-11-06T02:11:22.673 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.670+0000 7fe312e39640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe30c1659e0 con 0x7fe30c15a990 2023-11-06T02:11:22.673 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.670+0000 7fe310bd5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe30c06d640 0x7fe30c14afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:22.673 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.670+0000 7fe310bd5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe30c06d640 0x7fe30c14afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49440/0 (socket says 172.21.15.49:49440) 2023-11-06T02:11:22.673 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.670+0000 7fe310bd5640 1 -- 172.21.15.49:0/3912216941 learned_addr learned my addr 172.21.15.49:0/3912216941 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:22.673 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.674+0000 7fe30bfff640 1 --2- 172.21.15.49:0/3912216941 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe30c1595a0 0x7fe30c0cdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:22.674 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.674+0000 7fe3113d6640 1 --2- 172.21.15.49:0/3912216941 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe30c15a990 0x7fe30c142a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:22.674 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.674+0000 7fe310bd5640 1 -- 172.21.15.49:0/3912216941 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe30c15a990 msgr2=0x7fe30c142a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:22.674 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.674+0000 7fe310bd5640 1 --2- 172.21.15.49:0/3912216941 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe30c15a990 0x7fe30c142a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:22.674 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.674+0000 7fe310bd5640 1 -- 172.21.15.49:0/3912216941 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe30c1595a0 msgr2=0x7fe30c0cdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:22.674 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.674+0000 7fe310bd5640 1 --2- 172.21.15.49:0/3912216941 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe30c1595a0 0x7fe30c0cdba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:22.675 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.674+0000 7fe310bd5640 1 -- 172.21.15.49:0/3912216941 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe304009b30 con 0x7fe30c06d640 2023-11-06T02:11:22.675 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.674+0000 7fe30bfff640 1 --2- 172.21.15.49:0/3912216941 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe30c1595a0 0x7fe30c0cdba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:11:22.675 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.674+0000 7fe3113d6640 1 --2- 172.21.15.49:0/3912216941 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe30c15a990 0x7fe30c142a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:11:22.675 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.674+0000 7fe310bd5640 1 --2- 172.21.15.49:0/3912216941 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe30c06d640 0x7fe30c14afd0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fe30000d800 tx=0x7fe30000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:22.675 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.674+0000 7fe3097fa640 1 -- 172.21.15.49:0/3912216941 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe30000b6c0 con 0x7fe30c06d640 2023-11-06T02:11:22.676 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.674+0000 7fe3097fa640 1 -- 172.21.15.49:0/3912216941 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe30000b820 con 0x7fe30c06d640 2023-11-06T02:11:22.676 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.674+0000 7fe312e39640 1 -- 172.21.15.49:0/3912216941 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe30c165cc0 con 0x7fe30c06d640 2023-11-06T02:11:22.676 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.674+0000 7fe3097fa640 1 -- 172.21.15.49:0/3912216941 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe30000bb90 con 0x7fe30c06d640 2023-11-06T02:11:22.676 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.674+0000 7fe312e39640 1 -- 172.21.15.49:0/3912216941 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe30c142f50 con 0x7fe30c06d640 2023-11-06T02:11:22.676 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.674+0000 7fe312e39640 1 -- 172.21.15.49:0/3912216941 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe30c06c210 con 0x7fe30c06d640 2023-11-06T02:11:22.676 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.674+0000 7fe3097fa640 1 -- 172.21.15.49:0/3912216941 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fe30000bcf0 con 0x7fe30c06d640 2023-11-06T02:11:22.677 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.674+0000 7fe3097fa640 1 --2- 172.21.15.49:0/3912216941 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe2f0065a60 0x7fe2f0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:22.677 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.674+0000 7fe3097fa640 1 -- 172.21.15.49:0/3912216941 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fe3000834b0 con 0x7fe30c06d640 2023-11-06T02:11:22.677 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.674+0000 7fe30bfff640 1 --2- 172.21.15.49:0/3912216941 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe2f0065a60 0x7fe2f0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:22.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.674+0000 7fe30bfff640 1 --2- 172.21.15.49:0/3912216941 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe2f0065a60 0x7fe2f0067f20 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fe30c070520 tx=0x7fe2f8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:22.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.678+0000 7fe3097fa640 1 -- 172.21.15.49:0/3912216941 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fe30c06c210 con 0x7fe30c06d640 2023-11-06T02:11:22.840 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.842+0000 7fe312e39640 1 -- 172.21.15.49:0/3912216941 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fe30c06dab0 con 0x7fe30c06d640 2023-11-06T02:11:22.847 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.850+0000 7fe3097fa640 1 -- 172.21.15.49:0/3912216941 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe30c06dab0 con 0x7fe30c06d640 2023-11-06T02:11:22.850 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.854+0000 7fe312e39640 1 -- 172.21.15.49:0/3912216941 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe2f0065a60 msgr2=0x7fe2f0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:22.850 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.854+0000 7fe312e39640 1 --2- 172.21.15.49:0/3912216941 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe2f0065a60 0x7fe2f0067f20 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fe30c070520 tx=0x7fe2f8009040 comp rx=0 tx=0).stop 2023-11-06T02:11:22.850 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.854+0000 7fe312e39640 1 -- 172.21.15.49:0/3912216941 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe30c06d640 msgr2=0x7fe30c14afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:22.850 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.854+0000 7fe312e39640 1 --2- 172.21.15.49:0/3912216941 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe30c06d640 0x7fe30c14afd0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fe30000d800 tx=0x7fe30000dcc0 comp rx=0 tx=0).stop 2023-11-06T02:11:22.850 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.854+0000 7fe312e39640 1 -- 172.21.15.49:0/3912216941 shutdown_connections 2023-11-06T02:11:22.851 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.854+0000 7fe312e39640 1 --2- 172.21.15.49:0/3912216941 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe2f0065a60 0x7fe2f0067f20 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:22.851 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.854+0000 7fe312e39640 1 --2- 172.21.15.49:0/3912216941 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe30c15a990 0x7fe30c142a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:22.851 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.854+0000 7fe312e39640 1 --2- 172.21.15.49:0/3912216941 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe30c1595a0 0x7fe30c0cdba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:22.851 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.854+0000 7fe312e39640 1 --2- 172.21.15.49:0/3912216941 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe30c06d640 0x7fe30c14afd0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:22.852 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.854+0000 7fe312e39640 1 -- 172.21.15.49:0/3912216941 >> 172.21.15.49:0/3912216941 conn(0x7fe30c06a370 msgr2=0x7fe30c0ce5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:22.852 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.854+0000 7fe312e39640 1 -- 172.21.15.49:0/3912216941 shutdown_connections 2023-11-06T02:11:22.852 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.854+0000 7fe312e39640 1 -- 172.21.15.49:0/3912216941 wait complete. 2023-11-06T02:11:22.872 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair' 2023-11-06T02:11:22.993 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.998+0000 7f5629b68640 1 Processor -- start 2023-11-06T02:11:22.993 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.998+0000 7f5629b68640 1 -- start start 2023-11-06T02:11:22.995 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.998+0000 7f5629b68640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f56240b1f90 0x7f56240cda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:22.995 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.998+0000 7f5629b68640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f5624161650 0x7f56240d1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:22.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.998+0000 7f5629b68640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f56240d2190 0x7f56240714a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:22.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.998+0000 7f5629b68640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56240d17c0 con 0x7f56240d2190 2023-11-06T02:11:22.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.998+0000 7f5629b68640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56240ce6c0 con 0x7f5624161650 2023-11-06T02:11:22.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.998+0000 7f5629b68640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5624161ad0 con 0x7f56240b1f90 2023-11-06T02:11:22.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.998+0000 7f5623fff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f56240d2190 0x7f56240714a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:22.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.998+0000 7f5623fff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f56240d2190 0x7f56240714a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49456/0 (socket says 172.21.15.49:49456) 2023-11-06T02:11:22.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.998+0000 7f5623fff640 1 -- 172.21.15.49:0/2715793692 learned_addr learned my addr 172.21.15.49:0/2715793692 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:22.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.998+0000 7f5623fff640 1 -- 172.21.15.49:0/2715793692 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f56240b1f90 msgr2=0x7f56240cda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:22.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.998+0000 7f56237fe640 1 --2- 172.21.15.49:0/2715793692 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f56240b1f90 0x7f56240cda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:22.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.998+0000 7f5622ffd640 1 --2- 172.21.15.49:0/2715793692 >> v2:172.21.15.150:3300/0 conn(0x7f5624161650 0x7f56240d1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:22.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.998+0000 7f5623fff640 1 --2- 172.21.15.49:0/2715793692 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f56240b1f90 0x7f56240cda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:22.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.998+0000 7f5623fff640 1 -- 172.21.15.49:0/2715793692 >> v2:172.21.15.150:3300/0 conn(0x7f5624161650 msgr2=0x7f56240d1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:22.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.998+0000 7f5623fff640 1 --2- 172.21.15.49:0/2715793692 >> v2:172.21.15.150:3300/0 conn(0x7f5624161650 0x7f56240d1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:22.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.998+0000 7f5623fff640 1 -- 172.21.15.49:0/2715793692 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56240ce100 con 0x7f56240d2190 2023-11-06T02:11:22.999 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.998+0000 7f56237fe640 1 --2- 172.21.15.49:0/2715793692 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f56240b1f90 0x7f56240cda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:11:22.999 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:22.998+0000 7f5622ffd640 1 --2- 172.21.15.49:0/2715793692 >> v2:172.21.15.150:3300/0 conn(0x7f5624161650 0x7f56240d1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:11:23.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.002+0000 7f5623fff640 1 --2- 172.21.15.49:0/2715793692 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56240d2190 0x7f56240714a0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f56240b5660 tx=0x7f561c01c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e70be8d7e19b53ec server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:23.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.002+0000 7f56227fc640 1 -- 172.21.15.49:0/2715793692 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f561c0a5730 con 0x7f56240d2190 2023-11-06T02:11:23.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.002+0000 7f56227fc640 1 -- 172.21.15.49:0/2715793692 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f561c0a5890 con 0x7f56240d2190 2023-11-06T02:11:23.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.002+0000 7f56227fc640 1 -- 172.21.15.49:0/2715793692 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f561c0a5c20 con 0x7f56240d2190 2023-11-06T02:11:23.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.002+0000 7f5629b68640 1 -- 172.21.15.49:0/2715793692 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56240d2190 msgr2=0x7f56240714a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:23.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.002+0000 7f5629b68640 1 --2- 172.21.15.49:0/2715793692 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56240d2190 0x7f56240714a0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f56240b5660 tx=0x7f561c01c9a0 comp rx=0 tx=0).stop 2023-11-06T02:11:23.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.002+0000 7f5629b68640 1 -- 172.21.15.49:0/2715793692 shutdown_connections 2023-11-06T02:11:23.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.002+0000 7f5629b68640 1 --2- 172.21.15.49:0/2715793692 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56240d2190 0x7f56240714a0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:23.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.002+0000 7f5629b68640 1 --2- 172.21.15.49:0/2715793692 >> v2:172.21.15.150:3300/0 conn(0x7f5624161650 0x7f56240d1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:23.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.002+0000 7f5629b68640 1 --2- 172.21.15.49:0/2715793692 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f56240b1f90 0x7f56240cda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:23.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.002+0000 7f5629b68640 1 -- 172.21.15.49:0/2715793692 >> 172.21.15.49:0/2715793692 conn(0x7f56240d2aa0 msgr2=0x7f56240d2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:23.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.002+0000 7f5629b68640 1 -- 172.21.15.49:0/2715793692 shutdown_connections 2023-11-06T02:11:23.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.002+0000 7f5629b68640 1 -- 172.21.15.49:0/2715793692 wait complete. 2023-11-06T02:11:23.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.002+0000 7f5629b68640 1 Processor -- start 2023-11-06T02:11:23.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.002+0000 7f5629b68640 1 -- start start 2023-11-06T02:11:23.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.002+0000 7f5629b68640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f56240b1f90 0x7f56240c8850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:23.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.002+0000 7f5629b68640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56240d2190 0x7f562414e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:23.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.002+0000 7f5629b68640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5624161650 0x7f562415be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:23.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.002+0000 7f5629b68640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56240c8d90 con 0x7f56240d2190 2023-11-06T02:11:23.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.002+0000 7f5629b68640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56240cce40 con 0x7f5624161650 2023-11-06T02:11:23.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.006+0000 7f5629b68640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56240cc9b0 con 0x7f56240b1f90 2023-11-06T02:11:23.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.006+0000 7f5622ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56240d2190 0x7f562414e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:23.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.006+0000 7f5622ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56240d2190 0x7f562414e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49466/0 (socket says 172.21.15.49:49466) 2023-11-06T02:11:23.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.006+0000 7f5622ffd640 1 -- 172.21.15.49:0/317157979 learned_addr learned my addr 172.21.15.49:0/317157979 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:23.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.006+0000 7f56237fe640 1 --2- 172.21.15.49:0/317157979 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f56240b1f90 0x7f56240c8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:23.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.006+0000 7f5623fff640 1 --2- 172.21.15.49:0/317157979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5624161650 0x7f562415be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:23.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.006+0000 7f5622ffd640 1 -- 172.21.15.49:0/317157979 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f56240b1f90 msgr2=0x7f56240c8850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:23.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.006+0000 7f5622ffd640 1 --2- 172.21.15.49:0/317157979 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f56240b1f90 0x7f56240c8850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:23.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.006+0000 7f5622ffd640 1 -- 172.21.15.49:0/317157979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5624161650 msgr2=0x7f562415be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:23.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.006+0000 7f5622ffd640 1 --2- 172.21.15.49:0/317157979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5624161650 0x7f562415be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:23.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.006+0000 7f5622ffd640 1 -- 172.21.15.49:0/317157979 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f561c009b30 con 0x7f56240d2190 2023-11-06T02:11:23.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.006+0000 7f5622ffd640 1 --2- 172.21.15.49:0/317157979 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56240d2190 0x7f562414e300 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f561800c7f0 tx=0x7f561800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:23.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.006+0000 7f560bfff640 1 -- 172.21.15.49:0/317157979 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5618007820 con 0x7f56240d2190 2023-11-06T02:11:23.009 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.006+0000 7f560bfff640 1 -- 172.21.15.49:0/317157979 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5618007980 con 0x7f56240d2190 2023-11-06T02:11:23.009 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.006+0000 7f5629b68640 1 -- 172.21.15.49:0/317157979 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56240cbda0 con 0x7f56240d2190 2023-11-06T02:11:23.009 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.006+0000 7f560bfff640 1 -- 172.21.15.49:0/317157979 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5618007d30 con 0x7f56240d2190 2023-11-06T02:11:23.009 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.006+0000 7f5629b68640 1 -- 172.21.15.49:0/317157979 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56240cc360 con 0x7f56240d2190 2023-11-06T02:11:23.009 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.006+0000 7f5629b68640 1 -- 172.21.15.49:0/317157979 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56240d1100 con 0x7f56240d2190 2023-11-06T02:11:23.010 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.006+0000 7f560bfff640 1 -- 172.21.15.49:0/317157979 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f56180113f0 con 0x7f56240d2190 2023-11-06T02:11:23.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.006+0000 7f560bfff640 1 --2- 172.21.15.49:0/317157979 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5610065a60 0x7f5610067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:23.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.006+0000 7f560bfff640 1 -- 172.21.15.49:0/317157979 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f5618083260 con 0x7f56240d2190 2023-11-06T02:11:23.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.006+0000 7f56237fe640 1 --2- 172.21.15.49:0/317157979 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5610065a60 0x7f5610067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:23.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.010+0000 7f56237fe640 1 --2- 172.21.15.49:0/317157979 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5610065a60 0x7f5610067f20 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f5614006b20 tx=0x7f5614008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:23.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.010+0000 7f560bfff640 1 -- 172.21.15.49:0/317157979 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f5618050e50 con 0x7f56240d2190 2023-11-06T02:11:23.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.230+0000 7f5629b68640 1 -- 172.21.15.49:0/317157979 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]} v 0) v1 -- 0x7f56240ccb20 con 0x7f56240d2190 2023-11-06T02:11:23.529 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.534+0000 7f560bfff640 1 -- 172.21.15.49:0/317157979 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]}]=0 v165) v1 ==== 149+0+0 (secure 0 0 0) 0x7f5618054470 con 0x7f56240d2190 2023-11-06T02:11:23.532 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.534+0000 7f5629b68640 1 -- 172.21.15.49:0/317157979 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5610065a60 msgr2=0x7f5610067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:23.533 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.534+0000 7f5629b68640 1 --2- 172.21.15.49:0/317157979 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5610065a60 0x7f5610067f20 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f5614006b20 tx=0x7f5614008040 comp rx=0 tx=0).stop 2023-11-06T02:11:23.533 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.534+0000 7f5629b68640 1 -- 172.21.15.49:0/317157979 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56240d2190 msgr2=0x7f562414e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:23.533 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.534+0000 7f5629b68640 1 --2- 172.21.15.49:0/317157979 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56240d2190 0x7f562414e300 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f561800c7f0 tx=0x7f561800ccb0 comp rx=0 tx=0).stop 2023-11-06T02:11:23.533 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.534+0000 7f5629b68640 1 -- 172.21.15.49:0/317157979 shutdown_connections 2023-11-06T02:11:23.533 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.534+0000 7f5629b68640 1 --2- 172.21.15.49:0/317157979 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5610065a60 0x7f5610067f20 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:23.534 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.534+0000 7f5629b68640 1 --2- 172.21.15.49:0/317157979 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56240d2190 0x7f562414e300 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:23.534 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.534+0000 7f5629b68640 1 --2- 172.21.15.49:0/317157979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5624161650 0x7f562415be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:23.534 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.534+0000 7f5629b68640 1 --2- 172.21.15.49:0/317157979 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f56240b1f90 0x7f56240c8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:23.534 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.534+0000 7f5629b68640 1 -- 172.21.15.49:0/317157979 >> 172.21.15.49:0/317157979 conn(0x7f56240d2aa0 msgr2=0x7f56240b7fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:23.534 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.534+0000 7f5629b68640 1 -- 172.21.15.49:0/317157979 shutdown_connections 2023-11-06T02:11:23.535 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.534+0000 7f5629b68640 1 -- 172.21.15.49:0/317157979 wait complete. 2023-11-06T02:11:23.555 INFO:tasks.cephfs_test_runner:That the MDS can repair an inodes backtrace in the data pool ... ok 2023-11-06T02:11:23.556 INFO:tasks.cephfs_test_runner:Starting test: test_inotable_sync (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2023-11-06T02:11:23.557 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync' 2023-11-06T02:11:23.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.686+0000 7f8e810bd640 1 Processor -- start 2023-11-06T02:11:23.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.686+0000 7f8e810bd640 1 -- start start 2023-11-06T02:11:23.686 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.690+0000 7f8e810bd640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f8e7c159980 0x7f8e7c071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:23.686 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.690+0000 7f8e810bd640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f8e7c0b1450 0x7f8e7c0cdd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:23.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.690+0000 7f8e810bd640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e7c071890 con 0x7f8e7c0cb010 2023-11-06T02:11:23.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.690+0000 7f8e810bd640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e7c072410 con 0x7f8e7c159980 2023-11-06T02:11:23.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.690+0000 7f8e810bd640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e7c0b1890 con 0x7f8e7c0b1450 2023-11-06T02:11:23.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.690+0000 7f8e7ad9d640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f8e7c159980 0x7f8e7c071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:23.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.690+0000 7f8e7ad9d640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f8e7c159980 0x7f8e7c071290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49478/0 (socket says 172.21.15.49:49478) 2023-11-06T02:11:23.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.690+0000 7f8e7b59e640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f8e7c0cb010 0x7f8e7c0cc540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:45970/0 (socket says 172.21.15.49:45970) 2023-11-06T02:11:23.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.690+0000 7f8e7ad9d640 1 -- 172.21.15.49:0/2724844441 learned_addr learned my addr 172.21.15.49:0/2724844441 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:23.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.690+0000 7f8e7a59c640 1 --2- 172.21.15.49:0/2724844441 >> v2:172.21.15.150:3300/0 conn(0x7f8e7c0b1450 0x7f8e7c0cdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:23.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.690+0000 7f8e7ad9d640 1 -- 172.21.15.49:0/2724844441 >> v2:172.21.15.150:3300/0 conn(0x7f8e7c0b1450 msgr2=0x7f8e7c0cdd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:23.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.690+0000 7f8e7ad9d640 1 --2- 172.21.15.49:0/2724844441 >> v2:172.21.15.150:3300/0 conn(0x7f8e7c0b1450 0x7f8e7c0cdd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:23.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.690+0000 7f8e7ad9d640 1 -- 172.21.15.49:0/2724844441 >> v1:172.21.15.49:6789/0 conn(0x7f8e7c0cb010 legacy=0x7f8e7c0cc540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:23.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.690+0000 7f8e7ad9d640 1 -- 172.21.15.49:0/2724844441 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e7c071b70 con 0x7f8e7c159980 2023-11-06T02:11:23.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.690+0000 7f8e79d9b640 1 -- 172.21.15.49:0/2724844441 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 864664569 0 0) 0x7f8e7c071890 con 0x7f8e7c0cb010 2023-11-06T02:11:23.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.690+0000 7f8e7a59c640 1 --2- 172.21.15.49:0/2724844441 >> v2:172.21.15.150:3300/0 conn(0x7f8e7c0b1450 0x7f8e7c0cdd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:11:23.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.690+0000 7f8e7ad9d640 1 --2- 172.21.15.49:0/2724844441 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e7c159980 0x7f8e7c071290 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c15d2e0 tx=0x7f8e6c01c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16fc2342aaedd399 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:23.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.690+0000 7f8e79d9b640 1 -- 172.21.15.49:0/2724844441 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e6c0ab670 con 0x7f8e7c159980 2023-11-06T02:11:23.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.690+0000 7f8e79d9b640 1 -- 172.21.15.49:0/2724844441 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e6c0ab7d0 con 0x7f8e7c159980 2023-11-06T02:11:23.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.690+0000 7f8e79d9b640 1 -- 172.21.15.49:0/2724844441 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e6c0abb60 con 0x7f8e7c159980 2023-11-06T02:11:23.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.690+0000 7f8e810bd640 1 -- 172.21.15.49:0/2724844441 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e7c159980 msgr2=0x7f8e7c071290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:23.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.690+0000 7f8e810bd640 1 --2- 172.21.15.49:0/2724844441 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e7c159980 0x7f8e7c071290 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c15d2e0 tx=0x7f8e6c01c500 comp rx=0 tx=0).stop 2023-11-06T02:11:23.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.690+0000 7f8e810bd640 1 -- 172.21.15.49:0/2724844441 shutdown_connections 2023-11-06T02:11:23.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.690+0000 7f8e810bd640 1 --2- 172.21.15.49:0/2724844441 >> v2:172.21.15.150:3300/0 conn(0x7f8e7c0b1450 0x7f8e7c0cdd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:23.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.690+0000 7f8e810bd640 1 --2- 172.21.15.49:0/2724844441 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e7c159980 0x7f8e7c071290 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:23.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.690+0000 7f8e810bd640 1 -- 172.21.15.49:0/2724844441 >> 172.21.15.49:0/2724844441 conn(0x7f8e7c06c0a0 msgr2=0x7f8e7c06c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:23.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.694+0000 7f8e810bd640 1 -- 172.21.15.49:0/2724844441 shutdown_connections 2023-11-06T02:11:23.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.694+0000 7f8e810bd640 1 -- 172.21.15.49:0/2724844441 wait complete. 2023-11-06T02:11:23.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.694+0000 7f8e810bd640 1 Processor -- start 2023-11-06T02:11:23.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.694+0000 7f8e810bd640 1 -- start start 2023-11-06T02:11:23.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.694+0000 7f8e810bd640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8e7c0b1450 0x7f8e7c0ca450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:23.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.694+0000 7f8e810bd640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e7c0cb010 0x7f8e7c148a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:23.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.694+0000 7f8e810bd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e7c159980 0x7f8e7c0c42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:23.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.694+0000 7f8e810bd640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e7c0b0120 con 0x7f8e7c159980 2023-11-06T02:11:23.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.694+0000 7f8e810bd640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e7c15cac0 con 0x7f8e7c0cb010 2023-11-06T02:11:23.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.694+0000 7f8e810bd640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e7c0af7f0 con 0x7f8e7c0b1450 2023-11-06T02:11:23.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.694+0000 7f8e7b59e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e7c159980 0x7f8e7c0c42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:23.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.694+0000 7f8e7b59e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e7c159980 0x7f8e7c0c42e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49492/0 (socket says 172.21.15.49:49492) 2023-11-06T02:11:23.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.694+0000 7f8e7b59e640 1 -- 172.21.15.49:0/2613695886 learned_addr learned my addr 172.21.15.49:0/2613695886 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:23.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.694+0000 7f8e7a59c640 1 --2- 172.21.15.49:0/2613695886 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e7c0cb010 0x7f8e7c148a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:23.701 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.694+0000 7f8e7ad9d640 1 --2- 172.21.15.49:0/2613695886 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8e7c0b1450 0x7f8e7c0ca450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:23.701 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.694+0000 7f8e7b59e640 1 -- 172.21.15.49:0/2613695886 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8e7c0b1450 msgr2=0x7f8e7c0ca450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:23.701 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.694+0000 7f8e7b59e640 1 --2- 172.21.15.49:0/2613695886 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8e7c0b1450 0x7f8e7c0ca450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:23.701 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.694+0000 7f8e7b59e640 1 -- 172.21.15.49:0/2613695886 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e7c0cb010 msgr2=0x7f8e7c148a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:23.702 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.694+0000 7f8e7b59e640 1 --2- 172.21.15.49:0/2613695886 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e7c0cb010 0x7f8e7c148a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:23.702 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.694+0000 7f8e7b59e640 1 -- 172.21.15.49:0/2613695886 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e6c009480 con 0x7f8e7c159980 2023-11-06T02:11:23.702 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.694+0000 7f8e7b59e640 1 --2- 172.21.15.49:0/2613695886 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e7c159980 0x7f8e7c0c42e0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f8e74001570 tx=0x7f8e7400cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:23.702 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.694+0000 7f8e637fe640 1 -- 172.21.15.49:0/2613695886 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7400a640 con 0x7f8e7c159980 2023-11-06T02:11:23.703 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.694+0000 7f8e637fe640 1 -- 172.21.15.49:0/2613695886 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e7400a7c0 con 0x7f8e7c159980 2023-11-06T02:11:23.703 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.694+0000 7f8e810bd640 1 -- 172.21.15.49:0/2613695886 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e7c0ce3f0 con 0x7f8e7c159980 2023-11-06T02:11:23.703 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.694+0000 7f8e637fe640 1 -- 172.21.15.49:0/2613695886 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7400ab70 con 0x7f8e7c159980 2023-11-06T02:11:23.703 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.694+0000 7f8e810bd640 1 -- 172.21.15.49:0/2613695886 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e7c0afdb0 con 0x7f8e7c159980 2023-11-06T02:11:23.704 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.694+0000 7f8e810bd640 1 -- 172.21.15.49:0/2613695886 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e7c0b02d0 con 0x7f8e7c159980 2023-11-06T02:11:23.704 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.698+0000 7f8e637fe640 1 -- 172.21.15.49:0/2613695886 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f8e7400acd0 con 0x7f8e7c159980 2023-11-06T02:11:23.704 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.698+0000 7f8e637fe640 1 --2- 172.21.15.49:0/2613695886 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8e68065a90 0x7f8e68067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:23.704 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.698+0000 7f8e637fe640 1 -- 172.21.15.49:0/2613695886 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f8e740826b0 con 0x7f8e7c159980 2023-11-06T02:11:23.704 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.698+0000 7f8e7ad9d640 1 --2- 172.21.15.49:0/2613695886 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8e68065a90 0x7f8e68067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:23.705 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.698+0000 7f8e7ad9d640 1 --2- 172.21.15.49:0/2613695886 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8e68065a90 0x7f8e68067f50 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c0af000 tx=0x7f8e6c0a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:23.705 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.702+0000 7f8e637fe640 1 -- 172.21.15.49:0/2613695886 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f8e74050220 con 0x7f8e7c159980 2023-11-06T02:11:23.911 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:23.914+0000 7f8e810bd640 1 -- 172.21.15.49:0/2613695886 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]} v 0) v1 -- 0x7f8e7c074040 con 0x7f8e7c159980 2023-11-06T02:11:24.532 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:24.533+0000 7f8e637fe640 1 -- 172.21.15.49:0/2613695886 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]}]=0 v166) v1 ==== 149+0+0 (secure 0 0 0) 0x7f8e74018020 con 0x7f8e7c159980 2023-11-06T02:11:24.535 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:24.537+0000 7f8e810bd640 1 -- 172.21.15.49:0/2613695886 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8e68065a90 msgr2=0x7f8e68067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:24.536 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:24.537+0000 7f8e810bd640 1 --2- 172.21.15.49:0/2613695886 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8e68065a90 0x7f8e68067f50 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c0af000 tx=0x7f8e6c0a7000 comp rx=0 tx=0).stop 2023-11-06T02:11:24.536 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:24.537+0000 7f8e810bd640 1 -- 172.21.15.49:0/2613695886 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e7c159980 msgr2=0x7f8e7c0c42e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:24.537 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:24.537+0000 7f8e810bd640 1 --2- 172.21.15.49:0/2613695886 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e7c159980 0x7f8e7c0c42e0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f8e74001570 tx=0x7f8e7400cca0 comp rx=0 tx=0).stop 2023-11-06T02:11:24.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:24.537+0000 7f8e810bd640 1 -- 172.21.15.49:0/2613695886 shutdown_connections 2023-11-06T02:11:24.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:24.537+0000 7f8e810bd640 1 --2- 172.21.15.49:0/2613695886 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8e68065a90 0x7f8e68067f50 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:24.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:24.537+0000 7f8e810bd640 1 --2- 172.21.15.49:0/2613695886 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e7c159980 0x7f8e7c0c42e0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:24.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:24.537+0000 7f8e810bd640 1 --2- 172.21.15.49:0/2613695886 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e7c0cb010 0x7f8e7c148a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:24.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:24.537+0000 7f8e810bd640 1 --2- 172.21.15.49:0/2613695886 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8e7c0b1450 0x7f8e7c0ca450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:24.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:24.537+0000 7f8e810bd640 1 -- 172.21.15.49:0/2613695886 >> 172.21.15.49:0/2613695886 conn(0x7f8e7c06c0a0 msgr2=0x7f8e7c14c980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:24.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:24.537+0000 7f8e810bd640 1 -- 172.21.15.49:0/2613695886 shutdown_connections 2023-11-06T02:11:24.542 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:24.537+0000 7f8e810bd640 1 -- 172.21.15.49:0/2613695886 wait complete. 2023-11-06T02:11:24.556 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-06T02:11:24.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:24.701+0000 7f8f88a63640 1 Processor -- start 2023-11-06T02:11:24.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:24.701+0000 7f8f88a63640 1 -- start start 2023-11-06T02:11:24.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:24.701+0000 7f8f88a63640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f8f840cdea0 0x7f8f840cd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:24.701 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:24.701+0000 7f8f88a63640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f8f840ce280 0x7f8f8415d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:24.701 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:24.701+0000 7f8f88a63640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f84099080 con 0x7f8f840ae760 2023-11-06T02:11:24.701 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:24.701+0000 7f8f88a63640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f840b1100 con 0x7f8f840cdea0 2023-11-06T02:11:24.701 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:24.701+0000 7f8f88a63640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f840b0b40 con 0x7f8f840ce280 2023-11-06T02:11:24.702 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:24.701+0000 7f8f8259c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f8f840cdea0 0x7f8f840cd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:24.702 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:24.701+0000 7f8f82d9d640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f8f840ae760 0x7f8f8415df00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:45982/0 (socket says 172.21.15.49:45982) 2023-11-06T02:11:24.702 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:24.701+0000 7f8f82d9d640 1 -- 172.21.15.49:0/3820064396 learned_addr learned my addr 172.21.15.49:0/3820064396 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:11:24.702 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:24.701+0000 7f8f81d9b640 1 --2- 172.21.15.49:0/3820064396 >> v2:172.21.15.150:3300/0 conn(0x7f8f840ce280 0x7f8f8415d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:24.703 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:24.701+0000 7f8f8159a640 1 -- 172.21.15.49:0/3820064396 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3316940826 0 0) 0x7f8f84099080 con 0x7f8f840ae760 2023-11-06T02:11:24.703 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:24.701+0000 7f8f8159a640 1 -- 172.21.15.49:0/3820064396 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f70003530 con 0x7f8f840ae760 2023-11-06T02:11:24.703 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:24.701+0000 7f8f8259c640 1 -- 172.21.15.49:0/3820064396 >> v2:172.21.15.150:3300/0 conn(0x7f8f840ce280 msgr2=0x7f8f8415d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:24.703 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:24.701+0000 7f8f8259c640 1 --2- 172.21.15.49:0/3820064396 >> v2:172.21.15.150:3300/0 conn(0x7f8f840ce280 0x7f8f8415d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:24.704 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:24.701+0000 7f8f8259c640 1 -- 172.21.15.49:0/3820064396 >> v1:172.21.15.49:6789/0 conn(0x7f8f840ae760 legacy=0x7f8f8415df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:24.704 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:24.701+0000 7f8f8259c640 1 -- 172.21.15.49:0/3820064396 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f840b0e20 con 0x7f8f840cdea0 2023-11-06T02:11:24.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:24.705+0000 7f8f8259c640 1 --2- 172.21.15.49:0/3820064396 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f840cdea0 0x7f8f840cd430 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f8f840d0960 tx=0x7f8f7c01c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58dfb6592afbe422 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:24.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:24.705+0000 7f8f8159a640 1 -- 172.21.15.49:0/3820064396 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c0a58e0 con 0x7f8f840cdea0 2023-11-06T02:11:24.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:24.705+0000 7f8f8159a640 1 -- 172.21.15.49:0/3820064396 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f7c0a5a40 con 0x7f8f840cdea0 2023-11-06T02:11:24.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:24.705+0000 7f8f8159a640 1 -- 172.21.15.49:0/3820064396 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c0a5dd0 con 0x7f8f840cdea0 2023-11-06T02:11:24.709 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:24.705+0000 7f8f88a63640 1 -- 172.21.15.49:0/3820064396 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f840cdea0 msgr2=0x7f8f840cd430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:24.709 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:24.705+0000 7f8f88a63640 1 --2- 172.21.15.49:0/3820064396 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f840cdea0 0x7f8f840cd430 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f8f840d0960 tx=0x7f8f7c01c910 comp rx=0 tx=0).stop 2023-11-06T02:11:24.709 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:24.705+0000 7f8f88a63640 1 -- 172.21.15.49:0/3820064396 shutdown_connections 2023-11-06T02:11:24.709 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:24.705+0000 7f8f88a63640 1 --2- 172.21.15.49:0/3820064396 >> v2:172.21.15.150:3300/0 conn(0x7f8f840ce280 0x7f8f8415d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:24.709 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:24.705+0000 7f8f88a63640 1 --2- 172.21.15.49:0/3820064396 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f840cdea0 0x7f8f840cd430 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:24.710 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:24.705+0000 7f8f88a63640 1 -- 172.21.15.49:0/3820064396 >> 172.21.15.49:0/3820064396 conn(0x7f8f84098a40 msgr2=0x7f8f840cf860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:24.710 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:24.705+0000 7f8f88a63640 1 -- 172.21.15.49:0/3820064396 shutdown_connections 2023-11-06T02:11:24.710 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:24.705+0000 7f8f88a63640 1 -- 172.21.15.49:0/3820064396 wait complete. 2023-11-06T02:11:24.710 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:24.705+0000 7f8f88a63640 1 Processor -- start 2023-11-06T02:11:24.711 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:24.709+0000 7f8f88a63640 1 -- start start 2023-11-06T02:11:24.711 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:24.709+0000 7f8f88a63640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8f840ae760 0x7f8f84154660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:24.711 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:24.709+0000 7f8f88a63640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f840cdea0 0x7f8f8414ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:24.711 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:24.709+0000 7f8f88a63640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f840ce280 0x7f8f841518b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:24.712 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:24.709+0000 7f8f88a63640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f84154cf0 con 0x7f8f840cdea0 2023-11-06T02:11:24.712 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:24.709+0000 7f8f88a63640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f840c6200 con 0x7f8f840ce280 2023-11-06T02:11:24.712 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:24.709+0000 7f8f88a63640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f841504f0 con 0x7f8f840ae760 2023-11-06T02:11:24.712 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:24.709+0000 7f8f81d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f840cdea0 0x7f8f8414ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:24.713 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:24.709+0000 7f8f81d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f840cdea0 0x7f8f8414ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49502/0 (socket says 172.21.15.49:49502) 2023-11-06T02:11:24.713 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:24.709+0000 7f8f81d9b640 1 -- 172.21.15.49:0/979505403 learned_addr learned my addr 172.21.15.49:0/979505403 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:24.713 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:24.709+0000 7f8f82d9d640 1 --2- 172.21.15.49:0/979505403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f840ce280 0x7f8f841518b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:24.713 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:24.709+0000 7f8f8259c640 1 --2- 172.21.15.49:0/979505403 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8f840ae760 0x7f8f84154660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:24.713 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:24.709+0000 7f8f81d9b640 1 -- 172.21.15.49:0/979505403 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8f840ae760 msgr2=0x7f8f84154660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:24.714 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:24.709+0000 7f8f81d9b640 1 --2- 172.21.15.49:0/979505403 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8f840ae760 0x7f8f84154660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:24.714 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:24.709+0000 7f8f81d9b640 1 -- 172.21.15.49:0/979505403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f840ce280 msgr2=0x7f8f841518b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:24.714 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:24.709+0000 7f8f81d9b640 1 --2- 172.21.15.49:0/979505403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f840ce280 0x7f8f841518b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:24.714 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:24.709+0000 7f8f81d9b640 1 -- 172.21.15.49:0/979505403 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f7c009a70 con 0x7f8f840cdea0 2023-11-06T02:11:24.714 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:24.709+0000 7f8f81d9b640 1 --2- 172.21.15.49:0/979505403 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f840cdea0 0x7f8f8414ffb0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c00d620 tx=0x7f8f6c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:24.715 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:24.709+0000 7f8f7affd640 1 -- 172.21.15.49:0/979505403 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f6c00b620 con 0x7f8f840cdea0 2023-11-06T02:11:24.715 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:24.709+0000 7f8f7affd640 1 -- 172.21.15.49:0/979505403 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f6c00b780 con 0x7f8f840cdea0 2023-11-06T02:11:24.715 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:24.709+0000 7f8f88a63640 1 -- 172.21.15.49:0/979505403 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f84153660 con 0x7f8f840cdea0 2023-11-06T02:11:24.715 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:24.709+0000 7f8f7affd640 1 -- 172.21.15.49:0/979505403 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f6c00bb10 con 0x7f8f840cdea0 2023-11-06T02:11:24.715 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:24.709+0000 7f8f88a63640 1 -- 172.21.15.49:0/979505403 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f84153c20 con 0x7f8f840cdea0 2023-11-06T02:11:24.716 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:24.709+0000 7f8f88a63640 1 -- 172.21.15.49:0/979505403 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f8415d670 con 0x7f8f840cdea0 2023-11-06T02:11:24.716 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:24.709+0000 7f8f7affd640 1 -- 172.21.15.49:0/979505403 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f8f6c00bc70 con 0x7f8f840cdea0 2023-11-06T02:11:24.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:24.709+0000 7f8f7affd640 1 --2- 172.21.15.49:0/979505403 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8f64065a60 0x7f8f64067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:24.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:24.709+0000 7f8f7affd640 1 -- 172.21.15.49:0/979505403 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f8f6c0834b0 con 0x7f8f840cdea0 2023-11-06T02:11:24.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:24.713+0000 7f8f8259c640 1 --2- 172.21.15.49:0/979505403 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8f64065a60 0x7f8f64067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:24.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:24.713+0000 7f8f8259c640 1 --2- 172.21.15.49:0/979505403 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8f64065a60 0x7f8f64067f20 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c0af000 tx=0x7f8f7c0a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:24.718 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:24.717+0000 7f8f7affd640 1 -- 172.21.15.49:0/979505403 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f8f6c051010 con 0x7f8f840cdea0 2023-11-06T02:11:24.873 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:24.869+0000 7f8f88a63640 1 -- 172.21.15.49:0/979505403 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f8f84065d00 con 0x7f8f840cdea0 2023-11-06T02:11:24.877 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:24.881+0000 7f8f7affd640 1 -- 172.21.15.49:0/979505403 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7f8f6c007380 con 0x7f8f840cdea0 2023-11-06T02:11:24.881 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:24.881+0000 7f8f88a63640 1 -- 172.21.15.49:0/979505403 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8f64065a60 msgr2=0x7f8f64067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:24.881 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:24.881+0000 7f8f88a63640 1 --2- 172.21.15.49:0/979505403 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8f64065a60 0x7f8f64067f20 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c0af000 tx=0x7f8f7c0a7070 comp rx=0 tx=0).stop 2023-11-06T02:11:24.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:24.881+0000 7f8f88a63640 1 -- 172.21.15.49:0/979505403 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f840cdea0 msgr2=0x7f8f8414ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:24.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:24.881+0000 7f8f88a63640 1 --2- 172.21.15.49:0/979505403 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f840cdea0 0x7f8f8414ffb0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c00d620 tx=0x7f8f6c00dae0 comp rx=0 tx=0).stop 2023-11-06T02:11:24.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:24.881+0000 7f8f88a63640 1 -- 172.21.15.49:0/979505403 shutdown_connections 2023-11-06T02:11:24.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:24.881+0000 7f8f88a63640 1 --2- 172.21.15.49:0/979505403 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8f64065a60 0x7f8f64067f20 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:24.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:24.881+0000 7f8f88a63640 1 --2- 172.21.15.49:0/979505403 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f840cdea0 0x7f8f8414ffb0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:24.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:24.881+0000 7f8f88a63640 1 --2- 172.21.15.49:0/979505403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f840ce280 0x7f8f841518b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:24.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:24.881+0000 7f8f88a63640 1 --2- 172.21.15.49:0/979505403 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8f840ae760 0x7f8f84154660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:24.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:24.881+0000 7f8f88a63640 1 -- 172.21.15.49:0/979505403 >> 172.21.15.49:0/979505403 conn(0x7f8f84098a40 msgr2=0x7f8f8406ecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:24.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:24.881+0000 7f8f88a63640 1 -- 172.21.15.49:0/979505403 shutdown_connections 2023-11-06T02:11:24.885 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:24.881+0000 7f8f88a63640 1 -- 172.21.15.49:0/979505403 wait complete. 2023-11-06T02:11:24.897 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-06T02:11:24.898 DEBUG:teuthology.orchestra.run.smithi049:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T02:11:24.938 DEBUG:teuthology.orchestra.run.smithi150:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T02:11:24.990 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-06T02:11:24.991 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-11-06T02:11:24.991 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T02:11:24.999 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336476k,nr_inodes=4084119,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16105 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-06T02:11:24.999 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-11-06T02:11:25.000 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T02:11:25.007 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336552k,nr_inodes=4084138,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19862 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274692k,nr_inodes=818673,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-06T02:11:25.008 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-11-06T02:11:25.008 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T02:11:25.047 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336476k,nr_inodes=4084119,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16105 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-06T02:11:25.047 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-11-06T02:11:25.048 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T02:11:25.059 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336552k,nr_inodes=4084138,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19862 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274692k,nr_inodes=818673,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-06T02:11:25.059 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-06T02:11:25.059 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T02:11:25.216 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.217+0000 7ff6755ed640 1 Processor -- start 2023-11-06T02:11:25.217 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.217+0000 7ff6755ed640 1 -- start start 2023-11-06T02:11:25.218 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.221+0000 7ff6755ed640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7ff67015ab50 0x7ff6700c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:25.219 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.221+0000 7ff6755ed640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff6700cb010 0x7ff6700b0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:25.219 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.221+0000 7ff6755ed640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6700cab80 con 0x7ff6700b05c0 2023-11-06T02:11:25.219 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.221+0000 7ff6755ed640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6700b0a10 con 0x7ff6700cb010 2023-11-06T02:11:25.220 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.221+0000 7ff6755ed640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff67015af30 con 0x7ff67015ab50 2023-11-06T02:11:25.220 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.221+0000 7ff66e7fc640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff6700cb010 0x7ff6700b0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:25.220 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.221+0000 7ff66e7fc640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff6700cb010 0x7ff6700b0080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49508/0 (socket says 172.21.15.49:49508) 2023-11-06T02:11:25.220 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.221+0000 7ff66e7fc640 1 -- 172.21.15.49:0/1014137757 learned_addr learned my addr 172.21.15.49:0/1014137757 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:25.220 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.221+0000 7ff66effd640 1 --2- 172.21.15.49:0/1014137757 >> v2:172.21.15.150:3300/0 conn(0x7ff67015ab50 0x7ff6700c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:25.221 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.221+0000 7ff66e7fc640 1 -- 172.21.15.49:0/1014137757 >> v2:172.21.15.150:3300/0 conn(0x7ff67015ab50 msgr2=0x7ff6700c9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:25.221 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.221+0000 7ff66e7fc640 1 --2- 172.21.15.49:0/1014137757 >> v2:172.21.15.150:3300/0 conn(0x7ff67015ab50 0x7ff6700c9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:25.221 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.221+0000 7ff66e7fc640 1 -- 172.21.15.49:0/1014137757 >> v1:172.21.15.150:6789/0 conn(0x7ff6700b05c0 legacy=0x7ff6700700a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:25.221 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.221+0000 7ff66e7fc640 1 -- 172.21.15.49:0/1014137757 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6700cb5b0 con 0x7ff6700cb010 2023-11-06T02:11:25.222 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.221+0000 7ff66effd640 1 --2- 172.21.15.49:0/1014137757 >> v2:172.21.15.150:3300/0 conn(0x7ff67015ab50 0x7ff6700c9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:11:25.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.221+0000 7ff66e7fc640 1 --2- 172.21.15.49:0/1014137757 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6700cb010 0x7ff6700b0080 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7ff6700725a0 tx=0x7ff66401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6826b16567f7eaeb server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:25.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.221+0000 7ff66dffb640 1 -- 172.21.15.49:0/1014137757 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6640aa5f0 con 0x7ff6700cb010 2023-11-06T02:11:25.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.221+0000 7ff66dffb640 1 -- 172.21.15.49:0/1014137757 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6640aa750 con 0x7ff6700cb010 2023-11-06T02:11:25.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.221+0000 7ff66dffb640 1 -- 172.21.15.49:0/1014137757 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6640aaae0 con 0x7ff6700cb010 2023-11-06T02:11:25.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.225+0000 7ff6755ed640 1 -- 172.21.15.49:0/1014137757 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6700cb010 msgr2=0x7ff6700b0080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:25.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.225+0000 7ff6755ed640 1 --2- 172.21.15.49:0/1014137757 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6700cb010 0x7ff6700b0080 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7ff6700725a0 tx=0x7ff66401c490 comp rx=0 tx=0).stop 2023-11-06T02:11:25.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.225+0000 7ff6755ed640 1 -- 172.21.15.49:0/1014137757 shutdown_connections 2023-11-06T02:11:25.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.225+0000 7ff6755ed640 1 --2- 172.21.15.49:0/1014137757 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6700cb010 0x7ff6700b0080 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:25.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.225+0000 7ff6755ed640 1 --2- 172.21.15.49:0/1014137757 >> v2:172.21.15.150:3300/0 conn(0x7ff67015ab50 0x7ff6700c9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:25.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.225+0000 7ff6755ed640 1 -- 172.21.15.49:0/1014137757 >> 172.21.15.49:0/1014137757 conn(0x7ff67006c0a0 msgr2=0x7ff67006c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:25.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.225+0000 7ff6755ed640 1 -- 172.21.15.49:0/1014137757 shutdown_connections 2023-11-06T02:11:25.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.225+0000 7ff6755ed640 1 -- 172.21.15.49:0/1014137757 wait complete. 2023-11-06T02:11:25.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.225+0000 7ff6755ed640 1 Processor -- start 2023-11-06T02:11:25.229 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.225+0000 7ff6755ed640 1 -- start start 2023-11-06T02:11:25.229 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.225+0000 7ff6755ed640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff6700b05c0 0x7ff6700c8360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:25.229 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.225+0000 7ff6755ed640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6700cb010 0x7ff670157890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:25.229 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.225+0000 7ff6755ed640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff67015ab50 0x7ff6700c5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:25.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.225+0000 7ff6755ed640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff670158ef0 con 0x7ff6700cb010 2023-11-06T02:11:25.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.225+0000 7ff6755ed640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6701582e0 con 0x7ff67015ab50 2023-11-06T02:11:25.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.225+0000 7ff6755ed640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6701585c0 con 0x7ff6700b05c0 2023-11-06T02:11:25.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.225+0000 7ff66e7fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6700cb010 0x7ff670157890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:25.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.225+0000 7ff66e7fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6700cb010 0x7ff670157890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49520/0 (socket says 172.21.15.49:49520) 2023-11-06T02:11:25.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.225+0000 7ff66e7fc640 1 -- 172.21.15.49:0/2923406748 learned_addr learned my addr 172.21.15.49:0/2923406748 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:25.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.225+0000 7ff66f7fe640 1 --2- 172.21.15.49:0/2923406748 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff67015ab50 0x7ff6700c5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:25.232 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.225+0000 7ff66effd640 1 --2- 172.21.15.49:0/2923406748 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff6700b05c0 0x7ff6700c8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:25.232 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.225+0000 7ff66e7fc640 1 -- 172.21.15.49:0/2923406748 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff6700b05c0 msgr2=0x7ff6700c8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:25.232 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.225+0000 7ff66e7fc640 1 --2- 172.21.15.49:0/2923406748 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff6700b05c0 0x7ff6700c8360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:25.232 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.225+0000 7ff66e7fc640 1 -- 172.21.15.49:0/2923406748 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff67015ab50 msgr2=0x7ff6700c5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:25.232 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.225+0000 7ff66e7fc640 1 --2- 172.21.15.49:0/2923406748 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff67015ab50 0x7ff6700c5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:25.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.225+0000 7ff66e7fc640 1 -- 172.21.15.49:0/2923406748 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6640094d0 con 0x7ff6700cb010 2023-11-06T02:11:25.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.225+0000 7ff66effd640 1 --2- 172.21.15.49:0/2923406748 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff6700b05c0 0x7ff6700c8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:11:25.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.225+0000 7ff66e7fc640 1 --2- 172.21.15.49:0/2923406748 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6700cb010 0x7ff670157890 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7ff66401a0c0 tx=0x7ff66401acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:25.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.225+0000 7ff6577fe640 1 -- 172.21.15.49:0/2923406748 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6640b3800 con 0x7ff6700cb010 2023-11-06T02:11:25.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.225+0000 7ff6577fe640 1 -- 172.21.15.49:0/2923406748 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6640b3980 con 0x7ff6700cb010 2023-11-06T02:11:25.234 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.225+0000 7ff6755ed640 1 -- 172.21.15.49:0/2923406748 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6701588a0 con 0x7ff6700cb010 2023-11-06T02:11:25.234 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.225+0000 7ff6577fe640 1 -- 172.21.15.49:0/2923406748 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6640b3d30 con 0x7ff6700cb010 2023-11-06T02:11:25.234 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.225+0000 7ff6755ed640 1 -- 172.21.15.49:0/2923406748 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6700c88a0 con 0x7ff6700cb010 2023-11-06T02:11:25.234 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.229+0000 7ff6755ed640 1 -- 172.21.15.49:0/2923406748 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6700b0de0 con 0x7ff6700cb010 2023-11-06T02:11:25.234 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.229+0000 7ff6577fe640 1 -- 172.21.15.49:0/2923406748 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7ff6640c7420 con 0x7ff6700cb010 2023-11-06T02:11:25.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.229+0000 7ff6577fe640 1 --2- 172.21.15.49:0/2923406748 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff65c065a60 0x7ff65c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:25.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.229+0000 7ff6577fe640 1 -- 172.21.15.49:0/2923406748 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5296+0+0 (secure 0 0 0) 0x7ff6640a7070 con 0x7ff6700cb010 2023-11-06T02:11:25.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.229+0000 7ff66effd640 1 --2- 172.21.15.49:0/2923406748 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff65c065a60 0x7ff65c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:25.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.229+0000 7ff66effd640 1 --2- 172.21.15.49:0/2923406748 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff65c065a60 0x7ff65c067f20 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ff660006fd0 tx=0x7ff660008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:25.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.233+0000 7ff6577fe640 1 -- 172.21.15.49:0/2923406748 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7ff6640eb520 con 0x7ff6700cb010 2023-11-06T02:11:25.422 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.425+0000 7ff6755ed640 1 -- 172.21.15.49:0/2923406748 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff670074040 con 0x7ff6700cb010 2023-11-06T02:11:25.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.425+0000 7ff6577fe640 1 -- 172.21.15.49:0/2923406748 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+3535 (secure 0 0 0) 0x7ff6640eeb40 con 0x7ff6700cb010 2023-11-06T02:11:25.423 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:11:25.423 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":25,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5096,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1396196190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1396196190},{"type":"v1","addr":"172.21.15.150:6835","nonce":1396196190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5102,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/2465209170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":2465209170},{"type":"v1","addr":"172.21.15.150:6837","nonce":2465209170}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5871,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/1006951804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1006951804},{"type":"v1","addr":"172.21.15.49:6833","nonce":1006951804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":6276,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/2198944560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2198944560},{"type":"v1","addr":"172.21.15.49:6835","nonce":2198944560}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[]} 2023-11-06T02:11:25.424 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 25 2023-11-06T02:11:25.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.429+0000 7ff6755ed640 1 -- 172.21.15.49:0/2923406748 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff65c065a60 msgr2=0x7ff65c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:25.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.429+0000 7ff6755ed640 1 --2- 172.21.15.49:0/2923406748 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff65c065a60 0x7ff65c067f20 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ff660006fd0 tx=0x7ff660008040 comp rx=0 tx=0).stop 2023-11-06T02:11:25.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.429+0000 7ff6755ed640 1 -- 172.21.15.49:0/2923406748 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6700cb010 msgr2=0x7ff670157890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:25.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.429+0000 7ff6755ed640 1 --2- 172.21.15.49:0/2923406748 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6700cb010 0x7ff670157890 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7ff66401a0c0 tx=0x7ff66401acd0 comp rx=0 tx=0).stop 2023-11-06T02:11:25.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.429+0000 7ff6755ed640 1 -- 172.21.15.49:0/2923406748 shutdown_connections 2023-11-06T02:11:25.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.429+0000 7ff6755ed640 1 --2- 172.21.15.49:0/2923406748 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff65c065a60 0x7ff65c067f20 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:25.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.429+0000 7ff6755ed640 1 --2- 172.21.15.49:0/2923406748 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6700cb010 0x7ff670157890 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:25.427 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.429+0000 7ff6755ed640 1 --2- 172.21.15.49:0/2923406748 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff67015ab50 0x7ff6700c5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:25.427 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.429+0000 7ff6755ed640 1 --2- 172.21.15.49:0/2923406748 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff6700b05c0 0x7ff6700c8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:25.427 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.429+0000 7ff6755ed640 1 -- 172.21.15.49:0/2923406748 >> 172.21.15.49:0/2923406748 conn(0x7ff67006c0a0 msgr2=0x7ff670070080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:25.427 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.429+0000 7ff6755ed640 1 -- 172.21.15.49:0/2923406748 shutdown_connections 2023-11-06T02:11:25.427 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.429+0000 7ff6755ed640 1 -- 172.21.15.49:0/2923406748 wait complete. 2023-11-06T02:11:25.443 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-06T02:11:25.443 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-06T02:11:25.444 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-06T02:11:25.444 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T02:11:25.444 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-06T02:11:25.444 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-06T02:11:25.445 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-06T02:11:25.445 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T02:11:25.445 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-06T02:11:25.446 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-06T02:11:25.446 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-06T02:11:25.446 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T02:11:25.446 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-06T02:11:25.447 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-06T02:11:25.447 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-06T02:11:25.447 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T02:11:25.454 INFO:tasks.ceph.mds.a:Stopped 2023-11-06T02:11:25.455 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-06T02:11:25.457 INFO:tasks.ceph.mds.c:Stopped 2023-11-06T02:11:25.457 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-06T02:11:25.460 INFO:tasks.ceph.mds.b:Stopped 2023-11-06T02:11:25.460 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-06T02:11:25.462 INFO:tasks.ceph.mds.d:Stopped 2023-11-06T02:11:25.463 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-06T02:11:25.501 INFO:tasks.ceph.mds.a:Started 2023-11-06T02:11:25.503 DEBUG:teuthology.parallel:result is None 2023-11-06T02:11:25.504 INFO:tasks.ceph.mds.c:Started 2023-11-06T02:11:25.504 DEBUG:teuthology.parallel:result is None 2023-11-06T02:11:25.505 INFO:tasks.ceph.mds.b:Started 2023-11-06T02:11:25.506 DEBUG:teuthology.parallel:result is None 2023-11-06T02:11:25.507 INFO:tasks.ceph.mds.d:Started 2023-11-06T02:11:25.507 DEBUG:teuthology.parallel:result is None 2023-11-06T02:11:25.508 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-06T02:11:25.565 INFO:tasks.ceph.mds.a.smithi049.stdout:starting mds.a at 2023-11-06T02:11:25.575 INFO:tasks.ceph.mds.c.smithi049.stdout:starting mds.c at 2023-11-06T02:11:25.579 INFO:tasks.ceph.mds.d.smithi150.stdout:starting mds.d at 2023-11-06T02:11:25.580 INFO:tasks.ceph.mds.b.smithi150.stdout:starting mds.b at 2023-11-06T02:11:25.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.649+0000 7f2bb6020640 1 Processor -- start 2023-11-06T02:11:25.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.649+0000 7f2bb6020640 1 -- start start 2023-11-06T02:11:25.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.649+0000 7f2bb6020640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f2bb0159e60 0x7f2bb015a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:25.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.649+0000 7f2bb6020640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bb00cace0 con 0x7f2bb0153ea0 2023-11-06T02:11:25.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.649+0000 7f2bb6020640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bb00ca6f0 con 0x7f2bb00cdd10 2023-11-06T02:11:25.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.649+0000 7f2bb6020640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bb015a920 con 0x7f2bb0159e60 2023-11-06T02:11:25.650 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.649+0000 7f2baf7fe640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f2bb0153ea0 0x7f2bb00cb170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:45990/0 (socket says 172.21.15.49:45990) 2023-11-06T02:11:25.650 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.649+0000 7f2baf7fe640 1 -- 172.21.15.49:0/3195125159 learned_addr learned my addr 172.21.15.49:0/3195125159 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:11:25.650 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.649+0000 7f2baeffd640 1 --2- 172.21.15.49:0/3195125159 >> v2:172.21.15.150:3300/0 conn(0x7f2bb0159e60 0x7f2bb015a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:25.650 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.649+0000 7f2bae7fc640 1 -- 172.21.15.49:0/3195125159 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3582694057 0 0) 0x7f2bb00cace0 con 0x7f2bb0153ea0 2023-11-06T02:11:25.651 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.649+0000 7f2bae7fc640 1 -- 172.21.15.49:0/3195125159 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b98003530 con 0x7f2bb0153ea0 2023-11-06T02:11:25.651 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.649+0000 7f2bae7fc640 1 -- 172.21.15.49:0/3195125159 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2698967948 0 0) 0x7f2b98003530 con 0x7f2bb0153ea0 2023-11-06T02:11:25.651 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.649+0000 7f2bae7fc640 1 -- 172.21.15.49:0/3195125159 >> v2:172.21.15.150:3300/0 conn(0x7f2bb0159e60 msgr2=0x7f2bb015a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:25.651 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.649+0000 7f2bae7fc640 1 --2- 172.21.15.49:0/3195125159 >> v2:172.21.15.150:3300/0 conn(0x7f2bb0159e60 0x7f2bb015a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:25.651 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.649+0000 7f2bae7fc640 1 -- 172.21.15.49:0/3195125159 >> v1:172.21.15.150:6789/0 conn(0x7f2bb00cdd10 legacy=0x7f2bb0069d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:25.652 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.649+0000 7f2bae7fc640 1 -- 172.21.15.49:0/3195125159 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bb015ac00 con 0x7f2bb0153ea0 2023-11-06T02:11:25.652 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.649+0000 7f2bae7fc640 1 -- 172.21.15.49:0/3195125159 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7f2ba4001680 con 0x7f2bb0153ea0 2023-11-06T02:11:25.652 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.649+0000 7f2bae7fc640 1 -- 172.21.15.49:0/3195125159 >> v1:172.21.15.49:6789/0 conn(0x7f2bb0153ea0 legacy=0x7f2bb00cb170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:25.652 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.649+0000 7f2bae7fc640 1 --2- 172.21.15.49:0/3195125159 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b98003980 0x7f2b98003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:25.652 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.649+0000 7f2bae7fc640 1 --2- 172.21.15.49:0/3195125159 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b980047d0 0x7f2b98004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:25.653 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.649+0000 7f2bae7fc640 1 --2- 172.21.15.49:0/3195125159 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b98005190 0x7f2b9800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:25.653 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.649+0000 7f2bae7fc640 1 -- 172.21.15.49:0/3195125159 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb015a920 con 0x7f2b98003980 2023-11-06T02:11:25.653 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.649+0000 7f2bae7fc640 1 -- 172.21.15.49:0/3195125159 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b9800c030 con 0x7f2b98005190 2023-11-06T02:11:25.653 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.649+0000 7f2bae7fc640 1 -- 172.21.15.49:0/3195125159 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b9800c1f0 con 0x7f2b980047d0 2023-11-06T02:11:25.654 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.649+0000 7f2baf7fe640 1 --2- 172.21.15.49:0/3195125159 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b98003980 0x7f2b98003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:25.655 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.649+0000 7f2baffff640 1 --2- 172.21.15.49:0/3195125159 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b98005190 0x7f2b9800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:25.655 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.649+0000 7f2baeffd640 1 --2- 172.21.15.49:0/3195125159 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b980047d0 0x7f2b98004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:25.655 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.649+0000 7f2baf7fe640 1 -- 172.21.15.49:0/3195125159 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b980047d0 msgr2=0x7f2b98004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:25.655 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.649+0000 7f2baf7fe640 1 --2- 172.21.15.49:0/3195125159 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b980047d0 0x7f2b98004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:25.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.649+0000 7f2baf7fe640 1 -- 172.21.15.49:0/3195125159 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b98005190 msgr2=0x7f2b9800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:25.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.649+0000 7f2baf7fe640 1 --2- 172.21.15.49:0/3195125159 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b98005190 0x7f2b9800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:25.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.649+0000 7f2baf7fe640 1 -- 172.21.15.49:0/3195125159 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b98003530 con 0x7f2b98003980 2023-11-06T02:11:25.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.649+0000 7f2baeffd640 1 --2- 172.21.15.49:0/3195125159 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b980047d0 0x7f2b98004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:11:25.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.649+0000 7f2baffff640 1 --2- 172.21.15.49:0/3195125159 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b98005190 0x7f2b9800b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:11:25.657 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.649+0000 7f2baffff640 1 -- 172.21.15.49:0/3195125159 reap_dead start 2023-11-06T02:11:25.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.653+0000 7f2baf7fe640 1 --2- 172.21.15.49:0/3195125159 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b98003980 0x7f2b98003da0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f2ba400c1c0 tx=0x7f2ba401cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd13537c8b52ad37 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:25.662 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.653+0000 7f2bae7fc640 1 -- 172.21.15.49:0/3195125159 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba4005ae0 con 0x7f2b98003980 2023-11-06T02:11:25.662 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.653+0000 7f2bae7fc640 1 -- 172.21.15.49:0/3195125159 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ba4005c40 con 0x7f2b98003980 2023-11-06T02:11:25.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.653+0000 7f2bae7fc640 1 -- 172.21.15.49:0/3195125159 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba40a5a50 con 0x7f2b98003980 2023-11-06T02:11:25.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.653+0000 7f2bb6020640 1 -- 172.21.15.49:0/3195125159 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b98003980 msgr2=0x7f2b98003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:25.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.653+0000 7f2bb6020640 1 --2- 172.21.15.49:0/3195125159 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b98003980 0x7f2b98003da0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f2ba400c1c0 tx=0x7f2ba401cc20 comp rx=0 tx=0).stop 2023-11-06T02:11:25.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.653+0000 7f2bb6020640 1 -- 172.21.15.49:0/3195125159 shutdown_connections 2023-11-06T02:11:25.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.653+0000 7f2bb6020640 1 --2- 172.21.15.49:0/3195125159 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b98003980 0x7f2b98003da0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:25.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.653+0000 7f2bb6020640 1 -- 172.21.15.49:0/3195125159 >> 172.21.15.49:0/3195125159 conn(0x7f2bb00b0bd0 msgr2=0x7f2bb00b0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:25.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.653+0000 7f2bb6020640 1 -- 172.21.15.49:0/3195125159 shutdown_connections 2023-11-06T02:11:25.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.653+0000 7f2bb6020640 1 -- 172.21.15.49:0/3195125159 wait complete. 2023-11-06T02:11:25.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.653+0000 7f2bb6020640 1 Processor -- start 2023-11-06T02:11:25.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.653+0000 7f2bb6020640 1 -- start start 2023-11-06T02:11:25.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.657+0000 7f2bb6020640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b98003980 0x7f2bb0141a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:25.667 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.657+0000 7f2bb6020640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2bb0152e00 0x7f2bb0150380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:25.667 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.657+0000 7f2bb6020640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2bb014d590 0x7f2bb014e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:25.668 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.657+0000 7f2bb6020640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb0141fc0 con 0x7f2bb014d590 2023-11-06T02:11:25.668 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.657+0000 7f2bb6020640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb0152970 con 0x7f2bb0152e00 2023-11-06T02:11:25.668 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.657+0000 7f2bb6020640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb01508c0 con 0x7f2b98003980 2023-11-06T02:11:25.669 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.657+0000 7f2baffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2bb014d590 0x7f2bb014e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:25.669 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.657+0000 7f2baffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2bb014d590 0x7f2bb014e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49576/0 (socket says 172.21.15.49:49576) 2023-11-06T02:11:25.669 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.657+0000 7f2baffff640 1 -- 172.21.15.49:0/4097384253 learned_addr learned my addr 172.21.15.49:0/4097384253 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:25.670 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.657+0000 7f2baeffd640 1 --2- 172.21.15.49:0/4097384253 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2bb0152e00 0x7f2bb0150380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:25.670 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.657+0000 7f2baf7fe640 1 --2- 172.21.15.49:0/4097384253 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b98003980 0x7f2bb0141a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:25.670 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.657+0000 7f2baffff640 1 -- 172.21.15.49:0/4097384253 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b98003980 msgr2=0x7f2bb0141a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:25.670 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.657+0000 7f2baffff640 1 --2- 172.21.15.49:0/4097384253 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b98003980 0x7f2bb0141a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:25.670 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.657+0000 7f2baffff640 1 -- 172.21.15.49:0/4097384253 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2bb0152e00 msgr2=0x7f2bb0150380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:25.671 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.657+0000 7f2baffff640 1 --2- 172.21.15.49:0/4097384253 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2bb0152e00 0x7f2bb0150380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:25.671 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.657+0000 7f2baffff640 1 -- 172.21.15.49:0/4097384253 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bb0152060 con 0x7f2bb014d590 2023-11-06T02:11:25.671 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.657+0000 7f2baeffd640 1 --2- 172.21.15.49:0/4097384253 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2bb0152e00 0x7f2bb0150380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:11:25.671 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.657+0000 7f2baf7fe640 1 --2- 172.21.15.49:0/4097384253 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b98003980 0x7f2bb0141a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:11:25.672 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.657+0000 7f2baffff640 1 --2- 172.21.15.49:0/4097384253 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2bb014d590 0x7f2bb014e390 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f2bb015a240 tx=0x7f2ba800eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:25.672 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.657+0000 7f2b97fff640 1 -- 172.21.15.49:0/4097384253 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba800eed0 con 0x7f2bb014d590 2023-11-06T02:11:25.672 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.657+0000 7f2b97fff640 1 -- 172.21.15.49:0/4097384253 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ba8003980 con 0x7f2bb014d590 2023-11-06T02:11:25.672 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.657+0000 7f2bb6020640 1 -- 172.21.15.49:0/4097384253 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bb014dac0 con 0x7f2bb014d590 2023-11-06T02:11:25.672 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.657+0000 7f2b97fff640 1 -- 172.21.15.49:0/4097384253 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba8003d10 con 0x7f2bb014d590 2023-11-06T02:11:25.673 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.657+0000 7f2bb6020640 1 -- 172.21.15.49:0/4097384253 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bb0152600 con 0x7f2bb014d590 2023-11-06T02:11:25.673 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.657+0000 7f2b95ffb640 1 -- 172.21.15.49:0/4097384253 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b88005260 con 0x7f2bb014d590 2023-11-06T02:11:25.673 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.657+0000 7f2b97fff640 1 -- 172.21.15.49:0/4097384253 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f2ba80050d0 con 0x7f2bb014d590 2023-11-06T02:11:25.673 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.657+0000 7f2b97fff640 1 --2- 172.21.15.49:0/4097384253 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2b9c065a90 0x7f2b9c067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:25.673 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.657+0000 7f2b97fff640 1 -- 172.21.15.49:0/4097384253 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f2ba80828d0 con 0x7f2bb014d590 2023-11-06T02:11:25.674 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.657+0000 7f2baf7fe640 1 --2- 172.21.15.49:0/4097384253 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2b9c065a90 0x7f2b9c067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:25.674 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.661+0000 7f2baf7fe640 1 --2- 172.21.15.49:0/4097384253 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2b9c065a90 0x7f2b9c067f50 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2ba40ab000 tx=0x7f2ba40a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:25.674 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.661+0000 7f2b97fff640 1 -- 172.21.15.49:0/4097384253 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f2ba80504c0 con 0x7f2bb014d590 2023-11-06T02:11:25.831 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:25.833+0000 7f2b95ffb640 1 -- 172.21.15.49:0/4097384253 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f2b88005510 con 0x7f2bb014d590 2023-11-06T02:11:26.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:26.629+0000 7f2b97fff640 1 -- 172.21.15.49:0/4097384253 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v61) v1 ==== 97+0+0 (secure 0 0 0) 0x7f2ba8053ae0 con 0x7f2bb014d590 2023-11-06T02:11:26.625 INFO:teuthology.orchestra.run.smithi049.stderr: removed all blocklist entries 2023-11-06T02:11:26.628 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:26.629+0000 7f2b95ffb640 1 -- 172.21.15.49:0/4097384253 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2b9c065a90 msgr2=0x7f2b9c067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:26.628 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:26.629+0000 7f2b95ffb640 1 --2- 172.21.15.49:0/4097384253 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2b9c065a90 0x7f2b9c067f50 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2ba40ab000 tx=0x7f2ba40a7070 comp rx=0 tx=0).stop 2023-11-06T02:11:26.629 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:26.629+0000 7f2b95ffb640 1 -- 172.21.15.49:0/4097384253 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2bb014d590 msgr2=0x7f2bb014e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:26.629 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:26.629+0000 7f2b95ffb640 1 --2- 172.21.15.49:0/4097384253 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2bb014d590 0x7f2bb014e390 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f2bb015a240 tx=0x7f2ba800eb90 comp rx=0 tx=0).stop 2023-11-06T02:11:26.629 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:26.629+0000 7f2b95ffb640 1 -- 172.21.15.49:0/4097384253 shutdown_connections 2023-11-06T02:11:26.629 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:26.629+0000 7f2b95ffb640 1 --2- 172.21.15.49:0/4097384253 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2b9c065a90 0x7f2b9c067f50 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:26.630 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:26.629+0000 7f2b95ffb640 1 --2- 172.21.15.49:0/4097384253 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2bb014d590 0x7f2bb014e390 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:26.630 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:26.629+0000 7f2b95ffb640 1 --2- 172.21.15.49:0/4097384253 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2bb0152e00 0x7f2bb0150380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:26.630 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:26.629+0000 7f2b95ffb640 1 --2- 172.21.15.49:0/4097384253 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b98003980 0x7f2bb0141a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:26.631 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:26.629+0000 7f2b95ffb640 1 -- 172.21.15.49:0/4097384253 >> 172.21.15.49:0/4097384253 conn(0x7f2bb00b0bd0 msgr2=0x7f2bb0069d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:26.631 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:26.629+0000 7f2b95ffb640 1 -- 172.21.15.49:0/4097384253 shutdown_connections 2023-11-06T02:11:26.631 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:26.629+0000 7f2b95ffb640 1 -- 172.21.15.49:0/4097384253 wait complete. 2023-11-06T02:11:26.651 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-06T02:11:26.777 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:26.781+0000 7f6438d61640 1 Processor -- start 2023-11-06T02:11:26.778 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:26.781+0000 7f6438d61640 1 -- start start 2023-11-06T02:11:26.779 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:26.781+0000 7f6438d61640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f643415ceb0 0x7f64340cb970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:26.779 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:26.781+0000 7f6438d61640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f64340caf30 0x7f6434071120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:26.779 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:26.781+0000 7f6438d61640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64340cbeb0 con 0x7f64340cb370 2023-11-06T02:11:26.780 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:26.781+0000 7f6438d61640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64340caaa0 con 0x7f643415ceb0 2023-11-06T02:11:26.780 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:26.781+0000 7f6438d61640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64340ca1e0 con 0x7f64340caf30 2023-11-06T02:11:26.780 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:26.781+0000 7f643259c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f643415ceb0 0x7f64340cb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:26.780 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:26.781+0000 7f643259c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f643415ceb0 0x7f64340cb970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49582/0 (socket says 172.21.15.49:49582) 2023-11-06T02:11:26.780 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:26.781+0000 7f643259c640 1 -- 172.21.15.49:0/2507145908 learned_addr learned my addr 172.21.15.49:0/2507145908 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:26.782 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:26.781+0000 7f6431d9b640 1 --2- 172.21.15.49:0/2507145908 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f64340caf30 0x7f6434071120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:26.782 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:26.781+0000 7f643259c640 1 -- 172.21.15.49:0/2507145908 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f64340caf30 msgr2=0x7f6434071120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:26.783 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:26.781+0000 7f643259c640 1 --2- 172.21.15.49:0/2507145908 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f64340caf30 0x7f6434071120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:26.783 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:26.781+0000 7f643259c640 1 -- 172.21.15.49:0/2507145908 >> v1:172.21.15.150:6789/0 conn(0x7f64340cb370 legacy=0x7f6434069d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:26.783 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:26.781+0000 7f643259c640 1 -- 172.21.15.49:0/2507145908 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64340ce490 con 0x7f643415ceb0 2023-11-06T02:11:26.783 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:26.781+0000 7f6431d9b640 1 --2- 172.21.15.49:0/2507145908 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f64340caf30 0x7f6434071120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:11:26.784 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:26.785+0000 7f643259c640 1 --2- 172.21.15.49:0/2507145908 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f643415ceb0 0x7f64340cb970 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f6424009870 tx=0x7f642401c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f6d740d98092520 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:26.784 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:26.785+0000 7f643159a640 1 -- 172.21.15.49:0/2507145908 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64240ab600 con 0x7f643415ceb0 2023-11-06T02:11:26.784 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:26.785+0000 7f643159a640 1 -- 172.21.15.49:0/2507145908 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64240a7050 con 0x7f643415ceb0 2023-11-06T02:11:26.784 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:26.785+0000 7f643159a640 1 -- 172.21.15.49:0/2507145908 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64240ab980 con 0x7f643415ceb0 2023-11-06T02:11:26.787 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:26.785+0000 7f6438d61640 1 -- 172.21.15.49:0/2507145908 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f643415ceb0 msgr2=0x7f64340cb970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:26.787 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:26.785+0000 7f6438d61640 1 --2- 172.21.15.49:0/2507145908 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f643415ceb0 0x7f64340cb970 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f6424009870 tx=0x7f642401c540 comp rx=0 tx=0).stop 2023-11-06T02:11:26.788 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:26.785+0000 7f6438d61640 1 -- 172.21.15.49:0/2507145908 shutdown_connections 2023-11-06T02:11:26.788 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:26.785+0000 7f6438d61640 1 --2- 172.21.15.49:0/2507145908 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f64340caf30 0x7f6434071120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:26.788 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:26.785+0000 7f6438d61640 1 --2- 172.21.15.49:0/2507145908 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f643415ceb0 0x7f64340cb970 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:26.789 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:26.785+0000 7f6438d61640 1 -- 172.21.15.49:0/2507145908 >> 172.21.15.49:0/2507145908 conn(0x7f64340cc140 msgr2=0x7f64340cc550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:26.789 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:26.785+0000 7f6438d61640 1 -- 172.21.15.49:0/2507145908 shutdown_connections 2023-11-06T02:11:26.789 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:26.785+0000 7f6438d61640 1 -- 172.21.15.49:0/2507145908 wait complete. 2023-11-06T02:11:26.789 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:26.789+0000 7f6438d61640 1 Processor -- start 2023-11-06T02:11:26.789 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:26.789+0000 7f6438d61640 1 -- start start 2023-11-06T02:11:26.790 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:26.789+0000 7f6438d61640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f64340caf30 0x7f64340acf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:26.790 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:26.789+0000 7f6438d61640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f643415ceb0 0x7f64340ac830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:26.790 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:26.789+0000 7f6438d61640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6434155de0 0x7f6434157760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:26.790 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:26.789+0000 7f6438d61640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64340acd70 con 0x7f6434155de0 2023-11-06T02:11:26.790 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:26.789+0000 7f6438d61640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f643415fec0 con 0x7f643415ceb0 2023-11-06T02:11:26.791 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:26.789+0000 7f6438d61640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64340ad5c0 con 0x7f64340caf30 2023-11-06T02:11:26.791 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:26.789+0000 7f6432d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6434155de0 0x7f6434157760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:26.791 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:26.789+0000 7f6432d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6434155de0 0x7f6434157760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49584/0 (socket says 172.21.15.49:49584) 2023-11-06T02:11:26.791 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:26.789+0000 7f6432d9d640 1 -- 172.21.15.49:0/580291531 learned_addr learned my addr 172.21.15.49:0/580291531 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:26.791 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:26.789+0000 7f643259c640 1 --2- 172.21.15.49:0/580291531 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f64340caf30 0x7f64340acf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:26.792 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:26.789+0000 7f6431d9b640 1 --2- 172.21.15.49:0/580291531 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f643415ceb0 0x7f64340ac830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:26.792 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:26.789+0000 7f6432d9d640 1 -- 172.21.15.49:0/580291531 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f64340caf30 msgr2=0x7f64340acf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:26.792 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:26.789+0000 7f6432d9d640 1 --2- 172.21.15.49:0/580291531 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f64340caf30 0x7f64340acf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:26.792 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:26.789+0000 7f6432d9d640 1 -- 172.21.15.49:0/580291531 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f643415ceb0 msgr2=0x7f64340ac830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:26.792 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:26.789+0000 7f6432d9d640 1 --2- 172.21.15.49:0/580291531 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f643415ceb0 0x7f64340ac830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:26.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:26.789+0000 7f6432d9d640 1 -- 172.21.15.49:0/580291531 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64240094d0 con 0x7f6434155de0 2023-11-06T02:11:26.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:26.789+0000 7f6432d9d640 1 --2- 172.21.15.49:0/580291531 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6434155de0 0x7f6434157760 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f642c00c7a0 tx=0x7f642c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:26.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:26.789+0000 7f6422ffd640 1 -- 172.21.15.49:0/580291531 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f642c003b80 con 0x7f6434155de0 2023-11-06T02:11:26.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:26.789+0000 7f6422ffd640 1 -- 172.21.15.49:0/580291531 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f642c003ce0 con 0x7f6434155de0 2023-11-06T02:11:26.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:26.789+0000 7f6438d61640 1 -- 172.21.15.49:0/580291531 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64340ad8a0 con 0x7f6434155de0 2023-11-06T02:11:26.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:26.789+0000 7f6422ffd640 1 -- 172.21.15.49:0/580291531 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f642c00aa30 con 0x7f6434155de0 2023-11-06T02:11:26.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:26.789+0000 7f6438d61640 1 -- 172.21.15.49:0/580291531 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64340ade60 con 0x7f6434155de0 2023-11-06T02:11:26.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:26.789+0000 7f6438d61640 1 -- 172.21.15.49:0/580291531 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64340ae080 con 0x7f6434155de0 2023-11-06T02:11:26.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:26.789+0000 7f6422ffd640 1 -- 172.21.15.49:0/580291531 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f642c00ab90 con 0x7f6434155de0 2023-11-06T02:11:26.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:26.789+0000 7f6422ffd640 1 --2- 172.21.15.49:0/580291531 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6414065a60 0x7f6414067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:26.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:26.793+0000 7f6422ffd640 1 -- 172.21.15.49:0/580291531 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f642c082b80 con 0x7f6434155de0 2023-11-06T02:11:26.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:26.793+0000 7f643259c640 1 --2- 172.21.15.49:0/580291531 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6414065a60 0x7f6414067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:26.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:26.793+0000 7f643259c640 1 --2- 172.21.15.49:0/580291531 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6414065a60 0x7f6414067f20 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f64340b5830 tx=0x7f6424008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:26.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:26.797+0000 7f6422ffd640 1 -- 172.21.15.49:0/580291531 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f642c0507c0 con 0x7f6434155de0 2023-11-06T02:11:27.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.009+0000 7f6438d61640 1 -- 172.21.15.49:0/580291531 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f6434067140 con 0x7f6434155de0 2023-11-06T02:11:27.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.009+0000 7f6422ffd640 1 -- 172.21.15.49:0/580291531 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v12) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f642c053de0 con 0x7f6434155de0 2023-11-06T02:11:27.007 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:11:27.008 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-11-06T02:11:27.008 INFO:teuthology.orchestra.run.smithi049.stdout: "auth_dump": [ 2023-11-06T02:11:27.008 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:11:27.008 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "mds.a", 2023-11-06T02:11:27.009 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQBbSkhlbRFwIhAApA6xaqdIIeKDsOsNhyWCaA==", 2023-11-06T02:11:27.009 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-11-06T02:11:27.009 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": "allow", 2023-11-06T02:11:27.009 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow *", 2023-11-06T02:11:27.009 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow *", 2023-11-06T02:11:27.009 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-11-06T02:11:27.010 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:11:27.010 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:11:27.010 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:11:27.010 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "mds.b", 2023-11-06T02:11:27.010 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQBbSkhl6QhcMRAAgmOxZWghmuu11F0Gfut3sQ==", 2023-11-06T02:11:27.011 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-11-06T02:11:27.011 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": "allow", 2023-11-06T02:11:27.011 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow *", 2023-11-06T02:11:27.011 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow *", 2023-11-06T02:11:27.011 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-11-06T02:11:27.012 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:11:27.012 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:11:27.012 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:11:27.012 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "mds.c", 2023-11-06T02:11:27.012 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQBbSkhlMoZHKxAATV62bAItdXt3ShpsT4oVbA==", 2023-11-06T02:11:27.013 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-11-06T02:11:27.013 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": "allow", 2023-11-06T02:11:27.013 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow *", 2023-11-06T02:11:27.013 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow *", 2023-11-06T02:11:27.013 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-11-06T02:11:27.014 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:11:27.014 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:11:27.014 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:11:27.014 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "mds.d", 2023-11-06T02:11:27.014 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQBbSkhl376GORAAS2NIQJMzg6Es0JCYPak6fw==", 2023-11-06T02:11:27.015 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-11-06T02:11:27.015 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": "allow", 2023-11-06T02:11:27.015 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow *", 2023-11-06T02:11:27.015 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow *", 2023-11-06T02:11:27.016 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-11-06T02:11:27.016 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:11:27.016 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:11:27.016 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:11:27.017 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "osd.0", 2023-11-06T02:11:27.017 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQBdSkhlxuQtFRAAPttjVsYpkwexC5x2Pu2b+Q==", 2023-11-06T02:11:27.017 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-11-06T02:11:27.017 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow profile osd", 2023-11-06T02:11:27.017 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow profile osd", 2023-11-06T02:11:27.018 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-11-06T02:11:27.018 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:11:27.018 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:11:27.018 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:11:27.019 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "osd.1", 2023-11-06T02:11:27.019 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQBgSkhlCGeiHBAA8SofNV6C1+RQNajUsewVpw==", 2023-11-06T02:11:27.019 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-11-06T02:11:27.019 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow profile osd", 2023-11-06T02:11:27.019 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow profile osd", 2023-11-06T02:11:27.020 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-11-06T02:11:27.020 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:11:27.020 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:11:27.020 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:11:27.020 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "osd.2", 2023-11-06T02:11:27.021 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQBjSkhlBju+JBAAPJeIFngKDoamUvEjv1h8tw==", 2023-11-06T02:11:27.021 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-11-06T02:11:27.021 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow profile osd", 2023-11-06T02:11:27.021 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow profile osd", 2023-11-06T02:11:27.021 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-11-06T02:11:27.022 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:11:27.022 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:11:27.022 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:11:27.022 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "osd.3", 2023-11-06T02:11:27.022 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQBmSkhlMclTLRAABnbmQA9j3TLn40H1wYFx5g==", 2023-11-06T02:11:27.023 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-11-06T02:11:27.023 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow profile osd", 2023-11-06T02:11:27.023 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow profile osd", 2023-11-06T02:11:27.023 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-11-06T02:11:27.023 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:11:27.023 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:11:27.024 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:11:27.024 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "osd.4", 2023-11-06T02:11:27.024 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQBsSkhl7fmmLhAArv62qbJ+StgYMKhCftM5HA==", 2023-11-06T02:11:27.024 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-11-06T02:11:27.025 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow profile osd", 2023-11-06T02:11:27.025 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow profile osd", 2023-11-06T02:11:27.025 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-11-06T02:11:27.026 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:11:27.026 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:11:27.026 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:11:27.027 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "osd.5", 2023-11-06T02:11:27.027 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQBvSkhl5pAANhAArNFy9/YuEcXTdzM5XL46Xw==", 2023-11-06T02:11:27.027 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-11-06T02:11:27.027 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow profile osd", 2023-11-06T02:11:27.028 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow profile osd", 2023-11-06T02:11:27.028 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-11-06T02:11:27.028 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:11:27.028 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:11:27.028 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:11:27.029 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "osd.6", 2023-11-06T02:11:27.029 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQBzSkhlgfs2BBAAeHXVJ/tFiFagugkynGakFw==", 2023-11-06T02:11:27.029 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-11-06T02:11:27.029 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow profile osd", 2023-11-06T02:11:27.030 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow profile osd", 2023-11-06T02:11:27.030 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-11-06T02:11:27.030 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:11:27.030 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:11:27.030 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:11:27.031 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "osd.7", 2023-11-06T02:11:27.031 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQB2SkhlQbCYDRAAhE7vwbjJjqZDHfgDfdy1cA==", 2023-11-06T02:11:27.031 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-11-06T02:11:27.031 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow profile osd", 2023-11-06T02:11:27.031 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow profile osd", 2023-11-06T02:11:27.032 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-11-06T02:11:27.032 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:11:27.032 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:11:27.032 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:11:27.032 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "client.0", 2023-11-06T02:11:27.033 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQBcSkhlY46mAxAALClTH91L6USom3nVss6iiQ==", 2023-11-06T02:11:27.033 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-11-06T02:11:27.033 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": "allow", 2023-11-06T02:11:27.033 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow r", 2023-11-06T02:11:27.033 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-06T02:11:27.034 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:11:27.034 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:11:27.034 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:11:27.034 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "client.1", 2023-11-06T02:11:27.034 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQBcSkhllvSyCxAAfcSiBhHHNgaIr9mhpY11fg==", 2023-11-06T02:11:27.035 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-11-06T02:11:27.035 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": "allow", 2023-11-06T02:11:27.035 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow r", 2023-11-06T02:11:27.035 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow rw", 2023-11-06T02:11:27.035 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow rwx" 2023-11-06T02:11:27.035 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:11:27.036 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:11:27.036 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:11:27.036 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "client.2", 2023-11-06T02:11:27.036 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQBcSkhlURf6CBAA9g2y/UMD4x42IT0fgSgIcg==", 2023-11-06T02:11:27.036 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-11-06T02:11:27.037 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": "allow", 2023-11-06T02:11:27.037 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow r", 2023-11-06T02:11:27.037 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow rw", 2023-11-06T02:11:27.037 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow rwx" 2023-11-06T02:11:27.038 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:11:27.038 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:11:27.038 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:11:27.038 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "client.3", 2023-11-06T02:11:27.038 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQBcSkhlbjXXEBAA0xQGCY/avAWL18GLQtIjgg==", 2023-11-06T02:11:27.039 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-11-06T02:11:27.039 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": "allow", 2023-11-06T02:11:27.039 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow r", 2023-11-06T02:11:27.039 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow rw", 2023-11-06T02:11:27.039 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow rwx" 2023-11-06T02:11:27.040 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:11:27.040 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:11:27.040 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:11:27.040 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "client.admin", 2023-11-06T02:11:27.040 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQBbSkhloSJXCxAA++ZHscsqHENk8soyqq/GPQ==", 2023-11-06T02:11:27.041 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-11-06T02:11:27.041 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": "allow *", 2023-11-06T02:11:27.041 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow *", 2023-11-06T02:11:27.041 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow *", 2023-11-06T02:11:27.041 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-11-06T02:11:27.042 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:11:27.042 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:11:27.042 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:11:27.042 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "mgr.x", 2023-11-06T02:11:27.042 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQBbSkhlpJ4iHxAAGEz4XS5fILy7/NZbAenPFw==", 2023-11-06T02:11:27.043 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-11-06T02:11:27.043 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": "allow *", 2023-11-06T02:11:27.043 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow profile mgr", 2023-11-06T02:11:27.043 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-11-06T02:11:27.043 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:11:27.043 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:11:27.044 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:11:27.044 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "mgr.y", 2023-11-06T02:11:27.044 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQBbSkhln1JuHBAAymHRkniPpsI6pf9L2C2y+g==", 2023-11-06T02:11:27.044 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-11-06T02:11:27.044 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": "allow *", 2023-11-06T02:11:27.045 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow profile mgr", 2023-11-06T02:11:27.045 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-11-06T02:11:27.045 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:11:27.045 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:11:27.045 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2023-11-06T02:11:27.046 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-11-06T02:11:27.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.013+0000 7f6438d61640 1 -- 172.21.15.49:0/580291531 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6414065a60 msgr2=0x7f6414067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:27.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.013+0000 7f6438d61640 1 --2- 172.21.15.49:0/580291531 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6414065a60 0x7f6414067f20 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f64340b5830 tx=0x7f6424008040 comp rx=0 tx=0).stop 2023-11-06T02:11:27.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.013+0000 7f6438d61640 1 -- 172.21.15.49:0/580291531 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6434155de0 msgr2=0x7f6434157760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:27.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.013+0000 7f6438d61640 1 --2- 172.21.15.49:0/580291531 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6434155de0 0x7f6434157760 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f642c00c7a0 tx=0x7f642c00cc60 comp rx=0 tx=0).stop 2023-11-06T02:11:27.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.013+0000 7f6438d61640 1 -- 172.21.15.49:0/580291531 shutdown_connections 2023-11-06T02:11:27.049 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.013+0000 7f6438d61640 1 --2- 172.21.15.49:0/580291531 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6414065a60 0x7f6414067f20 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:27.049 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.013+0000 7f6438d61640 1 --2- 172.21.15.49:0/580291531 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6434155de0 0x7f6434157760 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:27.049 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.013+0000 7f6438d61640 1 --2- 172.21.15.49:0/580291531 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f643415ceb0 0x7f64340ac830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:27.049 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.013+0000 7f6438d61640 1 --2- 172.21.15.49:0/580291531 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f64340caf30 0x7f64340acf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:27.049 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.013+0000 7f6438d61640 1 -- 172.21.15.49:0/580291531 >> 172.21.15.49:0/580291531 conn(0x7f64340cc140 msgr2=0x7f64340c94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:27.050 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.013+0000 7f6438d61640 1 -- 172.21.15.49:0/580291531 shutdown_connections 2023-11-06T02:11:27.050 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.013+0000 7f6438d61640 1 -- 172.21.15.49:0/580291531 wait complete. 2023-11-06T02:11:27.050 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T02:11:27.176 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.177+0000 7fd8fe63b640 1 Processor -- start 2023-11-06T02:11:27.177 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.177+0000 7fd8fe63b640 1 -- start start 2023-11-06T02:11:27.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.181+0000 7fd8fe63b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd8f815ab50 0x7fd8f80c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:27.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.181+0000 7fd8fe63b640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fd8f80b0790 0x7fd8f80c2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:27.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.181+0000 7fd8fe63b640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8f80cab80 con 0x7fd8f80cb010 2023-11-06T02:11:27.179 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.181+0000 7fd8fe63b640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8f815af30 con 0x7fd8f80b0790 2023-11-06T02:11:27.179 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.181+0000 7fd8fe63b640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f80cb5b0 con 0x7fd8f815ab50 2023-11-06T02:11:27.179 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.181+0000 7fd8effff640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fd8f80cb010 0x7fd8f80b0080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:45994/0 (socket says 172.21.15.49:45994) 2023-11-06T02:11:27.179 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.181+0000 7fd8effff640 1 -- 172.21.15.49:0/3130199973 learned_addr learned my addr 172.21.15.49:0/3130199973 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:11:27.179 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.181+0000 7fd8f7fff640 1 --2- 172.21.15.49:0/3130199973 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd8f815ab50 0x7fd8f80c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:27.180 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.181+0000 7fd8fcbd8640 1 --2- 172.21.15.49:0/3130199973 >> v2:172.21.15.150:3300/0 conn(0x7fd8f80b0790 0x7fd8f80c2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:27.180 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.181+0000 7fd8f77fe640 1 -- 172.21.15.49:0/3130199973 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 225419942 0 0) 0x7fd8f80cab80 con 0x7fd8f80cb010 2023-11-06T02:11:27.180 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.181+0000 7fd8f77fe640 1 -- 172.21.15.49:0/3130199973 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8e4003530 con 0x7fd8f80cb010 2023-11-06T02:11:27.182 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.181+0000 7fd8f77fe640 1 -- 172.21.15.49:0/3130199973 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2746277830 0 0) 0x7fd8e4003530 con 0x7fd8f80cb010 2023-11-06T02:11:27.182 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.181+0000 7fd8f77fe640 1 -- 172.21.15.49:0/3130199973 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd8f815ab50 msgr2=0x7fd8f80c9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:27.182 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.181+0000 7fd8f77fe640 1 --2- 172.21.15.49:0/3130199973 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd8f815ab50 0x7fd8f80c9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:27.183 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.181+0000 7fd8f77fe640 1 -- 172.21.15.49:0/3130199973 >> v2:172.21.15.150:3300/0 conn(0x7fd8f80b0790 msgr2=0x7fd8f80c2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:27.183 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.181+0000 7fd8f77fe640 1 --2- 172.21.15.49:0/3130199973 >> v2:172.21.15.150:3300/0 conn(0x7fd8f80b0790 0x7fd8f80c2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:27.183 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.181+0000 7fd8f77fe640 1 -- 172.21.15.49:0/3130199973 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8f80cb890 con 0x7fd8f80cb010 2023-11-06T02:11:27.183 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.181+0000 7fd8f77fe640 1 -- 172.21.15.49:0/3130199973 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7fd8e0001680 con 0x7fd8f80cb010 2023-11-06T02:11:27.183 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.181+0000 7fd8f77fe640 1 -- 172.21.15.49:0/3130199973 >> v1:172.21.15.49:6789/0 conn(0x7fd8f80cb010 legacy=0x7fd8f80b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:27.184 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.181+0000 7fd8f77fe640 1 --2- 172.21.15.49:0/3130199973 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd8f815ab50 0x7fd8e4003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:27.184 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.181+0000 7fd8f77fe640 1 --2- 172.21.15.49:0/3130199973 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8e4003f00 0x7fd8e40047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:27.184 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.181+0000 7fd8f77fe640 1 --2- 172.21.15.49:0/3130199973 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd8e4004d10 0x7fd8e4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:27.184 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.181+0000 7fd8f77fe640 1 -- 172.21.15.49:0/3130199973 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f815af30 con 0x7fd8e4003f00 2023-11-06T02:11:27.184 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.181+0000 7fd8f77fe640 1 -- 172.21.15.49:0/3130199973 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f80cb5b0 con 0x7fd8e4004d10 2023-11-06T02:11:27.185 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.181+0000 7fd8f77fe640 1 -- 172.21.15.49:0/3130199973 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e400b990 con 0x7fd8f815ab50 2023-11-06T02:11:27.185 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.181+0000 7fd8effff640 1 --2- 172.21.15.49:0/3130199973 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8e4003f00 0x7fd8e40047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:27.185 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.181+0000 7fd8fcbd8640 1 --2- 172.21.15.49:0/3130199973 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd8e4004d10 0x7fd8e4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:27.185 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.181+0000 7fd8f7fff640 1 --2- 172.21.15.49:0/3130199973 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd8f815ab50 0x7fd8e4003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:27.185 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.181+0000 7fd8effff640 1 -- 172.21.15.49:0/3130199973 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd8f815ab50 msgr2=0x7fd8e4003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:27.186 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.181+0000 7fd8effff640 1 --2- 172.21.15.49:0/3130199973 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd8f815ab50 0x7fd8e4003980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:27.186 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.181+0000 7fd8effff640 1 -- 172.21.15.49:0/3130199973 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd8e4004d10 msgr2=0x7fd8e4005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:27.186 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.181+0000 7fd8effff640 1 --2- 172.21.15.49:0/3130199973 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd8e4004d10 0x7fd8e4005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:27.186 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.181+0000 7fd8effff640 1 -- 172.21.15.49:0/3130199973 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8e4003530 con 0x7fd8e4003f00 2023-11-06T02:11:27.186 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.185+0000 7fd8effff640 1 --2- 172.21.15.49:0/3130199973 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8e4003f00 0x7fd8e40047d0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fd8e000c290 tx=0x7fd8e001f3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d51e035ff66a84d8 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:27.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.185+0000 7fd8f77fe640 1 -- 172.21.15.49:0/3130199973 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e0005ae0 con 0x7fd8e4003f00 2023-11-06T02:11:27.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.185+0000 7fd8f77fe640 1 -- 172.21.15.49:0/3130199973 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8e00ab070 con 0x7fd8e4003f00 2023-11-06T02:11:27.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.185+0000 7fd8f77fe640 1 -- 172.21.15.49:0/3130199973 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e0005e80 con 0x7fd8e4003f00 2023-11-06T02:11:27.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.185+0000 7fd8fe63b640 1 -- 172.21.15.49:0/3130199973 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8e4003f00 msgr2=0x7fd8e40047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:27.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.185+0000 7fd8fe63b640 1 --2- 172.21.15.49:0/3130199973 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8e4003f00 0x7fd8e40047d0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fd8e000c290 tx=0x7fd8e001f3d0 comp rx=0 tx=0).stop 2023-11-06T02:11:27.188 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.185+0000 7fd8fcbd8640 1 -- 172.21.15.49:0/3130199973 reap_dead start 2023-11-06T02:11:27.188 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.185+0000 7fd8fe63b640 1 -- 172.21.15.49:0/3130199973 shutdown_connections 2023-11-06T02:11:27.188 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.185+0000 7fd8fe63b640 1 -- 172.21.15.49:0/3130199973 >> 172.21.15.49:0/3130199973 conn(0x7fd8f806c0a0 msgr2=0x7fd8f806c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:27.188 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.185+0000 7fd8fe63b640 1 -- 172.21.15.49:0/3130199973 shutdown_connections 2023-11-06T02:11:27.188 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.185+0000 7fd8fe63b640 1 -- 172.21.15.49:0/3130199973 wait complete. 2023-11-06T02:11:27.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.185+0000 7fd8fe63b640 1 Processor -- start 2023-11-06T02:11:27.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.185+0000 7fd8fe63b640 1 -- start start 2023-11-06T02:11:27.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.185+0000 7fd8fe63b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd8e4003f00 0x7fd8f81efcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:27.192 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.185+0000 7fd8fe63b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd8f8150560 0x7fd8f8152c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:27.192 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.185+0000 7fd8fe63b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8f80c2e70 0x7fd8f80c32f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:27.192 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.185+0000 7fd8fe63b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f81f0210 con 0x7fd8f80c2e70 2023-11-06T02:11:27.192 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.185+0000 7fd8fe63b640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f80c3830 con 0x7fd8e4003f00 2023-11-06T02:11:27.192 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.185+0000 7fd8fe63b640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f8151600 con 0x7fd8f8150560 2023-11-06T02:11:27.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.185+0000 7fd8fcbd8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8f80c2e70 0x7fd8f80c32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:27.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.185+0000 7fd8fcbd8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8f80c2e70 0x7fd8f80c32f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49602/0 (socket says 172.21.15.49:49602) 2023-11-06T02:11:27.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.185+0000 7fd8fcbd8640 1 -- 172.21.15.49:0/2877666704 learned_addr learned my addr 172.21.15.49:0/2877666704 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:27.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.185+0000 7fd8f7fff640 1 --2- 172.21.15.49:0/2877666704 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd8e4003f00 0x7fd8f81efcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:27.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.185+0000 7fd8fcbd8640 1 -- 172.21.15.49:0/2877666704 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd8f8150560 msgr2=0x7fd8f8152c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:27.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.185+0000 7fd8fcbd8640 1 --2- 172.21.15.49:0/2877666704 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd8f8150560 0x7fd8f8152c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:27.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.185+0000 7fd8fcbd8640 1 -- 172.21.15.49:0/2877666704 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd8e4003f00 msgr2=0x7fd8f81efcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:27.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.185+0000 7fd8fcbd8640 1 --2- 172.21.15.49:0/2877666704 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd8e4003f00 0x7fd8f81efcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:27.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.185+0000 7fd8fcbd8640 1 -- 172.21.15.49:0/2877666704 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8f8145ee0 con 0x7fd8f80c2e70 2023-11-06T02:11:27.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.189+0000 7fd8fcbd8640 1 --2- 172.21.15.49:0/2877666704 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8f80c2e70 0x7fd8f80c32f0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fd8f80ca5f0 tx=0x7fd8f00115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:27.195 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.189+0000 7fd8f57fa640 1 -- 172.21.15.49:0/2877666704 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f000e070 con 0x7fd8f80c2e70 2023-11-06T02:11:27.195 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.189+0000 7fd8f57fa640 1 -- 172.21.15.49:0/2877666704 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8f0011db0 con 0x7fd8f80c2e70 2023-11-06T02:11:27.195 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.189+0000 7fd8fe63b640 1 -- 172.21.15.49:0/2877666704 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8f81509f0 con 0x7fd8f80c2e70 2023-11-06T02:11:27.195 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.189+0000 7fd8f57fa640 1 -- 172.21.15.49:0/2877666704 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f00039a0 con 0x7fd8f80c2e70 2023-11-06T02:11:27.195 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.189+0000 7fd8fe63b640 1 -- 172.21.15.49:0/2877666704 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8f8150fb0 con 0x7fd8f80c2e70 2023-11-06T02:11:27.195 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.189+0000 7fd8fe63b640 1 -- 172.21.15.49:0/2877666704 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8f80b0de0 con 0x7fd8f80c2e70 2023-11-06T02:11:27.196 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.189+0000 7fd8f57fa640 1 -- 172.21.15.49:0/2877666704 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fd8f0003b00 con 0x7fd8f80c2e70 2023-11-06T02:11:27.196 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.189+0000 7fd8f57fa640 1 --2- 172.21.15.49:0/2877666704 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd8d8065a60 0x7fd8d8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:27.196 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.189+0000 7fd8f57fa640 1 -- 172.21.15.49:0/2877666704 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fd8f0083880 con 0x7fd8f80c2e70 2023-11-06T02:11:27.196 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.189+0000 7fd8f7fff640 1 --2- 172.21.15.49:0/2877666704 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd8d8065a60 0x7fd8d8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:27.196 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.189+0000 7fd8f7fff640 1 --2- 172.21.15.49:0/2877666704 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd8d8065a60 0x7fd8d8067f20 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fd8f814f9a0 tx=0x7fd8e8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:27.197 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.193+0000 7fd8f57fa640 1 -- 172.21.15.49:0/2877666704 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fd8f0051410 con 0x7fd8f80c2e70 2023-11-06T02:11:27.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.353+0000 7fd8fe63b640 1 -- 172.21.15.49:0/2877666704 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd8f8074040 con 0x7fd8f80c2e70 2023-11-06T02:11:27.351 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.353+0000 7fd8f57fa640 1 -- 172.21.15.49:0/2877666704 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) v1 ==== 74+0+14171 (secure 0 0 0) 0x7fd8f0054a30 con 0x7fd8f80c2e70 2023-11-06T02:11:27.351 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:11:27.352 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":61,"fsid":"1a8b035c-6ba4-499a-b876-aea47fd9c0d4","created":"2023-11-06T02:08:06.440857+0000","modified":"2023-11-06T02:11:26.622254+0000","last_up_change":"2023-11-06T02:08:17.982372+0000","last_in_change":"2023-11-06T02:08:08.906631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T02:08:19.252222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T02:08:24.776211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ad55af8-0571-4e6a-8722-b5d429b7496d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6809","nonce":38116},{"type":"v1","addr":"172.21.15.49:6811","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6813","nonce":38116},{"type":"v1","addr":"172.21.15.49:6815","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6821","nonce":38116},{"type":"v1","addr":"172.21.15.49:6823","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6817","nonce":38116},{"type":"v1","addr":"172.21.15.49:6819","nonce":38116}]},"public_addr":"172.21.15.49:6811/38116","cluster_addr":"172.21.15.49:6815/38116","heartbeat_back_addr":"172.21.15.49:6823/38116","heartbeat_front_addr":"172.21.15.49:6819/38116","state":["exists","up"]},{"osd":1,"uuid":"c8b31f85-a757-43ed-9419-351acc46611b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":38113},{"type":"v1","addr":"172.21.15.49:6810","nonce":38113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":38113},{"type":"v1","addr":"172.21.15.49:6814","nonce":38113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":38113},{"type":"v1","addr":"172.21.15.49:6822","nonce":38113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":38113},{"type":"v1","addr":"172.21.15.49:6818","nonce":38113}]},"public_addr":"172.21.15.49:6810/38113","cluster_addr":"172.21.15.49:6814/38113","heartbeat_back_addr":"172.21.15.49:6822/38113","heartbeat_front_addr":"172.21.15.49:6818/38113","state":["exists","up"]},{"osd":2,"uuid":"555b3bf3-0a70-4b91-8efd-e4204b5fc26e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":38114},{"type":"v1","addr":"172.21.15.49:6801","nonce":38114}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":38114},{"type":"v1","addr":"172.21.15.49:6803","nonce":38114}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":38114},{"type":"v1","addr":"172.21.15.49:6807","nonce":38114}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":38114},{"type":"v1","addr":"172.21.15.49:6805","nonce":38114}]},"public_addr":"172.21.15.49:6801/38114","cluster_addr":"172.21.15.49:6803/38114","heartbeat_back_addr":"172.21.15.49:6807/38114","heartbeat_front_addr":"172.21.15.49:6805/38114","state":["exists","up"]},{"osd":3,"uuid":"8d4e5d77-03ef-4b9b-bfcb-5ae93a46795c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":38115},{"type":"v1","addr":"172.21.15.49:6825","nonce":38115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":38115},{"type":"v1","addr":"172.21.15.49:6827","nonce":38115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":38115},{"type":"v1","addr":"172.21.15.49:6831","nonce":38115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":38115},{"type":"v1","addr":"172.21.15.49:6829","nonce":38115}]},"public_addr":"172.21.15.49:6825/38115","cluster_addr":"172.21.15.49:6827/38115","heartbeat_back_addr":"172.21.15.49:6831/38115","heartbeat_front_addr":"172.21.15.49:6829/38115","state":["exists","up"]},{"osd":4,"uuid":"dba03ad4-68a3-4cf3-9218-ac611f02eacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":38417},{"type":"v1","addr":"172.21.15.150:6809","nonce":38417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":38417},{"type":"v1","addr":"172.21.15.150:6811","nonce":38417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":38417},{"type":"v1","addr":"172.21.15.150:6815","nonce":38417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":38417},{"type":"v1","addr":"172.21.15.150:6813","nonce":38417}]},"public_addr":"172.21.15.150:6809/38417","cluster_addr":"172.21.15.150:6811/38417","heartbeat_back_addr":"172.21.15.150:6815/38417","heartbeat_front_addr":"172.21.15.150:6813/38417","state":["exists","up"]},{"osd":5,"uuid":"0df9c303-c255-4640-8e91-01f1c3efe18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":38418},{"type":"v1","addr":"172.21.15.150:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":38418},{"type":"v1","addr":"172.21.15.150:6803","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":38418},{"type":"v1","addr":"172.21.15.150:6807","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":38418},{"type":"v1","addr":"172.21.15.150:6805","nonce":38418}]},"public_addr":"172.21.15.150:6801/38418","cluster_addr":"172.21.15.150:6803/38418","heartbeat_back_addr":"172.21.15.150:6807/38418","heartbeat_front_addr":"172.21.15.150:6805/38418","state":["exists","up"]},{"osd":6,"uuid":"e59fa189-4b70-4c06-9722-c02b90828bf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":38428},{"type":"v1","addr":"172.21.15.150:6825","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":38428},{"type":"v1","addr":"172.21.15.150:6827","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":38428},{"type":"v1","addr":"172.21.15.150:6831","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":38428},{"type":"v1","addr":"172.21.15.150:6829","nonce":38428}]},"public_addr":"172.21.15.150:6825/38428","cluster_addr":"172.21.15.150:6827/38428","heartbeat_back_addr":"172.21.15.150:6831/38428","heartbeat_front_addr":"172.21.15.150:6829/38428","state":["exists","up"]},{"osd":7,"uuid":"a64f57e1-ef98-46bd-b304-385759e95898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":38421},{"type":"v1","addr":"172.21.15.150:6817","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":38421},{"type":"v1","addr":"172.21.15.150:6819","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":38421},{"type":"v1","addr":"172.21.15.150:6823","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":38421},{"type":"v1","addr":"172.21.15.150:6821","nonce":38421}]},"public_addr":"172.21.15.150:6817/38421","cluster_addr":"172.21.15.150:6819/38421","heartbeat_back_addr":"172.21.15.150:6823/38421","heartbeat_front_addr":"172.21.15.150:6821/38421","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.438552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.413280+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.296466+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.424966+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.276953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.230984+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.410235+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.480084+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T02:11:27.353 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.357+0000 7fd8fe63b640 1 -- 172.21.15.49:0/2877666704 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd8d8065a60 msgr2=0x7fd8d8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:27.353 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.357+0000 7fd8fe63b640 1 --2- 172.21.15.49:0/2877666704 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd8d8065a60 0x7fd8d8067f20 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fd8f814f9a0 tx=0x7fd8e8008040 comp rx=0 tx=0).stop 2023-11-06T02:11:27.354 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.357+0000 7fd8fe63b640 1 -- 172.21.15.49:0/2877666704 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8f80c2e70 msgr2=0x7fd8f80c32f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:27.354 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.357+0000 7fd8fe63b640 1 --2- 172.21.15.49:0/2877666704 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8f80c2e70 0x7fd8f80c32f0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fd8f80ca5f0 tx=0x7fd8f00115a0 comp rx=0 tx=0).stop 2023-11-06T02:11:27.354 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.357+0000 7fd8fe63b640 1 -- 172.21.15.49:0/2877666704 shutdown_connections 2023-11-06T02:11:27.354 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.357+0000 7fd8fe63b640 1 --2- 172.21.15.49:0/2877666704 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd8d8065a60 0x7fd8d8067f20 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:27.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.357+0000 7fd8fe63b640 1 --2- 172.21.15.49:0/2877666704 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8f80c2e70 0x7fd8f80c32f0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:27.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.357+0000 7fd8fe63b640 1 --2- 172.21.15.49:0/2877666704 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd8f8150560 0x7fd8f8152c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:27.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.357+0000 7fd8fe63b640 1 --2- 172.21.15.49:0/2877666704 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd8e4003f00 0x7fd8f81efcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:27.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.357+0000 7fd8fe63b640 1 -- 172.21.15.49:0/2877666704 >> 172.21.15.49:0/2877666704 conn(0x7fd8f806c0a0 msgr2=0x7fd8f80731d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:27.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.357+0000 7fd8fe63b640 1 -- 172.21.15.49:0/2877666704 shutdown_connections 2023-11-06T02:11:27.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.357+0000 7fd8fe63b640 1 -- 172.21.15.49:0/2877666704 wait complete. 2023-11-06T02:11:27.371 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T02:08:19.252222+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T02:08:24.776211+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T02:11:27.371 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T02:11:27.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.493+0000 7f1c68fe9640 1 Processor -- start 2023-11-06T02:11:27.493 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.493+0000 7f1c68fe9640 1 -- start start 2023-11-06T02:11:27.493 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.497+0000 7f1c68fe9640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1c64070870 0x7f1c640b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:27.494 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.497+0000 7f1c68fe9640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f1c6406d150 0x7f1c6406d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:27.494 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.497+0000 7f1c68fe9640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c6406db20 con 0x7f1c6415a880 2023-11-06T02:11:27.494 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.497+0000 7f1c68fe9640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c640ae360 con 0x7f1c6406d150 2023-11-06T02:11:27.494 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.497+0000 7f1c68fe9640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c640ae640 con 0x7f1c64070870 2023-11-06T02:11:27.495 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.497+0000 7f1c59d9b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1c64070870 0x7f1c640b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:27.495 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.497+0000 7f1c62d9d640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f1c6406d150 0x7f1c6406d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:27.496 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.497+0000 7f1c62d9d640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f1c6406d150 0x7f1c6406d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.49:60466/0 (socket says 172.21.15.49:60466) 2023-11-06T02:11:27.496 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.497+0000 7f1c62d9d640 1 -- 172.21.15.49:0/2622575276 learned_addr learned my addr 172.21.15.49:0/2622575276 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:27.496 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.497+0000 7f1c6259c640 1 --1- 172.21.15.49:0/2622575276 >> v1:172.21.15.150:6789/0 conn(0x7f1c6415a880 0x7f1c64070160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.49:60014/0 (socket says 172.21.15.49:60014) 2023-11-06T02:11:27.496 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.497+0000 7f1c62d9d640 1 -- 172.21.15.49:0/2622575276 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1c64070870 msgr2=0x7f1c640b15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:27.497 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.497+0000 7f1c62d9d640 1 --2- 172.21.15.49:0/2622575276 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1c64070870 0x7f1c640b15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:27.497 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.497+0000 7f1c62d9d640 1 -- 172.21.15.49:0/2622575276 >> v1:172.21.15.150:6789/0 conn(0x7f1c6415a880 legacy=0x7f1c64070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:27.497 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.497+0000 7f1c62d9d640 1 -- 172.21.15.49:0/2622575276 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c640ae920 con 0x7f1c6406d150 2023-11-06T02:11:27.497 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.497+0000 7f1c59d9b640 1 --2- 172.21.15.49:0/2622575276 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1c64070870 0x7f1c640b15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T02:11:27.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.501+0000 7f1c62d9d640 1 --2- 172.21.15.49:0/2622575276 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c6406d150 0x7f1c6406d5e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c009e50 tx=0x7f1c5c01ca10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=884bbd1a27ab4edd server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:27.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.501+0000 7f1c61d9b640 1 -- 172.21.15.49:0/2622575276 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5c0a57f0 con 0x7f1c6406d150 2023-11-06T02:11:27.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.501+0000 7f1c61d9b640 1 -- 172.21.15.49:0/2622575276 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c5c0a5950 con 0x7f1c6406d150 2023-11-06T02:11:27.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.501+0000 7f1c61d9b640 1 -- 172.21.15.49:0/2622575276 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5c0a5ce0 con 0x7f1c6406d150 2023-11-06T02:11:27.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.501+0000 7f1c68fe9640 1 -- 172.21.15.49:0/2622575276 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c6406d150 msgr2=0x7f1c6406d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:27.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.501+0000 7f1c68fe9640 1 --2- 172.21.15.49:0/2622575276 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c6406d150 0x7f1c6406d5e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c009e50 tx=0x7f1c5c01ca10 comp rx=0 tx=0).stop 2023-11-06T02:11:27.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.501+0000 7f1c68fe9640 1 -- 172.21.15.49:0/2622575276 shutdown_connections 2023-11-06T02:11:27.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.501+0000 7f1c68fe9640 1 --2- 172.21.15.49:0/2622575276 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1c64070870 0x7f1c640b15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:27.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.501+0000 7f1c68fe9640 1 --2- 172.21.15.49:0/2622575276 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c6406d150 0x7f1c6406d5e0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:27.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.501+0000 7f1c68fe9640 1 -- 172.21.15.49:0/2622575276 >> 172.21.15.49:0/2622575276 conn(0x7f1c6406a370 msgr2=0x7f1c640ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:27.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.501+0000 7f1c68fe9640 1 -- 172.21.15.49:0/2622575276 shutdown_connections 2023-11-06T02:11:27.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.501+0000 7f1c68fe9640 1 -- 172.21.15.49:0/2622575276 wait complete. 2023-11-06T02:11:27.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.501+0000 7f1c68fe9640 1 Processor -- start 2023-11-06T02:11:27.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.501+0000 7f1c68fe9640 1 -- start start 2023-11-06T02:11:27.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.501+0000 7f1c68fe9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c6406d150 0x7f1c6413fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:27.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.501+0000 7f1c68fe9640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c64070870 0x7f1c6415c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:27.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.501+0000 7f1c68fe9640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1c6415a880 0x7f1c6414aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:27.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.501+0000 7f1c68fe9640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c640c9360 con 0x7f1c6406d150 2023-11-06T02:11:27.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.501+0000 7f1c68fe9640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c640c9600 con 0x7f1c64070870 2023-11-06T02:11:27.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.501+0000 7f1c68fe9640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c640c98e0 con 0x7f1c6415a880 2023-11-06T02:11:27.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.501+0000 7f1c6259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c6406d150 0x7f1c6413fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:27.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.501+0000 7f1c6259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c6406d150 0x7f1c6413fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49612/0 (socket says 172.21.15.49:49612) 2023-11-06T02:11:27.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.501+0000 7f1c6259c640 1 -- 172.21.15.49:0/195527631 learned_addr learned my addr 172.21.15.49:0/195527631 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:27.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.501+0000 7f1c62d9d640 1 --2- 172.21.15.49:0/195527631 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1c6415a880 0x7f1c6414aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:27.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.501+0000 7f1c59d9b640 1 --2- 172.21.15.49:0/195527631 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c64070870 0x7f1c6415c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:27.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.505+0000 7f1c6259c640 1 -- 172.21.15.49:0/195527631 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1c6415a880 msgr2=0x7f1c6414aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:27.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.505+0000 7f1c6259c640 1 --2- 172.21.15.49:0/195527631 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1c6415a880 0x7f1c6414aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:27.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.505+0000 7f1c6259c640 1 -- 172.21.15.49:0/195527631 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c64070870 msgr2=0x7f1c6415c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:27.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.505+0000 7f1c6259c640 1 --2- 172.21.15.49:0/195527631 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c64070870 0x7f1c6415c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:27.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.505+0000 7f1c6259c640 1 -- 172.21.15.49:0/195527631 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c4c009420 con 0x7f1c6406d150 2023-11-06T02:11:27.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.505+0000 7f1c6259c640 1 --2- 172.21.15.49:0/195527631 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c6406d150 0x7f1c6413fff0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f1c5400c420 tx=0x7f1c5400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:27.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.505+0000 7f1c5b7fe640 1 -- 172.21.15.49:0/195527631 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c540074e0 con 0x7f1c6406d150 2023-11-06T02:11:27.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.505+0000 7f1c5b7fe640 1 -- 172.21.15.49:0/195527631 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c54007640 con 0x7f1c6406d150 2023-11-06T02:11:27.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.505+0000 7f1c68fe9640 1 -- 172.21.15.49:0/195527631 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c5c009b30 con 0x7f1c6406d150 2023-11-06T02:11:27.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.505+0000 7f1c5b7fe640 1 -- 172.21.15.49:0/195527631 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c54004a30 con 0x7f1c6406d150 2023-11-06T02:11:27.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.505+0000 7f1c68fe9640 1 -- 172.21.15.49:0/195527631 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c640c9ea0 con 0x7f1c6406d150 2023-11-06T02:11:27.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.505+0000 7f1c68fe9640 1 -- 172.21.15.49:0/195527631 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c640ca0c0 con 0x7f1c6406d150 2023-11-06T02:11:27.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.505+0000 7f1c5b7fe640 1 -- 172.21.15.49:0/195527631 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f1c54004b90 con 0x7f1c6406d150 2023-11-06T02:11:27.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.505+0000 7f1c5b7fe640 1 --2- 172.21.15.49:0/195527631 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1c44065a60 0x7f1c44067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:27.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.505+0000 7f1c5b7fe640 1 -- 172.21.15.49:0/195527631 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f1c54082950 con 0x7f1c6406d150 2023-11-06T02:11:27.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.505+0000 7f1c59d9b640 1 --2- 172.21.15.49:0/195527631 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1c44065a60 0x7f1c44067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:27.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.505+0000 7f1c59d9b640 1 --2- 172.21.15.49:0/195527631 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1c44065a60 0x7f1c44067f20 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f1c64159610 tx=0x7f1c4c009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:27.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.509+0000 7f1c5b7fe640 1 -- 172.21.15.49:0/195527631 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f1c54050530 con 0x7f1c6406d150 2023-11-06T02:11:27.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.669+0000 7f1c68fe9640 1 -- 172.21.15.49:0/195527631 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1c64074040 con 0x7f1c6406d150 2023-11-06T02:11:27.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.669+0000 7f1c5b7fe640 1 -- 172.21.15.49:0/195527631 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v61) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1c54053b50 con 0x7f1c6406d150 2023-11-06T02:11:27.666 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2023-11-06T02:11:27.668 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.669+0000 7f1c68fe9640 1 -- 172.21.15.49:0/195527631 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1c44065a60 msgr2=0x7f1c44067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:27.668 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.669+0000 7f1c68fe9640 1 --2- 172.21.15.49:0/195527631 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1c44065a60 0x7f1c44067f20 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f1c64159610 tx=0x7f1c4c009190 comp rx=0 tx=0).stop 2023-11-06T02:11:27.668 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.669+0000 7f1c68fe9640 1 -- 172.21.15.49:0/195527631 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c6406d150 msgr2=0x7f1c6413fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:27.668 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.669+0000 7f1c68fe9640 1 --2- 172.21.15.49:0/195527631 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c6406d150 0x7f1c6413fff0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f1c5400c420 tx=0x7f1c5400c8e0 comp rx=0 tx=0).stop 2023-11-06T02:11:27.669 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.669+0000 7f1c68fe9640 1 -- 172.21.15.49:0/195527631 shutdown_connections 2023-11-06T02:11:27.669 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.669+0000 7f1c68fe9640 1 --2- 172.21.15.49:0/195527631 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1c44065a60 0x7f1c44067f20 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:27.669 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.669+0000 7f1c68fe9640 1 --2- 172.21.15.49:0/195527631 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1c6415a880 0x7f1c6414aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:27.669 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.669+0000 7f1c68fe9640 1 --2- 172.21.15.49:0/195527631 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c64070870 0x7f1c6415c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:27.669 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.669+0000 7f1c68fe9640 1 --2- 172.21.15.49:0/195527631 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c6406d150 0x7f1c6413fff0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:27.670 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.669+0000 7f1c68fe9640 1 -- 172.21.15.49:0/195527631 >> 172.21.15.49:0/195527631 conn(0x7f1c6406a370 msgr2=0x7f1c640b3ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:27.670 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.669+0000 7f1c68fe9640 1 -- 172.21.15.49:0/195527631 shutdown_connections 2023-11-06T02:11:27.670 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.669+0000 7f1c68fe9640 1 -- 172.21.15.49:0/195527631 wait complete. 2023-11-06T02:11:27.685 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T02:11:27.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.813+0000 7fa183344640 1 Processor -- start 2023-11-06T02:11:27.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.813+0000 7fa183344640 1 -- start start 2023-11-06T02:11:27.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.813+0000 7fa183344640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa17c161260 0x7fa17c070160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:27.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.813+0000 7fa183344640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fa17c0c8790 0x7fa17c0706a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:27.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.813+0000 7fa183344640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa17c0c91f0 con 0x7fa17c0c93f0 2023-11-06T02:11:27.812 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.813+0000 7fa183344640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa17c06c580 con 0x7fa17c161260 2023-11-06T02:11:27.812 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.813+0000 7fa183344640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa17c06c230 con 0x7fa17c0c8790 2023-11-06T02:11:27.812 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.813+0000 7fa1810e0640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa17c161260 0x7fa17c070160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:27.812 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.813+0000 7fa1810e0640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa17c161260 0x7fa17c070160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49620/0 (socket says 172.21.15.49:49620) 2023-11-06T02:11:27.812 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.813+0000 7fa1810e0640 1 -- 172.21.15.49:0/1321078856 learned_addr learned my addr 172.21.15.49:0/1321078856 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:27.812 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.813+0000 7fa1808df640 1 --2- 172.21.15.49:0/1321078856 >> v2:172.21.15.150:3300/0 conn(0x7fa17c0c8790 0x7fa17c0706a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:27.813 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.813+0000 7fa1810e0640 1 -- 172.21.15.49:0/1321078856 >> v2:172.21.15.150:3300/0 conn(0x7fa17c0c8790 msgr2=0x7fa17c0706a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:27.813 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.813+0000 7fa1810e0640 1 --2- 172.21.15.49:0/1321078856 >> v2:172.21.15.150:3300/0 conn(0x7fa17c0c8790 0x7fa17c0706a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:27.813 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.813+0000 7fa1810e0640 1 -- 172.21.15.49:0/1321078856 >> v1:172.21.15.150:6789/0 conn(0x7fa17c0c93f0 legacy=0x7fa17c0af090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:27.813 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.813+0000 7fa1810e0640 1 -- 172.21.15.49:0/1321078856 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa17c0b02d0 con 0x7fa17c161260 2023-11-06T02:11:27.814 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.817+0000 7fa1810e0640 1 --2- 172.21.15.49:0/1321078856 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa17c161260 0x7fa17c070160 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fa17c06e400 tx=0x7fa17401c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82927ba0339cedba server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:27.814 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.817+0000 7fa17bfff640 1 -- 172.21.15.49:0/1321078856 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1740a5680 con 0x7fa17c161260 2023-11-06T02:11:27.815 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.817+0000 7fa17bfff640 1 -- 172.21.15.49:0/1321078856 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1740a57e0 con 0x7fa17c161260 2023-11-06T02:11:27.815 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.817+0000 7fa17bfff640 1 -- 172.21.15.49:0/1321078856 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1740a5b70 con 0x7fa17c161260 2023-11-06T02:11:27.816 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.817+0000 7fa183344640 1 -- 172.21.15.49:0/1321078856 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa17c161260 msgr2=0x7fa17c070160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:27.816 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.817+0000 7fa183344640 1 --2- 172.21.15.49:0/1321078856 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa17c161260 0x7fa17c070160 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fa17c06e400 tx=0x7fa17401c8b0 comp rx=0 tx=0).stop 2023-11-06T02:11:27.816 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.817+0000 7fa183344640 1 -- 172.21.15.49:0/1321078856 shutdown_connections 2023-11-06T02:11:27.816 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.817+0000 7fa183344640 1 --2- 172.21.15.49:0/1321078856 >> v2:172.21.15.150:3300/0 conn(0x7fa17c0c8790 0x7fa17c0706a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:27.816 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.817+0000 7fa183344640 1 --2- 172.21.15.49:0/1321078856 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa17c161260 0x7fa17c070160 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:27.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.817+0000 7fa183344640 1 -- 172.21.15.49:0/1321078856 >> 172.21.15.49:0/1321078856 conn(0x7fa17c06a370 msgr2=0x7fa17c0ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:27.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.817+0000 7fa183344640 1 -- 172.21.15.49:0/1321078856 shutdown_connections 2023-11-06T02:11:27.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.817+0000 7fa183344640 1 -- 172.21.15.49:0/1321078856 wait complete. 2023-11-06T02:11:27.820 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.817+0000 7fa183344640 1 Processor -- start 2023-11-06T02:11:27.820 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.817+0000 7fa183344640 1 -- start start 2023-11-06T02:11:27.820 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.821+0000 7fa183344640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa17c0c8790 0x7fa17c155e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:27.820 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.821+0000 7fa183344640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa17c0c93f0 0x7fa17c162500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:27.820 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.821+0000 7fa183344640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa17c161260 0x7fa17c156e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:27.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.821+0000 7fa183344640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa17c0cdbd0 con 0x7fa17c0c8790 2023-11-06T02:11:27.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.821+0000 7fa183344640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa17c155390 con 0x7fa17c161260 2023-11-06T02:11:27.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.821+0000 7fa183344640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa17c155670 con 0x7fa17c0c93f0 2023-11-06T02:11:27.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.821+0000 7fa1810e0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa17c0c8790 0x7fa17c155e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:27.822 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.821+0000 7fa1810e0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa17c0c8790 0x7fa17c155e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49628/0 (socket says 172.21.15.49:49628) 2023-11-06T02:11:27.822 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.821+0000 7fa1810e0640 1 -- 172.21.15.49:0/2045410530 learned_addr learned my addr 172.21.15.49:0/2045410530 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:27.822 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.821+0000 7fa1808df640 1 --2- 172.21.15.49:0/2045410530 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa17c0c93f0 0x7fa17c162500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:27.822 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.821+0000 7fa1818e1640 1 --2- 172.21.15.49:0/2045410530 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa17c161260 0x7fa17c156e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:27.822 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.821+0000 7fa1810e0640 1 -- 172.21.15.49:0/2045410530 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa17c0c93f0 msgr2=0x7fa17c162500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:27.823 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.821+0000 7fa1810e0640 1 --2- 172.21.15.49:0/2045410530 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa17c0c93f0 0x7fa17c162500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:27.823 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.821+0000 7fa1810e0640 1 -- 172.21.15.49:0/2045410530 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa17c161260 msgr2=0x7fa17c156e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:27.823 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.821+0000 7fa1810e0640 1 --2- 172.21.15.49:0/2045410530 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa17c161260 0x7fa17c156e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:27.823 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.821+0000 7fa1810e0640 1 -- 172.21.15.49:0/2045410530 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa174009b30 con 0x7fa17c0c8790 2023-11-06T02:11:27.823 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.821+0000 7fa1818e1640 1 --2- 172.21.15.49:0/2045410530 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa17c161260 0x7fa17c156e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:11:27.824 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.821+0000 7fa1808df640 1 --2- 172.21.15.49:0/2045410530 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa17c0c93f0 0x7fa17c162500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:11:27.824 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.821+0000 7fa1810e0640 1 --2- 172.21.15.49:0/2045410530 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa17c0c8790 0x7fa17c155e70 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fa1740af040 tx=0x7fa174019650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:27.824 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.821+0000 7fa179ffb640 1 -- 172.21.15.49:0/2045410530 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1740b8670 con 0x7fa17c0c8790 2023-11-06T02:11:27.824 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.821+0000 7fa179ffb640 1 -- 172.21.15.49:0/2045410530 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1740b87f0 con 0x7fa17c0c8790 2023-11-06T02:11:27.824 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.821+0000 7fa179ffb640 1 -- 172.21.15.49:0/2045410530 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1740b8ba0 con 0x7fa17c0c8790 2023-11-06T02:11:27.824 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.821+0000 7fa183344640 1 -- 172.21.15.49:0/2045410530 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa17c155950 con 0x7fa17c0c8790 2023-11-06T02:11:27.825 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.821+0000 7fa183344640 1 -- 172.21.15.49:0/2045410530 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa17c0cdeb0 con 0x7fa17c0c8790 2023-11-06T02:11:27.825 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.821+0000 7fa183344640 1 -- 172.21.15.49:0/2045410530 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa17c155b40 con 0x7fa17c0c8790 2023-11-06T02:11:27.825 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.821+0000 7fa179ffb640 1 -- 172.21.15.49:0/2045410530 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fa1740b8d00 con 0x7fa17c0c8790 2023-11-06T02:11:27.826 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.821+0000 7fa179ffb640 1 --2- 172.21.15.49:0/2045410530 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa168065a90 0x7fa168067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:27.826 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.821+0000 7fa179ffb640 1 -- 172.21.15.49:0/2045410530 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fa17411e010 con 0x7fa17c0c8790 2023-11-06T02:11:27.826 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.825+0000 7fa1808df640 1 --2- 172.21.15.49:0/2045410530 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa168065a90 0x7fa168067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:27.827 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.825+0000 7fa1808df640 1 --2- 172.21.15.49:0/2045410530 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa168065a90 0x7fa168067f50 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fa17c15e070 tx=0x7fa170008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:27.827 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.829+0000 7fa179ffb640 1 -- 172.21.15.49:0/2045410530 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fa1740ebc50 con 0x7fa17c0c8790 2023-11-06T02:11:27.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.977+0000 7fa183344640 1 -- 172.21.15.49:0/2045410530 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa17c074040 con 0x7fa17c0c8790 2023-11-06T02:11:27.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.977+0000 7fa179ffb640 1 -- 172.21.15.49:0/2045410530 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v61) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa174009510 con 0x7fa17c0c8790 2023-11-06T02:11:27.976 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 8 2023-11-06T02:11:27.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.981+0000 7fa183344640 1 -- 172.21.15.49:0/2045410530 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa168065a90 msgr2=0x7fa168067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:27.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.981+0000 7fa183344640 1 --2- 172.21.15.49:0/2045410530 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa168065a90 0x7fa168067f50 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fa17c15e070 tx=0x7fa170008040 comp rx=0 tx=0).stop 2023-11-06T02:11:27.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.981+0000 7fa183344640 1 -- 172.21.15.49:0/2045410530 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa17c0c8790 msgr2=0x7fa17c155e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:27.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.981+0000 7fa183344640 1 --2- 172.21.15.49:0/2045410530 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa17c0c8790 0x7fa17c155e70 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fa1740af040 tx=0x7fa174019650 comp rx=0 tx=0).stop 2023-11-06T02:11:27.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.981+0000 7fa183344640 1 -- 172.21.15.49:0/2045410530 shutdown_connections 2023-11-06T02:11:27.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.981+0000 7fa183344640 1 --2- 172.21.15.49:0/2045410530 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa168065a90 0x7fa168067f50 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:27.980 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.981+0000 7fa183344640 1 --2- 172.21.15.49:0/2045410530 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa17c161260 0x7fa17c156e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:27.980 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.981+0000 7fa183344640 1 --2- 172.21.15.49:0/2045410530 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa17c0c93f0 0x7fa17c162500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:27.980 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.981+0000 7fa183344640 1 --2- 172.21.15.49:0/2045410530 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa17c0c8790 0x7fa17c155e70 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:27.980 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.981+0000 7fa183344640 1 -- 172.21.15.49:0/2045410530 >> 172.21.15.49:0/2045410530 conn(0x7fa17c06a370 msgr2=0x7fa17c070060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:27.980 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.981+0000 7fa183344640 1 -- 172.21.15.49:0/2045410530 shutdown_connections 2023-11-06T02:11:27.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:27.981+0000 7fa183344640 1 -- 172.21.15.49:0/2045410530 wait complete. 2023-11-06T02:11:27.994 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-06T02:11:28.116 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.117+0000 7f11ebabb640 1 Processor -- start 2023-11-06T02:11:28.117 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.117+0000 7f11ebabb640 1 -- start start 2023-11-06T02:11:28.119 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.121+0000 7f11ebabb640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f11e40cbc10 0x7f11e4071120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:28.119 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.121+0000 7f11ebabb640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f11e40cf2a0 0x7f11e415d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:28.119 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.121+0000 7f11ebabb640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11e406b1b0 con 0x7f11e40b5290 2023-11-06T02:11:28.119 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.121+0000 7f11ebabb640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11e40cbff0 con 0x7f11e40cbc10 2023-11-06T02:11:28.120 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.121+0000 7f11ebabb640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11e406b490 con 0x7f11e40cf2a0 2023-11-06T02:11:28.120 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.121+0000 7f11e9857640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f11e40cbc10 0x7f11e4071120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:28.120 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.121+0000 7f11e9857640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f11e40cbc10 0x7f11e4071120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49644/0 (socket says 172.21.15.49:49644) 2023-11-06T02:11:28.120 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.121+0000 7f11e9857640 1 -- 172.21.15.49:0/2127786407 learned_addr learned my addr 172.21.15.49:0/2127786407 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:28.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.121+0000 7f11ea058640 1 --2- 172.21.15.49:0/2127786407 >> v2:172.21.15.150:3300/0 conn(0x7f11e40cf2a0 0x7f11e415d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:28.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.121+0000 7f11e9857640 1 -- 172.21.15.49:0/2127786407 >> v2:172.21.15.150:3300/0 conn(0x7f11e40cf2a0 msgr2=0x7f11e415d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:28.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.121+0000 7f11e9857640 1 --2- 172.21.15.49:0/2127786407 >> v2:172.21.15.150:3300/0 conn(0x7f11e40cf2a0 0x7f11e415d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:28.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.121+0000 7f11e9857640 1 -- 172.21.15.49:0/2127786407 >> v1:172.21.15.150:6789/0 conn(0x7f11e40b5290 legacy=0x7f11e40ceb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:28.122 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.121+0000 7f11e9857640 1 -- 172.21.15.49:0/2127786407 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11e406b770 con 0x7f11e40cbc10 2023-11-06T02:11:28.122 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.121+0000 7f11ea058640 1 --2- 172.21.15.49:0/2127786407 >> v2:172.21.15.150:3300/0 conn(0x7f11e40cf2a0 0x7f11e415d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:11:28.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.121+0000 7f11e9857640 1 --2- 172.21.15.49:0/2127786407 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11e40cbc10 0x7f11e4071120 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f11e40c9e10 tx=0x7f11dc01c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40d44365dbd9b4ef server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:28.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.121+0000 7f11e8855640 1 -- 172.21.15.49:0/2127786407 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11dc0a5660 con 0x7f11e40cbc10 2023-11-06T02:11:28.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.121+0000 7f11e8855640 1 -- 172.21.15.49:0/2127786407 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11dc0a57c0 con 0x7f11e40cbc10 2023-11-06T02:11:28.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.121+0000 7f11e8855640 1 -- 172.21.15.49:0/2127786407 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11dc0a5b50 con 0x7f11e40cbc10 2023-11-06T02:11:28.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.125+0000 7f11ebabb640 1 -- 172.21.15.49:0/2127786407 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11e40cbc10 msgr2=0x7f11e4071120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:28.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.125+0000 7f11ebabb640 1 --2- 172.21.15.49:0/2127786407 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11e40cbc10 0x7f11e4071120 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f11e40c9e10 tx=0x7f11dc01c990 comp rx=0 tx=0).stop 2023-11-06T02:11:28.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.125+0000 7f11ebabb640 1 -- 172.21.15.49:0/2127786407 shutdown_connections 2023-11-06T02:11:28.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.125+0000 7f11ebabb640 1 --2- 172.21.15.49:0/2127786407 >> v2:172.21.15.150:3300/0 conn(0x7f11e40cf2a0 0x7f11e415d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:28.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.125+0000 7f11ebabb640 1 --2- 172.21.15.49:0/2127786407 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11e40cbc10 0x7f11e4071120 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:28.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.125+0000 7f11ebabb640 1 -- 172.21.15.49:0/2127786407 >> 172.21.15.49:0/2127786407 conn(0x7f11e40cc140 msgr2=0x7f11e40cc550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:28.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.125+0000 7f11ebabb640 1 -- 172.21.15.49:0/2127786407 shutdown_connections 2023-11-06T02:11:28.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.125+0000 7f11ebabb640 1 -- 172.21.15.49:0/2127786407 wait complete. 2023-11-06T02:11:28.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.125+0000 7f11ebabb640 1 Processor -- start 2023-11-06T02:11:28.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.125+0000 7f11ebabb640 1 -- start start 2023-11-06T02:11:28.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.125+0000 7f11ebabb640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11e40b5290 0x7f11e40c8960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:28.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.125+0000 7f11ebabb640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11e40cbc10 0x7f11e415a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:28.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.125+0000 7f11ebabb640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f11e40cf2a0 0x7f11e40c3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:28.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.125+0000 7f11ebabb640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e415a5f0 con 0x7f11e40cbc10 2023-11-06T02:11:28.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.125+0000 7f11ebabb640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e41f56b0 con 0x7f11e40b5290 2023-11-06T02:11:28.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.125+0000 7f11ebabb640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11e41f5990 con 0x7f11e40cf2a0 2023-11-06T02:11:28.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.125+0000 7f11e9056640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11e40cbc10 0x7f11e415a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:28.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.125+0000 7f11e9056640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11e40cbc10 0x7f11e415a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49646/0 (socket says 172.21.15.49:49646) 2023-11-06T02:11:28.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.125+0000 7f11e9056640 1 -- 172.21.15.49:0/717710186 learned_addr learned my addr 172.21.15.49:0/717710186 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:28.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.125+0000 7f11e9857640 1 --2- 172.21.15.49:0/717710186 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11e40b5290 0x7f11e40c8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:28.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.125+0000 7f11ea058640 1 --2- 172.21.15.49:0/717710186 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f11e40cf2a0 0x7f11e40c3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:28.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.125+0000 7f11e9056640 1 -- 172.21.15.49:0/717710186 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f11e40cf2a0 msgr2=0x7f11e40c3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:28.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.125+0000 7f11e9056640 1 --2- 172.21.15.49:0/717710186 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f11e40cf2a0 0x7f11e40c3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:28.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.125+0000 7f11e9056640 1 -- 172.21.15.49:0/717710186 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11e40b5290 msgr2=0x7f11e40c8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:28.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.125+0000 7f11e9056640 1 --2- 172.21.15.49:0/717710186 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11e40b5290 0x7f11e40c8960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:28.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.125+0000 7f11e9056640 1 -- 172.21.15.49:0/717710186 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11dc009b30 con 0x7f11e40cbc10 2023-11-06T02:11:28.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.125+0000 7f11e9056640 1 --2- 172.21.15.49:0/717710186 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11e40cbc10 0x7f11e415a0b0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f11d400c450 tx=0x7f11d400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:28.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.125+0000 7f11da7fc640 1 -- 172.21.15.49:0/717710186 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11d4012830 con 0x7f11e40cbc10 2023-11-06T02:11:28.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.125+0000 7f11da7fc640 1 -- 172.21.15.49:0/717710186 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11d40129b0 con 0x7f11e40cbc10 2023-11-06T02:11:28.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.125+0000 7f11ebabb640 1 -- 172.21.15.49:0/717710186 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11e41f5c70 con 0x7f11e40cbc10 2023-11-06T02:11:28.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.125+0000 7f11da7fc640 1 -- 172.21.15.49:0/717710186 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11d4012d60 con 0x7f11e40cbc10 2023-11-06T02:11:28.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.125+0000 7f11ebabb640 1 -- 172.21.15.49:0/717710186 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11e41f6230 con 0x7f11e40cbc10 2023-11-06T02:11:28.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.125+0000 7f11ebabb640 1 -- 172.21.15.49:0/717710186 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11e4067140 con 0x7f11e40cbc10 2023-11-06T02:11:28.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.129+0000 7f11da7fc640 1 -- 172.21.15.49:0/717710186 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f11d401c460 con 0x7f11e40cbc10 2023-11-06T02:11:28.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.129+0000 7f11da7fc640 1 --2- 172.21.15.49:0/717710186 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f11c8065a60 0x7f11c8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:28.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.129+0000 7f11da7fc640 1 -- 172.21.15.49:0/717710186 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f11d4082db0 con 0x7f11e40cbc10 2023-11-06T02:11:28.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.129+0000 7f11e9857640 1 --2- 172.21.15.49:0/717710186 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f11c8065a60 0x7f11c8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:28.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.129+0000 7f11e9857640 1 --2- 172.21.15.49:0/717710186 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f11c8065a60 0x7f11c8067f20 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f11dc0ae000 tx=0x7f11dc0a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:28.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.133+0000 7f11da7fc640 1 -- 172.21.15.49:0/717710186 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f11d40509f0 con 0x7f11e40cbc10 2023-11-06T02:11:28.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.289+0000 7f11ebabb640 1 -- 172.21.15.49:0/717710186 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f11e40c2f40 con 0x7f11e40cbc10 2023-11-06T02:11:28.286 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.289+0000 7f11da7fc640 1 -- 172.21.15.49:0/717710186 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v61) v1 ==== 84+0+126 (secure 0 0 0) 0x7f11d4054010 con 0x7f11e40cbc10 2023-11-06T02:11:28.286 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:11:28.287 INFO:teuthology.orchestra.run.smithi049.stdout:[ 2023-11-06T02:11:28.287 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:11:28.287 INFO:teuthology.orchestra.run.smithi049.stdout: "poolnum": 1, 2023-11-06T02:11:28.287 INFO:teuthology.orchestra.run.smithi049.stdout: "poolname": ".mgr" 2023-11-06T02:11:28.287 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:11:28.288 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:11:28.288 INFO:teuthology.orchestra.run.smithi049.stdout: "poolnum": 2, 2023-11-06T02:11:28.288 INFO:teuthology.orchestra.run.smithi049.stdout: "poolname": "rbd" 2023-11-06T02:11:28.288 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:11:28.289 INFO:teuthology.orchestra.run.smithi049.stdout:] 2023-11-06T02:11:28.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.289+0000 7f11ebabb640 1 -- 172.21.15.49:0/717710186 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f11c8065a60 msgr2=0x7f11c8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:28.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.289+0000 7f11ebabb640 1 --2- 172.21.15.49:0/717710186 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f11c8065a60 0x7f11c8067f20 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f11dc0ae000 tx=0x7f11dc0a7070 comp rx=0 tx=0).stop 2023-11-06T02:11:28.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.289+0000 7f11ebabb640 1 -- 172.21.15.49:0/717710186 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11e40cbc10 msgr2=0x7f11e415a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:28.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.289+0000 7f11ebabb640 1 --2- 172.21.15.49:0/717710186 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11e40cbc10 0x7f11e415a0b0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f11d400c450 tx=0x7f11d400c910 comp rx=0 tx=0).stop 2023-11-06T02:11:28.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.289+0000 7f11ebabb640 1 -- 172.21.15.49:0/717710186 shutdown_connections 2023-11-06T02:11:28.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.289+0000 7f11ebabb640 1 --2- 172.21.15.49:0/717710186 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f11c8065a60 0x7f11c8067f20 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:28.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.289+0000 7f11ebabb640 1 --2- 172.21.15.49:0/717710186 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11e40cbc10 0x7f11e415a0b0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:28.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.289+0000 7f11ebabb640 1 --2- 172.21.15.49:0/717710186 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f11e40cf2a0 0x7f11e40c3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:28.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.289+0000 7f11ebabb640 1 --2- 172.21.15.49:0/717710186 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11e40b5290 0x7f11e40c8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:28.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.289+0000 7f11ebabb640 1 -- 172.21.15.49:0/717710186 >> 172.21.15.49:0/717710186 conn(0x7f11e40cc140 msgr2=0x7f11e40c5960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:28.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.289+0000 7f11ebabb640 1 -- 172.21.15.49:0/717710186 shutdown_connections 2023-11-06T02:11:28.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.289+0000 7f11ebabb640 1 -- 172.21.15.49:0/717710186 wait complete. 2023-11-06T02:11:28.306 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-06T02:11:28.306 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-06T02:11:28.430 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.433+0000 7ff6522fe640 1 Processor -- start 2023-11-06T02:11:28.430 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.433+0000 7ff6522fe640 1 -- start start 2023-11-06T02:11:28.433 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.433+0000 7ff6522fe640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff64c159c40 0x7ff64c0caa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:28.433 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.433+0000 7ff6522fe640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff64c071e80 con 0x7ff64c06a3b0 2023-11-06T02:11:28.433 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.433+0000 7ff6522fe640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff64c072160 con 0x7ff64c069fd0 2023-11-06T02:11:28.434 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.433+0000 7ff6522fe640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff64c0ca220 con 0x7ff64c159c40 2023-11-06T02:11:28.434 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.433+0000 7ff64b7fe640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7ff64c06a3b0 0x7ff64c159530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:46010/0 (socket says 172.21.15.49:46010) 2023-11-06T02:11:28.434 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.433+0000 7ff64b7fe640 1 -- 172.21.15.49:0/762372889 learned_addr learned my addr 172.21.15.49:0/762372889 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:11:28.434 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.433+0000 7ff65089b640 1 --2- 172.21.15.49:0/762372889 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff64c159c40 0x7ff64c0caa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:28.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.433+0000 7ff64affd640 1 -- 172.21.15.49:0/762372889 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3762267161 0 0) 0x7ff64c071e80 con 0x7ff64c06a3b0 2023-11-06T02:11:28.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.433+0000 7ff64affd640 1 -- 172.21.15.49:0/762372889 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff638005740 con 0x7ff64c06a3b0 2023-11-06T02:11:28.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.433+0000 7ff64affd640 1 -- 172.21.15.49:0/762372889 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3742553028 0 0) 0x7ff638005740 con 0x7ff64c06a3b0 2023-11-06T02:11:28.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.433+0000 7ff64affd640 1 -- 172.21.15.49:0/762372889 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff64c159c40 msgr2=0x7ff64c0caa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:28.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.433+0000 7ff64affd640 1 --2- 172.21.15.49:0/762372889 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff64c159c40 0x7ff64c0caa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:28.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.433+0000 7ff64affd640 1 -- 172.21.15.49:0/762372889 >> v1:172.21.15.150:6789/0 conn(0x7ff64c069fd0 legacy=0x7ff64c0715c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:28.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.433+0000 7ff64affd640 1 -- 172.21.15.49:0/762372889 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff64c0ca500 con 0x7ff64c06a3b0 2023-11-06T02:11:28.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.433+0000 7ff64affd640 1 -- 172.21.15.49:0/762372889 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7ff634001680 con 0x7ff64c06a3b0 2023-11-06T02:11:28.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.433+0000 7ff64affd640 1 -- 172.21.15.49:0/762372889 >> v1:172.21.15.49:6789/0 conn(0x7ff64c06a3b0 legacy=0x7ff64c159530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:28.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.433+0000 7ff64affd640 1 --2- 172.21.15.49:0/762372889 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff638005bc0 0x7ff638005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:28.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.433+0000 7ff64affd640 1 --2- 172.21.15.49:0/762372889 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6380069e0 0x7ff638006e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:28.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.433+0000 7ff64affd640 1 --2- 172.21.15.49:0/762372889 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff64c159c40 0x7ff6380073a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:28.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.433+0000 7ff64affd640 1 -- 172.21.15.49:0/762372889 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff64c0ca220 con 0x7ff6380069e0 2023-11-06T02:11:28.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.433+0000 7ff64affd640 1 -- 172.21.15.49:0/762372889 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff63800dc50 con 0x7ff638005bc0 2023-11-06T02:11:28.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.433+0000 7ff64affd640 1 -- 172.21.15.49:0/762372889 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff63800de10 con 0x7ff64c159c40 2023-11-06T02:11:28.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.433+0000 7ff64b7fe640 1 --2- 172.21.15.49:0/762372889 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6380069e0 0x7ff638006e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:28.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.433+0000 7ff64b7fe640 1 -- 172.21.15.49:0/762372889 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff64c159c40 msgr2=0x7ff6380073a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:28.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.433+0000 7ff65089b640 1 --2- 172.21.15.49:0/762372889 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff64c159c40 0x7ff6380073a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:28.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.433+0000 7ff64bfff640 1 --2- 172.21.15.49:0/762372889 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff638005bc0 0x7ff638005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:28.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.433+0000 7ff64b7fe640 1 --2- 172.21.15.49:0/762372889 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff64c159c40 0x7ff6380073a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:28.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.433+0000 7ff64b7fe640 1 -- 172.21.15.49:0/762372889 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff638005bc0 msgr2=0x7ff638005fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:28.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.437+0000 7ff64b7fe640 1 --2- 172.21.15.49:0/762372889 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff638005bc0 0x7ff638005fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:28.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.437+0000 7ff64b7fe640 1 -- 172.21.15.49:0/762372889 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff638005740 con 0x7ff6380069e0 2023-11-06T02:11:28.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.437+0000 7ff64bfff640 1 --2- 172.21.15.49:0/762372889 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff638005bc0 0x7ff638005fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:11:28.443 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.437+0000 7ff64b7fe640 1 --2- 172.21.15.49:0/762372889 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6380069e0 0x7ff638006e60 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7ff63400c210 tx=0x7ff63401f350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=193ecd5cef3e4f6e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:28.443 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.437+0000 7ff64affd640 1 -- 172.21.15.49:0/762372889 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff634005ae0 con 0x7ff6380069e0 2023-11-06T02:11:28.443 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.437+0000 7ff64affd640 1 -- 172.21.15.49:0/762372889 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6340ab070 con 0x7ff6380069e0 2023-11-06T02:11:28.443 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.437+0000 7ff64affd640 1 -- 172.21.15.49:0/762372889 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff634007070 con 0x7ff6380069e0 2023-11-06T02:11:28.444 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.437+0000 7ff6522fe640 1 -- 172.21.15.49:0/762372889 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6380069e0 msgr2=0x7ff638006e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:28.444 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.437+0000 7ff6522fe640 1 --2- 172.21.15.49:0/762372889 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6380069e0 0x7ff638006e60 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7ff63400c210 tx=0x7ff63401f350 comp rx=0 tx=0).stop 2023-11-06T02:11:28.444 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.437+0000 7ff65089b640 1 -- 172.21.15.49:0/762372889 reap_dead start 2023-11-06T02:11:28.444 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.437+0000 7ff6522fe640 1 -- 172.21.15.49:0/762372889 shutdown_connections 2023-11-06T02:11:28.444 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.437+0000 7ff6522fe640 1 -- 172.21.15.49:0/762372889 >> 172.21.15.49:0/762372889 conn(0x7ff64c0ce610 msgr2=0x7ff64c0cea20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:28.445 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.437+0000 7ff6522fe640 1 -- 172.21.15.49:0/762372889 shutdown_connections 2023-11-06T02:11:28.445 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.437+0000 7ff6522fe640 1 -- 172.21.15.49:0/762372889 wait complete. 2023-11-06T02:11:28.445 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.441+0000 7ff6522fe640 1 Processor -- start 2023-11-06T02:11:28.445 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.441+0000 7ff6522fe640 1 -- start start 2023-11-06T02:11:28.445 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.441+0000 7ff6522fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6380069e0 0x7ff64c0aee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:28.445 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.441+0000 7ff6522fe640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff64c0c7360 0x7ff64c0c77e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:28.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.441+0000 7ff6522fe640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff64c1e3740 0x7ff64c1e3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:28.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.441+0000 7ff6522fe640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff64c1e4100 con 0x7ff6380069e0 2023-11-06T02:11:28.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.441+0000 7ff6522fe640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff64c0c7ed0 con 0x7ff64c0c7360 2023-11-06T02:11:28.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.441+0000 7ff6522fe640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff64c0c81b0 con 0x7ff64c1e3740 2023-11-06T02:11:28.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.441+0000 7ff64bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6380069e0 0x7ff64c0aee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:28.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.441+0000 7ff64bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6380069e0 0x7ff64c0aee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49664/0 (socket says 172.21.15.49:49664) 2023-11-06T02:11:28.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.441+0000 7ff64bfff640 1 -- 172.21.15.49:0/3441845316 learned_addr learned my addr 172.21.15.49:0/3441845316 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:28.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.441+0000 7ff64b7fe640 1 --2- 172.21.15.49:0/3441845316 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff64c0c7360 0x7ff64c0c77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:28.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.441+0000 7ff65089b640 1 --2- 172.21.15.49:0/3441845316 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff64c1e3740 0x7ff64c1e3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:28.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.441+0000 7ff64bfff640 1 -- 172.21.15.49:0/3441845316 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff64c1e3740 msgr2=0x7ff64c1e3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:28.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.441+0000 7ff64bfff640 1 --2- 172.21.15.49:0/3441845316 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff64c1e3740 0x7ff64c1e3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:28.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.441+0000 7ff64bfff640 1 -- 172.21.15.49:0/3441845316 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff64c0c7360 msgr2=0x7ff64c0c77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:28.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.441+0000 7ff64bfff640 1 --2- 172.21.15.49:0/3441845316 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff64c0c7360 0x7ff64c0c77e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:28.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.441+0000 7ff64bfff640 1 -- 172.21.15.49:0/3441845316 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff64c0c8490 con 0x7ff6380069e0 2023-11-06T02:11:28.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.441+0000 7ff64b7fe640 1 --2- 172.21.15.49:0/3441845316 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff64c0c7360 0x7ff64c0c77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:11:28.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.441+0000 7ff65089b640 1 --2- 172.21.15.49:0/3441845316 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff64c1e3740 0x7ff64c1e3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:11:28.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.441+0000 7ff64bfff640 1 --2- 172.21.15.49:0/3441845316 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6380069e0 0x7ff64c0aee90 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7ff64400db10 tx=0x7ff6440020f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:28.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.441+0000 7ff648ff9640 1 -- 172.21.15.49:0/3441845316 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff644017070 con 0x7ff6380069e0 2023-11-06T02:11:28.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.441+0000 7ff648ff9640 1 -- 172.21.15.49:0/3441845316 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff644003bb0 con 0x7ff6380069e0 2023-11-06T02:11:28.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.441+0000 7ff6522fe640 1 -- 172.21.15.49:0/3441845316 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff64c0c8770 con 0x7ff6380069e0 2023-11-06T02:11:28.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.441+0000 7ff648ff9640 1 -- 172.21.15.49:0/3441845316 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff644006b60 con 0x7ff6380069e0 2023-11-06T02:11:28.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.441+0000 7ff6522fe640 1 -- 172.21.15.49:0/3441845316 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff64c0af3d0 con 0x7ff6380069e0 2023-11-06T02:11:28.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.441+0000 7ff6522fe640 1 -- 172.21.15.49:0/3441845316 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff64c1eec30 con 0x7ff6380069e0 2023-11-06T02:11:28.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.441+0000 7ff648ff9640 1 -- 172.21.15.49:0/3441845316 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7ff644006cc0 con 0x7ff6380069e0 2023-11-06T02:11:28.451 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.445+0000 7ff648ff9640 1 --2- 172.21.15.49:0/3441845316 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff62c065a60 0x7ff62c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:28.451 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.445+0000 7ff648ff9640 1 -- 172.21.15.49:0/3441845316 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5210+0+0 (secure 0 0 0) 0x7ff644082c90 con 0x7ff6380069e0 2023-11-06T02:11:28.452 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.445+0000 7ff64b7fe640 1 --2- 172.21.15.49:0/3441845316 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff62c065a60 0x7ff62c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:28.452 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.445+0000 7ff64b7fe640 1 --2- 172.21.15.49:0/3441845316 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff62c065a60 0x7ff62c067f20 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7ff6340af000 tx=0x7ff6340aa000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:28.453 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.449+0000 7ff648ff9640 1 -- 172.21.15.49:0/3441845316 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7ff644050840 con 0x7ff6380069e0 2023-11-06T02:11:28.600 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.601+0000 7ff6522fe640 1 -- 172.21.15.49:0/3441845316 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7ff64c0c89b0 con 0x7ff6380069e0 2023-11-06T02:11:28.702 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.705+0000 7ff648ff9640 1 -- 172.21.15.49:0/3441845316 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v62) v1 ==== 138+0+0 (secure 0 0 0) 0x7ff644053e60 con 0x7ff6380069e0 2023-11-06T02:11:28.703 INFO:teuthology.orchestra.run.smithi049.stderr:pool 'cephfs_metadata' created 2023-11-06T02:11:28.706 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.709+0000 7ff6522fe640 1 -- 172.21.15.49:0/3441845316 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff62c065a60 msgr2=0x7ff62c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:28.706 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.709+0000 7ff6522fe640 1 --2- 172.21.15.49:0/3441845316 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff62c065a60 0x7ff62c067f20 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7ff6340af000 tx=0x7ff6340aa000 comp rx=0 tx=0).stop 2023-11-06T02:11:28.706 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.709+0000 7ff6522fe640 1 -- 172.21.15.49:0/3441845316 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6380069e0 msgr2=0x7ff64c0aee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:28.706 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.709+0000 7ff6522fe640 1 --2- 172.21.15.49:0/3441845316 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6380069e0 0x7ff64c0aee90 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7ff64400db10 tx=0x7ff6440020f0 comp rx=0 tx=0).stop 2023-11-06T02:11:28.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.709+0000 7ff6522fe640 1 -- 172.21.15.49:0/3441845316 shutdown_connections 2023-11-06T02:11:28.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.709+0000 7ff6522fe640 1 --2- 172.21.15.49:0/3441845316 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff62c065a60 0x7ff62c067f20 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:28.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.709+0000 7ff6522fe640 1 --2- 172.21.15.49:0/3441845316 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff64c1e3740 0x7ff64c1e3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:28.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.709+0000 7ff6522fe640 1 --2- 172.21.15.49:0/3441845316 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff64c0c7360 0x7ff64c0c77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:28.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.709+0000 7ff6522fe640 1 --2- 172.21.15.49:0/3441845316 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6380069e0 0x7ff64c0aee90 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:28.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.709+0000 7ff6522fe640 1 -- 172.21.15.49:0/3441845316 >> 172.21.15.49:0/3441845316 conn(0x7ff64c0ce610 msgr2=0x7ff64c0c8bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:28.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.709+0000 7ff6522fe640 1 -- 172.21.15.49:0/3441845316 shutdown_connections 2023-11-06T02:11:28.709 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.709+0000 7ff6522fe640 1 -- 172.21.15.49:0/3441845316 wait complete. 2023-11-06T02:11:28.731 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-06T02:11:28.859 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.861+0000 7f18b2389640 1 Processor -- start 2023-11-06T02:11:28.860 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.861+0000 7f18b2389640 1 -- start start 2023-11-06T02:11:28.861 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.861+0000 7f18b2389640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f18ac15a880 0x7f18ac0b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:28.861 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.861+0000 7f18b2389640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f18ac06d860 0x7f18ac0705e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:28.861 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.861+0000 7f18b2389640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18ac0ae360 con 0x7f18ac070160 2023-11-06T02:11:28.862 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.861+0000 7f18b2389640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18ac070b20 con 0x7f18ac06d860 2023-11-06T02:11:28.862 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.861+0000 7f18b2389640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18ac0ae640 con 0x7f18ac15a880 2023-11-06T02:11:28.862 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.865+0000 7f18b0926640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f18ac06d860 0x7f18ac0705e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:28.862 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.865+0000 7f18b0926640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f18ac06d860 0x7f18ac0705e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48752/0 (socket says 172.21.15.49:48752) 2023-11-06T02:11:28.862 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.865+0000 7f18b0926640 1 -- 172.21.15.49:0/982460933 learned_addr learned my addr 172.21.15.49:0/982460933 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:28.863 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.865+0000 7f18abfff640 1 --2- 172.21.15.49:0/982460933 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f18ac15a880 0x7f18ac0b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:28.863 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.865+0000 7f18b0926640 1 -- 172.21.15.49:0/982460933 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f18ac15a880 msgr2=0x7f18ac0b15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:28.863 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.865+0000 7f18b0926640 1 --2- 172.21.15.49:0/982460933 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f18ac15a880 0x7f18ac0b15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:28.863 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.865+0000 7f18b0926640 1 -- 172.21.15.49:0/982460933 >> v1:172.21.15.150:6789/0 conn(0x7f18ac070160 legacy=0x7f18ac06d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:28.863 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.865+0000 7f18b0926640 1 -- 172.21.15.49:0/982460933 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18ac0ae920 con 0x7f18ac06d860 2023-11-06T02:11:28.865 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.865+0000 7f18b0926640 1 --2- 172.21.15.49:0/982460933 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f18ac06d860 0x7f18ac0705e0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f18a4009e50 tx=0x7f18a401ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=224b754106d0ce44 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:28.865 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.865+0000 7f18aaffd640 1 -- 172.21.15.49:0/982460933 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a40a57f0 con 0x7f18ac06d860 2023-11-06T02:11:28.865 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.865+0000 7f18aaffd640 1 -- 172.21.15.49:0/982460933 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18a40a5950 con 0x7f18ac06d860 2023-11-06T02:11:28.865 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.865+0000 7f18aaffd640 1 -- 172.21.15.49:0/982460933 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a40a5ce0 con 0x7f18ac06d860 2023-11-06T02:11:28.865 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.865+0000 7f18b2389640 1 -- 172.21.15.49:0/982460933 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f18ac06d860 msgr2=0x7f18ac0705e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:28.866 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.865+0000 7f18b2389640 1 --2- 172.21.15.49:0/982460933 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f18ac06d860 0x7f18ac0705e0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f18a4009e50 tx=0x7f18a401ca10 comp rx=0 tx=0).stop 2023-11-06T02:11:28.866 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.865+0000 7f18b2389640 1 -- 172.21.15.49:0/982460933 shutdown_connections 2023-11-06T02:11:28.866 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.865+0000 7f18b2389640 1 --2- 172.21.15.49:0/982460933 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f18ac06d860 0x7f18ac0705e0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:28.866 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.865+0000 7f18b2389640 1 --2- 172.21.15.49:0/982460933 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f18ac15a880 0x7f18ac0b15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:28.866 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.865+0000 7f18b2389640 1 -- 172.21.15.49:0/982460933 >> 172.21.15.49:0/982460933 conn(0x7f18ac06a370 msgr2=0x7f18ac0ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:28.867 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.865+0000 7f18b2389640 1 -- 172.21.15.49:0/982460933 shutdown_connections 2023-11-06T02:11:28.867 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.865+0000 7f18b2389640 1 -- 172.21.15.49:0/982460933 wait complete. 2023-11-06T02:11:28.869 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.869+0000 7f18b2389640 1 Processor -- start 2023-11-06T02:11:28.869 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.869+0000 7f18b2389640 1 -- start start 2023-11-06T02:11:28.870 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.869+0000 7f18b2389640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f18ac06d860 0x7f18ac164bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:28.870 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.869+0000 7f18b2389640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f18ac070160 0x7f18ac14f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:28.870 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.869+0000 7f18b2389640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f18ac15a880 0x7f18ac0c7940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:28.870 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.869+0000 7f18b2389640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18ac14ddf0 con 0x7f18ac15a880 2023-11-06T02:11:28.870 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.869+0000 7f18b2389640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18ac14e0d0 con 0x7f18ac06d860 2023-11-06T02:11:28.871 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.869+0000 7f18b2389640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18ac14e3b0 con 0x7f18ac070160 2023-11-06T02:11:28.871 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.869+0000 7f18b0926640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f18ac15a880 0x7f18ac0c7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:28.871 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.869+0000 7f18b0926640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f18ac15a880 0x7f18ac0c7940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48766/0 (socket says 172.21.15.49:48766) 2023-11-06T02:11:28.871 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.869+0000 7f18b0926640 1 -- 172.21.15.49:0/670631888 learned_addr learned my addr 172.21.15.49:0/670631888 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:28.871 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.869+0000 7f18ab7fe640 1 --2- 172.21.15.49:0/670631888 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f18ac070160 0x7f18ac14f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:28.872 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.869+0000 7f18abfff640 1 --2- 172.21.15.49:0/670631888 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f18ac06d860 0x7f18ac164bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:28.872 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.869+0000 7f18b0926640 1 -- 172.21.15.49:0/670631888 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f18ac070160 msgr2=0x7f18ac14f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:28.872 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.869+0000 7f18b0926640 1 --2- 172.21.15.49:0/670631888 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f18ac070160 0x7f18ac14f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:28.872 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.869+0000 7f18b0926640 1 -- 172.21.15.49:0/670631888 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f18ac06d860 msgr2=0x7f18ac164bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:28.872 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.869+0000 7f18b0926640 1 --2- 172.21.15.49:0/670631888 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f18ac06d860 0x7f18ac164bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:28.873 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.869+0000 7f18b0926640 1 -- 172.21.15.49:0/670631888 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18a4009b30 con 0x7f18ac15a880 2023-11-06T02:11:28.873 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.869+0000 7f18abfff640 1 --2- 172.21.15.49:0/670631888 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f18ac06d860 0x7f18ac164bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:11:28.873 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.869+0000 7f18b0926640 1 --2- 172.21.15.49:0/670631888 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f18ac15a880 0x7f18ac0c7940 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f18a401a640 tx=0x7f18a40197b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:28.873 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.869+0000 7f18a8ff9640 1 -- 172.21.15.49:0/670631888 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a40b8670 con 0x7f18ac15a880 2023-11-06T02:11:28.873 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.869+0000 7f18a8ff9640 1 -- 172.21.15.49:0/670631888 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18a40b87d0 con 0x7f18ac15a880 2023-11-06T02:11:28.874 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.869+0000 7f18a8ff9640 1 -- 172.21.15.49:0/670631888 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a40b8b60 con 0x7f18ac15a880 2023-11-06T02:11:28.874 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.869+0000 7f18b2389640 1 -- 172.21.15.49:0/670631888 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18ac14e690 con 0x7f18ac15a880 2023-11-06T02:11:28.874 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.869+0000 7f18b2389640 1 -- 172.21.15.49:0/670631888 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18ac14f640 con 0x7f18ac15a880 2023-11-06T02:11:28.874 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.869+0000 7f18b2389640 1 -- 172.21.15.49:0/670631888 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18ac14f7d0 con 0x7f18ac15a880 2023-11-06T02:11:28.874 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.873+0000 7f18a8ff9640 1 -- 172.21.15.49:0/670631888 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f18a40b8cc0 con 0x7f18ac15a880 2023-11-06T02:11:28.875 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.873+0000 7f18a8ff9640 1 --2- 172.21.15.49:0/670631888 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f188c065a60 0x7f188c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:28.875 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.873+0000 7f18a8ff9640 1 -- 172.21.15.49:0/670631888 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5580+0+0 (secure 0 0 0) 0x7f18a411e710 con 0x7f18ac15a880 2023-11-06T02:11:28.875 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.873+0000 7f18abfff640 1 --2- 172.21.15.49:0/670631888 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f188c065a60 0x7f188c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:28.876 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.873+0000 7f18abfff640 1 --2- 172.21.15.49:0/670631888 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f188c065a60 0x7f188c067f20 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f1894009a40 tx=0x7f1894008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:28.876 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:28.877+0000 7f18a8ff9640 1 -- 172.21.15.49:0/670631888 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f18a40ec380 con 0x7f18ac15a880 2023-11-06T02:11:29.031 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:29.033+0000 7f18b2389640 1 -- 172.21.15.49:0/670631888 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f18ac0cb0b0 con 0x7f18ac15a880 2023-11-06T02:11:29.714 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:29.717+0000 7f18a8ff9640 1 -- 172.21.15.49:0/670631888 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v63) v1 ==== 170+0+0 (secure 0 0 0) 0x7f18ac0cb0b0 con 0x7f18ac15a880 2023-11-06T02:11:29.714 INFO:teuthology.orchestra.run.smithi049.stderr:pool 'cephfs_data' created 2023-11-06T02:11:29.721 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:29.725+0000 7f18b2389640 1 -- 172.21.15.49:0/670631888 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f188c065a60 msgr2=0x7f188c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:29.721 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:29.725+0000 7f18b2389640 1 --2- 172.21.15.49:0/670631888 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f188c065a60 0x7f188c067f20 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f1894009a40 tx=0x7f1894008040 comp rx=0 tx=0).stop 2023-11-06T02:11:29.721 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:29.725+0000 7f18b2389640 1 -- 172.21.15.49:0/670631888 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f18ac15a880 msgr2=0x7f18ac0c7940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:29.722 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:29.725+0000 7f18b2389640 1 --2- 172.21.15.49:0/670631888 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f18ac15a880 0x7f18ac0c7940 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f18a401a640 tx=0x7f18a40197b0 comp rx=0 tx=0).stop 2023-11-06T02:11:29.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:29.733+0000 7f18b2389640 1 -- 172.21.15.49:0/670631888 shutdown_connections 2023-11-06T02:11:29.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:29.733+0000 7f18b2389640 1 --2- 172.21.15.49:0/670631888 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f188c065a60 0x7f188c067f20 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:29.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:29.733+0000 7f18b2389640 1 --2- 172.21.15.49:0/670631888 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f18ac15a880 0x7f18ac0c7940 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:29.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:29.733+0000 7f18b2389640 1 --2- 172.21.15.49:0/670631888 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f18ac070160 0x7f18ac14f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:29.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:29.733+0000 7f18b2389640 1 --2- 172.21.15.49:0/670631888 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f18ac06d860 0x7f18ac164bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:29.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:29.733+0000 7f18b2389640 1 -- 172.21.15.49:0/670631888 >> 172.21.15.49:0/670631888 conn(0x7f18ac06a370 msgr2=0x7f18ac15ae70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:29.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:29.733+0000 7f18b2389640 1 -- 172.21.15.49:0/670631888 shutdown_connections 2023-11-06T02:11:29.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:29.733+0000 7f18b2389640 1 -- 172.21.15.49:0/670631888 wait complete. 2023-11-06T02:11:29.760 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-06T02:11:29.881 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:29.885+0000 7fa49b197640 1 Processor -- start 2023-11-06T02:11:29.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:29.885+0000 7fa49b197640 1 -- start start 2023-11-06T02:11:29.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:29.885+0000 7fa49b197640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa494070870 0x7fa49406c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:29.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:29.885+0000 7fa49b197640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa4940c8790 0x7fa49406cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:29.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:29.885+0000 7fa49b197640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4940b00a0 con 0x7fa494161260 2023-11-06T02:11:29.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:29.885+0000 7fa49b197640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa49406c580 con 0x7fa494070870 2023-11-06T02:11:29.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:29.885+0000 7fa49b197640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4940c91f0 con 0x7fa4940c8790 2023-11-06T02:11:29.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:29.885+0000 7fa493fff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa494070870 0x7fa49406c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:29.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:29.885+0000 7fa493fff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa494070870 0x7fa49406c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48774/0 (socket says 172.21.15.49:48774) 2023-11-06T02:11:29.885 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:29.885+0000 7fa493fff640 1 -- 172.21.15.49:0/121321704 learned_addr learned my addr 172.21.15.49:0/121321704 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:29.885 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:29.885+0000 7fa499734640 1 --2- 172.21.15.49:0/121321704 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa4940c8790 0x7fa49406cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:29.885 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:29.885+0000 7fa493fff640 1 -- 172.21.15.49:0/121321704 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa4940c8790 msgr2=0x7fa49406cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:29.885 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:29.885+0000 7fa493fff640 1 --2- 172.21.15.49:0/121321704 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa4940c8790 0x7fa49406cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:29.885 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:29.885+0000 7fa493fff640 1 -- 172.21.15.49:0/121321704 >> v1:172.21.15.150:6789/0 conn(0x7fa494161260 legacy=0x7fa494070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:29.886 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:29.885+0000 7fa493fff640 1 -- 172.21.15.49:0/121321704 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa49406c230 con 0x7fa494070870 2023-11-06T02:11:29.886 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:29.885+0000 7fa499734640 1 --2- 172.21.15.49:0/121321704 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa4940c8790 0x7fa49406cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:11:29.887 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:29.889+0000 7fa493fff640 1 --2- 172.21.15.49:0/121321704 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa494070870 0x7fa49406c9a0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fa47c009870 tx=0x7fa47c01c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=453477db5a8103e3 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:29.888 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:29.889+0000 7fa4937fe640 1 -- 172.21.15.49:0/121321704 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa47c0ab600 con 0x7fa494070870 2023-11-06T02:11:29.888 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:29.889+0000 7fa4937fe640 1 -- 172.21.15.49:0/121321704 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa47c0a7050 con 0x7fa494070870 2023-11-06T02:11:29.888 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:29.889+0000 7fa4937fe640 1 -- 172.21.15.49:0/121321704 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa47c0ab980 con 0x7fa494070870 2023-11-06T02:11:29.888 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:29.889+0000 7fa49b197640 1 -- 172.21.15.49:0/121321704 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa494070870 msgr2=0x7fa49406c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:29.888 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:29.889+0000 7fa49b197640 1 --2- 172.21.15.49:0/121321704 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa494070870 0x7fa49406c9a0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fa47c009870 tx=0x7fa47c01c3e0 comp rx=0 tx=0).stop 2023-11-06T02:11:29.888 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:29.889+0000 7fa49b197640 1 -- 172.21.15.49:0/121321704 shutdown_connections 2023-11-06T02:11:29.889 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:29.889+0000 7fa49b197640 1 --2- 172.21.15.49:0/121321704 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa4940c8790 0x7fa49406cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:29.889 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:29.889+0000 7fa49b197640 1 --2- 172.21.15.49:0/121321704 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa494070870 0x7fa49406c9a0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:29.889 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:29.889+0000 7fa49b197640 1 -- 172.21.15.49:0/121321704 >> 172.21.15.49:0/121321704 conn(0x7fa49406a370 msgr2=0x7fa4940ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:29.889 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:29.889+0000 7fa49b197640 1 -- 172.21.15.49:0/121321704 shutdown_connections 2023-11-06T02:11:29.889 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:29.889+0000 7fa49b197640 1 -- 172.21.15.49:0/121321704 wait complete. 2023-11-06T02:11:29.890 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:29.889+0000 7fa49b197640 1 Processor -- start 2023-11-06T02:11:29.890 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:29.889+0000 7fa49b197640 1 -- start start 2023-11-06T02:11:29.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:29.889+0000 7fa49b197640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa494070870 0x7fa494151ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:29.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:29.889+0000 7fa49b197640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4940c8790 0x7fa49415ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:29.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:29.889+0000 7fa49b197640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa494161260 0x7fa494154370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:29.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:29.889+0000 7fa49b197640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa494162180 con 0x7fa4940c8790 2023-11-06T02:11:29.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:29.889+0000 7fa49b197640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa494162460 con 0x7fa494161260 2023-11-06T02:11:29.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:29.889+0000 7fa49b197640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa494162740 con 0x7fa494070870 2023-11-06T02:11:29.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:29.889+0000 7fa493fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4940c8790 0x7fa49415ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:29.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:29.889+0000 7fa493fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4940c8790 0x7fa49415ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48778/0 (socket says 172.21.15.49:48778) 2023-11-06T02:11:29.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:29.889+0000 7fa493fff640 1 -- 172.21.15.49:0/3139929994 learned_addr learned my addr 172.21.15.49:0/3139929994 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:29.895 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:29.889+0000 7fa498f33640 1 --2- 172.21.15.49:0/3139929994 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa494070870 0x7fa494151ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:29.895 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:29.889+0000 7fa499734640 1 --2- 172.21.15.49:0/3139929994 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa494161260 0x7fa494154370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:29.895 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:29.889+0000 7fa493fff640 1 -- 172.21.15.49:0/3139929994 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa494070870 msgr2=0x7fa494151ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:29.895 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:29.889+0000 7fa493fff640 1 --2- 172.21.15.49:0/3139929994 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa494070870 0x7fa494151ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:29.895 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:29.889+0000 7fa493fff640 1 -- 172.21.15.49:0/3139929994 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa494161260 msgr2=0x7fa494154370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:29.896 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:29.889+0000 7fa493fff640 1 --2- 172.21.15.49:0/3139929994 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa494161260 0x7fa494154370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:29.896 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:29.889+0000 7fa493fff640 1 -- 172.21.15.49:0/3139929994 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa47c0094d0 con 0x7fa4940c8790 2023-11-06T02:11:29.896 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:29.893+0000 7fa493fff640 1 --2- 172.21.15.49:0/3139929994 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4940c8790 0x7fa49415ab50 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fa47c0ae040 tx=0x7fa47c019180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:29.896 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:29.893+0000 7fa4917fa640 1 -- 172.21.15.49:0/3139929994 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa47c0b4640 con 0x7fa4940c8790 2023-11-06T02:11:29.896 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:29.893+0000 7fa4917fa640 1 -- 172.21.15.49:0/3139929994 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa47c0b47a0 con 0x7fa4940c8790 2023-11-06T02:11:29.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:29.893+0000 7fa49b197640 1 -- 172.21.15.49:0/3139929994 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa494162a20 con 0x7fa4940c8790 2023-11-06T02:11:29.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:29.893+0000 7fa4917fa640 1 -- 172.21.15.49:0/3139929994 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa47c0b4b50 con 0x7fa4940c8790 2023-11-06T02:11:29.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:29.893+0000 7fa49b197640 1 -- 172.21.15.49:0/3139929994 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa494152400 con 0x7fa4940c8790 2023-11-06T02:11:29.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:29.893+0000 7fa49b197640 1 -- 172.21.15.49:0/3139929994 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa494152590 con 0x7fa4940c8790 2023-11-06T02:11:29.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:29.893+0000 7fa4917fa640 1 -- 172.21.15.49:0/3139929994 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fa47c0b2070 con 0x7fa4940c8790 2023-11-06T02:11:29.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:29.893+0000 7fa4917fa640 1 --2- 172.21.15.49:0/3139929994 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa474065a60 0x7fa474067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:29.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:29.893+0000 7fa4917fa640 1 -- 172.21.15.49:0/3139929994 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5962+0+0 (secure 0 0 0) 0x7fa47c11d190 con 0x7fa4940c8790 2023-11-06T02:11:29.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:29.893+0000 7fa498f33640 1 --2- 172.21.15.49:0/3139929994 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa474065a60 0x7fa474067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:29.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:29.893+0000 7fa498f33640 1 --2- 172.21.15.49:0/3139929994 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa474065a60 0x7fa474067f20 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fa49415ba90 tx=0x7fa48c0094d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:29.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:29.897+0000 7fa4917fa640 1 -- 172.21.15.49:0/3139929994 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fa47c0eab70 con 0x7fa4940c8790 2023-11-06T02:11:30.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.085+0000 7fa49b197640 1 -- 172.21.15.49:0/3139929994 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fa494074040 con 0x7fa4940c8790 2023-11-06T02:11:30.096 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.097+0000 7fa4917fa640 1 -- 172.21.15.49:0/3139929994 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 9 and data pool 10 v27) v1 ==== 173+0+0 (secure 0 0 0) 0x7fa47c0ee190 con 0x7fa4940c8790 2023-11-06T02:11:30.096 INFO:teuthology.orchestra.run.smithi049.stderr:new fs with metadata pool 9 and data pool 10 2023-11-06T02:11:30.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.101+0000 7fa49b197640 1 -- 172.21.15.49:0/3139929994 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa474065a60 msgr2=0x7fa474067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:30.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.101+0000 7fa49b197640 1 --2- 172.21.15.49:0/3139929994 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa474065a60 0x7fa474067f20 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fa49415ba90 tx=0x7fa48c0094d0 comp rx=0 tx=0).stop 2023-11-06T02:11:30.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.101+0000 7fa49b197640 1 -- 172.21.15.49:0/3139929994 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4940c8790 msgr2=0x7fa49415ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:30.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.101+0000 7fa49b197640 1 --2- 172.21.15.49:0/3139929994 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4940c8790 0x7fa49415ab50 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fa47c0ae040 tx=0x7fa47c019180 comp rx=0 tx=0).stop 2023-11-06T02:11:30.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.105+0000 7fa49b197640 1 -- 172.21.15.49:0/3139929994 shutdown_connections 2023-11-06T02:11:30.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.105+0000 7fa49b197640 1 --2- 172.21.15.49:0/3139929994 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa474065a60 0x7fa474067f20 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:30.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.105+0000 7fa49b197640 1 --2- 172.21.15.49:0/3139929994 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4940c8790 0x7fa49415ab50 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:30.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.105+0000 7fa49b197640 1 --2- 172.21.15.49:0/3139929994 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa494161260 0x7fa494154370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:30.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.105+0000 7fa49b197640 1 --2- 172.21.15.49:0/3139929994 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa494070870 0x7fa494151ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:30.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.105+0000 7fa49b197640 1 -- 172.21.15.49:0/3139929994 >> 172.21.15.49:0/3139929994 conn(0x7fa49406a370 msgr2=0x7fa494070160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:30.105 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.105+0000 7fa49b197640 1 -- 172.21.15.49:0/3139929994 shutdown_connections 2023-11-06T02:11:30.105 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.105+0000 7fa49b197640 1 -- 172.21.15.49:0/3139929994 wait complete. 2023-11-06T02:11:30.128 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T02:11:30.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.245+0000 7fd58f519640 1 Processor -- start 2023-11-06T02:11:30.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.245+0000 7fd58f519640 1 -- start start 2023-11-06T02:11:30.246 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.249+0000 7fd58f519640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd58815ab50 0x7fd5880c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:30.246 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.249+0000 7fd58f519640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fd5880b0790 0x7fd5880c2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:30.247 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.249+0000 7fd58f519640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5880cab80 con 0x7fd5880cb010 2023-11-06T02:11:30.247 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.249+0000 7fd58f519640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd58815af30 con 0x7fd58815ab50 2023-11-06T02:11:30.247 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.249+0000 7fd58f519640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5880cb5b0 con 0x7fd5880b0790 2023-11-06T02:11:30.247 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.249+0000 7fd58d2b5640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd58815ab50 0x7fd5880c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:30.247 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.249+0000 7fd58d2b5640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd58815ab50 0x7fd5880c9bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48784/0 (socket says 172.21.15.49:48784) 2023-11-06T02:11:30.248 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.249+0000 7fd58d2b5640 1 -- 172.21.15.49:0/3143476257 learned_addr learned my addr 172.21.15.49:0/3143476257 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:30.248 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.249+0000 7fd58dab6640 1 --2- 172.21.15.49:0/3143476257 >> v2:172.21.15.150:3300/0 conn(0x7fd5880b0790 0x7fd5880c2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:30.248 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.249+0000 7fd58d2b5640 1 -- 172.21.15.49:0/3143476257 >> v2:172.21.15.150:3300/0 conn(0x7fd5880b0790 msgr2=0x7fd5880c2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:30.248 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.249+0000 7fd58d2b5640 1 --2- 172.21.15.49:0/3143476257 >> v2:172.21.15.150:3300/0 conn(0x7fd5880b0790 0x7fd5880c2710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:30.248 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.249+0000 7fd58d2b5640 1 -- 172.21.15.49:0/3143476257 >> v1:172.21.15.150:6789/0 conn(0x7fd5880cb010 legacy=0x7fd5880b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:30.249 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.249+0000 7fd58d2b5640 1 -- 172.21.15.49:0/3143476257 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5880cb890 con 0x7fd58815ab50 2023-11-06T02:11:30.250 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.249+0000 7fd58d2b5640 1 --2- 172.21.15.49:0/3143476257 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd58815ab50 0x7fd5880c9bb0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fd5880ca5d0 tx=0x7fd57c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=356c789cf313ee7b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:30.250 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.249+0000 7fd57bfff640 1 -- 172.21.15.49:0/3143476257 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57c0aa5f0 con 0x7fd58815ab50 2023-11-06T02:11:30.250 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.253+0000 7fd57bfff640 1 -- 172.21.15.49:0/3143476257 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd57c0aa750 con 0x7fd58815ab50 2023-11-06T02:11:30.250 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.253+0000 7fd57bfff640 1 -- 172.21.15.49:0/3143476257 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57c0aaae0 con 0x7fd58815ab50 2023-11-06T02:11:30.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.253+0000 7fd58f519640 1 -- 172.21.15.49:0/3143476257 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd58815ab50 msgr2=0x7fd5880c9bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:30.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.253+0000 7fd58f519640 1 --2- 172.21.15.49:0/3143476257 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd58815ab50 0x7fd5880c9bb0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fd5880ca5d0 tx=0x7fd57c01c490 comp rx=0 tx=0).stop 2023-11-06T02:11:30.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.253+0000 7fd58f519640 1 -- 172.21.15.49:0/3143476257 shutdown_connections 2023-11-06T02:11:30.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.253+0000 7fd58f519640 1 --2- 172.21.15.49:0/3143476257 >> v2:172.21.15.150:3300/0 conn(0x7fd5880b0790 0x7fd5880c2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:30.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.253+0000 7fd58f519640 1 --2- 172.21.15.49:0/3143476257 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd58815ab50 0x7fd5880c9bb0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:30.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.253+0000 7fd58f519640 1 -- 172.21.15.49:0/3143476257 >> 172.21.15.49:0/3143476257 conn(0x7fd58806c0a0 msgr2=0x7fd58806c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:30.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.253+0000 7fd58f519640 1 -- 172.21.15.49:0/3143476257 shutdown_connections 2023-11-06T02:11:30.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.253+0000 7fd58f519640 1 -- 172.21.15.49:0/3143476257 wait complete. 2023-11-06T02:11:30.255 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.253+0000 7fd58f519640 1 Processor -- start 2023-11-06T02:11:30.255 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.253+0000 7fd58f519640 1 -- start start 2023-11-06T02:11:30.255 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.253+0000 7fd58f519640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5880b0790 0x7fd5880ab8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:30.255 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.253+0000 7fd58f519640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd5880cb010 0x7fd58814a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:30.256 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.253+0000 7fd58f519640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd58815ab50 0x7fd5880ccbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:30.256 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.253+0000 7fd58f519640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd588153ce0 con 0x7fd5880cb010 2023-11-06T02:11:30.256 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.253+0000 7fd58f519640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5880abf50 con 0x7fd5880b0790 2023-11-06T02:11:30.256 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.253+0000 7fd58f519640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5880ac230 con 0x7fd58815ab50 2023-11-06T02:11:30.256 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.253+0000 7fd58cab4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd5880cb010 0x7fd58814a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:30.257 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.253+0000 7fd58cab4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd5880cb010 0x7fd58814a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48786/0 (socket says 172.21.15.49:48786) 2023-11-06T02:11:30.257 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.253+0000 7fd58cab4640 1 -- 172.21.15.49:0/1489676809 learned_addr learned my addr 172.21.15.49:0/1489676809 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:30.257 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.253+0000 7fd58d2b5640 1 --2- 172.21.15.49:0/1489676809 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5880b0790 0x7fd5880ab8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:30.257 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.253+0000 7fd58dab6640 1 --2- 172.21.15.49:0/1489676809 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd58815ab50 0x7fd5880ccbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:30.257 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.253+0000 7fd58cab4640 1 -- 172.21.15.49:0/1489676809 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd58815ab50 msgr2=0x7fd5880ccbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:30.258 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.253+0000 7fd58cab4640 1 --2- 172.21.15.49:0/1489676809 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd58815ab50 0x7fd5880ccbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:30.258 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.253+0000 7fd58cab4640 1 -- 172.21.15.49:0/1489676809 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5880b0790 msgr2=0x7fd5880ab8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:30.258 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.253+0000 7fd58cab4640 1 --2- 172.21.15.49:0/1489676809 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5880b0790 0x7fd5880ab8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:30.258 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.253+0000 7fd58cab4640 1 -- 172.21.15.49:0/1489676809 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd57c009480 con 0x7fd5880cb010 2023-11-06T02:11:30.258 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.253+0000 7fd58cab4640 1 --2- 172.21.15.49:0/1489676809 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd5880cb010 0x7fd58814a170 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fd58400c450 tx=0x7fd58400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:30.259 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.253+0000 7fd579ffb640 1 -- 172.21.15.49:0/1489676809 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd584012830 con 0x7fd5880cb010 2023-11-06T02:11:30.259 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.253+0000 7fd579ffb640 1 -- 172.21.15.49:0/1489676809 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5840129b0 con 0x7fd5880cb010 2023-11-06T02:11:30.259 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.253+0000 7fd58f519640 1 -- 172.21.15.49:0/1489676809 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5880ac510 con 0x7fd5880cb010 2023-11-06T02:11:30.259 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.253+0000 7fd579ffb640 1 -- 172.21.15.49:0/1489676809 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd584012d60 con 0x7fd5880cb010 2023-11-06T02:11:30.259 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.253+0000 7fd58f519640 1 -- 172.21.15.49:0/1489676809 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5880acad0 con 0x7fd5880cb010 2023-11-06T02:11:30.260 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.257+0000 7fd58f519640 1 -- 172.21.15.49:0/1489676809 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5880b0de0 con 0x7fd5880cb010 2023-11-06T02:11:30.260 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.257+0000 7fd579ffb640 1 -- 172.21.15.49:0/1489676809 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fd58401c460 con 0x7fd5880cb010 2023-11-06T02:11:30.260 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.257+0000 7fd579ffb640 1 --2- 172.21.15.49:0/1489676809 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd56c065a60 0x7fd56c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:30.260 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.257+0000 7fd579ffb640 1 -- 172.21.15.49:0/1489676809 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd584083200 con 0x7fd5880cb010 2023-11-06T02:11:30.260 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.257+0000 7fd58d2b5640 1 --2- 172.21.15.49:0/1489676809 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd56c065a60 0x7fd56c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:30.260 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.257+0000 7fd58d2b5640 1 --2- 172.21.15.49:0/1489676809 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd56c065a60 0x7fd56c067f20 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fd57c0ae000 tx=0x7fd57c0a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:30.261 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.261+0000 7fd579ffb640 1 -- 172.21.15.49:0/1489676809 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fd5840509c0 con 0x7fd5880cb010 2023-11-06T02:11:30.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.417+0000 7fd58f519640 1 -- 172.21.15.49:0/1489676809 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd588154a40 con 0x7fd5880cb010 2023-11-06T02:11:30.417 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.421+0000 7fd579ffb640 1 -- 172.21.15.49:0/1489676809 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+17838 (secure 0 0 0) 0x7fd584053fe0 con 0x7fd5880cb010 2023-11-06T02:11:30.418 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:11:30.418 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":64,"fsid":"1a8b035c-6ba4-499a-b876-aea47fd9c0d4","created":"2023-11-06T02:08:06.440857+0000","modified":"2023-11-06T02:11:30.090572+0000","last_up_change":"2023-11-06T02:08:17.982372+0000","last_in_change":"2023-11-06T02:08:08.906631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T02:08:19.252222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T02:08:24.776211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-06T02:11:28.648326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-06T02:11:29.079513+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ad55af8-0571-4e6a-8722-b5d429b7496d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6809","nonce":38116},{"type":"v1","addr":"172.21.15.49:6811","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6813","nonce":38116},{"type":"v1","addr":"172.21.15.49:6815","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6821","nonce":38116},{"type":"v1","addr":"172.21.15.49:6823","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6817","nonce":38116},{"type":"v1","addr":"172.21.15.49:6819","nonce":38116}]},"public_addr":"172.21.15.49:6811/38116","cluster_addr":"172.21.15.49:6815/38116","heartbeat_back_addr":"172.21.15.49:6823/38116","heartbeat_front_addr":"172.21.15.49:6819/38116","state":["exists","up"]},{"osd":1,"uuid":"c8b31f85-a757-43ed-9419-351acc46611b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":38113},{"type":"v1","addr":"172.21.15.49:6810","nonce":38113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":38113},{"type":"v1","addr":"172.21.15.49:6814","nonce":38113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":38113},{"type":"v1","addr":"172.21.15.49:6822","nonce":38113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":38113},{"type":"v1","addr":"172.21.15.49:6818","nonce":38113}]},"public_addr":"172.21.15.49:6810/38113","cluster_addr":"172.21.15.49:6814/38113","heartbeat_back_addr":"172.21.15.49:6822/38113","heartbeat_front_addr":"172.21.15.49:6818/38113","state":["exists","up"]},{"osd":2,"uuid":"555b3bf3-0a70-4b91-8efd-e4204b5fc26e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":38114},{"type":"v1","addr":"172.21.15.49:6801","nonce":38114}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":38114},{"type":"v1","addr":"172.21.15.49:6803","nonce":38114}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":38114},{"type":"v1","addr":"172.21.15.49:6807","nonce":38114}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":38114},{"type":"v1","addr":"172.21.15.49:6805","nonce":38114}]},"public_addr":"172.21.15.49:6801/38114","cluster_addr":"172.21.15.49:6803/38114","heartbeat_back_addr":"172.21.15.49:6807/38114","heartbeat_front_addr":"172.21.15.49:6805/38114","state":["exists","up"]},{"osd":3,"uuid":"8d4e5d77-03ef-4b9b-bfcb-5ae93a46795c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":38115},{"type":"v1","addr":"172.21.15.49:6825","nonce":38115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":38115},{"type":"v1","addr":"172.21.15.49:6827","nonce":38115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":38115},{"type":"v1","addr":"172.21.15.49:6831","nonce":38115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":38115},{"type":"v1","addr":"172.21.15.49:6829","nonce":38115}]},"public_addr":"172.21.15.49:6825/38115","cluster_addr":"172.21.15.49:6827/38115","heartbeat_back_addr":"172.21.15.49:6831/38115","heartbeat_front_addr":"172.21.15.49:6829/38115","state":["exists","up"]},{"osd":4,"uuid":"dba03ad4-68a3-4cf3-9218-ac611f02eacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":38417},{"type":"v1","addr":"172.21.15.150:6809","nonce":38417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":38417},{"type":"v1","addr":"172.21.15.150:6811","nonce":38417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":38417},{"type":"v1","addr":"172.21.15.150:6815","nonce":38417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":38417},{"type":"v1","addr":"172.21.15.150:6813","nonce":38417}]},"public_addr":"172.21.15.150:6809/38417","cluster_addr":"172.21.15.150:6811/38417","heartbeat_back_addr":"172.21.15.150:6815/38417","heartbeat_front_addr":"172.21.15.150:6813/38417","state":["exists","up"]},{"osd":5,"uuid":"0df9c303-c255-4640-8e91-01f1c3efe18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":38418},{"type":"v1","addr":"172.21.15.150:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":38418},{"type":"v1","addr":"172.21.15.150:6803","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":38418},{"type":"v1","addr":"172.21.15.150:6807","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":38418},{"type":"v1","addr":"172.21.15.150:6805","nonce":38418}]},"public_addr":"172.21.15.150:6801/38418","cluster_addr":"172.21.15.150:6803/38418","heartbeat_back_addr":"172.21.15.150:6807/38418","heartbeat_front_addr":"172.21.15.150:6805/38418","state":["exists","up"]},{"osd":6,"uuid":"e59fa189-4b70-4c06-9722-c02b90828bf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":38428},{"type":"v1","addr":"172.21.15.150:6825","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":38428},{"type":"v1","addr":"172.21.15.150:6827","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":38428},{"type":"v1","addr":"172.21.15.150:6831","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":38428},{"type":"v1","addr":"172.21.15.150:6829","nonce":38428}]},"public_addr":"172.21.15.150:6825/38428","cluster_addr":"172.21.15.150:6827/38428","heartbeat_back_addr":"172.21.15.150:6831/38428","heartbeat_front_addr":"172.21.15.150:6829/38428","state":["exists","up"]},{"osd":7,"uuid":"a64f57e1-ef98-46bd-b304-385759e95898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":38421},{"type":"v1","addr":"172.21.15.150:6817","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":38421},{"type":"v1","addr":"172.21.15.150:6819","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":38421},{"type":"v1","addr":"172.21.15.150:6823","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":38421},{"type":"v1","addr":"172.21.15.150:6821","nonce":38421}]},"public_addr":"172.21.15.150:6817/38421","cluster_addr":"172.21.15.150:6819/38421","heartbeat_back_addr":"172.21.15.150:6823/38421","heartbeat_front_addr":"172.21.15.150:6821/38421","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.438552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.413280+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.296466+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.424966+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.276953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.230984+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.410235+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.480084+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T02:11:30.419 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.421+0000 7fd58f519640 1 -- 172.21.15.49:0/1489676809 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd56c065a60 msgr2=0x7fd56c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:30.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.421+0000 7fd58f519640 1 --2- 172.21.15.49:0/1489676809 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd56c065a60 0x7fd56c067f20 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fd57c0ae000 tx=0x7fd57c0a7000 comp rx=0 tx=0).stop 2023-11-06T02:11:30.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.421+0000 7fd58f519640 1 -- 172.21.15.49:0/1489676809 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd5880cb010 msgr2=0x7fd58814a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:30.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.421+0000 7fd58f519640 1 --2- 172.21.15.49:0/1489676809 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd5880cb010 0x7fd58814a170 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fd58400c450 tx=0x7fd58400c910 comp rx=0 tx=0).stop 2023-11-06T02:11:30.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.421+0000 7fd58f519640 1 -- 172.21.15.49:0/1489676809 shutdown_connections 2023-11-06T02:11:30.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.421+0000 7fd58f519640 1 --2- 172.21.15.49:0/1489676809 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd56c065a60 0x7fd56c067f20 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:30.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.421+0000 7fd58f519640 1 --2- 172.21.15.49:0/1489676809 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd5880cb010 0x7fd58814a170 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:30.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.421+0000 7fd58f519640 1 --2- 172.21.15.49:0/1489676809 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd58815ab50 0x7fd5880ccbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:30.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.421+0000 7fd58f519640 1 --2- 172.21.15.49:0/1489676809 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5880b0790 0x7fd5880ab8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:30.422 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.421+0000 7fd58f519640 1 -- 172.21.15.49:0/1489676809 >> 172.21.15.49:0/1489676809 conn(0x7fd58806c0a0 msgr2=0x7fd5880c9bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:30.422 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.421+0000 7fd58f519640 1 -- 172.21.15.49:0/1489676809 shutdown_connections 2023-11-06T02:11:30.422 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.421+0000 7fd58f519640 1 -- 172.21.15.49:0/1489676809 wait complete. 2023-11-06T02:11:30.438 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T02:11:30.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.569+0000 7fd5388ab640 1 Processor -- start 2023-11-06T02:11:30.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.569+0000 7fd5388ab640 1 -- start start 2023-11-06T02:11:30.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.569+0000 7fd5388ab640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fd53415ab50 0x7fd5340c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:30.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.569+0000 7fd5388ab640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5340cab80 con 0x7fd5340cb010 2023-11-06T02:11:30.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.569+0000 7fd5388ab640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd53415af30 con 0x7fd5340b0790 2023-11-06T02:11:30.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.569+0000 7fd5388ab640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5340cb5b0 con 0x7fd53415ab50 2023-11-06T02:11:30.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.569+0000 7fd52bfff640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fd5340cb010 0x7fd5340b0080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:46642/0 (socket says 172.21.15.49:46642) 2023-11-06T02:11:30.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.569+0000 7fd52bfff640 1 -- 172.21.15.49:0/298200131 learned_addr learned my addr 172.21.15.49:0/298200131 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:11:30.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.569+0000 7fd53259c640 1 --2- 172.21.15.49:0/298200131 >> v2:172.21.15.150:3300/0 conn(0x7fd53415ab50 0x7fd5340c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:30.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.569+0000 7fd531d9b640 1 -- 172.21.15.49:0/298200131 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3750870599 0 0) 0x7fd5340cab80 con 0x7fd5340cb010 2023-11-06T02:11:30.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.569+0000 7fd531d9b640 1 -- 172.21.15.49:0/298200131 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd520003530 con 0x7fd5340cb010 2023-11-06T02:11:30.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.569+0000 7fd531d9b640 1 -- 172.21.15.49:0/298200131 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3876069018 0 0) 0x7fd520003530 con 0x7fd5340cb010 2023-11-06T02:11:30.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.569+0000 7fd53259c640 1 -- 172.21.15.49:0/298200131 >> v1:172.21.15.150:6789/0 conn(0x7fd5340b0790 legacy=0x7fd5340c2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:30.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.569+0000 7fd53259c640 1 -- 172.21.15.49:0/298200131 >> v1:172.21.15.49:6789/0 conn(0x7fd5340cb010 legacy=0x7fd5340b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:30.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.569+0000 7fd53259c640 1 -- 172.21.15.49:0/298200131 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5340cb890 con 0x7fd53415ab50 2023-11-06T02:11:30.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.573+0000 7fd53259c640 1 --2- 172.21.15.49:0/298200131 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd53415ab50 0x7fd5340c9bb0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd5240097a0 tx=0x7fd52401c350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c271f27884071846 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:30.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.573+0000 7fd531d9b640 1 -- 172.21.15.49:0/298200131 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd51c001680 con 0x7fd53415ab50 2023-11-06T02:11:30.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.573+0000 7fd531d9b640 1 -- 172.21.15.49:0/298200131 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5240a7050 con 0x7fd53415ab50 2023-11-06T02:11:30.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.573+0000 7fd531d9b640 1 -- 172.21.15.49:0/298200131 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5240ab600 con 0x7fd53415ab50 2023-11-06T02:11:30.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.573+0000 7fd5388ab640 1 -- 172.21.15.49:0/298200131 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd53415ab50 msgr2=0x7fd5340c9bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:30.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.573+0000 7fd5388ab640 1 --2- 172.21.15.49:0/298200131 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd53415ab50 0x7fd5340c9bb0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd5240097a0 tx=0x7fd52401c350 comp rx=0 tx=0).stop 2023-11-06T02:11:30.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.573+0000 7fd5388ab640 1 -- 172.21.15.49:0/298200131 shutdown_connections 2023-11-06T02:11:30.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.573+0000 7fd5388ab640 1 --2- 172.21.15.49:0/298200131 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd53415ab50 0x7fd5340c9bb0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:30.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.573+0000 7fd5388ab640 1 -- 172.21.15.49:0/298200131 >> 172.21.15.49:0/298200131 conn(0x7fd53406c0a0 msgr2=0x7fd53406c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:30.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.573+0000 7fd5388ab640 1 -- 172.21.15.49:0/298200131 shutdown_connections 2023-11-06T02:11:30.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.573+0000 7fd5388ab640 1 -- 172.21.15.49:0/298200131 wait complete. 2023-11-06T02:11:30.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.573+0000 7fd5388ab640 1 Processor -- start 2023-11-06T02:11:30.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.573+0000 7fd5388ab640 1 -- start start 2023-11-06T02:11:30.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.573+0000 7fd5388ab640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd5340b0790 0x7fd5341f68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:30.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.573+0000 7fd5388ab640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5340cb010 0x7fd53414eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:30.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.573+0000 7fd5388ab640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd53415ab50 0x7fd534148150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:30.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.573+0000 7fd5388ab640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5341f6e00 con 0x7fd5340b0790 2023-11-06T02:11:30.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.573+0000 7fd5388ab640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd53414de20 con 0x7fd5340cb010 2023-11-06T02:11:30.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.573+0000 7fd5388ab640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd53414e0e0 con 0x7fd53415ab50 2023-11-06T02:11:30.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.573+0000 7fd53259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd5340b0790 0x7fd5341f68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:30.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.573+0000 7fd53259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd5340b0790 0x7fd5341f68c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48794/0 (socket says 172.21.15.49:48794) 2023-11-06T02:11:30.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.573+0000 7fd53259c640 1 -- 172.21.15.49:0/1184649634 learned_addr learned my addr 172.21.15.49:0/1184649634 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:30.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.573+0000 7fd52bfff640 1 --2- 172.21.15.49:0/1184649634 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5340cb010 0x7fd53414eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:30.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.573+0000 7fd532d9d640 1 --2- 172.21.15.49:0/1184649634 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd53415ab50 0x7fd534148150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:30.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.577+0000 7fd53259c640 1 -- 172.21.15.49:0/1184649634 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd53415ab50 msgr2=0x7fd534148150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:30.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.577+0000 7fd53259c640 1 --2- 172.21.15.49:0/1184649634 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd53415ab50 0x7fd534148150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:30.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.577+0000 7fd53259c640 1 -- 172.21.15.49:0/1184649634 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5340cb010 msgr2=0x7fd53414eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:30.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.577+0000 7fd53259c640 1 --2- 172.21.15.49:0/1184649634 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5340cb010 0x7fd53414eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:30.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.577+0000 7fd53259c640 1 -- 172.21.15.49:0/1184649634 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5240093b0 con 0x7fd5340b0790 2023-11-06T02:11:30.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.577+0000 7fd52bfff640 1 --2- 172.21.15.49:0/1184649634 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5340cb010 0x7fd53414eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:11:30.579 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.577+0000 7fd53259c640 1 --2- 172.21.15.49:0/1184649634 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd5340b0790 0x7fd5341f68c0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fd524018f60 tx=0x7fd5240190f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:30.579 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.577+0000 7fd52affd640 1 -- 172.21.15.49:0/1184649634 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5240bb070 con 0x7fd5340b0790 2023-11-06T02:11:30.579 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.577+0000 7fd52affd640 1 -- 172.21.15.49:0/1184649634 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd524003830 con 0x7fd5340b0790 2023-11-06T02:11:30.579 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.577+0000 7fd5388ab640 1 -- 172.21.15.49:0/1184649634 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd53414e3c0 con 0x7fd5340b0790 2023-11-06T02:11:30.579 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.577+0000 7fd52affd640 1 -- 172.21.15.49:0/1184649634 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd524003ba0 con 0x7fd5340b0790 2023-11-06T02:11:30.580 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.577+0000 7fd5388ab640 1 -- 172.21.15.49:0/1184649634 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd53414e980 con 0x7fd5340b0790 2023-11-06T02:11:30.580 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.577+0000 7fd5388ab640 1 -- 172.21.15.49:0/1184649634 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5340b0de0 con 0x7fd5340b0790 2023-11-06T02:11:30.580 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.577+0000 7fd52affd640 1 -- 172.21.15.49:0/1184649634 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fd5240b2050 con 0x7fd5340b0790 2023-11-06T02:11:30.580 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.577+0000 7fd52affd640 1 --2- 172.21.15.49:0/1184649634 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd514065a60 0x7fd514067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:30.580 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.577+0000 7fd52affd640 1 -- 172.21.15.49:0/1184649634 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd52411ce10 con 0x7fd5340b0790 2023-11-06T02:11:30.581 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.577+0000 7fd52bfff640 1 --2- 172.21.15.49:0/1184649634 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd514065a60 0x7fd514067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:30.581 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.577+0000 7fd52bfff640 1 --2- 172.21.15.49:0/1184649634 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd514065a60 0x7fd514067f20 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fd51c0011f0 tx=0x7fd51c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:30.582 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.581+0000 7fd52affd640 1 -- 172.21.15.49:0/1184649634 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fd5240ea670 con 0x7fd5340b0790 2023-11-06T02:11:30.738 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.741+0000 7fd5388ab640 1 -- 172.21.15.49:0/1184649634 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd534074040 con 0x7fd5340b0790 2023-11-06T02:11:30.740 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.741+0000 7fd52affd640 1 -- 172.21.15.49:0/1184649634 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+17838 (secure 0 0 0) 0x7fd5240b0020 con 0x7fd5340b0790 2023-11-06T02:11:30.741 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:11:30.741 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":64,"fsid":"1a8b035c-6ba4-499a-b876-aea47fd9c0d4","created":"2023-11-06T02:08:06.440857+0000","modified":"2023-11-06T02:11:30.090572+0000","last_up_change":"2023-11-06T02:08:17.982372+0000","last_in_change":"2023-11-06T02:08:08.906631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T02:08:19.252222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T02:08:24.776211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-06T02:11:28.648326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-06T02:11:29.079513+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ad55af8-0571-4e6a-8722-b5d429b7496d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6809","nonce":38116},{"type":"v1","addr":"172.21.15.49:6811","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6813","nonce":38116},{"type":"v1","addr":"172.21.15.49:6815","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6821","nonce":38116},{"type":"v1","addr":"172.21.15.49:6823","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6817","nonce":38116},{"type":"v1","addr":"172.21.15.49:6819","nonce":38116}]},"public_addr":"172.21.15.49:6811/38116","cluster_addr":"172.21.15.49:6815/38116","heartbeat_back_addr":"172.21.15.49:6823/38116","heartbeat_front_addr":"172.21.15.49:6819/38116","state":["exists","up"]},{"osd":1,"uuid":"c8b31f85-a757-43ed-9419-351acc46611b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":38113},{"type":"v1","addr":"172.21.15.49:6810","nonce":38113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":38113},{"type":"v1","addr":"172.21.15.49:6814","nonce":38113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":38113},{"type":"v1","addr":"172.21.15.49:6822","nonce":38113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":38113},{"type":"v1","addr":"172.21.15.49:6818","nonce":38113}]},"public_addr":"172.21.15.49:6810/38113","cluster_addr":"172.21.15.49:6814/38113","heartbeat_back_addr":"172.21.15.49:6822/38113","heartbeat_front_addr":"172.21.15.49:6818/38113","state":["exists","up"]},{"osd":2,"uuid":"555b3bf3-0a70-4b91-8efd-e4204b5fc26e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":38114},{"type":"v1","addr":"172.21.15.49:6801","nonce":38114}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":38114},{"type":"v1","addr":"172.21.15.49:6803","nonce":38114}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":38114},{"type":"v1","addr":"172.21.15.49:6807","nonce":38114}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":38114},{"type":"v1","addr":"172.21.15.49:6805","nonce":38114}]},"public_addr":"172.21.15.49:6801/38114","cluster_addr":"172.21.15.49:6803/38114","heartbeat_back_addr":"172.21.15.49:6807/38114","heartbeat_front_addr":"172.21.15.49:6805/38114","state":["exists","up"]},{"osd":3,"uuid":"8d4e5d77-03ef-4b9b-bfcb-5ae93a46795c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":38115},{"type":"v1","addr":"172.21.15.49:6825","nonce":38115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":38115},{"type":"v1","addr":"172.21.15.49:6827","nonce":38115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":38115},{"type":"v1","addr":"172.21.15.49:6831","nonce":38115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":38115},{"type":"v1","addr":"172.21.15.49:6829","nonce":38115}]},"public_addr":"172.21.15.49:6825/38115","cluster_addr":"172.21.15.49:6827/38115","heartbeat_back_addr":"172.21.15.49:6831/38115","heartbeat_front_addr":"172.21.15.49:6829/38115","state":["exists","up"]},{"osd":4,"uuid":"dba03ad4-68a3-4cf3-9218-ac611f02eacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":38417},{"type":"v1","addr":"172.21.15.150:6809","nonce":38417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":38417},{"type":"v1","addr":"172.21.15.150:6811","nonce":38417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":38417},{"type":"v1","addr":"172.21.15.150:6815","nonce":38417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":38417},{"type":"v1","addr":"172.21.15.150:6813","nonce":38417}]},"public_addr":"172.21.15.150:6809/38417","cluster_addr":"172.21.15.150:6811/38417","heartbeat_back_addr":"172.21.15.150:6815/38417","heartbeat_front_addr":"172.21.15.150:6813/38417","state":["exists","up"]},{"osd":5,"uuid":"0df9c303-c255-4640-8e91-01f1c3efe18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":38418},{"type":"v1","addr":"172.21.15.150:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":38418},{"type":"v1","addr":"172.21.15.150:6803","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":38418},{"type":"v1","addr":"172.21.15.150:6807","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":38418},{"type":"v1","addr":"172.21.15.150:6805","nonce":38418}]},"public_addr":"172.21.15.150:6801/38418","cluster_addr":"172.21.15.150:6803/38418","heartbeat_back_addr":"172.21.15.150:6807/38418","heartbeat_front_addr":"172.21.15.150:6805/38418","state":["exists","up"]},{"osd":6,"uuid":"e59fa189-4b70-4c06-9722-c02b90828bf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":38428},{"type":"v1","addr":"172.21.15.150:6825","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":38428},{"type":"v1","addr":"172.21.15.150:6827","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":38428},{"type":"v1","addr":"172.21.15.150:6831","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":38428},{"type":"v1","addr":"172.21.15.150:6829","nonce":38428}]},"public_addr":"172.21.15.150:6825/38428","cluster_addr":"172.21.15.150:6827/38428","heartbeat_back_addr":"172.21.15.150:6831/38428","heartbeat_front_addr":"172.21.15.150:6829/38428","state":["exists","up"]},{"osd":7,"uuid":"a64f57e1-ef98-46bd-b304-385759e95898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":38421},{"type":"v1","addr":"172.21.15.150:6817","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":38421},{"type":"v1","addr":"172.21.15.150:6819","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":38421},{"type":"v1","addr":"172.21.15.150:6823","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":38421},{"type":"v1","addr":"172.21.15.150:6821","nonce":38421}]},"public_addr":"172.21.15.150:6817/38421","cluster_addr":"172.21.15.150:6819/38421","heartbeat_back_addr":"172.21.15.150:6823/38421","heartbeat_front_addr":"172.21.15.150:6821/38421","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.438552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.413280+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.296466+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.424966+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.276953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.230984+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.410235+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.480084+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T02:11:30.743 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.745+0000 7fd5388ab640 1 -- 172.21.15.49:0/1184649634 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd514065a60 msgr2=0x7fd514067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:30.743 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.745+0000 7fd5388ab640 1 --2- 172.21.15.49:0/1184649634 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd514065a60 0x7fd514067f20 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fd51c0011f0 tx=0x7fd51c009040 comp rx=0 tx=0).stop 2023-11-06T02:11:30.744 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.745+0000 7fd5388ab640 1 -- 172.21.15.49:0/1184649634 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd5340b0790 msgr2=0x7fd5341f68c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:30.744 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.745+0000 7fd5388ab640 1 --2- 172.21.15.49:0/1184649634 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd5340b0790 0x7fd5341f68c0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fd524018f60 tx=0x7fd5240190f0 comp rx=0 tx=0).stop 2023-11-06T02:11:30.744 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.745+0000 7fd5388ab640 1 -- 172.21.15.49:0/1184649634 shutdown_connections 2023-11-06T02:11:30.744 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.745+0000 7fd5388ab640 1 --2- 172.21.15.49:0/1184649634 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd514065a60 0x7fd514067f20 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:30.745 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.745+0000 7fd5388ab640 1 --2- 172.21.15.49:0/1184649634 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd53415ab50 0x7fd534148150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:30.745 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.745+0000 7fd5388ab640 1 --2- 172.21.15.49:0/1184649634 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5340cb010 0x7fd53414eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:30.745 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.745+0000 7fd5388ab640 1 --2- 172.21.15.49:0/1184649634 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd5340b0790 0x7fd5341f68c0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:30.745 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.745+0000 7fd5388ab640 1 -- 172.21.15.49:0/1184649634 >> 172.21.15.49:0/1184649634 conn(0x7fd53406c0a0 msgr2=0x7fd5340c9bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:30.745 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.745+0000 7fd5388ab640 1 -- 172.21.15.49:0/1184649634 shutdown_connections 2023-11-06T02:11:30.745 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.745+0000 7fd5388ab640 1 -- 172.21.15.49:0/1184649634 wait complete. 2023-11-06T02:11:30.761 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-06T02:11:30.886 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.889+0000 7f678a807640 1 Processor -- start 2023-11-06T02:11:30.887 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.889+0000 7f678a807640 1 -- start start 2023-11-06T02:11:30.888 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.889+0000 7f678a807640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f678415a880 0x7f67840b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:30.888 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.889+0000 7f678a807640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f6784070160 0x7f6784070610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:30.888 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.889+0000 7f678a807640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67840ae360 con 0x7f678406d150 2023-11-06T02:11:30.889 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.889+0000 7f678a807640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6784070b50 con 0x7f6784070160 2023-11-06T02:11:30.889 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.889+0000 7f678a807640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67840ae640 con 0x7f678415a880 2023-11-06T02:11:30.889 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.889+0000 7f67837fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f6784070160 0x7f6784070610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:30.889 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.889+0000 7f67837fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f6784070160 0x7f6784070610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48802/0 (socket says 172.21.15.49:48802) 2023-11-06T02:11:30.889 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.889+0000 7f67837fe640 1 -- 172.21.15.49:0/537048361 learned_addr learned my addr 172.21.15.49:0/537048361 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:30.890 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.889+0000 7f6783fff640 1 --2- 172.21.15.49:0/537048361 >> v2:172.21.15.150:3300/0 conn(0x7f678415a880 0x7f67840b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:30.890 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.889+0000 7f67837fe640 1 -- 172.21.15.49:0/537048361 >> v2:172.21.15.150:3300/0 conn(0x7f678415a880 msgr2=0x7f67840b15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:30.890 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.889+0000 7f67837fe640 1 --2- 172.21.15.49:0/537048361 >> v2:172.21.15.150:3300/0 conn(0x7f678415a880 0x7f67840b15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:30.890 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.889+0000 7f67837fe640 1 -- 172.21.15.49:0/537048361 >> v1:172.21.15.150:6789/0 conn(0x7f678406d150 legacy=0x7f678406d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:30.890 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.889+0000 7f67837fe640 1 -- 172.21.15.49:0/537048361 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67840ae920 con 0x7f6784070160 2023-11-06T02:11:30.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.893+0000 7f67837fe640 1 --2- 172.21.15.49:0/537048361 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6784070160 0x7f6784070610 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f67840cafc0 tx=0x7f677801c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b1dda537d5b2a49 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:30.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.893+0000 7f6782ffd640 1 -- 172.21.15.49:0/537048361 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67780ac630 con 0x7f6784070160 2023-11-06T02:11:30.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.893+0000 7f6782ffd640 1 -- 172.21.15.49:0/537048361 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67780ac790 con 0x7f6784070160 2023-11-06T02:11:30.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.893+0000 7f6782ffd640 1 -- 172.21.15.49:0/537048361 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67780acb20 con 0x7f6784070160 2023-11-06T02:11:30.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.893+0000 7f678a807640 1 -- 172.21.15.49:0/537048361 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6784070160 msgr2=0x7f6784070610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:30.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.893+0000 7f678a807640 1 --2- 172.21.15.49:0/537048361 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6784070160 0x7f6784070610 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f67840cafc0 tx=0x7f677801c4c0 comp rx=0 tx=0).stop 2023-11-06T02:11:30.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.893+0000 7f678a807640 1 -- 172.21.15.49:0/537048361 shutdown_connections 2023-11-06T02:11:30.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.893+0000 7f678a807640 1 --2- 172.21.15.49:0/537048361 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6784070160 0x7f6784070610 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:30.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.893+0000 7f678a807640 1 --2- 172.21.15.49:0/537048361 >> v2:172.21.15.150:3300/0 conn(0x7f678415a880 0x7f67840b15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:30.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.893+0000 7f678a807640 1 -- 172.21.15.49:0/537048361 >> 172.21.15.49:0/537048361 conn(0x7f678406a370 msgr2=0x7f67840ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:30.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.893+0000 7f678a807640 1 -- 172.21.15.49:0/537048361 shutdown_connections 2023-11-06T02:11:30.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.893+0000 7f678a807640 1 -- 172.21.15.49:0/537048361 wait complete. 2023-11-06T02:11:30.896 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.893+0000 7f678a807640 1 Processor -- start 2023-11-06T02:11:30.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.893+0000 7f678a807640 1 -- start start 2023-11-06T02:11:30.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.897+0000 7f678a807640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f678406d150 0x7f6784159ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:30.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.897+0000 7f678a807640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6784070160 0x7f678414cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:30.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.897+0000 7f678a807640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f678415a880 0x7f67840b2d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:30.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.897+0000 7f678a807640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f678414d150 con 0x7f678415a880 2023-11-06T02:11:30.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.897+0000 7f678a807640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67840c3660 con 0x7f678406d150 2023-11-06T02:11:30.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.897+0000 7f678a807640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67840c3940 con 0x7f6784070160 2023-11-06T02:11:30.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.897+0000 7f6788da4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f678415a880 0x7f67840b2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:30.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.897+0000 7f6788da4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f678415a880 0x7f67840b2d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48804/0 (socket says 172.21.15.49:48804) 2023-11-06T02:11:30.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.897+0000 7f6788da4640 1 -- 172.21.15.49:0/626348613 learned_addr learned my addr 172.21.15.49:0/626348613 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:30.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.897+0000 7f6783fff640 1 --2- 172.21.15.49:0/626348613 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f678406d150 0x7f6784159ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:30.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.897+0000 7f67837fe640 1 --2- 172.21.15.49:0/626348613 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6784070160 0x7f678414cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:30.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.897+0000 7f6788da4640 1 -- 172.21.15.49:0/626348613 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6784070160 msgr2=0x7f678414cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:30.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.897+0000 7f6788da4640 1 --2- 172.21.15.49:0/626348613 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6784070160 0x7f678414cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:30.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.897+0000 7f6788da4640 1 -- 172.21.15.49:0/626348613 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f678406d150 msgr2=0x7f6784159ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:30.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.897+0000 7f6788da4640 1 --2- 172.21.15.49:0/626348613 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f678406d150 0x7f6784159ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:30.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.897+0000 7f6788da4640 1 -- 172.21.15.49:0/626348613 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67780094d0 con 0x7f678415a880 2023-11-06T02:11:30.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.897+0000 7f67837fe640 1 --2- 172.21.15.49:0/626348613 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6784070160 0x7f678414cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:11:30.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.897+0000 7f6783fff640 1 --2- 172.21.15.49:0/626348613 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f678406d150 0x7f6784159ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:11:30.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.897+0000 7f6788da4640 1 --2- 172.21.15.49:0/626348613 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f678415a880 0x7f67840b2d10 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f677c00c780 tx=0x7f677c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:30.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.897+0000 7f6780ff9640 1 -- 172.21.15.49:0/626348613 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f677c004b70 con 0x7f678415a880 2023-11-06T02:11:30.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.897+0000 7f6780ff9640 1 -- 172.21.15.49:0/626348613 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f677c00ce90 con 0x7f678415a880 2023-11-06T02:11:30.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.897+0000 7f678a807640 1 -- 172.21.15.49:0/626348613 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67840c60b0 con 0x7f678415a880 2023-11-06T02:11:30.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.897+0000 7f6780ff9640 1 -- 172.21.15.49:0/626348613 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f677c00a850 con 0x7f678415a880 2023-11-06T02:11:30.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.897+0000 7f678a807640 1 -- 172.21.15.49:0/626348613 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67840c5cd0 con 0x7f678415a880 2023-11-06T02:11:30.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.897+0000 7f678a807640 1 -- 172.21.15.49:0/626348613 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6784074040 con 0x7f678415a880 2023-11-06T02:11:30.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.897+0000 7f6780ff9640 1 -- 172.21.15.49:0/626348613 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f677c00a9b0 con 0x7f678415a880 2023-11-06T02:11:30.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.897+0000 7f6780ff9640 1 --2- 172.21.15.49:0/626348613 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6770065a60 0x7f6770067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:30.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.897+0000 7f6780ff9640 1 -- 172.21.15.49:0/626348613 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f677c083420 con 0x7f678415a880 2023-11-06T02:11:30.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.901+0000 7f6783fff640 1 --2- 172.21.15.49:0/626348613 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6770065a60 0x7f6770067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:30.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.901+0000 7f6783fff640 1 --2- 172.21.15.49:0/626348613 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6770065a60 0x7f6770067f20 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f67840b3670 tx=0x7f6774008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:30.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:30.901+0000 7f6780ff9640 1 -- 172.21.15.49:0/626348613 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f677c050d10 con 0x7f678415a880 2023-11-06T02:11:31.088 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:31.089+0000 7f678a807640 1 -- 172.21.15.49:0/626348613 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f67840c6560 con 0x7f678415a880 2023-11-06T02:11:31.557 INFO:tasks.ceph.mds.c.smithi049.stderr:2023-11-06T02:11:31.561+0000 7fe85a2b1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-06T02:11:32.109 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:32.113+0000 7f6780ff9640 1 -- 172.21.15.49:0/626348613 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v29) v1 ==== 118+0+0 (secure 0 0 0) 0x7f677c054330 con 0x7f678415a880 2023-11-06T02:11:32.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:32.113+0000 7f678a807640 1 -- 172.21.15.49:0/626348613 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6770065a60 msgr2=0x7f6770067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:32.113 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:32.113+0000 7f678a807640 1 --2- 172.21.15.49:0/626348613 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6770065a60 0x7f6770067f20 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f67840b3670 tx=0x7f6774008040 comp rx=0 tx=0).stop 2023-11-06T02:11:32.113 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:32.117+0000 7f678a807640 1 -- 172.21.15.49:0/626348613 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f678415a880 msgr2=0x7f67840b2d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:32.113 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:32.117+0000 7f678a807640 1 --2- 172.21.15.49:0/626348613 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f678415a880 0x7f67840b2d10 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f677c00c780 tx=0x7f677c00cc40 comp rx=0 tx=0).stop 2023-11-06T02:11:32.114 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:32.117+0000 7f678a807640 1 -- 172.21.15.49:0/626348613 shutdown_connections 2023-11-06T02:11:32.114 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:32.117+0000 7f678a807640 1 --2- 172.21.15.49:0/626348613 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6770065a60 0x7f6770067f20 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:32.114 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:32.117+0000 7f678a807640 1 --2- 172.21.15.49:0/626348613 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f678415a880 0x7f67840b2d10 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:32.115 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:32.117+0000 7f678a807640 1 --2- 172.21.15.49:0/626348613 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6784070160 0x7f678414cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:32.115 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:32.117+0000 7f678a807640 1 --2- 172.21.15.49:0/626348613 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f678406d150 0x7f6784159ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:32.115 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:32.117+0000 7f678a807640 1 -- 172.21.15.49:0/626348613 >> 172.21.15.49:0/626348613 conn(0x7f678406a370 msgr2=0x7f678415c950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:32.115 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:32.117+0000 7f678a807640 1 -- 172.21.15.49:0/626348613 shutdown_connections 2023-11-06T02:11:32.115 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:32.117+0000 7f678a807640 1 -- 172.21.15.49:0/626348613 wait complete. 2023-11-06T02:11:32.136 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-06T02:11:32.136 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-06T02:11:32.273 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:32.273+0000 7f2f73b5a640 1 Processor -- start 2023-11-06T02:11:32.273 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:32.277+0000 7f2f73b5a640 1 -- start start 2023-11-06T02:11:32.275 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:32.277+0000 7f2f73b5a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2f6c0cf1f0 0x7f2f6c15dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:32.275 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:32.277+0000 7f2f73b5a640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f2f6c0cda10 0x7f2f6c0cf5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:32.275 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:32.277+0000 7f2f73b5a640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f6c0d2810 con 0x7f2f6c0d1fa0 2023-11-06T02:11:32.275 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:32.277+0000 7f2f73b5a640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f6c0d1b10 con 0x7f2f6c0cda10 2023-11-06T02:11:32.276 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:32.277+0000 7f2f73b5a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f6c0d0f70 con 0x7f2f6c0cf1f0 2023-11-06T02:11:32.276 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:32.277+0000 7f2f710f5640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f2f6c0cda10 0x7f2f6c0cf5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:32.276 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:32.277+0000 7f2f710f5640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f2f6c0cda10 0x7f2f6c0cf5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48812/0 (socket says 172.21.15.49:48812) 2023-11-06T02:11:32.276 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:32.277+0000 7f2f710f5640 1 -- 172.21.15.49:0/3568825589 learned_addr learned my addr 172.21.15.49:0/3568825589 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:32.277 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:32.277+0000 7f2f718f6640 1 --2- 172.21.15.49:0/3568825589 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2f6c0cf1f0 0x7f2f6c15dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:32.277 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:32.277+0000 7f2f710f5640 1 -- 172.21.15.49:0/3568825589 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2f6c0cf1f0 msgr2=0x7f2f6c15dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:32.277 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:32.277+0000 7f2f710f5640 1 --2- 172.21.15.49:0/3568825589 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2f6c0cf1f0 0x7f2f6c15dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:32.277 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:32.277+0000 7f2f710f5640 1 -- 172.21.15.49:0/3568825589 >> v1:172.21.15.150:6789/0 conn(0x7f2f6c0d1fa0 legacy=0x7f2f6c15e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:32.278 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:32.277+0000 7f2f710f5640 1 -- 172.21.15.49:0/3568825589 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f6c15e190 con 0x7f2f6c0cda10 2023-11-06T02:11:32.283 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:32.281+0000 7f2f710f5640 1 --2- 172.21.15.49:0/3568825589 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f6c0cda10 0x7f2f6c0cf5d0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c009870 tx=0x7f2f5c01c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=949a7045fc1aaa35 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:32.283 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:32.281+0000 7f2f708f4640 1 -- 172.21.15.49:0/3568825589 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f5c0aa430 con 0x7f2f6c0cda10 2023-11-06T02:11:32.283 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:32.281+0000 7f2f708f4640 1 -- 172.21.15.49:0/3568825589 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f5c0aa590 con 0x7f2f6c0cda10 2023-11-06T02:11:32.283 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:32.281+0000 7f2f708f4640 1 -- 172.21.15.49:0/3568825589 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f5c0aa920 con 0x7f2f6c0cda10 2023-11-06T02:11:32.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:32.281+0000 7f2f73b5a640 1 -- 172.21.15.49:0/3568825589 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f6c0cda10 msgr2=0x7f2f6c0cf5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:32.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:32.281+0000 7f2f73b5a640 1 --2- 172.21.15.49:0/3568825589 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f6c0cda10 0x7f2f6c0cf5d0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c009870 tx=0x7f2f5c01c510 comp rx=0 tx=0).stop 2023-11-06T02:11:32.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:32.281+0000 7f2f73b5a640 1 -- 172.21.15.49:0/3568825589 shutdown_connections 2023-11-06T02:11:32.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:32.281+0000 7f2f73b5a640 1 --2- 172.21.15.49:0/3568825589 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f6c0cda10 0x7f2f6c0cf5d0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:32.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:32.281+0000 7f2f73b5a640 1 --2- 172.21.15.49:0/3568825589 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2f6c0cf1f0 0x7f2f6c15dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:32.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:32.281+0000 7f2f73b5a640 1 -- 172.21.15.49:0/3568825589 >> 172.21.15.49:0/3568825589 conn(0x7f2f6c071aa0 msgr2=0x7f2f6c06b110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:32.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:32.281+0000 7f2f73b5a640 1 -- 172.21.15.49:0/3568825589 shutdown_connections 2023-11-06T02:11:32.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:32.281+0000 7f2f73b5a640 1 -- 172.21.15.49:0/3568825589 wait complete. 2023-11-06T02:11:32.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:32.281+0000 7f2f73b5a640 1 Processor -- start 2023-11-06T02:11:32.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:32.281+0000 7f2f73b5a640 1 -- start start 2023-11-06T02:11:32.286 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:32.281+0000 7f2f73b5a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f6c0cda10 0x7f2f6c1531e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:32.286 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:32.281+0000 7f2f73b5a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f6c0cf1f0 0x7f2f6c159d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:32.286 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:32.281+0000 7f2f73b5a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2f6c0d1fa0 0x7f2f6c159350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:32.286 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:32.281+0000 7f2f73b5a640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f6c0c65c0 con 0x7f2f6c0cf1f0 2023-11-06T02:11:32.286 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:32.281+0000 7f2f73b5a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f6c0cd090 con 0x7f2f6c0cda10 2023-11-06T02:11:32.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:32.281+0000 7f2f73b5a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f6c0c67d0 con 0x7f2f6c0d1fa0 2023-11-06T02:11:32.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:32.281+0000 7f2f710f5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f6c0cf1f0 0x7f2f6c159d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:32.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:32.281+0000 7f2f710f5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f6c0cf1f0 0x7f2f6c159d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48816/0 (socket says 172.21.15.49:48816) 2023-11-06T02:11:32.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:32.281+0000 7f2f710f5640 1 -- 172.21.15.49:0/1599847355 learned_addr learned my addr 172.21.15.49:0/1599847355 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:32.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:32.281+0000 7f2f718f6640 1 --2- 172.21.15.49:0/1599847355 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f6c0cda10 0x7f2f6c1531e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:32.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:32.281+0000 7f2f720f7640 1 --2- 172.21.15.49:0/1599847355 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2f6c0d1fa0 0x7f2f6c159350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:32.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:32.281+0000 7f2f710f5640 1 -- 172.21.15.49:0/1599847355 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2f6c0d1fa0 msgr2=0x7f2f6c159350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:32.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:32.281+0000 7f2f710f5640 1 --2- 172.21.15.49:0/1599847355 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2f6c0d1fa0 0x7f2f6c159350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:32.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:32.281+0000 7f2f710f5640 1 -- 172.21.15.49:0/1599847355 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f6c0cda10 msgr2=0x7f2f6c1531e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:32.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:32.281+0000 7f2f710f5640 1 --2- 172.21.15.49:0/1599847355 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f6c0cda10 0x7f2f6c1531e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:32.289 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:32.281+0000 7f2f710f5640 1 -- 172.21.15.49:0/1599847355 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f5c0094d0 con 0x7f2f6c0cf1f0 2023-11-06T02:11:32.289 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:32.281+0000 7f2f718f6640 1 --2- 172.21.15.49:0/1599847355 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f6c0cda10 0x7f2f6c1531e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:11:32.289 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:32.281+0000 7f2f720f7640 1 --2- 172.21.15.49:0/1599847355 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2f6c0d1fa0 0x7f2f6c159350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:11:32.289 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:32.281+0000 7f2f710f5640 1 --2- 172.21.15.49:0/1599847355 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f6c0cf1f0 0x7f2f6c159d40 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c01abc0 tx=0x7f2f5c01ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:32.289 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:32.281+0000 7f2f627fc640 1 -- 172.21.15.49:0/1599847355 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f5c0b3800 con 0x7f2f6c0cf1f0 2023-11-06T02:11:32.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:32.281+0000 7f2f627fc640 1 -- 172.21.15.49:0/1599847355 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f5c0b3960 con 0x7f2f6c0cf1f0 2023-11-06T02:11:32.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:32.281+0000 7f2f73b5a640 1 -- 172.21.15.49:0/1599847355 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f6c0c6ab0 con 0x7f2f6c0cf1f0 2023-11-06T02:11:32.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:32.281+0000 7f2f627fc640 1 -- 172.21.15.49:0/1599847355 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f5c0b3d10 con 0x7f2f6c0cf1f0 2023-11-06T02:11:32.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:32.281+0000 7f2f73b5a640 1 -- 172.21.15.49:0/1599847355 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f6c0c7070 con 0x7f2f6c0cf1f0 2023-11-06T02:11:32.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:32.285+0000 7f2f73b5a640 1 -- 172.21.15.49:0/1599847355 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f6c0c7260 con 0x7f2f6c0cf1f0 2023-11-06T02:11:32.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:32.285+0000 7f2f627fc640 1 -- 172.21.15.49:0/1599847355 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f2f5c0b2380 con 0x7f2f6c0cf1f0 2023-11-06T02:11:32.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:32.285+0000 7f2f627fc640 1 --2- 172.21.15.49:0/1599847355 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2f50065a60 0x7f2f50067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:32.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:32.285+0000 7f2f627fc640 1 -- 172.21.15.49:0/1599847355 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2f5c11d1e0 con 0x7f2f6c0cf1f0 2023-11-06T02:11:32.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:32.285+0000 7f2f718f6640 1 --2- 172.21.15.49:0/1599847355 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2f50065a60 0x7f2f50067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:32.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:32.285+0000 7f2f718f6640 1 --2- 172.21.15.49:0/1599847355 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2f50065a60 0x7f2f50067f20 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c159cd0 tx=0x7f2f68008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:32.293 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:32.289+0000 7f2f627fc640 1 -- 172.21.15.49:0/1599847355 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f2f5c0eaad0 con 0x7f2f6c0cf1f0 2023-11-06T02:11:32.480 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:32.481+0000 7f2f73b5a640 1 -- 172.21.15.49:0/1599847355 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f2f6c067140 con 0x7f2f6c0cf1f0 2023-11-06T02:11:33.175 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.177+0000 7f2f627fc640 1 -- 172.21.15.49:0/1599847355 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v30) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2f5c0ee0f0 con 0x7f2f6c0cf1f0 2023-11-06T02:11:33.179 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.181+0000 7f2f73b5a640 1 -- 172.21.15.49:0/1599847355 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2f50065a60 msgr2=0x7f2f50067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:33.179 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.181+0000 7f2f73b5a640 1 --2- 172.21.15.49:0/1599847355 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2f50065a60 0x7f2f50067f20 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c159cd0 tx=0x7f2f68008040 comp rx=0 tx=0).stop 2023-11-06T02:11:33.180 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.181+0000 7f2f73b5a640 1 -- 172.21.15.49:0/1599847355 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f6c0cf1f0 msgr2=0x7f2f6c159d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:33.180 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.181+0000 7f2f73b5a640 1 --2- 172.21.15.49:0/1599847355 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f6c0cf1f0 0x7f2f6c159d40 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c01abc0 tx=0x7f2f5c01ad50 comp rx=0 tx=0).stop 2023-11-06T02:11:33.180 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.181+0000 7f2f73b5a640 1 -- 172.21.15.49:0/1599847355 shutdown_connections 2023-11-06T02:11:33.181 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.181+0000 7f2f73b5a640 1 --2- 172.21.15.49:0/1599847355 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2f50065a60 0x7f2f50067f20 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:33.181 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.181+0000 7f2f73b5a640 1 --2- 172.21.15.49:0/1599847355 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f6c0cf1f0 0x7f2f6c159d40 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:33.181 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.181+0000 7f2f73b5a640 1 --2- 172.21.15.49:0/1599847355 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2f6c0d1fa0 0x7f2f6c159350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:33.182 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.181+0000 7f2f73b5a640 1 --2- 172.21.15.49:0/1599847355 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f6c0cda10 0x7f2f6c1531e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:33.182 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.181+0000 7f2f73b5a640 1 -- 172.21.15.49:0/1599847355 >> 172.21.15.49:0/1599847355 conn(0x7f2f6c071aa0 msgr2=0x7f2f6c156f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:33.182 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.181+0000 7f2f73b5a640 1 -- 172.21.15.49:0/1599847355 shutdown_connections 2023-11-06T02:11:33.183 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.181+0000 7f2f73b5a640 1 -- 172.21.15.49:0/1599847355 wait complete. 2023-11-06T02:11:33.201 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-06T02:11:33.202 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T02:11:33.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.333+0000 7f802413a640 1 Processor -- start 2023-11-06T02:11:33.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.333+0000 7f802413a640 1 -- start start 2023-11-06T02:11:33.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.333+0000 7f802413a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f801c0b1f90 0x7f801c0cda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:33.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.333+0000 7f802413a640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f801c0714a0 0x7f801c0718f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:33.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.333+0000 7f802413a640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f801c071e30 con 0x7f801c161650 2023-11-06T02:11:33.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.333+0000 7f802413a640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f801c0d17c0 con 0x7f801c0714a0 2023-11-06T02:11:33.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.333+0000 7f802413a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f801c0ce6c0 con 0x7f801c0b1f90 2023-11-06T02:11:33.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.333+0000 7f80226d7640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f801c0714a0 0x7f801c0718f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:33.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.333+0000 7f80226d7640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f801c0714a0 0x7f801c0718f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48828/0 (socket says 172.21.15.49:48828) 2023-11-06T02:11:33.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.333+0000 7f80226d7640 1 -- 172.21.15.49:0/485115927 learned_addr learned my addr 172.21.15.49:0/485115927 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:33.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.333+0000 7f8021ed6640 1 --2- 172.21.15.49:0/485115927 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f801c0b1f90 0x7f801c0cda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:33.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.333+0000 7f80226d7640 1 -- 172.21.15.49:0/485115927 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f801c0b1f90 msgr2=0x7f801c0cda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:33.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.333+0000 7f80226d7640 1 --2- 172.21.15.49:0/485115927 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f801c0b1f90 0x7f801c0cda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:33.334 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.333+0000 7f80226d7640 1 -- 172.21.15.49:0/485115927 >> v1:172.21.15.150:6789/0 conn(0x7f801c161650 legacy=0x7f801c0d1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:33.334 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.333+0000 7f80226d7640 1 -- 172.21.15.49:0/485115927 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f801c161ad0 con 0x7f801c0714a0 2023-11-06T02:11:33.335 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.337+0000 7f80226d7640 1 --2- 172.21.15.49:0/485115927 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f801c0714a0 0x7f801c0718f0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f8014009870 tx=0x7f801401c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=546991236bca7bf2 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:33.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.337+0000 7f8020ed4640 1 -- 172.21.15.49:0/485115927 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80140a6040 con 0x7f801c0714a0 2023-11-06T02:11:33.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.337+0000 7f8020ed4640 1 -- 172.21.15.49:0/485115927 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80140aa600 con 0x7f801c0714a0 2023-11-06T02:11:33.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.337+0000 7f8020ed4640 1 -- 172.21.15.49:0/485115927 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80140aa990 con 0x7f801c0714a0 2023-11-06T02:11:33.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.337+0000 7f802413a640 1 -- 172.21.15.49:0/485115927 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f801c0714a0 msgr2=0x7f801c0718f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:33.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.337+0000 7f802413a640 1 --2- 172.21.15.49:0/485115927 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f801c0714a0 0x7f801c0718f0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f8014009870 tx=0x7f801401c4e0 comp rx=0 tx=0).stop 2023-11-06T02:11:33.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.337+0000 7f802413a640 1 -- 172.21.15.49:0/485115927 shutdown_connections 2023-11-06T02:11:33.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.337+0000 7f802413a640 1 --2- 172.21.15.49:0/485115927 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f801c0714a0 0x7f801c0718f0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:33.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.337+0000 7f802413a640 1 --2- 172.21.15.49:0/485115927 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f801c0b1f90 0x7f801c0cda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:33.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.337+0000 7f802413a640 1 -- 172.21.15.49:0/485115927 >> 172.21.15.49:0/485115927 conn(0x7f801c0d2aa0 msgr2=0x7f801c0d2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:33.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.337+0000 7f802413a640 1 -- 172.21.15.49:0/485115927 shutdown_connections 2023-11-06T02:11:33.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.337+0000 7f802413a640 1 -- 172.21.15.49:0/485115927 wait complete. 2023-11-06T02:11:33.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.337+0000 7f802413a640 1 Processor -- start 2023-11-06T02:11:33.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.337+0000 7f802413a640 1 -- start start 2023-11-06T02:11:33.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.337+0000 7f802413a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f801c0714a0 0x7f801c159c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:33.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.337+0000 7f802413a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f801c0b1f90 0x7f801c14e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:33.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.337+0000 7f802413a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f801c161650 0x7f801c0cc340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:33.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.337+0000 7f802413a640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f801c154d90 con 0x7f801c161650 2023-11-06T02:11:33.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.337+0000 7f802413a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f801c0c9ae0 con 0x7f801c0b1f90 2023-11-06T02:11:33.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.337+0000 7f802413a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f801c0c9dc0 con 0x7f801c0714a0 2023-11-06T02:11:33.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.337+0000 7f80226d7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f801c161650 0x7f801c0cc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:33.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.337+0000 7f80226d7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f801c161650 0x7f801c0cc340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48842/0 (socket says 172.21.15.49:48842) 2023-11-06T02:11:33.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.337+0000 7f80226d7640 1 -- 172.21.15.49:0/1704441095 learned_addr learned my addr 172.21.15.49:0/1704441095 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:33.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.337+0000 7f8021ed6640 1 --2- 172.21.15.49:0/1704441095 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f801c0714a0 0x7f801c159c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:33.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.341+0000 7f80216d5640 1 --2- 172.21.15.49:0/1704441095 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f801c0b1f90 0x7f801c14e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:33.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.341+0000 7f80226d7640 1 -- 172.21.15.49:0/1704441095 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f801c0714a0 msgr2=0x7f801c159c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:33.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.341+0000 7f80226d7640 1 --2- 172.21.15.49:0/1704441095 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f801c0714a0 0x7f801c159c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:33.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.341+0000 7f80226d7640 1 -- 172.21.15.49:0/1704441095 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f801c0b1f90 msgr2=0x7f801c14e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:33.344 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.341+0000 7f80226d7640 1 --2- 172.21.15.49:0/1704441095 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f801c0b1f90 0x7f801c14e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:33.344 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.341+0000 7f80226d7640 1 -- 172.21.15.49:0/1704441095 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80140094d0 con 0x7f801c161650 2023-11-06T02:11:33.344 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.341+0000 7f80226d7640 1 --2- 172.21.15.49:0/1704441095 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f801c161650 0x7f801c0cc340 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f801401ab90 tx=0x7f801401ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:33.344 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.341+0000 7f801a7fc640 1 -- 172.21.15.49:0/1704441095 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80140b3600 con 0x7f801c161650 2023-11-06T02:11:33.344 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.341+0000 7f801a7fc640 1 -- 172.21.15.49:0/1704441095 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80140b3760 con 0x7f801c161650 2023-11-06T02:11:33.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.341+0000 7f802413a640 1 -- 172.21.15.49:0/1704441095 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f801c159190 con 0x7f801c161650 2023-11-06T02:11:33.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.341+0000 7f801a7fc640 1 -- 172.21.15.49:0/1704441095 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80140b3b10 con 0x7f801c161650 2023-11-06T02:11:33.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.341+0000 7f802413a640 1 -- 172.21.15.49:0/1704441095 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f801c159750 con 0x7f801c161650 2023-11-06T02:11:33.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.341+0000 7f802413a640 1 -- 172.21.15.49:0/1704441095 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f801c158ff0 con 0x7f801c161650 2023-11-06T02:11:33.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.341+0000 7f801a7fc640 1 -- 172.21.15.49:0/1704441095 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f80140b3c70 con 0x7f801c161650 2023-11-06T02:11:33.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.341+0000 7f801a7fc640 1 --2- 172.21.15.49:0/1704441095 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8008065a90 0x7f8008067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:33.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.341+0000 7f801a7fc640 1 -- 172.21.15.49:0/1704441095 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f801411d370 con 0x7f801c161650 2023-11-06T02:11:33.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.341+0000 7f8021ed6640 1 --2- 172.21.15.49:0/1704441095 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8008065a90 0x7f8008067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:33.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.341+0000 7f8021ed6640 1 --2- 172.21.15.49:0/1704441095 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8008065a90 0x7f8008067f50 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f800c003fd0 tx=0x7f800c0039c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:33.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.345+0000 7f801a7fc640 1 -- 172.21.15.49:0/1704441095 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f801c158ff0 con 0x7f801c161650 2023-11-06T02:11:33.536 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.537+0000 7f802413a640 1 -- 172.21.15.49:0/1704441095 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f801c071e30 con 0x7f801c161650 2023-11-06T02:11:33.537 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.541+0000 7f801a7fc640 1 -- 172.21.15.49:0/1704441095 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+4652 (secure 0 0 0) 0x7f801c071e30 con 0x7f801c161650 2023-11-06T02:11:33.537 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:11:33.538 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":30,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5635,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4170291905","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":4170291905},{"type":"v1","addr":"172.21.15.150:6835","nonce":4170291905}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5638,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/2719458937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":2719458937},{"type":"v1","addr":"172.21.15.150:6837","nonce":2719458937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":6339,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/2310338375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2310338375},{"type":"v1","addr":"172.21.15.49:6833","nonce":2310338375}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T02:11:30.091070+0000","modified":"2023-11-06T02:11:33.173821+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6348},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6348":{"gid":6348,"name":"c","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.49:6835/342908487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":342908487},{"type":"v1","addr":"172.21.15.49:6835","nonce":342908487}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-06T02:11:33.538 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 30 2023-11-06T02:11:33.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.541+0000 7f802413a640 1 -- 172.21.15.49:0/1704441095 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8008065a90 msgr2=0x7f8008067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:33.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.541+0000 7f802413a640 1 --2- 172.21.15.49:0/1704441095 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8008065a90 0x7f8008067f50 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f800c003fd0 tx=0x7f800c0039c0 comp rx=0 tx=0).stop 2023-11-06T02:11:33.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.541+0000 7f802413a640 1 -- 172.21.15.49:0/1704441095 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f801c161650 msgr2=0x7f801c0cc340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:33.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.541+0000 7f802413a640 1 --2- 172.21.15.49:0/1704441095 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f801c161650 0x7f801c0cc340 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f801401ab90 tx=0x7f801401ad20 comp rx=0 tx=0).stop 2023-11-06T02:11:33.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.541+0000 7f802413a640 1 -- 172.21.15.49:0/1704441095 shutdown_connections 2023-11-06T02:11:33.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.541+0000 7f802413a640 1 --2- 172.21.15.49:0/1704441095 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8008065a90 0x7f8008067f50 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:33.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.541+0000 7f802413a640 1 --2- 172.21.15.49:0/1704441095 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f801c161650 0x7f801c0cc340 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:33.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.541+0000 7f802413a640 1 --2- 172.21.15.49:0/1704441095 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f801c0b1f90 0x7f801c14e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:33.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.541+0000 7f802413a640 1 --2- 172.21.15.49:0/1704441095 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f801c0714a0 0x7f801c159c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:33.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.541+0000 7f802413a640 1 -- 172.21.15.49:0/1704441095 >> 172.21.15.49:0/1704441095 conn(0x7f801c0d2aa0 msgr2=0x7f801c0b2f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:33.542 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.541+0000 7f802413a640 1 -- 172.21.15.49:0/1704441095 shutdown_connections 2023-11-06T02:11:33.542 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.541+0000 7f802413a640 1 -- 172.21.15.49:0/1704441095 wait complete. 2023-11-06T02:11:33.557 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T02:11:33.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.685+0000 7ff809a28640 1 Processor -- start 2023-11-06T02:11:33.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.685+0000 7ff809a28640 1 -- start start 2023-11-06T02:11:33.686 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.689+0000 7ff809a28640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff8040cda70 0x7ff804161650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:33.686 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.689+0000 7ff809a28640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8040714a0 0x7ff8040718d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:33.686 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.689+0000 7ff809a28640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8040d17c0 con 0x7ff8040b1f90 2023-11-06T02:11:33.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.689+0000 7ff809a28640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8040cdf20 con 0x7ff8040cda70 2023-11-06T02:11:33.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.689+0000 7ff809a28640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff804161b90 con 0x7ff8040714a0 2023-11-06T02:11:33.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.689+0000 7ff8027fc640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff8040cda70 0x7ff804161650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:33.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.689+0000 7ff8027fc640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff8040cda70 0x7ff804161650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48848/0 (socket says 172.21.15.49:48848) 2023-11-06T02:11:33.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.689+0000 7ff8027fc640 1 -- 172.21.15.49:0/2929032966 learned_addr learned my addr 172.21.15.49:0/2929032966 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:33.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.689+0000 7ff8037fe640 1 --2- 172.21.15.49:0/2929032966 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8040714a0 0x7ff8040718d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:33.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.689+0000 7ff8027fc640 1 -- 172.21.15.49:0/2929032966 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8040714a0 msgr2=0x7ff8040718d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:33.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.689+0000 7ff8027fc640 1 --2- 172.21.15.49:0/2929032966 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8040714a0 0x7ff8040718d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:33.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.689+0000 7ff8027fc640 1 -- 172.21.15.49:0/2929032966 >> v1:172.21.15.150:6789/0 conn(0x7ff8040b1f90 legacy=0x7ff8040d1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:33.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.689+0000 7ff8027fc640 1 -- 172.21.15.49:0/2929032966 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff804071e10 con 0x7ff8040cda70 2023-11-06T02:11:33.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.689+0000 7ff8027fc640 1 --2- 172.21.15.49:0/2929032966 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8040cda70 0x7ff804161650 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec0098a0 tx=0x7ff7ec01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d354c0e775429c6 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:33.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.689+0000 7ff801ffb640 1 -- 172.21.15.49:0/2929032966 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec0ab610 con 0x7ff8040cda70 2023-11-06T02:11:33.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.689+0000 7ff801ffb640 1 -- 172.21.15.49:0/2929032966 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7ec0ab770 con 0x7ff8040cda70 2023-11-06T02:11:33.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.689+0000 7ff801ffb640 1 -- 172.21.15.49:0/2929032966 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec0abb00 con 0x7ff8040cda70 2023-11-06T02:11:33.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.693+0000 7ff809a28640 1 -- 172.21.15.49:0/2929032966 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8040cda70 msgr2=0x7ff804161650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:33.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.693+0000 7ff809a28640 1 --2- 172.21.15.49:0/2929032966 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8040cda70 0x7ff804161650 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec0098a0 tx=0x7ff7ec01c520 comp rx=0 tx=0).stop 2023-11-06T02:11:33.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.693+0000 7ff809a28640 1 -- 172.21.15.49:0/2929032966 shutdown_connections 2023-11-06T02:11:33.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.693+0000 7ff809a28640 1 --2- 172.21.15.49:0/2929032966 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8040714a0 0x7ff8040718d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:33.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.693+0000 7ff809a28640 1 --2- 172.21.15.49:0/2929032966 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8040cda70 0x7ff804161650 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:33.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.693+0000 7ff809a28640 1 -- 172.21.15.49:0/2929032966 >> 172.21.15.49:0/2929032966 conn(0x7ff8040d2aa0 msgr2=0x7ff8040d2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:33.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.693+0000 7ff809a28640 1 -- 172.21.15.49:0/2929032966 shutdown_connections 2023-11-06T02:11:33.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.693+0000 7ff809a28640 1 -- 172.21.15.49:0/2929032966 wait complete. 2023-11-06T02:11:33.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.693+0000 7ff809a28640 1 Processor -- start 2023-11-06T02:11:33.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.693+0000 7ff809a28640 1 -- start start 2023-11-06T02:11:33.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.693+0000 7ff809a28640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8040714a0 0x7ff8040b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:33.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.693+0000 7ff809a28640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8040b1f90 0x7ff804157430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:33.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.693+0000 7ff809a28640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8040cda70 0x7ff8040b4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:33.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.693+0000 7ff809a28640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff804157970 con 0x7ff8040b1f90 2023-11-06T02:11:33.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.693+0000 7ff809a28640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8040cc960 con 0x7ff8040cda70 2023-11-06T02:11:33.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.693+0000 7ff809a28640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8040cc3a0 con 0x7ff8040714a0 2023-11-06T02:11:33.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.693+0000 7ff8027fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8040b1f90 0x7ff804157430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:33.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.693+0000 7ff8027fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8040b1f90 0x7ff804157430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48856/0 (socket says 172.21.15.49:48856) 2023-11-06T02:11:33.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.693+0000 7ff8027fc640 1 -- 172.21.15.49:0/3661602545 learned_addr learned my addr 172.21.15.49:0/3661602545 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:33.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.693+0000 7ff802ffd640 1 --2- 172.21.15.49:0/3661602545 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8040714a0 0x7ff8040b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:33.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.693+0000 7ff8037fe640 1 --2- 172.21.15.49:0/3661602545 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8040cda70 0x7ff8040b4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:33.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.693+0000 7ff8027fc640 1 -- 172.21.15.49:0/3661602545 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8040714a0 msgr2=0x7ff8040b7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:33.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.693+0000 7ff8027fc640 1 --2- 172.21.15.49:0/3661602545 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8040714a0 0x7ff8040b7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:33.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.693+0000 7ff8027fc640 1 -- 172.21.15.49:0/3661602545 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8040cda70 msgr2=0x7ff8040b4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:33.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.693+0000 7ff8027fc640 1 --2- 172.21.15.49:0/3661602545 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8040cda70 0x7ff8040b4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:33.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.693+0000 7ff8027fc640 1 -- 172.21.15.49:0/3661602545 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7ec0094d0 con 0x7ff8040b1f90 2023-11-06T02:11:33.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.693+0000 7ff8037fe640 1 --2- 172.21.15.49:0/3661602545 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8040cda70 0x7ff8040b4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:11:33.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.693+0000 7ff8027fc640 1 --2- 172.21.15.49:0/3661602545 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8040b1f90 0x7ff804157430 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec0ad040 tx=0x7ff7ec01ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:33.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.693+0000 7ff7fbfff640 1 -- 172.21.15.49:0/3661602545 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec0b4800 con 0x7ff8040b1f90 2023-11-06T02:11:33.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.693+0000 7ff7fbfff640 1 -- 172.21.15.49:0/3661602545 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7ec0b49a0 con 0x7ff8040b1f90 2023-11-06T02:11:33.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.693+0000 7ff809a28640 1 -- 172.21.15.49:0/3661602545 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8040cc680 con 0x7ff8040b1f90 2023-11-06T02:11:33.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.693+0000 7ff7fbfff640 1 -- 172.21.15.49:0/3661602545 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec0b4d50 con 0x7ff8040b1f90 2023-11-06T02:11:33.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.693+0000 7ff809a28640 1 -- 172.21.15.49:0/3661602545 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8040b4540 con 0x7ff8040b1f90 2023-11-06T02:11:33.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.697+0000 7ff809a28640 1 -- 172.21.15.49:0/3661602545 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8040b4090 con 0x7ff8040b1f90 2023-11-06T02:11:33.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.697+0000 7ff7fbfff640 1 -- 172.21.15.49:0/3661602545 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7ff7ec0c8430 con 0x7ff8040b1f90 2023-11-06T02:11:33.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.697+0000 7ff7fbfff640 1 --2- 172.21.15.49:0/3661602545 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff7e4065a60 0x7ff7e4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:33.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.697+0000 7ff7fbfff640 1 -- 172.21.15.49:0/3661602545 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff7ec11dc40 con 0x7ff8040b1f90 2023-11-06T02:11:33.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.697+0000 7ff802ffd640 1 --2- 172.21.15.49:0/3661602545 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff7e4065a60 0x7ff7e4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:33.701 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.697+0000 7ff802ffd640 1 --2- 172.21.15.49:0/3661602545 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff7e4065a60 0x7ff7e4067f20 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc000f70 tx=0x7ff7fc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:33.701 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.701+0000 7ff7fbfff640 1 -- 172.21.15.49:0/3661602545 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7ff7ec0eb630 con 0x7ff8040b1f90 2023-11-06T02:11:33.859 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.861+0000 7ff809a28640 1 -- 172.21.15.49:0/3661602545 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff8040b4730 con 0x7ff8040b1f90 2023-11-06T02:11:33.862 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.865+0000 7ff7fbfff640 1 -- 172.21.15.49:0/3661602545 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+17825 (secure 0 0 0) 0x7ff7ec0eec50 con 0x7ff8040b1f90 2023-11-06T02:11:33.863 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:11:33.863 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":65,"fsid":"1a8b035c-6ba4-499a-b876-aea47fd9c0d4","created":"2023-11-06T02:08:06.440857+0000","modified":"2023-11-06T02:11:31.094207+0000","last_up_change":"2023-11-06T02:08:17.982372+0000","last_in_change":"2023-11-06T02:08:08.906631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T02:08:19.252222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T02:08:24.776211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-06T02:11:28.648326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-06T02:11:29.079513+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ad55af8-0571-4e6a-8722-b5d429b7496d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6809","nonce":38116},{"type":"v1","addr":"172.21.15.49:6811","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6813","nonce":38116},{"type":"v1","addr":"172.21.15.49:6815","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6821","nonce":38116},{"type":"v1","addr":"172.21.15.49:6823","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6817","nonce":38116},{"type":"v1","addr":"172.21.15.49:6819","nonce":38116}]},"public_addr":"172.21.15.49:6811/38116","cluster_addr":"172.21.15.49:6815/38116","heartbeat_back_addr":"172.21.15.49:6823/38116","heartbeat_front_addr":"172.21.15.49:6819/38116","state":["exists","up"]},{"osd":1,"uuid":"c8b31f85-a757-43ed-9419-351acc46611b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":38113},{"type":"v1","addr":"172.21.15.49:6810","nonce":38113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":38113},{"type":"v1","addr":"172.21.15.49:6814","nonce":38113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":38113},{"type":"v1","addr":"172.21.15.49:6822","nonce":38113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":38113},{"type":"v1","addr":"172.21.15.49:6818","nonce":38113}]},"public_addr":"172.21.15.49:6810/38113","cluster_addr":"172.21.15.49:6814/38113","heartbeat_back_addr":"172.21.15.49:6822/38113","heartbeat_front_addr":"172.21.15.49:6818/38113","state":["exists","up"]},{"osd":2,"uuid":"555b3bf3-0a70-4b91-8efd-e4204b5fc26e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":38114},{"type":"v1","addr":"172.21.15.49:6801","nonce":38114}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":38114},{"type":"v1","addr":"172.21.15.49:6803","nonce":38114}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":38114},{"type":"v1","addr":"172.21.15.49:6807","nonce":38114}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":38114},{"type":"v1","addr":"172.21.15.49:6805","nonce":38114}]},"public_addr":"172.21.15.49:6801/38114","cluster_addr":"172.21.15.49:6803/38114","heartbeat_back_addr":"172.21.15.49:6807/38114","heartbeat_front_addr":"172.21.15.49:6805/38114","state":["exists","up"]},{"osd":3,"uuid":"8d4e5d77-03ef-4b9b-bfcb-5ae93a46795c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":38115},{"type":"v1","addr":"172.21.15.49:6825","nonce":38115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":38115},{"type":"v1","addr":"172.21.15.49:6827","nonce":38115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":38115},{"type":"v1","addr":"172.21.15.49:6831","nonce":38115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":38115},{"type":"v1","addr":"172.21.15.49:6829","nonce":38115}]},"public_addr":"172.21.15.49:6825/38115","cluster_addr":"172.21.15.49:6827/38115","heartbeat_back_addr":"172.21.15.49:6831/38115","heartbeat_front_addr":"172.21.15.49:6829/38115","state":["exists","up"]},{"osd":4,"uuid":"dba03ad4-68a3-4cf3-9218-ac611f02eacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":38417},{"type":"v1","addr":"172.21.15.150:6809","nonce":38417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":38417},{"type":"v1","addr":"172.21.15.150:6811","nonce":38417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":38417},{"type":"v1","addr":"172.21.15.150:6815","nonce":38417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":38417},{"type":"v1","addr":"172.21.15.150:6813","nonce":38417}]},"public_addr":"172.21.15.150:6809/38417","cluster_addr":"172.21.15.150:6811/38417","heartbeat_back_addr":"172.21.15.150:6815/38417","heartbeat_front_addr":"172.21.15.150:6813/38417","state":["exists","up"]},{"osd":5,"uuid":"0df9c303-c255-4640-8e91-01f1c3efe18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":38418},{"type":"v1","addr":"172.21.15.150:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":38418},{"type":"v1","addr":"172.21.15.150:6803","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":38418},{"type":"v1","addr":"172.21.15.150:6807","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":38418},{"type":"v1","addr":"172.21.15.150:6805","nonce":38418}]},"public_addr":"172.21.15.150:6801/38418","cluster_addr":"172.21.15.150:6803/38418","heartbeat_back_addr":"172.21.15.150:6807/38418","heartbeat_front_addr":"172.21.15.150:6805/38418","state":["exists","up"]},{"osd":6,"uuid":"e59fa189-4b70-4c06-9722-c02b90828bf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":38428},{"type":"v1","addr":"172.21.15.150:6825","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":38428},{"type":"v1","addr":"172.21.15.150:6827","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":38428},{"type":"v1","addr":"172.21.15.150:6831","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":38428},{"type":"v1","addr":"172.21.15.150:6829","nonce":38428}]},"public_addr":"172.21.15.150:6825/38428","cluster_addr":"172.21.15.150:6827/38428","heartbeat_back_addr":"172.21.15.150:6831/38428","heartbeat_front_addr":"172.21.15.150:6829/38428","state":["exists","up"]},{"osd":7,"uuid":"a64f57e1-ef98-46bd-b304-385759e95898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":38421},{"type":"v1","addr":"172.21.15.150:6817","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":38421},{"type":"v1","addr":"172.21.15.150:6819","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":38421},{"type":"v1","addr":"172.21.15.150:6823","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":38421},{"type":"v1","addr":"172.21.15.150:6821","nonce":38421}]},"public_addr":"172.21.15.150:6817/38421","cluster_addr":"172.21.15.150:6819/38421","heartbeat_back_addr":"172.21.15.150:6823/38421","heartbeat_front_addr":"172.21.15.150:6821/38421","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.438552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.413280+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.296466+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.424966+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.276953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.230984+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.410235+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.480084+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T02:11:33.864 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.865+0000 7ff809a28640 1 -- 172.21.15.49:0/3661602545 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff7e4065a60 msgr2=0x7ff7e4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:33.865 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.865+0000 7ff809a28640 1 --2- 172.21.15.49:0/3661602545 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff7e4065a60 0x7ff7e4067f20 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc000f70 tx=0x7ff7fc009040 comp rx=0 tx=0).stop 2023-11-06T02:11:33.865 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.865+0000 7ff809a28640 1 -- 172.21.15.49:0/3661602545 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8040b1f90 msgr2=0x7ff804157430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:33.866 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.865+0000 7ff809a28640 1 --2- 172.21.15.49:0/3661602545 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8040b1f90 0x7ff804157430 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec0ad040 tx=0x7ff7ec01ad60 comp rx=0 tx=0).stop 2023-11-06T02:11:33.866 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.865+0000 7ff809a28640 1 -- 172.21.15.49:0/3661602545 shutdown_connections 2023-11-06T02:11:33.867 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.865+0000 7ff809a28640 1 --2- 172.21.15.49:0/3661602545 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff7e4065a60 0x7ff7e4067f20 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:33.867 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.865+0000 7ff809a28640 1 --2- 172.21.15.49:0/3661602545 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8040b1f90 0x7ff804157430 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:33.867 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.865+0000 7ff809a28640 1 --2- 172.21.15.49:0/3661602545 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8040cda70 0x7ff8040b4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:33.868 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.865+0000 7ff809a28640 1 --2- 172.21.15.49:0/3661602545 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8040714a0 0x7ff8040b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:33.868 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.865+0000 7ff809a28640 1 -- 172.21.15.49:0/3661602545 >> 172.21.15.49:0/3661602545 conn(0x7ff8040d2aa0 msgr2=0x7ff8040d2e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:33.869 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.865+0000 7ff809a28640 1 -- 172.21.15.49:0/3661602545 shutdown_connections 2023-11-06T02:11:33.869 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:33.865+0000 7ff809a28640 1 -- 172.21.15.49:0/3661602545 wait complete. 2023-11-06T02:11:33.882 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-06T02:11:33.882 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T02:11:34.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.001+0000 7fccdd224640 1 Processor -- start 2023-11-06T02:11:34.001 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.001+0000 7fccdd224640 1 -- start start 2023-11-06T02:11:34.001 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.005+0000 7fccdd224640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fccd815ab50 0x7fccd80c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:34.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.005+0000 7fccdd224640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fccd80cb010 0x7fccd80b0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:34.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.005+0000 7fccdd224640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccd80cab80 con 0x7fccd80b05c0 2023-11-06T02:11:34.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.005+0000 7fccdd224640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccd80b0a50 con 0x7fccd815ab50 2023-11-06T02:11:34.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.005+0000 7fccdd224640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccd815af30 con 0x7fccd80cb010 2023-11-06T02:11:34.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.005+0000 7fccd659c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fccd80cb010 0x7fccd80b0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:34.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.005+0000 7fccd659c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fccd80cb010 0x7fccd80b0080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.49:54934/0 (socket says 172.21.15.49:54934) 2023-11-06T02:11:34.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.005+0000 7fccd659c640 1 -- 172.21.15.49:0/2175985829 learned_addr learned my addr 172.21.15.49:0/2175985829 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:34.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.005+0000 7fccd6d9d640 1 --2- 172.21.15.49:0/2175985829 >> v2:172.21.15.150:3300/0 conn(0x7fccd815ab50 0x7fccd80c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:34.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.005+0000 7fccd659c640 1 -- 172.21.15.49:0/2175985829 >> v2:172.21.15.150:3300/0 conn(0x7fccd815ab50 msgr2=0x7fccd80c9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:34.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.005+0000 7fccd659c640 1 --2- 172.21.15.49:0/2175985829 >> v2:172.21.15.150:3300/0 conn(0x7fccd815ab50 0x7fccd80c9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:34.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.005+0000 7fccd659c640 1 -- 172.21.15.49:0/2175985829 >> v1:172.21.15.150:6789/0 conn(0x7fccd80b05c0 legacy=0x7fccd80700a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:34.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.005+0000 7fccd659c640 1 -- 172.21.15.49:0/2175985829 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccd80cb5b0 con 0x7fccd80cb010 2023-11-06T02:11:34.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.005+0000 7fccd5d9b640 1 -- 172.21.15.49:0/2175985829 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 825711123 0 0) 0x7fccd80cab80 con 0x7fccd80b05c0 2023-11-06T02:11:34.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.009+0000 7fccd659c640 1 --2- 172.21.15.49:0/2175985829 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fccd80cb010 0x7fccd80b0080 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcccc009870 tx=0x7fcccc01c510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f3d5b4aed763390 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:34.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.009+0000 7fccd5d9b640 1 -- 172.21.15.49:0/2175985829 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcccc0aa670 con 0x7fccd80cb010 2023-11-06T02:11:34.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.009+0000 7fccd5d9b640 1 -- 172.21.15.49:0/2175985829 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcccc0aa7d0 con 0x7fccd80cb010 2023-11-06T02:11:34.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.009+0000 7fccd5d9b640 1 -- 172.21.15.49:0/2175985829 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcccc0aab60 con 0x7fccd80cb010 2023-11-06T02:11:34.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.009+0000 7fccdd224640 1 -- 172.21.15.49:0/2175985829 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fccd80cb010 msgr2=0x7fccd80b0080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:34.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.009+0000 7fccdd224640 1 --2- 172.21.15.49:0/2175985829 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fccd80cb010 0x7fccd80b0080 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcccc009870 tx=0x7fcccc01c510 comp rx=0 tx=0).stop 2023-11-06T02:11:34.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.009+0000 7fccdd224640 1 -- 172.21.15.49:0/2175985829 shutdown_connections 2023-11-06T02:11:34.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.009+0000 7fccdd224640 1 --2- 172.21.15.49:0/2175985829 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fccd80cb010 0x7fccd80b0080 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:34.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.009+0000 7fccdd224640 1 --2- 172.21.15.49:0/2175985829 >> v2:172.21.15.150:3300/0 conn(0x7fccd815ab50 0x7fccd80c9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:34.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.009+0000 7fccdd224640 1 -- 172.21.15.49:0/2175985829 >> 172.21.15.49:0/2175985829 conn(0x7fccd806c0a0 msgr2=0x7fccd806c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:34.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.009+0000 7fccdd224640 1 -- 172.21.15.49:0/2175985829 shutdown_connections 2023-11-06T02:11:34.009 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.009+0000 7fccdd224640 1 -- 172.21.15.49:0/2175985829 wait complete. 2023-11-06T02:11:34.010 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.009+0000 7fccdd224640 1 Processor -- start 2023-11-06T02:11:34.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.009+0000 7fccdd224640 1 -- start start 2023-11-06T02:11:34.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.013+0000 7fccdd224640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fccd80b05c0 0x7fccd80a9580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:34.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.013+0000 7fccdd224640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fccd80cb010 0x7fccd80c1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:34.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.013+0000 7fccdd224640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fccd815ab50 0x7fccd80c7af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:34.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.013+0000 7fccdd224640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd8158550 con 0x7fccd80b05c0 2023-11-06T02:11:34.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.013+0000 7fccdd224640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd80a9ac0 con 0x7fccd815ab50 2023-11-06T02:11:34.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.013+0000 7fccdd224640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccd80c1cd0 con 0x7fccd80cb010 2023-11-06T02:11:34.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.013+0000 7fccd6d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fccd80b05c0 0x7fccd80a9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:34.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.013+0000 7fccd6d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fccd80b05c0 0x7fccd80a9580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48862/0 (socket says 172.21.15.49:48862) 2023-11-06T02:11:34.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.013+0000 7fccd6d9d640 1 -- 172.21.15.49:0/3940411243 learned_addr learned my addr 172.21.15.49:0/3940411243 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:34.013 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.013+0000 7fccd659c640 1 --2- 172.21.15.49:0/3940411243 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fccd80cb010 0x7fccd80c1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:34.013 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.013+0000 7fccd759e640 1 --2- 172.21.15.49:0/3940411243 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fccd815ab50 0x7fccd80c7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:34.013 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.013+0000 7fccd6d9d640 1 -- 172.21.15.49:0/3940411243 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fccd80cb010 msgr2=0x7fccd80c1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:34.013 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.013+0000 7fccd6d9d640 1 --2- 172.21.15.49:0/3940411243 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fccd80cb010 0x7fccd80c1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:34.013 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.013+0000 7fccd6d9d640 1 -- 172.21.15.49:0/3940411243 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fccd815ab50 msgr2=0x7fccd80c7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:34.014 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.013+0000 7fccd6d9d640 1 --2- 172.21.15.49:0/3940411243 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fccd815ab50 0x7fccd80c7af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:34.014 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.013+0000 7fccd6d9d640 1 -- 172.21.15.49:0/3940411243 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcccc0094d0 con 0x7fccd80b05c0 2023-11-06T02:11:34.014 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.013+0000 7fccd6d9d640 1 --2- 172.21.15.49:0/3940411243 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fccd80b05c0 0x7fccd80a9580 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fccd000efc0 tx=0x7fccd000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:34.014 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.013+0000 7fccbf7fe640 1 -- 172.21.15.49:0/3940411243 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd0014480 con 0x7fccd80b05c0 2023-11-06T02:11:34.014 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.013+0000 7fccbf7fe640 1 -- 172.21.15.49:0/3940411243 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccd000f040 con 0x7fccd80b05c0 2023-11-06T02:11:34.015 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.013+0000 7fccdd224640 1 -- 172.21.15.49:0/3940411243 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccd8146a00 con 0x7fccd80b05c0 2023-11-06T02:11:34.015 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.013+0000 7fccbf7fe640 1 -- 172.21.15.49:0/3940411243 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd0014800 con 0x7fccd80b05c0 2023-11-06T02:11:34.015 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.013+0000 7fccdd224640 1 -- 172.21.15.49:0/3940411243 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccd8146fc0 con 0x7fccd80b05c0 2023-11-06T02:11:34.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.013+0000 7fccdd224640 1 -- 172.21.15.49:0/3940411243 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccd80b0de0 con 0x7fccd80b05c0 2023-11-06T02:11:34.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.013+0000 7fccbf7fe640 1 -- 172.21.15.49:0/3940411243 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fccd0006d50 con 0x7fccd80b05c0 2023-11-06T02:11:34.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.013+0000 7fccbf7fe640 1 --2- 172.21.15.49:0/3940411243 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fccc4065a60 0x7fccc4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:34.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.013+0000 7fccbf7fe640 1 -- 172.21.15.49:0/3940411243 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fccd0083d80 con 0x7fccd80b05c0 2023-11-06T02:11:34.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.013+0000 7fccd659c640 1 --2- 172.21.15.49:0/3940411243 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fccc4065a60 0x7fccc4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:34.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.017+0000 7fccd659c640 1 --2- 172.21.15.49:0/3940411243 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fccc4065a60 0x7fccc4067f20 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fcccc0ae000 tx=0x7fcccc0a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:34.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.021+0000 7fccbf7fe640 1 -- 172.21.15.49:0/3940411243 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fccd00515e0 con 0x7fccd80b05c0 2023-11-06T02:11:34.169 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.173+0000 7fccdd224640 1 -- 172.21.15.49:0/3940411243 --> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fccd80cbe50 con 0x7fccc4065a60 2023-11-06T02:11:34.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.197+0000 7fccbf7fe640 1 -- 172.21.15.49:0/3940411243 <== mgr.4100 v2:172.21.15.150:6832/38104 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+296052 (secure 0 0 0) 0x7fcccc004040 con 0x7fccc4065a60 2023-11-06T02:11:34.196 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:11:34.200 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2023-11-06T02:11:34.202 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.201+0000 7fccdd224640 1 -- 172.21.15.49:0/3940411243 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fccc4065a60 msgr2=0x7fccc4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:34.202 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.201+0000 7fccdd224640 1 --2- 172.21.15.49:0/3940411243 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fccc4065a60 0x7fccc4067f20 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fcccc0ae000 tx=0x7fcccc0a7000 comp rx=0 tx=0).stop 2023-11-06T02:11:34.202 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.201+0000 7fccdd224640 1 -- 172.21.15.49:0/3940411243 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fccd80b05c0 msgr2=0x7fccd80a9580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:34.202 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.201+0000 7fccdd224640 1 --2- 172.21.15.49:0/3940411243 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fccd80b05c0 0x7fccd80a9580 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fccd000efc0 tx=0x7fccd000c490 comp rx=0 tx=0).stop 2023-11-06T02:11:34.202 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.201+0000 7fccdd224640 1 -- 172.21.15.49:0/3940411243 shutdown_connections 2023-11-06T02:11:34.203 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.201+0000 7fccdd224640 1 --2- 172.21.15.49:0/3940411243 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fccc4065a60 0x7fccc4067f20 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:34.203 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.201+0000 7fccdd224640 1 --2- 172.21.15.49:0/3940411243 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fccd815ab50 0x7fccd80c7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:34.203 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.201+0000 7fccdd224640 1 --2- 172.21.15.49:0/3940411243 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fccd80cb010 0x7fccd80c1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:34.203 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.201+0000 7fccdd224640 1 --2- 172.21.15.49:0/3940411243 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fccd80b05c0 0x7fccd80a9580 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:34.204 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.201+0000 7fccdd224640 1 -- 172.21.15.49:0/3940411243 >> 172.21.15.49:0/3940411243 conn(0x7fccd806c0a0 msgr2=0x7fccd80c9bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:34.204 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.201+0000 7fccdd224640 1 -- 172.21.15.49:0/3940411243 shutdown_connections 2023-11-06T02:11:34.204 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.201+0000 7fccdd224640 1 -- 172.21.15.49:0/3940411243 wait complete. 2023-11-06T02:11:34.214 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":156,"stamp":"2023-11-06T02:11:33.239742+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":28,"acting":28,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":36,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":238932,"kb_used_data":6932,"kb_used_omap":0,"kb_used_meta":231936,"kb_avail":754735788,"statfs":{"total":773094113280,"available":772849446912,"internally_reserved":0,"allocated":7098368,"data_stored":3763720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":237502464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.535345"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.109328+0000","last_change":"2023-11-06T02:11:30.139772+0000","last_active":"2023-11-06T02:11:31.109328+0000","last_peered":"2023-11-06T02:11:31.109328+0000","last_clean":"2023-11-06T02:11:31.109328+0000","last_became_active":"2023-11-06T02:11:30.139447+0000","last_became_peered":"2023-11-06T02:11:30.139447+0000","last_unstale":"2023-11-06T02:11:31.109328+0000","last_undegraded":"2023-11-06T02:11:31.109328+0000","last_fullsized":"2023-11-06T02:11:31.109328+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:24:49.653824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.109357+0000","last_change":"2023-11-06T02:11:30.140958+0000","last_active":"2023-11-06T02:11:31.109357+0000","last_peered":"2023-11-06T02:11:31.109357+0000","last_clean":"2023-11-06T02:11:31.109357+0000","last_became_active":"2023-11-06T02:11:30.140693+0000","last_became_peered":"2023-11-06T02:11:30.140693+0000","last_unstale":"2023-11-06T02:11:31.109357+0000","last_undegraded":"2023-11-06T02:11:31.109357+0000","last_fullsized":"2023-11-06T02:11:31.109357+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:49:19.353757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.109510+0000","last_change":"2023-11-06T02:11:30.123394+0000","last_active":"2023-11-06T02:11:31.109510+0000","last_peered":"2023-11-06T02:11:31.109510+0000","last_clean":"2023-11-06T02:11:31.109510+0000","last_became_active":"2023-11-06T02:11:30.123113+0000","last_became_peered":"2023-11-06T02:11:30.123113+0000","last_unstale":"2023-11-06T02:11:31.109510+0000","last_undegraded":"2023-11-06T02:11:31.109510+0000","last_fullsized":"2023-11-06T02:11:31.109510+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:44:35.667968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.110281+0000","last_change":"2023-11-06T02:11:30.113941+0000","last_active":"2023-11-06T02:11:31.110281+0000","last_peered":"2023-11-06T02:11:31.110281+0000","last_clean":"2023-11-06T02:11:31.110281+0000","last_became_active":"2023-11-06T02:11:30.113672+0000","last_became_peered":"2023-11-06T02:11:30.113672+0000","last_unstale":"2023-11-06T02:11:31.110281+0000","last_undegraded":"2023-11-06T02:11:31.110281+0000","last_fullsized":"2023-11-06T02:11:31.110281+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:06:25.213807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.110013+0000","last_change":"2023-11-06T02:11:30.150778+0000","last_active":"2023-11-06T02:11:31.110013+0000","last_peered":"2023-11-06T02:11:31.110013+0000","last_clean":"2023-11-06T02:11:31.110013+0000","last_became_active":"2023-11-06T02:11:30.150466+0000","last_became_peered":"2023-11-06T02:11:30.150466+0000","last_unstale":"2023-11-06T02:11:31.110013+0000","last_undegraded":"2023-11-06T02:11:31.110013+0000","last_fullsized":"2023-11-06T02:11:31.110013+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:21:14.911044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":200,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-06T02:11:25.673533+0000","last_change":"2023-11-06T02:08:20.523427+0000","last_active":"2023-11-06T02:11:25.673533+0000","last_peered":"2023-11-06T02:11:25.673533+0000","last_clean":"2023-11-06T02:11:25.673533+0000","last_became_active":"2023-11-06T02:08:20.521089+0000","last_became_peered":"2023-11-06T02:08:20.521089+0000","last_unstale":"2023-11-06T02:11:25.673533+0000","last_undegraded":"2023-11-06T02:11:25.673533+0000","last_fullsized":"2023-11-06T02:11:25.673533+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:19.252348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:19.252348+0000","last_clean_scrub_stamp":"2023-11-06T02:08:19.252348+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:30:15.566397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":46,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-06T02:11:25.674151+0000","last_change":"2023-11-06T02:08:26.059502+0000","last_active":"2023-11-06T02:11:25.674151+0000","last_peered":"2023-11-06T02:11:25.674151+0000","last_clean":"2023-11-06T02:11:25.674151+0000","last_became_active":"2023-11-06T02:08:26.058612+0000","last_became_peered":"2023-11-06T02:08:26.058612+0000","last_unstale":"2023-11-06T02:11:25.674151+0000","last_undegraded":"2023-11-06T02:11:25.674151+0000","last_fullsized":"2023-11-06T02:11:25.674151+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:10.626021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":47,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-06T02:11:25.673429+0000","last_change":"2023-11-06T02:08:26.060406+0000","last_active":"2023-11-06T02:11:25.673429+0000","last_peered":"2023-11-06T02:11:25.673429+0000","last_clean":"2023-11-06T02:11:25.673429+0000","last_became_active":"2023-11-06T02:08:26.059262+0000","last_became_peered":"2023-11-06T02:08:26.059262+0000","last_unstale":"2023-11-06T02:11:25.673429+0000","last_undegraded":"2023-11-06T02:11:25.673429+0000","last_fullsized":"2023-11-06T02:11:25.673429+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:10.626021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":52,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-06T02:11:30.105820+0000","last_change":"2023-11-06T02:08:26.559477+0000","last_active":"2023-11-06T02:11:30.105820+0000","last_peered":"2023-11-06T02:11:30.105820+0000","last_clean":"2023-11-06T02:11:30.105820+0000","last_became_active":"2023-11-06T02:08:26.559202+0000","last_became_peered":"2023-11-06T02:08:26.559202+0000","last_unstale":"2023-11-06T02:11:30.105820+0000","last_undegraded":"2023-11-06T02:11:30.105820+0000","last_fullsized":"2023-11-06T02:11:30.105820+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:55:04.552404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":46,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-06T02:11:25.674065+0000","last_change":"2023-11-06T02:08:26.059546+0000","last_active":"2023-11-06T02:11:25.674065+0000","last_peered":"2023-11-06T02:11:25.674065+0000","last_clean":"2023-11-06T02:11:25.674065+0000","last_became_active":"2023-11-06T02:08:26.058806+0000","last_became_peered":"2023-11-06T02:08:26.058806+0000","last_unstale":"2023-11-06T02:11:25.674065+0000","last_undegraded":"2023-11-06T02:11:25.674065+0000","last_fullsized":"2023-11-06T02:11:25.674065+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:23.468510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":45,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-06T02:11:25.672779+0000","last_change":"2023-11-06T02:08:26.436463+0000","last_active":"2023-11-06T02:11:25.672779+0000","last_peered":"2023-11-06T02:11:25.672779+0000","last_clean":"2023-11-06T02:11:25.672779+0000","last_became_active":"2023-11-06T02:08:26.436075+0000","last_became_peered":"2023-11-06T02:08:26.436075+0000","last_unstale":"2023-11-06T02:11:25.672779+0000","last_undegraded":"2023-11-06T02:11:25.672779+0000","last_fullsized":"2023-11-06T02:11:25.672779+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:23.468510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":47,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-06T02:11:25.673433+0000","last_change":"2023-11-06T02:08:26.436607+0000","last_active":"2023-11-06T02:11:25.673433+0000","last_peered":"2023-11-06T02:11:25.673433+0000","last_clean":"2023-11-06T02:11:25.673433+0000","last_became_active":"2023-11-06T02:08:26.435924+0000","last_became_peered":"2023-11-06T02:08:26.435924+0000","last_unstale":"2023-11-06T02:11:25.673433+0000","last_undegraded":"2023-11-06T02:11:25.673433+0000","last_fullsized":"2023-11-06T02:11:25.673433+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:23.468510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":45,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-06T02:11:25.672751+0000","last_change":"2023-11-06T02:08:26.057076+0000","last_active":"2023-11-06T02:11:25.672751+0000","last_peered":"2023-11-06T02:11:25.672751+0000","last_clean":"2023-11-06T02:11:25.672751+0000","last_became_active":"2023-11-06T02:08:26.056419+0000","last_became_peered":"2023-11-06T02:08:26.056419+0000","last_unstale":"2023-11-06T02:11:25.672751+0000","last_undegraded":"2023-11-06T02:11:25.672751+0000","last_fullsized":"2023-11-06T02:11:25.672751+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:10.626021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":53,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-06T02:11:25.617700+0000","last_change":"2023-11-06T02:08:26.058596+0000","last_active":"2023-11-06T02:11:25.617700+0000","last_peered":"2023-11-06T02:11:25.617700+0000","last_clean":"2023-11-06T02:11:25.617700+0000","last_became_active":"2023-11-06T02:08:26.057015+0000","last_became_peered":"2023-11-06T02:08:26.057015+0000","last_unstale":"2023-11-06T02:11:25.617700+0000","last_undegraded":"2023-11-06T02:11:25.617700+0000","last_fullsized":"2023-11-06T02:11:25.617700+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:55:04.552404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":10,"acting":10,"num_store_stats":1},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":1},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574889,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28248,"kb_used_data":1296,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343592,"statfs":{"total":96636764160,"available":96607838208,"internally_reserved":0,"allocated":1327104,"data_stored":910047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.86199999999999999,"15min":0.85299999999999998},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.1200000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.91300000000000003,"15min":0.91100000000000003},"min":{"1min":0.68300000000000005,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.1020000000000001,"5min":1.506,"15min":1.506},"last":0.46300000000000002}]},{"osd":1,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.82699999999999996,"15min":0.84299999999999997},"min":{"1min":0.55300000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.0329999999999999,"5min":1.161,"15min":1.161},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.91200000000000003,"15min":0.89600000000000002},"min":{"1min":0.63600000000000001,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.1910000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.61399999999999999}]},{"osd":2,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.96399999999999997,"15min":0.95699999999999996},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.2989999999999999,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.98699999999999999,"15min":0.996},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.5,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.49299999999999999}]},{"osd":3,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.88300000000000001,"15min":0.877},"min":{"1min":0.69099999999999995,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.1479999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":1.012,"5min":1.0700000000000001,"15min":1.0740000000000001},"min":{"1min":0.78900000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.2230000000000001,"5min":1.526,"15min":1.526},"last":0.64300000000000002}]},{"osd":4,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.95899999999999996,"15min":0.95799999999999996},"min":{"1min":0.64000000000000001,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.38200000000000001},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.0229999999999999,"15min":1.0369999999999999},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.47299999999999998}]},{"osd":5,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":1.0669999999999999,"15min":1.075},"min":{"1min":0.69999999999999996,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":0.503},{"interface":"front","average":{"1min":1.0780000000000001,"5min":1.117,"15min":1.129},"min":{"1min":0.73199999999999998,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.3700000000000001,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":0.55500000000000005}]},{"osd":6,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.196,"15min":1.2},"min":{"1min":0.83999999999999997,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.5009999999999999,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":0.42599999999999999},{"interface":"front","average":{"1min":1.048,"5min":1.131,"15min":1.149},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.331,"5min":1.74,"15min":1.74},"last":0.59499999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574888,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27668,"kb_used_data":716,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344172,"statfs":{"total":96636764160,"available":96608432128,"internally_reserved":0,"allocated":733184,"data_stored":319679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.52400000000000002,"5min":0.495,"15min":0.495},"max":{"1min":1.216,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.84099999999999997,"15min":0.81699999999999995},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.2370000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.496}]},{"osd":1,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.86599999999999999,"15min":0.872},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.86799999999999999,"15min":0.872},"min":{"1min":0.63,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.2330000000000001,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":1.0229999999999999}]},{"osd":2,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.877,"15min":0.89100000000000001},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.90400000000000003,"15min":0.92200000000000004},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.103,"5min":1.28,"15min":1.28},"last":1.1339999999999999}]},{"osd":3,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":0.95999999999999996,"15min":0.96799999999999997},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.3300000000000001,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.94699999999999995},{"interface":"front","average":{"1min":0.99399999999999999,"5min":0.99199999999999999,"15min":0.998},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.22,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.98799999999999999}]},{"osd":4,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.95099999999999996,"15min":0.97899999999999998},"min":{"1min":0.59499999999999997,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.1639999999999999,"5min":1.474,"15min":1.474},"last":1.163},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.99399999999999999,"15min":1.024},"min":{"1min":0.624,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.268,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":1.0669999999999999}]},{"osd":5,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.98399999999999999,"15min":1.0129999999999999},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.161,"5min":1.365,"15min":1.365},"last":1.1870000000000001},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.0760000000000001,"15min":1.1020000000000001},"min":{"1min":0.68400000000000005,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.3939999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":1.319}]},{"osd":7,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":1.083,"15min":1.1060000000000001},"min":{"1min":0.76800000000000002,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.3580000000000001,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":1.258},{"interface":"front","average":{"1min":0.99399999999999999,"5min":1.095,"15min":1.141},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.425,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":1.3340000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574889,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27668,"kb_used_data":716,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344172,"statfs":{"total":96636764160,"available":96608432128,"internally_reserved":0,"allocated":733184,"data_stored":319679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.85599999999999998,"15min":0.83799999999999997},"min":{"1min":0.63300000000000001,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.2390000000000001,"5min":1.381,"15min":1.381},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.85699999999999998,"15min":0.84699999999999998},"min":{"1min":0.63100000000000001,"5min":0.503,"15min":0.503},"max":{"1min":1.3200000000000001,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":1.1000000000000001}]},{"osd":1,"last update":"Mon Nov 6 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.86599999999999999,"15min":0.85499999999999998},"min":{"1min":0.626,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.1699999999999999,"5min":1.331,"15min":1.331},"last":1.1699999999999999},{"interface":"front","average":{"1min":0.87,"5min":0.83599999999999997,"15min":0.82299999999999995},"min":{"1min":0.57199999999999995,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.177,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.748}]},{"osd":2,"last update":"Mon Nov 6 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.93200000000000005,"15min":0.94099999999999995},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.2210000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.83199999999999996,"15min":0.80400000000000005},"min":{"1min":0.55800000000000005,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.323,"5min":1.377,"15min":1.377},"last":0.88400000000000001}]},{"osd":3,"last update":"Mon Nov 6 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":0.93200000000000005,"15min":0.92400000000000004},"min":{"1min":0.65900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.3580000000000001,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.95899999999999996},{"interface":"front","average":{"1min":1.0009999999999999,"5min":1.0409999999999999,"15min":1.0669999999999999},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.228,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":1.0009999999999999}]},{"osd":4,"last update":"Mon Nov 6 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.93100000000000005,"15min":0.92100000000000004},"min":{"1min":0.61299999999999999,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.2410000000000001,"5min":1.276,"15min":1.276},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.026},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.496,"5min":1.554,"15min":1.554},"last":1.2669999999999999}]},{"osd":6,"last update":"Mon Nov 6 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":1.0469999999999999,"15min":1.0609999999999999},"min":{"1min":0.67500000000000004,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":0.91800000000000004},{"interface":"front","average":{"1min":1.075,"5min":1.04,"15min":1.04},"min":{"1min":0.72999999999999998,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":1.0409999999999999}]},{"osd":7,"last update":"Mon Nov 6 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.083,"15min":1.0780000000000001},"min":{"1min":0.71599999999999997,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.5880000000000001,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":1.218},{"interface":"front","average":{"1min":1.161,"5min":1.1499999999999999,"15min":1.153},"min":{"1min":0.92400000000000004,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.544,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":1.137}]}]},{"osd":4,"up_from":13,"seq":55834574889,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27668,"kb_used_data":716,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344172,"statfs":{"total":96636764160,"available":96608432128,"internally_reserved":0,"allocated":733184,"data_stored":319679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:11:22 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.85599999999999998,"15min":0.83499999999999996},"min":{"1min":0.67100000000000004,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.343,"5min":1.343,"15min":1.343},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.79300000000000004,"15min":0.77700000000000002},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.0169999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.751}]},{"osd":1,"last update":"Mon Nov 6 02:11:22 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.86899999999999999,"15min":0.85199999999999998},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.083,"5min":1.462,"15min":1.462},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.92400000000000004,"15min":0.90200000000000002},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.3580000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":1.0189999999999999}]},{"osd":2,"last update":"Mon Nov 6 02:11:22 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.97599999999999998,"15min":1.0049999999999999},"min":{"1min":0.69399999999999995,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.198,"5min":1.615,"15min":1.615},"last":0.87},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.94099999999999995,"15min":0.93100000000000005},"min":{"1min":0.68000000000000005,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.3049999999999999,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":1.117}]},{"osd":3,"last update":"Mon Nov 6 02:11:22 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.90200000000000002,"15min":0.91300000000000003},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.103,"5min":1.52,"15min":1.52},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.97799999999999998,"5min":1.04,"15min":1.0600000000000001},"min":{"1min":0.63900000000000001,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.2470000000000001,"5min":1.595,"15min":1.595},"last":0.69999999999999996}]},{"osd":5,"last update":"Mon Nov 6 02:11:22 2023","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":1.0329999999999999,"15min":1.0449999999999999},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.25,"5min":1.377,"15min":1.377},"last":1.0720000000000001},{"interface":"front","average":{"1min":0.97999999999999998,"5min":1.0189999999999999,"15min":1.0429999999999999},"min":{"1min":0.67500000000000004,"5min":0.629,"15min":0.629},"max":{"1min":1.278,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":0.95399999999999996}]},{"osd":6,"last update":"Mon Nov 6 02:11:22 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":1.052,"15min":1.0800000000000001},"min":{"1min":0.68000000000000005,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.2470000000000001,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":0.79900000000000004},{"interface":"front","average":{"1min":1.071,"5min":1.157,"15min":1.1839999999999999},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.4079999999999999,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":1.167}]},{"osd":7,"last update":"Mon Nov 6 02:11:22 2023","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":1.0720000000000001,"15min":1.0860000000000001},"min":{"1min":0.748,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.2849999999999999,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.83499999999999996},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.1859999999999999,"15min":1.222},"min":{"1min":0.80000000000000004,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.3280000000000001,"5min":1.7410000000000001,"15min":1.7410000000000001},"last":0.98699999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574888,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31764,"kb_used_data":716,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340076,"statfs":{"total":96636764160,"available":96604237824,"internally_reserved":0,"allocated":733184,"data_stored":319679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.63900000000000001,"15min":0.64500000000000002},"min":{"1min":0.36499999999999999,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.95999999999999996,"5min":1.079,"15min":1.079},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.626,"5min":0.625,"15min":0.625},"min":{"1min":0.34599999999999997,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.93899999999999995,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.64600000000000002}]},{"osd":1,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.61899999999999999,"15min":0.627},"min":{"1min":0.27800000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.86799999999999999,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.746},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.72599999999999998,"15min":0.73099999999999998},"min":{"1min":0.307,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.0980000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.83599999999999997}]},{"osd":2,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.71399999999999997,"15min":0.72799999999999998},"min":{"1min":0.374,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.497},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.81200000000000006,"15min":0.81999999999999995},"min":{"1min":0.39000000000000001,"5min":0.308,"15min":0.308},"max":{"1min":1.1899999999999999,"5min":1.405,"15min":1.405},"last":0.78100000000000003}]},{"osd":4,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.92700000000000005,"15min":0.92800000000000005},"min":{"1min":0.65900000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.085,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":1.022},{"interface":"front","average":{"1min":0.98099999999999998,"5min":0.97599999999999998,"15min":0.99099999999999999},"min":{"1min":0.67800000000000005,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.236,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":1}]},{"osd":5,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":1.004,"15min":1.02},"min":{"1min":0.70699999999999996,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":1.137},{"interface":"front","average":{"1min":1.0880000000000001,"5min":1.0580000000000001,"15min":1.0740000000000001},"min":{"1min":0.74099999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.4330000000000001,"5min":2.21,"15min":2.21},"last":1.044}]},{"osd":6,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":1.0229999999999999,"15min":1.0249999999999999},"min":{"1min":0.76900000000000002,"5min":0.439,"15min":0.439},"max":{"1min":1.3799999999999999,"5min":1.395,"15min":1.395},"last":1.204},{"interface":"front","average":{"1min":1.0880000000000001,"5min":1.0600000000000001,"15min":1.0680000000000001},"min":{"1min":0.71699999999999997,"5min":0.375,"15min":0.375},"max":{"1min":1.4319999999999999,"5min":1.554,"15min":1.554},"last":1.069}]},{"osd":7,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.115,"15min":1.127},"min":{"1min":0.76300000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.5549999999999999,"5min":2.2829999999999999,"15min":2.2829999999999999},"last":1.1719999999999999},{"interface":"front","average":{"1min":1.171,"5min":1.1080000000000001,"15min":1.107},"min":{"1min":0.92200000000000004,"5min":0.497,"15min":0.497},"max":{"1min":1.504,"5min":1.504,"15min":1.504},"last":0.96399999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574889,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31812,"kb_used_data":764,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340028,"statfs":{"total":96636764160,"available":96604188672,"internally_reserved":0,"allocated":782336,"data_stored":346250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:11:20 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.53900000000000003,"15min":0.51200000000000001},"min":{"1min":0.48199999999999998,"5min":0.252,"15min":0.252},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.59099999999999997,"15min":0.55600000000000005},"min":{"1min":0.497,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":0.502}]},{"osd":1,"last update":"Mon Nov 6 02:11:20 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.60599999999999998,"15min":0.57499999999999996},"min":{"1min":0.44700000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.48,"5min":1.48,"15min":1.48},"last":0.623},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.64700000000000002,"15min":0.59399999999999997},"min":{"1min":0.45200000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.7230000000000001,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":0.75700000000000001}]},{"osd":3,"last update":"Mon Nov 6 02:11:20 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.65400000000000003,"15min":0.61599999999999999},"min":{"1min":0.58699999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.93200000000000005,"5min":1.155,"15min":1.155},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.69799999999999995,"15min":0.66000000000000003},"min":{"1min":0.52500000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.0960000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.58399999999999996}]},{"osd":4,"last update":"Mon Nov 6 02:11:20 2023","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":0.85799999999999998,"15min":0.81799999999999995},"min":{"1min":0.77500000000000002,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.89800000000000002},{"interface":"front","average":{"1min":1.0369999999999999,"5min":0.93600000000000005,"15min":0.92200000000000004},"min":{"1min":0.755,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.395,"5min":1.395,"15min":1.395},"last":0.95099999999999996}]},{"osd":5,"last update":"Mon Nov 6 02:11:20 2023","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":0.96199999999999997,"15min":0.94899999999999995},"min":{"1min":0.74099999999999999,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.7729999999999999,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":1.0189999999999999,"5min":0.95699999999999996,"15min":0.94099999999999995},"min":{"1min":0.67200000000000004,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.615,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.97099999999999997}]},{"osd":6,"last update":"Mon Nov 6 02:11:20 2023","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":0.96899999999999997,"15min":0.94799999999999995},"min":{"1min":0.79200000000000004,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":0.91700000000000004},{"interface":"front","average":{"1min":1.087,"5min":1.0129999999999999,"15min":1.002},"min":{"1min":0.81499999999999995,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.675,"5min":1.677,"15min":1.677},"last":1.0369999999999999}]},{"osd":7,"last update":"Mon Nov 6 02:11:20 2023","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":1.0089999999999999,"15min":0.98599999999999999},"min":{"1min":0.72499999999999998,"5min":0.622,"15min":0.622},"max":{"1min":1.659,"5min":1.659,"15min":1.659},"last":0.98699999999999999},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.0620000000000001,"15min":1.048},"min":{"1min":0.89300000000000002,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.4650000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":1.1160000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574889,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31764,"kb_used_data":716,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340076,"statfs":{"total":96636764160,"available":96604237824,"internally_reserved":0,"allocated":733184,"data_stored":319679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:11:22 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.63700000000000001,"15min":0.629},"min":{"1min":0.28699999999999998,"5min":0.224,"15min":0.224},"max":{"1min":1.603,"5min":1.603,"15min":1.603},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.67400000000000004,"15min":0.65900000000000003},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":1.0760000000000001}]},{"osd":2,"last update":"Mon Nov 6 02:11:22 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.63700000000000001,"15min":0.627},"min":{"1min":0.33600000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.649,"5min":1.649,"15min":1.649},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.755,"15min":0.748},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.1919999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":1.1539999999999999}]},{"osd":3,"last update":"Mon Nov 6 02:11:22 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.79500000000000004,"15min":0.80000000000000004},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.2350000000000001,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":1.0449999999999999},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.80800000000000005,"15min":0.81000000000000005},"min":{"1min":0.41299999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.5,"5min":1.8149999999999999,"15min":1.8149999999999999},"last":0.86299999999999999}]},{"osd":4,"last update":"Mon Nov 6 02:11:22 2023","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.97699999999999998,"15min":0.98699999999999999},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.5780000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":1.26},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.98099999999999998,"15min":0.98899999999999999},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.4099999999999999,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.81999999999999995}]},{"osd":5,"last update":"Mon Nov 6 02:11:22 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":0.97499999999999998,"15min":0.96899999999999997},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.718,"5min":1.718,"15min":1.718},"last":1.0169999999999999},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.069,"15min":1.075},"min":{"1min":0.66500000000000004,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.403,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":1.2110000000000001}]},{"osd":6,"last update":"Mon Nov 6 02:11:22 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.0920000000000001,"15min":1.099},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.6859999999999999,"5min":1.742,"15min":1.742},"last":1.1850000000000001},{"interface":"front","average":{"1min":1.0700000000000001,"5min":1.0669999999999999,"15min":1.073},"min":{"1min":0.64400000000000002,"5min":0.629,"15min":0.629},"max":{"1min":1.6339999999999999,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":1.129}]},{"osd":7,"last update":"Mon Nov 6 02:11:22 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.115,"15min":1.119},"min":{"1min":0.71499999999999997,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.534,"5min":1.534,"15min":1.534},"last":0.97199999999999998},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.1499999999999999,"15min":1.157},"min":{"1min":0.68700000000000006,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.7629999999999999,"5min":1.893,"15min":1.893},"last":1.3400000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574889,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32340,"kb_used_data":1292,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339500,"statfs":{"total":96636764160,"available":96603648000,"internally_reserved":0,"allocated":1323008,"data_stored":909028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 02:11:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.33700000000000002,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64000000000000001,"15min":0.63700000000000001},"min":{"1min":0.38700000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.93400000000000005,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.49299999999999999}]},{"osd":2,"last update":"Mon Nov 6 02:11:25 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.67100000000000004,"15min":0.66700000000000004},"min":{"1min":0.505,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.67800000000000005,"15min":0.67600000000000005},"min":{"1min":0.35199999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.95599999999999996,"5min":1.141,"15min":1.141},"last":0.64500000000000002}]},{"osd":3,"last update":"Mon Nov 6 02:11:25 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.64000000000000001,"15min":0.63200000000000001},"min":{"1min":0.376,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.0600000000000001,"5min":1.429,"15min":1.429},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.753,"15min":0.73799999999999999},"min":{"1min":0.44,"5min":0.317,"15min":0.317},"max":{"1min":1.1299999999999999,"5min":1.25,"15min":1.25},"last":0.96099999999999997}]},{"osd":4,"last update":"Mon Nov 6 02:11:25 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.85599999999999998,"15min":0.83599999999999997},"min":{"1min":0.59499999999999997,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.1579999999999999,"5min":1.341,"15min":1.341},"last":1.0189999999999999},{"interface":"front","average":{"1min":0.995,"5min":0.94299999999999995,"15min":0.93899999999999995},"min":{"1min":0.65100000000000002,"5min":0.5,"15min":0.5},"max":{"1min":1.3600000000000001,"5min":1.387,"15min":1.387},"last":1.0449999999999999}]},{"osd":5,"last update":"Mon Nov 6 02:11:25 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.94199999999999995,"15min":0.94599999999999995},"min":{"1min":0.63300000000000001,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.119,"5min":1.339,"15min":1.339},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.89700000000000002,"15min":0.88},"min":{"1min":0.72899999999999998,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.325,"5min":1.448,"15min":1.448},"last":1.0800000000000001}]},{"osd":6,"last update":"Mon Nov 6 02:11:25 2023","interfaces":[{"interface":"back","average":{"1min":1.0960000000000001,"5min":1.0409999999999999,"15min":1.038},"min":{"1min":0.66700000000000004,"5min":0.625,"15min":0.625},"max":{"1min":1.573,"5min":1.589,"15min":1.589},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.0309999999999999,"5min":1.0169999999999999,"15min":1.0049999999999999},"min":{"1min":0.67600000000000005,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.3899999999999999,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.95099999999999996}]},{"osd":7,"last update":"Mon Nov 6 02:11:25 2023","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":1.008,"15min":1.0149999999999999},"min":{"1min":0.71299999999999997,"5min":0.504,"15min":0.504},"max":{"1min":1.341,"5min":1.421,"15min":1.421},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.98599999999999999,"5min":0.98899999999999999,"15min":1.002},"min":{"1min":0.69899999999999995,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.3029999999999999,"5min":1.556,"15min":1.556},"last":0.81499999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T02:11:34.221 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T02:11:34.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.385+0000 7f27b1c11640 1 Processor -- start 2023-11-06T02:11:34.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.385+0000 7f27b1c11640 1 -- start start 2023-11-06T02:11:34.383 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.385+0000 7f27b1c11640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f27ac15ab50 0x7f27ac0c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:34.383 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.385+0000 7f27b1c11640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f27ac0cb010 0x7f27ac0b0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:34.383 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.385+0000 7f27b1c11640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f27ac0b05c0 0x7f27ac0700a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:34.384 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.385+0000 7f27b1c11640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27ac0b0a50 con 0x7f27ac15ab50 2023-11-06T02:11:34.384 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.385+0000 7f27b1c11640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27ac0cab80 con 0x7f27ac0b05c0 2023-11-06T02:11:34.384 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.385+0000 7f27b1c11640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27ac15af30 con 0x7f27ac0cb010 2023-11-06T02:11:34.384 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.385+0000 7f27ab7fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f27ac15ab50 0x7f27ac0c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:34.384 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.385+0000 7f27ab7fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f27ac15ab50 0x7f27ac0c9bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48868/0 (socket says 172.21.15.49:48868) 2023-11-06T02:11:34.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.385+0000 7f27ab7fe640 1 -- 172.21.15.49:0/2690585450 learned_addr learned my addr 172.21.15.49:0/2690585450 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:34.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.385+0000 7f27abfff640 1 --2- 172.21.15.49:0/2690585450 >> v2:172.21.15.150:3300/0 conn(0x7f27ac0b05c0 0x7f27ac0700a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:34.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.385+0000 7f27a2ffd640 1 --2- 172.21.15.49:0/2690585450 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f27ac0cb010 0x7f27ac0b0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:34.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.385+0000 7f27ab7fe640 1 -- 172.21.15.49:0/2690585450 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f27ac0cb010 msgr2=0x7f27ac0b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:34.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.385+0000 7f27ab7fe640 1 --2- 172.21.15.49:0/2690585450 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f27ac0cb010 0x7f27ac0b0080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:34.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.385+0000 7f27ab7fe640 1 -- 172.21.15.49:0/2690585450 >> v2:172.21.15.150:3300/0 conn(0x7f27ac0b05c0 msgr2=0x7f27ac0700a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:34.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.385+0000 7f27ab7fe640 1 --2- 172.21.15.49:0/2690585450 >> v2:172.21.15.150:3300/0 conn(0x7f27ac0b05c0 0x7f27ac0700a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:34.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.385+0000 7f27ab7fe640 1 -- 172.21.15.49:0/2690585450 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27ac0cb5b0 con 0x7f27ac15ab50 2023-11-06T02:11:34.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.385+0000 7f27a2ffd640 1 --2- 172.21.15.49:0/2690585450 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f27ac0cb010 0x7f27ac0b0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:11:34.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.385+0000 7f27ab7fe640 1 --2- 172.21.15.49:0/2690585450 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f27ac15ab50 0x7f27ac0c9bb0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f27ac0cae30 tx=0x7f279c01c210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=121864723673b32a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:34.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.389+0000 7f27aaffd640 1 -- 172.21.15.49:0/2690585450 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f279c0ab600 con 0x7f27ac15ab50 2023-11-06T02:11:34.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.389+0000 7f27aaffd640 1 -- 172.21.15.49:0/2690585450 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f279c0a7050 con 0x7f27ac15ab50 2023-11-06T02:11:34.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.389+0000 7f27aaffd640 1 -- 172.21.15.49:0/2690585450 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f279c0ab980 con 0x7f27ac15ab50 2023-11-06T02:11:34.391 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.389+0000 7f27b1c11640 1 -- 172.21.15.49:0/2690585450 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f27ac15ab50 msgr2=0x7f27ac0c9bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:34.391 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.389+0000 7f27b1c11640 1 --2- 172.21.15.49:0/2690585450 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f27ac15ab50 0x7f27ac0c9bb0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f27ac0cae30 tx=0x7f279c01c210 comp rx=0 tx=0).stop 2023-11-06T02:11:34.391 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.389+0000 7f27b1c11640 1 -- 172.21.15.49:0/2690585450 shutdown_connections 2023-11-06T02:11:34.391 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.389+0000 7f27b1c11640 1 --2- 172.21.15.49:0/2690585450 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f27ac0cb010 0x7f27ac0b0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:34.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.389+0000 7f27b1c11640 1 --2- 172.21.15.49:0/2690585450 >> v2:172.21.15.150:3300/0 conn(0x7f27ac0b05c0 0x7f27ac0700a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:34.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.389+0000 7f27b1c11640 1 --2- 172.21.15.49:0/2690585450 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f27ac15ab50 0x7f27ac0c9bb0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:34.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.389+0000 7f27b1c11640 1 -- 172.21.15.49:0/2690585450 >> 172.21.15.49:0/2690585450 conn(0x7f27ac06c0a0 msgr2=0x7f27ac06c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:34.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.389+0000 7f27b1c11640 1 -- 172.21.15.49:0/2690585450 shutdown_connections 2023-11-06T02:11:34.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.389+0000 7f27b1c11640 1 -- 172.21.15.49:0/2690585450 wait complete. 2023-11-06T02:11:34.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.389+0000 7f27b1c11640 1 Processor -- start 2023-11-06T02:11:34.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.389+0000 7f27b1c11640 1 -- start start 2023-11-06T02:11:34.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.389+0000 7f27b1c11640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f27ac0b05c0 0x7f27ac158430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:34.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.389+0000 7f27b1c11640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f27ac0cb010 0x7f27ac152690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:34.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.389+0000 7f27b1c11640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f27ac15ab50 0x7f27ac152bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:34.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.389+0000 7f27b1c11640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27ac0cc300 con 0x7f27ac0cb010 2023-11-06T02:11:34.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.389+0000 7f27b1c11640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27ac0cc5e0 con 0x7f27ac0b05c0 2023-11-06T02:11:34.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.389+0000 7f27b1c11640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27ac0cc8c0 con 0x7f27ac15ab50 2023-11-06T02:11:34.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.389+0000 7f27a2ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f27ac0cb010 0x7f27ac152690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:34.395 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.389+0000 7f27a2ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f27ac0cb010 0x7f27ac152690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48884/0 (socket says 172.21.15.49:48884) 2023-11-06T02:11:34.395 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.389+0000 7f27a2ffd640 1 -- 172.21.15.49:0/1756850319 learned_addr learned my addr 172.21.15.49:0/1756850319 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:34.395 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.389+0000 7f27ab7fe640 1 --2- 172.21.15.49:0/1756850319 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f27ac0b05c0 0x7f27ac158430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:34.395 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.389+0000 7f27a2ffd640 1 -- 172.21.15.49:0/1756850319 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f27ac15ab50 msgr2=0x7f27ac152bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:34.395 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.389+0000 7f27a2ffd640 1 --2- 172.21.15.49:0/1756850319 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f27ac15ab50 0x7f27ac152bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:34.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.389+0000 7f27a2ffd640 1 -- 172.21.15.49:0/1756850319 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f27ac0b05c0 msgr2=0x7f27ac158430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:34.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.389+0000 7f27a2ffd640 1 --2- 172.21.15.49:0/1756850319 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f27ac0b05c0 0x7f27ac158430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:34.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.389+0000 7f27a2ffd640 1 -- 172.21.15.49:0/1756850319 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f279c009480 con 0x7f27ac0cb010 2023-11-06T02:11:34.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.389+0000 7f27ab7fe640 1 --2- 172.21.15.49:0/1756850319 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f27ac0b05c0 0x7f27ac158430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:11:34.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.389+0000 7f27a2ffd640 1 --2- 172.21.15.49:0/1756850319 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f27ac0cb010 0x7f27ac152690 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f279400c7c0 tx=0x7f279400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:34.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.389+0000 7f27a8ff9640 1 -- 172.21.15.49:0/1756850319 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2794007a30 con 0x7f27ac0cb010 2023-11-06T02:11:34.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.389+0000 7f27a8ff9640 1 -- 172.21.15.49:0/1756850319 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2794007b90 con 0x7f27ac0cb010 2023-11-06T02:11:34.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.389+0000 7f27b1c11640 1 -- 172.21.15.49:0/1756850319 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27ac0ccba0 con 0x7f27ac0cb010 2023-11-06T02:11:34.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.389+0000 7f27a8ff9640 1 -- 172.21.15.49:0/1756850319 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f279400ce70 con 0x7f27ac0cb010 2023-11-06T02:11:34.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.389+0000 7f27b1c11640 1 -- 172.21.15.49:0/1756850319 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27ac0c6f20 con 0x7f27ac0cb010 2023-11-06T02:11:34.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.393+0000 7f27b1c11640 1 -- 172.21.15.49:0/1756850319 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27ac0b0de0 con 0x7f27ac0cb010 2023-11-06T02:11:34.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.393+0000 7f27a8ff9640 1 -- 172.21.15.49:0/1756850319 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f27940113f0 con 0x7f27ac0cb010 2023-11-06T02:11:34.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.393+0000 7f27a8ff9640 1 --2- 172.21.15.49:0/1756850319 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2790065a60 0x7f2790067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:34.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.393+0000 7f27a8ff9640 1 -- 172.21.15.49:0/1756850319 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2794083780 con 0x7f27ac0cb010 2023-11-06T02:11:34.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.393+0000 7f27ab7fe640 1 --2- 172.21.15.49:0/1756850319 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2790065a60 0x7f2790067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:34.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.393+0000 7f27ab7fe640 1 --2- 172.21.15.49:0/1756850319 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2790065a60 0x7f2790067f20 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f279c0ad000 tx=0x7f279c0a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:34.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.397+0000 7f27a8ff9640 1 -- 172.21.15.49:0/1756850319 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f2794051070 con 0x7f27ac0cb010 2023-11-06T02:11:34.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.549+0000 7f27b1c11640 1 -- 172.21.15.49:0/1756850319 --> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f27ac0cbe50 con 0x7f2790065a60 2023-11-06T02:11:34.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.573+0000 7f27a8ff9640 1 -- 172.21.15.49:0/1756850319 <== mgr.4100 v2:172.21.15.150:6832/38104 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+296052 (secure 0 0 0) 0x7f279c004220 con 0x7f2790065a60 2023-11-06T02:11:34.572 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:11:34.574 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2023-11-06T02:11:34.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.577+0000 7f27b1c11640 1 -- 172.21.15.49:0/1756850319 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2790065a60 msgr2=0x7f2790067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:34.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.577+0000 7f27b1c11640 1 --2- 172.21.15.49:0/1756850319 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2790065a60 0x7f2790067f20 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f279c0ad000 tx=0x7f279c0a6000 comp rx=0 tx=0).stop 2023-11-06T02:11:34.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.577+0000 7f27b1c11640 1 -- 172.21.15.49:0/1756850319 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f27ac0cb010 msgr2=0x7f27ac152690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:34.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.577+0000 7f27b1c11640 1 --2- 172.21.15.49:0/1756850319 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f27ac0cb010 0x7f27ac152690 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f279400c7c0 tx=0x7f279400cc80 comp rx=0 tx=0).stop 2023-11-06T02:11:34.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.577+0000 7f27b1c11640 1 -- 172.21.15.49:0/1756850319 shutdown_connections 2023-11-06T02:11:34.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.577+0000 7f27b1c11640 1 --2- 172.21.15.49:0/1756850319 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2790065a60 0x7f2790067f20 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:34.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.577+0000 7f27b1c11640 1 --2- 172.21.15.49:0/1756850319 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f27ac0cb010 0x7f27ac152690 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:34.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.577+0000 7f27b1c11640 1 --2- 172.21.15.49:0/1756850319 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f27ac15ab50 0x7f27ac152bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:34.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.577+0000 7f27b1c11640 1 --2- 172.21.15.49:0/1756850319 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f27ac0b05c0 0x7f27ac158430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:34.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.577+0000 7f27b1c11640 1 -- 172.21.15.49:0/1756850319 >> 172.21.15.49:0/1756850319 conn(0x7f27ac06c0a0 msgr2=0x7f27ac0c2c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:34.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.577+0000 7f27b1c11640 1 -- 172.21.15.49:0/1756850319 shutdown_connections 2023-11-06T02:11:34.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.577+0000 7f27b1c11640 1 -- 172.21.15.49:0/1756850319 wait complete. 2023-11-06T02:11:34.591 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":156,"stamp":"2023-11-06T02:11:33.239742+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":28,"acting":28,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":36,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":238932,"kb_used_data":6932,"kb_used_omap":0,"kb_used_meta":231936,"kb_avail":754735788,"statfs":{"total":773094113280,"available":772849446912,"internally_reserved":0,"allocated":7098368,"data_stored":3763720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":237502464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.535345"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.109328+0000","last_change":"2023-11-06T02:11:30.139772+0000","last_active":"2023-11-06T02:11:31.109328+0000","last_peered":"2023-11-06T02:11:31.109328+0000","last_clean":"2023-11-06T02:11:31.109328+0000","last_became_active":"2023-11-06T02:11:30.139447+0000","last_became_peered":"2023-11-06T02:11:30.139447+0000","last_unstale":"2023-11-06T02:11:31.109328+0000","last_undegraded":"2023-11-06T02:11:31.109328+0000","last_fullsized":"2023-11-06T02:11:31.109328+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:24:49.653824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.109357+0000","last_change":"2023-11-06T02:11:30.140958+0000","last_active":"2023-11-06T02:11:31.109357+0000","last_peered":"2023-11-06T02:11:31.109357+0000","last_clean":"2023-11-06T02:11:31.109357+0000","last_became_active":"2023-11-06T02:11:30.140693+0000","last_became_peered":"2023-11-06T02:11:30.140693+0000","last_unstale":"2023-11-06T02:11:31.109357+0000","last_undegraded":"2023-11-06T02:11:31.109357+0000","last_fullsized":"2023-11-06T02:11:31.109357+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:49:19.353757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.109510+0000","last_change":"2023-11-06T02:11:30.123394+0000","last_active":"2023-11-06T02:11:31.109510+0000","last_peered":"2023-11-06T02:11:31.109510+0000","last_clean":"2023-11-06T02:11:31.109510+0000","last_became_active":"2023-11-06T02:11:30.123113+0000","last_became_peered":"2023-11-06T02:11:30.123113+0000","last_unstale":"2023-11-06T02:11:31.109510+0000","last_undegraded":"2023-11-06T02:11:31.109510+0000","last_fullsized":"2023-11-06T02:11:31.109510+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:44:35.667968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.110281+0000","last_change":"2023-11-06T02:11:30.113941+0000","last_active":"2023-11-06T02:11:31.110281+0000","last_peered":"2023-11-06T02:11:31.110281+0000","last_clean":"2023-11-06T02:11:31.110281+0000","last_became_active":"2023-11-06T02:11:30.113672+0000","last_became_peered":"2023-11-06T02:11:30.113672+0000","last_unstale":"2023-11-06T02:11:31.110281+0000","last_undegraded":"2023-11-06T02:11:31.110281+0000","last_fullsized":"2023-11-06T02:11:31.110281+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:06:25.213807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.110013+0000","last_change":"2023-11-06T02:11:30.150778+0000","last_active":"2023-11-06T02:11:31.110013+0000","last_peered":"2023-11-06T02:11:31.110013+0000","last_clean":"2023-11-06T02:11:31.110013+0000","last_became_active":"2023-11-06T02:11:30.150466+0000","last_became_peered":"2023-11-06T02:11:30.150466+0000","last_unstale":"2023-11-06T02:11:31.110013+0000","last_undegraded":"2023-11-06T02:11:31.110013+0000","last_fullsized":"2023-11-06T02:11:31.110013+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:21:14.911044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":200,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-06T02:11:25.673533+0000","last_change":"2023-11-06T02:08:20.523427+0000","last_active":"2023-11-06T02:11:25.673533+0000","last_peered":"2023-11-06T02:11:25.673533+0000","last_clean":"2023-11-06T02:11:25.673533+0000","last_became_active":"2023-11-06T02:08:20.521089+0000","last_became_peered":"2023-11-06T02:08:20.521089+0000","last_unstale":"2023-11-06T02:11:25.673533+0000","last_undegraded":"2023-11-06T02:11:25.673533+0000","last_fullsized":"2023-11-06T02:11:25.673533+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:19.252348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:19.252348+0000","last_clean_scrub_stamp":"2023-11-06T02:08:19.252348+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:30:15.566397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":46,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-06T02:11:25.674151+0000","last_change":"2023-11-06T02:08:26.059502+0000","last_active":"2023-11-06T02:11:25.674151+0000","last_peered":"2023-11-06T02:11:25.674151+0000","last_clean":"2023-11-06T02:11:25.674151+0000","last_became_active":"2023-11-06T02:08:26.058612+0000","last_became_peered":"2023-11-06T02:08:26.058612+0000","last_unstale":"2023-11-06T02:11:25.674151+0000","last_undegraded":"2023-11-06T02:11:25.674151+0000","last_fullsized":"2023-11-06T02:11:25.674151+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:10.626021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":47,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-06T02:11:25.673429+0000","last_change":"2023-11-06T02:08:26.060406+0000","last_active":"2023-11-06T02:11:25.673429+0000","last_peered":"2023-11-06T02:11:25.673429+0000","last_clean":"2023-11-06T02:11:25.673429+0000","last_became_active":"2023-11-06T02:08:26.059262+0000","last_became_peered":"2023-11-06T02:08:26.059262+0000","last_unstale":"2023-11-06T02:11:25.673429+0000","last_undegraded":"2023-11-06T02:11:25.673429+0000","last_fullsized":"2023-11-06T02:11:25.673429+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:10.626021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":52,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-06T02:11:30.105820+0000","last_change":"2023-11-06T02:08:26.559477+0000","last_active":"2023-11-06T02:11:30.105820+0000","last_peered":"2023-11-06T02:11:30.105820+0000","last_clean":"2023-11-06T02:11:30.105820+0000","last_became_active":"2023-11-06T02:08:26.559202+0000","last_became_peered":"2023-11-06T02:08:26.559202+0000","last_unstale":"2023-11-06T02:11:30.105820+0000","last_undegraded":"2023-11-06T02:11:30.105820+0000","last_fullsized":"2023-11-06T02:11:30.105820+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:55:04.552404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":46,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-06T02:11:25.674065+0000","last_change":"2023-11-06T02:08:26.059546+0000","last_active":"2023-11-06T02:11:25.674065+0000","last_peered":"2023-11-06T02:11:25.674065+0000","last_clean":"2023-11-06T02:11:25.674065+0000","last_became_active":"2023-11-06T02:08:26.058806+0000","last_became_peered":"2023-11-06T02:08:26.058806+0000","last_unstale":"2023-11-06T02:11:25.674065+0000","last_undegraded":"2023-11-06T02:11:25.674065+0000","last_fullsized":"2023-11-06T02:11:25.674065+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:23.468510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":45,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-06T02:11:25.672779+0000","last_change":"2023-11-06T02:08:26.436463+0000","last_active":"2023-11-06T02:11:25.672779+0000","last_peered":"2023-11-06T02:11:25.672779+0000","last_clean":"2023-11-06T02:11:25.672779+0000","last_became_active":"2023-11-06T02:08:26.436075+0000","last_became_peered":"2023-11-06T02:08:26.436075+0000","last_unstale":"2023-11-06T02:11:25.672779+0000","last_undegraded":"2023-11-06T02:11:25.672779+0000","last_fullsized":"2023-11-06T02:11:25.672779+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:23.468510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":47,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-06T02:11:25.673433+0000","last_change":"2023-11-06T02:08:26.436607+0000","last_active":"2023-11-06T02:11:25.673433+0000","last_peered":"2023-11-06T02:11:25.673433+0000","last_clean":"2023-11-06T02:11:25.673433+0000","last_became_active":"2023-11-06T02:08:26.435924+0000","last_became_peered":"2023-11-06T02:08:26.435924+0000","last_unstale":"2023-11-06T02:11:25.673433+0000","last_undegraded":"2023-11-06T02:11:25.673433+0000","last_fullsized":"2023-11-06T02:11:25.673433+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:23.468510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":45,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-06T02:11:25.672751+0000","last_change":"2023-11-06T02:08:26.057076+0000","last_active":"2023-11-06T02:11:25.672751+0000","last_peered":"2023-11-06T02:11:25.672751+0000","last_clean":"2023-11-06T02:11:25.672751+0000","last_became_active":"2023-11-06T02:08:26.056419+0000","last_became_peered":"2023-11-06T02:08:26.056419+0000","last_unstale":"2023-11-06T02:11:25.672751+0000","last_undegraded":"2023-11-06T02:11:25.672751+0000","last_fullsized":"2023-11-06T02:11:25.672751+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:10.626021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":53,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-06T02:11:25.617700+0000","last_change":"2023-11-06T02:08:26.058596+0000","last_active":"2023-11-06T02:11:25.617700+0000","last_peered":"2023-11-06T02:11:25.617700+0000","last_clean":"2023-11-06T02:11:25.617700+0000","last_became_active":"2023-11-06T02:08:26.057015+0000","last_became_peered":"2023-11-06T02:08:26.057015+0000","last_unstale":"2023-11-06T02:11:25.617700+0000","last_undegraded":"2023-11-06T02:11:25.617700+0000","last_fullsized":"2023-11-06T02:11:25.617700+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:55:04.552404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":10,"acting":10,"num_store_stats":1},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":1},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574889,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28248,"kb_used_data":1296,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343592,"statfs":{"total":96636764160,"available":96607838208,"internally_reserved":0,"allocated":1327104,"data_stored":910047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.86199999999999999,"15min":0.85299999999999998},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.1200000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.91300000000000003,"15min":0.91100000000000003},"min":{"1min":0.68300000000000005,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.1020000000000001,"5min":1.506,"15min":1.506},"last":0.46300000000000002}]},{"osd":1,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.82699999999999996,"15min":0.84299999999999997},"min":{"1min":0.55300000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.0329999999999999,"5min":1.161,"15min":1.161},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.91200000000000003,"15min":0.89600000000000002},"min":{"1min":0.63600000000000001,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.1910000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.61399999999999999}]},{"osd":2,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.96399999999999997,"15min":0.95699999999999996},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.2989999999999999,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.98699999999999999,"15min":0.996},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.5,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.49299999999999999}]},{"osd":3,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.88300000000000001,"15min":0.877},"min":{"1min":0.69099999999999995,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.1479999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":1.012,"5min":1.0700000000000001,"15min":1.0740000000000001},"min":{"1min":0.78900000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.2230000000000001,"5min":1.526,"15min":1.526},"last":0.64300000000000002}]},{"osd":4,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.95899999999999996,"15min":0.95799999999999996},"min":{"1min":0.64000000000000001,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.38200000000000001},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.0229999999999999,"15min":1.0369999999999999},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.47299999999999998}]},{"osd":5,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":1.0669999999999999,"15min":1.075},"min":{"1min":0.69999999999999996,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":0.503},{"interface":"front","average":{"1min":1.0780000000000001,"5min":1.117,"15min":1.129},"min":{"1min":0.73199999999999998,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.3700000000000001,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":0.55500000000000005}]},{"osd":6,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.196,"15min":1.2},"min":{"1min":0.83999999999999997,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.5009999999999999,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":0.42599999999999999},{"interface":"front","average":{"1min":1.048,"5min":1.131,"15min":1.149},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.331,"5min":1.74,"15min":1.74},"last":0.59499999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574888,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27668,"kb_used_data":716,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344172,"statfs":{"total":96636764160,"available":96608432128,"internally_reserved":0,"allocated":733184,"data_stored":319679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.52400000000000002,"5min":0.495,"15min":0.495},"max":{"1min":1.216,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.84099999999999997,"15min":0.81699999999999995},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.2370000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.496}]},{"osd":1,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.86599999999999999,"15min":0.872},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.86799999999999999,"15min":0.872},"min":{"1min":0.63,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.2330000000000001,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":1.0229999999999999}]},{"osd":2,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.877,"15min":0.89100000000000001},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.90400000000000003,"15min":0.92200000000000004},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.103,"5min":1.28,"15min":1.28},"last":1.1339999999999999}]},{"osd":3,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":0.95999999999999996,"15min":0.96799999999999997},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.3300000000000001,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.94699999999999995},{"interface":"front","average":{"1min":0.99399999999999999,"5min":0.99199999999999999,"15min":0.998},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.22,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.98799999999999999}]},{"osd":4,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.95099999999999996,"15min":0.97899999999999998},"min":{"1min":0.59499999999999997,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.1639999999999999,"5min":1.474,"15min":1.474},"last":1.163},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.99399999999999999,"15min":1.024},"min":{"1min":0.624,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.268,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":1.0669999999999999}]},{"osd":5,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.98399999999999999,"15min":1.0129999999999999},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.161,"5min":1.365,"15min":1.365},"last":1.1870000000000001},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.0760000000000001,"15min":1.1020000000000001},"min":{"1min":0.68400000000000005,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.3939999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":1.319}]},{"osd":7,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":1.083,"15min":1.1060000000000001},"min":{"1min":0.76800000000000002,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.3580000000000001,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":1.258},{"interface":"front","average":{"1min":0.99399999999999999,"5min":1.095,"15min":1.141},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.425,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":1.3340000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574889,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27668,"kb_used_data":716,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344172,"statfs":{"total":96636764160,"available":96608432128,"internally_reserved":0,"allocated":733184,"data_stored":319679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.85599999999999998,"15min":0.83799999999999997},"min":{"1min":0.63300000000000001,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.2390000000000001,"5min":1.381,"15min":1.381},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.85699999999999998,"15min":0.84699999999999998},"min":{"1min":0.63100000000000001,"5min":0.503,"15min":0.503},"max":{"1min":1.3200000000000001,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":1.1000000000000001}]},{"osd":1,"last update":"Mon Nov 6 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.86599999999999999,"15min":0.85499999999999998},"min":{"1min":0.626,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.1699999999999999,"5min":1.331,"15min":1.331},"last":1.1699999999999999},{"interface":"front","average":{"1min":0.87,"5min":0.83599999999999997,"15min":0.82299999999999995},"min":{"1min":0.57199999999999995,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.177,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.748}]},{"osd":2,"last update":"Mon Nov 6 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.93200000000000005,"15min":0.94099999999999995},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.2210000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.83199999999999996,"15min":0.80400000000000005},"min":{"1min":0.55800000000000005,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.323,"5min":1.377,"15min":1.377},"last":0.88400000000000001}]},{"osd":3,"last update":"Mon Nov 6 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":0.93200000000000005,"15min":0.92400000000000004},"min":{"1min":0.65900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.3580000000000001,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.95899999999999996},{"interface":"front","average":{"1min":1.0009999999999999,"5min":1.0409999999999999,"15min":1.0669999999999999},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.228,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":1.0009999999999999}]},{"osd":4,"last update":"Mon Nov 6 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.93100000000000005,"15min":0.92100000000000004},"min":{"1min":0.61299999999999999,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.2410000000000001,"5min":1.276,"15min":1.276},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.026},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.496,"5min":1.554,"15min":1.554},"last":1.2669999999999999}]},{"osd":6,"last update":"Mon Nov 6 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":1.0469999999999999,"15min":1.0609999999999999},"min":{"1min":0.67500000000000004,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":0.91800000000000004},{"interface":"front","average":{"1min":1.075,"5min":1.04,"15min":1.04},"min":{"1min":0.72999999999999998,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":1.0409999999999999}]},{"osd":7,"last update":"Mon Nov 6 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.083,"15min":1.0780000000000001},"min":{"1min":0.71599999999999997,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.5880000000000001,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":1.218},{"interface":"front","average":{"1min":1.161,"5min":1.1499999999999999,"15min":1.153},"min":{"1min":0.92400000000000004,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.544,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":1.137}]}]},{"osd":4,"up_from":13,"seq":55834574889,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27668,"kb_used_data":716,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344172,"statfs":{"total":96636764160,"available":96608432128,"internally_reserved":0,"allocated":733184,"data_stored":319679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:11:22 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.85599999999999998,"15min":0.83499999999999996},"min":{"1min":0.67100000000000004,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.343,"5min":1.343,"15min":1.343},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.79300000000000004,"15min":0.77700000000000002},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.0169999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.751}]},{"osd":1,"last update":"Mon Nov 6 02:11:22 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.86899999999999999,"15min":0.85199999999999998},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.083,"5min":1.462,"15min":1.462},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.92400000000000004,"15min":0.90200000000000002},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.3580000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":1.0189999999999999}]},{"osd":2,"last update":"Mon Nov 6 02:11:22 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.97599999999999998,"15min":1.0049999999999999},"min":{"1min":0.69399999999999995,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.198,"5min":1.615,"15min":1.615},"last":0.87},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.94099999999999995,"15min":0.93100000000000005},"min":{"1min":0.68000000000000005,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.3049999999999999,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":1.117}]},{"osd":3,"last update":"Mon Nov 6 02:11:22 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.90200000000000002,"15min":0.91300000000000003},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.103,"5min":1.52,"15min":1.52},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.97799999999999998,"5min":1.04,"15min":1.0600000000000001},"min":{"1min":0.63900000000000001,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.2470000000000001,"5min":1.595,"15min":1.595},"last":0.69999999999999996}]},{"osd":5,"last update":"Mon Nov 6 02:11:22 2023","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":1.0329999999999999,"15min":1.0449999999999999},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.25,"5min":1.377,"15min":1.377},"last":1.0720000000000001},{"interface":"front","average":{"1min":0.97999999999999998,"5min":1.0189999999999999,"15min":1.0429999999999999},"min":{"1min":0.67500000000000004,"5min":0.629,"15min":0.629},"max":{"1min":1.278,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":0.95399999999999996}]},{"osd":6,"last update":"Mon Nov 6 02:11:22 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":1.052,"15min":1.0800000000000001},"min":{"1min":0.68000000000000005,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.2470000000000001,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":0.79900000000000004},{"interface":"front","average":{"1min":1.071,"5min":1.157,"15min":1.1839999999999999},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.4079999999999999,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":1.167}]},{"osd":7,"last update":"Mon Nov 6 02:11:22 2023","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":1.0720000000000001,"15min":1.0860000000000001},"min":{"1min":0.748,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.2849999999999999,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.83499999999999996},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.1859999999999999,"15min":1.222},"min":{"1min":0.80000000000000004,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.3280000000000001,"5min":1.7410000000000001,"15min":1.7410000000000001},"last":0.98699999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574888,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31764,"kb_used_data":716,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340076,"statfs":{"total":96636764160,"available":96604237824,"internally_reserved":0,"allocated":733184,"data_stored":319679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.63900000000000001,"15min":0.64500000000000002},"min":{"1min":0.36499999999999999,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.95999999999999996,"5min":1.079,"15min":1.079},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.626,"5min":0.625,"15min":0.625},"min":{"1min":0.34599999999999997,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.93899999999999995,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.64600000000000002}]},{"osd":1,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.61899999999999999,"15min":0.627},"min":{"1min":0.27800000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.86799999999999999,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.746},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.72599999999999998,"15min":0.73099999999999998},"min":{"1min":0.307,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.0980000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.83599999999999997}]},{"osd":2,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.71399999999999997,"15min":0.72799999999999998},"min":{"1min":0.374,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.497},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.81200000000000006,"15min":0.81999999999999995},"min":{"1min":0.39000000000000001,"5min":0.308,"15min":0.308},"max":{"1min":1.1899999999999999,"5min":1.405,"15min":1.405},"last":0.78100000000000003}]},{"osd":4,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.92700000000000005,"15min":0.92800000000000005},"min":{"1min":0.65900000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.085,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":1.022},{"interface":"front","average":{"1min":0.98099999999999998,"5min":0.97599999999999998,"15min":0.99099999999999999},"min":{"1min":0.67800000000000005,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.236,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":1}]},{"osd":5,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":1.004,"15min":1.02},"min":{"1min":0.70699999999999996,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":1.137},{"interface":"front","average":{"1min":1.0880000000000001,"5min":1.0580000000000001,"15min":1.0740000000000001},"min":{"1min":0.74099999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.4330000000000001,"5min":2.21,"15min":2.21},"last":1.044}]},{"osd":6,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":1.0229999999999999,"15min":1.0249999999999999},"min":{"1min":0.76900000000000002,"5min":0.439,"15min":0.439},"max":{"1min":1.3799999999999999,"5min":1.395,"15min":1.395},"last":1.204},{"interface":"front","average":{"1min":1.0880000000000001,"5min":1.0600000000000001,"15min":1.0680000000000001},"min":{"1min":0.71699999999999997,"5min":0.375,"15min":0.375},"max":{"1min":1.4319999999999999,"5min":1.554,"15min":1.554},"last":1.069}]},{"osd":7,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.115,"15min":1.127},"min":{"1min":0.76300000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.5549999999999999,"5min":2.2829999999999999,"15min":2.2829999999999999},"last":1.1719999999999999},{"interface":"front","average":{"1min":1.171,"5min":1.1080000000000001,"15min":1.107},"min":{"1min":0.92200000000000004,"5min":0.497,"15min":0.497},"max":{"1min":1.504,"5min":1.504,"15min":1.504},"last":0.96399999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574889,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31812,"kb_used_data":764,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340028,"statfs":{"total":96636764160,"available":96604188672,"internally_reserved":0,"allocated":782336,"data_stored":346250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:11:20 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.53900000000000003,"15min":0.51200000000000001},"min":{"1min":0.48199999999999998,"5min":0.252,"15min":0.252},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.59099999999999997,"15min":0.55600000000000005},"min":{"1min":0.497,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":0.502}]},{"osd":1,"last update":"Mon Nov 6 02:11:20 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.60599999999999998,"15min":0.57499999999999996},"min":{"1min":0.44700000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.48,"5min":1.48,"15min":1.48},"last":0.623},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.64700000000000002,"15min":0.59399999999999997},"min":{"1min":0.45200000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.7230000000000001,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":0.75700000000000001}]},{"osd":3,"last update":"Mon Nov 6 02:11:20 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.65400000000000003,"15min":0.61599999999999999},"min":{"1min":0.58699999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.93200000000000005,"5min":1.155,"15min":1.155},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.69799999999999995,"15min":0.66000000000000003},"min":{"1min":0.52500000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.0960000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.58399999999999996}]},{"osd":4,"last update":"Mon Nov 6 02:11:20 2023","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":0.85799999999999998,"15min":0.81799999999999995},"min":{"1min":0.77500000000000002,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.89800000000000002},{"interface":"front","average":{"1min":1.0369999999999999,"5min":0.93600000000000005,"15min":0.92200000000000004},"min":{"1min":0.755,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.395,"5min":1.395,"15min":1.395},"last":0.95099999999999996}]},{"osd":5,"last update":"Mon Nov 6 02:11:20 2023","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":0.96199999999999997,"15min":0.94899999999999995},"min":{"1min":0.74099999999999999,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.7729999999999999,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":1.0189999999999999,"5min":0.95699999999999996,"15min":0.94099999999999995},"min":{"1min":0.67200000000000004,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.615,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.97099999999999997}]},{"osd":6,"last update":"Mon Nov 6 02:11:20 2023","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":0.96899999999999997,"15min":0.94799999999999995},"min":{"1min":0.79200000000000004,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":0.91700000000000004},{"interface":"front","average":{"1min":1.087,"5min":1.0129999999999999,"15min":1.002},"min":{"1min":0.81499999999999995,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.675,"5min":1.677,"15min":1.677},"last":1.0369999999999999}]},{"osd":7,"last update":"Mon Nov 6 02:11:20 2023","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":1.0089999999999999,"15min":0.98599999999999999},"min":{"1min":0.72499999999999998,"5min":0.622,"15min":0.622},"max":{"1min":1.659,"5min":1.659,"15min":1.659},"last":0.98699999999999999},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.0620000000000001,"15min":1.048},"min":{"1min":0.89300000000000002,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.4650000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":1.1160000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574889,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31764,"kb_used_data":716,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340076,"statfs":{"total":96636764160,"available":96604237824,"internally_reserved":0,"allocated":733184,"data_stored":319679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:11:22 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.63700000000000001,"15min":0.629},"min":{"1min":0.28699999999999998,"5min":0.224,"15min":0.224},"max":{"1min":1.603,"5min":1.603,"15min":1.603},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.67400000000000004,"15min":0.65900000000000003},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":1.0760000000000001}]},{"osd":2,"last update":"Mon Nov 6 02:11:22 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.63700000000000001,"15min":0.627},"min":{"1min":0.33600000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.649,"5min":1.649,"15min":1.649},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.755,"15min":0.748},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.1919999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":1.1539999999999999}]},{"osd":3,"last update":"Mon Nov 6 02:11:22 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.79500000000000004,"15min":0.80000000000000004},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.2350000000000001,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":1.0449999999999999},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.80800000000000005,"15min":0.81000000000000005},"min":{"1min":0.41299999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.5,"5min":1.8149999999999999,"15min":1.8149999999999999},"last":0.86299999999999999}]},{"osd":4,"last update":"Mon Nov 6 02:11:22 2023","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.97699999999999998,"15min":0.98699999999999999},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.5780000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":1.26},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.98099999999999998,"15min":0.98899999999999999},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.4099999999999999,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.81999999999999995}]},{"osd":5,"last update":"Mon Nov 6 02:11:22 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":0.97499999999999998,"15min":0.96899999999999997},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.718,"5min":1.718,"15min":1.718},"last":1.0169999999999999},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.069,"15min":1.075},"min":{"1min":0.66500000000000004,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.403,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":1.2110000000000001}]},{"osd":6,"last update":"Mon Nov 6 02:11:22 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.0920000000000001,"15min":1.099},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.6859999999999999,"5min":1.742,"15min":1.742},"last":1.1850000000000001},{"interface":"front","average":{"1min":1.0700000000000001,"5min":1.0669999999999999,"15min":1.073},"min":{"1min":0.64400000000000002,"5min":0.629,"15min":0.629},"max":{"1min":1.6339999999999999,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":1.129}]},{"osd":7,"last update":"Mon Nov 6 02:11:22 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.115,"15min":1.119},"min":{"1min":0.71499999999999997,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.534,"5min":1.534,"15min":1.534},"last":0.97199999999999998},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.1499999999999999,"15min":1.157},"min":{"1min":0.68700000000000006,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.7629999999999999,"5min":1.893,"15min":1.893},"last":1.3400000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574889,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32340,"kb_used_data":1292,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339500,"statfs":{"total":96636764160,"available":96603648000,"internally_reserved":0,"allocated":1323008,"data_stored":909028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 02:11:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.33700000000000002,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64000000000000001,"15min":0.63700000000000001},"min":{"1min":0.38700000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.93400000000000005,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.49299999999999999}]},{"osd":2,"last update":"Mon Nov 6 02:11:25 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.67100000000000004,"15min":0.66700000000000004},"min":{"1min":0.505,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.67800000000000005,"15min":0.67600000000000005},"min":{"1min":0.35199999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.95599999999999996,"5min":1.141,"15min":1.141},"last":0.64500000000000002}]},{"osd":3,"last update":"Mon Nov 6 02:11:25 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.64000000000000001,"15min":0.63200000000000001},"min":{"1min":0.376,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.0600000000000001,"5min":1.429,"15min":1.429},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.753,"15min":0.73799999999999999},"min":{"1min":0.44,"5min":0.317,"15min":0.317},"max":{"1min":1.1299999999999999,"5min":1.25,"15min":1.25},"last":0.96099999999999997}]},{"osd":4,"last update":"Mon Nov 6 02:11:25 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.85599999999999998,"15min":0.83599999999999997},"min":{"1min":0.59499999999999997,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.1579999999999999,"5min":1.341,"15min":1.341},"last":1.0189999999999999},{"interface":"front","average":{"1min":0.995,"5min":0.94299999999999995,"15min":0.93899999999999995},"min":{"1min":0.65100000000000002,"5min":0.5,"15min":0.5},"max":{"1min":1.3600000000000001,"5min":1.387,"15min":1.387},"last":1.0449999999999999}]},{"osd":5,"last update":"Mon Nov 6 02:11:25 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.94199999999999995,"15min":0.94599999999999995},"min":{"1min":0.63300000000000001,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.119,"5min":1.339,"15min":1.339},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.89700000000000002,"15min":0.88},"min":{"1min":0.72899999999999998,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.325,"5min":1.448,"15min":1.448},"last":1.0800000000000001}]},{"osd":6,"last update":"Mon Nov 6 02:11:25 2023","interfaces":[{"interface":"back","average":{"1min":1.0960000000000001,"5min":1.0409999999999999,"15min":1.038},"min":{"1min":0.66700000000000004,"5min":0.625,"15min":0.625},"max":{"1min":1.573,"5min":1.589,"15min":1.589},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.0309999999999999,"5min":1.0169999999999999,"15min":1.0049999999999999},"min":{"1min":0.67600000000000005,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.3899999999999999,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.95099999999999996}]},{"osd":7,"last update":"Mon Nov 6 02:11:25 2023","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":1.008,"15min":1.0149999999999999},"min":{"1min":0.71299999999999997,"5min":0.504,"15min":0.504},"max":{"1min":1.341,"5min":1.421,"15min":1.421},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.98599999999999999,"5min":0.98899999999999999,"15min":1.002},"min":{"1min":0.69899999999999995,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.3029999999999999,"5min":1.556,"15min":1.556},"last":0.81499999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T02:11:34.598 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T02:11:34.749 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.753+0000 7fed3485a640 1 Processor -- start 2023-11-06T02:11:34.750 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.753+0000 7fed3485a640 1 -- start start 2023-11-06T02:11:34.752 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.753+0000 7fed3485a640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fed300b0d80 0x7fed3006b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:34.752 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.753+0000 7fed3485a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fed300b1420 0x7fed3006b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:34.752 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.753+0000 7fed3485a640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed300cf710 con 0x7fed300b1970 2023-11-06T02:11:34.752 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.753+0000 7fed3485a640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed300cffb0 con 0x7fed300b0d80 2023-11-06T02:11:34.753 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.753+0000 7fed3485a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed300cf9f0 con 0x7fed300b1420 2023-11-06T02:11:34.753 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.753+0000 7fed2e59c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fed300b0d80 0x7fed3006b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:34.753 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.753+0000 7fed2ed9d640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fed300b1970 0x7fed300cd280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:46656/0 (socket says 172.21.15.49:46656) 2023-11-06T02:11:34.753 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.753+0000 7fed2ed9d640 1 -- 172.21.15.49:0/4001154139 learned_addr learned my addr 172.21.15.49:0/4001154139 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:11:34.753 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.753+0000 7fed2dd9b640 1 --2- 172.21.15.49:0/4001154139 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fed300b1420 0x7fed3006b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:34.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.753+0000 7fed2e59c640 1 -- 172.21.15.49:0/4001154139 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fed300b1420 msgr2=0x7fed3006b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:34.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.753+0000 7fed2e59c640 1 --2- 172.21.15.49:0/4001154139 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fed300b1420 0x7fed3006b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:34.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.753+0000 7fed2e59c640 1 -- 172.21.15.49:0/4001154139 >> v1:172.21.15.49:6789/0 conn(0x7fed300b1970 legacy=0x7fed300cd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:34.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.753+0000 7fed2e59c640 1 -- 172.21.15.49:0/4001154139 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed300cfcd0 con 0x7fed300b0d80 2023-11-06T02:11:34.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.753+0000 7fed2dd9b640 1 --2- 172.21.15.49:0/4001154139 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fed300b1420 0x7fed3006b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:11:34.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.757+0000 7fed2e59c640 1 --2- 172.21.15.49:0/4001154139 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed300b0d80 0x7fed3006b090 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fed20009de0 tx=0x7fed2001c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=784c4e56d226edca server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:34.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.757+0000 7fed2d59a640 1 -- 172.21.15.49:0/4001154139 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed200a59e0 con 0x7fed300b0d80 2023-11-06T02:11:34.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.757+0000 7fed2d59a640 1 -- 172.21.15.49:0/4001154139 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed200a5b40 con 0x7fed300b0d80 2023-11-06T02:11:34.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.757+0000 7fed2d59a640 1 -- 172.21.15.49:0/4001154139 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed200ad440 con 0x7fed300b0d80 2023-11-06T02:11:34.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.757+0000 7fed3485a640 1 -- 172.21.15.49:0/4001154139 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed300b0d80 msgr2=0x7fed3006b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:34.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.757+0000 7fed3485a640 1 --2- 172.21.15.49:0/4001154139 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed300b0d80 0x7fed3006b090 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fed20009de0 tx=0x7fed2001c900 comp rx=0 tx=0).stop 2023-11-06T02:11:34.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.757+0000 7fed3485a640 1 -- 172.21.15.49:0/4001154139 shutdown_connections 2023-11-06T02:11:34.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.757+0000 7fed3485a640 1 --2- 172.21.15.49:0/4001154139 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fed300b1420 0x7fed3006b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:34.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.757+0000 7fed3485a640 1 --2- 172.21.15.49:0/4001154139 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed300b0d80 0x7fed3006b090 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:34.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.757+0000 7fed3485a640 1 -- 172.21.15.49:0/4001154139 >> 172.21.15.49:0/4001154139 conn(0x7fed300d0890 msgr2=0x7fed300d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:34.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.757+0000 7fed3485a640 1 -- 172.21.15.49:0/4001154139 shutdown_connections 2023-11-06T02:11:34.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.757+0000 7fed3485a640 1 -- 172.21.15.49:0/4001154139 wait complete. 2023-11-06T02:11:34.762 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.757+0000 7fed3485a640 1 Processor -- start 2023-11-06T02:11:34.762 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.757+0000 7fed3485a640 1 -- start start 2023-11-06T02:11:34.762 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.757+0000 7fed3485a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed300b0d80 0x7fed30153b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:34.762 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.757+0000 7fed3485a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fed300b1420 0x7fed3015ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:34.763 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.757+0000 7fed3485a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fed300b1970 0x7fed3015f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:34.763 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.757+0000 7fed3485a640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed300adbf0 con 0x7fed300b0d80 2023-11-06T02:11:34.763 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.757+0000 7fed3485a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed3015d290 con 0x7fed300b1970 2023-11-06T02:11:34.763 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.757+0000 7fed3485a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed301f6400 con 0x7fed300b1420 2023-11-06T02:11:34.763 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.757+0000 7fed2e59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed300b0d80 0x7fed30153b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:34.764 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.757+0000 7fed2e59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed300b0d80 0x7fed30153b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48914/0 (socket says 172.21.15.49:48914) 2023-11-06T02:11:34.764 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.757+0000 7fed2e59c640 1 -- 172.21.15.49:0/2834082017 learned_addr learned my addr 172.21.15.49:0/2834082017 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:34.764 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.761+0000 7fed2dd9b640 1 --2- 172.21.15.49:0/2834082017 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fed300b1420 0x7fed3015ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:34.764 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.761+0000 7fed2e59c640 1 -- 172.21.15.49:0/2834082017 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fed300b1420 msgr2=0x7fed3015ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:34.764 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.761+0000 7fed2ed9d640 1 --2- 172.21.15.49:0/2834082017 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fed300b1970 0x7fed3015f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:34.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.761+0000 7fed2e59c640 1 --2- 172.21.15.49:0/2834082017 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fed300b1420 0x7fed3015ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:34.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.761+0000 7fed2e59c640 1 -- 172.21.15.49:0/2834082017 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fed300b1970 msgr2=0x7fed3015f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:34.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.761+0000 7fed2e59c640 1 --2- 172.21.15.49:0/2834082017 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fed300b1970 0x7fed3015f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:34.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.761+0000 7fed2e59c640 1 -- 172.21.15.49:0/2834082017 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed20009a70 con 0x7fed300b0d80 2023-11-06T02:11:34.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.761+0000 7fed2ed9d640 1 --2- 172.21.15.49:0/2834082017 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fed300b1970 0x7fed3015f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:11:34.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.761+0000 7fed2e59c640 1 --2- 172.21.15.49:0/2834082017 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed300b0d80 0x7fed30153b90 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fed200b0040 tx=0x7fed200196a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:34.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.761+0000 7fed1effd640 1 -- 172.21.15.49:0/2834082017 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed200a59e0 con 0x7fed300b0d80 2023-11-06T02:11:34.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.761+0000 7fed1effd640 1 -- 172.21.15.49:0/2834082017 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed200b7410 con 0x7fed300b0d80 2023-11-06T02:11:34.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.761+0000 7fed1effd640 1 -- 172.21.15.49:0/2834082017 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed200b77c0 con 0x7fed300b0d80 2023-11-06T02:11:34.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.761+0000 7fed3485a640 1 -- 172.21.15.49:0/2834082017 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed301f61e0 con 0x7fed300b0d80 2023-11-06T02:11:34.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.761+0000 7fed3485a640 1 -- 172.21.15.49:0/2834082017 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed300ad530 con 0x7fed300b0d80 2023-11-06T02:11:34.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.761+0000 7fed3485a640 1 -- 172.21.15.49:0/2834082017 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed300add30 con 0x7fed300b0d80 2023-11-06T02:11:34.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.761+0000 7fed1effd640 1 -- 172.21.15.49:0/2834082017 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fed200b4050 con 0x7fed300b0d80 2023-11-06T02:11:34.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.761+0000 7fed1effd640 1 --2- 172.21.15.49:0/2834082017 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fed10065a60 0x7fed10067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:34.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.761+0000 7fed1effd640 1 -- 172.21.15.49:0/2834082017 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fed2011e630 con 0x7fed300b0d80 2023-11-06T02:11:34.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.761+0000 7fed2dd9b640 1 --2- 172.21.15.49:0/2834082017 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fed10065a60 0x7fed10067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:34.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.761+0000 7fed2dd9b640 1 --2- 172.21.15.49:0/2834082017 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fed10065a60 0x7fed10067f20 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fed24006fd0 tx=0x7fed24008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:34.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.765+0000 7fed1effd640 1 -- 172.21.15.49:0/2834082017 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fed200ebfe0 con 0x7fed300b0d80 2023-11-06T02:11:34.911 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.913+0000 7fed3485a640 1 -- 172.21.15.49:0/2834082017 --> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fed30169ad0 con 0x7fed10065a60 2023-11-06T02:11:34.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.933+0000 7fed1effd640 1 -- 172.21.15.49:0/2834082017 <== mgr.4100 v2:172.21.15.150:6832/38104 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+296052 (secure 0 0 0) 0x7fed2400c040 con 0x7fed10065a60 2023-11-06T02:11:34.937 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:11:34.941 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2023-11-06T02:11:34.942 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.941+0000 7fed3485a640 1 -- 172.21.15.49:0/2834082017 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fed10065a60 msgr2=0x7fed10067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:34.942 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.941+0000 7fed3485a640 1 --2- 172.21.15.49:0/2834082017 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fed10065a60 0x7fed10067f20 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fed24006fd0 tx=0x7fed24008040 comp rx=0 tx=0).stop 2023-11-06T02:11:34.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.941+0000 7fed3485a640 1 -- 172.21.15.49:0/2834082017 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed300b0d80 msgr2=0x7fed30153b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:34.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.941+0000 7fed3485a640 1 --2- 172.21.15.49:0/2834082017 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed300b0d80 0x7fed30153b90 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fed200b0040 tx=0x7fed200196a0 comp rx=0 tx=0).stop 2023-11-06T02:11:34.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.941+0000 7fed3485a640 1 -- 172.21.15.49:0/2834082017 shutdown_connections 2023-11-06T02:11:34.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.941+0000 7fed3485a640 1 --2- 172.21.15.49:0/2834082017 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fed10065a60 0x7fed10067f20 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:34.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.941+0000 7fed3485a640 1 --2- 172.21.15.49:0/2834082017 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fed300b1970 0x7fed3015f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:34.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.941+0000 7fed3485a640 1 --2- 172.21.15.49:0/2834082017 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fed300b1420 0x7fed3015ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:34.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.941+0000 7fed3485a640 1 --2- 172.21.15.49:0/2834082017 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed300b0d80 0x7fed30153b90 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:34.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.941+0000 7fed3485a640 1 -- 172.21.15.49:0/2834082017 >> 172.21.15.49:0/2834082017 conn(0x7fed300d0890 msgr2=0x7fed3006fb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:34.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.941+0000 7fed3485a640 1 -- 172.21.15.49:0/2834082017 shutdown_connections 2023-11-06T02:11:34.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:34.941+0000 7fed3485a640 1 -- 172.21.15.49:0/2834082017 wait complete. 2023-11-06T02:11:34.953 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":156,"stamp":"2023-11-06T02:11:33.239742+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":28,"acting":28,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":36,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":238932,"kb_used_data":6932,"kb_used_omap":0,"kb_used_meta":231936,"kb_avail":754735788,"statfs":{"total":773094113280,"available":772849446912,"internally_reserved":0,"allocated":7098368,"data_stored":3763720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":237502464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.535345"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.109328+0000","last_change":"2023-11-06T02:11:30.139772+0000","last_active":"2023-11-06T02:11:31.109328+0000","last_peered":"2023-11-06T02:11:31.109328+0000","last_clean":"2023-11-06T02:11:31.109328+0000","last_became_active":"2023-11-06T02:11:30.139447+0000","last_became_peered":"2023-11-06T02:11:30.139447+0000","last_unstale":"2023-11-06T02:11:31.109328+0000","last_undegraded":"2023-11-06T02:11:31.109328+0000","last_fullsized":"2023-11-06T02:11:31.109328+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:24:49.653824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.109357+0000","last_change":"2023-11-06T02:11:30.140958+0000","last_active":"2023-11-06T02:11:31.109357+0000","last_peered":"2023-11-06T02:11:31.109357+0000","last_clean":"2023-11-06T02:11:31.109357+0000","last_became_active":"2023-11-06T02:11:30.140693+0000","last_became_peered":"2023-11-06T02:11:30.140693+0000","last_unstale":"2023-11-06T02:11:31.109357+0000","last_undegraded":"2023-11-06T02:11:31.109357+0000","last_fullsized":"2023-11-06T02:11:31.109357+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:49:19.353757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.109510+0000","last_change":"2023-11-06T02:11:30.123394+0000","last_active":"2023-11-06T02:11:31.109510+0000","last_peered":"2023-11-06T02:11:31.109510+0000","last_clean":"2023-11-06T02:11:31.109510+0000","last_became_active":"2023-11-06T02:11:30.123113+0000","last_became_peered":"2023-11-06T02:11:30.123113+0000","last_unstale":"2023-11-06T02:11:31.109510+0000","last_undegraded":"2023-11-06T02:11:31.109510+0000","last_fullsized":"2023-11-06T02:11:31.109510+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:44:35.667968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.110281+0000","last_change":"2023-11-06T02:11:30.113941+0000","last_active":"2023-11-06T02:11:31.110281+0000","last_peered":"2023-11-06T02:11:31.110281+0000","last_clean":"2023-11-06T02:11:31.110281+0000","last_became_active":"2023-11-06T02:11:30.113672+0000","last_became_peered":"2023-11-06T02:11:30.113672+0000","last_unstale":"2023-11-06T02:11:31.110281+0000","last_undegraded":"2023-11-06T02:11:31.110281+0000","last_fullsized":"2023-11-06T02:11:31.110281+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:06:25.213807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.110013+0000","last_change":"2023-11-06T02:11:30.150778+0000","last_active":"2023-11-06T02:11:31.110013+0000","last_peered":"2023-11-06T02:11:31.110013+0000","last_clean":"2023-11-06T02:11:31.110013+0000","last_became_active":"2023-11-06T02:11:30.150466+0000","last_became_peered":"2023-11-06T02:11:30.150466+0000","last_unstale":"2023-11-06T02:11:31.110013+0000","last_undegraded":"2023-11-06T02:11:31.110013+0000","last_fullsized":"2023-11-06T02:11:31.110013+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:21:14.911044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":200,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-06T02:11:25.673533+0000","last_change":"2023-11-06T02:08:20.523427+0000","last_active":"2023-11-06T02:11:25.673533+0000","last_peered":"2023-11-06T02:11:25.673533+0000","last_clean":"2023-11-06T02:11:25.673533+0000","last_became_active":"2023-11-06T02:08:20.521089+0000","last_became_peered":"2023-11-06T02:08:20.521089+0000","last_unstale":"2023-11-06T02:11:25.673533+0000","last_undegraded":"2023-11-06T02:11:25.673533+0000","last_fullsized":"2023-11-06T02:11:25.673533+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:19.252348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:19.252348+0000","last_clean_scrub_stamp":"2023-11-06T02:08:19.252348+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:30:15.566397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":46,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-06T02:11:25.674151+0000","last_change":"2023-11-06T02:08:26.059502+0000","last_active":"2023-11-06T02:11:25.674151+0000","last_peered":"2023-11-06T02:11:25.674151+0000","last_clean":"2023-11-06T02:11:25.674151+0000","last_became_active":"2023-11-06T02:08:26.058612+0000","last_became_peered":"2023-11-06T02:08:26.058612+0000","last_unstale":"2023-11-06T02:11:25.674151+0000","last_undegraded":"2023-11-06T02:11:25.674151+0000","last_fullsized":"2023-11-06T02:11:25.674151+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:10.626021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":47,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-06T02:11:25.673429+0000","last_change":"2023-11-06T02:08:26.060406+0000","last_active":"2023-11-06T02:11:25.673429+0000","last_peered":"2023-11-06T02:11:25.673429+0000","last_clean":"2023-11-06T02:11:25.673429+0000","last_became_active":"2023-11-06T02:08:26.059262+0000","last_became_peered":"2023-11-06T02:08:26.059262+0000","last_unstale":"2023-11-06T02:11:25.673429+0000","last_undegraded":"2023-11-06T02:11:25.673429+0000","last_fullsized":"2023-11-06T02:11:25.673429+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:10.626021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":52,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-06T02:11:30.105820+0000","last_change":"2023-11-06T02:08:26.559477+0000","last_active":"2023-11-06T02:11:30.105820+0000","last_peered":"2023-11-06T02:11:30.105820+0000","last_clean":"2023-11-06T02:11:30.105820+0000","last_became_active":"2023-11-06T02:08:26.559202+0000","last_became_peered":"2023-11-06T02:08:26.559202+0000","last_unstale":"2023-11-06T02:11:30.105820+0000","last_undegraded":"2023-11-06T02:11:30.105820+0000","last_fullsized":"2023-11-06T02:11:30.105820+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:55:04.552404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":46,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-06T02:11:25.674065+0000","last_change":"2023-11-06T02:08:26.059546+0000","last_active":"2023-11-06T02:11:25.674065+0000","last_peered":"2023-11-06T02:11:25.674065+0000","last_clean":"2023-11-06T02:11:25.674065+0000","last_became_active":"2023-11-06T02:08:26.058806+0000","last_became_peered":"2023-11-06T02:08:26.058806+0000","last_unstale":"2023-11-06T02:11:25.674065+0000","last_undegraded":"2023-11-06T02:11:25.674065+0000","last_fullsized":"2023-11-06T02:11:25.674065+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:23.468510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":45,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-06T02:11:25.672779+0000","last_change":"2023-11-06T02:08:26.436463+0000","last_active":"2023-11-06T02:11:25.672779+0000","last_peered":"2023-11-06T02:11:25.672779+0000","last_clean":"2023-11-06T02:11:25.672779+0000","last_became_active":"2023-11-06T02:08:26.436075+0000","last_became_peered":"2023-11-06T02:08:26.436075+0000","last_unstale":"2023-11-06T02:11:25.672779+0000","last_undegraded":"2023-11-06T02:11:25.672779+0000","last_fullsized":"2023-11-06T02:11:25.672779+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:23.468510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":47,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-06T02:11:25.673433+0000","last_change":"2023-11-06T02:08:26.436607+0000","last_active":"2023-11-06T02:11:25.673433+0000","last_peered":"2023-11-06T02:11:25.673433+0000","last_clean":"2023-11-06T02:11:25.673433+0000","last_became_active":"2023-11-06T02:08:26.435924+0000","last_became_peered":"2023-11-06T02:08:26.435924+0000","last_unstale":"2023-11-06T02:11:25.673433+0000","last_undegraded":"2023-11-06T02:11:25.673433+0000","last_fullsized":"2023-11-06T02:11:25.673433+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:23.468510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":45,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-06T02:11:25.672751+0000","last_change":"2023-11-06T02:08:26.057076+0000","last_active":"2023-11-06T02:11:25.672751+0000","last_peered":"2023-11-06T02:11:25.672751+0000","last_clean":"2023-11-06T02:11:25.672751+0000","last_became_active":"2023-11-06T02:08:26.056419+0000","last_became_peered":"2023-11-06T02:08:26.056419+0000","last_unstale":"2023-11-06T02:11:25.672751+0000","last_undegraded":"2023-11-06T02:11:25.672751+0000","last_fullsized":"2023-11-06T02:11:25.672751+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:10.626021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:29.703192+0000","last_change":"2023-11-06T02:11:29.703192+0000","last_active":"2023-11-06T02:11:29.703192+0000","last_peered":"2023-11-06T02:11:29.703192+0000","last_clean":"2023-11-06T02:11:29.703192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:29.703192+0000","last_undegraded":"2023-11-06T02:11:29.703192+0000","last_fullsized":"2023-11-06T02:11:29.703192+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T02:11:28.698549+0000","last_change":"2023-11-06T02:11:28.698549+0000","last_active":"2023-11-06T02:11:28.698549+0000","last_peered":"2023-11-06T02:11:28.698549+0000","last_clean":"2023-11-06T02:11:28.698549+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:11:28.698549+0000","last_undegraded":"2023-11-06T02:11:28.698549+0000","last_fullsized":"2023-11-06T02:11:28.698549+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":53,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-06T02:11:25.617700+0000","last_change":"2023-11-06T02:08:26.058596+0000","last_active":"2023-11-06T02:11:25.617700+0000","last_peered":"2023-11-06T02:11:25.617700+0000","last_clean":"2023-11-06T02:11:25.617700+0000","last_became_active":"2023-11-06T02:08:26.057015+0000","last_became_peered":"2023-11-06T02:08:26.057015+0000","last_unstale":"2023-11-06T02:11:25.617700+0000","last_undegraded":"2023-11-06T02:11:25.617700+0000","last_fullsized":"2023-11-06T02:11:25.617700+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:55:04.552404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":10,"acting":10,"num_store_stats":1},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":1},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574889,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28248,"kb_used_data":1296,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343592,"statfs":{"total":96636764160,"available":96607838208,"internally_reserved":0,"allocated":1327104,"data_stored":910047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.86199999999999999,"15min":0.85299999999999998},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.1200000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.91300000000000003,"15min":0.91100000000000003},"min":{"1min":0.68300000000000005,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.1020000000000001,"5min":1.506,"15min":1.506},"last":0.46300000000000002}]},{"osd":1,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.82699999999999996,"15min":0.84299999999999997},"min":{"1min":0.55300000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.0329999999999999,"5min":1.161,"15min":1.161},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.91200000000000003,"15min":0.89600000000000002},"min":{"1min":0.63600000000000001,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.1910000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.61399999999999999}]},{"osd":2,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.96399999999999997,"15min":0.95699999999999996},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.2989999999999999,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.98699999999999999,"15min":0.996},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.5,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.49299999999999999}]},{"osd":3,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.88300000000000001,"15min":0.877},"min":{"1min":0.69099999999999995,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.1479999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":1.012,"5min":1.0700000000000001,"15min":1.0740000000000001},"min":{"1min":0.78900000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.2230000000000001,"5min":1.526,"15min":1.526},"last":0.64300000000000002}]},{"osd":4,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.95899999999999996,"15min":0.95799999999999996},"min":{"1min":0.64000000000000001,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.38200000000000001},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.0229999999999999,"15min":1.0369999999999999},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.47299999999999998}]},{"osd":5,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":1.0669999999999999,"15min":1.075},"min":{"1min":0.69999999999999996,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":0.503},{"interface":"front","average":{"1min":1.0780000000000001,"5min":1.117,"15min":1.129},"min":{"1min":0.73199999999999998,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.3700000000000001,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":0.55500000000000005}]},{"osd":6,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.196,"15min":1.2},"min":{"1min":0.83999999999999997,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.5009999999999999,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":0.42599999999999999},{"interface":"front","average":{"1min":1.048,"5min":1.131,"15min":1.149},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.331,"5min":1.74,"15min":1.74},"last":0.59499999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574888,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27668,"kb_used_data":716,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344172,"statfs":{"total":96636764160,"available":96608432128,"internally_reserved":0,"allocated":733184,"data_stored":319679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.52400000000000002,"5min":0.495,"15min":0.495},"max":{"1min":1.216,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.84099999999999997,"15min":0.81699999999999995},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.2370000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.496}]},{"osd":1,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.86599999999999999,"15min":0.872},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.86799999999999999,"15min":0.872},"min":{"1min":0.63,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.2330000000000001,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":1.0229999999999999}]},{"osd":2,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.877,"15min":0.89100000000000001},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.90400000000000003,"15min":0.92200000000000004},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.103,"5min":1.28,"15min":1.28},"last":1.1339999999999999}]},{"osd":3,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":0.95999999999999996,"15min":0.96799999999999997},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.3300000000000001,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.94699999999999995},{"interface":"front","average":{"1min":0.99399999999999999,"5min":0.99199999999999999,"15min":0.998},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.22,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.98799999999999999}]},{"osd":4,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.95099999999999996,"15min":0.97899999999999998},"min":{"1min":0.59499999999999997,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.1639999999999999,"5min":1.474,"15min":1.474},"last":1.163},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.99399999999999999,"15min":1.024},"min":{"1min":0.624,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.268,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":1.0669999999999999}]},{"osd":5,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.98399999999999999,"15min":1.0129999999999999},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.161,"5min":1.365,"15min":1.365},"last":1.1870000000000001},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.0760000000000001,"15min":1.1020000000000001},"min":{"1min":0.68400000000000005,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.3939999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":1.319}]},{"osd":7,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":1.083,"15min":1.1060000000000001},"min":{"1min":0.76800000000000002,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.3580000000000001,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":1.258},{"interface":"front","average":{"1min":0.99399999999999999,"5min":1.095,"15min":1.141},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.425,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":1.3340000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574889,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27668,"kb_used_data":716,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344172,"statfs":{"total":96636764160,"available":96608432128,"internally_reserved":0,"allocated":733184,"data_stored":319679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.85599999999999998,"15min":0.83799999999999997},"min":{"1min":0.63300000000000001,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.2390000000000001,"5min":1.381,"15min":1.381},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.85699999999999998,"15min":0.84699999999999998},"min":{"1min":0.63100000000000001,"5min":0.503,"15min":0.503},"max":{"1min":1.3200000000000001,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":1.1000000000000001}]},{"osd":1,"last update":"Mon Nov 6 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.86599999999999999,"15min":0.85499999999999998},"min":{"1min":0.626,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.1699999999999999,"5min":1.331,"15min":1.331},"last":1.1699999999999999},{"interface":"front","average":{"1min":0.87,"5min":0.83599999999999997,"15min":0.82299999999999995},"min":{"1min":0.57199999999999995,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.177,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.748}]},{"osd":2,"last update":"Mon Nov 6 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.93200000000000005,"15min":0.94099999999999995},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.2210000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.83199999999999996,"15min":0.80400000000000005},"min":{"1min":0.55800000000000005,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.323,"5min":1.377,"15min":1.377},"last":0.88400000000000001}]},{"osd":3,"last update":"Mon Nov 6 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":0.93200000000000005,"15min":0.92400000000000004},"min":{"1min":0.65900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.3580000000000001,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.95899999999999996},{"interface":"front","average":{"1min":1.0009999999999999,"5min":1.0409999999999999,"15min":1.0669999999999999},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.228,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":1.0009999999999999}]},{"osd":4,"last update":"Mon Nov 6 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.93100000000000005,"15min":0.92100000000000004},"min":{"1min":0.61299999999999999,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.2410000000000001,"5min":1.276,"15min":1.276},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.026},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.496,"5min":1.554,"15min":1.554},"last":1.2669999999999999}]},{"osd":6,"last update":"Mon Nov 6 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":1.0469999999999999,"15min":1.0609999999999999},"min":{"1min":0.67500000000000004,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":0.91800000000000004},{"interface":"front","average":{"1min":1.075,"5min":1.04,"15min":1.04},"min":{"1min":0.72999999999999998,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":1.0409999999999999}]},{"osd":7,"last update":"Mon Nov 6 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.083,"15min":1.0780000000000001},"min":{"1min":0.71599999999999997,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.5880000000000001,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":1.218},{"interface":"front","average":{"1min":1.161,"5min":1.1499999999999999,"15min":1.153},"min":{"1min":0.92400000000000004,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.544,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":1.137}]}]},{"osd":4,"up_from":13,"seq":55834574889,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27668,"kb_used_data":716,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344172,"statfs":{"total":96636764160,"available":96608432128,"internally_reserved":0,"allocated":733184,"data_stored":319679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:11:22 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.85599999999999998,"15min":0.83499999999999996},"min":{"1min":0.67100000000000004,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.343,"5min":1.343,"15min":1.343},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.79300000000000004,"15min":0.77700000000000002},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.0169999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.751}]},{"osd":1,"last update":"Mon Nov 6 02:11:22 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.86899999999999999,"15min":0.85199999999999998},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.083,"5min":1.462,"15min":1.462},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.92400000000000004,"15min":0.90200000000000002},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.3580000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":1.0189999999999999}]},{"osd":2,"last update":"Mon Nov 6 02:11:22 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.97599999999999998,"15min":1.0049999999999999},"min":{"1min":0.69399999999999995,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.198,"5min":1.615,"15min":1.615},"last":0.87},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.94099999999999995,"15min":0.93100000000000005},"min":{"1min":0.68000000000000005,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.3049999999999999,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":1.117}]},{"osd":3,"last update":"Mon Nov 6 02:11:22 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.90200000000000002,"15min":0.91300000000000003},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.103,"5min":1.52,"15min":1.52},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.97799999999999998,"5min":1.04,"15min":1.0600000000000001},"min":{"1min":0.63900000000000001,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.2470000000000001,"5min":1.595,"15min":1.595},"last":0.69999999999999996}]},{"osd":5,"last update":"Mon Nov 6 02:11:22 2023","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":1.0329999999999999,"15min":1.0449999999999999},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.25,"5min":1.377,"15min":1.377},"last":1.0720000000000001},{"interface":"front","average":{"1min":0.97999999999999998,"5min":1.0189999999999999,"15min":1.0429999999999999},"min":{"1min":0.67500000000000004,"5min":0.629,"15min":0.629},"max":{"1min":1.278,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":0.95399999999999996}]},{"osd":6,"last update":"Mon Nov 6 02:11:22 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":1.052,"15min":1.0800000000000001},"min":{"1min":0.68000000000000005,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.2470000000000001,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":0.79900000000000004},{"interface":"front","average":{"1min":1.071,"5min":1.157,"15min":1.1839999999999999},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.4079999999999999,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":1.167}]},{"osd":7,"last update":"Mon Nov 6 02:11:22 2023","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":1.0720000000000001,"15min":1.0860000000000001},"min":{"1min":0.748,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.2849999999999999,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.83499999999999996},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.1859999999999999,"15min":1.222},"min":{"1min":0.80000000000000004,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.3280000000000001,"5min":1.7410000000000001,"15min":1.7410000000000001},"last":0.98699999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574888,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31764,"kb_used_data":716,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340076,"statfs":{"total":96636764160,"available":96604237824,"internally_reserved":0,"allocated":733184,"data_stored":319679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.63900000000000001,"15min":0.64500000000000002},"min":{"1min":0.36499999999999999,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.95999999999999996,"5min":1.079,"15min":1.079},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.626,"5min":0.625,"15min":0.625},"min":{"1min":0.34599999999999997,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.93899999999999995,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.64600000000000002}]},{"osd":1,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.61899999999999999,"15min":0.627},"min":{"1min":0.27800000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.86799999999999999,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.746},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.72599999999999998,"15min":0.73099999999999998},"min":{"1min":0.307,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.0980000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.83599999999999997}]},{"osd":2,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.71399999999999997,"15min":0.72799999999999998},"min":{"1min":0.374,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.497},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.81200000000000006,"15min":0.81999999999999995},"min":{"1min":0.39000000000000001,"5min":0.308,"15min":0.308},"max":{"1min":1.1899999999999999,"5min":1.405,"15min":1.405},"last":0.78100000000000003}]},{"osd":4,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.92700000000000005,"15min":0.92800000000000005},"min":{"1min":0.65900000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.085,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":1.022},{"interface":"front","average":{"1min":0.98099999999999998,"5min":0.97599999999999998,"15min":0.99099999999999999},"min":{"1min":0.67800000000000005,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.236,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":1}]},{"osd":5,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":1.004,"15min":1.02},"min":{"1min":0.70699999999999996,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":1.137},{"interface":"front","average":{"1min":1.0880000000000001,"5min":1.0580000000000001,"15min":1.0740000000000001},"min":{"1min":0.74099999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.4330000000000001,"5min":2.21,"15min":2.21},"last":1.044}]},{"osd":6,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":1.0229999999999999,"15min":1.0249999999999999},"min":{"1min":0.76900000000000002,"5min":0.439,"15min":0.439},"max":{"1min":1.3799999999999999,"5min":1.395,"15min":1.395},"last":1.204},{"interface":"front","average":{"1min":1.0880000000000001,"5min":1.0600000000000001,"15min":1.0680000000000001},"min":{"1min":0.71699999999999997,"5min":0.375,"15min":0.375},"max":{"1min":1.4319999999999999,"5min":1.554,"15min":1.554},"last":1.069}]},{"osd":7,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.115,"15min":1.127},"min":{"1min":0.76300000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.5549999999999999,"5min":2.2829999999999999,"15min":2.2829999999999999},"last":1.1719999999999999},{"interface":"front","average":{"1min":1.171,"5min":1.1080000000000001,"15min":1.107},"min":{"1min":0.92200000000000004,"5min":0.497,"15min":0.497},"max":{"1min":1.504,"5min":1.504,"15min":1.504},"last":0.96399999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574889,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31812,"kb_used_data":764,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340028,"statfs":{"total":96636764160,"available":96604188672,"internally_reserved":0,"allocated":782336,"data_stored":346250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:11:20 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.53900000000000003,"15min":0.51200000000000001},"min":{"1min":0.48199999999999998,"5min":0.252,"15min":0.252},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.59099999999999997,"15min":0.55600000000000005},"min":{"1min":0.497,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":0.502}]},{"osd":1,"last update":"Mon Nov 6 02:11:20 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.60599999999999998,"15min":0.57499999999999996},"min":{"1min":0.44700000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.48,"5min":1.48,"15min":1.48},"last":0.623},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.64700000000000002,"15min":0.59399999999999997},"min":{"1min":0.45200000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.7230000000000001,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":0.75700000000000001}]},{"osd":3,"last update":"Mon Nov 6 02:11:20 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.65400000000000003,"15min":0.61599999999999999},"min":{"1min":0.58699999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.93200000000000005,"5min":1.155,"15min":1.155},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.69799999999999995,"15min":0.66000000000000003},"min":{"1min":0.52500000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.0960000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.58399999999999996}]},{"osd":4,"last update":"Mon Nov 6 02:11:20 2023","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":0.85799999999999998,"15min":0.81799999999999995},"min":{"1min":0.77500000000000002,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.89800000000000002},{"interface":"front","average":{"1min":1.0369999999999999,"5min":0.93600000000000005,"15min":0.92200000000000004},"min":{"1min":0.755,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.395,"5min":1.395,"15min":1.395},"last":0.95099999999999996}]},{"osd":5,"last update":"Mon Nov 6 02:11:20 2023","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":0.96199999999999997,"15min":0.94899999999999995},"min":{"1min":0.74099999999999999,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.7729999999999999,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":1.0189999999999999,"5min":0.95699999999999996,"15min":0.94099999999999995},"min":{"1min":0.67200000000000004,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.615,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.97099999999999997}]},{"osd":6,"last update":"Mon Nov 6 02:11:20 2023","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":0.96899999999999997,"15min":0.94799999999999995},"min":{"1min":0.79200000000000004,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":0.91700000000000004},{"interface":"front","average":{"1min":1.087,"5min":1.0129999999999999,"15min":1.002},"min":{"1min":0.81499999999999995,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.675,"5min":1.677,"15min":1.677},"last":1.0369999999999999}]},{"osd":7,"last update":"Mon Nov 6 02:11:20 2023","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":1.0089999999999999,"15min":0.98599999999999999},"min":{"1min":0.72499999999999998,"5min":0.622,"15min":0.622},"max":{"1min":1.659,"5min":1.659,"15min":1.659},"last":0.98699999999999999},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.0620000000000001,"15min":1.048},"min":{"1min":0.89300000000000002,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.4650000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":1.1160000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574889,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31764,"kb_used_data":716,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340076,"statfs":{"total":96636764160,"available":96604237824,"internally_reserved":0,"allocated":733184,"data_stored":319679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:11:22 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.63700000000000001,"15min":0.629},"min":{"1min":0.28699999999999998,"5min":0.224,"15min":0.224},"max":{"1min":1.603,"5min":1.603,"15min":1.603},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.67400000000000004,"15min":0.65900000000000003},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":1.0760000000000001}]},{"osd":2,"last update":"Mon Nov 6 02:11:22 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.63700000000000001,"15min":0.627},"min":{"1min":0.33600000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.649,"5min":1.649,"15min":1.649},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.755,"15min":0.748},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.1919999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":1.1539999999999999}]},{"osd":3,"last update":"Mon Nov 6 02:11:22 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.79500000000000004,"15min":0.80000000000000004},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.2350000000000001,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":1.0449999999999999},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.80800000000000005,"15min":0.81000000000000005},"min":{"1min":0.41299999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.5,"5min":1.8149999999999999,"15min":1.8149999999999999},"last":0.86299999999999999}]},{"osd":4,"last update":"Mon Nov 6 02:11:22 2023","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.97699999999999998,"15min":0.98699999999999999},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.5780000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":1.26},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.98099999999999998,"15min":0.98899999999999999},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.4099999999999999,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.81999999999999995}]},{"osd":5,"last update":"Mon Nov 6 02:11:22 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":0.97499999999999998,"15min":0.96899999999999997},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.718,"5min":1.718,"15min":1.718},"last":1.0169999999999999},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.069,"15min":1.075},"min":{"1min":0.66500000000000004,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.403,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":1.2110000000000001}]},{"osd":6,"last update":"Mon Nov 6 02:11:22 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.0920000000000001,"15min":1.099},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.6859999999999999,"5min":1.742,"15min":1.742},"last":1.1850000000000001},{"interface":"front","average":{"1min":1.0700000000000001,"5min":1.0669999999999999,"15min":1.073},"min":{"1min":0.64400000000000002,"5min":0.629,"15min":0.629},"max":{"1min":1.6339999999999999,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":1.129}]},{"osd":7,"last update":"Mon Nov 6 02:11:22 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.115,"15min":1.119},"min":{"1min":0.71499999999999997,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.534,"5min":1.534,"15min":1.534},"last":0.97199999999999998},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.1499999999999999,"15min":1.157},"min":{"1min":0.68700000000000006,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.7629999999999999,"5min":1.893,"15min":1.893},"last":1.3400000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574889,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32340,"kb_used_data":1292,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339500,"statfs":{"total":96636764160,"available":96603648000,"internally_reserved":0,"allocated":1323008,"data_stored":909028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 02:11:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.33700000000000002,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64000000000000001,"15min":0.63700000000000001},"min":{"1min":0.38700000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.93400000000000005,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.49299999999999999}]},{"osd":2,"last update":"Mon Nov 6 02:11:25 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.67100000000000004,"15min":0.66700000000000004},"min":{"1min":0.505,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.67800000000000005,"15min":0.67600000000000005},"min":{"1min":0.35199999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.95599999999999996,"5min":1.141,"15min":1.141},"last":0.64500000000000002}]},{"osd":3,"last update":"Mon Nov 6 02:11:25 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.64000000000000001,"15min":0.63200000000000001},"min":{"1min":0.376,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.0600000000000001,"5min":1.429,"15min":1.429},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.753,"15min":0.73799999999999999},"min":{"1min":0.44,"5min":0.317,"15min":0.317},"max":{"1min":1.1299999999999999,"5min":1.25,"15min":1.25},"last":0.96099999999999997}]},{"osd":4,"last update":"Mon Nov 6 02:11:25 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.85599999999999998,"15min":0.83599999999999997},"min":{"1min":0.59499999999999997,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.1579999999999999,"5min":1.341,"15min":1.341},"last":1.0189999999999999},{"interface":"front","average":{"1min":0.995,"5min":0.94299999999999995,"15min":0.93899999999999995},"min":{"1min":0.65100000000000002,"5min":0.5,"15min":0.5},"max":{"1min":1.3600000000000001,"5min":1.387,"15min":1.387},"last":1.0449999999999999}]},{"osd":5,"last update":"Mon Nov 6 02:11:25 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.94199999999999995,"15min":0.94599999999999995},"min":{"1min":0.63300000000000001,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.119,"5min":1.339,"15min":1.339},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.89700000000000002,"15min":0.88},"min":{"1min":0.72899999999999998,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.325,"5min":1.448,"15min":1.448},"last":1.0800000000000001}]},{"osd":6,"last update":"Mon Nov 6 02:11:25 2023","interfaces":[{"interface":"back","average":{"1min":1.0960000000000001,"5min":1.0409999999999999,"15min":1.038},"min":{"1min":0.66700000000000004,"5min":0.625,"15min":0.625},"max":{"1min":1.573,"5min":1.589,"15min":1.589},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.0309999999999999,"5min":1.0169999999999999,"15min":1.0049999999999999},"min":{"1min":0.67600000000000005,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.3899999999999999,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.95099999999999996}]},{"osd":7,"last update":"Mon Nov 6 02:11:25 2023","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":1.008,"15min":1.0149999999999999},"min":{"1min":0.71299999999999997,"5min":0.504,"15min":0.504},"max":{"1min":1.341,"5min":1.421,"15min":1.421},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.98599999999999999,"5min":0.98899999999999999,"15min":1.002},"min":{"1min":0.69899999999999995,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.3029999999999999,"5min":1.556,"15min":1.556},"last":0.81499999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T02:11:34.961 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3f is not active+clean 2023-11-06T02:11:34.962 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:29.703192+0000', 'last_change': '2023-11-06T02:11:29.703192+0000', 'last_active': '2023-11-06T02:11:29.703192+0000', 'last_peered': '2023-11-06T02:11:29.703192+0000', 'last_clean': '2023-11-06T02:11:29.703192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:29.703192+0000', 'last_undegraded': '2023-11-06T02:11:29.703192+0000', 'last_fullsized': '2023-11-06T02:11:29.703192+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:34.962 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3e is not active+clean 2023-11-06T02:11:34.963 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:29.703192+0000', 'last_change': '2023-11-06T02:11:29.703192+0000', 'last_active': '2023-11-06T02:11:29.703192+0000', 'last_peered': '2023-11-06T02:11:29.703192+0000', 'last_clean': '2023-11-06T02:11:29.703192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:29.703192+0000', 'last_undegraded': '2023-11-06T02:11:29.703192+0000', 'last_fullsized': '2023-11-06T02:11:29.703192+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:34.963 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3d is not active+clean 2023-11-06T02:11:34.964 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:29.703192+0000', 'last_change': '2023-11-06T02:11:29.703192+0000', 'last_active': '2023-11-06T02:11:29.703192+0000', 'last_peered': '2023-11-06T02:11:29.703192+0000', 'last_clean': '2023-11-06T02:11:29.703192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:29.703192+0000', 'last_undegraded': '2023-11-06T02:11:29.703192+0000', 'last_fullsized': '2023-11-06T02:11:29.703192+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:34.964 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3c is not active+clean 2023-11-06T02:11:34.965 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:29.703192+0000', 'last_change': '2023-11-06T02:11:29.703192+0000', 'last_active': '2023-11-06T02:11:29.703192+0000', 'last_peered': '2023-11-06T02:11:29.703192+0000', 'last_clean': '2023-11-06T02:11:29.703192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:29.703192+0000', 'last_undegraded': '2023-11-06T02:11:29.703192+0000', 'last_fullsized': '2023-11-06T02:11:29.703192+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:34.965 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3b is not active+clean 2023-11-06T02:11:34.965 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:29.703192+0000', 'last_change': '2023-11-06T02:11:29.703192+0000', 'last_active': '2023-11-06T02:11:29.703192+0000', 'last_peered': '2023-11-06T02:11:29.703192+0000', 'last_clean': '2023-11-06T02:11:29.703192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:29.703192+0000', 'last_undegraded': '2023-11-06T02:11:29.703192+0000', 'last_fullsized': '2023-11-06T02:11:29.703192+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:34.966 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3a is not active+clean 2023-11-06T02:11:34.967 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:29.703192+0000', 'last_change': '2023-11-06T02:11:29.703192+0000', 'last_active': '2023-11-06T02:11:29.703192+0000', 'last_peered': '2023-11-06T02:11:29.703192+0000', 'last_clean': '2023-11-06T02:11:29.703192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:29.703192+0000', 'last_undegraded': '2023-11-06T02:11:29.703192+0000', 'last_fullsized': '2023-11-06T02:11:29.703192+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:34.967 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.38 is not active+clean 2023-11-06T02:11:34.968 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:29.703192+0000', 'last_change': '2023-11-06T02:11:29.703192+0000', 'last_active': '2023-11-06T02:11:29.703192+0000', 'last_peered': '2023-11-06T02:11:29.703192+0000', 'last_clean': '2023-11-06T02:11:29.703192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:29.703192+0000', 'last_undegraded': '2023-11-06T02:11:29.703192+0000', 'last_fullsized': '2023-11-06T02:11:29.703192+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:34.968 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.37 is not active+clean 2023-11-06T02:11:34.969 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:29.703192+0000', 'last_change': '2023-11-06T02:11:29.703192+0000', 'last_active': '2023-11-06T02:11:29.703192+0000', 'last_peered': '2023-11-06T02:11:29.703192+0000', 'last_clean': '2023-11-06T02:11:29.703192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:29.703192+0000', 'last_undegraded': '2023-11-06T02:11:29.703192+0000', 'last_fullsized': '2023-11-06T02:11:29.703192+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:34.970 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.36 is not active+clean 2023-11-06T02:11:34.970 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:29.703192+0000', 'last_change': '2023-11-06T02:11:29.703192+0000', 'last_active': '2023-11-06T02:11:29.703192+0000', 'last_peered': '2023-11-06T02:11:29.703192+0000', 'last_clean': '2023-11-06T02:11:29.703192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:29.703192+0000', 'last_undegraded': '2023-11-06T02:11:29.703192+0000', 'last_fullsized': '2023-11-06T02:11:29.703192+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:34.971 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.35 is not active+clean 2023-11-06T02:11:34.971 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:29.703192+0000', 'last_change': '2023-11-06T02:11:29.703192+0000', 'last_active': '2023-11-06T02:11:29.703192+0000', 'last_peered': '2023-11-06T02:11:29.703192+0000', 'last_clean': '2023-11-06T02:11:29.703192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:29.703192+0000', 'last_undegraded': '2023-11-06T02:11:29.703192+0000', 'last_fullsized': '2023-11-06T02:11:29.703192+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:34.971 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.34 is not active+clean 2023-11-06T02:11:34.972 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:29.703192+0000', 'last_change': '2023-11-06T02:11:29.703192+0000', 'last_active': '2023-11-06T02:11:29.703192+0000', 'last_peered': '2023-11-06T02:11:29.703192+0000', 'last_clean': '2023-11-06T02:11:29.703192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:29.703192+0000', 'last_undegraded': '2023-11-06T02:11:29.703192+0000', 'last_fullsized': '2023-11-06T02:11:29.703192+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:34.972 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.33 is not active+clean 2023-11-06T02:11:34.973 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:29.703192+0000', 'last_change': '2023-11-06T02:11:29.703192+0000', 'last_active': '2023-11-06T02:11:29.703192+0000', 'last_peered': '2023-11-06T02:11:29.703192+0000', 'last_clean': '2023-11-06T02:11:29.703192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:29.703192+0000', 'last_undegraded': '2023-11-06T02:11:29.703192+0000', 'last_fullsized': '2023-11-06T02:11:29.703192+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:34.973 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.32 is not active+clean 2023-11-06T02:11:34.974 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:29.703192+0000', 'last_change': '2023-11-06T02:11:29.703192+0000', 'last_active': '2023-11-06T02:11:29.703192+0000', 'last_peered': '2023-11-06T02:11:29.703192+0000', 'last_clean': '2023-11-06T02:11:29.703192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:29.703192+0000', 'last_undegraded': '2023-11-06T02:11:29.703192+0000', 'last_fullsized': '2023-11-06T02:11:29.703192+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:34.974 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.31 is not active+clean 2023-11-06T02:11:34.975 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:29.703192+0000', 'last_change': '2023-11-06T02:11:29.703192+0000', 'last_active': '2023-11-06T02:11:29.703192+0000', 'last_peered': '2023-11-06T02:11:29.703192+0000', 'last_clean': '2023-11-06T02:11:29.703192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:29.703192+0000', 'last_undegraded': '2023-11-06T02:11:29.703192+0000', 'last_fullsized': '2023-11-06T02:11:29.703192+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:34.975 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2f is not active+clean 2023-11-06T02:11:34.975 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:29.703192+0000', 'last_change': '2023-11-06T02:11:29.703192+0000', 'last_active': '2023-11-06T02:11:29.703192+0000', 'last_peered': '2023-11-06T02:11:29.703192+0000', 'last_clean': '2023-11-06T02:11:29.703192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:29.703192+0000', 'last_undegraded': '2023-11-06T02:11:29.703192+0000', 'last_fullsized': '2023-11-06T02:11:29.703192+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:34.976 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2e is not active+clean 2023-11-06T02:11:34.976 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:29.703192+0000', 'last_change': '2023-11-06T02:11:29.703192+0000', 'last_active': '2023-11-06T02:11:29.703192+0000', 'last_peered': '2023-11-06T02:11:29.703192+0000', 'last_clean': '2023-11-06T02:11:29.703192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:29.703192+0000', 'last_undegraded': '2023-11-06T02:11:29.703192+0000', 'last_fullsized': '2023-11-06T02:11:29.703192+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:34.977 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2d is not active+clean 2023-11-06T02:11:34.977 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:29.703192+0000', 'last_change': '2023-11-06T02:11:29.703192+0000', 'last_active': '2023-11-06T02:11:29.703192+0000', 'last_peered': '2023-11-06T02:11:29.703192+0000', 'last_clean': '2023-11-06T02:11:29.703192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:29.703192+0000', 'last_undegraded': '2023-11-06T02:11:29.703192+0000', 'last_fullsized': '2023-11-06T02:11:29.703192+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:34.978 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2c is not active+clean 2023-11-06T02:11:34.978 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:29.703192+0000', 'last_change': '2023-11-06T02:11:29.703192+0000', 'last_active': '2023-11-06T02:11:29.703192+0000', 'last_peered': '2023-11-06T02:11:29.703192+0000', 'last_clean': '2023-11-06T02:11:29.703192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:29.703192+0000', 'last_undegraded': '2023-11-06T02:11:29.703192+0000', 'last_fullsized': '2023-11-06T02:11:29.703192+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:34.979 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2b is not active+clean 2023-11-06T02:11:34.979 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:29.703192+0000', 'last_change': '2023-11-06T02:11:29.703192+0000', 'last_active': '2023-11-06T02:11:29.703192+0000', 'last_peered': '2023-11-06T02:11:29.703192+0000', 'last_clean': '2023-11-06T02:11:29.703192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:29.703192+0000', 'last_undegraded': '2023-11-06T02:11:29.703192+0000', 'last_fullsized': '2023-11-06T02:11:29.703192+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:34.980 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2a is not active+clean 2023-11-06T02:11:34.980 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:29.703192+0000', 'last_change': '2023-11-06T02:11:29.703192+0000', 'last_active': '2023-11-06T02:11:29.703192+0000', 'last_peered': '2023-11-06T02:11:29.703192+0000', 'last_clean': '2023-11-06T02:11:29.703192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:29.703192+0000', 'last_undegraded': '2023-11-06T02:11:29.703192+0000', 'last_fullsized': '2023-11-06T02:11:29.703192+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:34.981 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.29 is not active+clean 2023-11-06T02:11:34.981 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:29.703192+0000', 'last_change': '2023-11-06T02:11:29.703192+0000', 'last_active': '2023-11-06T02:11:29.703192+0000', 'last_peered': '2023-11-06T02:11:29.703192+0000', 'last_clean': '2023-11-06T02:11:29.703192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:29.703192+0000', 'last_undegraded': '2023-11-06T02:11:29.703192+0000', 'last_fullsized': '2023-11-06T02:11:29.703192+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:34.982 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.28 is not active+clean 2023-11-06T02:11:34.983 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:29.703192+0000', 'last_change': '2023-11-06T02:11:29.703192+0000', 'last_active': '2023-11-06T02:11:29.703192+0000', 'last_peered': '2023-11-06T02:11:29.703192+0000', 'last_clean': '2023-11-06T02:11:29.703192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:29.703192+0000', 'last_undegraded': '2023-11-06T02:11:29.703192+0000', 'last_fullsized': '2023-11-06T02:11:29.703192+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:34.983 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.27 is not active+clean 2023-11-06T02:11:34.984 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:29.703192+0000', 'last_change': '2023-11-06T02:11:29.703192+0000', 'last_active': '2023-11-06T02:11:29.703192+0000', 'last_peered': '2023-11-06T02:11:29.703192+0000', 'last_clean': '2023-11-06T02:11:29.703192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:29.703192+0000', 'last_undegraded': '2023-11-06T02:11:29.703192+0000', 'last_fullsized': '2023-11-06T02:11:29.703192+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:34.984 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.26 is not active+clean 2023-11-06T02:11:34.985 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:29.703192+0000', 'last_change': '2023-11-06T02:11:29.703192+0000', 'last_active': '2023-11-06T02:11:29.703192+0000', 'last_peered': '2023-11-06T02:11:29.703192+0000', 'last_clean': '2023-11-06T02:11:29.703192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:29.703192+0000', 'last_undegraded': '2023-11-06T02:11:29.703192+0000', 'last_fullsized': '2023-11-06T02:11:29.703192+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:34.986 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.25 is not active+clean 2023-11-06T02:11:34.986 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:29.703192+0000', 'last_change': '2023-11-06T02:11:29.703192+0000', 'last_active': '2023-11-06T02:11:29.703192+0000', 'last_peered': '2023-11-06T02:11:29.703192+0000', 'last_clean': '2023-11-06T02:11:29.703192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:29.703192+0000', 'last_undegraded': '2023-11-06T02:11:29.703192+0000', 'last_fullsized': '2023-11-06T02:11:29.703192+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:34.987 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.24 is not active+clean 2023-11-06T02:11:34.987 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:29.703192+0000', 'last_change': '2023-11-06T02:11:29.703192+0000', 'last_active': '2023-11-06T02:11:29.703192+0000', 'last_peered': '2023-11-06T02:11:29.703192+0000', 'last_clean': '2023-11-06T02:11:29.703192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:29.703192+0000', 'last_undegraded': '2023-11-06T02:11:29.703192+0000', 'last_fullsized': '2023-11-06T02:11:29.703192+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:34.988 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.23 is not active+clean 2023-11-06T02:11:34.989 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:29.703192+0000', 'last_change': '2023-11-06T02:11:29.703192+0000', 'last_active': '2023-11-06T02:11:29.703192+0000', 'last_peered': '2023-11-06T02:11:29.703192+0000', 'last_clean': '2023-11-06T02:11:29.703192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:29.703192+0000', 'last_undegraded': '2023-11-06T02:11:29.703192+0000', 'last_fullsized': '2023-11-06T02:11:29.703192+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:34.989 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.22 is not active+clean 2023-11-06T02:11:34.990 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:29.703192+0000', 'last_change': '2023-11-06T02:11:29.703192+0000', 'last_active': '2023-11-06T02:11:29.703192+0000', 'last_peered': '2023-11-06T02:11:29.703192+0000', 'last_clean': '2023-11-06T02:11:29.703192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:29.703192+0000', 'last_undegraded': '2023-11-06T02:11:29.703192+0000', 'last_fullsized': '2023-11-06T02:11:29.703192+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:34.990 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.21 is not active+clean 2023-11-06T02:11:34.991 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:29.703192+0000', 'last_change': '2023-11-06T02:11:29.703192+0000', 'last_active': '2023-11-06T02:11:29.703192+0000', 'last_peered': '2023-11-06T02:11:29.703192+0000', 'last_clean': '2023-11-06T02:11:29.703192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:29.703192+0000', 'last_undegraded': '2023-11-06T02:11:29.703192+0000', 'last_fullsized': '2023-11-06T02:11:29.703192+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:34.991 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.20 is not active+clean 2023-11-06T02:11:34.992 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:29.703192+0000', 'last_change': '2023-11-06T02:11:29.703192+0000', 'last_active': '2023-11-06T02:11:29.703192+0000', 'last_peered': '2023-11-06T02:11:29.703192+0000', 'last_clean': '2023-11-06T02:11:29.703192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:29.703192+0000', 'last_undegraded': '2023-11-06T02:11:29.703192+0000', 'last_fullsized': '2023-11-06T02:11:29.703192+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:34.992 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1c is not active+clean 2023-11-06T02:11:34.993 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:29.703192+0000', 'last_change': '2023-11-06T02:11:29.703192+0000', 'last_active': '2023-11-06T02:11:29.703192+0000', 'last_peered': '2023-11-06T02:11:29.703192+0000', 'last_clean': '2023-11-06T02:11:29.703192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:29.703192+0000', 'last_undegraded': '2023-11-06T02:11:29.703192+0000', 'last_fullsized': '2023-11-06T02:11:29.703192+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:34.994 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1f is not active+clean 2023-11-06T02:11:34.994 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:28.698549+0000', 'last_change': '2023-11-06T02:11:28.698549+0000', 'last_active': '2023-11-06T02:11:28.698549+0000', 'last_peered': '2023-11-06T02:11:28.698549+0000', 'last_clean': '2023-11-06T02:11:28.698549+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:28.698549+0000', 'last_undegraded': '2023-11-06T02:11:28.698549+0000', 'last_fullsized': '2023-11-06T02:11:28.698549+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:34.995 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1d is not active+clean 2023-11-06T02:11:34.995 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:29.703192+0000', 'last_change': '2023-11-06T02:11:29.703192+0000', 'last_active': '2023-11-06T02:11:29.703192+0000', 'last_peered': '2023-11-06T02:11:29.703192+0000', 'last_clean': '2023-11-06T02:11:29.703192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:29.703192+0000', 'last_undegraded': '2023-11-06T02:11:29.703192+0000', 'last_fullsized': '2023-11-06T02:11:29.703192+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:34.996 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1e is not active+clean 2023-11-06T02:11:34.997 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:28.698549+0000', 'last_change': '2023-11-06T02:11:28.698549+0000', 'last_active': '2023-11-06T02:11:28.698549+0000', 'last_peered': '2023-11-06T02:11:28.698549+0000', 'last_clean': '2023-11-06T02:11:28.698549+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:28.698549+0000', 'last_undegraded': '2023-11-06T02:11:28.698549+0000', 'last_fullsized': '2023-11-06T02:11:28.698549+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:34.997 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1e is not active+clean 2023-11-06T02:11:34.998 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:29.703192+0000', 'last_change': '2023-11-06T02:11:29.703192+0000', 'last_active': '2023-11-06T02:11:29.703192+0000', 'last_peered': '2023-11-06T02:11:29.703192+0000', 'last_clean': '2023-11-06T02:11:29.703192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:29.703192+0000', 'last_undegraded': '2023-11-06T02:11:29.703192+0000', 'last_fullsized': '2023-11-06T02:11:29.703192+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:34.998 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1d is not active+clean 2023-11-06T02:11:34.999 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:28.698549+0000', 'last_change': '2023-11-06T02:11:28.698549+0000', 'last_active': '2023-11-06T02:11:28.698549+0000', 'last_peered': '2023-11-06T02:11:28.698549+0000', 'last_clean': '2023-11-06T02:11:28.698549+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:28.698549+0000', 'last_undegraded': '2023-11-06T02:11:28.698549+0000', 'last_fullsized': '2023-11-06T02:11:28.698549+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:35.000 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1f is not active+clean 2023-11-06T02:11:35.000 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:29.703192+0000', 'last_change': '2023-11-06T02:11:29.703192+0000', 'last_active': '2023-11-06T02:11:29.703192+0000', 'last_peered': '2023-11-06T02:11:29.703192+0000', 'last_clean': '2023-11-06T02:11:29.703192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:29.703192+0000', 'last_undegraded': '2023-11-06T02:11:29.703192+0000', 'last_fullsized': '2023-11-06T02:11:29.703192+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:35.001 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1c is not active+clean 2023-11-06T02:11:35.001 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:28.698549+0000', 'last_change': '2023-11-06T02:11:28.698549+0000', 'last_active': '2023-11-06T02:11:28.698549+0000', 'last_peered': '2023-11-06T02:11:28.698549+0000', 'last_clean': '2023-11-06T02:11:28.698549+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:28.698549+0000', 'last_undegraded': '2023-11-06T02:11:28.698549+0000', 'last_fullsized': '2023-11-06T02:11:28.698549+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:35.002 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1b is not active+clean 2023-11-06T02:11:35.002 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:28.698549+0000', 'last_change': '2023-11-06T02:11:28.698549+0000', 'last_active': '2023-11-06T02:11:28.698549+0000', 'last_peered': '2023-11-06T02:11:28.698549+0000', 'last_clean': '2023-11-06T02:11:28.698549+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:28.698549+0000', 'last_undegraded': '2023-11-06T02:11:28.698549+0000', 'last_fullsized': '2023-11-06T02:11:28.698549+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:35.003 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.19 is not active+clean 2023-11-06T02:11:35.004 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:29.703192+0000', 'last_change': '2023-11-06T02:11:29.703192+0000', 'last_active': '2023-11-06T02:11:29.703192+0000', 'last_peered': '2023-11-06T02:11:29.703192+0000', 'last_clean': '2023-11-06T02:11:29.703192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:29.703192+0000', 'last_undegraded': '2023-11-06T02:11:29.703192+0000', 'last_fullsized': '2023-11-06T02:11:29.703192+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:35.005 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1a is not active+clean 2023-11-06T02:11:35.005 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:28.698549+0000', 'last_change': '2023-11-06T02:11:28.698549+0000', 'last_active': '2023-11-06T02:11:28.698549+0000', 'last_peered': '2023-11-06T02:11:28.698549+0000', 'last_clean': '2023-11-06T02:11:28.698549+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:28.698549+0000', 'last_undegraded': '2023-11-06T02:11:28.698549+0000', 'last_fullsized': '2023-11-06T02:11:28.698549+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:35.005 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1a is not active+clean 2023-11-06T02:11:35.006 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:29.703192+0000', 'last_change': '2023-11-06T02:11:29.703192+0000', 'last_active': '2023-11-06T02:11:29.703192+0000', 'last_peered': '2023-11-06T02:11:29.703192+0000', 'last_clean': '2023-11-06T02:11:29.703192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:29.703192+0000', 'last_undegraded': '2023-11-06T02:11:29.703192+0000', 'last_fullsized': '2023-11-06T02:11:29.703192+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:35.006 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.19 is not active+clean 2023-11-06T02:11:35.007 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:28.698549+0000', 'last_change': '2023-11-06T02:11:28.698549+0000', 'last_active': '2023-11-06T02:11:28.698549+0000', 'last_peered': '2023-11-06T02:11:28.698549+0000', 'last_clean': '2023-11-06T02:11:28.698549+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:28.698549+0000', 'last_undegraded': '2023-11-06T02:11:28.698549+0000', 'last_fullsized': '2023-11-06T02:11:28.698549+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:35.007 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1b is not active+clean 2023-11-06T02:11:35.008 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:29.703192+0000', 'last_change': '2023-11-06T02:11:29.703192+0000', 'last_active': '2023-11-06T02:11:29.703192+0000', 'last_peered': '2023-11-06T02:11:29.703192+0000', 'last_clean': '2023-11-06T02:11:29.703192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:29.703192+0000', 'last_undegraded': '2023-11-06T02:11:29.703192+0000', 'last_fullsized': '2023-11-06T02:11:29.703192+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:35.009 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.18 is not active+clean 2023-11-06T02:11:35.009 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:28.698549+0000', 'last_change': '2023-11-06T02:11:28.698549+0000', 'last_active': '2023-11-06T02:11:28.698549+0000', 'last_peered': '2023-11-06T02:11:28.698549+0000', 'last_clean': '2023-11-06T02:11:28.698549+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:28.698549+0000', 'last_undegraded': '2023-11-06T02:11:28.698549+0000', 'last_fullsized': '2023-11-06T02:11:28.698549+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:35.010 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.14 is not active+clean 2023-11-06T02:11:35.011 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:29.703192+0000', 'last_change': '2023-11-06T02:11:29.703192+0000', 'last_active': '2023-11-06T02:11:29.703192+0000', 'last_peered': '2023-11-06T02:11:29.703192+0000', 'last_clean': '2023-11-06T02:11:29.703192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:29.703192+0000', 'last_undegraded': '2023-11-06T02:11:29.703192+0000', 'last_fullsized': '2023-11-06T02:11:29.703192+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:35.011 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.17 is not active+clean 2023-11-06T02:11:35.012 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:28.698549+0000', 'last_change': '2023-11-06T02:11:28.698549+0000', 'last_active': '2023-11-06T02:11:28.698549+0000', 'last_peered': '2023-11-06T02:11:28.698549+0000', 'last_clean': '2023-11-06T02:11:28.698549+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:28.698549+0000', 'last_undegraded': '2023-11-06T02:11:28.698549+0000', 'last_fullsized': '2023-11-06T02:11:28.698549+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:35.013 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.15 is not active+clean 2023-11-06T02:11:35.013 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:29.703192+0000', 'last_change': '2023-11-06T02:11:29.703192+0000', 'last_active': '2023-11-06T02:11:29.703192+0000', 'last_peered': '2023-11-06T02:11:29.703192+0000', 'last_clean': '2023-11-06T02:11:29.703192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:29.703192+0000', 'last_undegraded': '2023-11-06T02:11:29.703192+0000', 'last_fullsized': '2023-11-06T02:11:29.703192+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:35.014 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.16 is not active+clean 2023-11-06T02:11:35.015 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:28.698549+0000', 'last_change': '2023-11-06T02:11:28.698549+0000', 'last_active': '2023-11-06T02:11:28.698549+0000', 'last_peered': '2023-11-06T02:11:28.698549+0000', 'last_clean': '2023-11-06T02:11:28.698549+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:28.698549+0000', 'last_undegraded': '2023-11-06T02:11:28.698549+0000', 'last_fullsized': '2023-11-06T02:11:28.698549+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:35.015 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.16 is not active+clean 2023-11-06T02:11:35.016 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:29.703192+0000', 'last_change': '2023-11-06T02:11:29.703192+0000', 'last_active': '2023-11-06T02:11:29.703192+0000', 'last_peered': '2023-11-06T02:11:29.703192+0000', 'last_clean': '2023-11-06T02:11:29.703192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:29.703192+0000', 'last_undegraded': '2023-11-06T02:11:29.703192+0000', 'last_fullsized': '2023-11-06T02:11:29.703192+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:35.017 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.15 is not active+clean 2023-11-06T02:11:35.017 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:28.698549+0000', 'last_change': '2023-11-06T02:11:28.698549+0000', 'last_active': '2023-11-06T02:11:28.698549+0000', 'last_peered': '2023-11-06T02:11:28.698549+0000', 'last_clean': '2023-11-06T02:11:28.698549+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:28.698549+0000', 'last_undegraded': '2023-11-06T02:11:28.698549+0000', 'last_fullsized': '2023-11-06T02:11:28.698549+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:35.018 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.17 is not active+clean 2023-11-06T02:11:35.018 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:29.703192+0000', 'last_change': '2023-11-06T02:11:29.703192+0000', 'last_active': '2023-11-06T02:11:29.703192+0000', 'last_peered': '2023-11-06T02:11:29.703192+0000', 'last_clean': '2023-11-06T02:11:29.703192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:29.703192+0000', 'last_undegraded': '2023-11-06T02:11:29.703192+0000', 'last_fullsized': '2023-11-06T02:11:29.703192+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:35.019 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.14 is not active+clean 2023-11-06T02:11:35.019 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:28.698549+0000', 'last_change': '2023-11-06T02:11:28.698549+0000', 'last_active': '2023-11-06T02:11:28.698549+0000', 'last_peered': '2023-11-06T02:11:28.698549+0000', 'last_clean': '2023-11-06T02:11:28.698549+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:28.698549+0000', 'last_undegraded': '2023-11-06T02:11:28.698549+0000', 'last_fullsized': '2023-11-06T02:11:28.698549+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:35.020 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.10 is not active+clean 2023-11-06T02:11:35.020 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:29.703192+0000', 'last_change': '2023-11-06T02:11:29.703192+0000', 'last_active': '2023-11-06T02:11:29.703192+0000', 'last_peered': '2023-11-06T02:11:29.703192+0000', 'last_clean': '2023-11-06T02:11:29.703192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:29.703192+0000', 'last_undegraded': '2023-11-06T02:11:29.703192+0000', 'last_fullsized': '2023-11-06T02:11:29.703192+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:35.021 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.13 is not active+clean 2023-11-06T02:11:35.022 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:28.698549+0000', 'last_change': '2023-11-06T02:11:28.698549+0000', 'last_active': '2023-11-06T02:11:28.698549+0000', 'last_peered': '2023-11-06T02:11:28.698549+0000', 'last_clean': '2023-11-06T02:11:28.698549+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:28.698549+0000', 'last_undegraded': '2023-11-06T02:11:28.698549+0000', 'last_fullsized': '2023-11-06T02:11:28.698549+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:35.022 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.11 is not active+clean 2023-11-06T02:11:35.023 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:29.703192+0000', 'last_change': '2023-11-06T02:11:29.703192+0000', 'last_active': '2023-11-06T02:11:29.703192+0000', 'last_peered': '2023-11-06T02:11:29.703192+0000', 'last_clean': '2023-11-06T02:11:29.703192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:29.703192+0000', 'last_undegraded': '2023-11-06T02:11:29.703192+0000', 'last_fullsized': '2023-11-06T02:11:29.703192+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:35.024 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.12 is not active+clean 2023-11-06T02:11:35.025 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:28.698549+0000', 'last_change': '2023-11-06T02:11:28.698549+0000', 'last_active': '2023-11-06T02:11:28.698549+0000', 'last_peered': '2023-11-06T02:11:28.698549+0000', 'last_clean': '2023-11-06T02:11:28.698549+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:28.698549+0000', 'last_undegraded': '2023-11-06T02:11:28.698549+0000', 'last_fullsized': '2023-11-06T02:11:28.698549+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:35.026 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.12 is not active+clean 2023-11-06T02:11:35.026 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:29.703192+0000', 'last_change': '2023-11-06T02:11:29.703192+0000', 'last_active': '2023-11-06T02:11:29.703192+0000', 'last_peered': '2023-11-06T02:11:29.703192+0000', 'last_clean': '2023-11-06T02:11:29.703192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:29.703192+0000', 'last_undegraded': '2023-11-06T02:11:29.703192+0000', 'last_fullsized': '2023-11-06T02:11:29.703192+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:35.027 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.11 is not active+clean 2023-11-06T02:11:35.027 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:28.698549+0000', 'last_change': '2023-11-06T02:11:28.698549+0000', 'last_active': '2023-11-06T02:11:28.698549+0000', 'last_peered': '2023-11-06T02:11:28.698549+0000', 'last_clean': '2023-11-06T02:11:28.698549+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:28.698549+0000', 'last_undegraded': '2023-11-06T02:11:28.698549+0000', 'last_fullsized': '2023-11-06T02:11:28.698549+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:35.028 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.13 is not active+clean 2023-11-06T02:11:35.028 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:29.703192+0000', 'last_change': '2023-11-06T02:11:29.703192+0000', 'last_active': '2023-11-06T02:11:29.703192+0000', 'last_peered': '2023-11-06T02:11:29.703192+0000', 'last_clean': '2023-11-06T02:11:29.703192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:29.703192+0000', 'last_undegraded': '2023-11-06T02:11:29.703192+0000', 'last_fullsized': '2023-11-06T02:11:29.703192+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:35.029 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.10 is not active+clean 2023-11-06T02:11:35.029 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:28.698549+0000', 'last_change': '2023-11-06T02:11:28.698549+0000', 'last_active': '2023-11-06T02:11:28.698549+0000', 'last_peered': '2023-11-06T02:11:28.698549+0000', 'last_clean': '2023-11-06T02:11:28.698549+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:28.698549+0000', 'last_undegraded': '2023-11-06T02:11:28.698549+0000', 'last_fullsized': '2023-11-06T02:11:28.698549+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:35.030 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.4 is not active+clean 2023-11-06T02:11:35.030 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:29.703192+0000', 'last_change': '2023-11-06T02:11:29.703192+0000', 'last_active': '2023-11-06T02:11:29.703192+0000', 'last_peered': '2023-11-06T02:11:29.703192+0000', 'last_clean': '2023-11-06T02:11:29.703192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:29.703192+0000', 'last_undegraded': '2023-11-06T02:11:29.703192+0000', 'last_fullsized': '2023-11-06T02:11:29.703192+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:35.031 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.7 is not active+clean 2023-11-06T02:11:35.031 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:28.698549+0000', 'last_change': '2023-11-06T02:11:28.698549+0000', 'last_active': '2023-11-06T02:11:28.698549+0000', 'last_peered': '2023-11-06T02:11:28.698549+0000', 'last_clean': '2023-11-06T02:11:28.698549+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:28.698549+0000', 'last_undegraded': '2023-11-06T02:11:28.698549+0000', 'last_fullsized': '2023-11-06T02:11:28.698549+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:35.032 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.6 is not active+clean 2023-11-06T02:11:35.032 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:28.698549+0000', 'last_change': '2023-11-06T02:11:28.698549+0000', 'last_active': '2023-11-06T02:11:28.698549+0000', 'last_peered': '2023-11-06T02:11:28.698549+0000', 'last_clean': '2023-11-06T02:11:28.698549+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:28.698549+0000', 'last_undegraded': '2023-11-06T02:11:28.698549+0000', 'last_fullsized': '2023-11-06T02:11:28.698549+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:35.033 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.6 is not active+clean 2023-11-06T02:11:35.034 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:29.703192+0000', 'last_change': '2023-11-06T02:11:29.703192+0000', 'last_active': '2023-11-06T02:11:29.703192+0000', 'last_peered': '2023-11-06T02:11:29.703192+0000', 'last_clean': '2023-11-06T02:11:29.703192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:29.703192+0000', 'last_undegraded': '2023-11-06T02:11:29.703192+0000', 'last_fullsized': '2023-11-06T02:11:29.703192+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:35.035 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.5 is not active+clean 2023-11-06T02:11:35.036 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:28.698549+0000', 'last_change': '2023-11-06T02:11:28.698549+0000', 'last_active': '2023-11-06T02:11:28.698549+0000', 'last_peered': '2023-11-06T02:11:28.698549+0000', 'last_clean': '2023-11-06T02:11:28.698549+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:28.698549+0000', 'last_undegraded': '2023-11-06T02:11:28.698549+0000', 'last_fullsized': '2023-11-06T02:11:28.698549+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:35.036 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.7 is not active+clean 2023-11-06T02:11:35.037 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:29.703192+0000', 'last_change': '2023-11-06T02:11:29.703192+0000', 'last_active': '2023-11-06T02:11:29.703192+0000', 'last_peered': '2023-11-06T02:11:29.703192+0000', 'last_clean': '2023-11-06T02:11:29.703192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:29.703192+0000', 'last_undegraded': '2023-11-06T02:11:29.703192+0000', 'last_fullsized': '2023-11-06T02:11:29.703192+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:35.038 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.4 is not active+clean 2023-11-06T02:11:35.038 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:28.698549+0000', 'last_change': '2023-11-06T02:11:28.698549+0000', 'last_active': '2023-11-06T02:11:28.698549+0000', 'last_peered': '2023-11-06T02:11:28.698549+0000', 'last_clean': '2023-11-06T02:11:28.698549+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:28.698549+0000', 'last_undegraded': '2023-11-06T02:11:28.698549+0000', 'last_fullsized': '2023-11-06T02:11:28.698549+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:35.039 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.0 is not active+clean 2023-11-06T02:11:35.040 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:29.703192+0000', 'last_change': '2023-11-06T02:11:29.703192+0000', 'last_active': '2023-11-06T02:11:29.703192+0000', 'last_peered': '2023-11-06T02:11:29.703192+0000', 'last_clean': '2023-11-06T02:11:29.703192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:29.703192+0000', 'last_undegraded': '2023-11-06T02:11:29.703192+0000', 'last_fullsized': '2023-11-06T02:11:29.703192+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:35.041 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3 is not active+clean 2023-11-06T02:11:35.042 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:28.698549+0000', 'last_change': '2023-11-06T02:11:28.698549+0000', 'last_active': '2023-11-06T02:11:28.698549+0000', 'last_peered': '2023-11-06T02:11:28.698549+0000', 'last_clean': '2023-11-06T02:11:28.698549+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:28.698549+0000', 'last_undegraded': '2023-11-06T02:11:28.698549+0000', 'last_fullsized': '2023-11-06T02:11:28.698549+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:35.042 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1 is not active+clean 2023-11-06T02:11:35.043 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:29.703192+0000', 'last_change': '2023-11-06T02:11:29.703192+0000', 'last_active': '2023-11-06T02:11:29.703192+0000', 'last_peered': '2023-11-06T02:11:29.703192+0000', 'last_clean': '2023-11-06T02:11:29.703192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:29.703192+0000', 'last_undegraded': '2023-11-06T02:11:29.703192+0000', 'last_fullsized': '2023-11-06T02:11:29.703192+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:35.043 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2 is not active+clean 2023-11-06T02:11:35.043 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:28.698549+0000', 'last_change': '2023-11-06T02:11:28.698549+0000', 'last_active': '2023-11-06T02:11:28.698549+0000', 'last_peered': '2023-11-06T02:11:28.698549+0000', 'last_clean': '2023-11-06T02:11:28.698549+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:28.698549+0000', 'last_undegraded': '2023-11-06T02:11:28.698549+0000', 'last_fullsized': '2023-11-06T02:11:28.698549+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:35.044 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2 is not active+clean 2023-11-06T02:11:35.044 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:29.703192+0000', 'last_change': '2023-11-06T02:11:29.703192+0000', 'last_active': '2023-11-06T02:11:29.703192+0000', 'last_peered': '2023-11-06T02:11:29.703192+0000', 'last_clean': '2023-11-06T02:11:29.703192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:29.703192+0000', 'last_undegraded': '2023-11-06T02:11:29.703192+0000', 'last_fullsized': '2023-11-06T02:11:29.703192+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:35.045 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1 is not active+clean 2023-11-06T02:11:35.045 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:28.698549+0000', 'last_change': '2023-11-06T02:11:28.698549+0000', 'last_active': '2023-11-06T02:11:28.698549+0000', 'last_peered': '2023-11-06T02:11:28.698549+0000', 'last_clean': '2023-11-06T02:11:28.698549+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:28.698549+0000', 'last_undegraded': '2023-11-06T02:11:28.698549+0000', 'last_fullsized': '2023-11-06T02:11:28.698549+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:35.045 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.0 is not active+clean 2023-11-06T02:11:35.046 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:28.698549+0000', 'last_change': '2023-11-06T02:11:28.698549+0000', 'last_active': '2023-11-06T02:11:28.698549+0000', 'last_peered': '2023-11-06T02:11:28.698549+0000', 'last_clean': '2023-11-06T02:11:28.698549+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:28.698549+0000', 'last_undegraded': '2023-11-06T02:11:28.698549+0000', 'last_fullsized': '2023-11-06T02:11:28.698549+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:35.046 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.b is not active+clean 2023-11-06T02:11:35.047 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:29.703192+0000', 'last_change': '2023-11-06T02:11:29.703192+0000', 'last_active': '2023-11-06T02:11:29.703192+0000', 'last_peered': '2023-11-06T02:11:29.703192+0000', 'last_clean': '2023-11-06T02:11:29.703192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:29.703192+0000', 'last_undegraded': '2023-11-06T02:11:29.703192+0000', 'last_fullsized': '2023-11-06T02:11:29.703192+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:35.047 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.8 is not active+clean 2023-11-06T02:11:35.048 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:28.698549+0000', 'last_change': '2023-11-06T02:11:28.698549+0000', 'last_active': '2023-11-06T02:11:28.698549+0000', 'last_peered': '2023-11-06T02:11:28.698549+0000', 'last_clean': '2023-11-06T02:11:28.698549+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:28.698549+0000', 'last_undegraded': '2023-11-06T02:11:28.698549+0000', 'last_fullsized': '2023-11-06T02:11:28.698549+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:35.048 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.8 is not active+clean 2023-11-06T02:11:35.048 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:29.703192+0000', 'last_change': '2023-11-06T02:11:29.703192+0000', 'last_active': '2023-11-06T02:11:29.703192+0000', 'last_peered': '2023-11-06T02:11:29.703192+0000', 'last_clean': '2023-11-06T02:11:29.703192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:29.703192+0000', 'last_undegraded': '2023-11-06T02:11:29.703192+0000', 'last_fullsized': '2023-11-06T02:11:29.703192+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:35.049 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.b is not active+clean 2023-11-06T02:11:35.049 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:28.698549+0000', 'last_change': '2023-11-06T02:11:28.698549+0000', 'last_active': '2023-11-06T02:11:28.698549+0000', 'last_peered': '2023-11-06T02:11:28.698549+0000', 'last_clean': '2023-11-06T02:11:28.698549+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:28.698549+0000', 'last_undegraded': '2023-11-06T02:11:28.698549+0000', 'last_fullsized': '2023-11-06T02:11:28.698549+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:35.050 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.9 is not active+clean 2023-11-06T02:11:35.051 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:29.703192+0000', 'last_change': '2023-11-06T02:11:29.703192+0000', 'last_active': '2023-11-06T02:11:29.703192+0000', 'last_peered': '2023-11-06T02:11:29.703192+0000', 'last_clean': '2023-11-06T02:11:29.703192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:29.703192+0000', 'last_undegraded': '2023-11-06T02:11:29.703192+0000', 'last_fullsized': '2023-11-06T02:11:29.703192+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:35.051 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.a is not active+clean 2023-11-06T02:11:35.052 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:28.698549+0000', 'last_change': '2023-11-06T02:11:28.698549+0000', 'last_active': '2023-11-06T02:11:28.698549+0000', 'last_peered': '2023-11-06T02:11:28.698549+0000', 'last_clean': '2023-11-06T02:11:28.698549+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:28.698549+0000', 'last_undegraded': '2023-11-06T02:11:28.698549+0000', 'last_fullsized': '2023-11-06T02:11:28.698549+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:35.052 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.a is not active+clean 2023-11-06T02:11:35.053 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:29.703192+0000', 'last_change': '2023-11-06T02:11:29.703192+0000', 'last_active': '2023-11-06T02:11:29.703192+0000', 'last_peered': '2023-11-06T02:11:29.703192+0000', 'last_clean': '2023-11-06T02:11:29.703192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:29.703192+0000', 'last_undegraded': '2023-11-06T02:11:29.703192+0000', 'last_fullsized': '2023-11-06T02:11:29.703192+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:35.053 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.9 is not active+clean 2023-11-06T02:11:35.053 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:28.698549+0000', 'last_change': '2023-11-06T02:11:28.698549+0000', 'last_active': '2023-11-06T02:11:28.698549+0000', 'last_peered': '2023-11-06T02:11:28.698549+0000', 'last_clean': '2023-11-06T02:11:28.698549+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:28.698549+0000', 'last_undegraded': '2023-11-06T02:11:28.698549+0000', 'last_fullsized': '2023-11-06T02:11:28.698549+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:35.054 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.c is not active+clean 2023-11-06T02:11:35.054 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:29.703192+0000', 'last_change': '2023-11-06T02:11:29.703192+0000', 'last_active': '2023-11-06T02:11:29.703192+0000', 'last_peered': '2023-11-06T02:11:29.703192+0000', 'last_clean': '2023-11-06T02:11:29.703192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:29.703192+0000', 'last_undegraded': '2023-11-06T02:11:29.703192+0000', 'last_fullsized': '2023-11-06T02:11:29.703192+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:35.055 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.f is not active+clean 2023-11-06T02:11:35.055 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:28.698549+0000', 'last_change': '2023-11-06T02:11:28.698549+0000', 'last_active': '2023-11-06T02:11:28.698549+0000', 'last_peered': '2023-11-06T02:11:28.698549+0000', 'last_clean': '2023-11-06T02:11:28.698549+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:28.698549+0000', 'last_undegraded': '2023-11-06T02:11:28.698549+0000', 'last_fullsized': '2023-11-06T02:11:28.698549+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:35.055 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.d is not active+clean 2023-11-06T02:11:35.056 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:29.703192+0000', 'last_change': '2023-11-06T02:11:29.703192+0000', 'last_active': '2023-11-06T02:11:29.703192+0000', 'last_peered': '2023-11-06T02:11:29.703192+0000', 'last_clean': '2023-11-06T02:11:29.703192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:29.703192+0000', 'last_undegraded': '2023-11-06T02:11:29.703192+0000', 'last_fullsized': '2023-11-06T02:11:29.703192+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:35.056 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.e is not active+clean 2023-11-06T02:11:35.057 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:28.698549+0000', 'last_change': '2023-11-06T02:11:28.698549+0000', 'last_active': '2023-11-06T02:11:28.698549+0000', 'last_peered': '2023-11-06T02:11:28.698549+0000', 'last_clean': '2023-11-06T02:11:28.698549+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:28.698549+0000', 'last_undegraded': '2023-11-06T02:11:28.698549+0000', 'last_fullsized': '2023-11-06T02:11:28.698549+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:35.057 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.e is not active+clean 2023-11-06T02:11:35.057 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:29.703192+0000', 'last_change': '2023-11-06T02:11:29.703192+0000', 'last_active': '2023-11-06T02:11:29.703192+0000', 'last_peered': '2023-11-06T02:11:29.703192+0000', 'last_clean': '2023-11-06T02:11:29.703192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:29.703192+0000', 'last_undegraded': '2023-11-06T02:11:29.703192+0000', 'last_fullsized': '2023-11-06T02:11:29.703192+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:35.058 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.d is not active+clean 2023-11-06T02:11:35.058 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:28.698549+0000', 'last_change': '2023-11-06T02:11:28.698549+0000', 'last_active': '2023-11-06T02:11:28.698549+0000', 'last_peered': '2023-11-06T02:11:28.698549+0000', 'last_clean': '2023-11-06T02:11:28.698549+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:28.698549+0000', 'last_undegraded': '2023-11-06T02:11:28.698549+0000', 'last_fullsized': '2023-11-06T02:11:28.698549+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:35.059 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.f is not active+clean 2023-11-06T02:11:35.059 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:29.703192+0000', 'last_change': '2023-11-06T02:11:29.703192+0000', 'last_active': '2023-11-06T02:11:29.703192+0000', 'last_peered': '2023-11-06T02:11:29.703192+0000', 'last_clean': '2023-11-06T02:11:29.703192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:29.703192+0000', 'last_undegraded': '2023-11-06T02:11:29.703192+0000', 'last_fullsized': '2023-11-06T02:11:29.703192+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:29.703192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:35.060 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.c is not active+clean 2023-11-06T02:11:35.060 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T02:11:28.698549+0000', 'last_change': '2023-11-06T02:11:28.698549+0000', 'last_active': '2023-11-06T02:11:28.698549+0000', 'last_peered': '2023-11-06T02:11:28.698549+0000', 'last_clean': '2023-11-06T02:11:28.698549+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:11:28.698549+0000', 'last_undegraded': '2023-11-06T02:11:28.698549+0000', 'last_fullsized': '2023-11-06T02:11:28.698549+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'last_clean_scrub_stamp': '2023-11-06T02:11:28.698549+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T02:11:35.061 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-11-06T02:11:35.184 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.185+0000 7fe6c8d50640 1 Processor -- start 2023-11-06T02:11:35.185 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.185+0000 7fe6c8d50640 1 -- start start 2023-11-06T02:11:35.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.189+0000 7fe6c8d50640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fe6c40b0790 0x7fe6c40c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:35.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.189+0000 7fe6c8d50640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe6c40cb010 0x7fe6c40c2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:35.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.189+0000 7fe6c8d50640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6c40cab80 con 0x7fe6c415ab50 2023-11-06T02:11:35.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.189+0000 7fe6c8d50640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6c415af30 con 0x7fe6c40cb010 2023-11-06T02:11:35.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.189+0000 7fe6c8d50640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6c40cb5b0 con 0x7fe6c40b0790 2023-11-06T02:11:35.188 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.189+0000 7fe6c2d9d640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe6c40cb010 0x7fe6c40c2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:35.188 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.189+0000 7fe6c2d9d640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe6c40cb010 0x7fe6c40c2710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48922/0 (socket says 172.21.15.49:48922) 2023-11-06T02:11:35.188 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.189+0000 7fe6c2d9d640 1 -- 172.21.15.49:0/1095975042 learned_addr learned my addr 172.21.15.49:0/1095975042 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:35.188 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.189+0000 7fe6c1d9b640 1 --2- 172.21.15.49:0/1095975042 >> v2:172.21.15.150:3300/0 conn(0x7fe6c40b0790 0x7fe6c40c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:35.189 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.189+0000 7fe6c2d9d640 1 -- 172.21.15.49:0/1095975042 >> v2:172.21.15.150:3300/0 conn(0x7fe6c40b0790 msgr2=0x7fe6c40c9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:35.189 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.189+0000 7fe6c2d9d640 1 --2- 172.21.15.49:0/1095975042 >> v2:172.21.15.150:3300/0 conn(0x7fe6c40b0790 0x7fe6c40c9bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:35.189 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.189+0000 7fe6c2d9d640 1 -- 172.21.15.49:0/1095975042 >> v1:172.21.15.150:6789/0 conn(0x7fe6c415ab50 legacy=0x7fe6c40b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:35.189 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.189+0000 7fe6c2d9d640 1 -- 172.21.15.49:0/1095975042 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6c40cb890 con 0x7fe6c40cb010 2023-11-06T02:11:35.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.193+0000 7fe6c2d9d640 1 --2- 172.21.15.49:0/1095975042 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe6c40cb010 0x7fe6c40c2710 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fe6c406dde0 tx=0x7fe6bc01c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7316c5cb6bce0d9f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:35.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.193+0000 7fe6c159a640 1 -- 172.21.15.49:0/1095975042 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6bc0a5820 con 0x7fe6c40cb010 2023-11-06T02:11:35.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.193+0000 7fe6c159a640 1 -- 172.21.15.49:0/1095975042 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6bc0a5980 con 0x7fe6c40cb010 2023-11-06T02:11:35.195 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.193+0000 7fe6c159a640 1 -- 172.21.15.49:0/1095975042 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6bc0a5d10 con 0x7fe6c40cb010 2023-11-06T02:11:35.195 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.193+0000 7fe6c8d50640 1 -- 172.21.15.49:0/1095975042 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe6c40cb010 msgr2=0x7fe6c40c2710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:35.195 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.193+0000 7fe6c8d50640 1 --2- 172.21.15.49:0/1095975042 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe6c40cb010 0x7fe6c40c2710 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fe6c406dde0 tx=0x7fe6bc01c960 comp rx=0 tx=0).stop 2023-11-06T02:11:35.195 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.193+0000 7fe6c8d50640 1 -- 172.21.15.49:0/1095975042 shutdown_connections 2023-11-06T02:11:35.196 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.193+0000 7fe6c8d50640 1 --2- 172.21.15.49:0/1095975042 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe6c40cb010 0x7fe6c40c2710 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:35.196 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.193+0000 7fe6c8d50640 1 --2- 172.21.15.49:0/1095975042 >> v2:172.21.15.150:3300/0 conn(0x7fe6c40b0790 0x7fe6c40c9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:35.196 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.193+0000 7fe6c8d50640 1 -- 172.21.15.49:0/1095975042 >> 172.21.15.49:0/1095975042 conn(0x7fe6c406c0a0 msgr2=0x7fe6c406c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:35.196 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.193+0000 7fe6c8d50640 1 -- 172.21.15.49:0/1095975042 shutdown_connections 2023-11-06T02:11:35.196 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.193+0000 7fe6c8d50640 1 -- 172.21.15.49:0/1095975042 wait complete. 2023-11-06T02:11:35.197 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.193+0000 7fe6c8d50640 1 Processor -- start 2023-11-06T02:11:35.197 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.193+0000 7fe6c8d50640 1 -- start start 2023-11-06T02:11:35.197 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.193+0000 7fe6c8d50640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe6c40b0790 0x7fe6c40c0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:35.197 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.193+0000 7fe6c8d50640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6c40cb010 0x7fe6c40ae3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:35.198 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.193+0000 7fe6c8d50640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe6c415ab50 0x7fe6c40cd210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:35.198 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.193+0000 7fe6c8d50640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c40c1260 con 0x7fe6c40b0790 2023-11-06T02:11:35.198 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.193+0000 7fe6c8d50640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c40af970 con 0x7fe6c40cb010 2023-11-06T02:11:35.198 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.193+0000 7fe6c8d50640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c40aedf0 con 0x7fe6c415ab50 2023-11-06T02:11:35.199 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.193+0000 7fe6c259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe6c40b0790 0x7fe6c40c0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:35.199 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.193+0000 7fe6c259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe6c40b0790 0x7fe6c40c0d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48934/0 (socket says 172.21.15.49:48934) 2023-11-06T02:11:35.199 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.193+0000 7fe6c259c640 1 -- 172.21.15.49:0/2517005150 learned_addr learned my addr 172.21.15.49:0/2517005150 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:35.199 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.193+0000 7fe6c1d9b640 1 --2- 172.21.15.49:0/2517005150 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6c40cb010 0x7fe6c40ae3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:35.199 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.193+0000 7fe6c2d9d640 1 --2- 172.21.15.49:0/2517005150 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe6c415ab50 0x7fe6c40cd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:35.200 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.193+0000 7fe6c259c640 1 -- 172.21.15.49:0/2517005150 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe6c415ab50 msgr2=0x7fe6c40cd210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:35.200 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.193+0000 7fe6c259c640 1 --2- 172.21.15.49:0/2517005150 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe6c415ab50 0x7fe6c40cd210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:35.200 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.193+0000 7fe6c259c640 1 -- 172.21.15.49:0/2517005150 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6c40cb010 msgr2=0x7fe6c40ae3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:35.200 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.193+0000 7fe6c259c640 1 --2- 172.21.15.49:0/2517005150 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6c40cb010 0x7fe6c40ae3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:35.201 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.193+0000 7fe6c259c640 1 -- 172.21.15.49:0/2517005150 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6bc009b30 con 0x7fe6c40b0790 2023-11-06T02:11:35.201 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.193+0000 7fe6c2d9d640 1 --2- 172.21.15.49:0/2517005150 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe6c415ab50 0x7fe6c40cd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:11:35.201 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.193+0000 7fe6c259c640 1 --2- 172.21.15.49:0/2517005150 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe6c40b0790 0x7fe6c40c0d20 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fe6b400b3f0 tx=0x7fe6b400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:35.201 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.197+0000 7fe6b2ffd640 1 -- 172.21.15.49:0/2517005150 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b4011830 con 0x7fe6c40b0790 2023-11-06T02:11:35.202 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.197+0000 7fe6b2ffd640 1 -- 172.21.15.49:0/2517005150 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6b40119b0 con 0x7fe6c40b0790 2023-11-06T02:11:35.202 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.197+0000 7fe6c8d50640 1 -- 172.21.15.49:0/2517005150 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6c40af0d0 con 0x7fe6c40b0790 2023-11-06T02:11:35.202 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.197+0000 7fe6b2ffd640 1 -- 172.21.15.49:0/2517005150 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b4011d60 con 0x7fe6c40b0790 2023-11-06T02:11:35.202 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.197+0000 7fe6c8d50640 1 -- 172.21.15.49:0/2517005150 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6c40af690 con 0x7fe6c40b0790 2023-11-06T02:11:35.202 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.197+0000 7fe6c8d50640 1 -- 172.21.15.49:0/2517005150 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6c40b0de0 con 0x7fe6c40b0790 2023-11-06T02:11:35.203 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.197+0000 7fe6b2ffd640 1 -- 172.21.15.49:0/2517005150 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fe6b401a410 con 0x7fe6c40b0790 2023-11-06T02:11:35.204 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.197+0000 7fe6b2ffd640 1 --2- 172.21.15.49:0/2517005150 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe6a4065a60 0x7fe6a4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:35.204 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.197+0000 7fe6b2ffd640 1 -- 172.21.15.49:0/2517005150 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe6b40830f0 con 0x7fe6c40b0790 2023-11-06T02:11:35.204 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.197+0000 7fe6c1d9b640 1 --2- 172.21.15.49:0/2517005150 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe6a4065a60 0x7fe6a4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:35.205 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.197+0000 7fe6c1d9b640 1 --2- 172.21.15.49:0/2517005150 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe6a4065a60 0x7fe6a4067f20 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fe6c4159a20 tx=0x7fe6b8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:35.205 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.201+0000 7fe6b2ffd640 1 -- 172.21.15.49:0/2517005150 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fe6b40509e0 con 0x7fe6c40b0790 2023-11-06T02:11:35.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.413+0000 7fe6c8d50640 1 -- 172.21.15.49:0/2517005150 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fe6c40ab950 con 0x7fe6c40b0790 2023-11-06T02:11:35.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.413+0000 7fe6b2ffd640 1 -- 172.21.15.49:0/2517005150 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1443 (secure 0 0 0) 0x7fe6c40ab950 con 0x7fe6c40b0790 2023-11-06T02:11:35.410 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:11:35.411 INFO:teuthology.orchestra.run.smithi049.stdout:{"fsid":"1a8b035c-6ba4-499a-b876-aea47fd9c0d4","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":209,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":65,"num_osds":8,"num_up_osds":8,"osd_up_since":1699236497,"num_in_osds":8,"osd_in_since":1699236488,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":91},{"state_name":"active+clean","count":14}],"num_pgs":105,"num_pools":4,"num_objects":2,"data_bytes":590368,"bytes_used":244666368,"bytes_avail":772849446912,"bytes_total":773094113280,"unknown_pgs_ratio":0.86666667461395264},"fsmap":{"epoch":30,"id":4,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":4,"rank":0,"name":"c","status":"up:active","gid":6348}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2023-11-06T02:10:21.215293+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-11-06T02:11:35.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.413+0000 7fe6c8d50640 1 -- 172.21.15.49:0/2517005150 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe6a4065a60 msgr2=0x7fe6a4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:35.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.413+0000 7fe6c8d50640 1 --2- 172.21.15.49:0/2517005150 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe6a4065a60 0x7fe6a4067f20 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fe6c4159a20 tx=0x7fe6b8008040 comp rx=0 tx=0).stop 2023-11-06T02:11:35.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.413+0000 7fe6c8d50640 1 -- 172.21.15.49:0/2517005150 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe6c40b0790 msgr2=0x7fe6c40c0d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:35.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.413+0000 7fe6c8d50640 1 --2- 172.21.15.49:0/2517005150 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe6c40b0790 0x7fe6c40c0d20 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fe6b400b3f0 tx=0x7fe6b400b8b0 comp rx=0 tx=0).stop 2023-11-06T02:11:35.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.413+0000 7fe6c8d50640 1 -- 172.21.15.49:0/2517005150 shutdown_connections 2023-11-06T02:11:35.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.413+0000 7fe6c8d50640 1 --2- 172.21.15.49:0/2517005150 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe6a4065a60 0x7fe6a4067f20 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:35.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.413+0000 7fe6c8d50640 1 --2- 172.21.15.49:0/2517005150 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe6c415ab50 0x7fe6c40cd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:35.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.413+0000 7fe6c8d50640 1 --2- 172.21.15.49:0/2517005150 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6c40cb010 0x7fe6c40ae3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:35.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.413+0000 7fe6c8d50640 1 --2- 172.21.15.49:0/2517005150 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe6c40b0790 0x7fe6c40c0d20 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:35.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.413+0000 7fe6c8d50640 1 -- 172.21.15.49:0/2517005150 >> 172.21.15.49:0/2517005150 conn(0x7fe6c406c0a0 msgr2=0x7fe6c406ffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:35.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.413+0000 7fe6c8d50640 1 -- 172.21.15.49:0/2517005150 shutdown_connections 2023-11-06T02:11:35.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.413+0000 7fe6c8d50640 1 -- 172.21.15.49:0/2517005150 wait complete. 2023-11-06T02:11:35.428 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-11-06T02:11:35.429 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T02:11:35.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.545+0000 7f9792409640 1 Processor -- start 2023-11-06T02:11:35.542 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.545+0000 7f9792409640 1 -- start start 2023-11-06T02:11:35.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.545+0000 7f9792409640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f978c0cb010 0x7f978c0c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:35.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.545+0000 7f9792409640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f978c0cab80 con 0x7f978c15ab50 2023-11-06T02:11:35.545 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.545+0000 7f9792409640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f978c15af30 con 0x7f978c0b0790 2023-11-06T02:11:35.545 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.545+0000 7f9792409640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f978c0cb5b0 con 0x7f978c0cb010 2023-11-06T02:11:35.545 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.545+0000 7f978bfff640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f978c15ab50 0x7f978c0b0080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:46666/0 (socket says 172.21.15.49:46666) 2023-11-06T02:11:35.545 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.545+0000 7f978bfff640 1 -- 172.21.15.49:0/2081223231 learned_addr learned my addr 172.21.15.49:0/2081223231 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:11:35.545 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.545+0000 7f978affd640 1 -- 172.21.15.49:0/2081223231 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3691696670 0 0) 0x7f978c0cab80 con 0x7f978c15ab50 2023-11-06T02:11:35.546 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.545+0000 7f97909a6640 1 --2- 172.21.15.49:0/2081223231 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f978c0cb010 0x7f978c0c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:35.546 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.545+0000 7f978affd640 1 -- 172.21.15.49:0/2081223231 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9774003530 con 0x7f978c15ab50 2023-11-06T02:11:35.546 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.545+0000 7f978affd640 1 -- 172.21.15.49:0/2081223231 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2197440428 0 0) 0x7f9774003530 con 0x7f978c15ab50 2023-11-06T02:11:35.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.545+0000 7f978affd640 1 -- 172.21.15.49:0/2081223231 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f978c0cb010 msgr2=0x7f978c0c9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:35.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.545+0000 7f978affd640 1 --2- 172.21.15.49:0/2081223231 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f978c0cb010 0x7f978c0c9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:35.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.545+0000 7f978affd640 1 -- 172.21.15.49:0/2081223231 >> v1:172.21.15.150:6789/0 conn(0x7f978c0b0790 legacy=0x7f978c0c2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:35.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.545+0000 7f978affd640 1 -- 172.21.15.49:0/2081223231 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f978c0cb890 con 0x7f978c15ab50 2023-11-06T02:11:35.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.545+0000 7f978affd640 1 -- 172.21.15.49:0/2081223231 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7f977c001820 con 0x7f978c15ab50 2023-11-06T02:11:35.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.545+0000 7f97909a6640 1 --2- 172.21.15.49:0/2081223231 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f978c0cb010 0x7f978c0c9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:11:35.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.545+0000 7f978affd640 1 -- 172.21.15.49:0/2081223231 >> v1:172.21.15.49:6789/0 conn(0x7f978c15ab50 legacy=0x7f978c0b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:35.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.545+0000 7f978affd640 1 --2- 172.21.15.49:0/2081223231 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f978c0cb010 0x7f97740039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:35.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.545+0000 7f978affd640 1 --2- 172.21.15.49:0/2081223231 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9774003fa0 0x7f97740047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:35.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.545+0000 7f978affd640 1 --2- 172.21.15.49:0/2081223231 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9774004d10 0x7f9774005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:35.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.545+0000 7f978affd640 1 -- 172.21.15.49:0/2081223231 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f978c0cb5b0 con 0x7f9774004d10 2023-11-06T02:11:35.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.545+0000 7f978affd640 1 -- 172.21.15.49:0/2081223231 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f977400baf0 con 0x7f9774003fa0 2023-11-06T02:11:35.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.545+0000 7f978affd640 1 -- 172.21.15.49:0/2081223231 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f977400bcb0 con 0x7f978c0cb010 2023-11-06T02:11:35.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.545+0000 7f97909a6640 1 --2- 172.21.15.49:0/2081223231 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9774004d10 0x7f9774005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:35.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.545+0000 7f978bfff640 1 --2- 172.21.15.49:0/2081223231 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f978c0cb010 0x7f97740039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:35.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.545+0000 7f97909a6640 1 -- 172.21.15.49:0/2081223231 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f978c0cb010 msgr2=0x7f97740039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:35.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.545+0000 7f97909a6640 1 --2- 172.21.15.49:0/2081223231 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f978c0cb010 0x7f97740039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:35.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.545+0000 7f978b7fe640 1 --2- 172.21.15.49:0/2081223231 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9774003fa0 0x7f97740047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:35.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.545+0000 7f97909a6640 1 -- 172.21.15.49:0/2081223231 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9774003fa0 msgr2=0x7f97740047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:35.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.545+0000 7f97909a6640 1 --2- 172.21.15.49:0/2081223231 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9774003fa0 0x7f97740047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:35.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.545+0000 7f97909a6640 1 -- 172.21.15.49:0/2081223231 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9774003530 con 0x7f9774004d10 2023-11-06T02:11:35.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.545+0000 7f978b7fe640 1 --2- 172.21.15.49:0/2081223231 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9774003fa0 0x7f97740047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:11:35.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.549+0000 7f97909a6640 1 --2- 172.21.15.49:0/2081223231 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9774004d10 0x7f9774005190 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f97840073e0 tx=0x7f978401cea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecfab17449496cbf server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:35.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.549+0000 7f978affd640 1 -- 172.21.15.49:0/2081223231 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97840a5da0 con 0x7f9774004d10 2023-11-06T02:11:35.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.549+0000 7f978affd640 1 -- 172.21.15.49:0/2081223231 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97840ad440 con 0x7f9774004d10 2023-11-06T02:11:35.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.549+0000 7f978affd640 1 -- 172.21.15.49:0/2081223231 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97840ad7b0 con 0x7f9774004d10 2023-11-06T02:11:35.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.549+0000 7f9792409640 1 -- 172.21.15.49:0/2081223231 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9774004d10 msgr2=0x7f9774005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:35.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.549+0000 7f9792409640 1 --2- 172.21.15.49:0/2081223231 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9774004d10 0x7f9774005190 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f97840073e0 tx=0x7f978401cea0 comp rx=0 tx=0).stop 2023-11-06T02:11:35.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.549+0000 7f97909a6640 1 -- 172.21.15.49:0/2081223231 reap_dead start 2023-11-06T02:11:35.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.549+0000 7f9792409640 1 -- 172.21.15.49:0/2081223231 shutdown_connections 2023-11-06T02:11:35.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.549+0000 7f9792409640 1 -- 172.21.15.49:0/2081223231 >> 172.21.15.49:0/2081223231 conn(0x7f978c06c0a0 msgr2=0x7f978c06c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:35.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.549+0000 7f9792409640 1 -- 172.21.15.49:0/2081223231 shutdown_connections 2023-11-06T02:11:35.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.549+0000 7f9792409640 1 -- 172.21.15.49:0/2081223231 wait complete. 2023-11-06T02:11:35.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.549+0000 7f9792409640 1 Processor -- start 2023-11-06T02:11:35.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.549+0000 7f9792409640 1 -- start start 2023-11-06T02:11:35.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.553+0000 7f9792409640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f978c1edad0 0x7f978c151130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:35.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.553+0000 7f9792409640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f978c151670 0x7f978c1f06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:35.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.553+0000 7f9792409640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f978c1f0bf0 0x7f978c0af270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:35.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.553+0000 7f9792409640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f978c152b00 con 0x7f978c1edad0 2023-11-06T02:11:35.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.553+0000 7f9792409640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f978c151af0 con 0x7f978c151670 2023-11-06T02:11:35.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.553+0000 7f9792409640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f978c1f1430 con 0x7f978c1f0bf0 2023-11-06T02:11:35.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.553+0000 7f978bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f978c1edad0 0x7f978c151130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:35.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.553+0000 7f978bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f978c1edad0 0x7f978c151130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48946/0 (socket says 172.21.15.49:48946) 2023-11-06T02:11:35.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.553+0000 7f978bfff640 1 -- 172.21.15.49:0/995336865 learned_addr learned my addr 172.21.15.49:0/995336865 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:35.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.553+0000 7f97909a6640 1 --2- 172.21.15.49:0/995336865 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f978c1f0bf0 0x7f978c0af270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:35.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.553+0000 7f978b7fe640 1 --2- 172.21.15.49:0/995336865 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f978c151670 0x7f978c1f06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:35.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.553+0000 7f978bfff640 1 -- 172.21.15.49:0/995336865 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f978c1f0bf0 msgr2=0x7f978c0af270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:35.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.553+0000 7f978bfff640 1 --2- 172.21.15.49:0/995336865 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f978c1f0bf0 0x7f978c0af270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:35.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.553+0000 7f978bfff640 1 -- 172.21.15.49:0/995336865 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f978c151670 msgr2=0x7f978c1f06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:35.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.553+0000 7f978bfff640 1 --2- 172.21.15.49:0/995336865 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f978c151670 0x7f978c1f06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:35.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.553+0000 7f978bfff640 1 -- 172.21.15.49:0/995336865 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f978c1f1a10 con 0x7f978c1edad0 2023-11-06T02:11:35.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.553+0000 7f97909a6640 1 --2- 172.21.15.49:0/995336865 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f978c1f0bf0 0x7f978c0af270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:11:35.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.553+0000 7f978b7fe640 1 --2- 172.21.15.49:0/995336865 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f978c151670 0x7f978c1f06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:11:35.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.553+0000 7f978bfff640 1 --2- 172.21.15.49:0/995336865 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f978c1edad0 0x7f978c151130 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f977c000fe0 tx=0x7f977c00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:35.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.553+0000 7f9788ff9640 1 -- 172.21.15.49:0/995336865 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f977c0072e0 con 0x7f978c1edad0 2023-11-06T02:11:35.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.553+0000 7f9788ff9640 1 -- 172.21.15.49:0/995336865 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f977c007440 con 0x7f978c1edad0 2023-11-06T02:11:35.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.553+0000 7f9792409640 1 -- 172.21.15.49:0/995336865 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f978c1f1cf0 con 0x7f978c1edad0 2023-11-06T02:11:35.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.553+0000 7f9788ff9640 1 -- 172.21.15.49:0/995336865 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f977c0057b0 con 0x7f978c1edad0 2023-11-06T02:11:35.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.553+0000 7f9792409640 1 -- 172.21.15.49:0/995336865 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f978c1f1710 con 0x7f978c1edad0 2023-11-06T02:11:35.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.553+0000 7f9792409640 1 -- 172.21.15.49:0/995336865 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f978c0b0de0 con 0x7f978c1edad0 2023-11-06T02:11:35.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.553+0000 7f9788ff9640 1 -- 172.21.15.49:0/995336865 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f977c005910 con 0x7f978c1edad0 2023-11-06T02:11:35.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.553+0000 7f9788ff9640 1 --2- 172.21.15.49:0/995336865 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9778065a90 0x7f9778067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:35.563 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.553+0000 7f9788ff9640 1 -- 172.21.15.49:0/995336865 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f977c083450 con 0x7f978c1edad0 2023-11-06T02:11:35.563 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.553+0000 7f978b7fe640 1 --2- 172.21.15.49:0/995336865 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9778065a90 0x7f9778067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:35.563 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.553+0000 7f978b7fe640 1 --2- 172.21.15.49:0/995336865 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9778065a90 0x7f9778067f50 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f978c157d70 tx=0x7f9780009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:35.563 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.557+0000 7f9788ff9640 1 -- 172.21.15.49:0/995336865 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f977c050cb0 con 0x7f978c1edad0 2023-11-06T02:11:35.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.709+0000 7f9792409640 1 -- 172.21.15.49:0/995336865 --> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f978c0c21f0 con 0x7f9778065a90 2023-11-06T02:11:35.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.733+0000 7f9788ff9640 1 -- 172.21.15.49:0/995336865 <== mgr.4100 v2:172.21.15.150:6832/38104 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+309769 (secure 0 0 0) 0x7f978000d040 con 0x7f9778065a90 2023-11-06T02:11:35.733 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:11:35.735 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2023-11-06T02:11:35.736 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.737+0000 7f9792409640 1 -- 172.21.15.49:0/995336865 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9778065a90 msgr2=0x7f9778067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:35.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.737+0000 7f9792409640 1 --2- 172.21.15.49:0/995336865 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9778065a90 0x7f9778067f50 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f978c157d70 tx=0x7f9780009040 comp rx=0 tx=0).stop 2023-11-06T02:11:35.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.737+0000 7f9792409640 1 -- 172.21.15.49:0/995336865 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f978c1edad0 msgr2=0x7f978c151130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:35.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.737+0000 7f9792409640 1 --2- 172.21.15.49:0/995336865 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f978c1edad0 0x7f978c151130 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f977c000fe0 tx=0x7f977c00eca0 comp rx=0 tx=0).stop 2023-11-06T02:11:35.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.737+0000 7f9792409640 1 -- 172.21.15.49:0/995336865 shutdown_connections 2023-11-06T02:11:35.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.737+0000 7f9792409640 1 --2- 172.21.15.49:0/995336865 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9778065a90 0x7f9778067f50 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:35.738 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.737+0000 7f9792409640 1 --2- 172.21.15.49:0/995336865 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f978c1f0bf0 0x7f978c0af270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:35.738 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.737+0000 7f9792409640 1 --2- 172.21.15.49:0/995336865 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f978c151670 0x7f978c1f06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:35.738 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.737+0000 7f9792409640 1 --2- 172.21.15.49:0/995336865 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f978c1edad0 0x7f978c151130 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:35.738 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.737+0000 7f9792409640 1 -- 172.21.15.49:0/995336865 >> 172.21.15.49:0/995336865 conn(0x7f978c06c0a0 msgr2=0x7f978c0c8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:35.739 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.737+0000 7f9792409640 1 -- 172.21.15.49:0/995336865 shutdown_connections 2023-11-06T02:11:35.739 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:35.737+0000 7f9792409640 1 -- 172.21.15.49:0/995336865 wait complete. 2023-11-06T02:11:35.750 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":157,"stamp":"2023-11-06T02:11:35.240347+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":143,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":239332,"kb_used_data":7332,"kb_used_omap":0,"kb_used_meta":231936,"kb_avail":754735388,"statfs":{"total":773094113280,"available":772849037312,"internally_reserved":0,"allocated":7507968,"data_stored":3954441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":237502464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":69,"apply_latency_ms":69,"commit_latency_ns":69000000,"apply_latency_ns":69000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001775"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.110923+0000","last_change":"2023-11-06T02:11:30.149757+0000","last_active":"2023-11-06T02:11:31.110923+0000","last_peered":"2023-11-06T02:11:31.110923+0000","last_clean":"2023-11-06T02:11:31.110923+0000","last_became_active":"2023-11-06T02:11:30.149443+0000","last_became_peered":"2023-11-06T02:11:30.149443+0000","last_unstale":"2023-11-06T02:11:31.110923+0000","last_undegraded":"2023-11-06T02:11:31.110923+0000","last_fullsized":"2023-11-06T02:11:31.110923+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:51:26.553337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.110206+0000","last_change":"2023-11-06T02:11:30.116131+0000","last_active":"2023-11-06T02:11:31.110206+0000","last_peered":"2023-11-06T02:11:31.110206+0000","last_clean":"2023-11-06T02:11:31.110206+0000","last_became_active":"2023-11-06T02:11:30.115617+0000","last_became_peered":"2023-11-06T02:11:30.115617+0000","last_unstale":"2023-11-06T02:11:31.110206+0000","last_undegraded":"2023-11-06T02:11:31.110206+0000","last_fullsized":"2023-11-06T02:11:31.110206+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:17:20.665225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.111516+0000","last_change":"2023-11-06T02:11:30.157604+0000","last_active":"2023-11-06T02:11:31.111516+0000","last_peered":"2023-11-06T02:11:31.111516+0000","last_clean":"2023-11-06T02:11:31.111516+0000","last_became_active":"2023-11-06T02:11:30.156940+0000","last_became_peered":"2023-11-06T02:11:30.156940+0000","last_unstale":"2023-11-06T02:11:31.111516+0000","last_undegraded":"2023-11-06T02:11:31.111516+0000","last_fullsized":"2023-11-06T02:11:31.111516+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:51:26.553337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.105126+0000","last_change":"2023-11-06T02:11:30.112102+0000","last_active":"2023-11-06T02:11:31.105126+0000","last_peered":"2023-11-06T02:11:31.105126+0000","last_clean":"2023-11-06T02:11:31.105126+0000","last_became_active":"2023-11-06T02:11:30.111394+0000","last_became_peered":"2023-11-06T02:11:30.111394+0000","last_unstale":"2023-11-06T02:11:31.105126+0000","last_undegraded":"2023-11-06T02:11:31.105126+0000","last_fullsized":"2023-11-06T02:11:31.105126+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:05:39.414334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.104320+0000","last_change":"2023-11-06T02:11:30.163104+0000","last_active":"2023-11-06T02:11:31.104320+0000","last_peered":"2023-11-06T02:11:31.104320+0000","last_clean":"2023-11-06T02:11:31.104320+0000","last_became_active":"2023-11-06T02:11:30.162247+0000","last_became_peered":"2023-11-06T02:11:30.162247+0000","last_unstale":"2023-11-06T02:11:31.104320+0000","last_undegraded":"2023-11-06T02:11:31.104320+0000","last_fullsized":"2023-11-06T02:11:31.104320+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:38:56.722411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.101491+0000","last_change":"2023-11-06T02:11:30.157107+0000","last_active":"2023-11-06T02:11:31.101491+0000","last_peered":"2023-11-06T02:11:31.101491+0000","last_clean":"2023-11-06T02:11:31.101491+0000","last_became_active":"2023-11-06T02:11:30.156817+0000","last_became_peered":"2023-11-06T02:11:30.156817+0000","last_unstale":"2023-11-06T02:11:31.101491+0000","last_undegraded":"2023-11-06T02:11:31.101491+0000","last_fullsized":"2023-11-06T02:11:31.101491+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:04:33.551496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.109328+0000","last_change":"2023-11-06T02:11:30.139772+0000","last_active":"2023-11-06T02:11:31.109328+0000","last_peered":"2023-11-06T02:11:31.109328+0000","last_clean":"2023-11-06T02:11:31.109328+0000","last_became_active":"2023-11-06T02:11:30.139447+0000","last_became_peered":"2023-11-06T02:11:30.139447+0000","last_unstale":"2023-11-06T02:11:31.109328+0000","last_undegraded":"2023-11-06T02:11:31.109328+0000","last_fullsized":"2023-11-06T02:11:31.109328+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:24:49.653824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.113270+0000","last_change":"2023-11-06T02:11:30.170784+0000","last_active":"2023-11-06T02:11:31.113270+0000","last_peered":"2023-11-06T02:11:31.113270+0000","last_clean":"2023-11-06T02:11:31.113270+0000","last_became_active":"2023-11-06T02:11:30.170151+0000","last_became_peered":"2023-11-06T02:11:30.170151+0000","last_unstale":"2023-11-06T02:11:31.113270+0000","last_undegraded":"2023-11-06T02:11:31.113270+0000","last_fullsized":"2023-11-06T02:11:31.113270+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:33:33.543401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.115008+0000","last_change":"2023-11-06T02:11:30.164470+0000","last_active":"2023-11-06T02:11:31.115008+0000","last_peered":"2023-11-06T02:11:31.115008+0000","last_clean":"2023-11-06T02:11:31.115008+0000","last_became_active":"2023-11-06T02:11:30.163980+0000","last_became_peered":"2023-11-06T02:11:30.163980+0000","last_unstale":"2023-11-06T02:11:31.115008+0000","last_undegraded":"2023-11-06T02:11:31.115008+0000","last_fullsized":"2023-11-06T02:11:31.115008+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:03:45.538000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.112589+0000","last_change":"2023-11-06T02:11:30.157152+0000","last_active":"2023-11-06T02:11:31.112589+0000","last_peered":"2023-11-06T02:11:31.112589+0000","last_clean":"2023-11-06T02:11:31.112589+0000","last_became_active":"2023-11-06T02:11:30.156508+0000","last_became_peered":"2023-11-06T02:11:30.156508+0000","last_unstale":"2023-11-06T02:11:31.112589+0000","last_undegraded":"2023-11-06T02:11:31.112589+0000","last_fullsized":"2023-11-06T02:11:31.112589+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:09:18.888901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.103369+0000","last_change":"2023-11-06T02:11:30.156115+0000","last_active":"2023-11-06T02:11:31.103369+0000","last_peered":"2023-11-06T02:11:31.103369+0000","last_clean":"2023-11-06T02:11:31.103369+0000","last_became_active":"2023-11-06T02:11:30.154626+0000","last_became_peered":"2023-11-06T02:11:30.154626+0000","last_unstale":"2023-11-06T02:11:31.103369+0000","last_undegraded":"2023-11-06T02:11:31.103369+0000","last_fullsized":"2023-11-06T02:11:31.103369+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:03:45.538000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.113617+0000","last_change":"2023-11-06T02:11:30.170299+0000","last_active":"2023-11-06T02:11:31.113617+0000","last_peered":"2023-11-06T02:11:31.113617+0000","last_clean":"2023-11-06T02:11:31.113617+0000","last_became_active":"2023-11-06T02:11:30.169828+0000","last_became_peered":"2023-11-06T02:11:30.169828+0000","last_unstale":"2023-11-06T02:11:31.113617+0000","last_undegraded":"2023-11-06T02:11:31.113617+0000","last_fullsized":"2023-11-06T02:11:31.113617+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:02:35.764264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.113828+0000","last_change":"2023-11-06T02:11:30.165102+0000","last_active":"2023-11-06T02:11:31.113828+0000","last_peered":"2023-11-06T02:11:31.113828+0000","last_clean":"2023-11-06T02:11:31.113828+0000","last_became_active":"2023-11-06T02:11:30.164268+0000","last_became_peered":"2023-11-06T02:11:30.164268+0000","last_unstale":"2023-11-06T02:11:31.113828+0000","last_undegraded":"2023-11-06T02:11:31.113828+0000","last_fullsized":"2023-11-06T02:11:31.113828+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:01:56.680559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.111929+0000","last_change":"2023-11-06T02:11:30.157292+0000","last_active":"2023-11-06T02:11:31.111929+0000","last_peered":"2023-11-06T02:11:31.111929+0000","last_clean":"2023-11-06T02:11:31.111929+0000","last_became_active":"2023-11-06T02:11:30.156366+0000","last_became_peered":"2023-11-06T02:11:30.156366+0000","last_unstale":"2023-11-06T02:11:31.111929+0000","last_undegraded":"2023-11-06T02:11:31.111929+0000","last_fullsized":"2023-11-06T02:11:31.111929+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:42:52.898859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.111608+0000","last_change":"2023-11-06T02:11:30.157086+0000","last_active":"2023-11-06T02:11:31.111608+0000","last_peered":"2023-11-06T02:11:31.111608+0000","last_clean":"2023-11-06T02:11:31.111608+0000","last_became_active":"2023-11-06T02:11:30.156219+0000","last_became_peered":"2023-11-06T02:11:30.156219+0000","last_unstale":"2023-11-06T02:11:31.111608+0000","last_undegraded":"2023-11-06T02:11:31.111608+0000","last_fullsized":"2023-11-06T02:11:31.111608+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:24:57.712407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.109357+0000","last_change":"2023-11-06T02:11:30.140958+0000","last_active":"2023-11-06T02:11:31.109357+0000","last_peered":"2023-11-06T02:11:31.109357+0000","last_clean":"2023-11-06T02:11:31.109357+0000","last_became_active":"2023-11-06T02:11:30.140693+0000","last_became_peered":"2023-11-06T02:11:30.140693+0000","last_unstale":"2023-11-06T02:11:31.109357+0000","last_undegraded":"2023-11-06T02:11:31.109357+0000","last_fullsized":"2023-11-06T02:11:31.109357+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:49:19.353757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.111129+0000","last_change":"2023-11-06T02:11:30.155485+0000","last_active":"2023-11-06T02:11:31.111129+0000","last_peered":"2023-11-06T02:11:31.111129+0000","last_clean":"2023-11-06T02:11:31.111129+0000","last_became_active":"2023-11-06T02:11:30.155257+0000","last_became_peered":"2023-11-06T02:11:30.155257+0000","last_unstale":"2023-11-06T02:11:31.111129+0000","last_undegraded":"2023-11-06T02:11:31.111129+0000","last_fullsized":"2023-11-06T02:11:31.111129+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:25:05.779932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.103482+0000","last_change":"2023-11-06T02:11:30.112842+0000","last_active":"2023-11-06T02:11:31.103482+0000","last_peered":"2023-11-06T02:11:31.103482+0000","last_clean":"2023-11-06T02:11:31.103482+0000","last_became_active":"2023-11-06T02:11:30.112115+0000","last_became_peered":"2023-11-06T02:11:30.112115+0000","last_unstale":"2023-11-06T02:11:31.103482+0000","last_undegraded":"2023-11-06T02:11:31.103482+0000","last_fullsized":"2023-11-06T02:11:31.103482+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:01:08.304387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.104026+0000","last_change":"2023-11-06T02:11:30.130439+0000","last_active":"2023-11-06T02:11:31.104026+0000","last_peered":"2023-11-06T02:11:31.104026+0000","last_clean":"2023-11-06T02:11:31.104026+0000","last_became_active":"2023-11-06T02:11:30.129162+0000","last_became_peered":"2023-11-06T02:11:30.129162+0000","last_unstale":"2023-11-06T02:11:31.104026+0000","last_undegraded":"2023-11-06T02:11:31.104026+0000","last_fullsized":"2023-11-06T02:11:31.104026+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:04:12.430683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.108538+0000","last_change":"2023-11-06T02:11:30.132754+0000","last_active":"2023-11-06T02:11:31.108538+0000","last_peered":"2023-11-06T02:11:31.108538+0000","last_clean":"2023-11-06T02:11:31.108538+0000","last_became_active":"2023-11-06T02:11:30.132455+0000","last_became_peered":"2023-11-06T02:11:30.132455+0000","last_unstale":"2023-11-06T02:11:31.108538+0000","last_undegraded":"2023-11-06T02:11:31.108538+0000","last_fullsized":"2023-11-06T02:11:31.108538+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:00:58.743785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.113956+0000","last_change":"2023-11-06T02:11:30.150441+0000","last_active":"2023-11-06T02:11:31.113956+0000","last_peered":"2023-11-06T02:11:31.113956+0000","last_clean":"2023-11-06T02:11:31.113956+0000","last_became_active":"2023-11-06T02:11:30.148953+0000","last_became_peered":"2023-11-06T02:11:30.148953+0000","last_unstale":"2023-11-06T02:11:31.113956+0000","last_undegraded":"2023-11-06T02:11:31.113956+0000","last_fullsized":"2023-11-06T02:11:31.113956+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:03:45.538000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.109268+0000","last_change":"2023-11-06T02:11:30.136621+0000","last_active":"2023-11-06T02:11:31.109268+0000","last_peered":"2023-11-06T02:11:31.109268+0000","last_clean":"2023-11-06T02:11:31.109268+0000","last_became_active":"2023-11-06T02:11:30.136359+0000","last_became_peered":"2023-11-06T02:11:30.136359+0000","last_unstale":"2023-11-06T02:11:31.109268+0000","last_undegraded":"2023-11-06T02:11:31.109268+0000","last_fullsized":"2023-11-06T02:11:31.109268+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:40:12.076675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.108358+0000","last_change":"2023-11-06T02:11:30.137159+0000","last_active":"2023-11-06T02:11:31.108358+0000","last_peered":"2023-11-06T02:11:31.108358+0000","last_clean":"2023-11-06T02:11:31.108358+0000","last_became_active":"2023-11-06T02:11:30.136573+0000","last_became_peered":"2023-11-06T02:11:30.136573+0000","last_unstale":"2023-11-06T02:11:31.108358+0000","last_undegraded":"2023-11-06T02:11:31.108358+0000","last_fullsized":"2023-11-06T02:11:31.108358+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:17:41.584737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.111204+0000","last_change":"2023-11-06T02:11:30.159127+0000","last_active":"2023-11-06T02:11:31.111204+0000","last_peered":"2023-11-06T02:11:31.111204+0000","last_clean":"2023-11-06T02:11:31.111204+0000","last_became_active":"2023-11-06T02:11:30.158553+0000","last_became_peered":"2023-11-06T02:11:30.158553+0000","last_unstale":"2023-11-06T02:11:31.111204+0000","last_undegraded":"2023-11-06T02:11:31.111204+0000","last_fullsized":"2023-11-06T02:11:31.111204+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:48:50.245921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.114679+0000","last_change":"2023-11-06T02:11:30.149808+0000","last_active":"2023-11-06T02:11:31.114679+0000","last_peered":"2023-11-06T02:11:31.114679+0000","last_clean":"2023-11-06T02:11:31.114679+0000","last_became_active":"2023-11-06T02:11:30.149088+0000","last_became_peered":"2023-11-06T02:11:30.149088+0000","last_unstale":"2023-11-06T02:11:31.114679+0000","last_undegraded":"2023-11-06T02:11:31.114679+0000","last_fullsized":"2023-11-06T02:11:31.114679+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:39:35.439329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.113708+0000","last_change":"2023-11-06T02:11:30.151270+0000","last_active":"2023-11-06T02:11:31.113708+0000","last_peered":"2023-11-06T02:11:31.113708+0000","last_clean":"2023-11-06T02:11:31.113708+0000","last_became_active":"2023-11-06T02:11:30.150004+0000","last_became_peered":"2023-11-06T02:11:30.150004+0000","last_unstale":"2023-11-06T02:11:31.113708+0000","last_undegraded":"2023-11-06T02:11:31.113708+0000","last_fullsized":"2023-11-06T02:11:31.113708+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:06:06.524182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.103621+0000","last_change":"2023-11-06T02:11:30.155003+0000","last_active":"2023-11-06T02:11:31.103621+0000","last_peered":"2023-11-06T02:11:31.103621+0000","last_clean":"2023-11-06T02:11:31.103621+0000","last_became_active":"2023-11-06T02:11:30.154370+0000","last_became_peered":"2023-11-06T02:11:30.154370+0000","last_unstale":"2023-11-06T02:11:31.103621+0000","last_undegraded":"2023-11-06T02:11:31.103621+0000","last_fullsized":"2023-11-06T02:11:31.103621+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:20:36.621184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.110334+0000","last_change":"2023-11-06T02:11:30.142104+0000","last_active":"2023-11-06T02:11:31.110334+0000","last_peered":"2023-11-06T02:11:31.110334+0000","last_clean":"2023-11-06T02:11:31.110334+0000","last_became_active":"2023-11-06T02:11:30.141733+0000","last_became_peered":"2023-11-06T02:11:30.141733+0000","last_unstale":"2023-11-06T02:11:31.110334+0000","last_undegraded":"2023-11-06T02:11:31.110334+0000","last_fullsized":"2023-11-06T02:11:31.110334+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:17:21.200734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.111719+0000","last_change":"2023-11-06T02:11:30.163281+0000","last_active":"2023-11-06T02:11:31.111719+0000","last_peered":"2023-11-06T02:11:31.111719+0000","last_clean":"2023-11-06T02:11:31.111719+0000","last_became_active":"2023-11-06T02:11:30.162419+0000","last_became_peered":"2023-11-06T02:11:30.162419+0000","last_unstale":"2023-11-06T02:11:31.111719+0000","last_undegraded":"2023-11-06T02:11:31.111719+0000","last_fullsized":"2023-11-06T02:11:31.111719+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:10:19.972522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.113122+0000","last_change":"2023-11-06T02:11:30.170656+0000","last_active":"2023-11-06T02:11:31.113122+0000","last_peered":"2023-11-06T02:11:31.113122+0000","last_clean":"2023-11-06T02:11:31.113122+0000","last_became_active":"2023-11-06T02:11:30.170084+0000","last_became_peered":"2023-11-06T02:11:30.170084+0000","last_unstale":"2023-11-06T02:11:31.113122+0000","last_undegraded":"2023-11-06T02:11:31.113122+0000","last_fullsized":"2023-11-06T02:11:31.113122+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:32:50.421400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.103195+0000","last_change":"2023-11-06T02:11:30.144938+0000","last_active":"2023-11-06T02:11:31.103195+0000","last_peered":"2023-11-06T02:11:31.103195+0000","last_clean":"2023-11-06T02:11:31.103195+0000","last_became_active":"2023-11-06T02:11:30.135978+0000","last_became_peered":"2023-11-06T02:11:30.135978+0000","last_unstale":"2023-11-06T02:11:31.103195+0000","last_undegraded":"2023-11-06T02:11:31.103195+0000","last_fullsized":"2023-11-06T02:11:31.103195+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:38:11.812067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.100930+0000","last_change":"2023-11-06T02:11:30.118519+0000","last_active":"2023-11-06T02:11:31.100930+0000","last_peered":"2023-11-06T02:11:31.100930+0000","last_clean":"2023-11-06T02:11:31.100930+0000","last_became_active":"2023-11-06T02:11:30.118363+0000","last_became_peered":"2023-11-06T02:11:30.118363+0000","last_unstale":"2023-11-06T02:11:31.100930+0000","last_undegraded":"2023-11-06T02:11:31.100930+0000","last_fullsized":"2023-11-06T02:11:31.100930+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:07:25.816920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.104604+0000","last_change":"2023-11-06T02:11:30.131660+0000","last_active":"2023-11-06T02:11:31.104604+0000","last_peered":"2023-11-06T02:11:31.104604+0000","last_clean":"2023-11-06T02:11:31.104604+0000","last_became_active":"2023-11-06T02:11:30.130727+0000","last_became_peered":"2023-11-06T02:11:30.130727+0000","last_unstale":"2023-11-06T02:11:31.104604+0000","last_undegraded":"2023-11-06T02:11:31.104604+0000","last_fullsized":"2023-11-06T02:11:31.104604+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:59:55.787496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"64'4","reported_seq":15,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.111077+0000","last_change":"2023-11-06T02:11:29.759917+0000","last_active":"2023-11-06T02:11:31.111077+0000","last_peered":"2023-11-06T02:11:31.111077+0000","last_clean":"2023-11-06T02:11:31.111077+0000","last_became_active":"2023-11-06T02:11:29.758884+0000","last_became_peered":"2023-11-06T02:11:29.758884+0000","last_unstale":"2023-11-06T02:11:31.111077+0000","last_undegraded":"2023-11-06T02:11:31.111077+0000","last_fullsized":"2023-11-06T02:11:31.111077+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:10:27.614929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.103877+0000","last_change":"2023-11-06T02:11:30.153818+0000","last_active":"2023-11-06T02:11:31.103877+0000","last_peered":"2023-11-06T02:11:31.103877+0000","last_clean":"2023-11-06T02:11:31.103877+0000","last_became_active":"2023-11-06T02:11:30.153100+0000","last_became_peered":"2023-11-06T02:11:30.153100+0000","last_unstale":"2023-11-06T02:11:31.103877+0000","last_undegraded":"2023-11-06T02:11:31.103877+0000","last_fullsized":"2023-11-06T02:11:31.103877+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:41:12.940684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"64'1","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.101653+0000","last_change":"2023-11-06T02:11:29.756830+0000","last_active":"2023-11-06T02:11:31.101653+0000","last_peered":"2023-11-06T02:11:31.101653+0000","last_clean":"2023-11-06T02:11:31.101653+0000","last_became_active":"2023-11-06T02:11:29.756537+0000","last_became_peered":"2023-11-06T02:11:29.756537+0000","last_unstale":"2023-11-06T02:11:31.101653+0000","last_undegraded":"2023-11-06T02:11:31.101653+0000","last_fullsized":"2023-11-06T02:11:31.101653+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:24:56.707764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.101564+0000","last_change":"2023-11-06T02:11:30.116801+0000","last_active":"2023-11-06T02:11:31.101564+0000","last_peered":"2023-11-06T02:11:31.101564+0000","last_clean":"2023-11-06T02:11:31.101564+0000","last_became_active":"2023-11-06T02:11:30.116554+0000","last_became_peered":"2023-11-06T02:11:30.116554+0000","last_unstale":"2023-11-06T02:11:31.101564+0000","last_undegraded":"2023-11-06T02:11:31.101564+0000","last_fullsized":"2023-11-06T02:11:31.101564+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:58:42.053934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"64'2","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.101817+0000","last_change":"2023-11-06T02:11:29.767654+0000","last_active":"2023-11-06T02:11:31.101817+0000","last_peered":"2023-11-06T02:11:31.101817+0000","last_clean":"2023-11-06T02:11:31.101817+0000","last_became_active":"2023-11-06T02:11:29.767300+0000","last_became_peered":"2023-11-06T02:11:29.767300+0000","last_unstale":"2023-11-06T02:11:31.101817+0000","last_undegraded":"2023-11-06T02:11:31.101817+0000","last_fullsized":"2023-11-06T02:11:31.101817+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:35:46.481307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.114771+0000","last_change":"2023-11-06T02:11:30.158337+0000","last_active":"2023-11-06T02:11:31.114771+0000","last_peered":"2023-11-06T02:11:31.114771+0000","last_clean":"2023-11-06T02:11:31.114771+0000","last_became_active":"2023-11-06T02:11:30.157712+0000","last_became_peered":"2023-11-06T02:11:30.157712+0000","last_unstale":"2023-11-06T02:11:31.114771+0000","last_undegraded":"2023-11-06T02:11:31.114771+0000","last_fullsized":"2023-11-06T02:11:31.114771+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:01:56.680559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"64'2","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.101210+0000","last_change":"2023-11-06T02:11:29.742162+0000","last_active":"2023-11-06T02:11:31.101210+0000","last_peered":"2023-11-06T02:11:31.101210+0000","last_clean":"2023-11-06T02:11:31.101210+0000","last_became_active":"2023-11-06T02:11:29.741491+0000","last_became_peered":"2023-11-06T02:11:29.741491+0000","last_unstale":"2023-11-06T02:11:31.101210+0000","last_undegraded":"2023-11-06T02:11:31.101210+0000","last_fullsized":"2023-11-06T02:11:31.101210+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:09:17.884259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.109510+0000","last_change":"2023-11-06T02:11:30.123394+0000","last_active":"2023-11-06T02:11:31.109510+0000","last_peered":"2023-11-06T02:11:31.109510+0000","last_clean":"2023-11-06T02:11:31.109510+0000","last_became_active":"2023-11-06T02:11:30.123113+0000","last_became_peered":"2023-11-06T02:11:30.123113+0000","last_unstale":"2023-11-06T02:11:31.109510+0000","last_undegraded":"2023-11-06T02:11:31.109510+0000","last_fullsized":"2023-11-06T02:11:31.109510+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:44:35.667968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"64'1","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.108580+0000","last_change":"2023-11-06T02:11:29.737724+0000","last_active":"2023-11-06T02:11:31.108580+0000","last_peered":"2023-11-06T02:11:31.108580+0000","last_clean":"2023-11-06T02:11:31.108580+0000","last_became_active":"2023-11-06T02:11:29.737189+0000","last_became_peered":"2023-11-06T02:11:29.737189+0000","last_unstale":"2023-11-06T02:11:31.108580+0000","last_undegraded":"2023-11-06T02:11:31.108580+0000","last_fullsized":"2023-11-06T02:11:31.108580+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:56:30.903511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.109891+0000","last_change":"2023-11-06T02:11:30.154541+0000","last_active":"2023-11-06T02:11:31.109891+0000","last_peered":"2023-11-06T02:11:31.109891+0000","last_clean":"2023-11-06T02:11:31.109891+0000","last_became_active":"2023-11-06T02:11:30.154171+0000","last_became_peered":"2023-11-06T02:11:30.154171+0000","last_unstale":"2023-11-06T02:11:31.109891+0000","last_undegraded":"2023-11-06T02:11:31.109891+0000","last_fullsized":"2023-11-06T02:11:31.109891+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:10:42.238840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"64'1","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.109508+0000","last_change":"2023-11-06T02:11:29.736013+0000","last_active":"2023-11-06T02:11:31.109508+0000","last_peered":"2023-11-06T02:11:31.109508+0000","last_clean":"2023-11-06T02:11:31.109508+0000","last_became_active":"2023-11-06T02:11:29.735524+0000","last_became_peered":"2023-11-06T02:11:29.735524+0000","last_unstale":"2023-11-06T02:11:31.109508+0000","last_undegraded":"2023-11-06T02:11:31.109508+0000","last_fullsized":"2023-11-06T02:11:31.109508+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:33:32.538758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.104107+0000","last_change":"2023-11-06T02:11:30.132676+0000","last_active":"2023-11-06T02:11:31.104107+0000","last_peered":"2023-11-06T02:11:31.104107+0000","last_clean":"2023-11-06T02:11:31.104107+0000","last_became_active":"2023-11-06T02:11:30.131459+0000","last_became_peered":"2023-11-06T02:11:30.131459+0000","last_unstale":"2023-11-06T02:11:31.104107+0000","last_undegraded":"2023-11-06T02:11:31.104107+0000","last_fullsized":"2023-11-06T02:11:31.104107+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:37:58.300261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.108633+0000","last_change":"2023-11-06T02:11:29.745651+0000","last_active":"2023-11-06T02:11:31.108633+0000","last_peered":"2023-11-06T02:11:31.108633+0000","last_clean":"2023-11-06T02:11:31.108633+0000","last_became_active":"2023-11-06T02:11:29.744713+0000","last_became_peered":"2023-11-06T02:11:29.744713+0000","last_unstale":"2023-11-06T02:11:31.108633+0000","last_undegraded":"2023-11-06T02:11:31.108633+0000","last_fullsized":"2023-11-06T02:11:31.108633+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:06:05.519539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.108513+0000","last_change":"2023-11-06T02:11:30.111938+0000","last_active":"2023-11-06T02:11:31.108513+0000","last_peered":"2023-11-06T02:11:31.108513+0000","last_clean":"2023-11-06T02:11:31.108513+0000","last_became_active":"2023-11-06T02:11:30.111649+0000","last_became_peered":"2023-11-06T02:11:30.111649+0000","last_unstale":"2023-11-06T02:11:31.108513+0000","last_undegraded":"2023-11-06T02:11:31.108513+0000","last_fullsized":"2023-11-06T02:11:31.108513+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:47:22.667065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.103894+0000","last_change":"2023-11-06T02:11:29.759956+0000","last_active":"2023-11-06T02:11:31.103894+0000","last_peered":"2023-11-06T02:11:31.103894+0000","last_clean":"2023-11-06T02:11:31.103894+0000","last_became_active":"2023-11-06T02:11:29.759238+0000","last_became_peered":"2023-11-06T02:11:29.759238+0000","last_unstale":"2023-11-06T02:11:31.103894+0000","last_undegraded":"2023-11-06T02:11:31.103894+0000","last_fullsized":"2023-11-06T02:11:31.103894+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:24:48.649181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.110472+0000","last_change":"2023-11-06T02:11:30.127321+0000","last_active":"2023-11-06T02:11:31.110472+0000","last_peered":"2023-11-06T02:11:31.110472+0000","last_clean":"2023-11-06T02:11:31.110472+0000","last_became_active":"2023-11-06T02:11:30.127128+0000","last_became_peered":"2023-11-06T02:11:30.127128+0000","last_unstale":"2023-11-06T02:11:31.110472+0000","last_undegraded":"2023-11-06T02:11:31.110472+0000","last_fullsized":"2023-11-06T02:11:31.110472+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:43:26.651152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.115134+0000","last_change":"2023-11-06T02:11:29.765154+0000","last_active":"2023-11-06T02:11:31.115134+0000","last_peered":"2023-11-06T02:11:31.115134+0000","last_clean":"2023-11-06T02:11:31.115134+0000","last_became_active":"2023-11-06T02:11:29.764470+0000","last_became_peered":"2023-11-06T02:11:29.764470+0000","last_unstale":"2023-11-06T02:11:31.115134+0000","last_undegraded":"2023-11-06T02:11:31.115134+0000","last_fullsized":"2023-11-06T02:11:31.115134+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:59:54.782853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.108609+0000","last_change":"2023-11-06T02:11:30.150007+0000","last_active":"2023-11-06T02:11:31.108609+0000","last_peered":"2023-11-06T02:11:31.108609+0000","last_clean":"2023-11-06T02:11:31.108609+0000","last_became_active":"2023-11-06T02:11:30.149680+0000","last_became_peered":"2023-11-06T02:11:30.149680+0000","last_unstale":"2023-11-06T02:11:31.108609+0000","last_undegraded":"2023-11-06T02:11:31.108609+0000","last_fullsized":"2023-11-06T02:11:31.108609+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:10:05.661777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.103973+0000","last_change":"2023-11-06T02:11:29.752256+0000","last_active":"2023-11-06T02:11:31.103973+0000","last_peered":"2023-11-06T02:11:31.103973+0000","last_clean":"2023-11-06T02:11:31.103973+0000","last_became_active":"2023-11-06T02:11:29.751386+0000","last_became_peered":"2023-11-06T02:11:29.751386+0000","last_unstale":"2023-11-06T02:11:31.103973+0000","last_undegraded":"2023-11-06T02:11:31.103973+0000","last_fullsized":"2023-11-06T02:11:31.103973+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:08:31.634000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.104914+0000","last_change":"2023-11-06T02:11:30.150094+0000","last_active":"2023-11-06T02:11:31.104914+0000","last_peered":"2023-11-06T02:11:31.104914+0000","last_clean":"2023-11-06T02:11:31.104914+0000","last_became_active":"2023-11-06T02:11:30.149483+0000","last_became_peered":"2023-11-06T02:11:30.149483+0000","last_unstale":"2023-11-06T02:11:31.104914+0000","last_undegraded":"2023-11-06T02:11:31.104914+0000","last_fullsized":"2023-11-06T02:11:31.104914+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:35:11.189946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"64'3","reported_seq":14,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.112548+0000","last_change":"2023-11-06T02:11:29.763421+0000","last_active":"2023-11-06T02:11:31.112548+0000","last_peered":"2023-11-06T02:11:31.112548+0000","last_clean":"2023-11-06T02:11:31.112548+0000","last_became_active":"2023-11-06T02:11:29.762577+0000","last_became_peered":"2023-11-06T02:11:29.762577+0000","last_unstale":"2023-11-06T02:11:31.112548+0000","last_undegraded":"2023-11-06T02:11:31.112548+0000","last_fullsized":"2023-11-06T02:11:31.112548+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:34:29.023464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.103938+0000","last_change":"2023-11-06T02:11:30.153379+0000","last_active":"2023-11-06T02:11:31.103938+0000","last_peered":"2023-11-06T02:11:31.103938+0000","last_clean":"2023-11-06T02:11:31.103938+0000","last_became_active":"2023-11-06T02:11:30.152795+0000","last_became_peered":"2023-11-06T02:11:30.152795+0000","last_unstale":"2023-11-06T02:11:31.103938+0000","last_undegraded":"2023-11-06T02:11:31.103938+0000","last_fullsized":"2023-11-06T02:11:31.103938+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:42:52.898859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"64'6","reported_seq":17,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.110404+0000","last_change":"2023-11-06T02:11:29.760850+0000","last_active":"2023-11-06T02:11:31.110404+0000","last_peered":"2023-11-06T02:11:31.110404+0000","last_clean":"2023-11-06T02:11:31.110404+0000","last_became_active":"2023-11-06T02:11:29.760540+0000","last_became_peered":"2023-11-06T02:11:29.760540+0000","last_unstale":"2023-11-06T02:11:31.110404+0000","last_undegraded":"2023-11-06T02:11:31.110404+0000","last_fullsized":"2023-11-06T02:11:31.110404+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:37:57.295618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.102972+0000","last_change":"2023-11-06T02:11:30.151250+0000","last_active":"2023-11-06T02:11:31.102972+0000","last_peered":"2023-11-06T02:11:31.102972+0000","last_clean":"2023-11-06T02:11:31.102972+0000","last_became_active":"2023-11-06T02:11:30.147146+0000","last_became_peered":"2023-11-06T02:11:30.147146+0000","last_unstale":"2023-11-06T02:11:31.102972+0000","last_undegraded":"2023-11-06T02:11:31.102972+0000","last_fullsized":"2023-11-06T02:11:31.102972+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:24:57.712407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"64'2","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.111934+0000","last_change":"2023-11-06T02:11:29.761994+0000","last_active":"2023-11-06T02:11:31.111934+0000","last_peered":"2023-11-06T02:11:31.111934+0000","last_clean":"2023-11-06T02:11:31.111934+0000","last_became_active":"2023-11-06T02:11:29.761191+0000","last_became_peered":"2023-11-06T02:11:29.761191+0000","last_unstale":"2023-11-06T02:11:31.111934+0000","last_undegraded":"2023-11-06T02:11:31.111934+0000","last_fullsized":"2023-11-06T02:11:31.111934+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:38:55.717768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.113091+0000","last_change":"2023-11-06T02:11:30.139089+0000","last_active":"2023-11-06T02:11:31.113091+0000","last_peered":"2023-11-06T02:11:31.113091+0000","last_clean":"2023-11-06T02:11:31.113091+0000","last_became_active":"2023-11-06T02:11:30.138277+0000","last_became_peered":"2023-11-06T02:11:30.138277+0000","last_unstale":"2023-11-06T02:11:31.113091+0000","last_undegraded":"2023-11-06T02:11:31.113091+0000","last_fullsized":"2023-11-06T02:11:31.113091+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:31:04.064635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.113452+0000","last_change":"2023-11-06T02:11:29.767908+0000","last_active":"2023-11-06T02:11:31.113452+0000","last_peered":"2023-11-06T02:11:31.113452+0000","last_clean":"2023-11-06T02:11:31.113452+0000","last_became_active":"2023-11-06T02:11:29.767155+0000","last_became_peered":"2023-11-06T02:11:29.767155+0000","last_unstale":"2023-11-06T02:11:31.113452+0000","last_undegraded":"2023-11-06T02:11:31.113452+0000","last_fullsized":"2023-11-06T02:11:31.113452+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:20:25.114853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.104161+0000","last_change":"2023-11-06T02:11:30.132180+0000","last_active":"2023-11-06T02:11:31.104161+0000","last_peered":"2023-11-06T02:11:31.104161+0000","last_clean":"2023-11-06T02:11:31.104161+0000","last_became_active":"2023-11-06T02:11:30.129275+0000","last_became_peered":"2023-11-06T02:11:30.129275+0000","last_unstale":"2023-11-06T02:11:31.104161+0000","last_undegraded":"2023-11-06T02:11:31.104161+0000","last_fullsized":"2023-11-06T02:11:31.104161+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:10:28.619572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.112046+0000","last_change":"2023-11-06T02:11:29.756928+0000","last_active":"2023-11-06T02:11:31.112046+0000","last_peered":"2023-11-06T02:11:31.112046+0000","last_clean":"2023-11-06T02:11:31.112046+0000","last_became_active":"2023-11-06T02:11:29.756079+0000","last_became_peered":"2023-11-06T02:11:29.756079+0000","last_unstale":"2023-11-06T02:11:31.112046+0000","last_undegraded":"2023-11-06T02:11:31.112046+0000","last_fullsized":"2023-11-06T02:11:31.112046+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:39:30.010037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.110651+0000","last_change":"2023-11-06T02:11:30.139787+0000","last_active":"2023-11-06T02:11:31.110651+0000","last_peered":"2023-11-06T02:11:31.110651+0000","last_clean":"2023-11-06T02:11:31.110651+0000","last_became_active":"2023-11-06T02:11:30.139486+0000","last_became_peered":"2023-11-06T02:11:30.139486+0000","last_unstale":"2023-11-06T02:11:31.110651+0000","last_undegraded":"2023-11-06T02:11:31.110651+0000","last_fullsized":"2023-11-06T02:11:31.110651+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:42:52.898859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"64'8","reported_seq":19,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.103236+0000","last_change":"2023-11-06T02:11:29.758210+0000","last_active":"2023-11-06T02:11:31.103236+0000","last_peered":"2023-11-06T02:11:31.103236+0000","last_clean":"2023-11-06T02:11:31.103236+0000","last_became_active":"2023-11-06T02:11:29.757494+0000","last_became_peered":"2023-11-06T02:11:29.757494+0000","last_unstale":"2023-11-06T02:11:31.103236+0000","last_undegraded":"2023-11-06T02:11:31.103236+0000","last_fullsized":"2023-11-06T02:11:31.103236+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:20:00.073264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.101279+0000","last_change":"2023-11-06T02:11:30.154538+0000","last_active":"2023-11-06T02:11:31.101279+0000","last_peered":"2023-11-06T02:11:31.101279+0000","last_clean":"2023-11-06T02:11:31.101279+0000","last_became_active":"2023-11-06T02:11:30.154253+0000","last_became_peered":"2023-11-06T02:11:30.154253+0000","last_unstale":"2023-11-06T02:11:31.101279+0000","last_undegraded":"2023-11-06T02:11:31.101279+0000","last_fullsized":"2023-11-06T02:11:31.101279+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:47:05.853422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"64'2","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.115085+0000","last_change":"2023-11-06T02:11:29.767274+0000","last_active":"2023-11-06T02:11:31.115085+0000","last_peered":"2023-11-06T02:11:31.115085+0000","last_clean":"2023-11-06T02:11:31.115085+0000","last_became_active":"2023-11-06T02:11:29.766821+0000","last_became_peered":"2023-11-06T02:11:29.766821+0000","last_unstale":"2023-11-06T02:11:31.115085+0000","last_undegraded":"2023-11-06T02:11:31.115085+0000","last_fullsized":"2023-11-06T02:11:31.115085+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:51:25.548695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.110281+0000","last_change":"2023-11-06T02:11:30.113941+0000","last_active":"2023-11-06T02:11:31.110281+0000","last_peered":"2023-11-06T02:11:31.110281+0000","last_clean":"2023-11-06T02:11:31.110281+0000","last_became_active":"2023-11-06T02:11:30.113672+0000","last_became_peered":"2023-11-06T02:11:30.113672+0000","last_unstale":"2023-11-06T02:11:31.110281+0000","last_undegraded":"2023-11-06T02:11:31.110281+0000","last_fullsized":"2023-11-06T02:11:31.110281+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:06:25.213807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.114556+0000","last_change":"2023-11-06T02:11:29.755075+0000","last_active":"2023-11-06T02:11:31.114556+0000","last_peered":"2023-11-06T02:11:31.114556+0000","last_clean":"2023-11-06T02:11:31.114556+0000","last_became_active":"2023-11-06T02:11:29.754356+0000","last_became_peered":"2023-11-06T02:11:29.754356+0000","last_unstale":"2023-11-06T02:11:31.114556+0000","last_undegraded":"2023-11-06T02:11:31.114556+0000","last_fullsized":"2023-11-06T02:11:31.114556+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:17:20.196092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.114316+0000","last_change":"2023-11-06T02:11:30.150702+0000","last_active":"2023-11-06T02:11:31.114316+0000","last_peered":"2023-11-06T02:11:31.114316+0000","last_clean":"2023-11-06T02:11:31.114316+0000","last_became_active":"2023-11-06T02:11:30.148902+0000","last_became_peered":"2023-11-06T02:11:30.148902+0000","last_unstale":"2023-11-06T02:11:31.114316+0000","last_undegraded":"2023-11-06T02:11:31.114316+0000","last_fullsized":"2023-11-06T02:11:31.114316+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:17:40.439898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.101470+0000","last_change":"2023-11-06T02:11:29.769534+0000","last_active":"2023-11-06T02:11:31.101470+0000","last_peered":"2023-11-06T02:11:31.101470+0000","last_clean":"2023-11-06T02:11:31.101470+0000","last_became_active":"2023-11-06T02:11:29.769107+0000","last_became_peered":"2023-11-06T02:11:29.769107+0000","last_unstale":"2023-11-06T02:11:31.101470+0000","last_undegraded":"2023-11-06T02:11:31.101470+0000","last_fullsized":"2023-11-06T02:11:31.101470+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:06:05.519539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.105010+0000","last_change":"2023-11-06T02:11:30.130842+0000","last_active":"2023-11-06T02:11:31.105010+0000","last_peered":"2023-11-06T02:11:31.105010+0000","last_clean":"2023-11-06T02:11:31.105010+0000","last_became_active":"2023-11-06T02:11:30.129575+0000","last_became_peered":"2023-11-06T02:11:30.129575+0000","last_unstale":"2023-11-06T02:11:31.105010+0000","last_undegraded":"2023-11-06T02:11:31.105010+0000","last_fullsized":"2023-11-06T02:11:31.105010+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:08:32.638643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.110468+0000","last_change":"2023-11-06T02:11:29.760868+0000","last_active":"2023-11-06T02:11:31.110468+0000","last_peered":"2023-11-06T02:11:31.110468+0000","last_clean":"2023-11-06T02:11:31.110468+0000","last_became_active":"2023-11-06T02:11:29.760567+0000","last_became_peered":"2023-11-06T02:11:29.760567+0000","last_unstale":"2023-11-06T02:11:31.110468+0000","last_undegraded":"2023-11-06T02:11:31.110468+0000","last_fullsized":"2023-11-06T02:11:31.110468+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:40:36.082625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.103668+0000","last_change":"2023-11-06T02:11:30.129382+0000","last_active":"2023-11-06T02:11:31.103668+0000","last_peered":"2023-11-06T02:11:31.103668+0000","last_clean":"2023-11-06T02:11:31.103668+0000","last_became_active":"2023-11-06T02:11:30.128625+0000","last_became_peered":"2023-11-06T02:11:30.128625+0000","last_unstale":"2023-11-06T02:11:31.103668+0000","last_undegraded":"2023-11-06T02:11:31.103668+0000","last_fullsized":"2023-11-06T02:11:31.103668+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:55:14.474895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.112080+0000","last_change":"2023-11-06T02:11:29.757340+0000","last_active":"2023-11-06T02:11:31.112080+0000","last_peered":"2023-11-06T02:11:31.112080+0000","last_clean":"2023-11-06T02:11:31.112080+0000","last_became_active":"2023-11-06T02:11:29.756428+0000","last_became_peered":"2023-11-06T02:11:29.756428+0000","last_unstale":"2023-11-06T02:11:31.112080+0000","last_undegraded":"2023-11-06T02:11:31.112080+0000","last_fullsized":"2023-11-06T02:11:31.112080+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:30:05.695898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.103441+0000","last_change":"2023-11-06T02:11:30.146047+0000","last_active":"2023-11-06T02:11:31.103441+0000","last_peered":"2023-11-06T02:11:31.103441+0000","last_clean":"2023-11-06T02:11:31.103441+0000","last_became_active":"2023-11-06T02:11:30.144992+0000","last_became_peered":"2023-11-06T02:11:30.144992+0000","last_unstale":"2023-11-06T02:11:31.103441+0000","last_undegraded":"2023-11-06T02:11:31.103441+0000","last_fullsized":"2023-11-06T02:11:31.103441+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:10:19.972522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.109723+0000","last_change":"2023-11-06T02:11:29.741530+0000","last_active":"2023-11-06T02:11:31.109723+0000","last_peered":"2023-11-06T02:11:31.109723+0000","last_clean":"2023-11-06T02:11:31.109723+0000","last_became_active":"2023-11-06T02:11:29.741231+0000","last_became_peered":"2023-11-06T02:11:29.741231+0000","last_unstale":"2023-11-06T02:11:31.109723+0000","last_undegraded":"2023-11-06T02:11:31.109723+0000","last_fullsized":"2023-11-06T02:11:31.109723+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:35:46.481307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.109591+0000","last_change":"2023-11-06T02:11:30.154647+0000","last_active":"2023-11-06T02:11:31.109591+0000","last_peered":"2023-11-06T02:11:31.109591+0000","last_clean":"2023-11-06T02:11:31.109591+0000","last_became_active":"2023-11-06T02:11:30.154157+0000","last_became_peered":"2023-11-06T02:11:30.154157+0000","last_unstale":"2023-11-06T02:11:31.109591+0000","last_undegraded":"2023-11-06T02:11:31.109591+0000","last_fullsized":"2023-11-06T02:11:31.109591+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:20:26.119496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.108500+0000","last_change":"2023-11-06T02:11:29.746489+0000","last_active":"2023-11-06T02:11:31.108500+0000","last_peered":"2023-11-06T02:11:31.108500+0000","last_clean":"2023-11-06T02:11:31.108500+0000","last_became_active":"2023-11-06T02:11:29.745659+0000","last_became_peered":"2023-11-06T02:11:29.745659+0000","last_unstale":"2023-11-06T02:11:31.108500+0000","last_undegraded":"2023-11-06T02:11:31.108500+0000","last_fullsized":"2023-11-06T02:11:31.108500+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:32:49.416758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.110013+0000","last_change":"2023-11-06T02:11:30.150778+0000","last_active":"2023-11-06T02:11:31.110013+0000","last_peered":"2023-11-06T02:11:31.110013+0000","last_clean":"2023-11-06T02:11:31.110013+0000","last_became_active":"2023-11-06T02:11:30.150466+0000","last_became_peered":"2023-11-06T02:11:30.150466+0000","last_unstale":"2023-11-06T02:11:31.110013+0000","last_undegraded":"2023-11-06T02:11:31.110013+0000","last_fullsized":"2023-11-06T02:11:31.110013+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:21:14.911044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.109609+0000","last_change":"2023-11-06T02:11:29.741756+0000","last_active":"2023-11-06T02:11:31.109609+0000","last_peered":"2023-11-06T02:11:31.109609+0000","last_clean":"2023-11-06T02:11:31.109609+0000","last_became_active":"2023-11-06T02:11:29.740943+0000","last_became_peered":"2023-11-06T02:11:29.740943+0000","last_unstale":"2023-11-06T02:11:31.109609+0000","last_undegraded":"2023-11-06T02:11:31.109609+0000","last_fullsized":"2023-11-06T02:11:31.109609+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:09:39.841128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.104524+0000","last_change":"2023-11-06T02:11:30.129762+0000","last_active":"2023-11-06T02:11:31.104524+0000","last_peered":"2023-11-06T02:11:31.104524+0000","last_clean":"2023-11-06T02:11:31.104524+0000","last_became_active":"2023-11-06T02:11:30.128986+0000","last_became_peered":"2023-11-06T02:11:30.128986+0000","last_unstale":"2023-11-06T02:11:31.104524+0000","last_undegraded":"2023-11-06T02:11:31.104524+0000","last_fullsized":"2023-11-06T02:11:31.104524+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:09:40.845771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.108187+0000","last_change":"2023-11-06T02:11:29.737268+0000","last_active":"2023-11-06T02:11:31.108187+0000","last_peered":"2023-11-06T02:11:31.108187+0000","last_clean":"2023-11-06T02:11:31.108187+0000","last_became_active":"2023-11-06T02:11:29.736861+0000","last_became_peered":"2023-11-06T02:11:29.736861+0000","last_unstale":"2023-11-06T02:11:31.108187+0000","last_undegraded":"2023-11-06T02:11:31.108187+0000","last_fullsized":"2023-11-06T02:11:31.108187+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:02:34.759621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":204,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-06T02:11:30.116450+0000","last_change":"2023-11-06T02:08:20.523427+0000","last_active":"2023-11-06T02:11:30.116450+0000","last_peered":"2023-11-06T02:11:30.116450+0000","last_clean":"2023-11-06T02:11:30.116450+0000","last_became_active":"2023-11-06T02:08:20.521089+0000","last_became_peered":"2023-11-06T02:08:20.521089+0000","last_unstale":"2023-11-06T02:11:30.116450+0000","last_undegraded":"2023-11-06T02:11:30.116450+0000","last_fullsized":"2023-11-06T02:11:30.116450+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:19.252348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:19.252348+0000","last_clean_scrub_stamp":"2023-11-06T02:08:19.252348+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:30:15.566397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":50,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-06T02:11:30.126367+0000","last_change":"2023-11-06T02:08:26.059502+0000","last_active":"2023-11-06T02:11:30.126367+0000","last_peered":"2023-11-06T02:11:30.126367+0000","last_clean":"2023-11-06T02:11:30.126367+0000","last_became_active":"2023-11-06T02:08:26.058612+0000","last_became_peered":"2023-11-06T02:08:26.058612+0000","last_unstale":"2023-11-06T02:11:30.126367+0000","last_undegraded":"2023-11-06T02:11:30.126367+0000","last_fullsized":"2023-11-06T02:11:30.126367+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:10.626021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.101110+0000","last_change":"2023-11-06T02:11:30.118128+0000","last_active":"2023-11-06T02:11:31.101110+0000","last_peered":"2023-11-06T02:11:31.101110+0000","last_clean":"2023-11-06T02:11:31.101110+0000","last_became_active":"2023-11-06T02:11:30.117841+0000","last_became_peered":"2023-11-06T02:11:30.117841+0000","last_unstale":"2023-11-06T02:11:31.101110+0000","last_undegraded":"2023-11-06T02:11:31.101110+0000","last_fullsized":"2023-11-06T02:11:31.101110+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:15:08.905482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"64'2","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.101165+0000","last_change":"2023-11-06T02:11:29.741726+0000","last_active":"2023-11-06T02:11:31.101165+0000","last_peered":"2023-11-06T02:11:31.101165+0000","last_clean":"2023-11-06T02:11:31.101165+0000","last_became_active":"2023-11-06T02:11:29.741177+0000","last_became_peered":"2023-11-06T02:11:29.741177+0000","last_unstale":"2023-11-06T02:11:31.101165+0000","last_undegraded":"2023-11-06T02:11:31.101165+0000","last_fullsized":"2023-11-06T02:11:31.101165+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:31:03.059992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":51,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-06T02:11:30.116219+0000","last_change":"2023-11-06T02:08:26.060406+0000","last_active":"2023-11-06T02:11:30.116219+0000","last_peered":"2023-11-06T02:11:30.116219+0000","last_clean":"2023-11-06T02:11:30.116219+0000","last_became_active":"2023-11-06T02:08:26.059262+0000","last_became_peered":"2023-11-06T02:08:26.059262+0000","last_unstale":"2023-11-06T02:11:30.116219+0000","last_undegraded":"2023-11-06T02:11:30.116219+0000","last_fullsized":"2023-11-06T02:11:30.116219+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:10.626021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.113354+0000","last_change":"2023-11-06T02:11:30.140402+0000","last_active":"2023-11-06T02:11:31.113354+0000","last_peered":"2023-11-06T02:11:31.113354+0000","last_clean":"2023-11-06T02:11:31.113354+0000","last_became_active":"2023-11-06T02:11:30.139775+0000","last_became_peered":"2023-11-06T02:11:30.139775+0000","last_unstale":"2023-11-06T02:11:31.113354+0000","last_undegraded":"2023-11-06T02:11:31.113354+0000","last_fullsized":"2023-11-06T02:11:31.113354+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:35:47.485950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.113643+0000","last_change":"2023-11-06T02:11:29.755533+0000","last_active":"2023-11-06T02:11:31.113643+0000","last_peered":"2023-11-06T02:11:31.113643+0000","last_clean":"2023-11-06T02:11:31.113643+0000","last_became_active":"2023-11-06T02:11:29.754882+0000","last_became_peered":"2023-11-06T02:11:29.754882+0000","last_unstale":"2023-11-06T02:11:31.113643+0000","last_undegraded":"2023-11-06T02:11:31.113643+0000","last_fullsized":"2023-11-06T02:11:31.113643+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:43:25.646510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":52,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-06T02:11:30.105820+0000","last_change":"2023-11-06T02:08:26.559477+0000","last_active":"2023-11-06T02:11:30.105820+0000","last_peered":"2023-11-06T02:11:30.105820+0000","last_clean":"2023-11-06T02:11:30.105820+0000","last_became_active":"2023-11-06T02:08:26.559202+0000","last_became_peered":"2023-11-06T02:08:26.559202+0000","last_unstale":"2023-11-06T02:11:30.105820+0000","last_undegraded":"2023-11-06T02:11:30.105820+0000","last_fullsized":"2023-11-06T02:11:30.105820+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:55:04.552404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.104253+0000","last_change":"2023-11-06T02:11:30.133136+0000","last_active":"2023-11-06T02:11:31.104253+0000","last_peered":"2023-11-06T02:11:31.104253+0000","last_clean":"2023-11-06T02:11:31.104253+0000","last_became_active":"2023-11-06T02:11:30.130225+0000","last_became_peered":"2023-11-06T02:11:30.130225+0000","last_unstale":"2023-11-06T02:11:31.104253+0000","last_undegraded":"2023-11-06T02:11:31.104253+0000","last_fullsized":"2023-11-06T02:11:31.104253+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:40:37.087268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.100983+0000","last_change":"2023-11-06T02:11:29.753904+0000","last_active":"2023-11-06T02:11:31.100983+0000","last_peered":"2023-11-06T02:11:31.100983+0000","last_clean":"2023-11-06T02:11:31.100983+0000","last_became_active":"2023-11-06T02:11:29.753666+0000","last_became_peered":"2023-11-06T02:11:29.753666+0000","last_unstale":"2023-11-06T02:11:31.100983+0000","last_undegraded":"2023-11-06T02:11:31.100983+0000","last_fullsized":"2023-11-06T02:11:31.100983+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:17:39.435255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":50,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-06T02:11:30.127033+0000","last_change":"2023-11-06T02:08:26.059546+0000","last_active":"2023-11-06T02:11:30.127033+0000","last_peered":"2023-11-06T02:11:30.127033+0000","last_clean":"2023-11-06T02:11:30.127033+0000","last_became_active":"2023-11-06T02:08:26.058806+0000","last_became_peered":"2023-11-06T02:08:26.058806+0000","last_unstale":"2023-11-06T02:11:30.127033+0000","last_undegraded":"2023-11-06T02:11:30.127033+0000","last_fullsized":"2023-11-06T02:11:30.127033+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:23.468510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.112110+0000","last_change":"2023-11-06T02:11:30.164812+0000","last_active":"2023-11-06T02:11:31.112110+0000","last_peered":"2023-11-06T02:11:31.112110+0000","last_clean":"2023-11-06T02:11:31.112110+0000","last_became_active":"2023-11-06T02:11:30.163440+0000","last_became_peered":"2023-11-06T02:11:30.163440+0000","last_unstale":"2023-11-06T02:11:31.112110+0000","last_undegraded":"2023-11-06T02:11:31.112110+0000","last_fullsized":"2023-11-06T02:11:31.112110+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:41:12.940684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"64'1","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.114379+0000","last_change":"2023-11-06T02:11:29.769511+0000","last_active":"2023-11-06T02:11:31.114379+0000","last_peered":"2023-11-06T02:11:31.114379+0000","last_clean":"2023-11-06T02:11:31.114379+0000","last_became_active":"2023-11-06T02:11:29.768855+0000","last_became_peered":"2023-11-06T02:11:29.768855+0000","last_unstale":"2023-11-06T02:11:31.114379+0000","last_undegraded":"2023-11-06T02:11:31.114379+0000","last_fullsized":"2023-11-06T02:11:31.114379+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:10:41.234197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":49,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-06T02:11:30.111378+0000","last_change":"2023-11-06T02:08:26.436463+0000","last_active":"2023-11-06T02:11:30.111378+0000","last_peered":"2023-11-06T02:11:30.111378+0000","last_clean":"2023-11-06T02:11:30.111378+0000","last_became_active":"2023-11-06T02:08:26.436075+0000","last_became_peered":"2023-11-06T02:08:26.436075+0000","last_unstale":"2023-11-06T02:11:30.111378+0000","last_undegraded":"2023-11-06T02:11:30.111378+0000","last_fullsized":"2023-11-06T02:11:30.111378+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:23.468510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.104888+0000","last_change":"2023-11-06T02:11:30.163619+0000","last_active":"2023-11-06T02:11:31.104888+0000","last_peered":"2023-11-06T02:11:31.104888+0000","last_clean":"2023-11-06T02:11:31.104888+0000","last_became_active":"2023-11-06T02:11:30.163060+0000","last_became_peered":"2023-11-06T02:11:30.163060+0000","last_unstale":"2023-11-06T02:11:31.104888+0000","last_undegraded":"2023-11-06T02:11:31.104888+0000","last_fullsized":"2023-11-06T02:11:31.104888+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:34:30.028107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"64'2","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.114533+0000","last_change":"2023-11-06T02:11:29.766067+0000","last_active":"2023-11-06T02:11:31.114533+0000","last_peered":"2023-11-06T02:11:31.114533+0000","last_clean":"2023-11-06T02:11:31.114533+0000","last_became_active":"2023-11-06T02:11:29.765630+0000","last_became_peered":"2023-11-06T02:11:29.765630+0000","last_unstale":"2023-11-06T02:11:31.114533+0000","last_undegraded":"2023-11-06T02:11:31.114533+0000","last_fullsized":"2023-11-06T02:11:31.114533+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:35:10.185303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":51,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-06T02:11:30.121485+0000","last_change":"2023-11-06T02:08:26.436607+0000","last_active":"2023-11-06T02:11:30.121485+0000","last_peered":"2023-11-06T02:11:30.121485+0000","last_clean":"2023-11-06T02:11:30.121485+0000","last_became_active":"2023-11-06T02:08:26.435924+0000","last_became_peered":"2023-11-06T02:08:26.435924+0000","last_unstale":"2023-11-06T02:11:30.121485+0000","last_undegraded":"2023-11-06T02:11:30.121485+0000","last_fullsized":"2023-11-06T02:11:30.121485+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:23.468510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.103761+0000","last_change":"2023-11-06T02:11:30.135530+0000","last_active":"2023-11-06T02:11:31.103761+0000","last_peered":"2023-11-06T02:11:31.103761+0000","last_clean":"2023-11-06T02:11:31.103761+0000","last_became_active":"2023-11-06T02:11:30.134786+0000","last_became_peered":"2023-11-06T02:11:30.134786+0000","last_unstale":"2023-11-06T02:11:31.103761+0000","last_undegraded":"2023-11-06T02:11:31.103761+0000","last_fullsized":"2023-11-06T02:11:31.103761+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:48:50.245921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"64'1","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.112435+0000","last_change":"2023-11-06T02:11:29.762390+0000","last_active":"2023-11-06T02:11:31.112435+0000","last_peered":"2023-11-06T02:11:31.112435+0000","last_clean":"2023-11-06T02:11:31.112435+0000","last_became_active":"2023-11-06T02:11:29.761673+0000","last_became_peered":"2023-11-06T02:11:29.761673+0000","last_unstale":"2023-11-06T02:11:31.112435+0000","last_undegraded":"2023-11-06T02:11:31.112435+0000","last_fullsized":"2023-11-06T02:11:31.112435+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:07:34.988240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":49,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-06T02:11:30.110990+0000","last_change":"2023-11-06T02:08:26.057076+0000","last_active":"2023-11-06T02:11:30.110990+0000","last_peered":"2023-11-06T02:11:30.110990+0000","last_clean":"2023-11-06T02:11:30.110990+0000","last_became_active":"2023-11-06T02:08:26.056419+0000","last_became_peered":"2023-11-06T02:08:26.056419+0000","last_unstale":"2023-11-06T02:11:30.110990+0000","last_undegraded":"2023-11-06T02:11:30.110990+0000","last_fullsized":"2023-11-06T02:11:30.110990+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:10.626021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.105222+0000","last_change":"2023-11-06T02:11:30.130327+0000","last_active":"2023-11-06T02:11:31.105222+0000","last_peered":"2023-11-06T02:11:31.105222+0000","last_clean":"2023-11-06T02:11:31.105222+0000","last_became_active":"2023-11-06T02:11:30.128754+0000","last_became_peered":"2023-11-06T02:11:30.128754+0000","last_unstale":"2023-11-06T02:11:31.105222+0000","last_undegraded":"2023-11-06T02:11:31.105222+0000","last_fullsized":"2023-11-06T02:11:31.105222+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:20:01.077907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.108631+0000","last_change":"2023-11-06T02:11:29.755079+0000","last_active":"2023-11-06T02:11:31.108631+0000","last_peered":"2023-11-06T02:11:31.108631+0000","last_clean":"2023-11-06T02:11:31.108631+0000","last_became_active":"2023-11-06T02:11:29.754680+0000","last_became_peered":"2023-11-06T02:11:29.754680+0000","last_unstale":"2023-11-06T02:11:31.108631+0000","last_undegraded":"2023-11-06T02:11:31.108631+0000","last_fullsized":"2023-11-06T02:11:31.108631+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:39:34.434686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":57,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-06T02:11:30.111388+0000","last_change":"2023-11-06T02:08:26.058596+0000","last_active":"2023-11-06T02:11:30.111388+0000","last_peered":"2023-11-06T02:11:30.111388+0000","last_clean":"2023-11-06T02:11:30.111388+0000","last_became_active":"2023-11-06T02:08:26.057015+0000","last_became_peered":"2023-11-06T02:08:26.057015+0000","last_unstale":"2023-11-06T02:11:30.111388+0000","last_undegraded":"2023-11-06T02:11:30.111388+0000","last_fullsized":"2023-11-06T02:11:30.111388+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:55:04.552404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574890,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28312,"kb_used_data":1360,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343528,"statfs":{"total":96636764160,"available":96607772672,"internally_reserved":0,"allocated":1392640,"data_stored":937394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.86199999999999999,"15min":0.85299999999999998},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.1200000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.91300000000000003,"15min":0.91100000000000003},"min":{"1min":0.68300000000000005,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.1020000000000001,"5min":1.506,"15min":1.506},"last":0.98999999999999999}]},{"osd":1,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.82699999999999996,"15min":0.84299999999999997},"min":{"1min":0.55300000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.0329999999999999,"5min":1.161,"15min":1.161},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.91200000000000003,"15min":0.89600000000000002},"min":{"1min":0.63600000000000001,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.1910000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.877}]},{"osd":2,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.96399999999999997,"15min":0.95699999999999996},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.2989999999999999,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":1.109},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.98699999999999999,"15min":0.996},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.5,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.78500000000000003}]},{"osd":3,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.88300000000000001,"15min":0.877},"min":{"1min":0.69099999999999995,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.1479999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":1.276},{"interface":"front","average":{"1min":1.012,"5min":1.0700000000000001,"15min":1.0740000000000001},"min":{"1min":0.78900000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.2230000000000001,"5min":1.526,"15min":1.526},"last":1.032}]},{"osd":4,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.95899999999999996,"15min":0.95799999999999996},"min":{"1min":0.64000000000000001,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":1.0640000000000001},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.0229999999999999,"15min":1.0369999999999999},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":1.4199999999999999}]},{"osd":5,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":1.0669999999999999,"15min":1.075},"min":{"1min":0.69999999999999996,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":1.244},{"interface":"front","average":{"1min":1.0780000000000001,"5min":1.117,"15min":1.129},"min":{"1min":0.73199999999999998,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.3700000000000001,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":1.1499999999999999}]},{"osd":6,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.196,"15min":1.2},"min":{"1min":0.83999999999999997,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.5009999999999999,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":1.2050000000000001},{"interface":"front","average":{"1min":1.048,"5min":1.131,"15min":1.149},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.331,"5min":1.74,"15min":1.74},"last":1.369}]}]},{"osd":6,"up_from":13,"seq":55834574889,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27720,"kb_used_data":768,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344120,"statfs":{"total":96636764160,"available":96608378880,"internally_reserved":0,"allocated":786432,"data_stored":346284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.52400000000000002,"5min":0.495,"15min":0.495},"max":{"1min":1.216,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":1.032},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.84099999999999997,"15min":0.81699999999999995},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.2370000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":1.1060000000000001}]},{"osd":1,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.86599999999999999,"15min":0.872},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.86799999999999999,"15min":0.872},"min":{"1min":0.63,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.2330000000000001,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":1.129}]},{"osd":2,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.877,"15min":0.89100000000000001},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":1.252},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.90400000000000003,"15min":0.92200000000000004},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.103,"5min":1.28,"15min":1.28},"last":0.96299999999999997}]},{"osd":3,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":0.95999999999999996,"15min":0.96799999999999997},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.3300000000000001,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":1.1830000000000001},{"interface":"front","average":{"1min":0.99399999999999999,"5min":0.99199999999999999,"15min":0.998},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.22,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":1.056}]},{"osd":4,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.95099999999999996,"15min":0.97899999999999998},"min":{"1min":0.59499999999999997,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.1639999999999999,"5min":1.474,"15min":1.474},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.99399999999999999,"15min":1.024},"min":{"1min":0.624,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.268,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.91300000000000003}]},{"osd":5,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.98399999999999999,"15min":1.0129999999999999},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.161,"5min":1.365,"15min":1.365},"last":1.079},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.0760000000000001,"15min":1.1020000000000001},"min":{"1min":0.68400000000000005,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.3939999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":1.228}]},{"osd":7,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":1.083,"15min":1.1060000000000001},"min":{"1min":0.76800000000000002,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.3580000000000001,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":1.204},{"interface":"front","average":{"1min":0.99399999999999999,"5min":1.095,"15min":1.141},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.425,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":1.1579999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574890,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27720,"kb_used_data":768,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344120,"statfs":{"total":96636764160,"available":96608378880,"internally_reserved":0,"allocated":786432,"data_stored":346272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.85599999999999998,"15min":0.83799999999999997},"min":{"1min":0.63300000000000001,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.2390000000000001,"5min":1.381,"15min":1.381},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.85699999999999998,"15min":0.84699999999999998},"min":{"1min":0.63100000000000001,"5min":0.503,"15min":0.503},"max":{"1min":1.3200000000000001,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":1.0780000000000001}]},{"osd":1,"last update":"Mon Nov 6 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.86599999999999999,"15min":0.85499999999999998},"min":{"1min":0.626,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.1699999999999999,"5min":1.331,"15min":1.331},"last":0.93500000000000005},{"interface":"front","average":{"1min":0.87,"5min":0.83599999999999997,"15min":0.82299999999999995},"min":{"1min":0.57199999999999995,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.177,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":1.0569999999999999}]},{"osd":2,"last update":"Mon Nov 6 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.93200000000000005,"15min":0.94099999999999995},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.2210000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.83199999999999996,"15min":0.80400000000000005},"min":{"1min":0.55800000000000005,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.323,"5min":1.377,"15min":1.377},"last":1.1020000000000001}]},{"osd":3,"last update":"Mon Nov 6 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":0.93200000000000005,"15min":0.92400000000000004},"min":{"1min":0.65900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.3580000000000001,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":1.0269999999999999},{"interface":"front","average":{"1min":1.0009999999999999,"5min":1.0409999999999999,"15min":1.0669999999999999},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.228,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":0.82899999999999996}]},{"osd":4,"last update":"Mon Nov 6 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.93100000000000005,"15min":0.92100000000000004},"min":{"1min":0.61299999999999999,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.2410000000000001,"5min":1.276,"15min":1.276},"last":1.1819999999999999},{"interface":"front","average":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.026},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.496,"5min":1.554,"15min":1.554},"last":0.65800000000000003}]},{"osd":6,"last update":"Mon Nov 6 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":1.0469999999999999,"15min":1.0609999999999999},"min":{"1min":0.67500000000000004,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":0.90900000000000003},{"interface":"front","average":{"1min":1.075,"5min":1.04,"15min":1.04},"min":{"1min":0.72999999999999998,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.98299999999999998}]},{"osd":7,"last update":"Mon Nov 6 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.083,"15min":1.0780000000000001},"min":{"1min":0.71599999999999997,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.5880000000000001,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":0.77800000000000002},{"interface":"front","average":{"1min":1.161,"5min":1.1499999999999999,"15min":1.153},"min":{"1min":0.92400000000000004,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.544,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":0.72099999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574890,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27728,"kb_used_data":776,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344112,"statfs":{"total":96636764160,"available":96608370688,"internally_reserved":0,"allocated":794624,"data_stored":347814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:11:22 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.85599999999999998,"15min":0.83499999999999996},"min":{"1min":0.67100000000000004,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.343,"5min":1.343,"15min":1.343},"last":0.98199999999999998},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.79300000000000004,"15min":0.77700000000000002},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.0169999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.80600000000000005}]},{"osd":1,"last update":"Mon Nov 6 02:11:22 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.86899999999999999,"15min":0.85199999999999998},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.083,"5min":1.462,"15min":1.462},"last":1.008},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.92400000000000004,"15min":0.90200000000000002},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.3580000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.83899999999999997}]},{"osd":2,"last update":"Mon Nov 6 02:11:22 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.97599999999999998,"15min":1.0049999999999999},"min":{"1min":0.69399999999999995,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.198,"5min":1.615,"15min":1.615},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.94099999999999995,"15min":0.93100000000000005},"min":{"1min":0.68000000000000005,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.3049999999999999,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":1.0309999999999999}]},{"osd":3,"last update":"Mon Nov 6 02:11:22 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.90200000000000002,"15min":0.91300000000000003},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.103,"5min":1.52,"15min":1.52},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.97799999999999998,"5min":1.04,"15min":1.0600000000000001},"min":{"1min":0.63900000000000001,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.2470000000000001,"5min":1.595,"15min":1.595},"last":0.95699999999999996}]},{"osd":5,"last update":"Mon Nov 6 02:11:22 2023","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":1.0329999999999999,"15min":1.0449999999999999},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.25,"5min":1.377,"15min":1.377},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.97999999999999998,"5min":1.0189999999999999,"15min":1.0429999999999999},"min":{"1min":0.67500000000000004,"5min":0.629,"15min":0.629},"max":{"1min":1.278,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":0.66400000000000003}]},{"osd":6,"last update":"Mon Nov 6 02:11:22 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":1.052,"15min":1.0800000000000001},"min":{"1min":0.68000000000000005,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.2470000000000001,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":1.071,"5min":1.157,"15min":1.1839999999999999},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.4079999999999999,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":0.88900000000000001}]},{"osd":7,"last update":"Mon Nov 6 02:11:22 2023","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":1.0720000000000001,"15min":1.0860000000000001},"min":{"1min":0.748,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.2849999999999999,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":1.0529999999999999},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.1859999999999999,"15min":1.222},"min":{"1min":0.80000000000000004,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.3280000000000001,"5min":1.7410000000000001,"15min":1.7410000000000001},"last":0.85799999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574889,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31812,"kb_used_data":764,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340028,"statfs":{"total":96636764160,"available":96604188672,"internally_reserved":0,"allocated":782336,"data_stored":346250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.63900000000000001,"15min":0.64500000000000002},"min":{"1min":0.36499999999999999,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.95999999999999996,"5min":1.079,"15min":1.079},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.626,"5min":0.625,"15min":0.625},"min":{"1min":0.34599999999999997,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.93899999999999995,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.59199999999999997}]},{"osd":1,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.61899999999999999,"15min":0.627},"min":{"1min":0.27800000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.86799999999999999,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.72599999999999998,"15min":0.73099999999999998},"min":{"1min":0.307,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.0980000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.66600000000000004}]},{"osd":2,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.71399999999999997,"15min":0.72799999999999998},"min":{"1min":0.374,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.81200000000000006,"15min":0.81999999999999995},"min":{"1min":0.39000000000000001,"5min":0.308,"15min":0.308},"max":{"1min":1.1899999999999999,"5min":1.405,"15min":1.405},"last":0.84299999999999997}]},{"osd":4,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.92700000000000005,"15min":0.92800000000000005},"min":{"1min":0.65900000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.085,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.98099999999999998,"5min":0.97599999999999998,"15min":0.99099999999999999},"min":{"1min":0.67800000000000005,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.236,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":1.077}]},{"osd":5,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":1.004,"15min":1.02},"min":{"1min":0.70699999999999996,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":1.141},{"interface":"front","average":{"1min":1.0880000000000001,"5min":1.0580000000000001,"15min":1.0740000000000001},"min":{"1min":0.74099999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.4330000000000001,"5min":2.21,"15min":2.21},"last":1.1220000000000001}]},{"osd":6,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":1.0229999999999999,"15min":1.0249999999999999},"min":{"1min":0.76900000000000002,"5min":0.439,"15min":0.439},"max":{"1min":1.3799999999999999,"5min":1.395,"15min":1.395},"last":1.2989999999999999},{"interface":"front","average":{"1min":1.0880000000000001,"5min":1.0600000000000001,"15min":1.0680000000000001},"min":{"1min":0.71699999999999997,"5min":0.375,"15min":0.375},"max":{"1min":1.4319999999999999,"5min":1.554,"15min":1.554},"last":0.999}]},{"osd":7,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.115,"15min":1.127},"min":{"1min":0.76300000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.5549999999999999,"5min":2.2829999999999999,"15min":2.2829999999999999},"last":1.266},{"interface":"front","average":{"1min":1.171,"5min":1.1080000000000001,"15min":1.107},"min":{"1min":0.92200000000000004,"5min":0.497,"15min":0.497},"max":{"1min":1.504,"5min":1.504,"15min":1.504},"last":1.218}]}]},{"osd":2,"up_from":13,"seq":55834574889,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31812,"kb_used_data":764,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340028,"statfs":{"total":96636764160,"available":96604188672,"internally_reserved":0,"allocated":782336,"data_stored":346250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:11:20 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.53900000000000003,"15min":0.51200000000000001},"min":{"1min":0.48199999999999998,"5min":0.252,"15min":0.252},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.59099999999999997,"15min":0.55600000000000005},"min":{"1min":0.497,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":0.502}]},{"osd":1,"last update":"Mon Nov 6 02:11:20 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.60599999999999998,"15min":0.57499999999999996},"min":{"1min":0.44700000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.48,"5min":1.48,"15min":1.48},"last":0.623},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.64700000000000002,"15min":0.59399999999999997},"min":{"1min":0.45200000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.7230000000000001,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":0.75700000000000001}]},{"osd":3,"last update":"Mon Nov 6 02:11:20 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.65400000000000003,"15min":0.61599999999999999},"min":{"1min":0.58699999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.93200000000000005,"5min":1.155,"15min":1.155},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.69799999999999995,"15min":0.66000000000000003},"min":{"1min":0.52500000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.0960000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.58399999999999996}]},{"osd":4,"last update":"Mon Nov 6 02:11:20 2023","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":0.85799999999999998,"15min":0.81799999999999995},"min":{"1min":0.77500000000000002,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.89800000000000002},{"interface":"front","average":{"1min":1.0369999999999999,"5min":0.93600000000000005,"15min":0.92200000000000004},"min":{"1min":0.755,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.395,"5min":1.395,"15min":1.395},"last":0.95099999999999996}]},{"osd":5,"last update":"Mon Nov 6 02:11:20 2023","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":0.96199999999999997,"15min":0.94899999999999995},"min":{"1min":0.74099999999999999,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.7729999999999999,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":1.0189999999999999,"5min":0.95699999999999996,"15min":0.94099999999999995},"min":{"1min":0.67200000000000004,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.615,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.97099999999999997}]},{"osd":6,"last update":"Mon Nov 6 02:11:20 2023","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":0.96899999999999997,"15min":0.94799999999999995},"min":{"1min":0.79200000000000004,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":0.91700000000000004},{"interface":"front","average":{"1min":1.087,"5min":1.0129999999999999,"15min":1.002},"min":{"1min":0.81499999999999995,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.675,"5min":1.677,"15min":1.677},"last":1.0369999999999999}]},{"osd":7,"last update":"Mon Nov 6 02:11:20 2023","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":1.0089999999999999,"15min":0.98599999999999999},"min":{"1min":0.72499999999999998,"5min":0.622,"15min":0.622},"max":{"1min":1.659,"5min":1.659,"15min":1.659},"last":0.98699999999999999},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.0620000000000001,"15min":1.048},"min":{"1min":0.89300000000000002,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.4650000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":1.1160000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574890,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31840,"kb_used_data":792,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340000,"statfs":{"total":96636764160,"available":96604160000,"internally_reserved":0,"allocated":811008,"data_stored":348578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:11:22 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.63700000000000001,"15min":0.629},"min":{"1min":0.28699999999999998,"5min":0.224,"15min":0.224},"max":{"1min":1.603,"5min":1.603,"15min":1.603},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.67400000000000004,"15min":0.65900000000000003},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.76900000000000002}]},{"osd":2,"last update":"Mon Nov 6 02:11:22 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.63700000000000001,"15min":0.627},"min":{"1min":0.33600000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.649,"5min":1.649,"15min":1.649},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.755,"15min":0.748},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.1919999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.73699999999999999}]},{"osd":3,"last update":"Mon Nov 6 02:11:22 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.79500000000000004,"15min":0.80000000000000004},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.2350000000000001,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.98199999999999998},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.80800000000000005,"15min":0.81000000000000005},"min":{"1min":0.41299999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.5,"5min":1.8149999999999999,"15min":1.8149999999999999},"last":0.93999999999999995}]},{"osd":4,"last update":"Mon Nov 6 02:11:22 2023","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.97699999999999998,"15min":0.98699999999999999},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.5780000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":1.0660000000000001},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.98099999999999998,"15min":0.98899999999999999},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.4099999999999999,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":1.1799999999999999}]},{"osd":5,"last update":"Mon Nov 6 02:11:22 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":0.97499999999999998,"15min":0.96899999999999997},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.718,"5min":1.718,"15min":1.718},"last":1.036},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.069,"15min":1.075},"min":{"1min":0.66500000000000004,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.403,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":1.141}]},{"osd":6,"last update":"Mon Nov 6 02:11:22 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.0920000000000001,"15min":1.099},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.6859999999999999,"5min":1.742,"15min":1.742},"last":1.2070000000000001},{"interface":"front","average":{"1min":1.0700000000000001,"5min":1.0669999999999999,"15min":1.073},"min":{"1min":0.64400000000000002,"5min":0.629,"15min":0.629},"max":{"1min":1.6339999999999999,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":1.0960000000000001}]},{"osd":7,"last update":"Mon Nov 6 02:11:22 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.115,"15min":1.119},"min":{"1min":0.71499999999999997,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.534,"5min":1.534,"15min":1.534},"last":1.2310000000000001},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.1499999999999999,"15min":1.157},"min":{"1min":0.68700000000000006,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.7629999999999999,"5min":1.893,"15min":1.893},"last":1.1160000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574890,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32388,"kb_used_data":1340,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339452,"statfs":{"total":96636764160,"available":96603598848,"internally_reserved":0,"allocated":1372160,"data_stored":935599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 02:11:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.33700000000000002,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64000000000000001,"15min":0.63700000000000001},"min":{"1min":0.38700000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.93400000000000005,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.65400000000000003}]},{"osd":2,"last update":"Mon Nov 6 02:11:25 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.67100000000000004,"15min":0.66700000000000004},"min":{"1min":0.505,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.67800000000000005,"15min":0.67600000000000005},"min":{"1min":0.35199999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.95599999999999996,"5min":1.141,"15min":1.141},"last":0.83399999999999996}]},{"osd":3,"last update":"Mon Nov 6 02:11:25 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.64000000000000001,"15min":0.63200000000000001},"min":{"1min":0.376,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.0600000000000001,"5min":1.429,"15min":1.429},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.753,"15min":0.73799999999999999},"min":{"1min":0.44,"5min":0.317,"15min":0.317},"max":{"1min":1.1299999999999999,"5min":1.25,"15min":1.25},"last":1.141}]},{"osd":4,"last update":"Mon Nov 6 02:11:25 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.85599999999999998,"15min":0.83599999999999997},"min":{"1min":0.59499999999999997,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.1579999999999999,"5min":1.341,"15min":1.341},"last":1.0640000000000001},{"interface":"front","average":{"1min":0.995,"5min":0.94299999999999995,"15min":0.93899999999999995},"min":{"1min":0.65100000000000002,"5min":0.5,"15min":0.5},"max":{"1min":1.3600000000000001,"5min":1.387,"15min":1.387},"last":1.016}]},{"osd":5,"last update":"Mon Nov 6 02:11:25 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.94199999999999995,"15min":0.94599999999999995},"min":{"1min":0.63300000000000001,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.119,"5min":1.339,"15min":1.339},"last":0.97799999999999998},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.89700000000000002,"15min":0.88},"min":{"1min":0.72899999999999998,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.325,"5min":1.448,"15min":1.448},"last":1.2390000000000001}]},{"osd":6,"last update":"Mon Nov 6 02:11:25 2023","interfaces":[{"interface":"back","average":{"1min":1.0960000000000001,"5min":1.0409999999999999,"15min":1.038},"min":{"1min":0.66700000000000004,"5min":0.625,"15min":0.625},"max":{"1min":1.573,"5min":1.589,"15min":1.589},"last":1.268},{"interface":"front","average":{"1min":1.0309999999999999,"5min":1.0169999999999999,"15min":1.0049999999999999},"min":{"1min":0.67600000000000005,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.3899999999999999,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":1.1060000000000001}]},{"osd":7,"last update":"Mon Nov 6 02:11:25 2023","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":1.008,"15min":1.0149999999999999},"min":{"1min":0.71299999999999997,"5min":0.504,"15min":0.504},"max":{"1min":1.341,"5min":1.421,"15min":1.421},"last":1.1839999999999999},{"interface":"front","average":{"1min":0.98599999999999999,"5min":0.98899999999999999,"15min":1.002},"min":{"1min":0.69899999999999995,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.3029999999999999,"5min":1.556,"15min":1.556},"last":0.93500000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T02:11:38.758 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T02:11:38.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:38.893+0000 7ffa65c97640 1 Processor -- start 2023-11-06T02:11:38.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:38.897+0000 7ffa65c97640 1 -- start start 2023-11-06T02:11:38.896 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:38.897+0000 7ffa65c97640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7ffa600cda70 0x7ffa60161650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:38.896 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:38.897+0000 7ffa65c97640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa600d17c0 con 0x7ffa600b1f90 2023-11-06T02:11:38.896 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:38.897+0000 7ffa65c97640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa600ce6c0 con 0x7ffa600714a0 2023-11-06T02:11:38.896 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:38.897+0000 7ffa65c97640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa600cdf20 con 0x7ffa600cda70 2023-11-06T02:11:38.896 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:38.897+0000 7ffa5f7fe640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7ffa600b1f90 0x7ffa600d1c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:42110/0 (socket says 172.21.15.49:42110) 2023-11-06T02:11:38.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:38.897+0000 7ffa5f7fe640 1 -- 172.21.15.49:0/3450934615 learned_addr learned my addr 172.21.15.49:0/3450934615 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:11:38.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:38.897+0000 7ffa5effd640 1 --2- 172.21.15.49:0/3450934615 >> v2:172.21.15.150:3300/0 conn(0x7ffa600cda70 0x7ffa60161650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:38.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:38.897+0000 7ffa5e7fc640 1 -- 172.21.15.49:0/3450934615 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 542663598 0 0) 0x7ffa600d17c0 con 0x7ffa600b1f90 2023-11-06T02:11:38.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:38.897+0000 7ffa5e7fc640 1 -- 172.21.15.49:0/3450934615 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa48003530 con 0x7ffa600b1f90 2023-11-06T02:11:38.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:38.897+0000 7ffa5e7fc640 1 -- 172.21.15.49:0/3450934615 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2164836982 0 0) 0x7ffa48003530 con 0x7ffa600b1f90 2023-11-06T02:11:38.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:38.897+0000 7ffa5e7fc640 1 -- 172.21.15.49:0/3450934615 >> v2:172.21.15.150:3300/0 conn(0x7ffa600cda70 msgr2=0x7ffa60161650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:38.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:38.897+0000 7ffa5e7fc640 1 --2- 172.21.15.49:0/3450934615 >> v2:172.21.15.150:3300/0 conn(0x7ffa600cda70 0x7ffa60161650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:38.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:38.897+0000 7ffa5e7fc640 1 -- 172.21.15.49:0/3450934615 >> v1:172.21.15.150:6789/0 conn(0x7ffa600714a0 legacy=0x7ffa600718b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:38.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:38.897+0000 7ffa5e7fc640 1 -- 172.21.15.49:0/3450934615 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa600d2360 con 0x7ffa600b1f90 2023-11-06T02:11:38.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:38.897+0000 7ffa5e7fc640 1 -- 172.21.15.49:0/3450934615 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7ffa50001680 con 0x7ffa600b1f90 2023-11-06T02:11:38.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:38.897+0000 7ffa5e7fc640 1 -- 172.21.15.49:0/3450934615 >> v1:172.21.15.49:6789/0 conn(0x7ffa600b1f90 legacy=0x7ffa600d1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:38.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:38.897+0000 7ffa5e7fc640 1 --2- 172.21.15.49:0/3450934615 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffa48003980 0x7ffa48003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:38.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:38.897+0000 7ffa5e7fc640 1 --2- 172.21.15.49:0/3450934615 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa480047d0 0x7ffa48004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:38.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:38.897+0000 7ffa5e7fc640 1 --2- 172.21.15.49:0/3450934615 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffa48005190 0x7ffa4800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:38.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:38.897+0000 7ffa5e7fc640 1 -- 172.21.15.49:0/3450934615 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa600cdf20 con 0x7ffa480047d0 2023-11-06T02:11:38.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:38.897+0000 7ffa5e7fc640 1 -- 172.21.15.49:0/3450934615 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4800c030 con 0x7ffa48003980 2023-11-06T02:11:38.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:38.897+0000 7ffa5e7fc640 1 -- 172.21.15.49:0/3450934615 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4800c1f0 con 0x7ffa48005190 2023-11-06T02:11:38.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:38.897+0000 7ffa5effd640 1 --2- 172.21.15.49:0/3450934615 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa480047d0 0x7ffa48004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:38.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:38.897+0000 7ffa5f7fe640 1 --2- 172.21.15.49:0/3450934615 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffa48003980 0x7ffa48003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:38.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:38.897+0000 7ffa5ffff640 1 --2- 172.21.15.49:0/3450934615 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffa48005190 0x7ffa4800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:38.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:38.897+0000 7ffa5effd640 1 -- 172.21.15.49:0/3450934615 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffa48005190 msgr2=0x7ffa4800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:38.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:38.897+0000 7ffa5effd640 1 --2- 172.21.15.49:0/3450934615 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffa48005190 0x7ffa4800b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:38.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:38.897+0000 7ffa5effd640 1 -- 172.21.15.49:0/3450934615 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffa48003980 msgr2=0x7ffa48003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:38.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:38.897+0000 7ffa5effd640 1 --2- 172.21.15.49:0/3450934615 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffa48003980 0x7ffa48003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:38.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:38.897+0000 7ffa5effd640 1 -- 172.21.15.49:0/3450934615 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa48003530 con 0x7ffa480047d0 2023-11-06T02:11:38.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:38.897+0000 7ffa5ffff640 1 -- 172.21.15.49:0/3450934615 reap_dead start 2023-11-06T02:11:38.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:38.897+0000 7ffa5f7fe640 1 --2- 172.21.15.49:0/3450934615 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffa48003980 0x7ffa48003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:11:38.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:38.901+0000 7ffa5effd640 1 --2- 172.21.15.49:0/3450934615 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa480047d0 0x7ffa48004c50 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7ffa54009ae0 tx=0x7ffa5401ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0e94fb3873dada3 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:38.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:38.901+0000 7ffa5e7fc640 1 -- 172.21.15.49:0/3450934615 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa540a5660 con 0x7ffa480047d0 2023-11-06T02:11:38.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:38.901+0000 7ffa5e7fc640 1 -- 172.21.15.49:0/3450934615 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa50001d80 con 0x7ffa480047d0 2023-11-06T02:11:38.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:38.901+0000 7ffa5e7fc640 1 -- 172.21.15.49:0/3450934615 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa540a5a00 con 0x7ffa480047d0 2023-11-06T02:11:38.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:38.901+0000 7ffa65c97640 1 -- 172.21.15.49:0/3450934615 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa480047d0 msgr2=0x7ffa48004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:38.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:38.901+0000 7ffa65c97640 1 --2- 172.21.15.49:0/3450934615 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa480047d0 0x7ffa48004c50 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7ffa54009ae0 tx=0x7ffa5401ca70 comp rx=0 tx=0).stop 2023-11-06T02:11:38.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:38.901+0000 7ffa65c97640 1 -- 172.21.15.49:0/3450934615 shutdown_connections 2023-11-06T02:11:38.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:38.901+0000 7ffa65c97640 1 --2- 172.21.15.49:0/3450934615 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa480047d0 0x7ffa48004c50 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:38.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:38.901+0000 7ffa65c97640 1 -- 172.21.15.49:0/3450934615 >> 172.21.15.49:0/3450934615 conn(0x7ffa600d2aa0 msgr2=0x7ffa600d2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:38.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:38.901+0000 7ffa65c97640 1 -- 172.21.15.49:0/3450934615 shutdown_connections 2023-11-06T02:11:38.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:38.901+0000 7ffa65c97640 1 -- 172.21.15.49:0/3450934615 wait complete. 2023-11-06T02:11:38.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:38.901+0000 7ffa65c97640 1 Processor -- start 2023-11-06T02:11:38.908 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:38.901+0000 7ffa65c97640 1 -- start start 2023-11-06T02:11:38.908 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:38.901+0000 7ffa65c97640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffa480047d0 0x7ffa600cab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:38.908 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:38.901+0000 7ffa65c97640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffa600cc250 0x7ffa601f21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:38.908 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:38.901+0000 7ffa65c97640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa6015ab90 0x7ffa6015b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:38.908 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:38.901+0000 7ffa65c97640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa6015b550 con 0x7ffa6015ab90 2023-11-06T02:11:38.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:38.901+0000 7ffa65c97640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa601f26e0 con 0x7ffa600cc250 2023-11-06T02:11:38.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:38.901+0000 7ffa65c97640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa600cb230 con 0x7ffa480047d0 2023-11-06T02:11:38.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:38.901+0000 7ffa5ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa6015ab90 0x7ffa6015b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:38.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:38.901+0000 7ffa5ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa6015ab90 0x7ffa6015b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49372/0 (socket says 172.21.15.49:49372) 2023-11-06T02:11:38.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:38.901+0000 7ffa5ffff640 1 -- 172.21.15.49:0/1557518518 learned_addr learned my addr 172.21.15.49:0/1557518518 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:38.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:38.901+0000 7ffa5effd640 1 --2- 172.21.15.49:0/1557518518 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffa600cc250 0x7ffa601f21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:38.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:38.901+0000 7ffa5f7fe640 1 --2- 172.21.15.49:0/1557518518 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffa480047d0 0x7ffa600cab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:38.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:38.905+0000 7ffa5ffff640 1 -- 172.21.15.49:0/1557518518 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffa480047d0 msgr2=0x7ffa600cab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:38.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:38.905+0000 7ffa5ffff640 1 --2- 172.21.15.49:0/1557518518 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffa480047d0 0x7ffa600cab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:38.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:38.905+0000 7ffa5ffff640 1 -- 172.21.15.49:0/1557518518 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffa600cc250 msgr2=0x7ffa601f21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:38.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:38.905+0000 7ffa5ffff640 1 --2- 172.21.15.49:0/1557518518 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffa600cc250 0x7ffa601f21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:38.911 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:38.905+0000 7ffa5ffff640 1 -- 172.21.15.49:0/1557518518 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa600cb510 con 0x7ffa6015ab90 2023-11-06T02:11:38.911 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:38.905+0000 7ffa5ffff640 1 --2- 172.21.15.49:0/1557518518 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa6015ab90 0x7ffa6015b010 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7ffa6006c560 tx=0x7ffa5800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:38.911 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:38.905+0000 7ffa47fff640 1 -- 172.21.15.49:0/1557518518 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa58003b80 con 0x7ffa6015ab90 2023-11-06T02:11:38.911 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:38.905+0000 7ffa47fff640 1 -- 172.21.15.49:0/1557518518 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa58003ce0 con 0x7ffa6015ab90 2023-11-06T02:11:38.911 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:38.905+0000 7ffa47fff640 1 -- 172.21.15.49:0/1557518518 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa580052b0 con 0x7ffa6015ab90 2023-11-06T02:11:38.911 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:38.905+0000 7ffa65c97640 1 -- 172.21.15.49:0/1557518518 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa600cb7f0 con 0x7ffa6015ab90 2023-11-06T02:11:38.912 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:38.905+0000 7ffa65c97640 1 -- 172.21.15.49:0/1557518518 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa601600d0 con 0x7ffa6015ab90 2023-11-06T02:11:38.912 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:38.905+0000 7ffa65c97640 1 -- 172.21.15.49:0/1557518518 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa60074040 con 0x7ffa6015ab90 2023-11-06T02:11:38.912 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:38.905+0000 7ffa47fff640 1 -- 172.21.15.49:0/1557518518 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7ffa58005410 con 0x7ffa6015ab90 2023-11-06T02:11:38.912 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:38.905+0000 7ffa47fff640 1 --2- 172.21.15.49:0/1557518518 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ffa4c065a90 0x7ffa4c067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:38.912 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:38.905+0000 7ffa47fff640 1 -- 172.21.15.49:0/1557518518 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ffa58082c50 con 0x7ffa6015ab90 2023-11-06T02:11:38.913 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:38.905+0000 7ffa5f7fe640 1 --2- 172.21.15.49:0/1557518518 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ffa4c065a90 0x7ffa4c067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:38.913 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:38.905+0000 7ffa5f7fe640 1 --2- 172.21.15.49:0/1557518518 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ffa4c065a90 0x7ffa4c067f50 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7ffa50004880 tx=0x7ffa50001440 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:38.913 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:38.909+0000 7ffa47fff640 1 -- 172.21.15.49:0/1557518518 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7ffa58050570 con 0x7ffa6015ab90 2023-11-06T02:11:39.063 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.065+0000 7ffa65c97640 1 -- 172.21.15.49:0/1557518518 --> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ffa600ce3e0 con 0x7ffa4c065a90 2023-11-06T02:11:39.087 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.089+0000 7ffa47fff640 1 -- 172.21.15.49:0/1557518518 <== mgr.4100 v2:172.21.15.150:6832/38104 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+309769 (secure 0 0 0) 0x7ffa500056e0 con 0x7ffa4c065a90 2023-11-06T02:11:39.090 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:11:39.092 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2023-11-06T02:11:39.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.093+0000 7ffa65c97640 1 -- 172.21.15.49:0/1557518518 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ffa4c065a90 msgr2=0x7ffa4c067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:39.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.093+0000 7ffa65c97640 1 --2- 172.21.15.49:0/1557518518 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ffa4c065a90 0x7ffa4c067f50 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7ffa50004880 tx=0x7ffa50001440 comp rx=0 tx=0).stop 2023-11-06T02:11:39.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.093+0000 7ffa65c97640 1 -- 172.21.15.49:0/1557518518 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa6015ab90 msgr2=0x7ffa6015b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:39.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.093+0000 7ffa65c97640 1 --2- 172.21.15.49:0/1557518518 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa6015ab90 0x7ffa6015b010 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7ffa6006c560 tx=0x7ffa5800dc00 comp rx=0 tx=0).stop 2023-11-06T02:11:39.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.093+0000 7ffa65c97640 1 -- 172.21.15.49:0/1557518518 shutdown_connections 2023-11-06T02:11:39.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.093+0000 7ffa65c97640 1 --2- 172.21.15.49:0/1557518518 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ffa4c065a90 0x7ffa4c067f50 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:39.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.093+0000 7ffa65c97640 1 --2- 172.21.15.49:0/1557518518 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa6015ab90 0x7ffa6015b010 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:39.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.093+0000 7ffa65c97640 1 --2- 172.21.15.49:0/1557518518 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffa600cc250 0x7ffa601f21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:39.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.093+0000 7ffa65c97640 1 --2- 172.21.15.49:0/1557518518 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffa480047d0 0x7ffa600cab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:39.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.093+0000 7ffa65c97640 1 -- 172.21.15.49:0/1557518518 >> 172.21.15.49:0/1557518518 conn(0x7ffa600d2aa0 msgr2=0x7ffa6015ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:39.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.093+0000 7ffa65c97640 1 -- 172.21.15.49:0/1557518518 shutdown_connections 2023-11-06T02:11:39.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.093+0000 7ffa65c97640 1 -- 172.21.15.49:0/1557518518 wait complete. 2023-11-06T02:11:39.109 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":158,"stamp":"2023-11-06T02:11:37.241212+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":143,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":239332,"kb_used_data":7332,"kb_used_omap":0,"kb_used_meta":231936,"kb_avail":754735388,"statfs":{"total":773094113280,"available":772849037312,"internally_reserved":0,"allocated":7507968,"data_stored":3954441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":237502464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":69,"apply_latency_ms":69,"commit_latency_ns":69000000,"apply_latency_ns":69000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.525292"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.110923+0000","last_change":"2023-11-06T02:11:30.149757+0000","last_active":"2023-11-06T02:11:31.110923+0000","last_peered":"2023-11-06T02:11:31.110923+0000","last_clean":"2023-11-06T02:11:31.110923+0000","last_became_active":"2023-11-06T02:11:30.149443+0000","last_became_peered":"2023-11-06T02:11:30.149443+0000","last_unstale":"2023-11-06T02:11:31.110923+0000","last_undegraded":"2023-11-06T02:11:31.110923+0000","last_fullsized":"2023-11-06T02:11:31.110923+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:51:26.553337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.110206+0000","last_change":"2023-11-06T02:11:30.116131+0000","last_active":"2023-11-06T02:11:31.110206+0000","last_peered":"2023-11-06T02:11:31.110206+0000","last_clean":"2023-11-06T02:11:31.110206+0000","last_became_active":"2023-11-06T02:11:30.115617+0000","last_became_peered":"2023-11-06T02:11:30.115617+0000","last_unstale":"2023-11-06T02:11:31.110206+0000","last_undegraded":"2023-11-06T02:11:31.110206+0000","last_fullsized":"2023-11-06T02:11:31.110206+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:17:20.665225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.111516+0000","last_change":"2023-11-06T02:11:30.157604+0000","last_active":"2023-11-06T02:11:31.111516+0000","last_peered":"2023-11-06T02:11:31.111516+0000","last_clean":"2023-11-06T02:11:31.111516+0000","last_became_active":"2023-11-06T02:11:30.156940+0000","last_became_peered":"2023-11-06T02:11:30.156940+0000","last_unstale":"2023-11-06T02:11:31.111516+0000","last_undegraded":"2023-11-06T02:11:31.111516+0000","last_fullsized":"2023-11-06T02:11:31.111516+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:51:26.553337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.105126+0000","last_change":"2023-11-06T02:11:30.112102+0000","last_active":"2023-11-06T02:11:31.105126+0000","last_peered":"2023-11-06T02:11:31.105126+0000","last_clean":"2023-11-06T02:11:31.105126+0000","last_became_active":"2023-11-06T02:11:30.111394+0000","last_became_peered":"2023-11-06T02:11:30.111394+0000","last_unstale":"2023-11-06T02:11:31.105126+0000","last_undegraded":"2023-11-06T02:11:31.105126+0000","last_fullsized":"2023-11-06T02:11:31.105126+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:05:39.414334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.104320+0000","last_change":"2023-11-06T02:11:30.163104+0000","last_active":"2023-11-06T02:11:31.104320+0000","last_peered":"2023-11-06T02:11:31.104320+0000","last_clean":"2023-11-06T02:11:31.104320+0000","last_became_active":"2023-11-06T02:11:30.162247+0000","last_became_peered":"2023-11-06T02:11:30.162247+0000","last_unstale":"2023-11-06T02:11:31.104320+0000","last_undegraded":"2023-11-06T02:11:31.104320+0000","last_fullsized":"2023-11-06T02:11:31.104320+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:38:56.722411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.101491+0000","last_change":"2023-11-06T02:11:30.157107+0000","last_active":"2023-11-06T02:11:31.101491+0000","last_peered":"2023-11-06T02:11:31.101491+0000","last_clean":"2023-11-06T02:11:31.101491+0000","last_became_active":"2023-11-06T02:11:30.156817+0000","last_became_peered":"2023-11-06T02:11:30.156817+0000","last_unstale":"2023-11-06T02:11:31.101491+0000","last_undegraded":"2023-11-06T02:11:31.101491+0000","last_fullsized":"2023-11-06T02:11:31.101491+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:04:33.551496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.109328+0000","last_change":"2023-11-06T02:11:30.139772+0000","last_active":"2023-11-06T02:11:31.109328+0000","last_peered":"2023-11-06T02:11:31.109328+0000","last_clean":"2023-11-06T02:11:31.109328+0000","last_became_active":"2023-11-06T02:11:30.139447+0000","last_became_peered":"2023-11-06T02:11:30.139447+0000","last_unstale":"2023-11-06T02:11:31.109328+0000","last_undegraded":"2023-11-06T02:11:31.109328+0000","last_fullsized":"2023-11-06T02:11:31.109328+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:24:49.653824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.113270+0000","last_change":"2023-11-06T02:11:30.170784+0000","last_active":"2023-11-06T02:11:31.113270+0000","last_peered":"2023-11-06T02:11:31.113270+0000","last_clean":"2023-11-06T02:11:31.113270+0000","last_became_active":"2023-11-06T02:11:30.170151+0000","last_became_peered":"2023-11-06T02:11:30.170151+0000","last_unstale":"2023-11-06T02:11:31.113270+0000","last_undegraded":"2023-11-06T02:11:31.113270+0000","last_fullsized":"2023-11-06T02:11:31.113270+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:33:33.543401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.115008+0000","last_change":"2023-11-06T02:11:30.164470+0000","last_active":"2023-11-06T02:11:31.115008+0000","last_peered":"2023-11-06T02:11:31.115008+0000","last_clean":"2023-11-06T02:11:31.115008+0000","last_became_active":"2023-11-06T02:11:30.163980+0000","last_became_peered":"2023-11-06T02:11:30.163980+0000","last_unstale":"2023-11-06T02:11:31.115008+0000","last_undegraded":"2023-11-06T02:11:31.115008+0000","last_fullsized":"2023-11-06T02:11:31.115008+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:03:45.538000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.112589+0000","last_change":"2023-11-06T02:11:30.157152+0000","last_active":"2023-11-06T02:11:31.112589+0000","last_peered":"2023-11-06T02:11:31.112589+0000","last_clean":"2023-11-06T02:11:31.112589+0000","last_became_active":"2023-11-06T02:11:30.156508+0000","last_became_peered":"2023-11-06T02:11:30.156508+0000","last_unstale":"2023-11-06T02:11:31.112589+0000","last_undegraded":"2023-11-06T02:11:31.112589+0000","last_fullsized":"2023-11-06T02:11:31.112589+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:09:18.888901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.103369+0000","last_change":"2023-11-06T02:11:30.156115+0000","last_active":"2023-11-06T02:11:31.103369+0000","last_peered":"2023-11-06T02:11:31.103369+0000","last_clean":"2023-11-06T02:11:31.103369+0000","last_became_active":"2023-11-06T02:11:30.154626+0000","last_became_peered":"2023-11-06T02:11:30.154626+0000","last_unstale":"2023-11-06T02:11:31.103369+0000","last_undegraded":"2023-11-06T02:11:31.103369+0000","last_fullsized":"2023-11-06T02:11:31.103369+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:03:45.538000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.113617+0000","last_change":"2023-11-06T02:11:30.170299+0000","last_active":"2023-11-06T02:11:31.113617+0000","last_peered":"2023-11-06T02:11:31.113617+0000","last_clean":"2023-11-06T02:11:31.113617+0000","last_became_active":"2023-11-06T02:11:30.169828+0000","last_became_peered":"2023-11-06T02:11:30.169828+0000","last_unstale":"2023-11-06T02:11:31.113617+0000","last_undegraded":"2023-11-06T02:11:31.113617+0000","last_fullsized":"2023-11-06T02:11:31.113617+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:02:35.764264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.113828+0000","last_change":"2023-11-06T02:11:30.165102+0000","last_active":"2023-11-06T02:11:31.113828+0000","last_peered":"2023-11-06T02:11:31.113828+0000","last_clean":"2023-11-06T02:11:31.113828+0000","last_became_active":"2023-11-06T02:11:30.164268+0000","last_became_peered":"2023-11-06T02:11:30.164268+0000","last_unstale":"2023-11-06T02:11:31.113828+0000","last_undegraded":"2023-11-06T02:11:31.113828+0000","last_fullsized":"2023-11-06T02:11:31.113828+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:01:56.680559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.111929+0000","last_change":"2023-11-06T02:11:30.157292+0000","last_active":"2023-11-06T02:11:31.111929+0000","last_peered":"2023-11-06T02:11:31.111929+0000","last_clean":"2023-11-06T02:11:31.111929+0000","last_became_active":"2023-11-06T02:11:30.156366+0000","last_became_peered":"2023-11-06T02:11:30.156366+0000","last_unstale":"2023-11-06T02:11:31.111929+0000","last_undegraded":"2023-11-06T02:11:31.111929+0000","last_fullsized":"2023-11-06T02:11:31.111929+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:42:52.898859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.111608+0000","last_change":"2023-11-06T02:11:30.157086+0000","last_active":"2023-11-06T02:11:31.111608+0000","last_peered":"2023-11-06T02:11:31.111608+0000","last_clean":"2023-11-06T02:11:31.111608+0000","last_became_active":"2023-11-06T02:11:30.156219+0000","last_became_peered":"2023-11-06T02:11:30.156219+0000","last_unstale":"2023-11-06T02:11:31.111608+0000","last_undegraded":"2023-11-06T02:11:31.111608+0000","last_fullsized":"2023-11-06T02:11:31.111608+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:24:57.712407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.109357+0000","last_change":"2023-11-06T02:11:30.140958+0000","last_active":"2023-11-06T02:11:31.109357+0000","last_peered":"2023-11-06T02:11:31.109357+0000","last_clean":"2023-11-06T02:11:31.109357+0000","last_became_active":"2023-11-06T02:11:30.140693+0000","last_became_peered":"2023-11-06T02:11:30.140693+0000","last_unstale":"2023-11-06T02:11:31.109357+0000","last_undegraded":"2023-11-06T02:11:31.109357+0000","last_fullsized":"2023-11-06T02:11:31.109357+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:49:19.353757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.111129+0000","last_change":"2023-11-06T02:11:30.155485+0000","last_active":"2023-11-06T02:11:31.111129+0000","last_peered":"2023-11-06T02:11:31.111129+0000","last_clean":"2023-11-06T02:11:31.111129+0000","last_became_active":"2023-11-06T02:11:30.155257+0000","last_became_peered":"2023-11-06T02:11:30.155257+0000","last_unstale":"2023-11-06T02:11:31.111129+0000","last_undegraded":"2023-11-06T02:11:31.111129+0000","last_fullsized":"2023-11-06T02:11:31.111129+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:25:05.779932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.103482+0000","last_change":"2023-11-06T02:11:30.112842+0000","last_active":"2023-11-06T02:11:31.103482+0000","last_peered":"2023-11-06T02:11:31.103482+0000","last_clean":"2023-11-06T02:11:31.103482+0000","last_became_active":"2023-11-06T02:11:30.112115+0000","last_became_peered":"2023-11-06T02:11:30.112115+0000","last_unstale":"2023-11-06T02:11:31.103482+0000","last_undegraded":"2023-11-06T02:11:31.103482+0000","last_fullsized":"2023-11-06T02:11:31.103482+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:01:08.304387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.104026+0000","last_change":"2023-11-06T02:11:30.130439+0000","last_active":"2023-11-06T02:11:31.104026+0000","last_peered":"2023-11-06T02:11:31.104026+0000","last_clean":"2023-11-06T02:11:31.104026+0000","last_became_active":"2023-11-06T02:11:30.129162+0000","last_became_peered":"2023-11-06T02:11:30.129162+0000","last_unstale":"2023-11-06T02:11:31.104026+0000","last_undegraded":"2023-11-06T02:11:31.104026+0000","last_fullsized":"2023-11-06T02:11:31.104026+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:04:12.430683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.108538+0000","last_change":"2023-11-06T02:11:30.132754+0000","last_active":"2023-11-06T02:11:31.108538+0000","last_peered":"2023-11-06T02:11:31.108538+0000","last_clean":"2023-11-06T02:11:31.108538+0000","last_became_active":"2023-11-06T02:11:30.132455+0000","last_became_peered":"2023-11-06T02:11:30.132455+0000","last_unstale":"2023-11-06T02:11:31.108538+0000","last_undegraded":"2023-11-06T02:11:31.108538+0000","last_fullsized":"2023-11-06T02:11:31.108538+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:00:58.743785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.113956+0000","last_change":"2023-11-06T02:11:30.150441+0000","last_active":"2023-11-06T02:11:31.113956+0000","last_peered":"2023-11-06T02:11:31.113956+0000","last_clean":"2023-11-06T02:11:31.113956+0000","last_became_active":"2023-11-06T02:11:30.148953+0000","last_became_peered":"2023-11-06T02:11:30.148953+0000","last_unstale":"2023-11-06T02:11:31.113956+0000","last_undegraded":"2023-11-06T02:11:31.113956+0000","last_fullsized":"2023-11-06T02:11:31.113956+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:03:45.538000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.109268+0000","last_change":"2023-11-06T02:11:30.136621+0000","last_active":"2023-11-06T02:11:31.109268+0000","last_peered":"2023-11-06T02:11:31.109268+0000","last_clean":"2023-11-06T02:11:31.109268+0000","last_became_active":"2023-11-06T02:11:30.136359+0000","last_became_peered":"2023-11-06T02:11:30.136359+0000","last_unstale":"2023-11-06T02:11:31.109268+0000","last_undegraded":"2023-11-06T02:11:31.109268+0000","last_fullsized":"2023-11-06T02:11:31.109268+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:40:12.076675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.108358+0000","last_change":"2023-11-06T02:11:30.137159+0000","last_active":"2023-11-06T02:11:31.108358+0000","last_peered":"2023-11-06T02:11:31.108358+0000","last_clean":"2023-11-06T02:11:31.108358+0000","last_became_active":"2023-11-06T02:11:30.136573+0000","last_became_peered":"2023-11-06T02:11:30.136573+0000","last_unstale":"2023-11-06T02:11:31.108358+0000","last_undegraded":"2023-11-06T02:11:31.108358+0000","last_fullsized":"2023-11-06T02:11:31.108358+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:17:41.584737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.111204+0000","last_change":"2023-11-06T02:11:30.159127+0000","last_active":"2023-11-06T02:11:31.111204+0000","last_peered":"2023-11-06T02:11:31.111204+0000","last_clean":"2023-11-06T02:11:31.111204+0000","last_became_active":"2023-11-06T02:11:30.158553+0000","last_became_peered":"2023-11-06T02:11:30.158553+0000","last_unstale":"2023-11-06T02:11:31.111204+0000","last_undegraded":"2023-11-06T02:11:31.111204+0000","last_fullsized":"2023-11-06T02:11:31.111204+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:48:50.245921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.114679+0000","last_change":"2023-11-06T02:11:30.149808+0000","last_active":"2023-11-06T02:11:31.114679+0000","last_peered":"2023-11-06T02:11:31.114679+0000","last_clean":"2023-11-06T02:11:31.114679+0000","last_became_active":"2023-11-06T02:11:30.149088+0000","last_became_peered":"2023-11-06T02:11:30.149088+0000","last_unstale":"2023-11-06T02:11:31.114679+0000","last_undegraded":"2023-11-06T02:11:31.114679+0000","last_fullsized":"2023-11-06T02:11:31.114679+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:39:35.439329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.113708+0000","last_change":"2023-11-06T02:11:30.151270+0000","last_active":"2023-11-06T02:11:31.113708+0000","last_peered":"2023-11-06T02:11:31.113708+0000","last_clean":"2023-11-06T02:11:31.113708+0000","last_became_active":"2023-11-06T02:11:30.150004+0000","last_became_peered":"2023-11-06T02:11:30.150004+0000","last_unstale":"2023-11-06T02:11:31.113708+0000","last_undegraded":"2023-11-06T02:11:31.113708+0000","last_fullsized":"2023-11-06T02:11:31.113708+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:06:06.524182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.103621+0000","last_change":"2023-11-06T02:11:30.155003+0000","last_active":"2023-11-06T02:11:31.103621+0000","last_peered":"2023-11-06T02:11:31.103621+0000","last_clean":"2023-11-06T02:11:31.103621+0000","last_became_active":"2023-11-06T02:11:30.154370+0000","last_became_peered":"2023-11-06T02:11:30.154370+0000","last_unstale":"2023-11-06T02:11:31.103621+0000","last_undegraded":"2023-11-06T02:11:31.103621+0000","last_fullsized":"2023-11-06T02:11:31.103621+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:20:36.621184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.110334+0000","last_change":"2023-11-06T02:11:30.142104+0000","last_active":"2023-11-06T02:11:31.110334+0000","last_peered":"2023-11-06T02:11:31.110334+0000","last_clean":"2023-11-06T02:11:31.110334+0000","last_became_active":"2023-11-06T02:11:30.141733+0000","last_became_peered":"2023-11-06T02:11:30.141733+0000","last_unstale":"2023-11-06T02:11:31.110334+0000","last_undegraded":"2023-11-06T02:11:31.110334+0000","last_fullsized":"2023-11-06T02:11:31.110334+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:17:21.200734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.111719+0000","last_change":"2023-11-06T02:11:30.163281+0000","last_active":"2023-11-06T02:11:31.111719+0000","last_peered":"2023-11-06T02:11:31.111719+0000","last_clean":"2023-11-06T02:11:31.111719+0000","last_became_active":"2023-11-06T02:11:30.162419+0000","last_became_peered":"2023-11-06T02:11:30.162419+0000","last_unstale":"2023-11-06T02:11:31.111719+0000","last_undegraded":"2023-11-06T02:11:31.111719+0000","last_fullsized":"2023-11-06T02:11:31.111719+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:10:19.972522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.113122+0000","last_change":"2023-11-06T02:11:30.170656+0000","last_active":"2023-11-06T02:11:31.113122+0000","last_peered":"2023-11-06T02:11:31.113122+0000","last_clean":"2023-11-06T02:11:31.113122+0000","last_became_active":"2023-11-06T02:11:30.170084+0000","last_became_peered":"2023-11-06T02:11:30.170084+0000","last_unstale":"2023-11-06T02:11:31.113122+0000","last_undegraded":"2023-11-06T02:11:31.113122+0000","last_fullsized":"2023-11-06T02:11:31.113122+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:32:50.421400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.103195+0000","last_change":"2023-11-06T02:11:30.144938+0000","last_active":"2023-11-06T02:11:31.103195+0000","last_peered":"2023-11-06T02:11:31.103195+0000","last_clean":"2023-11-06T02:11:31.103195+0000","last_became_active":"2023-11-06T02:11:30.135978+0000","last_became_peered":"2023-11-06T02:11:30.135978+0000","last_unstale":"2023-11-06T02:11:31.103195+0000","last_undegraded":"2023-11-06T02:11:31.103195+0000","last_fullsized":"2023-11-06T02:11:31.103195+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:38:11.812067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.100930+0000","last_change":"2023-11-06T02:11:30.118519+0000","last_active":"2023-11-06T02:11:31.100930+0000","last_peered":"2023-11-06T02:11:31.100930+0000","last_clean":"2023-11-06T02:11:31.100930+0000","last_became_active":"2023-11-06T02:11:30.118363+0000","last_became_peered":"2023-11-06T02:11:30.118363+0000","last_unstale":"2023-11-06T02:11:31.100930+0000","last_undegraded":"2023-11-06T02:11:31.100930+0000","last_fullsized":"2023-11-06T02:11:31.100930+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:07:25.816920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.104604+0000","last_change":"2023-11-06T02:11:30.131660+0000","last_active":"2023-11-06T02:11:31.104604+0000","last_peered":"2023-11-06T02:11:31.104604+0000","last_clean":"2023-11-06T02:11:31.104604+0000","last_became_active":"2023-11-06T02:11:30.130727+0000","last_became_peered":"2023-11-06T02:11:30.130727+0000","last_unstale":"2023-11-06T02:11:31.104604+0000","last_undegraded":"2023-11-06T02:11:31.104604+0000","last_fullsized":"2023-11-06T02:11:31.104604+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:59:55.787496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"64'4","reported_seq":15,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.111077+0000","last_change":"2023-11-06T02:11:29.759917+0000","last_active":"2023-11-06T02:11:31.111077+0000","last_peered":"2023-11-06T02:11:31.111077+0000","last_clean":"2023-11-06T02:11:31.111077+0000","last_became_active":"2023-11-06T02:11:29.758884+0000","last_became_peered":"2023-11-06T02:11:29.758884+0000","last_unstale":"2023-11-06T02:11:31.111077+0000","last_undegraded":"2023-11-06T02:11:31.111077+0000","last_fullsized":"2023-11-06T02:11:31.111077+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:10:27.614929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.103877+0000","last_change":"2023-11-06T02:11:30.153818+0000","last_active":"2023-11-06T02:11:31.103877+0000","last_peered":"2023-11-06T02:11:31.103877+0000","last_clean":"2023-11-06T02:11:31.103877+0000","last_became_active":"2023-11-06T02:11:30.153100+0000","last_became_peered":"2023-11-06T02:11:30.153100+0000","last_unstale":"2023-11-06T02:11:31.103877+0000","last_undegraded":"2023-11-06T02:11:31.103877+0000","last_fullsized":"2023-11-06T02:11:31.103877+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:41:12.940684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"64'1","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.101653+0000","last_change":"2023-11-06T02:11:29.756830+0000","last_active":"2023-11-06T02:11:31.101653+0000","last_peered":"2023-11-06T02:11:31.101653+0000","last_clean":"2023-11-06T02:11:31.101653+0000","last_became_active":"2023-11-06T02:11:29.756537+0000","last_became_peered":"2023-11-06T02:11:29.756537+0000","last_unstale":"2023-11-06T02:11:31.101653+0000","last_undegraded":"2023-11-06T02:11:31.101653+0000","last_fullsized":"2023-11-06T02:11:31.101653+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:24:56.707764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.101564+0000","last_change":"2023-11-06T02:11:30.116801+0000","last_active":"2023-11-06T02:11:31.101564+0000","last_peered":"2023-11-06T02:11:31.101564+0000","last_clean":"2023-11-06T02:11:31.101564+0000","last_became_active":"2023-11-06T02:11:30.116554+0000","last_became_peered":"2023-11-06T02:11:30.116554+0000","last_unstale":"2023-11-06T02:11:31.101564+0000","last_undegraded":"2023-11-06T02:11:31.101564+0000","last_fullsized":"2023-11-06T02:11:31.101564+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:58:42.053934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"64'2","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.101817+0000","last_change":"2023-11-06T02:11:29.767654+0000","last_active":"2023-11-06T02:11:31.101817+0000","last_peered":"2023-11-06T02:11:31.101817+0000","last_clean":"2023-11-06T02:11:31.101817+0000","last_became_active":"2023-11-06T02:11:29.767300+0000","last_became_peered":"2023-11-06T02:11:29.767300+0000","last_unstale":"2023-11-06T02:11:31.101817+0000","last_undegraded":"2023-11-06T02:11:31.101817+0000","last_fullsized":"2023-11-06T02:11:31.101817+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:35:46.481307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.114771+0000","last_change":"2023-11-06T02:11:30.158337+0000","last_active":"2023-11-06T02:11:31.114771+0000","last_peered":"2023-11-06T02:11:31.114771+0000","last_clean":"2023-11-06T02:11:31.114771+0000","last_became_active":"2023-11-06T02:11:30.157712+0000","last_became_peered":"2023-11-06T02:11:30.157712+0000","last_unstale":"2023-11-06T02:11:31.114771+0000","last_undegraded":"2023-11-06T02:11:31.114771+0000","last_fullsized":"2023-11-06T02:11:31.114771+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:01:56.680559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"64'2","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.101210+0000","last_change":"2023-11-06T02:11:29.742162+0000","last_active":"2023-11-06T02:11:31.101210+0000","last_peered":"2023-11-06T02:11:31.101210+0000","last_clean":"2023-11-06T02:11:31.101210+0000","last_became_active":"2023-11-06T02:11:29.741491+0000","last_became_peered":"2023-11-06T02:11:29.741491+0000","last_unstale":"2023-11-06T02:11:31.101210+0000","last_undegraded":"2023-11-06T02:11:31.101210+0000","last_fullsized":"2023-11-06T02:11:31.101210+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:09:17.884259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.109510+0000","last_change":"2023-11-06T02:11:30.123394+0000","last_active":"2023-11-06T02:11:31.109510+0000","last_peered":"2023-11-06T02:11:31.109510+0000","last_clean":"2023-11-06T02:11:31.109510+0000","last_became_active":"2023-11-06T02:11:30.123113+0000","last_became_peered":"2023-11-06T02:11:30.123113+0000","last_unstale":"2023-11-06T02:11:31.109510+0000","last_undegraded":"2023-11-06T02:11:31.109510+0000","last_fullsized":"2023-11-06T02:11:31.109510+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:44:35.667968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"64'1","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.108580+0000","last_change":"2023-11-06T02:11:29.737724+0000","last_active":"2023-11-06T02:11:31.108580+0000","last_peered":"2023-11-06T02:11:31.108580+0000","last_clean":"2023-11-06T02:11:31.108580+0000","last_became_active":"2023-11-06T02:11:29.737189+0000","last_became_peered":"2023-11-06T02:11:29.737189+0000","last_unstale":"2023-11-06T02:11:31.108580+0000","last_undegraded":"2023-11-06T02:11:31.108580+0000","last_fullsized":"2023-11-06T02:11:31.108580+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:56:30.903511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.109891+0000","last_change":"2023-11-06T02:11:30.154541+0000","last_active":"2023-11-06T02:11:31.109891+0000","last_peered":"2023-11-06T02:11:31.109891+0000","last_clean":"2023-11-06T02:11:31.109891+0000","last_became_active":"2023-11-06T02:11:30.154171+0000","last_became_peered":"2023-11-06T02:11:30.154171+0000","last_unstale":"2023-11-06T02:11:31.109891+0000","last_undegraded":"2023-11-06T02:11:31.109891+0000","last_fullsized":"2023-11-06T02:11:31.109891+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:10:42.238840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"64'1","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.109508+0000","last_change":"2023-11-06T02:11:29.736013+0000","last_active":"2023-11-06T02:11:31.109508+0000","last_peered":"2023-11-06T02:11:31.109508+0000","last_clean":"2023-11-06T02:11:31.109508+0000","last_became_active":"2023-11-06T02:11:29.735524+0000","last_became_peered":"2023-11-06T02:11:29.735524+0000","last_unstale":"2023-11-06T02:11:31.109508+0000","last_undegraded":"2023-11-06T02:11:31.109508+0000","last_fullsized":"2023-11-06T02:11:31.109508+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:33:32.538758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.104107+0000","last_change":"2023-11-06T02:11:30.132676+0000","last_active":"2023-11-06T02:11:31.104107+0000","last_peered":"2023-11-06T02:11:31.104107+0000","last_clean":"2023-11-06T02:11:31.104107+0000","last_became_active":"2023-11-06T02:11:30.131459+0000","last_became_peered":"2023-11-06T02:11:30.131459+0000","last_unstale":"2023-11-06T02:11:31.104107+0000","last_undegraded":"2023-11-06T02:11:31.104107+0000","last_fullsized":"2023-11-06T02:11:31.104107+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:37:58.300261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.108633+0000","last_change":"2023-11-06T02:11:29.745651+0000","last_active":"2023-11-06T02:11:31.108633+0000","last_peered":"2023-11-06T02:11:31.108633+0000","last_clean":"2023-11-06T02:11:31.108633+0000","last_became_active":"2023-11-06T02:11:29.744713+0000","last_became_peered":"2023-11-06T02:11:29.744713+0000","last_unstale":"2023-11-06T02:11:31.108633+0000","last_undegraded":"2023-11-06T02:11:31.108633+0000","last_fullsized":"2023-11-06T02:11:31.108633+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:06:05.519539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.108513+0000","last_change":"2023-11-06T02:11:30.111938+0000","last_active":"2023-11-06T02:11:31.108513+0000","last_peered":"2023-11-06T02:11:31.108513+0000","last_clean":"2023-11-06T02:11:31.108513+0000","last_became_active":"2023-11-06T02:11:30.111649+0000","last_became_peered":"2023-11-06T02:11:30.111649+0000","last_unstale":"2023-11-06T02:11:31.108513+0000","last_undegraded":"2023-11-06T02:11:31.108513+0000","last_fullsized":"2023-11-06T02:11:31.108513+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:47:22.667065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.103894+0000","last_change":"2023-11-06T02:11:29.759956+0000","last_active":"2023-11-06T02:11:31.103894+0000","last_peered":"2023-11-06T02:11:31.103894+0000","last_clean":"2023-11-06T02:11:31.103894+0000","last_became_active":"2023-11-06T02:11:29.759238+0000","last_became_peered":"2023-11-06T02:11:29.759238+0000","last_unstale":"2023-11-06T02:11:31.103894+0000","last_undegraded":"2023-11-06T02:11:31.103894+0000","last_fullsized":"2023-11-06T02:11:31.103894+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:24:48.649181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.110472+0000","last_change":"2023-11-06T02:11:30.127321+0000","last_active":"2023-11-06T02:11:31.110472+0000","last_peered":"2023-11-06T02:11:31.110472+0000","last_clean":"2023-11-06T02:11:31.110472+0000","last_became_active":"2023-11-06T02:11:30.127128+0000","last_became_peered":"2023-11-06T02:11:30.127128+0000","last_unstale":"2023-11-06T02:11:31.110472+0000","last_undegraded":"2023-11-06T02:11:31.110472+0000","last_fullsized":"2023-11-06T02:11:31.110472+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:43:26.651152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.115134+0000","last_change":"2023-11-06T02:11:29.765154+0000","last_active":"2023-11-06T02:11:31.115134+0000","last_peered":"2023-11-06T02:11:31.115134+0000","last_clean":"2023-11-06T02:11:31.115134+0000","last_became_active":"2023-11-06T02:11:29.764470+0000","last_became_peered":"2023-11-06T02:11:29.764470+0000","last_unstale":"2023-11-06T02:11:31.115134+0000","last_undegraded":"2023-11-06T02:11:31.115134+0000","last_fullsized":"2023-11-06T02:11:31.115134+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:59:54.782853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.108609+0000","last_change":"2023-11-06T02:11:30.150007+0000","last_active":"2023-11-06T02:11:31.108609+0000","last_peered":"2023-11-06T02:11:31.108609+0000","last_clean":"2023-11-06T02:11:31.108609+0000","last_became_active":"2023-11-06T02:11:30.149680+0000","last_became_peered":"2023-11-06T02:11:30.149680+0000","last_unstale":"2023-11-06T02:11:31.108609+0000","last_undegraded":"2023-11-06T02:11:31.108609+0000","last_fullsized":"2023-11-06T02:11:31.108609+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:10:05.661777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.103973+0000","last_change":"2023-11-06T02:11:29.752256+0000","last_active":"2023-11-06T02:11:31.103973+0000","last_peered":"2023-11-06T02:11:31.103973+0000","last_clean":"2023-11-06T02:11:31.103973+0000","last_became_active":"2023-11-06T02:11:29.751386+0000","last_became_peered":"2023-11-06T02:11:29.751386+0000","last_unstale":"2023-11-06T02:11:31.103973+0000","last_undegraded":"2023-11-06T02:11:31.103973+0000","last_fullsized":"2023-11-06T02:11:31.103973+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:08:31.634000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.104914+0000","last_change":"2023-11-06T02:11:30.150094+0000","last_active":"2023-11-06T02:11:31.104914+0000","last_peered":"2023-11-06T02:11:31.104914+0000","last_clean":"2023-11-06T02:11:31.104914+0000","last_became_active":"2023-11-06T02:11:30.149483+0000","last_became_peered":"2023-11-06T02:11:30.149483+0000","last_unstale":"2023-11-06T02:11:31.104914+0000","last_undegraded":"2023-11-06T02:11:31.104914+0000","last_fullsized":"2023-11-06T02:11:31.104914+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:35:11.189946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"64'3","reported_seq":14,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.112548+0000","last_change":"2023-11-06T02:11:29.763421+0000","last_active":"2023-11-06T02:11:31.112548+0000","last_peered":"2023-11-06T02:11:31.112548+0000","last_clean":"2023-11-06T02:11:31.112548+0000","last_became_active":"2023-11-06T02:11:29.762577+0000","last_became_peered":"2023-11-06T02:11:29.762577+0000","last_unstale":"2023-11-06T02:11:31.112548+0000","last_undegraded":"2023-11-06T02:11:31.112548+0000","last_fullsized":"2023-11-06T02:11:31.112548+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:34:29.023464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.103938+0000","last_change":"2023-11-06T02:11:30.153379+0000","last_active":"2023-11-06T02:11:31.103938+0000","last_peered":"2023-11-06T02:11:31.103938+0000","last_clean":"2023-11-06T02:11:31.103938+0000","last_became_active":"2023-11-06T02:11:30.152795+0000","last_became_peered":"2023-11-06T02:11:30.152795+0000","last_unstale":"2023-11-06T02:11:31.103938+0000","last_undegraded":"2023-11-06T02:11:31.103938+0000","last_fullsized":"2023-11-06T02:11:31.103938+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:42:52.898859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"64'6","reported_seq":17,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.110404+0000","last_change":"2023-11-06T02:11:29.760850+0000","last_active":"2023-11-06T02:11:31.110404+0000","last_peered":"2023-11-06T02:11:31.110404+0000","last_clean":"2023-11-06T02:11:31.110404+0000","last_became_active":"2023-11-06T02:11:29.760540+0000","last_became_peered":"2023-11-06T02:11:29.760540+0000","last_unstale":"2023-11-06T02:11:31.110404+0000","last_undegraded":"2023-11-06T02:11:31.110404+0000","last_fullsized":"2023-11-06T02:11:31.110404+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:37:57.295618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.102972+0000","last_change":"2023-11-06T02:11:30.151250+0000","last_active":"2023-11-06T02:11:31.102972+0000","last_peered":"2023-11-06T02:11:31.102972+0000","last_clean":"2023-11-06T02:11:31.102972+0000","last_became_active":"2023-11-06T02:11:30.147146+0000","last_became_peered":"2023-11-06T02:11:30.147146+0000","last_unstale":"2023-11-06T02:11:31.102972+0000","last_undegraded":"2023-11-06T02:11:31.102972+0000","last_fullsized":"2023-11-06T02:11:31.102972+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:24:57.712407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"64'2","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.111934+0000","last_change":"2023-11-06T02:11:29.761994+0000","last_active":"2023-11-06T02:11:31.111934+0000","last_peered":"2023-11-06T02:11:31.111934+0000","last_clean":"2023-11-06T02:11:31.111934+0000","last_became_active":"2023-11-06T02:11:29.761191+0000","last_became_peered":"2023-11-06T02:11:29.761191+0000","last_unstale":"2023-11-06T02:11:31.111934+0000","last_undegraded":"2023-11-06T02:11:31.111934+0000","last_fullsized":"2023-11-06T02:11:31.111934+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:38:55.717768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.113091+0000","last_change":"2023-11-06T02:11:30.139089+0000","last_active":"2023-11-06T02:11:31.113091+0000","last_peered":"2023-11-06T02:11:31.113091+0000","last_clean":"2023-11-06T02:11:31.113091+0000","last_became_active":"2023-11-06T02:11:30.138277+0000","last_became_peered":"2023-11-06T02:11:30.138277+0000","last_unstale":"2023-11-06T02:11:31.113091+0000","last_undegraded":"2023-11-06T02:11:31.113091+0000","last_fullsized":"2023-11-06T02:11:31.113091+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:31:04.064635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.113452+0000","last_change":"2023-11-06T02:11:29.767908+0000","last_active":"2023-11-06T02:11:31.113452+0000","last_peered":"2023-11-06T02:11:31.113452+0000","last_clean":"2023-11-06T02:11:31.113452+0000","last_became_active":"2023-11-06T02:11:29.767155+0000","last_became_peered":"2023-11-06T02:11:29.767155+0000","last_unstale":"2023-11-06T02:11:31.113452+0000","last_undegraded":"2023-11-06T02:11:31.113452+0000","last_fullsized":"2023-11-06T02:11:31.113452+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:20:25.114853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.104161+0000","last_change":"2023-11-06T02:11:30.132180+0000","last_active":"2023-11-06T02:11:31.104161+0000","last_peered":"2023-11-06T02:11:31.104161+0000","last_clean":"2023-11-06T02:11:31.104161+0000","last_became_active":"2023-11-06T02:11:30.129275+0000","last_became_peered":"2023-11-06T02:11:30.129275+0000","last_unstale":"2023-11-06T02:11:31.104161+0000","last_undegraded":"2023-11-06T02:11:31.104161+0000","last_fullsized":"2023-11-06T02:11:31.104161+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:10:28.619572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.112046+0000","last_change":"2023-11-06T02:11:29.756928+0000","last_active":"2023-11-06T02:11:31.112046+0000","last_peered":"2023-11-06T02:11:31.112046+0000","last_clean":"2023-11-06T02:11:31.112046+0000","last_became_active":"2023-11-06T02:11:29.756079+0000","last_became_peered":"2023-11-06T02:11:29.756079+0000","last_unstale":"2023-11-06T02:11:31.112046+0000","last_undegraded":"2023-11-06T02:11:31.112046+0000","last_fullsized":"2023-11-06T02:11:31.112046+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:39:30.010037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.110651+0000","last_change":"2023-11-06T02:11:30.139787+0000","last_active":"2023-11-06T02:11:31.110651+0000","last_peered":"2023-11-06T02:11:31.110651+0000","last_clean":"2023-11-06T02:11:31.110651+0000","last_became_active":"2023-11-06T02:11:30.139486+0000","last_became_peered":"2023-11-06T02:11:30.139486+0000","last_unstale":"2023-11-06T02:11:31.110651+0000","last_undegraded":"2023-11-06T02:11:31.110651+0000","last_fullsized":"2023-11-06T02:11:31.110651+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:42:52.898859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"64'8","reported_seq":19,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.103236+0000","last_change":"2023-11-06T02:11:29.758210+0000","last_active":"2023-11-06T02:11:31.103236+0000","last_peered":"2023-11-06T02:11:31.103236+0000","last_clean":"2023-11-06T02:11:31.103236+0000","last_became_active":"2023-11-06T02:11:29.757494+0000","last_became_peered":"2023-11-06T02:11:29.757494+0000","last_unstale":"2023-11-06T02:11:31.103236+0000","last_undegraded":"2023-11-06T02:11:31.103236+0000","last_fullsized":"2023-11-06T02:11:31.103236+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:20:00.073264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.101279+0000","last_change":"2023-11-06T02:11:30.154538+0000","last_active":"2023-11-06T02:11:31.101279+0000","last_peered":"2023-11-06T02:11:31.101279+0000","last_clean":"2023-11-06T02:11:31.101279+0000","last_became_active":"2023-11-06T02:11:30.154253+0000","last_became_peered":"2023-11-06T02:11:30.154253+0000","last_unstale":"2023-11-06T02:11:31.101279+0000","last_undegraded":"2023-11-06T02:11:31.101279+0000","last_fullsized":"2023-11-06T02:11:31.101279+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:47:05.853422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"64'2","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.115085+0000","last_change":"2023-11-06T02:11:29.767274+0000","last_active":"2023-11-06T02:11:31.115085+0000","last_peered":"2023-11-06T02:11:31.115085+0000","last_clean":"2023-11-06T02:11:31.115085+0000","last_became_active":"2023-11-06T02:11:29.766821+0000","last_became_peered":"2023-11-06T02:11:29.766821+0000","last_unstale":"2023-11-06T02:11:31.115085+0000","last_undegraded":"2023-11-06T02:11:31.115085+0000","last_fullsized":"2023-11-06T02:11:31.115085+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:51:25.548695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.110281+0000","last_change":"2023-11-06T02:11:30.113941+0000","last_active":"2023-11-06T02:11:31.110281+0000","last_peered":"2023-11-06T02:11:31.110281+0000","last_clean":"2023-11-06T02:11:31.110281+0000","last_became_active":"2023-11-06T02:11:30.113672+0000","last_became_peered":"2023-11-06T02:11:30.113672+0000","last_unstale":"2023-11-06T02:11:31.110281+0000","last_undegraded":"2023-11-06T02:11:31.110281+0000","last_fullsized":"2023-11-06T02:11:31.110281+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:06:25.213807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.114556+0000","last_change":"2023-11-06T02:11:29.755075+0000","last_active":"2023-11-06T02:11:31.114556+0000","last_peered":"2023-11-06T02:11:31.114556+0000","last_clean":"2023-11-06T02:11:31.114556+0000","last_became_active":"2023-11-06T02:11:29.754356+0000","last_became_peered":"2023-11-06T02:11:29.754356+0000","last_unstale":"2023-11-06T02:11:31.114556+0000","last_undegraded":"2023-11-06T02:11:31.114556+0000","last_fullsized":"2023-11-06T02:11:31.114556+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:17:20.196092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.114316+0000","last_change":"2023-11-06T02:11:30.150702+0000","last_active":"2023-11-06T02:11:31.114316+0000","last_peered":"2023-11-06T02:11:31.114316+0000","last_clean":"2023-11-06T02:11:31.114316+0000","last_became_active":"2023-11-06T02:11:30.148902+0000","last_became_peered":"2023-11-06T02:11:30.148902+0000","last_unstale":"2023-11-06T02:11:31.114316+0000","last_undegraded":"2023-11-06T02:11:31.114316+0000","last_fullsized":"2023-11-06T02:11:31.114316+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:17:40.439898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.101470+0000","last_change":"2023-11-06T02:11:29.769534+0000","last_active":"2023-11-06T02:11:31.101470+0000","last_peered":"2023-11-06T02:11:31.101470+0000","last_clean":"2023-11-06T02:11:31.101470+0000","last_became_active":"2023-11-06T02:11:29.769107+0000","last_became_peered":"2023-11-06T02:11:29.769107+0000","last_unstale":"2023-11-06T02:11:31.101470+0000","last_undegraded":"2023-11-06T02:11:31.101470+0000","last_fullsized":"2023-11-06T02:11:31.101470+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:06:05.519539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.105010+0000","last_change":"2023-11-06T02:11:30.130842+0000","last_active":"2023-11-06T02:11:31.105010+0000","last_peered":"2023-11-06T02:11:31.105010+0000","last_clean":"2023-11-06T02:11:31.105010+0000","last_became_active":"2023-11-06T02:11:30.129575+0000","last_became_peered":"2023-11-06T02:11:30.129575+0000","last_unstale":"2023-11-06T02:11:31.105010+0000","last_undegraded":"2023-11-06T02:11:31.105010+0000","last_fullsized":"2023-11-06T02:11:31.105010+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:08:32.638643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.110468+0000","last_change":"2023-11-06T02:11:29.760868+0000","last_active":"2023-11-06T02:11:31.110468+0000","last_peered":"2023-11-06T02:11:31.110468+0000","last_clean":"2023-11-06T02:11:31.110468+0000","last_became_active":"2023-11-06T02:11:29.760567+0000","last_became_peered":"2023-11-06T02:11:29.760567+0000","last_unstale":"2023-11-06T02:11:31.110468+0000","last_undegraded":"2023-11-06T02:11:31.110468+0000","last_fullsized":"2023-11-06T02:11:31.110468+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:40:36.082625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.103668+0000","last_change":"2023-11-06T02:11:30.129382+0000","last_active":"2023-11-06T02:11:31.103668+0000","last_peered":"2023-11-06T02:11:31.103668+0000","last_clean":"2023-11-06T02:11:31.103668+0000","last_became_active":"2023-11-06T02:11:30.128625+0000","last_became_peered":"2023-11-06T02:11:30.128625+0000","last_unstale":"2023-11-06T02:11:31.103668+0000","last_undegraded":"2023-11-06T02:11:31.103668+0000","last_fullsized":"2023-11-06T02:11:31.103668+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:55:14.474895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.112080+0000","last_change":"2023-11-06T02:11:29.757340+0000","last_active":"2023-11-06T02:11:31.112080+0000","last_peered":"2023-11-06T02:11:31.112080+0000","last_clean":"2023-11-06T02:11:31.112080+0000","last_became_active":"2023-11-06T02:11:29.756428+0000","last_became_peered":"2023-11-06T02:11:29.756428+0000","last_unstale":"2023-11-06T02:11:31.112080+0000","last_undegraded":"2023-11-06T02:11:31.112080+0000","last_fullsized":"2023-11-06T02:11:31.112080+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:30:05.695898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.103441+0000","last_change":"2023-11-06T02:11:30.146047+0000","last_active":"2023-11-06T02:11:31.103441+0000","last_peered":"2023-11-06T02:11:31.103441+0000","last_clean":"2023-11-06T02:11:31.103441+0000","last_became_active":"2023-11-06T02:11:30.144992+0000","last_became_peered":"2023-11-06T02:11:30.144992+0000","last_unstale":"2023-11-06T02:11:31.103441+0000","last_undegraded":"2023-11-06T02:11:31.103441+0000","last_fullsized":"2023-11-06T02:11:31.103441+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:10:19.972522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.109723+0000","last_change":"2023-11-06T02:11:29.741530+0000","last_active":"2023-11-06T02:11:31.109723+0000","last_peered":"2023-11-06T02:11:31.109723+0000","last_clean":"2023-11-06T02:11:31.109723+0000","last_became_active":"2023-11-06T02:11:29.741231+0000","last_became_peered":"2023-11-06T02:11:29.741231+0000","last_unstale":"2023-11-06T02:11:31.109723+0000","last_undegraded":"2023-11-06T02:11:31.109723+0000","last_fullsized":"2023-11-06T02:11:31.109723+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:35:46.481307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.109591+0000","last_change":"2023-11-06T02:11:30.154647+0000","last_active":"2023-11-06T02:11:31.109591+0000","last_peered":"2023-11-06T02:11:31.109591+0000","last_clean":"2023-11-06T02:11:31.109591+0000","last_became_active":"2023-11-06T02:11:30.154157+0000","last_became_peered":"2023-11-06T02:11:30.154157+0000","last_unstale":"2023-11-06T02:11:31.109591+0000","last_undegraded":"2023-11-06T02:11:31.109591+0000","last_fullsized":"2023-11-06T02:11:31.109591+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:20:26.119496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.108500+0000","last_change":"2023-11-06T02:11:29.746489+0000","last_active":"2023-11-06T02:11:31.108500+0000","last_peered":"2023-11-06T02:11:31.108500+0000","last_clean":"2023-11-06T02:11:31.108500+0000","last_became_active":"2023-11-06T02:11:29.745659+0000","last_became_peered":"2023-11-06T02:11:29.745659+0000","last_unstale":"2023-11-06T02:11:31.108500+0000","last_undegraded":"2023-11-06T02:11:31.108500+0000","last_fullsized":"2023-11-06T02:11:31.108500+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:32:49.416758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.110013+0000","last_change":"2023-11-06T02:11:30.150778+0000","last_active":"2023-11-06T02:11:31.110013+0000","last_peered":"2023-11-06T02:11:31.110013+0000","last_clean":"2023-11-06T02:11:31.110013+0000","last_became_active":"2023-11-06T02:11:30.150466+0000","last_became_peered":"2023-11-06T02:11:30.150466+0000","last_unstale":"2023-11-06T02:11:31.110013+0000","last_undegraded":"2023-11-06T02:11:31.110013+0000","last_fullsized":"2023-11-06T02:11:31.110013+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:21:14.911044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.109609+0000","last_change":"2023-11-06T02:11:29.741756+0000","last_active":"2023-11-06T02:11:31.109609+0000","last_peered":"2023-11-06T02:11:31.109609+0000","last_clean":"2023-11-06T02:11:31.109609+0000","last_became_active":"2023-11-06T02:11:29.740943+0000","last_became_peered":"2023-11-06T02:11:29.740943+0000","last_unstale":"2023-11-06T02:11:31.109609+0000","last_undegraded":"2023-11-06T02:11:31.109609+0000","last_fullsized":"2023-11-06T02:11:31.109609+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:09:39.841128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.104524+0000","last_change":"2023-11-06T02:11:30.129762+0000","last_active":"2023-11-06T02:11:31.104524+0000","last_peered":"2023-11-06T02:11:31.104524+0000","last_clean":"2023-11-06T02:11:31.104524+0000","last_became_active":"2023-11-06T02:11:30.128986+0000","last_became_peered":"2023-11-06T02:11:30.128986+0000","last_unstale":"2023-11-06T02:11:31.104524+0000","last_undegraded":"2023-11-06T02:11:31.104524+0000","last_fullsized":"2023-11-06T02:11:31.104524+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:09:40.845771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.108187+0000","last_change":"2023-11-06T02:11:29.737268+0000","last_active":"2023-11-06T02:11:31.108187+0000","last_peered":"2023-11-06T02:11:31.108187+0000","last_clean":"2023-11-06T02:11:31.108187+0000","last_became_active":"2023-11-06T02:11:29.736861+0000","last_became_peered":"2023-11-06T02:11:29.736861+0000","last_unstale":"2023-11-06T02:11:31.108187+0000","last_undegraded":"2023-11-06T02:11:31.108187+0000","last_fullsized":"2023-11-06T02:11:31.108187+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:02:34.759621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":204,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-06T02:11:30.116450+0000","last_change":"2023-11-06T02:08:20.523427+0000","last_active":"2023-11-06T02:11:30.116450+0000","last_peered":"2023-11-06T02:11:30.116450+0000","last_clean":"2023-11-06T02:11:30.116450+0000","last_became_active":"2023-11-06T02:08:20.521089+0000","last_became_peered":"2023-11-06T02:08:20.521089+0000","last_unstale":"2023-11-06T02:11:30.116450+0000","last_undegraded":"2023-11-06T02:11:30.116450+0000","last_fullsized":"2023-11-06T02:11:30.116450+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:19.252348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:19.252348+0000","last_clean_scrub_stamp":"2023-11-06T02:08:19.252348+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:30:15.566397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":50,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-06T02:11:30.126367+0000","last_change":"2023-11-06T02:08:26.059502+0000","last_active":"2023-11-06T02:11:30.126367+0000","last_peered":"2023-11-06T02:11:30.126367+0000","last_clean":"2023-11-06T02:11:30.126367+0000","last_became_active":"2023-11-06T02:08:26.058612+0000","last_became_peered":"2023-11-06T02:08:26.058612+0000","last_unstale":"2023-11-06T02:11:30.126367+0000","last_undegraded":"2023-11-06T02:11:30.126367+0000","last_fullsized":"2023-11-06T02:11:30.126367+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:10.626021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.101110+0000","last_change":"2023-11-06T02:11:30.118128+0000","last_active":"2023-11-06T02:11:31.101110+0000","last_peered":"2023-11-06T02:11:31.101110+0000","last_clean":"2023-11-06T02:11:31.101110+0000","last_became_active":"2023-11-06T02:11:30.117841+0000","last_became_peered":"2023-11-06T02:11:30.117841+0000","last_unstale":"2023-11-06T02:11:31.101110+0000","last_undegraded":"2023-11-06T02:11:31.101110+0000","last_fullsized":"2023-11-06T02:11:31.101110+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:15:08.905482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"64'2","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.101165+0000","last_change":"2023-11-06T02:11:29.741726+0000","last_active":"2023-11-06T02:11:31.101165+0000","last_peered":"2023-11-06T02:11:31.101165+0000","last_clean":"2023-11-06T02:11:31.101165+0000","last_became_active":"2023-11-06T02:11:29.741177+0000","last_became_peered":"2023-11-06T02:11:29.741177+0000","last_unstale":"2023-11-06T02:11:31.101165+0000","last_undegraded":"2023-11-06T02:11:31.101165+0000","last_fullsized":"2023-11-06T02:11:31.101165+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:31:03.059992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":51,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-06T02:11:30.116219+0000","last_change":"2023-11-06T02:08:26.060406+0000","last_active":"2023-11-06T02:11:30.116219+0000","last_peered":"2023-11-06T02:11:30.116219+0000","last_clean":"2023-11-06T02:11:30.116219+0000","last_became_active":"2023-11-06T02:08:26.059262+0000","last_became_peered":"2023-11-06T02:08:26.059262+0000","last_unstale":"2023-11-06T02:11:30.116219+0000","last_undegraded":"2023-11-06T02:11:30.116219+0000","last_fullsized":"2023-11-06T02:11:30.116219+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:10.626021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.113354+0000","last_change":"2023-11-06T02:11:30.140402+0000","last_active":"2023-11-06T02:11:31.113354+0000","last_peered":"2023-11-06T02:11:31.113354+0000","last_clean":"2023-11-06T02:11:31.113354+0000","last_became_active":"2023-11-06T02:11:30.139775+0000","last_became_peered":"2023-11-06T02:11:30.139775+0000","last_unstale":"2023-11-06T02:11:31.113354+0000","last_undegraded":"2023-11-06T02:11:31.113354+0000","last_fullsized":"2023-11-06T02:11:31.113354+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:35:47.485950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.113643+0000","last_change":"2023-11-06T02:11:29.755533+0000","last_active":"2023-11-06T02:11:31.113643+0000","last_peered":"2023-11-06T02:11:31.113643+0000","last_clean":"2023-11-06T02:11:31.113643+0000","last_became_active":"2023-11-06T02:11:29.754882+0000","last_became_peered":"2023-11-06T02:11:29.754882+0000","last_unstale":"2023-11-06T02:11:31.113643+0000","last_undegraded":"2023-11-06T02:11:31.113643+0000","last_fullsized":"2023-11-06T02:11:31.113643+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:43:25.646510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":52,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-06T02:11:30.105820+0000","last_change":"2023-11-06T02:08:26.559477+0000","last_active":"2023-11-06T02:11:30.105820+0000","last_peered":"2023-11-06T02:11:30.105820+0000","last_clean":"2023-11-06T02:11:30.105820+0000","last_became_active":"2023-11-06T02:08:26.559202+0000","last_became_peered":"2023-11-06T02:08:26.559202+0000","last_unstale":"2023-11-06T02:11:30.105820+0000","last_undegraded":"2023-11-06T02:11:30.105820+0000","last_fullsized":"2023-11-06T02:11:30.105820+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:55:04.552404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.104253+0000","last_change":"2023-11-06T02:11:30.133136+0000","last_active":"2023-11-06T02:11:31.104253+0000","last_peered":"2023-11-06T02:11:31.104253+0000","last_clean":"2023-11-06T02:11:31.104253+0000","last_became_active":"2023-11-06T02:11:30.130225+0000","last_became_peered":"2023-11-06T02:11:30.130225+0000","last_unstale":"2023-11-06T02:11:31.104253+0000","last_undegraded":"2023-11-06T02:11:31.104253+0000","last_fullsized":"2023-11-06T02:11:31.104253+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:40:37.087268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.100983+0000","last_change":"2023-11-06T02:11:29.753904+0000","last_active":"2023-11-06T02:11:31.100983+0000","last_peered":"2023-11-06T02:11:31.100983+0000","last_clean":"2023-11-06T02:11:31.100983+0000","last_became_active":"2023-11-06T02:11:29.753666+0000","last_became_peered":"2023-11-06T02:11:29.753666+0000","last_unstale":"2023-11-06T02:11:31.100983+0000","last_undegraded":"2023-11-06T02:11:31.100983+0000","last_fullsized":"2023-11-06T02:11:31.100983+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:17:39.435255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":50,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-06T02:11:30.127033+0000","last_change":"2023-11-06T02:08:26.059546+0000","last_active":"2023-11-06T02:11:30.127033+0000","last_peered":"2023-11-06T02:11:30.127033+0000","last_clean":"2023-11-06T02:11:30.127033+0000","last_became_active":"2023-11-06T02:08:26.058806+0000","last_became_peered":"2023-11-06T02:08:26.058806+0000","last_unstale":"2023-11-06T02:11:30.127033+0000","last_undegraded":"2023-11-06T02:11:30.127033+0000","last_fullsized":"2023-11-06T02:11:30.127033+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:23.468510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.112110+0000","last_change":"2023-11-06T02:11:30.164812+0000","last_active":"2023-11-06T02:11:31.112110+0000","last_peered":"2023-11-06T02:11:31.112110+0000","last_clean":"2023-11-06T02:11:31.112110+0000","last_became_active":"2023-11-06T02:11:30.163440+0000","last_became_peered":"2023-11-06T02:11:30.163440+0000","last_unstale":"2023-11-06T02:11:31.112110+0000","last_undegraded":"2023-11-06T02:11:31.112110+0000","last_fullsized":"2023-11-06T02:11:31.112110+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:41:12.940684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"64'1","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.114379+0000","last_change":"2023-11-06T02:11:29.769511+0000","last_active":"2023-11-06T02:11:31.114379+0000","last_peered":"2023-11-06T02:11:31.114379+0000","last_clean":"2023-11-06T02:11:31.114379+0000","last_became_active":"2023-11-06T02:11:29.768855+0000","last_became_peered":"2023-11-06T02:11:29.768855+0000","last_unstale":"2023-11-06T02:11:31.114379+0000","last_undegraded":"2023-11-06T02:11:31.114379+0000","last_fullsized":"2023-11-06T02:11:31.114379+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:10:41.234197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":49,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-06T02:11:30.111378+0000","last_change":"2023-11-06T02:08:26.436463+0000","last_active":"2023-11-06T02:11:30.111378+0000","last_peered":"2023-11-06T02:11:30.111378+0000","last_clean":"2023-11-06T02:11:30.111378+0000","last_became_active":"2023-11-06T02:08:26.436075+0000","last_became_peered":"2023-11-06T02:08:26.436075+0000","last_unstale":"2023-11-06T02:11:30.111378+0000","last_undegraded":"2023-11-06T02:11:30.111378+0000","last_fullsized":"2023-11-06T02:11:30.111378+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:23.468510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.104888+0000","last_change":"2023-11-06T02:11:30.163619+0000","last_active":"2023-11-06T02:11:31.104888+0000","last_peered":"2023-11-06T02:11:31.104888+0000","last_clean":"2023-11-06T02:11:31.104888+0000","last_became_active":"2023-11-06T02:11:30.163060+0000","last_became_peered":"2023-11-06T02:11:30.163060+0000","last_unstale":"2023-11-06T02:11:31.104888+0000","last_undegraded":"2023-11-06T02:11:31.104888+0000","last_fullsized":"2023-11-06T02:11:31.104888+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:34:30.028107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"64'2","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.114533+0000","last_change":"2023-11-06T02:11:29.766067+0000","last_active":"2023-11-06T02:11:31.114533+0000","last_peered":"2023-11-06T02:11:31.114533+0000","last_clean":"2023-11-06T02:11:31.114533+0000","last_became_active":"2023-11-06T02:11:29.765630+0000","last_became_peered":"2023-11-06T02:11:29.765630+0000","last_unstale":"2023-11-06T02:11:31.114533+0000","last_undegraded":"2023-11-06T02:11:31.114533+0000","last_fullsized":"2023-11-06T02:11:31.114533+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:35:10.185303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":51,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-06T02:11:30.121485+0000","last_change":"2023-11-06T02:08:26.436607+0000","last_active":"2023-11-06T02:11:30.121485+0000","last_peered":"2023-11-06T02:11:30.121485+0000","last_clean":"2023-11-06T02:11:30.121485+0000","last_became_active":"2023-11-06T02:08:26.435924+0000","last_became_peered":"2023-11-06T02:08:26.435924+0000","last_unstale":"2023-11-06T02:11:30.121485+0000","last_undegraded":"2023-11-06T02:11:30.121485+0000","last_fullsized":"2023-11-06T02:11:30.121485+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:23.468510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.103761+0000","last_change":"2023-11-06T02:11:30.135530+0000","last_active":"2023-11-06T02:11:31.103761+0000","last_peered":"2023-11-06T02:11:31.103761+0000","last_clean":"2023-11-06T02:11:31.103761+0000","last_became_active":"2023-11-06T02:11:30.134786+0000","last_became_peered":"2023-11-06T02:11:30.134786+0000","last_unstale":"2023-11-06T02:11:31.103761+0000","last_undegraded":"2023-11-06T02:11:31.103761+0000","last_fullsized":"2023-11-06T02:11:31.103761+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:48:50.245921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"64'1","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.112435+0000","last_change":"2023-11-06T02:11:29.762390+0000","last_active":"2023-11-06T02:11:31.112435+0000","last_peered":"2023-11-06T02:11:31.112435+0000","last_clean":"2023-11-06T02:11:31.112435+0000","last_became_active":"2023-11-06T02:11:29.761673+0000","last_became_peered":"2023-11-06T02:11:29.761673+0000","last_unstale":"2023-11-06T02:11:31.112435+0000","last_undegraded":"2023-11-06T02:11:31.112435+0000","last_fullsized":"2023-11-06T02:11:31.112435+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:07:34.988240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":49,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-06T02:11:30.110990+0000","last_change":"2023-11-06T02:08:26.057076+0000","last_active":"2023-11-06T02:11:30.110990+0000","last_peered":"2023-11-06T02:11:30.110990+0000","last_clean":"2023-11-06T02:11:30.110990+0000","last_became_active":"2023-11-06T02:08:26.056419+0000","last_became_peered":"2023-11-06T02:08:26.056419+0000","last_unstale":"2023-11-06T02:11:30.110990+0000","last_undegraded":"2023-11-06T02:11:30.110990+0000","last_fullsized":"2023-11-06T02:11:30.110990+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:10.626021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.105222+0000","last_change":"2023-11-06T02:11:30.130327+0000","last_active":"2023-11-06T02:11:31.105222+0000","last_peered":"2023-11-06T02:11:31.105222+0000","last_clean":"2023-11-06T02:11:31.105222+0000","last_became_active":"2023-11-06T02:11:30.128754+0000","last_became_peered":"2023-11-06T02:11:30.128754+0000","last_unstale":"2023-11-06T02:11:31.105222+0000","last_undegraded":"2023-11-06T02:11:31.105222+0000","last_fullsized":"2023-11-06T02:11:31.105222+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:29.703192+0000","last_clean_scrub_stamp":"2023-11-06T02:11:29.703192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:20:01.077907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T02:11:31.108631+0000","last_change":"2023-11-06T02:11:29.755079+0000","last_active":"2023-11-06T02:11:31.108631+0000","last_peered":"2023-11-06T02:11:31.108631+0000","last_clean":"2023-11-06T02:11:31.108631+0000","last_became_active":"2023-11-06T02:11:29.754680+0000","last_became_peered":"2023-11-06T02:11:29.754680+0000","last_unstale":"2023-11-06T02:11:31.108631+0000","last_undegraded":"2023-11-06T02:11:31.108631+0000","last_fullsized":"2023-11-06T02:11:31.108631+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:11:28.698549+0000","last_clean_scrub_stamp":"2023-11-06T02:11:28.698549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:39:34.434686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":57,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-06T02:11:30.111388+0000","last_change":"2023-11-06T02:08:26.058596+0000","last_active":"2023-11-06T02:11:30.111388+0000","last_peered":"2023-11-06T02:11:30.111388+0000","last_clean":"2023-11-06T02:11:30.111388+0000","last_became_active":"2023-11-06T02:08:26.057015+0000","last_became_peered":"2023-11-06T02:08:26.057015+0000","last_unstale":"2023-11-06T02:11:30.111388+0000","last_undegraded":"2023-11-06T02:11:30.111388+0000","last_fullsized":"2023-11-06T02:11:30.111388+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:55:04.552404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574890,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28312,"kb_used_data":1360,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343528,"statfs":{"total":96636764160,"available":96607772672,"internally_reserved":0,"allocated":1392640,"data_stored":937394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.86199999999999999,"15min":0.85299999999999998},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.1200000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.91300000000000003,"15min":0.91100000000000003},"min":{"1min":0.68300000000000005,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.1020000000000001,"5min":1.506,"15min":1.506},"last":0.98999999999999999}]},{"osd":1,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.82699999999999996,"15min":0.84299999999999997},"min":{"1min":0.55300000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.0329999999999999,"5min":1.161,"15min":1.161},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.91200000000000003,"15min":0.89600000000000002},"min":{"1min":0.63600000000000001,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.1910000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.877}]},{"osd":2,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.96399999999999997,"15min":0.95699999999999996},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.2989999999999999,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":1.109},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.98699999999999999,"15min":0.996},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.5,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.78500000000000003}]},{"osd":3,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.88300000000000001,"15min":0.877},"min":{"1min":0.69099999999999995,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.1479999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":1.276},{"interface":"front","average":{"1min":1.012,"5min":1.0700000000000001,"15min":1.0740000000000001},"min":{"1min":0.78900000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.2230000000000001,"5min":1.526,"15min":1.526},"last":1.032}]},{"osd":4,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.95899999999999996,"15min":0.95799999999999996},"min":{"1min":0.64000000000000001,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":1.0640000000000001},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.0229999999999999,"15min":1.0369999999999999},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":1.4199999999999999}]},{"osd":5,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":1.0669999999999999,"15min":1.075},"min":{"1min":0.69999999999999996,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":1.244},{"interface":"front","average":{"1min":1.0780000000000001,"5min":1.117,"15min":1.129},"min":{"1min":0.73199999999999998,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.3700000000000001,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":1.1499999999999999}]},{"osd":6,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.196,"15min":1.2},"min":{"1min":0.83999999999999997,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.5009999999999999,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":1.2050000000000001},{"interface":"front","average":{"1min":1.048,"5min":1.131,"15min":1.149},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.331,"5min":1.74,"15min":1.74},"last":1.369}]}]},{"osd":6,"up_from":13,"seq":55834574889,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27720,"kb_used_data":768,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344120,"statfs":{"total":96636764160,"available":96608378880,"internally_reserved":0,"allocated":786432,"data_stored":346284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.52400000000000002,"5min":0.495,"15min":0.495},"max":{"1min":1.216,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":1.032},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.84099999999999997,"15min":0.81699999999999995},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.2370000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":1.1060000000000001}]},{"osd":1,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.86599999999999999,"15min":0.872},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.86799999999999999,"15min":0.872},"min":{"1min":0.63,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.2330000000000001,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":1.129}]},{"osd":2,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.877,"15min":0.89100000000000001},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":1.252},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.90400000000000003,"15min":0.92200000000000004},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.103,"5min":1.28,"15min":1.28},"last":0.96299999999999997}]},{"osd":3,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":0.95999999999999996,"15min":0.96799999999999997},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.3300000000000001,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":1.1830000000000001},{"interface":"front","average":{"1min":0.99399999999999999,"5min":0.99199999999999999,"15min":0.998},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.22,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":1.056}]},{"osd":4,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.95099999999999996,"15min":0.97899999999999998},"min":{"1min":0.59499999999999997,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.1639999999999999,"5min":1.474,"15min":1.474},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.99399999999999999,"15min":1.024},"min":{"1min":0.624,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.268,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.91300000000000003}]},{"osd":5,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.98399999999999999,"15min":1.0129999999999999},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.161,"5min":1.365,"15min":1.365},"last":1.079},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.0760000000000001,"15min":1.1020000000000001},"min":{"1min":0.68400000000000005,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.3939999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":1.228}]},{"osd":7,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":1.083,"15min":1.1060000000000001},"min":{"1min":0.76800000000000002,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.3580000000000001,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":1.204},{"interface":"front","average":{"1min":0.99399999999999999,"5min":1.095,"15min":1.141},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.425,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":1.1579999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574890,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27720,"kb_used_data":768,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344120,"statfs":{"total":96636764160,"available":96608378880,"internally_reserved":0,"allocated":786432,"data_stored":346272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.85599999999999998,"15min":0.83799999999999997},"min":{"1min":0.63300000000000001,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.2390000000000001,"5min":1.381,"15min":1.381},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.85699999999999998,"15min":0.84699999999999998},"min":{"1min":0.63100000000000001,"5min":0.503,"15min":0.503},"max":{"1min":1.3200000000000001,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":1.0780000000000001}]},{"osd":1,"last update":"Mon Nov 6 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.86599999999999999,"15min":0.85499999999999998},"min":{"1min":0.626,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.1699999999999999,"5min":1.331,"15min":1.331},"last":0.93500000000000005},{"interface":"front","average":{"1min":0.87,"5min":0.83599999999999997,"15min":0.82299999999999995},"min":{"1min":0.57199999999999995,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.177,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":1.0569999999999999}]},{"osd":2,"last update":"Mon Nov 6 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.93200000000000005,"15min":0.94099999999999995},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.2210000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.83199999999999996,"15min":0.80400000000000005},"min":{"1min":0.55800000000000005,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.323,"5min":1.377,"15min":1.377},"last":1.1020000000000001}]},{"osd":3,"last update":"Mon Nov 6 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":0.93200000000000005,"15min":0.92400000000000004},"min":{"1min":0.65900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.3580000000000001,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":1.0269999999999999},{"interface":"front","average":{"1min":1.0009999999999999,"5min":1.0409999999999999,"15min":1.0669999999999999},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.228,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":0.82899999999999996}]},{"osd":4,"last update":"Mon Nov 6 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.93100000000000005,"15min":0.92100000000000004},"min":{"1min":0.61299999999999999,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.2410000000000001,"5min":1.276,"15min":1.276},"last":1.1819999999999999},{"interface":"front","average":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.026},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.496,"5min":1.554,"15min":1.554},"last":0.65800000000000003}]},{"osd":6,"last update":"Mon Nov 6 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":1.0469999999999999,"15min":1.0609999999999999},"min":{"1min":0.67500000000000004,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":0.90900000000000003},{"interface":"front","average":{"1min":1.075,"5min":1.04,"15min":1.04},"min":{"1min":0.72999999999999998,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.98299999999999998}]},{"osd":7,"last update":"Mon Nov 6 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.083,"15min":1.0780000000000001},"min":{"1min":0.71599999999999997,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.5880000000000001,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":0.77800000000000002},{"interface":"front","average":{"1min":1.161,"5min":1.1499999999999999,"15min":1.153},"min":{"1min":0.92400000000000004,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.544,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":0.72099999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574890,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27728,"kb_used_data":776,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344112,"statfs":{"total":96636764160,"available":96608370688,"internally_reserved":0,"allocated":794624,"data_stored":347814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:11:22 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.85599999999999998,"15min":0.83499999999999996},"min":{"1min":0.67100000000000004,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.343,"5min":1.343,"15min":1.343},"last":0.98199999999999998},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.79300000000000004,"15min":0.77700000000000002},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.0169999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.80600000000000005}]},{"osd":1,"last update":"Mon Nov 6 02:11:22 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.86899999999999999,"15min":0.85199999999999998},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.083,"5min":1.462,"15min":1.462},"last":1.008},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.92400000000000004,"15min":0.90200000000000002},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.3580000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.83899999999999997}]},{"osd":2,"last update":"Mon Nov 6 02:11:22 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.97599999999999998,"15min":1.0049999999999999},"min":{"1min":0.69399999999999995,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.198,"5min":1.615,"15min":1.615},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.94099999999999995,"15min":0.93100000000000005},"min":{"1min":0.68000000000000005,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.3049999999999999,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":1.0309999999999999}]},{"osd":3,"last update":"Mon Nov 6 02:11:22 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.90200000000000002,"15min":0.91300000000000003},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.103,"5min":1.52,"15min":1.52},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.97799999999999998,"5min":1.04,"15min":1.0600000000000001},"min":{"1min":0.63900000000000001,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.2470000000000001,"5min":1.595,"15min":1.595},"last":0.95699999999999996}]},{"osd":5,"last update":"Mon Nov 6 02:11:22 2023","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":1.0329999999999999,"15min":1.0449999999999999},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.25,"5min":1.377,"15min":1.377},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.97999999999999998,"5min":1.0189999999999999,"15min":1.0429999999999999},"min":{"1min":0.67500000000000004,"5min":0.629,"15min":0.629},"max":{"1min":1.278,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":0.66400000000000003}]},{"osd":6,"last update":"Mon Nov 6 02:11:22 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":1.052,"15min":1.0800000000000001},"min":{"1min":0.68000000000000005,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.2470000000000001,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":1.071,"5min":1.157,"15min":1.1839999999999999},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.4079999999999999,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":0.88900000000000001}]},{"osd":7,"last update":"Mon Nov 6 02:11:22 2023","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":1.0720000000000001,"15min":1.0860000000000001},"min":{"1min":0.748,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.2849999999999999,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":1.0529999999999999},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.1859999999999999,"15min":1.222},"min":{"1min":0.80000000000000004,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.3280000000000001,"5min":1.7410000000000001,"15min":1.7410000000000001},"last":0.85799999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574889,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31812,"kb_used_data":764,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340028,"statfs":{"total":96636764160,"available":96604188672,"internally_reserved":0,"allocated":782336,"data_stored":346250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.63900000000000001,"15min":0.64500000000000002},"min":{"1min":0.36499999999999999,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.95999999999999996,"5min":1.079,"15min":1.079},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.626,"5min":0.625,"15min":0.625},"min":{"1min":0.34599999999999997,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.93899999999999995,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.59199999999999997}]},{"osd":1,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.61899999999999999,"15min":0.627},"min":{"1min":0.27800000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.86799999999999999,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.72599999999999998,"15min":0.73099999999999998},"min":{"1min":0.307,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.0980000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.66600000000000004}]},{"osd":2,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.71399999999999997,"15min":0.72799999999999998},"min":{"1min":0.374,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.81200000000000006,"15min":0.81999999999999995},"min":{"1min":0.39000000000000001,"5min":0.308,"15min":0.308},"max":{"1min":1.1899999999999999,"5min":1.405,"15min":1.405},"last":0.84299999999999997}]},{"osd":4,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.92700000000000005,"15min":0.92800000000000005},"min":{"1min":0.65900000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.085,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.98099999999999998,"5min":0.97599999999999998,"15min":0.99099999999999999},"min":{"1min":0.67800000000000005,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.236,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":1.077}]},{"osd":5,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":1.004,"15min":1.02},"min":{"1min":0.70699999999999996,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":1.141},{"interface":"front","average":{"1min":1.0880000000000001,"5min":1.0580000000000001,"15min":1.0740000000000001},"min":{"1min":0.74099999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.4330000000000001,"5min":2.21,"15min":2.21},"last":1.1220000000000001}]},{"osd":6,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":1.0229999999999999,"15min":1.0249999999999999},"min":{"1min":0.76900000000000002,"5min":0.439,"15min":0.439},"max":{"1min":1.3799999999999999,"5min":1.395,"15min":1.395},"last":1.2989999999999999},{"interface":"front","average":{"1min":1.0880000000000001,"5min":1.0600000000000001,"15min":1.0680000000000001},"min":{"1min":0.71699999999999997,"5min":0.375,"15min":0.375},"max":{"1min":1.4319999999999999,"5min":1.554,"15min":1.554},"last":0.999}]},{"osd":7,"last update":"Mon Nov 6 02:11:23 2023","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.115,"15min":1.127},"min":{"1min":0.76300000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.5549999999999999,"5min":2.2829999999999999,"15min":2.2829999999999999},"last":1.266},{"interface":"front","average":{"1min":1.171,"5min":1.1080000000000001,"15min":1.107},"min":{"1min":0.92200000000000004,"5min":0.497,"15min":0.497},"max":{"1min":1.504,"5min":1.504,"15min":1.504},"last":1.218}]}]},{"osd":2,"up_from":13,"seq":55834574889,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31812,"kb_used_data":764,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340028,"statfs":{"total":96636764160,"available":96604188672,"internally_reserved":0,"allocated":782336,"data_stored":346250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:11:20 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.53900000000000003,"15min":0.51200000000000001},"min":{"1min":0.48199999999999998,"5min":0.252,"15min":0.252},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.59099999999999997,"15min":0.55600000000000005},"min":{"1min":0.497,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":0.502}]},{"osd":1,"last update":"Mon Nov 6 02:11:20 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.60599999999999998,"15min":0.57499999999999996},"min":{"1min":0.44700000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.48,"5min":1.48,"15min":1.48},"last":0.623},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.64700000000000002,"15min":0.59399999999999997},"min":{"1min":0.45200000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.7230000000000001,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":0.75700000000000001}]},{"osd":3,"last update":"Mon Nov 6 02:11:20 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.65400000000000003,"15min":0.61599999999999999},"min":{"1min":0.58699999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.93200000000000005,"5min":1.155,"15min":1.155},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.69799999999999995,"15min":0.66000000000000003},"min":{"1min":0.52500000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.0960000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.58399999999999996}]},{"osd":4,"last update":"Mon Nov 6 02:11:20 2023","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":0.85799999999999998,"15min":0.81799999999999995},"min":{"1min":0.77500000000000002,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.89800000000000002},{"interface":"front","average":{"1min":1.0369999999999999,"5min":0.93600000000000005,"15min":0.92200000000000004},"min":{"1min":0.755,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.395,"5min":1.395,"15min":1.395},"last":0.95099999999999996}]},{"osd":5,"last update":"Mon Nov 6 02:11:20 2023","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":0.96199999999999997,"15min":0.94899999999999995},"min":{"1min":0.74099999999999999,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.7729999999999999,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":1.0189999999999999,"5min":0.95699999999999996,"15min":0.94099999999999995},"min":{"1min":0.67200000000000004,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.615,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.97099999999999997}]},{"osd":6,"last update":"Mon Nov 6 02:11:20 2023","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":0.96899999999999997,"15min":0.94799999999999995},"min":{"1min":0.79200000000000004,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":0.91700000000000004},{"interface":"front","average":{"1min":1.087,"5min":1.0129999999999999,"15min":1.002},"min":{"1min":0.81499999999999995,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.675,"5min":1.677,"15min":1.677},"last":1.0369999999999999}]},{"osd":7,"last update":"Mon Nov 6 02:11:20 2023","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":1.0089999999999999,"15min":0.98599999999999999},"min":{"1min":0.72499999999999998,"5min":0.622,"15min":0.622},"max":{"1min":1.659,"5min":1.659,"15min":1.659},"last":0.98699999999999999},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.0620000000000001,"15min":1.048},"min":{"1min":0.89300000000000002,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.4650000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":1.1160000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574890,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31840,"kb_used_data":792,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340000,"statfs":{"total":96636764160,"available":96604160000,"internally_reserved":0,"allocated":811008,"data_stored":348578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:11:22 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.63700000000000001,"15min":0.629},"min":{"1min":0.28699999999999998,"5min":0.224,"15min":0.224},"max":{"1min":1.603,"5min":1.603,"15min":1.603},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.67400000000000004,"15min":0.65900000000000003},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.76900000000000002}]},{"osd":2,"last update":"Mon Nov 6 02:11:22 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.63700000000000001,"15min":0.627},"min":{"1min":0.33600000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.649,"5min":1.649,"15min":1.649},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.755,"15min":0.748},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.1919999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.73699999999999999}]},{"osd":3,"last update":"Mon Nov 6 02:11:22 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.79500000000000004,"15min":0.80000000000000004},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.2350000000000001,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.98199999999999998},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.80800000000000005,"15min":0.81000000000000005},"min":{"1min":0.41299999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.5,"5min":1.8149999999999999,"15min":1.8149999999999999},"last":0.93999999999999995}]},{"osd":4,"last update":"Mon Nov 6 02:11:22 2023","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.97699999999999998,"15min":0.98699999999999999},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.5780000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":1.0660000000000001},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.98099999999999998,"15min":0.98899999999999999},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.4099999999999999,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":1.1799999999999999}]},{"osd":5,"last update":"Mon Nov 6 02:11:22 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":0.97499999999999998,"15min":0.96899999999999997},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.718,"5min":1.718,"15min":1.718},"last":1.036},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.069,"15min":1.075},"min":{"1min":0.66500000000000004,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.403,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":1.141}]},{"osd":6,"last update":"Mon Nov 6 02:11:22 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.0920000000000001,"15min":1.099},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.6859999999999999,"5min":1.742,"15min":1.742},"last":1.2070000000000001},{"interface":"front","average":{"1min":1.0700000000000001,"5min":1.0669999999999999,"15min":1.073},"min":{"1min":0.64400000000000002,"5min":0.629,"15min":0.629},"max":{"1min":1.6339999999999999,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":1.0960000000000001}]},{"osd":7,"last update":"Mon Nov 6 02:11:22 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.115,"15min":1.119},"min":{"1min":0.71499999999999997,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.534,"5min":1.534,"15min":1.534},"last":1.2310000000000001},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.1499999999999999,"15min":1.157},"min":{"1min":0.68700000000000006,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.7629999999999999,"5min":1.893,"15min":1.893},"last":1.1160000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574890,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32388,"kb_used_data":1340,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339452,"statfs":{"total":96636764160,"available":96603598848,"internally_reserved":0,"allocated":1372160,"data_stored":935599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 02:11:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.33700000000000002,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64000000000000001,"15min":0.63700000000000001},"min":{"1min":0.38700000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.93400000000000005,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.65400000000000003}]},{"osd":2,"last update":"Mon Nov 6 02:11:25 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.67100000000000004,"15min":0.66700000000000004},"min":{"1min":0.505,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.67800000000000005,"15min":0.67600000000000005},"min":{"1min":0.35199999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.95599999999999996,"5min":1.141,"15min":1.141},"last":0.83399999999999996}]},{"osd":3,"last update":"Mon Nov 6 02:11:25 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.64000000000000001,"15min":0.63200000000000001},"min":{"1min":0.376,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.0600000000000001,"5min":1.429,"15min":1.429},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.753,"15min":0.73799999999999999},"min":{"1min":0.44,"5min":0.317,"15min":0.317},"max":{"1min":1.1299999999999999,"5min":1.25,"15min":1.25},"last":1.141}]},{"osd":4,"last update":"Mon Nov 6 02:11:25 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.85599999999999998,"15min":0.83599999999999997},"min":{"1min":0.59499999999999997,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.1579999999999999,"5min":1.341,"15min":1.341},"last":1.0640000000000001},{"interface":"front","average":{"1min":0.995,"5min":0.94299999999999995,"15min":0.93899999999999995},"min":{"1min":0.65100000000000002,"5min":0.5,"15min":0.5},"max":{"1min":1.3600000000000001,"5min":1.387,"15min":1.387},"last":1.016}]},{"osd":5,"last update":"Mon Nov 6 02:11:25 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.94199999999999995,"15min":0.94599999999999995},"min":{"1min":0.63300000000000001,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.119,"5min":1.339,"15min":1.339},"last":0.97799999999999998},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.89700000000000002,"15min":0.88},"min":{"1min":0.72899999999999998,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.325,"5min":1.448,"15min":1.448},"last":1.2390000000000001}]},{"osd":6,"last update":"Mon Nov 6 02:11:25 2023","interfaces":[{"interface":"back","average":{"1min":1.0960000000000001,"5min":1.0409999999999999,"15min":1.038},"min":{"1min":0.66700000000000004,"5min":0.625,"15min":0.625},"max":{"1min":1.573,"5min":1.589,"15min":1.589},"last":1.268},{"interface":"front","average":{"1min":1.0309999999999999,"5min":1.0169999999999999,"15min":1.0049999999999999},"min":{"1min":0.67600000000000005,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.3899999999999999,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":1.1060000000000001}]},{"osd":7,"last update":"Mon Nov 6 02:11:25 2023","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":1.008,"15min":1.0149999999999999},"min":{"1min":0.71299999999999997,"5min":0.504,"15min":0.504},"max":{"1min":1.341,"5min":1.421,"15min":1.421},"last":1.1839999999999999},{"interface":"front","average":{"1min":0.98599999999999999,"5min":0.98899999999999999,"15min":1.002},"min":{"1min":0.69899999999999995,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.3029999999999999,"5min":1.556,"15min":1.556},"last":0.93500000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T02:11:39.116 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-06T02:11:39.116 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-06T02:11:39.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.285+0000 7f2e0ba8b640 1 Processor -- start 2023-11-06T02:11:39.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.285+0000 7f2e0ba8b640 1 -- start start 2023-11-06T02:11:39.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.289+0000 7f2e0ba8b640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f2e0415cbb0 0x7f2e040d1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:39.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.289+0000 7f2e0ba8b640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f2e040d24e0 0x7f2e040d2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:39.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.289+0000 7f2e0ba8b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2e040d2fa0 0x7f2e040b4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:39.289 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.289+0000 7f2e0ba8b640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e040d3430 con 0x7f2e040d24e0 2023-11-06T02:11:39.289 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.289+0000 7f2e0ba8b640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e0415cf90 con 0x7f2e0415cbb0 2023-11-06T02:11:39.289 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.289+0000 7f2e0ba8b640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e040d0b80 con 0x7f2e040d2fa0 2023-11-06T02:11:39.289 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.289+0000 7f2e09026640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f2e040d24e0 0x7f2e040d2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:39.289 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.289+0000 7f2e09026640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f2e040d24e0 0x7f2e040d2a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49388/0 (socket says 172.21.15.49:49388) 2023-11-06T02:11:39.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.289+0000 7f2e09026640 1 -- 172.21.15.49:0/422332429 learned_addr learned my addr 172.21.15.49:0/422332429 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:39.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.289+0000 7f2e09827640 1 --2- 172.21.15.49:0/422332429 >> v2:172.21.15.150:3300/0 conn(0x7f2e0415cbb0 0x7f2e040d1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:39.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.289+0000 7f2e0a028640 1 --2- 172.21.15.49:0/422332429 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2e040d2fa0 0x7f2e040b4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:39.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.289+0000 7f2e09026640 1 -- 172.21.15.49:0/422332429 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2e040d2fa0 msgr2=0x7f2e040b4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:39.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.289+0000 7f2e09026640 1 --2- 172.21.15.49:0/422332429 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2e040d2fa0 0x7f2e040b4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:39.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.289+0000 7f2e09026640 1 -- 172.21.15.49:0/422332429 >> v2:172.21.15.150:3300/0 conn(0x7f2e0415cbb0 msgr2=0x7f2e040d1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:39.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.289+0000 7f2e09026640 1 --2- 172.21.15.49:0/422332429 >> v2:172.21.15.150:3300/0 conn(0x7f2e0415cbb0 0x7f2e040d1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:39.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.289+0000 7f2e09026640 1 -- 172.21.15.49:0/422332429 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e040d0e60 con 0x7f2e040d24e0 2023-11-06T02:11:39.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.289+0000 7f2e0a028640 1 --2- 172.21.15.49:0/422332429 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2e040d2fa0 0x7f2e040b4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:11:39.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.289+0000 7f2e09026640 1 --2- 172.21.15.49:0/422332429 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e040d24e0 0x7f2e040d2a60 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f2e040cf4b0 tx=0x7f2df401ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8025a5704e8c715d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:39.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.293+0000 7f2e08825640 1 -- 172.21.15.49:0/422332429 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2df40a89b0 con 0x7f2e040d24e0 2023-11-06T02:11:39.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.293+0000 7f2e08825640 1 -- 172.21.15.49:0/422332429 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2df40a8b10 con 0x7f2e040d24e0 2023-11-06T02:11:39.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.293+0000 7f2e08825640 1 -- 172.21.15.49:0/422332429 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2df40a8ea0 con 0x7f2e040d24e0 2023-11-06T02:11:39.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.293+0000 7f2e0ba8b640 1 -- 172.21.15.49:0/422332429 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e040d24e0 msgr2=0x7f2e040d2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:39.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.293+0000 7f2e0ba8b640 1 --2- 172.21.15.49:0/422332429 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e040d24e0 0x7f2e040d2a60 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f2e040cf4b0 tx=0x7f2df401ecf0 comp rx=0 tx=0).stop 2023-11-06T02:11:39.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.293+0000 7f2e0ba8b640 1 -- 172.21.15.49:0/422332429 shutdown_connections 2023-11-06T02:11:39.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.293+0000 7f2e0ba8b640 1 --2- 172.21.15.49:0/422332429 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2e040d2fa0 0x7f2e040b4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:39.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.293+0000 7f2e0ba8b640 1 --2- 172.21.15.49:0/422332429 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e040d24e0 0x7f2e040d2a60 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:39.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.293+0000 7f2e0ba8b640 1 --2- 172.21.15.49:0/422332429 >> v2:172.21.15.150:3300/0 conn(0x7f2e0415cbb0 0x7f2e040d1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:39.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.293+0000 7f2e0ba8b640 1 -- 172.21.15.49:0/422332429 >> 172.21.15.49:0/422332429 conn(0x7f2e040718a0 msgr2=0x7f2e04071cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:39.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.293+0000 7f2e0ba8b640 1 -- 172.21.15.49:0/422332429 shutdown_connections 2023-11-06T02:11:39.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.293+0000 7f2e0ba8b640 1 -- 172.21.15.49:0/422332429 wait complete. 2023-11-06T02:11:39.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.293+0000 7f2e0ba8b640 1 Processor -- start 2023-11-06T02:11:39.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.293+0000 7f2e0ba8b640 1 -- start start 2023-11-06T02:11:39.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.293+0000 7f2e0ba8b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e040d24e0 0x7f2e040c8870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:39.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.293+0000 7f2e0ba8b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2e040d2fa0 0x7f2e040aba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:39.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.293+0000 7f2e0ba8b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2e0415cbb0 0x7f2e040abf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:39.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.293+0000 7f2e0ba8b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e041f2f10 con 0x7f2e040d24e0 2023-11-06T02:11:39.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.293+0000 7f2e0ba8b640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e0415e2e0 con 0x7f2e040d2fa0 2023-11-06T02:11:39.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.293+0000 7f2e0ba8b640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e0415e5c0 con 0x7f2e0415cbb0 2023-11-06T02:11:39.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.293+0000 7f2e09827640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e040d24e0 0x7f2e040c8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:39.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.293+0000 7f2e09827640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e040d24e0 0x7f2e040c8870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49394/0 (socket says 172.21.15.49:49394) 2023-11-06T02:11:39.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.293+0000 7f2e09827640 1 -- 172.21.15.49:0/3289736156 learned_addr learned my addr 172.21.15.49:0/3289736156 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:39.300 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.293+0000 7f2e09026640 1 --2- 172.21.15.49:0/3289736156 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2e040d2fa0 0x7f2e040aba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:39.300 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.293+0000 7f2e0a028640 1 --2- 172.21.15.49:0/3289736156 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2e0415cbb0 0x7f2e040abf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:39.300 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.293+0000 7f2e09827640 1 -- 172.21.15.49:0/3289736156 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2e0415cbb0 msgr2=0x7f2e040abf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:39.300 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.293+0000 7f2e09827640 1 --2- 172.21.15.49:0/3289736156 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2e0415cbb0 0x7f2e040abf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:39.300 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.293+0000 7f2e09827640 1 -- 172.21.15.49:0/3289736156 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2e040d2fa0 msgr2=0x7f2e040aba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:39.300 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.293+0000 7f2e09827640 1 --2- 172.21.15.49:0/3289736156 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2e040d2fa0 0x7f2e040aba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:39.301 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.293+0000 7f2e09827640 1 -- 172.21.15.49:0/3289736156 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2df40094d0 con 0x7f2e040d24e0 2023-11-06T02:11:39.301 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.293+0000 7f2e09026640 1 --2- 172.21.15.49:0/3289736156 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2e040d2fa0 0x7f2e040aba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:11:39.301 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.293+0000 7f2e0a028640 1 --2- 172.21.15.49:0/3289736156 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2e0415cbb0 0x7f2e040abf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:11:39.301 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.293+0000 7f2e09827640 1 --2- 172.21.15.49:0/3289736156 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e040d24e0 0x7f2e040c8870 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f2e0000d800 tx=0x7f2e0000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:39.301 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.297+0000 7f2dfa7fc640 1 -- 172.21.15.49:0/3289736156 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e0000b6c0 con 0x7f2e040d24e0 2023-11-06T02:11:39.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.297+0000 7f2dfa7fc640 1 -- 172.21.15.49:0/3289736156 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e0000b820 con 0x7f2e040d24e0 2023-11-06T02:11:39.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.297+0000 7f2e0ba8b640 1 -- 172.21.15.49:0/3289736156 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e0415e8a0 con 0x7f2e040d24e0 2023-11-06T02:11:39.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.297+0000 7f2dfa7fc640 1 -- 172.21.15.49:0/3289736156 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e0000bb90 con 0x7f2e040d24e0 2023-11-06T02:11:39.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.297+0000 7f2e0ba8b640 1 -- 172.21.15.49:0/3289736156 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e0415f0a0 con 0x7f2e040d24e0 2023-11-06T02:11:39.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.297+0000 7f2e0ba8b640 1 -- 172.21.15.49:0/3289736156 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e04074040 con 0x7f2e040d24e0 2023-11-06T02:11:39.303 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.297+0000 7f2dfa7fc640 1 -- 172.21.15.49:0/3289736156 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f2e0000bcf0 con 0x7f2e040d24e0 2023-11-06T02:11:39.303 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.297+0000 7f2dfa7fc640 1 --2- 172.21.15.49:0/3289736156 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2de8065a60 0x7f2de8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:39.303 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.297+0000 7f2dfa7fc640 1 -- 172.21.15.49:0/3289736156 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2e00083800 con 0x7f2e040d24e0 2023-11-06T02:11:39.303 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.297+0000 7f2e09026640 1 --2- 172.21.15.49:0/3289736156 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2de8065a60 0x7f2de8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:39.304 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.297+0000 7f2e09026640 1 --2- 172.21.15.49:0/3289736156 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2de8065a60 0x7f2de8067f20 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f2df40b2000 tx=0x7f2df40aa070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:39.304 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.301+0000 7f2dfa7fc640 1 -- 172.21.15.49:0/3289736156 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f2e000510f0 con 0x7f2e040d24e0 2023-11-06T02:11:39.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.509+0000 7f2e0ba8b640 1 -- 172.21.15.49:0/3289736156 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f2e0415bbf0 con 0x7f2e040d24e0 2023-11-06T02:11:39.516 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.517+0000 7f2dfa7fc640 1 -- 172.21.15.49:0/3289736156 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+0 (secure 0 0 0) 0x7f2e00054710 con 0x7f2e040d24e0 2023-11-06T02:11:39.516 INFO:teuthology.orchestra.run.smithi049.stderr:updated caps for client.0 2023-11-06T02:11:39.518 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.521+0000 7f2e0ba8b640 1 -- 172.21.15.49:0/3289736156 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2de8065a60 msgr2=0x7f2de8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:39.519 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.521+0000 7f2e0ba8b640 1 --2- 172.21.15.49:0/3289736156 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2de8065a60 0x7f2de8067f20 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f2df40b2000 tx=0x7f2df40aa070 comp rx=0 tx=0).stop 2023-11-06T02:11:39.519 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.521+0000 7f2e0ba8b640 1 -- 172.21.15.49:0/3289736156 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e040d24e0 msgr2=0x7f2e040c8870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:39.519 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.521+0000 7f2e0ba8b640 1 --2- 172.21.15.49:0/3289736156 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e040d24e0 0x7f2e040c8870 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f2e0000d800 tx=0x7f2e0000dcc0 comp rx=0 tx=0).stop 2023-11-06T02:11:39.519 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.521+0000 7f2e0ba8b640 1 -- 172.21.15.49:0/3289736156 shutdown_connections 2023-11-06T02:11:39.519 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.521+0000 7f2e0ba8b640 1 --2- 172.21.15.49:0/3289736156 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2de8065a60 0x7f2de8067f20 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:39.520 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.521+0000 7f2e0ba8b640 1 --2- 172.21.15.49:0/3289736156 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2e0415cbb0 0x7f2e040abf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:39.520 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.521+0000 7f2e0ba8b640 1 --2- 172.21.15.49:0/3289736156 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2e040d2fa0 0x7f2e040aba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:39.520 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.521+0000 7f2e0ba8b640 1 --2- 172.21.15.49:0/3289736156 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e040d24e0 0x7f2e040c8870 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:39.520 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.521+0000 7f2e0ba8b640 1 -- 172.21.15.49:0/3289736156 >> 172.21.15.49:0/3289736156 conn(0x7f2e040718a0 msgr2=0x7f2e0415c140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:39.520 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.521+0000 7f2e0ba8b640 1 -- 172.21.15.49:0/3289736156 shutdown_connections 2023-11-06T02:11:39.521 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.521+0000 7f2e0ba8b640 1 -- 172.21.15.49:0/3289736156 wait complete. 2023-11-06T02:11:39.533 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T02:11:39.653 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.657+0000 7ff9eceea640 1 Processor -- start 2023-11-06T02:11:39.654 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.657+0000 7ff9eceea640 1 -- start start 2023-11-06T02:11:39.655 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.657+0000 7ff9eceea640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff9e80b0790 0x7ff9e80c8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:39.655 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.657+0000 7ff9eceea640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff9e80adae0 0x7ff9e8070b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:39.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.657+0000 7ff9eceea640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9e81594d0 con 0x7ff9e81590f0 2023-11-06T02:11:39.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.657+0000 7ff9eceea640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9e815a140 con 0x7ff9e80adae0 2023-11-06T02:11:39.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.657+0000 7ff9eceea640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e80c92e0 con 0x7ff9e80b0790 2023-11-06T02:11:39.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.657+0000 7ff9e6d9d640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff9e80adae0 0x7ff9e8070b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:39.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.657+0000 7ff9e6d9d640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff9e80adae0 0x7ff9e8070b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49408/0 (socket says 172.21.15.49:49408) 2023-11-06T02:11:39.657 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.657+0000 7ff9e6d9d640 1 -- 172.21.15.49:0/4193907289 learned_addr learned my addr 172.21.15.49:0/4193907289 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:39.657 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.657+0000 7ff9e5d9b640 1 --2- 172.21.15.49:0/4193907289 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff9e80b0790 0x7ff9e80c8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:39.657 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.657+0000 7ff9e6d9d640 1 -- 172.21.15.49:0/4193907289 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff9e80b0790 msgr2=0x7ff9e80c8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:39.657 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.657+0000 7ff9e6d9d640 1 --2- 172.21.15.49:0/4193907289 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff9e80b0790 0x7ff9e80c8c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:39.658 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.657+0000 7ff9e6d9d640 1 -- 172.21.15.49:0/4193907289 >> v1:172.21.15.150:6789/0 conn(0x7ff9e81590f0 legacy=0x7ff9e80b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:39.658 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.657+0000 7ff9e6d9d640 1 -- 172.21.15.49:0/4193907289 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9e81597a0 con 0x7ff9e80adae0 2023-11-06T02:11:39.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.661+0000 7ff9e6d9d640 1 --2- 172.21.15.49:0/4193907289 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9e80adae0 0x7ff9e8070b20 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7ff9e0009e50 tx=0x7ff9e001c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8843d56799d1fdbc server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:39.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.661+0000 7ff9e559a640 1 -- 172.21.15.49:0/4193907289 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e00a64c0 con 0x7ff9e80adae0 2023-11-06T02:11:39.662 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.661+0000 7ff9e559a640 1 -- 172.21.15.49:0/4193907289 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9e00a6620 con 0x7ff9e80adae0 2023-11-06T02:11:39.662 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.661+0000 7ff9e559a640 1 -- 172.21.15.49:0/4193907289 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e00a69b0 con 0x7ff9e80adae0 2023-11-06T02:11:39.662 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.661+0000 7ff9eceea640 1 -- 172.21.15.49:0/4193907289 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9e80adae0 msgr2=0x7ff9e8070b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:39.662 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.661+0000 7ff9eceea640 1 --2- 172.21.15.49:0/4193907289 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9e80adae0 0x7ff9e8070b20 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7ff9e0009e50 tx=0x7ff9e001c980 comp rx=0 tx=0).stop 2023-11-06T02:11:39.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.661+0000 7ff9eceea640 1 -- 172.21.15.49:0/4193907289 shutdown_connections 2023-11-06T02:11:39.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.661+0000 7ff9eceea640 1 --2- 172.21.15.49:0/4193907289 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff9e80b0790 0x7ff9e80c8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:39.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.661+0000 7ff9eceea640 1 --2- 172.21.15.49:0/4193907289 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9e80adae0 0x7ff9e8070b20 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:39.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.661+0000 7ff9eceea640 1 -- 172.21.15.49:0/4193907289 >> 172.21.15.49:0/4193907289 conn(0x7ff9e806c0a0 msgr2=0x7ff9e806c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:39.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.661+0000 7ff9eceea640 1 -- 172.21.15.49:0/4193907289 shutdown_connections 2023-11-06T02:11:39.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.661+0000 7ff9eceea640 1 -- 172.21.15.49:0/4193907289 wait complete. 2023-11-06T02:11:39.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.661+0000 7ff9eceea640 1 Processor -- start 2023-11-06T02:11:39.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.661+0000 7ff9eceea640 1 -- start start 2023-11-06T02:11:39.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.661+0000 7ff9eceea640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff9e80adae0 0x7ff9e8151730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:39.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.661+0000 7ff9eceea640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff9e80b0790 0x7ff9e8155ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:39.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.661+0000 7ff9eceea640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9e81590f0 0x7ff9e80af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:39.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.661+0000 7ff9eceea640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e814e860 con 0x7ff9e81590f0 2023-11-06T02:11:39.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.661+0000 7ff9eceea640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e81512a0 con 0x7ff9e80adae0 2023-11-06T02:11:39.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.661+0000 7ff9eceea640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e81503d0 con 0x7ff9e80b0790 2023-11-06T02:11:39.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.661+0000 7ff9e6d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9e81590f0 0x7ff9e80af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:39.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.661+0000 7ff9e6d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9e81590f0 0x7ff9e80af8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49416/0 (socket says 172.21.15.49:49416) 2023-11-06T02:11:39.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.661+0000 7ff9e6d9d640 1 -- 172.21.15.49:0/2996381146 learned_addr learned my addr 172.21.15.49:0/2996381146 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:39.667 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.661+0000 7ff9e659c640 1 --2- 172.21.15.49:0/2996381146 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff9e80adae0 0x7ff9e8151730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:39.667 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.661+0000 7ff9e5d9b640 1 --2- 172.21.15.49:0/2996381146 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff9e80b0790 0x7ff9e8155ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:39.667 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.661+0000 7ff9e6d9d640 1 -- 172.21.15.49:0/2996381146 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff9e80b0790 msgr2=0x7ff9e8155ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:39.667 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.661+0000 7ff9e6d9d640 1 --2- 172.21.15.49:0/2996381146 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff9e80b0790 0x7ff9e8155ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:39.668 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.661+0000 7ff9e6d9d640 1 -- 172.21.15.49:0/2996381146 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff9e80adae0 msgr2=0x7ff9e8151730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:39.668 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.661+0000 7ff9e6d9d640 1 --2- 172.21.15.49:0/2996381146 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff9e80adae0 0x7ff9e8151730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:39.668 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.661+0000 7ff9e6d9d640 1 -- 172.21.15.49:0/2996381146 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9e0009b30 con 0x7ff9e81590f0 2023-11-06T02:11:39.668 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.665+0000 7ff9e6d9d640 1 --2- 172.21.15.49:0/2996381146 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9e81590f0 0x7ff9e80af8d0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7ff9e001a5b0 tx=0x7ff9e0019720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:39.668 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.665+0000 7ff9d6ffd640 1 -- 172.21.15.49:0/2996381146 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e00b3040 con 0x7ff9e81590f0 2023-11-06T02:11:39.669 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.665+0000 7ff9d6ffd640 1 -- 172.21.15.49:0/2996381146 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9e00a6ba0 con 0x7ff9e81590f0 2023-11-06T02:11:39.669 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.665+0000 7ff9eceea640 1 -- 172.21.15.49:0/2996381146 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9e8150690 con 0x7ff9e81590f0 2023-11-06T02:11:39.669 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.665+0000 7ff9d6ffd640 1 -- 172.21.15.49:0/2996381146 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e0009250 con 0x7ff9e81590f0 2023-11-06T02:11:39.670 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.665+0000 7ff9eceea640 1 -- 172.21.15.49:0/2996381146 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9e8150c50 con 0x7ff9e81590f0 2023-11-06T02:11:39.671 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.665+0000 7ff9eceea640 1 -- 172.21.15.49:0/2996381146 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9e8074040 con 0x7ff9e81590f0 2023-11-06T02:11:39.671 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.665+0000 7ff9d6ffd640 1 -- 172.21.15.49:0/2996381146 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7ff9e00af480 con 0x7ff9e81590f0 2023-11-06T02:11:39.672 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.665+0000 7ff9d6ffd640 1 --2- 172.21.15.49:0/2996381146 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff9c8065a60 0x7ff9c8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:39.672 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.665+0000 7ff9d6ffd640 1 -- 172.21.15.49:0/2996381146 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff9e00aa070 con 0x7ff9e81590f0 2023-11-06T02:11:39.672 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.665+0000 7ff9e659c640 1 --2- 172.21.15.49:0/2996381146 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff9c8065a60 0x7ff9c8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:39.672 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.665+0000 7ff9e659c640 1 --2- 172.21.15.49:0/2996381146 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff9c8065a60 0x7ff9c8067f20 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff9e81f64f0 tx=0x7ff9dc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:39.673 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.669+0000 7ff9d6ffd640 1 -- 172.21.15.49:0/2996381146 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7ff9e00ebaa0 con 0x7ff9e81590f0 2023-11-06T02:11:39.855 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.857+0000 7ff9eceea640 1 -- 172.21.15.49:0/2996381146 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff9e80cba50 con 0x7ff9e81590f0 2023-11-06T02:11:39.856 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.857+0000 7ff9d6ffd640 1 -- 172.21.15.49:0/2996381146 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+4652 (secure 0 0 0) 0x7ff9e00ef0c0 con 0x7ff9e81590f0 2023-11-06T02:11:39.856 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:11:39.857 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":30,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5635,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4170291905","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":4170291905},{"type":"v1","addr":"172.21.15.150:6835","nonce":4170291905}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5638,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/2719458937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":2719458937},{"type":"v1","addr":"172.21.15.150:6837","nonce":2719458937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":6339,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/2310338375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2310338375},{"type":"v1","addr":"172.21.15.49:6833","nonce":2310338375}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T02:11:30.091070+0000","modified":"2023-11-06T02:11:33.173821+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6348},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6348":{"gid":6348,"name":"c","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.49:6835/342908487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":342908487},{"type":"v1","addr":"172.21.15.49:6835","nonce":342908487}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-06T02:11:39.857 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 30 2023-11-06T02:11:39.859 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.861+0000 7ff9eceea640 1 -- 172.21.15.49:0/2996381146 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff9c8065a60 msgr2=0x7ff9c8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:39.859 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.861+0000 7ff9eceea640 1 --2- 172.21.15.49:0/2996381146 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff9c8065a60 0x7ff9c8067f20 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff9e81f64f0 tx=0x7ff9dc009040 comp rx=0 tx=0).stop 2023-11-06T02:11:39.859 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.861+0000 7ff9eceea640 1 -- 172.21.15.49:0/2996381146 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9e81590f0 msgr2=0x7ff9e80af8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:39.859 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.861+0000 7ff9eceea640 1 --2- 172.21.15.49:0/2996381146 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9e81590f0 0x7ff9e80af8d0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7ff9e001a5b0 tx=0x7ff9e0019720 comp rx=0 tx=0).stop 2023-11-06T02:11:39.860 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.861+0000 7ff9eceea640 1 -- 172.21.15.49:0/2996381146 shutdown_connections 2023-11-06T02:11:39.860 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.861+0000 7ff9eceea640 1 --2- 172.21.15.49:0/2996381146 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff9c8065a60 0x7ff9c8067f20 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:39.860 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.861+0000 7ff9eceea640 1 --2- 172.21.15.49:0/2996381146 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9e81590f0 0x7ff9e80af8d0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:39.860 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.861+0000 7ff9eceea640 1 --2- 172.21.15.49:0/2996381146 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff9e80b0790 0x7ff9e8155ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:39.860 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.861+0000 7ff9eceea640 1 --2- 172.21.15.49:0/2996381146 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff9e80adae0 0x7ff9e8151730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:39.860 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.861+0000 7ff9eceea640 1 -- 172.21.15.49:0/2996381146 >> 172.21.15.49:0/2996381146 conn(0x7ff9e806c0a0 msgr2=0x7ff9e80cc1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:39.861 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.861+0000 7ff9eceea640 1 -- 172.21.15.49:0/2996381146 shutdown_connections 2023-11-06T02:11:39.861 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:39.861+0000 7ff9eceea640 1 -- 172.21.15.49:0/2996381146 wait complete. 2023-11-06T02:11:40.277 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 30, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-06T02:11:30.091070+0000', 'modified': '2023-11-06T02:11:33.173821+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6348}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6348': {'gid': 6348, 'name': 'c', 'rank': 0, 'incarnation': 27, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.49:6835/342908487', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6834', 'nonce': 342908487}, {'type': 'v1', 'addr': '172.21.15.49:6835', 'nonce': 342908487}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-06T02:11:40.278 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-06T02:11:40.278 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-06T02:11:40.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.413+0000 7f8cb8a8b640 1 Processor -- start 2023-11-06T02:11:40.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.413+0000 7f8cb8a8b640 1 -- start start 2023-11-06T02:11:40.411 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.413+0000 7f8cb8a8b640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f8cb40b1f90 0x7f8cb40cda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:40.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.413+0000 7f8cb8a8b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8cb4161650 0x7f8cb40d1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:40.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.413+0000 7f8cb8a8b640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f8cb40d2190 0x7f8cb40714a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:40.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.413+0000 7f8cb8a8b640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cb40d17c0 con 0x7f8cb40b1f90 2023-11-06T02:11:40.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.413+0000 7f8cb8a8b640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cb40ce6c0 con 0x7f8cb40d2190 2023-11-06T02:11:40.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.413+0000 7f8cb8a8b640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb4161ad0 con 0x7f8cb4161650 2023-11-06T02:11:40.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.413+0000 7f8cb259c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f8cb40b1f90 0x7f8cb40cda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:40.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.413+0000 7f8cb259c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f8cb40b1f90 0x7f8cb40cda70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49432/0 (socket says 172.21.15.49:49432) 2023-11-06T02:11:40.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.413+0000 7f8cb259c640 1 -- 172.21.15.49:0/3298131308 learned_addr learned my addr 172.21.15.49:0/3298131308 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:40.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.413+0000 7f8cb2d9d640 1 --2- 172.21.15.49:0/3298131308 >> v2:172.21.15.150:3300/0 conn(0x7f8cb40d2190 0x7f8cb40714a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:40.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.413+0000 7f8cb259c640 1 -- 172.21.15.49:0/3298131308 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8cb4161650 msgr2=0x7f8cb40d1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:40.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.413+0000 7f8cb1d9b640 1 --2- 172.21.15.49:0/3298131308 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8cb4161650 0x7f8cb40d1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:40.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.413+0000 7f8cb259c640 1 --2- 172.21.15.49:0/3298131308 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8cb4161650 0x7f8cb40d1c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:40.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.413+0000 7f8cb259c640 1 -- 172.21.15.49:0/3298131308 >> v2:172.21.15.150:3300/0 conn(0x7f8cb40d2190 msgr2=0x7f8cb40714a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:40.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.413+0000 7f8cb259c640 1 --2- 172.21.15.49:0/3298131308 >> v2:172.21.15.150:3300/0 conn(0x7f8cb40d2190 0x7f8cb40714a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:40.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.413+0000 7f8cb259c640 1 -- 172.21.15.49:0/3298131308 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cb40ce100 con 0x7f8cb40b1f90 2023-11-06T02:11:40.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.413+0000 7f8cb2d9d640 1 --2- 172.21.15.49:0/3298131308 >> v2:172.21.15.150:3300/0 conn(0x7f8cb40d2190 0x7f8cb40714a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:11:40.417 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.417+0000 7f8cb259c640 1 --2- 172.21.15.49:0/3298131308 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8cb40b1f90 0x7f8cb40cda70 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f8cb406c560 tx=0x7f8ca401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7c7c6391a31e54d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:40.418 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.417+0000 7f8cb159a640 1 -- 172.21.15.49:0/3298131308 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca40ac600 con 0x7f8cb40b1f90 2023-11-06T02:11:40.418 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.417+0000 7f8cb159a640 1 -- 172.21.15.49:0/3298131308 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ca40a8050 con 0x7f8cb40b1f90 2023-11-06T02:11:40.418 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.417+0000 7f8cb159a640 1 -- 172.21.15.49:0/3298131308 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca40ac980 con 0x7f8cb40b1f90 2023-11-06T02:11:40.418 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.417+0000 7f8cb8a8b640 1 -- 172.21.15.49:0/3298131308 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8cb40b1f90 msgr2=0x7f8cb40cda70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:40.418 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.417+0000 7f8cb8a8b640 1 --2- 172.21.15.49:0/3298131308 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8cb40b1f90 0x7f8cb40cda70 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f8cb406c560 tx=0x7f8ca401c4d0 comp rx=0 tx=0).stop 2023-11-06T02:11:40.419 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.417+0000 7f8cb8a8b640 1 -- 172.21.15.49:0/3298131308 shutdown_connections 2023-11-06T02:11:40.419 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.417+0000 7f8cb8a8b640 1 --2- 172.21.15.49:0/3298131308 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8cb4161650 0x7f8cb40d1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:40.419 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.417+0000 7f8cb8a8b640 1 --2- 172.21.15.49:0/3298131308 >> v2:172.21.15.150:3300/0 conn(0x7f8cb40d2190 0x7f8cb40714a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:40.419 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.417+0000 7f8cb8a8b640 1 --2- 172.21.15.49:0/3298131308 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8cb40b1f90 0x7f8cb40cda70 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:40.419 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.417+0000 7f8cb8a8b640 1 -- 172.21.15.49:0/3298131308 >> 172.21.15.49:0/3298131308 conn(0x7f8cb40d2aa0 msgr2=0x7f8cb40d2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:40.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.417+0000 7f8cb8a8b640 1 -- 172.21.15.49:0/3298131308 shutdown_connections 2023-11-06T02:11:40.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.417+0000 7f8cb8a8b640 1 -- 172.21.15.49:0/3298131308 wait complete. 2023-11-06T02:11:40.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.417+0000 7f8cb8a8b640 1 Processor -- start 2023-11-06T02:11:40.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.417+0000 7f8cb8a8b640 1 -- start start 2023-11-06T02:11:40.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.417+0000 7f8cb8a8b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8cb40b1f90 0x7f8cb41527a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:40.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.417+0000 7f8cb8a8b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8cb40d2190 0x7f8cb414b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:40.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.417+0000 7f8cb8a8b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8cb4161650 0x7f8cb414b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:40.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.417+0000 7f8cb8a8b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb41504d0 con 0x7f8cb40b1f90 2023-11-06T02:11:40.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.417+0000 7f8cb8a8b640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb414fec0 con 0x7f8cb40d2190 2023-11-06T02:11:40.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.417+0000 7f8cb8a8b640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb4159640 con 0x7f8cb4161650 2023-11-06T02:11:40.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.417+0000 7f8cb259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8cb40b1f90 0x7f8cb41527a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:40.422 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.417+0000 7f8cb259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8cb40b1f90 0x7f8cb41527a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49442/0 (socket says 172.21.15.49:49442) 2023-11-06T02:11:40.422 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.417+0000 7f8cb259c640 1 -- 172.21.15.49:0/764612422 learned_addr learned my addr 172.21.15.49:0/764612422 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:40.422 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.417+0000 7f8cb1d9b640 1 --2- 172.21.15.49:0/764612422 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8cb40d2190 0x7f8cb414b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:40.422 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.417+0000 7f8cb2d9d640 1 --2- 172.21.15.49:0/764612422 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8cb4161650 0x7f8cb414b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:40.422 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.421+0000 7f8cb259c640 1 -- 172.21.15.49:0/764612422 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8cb4161650 msgr2=0x7f8cb414b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:40.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.421+0000 7f8cb259c640 1 --2- 172.21.15.49:0/764612422 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8cb4161650 0x7f8cb414b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:40.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.421+0000 7f8cb259c640 1 -- 172.21.15.49:0/764612422 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8cb40d2190 msgr2=0x7f8cb414b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:40.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.421+0000 7f8cb259c640 1 --2- 172.21.15.49:0/764612422 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8cb40d2190 0x7f8cb414b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:40.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.421+0000 7f8cb259c640 1 -- 172.21.15.49:0/764612422 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ca40094d0 con 0x7f8cb40b1f90 2023-11-06T02:11:40.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.421+0000 7f8cb2d9d640 1 --2- 172.21.15.49:0/764612422 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8cb4161650 0x7f8cb414b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:11:40.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.421+0000 7f8cb259c640 1 --2- 172.21.15.49:0/764612422 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8cb40b1f90 0x7f8cb41527a0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f8ca401ab80 tx=0x7f8ca401ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:40.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.421+0000 7f8caaffd640 1 -- 172.21.15.49:0/764612422 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca40bc070 con 0x7f8cb40b1f90 2023-11-06T02:11:40.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.421+0000 7f8caaffd640 1 -- 172.21.15.49:0/764612422 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ca40b7450 con 0x7f8cb40b1f90 2023-11-06T02:11:40.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.421+0000 7f8cb8a8b640 1 -- 172.21.15.49:0/764612422 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cb4150160 con 0x7f8cb40b1f90 2023-11-06T02:11:40.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.421+0000 7f8caaffd640 1 -- 172.21.15.49:0/764612422 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca40b77c0 con 0x7f8cb40b1f90 2023-11-06T02:11:40.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.421+0000 7f8cb8a8b640 1 -- 172.21.15.49:0/764612422 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cb4152ce0 con 0x7f8cb40b1f90 2023-11-06T02:11:40.430 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.421+0000 7f8caaffd640 1 -- 172.21.15.49:0/764612422 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f8ca40b3070 con 0x7f8cb40b1f90 2023-11-06T02:11:40.431 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.421+0000 7f8caaffd640 1 --2- 172.21.15.49:0/764612422 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8c94065a60 0x7f8c94067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:40.431 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.421+0000 7f8caaffd640 1 -- 172.21.15.49:0/764612422 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8ca411d0a0 con 0x7f8cb40b1f90 2023-11-06T02:11:40.431 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.421+0000 7f8cb1d9b640 1 --2- 172.21.15.49:0/764612422 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8c94065a60 0x7f8c94067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:40.431 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.421+0000 7f8cbbbb3000 1 Processor -- start 2023-11-06T02:11:40.431 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.421+0000 7f8cbbbb3000 1 -- start start 2023-11-06T02:11:40.432 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.421+0000 7f8cb1d9b640 1 --2- 172.21.15.49:0/764612422 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8c94065a60 0x7f8c94067f20 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c0070b0 tx=0x7f8c9c007040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:40.432 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.425+0000 7f8cbbbb3000 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5576859816e0 0x557685981ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:40.432 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.425+0000 7f8cbbbb3000 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557685982000 0x557685937550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:40.432 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.425+0000 7f8cbbbb3000 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557685937a90 0x557685937e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:40.432 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.425+0000 7f8cbbbb3000 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576857fffc0 con 0x557685937a90 2023-11-06T02:11:40.433 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.425+0000 7f8cbbbb3000 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55768578e1a0 con 0x557685982000 2023-11-06T02:11:40.433 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.425+0000 7f8cbbbb3000 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55768578b770 con 0x5576859816e0 2023-11-06T02:11:40.433 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.425+0000 7f8cb2d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557685937a90 0x557685937e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:40.433 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.425+0000 7f8cb2d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557685937a90 0x557685937e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49448/0 (socket says 172.21.15.49:49448) 2023-11-06T02:11:40.433 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.425+0000 7f8cb2d9d640 1 -- 172.21.15.49:0/2536443085 learned_addr learned my addr 172.21.15.49:0/2536443085 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:40.434 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.425+0000 7f8cb259c640 1 --2- 172.21.15.49:0/2536443085 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5576859816e0 0x557685981ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:40.434 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.425+0000 7f8cb1d9b640 1 --2- 172.21.15.49:0/2536443085 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557685982000 0x557685937550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:40.434 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.425+0000 7f8cb2d9d640 1 -- 172.21.15.49:0/2536443085 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5576859816e0 msgr2=0x557685981ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:40.434 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.425+0000 7f8cb2d9d640 1 --2- 172.21.15.49:0/2536443085 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5576859816e0 0x557685981ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:40.434 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.425+0000 7f8cb2d9d640 1 -- 172.21.15.49:0/2536443085 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557685982000 msgr2=0x557685937550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:40.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.425+0000 7f8cb2d9d640 1 --2- 172.21.15.49:0/2536443085 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557685982000 0x557685937550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:40.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.425+0000 7f8cb2d9d640 1 -- 172.21.15.49:0/2536443085 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557685483890 con 0x557685937a90 2023-11-06T02:11:40.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.425+0000 7f8cb1d9b640 1 --2- 172.21.15.49:0/2536443085 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557685982000 0x557685937550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:11:40.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.425+0000 7f8cb259c640 1 --2- 172.21.15.49:0/2536443085 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5576859816e0 0x557685981ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:11:40.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.425+0000 7f8cb2d9d640 1 --2- 172.21.15.49:0/2536443085 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557685937a90 0x557685937e70 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f8cac00b9d0 tx=0x7f8cac00be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a03c0771092bab37 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:40.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.425+0000 7f8c93fff640 1 -- 172.21.15.49:0/2536443085 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cac00dcd0 con 0x557685937a90 2023-11-06T02:11:40.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.425+0000 7f8c93fff640 1 -- 172.21.15.49:0/2536443085 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cac00c040 con 0x557685937a90 2023-11-06T02:11:40.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.425+0000 7f8c93fff640 1 -- 172.21.15.49:0/2536443085 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cac012660 con 0x557685937a90 2023-11-06T02:11:40.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.425+0000 7f8cbbbb3000 1 -- 172.21.15.49:0/2536443085 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557685937a90 msgr2=0x557685937e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:40.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.425+0000 7f8cbbbb3000 1 --2- 172.21.15.49:0/2536443085 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557685937a90 0x557685937e70 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f8cac00b9d0 tx=0x7f8cac00be90 comp rx=0 tx=0).stop 2023-11-06T02:11:40.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.425+0000 7f8cbbbb3000 1 -- 172.21.15.49:0/2536443085 shutdown_connections 2023-11-06T02:11:40.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.425+0000 7f8cbbbb3000 1 --2- 172.21.15.49:0/2536443085 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557685937a90 0x557685937e70 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:40.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.425+0000 7f8cbbbb3000 1 --2- 172.21.15.49:0/2536443085 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557685982000 0x557685937550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:40.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.425+0000 7f8cbbbb3000 1 --2- 172.21.15.49:0/2536443085 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5576859816e0 0x557685981ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:40.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.425+0000 7f8cbbbb3000 1 -- 172.21.15.49:0/2536443085 >> 172.21.15.49:0/2536443085 conn(0x55768593dbf0 msgr2=0x557685935e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:40.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.425+0000 7f8cbbbb3000 1 -- 172.21.15.49:0/2536443085 shutdown_connections 2023-11-06T02:11:40.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.425+0000 7f8cbbbb3000 1 -- 172.21.15.49:0/2536443085 wait complete. 2023-11-06T02:11:40.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.425+0000 7f8cbbbb3000 1 -- start start 2023-11-06T02:11:40.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.425+0000 7f8cbbbb3000 1 Processor -- start 2023-11-06T02:11:40.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.425+0000 7f8cbbbb3000 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557685937a90 0x557685942a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:40.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.425+0000 7f8cbbbb3000 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5576859816e0 0x557685942fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:40.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.425+0000 7f8cbbbb3000 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557685982000 0x557685943510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:40.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.425+0000 7f8cbbbb3000 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x557685787060 con 0x557685937a90 2023-11-06T02:11:40.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.425+0000 7f8cbbbb3000 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x557685726290 con 0x557685982000 2023-11-06T02:11:40.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.425+0000 7f8cbbbb3000 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x5576857446f0 con 0x5576859816e0 2023-11-06T02:11:40.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.425+0000 7f8cbbbb3000 10 client.0 _ll_register_callbacks cb 0x5576859231d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T02:11:40.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.425+0000 7f8cb259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557685937a90 0x557685942a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:40.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.425+0000 7f8cb259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557685937a90 0x557685942a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49462/0 (socket says 172.21.15.49:49462) 2023-11-06T02:11:40.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.425+0000 7f8cb259c640 1 -- 172.21.15.49:0/111768991 learned_addr learned my addr 172.21.15.49:0/111768991 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:40.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.425+0000 7f8cb1d9b640 1 --2- 172.21.15.49:0/111768991 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5576859816e0 0x557685942fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:40.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.425+0000 7f8cb2d9d640 1 --2- 172.21.15.49:0/111768991 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557685982000 0x557685943510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:40.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.425+0000 7f8cb259c640 1 -- 172.21.15.49:0/111768991 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5576859816e0 msgr2=0x557685942fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:40.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.425+0000 7f8cb259c640 1 --2- 172.21.15.49:0/111768991 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5576859816e0 0x557685942fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:40.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.425+0000 7f8cb259c640 1 -- 172.21.15.49:0/111768991 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557685982000 msgr2=0x557685943510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:40.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.425+0000 7f8cb259c640 1 --2- 172.21.15.49:0/111768991 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557685982000 0x557685943510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:40.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.425+0000 7f8cb259c640 1 -- 172.21.15.49:0/111768991 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cac00b600 con 0x557685937a90 2023-11-06T02:11:40.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.425+0000 7f8cb2d9d640 1 --2- 172.21.15.49:0/111768991 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557685982000 0x557685943510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:11:40.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.425+0000 7f8cb259c640 1 --2- 172.21.15.49:0/111768991 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557685937a90 0x557685942a90 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f8ca40af040 tx=0x7f8ca40eee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed18c18634334812 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:40.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.425+0000 7f8c917fa640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-11-06T02:11:40.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.425+0000 7f8c917fa640 1 -- 172.21.15.49:0/111768991 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca40f20d0 con 0x557685937a90 2023-11-06T02:11:40.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.425+0000 7f8c917fa640 1 -- 172.21.15.49:0/111768991 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8ca40f2230 con 0x557685937a90 2023-11-06T02:11:40.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.425+0000 7f8cbbbb3000 1 -- 172.21.15.49:0/111768991 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5576857859b0 con 0x557685937a90 2023-11-06T02:11:40.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.425+0000 7f8c917fa640 1 -- 172.21.15.49:0/111768991 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ca411b1a0 con 0x557685937a90 2023-11-06T02:11:40.443 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.425+0000 7f8cbbbb3000 10 client.6501 fetch_fsmap learned FSMap version 30 2023-11-06T02:11:40.443 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.425+0000 7f8cbbbb3000 1 -- 172.21.15.49:0/111768991 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=2+}) v3 -- 0x55768574c1e0 con 0x557685937a90 2023-11-06T02:11:40.443 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.425+0000 7f8c917fa640 1 -- 172.21.15.49:0/111768991 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5576857859b0 con 0x557685937a90 2023-11-06T02:11:40.443 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.429+0000 7f8c917fa640 1 -- 172.21.15.49:0/111768991 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 30) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f8ca40edb40 con 0x557685937a90 2023-11-06T02:11:40.443 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.429+0000 7f8cbbbb3000 10 client.6501 fetch_fsmap finished waiting for FSMap version 30 2023-11-06T02:11:40.444 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.429+0000 7f8cbbbb3000 10 client.6501 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-06T02:11:40.444 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.429+0000 7f8cbbbb3000 20 client.6501 populate_metadata read hostname 'smithi049' 2023-11-06T02:11:40.444 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.429+0000 7f8cbbbb3000 1 --2- 172.21.15.49:0/111768991 >> [v2:172.21.15.49:6834/342908487,v1:172.21.15.49:6835/342908487] conn(0x557685943de0 0x557685992c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:40.444 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.429+0000 7f8cbbbb3000 4 client.6501 mds_command: new command op to 6348 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T02:11:40.444 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.429+0000 7f8cbbbb3000 1 -- 172.21.15.49:0/111768991 --> [v2:172.21.15.49:6834/342908487,v1:172.21.15.49:6835/342908487] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5576857b64d0 con 0x557685943de0 2023-11-06T02:11:40.444 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.429+0000 7f8cb1d9b640 1 --2- 172.21.15.49:0/111768991 >> [v2:172.21.15.49:6834/342908487,v1:172.21.15.49:6835/342908487] conn(0x557685943de0 0x557685992c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:40.445 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.429+0000 7f8c917fa640 10 client.6501 ms_handle_connect on v2:172.21.15.49:6834/342908487 2023-11-06T02:11:40.445 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.429+0000 7f8cb1d9b640 1 --2- 172.21.15.49:0/111768991 >> [v2:172.21.15.49:6834/342908487,v1:172.21.15.49:6835/342908487] conn(0x557685943de0 0x557685992c50 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63d9fcb8e957257f server_cookie=61774f1939f85472 in_seq=0 out_seq=0 2023-11-06T02:11:40.445 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.429+0000 7f8c917fa640 1 -- 172.21.15.49:0/111768991 <== mds.0 v2:172.21.15.49:6834/342908487 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5576857b64d0 con 0x557685943de0 2023-11-06T02:11:40.445 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.429+0000 7f8c917fa640 10 client.6501 handle_command_reply: tid=0 2023-11-06T02:11:40.445 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.429+0000 7f8cbbbb3000 1 client.6501 shutdown 2023-11-06T02:11:40.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.429+0000 7f8cbbbb3000 1 -- 172.21.15.49:0/111768991 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557685937a90 msgr2=0x557685942a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:40.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.429+0000 7f8cbbbb3000 1 --2- 172.21.15.49:0/111768991 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557685937a90 0x557685942a90 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f8ca40af040 tx=0x7f8ca40eee90 comp rx=0 tx=0).stop 2023-11-06T02:11:40.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.429+0000 7f8cbbbb3000 1 -- 172.21.15.49:0/111768991 shutdown_connections 2023-11-06T02:11:40.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.429+0000 7f8cbbbb3000 1 --2- 172.21.15.49:0/111768991 >> [v2:172.21.15.49:6834/342908487,v1:172.21.15.49:6835/342908487] conn(0x557685943de0 0x557685992c50 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:40.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.429+0000 7f8cbbbb3000 1 --2- 172.21.15.49:0/111768991 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557685982000 0x557685943510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:40.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.429+0000 7f8cbbbb3000 1 --2- 172.21.15.49:0/111768991 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5576859816e0 0x557685942fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:40.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.429+0000 7f8c917fa640 0 client.6501 ms_handle_reset on v2:172.21.15.49:6834/342908487 2023-11-06T02:11:40.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.429+0000 7f8cbbbb3000 1 --2- 172.21.15.49:0/111768991 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557685937a90 0x557685942a90 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:40.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.429+0000 7f8cbbbb3000 1 -- 172.21.15.49:0/111768991 >> 172.21.15.49:0/111768991 conn(0x55768593dbf0 msgr2=0x557685936530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:40.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.429+0000 7f8cbbbb3000 1 -- 172.21.15.49:0/111768991 shutdown_connections 2023-11-06T02:11:40.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.429+0000 7f8cbbbb3000 1 -- 172.21.15.49:0/111768991 wait complete. 2023-11-06T02:11:40.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.429+0000 7f8cbbbb3000 20 client.6501 trim_cache size 0 max 16384 2023-11-06T02:11:40.453 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.445+0000 7f8cbbbb3000 1 Processor -- start 2023-11-06T02:11:40.453 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.445+0000 7f8cbbbb3000 1 -- start start 2023-11-06T02:11:40.453 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.445+0000 7f8cbbbb3000 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557685937a90 0x557685994370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:40.453 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.445+0000 7f8cbbbb3000 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5576859816e0 0x5576859948b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:40.454 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.445+0000 7f8cbbbb3000 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557685982000 0x55768597a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:40.454 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.445+0000 7f8cbbbb3000 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x557685444940 con 0x557685982000 2023-11-06T02:11:40.454 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.445+0000 7f8cbbbb3000 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x557685781400 con 0x557685937a90 2023-11-06T02:11:40.454 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.445+0000 7f8cbbbb3000 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x5576856be450 con 0x5576859816e0 2023-11-06T02:11:40.454 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.445+0000 7f8cb2d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557685982000 0x55768597a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:40.455 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.445+0000 7f8cb2d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557685982000 0x55768597a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49476/0 (socket says 172.21.15.49:49476) 2023-11-06T02:11:40.455 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.445+0000 7f8cb2d9d640 1 -- 172.21.15.49:0/3603353348 learned_addr learned my addr 172.21.15.49:0/3603353348 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:40.455 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.445+0000 7f8cb2d9d640 1 -- 172.21.15.49:0/3603353348 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5576859816e0 msgr2=0x5576859948b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:40.455 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.445+0000 7f8cb2d9d640 1 --2- 172.21.15.49:0/3603353348 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5576859816e0 0x5576859948b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:40.455 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.445+0000 7f8cb2d9d640 1 -- 172.21.15.49:0/3603353348 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557685937a90 msgr2=0x557685994370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:40.455 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.445+0000 7f8cb2d9d640 1 --2- 172.21.15.49:0/3603353348 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557685937a90 0x557685994370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:40.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.445+0000 7f8cb2d9d640 1 -- 172.21.15.49:0/3603353348 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557685732330 con 0x557685982000 2023-11-06T02:11:40.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.445+0000 7f8cb2d9d640 1 --2- 172.21.15.49:0/3603353348 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557685982000 0x55768597a370 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f8cac00d190 tx=0x7f8cac012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9132bb5aeeb3d95 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:40.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.445+0000 7f8ca8ff9640 1 -- 172.21.15.49:0/3603353348 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cac020070 con 0x557685982000 2023-11-06T02:11:40.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.445+0000 7f8ca8ff9640 1 -- 172.21.15.49:0/3603353348 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cac00dad0 con 0x557685982000 2023-11-06T02:11:40.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.445+0000 7f8ca8ff9640 1 -- 172.21.15.49:0/3603353348 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cac00dec0 con 0x557685982000 2023-11-06T02:11:40.457 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.445+0000 7f8cbbbb3000 1 -- 172.21.15.49:0/3603353348 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557685982000 msgr2=0x55768597a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:40.457 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.445+0000 7f8cbbbb3000 1 --2- 172.21.15.49:0/3603353348 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557685982000 0x55768597a370 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f8cac00d190 tx=0x7f8cac012000 comp rx=0 tx=0).stop 2023-11-06T02:11:40.457 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.445+0000 7f8cbbbb3000 1 -- 172.21.15.49:0/3603353348 shutdown_connections 2023-11-06T02:11:40.457 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.445+0000 7f8cbbbb3000 1 --2- 172.21.15.49:0/3603353348 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557685982000 0x55768597a370 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:40.457 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.445+0000 7f8cbbbb3000 1 --2- 172.21.15.49:0/3603353348 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5576859816e0 0x5576859948b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:40.458 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.445+0000 7f8cbbbb3000 1 --2- 172.21.15.49:0/3603353348 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557685937a90 0x557685994370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:40.458 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.445+0000 7f8cbbbb3000 1 -- 172.21.15.49:0/3603353348 >> 172.21.15.49:0/3603353348 conn(0x55768593dbf0 msgr2=0x557685992c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:40.458 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.445+0000 7f8cbbbb3000 1 -- 172.21.15.49:0/3603353348 shutdown_connections 2023-11-06T02:11:40.458 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.445+0000 7f8cbbbb3000 1 -- 172.21.15.49:0/3603353348 wait complete. 2023-11-06T02:11:40.458 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.445+0000 7f8cbbbb3000 1 -- start start 2023-11-06T02:11:40.459 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.445+0000 7f8cbbbb3000 1 Processor -- start 2023-11-06T02:11:40.459 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.445+0000 7f8cbbbb3000 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557685937a90 0x5576859ddea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:40.459 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.445+0000 7f8cbbbb3000 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5576859816e0 0x5576859de3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:40.459 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.445+0000 7f8cbbbb3000 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557685982000 0x5576859da4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:40.459 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.445+0000 7f8cbbbb3000 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x557685756090 con 0x557685982000 2023-11-06T02:11:40.459 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.445+0000 7f8cbbbb3000 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x557685787910 con 0x557685937a90 2023-11-06T02:11:40.460 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.445+0000 7f8cbbbb3000 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x5576857400f0 con 0x5576859816e0 2023-11-06T02:11:40.460 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.445+0000 7f8cbbbb3000 10 client.0 _ll_register_callbacks cb 0x5576859039e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T02:11:40.460 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.445+0000 7f8cb2d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557685982000 0x5576859da4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:40.460 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.445+0000 7f8cb2d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557685982000 0x5576859da4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49488/0 (socket says 172.21.15.49:49488) 2023-11-06T02:11:40.460 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.445+0000 7f8cb2d9d640 1 -- 172.21.15.49:0/496660843 learned_addr learned my addr 172.21.15.49:0/496660843 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:40.461 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.449+0000 7f8cb2d9d640 1 -- 172.21.15.49:0/496660843 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5576859816e0 msgr2=0x5576859de3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:40.461 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.449+0000 7f8cb2d9d640 1 --2- 172.21.15.49:0/496660843 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5576859816e0 0x5576859de3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:40.461 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.449+0000 7f8cb259c640 1 --2- 172.21.15.49:0/496660843 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557685937a90 0x5576859ddea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:40.461 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.449+0000 7f8cb2d9d640 1 -- 172.21.15.49:0/496660843 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557685937a90 msgr2=0x5576859ddea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:40.461 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.449+0000 7f8cb2d9d640 1 --2- 172.21.15.49:0/496660843 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557685937a90 0x5576859ddea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:40.462 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.449+0000 7f8cb2d9d640 1 -- 172.21.15.49:0/496660843 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557685483890 con 0x557685982000 2023-11-06T02:11:40.462 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.449+0000 7f8cb259c640 1 --2- 172.21.15.49:0/496660843 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557685937a90 0x5576859ddea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:11:40.462 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.449+0000 7f8cb2d9d640 1 --2- 172.21.15.49:0/496660843 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557685982000 0x5576859da4e0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f8cac00d190 tx=0x7f8cac004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e45d26cdadceacd server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:40.462 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.449+0000 7f8c917fa640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-11-06T02:11:40.462 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.449+0000 7f8c917fa640 1 -- 172.21.15.49:0/496660843 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cac0129a0 con 0x557685982000 2023-11-06T02:11:40.463 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.449+0000 7f8c917fa640 1 -- 172.21.15.49:0/496660843 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8cac012b00 con 0x557685982000 2023-11-06T02:11:40.463 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.449+0000 7f8cbbbb3000 1 -- 172.21.15.49:0/496660843 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557685756b90 con 0x557685982000 2023-11-06T02:11:40.463 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.449+0000 7f8c917fa640 1 -- 172.21.15.49:0/496660843 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8cac012cc0 con 0x557685982000 2023-11-06T02:11:40.463 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.449+0000 7f8cbbbb3000 10 client.6507 fetch_fsmap learned FSMap version 30 2023-11-06T02:11:40.463 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.449+0000 7f8cbbbb3000 1 -- 172.21.15.49:0/496660843 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=2+}) v3 -- 0x557685483cc0 con 0x557685982000 2023-11-06T02:11:40.463 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.449+0000 7f8c917fa640 1 -- 172.21.15.49:0/496660843 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557685756b90 con 0x557685982000 2023-11-06T02:11:40.464 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.449+0000 7f8c917fa640 1 -- 172.21.15.49:0/496660843 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 30) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f8cac01b450 con 0x557685982000 2023-11-06T02:11:40.464 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.449+0000 7f8cbbbb3000 10 client.6507 fetch_fsmap finished waiting for FSMap version 30 2023-11-06T02:11:40.464 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.449+0000 7f8cbbbb3000 10 client.6507 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-06T02:11:40.464 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.449+0000 7f8cbbbb3000 20 client.6507 populate_metadata read hostname 'smithi049' 2023-11-06T02:11:40.464 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.449+0000 7f8cbbbb3000 1 --2- 172.21.15.49:0/496660843 >> [v2:172.21.15.49:6834/342908487,v1:172.21.15.49:6835/342908487] conn(0x5576859daa20 0x5576859dae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:40.465 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.449+0000 7f8cbbbb3000 4 client.6507 mds_command: new command op to 6348 tid=0[{"prefix": "status"}] 2023-11-06T02:11:40.465 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.449+0000 7f8cbbbb3000 1 -- 172.21.15.49:0/496660843 --> [v2:172.21.15.49:6834/342908487,v1:172.21.15.49:6835/342908487] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5576857800e0 con 0x5576859daa20 2023-11-06T02:11:40.465 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.449+0000 7f8cb259c640 1 --2- 172.21.15.49:0/496660843 >> [v2:172.21.15.49:6834/342908487,v1:172.21.15.49:6835/342908487] conn(0x5576859daa20 0x5576859dae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:40.465 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.449+0000 7f8c917fa640 10 client.6507 ms_handle_connect on v2:172.21.15.49:6834/342908487 2023-11-06T02:11:40.465 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.449+0000 7f8cb259c640 1 --2- 172.21.15.49:0/496660843 >> [v2:172.21.15.49:6834/342908487,v1:172.21.15.49:6835/342908487] conn(0x5576859daa20 0x5576859dae00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9e513f1f5706c6b server_cookie=ed965fce2081e163 in_seq=0 out_seq=0 2023-11-06T02:11:40.466 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.449+0000 7f8c917fa640 1 -- 172.21.15.49:0/496660843 <== mds.0 v2:172.21.15.49:6834/342908487 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x5576857800e0 con 0x5576859daa20 2023-11-06T02:11:40.466 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.449+0000 7f8c917fa640 10 client.6507 handle_command_reply: tid=0 2023-11-06T02:11:40.466 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.449+0000 7f8cbbbb3000 1 client.6507 shutdown 2023-11-06T02:11:40.466 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.449+0000 7f8cbbbb3000 1 -- 172.21.15.49:0/496660843 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557685982000 msgr2=0x5576859da4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:40.466 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.449+0000 7f8cbbbb3000 1 --2- 172.21.15.49:0/496660843 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557685982000 0x5576859da4e0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f8cac00d190 tx=0x7f8cac004740 comp rx=0 tx=0).stop 2023-11-06T02:11:40.466 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.449+0000 7f8cbbbb3000 1 -- 172.21.15.49:0/496660843 shutdown_connections 2023-11-06T02:11:40.467 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.449+0000 7f8cbbbb3000 1 --2- 172.21.15.49:0/496660843 >> [v2:172.21.15.49:6834/342908487,v1:172.21.15.49:6835/342908487] conn(0x5576859daa20 0x5576859dae00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:40.467 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.449+0000 7f8cbbbb3000 1 --2- 172.21.15.49:0/496660843 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557685982000 0x5576859da4e0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:40.467 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.449+0000 7f8cbbbb3000 1 --2- 172.21.15.49:0/496660843 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5576859816e0 0x5576859de3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:40.467 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.449+0000 7f8c917fa640 0 client.6507 ms_handle_reset on v2:172.21.15.49:6834/342908487 2023-11-06T02:11:40.467 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.449+0000 7f8cbbbb3000 1 --2- 172.21.15.49:0/496660843 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557685937a90 0x5576859ddea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:40.468 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.449+0000 7f8cbbbb3000 1 -- 172.21.15.49:0/496660843 >> 172.21.15.49:0/496660843 conn(0x55768593dbf0 msgr2=0x557685992c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:40.468 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.449+0000 7f8cbbbb3000 1 -- 172.21.15.49:0/496660843 shutdown_connections 2023-11-06T02:11:40.468 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.449+0000 7f8cbbbb3000 1 -- 172.21.15.49:0/496660843 wait complete. 2023-11-06T02:11:40.468 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.449+0000 7f8cbbbb3000 20 client.6507 trim_cache size 0 max 16384 2023-11-06T02:11:40.469 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-11-06T02:11:40.469 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_fsid": "1a8b035c-6ba4-499a-b876-aea47fd9c0d4", 2023-11-06T02:11:40.469 INFO:teuthology.orchestra.run.smithi049.stdout: "whoami": 0, 2023-11-06T02:11:40.469 INFO:teuthology.orchestra.run.smithi049.stdout: "id": 6348, 2023-11-06T02:11:40.470 INFO:teuthology.orchestra.run.smithi049.stdout: "want_state": "up:active", 2023-11-06T02:11:40.470 INFO:teuthology.orchestra.run.smithi049.stdout: "state": "up:active", 2023-11-06T02:11:40.470 INFO:teuthology.orchestra.run.smithi049.stdout: "fs_name": "cephfs", 2023-11-06T02:11:40.470 INFO:teuthology.orchestra.run.smithi049.stdout: "rank_uptime": 10.357381759000001, 2023-11-06T02:11:40.470 INFO:teuthology.orchestra.run.smithi049.stdout: "mdsmap_epoch": 30, 2023-11-06T02:11:40.471 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch": 65, 2023-11-06T02:11:40.471 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch_barrier": 65, 2023-11-06T02:11:40.471 INFO:teuthology.orchestra.run.smithi049.stdout: "uptime": 14.868723396 2023-11-06T02:11:40.471 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-11-06T02:11:40.474 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.449+0000 7f8cb8a8b640 1 -- 172.21.15.49:0/764612422 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8c94065a60 msgr2=0x7f8c94067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:40.474 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.449+0000 7f8cb8a8b640 1 --2- 172.21.15.49:0/764612422 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8c94065a60 0x7f8c94067f20 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c0070b0 tx=0x7f8c9c007040 comp rx=0 tx=0).stop 2023-11-06T02:11:40.474 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.449+0000 7f8cb8a8b640 1 -- 172.21.15.49:0/764612422 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8cb40b1f90 msgr2=0x7f8cb41527a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:40.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.449+0000 7f8cb8a8b640 1 --2- 172.21.15.49:0/764612422 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8cb40b1f90 0x7f8cb41527a0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f8ca401ab80 tx=0x7f8ca401ad10 comp rx=0 tx=0).stop 2023-11-06T02:11:40.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.449+0000 7f8cb8a8b640 1 -- 172.21.15.49:0/764612422 shutdown_connections 2023-11-06T02:11:40.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.449+0000 7f8cb8a8b640 1 --2- 172.21.15.49:0/764612422 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8c94065a60 0x7f8c94067f20 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:40.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.449+0000 7f8cb8a8b640 1 --2- 172.21.15.49:0/764612422 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8cb4161650 0x7f8cb414b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:40.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.449+0000 7f8cb8a8b640 1 --2- 172.21.15.49:0/764612422 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8cb40d2190 0x7f8cb414b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:40.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.449+0000 7f8cb8a8b640 1 --2- 172.21.15.49:0/764612422 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8cb40b1f90 0x7f8cb41527a0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:40.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.449+0000 7f8cb8a8b640 1 -- 172.21.15.49:0/764612422 >> 172.21.15.49:0/764612422 conn(0x7f8cb40d2aa0 msgr2=0x7f8cb40b7000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:40.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.449+0000 7f8cb8a8b640 1 -- 172.21.15.49:0/764612422 shutdown_connections 2023-11-06T02:11:40.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.449+0000 7f8cb8a8b640 1 -- 172.21.15.49:0/764612422 wait complete. 2023-11-06T02:11:40.477 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-06T02:11:40.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.593+0000 7f69aa000640 1 Processor -- start 2023-11-06T02:11:40.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.593+0000 7f69aa000640 1 -- start start 2023-11-06T02:11:40.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.593+0000 7f69aa000640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f69a4071120 0x7f69a4071500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:40.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.593+0000 7f69aa000640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f69a40c9c80 0x7f69a40b2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:40.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.593+0000 7f69aa000640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f69a40b5270 0x7f69a40ca640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:40.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.593+0000 7f69aa000640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69a40ae9c0 con 0x7f69a4071120 2023-11-06T02:11:40.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.593+0000 7f69aa000640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69a40af010 con 0x7f69a40c9c80 2023-11-06T02:11:40.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.593+0000 7f69aa000640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69a4072090 con 0x7f69a40b5270 2023-11-06T02:11:40.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.593+0000 7f69a37fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f69a4071120 0x7f69a4071500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:40.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.593+0000 7f69a37fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f69a4071120 0x7f69a4071500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49504/0 (socket says 172.21.15.49:49504) 2023-11-06T02:11:40.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.593+0000 7f69a37fe640 1 -- 172.21.15.49:0/2228530574 learned_addr learned my addr 172.21.15.49:0/2228530574 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:40.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.593+0000 7f69a2ffd640 1 --2- 172.21.15.49:0/2228530574 >> v2:172.21.15.150:3300/0 conn(0x7f69a40c9c80 0x7f69a40b2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:40.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.593+0000 7f69a3fff640 1 --2- 172.21.15.49:0/2228530574 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f69a40b5270 0x7f69a40ca640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:40.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.593+0000 7f69a37fe640 1 -- 172.21.15.49:0/2228530574 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f69a40b5270 msgr2=0x7f69a40ca640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:40.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.593+0000 7f69a37fe640 1 --2- 172.21.15.49:0/2228530574 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f69a40b5270 0x7f69a40ca640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:40.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.593+0000 7f69a37fe640 1 -- 172.21.15.49:0/2228530574 >> v2:172.21.15.150:3300/0 conn(0x7f69a40c9c80 msgr2=0x7f69a40b2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:40.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.593+0000 7f69a37fe640 1 --2- 172.21.15.49:0/2228530574 >> v2:172.21.15.150:3300/0 conn(0x7f69a40c9c80 0x7f69a40b2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:40.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.593+0000 7f69a37fe640 1 -- 172.21.15.49:0/2228530574 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69a4071a40 con 0x7f69a4071120 2023-11-06T02:11:40.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.593+0000 7f69a37fe640 1 --2- 172.21.15.49:0/2228530574 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69a4071120 0x7f69a4071500 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f69a406cd80 tx=0x7f699401ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ff87c50e573d20c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:40.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.597+0000 7f69a27fc640 1 -- 172.21.15.49:0/2228530574 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69940a89b0 con 0x7f69a4071120 2023-11-06T02:11:40.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.597+0000 7f69a27fc640 1 -- 172.21.15.49:0/2228530574 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69940a8b10 con 0x7f69a4071120 2023-11-06T02:11:40.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.597+0000 7f69a27fc640 1 -- 172.21.15.49:0/2228530574 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69940a8ea0 con 0x7f69a4071120 2023-11-06T02:11:40.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.597+0000 7f69aa000640 1 -- 172.21.15.49:0/2228530574 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69a4071120 msgr2=0x7f69a4071500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:40.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.597+0000 7f69aa000640 1 --2- 172.21.15.49:0/2228530574 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69a4071120 0x7f69a4071500 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f69a406cd80 tx=0x7f699401ecf0 comp rx=0 tx=0).stop 2023-11-06T02:11:40.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.597+0000 7f69aa000640 1 -- 172.21.15.49:0/2228530574 shutdown_connections 2023-11-06T02:11:40.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.597+0000 7f69aa000640 1 --2- 172.21.15.49:0/2228530574 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f69a40b5270 0x7f69a40ca640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:40.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.597+0000 7f69aa000640 1 --2- 172.21.15.49:0/2228530574 >> v2:172.21.15.150:3300/0 conn(0x7f69a40c9c80 0x7f69a40b2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:40.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.597+0000 7f69aa000640 1 --2- 172.21.15.49:0/2228530574 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69a4071120 0x7f69a4071500 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:40.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.597+0000 7f69aa000640 1 -- 172.21.15.49:0/2228530574 >> 172.21.15.49:0/2228530574 conn(0x7f69a40cc140 msgr2=0x7f69a40cc550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:40.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.597+0000 7f69aa000640 1 -- 172.21.15.49:0/2228530574 shutdown_connections 2023-11-06T02:11:40.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.597+0000 7f69aa000640 1 -- 172.21.15.49:0/2228530574 wait complete. 2023-11-06T02:11:40.600 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.597+0000 7f69aa000640 1 Processor -- start 2023-11-06T02:11:40.600 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.597+0000 7f69aa000640 1 -- start start 2023-11-06T02:11:40.600 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.597+0000 7f69aa000640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69a4071120 0x7f69a40acc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:40.600 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.597+0000 7f69aa000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69a40b5270 0x7f69a40cd200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:40.600 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.597+0000 7f69aa000640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f69a40c9c80 0x7f69a40cd740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:40.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.597+0000 7f69aa000640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69a414f400 con 0x7f69a4071120 2023-11-06T02:11:40.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.597+0000 7f69aa000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69a414f8e0 con 0x7f69a40b5270 2023-11-06T02:11:40.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.597+0000 7f69aa000640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69a40ce9d0 con 0x7f69a40c9c80 2023-11-06T02:11:40.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.597+0000 7f69a37fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69a4071120 0x7f69a40acc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:40.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.597+0000 7f69a37fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69a4071120 0x7f69a40acc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49506/0 (socket says 172.21.15.49:49506) 2023-11-06T02:11:40.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.597+0000 7f69a37fe640 1 -- 172.21.15.49:0/3887542034 learned_addr learned my addr 172.21.15.49:0/3887542034 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:40.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.597+0000 7f69a37fe640 1 -- 172.21.15.49:0/3887542034 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f69a40c9c80 msgr2=0x7f69a40cd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:40.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.597+0000 7f69a37fe640 1 --2- 172.21.15.49:0/3887542034 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f69a40c9c80 0x7f69a40cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:40.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.597+0000 7f69a37fe640 1 -- 172.21.15.49:0/3887542034 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69a40b5270 msgr2=0x7f69a40cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:40.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.597+0000 7f69a37fe640 1 --2- 172.21.15.49:0/3887542034 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69a40b5270 0x7f69a40cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:40.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.597+0000 7f69a37fe640 1 -- 172.21.15.49:0/3887542034 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69940094d0 con 0x7f69a4071120 2023-11-06T02:11:40.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.601+0000 7f69a37fe640 1 --2- 172.21.15.49:0/3887542034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69a4071120 0x7f69a40acc00 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f699401d3a0 tx=0x7f699401ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:40.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.601+0000 7f698bfff640 1 -- 172.21.15.49:0/3887542034 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69940ba600 con 0x7f69a4071120 2023-11-06T02:11:40.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.601+0000 7f698bfff640 1 -- 172.21.15.49:0/3887542034 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69940ba780 con 0x7f69a4071120 2023-11-06T02:11:40.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.601+0000 7f69aa000640 1 -- 172.21.15.49:0/3887542034 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69a40cddc0 con 0x7f69a4071120 2023-11-06T02:11:40.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.601+0000 7f698bfff640 1 -- 172.21.15.49:0/3887542034 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69940bab30 con 0x7f69a4071120 2023-11-06T02:11:40.604 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.601+0000 7f69aa000640 1 -- 172.21.15.49:0/3887542034 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69a40ce380 con 0x7f69a4071120 2023-11-06T02:11:40.604 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.601+0000 7f69aa000640 1 -- 172.21.15.49:0/3887542034 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69a40c29a0 con 0x7f69a4071120 2023-11-06T02:11:40.604 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.601+0000 7f698bfff640 1 -- 172.21.15.49:0/3887542034 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f69940bac90 con 0x7f69a4071120 2023-11-06T02:11:40.604 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.601+0000 7f698bfff640 1 --2- 172.21.15.49:0/3887542034 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6990065a60 0x7f6990067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:40.604 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.601+0000 7f698bfff640 1 -- 172.21.15.49:0/3887542034 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6994121900 con 0x7f69a4071120 2023-11-06T02:11:40.604 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.601+0000 7f69a2ffd640 1 --2- 172.21.15.49:0/3887542034 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6990065a60 0x7f6990067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:40.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.601+0000 7f69a2ffd640 1 --2- 172.21.15.49:0/3887542034 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6990065a60 0x7f6990067f20 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f69a4156540 tx=0x7f6998008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:40.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.605+0000 7f698bfff640 1 -- 172.21.15.49:0/3887542034 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f69940ef160 con 0x7f69a4071120 2023-11-06T02:11:40.791 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.793+0000 7f69aa000640 1 -- 172.21.15.49:0/3887542034 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f69a4067140 con 0x7f69a4071120 2023-11-06T02:11:40.791 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.793+0000 7f698bfff640 1 -- 172.21.15.49:0/3887542034 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v30) v1 ==== 53+0+73 (secure 0 0 0) 0x7f69940b5020 con 0x7f69a4071120 2023-11-06T02:11:40.792 INFO:teuthology.orchestra.run.smithi049.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-06T02:11:40.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.797+0000 7f69aa000640 1 -- 172.21.15.49:0/3887542034 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6990065a60 msgr2=0x7f6990067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:40.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.797+0000 7f69aa000640 1 --2- 172.21.15.49:0/3887542034 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6990065a60 0x7f6990067f20 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f69a4156540 tx=0x7f6998008040 comp rx=0 tx=0).stop 2023-11-06T02:11:40.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.797+0000 7f69aa000640 1 -- 172.21.15.49:0/3887542034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69a4071120 msgr2=0x7f69a40acc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:40.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.797+0000 7f69aa000640 1 --2- 172.21.15.49:0/3887542034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69a4071120 0x7f69a40acc00 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f699401d3a0 tx=0x7f699401ba90 comp rx=0 tx=0).stop 2023-11-06T02:11:40.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.797+0000 7f69aa000640 1 -- 172.21.15.49:0/3887542034 shutdown_connections 2023-11-06T02:11:40.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.797+0000 7f69aa000640 1 --2- 172.21.15.49:0/3887542034 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6990065a60 0x7f6990067f20 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:40.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.797+0000 7f69aa000640 1 --2- 172.21.15.49:0/3887542034 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f69a40c9c80 0x7f69a40cd740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:40.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.797+0000 7f69aa000640 1 --2- 172.21.15.49:0/3887542034 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69a40b5270 0x7f69a40cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:40.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.797+0000 7f69aa000640 1 --2- 172.21.15.49:0/3887542034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69a4071120 0x7f69a40acc00 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:40.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.797+0000 7f69aa000640 1 -- 172.21.15.49:0/3887542034 >> 172.21.15.49:0/3887542034 conn(0x7f69a40cc140 msgr2=0x7f69a406a600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:40.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.797+0000 7f69aa000640 1 -- 172.21.15.49:0/3887542034 shutdown_connections 2023-11-06T02:11:40.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:40.797+0000 7f69aa000640 1 -- 172.21.15.49:0/3887542034 wait complete. 2023-11-06T02:11:40.809 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-06T02:11:40.809 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-06T02:11:40.810 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi049.front.sepia.ceph.com 2023-11-06T02:11:40.810 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-06T02:11:40.810 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-06T02:11:40.811 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-06T02:11:40.811 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-06T02:11:40.811 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-06T02:11:40.812 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-06T02:11:40.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:11:40.813 DEBUG:teuthology.orchestra.run.smithi049:> ip addr 2023-11-06T02:11:40.819 INFO:teuthology.orchestra.run.smithi049.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-06T02:11:40.820 INFO:teuthology.orchestra.run.smithi049.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-06T02:11:40.820 INFO:teuthology.orchestra.run.smithi049.stdout: inet 127.0.0.1/8 scope host lo 2023-11-06T02:11:40.820 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:11:40.820 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 ::1/128 scope host 2023-11-06T02:11:40.820 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:11:40.821 INFO:teuthology.orchestra.run.smithi049.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-06T02:11:40.821 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:6b:ff:84 brd ff:ff:ff:ff:ff:ff 2023-11-06T02:11:40.821 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp7s0f0 2023-11-06T02:11:40.821 INFO:teuthology.orchestra.run.smithi049.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-06T02:11:40.821 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:6b:ff:85 brd ff:ff:ff:ff:ff:ff 2023-11-06T02:11:40.822 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp7s0f1 2023-11-06T02:11:40.822 INFO:teuthology.orchestra.run.smithi049.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-06T02:11:40.822 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:bd:15:66 brd ff:ff:ff:ff:ff:ff 2023-11-06T02:11:40.822 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp1s0f0 2023-11-06T02:11:40.822 INFO:teuthology.orchestra.run.smithi049.stdout: inet 172.21.15.49/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-11-06T02:11:40.823 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft 42296sec preferred_lft 42296sec 2023-11-06T02:11:40.823 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 fe80::ec4:7aff:febd:1566/64 scope link 2023-11-06T02:11:40.823 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:11:40.823 INFO:teuthology.orchestra.run.smithi049.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-06T02:11:40.823 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:bd:15:67 brd ff:ff:ff:ff:ff:ff 2023-11-06T02:11:40.824 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp1s0f1 2023-11-06T02:11:40.824 INFO:teuthology.orchestra.run.smithi049.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-06T02:11:40.824 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 02:42:47:49:33:69 brd ff:ff:ff:ff:ff:ff 2023-11-06T02:11:40.824 INFO:teuthology.orchestra.run.smithi049.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-06T02:11:40.824 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:11:40.824 INFO:teuthology.orchestra.run.smithi049.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-06T02:11:40.825 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-11-06T02:11:40.825 INFO:teuthology.orchestra.run.smithi049.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-06T02:11:40.825 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:11:40.825 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-11-06T02:11:40.825 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:11:40.826 INFO:teuthology.orchestra.run.smithi049.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-06T02:11:40.826 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-06T02:11:40.826 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-11-06T02:11:40.826 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:11:40.826 INFO:teuthology.orchestra.run.smithi049.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-06T02:11:40.827 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-06T02:11:40.827 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-11-06T02:11:40.827 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:11:40.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:11:40.828 DEBUG:teuthology.orchestra.run.smithi049:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-06T02:11:40.871 INFO:teuthology.orchestra.run.smithi049.stdout:1 2023-11-06T02:11:40.872 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:11:40.873 DEBUG:teuthology.orchestra.run.smithi049:> route 2023-11-06T02:11:40.923 INFO:teuthology.orchestra.run.smithi049.stdout:Kernel IP routing table 2023-11-06T02:11:40.923 INFO:teuthology.orchestra.run.smithi049.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-06T02:11:40.923 INFO:teuthology.orchestra.run.smithi049.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-11-06T02:11:40.924 INFO:teuthology.orchestra.run.smithi049.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-06T02:11:40.924 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-11-06T02:11:40.924 INFO:teuthology.orchestra.run.smithi049.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-06T02:11:40.924 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:11:40.925 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec bash -c ' 2023-11-06T02:11:40.925 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2023-11-06T02:11:40.925 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-06T02:11:40.926 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-06T02:11:40.926 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-06T02:11:40.927 DEBUG:teuthology.orchestra.run.smithi049:> ') 2023-11-06T02:11:41.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:11:41.003 DEBUG:teuthology.orchestra.run.smithi049:> ip netns list 2023-11-06T02:11:41.053 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-06T02:11:41.053 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-06T02:11:41.054 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-06T02:11:41.054 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:11:41.055 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link set brx.0 up 2023-11-06T02:11:41.113 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-06T02:11:41.114 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-06T02:11:41.114 DEBUG:teuthology.orchestra.run.smithi049:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-06T02:11:41.162 INFO:teuthology.orchestra.run.smithi049.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-06T02:11:41.162 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-06T02:11:41.163 DEBUG:teuthology.orchestra.run.smithi049:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-06T02:11:41.210 DEBUG:teuthology.orchestra.run.smithi049:> sudo modprobe fuse 2023-11-06T02:11:41.268 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-06T02:11:41.268 DEBUG:teuthology.orchestra.run.smithi049:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-06T02:11:41.325 INFO:teuthology.orchestra.run.smithi049.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-06T02:11:41.326 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-06T02:11:41.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:11:41.328 DEBUG:teuthology.orchestra.run.smithi049:> ls /sys/fs/fuse/connections 2023-11-06T02:11:41.374 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-06T02:11:41.374 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-06T02:11:41.420 DEBUG:teuthology.orchestra.run.smithi049:> sudo modprobe fuse 2023-11-06T02:11:41.436 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-06T02:11:41.436 DEBUG:teuthology.orchestra.run.smithi049:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-06T02:11:41.487 INFO:teuthology.orchestra.run.smithi049.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-06T02:11:41.488 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-06T02:11:41.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:11:41.489 DEBUG:teuthology.orchestra.run.smithi049:> ls /sys/fs/fuse/connections 2023-11-06T02:11:41.505 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi049.stderr:2023-11-06T02:11:41.509+0000 7fd0302fa580 -1 init, newargv = 0x56368bdcbb90 newargc=15 2023-11-06T02:11:41.505 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi049.stderr:ceph-fuse[52005]: starting ceph client 2023-11-06T02:11:41.512 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi049.stderr:ceph-fuse[52005]: starting fuse 2023-11-06T02:11:42.612 DEBUG:teuthology.orchestra.run.smithi049:> sudo modprobe fuse 2023-11-06T02:11:42.630 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-06T02:11:42.630 DEBUG:teuthology.orchestra.run.smithi049:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-06T02:11:42.685 INFO:teuthology.orchestra.run.smithi049.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-06T02:11:42.686 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-06T02:11:42.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:11:42.687 DEBUG:teuthology.orchestra.run.smithi049:> ls /sys/fs/fuse/connections 2023-11-06T02:11:42.735 INFO:teuthology.orchestra.run.smithi049.stdout:45 2023-11-06T02:11:42.736 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2023-11-06T02:11:42.736 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-11-06T02:11:42.737 DEBUG:teuthology.orchestra.run.smithi049:> import glob 2023-11-06T02:11:42.737 DEBUG:teuthology.orchestra.run.smithi049:> import re 2023-11-06T02:11:42.737 DEBUG:teuthology.orchestra.run.smithi049:> import os 2023-11-06T02:11:42.738 DEBUG:teuthology.orchestra.run.smithi049:> import subprocess 2023-11-06T02:11:42.738 DEBUG:teuthology.orchestra.run.smithi049:> 2023-11-06T02:11:42.739 DEBUG:teuthology.orchestra.run.smithi049:> def _find_admin_socket(client_name): 2023-11-06T02:11:42.739 DEBUG:teuthology.orchestra.run.smithi049:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-06T02:11:42.739 DEBUG:teuthology.orchestra.run.smithi049:> files = glob.glob(asok_path) 2023-11-06T02:11:42.740 DEBUG:teuthology.orchestra.run.smithi049:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-06T02:11:42.740 DEBUG:teuthology.orchestra.run.smithi049:> 2023-11-06T02:11:42.741 DEBUG:teuthology.orchestra.run.smithi049:> # Given a non-glob path, it better be there 2023-11-06T02:11:42.741 DEBUG:teuthology.orchestra.run.smithi049:> if "*" not in asok_path: 2023-11-06T02:11:42.742 DEBUG:teuthology.orchestra.run.smithi049:> assert(len(files) == 1) 2023-11-06T02:11:42.742 DEBUG:teuthology.orchestra.run.smithi049:> return files[0] 2023-11-06T02:11:42.742 DEBUG:teuthology.orchestra.run.smithi049:> 2023-11-06T02:11:42.743 DEBUG:teuthology.orchestra.run.smithi049:> for f in files: 2023-11-06T02:11:42.743 DEBUG:teuthology.orchestra.run.smithi049:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-06T02:11:42.743 DEBUG:teuthology.orchestra.run.smithi049:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-06T02:11:42.744 DEBUG:teuthology.orchestra.run.smithi049:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-06T02:11:42.744 DEBUG:teuthology.orchestra.run.smithi049:> contents = proc_f.read() 2023-11-06T02:11:42.745 DEBUG:teuthology.orchestra.run.smithi049:> if mountpoint in contents: 2023-11-06T02:11:42.745 DEBUG:teuthology.orchestra.run.smithi049:> return f 2023-11-06T02:11:42.745 DEBUG:teuthology.orchestra.run.smithi049:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-06T02:11:42.746 DEBUG:teuthology.orchestra.run.smithi049:> 2023-11-06T02:11:42.746 DEBUG:teuthology.orchestra.run.smithi049:> print(_find_admin_socket("client.0")) 2023-11-06T02:11:42.747 DEBUG:teuthology.orchestra.run.smithi049:> ' 2023-11-06T02:11:42.857 INFO:teuthology.orchestra.run.smithi049.stdout:/var/run/ceph/ceph-client.0.52005.asok 2023-11-06T02:11:43.040 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.52005.asok 2023-11-06T02:11:43.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:11:43.041 DEBUG:teuthology.orchestra.run.smithi049:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.52005.asok status 2023-11-06T02:11:43.197 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-11-06T02:11:43.198 INFO:teuthology.orchestra.run.smithi049.stdout: "metadata": { 2023-11-06T02:11:43.198 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_sha1": "f7e9f9af51acf98e0ccc3037487018e4ae40caa5", 2023-11-06T02:11:43.198 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_version": "ceph version 18.2.0-1181-gf7e9f9af (f7e9f9af51acf98e0ccc3037487018e4ae40caa5) reef (stable)", 2023-11-06T02:11:43.198 INFO:teuthology.orchestra.run.smithi049.stdout: "entity_id": "0", 2023-11-06T02:11:43.199 INFO:teuthology.orchestra.run.smithi049.stdout: "hostname": "smithi049", 2023-11-06T02:11:43.199 INFO:teuthology.orchestra.run.smithi049.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-06T02:11:43.199 INFO:teuthology.orchestra.run.smithi049.stdout: "pid": "52005", 2023-11-06T02:11:43.199 INFO:teuthology.orchestra.run.smithi049.stdout: "root": "/" 2023-11-06T02:11:43.199 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:11:43.200 INFO:teuthology.orchestra.run.smithi049.stdout: "dentry_count": 0, 2023-11-06T02:11:43.200 INFO:teuthology.orchestra.run.smithi049.stdout: "dentry_pinned_count": 0, 2023-11-06T02:11:43.200 INFO:teuthology.orchestra.run.smithi049.stdout: "id": 6522, 2023-11-06T02:11:43.200 INFO:teuthology.orchestra.run.smithi049.stdout: "inst": { 2023-11-06T02:11:43.200 INFO:teuthology.orchestra.run.smithi049.stdout: "name": { 2023-11-06T02:11:43.201 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "client", 2023-11-06T02:11:43.201 INFO:teuthology.orchestra.run.smithi049.stdout: "num": 6522 2023-11-06T02:11:43.201 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:11:43.201 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": { 2023-11-06T02:11:43.201 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2023-11-06T02:11:43.202 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "192.168.0.1:0", 2023-11-06T02:11:43.202 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 1698210584 2023-11-06T02:11:43.202 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:11:43.202 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:11:43.202 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": { 2023-11-06T02:11:43.203 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2023-11-06T02:11:43.203 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "192.168.0.1:0", 2023-11-06T02:11:43.203 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 1698210584 2023-11-06T02:11:43.203 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:11:43.203 INFO:teuthology.orchestra.run.smithi049.stdout: "inst_str": "client.6522 192.168.0.1:0/1698210584", 2023-11-06T02:11:43.204 INFO:teuthology.orchestra.run.smithi049.stdout: "addr_str": "192.168.0.1:0/1698210584", 2023-11-06T02:11:43.204 INFO:teuthology.orchestra.run.smithi049.stdout: "inode_count": 1, 2023-11-06T02:11:43.205 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_epoch": 30, 2023-11-06T02:11:43.205 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_epoch": 65, 2023-11-06T02:11:43.205 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_epoch_barrier": 0, 2023-11-06T02:11:43.205 INFO:teuthology.orchestra.run.smithi049.stdout: "blocklisted": false, 2023-11-06T02:11:43.205 INFO:teuthology.orchestra.run.smithi049.stdout: "fs_name": "cephfs" 2023-11-06T02:11:43.206 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-11-06T02:11:43.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:11:43.212 DEBUG:teuthology.orchestra.run.smithi049:> stat --file-system '--printf=%T 2023-11-06T02:11:43.212 DEBUG:teuthology.orchestra.run.smithi049:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-06T02:11:43.219 INFO:teuthology.orchestra.run.smithi049.stdout:fuseblk 2023-11-06T02:11:43.219 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-06T02:11:43.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:11:43.220 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-06T02:11:43.275 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-11-06T02:11:43.275 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T02:11:43.322 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:11:43.323 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file1_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2023-11-06T02:11:43.471 INFO:teuthology.orchestra.run.smithi049.stderr:6+0 records in 2023-11-06T02:11:43.471 INFO:teuthology.orchestra.run.smithi049.stderr:6+0 records out 2023-11-06T02:11:43.472 INFO:teuthology.orchestra.run.smithi049.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.0868817 s, 72.4 MB/s 2023-11-06T02:11:43.473 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-11-06T02:11:43.473 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T02:11:43.480 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T02:11:43.653 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.653+0000 7f622d0e9640 1 Processor -- start 2023-11-06T02:11:43.653 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.657+0000 7f622d0e9640 1 -- start start 2023-11-06T02:11:43.654 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.657+0000 7f622d0e9640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f622815ab50 0x7f62280c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:43.654 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.657+0000 7f622d0e9640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f62280cb010 0x7f62280b0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:43.655 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.657+0000 7f622d0e9640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62280cab80 con 0x7f62280b05c0 2023-11-06T02:11:43.655 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.657+0000 7f622d0e9640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62280b0a50 con 0x7f62280cb010 2023-11-06T02:11:43.655 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.657+0000 7f622d0e9640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f622815af30 con 0x7f622815ab50 2023-11-06T02:11:43.655 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.657+0000 7f6226d9d640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f622815ab50 0x7f62280c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:43.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.657+0000 7f622659c640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f62280cb010 0x7f62280b0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:43.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.657+0000 7f6226d9d640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f622815ab50 0x7f62280c9bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.49:36204/0 (socket says 172.21.15.49:36204) 2023-11-06T02:11:43.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.657+0000 7f6226d9d640 1 -- 172.21.15.49:0/2973357536 learned_addr learned my addr 172.21.15.49:0/2973357536 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:43.657 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.657+0000 7f6226d9d640 1 -- 172.21.15.49:0/2973357536 >> v2:172.21.15.150:3300/0 conn(0x7f62280cb010 msgr2=0x7f62280b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:43.657 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.657+0000 7f6226d9d640 1 --2- 172.21.15.49:0/2973357536 >> v2:172.21.15.150:3300/0 conn(0x7f62280cb010 0x7f62280b0080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:43.657 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.657+0000 7f6226d9d640 1 -- 172.21.15.49:0/2973357536 >> v1:172.21.15.150:6789/0 conn(0x7f62280b05c0 legacy=0x7f62280700a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:43.657 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.657+0000 7f6226d9d640 1 -- 172.21.15.49:0/2973357536 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62280cb5b0 con 0x7f622815ab50 2023-11-06T02:11:43.658 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.657+0000 7f622659c640 1 --2- 172.21.15.49:0/2973357536 >> v2:172.21.15.150:3300/0 conn(0x7f62280cb010 0x7f62280b0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T02:11:43.659 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.661+0000 7f6226d9d640 1 --2- 172.21.15.49:0/2973357536 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f622815ab50 0x7f62280c9bb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6218006fd0 tx=0x7f621801c4f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30ffff5929ab1049 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:43.659 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.661+0000 7f6225d9b640 1 -- 172.21.15.49:0/2973357536 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62180aa670 con 0x7f622815ab50 2023-11-06T02:11:43.659 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.661+0000 7f6225d9b640 1 -- 172.21.15.49:0/2973357536 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62180aa7d0 con 0x7f622815ab50 2023-11-06T02:11:43.659 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.661+0000 7f6225d9b640 1 -- 172.21.15.49:0/2973357536 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62180aab60 con 0x7f622815ab50 2023-11-06T02:11:43.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.661+0000 7f622d0e9640 1 -- 172.21.15.49:0/2973357536 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f622815ab50 msgr2=0x7f62280c9bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:43.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.661+0000 7f622d0e9640 1 --2- 172.21.15.49:0/2973357536 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f622815ab50 0x7f62280c9bb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6218006fd0 tx=0x7f621801c4f0 comp rx=0 tx=0).stop 2023-11-06T02:11:43.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.661+0000 7f622d0e9640 1 -- 172.21.15.49:0/2973357536 shutdown_connections 2023-11-06T02:11:43.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.661+0000 7f622d0e9640 1 --2- 172.21.15.49:0/2973357536 >> v2:172.21.15.150:3300/0 conn(0x7f62280cb010 0x7f62280b0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:43.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.661+0000 7f622d0e9640 1 --2- 172.21.15.49:0/2973357536 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f622815ab50 0x7f62280c9bb0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:43.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.661+0000 7f622d0e9640 1 -- 172.21.15.49:0/2973357536 >> 172.21.15.49:0/2973357536 conn(0x7f622806c0a0 msgr2=0x7f622806c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:43.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.661+0000 7f622d0e9640 1 -- 172.21.15.49:0/2973357536 shutdown_connections 2023-11-06T02:11:43.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.661+0000 7f622d0e9640 1 -- 172.21.15.49:0/2973357536 wait complete. 2023-11-06T02:11:43.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.661+0000 7f622d0e9640 1 Processor -- start 2023-11-06T02:11:43.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.661+0000 7f622d0e9640 1 -- start start 2023-11-06T02:11:43.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.661+0000 7f622d0e9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62280b05c0 0x7f62280c0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:43.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.661+0000 7f622d0e9640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f62280cb010 0x7f62281529c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:43.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.661+0000 7f622d0e9640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f622815ab50 0x7f62280ae060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:43.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.661+0000 7f622d0e9640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6228153380 con 0x7f62280b05c0 2023-11-06T02:11:43.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.661+0000 7f622d0e9640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6228153660 con 0x7f62280cb010 2023-11-06T02:11:43.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.661+0000 7f622d0e9640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6228153940 con 0x7f622815ab50 2023-11-06T02:11:43.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.661+0000 7f6226d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62280b05c0 0x7f62280c0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:43.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.661+0000 7f6226d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62280b05c0 0x7f62280c0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49510/0 (socket says 172.21.15.49:49510) 2023-11-06T02:11:43.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.661+0000 7f6226d9d640 1 -- 172.21.15.49:0/3176341913 learned_addr learned my addr 172.21.15.49:0/3176341913 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:43.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.661+0000 7f622759e640 1 --2- 172.21.15.49:0/3176341913 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f622815ab50 0x7f62280ae060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:43.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.661+0000 7f622659c640 1 --2- 172.21.15.49:0/3176341913 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f62280cb010 0x7f62281529c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:43.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.665+0000 7f6226d9d640 1 -- 172.21.15.49:0/3176341913 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f622815ab50 msgr2=0x7f62280ae060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:43.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.665+0000 7f6226d9d640 1 --2- 172.21.15.49:0/3176341913 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f622815ab50 0x7f62280ae060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:43.667 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.665+0000 7f6226d9d640 1 -- 172.21.15.49:0/3176341913 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f62280cb010 msgr2=0x7f62281529c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:43.667 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.665+0000 7f6226d9d640 1 --2- 172.21.15.49:0/3176341913 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f62280cb010 0x7f62281529c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:43.667 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.665+0000 7f6226d9d640 1 -- 172.21.15.49:0/3176341913 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f621c009420 con 0x7f62280b05c0 2023-11-06T02:11:43.667 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.665+0000 7f6226d9d640 1 --2- 172.21.15.49:0/3176341913 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62280b05c0 0x7f62280c0b60 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f621801cd90 tx=0x7f6218019290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:43.667 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.665+0000 7f620f7fe640 1 -- 172.21.15.49:0/3176341913 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62180aad50 con 0x7f62280b05c0 2023-11-06T02:11:43.667 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.665+0000 7f620f7fe640 1 -- 172.21.15.49:0/3176341913 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62180aa630 con 0x7f62280b05c0 2023-11-06T02:11:43.668 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.665+0000 7f622d0e9640 1 -- 172.21.15.49:0/3176341913 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6218009480 con 0x7f62280b05c0 2023-11-06T02:11:43.668 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.665+0000 7f620f7fe640 1 -- 172.21.15.49:0/3176341913 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6218003860 con 0x7f62280b05c0 2023-11-06T02:11:43.668 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.665+0000 7f622d0e9640 1 -- 172.21.15.49:0/3176341913 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62281549c0 con 0x7f62280b05c0 2023-11-06T02:11:43.668 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.665+0000 7f622d0e9640 1 -- 172.21.15.49:0/3176341913 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62280b0de0 con 0x7f62280b05c0 2023-11-06T02:11:43.668 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.665+0000 7f620f7fe640 1 -- 172.21.15.49:0/3176341913 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f62180039c0 con 0x7f62280b05c0 2023-11-06T02:11:43.669 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.665+0000 7f620f7fe640 1 --2- 172.21.15.49:0/3176341913 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6214065a60 0x7f6214067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:43.669 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.665+0000 7f620f7fe640 1 -- 172.21.15.49:0/3176341913 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f621811ddb0 con 0x7f62280b05c0 2023-11-06T02:11:43.669 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.665+0000 7f622659c640 1 --2- 172.21.15.49:0/3176341913 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6214065a60 0x7f6214067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:43.670 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.665+0000 7f622659c640 1 --2- 172.21.15.49:0/3176341913 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6214065a60 0x7f6214067f20 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f62280aeca0 tx=0x7f621c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:43.670 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.669+0000 7f620f7fe640 1 -- 172.21.15.49:0/3176341913 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f6218121030 con 0x7f62280b05c0 2023-11-06T02:11:43.826 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.829+0000 7f622d0e9640 1 -- 172.21.15.49:0/3176341913 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6228074040 con 0x7f62280b05c0 2023-11-06T02:11:43.829 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.829+0000 7f620f7fe640 1 -- 172.21.15.49:0/3176341913 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+17825 (secure 0 0 0) 0x7f62180eb6a0 con 0x7f62280b05c0 2023-11-06T02:11:43.829 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:11:43.829 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":65,"fsid":"1a8b035c-6ba4-499a-b876-aea47fd9c0d4","created":"2023-11-06T02:08:06.440857+0000","modified":"2023-11-06T02:11:31.094207+0000","last_up_change":"2023-11-06T02:08:17.982372+0000","last_in_change":"2023-11-06T02:08:08.906631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T02:08:19.252222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T02:08:24.776211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-06T02:11:28.648326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-06T02:11:29.079513+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ad55af8-0571-4e6a-8722-b5d429b7496d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6809","nonce":38116},{"type":"v1","addr":"172.21.15.49:6811","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6813","nonce":38116},{"type":"v1","addr":"172.21.15.49:6815","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6821","nonce":38116},{"type":"v1","addr":"172.21.15.49:6823","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6817","nonce":38116},{"type":"v1","addr":"172.21.15.49:6819","nonce":38116}]},"public_addr":"172.21.15.49:6811/38116","cluster_addr":"172.21.15.49:6815/38116","heartbeat_back_addr":"172.21.15.49:6823/38116","heartbeat_front_addr":"172.21.15.49:6819/38116","state":["exists","up"]},{"osd":1,"uuid":"c8b31f85-a757-43ed-9419-351acc46611b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":38113},{"type":"v1","addr":"172.21.15.49:6810","nonce":38113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":38113},{"type":"v1","addr":"172.21.15.49:6814","nonce":38113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":38113},{"type":"v1","addr":"172.21.15.49:6822","nonce":38113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":38113},{"type":"v1","addr":"172.21.15.49:6818","nonce":38113}]},"public_addr":"172.21.15.49:6810/38113","cluster_addr":"172.21.15.49:6814/38113","heartbeat_back_addr":"172.21.15.49:6822/38113","heartbeat_front_addr":"172.21.15.49:6818/38113","state":["exists","up"]},{"osd":2,"uuid":"555b3bf3-0a70-4b91-8efd-e4204b5fc26e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":38114},{"type":"v1","addr":"172.21.15.49:6801","nonce":38114}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":38114},{"type":"v1","addr":"172.21.15.49:6803","nonce":38114}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":38114},{"type":"v1","addr":"172.21.15.49:6807","nonce":38114}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":38114},{"type":"v1","addr":"172.21.15.49:6805","nonce":38114}]},"public_addr":"172.21.15.49:6801/38114","cluster_addr":"172.21.15.49:6803/38114","heartbeat_back_addr":"172.21.15.49:6807/38114","heartbeat_front_addr":"172.21.15.49:6805/38114","state":["exists","up"]},{"osd":3,"uuid":"8d4e5d77-03ef-4b9b-bfcb-5ae93a46795c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":38115},{"type":"v1","addr":"172.21.15.49:6825","nonce":38115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":38115},{"type":"v1","addr":"172.21.15.49:6827","nonce":38115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":38115},{"type":"v1","addr":"172.21.15.49:6831","nonce":38115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":38115},{"type":"v1","addr":"172.21.15.49:6829","nonce":38115}]},"public_addr":"172.21.15.49:6825/38115","cluster_addr":"172.21.15.49:6827/38115","heartbeat_back_addr":"172.21.15.49:6831/38115","heartbeat_front_addr":"172.21.15.49:6829/38115","state":["exists","up"]},{"osd":4,"uuid":"dba03ad4-68a3-4cf3-9218-ac611f02eacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":38417},{"type":"v1","addr":"172.21.15.150:6809","nonce":38417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":38417},{"type":"v1","addr":"172.21.15.150:6811","nonce":38417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":38417},{"type":"v1","addr":"172.21.15.150:6815","nonce":38417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":38417},{"type":"v1","addr":"172.21.15.150:6813","nonce":38417}]},"public_addr":"172.21.15.150:6809/38417","cluster_addr":"172.21.15.150:6811/38417","heartbeat_back_addr":"172.21.15.150:6815/38417","heartbeat_front_addr":"172.21.15.150:6813/38417","state":["exists","up"]},{"osd":5,"uuid":"0df9c303-c255-4640-8e91-01f1c3efe18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":38418},{"type":"v1","addr":"172.21.15.150:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":38418},{"type":"v1","addr":"172.21.15.150:6803","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":38418},{"type":"v1","addr":"172.21.15.150:6807","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":38418},{"type":"v1","addr":"172.21.15.150:6805","nonce":38418}]},"public_addr":"172.21.15.150:6801/38418","cluster_addr":"172.21.15.150:6803/38418","heartbeat_back_addr":"172.21.15.150:6807/38418","heartbeat_front_addr":"172.21.15.150:6805/38418","state":["exists","up"]},{"osd":6,"uuid":"e59fa189-4b70-4c06-9722-c02b90828bf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":38428},{"type":"v1","addr":"172.21.15.150:6825","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":38428},{"type":"v1","addr":"172.21.15.150:6827","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":38428},{"type":"v1","addr":"172.21.15.150:6831","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":38428},{"type":"v1","addr":"172.21.15.150:6829","nonce":38428}]},"public_addr":"172.21.15.150:6825/38428","cluster_addr":"172.21.15.150:6827/38428","heartbeat_back_addr":"172.21.15.150:6831/38428","heartbeat_front_addr":"172.21.15.150:6829/38428","state":["exists","up"]},{"osd":7,"uuid":"a64f57e1-ef98-46bd-b304-385759e95898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":38421},{"type":"v1","addr":"172.21.15.150:6817","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":38421},{"type":"v1","addr":"172.21.15.150:6819","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":38421},{"type":"v1","addr":"172.21.15.150:6823","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":38421},{"type":"v1","addr":"172.21.15.150:6821","nonce":38421}]},"public_addr":"172.21.15.150:6817/38421","cluster_addr":"172.21.15.150:6819/38421","heartbeat_back_addr":"172.21.15.150:6823/38421","heartbeat_front_addr":"172.21.15.150:6821/38421","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.438552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.413280+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.296466+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.424966+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.276953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.230984+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.410235+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.480084+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T02:11:43.831 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.833+0000 7f622d0e9640 1 -- 172.21.15.49:0/3176341913 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6214065a60 msgr2=0x7f6214067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:43.831 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.833+0000 7f622d0e9640 1 --2- 172.21.15.49:0/3176341913 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6214065a60 0x7f6214067f20 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f62280aeca0 tx=0x7f621c008040 comp rx=0 tx=0).stop 2023-11-06T02:11:43.832 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.833+0000 7f622d0e9640 1 -- 172.21.15.49:0/3176341913 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62280b05c0 msgr2=0x7f62280c0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:43.832 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.833+0000 7f622d0e9640 1 --2- 172.21.15.49:0/3176341913 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62280b05c0 0x7f62280c0b60 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f621801cd90 tx=0x7f6218019290 comp rx=0 tx=0).stop 2023-11-06T02:11:43.832 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.833+0000 7f622d0e9640 1 -- 172.21.15.49:0/3176341913 shutdown_connections 2023-11-06T02:11:43.832 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.833+0000 7f622d0e9640 1 --2- 172.21.15.49:0/3176341913 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6214065a60 0x7f6214067f20 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:43.832 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.833+0000 7f622d0e9640 1 --2- 172.21.15.49:0/3176341913 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f622815ab50 0x7f62280ae060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:43.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.833+0000 7f622d0e9640 1 --2- 172.21.15.49:0/3176341913 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f62280cb010 0x7f62281529c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:43.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.833+0000 7f622d0e9640 1 --2- 172.21.15.49:0/3176341913 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62280b05c0 0x7f62280c0b60 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:43.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.833+0000 7f622d0e9640 1 -- 172.21.15.49:0/3176341913 >> 172.21.15.49:0/3176341913 conn(0x7f622806c0a0 msgr2=0x7f62280c3bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:43.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.833+0000 7f622d0e9640 1 -- 172.21.15.49:0/3176341913 shutdown_connections 2023-11-06T02:11:43.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.833+0000 7f622d0e9640 1 -- 172.21.15.49:0/3176341913 wait complete. 2023-11-06T02:11:43.848 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T02:08:19.252222+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T02:08:24.776211+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T02:11:28.648326+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T02:11:29.079513+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '65', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T02:11:43.849 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T02:11:43.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.985+0000 7f264b02b640 1 Processor -- start 2023-11-06T02:11:43.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.985+0000 7f264b02b640 1 -- start start 2023-11-06T02:11:43.983 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.985+0000 7f264b02b640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f26440d2a60 0x7f26440d2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:43.983 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.985+0000 7f264b02b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f26440b5d40 0x7f26440b61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:43.983 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.985+0000 7f264b02b640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f264415cf90 con 0x7f264415cbb0 2023-11-06T02:11:43.983 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.985+0000 7f264b02b640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26440d3410 con 0x7f26440d2a60 2023-11-06T02:11:43.984 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.985+0000 7f264b02b640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26440d0b80 con 0x7f26440b5d40 2023-11-06T02:11:43.984 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.985+0000 7f2648dc7640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f264415cbb0 0x7f26440d1fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.49:37020/0 (socket says 172.21.15.49:37020) 2023-11-06T02:11:43.984 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.985+0000 7f2648dc7640 1 -- 172.21.15.49:0/1324187975 learned_addr learned my addr 172.21.15.49:0/1324187975 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:11:43.984 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.985+0000 7f26495c8640 1 --2- 172.21.15.49:0/1324187975 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f26440b5d40 0x7f26440b61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:43.984 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.985+0000 7f2643fff640 1 --2- 172.21.15.49:0/1324187975 >> v2:172.21.15.150:3300/0 conn(0x7f26440d2a60 0x7f26440d2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:43.985 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.985+0000 7f26437fe640 1 -- 172.21.15.49:0/1324187975 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3006621105 0 0) 0x7f264415cf90 con 0x7f264415cbb0 2023-11-06T02:11:43.985 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.985+0000 7f26437fe640 1 -- 172.21.15.49:0/1324187975 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2634003530 con 0x7f264415cbb0 2023-11-06T02:11:43.985 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.985+0000 7f26495c8640 1 -- 172.21.15.49:0/1324187975 >> v2:172.21.15.150:3300/0 conn(0x7f26440d2a60 msgr2=0x7f26440d2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:43.985 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.985+0000 7f26495c8640 1 --2- 172.21.15.49:0/1324187975 >> v2:172.21.15.150:3300/0 conn(0x7f26440d2a60 0x7f26440d2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:43.985 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.985+0000 7f26495c8640 1 -- 172.21.15.49:0/1324187975 >> v1:172.21.15.150:6789/0 conn(0x7f264415cbb0 legacy=0x7f26440d1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:43.986 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.985+0000 7f26495c8640 1 -- 172.21.15.49:0/1324187975 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26440d0e60 con 0x7f26440b5d40 2023-11-06T02:11:43.987 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.989+0000 7f26495c8640 1 --2- 172.21.15.49:0/1324187975 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f26440b5d40 0x7f26440b61d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f263c009d90 tx=0x7f263c01f130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e2b18f1663a838f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:43.987 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.989+0000 7f26437fe640 1 -- 172.21.15.49:0/1324187975 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f263c0af600 con 0x7f26440b5d40 2023-11-06T02:11:43.987 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.989+0000 7f26437fe640 1 -- 172.21.15.49:0/1324187975 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f263c0ab050 con 0x7f26440b5d40 2023-11-06T02:11:43.987 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.989+0000 7f26437fe640 1 -- 172.21.15.49:0/1324187975 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f263c0af980 con 0x7f26440b5d40 2023-11-06T02:11:43.988 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.989+0000 7f264b02b640 1 -- 172.21.15.49:0/1324187975 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f26440b5d40 msgr2=0x7f26440b61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:43.988 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.989+0000 7f264b02b640 1 --2- 172.21.15.49:0/1324187975 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f26440b5d40 0x7f26440b61d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f263c009d90 tx=0x7f263c01f130 comp rx=0 tx=0).stop 2023-11-06T02:11:43.988 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.989+0000 7f264b02b640 1 -- 172.21.15.49:0/1324187975 shutdown_connections 2023-11-06T02:11:43.988 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.989+0000 7f264b02b640 1 --2- 172.21.15.49:0/1324187975 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f26440b5d40 0x7f26440b61d0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:43.988 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.989+0000 7f264b02b640 1 --2- 172.21.15.49:0/1324187975 >> v2:172.21.15.150:3300/0 conn(0x7f26440d2a60 0x7f26440d2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:43.989 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.989+0000 7f264b02b640 1 -- 172.21.15.49:0/1324187975 >> 172.21.15.49:0/1324187975 conn(0x7f26440718a0 msgr2=0x7f2644071cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:43.989 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.989+0000 7f264b02b640 1 -- 172.21.15.49:0/1324187975 shutdown_connections 2023-11-06T02:11:43.989 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.989+0000 7f264b02b640 1 -- 172.21.15.49:0/1324187975 wait complete. 2023-11-06T02:11:43.991 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.989+0000 7f264b02b640 1 Processor -- start 2023-11-06T02:11:43.992 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.989+0000 7f264b02b640 1 -- start start 2023-11-06T02:11:43.992 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.989+0000 7f264b02b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f26440b5d40 0x7f2644159e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:43.992 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.989+0000 7f264b02b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26440d2a60 0x7f26440aa590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:43.992 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.989+0000 7f264b02b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f264415cbb0 0x7f26440aaad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:43.992 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.989+0000 7f264b02b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26440ab950 con 0x7f264415cbb0 2023-11-06T02:11:43.993 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.989+0000 7f264b02b640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26441f1f50 con 0x7f26440d2a60 2023-11-06T02:11:43.993 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.989+0000 7f264b02b640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f264415a8e0 con 0x7f26440b5d40 2023-11-06T02:11:43.993 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.989+0000 7f26495c8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f264415cbb0 0x7f26440aaad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:43.993 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.989+0000 7f26495c8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f264415cbb0 0x7f26440aaad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49514/0 (socket says 172.21.15.49:49514) 2023-11-06T02:11:43.993 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.989+0000 7f26495c8640 1 -- 172.21.15.49:0/2592928354 learned_addr learned my addr 172.21.15.49:0/2592928354 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:43.994 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.989+0000 7f2648dc7640 1 --2- 172.21.15.49:0/2592928354 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f26440b5d40 0x7f2644159e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:43.994 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.989+0000 7f2643fff640 1 --2- 172.21.15.49:0/2592928354 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26440d2a60 0x7f26440aa590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:43.994 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.993+0000 7f26495c8640 1 -- 172.21.15.49:0/2592928354 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f26440b5d40 msgr2=0x7f2644159e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:43.994 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.993+0000 7f26495c8640 1 --2- 172.21.15.49:0/2592928354 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f26440b5d40 0x7f2644159e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:43.994 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.993+0000 7f26495c8640 1 -- 172.21.15.49:0/2592928354 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26440d2a60 msgr2=0x7f26440aa590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:43.995 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.993+0000 7f26495c8640 1 --2- 172.21.15.49:0/2592928354 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26440d2a60 0x7f26440aa590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:43.995 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.993+0000 7f26495c8640 1 -- 172.21.15.49:0/2592928354 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f263c009a70 con 0x7f264415cbb0 2023-11-06T02:11:43.995 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.993+0000 7f26495c8640 1 --2- 172.21.15.49:0/2592928354 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f264415cbb0 0x7f26440aaad0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f263c0b2040 tx=0x7f263c01bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:43.995 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.993+0000 7f26417fa640 1 -- 172.21.15.49:0/2592928354 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f263c009450 con 0x7f264415cbb0 2023-11-06T02:11:43.995 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.993+0000 7f264b02b640 1 -- 172.21.15.49:0/2592928354 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26440ab0c0 con 0x7f264415cbb0 2023-11-06T02:11:43.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.993+0000 7f26417fa640 1 -- 172.21.15.49:0/2592928354 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f263c0b9410 con 0x7f264415cbb0 2023-11-06T02:11:43.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.993+0000 7f26417fa640 1 -- 172.21.15.49:0/2592928354 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f263c0b97c0 con 0x7f264415cbb0 2023-11-06T02:11:43.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.993+0000 7f264b02b640 1 -- 172.21.15.49:0/2592928354 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26440abb60 con 0x7f264415cbb0 2023-11-06T02:11:43.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.993+0000 7f264b02b640 1 -- 172.21.15.49:0/2592928354 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2644074040 con 0x7f264415cbb0 2023-11-06T02:11:43.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.993+0000 7f26417fa640 1 -- 172.21.15.49:0/2592928354 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f263c0b6050 con 0x7f264415cbb0 2023-11-06T02:11:43.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.993+0000 7f26417fa640 1 --2- 172.21.15.49:0/2592928354 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2628065a60 0x7f2628067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:43.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.993+0000 7f26417fa640 1 -- 172.21.15.49:0/2592928354 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f263c121690 con 0x7f264415cbb0 2023-11-06T02:11:43.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.993+0000 7f2648dc7640 1 --2- 172.21.15.49:0/2592928354 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2628065a60 0x7f2628067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:43.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.993+0000 7f2648dc7640 1 --2- 172.21.15.49:0/2592928354 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2628065a60 0x7f2628067f20 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f2638005890 tx=0x7f2638009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:43.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:43.997+0000 7f26417fa640 1 -- 172.21.15.49:0/2592928354 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f263c0eee70 con 0x7f264415cbb0 2023-11-06T02:11:44.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.153+0000 7f264b02b640 1 -- 172.21.15.49:0/2592928354 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f26440cce50 con 0x7f264415cbb0 2023-11-06T02:11:44.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.153+0000 7f26417fa640 1 -- 172.21.15.49:0/2592928354 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v65) v1 ==== 93+0+10 (secure 0 0 0) 0x7f263c0f2490 con 0x7f264415cbb0 2023-11-06T02:11:44.152 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2023-11-06T02:11:44.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.157+0000 7f264b02b640 1 -- 172.21.15.49:0/2592928354 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2628065a60 msgr2=0x7f2628067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:44.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.157+0000 7f264b02b640 1 --2- 172.21.15.49:0/2592928354 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2628065a60 0x7f2628067f20 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f2638005890 tx=0x7f2638009040 comp rx=0 tx=0).stop 2023-11-06T02:11:44.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.157+0000 7f264b02b640 1 -- 172.21.15.49:0/2592928354 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f264415cbb0 msgr2=0x7f26440aaad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:44.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.157+0000 7f264b02b640 1 --2- 172.21.15.49:0/2592928354 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f264415cbb0 0x7f26440aaad0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f263c0b2040 tx=0x7f263c01bed0 comp rx=0 tx=0).stop 2023-11-06T02:11:44.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.157+0000 7f264b02b640 1 -- 172.21.15.49:0/2592928354 shutdown_connections 2023-11-06T02:11:44.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.157+0000 7f264b02b640 1 --2- 172.21.15.49:0/2592928354 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2628065a60 0x7f2628067f20 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:44.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.157+0000 7f264b02b640 1 --2- 172.21.15.49:0/2592928354 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f264415cbb0 0x7f26440aaad0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:44.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.157+0000 7f264b02b640 1 --2- 172.21.15.49:0/2592928354 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26440d2a60 0x7f26440aa590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:44.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.157+0000 7f264b02b640 1 --2- 172.21.15.49:0/2592928354 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f26440b5d40 0x7f2644159e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:44.156 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.157+0000 7f264b02b640 1 -- 172.21.15.49:0/2592928354 >> 172.21.15.49:0/2592928354 conn(0x7f26440718a0 msgr2=0x7f26440cd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:44.156 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.157+0000 7f264b02b640 1 -- 172.21.15.49:0/2592928354 shutdown_connections 2023-11-06T02:11:44.156 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.157+0000 7f264b02b640 1 -- 172.21.15.49:0/2592928354 wait complete. 2023-11-06T02:11:44.171 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T02:11:44.301 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.305+0000 7f2dae4b8640 1 Processor -- start 2023-11-06T02:11:44.301 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.305+0000 7f2dae4b8640 1 -- start start 2023-11-06T02:11:44.303 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.305+0000 7f2dae4b8640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f2da815a880 0x7f2da80b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:44.303 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.305+0000 7f2dae4b8640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f2da8070160 0x7f2da8070610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:44.303 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.305+0000 7f2dae4b8640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2da80ae360 con 0x7f2da806d150 2023-11-06T02:11:44.303 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.305+0000 7f2dae4b8640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2da8070b50 con 0x7f2da815a880 2023-11-06T02:11:44.304 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.305+0000 7f2dae4b8640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2da80ae640 con 0x7f2da8070160 2023-11-06T02:11:44.304 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.305+0000 7f2da7fff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f2da815a880 0x7f2da80b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:44.304 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.305+0000 7f2da7fff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f2da815a880 0x7f2da80b15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49526/0 (socket says 172.21.15.49:49526) 2023-11-06T02:11:44.304 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.305+0000 7f2da7fff640 1 -- 172.21.15.49:0/1389487639 learned_addr learned my addr 172.21.15.49:0/1389487639 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:44.304 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.305+0000 7f2da77fe640 1 --2- 172.21.15.49:0/1389487639 >> v2:172.21.15.150:3300/0 conn(0x7f2da8070160 0x7f2da8070610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:44.305 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.305+0000 7f2da7fff640 1 -- 172.21.15.49:0/1389487639 >> v2:172.21.15.150:3300/0 conn(0x7f2da8070160 msgr2=0x7f2da8070610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:44.305 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.305+0000 7f2da7fff640 1 --2- 172.21.15.49:0/1389487639 >> v2:172.21.15.150:3300/0 conn(0x7f2da8070160 0x7f2da8070610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:44.305 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.305+0000 7f2da7fff640 1 -- 172.21.15.49:0/1389487639 >> v1:172.21.15.150:6789/0 conn(0x7f2da806d150 legacy=0x7f2da806d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:44.305 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.305+0000 7f2da7fff640 1 -- 172.21.15.49:0/1389487639 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2da80ae920 con 0x7f2da815a880 2023-11-06T02:11:44.306 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.309+0000 7f2da7fff640 1 --2- 172.21.15.49:0/1389487639 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2da815a880 0x7f2da80b15c0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f2da815e3a0 tx=0x7f2d9801c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c681066ae797f809 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:44.306 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.309+0000 7f2da6ffd640 1 -- 172.21.15.49:0/1389487639 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d980ab610 con 0x7f2da815a880 2023-11-06T02:11:44.307 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.309+0000 7f2da6ffd640 1 -- 172.21.15.49:0/1389487639 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d980ab770 con 0x7f2da815a880 2023-11-06T02:11:44.307 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.309+0000 7f2da6ffd640 1 -- 172.21.15.49:0/1389487639 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d980abb00 con 0x7f2da815a880 2023-11-06T02:11:44.308 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.309+0000 7f2dae4b8640 1 -- 172.21.15.49:0/1389487639 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2da815a880 msgr2=0x7f2da80b15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:44.308 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.309+0000 7f2dae4b8640 1 --2- 172.21.15.49:0/1389487639 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2da815a880 0x7f2da80b15c0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f2da815e3a0 tx=0x7f2d9801c500 comp rx=0 tx=0).stop 2023-11-06T02:11:44.308 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.309+0000 7f2dae4b8640 1 -- 172.21.15.49:0/1389487639 shutdown_connections 2023-11-06T02:11:44.309 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.309+0000 7f2dae4b8640 1 --2- 172.21.15.49:0/1389487639 >> v2:172.21.15.150:3300/0 conn(0x7f2da8070160 0x7f2da8070610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:44.309 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.309+0000 7f2dae4b8640 1 --2- 172.21.15.49:0/1389487639 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2da815a880 0x7f2da80b15c0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:44.309 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.309+0000 7f2dae4b8640 1 -- 172.21.15.49:0/1389487639 >> 172.21.15.49:0/1389487639 conn(0x7f2da806a370 msgr2=0x7f2da80ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:44.309 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.309+0000 7f2dae4b8640 1 -- 172.21.15.49:0/1389487639 shutdown_connections 2023-11-06T02:11:44.309 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.309+0000 7f2dae4b8640 1 -- 172.21.15.49:0/1389487639 wait complete. 2023-11-06T02:11:44.309 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.309+0000 7f2dae4b8640 1 Processor -- start 2023-11-06T02:11:44.310 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.309+0000 7f2dae4b8640 1 -- start start 2023-11-06T02:11:44.312 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.313+0000 7f2dae4b8640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2da806d150 0x7f2da8158800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:44.312 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.313+0000 7f2dae4b8640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2da8070160 0x7f2da8158d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:44.312 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.313+0000 7f2dae4b8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2da815a880 0x7f2da815bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:44.313 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.313+0000 7f2dae4b8640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da80c9c30 con 0x7f2da815a880 2023-11-06T02:11:44.313 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.313+0000 7f2dae4b8640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da80c9780 con 0x7f2da806d150 2023-11-06T02:11:44.313 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.313+0000 7f2dae4b8640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2da80c9e80 con 0x7f2da8070160 2023-11-06T02:11:44.313 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.313+0000 7f2daca55640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2da815a880 0x7f2da815bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:44.313 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.313+0000 7f2daca55640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2da815a880 0x7f2da815bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49530/0 (socket says 172.21.15.49:49530) 2023-11-06T02:11:44.314 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.313+0000 7f2daca55640 1 -- 172.21.15.49:0/3428581766 learned_addr learned my addr 172.21.15.49:0/3428581766 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:44.314 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.313+0000 7f2da7fff640 1 --2- 172.21.15.49:0/3428581766 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2da806d150 0x7f2da8158800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:44.314 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.313+0000 7f2da77fe640 1 --2- 172.21.15.49:0/3428581766 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2da8070160 0x7f2da8158d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:44.314 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.313+0000 7f2daca55640 1 -- 172.21.15.49:0/3428581766 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2da8070160 msgr2=0x7f2da8158d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:44.315 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.313+0000 7f2daca55640 1 --2- 172.21.15.49:0/3428581766 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2da8070160 0x7f2da8158d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:44.315 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.313+0000 7f2daca55640 1 -- 172.21.15.49:0/3428581766 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2da806d150 msgr2=0x7f2da8158800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:44.315 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.313+0000 7f2daca55640 1 --2- 172.21.15.49:0/3428581766 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2da806d150 0x7f2da8158800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:44.315 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.313+0000 7f2daca55640 1 -- 172.21.15.49:0/3428581766 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d980094d0 con 0x7f2da815a880 2023-11-06T02:11:44.315 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.313+0000 7f2daca55640 1 --2- 172.21.15.49:0/3428581766 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2da815a880 0x7f2da815bec0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f2da000c770 tx=0x7f2da000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:44.316 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.313+0000 7f2da4ff9640 1 -- 172.21.15.49:0/3428581766 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da0003b80 con 0x7f2da815a880 2023-11-06T02:11:44.316 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.313+0000 7f2da4ff9640 1 -- 172.21.15.49:0/3428581766 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2da0003ce0 con 0x7f2da815a880 2023-11-06T02:11:44.316 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.313+0000 7f2dae4b8640 1 -- 172.21.15.49:0/3428581766 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2da80ca160 con 0x7f2da815a880 2023-11-06T02:11:44.316 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.313+0000 7f2da4ff9640 1 -- 172.21.15.49:0/3428581766 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da000aa90 con 0x7f2da815a880 2023-11-06T02:11:44.316 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.313+0000 7f2dae4b8640 1 -- 172.21.15.49:0/3428581766 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2da814d4e0 con 0x7f2da815a880 2023-11-06T02:11:44.316 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.313+0000 7f2dae4b8640 1 -- 172.21.15.49:0/3428581766 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2da80c9a60 con 0x7f2da815a880 2023-11-06T02:11:44.317 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.313+0000 7f2da4ff9640 1 -- 172.21.15.49:0/3428581766 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f2da000abf0 con 0x7f2da815a880 2023-11-06T02:11:44.317 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.313+0000 7f2da4ff9640 1 --2- 172.21.15.49:0/3428581766 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2d94065a90 0x7f2d94067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:44.317 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.313+0000 7f2da4ff9640 1 -- 172.21.15.49:0/3428581766 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2da0082fd0 con 0x7f2da815a880 2023-11-06T02:11:44.317 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.313+0000 7f2da7fff640 1 --2- 172.21.15.49:0/3428581766 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2d94065a90 0x7f2d94067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:44.318 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.317+0000 7f2da7fff640 1 --2- 172.21.15.49:0/3428581766 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2d94065a90 0x7f2d94067f50 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f2d980af000 tx=0x7f2d980b6040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:44.318 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.321+0000 7f2da4ff9640 1 -- 172.21.15.49:0/3428581766 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f2da00508e0 con 0x7f2da815a880 2023-11-06T02:11:44.472 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.473+0000 7f2dae4b8640 1 -- 172.21.15.49:0/3428581766 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f2da80adaf0 con 0x7f2da815a880 2023-11-06T02:11:44.473 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.473+0000 7f2da4ff9640 1 -- 172.21.15.49:0/3428581766 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v65) v1 ==== 92+0+10 (secure 0 0 0) 0x7f2da0053f00 con 0x7f2da815a880 2023-11-06T02:11:44.473 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 8 2023-11-06T02:11:44.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.477+0000 7f2dae4b8640 1 -- 172.21.15.49:0/3428581766 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2d94065a90 msgr2=0x7f2d94067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:44.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.477+0000 7f2dae4b8640 1 --2- 172.21.15.49:0/3428581766 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2d94065a90 0x7f2d94067f50 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f2d980af000 tx=0x7f2d980b6040 comp rx=0 tx=0).stop 2023-11-06T02:11:44.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.477+0000 7f2dae4b8640 1 -- 172.21.15.49:0/3428581766 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2da815a880 msgr2=0x7f2da815bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:44.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.477+0000 7f2dae4b8640 1 --2- 172.21.15.49:0/3428581766 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2da815a880 0x7f2da815bec0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f2da000c770 tx=0x7f2da000cc30 comp rx=0 tx=0).stop 2023-11-06T02:11:44.477 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.477+0000 7f2dae4b8640 1 -- 172.21.15.49:0/3428581766 shutdown_connections 2023-11-06T02:11:44.477 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.477+0000 7f2dae4b8640 1 --2- 172.21.15.49:0/3428581766 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2d94065a90 0x7f2d94067f50 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:44.478 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.477+0000 7f2dae4b8640 1 --2- 172.21.15.49:0/3428581766 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2da815a880 0x7f2da815bec0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:44.478 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.477+0000 7f2dae4b8640 1 --2- 172.21.15.49:0/3428581766 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2da8070160 0x7f2da8158d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:44.478 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.477+0000 7f2dae4b8640 1 --2- 172.21.15.49:0/3428581766 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2da806d150 0x7f2da8158800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:44.479 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.477+0000 7f2dae4b8640 1 -- 172.21.15.49:0/3428581766 >> 172.21.15.49:0/3428581766 conn(0x7f2da806a370 msgr2=0x7f2da814fd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:44.479 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.477+0000 7f2dae4b8640 1 -- 172.21.15.49:0/3428581766 shutdown_connections 2023-11-06T02:11:44.479 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.477+0000 7f2dae4b8640 1 -- 172.21.15.49:0/3428581766 wait complete. 2023-11-06T02:11:44.492 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T02:11:44.622 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.625+0000 7faba7729640 1 Processor -- start 2023-11-06T02:11:44.622 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.625+0000 7faba7729640 1 -- start start 2023-11-06T02:11:44.624 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.625+0000 7faba7729640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7faba015a880 0x7faba00b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:44.624 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.625+0000 7faba7729640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7faba0070160 0x7faba0070610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:44.624 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.625+0000 7faba7729640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faba00ae360 con 0x7faba006d150 2023-11-06T02:11:44.624 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.625+0000 7faba7729640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faba0070b50 con 0x7faba0070160 2023-11-06T02:11:44.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.625+0000 7faba7729640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faba00ae640 con 0x7faba015a880 2023-11-06T02:11:44.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.625+0000 7faba4cc4640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7faba0070160 0x7faba0070610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:44.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.625+0000 7faba4cc4640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7faba0070160 0x7faba0070610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49536/0 (socket says 172.21.15.49:49536) 2023-11-06T02:11:44.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.625+0000 7faba4cc4640 1 -- 172.21.15.49:0/1535759705 learned_addr learned my addr 172.21.15.49:0/1535759705 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:44.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.625+0000 7faba54c5640 1 --2- 172.21.15.49:0/1535759705 >> v2:172.21.15.150:3300/0 conn(0x7faba015a880 0x7faba00b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:44.626 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.625+0000 7faba4cc4640 1 -- 172.21.15.49:0/1535759705 >> v2:172.21.15.150:3300/0 conn(0x7faba015a880 msgr2=0x7faba00b15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:44.626 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.625+0000 7faba4cc4640 1 --2- 172.21.15.49:0/1535759705 >> v2:172.21.15.150:3300/0 conn(0x7faba015a880 0x7faba00b15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:44.626 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.625+0000 7faba4cc4640 1 -- 172.21.15.49:0/1535759705 >> v1:172.21.15.150:6789/0 conn(0x7faba006d150 legacy=0x7faba006d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:44.626 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.625+0000 7faba4cc4640 1 -- 172.21.15.49:0/1535759705 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faba00ae920 con 0x7faba0070160 2023-11-06T02:11:44.626 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.625+0000 7faba54c5640 1 --2- 172.21.15.49:0/1535759705 >> v2:172.21.15.150:3300/0 conn(0x7faba015a880 0x7faba00b15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:11:44.630 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.629+0000 7faba4cc4640 1 --2- 172.21.15.49:0/1535759705 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faba0070160 0x7faba0070610 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7faba00cafc0 tx=0x7fab9001c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d4afa1395a547f4 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:44.631 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.629+0000 7fab97fff640 1 -- 172.21.15.49:0/1535759705 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab900ab610 con 0x7faba0070160 2023-11-06T02:11:44.631 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.629+0000 7fab97fff640 1 -- 172.21.15.49:0/1535759705 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab900ab770 con 0x7faba0070160 2023-11-06T02:11:44.631 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.629+0000 7fab97fff640 1 -- 172.21.15.49:0/1535759705 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab900abb00 con 0x7faba0070160 2023-11-06T02:11:44.631 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.629+0000 7faba7729640 1 -- 172.21.15.49:0/1535759705 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faba0070160 msgr2=0x7faba0070610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:44.631 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.629+0000 7faba7729640 1 --2- 172.21.15.49:0/1535759705 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faba0070160 0x7faba0070610 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7faba00cafc0 tx=0x7fab9001c500 comp rx=0 tx=0).stop 2023-11-06T02:11:44.632 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.629+0000 7faba7729640 1 -- 172.21.15.49:0/1535759705 shutdown_connections 2023-11-06T02:11:44.632 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.629+0000 7faba7729640 1 --2- 172.21.15.49:0/1535759705 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faba0070160 0x7faba0070610 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:44.632 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.629+0000 7faba7729640 1 --2- 172.21.15.49:0/1535759705 >> v2:172.21.15.150:3300/0 conn(0x7faba015a880 0x7faba00b15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:44.632 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.629+0000 7faba7729640 1 -- 172.21.15.49:0/1535759705 >> 172.21.15.49:0/1535759705 conn(0x7faba006a370 msgr2=0x7faba00ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:44.632 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.629+0000 7faba7729640 1 -- 172.21.15.49:0/1535759705 shutdown_connections 2023-11-06T02:11:44.633 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.629+0000 7faba7729640 1 -- 172.21.15.49:0/1535759705 wait complete. 2023-11-06T02:11:44.633 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.629+0000 7faba7729640 1 Processor -- start 2023-11-06T02:11:44.633 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.629+0000 7faba7729640 1 -- start start 2023-11-06T02:11:44.633 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.633+0000 7faba7729640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faba006d150 0x7faba014a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:44.633 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.633+0000 7faba7729640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faba0070160 0x7faba0157ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:44.633 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.633+0000 7faba7729640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faba015a880 0x7faba00c8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:44.634 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.633+0000 7faba7729640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faba0156610 con 0x7faba015a880 2023-11-06T02:11:44.634 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.633+0000 7faba7729640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faba00c8a10 con 0x7faba006d150 2023-11-06T02:11:44.634 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.633+0000 7faba7729640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faba0152300 con 0x7faba0070160 2023-11-06T02:11:44.634 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.633+0000 7faba5cc6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faba015a880 0x7faba00c8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:44.634 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.633+0000 7faba5cc6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faba015a880 0x7faba00c8ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49540/0 (socket says 172.21.15.49:49540) 2023-11-06T02:11:44.635 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.633+0000 7faba5cc6640 1 -- 172.21.15.49:0/953178267 learned_addr learned my addr 172.21.15.49:0/953178267 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:44.635 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.633+0000 7faba54c5640 1 --2- 172.21.15.49:0/953178267 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faba006d150 0x7faba014a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:44.635 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.633+0000 7faba4cc4640 1 --2- 172.21.15.49:0/953178267 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faba0070160 0x7faba0157ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:44.635 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.633+0000 7faba5cc6640 1 -- 172.21.15.49:0/953178267 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faba0070160 msgr2=0x7faba0157ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:44.635 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.633+0000 7faba5cc6640 1 --2- 172.21.15.49:0/953178267 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faba0070160 0x7faba0157ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:44.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.633+0000 7faba5cc6640 1 -- 172.21.15.49:0/953178267 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faba006d150 msgr2=0x7faba014a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:44.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.633+0000 7faba5cc6640 1 --2- 172.21.15.49:0/953178267 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faba006d150 0x7faba014a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:44.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.633+0000 7faba5cc6640 1 -- 172.21.15.49:0/953178267 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab900094d0 con 0x7faba015a880 2023-11-06T02:11:44.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.633+0000 7faba5cc6640 1 --2- 172.21.15.49:0/953178267 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faba015a880 0x7faba00c8ea0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fab9800c770 tx=0x7fab9800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:44.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.633+0000 7fab95ffb640 1 -- 172.21.15.49:0/953178267 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab98003b80 con 0x7faba015a880 2023-11-06T02:11:44.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.633+0000 7fab95ffb640 1 -- 172.21.15.49:0/953178267 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab98003ce0 con 0x7faba015a880 2023-11-06T02:11:44.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.633+0000 7faba7729640 1 -- 172.21.15.49:0/953178267 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faba00c7e00 con 0x7faba015a880 2023-11-06T02:11:44.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.633+0000 7fab95ffb640 1 -- 172.21.15.49:0/953178267 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9800aa90 con 0x7faba015a880 2023-11-06T02:11:44.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.633+0000 7faba7729640 1 -- 172.21.15.49:0/953178267 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faba00c83c0 con 0x7faba015a880 2023-11-06T02:11:44.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.633+0000 7faba7729640 1 -- 172.21.15.49:0/953178267 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faba00c8b80 con 0x7faba015a880 2023-11-06T02:11:44.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.633+0000 7fab95ffb640 1 -- 172.21.15.49:0/953178267 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fab9800abf0 con 0x7faba015a880 2023-11-06T02:11:44.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.633+0000 7fab95ffb640 1 --2- 172.21.15.49:0/953178267 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fab84065a90 0x7fab84067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:44.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.633+0000 7fab95ffb640 1 -- 172.21.15.49:0/953178267 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fab98083090 con 0x7faba015a880 2023-11-06T02:11:44.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.633+0000 7faba54c5640 1 --2- 172.21.15.49:0/953178267 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fab84065a90 0x7fab84067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:44.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.637+0000 7faba54c5640 1 --2- 172.21.15.49:0/953178267 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fab84065a90 0x7fab84067f50 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7faba00cab70 tx=0x7fab9c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:44.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.637+0000 7fab95ffb640 1 -- 172.21.15.49:0/953178267 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fab98050980 con 0x7faba015a880 2023-11-06T02:11:44.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.797+0000 7faba7729640 1 -- 172.21.15.49:0/953178267 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7faba006c430 con 0x7faba015a880 2023-11-06T02:11:44.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.797+0000 7fab95ffb640 1 -- 172.21.15.49:0/953178267 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v65) v1 ==== 104+0+11 (secure 0 0 0) 0x7fab98053fa0 con 0x7faba015a880 2023-11-06T02:11:44.794 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2023-11-06T02:11:44.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.797+0000 7faba7729640 1 -- 172.21.15.49:0/953178267 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fab84065a90 msgr2=0x7fab84067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:44.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.797+0000 7faba7729640 1 --2- 172.21.15.49:0/953178267 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fab84065a90 0x7fab84067f50 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7faba00cab70 tx=0x7fab9c008040 comp rx=0 tx=0).stop 2023-11-06T02:11:44.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.797+0000 7faba7729640 1 -- 172.21.15.49:0/953178267 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faba015a880 msgr2=0x7faba00c8ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:44.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.797+0000 7faba7729640 1 --2- 172.21.15.49:0/953178267 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faba015a880 0x7faba00c8ea0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fab9800c770 tx=0x7fab9800cc30 comp rx=0 tx=0).stop 2023-11-06T02:11:44.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.797+0000 7faba7729640 1 -- 172.21.15.49:0/953178267 shutdown_connections 2023-11-06T02:11:44.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.797+0000 7faba7729640 1 --2- 172.21.15.49:0/953178267 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fab84065a90 0x7fab84067f50 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:44.798 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.797+0000 7faba7729640 1 --2- 172.21.15.49:0/953178267 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faba015a880 0x7faba00c8ea0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:44.798 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.797+0000 7faba7729640 1 --2- 172.21.15.49:0/953178267 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faba0070160 0x7faba0157ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:44.798 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.797+0000 7faba7729640 1 --2- 172.21.15.49:0/953178267 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faba006d150 0x7faba014a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:44.798 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.797+0000 7faba7729640 1 -- 172.21.15.49:0/953178267 >> 172.21.15.49:0/953178267 conn(0x7faba006a370 msgr2=0x7faba015d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:44.799 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.797+0000 7faba7729640 1 -- 172.21.15.49:0/953178267 shutdown_connections 2023-11-06T02:11:44.799 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.797+0000 7faba7729640 1 -- 172.21.15.49:0/953178267 wait complete. 2023-11-06T02:11:44.813 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T02:11:44.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.937+0000 7f287d301640 1 Processor -- start 2023-11-06T02:11:44.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.937+0000 7f287d301640 1 -- start start 2023-11-06T02:11:44.935 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.937+0000 7f287d301640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f28780cf1f0 0x7f287815dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:44.936 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.937+0000 7f287d301640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f28780cda10 0x7f28780cf5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:44.936 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.937+0000 7f287d301640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28780d2810 con 0x7f28780d1fa0 2023-11-06T02:11:44.936 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.937+0000 7f287d301640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28780d1b10 con 0x7f28780cf1f0 2023-11-06T02:11:44.936 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.937+0000 7f287d301640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28780d0f70 con 0x7f28780cda10 2023-11-06T02:11:44.936 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.937+0000 7f2876ffd640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f28780cf1f0 0x7f287815dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:44.937 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.937+0000 7f2876ffd640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f28780cf1f0 0x7f287815dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49546/0 (socket says 172.21.15.49:49546) 2023-11-06T02:11:44.937 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.937+0000 7f2876ffd640 1 -- 172.21.15.49:0/536361305 learned_addr learned my addr 172.21.15.49:0/536361305 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:44.937 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.937+0000 7f28767fc640 1 --2- 172.21.15.49:0/536361305 >> v2:172.21.15.150:3300/0 conn(0x7f28780cda10 0x7f28780cf5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:44.937 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.937+0000 7f2876ffd640 1 -- 172.21.15.49:0/536361305 >> v2:172.21.15.150:3300/0 conn(0x7f28780cda10 msgr2=0x7f28780cf5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:44.937 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.937+0000 7f2876ffd640 1 --2- 172.21.15.49:0/536361305 >> v2:172.21.15.150:3300/0 conn(0x7f28780cda10 0x7f28780cf5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:44.938 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.937+0000 7f2876ffd640 1 -- 172.21.15.49:0/536361305 >> v1:172.21.15.150:6789/0 conn(0x7f28780d1fa0 legacy=0x7f287815e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:44.938 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.937+0000 7f2876ffd640 1 -- 172.21.15.49:0/536361305 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f287815e190 con 0x7f28780cf1f0 2023-11-06T02:11:44.939 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.941+0000 7f2876ffd640 1 --2- 172.21.15.49:0/536361305 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f28780cf1f0 0x7f287815dc50 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f2868009870 tx=0x7f286801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1272e1c89ebf3208 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:44.939 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.941+0000 7f2875ffb640 1 -- 172.21.15.49:0/536361305 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28680ab650 con 0x7f28780cf1f0 2023-11-06T02:11:44.939 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.941+0000 7f2875ffb640 1 -- 172.21.15.49:0/536361305 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28680ab7b0 con 0x7f28780cf1f0 2023-11-06T02:11:44.940 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.941+0000 7f2875ffb640 1 -- 172.21.15.49:0/536361305 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28680abb40 con 0x7f28780cf1f0 2023-11-06T02:11:44.940 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.941+0000 7f287d301640 1 -- 172.21.15.49:0/536361305 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f28780cf1f0 msgr2=0x7f287815dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:44.940 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.941+0000 7f287d301640 1 --2- 172.21.15.49:0/536361305 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f28780cf1f0 0x7f287815dc50 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f2868009870 tx=0x7f286801c490 comp rx=0 tx=0).stop 2023-11-06T02:11:44.940 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.941+0000 7f287d301640 1 -- 172.21.15.49:0/536361305 shutdown_connections 2023-11-06T02:11:44.940 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.941+0000 7f287d301640 1 --2- 172.21.15.49:0/536361305 >> v2:172.21.15.150:3300/0 conn(0x7f28780cda10 0x7f28780cf5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:44.941 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.941+0000 7f287d301640 1 --2- 172.21.15.49:0/536361305 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f28780cf1f0 0x7f287815dc50 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:44.941 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.941+0000 7f287d301640 1 -- 172.21.15.49:0/536361305 >> 172.21.15.49:0/536361305 conn(0x7f2878071aa0 msgr2=0x7f287806b110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:44.941 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.941+0000 7f287d301640 1 -- 172.21.15.49:0/536361305 shutdown_connections 2023-11-06T02:11:44.941 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.941+0000 7f287d301640 1 -- 172.21.15.49:0/536361305 wait complete. 2023-11-06T02:11:44.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.941+0000 7f287d301640 1 Processor -- start 2023-11-06T02:11:44.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.941+0000 7f287d301640 1 -- start start 2023-11-06T02:11:44.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.941+0000 7f287d301640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f28780cda10 0x7f2878151570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:44.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.941+0000 7f287d301640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f28780cf1f0 0x7f28781537a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:44.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.941+0000 7f287d301640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f28780d1fa0 0x7f28780c8790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:44.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.945+0000 7f287d301640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2878153050 con 0x7f28780d1fa0 2023-11-06T02:11:44.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.945+0000 7f287d301640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28780cabe0 con 0x7f28780cda10 2023-11-06T02:11:44.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.945+0000 7f287d301640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2878156890 con 0x7f28780cf1f0 2023-11-06T02:11:44.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.945+0000 7f28777fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f28780d1fa0 0x7f28780c8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:44.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.945+0000 7f28777fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f28780d1fa0 0x7f28780c8790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49554/0 (socket says 172.21.15.49:49554) 2023-11-06T02:11:44.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.945+0000 7f28777fe640 1 -- 172.21.15.49:0/2360082342 learned_addr learned my addr 172.21.15.49:0/2360082342 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:44.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.945+0000 7f2876ffd640 1 --2- 172.21.15.49:0/2360082342 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f28780cda10 0x7f2878151570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:44.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.945+0000 7f28767fc640 1 --2- 172.21.15.49:0/2360082342 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f28780cf1f0 0x7f28781537a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:44.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.945+0000 7f28777fe640 1 -- 172.21.15.49:0/2360082342 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f28780cf1f0 msgr2=0x7f28781537a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:44.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.945+0000 7f28777fe640 1 --2- 172.21.15.49:0/2360082342 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f28780cf1f0 0x7f28781537a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:44.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.945+0000 7f28777fe640 1 -- 172.21.15.49:0/2360082342 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f28780cda10 msgr2=0x7f2878151570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:44.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.945+0000 7f28777fe640 1 --2- 172.21.15.49:0/2360082342 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f28780cda10 0x7f2878151570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:44.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.945+0000 7f28777fe640 1 -- 172.21.15.49:0/2360082342 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28680094d0 con 0x7f28780d1fa0 2023-11-06T02:11:44.948 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.945+0000 7f28777fe640 1 --2- 172.21.15.49:0/2360082342 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f28780d1fa0 0x7f28780c8790 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f287000b970 tx=0x7f287000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:44.948 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.945+0000 7f285f7fe640 1 -- 172.21.15.49:0/2360082342 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f287000cb40 con 0x7f28780d1fa0 2023-11-06T02:11:44.948 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.945+0000 7f285f7fe640 1 -- 172.21.15.49:0/2360082342 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f287000cca0 con 0x7f28780d1fa0 2023-11-06T02:11:44.948 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.945+0000 7f287d301640 1 -- 172.21.15.49:0/2360082342 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2878151ab0 con 0x7f28780d1fa0 2023-11-06T02:11:44.949 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.945+0000 7f285f7fe640 1 -- 172.21.15.49:0/2360082342 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2870014680 con 0x7f28780d1fa0 2023-11-06T02:11:44.949 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.945+0000 7f287d301640 1 -- 172.21.15.49:0/2360082342 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2878155ad0 con 0x7f28780d1fa0 2023-11-06T02:11:44.949 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.945+0000 7f287d301640 1 -- 172.21.15.49:0/2360082342 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28780c75b0 con 0x7f28780d1fa0 2023-11-06T02:11:44.949 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.945+0000 7f285f7fe640 1 -- 172.21.15.49:0/2360082342 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f287001d3b0 con 0x7f28780d1fa0 2023-11-06T02:11:44.949 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.945+0000 7f285f7fe640 1 --2- 172.21.15.49:0/2360082342 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2864065a60 0x7f2864067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:44.950 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.945+0000 7f285f7fe640 1 -- 172.21.15.49:0/2360082342 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2870083200 con 0x7f28780d1fa0 2023-11-06T02:11:44.950 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.945+0000 7f2876ffd640 1 --2- 172.21.15.49:0/2360082342 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2864065a60 0x7f2864067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:44.950 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.945+0000 7f2876ffd640 1 --2- 172.21.15.49:0/2360082342 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2864065a60 0x7f2864067f20 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f28680af000 tx=0x7f28680a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:44.951 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:44.949+0000 7f285f7fe640 1 -- 172.21.15.49:0/2360082342 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f28700509c0 con 0x7f28780d1fa0 2023-11-06T02:11:45.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.105+0000 7f287d301640 1 -- 172.21.15.49:0/2360082342 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2878067140 con 0x7f28780d1fa0 2023-11-06T02:11:45.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.105+0000 7f285f7fe640 1 -- 172.21.15.49:0/2360082342 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v65) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2870053fe0 con 0x7f28780d1fa0 2023-11-06T02:11:45.102 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2023-11-06T02:11:45.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.105+0000 7f287d301640 1 -- 172.21.15.49:0/2360082342 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2864065a60 msgr2=0x7f2864067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:45.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.105+0000 7f287d301640 1 --2- 172.21.15.49:0/2360082342 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2864065a60 0x7f2864067f20 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f28680af000 tx=0x7f28680a7000 comp rx=0 tx=0).stop 2023-11-06T02:11:45.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.105+0000 7f287d301640 1 -- 172.21.15.49:0/2360082342 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f28780d1fa0 msgr2=0x7f28780c8790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:45.105 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.105+0000 7f287d301640 1 --2- 172.21.15.49:0/2360082342 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f28780d1fa0 0x7f28780c8790 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f287000b970 tx=0x7f287000be30 comp rx=0 tx=0).stop 2023-11-06T02:11:45.105 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.105+0000 7f287d301640 1 -- 172.21.15.49:0/2360082342 shutdown_connections 2023-11-06T02:11:45.105 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.105+0000 7f287d301640 1 --2- 172.21.15.49:0/2360082342 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2864065a60 0x7f2864067f20 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:45.105 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.105+0000 7f287d301640 1 --2- 172.21.15.49:0/2360082342 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f28780d1fa0 0x7f28780c8790 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:45.105 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.105+0000 7f287d301640 1 --2- 172.21.15.49:0/2360082342 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f28780cf1f0 0x7f28781537a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:45.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.105+0000 7f287d301640 1 --2- 172.21.15.49:0/2360082342 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f28780cda10 0x7f2878151570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:45.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.105+0000 7f287d301640 1 -- 172.21.15.49:0/2360082342 >> 172.21.15.49:0/2360082342 conn(0x7f2878071aa0 msgr2=0x7f287815a030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:45.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.105+0000 7f287d301640 1 -- 172.21.15.49:0/2360082342 shutdown_connections 2023-11-06T02:11:45.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.105+0000 7f287d301640 1 -- 172.21.15.49:0/2360082342 wait complete. 2023-11-06T02:11:45.121 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-06T02:11:45.248 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.249+0000 7fbc58ff8640 1 Processor -- start 2023-11-06T02:11:45.249 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.249+0000 7fbc58ff8640 1 -- start start 2023-11-06T02:11:45.250 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.253+0000 7fbc58ff8640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fbc540b1f90 0x7fbc540cda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:45.250 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.253+0000 7fbc58ff8640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fbc54161650 0x7fbc540d1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:45.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.253+0000 7fbc58ff8640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbc540d2190 0x7fbc540714a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:45.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.253+0000 7fbc58ff8640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc540d17c0 con 0x7fbc54161650 2023-11-06T02:11:45.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.253+0000 7fbc58ff8640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc54161b00 con 0x7fbc540b1f90 2023-11-06T02:11:45.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.253+0000 7fbc58ff8640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc540ce6c0 con 0x7fbc540d2190 2023-11-06T02:11:45.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.253+0000 7fbc4bfff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fbc54161650 0x7fbc540d1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:45.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.253+0000 7fbc4bfff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fbc54161650 0x7fbc540d1c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49570/0 (socket says 172.21.15.49:49570) 2023-11-06T02:11:45.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.253+0000 7fbc4bfff640 1 -- 172.21.15.49:0/2787138674 learned_addr learned my addr 172.21.15.49:0/2787138674 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:45.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.253+0000 7fbc52d9d640 1 --2- 172.21.15.49:0/2787138674 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbc540d2190 0x7fbc540714a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:45.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.253+0000 7fbc5259c640 1 --2- 172.21.15.49:0/2787138674 >> v2:172.21.15.150:3300/0 conn(0x7fbc540b1f90 0x7fbc540cda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:45.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.253+0000 7fbc4bfff640 1 -- 172.21.15.49:0/2787138674 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbc540d2190 msgr2=0x7fbc540714a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:45.253 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.253+0000 7fbc4bfff640 1 --2- 172.21.15.49:0/2787138674 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbc540d2190 0x7fbc540714a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:45.253 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.253+0000 7fbc4bfff640 1 -- 172.21.15.49:0/2787138674 >> v2:172.21.15.150:3300/0 conn(0x7fbc540b1f90 msgr2=0x7fbc540cda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:45.253 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.253+0000 7fbc4bfff640 1 --2- 172.21.15.49:0/2787138674 >> v2:172.21.15.150:3300/0 conn(0x7fbc540b1f90 0x7fbc540cda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:45.253 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.253+0000 7fbc4bfff640 1 -- 172.21.15.49:0/2787138674 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc540ce100 con 0x7fbc54161650 2023-11-06T02:11:45.254 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.257+0000 7fbc4bfff640 1 --2- 172.21.15.49:0/2787138674 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbc54161650 0x7fbc540d1c50 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fbc54072180 tx=0x7fbc3c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae0f5391a8345c4a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:45.255 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.257+0000 7fbc51d9b640 1 -- 172.21.15.49:0/2787138674 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc3c0ac600 con 0x7fbc54161650 2023-11-06T02:11:45.255 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.257+0000 7fbc51d9b640 1 -- 172.21.15.49:0/2787138674 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc3c0a8050 con 0x7fbc54161650 2023-11-06T02:11:45.255 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.257+0000 7fbc51d9b640 1 -- 172.21.15.49:0/2787138674 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc3c0ac980 con 0x7fbc54161650 2023-11-06T02:11:45.255 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.257+0000 7fbc58ff8640 1 -- 172.21.15.49:0/2787138674 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbc54161650 msgr2=0x7fbc540d1c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:45.256 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.257+0000 7fbc58ff8640 1 --2- 172.21.15.49:0/2787138674 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbc54161650 0x7fbc540d1c50 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fbc54072180 tx=0x7fbc3c01c4d0 comp rx=0 tx=0).stop 2023-11-06T02:11:45.256 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.257+0000 7fbc58ff8640 1 -- 172.21.15.49:0/2787138674 shutdown_connections 2023-11-06T02:11:45.256 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.257+0000 7fbc58ff8640 1 --2- 172.21.15.49:0/2787138674 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbc540d2190 0x7fbc540714a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:45.256 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.257+0000 7fbc58ff8640 1 --2- 172.21.15.49:0/2787138674 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbc54161650 0x7fbc540d1c50 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:45.257 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.257+0000 7fbc58ff8640 1 --2- 172.21.15.49:0/2787138674 >> v2:172.21.15.150:3300/0 conn(0x7fbc540b1f90 0x7fbc540cda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:45.257 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.257+0000 7fbc58ff8640 1 -- 172.21.15.49:0/2787138674 >> 172.21.15.49:0/2787138674 conn(0x7fbc540d2aa0 msgr2=0x7fbc540d2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:45.257 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.257+0000 7fbc58ff8640 1 -- 172.21.15.49:0/2787138674 shutdown_connections 2023-11-06T02:11:45.257 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.257+0000 7fbc58ff8640 1 -- 172.21.15.49:0/2787138674 wait complete. 2023-11-06T02:11:45.260 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.257+0000 7fbc58ff8640 1 Processor -- start 2023-11-06T02:11:45.260 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.257+0000 7fbc58ff8640 1 -- start start 2023-11-06T02:11:45.260 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.257+0000 7fbc58ff8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbc540b1f90 0x7fbc540c8180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:45.260 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.257+0000 7fbc58ff8640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbc540d2190 0x7fbc54141640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:45.261 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.257+0000 7fbc58ff8640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbc54161650 0x7fbc54142e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:45.261 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.257+0000 7fbc58ff8640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc54141b80 con 0x7fbc540b1f90 2023-11-06T02:11:45.261 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.257+0000 7fbc58ff8640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc54141d70 con 0x7fbc54161650 2023-11-06T02:11:45.261 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.257+0000 7fbc58ff8640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc54142050 con 0x7fbc540d2190 2023-11-06T02:11:45.261 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.257+0000 7fbc5259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbc540b1f90 0x7fbc540c8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:45.262 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.257+0000 7fbc5259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbc540b1f90 0x7fbc540c8180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49574/0 (socket says 172.21.15.49:49574) 2023-11-06T02:11:45.262 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.257+0000 7fbc5259c640 1 -- 172.21.15.49:0/2606947353 learned_addr learned my addr 172.21.15.49:0/2606947353 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:45.262 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.257+0000 7fbc4bfff640 1 --2- 172.21.15.49:0/2606947353 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbc540d2190 0x7fbc54141640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:45.262 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.257+0000 7fbc52d9d640 1 --2- 172.21.15.49:0/2606947353 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbc54161650 0x7fbc54142e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:45.262 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.257+0000 7fbc5259c640 1 -- 172.21.15.49:0/2606947353 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbc540d2190 msgr2=0x7fbc54141640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:45.263 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.257+0000 7fbc5259c640 1 --2- 172.21.15.49:0/2606947353 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbc540d2190 0x7fbc54141640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:45.263 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.257+0000 7fbc5259c640 1 -- 172.21.15.49:0/2606947353 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbc54161650 msgr2=0x7fbc54142e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:45.263 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.257+0000 7fbc5259c640 1 --2- 172.21.15.49:0/2606947353 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbc54161650 0x7fbc54142e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:45.263 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.257+0000 7fbc5259c640 1 -- 172.21.15.49:0/2606947353 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc3c0094d0 con 0x7fbc540b1f90 2023-11-06T02:11:45.263 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.257+0000 7fbc5259c640 1 --2- 172.21.15.49:0/2606947353 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbc540b1f90 0x7fbc540c8180 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fbc4400d800 tx=0x7fbc4400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:45.264 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.261+0000 7fbc4affd640 1 -- 172.21.15.49:0/2606947353 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc440096a0 con 0x7fbc540b1f90 2023-11-06T02:11:45.264 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.261+0000 7fbc4affd640 1 -- 172.21.15.49:0/2606947353 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc44009800 con 0x7fbc540b1f90 2023-11-06T02:11:45.264 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.261+0000 7fbc58ff8640 1 -- 172.21.15.49:0/2606947353 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc54142330 con 0x7fbc540b1f90 2023-11-06T02:11:45.264 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.261+0000 7fbc4affd640 1 -- 172.21.15.49:0/2606947353 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc44009b70 con 0x7fbc540b1f90 2023-11-06T02:11:45.264 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.261+0000 7fbc58ff8640 1 -- 172.21.15.49:0/2606947353 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc541428f0 con 0x7fbc540b1f90 2023-11-06T02:11:45.265 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.261+0000 7fbc58ff8640 1 -- 172.21.15.49:0/2606947353 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc54074040 con 0x7fbc540b1f90 2023-11-06T02:11:45.265 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.261+0000 7fbc4affd640 1 -- 172.21.15.49:0/2606947353 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fbc44009cd0 con 0x7fbc540b1f90 2023-11-06T02:11:45.265 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.261+0000 7fbc4affd640 1 --2- 172.21.15.49:0/2606947353 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fbc34065a90 0x7fbc34067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:45.265 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.261+0000 7fbc4affd640 1 -- 172.21.15.49:0/2606947353 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbc44083f60 con 0x7fbc540b1f90 2023-11-06T02:11:45.265 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.261+0000 7fbc4bfff640 1 --2- 172.21.15.49:0/2606947353 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fbc34065a90 0x7fbc34067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:45.265 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.261+0000 7fbc4bfff640 1 --2- 172.21.15.49:0/2606947353 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fbc34065a90 0x7fbc34067f50 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c0ae000 tx=0x7fbc3c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:45.266 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.265+0000 7fbc4affd640 1 -- 172.21.15.49:0/2606947353 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fbc440518a0 con 0x7fbc540b1f90 2023-11-06T02:11:45.432 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.433+0000 7fbc58ff8640 1 -- 172.21.15.49:0/2606947353 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fbc540cd0a0 con 0x7fbc540b1f90 2023-11-06T02:11:45.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.433+0000 7fbc4affd640 1 -- 172.21.15.49:0/2606947353 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v65) v1 ==== 80+0+0 (secure 0 0 0) 0x7fbc44054ec0 con 0x7fbc540b1f90 2023-11-06T02:11:45.836 INFO:teuthology.orchestra.run.smithi049.stderr:listed 0 entries 2023-11-06T02:11:45.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.437+0000 7fbc58ff8640 1 -- 172.21.15.49:0/2606947353 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fbc34065a90 msgr2=0x7fbc34067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:45.839 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.437+0000 7fbc58ff8640 1 --2- 172.21.15.49:0/2606947353 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fbc34065a90 0x7fbc34067f50 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c0ae000 tx=0x7fbc3c008040 comp rx=0 tx=0).stop 2023-11-06T02:11:45.839 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.437+0000 7fbc58ff8640 1 -- 172.21.15.49:0/2606947353 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbc540b1f90 msgr2=0x7fbc540c8180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:45.839 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.437+0000 7fbc58ff8640 1 --2- 172.21.15.49:0/2606947353 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbc540b1f90 0x7fbc540c8180 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fbc4400d800 tx=0x7fbc4400dcc0 comp rx=0 tx=0).stop 2023-11-06T02:11:45.840 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.437+0000 7fbc58ff8640 1 -- 172.21.15.49:0/2606947353 shutdown_connections 2023-11-06T02:11:45.840 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.437+0000 7fbc58ff8640 1 --2- 172.21.15.49:0/2606947353 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fbc34065a90 0x7fbc34067f50 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:45.840 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.437+0000 7fbc58ff8640 1 --2- 172.21.15.49:0/2606947353 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbc54161650 0x7fbc54142e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:45.840 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.437+0000 7fbc58ff8640 1 --2- 172.21.15.49:0/2606947353 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbc540d2190 0x7fbc54141640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:45.841 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.437+0000 7fbc58ff8640 1 --2- 172.21.15.49:0/2606947353 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbc540b1f90 0x7fbc540c8180 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:45.841 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.437+0000 7fbc58ff8640 1 -- 172.21.15.49:0/2606947353 >> 172.21.15.49:0/2606947353 conn(0x7fbc540d2aa0 msgr2=0x7fbc540b83f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:45.841 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.437+0000 7fbc58ff8640 1 -- 172.21.15.49:0/2606947353 shutdown_connections 2023-11-06T02:11:45.842 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.437+0000 7fbc58ff8640 1 -- 172.21.15.49:0/2606947353 wait complete. 2023-11-06T02:11:45.842 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-11-06T02:11:45.843 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T02:11:45.852 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T02:11:45.985 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.989+0000 7f5fc4fe9640 1 Processor -- start 2023-11-06T02:11:45.985 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.989+0000 7f5fc4fe9640 1 -- start start 2023-11-06T02:11:45.987 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.989+0000 7f5fc4fe9640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f5fc00cda70 0x7f5fc0161650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:45.987 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.989+0000 7f5fc4fe9640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fc00d17c0 con 0x7f5fc00b1f90 2023-11-06T02:11:45.987 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.989+0000 7f5fc4fe9640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fc00ce6c0 con 0x7f5fc00714a0 2023-11-06T02:11:45.988 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.989+0000 7f5fc4fe9640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fc00cdf20 con 0x7f5fc00cda70 2023-11-06T02:11:45.988 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.989+0000 7f5fb7fff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f5fc00cda70 0x7f5fc0161650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:45.988 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.989+0000 7f5fb7fff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f5fc00cda70 0x7f5fc0161650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49580/0 (socket says 172.21.15.49:49580) 2023-11-06T02:11:45.988 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.989+0000 7f5fb7fff640 1 -- 172.21.15.49:0/1587021752 learned_addr learned my addr 172.21.15.49:0/1587021752 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:45.988 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.989+0000 7f5fbe59c640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f5fc00b1f90 0x7f5fc00d1c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:42122/0 (socket says 172.21.15.49:42122) 2023-11-06T02:11:45.989 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.989+0000 7f5fb7fff640 1 -- 172.21.15.49:0/1587021752 >> v1:172.21.15.150:6789/0 conn(0x7f5fc00714a0 legacy=0x7f5fc00718b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:45.989 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.989+0000 7f5fb7fff640 1 -- 172.21.15.49:0/1587021752 >> v1:172.21.15.49:6789/0 conn(0x7f5fc00b1f90 legacy=0x7f5fc00d1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:45.989 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.989+0000 7f5fbdd9b640 1 -- 172.21.15.49:0/1587021752 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3768938416 0 0) 0x7f5fc00d17c0 con 0x7f5fc00b1f90 2023-11-06T02:11:45.989 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.989+0000 7f5fb7fff640 1 -- 172.21.15.49:0/1587021752 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fc00d2360 con 0x7f5fc00cda70 2023-11-06T02:11:45.991 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.993+0000 7f5fb7fff640 1 --2- 172.21.15.49:0/1587021752 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5fc00cda70 0x7f5fc0161650 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f5fa8009870 tx=0x7f5fa801c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=208cd012f72e4188 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:45.991 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.993+0000 7f5fbdd9b640 1 -- 172.21.15.49:0/1587021752 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa80ab5f0 con 0x7f5fc00cda70 2023-11-06T02:11:45.991 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.993+0000 7f5fbdd9b640 1 -- 172.21.15.49:0/1587021752 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fa80ab750 con 0x7f5fc00cda70 2023-11-06T02:11:45.991 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.993+0000 7f5fbdd9b640 1 -- 172.21.15.49:0/1587021752 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa80abae0 con 0x7f5fc00cda70 2023-11-06T02:11:45.991 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.993+0000 7f5fc4fe9640 1 -- 172.21.15.49:0/1587021752 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5fc00cda70 msgr2=0x7f5fc0161650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:45.992 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.993+0000 7f5fc4fe9640 1 --2- 172.21.15.49:0/1587021752 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5fc00cda70 0x7f5fc0161650 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f5fa8009870 tx=0x7f5fa801c500 comp rx=0 tx=0).stop 2023-11-06T02:11:45.992 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.993+0000 7f5fc4fe9640 1 -- 172.21.15.49:0/1587021752 shutdown_connections 2023-11-06T02:11:45.992 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.993+0000 7f5fc4fe9640 1 --2- 172.21.15.49:0/1587021752 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5fc00cda70 0x7f5fc0161650 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:45.992 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.993+0000 7f5fc4fe9640 1 -- 172.21.15.49:0/1587021752 >> 172.21.15.49:0/1587021752 conn(0x7f5fc00d2aa0 msgr2=0x7f5fc00d2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:45.993 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.993+0000 7f5fc4fe9640 1 -- 172.21.15.49:0/1587021752 shutdown_connections 2023-11-06T02:11:45.993 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.993+0000 7f5fc4fe9640 1 -- 172.21.15.49:0/1587021752 wait complete. 2023-11-06T02:11:45.994 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.993+0000 7f5fc4fe9640 1 Processor -- start 2023-11-06T02:11:45.994 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.993+0000 7f5fc4fe9640 1 -- start start 2023-11-06T02:11:45.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.993+0000 7f5fc4fe9640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5fc00714a0 0x7f5fc00b2780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:45.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.993+0000 7f5fc4fe9640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5fc00b1f90 0x7f5fc015f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:45.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.993+0000 7f5fc4fe9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5fc00cda70 0x7f5fc00b80a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:45.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.993+0000 7f5fc4fe9640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc00b33d0 con 0x7f5fc00cda70 2023-11-06T02:11:45.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.993+0000 7f5fc4fe9640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc00b2e10 con 0x7f5fc00b1f90 2023-11-06T02:11:45.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.993+0000 7f5fc4fe9640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc00b30f0 con 0x7f5fc00714a0 2023-11-06T02:11:45.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.993+0000 7f5fbed9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5fc00cda70 0x7f5fc00b80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:45.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.993+0000 7f5fbed9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5fc00cda70 0x7f5fc00b80a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49592/0 (socket says 172.21.15.49:49592) 2023-11-06T02:11:45.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.993+0000 7f5fbed9d640 1 -- 172.21.15.49:0/1491283298 learned_addr learned my addr 172.21.15.49:0/1491283298 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:45.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.993+0000 7f5fb7fff640 1 --2- 172.21.15.49:0/1491283298 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5fc00b1f90 0x7f5fc015f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:45.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.993+0000 7f5fbe59c640 1 --2- 172.21.15.49:0/1491283298 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5fc00714a0 0x7f5fc00b2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:45.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.993+0000 7f5fbed9d640 1 -- 172.21.15.49:0/1491283298 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5fc00714a0 msgr2=0x7f5fc00b2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:45.999 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.993+0000 7f5fbed9d640 1 --2- 172.21.15.49:0/1491283298 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5fc00714a0 0x7f5fc00b2780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:45.999 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.993+0000 7f5fbed9d640 1 -- 172.21.15.49:0/1491283298 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5fc00b1f90 msgr2=0x7f5fc015f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:45.999 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.993+0000 7f5fbed9d640 1 --2- 172.21.15.49:0/1491283298 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5fc00b1f90 0x7f5fc015f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:45.999 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.993+0000 7f5fbed9d640 1 -- 172.21.15.49:0/1491283298 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fa80094d0 con 0x7f5fc00cda70 2023-11-06T02:11:46.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.993+0000 7f5fb7fff640 1 --2- 172.21.15.49:0/1491283298 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5fc00b1f90 0x7f5fc015f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:11:46.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.993+0000 7f5fbed9d640 1 --2- 172.21.15.49:0/1491283298 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5fc00cda70 0x7f5fc00b80a0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f5fb800c780 tx=0x7f5fb800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:46.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.993+0000 7f5fb6ffd640 1 -- 172.21.15.49:0/1491283298 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb8004b70 con 0x7f5fc00cda70 2023-11-06T02:11:46.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.993+0000 7f5fb6ffd640 1 -- 172.21.15.49:0/1491283298 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fb800ce90 con 0x7f5fc00cda70 2023-11-06T02:11:46.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.993+0000 7f5fc4fe9640 1 -- 172.21.15.49:0/1491283298 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fc015d930 con 0x7f5fc00cda70 2023-11-06T02:11:46.001 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.993+0000 7f5fb6ffd640 1 -- 172.21.15.49:0/1491283298 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb800a850 con 0x7f5fc00cda70 2023-11-06T02:11:46.001 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.993+0000 7f5fc4fe9640 1 -- 172.21.15.49:0/1491283298 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fc00b85e0 con 0x7f5fc00cda70 2023-11-06T02:11:46.001 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.997+0000 7f5fc4fe9640 1 -- 172.21.15.49:0/1491283298 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fc00b8830 con 0x7f5fc00cda70 2023-11-06T02:11:46.001 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.997+0000 7f5fb6ffd640 1 -- 172.21.15.49:0/1491283298 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f5fb800a9b0 con 0x7f5fc00cda70 2023-11-06T02:11:46.001 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.997+0000 7f5fb6ffd640 1 --2- 172.21.15.49:0/1491283298 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5fa0065a60 0x7f5fa0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:46.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.997+0000 7f5fb6ffd640 1 -- 172.21.15.49:0/1491283298 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5fb8083480 con 0x7f5fc00cda70 2023-11-06T02:11:46.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.997+0000 7f5fbe59c640 1 --2- 172.21.15.49:0/1491283298 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5fa0065a60 0x7f5fa0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:46.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:45.997+0000 7f5fbe59c640 1 --2- 172.21.15.49:0/1491283298 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5fa0065a60 0x7f5fa0067f20 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f5fb0000f70 tx=0x7f5fb0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:46.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.001+0000 7f5fb6ffd640 1 -- 172.21.15.49:0/1491283298 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f5fc00b8830 con 0x7f5fc00cda70 2023-11-06T02:11:46.161 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.161+0000 7f5fc4fe9640 1 -- 172.21.15.49:0/1491283298 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5fc00b8830 con 0x7f5fc00cda70 2023-11-06T02:11:46.163 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.165+0000 7f5fb6ffd640 1 -- 172.21.15.49:0/1491283298 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+17825 (secure 0 0 0) 0x7f5fc00b8830 con 0x7f5fc00cda70 2023-11-06T02:11:46.163 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:11:46.163 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":65,"fsid":"1a8b035c-6ba4-499a-b876-aea47fd9c0d4","created":"2023-11-06T02:08:06.440857+0000","modified":"2023-11-06T02:11:31.094207+0000","last_up_change":"2023-11-06T02:08:17.982372+0000","last_in_change":"2023-11-06T02:08:08.906631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T02:08:19.252222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T02:08:24.776211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-06T02:11:28.648326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-06T02:11:29.079513+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ad55af8-0571-4e6a-8722-b5d429b7496d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6809","nonce":38116},{"type":"v1","addr":"172.21.15.49:6811","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6813","nonce":38116},{"type":"v1","addr":"172.21.15.49:6815","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6821","nonce":38116},{"type":"v1","addr":"172.21.15.49:6823","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6817","nonce":38116},{"type":"v1","addr":"172.21.15.49:6819","nonce":38116}]},"public_addr":"172.21.15.49:6811/38116","cluster_addr":"172.21.15.49:6815/38116","heartbeat_back_addr":"172.21.15.49:6823/38116","heartbeat_front_addr":"172.21.15.49:6819/38116","state":["exists","up"]},{"osd":1,"uuid":"c8b31f85-a757-43ed-9419-351acc46611b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":38113},{"type":"v1","addr":"172.21.15.49:6810","nonce":38113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":38113},{"type":"v1","addr":"172.21.15.49:6814","nonce":38113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":38113},{"type":"v1","addr":"172.21.15.49:6822","nonce":38113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":38113},{"type":"v1","addr":"172.21.15.49:6818","nonce":38113}]},"public_addr":"172.21.15.49:6810/38113","cluster_addr":"172.21.15.49:6814/38113","heartbeat_back_addr":"172.21.15.49:6822/38113","heartbeat_front_addr":"172.21.15.49:6818/38113","state":["exists","up"]},{"osd":2,"uuid":"555b3bf3-0a70-4b91-8efd-e4204b5fc26e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":38114},{"type":"v1","addr":"172.21.15.49:6801","nonce":38114}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":38114},{"type":"v1","addr":"172.21.15.49:6803","nonce":38114}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":38114},{"type":"v1","addr":"172.21.15.49:6807","nonce":38114}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":38114},{"type":"v1","addr":"172.21.15.49:6805","nonce":38114}]},"public_addr":"172.21.15.49:6801/38114","cluster_addr":"172.21.15.49:6803/38114","heartbeat_back_addr":"172.21.15.49:6807/38114","heartbeat_front_addr":"172.21.15.49:6805/38114","state":["exists","up"]},{"osd":3,"uuid":"8d4e5d77-03ef-4b9b-bfcb-5ae93a46795c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":38115},{"type":"v1","addr":"172.21.15.49:6825","nonce":38115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":38115},{"type":"v1","addr":"172.21.15.49:6827","nonce":38115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":38115},{"type":"v1","addr":"172.21.15.49:6831","nonce":38115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":38115},{"type":"v1","addr":"172.21.15.49:6829","nonce":38115}]},"public_addr":"172.21.15.49:6825/38115","cluster_addr":"172.21.15.49:6827/38115","heartbeat_back_addr":"172.21.15.49:6831/38115","heartbeat_front_addr":"172.21.15.49:6829/38115","state":["exists","up"]},{"osd":4,"uuid":"dba03ad4-68a3-4cf3-9218-ac611f02eacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":38417},{"type":"v1","addr":"172.21.15.150:6809","nonce":38417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":38417},{"type":"v1","addr":"172.21.15.150:6811","nonce":38417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":38417},{"type":"v1","addr":"172.21.15.150:6815","nonce":38417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":38417},{"type":"v1","addr":"172.21.15.150:6813","nonce":38417}]},"public_addr":"172.21.15.150:6809/38417","cluster_addr":"172.21.15.150:6811/38417","heartbeat_back_addr":"172.21.15.150:6815/38417","heartbeat_front_addr":"172.21.15.150:6813/38417","state":["exists","up"]},{"osd":5,"uuid":"0df9c303-c255-4640-8e91-01f1c3efe18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":38418},{"type":"v1","addr":"172.21.15.150:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":38418},{"type":"v1","addr":"172.21.15.150:6803","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":38418},{"type":"v1","addr":"172.21.15.150:6807","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":38418},{"type":"v1","addr":"172.21.15.150:6805","nonce":38418}]},"public_addr":"172.21.15.150:6801/38418","cluster_addr":"172.21.15.150:6803/38418","heartbeat_back_addr":"172.21.15.150:6807/38418","heartbeat_front_addr":"172.21.15.150:6805/38418","state":["exists","up"]},{"osd":6,"uuid":"e59fa189-4b70-4c06-9722-c02b90828bf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":38428},{"type":"v1","addr":"172.21.15.150:6825","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":38428},{"type":"v1","addr":"172.21.15.150:6827","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":38428},{"type":"v1","addr":"172.21.15.150:6831","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":38428},{"type":"v1","addr":"172.21.15.150:6829","nonce":38428}]},"public_addr":"172.21.15.150:6825/38428","cluster_addr":"172.21.15.150:6827/38428","heartbeat_back_addr":"172.21.15.150:6831/38428","heartbeat_front_addr":"172.21.15.150:6829/38428","state":["exists","up"]},{"osd":7,"uuid":"a64f57e1-ef98-46bd-b304-385759e95898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":38421},{"type":"v1","addr":"172.21.15.150:6817","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":38421},{"type":"v1","addr":"172.21.15.150:6819","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":38421},{"type":"v1","addr":"172.21.15.150:6823","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":38421},{"type":"v1","addr":"172.21.15.150:6821","nonce":38421}]},"public_addr":"172.21.15.150:6817/38421","cluster_addr":"172.21.15.150:6819/38421","heartbeat_back_addr":"172.21.15.150:6823/38421","heartbeat_front_addr":"172.21.15.150:6821/38421","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.438552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.413280+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.296466+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.424966+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.276953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.230984+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.410235+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.480084+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T02:11:46.165 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.169+0000 7f5fc4fe9640 1 -- 172.21.15.49:0/1491283298 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5fa0065a60 msgr2=0x7f5fa0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:46.165 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.169+0000 7f5fc4fe9640 1 --2- 172.21.15.49:0/1491283298 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5fa0065a60 0x7f5fa0067f20 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f5fb0000f70 tx=0x7f5fb0009040 comp rx=0 tx=0).stop 2023-11-06T02:11:46.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.169+0000 7f5fc4fe9640 1 -- 172.21.15.49:0/1491283298 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5fc00cda70 msgr2=0x7f5fc00b80a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:46.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.169+0000 7f5fc4fe9640 1 --2- 172.21.15.49:0/1491283298 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5fc00cda70 0x7f5fc00b80a0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f5fb800c780 tx=0x7f5fb800cc40 comp rx=0 tx=0).stop 2023-11-06T02:11:46.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.169+0000 7f5fc4fe9640 1 -- 172.21.15.49:0/1491283298 shutdown_connections 2023-11-06T02:11:46.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.169+0000 7f5fc4fe9640 1 --2- 172.21.15.49:0/1491283298 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5fa0065a60 0x7f5fa0067f20 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:46.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.169+0000 7f5fc4fe9640 1 --2- 172.21.15.49:0/1491283298 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5fc00cda70 0x7f5fc00b80a0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:46.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.169+0000 7f5fc4fe9640 1 --2- 172.21.15.49:0/1491283298 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5fc00b1f90 0x7f5fc015f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:46.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.169+0000 7f5fc4fe9640 1 --2- 172.21.15.49:0/1491283298 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5fc00714a0 0x7f5fc00b2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:46.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.169+0000 7f5fc4fe9640 1 -- 172.21.15.49:0/1491283298 >> 172.21.15.49:0/1491283298 conn(0x7f5fc00d2aa0 msgr2=0x7f5fc00c6e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:46.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.169+0000 7f5fc4fe9640 1 -- 172.21.15.49:0/1491283298 shutdown_connections 2023-11-06T02:11:46.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.169+0000 7f5fc4fe9640 1 -- 172.21.15.49:0/1491283298 wait complete. 2023-11-06T02:11:46.183 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T02:08:19.252222+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T02:08:24.776211+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T02:11:28.648326+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T02:11:29.079513+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '65', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T02:11:46.184 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T02:11:46.316 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.317+0000 7fb8585ee640 1 Processor -- start 2023-11-06T02:11:46.316 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.317+0000 7fb8585ee640 1 -- start start 2023-11-06T02:11:46.320 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.317+0000 7fb8585ee640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fb850071300 0x7fb8500716e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:46.321 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.317+0000 7fb8585ee640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8500cacc0 con 0x7fb850069d70 2023-11-06T02:11:46.321 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.317+0000 7fb8585ee640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb850071c20 con 0x7fb8500cd3b0 2023-11-06T02:11:46.321 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.317+0000 7fb8585ee640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8500cd840 con 0x7fb850071300 2023-11-06T02:11:46.322 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.321+0000 7fb856b8b640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fb850069d70 0x7fb8500cb760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:42124/0 (socket says 172.21.15.49:42124) 2023-11-06T02:11:46.322 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.321+0000 7fb856b8b640 1 -- 172.21.15.49:0/599667206 learned_addr learned my addr 172.21.15.49:0/599667206 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:11:46.322 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.321+0000 7fb85638a640 1 --2- 172.21.15.49:0/599667206 >> v2:172.21.15.150:3300/0 conn(0x7fb850071300 0x7fb8500716e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:46.322 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.321+0000 7fb855388640 1 -- 172.21.15.49:0/599667206 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 521298524 0 0) 0x7fb8500cacc0 con 0x7fb850069d70 2023-11-06T02:11:46.323 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.321+0000 7fb855388640 1 -- 172.21.15.49:0/599667206 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb83c003530 con 0x7fb850069d70 2023-11-06T02:11:46.323 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.321+0000 7fb855388640 1 -- 172.21.15.49:0/599667206 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3106175190 0 0) 0x7fb83c003530 con 0x7fb850069d70 2023-11-06T02:11:46.323 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.321+0000 7fb855388640 1 -- 172.21.15.49:0/599667206 >> v2:172.21.15.150:3300/0 conn(0x7fb850071300 msgr2=0x7fb8500716e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:46.324 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.321+0000 7fb855388640 1 --2- 172.21.15.49:0/599667206 >> v2:172.21.15.150:3300/0 conn(0x7fb850071300 0x7fb8500716e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:46.324 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.321+0000 7fb855388640 1 -- 172.21.15.49:0/599667206 >> v1:172.21.15.150:6789/0 conn(0x7fb8500cd3b0 legacy=0x7fb8500cde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:46.324 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.321+0000 7fb855388640 1 -- 172.21.15.49:0/599667206 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8500caf60 con 0x7fb850069d70 2023-11-06T02:11:46.324 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.321+0000 7fb855388640 1 -- 172.21.15.49:0/599667206 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7fb848001a00 con 0x7fb850069d70 2023-11-06T02:11:46.325 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.321+0000 7fb855388640 1 -- 172.21.15.49:0/599667206 >> v1:172.21.15.49:6789/0 conn(0x7fb850069d70 legacy=0x7fb8500cb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:46.325 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.321+0000 7fb855388640 1 --2- 172.21.15.49:0/599667206 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb83c003980 0x7fb83c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:46.325 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.321+0000 7fb855388640 1 --2- 172.21.15.49:0/599667206 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb83c0047d0 0x7fb83c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:46.326 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.321+0000 7fb855388640 1 --2- 172.21.15.49:0/599667206 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb83c005190 0x7fb83c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:46.326 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.321+0000 7fb855388640 1 -- 172.21.15.49:0/599667206 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8500cd840 con 0x7fb83c005190 2023-11-06T02:11:46.326 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.321+0000 7fb855388640 1 -- 172.21.15.49:0/599667206 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb83c00c130 con 0x7fb83c0047d0 2023-11-06T02:11:46.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.321+0000 7fb855388640 1 -- 172.21.15.49:0/599667206 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb83c00c2f0 con 0x7fb83c003980 2023-11-06T02:11:46.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.321+0000 7fb856b8b640 1 --2- 172.21.15.49:0/599667206 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb83c005190 0x7fb83c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:46.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.321+0000 7fb855b89640 1 --2- 172.21.15.49:0/599667206 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb83c0047d0 0x7fb83c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:46.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.321+0000 7fb85638a640 1 --2- 172.21.15.49:0/599667206 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb83c003980 0x7fb83c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:46.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.321+0000 7fb856b8b640 1 -- 172.21.15.49:0/599667206 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb83c003980 msgr2=0x7fb83c003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:46.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.321+0000 7fb856b8b640 1 --2- 172.21.15.49:0/599667206 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb83c003980 0x7fb83c003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:46.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.321+0000 7fb856b8b640 1 -- 172.21.15.49:0/599667206 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb83c0047d0 msgr2=0x7fb83c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:46.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.321+0000 7fb856b8b640 1 --2- 172.21.15.49:0/599667206 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb83c0047d0 0x7fb83c004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:46.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.321+0000 7fb856b8b640 1 -- 172.21.15.49:0/599667206 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb83c003530 con 0x7fb83c005190 2023-11-06T02:11:46.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.321+0000 7fb85638a640 1 --2- 172.21.15.49:0/599667206 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb83c003980 0x7fb83c003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:11:46.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.321+0000 7fb856b8b640 1 -- 172.21.15.49:0/599667206 reap_dead start 2023-11-06T02:11:46.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.321+0000 7fb856b8b640 1 --2- 172.21.15.49:0/599667206 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb83c005190 0x7fb83c00b990 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fb84800a830 tx=0x7fb84801faa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa3af93c02999227 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:46.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.321+0000 7fb855388640 1 -- 172.21.15.49:0/599667206 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8480aa6d0 con 0x7fb83c005190 2023-11-06T02:11:46.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.321+0000 7fb855388640 1 -- 172.21.15.49:0/599667206 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8480aa870 con 0x7fb83c005190 2023-11-06T02:11:46.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.321+0000 7fb855388640 1 -- 172.21.15.49:0/599667206 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8480aac20 con 0x7fb83c005190 2023-11-06T02:11:46.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.325+0000 7fb8585ee640 1 -- 172.21.15.49:0/599667206 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb83c005190 msgr2=0x7fb83c00b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:46.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.325+0000 7fb8585ee640 1 --2- 172.21.15.49:0/599667206 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb83c005190 0x7fb83c00b990 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fb84800a830 tx=0x7fb84801faa0 comp rx=0 tx=0).stop 2023-11-06T02:11:46.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.325+0000 7fb8585ee640 1 -- 172.21.15.49:0/599667206 shutdown_connections 2023-11-06T02:11:46.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.325+0000 7fb8585ee640 1 --2- 172.21.15.49:0/599667206 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb83c005190 0x7fb83c00b990 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:46.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.325+0000 7fb8585ee640 1 -- 172.21.15.49:0/599667206 >> 172.21.15.49:0/599667206 conn(0x7fb8500cf7e0 msgr2=0x7fb8500cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:46.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.325+0000 7fb8585ee640 1 -- 172.21.15.49:0/599667206 shutdown_connections 2023-11-06T02:11:46.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.325+0000 7fb8585ee640 1 -- 172.21.15.49:0/599667206 wait complete. 2023-11-06T02:11:46.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.325+0000 7fb8585ee640 1 Processor -- start 2023-11-06T02:11:46.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.325+0000 7fb8585ee640 1 -- start start 2023-11-06T02:11:46.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.325+0000 7fb8585ee640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb83c005190 0x7fb850157b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:46.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.325+0000 7fb8585ee640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb8500c5f30 0x7fb8500c4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:46.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.325+0000 7fb8585ee640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8500b3610 0x7fb85006a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:46.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.325+0000 7fb8585ee640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8500c4e90 con 0x7fb83c005190 2023-11-06T02:11:46.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.325+0000 7fb8585ee640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8500c5170 con 0x7fb8500b3610 2023-11-06T02:11:46.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.325+0000 7fb8585ee640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8500c5450 con 0x7fb8500c5f30 2023-11-06T02:11:46.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.325+0000 7fb85638a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb83c005190 0x7fb850157b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:46.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.325+0000 7fb85638a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb83c005190 0x7fb850157b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49610/0 (socket says 172.21.15.49:49610) 2023-11-06T02:11:46.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.325+0000 7fb85638a640 1 -- 172.21.15.49:0/2240809005 learned_addr learned my addr 172.21.15.49:0/2240809005 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:46.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.325+0000 7fb856b8b640 1 --2- 172.21.15.49:0/2240809005 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8500b3610 0x7fb85006a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:46.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.325+0000 7fb855b89640 1 --2- 172.21.15.49:0/2240809005 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb8500c5f30 0x7fb8500c4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:46.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.325+0000 7fb85638a640 1 -- 172.21.15.49:0/2240809005 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb8500c5f30 msgr2=0x7fb8500c4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:46.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.325+0000 7fb85638a640 1 --2- 172.21.15.49:0/2240809005 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb8500c5f30 0x7fb8500c4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:46.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.325+0000 7fb85638a640 1 -- 172.21.15.49:0/2240809005 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8500b3610 msgr2=0x7fb85006a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:46.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.325+0000 7fb85638a640 1 --2- 172.21.15.49:0/2240809005 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8500b3610 0x7fb85006a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:46.344 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.325+0000 7fb85638a640 1 -- 172.21.15.49:0/2240809005 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8500c5730 con 0x7fb83c005190 2023-11-06T02:11:46.344 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.325+0000 7fb85638a640 1 --2- 172.21.15.49:0/2240809005 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb83c005190 0x7fb850157b30 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fb84c0108d0 tx=0x7fb84c010d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:46.344 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.325+0000 7fb846ffd640 1 -- 172.21.15.49:0/2240809005 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb84c004220 con 0x7fb83c005190 2023-11-06T02:11:46.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.325+0000 7fb846ffd640 1 -- 172.21.15.49:0/2240809005 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb84c004380 con 0x7fb83c005190 2023-11-06T02:11:46.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.325+0000 7fb8585ee640 1 -- 172.21.15.49:0/2240809005 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8500c5a10 con 0x7fb83c005190 2023-11-06T02:11:46.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.325+0000 7fb846ffd640 1 -- 172.21.15.49:0/2240809005 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb84c00eba0 con 0x7fb83c005190 2023-11-06T02:11:46.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.325+0000 7fb8585ee640 1 -- 172.21.15.49:0/2240809005 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8500c4aa0 con 0x7fb83c005190 2023-11-06T02:11:46.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.329+0000 7fb8585ee640 1 -- 172.21.15.49:0/2240809005 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8500b5760 con 0x7fb83c005190 2023-11-06T02:11:46.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.329+0000 7fb846ffd640 1 -- 172.21.15.49:0/2240809005 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fb84c00ed00 con 0x7fb83c005190 2023-11-06T02:11:46.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.329+0000 7fb846ffd640 1 --2- 172.21.15.49:0/2240809005 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb834065a60 0x7fb834067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:46.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.329+0000 7fb846ffd640 1 -- 172.21.15.49:0/2240809005 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb84c0832b0 con 0x7fb83c005190 2023-11-06T02:11:46.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.329+0000 7fb855b89640 1 --2- 172.21.15.49:0/2240809005 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb834065a60 0x7fb834067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:46.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.329+0000 7fb855b89640 1 --2- 172.21.15.49:0/2240809005 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb834065a60 0x7fb834067f20 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb8500b4aa0 tx=0x7fb840009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:46.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.333+0000 7fb846ffd640 1 -- 172.21.15.49:0/2240809005 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fb84c050b20 con 0x7fb83c005190 2023-11-06T02:11:46.485 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.489+0000 7fb8585ee640 1 -- 172.21.15.49:0/2240809005 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb8500ca860 con 0x7fb83c005190 2023-11-06T02:11:46.486 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.489+0000 7fb846ffd640 1 -- 172.21.15.49:0/2240809005 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v65) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb84c054140 con 0x7fb83c005190 2023-11-06T02:11:46.486 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2023-11-06T02:11:46.488 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.489+0000 7fb8585ee640 1 -- 172.21.15.49:0/2240809005 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb834065a60 msgr2=0x7fb834067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:46.489 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.489+0000 7fb8585ee640 1 --2- 172.21.15.49:0/2240809005 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb834065a60 0x7fb834067f20 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb8500b4aa0 tx=0x7fb840009040 comp rx=0 tx=0).stop 2023-11-06T02:11:46.489 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.489+0000 7fb8585ee640 1 -- 172.21.15.49:0/2240809005 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb83c005190 msgr2=0x7fb850157b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:46.489 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.489+0000 7fb8585ee640 1 --2- 172.21.15.49:0/2240809005 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb83c005190 0x7fb850157b30 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fb84c0108d0 tx=0x7fb84c010d90 comp rx=0 tx=0).stop 2023-11-06T02:11:46.489 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.489+0000 7fb8585ee640 1 -- 172.21.15.49:0/2240809005 shutdown_connections 2023-11-06T02:11:46.489 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.489+0000 7fb8585ee640 1 --2- 172.21.15.49:0/2240809005 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb834065a60 0x7fb834067f20 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:46.490 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.489+0000 7fb8585ee640 1 --2- 172.21.15.49:0/2240809005 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8500b3610 0x7fb85006a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:46.490 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.489+0000 7fb8585ee640 1 --2- 172.21.15.49:0/2240809005 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb8500c5f30 0x7fb8500c4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:46.490 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.489+0000 7fb8585ee640 1 --2- 172.21.15.49:0/2240809005 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb83c005190 0x7fb850157b30 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:46.490 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.489+0000 7fb8585ee640 1 -- 172.21.15.49:0/2240809005 >> 172.21.15.49:0/2240809005 conn(0x7fb8500cf7e0 msgr2=0x7fb85015f300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:46.490 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.489+0000 7fb8585ee640 1 -- 172.21.15.49:0/2240809005 shutdown_connections 2023-11-06T02:11:46.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.489+0000 7fb8585ee640 1 -- 172.21.15.49:0/2240809005 wait complete. 2023-11-06T02:11:46.505 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T02:11:46.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.641+0000 7ffae062d640 1 Processor -- start 2023-11-06T02:11:46.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.641+0000 7ffae062d640 1 -- start start 2023-11-06T02:11:46.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.641+0000 7ffae062d640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffad815ad90 0x7ffad80cde40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:46.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.641+0000 7ffae062d640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ffad80cd2a0 0x7ffad80cd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:46.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.641+0000 7ffae062d640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffad80b3720 con 0x7ffad815a9b0 2023-11-06T02:11:46.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.641+0000 7ffae062d640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffad80cdc70 con 0x7ffad80cd2a0 2023-11-06T02:11:46.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.641+0000 7ffae062d640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffad80cf690 con 0x7ffad815ad90 2023-11-06T02:11:46.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.641+0000 7ffadebca640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ffad80cd2a0 0x7ffad80cd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:46.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.641+0000 7ffadebca640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ffad80cd2a0 0x7ffad80cd730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49616/0 (socket says 172.21.15.49:49616) 2023-11-06T02:11:46.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.641+0000 7ffadebca640 1 -- 172.21.15.49:0/2913336857 learned_addr learned my addr 172.21.15.49:0/2913336857 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:46.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.641+0000 7ffaddbc8640 1 --2- 172.21.15.49:0/2913336857 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffad815ad90 0x7ffad80cde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:46.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.641+0000 7ffadebca640 1 -- 172.21.15.49:0/2913336857 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffad815ad90 msgr2=0x7ffad80cde40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:46.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.641+0000 7ffadebca640 1 --2- 172.21.15.49:0/2913336857 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffad815ad90 0x7ffad80cde40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:46.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.641+0000 7ffadebca640 1 -- 172.21.15.49:0/2913336857 >> v1:172.21.15.49:6789/0 conn(0x7ffad815a9b0 legacy=0x7ffad80cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:46.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.641+0000 7ffadebca640 1 -- 172.21.15.49:0/2913336857 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffad80b3970 con 0x7ffad80cd2a0 2023-11-06T02:11:46.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.641+0000 7ffadd3c7640 1 -- 172.21.15.49:0/2913336857 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1345950996 0 0) 0x7ffad80b3720 con 0x7ffad815a9b0 2023-11-06T02:11:46.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.645+0000 7ffadebca640 1 --2- 172.21.15.49:0/2913336857 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffad80cd2a0 0x7ffad80cd730 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7ffad0009e50 tx=0x7ffad001ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=771b5219e8eacbef server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:46.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.645+0000 7ffadd3c7640 1 -- 172.21.15.49:0/2913336857 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffad00a57d0 con 0x7ffad80cd2a0 2023-11-06T02:11:46.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.645+0000 7ffadd3c7640 1 -- 172.21.15.49:0/2913336857 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffad00a5930 con 0x7ffad80cd2a0 2023-11-06T02:11:46.650 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.645+0000 7ffadd3c7640 1 -- 172.21.15.49:0/2913336857 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffad00a5cc0 con 0x7ffad80cd2a0 2023-11-06T02:11:46.650 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.645+0000 7ffae062d640 1 -- 172.21.15.49:0/2913336857 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffad80cd2a0 msgr2=0x7ffad80cd730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:46.650 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.645+0000 7ffae062d640 1 --2- 172.21.15.49:0/2913336857 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffad80cd2a0 0x7ffad80cd730 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7ffad0009e50 tx=0x7ffad001ca10 comp rx=0 tx=0).stop 2023-11-06T02:11:46.650 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.645+0000 7ffae062d640 1 -- 172.21.15.49:0/2913336857 shutdown_connections 2023-11-06T02:11:46.650 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.645+0000 7ffae062d640 1 --2- 172.21.15.49:0/2913336857 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffad80cd2a0 0x7ffad80cd730 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:46.651 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.645+0000 7ffae062d640 1 --2- 172.21.15.49:0/2913336857 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffad815ad90 0x7ffad80cde40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:46.651 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.645+0000 7ffae062d640 1 -- 172.21.15.49:0/2913336857 >> 172.21.15.49:0/2913336857 conn(0x7ffad80d08d0 msgr2=0x7ffad80d0cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:46.651 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.645+0000 7ffae062d640 1 -- 172.21.15.49:0/2913336857 shutdown_connections 2023-11-06T02:11:46.651 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.645+0000 7ffae062d640 1 -- 172.21.15.49:0/2913336857 wait complete. 2023-11-06T02:11:46.651 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.645+0000 7ffae062d640 1 Processor -- start 2023-11-06T02:11:46.651 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.645+0000 7ffae062d640 1 -- start start 2023-11-06T02:11:46.652 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.645+0000 7ffae062d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffad80cd2a0 0x7ffad81540f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:46.652 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.645+0000 7ffae062d640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffad815a9b0 0x7ffad806bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:46.652 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.645+0000 7ffae062d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffad815ad90 0x7ffad80c4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:46.652 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.645+0000 7ffae062d640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad806c820 con 0x7ffad80cd2a0 2023-11-06T02:11:46.652 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.645+0000 7ffae062d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad806cb00 con 0x7ffad815ad90 2023-11-06T02:11:46.653 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.645+0000 7ffae062d640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffad80c4460 con 0x7ffad815a9b0 2023-11-06T02:11:46.653 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.645+0000 7ffade3c9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffad80cd2a0 0x7ffad81540f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:46.653 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.645+0000 7ffade3c9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffad80cd2a0 0x7ffad81540f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49630/0 (socket says 172.21.15.49:49630) 2023-11-06T02:11:46.653 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.645+0000 7ffade3c9640 1 -- 172.21.15.49:0/4057471535 learned_addr learned my addr 172.21.15.49:0/4057471535 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:46.653 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.645+0000 7ffadebca640 1 --2- 172.21.15.49:0/4057471535 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffad815ad90 0x7ffad80c4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:46.654 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.645+0000 7ffade3c9640 1 -- 172.21.15.49:0/4057471535 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffad815a9b0 msgr2=0x7ffad806bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:46.654 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.645+0000 7ffade3c9640 1 --2- 172.21.15.49:0/4057471535 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffad815a9b0 0x7ffad806bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:46.654 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.645+0000 7ffade3c9640 1 -- 172.21.15.49:0/4057471535 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffad815ad90 msgr2=0x7ffad80c4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:46.654 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.645+0000 7ffade3c9640 1 --2- 172.21.15.49:0/4057471535 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffad815ad90 0x7ffad80c4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:46.654 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.645+0000 7ffade3c9640 1 -- 172.21.15.49:0/4057471535 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffad0009b30 con 0x7ffad80cd2a0 2023-11-06T02:11:46.655 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.645+0000 7ffadebca640 1 --2- 172.21.15.49:0/4057471535 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffad815ad90 0x7ffad80c4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:11:46.655 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.645+0000 7ffade3c9640 1 --2- 172.21.15.49:0/4057471535 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffad80cd2a0 0x7ffad81540f0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7ffad400c550 tx=0x7ffad400ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:46.655 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.645+0000 7ffaceffd640 1 -- 172.21.15.49:0/4057471535 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffad4005480 con 0x7ffad80cd2a0 2023-11-06T02:11:46.655 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.645+0000 7ffaceffd640 1 -- 172.21.15.49:0/4057471535 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffad4003600 con 0x7ffad80cd2a0 2023-11-06T02:11:46.655 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.645+0000 7ffae062d640 1 -- 172.21.15.49:0/4057471535 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffad80c4740 con 0x7ffad80cd2a0 2023-11-06T02:11:46.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.645+0000 7ffaceffd640 1 -- 172.21.15.49:0/4057471535 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffad40039b0 con 0x7ffad80cd2a0 2023-11-06T02:11:46.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.645+0000 7ffae062d640 1 -- 172.21.15.49:0/4057471535 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffad806c480 con 0x7ffad80cd2a0 2023-11-06T02:11:46.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.649+0000 7ffae062d640 1 -- 172.21.15.49:0/4057471535 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffad80c42c0 con 0x7ffad80cd2a0 2023-11-06T02:11:46.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.649+0000 7ffaceffd640 1 -- 172.21.15.49:0/4057471535 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7ffad4003b10 con 0x7ffad80cd2a0 2023-11-06T02:11:46.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.649+0000 7ffaceffd640 1 --2- 172.21.15.49:0/4057471535 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ffabc065a60 0x7ffabc067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:46.657 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.649+0000 7ffaceffd640 1 -- 172.21.15.49:0/4057471535 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ffad4082880 con 0x7ffad80cd2a0 2023-11-06T02:11:46.657 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.649+0000 7ffaddbc8640 1 --2- 172.21.15.49:0/4057471535 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ffabc065a60 0x7ffabc067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:46.657 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.649+0000 7ffaddbc8640 1 --2- 172.21.15.49:0/4057471535 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ffabc065a60 0x7ffabc067f20 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ffac8006aa0 tx=0x7ffac8006a30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:46.658 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.653+0000 7ffaceffd640 1 -- 172.21.15.49:0/4057471535 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7ffad80c42c0 con 0x7ffad80cd2a0 2023-11-06T02:11:46.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.809+0000 7ffae062d640 1 -- 172.21.15.49:0/4057471535 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ffad80c42c0 con 0x7ffad80cd2a0 2023-11-06T02:11:46.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.813+0000 7ffaceffd640 1 -- 172.21.15.49:0/4057471535 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v65) v1 ==== 92+0+10 (secure 0 0 0) 0x7ffad80c42c0 con 0x7ffad80cd2a0 2023-11-06T02:11:46.810 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 8 2023-11-06T02:11:46.812 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.813+0000 7ffae062d640 1 -- 172.21.15.49:0/4057471535 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ffabc065a60 msgr2=0x7ffabc067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:46.812 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.813+0000 7ffae062d640 1 --2- 172.21.15.49:0/4057471535 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ffabc065a60 0x7ffabc067f20 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ffac8006aa0 tx=0x7ffac8006a30 comp rx=0 tx=0).stop 2023-11-06T02:11:46.812 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.813+0000 7ffae062d640 1 -- 172.21.15.49:0/4057471535 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffad80cd2a0 msgr2=0x7ffad81540f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:46.812 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.813+0000 7ffae062d640 1 --2- 172.21.15.49:0/4057471535 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffad80cd2a0 0x7ffad81540f0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7ffad400c550 tx=0x7ffad400ca10 comp rx=0 tx=0).stop 2023-11-06T02:11:46.813 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.813+0000 7ffae062d640 1 -- 172.21.15.49:0/4057471535 shutdown_connections 2023-11-06T02:11:46.813 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.813+0000 7ffae062d640 1 --2- 172.21.15.49:0/4057471535 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ffabc065a60 0x7ffabc067f20 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:46.813 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.813+0000 7ffae062d640 1 --2- 172.21.15.49:0/4057471535 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffad815ad90 0x7ffad80c4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:46.813 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.813+0000 7ffae062d640 1 --2- 172.21.15.49:0/4057471535 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffad815a9b0 0x7ffad806bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:46.813 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.813+0000 7ffae062d640 1 --2- 172.21.15.49:0/4057471535 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffad80cd2a0 0x7ffad81540f0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:46.813 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.813+0000 7ffae062d640 1 -- 172.21.15.49:0/4057471535 >> 172.21.15.49:0/4057471535 conn(0x7ffad80d08d0 msgr2=0x7ffad80d0cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:46.814 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.813+0000 7ffae062d640 1 -- 172.21.15.49:0/4057471535 shutdown_connections 2023-11-06T02:11:46.814 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.813+0000 7ffae062d640 1 -- 172.21.15.49:0/4057471535 wait complete. 2023-11-06T02:11:46.829 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T02:11:46.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.961+0000 7fb8906fc640 1 Processor -- start 2023-11-06T02:11:46.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.961+0000 7fb8906fc640 1 -- start start 2023-11-06T02:11:46.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.961+0000 7fb8906fc640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fb8881595a0 0x7fb88815dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:46.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.961+0000 7fb8906fc640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb88806d640 0x7fb8880b0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:46.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.961+0000 7fb8906fc640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb8880b0760 0x7fb8880c8e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:46.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.961+0000 7fb8906fc640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb88815e250 con 0x7fb8880b0760 2023-11-06T02:11:46.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.961+0000 7fb8906fc640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb88806c5a0 con 0x7fb8881595a0 2023-11-06T02:11:46.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.961+0000 7fb8906fc640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb88806c880 con 0x7fb88806d640 2023-11-06T02:11:46.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.961+0000 7fb88ec99640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb8880b0760 0x7fb8880c8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:46.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.961+0000 7fb88ec99640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb8880b0760 0x7fb8880c8e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49648/0 (socket says 172.21.15.49:49648) 2023-11-06T02:11:46.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.961+0000 7fb88ec99640 1 -- 172.21.15.49:0/2069422373 learned_addr learned my addr 172.21.15.49:0/2069422373 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:46.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.961+0000 7fb88e498640 1 --2- 172.21.15.49:0/2069422373 >> v2:172.21.15.150:3300/0 conn(0x7fb8881595a0 0x7fb88815dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:46.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.961+0000 7fb88dc97640 1 --2- 172.21.15.49:0/2069422373 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb88806d640 0x7fb8880b0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:46.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.961+0000 7fb88ec99640 1 -- 172.21.15.49:0/2069422373 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb88806d640 msgr2=0x7fb8880b0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:46.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.961+0000 7fb88ec99640 1 --2- 172.21.15.49:0/2069422373 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb88806d640 0x7fb8880b0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:46.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.961+0000 7fb88ec99640 1 -- 172.21.15.49:0/2069422373 >> v2:172.21.15.150:3300/0 conn(0x7fb8881595a0 msgr2=0x7fb88815dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:46.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.961+0000 7fb88ec99640 1 --2- 172.21.15.49:0/2069422373 >> v2:172.21.15.150:3300/0 conn(0x7fb8881595a0 0x7fb88815dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:46.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.961+0000 7fb88ec99640 1 -- 172.21.15.49:0/2069422373 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb88806cb60 con 0x7fb8880b0760 2023-11-06T02:11:46.966 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.965+0000 7fb88ec99640 1 --2- 172.21.15.49:0/2069422373 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb8880b0760 0x7fb8880c8e90 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fb888071420 tx=0x7fb88001c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed2c8eebdc07d9a3 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:46.966 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.965+0000 7fb88d496640 1 -- 172.21.15.49:0/2069422373 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8800a57e0 con 0x7fb8880b0760 2023-11-06T02:11:46.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.965+0000 7fb88d496640 1 -- 172.21.15.49:0/2069422373 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8800a5940 con 0x7fb8880b0760 2023-11-06T02:11:46.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.965+0000 7fb88d496640 1 -- 172.21.15.49:0/2069422373 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8800a5cd0 con 0x7fb8880b0760 2023-11-06T02:11:46.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.965+0000 7fb8906fc640 1 -- 172.21.15.49:0/2069422373 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb8880b0760 msgr2=0x7fb8880c8e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:46.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.965+0000 7fb8906fc640 1 --2- 172.21.15.49:0/2069422373 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb8880b0760 0x7fb8880c8e90 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fb888071420 tx=0x7fb88001c9a0 comp rx=0 tx=0).stop 2023-11-06T02:11:46.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.965+0000 7fb8906fc640 1 -- 172.21.15.49:0/2069422373 shutdown_connections 2023-11-06T02:11:46.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.965+0000 7fb8906fc640 1 --2- 172.21.15.49:0/2069422373 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb88806d640 0x7fb8880b0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:46.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.965+0000 7fb8906fc640 1 --2- 172.21.15.49:0/2069422373 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb8880b0760 0x7fb8880c8e90 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:46.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.965+0000 7fb8906fc640 1 --2- 172.21.15.49:0/2069422373 >> v2:172.21.15.150:3300/0 conn(0x7fb8881595a0 0x7fb88815dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:46.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.965+0000 7fb8906fc640 1 -- 172.21.15.49:0/2069422373 >> 172.21.15.49:0/2069422373 conn(0x7fb88806a370 msgr2=0x7fb8880ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:46.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.965+0000 7fb8906fc640 1 -- 172.21.15.49:0/2069422373 shutdown_connections 2023-11-06T02:11:46.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.965+0000 7fb8906fc640 1 -- 172.21.15.49:0/2069422373 wait complete. 2023-11-06T02:11:46.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.965+0000 7fb8906fc640 1 Processor -- start 2023-11-06T02:11:46.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.965+0000 7fb8906fc640 1 -- start start 2023-11-06T02:11:46.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.965+0000 7fb8906fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb88806d640 0x7fb888135fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:46.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.965+0000 7fb8906fc640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb8880b0760 0x7fb888148930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:46.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.965+0000 7fb8906fc640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8881595a0 0x7fb888148e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:46.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.965+0000 7fb8906fc640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb88815a5f0 con 0x7fb88806d640 2023-11-06T02:11:46.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.965+0000 7fb8906fc640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb888159a20 con 0x7fb8881595a0 2023-11-06T02:11:46.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.965+0000 7fb8906fc640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8881364e0 con 0x7fb8880b0760 2023-11-06T02:11:46.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.965+0000 7fb88e498640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb88806d640 0x7fb888135fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:46.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.965+0000 7fb88e498640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb88806d640 0x7fb888135fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49660/0 (socket says 172.21.15.49:49660) 2023-11-06T02:11:46.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.965+0000 7fb88e498640 1 -- 172.21.15.49:0/3476857588 learned_addr learned my addr 172.21.15.49:0/3476857588 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:46.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.969+0000 7fb88dc97640 1 --2- 172.21.15.49:0/3476857588 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb8880b0760 0x7fb888148930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:46.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.969+0000 7fb88ec99640 1 --2- 172.21.15.49:0/3476857588 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8881595a0 0x7fb888148e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:46.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.969+0000 7fb88e498640 1 -- 172.21.15.49:0/3476857588 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb8880b0760 msgr2=0x7fb888148930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:46.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.969+0000 7fb88e498640 1 --2- 172.21.15.49:0/3476857588 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb8880b0760 0x7fb888148930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:46.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.969+0000 7fb88e498640 1 -- 172.21.15.49:0/3476857588 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8881595a0 msgr2=0x7fb888148e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:46.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.969+0000 7fb88e498640 1 --2- 172.21.15.49:0/3476857588 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8881595a0 0x7fb888148e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:46.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.969+0000 7fb88e498640 1 -- 172.21.15.49:0/3476857588 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb880009b30 con 0x7fb88806d640 2023-11-06T02:11:46.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.969+0000 7fb88ec99640 1 --2- 172.21.15.49:0/3476857588 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8881595a0 0x7fb888148e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:11:46.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.969+0000 7fb88e498640 1 --2- 172.21.15.49:0/3476857588 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb88806d640 0x7fb888135fa0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fb88400e870 tx=0x7fb88400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:46.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.969+0000 7fb87effd640 1 -- 172.21.15.49:0/3476857588 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb884009510 con 0x7fb88806d640 2023-11-06T02:11:46.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.969+0000 7fb87effd640 1 -- 172.21.15.49:0/3476857588 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb88400eec0 con 0x7fb88806d640 2023-11-06T02:11:46.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.969+0000 7fb8906fc640 1 -- 172.21.15.49:0/3476857588 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb888159cc0 con 0x7fb88806d640 2023-11-06T02:11:46.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.969+0000 7fb87effd640 1 -- 172.21.15.49:0/3476857588 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb88400cb40 con 0x7fb88806d640 2023-11-06T02:11:46.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.969+0000 7fb8906fc640 1 -- 172.21.15.49:0/3476857588 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb88815a280 con 0x7fb88806d640 2023-11-06T02:11:46.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.969+0000 7fb8906fc640 1 -- 172.21.15.49:0/3476857588 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb88806c210 con 0x7fb88806d640 2023-11-06T02:11:46.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.969+0000 7fb87effd640 1 -- 172.21.15.49:0/3476857588 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fb88400cca0 con 0x7fb88806d640 2023-11-06T02:11:46.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.969+0000 7fb87effd640 1 --2- 172.21.15.49:0/3476857588 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb86c065a60 0x7fb86c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:46.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.969+0000 7fb87effd640 1 -- 172.21.15.49:0/3476857588 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb884083310 con 0x7fb88806d640 2023-11-06T02:11:46.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.969+0000 7fb88dc97640 1 --2- 172.21.15.49:0/3476857588 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb86c065a60 0x7fb86c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:46.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.969+0000 7fb88dc97640 1 --2- 172.21.15.49:0/3476857588 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb86c065a60 0x7fb86c067f20 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fb88815c6c0 tx=0x7fb878008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:46.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:46.973+0000 7fb87effd640 1 -- 172.21.15.49:0/3476857588 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fb88806c210 con 0x7fb88806d640 2023-11-06T02:11:47.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.129+0000 7fb8906fc640 1 -- 172.21.15.49:0/3476857588 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb88806c210 con 0x7fb88806d640 2023-11-06T02:11:47.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.129+0000 7fb87effd640 1 -- 172.21.15.49:0/3476857588 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v65) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb88806c210 con 0x7fb88806d640 2023-11-06T02:11:47.127 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2023-11-06T02:11:47.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.133+0000 7fb8906fc640 1 -- 172.21.15.49:0/3476857588 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb86c065a60 msgr2=0x7fb86c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:47.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.133+0000 7fb8906fc640 1 --2- 172.21.15.49:0/3476857588 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb86c065a60 0x7fb86c067f20 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fb88815c6c0 tx=0x7fb878008040 comp rx=0 tx=0).stop 2023-11-06T02:11:47.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.133+0000 7fb8906fc640 1 -- 172.21.15.49:0/3476857588 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb88806d640 msgr2=0x7fb888135fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:47.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.133+0000 7fb8906fc640 1 --2- 172.21.15.49:0/3476857588 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb88806d640 0x7fb888135fa0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fb88400e870 tx=0x7fb88400ec70 comp rx=0 tx=0).stop 2023-11-06T02:11:47.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.133+0000 7fb8906fc640 1 -- 172.21.15.49:0/3476857588 shutdown_connections 2023-11-06T02:11:47.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.133+0000 7fb8906fc640 1 --2- 172.21.15.49:0/3476857588 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb86c065a60 0x7fb86c067f20 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:47.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.133+0000 7fb8906fc640 1 --2- 172.21.15.49:0/3476857588 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8881595a0 0x7fb888148e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:47.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.133+0000 7fb8906fc640 1 --2- 172.21.15.49:0/3476857588 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb8880b0760 0x7fb888148930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:47.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.133+0000 7fb8906fc640 1 --2- 172.21.15.49:0/3476857588 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb88806d640 0x7fb888135fa0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:47.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.133+0000 7fb8906fc640 1 -- 172.21.15.49:0/3476857588 >> 172.21.15.49:0/3476857588 conn(0x7fb88806a370 msgr2=0x7fb8880b2bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:47.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.133+0000 7fb8906fc640 1 -- 172.21.15.49:0/3476857588 shutdown_connections 2023-11-06T02:11:47.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.133+0000 7fb8906fc640 1 -- 172.21.15.49:0/3476857588 wait complete. 2023-11-06T02:11:47.147 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T02:11:47.278 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.281+0000 7f386f6ae640 1 Processor -- start 2023-11-06T02:11:47.278 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.281+0000 7f386f6ae640 1 -- start start 2023-11-06T02:11:47.279 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.281+0000 7f386f6ae640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f386815a880 0x7f38680b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:47.280 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.281+0000 7f386f6ae640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f386806d860 0x7f38680705e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:47.280 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.281+0000 7f386f6ae640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38680ae360 con 0x7f3868070160 2023-11-06T02:11:47.280 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.281+0000 7f386f6ae640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3868070b20 con 0x7f386806d860 2023-11-06T02:11:47.280 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.281+0000 7f386f6ae640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38680ae640 con 0x7f386815a880 2023-11-06T02:11:47.280 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.281+0000 7f386dc4b640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f386806d860 0x7f38680705e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:47.281 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.281+0000 7f386dc4b640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f386806d860 0x7f38680705e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49670/0 (socket says 172.21.15.49:49670) 2023-11-06T02:11:47.281 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.281+0000 7f386dc4b640 1 -- 172.21.15.49:0/75617267 learned_addr learned my addr 172.21.15.49:0/75617267 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:47.281 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.281+0000 7f386d44a640 1 --2- 172.21.15.49:0/75617267 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f386815a880 0x7f38680b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:47.281 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.281+0000 7f386dc4b640 1 -- 172.21.15.49:0/75617267 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f386815a880 msgr2=0x7f38680b15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:47.281 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.281+0000 7f386dc4b640 1 --2- 172.21.15.49:0/75617267 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f386815a880 0x7f38680b15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:47.282 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.281+0000 7f386dc4b640 1 -- 172.21.15.49:0/75617267 >> v1:172.21.15.150:6789/0 conn(0x7f3868070160 legacy=0x7f386806d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:47.282 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.281+0000 7f386dc4b640 1 -- 172.21.15.49:0/75617267 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38680ae920 con 0x7f386806d860 2023-11-06T02:11:47.283 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.285+0000 7f386dc4b640 1 --2- 172.21.15.49:0/75617267 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f386806d860 0x7f38680705e0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f3860009e50 tx=0x7f386001ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0ffa5dd7bc90d46 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:47.283 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.285+0000 7f385ffff640 1 -- 172.21.15.49:0/75617267 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38600a57f0 con 0x7f386806d860 2023-11-06T02:11:47.283 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.285+0000 7f385ffff640 1 -- 172.21.15.49:0/75617267 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38600a5950 con 0x7f386806d860 2023-11-06T02:11:47.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.285+0000 7f385ffff640 1 -- 172.21.15.49:0/75617267 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38600a5ce0 con 0x7f386806d860 2023-11-06T02:11:47.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.285+0000 7f386f6ae640 1 -- 172.21.15.49:0/75617267 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f386806d860 msgr2=0x7f38680705e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:47.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.285+0000 7f386f6ae640 1 --2- 172.21.15.49:0/75617267 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f386806d860 0x7f38680705e0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f3860009e50 tx=0x7f386001ca10 comp rx=0 tx=0).stop 2023-11-06T02:11:47.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.285+0000 7f386f6ae640 1 -- 172.21.15.49:0/75617267 shutdown_connections 2023-11-06T02:11:47.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.285+0000 7f386f6ae640 1 --2- 172.21.15.49:0/75617267 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f386806d860 0x7f38680705e0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:47.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.285+0000 7f386f6ae640 1 --2- 172.21.15.49:0/75617267 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f386815a880 0x7f38680b15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:47.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.285+0000 7f386f6ae640 1 -- 172.21.15.49:0/75617267 >> 172.21.15.49:0/75617267 conn(0x7f386806a370 msgr2=0x7f38680ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:47.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.285+0000 7f386f6ae640 1 -- 172.21.15.49:0/75617267 shutdown_connections 2023-11-06T02:11:47.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.285+0000 7f386f6ae640 1 -- 172.21.15.49:0/75617267 wait complete. 2023-11-06T02:11:47.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.285+0000 7f386f6ae640 1 Processor -- start 2023-11-06T02:11:47.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.285+0000 7f386f6ae640 1 -- start start 2023-11-06T02:11:47.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.285+0000 7f386f6ae640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f386806d860 0x7f3868154e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:47.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.285+0000 7f386f6ae640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3868070160 0x7f386814b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:47.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.285+0000 7f386f6ae640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f386815a880 0x7f386814bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:47.289 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.285+0000 7f386f6ae640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f386814bb30 con 0x7f386815a880 2023-11-06T02:11:47.289 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.285+0000 7f386f6ae640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38680cbab0 con 0x7f386806d860 2023-11-06T02:11:47.289 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.285+0000 7f386f6ae640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f386814c5d0 con 0x7f3868070160 2023-11-06T02:11:47.289 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.285+0000 7f386dc4b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f386815a880 0x7f386814bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:47.289 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.285+0000 7f386dc4b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f386815a880 0x7f386814bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49686/0 (socket says 172.21.15.49:49686) 2023-11-06T02:11:47.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.285+0000 7f386dc4b640 1 -- 172.21.15.49:0/1753567170 learned_addr learned my addr 172.21.15.49:0/1753567170 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:47.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.289+0000 7f386d44a640 1 --2- 172.21.15.49:0/1753567170 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f386806d860 0x7f3868154e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:47.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.289+0000 7f386cc49640 1 --2- 172.21.15.49:0/1753567170 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3868070160 0x7f386814b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:47.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.289+0000 7f386dc4b640 1 -- 172.21.15.49:0/1753567170 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3868070160 msgr2=0x7f386814b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:47.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.289+0000 7f386dc4b640 1 --2- 172.21.15.49:0/1753567170 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3868070160 0x7f386814b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:47.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.289+0000 7f386dc4b640 1 -- 172.21.15.49:0/1753567170 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f386806d860 msgr2=0x7f3868154e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:47.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.289+0000 7f386dc4b640 1 --2- 172.21.15.49:0/1753567170 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f386806d860 0x7f3868154e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:47.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.289+0000 7f386dc4b640 1 -- 172.21.15.49:0/1753567170 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3860009b30 con 0x7f386815a880 2023-11-06T02:11:47.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.289+0000 7f386dc4b640 1 --2- 172.21.15.49:0/1753567170 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f386815a880 0x7f386814bcd0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f38600af040 tx=0x7f38600197b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:47.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.289+0000 7f385dffb640 1 -- 172.21.15.49:0/1753567170 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38600b8670 con 0x7f386815a880 2023-11-06T02:11:47.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.289+0000 7f385dffb640 1 -- 172.21.15.49:0/1753567170 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38600b87f0 con 0x7f386815a880 2023-11-06T02:11:47.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.289+0000 7f386f6ae640 1 -- 172.21.15.49:0/1753567170 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f386814c8b0 con 0x7f386815a880 2023-11-06T02:11:47.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.289+0000 7f385dffb640 1 -- 172.21.15.49:0/1753567170 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38600b8ba0 con 0x7f386815a880 2023-11-06T02:11:47.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.289+0000 7f386f6ae640 1 -- 172.21.15.49:0/1753567170 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f386814ce70 con 0x7f386815a880 2023-11-06T02:11:47.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.289+0000 7f386f6ae640 1 -- 172.21.15.49:0/1753567170 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38680b3df0 con 0x7f386815a880 2023-11-06T02:11:47.293 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.289+0000 7f385dffb640 1 -- 172.21.15.49:0/1753567170 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f38600b8d00 con 0x7f386815a880 2023-11-06T02:11:47.293 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.289+0000 7f385dffb640 1 --2- 172.21.15.49:0/1753567170 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f384c065a90 0x7f384c067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:47.293 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.289+0000 7f385dffb640 1 -- 172.21.15.49:0/1753567170 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f386011e760 con 0x7f386815a880 2023-11-06T02:11:47.293 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.289+0000 7f386d44a640 1 --2- 172.21.15.49:0/1753567170 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f384c065a90 0x7f384c067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:47.294 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.289+0000 7f386d44a640 1 --2- 172.21.15.49:0/1753567170 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f384c065a90 0x7f384c067f50 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f386400b930 tx=0x7f3864008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:47.294 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.293+0000 7f385dffb640 1 -- 172.21.15.49:0/1753567170 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f38600ec070 con 0x7f386815a880 2023-11-06T02:11:47.445 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.449+0000 7f386f6ae640 1 -- 172.21.15.49:0/1753567170 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3868074040 con 0x7f386815a880 2023-11-06T02:11:47.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.449+0000 7f385dffb640 1 -- 172.21.15.49:0/1753567170 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v65) v1 ==== 100+0+11 (secure 0 0 0) 0x7f38600ef690 con 0x7f386815a880 2023-11-06T02:11:47.447 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2023-11-06T02:11:47.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.449+0000 7f386f6ae640 1 -- 172.21.15.49:0/1753567170 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f384c065a90 msgr2=0x7f384c067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:47.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.449+0000 7f386f6ae640 1 --2- 172.21.15.49:0/1753567170 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f384c065a90 0x7f384c067f50 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f386400b930 tx=0x7f3864008040 comp rx=0 tx=0).stop 2023-11-06T02:11:47.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.449+0000 7f386f6ae640 1 -- 172.21.15.49:0/1753567170 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f386815a880 msgr2=0x7f386814bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:47.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.449+0000 7f386f6ae640 1 --2- 172.21.15.49:0/1753567170 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f386815a880 0x7f386814bcd0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f38600af040 tx=0x7f38600197b0 comp rx=0 tx=0).stop 2023-11-06T02:11:47.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.449+0000 7f386f6ae640 1 -- 172.21.15.49:0/1753567170 shutdown_connections 2023-11-06T02:11:47.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.449+0000 7f386f6ae640 1 --2- 172.21.15.49:0/1753567170 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f384c065a90 0x7f384c067f50 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:47.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.449+0000 7f386f6ae640 1 --2- 172.21.15.49:0/1753567170 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f386815a880 0x7f386814bcd0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:47.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.449+0000 7f386f6ae640 1 --2- 172.21.15.49:0/1753567170 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3868070160 0x7f386814b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:47.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.449+0000 7f386f6ae640 1 --2- 172.21.15.49:0/1753567170 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f386806d860 0x7f3868154e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:47.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.449+0000 7f386f6ae640 1 -- 172.21.15.49:0/1753567170 >> 172.21.15.49:0/1753567170 conn(0x7f386806a370 msgr2=0x7f38680b31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:47.451 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.449+0000 7f386f6ae640 1 -- 172.21.15.49:0/1753567170 shutdown_connections 2023-11-06T02:11:47.451 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.449+0000 7f386f6ae640 1 -- 172.21.15.49:0/1753567170 wait complete. 2023-11-06T02:11:47.465 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-06T02:11:47.580 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.581+0000 7fee75454640 1 Processor -- start 2023-11-06T02:11:47.581 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.581+0000 7fee75454640 1 -- start start 2023-11-06T02:11:47.582 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.585+0000 7fee75454640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fee70159980 0x7fee70071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:47.582 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.585+0000 7fee75454640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fee700b1450 0x7fee700cdd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:47.582 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.585+0000 7fee75454640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fee700cb010 0x7fee700cb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:47.582 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.585+0000 7fee75454640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee70072410 con 0x7fee700cb010 2023-11-06T02:11:47.583 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.585+0000 7fee75454640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee700b1890 con 0x7fee70159980 2023-11-06T02:11:47.583 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.585+0000 7fee75454640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee700cb9e0 con 0x7fee700b1450 2023-11-06T02:11:47.583 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.585+0000 7fee6f7fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fee700cb010 0x7fee700cb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:47.583 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.585+0000 7fee6f7fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fee700cb010 0x7fee700cb4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49690/0 (socket says 172.21.15.49:49690) 2023-11-06T02:11:47.583 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.585+0000 7fee6f7fe640 1 -- 172.21.15.49:0/2635553221 learned_addr learned my addr 172.21.15.49:0/2635553221 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:47.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.585+0000 7fee6effd640 1 --2- 172.21.15.49:0/2635553221 >> v2:172.21.15.150:3300/0 conn(0x7fee70159980 0x7fee70071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:47.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.585+0000 7fee6e7fc640 1 --2- 172.21.15.49:0/2635553221 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fee700b1450 0x7fee700cdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:47.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.585+0000 7fee6f7fe640 1 -- 172.21.15.49:0/2635553221 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fee700b1450 msgr2=0x7fee700cdd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:47.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.585+0000 7fee6f7fe640 1 --2- 172.21.15.49:0/2635553221 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fee700b1450 0x7fee700cdd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:47.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.585+0000 7fee6f7fe640 1 -- 172.21.15.49:0/2635553221 >> v2:172.21.15.150:3300/0 conn(0x7fee70159980 msgr2=0x7fee70071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:47.585 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.585+0000 7fee6f7fe640 1 --2- 172.21.15.49:0/2635553221 >> v2:172.21.15.150:3300/0 conn(0x7fee70159980 0x7fee70071290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:47.585 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.585+0000 7fee6f7fe640 1 -- 172.21.15.49:0/2635553221 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee70071890 con 0x7fee700cb010 2023-11-06T02:11:47.588 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.585+0000 7fee6f7fe640 1 --2- 172.21.15.49:0/2635553221 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee700cb010 0x7fee700cb4a0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fee700cc140 tx=0x7fee6801c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8be195f2def30c45 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:47.588 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.585+0000 7fee6dffb640 1 -- 172.21.15.49:0/2635553221 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee680a5850 con 0x7fee700cb010 2023-11-06T02:11:47.588 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.585+0000 7fee6dffb640 1 -- 172.21.15.49:0/2635553221 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee680a59b0 con 0x7fee700cb010 2023-11-06T02:11:47.588 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.585+0000 7fee6dffb640 1 -- 172.21.15.49:0/2635553221 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee680a5d40 con 0x7fee700cb010 2023-11-06T02:11:47.588 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.589+0000 7fee75454640 1 -- 172.21.15.49:0/2635553221 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee700cb010 msgr2=0x7fee700cb4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:47.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.589+0000 7fee75454640 1 --2- 172.21.15.49:0/2635553221 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee700cb010 0x7fee700cb4a0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fee700cc140 tx=0x7fee6801c9d0 comp rx=0 tx=0).stop 2023-11-06T02:11:47.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.589+0000 7fee75454640 1 -- 172.21.15.49:0/2635553221 shutdown_connections 2023-11-06T02:11:47.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.589+0000 7fee75454640 1 --2- 172.21.15.49:0/2635553221 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fee700b1450 0x7fee700cdd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:47.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.589+0000 7fee75454640 1 --2- 172.21.15.49:0/2635553221 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee700cb010 0x7fee700cb4a0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:47.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.589+0000 7fee75454640 1 --2- 172.21.15.49:0/2635553221 >> v2:172.21.15.150:3300/0 conn(0x7fee70159980 0x7fee70071290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:47.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.589+0000 7fee75454640 1 -- 172.21.15.49:0/2635553221 >> 172.21.15.49:0/2635553221 conn(0x7fee7006c0a0 msgr2=0x7fee7006c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:47.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.589+0000 7fee75454640 1 -- 172.21.15.49:0/2635553221 shutdown_connections 2023-11-06T02:11:47.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.589+0000 7fee75454640 1 -- 172.21.15.49:0/2635553221 wait complete. 2023-11-06T02:11:47.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.589+0000 7fee75454640 1 Processor -- start 2023-11-06T02:11:47.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.589+0000 7fee75454640 1 -- start start 2023-11-06T02:11:47.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.589+0000 7fee75454640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee700b1450 0x7fee70154a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:47.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.589+0000 7fee75454640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee700cb010 0x7fee70143a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:47.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.589+0000 7fee75454640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fee70159980 0x7fee7015a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:47.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.589+0000 7fee75454640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee70154fd0 con 0x7fee700cb010 2023-11-06T02:11:47.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.589+0000 7fee75454640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee70155a60 con 0x7fee700b1450 2023-11-06T02:11:47.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.589+0000 7fee75454640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee700b4090 con 0x7fee70159980 2023-11-06T02:11:47.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.589+0000 7fee6e7fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee700cb010 0x7fee70143a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:47.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.589+0000 7fee6e7fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee700cb010 0x7fee70143a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49692/0 (socket says 172.21.15.49:49692) 2023-11-06T02:11:47.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.589+0000 7fee6e7fc640 1 -- 172.21.15.49:0/2442522359 learned_addr learned my addr 172.21.15.49:0/2442522359 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:47.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.589+0000 7fee6f7fe640 1 --2- 172.21.15.49:0/2442522359 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fee70159980 0x7fee7015a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:47.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.589+0000 7fee6effd640 1 --2- 172.21.15.49:0/2442522359 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee700b1450 0x7fee70154a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:47.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.589+0000 7fee6e7fc640 1 -- 172.21.15.49:0/2442522359 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fee70159980 msgr2=0x7fee7015a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:47.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.589+0000 7fee6e7fc640 1 --2- 172.21.15.49:0/2442522359 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fee70159980 0x7fee7015a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:47.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.589+0000 7fee6e7fc640 1 -- 172.21.15.49:0/2442522359 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee700b1450 msgr2=0x7fee70154a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:47.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.589+0000 7fee6e7fc640 1 --2- 172.21.15.49:0/2442522359 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee700b1450 0x7fee70154a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:47.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.589+0000 7fee6e7fc640 1 -- 172.21.15.49:0/2442522359 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee68009b30 con 0x7fee700cb010 2023-11-06T02:11:47.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.589+0000 7fee6e7fc640 1 --2- 172.21.15.49:0/2442522359 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee700cb010 0x7fee70143a70 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fee6400d830 tx=0x7fee6400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:47.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.589+0000 7fee577fe640 1 -- 172.21.15.49:0/2442522359 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee6400be10 con 0x7fee700cb010 2023-11-06T02:11:47.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.589+0000 7fee577fe640 1 -- 172.21.15.49:0/2442522359 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee640098b0 con 0x7fee700cb010 2023-11-06T02:11:47.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.589+0000 7fee75454640 1 -- 172.21.15.49:0/2442522359 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee700ce3f0 con 0x7fee700cb010 2023-11-06T02:11:47.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.589+0000 7fee577fe640 1 -- 172.21.15.49:0/2442522359 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee64009a40 con 0x7fee700cb010 2023-11-06T02:11:47.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.589+0000 7fee75454640 1 -- 172.21.15.49:0/2442522359 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee700b20d0 con 0x7fee700cb010 2023-11-06T02:11:47.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.589+0000 7fee75454640 1 -- 172.21.15.49:0/2442522359 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee70074040 con 0x7fee700cb010 2023-11-06T02:11:47.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.593+0000 7fee577fe640 1 -- 172.21.15.49:0/2442522359 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fee64009ba0 con 0x7fee700cb010 2023-11-06T02:11:47.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.593+0000 7fee577fe640 1 --2- 172.21.15.49:0/2442522359 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fee5c065a60 0x7fee5c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:47.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.593+0000 7fee577fe640 1 -- 172.21.15.49:0/2442522359 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fee64083670 con 0x7fee700cb010 2023-11-06T02:11:47.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.593+0000 7fee6effd640 1 --2- 172.21.15.49:0/2442522359 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fee5c065a60 0x7fee5c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:47.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.593+0000 7fee6effd640 1 --2- 172.21.15.49:0/2442522359 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fee5c065a60 0x7fee5c067f20 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fee60006fd0 tx=0x7fee60008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:47.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.597+0000 7fee577fe640 1 -- 172.21.15.49:0/2442522359 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fee64050ec0 con 0x7fee700cb010 2023-11-06T02:11:47.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.761+0000 7fee75454640 1 -- 172.21.15.49:0/2442522359 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fee7006c690 con 0x7fee700cb010 2023-11-06T02:11:47.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.761+0000 7fee577fe640 1 -- 172.21.15.49:0/2442522359 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v65) v1 ==== 80+0+0 (secure 0 0 0) 0x7fee6400b3a0 con 0x7fee700cb010 2023-11-06T02:11:47.761 INFO:teuthology.orchestra.run.smithi049.stderr:listed 0 entries 2023-11-06T02:11:47.762 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.765+0000 7fee75454640 1 -- 172.21.15.49:0/2442522359 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fee5c065a60 msgr2=0x7fee5c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:47.763 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.765+0000 7fee75454640 1 --2- 172.21.15.49:0/2442522359 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fee5c065a60 0x7fee5c067f20 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fee60006fd0 tx=0x7fee60008040 comp rx=0 tx=0).stop 2023-11-06T02:11:47.763 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.765+0000 7fee75454640 1 -- 172.21.15.49:0/2442522359 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee700cb010 msgr2=0x7fee70143a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:47.763 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.765+0000 7fee75454640 1 --2- 172.21.15.49:0/2442522359 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee700cb010 0x7fee70143a70 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fee6400d830 tx=0x7fee6400dcf0 comp rx=0 tx=0).stop 2023-11-06T02:11:47.763 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.765+0000 7fee75454640 1 -- 172.21.15.49:0/2442522359 shutdown_connections 2023-11-06T02:11:47.764 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.765+0000 7fee75454640 1 --2- 172.21.15.49:0/2442522359 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fee5c065a60 0x7fee5c067f20 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:47.764 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.765+0000 7fee75454640 1 --2- 172.21.15.49:0/2442522359 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fee700cb010 0x7fee70143a70 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:47.764 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.765+0000 7fee75454640 1 --2- 172.21.15.49:0/2442522359 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fee70159980 0x7fee7015a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:47.764 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.765+0000 7fee75454640 1 --2- 172.21.15.49:0/2442522359 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee700b1450 0x7fee70154a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:47.764 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.765+0000 7fee75454640 1 -- 172.21.15.49:0/2442522359 >> 172.21.15.49:0/2442522359 conn(0x7fee7006c0a0 msgr2=0x7fee700acd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:47.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.765+0000 7fee75454640 1 -- 172.21.15.49:0/2442522359 shutdown_connections 2023-11-06T02:11:47.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:47.765+0000 7fee75454640 1 -- 172.21.15.49:0/2442522359 wait complete. 2023-11-06T02:11:47.778 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi049.front.sepia.ceph.com... 2023-11-06T02:11:47.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:11:47.779 DEBUG:teuthology.orchestra.run.smithi049:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-06T02:11:47.790 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi049.stderr:ceph-fuse[52005]: fuse finished with error 0 and tester_r 0 2023-11-06T02:11:47.792 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T02:11:51.859 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T02:11:51.862 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T02:11:53.894 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi049.front.sepia.ceph.com 2023-11-06T02:11:53.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:11:53.895 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-06T02:11:53.902 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T02:11:54.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.080+0000 7fc7ff18e640 1 Processor -- start 2023-11-06T02:11:54.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.080+0000 7fc7ff18e640 1 -- start start 2023-11-06T02:11:54.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.084+0000 7fc7ff18e640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc7f815ab50 0x7fc7f80c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:54.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.084+0000 7fc7ff18e640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7f80cab80 con 0x7fc7f80b0790 2023-11-06T02:11:54.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.084+0000 7fc7ff18e640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7f815af30 con 0x7fc7f80cb010 2023-11-06T02:11:54.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.084+0000 7fc7ff18e640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7f80cb5b0 con 0x7fc7f815ab50 2023-11-06T02:11:54.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.084+0000 7fc7fd72b640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fc7f80b0790 0x7fc7f80c2710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:55172/0 (socket says 172.21.15.49:55172) 2023-11-06T02:11:54.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.084+0000 7fc7fd72b640 1 -- 172.21.15.49:0/553852758 learned_addr learned my addr 172.21.15.49:0/553852758 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:11:54.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.084+0000 7fc7fcf2a640 1 --2- 172.21.15.49:0/553852758 >> v2:172.21.15.49:3300/0 conn(0x7fc7f815ab50 0x7fc7f80c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:54.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.084+0000 7fc7f77fe640 1 -- 172.21.15.49:0/553852758 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3730838183 0 0) 0x7fc7f80cab80 con 0x7fc7f80b0790 2023-11-06T02:11:54.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.084+0000 7fc7f77fe640 1 -- 172.21.15.49:0/553852758 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7e4003530 con 0x7fc7f80b0790 2023-11-06T02:11:54.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.084+0000 7fc7fcf2a640 1 -- 172.21.15.49:0/553852758 >> v1:172.21.15.150:6789/0 conn(0x7fc7f80cb010 legacy=0x7fc7f80b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:54.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.084+0000 7fc7fcf2a640 1 -- 172.21.15.49:0/553852758 >> v1:172.21.15.49:6789/0 conn(0x7fc7f80b0790 legacy=0x7fc7f80c2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:54.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.084+0000 7fc7fcf2a640 1 -- 172.21.15.49:0/553852758 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7f80cb890 con 0x7fc7f815ab50 2023-11-06T02:11:54.086 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.084+0000 7fc7fcf2a640 1 --2- 172.21.15.49:0/553852758 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7f815ab50 0x7fc7f80c9bb0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fc7e80097a0 tx=0x7fc7e801c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cce66d94a4b20d17 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:54.086 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.084+0000 7fc7f77fe640 1 -- 172.21.15.49:0/553852758 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e80a7070 con 0x7fc7f815ab50 2023-11-06T02:11:54.086 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.088+0000 7fc7f77fe640 1 -- 172.21.15.49:0/553852758 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7e80ab430 con 0x7fc7f815ab50 2023-11-06T02:11:54.086 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.088+0000 7fc7f77fe640 1 -- 172.21.15.49:0/553852758 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e80ab7a0 con 0x7fc7f815ab50 2023-11-06T02:11:54.087 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.088+0000 7fc7ff18e640 1 -- 172.21.15.49:0/553852758 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7f815ab50 msgr2=0x7fc7f80c9bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:54.087 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.088+0000 7fc7ff18e640 1 --2- 172.21.15.49:0/553852758 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7f815ab50 0x7fc7f80c9bb0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fc7e80097a0 tx=0x7fc7e801c430 comp rx=0 tx=0).stop 2023-11-06T02:11:54.087 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.088+0000 7fc7ff18e640 1 -- 172.21.15.49:0/553852758 shutdown_connections 2023-11-06T02:11:54.087 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.088+0000 7fc7ff18e640 1 --2- 172.21.15.49:0/553852758 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7f815ab50 0x7fc7f80c9bb0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:54.087 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.088+0000 7fc7ff18e640 1 -- 172.21.15.49:0/553852758 >> 172.21.15.49:0/553852758 conn(0x7fc7f806c0a0 msgr2=0x7fc7f806c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:54.088 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.088+0000 7fc7ff18e640 1 -- 172.21.15.49:0/553852758 shutdown_connections 2023-11-06T02:11:54.088 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.088+0000 7fc7ff18e640 1 -- 172.21.15.49:0/553852758 wait complete. 2023-11-06T02:11:54.091 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.088+0000 7fc7ff18e640 1 Processor -- start 2023-11-06T02:11:54.091 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.088+0000 7fc7ff18e640 1 -- start start 2023-11-06T02:11:54.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.088+0000 7fc7ff18e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7f80b0790 0x7fc7f81513b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:54.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.088+0000 7fc7ff18e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc7f80cb010 0x7fc7f814d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:54.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.088+0000 7fc7ff18e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc7f815ab50 0x7fc7f8155b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:54.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.088+0000 7fc7ff18e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f814af00 con 0x7fc7f80b0790 2023-11-06T02:11:54.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.088+0000 7fc7ff18e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f814dca0 con 0x7fc7f815ab50 2023-11-06T02:11:54.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.088+0000 7fc7ff18e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f80afdb0 con 0x7fc7f80cb010 2023-11-06T02:11:54.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.088+0000 7fc7fcf2a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7f80b0790 0x7fc7f81513b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:54.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.088+0000 7fc7fcf2a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7f80b0790 0x7fc7f81513b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59744/0 (socket says 172.21.15.49:59744) 2023-11-06T02:11:54.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.088+0000 7fc7fcf2a640 1 -- 172.21.15.49:0/1936155041 learned_addr learned my addr 172.21.15.49:0/1936155041 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:54.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.088+0000 7fc7fcf2a640 1 -- 172.21.15.49:0/1936155041 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc7f80cb010 msgr2=0x7fc7f814d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:54.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.088+0000 7fc7fd72b640 1 --2- 172.21.15.49:0/1936155041 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc7f815ab50 0x7fc7f8155b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:54.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.088+0000 7fc7f7fff640 1 --2- 172.21.15.49:0/1936155041 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc7f80cb010 0x7fc7f814d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:54.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.088+0000 7fc7fcf2a640 1 --2- 172.21.15.49:0/1936155041 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc7f80cb010 0x7fc7f814d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:54.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.088+0000 7fc7fcf2a640 1 -- 172.21.15.49:0/1936155041 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc7f815ab50 msgr2=0x7fc7f8155b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:54.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.088+0000 7fc7fcf2a640 1 --2- 172.21.15.49:0/1936155041 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc7f815ab50 0x7fc7f8155b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:54.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.088+0000 7fc7fcf2a640 1 -- 172.21.15.49:0/1936155041 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7e80093b0 con 0x7fc7f80b0790 2023-11-06T02:11:54.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.088+0000 7fc7fd72b640 1 --2- 172.21.15.49:0/1936155041 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc7f815ab50 0x7fc7f8155b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:11:54.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.088+0000 7fc7fcf2a640 1 --2- 172.21.15.49:0/1936155041 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7f80b0790 0x7fc7f81513b0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fc7e801a060 tx=0x7fc7e80191d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:54.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.088+0000 7fc7f57fa640 1 -- 172.21.15.49:0/1936155041 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e8005ba0 con 0x7fc7f80b0790 2023-11-06T02:11:54.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.088+0000 7fc7f57fa640 1 -- 172.21.15.49:0/1936155041 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7e8005d00 con 0x7fc7f80b0790 2023-11-06T02:11:54.096 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.088+0000 7fc7ff18e640 1 -- 172.21.15.49:0/1936155041 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7f8150610 con 0x7fc7f80b0790 2023-11-06T02:11:54.096 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.088+0000 7fc7f57fa640 1 -- 172.21.15.49:0/1936155041 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e8003840 con 0x7fc7f80b0790 2023-11-06T02:11:54.096 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.088+0000 7fc7ff18e640 1 -- 172.21.15.49:0/1936155041 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7f8150bd0 con 0x7fc7f80b0790 2023-11-06T02:11:54.096 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.092+0000 7fc7ff18e640 1 -- 172.21.15.49:0/1936155041 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7f80b0de0 con 0x7fc7f80b0790 2023-11-06T02:11:54.096 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.092+0000 7fc7f57fa640 1 -- 172.21.15.49:0/1936155041 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fc7e80039a0 con 0x7fc7f80b0790 2023-11-06T02:11:54.097 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.092+0000 7fc7f57fa640 1 --2- 172.21.15.49:0/1936155041 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc7d8065a60 0x7fc7d8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:54.097 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.092+0000 7fc7f57fa640 1 -- 172.21.15.49:0/1936155041 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc7e811d1b0 con 0x7fc7f80b0790 2023-11-06T02:11:54.097 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.092+0000 7fc7f7fff640 1 --2- 172.21.15.49:0/1936155041 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc7d8065a60 0x7fc7d8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:54.097 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.092+0000 7fc7f7fff640 1 --2- 172.21.15.49:0/1936155041 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc7d8065a60 0x7fc7d8067f20 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fc7e0007fd0 tx=0x7fc7e0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:54.098 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.096+0000 7fc7f57fa640 1 -- 172.21.15.49:0/1936155041 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fc7e80ea970 con 0x7fc7f80b0790 2023-11-06T02:11:54.286 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.288+0000 7fc7ff18e640 1 -- 172.21.15.49:0/1936155041 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc7f8074040 con 0x7fc7f80b0790 2023-11-06T02:11:54.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.288+0000 7fc7f57fa640 1 -- 172.21.15.49:0/1936155041 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+4652 (secure 0 0 0) 0x7fc7e80edf90 con 0x7fc7f80b0790 2023-11-06T02:11:54.288 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:11:54.288 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":30,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5635,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4170291905","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":4170291905},{"type":"v1","addr":"172.21.15.150:6835","nonce":4170291905}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5638,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/2719458937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":2719458937},{"type":"v1","addr":"172.21.15.150:6837","nonce":2719458937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":6339,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/2310338375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2310338375},{"type":"v1","addr":"172.21.15.49:6833","nonce":2310338375}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T02:11:30.091070+0000","modified":"2023-11-06T02:11:33.173821+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6348},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6348":{"gid":6348,"name":"c","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.49:6835/342908487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":342908487},{"type":"v1","addr":"172.21.15.49:6835","nonce":342908487}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-06T02:11:54.289 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 30 2023-11-06T02:11:54.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.292+0000 7fc7ff18e640 1 -- 172.21.15.49:0/1936155041 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc7d8065a60 msgr2=0x7fc7d8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:54.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.292+0000 7fc7ff18e640 1 --2- 172.21.15.49:0/1936155041 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc7d8065a60 0x7fc7d8067f20 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fc7e0007fd0 tx=0x7fc7e0009040 comp rx=0 tx=0).stop 2023-11-06T02:11:54.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.292+0000 7fc7ff18e640 1 -- 172.21.15.49:0/1936155041 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7f80b0790 msgr2=0x7fc7f81513b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:54.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.292+0000 7fc7ff18e640 1 --2- 172.21.15.49:0/1936155041 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7f80b0790 0x7fc7f81513b0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fc7e801a060 tx=0x7fc7e80191d0 comp rx=0 tx=0).stop 2023-11-06T02:11:54.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.292+0000 7fc7ff18e640 1 -- 172.21.15.49:0/1936155041 shutdown_connections 2023-11-06T02:11:54.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.292+0000 7fc7ff18e640 1 --2- 172.21.15.49:0/1936155041 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc7d8065a60 0x7fc7d8067f20 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:54.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.292+0000 7fc7ff18e640 1 --2- 172.21.15.49:0/1936155041 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc7f815ab50 0x7fc7f8155b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:54.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.292+0000 7fc7ff18e640 1 --2- 172.21.15.49:0/1936155041 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc7f80cb010 0x7fc7f814d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:54.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.292+0000 7fc7ff18e640 1 --2- 172.21.15.49:0/1936155041 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7f80b0790 0x7fc7f81513b0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:54.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.292+0000 7fc7ff18e640 1 -- 172.21.15.49:0/1936155041 >> 172.21.15.49:0/1936155041 conn(0x7fc7f806c0a0 msgr2=0x7fc7f80ce570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:54.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.292+0000 7fc7ff18e640 1 -- 172.21.15.49:0/1936155041 shutdown_connections 2023-11-06T02:11:54.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.292+0000 7fc7ff18e640 1 -- 172.21.15.49:0/1936155041 wait complete. 2023-11-06T02:11:54.306 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2023-11-06T02:11:54.455 INFO:teuthology.orchestra.run.smithi049.stdout:{"message":"","return_code":0} 2023-11-06T02:11:54.456 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2023-11-06T02:11:54.456 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T02:11:54.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.576+0000 7f2277eff640 1 Processor -- start 2023-11-06T02:11:54.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.576+0000 7f2277eff640 1 -- start start 2023-11-06T02:11:54.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.580+0000 7f2277eff640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f22700b0790 0x7f22700c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:54.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.580+0000 7f2277eff640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f22700cb010 0x7f22700c2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:54.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.580+0000 7f2277eff640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22700cab80 con 0x7f227015ab50 2023-11-06T02:11:54.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.580+0000 7f2277eff640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f227015af30 con 0x7f22700b0790 2023-11-06T02:11:54.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.580+0000 7f2277eff640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22700cb5b0 con 0x7f22700cb010 2023-11-06T02:11:54.579 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.580+0000 7f227649c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f22700cb010 0x7f22700c2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:54.579 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.580+0000 7f227549a640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f22700b0790 0x7f22700c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:54.580 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.580+0000 7f227649c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f22700cb010 0x7f22700c2710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.49:51098/0 (socket says 172.21.15.49:51098) 2023-11-06T02:11:54.580 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.580+0000 7f2275c9b640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f227015ab50 0x7f22700b0080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.49:41688/0 (socket says 172.21.15.49:41688) 2023-11-06T02:11:54.580 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.580+0000 7f227649c640 1 -- 172.21.15.49:0/2048571292 learned_addr learned my addr 172.21.15.49:0/2048571292 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:54.580 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.580+0000 7f227549a640 1 -- 172.21.15.49:0/2048571292 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f22700cb010 msgr2=0x7f22700c2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:54.580 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.580+0000 7f227549a640 1 --2- 172.21.15.49:0/2048571292 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f22700cb010 0x7f22700c2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:54.581 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.580+0000 7f227549a640 1 -- 172.21.15.49:0/2048571292 >> v1:172.21.15.150:6789/0 conn(0x7f227015ab50 legacy=0x7f22700b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:54.581 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.580+0000 7f227549a640 1 -- 172.21.15.49:0/2048571292 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22700cb890 con 0x7f22700b0790 2023-11-06T02:11:54.582 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.584+0000 7f227549a640 1 --2- 172.21.15.49:0/2048571292 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22700b0790 0x7f22700c9bb0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f22600097c0 tx=0x7f226001c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4bffc44c5963382b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:54.582 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.584+0000 7f2274c99640 1 -- 172.21.15.49:0/2048571292 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22600ab650 con 0x7f22700b0790 2023-11-06T02:11:54.582 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.584+0000 7f2274c99640 1 -- 172.21.15.49:0/2048571292 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22600ab7b0 con 0x7f22700b0790 2023-11-06T02:11:54.583 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.584+0000 7f2274c99640 1 -- 172.21.15.49:0/2048571292 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22600abb40 con 0x7f22700b0790 2023-11-06T02:11:54.583 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.584+0000 7f2277eff640 1 -- 172.21.15.49:0/2048571292 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22700b0790 msgr2=0x7f22700c9bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:54.583 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.584+0000 7f2277eff640 1 --2- 172.21.15.49:0/2048571292 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22700b0790 0x7f22700c9bb0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f22600097c0 tx=0x7f226001c450 comp rx=0 tx=0).stop 2023-11-06T02:11:54.583 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.584+0000 7f2277eff640 1 -- 172.21.15.49:0/2048571292 shutdown_connections 2023-11-06T02:11:54.583 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.584+0000 7f2277eff640 1 --2- 172.21.15.49:0/2048571292 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f22700cb010 0x7f22700c2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:54.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.584+0000 7f2277eff640 1 --2- 172.21.15.49:0/2048571292 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22700b0790 0x7f22700c9bb0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:54.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.584+0000 7f2277eff640 1 -- 172.21.15.49:0/2048571292 >> 172.21.15.49:0/2048571292 conn(0x7f227006c0a0 msgr2=0x7f227006c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:54.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.584+0000 7f2277eff640 1 -- 172.21.15.49:0/2048571292 shutdown_connections 2023-11-06T02:11:54.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.584+0000 7f2277eff640 1 -- 172.21.15.49:0/2048571292 wait complete. 2023-11-06T02:11:54.587 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.584+0000 7f2277eff640 1 Processor -- start 2023-11-06T02:11:54.587 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.584+0000 7f2277eff640 1 -- start start 2023-11-06T02:11:54.587 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.584+0000 7f2277eff640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f22700b0790 0x7f227014fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:54.587 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.584+0000 7f2277eff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f22700cb010 0x7f22700c39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:54.587 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.584+0000 7f2277eff640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f227015ab50 0x7f2270154d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:54.588 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.584+0000 7f2277eff640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2270152d00 con 0x7f22700cb010 2023-11-06T02:11:54.588 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.584+0000 7f2277eff640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22700acf50 con 0x7f227015ab50 2023-11-06T02:11:54.588 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.584+0000 7f2277eff640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22700c4330 con 0x7f22700b0790 2023-11-06T02:11:54.588 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.584+0000 7f227549a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f22700cb010 0x7f22700c39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:54.588 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.584+0000 7f227549a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f22700cb010 0x7f22700c39e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59748/0 (socket says 172.21.15.49:59748) 2023-11-06T02:11:54.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.584+0000 7f227549a640 1 -- 172.21.15.49:0/758338516 learned_addr learned my addr 172.21.15.49:0/758338516 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:54.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.584+0000 7f2275c9b640 1 --2- 172.21.15.49:0/758338516 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f22700b0790 0x7f227014fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:54.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.584+0000 7f227649c640 1 --2- 172.21.15.49:0/758338516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f227015ab50 0x7f2270154d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:54.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.584+0000 7f227549a640 1 -- 172.21.15.49:0/758338516 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f22700b0790 msgr2=0x7f227014fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:54.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.584+0000 7f227549a640 1 --2- 172.21.15.49:0/758338516 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f22700b0790 0x7f227014fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:54.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.584+0000 7f227549a640 1 -- 172.21.15.49:0/758338516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f227015ab50 msgr2=0x7f2270154d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:54.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.584+0000 7f227549a640 1 --2- 172.21.15.49:0/758338516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f227015ab50 0x7f2270154d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:54.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.584+0000 7f227549a640 1 -- 172.21.15.49:0/758338516 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2260009420 con 0x7f22700cb010 2023-11-06T02:11:54.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.584+0000 7f227649c640 1 --2- 172.21.15.49:0/758338516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f227015ab50 0x7f2270154d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:11:54.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.588+0000 7f227549a640 1 --2- 172.21.15.49:0/758338516 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f22700cb010 0x7f22700c39e0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f22600ad040 tx=0x7f226001ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:54.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.588+0000 7f22667fc640 1 -- 172.21.15.49:0/758338516 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22600bf610 con 0x7f22700cb010 2023-11-06T02:11:54.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.588+0000 7f22667fc640 1 -- 172.21.15.49:0/758338516 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22600bf770 con 0x7f22700cb010 2023-11-06T02:11:54.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.588+0000 7f2277eff640 1 -- 172.21.15.49:0/758338516 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22700c4610 con 0x7f22700cb010 2023-11-06T02:11:54.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.588+0000 7f22667fc640 1 -- 172.21.15.49:0/758338516 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22600bfae0 con 0x7f22700cb010 2023-11-06T02:11:54.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.588+0000 7f2277eff640 1 -- 172.21.15.49:0/758338516 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2270152990 con 0x7f22700cb010 2023-11-06T02:11:54.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.588+0000 7f2277eff640 1 -- 172.21.15.49:0/758338516 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22700b0de0 con 0x7f22700cb010 2023-11-06T02:11:54.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.588+0000 7f22667fc640 1 -- 172.21.15.49:0/758338516 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f22600bfc40 con 0x7f22700cb010 2023-11-06T02:11:54.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.588+0000 7f22667fc640 1 --2- 172.21.15.49:0/758338516 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2254065a90 0x7f2254067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:54.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.588+0000 7f22667fc640 1 -- 172.21.15.49:0/758338516 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f226011e1c0 con 0x7f22700cb010 2023-11-06T02:11:54.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.588+0000 7f2275c9b640 1 --2- 172.21.15.49:0/758338516 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2254065a90 0x7f2254067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:54.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.588+0000 7f2275c9b640 1 --2- 172.21.15.49:0/758338516 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2254065a90 0x7f2254067f50 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f226c000fc0 tx=0x7f226c007ce0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:54.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.592+0000 7f22667fc640 1 -- 172.21.15.49:0/758338516 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f22600ebab0 con 0x7f22700cb010 2023-11-06T02:11:54.781 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.784+0000 7f2277eff640 1 -- 172.21.15.49:0/758338516 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2270074040 con 0x7f22700cb010 2023-11-06T02:11:54.782 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.784+0000 7f22667fc640 1 -- 172.21.15.49:0/758338516 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+4652 (secure 0 0 0) 0x7f22600ef0d0 con 0x7f22700cb010 2023-11-06T02:11:54.783 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:11:54.783 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":30,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5635,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4170291905","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":4170291905},{"type":"v1","addr":"172.21.15.150:6835","nonce":4170291905}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5638,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/2719458937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":2719458937},{"type":"v1","addr":"172.21.15.150:6837","nonce":2719458937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":6339,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/2310338375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2310338375},{"type":"v1","addr":"172.21.15.49:6833","nonce":2310338375}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T02:11:30.091070+0000","modified":"2023-11-06T02:11:33.173821+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6348},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6348":{"gid":6348,"name":"c","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.49:6835/342908487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":342908487},{"type":"v1","addr":"172.21.15.49:6835","nonce":342908487}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-06T02:11:54.783 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 30 2023-11-06T02:11:54.785 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.784+0000 7f2277eff640 1 -- 172.21.15.49:0/758338516 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2254065a90 msgr2=0x7f2254067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:54.785 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.784+0000 7f2277eff640 1 --2- 172.21.15.49:0/758338516 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2254065a90 0x7f2254067f50 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f226c000fc0 tx=0x7f226c007ce0 comp rx=0 tx=0).stop 2023-11-06T02:11:54.785 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.784+0000 7f2277eff640 1 -- 172.21.15.49:0/758338516 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f22700cb010 msgr2=0x7f22700c39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:54.785 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.784+0000 7f2277eff640 1 --2- 172.21.15.49:0/758338516 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f22700cb010 0x7f22700c39e0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f22600ad040 tx=0x7f226001ac90 comp rx=0 tx=0).stop 2023-11-06T02:11:54.785 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.784+0000 7f2277eff640 1 -- 172.21.15.49:0/758338516 shutdown_connections 2023-11-06T02:11:54.786 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.784+0000 7f2277eff640 1 --2- 172.21.15.49:0/758338516 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2254065a90 0x7f2254067f50 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:54.786 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.784+0000 7f2277eff640 1 --2- 172.21.15.49:0/758338516 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f22700cb010 0x7f22700c39e0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:54.786 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.784+0000 7f2277eff640 1 --2- 172.21.15.49:0/758338516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f227015ab50 0x7f2270154d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:54.786 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.784+0000 7f2277eff640 1 --2- 172.21.15.49:0/758338516 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f22700b0790 0x7f227014fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:54.787 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.784+0000 7f2277eff640 1 -- 172.21.15.49:0/758338516 >> 172.21.15.49:0/758338516 conn(0x7f227006c0a0 msgr2=0x7f22700ce570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:54.787 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.784+0000 7f2277eff640 1 -- 172.21.15.49:0/758338516 shutdown_connections 2023-11-06T02:11:54.787 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.788+0000 7f2277eff640 1 -- 172.21.15.49:0/758338516 wait complete. 2023-11-06T02:11:54.802 DEBUG:teuthology.orchestra.run.smithi049:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata get mds0_inotable - 2023-11-06T02:11:54.840 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.840+0000 7ff5965ce080 1 Processor -- start 2023-11-06T02:11:54.841 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.840+0000 7ff5965ce080 1 -- start start 2023-11-06T02:11:54.843 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.844+0000 7ff5965ce080 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff57c0029e0 0x55ac00fff520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:54.843 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.844+0000 7ff5965ce080 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ac00ff75f0 0x55ac00ffdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:54.843 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.844+0000 7ff5965ce080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ac00f26a00 0x55ac00ff9f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:54.844 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.844+0000 7ff5965ce080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac00ed0940 con 0x55ac00f26a00 2023-11-06T02:11:54.844 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.844+0000 7ff5965ce080 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac00e6c450 con 0x7ff57c0029e0 2023-11-06T02:11:54.844 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.844+0000 7ff5965ce080 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ac00e6c150 con 0x55ac00ff75f0 2023-11-06T02:11:54.844 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.844+0000 7ff5948e9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ac00f26a00 0x55ac00ff9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:54.844 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.844+0000 7ff5948e9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ac00f26a00 0x55ac00ff9f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59754/0 (socket says 172.21.15.49:59754) 2023-11-06T02:11:54.845 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.844+0000 7ff5948e9640 1 -- 172.21.15.49:0/2009502186 learned_addr learned my addr 172.21.15.49:0/2009502186 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:54.845 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.844+0000 7ff5940e8640 1 --2- 172.21.15.49:0/2009502186 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff57c0029e0 0x55ac00fff520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:54.845 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.844+0000 7ff5948e9640 1 -- 172.21.15.49:0/2009502186 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ac00ff75f0 msgr2=0x55ac00ffdd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:54.845 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.844+0000 7ff5938e7640 1 --2- 172.21.15.49:0/2009502186 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ac00ff75f0 0x55ac00ffdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:54.845 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.844+0000 7ff5948e9640 1 --2- 172.21.15.49:0/2009502186 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ac00ff75f0 0x55ac00ffdd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:54.846 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.844+0000 7ff5948e9640 1 -- 172.21.15.49:0/2009502186 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff57c0029e0 msgr2=0x55ac00fff520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:54.846 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.844+0000 7ff5948e9640 1 --2- 172.21.15.49:0/2009502186 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff57c0029e0 0x55ac00fff520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:54.846 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.844+0000 7ff5948e9640 1 -- 172.21.15.49:0/2009502186 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ac00e6e430 con 0x55ac00f26a00 2023-11-06T02:11:54.846 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.844+0000 7ff5940e8640 1 --2- 172.21.15.49:0/2009502186 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff57c0029e0 0x55ac00fff520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:11:54.846 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.844+0000 7ff5948e9640 1 --2- 172.21.15.49:0/2009502186 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ac00f26a00 0x55ac00ff9f60 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7ff58c002470 tx=0x7ff58c005d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e4d5166f301efa server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:54.847 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.844+0000 7ff5928e5640 1 -- 172.21.15.49:0/2009502186 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58c0124c0 con 0x55ac00f26a00 2023-11-06T02:11:54.847 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.844+0000 7ff5928e5640 1 -- 172.21.15.49:0/2009502186 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff58c012620 con 0x55ac00f26a00 2023-11-06T02:11:54.847 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.844+0000 7ff5928e5640 1 -- 172.21.15.49:0/2009502186 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58c0129d0 con 0x55ac00f26a00 2023-11-06T02:11:54.847 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.844+0000 7ff5965ce080 1 -- 172.21.15.49:0/2009502186 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ac00f26a00 msgr2=0x55ac00ff9f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:54.847 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.844+0000 7ff5965ce080 1 --2- 172.21.15.49:0/2009502186 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ac00f26a00 0x55ac00ff9f60 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7ff58c002470 tx=0x7ff58c005d90 comp rx=0 tx=0).stop 2023-11-06T02:11:54.848 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.844+0000 7ff5965ce080 1 -- 172.21.15.49:0/2009502186 shutdown_connections 2023-11-06T02:11:54.848 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.844+0000 7ff5965ce080 1 --2- 172.21.15.49:0/2009502186 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ac00f26a00 0x55ac00ff9f60 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:54.848 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.844+0000 7ff5965ce080 1 --2- 172.21.15.49:0/2009502186 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ac00ff75f0 0x55ac00ffdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:54.848 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.844+0000 7ff5965ce080 1 --2- 172.21.15.49:0/2009502186 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff57c0029e0 0x55ac00fff520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:54.848 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.844+0000 7ff5965ce080 1 -- 172.21.15.49:0/2009502186 >> 172.21.15.49:0/2009502186 conn(0x55ac00f50cf0 msgr2=0x55ac01002fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:54.849 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.844+0000 7ff5965ce080 1 -- 172.21.15.49:0/2009502186 shutdown_connections 2023-11-06T02:11:54.849 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.844+0000 7ff5965ce080 1 -- 172.21.15.49:0/2009502186 wait complete. 2023-11-06T02:11:54.849 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.844+0000 7ff5965ce080 1 Processor -- start 2023-11-06T02:11:54.849 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.844+0000 7ff5965ce080 1 -- start start 2023-11-06T02:11:54.852 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.844+0000 7ff5965ce080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ac00f26a00 0x55ac00ffb880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:54.852 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.844+0000 7ff5965ce080 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ac00ff75f0 0x55ac00fe37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:54.852 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.844+0000 7ff5965ce080 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff57c0029e0 0x55ac00fe3d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:54.853 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.844+0000 7ff5965ce080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac00e62dc0 con 0x55ac00f26a00 2023-11-06T02:11:54.853 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.844+0000 7ff5965ce080 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac00fe6840 con 0x55ac00ff75f0 2023-11-06T02:11:54.853 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.844+0000 7ff5965ce080 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ac00fe6ae0 con 0x7ff57c0029e0 2023-11-06T02:11:54.853 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.844+0000 7ff5940e8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ac00f26a00 0x55ac00ffb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:54.853 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.844+0000 7ff5940e8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ac00f26a00 0x55ac00ffb880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59758/0 (socket says 172.21.15.49:59758) 2023-11-06T02:11:54.854 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.844+0000 7ff5940e8640 1 -- 172.21.15.49:0/2513676115 learned_addr learned my addr 172.21.15.49:0/2513676115 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:54.854 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.844+0000 7ff5948e9640 1 --2- 172.21.15.49:0/2513676115 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff57c0029e0 0x55ac00fe3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:54.854 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.844+0000 7ff5938e7640 1 --2- 172.21.15.49:0/2513676115 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ac00ff75f0 0x55ac00fe37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:54.854 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.844+0000 7ff5940e8640 1 -- 172.21.15.49:0/2513676115 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff57c0029e0 msgr2=0x55ac00fe3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:54.854 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.844+0000 7ff5940e8640 1 --2- 172.21.15.49:0/2513676115 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff57c0029e0 0x55ac00fe3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:54.855 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.844+0000 7ff5940e8640 1 -- 172.21.15.49:0/2513676115 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ac00ff75f0 msgr2=0x55ac00fe37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:54.855 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.844+0000 7ff5940e8640 1 --2- 172.21.15.49:0/2513676115 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ac00ff75f0 0x55ac00fe37d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:54.855 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.844+0000 7ff5940e8640 1 -- 172.21.15.49:0/2513676115 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff58c00d040 con 0x55ac00f26a00 2023-11-06T02:11:54.855 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.848+0000 7ff5940e8640 1 --2- 172.21.15.49:0/2513676115 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ac00f26a00 0x55ac00ffb880 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7ff58400cf70 tx=0x7ff58400a5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:54.855 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.848+0000 7ff5918e3640 1 -- 172.21.15.49:0/2513676115 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff584010680 con 0x55ac00f26a00 2023-11-06T02:11:54.856 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.848+0000 7ff5918e3640 1 -- 172.21.15.49:0/2513676115 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5840107e0 con 0x55ac00f26a00 2023-11-06T02:11:54.856 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.848+0000 7ff5965ce080 1 -- 172.21.15.49:0/2513676115 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ac00e6de90 con 0x55ac00f26a00 2023-11-06T02:11:54.856 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.848+0000 7ff5918e3640 1 -- 172.21.15.49:0/2513676115 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff584010b70 con 0x55ac00f26a00 2023-11-06T02:11:54.856 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.848+0000 7ff5965ce080 1 -- 172.21.15.49:0/2513676115 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ac00fe7020 con 0x55ac00f26a00 2023-11-06T02:11:54.856 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.848+0000 7ff5918e3640 1 -- 172.21.15.49:0/2513676115 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7ff584010cd0 con 0x55ac00f26a00 2023-11-06T02:11:54.857 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.848+0000 7ff5918e3640 1 --2- 172.21.15.49:0/2513676115 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff580064410 0x7ff5800668d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:54.857 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.848+0000 7ff5918e3640 1 -- 172.21.15.49:0/2513676115 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff584082530 con 0x55ac00f26a00 2023-11-06T02:11:54.857 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.848+0000 7ff5965ce080 1 --2- 172.21.15.49:0/2513676115 >> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] conn(0x55ac00ffbdc0 0x55ac00f54530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:54.857 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.848+0000 7ff5965ce080 1 -- 172.21.15.49:0/2513676115 --> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] -- osd_op(unknown.0.0:1 9.13 9:c91d4a1d:::mds0_inotable:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65) v8 -- 0x55ac00f54a70 con 0x55ac00ffbdc0 2023-11-06T02:11:54.857 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.848+0000 7ff5938e7640 1 --2- 172.21.15.49:0/2513676115 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff580064410 0x7ff5800668d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:54.858 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.848+0000 7ff5948e9640 1 --2- 172.21.15.49:0/2513676115 >> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] conn(0x55ac00ffbdc0 0x55ac00f54530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:54.858 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.848+0000 7ff5938e7640 1 --2- 172.21.15.49:0/2513676115 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff580064410 0x7ff5800668d0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff588009900 tx=0x55ac00f65550 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:54.858 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.848+0000 7ff5948e9640 1 --2- 172.21.15.49:0/2513676115 >> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] conn(0x55ac00ffbdc0 0x55ac00f54530 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:54.858 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.852+0000 7ff5948e9640 1 -- 172.21.15.49:0/2513676115 <== osd.7 v2:172.21.15.150:6816/38421 1 ==== osd_op_reply(1 mds0_inotable [read 0~34 out=34b] v0'0 uv3 ondisk = 0) v8 ==== 157+0+34 (crc 0 0 0) 0x7ff58c018070 con 0x55ac00ffbdc0 2023-11-06T02:11:54.860 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.852+0000 7ff5965ce080 1 -- 172.21.15.49:0/2513676115 >> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] conn(0x55ac00ffbdc0 msgr2=0x55ac00f54530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:54.860 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.852+0000 7ff5965ce080 1 --2- 172.21.15.49:0/2513676115 >> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] conn(0x55ac00ffbdc0 0x55ac00f54530 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:54.860 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.852+0000 7ff5965ce080 1 -- 172.21.15.49:0/2513676115 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff580064410 msgr2=0x7ff5800668d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:54.860 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.852+0000 7ff5965ce080 1 --2- 172.21.15.49:0/2513676115 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff580064410 0x7ff5800668d0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff588009900 tx=0x55ac00f65550 comp rx=0 tx=0).stop 2023-11-06T02:11:54.861 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.852+0000 7ff5965ce080 1 -- 172.21.15.49:0/2513676115 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ac00f26a00 msgr2=0x55ac00ffb880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:54.861 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.852+0000 7ff5965ce080 1 --2- 172.21.15.49:0/2513676115 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ac00f26a00 0x55ac00ffb880 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7ff58400cf70 tx=0x7ff58400a5b0 comp rx=0 tx=0).stop 2023-11-06T02:11:54.861 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.852+0000 7ff5948e9640 1 -- 172.21.15.49:0/2513676115 reap_dead start 2023-11-06T02:11:54.861 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.852+0000 7ff5965ce080 1 -- 172.21.15.49:0/2513676115 shutdown_connections 2023-11-06T02:11:54.861 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.852+0000 7ff5965ce080 1 -- 172.21.15.49:0/2513676115 >> 172.21.15.49:0/2513676115 conn(0x55ac00f50cf0 msgr2=0x55ac00f24410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:54.862 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.852+0000 7ff5965ce080 1 -- 172.21.15.49:0/2513676115 shutdown_connections 2023-11-06T02:11:54.862 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:54.852+0000 7ff5965ce080 1 -- 172.21.15.49:0/2513676115 wait complete. 2023-11-06T02:11:54.862 INFO:teuthology.orchestra.run.smithi049.stdout:����� 2023-11-06T02:11:54.862 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-06T02:11:55.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:55.028+0000 7faa55428640 1 Processor -- start 2023-11-06T02:11:55.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:55.028+0000 7faa55428640 1 -- start start 2023-11-06T02:11:55.026 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:55.028+0000 7faa55428640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7faa5015ab50 0x7faa500c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:55.026 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:55.028+0000 7faa55428640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa500cab80 con 0x7faa500cb010 2023-11-06T02:11:55.026 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:55.028+0000 7faa55428640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa5015af30 con 0x7faa500b0790 2023-11-06T02:11:55.026 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:55.028+0000 7faa55428640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa500cb5b0 con 0x7faa5015ab50 2023-11-06T02:11:55.027 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:55.028+0000 7faa4effd640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7faa5015ab50 0x7faa500c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:55.027 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:55.028+0000 7faa4e7fc640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7faa500cb010 0x7faa500b0080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:55180/0 (socket says 172.21.15.49:55180) 2023-11-06T02:11:55.027 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:55.028+0000 7faa4e7fc640 1 -- 172.21.15.49:0/1091824822 learned_addr learned my addr 172.21.15.49:0/1091824822 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:11:55.028 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:55.028+0000 7faa4effd640 1 -- 172.21.15.49:0/1091824822 >> v1:172.21.15.150:6789/0 conn(0x7faa500b0790 legacy=0x7faa500c2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:55.028 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:55.028+0000 7faa4effd640 1 -- 172.21.15.49:0/1091824822 >> v1:172.21.15.49:6789/0 conn(0x7faa500cb010 legacy=0x7faa500b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:55.028 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:55.028+0000 7faa4effd640 1 -- 172.21.15.49:0/1091824822 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa500cb890 con 0x7faa5015ab50 2023-11-06T02:11:55.029 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:55.028+0000 7faa4effd640 1 --2- 172.21.15.49:0/1091824822 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa5015ab50 0x7faa500c9bb0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7faa40009de0 tx=0x7faa4001c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7872f783a62325f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:55.029 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:55.028+0000 7faa4dffb640 1 -- 172.21.15.49:0/1091824822 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa400a59e0 con 0x7faa5015ab50 2023-11-06T02:11:55.029 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:55.028+0000 7faa4dffb640 1 -- 172.21.15.49:0/1091824822 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa400a5b40 con 0x7faa5015ab50 2023-11-06T02:11:55.030 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:55.028+0000 7faa4dffb640 1 -- 172.21.15.49:0/1091824822 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa400ad440 con 0x7faa5015ab50 2023-11-06T02:11:55.030 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:55.032+0000 7faa55428640 1 -- 172.21.15.49:0/1091824822 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa5015ab50 msgr2=0x7faa500c9bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:55.030 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:55.032+0000 7faa55428640 1 --2- 172.21.15.49:0/1091824822 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa5015ab50 0x7faa500c9bb0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7faa40009de0 tx=0x7faa4001c920 comp rx=0 tx=0).stop 2023-11-06T02:11:55.030 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:55.032+0000 7faa55428640 1 -- 172.21.15.49:0/1091824822 shutdown_connections 2023-11-06T02:11:55.031 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:55.032+0000 7faa55428640 1 --2- 172.21.15.49:0/1091824822 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa5015ab50 0x7faa500c9bb0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:55.031 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:55.032+0000 7faa55428640 1 -- 172.21.15.49:0/1091824822 >> 172.21.15.49:0/1091824822 conn(0x7faa5006c0a0 msgr2=0x7faa5006c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:55.031 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:55.032+0000 7faa55428640 1 -- 172.21.15.49:0/1091824822 shutdown_connections 2023-11-06T02:11:55.031 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:55.032+0000 7faa55428640 1 -- 172.21.15.49:0/1091824822 wait complete. 2023-11-06T02:11:55.034 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:55.032+0000 7faa55428640 1 Processor -- start 2023-11-06T02:11:55.034 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:55.032+0000 7faa55428640 1 -- start start 2023-11-06T02:11:55.034 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:55.032+0000 7faa55428640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa500b0790 0x7faa50152850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:55.035 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:55.032+0000 7faa55428640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa500cb010 0x7faa5015d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:55.035 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:55.032+0000 7faa55428640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa5015ab50 0x7faa500ccca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:55.035 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:55.032+0000 7faa55428640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa501583e0 con 0x7faa5015ab50 2023-11-06T02:11:55.035 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:55.032+0000 7faa55428640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa50158100 con 0x7faa500b0790 2023-11-06T02:11:55.035 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:55.032+0000 7faa55428640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa5015cdd0 con 0x7faa500cb010 2023-11-06T02:11:55.036 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:55.032+0000 7faa4f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa5015ab50 0x7faa500ccca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:55.036 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:55.032+0000 7faa4f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa5015ab50 0x7faa500ccca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59782/0 (socket says 172.21.15.49:59782) 2023-11-06T02:11:55.036 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:55.032+0000 7faa4f7fe640 1 -- 172.21.15.49:0/3928542794 learned_addr learned my addr 172.21.15.49:0/3928542794 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:55.036 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:55.032+0000 7faa4effd640 1 --2- 172.21.15.49:0/3928542794 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa500b0790 0x7faa50152850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:55.036 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:55.032+0000 7faa4f7fe640 1 -- 172.21.15.49:0/3928542794 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa500cb010 msgr2=0x7faa5015d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:55.036 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:55.032+0000 7faa4e7fc640 1 --2- 172.21.15.49:0/3928542794 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa500cb010 0x7faa5015d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:55.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:55.032+0000 7faa4f7fe640 1 --2- 172.21.15.49:0/3928542794 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa500cb010 0x7faa5015d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:55.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:55.032+0000 7faa4f7fe640 1 -- 172.21.15.49:0/3928542794 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa500b0790 msgr2=0x7faa50152850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:55.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:55.032+0000 7faa4f7fe640 1 --2- 172.21.15.49:0/3928542794 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa500b0790 0x7faa50152850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:55.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:55.032+0000 7faa4f7fe640 1 -- 172.21.15.49:0/3928542794 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa40009a70 con 0x7faa5015ab50 2023-11-06T02:11:55.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:55.032+0000 7faa4effd640 1 --2- 172.21.15.49:0/3928542794 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa500b0790 0x7faa50152850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:11:55.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:55.032+0000 7faa4f7fe640 1 --2- 172.21.15.49:0/3928542794 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa5015ab50 0x7faa500ccca0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7faa4800c390 tx=0x7faa4800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:55.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:55.032+0000 7faa377fe640 1 -- 172.21.15.49:0/3928542794 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa48011790 con 0x7faa5015ab50 2023-11-06T02:11:55.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:55.032+0000 7faa377fe640 1 -- 172.21.15.49:0/3928542794 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa480118f0 con 0x7faa5015ab50 2023-11-06T02:11:55.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:55.032+0000 7faa55428640 1 -- 172.21.15.49:0/3928542794 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa5015d0b0 con 0x7faa5015ab50 2023-11-06T02:11:55.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:55.032+0000 7faa377fe640 1 -- 172.21.15.49:0/3928542794 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa48011ca0 con 0x7faa5015ab50 2023-11-06T02:11:55.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:55.032+0000 7faa55428640 1 -- 172.21.15.49:0/3928542794 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa5015ddf0 con 0x7faa5015ab50 2023-11-06T02:11:55.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:55.036+0000 7faa55428640 1 -- 172.21.15.49:0/3928542794 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa500b0de0 con 0x7faa5015ab50 2023-11-06T02:11:55.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:55.036+0000 7faa377fe640 1 -- 172.21.15.49:0/3928542794 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7faa4801c460 con 0x7faa5015ab50 2023-11-06T02:11:55.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:55.036+0000 7faa377fe640 1 --2- 172.21.15.49:0/3928542794 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7faa3c065a60 0x7faa3c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:55.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:55.036+0000 7faa377fe640 1 -- 172.21.15.49:0/3928542794 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7faa48082f10 con 0x7faa5015ab50 2023-11-06T02:11:55.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:55.036+0000 7faa4effd640 1 --2- 172.21.15.49:0/3928542794 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7faa3c065a60 0x7faa3c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:55.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:55.036+0000 7faa4effd640 1 --2- 172.21.15.49:0/3928542794 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7faa3c065a60 0x7faa3c067f20 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7faa400af000 tx=0x7faa400a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:55.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:55.040+0000 7faa377fe640 1 -- 172.21.15.49:0/3928542794 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7faa48050800 con 0x7faa5015ab50 2023-11-06T02:11:55.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:55.228+0000 7faa55428640 1 -- 172.21.15.49:0/3928542794 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7faa50074040 con 0x7faa5015ab50 2023-11-06T02:11:55.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:55.228+0000 7faa377fe640 1 -- 172.21.15.49:0/3928542794 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v30) v1 ==== 53+0+73 (secure 0 0 0) 0x7faa48053e20 con 0x7faa5015ab50 2023-11-06T02:11:55.229 INFO:teuthology.orchestra.run.smithi049.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-06T02:11:55.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:55.232+0000 7faa55428640 1 -- 172.21.15.49:0/3928542794 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7faa3c065a60 msgr2=0x7faa3c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:55.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:55.232+0000 7faa55428640 1 --2- 172.21.15.49:0/3928542794 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7faa3c065a60 0x7faa3c067f20 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7faa400af000 tx=0x7faa400a7070 comp rx=0 tx=0).stop 2023-11-06T02:11:55.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:55.232+0000 7faa55428640 1 -- 172.21.15.49:0/3928542794 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa5015ab50 msgr2=0x7faa500ccca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:55.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:55.232+0000 7faa55428640 1 --2- 172.21.15.49:0/3928542794 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa5015ab50 0x7faa500ccca0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7faa4800c390 tx=0x7faa4800c850 comp rx=0 tx=0).stop 2023-11-06T02:11:55.232 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:55.232+0000 7faa55428640 1 -- 172.21.15.49:0/3928542794 shutdown_connections 2023-11-06T02:11:55.232 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:55.232+0000 7faa55428640 1 --2- 172.21.15.49:0/3928542794 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7faa3c065a60 0x7faa3c067f20 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:55.232 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:55.232+0000 7faa55428640 1 --2- 172.21.15.49:0/3928542794 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faa5015ab50 0x7faa500ccca0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:55.232 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:55.232+0000 7faa55428640 1 --2- 172.21.15.49:0/3928542794 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa500cb010 0x7faa5015d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:55.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:55.232+0000 7faa55428640 1 --2- 172.21.15.49:0/3928542794 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa500b0790 0x7faa50152850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:55.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:55.232+0000 7faa55428640 1 -- 172.21.15.49:0/3928542794 >> 172.21.15.49:0/3928542794 conn(0x7faa5006c0a0 msgr2=0x7faa500cc1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:55.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:55.232+0000 7faa55428640 1 -- 172.21.15.49:0/3928542794 shutdown_connections 2023-11-06T02:11:55.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:55.232+0000 7faa55428640 1 -- 172.21.15.49:0/3928542794 wait complete. 2023-11-06T02:11:55.247 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-06T02:11:55.248 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-06T02:11:55.248 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi049.front.sepia.ceph.com 2023-11-06T02:11:55.248 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-06T02:11:55.249 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-06T02:11:55.249 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-06T02:11:55.249 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-06T02:11:55.250 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-06T02:11:55.250 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-06T02:11:55.251 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:11:55.251 DEBUG:teuthology.orchestra.run.smithi049:> ip addr 2023-11-06T02:11:55.257 INFO:teuthology.orchestra.run.smithi049.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-06T02:11:55.257 INFO:teuthology.orchestra.run.smithi049.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-06T02:11:55.258 INFO:teuthology.orchestra.run.smithi049.stdout: inet 127.0.0.1/8 scope host lo 2023-11-06T02:11:55.258 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:11:55.258 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 ::1/128 scope host 2023-11-06T02:11:55.258 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:11:55.258 INFO:teuthology.orchestra.run.smithi049.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-06T02:11:55.259 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:6b:ff:84 brd ff:ff:ff:ff:ff:ff 2023-11-06T02:11:55.259 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp7s0f0 2023-11-06T02:11:55.259 INFO:teuthology.orchestra.run.smithi049.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-06T02:11:55.259 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:6b:ff:85 brd ff:ff:ff:ff:ff:ff 2023-11-06T02:11:55.259 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp7s0f1 2023-11-06T02:11:55.260 INFO:teuthology.orchestra.run.smithi049.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-06T02:11:55.260 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:bd:15:66 brd ff:ff:ff:ff:ff:ff 2023-11-06T02:11:55.260 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp1s0f0 2023-11-06T02:11:55.260 INFO:teuthology.orchestra.run.smithi049.stdout: inet 172.21.15.49/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-11-06T02:11:55.260 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft 42282sec preferred_lft 42282sec 2023-11-06T02:11:55.261 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 fe80::ec4:7aff:febd:1566/64 scope link 2023-11-06T02:11:55.261 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:11:55.261 INFO:teuthology.orchestra.run.smithi049.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-06T02:11:55.261 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:bd:15:67 brd ff:ff:ff:ff:ff:ff 2023-11-06T02:11:55.261 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp1s0f1 2023-11-06T02:11:55.262 INFO:teuthology.orchestra.run.smithi049.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-06T02:11:55.262 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 02:42:47:49:33:69 brd ff:ff:ff:ff:ff:ff 2023-11-06T02:11:55.262 INFO:teuthology.orchestra.run.smithi049.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-06T02:11:55.262 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:11:55.262 INFO:teuthology.orchestra.run.smithi049.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-06T02:11:55.263 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-11-06T02:11:55.263 INFO:teuthology.orchestra.run.smithi049.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-06T02:11:55.263 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:11:55.263 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-11-06T02:11:55.263 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:11:55.264 INFO:teuthology.orchestra.run.smithi049.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-06T02:11:55.264 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-06T02:11:55.264 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-11-06T02:11:55.264 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:11:55.264 INFO:teuthology.orchestra.run.smithi049.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-06T02:11:55.265 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-06T02:11:55.265 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-11-06T02:11:55.265 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:11:55.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:11:55.266 DEBUG:teuthology.orchestra.run.smithi049:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-06T02:11:55.311 INFO:teuthology.orchestra.run.smithi049.stdout:1 2023-11-06T02:11:55.312 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:11:55.313 DEBUG:teuthology.orchestra.run.smithi049:> route 2023-11-06T02:11:55.362 INFO:teuthology.orchestra.run.smithi049.stdout:Kernel IP routing table 2023-11-06T02:11:55.362 INFO:teuthology.orchestra.run.smithi049.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-06T02:11:55.363 INFO:teuthology.orchestra.run.smithi049.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-11-06T02:11:55.363 INFO:teuthology.orchestra.run.smithi049.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-06T02:11:55.363 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-11-06T02:11:55.364 INFO:teuthology.orchestra.run.smithi049.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-06T02:11:55.364 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:11:55.365 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec bash -c ' 2023-11-06T02:11:55.365 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2023-11-06T02:11:55.366 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-06T02:11:55.367 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-06T02:11:55.367 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-06T02:11:55.368 DEBUG:teuthology.orchestra.run.smithi049:> ') 2023-11-06T02:11:55.443 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:11:55.444 DEBUG:teuthology.orchestra.run.smithi049:> ip netns list 2023-11-06T02:11:55.493 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-06T02:11:55.494 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-06T02:11:55.494 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-06T02:11:55.495 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:11:55.495 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link set brx.0 up 2023-11-06T02:11:55.549 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-06T02:11:55.550 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-06T02:11:55.550 DEBUG:teuthology.orchestra.run.smithi049:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-06T02:11:55.598 INFO:teuthology.orchestra.run.smithi049.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-06T02:11:55.598 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-06T02:11:55.599 DEBUG:teuthology.orchestra.run.smithi049:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-06T02:11:55.645 DEBUG:teuthology.orchestra.run.smithi049:> sudo modprobe fuse 2023-11-06T02:11:55.703 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-06T02:11:55.704 DEBUG:teuthology.orchestra.run.smithi049:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-06T02:11:55.757 INFO:teuthology.orchestra.run.smithi049.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-06T02:11:55.758 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-06T02:11:55.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:11:55.759 DEBUG:teuthology.orchestra.run.smithi049:> ls /sys/fs/fuse/connections 2023-11-06T02:11:55.805 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-06T02:11:55.806 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-06T02:11:55.850 DEBUG:teuthology.orchestra.run.smithi049:> sudo modprobe fuse 2023-11-06T02:11:55.866 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-06T02:11:55.867 DEBUG:teuthology.orchestra.run.smithi049:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-06T02:11:55.920 INFO:teuthology.orchestra.run.smithi049.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-06T02:11:55.921 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-06T02:11:55.922 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:11:55.922 DEBUG:teuthology.orchestra.run.smithi049:> ls /sys/fs/fuse/connections 2023-11-06T02:11:55.929 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi049.stderr:2023-11-06T02:11:55.932+0000 7fc476561580 -1 init, newargv = 0x565358a38030 newargc=15 2023-11-06T02:11:55.930 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi049.stderr:ceph-fuse[52514]: starting ceph client 2023-11-06T02:11:55.935 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi049.stderr:ceph-fuse[52514]: starting fuse 2023-11-06T02:11:57.036 DEBUG:teuthology.orchestra.run.smithi049:> sudo modprobe fuse 2023-11-06T02:11:57.052 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-06T02:11:57.053 DEBUG:teuthology.orchestra.run.smithi049:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-06T02:11:57.108 INFO:teuthology.orchestra.run.smithi049.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-06T02:11:57.109 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-06T02:11:57.110 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:11:57.111 DEBUG:teuthology.orchestra.run.smithi049:> ls /sys/fs/fuse/connections 2023-11-06T02:11:57.158 INFO:teuthology.orchestra.run.smithi049.stdout:45 2023-11-06T02:11:57.160 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2023-11-06T02:11:57.161 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-11-06T02:11:57.161 DEBUG:teuthology.orchestra.run.smithi049:> import glob 2023-11-06T02:11:57.162 DEBUG:teuthology.orchestra.run.smithi049:> import re 2023-11-06T02:11:57.162 DEBUG:teuthology.orchestra.run.smithi049:> import os 2023-11-06T02:11:57.162 DEBUG:teuthology.orchestra.run.smithi049:> import subprocess 2023-11-06T02:11:57.163 DEBUG:teuthology.orchestra.run.smithi049:> 2023-11-06T02:11:57.163 DEBUG:teuthology.orchestra.run.smithi049:> def _find_admin_socket(client_name): 2023-11-06T02:11:57.164 DEBUG:teuthology.orchestra.run.smithi049:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-06T02:11:57.164 DEBUG:teuthology.orchestra.run.smithi049:> files = glob.glob(asok_path) 2023-11-06T02:11:57.164 DEBUG:teuthology.orchestra.run.smithi049:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-06T02:11:57.165 DEBUG:teuthology.orchestra.run.smithi049:> 2023-11-06T02:11:57.165 DEBUG:teuthology.orchestra.run.smithi049:> # Given a non-glob path, it better be there 2023-11-06T02:11:57.166 DEBUG:teuthology.orchestra.run.smithi049:> if "*" not in asok_path: 2023-11-06T02:11:57.166 DEBUG:teuthology.orchestra.run.smithi049:> assert(len(files) == 1) 2023-11-06T02:11:57.166 DEBUG:teuthology.orchestra.run.smithi049:> return files[0] 2023-11-06T02:11:57.167 DEBUG:teuthology.orchestra.run.smithi049:> 2023-11-06T02:11:57.167 DEBUG:teuthology.orchestra.run.smithi049:> for f in files: 2023-11-06T02:11:57.168 DEBUG:teuthology.orchestra.run.smithi049:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-06T02:11:57.168 DEBUG:teuthology.orchestra.run.smithi049:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-06T02:11:57.169 DEBUG:teuthology.orchestra.run.smithi049:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-06T02:11:57.169 DEBUG:teuthology.orchestra.run.smithi049:> contents = proc_f.read() 2023-11-06T02:11:57.170 DEBUG:teuthology.orchestra.run.smithi049:> if mountpoint in contents: 2023-11-06T02:11:57.170 DEBUG:teuthology.orchestra.run.smithi049:> return f 2023-11-06T02:11:57.170 DEBUG:teuthology.orchestra.run.smithi049:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-06T02:11:57.171 DEBUG:teuthology.orchestra.run.smithi049:> 2023-11-06T02:11:57.171 DEBUG:teuthology.orchestra.run.smithi049:> print(_find_admin_socket("client.0")) 2023-11-06T02:11:57.172 DEBUG:teuthology.orchestra.run.smithi049:> ' 2023-11-06T02:11:57.283 INFO:teuthology.orchestra.run.smithi049.stdout:/var/run/ceph/ceph-client.0.52514.asok 2023-11-06T02:11:57.464 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.52514.asok 2023-11-06T02:11:57.465 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:11:57.465 DEBUG:teuthology.orchestra.run.smithi049:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.52514.asok status 2023-11-06T02:11:57.633 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-11-06T02:11:57.633 INFO:teuthology.orchestra.run.smithi049.stdout: "metadata": { 2023-11-06T02:11:57.633 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_sha1": "f7e9f9af51acf98e0ccc3037487018e4ae40caa5", 2023-11-06T02:11:57.634 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_version": "ceph version 18.2.0-1181-gf7e9f9af (f7e9f9af51acf98e0ccc3037487018e4ae40caa5) reef (stable)", 2023-11-06T02:11:57.634 INFO:teuthology.orchestra.run.smithi049.stdout: "entity_id": "0", 2023-11-06T02:11:57.634 INFO:teuthology.orchestra.run.smithi049.stdout: "hostname": "smithi049", 2023-11-06T02:11:57.634 INFO:teuthology.orchestra.run.smithi049.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-06T02:11:57.634 INFO:teuthology.orchestra.run.smithi049.stdout: "pid": "52514", 2023-11-06T02:11:57.635 INFO:teuthology.orchestra.run.smithi049.stdout: "root": "/" 2023-11-06T02:11:57.635 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:11:57.635 INFO:teuthology.orchestra.run.smithi049.stdout: "dentry_count": 0, 2023-11-06T02:11:57.635 INFO:teuthology.orchestra.run.smithi049.stdout: "dentry_pinned_count": 0, 2023-11-06T02:11:57.635 INFO:teuthology.orchestra.run.smithi049.stdout: "id": 6630, 2023-11-06T02:11:57.636 INFO:teuthology.orchestra.run.smithi049.stdout: "inst": { 2023-11-06T02:11:57.636 INFO:teuthology.orchestra.run.smithi049.stdout: "name": { 2023-11-06T02:11:57.636 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "client", 2023-11-06T02:11:57.636 INFO:teuthology.orchestra.run.smithi049.stdout: "num": 6630 2023-11-06T02:11:57.636 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:11:57.637 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": { 2023-11-06T02:11:57.637 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2023-11-06T02:11:57.637 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "192.168.0.1:0", 2023-11-06T02:11:57.637 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 1289352588 2023-11-06T02:11:57.637 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:11:57.637 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:11:57.638 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": { 2023-11-06T02:11:57.638 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2023-11-06T02:11:57.638 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "192.168.0.1:0", 2023-11-06T02:11:57.638 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 1289352588 2023-11-06T02:11:57.638 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:11:57.639 INFO:teuthology.orchestra.run.smithi049.stdout: "inst_str": "client.6630 192.168.0.1:0/1289352588", 2023-11-06T02:11:57.639 INFO:teuthology.orchestra.run.smithi049.stdout: "addr_str": "192.168.0.1:0/1289352588", 2023-11-06T02:11:57.639 INFO:teuthology.orchestra.run.smithi049.stdout: "inode_count": 1, 2023-11-06T02:11:57.639 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_epoch": 30, 2023-11-06T02:11:57.639 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_epoch": 65, 2023-11-06T02:11:57.639 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_epoch_barrier": 0, 2023-11-06T02:11:57.640 INFO:teuthology.orchestra.run.smithi049.stdout: "blocklisted": false, 2023-11-06T02:11:57.640 INFO:teuthology.orchestra.run.smithi049.stdout: "fs_name": "cephfs" 2023-11-06T02:11:57.640 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-11-06T02:11:57.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:11:57.647 DEBUG:teuthology.orchestra.run.smithi049:> stat --file-system '--printf=%T 2023-11-06T02:11:57.648 DEBUG:teuthology.orchestra.run.smithi049:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-06T02:11:57.654 INFO:teuthology.orchestra.run.smithi049.stdout:fuseblk 2023-11-06T02:11:57.654 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-06T02:11:57.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:11:57.655 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-06T02:11:57.707 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-11-06T02:11:57.707 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T02:11:57.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:11:57.754 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file2_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2023-11-06T02:11:57.872 INFO:teuthology.orchestra.run.smithi049.stderr:6+0 records in 2023-11-06T02:11:57.872 INFO:teuthology.orchestra.run.smithi049.stderr:6+0 records out 2023-11-06T02:11:57.872 INFO:teuthology.orchestra.run.smithi049.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.0617939 s, 102 MB/s 2023-11-06T02:11:57.873 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-11-06T02:11:57.874 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T02:11:57.884 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:11:57.885 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file3_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2023-11-06T02:11:58.000 INFO:teuthology.orchestra.run.smithi049.stderr:6+0 records in 2023-11-06T02:11:58.001 INFO:teuthology.orchestra.run.smithi049.stderr:6+0 records out 2023-11-06T02:11:58.001 INFO:teuthology.orchestra.run.smithi049.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.0620648 s, 101 MB/s 2023-11-06T02:11:58.002 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:11:58.002 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2023-11-06T02:11:58.014 INFO:teuthology.orchestra.run.smithi049.stdout:./ 2023-11-06T02:11:58.014 INFO:teuthology.orchestra.run.smithi049.stdout:./file2_sixmegs 2023-11-06T02:11:58.015 INFO:teuthology.orchestra.run.smithi049.stdout:./file3_sixmegs 2023-11-06T02:11:58.015 INFO:teuthology.orchestra.run.smithi049.stdout:./file1_sixmegs 2023-11-06T02:11:58.016 DEBUG:teuthology.orchestra.run.smithi049:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-06T02:11:58.016 DEBUG:teuthology.orchestra.run.smithi049:> import os 2023-11-06T02:11:58.017 DEBUG:teuthology.orchestra.run.smithi049:> import stat 2023-11-06T02:11:58.017 DEBUG:teuthology.orchestra.run.smithi049:> 2023-11-06T02:11:58.017 DEBUG:teuthology.orchestra.run.smithi049:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2023-11-06T02:11:58.018 DEBUG:teuthology.orchestra.run.smithi049:> ' 2023-11-06T02:11:58.122 INFO:teuthology.orchestra.run.smithi049.stdout:1 2023-11-06T02:11:58.311 DEBUG:teuthology.orchestra.run.smithi049:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-06T02:11:58.312 DEBUG:teuthology.orchestra.run.smithi049:> import os 2023-11-06T02:11:58.312 DEBUG:teuthology.orchestra.run.smithi049:> import stat 2023-11-06T02:11:58.313 DEBUG:teuthology.orchestra.run.smithi049:> 2023-11-06T02:11:58.313 DEBUG:teuthology.orchestra.run.smithi049:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file2_sixmegs").st_ino) 2023-11-06T02:11:58.313 DEBUG:teuthology.orchestra.run.smithi049:> ' 2023-11-06T02:11:58.415 INFO:teuthology.orchestra.run.smithi049.stdout:1099511627777 2023-11-06T02:11:58.606 DEBUG:teuthology.orchestra.run.smithi049:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-06T02:11:58.606 DEBUG:teuthology.orchestra.run.smithi049:> import os 2023-11-06T02:11:58.607 DEBUG:teuthology.orchestra.run.smithi049:> import stat 2023-11-06T02:11:58.607 DEBUG:teuthology.orchestra.run.smithi049:> 2023-11-06T02:11:58.607 DEBUG:teuthology.orchestra.run.smithi049:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file3_sixmegs").st_ino) 2023-11-06T02:11:58.608 DEBUG:teuthology.orchestra.run.smithi049:> ' 2023-11-06T02:11:58.713 INFO:teuthology.orchestra.run.smithi049.stdout:1099511627778 2023-11-06T02:11:58.901 DEBUG:teuthology.orchestra.run.smithi049:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-06T02:11:58.901 DEBUG:teuthology.orchestra.run.smithi049:> import os 2023-11-06T02:11:58.902 DEBUG:teuthology.orchestra.run.smithi049:> import stat 2023-11-06T02:11:58.902 DEBUG:teuthology.orchestra.run.smithi049:> 2023-11-06T02:11:58.903 DEBUG:teuthology.orchestra.run.smithi049:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file1_sixmegs").st_ino) 2023-11-06T02:11:58.903 DEBUG:teuthology.orchestra.run.smithi049:> ' 2023-11-06T02:11:59.012 INFO:teuthology.orchestra.run.smithi049.stdout:1099511627776 2023-11-06T02:11:59.200 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-11-06T02:11:59.200 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T02:11:59.247 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T02:11:59.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.424+0000 7f1ba4cd2640 1 Processor -- start 2023-11-06T02:11:59.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.424+0000 7f1ba4cd2640 1 -- start start 2023-11-06T02:11:59.422 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.424+0000 7f1ba4cd2640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f1ba0160a60 0x7f1ba00cdc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:59.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.424+0000 7f1ba4cd2640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ba015db10 con 0x7f1ba015aa20 2023-11-06T02:11:59.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.424+0000 7f1ba4cd2640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ba015ae00 con 0x7f1ba00ce1d0 2023-11-06T02:11:59.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.424+0000 7f1ba4cd2640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ba015e310 con 0x7f1ba0160a60 2023-11-06T02:11:59.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.424+0000 7f1b9e59c640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f1ba015aa20 0x7f1ba015d400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:54718/0 (socket says 172.21.15.49:54718) 2023-11-06T02:11:59.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.424+0000 7f1b9e59c640 1 -- 172.21.15.49:0/3378080101 learned_addr learned my addr 172.21.15.49:0/3378080101 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:11:59.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.424+0000 7f1b9d59a640 1 -- 172.21.15.49:0/3378080101 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2799917942 0 0) 0x7f1b88001120 con 0x7f1ba015aa20 2023-11-06T02:11:59.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.424+0000 7f1b9dd9b640 1 --2- 172.21.15.49:0/3378080101 >> v2:172.21.15.150:3300/0 conn(0x7f1ba0160a60 0x7f1ba00cdc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:59.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.424+0000 7f1b9d59a640 1 -- 172.21.15.49:0/3378080101 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b8c003530 con 0x7f1ba015aa20 2023-11-06T02:11:59.427 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.424+0000 7f1b9d59a640 1 -- 172.21.15.49:0/3378080101 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1937833900 0 0) 0x7f1b8c003530 con 0x7f1ba015aa20 2023-11-06T02:11:59.427 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.424+0000 7f1b9d59a640 1 -- 172.21.15.49:0/3378080101 >> v2:172.21.15.150:3300/0 conn(0x7f1ba0160a60 msgr2=0x7f1ba00cdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:59.427 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.424+0000 7f1b9d59a640 1 --2- 172.21.15.49:0/3378080101 >> v2:172.21.15.150:3300/0 conn(0x7f1ba0160a60 0x7f1ba00cdc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:59.427 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.424+0000 7f1b9d59a640 1 -- 172.21.15.49:0/3378080101 >> v1:172.21.15.150:6789/0 conn(0x7f1ba00ce1d0 legacy=0x7f1ba00d0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:59.427 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.424+0000 7f1b9d59a640 1 -- 172.21.15.49:0/3378080101 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ba015dd50 con 0x7f1ba015aa20 2023-11-06T02:11:59.428 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.424+0000 7f1b9d59a640 1 -- 172.21.15.49:0/3378080101 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7f1b88001810 con 0x7f1ba015aa20 2023-11-06T02:11:59.428 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.424+0000 7f1b9d59a640 1 -- 172.21.15.49:0/3378080101 >> v1:172.21.15.49:6789/0 conn(0x7f1ba015aa20 legacy=0x7f1ba015d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:59.428 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.424+0000 7f1b9d59a640 1 --2- 172.21.15.49:0/3378080101 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1b8c003980 0x7f1b8c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:59.428 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.424+0000 7f1b9d59a640 1 --2- 172.21.15.49:0/3378080101 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b8c0047d0 0x7f1b8c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:59.428 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.424+0000 7f1b9d59a640 1 --2- 172.21.15.49:0/3378080101 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b8c005190 0x7f1b8c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:59.429 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.424+0000 7f1b9d59a640 1 -- 172.21.15.49:0/3378080101 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba015e310 con 0x7f1b8c0047d0 2023-11-06T02:11:59.429 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.424+0000 7f1b9d59a640 1 -- 172.21.15.49:0/3378080101 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b8c00c130 con 0x7f1b8c005190 2023-11-06T02:11:59.429 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.424+0000 7f1b9d59a640 1 -- 172.21.15.49:0/3378080101 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b8c00c2f0 con 0x7f1b8c003980 2023-11-06T02:11:59.429 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.424+0000 7f1b9dd9b640 1 --2- 172.21.15.49:0/3378080101 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b8c0047d0 0x7f1b8c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:59.429 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.424+0000 7f1b9e59c640 1 --2- 172.21.15.49:0/3378080101 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1b8c003980 0x7f1b8c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:59.430 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.424+0000 7f1b9ed9d640 1 --2- 172.21.15.49:0/3378080101 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b8c005190 0x7f1b8c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:59.430 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.424+0000 7f1b9dd9b640 1 -- 172.21.15.49:0/3378080101 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1b8c003980 msgr2=0x7f1b8c003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:59.430 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.424+0000 7f1b9dd9b640 1 --2- 172.21.15.49:0/3378080101 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1b8c003980 0x7f1b8c003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:59.430 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.424+0000 7f1b9dd9b640 1 -- 172.21.15.49:0/3378080101 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b8c005190 msgr2=0x7f1b8c00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:11:59.430 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.424+0000 7f1b9dd9b640 1 --2- 172.21.15.49:0/3378080101 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b8c005190 0x7f1b8c00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:59.431 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.424+0000 7f1b9ed9d640 1 -- 172.21.15.49:0/3378080101 reap_dead start 2023-11-06T02:11:59.431 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.424+0000 7f1b9dd9b640 1 -- 172.21.15.49:0/3378080101 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b8c003530 con 0x7f1b8c0047d0 2023-11-06T02:11:59.431 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.428+0000 7f1b9dd9b640 1 --2- 172.21.15.49:0/3378080101 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b8c0047d0 0x7f1b8c004c50 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f1b90009ae0 tx=0x7f1b9001ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaae17666c4ef365 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:59.431 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.428+0000 7f1b9d59a640 1 -- 172.21.15.49:0/3378080101 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b900a6510 con 0x7f1b8c0047d0 2023-11-06T02:11:59.431 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.428+0000 7f1b9d59a640 1 -- 172.21.15.49:0/3378080101 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b900a6670 con 0x7f1b8c0047d0 2023-11-06T02:11:59.432 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.428+0000 7f1b9d59a640 1 -- 172.21.15.49:0/3378080101 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b900a6a20 con 0x7f1b8c0047d0 2023-11-06T02:11:59.432 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.428+0000 7f1ba4cd2640 1 -- 172.21.15.49:0/3378080101 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b8c0047d0 msgr2=0x7f1b8c004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:59.432 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.428+0000 7f1ba4cd2640 1 --2- 172.21.15.49:0/3378080101 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b8c0047d0 0x7f1b8c004c50 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f1b90009ae0 tx=0x7f1b9001ca20 comp rx=0 tx=0).stop 2023-11-06T02:11:59.432 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.428+0000 7f1ba4cd2640 1 -- 172.21.15.49:0/3378080101 shutdown_connections 2023-11-06T02:11:59.432 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.428+0000 7f1ba4cd2640 1 --2- 172.21.15.49:0/3378080101 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b8c0047d0 0x7f1b8c004c50 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:59.433 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.428+0000 7f1ba4cd2640 1 -- 172.21.15.49:0/3378080101 >> 172.21.15.49:0/3378080101 conn(0x7f1ba00714a0 msgr2=0x7f1ba00718b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:59.433 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.428+0000 7f1ba4cd2640 1 -- 172.21.15.49:0/3378080101 shutdown_connections 2023-11-06T02:11:59.433 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.428+0000 7f1ba4cd2640 1 -- 172.21.15.49:0/3378080101 wait complete. 2023-11-06T02:11:59.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.428+0000 7f1ba4cd2640 1 Processor -- start 2023-11-06T02:11:59.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.428+0000 7f1ba4cd2640 1 -- start start 2023-11-06T02:11:59.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.432+0000 7f1ba4cd2640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b8c0047d0 0x7f1ba00b4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:59.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.432+0000 7f1ba4cd2640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1ba00b49b0 0x7f1ba01f5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:59.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.432+0000 7f1ba4cd2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ba01f5950 0x7f1ba01f8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:59.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.432+0000 7f1ba4cd2640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba00aa420 con 0x7f1ba01f5950 2023-11-06T02:11:59.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.432+0000 7f1ba4cd2640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba00a97a0 con 0x7f1b8c0047d0 2023-11-06T02:11:59.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.432+0000 7f1ba4cd2640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba00a9a80 con 0x7f1ba00b49b0 2023-11-06T02:11:59.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.432+0000 7f1b9ed9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ba01f5950 0x7f1ba01f8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:59.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.432+0000 7f1b9ed9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ba01f5950 0x7f1ba01f8dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39820/0 (socket says 172.21.15.49:39820) 2023-11-06T02:11:59.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.432+0000 7f1b9ed9d640 1 -- 172.21.15.49:0/1801994781 learned_addr learned my addr 172.21.15.49:0/1801994781 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:59.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.432+0000 7f1b9dd9b640 1 --2- 172.21.15.49:0/1801994781 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1ba00b49b0 0x7f1ba01f5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:59.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.432+0000 7f1b9e59c640 1 --2- 172.21.15.49:0/1801994781 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b8c0047d0 0x7f1ba00b4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:59.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.432+0000 7f1b9ed9d640 1 -- 172.21.15.49:0/1801994781 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1ba00b49b0 msgr2=0x7f1ba01f5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:59.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.432+0000 7f1b9ed9d640 1 --2- 172.21.15.49:0/1801994781 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1ba00b49b0 0x7f1ba01f5410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:59.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.432+0000 7f1b9ed9d640 1 -- 172.21.15.49:0/1801994781 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b8c0047d0 msgr2=0x7f1ba00b4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:59.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.432+0000 7f1b9ed9d640 1 --2- 172.21.15.49:0/1801994781 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b8c0047d0 0x7f1ba00b4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:59.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.432+0000 7f1b9ed9d640 1 -- 172.21.15.49:0/1801994781 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ba00a9d60 con 0x7f1ba01f5950 2023-11-06T02:11:59.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.432+0000 7f1b9e59c640 1 --2- 172.21.15.49:0/1801994781 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b8c0047d0 0x7f1ba00b4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:11:59.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.432+0000 7f1b9ed9d640 1 --2- 172.21.15.49:0/1801994781 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ba01f5950 0x7f1ba01f8dc0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f1ba00b5c20 tx=0x7f1b9800e9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:59.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.432+0000 7f1b96ffd640 1 -- 172.21.15.49:0/1801994781 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b98006e30 con 0x7f1ba01f5950 2023-11-06T02:11:59.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.432+0000 7f1b96ffd640 1 -- 172.21.15.49:0/1801994781 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b98003980 con 0x7f1ba01f5950 2023-11-06T02:11:59.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.432+0000 7f1b96ffd640 1 -- 172.21.15.49:0/1801994781 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b98003d30 con 0x7f1ba01f5950 2023-11-06T02:11:59.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.432+0000 7f1ba4cd2640 1 -- 172.21.15.49:0/1801994781 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ba00aa040 con 0x7f1ba01f5950 2023-11-06T02:11:59.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.432+0000 7f1ba4cd2640 1 -- 172.21.15.49:0/1801994781 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ba01f9300 con 0x7f1ba01f5950 2023-11-06T02:11:59.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.432+0000 7f1ba4cd2640 1 -- 172.21.15.49:0/1801994781 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ba00aa560 con 0x7f1ba01f5950 2023-11-06T02:11:59.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.432+0000 7f1b96ffd640 1 -- 172.21.15.49:0/1801994781 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f1b98005110 con 0x7f1ba01f5950 2023-11-06T02:11:59.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.432+0000 7f1b96ffd640 1 --2- 172.21.15.49:0/1801994781 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1b80065a60 0x7f1b80067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:59.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.432+0000 7f1b96ffd640 1 -- 172.21.15.49:0/1801994781 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1b98082ad0 con 0x7f1ba01f5950 2023-11-06T02:11:59.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.432+0000 7f1b9e59c640 1 --2- 172.21.15.49:0/1801994781 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1b80065a60 0x7f1b80067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:59.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.432+0000 7f1b9e59c640 1 --2- 172.21.15.49:0/1801994781 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1b80065a60 0x7f1b80067f20 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f1b88001e30 tx=0x7f1b88001020 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:59.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.436+0000 7f1b96ffd640 1 -- 172.21.15.49:0/1801994781 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f1ba00aa560 con 0x7f1ba01f5950 2023-11-06T02:11:59.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.596+0000 7f1ba4cd2640 1 -- 172.21.15.49:0/1801994781 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1ba00aa560 con 0x7f1ba01f5950 2023-11-06T02:11:59.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.596+0000 7f1b96ffd640 1 -- 172.21.15.49:0/1801994781 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+17825 (secure 0 0 0) 0x7f1ba00aa560 con 0x7f1ba01f5950 2023-11-06T02:11:59.597 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:11:59.597 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":65,"fsid":"1a8b035c-6ba4-499a-b876-aea47fd9c0d4","created":"2023-11-06T02:08:06.440857+0000","modified":"2023-11-06T02:11:31.094207+0000","last_up_change":"2023-11-06T02:08:17.982372+0000","last_in_change":"2023-11-06T02:08:08.906631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T02:08:19.252222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T02:08:24.776211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-06T02:11:28.648326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-06T02:11:29.079513+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ad55af8-0571-4e6a-8722-b5d429b7496d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6809","nonce":38116},{"type":"v1","addr":"172.21.15.49:6811","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6813","nonce":38116},{"type":"v1","addr":"172.21.15.49:6815","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6821","nonce":38116},{"type":"v1","addr":"172.21.15.49:6823","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6817","nonce":38116},{"type":"v1","addr":"172.21.15.49:6819","nonce":38116}]},"public_addr":"172.21.15.49:6811/38116","cluster_addr":"172.21.15.49:6815/38116","heartbeat_back_addr":"172.21.15.49:6823/38116","heartbeat_front_addr":"172.21.15.49:6819/38116","state":["exists","up"]},{"osd":1,"uuid":"c8b31f85-a757-43ed-9419-351acc46611b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":38113},{"type":"v1","addr":"172.21.15.49:6810","nonce":38113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":38113},{"type":"v1","addr":"172.21.15.49:6814","nonce":38113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":38113},{"type":"v1","addr":"172.21.15.49:6822","nonce":38113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":38113},{"type":"v1","addr":"172.21.15.49:6818","nonce":38113}]},"public_addr":"172.21.15.49:6810/38113","cluster_addr":"172.21.15.49:6814/38113","heartbeat_back_addr":"172.21.15.49:6822/38113","heartbeat_front_addr":"172.21.15.49:6818/38113","state":["exists","up"]},{"osd":2,"uuid":"555b3bf3-0a70-4b91-8efd-e4204b5fc26e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":38114},{"type":"v1","addr":"172.21.15.49:6801","nonce":38114}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":38114},{"type":"v1","addr":"172.21.15.49:6803","nonce":38114}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":38114},{"type":"v1","addr":"172.21.15.49:6807","nonce":38114}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":38114},{"type":"v1","addr":"172.21.15.49:6805","nonce":38114}]},"public_addr":"172.21.15.49:6801/38114","cluster_addr":"172.21.15.49:6803/38114","heartbeat_back_addr":"172.21.15.49:6807/38114","heartbeat_front_addr":"172.21.15.49:6805/38114","state":["exists","up"]},{"osd":3,"uuid":"8d4e5d77-03ef-4b9b-bfcb-5ae93a46795c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":38115},{"type":"v1","addr":"172.21.15.49:6825","nonce":38115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":38115},{"type":"v1","addr":"172.21.15.49:6827","nonce":38115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":38115},{"type":"v1","addr":"172.21.15.49:6831","nonce":38115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":38115},{"type":"v1","addr":"172.21.15.49:6829","nonce":38115}]},"public_addr":"172.21.15.49:6825/38115","cluster_addr":"172.21.15.49:6827/38115","heartbeat_back_addr":"172.21.15.49:6831/38115","heartbeat_front_addr":"172.21.15.49:6829/38115","state":["exists","up"]},{"osd":4,"uuid":"dba03ad4-68a3-4cf3-9218-ac611f02eacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":38417},{"type":"v1","addr":"172.21.15.150:6809","nonce":38417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":38417},{"type":"v1","addr":"172.21.15.150:6811","nonce":38417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":38417},{"type":"v1","addr":"172.21.15.150:6815","nonce":38417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":38417},{"type":"v1","addr":"172.21.15.150:6813","nonce":38417}]},"public_addr":"172.21.15.150:6809/38417","cluster_addr":"172.21.15.150:6811/38417","heartbeat_back_addr":"172.21.15.150:6815/38417","heartbeat_front_addr":"172.21.15.150:6813/38417","state":["exists","up"]},{"osd":5,"uuid":"0df9c303-c255-4640-8e91-01f1c3efe18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":38418},{"type":"v1","addr":"172.21.15.150:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":38418},{"type":"v1","addr":"172.21.15.150:6803","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":38418},{"type":"v1","addr":"172.21.15.150:6807","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":38418},{"type":"v1","addr":"172.21.15.150:6805","nonce":38418}]},"public_addr":"172.21.15.150:6801/38418","cluster_addr":"172.21.15.150:6803/38418","heartbeat_back_addr":"172.21.15.150:6807/38418","heartbeat_front_addr":"172.21.15.150:6805/38418","state":["exists","up"]},{"osd":6,"uuid":"e59fa189-4b70-4c06-9722-c02b90828bf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":38428},{"type":"v1","addr":"172.21.15.150:6825","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":38428},{"type":"v1","addr":"172.21.15.150:6827","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":38428},{"type":"v1","addr":"172.21.15.150:6831","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":38428},{"type":"v1","addr":"172.21.15.150:6829","nonce":38428}]},"public_addr":"172.21.15.150:6825/38428","cluster_addr":"172.21.15.150:6827/38428","heartbeat_back_addr":"172.21.15.150:6831/38428","heartbeat_front_addr":"172.21.15.150:6829/38428","state":["exists","up"]},{"osd":7,"uuid":"a64f57e1-ef98-46bd-b304-385759e95898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":38421},{"type":"v1","addr":"172.21.15.150:6817","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":38421},{"type":"v1","addr":"172.21.15.150:6819","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":38421},{"type":"v1","addr":"172.21.15.150:6823","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":38421},{"type":"v1","addr":"172.21.15.150:6821","nonce":38421}]},"public_addr":"172.21.15.150:6817/38421","cluster_addr":"172.21.15.150:6819/38421","heartbeat_back_addr":"172.21.15.150:6823/38421","heartbeat_front_addr":"172.21.15.150:6821/38421","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.438552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.413280+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.296466+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.424966+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.276953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.230984+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.410235+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.480084+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T02:11:59.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.600+0000 7f1ba4cd2640 1 -- 172.21.15.49:0/1801994781 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1b80065a60 msgr2=0x7f1b80067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:59.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.600+0000 7f1ba4cd2640 1 --2- 172.21.15.49:0/1801994781 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1b80065a60 0x7f1b80067f20 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f1b88001e30 tx=0x7f1b88001020 comp rx=0 tx=0).stop 2023-11-06T02:11:59.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.600+0000 7f1ba4cd2640 1 -- 172.21.15.49:0/1801994781 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ba01f5950 msgr2=0x7f1ba01f8dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:59.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.600+0000 7f1ba4cd2640 1 --2- 172.21.15.49:0/1801994781 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ba01f5950 0x7f1ba01f8dc0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f1ba00b5c20 tx=0x7f1b9800e9e0 comp rx=0 tx=0).stop 2023-11-06T02:11:59.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.600+0000 7f1ba4cd2640 1 -- 172.21.15.49:0/1801994781 shutdown_connections 2023-11-06T02:11:59.600 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.600+0000 7f1ba4cd2640 1 --2- 172.21.15.49:0/1801994781 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1b80065a60 0x7f1b80067f20 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:59.600 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.600+0000 7f1ba4cd2640 1 --2- 172.21.15.49:0/1801994781 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ba01f5950 0x7f1ba01f8dc0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:59.600 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.600+0000 7f1ba4cd2640 1 --2- 172.21.15.49:0/1801994781 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1ba00b49b0 0x7f1ba01f5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:59.600 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.600+0000 7f1ba4cd2640 1 --2- 172.21.15.49:0/1801994781 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b8c0047d0 0x7f1ba00b4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:59.600 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.600+0000 7f1ba4cd2640 1 -- 172.21.15.49:0/1801994781 >> 172.21.15.49:0/1801994781 conn(0x7f1ba00714a0 msgr2=0x7f1ba00d32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:59.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.600+0000 7f1ba4cd2640 1 -- 172.21.15.49:0/1801994781 shutdown_connections 2023-11-06T02:11:59.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.600+0000 7f1ba4cd2640 1 -- 172.21.15.49:0/1801994781 wait complete. 2023-11-06T02:11:59.616 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T02:08:19.252222+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T02:08:24.776211+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T02:11:28.648326+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T02:11:29.079513+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '65', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T02:11:59.617 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T02:11:59.745 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.748+0000 7fdcfb8cd640 1 Processor -- start 2023-11-06T02:11:59.745 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.748+0000 7fdcfb8cd640 1 -- start start 2023-11-06T02:11:59.746 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.748+0000 7fdcfb8cd640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fdcf40d2ed0 0x7fdcf40b6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:59.746 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.748+0000 7fdcfb8cd640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcf415cf90 con 0x7fdcf415cbb0 2023-11-06T02:11:59.747 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.748+0000 7fdcfb8cd640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcf40d3360 con 0x7fdcf40d2a60 2023-11-06T02:11:59.747 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.748+0000 7fdcfb8cd640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcf40d0b80 con 0x7fdcf40d2ed0 2023-11-06T02:11:59.747 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.748+0000 7fdcf9e6a640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fdcf40d2ed0 0x7fdcf40b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:59.747 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.748+0000 7fdcf9669640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fdcf415cbb0 0x7fdcf40d1fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:54726/0 (socket says 172.21.15.49:54726) 2023-11-06T02:11:59.747 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.748+0000 7fdcf9669640 1 -- 172.21.15.49:0/3919474861 learned_addr learned my addr 172.21.15.49:0/3919474861 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:11:59.748 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.748+0000 7fdcf9e6a640 1 -- 172.21.15.49:0/3919474861 >> v1:172.21.15.150:6789/0 conn(0x7fdcf40d2a60 legacy=0x7fdcf40b5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:59.748 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.748+0000 7fdcf9e6a640 1 -- 172.21.15.49:0/3919474861 >> v1:172.21.15.49:6789/0 conn(0x7fdcf415cbb0 legacy=0x7fdcf40d1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:59.748 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.748+0000 7fdcf9e6a640 1 -- 172.21.15.49:0/3919474861 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcf40d0e60 con 0x7fdcf40d2ed0 2023-11-06T02:11:59.749 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.748+0000 7fdcebfff640 1 -- 172.21.15.49:0/3919474861 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2793820382 0 0) 0x7fdcf415cf90 con 0x7fdcf415cbb0 2023-11-06T02:11:59.750 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.752+0000 7fdcf9e6a640 1 --2- 172.21.15.49:0/3919474861 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcf40d2ed0 0x7fdcf40b6450 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fdcec009d90 tx=0x7fdcec01f130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7fcf99222d8e5c4 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:59.750 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.752+0000 7fdcebfff640 1 -- 172.21.15.49:0/3919474861 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcec0af600 con 0x7fdcf40d2ed0 2023-11-06T02:11:59.750 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.752+0000 7fdcebfff640 1 -- 172.21.15.49:0/3919474861 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcec0ab050 con 0x7fdcf40d2ed0 2023-11-06T02:11:59.751 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.752+0000 7fdcebfff640 1 -- 172.21.15.49:0/3919474861 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcec0af980 con 0x7fdcf40d2ed0 2023-11-06T02:11:59.751 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.752+0000 7fdcfb8cd640 1 -- 172.21.15.49:0/3919474861 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcf40d2ed0 msgr2=0x7fdcf40b6450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:59.751 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.752+0000 7fdcfb8cd640 1 --2- 172.21.15.49:0/3919474861 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcf40d2ed0 0x7fdcf40b6450 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fdcec009d90 tx=0x7fdcec01f130 comp rx=0 tx=0).stop 2023-11-06T02:11:59.751 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.752+0000 7fdcfb8cd640 1 -- 172.21.15.49:0/3919474861 shutdown_connections 2023-11-06T02:11:59.751 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.752+0000 7fdcfb8cd640 1 --2- 172.21.15.49:0/3919474861 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcf40d2ed0 0x7fdcf40b6450 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:59.752 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.752+0000 7fdcfb8cd640 1 -- 172.21.15.49:0/3919474861 >> 172.21.15.49:0/3919474861 conn(0x7fdcf40718a0 msgr2=0x7fdcf4071cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:59.752 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.752+0000 7fdcfb8cd640 1 -- 172.21.15.49:0/3919474861 shutdown_connections 2023-11-06T02:11:59.752 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.752+0000 7fdcfb8cd640 1 -- 172.21.15.49:0/3919474861 wait complete. 2023-11-06T02:11:59.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.752+0000 7fdcfb8cd640 1 Processor -- start 2023-11-06T02:11:59.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.752+0000 7fdcfb8cd640 1 -- start start 2023-11-06T02:11:59.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.752+0000 7fdcfb8cd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcf40d2a60 0x7fdcf41f2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:59.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.752+0000 7fdcfb8cd640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdcf40d2ed0 0x7fdcf415d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:59.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.752+0000 7fdcfb8cd640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdcf415cbb0 0x7fdcf41f1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:59.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.752+0000 7fdcfb8cd640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf41f3190 con 0x7fdcf40d2a60 2023-11-06T02:11:59.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.752+0000 7fdcfb8cd640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf41f3380 con 0x7fdcf40d2ed0 2023-11-06T02:11:59.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.752+0000 7fdcfb8cd640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf40af050 con 0x7fdcf415cbb0 2023-11-06T02:11:59.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.752+0000 7fdcf9669640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcf40d2a60 0x7fdcf41f2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:59.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.752+0000 7fdcf9669640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcf40d2a60 0x7fdcf41f2c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39850/0 (socket says 172.21.15.49:39850) 2023-11-06T02:11:59.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.752+0000 7fdcf9669640 1 -- 172.21.15.49:0/4014891473 learned_addr learned my addr 172.21.15.49:0/4014891473 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:11:59.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.752+0000 7fdcf8e68640 1 --2- 172.21.15.49:0/4014891473 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdcf40d2ed0 0x7fdcf415d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:59.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.752+0000 7fdcf9e6a640 1 --2- 172.21.15.49:0/4014891473 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdcf415cbb0 0x7fdcf41f1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:59.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.752+0000 7fdcf9669640 1 -- 172.21.15.49:0/4014891473 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdcf415cbb0 msgr2=0x7fdcf41f1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:59.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.752+0000 7fdcf9669640 1 --2- 172.21.15.49:0/4014891473 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdcf415cbb0 0x7fdcf41f1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:59.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.752+0000 7fdcf9669640 1 -- 172.21.15.49:0/4014891473 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdcf40d2ed0 msgr2=0x7fdcf415d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:59.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.752+0000 7fdcf9669640 1 --2- 172.21.15.49:0/4014891473 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdcf40d2ed0 0x7fdcf415d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:59.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.752+0000 7fdcf9669640 1 -- 172.21.15.49:0/4014891473 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcec009a70 con 0x7fdcf40d2a60 2023-11-06T02:11:59.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.752+0000 7fdcf9669640 1 --2- 172.21.15.49:0/4014891473 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcf40d2a60 0x7fdcf41f2c50 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fdcf000c4d0 tx=0x7fdcf000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:59.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.752+0000 7fdce9ffb640 1 -- 172.21.15.49:0/4014891473 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf00074e0 con 0x7fdcf40d2a60 2023-11-06T02:11:59.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.752+0000 7fdce9ffb640 1 -- 172.21.15.49:0/4014891473 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcf0007640 con 0x7fdcf40d2a60 2023-11-06T02:11:59.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.752+0000 7fdcfb8cd640 1 -- 172.21.15.49:0/4014891473 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcf41f5d10 con 0x7fdcf40d2a60 2023-11-06T02:11:59.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.752+0000 7fdce9ffb640 1 -- 172.21.15.49:0/4014891473 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf0004a30 con 0x7fdcf40d2a60 2023-11-06T02:11:59.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.752+0000 7fdcfb8cd640 1 -- 172.21.15.49:0/4014891473 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcf41f62d0 con 0x7fdcf40d2a60 2023-11-06T02:11:59.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.756+0000 7fdcfb8cd640 1 -- 172.21.15.49:0/4014891473 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcf41f3550 con 0x7fdcf40d2a60 2023-11-06T02:11:59.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.756+0000 7fdce9ffb640 1 -- 172.21.15.49:0/4014891473 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fdcf0004b90 con 0x7fdcf40d2a60 2023-11-06T02:11:59.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.756+0000 7fdce9ffb640 1 --2- 172.21.15.49:0/4014891473 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fdcd8065a60 0x7fdcd8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:11:59.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.756+0000 7fdce9ffb640 1 -- 172.21.15.49:0/4014891473 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdcf00825b0 con 0x7fdcf40d2a60 2023-11-06T02:11:59.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.756+0000 7fdcf8e68640 1 --2- 172.21.15.49:0/4014891473 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fdcd8065a60 0x7fdcd8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:11:59.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.756+0000 7fdcf8e68640 1 --2- 172.21.15.49:0/4014891473 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fdcd8065a60 0x7fdcd8067f20 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fdcf40aec90 tx=0x7fdce4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:11:59.762 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.760+0000 7fdce9ffb640 1 -- 172.21.15.49:0/4014891473 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fdcf41f3550 con 0x7fdcf40d2a60 2023-11-06T02:11:59.918 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.916+0000 7fdcfb8cd640 1 -- 172.21.15.49:0/4014891473 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdcf41f3550 con 0x7fdcf40d2a60 2023-11-06T02:11:59.918 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.916+0000 7fdce9ffb640 1 -- 172.21.15.49:0/4014891473 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v65) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdcf41f3550 con 0x7fdcf40d2a60 2023-11-06T02:11:59.918 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2023-11-06T02:11:59.919 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.920+0000 7fdcfb8cd640 1 -- 172.21.15.49:0/4014891473 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fdcd8065a60 msgr2=0x7fdcd8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:59.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.920+0000 7fdcfb8cd640 1 --2- 172.21.15.49:0/4014891473 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fdcd8065a60 0x7fdcd8067f20 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fdcf40aec90 tx=0x7fdce4009040 comp rx=0 tx=0).stop 2023-11-06T02:11:59.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.920+0000 7fdcfb8cd640 1 -- 172.21.15.49:0/4014891473 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcf40d2a60 msgr2=0x7fdcf41f2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:11:59.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.920+0000 7fdcfb8cd640 1 --2- 172.21.15.49:0/4014891473 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcf40d2a60 0x7fdcf41f2c50 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fdcf000c4d0 tx=0x7fdcf000c990 comp rx=0 tx=0).stop 2023-11-06T02:11:59.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.920+0000 7fdcfb8cd640 1 -- 172.21.15.49:0/4014891473 shutdown_connections 2023-11-06T02:11:59.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.920+0000 7fdcfb8cd640 1 --2- 172.21.15.49:0/4014891473 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fdcd8065a60 0x7fdcd8067f20 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:59.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.920+0000 7fdcfb8cd640 1 --2- 172.21.15.49:0/4014891473 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdcf415cbb0 0x7fdcf41f1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:59.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.920+0000 7fdcfb8cd640 1 --2- 172.21.15.49:0/4014891473 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdcf40d2ed0 0x7fdcf415d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:59.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.920+0000 7fdcfb8cd640 1 --2- 172.21.15.49:0/4014891473 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcf40d2a60 0x7fdcf41f2c50 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:11:59.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.920+0000 7fdcfb8cd640 1 -- 172.21.15.49:0/4014891473 >> 172.21.15.49:0/4014891473 conn(0x7fdcf40718a0 msgr2=0x7fdcf415c190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:11:59.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.920+0000 7fdcfb8cd640 1 -- 172.21.15.49:0/4014891473 shutdown_connections 2023-11-06T02:11:59.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:11:59.920+0000 7fdcfb8cd640 1 -- 172.21.15.49:0/4014891473 wait complete. 2023-11-06T02:11:59.936 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T02:12:00.069 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.072+0000 7f7cc6300640 1 Processor -- start 2023-11-06T02:12:00.070 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.072+0000 7f7cc6300640 1 -- start start 2023-11-06T02:12:00.071 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.072+0000 7f7cc6300640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f7cc006a480 0x7f7cc0164f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:00.071 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.072+0000 7f7cc6300640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cc0159a00 con 0x7f7cc0065930 2023-11-06T02:12:00.072 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.072+0000 7f7cc6300640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cc006a950 con 0x7f7cc00c9d10 2023-11-06T02:12:00.072 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.072+0000 7f7cc6300640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cc006ac30 con 0x7f7cc006a480 2023-11-06T02:12:00.072 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.072+0000 7f7cbffff640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f7cc0065930 0x7f7cc00c9600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:54736/0 (socket says 172.21.15.49:54736) 2023-11-06T02:12:00.072 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.072+0000 7f7cbffff640 1 -- 172.21.15.49:0/2751892017 learned_addr learned my addr 172.21.15.49:0/2751892017 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:12:00.073 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.072+0000 7f7cc489d640 1 --2- 172.21.15.49:0/2751892017 >> v2:172.21.15.49:3300/0 conn(0x7f7cc006a480 0x7f7cc0164f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:00.073 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.072+0000 7f7cbeffd640 1 -- 172.21.15.49:0/2751892017 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3146761258 0 0) 0x7f7cc0159a00 con 0x7f7cc0065930 2023-11-06T02:12:00.073 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.072+0000 7f7cbeffd640 1 -- 172.21.15.49:0/2751892017 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7cac003530 con 0x7f7cc0065930 2023-11-06T02:12:00.073 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.072+0000 7f7cc489d640 1 -- 172.21.15.49:0/2751892017 >> v1:172.21.15.150:6789/0 conn(0x7f7cc00c9d10 legacy=0x7f7cc0069d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:00.073 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.072+0000 7f7cc489d640 1 -- 172.21.15.49:0/2751892017 >> v1:172.21.15.49:6789/0 conn(0x7f7cc0065930 legacy=0x7f7cc00c9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:00.074 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.072+0000 7f7cc489d640 1 -- 172.21.15.49:0/2751892017 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cc006af10 con 0x7f7cc006a480 2023-11-06T02:12:00.075 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.076+0000 7f7cc489d640 1 --2- 172.21.15.49:0/2751892017 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7cc006a480 0x7f7cc0164f20 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f7cb80097c0 tx=0x7f7cb801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9872165df26ef916 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:00.075 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.076+0000 7f7cbeffd640 1 -- 172.21.15.49:0/2751892017 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb80ab650 con 0x7f7cc006a480 2023-11-06T02:12:00.076 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.076+0000 7f7cbeffd640 1 -- 172.21.15.49:0/2751892017 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cb80ab7b0 con 0x7f7cc006a480 2023-11-06T02:12:00.076 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.076+0000 7f7cbeffd640 1 -- 172.21.15.49:0/2751892017 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb80abb20 con 0x7f7cc006a480 2023-11-06T02:12:00.076 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.076+0000 7f7cc6300640 1 -- 172.21.15.49:0/2751892017 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7cc006a480 msgr2=0x7f7cc0164f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:00.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.076+0000 7f7cc6300640 1 --2- 172.21.15.49:0/2751892017 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7cc006a480 0x7f7cc0164f20 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f7cb80097c0 tx=0x7f7cb801c450 comp rx=0 tx=0).stop 2023-11-06T02:12:00.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.076+0000 7f7cc6300640 1 -- 172.21.15.49:0/2751892017 shutdown_connections 2023-11-06T02:12:00.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.076+0000 7f7cc6300640 1 --2- 172.21.15.49:0/2751892017 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7cc006a480 0x7f7cc0164f20 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:00.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.076+0000 7f7cc6300640 1 -- 172.21.15.49:0/2751892017 >> 172.21.15.49:0/2751892017 conn(0x7f7cc00b0aa0 msgr2=0x7f7cc00b0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:12:00.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.076+0000 7f7cc6300640 1 -- 172.21.15.49:0/2751892017 shutdown_connections 2023-11-06T02:12:00.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.076+0000 7f7cc6300640 1 -- 172.21.15.49:0/2751892017 wait complete. 2023-11-06T02:12:00.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.076+0000 7f7cc6300640 1 Processor -- start 2023-11-06T02:12:00.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.076+0000 7f7cc6300640 1 -- start start 2023-11-06T02:12:00.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.076+0000 7f7cc6300640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7cc0065930 0x7f7cc0152780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:00.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.076+0000 7f7cc6300640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7cc006a480 0x7f7cc0071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:00.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.076+0000 7f7cc6300640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7cc00c9d10 0x7f7cc00716d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:00.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.076+0000 7f7cc6300640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc01544a0 con 0x7f7cc00c9d10 2023-11-06T02:12:00.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.076+0000 7f7cc6300640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc00c15c0 con 0x7f7cc006a480 2023-11-06T02:12:00.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.076+0000 7f7cc6300640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc0152cc0 con 0x7f7cc0065930 2023-11-06T02:12:00.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.076+0000 7f7cc489d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7cc00c9d10 0x7f7cc00716d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:00.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.076+0000 7f7cc489d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7cc00c9d10 0x7f7cc00716d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39878/0 (socket says 172.21.15.49:39878) 2023-11-06T02:12:00.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.076+0000 7f7cc489d640 1 -- 172.21.15.49:0/1229972000 learned_addr learned my addr 172.21.15.49:0/1229972000 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:12:00.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.076+0000 7f7cbf7fe640 1 --2- 172.21.15.49:0/1229972000 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7cc006a480 0x7f7cc0071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:00.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.080+0000 7f7cbffff640 1 --2- 172.21.15.49:0/1229972000 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7cc0065930 0x7f7cc0152780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:00.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.080+0000 7f7cc489d640 1 -- 172.21.15.49:0/1229972000 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7cc0065930 msgr2=0x7f7cc0152780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:00.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.080+0000 7f7cc489d640 1 --2- 172.21.15.49:0/1229972000 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7cc0065930 0x7f7cc0152780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:00.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.080+0000 7f7cc489d640 1 -- 172.21.15.49:0/1229972000 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7cc006a480 msgr2=0x7f7cc0071190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:00.085 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.080+0000 7f7cc489d640 1 --2- 172.21.15.49:0/1229972000 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7cc006a480 0x7f7cc0071190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:00.085 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.080+0000 7f7cc489d640 1 -- 172.21.15.49:0/1229972000 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cb8009420 con 0x7f7cc00c9d10 2023-11-06T02:12:00.085 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.080+0000 7f7cbf7fe640 1 --2- 172.21.15.49:0/1229972000 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7cc006a480 0x7f7cc0071190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:12:00.085 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.080+0000 7f7cc489d640 1 --2- 172.21.15.49:0/1229972000 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7cc00c9d10 0x7f7cc00716d0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f7cb80097c0 tx=0x7f7cb801ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:00.086 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.080+0000 7f7cbcff9640 1 -- 172.21.15.49:0/1229972000 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb8004230 con 0x7f7cc00c9d10 2023-11-06T02:12:00.086 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.080+0000 7f7cbcff9640 1 -- 172.21.15.49:0/1229972000 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cb80b9070 con 0x7f7cc00c9d10 2023-11-06T02:12:00.086 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.080+0000 7f7cc6300640 1 -- 172.21.15.49:0/1229972000 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cc00c1830 con 0x7f7cc00c9d10 2023-11-06T02:12:00.087 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.080+0000 7f7cbcff9640 1 -- 172.21.15.49:0/1229972000 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb8003620 con 0x7f7cc00c9d10 2023-11-06T02:12:00.087 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.080+0000 7f7cc6300640 1 -- 172.21.15.49:0/1229972000 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cc00c1df0 con 0x7f7cc00c9d10 2023-11-06T02:12:00.087 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.080+0000 7f7cc6300640 1 -- 172.21.15.49:0/1229972000 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cc00c1fe0 con 0x7f7cc00c9d10 2023-11-06T02:12:00.087 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.080+0000 7f7cbcff9640 1 -- 172.21.15.49:0/1229972000 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f7cb8003780 con 0x7f7cc00c9d10 2023-11-06T02:12:00.087 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.080+0000 7f7cbcff9640 1 --2- 172.21.15.49:0/1229972000 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7ca0065a60 0x7f7ca0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:00.088 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.080+0000 7f7cbcff9640 1 -- 172.21.15.49:0/1229972000 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7cb80bb070 con 0x7f7cc00c9d10 2023-11-06T02:12:00.088 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.080+0000 7f7cbffff640 1 --2- 172.21.15.49:0/1229972000 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7ca0065a60 0x7f7ca0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:00.088 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.080+0000 7f7cbffff640 1 --2- 172.21.15.49:0/1229972000 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7ca0065a60 0x7f7ca0067f20 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f7cb0001570 tx=0x7f7cb0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:00.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.084+0000 7f7cbcff9640 1 -- 172.21.15.49:0/1229972000 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f7cb80eb9d0 con 0x7f7cc00c9d10 2023-11-06T02:12:00.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.240+0000 7f7cc6300640 1 -- 172.21.15.49:0/1229972000 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f7cc00b18d0 con 0x7f7cc00c9d10 2023-11-06T02:12:00.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.244+0000 7f7cbcff9640 1 -- 172.21.15.49:0/1229972000 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v65) v1 ==== 92+0+10 (secure 0 0 0) 0x7f7cb80c4020 con 0x7f7cc00c9d10 2023-11-06T02:12:00.241 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 8 2023-11-06T02:12:00.243 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.244+0000 7f7cc6300640 1 -- 172.21.15.49:0/1229972000 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7ca0065a60 msgr2=0x7f7ca0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:00.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.244+0000 7f7cc6300640 1 --2- 172.21.15.49:0/1229972000 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7ca0065a60 0x7f7ca0067f20 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f7cb0001570 tx=0x7f7cb0009040 comp rx=0 tx=0).stop 2023-11-06T02:12:00.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.244+0000 7f7cc6300640 1 -- 172.21.15.49:0/1229972000 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7cc00c9d10 msgr2=0x7f7cc00716d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:00.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.244+0000 7f7cc6300640 1 --2- 172.21.15.49:0/1229972000 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7cc00c9d10 0x7f7cc00716d0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f7cb80097c0 tx=0x7f7cb801ac90 comp rx=0 tx=0).stop 2023-11-06T02:12:00.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.244+0000 7f7cc6300640 1 -- 172.21.15.49:0/1229972000 shutdown_connections 2023-11-06T02:12:00.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.244+0000 7f7cc6300640 1 --2- 172.21.15.49:0/1229972000 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7ca0065a60 0x7f7ca0067f20 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:00.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.244+0000 7f7cc6300640 1 --2- 172.21.15.49:0/1229972000 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7cc00c9d10 0x7f7cc00716d0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:00.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.244+0000 7f7cc6300640 1 --2- 172.21.15.49:0/1229972000 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7cc006a480 0x7f7cc0071190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:00.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.244+0000 7f7cc6300640 1 --2- 172.21.15.49:0/1229972000 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7cc0065930 0x7f7cc0152780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:00.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.244+0000 7f7cc6300640 1 -- 172.21.15.49:0/1229972000 >> 172.21.15.49:0/1229972000 conn(0x7f7cc00b0aa0 msgr2=0x7f7cc006df00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:12:00.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.244+0000 7f7cc6300640 1 -- 172.21.15.49:0/1229972000 shutdown_connections 2023-11-06T02:12:00.246 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.244+0000 7f7cc6300640 1 -- 172.21.15.49:0/1229972000 wait complete. 2023-11-06T02:12:00.260 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T02:12:00.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.396+0000 7fbc628ea640 1 Processor -- start 2023-11-06T02:12:00.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.396+0000 7fbc628ea640 1 -- start start 2023-11-06T02:12:00.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.396+0000 7fbc628ea640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbc5c15e0c0 0x7fbc5c0ce840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:00.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.396+0000 7fbc628ea640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fbc5c15e4a0 0x7fbc5c0cdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:00.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.396+0000 7fbc628ea640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc5c0ce230 con 0x7fbc5c0b16e0 2023-11-06T02:12:00.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.396+0000 7fbc628ea640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc5c0cd8d0 con 0x7fbc5c15e4a0 2023-11-06T02:12:00.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.396+0000 7fbc628ea640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc5c0d1430 con 0x7fbc5c15e0c0 2023-11-06T02:12:00.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.396+0000 7fbc60e87640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fbc5c0b16e0 0x7fbc5c0b0100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:54752/0 (socket says 172.21.15.49:54752) 2023-11-06T02:12:00.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.396+0000 7fbc60e87640 1 -- 172.21.15.49:0/119187616 learned_addr learned my addr 172.21.15.49:0/119187616 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:12:00.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.396+0000 7fbc5b7fe640 1 --2- 172.21.15.49:0/119187616 >> v2:172.21.15.150:3300/0 conn(0x7fbc5c15e4a0 0x7fbc5c0cdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:00.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.396+0000 7fbc5bfff640 1 --2- 172.21.15.49:0/119187616 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbc5c15e0c0 0x7fbc5c0ce840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:00.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.396+0000 7fbc5affd640 1 -- 172.21.15.49:0/119187616 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 648690628 0 0) 0x7fbc5c0ce230 con 0x7fbc5c0b16e0 2023-11-06T02:12:00.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.396+0000 7fbc5affd640 1 -- 172.21.15.49:0/119187616 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc44003530 con 0x7fbc5c0b16e0 2023-11-06T02:12:00.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.396+0000 7fbc5affd640 1 -- 172.21.15.49:0/119187616 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2429544532 0 0) 0x7fbc44003530 con 0x7fbc5c0b16e0 2023-11-06T02:12:00.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.396+0000 7fbc5affd640 1 -- 172.21.15.49:0/119187616 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbc5c15e0c0 msgr2=0x7fbc5c0ce840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:00.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.396+0000 7fbc5affd640 1 --2- 172.21.15.49:0/119187616 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbc5c15e0c0 0x7fbc5c0ce840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:00.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.396+0000 7fbc5affd640 1 -- 172.21.15.49:0/119187616 >> v2:172.21.15.150:3300/0 conn(0x7fbc5c15e4a0 msgr2=0x7fbc5c0cdcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:00.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.396+0000 7fbc5affd640 1 --2- 172.21.15.49:0/119187616 >> v2:172.21.15.150:3300/0 conn(0x7fbc5c15e4a0 0x7fbc5c0cdcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:00.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.396+0000 7fbc5affd640 1 -- 172.21.15.49:0/119187616 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc5c0d1710 con 0x7fbc5c0b16e0 2023-11-06T02:12:00.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.396+0000 7fbc5affd640 1 -- 172.21.15.49:0/119187616 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7fbc54001a00 con 0x7fbc5c0b16e0 2023-11-06T02:12:00.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.396+0000 7fbc5affd640 1 -- 172.21.15.49:0/119187616 >> v1:172.21.15.49:6789/0 conn(0x7fbc5c0b16e0 legacy=0x7fbc5c0b0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:00.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.396+0000 7fbc5affd640 1 --2- 172.21.15.49:0/119187616 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbc44003980 0x7fbc44003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:00.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.396+0000 7fbc5affd640 1 --2- 172.21.15.49:0/119187616 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbc5c15e0c0 0x7fbc440047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:00.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.396+0000 7fbc5affd640 1 --2- 172.21.15.49:0/119187616 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbc44004d10 0x7fbc44005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:00.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.396+0000 7fbc5affd640 1 -- 172.21.15.49:0/119187616 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc5c0cd8d0 con 0x7fbc44003980 2023-11-06T02:12:00.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.396+0000 7fbc5affd640 1 -- 172.21.15.49:0/119187616 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc5c0d1430 con 0x7fbc44004d10 2023-11-06T02:12:00.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.396+0000 7fbc5affd640 1 -- 172.21.15.49:0/119187616 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc4400b990 con 0x7fbc5c15e0c0 2023-11-06T02:12:00.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.396+0000 7fbc5bfff640 1 --2- 172.21.15.49:0/119187616 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbc44003980 0x7fbc44003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:00.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.396+0000 7fbc60e87640 1 --2- 172.21.15.49:0/119187616 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbc44004d10 0x7fbc44005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:00.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.396+0000 7fbc5b7fe640 1 --2- 172.21.15.49:0/119187616 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbc5c15e0c0 0x7fbc440047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:00.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.396+0000 7fbc5bfff640 1 -- 172.21.15.49:0/119187616 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbc5c15e0c0 msgr2=0x7fbc440047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:00.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.396+0000 7fbc5bfff640 1 --2- 172.21.15.49:0/119187616 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbc5c15e0c0 0x7fbc440047d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:00.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.396+0000 7fbc5bfff640 1 -- 172.21.15.49:0/119187616 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbc44004d10 msgr2=0x7fbc44005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:00.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.396+0000 7fbc5bfff640 1 --2- 172.21.15.49:0/119187616 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbc44004d10 0x7fbc44005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:00.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.396+0000 7fbc5bfff640 1 -- 172.21.15.49:0/119187616 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc44003530 con 0x7fbc44003980 2023-11-06T02:12:00.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.400+0000 7fbc5bfff640 1 --2- 172.21.15.49:0/119187616 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbc44003980 0x7fbc44003da0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c009ae0 tx=0x7fbc4c01ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=611a1a36ecf72319 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:00.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.400+0000 7fbc5affd640 1 -- 172.21.15.49:0/119187616 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc4c0a55f0 con 0x7fbc44003980 2023-11-06T02:12:00.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.400+0000 7fbc5affd640 1 -- 172.21.15.49:0/119187616 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc4c0a5750 con 0x7fbc44003980 2023-11-06T02:12:00.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.400+0000 7fbc5affd640 1 -- 172.21.15.49:0/119187616 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc4c0a5b00 con 0x7fbc44003980 2023-11-06T02:12:00.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.400+0000 7fbc628ea640 1 -- 172.21.15.49:0/119187616 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbc44003980 msgr2=0x7fbc44003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:00.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.400+0000 7fbc628ea640 1 --2- 172.21.15.49:0/119187616 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbc44003980 0x7fbc44003da0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c009ae0 tx=0x7fbc4c01ca20 comp rx=0 tx=0).stop 2023-11-06T02:12:00.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.400+0000 7fbc60e87640 1 -- 172.21.15.49:0/119187616 reap_dead start 2023-11-06T02:12:00.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.400+0000 7fbc628ea640 1 -- 172.21.15.49:0/119187616 shutdown_connections 2023-11-06T02:12:00.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.400+0000 7fbc628ea640 1 -- 172.21.15.49:0/119187616 >> 172.21.15.49:0/119187616 conn(0x7fbc5c099c50 msgr2=0x7fbc5c0d0940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:12:00.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.400+0000 7fbc628ea640 1 -- 172.21.15.49:0/119187616 shutdown_connections 2023-11-06T02:12:00.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.400+0000 7fbc628ea640 1 -- 172.21.15.49:0/119187616 wait complete. 2023-11-06T02:12:00.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.400+0000 7fbc628ea640 1 Processor -- start 2023-11-06T02:12:00.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.400+0000 7fbc628ea640 1 -- start start 2023-11-06T02:12:00.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.400+0000 7fbc628ea640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbc44003980 0x7fbc5c14ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:00.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.400+0000 7fbc628ea640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbc5c1f36f0 0x7fbc5c164550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:00.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.400+0000 7fbc628ea640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbc5c164a90 0x7fbc5c06e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:00.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.400+0000 7fbc628ea640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc5c14f280 con 0x7fbc5c1f36f0 2023-11-06T02:12:00.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.400+0000 7fbc628ea640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc5c0b47c0 con 0x7fbc44003980 2023-11-06T02:12:00.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.400+0000 7fbc628ea640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc5c0b4aa0 con 0x7fbc5c164a90 2023-11-06T02:12:00.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.404+0000 7fbc5b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbc5c1f36f0 0x7fbc5c164550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:00.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.404+0000 7fbc5b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbc5c1f36f0 0x7fbc5c164550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39898/0 (socket says 172.21.15.49:39898) 2023-11-06T02:12:00.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.404+0000 7fbc5b7fe640 1 -- 172.21.15.49:0/3174254107 learned_addr learned my addr 172.21.15.49:0/3174254107 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:12:00.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.404+0000 7fbc60e87640 1 --2- 172.21.15.49:0/3174254107 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbc5c164a90 0x7fbc5c06e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:00.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.404+0000 7fbc5bfff640 1 --2- 172.21.15.49:0/3174254107 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbc44003980 0x7fbc5c14ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:00.411 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.404+0000 7fbc5b7fe640 1 -- 172.21.15.49:0/3174254107 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbc5c164a90 msgr2=0x7fbc5c06e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:00.411 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.404+0000 7fbc5b7fe640 1 --2- 172.21.15.49:0/3174254107 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbc5c164a90 0x7fbc5c06e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:00.411 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.404+0000 7fbc5b7fe640 1 -- 172.21.15.49:0/3174254107 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbc44003980 msgr2=0x7fbc5c14ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:00.411 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.404+0000 7fbc5b7fe640 1 --2- 172.21.15.49:0/3174254107 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbc44003980 0x7fbc5c14ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:00.411 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.404+0000 7fbc5b7fe640 1 -- 172.21.15.49:0/3174254107 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc5c0b4d80 con 0x7fbc5c1f36f0 2023-11-06T02:12:00.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.404+0000 7fbc5b7fe640 1 --2- 172.21.15.49:0/3174254107 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbc5c1f36f0 0x7fbc5c164550 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fbc50010900 tx=0x7fbc50010d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:00.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.404+0000 7fbc58ff9640 1 -- 172.21.15.49:0/3174254107 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc500096b0 con 0x7fbc5c1f36f0 2023-11-06T02:12:00.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.404+0000 7fbc58ff9640 1 -- 172.21.15.49:0/3174254107 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc5000eb60 con 0x7fbc5c1f36f0 2023-11-06T02:12:00.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.404+0000 7fbc58ff9640 1 -- 172.21.15.49:0/3174254107 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc5001c400 con 0x7fbc5c1f36f0 2023-11-06T02:12:00.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.404+0000 7fbc628ea640 1 -- 172.21.15.49:0/3174254107 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc5c0b5060 con 0x7fbc5c1f36f0 2023-11-06T02:12:00.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.404+0000 7fbc628ea640 1 -- 172.21.15.49:0/3174254107 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc5c1f3b50 con 0x7fbc5c1f36f0 2023-11-06T02:12:00.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.404+0000 7fbc628ea640 1 -- 172.21.15.49:0/3174254107 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc5c067140 con 0x7fbc5c1f36f0 2023-11-06T02:12:00.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.404+0000 7fbc58ff9640 1 -- 172.21.15.49:0/3174254107 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fbc5001c560 con 0x7fbc5c1f36f0 2023-11-06T02:12:00.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.404+0000 7fbc58ff9640 1 --2- 172.21.15.49:0/3174254107 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fbc48065a60 0x7fbc48067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:00.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.404+0000 7fbc58ff9640 1 -- 172.21.15.49:0/3174254107 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbc500835d0 con 0x7fbc5c1f36f0 2023-11-06T02:12:00.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.404+0000 7fbc5bfff640 1 --2- 172.21.15.49:0/3174254107 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fbc48065a60 0x7fbc48067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:00.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.408+0000 7fbc5bfff640 1 --2- 172.21.15.49:0/3174254107 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fbc48065a60 0x7fbc48067f20 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c0af000 tx=0x7fbc4c0046d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:00.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.408+0000 7fbc58ff9640 1 -- 172.21.15.49:0/3174254107 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fbc50050ec0 con 0x7fbc5c1f36f0 2023-11-06T02:12:00.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.564+0000 7fbc628ea640 1 -- 172.21.15.49:0/3174254107 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbc5c09a030 con 0x7fbc5c1f36f0 2023-11-06T02:12:00.563 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.564+0000 7fbc58ff9640 1 -- 172.21.15.49:0/3174254107 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v65) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbc500544e0 con 0x7fbc5c1f36f0 2023-11-06T02:12:00.564 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2023-11-06T02:12:00.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.568+0000 7fbc628ea640 1 -- 172.21.15.49:0/3174254107 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fbc48065a60 msgr2=0x7fbc48067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:00.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.568+0000 7fbc628ea640 1 --2- 172.21.15.49:0/3174254107 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fbc48065a60 0x7fbc48067f20 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c0af000 tx=0x7fbc4c0046d0 comp rx=0 tx=0).stop 2023-11-06T02:12:00.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.568+0000 7fbc628ea640 1 -- 172.21.15.49:0/3174254107 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbc5c1f36f0 msgr2=0x7fbc5c164550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:00.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.568+0000 7fbc628ea640 1 --2- 172.21.15.49:0/3174254107 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbc5c1f36f0 0x7fbc5c164550 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fbc50010900 tx=0x7fbc50010d00 comp rx=0 tx=0).stop 2023-11-06T02:12:00.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.568+0000 7fbc628ea640 1 -- 172.21.15.49:0/3174254107 shutdown_connections 2023-11-06T02:12:00.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.568+0000 7fbc628ea640 1 --2- 172.21.15.49:0/3174254107 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fbc48065a60 0x7fbc48067f20 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:00.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.568+0000 7fbc628ea640 1 --2- 172.21.15.49:0/3174254107 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbc5c1f36f0 0x7fbc5c164550 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:00.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.568+0000 7fbc628ea640 1 --2- 172.21.15.49:0/3174254107 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbc5c164a90 0x7fbc5c06e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:00.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.568+0000 7fbc628ea640 1 --2- 172.21.15.49:0/3174254107 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbc44003980 0x7fbc5c14ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:00.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.568+0000 7fbc628ea640 1 -- 172.21.15.49:0/3174254107 >> 172.21.15.49:0/3174254107 conn(0x7fbc5c099c50 msgr2=0x7fbc5c0b0ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:12:00.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.568+0000 7fbc628ea640 1 -- 172.21.15.49:0/3174254107 shutdown_connections 2023-11-06T02:12:00.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.568+0000 7fbc628ea640 1 -- 172.21.15.49:0/3174254107 wait complete. 2023-11-06T02:12:00.583 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T02:12:00.713 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.712+0000 7f8620008640 1 Processor -- start 2023-11-06T02:12:00.713 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.712+0000 7f8620008640 1 -- start start 2023-11-06T02:12:00.714 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.712+0000 7f8620008640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f861814ffc0 0x7f86180ada30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:00.714 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.712+0000 7f8620008640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f8618070870 0x7f861806c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:00.714 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.712+0000 7f8620008640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86180c8c80 con 0x7f86180adf70 2023-11-06T02:12:00.714 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.712+0000 7f8620008640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86180ae4d0 con 0x7f8618070870 2023-11-06T02:12:00.715 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.712+0000 7f8620008640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86181594f0 con 0x7f861814ffc0 2023-11-06T02:12:00.715 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.712+0000 7f861e5a5640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f8618070870 0x7f861806c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:00.715 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.712+0000 7f861d5a3640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f86180adf70 0x7f8618070160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:54758/0 (socket says 172.21.15.49:54758) 2023-11-06T02:12:00.715 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.712+0000 7f861d5a3640 1 -- 172.21.15.49:0/1891293872 learned_addr learned my addr 172.21.15.49:0/1891293872 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:12:00.715 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.712+0000 7f861e5a5640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f8618070870 0x7f861806c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39900/0 (socket says 172.21.15.49:39900) 2023-11-06T02:12:00.716 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.712+0000 7f861dda4640 1 --2- 172.21.15.49:0/1891293872 >> v2:172.21.15.150:3300/0 conn(0x7f861814ffc0 0x7f86180ada30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:00.716 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.712+0000 7f861e5a5640 1 -- 172.21.15.49:0/1891293872 >> v2:172.21.15.150:3300/0 conn(0x7f861814ffc0 msgr2=0x7f86180ada30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:00.716 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.712+0000 7f861e5a5640 1 --2- 172.21.15.49:0/1891293872 >> v2:172.21.15.150:3300/0 conn(0x7f861814ffc0 0x7f86180ada30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:00.716 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.712+0000 7f861e5a5640 1 -- 172.21.15.49:0/1891293872 >> v1:172.21.15.49:6789/0 conn(0x7f86180adf70 legacy=0x7f8618070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:00.716 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.712+0000 7f861cda2640 1 -- 172.21.15.49:0/1891293872 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2179646003 0 0) 0x7f86180c8c80 con 0x7f86180adf70 2023-11-06T02:12:00.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.712+0000 7f861e5a5640 1 -- 172.21.15.49:0/1891293872 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8618159700 con 0x7f8618070870 2023-11-06T02:12:00.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.716+0000 7f861e5a5640 1 --2- 172.21.15.49:0/1891293872 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8618070870 0x7f861806c230 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f861806e390 tx=0x7f861001da30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d05d2435e4777fd4 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:00.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.716+0000 7f861cda2640 1 -- 172.21.15.49:0/1891293872 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f861000a2d0 con 0x7f8618070870 2023-11-06T02:12:00.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.716+0000 7f861cda2640 1 -- 172.21.15.49:0/1891293872 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86100a6760 con 0x7f8618070870 2023-11-06T02:12:00.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.716+0000 7f861cda2640 1 -- 172.21.15.49:0/1891293872 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86100a6af0 con 0x7f8618070870 2023-11-06T02:12:00.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.716+0000 7f8620008640 1 -- 172.21.15.49:0/1891293872 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8618070870 msgr2=0x7f861806c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:00.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.716+0000 7f8620008640 1 --2- 172.21.15.49:0/1891293872 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8618070870 0x7f861806c230 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f861806e390 tx=0x7f861001da30 comp rx=0 tx=0).stop 2023-11-06T02:12:00.721 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.716+0000 7f8620008640 1 -- 172.21.15.49:0/1891293872 shutdown_connections 2023-11-06T02:12:00.721 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.716+0000 7f8620008640 1 --2- 172.21.15.49:0/1891293872 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8618070870 0x7f861806c230 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:00.721 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.716+0000 7f8620008640 1 --2- 172.21.15.49:0/1891293872 >> v2:172.21.15.150:3300/0 conn(0x7f861814ffc0 0x7f86180ada30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:00.721 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.716+0000 7f8620008640 1 -- 172.21.15.49:0/1891293872 >> 172.21.15.49:0/1891293872 conn(0x7f861806a370 msgr2=0x7f86180ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:12:00.722 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.716+0000 7f8620008640 1 -- 172.21.15.49:0/1891293872 shutdown_connections 2023-11-06T02:12:00.722 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.716+0000 7f8620008640 1 -- 172.21.15.49:0/1891293872 wait complete. 2023-11-06T02:12:00.722 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.716+0000 7f8620008640 1 Processor -- start 2023-11-06T02:12:00.722 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.716+0000 7f8620008640 1 -- start start 2023-11-06T02:12:00.722 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.720+0000 7f8620008640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8618070870 0x7f8618152850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:00.723 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.720+0000 7f8620008640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f86180adf70 0x7f86180b3300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:00.723 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.720+0000 7f8620008640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f861814ffc0 0x7f861814e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:00.723 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.720+0000 7f8620008640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86181515a0 con 0x7f8618070870 2023-11-06T02:12:00.723 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.720+0000 7f8620008640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8618151a90 con 0x7f861814ffc0 2023-11-06T02:12:00.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.720+0000 7f8620008640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8618151d70 con 0x7f86180adf70 2023-11-06T02:12:00.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.720+0000 7f861dda4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8618070870 0x7f8618152850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:00.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.720+0000 7f861dda4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8618070870 0x7f8618152850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39916/0 (socket says 172.21.15.49:39916) 2023-11-06T02:12:00.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.720+0000 7f861dda4640 1 -- 172.21.15.49:0/2162175214 learned_addr learned my addr 172.21.15.49:0/2162175214 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:12:00.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.720+0000 7f861e5a5640 1 --2- 172.21.15.49:0/2162175214 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f861814ffc0 0x7f861814e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:00.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.720+0000 7f861dda4640 1 -- 172.21.15.49:0/2162175214 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f86180adf70 msgr2=0x7f86180b3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:00.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.720+0000 7f861dda4640 1 --2- 172.21.15.49:0/2162175214 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f86180adf70 0x7f86180b3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:00.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.720+0000 7f861dda4640 1 -- 172.21.15.49:0/2162175214 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f861814ffc0 msgr2=0x7f861814e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:00.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.720+0000 7f861dda4640 1 --2- 172.21.15.49:0/2162175214 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f861814ffc0 0x7f861814e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:00.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.720+0000 7f861dda4640 1 -- 172.21.15.49:0/2162175214 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f861000ab80 con 0x7f8618070870 2023-11-06T02:12:00.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.720+0000 7f861dda4640 1 --2- 172.21.15.49:0/2162175214 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8618070870 0x7f8618152850 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f861400d5f0 tx=0x7f861400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:00.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.720+0000 7f860e7fc640 1 -- 172.21.15.49:0/2162175214 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86140125f0 con 0x7f8618070870 2023-11-06T02:12:00.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.720+0000 7f860e7fc640 1 -- 172.21.15.49:0/2162175214 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8614012750 con 0x7f8618070870 2023-11-06T02:12:00.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.720+0000 7f860e7fc640 1 -- 172.21.15.49:0/2162175214 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8614012ae0 con 0x7f8618070870 2023-11-06T02:12:00.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.720+0000 7f8620008640 1 -- 172.21.15.49:0/2162175214 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8618152050 con 0x7f8618070870 2023-11-06T02:12:00.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.720+0000 7f8620008640 1 -- 172.21.15.49:0/2162175214 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8618152d90 con 0x7f8618070870 2023-11-06T02:12:00.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.720+0000 7f8620008640 1 -- 172.21.15.49:0/2162175214 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8618074040 con 0x7f8618070870 2023-11-06T02:12:00.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.720+0000 7f860e7fc640 1 -- 172.21.15.49:0/2162175214 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f8614012c40 con 0x7f8618070870 2023-11-06T02:12:00.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.720+0000 7f860e7fc640 1 --2- 172.21.15.49:0/2162175214 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f85fc065a60 0x7f85fc067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:00.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.720+0000 7f860e7fc640 1 -- 172.21.15.49:0/2162175214 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8614083880 con 0x7f8618070870 2023-11-06T02:12:00.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.720+0000 7f861d5a3640 1 --2- 172.21.15.49:0/2162175214 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f85fc065a60 0x7f85fc067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:00.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.724+0000 7f861d5a3640 1 --2- 172.21.15.49:0/2162175214 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f85fc065a60 0x7f85fc067f20 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f86080011f0 tx=0x7f8608007930 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:00.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.724+0000 7f860e7fc640 1 -- 172.21.15.49:0/2162175214 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f8614051050 con 0x7f8618070870 2023-11-06T02:12:00.878 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.880+0000 7f8620008640 1 -- 172.21.15.49:0/2162175214 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f861814f760 con 0x7f8618070870 2023-11-06T02:12:00.879 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.880+0000 7f860e7fc640 1 -- 172.21.15.49:0/2162175214 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v65) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8614054670 con 0x7f8618070870 2023-11-06T02:12:00.880 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2023-11-06T02:12:00.881 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.884+0000 7f8620008640 1 -- 172.21.15.49:0/2162175214 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f85fc065a60 msgr2=0x7f85fc067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:00.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.884+0000 7f8620008640 1 --2- 172.21.15.49:0/2162175214 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f85fc065a60 0x7f85fc067f20 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f86080011f0 tx=0x7f8608007930 comp rx=0 tx=0).stop 2023-11-06T02:12:00.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.884+0000 7f8620008640 1 -- 172.21.15.49:0/2162175214 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8618070870 msgr2=0x7f8618152850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:00.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.884+0000 7f8620008640 1 --2- 172.21.15.49:0/2162175214 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8618070870 0x7f8618152850 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f861400d5f0 tx=0x7f861400dab0 comp rx=0 tx=0).stop 2023-11-06T02:12:00.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.884+0000 7f8620008640 1 -- 172.21.15.49:0/2162175214 shutdown_connections 2023-11-06T02:12:00.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.884+0000 7f8620008640 1 --2- 172.21.15.49:0/2162175214 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f85fc065a60 0x7f85fc067f20 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:00.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.884+0000 7f8620008640 1 --2- 172.21.15.49:0/2162175214 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f861814ffc0 0x7f861814e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:00.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.884+0000 7f8620008640 1 --2- 172.21.15.49:0/2162175214 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f86180adf70 0x7f86180b3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:00.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.884+0000 7f8620008640 1 --2- 172.21.15.49:0/2162175214 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8618070870 0x7f8618152850 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:00.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.884+0000 7f8620008640 1 -- 172.21.15.49:0/2162175214 >> 172.21.15.49:0/2162175214 conn(0x7f861806a370 msgr2=0x7f86180cb1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:12:00.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.884+0000 7f8620008640 1 -- 172.21.15.49:0/2162175214 shutdown_connections 2023-11-06T02:12:00.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:00.884+0000 7f8620008640 1 -- 172.21.15.49:0/2162175214 wait complete. 2023-11-06T02:12:00.898 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-06T02:12:01.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.024+0000 7fe0b859b640 1 Processor -- start 2023-11-06T02:12:01.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.024+0000 7fe0b859b640 1 -- start start 2023-11-06T02:12:01.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.024+0000 7fe0b859b640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fe0b015cd70 0x7fe0b006b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:01.026 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.024+0000 7fe0b859b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe0b006bfa0 0x7fe0b00b2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:01.026 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.024+0000 7fe0b859b640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0b00aff70 con 0x7fe0b00b0830 2023-11-06T02:12:01.026 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.024+0000 7fe0b859b640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0b00b0dd0 con 0x7fe0b015cd70 2023-11-06T02:12:01.026 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.024+0000 7fe0b859b640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b00b10b0 con 0x7fe0b006bfa0 2023-11-06T02:12:01.027 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.024+0000 7fe0b5b36640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fe0b00b0830 0x7fe0b006b890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:54774/0 (socket says 172.21.15.49:54774) 2023-11-06T02:12:01.027 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.024+0000 7fe0b5b36640 1 -- 172.21.15.49:0/4291063328 learned_addr learned my addr 172.21.15.49:0/4291063328 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:12:01.027 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.024+0000 7fe0b6337640 1 --2- 172.21.15.49:0/4291063328 >> v2:172.21.15.150:3300/0 conn(0x7fe0b015cd70 0x7fe0b006b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:01.027 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.024+0000 7fe0b6b38640 1 --2- 172.21.15.49:0/4291063328 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe0b006bfa0 0x7fe0b00b2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:01.027 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.024+0000 7fe0b5335640 1 -- 172.21.15.49:0/4291063328 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 329815423 0 0) 0x7fe0b00aff70 con 0x7fe0b00b0830 2023-11-06T02:12:01.028 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.024+0000 7fe0b5335640 1 -- 172.21.15.49:0/4291063328 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe09c003530 con 0x7fe0b00b0830 2023-11-06T02:12:01.028 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.024+0000 7fe0b5335640 1 -- 172.21.15.49:0/4291063328 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3015298358 0 0) 0x7fe09c003530 con 0x7fe0b00b0830 2023-11-06T02:12:01.028 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.024+0000 7fe0b5335640 1 -- 172.21.15.49:0/4291063328 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe0b006bfa0 msgr2=0x7fe0b00b2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:01.028 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.024+0000 7fe0b5335640 1 --2- 172.21.15.49:0/4291063328 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe0b006bfa0 0x7fe0b00b2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:01.029 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.024+0000 7fe0b5335640 1 -- 172.21.15.49:0/4291063328 >> v2:172.21.15.150:3300/0 conn(0x7fe0b015cd70 msgr2=0x7fe0b006b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:01.029 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.024+0000 7fe0b5335640 1 --2- 172.21.15.49:0/4291063328 >> v2:172.21.15.150:3300/0 conn(0x7fe0b015cd70 0x7fe0b006b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:01.029 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.024+0000 7fe0b5335640 1 -- 172.21.15.49:0/4291063328 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0b00b1390 con 0x7fe0b00b0830 2023-11-06T02:12:01.029 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.024+0000 7fe0b5335640 1 -- 172.21.15.49:0/4291063328 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7fe0a0001680 con 0x7fe0b00b0830 2023-11-06T02:12:01.030 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.028+0000 7fe0b5335640 1 -- 172.21.15.49:0/4291063328 >> v1:172.21.15.49:6789/0 conn(0x7fe0b00b0830 legacy=0x7fe0b006b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:01.030 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.028+0000 7fe0b5335640 1 --2- 172.21.15.49:0/4291063328 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe0b006bfa0 0x7fe09c003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:01.030 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.028+0000 7fe0b5335640 1 --2- 172.21.15.49:0/4291063328 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe09c003f00 0x7fe09c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:01.030 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.028+0000 7fe0b5335640 1 --2- 172.21.15.49:0/4291063328 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe09c004d10 0x7fe09c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:01.031 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.028+0000 7fe0b5335640 1 -- 172.21.15.49:0/4291063328 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b00b0dd0 con 0x7fe09c003f00 2023-11-06T02:12:01.031 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.028+0000 7fe0b5335640 1 -- 172.21.15.49:0/4291063328 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b00b10b0 con 0x7fe09c004d10 2023-11-06T02:12:01.031 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.028+0000 7fe0b5335640 1 -- 172.21.15.49:0/4291063328 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe09c00ba70 con 0x7fe0b006bfa0 2023-11-06T02:12:01.031 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.028+0000 7fe0b5b36640 1 --2- 172.21.15.49:0/4291063328 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe09c003f00 0x7fe09c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:01.031 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.028+0000 7fe0b6337640 1 --2- 172.21.15.49:0/4291063328 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe0b006bfa0 0x7fe09c003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:01.032 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.028+0000 7fe0b6b38640 1 --2- 172.21.15.49:0/4291063328 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe09c004d10 0x7fe09c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:01.032 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.028+0000 7fe0b5b36640 1 -- 172.21.15.49:0/4291063328 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe0b006bfa0 msgr2=0x7fe09c003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:01.032 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.028+0000 7fe0b5b36640 1 --2- 172.21.15.49:0/4291063328 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe0b006bfa0 0x7fe09c003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:01.032 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.028+0000 7fe0b5b36640 1 -- 172.21.15.49:0/4291063328 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe09c004d10 msgr2=0x7fe09c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:01.032 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.028+0000 7fe0b5b36640 1 --2- 172.21.15.49:0/4291063328 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe09c004d10 0x7fe09c005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:01.033 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.028+0000 7fe0b5b36640 1 -- 172.21.15.49:0/4291063328 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe09c003530 con 0x7fe09c003f00 2023-11-06T02:12:01.033 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.028+0000 7fe0b6b38640 1 --2- 172.21.15.49:0/4291063328 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe09c004d10 0x7fe09c005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:12:01.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.028+0000 7fe0b5b36640 1 --2- 172.21.15.49:0/4291063328 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe09c003f00 0x7fe09c0047d0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fe0a000c290 tx=0x7fe0a001cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d6b8f2a2b5b989f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:01.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.028+0000 7fe0b5335640 1 -- 172.21.15.49:0/4291063328 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0a00a5950 con 0x7fe09c003f00 2023-11-06T02:12:01.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.028+0000 7fe0b5335640 1 -- 172.21.15.49:0/4291063328 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0a00a5ab0 con 0x7fe09c003f00 2023-11-06T02:12:01.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.028+0000 7fe0b5335640 1 -- 172.21.15.49:0/4291063328 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0a00a5e60 con 0x7fe09c003f00 2023-11-06T02:12:01.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.028+0000 7fe0b859b640 1 -- 172.21.15.49:0/4291063328 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe09c003f00 msgr2=0x7fe09c0047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:01.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.028+0000 7fe0b859b640 1 --2- 172.21.15.49:0/4291063328 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe09c003f00 0x7fe09c0047d0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fe0a000c290 tx=0x7fe0a001cce0 comp rx=0 tx=0).stop 2023-11-06T02:12:01.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.028+0000 7fe0b6b38640 1 -- 172.21.15.49:0/4291063328 reap_dead start 2023-11-06T02:12:01.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.028+0000 7fe0b859b640 1 -- 172.21.15.49:0/4291063328 shutdown_connections 2023-11-06T02:12:01.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.028+0000 7fe0b859b640 1 -- 172.21.15.49:0/4291063328 >> 172.21.15.49:0/4291063328 conn(0x7fe0b0089fd0 msgr2=0x7fe0b0099c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:12:01.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.028+0000 7fe0b859b640 1 -- 172.21.15.49:0/4291063328 shutdown_connections 2023-11-06T02:12:01.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.028+0000 7fe0b859b640 1 -- 172.21.15.49:0/4291063328 wait complete. 2023-11-06T02:12:01.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.032+0000 7fe0b859b640 1 Processor -- start 2023-11-06T02:12:01.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.032+0000 7fe0b859b640 1 -- start start 2023-11-06T02:12:01.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.032+0000 7fe0b859b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe09c003f00 0x7fe0b015f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:01.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.032+0000 7fe0b859b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe0b00b5e00 0x7fe0b015f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:01.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.032+0000 7fe0b859b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe0b015d150 0x7fe0b015d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:01.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.032+0000 7fe0b859b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b015db40 con 0x7fe0b015d150 2023-11-06T02:12:01.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.032+0000 7fe0b859b640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b015ffa0 con 0x7fe0b00b5e00 2023-11-06T02:12:01.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.032+0000 7fe0b859b640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b015fcc0 con 0x7fe09c003f00 2023-11-06T02:12:01.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.032+0000 7fe0b6b38640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe0b015d150 0x7fe0b015d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:01.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.032+0000 7fe0b6b38640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe0b015d150 0x7fe0b015d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39926/0 (socket says 172.21.15.49:39926) 2023-11-06T02:12:01.042 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.032+0000 7fe0b6b38640 1 -- 172.21.15.49:0/3055968855 learned_addr learned my addr 172.21.15.49:0/3055968855 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:12:01.042 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.032+0000 7fe0b6337640 1 --2- 172.21.15.49:0/3055968855 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe09c003f00 0x7fe0b015f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:01.042 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.032+0000 7fe0b5b36640 1 --2- 172.21.15.49:0/3055968855 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe0b00b5e00 0x7fe0b015f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:01.042 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.032+0000 7fe0b6b38640 1 -- 172.21.15.49:0/3055968855 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe09c003f00 msgr2=0x7fe0b015f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:01.042 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.032+0000 7fe0b6b38640 1 --2- 172.21.15.49:0/3055968855 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe09c003f00 0x7fe0b015f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:01.043 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.032+0000 7fe0b6b38640 1 -- 172.21.15.49:0/3055968855 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe0b00b5e00 msgr2=0x7fe0b015f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:01.043 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.032+0000 7fe0b6b38640 1 --2- 172.21.15.49:0/3055968855 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe0b00b5e00 0x7fe0b015f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:01.043 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.032+0000 7fe0b6b38640 1 -- 172.21.15.49:0/3055968855 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0b01545d0 con 0x7fe0b015d150 2023-11-06T02:12:01.043 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.032+0000 7fe0b6337640 1 --2- 172.21.15.49:0/3055968855 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe09c003f00 0x7fe0b015f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:12:01.043 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.032+0000 7fe0b6b38640 1 --2- 172.21.15.49:0/3055968855 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe0b015d150 0x7fe0b015d600 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fe0b0072200 tx=0x7fe0a800f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:01.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.032+0000 7fe0a6ffd640 1 -- 172.21.15.49:0/3055968855 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0a8017400 con 0x7fe0b015d150 2023-11-06T02:12:01.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.032+0000 7fe0a6ffd640 1 -- 172.21.15.49:0/3055968855 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0a8011040 con 0x7fe0b015d150 2023-11-06T02:12:01.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.032+0000 7fe0b859b640 1 -- 172.21.15.49:0/3055968855 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0b01548b0 con 0x7fe0b015d150 2023-11-06T02:12:01.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.032+0000 7fe0a6ffd640 1 -- 172.21.15.49:0/3055968855 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0a80177c0 con 0x7fe0b015d150 2023-11-06T02:12:01.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.032+0000 7fe0b859b640 1 -- 172.21.15.49:0/3055968855 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0b01540f0 con 0x7fe0b015d150 2023-11-06T02:12:01.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.032+0000 7fe0b859b640 1 -- 172.21.15.49:0/3055968855 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0b0154340 con 0x7fe0b015d150 2023-11-06T02:12:01.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.032+0000 7fe0a6ffd640 1 -- 172.21.15.49:0/3055968855 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fe0a80203b0 con 0x7fe0b015d150 2023-11-06T02:12:01.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.036+0000 7fe0a6ffd640 1 --2- 172.21.15.49:0/3055968855 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe094065a90 0x7fe094067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:01.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.036+0000 7fe0a6ffd640 1 -- 172.21.15.49:0/3055968855 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe0a80833d0 con 0x7fe0b015d150 2023-11-06T02:12:01.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.036+0000 7fe0b6337640 1 --2- 172.21.15.49:0/3055968855 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe094065a90 0x7fe094067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:01.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.036+0000 7fe0b6337640 1 --2- 172.21.15.49:0/3055968855 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe094065a90 0x7fe094067f50 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fe0b015c0d0 tx=0x7fe0ac008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:01.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.040+0000 7fe0a6ffd640 1 -- 172.21.15.49:0/3055968855 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fe0a8050c30 con 0x7fe0b015d150 2023-11-06T02:12:01.208 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.208+0000 7fe0b859b640 1 -- 172.21.15.49:0/3055968855 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe0b00b03e0 con 0x7fe0b015d150 2023-11-06T02:12:01.208 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.208+0000 7fe0a6ffd640 1 -- 172.21.15.49:0/3055968855 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v65) v1 ==== 80+0+0 (secure 0 0 0) 0x7fe0a8054250 con 0x7fe0b015d150 2023-11-06T02:12:01.209 INFO:teuthology.orchestra.run.smithi049.stderr:listed 0 entries 2023-11-06T02:12:01.211 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.212+0000 7fe0b859b640 1 -- 172.21.15.49:0/3055968855 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe094065a90 msgr2=0x7fe094067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:01.211 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.212+0000 7fe0b859b640 1 --2- 172.21.15.49:0/3055968855 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe094065a90 0x7fe094067f50 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fe0b015c0d0 tx=0x7fe0ac008040 comp rx=0 tx=0).stop 2023-11-06T02:12:01.211 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.212+0000 7fe0b859b640 1 -- 172.21.15.49:0/3055968855 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe0b015d150 msgr2=0x7fe0b015d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:01.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.212+0000 7fe0b859b640 1 --2- 172.21.15.49:0/3055968855 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe0b015d150 0x7fe0b015d600 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fe0b0072200 tx=0x7fe0a800f4a0 comp rx=0 tx=0).stop 2023-11-06T02:12:01.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.212+0000 7fe0b859b640 1 -- 172.21.15.49:0/3055968855 shutdown_connections 2023-11-06T02:12:01.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.212+0000 7fe0b859b640 1 --2- 172.21.15.49:0/3055968855 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe094065a90 0x7fe094067f50 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:01.213 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.212+0000 7fe0b859b640 1 --2- 172.21.15.49:0/3055968855 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe0b015d150 0x7fe0b015d600 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:01.213 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.212+0000 7fe0b859b640 1 --2- 172.21.15.49:0/3055968855 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe0b00b5e00 0x7fe0b015f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:01.213 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.212+0000 7fe0b859b640 1 --2- 172.21.15.49:0/3055968855 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe09c003f00 0x7fe0b015f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:01.214 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.212+0000 7fe0b859b640 1 -- 172.21.15.49:0/3055968855 >> 172.21.15.49:0/3055968855 conn(0x7fe0b0089fd0 msgr2=0x7fe0b015a020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:12:01.214 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.212+0000 7fe0b859b640 1 -- 172.21.15.49:0/3055968855 shutdown_connections 2023-11-06T02:12:01.214 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.212+0000 7fe0b859b640 1 -- 172.21.15.49:0/3055968855 wait complete. 2023-11-06T02:12:01.228 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-11-06T02:12:01.228 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T02:12:01.236 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T02:12:01.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.416+0000 7f616b769640 1 Processor -- start 2023-11-06T02:12:01.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.416+0000 7f616b769640 1 -- start start 2023-11-06T02:12:01.418 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.416+0000 7f616b769640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61640adae0 0x7f61640c8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:01.418 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.416+0000 7f616b769640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61641594d0 con 0x7f61641590f0 2023-11-06T02:12:01.418 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.416+0000 7f616b769640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61641597a0 con 0x7f61640b0790 2023-11-06T02:12:01.418 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.416+0000 7f616b769640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f616415a140 con 0x7f61640adae0 2023-11-06T02:12:01.419 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.416+0000 7f6169505640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f61641590f0 0x7f61640b0080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:54788/0 (socket says 172.21.15.49:54788) 2023-11-06T02:12:01.419 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.416+0000 7f6169505640 1 -- 172.21.15.49:0/2807099412 learned_addr learned my addr 172.21.15.49:0/2807099412 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:12:01.419 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.416+0000 7f615bfff640 1 -- 172.21.15.49:0/2807099412 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1852361784 0 0) 0x7f61641594d0 con 0x7f61641590f0 2023-11-06T02:12:01.419 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.416+0000 7f615bfff640 1 -- 172.21.15.49:0/2807099412 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6150005740 con 0x7f61641590f0 2023-11-06T02:12:01.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.416+0000 7f6169d06640 1 --2- 172.21.15.49:0/2807099412 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61640adae0 0x7f61640c8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:01.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.416+0000 7f615bfff640 1 -- 172.21.15.49:0/2807099412 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3256691395 0 0) 0x7f6150005740 con 0x7f61641590f0 2023-11-06T02:12:01.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.416+0000 7f615bfff640 1 -- 172.21.15.49:0/2807099412 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61640adae0 msgr2=0x7f61640c8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:01.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.416+0000 7f615bfff640 1 --2- 172.21.15.49:0/2807099412 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61640adae0 0x7f61640c8c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:01.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.416+0000 7f615bfff640 1 -- 172.21.15.49:0/2807099412 >> v1:172.21.15.150:6789/0 conn(0x7f61640b0790 legacy=0x7f6164070b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:01.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.416+0000 7f615bfff640 1 -- 172.21.15.49:0/2807099412 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6164159a80 con 0x7f61641590f0 2023-11-06T02:12:01.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.416+0000 7f615bfff640 1 -- 172.21.15.49:0/2807099412 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7f6160001800 con 0x7f61641590f0 2023-11-06T02:12:01.422 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.416+0000 7f615bfff640 1 -- 172.21.15.49:0/2807099412 >> v1:172.21.15.49:6789/0 conn(0x7f61641590f0 legacy=0x7f61640b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:01.422 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.416+0000 7f615bfff640 1 --2- 172.21.15.49:0/2807099412 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61640adae0 0x7f6150005bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:01.422 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.416+0000 7f615bfff640 1 --2- 172.21.15.49:0/2807099412 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61500061f0 0x7f61500069e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:01.422 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.416+0000 7f615bfff640 1 --2- 172.21.15.49:0/2807099412 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6150006f70 0x7f61500073f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:01.422 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.416+0000 7f615bfff640 1 -- 172.21.15.49:0/2807099412 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f616415a140 con 0x7f6150006f70 2023-11-06T02:12:01.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.416+0000 7f615bfff640 1 -- 172.21.15.49:0/2807099412 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f615000de80 con 0x7f61500061f0 2023-11-06T02:12:01.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.416+0000 7f615bfff640 1 -- 172.21.15.49:0/2807099412 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f615000e040 con 0x7f61640adae0 2023-11-06T02:12:01.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.416+0000 7f6169d06640 1 --2- 172.21.15.49:0/2807099412 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6150006f70 0x7f61500073f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:01.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.416+0000 7f6169505640 1 --2- 172.21.15.49:0/2807099412 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61640adae0 0x7f6150005bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:01.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.416+0000 7f6168d04640 1 --2- 172.21.15.49:0/2807099412 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61500061f0 0x7f61500069e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:01.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.416+0000 7f6169d06640 1 -- 172.21.15.49:0/2807099412 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61640adae0 msgr2=0x7f6150005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:01.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.416+0000 7f6169d06640 1 --2- 172.21.15.49:0/2807099412 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61640adae0 0x7f6150005bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:01.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.416+0000 7f6169d06640 1 -- 172.21.15.49:0/2807099412 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61500061f0 msgr2=0x7f61500069e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:01.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.416+0000 7f6169d06640 1 --2- 172.21.15.49:0/2807099412 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61500061f0 0x7f61500069e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:01.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.416+0000 7f6169d06640 1 -- 172.21.15.49:0/2807099412 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6150005740 con 0x7f6150006f70 2023-11-06T02:12:01.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.416+0000 7f6168d04640 1 --2- 172.21.15.49:0/2807099412 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61500061f0 0x7f61500069e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:12:01.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.420+0000 7f6169d06640 1 --2- 172.21.15.49:0/2807099412 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6150006f70 0x7f61500073f0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f615c007330 tx=0x7f615c01ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a61871a6853bc531 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:01.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.420+0000 7f615bfff640 1 -- 172.21.15.49:0/2807099412 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f615c0a5a60 con 0x7f6150006f70 2023-11-06T02:12:01.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.420+0000 7f615bfff640 1 -- 172.21.15.49:0/2807099412 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f615c0a5bc0 con 0x7f6150006f70 2023-11-06T02:12:01.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.420+0000 7f615bfff640 1 -- 172.21.15.49:0/2807099412 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f615c0ad440 con 0x7f6150006f70 2023-11-06T02:12:01.430 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.420+0000 7f616b769640 1 -- 172.21.15.49:0/2807099412 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6150006f70 msgr2=0x7f61500073f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:01.430 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.420+0000 7f616b769640 1 --2- 172.21.15.49:0/2807099412 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6150006f70 0x7f61500073f0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f615c007330 tx=0x7f615c01ce00 comp rx=0 tx=0).stop 2023-11-06T02:12:01.431 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.420+0000 7f6169d06640 1 -- 172.21.15.49:0/2807099412 reap_dead start 2023-11-06T02:12:01.431 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.420+0000 7f616b769640 1 -- 172.21.15.49:0/2807099412 shutdown_connections 2023-11-06T02:12:01.431 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.420+0000 7f616b769640 1 -- 172.21.15.49:0/2807099412 >> 172.21.15.49:0/2807099412 conn(0x7f616406c0a0 msgr2=0x7f616406c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:12:01.432 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.420+0000 7f616b769640 1 -- 172.21.15.49:0/2807099412 shutdown_connections 2023-11-06T02:12:01.432 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.420+0000 7f616b769640 1 -- 172.21.15.49:0/2807099412 wait complete. 2023-11-06T02:12:01.432 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.420+0000 7f616b769640 1 Processor -- start 2023-11-06T02:12:01.433 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.420+0000 7f616b769640 1 -- start start 2023-11-06T02:12:01.433 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.424+0000 7f616b769640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6150006f70 0x7f61640b0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:01.433 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.424+0000 7f616b769640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f616414ed90 0x7f61640c68d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:01.433 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.424+0000 7f616b769640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61640c6e10 0x7f61641533e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:01.433 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.424+0000 7f616b769640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61640afc00 con 0x7f616414ed90 2023-11-06T02:12:01.434 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.424+0000 7f616b769640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61640b05d0 con 0x7f61640c6e10 2023-11-06T02:12:01.434 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.424+0000 7f616b769640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61640aeff0 con 0x7f6150006f70 2023-11-06T02:12:01.434 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.424+0000 7f6168d04640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f616414ed90 0x7f61640c68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:01.434 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.424+0000 7f6168d04640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f616414ed90 0x7f61640c68d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39938/0 (socket says 172.21.15.49:39938) 2023-11-06T02:12:01.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.424+0000 7f6168d04640 1 -- 172.21.15.49:0/1592349841 learned_addr learned my addr 172.21.15.49:0/1592349841 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:12:01.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.424+0000 7f6169d06640 1 --2- 172.21.15.49:0/1592349841 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61640c6e10 0x7f61641533e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:01.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.424+0000 7f6169505640 1 --2- 172.21.15.49:0/1592349841 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6150006f70 0x7f61640b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:01.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.424+0000 7f6168d04640 1 -- 172.21.15.49:0/1592349841 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6150006f70 msgr2=0x7f61640b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:01.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.424+0000 7f6168d04640 1 --2- 172.21.15.49:0/1592349841 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6150006f70 0x7f61640b0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:01.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.424+0000 7f6168d04640 1 -- 172.21.15.49:0/1592349841 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61640c6e10 msgr2=0x7f61641533e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:01.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.424+0000 7f6168d04640 1 --2- 172.21.15.49:0/1592349841 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61640c6e10 0x7f61641533e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:01.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.424+0000 7f6168d04640 1 -- 172.21.15.49:0/1592349841 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61640af2d0 con 0x7f616414ed90 2023-11-06T02:12:01.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.424+0000 7f6169d06640 1 --2- 172.21.15.49:0/1592349841 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61640c6e10 0x7f61641533e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:12:01.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.424+0000 7f6168d04640 1 --2- 172.21.15.49:0/1592349841 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f616414ed90 0x7f61640c68d0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f615400d770 tx=0x7f615400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:01.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.424+0000 7f6159ffb640 1 -- 172.21.15.49:0/1592349841 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6154005270 con 0x7f616414ed90 2023-11-06T02:12:01.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.424+0000 7f6159ffb640 1 -- 172.21.15.49:0/1592349841 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61540053d0 con 0x7f616414ed90 2023-11-06T02:12:01.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.424+0000 7f6159ffb640 1 -- 172.21.15.49:0/1592349841 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f615400b8d0 con 0x7f616414ed90 2023-11-06T02:12:01.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.424+0000 7f616b769640 1 -- 172.21.15.49:0/1592349841 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61640af5b0 con 0x7f616414ed90 2023-11-06T02:12:01.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.424+0000 7f616b769640 1 -- 172.21.15.49:0/1592349841 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61640aec80 con 0x7f616414ed90 2023-11-06T02:12:01.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.424+0000 7f616b769640 1 -- 172.21.15.49:0/1592349841 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6164074040 con 0x7f616414ed90 2023-11-06T02:12:01.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.424+0000 7f6159ffb640 1 -- 172.21.15.49:0/1592349841 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f615400ba30 con 0x7f616414ed90 2023-11-06T02:12:01.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.424+0000 7f6159ffb640 1 --2- 172.21.15.49:0/1592349841 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6148065a60 0x7f6148067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:01.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.424+0000 7f6159ffb640 1 -- 172.21.15.49:0/1592349841 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f61540832e0 con 0x7f616414ed90 2023-11-06T02:12:01.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.424+0000 7f6169505640 1 --2- 172.21.15.49:0/1592349841 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6148065a60 0x7f6148067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:01.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.428+0000 7f6169505640 1 --2- 172.21.15.49:0/1592349841 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6148065a60 0x7f6148067f20 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f6160001270 tx=0x7f6160009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:01.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.432+0000 7f6159ffb640 1 -- 172.21.15.49:0/1592349841 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f6154050b20 con 0x7f616414ed90 2023-11-06T02:12:01.587 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.588+0000 7f616b769640 1 -- 172.21.15.49:0/1592349841 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f61640ad840 con 0x7f616414ed90 2023-11-06T02:12:01.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.592+0000 7f6159ffb640 1 -- 172.21.15.49:0/1592349841 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+17825 (secure 0 0 0) 0x7f6154019310 con 0x7f616414ed90 2023-11-06T02:12:01.590 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:12:01.591 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":65,"fsid":"1a8b035c-6ba4-499a-b876-aea47fd9c0d4","created":"2023-11-06T02:08:06.440857+0000","modified":"2023-11-06T02:11:31.094207+0000","last_up_change":"2023-11-06T02:08:17.982372+0000","last_in_change":"2023-11-06T02:08:08.906631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T02:08:19.252222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T02:08:24.776211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-06T02:11:28.648326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-06T02:11:29.079513+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ad55af8-0571-4e6a-8722-b5d429b7496d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6809","nonce":38116},{"type":"v1","addr":"172.21.15.49:6811","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6813","nonce":38116},{"type":"v1","addr":"172.21.15.49:6815","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6821","nonce":38116},{"type":"v1","addr":"172.21.15.49:6823","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6817","nonce":38116},{"type":"v1","addr":"172.21.15.49:6819","nonce":38116}]},"public_addr":"172.21.15.49:6811/38116","cluster_addr":"172.21.15.49:6815/38116","heartbeat_back_addr":"172.21.15.49:6823/38116","heartbeat_front_addr":"172.21.15.49:6819/38116","state":["exists","up"]},{"osd":1,"uuid":"c8b31f85-a757-43ed-9419-351acc46611b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":38113},{"type":"v1","addr":"172.21.15.49:6810","nonce":38113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":38113},{"type":"v1","addr":"172.21.15.49:6814","nonce":38113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":38113},{"type":"v1","addr":"172.21.15.49:6822","nonce":38113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":38113},{"type":"v1","addr":"172.21.15.49:6818","nonce":38113}]},"public_addr":"172.21.15.49:6810/38113","cluster_addr":"172.21.15.49:6814/38113","heartbeat_back_addr":"172.21.15.49:6822/38113","heartbeat_front_addr":"172.21.15.49:6818/38113","state":["exists","up"]},{"osd":2,"uuid":"555b3bf3-0a70-4b91-8efd-e4204b5fc26e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":38114},{"type":"v1","addr":"172.21.15.49:6801","nonce":38114}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":38114},{"type":"v1","addr":"172.21.15.49:6803","nonce":38114}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":38114},{"type":"v1","addr":"172.21.15.49:6807","nonce":38114}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":38114},{"type":"v1","addr":"172.21.15.49:6805","nonce":38114}]},"public_addr":"172.21.15.49:6801/38114","cluster_addr":"172.21.15.49:6803/38114","heartbeat_back_addr":"172.21.15.49:6807/38114","heartbeat_front_addr":"172.21.15.49:6805/38114","state":["exists","up"]},{"osd":3,"uuid":"8d4e5d77-03ef-4b9b-bfcb-5ae93a46795c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":38115},{"type":"v1","addr":"172.21.15.49:6825","nonce":38115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":38115},{"type":"v1","addr":"172.21.15.49:6827","nonce":38115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":38115},{"type":"v1","addr":"172.21.15.49:6831","nonce":38115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":38115},{"type":"v1","addr":"172.21.15.49:6829","nonce":38115}]},"public_addr":"172.21.15.49:6825/38115","cluster_addr":"172.21.15.49:6827/38115","heartbeat_back_addr":"172.21.15.49:6831/38115","heartbeat_front_addr":"172.21.15.49:6829/38115","state":["exists","up"]},{"osd":4,"uuid":"dba03ad4-68a3-4cf3-9218-ac611f02eacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":38417},{"type":"v1","addr":"172.21.15.150:6809","nonce":38417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":38417},{"type":"v1","addr":"172.21.15.150:6811","nonce":38417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":38417},{"type":"v1","addr":"172.21.15.150:6815","nonce":38417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":38417},{"type":"v1","addr":"172.21.15.150:6813","nonce":38417}]},"public_addr":"172.21.15.150:6809/38417","cluster_addr":"172.21.15.150:6811/38417","heartbeat_back_addr":"172.21.15.150:6815/38417","heartbeat_front_addr":"172.21.15.150:6813/38417","state":["exists","up"]},{"osd":5,"uuid":"0df9c303-c255-4640-8e91-01f1c3efe18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":38418},{"type":"v1","addr":"172.21.15.150:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":38418},{"type":"v1","addr":"172.21.15.150:6803","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":38418},{"type":"v1","addr":"172.21.15.150:6807","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":38418},{"type":"v1","addr":"172.21.15.150:6805","nonce":38418}]},"public_addr":"172.21.15.150:6801/38418","cluster_addr":"172.21.15.150:6803/38418","heartbeat_back_addr":"172.21.15.150:6807/38418","heartbeat_front_addr":"172.21.15.150:6805/38418","state":["exists","up"]},{"osd":6,"uuid":"e59fa189-4b70-4c06-9722-c02b90828bf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":38428},{"type":"v1","addr":"172.21.15.150:6825","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":38428},{"type":"v1","addr":"172.21.15.150:6827","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":38428},{"type":"v1","addr":"172.21.15.150:6831","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":38428},{"type":"v1","addr":"172.21.15.150:6829","nonce":38428}]},"public_addr":"172.21.15.150:6825/38428","cluster_addr":"172.21.15.150:6827/38428","heartbeat_back_addr":"172.21.15.150:6831/38428","heartbeat_front_addr":"172.21.15.150:6829/38428","state":["exists","up"]},{"osd":7,"uuid":"a64f57e1-ef98-46bd-b304-385759e95898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":38421},{"type":"v1","addr":"172.21.15.150:6817","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":38421},{"type":"v1","addr":"172.21.15.150:6819","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":38421},{"type":"v1","addr":"172.21.15.150:6823","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":38421},{"type":"v1","addr":"172.21.15.150:6821","nonce":38421}]},"public_addr":"172.21.15.150:6817/38421","cluster_addr":"172.21.15.150:6819/38421","heartbeat_back_addr":"172.21.15.150:6823/38421","heartbeat_front_addr":"172.21.15.150:6821/38421","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.438552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.413280+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.296466+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.424966+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.276953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.230984+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.410235+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.480084+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T02:12:01.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.592+0000 7f616b769640 1 -- 172.21.15.49:0/1592349841 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6148065a60 msgr2=0x7f6148067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:01.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.592+0000 7f616b769640 1 --2- 172.21.15.49:0/1592349841 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6148065a60 0x7f6148067f20 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f6160001270 tx=0x7f6160009040 comp rx=0 tx=0).stop 2023-11-06T02:12:01.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.592+0000 7f616b769640 1 -- 172.21.15.49:0/1592349841 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f616414ed90 msgr2=0x7f61640c68d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:01.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.592+0000 7f616b769640 1 --2- 172.21.15.49:0/1592349841 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f616414ed90 0x7f61640c68d0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f615400d770 tx=0x7f615400dc30 comp rx=0 tx=0).stop 2023-11-06T02:12:01.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.592+0000 7f616b769640 1 -- 172.21.15.49:0/1592349841 shutdown_connections 2023-11-06T02:12:01.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.592+0000 7f616b769640 1 --2- 172.21.15.49:0/1592349841 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6148065a60 0x7f6148067f20 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:01.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.592+0000 7f616b769640 1 --2- 172.21.15.49:0/1592349841 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f616414ed90 0x7f61640c68d0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:01.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.592+0000 7f616b769640 1 --2- 172.21.15.49:0/1592349841 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61640c6e10 0x7f61641533e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:01.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.592+0000 7f616b769640 1 --2- 172.21.15.49:0/1592349841 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6150006f70 0x7f61640b0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:01.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.592+0000 7f616b769640 1 -- 172.21.15.49:0/1592349841 >> 172.21.15.49:0/1592349841 conn(0x7f616406c0a0 msgr2=0x7f61640cc1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:12:01.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.592+0000 7f616b769640 1 -- 172.21.15.49:0/1592349841 shutdown_connections 2023-11-06T02:12:01.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.592+0000 7f616b769640 1 -- 172.21.15.49:0/1592349841 wait complete. 2023-11-06T02:12:01.610 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T02:08:19.252222+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T02:08:24.776211+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T02:11:28.648326+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T02:11:29.079513+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '65', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T02:12:01.611 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T02:12:01.742 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.744+0000 7fa935ef5640 1 Processor -- start 2023-11-06T02:12:01.743 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.744+0000 7fa935ef5640 1 -- start start 2023-11-06T02:12:01.744 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.744+0000 7fa935ef5640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fa9300d2a60 0x7fa9300d2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:01.744 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.744+0000 7fa935ef5640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa9300b5d40 0x7fa9300b61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:01.744 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.744+0000 7fa935ef5640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa93015cf90 con 0x7fa93015cbb0 2023-11-06T02:12:01.745 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.744+0000 7fa935ef5640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9300d3410 con 0x7fa9300b5d40 2023-11-06T02:12:01.745 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.744+0000 7fa935ef5640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9300d0b80 con 0x7fa9300d2a60 2023-11-06T02:12:01.745 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.744+0000 7fa92ffff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa9300b5d40 0x7fa9300b61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:01.745 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.744+0000 7fa92ffff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa9300b5d40 0x7fa9300b61d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39950/0 (socket says 172.21.15.49:39950) 2023-11-06T02:12:01.745 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.744+0000 7fa92ffff640 1 -- 172.21.15.49:0/1933017316 learned_addr learned my addr 172.21.15.49:0/1933017316 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:12:01.746 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.744+0000 7fa92effd640 1 --2- 172.21.15.49:0/1933017316 >> v2:172.21.15.150:3300/0 conn(0x7fa9300d2a60 0x7fa9300d2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:01.746 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.744+0000 7fa92ffff640 1 -- 172.21.15.49:0/1933017316 >> v2:172.21.15.150:3300/0 conn(0x7fa9300d2a60 msgr2=0x7fa9300d2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:01.746 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.744+0000 7fa92ffff640 1 --2- 172.21.15.49:0/1933017316 >> v2:172.21.15.150:3300/0 conn(0x7fa9300d2a60 0x7fa9300d2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:01.746 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.744+0000 7fa92ffff640 1 -- 172.21.15.49:0/1933017316 >> v1:172.21.15.150:6789/0 conn(0x7fa93015cbb0 legacy=0x7fa9300d1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:01.746 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.744+0000 7fa92ffff640 1 -- 172.21.15.49:0/1933017316 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9300d0e60 con 0x7fa9300b5d40 2023-11-06T02:12:01.748 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.748+0000 7fa92ffff640 1 --2- 172.21.15.49:0/1933017316 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9300b5d40 0x7fa9300b61d0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fa9300c65d0 tx=0x7fa92801ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbff467ef8e02bbd server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:01.748 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.748+0000 7fa92e7fc640 1 -- 172.21.15.49:0/1933017316 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9280a89f0 con 0x7fa9300b5d40 2023-11-06T02:12:01.748 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.748+0000 7fa92e7fc640 1 -- 172.21.15.49:0/1933017316 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9280a8b50 con 0x7fa9300b5d40 2023-11-06T02:12:01.748 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.748+0000 7fa92e7fc640 1 -- 172.21.15.49:0/1933017316 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9280b0440 con 0x7fa9300b5d40 2023-11-06T02:12:01.748 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.748+0000 7fa935ef5640 1 -- 172.21.15.49:0/1933017316 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9300b5d40 msgr2=0x7fa9300b61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:01.749 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.748+0000 7fa935ef5640 1 --2- 172.21.15.49:0/1933017316 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9300b5d40 0x7fa9300b61d0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fa9300c65d0 tx=0x7fa92801ecf0 comp rx=0 tx=0).stop 2023-11-06T02:12:01.749 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.748+0000 7fa935ef5640 1 -- 172.21.15.49:0/1933017316 shutdown_connections 2023-11-06T02:12:01.749 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.748+0000 7fa935ef5640 1 --2- 172.21.15.49:0/1933017316 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9300b5d40 0x7fa9300b61d0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:01.749 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.748+0000 7fa935ef5640 1 --2- 172.21.15.49:0/1933017316 >> v2:172.21.15.150:3300/0 conn(0x7fa9300d2a60 0x7fa9300d2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:01.749 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.748+0000 7fa935ef5640 1 -- 172.21.15.49:0/1933017316 >> 172.21.15.49:0/1933017316 conn(0x7fa9300718a0 msgr2=0x7fa930071cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:12:01.750 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.748+0000 7fa935ef5640 1 -- 172.21.15.49:0/1933017316 shutdown_connections 2023-11-06T02:12:01.750 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.748+0000 7fa935ef5640 1 -- 172.21.15.49:0/1933017316 wait complete. 2023-11-06T02:12:01.752 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.748+0000 7fa935ef5640 1 Processor -- start 2023-11-06T02:12:01.752 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.748+0000 7fa935ef5640 1 -- start start 2023-11-06T02:12:01.753 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.752+0000 7fa935ef5640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa9300b5d40 0x7fa930155c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:01.753 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.752+0000 7fa935ef5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa9300d2a60 0x7fa9300c88d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:01.753 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.752+0000 7fa935ef5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa93015cbb0 0x7fa9300ca060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:01.753 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.752+0000 7fa935ef5640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9301562d0 con 0x7fa93015cbb0 2023-11-06T02:12:01.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.752+0000 7fa935ef5640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa930157f40 con 0x7fa9300b5d40 2023-11-06T02:12:01.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.752+0000 7fa935ef5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa930158220 con 0x7fa9300d2a60 2023-11-06T02:12:01.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.752+0000 7fa92ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa93015cbb0 0x7fa9300ca060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:01.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.752+0000 7fa92ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa93015cbb0 0x7fa9300ca060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39954/0 (socket says 172.21.15.49:39954) 2023-11-06T02:12:01.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.752+0000 7fa92ffff640 1 -- 172.21.15.49:0/759237938 learned_addr learned my addr 172.21.15.49:0/759237938 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:12:01.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.752+0000 7fa92f7fe640 1 --2- 172.21.15.49:0/759237938 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa9300b5d40 0x7fa930155c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:01.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.752+0000 7fa92effd640 1 --2- 172.21.15.49:0/759237938 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa9300d2a60 0x7fa9300c88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:01.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.752+0000 7fa92ffff640 1 -- 172.21.15.49:0/759237938 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa9300d2a60 msgr2=0x7fa9300c88d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:01.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.752+0000 7fa92ffff640 1 --2- 172.21.15.49:0/759237938 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa9300d2a60 0x7fa9300c88d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:01.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.752+0000 7fa92ffff640 1 -- 172.21.15.49:0/759237938 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa9300b5d40 msgr2=0x7fa930155c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:01.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.752+0000 7fa92ffff640 1 --2- 172.21.15.49:0/759237938 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa9300b5d40 0x7fa930155c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:01.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.752+0000 7fa92ffff640 1 -- 172.21.15.49:0/759237938 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9280094d0 con 0x7fa93015cbb0 2023-11-06T02:12:01.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.752+0000 7fa92ffff640 1 --2- 172.21.15.49:0/759237938 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa93015cbb0 0x7fa9300ca060 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fa9280b3040 tx=0x7fa92801ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:01.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.752+0000 7fa927fff640 1 -- 172.21.15.49:0/759237938 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9280b0d50 con 0x7fa93015cbb0 2023-11-06T02:12:01.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.752+0000 7fa927fff640 1 -- 172.21.15.49:0/759237938 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9280bb460 con 0x7fa93015cbb0 2023-11-06T02:12:01.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.752+0000 7fa935ef5640 1 -- 172.21.15.49:0/759237938 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa930158500 con 0x7fa93015cbb0 2023-11-06T02:12:01.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.752+0000 7fa927fff640 1 -- 172.21.15.49:0/759237938 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9280bb810 con 0x7fa93015cbb0 2023-11-06T02:12:01.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.752+0000 7fa935ef5640 1 -- 172.21.15.49:0/759237938 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9301494d0 con 0x7fa93015cbb0 2023-11-06T02:12:01.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.752+0000 7fa935ef5640 1 -- 172.21.15.49:0/759237938 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa930157da0 con 0x7fa93015cbb0 2023-11-06T02:12:01.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.752+0000 7fa927fff640 1 -- 172.21.15.49:0/759237938 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fa9280b83f0 con 0x7fa93015cbb0 2023-11-06T02:12:01.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.752+0000 7fa927fff640 1 --2- 172.21.15.49:0/759237938 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa910065a60 0x7fa910067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:01.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.752+0000 7fa927fff640 1 -- 172.21.15.49:0/759237938 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa928120bd0 con 0x7fa93015cbb0 2023-11-06T02:12:01.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.756+0000 7fa92f7fe640 1 --2- 172.21.15.49:0/759237938 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa910065a60 0x7fa910067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:01.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.756+0000 7fa92f7fe640 1 --2- 172.21.15.49:0/759237938 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa910065a60 0x7fa910067f20 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fa918007fd0 tx=0x7fa918009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:01.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.756+0000 7fa927fff640 1 -- 172.21.15.49:0/759237938 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fa930157da0 con 0x7fa93015cbb0 2023-11-06T02:12:01.913 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.916+0000 7fa935ef5640 1 -- 172.21.15.49:0/759237938 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa930157da0 con 0x7fa93015cbb0 2023-11-06T02:12:01.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.916+0000 7fa927fff640 1 -- 172.21.15.49:0/759237938 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v65) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa9280b5020 con 0x7fa93015cbb0 2023-11-06T02:12:01.914 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2023-11-06T02:12:01.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.916+0000 7fa935ef5640 1 -- 172.21.15.49:0/759237938 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa910065a60 msgr2=0x7fa910067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:01.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.916+0000 7fa935ef5640 1 --2- 172.21.15.49:0/759237938 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa910065a60 0x7fa910067f20 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fa918007fd0 tx=0x7fa918009040 comp rx=0 tx=0).stop 2023-11-06T02:12:01.917 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.916+0000 7fa935ef5640 1 -- 172.21.15.49:0/759237938 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa93015cbb0 msgr2=0x7fa9300ca060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:01.917 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.916+0000 7fa935ef5640 1 --2- 172.21.15.49:0/759237938 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa93015cbb0 0x7fa9300ca060 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fa9280b3040 tx=0x7fa92801ba90 comp rx=0 tx=0).stop 2023-11-06T02:12:01.917 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.916+0000 7fa935ef5640 1 -- 172.21.15.49:0/759237938 shutdown_connections 2023-11-06T02:12:01.917 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.916+0000 7fa935ef5640 1 --2- 172.21.15.49:0/759237938 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa910065a60 0x7fa910067f20 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:01.917 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.916+0000 7fa935ef5640 1 --2- 172.21.15.49:0/759237938 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa93015cbb0 0x7fa9300ca060 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:01.918 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.916+0000 7fa935ef5640 1 --2- 172.21.15.49:0/759237938 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa9300d2a60 0x7fa9300c88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:01.918 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.916+0000 7fa935ef5640 1 --2- 172.21.15.49:0/759237938 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa9300b5d40 0x7fa930155c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:01.918 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.916+0000 7fa935ef5640 1 -- 172.21.15.49:0/759237938 >> 172.21.15.49:0/759237938 conn(0x7fa9300718a0 msgr2=0x7fa9300cd0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:12:01.918 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.916+0000 7fa935ef5640 1 -- 172.21.15.49:0/759237938 shutdown_connections 2023-11-06T02:12:01.918 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:01.916+0000 7fa935ef5640 1 -- 172.21.15.49:0/759237938 wait complete. 2023-11-06T02:12:01.934 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T02:12:02.057 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.060+0000 7f4a1f0c5640 1 Processor -- start 2023-11-06T02:12:02.058 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.060+0000 7f4a1f0c5640 1 -- start start 2023-11-06T02:12:02.059 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.060+0000 7f4a1f0c5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4a1815cbb0 0x7f4a180d1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:02.059 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.060+0000 7f4a1f0c5640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a1815cf90 con 0x7f4a180d24e0 2023-11-06T02:12:02.060 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.060+0000 7f4a1f0c5640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a180d0b80 con 0x7f4a180d3170 2023-11-06T02:12:02.060 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.060+0000 7f4a1f0c5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a180d0e60 con 0x7f4a1815cbb0 2023-11-06T02:12:02.060 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.060+0000 7f4a0ffff640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f4a180d24e0 0x7f4a180d2a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:54802/0 (socket says 172.21.15.49:54802) 2023-11-06T02:12:02.060 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.060+0000 7f4a0ffff640 1 -- 172.21.15.49:0/597423651 learned_addr learned my addr 172.21.15.49:0/597423651 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:12:02.060 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.060+0000 7f4a1ce61640 1 --2- 172.21.15.49:0/597423651 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4a1815cbb0 0x7f4a180d1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:02.061 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.060+0000 7f4a0f7fe640 1 -- 172.21.15.49:0/597423651 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1647341461 0 0) 0x7f4a1815cf90 con 0x7f4a180d24e0 2023-11-06T02:12:02.061 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.060+0000 7f4a0f7fe640 1 -- 172.21.15.49:0/597423651 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a00003530 con 0x7f4a180d24e0 2023-11-06T02:12:02.061 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.060+0000 7f4a1ce61640 1 -- 172.21.15.49:0/597423651 >> v1:172.21.15.150:6789/0 conn(0x7f4a180d3170 legacy=0x7f4a180b5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:02.061 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.060+0000 7f4a1ce61640 1 -- 172.21.15.49:0/597423651 >> v1:172.21.15.49:6789/0 conn(0x7f4a180d24e0 legacy=0x7f4a180d2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:02.062 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.060+0000 7f4a0f7fe640 1 -- 172.21.15.49:0/597423651 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2739598281 0 0) 0x7f4a00003530 con 0x7f4a180d24e0 2023-11-06T02:12:02.062 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.060+0000 7f4a1ce61640 1 -- 172.21.15.49:0/597423651 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a180d1140 con 0x7f4a1815cbb0 2023-11-06T02:12:02.063 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.064+0000 7f4a1ce61640 1 --2- 172.21.15.49:0/597423651 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4a1815cbb0 0x7f4a180d1fa0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4a140097f0 tx=0x7f4a1401ebd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c81334f529aa0b89 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:02.063 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.064+0000 7f4a0f7fe640 1 -- 172.21.15.49:0/597423651 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a08001680 con 0x7f4a1815cbb0 2023-11-06T02:12:02.064 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.064+0000 7f4a0f7fe640 1 -- 172.21.15.49:0/597423651 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a140a8650 con 0x7f4a1815cbb0 2023-11-06T02:12:02.064 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.064+0000 7f4a0f7fe640 1 -- 172.21.15.49:0/597423651 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a140a89c0 con 0x7f4a1815cbb0 2023-11-06T02:12:02.064 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.064+0000 7f4a1f0c5640 1 -- 172.21.15.49:0/597423651 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4a1815cbb0 msgr2=0x7f4a180d1fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:02.064 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.064+0000 7f4a1f0c5640 1 --2- 172.21.15.49:0/597423651 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4a1815cbb0 0x7f4a180d1fa0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4a140097f0 tx=0x7f4a1401ebd0 comp rx=0 tx=0).stop 2023-11-06T02:12:02.064 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.064+0000 7f4a1f0c5640 1 -- 172.21.15.49:0/597423651 shutdown_connections 2023-11-06T02:12:02.065 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.064+0000 7f4a1f0c5640 1 --2- 172.21.15.49:0/597423651 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4a1815cbb0 0x7f4a180d1fa0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:02.065 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.064+0000 7f4a1f0c5640 1 -- 172.21.15.49:0/597423651 >> 172.21.15.49:0/597423651 conn(0x7f4a180718a0 msgr2=0x7f4a18071cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:12:02.065 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.064+0000 7f4a1f0c5640 1 -- 172.21.15.49:0/597423651 shutdown_connections 2023-11-06T02:12:02.065 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.064+0000 7f4a1f0c5640 1 -- 172.21.15.49:0/597423651 wait complete. 2023-11-06T02:12:02.068 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.064+0000 7f4a1f0c5640 1 Processor -- start 2023-11-06T02:12:02.068 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.064+0000 7f4a1f0c5640 1 -- start start 2023-11-06T02:12:02.069 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.064+0000 7f4a1f0c5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a180d24e0 0x7f4a180cc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:02.069 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.064+0000 7f4a1f0c5640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a180d3170 0x7f4a181f56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:02.069 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.064+0000 7f4a1f0c5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4a1815cbb0 0x7f4a1814c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:02.069 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.064+0000 7f4a1f0c5640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a18153d80 con 0x7f4a180d24e0 2023-11-06T02:12:02.069 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.064+0000 7f4a1f0c5640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a180cc9f0 con 0x7f4a180d3170 2023-11-06T02:12:02.070 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.064+0000 7f4a1f0c5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a181537c0 con 0x7f4a1815cbb0 2023-11-06T02:12:02.070 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.068+0000 7f4a1ce61640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a180d24e0 0x7f4a180cc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:02.070 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.068+0000 7f4a1ce61640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a180d24e0 0x7f4a180cc4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39958/0 (socket says 172.21.15.49:39958) 2023-11-06T02:12:02.070 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.068+0000 7f4a1ce61640 1 -- 172.21.15.49:0/3746789602 learned_addr learned my addr 172.21.15.49:0/3746789602 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:12:02.071 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.068+0000 7f4a0ffff640 1 --2- 172.21.15.49:0/3746789602 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a180d3170 0x7f4a181f56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:02.071 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.068+0000 7f4a1d662640 1 --2- 172.21.15.49:0/3746789602 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4a1815cbb0 0x7f4a1814c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:02.071 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.068+0000 7f4a1ce61640 1 -- 172.21.15.49:0/3746789602 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4a1815cbb0 msgr2=0x7f4a1814c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:02.071 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.068+0000 7f4a1ce61640 1 --2- 172.21.15.49:0/3746789602 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4a1815cbb0 0x7f4a1814c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:02.071 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.068+0000 7f4a1ce61640 1 -- 172.21.15.49:0/3746789602 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a180d3170 msgr2=0x7f4a181f56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:02.072 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.068+0000 7f4a1ce61640 1 --2- 172.21.15.49:0/3746789602 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a180d3170 0x7f4a181f56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:02.072 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.068+0000 7f4a1ce61640 1 -- 172.21.15.49:0/3746789602 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a14009420 con 0x7f4a180d24e0 2023-11-06T02:12:02.072 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.068+0000 7f4a1ce61640 1 --2- 172.21.15.49:0/3746789602 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a180d24e0 0x7f4a180cc4b0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f4a1401c800 tx=0x7f4a1401b970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:02.072 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.068+0000 7f4a0d7fa640 1 -- 172.21.15.49:0/3746789602 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a140c2070 con 0x7f4a180d24e0 2023-11-06T02:12:02.072 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.068+0000 7f4a0d7fa640 1 -- 172.21.15.49:0/3746789602 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a140a9a60 con 0x7f4a180d24e0 2023-11-06T02:12:02.073 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.068+0000 7f4a1f0c5640 1 -- 172.21.15.49:0/3746789602 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a18153aa0 con 0x7f4a180d24e0 2023-11-06T02:12:02.073 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.068+0000 7f4a0d7fa640 1 -- 172.21.15.49:0/3746789602 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a140a9dd0 con 0x7f4a180d24e0 2023-11-06T02:12:02.073 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.068+0000 7f4a1f0c5640 1 -- 172.21.15.49:0/3746789602 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a180c9fb0 con 0x7f4a180d24e0 2023-11-06T02:12:02.073 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.068+0000 7f4a1f0c5640 1 -- 172.21.15.49:0/3746789602 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a18074040 con 0x7f4a180d24e0 2023-11-06T02:12:02.074 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.068+0000 7f4a0d7fa640 1 -- 172.21.15.49:0/3746789602 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f4a140b7050 con 0x7f4a180d24e0 2023-11-06T02:12:02.074 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.068+0000 7f4a0d7fa640 1 --2- 172.21.15.49:0/3746789602 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4a04065a60 0x7f4a04067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:02.074 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.068+0000 7f4a0d7fa640 1 -- 172.21.15.49:0/3746789602 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4a141206c0 con 0x7f4a180d24e0 2023-11-06T02:12:02.074 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.068+0000 7f4a0ffff640 1 --2- 172.21.15.49:0/3746789602 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4a04065a60 0x7f4a04067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:02.075 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.068+0000 7f4a0ffff640 1 --2- 172.21.15.49:0/3746789602 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4a04065a60 0x7f4a04067f20 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f4a18151de0 tx=0x7f4a08009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:02.075 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.072+0000 7f4a0d7fa640 1 -- 172.21.15.49:0/3746789602 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f4a140ee0d0 con 0x7f4a180d24e0 2023-11-06T02:12:02.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.236+0000 7f4a1f0c5640 1 -- 172.21.15.49:0/3746789602 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f4a1815c2c0 con 0x7f4a180d24e0 2023-11-06T02:12:02.234 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.236+0000 7f4a0d7fa640 1 -- 172.21.15.49:0/3746789602 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v65) v1 ==== 92+0+10 (secure 0 0 0) 0x7f4a140f16f0 con 0x7f4a180d24e0 2023-11-06T02:12:02.234 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 8 2023-11-06T02:12:02.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.236+0000 7f4a1f0c5640 1 -- 172.21.15.49:0/3746789602 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4a04065a60 msgr2=0x7f4a04067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:02.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.236+0000 7f4a1f0c5640 1 --2- 172.21.15.49:0/3746789602 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4a04065a60 0x7f4a04067f20 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f4a18151de0 tx=0x7f4a08009040 comp rx=0 tx=0).stop 2023-11-06T02:12:02.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.236+0000 7f4a1f0c5640 1 -- 172.21.15.49:0/3746789602 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a180d24e0 msgr2=0x7f4a180cc4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:02.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.236+0000 7f4a1f0c5640 1 --2- 172.21.15.49:0/3746789602 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a180d24e0 0x7f4a180cc4b0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f4a1401c800 tx=0x7f4a1401b970 comp rx=0 tx=0).stop 2023-11-06T02:12:02.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.236+0000 7f4a1f0c5640 1 -- 172.21.15.49:0/3746789602 shutdown_connections 2023-11-06T02:12:02.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.236+0000 7f4a1f0c5640 1 --2- 172.21.15.49:0/3746789602 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4a04065a60 0x7f4a04067f20 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:02.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.236+0000 7f4a1f0c5640 1 --2- 172.21.15.49:0/3746789602 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4a1815cbb0 0x7f4a1814c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:02.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.236+0000 7f4a1f0c5640 1 --2- 172.21.15.49:0/3746789602 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a180d3170 0x7f4a181f56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:02.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.236+0000 7f4a1f0c5640 1 --2- 172.21.15.49:0/3746789602 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4a180d24e0 0x7f4a180cc4b0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:02.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.236+0000 7f4a1f0c5640 1 -- 172.21.15.49:0/3746789602 >> 172.21.15.49:0/3746789602 conn(0x7f4a180718a0 msgr2=0x7f4a18150d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:12:02.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.236+0000 7f4a1f0c5640 1 -- 172.21.15.49:0/3746789602 shutdown_connections 2023-11-06T02:12:02.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.236+0000 7f4a1f0c5640 1 -- 172.21.15.49:0/3746789602 wait complete. 2023-11-06T02:12:02.253 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T02:12:02.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.376+0000 7f6a9c7f4640 1 Processor -- start 2023-11-06T02:12:02.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.376+0000 7f6a9c7f4640 1 -- start start 2023-11-06T02:12:02.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.376+0000 7f6a9c7f4640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6a9415a880 0x7f6a940b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:02.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.376+0000 7f6a9c7f4640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a940ae360 con 0x7f6a9406d860 2023-11-06T02:12:02.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.376+0000 7f6a9c7f4640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a940ae640 con 0x7f6a94070160 2023-11-06T02:12:02.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.376+0000 7f6a9c7f4640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a940ae920 con 0x7f6a9415a880 2023-11-06T02:12:02.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.376+0000 7f6a9ad91640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f6a9406d860 0x7f6a940af120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:54808/0 (socket says 172.21.15.49:54808) 2023-11-06T02:12:02.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.376+0000 7f6a9ad91640 1 -- 172.21.15.49:0/206524037 learned_addr learned my addr 172.21.15.49:0/206524037 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:12:02.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.376+0000 7f6a9a590640 1 --2- 172.21.15.49:0/206524037 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6a9415a880 0x7f6a940b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:02.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.376+0000 7f6a9958e640 1 -- 172.21.15.49:0/206524037 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2265758856 0 0) 0x7f6a940ae360 con 0x7f6a9406d860 2023-11-06T02:12:02.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.376+0000 7f6a9958e640 1 -- 172.21.15.49:0/206524037 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a80003530 con 0x7f6a9406d860 2023-11-06T02:12:02.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.376+0000 7f6a9958e640 1 -- 172.21.15.49:0/206524037 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3721574701 0 0) 0x7f6a80003530 con 0x7f6a9406d860 2023-11-06T02:12:02.378 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.376+0000 7f6a9958e640 1 -- 172.21.15.49:0/206524037 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6a9415a880 msgr2=0x7f6a940b15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:02.378 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.376+0000 7f6a9958e640 1 --2- 172.21.15.49:0/206524037 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6a9415a880 0x7f6a940b15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:02.378 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.376+0000 7f6a9958e640 1 -- 172.21.15.49:0/206524037 >> v1:172.21.15.150:6789/0 conn(0x7f6a94070160 legacy=0x7f6a9406d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:02.378 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.376+0000 7f6a9958e640 1 -- 172.21.15.49:0/206524037 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a940aec00 con 0x7f6a9406d860 2023-11-06T02:12:02.378 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.376+0000 7f6a9958e640 1 -- 172.21.15.49:0/206524037 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7f6a8c001a00 con 0x7f6a9406d860 2023-11-06T02:12:02.379 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.376+0000 7f6a9958e640 1 -- 172.21.15.49:0/206524037 >> v1:172.21.15.49:6789/0 conn(0x7f6a9406d860 legacy=0x7f6a940af120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:02.379 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.376+0000 7f6a9958e640 1 --2- 172.21.15.49:0/206524037 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6a9415a880 0x7f6a800039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:02.379 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.376+0000 7f6a9958e640 1 --2- 172.21.15.49:0/206524037 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a80003f30 0x7f6a800047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:02.379 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.376+0000 7f6a9958e640 1 --2- 172.21.15.49:0/206524037 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a80004d10 0x7f6a80005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:02.379 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.376+0000 7f6a9958e640 1 -- 172.21.15.49:0/206524037 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a940ae920 con 0x7f6a80003f30 2023-11-06T02:12:02.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.376+0000 7f6a9958e640 1 -- 172.21.15.49:0/206524037 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a8000ba70 con 0x7f6a80004d10 2023-11-06T02:12:02.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.376+0000 7f6a9958e640 1 -- 172.21.15.49:0/206524037 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a8000bc30 con 0x7f6a9415a880 2023-11-06T02:12:02.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.376+0000 7f6a99d8f640 1 --2- 172.21.15.49:0/206524037 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a80003f30 0x7f6a800047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:02.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.376+0000 7f6a9ad91640 1 --2- 172.21.15.49:0/206524037 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a80004d10 0x7f6a80005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:02.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.376+0000 7f6a99d8f640 1 -- 172.21.15.49:0/206524037 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6a9415a880 msgr2=0x7f6a800039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:02.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.376+0000 7f6a99d8f640 1 --2- 172.21.15.49:0/206524037 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6a9415a880 0x7f6a800039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:02.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.376+0000 7f6a99d8f640 1 -- 172.21.15.49:0/206524037 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a80004d10 msgr2=0x7f6a80005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:02.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.376+0000 7f6a99d8f640 1 --2- 172.21.15.49:0/206524037 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a80004d10 0x7f6a80005190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:02.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.376+0000 7f6a99d8f640 1 -- 172.21.15.49:0/206524037 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a80003530 con 0x7f6a80003f30 2023-11-06T02:12:02.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.380+0000 7f6a99d8f640 1 --2- 172.21.15.49:0/206524037 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a80003f30 0x7f6a800047d0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f6a84001780 tx=0x7f6a8401c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd04faadcfbae316 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:02.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.380+0000 7f6a9958e640 1 -- 172.21.15.49:0/206524037 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a840052d0 con 0x7f6a80003f30 2023-11-06T02:12:02.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.380+0000 7f6a9958e640 1 -- 172.21.15.49:0/206524037 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a840a7050 con 0x7f6a80003f30 2023-11-06T02:12:02.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.380+0000 7f6a9958e640 1 -- 172.21.15.49:0/206524037 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a84003600 con 0x7f6a80003f30 2023-11-06T02:12:02.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.380+0000 7f6a9c7f4640 1 -- 172.21.15.49:0/206524037 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a80003f30 msgr2=0x7f6a800047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:02.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.380+0000 7f6a9c7f4640 1 --2- 172.21.15.49:0/206524037 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a80003f30 0x7f6a800047d0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f6a84001780 tx=0x7f6a8401c400 comp rx=0 tx=0).stop 2023-11-06T02:12:02.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.380+0000 7f6a9ad91640 1 -- 172.21.15.49:0/206524037 reap_dead start 2023-11-06T02:12:02.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.380+0000 7f6a9c7f4640 1 -- 172.21.15.49:0/206524037 shutdown_connections 2023-11-06T02:12:02.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.380+0000 7f6a9c7f4640 1 -- 172.21.15.49:0/206524037 >> 172.21.15.49:0/206524037 conn(0x7f6a9406a370 msgr2=0x7f6a940ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:12:02.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.380+0000 7f6a9c7f4640 1 -- 172.21.15.49:0/206524037 shutdown_connections 2023-11-06T02:12:02.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.380+0000 7f6a9c7f4640 1 -- 172.21.15.49:0/206524037 wait complete. 2023-11-06T02:12:02.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.380+0000 7f6a9c7f4640 1 Processor -- start 2023-11-06T02:12:02.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.380+0000 7f6a9c7f4640 1 -- start start 2023-11-06T02:12:02.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.380+0000 7f6a9c7f4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a80003f30 0x7f6a94152c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:02.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.380+0000 7f6a9c7f4640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a940b3180 0x7f6a9414afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:02.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.380+0000 7f6a9c7f4640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6a940c9b10 0x7f6a940c9f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:02.389 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.380+0000 7f6a9c7f4640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a940ca4d0 con 0x7f6a80003f30 2023-11-06T02:12:02.389 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.380+0000 7f6a9c7f4640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a94153430 con 0x7f6a940b3180 2023-11-06T02:12:02.389 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.380+0000 7f6a9c7f4640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a94153710 con 0x7f6a940c9b10 2023-11-06T02:12:02.389 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.380+0000 7f6a9a590640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a80003f30 0x7f6a94152c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:02.389 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.380+0000 7f6a9a590640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a80003f30 0x7f6a94152c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39976/0 (socket says 172.21.15.49:39976) 2023-11-06T02:12:02.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.380+0000 7f6a9a590640 1 -- 172.21.15.49:0/2687764763 learned_addr learned my addr 172.21.15.49:0/2687764763 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:12:02.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.384+0000 7f6a9ad91640 1 --2- 172.21.15.49:0/2687764763 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6a940c9b10 0x7f6a940c9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:02.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.384+0000 7f6a99d8f640 1 --2- 172.21.15.49:0/2687764763 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a940b3180 0x7f6a9414afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:02.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.384+0000 7f6a9a590640 1 -- 172.21.15.49:0/2687764763 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6a940c9b10 msgr2=0x7f6a940c9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:02.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.384+0000 7f6a9a590640 1 --2- 172.21.15.49:0/2687764763 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6a940c9b10 0x7f6a940c9f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:02.391 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.384+0000 7f6a9a590640 1 -- 172.21.15.49:0/2687764763 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a940b3180 msgr2=0x7f6a9414afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:02.391 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.384+0000 7f6a9a590640 1 --2- 172.21.15.49:0/2687764763 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a940b3180 0x7f6a9414afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:02.391 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.384+0000 7f6a9a590640 1 -- 172.21.15.49:0/2687764763 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a9415ade0 con 0x7f6a80003f30 2023-11-06T02:12:02.391 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.384+0000 7f6a9a590640 1 --2- 172.21.15.49:0/2687764763 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a80003f30 0x7f6a94152c00 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f6a9000e7f0 tx=0x7f6a9000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:02.391 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.384+0000 7f6a8affd640 1 -- 172.21.15.49:0/2687764763 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a9000c940 con 0x7f6a80003f30 2023-11-06T02:12:02.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.384+0000 7f6a8affd640 1 -- 172.21.15.49:0/2687764763 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a9000caa0 con 0x7f6a80003f30 2023-11-06T02:12:02.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.384+0000 7f6a9c7f4640 1 -- 172.21.15.49:0/2687764763 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a941539f0 con 0x7f6a80003f30 2023-11-06T02:12:02.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.384+0000 7f6a8affd640 1 -- 172.21.15.49:0/2687764763 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a9000ce50 con 0x7f6a80003f30 2023-11-06T02:12:02.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.384+0000 7f6a9c7f4640 1 -- 172.21.15.49:0/2687764763 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a94153fb0 con 0x7f6a80003f30 2023-11-06T02:12:02.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.384+0000 7f6a9c7f4640 1 -- 172.21.15.49:0/2687764763 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a94074040 con 0x7f6a80003f30 2023-11-06T02:12:02.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.384+0000 7f6a8affd640 1 -- 172.21.15.49:0/2687764763 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f6a90013470 con 0x7f6a80003f30 2023-11-06T02:12:02.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.384+0000 7f6a8affd640 1 --2- 172.21.15.49:0/2687764763 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6a78065a60 0x7f6a78067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:02.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.384+0000 7f6a8affd640 1 -- 172.21.15.49:0/2687764763 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6a90083790 con 0x7f6a80003f30 2023-11-06T02:12:02.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.384+0000 7f6a99d8f640 1 --2- 172.21.15.49:0/2687764763 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6a78065a60 0x7f6a78067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:02.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.384+0000 7f6a99d8f640 1 --2- 172.21.15.49:0/2687764763 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6a78065a60 0x7f6a78067f20 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f6a84004000 tx=0x7f6a840011a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:02.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.388+0000 7f6a8affd640 1 -- 172.21.15.49:0/2687764763 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f6a90050ff0 con 0x7f6a80003f30 2023-11-06T02:12:02.537 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.540+0000 7f6a9c7f4640 1 -- 172.21.15.49:0/2687764763 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6a940b42a0 con 0x7f6a80003f30 2023-11-06T02:12:02.537 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.540+0000 7f6a8affd640 1 -- 172.21.15.49:0/2687764763 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v65) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6a90054610 con 0x7f6a80003f30 2023-11-06T02:12:02.538 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2023-11-06T02:12:02.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.540+0000 7f6a9c7f4640 1 -- 172.21.15.49:0/2687764763 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6a78065a60 msgr2=0x7f6a78067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:02.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.540+0000 7f6a9c7f4640 1 --2- 172.21.15.49:0/2687764763 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6a78065a60 0x7f6a78067f20 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f6a84004000 tx=0x7f6a840011a0 comp rx=0 tx=0).stop 2023-11-06T02:12:02.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.540+0000 7f6a9c7f4640 1 -- 172.21.15.49:0/2687764763 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a80003f30 msgr2=0x7f6a94152c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:02.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.540+0000 7f6a9c7f4640 1 --2- 172.21.15.49:0/2687764763 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a80003f30 0x7f6a94152c00 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f6a9000e7f0 tx=0x7f6a9000ecb0 comp rx=0 tx=0).stop 2023-11-06T02:12:02.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.540+0000 7f6a9c7f4640 1 -- 172.21.15.49:0/2687764763 shutdown_connections 2023-11-06T02:12:02.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.540+0000 7f6a9c7f4640 1 --2- 172.21.15.49:0/2687764763 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6a78065a60 0x7f6a78067f20 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:02.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.540+0000 7f6a9c7f4640 1 --2- 172.21.15.49:0/2687764763 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6a940c9b10 0x7f6a940c9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:02.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.540+0000 7f6a9c7f4640 1 --2- 172.21.15.49:0/2687764763 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a940b3180 0x7f6a9414afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:02.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.540+0000 7f6a9c7f4640 1 --2- 172.21.15.49:0/2687764763 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a80003f30 0x7f6a94152c00 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:02.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.540+0000 7f6a9c7f4640 1 -- 172.21.15.49:0/2687764763 >> 172.21.15.49:0/2687764763 conn(0x7f6a9406a370 msgr2=0x7f6a9415d820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:12:02.542 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.540+0000 7f6a9c7f4640 1 -- 172.21.15.49:0/2687764763 shutdown_connections 2023-11-06T02:12:02.542 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.540+0000 7f6a9c7f4640 1 -- 172.21.15.49:0/2687764763 wait complete. 2023-11-06T02:12:02.555 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T02:12:02.669 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.672+0000 7fabcbe12640 1 Processor -- start 2023-11-06T02:12:02.669 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.672+0000 7fabcbe12640 1 -- start start 2023-11-06T02:12:02.670 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.672+0000 7fabcbe12640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fabc415a880 0x7fabc40b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:02.671 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.672+0000 7fabcbe12640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fabc4070160 0x7fabc40705e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:02.671 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.672+0000 7fabcbe12640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabc40ae360 con 0x7fabc406d150 2023-11-06T02:12:02.671 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.672+0000 7fabcbe12640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabc4070b20 con 0x7fabc4070160 2023-11-06T02:12:02.671 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.672+0000 7fabcbe12640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabc40ae640 con 0x7fabc415a880 2023-11-06T02:12:02.671 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.672+0000 7fabc9bae640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fabc415a880 0x7fabc40b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:02.672 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.672+0000 7fabc9bae640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fabc415a880 0x7fabc40b15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.49:55066/0 (socket says 172.21.15.49:55066) 2023-11-06T02:12:02.672 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.672+0000 7fabc9bae640 1 -- 172.21.15.49:0/4254959185 learned_addr learned my addr 172.21.15.49:0/4254959185 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:12:02.672 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.672+0000 7fabc93ad640 1 --2- 172.21.15.49:0/4254959185 >> v2:172.21.15.150:3300/0 conn(0x7fabc4070160 0x7fabc40705e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:02.673 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.672+0000 7fabc9bae640 1 -- 172.21.15.49:0/4254959185 >> v2:172.21.15.150:3300/0 conn(0x7fabc4070160 msgr2=0x7fabc40705e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:02.673 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.672+0000 7fabc9bae640 1 --2- 172.21.15.49:0/4254959185 >> v2:172.21.15.150:3300/0 conn(0x7fabc4070160 0x7fabc40705e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:02.673 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.672+0000 7fabc9bae640 1 -- 172.21.15.49:0/4254959185 >> v1:172.21.15.150:6789/0 conn(0x7fabc406d150 legacy=0x7fabc406d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:02.673 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.672+0000 7fabc9bae640 1 -- 172.21.15.49:0/4254959185 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabc40ae920 con 0x7fabc415a880 2023-11-06T02:12:02.674 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.676+0000 7fabc9bae640 1 --2- 172.21.15.49:0/4254959185 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fabc415a880 0x7fabc40b15c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fabc0009870 tx=0x7fabc001c540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab4fa404c5f673b6 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:02.674 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.676+0000 7fabc8bac640 1 -- 172.21.15.49:0/4254959185 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc00ab650 con 0x7fabc415a880 2023-11-06T02:12:02.675 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.676+0000 7fabc8bac640 1 -- 172.21.15.49:0/4254959185 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabc00ab7b0 con 0x7fabc415a880 2023-11-06T02:12:02.675 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.676+0000 7fabc8bac640 1 -- 172.21.15.49:0/4254959185 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc00abb40 con 0x7fabc415a880 2023-11-06T02:12:02.676 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.676+0000 7fabcbe12640 1 -- 172.21.15.49:0/4254959185 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fabc415a880 msgr2=0x7fabc40b15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:02.676 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.676+0000 7fabcbe12640 1 --2- 172.21.15.49:0/4254959185 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fabc415a880 0x7fabc40b15c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fabc0009870 tx=0x7fabc001c540 comp rx=0 tx=0).stop 2023-11-06T02:12:02.676 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.676+0000 7fabcbe12640 1 -- 172.21.15.49:0/4254959185 shutdown_connections 2023-11-06T02:12:02.676 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.676+0000 7fabcbe12640 1 --2- 172.21.15.49:0/4254959185 >> v2:172.21.15.150:3300/0 conn(0x7fabc4070160 0x7fabc40705e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:02.677 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.676+0000 7fabcbe12640 1 --2- 172.21.15.49:0/4254959185 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fabc415a880 0x7fabc40b15c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:02.677 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.676+0000 7fabcbe12640 1 -- 172.21.15.49:0/4254959185 >> 172.21.15.49:0/4254959185 conn(0x7fabc406a370 msgr2=0x7fabc40ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:12:02.677 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.676+0000 7fabcbe12640 1 -- 172.21.15.49:0/4254959185 shutdown_connections 2023-11-06T02:12:02.677 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.676+0000 7fabcbe12640 1 -- 172.21.15.49:0/4254959185 wait complete. 2023-11-06T02:12:02.679 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.676+0000 7fabcbe12640 1 Processor -- start 2023-11-06T02:12:02.679 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.676+0000 7fabcbe12640 1 -- start start 2023-11-06T02:12:02.680 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.680+0000 7fabcbe12640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fabc406d150 0x7fabc414c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:02.680 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.680+0000 7fabcbe12640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fabc4070160 0x7fabc415c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:02.680 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.680+0000 7fabcbe12640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fabc415a880 0x7fabc40cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:02.680 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.680+0000 7fabcbe12640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc415ae70 con 0x7fabc415a880 2023-11-06T02:12:02.680 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.680+0000 7fabcbe12640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc4157040 con 0x7fabc4070160 2023-11-06T02:12:02.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.680+0000 7fabcbe12640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabc415b0c0 con 0x7fabc406d150 2023-11-06T02:12:02.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.680+0000 7fabca3af640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fabc415a880 0x7fabc40cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:02.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.680+0000 7fabca3af640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fabc415a880 0x7fabc40cd7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39986/0 (socket says 172.21.15.49:39986) 2023-11-06T02:12:02.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.680+0000 7fabca3af640 1 -- 172.21.15.49:0/4002476630 learned_addr learned my addr 172.21.15.49:0/4002476630 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:12:02.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.680+0000 7fabc9bae640 1 --2- 172.21.15.49:0/4002476630 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fabc406d150 0x7fabc414c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:02.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.680+0000 7fabc93ad640 1 --2- 172.21.15.49:0/4002476630 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fabc4070160 0x7fabc415c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:02.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.680+0000 7fabca3af640 1 -- 172.21.15.49:0/4002476630 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fabc406d150 msgr2=0x7fabc414c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:02.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.680+0000 7fabca3af640 1 --2- 172.21.15.49:0/4002476630 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fabc406d150 0x7fabc414c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:02.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.680+0000 7fabca3af640 1 -- 172.21.15.49:0/4002476630 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fabc4070160 msgr2=0x7fabc415c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:02.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.680+0000 7fabca3af640 1 --2- 172.21.15.49:0/4002476630 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fabc4070160 0x7fabc415c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:02.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.680+0000 7fabca3af640 1 -- 172.21.15.49:0/4002476630 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabc00094d0 con 0x7fabc415a880 2023-11-06T02:12:02.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.680+0000 7fabca3af640 1 --2- 172.21.15.49:0/4002476630 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fabc415a880 0x7fabc40cd7f0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fabbc00c770 tx=0x7fabbc00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:02.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.680+0000 7fabba7fc640 1 -- 172.21.15.49:0/4002476630 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabbc003b80 con 0x7fabc415a880 2023-11-06T02:12:02.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.680+0000 7fabba7fc640 1 -- 172.21.15.49:0/4002476630 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabbc003ce0 con 0x7fabc415a880 2023-11-06T02:12:02.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.680+0000 7fabcbe12640 1 -- 172.21.15.49:0/4002476630 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabc415b3a0 con 0x7fabc415a880 2023-11-06T02:12:02.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.680+0000 7fabba7fc640 1 -- 172.21.15.49:0/4002476630 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabbc00aa90 con 0x7fabc415a880 2023-11-06T02:12:02.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.680+0000 7fabcbe12640 1 -- 172.21.15.49:0/4002476630 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabc415b960 con 0x7fabc415a880 2023-11-06T02:12:02.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.680+0000 7fabcbe12640 1 -- 172.21.15.49:0/4002476630 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabc4074040 con 0x7fabc415a880 2023-11-06T02:12:02.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.680+0000 7fabba7fc640 1 -- 172.21.15.49:0/4002476630 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fabbc00abf0 con 0x7fabc415a880 2023-11-06T02:12:02.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.680+0000 7fabba7fc640 1 --2- 172.21.15.49:0/4002476630 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7faba8065a60 0x7faba8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:02.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.680+0000 7fabba7fc640 1 -- 172.21.15.49:0/4002476630 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fabbc083130 con 0x7fabc415a880 2023-11-06T02:12:02.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.680+0000 7fabc9bae640 1 --2- 172.21.15.49:0/4002476630 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7faba8065a60 0x7faba8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:02.686 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.684+0000 7fabc9bae640 1 --2- 172.21.15.49:0/4002476630 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7faba8065a60 0x7faba8067f20 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fabc00af000 tx=0x7fabc00a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:02.686 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.684+0000 7fabba7fc640 1 -- 172.21.15.49:0/4002476630 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fabbc050990 con 0x7fabc415a880 2023-11-06T02:12:02.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.840+0000 7fabcbe12640 1 -- 172.21.15.49:0/4002476630 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fabc406c480 con 0x7fabc415a880 2023-11-06T02:12:02.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.840+0000 7fabba7fc640 1 -- 172.21.15.49:0/4002476630 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v65) v1 ==== 100+0+11 (secure 0 0 0) 0x7fabbc053fb0 con 0x7fabc415a880 2023-11-06T02:12:02.839 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2023-11-06T02:12:02.841 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.840+0000 7fabcbe12640 1 -- 172.21.15.49:0/4002476630 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7faba8065a60 msgr2=0x7faba8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:02.841 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.840+0000 7fabcbe12640 1 --2- 172.21.15.49:0/4002476630 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7faba8065a60 0x7faba8067f20 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fabc00af000 tx=0x7fabc00a7000 comp rx=0 tx=0).stop 2023-11-06T02:12:02.841 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.840+0000 7fabcbe12640 1 -- 172.21.15.49:0/4002476630 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fabc415a880 msgr2=0x7fabc40cd7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:02.841 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.840+0000 7fabcbe12640 1 --2- 172.21.15.49:0/4002476630 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fabc415a880 0x7fabc40cd7f0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fabbc00c770 tx=0x7fabbc00cc30 comp rx=0 tx=0).stop 2023-11-06T02:12:02.842 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.840+0000 7fabcbe12640 1 -- 172.21.15.49:0/4002476630 shutdown_connections 2023-11-06T02:12:02.842 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.840+0000 7fabcbe12640 1 --2- 172.21.15.49:0/4002476630 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7faba8065a60 0x7faba8067f20 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:02.842 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.840+0000 7fabcbe12640 1 --2- 172.21.15.49:0/4002476630 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fabc415a880 0x7fabc40cd7f0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:02.842 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.840+0000 7fabcbe12640 1 --2- 172.21.15.49:0/4002476630 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fabc4070160 0x7fabc415c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:02.842 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.840+0000 7fabcbe12640 1 --2- 172.21.15.49:0/4002476630 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fabc406d150 0x7fabc414c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:02.843 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.840+0000 7fabcbe12640 1 -- 172.21.15.49:0/4002476630 >> 172.21.15.49:0/4002476630 conn(0x7fabc406a370 msgr2=0x7fabc414c8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:12:02.843 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.840+0000 7fabcbe12640 1 -- 172.21.15.49:0/4002476630 shutdown_connections 2023-11-06T02:12:02.843 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.840+0000 7fabcbe12640 1 -- 172.21.15.49:0/4002476630 wait complete. 2023-11-06T02:12:02.857 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-06T02:12:02.982 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.984+0000 7f5b6b5ce640 1 Processor -- start 2023-11-06T02:12:02.983 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.984+0000 7f5b6b5ce640 1 -- start start 2023-11-06T02:12:02.984 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.984+0000 7f5b6b5ce640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5b64159b40 0x7f5b640ca980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:02.984 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.984+0000 7f5b6b5ce640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f5b640cdd50 0x7f5b640cc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:02.984 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.984+0000 7f5b6b5ce640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b640723a0 con 0x7f5b640cb010 2023-11-06T02:12:02.985 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.984+0000 7f5b6b5ce640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b640715e0 con 0x7f5b640cdd50 2023-11-06T02:12:02.985 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.984+0000 7f5b6b5ce640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b640718c0 con 0x7f5b64159b40 2023-11-06T02:12:02.985 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.984+0000 7f5b6936a640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f5b640cb010 0x7f5b640cb3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:54824/0 (socket says 172.21.15.49:54824) 2023-11-06T02:12:02.985 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.984+0000 7f5b6936a640 1 -- 172.21.15.49:0/321095625 learned_addr learned my addr 172.21.15.49:0/321095625 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:12:02.985 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.984+0000 7f5b69b6b640 1 --2- 172.21.15.49:0/321095625 >> v2:172.21.15.49:3300/0 conn(0x7f5b640cdd50 0x7f5b640cc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:02.986 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.984+0000 7f5b68b69640 1 --2- 172.21.15.49:0/321095625 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5b64159b40 0x7f5b640ca980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:02.986 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.984+0000 7f5b5bfff640 1 -- 172.21.15.49:0/321095625 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4281153891 0 0) 0x7f5b640723a0 con 0x7f5b640cb010 2023-11-06T02:12:02.986 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.984+0000 7f5b5bfff640 1 -- 172.21.15.49:0/321095625 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b50003530 con 0x7f5b640cb010 2023-11-06T02:12:02.986 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.984+0000 7f5b69b6b640 1 -- 172.21.15.49:0/321095625 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5b64159b40 msgr2=0x7f5b640ca980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:02.986 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.984+0000 7f5b69b6b640 1 --2- 172.21.15.49:0/321095625 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5b64159b40 0x7f5b640ca980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:02.987 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.984+0000 7f5b69b6b640 1 -- 172.21.15.49:0/321095625 >> v1:172.21.15.49:6789/0 conn(0x7f5b640cb010 legacy=0x7f5b640cb3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:02.987 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.984+0000 7f5b69b6b640 1 -- 172.21.15.49:0/321095625 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b64071ba0 con 0x7f5b640cdd50 2023-11-06T02:12:02.988 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.988+0000 7f5b69b6b640 1 --2- 172.21.15.49:0/321095625 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b640cdd50 0x7f5b640cc1d0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c009d90 tx=0x7f5b5c01c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=342808580fd82a3c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:02.988 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.988+0000 7f5b5bfff640 1 -- 172.21.15.49:0/321095625 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b5c0a5a00 con 0x7f5b640cdd50 2023-11-06T02:12:02.989 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.988+0000 7f5b5bfff640 1 -- 172.21.15.49:0/321095625 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b5c0a5b60 con 0x7f5b640cdd50 2023-11-06T02:12:02.989 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.988+0000 7f5b5bfff640 1 -- 172.21.15.49:0/321095625 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b5c0ad440 con 0x7f5b640cdd50 2023-11-06T02:12:02.989 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.988+0000 7f5b6b5ce640 1 -- 172.21.15.49:0/321095625 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b640cdd50 msgr2=0x7f5b640cc1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:02.989 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.988+0000 7f5b6b5ce640 1 --2- 172.21.15.49:0/321095625 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b640cdd50 0x7f5b640cc1d0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c009d90 tx=0x7f5b5c01c920 comp rx=0 tx=0).stop 2023-11-06T02:12:02.989 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.988+0000 7f5b6b5ce640 1 -- 172.21.15.49:0/321095625 shutdown_connections 2023-11-06T02:12:02.990 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.988+0000 7f5b6b5ce640 1 --2- 172.21.15.49:0/321095625 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b640cdd50 0x7f5b640cc1d0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:02.990 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.988+0000 7f5b6b5ce640 1 --2- 172.21.15.49:0/321095625 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5b64159b40 0x7f5b640ca980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:02.990 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.988+0000 7f5b6b5ce640 1 -- 172.21.15.49:0/321095625 >> 172.21.15.49:0/321095625 conn(0x7f5b6406c0a0 msgr2=0x7f5b6406c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:12:02.990 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.988+0000 7f5b6b5ce640 1 -- 172.21.15.49:0/321095625 shutdown_connections 2023-11-06T02:12:02.990 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.988+0000 7f5b6b5ce640 1 -- 172.21.15.49:0/321095625 wait complete. 2023-11-06T02:12:02.990 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.988+0000 7f5b6b5ce640 1 Processor -- start 2023-11-06T02:12:02.991 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.988+0000 7f5b6b5ce640 1 -- start start 2023-11-06T02:12:02.993 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.992+0000 7f5b6b5ce640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b640cb010 0x7f5b64149160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:02.993 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.992+0000 7f5b6b5ce640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b640cdd50 0x7f5b640b2550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:02.994 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.992+0000 7f5b6b5ce640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5b64159b40 0x7f5b64154e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:02.994 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.992+0000 7f5b6b5ce640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b6414d080 con 0x7f5b640cdd50 2023-11-06T02:12:02.994 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.992+0000 7f5b6b5ce640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b640aca30 con 0x7f5b640cb010 2023-11-06T02:12:02.994 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.992+0000 7f5b6b5ce640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b640acd10 con 0x7f5b64159b40 2023-11-06T02:12:02.994 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.992+0000 7f5b68b69640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b640cdd50 0x7f5b640b2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:02.995 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.992+0000 7f5b68b69640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b640cdd50 0x7f5b640b2550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:40006/0 (socket says 172.21.15.49:40006) 2023-11-06T02:12:02.995 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.992+0000 7f5b68b69640 1 -- 172.21.15.49:0/1443817513 learned_addr learned my addr 172.21.15.49:0/1443817513 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:12:02.995 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.992+0000 7f5b6936a640 1 --2- 172.21.15.49:0/1443817513 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b640cb010 0x7f5b64149160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:02.995 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.992+0000 7f5b69b6b640 1 --2- 172.21.15.49:0/1443817513 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5b64159b40 0x7f5b64154e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:02.995 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.992+0000 7f5b68b69640 1 -- 172.21.15.49:0/1443817513 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5b64159b40 msgr2=0x7f5b64154e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:02.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.992+0000 7f5b68b69640 1 --2- 172.21.15.49:0/1443817513 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5b64159b40 0x7f5b64154e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:02.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.992+0000 7f5b68b69640 1 -- 172.21.15.49:0/1443817513 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b640cb010 msgr2=0x7f5b64149160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:02.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.992+0000 7f5b68b69640 1 --2- 172.21.15.49:0/1443817513 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b640cb010 0x7f5b64149160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:02.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.992+0000 7f5b68b69640 1 -- 172.21.15.49:0/1443817513 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b5c009a70 con 0x7f5b640cdd50 2023-11-06T02:12:02.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.992+0000 7f5b68b69640 1 --2- 172.21.15.49:0/1443817513 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b640cdd50 0x7f5b640b2550 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f5b5400d5f0 tx=0x7f5b5400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:02.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.992+0000 7f5b59ffb640 1 -- 172.21.15.49:0/1443817513 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b540125f0 con 0x7f5b640cdd50 2023-11-06T02:12:02.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.992+0000 7f5b59ffb640 1 -- 172.21.15.49:0/1443817513 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b54012750 con 0x7f5b640cdd50 2023-11-06T02:12:02.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.992+0000 7f5b6b5ce640 1 -- 172.21.15.49:0/1443817513 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b640acff0 con 0x7f5b640cdd50 2023-11-06T02:12:02.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.992+0000 7f5b59ffb640 1 -- 172.21.15.49:0/1443817513 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b54012ae0 con 0x7f5b640cdd50 2023-11-06T02:12:02.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.992+0000 7f5b6b5ce640 1 -- 172.21.15.49:0/1443817513 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b640ad2d0 con 0x7f5b640cdd50 2023-11-06T02:12:02.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.992+0000 7f5b6b5ce640 1 -- 172.21.15.49:0/1443817513 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b64074040 con 0x7f5b640cdd50 2023-11-06T02:12:02.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.992+0000 7f5b59ffb640 1 -- 172.21.15.49:0/1443817513 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f5b54012c40 con 0x7f5b640cdd50 2023-11-06T02:12:02.999 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.992+0000 7f5b59ffb640 1 --2- 172.21.15.49:0/1443817513 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5b48065a60 0x7f5b48067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:02.999 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.992+0000 7f5b59ffb640 1 -- 172.21.15.49:0/1443817513 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5b54083760 con 0x7f5b640cdd50 2023-11-06T02:12:02.999 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.992+0000 7f5b6936a640 1 --2- 172.21.15.49:0/1443817513 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5b48065a60 0x7f5b48067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:02.999 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.996+0000 7f5b6936a640 1 --2- 172.21.15.49:0/1443817513 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5b48065a60 0x7f5b48067f20 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f5b60007fd0 tx=0x7f5b60009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:03.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:02.996+0000 7f5b59ffb640 1 -- 172.21.15.49:0/1443817513 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f5b54051050 con 0x7f5b640cdd50 2023-11-06T02:12:03.165 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:03.168+0000 7f5b6b5ce640 1 -- 172.21.15.49:0/1443817513 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f5b64070e70 con 0x7f5b640cdd50 2023-11-06T02:12:03.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:03.168+0000 7f5b59ffb640 1 -- 172.21.15.49:0/1443817513 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v65) v1 ==== 80+0+0 (secure 0 0 0) 0x7f5b54054670 con 0x7f5b640cdd50 2023-11-06T02:12:03.166 INFO:teuthology.orchestra.run.smithi049.stderr:listed 0 entries 2023-11-06T02:12:03.169 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:03.168+0000 7f5b6b5ce640 1 -- 172.21.15.49:0/1443817513 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5b48065a60 msgr2=0x7f5b48067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:03.169 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:03.168+0000 7f5b6b5ce640 1 --2- 172.21.15.49:0/1443817513 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5b48065a60 0x7f5b48067f20 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f5b60007fd0 tx=0x7f5b60009040 comp rx=0 tx=0).stop 2023-11-06T02:12:03.169 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:03.168+0000 7f5b6b5ce640 1 -- 172.21.15.49:0/1443817513 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b640cdd50 msgr2=0x7f5b640b2550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:03.170 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:03.168+0000 7f5b6b5ce640 1 --2- 172.21.15.49:0/1443817513 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b640cdd50 0x7f5b640b2550 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f5b5400d5f0 tx=0x7f5b5400dab0 comp rx=0 tx=0).stop 2023-11-06T02:12:03.170 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:03.168+0000 7f5b6b5ce640 1 -- 172.21.15.49:0/1443817513 shutdown_connections 2023-11-06T02:12:03.170 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:03.168+0000 7f5b6b5ce640 1 --2- 172.21.15.49:0/1443817513 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5b48065a60 0x7f5b48067f20 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:03.170 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:03.168+0000 7f5b6b5ce640 1 --2- 172.21.15.49:0/1443817513 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b640cdd50 0x7f5b640b2550 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:03.170 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:03.168+0000 7f5b6b5ce640 1 --2- 172.21.15.49:0/1443817513 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5b64159b40 0x7f5b64154e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:03.171 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:03.168+0000 7f5b6b5ce640 1 --2- 172.21.15.49:0/1443817513 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b640cb010 0x7f5b64149160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:03.171 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:03.168+0000 7f5b6b5ce640 1 -- 172.21.15.49:0/1443817513 >> 172.21.15.49:0/1443817513 conn(0x7f5b6406c0a0 msgr2=0x7f5b640af610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:12:03.171 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:03.168+0000 7f5b6b5ce640 1 -- 172.21.15.49:0/1443817513 shutdown_connections 2023-11-06T02:12:03.171 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:03.168+0000 7f5b6b5ce640 1 -- 172.21.15.49:0/1443817513 wait complete. 2023-11-06T02:12:03.186 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi049.front.sepia.ceph.com... 2023-11-06T02:12:03.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:12:03.187 DEBUG:teuthology.orchestra.run.smithi049:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-06T02:12:03.199 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi049.stderr:ceph-fuse[52514]: fuse finished with error 0 and tester_r 0 2023-11-06T02:12:03.202 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T02:12:09.304 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi049.front.sepia.ceph.com 2023-11-06T02:12:09.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:12:09.305 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-06T02:12:09.313 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T02:12:09.490 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:09.492+0000 7f5baf1bb640 1 Processor -- start 2023-11-06T02:12:09.490 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:09.492+0000 7f5baf1bb640 1 -- start start 2023-11-06T02:12:09.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:09.492+0000 7f5baf1bb640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ba80cda70 0x7f5ba8161650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:09.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:09.492+0000 7f5baf1bb640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f5ba80714a0 0x7f5ba80718f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:09.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:09.492+0000 7f5baf1bb640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ba8071e30 con 0x7f5ba80b1f90 2023-11-06T02:12:09.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:09.492+0000 7f5baf1bb640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ba8161b90 con 0x7f5ba80714a0 2023-11-06T02:12:09.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:09.492+0000 7f5baf1bb640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba80cdef0 con 0x7f5ba80cda70 2023-11-06T02:12:09.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:09.492+0000 7f5bacf57640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f5ba80b1f90 0x7f5ba80d1c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:35110/0 (socket says 172.21.15.49:35110) 2023-11-06T02:12:09.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:09.492+0000 7f5bacf57640 1 -- 172.21.15.49:0/3553283985 learned_addr learned my addr 172.21.15.49:0/3553283985 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:12:09.493 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:09.492+0000 7f5bad758640 1 --2- 172.21.15.49:0/3553283985 >> v2:172.21.15.49:3300/0 conn(0x7f5ba80714a0 0x7f5ba80718f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:09.493 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:09.492+0000 7f5ba7fff640 1 --2- 172.21.15.49:0/3553283985 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ba80cda70 0x7f5ba8161650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:09.493 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:09.492+0000 7f5ba77fe640 1 -- 172.21.15.49:0/3553283985 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 291653927 0 0) 0x7f5b98001120 con 0x7f5ba80b1f90 2023-11-06T02:12:09.493 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:09.492+0000 7f5ba77fe640 1 -- 172.21.15.49:0/3553283985 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b88003530 con 0x7f5ba80b1f90 2023-11-06T02:12:09.493 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:09.492+0000 7f5bad758640 1 -- 172.21.15.49:0/3553283985 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ba80cda70 msgr2=0x7f5ba8161650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:09.494 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:09.492+0000 7f5bad758640 1 --2- 172.21.15.49:0/3553283985 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ba80cda70 0x7f5ba8161650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:09.494 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:09.492+0000 7f5bad758640 1 -- 172.21.15.49:0/3553283985 >> v1:172.21.15.49:6789/0 conn(0x7f5ba80b1f90 legacy=0x7f5ba80d1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:09.494 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:09.492+0000 7f5bad758640 1 -- 172.21.15.49:0/3553283985 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ba80d17c0 con 0x7f5ba80714a0 2023-11-06T02:12:09.494 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:09.492+0000 7f5ba7fff640 1 --2- 172.21.15.49:0/3553283985 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ba80cda70 0x7f5ba8161650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:12:09.496 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:09.496+0000 7f5bad758640 1 --2- 172.21.15.49:0/3553283985 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ba80714a0 0x7f5ba80718f0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f5ba0009d90 tx=0x7f5ba001c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8fe473937f62022 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:09.496 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:09.496+0000 7f5ba77fe640 1 -- 172.21.15.49:0/3553283985 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ba00a5900 con 0x7f5ba80714a0 2023-11-06T02:12:09.496 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:09.496+0000 7f5ba77fe640 1 -- 172.21.15.49:0/3553283985 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ba00a5a60 con 0x7f5ba80714a0 2023-11-06T02:12:09.496 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:09.496+0000 7f5ba77fe640 1 -- 172.21.15.49:0/3553283985 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ba00a5dd0 con 0x7f5ba80714a0 2023-11-06T02:12:09.496 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:09.496+0000 7f5baf1bb640 1 -- 172.21.15.49:0/3553283985 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ba80714a0 msgr2=0x7f5ba80718f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:09.497 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:09.496+0000 7f5baf1bb640 1 --2- 172.21.15.49:0/3553283985 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ba80714a0 0x7f5ba80718f0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f5ba0009d90 tx=0x7f5ba001c950 comp rx=0 tx=0).stop 2023-11-06T02:12:09.497 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:09.496+0000 7f5baf1bb640 1 -- 172.21.15.49:0/3553283985 shutdown_connections 2023-11-06T02:12:09.497 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:09.496+0000 7f5baf1bb640 1 --2- 172.21.15.49:0/3553283985 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ba80714a0 0x7f5ba80718f0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:09.497 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:09.496+0000 7f5baf1bb640 1 --2- 172.21.15.49:0/3553283985 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ba80cda70 0x7f5ba8161650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:09.497 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:09.496+0000 7f5baf1bb640 1 -- 172.21.15.49:0/3553283985 >> 172.21.15.49:0/3553283985 conn(0x7f5ba80d2aa0 msgr2=0x7f5ba80d2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:12:09.498 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:09.496+0000 7f5baf1bb640 1 -- 172.21.15.49:0/3553283985 shutdown_connections 2023-11-06T02:12:09.498 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:09.496+0000 7f5baf1bb640 1 -- 172.21.15.49:0/3553283985 wait complete. 2023-11-06T02:12:09.498 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:09.496+0000 7f5baf1bb640 1 Processor -- start 2023-11-06T02:12:09.498 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:09.496+0000 7f5baf1bb640 1 -- start start 2023-11-06T02:12:09.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:09.500+0000 7f5baf1bb640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ba80714a0 0x7f5ba8157c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:09.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:09.500+0000 7f5baf1bb640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ba80b1f90 0x7f5ba814fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:09.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:09.500+0000 7f5baf1bb640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ba80cda70 0x7f5ba81551d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:09.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:09.500+0000 7f5baf1bb640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba81571c0 con 0x7f5ba80b1f90 2023-11-06T02:12:09.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:09.500+0000 7f5baf1bb640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba81574a0 con 0x7f5ba80714a0 2023-11-06T02:12:09.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:09.500+0000 7f5baf1bb640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba8157780 con 0x7f5ba80cda70 2023-11-06T02:12:09.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:09.500+0000 7f5ba7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ba80b1f90 0x7f5ba814fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:09.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:09.500+0000 7f5ba7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ba80b1f90 0x7f5ba814fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36964/0 (socket says 172.21.15.49:36964) 2023-11-06T02:12:09.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:09.500+0000 7f5ba7fff640 1 -- 172.21.15.49:0/2821697575 learned_addr learned my addr 172.21.15.49:0/2821697575 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:12:09.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:09.500+0000 7f5bacf57640 1 --2- 172.21.15.49:0/2821697575 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ba80714a0 0x7f5ba8157c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:09.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:09.500+0000 7f5bad758640 1 --2- 172.21.15.49:0/2821697575 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ba80cda70 0x7f5ba81551d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:09.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:09.500+0000 7f5ba7fff640 1 -- 172.21.15.49:0/2821697575 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ba80cda70 msgr2=0x7f5ba81551d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:09.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:09.500+0000 7f5ba7fff640 1 --2- 172.21.15.49:0/2821697575 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ba80cda70 0x7f5ba81551d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:09.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:09.500+0000 7f5ba7fff640 1 -- 172.21.15.49:0/2821697575 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ba80714a0 msgr2=0x7f5ba8157c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:09.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:09.500+0000 7f5ba7fff640 1 --2- 172.21.15.49:0/2821697575 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ba80714a0 0x7f5ba8157c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:09.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:09.500+0000 7f5ba7fff640 1 -- 172.21.15.49:0/2821697575 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ba0009a70 con 0x7f5ba80b1f90 2023-11-06T02:12:09.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:09.500+0000 7f5bacf57640 1 --2- 172.21.15.49:0/2821697575 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ba80714a0 0x7f5ba8157c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:12:09.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:09.500+0000 7f5ba7fff640 1 --2- 172.21.15.49:0/2821697575 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ba80b1f90 0x7f5ba814fee0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f5b9000e810 tx=0x7f5b9000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:09.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:09.500+0000 7f5ba57fa640 1 -- 172.21.15.49:0/2821697575 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b9000c8d0 con 0x7f5ba80b1f90 2023-11-06T02:12:09.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:09.500+0000 7f5ba57fa640 1 -- 172.21.15.49:0/2821697575 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b9000ca30 con 0x7f5ba80b1f90 2023-11-06T02:12:09.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:09.500+0000 7f5baf1bb640 1 -- 172.21.15.49:0/2821697575 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ba80ca590 con 0x7f5ba80b1f90 2023-11-06T02:12:09.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:09.500+0000 7f5ba57fa640 1 -- 172.21.15.49:0/2821697575 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b9000cdc0 con 0x7f5ba80b1f90 2023-11-06T02:12:09.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:09.500+0000 7f5baf1bb640 1 -- 172.21.15.49:0/2821697575 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ba8150420 con 0x7f5ba80b1f90 2023-11-06T02:12:09.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:09.500+0000 7f5baf1bb640 1 -- 172.21.15.49:0/2821697575 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ba8074040 con 0x7f5ba80b1f90 2023-11-06T02:12:09.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:09.500+0000 7f5ba57fa640 1 -- 172.21.15.49:0/2821697575 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f5b90013410 con 0x7f5ba80b1f90 2023-11-06T02:12:09.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:09.500+0000 7f5ba57fa640 1 --2- 172.21.15.49:0/2821697575 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5b8c065a60 0x7f5b8c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:09.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:09.500+0000 7f5ba57fa640 1 -- 172.21.15.49:0/2821697575 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5b90083110 con 0x7f5ba80b1f90 2023-11-06T02:12:09.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:09.500+0000 7f5bacf57640 1 --2- 172.21.15.49:0/2821697575 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5b8c065a60 0x7f5b8c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:09.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:09.504+0000 7f5bacf57640 1 --2- 172.21.15.49:0/2821697575 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5b8c065a60 0x7f5b8c067f20 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f5b98005890 tx=0x7f5b98009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:09.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:09.504+0000 7f5ba57fa640 1 -- 172.21.15.49:0/2821697575 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f5b90050970 con 0x7f5ba80b1f90 2023-11-06T02:12:09.701 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:09.704+0000 7f5baf1bb640 1 -- 172.21.15.49:0/2821697575 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5ba80d0c30 con 0x7f5ba80b1f90 2023-11-06T02:12:09.701 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:09.704+0000 7f5ba57fa640 1 -- 172.21.15.49:0/2821697575 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+4652 (secure 0 0 0) 0x7f5b90009480 con 0x7f5ba80b1f90 2023-11-06T02:12:09.702 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:12:09.702 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":30,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5635,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4170291905","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":4170291905},{"type":"v1","addr":"172.21.15.150:6835","nonce":4170291905}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5638,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/2719458937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":2719458937},{"type":"v1","addr":"172.21.15.150:6837","nonce":2719458937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":6339,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/2310338375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2310338375},{"type":"v1","addr":"172.21.15.49:6833","nonce":2310338375}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T02:11:30.091070+0000","modified":"2023-11-06T02:11:33.173821+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6348},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6348":{"gid":6348,"name":"c","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.49:6835/342908487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":342908487},{"type":"v1","addr":"172.21.15.49:6835","nonce":342908487}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-06T02:12:09.703 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 30 2023-11-06T02:12:09.704 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:09.704+0000 7f5baf1bb640 1 -- 172.21.15.49:0/2821697575 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5b8c065a60 msgr2=0x7f5b8c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:09.704 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:09.704+0000 7f5baf1bb640 1 --2- 172.21.15.49:0/2821697575 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5b8c065a60 0x7f5b8c067f20 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f5b98005890 tx=0x7f5b98009040 comp rx=0 tx=0).stop 2023-11-06T02:12:09.704 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:09.704+0000 7f5baf1bb640 1 -- 172.21.15.49:0/2821697575 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ba80b1f90 msgr2=0x7f5ba814fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:09.704 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:09.704+0000 7f5baf1bb640 1 --2- 172.21.15.49:0/2821697575 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ba80b1f90 0x7f5ba814fee0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f5b9000e810 tx=0x7f5b9000ecd0 comp rx=0 tx=0).stop 2023-11-06T02:12:09.705 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:09.704+0000 7f5baf1bb640 1 -- 172.21.15.49:0/2821697575 shutdown_connections 2023-11-06T02:12:09.705 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:09.704+0000 7f5baf1bb640 1 --2- 172.21.15.49:0/2821697575 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5b8c065a60 0x7f5b8c067f20 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:09.705 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:09.704+0000 7f5baf1bb640 1 --2- 172.21.15.49:0/2821697575 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ba80b1f90 0x7f5ba814fee0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:09.705 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:09.704+0000 7f5baf1bb640 1 --2- 172.21.15.49:0/2821697575 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ba80cda70 0x7f5ba81551d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:09.705 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:09.704+0000 7f5baf1bb640 1 --2- 172.21.15.49:0/2821697575 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ba80714a0 0x7f5ba8157c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:09.706 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:09.704+0000 7f5baf1bb640 1 -- 172.21.15.49:0/2821697575 >> 172.21.15.49:0/2821697575 conn(0x7f5ba80d2aa0 msgr2=0x7f5ba80b3150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:12:09.706 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:09.704+0000 7f5baf1bb640 1 -- 172.21.15.49:0/2821697575 shutdown_connections 2023-11-06T02:12:09.706 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:09.704+0000 7f5baf1bb640 1 -- 172.21.15.49:0/2821697575 wait complete. 2023-11-06T02:12:09.721 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2023-11-06T02:12:09.867 INFO:teuthology.orchestra.run.smithi049.stdout:{"message":"","return_code":0} 2023-11-06T02:12:09.867 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2023-11-06T02:12:09.868 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-11-06T02:12:09.868 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T02:12:09.874 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336476k,nr_inodes=4084119,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16105 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-06T02:12:09.874 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse client.0 is not mounted at ubuntu@smithi049.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0 2023-11-06T02:12:09.875 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi049.front.sepia.ceph.com 2023-11-06T02:12:09.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:12:09.875 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-06T02:12:09.921 INFO:teuthology.orchestra.run.smithi049.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-11-06T02:12:09.921 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T02:12:09.922 DEBUG:teuthology.orchestra.run.smithi049:> sudo daemon-helper kill ceph --cluster ceph -w 2023-11-06T02:12:09.966 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 scrub start / repair,recursive 2023-11-06T02:12:10.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.096+0000 7f69c922d640 1 Processor -- start 2023-11-06T02:12:10.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.096+0000 7f69c922d640 1 -- start start 2023-11-06T02:12:10.096 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.096+0000 7f69c922d640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f69c40cbc10 0x7f69c4071120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:10.096 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.096+0000 7f69c922d640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f69c40b5290 0x7f69c40ceb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:10.097 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.096+0000 7f69c922d640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69c406b1b0 con 0x7f69c40b4d40 2023-11-06T02:12:10.097 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.096+0000 7f69c922d640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69c40cbff0 con 0x7f69c40cbc10 2023-11-06T02:12:10.097 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.096+0000 7f69c922d640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69c406b490 con 0x7f69c40b5290 2023-11-06T02:12:10.097 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.096+0000 7f69c2d9d640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f69c40cbc10 0x7f69c4071120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:10.097 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.096+0000 7f69c2d9d640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f69c40cbc10 0x7f69c4071120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36968/0 (socket says 172.21.15.49:36968) 2023-11-06T02:12:10.098 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.096+0000 7f69c2d9d640 1 -- 172.21.15.49:0/540191861 learned_addr learned my addr 172.21.15.49:0/540191861 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:12:10.098 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.096+0000 7f69c259c640 1 --2- 172.21.15.49:0/540191861 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f69c40b5290 0x7f69c40ceb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:10.098 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.096+0000 7f69c2d9d640 1 -- 172.21.15.49:0/540191861 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f69c40b5290 msgr2=0x7f69c40ceb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:10.098 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.096+0000 7f69c2d9d640 1 --2- 172.21.15.49:0/540191861 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f69c40b5290 0x7f69c40ceb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:10.098 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.096+0000 7f69c2d9d640 1 -- 172.21.15.49:0/540191861 >> v1:172.21.15.150:6789/0 conn(0x7f69c40b4d40 legacy=0x7f69c415d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:10.099 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.096+0000 7f69c2d9d640 1 -- 172.21.15.49:0/540191861 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69c406b770 con 0x7f69c40cbc10 2023-11-06T02:12:10.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.100+0000 7f69c2d9d640 1 --2- 172.21.15.49:0/540191861 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69c40cbc10 0x7f69c4071120 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f69bc009870 tx=0x7f69bc01c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bdc03cda38be1c7 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:10.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.100+0000 7f69c1d9b640 1 -- 172.21.15.49:0/540191861 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69bc0ab600 con 0x7f69c40cbc10 2023-11-06T02:12:10.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.100+0000 7f69c1d9b640 1 -- 172.21.15.49:0/540191861 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69bc0a7050 con 0x7f69c40cbc10 2023-11-06T02:12:10.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.100+0000 7f69c1d9b640 1 -- 172.21.15.49:0/540191861 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69bc0ab980 con 0x7f69c40cbc10 2023-11-06T02:12:10.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.100+0000 7f69c922d640 1 -- 172.21.15.49:0/540191861 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69c40cbc10 msgr2=0x7f69c4071120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:10.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.100+0000 7f69c922d640 1 --2- 172.21.15.49:0/540191861 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69c40cbc10 0x7f69c4071120 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f69bc009870 tx=0x7f69bc01c500 comp rx=0 tx=0).stop 2023-11-06T02:12:10.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.100+0000 7f69c922d640 1 -- 172.21.15.49:0/540191861 shutdown_connections 2023-11-06T02:12:10.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.100+0000 7f69c922d640 1 --2- 172.21.15.49:0/540191861 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f69c40b5290 0x7f69c40ceb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:10.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.100+0000 7f69c922d640 1 --2- 172.21.15.49:0/540191861 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69c40cbc10 0x7f69c4071120 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:10.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.100+0000 7f69c922d640 1 -- 172.21.15.49:0/540191861 >> 172.21.15.49:0/540191861 conn(0x7f69c40cc140 msgr2=0x7f69c40cc550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:12:10.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.100+0000 7f69c922d640 1 -- 172.21.15.49:0/540191861 shutdown_connections 2023-11-06T02:12:10.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.100+0000 7f69c922d640 1 -- 172.21.15.49:0/540191861 wait complete. 2023-11-06T02:12:10.105 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.104+0000 7f69c922d640 1 Processor -- start 2023-11-06T02:12:10.105 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.104+0000 7f69c922d640 1 -- start start 2023-11-06T02:12:10.105 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.104+0000 7f69c922d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69c40b4d40 0x7f69c4166b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:10.105 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.104+0000 7f69c922d640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f69c40b5290 0x7f69c4155af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:10.105 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.104+0000 7f69c922d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69c40cbc10 0x7f69c414e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:10.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.104+0000 7f69c922d640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69c4156030 con 0x7f69c40b4d40 2023-11-06T02:12:10.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.104+0000 7f69c922d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69c415c750 con 0x7f69c40cbc10 2023-11-06T02:12:10.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.104+0000 7f69c922d640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69c4167250 con 0x7f69c40b5290 2023-11-06T02:12:10.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.104+0000 7f69c2d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69c40b4d40 0x7f69c4166b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:10.107 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.104+0000 7f69c2d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69c40b4d40 0x7f69c4166b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36984/0 (socket says 172.21.15.49:36984) 2023-11-06T02:12:10.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.104+0000 7f69c2d9d640 1 -- 172.21.15.49:0/1287556246 learned_addr learned my addr 172.21.15.49:0/1287556246 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:12:10.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.104+0000 7f69c259c640 1 --2- 172.21.15.49:0/1287556246 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f69c40b5290 0x7f69c4155af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:10.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.104+0000 7f69c359e640 1 --2- 172.21.15.49:0/1287556246 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69c40cbc10 0x7f69c414e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:10.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.104+0000 7f69c2d9d640 1 -- 172.21.15.49:0/1287556246 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f69c40b5290 msgr2=0x7f69c4155af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:10.109 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.104+0000 7f69c2d9d640 1 --2- 172.21.15.49:0/1287556246 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f69c40b5290 0x7f69c4155af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:10.109 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.104+0000 7f69c2d9d640 1 -- 172.21.15.49:0/1287556246 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69c40cbc10 msgr2=0x7f69c414e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:10.109 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.104+0000 7f69c2d9d640 1 --2- 172.21.15.49:0/1287556246 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69c40cbc10 0x7f69c414e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:10.109 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.104+0000 7f69c2d9d640 1 -- 172.21.15.49:0/1287556246 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69bc0094d0 con 0x7f69c40b4d40 2023-11-06T02:12:10.109 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.104+0000 7f69c359e640 1 --2- 172.21.15.49:0/1287556246 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69c40cbc10 0x7f69c414e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:12:10.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.104+0000 7f69c259c640 1 --2- 172.21.15.49:0/1287556246 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f69c40b5290 0x7f69c4155af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:12:10.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.104+0000 7f69c2d9d640 1 --2- 172.21.15.49:0/1287556246 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69c40b4d40 0x7f69c4166b90 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f69bc0ae040 tx=0x7f69bc01ad40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:10.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.104+0000 7f69bb7fe640 1 -- 172.21.15.49:0/1287556246 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69bc0b6620 con 0x7f69c40b4d40 2023-11-06T02:12:10.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.104+0000 7f69c922d640 1 -- 172.21.15.49:0/1287556246 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69c4167530 con 0x7f69c40b4d40 2023-11-06T02:12:10.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.104+0000 7f69bb7fe640 1 -- 172.21.15.49:0/1287556246 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69bc0b6780 con 0x7f69c40b4d40 2023-11-06T02:12:10.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.104+0000 7f69bb7fe640 1 -- 172.21.15.49:0/1287556246 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69bc0b6b30 con 0x7f69c40b4d40 2023-11-06T02:12:10.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.104+0000 7f69c922d640 1 -- 172.21.15.49:0/1287556246 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69c415c3e0 con 0x7f69c40b4d40 2023-11-06T02:12:10.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.108+0000 7f69bb7fe640 1 -- 172.21.15.49:0/1287556246 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f69bc0b2040 con 0x7f69c40b4d40 2023-11-06T02:12:10.118 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.108+0000 7f69bb7fe640 1 --2- 172.21.15.49:0/1287556246 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f69a4065a90 0x7f69a4067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:10.118 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.108+0000 7f69bb7fe640 1 -- 172.21.15.49:0/1287556246 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f69bc11d5c0 con 0x7f69c40b4d40 2023-11-06T02:12:10.118 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.108+0000 7f69c259c640 1 --2- 172.21.15.49:0/1287556246 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f69a4065a90 0x7f69a4067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:10.118 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.108+0000 7f69cc355000 1 Processor -- start 2023-11-06T02:12:10.119 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.108+0000 7f69cc355000 1 -- start start 2023-11-06T02:12:10.119 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.108+0000 7f69c259c640 1 --2- 172.21.15.49:0/1287556246 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f69a4065a90 0x7f69a4067f50 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f69ac006fd0 tx=0x7f69ac008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:10.119 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.108+0000 7f69cc355000 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e7c45176a0 0x55e7c4517a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:10.119 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.108+0000 7f69cc355000 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e7c4517fc0 0x55e7c44fbec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:10.120 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.108+0000 7f69cc355000 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7c44fc400 0x55e7c44fc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:10.120 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.108+0000 7f69cc355000 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7c4395a90 con 0x55e7c44fc400 2023-11-06T02:12:10.120 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.108+0000 7f69cc355000 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7c4323490 con 0x55e7c45176a0 2023-11-06T02:12:10.120 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.108+0000 7f69cc355000 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e7c4320a60 con 0x55e7c4517fc0 2023-11-06T02:12:10.120 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.108+0000 7f69c359e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7c44fc400 0x55e7c44fc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:10.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.108+0000 7f69c359e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7c44fc400 0x55e7c44fc7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36994/0 (socket says 172.21.15.49:36994) 2023-11-06T02:12:10.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.108+0000 7f69c359e640 1 -- 172.21.15.49:0/380871212 learned_addr learned my addr 172.21.15.49:0/380871212 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:12:10.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.108+0000 7f69c2d9d640 1 --2- 172.21.15.49:0/380871212 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e7c45176a0 0x55e7c4517a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:10.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.108+0000 7f69c259c640 1 --2- 172.21.15.49:0/380871212 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e7c4517fc0 0x55e7c44fbec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:10.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.108+0000 7f69c359e640 1 -- 172.21.15.49:0/380871212 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e7c4517fc0 msgr2=0x55e7c44fbec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:10.122 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.108+0000 7f69c359e640 1 --2- 172.21.15.49:0/380871212 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e7c4517fc0 0x55e7c44fbec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:10.122 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.108+0000 7f69c359e640 1 -- 172.21.15.49:0/380871212 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e7c45176a0 msgr2=0x55e7c4517a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:10.122 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.108+0000 7f69c359e640 1 --2- 172.21.15.49:0/380871212 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e7c45176a0 0x55e7c4517a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:10.122 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.108+0000 7f69c359e640 1 -- 172.21.15.49:0/380871212 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e7c4019070 con 0x55e7c44fc400 2023-11-06T02:12:10.123 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.108+0000 7f69c2d9d640 1 --2- 172.21.15.49:0/380871212 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e7c45176a0 0x55e7c4517a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:12:10.123 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.108+0000 7f69c359e640 1 --2- 172.21.15.49:0/380871212 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7c44fc400 0x55e7c44fc7e0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f69b40045a0 tx=0x7f69b400d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3453e852105dcb4 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:10.123 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.108+0000 7f69b8ff9640 1 -- 172.21.15.49:0/380871212 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b4006b60 con 0x55e7c44fc400 2023-11-06T02:12:10.123 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.108+0000 7f69b8ff9640 1 -- 172.21.15.49:0/380871212 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69b4006cc0 con 0x55e7c44fc400 2023-11-06T02:12:10.123 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.108+0000 7f69b8ff9640 1 -- 172.21.15.49:0/380871212 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b40049d0 con 0x55e7c44fc400 2023-11-06T02:12:10.124 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.108+0000 7f69cc355000 1 -- 172.21.15.49:0/380871212 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7c44fc400 msgr2=0x55e7c44fc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:10.124 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.108+0000 7f69cc355000 1 --2- 172.21.15.49:0/380871212 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7c44fc400 0x55e7c44fc7e0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f69b40045a0 tx=0x7f69b400d520 comp rx=0 tx=0).stop 2023-11-06T02:12:10.124 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.108+0000 7f69cc355000 1 -- 172.21.15.49:0/380871212 shutdown_connections 2023-11-06T02:12:10.124 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.108+0000 7f69cc355000 1 --2- 172.21.15.49:0/380871212 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7c44fc400 0x55e7c44fc7e0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:10.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.108+0000 7f69cc355000 1 --2- 172.21.15.49:0/380871212 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e7c4517fc0 0x55e7c44fbec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:10.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.108+0000 7f69cc355000 1 --2- 172.21.15.49:0/380871212 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e7c45176a0 0x55e7c4517a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:10.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.108+0000 7f69cc355000 1 -- 172.21.15.49:0/380871212 >> 172.21.15.49:0/380871212 conn(0x55e7c44c4bc0 msgr2=0x55e7c44d3bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:12:10.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.112+0000 7f69cc355000 1 -- 172.21.15.49:0/380871212 shutdown_connections 2023-11-06T02:12:10.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.112+0000 7f69cc355000 1 -- 172.21.15.49:0/380871212 wait complete. 2023-11-06T02:12:10.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.112+0000 7f69cc355000 1 -- start start 2023-11-06T02:12:10.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.112+0000 7f69cc355000 1 Processor -- start 2023-11-06T02:12:10.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.112+0000 7f69cc355000 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7c44fc400 0x55e7c44fbe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:10.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.112+0000 7f69cc355000 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e7c45176a0 0x55e7c44d10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:10.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.112+0000 7f69cc355000 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e7c4517fc0 0x55e7c44d15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:10.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.112+0000 7f69cc355000 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7c431c350 con 0x55e7c44fc400 2023-11-06T02:12:10.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.112+0000 7f69cc355000 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7c42bb610 con 0x55e7c4517fc0 2023-11-06T02:12:10.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.112+0000 7f69cc355000 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e7c42d9800 con 0x55e7c45176a0 2023-11-06T02:12:10.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.112+0000 7f69cc355000 10 client.0 _ll_register_callbacks cb 0x55e7c4311960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T02:12:10.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.112+0000 7f69c2d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7c44fc400 0x55e7c44fbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:10.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.112+0000 7f69c2d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7c44fc400 0x55e7c44fbe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37010/0 (socket says 172.21.15.49:37010) 2023-11-06T02:12:10.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.112+0000 7f69c2d9d640 1 -- 172.21.15.49:0/3364301622 learned_addr learned my addr 172.21.15.49:0/3364301622 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:12:10.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.112+0000 7f69c259c640 1 --2- 172.21.15.49:0/3364301622 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e7c45176a0 0x55e7c44d10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:10.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.112+0000 7f69c359e640 1 --2- 172.21.15.49:0/3364301622 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e7c4517fc0 0x55e7c44d15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:10.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.112+0000 7f69c2d9d640 1 -- 172.21.15.49:0/3364301622 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e7c45176a0 msgr2=0x55e7c44d10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:10.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.112+0000 7f69c2d9d640 1 --2- 172.21.15.49:0/3364301622 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e7c45176a0 0x55e7c44d10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:10.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.112+0000 7f69c2d9d640 1 -- 172.21.15.49:0/3364301622 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e7c4517fc0 msgr2=0x55e7c44d15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:10.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.112+0000 7f69c2d9d640 1 --2- 172.21.15.49:0/3364301622 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e7c4517fc0 0x55e7c44d15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:10.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.112+0000 7f69c2d9d640 1 -- 172.21.15.49:0/3364301622 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69b4004200 con 0x55e7c44fc400 2023-11-06T02:12:10.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.112+0000 7f69c359e640 1 --2- 172.21.15.49:0/3364301622 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e7c4517fc0 0x55e7c44d15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:12:10.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.112+0000 7f69c2d9d640 1 --2- 172.21.15.49:0/3364301622 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7c44fc400 0x55e7c44fbe90 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f69bc01cda0 tx=0x7f69bc0ee480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51d30315a1943f3e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:10.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.112+0000 7f69a1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-11-06T02:12:10.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.112+0000 7f69a1ffb640 1 -- 172.21.15.49:0/3364301622 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69bc0f2890 con 0x55e7c44fc400 2023-11-06T02:12:10.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.112+0000 7f69cc355000 1 -- 172.21.15.49:0/3364301622 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e7c42ebc80 con 0x55e7c44fc400 2023-11-06T02:12:10.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.112+0000 7f69a1ffb640 1 -- 172.21.15.49:0/3364301622 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f69bc0f6a40 con 0x55e7c44fc400 2023-11-06T02:12:10.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.112+0000 7f69a1ffb640 1 -- 172.21.15.49:0/3364301622 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f69bc106020 con 0x55e7c44fc400 2023-11-06T02:12:10.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.112+0000 7f69cc355000 10 client.6732 fetch_fsmap learned FSMap version 30 2023-11-06T02:12:10.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.112+0000 7f69cc355000 1 -- 172.21.15.49:0/3364301622 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=2+}) v3 -- 0x55e7c42e12d0 con 0x55e7c44fc400 2023-11-06T02:12:10.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.112+0000 7f69a1ffb640 1 -- 172.21.15.49:0/3364301622 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e7c42ebc80 con 0x55e7c44fc400 2023-11-06T02:12:10.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.112+0000 7f69a1ffb640 1 -- 172.21.15.49:0/3364301622 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 30) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f69bc0eecc0 con 0x55e7c44fc400 2023-11-06T02:12:10.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.112+0000 7f69cc355000 10 client.6732 fetch_fsmap finished waiting for FSMap version 30 2023-11-06T02:12:10.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.112+0000 7f69cc355000 10 client.6732 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2023-11-06T02:12:10.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.112+0000 7f69cc355000 20 client.6732 populate_metadata read hostname 'smithi049' 2023-11-06T02:12:10.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.112+0000 7f69cc355000 1 --2- 172.21.15.49:0/3364301622 >> [v2:172.21.15.49:6834/342908487,v1:172.21.15.49:6835/342908487] conn(0x55e7c44d2390 0x55e7c4529750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:10.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.112+0000 7f69cc355000 4 client.6732 mds_command: new command op to 6348 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T02:12:10.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.112+0000 7f69cc355000 1 -- 172.21.15.49:0/3364301622 --> [v2:172.21.15.49:6834/342908487,v1:172.21.15.49:6835/342908487] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e7c431aca0 con 0x55e7c44d2390 2023-11-06T02:12:10.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.112+0000 7f69c259c640 1 --2- 172.21.15.49:0/3364301622 >> [v2:172.21.15.49:6834/342908487,v1:172.21.15.49:6835/342908487] conn(0x55e7c44d2390 0x55e7c4529750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:10.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.112+0000 7f69a1ffb640 10 client.6732 ms_handle_connect on v2:172.21.15.49:6834/342908487 2023-11-06T02:12:10.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.112+0000 7f69c259c640 1 --2- 172.21.15.49:0/3364301622 >> [v2:172.21.15.49:6834/342908487,v1:172.21.15.49:6835/342908487] conn(0x55e7c44d2390 0x55e7c4529750 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f8f7108864fb039 server_cookie=8444146c9223c014 in_seq=0 out_seq=0 2023-11-06T02:12:10.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.116+0000 7f69a1ffb640 1 -- 172.21.15.49:0/3364301622 <== mds.0 v2:172.21.15.49:6834/342908487 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55e7c431aca0 con 0x55e7c44d2390 2023-11-06T02:12:10.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.116+0000 7f69a1ffb640 10 client.6732 handle_command_reply: tid=0 2023-11-06T02:12:10.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.116+0000 7f69cc355000 1 client.6732 shutdown 2023-11-06T02:12:10.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.116+0000 7f69cc355000 1 -- 172.21.15.49:0/3364301622 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7c44fc400 msgr2=0x55e7c44fbe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:10.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.116+0000 7f69cc355000 1 --2- 172.21.15.49:0/3364301622 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7c44fc400 0x55e7c44fbe90 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f69bc01cda0 tx=0x7f69bc0ee480 comp rx=0 tx=0).stop 2023-11-06T02:12:10.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.116+0000 7f69cc355000 1 -- 172.21.15.49:0/3364301622 shutdown_connections 2023-11-06T02:12:10.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.116+0000 7f69cc355000 1 --2- 172.21.15.49:0/3364301622 >> [v2:172.21.15.49:6834/342908487,v1:172.21.15.49:6835/342908487] conn(0x55e7c44d2390 0x55e7c4529750 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:10.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.116+0000 7f69cc355000 1 --2- 172.21.15.49:0/3364301622 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e7c4517fc0 0x55e7c44d15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:10.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.116+0000 7f69cc355000 1 --2- 172.21.15.49:0/3364301622 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e7c45176a0 0x55e7c44d10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:10.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.116+0000 7f69cc355000 1 --2- 172.21.15.49:0/3364301622 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7c44fc400 0x55e7c44fbe90 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:10.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.116+0000 7f69cc355000 1 -- 172.21.15.49:0/3364301622 >> 172.21.15.49:0/3364301622 conn(0x55e7c44c4bc0 msgr2=0x55e7c44fd280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:12:10.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.116+0000 7f69a1ffb640 0 client.6732 ms_handle_reset on v2:172.21.15.49:6834/342908487 2023-11-06T02:12:10.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.116+0000 7f69cc355000 1 -- 172.21.15.49:0/3364301622 shutdown_connections 2023-11-06T02:12:10.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.116+0000 7f69cc355000 1 -- 172.21.15.49:0/3364301622 wait complete. 2023-11-06T02:12:10.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.116+0000 7f69cc355000 20 client.6732 trim_cache size 0 max 16384 2023-11-06T02:12:10.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.120+0000 7f69872d8640 1 Processor -- start 2023-11-06T02:12:10.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.120+0000 7f69872d8640 1 -- start start 2023-11-06T02:12:10.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.124+0000 7f69872d8640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f698015cbd0 0x7f69800d3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:10.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.124+0000 7f69872d8640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f69800cd450 0x7f69800d0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:10.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.124+0000 7f69872d8640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f69800d3590 0x7f69800d13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:10.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.124+0000 7f69872d8640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69800cd8d0 con 0x7f698015cbd0 2023-11-06T02:12:10.143 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.124+0000 7f69872d8640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69800cdbb0 con 0x7f69800d3590 2023-11-06T02:12:10.143 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.124+0000 7f69872d8640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69800cde90 con 0x7f69800cd450 2023-11-06T02:12:10.143 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.124+0000 7f6985074640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f698015cbd0 0x7f69800d3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:10.143 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.124+0000 7f6985074640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f698015cbd0 0x7f69800d3050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37016/0 (socket says 172.21.15.49:37016) 2023-11-06T02:12:10.143 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.124+0000 7f6985074640 1 -- 172.21.15.49:0/763160518 learned_addr learned my addr 172.21.15.49:0/763160518 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:12:10.144 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.124+0000 7f6985875640 1 --2- 172.21.15.49:0/763160518 >> v2:172.21.15.150:3300/0 conn(0x7f69800d3590 0x7f69800d13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:10.144 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.124+0000 7f6984873640 1 --2- 172.21.15.49:0/763160518 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f69800cd450 0x7f69800d0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:10.144 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.124+0000 7f6985074640 1 -- 172.21.15.49:0/763160518 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f69800cd450 msgr2=0x7f69800d0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:10.144 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.124+0000 7f6985074640 1 --2- 172.21.15.49:0/763160518 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f69800cd450 0x7f69800d0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:10.144 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.124+0000 7f6985074640 1 -- 172.21.15.49:0/763160518 >> v2:172.21.15.150:3300/0 conn(0x7f69800d3590 msgr2=0x7f69800d13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:10.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.124+0000 7f6985074640 1 --2- 172.21.15.49:0/763160518 >> v2:172.21.15.150:3300/0 conn(0x7f69800d3590 0x7f69800d13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:10.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.124+0000 7f6985074640 1 -- 172.21.15.49:0/763160518 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69800ce170 con 0x7f698015cbd0 2023-11-06T02:12:10.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.124+0000 7f6985074640 1 --2- 172.21.15.49:0/763160518 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f698015cbd0 0x7f69800d3050 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f698006d000 tx=0x7f697c01c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79ae5968642816c4 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:10.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.124+0000 7f6977fff640 1 -- 172.21.15.49:0/763160518 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f698015d1c0 con 0x7f698015cbd0 2023-11-06T02:12:10.146 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.124+0000 7f6977fff640 1 -- 172.21.15.49:0/763160518 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f697c0ab620 con 0x7f698015cbd0 2023-11-06T02:12:10.146 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.124+0000 7f6977fff640 1 -- 172.21.15.49:0/763160518 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f697c0ab9b0 con 0x7f698015cbd0 2023-11-06T02:12:10.146 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.124+0000 7f69872d8640 1 -- 172.21.15.49:0/763160518 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f698015cbd0 msgr2=0x7f69800d3050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:10.146 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.124+0000 7f69872d8640 1 --2- 172.21.15.49:0/763160518 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f698015cbd0 0x7f69800d3050 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f698006d000 tx=0x7f697c01c440 comp rx=0 tx=0).stop 2023-11-06T02:12:10.146 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.124+0000 7f69872d8640 1 -- 172.21.15.49:0/763160518 shutdown_connections 2023-11-06T02:12:10.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.124+0000 7f69872d8640 1 --2- 172.21.15.49:0/763160518 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f69800cd450 0x7f69800d0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:10.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.124+0000 7f69872d8640 1 --2- 172.21.15.49:0/763160518 >> v2:172.21.15.150:3300/0 conn(0x7f69800d3590 0x7f69800d13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:10.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.124+0000 7f69872d8640 1 --2- 172.21.15.49:0/763160518 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f698015cbd0 0x7f69800d3050 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:10.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.124+0000 7f69872d8640 1 -- 172.21.15.49:0/763160518 >> 172.21.15.49:0/763160518 conn(0x7f698009bc10 msgr2=0x7f69800d2940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:12:10.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.124+0000 7f69872d8640 1 -- 172.21.15.49:0/763160518 shutdown_connections 2023-11-06T02:12:10.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.128+0000 7f69872d8640 1 -- 172.21.15.49:0/763160518 wait complete. 2023-11-06T02:12:10.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.128+0000 7f69872d8640 1 Processor -- start 2023-11-06T02:12:10.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.128+0000 7f69872d8640 1 -- start start 2023-11-06T02:12:10.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.128+0000 7f69872d8640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69800cd450 0x7f69801552d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:10.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.128+0000 7f69872d8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69800d3590 0x7f6980155810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:10.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.128+0000 7f69872d8640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f698015cbd0 0x7f6980155d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:10.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.128+0000 7f69872d8640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69800b8460 con 0x7f69800d3590 2023-11-06T02:12:10.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.128+0000 7f69872d8640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f698015d380 con 0x7f69800cd450 2023-11-06T02:12:10.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.128+0000 7f69872d8640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f698015d660 con 0x7f698015cbd0 2023-11-06T02:12:10.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.128+0000 7f6984873640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69800d3590 0x7f6980155810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:10.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.128+0000 7f6984873640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69800d3590 0x7f6980155810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37024/0 (socket says 172.21.15.49:37024) 2023-11-06T02:12:10.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.128+0000 7f6984873640 1 -- 172.21.15.49:0/1194616221 learned_addr learned my addr 172.21.15.49:0/1194616221 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:12:10.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.128+0000 7f6985074640 1 --2- 172.21.15.49:0/1194616221 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69800cd450 0x7f69801552d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:10.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.128+0000 7f6985875640 1 --2- 172.21.15.49:0/1194616221 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f698015cbd0 0x7f6980155d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:10.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.128+0000 7f6984873640 1 -- 172.21.15.49:0/1194616221 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f698015cbd0 msgr2=0x7f6980155d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:10.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.128+0000 7f6984873640 1 --2- 172.21.15.49:0/1194616221 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f698015cbd0 0x7f6980155d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:10.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.128+0000 7f6984873640 1 -- 172.21.15.49:0/1194616221 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69800cd450 msgr2=0x7f69801552d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:10.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.128+0000 7f6984873640 1 --2- 172.21.15.49:0/1194616221 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69800cd450 0x7f69801552d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:10.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.128+0000 7f6984873640 1 -- 172.21.15.49:0/1194616221 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f697c0094d0 con 0x7f69800d3590 2023-11-06T02:12:10.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.128+0000 7f6985875640 1 --2- 172.21.15.49:0/1194616221 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f698015cbd0 0x7f6980155d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:12:10.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.128+0000 7f6985074640 1 --2- 172.21.15.49:0/1194616221 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69800cd450 0x7f69801552d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:12:10.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.128+0000 7f6984873640 1 --2- 172.21.15.49:0/1194616221 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69800d3590 0x7f6980155810 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f697000d5f0 tx=0x7f697000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:10.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.128+0000 7f6975ffb640 1 -- 172.21.15.49:0/1194616221 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69700125f0 con 0x7f69800d3590 2023-11-06T02:12:10.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.128+0000 7f6975ffb640 1 -- 172.21.15.49:0/1194616221 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6970012750 con 0x7f69800d3590 2023-11-06T02:12:10.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.128+0000 7f69872d8640 1 -- 172.21.15.49:0/1194616221 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f698015d940 con 0x7f69800d3590 2023-11-06T02:12:10.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.128+0000 7f6975ffb640 1 -- 172.21.15.49:0/1194616221 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6970012ae0 con 0x7f69800d3590 2023-11-06T02:12:10.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.128+0000 7f69872d8640 1 -- 172.21.15.49:0/1194616221 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69800b56b0 con 0x7f69800d3590 2023-11-06T02:12:10.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.128+0000 7f69872d8640 1 -- 172.21.15.49:0/1194616221 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f698004e8a0 con 0x7f69800d3590 2023-11-06T02:12:10.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.132+0000 7f6975ffb640 1 -- 172.21.15.49:0/1194616221 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f6970012c40 con 0x7f69800d3590 2023-11-06T02:12:10.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.132+0000 7f6975ffb640 1 --2- 172.21.15.49:0/1194616221 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6964065a60 0x7f6964067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:10.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.132+0000 7f6975ffb640 1 -- 172.21.15.49:0/1194616221 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6970084180 con 0x7f69800d3590 2023-11-06T02:12:10.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.132+0000 7f6975ffb640 1 -- 172.21.15.49:0/1194616221 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+513 (secure 0 0 0) 0x7f69700845f0 con 0x7f69800d3590 2023-11-06T02:12:10.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.132+0000 7f69cc355000 1 Processor -- start 2023-11-06T02:12:10.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.132+0000 7f6985074640 1 --2- 172.21.15.49:0/1194616221 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6964065a60 0x7f6964067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:10.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.132+0000 7f69cc355000 1 -- start start 2023-11-06T02:12:10.156 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.132+0000 7f69872d8640 1 -- 172.21.15.49:0/1194616221 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({log-info=0+}) v3 -- 0x7f69800b5030 con 0x7f69800d3590 2023-11-06T02:12:10.156 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.132+0000 7f69cc355000 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e7c44fc400 0x55e7c456fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:10.157 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.132+0000 7f69cc355000 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7c45176a0 0x55e7c4570120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:10.157 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.132+0000 7f69cc355000 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e7c4517fc0 0x55e7c4570660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:10.157 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.132+0000 7f69cc355000 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7c3fd9cf0 con 0x55e7c45176a0 2023-11-06T02:12:10.157 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.132+0000 7f69cc355000 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7c42eb8e0 con 0x55e7c4517fc0 2023-11-06T02:12:10.158 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.132+0000 7f69cc355000 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e7c42eafc0 con 0x55e7c44fc400 2023-11-06T02:12:10.158 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.132+0000 7f6985074640 1 --2- 172.21.15.49:0/1194616221 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6964065a60 0x7f6964067f20 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f697c0af000 tx=0x7f697c0a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:10.163 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.132+0000 7f69c259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7c45176a0 0x55e7c4570120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:10.163 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.132+0000 7f69c259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7c45176a0 0x55e7c4570120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37036/0 (socket says 172.21.15.49:37036) 2023-11-06T02:12:10.164 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.132+0000 7f69c259c640 1 -- 172.21.15.49:0/64771552 learned_addr learned my addr 172.21.15.49:0/64771552 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:12:10.164 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.132+0000 7f69c2d9d640 1 --2- 172.21.15.49:0/64771552 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e7c44fc400 0x55e7c456fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:10.164 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.132+0000 7f69c359e640 1 --2- 172.21.15.49:0/64771552 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e7c4517fc0 0x55e7c4570660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:10.164 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.132+0000 7f69c259c640 1 -- 172.21.15.49:0/64771552 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e7c44fc400 msgr2=0x55e7c456fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:10.165 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.132+0000 7f69c259c640 1 --2- 172.21.15.49:0/64771552 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e7c44fc400 0x55e7c456fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:10.165 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.132+0000 7f69c259c640 1 -- 172.21.15.49:0/64771552 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e7c4517fc0 msgr2=0x55e7c4570660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:10.165 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.132+0000 7f69c259c640 1 --2- 172.21.15.49:0/64771552 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e7c4517fc0 0x55e7c4570660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:10.165 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.132+0000 7f69c259c640 1 -- 172.21.15.49:0/64771552 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e7c42c7620 con 0x55e7c45176a0 2023-11-06T02:12:10.165 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.132+0000 7f69c2d9d640 1 --2- 172.21.15.49:0/64771552 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e7c44fc400 0x55e7c456fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:12:10.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.132+0000 7f69c259c640 1 --2- 172.21.15.49:0/64771552 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7c45176a0 0x55e7c4570120 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f69ac00daa0 tx=0x7f69ac016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d822d5de4da0028 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:10.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.132+0000 7f69b97fa640 1 -- 172.21.15.49:0/64771552 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69ac026040 con 0x55e7c45176a0 2023-11-06T02:12:10.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.132+0000 7f69b97fa640 1 -- 172.21.15.49:0/64771552 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69ac00c040 con 0x55e7c45176a0 2023-11-06T02:12:10.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.132+0000 7f69b97fa640 1 -- 172.21.15.49:0/64771552 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69ac00b050 con 0x55e7c45176a0 2023-11-06T02:12:10.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.136+0000 7f69cc355000 1 -- 172.21.15.49:0/64771552 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7c45176a0 msgr2=0x55e7c4570120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:10.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.136+0000 7f69cc355000 1 --2- 172.21.15.49:0/64771552 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7c45176a0 0x55e7c4570120 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f69ac00daa0 tx=0x7f69ac016040 comp rx=0 tx=0).stop 2023-11-06T02:12:10.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.136+0000 7f69cc355000 1 -- 172.21.15.49:0/64771552 shutdown_connections 2023-11-06T02:12:10.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.136+0000 7f69cc355000 1 --2- 172.21.15.49:0/64771552 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7c45176a0 0x55e7c4570120 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:10.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.136+0000 7f69cc355000 1 --2- 172.21.15.49:0/64771552 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e7c4517fc0 0x55e7c4570660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:10.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.136+0000 7f69cc355000 1 --2- 172.21.15.49:0/64771552 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e7c44fc400 0x55e7c456fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:10.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.136+0000 7f69cc355000 1 -- 172.21.15.49:0/64771552 >> 172.21.15.49:0/64771552 conn(0x55e7c44c4bc0 msgr2=0x55e7c44cfc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:12:10.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.136+0000 7f69cc355000 1 -- 172.21.15.49:0/64771552 shutdown_connections 2023-11-06T02:12:10.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.136+0000 7f69cc355000 1 -- 172.21.15.49:0/64771552 wait complete. 2023-11-06T02:12:10.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.136+0000 7f69cc355000 1 -- start start 2023-11-06T02:12:10.169 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.136+0000 7f69cc355000 1 Processor -- start 2023-11-06T02:12:10.169 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.136+0000 7f69cc355000 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e7c44fc400 0x55e7c44f7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:10.169 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.136+0000 7f69cc355000 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7c45176a0 0x55e7c44f8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:10.169 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.136+0000 7f69cc355000 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e7c4517fc0 0x55e7c44f8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:10.169 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.136+0000 7f69cc355000 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7c44c6670 con 0x55e7c45176a0 2023-11-06T02:12:10.170 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.136+0000 7f69cc355000 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7c4317cc0 con 0x55e7c4517fc0 2023-11-06T02:12:10.170 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.136+0000 7f69cc355000 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e7c431b1e0 con 0x55e7c44fc400 2023-11-06T02:12:10.170 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.136+0000 7f69cc355000 10 client.0 _ll_register_callbacks cb 0x55e7c43220a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T02:12:10.170 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.136+0000 7f69c259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7c45176a0 0x55e7c44f8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:10.170 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.136+0000 7f69c259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7c45176a0 0x55e7c44f8140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37046/0 (socket says 172.21.15.49:37046) 2023-11-06T02:12:10.171 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.136+0000 7f69c259c640 1 -- 172.21.15.49:0/4217463649 learned_addr learned my addr 172.21.15.49:0/4217463649 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:12:10.171 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.136+0000 7f69c2d9d640 1 --2- 172.21.15.49:0/4217463649 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e7c44fc400 0x55e7c44f7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:10.171 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.136+0000 7f69c359e640 1 --2- 172.21.15.49:0/4217463649 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e7c4517fc0 0x55e7c44f8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:10.171 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.136+0000 7f69c259c640 1 -- 172.21.15.49:0/4217463649 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e7c44fc400 msgr2=0x55e7c44f7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:10.172 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.136+0000 7f69c259c640 1 --2- 172.21.15.49:0/4217463649 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e7c44fc400 0x55e7c44f7c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:10.172 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.136+0000 7f69c259c640 1 -- 172.21.15.49:0/4217463649 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e7c4517fc0 msgr2=0x55e7c44f8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:10.172 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.136+0000 7f69c259c640 1 --2- 172.21.15.49:0/4217463649 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e7c4517fc0 0x55e7c44f8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:10.172 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.136+0000 7f69c259c640 1 -- 172.21.15.49:0/4217463649 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69ac00d810 con 0x55e7c45176a0 2023-11-06T02:12:10.172 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.136+0000 7f69c259c640 1 --2- 172.21.15.49:0/4217463649 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7c45176a0 0x55e7c44f8140 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f69ac008010 tx=0x7f69ac004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a215d2ec05406fab server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:10.173 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.136+0000 7f69a1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-11-06T02:12:10.173 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.136+0000 7f69a1ffb640 1 -- 172.21.15.49:0/4217463649 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69ac00b040 con 0x55e7c45176a0 2023-11-06T02:12:10.173 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.136+0000 7f69a1ffb640 1 -- 172.21.15.49:0/4217463649 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f69ac027620 con 0x55e7c45176a0 2023-11-06T02:12:10.173 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.136+0000 7f69cc355000 1 -- 172.21.15.49:0/4217463649 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e7c44d37e0 con 0x55e7c45176a0 2023-11-06T02:12:10.173 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.136+0000 7f69a1ffb640 1 -- 172.21.15.49:0/4217463649 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f69ac021e90 con 0x55e7c45176a0 2023-11-06T02:12:10.174 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.136+0000 7f69cc355000 10 client.6744 fetch_fsmap learned FSMap version 30 2023-11-06T02:12:10.174 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.136+0000 7f69cc355000 1 -- 172.21.15.49:0/4217463649 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=2+}) v3 -- 0x55e7c42bb190 con 0x55e7c45176a0 2023-11-06T02:12:10.174 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.136+0000 7f69a1ffb640 1 -- 172.21.15.49:0/4217463649 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e7c44d37e0 con 0x55e7c45176a0 2023-11-06T02:12:10.174 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.136+0000 7f69a1ffb640 1 -- 172.21.15.49:0/4217463649 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 30) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f69ac0277e0 con 0x55e7c45176a0 2023-11-06T02:12:10.174 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.136+0000 7f69cc355000 10 client.6744 fetch_fsmap finished waiting for FSMap version 30 2023-11-06T02:12:10.175 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.136+0000 7f69cc355000 10 client.6744 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2023-11-06T02:12:10.175 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.136+0000 7f69cc355000 20 client.6744 populate_metadata read hostname 'smithi049' 2023-11-06T02:12:10.175 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.136+0000 7f69cc355000 1 --2- 172.21.15.49:0/4217463649 >> [v2:172.21.15.49:6834/342908487,v1:172.21.15.49:6835/342908487] conn(0x55e7c4579e60 0x55e7c457c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:10.175 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.136+0000 7f69cc355000 4 client.6744 mds_command: new command op to 6348 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2023-11-06T02:12:10.176 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.136+0000 7f69cc355000 1 -- 172.21.15.49:0/4217463649 --> [v2:172.21.15.49:6834/342908487,v1:172.21.15.49:6835/342908487] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) v1 -- 0x55e7c42cbd50 con 0x55e7c4579e60 2023-11-06T02:12:10.176 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.136+0000 7f69c2d9d640 1 --2- 172.21.15.49:0/4217463649 >> [v2:172.21.15.49:6834/342908487,v1:172.21.15.49:6835/342908487] conn(0x55e7c4579e60 0x55e7c457c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:10.176 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.136+0000 7f69a1ffb640 10 client.6744 ms_handle_connect on v2:172.21.15.49:6834/342908487 2023-11-06T02:12:10.176 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.136+0000 7f69c2d9d640 1 --2- 172.21.15.49:0/4217463649 >> [v2:172.21.15.49:6834/342908487,v1:172.21.15.49:6835/342908487] conn(0x55e7c4579e60 0x55e7c457c250 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e59195635b8a953b server_cookie=5b716e32fa2f5823 in_seq=0 out_seq=0 2023-11-06T02:12:10.176 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.140+0000 7f69a1ffb640 1 -- 172.21.15.49:0/4217463649 <== mds.0 v2:172.21.15.49:6834/342908487 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55e7c42cbd50 con 0x55e7c4579e60 2023-11-06T02:12:10.177 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.140+0000 7f69a1ffb640 10 client.6744 handle_command_reply: tid=0 2023-11-06T02:12:10.177 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.140+0000 7f69cc355000 1 client.6744 shutdown 2023-11-06T02:12:10.177 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.140+0000 7f69cc355000 1 -- 172.21.15.49:0/4217463649 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7c45176a0 msgr2=0x55e7c44f8140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:10.177 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.140+0000 7f69cc355000 1 --2- 172.21.15.49:0/4217463649 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7c45176a0 0x55e7c44f8140 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f69ac008010 tx=0x7f69ac004740 comp rx=0 tx=0).stop 2023-11-06T02:12:10.177 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.140+0000 7f69cc355000 1 -- 172.21.15.49:0/4217463649 shutdown_connections 2023-11-06T02:12:10.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.140+0000 7f69cc355000 1 --2- 172.21.15.49:0/4217463649 >> [v2:172.21.15.49:6834/342908487,v1:172.21.15.49:6835/342908487] conn(0x55e7c4579e60 0x55e7c457c250 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:10.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.140+0000 7f69a1ffb640 0 client.6744 ms_handle_reset on v2:172.21.15.49:6834/342908487 2023-11-06T02:12:10.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.140+0000 7f69cc355000 1 --2- 172.21.15.49:0/4217463649 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7c45176a0 0x55e7c44f8140 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:10.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.140+0000 7f69cc355000 1 --2- 172.21.15.49:0/4217463649 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e7c4517fc0 0x55e7c44f8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:10.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.140+0000 7f69cc355000 1 --2- 172.21.15.49:0/4217463649 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e7c44fc400 0x55e7c44f7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:10.179 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.140+0000 7f69cc355000 1 -- 172.21.15.49:0/4217463649 >> 172.21.15.49:0/4217463649 conn(0x55e7c44c4bc0 msgr2=0x55e7c44d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:12:10.179 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.140+0000 7f69cc355000 1 -- 172.21.15.49:0/4217463649 shutdown_connections 2023-11-06T02:12:10.179 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.140+0000 7f69cc355000 1 -- 172.21.15.49:0/4217463649 wait complete. 2023-11-06T02:12:10.179 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.140+0000 7f69cc355000 20 client.6744 trim_cache size 0 max 16384 2023-11-06T02:12:10.180 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-11-06T02:12:10.180 INFO:teuthology.orchestra.run.smithi049.stdout: "return_code": 0, 2023-11-06T02:12:10.180 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_tag": "93f7c4f5-ab96-448b-808d-f2d0cb201719", 2023-11-06T02:12:10.180 INFO:teuthology.orchestra.run.smithi049.stdout: "mode": "asynchronous" 2023-11-06T02:12:10.181 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-11-06T02:12:10.182 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.140+0000 7f69c922d640 1 -- 172.21.15.49:0/1287556246 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f69a4065a90 msgr2=0x7f69a4067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:10.182 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.140+0000 7f69c922d640 1 --2- 172.21.15.49:0/1287556246 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f69a4065a90 0x7f69a4067f50 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f69ac006fd0 tx=0x7f69ac008040 comp rx=0 tx=0).stop 2023-11-06T02:12:10.183 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.140+0000 7f69c922d640 1 -- 172.21.15.49:0/1287556246 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69c40b4d40 msgr2=0x7f69c4166b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:10.183 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.140+0000 7f69c922d640 1 --2- 172.21.15.49:0/1287556246 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69c40b4d40 0x7f69c4166b90 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f69bc0ae040 tx=0x7f69bc01ad40 comp rx=0 tx=0).stop 2023-11-06T02:12:10.183 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.140+0000 7f69c922d640 1 -- 172.21.15.49:0/1287556246 shutdown_connections 2023-11-06T02:12:10.183 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.140+0000 7f69c922d640 1 --2- 172.21.15.49:0/1287556246 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f69a4065a90 0x7f69a4067f50 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:10.184 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.140+0000 7f69c922d640 1 --2- 172.21.15.49:0/1287556246 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69c40cbc10 0x7f69c414e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:10.184 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.140+0000 7f69c922d640 1 --2- 172.21.15.49:0/1287556246 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f69c40b5290 0x7f69c4155af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:10.184 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.140+0000 7f69c922d640 1 --2- 172.21.15.49:0/1287556246 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69c40b4d40 0x7f69c4166b90 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:10.184 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.140+0000 7f69c922d640 1 -- 172.21.15.49:0/1287556246 >> 172.21.15.49:0/1287556246 conn(0x7f69c40cc140 msgr2=0x7f69c40c56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:12:10.184 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.140+0000 7f69c922d640 1 -- 172.21.15.49:0/1287556246 shutdown_connections 2023-11-06T02:12:10.185 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.140+0000 7f69c922d640 1 -- 172.21.15.49:0/1287556246 wait complete. 2023-11-06T02:12:10.185 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 scrub status 2023-11-06T02:12:10.305 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.308+0000 7fb7911ab640 1 Processor -- start 2023-11-06T02:12:10.305 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.308+0000 7fb7911ab640 1 -- start start 2023-11-06T02:12:10.308 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.308+0000 7fb7911ab640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fb78c15cbb0 0x7fb78c0d1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:10.308 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.308+0000 7fb7911ab640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb78c0d3170 0x7fb78c0b5d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:10.308 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.308+0000 7fb7911ab640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb78c15cf90 con 0x7fb78c0d24e0 2023-11-06T02:12:10.309 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.308+0000 7fb7911ab640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb78c0d0b80 con 0x7fb78c15cbb0 2023-11-06T02:12:10.309 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.308+0000 7fb7911ab640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb78c0d0e60 con 0x7fb78c0d3170 2023-11-06T02:12:10.309 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.308+0000 7fb78a59c640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fb78c0d24e0 0x7fb78c0d2a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:35116/0 (socket says 172.21.15.49:35116) 2023-11-06T02:12:10.309 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.308+0000 7fb78a59c640 1 -- 172.21.15.49:0/3437281397 learned_addr learned my addr 172.21.15.49:0/3437281397 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:12:10.309 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.308+0000 7fb78ad9d640 1 --2- 172.21.15.49:0/3437281397 >> v2:172.21.15.150:3300/0 conn(0x7fb78c15cbb0 0x7fb78c0d1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:10.310 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.308+0000 7fb78b59e640 1 --2- 172.21.15.49:0/3437281397 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb78c0d3170 0x7fb78c0b5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:10.310 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.308+0000 7fb789d9b640 1 -- 172.21.15.49:0/3437281397 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 740509308 0 0) 0x7fb78c15cf90 con 0x7fb78c0d24e0 2023-11-06T02:12:10.310 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.308+0000 7fb789d9b640 1 -- 172.21.15.49:0/3437281397 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb774003530 con 0x7fb78c0d24e0 2023-11-06T02:12:10.310 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.308+0000 7fb789d9b640 1 -- 172.21.15.49:0/3437281397 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 598629980 0 0) 0x7fb774003530 con 0x7fb78c0d24e0 2023-11-06T02:12:10.310 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.308+0000 7fb789d9b640 1 -- 172.21.15.49:0/3437281397 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb78c0d3170 msgr2=0x7fb78c0b5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:10.311 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.308+0000 7fb789d9b640 1 --2- 172.21.15.49:0/3437281397 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb78c0d3170 0x7fb78c0b5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:10.311 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.308+0000 7fb789d9b640 1 -- 172.21.15.49:0/3437281397 >> v2:172.21.15.150:3300/0 conn(0x7fb78c15cbb0 msgr2=0x7fb78c0d1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:10.311 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.308+0000 7fb789d9b640 1 --2- 172.21.15.49:0/3437281397 >> v2:172.21.15.150:3300/0 conn(0x7fb78c15cbb0 0x7fb78c0d1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:10.311 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.308+0000 7fb789d9b640 1 -- 172.21.15.49:0/3437281397 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb78c0d1140 con 0x7fb78c0d24e0 2023-11-06T02:12:10.311 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.308+0000 7fb789d9b640 1 -- 172.21.15.49:0/3437281397 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7fb780001680 con 0x7fb78c0d24e0 2023-11-06T02:12:10.312 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.308+0000 7fb789d9b640 1 -- 172.21.15.49:0/3437281397 >> v1:172.21.15.49:6789/0 conn(0x7fb78c0d24e0 legacy=0x7fb78c0d2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:10.312 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.308+0000 7fb789d9b640 1 --2- 172.21.15.49:0/3437281397 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb774003980 0x7fb774003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:10.312 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.308+0000 7fb789d9b640 1 --2- 172.21.15.49:0/3437281397 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb7740047d0 0x7fb774004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:10.312 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.308+0000 7fb789d9b640 1 --2- 172.21.15.49:0/3437281397 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb78c0d3170 0x7fb774005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:10.312 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.308+0000 7fb789d9b640 1 -- 172.21.15.49:0/3437281397 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb78c0d0b80 con 0x7fb774003980 2023-11-06T02:12:10.312 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.308+0000 7fb789d9b640 1 -- 172.21.15.49:0/3437281397 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb78c0d0e60 con 0x7fb7740047d0 2023-11-06T02:12:10.313 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.308+0000 7fb789d9b640 1 -- 172.21.15.49:0/3437281397 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb77400b9c0 con 0x7fb78c0d3170 2023-11-06T02:12:10.313 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.308+0000 7fb78ad9d640 1 --2- 172.21.15.49:0/3437281397 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb774003980 0x7fb774003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:10.313 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.308+0000 7fb78a59c640 1 --2- 172.21.15.49:0/3437281397 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb7740047d0 0x7fb774004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:10.313 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.308+0000 7fb78b59e640 1 --2- 172.21.15.49:0/3437281397 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb78c0d3170 0x7fb774005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:10.313 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.308+0000 7fb78ad9d640 1 -- 172.21.15.49:0/3437281397 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb78c0d3170 msgr2=0x7fb774005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:10.314 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.308+0000 7fb78ad9d640 1 --2- 172.21.15.49:0/3437281397 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb78c0d3170 0x7fb774005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:10.314 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.308+0000 7fb78ad9d640 1 -- 172.21.15.49:0/3437281397 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb7740047d0 msgr2=0x7fb774004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:10.314 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.308+0000 7fb78ad9d640 1 --2- 172.21.15.49:0/3437281397 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb7740047d0 0x7fb774004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:10.314 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.308+0000 7fb78ad9d640 1 -- 172.21.15.49:0/3437281397 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb774003530 con 0x7fb774003980 2023-11-06T02:12:10.320 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.312+0000 7fb78ad9d640 1 --2- 172.21.15.49:0/3437281397 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb774003980 0x7fb774003da0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fb77c009ae0 tx=0x7fb77c01ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57366db50fe0b560 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:10.320 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.312+0000 7fb789d9b640 1 -- 172.21.15.49:0/3437281397 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb77c0a64a0 con 0x7fb774003980 2023-11-06T02:12:10.320 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.312+0000 7fb789d9b640 1 -- 172.21.15.49:0/3437281397 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb77c0a6600 con 0x7fb774003980 2023-11-06T02:12:10.320 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.312+0000 7fb789d9b640 1 -- 172.21.15.49:0/3437281397 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb77c0a69b0 con 0x7fb774003980 2023-11-06T02:12:10.321 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.312+0000 7fb7911ab640 1 -- 172.21.15.49:0/3437281397 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb774003980 msgr2=0x7fb774003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:10.321 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.312+0000 7fb7911ab640 1 --2- 172.21.15.49:0/3437281397 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb774003980 0x7fb774003da0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fb77c009ae0 tx=0x7fb77c01ca90 comp rx=0 tx=0).stop 2023-11-06T02:12:10.321 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.312+0000 7fb78b59e640 1 -- 172.21.15.49:0/3437281397 reap_dead start 2023-11-06T02:12:10.321 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.312+0000 7fb7911ab640 1 -- 172.21.15.49:0/3437281397 shutdown_connections 2023-11-06T02:12:10.321 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.312+0000 7fb7911ab640 1 -- 172.21.15.49:0/3437281397 >> 172.21.15.49:0/3437281397 conn(0x7fb78c0718a0 msgr2=0x7fb78c071cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:12:10.322 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.312+0000 7fb7911ab640 1 -- 172.21.15.49:0/3437281397 shutdown_connections 2023-11-06T02:12:10.322 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.312+0000 7fb7911ab640 1 -- 172.21.15.49:0/3437281397 wait complete. 2023-11-06T02:12:10.322 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.312+0000 7fb7911ab640 1 Processor -- start 2023-11-06T02:12:10.322 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.312+0000 7fb7911ab640 1 -- start start 2023-11-06T02:12:10.322 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.316+0000 7fb7911ab640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb774003980 0x7fb78c0b6d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:10.323 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.316+0000 7fb7911ab640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb78c158f30 0x7fb78c0ad000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:10.323 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.316+0000 7fb7911ab640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb78c1593b0 0x7fb78c0ad540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:10.323 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.316+0000 7fb7911ab640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb78c1f37e0 con 0x7fb774003980 2023-11-06T02:12:10.323 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.316+0000 7fb7911ab640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb78c0b72a0 con 0x7fb78c158f30 2023-11-06T02:12:10.323 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.316+0000 7fb7911ab640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb78c1f2bd0 con 0x7fb78c1593b0 2023-11-06T02:12:10.323 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.316+0000 7fb78ad9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb774003980 0x7fb78c0b6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:10.324 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.316+0000 7fb78ad9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb774003980 0x7fb78c0b6d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37074/0 (socket says 172.21.15.49:37074) 2023-11-06T02:12:10.324 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.316+0000 7fb78ad9d640 1 -- 172.21.15.49:0/2917487584 learned_addr learned my addr 172.21.15.49:0/2917487584 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:12:10.324 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.316+0000 7fb78a59c640 1 --2- 172.21.15.49:0/2917487584 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb78c158f30 0x7fb78c0ad000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:10.324 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.316+0000 7fb78b59e640 1 --2- 172.21.15.49:0/2917487584 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb78c1593b0 0x7fb78c0ad540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:10.324 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.316+0000 7fb78ad9d640 1 -- 172.21.15.49:0/2917487584 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb78c1593b0 msgr2=0x7fb78c0ad540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:10.325 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.316+0000 7fb78ad9d640 1 --2- 172.21.15.49:0/2917487584 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb78c1593b0 0x7fb78c0ad540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:10.325 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.316+0000 7fb78ad9d640 1 -- 172.21.15.49:0/2917487584 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb78c158f30 msgr2=0x7fb78c0ad000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:10.325 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.316+0000 7fb78ad9d640 1 --2- 172.21.15.49:0/2917487584 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb78c158f30 0x7fb78c0ad000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:10.325 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.316+0000 7fb78ad9d640 1 -- 172.21.15.49:0/2917487584 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb78c1f2eb0 con 0x7fb774003980 2023-11-06T02:12:10.325 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.316+0000 7fb78ad9d640 1 --2- 172.21.15.49:0/2917487584 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb774003980 0x7fb78c0b6d60 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fb77c009ae0 tx=0x7fb774005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:10.326 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.316+0000 7fb7737fe640 1 -- 172.21.15.49:0/2917487584 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb77c0b5070 con 0x7fb774003980 2023-11-06T02:12:10.326 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.316+0000 7fb7737fe640 1 -- 172.21.15.49:0/2917487584 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb77c00f000 con 0x7fb774003980 2023-11-06T02:12:10.326 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.316+0000 7fb7911ab640 1 -- 172.21.15.49:0/2917487584 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb78c1f3190 con 0x7fb774003980 2023-11-06T02:12:10.326 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.316+0000 7fb7737fe640 1 -- 172.21.15.49:0/2917487584 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb77c0a64a0 con 0x7fb774003980 2023-11-06T02:12:10.326 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.316+0000 7fb7911ab640 1 -- 172.21.15.49:0/2917487584 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb78c0b52a0 con 0x7fb774003980 2023-11-06T02:12:10.326 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.316+0000 7fb7737fe640 1 -- 172.21.15.49:0/2917487584 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fb77c0b9070 con 0x7fb774003980 2023-11-06T02:12:10.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.316+0000 7fb7737fe640 1 --2- 172.21.15.49:0/2917487584 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb778065a60 0x7fb778067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:10.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.316+0000 7fb7737fe640 1 -- 172.21.15.49:0/2917487584 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb77c11d200 con 0x7fb774003980 2023-11-06T02:12:10.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.316+0000 7fb78a59c640 1 --2- 172.21.15.49:0/2917487584 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb778065a60 0x7fb778067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:10.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.320+0000 7fb7942d3000 1 Processor -- start 2023-11-06T02:12:10.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.320+0000 7fb7942d3000 1 -- start start 2023-11-06T02:12:10.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.320+0000 7fb78a59c640 1 --2- 172.21.15.49:0/2917487584 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb778065a60 0x7fb778067f20 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fb78c0b4840 tx=0x7fb780009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:10.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.320+0000 7fb7942d3000 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55bdf3deb130 0x55bdf3deb510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:10.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.320+0000 7fb7942d3000 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdf3deba50 0x55bdf3da0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:10.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.320+0000 7fb7942d3000 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bdf3da1260 0x55bdf3da1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:10.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.320+0000 7fb7942d3000 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdf3c68c10 con 0x55bdf3deba50 2023-11-06T02:12:10.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.320+0000 7fb7942d3000 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdf3bf73e0 con 0x55bdf3da1260 2023-11-06T02:12:10.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.320+0000 7fb7942d3000 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bdf3bf49b0 con 0x55bdf3deb130 2023-11-06T02:12:10.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.320+0000 7fb78a59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdf3deba50 0x55bdf3da0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:10.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.320+0000 7fb78a59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdf3deba50 0x55bdf3da0d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37078/0 (socket says 172.21.15.49:37078) 2023-11-06T02:12:10.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.320+0000 7fb78a59c640 1 -- 172.21.15.49:0/2019530038 learned_addr learned my addr 172.21.15.49:0/2019530038 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:12:10.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.320+0000 7fb78ad9d640 1 --2- 172.21.15.49:0/2019530038 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55bdf3deb130 0x55bdf3deb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:10.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.320+0000 7fb78b59e640 1 --2- 172.21.15.49:0/2019530038 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bdf3da1260 0x55bdf3da1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:10.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.320+0000 7fb78a59c640 1 -- 172.21.15.49:0/2019530038 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55bdf3deb130 msgr2=0x55bdf3deb510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:10.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.320+0000 7fb78a59c640 1 --2- 172.21.15.49:0/2019530038 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55bdf3deb130 0x55bdf3deb510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:10.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.320+0000 7fb78a59c640 1 -- 172.21.15.49:0/2019530038 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bdf3da1260 msgr2=0x55bdf3da1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:10.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.320+0000 7fb78a59c640 1 --2- 172.21.15.49:0/2019530038 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bdf3da1260 0x55bdf3da1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:10.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.320+0000 7fb78a59c640 1 -- 172.21.15.49:0/2019530038 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bdf38ece20 con 0x55bdf3deba50 2023-11-06T02:12:10.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.320+0000 7fb78b59e640 1 --2- 172.21.15.49:0/2019530038 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bdf3da1260 0x55bdf3da1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:12:10.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.320+0000 7fb78a59c640 1 --2- 172.21.15.49:0/2019530038 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdf3deba50 0x55bdf3da0d20 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fb780001490 tx=0x7fb780016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3007a408cb26ba2b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:10.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.320+0000 7fb770ff9640 1 -- 172.21.15.49:0/2019530038 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb78001b040 con 0x55bdf3deba50 2023-11-06T02:12:10.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.320+0000 7fb770ff9640 1 -- 172.21.15.49:0/2019530038 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb780016c00 con 0x55bdf3deba50 2023-11-06T02:12:10.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.320+0000 7fb770ff9640 1 -- 172.21.15.49:0/2019530038 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7800223c0 con 0x55bdf3deba50 2023-11-06T02:12:10.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.320+0000 7fb7942d3000 1 -- 172.21.15.49:0/2019530038 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdf3deba50 msgr2=0x55bdf3da0d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:10.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.320+0000 7fb7942d3000 1 --2- 172.21.15.49:0/2019530038 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdf3deba50 0x55bdf3da0d20 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fb780001490 tx=0x7fb780016000 comp rx=0 tx=0).stop 2023-11-06T02:12:10.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.320+0000 7fb7942d3000 1 -- 172.21.15.49:0/2019530038 shutdown_connections 2023-11-06T02:12:10.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.320+0000 7fb7942d3000 1 --2- 172.21.15.49:0/2019530038 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdf3deba50 0x55bdf3da0d20 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:10.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.320+0000 7fb7942d3000 1 --2- 172.21.15.49:0/2019530038 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bdf3da1260 0x55bdf3da1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:10.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.320+0000 7fb7942d3000 1 --2- 172.21.15.49:0/2019530038 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55bdf3deb130 0x55bdf3deb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:10.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.320+0000 7fb7942d3000 1 -- 172.21.15.49:0/2019530038 >> 172.21.15.49:0/2019530038 conn(0x55bdf3da79f0 msgr2=0x55bdf3d9f600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:12:10.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.320+0000 7fb7942d3000 1 -- 172.21.15.49:0/2019530038 shutdown_connections 2023-11-06T02:12:10.334 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.320+0000 7fb7942d3000 1 -- 172.21.15.49:0/2019530038 wait complete. 2023-11-06T02:12:10.334 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.320+0000 7fb7942d3000 1 -- start start 2023-11-06T02:12:10.334 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.320+0000 7fb7942d3000 1 Processor -- start 2023-11-06T02:12:10.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.324+0000 7fb7942d3000 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bdf3da1260 0x55bdf3dea6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:10.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.324+0000 7fb7942d3000 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55bdf3deb130 0x55bdf3dec040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:10.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.324+0000 7fb7942d3000 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdf3deba50 0x55bdf3dec580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:10.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.324+0000 7fb7942d3000 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdf3bf02a0 con 0x55bdf3deba50 2023-11-06T02:12:10.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.324+0000 7fb7942d3000 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdf3b8f470 con 0x55bdf3da1260 2023-11-06T02:12:10.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.324+0000 7fb7942d3000 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bdf3bad660 con 0x55bdf3deb130 2023-11-06T02:12:10.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.324+0000 7fb7942d3000 10 client.0 _ll_register_callbacks cb 0x55bdf3d8cc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T02:12:10.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.324+0000 7fb78b59e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdf3deba50 0x55bdf3dec580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:10.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.324+0000 7fb78b59e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdf3deba50 0x55bdf3dec580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37080/0 (socket says 172.21.15.49:37080) 2023-11-06T02:12:10.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.324+0000 7fb78b59e640 1 -- 172.21.15.49:0/2721959340 learned_addr learned my addr 172.21.15.49:0/2721959340 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:12:10.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.324+0000 7fb78ad9d640 1 --2- 172.21.15.49:0/2721959340 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bdf3da1260 0x55bdf3dea6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:10.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.324+0000 7fb78a59c640 1 --2- 172.21.15.49:0/2721959340 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55bdf3deb130 0x55bdf3dec040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:10.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.324+0000 7fb78b59e640 1 -- 172.21.15.49:0/2721959340 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55bdf3deb130 msgr2=0x55bdf3dec040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:10.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.324+0000 7fb78b59e640 1 --2- 172.21.15.49:0/2721959340 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55bdf3deb130 0x55bdf3dec040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:10.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.324+0000 7fb78b59e640 1 -- 172.21.15.49:0/2721959340 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bdf3da1260 msgr2=0x55bdf3dea6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:10.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.324+0000 7fb78b59e640 1 --2- 172.21.15.49:0/2721959340 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bdf3da1260 0x55bdf3dea6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:10.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.324+0000 7fb78b59e640 1 -- 172.21.15.49:0/2721959340 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb78000f030 con 0x55bdf3deba50 2023-11-06T02:12:10.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.324+0000 7fb769ffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-11-06T02:12:10.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.324+0000 7fb78b59e640 1 --2- 172.21.15.49:0/2721959340 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdf3deba50 0x55bdf3dec580 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fb78c0c9b50 tx=0x7fb784004700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e0a7d3ecd203426 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:10.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.324+0000 7fb769ffb640 1 -- 172.21.15.49:0/2721959340 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb78400e040 con 0x55bdf3deba50 2023-11-06T02:12:10.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.324+0000 7fb769ffb640 1 -- 172.21.15.49:0/2721959340 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb784007a70 con 0x55bdf3deba50 2023-11-06T02:12:10.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.324+0000 7fb7942d3000 1 -- 172.21.15.49:0/2721959340 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bdf3bad0a0 con 0x55bdf3deba50 2023-11-06T02:12:10.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.324+0000 7fb769ffb640 1 -- 172.21.15.49:0/2721959340 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb784007c30 con 0x55bdf3deba50 2023-11-06T02:12:10.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.324+0000 7fb7942d3000 10 client.6756 fetch_fsmap learned FSMap version 30 2023-11-06T02:12:10.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.324+0000 7fb7942d3000 1 -- 172.21.15.49:0/2721959340 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=2+}) v3 -- 0x55bdf3bb51d0 con 0x55bdf3deba50 2023-11-06T02:12:10.344 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.324+0000 7fb769ffb640 1 -- 172.21.15.49:0/2721959340 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bdf3bad0a0 con 0x55bdf3deba50 2023-11-06T02:12:10.344 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.324+0000 7fb769ffb640 1 -- 172.21.15.49:0/2721959340 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 30) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fb784017470 con 0x55bdf3deba50 2023-11-06T02:12:10.344 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.324+0000 7fb7942d3000 10 client.6756 fetch_fsmap finished waiting for FSMap version 30 2023-11-06T02:12:10.344 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.324+0000 7fb7942d3000 10 client.6756 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2023-11-06T02:12:10.344 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.324+0000 7fb7942d3000 20 client.6756 populate_metadata read hostname 'smithi049' 2023-11-06T02:12:10.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.324+0000 7fb7942d3000 1 --2- 172.21.15.49:0/2721959340 >> [v2:172.21.15.49:6834/342908487,v1:172.21.15.49:6835/342908487] conn(0x55bdf3ded320 0x55bdf3dfd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:10.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.324+0000 7fb7942d3000 4 client.6756 mds_command: new command op to 6348 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T02:12:10.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.324+0000 7fb7942d3000 1 -- 172.21.15.49:0/2721959340 --> [v2:172.21.15.49:6834/342908487,v1:172.21.15.49:6835/342908487] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bdf3beebf0 con 0x55bdf3ded320 2023-11-06T02:12:10.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.324+0000 7fb78ad9d640 1 --2- 172.21.15.49:0/2721959340 >> [v2:172.21.15.49:6834/342908487,v1:172.21.15.49:6835/342908487] conn(0x55bdf3ded320 0x55bdf3dfd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:10.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.324+0000 7fb769ffb640 10 client.6756 ms_handle_connect on v2:172.21.15.49:6834/342908487 2023-11-06T02:12:10.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.324+0000 7fb78ad9d640 1 --2- 172.21.15.49:0/2721959340 >> [v2:172.21.15.49:6834/342908487,v1:172.21.15.49:6835/342908487] conn(0x55bdf3ded320 0x55bdf3dfd4d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fb1f8d6bbc95be8 server_cookie=262d513b39ab5683 in_seq=0 out_seq=0 2023-11-06T02:12:10.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.324+0000 7fb769ffb640 1 -- 172.21.15.49:0/2721959340 <== mds.0 v2:172.21.15.49:6834/342908487 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55bdf3beebf0 con 0x55bdf3ded320 2023-11-06T02:12:10.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.324+0000 7fb769ffb640 10 client.6756 handle_command_reply: tid=0 2023-11-06T02:12:10.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.324+0000 7fb7942d3000 1 client.6756 shutdown 2023-11-06T02:12:10.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.324+0000 7fb7942d3000 1 -- 172.21.15.49:0/2721959340 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdf3deba50 msgr2=0x55bdf3dec580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:10.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.324+0000 7fb7942d3000 1 --2- 172.21.15.49:0/2721959340 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdf3deba50 0x55bdf3dec580 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fb78c0c9b50 tx=0x7fb784004700 comp rx=0 tx=0).stop 2023-11-06T02:12:10.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.324+0000 7fb7942d3000 1 -- 172.21.15.49:0/2721959340 shutdown_connections 2023-11-06T02:12:10.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.324+0000 7fb7942d3000 1 --2- 172.21.15.49:0/2721959340 >> [v2:172.21.15.49:6834/342908487,v1:172.21.15.49:6835/342908487] conn(0x55bdf3ded320 0x55bdf3dfd4d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:10.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.324+0000 7fb7942d3000 1 --2- 172.21.15.49:0/2721959340 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdf3deba50 0x55bdf3dec580 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:10.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.324+0000 7fb769ffb640 0 client.6756 ms_handle_reset on v2:172.21.15.49:6834/342908487 2023-11-06T02:12:10.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.324+0000 7fb7942d3000 1 --2- 172.21.15.49:0/2721959340 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55bdf3deb130 0x55bdf3dec040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:10.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.324+0000 7fb7942d3000 1 --2- 172.21.15.49:0/2721959340 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bdf3da1260 0x55bdf3dea6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:10.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.324+0000 7fb7942d3000 1 -- 172.21.15.49:0/2721959340 >> 172.21.15.49:0/2721959340 conn(0x55bdf3da79f0 msgr2=0x55bdf3d9fd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:12:10.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.324+0000 7fb7942d3000 1 -- 172.21.15.49:0/2721959340 shutdown_connections 2023-11-06T02:12:10.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.324+0000 7fb7942d3000 1 -- 172.21.15.49:0/2721959340 wait complete. 2023-11-06T02:12:10.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.324+0000 7fb7942d3000 20 client.6756 trim_cache size 0 max 16384 2023-11-06T02:12:10.354 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.340+0000 7fb7942d3000 1 Processor -- start 2023-11-06T02:12:10.354 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.340+0000 7fb7942d3000 1 -- start start 2023-11-06T02:12:10.354 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.340+0000 7fb7942d3000 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bdf3da1260 0x55bdf3da6950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:10.354 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.340+0000 7fb7942d3000 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55bdf3deb130 0x55bdf3da6e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:10.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.340+0000 7fb7942d3000 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdf3deba50 0x55bdf3def710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:10.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.340+0000 7fb7942d3000 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdf38adaa0 con 0x55bdf3deba50 2023-11-06T02:12:10.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.340+0000 7fb7942d3000 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdf3bebc10 con 0x55bdf3da1260 2023-11-06T02:12:10.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.340+0000 7fb7942d3000 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bdf3bef130 con 0x55bdf3deb130 2023-11-06T02:12:10.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.340+0000 7fb78b59e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdf3deba50 0x55bdf3def710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:10.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.340+0000 7fb78b59e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdf3deba50 0x55bdf3def710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37086/0 (socket says 172.21.15.49:37086) 2023-11-06T02:12:10.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.340+0000 7fb78b59e640 1 -- 172.21.15.49:0/3845759793 learned_addr learned my addr 172.21.15.49:0/3845759793 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:12:10.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.340+0000 7fb78a59c640 1 --2- 172.21.15.49:0/3845759793 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55bdf3deb130 0x55bdf3da6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:10.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.340+0000 7fb78ad9d640 1 --2- 172.21.15.49:0/3845759793 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bdf3da1260 0x55bdf3da6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:10.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.340+0000 7fb78b59e640 1 -- 172.21.15.49:0/3845759793 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55bdf3deb130 msgr2=0x55bdf3da6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:10.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.340+0000 7fb78b59e640 1 --2- 172.21.15.49:0/3845759793 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55bdf3deb130 0x55bdf3da6e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:10.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.340+0000 7fb78b59e640 1 -- 172.21.15.49:0/3845759793 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bdf3da1260 msgr2=0x55bdf3da6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:10.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.340+0000 7fb78b59e640 1 --2- 172.21.15.49:0/3845759793 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bdf3da1260 0x55bdf3da6950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:10.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.340+0000 7fb78b59e640 1 -- 172.21.15.49:0/3845759793 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bdf3b9b520 con 0x55bdf3deba50 2023-11-06T02:12:10.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.340+0000 7fb78a59c640 1 --2- 172.21.15.49:0/3845759793 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55bdf3deb130 0x55bdf3da6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:12:10.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.340+0000 7fb78ad9d640 1 --2- 172.21.15.49:0/3845759793 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bdf3da1260 0x55bdf3da6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:12:10.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.340+0000 7fb78b59e640 1 --2- 172.21.15.49:0/3845759793 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdf3deba50 0x55bdf3def710 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fb784003dc0 tx=0x7fb784017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3eff7e9af3cdb58 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:10.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.340+0000 7fb7717fa640 1 -- 172.21.15.49:0/3845759793 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb78400e070 con 0x55bdf3deba50 2023-11-06T02:12:10.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.340+0000 7fb7717fa640 1 -- 172.21.15.49:0/3845759793 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb784017980 con 0x55bdf3deba50 2023-11-06T02:12:10.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.340+0000 7fb7717fa640 1 -- 172.21.15.49:0/3845759793 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb784009b10 con 0x55bdf3deba50 2023-11-06T02:12:10.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.340+0000 7fb7942d3000 1 -- 172.21.15.49:0/3845759793 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdf3deba50 msgr2=0x55bdf3def710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:10.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.340+0000 7fb7942d3000 1 --2- 172.21.15.49:0/3845759793 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdf3deba50 0x55bdf3def710 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fb784003dc0 tx=0x7fb784017000 comp rx=0 tx=0).stop 2023-11-06T02:12:10.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.340+0000 7fb7942d3000 1 -- 172.21.15.49:0/3845759793 shutdown_connections 2023-11-06T02:12:10.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.340+0000 7fb7942d3000 1 --2- 172.21.15.49:0/3845759793 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdf3deba50 0x55bdf3def710 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:10.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.340+0000 7fb7942d3000 1 --2- 172.21.15.49:0/3845759793 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55bdf3deb130 0x55bdf3da6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:10.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.340+0000 7fb7942d3000 1 --2- 172.21.15.49:0/3845759793 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bdf3da1260 0x55bdf3da6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:10.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.340+0000 7fb7942d3000 1 -- 172.21.15.49:0/3845759793 >> 172.21.15.49:0/3845759793 conn(0x55bdf3da79f0 msgr2=0x55bdf3df89d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:12:10.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.344+0000 7fb7942d3000 1 -- 172.21.15.49:0/3845759793 shutdown_connections 2023-11-06T02:12:10.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.344+0000 7fb7942d3000 1 -- 172.21.15.49:0/3845759793 wait complete. 2023-11-06T02:12:10.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.344+0000 7fb7942d3000 1 -- start start 2023-11-06T02:12:10.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.344+0000 7fb7942d3000 1 Processor -- start 2023-11-06T02:12:10.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.344+0000 7fb7942d3000 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bdf3da1260 0x55bdf3e480b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:10.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.344+0000 7fb7942d3000 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdf3deb130 0x55bdf3e485f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:10.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.344+0000 7fb7942d3000 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55bdf3deba50 0x55bdf3e2bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:10.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.344+0000 7fb7942d3000 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdf3da3880 con 0x55bdf3deb130 2023-11-06T02:12:10.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.344+0000 7fb7942d3000 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdf3c32cd0 con 0x55bdf3da1260 2023-11-06T02:12:10.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.344+0000 7fb7942d3000 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bdf3be8840 con 0x55bdf3deba50 2023-11-06T02:12:10.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.344+0000 7fb7942d3000 10 client.0 _ll_register_callbacks cb 0x55bdf3bb4cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T02:12:10.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.344+0000 7fb78a59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdf3deb130 0x55bdf3e485f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:10.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.344+0000 7fb78a59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdf3deb130 0x55bdf3e485f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37100/0 (socket says 172.21.15.49:37100) 2023-11-06T02:12:10.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.344+0000 7fb78a59c640 1 -- 172.21.15.49:0/4071617790 learned_addr learned my addr 172.21.15.49:0/4071617790 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:12:10.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.344+0000 7fb78ad9d640 1 --2- 172.21.15.49:0/4071617790 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bdf3da1260 0x55bdf3e480b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:10.365 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.344+0000 7fb78b59e640 1 --2- 172.21.15.49:0/4071617790 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55bdf3deba50 0x55bdf3e2bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:10.365 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.344+0000 7fb78a59c640 1 -- 172.21.15.49:0/4071617790 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55bdf3deba50 msgr2=0x55bdf3e2bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:10.365 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.344+0000 7fb78a59c640 1 --2- 172.21.15.49:0/4071617790 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55bdf3deba50 0x55bdf3e2bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:10.365 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.344+0000 7fb78a59c640 1 -- 172.21.15.49:0/4071617790 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bdf3da1260 msgr2=0x55bdf3e480b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:10.365 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.344+0000 7fb78a59c640 1 --2- 172.21.15.49:0/4071617790 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bdf3da1260 0x55bdf3e480b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:10.366 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.344+0000 7fb78a59c640 1 -- 172.21.15.49:0/4071617790 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bdf3bb51d0 con 0x55bdf3deb130 2023-11-06T02:12:10.366 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.344+0000 7fb78a59c640 1 --2- 172.21.15.49:0/4071617790 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdf3deb130 0x55bdf3e485f0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fb780009010 tx=0x7fb780001a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b141a9cdd6e2428f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:10.366 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.344+0000 7fb769ffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-11-06T02:12:10.366 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.344+0000 7fb769ffb640 1 -- 172.21.15.49:0/4071617790 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb780023830 con 0x55bdf3deb130 2023-11-06T02:12:10.366 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.344+0000 7fb769ffb640 1 -- 172.21.15.49:0/4071617790 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(65..65 src has 1..65) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb780023990 con 0x55bdf3deb130 2023-11-06T02:12:10.367 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.344+0000 7fb7942d3000 1 -- 172.21.15.49:0/4071617790 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bdf3bb9620 con 0x55bdf3deb130 2023-11-06T02:12:10.367 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.344+0000 7fb769ffb640 1 -- 172.21.15.49:0/4071617790 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb780023b50 con 0x55bdf3deb130 2023-11-06T02:12:10.367 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.344+0000 7fb7942d3000 10 client.6762 fetch_fsmap learned FSMap version 30 2023-11-06T02:12:10.367 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.344+0000 7fb7942d3000 1 -- 172.21.15.49:0/4071617790 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=2+}) v3 -- 0x55bdf3b8f090 con 0x55bdf3deb130 2023-11-06T02:12:10.367 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.344+0000 7fb769ffb640 1 -- 172.21.15.49:0/4071617790 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bdf3bb9620 con 0x55bdf3deb130 2023-11-06T02:12:10.367 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.344+0000 7fb769ffb640 1 -- 172.21.15.49:0/4071617790 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 30) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fb780021040 con 0x55bdf3deb130 2023-11-06T02:12:10.368 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.344+0000 7fb7942d3000 10 client.6762 fetch_fsmap finished waiting for FSMap version 30 2023-11-06T02:12:10.368 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.344+0000 7fb7942d3000 10 client.6762 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2023-11-06T02:12:10.368 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.344+0000 7fb7942d3000 20 client.6762 populate_metadata read hostname 'smithi049' 2023-11-06T02:12:10.368 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.344+0000 7fb7942d3000 1 --2- 172.21.15.49:0/4071617790 >> [v2:172.21.15.49:6834/342908487,v1:172.21.15.49:6835/342908487] conn(0x55bdf3e2c140 0x55bdf3e2c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:10.368 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.344+0000 7fb7942d3000 4 client.6762 mds_command: new command op to 6348 tid=0[{"prefix": "scrub status"}] 2023-11-06T02:12:10.369 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.344+0000 7fb7942d3000 1 -- 172.21.15.49:0/4071617790 --> [v2:172.21.15.49:6834/342908487,v1:172.21.15.49:6835/342908487] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55bdf3be9320 con 0x55bdf3e2c140 2023-11-06T02:12:10.369 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.344+0000 7fb78ad9d640 1 --2- 172.21.15.49:0/4071617790 >> [v2:172.21.15.49:6834/342908487,v1:172.21.15.49:6835/342908487] conn(0x55bdf3e2c140 0x55bdf3e2c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:10.369 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.344+0000 7fb78ad9d640 1 --2- 172.21.15.49:0/4071617790 >> [v2:172.21.15.49:6834/342908487,v1:172.21.15.49:6835/342908487] conn(0x55bdf3e2c140 0x55bdf3e2c520 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c7b883db26ddd43 server_cookie=bdbaf11b816d9441 in_seq=0 out_seq=0 2023-11-06T02:12:10.369 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.344+0000 7fb769ffb640 10 client.6762 ms_handle_connect on v2:172.21.15.49:6834/342908487 2023-11-06T02:12:10.369 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.344+0000 7fb769ffb640 1 -- 172.21.15.49:0/4071617790 <== mds.0 v2:172.21.15.49:6834/342908487 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+260 (crc 0 0 0) 0x55bdf3be9320 con 0x55bdf3e2c140 2023-11-06T02:12:10.370 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.344+0000 7fb769ffb640 10 client.6762 handle_command_reply: tid=0 2023-11-06T02:12:10.370 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.344+0000 7fb7942d3000 1 client.6762 shutdown 2023-11-06T02:12:10.370 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.344+0000 7fb7942d3000 1 -- 172.21.15.49:0/4071617790 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdf3deb130 msgr2=0x55bdf3e485f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:10.370 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.344+0000 7fb7942d3000 1 --2- 172.21.15.49:0/4071617790 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdf3deb130 0x55bdf3e485f0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fb780009010 tx=0x7fb780001a10 comp rx=0 tx=0).stop 2023-11-06T02:12:10.370 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.344+0000 7fb7942d3000 1 -- 172.21.15.49:0/4071617790 shutdown_connections 2023-11-06T02:12:10.370 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.344+0000 7fb7942d3000 1 --2- 172.21.15.49:0/4071617790 >> [v2:172.21.15.49:6834/342908487,v1:172.21.15.49:6835/342908487] conn(0x55bdf3e2c140 0x55bdf3e2c520 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:10.371 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.344+0000 7fb769ffb640 0 client.6762 ms_handle_reset on v2:172.21.15.49:6834/342908487 2023-11-06T02:12:10.371 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.344+0000 7fb7942d3000 1 --2- 172.21.15.49:0/4071617790 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bdf3deb130 0x55bdf3e485f0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:10.371 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.344+0000 7fb7942d3000 1 --2- 172.21.15.49:0/4071617790 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55bdf3deba50 0x55bdf3e2bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:10.371 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.344+0000 7fb7942d3000 1 --2- 172.21.15.49:0/4071617790 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bdf3da1260 0x55bdf3e480b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:10.371 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.344+0000 7fb7942d3000 1 -- 172.21.15.49:0/4071617790 >> 172.21.15.49:0/4071617790 conn(0x55bdf3da79f0 msgr2=0x55bdf3da5940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:12:10.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.344+0000 7fb7942d3000 1 -- 172.21.15.49:0/4071617790 shutdown_connections 2023-11-06T02:12:10.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.344+0000 7fb7942d3000 1 -- 172.21.15.49:0/4071617790 wait complete. 2023-11-06T02:12:10.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.344+0000 7fb7942d3000 20 client.6762 trim_cache size 0 max 16384 2023-11-06T02:12:10.373 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-11-06T02:12:10.373 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-06T02:12:10.373 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": { 2023-11-06T02:12:10.373 INFO:teuthology.orchestra.run.smithi049.stdout: "93f7c4f5-ab96-448b-808d-f2d0cb201719": { 2023-11-06T02:12:10.373 INFO:teuthology.orchestra.run.smithi049.stdout: "path": "/", 2023-11-06T02:12:10.374 INFO:teuthology.orchestra.run.smithi049.stdout: "tag": "93f7c4f5-ab96-448b-808d-f2d0cb201719", 2023-11-06T02:12:10.374 INFO:teuthology.orchestra.run.smithi049.stdout: "options": "recursive,repair" 2023-11-06T02:12:10.374 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:12:10.374 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:12:10.374 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-11-06T02:12:10.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.348+0000 7fb7911ab640 1 -- 172.21.15.49:0/2917487584 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb778065a60 msgr2=0x7fb778067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:10.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.348+0000 7fb7911ab640 1 --2- 172.21.15.49:0/2917487584 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb778065a60 0x7fb778067f20 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fb78c0b4840 tx=0x7fb780009040 comp rx=0 tx=0).stop 2023-11-06T02:12:10.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.348+0000 7fb7911ab640 1 -- 172.21.15.49:0/2917487584 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb774003980 msgr2=0x7fb78c0b6d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:10.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.348+0000 7fb7911ab640 1 --2- 172.21.15.49:0/2917487584 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb774003980 0x7fb78c0b6d60 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fb77c009ae0 tx=0x7fb774005800 comp rx=0 tx=0).stop 2023-11-06T02:12:10.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.348+0000 7fb7911ab640 1 -- 172.21.15.49:0/2917487584 shutdown_connections 2023-11-06T02:12:10.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.348+0000 7fb7911ab640 1 --2- 172.21.15.49:0/2917487584 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb778065a60 0x7fb778067f20 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:10.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.348+0000 7fb7911ab640 1 --2- 172.21.15.49:0/2917487584 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb78c1593b0 0x7fb78c0ad540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:10.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.348+0000 7fb7911ab640 1 --2- 172.21.15.49:0/2917487584 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb78c158f30 0x7fb78c0ad000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:10.378 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.348+0000 7fb7911ab640 1 --2- 172.21.15.49:0/2917487584 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb774003980 0x7fb78c0b6d60 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:10.378 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.348+0000 7fb7911ab640 1 -- 172.21.15.49:0/2917487584 >> 172.21.15.49:0/2917487584 conn(0x7fb78c0718a0 msgr2=0x7fb78c0c9ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:12:10.378 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.348+0000 7fb7911ab640 1 -- 172.21.15.49:0/2917487584 shutdown_connections 2023-11-06T02:12:10.378 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:10.348+0000 7fb7911ab640 1 -- 172.21.15.49:0/2917487584 wait complete. 2023-11-06T02:12:10.378 INFO:tasks.cephfs.filesystem:scrub status for tag:93f7c4f5-ab96-448b-808d-f2d0cb201719 - {'path': '/', 'tag': '93f7c4f5-ab96-448b-808d-f2d0cb201719', 'options': 'recursive,repair'} 2023-11-06T02:12:11.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:11.356+0000 7f6975ffb640 1 -- 172.21.15.49:0/1194616221 <== mon.0 v2:172.21.15.49:3300/0 7 ==== log(5 entries from seq 1 at 2023-11-06T02:12:10.143060+0000) v1 ==== 846+0+0 (secure 0 0 0) 0x7f6970055e30 con 0x7f69800d3590 2023-11-06T02:12:11.356 INFO:teuthology.orchestra.run.smithi049.stdout: cluster: 2023-11-06T02:12:11.356 INFO:teuthology.orchestra.run.smithi049.stdout: id: 1a8b035c-6ba4-499a-b876-aea47fd9c0d4 2023-11-06T02:12:11.356 INFO:teuthology.orchestra.run.smithi049.stdout: health: HEALTH_OK 2023-11-06T02:12:11.356 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:12:11.357 INFO:teuthology.orchestra.run.smithi049.stdout: services: 2023-11-06T02:12:11.357 INFO:teuthology.orchestra.run.smithi049.stdout: mon: 3 daemons, quorum a,b,c (age 4m) 2023-11-06T02:12:11.357 INFO:teuthology.orchestra.run.smithi049.stdout: mgr: x(active, since 3m), standbys: y 2023-11-06T02:12:11.357 INFO:teuthology.orchestra.run.smithi049.stdout: mds: 1/1 daemons up, 3 standby 2023-11-06T02:12:11.357 INFO:teuthology.orchestra.run.smithi049.stdout: osd: 8 osds: 8 up (since 3m), 8 in (since 4m) 2023-11-06T02:12:11.358 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:12:11.358 INFO:teuthology.orchestra.run.smithi049.stdout: data: 2023-11-06T02:12:11.358 INFO:teuthology.orchestra.run.smithi049.stdout: volumes: 1/1 healthy 2023-11-06T02:12:11.358 INFO:teuthology.orchestra.run.smithi049.stdout: pools: 4 pools, 105 pgs 2023-11-06T02:12:11.358 INFO:teuthology.orchestra.run.smithi049.stdout: objects: 30 objects, 19 MiB 2023-11-06T02:12:11.359 INFO:teuthology.orchestra.run.smithi049.stdout: usage: 274 MiB used, 720 GiB / 720 GiB avail 2023-11-06T02:12:11.359 INFO:teuthology.orchestra.run.smithi049.stdout: pgs: 105 active+clean 2023-11-06T02:12:11.359 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:12:11.359 INFO:teuthology.orchestra.run.smithi049.stdout: io: 2023-11-06T02:12:11.359 INFO:teuthology.orchestra.run.smithi049.stdout: client: 85 B/s rd, 1.0 MiB/s wr, 0 op/s rd, 2 op/s wr 2023-11-06T02:12:11.360 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:12:11.360 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:12:11.360 INFO:teuthology.orchestra.run.smithi049.stdout:2023-11-06T02:12:10.143060+0000 mds.c [INF] scrub queued for path: / 2023-11-06T02:12:11.360 INFO:teuthology.orchestra.run.smithi049.stdout:2023-11-06T02:12:10.143078+0000 mds.c [INF] scrub summary: idle+waiting paths [/] 2023-11-06T02:12:11.360 INFO:teuthology.orchestra.run.smithi049.stdout:2023-11-06T02:12:10.143086+0000 mds.c [INF] scrub summary: active paths [/] 2023-11-06T02:12:11.360 INFO:teuthology.orchestra.run.smithi049.stdout:2023-11-06T02:12:10.144343+0000 mds.c [INF] scrub summary: idle+waiting paths [/] 2023-11-06T02:12:11.361 INFO:teuthology.orchestra.run.smithi049.stdout:2023-11-06T02:12:10.586462+0000 mds.c [INF] scrub summary: idle 2023-11-06T02:12:14.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:14.376+0000 7f6975ffb640 1 -- 172.21.15.49:0/1194616221 <== mon.0 v2:172.21.15.49:3300/0 8 ==== log(8 entries from seq 112 at 2023-11-06T02:12:14.201339+0000) v1 ==== 2192+0+0 (secure 0 0 0) 0x7f6970055730 con 0x7f69800d3590 2023-11-06T02:12:15.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:15.396+0000 7f6975ffb640 1 -- 172.21.15.49:0/1194616221 <== mon.0 v2:172.21.15.49:3300/0 9 ==== log(4 entries from seq 624 at 2023-11-06T02:12:14.374830+0000) v1 ==== 1081+0+0 (secure 0 0 0) 0x7f6970082660 con 0x7f69800d3590 2023-11-06T02:12:21.965 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T02:12:21.968 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T02:12:40.380 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 scrub status 2023-11-06T02:12:40.517 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.519+0000 7f42835c9640 1 Processor -- start 2023-11-06T02:12:40.517 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.519+0000 7f42835c9640 1 -- start start 2023-11-06T02:12:40.518 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.519+0000 7f42835c9640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f427c15e0c0 0x7f427c0ce840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:40.519 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.519+0000 7f42835c9640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f427c15e4a0 0x7f427c0cdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:40.519 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.519+0000 7f42835c9640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f427c0ce230 con 0x7f427c0b16e0 2023-11-06T02:12:40.519 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.519+0000 7f42835c9640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f427c0cd8d0 con 0x7f427c15e0c0 2023-11-06T02:12:40.519 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.519+0000 7f42835c9640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f427c0d1430 con 0x7f427c15e4a0 2023-11-06T02:12:40.520 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.519+0000 7f4281365640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f427c15e0c0 0x7f427c0ce840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:40.520 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.519+0000 7f4281b66640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f427c0b16e0 0x7f427c0b0100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:42658/0 (socket says 172.21.15.49:42658) 2023-11-06T02:12:40.520 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.519+0000 7f4281b66640 1 -- 172.21.15.49:0/2165054118 learned_addr learned my addr 172.21.15.49:0/2165054118 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:12:40.520 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.519+0000 7f4280b64640 1 --2- 172.21.15.49:0/2165054118 >> v2:172.21.15.150:3300/0 conn(0x7f427c15e4a0 0x7f427c0cdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:40.521 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.519+0000 7f4281365640 1 -- 172.21.15.49:0/2165054118 >> v2:172.21.15.150:3300/0 conn(0x7f427c15e4a0 msgr2=0x7f427c0cdcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:40.521 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.519+0000 7f4281365640 1 --2- 172.21.15.49:0/2165054118 >> v2:172.21.15.150:3300/0 conn(0x7f427c15e4a0 0x7f427c0cdcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:40.521 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.519+0000 7f4281365640 1 -- 172.21.15.49:0/2165054118 >> v1:172.21.15.49:6789/0 conn(0x7f427c0b16e0 legacy=0x7f427c0b0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:40.521 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.519+0000 7f4273fff640 1 -- 172.21.15.49:0/2165054118 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3914260405 0 0) 0x7f427c0ce230 con 0x7f427c0b16e0 2023-11-06T02:12:40.522 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.519+0000 7f4281365640 1 -- 172.21.15.49:0/2165054118 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f427c0d1710 con 0x7f427c15e0c0 2023-11-06T02:12:40.522 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.523+0000 7f4281365640 1 --2- 172.21.15.49:0/2165054118 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f427c15e0c0 0x7f427c0ce840 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f427c073c70 tx=0x7f426c01c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7a3988f0c60c14 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:40.523 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.523+0000 7f4273fff640 1 -- 172.21.15.49:0/2165054118 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426c0ab600 con 0x7f427c15e0c0 2023-11-06T02:12:40.523 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.523+0000 7f4273fff640 1 -- 172.21.15.49:0/2165054118 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f426c0a7050 con 0x7f427c15e0c0 2023-11-06T02:12:40.524 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.523+0000 7f4273fff640 1 -- 172.21.15.49:0/2165054118 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426c0ab980 con 0x7f427c15e0c0 2023-11-06T02:12:40.524 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.523+0000 7f42835c9640 1 -- 172.21.15.49:0/2165054118 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f427c15e0c0 msgr2=0x7f427c0ce840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:40.524 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.523+0000 7f42835c9640 1 --2- 172.21.15.49:0/2165054118 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f427c15e0c0 0x7f427c0ce840 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f427c073c70 tx=0x7f426c01c3d0 comp rx=0 tx=0).stop 2023-11-06T02:12:40.524 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.523+0000 7f42835c9640 1 -- 172.21.15.49:0/2165054118 shutdown_connections 2023-11-06T02:12:40.524 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.523+0000 7f42835c9640 1 --2- 172.21.15.49:0/2165054118 >> v2:172.21.15.150:3300/0 conn(0x7f427c15e4a0 0x7f427c0cdcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:40.525 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.523+0000 7f42835c9640 1 --2- 172.21.15.49:0/2165054118 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f427c15e0c0 0x7f427c0ce840 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:40.525 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.523+0000 7f42835c9640 1 -- 172.21.15.49:0/2165054118 >> 172.21.15.49:0/2165054118 conn(0x7f427c099c50 msgr2=0x7f427c0d0940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:12:40.525 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.523+0000 7f42835c9640 1 -- 172.21.15.49:0/2165054118 shutdown_connections 2023-11-06T02:12:40.525 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.523+0000 7f42835c9640 1 -- 172.21.15.49:0/2165054118 wait complete. 2023-11-06T02:12:40.525 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.523+0000 7f42835c9640 1 Processor -- start 2023-11-06T02:12:40.526 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.523+0000 7f42835c9640 1 -- start start 2023-11-06T02:12:40.528 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.523+0000 7f42835c9640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f427c0b16e0 0x7f427c155e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:40.528 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.523+0000 7f42835c9640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f427c15e0c0 0x7f427c0b3560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:40.528 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.523+0000 7f42835c9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f427c15e4a0 0x7f427c0b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:40.528 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.523+0000 7f42835c9640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f427c0b4aa0 con 0x7f427c15e4a0 2023-11-06T02:12:40.529 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.523+0000 7f42835c9640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f427c0b3e20 con 0x7f427c0b16e0 2023-11-06T02:12:40.529 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.523+0000 7f42835c9640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f427c0b4100 con 0x7f427c15e0c0 2023-11-06T02:12:40.529 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.523+0000 7f4281b66640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f427c15e4a0 0x7f427c0b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:40.529 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.523+0000 7f4281b66640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f427c15e4a0 0x7f427c0b4ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58580/0 (socket says 172.21.15.49:58580) 2023-11-06T02:12:40.529 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.523+0000 7f4281b66640 1 -- 172.21.15.49:0/1814667216 learned_addr learned my addr 172.21.15.49:0/1814667216 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:12:40.530 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.527+0000 7f4281365640 1 --2- 172.21.15.49:0/1814667216 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f427c0b16e0 0x7f427c155e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:40.530 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.527+0000 7f4280b64640 1 --2- 172.21.15.49:0/1814667216 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f427c15e0c0 0x7f427c0b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:40.530 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.527+0000 7f4281b66640 1 -- 172.21.15.49:0/1814667216 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f427c15e0c0 msgr2=0x7f427c0b3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:40.530 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.527+0000 7f4281b66640 1 --2- 172.21.15.49:0/1814667216 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f427c15e0c0 0x7f427c0b3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:40.530 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.527+0000 7f4281b66640 1 -- 172.21.15.49:0/1814667216 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f427c0b16e0 msgr2=0x7f427c155e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:40.530 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.527+0000 7f4281b66640 1 --2- 172.21.15.49:0/1814667216 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f427c0b16e0 0x7f427c155e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:40.531 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.527+0000 7f4281b66640 1 -- 172.21.15.49:0/1814667216 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f426c009420 con 0x7f427c15e4a0 2023-11-06T02:12:40.531 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.527+0000 7f4280b64640 1 --2- 172.21.15.49:0/1814667216 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f427c15e0c0 0x7f427c0b3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:12:40.531 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.527+0000 7f4281365640 1 --2- 172.21.15.49:0/1814667216 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f427c0b16e0 0x7f427c155e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:12:40.531 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.527+0000 7f4281b66640 1 --2- 172.21.15.49:0/1814667216 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f427c15e4a0 0x7f427c0b4ec0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f427400c780 tx=0x7f427400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:40.531 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.527+0000 7f4271ffb640 1 -- 172.21.15.49:0/1814667216 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f427400ce90 con 0x7f427c15e4a0 2023-11-06T02:12:40.532 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.527+0000 7f4271ffb640 1 -- 172.21.15.49:0/1814667216 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f427400a640 con 0x7f427c15e4a0 2023-11-06T02:12:40.532 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.527+0000 7f42835c9640 1 -- 172.21.15.49:0/1814667216 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f427c0b43e0 con 0x7f427c15e4a0 2023-11-06T02:12:40.532 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.527+0000 7f4271ffb640 1 -- 172.21.15.49:0/1814667216 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f427400a9d0 con 0x7f427c15e4a0 2023-11-06T02:12:40.532 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.527+0000 7f42835c9640 1 -- 172.21.15.49:0/1814667216 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f427c0b3aa0 con 0x7f427c15e4a0 2023-11-06T02:12:40.532 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.527+0000 7f4271ffb640 1 -- 172.21.15.49:0/1814667216 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f427400ab30 con 0x7f427c15e4a0 2023-11-06T02:12:40.533 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.527+0000 7f4271ffb640 1 --2- 172.21.15.49:0/1814667216 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4260065a60 0x7f4260067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:40.533 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.527+0000 7f4271ffb640 1 -- 172.21.15.49:0/1814667216 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f4274082b50 con 0x7f427c15e4a0 2023-11-06T02:12:40.533 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.527+0000 7f4281365640 1 --2- 172.21.15.49:0/1814667216 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4260065a60 0x7f4260067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:40.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.527+0000 7f42866f1000 1 Processor -- start 2023-11-06T02:12:40.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.527+0000 7f42866f1000 1 -- start start 2023-11-06T02:12:40.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.531+0000 7f4281365640 1 --2- 172.21.15.49:0/1814667216 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4260065a60 0x7f4260067f20 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f426c0ad000 tx=0x7f426c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:40.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.531+0000 7f42866f1000 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55693717f130 0x55693717f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:40.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.531+0000 7f42866f1000 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55693717fa50 0x556937134d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:40.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.531+0000 7f42866f1000 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556937135260 0x556937135640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:40.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.531+0000 7f42866f1000 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x556936ffcc10 con 0x556937135260 2023-11-06T02:12:40.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.531+0000 7f42866f1000 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x556936f8b3e0 con 0x55693717fa50 2023-11-06T02:12:40.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.531+0000 7f42866f1000 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x556936f889b0 con 0x55693717f130 2023-11-06T02:12:40.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.531+0000 7f4281b66640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556937135260 0x556937135640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:40.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.531+0000 7f4281b66640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556937135260 0x556937135640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58586/0 (socket says 172.21.15.49:58586) 2023-11-06T02:12:40.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.531+0000 7f4281b66640 1 -- 172.21.15.49:0/4053730574 learned_addr learned my addr 172.21.15.49:0/4053730574 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:12:40.542 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.531+0000 7f4281365640 1 --2- 172.21.15.49:0/4053730574 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55693717f130 0x55693717f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:40.542 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.531+0000 7f4280b64640 1 --2- 172.21.15.49:0/4053730574 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55693717fa50 0x556937134d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:40.542 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.531+0000 7f4281b66640 1 -- 172.21.15.49:0/4053730574 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55693717f130 msgr2=0x55693717f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:40.542 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.531+0000 7f4281b66640 1 --2- 172.21.15.49:0/4053730574 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55693717f130 0x55693717f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:40.542 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.531+0000 7f4281b66640 1 -- 172.21.15.49:0/4053730574 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55693717fa50 msgr2=0x556937134d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:40.543 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.531+0000 7f4281b66640 1 --2- 172.21.15.49:0/4053730574 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55693717fa50 0x556937134d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:40.543 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.531+0000 7f4281b66640 1 -- 172.21.15.49:0/4053730574 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556936c80e20 con 0x556937135260 2023-11-06T02:12:40.543 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.531+0000 7f4280b64640 1 --2- 172.21.15.49:0/4053730574 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55693717fa50 0x556937134d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:12:40.543 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.531+0000 7f4281b66640 1 --2- 172.21.15.49:0/4053730574 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556937135260 0x556937135640 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f4274080e40 tx=0x7f4274053c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c6d27f2d300ad1e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:40.543 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.531+0000 7f4266ffd640 1 -- 172.21.15.49:0/4053730574 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f427405bfd0 con 0x556937135260 2023-11-06T02:12:40.543 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.531+0000 7f4266ffd640 1 -- 172.21.15.49:0/4053730574 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4274052130 con 0x556937135260 2023-11-06T02:12:40.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.531+0000 7f4266ffd640 1 -- 172.21.15.49:0/4053730574 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4274050700 con 0x556937135260 2023-11-06T02:12:40.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.531+0000 7f42866f1000 1 -- 172.21.15.49:0/4053730574 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556937135260 msgr2=0x556937135640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:40.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.531+0000 7f42866f1000 1 --2- 172.21.15.49:0/4053730574 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556937135260 0x556937135640 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f4274080e40 tx=0x7f4274053c60 comp rx=0 tx=0).stop 2023-11-06T02:12:40.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.531+0000 7f42866f1000 1 -- 172.21.15.49:0/4053730574 shutdown_connections 2023-11-06T02:12:40.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.531+0000 7f42866f1000 1 --2- 172.21.15.49:0/4053730574 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556937135260 0x556937135640 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:40.545 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.531+0000 7f42866f1000 1 --2- 172.21.15.49:0/4053730574 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55693717fa50 0x556937134d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:40.545 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.531+0000 7f42866f1000 1 --2- 172.21.15.49:0/4053730574 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55693717f130 0x55693717f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:40.545 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.531+0000 7f42866f1000 1 -- 172.21.15.49:0/4053730574 >> 172.21.15.49:0/4053730574 conn(0x55693713b9f0 msgr2=0x556937133600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:12:40.545 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.531+0000 7f42866f1000 1 -- 172.21.15.49:0/4053730574 shutdown_connections 2023-11-06T02:12:40.545 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.531+0000 7f42866f1000 1 -- 172.21.15.49:0/4053730574 wait complete. 2023-11-06T02:12:40.546 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.531+0000 7f42866f1000 1 -- start start 2023-11-06T02:12:40.546 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.531+0000 7f42866f1000 1 Processor -- start 2023-11-06T02:12:40.546 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.531+0000 7f42866f1000 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556937135260 0x556937186570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:40.546 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.531+0000 7f42866f1000 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55693717f130 0x556937186ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:40.546 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.531+0000 7f42866f1000 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55693717fa50 0x556937180040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:40.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.531+0000 7f42866f1000 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x556936f842a0 con 0x556937135260 2023-11-06T02:12:40.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.531+0000 7f42866f1000 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x556936f23470 con 0x55693717fa50 2023-11-06T02:12:40.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.531+0000 7f42866f1000 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x556936f41660 con 0x55693717f130 2023-11-06T02:12:40.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.531+0000 7f42866f1000 10 client.0 _ll_register_callbacks cb 0x556937120c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T02:12:40.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.531+0000 7f4281365640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556937135260 0x556937186570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:40.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.531+0000 7f4281365640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556937135260 0x556937186570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58594/0 (socket says 172.21.15.49:58594) 2023-11-06T02:12:40.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.531+0000 7f4281365640 1 -- 172.21.15.49:0/3085301993 learned_addr learned my addr 172.21.15.49:0/3085301993 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:12:40.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.535+0000 7f4280b64640 1 --2- 172.21.15.49:0/3085301993 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55693717f130 0x556937186ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:40.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.535+0000 7f4281b66640 1 --2- 172.21.15.49:0/3085301993 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55693717fa50 0x556937180040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:40.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.535+0000 7f4281365640 1 -- 172.21.15.49:0/3085301993 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55693717f130 msgr2=0x556937186ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:40.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.535+0000 7f4281365640 1 --2- 172.21.15.49:0/3085301993 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55693717f130 0x556937186ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:40.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.535+0000 7f4281365640 1 -- 172.21.15.49:0/3085301993 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55693717fa50 msgr2=0x556937180040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:40.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.535+0000 7f4281365640 1 --2- 172.21.15.49:0/3085301993 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55693717fa50 0x556937180040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:40.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.535+0000 7f4281365640 1 -- 172.21.15.49:0/3085301993 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f427c0b3aa0 con 0x556937135260 2023-11-06T02:12:40.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.535+0000 7f4281365640 1 --2- 172.21.15.49:0/3085301993 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556937135260 0x556937186570 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f426c0ae040 tx=0x7f426c01ac10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a87fc57e1743f8 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:40.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.535+0000 7f425bfff640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-11-06T02:12:40.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.535+0000 7f425bfff640 1 -- 172.21.15.49:0/3085301993 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426c0b5850 con 0x556937135260 2023-11-06T02:12:40.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.535+0000 7f425bfff640 1 -- 172.21.15.49:0/3085301993 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(67..67 src has 1..67) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f426c0b59b0 con 0x556937135260 2023-11-06T02:12:40.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.535+0000 7f42866f1000 1 -- 172.21.15.49:0/3085301993 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556936f410a0 con 0x556937135260 2023-11-06T02:12:40.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.535+0000 7f425bfff640 1 -- 172.21.15.49:0/3085301993 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f426c0b5b70 con 0x556937135260 2023-11-06T02:12:40.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.535+0000 7f42866f1000 10 client.6777 fetch_fsmap learned FSMap version 30 2023-11-06T02:12:40.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.535+0000 7f42866f1000 1 -- 172.21.15.49:0/3085301993 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=2+}) v3 -- 0x556936f491d0 con 0x556937135260 2023-11-06T02:12:40.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.535+0000 7f425bfff640 1 -- 172.21.15.49:0/3085301993 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556936f410a0 con 0x556937135260 2023-11-06T02:12:40.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.535+0000 7f425bfff640 1 -- 172.21.15.49:0/3085301993 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 30) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f426c0b5cd0 con 0x556937135260 2023-11-06T02:12:40.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.535+0000 7f42866f1000 10 client.6777 fetch_fsmap finished waiting for FSMap version 30 2023-11-06T02:12:40.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.535+0000 7f42866f1000 10 client.6777 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2023-11-06T02:12:40.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.535+0000 7f42866f1000 20 client.6777 populate_metadata read hostname 'smithi049' 2023-11-06T02:12:40.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.535+0000 7f42866f1000 1 --2- 172.21.15.49:0/3085301993 >> [v2:172.21.15.49:6834/342908487,v1:172.21.15.49:6835/342908487] conn(0x556937180db0 0x5569371811c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:40.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.535+0000 7f42866f1000 4 client.6777 mds_command: new command op to 6348 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T02:12:40.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.535+0000 7f42866f1000 1 -- 172.21.15.49:0/3085301993 --> [v2:172.21.15.49:6834/342908487,v1:172.21.15.49:6835/342908487] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556936f82bf0 con 0x556937180db0 2023-11-06T02:12:40.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.535+0000 7f4280b64640 1 --2- 172.21.15.49:0/3085301993 >> [v2:172.21.15.49:6834/342908487,v1:172.21.15.49:6835/342908487] conn(0x556937180db0 0x5569371811c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:40.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.535+0000 7f4280b64640 1 --2- 172.21.15.49:0/3085301993 >> [v2:172.21.15.49:6834/342908487,v1:172.21.15.49:6835/342908487] conn(0x556937180db0 0x5569371811c0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d1cd877ad1c5a34 server_cookie=6a5884028925258d in_seq=0 out_seq=0 2023-11-06T02:12:40.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.535+0000 7f425bfff640 10 client.6777 ms_handle_connect on v2:172.21.15.49:6834/342908487 2023-11-06T02:12:40.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.535+0000 7f425bfff640 1 -- 172.21.15.49:0/3085301993 <== mds.0 v2:172.21.15.49:6834/342908487 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x556936f82bf0 con 0x556937180db0 2023-11-06T02:12:40.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.535+0000 7f425bfff640 10 client.6777 handle_command_reply: tid=0 2023-11-06T02:12:40.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.535+0000 7f42866f1000 1 client.6777 shutdown 2023-11-06T02:12:40.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.535+0000 7f42866f1000 1 -- 172.21.15.49:0/3085301993 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556937135260 msgr2=0x556937186570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:40.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.535+0000 7f42866f1000 1 --2- 172.21.15.49:0/3085301993 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556937135260 0x556937186570 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f426c0ae040 tx=0x7f426c01ac10 comp rx=0 tx=0).stop 2023-11-06T02:12:40.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.535+0000 7f42866f1000 1 -- 172.21.15.49:0/3085301993 shutdown_connections 2023-11-06T02:12:40.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.535+0000 7f42866f1000 1 --2- 172.21.15.49:0/3085301993 >> [v2:172.21.15.49:6834/342908487,v1:172.21.15.49:6835/342908487] conn(0x556937180db0 0x5569371811c0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:40.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.535+0000 7f42866f1000 1 --2- 172.21.15.49:0/3085301993 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55693717fa50 0x556937180040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:40.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.535+0000 7f42866f1000 1 --2- 172.21.15.49:0/3085301993 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55693717f130 0x556937186ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:40.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.535+0000 7f425bfff640 0 client.6777 ms_handle_reset on v2:172.21.15.49:6834/342908487 2023-11-06T02:12:40.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.535+0000 7f42866f1000 1 --2- 172.21.15.49:0/3085301993 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556937135260 0x556937186570 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:40.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.535+0000 7f42866f1000 1 -- 172.21.15.49:0/3085301993 >> 172.21.15.49:0/3085301993 conn(0x55693713b9f0 msgr2=0x556937133d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:12:40.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.535+0000 7f42866f1000 1 -- 172.21.15.49:0/3085301993 shutdown_connections 2023-11-06T02:12:40.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.535+0000 7f42866f1000 1 -- 172.21.15.49:0/3085301993 wait complete. 2023-11-06T02:12:40.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.535+0000 7f42866f1000 20 client.6777 trim_cache size 0 max 16384 2023-11-06T02:12:40.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.551+0000 7f42866f1000 1 Processor -- start 2023-11-06T02:12:40.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.551+0000 7f42866f1000 1 -- start start 2023-11-06T02:12:40.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.551+0000 7f42866f1000 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556937135260 0x55693713a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:40.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.551+0000 7f42866f1000 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55693717f130 0x55693713adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:40.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.551+0000 7f42866f1000 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55693717fa50 0x5569371841e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:40.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.551+0000 7f42866f1000 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x556936c41aa0 con 0x556937135260 2023-11-06T02:12:40.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.551+0000 7f42866f1000 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x556936f81ab0 con 0x55693717fa50 2023-11-06T02:12:40.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.551+0000 7f42866f1000 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x556936f297a0 con 0x55693717f130 2023-11-06T02:12:40.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.551+0000 7f4281365640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556937135260 0x55693713a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:40.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.551+0000 7f4281365640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556937135260 0x55693713a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58608/0 (socket says 172.21.15.49:58608) 2023-11-06T02:12:40.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.551+0000 7f4281365640 1 -- 172.21.15.49:0/3572337325 learned_addr learned my addr 172.21.15.49:0/3572337325 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:12:40.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.551+0000 7f4281365640 1 -- 172.21.15.49:0/3572337325 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55693717f130 msgr2=0x55693713adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:40.563 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.551+0000 7f4281365640 1 --2- 172.21.15.49:0/3572337325 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55693717f130 0x55693713adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:40.563 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.551+0000 7f4281365640 1 -- 172.21.15.49:0/3572337325 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55693717fa50 msgr2=0x5569371841e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:40.563 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.551+0000 7f4281365640 1 --2- 172.21.15.49:0/3572337325 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55693717fa50 0x5569371841e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:40.563 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.551+0000 7f4281365640 1 -- 172.21.15.49:0/3572337325 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556936f2f520 con 0x556937135260 2023-11-06T02:12:40.563 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.551+0000 7f4281365640 1 --2- 172.21.15.49:0/3572337325 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556937135260 0x55693713a880 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f426c01c410 tx=0x7f426c0be000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7ab6f0ed16590ed server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:40.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.551+0000 7f42677fe640 1 -- 172.21.15.49:0/3572337325 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426c0c19a0 con 0x556937135260 2023-11-06T02:12:40.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.551+0000 7f42677fe640 1 -- 172.21.15.49:0/3572337325 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f426c0c1b00 con 0x556937135260 2023-11-06T02:12:40.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.551+0000 7f42677fe640 1 -- 172.21.15.49:0/3572337325 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426c0b5850 con 0x556937135260 2023-11-06T02:12:40.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.551+0000 7f42866f1000 1 -- 172.21.15.49:0/3572337325 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556937135260 msgr2=0x55693713a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:40.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.551+0000 7f42866f1000 1 --2- 172.21.15.49:0/3572337325 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556937135260 0x55693713a880 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f426c01c410 tx=0x7f426c0be000 comp rx=0 tx=0).stop 2023-11-06T02:12:40.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.551+0000 7f42866f1000 1 -- 172.21.15.49:0/3572337325 shutdown_connections 2023-11-06T02:12:40.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.551+0000 7f42866f1000 1 --2- 172.21.15.49:0/3572337325 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55693717fa50 0x5569371841e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:40.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.551+0000 7f42866f1000 1 --2- 172.21.15.49:0/3572337325 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55693717f130 0x55693713adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:40.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.551+0000 7f42866f1000 1 --2- 172.21.15.49:0/3572337325 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556937135260 0x55693713a880 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:40.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.551+0000 7f42866f1000 1 -- 172.21.15.49:0/3572337325 >> 172.21.15.49:0/3572337325 conn(0x55693713b9f0 msgr2=0x55693713a170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:12:40.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.551+0000 7f42866f1000 1 -- 172.21.15.49:0/3572337325 shutdown_connections 2023-11-06T02:12:40.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.551+0000 7f42866f1000 1 -- 172.21.15.49:0/3572337325 wait complete. 2023-11-06T02:12:40.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.551+0000 7f42866f1000 1 -- start start 2023-11-06T02:12:40.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.551+0000 7f42866f1000 1 Processor -- start 2023-11-06T02:12:40.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.555+0000 7f42866f1000 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556937135260 0x5569371d9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:40.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.555+0000 7f42866f1000 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55693717f130 0x5569371d95d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:40.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.555+0000 7f42866f1000 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55693717fa50 0x5569371d9b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:40.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.555+0000 7f42866f1000 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x556937180290 con 0x556937135260 2023-11-06T02:12:40.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.555+0000 7f42866f1000 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x556937138320 con 0x55693717f130 2023-11-06T02:12:40.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.555+0000 7f42866f1000 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x556937195930 con 0x55693717fa50 2023-11-06T02:12:40.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.555+0000 7f42866f1000 10 client.0 _ll_register_callbacks cb 0x556936f48cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T02:12:40.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.555+0000 7f4281365640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556937135260 0x5569371d9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:40.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.555+0000 7f4281365640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556937135260 0x5569371d9090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58620/0 (socket says 172.21.15.49:58620) 2023-11-06T02:12:40.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.555+0000 7f4281365640 1 -- 172.21.15.49:0/384726348 learned_addr learned my addr 172.21.15.49:0/384726348 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:12:40.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.555+0000 7f4280b64640 1 --2- 172.21.15.49:0/384726348 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55693717f130 0x5569371d95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:40.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.555+0000 7f4281365640 1 -- 172.21.15.49:0/384726348 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55693717fa50 msgr2=0x5569371d9b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:40.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.555+0000 7f4281365640 1 --2- 172.21.15.49:0/384726348 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55693717fa50 0x5569371d9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:40.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.555+0000 7f4281365640 1 -- 172.21.15.49:0/384726348 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55693717f130 msgr2=0x5569371d95d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:40.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.555+0000 7f4281365640 1 --2- 172.21.15.49:0/384726348 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55693717f130 0x5569371d95d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:40.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.555+0000 7f4281365640 1 -- 172.21.15.49:0/384726348 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556936f491d0 con 0x556937135260 2023-11-06T02:12:40.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.555+0000 7f4281365640 1 --2- 172.21.15.49:0/384726348 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556937135260 0x5569371d9090 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f426c097d90 tx=0x7f426c0a8000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=112e4b1c27d15d5e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:40.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.555+0000 7f425bfff640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-11-06T02:12:40.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.555+0000 7f425bfff640 1 -- 172.21.15.49:0/384726348 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426c0b5cf0 con 0x556937135260 2023-11-06T02:12:40.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.555+0000 7f425bfff640 1 -- 172.21.15.49:0/384726348 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(67..67 src has 1..67) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f426c0decc0 con 0x556937135260 2023-11-06T02:12:40.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.555+0000 7f42866f1000 1 -- 172.21.15.49:0/384726348 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556936f4d620 con 0x556937135260 2023-11-06T02:12:40.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.555+0000 7f425bfff640 1 -- 172.21.15.49:0/384726348 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f426c0b5e50 con 0x556937135260 2023-11-06T02:12:40.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.555+0000 7f42866f1000 10 client.6783 fetch_fsmap learned FSMap version 30 2023-11-06T02:12:40.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.555+0000 7f42866f1000 1 -- 172.21.15.49:0/384726348 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=2+}) v3 -- 0x556936f23090 con 0x556937135260 2023-11-06T02:12:40.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.555+0000 7f425bfff640 1 -- 172.21.15.49:0/384726348 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556936f4d620 con 0x556937135260 2023-11-06T02:12:40.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.555+0000 7f425bfff640 1 -- 172.21.15.49:0/384726348 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 30) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f426c0c1d20 con 0x556937135260 2023-11-06T02:12:40.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.555+0000 7f42866f1000 10 client.6783 fetch_fsmap finished waiting for FSMap version 30 2023-11-06T02:12:40.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.555+0000 7f42866f1000 10 client.6783 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2023-11-06T02:12:40.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.555+0000 7f42866f1000 20 client.6783 populate_metadata read hostname 'smithi049' 2023-11-06T02:12:40.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.555+0000 7f42866f1000 1 --2- 172.21.15.49:0/384726348 >> [v2:172.21.15.49:6834/342908487,v1:172.21.15.49:6835/342908487] conn(0x5569371da050 0x5569371e4840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:40.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.555+0000 7f42866f1000 4 client.6783 mds_command: new command op to 6348 tid=0[{"prefix": "scrub status"}] 2023-11-06T02:12:40.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.555+0000 7f42866f1000 1 -- 172.21.15.49:0/384726348 --> [v2:172.21.15.49:6834/342908487,v1:172.21.15.49:6835/342908487] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556936f7d320 con 0x5569371da050 2023-11-06T02:12:40.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.555+0000 7f4280b64640 1 --2- 172.21.15.49:0/384726348 >> [v2:172.21.15.49:6834/342908487,v1:172.21.15.49:6835/342908487] conn(0x5569371da050 0x5569371e4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:40.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.555+0000 7f425bfff640 10 client.6783 ms_handle_connect on v2:172.21.15.49:6834/342908487 2023-11-06T02:12:40.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.555+0000 7f4280b64640 1 --2- 172.21.15.49:0/384726348 >> [v2:172.21.15.49:6834/342908487,v1:172.21.15.49:6835/342908487] conn(0x5569371da050 0x5569371e4840 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3075d05c4be5eb0 server_cookie=ada94163cf65d23b in_seq=0 out_seq=0 2023-11-06T02:12:40.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.555+0000 7f425bfff640 1 -- 172.21.15.49:0/384726348 <== mds.0 v2:172.21.15.49:6834/342908487 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x556936f7d320 con 0x5569371da050 2023-11-06T02:12:40.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.555+0000 7f425bfff640 10 client.6783 handle_command_reply: tid=0 2023-11-06T02:12:40.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.555+0000 7f42866f1000 1 client.6783 shutdown 2023-11-06T02:12:40.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.555+0000 7f42866f1000 1 -- 172.21.15.49:0/384726348 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556937135260 msgr2=0x5569371d9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:40.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.555+0000 7f42866f1000 1 --2- 172.21.15.49:0/384726348 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556937135260 0x5569371d9090 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f426c097d90 tx=0x7f426c0a8000 comp rx=0 tx=0).stop 2023-11-06T02:12:40.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.555+0000 7f42866f1000 1 -- 172.21.15.49:0/384726348 shutdown_connections 2023-11-06T02:12:40.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.555+0000 7f42866f1000 1 --2- 172.21.15.49:0/384726348 >> [v2:172.21.15.49:6834/342908487,v1:172.21.15.49:6835/342908487] conn(0x5569371da050 0x5569371e4840 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:40.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.555+0000 7f42866f1000 1 --2- 172.21.15.49:0/384726348 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55693717fa50 0x5569371d9b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:40.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.555+0000 7f42866f1000 1 --2- 172.21.15.49:0/384726348 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55693717f130 0x5569371d95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:40.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.555+0000 7f425bfff640 0 client.6783 ms_handle_reset on v2:172.21.15.49:6834/342908487 2023-11-06T02:12:40.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.555+0000 7f42866f1000 1 --2- 172.21.15.49:0/384726348 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556937135260 0x5569371d9090 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:40.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.555+0000 7f42866f1000 1 -- 172.21.15.49:0/384726348 >> 172.21.15.49:0/384726348 conn(0x55693713b9f0 msgr2=0x55693713a170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:12:40.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.555+0000 7f42866f1000 1 -- 172.21.15.49:0/384726348 shutdown_connections 2023-11-06T02:12:40.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.555+0000 7f42866f1000 1 -- 172.21.15.49:0/384726348 wait complete. 2023-11-06T02:12:40.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.555+0000 7f42866f1000 20 client.6783 trim_cache size 0 max 16384 2023-11-06T02:12:40.576 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-11-06T02:12:40.576 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "no active scrubs running", 2023-11-06T02:12:40.577 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": {} 2023-11-06T02:12:40.577 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-11-06T02:12:40.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.555+0000 7f42835c9640 1 -- 172.21.15.49:0/1814667216 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4260065a60 msgr2=0x7f4260067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:40.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.555+0000 7f42835c9640 1 --2- 172.21.15.49:0/1814667216 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4260065a60 0x7f4260067f20 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f426c0ad000 tx=0x7f426c008040 comp rx=0 tx=0).stop 2023-11-06T02:12:40.579 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.555+0000 7f42835c9640 1 -- 172.21.15.49:0/1814667216 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f427c15e4a0 msgr2=0x7f427c0b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:40.579 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.555+0000 7f42835c9640 1 --2- 172.21.15.49:0/1814667216 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f427c15e4a0 0x7f427c0b4ec0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f427400c780 tx=0x7f427400cc40 comp rx=0 tx=0).stop 2023-11-06T02:12:40.579 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.555+0000 7f42835c9640 1 -- 172.21.15.49:0/1814667216 shutdown_connections 2023-11-06T02:12:40.579 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.555+0000 7f42835c9640 1 --2- 172.21.15.49:0/1814667216 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4260065a60 0x7f4260067f20 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:40.579 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.555+0000 7f42835c9640 1 --2- 172.21.15.49:0/1814667216 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f427c15e4a0 0x7f427c0b4ec0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:40.580 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.555+0000 7f42835c9640 1 --2- 172.21.15.49:0/1814667216 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f427c15e0c0 0x7f427c0b3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:40.580 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.555+0000 7f42835c9640 1 --2- 172.21.15.49:0/1814667216 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f427c0b16e0 0x7f427c155e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:40.580 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.555+0000 7f42835c9640 1 -- 172.21.15.49:0/1814667216 >> 172.21.15.49:0/1814667216 conn(0x7f427c099c50 msgr2=0x7f427c159780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:12:40.580 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.555+0000 7f42835c9640 1 -- 172.21.15.49:0/1814667216 shutdown_connections 2023-11-06T02:12:40.580 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.555+0000 7f42835c9640 1 -- 172.21.15.49:0/1814667216 wait complete. 2023-11-06T02:12:40.581 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-06T02:12:40.682 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-06T02:12:40.818 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.819+0000 7f35aa3c0640 1 Processor -- start 2023-11-06T02:12:40.819 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.819+0000 7f35aa3c0640 1 -- start start 2023-11-06T02:12:40.820 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.819+0000 7f35aa3c0640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f35a415ab50 0x7f35a40c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:40.820 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.819+0000 7f35aa3c0640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f35a40b0790 0x7f35a40c2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:40.820 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.819+0000 7f35aa3c0640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35a40cab80 con 0x7f35a40cb010 2023-11-06T02:12:40.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.819+0000 7f35aa3c0640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35a415af30 con 0x7f35a415ab50 2023-11-06T02:12:40.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.819+0000 7f35aa3c0640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35a40cb5b0 con 0x7f35a40b0790 2023-11-06T02:12:40.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.819+0000 7f35a3fff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f35a415ab50 0x7f35a40c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:40.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.819+0000 7f35a3fff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f35a415ab50 0x7f35a40c9bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58632/0 (socket says 172.21.15.49:58632) 2023-11-06T02:12:40.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.819+0000 7f35a3fff640 1 -- 172.21.15.49:0/3271594647 learned_addr learned my addr 172.21.15.49:0/3271594647 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:12:40.822 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.819+0000 7f35a895d640 1 --2- 172.21.15.49:0/3271594647 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f35a40b0790 0x7f35a40c2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:40.822 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.819+0000 7f35a3fff640 1 -- 172.21.15.49:0/3271594647 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f35a40b0790 msgr2=0x7f35a40c2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:40.822 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.819+0000 7f35a3fff640 1 --2- 172.21.15.49:0/3271594647 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f35a40b0790 0x7f35a40c2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:40.822 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.819+0000 7f35a3fff640 1 -- 172.21.15.49:0/3271594647 >> v1:172.21.15.150:6789/0 conn(0x7f35a40cb010 legacy=0x7f35a40b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:40.823 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.819+0000 7f35a3fff640 1 -- 172.21.15.49:0/3271594647 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35a40cb890 con 0x7f35a415ab50 2023-11-06T02:12:40.824 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.823+0000 7f35a3fff640 1 --2- 172.21.15.49:0/3271594647 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35a415ab50 0x7f35a40c9bb0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f35a40725a0 tx=0x7f359801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=887bbcd4456768d2 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:40.824 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.823+0000 7f35a2ffd640 1 -- 172.21.15.49:0/3271594647 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35980ab650 con 0x7f35a415ab50 2023-11-06T02:12:40.824 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.823+0000 7f35a2ffd640 1 -- 172.21.15.49:0/3271594647 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35980ab7b0 con 0x7f35a415ab50 2023-11-06T02:12:40.825 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.823+0000 7f35a2ffd640 1 -- 172.21.15.49:0/3271594647 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35980abb40 con 0x7f35a415ab50 2023-11-06T02:12:40.825 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.823+0000 7f35aa3c0640 1 -- 172.21.15.49:0/3271594647 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35a415ab50 msgr2=0x7f35a40c9bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:40.825 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.823+0000 7f35aa3c0640 1 --2- 172.21.15.49:0/3271594647 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35a415ab50 0x7f35a40c9bb0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f35a40725a0 tx=0x7f359801c490 comp rx=0 tx=0).stop 2023-11-06T02:12:40.825 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.823+0000 7f35aa3c0640 1 -- 172.21.15.49:0/3271594647 shutdown_connections 2023-11-06T02:12:40.825 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.823+0000 7f35aa3c0640 1 --2- 172.21.15.49:0/3271594647 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f35a40b0790 0x7f35a40c2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:40.825 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.823+0000 7f35aa3c0640 1 --2- 172.21.15.49:0/3271594647 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35a415ab50 0x7f35a40c9bb0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:40.826 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.823+0000 7f35aa3c0640 1 -- 172.21.15.49:0/3271594647 >> 172.21.15.49:0/3271594647 conn(0x7f35a406c0a0 msgr2=0x7f35a406c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:12:40.826 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.823+0000 7f35aa3c0640 1 -- 172.21.15.49:0/3271594647 shutdown_connections 2023-11-06T02:12:40.826 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.823+0000 7f35aa3c0640 1 -- 172.21.15.49:0/3271594647 wait complete. 2023-11-06T02:12:40.826 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.823+0000 7f35aa3c0640 1 Processor -- start 2023-11-06T02:12:40.826 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.823+0000 7f35aa3c0640 1 -- start start 2023-11-06T02:12:40.829 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.827+0000 7f35aa3c0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35a40b0790 0x7f35a41520a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:40.829 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.827+0000 7f35aa3c0640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f35a40cb010 0x7f35a41f4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:40.829 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.827+0000 7f35aa3c0640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f35a415ab50 0x7f35a40a82d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:40.829 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.827+0000 7f35aa3c0640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35a4152a60 con 0x7f35a40b0790 2023-11-06T02:12:40.829 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.827+0000 7f35aa3c0640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35a40c69a0 con 0x7f35a415ab50 2023-11-06T02:12:40.830 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.827+0000 7f35aa3c0640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35a40c6bf0 con 0x7f35a40cb010 2023-11-06T02:12:40.830 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.827+0000 7f35a3fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35a40b0790 0x7f35a41520a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:40.830 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.827+0000 7f35a3fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35a40b0790 0x7f35a41520a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58648/0 (socket says 172.21.15.49:58648) 2023-11-06T02:12:40.830 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.827+0000 7f35a3fff640 1 -- 172.21.15.49:0/1475232324 learned_addr learned my addr 172.21.15.49:0/1475232324 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:12:40.830 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.827+0000 7f35a37fe640 1 --2- 172.21.15.49:0/1475232324 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f35a40cb010 0x7f35a41f4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:40.831 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.827+0000 7f35a895d640 1 --2- 172.21.15.49:0/1475232324 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f35a415ab50 0x7f35a40a82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:40.831 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.827+0000 7f35a3fff640 1 -- 172.21.15.49:0/1475232324 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f35a40cb010 msgr2=0x7f35a41f4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:40.831 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.827+0000 7f35a3fff640 1 --2- 172.21.15.49:0/1475232324 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f35a40cb010 0x7f35a41f4740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:40.831 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.827+0000 7f35a3fff640 1 -- 172.21.15.49:0/1475232324 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f35a415ab50 msgr2=0x7f35a40a82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:40.831 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.827+0000 7f35a3fff640 1 --2- 172.21.15.49:0/1475232324 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f35a415ab50 0x7f35a40a82d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:40.832 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.827+0000 7f35a3fff640 1 -- 172.21.15.49:0/1475232324 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3598009480 con 0x7f35a40b0790 2023-11-06T02:12:40.832 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.827+0000 7f35a895d640 1 --2- 172.21.15.49:0/1475232324 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f35a415ab50 0x7f35a40a82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:12:40.832 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.827+0000 7f35a3fff640 1 --2- 172.21.15.49:0/1475232324 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35a40b0790 0x7f35a41520a0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f359801cd30 tx=0x7f3598019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:40.832 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.827+0000 7f35a0ff9640 1 -- 172.21.15.49:0/1475232324 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35980abd30 con 0x7f35a40b0790 2023-11-06T02:12:40.832 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.827+0000 7f35a0ff9640 1 -- 172.21.15.49:0/1475232324 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35980abe90 con 0x7f35a40b0790 2023-11-06T02:12:40.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.827+0000 7f35aa3c0640 1 -- 172.21.15.49:0/1475232324 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35a40c6ed0 con 0x7f35a40b0790 2023-11-06T02:12:40.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.827+0000 7f35a0ff9640 1 -- 172.21.15.49:0/1475232324 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3598005bc0 con 0x7f35a40b0790 2023-11-06T02:12:40.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.827+0000 7f35aa3c0640 1 -- 172.21.15.49:0/1475232324 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35a40c7490 con 0x7f35a40b0790 2023-11-06T02:12:40.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.827+0000 7f35aa3c0640 1 -- 172.21.15.49:0/1475232324 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35a40b0de0 con 0x7f35a40b0790 2023-11-06T02:12:40.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.827+0000 7f35a0ff9640 1 -- 172.21.15.49:0/1475232324 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f3598003620 con 0x7f35a40b0790 2023-11-06T02:12:40.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.827+0000 7f35a0ff9640 1 --2- 172.21.15.49:0/1475232324 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3590065a60 0x7f3590067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:40.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.827+0000 7f35a0ff9640 1 -- 172.21.15.49:0/1475232324 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f359811d480 con 0x7f35a40b0790 2023-11-06T02:12:40.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.827+0000 7f35a37fe640 1 --2- 172.21.15.49:0/1475232324 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3590065a60 0x7f3590067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:40.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.831+0000 7f35a37fe640 1 --2- 172.21.15.49:0/1475232324 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3590065a60 0x7f3590067f20 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f35a40c43f0 tx=0x7f3594009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:40.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:40.831+0000 7f35a0ff9640 1 -- 172.21.15.49:0/1475232324 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f35980eac60 con 0x7f35a40b0790 2023-11-06T02:12:41.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.023+0000 7f35aa3c0640 1 -- 172.21.15.49:0/1475232324 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f35a4074040 con 0x7f35a40b0790 2023-11-06T02:12:41.025 INFO:tasks.ceph.mon.a.smithi049.stderr:2023-11-06T02:12:41.027+0000 7f0164e5a640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-06T02:12:41.043 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.043+0000 7f35a0ff9640 1 -- 172.21.15.49:0/1475232324 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v31) v1 ==== 161+0+0 (secure 0 0 0) 0x7f35980ab610 con 0x7f35a40b0790 2023-11-06T02:12:41.043 INFO:teuthology.orchestra.run.smithi049.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-06T02:12:41.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.047+0000 7f35aa3c0640 1 -- 172.21.15.49:0/1475232324 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3590065a60 msgr2=0x7f3590067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:41.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.047+0000 7f35aa3c0640 1 --2- 172.21.15.49:0/1475232324 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3590065a60 0x7f3590067f20 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f35a40c43f0 tx=0x7f3594009040 comp rx=0 tx=0).stop 2023-11-06T02:12:41.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.047+0000 7f35aa3c0640 1 -- 172.21.15.49:0/1475232324 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35a40b0790 msgr2=0x7f35a41520a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:41.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.047+0000 7f35aa3c0640 1 --2- 172.21.15.49:0/1475232324 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35a40b0790 0x7f35a41520a0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f359801cd30 tx=0x7f3598019230 comp rx=0 tx=0).stop 2023-11-06T02:12:41.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.047+0000 7f35aa3c0640 1 -- 172.21.15.49:0/1475232324 shutdown_connections 2023-11-06T02:12:41.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.047+0000 7f35aa3c0640 1 --2- 172.21.15.49:0/1475232324 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3590065a60 0x7f3590067f20 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:41.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.047+0000 7f35aa3c0640 1 --2- 172.21.15.49:0/1475232324 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f35a415ab50 0x7f35a40a82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:41.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.047+0000 7f35aa3c0640 1 --2- 172.21.15.49:0/1475232324 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f35a40cb010 0x7f35a41f4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:41.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.047+0000 7f35aa3c0640 1 --2- 172.21.15.49:0/1475232324 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35a40b0790 0x7f35a41520a0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:41.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.047+0000 7f35aa3c0640 1 -- 172.21.15.49:0/1475232324 >> 172.21.15.49:0/1475232324 conn(0x7f35a406c0a0 msgr2=0x7f35a4159d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:12:41.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.047+0000 7f35aa3c0640 1 -- 172.21.15.49:0/1475232324 shutdown_connections 2023-11-06T02:12:41.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.047+0000 7f35aa3c0640 1 -- 172.21.15.49:0/1475232324 wait complete. 2023-11-06T02:12:41.050 INFO:tasks.ceph.mds.c.smithi049.stderr: -6267> 2023-11-06T02:11:31.561+0000 7fe85a2b1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-06T02:12:41.063 DEBUG:teuthology.orchestra.run.smithi049:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 event splice --inode=1099511627777 summary 2023-11-06T02:12:41.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.103+0000 7fbb899ed280 1 -- start start 2023-11-06T02:12:41.105 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.103+0000 7fbb899ed280 1 Processor -- start 2023-11-06T02:12:41.107 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.107+0000 7fbb899ed280 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f82fc63000 0x55f82fcefb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:41.107 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.107+0000 7fbb899ed280 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f82fc63400 0x55f82fcef600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:41.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.107+0000 7fbb899ed280 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f82fc63800 0x55f82fcef080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:41.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.107+0000 7fbb899ed280 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f82fd30640 con 0x55f82fc63800 2023-11-06T02:12:41.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.107+0000 7fbb899ed280 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f82fd30500 con 0x55f82fc63000 2023-11-06T02:12:41.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.107+0000 7fbb899ed280 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f82fd303c0 con 0x55f82fc63400 2023-11-06T02:12:41.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.107+0000 7fbb889ae640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f82fc63800 0x55f82fcef080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:41.109 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.107+0000 7fbb889ae640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f82fc63800 0x55f82fcef080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58660/0 (socket says 172.21.15.49:58660) 2023-11-06T02:12:41.109 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.107+0000 7fbb889ae640 1 -- 172.21.15.49:0/2831070338 learned_addr learned my addr 172.21.15.49:0/2831070338 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:12:41.109 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.107+0000 7fbb879ac640 1 --2- 172.21.15.49:0/2831070338 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f82fc63400 0x55f82fcef600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:41.109 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.107+0000 7fbb881ad640 1 --2- 172.21.15.49:0/2831070338 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f82fc63000 0x55f82fcefb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:41.109 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.107+0000 7fbb889ae640 1 -- 172.21.15.49:0/2831070338 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f82fc63400 msgr2=0x55f82fcef600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:41.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.107+0000 7fbb889ae640 1 --2- 172.21.15.49:0/2831070338 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f82fc63400 0x55f82fcef600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:41.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.107+0000 7fbb889ae640 1 -- 172.21.15.49:0/2831070338 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f82fc63000 msgr2=0x55f82fcefb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:41.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.107+0000 7fbb889ae640 1 --2- 172.21.15.49:0/2831070338 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f82fc63000 0x55f82fcefb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:41.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.107+0000 7fbb889ae640 1 -- 172.21.15.49:0/2831070338 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f82fc98b60 con 0x55f82fc63800 2023-11-06T02:12:41.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.107+0000 7fbb889ae640 1 --2- 172.21.15.49:0/2831070338 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f82fc63800 0x55f82fcef080 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x55f830a487b0 tx=0x55f830a385a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47fb809401803737 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:41.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.107+0000 7fbb859a8640 1 -- 172.21.15.49:0/2831070338 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f82fcc7600 con 0x55f82fc63800 2023-11-06T02:12:41.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.107+0000 7fbb859a8640 1 -- 172.21.15.49:0/2831070338 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f830a4a780 con 0x55f82fc63800 2023-11-06T02:12:41.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.107+0000 7fbb859a8640 1 -- 172.21.15.49:0/2831070338 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f82fcc71e0 con 0x55f82fc63800 2023-11-06T02:12:41.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.107+0000 7fbb899ed280 1 -- 172.21.15.49:0/2831070338 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f82fc98340 con 0x55f82fc63800 2023-11-06T02:12:41.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.107+0000 7fbb899ed280 1 -- 172.21.15.49:0/2831070338 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55f82fc98000 con 0x55f82fc63800 2023-11-06T02:12:41.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.107+0000 7fbb899ed280 4 waiting for MDS map... 2023-11-06T02:12:41.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.107+0000 7fbb859a8640 1 -- 172.21.15.49:0/2831070338 <== mon.0 v2:172.21.15.49:3300/0 4 ==== osd_map(68..68 src has 1..68) v4 ==== 6264+0+0 (secure 0 0 0) 0x55f82fca2c40 con 0x55f82fc63800 2023-11-06T02:12:41.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.107+0000 7fbb859a8640 1 -- 172.21.15.49:0/2831070338 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 31) v1 ==== 2565+0+0 (secure 0 0 0) 0x55f82fcb0f00 con 0x55f82fc63800 2023-11-06T02:12:41.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.107+0000 7fbb899ed280 4 Got MDS map 31 2023-11-06T02:12:41.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.107+0000 7fbb899ed280 10 main: JournalTool::main 2023-11-06T02:12:41.113 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.107+0000 7fbb899ed280 4 main: JournalTool: connecting to RADOS... 2023-11-06T02:12:41.113 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.107+0000 7fbb899ed280 1 Processor -- start 2023-11-06T02:12:41.113 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.107+0000 7fbb899ed280 1 -- start start 2023-11-06T02:12:41.117 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.107+0000 7fbb899ed280 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f830be2400 0x55f82fceeb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:41.117 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.107+0000 7fbb899ed280 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f830be2800 0x55f82fcee580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:41.118 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.107+0000 7fbb899ed280 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f830be2c00 0x55f82fcf0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:41.118 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.107+0000 7fbb899ed280 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f82fd303c0 con 0x55f830be2c00 2023-11-06T02:12:41.118 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.107+0000 7fbb899ed280 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f82fd30500 con 0x55f830be2400 2023-11-06T02:12:41.119 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.107+0000 7fbb899ed280 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f82fd30a00 con 0x55f830be2800 2023-11-06T02:12:41.119 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.107+0000 7fbb879ac640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f830be2c00 0x55f82fcf0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:41.119 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.107+0000 7fbb879ac640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f830be2c00 0x55f82fcf0100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58672/0 (socket says 172.21.15.49:58672) 2023-11-06T02:12:41.119 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.107+0000 7fbb879ac640 1 -- 172.21.15.49:0/2223411272 learned_addr learned my addr 172.21.15.49:0/2223411272 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:12:41.119 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.107+0000 7fbb879ac640 1 --2- 172.21.15.49:0/2223411272 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f830be2400 0x55f82fceeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:41.120 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.107+0000 7fbb881ad640 1 --2- 172.21.15.49:0/2223411272 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f830be2800 0x55f82fcee580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:41.120 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.107+0000 7fbb879ac640 1 -- 172.21.15.49:0/2223411272 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f830be2800 msgr2=0x55f82fcee580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:41.120 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.107+0000 7fbb879ac640 1 --2- 172.21.15.49:0/2223411272 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f830be2800 0x55f82fcee580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:41.120 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.107+0000 7fbb879ac640 1 -- 172.21.15.49:0/2223411272 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f830be2400 msgr2=0x55f82fceeb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:41.120 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.107+0000 7fbb879ac640 1 --2- 172.21.15.49:0/2223411272 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f830be2400 0x55f82fceeb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:41.120 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.107+0000 7fbb879ac640 1 -- 172.21.15.49:0/2223411272 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f82fc98ea0 con 0x55f830be2c00 2023-11-06T02:12:41.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.111+0000 7fbb879ac640 1 --2- 172.21.15.49:0/2223411272 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f830be2c00 0x55f82fcf0100 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x55f830a48270 tx=0x55f830a383c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bdeb7d52674d976 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:41.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.111+0000 7fbb839a4640 1 -- 172.21.15.49:0/2223411272 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f82fcc7080 con 0x55f830be2c00 2023-11-06T02:12:41.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.111+0000 7fbb839a4640 1 -- 172.21.15.49:0/2223411272 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f830a4a480 con 0x55f830be2c00 2023-11-06T02:12:41.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.111+0000 7fbb839a4640 1 -- 172.21.15.49:0/2223411272 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f82fcc7600 con 0x55f830be2c00 2023-11-06T02:12:41.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.111+0000 7fbb899ed280 1 -- 172.21.15.49:0/2223411272 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f830be2c00 msgr2=0x55f82fcf0100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:41.122 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.111+0000 7fbb899ed280 1 --2- 172.21.15.49:0/2223411272 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f830be2c00 0x55f82fcf0100 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x55f830a48270 tx=0x55f830a383c0 comp rx=0 tx=0).stop 2023-11-06T02:12:41.122 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.111+0000 7fbb899ed280 1 -- 172.21.15.49:0/2223411272 shutdown_connections 2023-11-06T02:12:41.122 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.111+0000 7fbb899ed280 1 --2- 172.21.15.49:0/2223411272 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f830be2c00 0x55f82fcf0100 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:41.122 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.111+0000 7fbb899ed280 1 --2- 172.21.15.49:0/2223411272 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f830be2800 0x55f82fcee580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:41.122 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.111+0000 7fbb899ed280 1 --2- 172.21.15.49:0/2223411272 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f830be2400 0x55f82fceeb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:41.123 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.111+0000 7fbb899ed280 1 -- 172.21.15.49:0/2223411272 >> 172.21.15.49:0/2223411272 conn(0x55f830be2000 msgr2=0x55f82fc91800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:12:41.123 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.111+0000 7fbb899ed280 1 -- 172.21.15.49:0/2223411272 shutdown_connections 2023-11-06T02:12:41.123 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.111+0000 7fbb899ed280 1 -- 172.21.15.49:0/2223411272 wait complete. 2023-11-06T02:12:41.123 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.111+0000 7fbb899ed280 1 Processor -- start 2023-11-06T02:12:41.123 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.111+0000 7fbb899ed280 1 -- start start 2023-11-06T02:12:41.123 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.111+0000 7fbb899ed280 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f830be2400 0x55f82fceeb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:41.124 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.111+0000 7fbb899ed280 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f830be2800 0x55f82fcee580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:41.124 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.111+0000 7fbb899ed280 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f830be2c00 0x55f82fcf0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:41.124 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.111+0000 7fbb899ed280 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f82fd30280 con 0x55f830be2400 2023-11-06T02:12:41.124 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.111+0000 7fbb899ed280 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f82fd30140 con 0x55f830be2800 2023-11-06T02:12:41.124 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.111+0000 7fbb899ed280 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f82fd30780 con 0x55f830be2c00 2023-11-06T02:12:41.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.111+0000 7fbb879ac640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f830be2400 0x55f82fceeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:41.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.111+0000 7fbb879ac640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f830be2400 0x55f82fceeb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58686/0 (socket says 172.21.15.49:58686) 2023-11-06T02:12:41.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.111+0000 7fbb879ac640 1 -- 172.21.15.49:0/2089832632 learned_addr learned my addr 172.21.15.49:0/2089832632 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:12:41.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.111+0000 7fbb881ad640 1 --2- 172.21.15.49:0/2089832632 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f830be2800 0x55f82fcee580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:41.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.111+0000 7fbb879ac640 1 --2- 172.21.15.49:0/2089832632 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f830be2c00 0x55f82fcf0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:41.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.111+0000 7fbb879ac640 1 -- 172.21.15.49:0/2089832632 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f830be2c00 msgr2=0x55f82fcf0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:41.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.111+0000 7fbb879ac640 1 --2- 172.21.15.49:0/2089832632 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f830be2c00 0x55f82fcf0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:41.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.111+0000 7fbb879ac640 1 -- 172.21.15.49:0/2089832632 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f830be2800 msgr2=0x55f82fcee580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:41.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.111+0000 7fbb879ac640 1 --2- 172.21.15.49:0/2089832632 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f830be2800 0x55f82fcee580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:41.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.111+0000 7fbb879ac640 1 -- 172.21.15.49:0/2089832632 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f82fc991e0 con 0x55f830be2400 2023-11-06T02:12:41.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.111+0000 7fbb879ac640 1 --2- 172.21.15.49:0/2089832632 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f830be2400 0x55f82fceeb00 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x55f830a49c50 tx=0x55f830a384e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:41.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.111+0000 7fbb821a1640 1 -- 172.21.15.49:0/2089832632 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f82fcc71e0 con 0x55f830be2400 2023-11-06T02:12:41.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.111+0000 7fbb899ed280 1 -- 172.21.15.49:0/2089832632 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f82fc98ea0 con 0x55f830be2400 2023-11-06T02:12:41.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.111+0000 7fbb821a1640 1 -- 172.21.15.49:0/2089832632 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f830a4a600 con 0x55f830be2400 2023-11-06T02:12:41.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.111+0000 7fbb821a1640 1 -- 172.21.15.49:0/2089832632 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f82fcc6dc0 con 0x55f830be2400 2023-11-06T02:12:41.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.111+0000 7fbb899ed280 1 -- 172.21.15.49:0/2089832632 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f82fc98b60 con 0x55f830be2400 2023-11-06T02:12:41.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.111+0000 7fbb899ed280 4 main: JournalTool: resolving pool 9 2023-11-06T02:12:41.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.115+0000 7fbb821a1640 1 -- 172.21.15.49:0/2089832632 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x55f82fd2c840 con 0x55f830be2400 2023-11-06T02:12:41.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.115+0000 7fbb821a1640 1 --2- 172.21.15.49:0/2089832632 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x55f830be3000 0x55f83003c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:41.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.115+0000 7fbb821a1640 1 -- 172.21.15.49:0/2089832632 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6264+0+0 (secure 0 0 0) 0x55f82fca2a80 con 0x55f830be2400 2023-11-06T02:12:41.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.115+0000 7fbb899ed280 4 main: JournalTool: creating IoCtx.. 2023-11-06T02:12:41.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.115+0000 7fbb899ed280 4 main: Executing for rank 0 2023-11-06T02:12:41.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.115+0000 7fbb899ed280 4 Filtering by inode '1099511627777' 2023-11-06T02:12:41.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.115+0000 7fbb899ed280 1 --2- 172.21.15.49:0/2089832632 >> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] conn(0x55f830be3400 0x55f82fcf0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:41.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.115+0000 7fbb899ed280 1 -- 172.21.15.49:0/2089832632 --> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] -- osd_op(unknown.0.0:1 9.f 9:f1f69726:::400.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68) v8 -- 0x55f830be3800 con 0x55f830be3400 2023-11-06T02:12:41.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.115+0000 7fbb881ad640 1 --2- 172.21.15.49:0/2089832632 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x55f830be3000 0x55f83003c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:41.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.115+0000 7fbb879ac640 1 --2- 172.21.15.49:0/2089832632 >> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] conn(0x55f830be3400 0x55f82fcf0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:41.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.115+0000 7fbb881ad640 1 --2- 172.21.15.49:0/2089832632 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x55f830be3000 0x55f83003c000 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x55f830a3e030 tx=0x55f830a37500 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:41.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.115+0000 7fbb879ac640 1 --2- 172.21.15.49:0/2089832632 >> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] conn(0x55f830be3400 0x55f82fcf0680 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:41.130 INFO:tasks.ceph.mds.c.smithi049.stdout:starting mds.c at 2023-11-06T02:12:41.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.123+0000 7fbb879ac640 1 -- 172.21.15.49:0/2089832632 <== osd.5 v2:172.21.15.150:6800/38418 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) v8 ==== 156+0+22 (crc 0 0 0) 0x55f830b70fc0 con 0x55f830be3400 2023-11-06T02:12:41.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.123+0000 7fbb899ed280 4 Pointer 400.00000000 is readable 2023-11-06T02:12:41.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.123+0000 7fbb899ed280 4 JournalScanner::scan: reading header object '200.00000000' 2023-11-06T02:12:41.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.123+0000 7fbb899ed280 1 --2- 172.21.15.49:0/2089832632 >> [v2:172.21.15.49:6808/38113,v1:172.21.15.49:6810/38113] conn(0x55f830be3c00 0x55f82fcf0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:41.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.123+0000 7fbb899ed280 1 -- 172.21.15.49:0/2089832632 --> [v2:172.21.15.49:6808/38113,v1:172.21.15.49:6810/38113] -- osd_op(unknown.0.0:2 9.14 9:292cf221:::200.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68) v8 -- 0x55f830048c00 con 0x55f830be3c00 2023-11-06T02:12:41.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.123+0000 7fbb889ae640 1 --2- 172.21.15.49:0/2089832632 >> [v2:172.21.15.49:6808/38113,v1:172.21.15.49:6810/38113] conn(0x55f830be3c00 0x55f82fcf0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:41.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.123+0000 7fbb889ae640 1 --2- 172.21.15.49:0/2089832632 >> [v2:172.21.15.49:6808/38113,v1:172.21.15.49:6810/38113] conn(0x55f830be3c00 0x55f82fcf0c00 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:41.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.127+0000 7fbb889ae640 1 -- 172.21.15.49:0/2089832632 <== osd.1 v2:172.21.15.49:6808/38113 1 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv31 ondisk = 0) v8 ==== 156+0+90 (crc 0 0 0) 0x55f830b71200 con 0x55f830be3c00 2023-11-06T02:12:41.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.127+0000 7fbb899ed280 10 Header 0x400000 0x405ee2 0x40625e 2023-11-06T02:12:41.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.127+0000 7fbb899ed280 10 Starting journal scan from offset 0x405ee2 2023-11-06T02:12:41.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.127+0000 7fbb899ed280 1 -- 172.21.15.49:0/2089832632 --> [v2:172.21.15.49:6808/38113,v1:172.21.15.49:6810/38113] -- osd_op(unknown.0.0:3 9.14 9:2e2fa760:::200.00000001:head [read 24290~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68) v8 -- 0x55f830049000 con 0x55f830be3c00 2023-11-06T02:12:41.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.127+0000 7fbb889ae640 1 -- 172.21.15.49:0/2089832632 <== osd.1 v2:172.21.15.49:6808/38113 2 ==== osd_op_reply(3 200.00000001 [read 24290~892 out=892b] v0'0 uv27 ondisk = 0) v8 ==== 156+0+892 (crc 0 0 0) 0x55f830b71200 con 0x55f830be3c00 2023-11-06T02:12:41.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.127+0000 7fbb899ed280 4 Read 0x37c bytes from 200.00000001 gap=0 2023-11-06T02:12:41.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.127+0000 7fbb899ed280 10 Parsing data, 0x37c bytes available 2023-11-06T02:12:41.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.127+0000 7fbb899ed280 10 Attempting decode at 0x405ee2 2023-11-06T02:12:41.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.127+0000 7fbb899ed280 10 Consumed 0x37c bytes 2023-11-06T02:12:41.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.127+0000 7fbb899ed280 10 Valid entry at 0x405ee2 2023-11-06T02:12:41.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.127+0000 7fbb899ed280 1 --2- 172.21.15.49:0/2089832632 >> [v2:172.21.15.49:6824/38115,v1:172.21.15.49:6825/38115] conn(0x55f830049400 0x55f82fcf1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:41.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.127+0000 7fbb899ed280 1 -- 172.21.15.49:0/2089832632 --> [v2:172.21.15.49:6824/38115,v1:172.21.15.49:6825/38115] -- osd_op(unknown.0.0:4 9.1b 9:ddb4e4d7:::200.00000002:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68) v8 -- 0x55f830049800 con 0x55f830049400 2023-11-06T02:12:41.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.127+0000 7fbb881ad640 1 --2- 172.21.15.49:0/2089832632 >> [v2:172.21.15.49:6824/38115,v1:172.21.15.49:6825/38115] conn(0x55f830049400 0x55f82fcf1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:41.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.127+0000 7fbb881ad640 1 --2- 172.21.15.49:0/2089832632 >> [v2:172.21.15.49:6824/38115,v1:172.21.15.49:6825/38115] conn(0x55f830049400 0x55f82fcf1180 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:41.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.127+0000 7fbb881ad640 1 -- 172.21.15.49:0/2089832632 <== osd.3 v2:172.21.15.49:6824/38115 1 ==== osd_op_reply(4 200.00000002 [read 0~2147483647] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55f830b71440 con 0x55f830049400 2023-11-06T02:12:41.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.127+0000 7fbb899ed280 4 Reached end of journal objects 2023-11-06T02:12:41.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.127+0000 7fbb899ed280 4 Scanned objects, 0 missing, 1 valid 2023-11-06T02:12:41.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.127+0000 7fbb899ed280 4 Events scanned, 0 gaps 2023-11-06T02:12:41.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.127+0000 7fbb899ed280 4 Found 1 valid events 2023-11-06T02:12:41.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.127+0000 7fbb899ed280 4 Selected 0 events events for processing 2023-11-06T02:12:41.138 INFO:teuthology.orchestra.run.smithi049.stdout:Events by type: 2023-11-06T02:12:41.138 INFO:teuthology.orchestra.run.smithi049.stdout:Errors: 0 2023-11-06T02:12:41.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.127+0000 7fbb899ed280 4 0 events 2023-11-06T02:12:41.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.127+0000 7fbb899ed280 1 -- 172.21.15.49:0/2089832632 >> [v2:172.21.15.49:6808/38113,v1:172.21.15.49:6810/38113] conn(0x55f830be3c00 msgr2=0x55f82fcf0c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:41.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.127+0000 7fbb899ed280 1 --2- 172.21.15.49:0/2089832632 >> [v2:172.21.15.49:6808/38113,v1:172.21.15.49:6810/38113] conn(0x55f830be3c00 0x55f82fcf0c00 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:41.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.127+0000 7fbb899ed280 1 -- 172.21.15.49:0/2089832632 >> [v2:172.21.15.49:6824/38115,v1:172.21.15.49:6825/38115] conn(0x55f830049400 msgr2=0x55f82fcf1180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:41.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.127+0000 7fbb899ed280 1 --2- 172.21.15.49:0/2089832632 >> [v2:172.21.15.49:6824/38115,v1:172.21.15.49:6825/38115] conn(0x55f830049400 0x55f82fcf1180 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:41.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.127+0000 7fbb899ed280 1 -- 172.21.15.49:0/2089832632 >> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] conn(0x55f830be3400 msgr2=0x55f82fcf0680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:41.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.127+0000 7fbb899ed280 1 --2- 172.21.15.49:0/2089832632 >> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] conn(0x55f830be3400 0x55f82fcf0680 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:41.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.127+0000 7fbb881ad640 1 -- 172.21.15.49:0/2089832632 reap_dead start 2023-11-06T02:12:41.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.127+0000 7fbb899ed280 1 -- 172.21.15.49:0/2089832632 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x55f830be3000 msgr2=0x55f83003c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:41.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.127+0000 7fbb899ed280 1 --2- 172.21.15.49:0/2089832632 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x55f830be3000 0x55f83003c000 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x55f830a3e030 tx=0x55f830a37500 comp rx=0 tx=0).stop 2023-11-06T02:12:41.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.127+0000 7fbb899ed280 1 -- 172.21.15.49:0/2089832632 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f830be2400 msgr2=0x55f82fceeb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:41.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.127+0000 7fbb899ed280 1 --2- 172.21.15.49:0/2089832632 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f830be2400 0x55f82fceeb00 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x55f830a49c50 tx=0x55f830a384e0 comp rx=0 tx=0).stop 2023-11-06T02:12:41.143 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.127+0000 7fbb899ed280 1 -- 172.21.15.49:0/2089832632 shutdown_connections 2023-11-06T02:12:41.143 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.127+0000 7fbb899ed280 1 --2- 172.21.15.49:0/2089832632 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x55f830be3000 0x55f83003c000 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:41.143 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.127+0000 7fbb899ed280 1 --2- 172.21.15.49:0/2089832632 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f830be2400 0x55f82fceeb00 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:41.143 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.127+0000 7fbb899ed280 1 -- 172.21.15.49:0/2089832632 >> 172.21.15.49:0/2089832632 conn(0x55f830be2000 msgr2=0x55f82fc91800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:12:41.143 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.127+0000 7fbb899ed280 1 -- 172.21.15.49:0/2089832632 shutdown_connections 2023-11-06T02:12:41.143 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.127+0000 7fbb899ed280 1 -- 172.21.15.49:0/2089832632 wait complete. 2023-11-06T02:12:41.144 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.131+0000 7fbb899ed280 1 -- 172.21.15.49:0/2831070338 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f82fc63800 msgr2=0x55f82fcef080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:41.144 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.131+0000 7fbb899ed280 1 --2- 172.21.15.49:0/2831070338 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f82fc63800 0x55f82fcef080 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x55f830a487b0 tx=0x55f830a385a0 comp rx=0 tx=0).stop 2023-11-06T02:12:41.144 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.131+0000 7fbb899ed280 1 -- 172.21.15.49:0/2831070338 shutdown_connections 2023-11-06T02:12:41.144 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.131+0000 7fbb899ed280 1 --2- 172.21.15.49:0/2831070338 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f82fc63800 0x55f82fcef080 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:41.144 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.131+0000 7fbb899ed280 1 --2- 172.21.15.49:0/2831070338 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f82fc63400 0x55f82fcef600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:41.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.131+0000 7fbb899ed280 1 --2- 172.21.15.49:0/2831070338 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f82fc63000 0x55f82fcefb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:41.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.131+0000 7fbb899ed280 1 -- 172.21.15.49:0/2831070338 >> 172.21.15.49:0/2831070338 conn(0x55f82fc62c00 msgr2=0x55f82fc91000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:12:41.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.131+0000 7fbb899ed280 1 -- 172.21.15.49:0/2831070338 shutdown_connections 2023-11-06T02:12:41.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.131+0000 7fbb899ed280 1 -- 172.21.15.49:0/2831070338 wait complete. 2023-11-06T02:12:41.145 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'event', 'splice', '--inode=1099511627777', 'summary'] in time 0:00:00.081992, result: Events by type: Errors: 0 2023-11-06T02:12:41.146 DEBUG:teuthology.orchestra.run.smithi049:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 event splice --inode=1099511627778 summary 2023-11-06T02:12:41.218 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.219+0000 7f59f68b5280 1 -- start start 2023-11-06T02:12:41.218 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.219+0000 7f59f68b5280 1 Processor -- start 2023-11-06T02:12:41.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.219+0000 7f59f68b5280 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561137dc5800 0x561137e50580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:41.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.219+0000 7f59f68b5280 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561137dc5400 0x561137e50b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:41.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.219+0000 7f59f68b5280 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x561137dc5000 0x561137e51080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:41.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.219+0000 7f59f68b5280 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x561137e92a00 con 0x561137dc5800 2023-11-06T02:12:41.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.219+0000 7f59f68b5280 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x561137e92780 con 0x561137dc5400 2023-11-06T02:12:41.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.219+0000 7f59f68b5280 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x561137e92640 con 0x561137dc5000 2023-11-06T02:12:41.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.219+0000 7f59f5075640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561137dc5800 0x561137e50580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:41.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.219+0000 7f59f5075640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561137dc5800 0x561137e50580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58730/0 (socket says 172.21.15.49:58730) 2023-11-06T02:12:41.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.219+0000 7f59f5075640 1 -- 172.21.15.49:0/3213561090 learned_addr learned my addr 172.21.15.49:0/3213561090 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:12:41.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.219+0000 7f59f4874640 1 --2- 172.21.15.49:0/3213561090 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561137dc5400 0x561137e50b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:41.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.219+0000 7f59f5876640 1 --2- 172.21.15.49:0/3213561090 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x561137dc5000 0x561137e51080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:41.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.219+0000 7f59f5075640 1 -- 172.21.15.49:0/3213561090 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x561137dc5000 msgr2=0x561137e51080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:41.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.219+0000 7f59f5075640 1 --2- 172.21.15.49:0/3213561090 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x561137dc5000 0x561137e51080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:41.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.219+0000 7f59f5075640 1 -- 172.21.15.49:0/3213561090 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561137dc5400 msgr2=0x561137e50b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:41.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.219+0000 7f59f5075640 1 --2- 172.21.15.49:0/3213561090 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561137dc5400 0x561137e50b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:41.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.219+0000 7f59f5075640 1 -- 172.21.15.49:0/3213561090 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561137dfab60 con 0x561137dc5800 2023-11-06T02:12:41.229 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.219+0000 7f59f5075640 1 --2- 172.21.15.49:0/3213561090 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561137dc5800 0x561137e50580 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x561138cc46c0 tx=0x561137ebb140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8149c3601e4ee9e9 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:41.229 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.219+0000 7f59f2870640 1 -- 172.21.15.49:0/3213561090 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x561137e29600 con 0x561137dc5800 2023-11-06T02:12:41.229 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.219+0000 7f59f2870640 1 -- 172.21.15.49:0/3213561090 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561137e20480 con 0x561137dc5800 2023-11-06T02:12:41.229 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.219+0000 7f59f68b5280 1 -- 172.21.15.49:0/3213561090 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561137dfa340 con 0x561137dc5800 2023-11-06T02:12:41.229 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.219+0000 7f59f68b5280 1 -- 172.21.15.49:0/3213561090 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x561137dfa000 con 0x561137dc5800 2023-11-06T02:12:41.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.219+0000 7f59f68b5280 4 waiting for MDS map... 2023-11-06T02:12:41.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.219+0000 7f59f2870640 1 -- 172.21.15.49:0/3213561090 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x561137e291e0 con 0x561137dc5800 2023-11-06T02:12:41.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.219+0000 7f59f2870640 1 -- 172.21.15.49:0/3213561090 <== mon.0 v2:172.21.15.49:3300/0 4 ==== osd_map(68..68 src has 1..68) v4 ==== 6264+0+0 (secure 0 0 0) 0x561137e04c40 con 0x561137dc5800 2023-11-06T02:12:41.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.219+0000 7f59f2870640 1 -- 172.21.15.49:0/3213561090 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 31) v1 ==== 2565+0+0 (secure 0 0 0) 0x561137e12f00 con 0x561137dc5800 2023-11-06T02:12:41.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.219+0000 7f59f68b5280 4 Got MDS map 31 2023-11-06T02:12:41.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.219+0000 7f59f68b5280 10 main: JournalTool::main 2023-11-06T02:12:41.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.219+0000 7f59f68b5280 4 main: JournalTool: connecting to RADOS... 2023-11-06T02:12:41.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.223+0000 7f59f68b5280 1 Processor -- start 2023-11-06T02:12:41.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.223+0000 7f59f68b5280 1 -- start start 2023-11-06T02:12:41.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.223+0000 7f59f68b5280 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x561138cdec00 0x561137e51600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:41.232 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.223+0000 7f59f68b5280 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561138cdf000 0x561137e51b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:41.232 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.223+0000 7f59f68b5280 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561138cdf400 0x561137e52100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:41.232 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.223+0000 7f59f68b5280 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x561137e0be00 con 0x561138cdf400 2023-11-06T02:12:41.232 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.223+0000 7f59f68b5280 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x561137e92780 con 0x561138cdf000 2023-11-06T02:12:41.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.223+0000 7f59f68b5280 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x561137e92500 con 0x561138cdec00 2023-11-06T02:12:41.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.223+0000 7f59f4874640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561138cdf400 0x561137e52100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:41.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.223+0000 7f59f4874640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561138cdf400 0x561137e52100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58736/0 (socket says 172.21.15.49:58736) 2023-11-06T02:12:41.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.223+0000 7f59f4874640 1 -- 172.21.15.49:0/3202356892 learned_addr learned my addr 172.21.15.49:0/3202356892 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:12:41.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.223+0000 7f59f5876640 1 --2- 172.21.15.49:0/3202356892 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x561138cdec00 0x561137e51600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:41.234 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.223+0000 7f59f5075640 1 --2- 172.21.15.49:0/3202356892 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561138cdf000 0x561137e51b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:41.234 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.223+0000 7f59f4874640 1 -- 172.21.15.49:0/3202356892 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x561138cdec00 msgr2=0x561137e51600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:41.234 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.223+0000 7f59f4874640 1 --2- 172.21.15.49:0/3202356892 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x561138cdec00 0x561137e51600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:41.234 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.223+0000 7f59f4874640 1 -- 172.21.15.49:0/3202356892 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561138cdf000 msgr2=0x561137e51b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:41.234 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.223+0000 7f59f4874640 1 --2- 172.21.15.49:0/3202356892 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561138cdf000 0x561137e51b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:41.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.223+0000 7f59f4874640 1 -- 172.21.15.49:0/3202356892 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561137dfaea0 con 0x561138cdf400 2023-11-06T02:12:41.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.223+0000 7f59f4874640 1 --2- 172.21.15.49:0/3202356892 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561138cdf400 0x561137e52100 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x561138cc5ad0 tx=0x561137eba000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6993a0360fa9a626 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:41.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.223+0000 7f59f086c640 1 -- 172.21.15.49:0/3202356892 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x561137e29080 con 0x561138cdf400 2023-11-06T02:12:41.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.223+0000 7f59f086c640 1 -- 172.21.15.49:0/3202356892 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561138b9c900 con 0x561138cdf400 2023-11-06T02:12:41.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.223+0000 7f59f086c640 1 -- 172.21.15.49:0/3202356892 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x561137e29600 con 0x561138cdf400 2023-11-06T02:12:41.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.223+0000 7f59f68b5280 1 -- 172.21.15.49:0/3202356892 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561138cdf400 msgr2=0x561137e52100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:41.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.223+0000 7f59f68b5280 1 --2- 172.21.15.49:0/3202356892 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561138cdf400 0x561137e52100 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x561138cc5ad0 tx=0x561137eba000 comp rx=0 tx=0).stop 2023-11-06T02:12:41.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.223+0000 7f59f68b5280 1 -- 172.21.15.49:0/3202356892 shutdown_connections 2023-11-06T02:12:41.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.223+0000 7f59f68b5280 1 --2- 172.21.15.49:0/3202356892 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561138cdf400 0x561137e52100 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:41.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.223+0000 7f59f68b5280 1 --2- 172.21.15.49:0/3202356892 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561138cdf000 0x561137e51b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:41.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.223+0000 7f59f68b5280 1 --2- 172.21.15.49:0/3202356892 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x561138cdec00 0x561137e51600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:41.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.223+0000 7f59f68b5280 1 -- 172.21.15.49:0/3202356892 >> 172.21.15.49:0/3202356892 conn(0x561137dc4c00 msgr2=0x561137df3800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:12:41.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.223+0000 7f59f68b5280 1 -- 172.21.15.49:0/3202356892 shutdown_connections 2023-11-06T02:12:41.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.223+0000 7f59f68b5280 1 -- 172.21.15.49:0/3202356892 wait complete. 2023-11-06T02:12:41.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.223+0000 7f59f68b5280 1 Processor -- start 2023-11-06T02:12:41.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.223+0000 7f59f68b5280 1 -- start start 2023-11-06T02:12:41.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.223+0000 7f59f68b5280 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561138cdec00 0x561137e53180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:41.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.223+0000 7f59f68b5280 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561138cdf000 0x561137e52c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:41.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.223+0000 7f59f68b5280 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x561138cdf400 0x561137e52680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:41.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.223+0000 7f59f68b5280 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x561137e923c0 con 0x561138cdec00 2023-11-06T02:12:41.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.223+0000 7f59f68b5280 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x561137e92280 con 0x561138cdf000 2023-11-06T02:12:41.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.223+0000 7f59f68b5280 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x561137e92140 con 0x561138cdf400 2023-11-06T02:12:41.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.223+0000 7f59f5876640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561138cdec00 0x561137e53180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:41.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.223+0000 7f59f5876640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561138cdec00 0x561137e53180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58748/0 (socket says 172.21.15.49:58748) 2023-11-06T02:12:41.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.223+0000 7f59f5876640 1 -- 172.21.15.49:0/2221323911 learned_addr learned my addr 172.21.15.49:0/2221323911 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:12:41.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.227+0000 7f59f4874640 1 --2- 172.21.15.49:0/2221323911 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x561138cdf400 0x561137e52680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:41.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.227+0000 7f59f5075640 1 --2- 172.21.15.49:0/2221323911 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561138cdf000 0x561137e52c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:41.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.227+0000 7f59f5876640 1 -- 172.21.15.49:0/2221323911 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x561138cdf400 msgr2=0x561137e52680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:41.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.227+0000 7f59f5876640 1 --2- 172.21.15.49:0/2221323911 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x561138cdf400 0x561137e52680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:41.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.227+0000 7f59f5876640 1 -- 172.21.15.49:0/2221323911 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561138cdf000 msgr2=0x561137e52c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:41.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.227+0000 7f59f5876640 1 --2- 172.21.15.49:0/2221323911 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561138cdf000 0x561137e52c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:41.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.227+0000 7f59f5876640 1 -- 172.21.15.49:0/2221323911 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561137dfb1e0 con 0x561138cdec00 2023-11-06T02:12:41.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.227+0000 7f59f5876640 1 --2- 172.21.15.49:0/2221323911 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561138cdec00 0x561137e53180 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x561138cd4cf0 tx=0x561138cf4c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:41.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.227+0000 7f59ef069640 1 -- 172.21.15.49:0/2221323911 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x561137e29080 con 0x561138cdec00 2023-11-06T02:12:41.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.227+0000 7f59ef069640 1 -- 172.21.15.49:0/2221323911 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561137e20180 con 0x561138cdec00 2023-11-06T02:12:41.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.227+0000 7f59f68b5280 1 -- 172.21.15.49:0/2221323911 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x561137dfaea0 con 0x561138cdec00 2023-11-06T02:12:41.242 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.227+0000 7f59ef069640 1 -- 172.21.15.49:0/2221323911 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x561137e29600 con 0x561138cdec00 2023-11-06T02:12:41.242 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.227+0000 7f59f68b5280 1 -- 172.21.15.49:0/2221323911 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561137dfab60 con 0x561138cdec00 2023-11-06T02:12:41.242 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.227+0000 7f59f68b5280 4 main: JournalTool: resolving pool 9 2023-11-06T02:12:41.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.227+0000 7f59ef069640 1 -- 172.21.15.49:0/2221323911 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x561137e8e840 con 0x561138cdec00 2023-11-06T02:12:41.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.227+0000 7f59ef069640 1 --2- 172.21.15.49:0/2221323911 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x5611380a8c00 0x561137e53700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:41.246 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.227+0000 7f59ef069640 1 -- 172.21.15.49:0/2221323911 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6264+0+0 (secure 0 0 0) 0x561137e04a80 con 0x561138cdec00 2023-11-06T02:12:41.246 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.227+0000 7f59f68b5280 4 main: JournalTool: creating IoCtx.. 2023-11-06T02:12:41.246 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.227+0000 7f59f68b5280 4 main: Executing for rank 0 2023-11-06T02:12:41.246 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.227+0000 7f59f68b5280 4 Filtering by inode '1099511627778' 2023-11-06T02:12:41.246 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.227+0000 7f59f68b5280 1 --2- 172.21.15.49:0/2221323911 >> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] conn(0x561138cdf800 0x561137e52100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:41.247 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.227+0000 7f59f68b5280 1 -- 172.21.15.49:0/2221323911 --> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] -- osd_op(unknown.0.0:1 9.f 9:f1f69726:::400.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68) v8 -- 0x561138cdfc00 con 0x561138cdf800 2023-11-06T02:12:41.247 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.227+0000 7f59f5075640 1 --2- 172.21.15.49:0/2221323911 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x5611380a8c00 0x561137e53700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:41.247 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.227+0000 7f59f4874640 1 --2- 172.21.15.49:0/2221323911 >> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] conn(0x561138cdf800 0x561137e52100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:41.247 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.227+0000 7f59f4874640 1 --2- 172.21.15.49:0/2221323911 >> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] conn(0x561138cdf800 0x561137e52100 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:41.248 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.227+0000 7f59f5075640 1 --2- 172.21.15.49:0/2221323911 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x5611380a8c00 0x561137e53700 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x561138cc4990 tx=0x561137eba120 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:41.248 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.231+0000 7f59f4874640 1 -- 172.21.15.49:0/2221323911 <== osd.5 v2:172.21.15.150:6800/38418 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) v8 ==== 156+0+22 (crc 0 0 0) 0x561138cc0fc0 con 0x561138cdf800 2023-11-06T02:12:41.248 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.231+0000 7f59f68b5280 4 Pointer 400.00000000 is readable 2023-11-06T02:12:41.248 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.231+0000 7f59f68b5280 4 JournalScanner::scan: reading header object '200.00000000' 2023-11-06T02:12:41.248 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.231+0000 7f59f68b5280 1 --2- 172.21.15.49:0/2221323911 >> [v2:172.21.15.49:6808/38113,v1:172.21.15.49:6810/38113] conn(0x5611380a9000 0x561137e51b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:41.248 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.231+0000 7f59f68b5280 1 -- 172.21.15.49:0/2221323911 --> [v2:172.21.15.49:6808/38113,v1:172.21.15.49:6810/38113] -- osd_op(unknown.0.0:2 9.14 9:292cf221:::200.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68) v8 -- 0x5611380a9400 con 0x5611380a9000 2023-11-06T02:12:41.249 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.231+0000 7f59f5876640 1 --2- 172.21.15.49:0/2221323911 >> [v2:172.21.15.49:6808/38113,v1:172.21.15.49:6810/38113] conn(0x5611380a9000 0x561137e51b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:41.249 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.231+0000 7f59f5876640 1 --2- 172.21.15.49:0/2221323911 >> [v2:172.21.15.49:6808/38113,v1:172.21.15.49:6810/38113] conn(0x5611380a9000 0x561137e51b80 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:41.249 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.231+0000 7f59f5876640 1 -- 172.21.15.49:0/2221323911 <== osd.1 v2:172.21.15.49:6808/38113 1 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv31 ondisk = 0) v8 ==== 156+0+90 (crc 0 0 0) 0x561138cc1200 con 0x5611380a9000 2023-11-06T02:12:41.249 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.231+0000 7f59f68b5280 10 Header 0x400000 0x405ee2 0x40625e 2023-11-06T02:12:41.249 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.231+0000 7f59f68b5280 10 Starting journal scan from offset 0x405ee2 2023-11-06T02:12:41.250 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.231+0000 7f59f68b5280 1 -- 172.21.15.49:0/2221323911 --> [v2:172.21.15.49:6808/38113,v1:172.21.15.49:6810/38113] -- osd_op(unknown.0.0:3 9.14 9:2e2fa760:::200.00000001:head [read 24290~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68) v8 -- 0x561138cdfc00 con 0x5611380a9000 2023-11-06T02:12:41.250 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.231+0000 7f59f5876640 1 -- 172.21.15.49:0/2221323911 <== osd.1 v2:172.21.15.49:6808/38113 2 ==== osd_op_reply(3 200.00000001 [read 24290~892 out=892b] v0'0 uv27 ondisk = 0) v8 ==== 156+0+892 (crc 0 0 0) 0x561138cc1200 con 0x5611380a9000 2023-11-06T02:12:41.250 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.231+0000 7f59f68b5280 4 Read 0x37c bytes from 200.00000001 gap=0 2023-11-06T02:12:41.250 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.231+0000 7f59f68b5280 10 Parsing data, 0x37c bytes available 2023-11-06T02:12:41.250 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.231+0000 7f59f68b5280 10 Attempting decode at 0x405ee2 2023-11-06T02:12:41.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.231+0000 7f59f68b5280 10 Consumed 0x37c bytes 2023-11-06T02:12:41.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.231+0000 7f59f68b5280 10 Valid entry at 0x405ee2 2023-11-06T02:12:41.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.231+0000 7f59f68b5280 1 --2- 172.21.15.49:0/2221323911 >> [v2:172.21.15.49:6824/38115,v1:172.21.15.49:6825/38115] conn(0x561138b9f400 0x561137e51600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:41.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.231+0000 7f59f68b5280 1 -- 172.21.15.49:0/2221323911 --> [v2:172.21.15.49:6824/38115,v1:172.21.15.49:6825/38115] -- osd_op(unknown.0.0:4 9.1b 9:ddb4e4d7:::200.00000002:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68) v8 -- 0x561138b9f000 con 0x561138b9f400 2023-11-06T02:12:41.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.231+0000 7f59f5075640 1 --2- 172.21.15.49:0/2221323911 >> [v2:172.21.15.49:6824/38115,v1:172.21.15.49:6825/38115] conn(0x561138b9f400 0x561137e51600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:41.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.231+0000 7f59f5075640 1 --2- 172.21.15.49:0/2221323911 >> [v2:172.21.15.49:6824/38115,v1:172.21.15.49:6825/38115] conn(0x561138b9f400 0x561137e51600 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:41.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.231+0000 7f59f5075640 1 -- 172.21.15.49:0/2221323911 <== osd.3 v2:172.21.15.49:6824/38115 1 ==== osd_op_reply(4 200.00000002 [read 0~2147483647] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x561138cc1440 con 0x561138b9f400 2023-11-06T02:12:41.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.231+0000 7f59f68b5280 4 Reached end of journal objects 2023-11-06T02:12:41.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.231+0000 7f59f68b5280 4 Scanned objects, 0 missing, 1 valid 2023-11-06T02:12:41.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.231+0000 7f59f68b5280 4 Events scanned, 0 gaps 2023-11-06T02:12:41.253 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.231+0000 7f59f68b5280 4 Found 1 valid events 2023-11-06T02:12:41.253 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.231+0000 7f59f68b5280 4 Selected 0 events events for processing 2023-11-06T02:12:41.253 INFO:teuthology.orchestra.run.smithi049.stdout:Events by type: 2023-11-06T02:12:41.253 INFO:teuthology.orchestra.run.smithi049.stdout:Errors: 0 2023-11-06T02:12:41.256 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.231+0000 7f59f68b5280 4 0 events 2023-11-06T02:12:41.256 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.231+0000 7f59f68b5280 1 -- 172.21.15.49:0/2221323911 >> [v2:172.21.15.49:6808/38113,v1:172.21.15.49:6810/38113] conn(0x5611380a9000 msgr2=0x561137e51b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:41.256 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.231+0000 7f59f68b5280 1 --2- 172.21.15.49:0/2221323911 >> [v2:172.21.15.49:6808/38113,v1:172.21.15.49:6810/38113] conn(0x5611380a9000 0x561137e51b80 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:41.256 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.231+0000 7f59f68b5280 1 -- 172.21.15.49:0/2221323911 >> [v2:172.21.15.49:6824/38115,v1:172.21.15.49:6825/38115] conn(0x561138b9f400 msgr2=0x561137e51600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:41.256 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.231+0000 7f59f68b5280 1 --2- 172.21.15.49:0/2221323911 >> [v2:172.21.15.49:6824/38115,v1:172.21.15.49:6825/38115] conn(0x561138b9f400 0x561137e51600 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:41.257 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.231+0000 7f59f68b5280 1 -- 172.21.15.49:0/2221323911 >> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] conn(0x561138cdf800 msgr2=0x561137e52100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:41.257 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.231+0000 7f59f68b5280 1 --2- 172.21.15.49:0/2221323911 >> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] conn(0x561138cdf800 0x561137e52100 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:41.257 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.231+0000 7f59f4874640 1 -- 172.21.15.49:0/2221323911 reap_dead start 2023-11-06T02:12:41.257 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.231+0000 7f59f68b5280 1 -- 172.21.15.49:0/2221323911 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x5611380a8c00 msgr2=0x561137e53700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:41.257 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.231+0000 7f59f68b5280 1 --2- 172.21.15.49:0/2221323911 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x5611380a8c00 0x561137e53700 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x561138cc4990 tx=0x561137eba120 comp rx=0 tx=0).stop 2023-11-06T02:12:41.258 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.231+0000 7f59f68b5280 1 -- 172.21.15.49:0/2221323911 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561138cdec00 msgr2=0x561137e53180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:41.258 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.231+0000 7f59f68b5280 1 --2- 172.21.15.49:0/2221323911 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561138cdec00 0x561137e53180 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x561138cd4cf0 tx=0x561138cf4c00 comp rx=0 tx=0).stop 2023-11-06T02:12:41.258 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.231+0000 7f59f68b5280 1 -- 172.21.15.49:0/2221323911 shutdown_connections 2023-11-06T02:12:41.258 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.231+0000 7f59f68b5280 1 --2- 172.21.15.49:0/2221323911 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x5611380a8c00 0x561137e53700 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:41.259 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.231+0000 7f59f68b5280 1 --2- 172.21.15.49:0/2221323911 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561138cdec00 0x561137e53180 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:41.259 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.231+0000 7f59f68b5280 1 -- 172.21.15.49:0/2221323911 >> 172.21.15.49:0/2221323911 conn(0x561137dc4c00 msgr2=0x561137df3800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:12:41.259 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.231+0000 7f59f68b5280 1 -- 172.21.15.49:0/2221323911 shutdown_connections 2023-11-06T02:12:41.259 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.231+0000 7f59f68b5280 1 -- 172.21.15.49:0/2221323911 wait complete. 2023-11-06T02:12:41.259 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.231+0000 7f59f68b5280 1 -- 172.21.15.49:0/3213561090 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561137dc5800 msgr2=0x561137e50580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:41.260 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.231+0000 7f59f68b5280 1 --2- 172.21.15.49:0/3213561090 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561137dc5800 0x561137e50580 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x561138cc46c0 tx=0x561137ebb140 comp rx=0 tx=0).stop 2023-11-06T02:12:41.260 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.231+0000 7f59f68b5280 1 -- 172.21.15.49:0/3213561090 shutdown_connections 2023-11-06T02:12:41.260 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.231+0000 7f59f68b5280 1 --2- 172.21.15.49:0/3213561090 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x561137dc5000 0x561137e51080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:41.260 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.231+0000 7f59f68b5280 1 --2- 172.21.15.49:0/3213561090 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561137dc5400 0x561137e50b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:41.260 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.231+0000 7f59f68b5280 1 --2- 172.21.15.49:0/3213561090 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561137dc5800 0x561137e50580 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:41.261 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.231+0000 7f59f68b5280 1 -- 172.21.15.49:0/3213561090 >> 172.21.15.49:0/3213561090 conn(0x561137dc5c00 msgr2=0x561137df3000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:12:41.261 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.231+0000 7f59f68b5280 1 -- 172.21.15.49:0/3213561090 shutdown_connections 2023-11-06T02:12:41.261 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.231+0000 7f59f68b5280 1 -- 172.21.15.49:0/3213561090 wait complete. 2023-11-06T02:12:41.261 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'event', 'splice', '--inode=1099511627778', 'summary'] in time 0:00:00.115556, result: Events by type: Errors: 0 2023-11-06T02:12:41.262 DEBUG:teuthology.orchestra.run.smithi049:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put mds0_inotable - 2023-11-06T02:12:41.344 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.343+0000 7ff097266080 1 Processor -- start 2023-11-06T02:12:41.344 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.343+0000 7ff097266080 1 -- start start 2023-11-06T02:12:41.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.343+0000 7ff097266080 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d7a8edc8f0 0x55d7a8f632d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:41.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.343+0000 7ff097266080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d7a8ef8fa0 0x55d7a8ef4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:41.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.343+0000 7ff097266080 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d7a8f84540 0x55d7a8f81090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:41.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.343+0000 7ff097266080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7a8df6450 con 0x55d7a8ef8fa0 2023-11-06T02:12:41.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.343+0000 7ff097266080 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7a8df6150 con 0x55d7a8edc8f0 2023-11-06T02:12:41.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.343+0000 7ff097266080 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d7a8decdc0 con 0x55d7a8f84540 2023-11-06T02:12:41.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.347+0000 7ff09457f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d7a8ef8fa0 0x55d7a8ef4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:41.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.347+0000 7ff09457f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d7a8ef8fa0 0x55d7a8ef4910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58760/0 (socket says 172.21.15.49:58760) 2023-11-06T02:12:41.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.347+0000 7ff09457f640 1 -- 172.21.15.49:0/3194967990 learned_addr learned my addr 172.21.15.49:0/3194967990 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:12:41.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.347+0000 7ff094d80640 1 --2- 172.21.15.49:0/3194967990 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d7a8edc8f0 0x55d7a8f632d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:41.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.347+0000 7ff095581640 1 --2- 172.21.15.49:0/3194967990 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d7a8f84540 0x55d7a8f81090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:41.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.347+0000 7ff09457f640 1 -- 172.21.15.49:0/3194967990 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d7a8f84540 msgr2=0x55d7a8f81090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:41.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.347+0000 7ff09457f640 1 --2- 172.21.15.49:0/3194967990 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d7a8f84540 0x55d7a8f81090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:41.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.347+0000 7ff09457f640 1 -- 172.21.15.49:0/3194967990 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d7a8edc8f0 msgr2=0x55d7a8f632d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:41.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.347+0000 7ff09457f640 1 --2- 172.21.15.49:0/3194967990 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d7a8edc8f0 0x55d7a8f632d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:41.350 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.347+0000 7ff09457f640 1 -- 172.21.15.49:0/3194967990 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff07c005d30 con 0x55d7a8ef8fa0 2023-11-06T02:12:41.350 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.347+0000 7ff09457f640 1 --2- 172.21.15.49:0/3194967990 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d7a8ef8fa0 0x55d7a8ef4910 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7ff07c00a500 tx=0x7ff07c003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb42f3fc84381d79 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:41.350 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.347+0000 7ff09357d640 1 -- 172.21.15.49:0/3194967990 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff07c0aa740 con 0x55d7a8ef8fa0 2023-11-06T02:12:41.350 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.347+0000 7ff09357d640 1 -- 172.21.15.49:0/3194967990 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff07c0aa8c0 con 0x55d7a8ef8fa0 2023-11-06T02:12:41.350 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.347+0000 7ff09357d640 1 -- 172.21.15.49:0/3194967990 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff07c0aac70 con 0x55d7a8ef8fa0 2023-11-06T02:12:41.351 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.347+0000 7ff097266080 1 -- 172.21.15.49:0/3194967990 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d7a8ef8fa0 msgr2=0x55d7a8ef4910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:41.351 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.347+0000 7ff097266080 1 --2- 172.21.15.49:0/3194967990 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d7a8ef8fa0 0x55d7a8ef4910 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7ff07c00a500 tx=0x7ff07c003030 comp rx=0 tx=0).stop 2023-11-06T02:12:41.351 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.347+0000 7ff097266080 1 -- 172.21.15.49:0/3194967990 shutdown_connections 2023-11-06T02:12:41.351 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.347+0000 7ff097266080 1 --2- 172.21.15.49:0/3194967990 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d7a8ef8fa0 0x55d7a8ef4910 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:41.351 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.347+0000 7ff097266080 1 --2- 172.21.15.49:0/3194967990 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d7a8f84540 0x55d7a8f81090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:41.351 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.347+0000 7ff097266080 1 --2- 172.21.15.49:0/3194967990 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d7a8edc8f0 0x55d7a8f632d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:41.352 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.347+0000 7ff097266080 1 -- 172.21.15.49:0/3194967990 >> 172.21.15.49:0/3194967990 conn(0x55d7a8edaf90 msgr2=0x55d7a8edbc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:12:41.352 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.347+0000 7ff097266080 1 -- 172.21.15.49:0/3194967990 shutdown_connections 2023-11-06T02:12:41.352 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.347+0000 7ff097266080 1 -- 172.21.15.49:0/3194967990 wait complete. 2023-11-06T02:12:41.352 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.347+0000 7ff097266080 1 Processor -- start 2023-11-06T02:12:41.352 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.347+0000 7ff097266080 1 -- start start 2023-11-06T02:12:41.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.347+0000 7ff097266080 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d7a8edc8f0 0x55d7a8f7efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:41.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.347+0000 7ff097266080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d7a8ef8fa0 0x55d7a8f7f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:41.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.347+0000 7ff097266080 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d7a8f84540 0x55d7a8eb1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:41.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.347+0000 7ff097266080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7a8f7cd00 con 0x55d7a8ef8fa0 2023-11-06T02:12:41.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.347+0000 7ff097266080 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7a8f7cf30 con 0x55d7a8f84540 2023-11-06T02:12:41.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.347+0000 7ff097266080 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d7a8f7d1d0 con 0x55d7a8edc8f0 2023-11-06T02:12:41.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.347+0000 7ff09457f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d7a8ef8fa0 0x55d7a8f7f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:41.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.347+0000 7ff09457f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d7a8ef8fa0 0x55d7a8f7f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58762/0 (socket says 172.21.15.49:58762) 2023-11-06T02:12:41.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.347+0000 7ff09457f640 1 -- 172.21.15.49:0/2475378346 learned_addr learned my addr 172.21.15.49:0/2475378346 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:12:41.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.347+0000 7ff095581640 1 --2- 172.21.15.49:0/2475378346 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d7a8f84540 0x55d7a8eb1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:41.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.347+0000 7ff094d80640 1 --2- 172.21.15.49:0/2475378346 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d7a8edc8f0 0x55d7a8f7efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:41.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.347+0000 7ff09457f640 1 -- 172.21.15.49:0/2475378346 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d7a8edc8f0 msgr2=0x55d7a8f7efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:41.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.347+0000 7ff09457f640 1 --2- 172.21.15.49:0/2475378346 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55d7a8edc8f0 0x55d7a8f7efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:41.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.347+0000 7ff09457f640 1 -- 172.21.15.49:0/2475378346 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d7a8f84540 msgr2=0x55d7a8eb1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:41.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.347+0000 7ff09457f640 1 --2- 172.21.15.49:0/2475378346 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d7a8f84540 0x55d7a8eb1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:41.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.347+0000 7ff09457f640 1 -- 172.21.15.49:0/2475378346 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d7a8e1f190 con 0x55d7a8ef8fa0 2023-11-06T02:12:41.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.347+0000 7ff09457f640 1 --2- 172.21.15.49:0/2475378346 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d7a8ef8fa0 0x55d7a8f7f4f0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7ff07c017f10 tx=0x7ff07c019b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:41.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.347+0000 7ff09257b640 1 -- 172.21.15.49:0/2475378346 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff07c0c5650 con 0x55d7a8ef8fa0 2023-11-06T02:12:41.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.351+0000 7ff09257b640 1 -- 172.21.15.49:0/2475378346 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff07c0aa750 con 0x55d7a8ef8fa0 2023-11-06T02:12:41.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.351+0000 7ff09257b640 1 -- 172.21.15.49:0/2475378346 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff07c0c59b0 con 0x55d7a8ef8fa0 2023-11-06T02:12:41.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.351+0000 7ff097266080 1 -- 172.21.15.49:0/2475378346 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55d7a8e04ff0 con 0x55d7a8ef8fa0 2023-11-06T02:12:41.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.351+0000 7ff097266080 1 -- 172.21.15.49:0/2475378346 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d7a8df8430 con 0x55d7a8ef8fa0 2023-11-06T02:12:41.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.351+0000 7ff09257b640 1 -- 172.21.15.49:0/2475378346 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7ff07c0b1070 con 0x55d7a8ef8fa0 2023-11-06T02:12:41.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.351+0000 7ff09257b640 1 --2- 172.21.15.49:0/2475378346 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff074064440 0x7ff074066900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:41.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.351+0000 7ff09257b640 1 -- 172.21.15.49:0/2475378346 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6264+0+0 (secure 0 0 0) 0x7ff07c11e080 con 0x55d7a8ef8fa0 2023-11-06T02:12:41.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.351+0000 7ff097266080 1 --2- 172.21.15.49:0/2475378346 >> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] conn(0x55d7a8eb2340 0x55d7a8eaecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:41.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.351+0000 7ff097266080 1 -- 172.21.15.49:0/2475378346 --> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] -- osd_op(unknown.0.0:1 9.13 9:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68) v8 -- 0x55d7a901a910 con 0x55d7a8eb2340 2023-11-06T02:12:41.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.351+0000 7ff094d80640 1 --2- 172.21.15.49:0/2475378346 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff074064440 0x7ff074066900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:41.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.351+0000 7ff095581640 1 --2- 172.21.15.49:0/2475378346 >> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] conn(0x55d7a8eb2340 0x55d7a8eaecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:41.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.351+0000 7ff094d80640 1 --2- 172.21.15.49:0/2475378346 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff074064440 0x7ff074066900 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7ff084000f70 tx=0x7ff084006640 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:41.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.351+0000 7ff095581640 1 --2- 172.21.15.49:0/2475378346 >> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] conn(0x55d7a8eb2340 0x55d7a8eaecb0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:41.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.355+0000 7ff095581640 1 -- 172.21.15.49:0/2475378346 <== osd.7 v2:172.21.15.150:6816/38421 1 ==== osd_op_reply(1 mds0_inotable [writefull 0~34] v68'5 uv5 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7ff08c002940 con 0x55d7a8eb2340 2023-11-06T02:12:41.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.355+0000 7ff097266080 1 -- 172.21.15.49:0/2475378346 >> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] conn(0x55d7a8eb2340 msgr2=0x55d7a8eaecb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:41.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.355+0000 7ff097266080 1 --2- 172.21.15.49:0/2475378346 >> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] conn(0x55d7a8eb2340 0x55d7a8eaecb0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:41.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.355+0000 7ff097266080 1 -- 172.21.15.49:0/2475378346 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff074064440 msgr2=0x7ff074066900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:41.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.355+0000 7ff097266080 1 --2- 172.21.15.49:0/2475378346 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff074064440 0x7ff074066900 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7ff084000f70 tx=0x7ff084006640 comp rx=0 tx=0).stop 2023-11-06T02:12:41.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.355+0000 7ff097266080 1 -- 172.21.15.49:0/2475378346 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d7a8ef8fa0 msgr2=0x55d7a8f7f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:41.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.355+0000 7ff097266080 1 --2- 172.21.15.49:0/2475378346 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d7a8ef8fa0 0x55d7a8f7f4f0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7ff07c017f10 tx=0x7ff07c019b40 comp rx=0 tx=0).stop 2023-11-06T02:12:41.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.355+0000 7ff095581640 1 -- 172.21.15.49:0/2475378346 reap_dead start 2023-11-06T02:12:41.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.355+0000 7ff097266080 1 -- 172.21.15.49:0/2475378346 shutdown_connections 2023-11-06T02:12:41.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.355+0000 7ff097266080 1 -- 172.21.15.49:0/2475378346 >> 172.21.15.49:0/2475378346 conn(0x55d7a8edaf90 msgr2=0x55d7a8edb370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:12:41.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.355+0000 7ff097266080 1 -- 172.21.15.49:0/2475378346 shutdown_connections 2023-11-06T02:12:41.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.355+0000 7ff097266080 1 -- 172.21.15.49:0/2475378346 wait complete. 2023-11-06T02:12:41.365 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-11-06T02:12:41.533 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.535+0000 7f2dd6090640 1 Processor -- start 2023-11-06T02:12:41.534 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.535+0000 7f2dd6090640 1 -- start start 2023-11-06T02:12:41.535 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.535+0000 7f2dd6090640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f2dd015a990 0x7f2dd015dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:41.535 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.535+0000 7f2dd6090640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2dd006c5a0 con 0x7f2dd01595a0 2023-11-06T02:12:41.535 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.535+0000 7f2dd6090640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2dd006c880 con 0x7f2dd00b0220 2023-11-06T02:12:41.535 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.535+0000 7f2dd6090640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2dd015e250 con 0x7f2dd015a990 2023-11-06T02:12:41.535 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.535+0000 7f2dceffd640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f2dd015a990 0x7f2dd015dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:41.536 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.535+0000 7f2dcf7fe640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f2dd01595a0 0x7f2dd006d640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:42678/0 (socket says 172.21.15.49:42678) 2023-11-06T02:12:41.536 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.535+0000 7f2dceffd640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f2dd015a990 0x7f2dd015dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58766/0 (socket says 172.21.15.49:58766) 2023-11-06T02:12:41.536 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.535+0000 7f2dcf7fe640 1 -- 172.21.15.49:0/565233206 learned_addr learned my addr 172.21.15.49:0/565233206 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:12:41.537 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.535+0000 7f2dceffd640 1 -- 172.21.15.49:0/565233206 >> v1:172.21.15.150:6789/0 conn(0x7f2dd00b0220 legacy=0x7f2dd00b0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:41.537 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.535+0000 7f2dceffd640 1 -- 172.21.15.49:0/565233206 >> v1:172.21.15.49:6789/0 conn(0x7f2dd01595a0 legacy=0x7f2dd006d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:41.537 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.535+0000 7f2dceffd640 1 -- 172.21.15.49:0/565233206 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dd006cb60 con 0x7f2dd015a990 2023-11-06T02:12:41.538 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.539+0000 7f2dceffd640 1 --2- 172.21.15.49:0/565233206 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2dd015a990 0x7f2dd015dd10 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f2dc40097c0 tx=0x7f2dc401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc7a659abd62d064 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:41.538 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.539+0000 7f2dce7fc640 1 -- 172.21.15.49:0/565233206 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc40ab650 con 0x7f2dd015a990 2023-11-06T02:12:41.538 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.539+0000 7f2dce7fc640 1 -- 172.21.15.49:0/565233206 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dc40ab7b0 con 0x7f2dd015a990 2023-11-06T02:12:41.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.539+0000 7f2dce7fc640 1 -- 172.21.15.49:0/565233206 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc40abb40 con 0x7f2dd015a990 2023-11-06T02:12:41.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.539+0000 7f2dd6090640 1 -- 172.21.15.49:0/565233206 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2dd015a990 msgr2=0x7f2dd015dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:41.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.539+0000 7f2dd6090640 1 --2- 172.21.15.49:0/565233206 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2dd015a990 0x7f2dd015dd10 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f2dc40097c0 tx=0x7f2dc401c450 comp rx=0 tx=0).stop 2023-11-06T02:12:41.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.539+0000 7f2dd6090640 1 -- 172.21.15.49:0/565233206 shutdown_connections 2023-11-06T02:12:41.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.539+0000 7f2dd6090640 1 --2- 172.21.15.49:0/565233206 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2dd015a990 0x7f2dd015dd10 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:41.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.539+0000 7f2dd6090640 1 -- 172.21.15.49:0/565233206 >> 172.21.15.49:0/565233206 conn(0x7f2dd006a370 msgr2=0x7f2dd00ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:12:41.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.539+0000 7f2dd6090640 1 -- 172.21.15.49:0/565233206 shutdown_connections 2023-11-06T02:12:41.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.539+0000 7f2dd6090640 1 -- 172.21.15.49:0/565233206 wait complete. 2023-11-06T02:12:41.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.539+0000 7f2dd6090640 1 Processor -- start 2023-11-06T02:12:41.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.539+0000 7f2dd6090640 1 -- start start 2023-11-06T02:12:41.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.543+0000 7f2dd6090640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2dd00b0220 0x7f2dd014b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:41.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.543+0000 7f2dd6090640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2dd01595a0 0x7f2dd0147260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:41.545 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.543+0000 7f2dd6090640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2dd015a990 0x7f2dd01499b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:41.545 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.543+0000 7f2dd6090640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd00c36d0 con 0x7f2dd015a990 2023-11-06T02:12:41.545 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.543+0000 7f2dd6090640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd014ba30 con 0x7f2dd01595a0 2023-11-06T02:12:41.545 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.543+0000 7f2dd6090640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd01477a0 con 0x7f2dd00b0220 2023-11-06T02:12:41.545 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.543+0000 7f2dcffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2dd015a990 0x7f2dd01499b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:41.546 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.543+0000 7f2dcffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2dd015a990 0x7f2dd01499b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58770/0 (socket says 172.21.15.49:58770) 2023-11-06T02:12:41.546 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.543+0000 7f2dcffff640 1 -- 172.21.15.49:0/2627985156 learned_addr learned my addr 172.21.15.49:0/2627985156 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:12:41.546 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.543+0000 7f2dceffd640 1 --2- 172.21.15.49:0/2627985156 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2dd01595a0 0x7f2dd0147260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:41.546 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.543+0000 7f2dcf7fe640 1 --2- 172.21.15.49:0/2627985156 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2dd00b0220 0x7f2dd014b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:41.546 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.543+0000 7f2dcffff640 1 -- 172.21.15.49:0/2627985156 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2dd00b0220 msgr2=0x7f2dd014b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:41.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.543+0000 7f2dcffff640 1 --2- 172.21.15.49:0/2627985156 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2dd00b0220 0x7f2dd014b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:41.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.543+0000 7f2dcffff640 1 -- 172.21.15.49:0/2627985156 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2dd01595a0 msgr2=0x7f2dd0147260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:41.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.543+0000 7f2dcffff640 1 --2- 172.21.15.49:0/2627985156 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2dd01595a0 0x7f2dd0147260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:41.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.543+0000 7f2dcffff640 1 -- 172.21.15.49:0/2627985156 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dc4009420 con 0x7f2dd015a990 2023-11-06T02:12:41.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.543+0000 7f2dcf7fe640 1 --2- 172.21.15.49:0/2627985156 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2dd00b0220 0x7f2dd014b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:12:41.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.543+0000 7f2dcffff640 1 --2- 172.21.15.49:0/2627985156 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2dd015a990 0x7f2dd01499b0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f2dc800c770 tx=0x7f2dc800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:41.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.543+0000 7f2db7fff640 1 -- 172.21.15.49:0/2627985156 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc8003b80 con 0x7f2dd015a990 2023-11-06T02:12:41.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.543+0000 7f2db7fff640 1 -- 172.21.15.49:0/2627985156 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dc8003ce0 con 0x7f2dd015a990 2023-11-06T02:12:41.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.543+0000 7f2dd6090640 1 -- 172.21.15.49:0/2627985156 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dd015b8e0 con 0x7f2dd015a990 2023-11-06T02:12:41.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.543+0000 7f2db7fff640 1 -- 172.21.15.49:0/2627985156 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc800aa90 con 0x7f2dd015a990 2023-11-06T02:12:41.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.543+0000 7f2dd6090640 1 -- 172.21.15.49:0/2627985156 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dd015bea0 con 0x7f2dd015a990 2023-11-06T02:12:41.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.543+0000 7f2dd6090640 1 -- 172.21.15.49:0/2627985156 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dd0074040 con 0x7f2dd015a990 2023-11-06T02:12:41.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.543+0000 7f2db7fff640 1 -- 172.21.15.49:0/2627985156 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f2dc800abf0 con 0x7f2dd015a990 2023-11-06T02:12:41.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.543+0000 7f2db7fff640 1 --2- 172.21.15.49:0/2627985156 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2dbc065a60 0x7f2dbc067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:41.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.543+0000 7f2db7fff640 1 -- 172.21.15.49:0/2627985156 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6264+0+0 (secure 0 0 0) 0x7f2dc80831f0 con 0x7f2dd015a990 2023-11-06T02:12:41.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.543+0000 7f2dcf7fe640 1 --2- 172.21.15.49:0/2627985156 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2dbc065a60 0x7f2dbc067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:41.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.547+0000 7f2dcf7fe640 1 --2- 172.21.15.49:0/2627985156 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2dbc065a60 0x7f2dbc067f20 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0005890 tx=0x7f2dc0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:41.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.547+0000 7f2db7fff640 1 -- 172.21.15.49:0/2627985156 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f2dc8050980 con 0x7f2dd015a990 2023-11-06T02:12:41.743 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:41.743+0000 7f2dd6090640 1 -- 172.21.15.49:0/2627985156 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f2dd00cd5b0 con 0x7f2dd015a990 2023-11-06T02:12:42.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:42.047+0000 7f2db7fff640 1 -- 172.21.15.49:0/2627985156 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v32) v1 ==== 162+0+0 (secure 0 0 0) 0x7f2dc8053fa0 con 0x7f2dd015a990 2023-11-06T02:12:42.048 INFO:teuthology.orchestra.run.smithi049.stderr:cephfs marked joinable; MDS may join as newly active. 2023-11-06T02:12:42.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:42.051+0000 7f2dd6090640 1 -- 172.21.15.49:0/2627985156 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2dbc065a60 msgr2=0x7f2dbc067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:42.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:42.051+0000 7f2dd6090640 1 --2- 172.21.15.49:0/2627985156 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2dbc065a60 0x7f2dbc067f20 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0005890 tx=0x7f2dc0009040 comp rx=0 tx=0).stop 2023-11-06T02:12:42.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:42.051+0000 7f2dd6090640 1 -- 172.21.15.49:0/2627985156 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2dd015a990 msgr2=0x7f2dd01499b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:42.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:42.051+0000 7f2dd6090640 1 --2- 172.21.15.49:0/2627985156 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2dd015a990 0x7f2dd01499b0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f2dc800c770 tx=0x7f2dc800cc30 comp rx=0 tx=0).stop 2023-11-06T02:12:42.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:42.051+0000 7f2dd6090640 1 -- 172.21.15.49:0/2627985156 shutdown_connections 2023-11-06T02:12:42.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:42.051+0000 7f2dd6090640 1 --2- 172.21.15.49:0/2627985156 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2dbc065a60 0x7f2dbc067f20 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:42.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:42.051+0000 7f2dd6090640 1 --2- 172.21.15.49:0/2627985156 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2dd015a990 0x7f2dd01499b0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:42.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:42.051+0000 7f2dd6090640 1 --2- 172.21.15.49:0/2627985156 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2dd01595a0 0x7f2dd0147260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:42.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:42.051+0000 7f2dd6090640 1 --2- 172.21.15.49:0/2627985156 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2dd00b0220 0x7f2dd014b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:42.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:42.051+0000 7f2dd6090640 1 -- 172.21.15.49:0/2627985156 >> 172.21.15.49:0/2627985156 conn(0x7f2dd006a370 msgr2=0x7f2dd00b0650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:12:42.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:42.051+0000 7f2dd6090640 1 -- 172.21.15.49:0/2627985156 shutdown_connections 2023-11-06T02:12:42.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:42.051+0000 7f2dd6090640 1 -- 172.21.15.49:0/2627985156 wait complete. 2023-11-06T02:12:42.077 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T02:12:42.196 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:42.195+0000 7fe6a2418640 1 Processor -- start 2023-11-06T02:12:42.197 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:42.195+0000 7fe6a2418640 1 -- start start 2023-11-06T02:12:42.198 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:42.199+0000 7fe6a2418640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe69c15cda0 0x7fe69c0cf110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:42.198 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:42.199+0000 7fe6a2418640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe69c06c7a0 0x7fe69c0cc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:42.198 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:42.199+0000 7fe6a2418640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe69c0cec80 con 0x7fe69c0b22d0 2023-11-06T02:12:42.199 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:42.199+0000 7fe6a2418640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe69c15d180 con 0x7fe69c06c7a0 2023-11-06T02:12:42.199 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:42.199+0000 7fe6a2418640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe69c0ce420 con 0x7fe69c15cda0 2023-11-06T02:12:42.199 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:42.199+0000 7fe69b7fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe69c06c7a0 0x7fe69c0cc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:42.199 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:42.199+0000 7fe6a09b5640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fe69c0b22d0 0x7fe69c06fe20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:42694/0 (socket says 172.21.15.49:42694) 2023-11-06T02:12:42.199 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:42.199+0000 7fe6a09b5640 1 -- 172.21.15.49:0/1700598526 learned_addr learned my addr 172.21.15.49:0/1700598526 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:12:42.200 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:42.199+0000 7fe69bfff640 1 --2- 172.21.15.49:0/1700598526 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe69c15cda0 0x7fe69c0cf110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:42.200 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:42.199+0000 7fe69affd640 1 -- 172.21.15.49:0/1700598526 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1823251371 0 0) 0x7fe69c0cec80 con 0x7fe69c0b22d0 2023-11-06T02:12:42.200 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:42.199+0000 7fe69affd640 1 -- 172.21.15.49:0/1700598526 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe688003530 con 0x7fe69c0b22d0 2023-11-06T02:12:42.200 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:42.199+0000 7fe69b7fe640 1 -- 172.21.15.49:0/1700598526 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe69c15cda0 msgr2=0x7fe69c0cf110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:42.200 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:42.199+0000 7fe69b7fe640 1 --2- 172.21.15.49:0/1700598526 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe69c15cda0 0x7fe69c0cf110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:42.201 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:42.199+0000 7fe69b7fe640 1 -- 172.21.15.49:0/1700598526 >> v1:172.21.15.49:6789/0 conn(0x7fe69c0b22d0 legacy=0x7fe69c06fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:42.201 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:42.199+0000 7fe69b7fe640 1 -- 172.21.15.49:0/1700598526 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe69c161520 con 0x7fe69c06c7a0 2023-11-06T02:12:42.202 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:42.199+0000 7fe69b7fe640 1 --2- 172.21.15.49:0/1700598526 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe69c06c7a0 0x7fe69c0cc8e0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fe68c0097c0 tx=0x7fe68c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d04aae606cdda5f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:42.203 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:42.199+0000 7fe69affd640 1 -- 172.21.15.49:0/1700598526 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe68c0ab600 con 0x7fe69c06c7a0 2023-11-06T02:12:42.203 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:42.199+0000 7fe69affd640 1 -- 172.21.15.49:0/1700598526 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe68c0a7050 con 0x7fe69c06c7a0 2023-11-06T02:12:42.203 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:42.199+0000 7fe69affd640 1 -- 172.21.15.49:0/1700598526 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe68c0ab980 con 0x7fe69c06c7a0 2023-11-06T02:12:42.203 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:42.203+0000 7fe6a2418640 1 -- 172.21.15.49:0/1700598526 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe69c06c7a0 msgr2=0x7fe69c0cc8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:42.203 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:42.203+0000 7fe6a2418640 1 --2- 172.21.15.49:0/1700598526 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe69c06c7a0 0x7fe69c0cc8e0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fe68c0097c0 tx=0x7fe68c01c480 comp rx=0 tx=0).stop 2023-11-06T02:12:42.204 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:42.203+0000 7fe6a2418640 1 -- 172.21.15.49:0/1700598526 shutdown_connections 2023-11-06T02:12:42.204 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:42.203+0000 7fe6a2418640 1 --2- 172.21.15.49:0/1700598526 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe69c06c7a0 0x7fe69c0cc8e0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:42.204 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:42.203+0000 7fe6a2418640 1 --2- 172.21.15.49:0/1700598526 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe69c15cda0 0x7fe69c0cf110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:42.204 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:42.203+0000 7fe6a2418640 1 -- 172.21.15.49:0/1700598526 >> 172.21.15.49:0/1700598526 conn(0x7fe69c0cd2d0 msgr2=0x7fe69c0cd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:12:42.204 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:42.203+0000 7fe6a2418640 1 -- 172.21.15.49:0/1700598526 shutdown_connections 2023-11-06T02:12:42.204 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:42.203+0000 7fe6a2418640 1 -- 172.21.15.49:0/1700598526 wait complete. 2023-11-06T02:12:42.205 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:42.203+0000 7fe6a2418640 1 Processor -- start 2023-11-06T02:12:42.205 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:42.203+0000 7fe6a2418640 1 -- start start 2023-11-06T02:12:42.207 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:42.203+0000 7fe6a2418640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe69c06c7a0 0x7fe69c0b3470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:42.207 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:42.203+0000 7fe6a2418640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe69c0b22d0 0x7fe69c15c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:42.208 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:42.203+0000 7fe6a2418640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe69c15cda0 0x7fe69c0b4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:42.208 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:42.203+0000 7fe6a2418640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe69c152d30 con 0x7fe69c0b22d0 2023-11-06T02:12:42.208 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:42.203+0000 7fe6a2418640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe69c0b3a50 con 0x7fe69c15cda0 2023-11-06T02:12:42.208 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:42.203+0000 7fe6a2418640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe69c0b3d30 con 0x7fe69c06c7a0 2023-11-06T02:12:42.208 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:42.203+0000 7fe69b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe69c0b22d0 0x7fe69c15c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:42.209 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:42.203+0000 7fe69b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe69c0b22d0 0x7fe69c15c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58782/0 (socket says 172.21.15.49:58782) 2023-11-06T02:12:42.209 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:42.203+0000 7fe69b7fe640 1 -- 172.21.15.49:0/1923742971 learned_addr learned my addr 172.21.15.49:0/1923742971 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:12:42.209 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:42.203+0000 7fe69b7fe640 1 -- 172.21.15.49:0/1923742971 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe69c06c7a0 msgr2=0x7fe69c0b3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:42.209 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:42.203+0000 7fe69bfff640 1 --2- 172.21.15.49:0/1923742971 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe69c06c7a0 0x7fe69c0b3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:42.209 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:42.203+0000 7fe6a09b5640 1 --2- 172.21.15.49:0/1923742971 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe69c15cda0 0x7fe69c0b4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:42.210 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:42.203+0000 7fe69b7fe640 1 --2- 172.21.15.49:0/1923742971 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe69c06c7a0 0x7fe69c0b3470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:42.210 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:42.203+0000 7fe69b7fe640 1 -- 172.21.15.49:0/1923742971 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe69c15cda0 msgr2=0x7fe69c0b4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:42.210 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:42.203+0000 7fe69b7fe640 1 --2- 172.21.15.49:0/1923742971 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe69c15cda0 0x7fe69c0b4af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:42.210 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:42.203+0000 7fe69b7fe640 1 -- 172.21.15.49:0/1923742971 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe68c009420 con 0x7fe69c0b22d0 2023-11-06T02:12:42.210 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:42.203+0000 7fe69b7fe640 1 --2- 172.21.15.49:0/1923742971 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe69c0b22d0 0x7fe69c15c1b0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fe68c0ae040 tx=0x7fe68c019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:42.211 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:42.203+0000 7fe698ff9640 1 -- 172.21.15.49:0/1923742971 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe68c0b4840 con 0x7fe69c0b22d0 2023-11-06T02:12:42.211 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:42.203+0000 7fe698ff9640 1 -- 172.21.15.49:0/1923742971 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe68c0b49a0 con 0x7fe69c0b22d0 2023-11-06T02:12:42.211 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:42.203+0000 7fe6a2418640 1 -- 172.21.15.49:0/1923742971 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe69c0b4010 con 0x7fe69c0b22d0 2023-11-06T02:12:42.211 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:42.203+0000 7fe698ff9640 1 -- 172.21.15.49:0/1923742971 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe68c0b4d50 con 0x7fe69c0b22d0 2023-11-06T02:12:42.211 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:42.203+0000 7fe6a2418640 1 -- 172.21.15.49:0/1923742971 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe69c0b45d0 con 0x7fe69c0b22d0 2023-11-06T02:12:42.211 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:42.207+0000 7fe6a2418640 1 -- 172.21.15.49:0/1923742971 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe69c0b47c0 con 0x7fe69c0b22d0 2023-11-06T02:12:42.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:42.207+0000 7fe698ff9640 1 -- 172.21.15.49:0/1923742971 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fe68c0b2040 con 0x7fe69c0b22d0 2023-11-06T02:12:42.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:42.207+0000 7fe698ff9640 1 --2- 172.21.15.49:0/1923742971 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe67c065a60 0x7fe67c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:42.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:42.207+0000 7fe698ff9640 1 -- 172.21.15.49:0/1923742971 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6264+0+0 (secure 0 0 0) 0x7fe68c11d6d0 con 0x7fe69c0b22d0 2023-11-06T02:12:42.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:42.207+0000 7fe69bfff640 1 --2- 172.21.15.49:0/1923742971 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe67c065a60 0x7fe67c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:42.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:42.207+0000 7fe69bfff640 1 --2- 172.21.15.49:0/1923742971 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe67c065a60 0x7fe67c067f20 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fe69c066f80 tx=0x7fe684008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:42.213 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:42.211+0000 7fe698ff9640 1 -- 172.21.15.49:0/1923742971 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fe68c0eadd0 con 0x7fe69c0b22d0 2023-11-06T02:12:42.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:42.399+0000 7fe6a2418640 1 -- 172.21.15.49:0/1923742971 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe69c06c580 con 0x7fe69c0b22d0 2023-11-06T02:12:42.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:42.399+0000 7fe698ff9640 1 -- 172.21.15.49:0/1923742971 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4656 (secure 0 0 0) 0x7fe68c0b0020 con 0x7fe69c0b22d0 2023-11-06T02:12:42.399 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:12:42.399 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5635,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4170291905","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":4170291905},{"type":"v1","addr":"172.21.15.150:6835","nonce":4170291905}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5638,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/2719458937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":2719458937},{"type":"v1","addr":"172.21.15.150:6837","nonce":2719458937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":6339,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/2310338375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2310338375},{"type":"v1","addr":"172.21.15.49:6833","nonce":2310338375}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T02:11:30.091070+0000","modified":"2023-11-06T02:12:42.053599+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":68,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6807},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6807":{"gid":6807,"name":"c","rank":0,"incarnation":33,"state":"up:replay","state_seq":1,"addr":"172.21.15.49:6835/3698847494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3698847494},{"type":"v1","addr":"172.21.15.49:6835","nonce":3698847494}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-06T02:12:42.400 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 33 2023-11-06T02:12:42.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:42.403+0000 7fe6a2418640 1 -- 172.21.15.49:0/1923742971 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe67c065a60 msgr2=0x7fe67c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:42.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:42.403+0000 7fe6a2418640 1 --2- 172.21.15.49:0/1923742971 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe67c065a60 0x7fe67c067f20 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fe69c066f80 tx=0x7fe684008040 comp rx=0 tx=0).stop 2023-11-06T02:12:42.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:42.403+0000 7fe6a2418640 1 -- 172.21.15.49:0/1923742971 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe69c0b22d0 msgr2=0x7fe69c15c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:42.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:42.403+0000 7fe6a2418640 1 --2- 172.21.15.49:0/1923742971 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe69c0b22d0 0x7fe69c15c1b0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fe68c0ae040 tx=0x7fe68c019220 comp rx=0 tx=0).stop 2023-11-06T02:12:42.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:42.403+0000 7fe6a2418640 1 -- 172.21.15.49:0/1923742971 shutdown_connections 2023-11-06T02:12:42.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:42.403+0000 7fe6a2418640 1 --2- 172.21.15.49:0/1923742971 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe67c065a60 0x7fe67c067f20 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:42.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:42.403+0000 7fe6a2418640 1 --2- 172.21.15.49:0/1923742971 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe69c0b22d0 0x7fe69c15c1b0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:42.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:42.403+0000 7fe6a2418640 1 --2- 172.21.15.49:0/1923742971 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe69c15cda0 0x7fe69c0b4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:42.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:42.403+0000 7fe6a2418640 1 --2- 172.21.15.49:0/1923742971 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe69c06c7a0 0x7fe69c0b3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:42.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:42.403+0000 7fe6a2418640 1 -- 172.21.15.49:0/1923742971 >> 172.21.15.49:0/1923742971 conn(0x7fe69c0cd2d0 msgr2=0x7fe69c160770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:12:42.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:42.403+0000 7fe6a2418640 1 -- 172.21.15.49:0/1923742971 shutdown_connections 2023-11-06T02:12:42.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:42.403+0000 7fe6a2418640 1 -- 172.21.15.49:0/1923742971 wait complete. 2023-11-06T02:12:42.820 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 33, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-06T02:11:30.091070+0000', 'modified': '2023-11-06T02:12:42.053599+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 68, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6807}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6807': {'gid': 6807, 'name': 'c', 'rank': 0, 'incarnation': 33, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.49:6835/3698847494', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6834', 'nonce': 3698847494}, {'type': 'v1', 'addr': '172.21.15.49:6835', 'nonce': 3698847494}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-06T02:12:42.821 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6807:up:replay 2023-11-06T02:12:43.822 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T02:12:43.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:43.959+0000 7fa26ffe4640 1 Processor -- start 2023-11-06T02:12:43.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:43.959+0000 7fa26ffe4640 1 -- start start 2023-11-06T02:12:43.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:43.959+0000 7fa26ffe4640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa26815aab0 0x7fa2680cb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:43.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:43.959+0000 7fa26ffe4640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fa2680cb970 0x7fa2680cd280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:43.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:43.959+0000 7fa26ffe4640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2680ce3f0 con 0x7fa2680b2250 2023-11-06T02:12:43.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:43.959+0000 7fa26ffe4640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa26815ae90 con 0x7fa26815aab0 2023-11-06T02:12:43.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:43.959+0000 7fa26ffe4640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2680cbdf0 con 0x7fa2680cb970 2023-11-06T02:12:43.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:43.959+0000 7fa26e581640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fa2680b2250 0x7fa26815e510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:42706/0 (socket says 172.21.15.49:42706) 2023-11-06T02:12:43.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:43.959+0000 7fa26dd80640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa26815aab0 0x7fa2680cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:43.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:43.959+0000 7fa26e581640 1 -- 172.21.15.49:0/1814626196 learned_addr learned my addr 172.21.15.49:0/1814626196 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:12:43.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:43.959+0000 7fa26d57f640 1 --2- 172.21.15.49:0/1814626196 >> v2:172.21.15.150:3300/0 conn(0x7fa2680cb970 0x7fa2680cd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:43.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:43.959+0000 7fa26cd7e640 1 -- 172.21.15.49:0/1814626196 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2023566299 0 0) 0x7fa2680ce3f0 con 0x7fa2680b2250 2023-11-06T02:12:43.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:43.959+0000 7fa26cd7e640 1 -- 172.21.15.49:0/1814626196 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa254003530 con 0x7fa2680b2250 2023-11-06T02:12:43.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:43.959+0000 7fa26dd80640 1 -- 172.21.15.49:0/1814626196 >> v2:172.21.15.150:3300/0 conn(0x7fa2680cb970 msgr2=0x7fa2680cd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:43.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:43.959+0000 7fa26dd80640 1 --2- 172.21.15.49:0/1814626196 >> v2:172.21.15.150:3300/0 conn(0x7fa2680cb970 0x7fa2680cd280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:43.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:43.959+0000 7fa26dd80640 1 -- 172.21.15.49:0/1814626196 >> v1:172.21.15.49:6789/0 conn(0x7fa2680b2250 legacy=0x7fa26815e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:43.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:43.959+0000 7fa26dd80640 1 -- 172.21.15.49:0/1814626196 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2680cc0d0 con 0x7fa26815aab0 2023-11-06T02:12:43.964 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:43.963+0000 7fa26dd80640 1 --2- 172.21.15.49:0/1814626196 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa26815aab0 0x7fa2680cb430 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fa2680d18c0 tx=0x7fa26401c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e933e4e8f727ecb server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:43.964 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:43.963+0000 7fa26cd7e640 1 -- 172.21.15.49:0/1814626196 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2640ab600 con 0x7fa26815aab0 2023-11-06T02:12:43.964 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:43.963+0000 7fa26cd7e640 1 -- 172.21.15.49:0/1814626196 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2640a7050 con 0x7fa26815aab0 2023-11-06T02:12:43.964 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:43.963+0000 7fa26cd7e640 1 -- 172.21.15.49:0/1814626196 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2640ab980 con 0x7fa26815aab0 2023-11-06T02:12:43.964 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:43.963+0000 7fa26ffe4640 1 -- 172.21.15.49:0/1814626196 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa26815aab0 msgr2=0x7fa2680cb430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:43.965 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:43.963+0000 7fa26ffe4640 1 --2- 172.21.15.49:0/1814626196 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa26815aab0 0x7fa2680cb430 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fa2680d18c0 tx=0x7fa26401c380 comp rx=0 tx=0).stop 2023-11-06T02:12:43.965 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:43.963+0000 7fa26ffe4640 1 -- 172.21.15.49:0/1814626196 shutdown_connections 2023-11-06T02:12:43.965 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:43.963+0000 7fa26ffe4640 1 --2- 172.21.15.49:0/1814626196 >> v2:172.21.15.150:3300/0 conn(0x7fa2680cb970 0x7fa2680cd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:43.965 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:43.963+0000 7fa26ffe4640 1 --2- 172.21.15.49:0/1814626196 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa26815aab0 0x7fa2680cb430 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:43.965 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:43.963+0000 7fa26ffe4640 1 -- 172.21.15.49:0/1814626196 >> 172.21.15.49:0/1814626196 conn(0x7fa268089fd0 msgr2=0x7fa268099c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:12:43.966 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:43.963+0000 7fa26ffe4640 1 -- 172.21.15.49:0/1814626196 shutdown_connections 2023-11-06T02:12:43.966 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:43.963+0000 7fa26ffe4640 1 -- 172.21.15.49:0/1814626196 wait complete. 2023-11-06T02:12:43.966 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:43.963+0000 7fa26ffe4640 1 Processor -- start 2023-11-06T02:12:43.966 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:43.963+0000 7fa26ffe4640 1 -- start start 2023-11-06T02:12:43.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:43.963+0000 7fa26ffe4640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2680b2250 0x7fa268150a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:43.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:43.963+0000 7fa26ffe4640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa2680cb970 0x7fa2680ade10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:43.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:43.967+0000 7fa26ffe4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2680ae350 0x7fa26806f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:43.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:43.967+0000 7fa26ffe4640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2680c4460 con 0x7fa2680ae350 2023-11-06T02:12:43.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:43.967+0000 7fa26ffe4640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2680ae7f0 con 0x7fa2680b2250 2023-11-06T02:12:43.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:43.967+0000 7fa26ffe4640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa26815b0b0 con 0x7fa2680cb970 2023-11-06T02:12:43.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:43.967+0000 7fa26e581640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2680ae350 0x7fa26806f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:43.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:43.967+0000 7fa26e581640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2680ae350 0x7fa26806f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58796/0 (socket says 172.21.15.49:58796) 2023-11-06T02:12:43.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:43.967+0000 7fa26e581640 1 -- 172.21.15.49:0/2106502941 learned_addr learned my addr 172.21.15.49:0/2106502941 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:12:43.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:43.967+0000 7fa26d57f640 1 --2- 172.21.15.49:0/2106502941 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa2680cb970 0x7fa2680ade10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:43.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:43.967+0000 7fa26dd80640 1 --2- 172.21.15.49:0/2106502941 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2680b2250 0x7fa268150a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:43.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:43.967+0000 7fa26e581640 1 -- 172.21.15.49:0/2106502941 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa2680cb970 msgr2=0x7fa2680ade10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:43.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:43.967+0000 7fa26e581640 1 --2- 172.21.15.49:0/2106502941 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa2680cb970 0x7fa2680ade10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:43.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:43.967+0000 7fa26e581640 1 -- 172.21.15.49:0/2106502941 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2680b2250 msgr2=0x7fa268150a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:43.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:43.967+0000 7fa26e581640 1 --2- 172.21.15.49:0/2106502941 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2680b2250 0x7fa268150a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:43.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:43.967+0000 7fa26e581640 1 -- 172.21.15.49:0/2106502941 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2640093b0 con 0x7fa2680ae350 2023-11-06T02:12:43.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:43.967+0000 7fa26e581640 1 --2- 172.21.15.49:0/2106502941 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2680ae350 0x7fa26806f060 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fa26000caa0 tx=0x7fa26000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:43.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:43.967+0000 7fa25e7fc640 1 -- 172.21.15.49:0/2106502941 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa26000ac00 con 0x7fa2680ae350 2023-11-06T02:12:43.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:43.967+0000 7fa25e7fc640 1 -- 172.21.15.49:0/2106502941 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa26000ad60 con 0x7fa2680ae350 2023-11-06T02:12:43.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:43.967+0000 7fa25e7fc640 1 -- 172.21.15.49:0/2106502941 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa260012620 con 0x7fa2680ae350 2023-11-06T02:12:43.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:43.967+0000 7fa26ffe4640 1 -- 172.21.15.49:0/2106502941 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa26815b390 con 0x7fa2680ae350 2023-11-06T02:12:43.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:43.967+0000 7fa26ffe4640 1 -- 172.21.15.49:0/2106502941 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa26815b950 con 0x7fa2680ae350 2023-11-06T02:12:43.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:43.967+0000 7fa26ffe4640 1 -- 172.21.15.49:0/2106502941 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa26815bb70 con 0x7fa2680ae350 2023-11-06T02:12:43.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:43.967+0000 7fa25e7fc640 1 -- 172.21.15.49:0/2106502941 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fa260007070 con 0x7fa2680ae350 2023-11-06T02:12:43.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:43.967+0000 7fa25e7fc640 1 --2- 172.21.15.49:0/2106502941 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa24c065a90 0x7fa24c067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:43.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:43.967+0000 7fa25e7fc640 1 -- 172.21.15.49:0/2106502941 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6264+0+0 (secure 0 0 0) 0x7fa260083250 con 0x7fa2680ae350 2023-11-06T02:12:43.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:43.967+0000 7fa26dd80640 1 --2- 172.21.15.49:0/2106502941 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa24c065a90 0x7fa24c067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:43.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:43.971+0000 7fa26dd80640 1 --2- 172.21.15.49:0/2106502941 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa24c065a90 0x7fa24c067f50 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fa2640ad000 tx=0x7fa2640a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:43.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:43.971+0000 7fa25e7fc640 1 -- 172.21.15.49:0/2106502941 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fa260050a70 con 0x7fa2680ae350 2023-11-06T02:12:44.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:44.167+0000 7fa26ffe4640 1 -- 172.21.15.49:0/2106502941 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa2680b03e0 con 0x7fa2680ae350 2023-11-06T02:12:44.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:44.167+0000 7fa25e7fc640 1 -- 172.21.15.49:0/2106502941 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4656 (secure 0 0 0) 0x7fa260054090 con 0x7fa2680ae350 2023-11-06T02:12:44.167 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:12:44.167 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5635,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4170291905","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":4170291905},{"type":"v1","addr":"172.21.15.150:6835","nonce":4170291905}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5638,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/2719458937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":2719458937},{"type":"v1","addr":"172.21.15.150:6837","nonce":2719458937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":6339,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/2310338375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2310338375},{"type":"v1","addr":"172.21.15.49:6833","nonce":2310338375}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T02:11:30.091070+0000","modified":"2023-11-06T02:12:44.061318+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":68,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6807},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6807":{"gid":6807,"name":"c","rank":0,"incarnation":33,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.49:6835/3698847494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3698847494},{"type":"v1","addr":"172.21.15.49:6835","nonce":3698847494}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-06T02:12:44.167 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 35 2023-11-06T02:12:44.169 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:44.167+0000 7fa26ffe4640 1 -- 172.21.15.49:0/2106502941 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa24c065a90 msgr2=0x7fa24c067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:44.169 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:44.167+0000 7fa26ffe4640 1 --2- 172.21.15.49:0/2106502941 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa24c065a90 0x7fa24c067f50 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fa2640ad000 tx=0x7fa2640a6000 comp rx=0 tx=0).stop 2023-11-06T02:12:44.169 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:44.167+0000 7fa26ffe4640 1 -- 172.21.15.49:0/2106502941 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2680ae350 msgr2=0x7fa26806f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:44.169 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:44.167+0000 7fa26ffe4640 1 --2- 172.21.15.49:0/2106502941 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2680ae350 0x7fa26806f060 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fa26000caa0 tx=0x7fa26000cf60 comp rx=0 tx=0).stop 2023-11-06T02:12:44.169 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:44.167+0000 7fa26ffe4640 1 -- 172.21.15.49:0/2106502941 shutdown_connections 2023-11-06T02:12:44.170 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:44.167+0000 7fa26ffe4640 1 --2- 172.21.15.49:0/2106502941 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa24c065a90 0x7fa24c067f50 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:44.170 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:44.167+0000 7fa26ffe4640 1 --2- 172.21.15.49:0/2106502941 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2680ae350 0x7fa26806f060 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:44.170 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:44.167+0000 7fa26ffe4640 1 --2- 172.21.15.49:0/2106502941 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa2680cb970 0x7fa2680ade10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:44.170 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:44.167+0000 7fa26ffe4640 1 --2- 172.21.15.49:0/2106502941 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2680b2250 0x7fa268150a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:44.170 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:44.167+0000 7fa26ffe4640 1 -- 172.21.15.49:0/2106502941 >> 172.21.15.49:0/2106502941 conn(0x7fa268089fd0 msgr2=0x7fa26815e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:12:44.171 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:44.167+0000 7fa26ffe4640 1 -- 172.21.15.49:0/2106502941 shutdown_connections 2023-11-06T02:12:44.171 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:44.171+0000 7fa26ffe4640 1 -- 172.21.15.49:0/2106502941 wait complete. 2023-11-06T02:12:44.588 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 35, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-06T02:11:30.091070+0000', 'modified': '2023-11-06T02:12:44.061318+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 68, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6807}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6807': {'gid': 6807, 'name': 'c', 'rank': 0, 'incarnation': 33, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.49:6835/3698847494', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6834', 'nonce': 3698847494}, {'type': 'v1', 'addr': '172.21.15.49:6835', 'nonce': 3698847494}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-06T02:12:44.589 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6807:up:rejoin 2023-11-06T02:12:45.590 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T02:12:45.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:45.727+0000 7f2cf1053640 1 Processor -- start 2023-11-06T02:12:45.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:45.727+0000 7f2cf1053640 1 -- start start 2023-11-06T02:12:45.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:45.727+0000 7f2cf1053640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f2cec0ca980 0x7f2cec1644c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:45.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:45.727+0000 7f2cf1053640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cec0cad90 con 0x7f2cec159b40 2023-11-06T02:12:45.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:45.727+0000 7f2cf1053640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cec0723a0 con 0x7f2cec0cb010 2023-11-06T02:12:45.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:45.727+0000 7f2cf1053640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cec0715e0 con 0x7f2cec0ca980 2023-11-06T02:12:45.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:45.727+0000 7f2ceb59e640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f2cec0ca980 0x7f2cec1644c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:45.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:45.727+0000 7f2cea59c640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f2cec159b40 0x7f2cec0cdd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:42712/0 (socket says 172.21.15.49:42712) 2023-11-06T02:12:45.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:45.727+0000 7f2ceb59e640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f2cec0ca980 0x7f2cec1644c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58800/0 (socket says 172.21.15.49:58800) 2023-11-06T02:12:45.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:45.727+0000 7f2cea59c640 1 -- 172.21.15.49:0/72916773 learned_addr learned my addr 172.21.15.49:0/72916773 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:12:45.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:45.727+0000 7f2ceb59e640 1 -- 172.21.15.49:0/72916773 >> v1:172.21.15.150:6789/0 conn(0x7f2cec0cb010 legacy=0x7f2cec0cb3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:45.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:45.727+0000 7f2ceb59e640 1 -- 172.21.15.49:0/72916773 >> v1:172.21.15.49:6789/0 conn(0x7f2cec159b40 legacy=0x7f2cec0cdd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:45.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:45.727+0000 7f2ceb59e640 1 -- 172.21.15.49:0/72916773 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cec0718c0 con 0x7f2cec0ca980 2023-11-06T02:12:45.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:45.727+0000 7f2ce9d9b640 1 -- 172.21.15.49:0/72916773 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2030908846 0 0) 0x7f2cec0cad90 con 0x7f2cec159b40 2023-11-06T02:12:45.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:45.727+0000 7f2ceb59e640 1 --2- 172.21.15.49:0/72916773 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cec0ca980 0x7f2cec1644c0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f2ce4009d90 tx=0x7f2ce401c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4ec8fcbd9f9bbbe server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:45.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:45.731+0000 7f2ce9d9b640 1 -- 172.21.15.49:0/72916773 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce40a59e0 con 0x7f2cec0ca980 2023-11-06T02:12:45.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:45.731+0000 7f2ce9d9b640 1 -- 172.21.15.49:0/72916773 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ce40a5b40 con 0x7f2cec0ca980 2023-11-06T02:12:45.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:45.731+0000 7f2ce9d9b640 1 -- 172.21.15.49:0/72916773 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce40ad440 con 0x7f2cec0ca980 2023-11-06T02:12:45.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:45.731+0000 7f2cf1053640 1 -- 172.21.15.49:0/72916773 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cec0ca980 msgr2=0x7f2cec1644c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:45.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:45.731+0000 7f2cf1053640 1 --2- 172.21.15.49:0/72916773 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cec0ca980 0x7f2cec1644c0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f2ce4009d90 tx=0x7f2ce401c920 comp rx=0 tx=0).stop 2023-11-06T02:12:45.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:45.731+0000 7f2cf1053640 1 -- 172.21.15.49:0/72916773 shutdown_connections 2023-11-06T02:12:45.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:45.731+0000 7f2cf1053640 1 --2- 172.21.15.49:0/72916773 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cec0ca980 0x7f2cec1644c0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:45.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:45.731+0000 7f2cf1053640 1 -- 172.21.15.49:0/72916773 >> 172.21.15.49:0/72916773 conn(0x7f2cec06c0a0 msgr2=0x7f2cec06c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:12:45.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:45.731+0000 7f2cf1053640 1 -- 172.21.15.49:0/72916773 shutdown_connections 2023-11-06T02:12:45.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:45.731+0000 7f2cf1053640 1 -- 172.21.15.49:0/72916773 wait complete. 2023-11-06T02:12:45.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:45.731+0000 7f2cf1053640 1 Processor -- start 2023-11-06T02:12:45.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:45.731+0000 7f2cf1053640 1 -- start start 2023-11-06T02:12:45.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:45.731+0000 7f2cf1053640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cec0ca980 0x7f2cec150b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:45.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:45.731+0000 7f2cf1053640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2cec0cb010 0x7f2cec0c7c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:45.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:45.731+0000 7f2cf1053640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2cec159b40 0x7f2cec0c8180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:45.736 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:45.731+0000 7f2cf1053640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cec15b770 con 0x7f2cec0ca980 2023-11-06T02:12:45.736 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:45.731+0000 7f2cf1053640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cec15ba50 con 0x7f2cec159b40 2023-11-06T02:12:45.736 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:45.731+0000 7f2cf1053640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cec0c3370 con 0x7f2cec0cb010 2023-11-06T02:12:45.736 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:45.731+0000 7f2cead9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cec0ca980 0x7f2cec150b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:45.736 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:45.731+0000 7f2cead9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cec0ca980 0x7f2cec150b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58816/0 (socket says 172.21.15.49:58816) 2023-11-06T02:12:45.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:45.731+0000 7f2cead9d640 1 -- 172.21.15.49:0/2577053139 learned_addr learned my addr 172.21.15.49:0/2577053139 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:12:45.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:45.731+0000 7f2cea59c640 1 --2- 172.21.15.49:0/2577053139 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2cec0cb010 0x7f2cec0c7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:45.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:45.731+0000 7f2ceb59e640 1 --2- 172.21.15.49:0/2577053139 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2cec159b40 0x7f2cec0c8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:45.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:45.731+0000 7f2cead9d640 1 -- 172.21.15.49:0/2577053139 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2cec0cb010 msgr2=0x7f2cec0c7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:45.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:45.731+0000 7f2cead9d640 1 --2- 172.21.15.49:0/2577053139 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2cec0cb010 0x7f2cec0c7c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:45.738 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:45.731+0000 7f2cead9d640 1 -- 172.21.15.49:0/2577053139 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2cec159b40 msgr2=0x7f2cec0c8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:45.738 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:45.731+0000 7f2cead9d640 1 --2- 172.21.15.49:0/2577053139 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2cec159b40 0x7f2cec0c8180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:45.738 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:45.731+0000 7f2cead9d640 1 -- 172.21.15.49:0/2577053139 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ce4009a70 con 0x7f2cec0ca980 2023-11-06T02:12:45.738 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:45.731+0000 7f2cea59c640 1 --2- 172.21.15.49:0/2577053139 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2cec0cb010 0x7f2cec0c7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:12:45.738 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:45.731+0000 7f2cead9d640 1 --2- 172.21.15.49:0/2577053139 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cec0ca980 0x7f2cec150b50 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc00c400 tx=0x7f2cdc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:45.739 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:45.731+0000 7f2cd37fe640 1 -- 172.21.15.49:0/2577053139 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cdc0052d0 con 0x7f2cec0ca980 2023-11-06T02:12:45.739 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:45.731+0000 7f2cd37fe640 1 -- 172.21.15.49:0/2577053139 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cdc005430 con 0x7f2cec0ca980 2023-11-06T02:12:45.739 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:45.731+0000 7f2cf1053640 1 -- 172.21.15.49:0/2577053139 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cec0c3650 con 0x7f2cec0ca980 2023-11-06T02:12:45.739 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:45.731+0000 7f2cd37fe640 1 -- 172.21.15.49:0/2577053139 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cdc003890 con 0x7f2cec0ca980 2023-11-06T02:12:45.739 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:45.731+0000 7f2cf1053640 1 -- 172.21.15.49:0/2577053139 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cec14f280 con 0x7f2cec0ca980 2023-11-06T02:12:45.740 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:45.735+0000 7f2cf1053640 1 -- 172.21.15.49:0/2577053139 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cec0c3890 con 0x7f2cec0ca980 2023-11-06T02:12:45.740 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:45.735+0000 7f2cd37fe640 1 -- 172.21.15.49:0/2577053139 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f2cdc0039f0 con 0x7f2cec0ca980 2023-11-06T02:12:45.740 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:45.735+0000 7f2cd37fe640 1 --2- 172.21.15.49:0/2577053139 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2cd8065a60 0x7f2cd8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:45.740 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:45.735+0000 7f2cd37fe640 1 -- 172.21.15.49:0/2577053139 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6264+0+0 (secure 0 0 0) 0x7f2cdc082900 con 0x7f2cec0ca980 2023-11-06T02:12:45.740 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:45.735+0000 7f2cea59c640 1 --2- 172.21.15.49:0/2577053139 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2cd8065a60 0x7f2cd8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:45.741 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:45.735+0000 7f2cea59c640 1 --2- 172.21.15.49:0/2577053139 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2cd8065a60 0x7f2cd8067f20 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f2cec13e340 tx=0x7f2ce0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:45.741 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:45.739+0000 7f2cd37fe640 1 -- 172.21.15.49:0/2577053139 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f2cec0c3890 con 0x7f2cec0ca980 2023-11-06T02:12:45.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:45.927+0000 7f2cf1053640 1 -- 172.21.15.49:0/2577053139 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2cec0c3160 con 0x7f2cec0ca980 2023-11-06T02:12:45.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:45.931+0000 7f2cd37fe640 1 -- 172.21.15.49:0/2577053139 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+4656 (secure 0 0 0) 0x7f2cdc017090 con 0x7f2cec0ca980 2023-11-06T02:12:45.930 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:12:45.930 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":36,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5635,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4170291905","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":4170291905},{"type":"v1","addr":"172.21.15.150:6835","nonce":4170291905}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5638,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/2719458937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":2719458937},{"type":"v1","addr":"172.21.15.150:6837","nonce":2719458937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":6339,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/2310338375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2310338375},{"type":"v1","addr":"172.21.15.49:6833","nonce":2310338375}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T02:11:30.091070+0000","modified":"2023-11-06T02:12:45.070499+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":68,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6807},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6807":{"gid":6807,"name":"c","rank":0,"incarnation":33,"state":"up:active","state_seq":4,"addr":"172.21.15.49:6835/3698847494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3698847494},{"type":"v1","addr":"172.21.15.49:6835","nonce":3698847494}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-06T02:12:45.931 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 36 2023-11-06T02:12:45.932 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:45.931+0000 7f2cf1053640 1 -- 172.21.15.49:0/2577053139 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2cd8065a60 msgr2=0x7f2cd8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:45.932 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:45.931+0000 7f2cf1053640 1 --2- 172.21.15.49:0/2577053139 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2cd8065a60 0x7f2cd8067f20 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f2cec13e340 tx=0x7f2ce0009040 comp rx=0 tx=0).stop 2023-11-06T02:12:45.932 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:45.931+0000 7f2cf1053640 1 -- 172.21.15.49:0/2577053139 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cec0ca980 msgr2=0x7f2cec150b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:45.932 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:45.931+0000 7f2cf1053640 1 --2- 172.21.15.49:0/2577053139 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cec0ca980 0x7f2cec150b50 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc00c400 tx=0x7f2cdc00c8c0 comp rx=0 tx=0).stop 2023-11-06T02:12:45.932 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:45.931+0000 7f2cf1053640 1 -- 172.21.15.49:0/2577053139 shutdown_connections 2023-11-06T02:12:45.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:45.931+0000 7f2cf1053640 1 --2- 172.21.15.49:0/2577053139 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2cd8065a60 0x7f2cd8067f20 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:45.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:45.931+0000 7f2cf1053640 1 --2- 172.21.15.49:0/2577053139 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2cec159b40 0x7f2cec0c8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:45.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:45.931+0000 7f2cf1053640 1 --2- 172.21.15.49:0/2577053139 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2cec0cb010 0x7f2cec0c7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:45.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:45.931+0000 7f2cf1053640 1 --2- 172.21.15.49:0/2577053139 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cec0ca980 0x7f2cec150b50 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:45.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:45.931+0000 7f2cf1053640 1 -- 172.21.15.49:0/2577053139 >> 172.21.15.49:0/2577053139 conn(0x7f2cec06c0a0 msgr2=0x7f2cec072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:12:45.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:45.931+0000 7f2cf1053640 1 -- 172.21.15.49:0/2577053139 shutdown_connections 2023-11-06T02:12:45.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:45.931+0000 7f2cf1053640 1 -- 172.21.15.49:0/2577053139 wait complete. 2023-11-06T02:12:46.351 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 36, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-06T02:11:30.091070+0000', 'modified': '2023-11-06T02:12:45.070499+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 68, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6807}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6807': {'gid': 6807, 'name': 'c', 'rank': 0, 'incarnation': 33, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.49:6835/3698847494', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6834', 'nonce': 3698847494}, {'type': 'v1', 'addr': '172.21.15.49:6835', 'nonce': 3698847494}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-06T02:12:46.352 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-06T02:12:46.352 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-06T02:12:46.489 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.491+0000 7f4955e50640 1 Processor -- start 2023-11-06T02:12:46.489 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.491+0000 7f4955e50640 1 -- start start 2023-11-06T02:12:46.490 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.491+0000 7f4955e50640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f49500b5470 0x7f49500b1150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:46.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.491+0000 7f4955e50640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4950072360 con 0x7f495015e1c0 2023-11-06T02:12:46.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.491+0000 7f4955e50640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4950071e20 con 0x7f49501599d0 2023-11-06T02:12:46.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.491+0000 7f4955e50640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4950072080 con 0x7f49500b5470 2023-11-06T02:12:46.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.491+0000 7f4946ffd640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f49500b5470 0x7f49500b1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:46.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.491+0000 7f494ffff640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f495015e1c0 0x7f49500caf10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:42718/0 (socket says 172.21.15.49:42718) 2023-11-06T02:12:46.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.491+0000 7f494ffff640 1 -- 172.21.15.49:0/4098782418 learned_addr learned my addr 172.21.15.49:0/4098782418 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:12:46.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.491+0000 7f4946ffd640 1 -- 172.21.15.49:0/4098782418 >> v1:172.21.15.150:6789/0 conn(0x7f49501599d0 legacy=0x7f4950071710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:46.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.491+0000 7f4946ffd640 1 -- 172.21.15.49:0/4098782418 >> v1:172.21.15.49:6789/0 conn(0x7f495015e1c0 legacy=0x7f49500caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:46.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.491+0000 7f4946ffd640 1 -- 172.21.15.49:0/4098782418 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f495015d6e0 con 0x7f49500b5470 2023-11-06T02:12:46.493 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.495+0000 7f4946ffd640 1 --2- 172.21.15.49:0/4098782418 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49500b5470 0x7f49500b1150 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f49380097c0 tx=0x7f493801c250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f58a395eb556d2a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:46.493 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.495+0000 7f494effd640 1 -- 172.21.15.49:0/4098782418 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49380ab600 con 0x7f49500b5470 2023-11-06T02:12:46.493 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.495+0000 7f494effd640 1 -- 172.21.15.49:0/4098782418 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49380a7050 con 0x7f49500b5470 2023-11-06T02:12:46.494 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.495+0000 7f494effd640 1 -- 172.21.15.49:0/4098782418 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49380ab980 con 0x7f49500b5470 2023-11-06T02:12:46.495 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.495+0000 7f4955e50640 1 -- 172.21.15.49:0/4098782418 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49500b5470 msgr2=0x7f49500b1150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:46.495 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.495+0000 7f4955e50640 1 --2- 172.21.15.49:0/4098782418 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49500b5470 0x7f49500b1150 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f49380097c0 tx=0x7f493801c250 comp rx=0 tx=0).stop 2023-11-06T02:12:46.495 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.495+0000 7f4955e50640 1 -- 172.21.15.49:0/4098782418 shutdown_connections 2023-11-06T02:12:46.495 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.495+0000 7f4955e50640 1 --2- 172.21.15.49:0/4098782418 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49500b5470 0x7f49500b1150 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.495 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.495+0000 7f4955e50640 1 -- 172.21.15.49:0/4098782418 >> 172.21.15.49:0/4098782418 conn(0x7f49500cc150 msgr2=0x7f49500cc560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:12:46.496 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.495+0000 7f4955e50640 1 -- 172.21.15.49:0/4098782418 shutdown_connections 2023-11-06T02:12:46.496 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.495+0000 7f4955e50640 1 -- 172.21.15.49:0/4098782418 wait complete. 2023-11-06T02:12:46.498 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.495+0000 7f4955e50640 1 Processor -- start 2023-11-06T02:12:46.498 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.495+0000 7f4955e50640 1 -- start start 2023-11-06T02:12:46.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.495+0000 7f4955e50640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49500b5470 0x7f49500c8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:46.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.495+0000 7f4955e50640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f49501599d0 0x7f495014ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:46.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.495+0000 7f4955e50640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f495015e1c0 0x7f495015b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:46.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.495+0000 7f4955e50640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49500c90d0 con 0x7f49500b5470 2023-11-06T02:12:46.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.495+0000 7f4955e50640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f495014ef40 con 0x7f49501599d0 2023-11-06T02:12:46.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.495+0000 7f4955e50640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49500c7af0 con 0x7f495015e1c0 2023-11-06T02:12:46.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.499+0000 7f494f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49500b5470 0x7f49500c8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:46.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.499+0000 7f494f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49500b5470 0x7f49500c8b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58832/0 (socket says 172.21.15.49:58832) 2023-11-06T02:12:46.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.499+0000 7f494f7fe640 1 -- 172.21.15.49:0/3254264044 learned_addr learned my addr 172.21.15.49:0/3254264044 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:12:46.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.499+0000 7f4946ffd640 1 --2- 172.21.15.49:0/3254264044 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f49501599d0 0x7f495014ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:46.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.499+0000 7f494ffff640 1 --2- 172.21.15.49:0/3254264044 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f495015e1c0 0x7f495015b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:46.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.499+0000 7f494f7fe640 1 -- 172.21.15.49:0/3254264044 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f495015e1c0 msgr2=0x7f495015b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:46.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.499+0000 7f494f7fe640 1 --2- 172.21.15.49:0/3254264044 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f495015e1c0 0x7f495015b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.499+0000 7f494f7fe640 1 -- 172.21.15.49:0/3254264044 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f49501599d0 msgr2=0x7f495014ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:46.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.499+0000 7f494f7fe640 1 --2- 172.21.15.49:0/3254264044 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f49501599d0 0x7f495014ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.499+0000 7f494f7fe640 1 -- 172.21.15.49:0/3254264044 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4938009420 con 0x7f49500b5470 2023-11-06T02:12:46.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.499+0000 7f494ffff640 1 --2- 172.21.15.49:0/3254264044 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f495015e1c0 0x7f495015b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:12:46.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.499+0000 7f4946ffd640 1 --2- 172.21.15.49:0/3254264044 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f49501599d0 0x7f495014ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:12:46.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.499+0000 7f494f7fe640 1 --2- 172.21.15.49:0/3254264044 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49500b5470 0x7f49500c8b90 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f494000c500 tx=0x7f494000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:46.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.499+0000 7f494cff9640 1 -- 172.21.15.49:0/3254264044 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4940005220 con 0x7f49500b5470 2023-11-06T02:12:46.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.499+0000 7f494cff9640 1 -- 172.21.15.49:0/3254264044 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4940005380 con 0x7f49500b5470 2023-11-06T02:12:46.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.499+0000 7f4955e50640 1 -- 172.21.15.49:0/3254264044 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49500c7dd0 con 0x7f49500b5470 2023-11-06T02:12:46.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.499+0000 7f494cff9640 1 -- 172.21.15.49:0/3254264044 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4940007270 con 0x7f49500b5470 2023-11-06T02:12:46.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.499+0000 7f4955e50640 1 -- 172.21.15.49:0/3254264044 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49500c8390 con 0x7f49500b5470 2023-11-06T02:12:46.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.499+0000 7f494cff9640 1 -- 172.21.15.49:0/3254264044 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f49400073d0 con 0x7f49500b5470 2023-11-06T02:12:46.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.499+0000 7f494cff9640 1 --2- 172.21.15.49:0/3254264044 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4930065a90 0x7f4930067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:46.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.499+0000 7f494cff9640 1 -- 172.21.15.49:0/3254264044 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6264+0+0 (secure 0 0 0) 0x7f4940082780 con 0x7f49500b5470 2023-11-06T02:12:46.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.499+0000 7f4946ffd640 1 --2- 172.21.15.49:0/3254264044 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4930065a90 0x7f4930067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:46.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.503+0000 7f4946ffd640 1 --2- 172.21.15.49:0/3254264044 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4930065a90 0x7f4930067f50 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f49380ad000 tx=0x7f49380a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:46.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.503+0000 7f4958f78000 1 Processor -- start 2023-11-06T02:12:46.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.503+0000 7f4958f78000 1 -- start start 2023-11-06T02:12:46.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.503+0000 7f4958f78000 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db8913e910 0x55db8913ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:46.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.503+0000 7f4958f78000 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55db8913f230 0x55db890f4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:46.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.503+0000 7f4958f78000 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db890f4ca0 0x55db890f5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:46.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.503+0000 7f4958f78000 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db88fbafc0 con 0x55db8913e910 2023-11-06T02:12:46.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.503+0000 7f4958f78000 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db88f491a0 con 0x55db890f4ca0 2023-11-06T02:12:46.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.503+0000 7f4958f78000 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55db88f46770 con 0x55db8913f230 2023-11-06T02:12:46.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.503+0000 7f494f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db8913e910 0x55db8913ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:46.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.503+0000 7f494f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db8913e910 0x55db8913ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58834/0 (socket says 172.21.15.49:58834) 2023-11-06T02:12:46.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.503+0000 7f494f7fe640 1 -- 172.21.15.49:0/1184059535 learned_addr learned my addr 172.21.15.49:0/1184059535 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:12:46.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.503+0000 7f4946ffd640 1 --2- 172.21.15.49:0/1184059535 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55db8913f230 0x55db890f4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:46.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.503+0000 7f494ffff640 1 --2- 172.21.15.49:0/1184059535 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db890f4ca0 0x55db890f5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:46.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.503+0000 7f494f7fe640 1 -- 172.21.15.49:0/1184059535 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55db8913f230 msgr2=0x55db890f4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:46.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.503+0000 7f494f7fe640 1 --2- 172.21.15.49:0/1184059535 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55db8913f230 0x55db890f4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.503+0000 7f494f7fe640 1 -- 172.21.15.49:0/1184059535 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db890f4ca0 msgr2=0x55db890f5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:46.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.503+0000 7f494f7fe640 1 --2- 172.21.15.49:0/1184059535 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db890f4ca0 0x55db890f5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.503+0000 7f494f7fe640 1 -- 172.21.15.49:0/1184059535 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55db88c3e890 con 0x55db8913e910 2023-11-06T02:12:46.514 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.503+0000 7f494ffff640 1 --2- 172.21.15.49:0/1184059535 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db890f4ca0 0x55db890f5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:12:46.514 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.503+0000 7f494f7fe640 1 --2- 172.21.15.49:0/1184059535 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db8913e910 0x55db8913ecf0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f494005b5f0 tx=0x7f4940053560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a600ac76547e2ef server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:46.514 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.503+0000 7f49457fa640 1 -- 172.21.15.49:0/1184059535 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4940053b40 con 0x55db8913e910 2023-11-06T02:12:46.514 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.503+0000 7f49457fa640 1 -- 172.21.15.49:0/1184059535 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4940057970 con 0x55db8913e910 2023-11-06T02:12:46.514 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.503+0000 7f49457fa640 1 -- 172.21.15.49:0/1184059535 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f494005bb20 con 0x55db8913e910 2023-11-06T02:12:46.515 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.503+0000 7f4958f78000 1 -- 172.21.15.49:0/1184059535 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db8913e910 msgr2=0x55db8913ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:46.515 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.503+0000 7f4958f78000 1 --2- 172.21.15.49:0/1184059535 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db8913e910 0x55db8913ecf0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f494005b5f0 tx=0x7f4940053560 comp rx=0 tx=0).stop 2023-11-06T02:12:46.515 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.503+0000 7f4958f78000 1 -- 172.21.15.49:0/1184059535 shutdown_connections 2023-11-06T02:12:46.515 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.503+0000 7f4958f78000 1 --2- 172.21.15.49:0/1184059535 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db890f4ca0 0x55db890f5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.515 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.503+0000 7f4958f78000 1 --2- 172.21.15.49:0/1184059535 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55db8913f230 0x55db890f4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.515 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.503+0000 7f4958f78000 1 --2- 172.21.15.49:0/1184059535 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db8913e910 0x55db8913ecf0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.516 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.503+0000 7f4958f78000 1 -- 172.21.15.49:0/1184059535 >> 172.21.15.49:0/1184059535 conn(0x55db890f8de0 msgr2=0x55db890f3040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:12:46.516 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.503+0000 7f4958f78000 1 -- 172.21.15.49:0/1184059535 shutdown_connections 2023-11-06T02:12:46.516 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.503+0000 7f4958f78000 1 -- 172.21.15.49:0/1184059535 wait complete. 2023-11-06T02:12:46.516 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.503+0000 7f4958f78000 1 -- start start 2023-11-06T02:12:46.516 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.503+0000 7f4958f78000 1 Processor -- start 2023-11-06T02:12:46.517 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.507+0000 7f4958f78000 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db890f4ca0 0x55db8912b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:46.517 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.507+0000 7f4958f78000 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db8913e910 0x55db8913f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:46.517 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.507+0000 7f4958f78000 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55db8913f230 0x55db8913fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:46.517 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.507+0000 7f4958f78000 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db88f42060 con 0x55db890f4ca0 2023-11-06T02:12:46.517 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.507+0000 7f4958f78000 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db88ee1290 con 0x55db8913e910 2023-11-06T02:12:46.518 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.507+0000 7f4958f78000 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55db88eff6f0 con 0x55db8913f230 2023-11-06T02:12:46.518 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.507+0000 7f4958f78000 10 client.0 _ll_register_callbacks cb 0x55db890de1d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T02:12:46.518 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.507+0000 7f494f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db890f4ca0 0x55db8912b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:46.518 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.507+0000 7f494f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db890f4ca0 0x55db8912b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58844/0 (socket says 172.21.15.49:58844) 2023-11-06T02:12:46.518 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.507+0000 7f494f7fe640 1 -- 172.21.15.49:0/1473540789 learned_addr learned my addr 172.21.15.49:0/1473540789 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:12:46.519 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.507+0000 7f494ffff640 1 --2- 172.21.15.49:0/1473540789 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55db8913f230 0x55db8913fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:46.519 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.507+0000 7f4946ffd640 1 --2- 172.21.15.49:0/1473540789 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db8913e910 0x55db8913f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:46.519 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.507+0000 7f494f7fe640 1 -- 172.21.15.49:0/1473540789 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55db8913f230 msgr2=0x55db8913fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:46.519 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.507+0000 7f494f7fe640 1 --2- 172.21.15.49:0/1473540789 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55db8913f230 0x55db8913fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.519 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.507+0000 7f494f7fe640 1 -- 172.21.15.49:0/1473540789 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db8913e910 msgr2=0x55db8913f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:46.520 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.507+0000 7f494f7fe640 1 --2- 172.21.15.49:0/1473540789 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db8913e910 0x55db8913f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.520 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.507+0000 7f494f7fe640 1 -- 172.21.15.49:0/1473540789 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49500c8390 con 0x55db890f4ca0 2023-11-06T02:12:46.520 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.507+0000 7f494f7fe640 1 --2- 172.21.15.49:0/1473540789 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db890f4ca0 0x55db8912b200 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f494005b5f0 tx=0x7f4940001130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8ca99c9b745ef32 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:46.520 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.507+0000 7f492e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-11-06T02:12:46.520 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.507+0000 7f492e7fc640 1 -- 172.21.15.49:0/1473540789 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4940053b40 con 0x55db890f4ca0 2023-11-06T02:12:46.521 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.507+0000 7f492e7fc640 1 -- 172.21.15.49:0/1473540789 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(68..68 src has 1..68) v4 ==== 6264+0+0 (secure 0 0 0) 0x7f4940080e40 con 0x55db890f4ca0 2023-11-06T02:12:46.521 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.507+0000 7f4958f78000 1 -- 172.21.15.49:0/1473540789 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55db88f409b0 con 0x55db890f4ca0 2023-11-06T02:12:46.521 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.507+0000 7f492e7fc640 1 -- 172.21.15.49:0/1473540789 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4940057970 con 0x55db890f4ca0 2023-11-06T02:12:46.521 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.507+0000 7f4958f78000 10 client.6879 fetch_fsmap learned FSMap version 36 2023-11-06T02:12:46.521 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.507+0000 7f4958f78000 1 -- 172.21.15.49:0/1473540789 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) v3 -- 0x55db88f071e0 con 0x55db890f4ca0 2023-11-06T02:12:46.521 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.507+0000 7f492e7fc640 1 -- 172.21.15.49:0/1473540789 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4940079020 con 0x55db890f4ca0 2023-11-06T02:12:46.522 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.507+0000 7f492e7fc640 1 -- 172.21.15.49:0/1473540789 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 36) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f4940019030 con 0x55db890f4ca0 2023-11-06T02:12:46.522 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.507+0000 7f4958f78000 10 client.6879 fetch_fsmap finished waiting for FSMap version 36 2023-11-06T02:12:46.522 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.507+0000 7f4958f78000 10 client.6879 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-06T02:12:46.522 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.507+0000 7f4958f78000 20 client.6879 populate_metadata read hostname 'smithi049' 2023-11-06T02:12:46.522 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.507+0000 7f4958f78000 1 --2- 172.21.15.49:0/1473540789 >> [v2:172.21.15.49:6834/3698847494,v1:172.21.15.49:6835/3698847494] conn(0x55db89140b00 0x55db891504f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:46.523 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.507+0000 7f4958f78000 4 client.6879 mds_command: new command op to 6807 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T02:12:46.523 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.507+0000 7f4958f78000 1 -- 172.21.15.49:0/1473540789 --> [v2:172.21.15.49:6834/3698847494,v1:172.21.15.49:6835/3698847494] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55db88f714d0 con 0x55db89140b00 2023-11-06T02:12:46.523 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.507+0000 7f4946ffd640 1 --2- 172.21.15.49:0/1473540789 >> [v2:172.21.15.49:6834/3698847494,v1:172.21.15.49:6835/3698847494] conn(0x55db89140b00 0x55db891504f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:46.523 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.507+0000 7f492e7fc640 10 client.6879 ms_handle_connect on v2:172.21.15.49:6834/3698847494 2023-11-06T02:12:46.523 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.507+0000 7f4946ffd640 1 --2- 172.21.15.49:0/1473540789 >> [v2:172.21.15.49:6834/3698847494,v1:172.21.15.49:6835/3698847494] conn(0x55db89140b00 0x55db891504f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1850be50d8bad102 server_cookie=6fcbacbd8a46a232 in_seq=0 out_seq=0 2023-11-06T02:12:46.524 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.507+0000 7f492e7fc640 1 -- 172.21.15.49:0/1473540789 <== mds.0 v2:172.21.15.49:6834/3698847494 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55db88f714d0 con 0x55db89140b00 2023-11-06T02:12:46.524 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.507+0000 7f492e7fc640 10 client.6879 handle_command_reply: tid=0 2023-11-06T02:12:46.524 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.507+0000 7f4958f78000 1 client.6879 shutdown 2023-11-06T02:12:46.524 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.507+0000 7f4958f78000 1 -- 172.21.15.49:0/1473540789 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db890f4ca0 msgr2=0x55db8912b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:46.524 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.507+0000 7f4958f78000 1 --2- 172.21.15.49:0/1473540789 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db890f4ca0 0x55db8912b200 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f494005b5f0 tx=0x7f4940001130 comp rx=0 tx=0).stop 2023-11-06T02:12:46.525 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.507+0000 7f4958f78000 1 -- 172.21.15.49:0/1473540789 shutdown_connections 2023-11-06T02:12:46.525 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.507+0000 7f4958f78000 1 --2- 172.21.15.49:0/1473540789 >> [v2:172.21.15.49:6834/3698847494,v1:172.21.15.49:6835/3698847494] conn(0x55db89140b00 0x55db891504f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.525 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.507+0000 7f4958f78000 1 --2- 172.21.15.49:0/1473540789 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55db8913f230 0x55db8913fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.525 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.507+0000 7f4958f78000 1 --2- 172.21.15.49:0/1473540789 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db8913e910 0x55db8913f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.525 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.507+0000 7f4958f78000 1 --2- 172.21.15.49:0/1473540789 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db890f4ca0 0x55db8912b200 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.526 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.507+0000 7f492e7fc640 0 client.6879 ms_handle_reset on v2:172.21.15.49:6834/3698847494 2023-11-06T02:12:46.526 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.507+0000 7f4958f78000 1 -- 172.21.15.49:0/1473540789 >> 172.21.15.49:0/1473540789 conn(0x55db890f8de0 msgr2=0x55db890f7920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:12:46.526 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.507+0000 7f4958f78000 1 -- 172.21.15.49:0/1473540789 shutdown_connections 2023-11-06T02:12:46.526 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.507+0000 7f4958f78000 1 -- 172.21.15.49:0/1473540789 wait complete. 2023-11-06T02:12:46.526 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.507+0000 7f4958f78000 20 client.6879 trim_cache size 0 max 16384 2023-11-06T02:12:46.532 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.523+0000 7f4958f78000 1 Processor -- start 2023-11-06T02:12:46.533 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.523+0000 7f4958f78000 1 -- start start 2023-11-06T02:12:46.533 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.523+0000 7f4958f78000 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db890f4ca0 0x55db890f82b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:46.533 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.523+0000 7f4958f78000 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55db8913e910 0x55db89129f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:46.533 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.523+0000 7f4958f78000 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db8913f230 0x55db8912a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:46.533 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.523+0000 7f4958f78000 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db88bff940 con 0x55db8913f230 2023-11-06T02:12:46.534 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.523+0000 7f4958f78000 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db88f435f0 con 0x55db890f4ca0 2023-11-06T02:12:46.534 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.523+0000 7f4958f78000 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55db88f36d30 con 0x55db8913e910 2023-11-06T02:12:46.534 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.523+0000 7f494ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db8913f230 0x55db8912a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:46.534 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.523+0000 7f494ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db8913f230 0x55db8912a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58846/0 (socket says 172.21.15.49:58846) 2023-11-06T02:12:46.534 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.523+0000 7f494ffff640 1 -- 172.21.15.49:0/653143587 learned_addr learned my addr 172.21.15.49:0/653143587 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:12:46.535 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.523+0000 7f494f7fe640 1 --2- 172.21.15.49:0/653143587 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db890f4ca0 0x55db890f82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:46.535 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.523+0000 7f4946ffd640 1 --2- 172.21.15.49:0/653143587 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55db8913e910 0x55db89129f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:46.535 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.523+0000 7f494ffff640 1 -- 172.21.15.49:0/653143587 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55db8913e910 msgr2=0x55db89129f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:46.535 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.523+0000 7f494ffff640 1 --2- 172.21.15.49:0/653143587 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55db8913e910 0x55db89129f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.535 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.523+0000 7f494ffff640 1 -- 172.21.15.49:0/653143587 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db890f4ca0 msgr2=0x55db890f82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:46.536 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.527+0000 7f494ffff640 1 --2- 172.21.15.49:0/653143587 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db890f4ca0 0x55db890f82b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.536 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.527+0000 7f494ffff640 1 -- 172.21.15.49:0/653143587 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55db88eed330 con 0x55db8913f230 2023-11-06T02:12:46.536 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.527+0000 7f4946ffd640 1 --2- 172.21.15.49:0/653143587 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55db8913e910 0x55db89129f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:12:46.536 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.527+0000 7f494ffff640 1 --2- 172.21.15.49:0/653143587 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db8913f230 0x55db8912a450 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f4948001de0 tx=0x7f494800f600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6690224f6775c96 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:46.536 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.527+0000 7f4945ffb640 1 -- 172.21.15.49:0/653143587 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4948018070 con 0x55db8913f230 2023-11-06T02:12:46.537 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.527+0000 7f4945ffb640 1 -- 172.21.15.49:0/653143587 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4948001ed0 con 0x55db8913f230 2023-11-06T02:12:46.537 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.527+0000 7f4945ffb640 1 -- 172.21.15.49:0/653143587 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4948004b70 con 0x55db8913f230 2023-11-06T02:12:46.537 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.527+0000 7f4958f78000 1 -- 172.21.15.49:0/653143587 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db8913f230 msgr2=0x55db8912a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:46.537 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.527+0000 7f4958f78000 1 --2- 172.21.15.49:0/653143587 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db8913f230 0x55db8912a450 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f4948001de0 tx=0x7f494800f600 comp rx=0 tx=0).stop 2023-11-06T02:12:46.537 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.527+0000 7f4958f78000 1 -- 172.21.15.49:0/653143587 shutdown_connections 2023-11-06T02:12:46.538 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.527+0000 7f4958f78000 1 --2- 172.21.15.49:0/653143587 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db8913f230 0x55db8912a450 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.538 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.527+0000 7f4958f78000 1 --2- 172.21.15.49:0/653143587 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55db8913e910 0x55db89129f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.538 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.527+0000 7f4958f78000 1 --2- 172.21.15.49:0/653143587 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db890f4ca0 0x55db890f82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.538 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.527+0000 7f4958f78000 1 -- 172.21.15.49:0/653143587 >> 172.21.15.49:0/653143587 conn(0x55db890f8de0 msgr2=0x55db89129800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:12:46.538 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.527+0000 7f4958f78000 1 -- 172.21.15.49:0/653143587 shutdown_connections 2023-11-06T02:12:46.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.527+0000 7f4958f78000 1 -- 172.21.15.49:0/653143587 wait complete. 2023-11-06T02:12:46.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.527+0000 7f4958f78000 1 -- start start 2023-11-06T02:12:46.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.527+0000 7f4958f78000 1 Processor -- start 2023-11-06T02:12:46.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.527+0000 7f4958f78000 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db890f4ca0 0x55db8919c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:46.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.527+0000 7f4958f78000 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db8913e910 0x55db8919c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:46.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.527+0000 7f4958f78000 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55db8913f230 0x55db8914f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:46.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.527+0000 7f4958f78000 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db88f3d9d0 con 0x55db890f4ca0 2023-11-06T02:12:46.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.527+0000 7f4958f78000 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db88f4a430 con 0x55db8913e910 2023-11-06T02:12:46.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.527+0000 7f4958f78000 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55db88f85560 con 0x55db8913f230 2023-11-06T02:12:46.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.527+0000 7f4958f78000 10 client.0 _ll_register_callbacks cb 0x55db890be9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T02:12:46.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.527+0000 7f494f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db890f4ca0 0x55db8919c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:46.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.527+0000 7f494f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db890f4ca0 0x55db8919c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58852/0 (socket says 172.21.15.49:58852) 2023-11-06T02:12:46.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.527+0000 7f494f7fe640 1 -- 172.21.15.49:0/345046684 learned_addr learned my addr 172.21.15.49:0/345046684 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:12:46.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.527+0000 7f4946ffd640 1 --2- 172.21.15.49:0/345046684 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db8913e910 0x55db8919c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:46.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.527+0000 7f494ffff640 1 --2- 172.21.15.49:0/345046684 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55db8913f230 0x55db8914f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:46.542 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.527+0000 7f494f7fe640 1 -- 172.21.15.49:0/345046684 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55db8913f230 msgr2=0x55db8914f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:46.542 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.527+0000 7f494f7fe640 1 --2- 172.21.15.49:0/345046684 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55db8913f230 0x55db8914f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.542 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.527+0000 7f494f7fe640 1 -- 172.21.15.49:0/345046684 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db8913e910 msgr2=0x55db8919c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:46.542 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.527+0000 7f494f7fe640 1 --2- 172.21.15.49:0/345046684 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db8913e910 0x55db8919c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.542 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.527+0000 7f494f7fe640 1 -- 172.21.15.49:0/345046684 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4948001a10 con 0x55db890f4ca0 2023-11-06T02:12:46.543 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.527+0000 7f494ffff640 1 --2- 172.21.15.49:0/345046684 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55db8913f230 0x55db8914f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:12:46.543 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.527+0000 7f494f7fe640 1 --2- 172.21.15.49:0/345046684 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db890f4ca0 0x55db8919c380 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f4940052110 tx=0x7f49400055f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21014ab5339be32 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:46.543 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.527+0000 7f492e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-11-06T02:12:46.543 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.527+0000 7f492e7fc640 1 -- 172.21.15.49:0/345046684 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4940057970 con 0x55db890f4ca0 2023-11-06T02:12:46.543 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.527+0000 7f492e7fc640 1 -- 172.21.15.49:0/345046684 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(68..68 src has 1..68) v4 ==== 6264+0+0 (secure 0 0 0) 0x7f494007a080 con 0x55db890f4ca0 2023-11-06T02:12:46.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.527+0000 7f4958f78000 1 -- 172.21.15.49:0/345046684 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55db89141b00 con 0x55db890f4ca0 2023-11-06T02:12:46.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.527+0000 7f492e7fc640 1 -- 172.21.15.49:0/345046684 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4940053b40 con 0x55db890f4ca0 2023-11-06T02:12:46.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.527+0000 7f4958f78000 10 client.6885 fetch_fsmap learned FSMap version 36 2023-11-06T02:12:46.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.527+0000 7f4958f78000 1 -- 172.21.15.49:0/345046684 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) v3 -- 0x55db88c3ecc0 con 0x55db890f4ca0 2023-11-06T02:12:46.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.527+0000 7f492e7fc640 1 -- 172.21.15.49:0/345046684 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55db89141b00 con 0x55db890f4ca0 2023-11-06T02:12:46.545 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.527+0000 7f492e7fc640 1 -- 172.21.15.49:0/345046684 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 36) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f4940019040 con 0x55db890f4ca0 2023-11-06T02:12:46.545 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.527+0000 7f4958f78000 10 client.6885 fetch_fsmap finished waiting for FSMap version 36 2023-11-06T02:12:46.545 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.527+0000 7f4958f78000 10 client.6885 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-06T02:12:46.545 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.527+0000 7f4958f78000 20 client.6885 populate_metadata read hostname 'smithi049' 2023-11-06T02:12:46.545 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.527+0000 7f4958f78000 1 --2- 172.21.15.49:0/345046684 >> [v2:172.21.15.49:6834/3698847494,v1:172.21.15.49:6835/3698847494] conn(0x55db8914fed0 0x55db891502b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:46.545 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.527+0000 7f4958f78000 4 client.6885 mds_command: new command op to 6807 tid=0[{"prefix": "status"}] 2023-11-06T02:12:46.546 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.527+0000 7f4958f78000 1 -- 172.21.15.49:0/345046684 --> [v2:172.21.15.49:6834/3698847494,v1:172.21.15.49:6835/3698847494] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55db88f11b90 con 0x55db8914fed0 2023-11-06T02:12:46.546 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.527+0000 7f4946ffd640 1 --2- 172.21.15.49:0/345046684 >> [v2:172.21.15.49:6834/3698847494,v1:172.21.15.49:6835/3698847494] conn(0x55db8914fed0 0x55db891502b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:46.546 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.531+0000 7f492e7fc640 10 client.6885 ms_handle_connect on v2:172.21.15.49:6834/3698847494 2023-11-06T02:12:46.546 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.531+0000 7f4946ffd640 1 --2- 172.21.15.49:0/345046684 >> [v2:172.21.15.49:6834/3698847494,v1:172.21.15.49:6835/3698847494] conn(0x55db8914fed0 0x55db891502b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8af20814041aebd9 server_cookie=ad6f9c73bf5512fa in_seq=0 out_seq=0 2023-11-06T02:12:46.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.531+0000 7f492e7fc640 1 -- 172.21.15.49:0/345046684 <== mds.0 v2:172.21.15.49:6834/3698847494 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55db88f11b90 con 0x55db8914fed0 2023-11-06T02:12:46.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.531+0000 7f492e7fc640 10 client.6885 handle_command_reply: tid=0 2023-11-06T02:12:46.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.531+0000 7f4958f78000 1 client.6885 shutdown 2023-11-06T02:12:46.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.531+0000 7f4958f78000 1 -- 172.21.15.49:0/345046684 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db890f4ca0 msgr2=0x55db8919c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:46.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.531+0000 7f4958f78000 1 --2- 172.21.15.49:0/345046684 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db890f4ca0 0x55db8919c380 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f4940052110 tx=0x7f49400055f0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.531+0000 7f4958f78000 1 -- 172.21.15.49:0/345046684 shutdown_connections 2023-11-06T02:12:46.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.531+0000 7f4958f78000 1 --2- 172.21.15.49:0/345046684 >> [v2:172.21.15.49:6834/3698847494,v1:172.21.15.49:6835/3698847494] conn(0x55db8914fed0 0x55db891502b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.531+0000 7f4958f78000 1 --2- 172.21.15.49:0/345046684 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55db8913f230 0x55db8914f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.531+0000 7f4958f78000 1 --2- 172.21.15.49:0/345046684 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55db8913e910 0x55db8919c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.531+0000 7f492e7fc640 0 client.6885 ms_handle_reset on v2:172.21.15.49:6834/3698847494 2023-11-06T02:12:46.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.531+0000 7f4958f78000 1 --2- 172.21.15.49:0/345046684 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55db890f4ca0 0x55db8919c380 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.531+0000 7f4958f78000 1 -- 172.21.15.49:0/345046684 >> 172.21.15.49:0/345046684 conn(0x55db890f8de0 msgr2=0x55db890f7090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:12:46.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.531+0000 7f4958f78000 1 -- 172.21.15.49:0/345046684 shutdown_connections 2023-11-06T02:12:46.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.531+0000 7f4958f78000 1 -- 172.21.15.49:0/345046684 wait complete. 2023-11-06T02:12:46.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.531+0000 7f4958f78000 20 client.6885 trim_cache size 0 max 16384 2023-11-06T02:12:46.550 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-11-06T02:12:46.550 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_fsid": "1a8b035c-6ba4-499a-b876-aea47fd9c0d4", 2023-11-06T02:12:46.550 INFO:teuthology.orchestra.run.smithi049.stdout: "whoami": 0, 2023-11-06T02:12:46.550 INFO:teuthology.orchestra.run.smithi049.stdout: "id": 6807, 2023-11-06T02:12:46.551 INFO:teuthology.orchestra.run.smithi049.stdout: "want_state": "up:active", 2023-11-06T02:12:46.551 INFO:teuthology.orchestra.run.smithi049.stdout: "state": "up:active", 2023-11-06T02:12:46.551 INFO:teuthology.orchestra.run.smithi049.stdout: "fs_name": "cephfs", 2023-11-06T02:12:46.551 INFO:teuthology.orchestra.run.smithi049.stdout: "rank_uptime": 4.4735489209999999, 2023-11-06T02:12:46.552 INFO:teuthology.orchestra.run.smithi049.stdout: "mdsmap_epoch": 36, 2023-11-06T02:12:46.552 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch": 68, 2023-11-06T02:12:46.552 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch_barrier": 68, 2023-11-06T02:12:46.552 INFO:teuthology.orchestra.run.smithi049.stdout: "uptime": 5.4112688589999998 2023-11-06T02:12:46.552 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-11-06T02:12:46.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.531+0000 7f4955e50640 1 -- 172.21.15.49:0/3254264044 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4930065a90 msgr2=0x7f4930067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:46.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.531+0000 7f4955e50640 1 --2- 172.21.15.49:0/3254264044 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4930065a90 0x7f4930067f50 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f49380ad000 tx=0x7f49380a6000 comp rx=0 tx=0).stop 2023-11-06T02:12:46.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.531+0000 7f4955e50640 1 -- 172.21.15.49:0/3254264044 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49500b5470 msgr2=0x7f49500c8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:46.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.531+0000 7f4955e50640 1 --2- 172.21.15.49:0/3254264044 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49500b5470 0x7f49500c8b90 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f494000c500 tx=0x7f494000c9c0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.531+0000 7f4955e50640 1 -- 172.21.15.49:0/3254264044 shutdown_connections 2023-11-06T02:12:46.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.531+0000 7f4955e50640 1 --2- 172.21.15.49:0/3254264044 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4930065a90 0x7f4930067f50 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.531+0000 7f4955e50640 1 --2- 172.21.15.49:0/3254264044 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f495015e1c0 0x7f495015b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.531+0000 7f4955e50640 1 --2- 172.21.15.49:0/3254264044 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f49501599d0 0x7f495014ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.531+0000 7f4955e50640 1 --2- 172.21.15.49:0/3254264044 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49500b5470 0x7f49500c8b90 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.531+0000 7f4955e50640 1 -- 172.21.15.49:0/3254264044 >> 172.21.15.49:0/3254264044 conn(0x7f49500cc150 msgr2=0x7f4950154ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:12:46.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.531+0000 7f4955e50640 1 -- 172.21.15.49:0/3254264044 shutdown_connections 2023-11-06T02:12:46.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.531+0000 7f4955e50640 1 -- 172.21.15.49:0/3254264044 wait complete. 2023-11-06T02:12:46.556 DEBUG:teuthology.orchestra.run.smithi049:> sudo daemon-helper kill ceph --cluster ceph -w 2023-11-06T02:12:46.559 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 scrub start / repair,recursive 2023-11-06T02:12:46.680 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.679+0000 7f9dcbdb5640 1 Processor -- start 2023-11-06T02:12:46.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.679+0000 7f9dcbdb5640 1 -- start start 2023-11-06T02:12:46.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.683+0000 7f9dcbdb5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9dc415a930 0x7f9dc40cb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:46.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.683+0000 7f9dcbdb5640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f9dc40b29e0 0x7f9dc40ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:46.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.683+0000 7f9dcbdb5640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dc415e6f0 con 0x7f9dc40b6020 2023-11-06T02:12:46.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.683+0000 7f9dcbdb5640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dc40b6830 con 0x7f9dc40b29e0 2023-11-06T02:12:46.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.683+0000 7f9dcbdb5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc415f4b0 con 0x7f9dc415a930 2023-11-06T02:12:46.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.683+0000 7f9dca352640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f9dc40b29e0 0x7f9dc40ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:46.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.683+0000 7f9dca352640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f9dc40b29e0 0x7f9dc40ce930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58860/0 (socket says 172.21.15.49:58860) 2023-11-06T02:12:46.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.683+0000 7f9dca352640 1 -- 172.21.15.49:0/3967447117 learned_addr learned my addr 172.21.15.49:0/3967447117 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:12:46.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.683+0000 7f9dc9b51640 1 --2- 172.21.15.49:0/3967447117 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9dc415a930 0x7f9dc40cb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:46.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.683+0000 7f9dca352640 1 -- 172.21.15.49:0/3967447117 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9dc415a930 msgr2=0x7f9dc40cb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:46.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.683+0000 7f9dca352640 1 --2- 172.21.15.49:0/3967447117 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9dc415a930 0x7f9dc40cb8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.683+0000 7f9dca352640 1 -- 172.21.15.49:0/3967447117 >> v1:172.21.15.150:6789/0 conn(0x7f9dc40b6020 legacy=0x7f9dc40b22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:46.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.683+0000 7f9dca352640 1 -- 172.21.15.49:0/3967447117 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dc415e9d0 con 0x7f9dc40b29e0 2023-11-06T02:12:46.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.683+0000 7f9dca352640 1 --2- 172.21.15.49:0/3967447117 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9dc40b29e0 0x7f9dc40ce930 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f9db4009e80 tx=0x7f9db401c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=447d37455f7b95cb server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:46.686 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.683+0000 7f9dc8b4f640 1 -- 172.21.15.49:0/3967447117 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db40a5770 con 0x7f9dc40b29e0 2023-11-06T02:12:46.686 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.683+0000 7f9dc8b4f640 1 -- 172.21.15.49:0/3967447117 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9db40a58d0 con 0x7f9dc40b29e0 2023-11-06T02:12:46.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.683+0000 7f9dc8b4f640 1 -- 172.21.15.49:0/3967447117 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db40a5c60 con 0x7f9dc40b29e0 2023-11-06T02:12:46.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.687+0000 7f9dcbdb5640 1 -- 172.21.15.49:0/3967447117 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9dc40b29e0 msgr2=0x7f9dc40ce930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:46.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.687+0000 7f9dcbdb5640 1 --2- 172.21.15.49:0/3967447117 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9dc40b29e0 0x7f9dc40ce930 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f9db4009e80 tx=0x7f9db401c8f0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.687+0000 7f9dcbdb5640 1 -- 172.21.15.49:0/3967447117 shutdown_connections 2023-11-06T02:12:46.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.687+0000 7f9dcbdb5640 1 --2- 172.21.15.49:0/3967447117 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9dc40b29e0 0x7f9dc40ce930 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.687+0000 7f9dcbdb5640 1 --2- 172.21.15.49:0/3967447117 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9dc415a930 0x7f9dc40cb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.687+0000 7f9dcbdb5640 1 -- 172.21.15.49:0/3967447117 >> 172.21.15.49:0/3967447117 conn(0x7f9dc40cd2d0 msgr2=0x7f9dc40cd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:12:46.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.687+0000 7f9dcbdb5640 1 -- 172.21.15.49:0/3967447117 shutdown_connections 2023-11-06T02:12:46.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.687+0000 7f9dcbdb5640 1 -- 172.21.15.49:0/3967447117 wait complete. 2023-11-06T02:12:46.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.687+0000 7f9dcbdb5640 1 Processor -- start 2023-11-06T02:12:46.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.687+0000 7f9dcbdb5640 1 -- start start 2023-11-06T02:12:46.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.687+0000 7f9dcbdb5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9dc40b29e0 0x7f9dc414f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:46.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.687+0000 7f9dcbdb5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9dc40b6020 0x7f9dc40b0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:46.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.687+0000 7f9dcbdb5640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9dc415a930 0x7f9dc40c93d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:46.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.687+0000 7f9dcbdb5640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc40c8330 con 0x7f9dc40b29e0 2023-11-06T02:12:46.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.687+0000 7f9dcbdb5640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc40c8610 con 0x7f9dc415a930 2023-11-06T02:12:46.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.687+0000 7f9dcbdb5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc40c88f0 con 0x7f9dc40b6020 2023-11-06T02:12:46.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.687+0000 7f9dc9b51640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9dc40b29e0 0x7f9dc414f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:46.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.687+0000 7f9dc9b51640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9dc40b29e0 0x7f9dc414f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58872/0 (socket says 172.21.15.49:58872) 2023-11-06T02:12:46.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.687+0000 7f9dc9b51640 1 -- 172.21.15.49:0/92031572 learned_addr learned my addr 172.21.15.49:0/92031572 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:12:46.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.687+0000 7f9dca352640 1 --2- 172.21.15.49:0/92031572 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9dc415a930 0x7f9dc40c93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:46.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.687+0000 7f9dc9350640 1 --2- 172.21.15.49:0/92031572 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9dc40b6020 0x7f9dc40b0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:46.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.687+0000 7f9dc9b51640 1 -- 172.21.15.49:0/92031572 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9dc40b6020 msgr2=0x7f9dc40b0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:46.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.687+0000 7f9dc9b51640 1 --2- 172.21.15.49:0/92031572 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9dc40b6020 0x7f9dc40b0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.687+0000 7f9dc9b51640 1 -- 172.21.15.49:0/92031572 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9dc415a930 msgr2=0x7f9dc40c93d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:46.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.687+0000 7f9dc9b51640 1 --2- 172.21.15.49:0/92031572 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9dc415a930 0x7f9dc40c93d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.687+0000 7f9dc9b51640 1 -- 172.21.15.49:0/92031572 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9db4009b30 con 0x7f9dc40b29e0 2023-11-06T02:12:46.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.687+0000 7f9dc9b51640 1 --2- 172.21.15.49:0/92031572 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9dc40b29e0 0x7f9dc414f650 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc00d800 tx=0x7f9dbc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:46.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.687+0000 7f9dc27fc640 1 -- 172.21.15.49:0/92031572 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dbc00b6c0 con 0x7f9dc40b29e0 2023-11-06T02:12:46.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.687+0000 7f9dc27fc640 1 -- 172.21.15.49:0/92031572 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dbc00b820 con 0x7f9dc40b29e0 2023-11-06T02:12:46.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.687+0000 7f9dcbdb5640 1 -- 172.21.15.49:0/92031572 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dc40b6600 con 0x7f9dc40b29e0 2023-11-06T02:12:46.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.687+0000 7f9dc27fc640 1 -- 172.21.15.49:0/92031572 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dbc00bb90 con 0x7f9dc40b29e0 2023-11-06T02:12:46.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.687+0000 7f9dcbdb5640 1 -- 172.21.15.49:0/92031572 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dc40c8eb0 con 0x7f9dc40b29e0 2023-11-06T02:12:46.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.691+0000 7f9dc27fc640 1 -- 172.21.15.49:0/92031572 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f9dbc00bcf0 con 0x7f9dc40b29e0 2023-11-06T02:12:46.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.691+0000 7f9dc27fc640 1 --2- 172.21.15.49:0/92031572 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9db0065a90 0x7f9db0067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:46.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.691+0000 7f9dc27fc640 1 -- 172.21.15.49:0/92031572 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6264+0+0 (secure 0 0 0) 0x7f9dbc083940 con 0x7f9dc40b29e0 2023-11-06T02:12:46.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.691+0000 7f9dc9350640 1 --2- 172.21.15.49:0/92031572 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9db0065a90 0x7f9db0067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:46.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.691+0000 7f9dceedd000 1 Processor -- start 2023-11-06T02:12:46.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.691+0000 7f9dceedd000 1 -- start start 2023-11-06T02:12:46.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.691+0000 7f9dc9350640 1 --2- 172.21.15.49:0/92031572 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9db0065a90 0x7f9db0067f50 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f9db8004e30 tx=0x7f9db8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:46.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.691+0000 7f9dceedd000 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ee610b96a0 0x55ee610b9a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:46.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.691+0000 7f9dceedd000 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ee610b9fc0 0x55ee6109dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:46.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.691+0000 7f9dceedd000 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee6109e400 0x55ee6109e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:46.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.691+0000 7f9dceedd000 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee60f37a90 con 0x55ee6109e400 2023-11-06T02:12:46.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.691+0000 7f9dceedd000 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee60ec5490 con 0x55ee610b96a0 2023-11-06T02:12:46.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.691+0000 7f9dceedd000 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ee60ec2a60 con 0x55ee610b9fc0 2023-11-06T02:12:46.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.691+0000 7f9dca352640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee6109e400 0x55ee6109e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:46.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.691+0000 7f9dca352640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee6109e400 0x55ee6109e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58874/0 (socket says 172.21.15.49:58874) 2023-11-06T02:12:46.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.691+0000 7f9dca352640 1 -- 172.21.15.49:0/201951250 learned_addr learned my addr 172.21.15.49:0/201951250 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:12:46.701 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.691+0000 7f9dc9b51640 1 --2- 172.21.15.49:0/201951250 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ee610b96a0 0x55ee610b9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:46.701 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.691+0000 7f9dc9350640 1 --2- 172.21.15.49:0/201951250 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ee610b9fc0 0x55ee6109dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:46.701 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.691+0000 7f9dca352640 1 -- 172.21.15.49:0/201951250 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ee610b9fc0 msgr2=0x55ee6109dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:46.701 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.691+0000 7f9dca352640 1 --2- 172.21.15.49:0/201951250 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ee610b9fc0 0x55ee6109dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.702 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.691+0000 7f9dca352640 1 -- 172.21.15.49:0/201951250 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ee610b96a0 msgr2=0x55ee610b9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:46.702 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.691+0000 7f9dca352640 1 --2- 172.21.15.49:0/201951250 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ee610b96a0 0x55ee610b9a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.702 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.691+0000 7f9dca352640 1 -- 172.21.15.49:0/201951250 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ee60bbb070 con 0x55ee6109e400 2023-11-06T02:12:46.702 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.691+0000 7f9dca352640 1 --2- 172.21.15.49:0/201951250 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee6109e400 0x55ee6109e7e0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f9db401afa0 tx=0x7f9db4004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=818eb47a799ce9b3 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:46.702 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.691+0000 7f9dab7fe640 1 -- 172.21.15.49:0/201951250 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db40a5a40 con 0x55ee6109e400 2023-11-06T02:12:46.703 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.691+0000 7f9dab7fe640 1 -- 172.21.15.49:0/201951250 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9db40a5ba0 con 0x55ee6109e400 2023-11-06T02:12:46.703 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.691+0000 7f9dab7fe640 1 -- 172.21.15.49:0/201951250 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db40b6420 con 0x55ee6109e400 2023-11-06T02:12:46.703 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.695+0000 7f9dceedd000 1 -- 172.21.15.49:0/201951250 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee6109e400 msgr2=0x55ee6109e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:46.703 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.695+0000 7f9dceedd000 1 --2- 172.21.15.49:0/201951250 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee6109e400 0x55ee6109e7e0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f9db401afa0 tx=0x7f9db4004ac0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.703 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.695+0000 7f9dceedd000 1 -- 172.21.15.49:0/201951250 shutdown_connections 2023-11-06T02:12:46.704 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.695+0000 7f9dceedd000 1 --2- 172.21.15.49:0/201951250 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee6109e400 0x55ee6109e7e0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.704 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.695+0000 7f9dceedd000 1 --2- 172.21.15.49:0/201951250 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ee610b9fc0 0x55ee6109dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.704 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.695+0000 7f9dceedd000 1 --2- 172.21.15.49:0/201951250 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ee610b96a0 0x55ee610b9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.704 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.695+0000 7f9dceedd000 1 -- 172.21.15.49:0/201951250 >> 172.21.15.49:0/201951250 conn(0x55ee61066bc0 msgr2=0x55ee61075bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:12:46.704 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.695+0000 7f9dceedd000 1 -- 172.21.15.49:0/201951250 shutdown_connections 2023-11-06T02:12:46.705 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.695+0000 7f9dceedd000 1 -- 172.21.15.49:0/201951250 wait complete. 2023-11-06T02:12:46.705 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.695+0000 7ff1121ab640 1 Processor -- start 2023-11-06T02:12:46.705 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.695+0000 7ff1121ab640 1 -- start start 2023-11-06T02:12:46.706 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.695+0000 7f9dceedd000 1 -- start start 2023-11-06T02:12:46.706 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.695+0000 7f9dceedd000 1 Processor -- start 2023-11-06T02:12:46.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.695+0000 7ff1121ab640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff10c0cdc80 0x7ff10c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:46.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.695+0000 7ff1121ab640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff10c06f2f0 0x7ff10c069d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:46.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.695+0000 7ff1121ab640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff10c06fac0 con 0x7ff10c1611e0 2023-11-06T02:12:46.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.695+0000 7ff1121ab640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff10c160d50 con 0x7ff10c06f2f0 2023-11-06T02:12:46.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.695+0000 7ff1121ab640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff10c06f7e0 con 0x7ff10c0cdc80 2023-11-06T02:12:46.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.695+0000 7ff10affd640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff10c06f2f0 0x7ff10c069d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:46.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.695+0000 7ff10bfff640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7ff10c1611e0 0x7ff10c071160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:42728/0 (socket says 172.21.15.49:42728) 2023-11-06T02:12:46.709 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.695+0000 7ff10bfff640 1 -- 172.21.15.49:0/2080568586 learned_addr learned my addr 172.21.15.49:0/2080568586 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:12:46.709 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.695+0000 7ff10b7fe640 1 --2- 172.21.15.49:0/2080568586 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff10c0cdc80 0x7ff10c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:46.709 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.695+0000 7ff10affd640 1 -- 172.21.15.49:0/2080568586 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff10c0cdc80 msgr2=0x7ff10c06edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:46.709 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.695+0000 7ff10affd640 1 --2- 172.21.15.49:0/2080568586 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff10c0cdc80 0x7ff10c06edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.709 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.695+0000 7ff10affd640 1 -- 172.21.15.49:0/2080568586 >> v1:172.21.15.49:6789/0 conn(0x7ff10c1611e0 legacy=0x7ff10c071160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:46.710 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.695+0000 7ff10affd640 1 -- 172.21.15.49:0/2080568586 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff10c160700 con 0x7ff10c06f2f0 2023-11-06T02:12:46.712 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.695+0000 7f9dceedd000 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee6109e400 0x55ee6109de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:46.712 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.695+0000 7f9dceedd000 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ee610b96a0 0x55ee61073070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:46.713 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.695+0000 7f9dceedd000 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ee610b9fc0 0x55ee610735b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:46.713 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.695+0000 7f9dceedd000 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee60ebe350 con 0x55ee6109e400 2023-11-06T02:12:46.713 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.695+0000 7f9dceedd000 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee60e5d610 con 0x55ee610b9fc0 2023-11-06T02:12:46.713 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.695+0000 7f9dceedd000 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ee60e7b800 con 0x55ee610b96a0 2023-11-06T02:12:46.713 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.695+0000 7f9dc9b51640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee6109e400 0x55ee6109de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:46.714 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.695+0000 7f9dceedd000 10 client.0 _ll_register_callbacks cb 0x55ee60eb3960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T02:12:46.714 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.695+0000 7f9dc9b51640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee6109e400 0x55ee6109de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58892/0 (socket says 172.21.15.49:58892) 2023-11-06T02:12:46.714 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.695+0000 7f9dc9b51640 1 -- 172.21.15.49:0/1600874319 learned_addr learned my addr 172.21.15.49:0/1600874319 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:12:46.714 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.695+0000 7f9dc9350640 1 --2- 172.21.15.49:0/1600874319 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ee610b96a0 0x55ee61073070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:46.714 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.695+0000 7f9dca352640 1 --2- 172.21.15.49:0/1600874319 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ee610b9fc0 0x55ee610735b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:46.715 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.695+0000 7f9dc9b51640 1 -- 172.21.15.49:0/1600874319 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ee610b96a0 msgr2=0x55ee61073070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:46.715 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.695+0000 7f9dc9b51640 1 --2- 172.21.15.49:0/1600874319 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ee610b96a0 0x55ee61073070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.715 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.695+0000 7f9dc9b51640 1 -- 172.21.15.49:0/1600874319 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ee610b9fc0 msgr2=0x55ee610735b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:46.715 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.695+0000 7f9dc9b51640 1 --2- 172.21.15.49:0/1600874319 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ee610b9fc0 0x55ee610735b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.715 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.695+0000 7f9dc9b51640 1 -- 172.21.15.49:0/1600874319 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dc415e9d0 con 0x55ee6109e400 2023-11-06T02:12:46.716 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.695+0000 7f9dc9b51640 1 --2- 172.21.15.49:0/1600874319 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee6109e400 0x55ee6109de90 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc054d20 tx=0x7f9dbc054e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ad4694c334a5bda server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:46.716 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.695+0000 7f9da8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-11-06T02:12:46.716 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.695+0000 7f9da8ff9640 1 -- 172.21.15.49:0/1600874319 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dbc06c1d0 con 0x55ee6109e400 2023-11-06T02:12:46.716 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.695+0000 7f9da8ff9640 1 -- 172.21.15.49:0/1600874319 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(68..68 src has 1..68) v4 ==== 6264+0+0 (secure 0 0 0) 0x7f9dbc06c330 con 0x55ee6109e400 2023-11-06T02:12:46.716 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.695+0000 7f9dceedd000 1 -- 172.21.15.49:0/1600874319 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ee60e8dc80 con 0x55ee6109e400 2023-11-06T02:12:46.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.695+0000 7f9da8ff9640 1 -- 172.21.15.49:0/1600874319 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9dbc054280 con 0x55ee6109e400 2023-11-06T02:12:46.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.695+0000 7f9dceedd000 10 client.6903 fetch_fsmap learned FSMap version 36 2023-11-06T02:12:46.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.695+0000 7f9dceedd000 1 -- 172.21.15.49:0/1600874319 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) v3 -- 0x55ee60e832d0 con 0x55ee6109e400 2023-11-06T02:12:46.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.695+0000 7f9da8ff9640 1 -- 172.21.15.49:0/1600874319 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ee60e8dc80 con 0x55ee6109e400 2023-11-06T02:12:46.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.695+0000 7f9da8ff9640 1 -- 172.21.15.49:0/1600874319 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 36) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f9dbc05c8e0 con 0x55ee6109e400 2023-11-06T02:12:46.718 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.695+0000 7f9dceedd000 10 client.6903 fetch_fsmap finished waiting for FSMap version 36 2023-11-06T02:12:46.718 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.695+0000 7f9dceedd000 10 client.6903 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2023-11-06T02:12:46.718 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.695+0000 7f9dceedd000 20 client.6903 populate_metadata read hostname 'smithi049' 2023-11-06T02:12:46.718 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.695+0000 7f9dceedd000 1 --2- 172.21.15.49:0/1600874319 >> [v2:172.21.15.49:6834/3698847494,v1:172.21.15.49:6835/3698847494] conn(0x55ee610742f0 0x55ee610cb750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:46.718 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.695+0000 7f9dceedd000 4 client.6903 mds_command: new command op to 6807 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T02:12:46.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.695+0000 7f9dceedd000 1 -- 172.21.15.49:0/1600874319 --> [v2:172.21.15.49:6834/3698847494,v1:172.21.15.49:6835/3698847494] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ee60ebcca0 con 0x55ee610742f0 2023-11-06T02:12:46.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.695+0000 7f9dc9350640 1 --2- 172.21.15.49:0/1600874319 >> [v2:172.21.15.49:6834/3698847494,v1:172.21.15.49:6835/3698847494] conn(0x55ee610742f0 0x55ee610cb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:46.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.699+0000 7f9dc9350640 1 --2- 172.21.15.49:0/1600874319 >> [v2:172.21.15.49:6834/3698847494,v1:172.21.15.49:6835/3698847494] conn(0x55ee610742f0 0x55ee610cb750 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f84bfc1ea7fe8eca server_cookie=ed350064b0895000 in_seq=0 out_seq=0 2023-11-06T02:12:46.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.699+0000 7f9da8ff9640 10 client.6903 ms_handle_connect on v2:172.21.15.49:6834/3698847494 2023-11-06T02:12:46.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.699+0000 7ff10affd640 1 --2- 172.21.15.49:0/2080568586 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff10c06f2f0 0x7ff10c069d70 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7ff1000097c0 tx=0x7ff10001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55a0b4507cbaaf33 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:46.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.699+0000 7ff10a7fc640 1 -- 172.21.15.49:0/2080568586 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1000ab600 con 0x7ff10c06f2f0 2023-11-06T02:12:46.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.699+0000 7ff10a7fc640 1 -- 172.21.15.49:0/2080568586 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1000a7050 con 0x7ff10c06f2f0 2023-11-06T02:12:46.721 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.699+0000 7ff10a7fc640 1 -- 172.21.15.49:0/2080568586 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1000ab980 con 0x7ff10c06f2f0 2023-11-06T02:12:46.722 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.699+0000 7f9da8ff9640 1 -- 172.21.15.49:0/1600874319 <== mds.0 v2:172.21.15.49:6834/3698847494 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55ee60ebcca0 con 0x55ee610742f0 2023-11-06T02:12:46.722 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.699+0000 7f9da8ff9640 10 client.6903 handle_command_reply: tid=0 2023-11-06T02:12:46.722 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.699+0000 7f9dceedd000 1 client.6903 shutdown 2023-11-06T02:12:46.722 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.699+0000 7f9dceedd000 1 -- 172.21.15.49:0/1600874319 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee6109e400 msgr2=0x55ee6109de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:46.723 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.699+0000 7f9dceedd000 1 --2- 172.21.15.49:0/1600874319 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee6109e400 0x55ee6109de90 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc054d20 tx=0x7f9dbc054e50 comp rx=0 tx=0).stop 2023-11-06T02:12:46.723 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.699+0000 7f9dceedd000 1 -- 172.21.15.49:0/1600874319 shutdown_connections 2023-11-06T02:12:46.723 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.699+0000 7f9dceedd000 1 --2- 172.21.15.49:0/1600874319 >> [v2:172.21.15.49:6834/3698847494,v1:172.21.15.49:6835/3698847494] conn(0x55ee610742f0 0x55ee610cb750 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.723 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.699+0000 7f9dceedd000 1 --2- 172.21.15.49:0/1600874319 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ee610b9fc0 0x55ee610735b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.723 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.699+0000 7f9da8ff9640 0 client.6903 ms_handle_reset on v2:172.21.15.49:6834/3698847494 2023-11-06T02:12:46.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.699+0000 7f9dceedd000 1 --2- 172.21.15.49:0/1600874319 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ee610b96a0 0x55ee61073070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.699+0000 7f9dceedd000 1 --2- 172.21.15.49:0/1600874319 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee6109e400 0x55ee6109de90 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.699+0000 7f9dceedd000 1 -- 172.21.15.49:0/1600874319 >> 172.21.15.49:0/1600874319 conn(0x55ee61066bc0 msgr2=0x55ee6109f280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:12:46.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.699+0000 7f9dceedd000 1 -- 172.21.15.49:0/1600874319 shutdown_connections 2023-11-06T02:12:46.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.699+0000 7f9dceedd000 1 -- 172.21.15.49:0/1600874319 wait complete. 2023-11-06T02:12:46.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.699+0000 7f9dceedd000 20 client.6903 trim_cache size 0 max 16384 2023-11-06T02:12:46.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.699+0000 7ff1121ab640 1 -- 172.21.15.49:0/2080568586 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff10c06f2f0 msgr2=0x7ff10c069d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:46.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.699+0000 7ff1121ab640 1 --2- 172.21.15.49:0/2080568586 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff10c06f2f0 0x7ff10c069d70 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7ff1000097c0 tx=0x7ff10001c3f0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.699+0000 7ff1121ab640 1 -- 172.21.15.49:0/2080568586 shutdown_connections 2023-11-06T02:12:46.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.699+0000 7ff1121ab640 1 --2- 172.21.15.49:0/2080568586 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff10c06f2f0 0x7ff10c069d70 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.699+0000 7ff1121ab640 1 --2- 172.21.15.49:0/2080568586 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff10c0cdc80 0x7ff10c06edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.699+0000 7ff1121ab640 1 -- 172.21.15.49:0/2080568586 >> 172.21.15.49:0/2080568586 conn(0x7ff10c0cc050 msgr2=0x7ff10c0cc460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:12:46.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.699+0000 7ff1121ab640 1 -- 172.21.15.49:0/2080568586 shutdown_connections 2023-11-06T02:12:46.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.699+0000 7ff1121ab640 1 -- 172.21.15.49:0/2080568586 wait complete. 2023-11-06T02:12:46.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.699+0000 7ff1121ab640 1 Processor -- start 2023-11-06T02:12:46.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.699+0000 7ff1121ab640 1 -- start start 2023-11-06T02:12:46.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.699+0000 7ff1121ab640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff10c06f2f0 0x7ff10c150f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:46.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.699+0000 7ff1121ab640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff10c0cdc80 0x7ff10c14c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:46.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.699+0000 7ff1121ab640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff10c1611e0 0x7ff10c0c4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:46.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.699+0000 7ff1121ab640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff10c1502e0 con 0x7ff10c0cdc80 2023-11-06T02:12:46.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.699+0000 7ff1121ab640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff10c0c3fe0 con 0x7ff10c1611e0 2023-11-06T02:12:46.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.699+0000 7ff1121ab640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff10c151450 con 0x7ff10c06f2f0 2023-11-06T02:12:46.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.699+0000 7ff10affd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff10c0cdc80 0x7ff10c14c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:46.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.699+0000 7ff10affd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff10c0cdc80 0x7ff10c14c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58902/0 (socket says 172.21.15.49:58902) 2023-11-06T02:12:46.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.699+0000 7ff10affd640 1 -- 172.21.15.49:0/1316750628 learned_addr learned my addr 172.21.15.49:0/1316750628 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:12:46.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.699+0000 7ff10b7fe640 1 --2- 172.21.15.49:0/1316750628 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff10c06f2f0 0x7ff10c150f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:46.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.699+0000 7ff10bfff640 1 --2- 172.21.15.49:0/1316750628 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff10c1611e0 0x7ff10c0c4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:46.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.699+0000 7ff10affd640 1 -- 172.21.15.49:0/1316750628 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff10c06f2f0 msgr2=0x7ff10c150f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:46.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.699+0000 7ff10affd640 1 --2- 172.21.15.49:0/1316750628 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff10c06f2f0 0x7ff10c150f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.699+0000 7ff10affd640 1 -- 172.21.15.49:0/1316750628 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff10c1611e0 msgr2=0x7ff10c0c4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:46.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.699+0000 7ff10affd640 1 --2- 172.21.15.49:0/1316750628 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff10c1611e0 0x7ff10c0c4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.699+0000 7ff10affd640 1 -- 172.21.15.49:0/1316750628 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff100009420 con 0x7ff10c0cdc80 2023-11-06T02:12:46.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.699+0000 7ff10bfff640 1 --2- 172.21.15.49:0/1316750628 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff10c1611e0 0x7ff10c0c4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:12:46.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.703+0000 7ff10affd640 1 --2- 172.21.15.49:0/1316750628 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff10c0cdc80 0x7ff10c14c9c0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7ff1000ae040 tx=0x7ff100019190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:46.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.703+0000 7ff0f3fff640 1 -- 172.21.15.49:0/1316750628 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1000b4840 con 0x7ff10c0cdc80 2023-11-06T02:12:46.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.703+0000 7ff0f3fff640 1 -- 172.21.15.49:0/1316750628 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1000b49c0 con 0x7ff10c0cdc80 2023-11-06T02:12:46.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.703+0000 7ff1121ab640 1 -- 172.21.15.49:0/1316750628 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff10c150000 con 0x7ff10c0cdc80 2023-11-06T02:12:46.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.703+0000 7ff0f3fff640 1 -- 172.21.15.49:0/1316750628 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1000b4d70 con 0x7ff10c0cdc80 2023-11-06T02:12:46.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.703+0000 7ff1121ab640 1 -- 172.21.15.49:0/1316750628 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff10c0c3c70 con 0x7ff10c0cdc80 2023-11-06T02:12:46.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.703+0000 7ff1121ab640 1 -- 172.21.15.49:0/1316750628 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7ff10c0c4150 con 0x7ff10c0cdc80 2023-11-06T02:12:46.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.703+0000 7ff0f3fff640 1 -- 172.21.15.49:0/1316750628 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7ff1000b2040 con 0x7ff10c0cdc80 2023-11-06T02:12:46.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.703+0000 7ff0f3fff640 1 --2- 172.21.15.49:0/1316750628 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff0f8065a60 0x7ff0f8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:46.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.703+0000 7ff0f3fff640 1 -- 172.21.15.49:0/1316750628 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6264+0+0 (secure 0 0 0) 0x7ff10011dae0 con 0x7ff10c0cdc80 2023-11-06T02:12:46.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.703+0000 7ff0f3fff640 1 -- 172.21.15.49:0/1316750628 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+514 (secure 0 0 0) 0x7ff10011e2e0 con 0x7ff10c0cdc80 2023-11-06T02:12:46.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.703+0000 7ff10b7fe640 1 --2- 172.21.15.49:0/1316750628 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff0f8065a60 0x7ff0f8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:46.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.703+0000 7ff1121ab640 1 -- 172.21.15.49:0/1316750628 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({log-info=0+}) v3 -- 0x7ff10c04e8a0 con 0x7ff10c0cdc80 2023-11-06T02:12:46.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.703+0000 7ff10b7fe640 1 --2- 172.21.15.49:0/1316750628 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff0f8065a60 0x7ff0f8067f20 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc00b9d0 tx=0x7ff0fc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:46.741 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.715+0000 7f9dceedd000 1 Processor -- start 2023-11-06T02:12:46.741 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.715+0000 7f9dceedd000 1 -- start start 2023-11-06T02:12:46.741 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.715+0000 7f9dceedd000 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee6109e400 0x55ee61072bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:46.742 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.715+0000 7f9dceedd000 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ee610b96a0 0x55ee61073110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:46.742 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.715+0000 7f9dceedd000 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ee610b9fc0 0x55ee610cc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:46.742 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.715+0000 7f9dceedd000 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee60b7bcf0 con 0x55ee6109e400 2023-11-06T02:12:46.742 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.715+0000 7f9dceedd000 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee60e6b560 con 0x55ee610b96a0 2023-11-06T02:12:46.742 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.715+0000 7f9dceedd000 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ee60e8d180 con 0x55ee610b9fc0 2023-11-06T02:12:46.743 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.715+0000 7f9dc9b51640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee6109e400 0x55ee61072bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:46.743 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.715+0000 7f9dc9b51640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee6109e400 0x55ee61072bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58912/0 (socket says 172.21.15.49:58912) 2023-11-06T02:12:46.743 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.715+0000 7f9dc9b51640 1 -- 172.21.15.49:0/671114728 learned_addr learned my addr 172.21.15.49:0/671114728 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:12:46.743 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.715+0000 7f9dc9350640 1 --2- 172.21.15.49:0/671114728 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ee610b96a0 0x55ee61073110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:46.743 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.715+0000 7f9dca352640 1 --2- 172.21.15.49:0/671114728 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ee610b9fc0 0x55ee610cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:46.744 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.715+0000 7f9dc9b51640 1 -- 172.21.15.49:0/671114728 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ee610b9fc0 msgr2=0x55ee610cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:46.744 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.715+0000 7f9dc9b51640 1 --2- 172.21.15.49:0/671114728 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ee610b9fc0 0x55ee610cc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.744 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.715+0000 7f9dc9b51640 1 -- 172.21.15.49:0/671114728 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ee610b96a0 msgr2=0x55ee61073110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:46.744 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.715+0000 7f9dc9b51640 1 --2- 172.21.15.49:0/671114728 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ee610b96a0 0x55ee61073110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.744 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.715+0000 7f9dc9b51640 1 -- 172.21.15.49:0/671114728 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ee60e69620 con 0x55ee6109e400 2023-11-06T02:12:46.745 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.715+0000 7f9dc9350640 1 --2- 172.21.15.49:0/671114728 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ee610b96a0 0x55ee61073110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:12:46.745 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.715+0000 7f9dca352640 1 --2- 172.21.15.49:0/671114728 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ee610b9fc0 0x55ee610cc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:12:46.745 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.715+0000 7f9dc9b51640 1 --2- 172.21.15.49:0/671114728 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee6109e400 0x55ee61072bd0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc05c840 tx=0x7f9dbc057000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e5e96a38e6f9357 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:46.745 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.715+0000 7f9dabfff640 1 -- 172.21.15.49:0/671114728 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dbc054280 con 0x55ee6109e400 2023-11-06T02:12:46.745 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.715+0000 7f9dabfff640 1 -- 172.21.15.49:0/671114728 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dbc058bc0 con 0x55ee6109e400 2023-11-06T02:12:46.746 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.715+0000 7f9dabfff640 1 -- 172.21.15.49:0/671114728 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dbc0514a0 con 0x55ee6109e400 2023-11-06T02:12:46.746 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.715+0000 7f9dceedd000 1 -- 172.21.15.49:0/671114728 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee6109e400 msgr2=0x55ee61072bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:46.746 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.715+0000 7f9dceedd000 1 --2- 172.21.15.49:0/671114728 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee6109e400 0x55ee61072bd0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc05c840 tx=0x7f9dbc057000 comp rx=0 tx=0).stop 2023-11-06T02:12:46.746 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.715+0000 7f9dceedd000 1 -- 172.21.15.49:0/671114728 shutdown_connections 2023-11-06T02:12:46.746 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.715+0000 7f9dceedd000 1 --2- 172.21.15.49:0/671114728 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ee610b9fc0 0x55ee610cc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.747 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.715+0000 7f9dceedd000 1 --2- 172.21.15.49:0/671114728 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ee610b96a0 0x55ee61073110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.747 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.715+0000 7f9dceedd000 1 --2- 172.21.15.49:0/671114728 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee6109e400 0x55ee61072bd0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.747 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.715+0000 7f9dceedd000 1 -- 172.21.15.49:0/671114728 >> 172.21.15.49:0/671114728 conn(0x55ee61066bc0 msgr2=0x55ee610724c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:12:46.747 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.715+0000 7f9dceedd000 1 -- 172.21.15.49:0/671114728 shutdown_connections 2023-11-06T02:12:46.747 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.715+0000 7f9dceedd000 1 -- 172.21.15.49:0/671114728 wait complete. 2023-11-06T02:12:46.748 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.715+0000 7f9dceedd000 1 -- start start 2023-11-06T02:12:46.748 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.715+0000 7f9dceedd000 1 Processor -- start 2023-11-06T02:12:46.748 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.719+0000 7f9dceedd000 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee6109e400 0x55ee61115c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:46.748 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.719+0000 7f9dceedd000 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ee610b96a0 0x55ee6109a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:46.748 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.719+0000 7f9dceedd000 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ee610b9fc0 0x55ee6109abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:46.748 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.719+0000 7f9dceedd000 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee60e80ae0 con 0x55ee6109e400 2023-11-06T02:12:46.749 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.719+0000 7f9dceedd000 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee60e8cfc0 con 0x55ee610b9fc0 2023-11-06T02:12:46.749 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.719+0000 7f9dceedd000 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ee60e82d20 con 0x55ee610b96a0 2023-11-06T02:12:46.749 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.719+0000 7f9dceedd000 10 client.0 _ll_register_callbacks cb 0x55ee60ec40a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T02:12:46.749 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.719+0000 7f9dc9b51640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee6109e400 0x55ee61115c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:46.749 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.719+0000 7f9dc9b51640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee6109e400 0x55ee61115c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58920/0 (socket says 172.21.15.49:58920) 2023-11-06T02:12:46.750 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.719+0000 7f9dc9b51640 1 -- 172.21.15.49:0/3799030530 learned_addr learned my addr 172.21.15.49:0/3799030530 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:12:46.750 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.719+0000 7f9dc9350640 1 --2- 172.21.15.49:0/3799030530 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ee610b96a0 0x55ee6109a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:46.750 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.719+0000 7f9dca352640 1 --2- 172.21.15.49:0/3799030530 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ee610b9fc0 0x55ee6109abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:46.750 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.719+0000 7f9dc9b51640 1 -- 172.21.15.49:0/3799030530 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ee610b96a0 msgr2=0x55ee6109a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:46.750 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.719+0000 7f9dc9b51640 1 --2- 172.21.15.49:0/3799030530 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ee610b96a0 0x55ee6109a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.751 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.719+0000 7f9dc9b51640 1 -- 172.21.15.49:0/3799030530 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ee610b9fc0 msgr2=0x55ee6109abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:46.751 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.719+0000 7f9dc9b51640 1 --2- 172.21.15.49:0/3799030530 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ee610b9fc0 0x55ee6109abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.751 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.719+0000 7f9dc9b51640 1 -- 172.21.15.49:0/3799030530 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ee60e832d0 con 0x55ee6109e400 2023-11-06T02:12:46.751 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.719+0000 7f9dc9350640 1 --2- 172.21.15.49:0/3799030530 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ee610b96a0 0x55ee6109a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:12:46.751 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.719+0000 7f9dca352640 1 --2- 172.21.15.49:0/3799030530 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ee610b9fc0 0x55ee6109abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:12:46.752 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.719+0000 7f9dc9b51640 1 --2- 172.21.15.49:0/3799030530 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee6109e400 0x55ee61115c90 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc006f10 tx=0x7f9dbc004710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f146392c2d91d457 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:46.752 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.719+0000 7f9da8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-11-06T02:12:46.752 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.719+0000 7f9da8ff9640 1 -- 172.21.15.49:0/3799030530 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dbc054480 con 0x55ee6109e400 2023-11-06T02:12:46.752 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.719+0000 7f9da8ff9640 1 -- 172.21.15.49:0/3799030530 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(68..68 src has 1..68) v4 ==== 6264+0+0 (secure 0 0 0) 0x7f9dbc050b80 con 0x55ee6109e400 2023-11-06T02:12:46.752 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.719+0000 7f9dceedd000 1 -- 172.21.15.49:0/3799030530 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ee60e6dd50 con 0x55ee6109e400 2023-11-06T02:12:46.752 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.719+0000 7f9da8ff9640 1 -- 172.21.15.49:0/3799030530 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9dbc050d60 con 0x55ee6109e400 2023-11-06T02:12:46.753 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.719+0000 7f9dceedd000 10 client.6912 fetch_fsmap learned FSMap version 36 2023-11-06T02:12:46.753 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.719+0000 7f9dceedd000 1 -- 172.21.15.49:0/3799030530 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) v3 -- 0x55ee60e5d190 con 0x55ee6109e400 2023-11-06T02:12:46.753 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.719+0000 7f9da8ff9640 1 -- 172.21.15.49:0/3799030530 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ee60e6dd50 con 0x55ee6109e400 2023-11-06T02:12:46.753 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.719+0000 7f9da8ff9640 1 -- 172.21.15.49:0/3799030530 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 36) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f9dbc068240 con 0x55ee6109e400 2023-11-06T02:12:46.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.719+0000 7f9dceedd000 10 client.6912 fetch_fsmap finished waiting for FSMap version 36 2023-11-06T02:12:46.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.719+0000 7f9dceedd000 10 client.6912 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2023-11-06T02:12:46.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.719+0000 7f9dceedd000 20 client.6912 populate_metadata read hostname 'smithi049' 2023-11-06T02:12:46.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.719+0000 7f9dceedd000 1 --2- 172.21.15.49:0/3799030530 >> [v2:172.21.15.49:6834/3698847494,v1:172.21.15.49:6835/3698847494] conn(0x55ee6109b5e0 0x55ee6109b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:46.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.719+0000 7f9dceedd000 4 client.6912 mds_command: new command op to 6807 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2023-11-06T02:12:46.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.719+0000 7f9dceedd000 1 -- 172.21.15.49:0/3799030530 --> [v2:172.21.15.49:6834/3698847494,v1:172.21.15.49:6835/3698847494] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) v1 -- 0x55ee60eb73d0 con 0x55ee6109b5e0 2023-11-06T02:12:46.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.719+0000 7f9dc9350640 1 --2- 172.21.15.49:0/3799030530 >> [v2:172.21.15.49:6834/3698847494,v1:172.21.15.49:6835/3698847494] conn(0x55ee6109b5e0 0x55ee6109b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:46.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.719+0000 7f9da8ff9640 10 client.6912 ms_handle_connect on v2:172.21.15.49:6834/3698847494 2023-11-06T02:12:46.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.719+0000 7f9dc9350640 1 --2- 172.21.15.49:0/3799030530 >> [v2:172.21.15.49:6834/3698847494,v1:172.21.15.49:6835/3698847494] conn(0x55ee6109b5e0 0x55ee6109b9f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af8609f553da42b3 server_cookie=ed36b43b900e5128 in_seq=0 out_seq=0 2023-11-06T02:12:46.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.719+0000 7f9da8ff9640 1 -- 172.21.15.49:0/3799030530 <== mds.0 v2:172.21.15.49:6834/3698847494 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55ee60eb73d0 con 0x55ee6109b5e0 2023-11-06T02:12:46.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.719+0000 7f9da8ff9640 10 client.6912 handle_command_reply: tid=0 2023-11-06T02:12:46.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.719+0000 7f9dceedd000 1 client.6912 shutdown 2023-11-06T02:12:46.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.719+0000 7f9dceedd000 1 -- 172.21.15.49:0/3799030530 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee6109e400 msgr2=0x55ee61115c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:46.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.719+0000 7f9dceedd000 1 --2- 172.21.15.49:0/3799030530 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee6109e400 0x55ee61115c90 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc006f10 tx=0x7f9dbc004710 comp rx=0 tx=0).stop 2023-11-06T02:12:46.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.719+0000 7f9dceedd000 1 -- 172.21.15.49:0/3799030530 shutdown_connections 2023-11-06T02:12:46.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.719+0000 7f9dceedd000 1 --2- 172.21.15.49:0/3799030530 >> [v2:172.21.15.49:6834/3698847494,v1:172.21.15.49:6835/3698847494] conn(0x55ee6109b5e0 0x55ee6109b9f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.723+0000 7f9dceedd000 1 --2- 172.21.15.49:0/3799030530 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ee610b9fc0 0x55ee6109abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.723+0000 7f9dceedd000 1 --2- 172.21.15.49:0/3799030530 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ee610b96a0 0x55ee6109a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.723+0000 7f9dceedd000 1 --2- 172.21.15.49:0/3799030530 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ee6109e400 0x55ee61115c90 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.723+0000 7f9dceedd000 1 -- 172.21.15.49:0/3799030530 >> 172.21.15.49:0/3799030530 conn(0x55ee61066bc0 msgr2=0x55ee610724c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:12:46.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.723+0000 7f9da8ff9640 0 client.6912 ms_handle_reset on v2:172.21.15.49:6834/3698847494 2023-11-06T02:12:46.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.723+0000 7f9dceedd000 1 -- 172.21.15.49:0/3799030530 shutdown_connections 2023-11-06T02:12:46.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.723+0000 7f9dceedd000 1 -- 172.21.15.49:0/3799030530 wait complete. 2023-11-06T02:12:46.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.723+0000 7f9dceedd000 20 client.6912 trim_cache size 0 max 16384 2023-11-06T02:12:46.760 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-11-06T02:12:46.760 INFO:teuthology.orchestra.run.smithi049.stdout: "return_code": 0, 2023-11-06T02:12:46.761 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_tag": "8f82abe6-c08d-40a5-a969-67de1c6c4ed5", 2023-11-06T02:12:46.761 INFO:teuthology.orchestra.run.smithi049.stdout: "mode": "asynchronous" 2023-11-06T02:12:46.761 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-11-06T02:12:46.762 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.723+0000 7f9dcbdb5640 1 -- 172.21.15.49:0/92031572 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9db0065a90 msgr2=0x7f9db0067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:46.762 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.723+0000 7f9dcbdb5640 1 --2- 172.21.15.49:0/92031572 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9db0065a90 0x7f9db0067f50 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f9db8004e30 tx=0x7f9db8009040 comp rx=0 tx=0).stop 2023-11-06T02:12:46.762 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.723+0000 7f9dcbdb5640 1 -- 172.21.15.49:0/92031572 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9dc40b29e0 msgr2=0x7f9dc414f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:46.763 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.723+0000 7f9dcbdb5640 1 --2- 172.21.15.49:0/92031572 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9dc40b29e0 0x7f9dc414f650 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc00d800 tx=0x7f9dbc00dcc0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.763 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.723+0000 7f9dcbdb5640 1 -- 172.21.15.49:0/92031572 shutdown_connections 2023-11-06T02:12:46.763 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.723+0000 7f9dcbdb5640 1 --2- 172.21.15.49:0/92031572 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9db0065a90 0x7f9db0067f50 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.763 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.723+0000 7f9dcbdb5640 1 --2- 172.21.15.49:0/92031572 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9dc415a930 0x7f9dc40c93d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.763 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.723+0000 7f9dcbdb5640 1 --2- 172.21.15.49:0/92031572 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9dc40b6020 0x7f9dc40b0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.764 INFO:tasks.ceph.mds.c.smithi049.stderr:2023-11-06T02:12:46.723+0000 7fbec3de3640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000001 2023-11-06T02:12:46.764 INFO:tasks.ceph.mds.c.smithi049.stderr:2023-11-06T02:12:46.723+0000 7fbec3de3640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000001 2023-11-06T02:12:46.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.723+0000 7f9dcbdb5640 1 --2- 172.21.15.49:0/92031572 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9dc40b29e0 0x7f9dc414f650 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.723+0000 7f9dcbdb5640 1 -- 172.21.15.49:0/92031572 >> 172.21.15.49:0/92031572 conn(0x7f9dc40cd2d0 msgr2=0x7f9dc4159820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:12:46.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.723+0000 7f9dcbdb5640 1 -- 172.21.15.49:0/92031572 shutdown_connections 2023-11-06T02:12:46.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.723+0000 7f9dcbdb5640 1 -- 172.21.15.49:0/92031572 wait complete. 2023-11-06T02:12:46.766 INFO:tasks.ceph.mds.c.smithi049.stderr:2023-11-06T02:12:46.723+0000 7fbec3de3640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000002 2023-11-06T02:12:46.766 INFO:tasks.ceph.mds.c.smithi049.stderr:2023-11-06T02:12:46.723+0000 7fbec3de3640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000002 2023-11-06T02:12:46.767 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 scrub status 2023-11-06T02:12:46.889 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.891+0000 7f1ebbffb640 1 Processor -- start 2023-11-06T02:12:46.889 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.891+0000 7f1ebbffb640 1 -- start start 2023-11-06T02:12:46.891 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.891+0000 7f1ebbffb640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1eb4070160 0x7f1eb40705a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:46.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.891+0000 7f1ebbffb640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f1eb406c230 0x7f1eb406c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:46.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.891+0000 7f1ebbffb640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1eb40c8c80 con 0x7f1eb414ffc0 2023-11-06T02:12:46.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.891+0000 7f1ebbffb640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1eb40ae4d0 con 0x7f1eb406c230 2023-11-06T02:12:46.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.891+0000 7f1ebbffb640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb41594f0 con 0x7f1eb4070160 2023-11-06T02:12:46.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.891+0000 7f1eba598640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f1eb406c230 0x7f1eb406c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:46.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.891+0000 7f1eba598640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f1eb406c230 0x7f1eb406c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58930/0 (socket says 172.21.15.49:58930) 2023-11-06T02:12:46.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.891+0000 7f1eba598640 1 -- 172.21.15.49:0/3477272959 learned_addr learned my addr 172.21.15.49:0/3477272959 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:12:46.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.891+0000 7f1eb9596640 1 --2- 172.21.15.49:0/3477272959 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1eb4070160 0x7f1eb40705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:46.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.891+0000 7f1eba598640 1 -- 172.21.15.49:0/3477272959 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1eb4070160 msgr2=0x7f1eb40705a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:46.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.891+0000 7f1eba598640 1 --2- 172.21.15.49:0/3477272959 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1eb4070160 0x7f1eb40705a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.891+0000 7f1eba598640 1 -- 172.21.15.49:0/3477272959 >> v1:172.21.15.150:6789/0 conn(0x7f1eb414ffc0 legacy=0x7f1eb40ada30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:46.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.891+0000 7f1eba598640 1 -- 172.21.15.49:0/3477272959 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eb4070ae0 con 0x7f1eb406c230 2023-11-06T02:12:46.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.895+0000 7f1eba598640 1 --2- 172.21.15.49:0/3477272959 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1eb406c230 0x7f1eb406c6c0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f1eac009e50 tx=0x7f1eac01db20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28f3274043635cf server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:46.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.895+0000 7f1eb8d95640 1 -- 172.21.15.49:0/3477272959 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eac0a6850 con 0x7f1eb406c230 2023-11-06T02:12:46.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.895+0000 7f1eb8d95640 1 -- 172.21.15.49:0/3477272959 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eac0a69b0 con 0x7f1eb406c230 2023-11-06T02:12:46.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.895+0000 7f1eb8d95640 1 -- 172.21.15.49:0/3477272959 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eac0a6d40 con 0x7f1eb406c230 2023-11-06T02:12:46.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.895+0000 7f1ebbffb640 1 -- 172.21.15.49:0/3477272959 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1eb406c230 msgr2=0x7f1eb406c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:46.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.895+0000 7f1ebbffb640 1 --2- 172.21.15.49:0/3477272959 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1eb406c230 0x7f1eb406c6c0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f1eac009e50 tx=0x7f1eac01db20 comp rx=0 tx=0).stop 2023-11-06T02:12:46.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.895+0000 7f1ebbffb640 1 -- 172.21.15.49:0/3477272959 shutdown_connections 2023-11-06T02:12:46.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.895+0000 7f1ebbffb640 1 --2- 172.21.15.49:0/3477272959 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1eb4070160 0x7f1eb40705a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.895+0000 7f1ebbffb640 1 --2- 172.21.15.49:0/3477272959 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1eb406c230 0x7f1eb406c6c0 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.895+0000 7f1ebbffb640 1 -- 172.21.15.49:0/3477272959 >> 172.21.15.49:0/3477272959 conn(0x7f1eb406a370 msgr2=0x7f1eb40ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:12:46.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.895+0000 7f1ebbffb640 1 -- 172.21.15.49:0/3477272959 shutdown_connections 2023-11-06T02:12:46.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.895+0000 7f1ebbffb640 1 -- 172.21.15.49:0/3477272959 wait complete. 2023-11-06T02:12:46.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.895+0000 7f1ebbffb640 1 Processor -- start 2023-11-06T02:12:46.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.895+0000 7f1ebbffb640 1 -- start start 2023-11-06T02:12:46.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.895+0000 7f1ebbffb640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1eb406c230 0x7f1eb4145240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:46.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.895+0000 7f1ebbffb640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1eb4070160 0x7f1eb414ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:46.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.895+0000 7f1ebbffb640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1eb414ffc0 0x7f1eb414a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:46.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.895+0000 7f1ebbffb640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb4149230 con 0x7f1eb406c230 2023-11-06T02:12:46.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.895+0000 7f1ebbffb640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb4145780 con 0x7f1eb4070160 2023-11-06T02:12:46.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.895+0000 7f1ebbffb640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb4149780 con 0x7f1eb414ffc0 2023-11-06T02:12:46.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.899+0000 7f1eb9d97640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1eb406c230 0x7f1eb4145240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:46.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.899+0000 7f1eb9d97640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1eb406c230 0x7f1eb4145240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58932/0 (socket says 172.21.15.49:58932) 2023-11-06T02:12:46.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.899+0000 7f1eb9d97640 1 -- 172.21.15.49:0/2076180871 learned_addr learned my addr 172.21.15.49:0/2076180871 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:12:46.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.899+0000 7f1eba598640 1 --2- 172.21.15.49:0/2076180871 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1eb414ffc0 0x7f1eb414a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:46.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.899+0000 7f1eb9596640 1 --2- 172.21.15.49:0/2076180871 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1eb4070160 0x7f1eb414ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:46.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.899+0000 7f1eb9d97640 1 -- 172.21.15.49:0/2076180871 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1eb414ffc0 msgr2=0x7f1eb414a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:46.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.899+0000 7f1eb9d97640 1 --2- 172.21.15.49:0/2076180871 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1eb414ffc0 0x7f1eb414a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.899+0000 7f1eb9d97640 1 -- 172.21.15.49:0/2076180871 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1eb4070160 msgr2=0x7f1eb414ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:46.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.899+0000 7f1eb9d97640 1 --2- 172.21.15.49:0/2076180871 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1eb4070160 0x7f1eb414ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.899+0000 7f1eb9d97640 1 -- 172.21.15.49:0/2076180871 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eac009b30 con 0x7f1eb406c230 2023-11-06T02:12:46.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.899+0000 7f1eb9d97640 1 --2- 172.21.15.49:0/2076180871 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1eb406c230 0x7f1eb4145240 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f1eb0001e50 tx=0x7f1eb0001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:46.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.899+0000 7f1eaa7fc640 1 -- 172.21.15.49:0/2076180871 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb00052d0 con 0x7f1eb406c230 2023-11-06T02:12:46.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.899+0000 7f1eaa7fc640 1 -- 172.21.15.49:0/2076180871 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eb0005430 con 0x7f1eb406c230 2023-11-06T02:12:46.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.899+0000 7f1ebbffb640 1 -- 172.21.15.49:0/2076180871 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1eb414b080 con 0x7f1eb406c230 2023-11-06T02:12:46.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.899+0000 7f1eaa7fc640 1 -- 172.21.15.49:0/2076180871 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb00108e0 con 0x7f1eb406c230 2023-11-06T02:12:46.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.899+0000 7f1ebbffb640 1 -- 172.21.15.49:0/2076180871 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1eb4151250 con 0x7f1eb406c230 2023-11-06T02:12:46.912 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.899+0000 7f1eaa7fc640 1 -- 172.21.15.49:0/2076180871 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f1eb0010a40 con 0x7f1eb406c230 2023-11-06T02:12:46.912 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.899+0000 7f1eaa7fc640 1 --2- 172.21.15.49:0/2076180871 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1e98065a60 0x7f1e98067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:46.913 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.899+0000 7f1eaa7fc640 1 -- 172.21.15.49:0/2076180871 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6264+0+0 (secure 0 0 0) 0x7f1eb0085190 con 0x7f1eb406c230 2023-11-06T02:12:46.913 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.899+0000 7f1eb9596640 1 --2- 172.21.15.49:0/2076180871 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1e98065a60 0x7f1e98067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:46.913 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.903+0000 7f1ebf123000 1 Processor -- start 2023-11-06T02:12:46.913 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.903+0000 7f1ebf123000 1 -- start start 2023-11-06T02:12:46.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.903+0000 7f1eb9596640 1 --2- 172.21.15.49:0/2076180871 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1e98065a60 0x7f1e98067f20 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f1eb406c180 tx=0x7f1ea4009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:46.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.903+0000 7f1ebf123000 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e7b4ee5130 0x55e7b4ee5510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:46.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.903+0000 7f1ebf123000 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7b4ee5a50 0x55e7b4e9ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:46.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.903+0000 7f1ebf123000 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e7b4e9b260 0x55e7b4e9b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:46.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.903+0000 7f1ebf123000 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7b4d62c10 con 0x55e7b4ee5a50 2023-11-06T02:12:46.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.903+0000 7f1ebf123000 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7b4cf13e0 con 0x55e7b4ee5130 2023-11-06T02:12:46.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.903+0000 7f1ebf123000 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e7b4cee9b0 con 0x55e7b4e9b260 2023-11-06T02:12:46.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.903+0000 7f1eb9596640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7b4ee5a50 0x55e7b4e9ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:46.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.903+0000 7f1eb9596640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7b4ee5a50 0x55e7b4e9ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58940/0 (socket says 172.21.15.49:58940) 2023-11-06T02:12:46.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.903+0000 7f1eb9596640 1 -- 172.21.15.49:0/1386107535 learned_addr learned my addr 172.21.15.49:0/1386107535 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:12:46.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.903+0000 7f1eba598640 1 --2- 172.21.15.49:0/1386107535 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e7b4e9b260 0x55e7b4e9b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:46.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.903+0000 7f1eb9d97640 1 --2- 172.21.15.49:0/1386107535 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e7b4ee5130 0x55e7b4ee5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:46.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.903+0000 7f1eb9596640 1 -- 172.21.15.49:0/1386107535 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e7b4e9b260 msgr2=0x55e7b4e9b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:46.917 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.903+0000 7f1eb9596640 1 --2- 172.21.15.49:0/1386107535 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e7b4e9b260 0x55e7b4e9b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.917 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.903+0000 7f1eb9596640 1 -- 172.21.15.49:0/1386107535 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e7b4ee5130 msgr2=0x55e7b4ee5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:46.917 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.903+0000 7f1eb9596640 1 --2- 172.21.15.49:0/1386107535 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e7b4ee5130 0x55e7b4ee5510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.918 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.903+0000 7f1eb9596640 1 -- 172.21.15.49:0/1386107535 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e7b49e6e20 con 0x55e7b4ee5a50 2023-11-06T02:12:46.918 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.903+0000 7f1eba598640 1 --2- 172.21.15.49:0/1386107535 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e7b4e9b260 0x55e7b4e9b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:12:46.918 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.903+0000 7f1eb9596640 1 --2- 172.21.15.49:0/1386107535 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7b4ee5a50 0x55e7b4e9ad20 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f1ea4006c10 tx=0x7f1ea4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73c99ae6228c377f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:46.918 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.903+0000 7f1e977fe640 1 -- 172.21.15.49:0/1386107535 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea401c040 con 0x55e7b4ee5a50 2023-11-06T02:12:46.919 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.903+0000 7f1e977fe640 1 -- 172.21.15.49:0/1386107535 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ea4004500 con 0x55e7b4ee5a50 2023-11-06T02:12:46.919 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.903+0000 7f1e977fe640 1 -- 172.21.15.49:0/1386107535 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea4003a30 con 0x55e7b4ee5a50 2023-11-06T02:12:46.919 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.903+0000 7f1ebf123000 1 -- 172.21.15.49:0/1386107535 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7b4ee5a50 msgr2=0x55e7b4e9ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:46.919 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.903+0000 7f1ebf123000 1 --2- 172.21.15.49:0/1386107535 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7b4ee5a50 0x55e7b4e9ad20 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f1ea4006c10 tx=0x7f1ea4014000 comp rx=0 tx=0).stop 2023-11-06T02:12:46.919 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.903+0000 7f1ebf123000 1 -- 172.21.15.49:0/1386107535 shutdown_connections 2023-11-06T02:12:46.919 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.903+0000 7f1ebf123000 1 --2- 172.21.15.49:0/1386107535 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7b4ee5a50 0x55e7b4e9ad20 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.903+0000 7f1ebf123000 1 --2- 172.21.15.49:0/1386107535 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e7b4e9b260 0x55e7b4e9b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.903+0000 7f1ebf123000 1 --2- 172.21.15.49:0/1386107535 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e7b4ee5130 0x55e7b4ee5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.903+0000 7f1ebf123000 1 -- 172.21.15.49:0/1386107535 >> 172.21.15.49:0/1386107535 conn(0x55e7b4ea19f0 msgr2=0x55e7b4e99600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:12:46.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.903+0000 7f1ebf123000 1 -- 172.21.15.49:0/1386107535 shutdown_connections 2023-11-06T02:12:46.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.903+0000 7f1ebf123000 1 -- 172.21.15.49:0/1386107535 wait complete. 2023-11-06T02:12:46.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.903+0000 7f1ebf123000 1 -- start start 2023-11-06T02:12:46.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.903+0000 7f1ebf123000 1 Processor -- start 2023-11-06T02:12:46.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.907+0000 7f1ebf123000 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e7b4e9b260 0x55e7b4ee4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:46.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.907+0000 7f1ebf123000 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7b4ee5130 0x55e7b4ee5e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:46.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.907+0000 7f1ebf123000 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e7b4ee5a50 0x55e7b4ee6370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:46.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.907+0000 7f1ebf123000 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7b4cea2a0 con 0x55e7b4ee5130 2023-11-06T02:12:46.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.907+0000 7f1ebf123000 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7b4c89470 con 0x55e7b4ee5a50 2023-11-06T02:12:46.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.907+0000 7f1ebf123000 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e7b4ca7660 con 0x55e7b4e9b260 2023-11-06T02:12:46.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.907+0000 7f1ebf123000 10 client.0 _ll_register_callbacks cb 0x55e7b4e86c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T02:12:46.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.907+0000 7f1eb9596640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7b4ee5130 0x55e7b4ee5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:46.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.907+0000 7f1eb9596640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7b4ee5130 0x55e7b4ee5e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58954/0 (socket says 172.21.15.49:58954) 2023-11-06T02:12:46.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.907+0000 7f1eb9596640 1 -- 172.21.15.49:0/1771744833 learned_addr learned my addr 172.21.15.49:0/1771744833 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:12:46.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.907+0000 7f1eb9d97640 1 --2- 172.21.15.49:0/1771744833 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e7b4e9b260 0x55e7b4ee4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:46.924 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.907+0000 7f1eba598640 1 --2- 172.21.15.49:0/1771744833 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e7b4ee5a50 0x55e7b4ee6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:46.924 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.907+0000 7f1eb9596640 1 -- 172.21.15.49:0/1771744833 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e7b4e9b260 msgr2=0x55e7b4ee4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:46.924 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.907+0000 7f1eb9596640 1 --2- 172.21.15.49:0/1771744833 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e7b4e9b260 0x55e7b4ee4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.924 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.907+0000 7f1eb9596640 1 -- 172.21.15.49:0/1771744833 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e7b4ee5a50 msgr2=0x55e7b4ee6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:46.925 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.907+0000 7f1eb9596640 1 --2- 172.21.15.49:0/1771744833 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e7b4ee5a50 0x55e7b4ee6370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.925 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.907+0000 7f1eb9596640 1 -- 172.21.15.49:0/1771744833 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ea4004120 con 0x55e7b4ee5130 2023-11-06T02:12:46.925 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.907+0000 7f1eb9d97640 1 --2- 172.21.15.49:0/1771744833 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e7b4e9b260 0x55e7b4ee4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:12:46.925 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.907+0000 7f1eb9596640 1 --2- 172.21.15.49:0/1771744833 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7b4ee5130 0x55e7b4ee5e30 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f1ea4024040 tx=0x7f1ea4004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c384fdfd9280b88 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:46.926 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.907+0000 7f1e94ff9640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-11-06T02:12:46.926 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.907+0000 7f1e94ff9640 1 -- 172.21.15.49:0/1771744833 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea4022670 con 0x55e7b4ee5130 2023-11-06T02:12:46.926 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.907+0000 7f1e94ff9640 1 -- 172.21.15.49:0/1771744833 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(68..68 src has 1..68) v4 ==== 6264+0+0 (secure 0 0 0) 0x7f1ea4037d10 con 0x55e7b4ee5130 2023-11-06T02:12:46.926 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.907+0000 7f1ebf123000 1 -- 172.21.15.49:0/1771744833 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e7b4ca70a0 con 0x55e7b4ee5130 2023-11-06T02:12:46.926 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.907+0000 7f1e94ff9640 1 -- 172.21.15.49:0/1771744833 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ea40227d0 con 0x55e7b4ee5130 2023-11-06T02:12:46.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.907+0000 7f1ebf123000 10 client.6924 fetch_fsmap learned FSMap version 36 2023-11-06T02:12:46.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.907+0000 7f1ebf123000 1 -- 172.21.15.49:0/1771744833 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) v3 -- 0x55e7b4caf1d0 con 0x55e7b4ee5130 2023-11-06T02:12:46.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.907+0000 7f1e94ff9640 1 -- 172.21.15.49:0/1771744833 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e7b4ca70a0 con 0x55e7b4ee5130 2023-11-06T02:12:46.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.907+0000 7f1e94ff9640 1 -- 172.21.15.49:0/1771744833 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 36) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f1ea4022930 con 0x55e7b4ee5130 2023-11-06T02:12:46.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.907+0000 7f1ebf123000 10 client.6924 fetch_fsmap finished waiting for FSMap version 36 2023-11-06T02:12:46.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.907+0000 7f1ebf123000 10 client.6924 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2023-11-06T02:12:46.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.907+0000 7f1ebf123000 20 client.6924 populate_metadata read hostname 'smithi049' 2023-11-06T02:12:46.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.907+0000 7f1ebf123000 1 --2- 172.21.15.49:0/1771744833 >> [v2:172.21.15.49:6834/3698847494,v1:172.21.15.49:6835/3698847494] conn(0x55e7b4ee6d80 0x55e7b4ee7160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:46.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.907+0000 7f1ebf123000 4 client.6924 mds_command: new command op to 6807 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T02:12:46.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.907+0000 7f1ebf123000 1 -- 172.21.15.49:0/1771744833 --> [v2:172.21.15.49:6834/3698847494,v1:172.21.15.49:6835/3698847494] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e7b4ce8bf0 con 0x55e7b4ee6d80 2023-11-06T02:12:46.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.907+0000 7f1eb9d97640 1 --2- 172.21.15.49:0/1771744833 >> [v2:172.21.15.49:6834/3698847494,v1:172.21.15.49:6835/3698847494] conn(0x55e7b4ee6d80 0x55e7b4ee7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:46.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.907+0000 7f1eb9d97640 1 --2- 172.21.15.49:0/1771744833 >> [v2:172.21.15.49:6834/3698847494,v1:172.21.15.49:6835/3698847494] conn(0x55e7b4ee6d80 0x55e7b4ee7160 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=847ff963499e1312 server_cookie=422ee2d21fa05130 in_seq=0 out_seq=0 2023-11-06T02:12:46.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.907+0000 7f1e94ff9640 10 client.6924 ms_handle_connect on v2:172.21.15.49:6834/3698847494 2023-11-06T02:12:46.930 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.907+0000 7f1e94ff9640 1 -- 172.21.15.49:0/1771744833 <== mds.0 v2:172.21.15.49:6834/3698847494 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55e7b4ce8bf0 con 0x55e7b4ee6d80 2023-11-06T02:12:46.930 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.907+0000 7f1e94ff9640 10 client.6924 handle_command_reply: tid=0 2023-11-06T02:12:46.930 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.907+0000 7f1ebf123000 1 client.6924 shutdown 2023-11-06T02:12:46.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.907+0000 7f1ebf123000 1 -- 172.21.15.49:0/1771744833 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7b4ee5130 msgr2=0x55e7b4ee5e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:46.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.907+0000 7f1ebf123000 1 --2- 172.21.15.49:0/1771744833 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7b4ee5130 0x55e7b4ee5e30 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f1ea4024040 tx=0x7f1ea4004740 comp rx=0 tx=0).stop 2023-11-06T02:12:46.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.907+0000 7f1ebf123000 1 -- 172.21.15.49:0/1771744833 shutdown_connections 2023-11-06T02:12:46.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.907+0000 7f1ebf123000 1 --2- 172.21.15.49:0/1771744833 >> [v2:172.21.15.49:6834/3698847494,v1:172.21.15.49:6835/3698847494] conn(0x55e7b4ee6d80 0x55e7b4ee7160 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.907+0000 7f1ebf123000 1 --2- 172.21.15.49:0/1771744833 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7b4ee5130 0x55e7b4ee5e30 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.932 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.907+0000 7f1e94ff9640 0 client.6924 ms_handle_reset on v2:172.21.15.49:6834/3698847494 2023-11-06T02:12:46.932 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.907+0000 7f1ebf123000 1 --2- 172.21.15.49:0/1771744833 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e7b4ee5a50 0x55e7b4ee6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.932 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.907+0000 7f1ebf123000 1 --2- 172.21.15.49:0/1771744833 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e7b4e9b260 0x55e7b4ee4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.932 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.907+0000 7f1ebf123000 1 -- 172.21.15.49:0/1771744833 >> 172.21.15.49:0/1771744833 conn(0x55e7b4ea19f0 msgr2=0x55e7b4e99d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:12:46.932 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.907+0000 7f1ebf123000 1 -- 172.21.15.49:0/1771744833 shutdown_connections 2023-11-06T02:12:46.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.907+0000 7f1ebf123000 1 -- 172.21.15.49:0/1771744833 wait complete. 2023-11-06T02:12:46.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.907+0000 7f1ebf123000 20 client.6924 trim_cache size 0 max 16384 2023-11-06T02:12:46.939 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.923+0000 7f1ebf123000 1 Processor -- start 2023-11-06T02:12:46.939 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.923+0000 7f1ebf123000 1 -- start start 2023-11-06T02:12:46.940 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.923+0000 7f1ebf123000 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e7b4e9b260 0x55e7b4f40c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:46.940 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.923+0000 7f1ebf123000 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e7b4ee5130 0x55e7b4f41160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:46.940 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.923+0000 7f1ebf123000 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7b4ee5a50 0x55e7b4f416a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:46.940 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.923+0000 7f1ebf123000 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7b49a7aa0 con 0x55e7b4ee5a50 2023-11-06T02:12:46.940 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.923+0000 7f1ebf123000 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7b4ce7ab0 con 0x55e7b4e9b260 2023-11-06T02:12:46.941 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.923+0000 7f1ebf123000 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e7b4ca6a30 con 0x55e7b4ee5130 2023-11-06T02:12:46.941 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.923+0000 7f1eba598640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7b4ee5a50 0x55e7b4f416a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:46.941 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.923+0000 7f1eba598640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7b4ee5a50 0x55e7b4f416a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58966/0 (socket says 172.21.15.49:58966) 2023-11-06T02:12:46.941 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.923+0000 7f1eba598640 1 -- 172.21.15.49:0/1734900132 learned_addr learned my addr 172.21.15.49:0/1734900132 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:12:46.941 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.923+0000 7f1eb9d97640 1 --2- 172.21.15.49:0/1734900132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e7b4e9b260 0x55e7b4f40c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:46.942 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.923+0000 7f1eba598640 1 -- 172.21.15.49:0/1734900132 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e7b4ee5130 msgr2=0x55e7b4f41160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:46.942 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.923+0000 7f1eb9596640 1 --2- 172.21.15.49:0/1734900132 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e7b4ee5130 0x55e7b4f41160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:46.942 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.923+0000 7f1eba598640 1 --2- 172.21.15.49:0/1734900132 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e7b4ee5130 0x55e7b4f41160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.942 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.923+0000 7f1eba598640 1 -- 172.21.15.49:0/1734900132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e7b4e9b260 msgr2=0x55e7b4f40c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:46.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.923+0000 7f1eba598640 1 --2- 172.21.15.49:0/1734900132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e7b4e9b260 0x55e7b4f40c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.923+0000 7f1eba598640 1 -- 172.21.15.49:0/1734900132 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e7b4c95520 con 0x55e7b4ee5a50 2023-11-06T02:12:46.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.923+0000 7f1eb9d97640 1 --2- 172.21.15.49:0/1734900132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e7b4e9b260 0x55e7b4f40c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:12:46.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.923+0000 7f1eb9596640 1 --2- 172.21.15.49:0/1734900132 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e7b4ee5130 0x55e7b4f41160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:12:46.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.923+0000 7f1eba598640 1 --2- 172.21.15.49:0/1734900132 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7b4ee5a50 0x55e7b4f416a0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f1eac01b750 tx=0x7f1eac004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf102aaa00842450 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:46.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.923+0000 7f1e97fff640 1 -- 172.21.15.49:0/1734900132 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eac0a6b20 con 0x55e7b4ee5a50 2023-11-06T02:12:46.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.923+0000 7f1e97fff640 1 -- 172.21.15.49:0/1734900132 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eac0a6c80 con 0x55e7b4ee5a50 2023-11-06T02:12:46.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.923+0000 7f1e97fff640 1 -- 172.21.15.49:0/1734900132 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eac0c2450 con 0x55e7b4ee5a50 2023-11-06T02:12:46.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.923+0000 7f1ebf123000 1 -- 172.21.15.49:0/1734900132 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7b4ee5a50 msgr2=0x55e7b4f416a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:46.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.923+0000 7f1ebf123000 1 --2- 172.21.15.49:0/1734900132 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7b4ee5a50 0x55e7b4f416a0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f1eac01b750 tx=0x7f1eac004ac0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.923+0000 7f1ebf123000 1 -- 172.21.15.49:0/1734900132 shutdown_connections 2023-11-06T02:12:46.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.923+0000 7f1ebf123000 1 --2- 172.21.15.49:0/1734900132 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7b4ee5a50 0x55e7b4f416a0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.923+0000 7f1ebf123000 1 --2- 172.21.15.49:0/1734900132 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e7b4ee5130 0x55e7b4f41160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.923+0000 7f1ebf123000 1 --2- 172.21.15.49:0/1734900132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e7b4e9b260 0x55e7b4f40c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.923+0000 7f1ebf123000 1 -- 172.21.15.49:0/1734900132 >> 172.21.15.49:0/1734900132 conn(0x55e7b4ea19f0 msgr2=0x55e7b4e9fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:12:46.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.923+0000 7f1ebf123000 1 -- 172.21.15.49:0/1734900132 shutdown_connections 2023-11-06T02:12:46.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.923+0000 7f1ebf123000 1 -- 172.21.15.49:0/1734900132 wait complete. 2023-11-06T02:12:46.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.927+0000 7f1ebf123000 1 -- start start 2023-11-06T02:12:46.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.927+0000 7f1ebf123000 1 Processor -- start 2023-11-06T02:12:46.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.927+0000 7f1ebf123000 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e7b4e9b260 0x55e7b4f2a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:46.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.927+0000 7f1ebf123000 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7b4ee5130 0x55e7b4f2a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:46.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.927+0000 7f1ebf123000 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e7b4ee5a50 0x55e7b4f2ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:46.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.927+0000 7f1ebf123000 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7b4d2ccd0 con 0x55e7b4ee5130 2023-11-06T02:12:46.948 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.927+0000 7f1ebf123000 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7b4c8f7a0 con 0x55e7b4e9b260 2023-11-06T02:12:46.948 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.927+0000 7f1ebf123000 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e7b4d852c0 con 0x55e7b4ee5a50 2023-11-06T02:12:46.948 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.927+0000 7f1ebf123000 10 client.0 _ll_register_callbacks cb 0x55e7b4caecb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T02:12:46.948 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.927+0000 7f1eb9596640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7b4ee5130 0x55e7b4f2a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:46.948 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.927+0000 7f1eb9596640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7b4ee5130 0x55e7b4f2a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58976/0 (socket says 172.21.15.49:58976) 2023-11-06T02:12:46.949 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.927+0000 7f1eb9596640 1 -- 172.21.15.49:0/4185981878 learned_addr learned my addr 172.21.15.49:0/4185981878 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:12:46.949 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.927+0000 7f1eba598640 1 --2- 172.21.15.49:0/4185981878 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e7b4ee5a50 0x55e7b4f2ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:46.949 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.927+0000 7f1eb9d97640 1 --2- 172.21.15.49:0/4185981878 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e7b4e9b260 0x55e7b4f2a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:46.950 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.927+0000 7f1eb9596640 1 -- 172.21.15.49:0/4185981878 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e7b4ee5a50 msgr2=0x55e7b4f2ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:46.950 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.927+0000 7f1eb9596640 1 --2- 172.21.15.49:0/4185981878 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e7b4ee5a50 0x55e7b4f2ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.950 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.927+0000 7f1eb9596640 1 -- 172.21.15.49:0/4185981878 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e7b4e9b260 msgr2=0x55e7b4f2a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:12:46.950 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.927+0000 7f1eb9596640 1 --2- 172.21.15.49:0/4185981878 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e7b4e9b260 0x55e7b4f2a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.950 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.927+0000 7f1eb9596640 1 -- 172.21.15.49:0/4185981878 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1eb4070ae0 con 0x55e7b4ee5130 2023-11-06T02:12:46.951 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.927+0000 7f1eba598640 1 --2- 172.21.15.49:0/4185981878 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e7b4ee5a50 0x55e7b4f2ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:12:46.951 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.927+0000 7f1eb9596640 1 --2- 172.21.15.49:0/4185981878 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7b4ee5130 0x55e7b4f2a910 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f1ea4006c10 tx=0x7f1ea4018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de7a81ae7fc93ca3 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:12:46.951 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.927+0000 7f1e94ff9640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-11-06T02:12:46.951 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.927+0000 7f1e94ff9640 1 -- 172.21.15.49:0/4185981878 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea4022880 con 0x55e7b4ee5130 2023-11-06T02:12:46.951 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.927+0000 7f1e94ff9640 1 -- 172.21.15.49:0/4185981878 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(68..68 src has 1..68) v4 ==== 6264+0+0 (secure 0 0 0) 0x7f1ea4004500 con 0x55e7b4ee5130 2023-11-06T02:12:46.952 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.927+0000 7f1ebf123000 1 -- 172.21.15.49:0/4185981878 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e7b4cb3620 con 0x55e7b4ee5130 2023-11-06T02:12:46.952 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.927+0000 7f1e94ff9640 1 -- 172.21.15.49:0/4185981878 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ea4011da0 con 0x55e7b4ee5130 2023-11-06T02:12:46.952 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.927+0000 7f1ebf123000 10 client.6930 fetch_fsmap learned FSMap version 36 2023-11-06T02:12:46.952 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.927+0000 7f1ebf123000 1 -- 172.21.15.49:0/4185981878 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) v3 -- 0x55e7b4c89090 con 0x55e7b4ee5130 2023-11-06T02:12:46.953 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.927+0000 7f1e94ff9640 1 -- 172.21.15.49:0/4185981878 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e7b4cb3620 con 0x55e7b4ee5130 2023-11-06T02:12:46.953 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.927+0000 7f1e94ff9640 1 -- 172.21.15.49:0/4185981878 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 36) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f1ea404ad20 con 0x55e7b4ee5130 2023-11-06T02:12:46.953 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.927+0000 7f1ebf123000 10 client.6930 fetch_fsmap finished waiting for FSMap version 36 2023-11-06T02:12:46.953 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.927+0000 7f1ebf123000 10 client.6930 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2023-11-06T02:12:46.954 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.927+0000 7f1ebf123000 20 client.6930 populate_metadata read hostname 'smithi049' 2023-11-06T02:12:46.954 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.927+0000 7f1ebf123000 1 --2- 172.21.15.49:0/4185981878 >> [v2:172.21.15.49:6834/3698847494,v1:172.21.15.49:6835/3698847494] conn(0x55e7b4f2b390 0x55e7b4f2b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:12:46.954 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.927+0000 7f1ebf123000 4 client.6930 mds_command: new command op to 6807 tid=0[{"prefix": "scrub status"}] 2023-11-06T02:12:46.954 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.927+0000 7f1ebf123000 1 -- 172.21.15.49:0/4185981878 --> [v2:172.21.15.49:6834/3698847494,v1:172.21.15.49:6835/3698847494] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e7b4ce3320 con 0x55e7b4f2b390 2023-11-06T02:12:46.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.927+0000 7f1eb9d97640 1 --2- 172.21.15.49:0/4185981878 >> [v2:172.21.15.49:6834/3698847494,v1:172.21.15.49:6835/3698847494] conn(0x55e7b4f2b390 0x55e7b4f2b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:12:46.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.927+0000 7f1eb9d97640 1 --2- 172.21.15.49:0/4185981878 >> [v2:172.21.15.49:6834/3698847494,v1:172.21.15.49:6835/3698847494] conn(0x55e7b4f2b390 0x55e7b4f2b770 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e86ee2cbee1fb386 server_cookie=71b2d8376258a36a in_seq=0 out_seq=0 2023-11-06T02:12:46.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.927+0000 7f1e94ff9640 10 client.6930 ms_handle_connect on v2:172.21.15.49:6834/3698847494 2023-11-06T02:12:46.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.927+0000 7f1e94ff9640 1 -- 172.21.15.49:0/4185981878 <== mds.0 v2:172.21.15.49:6834/3698847494 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+260 (crc 0 0 0) 0x55e7b4ce3320 con 0x55e7b4f2b390 2023-11-06T02:12:46.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.927+0000 7f1e94ff9640 10 client.6930 handle_command_reply: tid=0 2023-11-06T02:12:46.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.927+0000 7f1ebf123000 1 client.6930 shutdown 2023-11-06T02:12:46.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.927+0000 7f1ebf123000 1 -- 172.21.15.49:0/4185981878 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7b4ee5130 msgr2=0x55e7b4f2a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:46.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.927+0000 7f1ebf123000 1 --2- 172.21.15.49:0/4185981878 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7b4ee5130 0x55e7b4f2a910 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f1ea4006c10 tx=0x7f1ea4018000 comp rx=0 tx=0).stop 2023-11-06T02:12:46.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.927+0000 7f1ebf123000 1 -- 172.21.15.49:0/4185981878 shutdown_connections 2023-11-06T02:12:46.957 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.927+0000 7f1ebf123000 1 --2- 172.21.15.49:0/4185981878 >> [v2:172.21.15.49:6834/3698847494,v1:172.21.15.49:6835/3698847494] conn(0x55e7b4f2b390 0x55e7b4f2b770 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.957 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.927+0000 7f1ebf123000 1 --2- 172.21.15.49:0/4185981878 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e7b4ee5130 0x55e7b4f2a910 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.957 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.927+0000 7f1ebf123000 1 --2- 172.21.15.49:0/4185981878 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e7b4ee5a50 0x55e7b4f2ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.957 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.927+0000 7f1e94ff9640 0 client.6930 ms_handle_reset on v2:172.21.15.49:6834/3698847494 2023-11-06T02:12:46.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.927+0000 7f1ebf123000 1 --2- 172.21.15.49:0/4185981878 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e7b4e9b260 0x55e7b4f2a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.927+0000 7f1ebf123000 1 -- 172.21.15.49:0/4185981878 >> 172.21.15.49:0/4185981878 conn(0x55e7b4ea19f0 msgr2=0x55e7b4ea03e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:12:46.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.927+0000 7f1ebf123000 1 -- 172.21.15.49:0/4185981878 shutdown_connections 2023-11-06T02:12:46.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.927+0000 7f1ebf123000 1 -- 172.21.15.49:0/4185981878 wait complete. 2023-11-06T02:12:46.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.927+0000 7f1ebf123000 20 client.6930 trim_cache size 0 max 16384 2023-11-06T02:12:46.959 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-11-06T02:12:46.959 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-06T02:12:46.959 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": { 2023-11-06T02:12:46.960 INFO:teuthology.orchestra.run.smithi049.stdout: "8f82abe6-c08d-40a5-a969-67de1c6c4ed5": { 2023-11-06T02:12:46.960 INFO:teuthology.orchestra.run.smithi049.stdout: "path": "/", 2023-11-06T02:12:46.960 INFO:teuthology.orchestra.run.smithi049.stdout: "tag": "8f82abe6-c08d-40a5-a969-67de1c6c4ed5", 2023-11-06T02:12:46.960 INFO:teuthology.orchestra.run.smithi049.stdout: "options": "recursive,repair" 2023-11-06T02:12:46.961 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:12:46.961 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:12:46.961 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-11-06T02:12:46.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.931+0000 7f1ebbffb640 1 -- 172.21.15.49:0/2076180871 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1e98065a60 msgr2=0x7f1e98067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:46.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.931+0000 7f1ebbffb640 1 --2- 172.21.15.49:0/2076180871 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1e98065a60 0x7f1e98067f20 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f1eb406c180 tx=0x7f1ea4009190 comp rx=0 tx=0).stop 2023-11-06T02:12:46.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.931+0000 7f1ebbffb640 1 -- 172.21.15.49:0/2076180871 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1eb406c230 msgr2=0x7f1eb4145240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:12:46.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.931+0000 7f1ebbffb640 1 --2- 172.21.15.49:0/2076180871 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1eb406c230 0x7f1eb4145240 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f1eb0001e50 tx=0x7f1eb0001e80 comp rx=0 tx=0).stop 2023-11-06T02:12:46.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.931+0000 7f1ebbffb640 1 -- 172.21.15.49:0/2076180871 shutdown_connections 2023-11-06T02:12:46.964 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.931+0000 7f1ebbffb640 1 --2- 172.21.15.49:0/2076180871 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1e98065a60 0x7f1e98067f20 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.964 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.931+0000 7f1ebbffb640 1 --2- 172.21.15.49:0/2076180871 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1eb414ffc0 0x7f1eb414a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.964 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.931+0000 7f1ebbffb640 1 --2- 172.21.15.49:0/2076180871 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1eb4070160 0x7f1eb414ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.964 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.931+0000 7f1ebbffb640 1 --2- 172.21.15.49:0/2076180871 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1eb406c230 0x7f1eb4145240 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:12:46.965 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.931+0000 7f1ebbffb640 1 -- 172.21.15.49:0/2076180871 >> 172.21.15.49:0/2076180871 conn(0x7f1eb406a370 msgr2=0x7f1eb40cd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:12:46.965 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.931+0000 7f1ebbffb640 1 -- 172.21.15.49:0/2076180871 shutdown_connections 2023-11-06T02:12:46.965 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:46.931+0000 7f1ebbffb640 1 -- 172.21.15.49:0/2076180871 wait complete. 2023-11-06T02:12:46.966 INFO:tasks.cephfs.filesystem:scrub status for tag:8f82abe6-c08d-40a5-a969-67de1c6c4ed5 - {'path': '/', 'tag': '8f82abe6-c08d-40a5-a969-67de1c6c4ed5', 'options': 'recursive,repair'} 2023-11-06T02:12:47.352 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:47.351+0000 7ff0f3fff640 1 -- 172.21.15.49:0/1316750628 <== mon.0 v2:172.21.15.49:3300/0 7 ==== log(8 entries from seq 1 at 2023-11-06T02:12:46.726318+0000) v1 ==== 1407+0+0 (secure 0 0 0) 0x7ff1000ef6c0 con 0x7ff10c0cdc80 2023-11-06T02:12:47.353 INFO:teuthology.orchestra.run.smithi049.stdout: cluster: 2023-11-06T02:12:47.353 INFO:teuthology.orchestra.run.smithi049.stdout: id: 1a8b035c-6ba4-499a-b876-aea47fd9c0d4 2023-11-06T02:12:47.353 INFO:teuthology.orchestra.run.smithi049.stdout: health: HEALTH_OK 2023-11-06T02:12:47.353 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:12:47.354 INFO:teuthology.orchestra.run.smithi049.stdout: services: 2023-11-06T02:12:47.354 INFO:teuthology.orchestra.run.smithi049.stdout: mon: 3 daemons, quorum a,b,c (age 4m) 2023-11-06T02:12:47.354 INFO:teuthology.orchestra.run.smithi049.stdout: mgr: x(active, since 4m), standbys: y 2023-11-06T02:12:47.354 INFO:teuthology.orchestra.run.smithi049.stdout: mds: 1/1 daemons up, 3 standby 2023-11-06T02:12:47.354 INFO:teuthology.orchestra.run.smithi049.stdout: osd: 8 osds: 8 up (since 4m), 8 in (since 4m) 2023-11-06T02:12:47.355 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:12:47.355 INFO:teuthology.orchestra.run.smithi049.stdout: data: 2023-11-06T02:12:47.355 INFO:teuthology.orchestra.run.smithi049.stdout: volumes: 1/1 healthy 2023-11-06T02:12:47.355 INFO:teuthology.orchestra.run.smithi049.stdout: pools: 4 pools, 105 pgs 2023-11-06T02:12:47.355 INFO:teuthology.orchestra.run.smithi049.stdout: objects: 30 objects, 19 MiB 2023-11-06T02:12:47.355 INFO:teuthology.orchestra.run.smithi049.stdout: usage: 274 MiB used, 720 GiB / 720 GiB avail 2023-11-06T02:12:47.356 INFO:teuthology.orchestra.run.smithi049.stdout: pgs: 105 active+clean 2023-11-06T02:12:47.356 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:12:47.356 INFO:teuthology.orchestra.run.smithi049.stdout: io: 2023-11-06T02:12:47.356 INFO:teuthology.orchestra.run.smithi049.stdout: client: 1.8 KiB/s rd, 102 B/s wr, 1 op/s rd, 0 op/s wr 2023-11-06T02:12:47.356 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:12:47.357 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:12:47.357 INFO:teuthology.orchestra.run.smithi049.stdout:2023-11-06T02:12:46.726318+0000 mds.c [INF] scrub queued for path: / 2023-11-06T02:12:47.357 INFO:teuthology.orchestra.run.smithi049.stdout:2023-11-06T02:12:46.726322+0000 mds.c [INF] scrub summary: idle+waiting paths [/] 2023-11-06T02:12:47.357 INFO:teuthology.orchestra.run.smithi049.stdout:2023-11-06T02:12:46.726329+0000 mds.c [INF] scrub summary: active paths [/] 2023-11-06T02:12:47.357 INFO:teuthology.orchestra.run.smithi049.stdout:2023-11-06T02:12:46.728257+0000 mds.c [ERR] scrub: inode wrongly marked free: 0x10000000001 2023-11-06T02:12:47.358 INFO:teuthology.orchestra.run.smithi049.stdout:2023-11-06T02:12:46.728266+0000 mds.c [ERR] inode table repaired for inode: 0x10000000001 2023-11-06T02:12:47.358 INFO:teuthology.orchestra.run.smithi049.stdout:2023-11-06T02:12:46.728554+0000 mds.c [ERR] scrub: inode wrongly marked free: 0x10000000002 2023-11-06T02:12:47.358 INFO:teuthology.orchestra.run.smithi049.stdout:2023-11-06T02:12:46.728561+0000 mds.c [ERR] inode table repaired for inode: 0x10000000002 2023-11-06T02:12:47.358 INFO:teuthology.orchestra.run.smithi049.stdout:2023-11-06T02:12:46.728610+0000 mds.c [INF] scrub summary: idle+waiting paths [/] 2023-11-06T02:12:48.056 INFO:tasks.ceph.mds.c.smithi049.stderr:2023-11-06T02:12:48.055+0000 7fbec75ea640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-06T02:12:52.011 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T02:12:52.014 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T02:12:52.208 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:12:52.207+0000 7ff0f3fff640 1 -- 172.21.15.49:0/1316750628 <== mon.0 v2:172.21.15.49:3300/0 8 ==== log(1 entries from seq 9 at 2023-11-06T02:12:51.126520+0000) v1 ==== 189+0+0 (secure 0 0 0) 0x7ff1000f2df0 con 0x7ff10c0cdc80 2023-11-06T02:12:52.208 INFO:teuthology.orchestra.run.smithi049.stdout:2023-11-06T02:12:51.126520+0000 mds.c [INF] scrub summary: idle 2023-11-06T02:13:16.967 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 scrub status 2023-11-06T02:13:17.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.106+0000 7fc5fbaa0640 1 Processor -- start 2023-11-06T02:13:17.105 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.106+0000 7fc5fbaa0640 1 -- start start 2023-11-06T02:13:17.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.110+0000 7fc5fbaa0640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc5f415cbb0 0x7fc5f40d1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:17.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.110+0000 7fc5fbaa0640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc5f40d3170 0x7fc5f40b5d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:17.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.110+0000 7fc5fbaa0640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5f415cf90 con 0x7fc5f40d24e0 2023-11-06T02:13:17.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.110+0000 7fc5fbaa0640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5f40d0b80 con 0x7fc5f415cbb0 2023-11-06T02:13:17.107 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.110+0000 7fc5fbaa0640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f40d0e60 con 0x7fc5f40d3170 2023-11-06T02:13:17.107 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.110+0000 7fc5f983c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc5f415cbb0 0x7fc5f40d1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:17.107 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.110+0000 7fc5f983c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc5f415cbb0 0x7fc5f40d1fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52628/0 (socket says 172.21.15.49:52628) 2023-11-06T02:13:17.107 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.110+0000 7fc5f983c640 1 -- 172.21.15.49:0/2511013065 learned_addr learned my addr 172.21.15.49:0/2511013065 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:17.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.110+0000 7fc5fa03d640 1 --2- 172.21.15.49:0/2511013065 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc5f40d3170 0x7fc5f40b5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:17.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.110+0000 7fc5f983c640 1 -- 172.21.15.49:0/2511013065 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc5f40d3170 msgr2=0x7fc5f40b5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:17.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.110+0000 7fc5f983c640 1 --2- 172.21.15.49:0/2511013065 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc5f40d3170 0x7fc5f40b5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:17.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.110+0000 7fc5f983c640 1 -- 172.21.15.49:0/2511013065 >> v1:172.21.15.49:6789/0 conn(0x7fc5f40d24e0 legacy=0x7fc5f40d2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:17.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.110+0000 7fc5f983c640 1 -- 172.21.15.49:0/2511013065 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5f40d1140 con 0x7fc5f415cbb0 2023-11-06T02:13:17.109 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.110+0000 7fc5f983c640 1 --2- 172.21.15.49:0/2511013065 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5f415cbb0 0x7fc5f40d1fa0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec009e50 tx=0x7fc5ec01f1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6628251a1298b88c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:17.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.110+0000 7fc5f883a640 1 -- 172.21.15.49:0/2511013065 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5ec0af400 con 0x7fc5f415cbb0 2023-11-06T02:13:17.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.110+0000 7fc5f883a640 1 -- 172.21.15.49:0/2511013065 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5ec0ab050 con 0x7fc5f415cbb0 2023-11-06T02:13:17.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.110+0000 7fc5f883a640 1 -- 172.21.15.49:0/2511013065 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5ec0af780 con 0x7fc5f415cbb0 2023-11-06T02:13:17.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.114+0000 7fc5fbaa0640 1 -- 172.21.15.49:0/2511013065 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5f415cbb0 msgr2=0x7fc5f40d1fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:17.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.114+0000 7fc5fbaa0640 1 --2- 172.21.15.49:0/2511013065 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5f415cbb0 0x7fc5f40d1fa0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec009e50 tx=0x7fc5ec01f1f0 comp rx=0 tx=0).stop 2023-11-06T02:13:17.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.114+0000 7fc5fbaa0640 1 -- 172.21.15.49:0/2511013065 shutdown_connections 2023-11-06T02:13:17.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.114+0000 7fc5fbaa0640 1 --2- 172.21.15.49:0/2511013065 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc5f40d3170 0x7fc5f40b5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:17.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.114+0000 7fc5fbaa0640 1 --2- 172.21.15.49:0/2511013065 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5f415cbb0 0x7fc5f40d1fa0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:17.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.114+0000 7fc5fbaa0640 1 -- 172.21.15.49:0/2511013065 >> 172.21.15.49:0/2511013065 conn(0x7fc5f40718a0 msgr2=0x7fc5f4071cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:17.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.114+0000 7fc5fbaa0640 1 -- 172.21.15.49:0/2511013065 shutdown_connections 2023-11-06T02:13:17.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.114+0000 7fc5fbaa0640 1 -- 172.21.15.49:0/2511013065 wait complete. 2023-11-06T02:13:17.114 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.114+0000 7fc5fbaa0640 1 Processor -- start 2023-11-06T02:13:17.114 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.114+0000 7fc5fbaa0640 1 -- start start 2023-11-06T02:13:17.114 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.114+0000 7fc5fbaa0640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc5f40d24e0 0x7fc5f41ef9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:17.114 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.114+0000 7fc5fbaa0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5f40d3170 0x7fc5f41f5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:17.115 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.114+0000 7fc5fbaa0640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc5f415cbb0 0x7fc5f41f6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:17.115 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.114+0000 7fc5fbaa0640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f41eff20 con 0x7fc5f40d3170 2023-11-06T02:13:17.115 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.114+0000 7fc5fbaa0640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f40aaca0 con 0x7fc5f415cbb0 2023-11-06T02:13:17.115 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.114+0000 7fc5fbaa0640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f40aaf80 con 0x7fc5f40d24e0 2023-11-06T02:13:17.115 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.114+0000 7fc5f903b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5f40d3170 0x7fc5f41f5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:17.116 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.114+0000 7fc5f903b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5f40d3170 0x7fc5f41f5b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52636/0 (socket says 172.21.15.49:52636) 2023-11-06T02:13:17.116 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.114+0000 7fc5f903b640 1 -- 172.21.15.49:0/3658462536 learned_addr learned my addr 172.21.15.49:0/3658462536 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:17.116 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.114+0000 7fc5fa03d640 1 --2- 172.21.15.49:0/3658462536 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc5f415cbb0 0x7fc5f41f6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:17.116 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.114+0000 7fc5f983c640 1 --2- 172.21.15.49:0/3658462536 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc5f40d24e0 0x7fc5f41ef9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:17.116 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.114+0000 7fc5f903b640 1 -- 172.21.15.49:0/3658462536 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc5f40d24e0 msgr2=0x7fc5f41ef9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:17.116 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.114+0000 7fc5f903b640 1 --2- 172.21.15.49:0/3658462536 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc5f40d24e0 0x7fc5f41ef9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:17.117 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.114+0000 7fc5f903b640 1 -- 172.21.15.49:0/3658462536 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc5f415cbb0 msgr2=0x7fc5f41f6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:17.117 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.114+0000 7fc5f903b640 1 --2- 172.21.15.49:0/3658462536 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc5f415cbb0 0x7fc5f41f6050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:17.117 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.114+0000 7fc5f903b640 1 -- 172.21.15.49:0/3658462536 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5ec009b30 con 0x7fc5f40d3170 2023-11-06T02:13:17.117 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.114+0000 7fc5f983c640 1 --2- 172.21.15.49:0/3658462536 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc5f40d24e0 0x7fc5f41ef9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:13:17.117 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.114+0000 7fc5fa03d640 1 --2- 172.21.15.49:0/3658462536 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc5f415cbb0 0x7fc5f41f6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:13:17.118 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.114+0000 7fc5f903b640 1 --2- 172.21.15.49:0/3658462536 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5f40d3170 0x7fc5f41f5b10 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fc5e400c4d0 tx=0x7fc5e400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:17.118 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.118+0000 7fc5ea7fc640 1 -- 172.21.15.49:0/3658462536 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e40052d0 con 0x7fc5f40d3170 2023-11-06T02:13:17.118 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.118+0000 7fc5ea7fc640 1 -- 172.21.15.49:0/3658462536 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5e4005430 con 0x7fc5f40d3170 2023-11-06T02:13:17.118 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.118+0000 7fc5fbaa0640 1 -- 172.21.15.49:0/3658462536 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5f41f3780 con 0x7fc5f40d3170 2023-11-06T02:13:17.118 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.118+0000 7fc5ea7fc640 1 -- 172.21.15.49:0/3658462536 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e4012830 con 0x7fc5f40d3170 2023-11-06T02:13:17.119 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.118+0000 7fc5fbaa0640 1 -- 172.21.15.49:0/3658462536 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5f40ab260 con 0x7fc5f40d3170 2023-11-06T02:13:17.119 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.118+0000 7fc5ea7fc640 1 -- 172.21.15.49:0/3658462536 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fc5e4012990 con 0x7fc5f40d3170 2023-11-06T02:13:17.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.118+0000 7fc5ea7fc640 1 --2- 172.21.15.49:0/3658462536 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc5d8065a60 0x7fc5d8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:17.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.118+0000 7fc5ea7fc640 1 -- 172.21.15.49:0/3658462536 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6264+0+0 (secure 0 0 0) 0x7fc5e4082e40 con 0x7fc5f40d3170 2023-11-06T02:13:17.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.118+0000 7fc5f983c640 1 --2- 172.21.15.49:0/3658462536 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc5d8065a60 0x7fc5d8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:17.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.118+0000 7fc5febc8000 1 Processor -- start 2023-11-06T02:13:17.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.118+0000 7fc5febc8000 1 -- start start 2023-11-06T02:13:17.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.118+0000 7fc5f983c640 1 --2- 172.21.15.49:0/3658462536 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc5d8065a60 0x7fc5d8067f20 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec0b1000 tx=0x7fc5ec0aa000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:17.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.118+0000 7fc5febc8000 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1571ad130 0x55b1571ad510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:17.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.118+0000 7fc5febc8000 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b1571ada50 0x55b157162d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:17.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.118+0000 7fc5febc8000 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b157163260 0x55b157163640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:17.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.118+0000 7fc5febc8000 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b15702ac10 con 0x55b1571ad130 2023-11-06T02:13:17.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.118+0000 7fc5febc8000 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b156fb93e0 con 0x55b1571ada50 2023-11-06T02:13:17.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.118+0000 7fc5febc8000 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b156fb69b0 con 0x55b157163260 2023-11-06T02:13:17.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.122+0000 7fc5f983c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1571ad130 0x55b1571ad510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:17.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.122+0000 7fc5f983c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1571ad130 0x55b1571ad510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52638/0 (socket says 172.21.15.49:52638) 2023-11-06T02:13:17.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.122+0000 7fc5f983c640 1 -- 172.21.15.49:0/3985782157 learned_addr learned my addr 172.21.15.49:0/3985782157 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:17.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.122+0000 7fc5f903b640 1 --2- 172.21.15.49:0/3985782157 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b1571ada50 0x55b157162d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:17.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.122+0000 7fc5fa03d640 1 --2- 172.21.15.49:0/3985782157 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b157163260 0x55b157163640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:17.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.122+0000 7fc5f983c640 1 -- 172.21.15.49:0/3985782157 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b157163260 msgr2=0x55b157163640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:17.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.122+0000 7fc5f983c640 1 --2- 172.21.15.49:0/3985782157 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b157163260 0x55b157163640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:17.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.122+0000 7fc5f983c640 1 -- 172.21.15.49:0/3985782157 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b1571ada50 msgr2=0x55b157162d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:17.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.122+0000 7fc5f983c640 1 --2- 172.21.15.49:0/3985782157 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b1571ada50 0x55b157162d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:17.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.122+0000 7fc5f983c640 1 -- 172.21.15.49:0/3985782157 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b156caee20 con 0x55b1571ad130 2023-11-06T02:13:17.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.122+0000 7fc5f903b640 1 --2- 172.21.15.49:0/3985782157 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b1571ada50 0x55b157162d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:13:17.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.122+0000 7fc5f983c640 1 --2- 172.21.15.49:0/3985782157 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1571ad130 0x55b1571ad510 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec09bb30 tx=0x7fc5ec004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ffb177e1b9ca3a7 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:17.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.122+0000 7fc5d77fe640 1 -- 172.21.15.49:0/3985782157 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5ec0bb390 con 0x55b1571ad130 2023-11-06T02:13:17.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.122+0000 7fc5d77fe640 1 -- 172.21.15.49:0/3985782157 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5ec0c0040 con 0x55b1571ad130 2023-11-06T02:13:17.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.122+0000 7fc5d77fe640 1 -- 172.21.15.49:0/3985782157 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5ec0b9750 con 0x55b1571ad130 2023-11-06T02:13:17.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.122+0000 7fc5febc8000 1 -- 172.21.15.49:0/3985782157 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1571ad130 msgr2=0x55b1571ad510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:17.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.122+0000 7fc5febc8000 1 --2- 172.21.15.49:0/3985782157 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1571ad130 0x55b1571ad510 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec09bb30 tx=0x7fc5ec004ac0 comp rx=0 tx=0).stop 2023-11-06T02:13:17.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.122+0000 7fc5febc8000 1 -- 172.21.15.49:0/3985782157 shutdown_connections 2023-11-06T02:13:17.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.122+0000 7fc5febc8000 1 --2- 172.21.15.49:0/3985782157 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b157163260 0x55b157163640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:17.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.122+0000 7fc5febc8000 1 --2- 172.21.15.49:0/3985782157 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b1571ada50 0x55b157162d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:17.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.122+0000 7fc5febc8000 1 --2- 172.21.15.49:0/3985782157 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1571ad130 0x55b1571ad510 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:17.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.122+0000 7fc5febc8000 1 -- 172.21.15.49:0/3985782157 >> 172.21.15.49:0/3985782157 conn(0x55b1571699f0 msgr2=0x55b157161600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:17.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.122+0000 7fc5febc8000 1 -- 172.21.15.49:0/3985782157 shutdown_connections 2023-11-06T02:13:17.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.122+0000 7fc5febc8000 1 -- 172.21.15.49:0/3985782157 wait complete. 2023-11-06T02:13:17.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.122+0000 7fc5febc8000 1 -- start start 2023-11-06T02:13:17.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.122+0000 7fc5febc8000 1 Processor -- start 2023-11-06T02:13:17.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.122+0000 7fc5febc8000 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b157163260 0x55b1571acaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:17.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.122+0000 7fc5febc8000 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b1571ad130 0x55b1571ae040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:17.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.122+0000 7fc5febc8000 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1571ada50 0x55b1571ae580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:17.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.122+0000 7fc5febc8000 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b156fb22a0 con 0x55b1571ada50 2023-11-06T02:13:17.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.122+0000 7fc5febc8000 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b156f51470 con 0x55b1571ad130 2023-11-06T02:13:17.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.122+0000 7fc5febc8000 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b156f6f660 con 0x55b157163260 2023-11-06T02:13:17.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.122+0000 7fc5febc8000 10 client.0 _ll_register_callbacks cb 0x55b15714ec20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T02:13:17.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.122+0000 7fc5fa03d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1571ada50 0x55b1571ae580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:17.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.122+0000 7fc5fa03d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1571ada50 0x55b1571ae580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52642/0 (socket says 172.21.15.49:52642) 2023-11-06T02:13:17.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.122+0000 7fc5fa03d640 1 -- 172.21.15.49:0/2109336339 learned_addr learned my addr 172.21.15.49:0/2109336339 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:17.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.122+0000 7fc5f903b640 1 --2- 172.21.15.49:0/2109336339 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b1571ad130 0x55b1571ae040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:17.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.122+0000 7fc5f983c640 1 --2- 172.21.15.49:0/2109336339 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b157163260 0x55b1571acaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:17.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.122+0000 7fc5fa03d640 1 -- 172.21.15.49:0/2109336339 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b157163260 msgr2=0x55b1571acaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:17.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.122+0000 7fc5fa03d640 1 --2- 172.21.15.49:0/2109336339 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b157163260 0x55b1571acaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:17.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.122+0000 7fc5fa03d640 1 -- 172.21.15.49:0/2109336339 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b1571ad130 msgr2=0x55b1571ae040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:17.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.122+0000 7fc5fa03d640 1 --2- 172.21.15.49:0/2109336339 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b1571ad130 0x55b1571ae040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:17.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.122+0000 7fc5fa03d640 1 -- 172.21.15.49:0/2109336339 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5f40d1140 con 0x55b1571ada50 2023-11-06T02:13:17.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.122+0000 7fc5fa03d640 1 --2- 172.21.15.49:0/2109336339 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1571ada50 0x55b1571ae580 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fc5f0011290 tx=0x7fc5f0011750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3ab59f7be95013d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:17.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.122+0000 7fc5d4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-11-06T02:13:17.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.126+0000 7fc5d4ff9640 1 -- 172.21.15.49:0/2109336339 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f0015650 con 0x55b1571ada50 2023-11-06T02:13:17.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.126+0000 7fc5d4ff9640 1 -- 172.21.15.49:0/2109336339 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(68..68 src has 1..68) v4 ==== 6264+0+0 (secure 0 0 0) 0x7fc5f00157b0 con 0x55b1571ada50 2023-11-06T02:13:17.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.126+0000 7fc5febc8000 1 -- 172.21.15.49:0/2109336339 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b156f6f0a0 con 0x55b1571ada50 2023-11-06T02:13:17.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.126+0000 7fc5d4ff9640 1 -- 172.21.15.49:0/2109336339 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc5f0015970 con 0x55b1571ada50 2023-11-06T02:13:17.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.126+0000 7fc5febc8000 10 client.6942 fetch_fsmap learned FSMap version 36 2023-11-06T02:13:17.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.126+0000 7fc5febc8000 1 -- 172.21.15.49:0/2109336339 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) v3 -- 0x55b156f771d0 con 0x55b1571ada50 2023-11-06T02:13:17.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.126+0000 7fc5d4ff9640 1 -- 172.21.15.49:0/2109336339 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b156f6f0a0 con 0x55b1571ada50 2023-11-06T02:13:17.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.126+0000 7fc5d4ff9640 1 -- 172.21.15.49:0/2109336339 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 36) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fc5f001fd20 con 0x55b1571ada50 2023-11-06T02:13:17.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.126+0000 7fc5febc8000 10 client.6942 fetch_fsmap finished waiting for FSMap version 36 2023-11-06T02:13:17.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.126+0000 7fc5febc8000 10 client.6942 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2023-11-06T02:13:17.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.126+0000 7fc5febc8000 20 client.6942 populate_metadata read hostname 'smithi049' 2023-11-06T02:13:17.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.126+0000 7fc5febc8000 1 --2- 172.21.15.49:0/2109336339 >> [v2:172.21.15.49:6834/3698847494,v1:172.21.15.49:6835/3698847494] conn(0x55b1571af320 0x55b1571bf560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:17.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.126+0000 7fc5febc8000 4 client.6942 mds_command: new command op to 6807 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T02:13:17.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.126+0000 7fc5febc8000 1 -- 172.21.15.49:0/2109336339 --> [v2:172.21.15.49:6834/3698847494,v1:172.21.15.49:6835/3698847494] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b156fb0bf0 con 0x55b1571af320 2023-11-06T02:13:17.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.126+0000 7fc5f983c640 1 --2- 172.21.15.49:0/2109336339 >> [v2:172.21.15.49:6834/3698847494,v1:172.21.15.49:6835/3698847494] conn(0x55b1571af320 0x55b1571bf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:17.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.126+0000 7fc5f983c640 1 --2- 172.21.15.49:0/2109336339 >> [v2:172.21.15.49:6834/3698847494,v1:172.21.15.49:6835/3698847494] conn(0x55b1571af320 0x55b1571bf560 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=188ab63d935a811a server_cookie=1d106fca4ff54f80 in_seq=0 out_seq=0 2023-11-06T02:13:17.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.126+0000 7fc5d4ff9640 10 client.6942 ms_handle_connect on v2:172.21.15.49:6834/3698847494 2023-11-06T02:13:17.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.126+0000 7fc5d4ff9640 1 -- 172.21.15.49:0/2109336339 <== mds.0 v2:172.21.15.49:6834/3698847494 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55b156fb0bf0 con 0x55b1571af320 2023-11-06T02:13:17.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.126+0000 7fc5d4ff9640 10 client.6942 handle_command_reply: tid=0 2023-11-06T02:13:17.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.126+0000 7fc5febc8000 1 client.6942 shutdown 2023-11-06T02:13:17.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.126+0000 7fc5febc8000 1 -- 172.21.15.49:0/2109336339 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1571ada50 msgr2=0x55b1571ae580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:17.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.126+0000 7fc5febc8000 1 --2- 172.21.15.49:0/2109336339 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1571ada50 0x55b1571ae580 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fc5f0011290 tx=0x7fc5f0011750 comp rx=0 tx=0).stop 2023-11-06T02:13:17.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.126+0000 7fc5febc8000 1 -- 172.21.15.49:0/2109336339 shutdown_connections 2023-11-06T02:13:17.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.126+0000 7fc5febc8000 1 --2- 172.21.15.49:0/2109336339 >> [v2:172.21.15.49:6834/3698847494,v1:172.21.15.49:6835/3698847494] conn(0x55b1571af320 0x55b1571bf560 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:17.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.126+0000 7fc5febc8000 1 --2- 172.21.15.49:0/2109336339 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1571ada50 0x55b1571ae580 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:17.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.126+0000 7fc5febc8000 1 --2- 172.21.15.49:0/2109336339 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b1571ad130 0x55b1571ae040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:17.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.126+0000 7fc5febc8000 1 --2- 172.21.15.49:0/2109336339 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b157163260 0x55b1571acaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:17.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.126+0000 7fc5d4ff9640 0 client.6942 ms_handle_reset on v2:172.21.15.49:6834/3698847494 2023-11-06T02:13:17.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.126+0000 7fc5febc8000 1 -- 172.21.15.49:0/2109336339 >> 172.21.15.49:0/2109336339 conn(0x55b1571699f0 msgr2=0x55b157161d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:17.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.126+0000 7fc5febc8000 1 -- 172.21.15.49:0/2109336339 shutdown_connections 2023-11-06T02:13:17.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.126+0000 7fc5febc8000 1 -- 172.21.15.49:0/2109336339 wait complete. 2023-11-06T02:13:17.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.126+0000 7fc5febc8000 20 client.6942 trim_cache size 0 max 16384 2023-11-06T02:13:17.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.142+0000 7fc5febc8000 1 Processor -- start 2023-11-06T02:13:17.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.142+0000 7fc5febc8000 1 -- start start 2023-11-06T02:13:17.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.142+0000 7fc5febc8000 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b157163260 0x55b157168950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:17.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.142+0000 7fc5febc8000 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b1571ad130 0x55b157168e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:17.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.142+0000 7fc5febc8000 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1571ada50 0x55b1571bf560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:17.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.142+0000 7fc5febc8000 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b156c6faa0 con 0x55b1571ada50 2023-11-06T02:13:17.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.142+0000 7fc5febc8000 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b156fafab0 con 0x55b157163260 2023-11-06T02:13:17.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.142+0000 7fc5febc8000 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b156f577a0 con 0x55b1571ad130 2023-11-06T02:13:17.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.142+0000 7fc5fa03d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1571ada50 0x55b1571bf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:17.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.142+0000 7fc5fa03d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1571ada50 0x55b1571bf560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52658/0 (socket says 172.21.15.49:52658) 2023-11-06T02:13:17.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.142+0000 7fc5fa03d640 1 -- 172.21.15.49:0/2947975791 learned_addr learned my addr 172.21.15.49:0/2947975791 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:17.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.142+0000 7fc5fa03d640 1 -- 172.21.15.49:0/2947975791 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b1571ad130 msgr2=0x55b157168e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:17.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.142+0000 7fc5f903b640 1 --2- 172.21.15.49:0/2947975791 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b1571ad130 0x55b157168e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:17.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.142+0000 7fc5f983c640 1 --2- 172.21.15.49:0/2947975791 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b157163260 0x55b157168950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:17.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.142+0000 7fc5fa03d640 1 --2- 172.21.15.49:0/2947975791 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b1571ad130 0x55b157168e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:17.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.142+0000 7fc5fa03d640 1 -- 172.21.15.49:0/2947975791 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b157163260 msgr2=0x55b157168950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:17.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.142+0000 7fc5fa03d640 1 --2- 172.21.15.49:0/2947975791 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b157163260 0x55b157168950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:17.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.142+0000 7fc5fa03d640 1 -- 172.21.15.49:0/2947975791 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b156f5d520 con 0x55b1571ada50 2023-11-06T02:13:17.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.142+0000 7fc5fa03d640 1 --2- 172.21.15.49:0/2947975791 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1571ada50 0x55b1571bf560 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fc5f000bb00 tx=0x7fc5f0009000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d42486a0038e4f5 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:17.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.142+0000 7fc5d7fff640 1 -- 172.21.15.49:0/2947975791 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f0015e50 con 0x55b1571ada50 2023-11-06T02:13:17.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.142+0000 7fc5d7fff640 1 -- 172.21.15.49:0/2947975791 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5f002c070 con 0x55b1571ada50 2023-11-06T02:13:17.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.142+0000 7fc5d7fff640 1 -- 172.21.15.49:0/2947975791 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f0015c10 con 0x55b1571ada50 2023-11-06T02:13:17.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.142+0000 7fc5febc8000 1 -- 172.21.15.49:0/2947975791 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1571ada50 msgr2=0x55b1571bf560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:17.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.142+0000 7fc5febc8000 1 --2- 172.21.15.49:0/2947975791 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1571ada50 0x55b1571bf560 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fc5f000bb00 tx=0x7fc5f0009000 comp rx=0 tx=0).stop 2023-11-06T02:13:17.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.142+0000 7fc5febc8000 1 -- 172.21.15.49:0/2947975791 shutdown_connections 2023-11-06T02:13:17.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.142+0000 7fc5febc8000 1 --2- 172.21.15.49:0/2947975791 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b1571ada50 0x55b1571bf560 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:17.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.142+0000 7fc5febc8000 1 --2- 172.21.15.49:0/2947975791 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b1571ad130 0x55b157168e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:17.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.142+0000 7fc5febc8000 1 --2- 172.21.15.49:0/2947975791 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b157163260 0x55b157168950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:17.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.142+0000 7fc5febc8000 1 -- 172.21.15.49:0/2947975791 >> 172.21.15.49:0/2947975791 conn(0x55b1571699f0 msgr2=0x55b1571ba8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:17.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.142+0000 7fc5febc8000 1 -- 172.21.15.49:0/2947975791 shutdown_connections 2023-11-06T02:13:17.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.142+0000 7fc5febc8000 1 -- 172.21.15.49:0/2947975791 wait complete. 2023-11-06T02:13:17.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.142+0000 7fc5febc8000 1 -- start start 2023-11-06T02:13:17.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.142+0000 7fc5febc8000 1 Processor -- start 2023-11-06T02:13:17.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.146+0000 7fc5febc8000 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b157163260 0x55b1571ed480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:17.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.146+0000 7fc5febc8000 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b1571ad130 0x55b1571edea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:17.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.146+0000 7fc5febc8000 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b1571ada50 0x55b1571ee3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:17.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.146+0000 7fc5febc8000 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b156fac640 con 0x55b157163260 2023-11-06T02:13:17.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.146+0000 7fc5febc8000 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b156fadc10 con 0x55b1571ada50 2023-11-06T02:13:17.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.146+0000 7fc5febc8000 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b15703ac70 con 0x55b1571ad130 2023-11-06T02:13:17.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.146+0000 7fc5febc8000 10 client.0 _ll_register_callbacks cb 0x55b156f76cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T02:13:17.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.146+0000 7fc5f983c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b157163260 0x55b1571ed480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:17.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.146+0000 7fc5f983c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b157163260 0x55b1571ed480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52660/0 (socket says 172.21.15.49:52660) 2023-11-06T02:13:17.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.146+0000 7fc5f983c640 1 -- 172.21.15.49:0/1724941394 learned_addr learned my addr 172.21.15.49:0/1724941394 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:17.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.146+0000 7fc5f903b640 1 --2- 172.21.15.49:0/1724941394 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b1571ad130 0x55b1571edea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:17.156 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.146+0000 7fc5fa03d640 1 --2- 172.21.15.49:0/1724941394 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b1571ada50 0x55b1571ee3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:17.156 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.146+0000 7fc5f983c640 1 -- 172.21.15.49:0/1724941394 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b1571ad130 msgr2=0x55b1571edea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:17.156 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.146+0000 7fc5f983c640 1 --2- 172.21.15.49:0/1724941394 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b1571ad130 0x55b1571edea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:17.156 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.146+0000 7fc5f983c640 1 -- 172.21.15.49:0/1724941394 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b1571ada50 msgr2=0x55b1571ee3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:17.156 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.146+0000 7fc5f983c640 1 --2- 172.21.15.49:0/1724941394 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b1571ada50 0x55b1571ee3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:17.157 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.146+0000 7fc5f983c640 1 -- 172.21.15.49:0/1724941394 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b156f771d0 con 0x55b157163260 2023-11-06T02:13:17.157 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.146+0000 7fc5f903b640 1 --2- 172.21.15.49:0/1724941394 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b1571ad130 0x55b1571edea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:13:17.157 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.146+0000 7fc5f983c640 1 --2- 172.21.15.49:0/1724941394 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b157163260 0x55b1571ed480 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec025f40 tx=0x7fc5ec01da30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e62d4bfdb702052 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:17.157 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.146+0000 7fc5d4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-11-06T02:13:17.157 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.146+0000 7fc5d4ff9640 1 -- 172.21.15.49:0/1724941394 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5ec0b97f0 con 0x55b157163260 2023-11-06T02:13:17.157 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.146+0000 7fc5d4ff9640 1 -- 172.21.15.49:0/1724941394 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(68..68 src has 1..68) v4 ==== 6264+0+0 (secure 0 0 0) 0x7fc5ec0b9950 con 0x55b157163260 2023-11-06T02:13:17.158 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.146+0000 7fc5febc8000 1 -- 172.21.15.49:0/1724941394 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b156f7b620 con 0x55b157163260 2023-11-06T02:13:17.158 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.146+0000 7fc5d4ff9640 1 -- 172.21.15.49:0/1724941394 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc5ec0b9b10 con 0x55b157163260 2023-11-06T02:13:17.158 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.146+0000 7fc5febc8000 10 client.6948 fetch_fsmap learned FSMap version 36 2023-11-06T02:13:17.158 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.146+0000 7fc5febc8000 1 -- 172.21.15.49:0/1724941394 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) v3 -- 0x55b156f51090 con 0x55b157163260 2023-11-06T02:13:17.159 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.146+0000 7fc5d4ff9640 1 -- 172.21.15.49:0/1724941394 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b156f7b620 con 0x55b157163260 2023-11-06T02:13:17.159 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.146+0000 7fc5d4ff9640 1 -- 172.21.15.49:0/1724941394 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 36) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fc5ec0b9c70 con 0x55b157163260 2023-11-06T02:13:17.159 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.146+0000 7fc5febc8000 10 client.6948 fetch_fsmap finished waiting for FSMap version 36 2023-11-06T02:13:17.159 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.146+0000 7fc5febc8000 10 client.6948 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2023-11-06T02:13:17.159 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.146+0000 7fc5febc8000 20 client.6948 populate_metadata read hostname 'smithi049' 2023-11-06T02:13:17.160 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.146+0000 7fc5febc8000 1 --2- 172.21.15.49:0/1724941394 >> [v2:172.21.15.49:6834/3698847494,v1:172.21.15.49:6835/3698847494] conn(0x55b1571ee920 0x55b1571eed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:17.160 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.146+0000 7fc5febc8000 4 client.6948 mds_command: new command op to 6807 tid=0[{"prefix": "scrub status"}] 2023-11-06T02:13:17.160 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.146+0000 7fc5febc8000 1 -- 172.21.15.49:0/1724941394 --> [v2:172.21.15.49:6834/3698847494,v1:172.21.15.49:6835/3698847494] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b156fab320 con 0x55b1571ee920 2023-11-06T02:13:17.160 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.146+0000 7fc5f903b640 1 --2- 172.21.15.49:0/1724941394 >> [v2:172.21.15.49:6834/3698847494,v1:172.21.15.49:6835/3698847494] conn(0x55b1571ee920 0x55b1571eed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:17.160 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.146+0000 7fc5d4ff9640 10 client.6948 ms_handle_connect on v2:172.21.15.49:6834/3698847494 2023-11-06T02:13:17.160 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.146+0000 7fc5f903b640 1 --2- 172.21.15.49:0/1724941394 >> [v2:172.21.15.49:6834/3698847494,v1:172.21.15.49:6835/3698847494] conn(0x55b1571ee920 0x55b1571eed30 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59fcf0eff269d847 server_cookie=a399e76a8beb194a in_seq=0 out_seq=0 2023-11-06T02:13:17.161 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.146+0000 7fc5d4ff9640 1 -- 172.21.15.49:0/1724941394 <== mds.0 v2:172.21.15.49:6834/3698847494 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55b156fab320 con 0x55b1571ee920 2023-11-06T02:13:17.161 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.146+0000 7fc5d4ff9640 10 client.6948 handle_command_reply: tid=0 2023-11-06T02:13:17.161 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.146+0000 7fc5febc8000 1 client.6948 shutdown 2023-11-06T02:13:17.161 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.146+0000 7fc5febc8000 1 -- 172.21.15.49:0/1724941394 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b157163260 msgr2=0x55b1571ed480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:17.161 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.146+0000 7fc5febc8000 1 --2- 172.21.15.49:0/1724941394 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b157163260 0x55b1571ed480 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec025f40 tx=0x7fc5ec01da30 comp rx=0 tx=0).stop 2023-11-06T02:13:17.162 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.146+0000 7fc5febc8000 1 -- 172.21.15.49:0/1724941394 shutdown_connections 2023-11-06T02:13:17.162 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.146+0000 7fc5febc8000 1 --2- 172.21.15.49:0/1724941394 >> [v2:172.21.15.49:6834/3698847494,v1:172.21.15.49:6835/3698847494] conn(0x55b1571ee920 0x55b1571eed30 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:17.162 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.146+0000 7fc5febc8000 1 --2- 172.21.15.49:0/1724941394 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b1571ada50 0x55b1571ee3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:17.162 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.146+0000 7fc5febc8000 1 --2- 172.21.15.49:0/1724941394 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b1571ad130 0x55b1571edea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:17.162 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.146+0000 7fc5febc8000 1 --2- 172.21.15.49:0/1724941394 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b157163260 0x55b1571ed480 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:17.163 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.146+0000 7fc5d4ff9640 0 client.6948 ms_handle_reset on v2:172.21.15.49:6834/3698847494 2023-11-06T02:13:17.163 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.146+0000 7fc5febc8000 1 -- 172.21.15.49:0/1724941394 >> 172.21.15.49:0/1724941394 conn(0x55b1571699f0 msgr2=0x55b1571681d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:17.163 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.146+0000 7fc5febc8000 1 -- 172.21.15.49:0/1724941394 shutdown_connections 2023-11-06T02:13:17.163 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.146+0000 7fc5febc8000 1 -- 172.21.15.49:0/1724941394 wait complete. 2023-11-06T02:13:17.163 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.146+0000 7fc5febc8000 20 client.6948 trim_cache size 0 max 16384 2023-11-06T02:13:17.164 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-11-06T02:13:17.164 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "no active scrubs running", 2023-11-06T02:13:17.164 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": {} 2023-11-06T02:13:17.164 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-11-06T02:13:17.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.146+0000 7fc5fbaa0640 1 -- 172.21.15.49:0/3658462536 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc5d8065a60 msgr2=0x7fc5d8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:17.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.146+0000 7fc5fbaa0640 1 --2- 172.21.15.49:0/3658462536 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc5d8065a60 0x7fc5d8067f20 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec0b1000 tx=0x7fc5ec0aa000 comp rx=0 tx=0).stop 2023-11-06T02:13:17.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.146+0000 7fc5fbaa0640 1 -- 172.21.15.49:0/3658462536 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5f40d3170 msgr2=0x7fc5f41f5b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:17.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.146+0000 7fc5fbaa0640 1 --2- 172.21.15.49:0/3658462536 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5f40d3170 0x7fc5f41f5b10 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fc5e400c4d0 tx=0x7fc5e400c990 comp rx=0 tx=0).stop 2023-11-06T02:13:17.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.146+0000 7fc5fbaa0640 1 -- 172.21.15.49:0/3658462536 shutdown_connections 2023-11-06T02:13:17.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.146+0000 7fc5fbaa0640 1 --2- 172.21.15.49:0/3658462536 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc5d8065a60 0x7fc5d8067f20 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:17.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.146+0000 7fc5fbaa0640 1 --2- 172.21.15.49:0/3658462536 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5f40d3170 0x7fc5f41f5b10 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:17.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.146+0000 7fc5fbaa0640 1 --2- 172.21.15.49:0/3658462536 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc5f415cbb0 0x7fc5f41f6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:17.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.146+0000 7fc5fbaa0640 1 --2- 172.21.15.49:0/3658462536 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc5f40d24e0 0x7fc5f41ef9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:17.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.146+0000 7fc5fbaa0640 1 -- 172.21.15.49:0/3658462536 >> 172.21.15.49:0/3658462536 conn(0x7fc5f40718a0 msgr2=0x7fc5f406b000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:17.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.146+0000 7fc5fbaa0640 1 -- 172.21.15.49:0/3658462536 shutdown_connections 2023-11-06T02:13:17.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.146+0000 7fc5fbaa0640 1 -- 172.21.15.49:0/3658462536 wait complete. 2023-11-06T02:13:17.168 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-06T02:13:17.270 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-06T02:13:17.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.406+0000 7fbdcb4f5640 1 Processor -- start 2023-11-06T02:13:17.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.406+0000 7fbdcb4f5640 1 -- start start 2023-11-06T02:13:17.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.406+0000 7fbdcb4f5640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fbdc40c9d50 0x7fbdc4158b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:17.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.406+0000 7fbdcb4f5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbdc4071520 0x7fbdc415c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:17.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.406+0000 7fbdcb4f5640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdc4071090 con 0x7fbdc406e190 2023-11-06T02:13:17.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.406+0000 7fbdcb4f5640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdc40719d0 con 0x7fbdc40c9d50 2023-11-06T02:13:17.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.406+0000 7fbdcb4f5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc415d480 con 0x7fbdc4071520 2023-11-06T02:13:17.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.406+0000 7fbdc9291640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7fbdc406e190 0x7fbdc40c9640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.49:42352/0 (socket says 172.21.15.49:42352) 2023-11-06T02:13:17.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.406+0000 7fbdc9291640 1 -- 172.21.15.49:0/1586612299 learned_addr learned my addr 172.21.15.49:0/1586612299 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:13:17.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.406+0000 7fbdc9a92640 1 --2- 172.21.15.49:0/1586612299 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbdc4071520 0x7fbdc415c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:17.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.406+0000 7fbdc8a90640 1 --2- 172.21.15.49:0/1586612299 >> v2:172.21.15.150:3300/0 conn(0x7fbdc40c9d50 0x7fbdc4158b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:17.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.406+0000 7fbdbbfff640 1 -- 172.21.15.49:0/1586612299 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2472255220 0 0) 0x7fbdc4071090 con 0x7fbdc406e190 2023-11-06T02:13:17.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.406+0000 7fbdbbfff640 1 -- 172.21.15.49:0/1586612299 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbdb0003530 con 0x7fbdc406e190 2023-11-06T02:13:17.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.406+0000 7fbdc9a92640 1 -- 172.21.15.49:0/1586612299 >> v2:172.21.15.150:3300/0 conn(0x7fbdc40c9d50 msgr2=0x7fbdc4158b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:17.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.406+0000 7fbdc9a92640 1 --2- 172.21.15.49:0/1586612299 >> v2:172.21.15.150:3300/0 conn(0x7fbdc40c9d50 0x7fbdc4158b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:17.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.406+0000 7fbdc9a92640 1 -- 172.21.15.49:0/1586612299 >> v1:172.21.15.150:6789/0 conn(0x7fbdc406e190 legacy=0x7fbdc40c9640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:17.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.406+0000 7fbdc9a92640 1 -- 172.21.15.49:0/1586612299 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdc4070a40 con 0x7fbdc4071520 2023-11-06T02:13:17.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.410+0000 7fbdc9a92640 1 --2- 172.21.15.49:0/1586612299 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbdc4071520 0x7fbdc415c710 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc009d90 tx=0x7fbdbc01f130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b968d09c3b1a350 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:17.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.410+0000 7fbdbbfff640 1 -- 172.21.15.49:0/1586612299 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdbc0af600 con 0x7fbdc4071520 2023-11-06T02:13:17.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.410+0000 7fbdbbfff640 1 -- 172.21.15.49:0/1586612299 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdbc0ab050 con 0x7fbdc4071520 2023-11-06T02:13:17.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.410+0000 7fbdbbfff640 1 -- 172.21.15.49:0/1586612299 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdbc0af960 con 0x7fbdc4071520 2023-11-06T02:13:17.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.410+0000 7fbdcb4f5640 1 -- 172.21.15.49:0/1586612299 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbdc4071520 msgr2=0x7fbdc415c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:17.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.410+0000 7fbdcb4f5640 1 --2- 172.21.15.49:0/1586612299 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbdc4071520 0x7fbdc415c710 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc009d90 tx=0x7fbdbc01f130 comp rx=0 tx=0).stop 2023-11-06T02:13:17.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.410+0000 7fbdcb4f5640 1 -- 172.21.15.49:0/1586612299 shutdown_connections 2023-11-06T02:13:17.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.410+0000 7fbdcb4f5640 1 --2- 172.21.15.49:0/1586612299 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbdc4071520 0x7fbdc415c710 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:17.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.410+0000 7fbdcb4f5640 1 --2- 172.21.15.49:0/1586612299 >> v2:172.21.15.150:3300/0 conn(0x7fbdc40c9d50 0x7fbdc4158b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:17.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.410+0000 7fbdcb4f5640 1 -- 172.21.15.49:0/1586612299 >> 172.21.15.49:0/1586612299 conn(0x7fbdc406c0a0 msgr2=0x7fbdc406c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:17.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.410+0000 7fbdcb4f5640 1 -- 172.21.15.49:0/1586612299 shutdown_connections 2023-11-06T02:13:17.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.410+0000 7fbdcb4f5640 1 -- 172.21.15.49:0/1586612299 wait complete. 2023-11-06T02:13:17.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.414+0000 7fbdcb4f5640 1 Processor -- start 2023-11-06T02:13:17.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.414+0000 7fbdcb4f5640 1 -- start start 2023-11-06T02:13:17.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.414+0000 7fbdcb4f5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbdc406e190 0x7fbdc40afb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:17.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.414+0000 7fbdcb4f5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbdc4071520 0x7fbdc40b0040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:17.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.414+0000 7fbdcb4f5640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbdc40c9d50 0x7fbdc4146130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:17.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.414+0000 7fbdcb4f5640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc415b650 con 0x7fbdc406e190 2023-11-06T02:13:17.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.414+0000 7fbdcb4f5640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc415aa40 con 0x7fbdc40c9d50 2023-11-06T02:13:17.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.414+0000 7fbdcb4f5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc415ba70 con 0x7fbdc4071520 2023-11-06T02:13:17.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.414+0000 7fbdc9291640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbdc406e190 0x7fbdc40afb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:17.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.414+0000 7fbdc9291640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbdc406e190 0x7fbdc40afb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52668/0 (socket says 172.21.15.49:52668) 2023-11-06T02:13:17.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.414+0000 7fbdc9291640 1 -- 172.21.15.49:0/3926553976 learned_addr learned my addr 172.21.15.49:0/3926553976 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:17.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.414+0000 7fbdc8a90640 1 --2- 172.21.15.49:0/3926553976 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbdc4071520 0x7fbdc40b0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:17.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.414+0000 7fbdc9a92640 1 --2- 172.21.15.49:0/3926553976 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbdc40c9d50 0x7fbdc4146130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:17.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.414+0000 7fbdc9291640 1 -- 172.21.15.49:0/3926553976 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbdc4071520 msgr2=0x7fbdc40b0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:17.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.414+0000 7fbdc9291640 1 --2- 172.21.15.49:0/3926553976 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbdc4071520 0x7fbdc40b0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:17.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.414+0000 7fbdc9291640 1 -- 172.21.15.49:0/3926553976 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbdc40c9d50 msgr2=0x7fbdc4146130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:17.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.414+0000 7fbdc9291640 1 --2- 172.21.15.49:0/3926553976 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbdc40c9d50 0x7fbdc4146130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:17.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.414+0000 7fbdc9291640 1 -- 172.21.15.49:0/3926553976 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdbc009a70 con 0x7fbdc406e190 2023-11-06T02:13:17.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.414+0000 7fbdc9291640 1 --2- 172.21.15.49:0/3926553976 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbdc406e190 0x7fbdc40afb00 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fbdc000c430 tx=0x7fbdc000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:17.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.414+0000 7fbdb9ffb640 1 -- 172.21.15.49:0/3926553976 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc0005d40 con 0x7fbdc406e190 2023-11-06T02:13:17.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.414+0000 7fbdb9ffb640 1 -- 172.21.15.49:0/3926553976 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdc00070d0 con 0x7fbdc406e190 2023-11-06T02:13:17.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.414+0000 7fbdcb4f5640 1 -- 172.21.15.49:0/3926553976 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdc415acb0 con 0x7fbdc406e190 2023-11-06T02:13:17.417 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.414+0000 7fbdb9ffb640 1 -- 172.21.15.49:0/3926553976 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc0007480 con 0x7fbdc406e190 2023-11-06T02:13:17.417 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.414+0000 7fbdcb4f5640 1 -- 172.21.15.49:0/3926553976 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdc415b270 con 0x7fbdc406e190 2023-11-06T02:13:17.417 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.414+0000 7fbdcb4f5640 1 -- 172.21.15.49:0/3926553976 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdc415b8c0 con 0x7fbdc406e190 2023-11-06T02:13:17.417 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.414+0000 7fbdb9ffb640 1 -- 172.21.15.49:0/3926553976 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fbdc00075e0 con 0x7fbdc406e190 2023-11-06T02:13:17.417 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.418+0000 7fbdb9ffb640 1 --2- 172.21.15.49:0/3926553976 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fbda8065a60 0x7fbda8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:17.417 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.418+0000 7fbdb9ffb640 1 -- 172.21.15.49:0/3926553976 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6264+0+0 (secure 0 0 0) 0x7fbdc00829c0 con 0x7fbdc406e190 2023-11-06T02:13:17.418 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.418+0000 7fbdc8a90640 1 --2- 172.21.15.49:0/3926553976 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fbda8065a60 0x7fbda8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:17.418 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.418+0000 7fbdc8a90640 1 --2- 172.21.15.49:0/3926553976 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fbda8065a60 0x7fbda8067f20 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fbdc40cc640 tx=0x7fbdb4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:17.419 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.422+0000 7fbdb9ffb640 1 -- 172.21.15.49:0/3926553976 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fbdc0050210 con 0x7fbdc406e190 2023-11-06T02:13:17.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.610+0000 7fbdcb4f5640 1 -- 172.21.15.49:0/3926553976 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fbdc4074040 con 0x7fbdc406e190 2023-11-06T02:13:17.608 INFO:tasks.ceph.mon.a.smithi049.stderr:2023-11-06T02:13:17.614+0000 7f0164e5a640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-06T02:13:17.622 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.626+0000 7fbdb9ffb640 1 -- 172.21.15.49:0/3926553976 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v37) v1 ==== 161+0+0 (secure 0 0 0) 0x7fbdc0017020 con 0x7fbdc406e190 2023-11-06T02:13:17.623 INFO:teuthology.orchestra.run.smithi049.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-06T02:13:17.626 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.630+0000 7fbdcb4f5640 1 -- 172.21.15.49:0/3926553976 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fbda8065a60 msgr2=0x7fbda8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:17.626 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.630+0000 7fbdcb4f5640 1 --2- 172.21.15.49:0/3926553976 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fbda8065a60 0x7fbda8067f20 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fbdc40cc640 tx=0x7fbdb4008040 comp rx=0 tx=0).stop 2023-11-06T02:13:17.626 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.630+0000 7fbdcb4f5640 1 -- 172.21.15.49:0/3926553976 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbdc406e190 msgr2=0x7fbdc40afb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:17.627 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.630+0000 7fbdcb4f5640 1 --2- 172.21.15.49:0/3926553976 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbdc406e190 0x7fbdc40afb00 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fbdc000c430 tx=0x7fbdc000c8f0 comp rx=0 tx=0).stop 2023-11-06T02:13:17.627 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.630+0000 7fbdcb4f5640 1 -- 172.21.15.49:0/3926553976 shutdown_connections 2023-11-06T02:13:17.627 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.630+0000 7fbdcb4f5640 1 --2- 172.21.15.49:0/3926553976 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fbda8065a60 0x7fbda8067f20 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:17.627 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.630+0000 7fbdcb4f5640 1 --2- 172.21.15.49:0/3926553976 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbdc40c9d50 0x7fbdc4146130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:17.627 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.630+0000 7fbdcb4f5640 1 --2- 172.21.15.49:0/3926553976 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbdc4071520 0x7fbdc40b0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:17.628 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.630+0000 7fbdcb4f5640 1 --2- 172.21.15.49:0/3926553976 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbdc406e190 0x7fbdc40afb00 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:17.628 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.630+0000 7fbdcb4f5640 1 -- 172.21.15.49:0/3926553976 >> 172.21.15.49:0/3926553976 conn(0x7fbdc406c0a0 msgr2=0x7fbdc40cc690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:17.628 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.630+0000 7fbdcb4f5640 1 -- 172.21.15.49:0/3926553976 shutdown_connections 2023-11-06T02:13:17.628 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.630+0000 7fbdcb4f5640 1 -- 172.21.15.49:0/3926553976 wait complete. 2023-11-06T02:13:17.629 INFO:tasks.ceph.mds.c.smithi049.stderr: -887> 2023-11-06T02:12:46.723+0000 7fbec3de3640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000001 2023-11-06T02:13:17.629 INFO:tasks.ceph.mds.c.smithi049.stderr: -884> 2023-11-06T02:12:46.723+0000 7fbec3de3640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000001 2023-11-06T02:13:17.629 INFO:tasks.ceph.mds.c.smithi049.stderr: -867> 2023-11-06T02:12:46.723+0000 7fbec3de3640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000002 2023-11-06T02:13:17.629 INFO:tasks.ceph.mds.c.smithi049.stderr: -864> 2023-11-06T02:12:46.723+0000 7fbec3de3640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000002 2023-11-06T02:13:17.630 INFO:tasks.ceph.mds.c.smithi049.stderr: -764> 2023-11-06T02:12:48.055+0000 7fbec75ea640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-06T02:13:17.642 DEBUG:teuthology.orchestra.run.smithi049:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 0 show inode 2023-11-06T02:13:17.659 INFO:tasks.ceph.mds.c.smithi049.stdout:starting mds.c at 2023-11-06T02:13:17.675 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.678+0000 7fd1e4a51280 1 -- start start 2023-11-06T02:13:17.675 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.678+0000 7fd1e4a51280 1 Processor -- start 2023-11-06T02:13:17.677 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.678+0000 7fd1e4a51280 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564f75285800 0x564f75310580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:17.677 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.678+0000 7fd1e4a51280 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564f75285400 0x564f75310b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:17.677 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.678+0000 7fd1e4a51280 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564f75285000 0x564f75311080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:17.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.678+0000 7fd1e4a51280 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f75352a00 con 0x564f75285800 2023-11-06T02:13:17.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.678+0000 7fd1e4a51280 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f75352780 con 0x564f75285000 2023-11-06T02:13:17.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.678+0000 7fd1e4a51280 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x564f75352640 con 0x564f75285400 2023-11-06T02:13:17.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.678+0000 7fd1e3211640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564f75285800 0x564f75310580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:17.679 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.678+0000 7fd1e3211640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564f75285800 0x564f75310580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52702/0 (socket says 172.21.15.49:52702) 2023-11-06T02:13:17.679 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.678+0000 7fd1e3211640 1 -- 172.21.15.49:0/2903621878 learned_addr learned my addr 172.21.15.49:0/2903621878 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:17.679 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.678+0000 7fd1e3211640 1 -- 172.21.15.49:0/2903621878 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564f75285400 msgr2=0x564f75310b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:17.679 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.678+0000 7fd1e3211640 1 --2- 172.21.15.49:0/2903621878 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564f75285400 0x564f75310b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:17.679 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.678+0000 7fd1e3211640 1 -- 172.21.15.49:0/2903621878 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564f75285000 msgr2=0x564f75311080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:17.679 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.678+0000 7fd1e3211640 1 --2- 172.21.15.49:0/2903621878 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564f75285000 0x564f75311080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:17.680 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.678+0000 7fd1e3211640 1 -- 172.21.15.49:0/2903621878 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f752bab60 con 0x564f75285800 2023-11-06T02:13:17.680 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.678+0000 7fd1e3211640 1 --2- 172.21.15.49:0/2903621878 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564f75285800 0x564f75310580 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x564f76180660 tx=0x564f76044000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f51195af56b1e0bf server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:17.680 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.678+0000 7fd1e0a0c640 1 -- 172.21.15.49:0/2903621878 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x564f752e9080 con 0x564f75285800 2023-11-06T02:13:17.680 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.678+0000 7fd1e0a0c640 1 -- 172.21.15.49:0/2903621878 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564f752e0480 con 0x564f75285800 2023-11-06T02:13:17.680 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.678+0000 7fd1e0a0c640 1 -- 172.21.15.49:0/2903621878 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x564f752e9600 con 0x564f75285800 2023-11-06T02:13:17.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.682+0000 7fd1e4a51280 1 -- 172.21.15.49:0/2903621878 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564f752bb1e0 con 0x564f75285800 2023-11-06T02:13:17.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.682+0000 7fd1e4a51280 1 -- 172.21.15.49:0/2903621878 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x564f752bab60 con 0x564f75285800 2023-11-06T02:13:17.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.682+0000 7fd1e4a51280 4 waiting for MDS map... 2023-11-06T02:13:17.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.682+0000 7fd1e0a0c640 1 -- 172.21.15.49:0/2903621878 <== mon.0 v2:172.21.15.49:3300/0 4 ==== osd_map(69..69 src has 1..69) v4 ==== 6350+0+0 (secure 0 0 0) 0x564f752c4c40 con 0x564f75285800 2023-11-06T02:13:17.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.682+0000 7fd1e0a0c640 1 -- 172.21.15.49:0/2903621878 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 37) v1 ==== 2565+0+0 (secure 0 0 0) 0x564f752d3200 con 0x564f75285800 2023-11-06T02:13:17.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.682+0000 7fd1e4a51280 4 Got MDS map 37 2023-11-06T02:13:17.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.682+0000 7fd1e4a51280 10 main: main 2023-11-06T02:13:17.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.682+0000 7fd1e4a51280 4 main: connecting to RADOS... 2023-11-06T02:13:17.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.682+0000 7fd1e4a51280 1 Processor -- start 2023-11-06T02:13:17.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.682+0000 7fd1e4a51280 1 -- start start 2023-11-06T02:13:17.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.682+0000 7fd1e4a51280 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564f76055800 0x564f75311600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:17.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.682+0000 7fd1e4a51280 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564f76055c00 0x564f75311b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:17.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.682+0000 7fd1e4a51280 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564f761fc000 0x564f75312100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:17.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.682+0000 7fd1e4a51280 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f752cbe00 con 0x564f76055c00 2023-11-06T02:13:17.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.682+0000 7fd1e4a51280 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f75352780 con 0x564f76055800 2023-11-06T02:13:17.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.682+0000 7fd1e4a51280 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x564f75352a00 con 0x564f761fc000 2023-11-06T02:13:17.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.682+0000 7fd1e3211640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564f76055c00 0x564f75311b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:17.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.682+0000 7fd1e3211640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564f76055c00 0x564f75311b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52716/0 (socket says 172.21.15.49:52716) 2023-11-06T02:13:17.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.682+0000 7fd1e3211640 1 -- 172.21.15.49:0/2940979313 learned_addr learned my addr 172.21.15.49:0/2940979313 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:17.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.682+0000 7fd1e3211640 1 -- 172.21.15.49:0/2940979313 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564f761fc000 msgr2=0x564f75312100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:17.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.682+0000 7fd1e3211640 1 --2- 172.21.15.49:0/2940979313 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564f761fc000 0x564f75312100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:17.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.682+0000 7fd1e3211640 1 -- 172.21.15.49:0/2940979313 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564f76055800 msgr2=0x564f75311600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:17.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.682+0000 7fd1e3211640 1 --2- 172.21.15.49:0/2940979313 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564f76055800 0x564f75311600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:17.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.682+0000 7fd1e3211640 1 -- 172.21.15.49:0/2940979313 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f752ba340 con 0x564f76055c00 2023-11-06T02:13:17.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.682+0000 7fd1e3211640 1 --2- 172.21.15.49:0/2940979313 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564f76055c00 0x564f75311b80 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x564f76180c60 tx=0x564f760442a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33200fdaaf621c66 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:17.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.682+0000 7fd1dea08640 1 -- 172.21.15.49:0/2940979313 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x564f752e91e0 con 0x564f76055c00 2023-11-06T02:13:17.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.682+0000 7fd1dea08640 1 -- 172.21.15.49:0/2940979313 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564f76052480 con 0x564f76055c00 2023-11-06T02:13:17.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.682+0000 7fd1dea08640 1 -- 172.21.15.49:0/2940979313 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x564f752e8f20 con 0x564f76055c00 2023-11-06T02:13:17.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.682+0000 7fd1e4a51280 1 -- 172.21.15.49:0/2940979313 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564f76055c00 msgr2=0x564f75311b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:17.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.682+0000 7fd1e4a51280 1 --2- 172.21.15.49:0/2940979313 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564f76055c00 0x564f75311b80 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x564f76180c60 tx=0x564f760442a0 comp rx=0 tx=0).stop 2023-11-06T02:13:17.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.682+0000 7fd1e4a51280 1 -- 172.21.15.49:0/2940979313 shutdown_connections 2023-11-06T02:13:17.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.682+0000 7fd1e4a51280 1 --2- 172.21.15.49:0/2940979313 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564f76055c00 0x564f75311b80 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:17.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.682+0000 7fd1e4a51280 1 --2- 172.21.15.49:0/2940979313 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564f761fc000 0x564f75312100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:17.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.682+0000 7fd1e4a51280 1 --2- 172.21.15.49:0/2940979313 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564f76055800 0x564f75311600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:17.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.682+0000 7fd1e4a51280 1 -- 172.21.15.49:0/2940979313 >> 172.21.15.49:0/2940979313 conn(0x564f75284c00 msgr2=0x564f752b3800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:17.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.682+0000 7fd1e4a51280 1 -- 172.21.15.49:0/2940979313 shutdown_connections 2023-11-06T02:13:17.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.682+0000 7fd1e4a51280 1 -- 172.21.15.49:0/2940979313 wait complete. 2023-11-06T02:13:17.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.682+0000 7fd1e4a51280 1 Processor -- start 2023-11-06T02:13:17.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.682+0000 7fd1e4a51280 1 -- start start 2023-11-06T02:13:17.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.682+0000 7fd1e4a51280 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564f76055c00 0x564f75313180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:17.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.682+0000 7fd1e4a51280 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564f76055800 0x564f75312c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:17.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.682+0000 7fd1e4a51280 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564f761fd800 0x564f75312680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:17.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.682+0000 7fd1e4a51280 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f75352500 con 0x564f761fd800 2023-11-06T02:13:17.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.682+0000 7fd1e4a51280 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f753523c0 con 0x564f76055c00 2023-11-06T02:13:17.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.682+0000 7fd1e4a51280 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x564f75352280 con 0x564f76055800 2023-11-06T02:13:17.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.686+0000 7fd1e2a10640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564f761fd800 0x564f75312680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:17.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.686+0000 7fd1e2a10640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564f761fd800 0x564f75312680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52728/0 (socket says 172.21.15.49:52728) 2023-11-06T02:13:17.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.686+0000 7fd1e2a10640 1 -- 172.21.15.49:0/2745788312 learned_addr learned my addr 172.21.15.49:0/2745788312 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:17.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.686+0000 7fd1e3211640 1 --2- 172.21.15.49:0/2745788312 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564f76055800 0x564f75312c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:17.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.686+0000 7fd1e3a12640 1 --2- 172.21.15.49:0/2745788312 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564f76055c00 0x564f75313180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:17.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.686+0000 7fd1e2a10640 1 -- 172.21.15.49:0/2745788312 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564f76055800 msgr2=0x564f75312c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:17.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.686+0000 7fd1e2a10640 1 --2- 172.21.15.49:0/2745788312 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564f76055800 0x564f75312c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:17.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.686+0000 7fd1e2a10640 1 -- 172.21.15.49:0/2745788312 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564f76055c00 msgr2=0x564f75313180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:17.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.686+0000 7fd1e2a10640 1 --2- 172.21.15.49:0/2745788312 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564f76055c00 0x564f75313180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:17.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.686+0000 7fd1e2a10640 1 -- 172.21.15.49:0/2745788312 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f752bb1e0 con 0x564f761fd800 2023-11-06T02:13:17.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.686+0000 7fd1e2a10640 1 --2- 172.21.15.49:0/2745788312 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564f761fd800 0x564f75312680 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x564f76204cc0 tx=0x564f761a0e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:17.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.686+0000 7fd1dd205640 1 -- 172.21.15.49:0/2745788312 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x564f752e91e0 con 0x564f761fd800 2023-11-06T02:13:17.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.686+0000 7fd1dd205640 1 -- 172.21.15.49:0/2745788312 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564f76053800 con 0x564f761fd800 2023-11-06T02:13:17.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.686+0000 7fd1e4a51280 1 -- 172.21.15.49:0/2745788312 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x564f752bab60 con 0x564f761fd800 2023-11-06T02:13:17.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.686+0000 7fd1dd205640 1 -- 172.21.15.49:0/2745788312 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x564f752e8f20 con 0x564f761fd800 2023-11-06T02:13:17.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.686+0000 7fd1e4a51280 1 -- 172.21.15.49:0/2745788312 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564f752ba340 con 0x564f761fd800 2023-11-06T02:13:17.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.686+0000 7fd1e4a51280 4 main: resolving pool 9 2023-11-06T02:13:17.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.686+0000 7fd1dd205640 1 -- 172.21.15.49:0/2745788312 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x564f7534e840 con 0x564f761fd800 2023-11-06T02:13:17.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.686+0000 7fd1dd205640 1 --2- 172.21.15.49:0/2745788312 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x564f7556ac00 0x564f75313700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:17.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.686+0000 7fd1dd205640 1 -- 172.21.15.49:0/2745788312 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6350+0+0 (secure 0 0 0) 0x564f752c4a80 con 0x564f761fd800 2023-11-06T02:13:17.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.686+0000 7fd1e4a51280 4 main: creating IoCtx.. 2023-11-06T02:13:17.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.686+0000 7fd1e4a51280 1 --2- 172.21.15.49:0/2745788312 >> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] conn(0x564f761fd400 0x564f75312100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:17.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.686+0000 7fd1e4a51280 1 -- 172.21.15.49:0/2745788312 --> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] -- osd_op(unknown.0.0:1 9.13 9:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69) v8 -- 0x564f761fd000 con 0x564f761fd400 2023-11-06T02:13:17.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.686+0000 7fd1e3a12640 1 --2- 172.21.15.49:0/2745788312 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x564f7556ac00 0x564f75313700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:17.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.686+0000 7fd1e3211640 1 --2- 172.21.15.49:0/2745788312 >> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] conn(0x564f761fd400 0x564f75312100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:17.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.686+0000 7fd1e3211640 1 --2- 172.21.15.49:0/2745788312 >> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] conn(0x564f761fd400 0x564f75312100 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:17.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.686+0000 7fd1e3a12640 1 --2- 172.21.15.49:0/2745788312 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x564f7556ac00 0x564f75313700 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x564f75ff9c80 tx=0x564f761a12c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:17.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.694+0000 7fd1e3211640 1 -- 172.21.15.49:0/2745788312 <== osd.7 v2:172.21.15.150:6816/38421 1 ==== osd_op_reply(1 mds0_inotable [read 0~34 out=34b] v0'0 uv7 ondisk = 0) v8 ==== 157+0+34 (crc 0 0 0) 0x564f7617cfc0 con 0x564f761fd400 2023-11-06T02:13:17.700 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-11-06T02:13:17.700 INFO:teuthology.orchestra.run.smithi049.stdout: "0": { 2023-11-06T02:13:17.700 INFO:teuthology.orchestra.run.smithi049.stdout: "data": { 2023-11-06T02:13:17.700 INFO:teuthology.orchestra.run.smithi049.stdout: "version": 5, 2023-11-06T02:13:17.700 INFO:teuthology.orchestra.run.smithi049.stdout: "inotable": { 2023-11-06T02:13:17.701 INFO:teuthology.orchestra.run.smithi049.stdout: "projected_free": [ 2023-11-06T02:13:17.701 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:13:17.701 INFO:teuthology.orchestra.run.smithi049.stdout: "start": 1099511627779, 2023-11-06T02:13:17.701 INFO:teuthology.orchestra.run.smithi049.stdout: "len": 1099511627773 2023-11-06T02:13:17.701 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:13:17.702 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-11-06T02:13:17.702 INFO:teuthology.orchestra.run.smithi049.stdout: "free": [ 2023-11-06T02:13:17.702 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:13:17.702 INFO:teuthology.orchestra.run.smithi049.stdout: "start": 1099511627779, 2023-11-06T02:13:17.702 INFO:teuthology.orchestra.run.smithi049.stdout: "len": 1099511627773 2023-11-06T02:13:17.703 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:13:17.703 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2023-11-06T02:13:17.703 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:13:17.703 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:13:17.703 INFO:teuthology.orchestra.run.smithi049.stdout: "result": 0 2023-11-06T02:13:17.704 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:13:17.704 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-11-06T02:13:17.704 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:13:17.706 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.694+0000 7fd1e4a51280 1 -- 172.21.15.49:0/2745788312 >> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] conn(0x564f761fd400 msgr2=0x564f75312100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:17.706 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.694+0000 7fd1e4a51280 1 --2- 172.21.15.49:0/2745788312 >> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] conn(0x564f761fd400 0x564f75312100 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:17.706 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.694+0000 7fd1e4a51280 1 -- 172.21.15.49:0/2745788312 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x564f7556ac00 msgr2=0x564f75313700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:17.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.694+0000 7fd1e4a51280 1 --2- 172.21.15.49:0/2745788312 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x564f7556ac00 0x564f75313700 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x564f75ff9c80 tx=0x564f761a12c0 comp rx=0 tx=0).stop 2023-11-06T02:13:17.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.694+0000 7fd1e4a51280 1 -- 172.21.15.49:0/2745788312 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564f761fd800 msgr2=0x564f75312680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:17.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.694+0000 7fd1e4a51280 1 --2- 172.21.15.49:0/2745788312 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564f761fd800 0x564f75312680 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x564f76204cc0 tx=0x564f761a0e40 comp rx=0 tx=0).stop 2023-11-06T02:13:17.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.694+0000 7fd1e2a10640 1 -- 172.21.15.49:0/2745788312 reap_dead start 2023-11-06T02:13:17.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.694+0000 7fd1e4a51280 1 -- 172.21.15.49:0/2745788312 shutdown_connections 2023-11-06T02:13:17.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.694+0000 7fd1e4a51280 1 -- 172.21.15.49:0/2745788312 >> 172.21.15.49:0/2745788312 conn(0x564f75284c00 msgr2=0x564f752b3800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:17.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.694+0000 7fd1e4a51280 1 -- 172.21.15.49:0/2745788312 shutdown_connections 2023-11-06T02:13:17.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.694+0000 7fd1e4a51280 1 -- 172.21.15.49:0/2745788312 wait complete. 2023-11-06T02:13:17.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.694+0000 7fd1e4a51280 1 -- 172.21.15.49:0/2903621878 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564f75285800 msgr2=0x564f75310580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:17.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.694+0000 7fd1e4a51280 1 --2- 172.21.15.49:0/2903621878 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564f75285800 0x564f75310580 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x564f76180660 tx=0x564f76044000 comp rx=0 tx=0).stop 2023-11-06T02:13:17.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.694+0000 7fd1e4a51280 1 -- 172.21.15.49:0/2903621878 shutdown_connections 2023-11-06T02:13:17.709 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.694+0000 7fd1e4a51280 1 --2- 172.21.15.49:0/2903621878 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564f75285000 0x564f75311080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:17.709 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.694+0000 7fd1e4a51280 1 --2- 172.21.15.49:0/2903621878 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x564f75285400 0x564f75310b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:17.709 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.694+0000 7fd1e4a51280 1 --2- 172.21.15.49:0/2903621878 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564f75285800 0x564f75310580 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:17.709 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.694+0000 7fd1e4a51280 1 -- 172.21.15.49:0/2903621878 >> 172.21.15.49:0/2903621878 conn(0x564f75285c00 msgr2=0x564f752b3000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:17.709 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.694+0000 7fd1e4a51280 1 -- 172.21.15.49:0/2903621878 shutdown_connections 2023-11-06T02:13:17.709 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:17.694+0000 7fd1e4a51280 1 -- 172.21.15.49:0/2903621878 wait complete. 2023-11-06T02:13:17.710 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '0', 'show', 'inode'] in time 0:00:00.068236, result: { "0": { "data": { "version": 5, "inotable": { "projected_free": [ { "start": 1099511627779, "len": 1099511627773 } ], "free": [ { "start": 1099511627779, "len": 1099511627773 } ] } }, "result": 0 } } 2023-11-06T02:13:17.710 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-06T02:13:17.711 DEBUG:teuthology.orchestra.run.smithi049:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T02:13:17.790 DEBUG:teuthology.orchestra.run.smithi150:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T02:13:17.846 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-06T02:13:17.847 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-11-06T02:13:17.847 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T02:13:17.853 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336476k,nr_inodes=4084119,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16105 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-06T02:13:17.853 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi049.front.sepia.ceph.com 2023-11-06T02:13:17.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:13:17.854 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-06T02:13:17.900 INFO:teuthology.orchestra.run.smithi049.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-11-06T02:13:17.901 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T02:13:17.902 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-11-06T02:13:17.903 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T02:13:17.909 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336552k,nr_inodes=4084138,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19862 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274692k,nr_inodes=818673,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-06T02:13:17.909 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi150.front.sepia.ceph.com 2023-11-06T02:13:17.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:13:17.910 DEBUG:teuthology.orchestra.run.smithi150:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-06T02:13:17.957 INFO:teuthology.orchestra.run.smithi150.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-11-06T02:13:17.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T02:13:17.958 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-11-06T02:13:17.959 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T02:13:17.965 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336476k,nr_inodes=4084119,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16105 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-06T02:13:17.965 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi049.front.sepia.ceph.com 2023-11-06T02:13:17.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:13:17.966 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-06T02:13:18.014 INFO:teuthology.orchestra.run.smithi049.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-11-06T02:13:18.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T02:13:18.015 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-11-06T02:13:18.016 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T02:13:18.021 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336552k,nr_inodes=4084138,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19862 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274692k,nr_inodes=818673,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-06T02:13:18.022 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi150.front.sepia.ceph.com 2023-11-06T02:13:18.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:13:18.023 DEBUG:teuthology.orchestra.run.smithi150:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-06T02:13:18.069 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T02:13:18.070 INFO:teuthology.orchestra.run.smithi150.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-11-06T02:13:18.070 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T02:13:18.204 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.206+0000 7f77bcd10640 1 Processor -- start 2023-11-06T02:13:18.205 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.206+0000 7f77bcd10640 1 -- start start 2023-11-06T02:13:18.206 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.210+0000 7f77bcd10640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f77b8071290 0x7f77b80b1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:18.206 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.210+0000 7f77bcd10640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f77b80cb010 0x7f77b80cb480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:18.207 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.210+0000 7f77bcd10640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77b8071890 con 0x7f77b8159980 2023-11-06T02:13:18.207 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.210+0000 7f77bcd10640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77b8072410 con 0x7f77b8071290 2023-11-06T02:13:18.207 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.210+0000 7f77bcd10640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77b80ce440 con 0x7f77b80cb010 2023-11-06T02:13:18.207 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.210+0000 7f77b5d9b640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f77b8071290 0x7f77b80b1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:18.207 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.210+0000 7f77b5d9b640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f77b8071290 0x7f77b80b1450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52744/0 (socket says 172.21.15.49:52744) 2023-11-06T02:13:18.208 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.210+0000 7f77b5d9b640 1 -- 172.21.15.49:0/1365754822 learned_addr learned my addr 172.21.15.49:0/1365754822 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:18.208 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.210+0000 7f77b5d9b640 1 -- 172.21.15.49:0/1365754822 >> v2:172.21.15.150:3300/0 conn(0x7f77b80cb010 msgr2=0x7f77b80cb480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:18.208 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.210+0000 7f77b6d9d640 1 --2- 172.21.15.49:0/1365754822 >> v2:172.21.15.150:3300/0 conn(0x7f77b80cb010 0x7f77b80cb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:18.208 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.210+0000 7f77b5d9b640 1 --2- 172.21.15.49:0/1365754822 >> v2:172.21.15.150:3300/0 conn(0x7f77b80cb010 0x7f77b80cb480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:18.208 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.210+0000 7f77b5d9b640 1 -- 172.21.15.49:0/1365754822 >> v1:172.21.15.150:6789/0 conn(0x7f77b8159980 legacy=0x7f77b80cdd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:18.209 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.210+0000 7f77b5d9b640 1 -- 172.21.15.49:0/1365754822 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77b8071b70 con 0x7f77b8071290 2023-11-06T02:13:18.210 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.210+0000 7f77b5d9b640 1 --2- 172.21.15.49:0/1365754822 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77b8071290 0x7f77b80b1450 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f77b815d2e0 tx=0x7f77ac01eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c6c9968c9e77f59 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:18.210 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.214+0000 7f77b559a640 1 -- 172.21.15.49:0/1365754822 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77ac0a7b30 con 0x7f77b8071290 2023-11-06T02:13:18.210 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.214+0000 7f77b559a640 1 -- 172.21.15.49:0/1365754822 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77ac0a7c90 con 0x7f77b8071290 2023-11-06T02:13:18.211 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.214+0000 7f77b559a640 1 -- 172.21.15.49:0/1365754822 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77ac0af440 con 0x7f77b8071290 2023-11-06T02:13:18.211 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.214+0000 7f77bcd10640 1 -- 172.21.15.49:0/1365754822 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77b8071290 msgr2=0x7f77b80b1450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:18.211 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.214+0000 7f77bcd10640 1 --2- 172.21.15.49:0/1365754822 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77b8071290 0x7f77b80b1450 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f77b815d2e0 tx=0x7f77ac01eb50 comp rx=0 tx=0).stop 2023-11-06T02:13:18.211 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.214+0000 7f77bcd10640 1 -- 172.21.15.49:0/1365754822 shutdown_connections 2023-11-06T02:13:18.211 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.214+0000 7f77bcd10640 1 --2- 172.21.15.49:0/1365754822 >> v2:172.21.15.150:3300/0 conn(0x7f77b80cb010 0x7f77b80cb480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:18.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.214+0000 7f77bcd10640 1 --2- 172.21.15.49:0/1365754822 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77b8071290 0x7f77b80b1450 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:18.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.214+0000 7f77bcd10640 1 -- 172.21.15.49:0/1365754822 >> 172.21.15.49:0/1365754822 conn(0x7f77b806c0a0 msgr2=0x7f77b806c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:18.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.214+0000 7f77bcd10640 1 -- 172.21.15.49:0/1365754822 shutdown_connections 2023-11-06T02:13:18.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.214+0000 7f77bcd10640 1 -- 172.21.15.49:0/1365754822 wait complete. 2023-11-06T02:13:18.215 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.214+0000 7f77bcd10640 1 Processor -- start 2023-11-06T02:13:18.215 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.214+0000 7f77bcd10640 1 -- start start 2023-11-06T02:13:18.215 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.214+0000 7f77bcd10640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77b8071290 0x7f77b8150be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:18.215 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.214+0000 7f77bcd10640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f77b80cb010 0x7f77b8157000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:18.216 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.214+0000 7f77bcd10640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f77b8159980 0x7f77b81493c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:18.216 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.214+0000 7f77bcd10640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77b815af30 con 0x7f77b8071290 2023-11-06T02:13:18.216 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.214+0000 7f77bcd10640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77b8156c60 con 0x7f77b80cb010 2023-11-06T02:13:18.216 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.214+0000 7f77bcd10640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77b8151120 con 0x7f77b8159980 2023-11-06T02:13:18.216 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.214+0000 7f77b659c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77b8071290 0x7f77b8150be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:18.217 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.214+0000 7f77b659c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77b8071290 0x7f77b8150be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52748/0 (socket says 172.21.15.49:52748) 2023-11-06T02:13:18.217 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.214+0000 7f77b659c640 1 -- 172.21.15.49:0/3237933663 learned_addr learned my addr 172.21.15.49:0/3237933663 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:18.217 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.214+0000 7f77b6d9d640 1 --2- 172.21.15.49:0/3237933663 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f77b8159980 0x7f77b81493c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:18.217 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.214+0000 7f77b5d9b640 1 --2- 172.21.15.49:0/3237933663 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f77b80cb010 0x7f77b8157000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:18.217 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.214+0000 7f77b659c640 1 -- 172.21.15.49:0/3237933663 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f77b8159980 msgr2=0x7f77b81493c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:18.218 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.214+0000 7f77b659c640 1 --2- 172.21.15.49:0/3237933663 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f77b8159980 0x7f77b81493c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:18.218 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.214+0000 7f77b659c640 1 -- 172.21.15.49:0/3237933663 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f77b80cb010 msgr2=0x7f77b8157000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:18.218 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.214+0000 7f77b659c640 1 --2- 172.21.15.49:0/3237933663 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f77b80cb010 0x7f77b8157000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:18.218 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.214+0000 7f77b659c640 1 -- 172.21.15.49:0/3237933663 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77ac00b770 con 0x7f77b8071290 2023-11-06T02:13:18.218 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.214+0000 7f77b5d9b640 1 --2- 172.21.15.49:0/3237933663 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f77b80cb010 0x7f77b8157000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:13:18.219 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.218+0000 7f77b659c640 1 --2- 172.21.15.49:0/3237933663 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77b8071290 0x7f77b8150be0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f77a800b3f0 tx=0x7f77a800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:18.219 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.218+0000 7f77a6ffd640 1 -- 172.21.15.49:0/3237933663 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a8011830 con 0x7f77b8071290 2023-11-06T02:13:18.219 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.218+0000 7f77a6ffd640 1 -- 172.21.15.49:0/3237933663 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77a80119b0 con 0x7f77b8071290 2023-11-06T02:13:18.219 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.218+0000 7f77bcd10640 1 -- 172.21.15.49:0/3237933663 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77b814f250 con 0x7f77b8071290 2023-11-06T02:13:18.219 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.218+0000 7f77a6ffd640 1 -- 172.21.15.49:0/3237933663 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a8011d60 con 0x7f77b8071290 2023-11-06T02:13:18.220 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.218+0000 7f77bcd10640 1 -- 172.21.15.49:0/3237933663 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77b814f810 con 0x7f77b8071290 2023-11-06T02:13:18.220 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.218+0000 7f77bcd10640 1 -- 172.21.15.49:0/3237933663 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77b8074040 con 0x7f77b8071290 2023-11-06T02:13:18.220 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.218+0000 7f77a6ffd640 1 -- 172.21.15.49:0/3237933663 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f77a801a410 con 0x7f77b8071290 2023-11-06T02:13:18.220 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.218+0000 7f77a6ffd640 1 --2- 172.21.15.49:0/3237933663 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7798065a60 0x7f7798067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:18.220 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.218+0000 7f77a6ffd640 1 -- 172.21.15.49:0/3237933663 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6350+0+0 (secure 0 0 0) 0x7f77a80832b0 con 0x7f77b8071290 2023-11-06T02:13:18.221 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.218+0000 7f77b5d9b640 1 --2- 172.21.15.49:0/3237933663 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7798065a60 0x7f7798067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:18.222 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.218+0000 7f77b5d9b640 1 --2- 172.21.15.49:0/3237933663 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7798065a60 0x7f7798067f20 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f77ac0b1000 tx=0x7f77ac0a9070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:18.222 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.222+0000 7f77a6ffd640 1 -- 172.21.15.49:0/3237933663 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f77a80509f0 con 0x7f77b8071290 2023-11-06T02:13:18.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.414+0000 7f77bcd10640 1 -- 172.21.15.49:0/3237933663 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f77b8158d00 con 0x7f77b8071290 2023-11-06T02:13:18.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.414+0000 7f77a6ffd640 1 -- 172.21.15.49:0/3237933663 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+3891 (secure 0 0 0) 0x7f77a8054010 con 0x7f77b8071290 2023-11-06T02:13:18.411 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:13:18.411 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":37,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5635,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4170291905","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":4170291905},{"type":"v1","addr":"172.21.15.150:6835","nonce":4170291905}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5638,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/2719458937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":2719458937},{"type":"v1","addr":"172.21.15.150:6837","nonce":2719458937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":6339,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/2310338375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2310338375},{"type":"v1","addr":"172.21.15.49:6833","nonce":2310338375}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":37,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T02:11:30.091070+0000","modified":"2023-11-06T02:13:17.615097+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-06T02:13:18.411 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 37 2023-11-06T02:13:18.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.418+0000 7f77bcd10640 1 -- 172.21.15.49:0/3237933663 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7798065a60 msgr2=0x7f7798067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:18.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.418+0000 7f77bcd10640 1 --2- 172.21.15.49:0/3237933663 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7798065a60 0x7f7798067f20 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f77ac0b1000 tx=0x7f77ac0a9070 comp rx=0 tx=0).stop 2023-11-06T02:13:18.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.418+0000 7f77bcd10640 1 -- 172.21.15.49:0/3237933663 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77b8071290 msgr2=0x7f77b8150be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:18.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.418+0000 7f77bcd10640 1 --2- 172.21.15.49:0/3237933663 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77b8071290 0x7f77b8150be0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f77a800b3f0 tx=0x7f77a800b8b0 comp rx=0 tx=0).stop 2023-11-06T02:13:18.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.418+0000 7f77bcd10640 1 -- 172.21.15.49:0/3237933663 shutdown_connections 2023-11-06T02:13:18.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.418+0000 7f77bcd10640 1 --2- 172.21.15.49:0/3237933663 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7798065a60 0x7f7798067f20 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:18.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.418+0000 7f77bcd10640 1 --2- 172.21.15.49:0/3237933663 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f77b8159980 0x7f77b81493c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:18.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.418+0000 7f77bcd10640 1 --2- 172.21.15.49:0/3237933663 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f77b80cb010 0x7f77b8157000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:18.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.418+0000 7f77bcd10640 1 --2- 172.21.15.49:0/3237933663 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77b8071290 0x7f77b8150be0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:18.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.418+0000 7f77bcd10640 1 -- 172.21.15.49:0/3237933663 >> 172.21.15.49:0/3237933663 conn(0x7f77b806c0a0 msgr2=0x7f77b80b19d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:18.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.418+0000 7f77bcd10640 1 -- 172.21.15.49:0/3237933663 shutdown_connections 2023-11-06T02:13:18.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.418+0000 7f77bcd10640 1 -- 172.21.15.49:0/3237933663 wait complete. 2023-11-06T02:13:18.430 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T02:13:18.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.550+0000 7fe890e13640 1 Processor -- start 2023-11-06T02:13:18.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.550+0000 7fe890e13640 1 -- start start 2023-11-06T02:13:18.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.554+0000 7fe890e13640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fe88c071290 0x7fe88c0b1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:18.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.554+0000 7fe890e13640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe88c0cb010 0x7fe88c0cb480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:18.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.554+0000 7fe890e13640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe88c071890 con 0x7fe88c159980 2023-11-06T02:13:18.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.554+0000 7fe890e13640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe88c072410 con 0x7fe88c0cb010 2023-11-06T02:13:18.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.554+0000 7fe890e13640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe88c0ce440 con 0x7fe88c071290 2023-11-06T02:13:18.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.554+0000 7fe88ad9d640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe88c0cb010 0x7fe88c0cb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:18.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.554+0000 7fe88ad9d640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe88c0cb010 0x7fe88c0cb480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48612/0 (socket says 172.21.15.49:48612) 2023-11-06T02:13:18.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.554+0000 7fe88ad9d640 1 -- 172.21.15.49:0/1892321410 learned_addr learned my addr 172.21.15.49:0/1892321410 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:18.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.554+0000 7fe881d9b640 1 --2- 172.21.15.49:0/1892321410 >> v2:172.21.15.150:3300/0 conn(0x7fe88c071290 0x7fe88c0b1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:18.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.554+0000 7fe88ad9d640 1 -- 172.21.15.49:0/1892321410 >> v2:172.21.15.150:3300/0 conn(0x7fe88c071290 msgr2=0x7fe88c0b1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:18.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.554+0000 7fe88ad9d640 1 --2- 172.21.15.49:0/1892321410 >> v2:172.21.15.150:3300/0 conn(0x7fe88c071290 0x7fe88c0b1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:18.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.554+0000 7fe88ad9d640 1 -- 172.21.15.49:0/1892321410 >> v1:172.21.15.150:6789/0 conn(0x7fe88c159980 legacy=0x7fe88c0cdd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:18.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.554+0000 7fe88ad9d640 1 -- 172.21.15.49:0/1892321410 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe88c071b70 con 0x7fe88c0cb010 2023-11-06T02:13:18.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.558+0000 7fe88ad9d640 1 --2- 172.21.15.49:0/1892321410 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe88c0cb010 0x7fe88c0cb480 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fe88c06d090 tx=0x7fe88401c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=244fa532c08f40a4 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:18.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.558+0000 7fe889d9b640 1 -- 172.21.15.49:0/1892321410 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8840a5720 con 0x7fe88c0cb010 2023-11-06T02:13:18.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.558+0000 7fe889d9b640 1 -- 172.21.15.49:0/1892321410 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8840a5880 con 0x7fe88c0cb010 2023-11-06T02:13:18.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.558+0000 7fe889d9b640 1 -- 172.21.15.49:0/1892321410 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8840a5c10 con 0x7fe88c0cb010 2023-11-06T02:13:18.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.558+0000 7fe890e13640 1 -- 172.21.15.49:0/1892321410 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe88c0cb010 msgr2=0x7fe88c0cb480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:18.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.558+0000 7fe890e13640 1 --2- 172.21.15.49:0/1892321410 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe88c0cb010 0x7fe88c0cb480 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fe88c06d090 tx=0x7fe88401c950 comp rx=0 tx=0).stop 2023-11-06T02:13:18.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.558+0000 7fe890e13640 1 -- 172.21.15.49:0/1892321410 shutdown_connections 2023-11-06T02:13:18.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.558+0000 7fe890e13640 1 --2- 172.21.15.49:0/1892321410 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe88c0cb010 0x7fe88c0cb480 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:18.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.558+0000 7fe890e13640 1 --2- 172.21.15.49:0/1892321410 >> v2:172.21.15.150:3300/0 conn(0x7fe88c071290 0x7fe88c0b1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:18.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.558+0000 7fe890e13640 1 -- 172.21.15.49:0/1892321410 >> 172.21.15.49:0/1892321410 conn(0x7fe88c06c0a0 msgr2=0x7fe88c06c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:18.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.558+0000 7fe890e13640 1 -- 172.21.15.49:0/1892321410 shutdown_connections 2023-11-06T02:13:18.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.558+0000 7fe890e13640 1 -- 172.21.15.49:0/1892321410 wait complete. 2023-11-06T02:13:18.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.558+0000 7fe890e13640 1 Processor -- start 2023-11-06T02:13:18.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.558+0000 7fe890e13640 1 -- start start 2023-11-06T02:13:18.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.558+0000 7fe890e13640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe88c071290 0x7fe88c14fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:18.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.558+0000 7fe890e13640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe88c0cb010 0x7fe88c0b38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:18.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.558+0000 7fe890e13640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe88c159980 0x7fe88c159fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:18.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.558+0000 7fe890e13640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe88c0b2830 con 0x7fe88c0cb010 2023-11-06T02:13:18.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.558+0000 7fe890e13640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe88c0b2b10 con 0x7fe88c071290 2023-11-06T02:13:18.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.558+0000 7fe890e13640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe88c0b2df0 con 0x7fe88c159980 2023-11-06T02:13:18.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.558+0000 7fe881d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe88c0cb010 0x7fe88c0b38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:18.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.558+0000 7fe881d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe88c0cb010 0x7fe88c0b38d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48614/0 (socket says 172.21.15.49:48614) 2023-11-06T02:13:18.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.558+0000 7fe881d9b640 1 -- 172.21.15.49:0/404583617 learned_addr learned my addr 172.21.15.49:0/404583617 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:18.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.562+0000 7fe88a59c640 1 --2- 172.21.15.49:0/404583617 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe88c071290 0x7fe88c14fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:18.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.562+0000 7fe88ad9d640 1 --2- 172.21.15.49:0/404583617 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe88c159980 0x7fe88c159fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:18.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.562+0000 7fe881d9b640 1 -- 172.21.15.49:0/404583617 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe88c159980 msgr2=0x7fe88c159fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:18.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.562+0000 7fe881d9b640 1 --2- 172.21.15.49:0/404583617 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe88c159980 0x7fe88c159fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:18.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.562+0000 7fe881d9b640 1 -- 172.21.15.49:0/404583617 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe88c071290 msgr2=0x7fe88c14fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:18.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.562+0000 7fe881d9b640 1 --2- 172.21.15.49:0/404583617 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe88c071290 0x7fe88c14fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:18.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.562+0000 7fe881d9b640 1 -- 172.21.15.49:0/404583617 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe884009b30 con 0x7fe88c0cb010 2023-11-06T02:13:18.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.562+0000 7fe88a59c640 1 --2- 172.21.15.49:0/404583617 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe88c071290 0x7fe88c14fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:13:18.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.562+0000 7fe881d9b640 1 --2- 172.21.15.49:0/404583617 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe88c0cb010 0x7fe88c0b38d0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fe87400d800 tx=0x7fe87400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:18.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.562+0000 7fe8837fe640 1 -- 172.21.15.49:0/404583617 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe87400b6c0 con 0x7fe88c0cb010 2023-11-06T02:13:18.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.562+0000 7fe8837fe640 1 -- 172.21.15.49:0/404583617 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe87400b820 con 0x7fe88c0cb010 2023-11-06T02:13:18.563 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.562+0000 7fe890e13640 1 -- 172.21.15.49:0/404583617 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe88c0b30d0 con 0x7fe88c0cb010 2023-11-06T02:13:18.563 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.562+0000 7fe8837fe640 1 -- 172.21.15.49:0/404583617 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe87400bb90 con 0x7fe88c0cb010 2023-11-06T02:13:18.563 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.562+0000 7fe890e13640 1 -- 172.21.15.49:0/404583617 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe88c150010 con 0x7fe88c0cb010 2023-11-06T02:13:18.563 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.562+0000 7fe890e13640 1 -- 172.21.15.49:0/404583617 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe88c074040 con 0x7fe88c0cb010 2023-11-06T02:13:18.563 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.562+0000 7fe8837fe640 1 -- 172.21.15.49:0/404583617 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fe87400bcf0 con 0x7fe88c0cb010 2023-11-06T02:13:18.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.562+0000 7fe8837fe640 1 --2- 172.21.15.49:0/404583617 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe86c065a60 0x7fe86c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:18.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.562+0000 7fe8837fe640 1 -- 172.21.15.49:0/404583617 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6350+0+0 (secure 0 0 0) 0x7fe8740839b0 con 0x7fe88c0cb010 2023-11-06T02:13:18.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.562+0000 7fe88a59c640 1 --2- 172.21.15.49:0/404583617 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe86c065a60 0x7fe86c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:18.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.562+0000 7fe88a59c640 1 --2- 172.21.15.49:0/404583617 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe86c065a60 0x7fe86c067f20 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fe87c00a830 tx=0x7fe87c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:18.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.566+0000 7fe8837fe640 1 -- 172.21.15.49:0/404583617 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fe8740510f0 con 0x7fe88c0cb010 2023-11-06T02:13:18.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.730+0000 7fe890e13640 1 -- 172.21.15.49:0/404583617 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe88c159000 con 0x7fe88c0cb010 2023-11-06T02:13:18.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.730+0000 7fe8837fe640 1 -- 172.21.15.49:0/404583617 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) v1 ==== 74+0+18275 (secure 0 0 0) 0x7fe874054710 con 0x7fe88c0cb010 2023-11-06T02:13:18.728 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:13:18.728 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":69,"fsid":"1a8b035c-6ba4-499a-b876-aea47fd9c0d4","created":"2023-11-06T02:08:06.440857+0000","modified":"2023-11-06T02:13:17.614076+0000","last_up_change":"2023-11-06T02:08:17.982372+0000","last_in_change":"2023-11-06T02:08:08.906631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T02:08:19.252222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T02:08:24.776211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-06T02:11:28.648326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-06T02:11:29.079513+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ad55af8-0571-4e6a-8722-b5d429b7496d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6809","nonce":38116},{"type":"v1","addr":"172.21.15.49:6811","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6813","nonce":38116},{"type":"v1","addr":"172.21.15.49:6815","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6821","nonce":38116},{"type":"v1","addr":"172.21.15.49:6823","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6817","nonce":38116},{"type":"v1","addr":"172.21.15.49:6819","nonce":38116}]},"public_addr":"172.21.15.49:6811/38116","cluster_addr":"172.21.15.49:6815/38116","heartbeat_back_addr":"172.21.15.49:6823/38116","heartbeat_front_addr":"172.21.15.49:6819/38116","state":["exists","up"]},{"osd":1,"uuid":"c8b31f85-a757-43ed-9419-351acc46611b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":38113},{"type":"v1","addr":"172.21.15.49:6810","nonce":38113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":38113},{"type":"v1","addr":"172.21.15.49:6814","nonce":38113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":38113},{"type":"v1","addr":"172.21.15.49:6822","nonce":38113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":38113},{"type":"v1","addr":"172.21.15.49:6818","nonce":38113}]},"public_addr":"172.21.15.49:6810/38113","cluster_addr":"172.21.15.49:6814/38113","heartbeat_back_addr":"172.21.15.49:6822/38113","heartbeat_front_addr":"172.21.15.49:6818/38113","state":["exists","up"]},{"osd":2,"uuid":"555b3bf3-0a70-4b91-8efd-e4204b5fc26e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":38114},{"type":"v1","addr":"172.21.15.49:6801","nonce":38114}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":38114},{"type":"v1","addr":"172.21.15.49:6803","nonce":38114}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":38114},{"type":"v1","addr":"172.21.15.49:6807","nonce":38114}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":38114},{"type":"v1","addr":"172.21.15.49:6805","nonce":38114}]},"public_addr":"172.21.15.49:6801/38114","cluster_addr":"172.21.15.49:6803/38114","heartbeat_back_addr":"172.21.15.49:6807/38114","heartbeat_front_addr":"172.21.15.49:6805/38114","state":["exists","up"]},{"osd":3,"uuid":"8d4e5d77-03ef-4b9b-bfcb-5ae93a46795c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":38115},{"type":"v1","addr":"172.21.15.49:6825","nonce":38115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":38115},{"type":"v1","addr":"172.21.15.49:6827","nonce":38115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":38115},{"type":"v1","addr":"172.21.15.49:6831","nonce":38115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":38115},{"type":"v1","addr":"172.21.15.49:6829","nonce":38115}]},"public_addr":"172.21.15.49:6825/38115","cluster_addr":"172.21.15.49:6827/38115","heartbeat_back_addr":"172.21.15.49:6831/38115","heartbeat_front_addr":"172.21.15.49:6829/38115","state":["exists","up"]},{"osd":4,"uuid":"dba03ad4-68a3-4cf3-9218-ac611f02eacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":38417},{"type":"v1","addr":"172.21.15.150:6809","nonce":38417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":38417},{"type":"v1","addr":"172.21.15.150:6811","nonce":38417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":38417},{"type":"v1","addr":"172.21.15.150:6815","nonce":38417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":38417},{"type":"v1","addr":"172.21.15.150:6813","nonce":38417}]},"public_addr":"172.21.15.150:6809/38417","cluster_addr":"172.21.15.150:6811/38417","heartbeat_back_addr":"172.21.15.150:6815/38417","heartbeat_front_addr":"172.21.15.150:6813/38417","state":["exists","up"]},{"osd":5,"uuid":"0df9c303-c255-4640-8e91-01f1c3efe18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":38418},{"type":"v1","addr":"172.21.15.150:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":38418},{"type":"v1","addr":"172.21.15.150:6803","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":38418},{"type":"v1","addr":"172.21.15.150:6807","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":38418},{"type":"v1","addr":"172.21.15.150:6805","nonce":38418}]},"public_addr":"172.21.15.150:6801/38418","cluster_addr":"172.21.15.150:6803/38418","heartbeat_back_addr":"172.21.15.150:6807/38418","heartbeat_front_addr":"172.21.15.150:6805/38418","state":["exists","up"]},{"osd":6,"uuid":"e59fa189-4b70-4c06-9722-c02b90828bf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":38428},{"type":"v1","addr":"172.21.15.150:6825","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":38428},{"type":"v1","addr":"172.21.15.150:6827","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":38428},{"type":"v1","addr":"172.21.15.150:6831","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":38428},{"type":"v1","addr":"172.21.15.150:6829","nonce":38428}]},"public_addr":"172.21.15.150:6825/38428","cluster_addr":"172.21.15.150:6827/38428","heartbeat_back_addr":"172.21.15.150:6831/38428","heartbeat_front_addr":"172.21.15.150:6829/38428","state":["exists","up"]},{"osd":7,"uuid":"a64f57e1-ef98-46bd-b304-385759e95898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":38421},{"type":"v1","addr":"172.21.15.150:6817","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":38421},{"type":"v1","addr":"172.21.15.150:6819","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":38421},{"type":"v1","addr":"172.21.15.150:6823","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":38421},{"type":"v1","addr":"172.21.15.150:6821","nonce":38421}]},"public_addr":"172.21.15.150:6817/38421","cluster_addr":"172.21.15.150:6819/38421","heartbeat_back_addr":"172.21.15.150:6823/38421","heartbeat_front_addr":"172.21.15.150:6821/38421","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.438552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.413280+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.296466+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.424966+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.276953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.230984+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.410235+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.480084+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.10","mappings":[{"from":7,"to":2}]},{"pgid":"10.1d","mappings":[{"from":4,"to":3}]},{"pgid":"10.21","mappings":[{"from":4,"to":3}]},{"pgid":"10.3a","mappings":[{"from":7,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:6835/3698847494":"2023-11-07T02:13:17.614029+0000","172.21.15.49:6834/3698847494":"2023-11-07T02:13:17.614029+0000","172.21.15.49:6835/342908487":"2023-11-07T02:12:41.031290+0000","172.21.15.49:6834/342908487":"2023-11-07T02:12:41.031290+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T02:13:18.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.734+0000 7fe890e13640 1 -- 172.21.15.49:0/404583617 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe86c065a60 msgr2=0x7fe86c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:18.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.734+0000 7fe890e13640 1 --2- 172.21.15.49:0/404583617 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe86c065a60 0x7fe86c067f20 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fe87c00a830 tx=0x7fe87c008040 comp rx=0 tx=0).stop 2023-11-06T02:13:18.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.734+0000 7fe890e13640 1 -- 172.21.15.49:0/404583617 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe88c0cb010 msgr2=0x7fe88c0b38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:18.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.734+0000 7fe890e13640 1 --2- 172.21.15.49:0/404583617 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe88c0cb010 0x7fe88c0b38d0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fe87400d800 tx=0x7fe87400dcc0 comp rx=0 tx=0).stop 2023-11-06T02:13:18.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.734+0000 7fe890e13640 1 -- 172.21.15.49:0/404583617 shutdown_connections 2023-11-06T02:13:18.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.734+0000 7fe890e13640 1 --2- 172.21.15.49:0/404583617 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe86c065a60 0x7fe86c067f20 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:18.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.734+0000 7fe890e13640 1 --2- 172.21.15.49:0/404583617 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe88c0cb010 0x7fe88c0b38d0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:18.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.734+0000 7fe890e13640 1 --2- 172.21.15.49:0/404583617 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe88c159980 0x7fe88c159fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:18.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.734+0000 7fe890e13640 1 --2- 172.21.15.49:0/404583617 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe88c071290 0x7fe88c14fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:18.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.734+0000 7fe890e13640 1 -- 172.21.15.49:0/404583617 >> 172.21.15.49:0/404583617 conn(0x7fe88c06c0a0 msgr2=0x7fe88c150c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:18.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.734+0000 7fe890e13640 1 -- 172.21.15.49:0/404583617 shutdown_connections 2023-11-06T02:13:18.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.734+0000 7fe890e13640 1 -- 172.21.15.49:0/404583617 wait complete. 2023-11-06T02:13:18.747 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T02:08:19.252222+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T02:08:24.776211+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T02:11:28.648326+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T02:11:29.079513+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '65', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T02:13:18.748 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T02:13:18.877 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.882+0000 7f7987d5f640 1 Processor -- start 2023-11-06T02:13:18.877 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.882+0000 7f7987d5f640 1 -- start start 2023-11-06T02:13:18.878 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.882+0000 7f7987d5f640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f79800afe20 0x7f79800ce9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:18.878 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.882+0000 7f7987d5f640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f79800d0fe0 0x7f79800b0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:18.879 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.882+0000 7f7987d5f640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79800cf340 con 0x7f798006bda0 2023-11-06T02:13:18.879 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.882+0000 7f7987d5f640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79800cf620 con 0x7f79800afe20 2023-11-06T02:13:18.879 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.882+0000 7f7987d5f640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79800cf900 con 0x7f79800d0fe0 2023-11-06T02:13:18.879 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.882+0000 7f7985afb640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f79800afe20 0x7f79800ce9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:18.879 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.882+0000 7f7985afb640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f79800afe20 0x7f79800ce9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48626/0 (socket says 172.21.15.49:48626) 2023-11-06T02:13:18.880 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.882+0000 7f7985afb640 1 -- 172.21.15.49:0/3685010072 learned_addr learned my addr 172.21.15.49:0/3685010072 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:18.880 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.882+0000 7f79862fc640 1 --2- 172.21.15.49:0/3685010072 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f79800d0fe0 0x7f79800b0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:18.880 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.882+0000 7f7985afb640 1 -- 172.21.15.49:0/3685010072 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f79800d0fe0 msgr2=0x7f79800b0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:18.880 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.882+0000 7f7985afb640 1 --2- 172.21.15.49:0/3685010072 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f79800d0fe0 0x7f79800b0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:18.880 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.882+0000 7f7985afb640 1 -- 172.21.15.49:0/3685010072 >> v1:172.21.15.150:6789/0 conn(0x7f798006bda0 legacy=0x7f79800d08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:18.881 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.882+0000 7f7985afb640 1 -- 172.21.15.49:0/3685010072 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79800cfbe0 con 0x7f79800afe20 2023-11-06T02:13:18.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.882+0000 7f7985afb640 1 --2- 172.21.15.49:0/3685010072 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f79800afe20 0x7f79800ce9f0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f7978009870 tx=0x7f797801c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d356efac93ce52b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:18.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.886+0000 7f7984af9640 1 -- 172.21.15.49:0/3685010072 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79800cf140 con 0x7f79800afe20 2023-11-06T02:13:18.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.886+0000 7f7984af9640 1 -- 172.21.15.49:0/3685010072 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79780ab650 con 0x7f79800afe20 2023-11-06T02:13:18.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.886+0000 7f7984af9640 1 -- 172.21.15.49:0/3685010072 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79780ab9e0 con 0x7f79800afe20 2023-11-06T02:13:18.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.886+0000 7f7987d5f640 1 -- 172.21.15.49:0/3685010072 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f79800afe20 msgr2=0x7f79800ce9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:18.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.886+0000 7f7987d5f640 1 --2- 172.21.15.49:0/3685010072 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f79800afe20 0x7f79800ce9f0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f7978009870 tx=0x7f797801c4b0 comp rx=0 tx=0).stop 2023-11-06T02:13:18.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.886+0000 7f7987d5f640 1 -- 172.21.15.49:0/3685010072 shutdown_connections 2023-11-06T02:13:18.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.886+0000 7f7987d5f640 1 --2- 172.21.15.49:0/3685010072 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f79800d0fe0 0x7f79800b0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:18.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.886+0000 7f7987d5f640 1 --2- 172.21.15.49:0/3685010072 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f79800afe20 0x7f79800ce9f0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:18.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.886+0000 7f7987d5f640 1 -- 172.21.15.49:0/3685010072 >> 172.21.15.49:0/3685010072 conn(0x7f7980089fd0 msgr2=0x7f7980099c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:18.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.886+0000 7f7987d5f640 1 -- 172.21.15.49:0/3685010072 shutdown_connections 2023-11-06T02:13:18.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.886+0000 7f7987d5f640 1 -- 172.21.15.49:0/3685010072 wait complete. 2023-11-06T02:13:18.886 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.886+0000 7f7987d5f640 1 Processor -- start 2023-11-06T02:13:18.886 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.886+0000 7f7987d5f640 1 -- start start 2023-11-06T02:13:18.886 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.886+0000 7f7987d5f640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f798006bda0 0x7f798014f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:18.886 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.886+0000 7f7987d5f640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79800afe20 0x7f79800cb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:18.887 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.886+0000 7f7987d5f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f79800d0fe0 0x7f79800cbee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:18.887 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.886+0000 7f7987d5f640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f798014ffe0 con 0x7f79800d0fe0 2023-11-06T02:13:18.887 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.886+0000 7f7987d5f640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79801502c0 con 0x7f79800afe20 2023-11-06T02:13:18.887 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.886+0000 7f7987d5f640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79801505a0 con 0x7f798006bda0 2023-11-06T02:13:18.887 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.886+0000 7f79862fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f79800d0fe0 0x7f79800cbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:18.888 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.886+0000 7f79862fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f79800d0fe0 0x7f79800cbee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48636/0 (socket says 172.21.15.49:48636) 2023-11-06T02:13:18.888 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.886+0000 7f79862fc640 1 -- 172.21.15.49:0/3079156654 learned_addr learned my addr 172.21.15.49:0/3079156654 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:18.888 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.886+0000 7f7985afb640 1 --2- 172.21.15.49:0/3079156654 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f798006bda0 0x7f798014f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:18.888 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.886+0000 7f79852fa640 1 --2- 172.21.15.49:0/3079156654 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79800afe20 0x7f79800cb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:18.888 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.886+0000 7f79862fc640 1 -- 172.21.15.49:0/3079156654 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f798006bda0 msgr2=0x7f798014f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:18.888 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.886+0000 7f79862fc640 1 --2- 172.21.15.49:0/3079156654 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f798006bda0 0x7f798014f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:18.889 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.886+0000 7f79862fc640 1 -- 172.21.15.49:0/3079156654 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79800afe20 msgr2=0x7f79800cb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:18.889 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.886+0000 7f79862fc640 1 --2- 172.21.15.49:0/3079156654 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79800afe20 0x7f79800cb9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:18.889 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.886+0000 7f79862fc640 1 -- 172.21.15.49:0/3079156654 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79780094d0 con 0x7f79800d0fe0 2023-11-06T02:13:18.889 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.886+0000 7f79862fc640 1 --2- 172.21.15.49:0/3079156654 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f79800d0fe0 0x7f79800cbee0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f797c00e7f0 tx=0x7f797c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:18.889 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.886+0000 7f79727fc640 1 -- 172.21.15.49:0/3079156654 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f797c0094d0 con 0x7f79800d0fe0 2023-11-06T02:13:18.889 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.886+0000 7f79727fc640 1 -- 172.21.15.49:0/3079156654 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f797c00eea0 con 0x7f79800d0fe0 2023-11-06T02:13:18.890 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.886+0000 7f79727fc640 1 -- 172.21.15.49:0/3079156654 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f797c00cb60 con 0x7f79800d0fe0 2023-11-06T02:13:18.890 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.886+0000 7f7987d5f640 1 -- 172.21.15.49:0/3079156654 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7980150880 con 0x7f79800d0fe0 2023-11-06T02:13:18.890 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.886+0000 7f7987d5f640 1 -- 172.21.15.49:0/3079156654 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79800ca630 con 0x7f79800d0fe0 2023-11-06T02:13:18.890 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.890+0000 7f7987d5f640 1 -- 172.21.15.49:0/3079156654 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7980067140 con 0x7f79800d0fe0 2023-11-06T02:13:18.890 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.890+0000 7f79727fc640 1 -- 172.21.15.49:0/3079156654 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f797c00ccc0 con 0x7f79800d0fe0 2023-11-06T02:13:18.891 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.890+0000 7f79727fc640 1 --2- 172.21.15.49:0/3079156654 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7964065a90 0x7f7964067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:18.891 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.890+0000 7f79727fc640 1 -- 172.21.15.49:0/3079156654 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6350+0+0 (secure 0 0 0) 0x7f797c083470 con 0x7f79800d0fe0 2023-11-06T02:13:18.891 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.890+0000 7f7985afb640 1 --2- 172.21.15.49:0/3079156654 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7964065a90 0x7f7964067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:18.891 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.890+0000 7f7985afb640 1 --2- 172.21.15.49:0/3079156654 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7964065a90 0x7f7964067f50 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f79780af000 tx=0x7f79780a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:18.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:18.894+0000 7f79727fc640 1 -- 172.21.15.49:0/3079156654 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f797c050bb0 con 0x7f79800d0fe0 2023-11-06T02:13:19.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.042+0000 7f7987d5f640 1 -- 172.21.15.49:0/3079156654 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7980150d30 con 0x7f79800d0fe0 2023-11-06T02:13:19.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.042+0000 7f79727fc640 1 -- 172.21.15.49:0/3079156654 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v69) v1 ==== 93+0+10 (secure 0 0 0) 0x7f797c0541d0 con 0x7f79800d0fe0 2023-11-06T02:13:19.041 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2023-11-06T02:13:19.042 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.046+0000 7f7987d5f640 1 -- 172.21.15.49:0/3079156654 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7964065a90 msgr2=0x7f7964067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:19.042 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.046+0000 7f7987d5f640 1 --2- 172.21.15.49:0/3079156654 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7964065a90 0x7f7964067f50 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f79780af000 tx=0x7f79780a7000 comp rx=0 tx=0).stop 2023-11-06T02:13:19.043 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.046+0000 7f7987d5f640 1 -- 172.21.15.49:0/3079156654 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f79800d0fe0 msgr2=0x7f79800cbee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:19.043 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.046+0000 7f7987d5f640 1 --2- 172.21.15.49:0/3079156654 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f79800d0fe0 0x7f79800cbee0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f797c00e7f0 tx=0x7f797c00ecb0 comp rx=0 tx=0).stop 2023-11-06T02:13:19.043 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.046+0000 7f7987d5f640 1 -- 172.21.15.49:0/3079156654 shutdown_connections 2023-11-06T02:13:19.043 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.046+0000 7f7987d5f640 1 --2- 172.21.15.49:0/3079156654 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7964065a90 0x7f7964067f50 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:19.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.046+0000 7f7987d5f640 1 --2- 172.21.15.49:0/3079156654 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f79800d0fe0 0x7f79800cbee0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:19.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.046+0000 7f7987d5f640 1 --2- 172.21.15.49:0/3079156654 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79800afe20 0x7f79800cb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:19.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.046+0000 7f7987d5f640 1 --2- 172.21.15.49:0/3079156654 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f798006bda0 0x7f798014f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:19.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.046+0000 7f7987d5f640 1 -- 172.21.15.49:0/3079156654 >> 172.21.15.49:0/3079156654 conn(0x7f7980089fd0 msgr2=0x7f798006fb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:19.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.046+0000 7f7987d5f640 1 -- 172.21.15.49:0/3079156654 shutdown_connections 2023-11-06T02:13:19.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.046+0000 7f7987d5f640 1 -- 172.21.15.49:0/3079156654 wait complete. 2023-11-06T02:13:19.058 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T02:13:19.186 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.190+0000 7ff0cc1ba640 1 Processor -- start 2023-11-06T02:13:19.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.190+0000 7ff0cc1ba640 1 -- start start 2023-11-06T02:13:19.199 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.190+0000 7ff0cc1ba640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7ff0c415a880 0x7ff0c40b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:19.199 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.190+0000 7ff0cc1ba640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff0c4070160 0x7ff0c4070610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:19.199 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.190+0000 7ff0cc1ba640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0c40ae360 con 0x7ff0c406d150 2023-11-06T02:13:19.200 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.190+0000 7ff0cc1ba640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0c4070b50 con 0x7ff0c415a880 2023-11-06T02:13:19.201 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.190+0000 7ff0cc1ba640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c40ae640 con 0x7ff0c4070160 2023-11-06T02:13:19.201 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.190+0000 7ff0ca757640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7ff0c406d150 0x7ff0c406d5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:46726/0 (socket says 172.21.15.49:46726) 2023-11-06T02:13:19.201 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.190+0000 7ff0ca757640 1 -- 172.21.15.49:0/1878691528 learned_addr learned my addr 172.21.15.49:0/1878691528 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:13:19.202 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.190+0000 7ff0c9755640 1 --2- 172.21.15.49:0/1878691528 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff0c4070160 0x7ff0c4070610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:19.202 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.190+0000 7ff0c9f56640 1 --2- 172.21.15.49:0/1878691528 >> v2:172.21.15.150:3300/0 conn(0x7ff0c415a880 0x7ff0c40b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:19.202 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.190+0000 7ff0c8f54640 1 -- 172.21.15.49:0/1878691528 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1574852339 0 0) 0x7ff0c40ae360 con 0x7ff0c406d150 2023-11-06T02:13:19.202 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.190+0000 7ff0c8f54640 1 -- 172.21.15.49:0/1878691528 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0b0003530 con 0x7ff0c406d150 2023-11-06T02:13:19.202 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.190+0000 7ff0c8f54640 1 -- 172.21.15.49:0/1878691528 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2714258551 0 0) 0x7ff0b0003530 con 0x7ff0c406d150 2023-11-06T02:13:19.203 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.190+0000 7ff0c8f54640 1 -- 172.21.15.49:0/1878691528 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff0c4070160 msgr2=0x7ff0c4070610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:19.203 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.190+0000 7ff0c8f54640 1 --2- 172.21.15.49:0/1878691528 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff0c4070160 0x7ff0c4070610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:19.203 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.190+0000 7ff0c8f54640 1 -- 172.21.15.49:0/1878691528 >> v2:172.21.15.150:3300/0 conn(0x7ff0c415a880 msgr2=0x7ff0c40b15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:19.203 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.190+0000 7ff0c8f54640 1 --2- 172.21.15.49:0/1878691528 >> v2:172.21.15.150:3300/0 conn(0x7ff0c415a880 0x7ff0c40b15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:19.203 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.190+0000 7ff0c8f54640 1 -- 172.21.15.49:0/1878691528 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0c40ae920 con 0x7ff0c406d150 2023-11-06T02:13:19.204 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.190+0000 7ff0c8f54640 1 -- 172.21.15.49:0/1878691528 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7ff0bc001a00 con 0x7ff0c406d150 2023-11-06T02:13:19.204 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.190+0000 7ff0c8f54640 1 -- 172.21.15.49:0/1878691528 >> v1:172.21.15.49:6789/0 conn(0x7ff0c406d150 legacy=0x7ff0c406d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:19.204 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.190+0000 7ff0c8f54640 1 --2- 172.21.15.49:0/1878691528 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0b0003980 0x7ff0b0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:19.204 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.190+0000 7ff0c8f54640 1 --2- 172.21.15.49:0/1878691528 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff0c4070160 0x7ff0b00047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:19.204 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.190+0000 7ff0c8f54640 1 --2- 172.21.15.49:0/1878691528 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff0b0004d10 0x7ff0b0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:19.205 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.190+0000 7ff0c8f54640 1 -- 172.21.15.49:0/1878691528 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c4070b50 con 0x7ff0b0004d10 2023-11-06T02:13:19.205 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.190+0000 7ff0c8f54640 1 -- 172.21.15.49:0/1878691528 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c40ae640 con 0x7ff0b0003980 2023-11-06T02:13:19.205 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.190+0000 7ff0c8f54640 1 -- 172.21.15.49:0/1878691528 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b000b990 con 0x7ff0c4070160 2023-11-06T02:13:19.205 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.190+0000 7ff0ca757640 1 --2- 172.21.15.49:0/1878691528 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff0b0004d10 0x7ff0b0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:19.205 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.194+0000 7ff0ca757640 1 -- 172.21.15.49:0/1878691528 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff0c4070160 msgr2=0x7ff0b00047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:19.206 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.194+0000 7ff0c9755640 1 --2- 172.21.15.49:0/1878691528 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff0c4070160 0x7ff0b00047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:19.206 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.194+0000 7ff0c9f56640 1 --2- 172.21.15.49:0/1878691528 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0b0003980 0x7ff0b0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:19.206 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.194+0000 7ff0ca757640 1 --2- 172.21.15.49:0/1878691528 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff0c4070160 0x7ff0b00047d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:19.206 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.194+0000 7ff0ca757640 1 -- 172.21.15.49:0/1878691528 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0b0003980 msgr2=0x7ff0b0003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:19.206 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.194+0000 7ff0ca757640 1 --2- 172.21.15.49:0/1878691528 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0b0003980 0x7ff0b0003da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:19.207 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.194+0000 7ff0ca757640 1 -- 172.21.15.49:0/1878691528 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0b0003530 con 0x7ff0b0004d10 2023-11-06T02:13:19.207 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.194+0000 7ff0ca757640 1 --2- 172.21.15.49:0/1878691528 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff0b0004d10 0x7ff0b0005190 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc00a900 tx=0x7ff0bc01d3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77f18d7447374474 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:19.207 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.194+0000 7ff0c8f54640 1 -- 172.21.15.49:0/1878691528 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0bc0ad630 con 0x7ff0b0004d10 2023-11-06T02:13:19.207 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.194+0000 7ff0c8f54640 1 -- 172.21.15.49:0/1878691528 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0bc0a8070 con 0x7ff0b0004d10 2023-11-06T02:13:19.207 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.194+0000 7ff0c8f54640 1 -- 172.21.15.49:0/1878691528 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0bc0ad9d0 con 0x7ff0b0004d10 2023-11-06T02:13:19.208 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.194+0000 7ff0cc1ba640 1 -- 172.21.15.49:0/1878691528 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff0b0004d10 msgr2=0x7ff0b0005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:19.208 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.194+0000 7ff0cc1ba640 1 --2- 172.21.15.49:0/1878691528 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff0b0004d10 0x7ff0b0005190 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc00a900 tx=0x7ff0bc01d3c0 comp rx=0 tx=0).stop 2023-11-06T02:13:19.208 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.194+0000 7ff0ca757640 1 -- 172.21.15.49:0/1878691528 reap_dead start 2023-11-06T02:13:19.208 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.194+0000 7ff0cc1ba640 1 -- 172.21.15.49:0/1878691528 shutdown_connections 2023-11-06T02:13:19.208 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.194+0000 7ff0cc1ba640 1 -- 172.21.15.49:0/1878691528 >> 172.21.15.49:0/1878691528 conn(0x7ff0c406a370 msgr2=0x7ff0c40ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:19.209 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.194+0000 7ff0cc1ba640 1 -- 172.21.15.49:0/1878691528 shutdown_connections 2023-11-06T02:13:19.209 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.194+0000 7ff0cc1ba640 1 -- 172.21.15.49:0/1878691528 wait complete. 2023-11-06T02:13:19.209 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.194+0000 7ff0cc1ba640 1 Processor -- start 2023-11-06T02:13:19.209 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.198+0000 7ff0cc1ba640 1 -- start start 2023-11-06T02:13:19.209 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.198+0000 7ff0cc1ba640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff0c4154330 0x7ff0c4158360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:19.210 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.198+0000 7ff0cc1ba640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff0c40ca3e0 0x7ff0c40ca860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:19.210 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.198+0000 7ff0cc1ba640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0c41f1f20 0x7ff0c41f23d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:19.210 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.198+0000 7ff0cc1ba640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c40c9fc0 con 0x7ff0c4154330 2023-11-06T02:13:19.210 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.198+0000 7ff0cc1ba640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c4158110 con 0x7ff0c41f1f20 2023-11-06T02:13:19.210 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.198+0000 7ff0cc1ba640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c4157b50 con 0x7ff0c40ca3e0 2023-11-06T02:13:19.211 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.198+0000 7ff0c9f56640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff0c4154330 0x7ff0c4158360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:19.211 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.198+0000 7ff0c9f56640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff0c4154330 0x7ff0c4158360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48660/0 (socket says 172.21.15.49:48660) 2023-11-06T02:13:19.211 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.198+0000 7ff0c9f56640 1 -- 172.21.15.49:0/4263698777 learned_addr learned my addr 172.21.15.49:0/4263698777 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:19.211 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.198+0000 7ff0ca757640 1 --2- 172.21.15.49:0/4263698777 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0c41f1f20 0x7ff0c41f23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:19.211 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.198+0000 7ff0c9f56640 1 -- 172.21.15.49:0/4263698777 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff0c40ca3e0 msgr2=0x7ff0c40ca860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:19.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.198+0000 7ff0c9755640 1 --2- 172.21.15.49:0/4263698777 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff0c40ca3e0 0x7ff0c40ca860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:19.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.198+0000 7ff0c9f56640 1 --2- 172.21.15.49:0/4263698777 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff0c40ca3e0 0x7ff0c40ca860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:19.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.198+0000 7ff0c9f56640 1 -- 172.21.15.49:0/4263698777 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0c41f1f20 msgr2=0x7ff0c41f23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:19.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.198+0000 7ff0c9f56640 1 --2- 172.21.15.49:0/4263698777 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0c41f1f20 0x7ff0c41f23d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:19.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.198+0000 7ff0c9f56640 1 -- 172.21.15.49:0/4263698777 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0c415ade0 con 0x7ff0c4154330 2023-11-06T02:13:19.213 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.198+0000 7ff0c9f56640 1 --2- 172.21.15.49:0/4263698777 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff0c4154330 0x7ff0c4158360 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7ff0c0010900 tx=0x7ff0c0010dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:19.213 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.198+0000 7ff0b67fc640 1 -- 172.21.15.49:0/4263698777 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c00096b0 con 0x7ff0c4154330 2023-11-06T02:13:19.213 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.198+0000 7ff0b67fc640 1 -- 172.21.15.49:0/4263698777 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0c000ec20 con 0x7ff0c4154330 2023-11-06T02:13:19.213 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.198+0000 7ff0cc1ba640 1 -- 172.21.15.49:0/4263698777 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0c4157e30 con 0x7ff0c4154330 2023-11-06T02:13:19.213 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.198+0000 7ff0b67fc640 1 -- 172.21.15.49:0/4263698777 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c001c400 con 0x7ff0c4154330 2023-11-06T02:13:19.214 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.198+0000 7ff0cc1ba640 1 -- 172.21.15.49:0/4263698777 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0c4154710 con 0x7ff0c4154330 2023-11-06T02:13:19.214 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.198+0000 7ff0cc1ba640 1 -- 172.21.15.49:0/4263698777 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0c40c9a40 con 0x7ff0c4154330 2023-11-06T02:13:19.214 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.198+0000 7ff0b67fc640 1 -- 172.21.15.49:0/4263698777 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7ff0c001c560 con 0x7ff0c4154330 2023-11-06T02:13:19.214 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.198+0000 7ff0b67fc640 1 --2- 172.21.15.49:0/4263698777 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff0a8065a60 0x7ff0a8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:19.214 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.198+0000 7ff0b67fc640 1 -- 172.21.15.49:0/4263698777 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6350+0+0 (secure 0 0 0) 0x7ff0c0083a80 con 0x7ff0c4154330 2023-11-06T02:13:19.215 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.198+0000 7ff0c9755640 1 --2- 172.21.15.49:0/4263698777 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff0a8065a60 0x7ff0a8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:19.215 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.202+0000 7ff0c9755640 1 --2- 172.21.15.49:0/4263698777 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff0a8065a60 0x7ff0a8067f20 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7ff0c40b2610 tx=0x7ff0b8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:19.215 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.202+0000 7ff0b67fc640 1 -- 172.21.15.49:0/4263698777 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7ff0c40c9a40 con 0x7ff0c4154330 2023-11-06T02:13:19.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.358+0000 7ff0cc1ba640 1 -- 172.21.15.49:0/4263698777 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff0c40c9a40 con 0x7ff0c4154330 2023-11-06T02:13:19.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.362+0000 7ff0b67fc640 1 -- 172.21.15.49:0/4263698777 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v69) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff0c40c9a40 con 0x7ff0c4154330 2023-11-06T02:13:19.358 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 8 2023-11-06T02:13:19.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.362+0000 7ff0cc1ba640 1 -- 172.21.15.49:0/4263698777 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff0a8065a60 msgr2=0x7ff0a8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:19.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.362+0000 7ff0cc1ba640 1 --2- 172.21.15.49:0/4263698777 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff0a8065a60 0x7ff0a8067f20 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7ff0c40b2610 tx=0x7ff0b8008040 comp rx=0 tx=0).stop 2023-11-06T02:13:19.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.362+0000 7ff0cc1ba640 1 -- 172.21.15.49:0/4263698777 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff0c4154330 msgr2=0x7ff0c4158360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:19.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.362+0000 7ff0cc1ba640 1 --2- 172.21.15.49:0/4263698777 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff0c4154330 0x7ff0c4158360 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7ff0c0010900 tx=0x7ff0c0010dc0 comp rx=0 tx=0).stop 2023-11-06T02:13:19.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.362+0000 7ff0cc1ba640 1 -- 172.21.15.49:0/4263698777 shutdown_connections 2023-11-06T02:13:19.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.362+0000 7ff0cc1ba640 1 --2- 172.21.15.49:0/4263698777 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff0a8065a60 0x7ff0a8067f20 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:19.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.362+0000 7ff0cc1ba640 1 --2- 172.21.15.49:0/4263698777 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0c41f1f20 0x7ff0c41f23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:19.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.362+0000 7ff0cc1ba640 1 --2- 172.21.15.49:0/4263698777 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff0c40ca3e0 0x7ff0c40ca860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:19.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.362+0000 7ff0cc1ba640 1 --2- 172.21.15.49:0/4263698777 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff0c4154330 0x7ff0c4158360 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:19.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.362+0000 7ff0cc1ba640 1 -- 172.21.15.49:0/4263698777 >> 172.21.15.49:0/4263698777 conn(0x7ff0c406a370 msgr2=0x7ff0c40b1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:19.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.362+0000 7ff0cc1ba640 1 -- 172.21.15.49:0/4263698777 shutdown_connections 2023-11-06T02:13:19.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.362+0000 7ff0cc1ba640 1 -- 172.21.15.49:0/4263698777 wait complete. 2023-11-06T02:13:19.378 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T02:13:19.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.502+0000 7f0a4ae7c640 1 Processor -- start 2023-11-06T02:13:19.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.502+0000 7f0a4ae7c640 1 -- start start 2023-11-06T02:13:19.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.502+0000 7f0a4ae7c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f0a4415d160 0x7f0a4415e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:19.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.502+0000 7f0a4ae7c640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a4415e100 con 0x7f0a44153d50 2023-11-06T02:13:19.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.502+0000 7f0a4ae7c640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a4415d5f0 con 0x7f0a4415e860 2023-11-06T02:13:19.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.502+0000 7f0a4ae7c640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a4406e0a0 con 0x7f0a4415d160 2023-11-06T02:13:19.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.502+0000 7f0a48c18640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f0a44153d50 0x7f0a440c9100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:46732/0 (socket says 172.21.15.49:46732) 2023-11-06T02:13:19.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.502+0000 7f0a48c18640 1 -- 172.21.15.49:0/605773937 learned_addr learned my addr 172.21.15.49:0/605773937 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:13:19.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.502+0000 7f0a43fff640 1 --2- 172.21.15.49:0/605773937 >> v2:172.21.15.49:3300/0 conn(0x7f0a4415d160 0x7f0a4415e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:19.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.502+0000 7f0a437fe640 1 -- 172.21.15.49:0/605773937 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3862351235 0 0) 0x7f0a4415e100 con 0x7f0a44153d50 2023-11-06T02:13:19.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.502+0000 7f0a437fe640 1 -- 172.21.15.49:0/605773937 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a2c003530 con 0x7f0a44153d50 2023-11-06T02:13:19.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.502+0000 7f0a43fff640 1 -- 172.21.15.49:0/605773937 >> v1:172.21.15.150:6789/0 conn(0x7f0a4415e860 legacy=0x7f0a440707b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:19.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.502+0000 7f0a43fff640 1 -- 172.21.15.49:0/605773937 >> v1:172.21.15.49:6789/0 conn(0x7f0a44153d50 legacy=0x7f0a440c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:19.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.502+0000 7f0a43fff640 1 -- 172.21.15.49:0/605773937 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a4415d860 con 0x7f0a4415d160 2023-11-06T02:13:19.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.506+0000 7f0a43fff640 1 --2- 172.21.15.49:0/605773937 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0a4415d160 0x7f0a4415e320 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f0a380097c0 tx=0x7f0a3801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0a94e82f45eae server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:19.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.506+0000 7f0a437fe640 1 -- 172.21.15.49:0/605773937 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a380ab600 con 0x7f0a4415d160 2023-11-06T02:13:19.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.506+0000 7f0a437fe640 1 -- 172.21.15.49:0/605773937 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a380a7050 con 0x7f0a4415d160 2023-11-06T02:13:19.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.506+0000 7f0a437fe640 1 -- 172.21.15.49:0/605773937 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a380ab980 con 0x7f0a4415d160 2023-11-06T02:13:19.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.506+0000 7f0a4ae7c640 1 -- 172.21.15.49:0/605773937 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0a4415d160 msgr2=0x7f0a4415e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:19.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.506+0000 7f0a4ae7c640 1 --2- 172.21.15.49:0/605773937 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0a4415d160 0x7f0a4415e320 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f0a380097c0 tx=0x7f0a3801c3f0 comp rx=0 tx=0).stop 2023-11-06T02:13:19.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.506+0000 7f0a4ae7c640 1 -- 172.21.15.49:0/605773937 shutdown_connections 2023-11-06T02:13:19.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.506+0000 7f0a4ae7c640 1 --2- 172.21.15.49:0/605773937 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0a4415d160 0x7f0a4415e320 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:19.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.506+0000 7f0a4ae7c640 1 -- 172.21.15.49:0/605773937 >> 172.21.15.49:0/605773937 conn(0x7f0a4406a370 msgr2=0x7f0a440ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:19.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.506+0000 7f0a4ae7c640 1 -- 172.21.15.49:0/605773937 shutdown_connections 2023-11-06T02:13:19.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.506+0000 7f0a4ae7c640 1 -- 172.21.15.49:0/605773937 wait complete. 2023-11-06T02:13:19.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.506+0000 7f0a4ae7c640 1 Processor -- start 2023-11-06T02:13:19.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.506+0000 7f0a4ae7c640 1 -- start start 2023-11-06T02:13:19.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.506+0000 7f0a4ae7c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0a44153d50 0x7f0a440cdbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:19.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.506+0000 7f0a4ae7c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a4415d160 0x7f0a4414bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:19.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.506+0000 7f0a4ae7c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0a4415e860 0x7f0a4415f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:19.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.506+0000 7f0a4ae7c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a44157820 con 0x7f0a44153d50 2023-11-06T02:13:19.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.506+0000 7f0a4ae7c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a44157a70 con 0x7f0a4415d160 2023-11-06T02:13:19.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.506+0000 7f0a4ae7c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a44157d50 con 0x7f0a4415e860 2023-11-06T02:13:19.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.506+0000 7f0a48c18640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0a44153d50 0x7f0a440cdbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:19.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.506+0000 7f0a48c18640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0a44153d50 0x7f0a440cdbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48682/0 (socket says 172.21.15.49:48682) 2023-11-06T02:13:19.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.506+0000 7f0a48c18640 1 -- 172.21.15.49:0/667023091 learned_addr learned my addr 172.21.15.49:0/667023091 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:19.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.506+0000 7f0a43fff640 1 --2- 172.21.15.49:0/667023091 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a4415d160 0x7f0a4414bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:19.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.506+0000 7f0a49419640 1 --2- 172.21.15.49:0/667023091 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0a4415e860 0x7f0a4415f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:19.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.506+0000 7f0a48c18640 1 -- 172.21.15.49:0/667023091 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0a4415e860 msgr2=0x7f0a4415f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:19.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.506+0000 7f0a48c18640 1 --2- 172.21.15.49:0/667023091 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0a4415e860 0x7f0a4415f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:19.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.506+0000 7f0a48c18640 1 -- 172.21.15.49:0/667023091 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a4415d160 msgr2=0x7f0a4414bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:19.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.506+0000 7f0a48c18640 1 --2- 172.21.15.49:0/667023091 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a4415d160 0x7f0a4414bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:19.514 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.506+0000 7f0a48c18640 1 -- 172.21.15.49:0/667023091 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a38009420 con 0x7f0a44153d50 2023-11-06T02:13:19.514 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.506+0000 7f0a43fff640 1 --2- 172.21.15.49:0/667023091 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a4415d160 0x7f0a4414bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:13:19.514 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.506+0000 7f0a48c18640 1 --2- 172.21.15.49:0/667023091 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0a44153d50 0x7f0a440cdbc0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f0a3400c4d0 tx=0x7f0a3400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:19.514 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.506+0000 7f0a417fa640 1 -- 172.21.15.49:0/667023091 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a34005cf0 con 0x7f0a44153d50 2023-11-06T02:13:19.515 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.506+0000 7f0a417fa640 1 -- 172.21.15.49:0/667023091 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a34005e50 con 0x7f0a44153d50 2023-11-06T02:13:19.515 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.506+0000 7f0a4ae7c640 1 -- 172.21.15.49:0/667023091 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a44158030 con 0x7f0a44153d50 2023-11-06T02:13:19.515 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.506+0000 7f0a417fa640 1 -- 172.21.15.49:0/667023091 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a34007270 con 0x7f0a44153d50 2023-11-06T02:13:19.515 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.506+0000 7f0a4ae7c640 1 -- 172.21.15.49:0/667023091 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a44158830 con 0x7f0a44153d50 2023-11-06T02:13:19.515 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.510+0000 7f0a4ae7c640 1 -- 172.21.15.49:0/667023091 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a44074040 con 0x7f0a44153d50 2023-11-06T02:13:19.516 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.510+0000 7f0a417fa640 1 -- 172.21.15.49:0/667023091 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f0a340073d0 con 0x7f0a44153d50 2023-11-06T02:13:19.516 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.510+0000 7f0a417fa640 1 --2- 172.21.15.49:0/667023091 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0a30065a60 0x7f0a30067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:19.516 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.510+0000 7f0a417fa640 1 -- 172.21.15.49:0/667023091 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6350+0+0 (secure 0 0 0) 0x7f0a34082860 con 0x7f0a44153d50 2023-11-06T02:13:19.517 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.510+0000 7f0a43fff640 1 --2- 172.21.15.49:0/667023091 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0a30065a60 0x7f0a30067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:19.517 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.510+0000 7f0a43fff640 1 --2- 172.21.15.49:0/667023091 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0a30065a60 0x7f0a30067f20 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f0a380ad000 tx=0x7f0a38008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:19.517 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.514+0000 7f0a417fa640 1 -- 172.21.15.49:0/667023091 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f0a340500e0 con 0x7f0a44153d50 2023-11-06T02:13:19.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.670+0000 7f0a4ae7c640 1 -- 172.21.15.49:0/667023091 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0a440ca770 con 0x7f0a44153d50 2023-11-06T02:13:19.667 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.670+0000 7f0a417fa640 1 -- 172.21.15.49:0/667023091 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v69) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0a34053700 con 0x7f0a44153d50 2023-11-06T02:13:19.668 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2023-11-06T02:13:19.670 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.674+0000 7f0a4ae7c640 1 -- 172.21.15.49:0/667023091 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0a30065a60 msgr2=0x7f0a30067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:19.670 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.674+0000 7f0a4ae7c640 1 --2- 172.21.15.49:0/667023091 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0a30065a60 0x7f0a30067f20 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f0a380ad000 tx=0x7f0a38008040 comp rx=0 tx=0).stop 2023-11-06T02:13:19.671 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.674+0000 7f0a4ae7c640 1 -- 172.21.15.49:0/667023091 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0a44153d50 msgr2=0x7f0a440cdbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:19.671 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.674+0000 7f0a4ae7c640 1 --2- 172.21.15.49:0/667023091 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0a44153d50 0x7f0a440cdbc0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f0a3400c4d0 tx=0x7f0a3400c990 comp rx=0 tx=0).stop 2023-11-06T02:13:19.673 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.674+0000 7f0a4ae7c640 1 -- 172.21.15.49:0/667023091 shutdown_connections 2023-11-06T02:13:19.673 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.674+0000 7f0a4ae7c640 1 --2- 172.21.15.49:0/667023091 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0a30065a60 0x7f0a30067f20 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:19.673 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.674+0000 7f0a4ae7c640 1 --2- 172.21.15.49:0/667023091 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0a4415e860 0x7f0a4415f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:19.674 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.674+0000 7f0a4ae7c640 1 --2- 172.21.15.49:0/667023091 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a4415d160 0x7f0a4414bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:19.674 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.674+0000 7f0a4ae7c640 1 --2- 172.21.15.49:0/667023091 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0a44153d50 0x7f0a440cdbc0 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:19.674 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.674+0000 7f0a4ae7c640 1 -- 172.21.15.49:0/667023091 >> 172.21.15.49:0/667023091 conn(0x7f0a4406a370 msgr2=0x7f0a4414ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:19.675 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.674+0000 7f0a4ae7c640 1 -- 172.21.15.49:0/667023091 shutdown_connections 2023-11-06T02:13:19.675 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.674+0000 7f0a4ae7c640 1 -- 172.21.15.49:0/667023091 wait complete. 2023-11-06T02:13:19.685 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T02:13:19.815 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.818+0000 7f4142fdb640 1 Processor -- start 2023-11-06T02:13:19.815 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.818+0000 7f4142fdb640 1 -- start start 2023-11-06T02:13:19.818 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.818+0000 7f4142fdb640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f413c15a9b0 0x7f413c0cde40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:19.818 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.818+0000 7f4142fdb640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f413c0cd2a0 0x7f413c0cd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:19.819 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.818+0000 7f4142fdb640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f413c0b3720 con 0x7f413c15ad90 2023-11-06T02:13:19.819 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.818+0000 7f4142fdb640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f413c0cdc70 con 0x7f413c0cd2a0 2023-11-06T02:13:19.819 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.818+0000 7f4142fdb640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f413c0cf690 con 0x7f413c15a9b0 2023-11-06T02:13:19.819 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.818+0000 7f413bfff640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f413c15ad90 0x7f413c0cee80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:46742/0 (socket says 172.21.15.49:46742) 2023-11-06T02:13:19.820 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.818+0000 7f413bfff640 1 -- 172.21.15.49:0/383734501 learned_addr learned my addr 172.21.15.49:0/383734501 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:13:19.820 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.818+0000 7f4141578640 1 --2- 172.21.15.49:0/383734501 >> v2:172.21.15.150:3300/0 conn(0x7f413c0cd2a0 0x7f413c0cd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:19.820 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.818+0000 7f4140d77640 1 --2- 172.21.15.49:0/383734501 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f413c15a9b0 0x7f413c0cde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:19.820 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.818+0000 7f413b7fe640 1 -- 172.21.15.49:0/383734501 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 614539218 0 0) 0x7f413c0b3720 con 0x7f413c15ad90 2023-11-06T02:13:19.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.818+0000 7f413b7fe640 1 -- 172.21.15.49:0/383734501 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4128003530 con 0x7f413c15ad90 2023-11-06T02:13:19.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.818+0000 7f413b7fe640 1 -- 172.21.15.49:0/383734501 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 708515461 0 0) 0x7f4128003530 con 0x7f413c15ad90 2023-11-06T02:13:19.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.818+0000 7f413b7fe640 1 -- 172.21.15.49:0/383734501 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f413c15a9b0 msgr2=0x7f413c0cde40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:19.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.818+0000 7f413b7fe640 1 --2- 172.21.15.49:0/383734501 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f413c15a9b0 0x7f413c0cde40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:19.822 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.818+0000 7f413b7fe640 1 -- 172.21.15.49:0/383734501 >> v2:172.21.15.150:3300/0 conn(0x7f413c0cd2a0 msgr2=0x7f413c0cd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:19.822 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.818+0000 7f413b7fe640 1 --2- 172.21.15.49:0/383734501 >> v2:172.21.15.150:3300/0 conn(0x7f413c0cd2a0 0x7f413c0cd730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:19.822 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.818+0000 7f413b7fe640 1 -- 172.21.15.49:0/383734501 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f413c0b3970 con 0x7f413c15ad90 2023-11-06T02:13:19.822 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.818+0000 7f413b7fe640 1 -- 172.21.15.49:0/383734501 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7f4124001a00 con 0x7f413c15ad90 2023-11-06T02:13:19.823 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.818+0000 7f413b7fe640 1 -- 172.21.15.49:0/383734501 >> v1:172.21.15.49:6789/0 conn(0x7f413c15ad90 legacy=0x7f413c0cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:19.823 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.818+0000 7f413b7fe640 1 --2- 172.21.15.49:0/383734501 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4128003980 0x7f4128003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:19.823 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.818+0000 7f413b7fe640 1 --2- 172.21.15.49:0/383734501 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41280047d0 0x7f4128004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:19.823 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.818+0000 7f413b7fe640 1 --2- 172.21.15.49:0/383734501 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f413c15a9b0 0x7f4128005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:19.823 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.818+0000 7f413b7fe640 1 -- 172.21.15.49:0/383734501 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f413c0cdc70 con 0x7f41280047d0 2023-11-06T02:13:19.824 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.818+0000 7f413b7fe640 1 -- 172.21.15.49:0/383734501 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f413c0cf690 con 0x7f4128003980 2023-11-06T02:13:19.824 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.818+0000 7f413b7fe640 1 -- 172.21.15.49:0/383734501 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f412800b990 con 0x7f413c15a9b0 2023-11-06T02:13:19.824 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.818+0000 7f413bfff640 1 --2- 172.21.15.49:0/383734501 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41280047d0 0x7f4128004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:19.825 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.822+0000 7f4140d77640 1 --2- 172.21.15.49:0/383734501 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4128003980 0x7f4128003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:19.825 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.822+0000 7f4141578640 1 --2- 172.21.15.49:0/383734501 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f413c15a9b0 0x7f4128005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:19.825 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.822+0000 7f413bfff640 1 -- 172.21.15.49:0/383734501 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f413c15a9b0 msgr2=0x7f4128005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:19.825 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.822+0000 7f413bfff640 1 --2- 172.21.15.49:0/383734501 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f413c15a9b0 0x7f4128005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:19.826 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.822+0000 7f413bfff640 1 -- 172.21.15.49:0/383734501 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4128003980 msgr2=0x7f4128003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:19.826 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.822+0000 7f413bfff640 1 --2- 172.21.15.49:0/383734501 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4128003980 0x7f4128003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:19.826 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.822+0000 7f413bfff640 1 -- 172.21.15.49:0/383734501 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4128003530 con 0x7f41280047d0 2023-11-06T02:13:19.831 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.822+0000 7f413bfff640 1 --2- 172.21.15.49:0/383734501 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41280047d0 0x7f4128004c50 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f412400a8d0 tx=0x7f412401fb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ee93c8ee96961c1 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:19.831 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.822+0000 7f413b7fe640 1 -- 172.21.15.49:0/383734501 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41240ac660 con 0x7f41280047d0 2023-11-06T02:13:19.831 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.822+0000 7f413b7fe640 1 -- 172.21.15.49:0/383734501 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41240ac7c0 con 0x7f41280047d0 2023-11-06T02:13:19.832 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.822+0000 7f413b7fe640 1 -- 172.21.15.49:0/383734501 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41240acb70 con 0x7f41280047d0 2023-11-06T02:13:19.832 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.822+0000 7f4142fdb640 1 -- 172.21.15.49:0/383734501 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41280047d0 msgr2=0x7f4128004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:19.832 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.822+0000 7f4142fdb640 1 --2- 172.21.15.49:0/383734501 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41280047d0 0x7f4128004c50 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f412400a8d0 tx=0x7f412401fb40 comp rx=0 tx=0).stop 2023-11-06T02:13:19.832 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.822+0000 7f4141578640 1 -- 172.21.15.49:0/383734501 reap_dead start 2023-11-06T02:13:19.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.822+0000 7f4142fdb640 1 -- 172.21.15.49:0/383734501 shutdown_connections 2023-11-06T02:13:19.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.822+0000 7f4142fdb640 1 -- 172.21.15.49:0/383734501 >> 172.21.15.49:0/383734501 conn(0x7f413c0d08d0 msgr2=0x7f413c0d0cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:19.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.822+0000 7f4142fdb640 1 -- 172.21.15.49:0/383734501 shutdown_connections 2023-11-06T02:13:19.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.822+0000 7f4142fdb640 1 -- 172.21.15.49:0/383734501 wait complete. 2023-11-06T02:13:19.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.826+0000 7f4142fdb640 1 Processor -- start 2023-11-06T02:13:19.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.826+0000 7f4142fdb640 1 -- start start 2023-11-06T02:13:19.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.826+0000 7f4142fdb640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41280047d0 0x7f413c14e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:19.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.826+0000 7f4142fdb640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f413c1e3fb0 0x7f413c0c79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:19.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.826+0000 7f4142fdb640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f413c1e4430 0x7f413c0c7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:19.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.826+0000 7f4142fdb640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f413c15b6c0 con 0x7f41280047d0 2023-11-06T02:13:19.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.826+0000 7f4142fdb640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f413c0b6460 con 0x7f413c1e3fb0 2023-11-06T02:13:19.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.826+0000 7f4142fdb640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f413c0b58e0 con 0x7f413c1e4430 2023-11-06T02:13:19.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.826+0000 7f4140d77640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41280047d0 0x7f413c14e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:19.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.826+0000 7f4140d77640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41280047d0 0x7f413c14e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48696/0 (socket says 172.21.15.49:48696) 2023-11-06T02:13:19.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.826+0000 7f4140d77640 1 -- 172.21.15.49:0/998595790 learned_addr learned my addr 172.21.15.49:0/998595790 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:19.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.826+0000 7f413bfff640 1 --2- 172.21.15.49:0/998595790 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f413c1e3fb0 0x7f413c0c79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:19.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.826+0000 7f4141578640 1 --2- 172.21.15.49:0/998595790 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f413c1e4430 0x7f413c0c7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:19.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.826+0000 7f4140d77640 1 -- 172.21.15.49:0/998595790 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f413c1e4430 msgr2=0x7f413c0c7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:19.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.826+0000 7f4140d77640 1 --2- 172.21.15.49:0/998595790 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f413c1e4430 0x7f413c0c7ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:19.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.826+0000 7f4140d77640 1 -- 172.21.15.49:0/998595790 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f413c1e3fb0 msgr2=0x7f413c0c79a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:19.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.826+0000 7f4140d77640 1 --2- 172.21.15.49:0/998595790 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f413c1e3fb0 0x7f413c0c79a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:19.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.826+0000 7f4140d77640 1 -- 172.21.15.49:0/998595790 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f413c0b5bc0 con 0x7f41280047d0 2023-11-06T02:13:19.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.826+0000 7f4140d77640 1 --2- 172.21.15.49:0/998595790 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41280047d0 0x7f413c14e050 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f412c0115c0 tx=0x7f412c011a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:19.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.826+0000 7f41397fa640 1 -- 172.21.15.49:0/998595790 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f412c00f8f0 con 0x7f41280047d0 2023-11-06T02:13:19.839 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.826+0000 7f41397fa640 1 -- 172.21.15.49:0/998595790 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f412c00fa50 con 0x7f41280047d0 2023-11-06T02:13:19.839 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.826+0000 7f4142fdb640 1 -- 172.21.15.49:0/998595790 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f413c0b5ea0 con 0x7f41280047d0 2023-11-06T02:13:19.839 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.826+0000 7f41397fa640 1 -- 172.21.15.49:0/998595790 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f412c00fdc0 con 0x7f41280047d0 2023-11-06T02:13:19.839 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.826+0000 7f4142fdb640 1 -- 172.21.15.49:0/998595790 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f413c14e590 con 0x7f41280047d0 2023-11-06T02:13:19.839 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.826+0000 7f4142fdb640 1 -- 172.21.15.49:0/998595790 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f413c074040 con 0x7f41280047d0 2023-11-06T02:13:19.840 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.826+0000 7f41397fa640 1 -- 172.21.15.49:0/998595790 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f412c015460 con 0x7f41280047d0 2023-11-06T02:13:19.840 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.830+0000 7f41397fa640 1 --2- 172.21.15.49:0/998595790 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f411c065a60 0x7f411c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:19.840 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.830+0000 7f41397fa640 1 -- 172.21.15.49:0/998595790 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6350+0+0 (secure 0 0 0) 0x7f412c083b60 con 0x7f41280047d0 2023-11-06T02:13:19.840 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.830+0000 7f413bfff640 1 --2- 172.21.15.49:0/998595790 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f411c065a60 0x7f411c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:19.841 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.830+0000 7f413bfff640 1 --2- 172.21.15.49:0/998595790 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f411c065a60 0x7f411c067f20 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f41240b0000 tx=0x7f4124001d90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:19.841 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.834+0000 7f41397fa640 1 -- 172.21.15.49:0/998595790 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f412c0512a0 con 0x7f41280047d0 2023-11-06T02:13:19.985 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.990+0000 7f4142fdb640 1 -- 172.21.15.49:0/998595790 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f413c0cb540 con 0x7f41280047d0 2023-11-06T02:13:19.985 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.990+0000 7f41397fa640 1 -- 172.21.15.49:0/998595790 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v69) v1 ==== 100+0+11 (secure 0 0 0) 0x7f412c0548c0 con 0x7f41280047d0 2023-11-06T02:13:19.986 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2023-11-06T02:13:19.988 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.990+0000 7f4142fdb640 1 -- 172.21.15.49:0/998595790 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f411c065a60 msgr2=0x7f411c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:19.988 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.990+0000 7f4142fdb640 1 --2- 172.21.15.49:0/998595790 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f411c065a60 0x7f411c067f20 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f41240b0000 tx=0x7f4124001d90 comp rx=0 tx=0).stop 2023-11-06T02:13:19.989 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.990+0000 7f4142fdb640 1 -- 172.21.15.49:0/998595790 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41280047d0 msgr2=0x7f413c14e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:19.989 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.990+0000 7f4142fdb640 1 --2- 172.21.15.49:0/998595790 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41280047d0 0x7f413c14e050 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f412c0115c0 tx=0x7f412c011a80 comp rx=0 tx=0).stop 2023-11-06T02:13:19.989 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.990+0000 7f4142fdb640 1 -- 172.21.15.49:0/998595790 shutdown_connections 2023-11-06T02:13:19.989 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.990+0000 7f4142fdb640 1 --2- 172.21.15.49:0/998595790 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f411c065a60 0x7f411c067f20 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:19.990 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.990+0000 7f4142fdb640 1 --2- 172.21.15.49:0/998595790 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f413c1e4430 0x7f413c0c7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:19.990 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.990+0000 7f4142fdb640 1 --2- 172.21.15.49:0/998595790 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f413c1e3fb0 0x7f413c0c79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:19.990 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.990+0000 7f4142fdb640 1 --2- 172.21.15.49:0/998595790 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41280047d0 0x7f413c14e050 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:19.990 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.990+0000 7f4142fdb640 1 -- 172.21.15.49:0/998595790 >> 172.21.15.49:0/998595790 conn(0x7f413c0d08d0 msgr2=0x7f413c0ce1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:19.991 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.990+0000 7f4142fdb640 1 -- 172.21.15.49:0/998595790 shutdown_connections 2023-11-06T02:13:19.991 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:19.990+0000 7f4142fdb640 1 -- 172.21.15.49:0/998595790 wait complete. 2023-11-06T02:13:20.007 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T02:13:20.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.138+0000 7faa07e1c640 1 Processor -- start 2023-11-06T02:13:20.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.138+0000 7faa07e1c640 1 -- start start 2023-11-06T02:13:20.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.138+0000 7faa07e1c640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7faa000b0790 0x7faa000c8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:20.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.138+0000 7faa07e1c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa000adae0 0x7faa00070b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:20.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.138+0000 7faa07e1c640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa001594d0 con 0x7faa001590f0 2023-11-06T02:13:20.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.138+0000 7faa07e1c640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa0015a140 con 0x7faa000b0790 2023-11-06T02:13:20.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.138+0000 7faa07e1c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa000c92e0 con 0x7faa000adae0 2023-11-06T02:13:20.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.138+0000 7faa05bb8640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7faa001590f0 0x7faa000b0080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:46750/0 (socket says 172.21.15.49:46750) 2023-11-06T02:13:20.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.138+0000 7faa05bb8640 1 -- 172.21.15.49:0/1308942167 learned_addr learned my addr 172.21.15.49:0/1308942167 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:13:20.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.138+0000 7faa053b7640 1 --2- 172.21.15.49:0/1308942167 >> v2:172.21.15.150:3300/0 conn(0x7faa000b0790 0x7faa000c8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:20.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.138+0000 7faa063b9640 1 --2- 172.21.15.49:0/1308942167 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa000adae0 0x7faa00070b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:20.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.138+0000 7faa04bb6640 1 -- 172.21.15.49:0/1308942167 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2032270994 0 0) 0x7faa001594d0 con 0x7faa001590f0 2023-11-06T02:13:20.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.138+0000 7faa04bb6640 1 -- 172.21.15.49:0/1308942167 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9ec003530 con 0x7faa001590f0 2023-11-06T02:13:20.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.138+0000 7faa04bb6640 1 -- 172.21.15.49:0/1308942167 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1661030205 0 0) 0x7fa9ec003530 con 0x7faa001590f0 2023-11-06T02:13:20.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.138+0000 7faa04bb6640 1 -- 172.21.15.49:0/1308942167 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa000adae0 msgr2=0x7faa00070b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:20.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.138+0000 7faa04bb6640 1 --2- 172.21.15.49:0/1308942167 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa000adae0 0x7faa00070b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:20.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.138+0000 7faa04bb6640 1 -- 172.21.15.49:0/1308942167 >> v2:172.21.15.150:3300/0 conn(0x7faa000b0790 msgr2=0x7faa000c8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:20.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.138+0000 7faa04bb6640 1 --2- 172.21.15.49:0/1308942167 >> v2:172.21.15.150:3300/0 conn(0x7faa000b0790 0x7faa000c8c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:20.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.138+0000 7faa04bb6640 1 -- 172.21.15.49:0/1308942167 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa001597a0 con 0x7faa001590f0 2023-11-06T02:13:20.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.138+0000 7faa04bb6640 1 -- 172.21.15.49:0/1308942167 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7fa9fc001800 con 0x7faa001590f0 2023-11-06T02:13:20.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.138+0000 7faa04bb6640 1 -- 172.21.15.49:0/1308942167 >> v1:172.21.15.49:6789/0 conn(0x7faa001590f0 legacy=0x7faa000b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:20.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.138+0000 7faa04bb6640 1 --2- 172.21.15.49:0/1308942167 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9ec003980 0x7fa9ec003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:20.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.138+0000 7faa04bb6640 1 --2- 172.21.15.49:0/1308942167 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa9ec0047d0 0x7fa9ec004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:20.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.138+0000 7faa04bb6640 1 --2- 172.21.15.49:0/1308942167 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa000adae0 0x7fa9ec005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:20.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.138+0000 7faa04bb6640 1 -- 172.21.15.49:0/1308942167 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa0015a140 con 0x7fa9ec003980 2023-11-06T02:13:20.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.138+0000 7faa04bb6640 1 -- 172.21.15.49:0/1308942167 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9ec00b990 con 0x7fa9ec0047d0 2023-11-06T02:13:20.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.138+0000 7faa04bb6640 1 -- 172.21.15.49:0/1308942167 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9ec00bad0 con 0x7faa000adae0 2023-11-06T02:13:20.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.138+0000 7faa05bb8640 1 --2- 172.21.15.49:0/1308942167 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9ec003980 0x7fa9ec003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:20.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.138+0000 7faa053b7640 1 --2- 172.21.15.49:0/1308942167 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa9ec0047d0 0x7fa9ec004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:20.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.138+0000 7faa05bb8640 1 -- 172.21.15.49:0/1308942167 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa000adae0 msgr2=0x7fa9ec005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:20.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.138+0000 7faa05bb8640 1 --2- 172.21.15.49:0/1308942167 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa000adae0 0x7fa9ec005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:20.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.138+0000 7faa05bb8640 1 -- 172.21.15.49:0/1308942167 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa9ec0047d0 msgr2=0x7fa9ec004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:20.143 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.138+0000 7faa05bb8640 1 --2- 172.21.15.49:0/1308942167 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa9ec0047d0 0x7fa9ec004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:20.143 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.138+0000 7faa05bb8640 1 -- 172.21.15.49:0/1308942167 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9ec003530 con 0x7fa9ec003980 2023-11-06T02:13:20.143 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.138+0000 7faa053b7640 1 --2- 172.21.15.49:0/1308942167 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa9ec0047d0 0x7fa9ec004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:13:20.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.142+0000 7faa05bb8640 1 --2- 172.21.15.49:0/1308942167 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9ec003980 0x7fa9ec003d60 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc003f10 tx=0x7fa9fc01d2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b175c2aa8ebb768 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:20.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.142+0000 7faa04bb6640 1 -- 172.21.15.49:0/1308942167 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9fc0a5e20 con 0x7fa9ec003980 2023-11-06T02:13:20.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.142+0000 7faa04bb6640 1 -- 172.21.15.49:0/1308942167 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9fc0a9070 con 0x7fa9ec003980 2023-11-06T02:13:20.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.142+0000 7faa04bb6640 1 -- 172.21.15.49:0/1308942167 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9fc0083b0 con 0x7fa9ec003980 2023-11-06T02:13:20.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.142+0000 7faa07e1c640 1 -- 172.21.15.49:0/1308942167 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9ec003980 msgr2=0x7fa9ec003d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:20.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.142+0000 7faa07e1c640 1 --2- 172.21.15.49:0/1308942167 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9ec003980 0x7fa9ec003d60 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc003f10 tx=0x7fa9fc01d2a0 comp rx=0 tx=0).stop 2023-11-06T02:13:20.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.142+0000 7faa063b9640 1 -- 172.21.15.49:0/1308942167 reap_dead start 2023-11-06T02:13:20.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.142+0000 7faa07e1c640 1 -- 172.21.15.49:0/1308942167 shutdown_connections 2023-11-06T02:13:20.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.142+0000 7faa07e1c640 1 -- 172.21.15.49:0/1308942167 >> 172.21.15.49:0/1308942167 conn(0x7faa0006c0a0 msgr2=0x7faa0006c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:20.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.142+0000 7faa07e1c640 1 -- 172.21.15.49:0/1308942167 shutdown_connections 2023-11-06T02:13:20.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.142+0000 7faa07e1c640 1 -- 172.21.15.49:0/1308942167 wait complete. 2023-11-06T02:13:20.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.142+0000 7faa07e1c640 1 Processor -- start 2023-11-06T02:13:20.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.142+0000 7faa07e1c640 1 -- start start 2023-11-06T02:13:20.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.146+0000 7faa07e1c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9ec003980 0x7faa000ac750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:20.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.146+0000 7faa07e1c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa0014f570 0x7faa0014d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:20.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.146+0000 7faa07e1c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa0014eb20 0x7faa0014efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:20.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.146+0000 7faa07e1c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa000acc90 con 0x7fa9ec003980 2023-11-06T02:13:20.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.146+0000 7faa07e1c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa00150310 con 0x7faa0014eb20 2023-11-06T02:13:20.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.146+0000 7faa07e1c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa000ac430 con 0x7faa0014f570 2023-11-06T02:13:20.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.146+0000 7faa05bb8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9ec003980 0x7faa000ac750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:20.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.146+0000 7faa05bb8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9ec003980 0x7faa000ac750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48708/0 (socket says 172.21.15.49:48708) 2023-11-06T02:13:20.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.146+0000 7faa05bb8640 1 -- 172.21.15.49:0/3835721322 learned_addr learned my addr 172.21.15.49:0/3835721322 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:20.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.146+0000 7faa053b7640 1 --2- 172.21.15.49:0/3835721322 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa0014f570 0x7faa0014d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:20.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.146+0000 7faa05bb8640 1 -- 172.21.15.49:0/3835721322 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa0014f570 msgr2=0x7faa0014d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:20.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.146+0000 7faa05bb8640 1 --2- 172.21.15.49:0/3835721322 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa0014f570 0x7faa0014d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:20.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.146+0000 7faa05bb8640 1 -- 172.21.15.49:0/3835721322 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa0014eb20 msgr2=0x7faa0014efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:20.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.146+0000 7faa05bb8640 1 --2- 172.21.15.49:0/3835721322 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa0014eb20 0x7faa0014efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:20.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.146+0000 7faa05bb8640 1 -- 172.21.15.49:0/3835721322 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa0014fa70 con 0x7fa9ec003980 2023-11-06T02:13:20.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.146+0000 7faa05bb8640 1 --2- 172.21.15.49:0/3835721322 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9ec003980 0x7faa000ac750 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc01cd10 tx=0x7fa9ec005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:20.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.146+0000 7fa9f67fc640 1 -- 172.21.15.49:0/3835721322 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9fc003600 con 0x7fa9ec003980 2023-11-06T02:13:20.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.146+0000 7fa9f67fc640 1 -- 172.21.15.49:0/3835721322 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9fc0037a0 con 0x7fa9ec003980 2023-11-06T02:13:20.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.146+0000 7fa9f67fc640 1 -- 172.21.15.49:0/3835721322 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9fc001800 con 0x7fa9ec003980 2023-11-06T02:13:20.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.146+0000 7faa07e1c640 1 -- 172.21.15.49:0/3835721322 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa0014fd50 con 0x7fa9ec003980 2023-11-06T02:13:20.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.146+0000 7faa07e1c640 1 -- 172.21.15.49:0/3835721322 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa0014e600 con 0x7fa9ec003980 2023-11-06T02:13:20.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.146+0000 7faa07e1c640 1 -- 172.21.15.49:0/3835721322 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa000cbe50 con 0x7fa9ec003980 2023-11-06T02:13:20.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.146+0000 7fa9f67fc640 1 -- 172.21.15.49:0/3835721322 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fa9fc0ab070 con 0x7fa9ec003980 2023-11-06T02:13:20.156 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.146+0000 7fa9f67fc640 1 --2- 172.21.15.49:0/3835721322 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa9e4065a80 0x7fa9e4067f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:20.156 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.146+0000 7fa9f67fc640 1 -- 172.21.15.49:0/3835721322 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6350+0+0 (secure 0 0 0) 0x7fa9fc11c690 con 0x7fa9ec003980 2023-11-06T02:13:20.157 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.146+0000 7faa053b7640 1 --2- 172.21.15.49:0/3835721322 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa9e4065a80 0x7fa9e4067f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:20.157 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.150+0000 7faa053b7640 1 --2- 172.21.15.49:0/3835721322 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa9e4065a80 0x7fa9e4067f40 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7faa001567a0 tx=0x7fa9f0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:20.158 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.150+0000 7fa9f67fc640 1 -- 172.21.15.49:0/3835721322 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fa9fc0e9e60 con 0x7fa9ec003980 2023-11-06T02:13:20.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.338+0000 7faa07e1c640 1 -- 172.21.15.49:0/3835721322 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faa000acc90 con 0x7fa9ec003980 2023-11-06T02:13:20.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.342+0000 7fa9f67fc640 1 -- 172.21.15.49:0/3835721322 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4645 (secure 0 0 0) 0x7fa9fc0ed480 con 0x7fa9ec003980 2023-11-06T02:13:20.337 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:13:20.338 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":38,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5635,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4170291905","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":4170291905},{"type":"v1","addr":"172.21.15.150:6835","nonce":4170291905}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5638,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/2719458937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":2719458937},{"type":"v1","addr":"172.21.15.150:6837","nonce":2719458937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":6339,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/2310338375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2310338375},{"type":"v1","addr":"172.21.15.49:6833","nonce":2310338375}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":6960,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/322306580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":322306580},{"type":"v1","addr":"172.21.15.49:6835","nonce":322306580}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":37,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T02:11:30.091070+0000","modified":"2023-11-06T02:13:17.615097+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-06T02:13:20.338 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 38 2023-11-06T02:13:20.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.342+0000 7faa07e1c640 1 -- 172.21.15.49:0/3835721322 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa9e4065a80 msgr2=0x7fa9e4067f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:20.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.342+0000 7faa07e1c640 1 --2- 172.21.15.49:0/3835721322 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa9e4065a80 0x7fa9e4067f40 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7faa001567a0 tx=0x7fa9f0008040 comp rx=0 tx=0).stop 2023-11-06T02:13:20.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.342+0000 7faa07e1c640 1 -- 172.21.15.49:0/3835721322 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9ec003980 msgr2=0x7faa000ac750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:20.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.342+0000 7faa07e1c640 1 --2- 172.21.15.49:0/3835721322 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9ec003980 0x7faa000ac750 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc01cd10 tx=0x7fa9ec005800 comp rx=0 tx=0).stop 2023-11-06T02:13:20.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.342+0000 7faa07e1c640 1 -- 172.21.15.49:0/3835721322 shutdown_connections 2023-11-06T02:13:20.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.342+0000 7faa07e1c640 1 --2- 172.21.15.49:0/3835721322 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa9e4065a80 0x7fa9e4067f40 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:20.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.342+0000 7faa07e1c640 1 --2- 172.21.15.49:0/3835721322 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa0014eb20 0x7faa0014efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:20.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.342+0000 7faa07e1c640 1 --2- 172.21.15.49:0/3835721322 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa0014f570 0x7faa0014d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:20.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.342+0000 7faa07e1c640 1 --2- 172.21.15.49:0/3835721322 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9ec003980 0x7faa000ac750 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:20.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.342+0000 7faa07e1c640 1 -- 172.21.15.49:0/3835721322 >> 172.21.15.49:0/3835721322 conn(0x7faa0006c0a0 msgr2=0x7faa00151090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:20.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.342+0000 7faa07e1c640 1 -- 172.21.15.49:0/3835721322 shutdown_connections 2023-11-06T02:13:20.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.342+0000 7faa07e1c640 1 -- 172.21.15.49:0/3835721322 wait complete. 2023-11-06T02:13:20.358 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T02:13:20.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.478+0000 7f2fcafd6640 1 Processor -- start 2023-11-06T02:13:20.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.478+0000 7f2fcafd6640 1 -- start start 2023-11-06T02:13:20.478 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.482+0000 7f2fcafd6640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f2fc40b0790 0x7f2fc40c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:20.478 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.482+0000 7f2fcafd6640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f2fc40cb010 0x7f2fc40c2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:20.478 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.482+0000 7f2fcafd6640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fc40cab80 con 0x7f2fc415ab50 2023-11-06T02:13:20.479 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.482+0000 7f2fcafd6640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fc415af30 con 0x7f2fc40cb010 2023-11-06T02:13:20.479 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.482+0000 7f2fcafd6640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fc40cb5b0 con 0x7f2fc40b0790 2023-11-06T02:13:20.479 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.482+0000 7f2fc9573640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f2fc40cb010 0x7f2fc40c2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:20.479 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.482+0000 7f2fc8d72640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f2fc415ab50 0x7f2fc40b0080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:46754/0 (socket says 172.21.15.49:46754) 2023-11-06T02:13:20.479 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.482+0000 7f2fc8d72640 1 -- 172.21.15.49:0/3141060495 learned_addr learned my addr 172.21.15.49:0/3141060495 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:13:20.479 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.482+0000 7f2fc3fff640 1 --2- 172.21.15.49:0/3141060495 >> v2:172.21.15.150:3300/0 conn(0x7f2fc40b0790 0x7f2fc40c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:20.480 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.482+0000 7f2fc9573640 1 -- 172.21.15.49:0/3141060495 >> v2:172.21.15.150:3300/0 conn(0x7f2fc40b0790 msgr2=0x7f2fc40c9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:20.480 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.482+0000 7f2fc9573640 1 --2- 172.21.15.49:0/3141060495 >> v2:172.21.15.150:3300/0 conn(0x7f2fc40b0790 0x7f2fc40c9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:20.480 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.482+0000 7f2fc9573640 1 -- 172.21.15.49:0/3141060495 >> v1:172.21.15.49:6789/0 conn(0x7f2fc415ab50 legacy=0x7f2fc40b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:20.480 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.482+0000 7f2fc9573640 1 -- 172.21.15.49:0/3141060495 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fc40cb890 con 0x7f2fc40cb010 2023-11-06T02:13:20.481 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.482+0000 7f2fc37fe640 1 -- 172.21.15.49:0/3141060495 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2900938105 0 0) 0x7f2fc40cab80 con 0x7f2fc415ab50 2023-11-06T02:13:20.481 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.482+0000 7f2fc3fff640 1 --2- 172.21.15.49:0/3141060495 >> v2:172.21.15.150:3300/0 conn(0x7f2fc40b0790 0x7f2fc40c9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:13:20.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.482+0000 7f2fc9573640 1 --2- 172.21.15.49:0/3141060495 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2fc40cb010 0x7f2fc40c2710 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f2fc406dde0 tx=0x7f2fbc01c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0f2d093c6ee776b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:20.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.482+0000 7f2fc37fe640 1 -- 172.21.15.49:0/3141060495 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fbc0a59a0 con 0x7f2fc40cb010 2023-11-06T02:13:20.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.482+0000 7f2fc37fe640 1 -- 172.21.15.49:0/3141060495 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fbc0a5b00 con 0x7f2fc40cb010 2023-11-06T02:13:20.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.482+0000 7f2fc37fe640 1 -- 172.21.15.49:0/3141060495 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fbc0a5e90 con 0x7f2fc40cb010 2023-11-06T02:13:20.485 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.486+0000 7f2fcafd6640 1 -- 172.21.15.49:0/3141060495 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2fc40cb010 msgr2=0x7f2fc40c2710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:20.485 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.486+0000 7f2fcafd6640 1 --2- 172.21.15.49:0/3141060495 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2fc40cb010 0x7f2fc40c2710 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f2fc406dde0 tx=0x7f2fbc01c8e0 comp rx=0 tx=0).stop 2023-11-06T02:13:20.485 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.486+0000 7f2fcafd6640 1 -- 172.21.15.49:0/3141060495 shutdown_connections 2023-11-06T02:13:20.485 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.486+0000 7f2fcafd6640 1 --2- 172.21.15.49:0/3141060495 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2fc40cb010 0x7f2fc40c2710 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:20.485 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.486+0000 7f2fcafd6640 1 --2- 172.21.15.49:0/3141060495 >> v2:172.21.15.150:3300/0 conn(0x7f2fc40b0790 0x7f2fc40c9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:20.486 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.486+0000 7f2fcafd6640 1 -- 172.21.15.49:0/3141060495 >> 172.21.15.49:0/3141060495 conn(0x7f2fc406c0a0 msgr2=0x7f2fc406c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:20.486 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.486+0000 7f2fcafd6640 1 -- 172.21.15.49:0/3141060495 shutdown_connections 2023-11-06T02:13:20.486 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.486+0000 7f2fcafd6640 1 -- 172.21.15.49:0/3141060495 wait complete. 2023-11-06T02:13:20.486 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.486+0000 7f2fcafd6640 1 Processor -- start 2023-11-06T02:13:20.486 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.486+0000 7f2fcafd6640 1 -- start start 2023-11-06T02:13:20.487 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.486+0000 7f2fcafd6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2fc40b0790 0x7f2fc4147c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:20.487 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.486+0000 7f2fcafd6640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2fc40cb010 0x7f2fc40af860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:20.487 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.486+0000 7f2fcafd6640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2fc415ab50 0x7f2fc40c3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:20.487 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.486+0000 7f2fcafd6640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc4148180 con 0x7f2fc40b0790 2023-11-06T02:13:20.487 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.486+0000 7f2fcafd6640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc40c5e00 con 0x7f2fc415ab50 2023-11-06T02:13:20.488 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.486+0000 7f2fcafd6640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc40ae540 con 0x7f2fc40cb010 2023-11-06T02:13:20.488 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.486+0000 7f2fc8d72640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2fc40b0790 0x7f2fc4147c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:20.488 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.486+0000 7f2fc8d72640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2fc40b0790 0x7f2fc4147c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48714/0 (socket says 172.21.15.49:48714) 2023-11-06T02:13:20.488 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.486+0000 7f2fc8d72640 1 -- 172.21.15.49:0/52582698 learned_addr learned my addr 172.21.15.49:0/52582698 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:20.488 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.486+0000 7f2fc3fff640 1 --2- 172.21.15.49:0/52582698 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2fc40cb010 0x7f2fc40af860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:20.489 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.486+0000 7f2fc8d72640 1 -- 172.21.15.49:0/52582698 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2fc40cb010 msgr2=0x7f2fc40af860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:20.489 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.486+0000 7f2fc9573640 1 --2- 172.21.15.49:0/52582698 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2fc415ab50 0x7f2fc40c3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:20.489 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.486+0000 7f2fc8d72640 1 --2- 172.21.15.49:0/52582698 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2fc40cb010 0x7f2fc40af860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:20.489 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.486+0000 7f2fc8d72640 1 -- 172.21.15.49:0/52582698 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2fc415ab50 msgr2=0x7f2fc40c3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:20.489 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.486+0000 7f2fc8d72640 1 --2- 172.21.15.49:0/52582698 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2fc415ab50 0x7f2fc40c3a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:20.489 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.486+0000 7f2fc8d72640 1 -- 172.21.15.49:0/52582698 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fbc009a70 con 0x7f2fc40b0790 2023-11-06T02:13:20.490 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.486+0000 7f2fc3fff640 1 --2- 172.21.15.49:0/52582698 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2fc40cb010 0x7f2fc40af860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:13:20.490 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.486+0000 7f2fc8d72640 1 --2- 172.21.15.49:0/52582698 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2fc40b0790 0x7f2fc4147c40 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f2fb800c430 tx=0x7f2fb800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:20.490 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.486+0000 7f2fc17fa640 1 -- 172.21.15.49:0/52582698 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb80074e0 con 0x7f2fc40b0790 2023-11-06T02:13:20.490 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.486+0000 7f2fc17fa640 1 -- 172.21.15.49:0/52582698 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fb8007640 con 0x7f2fc40b0790 2023-11-06T02:13:20.490 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.486+0000 7f2fc17fa640 1 -- 172.21.15.49:0/52582698 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb80059d0 con 0x7f2fc40b0790 2023-11-06T02:13:20.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.486+0000 7f2fcafd6640 1 -- 172.21.15.49:0/52582698 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fc40ae820 con 0x7f2fc40b0790 2023-11-06T02:13:20.494 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.486+0000 7f2fcafd6640 1 -- 172.21.15.49:0/52582698 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fc40aede0 con 0x7f2fc40b0790 2023-11-06T02:13:20.494 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.490+0000 7f2fcafd6640 1 -- 172.21.15.49:0/52582698 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fc40b0de0 con 0x7f2fc40b0790 2023-11-06T02:13:20.494 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.490+0000 7f2fc17fa640 1 -- 172.21.15.49:0/52582698 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f2fb8005b30 con 0x7f2fc40b0790 2023-11-06T02:13:20.494 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.490+0000 7f2fc17fa640 1 --2- 172.21.15.49:0/52582698 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2fb0065a60 0x7f2fb0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:20.495 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.490+0000 7f2fc17fa640 1 -- 172.21.15.49:0/52582698 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6350+0+0 (secure 0 0 0) 0x7f2fb80829e0 con 0x7f2fc40b0790 2023-11-06T02:13:20.495 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.490+0000 7f2fc3fff640 1 --2- 172.21.15.49:0/52582698 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2fb0065a60 0x7f2fb0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:20.495 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.490+0000 7f2fc3fff640 1 --2- 172.21.15.49:0/52582698 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2fb0065a60 0x7f2fb0067f20 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f2fc40c1670 tx=0x7f2fb4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:20.495 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.494+0000 7f2fc17fa640 1 -- 172.21.15.49:0/52582698 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f2fb8050270 con 0x7f2fc40b0790 2023-11-06T02:13:20.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.650+0000 7f2fcafd6640 1 -- 172.21.15.49:0/52582698 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2fc4074040 con 0x7f2fc40b0790 2023-11-06T02:13:20.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.650+0000 7f2fc17fa640 1 -- 172.21.15.49:0/52582698 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) v1 ==== 74+0+18275 (secure 0 0 0) 0x7f2fb807ae60 con 0x7f2fc40b0790 2023-11-06T02:13:20.648 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:13:20.648 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":69,"fsid":"1a8b035c-6ba4-499a-b876-aea47fd9c0d4","created":"2023-11-06T02:08:06.440857+0000","modified":"2023-11-06T02:13:17.614076+0000","last_up_change":"2023-11-06T02:08:17.982372+0000","last_in_change":"2023-11-06T02:08:08.906631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T02:08:19.252222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T02:08:24.776211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-06T02:11:28.648326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-06T02:11:29.079513+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ad55af8-0571-4e6a-8722-b5d429b7496d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6809","nonce":38116},{"type":"v1","addr":"172.21.15.49:6811","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6813","nonce":38116},{"type":"v1","addr":"172.21.15.49:6815","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6821","nonce":38116},{"type":"v1","addr":"172.21.15.49:6823","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6817","nonce":38116},{"type":"v1","addr":"172.21.15.49:6819","nonce":38116}]},"public_addr":"172.21.15.49:6811/38116","cluster_addr":"172.21.15.49:6815/38116","heartbeat_back_addr":"172.21.15.49:6823/38116","heartbeat_front_addr":"172.21.15.49:6819/38116","state":["exists","up"]},{"osd":1,"uuid":"c8b31f85-a757-43ed-9419-351acc46611b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":38113},{"type":"v1","addr":"172.21.15.49:6810","nonce":38113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":38113},{"type":"v1","addr":"172.21.15.49:6814","nonce":38113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":38113},{"type":"v1","addr":"172.21.15.49:6822","nonce":38113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":38113},{"type":"v1","addr":"172.21.15.49:6818","nonce":38113}]},"public_addr":"172.21.15.49:6810/38113","cluster_addr":"172.21.15.49:6814/38113","heartbeat_back_addr":"172.21.15.49:6822/38113","heartbeat_front_addr":"172.21.15.49:6818/38113","state":["exists","up"]},{"osd":2,"uuid":"555b3bf3-0a70-4b91-8efd-e4204b5fc26e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":38114},{"type":"v1","addr":"172.21.15.49:6801","nonce":38114}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":38114},{"type":"v1","addr":"172.21.15.49:6803","nonce":38114}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":38114},{"type":"v1","addr":"172.21.15.49:6807","nonce":38114}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":38114},{"type":"v1","addr":"172.21.15.49:6805","nonce":38114}]},"public_addr":"172.21.15.49:6801/38114","cluster_addr":"172.21.15.49:6803/38114","heartbeat_back_addr":"172.21.15.49:6807/38114","heartbeat_front_addr":"172.21.15.49:6805/38114","state":["exists","up"]},{"osd":3,"uuid":"8d4e5d77-03ef-4b9b-bfcb-5ae93a46795c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":38115},{"type":"v1","addr":"172.21.15.49:6825","nonce":38115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":38115},{"type":"v1","addr":"172.21.15.49:6827","nonce":38115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":38115},{"type":"v1","addr":"172.21.15.49:6831","nonce":38115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":38115},{"type":"v1","addr":"172.21.15.49:6829","nonce":38115}]},"public_addr":"172.21.15.49:6825/38115","cluster_addr":"172.21.15.49:6827/38115","heartbeat_back_addr":"172.21.15.49:6831/38115","heartbeat_front_addr":"172.21.15.49:6829/38115","state":["exists","up"]},{"osd":4,"uuid":"dba03ad4-68a3-4cf3-9218-ac611f02eacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":38417},{"type":"v1","addr":"172.21.15.150:6809","nonce":38417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":38417},{"type":"v1","addr":"172.21.15.150:6811","nonce":38417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":38417},{"type":"v1","addr":"172.21.15.150:6815","nonce":38417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":38417},{"type":"v1","addr":"172.21.15.150:6813","nonce":38417}]},"public_addr":"172.21.15.150:6809/38417","cluster_addr":"172.21.15.150:6811/38417","heartbeat_back_addr":"172.21.15.150:6815/38417","heartbeat_front_addr":"172.21.15.150:6813/38417","state":["exists","up"]},{"osd":5,"uuid":"0df9c303-c255-4640-8e91-01f1c3efe18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":38418},{"type":"v1","addr":"172.21.15.150:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":38418},{"type":"v1","addr":"172.21.15.150:6803","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":38418},{"type":"v1","addr":"172.21.15.150:6807","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":38418},{"type":"v1","addr":"172.21.15.150:6805","nonce":38418}]},"public_addr":"172.21.15.150:6801/38418","cluster_addr":"172.21.15.150:6803/38418","heartbeat_back_addr":"172.21.15.150:6807/38418","heartbeat_front_addr":"172.21.15.150:6805/38418","state":["exists","up"]},{"osd":6,"uuid":"e59fa189-4b70-4c06-9722-c02b90828bf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":38428},{"type":"v1","addr":"172.21.15.150:6825","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":38428},{"type":"v1","addr":"172.21.15.150:6827","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":38428},{"type":"v1","addr":"172.21.15.150:6831","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":38428},{"type":"v1","addr":"172.21.15.150:6829","nonce":38428}]},"public_addr":"172.21.15.150:6825/38428","cluster_addr":"172.21.15.150:6827/38428","heartbeat_back_addr":"172.21.15.150:6831/38428","heartbeat_front_addr":"172.21.15.150:6829/38428","state":["exists","up"]},{"osd":7,"uuid":"a64f57e1-ef98-46bd-b304-385759e95898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":38421},{"type":"v1","addr":"172.21.15.150:6817","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":38421},{"type":"v1","addr":"172.21.15.150:6819","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":38421},{"type":"v1","addr":"172.21.15.150:6823","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":38421},{"type":"v1","addr":"172.21.15.150:6821","nonce":38421}]},"public_addr":"172.21.15.150:6817/38421","cluster_addr":"172.21.15.150:6819/38421","heartbeat_back_addr":"172.21.15.150:6823/38421","heartbeat_front_addr":"172.21.15.150:6821/38421","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.438552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.413280+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.296466+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.424966+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.276953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.230984+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.410235+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.480084+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.10","mappings":[{"from":7,"to":2}]},{"pgid":"10.1d","mappings":[{"from":4,"to":3}]},{"pgid":"10.21","mappings":[{"from":4,"to":3}]},{"pgid":"10.3a","mappings":[{"from":7,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:6835/3698847494":"2023-11-07T02:13:17.614029+0000","172.21.15.49:6834/3698847494":"2023-11-07T02:13:17.614029+0000","172.21.15.49:6835/342908487":"2023-11-07T02:12:41.031290+0000","172.21.15.49:6834/342908487":"2023-11-07T02:12:41.031290+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T02:13:20.650 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.654+0000 7f2fcafd6640 1 -- 172.21.15.49:0/52582698 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2fb0065a60 msgr2=0x7f2fb0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:20.650 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.654+0000 7f2fcafd6640 1 --2- 172.21.15.49:0/52582698 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2fb0065a60 0x7f2fb0067f20 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f2fc40c1670 tx=0x7f2fb4008040 comp rx=0 tx=0).stop 2023-11-06T02:13:20.650 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.654+0000 7f2fcafd6640 1 -- 172.21.15.49:0/52582698 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2fc40b0790 msgr2=0x7f2fc4147c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:20.651 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.654+0000 7f2fcafd6640 1 --2- 172.21.15.49:0/52582698 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2fc40b0790 0x7f2fc4147c40 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f2fb800c430 tx=0x7f2fb800c8f0 comp rx=0 tx=0).stop 2023-11-06T02:13:20.651 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.654+0000 7f2fcafd6640 1 -- 172.21.15.49:0/52582698 shutdown_connections 2023-11-06T02:13:20.651 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.654+0000 7f2fcafd6640 1 --2- 172.21.15.49:0/52582698 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2fb0065a60 0x7f2fb0067f20 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:20.651 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.654+0000 7f2fcafd6640 1 --2- 172.21.15.49:0/52582698 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2fc415ab50 0x7f2fc40c3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:20.652 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.654+0000 7f2fcafd6640 1 --2- 172.21.15.49:0/52582698 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2fc40cb010 0x7f2fc40af860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:20.652 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.654+0000 7f2fcafd6640 1 --2- 172.21.15.49:0/52582698 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2fc40b0790 0x7f2fc4147c40 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:20.652 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.654+0000 7f2fcafd6640 1 -- 172.21.15.49:0/52582698 >> 172.21.15.49:0/52582698 conn(0x7f2fc406c0a0 msgr2=0x7f2fc40c9bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:20.652 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.654+0000 7f2fcafd6640 1 -- 172.21.15.49:0/52582698 shutdown_connections 2023-11-06T02:13:20.652 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.654+0000 7f2fcafd6640 1 -- 172.21.15.49:0/52582698 wait complete. 2023-11-06T02:13:20.666 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-06T02:13:20.667 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T02:13:20.781 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.786+0000 7fd67e25f640 1 Processor -- start 2023-11-06T02:13:20.781 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.786+0000 7fd67e25f640 1 -- start start 2023-11-06T02:13:20.784 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.786+0000 7fd67e25f640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd6780cda70 0x7fd678161650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:20.784 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.786+0000 7fd67e25f640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fd6780714a0 0x7fd6780718f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:20.784 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.786+0000 7fd67e25f640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd678071e30 con 0x7fd6780b1f90 2023-11-06T02:13:20.785 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.786+0000 7fd67e25f640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd678161b90 con 0x7fd6780714a0 2023-11-06T02:13:20.785 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.786+0000 7fd67e25f640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6780cdef0 con 0x7fd6780cda70 2023-11-06T02:13:20.785 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.786+0000 7fd6777fe640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fd6780b1f90 0x7fd6780d1c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:46760/0 (socket says 172.21.15.49:46760) 2023-11-06T02:13:20.785 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.786+0000 7fd6777fe640 1 -- 172.21.15.49:0/1038176148 learned_addr learned my addr 172.21.15.49:0/1038176148 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:13:20.785 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.786+0000 7fd677fff640 1 --2- 172.21.15.49:0/1038176148 >> v2:172.21.15.150:3300/0 conn(0x7fd6780714a0 0x7fd6780718f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:20.786 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.786+0000 7fd676ffd640 1 --2- 172.21.15.49:0/1038176148 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd6780cda70 0x7fd678161650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:20.786 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.786+0000 7fd6767fc640 1 -- 172.21.15.49:0/1038176148 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2199349264 0 0) 0x7fd668001120 con 0x7fd6780b1f90 2023-11-06T02:13:20.786 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.786+0000 7fd6767fc640 1 -- 172.21.15.49:0/1038176148 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd65c003530 con 0x7fd6780b1f90 2023-11-06T02:13:20.786 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.786+0000 7fd6767fc640 1 -- 172.21.15.49:0/1038176148 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2335664814 0 0) 0x7fd65c003530 con 0x7fd6780b1f90 2023-11-06T02:13:20.787 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.786+0000 7fd6767fc640 1 -- 172.21.15.49:0/1038176148 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd6780cda70 msgr2=0x7fd678161650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:20.787 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.786+0000 7fd6767fc640 1 --2- 172.21.15.49:0/1038176148 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd6780cda70 0x7fd678161650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:20.787 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.786+0000 7fd6767fc640 1 -- 172.21.15.49:0/1038176148 >> v2:172.21.15.150:3300/0 conn(0x7fd6780714a0 msgr2=0x7fd6780718f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:20.787 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.786+0000 7fd6767fc640 1 --2- 172.21.15.49:0/1038176148 >> v2:172.21.15.150:3300/0 conn(0x7fd6780714a0 0x7fd6780718f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:20.787 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.786+0000 7fd6767fc640 1 -- 172.21.15.49:0/1038176148 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6780d17c0 con 0x7fd6780b1f90 2023-11-06T02:13:20.788 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.786+0000 7fd6767fc640 1 -- 172.21.15.49:0/1038176148 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7fd668001810 con 0x7fd6780b1f90 2023-11-06T02:13:20.788 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.786+0000 7fd6767fc640 1 -- 172.21.15.49:0/1038176148 >> v1:172.21.15.49:6789/0 conn(0x7fd6780b1f90 legacy=0x7fd6780d1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:20.788 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.786+0000 7fd6767fc640 1 --2- 172.21.15.49:0/1038176148 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd65c003980 0x7fd65c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:20.788 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.786+0000 7fd6767fc640 1 --2- 172.21.15.49:0/1038176148 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd6780cda70 0x7fd65c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:20.788 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.786+0000 7fd6767fc640 1 --2- 172.21.15.49:0/1038176148 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd65c004d10 0x7fd65c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:20.789 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.786+0000 7fd6767fc640 1 -- 172.21.15.49:0/1038176148 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6780cdef0 con 0x7fd65c004d10 2023-11-06T02:13:20.789 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.786+0000 7fd6767fc640 1 -- 172.21.15.49:0/1038176148 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd65c00b990 con 0x7fd65c003980 2023-11-06T02:13:20.789 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.786+0000 7fd6767fc640 1 -- 172.21.15.49:0/1038176148 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd65c00bad0 con 0x7fd6780cda70 2023-11-06T02:13:20.789 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.786+0000 7fd677fff640 1 --2- 172.21.15.49:0/1038176148 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd65c004d10 0x7fd65c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:20.789 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.786+0000 7fd676ffd640 1 --2- 172.21.15.49:0/1038176148 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd6780cda70 0x7fd65c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:20.789 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.786+0000 7fd677fff640 1 -- 172.21.15.49:0/1038176148 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd6780cda70 msgr2=0x7fd65c0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:20.790 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.786+0000 7fd677fff640 1 --2- 172.21.15.49:0/1038176148 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd6780cda70 0x7fd65c0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:20.790 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.786+0000 7fd6777fe640 1 --2- 172.21.15.49:0/1038176148 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd65c003980 0x7fd65c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:20.790 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.786+0000 7fd677fff640 1 -- 172.21.15.49:0/1038176148 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd65c003980 msgr2=0x7fd65c003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:20.790 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.786+0000 7fd677fff640 1 --2- 172.21.15.49:0/1038176148 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd65c003980 0x7fd65c003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:20.790 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.786+0000 7fd677fff640 1 -- 172.21.15.49:0/1038176148 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd65c003530 con 0x7fd65c004d10 2023-11-06T02:13:20.791 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.786+0000 7fd6777fe640 1 --2- 172.21.15.49:0/1038176148 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd65c003980 0x7fd65c003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:13:20.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.790+0000 7fd677fff640 1 --2- 172.21.15.49:0/1038176148 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd65c004d10 0x7fd65c005190 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fd67000e170 tx=0x7fd67001cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60489b407c704b81 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:20.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.790+0000 7fd6767fc640 1 -- 172.21.15.49:0/1038176148 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6700a5c90 con 0x7fd65c004d10 2023-11-06T02:13:20.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.790+0000 7fd6767fc640 1 -- 172.21.15.49:0/1038176148 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd67000f040 con 0x7fd65c004d10 2023-11-06T02:13:20.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.790+0000 7fd6767fc640 1 -- 172.21.15.49:0/1038176148 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6700ad440 con 0x7fd65c004d10 2023-11-06T02:13:20.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.790+0000 7fd67e25f640 1 -- 172.21.15.49:0/1038176148 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd65c004d10 msgr2=0x7fd65c005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:20.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.790+0000 7fd67e25f640 1 --2- 172.21.15.49:0/1038176148 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd65c004d10 0x7fd65c005190 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fd67000e170 tx=0x7fd67001cc80 comp rx=0 tx=0).stop 2023-11-06T02:13:20.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.790+0000 7fd677fff640 1 -- 172.21.15.49:0/1038176148 reap_dead start 2023-11-06T02:13:20.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.790+0000 7fd67e25f640 1 -- 172.21.15.49:0/1038176148 shutdown_connections 2023-11-06T02:13:20.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.790+0000 7fd67e25f640 1 -- 172.21.15.49:0/1038176148 >> 172.21.15.49:0/1038176148 conn(0x7fd6780d2aa0 msgr2=0x7fd6780d2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:20.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.790+0000 7fd67e25f640 1 -- 172.21.15.49:0/1038176148 shutdown_connections 2023-11-06T02:13:20.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.790+0000 7fd67e25f640 1 -- 172.21.15.49:0/1038176148 wait complete. 2023-11-06T02:13:20.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.790+0000 7fd67e25f640 1 Processor -- start 2023-11-06T02:13:20.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.790+0000 7fd67e25f640 1 -- start start 2023-11-06T02:13:20.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.794+0000 7fd67e25f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd678155ef0 0x7fd678157cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:20.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.794+0000 7fd67e25f640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd6781562d0 0x7fd6781f2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:20.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.794+0000 7fd67e25f640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd678156b50 0x7fd6780cbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:20.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.794+0000 7fd67e25f640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd67815d120 con 0x7fd678155ef0 2023-11-06T02:13:20.798 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.794+0000 7fd67e25f640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd67815a600 con 0x7fd678156b50 2023-11-06T02:13:20.798 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.794+0000 7fd67e25f640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6781f3370 con 0x7fd6781562d0 2023-11-06T02:13:20.798 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.794+0000 7fd6777fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd678155ef0 0x7fd678157cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:20.798 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.794+0000 7fd6777fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd678155ef0 0x7fd678157cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48730/0 (socket says 172.21.15.49:48730) 2023-11-06T02:13:20.798 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.794+0000 7fd6777fe640 1 -- 172.21.15.49:0/630346281 learned_addr learned my addr 172.21.15.49:0/630346281 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:20.799 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.794+0000 7fd676ffd640 1 --2- 172.21.15.49:0/630346281 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd6781562d0 0x7fd6781f2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:20.799 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.794+0000 7fd677fff640 1 --2- 172.21.15.49:0/630346281 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd678156b50 0x7fd6780cbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:20.799 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.794+0000 7fd6777fe640 1 -- 172.21.15.49:0/630346281 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd6781562d0 msgr2=0x7fd6781f2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:20.799 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.794+0000 7fd6777fe640 1 --2- 172.21.15.49:0/630346281 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd6781562d0 0x7fd6781f2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:20.799 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.794+0000 7fd6777fe640 1 -- 172.21.15.49:0/630346281 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd678156b50 msgr2=0x7fd6780cbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:20.800 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.794+0000 7fd6777fe640 1 --2- 172.21.15.49:0/630346281 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd678156b50 0x7fd6780cbcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:20.800 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.794+0000 7fd6777fe640 1 -- 172.21.15.49:0/630346281 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6781570c0 con 0x7fd678155ef0 2023-11-06T02:13:20.800 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.794+0000 7fd676ffd640 1 --2- 172.21.15.49:0/630346281 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd6781562d0 0x7fd6781f2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:13:20.800 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.794+0000 7fd6777fe640 1 --2- 172.21.15.49:0/630346281 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd678155ef0 0x7fd678157cf0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fd66800da60 tx=0x7fd66800df20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:20.800 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.794+0000 7fd66ffff640 1 -- 172.21.15.49:0/630346281 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd668004cc0 con 0x7fd678155ef0 2023-11-06T02:13:20.800 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.794+0000 7fd66ffff640 1 -- 172.21.15.49:0/630346281 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd668004e20 con 0x7fd678155ef0 2023-11-06T02:13:20.801 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.794+0000 7fd67e25f640 1 -- 172.21.15.49:0/630346281 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6781573a0 con 0x7fd678155ef0 2023-11-06T02:13:20.801 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.794+0000 7fd66ffff640 1 -- 172.21.15.49:0/630346281 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6680072d0 con 0x7fd678155ef0 2023-11-06T02:13:20.801 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.794+0000 7fd67e25f640 1 -- 172.21.15.49:0/630346281 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd678157960 con 0x7fd678155ef0 2023-11-06T02:13:20.801 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.794+0000 7fd67e25f640 1 -- 172.21.15.49:0/630346281 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd678074040 con 0x7fd678155ef0 2023-11-06T02:13:20.801 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.794+0000 7fd66ffff640 1 -- 172.21.15.49:0/630346281 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fd668007430 con 0x7fd678155ef0 2023-11-06T02:13:20.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.794+0000 7fd66ffff640 1 --2- 172.21.15.49:0/630346281 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd660065a90 0x7fd660067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:20.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.794+0000 7fd66ffff640 1 -- 172.21.15.49:0/630346281 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6350+0+0 (secure 0 0 0) 0x7fd668083090 con 0x7fd678155ef0 2023-11-06T02:13:20.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.794+0000 7fd676ffd640 1 --2- 172.21.15.49:0/630346281 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd660065a90 0x7fd660067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:20.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.798+0000 7fd676ffd640 1 --2- 172.21.15.49:0/630346281 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd660065a90 0x7fd660067f50 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fd6781566b0 tx=0x7fd664008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:20.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.798+0000 7fd66ffff640 1 -- 172.21.15.49:0/630346281 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fd6680507d0 con 0x7fd678155ef0 2023-11-06T02:13:20.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.982+0000 7fd67e25f640 1 -- 172.21.15.49:0/630346281 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd678071e30 con 0x7fd678155ef0 2023-11-06T02:13:20.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.982+0000 7fd66ffff640 1 -- 172.21.15.49:0/630346281 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4645 (secure 0 0 0) 0x7fd678071e30 con 0x7fd678155ef0 2023-11-06T02:13:20.978 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:13:20.979 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":38,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5635,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4170291905","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":4170291905},{"type":"v1","addr":"172.21.15.150:6835","nonce":4170291905}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5638,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/2719458937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":2719458937},{"type":"v1","addr":"172.21.15.150:6837","nonce":2719458937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":6339,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/2310338375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2310338375},{"type":"v1","addr":"172.21.15.49:6833","nonce":2310338375}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":6960,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/322306580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":322306580},{"type":"v1","addr":"172.21.15.49:6835","nonce":322306580}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":37,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T02:11:30.091070+0000","modified":"2023-11-06T02:13:17.615097+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-06T02:13:20.981 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 38 2023-11-06T02:13:20.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.982+0000 7fd67e25f640 1 -- 172.21.15.49:0/630346281 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd660065a90 msgr2=0x7fd660067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:20.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.982+0000 7fd67e25f640 1 --2- 172.21.15.49:0/630346281 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd660065a90 0x7fd660067f50 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fd6781566b0 tx=0x7fd664008040 comp rx=0 tx=0).stop 2023-11-06T02:13:20.982 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.982+0000 7fd67e25f640 1 -- 172.21.15.49:0/630346281 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd678155ef0 msgr2=0x7fd678157cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:20.982 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.982+0000 7fd67e25f640 1 --2- 172.21.15.49:0/630346281 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd678155ef0 0x7fd678157cf0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fd66800da60 tx=0x7fd66800df20 comp rx=0 tx=0).stop 2023-11-06T02:13:20.983 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.982+0000 7fd67e25f640 1 -- 172.21.15.49:0/630346281 shutdown_connections 2023-11-06T02:13:20.983 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.982+0000 7fd67e25f640 1 --2- 172.21.15.49:0/630346281 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd660065a90 0x7fd660067f50 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:20.984 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.982+0000 7fd67e25f640 1 --2- 172.21.15.49:0/630346281 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd678156b50 0x7fd6780cbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:20.984 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.982+0000 7fd67e25f640 1 --2- 172.21.15.49:0/630346281 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd6781562d0 0x7fd6781f2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:20.984 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.982+0000 7fd67e25f640 1 --2- 172.21.15.49:0/630346281 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd678155ef0 0x7fd678157cf0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:20.985 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.982+0000 7fd67e25f640 1 -- 172.21.15.49:0/630346281 >> 172.21.15.49:0/630346281 conn(0x7fd6780d2aa0 msgr2=0x7fd6780c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:20.985 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.982+0000 7fd67e25f640 1 -- 172.21.15.49:0/630346281 shutdown_connections 2023-11-06T02:13:20.986 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:20.982+0000 7fd67e25f640 1 -- 172.21.15.49:0/630346281 wait complete. 2023-11-06T02:13:20.996 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T02:13:21.116 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.118+0000 7f1590d56640 1 Processor -- start 2023-11-06T02:13:21.117 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.118+0000 7f1590d56640 1 -- start start 2023-11-06T02:13:21.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.122+0000 7f1590d56640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f158c0b0790 0x7f158c0c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:21.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.122+0000 7f1590d56640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f158c0cab80 con 0x7f158c0cb010 2023-11-06T02:13:21.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.122+0000 7f1590d56640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f158c15af30 con 0x7f158c15ab50 2023-11-06T02:13:21.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.122+0000 7f1590d56640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f158c0cb5b0 con 0x7f158c0b0790 2023-11-06T02:13:21.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.122+0000 7f158ad9d640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f158c0cb010 0x7f158c0c2710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:46766/0 (socket says 172.21.15.49:46766) 2023-11-06T02:13:21.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.122+0000 7f158ad9d640 1 -- 172.21.15.49:0/3487191947 learned_addr learned my addr 172.21.15.49:0/3487191947 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:13:21.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.122+0000 7f1589d9b640 1 --2- 172.21.15.49:0/3487191947 >> v2:172.21.15.150:3300/0 conn(0x7f158c0b0790 0x7f158c0c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:21.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.122+0000 7f158959a640 1 -- 172.21.15.49:0/3487191947 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 130510700 0 0) 0x7f158c0cab80 con 0x7f158c0cb010 2023-11-06T02:13:21.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.122+0000 7f158959a640 1 -- 172.21.15.49:0/3487191947 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1574003530 con 0x7f158c0cb010 2023-11-06T02:13:21.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.122+0000 7f158959a640 1 -- 172.21.15.49:0/3487191947 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1893887556 0 0) 0x7f1574003530 con 0x7f158c0cb010 2023-11-06T02:13:21.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.122+0000 7f158959a640 1 -- 172.21.15.49:0/3487191947 >> v2:172.21.15.150:3300/0 conn(0x7f158c0b0790 msgr2=0x7f158c0c9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:21.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.122+0000 7f158959a640 1 --2- 172.21.15.49:0/3487191947 >> v2:172.21.15.150:3300/0 conn(0x7f158c0b0790 0x7f158c0c9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:21.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.122+0000 7f158959a640 1 -- 172.21.15.49:0/3487191947 >> v1:172.21.15.150:6789/0 conn(0x7f158c15ab50 legacy=0x7f158c0b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:21.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.122+0000 7f158959a640 1 -- 172.21.15.49:0/3487191947 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f158c0cb890 con 0x7f158c0cb010 2023-11-06T02:13:21.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.122+0000 7f158959a640 1 -- 172.21.15.49:0/3487191947 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7f1584001a00 con 0x7f158c0cb010 2023-11-06T02:13:21.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.122+0000 7f158959a640 1 -- 172.21.15.49:0/3487191947 >> v1:172.21.15.49:6789/0 conn(0x7f158c0cb010 legacy=0x7f158c0c2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:21.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.122+0000 7f158959a640 1 --2- 172.21.15.49:0/3487191947 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1574003980 0x7f1574003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:21.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.122+0000 7f158959a640 1 --2- 172.21.15.49:0/3487191947 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1574004380 0x7f1574004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:21.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.122+0000 7f158959a640 1 --2- 172.21.15.49:0/3487191947 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1574005050 0x7f157400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:21.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.122+0000 7f158959a640 1 -- 172.21.15.49:0/3487191947 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f158c0cb5b0 con 0x7f1574003980 2023-11-06T02:13:21.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.122+0000 7f158959a640 1 -- 172.21.15.49:0/3487191947 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f157400bfb0 con 0x7f1574004380 2023-11-06T02:13:21.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.122+0000 7f158959a640 1 -- 172.21.15.49:0/3487191947 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f157400c170 con 0x7f1574005050 2023-11-06T02:13:21.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.122+0000 7f158a59c640 1 --2- 172.21.15.49:0/3487191947 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1574003980 0x7f1574003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:21.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.122+0000 7f1589d9b640 1 --2- 172.21.15.49:0/3487191947 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1574004380 0x7f1574004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:21.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.122+0000 7f158ad9d640 1 --2- 172.21.15.49:0/3487191947 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1574005050 0x7f157400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:21.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.122+0000 7f158a59c640 1 -- 172.21.15.49:0/3487191947 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1574005050 msgr2=0x7f157400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:21.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.122+0000 7f158a59c640 1 --2- 172.21.15.49:0/3487191947 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1574005050 0x7f157400b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:21.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.122+0000 7f158a59c640 1 -- 172.21.15.49:0/3487191947 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1574004380 msgr2=0x7f1574004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:21.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.122+0000 7f158a59c640 1 --2- 172.21.15.49:0/3487191947 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1574004380 0x7f1574004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:21.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.122+0000 7f158a59c640 1 -- 172.21.15.49:0/3487191947 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1574003530 con 0x7f1574003980 2023-11-06T02:13:21.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.122+0000 7f158ad9d640 1 -- 172.21.15.49:0/3487191947 reap_dead start 2023-11-06T02:13:21.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.126+0000 7f158a59c640 1 --2- 172.21.15.49:0/3487191947 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1574003980 0x7f1574003d60 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f15800018c0 tx=0x7f158001c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=238bbf34493d88ca server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:21.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.126+0000 7f158959a640 1 -- 172.21.15.49:0/3487191947 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15800ab600 con 0x7f1574003980 2023-11-06T02:13:21.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.126+0000 7f158959a640 1 -- 172.21.15.49:0/3487191947 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15800a7050 con 0x7f1574003980 2023-11-06T02:13:21.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.126+0000 7f158959a640 1 -- 172.21.15.49:0/3487191947 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15800ab9c0 con 0x7f1574003980 2023-11-06T02:13:21.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.126+0000 7f1590d56640 1 -- 172.21.15.49:0/3487191947 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1574003980 msgr2=0x7f1574003d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:21.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.126+0000 7f1590d56640 1 --2- 172.21.15.49:0/3487191947 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1574003980 0x7f1574003d60 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f15800018c0 tx=0x7f158001c4b0 comp rx=0 tx=0).stop 2023-11-06T02:13:21.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.126+0000 7f1590d56640 1 -- 172.21.15.49:0/3487191947 shutdown_connections 2023-11-06T02:13:21.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.126+0000 7f1590d56640 1 --2- 172.21.15.49:0/3487191947 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1574003980 0x7f1574003d60 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:21.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.126+0000 7f1590d56640 1 -- 172.21.15.49:0/3487191947 >> 172.21.15.49:0/3487191947 conn(0x7f158c06c0a0 msgr2=0x7f158c06c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:21.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.126+0000 7f1590d56640 1 -- 172.21.15.49:0/3487191947 shutdown_connections 2023-11-06T02:13:21.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.126+0000 7f1590d56640 1 -- 172.21.15.49:0/3487191947 wait complete. 2023-11-06T02:13:21.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.126+0000 7f1590d56640 1 Processor -- start 2023-11-06T02:13:21.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.126+0000 7f1590d56640 1 -- start start 2023-11-06T02:13:21.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.126+0000 7f1590d56640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1574003980 0x7f158c0c8670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:21.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.126+0000 7f1590d56640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f158c156490 0x7f158c0c56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:21.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.126+0000 7f1590d56640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f158c0c7060 0x7f158c158470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:21.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.126+0000 7f1590d56640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f158c0c5c10 con 0x7f1574003980 2023-11-06T02:13:21.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.126+0000 7f1590d56640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f158c1f50a0 con 0x7f158c156490 2023-11-06T02:13:21.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.126+0000 7f1590d56640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f158c0c75d0 con 0x7f158c0c7060 2023-11-06T02:13:21.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.126+0000 7f158a59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1574003980 0x7f158c0c8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:21.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.126+0000 7f158a59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1574003980 0x7f158c0c8670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48736/0 (socket says 172.21.15.49:48736) 2023-11-06T02:13:21.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.126+0000 7f158a59c640 1 -- 172.21.15.49:0/2962717795 learned_addr learned my addr 172.21.15.49:0/2962717795 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:21.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.126+0000 7f1589d9b640 1 --2- 172.21.15.49:0/2962717795 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f158c156490 0x7f158c0c56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:21.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.126+0000 7f158a59c640 1 -- 172.21.15.49:0/2962717795 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f158c0c7060 msgr2=0x7f158c158470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:21.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.126+0000 7f158a59c640 1 --2- 172.21.15.49:0/2962717795 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f158c0c7060 0x7f158c158470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:21.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.126+0000 7f158a59c640 1 -- 172.21.15.49:0/2962717795 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f158c156490 msgr2=0x7f158c0c56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:21.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.126+0000 7f158a59c640 1 --2- 172.21.15.49:0/2962717795 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f158c156490 0x7f158c0c56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:21.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.126+0000 7f158a59c640 1 -- 172.21.15.49:0/2962717795 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f158c0c78b0 con 0x7f1574003980 2023-11-06T02:13:21.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.126+0000 7f158a59c640 1 --2- 172.21.15.49:0/2962717795 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1574003980 0x7f158c0c8670 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f158001c2e0 tx=0x7f1574005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:21.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.126+0000 7f157affd640 1 -- 172.21.15.49:0/2962717795 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1580003910 con 0x7f1574003980 2023-11-06T02:13:21.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.126+0000 7f157affd640 1 -- 172.21.15.49:0/2962717795 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1580003a70 con 0x7f1574003980 2023-11-06T02:13:21.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.126+0000 7f1590d56640 1 -- 172.21.15.49:0/2962717795 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f158c0c7b90 con 0x7f1574003980 2023-11-06T02:13:21.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.126+0000 7f157affd640 1 -- 172.21.15.49:0/2962717795 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1580003e00 con 0x7f1574003980 2023-11-06T02:13:21.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.126+0000 7f1590d56640 1 -- 172.21.15.49:0/2962717795 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f158c0c8150 con 0x7f1574003980 2023-11-06T02:13:21.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.130+0000 7f1590d56640 1 -- 172.21.15.49:0/2962717795 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f158c0b0de0 con 0x7f1574003980 2023-11-06T02:13:21.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.130+0000 7f157affd640 1 -- 172.21.15.49:0/2962717795 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f15800b2040 con 0x7f1574003980 2023-11-06T02:13:21.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.130+0000 7f157affd640 1 --2- 172.21.15.49:0/2962717795 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f156c065a90 0x7f156c067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:21.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.130+0000 7f157affd640 1 -- 172.21.15.49:0/2962717795 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6350+0+0 (secure 0 0 0) 0x7f158011c570 con 0x7f1574003980 2023-11-06T02:13:21.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.130+0000 7f1589d9b640 1 --2- 172.21.15.49:0/2962717795 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f156c065a90 0x7f156c067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:21.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.130+0000 7f1589d9b640 1 --2- 172.21.15.49:0/2962717795 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f156c065a90 0x7f156c067f50 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f158c071b40 tx=0x7f157c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:21.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.134+0000 7f157affd640 1 -- 172.21.15.49:0/2962717795 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f15800e9cc0 con 0x7f1574003980 2023-11-06T02:13:21.318 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.322+0000 7f1590d56640 1 -- 172.21.15.49:0/2962717795 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f158c0cbe20 con 0x7f1574003980 2023-11-06T02:13:21.319 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.322+0000 7f157affd640 1 -- 172.21.15.49:0/2962717795 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4645 (secure 0 0 0) 0x7f15800ed2e0 con 0x7f1574003980 2023-11-06T02:13:21.320 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:13:21.320 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":38,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5635,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4170291905","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":4170291905},{"type":"v1","addr":"172.21.15.150:6835","nonce":4170291905}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5638,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/2719458937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":2719458937},{"type":"v1","addr":"172.21.15.150:6837","nonce":2719458937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":6339,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/2310338375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2310338375},{"type":"v1","addr":"172.21.15.49:6833","nonce":2310338375}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":6960,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/322306580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":322306580},{"type":"v1","addr":"172.21.15.49:6835","nonce":322306580}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":37,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T02:11:30.091070+0000","modified":"2023-11-06T02:13:17.615097+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-06T02:13:21.320 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 38 2023-11-06T02:13:21.322 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.326+0000 7f1590d56640 1 -- 172.21.15.49:0/2962717795 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f156c065a90 msgr2=0x7f156c067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:21.322 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.326+0000 7f1590d56640 1 --2- 172.21.15.49:0/2962717795 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f156c065a90 0x7f156c067f50 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f158c071b40 tx=0x7f157c008040 comp rx=0 tx=0).stop 2023-11-06T02:13:21.322 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.326+0000 7f1590d56640 1 -- 172.21.15.49:0/2962717795 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1574003980 msgr2=0x7f158c0c8670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:21.322 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.326+0000 7f1590d56640 1 --2- 172.21.15.49:0/2962717795 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1574003980 0x7f158c0c8670 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f158001c2e0 tx=0x7f1574005800 comp rx=0 tx=0).stop 2023-11-06T02:13:21.323 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.326+0000 7f1590d56640 1 -- 172.21.15.49:0/2962717795 shutdown_connections 2023-11-06T02:13:21.323 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.326+0000 7f1590d56640 1 --2- 172.21.15.49:0/2962717795 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f156c065a90 0x7f156c067f50 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:21.323 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.326+0000 7f1590d56640 1 --2- 172.21.15.49:0/2962717795 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f158c0c7060 0x7f158c158470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:21.323 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.326+0000 7f1590d56640 1 --2- 172.21.15.49:0/2962717795 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f158c156490 0x7f158c0c56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:21.323 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.326+0000 7f1590d56640 1 --2- 172.21.15.49:0/2962717795 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1574003980 0x7f158c0c8670 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:21.324 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.326+0000 7f1590d56640 1 -- 172.21.15.49:0/2962717795 >> 172.21.15.49:0/2962717795 conn(0x7f158c06c0a0 msgr2=0x7f158c0c3020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:21.324 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.326+0000 7f1590d56640 1 -- 172.21.15.49:0/2962717795 shutdown_connections 2023-11-06T02:13:21.324 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.326+0000 7f1590d56640 1 -- 172.21.15.49:0/2962717795 wait complete. 2023-11-06T02:13:21.338 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T02:13:21.461 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.466+0000 7fa41b337640 1 Processor -- start 2023-11-06T02:13:21.461 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.466+0000 7fa41b337640 1 -- start start 2023-11-06T02:13:21.462 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.466+0000 7fa41b337640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa41406b590 0x7fa4140cae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:21.463 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.466+0000 7fa41b337640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa414161440 con 0x7fa4140cb3b0 2023-11-06T02:13:21.463 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.466+0000 7fa41b337640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4140ae410 con 0x7fa414161060 2023-11-06T02:13:21.463 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.466+0000 7fa41b337640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4140cbba0 con 0x7fa41406b590 2023-11-06T02:13:21.463 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.466+0000 7fa4188d2640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa41406b590 0x7fa4140cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:21.464 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.466+0000 7fa4198d4640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fa4140cb3b0 0x7fa4140ccc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:46780/0 (socket says 172.21.15.49:46780) 2023-11-06T02:13:21.464 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.466+0000 7fa4198d4640 1 -- 172.21.15.49:0/3485051953 learned_addr learned my addr 172.21.15.49:0/3485051953 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:13:21.464 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.466+0000 7fa4188d2640 1 -- 172.21.15.49:0/3485051953 >> v1:172.21.15.150:6789/0 conn(0x7fa414161060 legacy=0x7fa41406ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:21.464 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.466+0000 7fa4188d2640 1 -- 172.21.15.49:0/3485051953 >> v1:172.21.15.49:6789/0 conn(0x7fa4140cb3b0 legacy=0x7fa4140ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:21.464 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.466+0000 7fa4188d2640 1 -- 172.21.15.49:0/3485051953 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4140cbe80 con 0x7fa41406b590 2023-11-06T02:13:21.464 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.466+0000 7fa413fff640 1 -- 172.21.15.49:0/3485051953 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3406368735 0 0) 0x7fa414161440 con 0x7fa4140cb3b0 2023-11-06T02:13:21.465 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.470+0000 7fa4188d2640 1 --2- 172.21.15.49:0/3485051953 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa41406b590 0x7fa4140cae70 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fa4080097c0 tx=0x7fa40801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37a1bac7d755122d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:21.465 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.470+0000 7fa413fff640 1 -- 172.21.15.49:0/3485051953 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4080ab600 con 0x7fa41406b590 2023-11-06T02:13:21.466 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.470+0000 7fa413fff640 1 -- 172.21.15.49:0/3485051953 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4080a7050 con 0x7fa41406b590 2023-11-06T02:13:21.466 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.470+0000 7fa413fff640 1 -- 172.21.15.49:0/3485051953 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4080ab980 con 0x7fa41406b590 2023-11-06T02:13:21.467 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.470+0000 7fa41b337640 1 -- 172.21.15.49:0/3485051953 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa41406b590 msgr2=0x7fa4140cae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:21.467 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.470+0000 7fa41b337640 1 --2- 172.21.15.49:0/3485051953 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa41406b590 0x7fa4140cae70 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fa4080097c0 tx=0x7fa40801c450 comp rx=0 tx=0).stop 2023-11-06T02:13:21.467 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.470+0000 7fa41b337640 1 -- 172.21.15.49:0/3485051953 shutdown_connections 2023-11-06T02:13:21.467 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.470+0000 7fa41b337640 1 --2- 172.21.15.49:0/3485051953 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa41406b590 0x7fa4140cae70 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:21.468 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.470+0000 7fa41b337640 1 -- 172.21.15.49:0/3485051953 >> 172.21.15.49:0/3485051953 conn(0x7fa414089fd0 msgr2=0x7fa414099c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:21.468 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.470+0000 7fa41b337640 1 -- 172.21.15.49:0/3485051953 shutdown_connections 2023-11-06T02:13:21.468 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.470+0000 7fa41b337640 1 -- 172.21.15.49:0/3485051953 wait complete. 2023-11-06T02:13:21.468 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.470+0000 7fa41b337640 1 Processor -- start 2023-11-06T02:13:21.468 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.470+0000 7fa41b337640 1 -- start start 2023-11-06T02:13:21.470 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.470+0000 7fa41b337640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa41406b590 0x7fa4140c9060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:21.471 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.470+0000 7fa41b337640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4140cb3b0 0x7fa4140c4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:21.471 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.470+0000 7fa41b337640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa414161060 0x7fa4140c8300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:21.471 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.470+0000 7fa41b337640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4140b5f80 con 0x7fa4140cb3b0 2023-11-06T02:13:21.471 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.470+0000 7fa41b337640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4140b3db0 con 0x7fa414161060 2023-11-06T02:13:21.472 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.470+0000 7fa41b337640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4140b3480 con 0x7fa41406b590 2023-11-06T02:13:21.472 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.470+0000 7fa4188d2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4140cb3b0 0x7fa4140c4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:21.472 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.470+0000 7fa4188d2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4140cb3b0 0x7fa4140c4270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48758/0 (socket says 172.21.15.49:48758) 2023-11-06T02:13:21.472 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.470+0000 7fa4188d2640 1 -- 172.21.15.49:0/1076611407 learned_addr learned my addr 172.21.15.49:0/1076611407 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:21.472 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.470+0000 7fa4190d3640 1 --2- 172.21.15.49:0/1076611407 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa41406b590 0x7fa4140c9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:21.473 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.474+0000 7fa4198d4640 1 --2- 172.21.15.49:0/1076611407 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa414161060 0x7fa4140c8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:21.473 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.474+0000 7fa4188d2640 1 -- 172.21.15.49:0/1076611407 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa41406b590 msgr2=0x7fa4140c9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:21.473 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.474+0000 7fa4188d2640 1 --2- 172.21.15.49:0/1076611407 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa41406b590 0x7fa4140c9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:21.473 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.474+0000 7fa4188d2640 1 -- 172.21.15.49:0/1076611407 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa414161060 msgr2=0x7fa4140c8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:21.473 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.474+0000 7fa4188d2640 1 --2- 172.21.15.49:0/1076611407 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa414161060 0x7fa4140c8300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:21.473 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.474+0000 7fa4188d2640 1 -- 172.21.15.49:0/1076611407 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa408009420 con 0x7fa4140cb3b0 2023-11-06T02:13:21.474 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.474+0000 7fa4188d2640 1 --2- 172.21.15.49:0/1076611407 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4140cb3b0 0x7fa4140c4270 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fa4080ae040 tx=0x7fa4080191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:21.474 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.474+0000 7fa411ffb640 1 -- 172.21.15.49:0/1076611407 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4080b4840 con 0x7fa4140cb3b0 2023-11-06T02:13:21.474 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.474+0000 7fa411ffb640 1 -- 172.21.15.49:0/1076611407 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4080b49a0 con 0x7fa4140cb3b0 2023-11-06T02:13:21.474 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.474+0000 7fa41b337640 1 -- 172.21.15.49:0/1076611407 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4140b3760 con 0x7fa4140cb3b0 2023-11-06T02:13:21.474 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.474+0000 7fa411ffb640 1 -- 172.21.15.49:0/1076611407 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4080b4d50 con 0x7fa4140cb3b0 2023-11-06T02:13:21.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.474+0000 7fa41b337640 1 -- 172.21.15.49:0/1076611407 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4140c9690 con 0x7fa4140cb3b0 2023-11-06T02:13:21.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.474+0000 7fa41b337640 1 -- 172.21.15.49:0/1076611407 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4140b3f20 con 0x7fa4140cb3b0 2023-11-06T02:13:21.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.474+0000 7fa411ffb640 1 -- 172.21.15.49:0/1076611407 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fa4080b2040 con 0x7fa4140cb3b0 2023-11-06T02:13:21.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.474+0000 7fa411ffb640 1 --2- 172.21.15.49:0/1076611407 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa400065a90 0x7fa400067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:21.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.474+0000 7fa411ffb640 1 -- 172.21.15.49:0/1076611407 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6350+0+0 (secure 0 0 0) 0x7fa40811d7b0 con 0x7fa4140cb3b0 2023-11-06T02:13:21.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.474+0000 7fa4190d3640 1 --2- 172.21.15.49:0/1076611407 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa400065a90 0x7fa400067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:21.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.474+0000 7fa4190d3640 1 --2- 172.21.15.49:0/1076611407 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa400065a90 0x7fa400067f50 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fa4140c8ff0 tx=0x7fa40c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:21.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.478+0000 7fa411ffb640 1 -- 172.21.15.49:0/1076611407 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fa4080eaf80 con 0x7fa4140cb3b0 2023-11-06T02:13:21.631 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.634+0000 7fa41b337640 1 -- 172.21.15.49:0/1076611407 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa4140c3950 con 0x7fa4140cb3b0 2023-11-06T02:13:21.634 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.638+0000 7fa411ffb640 1 -- 172.21.15.49:0/1076611407 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) v1 ==== 74+0+18275 (secure 0 0 0) 0x7fa4080ee5a0 con 0x7fa4140cb3b0 2023-11-06T02:13:21.635 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:13:21.635 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":69,"fsid":"1a8b035c-6ba4-499a-b876-aea47fd9c0d4","created":"2023-11-06T02:08:06.440857+0000","modified":"2023-11-06T02:13:17.614076+0000","last_up_change":"2023-11-06T02:08:17.982372+0000","last_in_change":"2023-11-06T02:08:08.906631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T02:08:19.252222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T02:08:24.776211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-06T02:11:28.648326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-06T02:11:29.079513+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ad55af8-0571-4e6a-8722-b5d429b7496d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6809","nonce":38116},{"type":"v1","addr":"172.21.15.49:6811","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6813","nonce":38116},{"type":"v1","addr":"172.21.15.49:6815","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6821","nonce":38116},{"type":"v1","addr":"172.21.15.49:6823","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6817","nonce":38116},{"type":"v1","addr":"172.21.15.49:6819","nonce":38116}]},"public_addr":"172.21.15.49:6811/38116","cluster_addr":"172.21.15.49:6815/38116","heartbeat_back_addr":"172.21.15.49:6823/38116","heartbeat_front_addr":"172.21.15.49:6819/38116","state":["exists","up"]},{"osd":1,"uuid":"c8b31f85-a757-43ed-9419-351acc46611b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":38113},{"type":"v1","addr":"172.21.15.49:6810","nonce":38113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":38113},{"type":"v1","addr":"172.21.15.49:6814","nonce":38113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":38113},{"type":"v1","addr":"172.21.15.49:6822","nonce":38113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":38113},{"type":"v1","addr":"172.21.15.49:6818","nonce":38113}]},"public_addr":"172.21.15.49:6810/38113","cluster_addr":"172.21.15.49:6814/38113","heartbeat_back_addr":"172.21.15.49:6822/38113","heartbeat_front_addr":"172.21.15.49:6818/38113","state":["exists","up"]},{"osd":2,"uuid":"555b3bf3-0a70-4b91-8efd-e4204b5fc26e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":38114},{"type":"v1","addr":"172.21.15.49:6801","nonce":38114}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":38114},{"type":"v1","addr":"172.21.15.49:6803","nonce":38114}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":38114},{"type":"v1","addr":"172.21.15.49:6807","nonce":38114}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":38114},{"type":"v1","addr":"172.21.15.49:6805","nonce":38114}]},"public_addr":"172.21.15.49:6801/38114","cluster_addr":"172.21.15.49:6803/38114","heartbeat_back_addr":"172.21.15.49:6807/38114","heartbeat_front_addr":"172.21.15.49:6805/38114","state":["exists","up"]},{"osd":3,"uuid":"8d4e5d77-03ef-4b9b-bfcb-5ae93a46795c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":38115},{"type":"v1","addr":"172.21.15.49:6825","nonce":38115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":38115},{"type":"v1","addr":"172.21.15.49:6827","nonce":38115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":38115},{"type":"v1","addr":"172.21.15.49:6831","nonce":38115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":38115},{"type":"v1","addr":"172.21.15.49:6829","nonce":38115}]},"public_addr":"172.21.15.49:6825/38115","cluster_addr":"172.21.15.49:6827/38115","heartbeat_back_addr":"172.21.15.49:6831/38115","heartbeat_front_addr":"172.21.15.49:6829/38115","state":["exists","up"]},{"osd":4,"uuid":"dba03ad4-68a3-4cf3-9218-ac611f02eacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":38417},{"type":"v1","addr":"172.21.15.150:6809","nonce":38417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":38417},{"type":"v1","addr":"172.21.15.150:6811","nonce":38417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":38417},{"type":"v1","addr":"172.21.15.150:6815","nonce":38417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":38417},{"type":"v1","addr":"172.21.15.150:6813","nonce":38417}]},"public_addr":"172.21.15.150:6809/38417","cluster_addr":"172.21.15.150:6811/38417","heartbeat_back_addr":"172.21.15.150:6815/38417","heartbeat_front_addr":"172.21.15.150:6813/38417","state":["exists","up"]},{"osd":5,"uuid":"0df9c303-c255-4640-8e91-01f1c3efe18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":38418},{"type":"v1","addr":"172.21.15.150:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":38418},{"type":"v1","addr":"172.21.15.150:6803","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":38418},{"type":"v1","addr":"172.21.15.150:6807","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":38418},{"type":"v1","addr":"172.21.15.150:6805","nonce":38418}]},"public_addr":"172.21.15.150:6801/38418","cluster_addr":"172.21.15.150:6803/38418","heartbeat_back_addr":"172.21.15.150:6807/38418","heartbeat_front_addr":"172.21.15.150:6805/38418","state":["exists","up"]},{"osd":6,"uuid":"e59fa189-4b70-4c06-9722-c02b90828bf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":38428},{"type":"v1","addr":"172.21.15.150:6825","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":38428},{"type":"v1","addr":"172.21.15.150:6827","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":38428},{"type":"v1","addr":"172.21.15.150:6831","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":38428},{"type":"v1","addr":"172.21.15.150:6829","nonce":38428}]},"public_addr":"172.21.15.150:6825/38428","cluster_addr":"172.21.15.150:6827/38428","heartbeat_back_addr":"172.21.15.150:6831/38428","heartbeat_front_addr":"172.21.15.150:6829/38428","state":["exists","up"]},{"osd":7,"uuid":"a64f57e1-ef98-46bd-b304-385759e95898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":38421},{"type":"v1","addr":"172.21.15.150:6817","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":38421},{"type":"v1","addr":"172.21.15.150:6819","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":38421},{"type":"v1","addr":"172.21.15.150:6823","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":38421},{"type":"v1","addr":"172.21.15.150:6821","nonce":38421}]},"public_addr":"172.21.15.150:6817/38421","cluster_addr":"172.21.15.150:6819/38421","heartbeat_back_addr":"172.21.15.150:6823/38421","heartbeat_front_addr":"172.21.15.150:6821/38421","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.438552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.413280+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.296466+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.424966+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.276953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.230984+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.410235+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.480084+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.10","mappings":[{"from":7,"to":2}]},{"pgid":"10.1d","mappings":[{"from":4,"to":3}]},{"pgid":"10.21","mappings":[{"from":4,"to":3}]},{"pgid":"10.3a","mappings":[{"from":7,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:6835/3698847494":"2023-11-07T02:13:17.614029+0000","172.21.15.49:6834/3698847494":"2023-11-07T02:13:17.614029+0000","172.21.15.49:6835/342908487":"2023-11-07T02:12:41.031290+0000","172.21.15.49:6834/342908487":"2023-11-07T02:12:41.031290+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T02:13:21.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.642+0000 7fa41b337640 1 -- 172.21.15.49:0/1076611407 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa400065a90 msgr2=0x7fa400067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:21.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.642+0000 7fa41b337640 1 --2- 172.21.15.49:0/1076611407 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa400065a90 0x7fa400067f50 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fa4140c8ff0 tx=0x7fa40c009040 comp rx=0 tx=0).stop 2023-11-06T02:13:21.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.642+0000 7fa41b337640 1 -- 172.21.15.49:0/1076611407 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4140cb3b0 msgr2=0x7fa4140c4270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:21.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.642+0000 7fa41b337640 1 --2- 172.21.15.49:0/1076611407 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4140cb3b0 0x7fa4140c4270 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fa4080ae040 tx=0x7fa4080191f0 comp rx=0 tx=0).stop 2023-11-06T02:13:21.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.642+0000 7fa41b337640 1 -- 172.21.15.49:0/1076611407 shutdown_connections 2023-11-06T02:13:21.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.642+0000 7fa41b337640 1 --2- 172.21.15.49:0/1076611407 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa400065a90 0x7fa400067f50 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:21.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.642+0000 7fa41b337640 1 --2- 172.21.15.49:0/1076611407 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4140cb3b0 0x7fa4140c4270 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:21.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.642+0000 7fa41b337640 1 --2- 172.21.15.49:0/1076611407 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa414161060 0x7fa4140c8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:21.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.642+0000 7fa41b337640 1 --2- 172.21.15.49:0/1076611407 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa41406b590 0x7fa4140c9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:21.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.642+0000 7fa41b337640 1 -- 172.21.15.49:0/1076611407 >> 172.21.15.49:0/1076611407 conn(0x7fa414089fd0 msgr2=0x7fa41415f380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:21.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.642+0000 7fa41b337640 1 -- 172.21.15.49:0/1076611407 shutdown_connections 2023-11-06T02:13:21.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.642+0000 7fa41b337640 1 -- 172.21.15.49:0/1076611407 wait complete. 2023-11-06T02:13:21.654 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-06T02:13:21.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.778+0000 7f9d35815640 1 Processor -- start 2023-11-06T02:13:21.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.778+0000 7f9d35815640 1 -- start start 2023-11-06T02:13:21.777 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.778+0000 7f9d35815640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9d300cbc10 0x7f9d30071120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:21.777 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.778+0000 7f9d35815640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f9d300b5290 0x7f9d300ceb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:21.778 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.778+0000 7f9d35815640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d3006b1b0 con 0x7f9d300b4d40 2023-11-06T02:13:21.778 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.778+0000 7f9d35815640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d300cbff0 con 0x7f9d300b5290 2023-11-06T02:13:21.778 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.778+0000 7f9d35815640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d3006b490 con 0x7f9d300cbc10 2023-11-06T02:13:21.778 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.782+0000 7f9d2e7fc640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f9d300b5290 0x7f9d300ceb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:21.778 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.782+0000 7f9d2f7fe640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f9d300b4d40 0x7f9d3015d850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:46796/0 (socket says 172.21.15.49:46796) 2023-11-06T02:13:21.779 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.782+0000 7f9d2f7fe640 1 -- 172.21.15.49:0/1787489515 learned_addr learned my addr 172.21.15.49:0/1787489515 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:13:21.779 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.782+0000 7f9d2effd640 1 --2- 172.21.15.49:0/1787489515 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9d300cbc10 0x7f9d30071120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:21.779 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.782+0000 7f9d2dffb640 1 -- 172.21.15.49:0/1787489515 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 904823654 0 0) 0x7f9d3006b1b0 con 0x7f9d300b4d40 2023-11-06T02:13:21.779 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.782+0000 7f9d2dffb640 1 -- 172.21.15.49:0/1787489515 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d18003530 con 0x7f9d300b4d40 2023-11-06T02:13:21.779 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.782+0000 7f9d2e7fc640 1 -- 172.21.15.49:0/1787489515 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9d300cbc10 msgr2=0x7f9d30071120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:21.780 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.782+0000 7f9d2e7fc640 1 --2- 172.21.15.49:0/1787489515 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9d300cbc10 0x7f9d30071120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:21.780 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.782+0000 7f9d2e7fc640 1 -- 172.21.15.49:0/1787489515 >> v1:172.21.15.49:6789/0 conn(0x7f9d300b4d40 legacy=0x7f9d3015d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:21.780 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.782+0000 7f9d2e7fc640 1 -- 172.21.15.49:0/1787489515 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d3006b770 con 0x7f9d300b5290 2023-11-06T02:13:21.783 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.782+0000 7f9d2e7fc640 1 --2- 172.21.15.49:0/1787489515 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d300b5290 0x7f9d300ceb90 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f9d200097c0 tx=0x7f9d2001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2efdde487e220df5 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:21.783 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.782+0000 7f9d2dffb640 1 -- 172.21.15.49:0/1787489515 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d200ad640 con 0x7f9d300b5290 2023-11-06T02:13:21.784 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.782+0000 7f9d2dffb640 1 -- 172.21.15.49:0/1787489515 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d200ad7a0 con 0x7f9d300b5290 2023-11-06T02:13:21.784 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.782+0000 7f9d2dffb640 1 -- 172.21.15.49:0/1787489515 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d200adb30 con 0x7f9d300b5290 2023-11-06T02:13:21.784 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.782+0000 7f9d35815640 1 -- 172.21.15.49:0/1787489515 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d300b5290 msgr2=0x7f9d300ceb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:21.784 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.782+0000 7f9d35815640 1 --2- 172.21.15.49:0/1787489515 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d300b5290 0x7f9d300ceb90 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f9d200097c0 tx=0x7f9d2001c480 comp rx=0 tx=0).stop 2023-11-06T02:13:21.784 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.782+0000 7f9d35815640 1 -- 172.21.15.49:0/1787489515 shutdown_connections 2023-11-06T02:13:21.785 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.782+0000 7f9d35815640 1 --2- 172.21.15.49:0/1787489515 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d300b5290 0x7f9d300ceb90 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:21.785 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.782+0000 7f9d35815640 1 --2- 172.21.15.49:0/1787489515 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9d300cbc10 0x7f9d30071120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:21.785 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.782+0000 7f9d35815640 1 -- 172.21.15.49:0/1787489515 >> 172.21.15.49:0/1787489515 conn(0x7f9d300cc140 msgr2=0x7f9d300cc550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:21.785 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.782+0000 7f9d35815640 1 -- 172.21.15.49:0/1787489515 shutdown_connections 2023-11-06T02:13:21.785 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.782+0000 7f9d35815640 1 -- 172.21.15.49:0/1787489515 wait complete. 2023-11-06T02:13:21.786 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.786+0000 7f9d35815640 1 Processor -- start 2023-11-06T02:13:21.786 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.786+0000 7f9d35815640 1 -- start start 2023-11-06T02:13:21.786 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.786+0000 7f9d35815640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d300b4d40 0x7f9d3014edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:21.786 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.786+0000 7f9d35815640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d300b5290 0x7f9d300cc520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:21.787 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.786+0000 7f9d35815640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9d300cbc10 0x7f9d300cca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:21.787 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.786+0000 7f9d35815640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d30153c30 con 0x7f9d300b5290 2023-11-06T02:13:21.787 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.786+0000 7f9d35815640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d30154c90 con 0x7f9d300b4d40 2023-11-06T02:13:21.787 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.786+0000 7f9d35815640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d30153ed0 con 0x7f9d300cbc10 2023-11-06T02:13:21.787 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.786+0000 7f9d2e7fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d300b5290 0x7f9d300cc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:21.788 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.786+0000 7f9d2e7fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d300b5290 0x7f9d300cc520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48780/0 (socket says 172.21.15.49:48780) 2023-11-06T02:13:21.788 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.786+0000 7f9d2e7fc640 1 -- 172.21.15.49:0/3366693541 learned_addr learned my addr 172.21.15.49:0/3366693541 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:21.788 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.786+0000 7f9d2e7fc640 1 -- 172.21.15.49:0/3366693541 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9d300cbc10 msgr2=0x7f9d300cca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:21.788 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.786+0000 7f9d2e7fc640 1 --2- 172.21.15.49:0/3366693541 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9d300cbc10 0x7f9d300cca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:21.789 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.786+0000 7f9d2e7fc640 1 -- 172.21.15.49:0/3366693541 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d300b4d40 msgr2=0x7f9d3014edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:21.789 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.786+0000 7f9d2e7fc640 1 --2- 172.21.15.49:0/3366693541 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d300b4d40 0x7f9d3014edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:21.789 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.786+0000 7f9d2e7fc640 1 -- 172.21.15.49:0/3366693541 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d20009420 con 0x7f9d300b5290 2023-11-06T02:13:21.789 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.786+0000 7f9d2e7fc640 1 --2- 172.21.15.49:0/3366693541 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d300b5290 0x7f9d300cc520 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f9d200b0040 tx=0x7f9d20019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:21.789 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.786+0000 7f9d17fff640 1 -- 172.21.15.49:0/3366693541 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d200b6610 con 0x7f9d300b5290 2023-11-06T02:13:21.790 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.786+0000 7f9d17fff640 1 -- 172.21.15.49:0/3366693541 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d200a6040 con 0x7f9d300b5290 2023-11-06T02:13:21.790 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.786+0000 7f9d17fff640 1 -- 172.21.15.49:0/3366693541 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d200b69b0 con 0x7f9d300b5290 2023-11-06T02:13:21.790 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.786+0000 7f9d35815640 1 -- 172.21.15.49:0/3366693541 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d301541b0 con 0x7f9d300b5290 2023-11-06T02:13:21.790 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.786+0000 7f9d35815640 1 -- 172.21.15.49:0/3366693541 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d30154770 con 0x7f9d300b5290 2023-11-06T02:13:21.792 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.786+0000 7f9d35815640 1 -- 172.21.15.49:0/3366693541 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d30154960 con 0x7f9d300b5290 2023-11-06T02:13:21.792 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.790+0000 7f9d17fff640 1 -- 172.21.15.49:0/3366693541 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f9d200b4040 con 0x7f9d300b5290 2023-11-06T02:13:21.792 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.790+0000 7f9d17fff640 1 --2- 172.21.15.49:0/3366693541 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9d1c065a60 0x7f9d1c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:21.792 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.790+0000 7f9d17fff640 1 -- 172.21.15.49:0/3366693541 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6350+0+0 (secure 0 0 0) 0x7f9d2011d530 con 0x7f9d300b5290 2023-11-06T02:13:21.792 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.790+0000 7f9d2effd640 1 --2- 172.21.15.49:0/3366693541 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9d1c065a60 0x7f9d1c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:21.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.790+0000 7f9d2effd640 1 --2- 172.21.15.49:0/3366693541 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9d1c065a60 0x7f9d1c067f20 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f9d2400b850 tx=0x7f9d24008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:21.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.794+0000 7f9d17fff640 1 -- 172.21.15.49:0/3366693541 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f9d200eac70 con 0x7f9d300b5290 2023-11-06T02:13:21.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:21.982+0000 7f9d35815640 1 -- 172.21.15.49:0/3366693541 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f9d30067140 con 0x7f9d300b5290 2023-11-06T02:13:22.129 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T02:13:22.132 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T02:13:22.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:22.358+0000 7f9d17fff640 1 -- 172.21.15.49:0/3366693541 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v39) v1 ==== 161+0+0 (secure 0 0 0) 0x7f9d200ee290 con 0x7f9d300b5290 2023-11-06T02:13:22.355 INFO:teuthology.orchestra.run.smithi049.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-06T02:13:22.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:22.362+0000 7f9d35815640 1 -- 172.21.15.49:0/3366693541 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9d1c065a60 msgr2=0x7f9d1c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:22.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:22.362+0000 7f9d35815640 1 --2- 172.21.15.49:0/3366693541 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9d1c065a60 0x7f9d1c067f20 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f9d2400b850 tx=0x7f9d24008040 comp rx=0 tx=0).stop 2023-11-06T02:13:22.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:22.362+0000 7f9d35815640 1 -- 172.21.15.49:0/3366693541 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d300b5290 msgr2=0x7f9d300cc520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:22.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:22.362+0000 7f9d35815640 1 --2- 172.21.15.49:0/3366693541 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d300b5290 0x7f9d300cc520 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f9d200b0040 tx=0x7f9d20019220 comp rx=0 tx=0).stop 2023-11-06T02:13:22.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:22.362+0000 7f9d35815640 1 -- 172.21.15.49:0/3366693541 shutdown_connections 2023-11-06T02:13:22.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:22.362+0000 7f9d35815640 1 --2- 172.21.15.49:0/3366693541 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9d1c065a60 0x7f9d1c067f20 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:22.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:22.362+0000 7f9d35815640 1 --2- 172.21.15.49:0/3366693541 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d300b5290 0x7f9d300cc520 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:22.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:22.362+0000 7f9d35815640 1 --2- 172.21.15.49:0/3366693541 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9d300cbc10 0x7f9d300cca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:22.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:22.362+0000 7f9d35815640 1 --2- 172.21.15.49:0/3366693541 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d300b4d40 0x7f9d3014edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:22.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:22.362+0000 7f9d35815640 1 -- 172.21.15.49:0/3366693541 >> 172.21.15.49:0/3366693541 conn(0x7f9d300cc140 msgr2=0x7f9d3015b310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:22.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:22.362+0000 7f9d35815640 1 -- 172.21.15.49:0/3366693541 shutdown_connections 2023-11-06T02:13:22.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:22.362+0000 7f9d35815640 1 -- 172.21.15.49:0/3366693541 wait complete. 2023-11-06T02:13:22.383 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-06T02:13:22.515 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:22.518+0000 7f094cbb3640 1 Processor -- start 2023-11-06T02:13:22.516 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:22.518+0000 7f094cbb3640 1 -- start start 2023-11-06T02:13:22.517 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:22.522+0000 7f094cbb3640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f09480afe20 0x7f09480ce9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:22.517 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:22.522+0000 7f094cbb3640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f09480d0fe0 0x7f09480b0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:22.517 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:22.522+0000 7f094cbb3640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09480cf340 con 0x7f094806bda0 2023-11-06T02:13:22.518 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:22.522+0000 7f094cbb3640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09480cf620 con 0x7f09480afe20 2023-11-06T02:13:22.518 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:22.522+0000 7f094cbb3640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09480cf900 con 0x7f09480d0fe0 2023-11-06T02:13:22.518 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:22.522+0000 7f094659c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f09480afe20 0x7f09480ce9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:22.518 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:22.522+0000 7f094659c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f09480afe20 0x7f09480ce9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48784/0 (socket says 172.21.15.49:48784) 2023-11-06T02:13:22.519 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:22.522+0000 7f094659c640 1 -- 172.21.15.49:0/409962098 learned_addr learned my addr 172.21.15.49:0/409962098 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:22.519 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:22.522+0000 7f0946d9d640 1 --2- 172.21.15.49:0/409962098 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f09480d0fe0 0x7f09480b0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:22.519 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:22.522+0000 7f094659c640 1 -- 172.21.15.49:0/409962098 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f09480d0fe0 msgr2=0x7f09480b0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:22.519 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:22.522+0000 7f094659c640 1 --2- 172.21.15.49:0/409962098 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f09480d0fe0 0x7f09480b0a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:22.519 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:22.522+0000 7f094659c640 1 -- 172.21.15.49:0/409962098 >> v1:172.21.15.150:6789/0 conn(0x7f094806bda0 legacy=0x7f09480d08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:22.519 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:22.522+0000 7f094659c640 1 -- 172.21.15.49:0/409962098 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09480cfbe0 con 0x7f09480afe20 2023-11-06T02:13:22.521 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:22.522+0000 7f094659c640 1 --2- 172.21.15.49:0/409962098 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09480afe20 0x7f09480ce9f0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f09480ce420 tx=0x7f093801c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0236f206c203df server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:22.521 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:22.522+0000 7f094559a640 1 -- 172.21.15.49:0/409962098 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09480cf140 con 0x7f09480afe20 2023-11-06T02:13:22.521 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:22.522+0000 7f094559a640 1 -- 172.21.15.49:0/409962098 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09380ad640 con 0x7f09480afe20 2023-11-06T02:13:22.521 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:22.522+0000 7f094559a640 1 -- 172.21.15.49:0/409962098 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09380ad9d0 con 0x7f09480afe20 2023-11-06T02:13:22.521 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:22.522+0000 7f094cbb3640 1 -- 172.21.15.49:0/409962098 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09480afe20 msgr2=0x7f09480ce9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:22.522 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:22.522+0000 7f094cbb3640 1 --2- 172.21.15.49:0/409962098 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09480afe20 0x7f09480ce9f0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f09480ce420 tx=0x7f093801c430 comp rx=0 tx=0).stop 2023-11-06T02:13:22.522 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:22.522+0000 7f094cbb3640 1 -- 172.21.15.49:0/409962098 shutdown_connections 2023-11-06T02:13:22.522 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:22.522+0000 7f094cbb3640 1 --2- 172.21.15.49:0/409962098 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f09480d0fe0 0x7f09480b0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:22.522 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:22.522+0000 7f094cbb3640 1 --2- 172.21.15.49:0/409962098 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09480afe20 0x7f09480ce9f0 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:22.522 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:22.522+0000 7f094cbb3640 1 -- 172.21.15.49:0/409962098 >> 172.21.15.49:0/409962098 conn(0x7f0948089fd0 msgr2=0x7f0948099c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:22.523 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:22.522+0000 7f094cbb3640 1 -- 172.21.15.49:0/409962098 shutdown_connections 2023-11-06T02:13:22.523 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:22.522+0000 7f094cbb3640 1 -- 172.21.15.49:0/409962098 wait complete. 2023-11-06T02:13:22.525 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:22.526+0000 7f094cbb3640 1 Processor -- start 2023-11-06T02:13:22.526 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:22.526+0000 7f094cbb3640 1 -- start start 2023-11-06T02:13:22.526 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:22.526+0000 7f094cbb3640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f094806bda0 0x7f094814f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:22.526 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:22.526+0000 7f094cbb3640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09480afe20 0x7f09480c80c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:22.526 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:22.526+0000 7f094cbb3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09480d0fe0 0x7f09480c8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:22.527 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:22.526+0000 7f094cbb3640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0948158a00 con 0x7f09480d0fe0 2023-11-06T02:13:22.527 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:22.526+0000 7f094cbb3640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f094814fd40 con 0x7f09480afe20 2023-11-06T02:13:22.527 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:22.526+0000 7f094cbb3640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09481580d0 con 0x7f094806bda0 2023-11-06T02:13:22.527 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:22.526+0000 7f0946d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09480d0fe0 0x7f09480c8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:22.527 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:22.526+0000 7f0946d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09480d0fe0 0x7f09480c8600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48794/0 (socket says 172.21.15.49:48794) 2023-11-06T02:13:22.527 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:22.526+0000 7f0946d9d640 1 -- 172.21.15.49:0/426390537 learned_addr learned my addr 172.21.15.49:0/426390537 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:22.528 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:22.526+0000 7f094659c640 1 --2- 172.21.15.49:0/426390537 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f094806bda0 0x7f094814f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:22.528 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:22.526+0000 7f0946d9d640 1 -- 172.21.15.49:0/426390537 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f094806bda0 msgr2=0x7f094814f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:22.528 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:22.526+0000 7f0945d9b640 1 --2- 172.21.15.49:0/426390537 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09480afe20 0x7f09480c80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:22.528 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:22.526+0000 7f0946d9d640 1 --2- 172.21.15.49:0/426390537 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f094806bda0 0x7f094814f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:22.528 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:22.526+0000 7f0946d9d640 1 -- 172.21.15.49:0/426390537 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09480afe20 msgr2=0x7f09480c80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:22.529 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:22.526+0000 7f0946d9d640 1 --2- 172.21.15.49:0/426390537 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09480afe20 0x7f09480c80c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:22.529 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:22.526+0000 7f0946d9d640 1 -- 172.21.15.49:0/426390537 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09380094d0 con 0x7f09480d0fe0 2023-11-06T02:13:22.529 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:22.526+0000 7f0945d9b640 1 --2- 172.21.15.49:0/426390537 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09480afe20 0x7f09480c80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:13:22.529 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:22.526+0000 7f0946d9d640 1 --2- 172.21.15.49:0/426390537 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09480d0fe0 0x7f09480c8600 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f094000cd40 tx=0x7f094000a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:22.529 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:22.526+0000 7f0936ffd640 1 -- 172.21.15.49:0/426390537 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0940003ba0 con 0x7f09480d0fe0 2023-11-06T02:13:22.529 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:22.526+0000 7f0936ffd640 1 -- 172.21.15.49:0/426390537 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0940003d00 con 0x7f09480d0fe0 2023-11-06T02:13:22.530 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:22.526+0000 7f094cbb3640 1 -- 172.21.15.49:0/426390537 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09481583b0 con 0x7f09480d0fe0 2023-11-06T02:13:22.530 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:22.526+0000 7f0936ffd640 1 -- 172.21.15.49:0/426390537 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0940004240 con 0x7f09480d0fe0 2023-11-06T02:13:22.530 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:22.526+0000 7f094cbb3640 1 -- 172.21.15.49:0/426390537 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09480c9070 con 0x7f09480d0fe0 2023-11-06T02:13:22.530 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:22.526+0000 7f094cbb3640 1 -- 172.21.15.49:0/426390537 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0948158b70 con 0x7f09480d0fe0 2023-11-06T02:13:22.530 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:22.530+0000 7f0936ffd640 1 -- 172.21.15.49:0/426390537 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f09400043a0 con 0x7f09480d0fe0 2023-11-06T02:13:22.531 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:22.530+0000 7f0936ffd640 1 --2- 172.21.15.49:0/426390537 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0928065a60 0x7f0928067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:22.531 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:22.530+0000 7f0936ffd640 1 -- 172.21.15.49:0/426390537 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6350+0+0 (secure 0 0 0) 0x7f0940012070 con 0x7f09480d0fe0 2023-11-06T02:13:22.531 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:22.530+0000 7f094659c640 1 --2- 172.21.15.49:0/426390537 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0928065a60 0x7f0928067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:22.532 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:22.530+0000 7f094659c640 1 --2- 172.21.15.49:0/426390537 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0928065a60 0x7f0928067f20 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f09380af000 tx=0x7f0938008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:22.532 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:22.534+0000 7f0936ffd640 1 -- 172.21.15.49:0/426390537 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f094008a080 con 0x7f09480d0fe0 2023-11-06T02:13:22.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:22.730+0000 7f094cbb3640 1 -- 172.21.15.49:0/426390537 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f0948067140 con 0x7f09480d0fe0 2023-11-06T02:13:23.365 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:23.370+0000 7f0936ffd640 1 -- 172.21.15.49:0/426390537 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v40) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0940051500 con 0x7f09480d0fe0 2023-11-06T02:13:23.370 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:23.370+0000 7f094cbb3640 1 -- 172.21.15.49:0/426390537 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0928065a60 msgr2=0x7f0928067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:23.370 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:23.370+0000 7f094cbb3640 1 --2- 172.21.15.49:0/426390537 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0928065a60 0x7f0928067f20 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f09380af000 tx=0x7f0938008040 comp rx=0 tx=0).stop 2023-11-06T02:13:23.370 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:23.370+0000 7f094cbb3640 1 -- 172.21.15.49:0/426390537 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09480d0fe0 msgr2=0x7f09480c8600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:23.370 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:23.370+0000 7f094cbb3640 1 --2- 172.21.15.49:0/426390537 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09480d0fe0 0x7f09480c8600 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f094000cd40 tx=0x7f094000a660 comp rx=0 tx=0).stop 2023-11-06T02:13:23.371 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:23.370+0000 7f094cbb3640 1 -- 172.21.15.49:0/426390537 shutdown_connections 2023-11-06T02:13:23.371 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:23.370+0000 7f094cbb3640 1 --2- 172.21.15.49:0/426390537 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0928065a60 0x7f0928067f20 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:23.371 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:23.370+0000 7f094cbb3640 1 --2- 172.21.15.49:0/426390537 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09480d0fe0 0x7f09480c8600 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:23.371 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:23.370+0000 7f094cbb3640 1 --2- 172.21.15.49:0/426390537 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09480afe20 0x7f09480c80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:23.371 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:23.370+0000 7f094cbb3640 1 --2- 172.21.15.49:0/426390537 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f094806bda0 0x7f094814f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:23.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:23.370+0000 7f094cbb3640 1 -- 172.21.15.49:0/426390537 >> 172.21.15.49:0/426390537 conn(0x7f0948089fd0 msgr2=0x7f094806fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:23.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:23.374+0000 7f094cbb3640 1 -- 172.21.15.49:0/426390537 shutdown_connections 2023-11-06T02:13:23.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:23.374+0000 7f094cbb3640 1 -- 172.21.15.49:0/426390537 wait complete. 2023-11-06T02:13:23.392 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-06T02:13:23.526 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:23.530+0000 7fe9fd005640 1 Processor -- start 2023-11-06T02:13:23.527 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:23.530+0000 7fe9fd005640 1 -- start start 2023-11-06T02:13:23.528 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:23.530+0000 7fe9fd005640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe9f8069fd0 0x7fe9f80caa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:23.528 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:23.530+0000 7fe9fd005640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fe9f80ad720 0x7fe9f8159530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:23.528 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:23.530+0000 7fe9fd005640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9f8071e80 con 0x7fe9f806a3b0 2023-11-06T02:13:23.528 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:23.530+0000 7fe9fd005640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9f8072160 con 0x7fe9f8069fd0 2023-11-06T02:13:23.529 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:23.530+0000 7fe9fd005640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9f80ca220 con 0x7fe9f80ad720 2023-11-06T02:13:23.529 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:23.530+0000 7fe9f6d9d640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe9f8069fd0 0x7fe9f80caa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:23.529 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:23.530+0000 7fe9f6d9d640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe9f8069fd0 0x7fe9f80caa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48806/0 (socket says 172.21.15.49:48806) 2023-11-06T02:13:23.529 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:23.530+0000 7fe9f6d9d640 1 -- 172.21.15.49:0/2561403214 learned_addr learned my addr 172.21.15.49:0/2561403214 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:23.529 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:23.530+0000 7fe9f759e640 1 --2- 172.21.15.49:0/2561403214 >> v2:172.21.15.150:3300/0 conn(0x7fe9f80ad720 0x7fe9f8159530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:23.530 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:23.530+0000 7fe9f6d9d640 1 -- 172.21.15.49:0/2561403214 >> v2:172.21.15.150:3300/0 conn(0x7fe9f80ad720 msgr2=0x7fe9f8159530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:23.530 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:23.530+0000 7fe9f6d9d640 1 --2- 172.21.15.49:0/2561403214 >> v2:172.21.15.150:3300/0 conn(0x7fe9f80ad720 0x7fe9f8159530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:23.530 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:23.530+0000 7fe9f6d9d640 1 -- 172.21.15.49:0/2561403214 >> v1:172.21.15.49:6789/0 conn(0x7fe9f806a3b0 legacy=0x7fe9f80715c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:23.530 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:23.530+0000 7fe9f6d9d640 1 -- 172.21.15.49:0/2561403214 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9f80ca500 con 0x7fe9f8069fd0 2023-11-06T02:13:23.531 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:23.534+0000 7fe9f6d9d640 1 --2- 172.21.15.49:0/2561403214 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9f8069fd0 0x7fe9f80caa20 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fe9f8071510 tx=0x7fe9e801c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ff3ad2d6e0eba14 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:23.532 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:23.534+0000 7fe9f5d9b640 1 -- 172.21.15.49:0/2561403214 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e80ab610 con 0x7fe9f8069fd0 2023-11-06T02:13:23.532 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:23.534+0000 7fe9f5d9b640 1 -- 172.21.15.49:0/2561403214 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9e80ab770 con 0x7fe9f8069fd0 2023-11-06T02:13:23.532 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:23.534+0000 7fe9f5d9b640 1 -- 172.21.15.49:0/2561403214 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e80abb00 con 0x7fe9f8069fd0 2023-11-06T02:13:23.532 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:23.534+0000 7fe9fd005640 1 -- 172.21.15.49:0/2561403214 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9f8069fd0 msgr2=0x7fe9f80caa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:23.532 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:23.534+0000 7fe9fd005640 1 --2- 172.21.15.49:0/2561403214 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9f8069fd0 0x7fe9f80caa20 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fe9f8071510 tx=0x7fe9e801c500 comp rx=0 tx=0).stop 2023-11-06T02:13:23.533 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:23.534+0000 7fe9fd005640 1 -- 172.21.15.49:0/2561403214 shutdown_connections 2023-11-06T02:13:23.533 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:23.534+0000 7fe9fd005640 1 --2- 172.21.15.49:0/2561403214 >> v2:172.21.15.150:3300/0 conn(0x7fe9f80ad720 0x7fe9f8159530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:23.533 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:23.534+0000 7fe9fd005640 1 --2- 172.21.15.49:0/2561403214 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9f8069fd0 0x7fe9f80caa20 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:23.533 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:23.534+0000 7fe9fd005640 1 -- 172.21.15.49:0/2561403214 >> 172.21.15.49:0/2561403214 conn(0x7fe9f80ce610 msgr2=0x7fe9f80cea20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:23.533 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:23.534+0000 7fe9fd005640 1 -- 172.21.15.49:0/2561403214 shutdown_connections 2023-11-06T02:13:23.534 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:23.534+0000 7fe9fd005640 1 -- 172.21.15.49:0/2561403214 wait complete. 2023-11-06T02:13:23.537 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:23.534+0000 7fe9fd005640 1 Processor -- start 2023-11-06T02:13:23.537 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:23.538+0000 7fe9fd005640 1 -- start start 2023-11-06T02:13:23.537 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:23.538+0000 7fe9fd005640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe9f8069fd0 0x7fe9f80c2540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:23.537 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:23.538+0000 7fe9fd005640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe9f806a3b0 0x7fe9f8153470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:23.538 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:23.538+0000 7fe9fd005640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9f80ad720 0x7fe9f81539b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:23.538 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:23.538+0000 7fe9fd005640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f814c220 con 0x7fe9f80ad720 2023-11-06T02:13:23.538 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:23.538+0000 7fe9fd005640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f80c45a0 con 0x7fe9f8069fd0 2023-11-06T02:13:23.538 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:23.538+0000 7fe9fd005640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f80c4880 con 0x7fe9f806a3b0 2023-11-06T02:13:23.538 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:23.538+0000 7fe9f759e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9f80ad720 0x7fe9f81539b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:23.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:23.538+0000 7fe9f759e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9f80ad720 0x7fe9f81539b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48814/0 (socket says 172.21.15.49:48814) 2023-11-06T02:13:23.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:23.538+0000 7fe9f759e640 1 -- 172.21.15.49:0/541663477 learned_addr learned my addr 172.21.15.49:0/541663477 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:23.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:23.538+0000 7fe9f759e640 1 -- 172.21.15.49:0/541663477 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe9f806a3b0 msgr2=0x7fe9f8153470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:23.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:23.538+0000 7fe9f659c640 1 --2- 172.21.15.49:0/541663477 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe9f806a3b0 0x7fe9f8153470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:23.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:23.538+0000 7fe9f6d9d640 1 --2- 172.21.15.49:0/541663477 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe9f8069fd0 0x7fe9f80c2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:23.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:23.538+0000 7fe9f759e640 1 --2- 172.21.15.49:0/541663477 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe9f806a3b0 0x7fe9f8153470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:23.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:23.538+0000 7fe9f759e640 1 -- 172.21.15.49:0/541663477 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe9f8069fd0 msgr2=0x7fe9f80c2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:23.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:23.538+0000 7fe9f759e640 1 --2- 172.21.15.49:0/541663477 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe9f8069fd0 0x7fe9f80c2540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:23.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:23.538+0000 7fe9f759e640 1 -- 172.21.15.49:0/541663477 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9e80094d0 con 0x7fe9f80ad720 2023-11-06T02:13:23.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:23.538+0000 7fe9f759e640 1 --2- 172.21.15.49:0/541663477 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9f80ad720 0x7fe9f81539b0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fe9f0007e60 tx=0x7fe9f000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:23.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:23.538+0000 7fe9ef7fe640 1 -- 172.21.15.49:0/541663477 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f0014420 con 0x7fe9f80ad720 2023-11-06T02:13:23.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:23.538+0000 7fe9ef7fe640 1 -- 172.21.15.49:0/541663477 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9f00145a0 con 0x7fe9f80ad720 2023-11-06T02:13:23.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:23.538+0000 7fe9fd005640 1 -- 172.21.15.49:0/541663477 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9f80c4b60 con 0x7fe9f80ad720 2023-11-06T02:13:23.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:23.538+0000 7fe9ef7fe640 1 -- 172.21.15.49:0/541663477 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f0014950 con 0x7fe9f80ad720 2023-11-06T02:13:23.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:23.538+0000 7fe9fd005640 1 -- 172.21.15.49:0/541663477 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9f80c2a80 con 0x7fe9f80ad720 2023-11-06T02:13:23.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:23.538+0000 7fe9fd005640 1 -- 172.21.15.49:0/541663477 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9f814c3d0 con 0x7fe9f80ad720 2023-11-06T02:13:23.542 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:23.538+0000 7fe9ef7fe640 1 -- 172.21.15.49:0/541663477 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fe9f0014ab0 con 0x7fe9f80ad720 2023-11-06T02:13:23.542 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:23.538+0000 7fe9ef7fe640 1 --2- 172.21.15.49:0/541663477 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe9d8065a60 0x7fe9d8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:23.542 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:23.538+0000 7fe9ef7fe640 1 -- 172.21.15.49:0/541663477 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6350+0+0 (secure 0 0 0) 0x7fe9f0083c80 con 0x7fe9f80ad720 2023-11-06T02:13:23.542 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:23.538+0000 7fe9f6d9d640 1 --2- 172.21.15.49:0/541663477 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe9d8065a60 0x7fe9d8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:23.542 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:23.542+0000 7fe9f6d9d640 1 --2- 172.21.15.49:0/541663477 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe9d8065a60 0x7fe9d8067f20 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fe9e80af000 tx=0x7fe9e8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:23.543 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:23.542+0000 7fe9ef7fe640 1 -- 172.21.15.49:0/541663477 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fe9f814c3d0 con 0x7fe9f80ad720 2023-11-06T02:13:23.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:23.698+0000 7fe9fd005640 1 -- 172.21.15.49:0/541663477 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe9f8074040 con 0x7fe9f80ad720 2023-11-06T02:13:24.391 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:24.394+0000 7fe9ef7fe640 1 -- 172.21.15.49:0/541663477 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v70) v1 ==== 181+0+0 (secure 0 0 0) 0x7fe9f0051450 con 0x7fe9f80ad720 2023-11-06T02:13:24.393 INFO:teuthology.orchestra.run.smithi049.stderr:pool 'cephfs_metadata' removed 2023-11-06T02:13:24.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:24.398+0000 7fe9fd005640 1 -- 172.21.15.49:0/541663477 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe9d8065a60 msgr2=0x7fe9d8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:24.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:24.398+0000 7fe9fd005640 1 --2- 172.21.15.49:0/541663477 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe9d8065a60 0x7fe9d8067f20 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fe9e80af000 tx=0x7fe9e8008040 comp rx=0 tx=0).stop 2023-11-06T02:13:24.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:24.398+0000 7fe9fd005640 1 -- 172.21.15.49:0/541663477 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9f80ad720 msgr2=0x7fe9f81539b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:24.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:24.398+0000 7fe9fd005640 1 --2- 172.21.15.49:0/541663477 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9f80ad720 0x7fe9f81539b0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fe9f0007e60 tx=0x7fe9f000e4b0 comp rx=0 tx=0).stop 2023-11-06T02:13:24.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:24.402+0000 7fe9fd005640 1 -- 172.21.15.49:0/541663477 shutdown_connections 2023-11-06T02:13:24.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:24.402+0000 7fe9fd005640 1 --2- 172.21.15.49:0/541663477 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe9d8065a60 0x7fe9d8067f20 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:24.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:24.402+0000 7fe9fd005640 1 --2- 172.21.15.49:0/541663477 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9f80ad720 0x7fe9f81539b0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:24.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:24.402+0000 7fe9fd005640 1 --2- 172.21.15.49:0/541663477 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe9f806a3b0 0x7fe9f8153470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:24.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:24.402+0000 7fe9fd005640 1 --2- 172.21.15.49:0/541663477 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe9f8069fd0 0x7fe9f80c2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:24.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:24.402+0000 7fe9fd005640 1 -- 172.21.15.49:0/541663477 >> 172.21.15.49:0/541663477 conn(0x7fe9f80ce610 msgr2=0x7fe9f80cb8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:24.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:24.402+0000 7fe9fd005640 1 -- 172.21.15.49:0/541663477 shutdown_connections 2023-11-06T02:13:24.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:24.402+0000 7fe9fd005640 1 -- 172.21.15.49:0/541663477 wait complete. 2023-11-06T02:13:24.422 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-06T02:13:24.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:24.562+0000 7ff54e2f6640 1 Processor -- start 2023-11-06T02:13:24.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:24.562+0000 7ff54e2f6640 1 -- start start 2023-11-06T02:13:24.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:24.562+0000 7ff54e2f6640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff548069fd0 0x7ff5480caa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:24.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:24.562+0000 7ff54e2f6640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7ff54806a3b0 0x7ff5480715c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:24.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:24.562+0000 7ff54e2f6640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff548071e80 con 0x7ff5480ad720 2023-11-06T02:13:24.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:24.562+0000 7ff54e2f6640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff548072160 con 0x7ff548069fd0 2023-11-06T02:13:24.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:24.562+0000 7ff54e2f6640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5480ca220 con 0x7ff54806a3b0 2023-11-06T02:13:24.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:24.562+0000 7ff547fff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff548069fd0 0x7ff5480caa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:24.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:24.562+0000 7ff547fff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff548069fd0 0x7ff5480caa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48822/0 (socket says 172.21.15.49:48822) 2023-11-06T02:13:24.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:24.562+0000 7ff54c893640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7ff5480ad720 0x7ff548159530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:46814/0 (socket says 172.21.15.49:46814) 2023-11-06T02:13:24.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:24.562+0000 7ff547fff640 1 -- 172.21.15.49:0/4037329090 learned_addr learned my addr 172.21.15.49:0/4037329090 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:24.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:24.562+0000 7ff547fff640 1 -- 172.21.15.49:0/4037329090 >> v2:172.21.15.150:3300/0 conn(0x7ff54806a3b0 msgr2=0x7ff5480715c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:24.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:24.562+0000 7ff5477fe640 1 --2- 172.21.15.49:0/4037329090 >> v2:172.21.15.150:3300/0 conn(0x7ff54806a3b0 0x7ff5480715c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:24.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:24.562+0000 7ff547fff640 1 --2- 172.21.15.49:0/4037329090 >> v2:172.21.15.150:3300/0 conn(0x7ff54806a3b0 0x7ff5480715c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:24.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:24.562+0000 7ff547fff640 1 -- 172.21.15.49:0/4037329090 >> v1:172.21.15.49:6789/0 conn(0x7ff5480ad720 legacy=0x7ff548159530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:24.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:24.562+0000 7ff547fff640 1 -- 172.21.15.49:0/4037329090 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5480ca500 con 0x7ff548069fd0 2023-11-06T02:13:24.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:24.562+0000 7ff546ffd640 1 -- 172.21.15.49:0/4037329090 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3385945785 0 0) 0x7ff548071e80 con 0x7ff5480ad720 2023-11-06T02:13:24.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:24.562+0000 7ff5477fe640 1 --2- 172.21.15.49:0/4037329090 >> v2:172.21.15.150:3300/0 conn(0x7ff54806a3b0 0x7ff5480715c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:13:24.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:24.566+0000 7ff547fff640 1 --2- 172.21.15.49:0/4037329090 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff548069fd0 0x7ff5480caa20 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7ff5480cd5e0 tx=0x7ff53801c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a61da573b03ea578 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:24.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:24.566+0000 7ff546ffd640 1 -- 172.21.15.49:0/4037329090 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5380aa680 con 0x7ff548069fd0 2023-11-06T02:13:24.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:24.566+0000 7ff546ffd640 1 -- 172.21.15.49:0/4037329090 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5380aa7e0 con 0x7ff548069fd0 2023-11-06T02:13:24.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:24.566+0000 7ff546ffd640 1 -- 172.21.15.49:0/4037329090 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5380aab70 con 0x7ff548069fd0 2023-11-06T02:13:24.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:24.566+0000 7ff54e2f6640 1 -- 172.21.15.49:0/4037329090 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff548069fd0 msgr2=0x7ff5480caa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:24.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:24.566+0000 7ff54e2f6640 1 --2- 172.21.15.49:0/4037329090 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff548069fd0 0x7ff5480caa20 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7ff5480cd5e0 tx=0x7ff53801c500 comp rx=0 tx=0).stop 2023-11-06T02:13:24.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:24.566+0000 7ff54e2f6640 1 -- 172.21.15.49:0/4037329090 shutdown_connections 2023-11-06T02:13:24.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:24.566+0000 7ff54e2f6640 1 --2- 172.21.15.49:0/4037329090 >> v2:172.21.15.150:3300/0 conn(0x7ff54806a3b0 0x7ff5480715c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:24.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:24.566+0000 7ff54e2f6640 1 --2- 172.21.15.49:0/4037329090 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff548069fd0 0x7ff5480caa20 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:24.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:24.566+0000 7ff54e2f6640 1 -- 172.21.15.49:0/4037329090 >> 172.21.15.49:0/4037329090 conn(0x7ff5480ce610 msgr2=0x7ff5480cea20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:24.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:24.566+0000 7ff54e2f6640 1 -- 172.21.15.49:0/4037329090 shutdown_connections 2023-11-06T02:13:24.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:24.566+0000 7ff54e2f6640 1 -- 172.21.15.49:0/4037329090 wait complete. 2023-11-06T02:13:24.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:24.566+0000 7ff54e2f6640 1 Processor -- start 2023-11-06T02:13:24.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:24.566+0000 7ff54e2f6640 1 -- start start 2023-11-06T02:13:24.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:24.566+0000 7ff54e2f6640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff548069fd0 0x7ff548153fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:24.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:24.566+0000 7ff54e2f6640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff54806a3b0 0x7ff54815aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:24.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:24.566+0000 7ff54e2f6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff5480ad720 0x7ff548158600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:24.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:24.566+0000 7ff54e2f6640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54814e460 con 0x7ff5480ad720 2023-11-06T02:13:24.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:24.566+0000 7ff54e2f6640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54814e740 con 0x7ff548069fd0 2023-11-06T02:13:24.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:24.566+0000 7ff54e2f6640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff54814ea20 con 0x7ff54806a3b0 2023-11-06T02:13:24.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:24.566+0000 7ff54c893640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff5480ad720 0x7ff548158600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:24.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:24.566+0000 7ff54c893640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff5480ad720 0x7ff548158600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48836/0 (socket says 172.21.15.49:48836) 2023-11-06T02:13:24.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:24.566+0000 7ff54c893640 1 -- 172.21.15.49:0/1267046137 learned_addr learned my addr 172.21.15.49:0/1267046137 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:24.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:24.566+0000 7ff5477fe640 1 --2- 172.21.15.49:0/1267046137 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff54806a3b0 0x7ff54815aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:24.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:24.566+0000 7ff547fff640 1 --2- 172.21.15.49:0/1267046137 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff548069fd0 0x7ff548153fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:24.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:24.566+0000 7ff54c893640 1 -- 172.21.15.49:0/1267046137 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff54806a3b0 msgr2=0x7ff54815aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:24.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:24.566+0000 7ff54c893640 1 --2- 172.21.15.49:0/1267046137 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff54806a3b0 0x7ff54815aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:24.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:24.566+0000 7ff54c893640 1 -- 172.21.15.49:0/1267046137 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff548069fd0 msgr2=0x7ff548153fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:24.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:24.566+0000 7ff54c893640 1 --2- 172.21.15.49:0/1267046137 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff548069fd0 0x7ff548153fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:24.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:24.566+0000 7ff54c893640 1 -- 172.21.15.49:0/1267046137 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5380094d0 con 0x7ff5480ad720 2023-11-06T02:13:24.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:24.566+0000 7ff5477fe640 1 --2- 172.21.15.49:0/1267046137 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff54806a3b0 0x7ff54815aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:13:24.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:24.566+0000 7ff547fff640 1 --2- 172.21.15.49:0/1267046137 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff548069fd0 0x7ff548153fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:13:24.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:24.566+0000 7ff54c893640 1 --2- 172.21.15.49:0/1267046137 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff5480ad720 0x7ff548158600 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7ff54000c780 tx=0x7ff54000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:24.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:24.570+0000 7ff544ff9640 1 -- 172.21.15.49:0/1267046137 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff54000ce90 con 0x7ff5480ad720 2023-11-06T02:13:24.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:24.570+0000 7ff544ff9640 1 -- 172.21.15.49:0/1267046137 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff54000a640 con 0x7ff5480ad720 2023-11-06T02:13:24.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:24.570+0000 7ff544ff9640 1 -- 172.21.15.49:0/1267046137 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff54000a9d0 con 0x7ff5480ad720 2023-11-06T02:13:24.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:24.570+0000 7ff54e2f6640 1 -- 172.21.15.49:0/1267046137 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff54814ed00 con 0x7ff5480ad720 2023-11-06T02:13:24.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:24.570+0000 7ff54e2f6640 1 -- 172.21.15.49:0/1267046137 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff54814f500 con 0x7ff5480ad720 2023-11-06T02:13:24.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:24.570+0000 7ff54e2f6640 1 -- 172.21.15.49:0/1267046137 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff54814f220 con 0x7ff5480ad720 2023-11-06T02:13:24.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:24.570+0000 7ff544ff9640 1 -- 172.21.15.49:0/1267046137 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7ff54000ab30 con 0x7ff5480ad720 2023-11-06T02:13:24.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:24.570+0000 7ff544ff9640 1 --2- 172.21.15.49:0/1267046137 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff534065a60 0x7ff534067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:24.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:24.570+0000 7ff544ff9640 1 -- 172.21.15.49:0/1267046137 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5883+0+0 (secure 0 0 0) 0x7ff5400827a0 con 0x7ff5480ad720 2023-11-06T02:13:24.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:24.570+0000 7ff547fff640 1 --2- 172.21.15.49:0/1267046137 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff534065a60 0x7ff534067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:24.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:24.570+0000 7ff547fff640 1 --2- 172.21.15.49:0/1267046137 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff534065a60 0x7ff534067f20 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7ff5380ae000 tx=0x7ff538008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:24.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:24.574+0000 7ff544ff9640 1 -- 172.21.15.49:0/1267046137 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7ff54814f220 con 0x7ff5480ad720 2023-11-06T02:13:24.722 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:24.726+0000 7ff54e2f6640 1 -- 172.21.15.49:0/1267046137 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff54814f220 con 0x7ff5480ad720 2023-11-06T02:13:25.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.410+0000 7ff544ff9640 1 -- 172.21.15.49:0/1267046137 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v71) v1 ==== 169+0+0 (secure 0 0 0) 0x7ff54814f220 con 0x7ff5480ad720 2023-11-06T02:13:25.406 INFO:teuthology.orchestra.run.smithi049.stderr:pool 'cephfs_data' removed 2023-11-06T02:13:25.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.414+0000 7ff54e2f6640 1 -- 172.21.15.49:0/1267046137 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff534065a60 msgr2=0x7ff534067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:25.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.414+0000 7ff54e2f6640 1 --2- 172.21.15.49:0/1267046137 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff534065a60 0x7ff534067f20 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7ff5380ae000 tx=0x7ff538008040 comp rx=0 tx=0).stop 2023-11-06T02:13:25.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.414+0000 7ff54e2f6640 1 -- 172.21.15.49:0/1267046137 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff5480ad720 msgr2=0x7ff548158600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:25.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.414+0000 7ff54e2f6640 1 --2- 172.21.15.49:0/1267046137 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff5480ad720 0x7ff548158600 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7ff54000c780 tx=0x7ff54000cc40 comp rx=0 tx=0).stop 2023-11-06T02:13:25.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.414+0000 7ff54e2f6640 1 -- 172.21.15.49:0/1267046137 shutdown_connections 2023-11-06T02:13:25.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.414+0000 7ff54e2f6640 1 --2- 172.21.15.49:0/1267046137 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff534065a60 0x7ff534067f20 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:25.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.414+0000 7ff54e2f6640 1 --2- 172.21.15.49:0/1267046137 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff5480ad720 0x7ff548158600 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:25.411 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.414+0000 7ff54e2f6640 1 --2- 172.21.15.49:0/1267046137 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff54806a3b0 0x7ff54815aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:25.411 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.414+0000 7ff54e2f6640 1 --2- 172.21.15.49:0/1267046137 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff548069fd0 0x7ff548153fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:25.411 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.414+0000 7ff54e2f6640 1 -- 172.21.15.49:0/1267046137 >> 172.21.15.49:0/1267046137 conn(0x7ff5480ce610 msgr2=0x7ff5480cd860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:25.411 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.414+0000 7ff54e2f6640 1 -- 172.21.15.49:0/1267046137 shutdown_connections 2023-11-06T02:13:25.411 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.414+0000 7ff54e2f6640 1 -- 172.21.15.49:0/1267046137 wait complete. 2023-11-06T02:13:25.435 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-06T02:13:25.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.570+0000 7f94a5757640 1 Processor -- start 2023-11-06T02:13:25.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.570+0000 7f94a5757640 1 -- start start 2023-11-06T02:13:25.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.570+0000 7f94a5757640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f94a006b2f0 0x7f94a00b1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:25.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.570+0000 7f94a5757640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f94a00b0810 0x7f94a015c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:25.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.570+0000 7f94a5757640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94a00b1990 con 0x7f94a015cd40 2023-11-06T02:13:25.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.570+0000 7f94a5757640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94a015d190 con 0x7f94a006b2f0 2023-11-06T02:13:25.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.570+0000 7f94a5757640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94a00b0d80 con 0x7f94a00b0810 2023-11-06T02:13:25.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.570+0000 7f949effd640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f94a006b2f0 0x7f94a00b1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:25.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.570+0000 7f949effd640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f94a006b2f0 0x7f94a00b1e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48852/0 (socket says 172.21.15.49:48852) 2023-11-06T02:13:25.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.570+0000 7f949effd640 1 -- 172.21.15.49:0/4253880346 learned_addr learned my addr 172.21.15.49:0/4253880346 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:25.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.570+0000 7f949e7fc640 1 --2- 172.21.15.49:0/4253880346 >> v2:172.21.15.150:3300/0 conn(0x7f94a00b0810 0x7f94a015c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:25.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.570+0000 7f949effd640 1 -- 172.21.15.49:0/4253880346 >> v2:172.21.15.150:3300/0 conn(0x7f94a00b0810 msgr2=0x7f94a015c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:25.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.570+0000 7f949effd640 1 --2- 172.21.15.49:0/4253880346 >> v2:172.21.15.150:3300/0 conn(0x7f94a00b0810 0x7f94a015c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:25.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.570+0000 7f949effd640 1 -- 172.21.15.49:0/4253880346 >> v1:172.21.15.49:6789/0 conn(0x7f94a015cd40 legacy=0x7f94a006b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:25.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.570+0000 7f949dffb640 1 -- 172.21.15.49:0/4253880346 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 342016302 0 0) 0x7f94a00b1990 con 0x7f94a015cd40 2023-11-06T02:13:25.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.570+0000 7f949effd640 1 -- 172.21.15.49:0/4253880346 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94a00b1060 con 0x7f94a006b2f0 2023-11-06T02:13:25.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.574+0000 7f949effd640 1 --2- 172.21.15.49:0/4253880346 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f94a006b2f0 0x7f94a00b1e20 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f94a0160810 tx=0x7f949001ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f976ec57f75673c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:25.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.574+0000 7f949dffb640 1 -- 172.21.15.49:0/4253880346 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94900a89f0 con 0x7f94a006b2f0 2023-11-06T02:13:25.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.574+0000 7f949dffb640 1 -- 172.21.15.49:0/4253880346 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94900a8b50 con 0x7f94a006b2f0 2023-11-06T02:13:25.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.574+0000 7f949dffb640 1 -- 172.21.15.49:0/4253880346 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94900b0440 con 0x7f94a006b2f0 2023-11-06T02:13:25.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.574+0000 7f94a5757640 1 -- 172.21.15.49:0/4253880346 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f94a006b2f0 msgr2=0x7f94a00b1e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:25.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.574+0000 7f94a5757640 1 --2- 172.21.15.49:0/4253880346 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f94a006b2f0 0x7f94a00b1e20 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f94a0160810 tx=0x7f949001ecf0 comp rx=0 tx=0).stop 2023-11-06T02:13:25.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.574+0000 7f94a5757640 1 -- 172.21.15.49:0/4253880346 shutdown_connections 2023-11-06T02:13:25.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.574+0000 7f94a5757640 1 --2- 172.21.15.49:0/4253880346 >> v2:172.21.15.150:3300/0 conn(0x7f94a00b0810 0x7f94a015c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:25.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.574+0000 7f94a5757640 1 --2- 172.21.15.49:0/4253880346 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f94a006b2f0 0x7f94a00b1e20 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:25.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.574+0000 7f94a5757640 1 -- 172.21.15.49:0/4253880346 >> 172.21.15.49:0/4253880346 conn(0x7f94a00d08d0 msgr2=0x7f94a00d0cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:25.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.574+0000 7f94a5757640 1 -- 172.21.15.49:0/4253880346 shutdown_connections 2023-11-06T02:13:25.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.574+0000 7f94a5757640 1 -- 172.21.15.49:0/4253880346 wait complete. 2023-11-06T02:13:25.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.574+0000 7f94a5757640 1 Processor -- start 2023-11-06T02:13:25.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.574+0000 7f94a5757640 1 -- start start 2023-11-06T02:13:25.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.578+0000 7f94a5757640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f94a006b2f0 0x7f94a00b5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:25.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.578+0000 7f94a5757640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f94a00b0810 0x7f94a006fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:25.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.578+0000 7f94a5757640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f94a015cd40 0x7f94a0158500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:25.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.578+0000 7f94a5757640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a00b6330 con 0x7f94a015cd40 2023-11-06T02:13:25.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.578+0000 7f94a5757640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a00b35b0 con 0x7f94a006b2f0 2023-11-06T02:13:25.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.578+0000 7f94a5757640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94a00b3890 con 0x7f94a00b0810 2023-11-06T02:13:25.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.578+0000 7f949f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f94a015cd40 0x7f94a0158500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:25.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.578+0000 7f949f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f94a015cd40 0x7f94a0158500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48856/0 (socket says 172.21.15.49:48856) 2023-11-06T02:13:25.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.578+0000 7f949f7fe640 1 -- 172.21.15.49:0/2424964668 learned_addr learned my addr 172.21.15.49:0/2424964668 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:25.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.578+0000 7f949e7fc640 1 --2- 172.21.15.49:0/2424964668 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f94a00b0810 0x7f94a006fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:25.579 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.578+0000 7f949f7fe640 1 -- 172.21.15.49:0/2424964668 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f94a00b0810 msgr2=0x7f94a006fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:25.579 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.578+0000 7f949f7fe640 1 --2- 172.21.15.49:0/2424964668 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f94a00b0810 0x7f94a006fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:25.579 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.578+0000 7f949f7fe640 1 -- 172.21.15.49:0/2424964668 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f94a006b2f0 msgr2=0x7f94a00b5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:25.579 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.578+0000 7f949f7fe640 1 --2- 172.21.15.49:0/2424964668 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f94a006b2f0 0x7f94a00b5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:25.579 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.578+0000 7f949f7fe640 1 -- 172.21.15.49:0/2424964668 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9490009480 con 0x7f94a015cd40 2023-11-06T02:13:25.580 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.578+0000 7f949f7fe640 1 --2- 172.21.15.49:0/2424964668 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f94a015cd40 0x7f94a0158500 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f949800c750 tx=0x7f949800cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:25.580 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.578+0000 7f94877fe640 1 -- 172.21.15.49:0/2424964668 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9498003bb0 con 0x7f94a015cd40 2023-11-06T02:13:25.580 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.578+0000 7f94877fe640 1 -- 172.21.15.49:0/2424964668 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9498003d10 con 0x7f94a015cd40 2023-11-06T02:13:25.580 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.578+0000 7f94a5757640 1 -- 172.21.15.49:0/2424964668 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94a00b3b70 con 0x7f94a015cd40 2023-11-06T02:13:25.580 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.578+0000 7f94877fe640 1 -- 172.21.15.49:0/2424964668 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9498005270 con 0x7f94a015cd40 2023-11-06T02:13:25.581 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.578+0000 7f94a5757640 1 -- 172.21.15.49:0/2424964668 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94a0158a40 con 0x7f94a015cd40 2023-11-06T02:13:25.581 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.578+0000 7f94a5757640 1 -- 172.21.15.49:0/2424964668 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94a0074040 con 0x7f94a015cd40 2023-11-06T02:13:25.581 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.578+0000 7f94877fe640 1 -- 172.21.15.49:0/2424964668 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f94980053d0 con 0x7f94a015cd40 2023-11-06T02:13:25.581 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.578+0000 7f94877fe640 1 --2- 172.21.15.49:0/2424964668 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f948c065a60 0x7f948c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:25.581 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.578+0000 7f94877fe640 1 -- 172.21.15.49:0/2424964668 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f9498082700 con 0x7f94a015cd40 2023-11-06T02:13:25.582 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.578+0000 7f949effd640 1 --2- 172.21.15.49:0/2424964668 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f948c065a60 0x7f948c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:25.582 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.582+0000 7f949effd640 1 --2- 172.21.15.49:0/2424964668 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f948c065a60 0x7f948c067f20 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f94900b2000 tx=0x7f9490008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:25.583 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.582+0000 7f94877fe640 1 -- 172.21.15.49:0/2424964668 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f94980502b0 con 0x7f94a015cd40 2023-11-06T02:13:25.742 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.746+0000 7f94a5757640 1 -- 172.21.15.49:0/2424964668 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f94a00b4020 con 0x7f94a015cd40 2023-11-06T02:13:25.746 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.750+0000 7f94877fe640 1 -- 172.21.15.49:0/2424964668 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7f94980538d0 con 0x7f94a015cd40 2023-11-06T02:13:25.749 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.754+0000 7f94a5757640 1 -- 172.21.15.49:0/2424964668 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f948c065a60 msgr2=0x7f948c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:25.749 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.754+0000 7f94a5757640 1 --2- 172.21.15.49:0/2424964668 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f948c065a60 0x7f948c067f20 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f94900b2000 tx=0x7f9490008040 comp rx=0 tx=0).stop 2023-11-06T02:13:25.749 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.754+0000 7f94a5757640 1 -- 172.21.15.49:0/2424964668 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f94a015cd40 msgr2=0x7f94a0158500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:25.750 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.754+0000 7f94a5757640 1 --2- 172.21.15.49:0/2424964668 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f94a015cd40 0x7f94a0158500 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f949800c750 tx=0x7f949800cc10 comp rx=0 tx=0).stop 2023-11-06T02:13:25.750 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.754+0000 7f94a5757640 1 -- 172.21.15.49:0/2424964668 shutdown_connections 2023-11-06T02:13:25.750 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.754+0000 7f94a5757640 1 --2- 172.21.15.49:0/2424964668 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f948c065a60 0x7f948c067f20 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:25.750 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.754+0000 7f94a5757640 1 --2- 172.21.15.49:0/2424964668 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f94a015cd40 0x7f94a0158500 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:25.750 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.754+0000 7f94a5757640 1 --2- 172.21.15.49:0/2424964668 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f94a00b0810 0x7f94a006fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:25.751 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.754+0000 7f94a5757640 1 --2- 172.21.15.49:0/2424964668 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f94a006b2f0 0x7f94a00b5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:25.751 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.754+0000 7f94a5757640 1 -- 172.21.15.49:0/2424964668 >> 172.21.15.49:0/2424964668 conn(0x7f94a00d08d0 msgr2=0x7f94a006b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:25.751 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.754+0000 7f94a5757640 1 -- 172.21.15.49:0/2424964668 shutdown_connections 2023-11-06T02:13:25.751 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.754+0000 7f94a5757640 1 -- 172.21.15.49:0/2424964668 wait complete. 2023-11-06T02:13:25.766 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync' 2023-11-06T02:13:25.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.886+0000 7fe22f84a640 1 Processor -- start 2023-11-06T02:13:25.885 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.886+0000 7fe22f84a640 1 -- start start 2023-11-06T02:13:25.886 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.890+0000 7fe22f84a640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe2281590f0 0x7fe2280c8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:25.886 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.890+0000 7fe22f84a640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fe2280adae0 0x7fe2280b0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:25.887 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.890+0000 7fe22f84a640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2281594d0 con 0x7fe2280b05c0 2023-11-06T02:13:25.887 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.890+0000 7fe22f84a640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe22815a140 con 0x7fe2281590f0 2023-11-06T02:13:25.887 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.890+0000 7fe22f84a640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2280c92e0 con 0x7fe2280adae0 2023-11-06T02:13:25.887 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.890+0000 7fe22d5e6640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe2281590f0 0x7fe2280c8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:25.888 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.890+0000 7fe22d5e6640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe2281590f0 0x7fe2280c8c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48862/0 (socket says 172.21.15.49:48862) 2023-11-06T02:13:25.888 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.890+0000 7fe22d5e6640 1 -- 172.21.15.49:0/50632768 learned_addr learned my addr 172.21.15.49:0/50632768 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:25.888 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.890+0000 7fe22cde5640 1 --2- 172.21.15.49:0/50632768 >> v2:172.21.15.150:3300/0 conn(0x7fe2280adae0 0x7fe2280b0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:25.888 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.890+0000 7fe22d5e6640 1 -- 172.21.15.49:0/50632768 >> v2:172.21.15.150:3300/0 conn(0x7fe2280adae0 msgr2=0x7fe2280b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:25.888 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.890+0000 7fe22d5e6640 1 --2- 172.21.15.49:0/50632768 >> v2:172.21.15.150:3300/0 conn(0x7fe2280adae0 0x7fe2280b0080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:25.889 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.890+0000 7fe22d5e6640 1 -- 172.21.15.49:0/50632768 >> v1:172.21.15.150:6789/0 conn(0x7fe2280b05c0 legacy=0x7fe22815cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:25.889 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.890+0000 7fe22d5e6640 1 -- 172.21.15.49:0/50632768 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2281597a0 con 0x7fe2281590f0 2023-11-06T02:13:25.890 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.894+0000 7fe22d5e6640 1 --2- 172.21.15.49:0/50632768 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2281590f0 0x7fe2280c8c50 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fe22806adc0 tx=0x7fe22401eb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa4d2322b3498b59 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:25.890 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.894+0000 7fe21ffff640 1 -- 172.21.15.49:0/50632768 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2240a7950 con 0x7fe2281590f0 2023-11-06T02:13:25.890 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.894+0000 7fe21ffff640 1 -- 172.21.15.49:0/50632768 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2240a7ab0 con 0x7fe2281590f0 2023-11-06T02:13:25.891 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.894+0000 7fe21ffff640 1 -- 172.21.15.49:0/50632768 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2240a7e40 con 0x7fe2281590f0 2023-11-06T02:13:25.891 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.894+0000 7fe22f84a640 1 -- 172.21.15.49:0/50632768 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2281590f0 msgr2=0x7fe2280c8c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:25.891 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.894+0000 7fe22f84a640 1 --2- 172.21.15.49:0/50632768 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2281590f0 0x7fe2280c8c50 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fe22806adc0 tx=0x7fe22401eb00 comp rx=0 tx=0).stop 2023-11-06T02:13:25.891 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.894+0000 7fe22f84a640 1 -- 172.21.15.49:0/50632768 shutdown_connections 2023-11-06T02:13:25.891 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.894+0000 7fe22f84a640 1 --2- 172.21.15.49:0/50632768 >> v2:172.21.15.150:3300/0 conn(0x7fe2280adae0 0x7fe2280b0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:25.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.894+0000 7fe22f84a640 1 --2- 172.21.15.49:0/50632768 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2281590f0 0x7fe2280c8c50 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:25.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.894+0000 7fe22f84a640 1 -- 172.21.15.49:0/50632768 >> 172.21.15.49:0/50632768 conn(0x7fe22806c0a0 msgr2=0x7fe22806c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:25.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.894+0000 7fe22f84a640 1 -- 172.21.15.49:0/50632768 shutdown_connections 2023-11-06T02:13:25.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.894+0000 7fe22f84a640 1 -- 172.21.15.49:0/50632768 wait complete. 2023-11-06T02:13:25.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.894+0000 7fe22f84a640 1 Processor -- start 2023-11-06T02:13:25.895 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.894+0000 7fe22f84a640 1 -- start start 2023-11-06T02:13:25.895 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.894+0000 7fe22f84a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe2280adae0 0x7fe22814bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:25.895 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.894+0000 7fe22f84a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2280b05c0 0x7fe22814da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:25.896 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.894+0000 7fe22f84a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe2281590f0 0x7fe228151350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:25.896 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.894+0000 7fe22f84a640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe22814c110 con 0x7fe2280b05c0 2023-11-06T02:13:25.896 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.894+0000 7fe22f84a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe22814d580 con 0x7fe2281590f0 2023-11-06T02:13:25.896 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.894+0000 7fe22f84a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe22814c9b0 con 0x7fe2280adae0 2023-11-06T02:13:25.896 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.894+0000 7fe22cde5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2280b05c0 0x7fe22814da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:25.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.894+0000 7fe22cde5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2280b05c0 0x7fe22814da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48876/0 (socket says 172.21.15.49:48876) 2023-11-06T02:13:25.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.894+0000 7fe22cde5640 1 -- 172.21.15.49:0/1574723689 learned_addr learned my addr 172.21.15.49:0/1574723689 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:25.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.898+0000 7fe22d5e6640 1 --2- 172.21.15.49:0/1574723689 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe2280adae0 0x7fe22814bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:25.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.898+0000 7fe22dde7640 1 --2- 172.21.15.49:0/1574723689 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe2281590f0 0x7fe228151350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:25.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.898+0000 7fe22cde5640 1 -- 172.21.15.49:0/1574723689 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe2280adae0 msgr2=0x7fe22814bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:25.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.898+0000 7fe22cde5640 1 --2- 172.21.15.49:0/1574723689 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe2280adae0 0x7fe22814bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:25.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.898+0000 7fe22cde5640 1 -- 172.21.15.49:0/1574723689 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe2281590f0 msgr2=0x7fe228151350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:25.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.898+0000 7fe22cde5640 1 --2- 172.21.15.49:0/1574723689 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe2281590f0 0x7fe228151350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:25.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.898+0000 7fe22cde5640 1 -- 172.21.15.49:0/1574723689 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe22400b720 con 0x7fe2280b05c0 2023-11-06T02:13:25.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.898+0000 7fe22d5e6640 1 --2- 172.21.15.49:0/1574723689 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe2280adae0 0x7fe22814bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:13:25.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.898+0000 7fe22cde5640 1 --2- 172.21.15.49:0/1574723689 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2280b05c0 0x7fe22814da10 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fe21800d7d0 tx=0x7fe21800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:25.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.898+0000 7fe21dffb640 1 -- 172.21.15.49:0/1574723689 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe218018070 con 0x7fe2280b05c0 2023-11-06T02:13:25.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.898+0000 7fe21dffb640 1 -- 172.21.15.49:0/1574723689 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe218004020 con 0x7fe2280b05c0 2023-11-06T02:13:25.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.898+0000 7fe22f84a640 1 -- 172.21.15.49:0/1574723689 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe22814cc50 con 0x7fe2280b05c0 2023-11-06T02:13:25.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.898+0000 7fe21dffb640 1 -- 172.21.15.49:0/1574723689 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2180041d0 con 0x7fe2280b05c0 2023-11-06T02:13:25.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.898+0000 7fe22f84a640 1 -- 172.21.15.49:0/1574723689 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe22814d210 con 0x7fe2280b05c0 2023-11-06T02:13:25.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.898+0000 7fe22f84a640 1 -- 172.21.15.49:0/1574723689 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe22814d710 con 0x7fe2280b05c0 2023-11-06T02:13:25.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.898+0000 7fe21dffb640 1 -- 172.21.15.49:0/1574723689 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fe218004330 con 0x7fe2280b05c0 2023-11-06T02:13:25.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.898+0000 7fe21dffb640 1 --2- 172.21.15.49:0/1574723689 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe20c065a60 0x7fe20c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:25.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.898+0000 7fe21dffb640 1 -- 172.21.15.49:0/1574723689 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5382+0+0 (secure 0 0 0) 0x7fe218082e40 con 0x7fe2280b05c0 2023-11-06T02:13:25.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.898+0000 7fe22d5e6640 1 --2- 172.21.15.49:0/1574723689 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe20c065a60 0x7fe20c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:25.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.898+0000 7fe22d5e6640 1 --2- 172.21.15.49:0/1574723689 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe20c065a60 0x7fe20c067f20 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fe2240b1000 tx=0x7fe224008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:25.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:25.902+0000 7fe21dffb640 1 -- 172.21.15.49:0/1574723689 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fe2180509d0 con 0x7fe2280b05c0 2023-11-06T02:13:26.115 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:26.118+0000 7fe22f84a640 1 -- 172.21.15.49:0/1574723689 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]} v 0) v1 -- 0x7fe228074040 con 0x7fe2280b05c0 2023-11-06T02:13:26.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:26.406+0000 7fe21dffb640 1 -- 172.21.15.49:0/1574723689 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]}]=0 v249) v1 ==== 146+0+0 (secure 0 0 0) 0x7fe218053ff0 con 0x7fe2280b05c0 2023-11-06T02:13:26.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:26.414+0000 7fe22f84a640 1 -- 172.21.15.49:0/1574723689 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe20c065a60 msgr2=0x7fe20c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:26.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:26.414+0000 7fe22f84a640 1 --2- 172.21.15.49:0/1574723689 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe20c065a60 0x7fe20c067f20 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fe2240b1000 tx=0x7fe224008040 comp rx=0 tx=0).stop 2023-11-06T02:13:26.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:26.414+0000 7fe22f84a640 1 -- 172.21.15.49:0/1574723689 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2280b05c0 msgr2=0x7fe22814da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:26.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:26.414+0000 7fe22f84a640 1 --2- 172.21.15.49:0/1574723689 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2280b05c0 0x7fe22814da10 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fe21800d7d0 tx=0x7fe21800dc90 comp rx=0 tx=0).stop 2023-11-06T02:13:26.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:26.414+0000 7fe22f84a640 1 -- 172.21.15.49:0/1574723689 shutdown_connections 2023-11-06T02:13:26.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:26.414+0000 7fe22f84a640 1 --2- 172.21.15.49:0/1574723689 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe20c065a60 0x7fe20c067f20 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:26.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:26.414+0000 7fe22f84a640 1 --2- 172.21.15.49:0/1574723689 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe2280b05c0 0x7fe22814da10 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:26.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:26.414+0000 7fe22f84a640 1 --2- 172.21.15.49:0/1574723689 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe2281590f0 0x7fe228151350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:26.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:26.414+0000 7fe22f84a640 1 --2- 172.21.15.49:0/1574723689 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe2280adae0 0x7fe22814bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:26.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:26.414+0000 7fe22f84a640 1 -- 172.21.15.49:0/1574723689 >> 172.21.15.49:0/1574723689 conn(0x7fe22806c0a0 msgr2=0x7fe22806c480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:26.411 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:26.414+0000 7fe22f84a640 1 -- 172.21.15.49:0/1574723689 shutdown_connections 2023-11-06T02:13:26.411 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:26.414+0000 7fe22f84a640 1 -- 172.21.15.49:0/1574723689 wait complete. 2023-11-06T02:13:26.431 INFO:tasks.cephfs_test_runner:test_inotable_sync (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2023-11-06T02:13:26.431 INFO:tasks.cephfs_test_runner:Starting test: test_orphan_scan (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2023-11-06T02:13:26.432 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan' 2023-11-06T02:13:26.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:26.566+0000 7f8974748640 1 Processor -- start 2023-11-06T02:13:26.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:26.566+0000 7f8974748640 1 -- start start 2023-11-06T02:13:26.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:26.566+0000 7f8974748640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f896c15ab50 0x7f896c0c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:26.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:26.566+0000 7f8974748640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f896c0cb010 0x7f896c0b0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:26.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:26.566+0000 7f8974748640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f896c0cab80 con 0x7f896c0b05c0 2023-11-06T02:13:26.563 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:26.566+0000 7f8974748640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f896c0b0a50 con 0x7f896c15ab50 2023-11-06T02:13:26.563 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:26.566+0000 7f8974748640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f896c15af30 con 0x7f896c0cb010 2023-11-06T02:13:26.563 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:26.566+0000 7f8971ce3640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f896c0cb010 0x7f896c0b0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:26.563 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:26.566+0000 7f89724e4640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f896c15ab50 0x7f896c0c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:26.563 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:26.566+0000 7f89724e4640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f896c15ab50 0x7f896c0c9bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.49:52130/0 (socket says 172.21.15.49:52130) 2023-11-06T02:13:26.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:26.566+0000 7f89724e4640 1 -- 172.21.15.49:0/1940636266 learned_addr learned my addr 172.21.15.49:0/1940636266 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:26.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:26.566+0000 7f8972ce5640 1 --1- 172.21.15.49:0/1940636266 >> v1:172.21.15.150:6789/0 conn(0x7f896c0b05c0 0x7f896c0700a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.49:51938/0 (socket says 172.21.15.49:51938) 2023-11-06T02:13:26.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:26.566+0000 7f8971ce3640 1 -- 172.21.15.49:0/1940636266 >> v2:172.21.15.150:3300/0 conn(0x7f896c15ab50 msgr2=0x7f896c0c9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:26.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:26.566+0000 7f8971ce3640 1 --2- 172.21.15.49:0/1940636266 >> v2:172.21.15.150:3300/0 conn(0x7f896c15ab50 0x7f896c0c9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:26.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:26.566+0000 7f8971ce3640 1 -- 172.21.15.49:0/1940636266 >> v1:172.21.15.150:6789/0 conn(0x7f896c0b05c0 legacy=0x7f896c0700a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:26.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:26.566+0000 7f8971ce3640 1 -- 172.21.15.49:0/1940636266 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f896c0cb5b0 con 0x7f896c0cb010 2023-11-06T02:13:26.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:26.566+0000 7f89724e4640 1 --2- 172.21.15.49:0/1940636266 >> v2:172.21.15.150:3300/0 conn(0x7f896c15ab50 0x7f896c0c9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T02:13:26.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:26.570+0000 7f8971ce3640 1 --2- 172.21.15.49:0/1940636266 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f896c0cb010 0x7f896c0b0080 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8960009d90 tx=0x7f896001c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9b6e15ec4338e26 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:26.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:26.570+0000 7f89714e2640 1 -- 172.21.15.49:0/1940636266 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89600a59e0 con 0x7f896c0cb010 2023-11-06T02:13:26.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:26.570+0000 7f89714e2640 1 -- 172.21.15.49:0/1940636266 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89600a5b40 con 0x7f896c0cb010 2023-11-06T02:13:26.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:26.570+0000 7f89714e2640 1 -- 172.21.15.49:0/1940636266 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89600ad440 con 0x7f896c0cb010 2023-11-06T02:13:26.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:26.570+0000 7f8974748640 1 -- 172.21.15.49:0/1940636266 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f896c0cb010 msgr2=0x7f896c0b0080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:26.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:26.570+0000 7f8974748640 1 --2- 172.21.15.49:0/1940636266 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f896c0cb010 0x7f896c0b0080 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8960009d90 tx=0x7f896001c920 comp rx=0 tx=0).stop 2023-11-06T02:13:26.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:26.570+0000 7f8974748640 1 -- 172.21.15.49:0/1940636266 shutdown_connections 2023-11-06T02:13:26.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:26.570+0000 7f8974748640 1 --2- 172.21.15.49:0/1940636266 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f896c0cb010 0x7f896c0b0080 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:26.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:26.570+0000 7f8974748640 1 --2- 172.21.15.49:0/1940636266 >> v2:172.21.15.150:3300/0 conn(0x7f896c15ab50 0x7f896c0c9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:26.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:26.570+0000 7f8974748640 1 -- 172.21.15.49:0/1940636266 >> 172.21.15.49:0/1940636266 conn(0x7f896c06c0a0 msgr2=0x7f896c06c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:26.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:26.570+0000 7f8974748640 1 -- 172.21.15.49:0/1940636266 shutdown_connections 2023-11-06T02:13:26.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:26.570+0000 7f8974748640 1 -- 172.21.15.49:0/1940636266 wait complete. 2023-11-06T02:13:26.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:26.570+0000 7f8974748640 1 Processor -- start 2023-11-06T02:13:26.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:26.570+0000 7f8974748640 1 -- start start 2023-11-06T02:13:26.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:26.574+0000 7f8974748640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f896c0b05c0 0x7f896c0accb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:26.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:26.574+0000 7f8974748640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f896c0cb010 0x7f896c154710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:26.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:26.574+0000 7f8974748640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f896c15ab50 0x7f896c0c7060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:26.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:26.574+0000 7f8974748640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f896c0abc10 con 0x7f896c15ab50 2023-11-06T02:13:26.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:26.574+0000 7f8974748640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f896c0abef0 con 0x7f896c0b05c0 2023-11-06T02:13:26.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:26.574+0000 7f8974748640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f896c0ac1d0 con 0x7f896c0cb010 2023-11-06T02:13:26.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:26.574+0000 7f8972ce5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f896c15ab50 0x7f896c0c7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:26.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:26.574+0000 7f8972ce5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f896c15ab50 0x7f896c0c7060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48892/0 (socket says 172.21.15.49:48892) 2023-11-06T02:13:26.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:26.574+0000 7f8972ce5640 1 -- 172.21.15.49:0/1503703217 learned_addr learned my addr 172.21.15.49:0/1503703217 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:26.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:26.574+0000 7f89724e4640 1 --2- 172.21.15.49:0/1503703217 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f896c0b05c0 0x7f896c0accb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:26.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:26.574+0000 7f8971ce3640 1 --2- 172.21.15.49:0/1503703217 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f896c0cb010 0x7f896c154710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:26.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:26.574+0000 7f8972ce5640 1 -- 172.21.15.49:0/1503703217 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f896c0cb010 msgr2=0x7f896c154710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:26.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:26.574+0000 7f8972ce5640 1 --2- 172.21.15.49:0/1503703217 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f896c0cb010 0x7f896c154710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:26.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:26.574+0000 7f8972ce5640 1 -- 172.21.15.49:0/1503703217 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f896c0b05c0 msgr2=0x7f896c0accb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:26.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:26.574+0000 7f8972ce5640 1 --2- 172.21.15.49:0/1503703217 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f896c0b05c0 0x7f896c0accb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:26.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:26.574+0000 7f8972ce5640 1 -- 172.21.15.49:0/1503703217 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8964009480 con 0x7f896c15ab50 2023-11-06T02:13:26.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:26.574+0000 7f8972ce5640 1 --2- 172.21.15.49:0/1503703217 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f896c15ab50 0x7f896c0c7060 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f895c00c390 tx=0x7f895c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:26.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:26.574+0000 7f896affd640 1 -- 172.21.15.49:0/1503703217 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f895c0074e0 con 0x7f896c15ab50 2023-11-06T02:13:26.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:26.574+0000 7f896affd640 1 -- 172.21.15.49:0/1503703217 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f895c007640 con 0x7f896c15ab50 2023-11-06T02:13:26.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:26.574+0000 7f8974748640 1 -- 172.21.15.49:0/1503703217 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8960009a70 con 0x7f896c15ab50 2023-11-06T02:13:26.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:26.574+0000 7f896affd640 1 -- 172.21.15.49:0/1503703217 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f895c004a10 con 0x7f896c15ab50 2023-11-06T02:13:26.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:26.574+0000 7f8974748640 1 -- 172.21.15.49:0/1503703217 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f896c0ac790 con 0x7f896c15ab50 2023-11-06T02:13:26.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:26.574+0000 7f8974748640 1 -- 172.21.15.49:0/1503703217 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f896c0b0de0 con 0x7f896c15ab50 2023-11-06T02:13:26.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:26.574+0000 7f896affd640 1 -- 172.21.15.49:0/1503703217 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f895c004b70 con 0x7f896c15ab50 2023-11-06T02:13:26.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:26.574+0000 7f896affd640 1 --2- 172.21.15.49:0/1503703217 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8958065a90 0x7f8958067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:26.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:26.574+0000 7f896affd640 1 -- 172.21.15.49:0/1503703217 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f895c0828d0 con 0x7f896c15ab50 2023-11-06T02:13:26.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:26.574+0000 7f89724e4640 1 --2- 172.21.15.49:0/1503703217 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8958065a90 0x7f8958067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:26.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:26.578+0000 7f89724e4640 1 --2- 172.21.15.49:0/1503703217 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8958065a90 0x7f8958067f50 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f8964006fd0 tx=0x7f8964008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:26.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:26.578+0000 7f896affd640 1 -- 172.21.15.49:0/1503703217 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f895c050460 con 0x7f896c15ab50 2023-11-06T02:13:26.790 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:26.794+0000 7f8974748640 1 -- 172.21.15.49:0/1503703217 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]} v 0) v1 -- 0x7f896c0ac980 con 0x7f896c15ab50 2023-11-06T02:13:27.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:27.418+0000 7f896affd640 1 -- 172.21.15.49:0/1503703217 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]}]=0 v250) v1 ==== 147+0+0 (secure 0 0 0) 0x7f895c053a80 con 0x7f896c15ab50 2023-11-06T02:13:27.418 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:27.422+0000 7f8974748640 1 -- 172.21.15.49:0/1503703217 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8958065a90 msgr2=0x7f8958067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:27.419 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:27.422+0000 7f8974748640 1 --2- 172.21.15.49:0/1503703217 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8958065a90 0x7f8958067f50 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f8964006fd0 tx=0x7f8964008040 comp rx=0 tx=0).stop 2023-11-06T02:13:27.419 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:27.422+0000 7f8974748640 1 -- 172.21.15.49:0/1503703217 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f896c15ab50 msgr2=0x7f896c0c7060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:27.419 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:27.422+0000 7f8974748640 1 --2- 172.21.15.49:0/1503703217 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f896c15ab50 0x7f896c0c7060 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f895c00c390 tx=0x7f895c00c850 comp rx=0 tx=0).stop 2023-11-06T02:13:27.419 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:27.422+0000 7f8974748640 1 -- 172.21.15.49:0/1503703217 shutdown_connections 2023-11-06T02:13:27.419 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:27.422+0000 7f8974748640 1 --2- 172.21.15.49:0/1503703217 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8958065a90 0x7f8958067f50 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:27.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:27.422+0000 7f8974748640 1 --2- 172.21.15.49:0/1503703217 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f896c15ab50 0x7f896c0c7060 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:27.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:27.422+0000 7f8974748640 1 --2- 172.21.15.49:0/1503703217 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f896c0cb010 0x7f896c154710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:27.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:27.422+0000 7f8974748640 1 --2- 172.21.15.49:0/1503703217 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f896c0b05c0 0x7f896c0accb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:27.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:27.422+0000 7f8974748640 1 -- 172.21.15.49:0/1503703217 >> 172.21.15.49:0/1503703217 conn(0x7f896c06c0a0 msgr2=0x7f896c0ce570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:27.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:27.422+0000 7f8974748640 1 -- 172.21.15.49:0/1503703217 shutdown_connections 2023-11-06T02:13:27.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:27.422+0000 7f8974748640 1 -- 172.21.15.49:0/1503703217 wait complete. 2023-11-06T02:13:27.441 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-06T02:13:27.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:27.578+0000 7f1944402640 1 Processor -- start 2023-11-06T02:13:27.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:27.578+0000 7f1944402640 1 -- start start 2023-11-06T02:13:27.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:27.582+0000 7f1944402640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f193c158ea0 0x7f193c06d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:27.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:27.582+0000 7f1944402640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f193c06dc80 0x7f193c0ca500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:27.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:27.582+0000 7f1944402640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f193c06c230 con 0x7f193c0caa40 2023-11-06T02:13:27.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:27.582+0000 7f1944402640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f193c06ce20 con 0x7f193c158ea0 2023-11-06T02:13:27.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:27.582+0000 7f1944402640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f193c0caef0 con 0x7f193c06dc80 2023-11-06T02:13:27.579 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:27.582+0000 7f194199d640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f193c06dc80 0x7f193c0ca500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:27.579 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:27.582+0000 7f194199d640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f193c06dc80 0x7f193c0ca500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.49:49892/0 (socket says 172.21.15.49:49892) 2023-11-06T02:13:27.579 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:27.582+0000 7f194199d640 1 -- 172.21.15.49:0/3859933151 learned_addr learned my addr 172.21.15.49:0/3859933151 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:27.580 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:27.582+0000 7f194219e640 1 --2- 172.21.15.49:0/3859933151 >> v2:172.21.15.150:3300/0 conn(0x7f193c158ea0 0x7f193c06d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:27.580 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:27.582+0000 7f194199d640 1 -- 172.21.15.49:0/3859933151 >> v2:172.21.15.150:3300/0 conn(0x7f193c158ea0 msgr2=0x7f193c06d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:27.580 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:27.582+0000 7f194199d640 1 --2- 172.21.15.49:0/3859933151 >> v2:172.21.15.150:3300/0 conn(0x7f193c158ea0 0x7f193c06d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:27.580 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:27.582+0000 7f194199d640 1 -- 172.21.15.49:0/3859933151 >> v1:172.21.15.150:6789/0 conn(0x7f193c0caa40 legacy=0x7f193c15ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:27.580 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:27.582+0000 7f194199d640 1 -- 172.21.15.49:0/3859933151 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f193c06c480 con 0x7f193c06dc80 2023-11-06T02:13:27.581 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:27.586+0000 7f194199d640 1 --2- 172.21.15.49:0/3859933151 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f193c06dc80 0x7f193c0ca500 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f192c00bb10 tx=0x7f192c01eb50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70fc082ba10a5b36 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:27.582 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:27.586+0000 7f194119c640 1 -- 172.21.15.49:0/3859933151 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f192c0a7ba0 con 0x7f193c06dc80 2023-11-06T02:13:27.582 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:27.586+0000 7f194119c640 1 -- 172.21.15.49:0/3859933151 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f192c0a7d00 con 0x7f193c06dc80 2023-11-06T02:13:27.582 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:27.586+0000 7f194119c640 1 -- 172.21.15.49:0/3859933151 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f192c0af660 con 0x7f193c06dc80 2023-11-06T02:13:27.583 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:27.586+0000 7f1944402640 1 -- 172.21.15.49:0/3859933151 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f193c06dc80 msgr2=0x7f193c0ca500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:27.583 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:27.586+0000 7f1944402640 1 --2- 172.21.15.49:0/3859933151 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f193c06dc80 0x7f193c0ca500 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f192c00bb10 tx=0x7f192c01eb50 comp rx=0 tx=0).stop 2023-11-06T02:13:27.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:27.586+0000 7f1944402640 1 -- 172.21.15.49:0/3859933151 shutdown_connections 2023-11-06T02:13:27.585 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:27.586+0000 7f1944402640 1 --2- 172.21.15.49:0/3859933151 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f193c06dc80 0x7f193c0ca500 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:27.585 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:27.586+0000 7f1944402640 1 --2- 172.21.15.49:0/3859933151 >> v2:172.21.15.150:3300/0 conn(0x7f193c158ea0 0x7f193c06d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:27.585 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:27.586+0000 7f1944402640 1 -- 172.21.15.49:0/3859933151 >> 172.21.15.49:0/3859933151 conn(0x7f193c06a370 msgr2=0x7f193c0ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:27.585 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:27.586+0000 7f1944402640 1 -- 172.21.15.49:0/3859933151 shutdown_connections 2023-11-06T02:13:27.586 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:27.586+0000 7f1944402640 1 -- 172.21.15.49:0/3859933151 wait complete. 2023-11-06T02:13:27.588 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:27.586+0000 7f1944402640 1 Processor -- start 2023-11-06T02:13:27.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:27.586+0000 7f1944402640 1 -- start start 2023-11-06T02:13:27.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:27.590+0000 7f1944402640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f193c06dc80 0x7f193c147230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:27.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:27.590+0000 7f1944402640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f193c0caa40 0x7f193c0cbb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:27.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:27.590+0000 7f1944402640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f193c158ea0 0x7f193c157760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:27.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:27.590+0000 7f1944402640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f193c147ba0 con 0x7f193c06dc80 2023-11-06T02:13:27.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:27.590+0000 7f1944402640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f193c142c00 con 0x7f193c158ea0 2023-11-06T02:13:27.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:27.590+0000 7f1944402640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f193c14fcc0 con 0x7f193c0caa40 2023-11-06T02:13:27.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:27.590+0000 7f194219e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f193c06dc80 0x7f193c147230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:27.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:27.590+0000 7f194219e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f193c06dc80 0x7f193c147230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48894/0 (socket says 172.21.15.49:48894) 2023-11-06T02:13:27.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:27.590+0000 7f194219e640 1 -- 172.21.15.49:0/4130486345 learned_addr learned my addr 172.21.15.49:0/4130486345 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:27.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:27.590+0000 7f194199d640 1 --2- 172.21.15.49:0/4130486345 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f193c0caa40 0x7f193c0cbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:27.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:27.590+0000 7f194299f640 1 --2- 172.21.15.49:0/4130486345 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f193c158ea0 0x7f193c157760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:27.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:27.590+0000 7f194219e640 1 -- 172.21.15.49:0/4130486345 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f193c0caa40 msgr2=0x7f193c0cbb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:27.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:27.590+0000 7f194219e640 1 --2- 172.21.15.49:0/4130486345 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f193c0caa40 0x7f193c0cbb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:27.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:27.590+0000 7f194219e640 1 -- 172.21.15.49:0/4130486345 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f193c158ea0 msgr2=0x7f193c157760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:27.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:27.590+0000 7f194219e640 1 --2- 172.21.15.49:0/4130486345 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f193c158ea0 0x7f193c157760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:27.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:27.590+0000 7f194219e640 1 -- 172.21.15.49:0/4130486345 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f192c00b770 con 0x7f193c06dc80 2023-11-06T02:13:27.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:27.590+0000 7f194219e640 1 --2- 172.21.15.49:0/4130486345 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f193c06dc80 0x7f193c147230 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f193800e930 tx=0x7f193800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:27.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:27.590+0000 7f1932ffd640 1 -- 172.21.15.49:0/4130486345 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1938009690 con 0x7f193c06dc80 2023-11-06T02:13:27.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:27.590+0000 7f1932ffd640 1 -- 172.21.15.49:0/4130486345 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f193800ca00 con 0x7f193c06dc80 2023-11-06T02:13:27.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:27.590+0000 7f1944402640 1 -- 172.21.15.49:0/4130486345 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f193c0cc080 con 0x7f193c06dc80 2023-11-06T02:13:27.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:27.590+0000 7f1932ffd640 1 -- 172.21.15.49:0/4130486345 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193800cd70 con 0x7f193c06dc80 2023-11-06T02:13:27.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:27.590+0000 7f1944402640 1 -- 172.21.15.49:0/4130486345 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f193c14ff60 con 0x7f193c06dc80 2023-11-06T02:13:27.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:27.590+0000 7f1944402640 1 -- 172.21.15.49:0/4130486345 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f193c074040 con 0x7f193c06dc80 2023-11-06T02:13:27.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:27.590+0000 7f1932ffd640 1 -- 172.21.15.49:0/4130486345 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f1938013470 con 0x7f193c06dc80 2023-11-06T02:13:27.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:27.590+0000 7f1932ffd640 1 --2- 172.21.15.49:0/4130486345 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1920065a60 0x7f1920067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:27.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:27.594+0000 7f1932ffd640 1 -- 172.21.15.49:0/4130486345 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f1938083230 con 0x7f193c06dc80 2023-11-06T02:13:27.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:27.594+0000 7f194199d640 1 --2- 172.21.15.49:0/4130486345 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1920065a60 0x7f1920067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:27.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:27.594+0000 7f194199d640 1 --2- 172.21.15.49:0/4130486345 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1920065a60 0x7f1920067f20 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f192c0b1000 tx=0x7f192c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:27.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:27.598+0000 7f1932ffd640 1 -- 172.21.15.49:0/4130486345 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f1938050e90 con 0x7f193c06dc80 2023-11-06T02:13:27.747 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:27.750+0000 7f1944402640 1 -- 172.21.15.49:0/4130486345 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f193c070db0 con 0x7f193c06dc80 2023-11-06T02:13:27.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:27.758+0000 7f1932ffd640 1 -- 172.21.15.49:0/4130486345 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f19380544b0 con 0x7f193c06dc80 2023-11-06T02:13:27.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:27.762+0000 7f1944402640 1 -- 172.21.15.49:0/4130486345 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1920065a60 msgr2=0x7f1920067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:27.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:27.762+0000 7f1944402640 1 --2- 172.21.15.49:0/4130486345 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1920065a60 0x7f1920067f20 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f192c0b1000 tx=0x7f192c008040 comp rx=0 tx=0).stop 2023-11-06T02:13:27.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:27.762+0000 7f1944402640 1 -- 172.21.15.49:0/4130486345 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f193c06dc80 msgr2=0x7f193c147230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:27.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:27.762+0000 7f1944402640 1 --2- 172.21.15.49:0/4130486345 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f193c06dc80 0x7f193c147230 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f193800e930 tx=0x7f193800edf0 comp rx=0 tx=0).stop 2023-11-06T02:13:27.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:27.762+0000 7f1944402640 1 -- 172.21.15.49:0/4130486345 shutdown_connections 2023-11-06T02:13:27.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:27.762+0000 7f1944402640 1 --2- 172.21.15.49:0/4130486345 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1920065a60 0x7f1920067f20 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:27.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:27.762+0000 7f1944402640 1 --2- 172.21.15.49:0/4130486345 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f193c158ea0 0x7f193c157760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:27.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:27.762+0000 7f1944402640 1 --2- 172.21.15.49:0/4130486345 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f193c0caa40 0x7f193c0cbb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:27.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:27.762+0000 7f1944402640 1 --2- 172.21.15.49:0/4130486345 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f193c06dc80 0x7f193c147230 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:27.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:27.762+0000 7f1944402640 1 -- 172.21.15.49:0/4130486345 >> 172.21.15.49:0/4130486345 conn(0x7f193c06a370 msgr2=0x7f193c15be10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:27.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:27.762+0000 7f1944402640 1 -- 172.21.15.49:0/4130486345 shutdown_connections 2023-11-06T02:13:27.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:27.762+0000 7f1944402640 1 -- 172.21.15.49:0/4130486345 wait complete. 2023-11-06T02:13:27.775 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-06T02:13:27.775 DEBUG:teuthology.orchestra.run.smithi049:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T02:13:27.806 DEBUG:teuthology.orchestra.run.smithi150:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T02:13:27.858 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-06T02:13:27.858 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-11-06T02:13:27.859 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T02:13:27.865 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336476k,nr_inodes=4084119,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16105 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-06T02:13:27.865 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-11-06T02:13:27.865 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T02:13:27.872 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336552k,nr_inodes=4084138,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19862 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274692k,nr_inodes=818673,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-06T02:13:27.872 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-11-06T02:13:27.872 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T02:13:27.913 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336476k,nr_inodes=4084119,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16105 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-06T02:13:27.914 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-11-06T02:13:27.914 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T02:13:27.925 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336552k,nr_inodes=4084138,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19862 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274692k,nr_inodes=818673,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-06T02:13:27.926 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-06T02:13:27.926 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T02:13:28.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.086+0000 7f83bf8c7640 1 Processor -- start 2023-11-06T02:13:28.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.086+0000 7f83bf8c7640 1 -- start start 2023-11-06T02:13:28.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.086+0000 7f83bf8c7640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f83b80b0790 0x7f83b80cc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:28.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.086+0000 7f83bf8c7640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83b8165b80 con 0x7f83b81646f0 2023-11-06T02:13:28.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.086+0000 7f83bf8c7640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83b8165600 con 0x7f83b80c8c50 2023-11-06T02:13:28.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.086+0000 7f83bf8c7640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83b81658a0 con 0x7f83b80b0790 2023-11-06T02:13:28.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.086+0000 7f83bd663640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f83b81646f0 0x7f83b80b0080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:46826/0 (socket says 172.21.15.49:46826) 2023-11-06T02:13:28.085 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.086+0000 7f83bd663640 1 -- 172.21.15.49:0/1736654309 learned_addr learned my addr 172.21.15.49:0/1736654309 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:13:28.085 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.086+0000 7f83b7fff640 1 -- 172.21.15.49:0/1736654309 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3520982595 0 0) 0x7f83b8165b80 con 0x7f83b81646f0 2023-11-06T02:13:28.085 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.086+0000 7f83bce62640 1 --2- 172.21.15.49:0/1736654309 >> v2:172.21.15.150:3300/0 conn(0x7f83b80b0790 0x7f83b80cc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:28.085 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.086+0000 7f83b7fff640 1 -- 172.21.15.49:0/1736654309 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83a0003530 con 0x7f83b81646f0 2023-11-06T02:13:28.086 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.086+0000 7f83b7fff640 1 -- 172.21.15.49:0/1736654309 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 366412901 0 0) 0x7f83a0003530 con 0x7f83b81646f0 2023-11-06T02:13:28.086 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.086+0000 7f83b7fff640 1 -- 172.21.15.49:0/1736654309 >> v2:172.21.15.150:3300/0 conn(0x7f83b80b0790 msgr2=0x7f83b80cc990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:28.086 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.086+0000 7f83b7fff640 1 --2- 172.21.15.49:0/1736654309 >> v2:172.21.15.150:3300/0 conn(0x7f83b80b0790 0x7f83b80cc990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:28.086 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.086+0000 7f83b7fff640 1 -- 172.21.15.49:0/1736654309 >> v1:172.21.15.150:6789/0 conn(0x7f83b80c8c50 legacy=0x7f83b80cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:28.086 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.086+0000 7f83b7fff640 1 -- 172.21.15.49:0/1736654309 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83b80c96f0 con 0x7f83b81646f0 2023-11-06T02:13:28.087 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.086+0000 7f83b7fff640 1 -- 172.21.15.49:0/1736654309 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7f83a8001800 con 0x7f83b81646f0 2023-11-06T02:13:28.087 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.086+0000 7f83b7fff640 1 -- 172.21.15.49:0/1736654309 >> v1:172.21.15.49:6789/0 conn(0x7f83b81646f0 legacy=0x7f83b80b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:28.087 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.086+0000 7f83b7fff640 1 --2- 172.21.15.49:0/1736654309 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83a0003980 0x7f83a0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:28.087 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.086+0000 7f83b7fff640 1 --2- 172.21.15.49:0/1736654309 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83a00047d0 0x7f83a0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:28.088 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.086+0000 7f83b7fff640 1 --2- 172.21.15.49:0/1736654309 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f83a0005190 0x7f83a000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:28.088 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.086+0000 7f83b7fff640 1 -- 172.21.15.49:0/1736654309 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83b81658a0 con 0x7f83a00047d0 2023-11-06T02:13:28.088 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.086+0000 7f83b7fff640 1 -- 172.21.15.49:0/1736654309 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83a000c030 con 0x7f83a0005190 2023-11-06T02:13:28.088 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.086+0000 7f83b7fff640 1 -- 172.21.15.49:0/1736654309 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83a000c1f0 con 0x7f83a0003980 2023-11-06T02:13:28.088 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.086+0000 7f83bce62640 1 --2- 172.21.15.49:0/1736654309 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83a00047d0 0x7f83a0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:28.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.086+0000 7f83bd663640 1 --2- 172.21.15.49:0/1736654309 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83a0003980 0x7f83a0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:28.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.086+0000 7f83bce62640 1 -- 172.21.15.49:0/1736654309 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83a0003980 msgr2=0x7f83a0003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:28.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.086+0000 7f83bce62640 1 --2- 172.21.15.49:0/1736654309 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83a0003980 0x7f83a0003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:28.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.086+0000 7f83bce62640 1 -- 172.21.15.49:0/1736654309 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f83a0005190 msgr2=0x7f83a000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:28.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.086+0000 7f83bde64640 1 --2- 172.21.15.49:0/1736654309 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f83a0005190 0x7f83a000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:28.091 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.086+0000 7f83bce62640 1 --2- 172.21.15.49:0/1736654309 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f83a0005190 0x7f83a000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:28.091 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.086+0000 7f83bce62640 1 -- 172.21.15.49:0/1736654309 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83a0003530 con 0x7f83a00047d0 2023-11-06T02:13:28.091 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.086+0000 7f83bde64640 1 --2- 172.21.15.49:0/1736654309 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f83a0005190 0x7f83a000b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:13:28.091 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.086+0000 7f83bde64640 1 -- 172.21.15.49:0/1736654309 reap_dead start 2023-11-06T02:13:28.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.090+0000 7f83bce62640 1 --2- 172.21.15.49:0/1736654309 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83a00047d0 0x7f83a0004c50 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f83ac009ae0 tx=0x7f83ac01ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83271363683995b0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:28.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.090+0000 7f83b7fff640 1 -- 172.21.15.49:0/1736654309 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83ac0a5610 con 0x7f83a00047d0 2023-11-06T02:13:28.097 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.090+0000 7f83b7fff640 1 -- 172.21.15.49:0/1736654309 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83ac0a5770 con 0x7f83a00047d0 2023-11-06T02:13:28.097 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.090+0000 7f83b7fff640 1 -- 172.21.15.49:0/1736654309 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83ac0a5b20 con 0x7f83a00047d0 2023-11-06T02:13:28.098 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.090+0000 7f83bf8c7640 1 -- 172.21.15.49:0/1736654309 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83a00047d0 msgr2=0x7f83a0004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:28.098 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.090+0000 7f83bf8c7640 1 --2- 172.21.15.49:0/1736654309 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83a00047d0 0x7f83a0004c50 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f83ac009ae0 tx=0x7f83ac01ca50 comp rx=0 tx=0).stop 2023-11-06T02:13:28.098 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.090+0000 7f83bf8c7640 1 -- 172.21.15.49:0/1736654309 shutdown_connections 2023-11-06T02:13:28.099 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.090+0000 7f83bf8c7640 1 --2- 172.21.15.49:0/1736654309 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83a00047d0 0x7f83a0004c50 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:28.099 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.090+0000 7f83bf8c7640 1 -- 172.21.15.49:0/1736654309 >> 172.21.15.49:0/1736654309 conn(0x7f83b806c0a0 msgr2=0x7f83b806c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:28.099 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.090+0000 7f83bf8c7640 1 -- 172.21.15.49:0/1736654309 shutdown_connections 2023-11-06T02:13:28.099 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.090+0000 7f83bf8c7640 1 -- 172.21.15.49:0/1736654309 wait complete. 2023-11-06T02:13:28.099 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.090+0000 7f83bf8c7640 1 Processor -- start 2023-11-06T02:13:28.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.090+0000 7f83bf8c7640 1 -- start start 2023-11-06T02:13:28.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.094+0000 7f83bf8c7640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f83a00047d0 0x7f83b814e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:28.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.094+0000 7f83bf8c7640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83b80b33d0 0x7f83b8070c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:28.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.094+0000 7f83bf8c7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83b8071170 0x7f83b80af2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:28.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.094+0000 7f83bf8c7640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83b80715f0 con 0x7f83b8071170 2023-11-06T02:13:28.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.094+0000 7f83bf8c7640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83b80ae560 con 0x7f83a00047d0 2023-11-06T02:13:28.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.094+0000 7f83bf8c7640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83b80b3850 con 0x7f83b80b33d0 2023-11-06T02:13:28.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.094+0000 7f83bde64640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83b8071170 0x7f83b80af2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:28.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.094+0000 7f83bde64640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83b8071170 0x7f83b80af2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48922/0 (socket says 172.21.15.49:48922) 2023-11-06T02:13:28.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.094+0000 7f83bde64640 1 -- 172.21.15.49:0/2442300032 learned_addr learned my addr 172.21.15.49:0/2442300032 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:28.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.094+0000 7f83bd663640 1 --2- 172.21.15.49:0/2442300032 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f83a00047d0 0x7f83b814e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:28.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.094+0000 7f83bce62640 1 --2- 172.21.15.49:0/2442300032 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83b80b33d0 0x7f83b8070c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:28.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.094+0000 7f83bde64640 1 -- 172.21.15.49:0/2442300032 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83b80b33d0 msgr2=0x7f83b8070c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:28.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.094+0000 7f83bde64640 1 --2- 172.21.15.49:0/2442300032 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83b80b33d0 0x7f83b8070c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:28.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.094+0000 7f83bde64640 1 -- 172.21.15.49:0/2442300032 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f83a00047d0 msgr2=0x7f83b814e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:28.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.094+0000 7f83bde64640 1 --2- 172.21.15.49:0/2442300032 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f83a00047d0 0x7f83b814e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:28.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.094+0000 7f83bde64640 1 -- 172.21.15.49:0/2442300032 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83b80c21a0 con 0x7f83b8071170 2023-11-06T02:13:28.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.094+0000 7f83bd663640 1 --2- 172.21.15.49:0/2442300032 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f83a00047d0 0x7f83b814e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:13:28.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.094+0000 7f83bde64640 1 --2- 172.21.15.49:0/2442300032 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83b8071170 0x7f83b80af2f0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f83b806cde0 tx=0x7f83b000d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:28.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.094+0000 7f83b5ffb640 1 -- 172.21.15.49:0/2442300032 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b0003d90 con 0x7f83b8071170 2023-11-06T02:13:28.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.094+0000 7f83b5ffb640 1 -- 172.21.15.49:0/2442300032 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83b0005070 con 0x7f83b8071170 2023-11-06T02:13:28.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.094+0000 7f83bf8c7640 1 -- 172.21.15.49:0/2442300032 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83b80c2480 con 0x7f83b8071170 2023-11-06T02:13:28.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.094+0000 7f83b5ffb640 1 -- 172.21.15.49:0/2442300032 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b0005420 con 0x7f83b8071170 2023-11-06T02:13:28.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.094+0000 7f83bf8c7640 1 -- 172.21.15.49:0/2442300032 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83b80c2a40 con 0x7f83b8071170 2023-11-06T02:13:28.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.094+0000 7f83bf8c7640 1 -- 172.21.15.49:0/2442300032 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83b8074040 con 0x7f83b8071170 2023-11-06T02:13:28.105 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.094+0000 7f83b5ffb640 1 -- 172.21.15.49:0/2442300032 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f83b0005580 con 0x7f83b8071170 2023-11-06T02:13:28.105 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.094+0000 7f83b5ffb640 1 --2- 172.21.15.49:0/2442300032 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f83a4065a60 0x7f83a4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:28.105 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.094+0000 7f83b5ffb640 1 -- 172.21.15.49:0/2442300032 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f83b0082880 con 0x7f83b8071170 2023-11-06T02:13:28.105 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.094+0000 7f83bd663640 1 --2- 172.21.15.49:0/2442300032 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f83a4065a60 0x7f83a4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:28.105 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.094+0000 7f83bd663640 1 --2- 172.21.15.49:0/2442300032 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f83a4065a60 0x7f83a4067f20 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f83a8000f00 tx=0x7f83a8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:28.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.098+0000 7f83b5ffb640 1 -- 172.21.15.49:0/2442300032 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f83b0050410 con 0x7f83b8071170 2023-11-06T02:13:28.283 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.286+0000 7f83bf8c7640 1 -- 172.21.15.49:0/2442300032 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f83b80b4580 con 0x7f83b8071170 2023-11-06T02:13:28.283 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.286+0000 7f83b5ffb640 1 -- 172.21.15.49:0/2442300032 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+3532 (secure 0 0 0) 0x7f83b0018020 con 0x7f83b8071170 2023-11-06T02:13:28.284 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:13:28.284 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":40,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5635,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4170291905","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":4170291905},{"type":"v1","addr":"172.21.15.150:6835","nonce":4170291905}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5638,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/2719458937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":2719458937},{"type":"v1","addr":"172.21.15.150:6837","nonce":2719458937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":6339,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/2310338375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2310338375},{"type":"v1","addr":"172.21.15.49:6833","nonce":2310338375}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":6960,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/322306580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":322306580},{"type":"v1","addr":"172.21.15.49:6835","nonce":322306580}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38}],"filesystems":[]} 2023-11-06T02:13:28.284 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 40 2023-11-06T02:13:28.286 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.290+0000 7f83bf8c7640 1 -- 172.21.15.49:0/2442300032 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f83a4065a60 msgr2=0x7f83a4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:28.286 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.290+0000 7f83bf8c7640 1 --2- 172.21.15.49:0/2442300032 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f83a4065a60 0x7f83a4067f20 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f83a8000f00 tx=0x7f83a8009040 comp rx=0 tx=0).stop 2023-11-06T02:13:28.286 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.290+0000 7f83bf8c7640 1 -- 172.21.15.49:0/2442300032 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83b8071170 msgr2=0x7f83b80af2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:28.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.290+0000 7f83bf8c7640 1 --2- 172.21.15.49:0/2442300032 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83b8071170 0x7f83b80af2f0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f83b806cde0 tx=0x7f83b000d9d0 comp rx=0 tx=0).stop 2023-11-06T02:13:28.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.290+0000 7f83bf8c7640 1 -- 172.21.15.49:0/2442300032 shutdown_connections 2023-11-06T02:13:28.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.290+0000 7f83bf8c7640 1 --2- 172.21.15.49:0/2442300032 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f83a4065a60 0x7f83a4067f20 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:28.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.290+0000 7f83bf8c7640 1 --2- 172.21.15.49:0/2442300032 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83b8071170 0x7f83b80af2f0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:28.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.290+0000 7f83bf8c7640 1 --2- 172.21.15.49:0/2442300032 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f83b80b33d0 0x7f83b8070c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:28.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.290+0000 7f83bf8c7640 1 --2- 172.21.15.49:0/2442300032 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f83a00047d0 0x7f83b814e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:28.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.290+0000 7f83bf8c7640 1 -- 172.21.15.49:0/2442300032 >> 172.21.15.49:0/2442300032 conn(0x7f83b806c0a0 msgr2=0x7f83b80ce590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:28.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.290+0000 7f83bf8c7640 1 -- 172.21.15.49:0/2442300032 shutdown_connections 2023-11-06T02:13:28.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.290+0000 7f83bf8c7640 1 -- 172.21.15.49:0/2442300032 wait complete. 2023-11-06T02:13:28.302 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-06T02:13:28.302 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-06T02:13:28.302 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-06T02:13:28.303 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T02:13:28.303 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-06T02:13:28.303 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-06T02:13:28.304 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-06T02:13:28.304 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T02:13:28.304 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-06T02:13:28.304 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-06T02:13:28.305 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-06T02:13:28.305 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T02:13:28.305 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-06T02:13:28.306 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-06T02:13:28.306 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-06T02:13:28.306 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T02:13:28.312 INFO:tasks.ceph.mds.a:Stopped 2023-11-06T02:13:28.312 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-06T02:13:28.314 INFO:tasks.ceph.mds.c:Stopped 2023-11-06T02:13:28.314 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-06T02:13:28.319 INFO:tasks.ceph.mds.d:Stopped 2023-11-06T02:13:28.319 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-06T02:13:28.320 INFO:tasks.ceph.mds.b:Stopped 2023-11-06T02:13:28.321 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-06T02:13:28.357 INFO:tasks.ceph.mds.a:Started 2023-11-06T02:13:28.358 DEBUG:teuthology.parallel:result is None 2023-11-06T02:13:28.359 INFO:tasks.ceph.mds.c:Started 2023-11-06T02:13:28.359 DEBUG:teuthology.parallel:result is None 2023-11-06T02:13:28.362 INFO:tasks.ceph.mds.d:Started 2023-11-06T02:13:28.362 DEBUG:teuthology.parallel:result is None 2023-11-06T02:13:28.363 INFO:tasks.ceph.mds.b:Started 2023-11-06T02:13:28.363 DEBUG:teuthology.parallel:result is None 2023-11-06T02:13:28.364 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-06T02:13:28.417 INFO:tasks.ceph.mds.a.smithi049.stdout:starting mds.a at 2023-11-06T02:13:28.424 INFO:tasks.ceph.mds.c.smithi049.stdout:starting mds.c at 2023-11-06T02:13:28.431 INFO:tasks.ceph.mds.d.smithi150.stdout:starting mds.d at 2023-11-06T02:13:28.436 INFO:tasks.ceph.mds.b.smithi150.stdout:starting mds.b at 2023-11-06T02:13:28.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.502+0000 7ff2b6895640 1 Processor -- start 2023-11-06T02:13:28.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.502+0000 7ff2b6895640 1 -- start start 2023-11-06T02:13:28.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.506+0000 7ff2b6895640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7ff2b00ced80 0x7ff2b0165740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:28.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.506+0000 7ff2b6895640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2b00b0d70 con 0x7ff2b015a970 2023-11-06T02:13:28.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.506+0000 7ff2b6895640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2b00b01f0 con 0x7ff2b015ad50 2023-11-06T02:13:28.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.506+0000 7ff2b6895640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2b00b0440 con 0x7ff2b00ced80 2023-11-06T02:13:28.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.506+0000 7ff2affff640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7ff2b015a970 0x7ff2b00cd8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:60492/0 (socket says 172.21.15.49:60492) 2023-11-06T02:13:28.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.506+0000 7ff2affff640 1 -- 172.21.15.49:0/2335093843 learned_addr learned my addr 172.21.15.49:0/2335093843 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:13:28.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.506+0000 7ff2b4e32640 1 --2- 172.21.15.49:0/2335093843 >> v2:172.21.15.150:3300/0 conn(0x7ff2b00ced80 0x7ff2b0165740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:28.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.506+0000 7ff2aeffd640 1 -- 172.21.15.49:0/2335093843 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3463966956 0 0) 0x7ff2b00b0d70 con 0x7ff2b015a970 2023-11-06T02:13:28.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.506+0000 7ff2aeffd640 1 -- 172.21.15.49:0/2335093843 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff298003530 con 0x7ff2b015a970 2023-11-06T02:13:28.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.506+0000 7ff2aeffd640 1 -- 172.21.15.49:0/2335093843 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1935346451 0 0) 0x7ff298003530 con 0x7ff2b015a970 2023-11-06T02:13:28.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.506+0000 7ff2aeffd640 1 -- 172.21.15.49:0/2335093843 >> v2:172.21.15.150:3300/0 conn(0x7ff2b00ced80 msgr2=0x7ff2b0165740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:28.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.506+0000 7ff2aeffd640 1 --2- 172.21.15.49:0/2335093843 >> v2:172.21.15.150:3300/0 conn(0x7ff2b00ced80 0x7ff2b0165740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:28.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.506+0000 7ff2aeffd640 1 -- 172.21.15.49:0/2335093843 >> v1:172.21.15.150:6789/0 conn(0x7ff2b015ad50 legacy=0x7ff2b00ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:28.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.506+0000 7ff2aeffd640 1 -- 172.21.15.49:0/2335093843 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2b00b0720 con 0x7ff2b015a970 2023-11-06T02:13:28.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.506+0000 7ff2aeffd640 1 -- 172.21.15.49:0/2335093843 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7ff2a8001b80 con 0x7ff2b015a970 2023-11-06T02:13:28.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.506+0000 7ff2aeffd640 1 -- 172.21.15.49:0/2335093843 >> v1:172.21.15.49:6789/0 conn(0x7ff2b015a970 legacy=0x7ff2b00cd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:28.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.506+0000 7ff2aeffd640 1 --2- 172.21.15.49:0/2335093843 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff298003980 0x7ff298003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:28.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.506+0000 7ff2aeffd640 1 --2- 172.21.15.49:0/2335093843 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff2980047d0 0x7ff298004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:28.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.506+0000 7ff2aeffd640 1 --2- 172.21.15.49:0/2335093843 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff298005190 0x7ff29800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:28.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.506+0000 7ff2aeffd640 1 -- 172.21.15.49:0/2335093843 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b00b0440 con 0x7ff298005190 2023-11-06T02:13:28.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.506+0000 7ff2aeffd640 1 -- 172.21.15.49:0/2335093843 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff29800c030 con 0x7ff298003980 2023-11-06T02:13:28.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.506+0000 7ff2aeffd640 1 -- 172.21.15.49:0/2335093843 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff29800c1f0 con 0x7ff2980047d0 2023-11-06T02:13:28.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.506+0000 7ff2b4e32640 1 --2- 172.21.15.49:0/2335093843 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff298005190 0x7ff29800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:28.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.506+0000 7ff2affff640 1 --2- 172.21.15.49:0/2335093843 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff298003980 0x7ff298003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:28.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.506+0000 7ff2af7fe640 1 --2- 172.21.15.49:0/2335093843 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff2980047d0 0x7ff298004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:28.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.506+0000 7ff2b4e32640 1 -- 172.21.15.49:0/2335093843 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff2980047d0 msgr2=0x7ff298004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:28.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.506+0000 7ff2b4e32640 1 --2- 172.21.15.49:0/2335093843 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff2980047d0 0x7ff298004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:28.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.506+0000 7ff2b4e32640 1 -- 172.21.15.49:0/2335093843 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff298003980 msgr2=0x7ff298003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:28.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.506+0000 7ff2b4e32640 1 --2- 172.21.15.49:0/2335093843 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff298003980 0x7ff298003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:28.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.506+0000 7ff2b4e32640 1 -- 172.21.15.49:0/2335093843 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff298003530 con 0x7ff298005190 2023-11-06T02:13:28.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.506+0000 7ff2af7fe640 1 --2- 172.21.15.49:0/2335093843 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff2980047d0 0x7ff298004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:13:28.514 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.506+0000 7ff2b4e32640 1 -- 172.21.15.49:0/2335093843 reap_dead start 2023-11-06T02:13:28.514 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.506+0000 7ff2b4e32640 1 --2- 172.21.15.49:0/2335093843 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff298005190 0x7ff29800b990 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7ff2a0009ae0 tx=0x7ff2a001ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76d9b69ec363e91e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:28.514 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.506+0000 7ff2aeffd640 1 -- 172.21.15.49:0/2335093843 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a0004020 con 0x7ff298005190 2023-11-06T02:13:28.514 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.506+0000 7ff2aeffd640 1 -- 172.21.15.49:0/2335093843 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2a00041c0 con 0x7ff298005190 2023-11-06T02:13:28.514 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.506+0000 7ff2aeffd640 1 -- 172.21.15.49:0/2335093843 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a00a5680 con 0x7ff298005190 2023-11-06T02:13:28.515 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.506+0000 7ff2b6895640 1 -- 172.21.15.49:0/2335093843 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff298005190 msgr2=0x7ff29800b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:28.515 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.506+0000 7ff2b6895640 1 --2- 172.21.15.49:0/2335093843 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff298005190 0x7ff29800b990 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7ff2a0009ae0 tx=0x7ff2a001ca90 comp rx=0 tx=0).stop 2023-11-06T02:13:28.515 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.510+0000 7ff2b6895640 1 -- 172.21.15.49:0/2335093843 shutdown_connections 2023-11-06T02:13:28.515 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.510+0000 7ff2b6895640 1 --2- 172.21.15.49:0/2335093843 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff298005190 0x7ff29800b990 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:28.515 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.510+0000 7ff2b6895640 1 -- 172.21.15.49:0/2335093843 >> 172.21.15.49:0/2335093843 conn(0x7ff2b00d0890 msgr2=0x7ff2b00d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:28.516 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.510+0000 7ff2b6895640 1 -- 172.21.15.49:0/2335093843 shutdown_connections 2023-11-06T02:13:28.516 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.510+0000 7ff2b6895640 1 -- 172.21.15.49:0/2335093843 wait complete. 2023-11-06T02:13:28.516 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.510+0000 7ff2b6895640 1 Processor -- start 2023-11-06T02:13:28.516 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.510+0000 7ff2b6895640 1 -- start start 2023-11-06T02:13:28.516 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.510+0000 7ff2b6895640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff298005190 0x7ff2b014cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:28.517 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.510+0000 7ff2b6895640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2b015b2c0 0x7ff2b006f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:28.517 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.510+0000 7ff2b6895640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff2b00c5750 0x7ff2b006f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:28.517 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.510+0000 7ff2b6895640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b00c43c0 con 0x7ff298005190 2023-11-06T02:13:28.517 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.510+0000 7ff2b6895640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b00c5c30 con 0x7ff2b015b2c0 2023-11-06T02:13:28.517 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.510+0000 7ff2b6895640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b00c4610 con 0x7ff2b00c5750 2023-11-06T02:13:28.518 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.510+0000 7ff2affff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff298005190 0x7ff2b014cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:28.518 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.510+0000 7ff2affff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff298005190 0x7ff2b014cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51088/0 (socket says 172.21.15.49:51088) 2023-11-06T02:13:28.518 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.510+0000 7ff2affff640 1 -- 172.21.15.49:0/85040594 learned_addr learned my addr 172.21.15.49:0/85040594 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:28.518 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.510+0000 7ff2af7fe640 1 --2- 172.21.15.49:0/85040594 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2b015b2c0 0x7ff2b006f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:28.518 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.510+0000 7ff2b4e32640 1 --2- 172.21.15.49:0/85040594 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff2b00c5750 0x7ff2b006f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:28.518 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.510+0000 7ff2affff640 1 -- 172.21.15.49:0/85040594 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff2b00c5750 msgr2=0x7ff2b006f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:28.519 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.510+0000 7ff2affff640 1 --2- 172.21.15.49:0/85040594 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff2b00c5750 0x7ff2b006f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:28.519 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.510+0000 7ff2affff640 1 -- 172.21.15.49:0/85040594 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2b015b2c0 msgr2=0x7ff2b006f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:28.519 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.510+0000 7ff2affff640 1 --2- 172.21.15.49:0/85040594 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2b015b2c0 0x7ff2b006f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:28.519 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.510+0000 7ff2affff640 1 -- 172.21.15.49:0/85040594 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2b00c48f0 con 0x7ff298005190 2023-11-06T02:13:28.519 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.510+0000 7ff2affff640 1 --2- 172.21.15.49:0/85040594 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff298005190 0x7ff2b014cce0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7ff2a8001b70 tx=0x7ff2a800d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:28.520 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.510+0000 7ff2acff9640 1 -- 172.21.15.49:0/85040594 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a800fed0 con 0x7ff298005190 2023-11-06T02:13:28.520 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.510+0000 7ff2acff9640 1 -- 172.21.15.49:0/85040594 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2a800ddc0 con 0x7ff298005190 2023-11-06T02:13:28.520 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.510+0000 7ff2b6895640 1 -- 172.21.15.49:0/85040594 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2b00c4bd0 con 0x7ff298005190 2023-11-06T02:13:28.520 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.510+0000 7ff2acff9640 1 -- 172.21.15.49:0/85040594 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a8005d70 con 0x7ff298005190 2023-11-06T02:13:28.520 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.510+0000 7ff2b6895640 1 -- 172.21.15.49:0/85040594 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2b00c52c0 con 0x7ff298005190 2023-11-06T02:13:28.521 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.514+0000 7ff2b6895640 1 -- 172.21.15.49:0/85040594 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2b00c4e10 con 0x7ff298005190 2023-11-06T02:13:28.521 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.514+0000 7ff2acff9640 1 -- 172.21.15.49:0/85040594 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7ff2a80070d0 con 0x7ff298005190 2023-11-06T02:13:28.521 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.514+0000 7ff2acff9640 1 --2- 172.21.15.49:0/85040594 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff29c065a60 0x7ff29c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:28.521 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.514+0000 7ff2acff9640 1 -- 172.21.15.49:0/85040594 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5382+0+0 (secure 0 0 0) 0x7ff2a8083530 con 0x7ff298005190 2023-11-06T02:13:28.521 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.514+0000 7ff2af7fe640 1 --2- 172.21.15.49:0/85040594 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff29c065a60 0x7ff29c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:28.522 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.514+0000 7ff2af7fe640 1 --2- 172.21.15.49:0/85040594 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff29c065a60 0x7ff29c067f20 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7ff2a4005450 tx=0x7ff2a4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:28.522 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.518+0000 7ff2acff9640 1 -- 172.21.15.49:0/85040594 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7ff2a80510c0 con 0x7ff298005190 2023-11-06T02:13:28.676 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:28.682+0000 7ff2b6895640 1 -- 172.21.15.49:0/85040594 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7ff2b0074040 con 0x7ff298005190 2023-11-06T02:13:29.467 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.470+0000 7ff2acff9640 1 -- 172.21.15.49:0/85040594 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v76) v1 ==== 97+0+0 (secure 0 0 0) 0x7ff2a8001300 con 0x7ff298005190 2023-11-06T02:13:29.468 INFO:teuthology.orchestra.run.smithi049.stderr: removed all blocklist entries 2023-11-06T02:13:29.470 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.474+0000 7ff2b6895640 1 -- 172.21.15.49:0/85040594 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff29c065a60 msgr2=0x7ff29c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:29.470 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.474+0000 7ff2b6895640 1 --2- 172.21.15.49:0/85040594 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff29c065a60 0x7ff29c067f20 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7ff2a4005450 tx=0x7ff2a4009040 comp rx=0 tx=0).stop 2023-11-06T02:13:29.471 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.474+0000 7ff2b6895640 1 -- 172.21.15.49:0/85040594 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff298005190 msgr2=0x7ff2b014cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:29.471 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.474+0000 7ff2b6895640 1 --2- 172.21.15.49:0/85040594 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff298005190 0x7ff2b014cce0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7ff2a8001b70 tx=0x7ff2a800d520 comp rx=0 tx=0).stop 2023-11-06T02:13:29.471 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.474+0000 7ff2b6895640 1 -- 172.21.15.49:0/85040594 shutdown_connections 2023-11-06T02:13:29.471 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.474+0000 7ff2b6895640 1 --2- 172.21.15.49:0/85040594 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff29c065a60 0x7ff29c067f20 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:29.471 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.474+0000 7ff2b6895640 1 --2- 172.21.15.49:0/85040594 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff2b00c5750 0x7ff2b006f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:29.472 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.474+0000 7ff2b6895640 1 --2- 172.21.15.49:0/85040594 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2b015b2c0 0x7ff2b006f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:29.472 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.474+0000 7ff2b6895640 1 --2- 172.21.15.49:0/85040594 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff298005190 0x7ff2b014cce0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:29.472 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.474+0000 7ff2b6895640 1 -- 172.21.15.49:0/85040594 >> 172.21.15.49:0/85040594 conn(0x7ff2b00d0890 msgr2=0x7ff2b0159b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:29.472 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.474+0000 7ff2b6895640 1 -- 172.21.15.49:0/85040594 shutdown_connections 2023-11-06T02:13:29.472 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.474+0000 7ff2b6895640 1 -- 172.21.15.49:0/85040594 wait complete. 2023-11-06T02:13:29.493 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-06T02:13:29.616 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.622+0000 7fcc3c76f640 1 Processor -- start 2023-11-06T02:13:29.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.622+0000 7fcc3c76f640 1 -- start start 2023-11-06T02:13:29.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.622+0000 7fcc3c76f640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcc3415ab50 0x7fcc340c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:29.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.622+0000 7fcc3c76f640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc340cab80 con 0x7fcc340cb010 2023-11-06T02:13:29.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.622+0000 7fcc3c76f640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc3415af30 con 0x7fcc340b0790 2023-11-06T02:13:29.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.622+0000 7fcc3c76f640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc340cb5b0 con 0x7fcc3415ab50 2023-11-06T02:13:29.621 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.622+0000 7fcc39d0a640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fcc340cb010 0x7fcc340b0080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:60508/0 (socket says 172.21.15.49:60508) 2023-11-06T02:13:29.621 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.622+0000 7fcc39d0a640 1 -- 172.21.15.49:0/1377448991 learned_addr learned my addr 172.21.15.49:0/1377448991 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:13:29.621 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.622+0000 7fcc3a50b640 1 --2- 172.21.15.49:0/1377448991 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcc3415ab50 0x7fcc340c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:29.622 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.622+0000 7fcc39509640 1 -- 172.21.15.49:0/1377448991 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2410022765 0 0) 0x7fcc340cab80 con 0x7fcc340cb010 2023-11-06T02:13:29.622 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.622+0000 7fcc39509640 1 -- 172.21.15.49:0/1377448991 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc20003530 con 0x7fcc340cb010 2023-11-06T02:13:29.622 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.622+0000 7fcc39509640 1 -- 172.21.15.49:0/1377448991 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2586374543 0 0) 0x7fcc20003530 con 0x7fcc340cb010 2023-11-06T02:13:29.622 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.622+0000 7fcc39509640 1 -- 172.21.15.49:0/1377448991 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcc3415ab50 msgr2=0x7fcc340c9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:29.622 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.622+0000 7fcc39509640 1 --2- 172.21.15.49:0/1377448991 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcc3415ab50 0x7fcc340c9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:29.623 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.622+0000 7fcc39509640 1 -- 172.21.15.49:0/1377448991 >> v1:172.21.15.150:6789/0 conn(0x7fcc340b0790 legacy=0x7fcc340c2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:29.623 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.622+0000 7fcc39509640 1 -- 172.21.15.49:0/1377448991 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc340cb890 con 0x7fcc340cb010 2023-11-06T02:13:29.623 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.622+0000 7fcc39509640 1 -- 172.21.15.49:0/1377448991 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7fcc24001680 con 0x7fcc340cb010 2023-11-06T02:13:29.623 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.622+0000 7fcc39509640 1 -- 172.21.15.49:0/1377448991 >> v1:172.21.15.49:6789/0 conn(0x7fcc340cb010 legacy=0x7fcc340b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:29.623 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.622+0000 7fcc39509640 1 --2- 172.21.15.49:0/1377448991 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcc3415ab50 0x7fcc200039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:29.624 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.622+0000 7fcc39509640 1 --2- 172.21.15.49:0/1377448991 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc20003ef0 0x7fcc200047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:29.624 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.622+0000 7fcc39509640 1 --2- 172.21.15.49:0/1377448991 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcc20004d10 0x7fcc20005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:29.624 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.622+0000 7fcc39509640 1 -- 172.21.15.49:0/1377448991 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc340cb5b0 con 0x7fcc20004d10 2023-11-06T02:13:29.624 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.622+0000 7fcc39509640 1 -- 172.21.15.49:0/1377448991 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2000b9f0 con 0x7fcc20003ef0 2023-11-06T02:13:29.624 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.622+0000 7fcc39509640 1 -- 172.21.15.49:0/1377448991 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2000bbb0 con 0x7fcc3415ab50 2023-11-06T02:13:29.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.622+0000 7fcc3ad0c640 1 --2- 172.21.15.49:0/1377448991 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcc20004d10 0x7fcc20005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:29.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.622+0000 7fcc3a50b640 1 --2- 172.21.15.49:0/1377448991 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcc3415ab50 0x7fcc200039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:29.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.622+0000 7fcc39d0a640 1 --2- 172.21.15.49:0/1377448991 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc20003ef0 0x7fcc200047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:29.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.622+0000 7fcc3ad0c640 1 -- 172.21.15.49:0/1377448991 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcc3415ab50 msgr2=0x7fcc200039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:29.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.622+0000 7fcc3ad0c640 1 --2- 172.21.15.49:0/1377448991 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcc3415ab50 0x7fcc200039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:29.626 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.622+0000 7fcc3ad0c640 1 -- 172.21.15.49:0/1377448991 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc20003ef0 msgr2=0x7fcc200047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:29.626 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.622+0000 7fcc3ad0c640 1 --2- 172.21.15.49:0/1377448991 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc20003ef0 0x7fcc200047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:29.626 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.622+0000 7fcc3ad0c640 1 -- 172.21.15.49:0/1377448991 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc20003530 con 0x7fcc20004d10 2023-11-06T02:13:29.626 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.622+0000 7fcc39d0a640 1 --2- 172.21.15.49:0/1377448991 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc20003ef0 0x7fcc200047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:13:29.630 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.626+0000 7fcc3ad0c640 1 --2- 172.21.15.49:0/1377448991 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcc20004d10 0x7fcc20005190 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fcc340ca5f0 tx=0x7fcc2c01c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=323ae232a6c5f0eb server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:29.630 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.626+0000 7fcc39509640 1 -- 172.21.15.49:0/1377448991 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc2c0a6700 con 0x7fcc20004d10 2023-11-06T02:13:29.630 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.626+0000 7fcc39509640 1 -- 172.21.15.49:0/1377448991 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc24001d80 con 0x7fcc20004d10 2023-11-06T02:13:29.631 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.626+0000 7fcc39509640 1 -- 172.21.15.49:0/1377448991 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc2c0a6ae0 con 0x7fcc20004d10 2023-11-06T02:13:29.631 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.626+0000 7fcc3c76f640 1 -- 172.21.15.49:0/1377448991 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcc20004d10 msgr2=0x7fcc20005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:29.631 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.626+0000 7fcc3c76f640 1 --2- 172.21.15.49:0/1377448991 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcc20004d10 0x7fcc20005190 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fcc340ca5f0 tx=0x7fcc2c01c800 comp rx=0 tx=0).stop 2023-11-06T02:13:29.631 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.626+0000 7fcc3ad0c640 1 -- 172.21.15.49:0/1377448991 reap_dead start 2023-11-06T02:13:29.631 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.626+0000 7fcc3c76f640 1 -- 172.21.15.49:0/1377448991 shutdown_connections 2023-11-06T02:13:29.632 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.626+0000 7fcc3c76f640 1 -- 172.21.15.49:0/1377448991 >> 172.21.15.49:0/1377448991 conn(0x7fcc3406c0a0 msgr2=0x7fcc3406c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:29.632 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.626+0000 7fcc3c76f640 1 -- 172.21.15.49:0/1377448991 shutdown_connections 2023-11-06T02:13:29.632 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.626+0000 7fcc3c76f640 1 -- 172.21.15.49:0/1377448991 wait complete. 2023-11-06T02:13:29.632 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.626+0000 7fcc3c76f640 1 Processor -- start 2023-11-06T02:13:29.632 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.626+0000 7fcc3c76f640 1 -- start start 2023-11-06T02:13:29.633 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.626+0000 7fcc3c76f640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcc3413de40 0x7fcc3413c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:29.633 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.626+0000 7fcc3c76f640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc3413cb40 0x7fcc3413cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:29.633 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.626+0000 7fcc3c76f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcc34151e10 0x7fcc34152290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:29.633 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.626+0000 7fcc3c76f640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc34153370 con 0x7fcc34151e10 2023-11-06T02:13:29.635 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.626+0000 7fcc3c76f640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc340af4d0 con 0x7fcc3413cb40 2023-11-06T02:13:29.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.626+0000 7fcc3c76f640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc3413d500 con 0x7fcc3413de40 2023-11-06T02:13:29.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.630+0000 7fcc3ad0c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcc34151e10 0x7fcc34152290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:29.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.630+0000 7fcc3ad0c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcc34151e10 0x7fcc34152290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51112/0 (socket says 172.21.15.49:51112) 2023-11-06T02:13:29.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.630+0000 7fcc3ad0c640 1 -- 172.21.15.49:0/2756461845 learned_addr learned my addr 172.21.15.49:0/2756461845 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:29.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.630+0000 7fcc3a50b640 1 --2- 172.21.15.49:0/2756461845 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcc3413de40 0x7fcc3413c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:29.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.630+0000 7fcc39d0a640 1 --2- 172.21.15.49:0/2756461845 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc3413cb40 0x7fcc3413cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:29.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.630+0000 7fcc3ad0c640 1 -- 172.21.15.49:0/2756461845 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcc3413de40 msgr2=0x7fcc3413c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:29.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.630+0000 7fcc3ad0c640 1 --2- 172.21.15.49:0/2756461845 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcc3413de40 0x7fcc3413c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:29.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.630+0000 7fcc3ad0c640 1 -- 172.21.15.49:0/2756461845 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc3413cb40 msgr2=0x7fcc3413cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:29.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.630+0000 7fcc3ad0c640 1 --2- 172.21.15.49:0/2756461845 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc3413cb40 0x7fcc3413cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:29.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.630+0000 7fcc3ad0c640 1 -- 172.21.15.49:0/2756461845 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc340aeba0 con 0x7fcc34151e10 2023-11-06T02:13:29.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.630+0000 7fcc3ad0c640 1 --2- 172.21.15.49:0/2756461845 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcc34151e10 0x7fcc34152290 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c0b0040 tx=0x7fcc20005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:29.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.630+0000 7fcc2affd640 1 -- 172.21.15.49:0/2756461845 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc2c005480 con 0x7fcc34151e10 2023-11-06T02:13:29.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.630+0000 7fcc2affd640 1 -- 172.21.15.49:0/2756461845 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc2c0055e0 con 0x7fcc34151e10 2023-11-06T02:13:29.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.630+0000 7fcc3c76f640 1 -- 172.21.15.49:0/2756461845 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc340aee80 con 0x7fcc34151e10 2023-11-06T02:13:29.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.630+0000 7fcc2affd640 1 -- 172.21.15.49:0/2756461845 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc2c0011d0 con 0x7fcc34151e10 2023-11-06T02:13:29.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.630+0000 7fcc3c76f640 1 -- 172.21.15.49:0/2756461845 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc3414ddf0 con 0x7fcc34151e10 2023-11-06T02:13:29.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.630+0000 7fcc3c76f640 1 -- 172.21.15.49:0/2756461845 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc340b0de0 con 0x7fcc34151e10 2023-11-06T02:13:29.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.630+0000 7fcc2affd640 1 -- 172.21.15.49:0/2756461845 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fcc2c0a6500 con 0x7fcc34151e10 2023-11-06T02:13:29.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.630+0000 7fcc2affd640 1 --2- 172.21.15.49:0/2756461845 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fcc18065a60 0x7fcc18067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:29.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.630+0000 7fcc2affd640 1 -- 172.21.15.49:0/2756461845 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fcc2c0aa070 con 0x7fcc34151e10 2023-11-06T02:13:29.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.630+0000 7fcc3a50b640 1 --2- 172.21.15.49:0/2756461845 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fcc18065a60 0x7fcc18067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:29.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.634+0000 7fcc3a50b640 1 --2- 172.21.15.49:0/2756461845 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fcc18065a60 0x7fcc18067f20 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fcc30009a10 tx=0x7fcc30008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:29.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.634+0000 7fcc2affd640 1 -- 172.21.15.49:0/2756461845 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fcc2c0ead70 con 0x7fcc34151e10 2023-11-06T02:13:29.845 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.850+0000 7fcc3c76f640 1 -- 172.21.15.49:0/2756461845 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fcc34074040 con 0x7fcc34151e10 2023-11-06T02:13:29.847 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.850+0000 7fcc2affd640 1 -- 172.21.15.49:0/2756461845 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fcc2c003e60 con 0x7fcc34151e10 2023-11-06T02:13:29.847 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:13:29.847 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-11-06T02:13:29.847 INFO:teuthology.orchestra.run.smithi049.stdout: "auth_dump": [ 2023-11-06T02:13:29.848 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:13:29.848 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "mds.a", 2023-11-06T02:13:29.848 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQBbSkhlbRFwIhAApA6xaqdIIeKDsOsNhyWCaA==", 2023-11-06T02:13:29.848 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-11-06T02:13:29.848 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": "allow", 2023-11-06T02:13:29.849 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow *", 2023-11-06T02:13:29.849 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow *", 2023-11-06T02:13:29.849 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-11-06T02:13:29.849 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:13:29.849 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:13:29.850 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:13:29.850 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "mds.b", 2023-11-06T02:13:29.850 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQBbSkhl6QhcMRAAgmOxZWghmuu11F0Gfut3sQ==", 2023-11-06T02:13:29.850 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-11-06T02:13:29.850 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": "allow", 2023-11-06T02:13:29.851 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow *", 2023-11-06T02:13:29.851 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow *", 2023-11-06T02:13:29.851 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-11-06T02:13:29.851 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:13:29.851 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:13:29.852 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:13:29.852 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "mds.c", 2023-11-06T02:13:29.852 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQBbSkhlMoZHKxAATV62bAItdXt3ShpsT4oVbA==", 2023-11-06T02:13:29.852 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-11-06T02:13:29.852 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": "allow", 2023-11-06T02:13:29.853 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow *", 2023-11-06T02:13:29.853 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow *", 2023-11-06T02:13:29.853 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-11-06T02:13:29.853 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:13:29.853 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:13:29.854 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:13:29.854 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "mds.d", 2023-11-06T02:13:29.854 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQBbSkhl376GORAAS2NIQJMzg6Es0JCYPak6fw==", 2023-11-06T02:13:29.854 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-11-06T02:13:29.855 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": "allow", 2023-11-06T02:13:29.855 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow *", 2023-11-06T02:13:29.855 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow *", 2023-11-06T02:13:29.855 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-11-06T02:13:29.855 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:13:29.856 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:13:29.856 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:13:29.856 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "osd.0", 2023-11-06T02:13:29.856 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQBdSkhlxuQtFRAAPttjVsYpkwexC5x2Pu2b+Q==", 2023-11-06T02:13:29.856 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-11-06T02:13:29.857 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow profile osd", 2023-11-06T02:13:29.857 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow profile osd", 2023-11-06T02:13:29.857 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-11-06T02:13:29.857 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:13:29.857 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:13:29.858 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:13:29.858 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "osd.1", 2023-11-06T02:13:29.858 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQBgSkhlCGeiHBAA8SofNV6C1+RQNajUsewVpw==", 2023-11-06T02:13:29.858 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-11-06T02:13:29.858 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow profile osd", 2023-11-06T02:13:29.859 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow profile osd", 2023-11-06T02:13:29.859 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-11-06T02:13:29.859 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:13:29.859 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:13:29.859 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:13:29.860 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "osd.2", 2023-11-06T02:13:29.860 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQBjSkhlBju+JBAAPJeIFngKDoamUvEjv1h8tw==", 2023-11-06T02:13:29.860 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-11-06T02:13:29.860 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow profile osd", 2023-11-06T02:13:29.860 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow profile osd", 2023-11-06T02:13:29.861 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-11-06T02:13:29.861 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:13:29.861 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:13:29.861 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:13:29.862 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "osd.3", 2023-11-06T02:13:29.865 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQBmSkhlMclTLRAABnbmQA9j3TLn40H1wYFx5g==", 2023-11-06T02:13:29.866 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-11-06T02:13:29.866 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow profile osd", 2023-11-06T02:13:29.867 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow profile osd", 2023-11-06T02:13:29.867 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-11-06T02:13:29.867 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:13:29.867 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:13:29.867 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:13:29.868 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "osd.4", 2023-11-06T02:13:29.868 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQBsSkhl7fmmLhAArv62qbJ+StgYMKhCftM5HA==", 2023-11-06T02:13:29.868 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-11-06T02:13:29.868 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow profile osd", 2023-11-06T02:13:29.869 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow profile osd", 2023-11-06T02:13:29.869 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-11-06T02:13:29.869 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:13:29.869 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:13:29.869 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:13:29.870 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "osd.5", 2023-11-06T02:13:29.870 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQBvSkhl5pAANhAArNFy9/YuEcXTdzM5XL46Xw==", 2023-11-06T02:13:29.870 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-11-06T02:13:29.870 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow profile osd", 2023-11-06T02:13:29.870 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow profile osd", 2023-11-06T02:13:29.871 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-11-06T02:13:29.871 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:13:29.871 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:13:29.871 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:13:29.872 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "osd.6", 2023-11-06T02:13:29.872 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQBzSkhlgfs2BBAAeHXVJ/tFiFagugkynGakFw==", 2023-11-06T02:13:29.872 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-11-06T02:13:29.872 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow profile osd", 2023-11-06T02:13:29.872 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow profile osd", 2023-11-06T02:13:29.873 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-11-06T02:13:29.873 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:13:29.873 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:13:29.873 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:13:29.873 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "osd.7", 2023-11-06T02:13:29.874 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQB2SkhlQbCYDRAAhE7vwbjJjqZDHfgDfdy1cA==", 2023-11-06T02:13:29.874 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-11-06T02:13:29.874 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow profile osd", 2023-11-06T02:13:29.874 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow profile osd", 2023-11-06T02:13:29.874 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-11-06T02:13:29.875 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:13:29.875 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:13:29.875 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:13:29.875 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "client.0", 2023-11-06T02:13:29.875 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQBcSkhlY46mAxAALClTH91L6USom3nVss6iiQ==", 2023-11-06T02:13:29.876 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-11-06T02:13:29.876 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": "allow", 2023-11-06T02:13:29.876 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow r", 2023-11-06T02:13:29.876 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-06T02:13:29.877 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:13:29.877 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:13:29.877 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:13:29.877 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "client.1", 2023-11-06T02:13:29.877 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQBcSkhllvSyCxAAfcSiBhHHNgaIr9mhpY11fg==", 2023-11-06T02:13:29.878 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-11-06T02:13:29.878 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": "allow", 2023-11-06T02:13:29.878 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow r", 2023-11-06T02:13:29.878 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow rw", 2023-11-06T02:13:29.878 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow rwx" 2023-11-06T02:13:29.879 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:13:29.879 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:13:29.879 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:13:29.879 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "client.2", 2023-11-06T02:13:29.880 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQBcSkhlURf6CBAA9g2y/UMD4x42IT0fgSgIcg==", 2023-11-06T02:13:29.880 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-11-06T02:13:29.880 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": "allow", 2023-11-06T02:13:29.880 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow r", 2023-11-06T02:13:29.880 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow rw", 2023-11-06T02:13:29.881 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow rwx" 2023-11-06T02:13:29.881 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:13:29.881 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:13:29.881 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:13:29.882 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "client.3", 2023-11-06T02:13:29.882 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQBcSkhlbjXXEBAA0xQGCY/avAWL18GLQtIjgg==", 2023-11-06T02:13:29.882 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-11-06T02:13:29.882 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": "allow", 2023-11-06T02:13:29.883 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow r", 2023-11-06T02:13:29.883 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow rw", 2023-11-06T02:13:29.883 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow rwx" 2023-11-06T02:13:29.883 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:13:29.884 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:13:29.884 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:13:29.884 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "client.admin", 2023-11-06T02:13:29.884 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQBbSkhloSJXCxAA++ZHscsqHENk8soyqq/GPQ==", 2023-11-06T02:13:29.884 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-11-06T02:13:29.885 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": "allow *", 2023-11-06T02:13:29.885 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": "allow *", 2023-11-06T02:13:29.885 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow *", 2023-11-06T02:13:29.885 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-11-06T02:13:29.886 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:13:29.886 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:13:29.886 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:13:29.886 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "mgr.x", 2023-11-06T02:13:29.887 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQBbSkhlpJ4iHxAAGEz4XS5fILy7/NZbAenPFw==", 2023-11-06T02:13:29.887 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-11-06T02:13:29.887 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": "allow *", 2023-11-06T02:13:29.887 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow profile mgr", 2023-11-06T02:13:29.887 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-11-06T02:13:29.888 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:13:29.888 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:13:29.888 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:13:29.888 INFO:teuthology.orchestra.run.smithi049.stdout: "entity": "mgr.y", 2023-11-06T02:13:29.889 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "AQBbSkhln1JuHBAAymHRkniPpsI6pf9L2C2y+g==", 2023-11-06T02:13:29.889 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": { 2023-11-06T02:13:29.889 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": "allow *", 2023-11-06T02:13:29.889 INFO:teuthology.orchestra.run.smithi049.stdout: "mon": "allow profile mgr", 2023-11-06T02:13:29.890 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": "allow *" 2023-11-06T02:13:29.890 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:13:29.890 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:13:29.890 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2023-11-06T02:13:29.891 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-11-06T02:13:29.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.854+0000 7fcc3c76f640 1 -- 172.21.15.49:0/2756461845 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fcc18065a60 msgr2=0x7fcc18067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:29.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.854+0000 7fcc3c76f640 1 --2- 172.21.15.49:0/2756461845 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fcc18065a60 0x7fcc18067f20 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fcc30009a10 tx=0x7fcc30008040 comp rx=0 tx=0).stop 2023-11-06T02:13:29.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.854+0000 7fcc3c76f640 1 -- 172.21.15.49:0/2756461845 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcc34151e10 msgr2=0x7fcc34152290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:29.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.854+0000 7fcc3c76f640 1 --2- 172.21.15.49:0/2756461845 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcc34151e10 0x7fcc34152290 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c0b0040 tx=0x7fcc20005800 comp rx=0 tx=0).stop 2023-11-06T02:13:29.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.854+0000 7fcc3c76f640 1 -- 172.21.15.49:0/2756461845 shutdown_connections 2023-11-06T02:13:29.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.854+0000 7fcc3c76f640 1 --2- 172.21.15.49:0/2756461845 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fcc18065a60 0x7fcc18067f20 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:29.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.854+0000 7fcc3c76f640 1 --2- 172.21.15.49:0/2756461845 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcc34151e10 0x7fcc34152290 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:29.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.854+0000 7fcc3c76f640 1 --2- 172.21.15.49:0/2756461845 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc3413cb40 0x7fcc3413cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:29.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.854+0000 7fcc3c76f640 1 --2- 172.21.15.49:0/2756461845 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcc3413de40 0x7fcc3413c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:29.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.854+0000 7fcc3c76f640 1 -- 172.21.15.49:0/2756461845 >> 172.21.15.49:0/2756461845 conn(0x7fcc3406c0a0 msgr2=0x7fcc340c2430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:29.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.854+0000 7fcc3c76f640 1 -- 172.21.15.49:0/2756461845 shutdown_connections 2023-11-06T02:13:29.895 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:29.854+0000 7fcc3c76f640 1 -- 172.21.15.49:0/2756461845 wait complete. 2023-11-06T02:13:29.895 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T02:13:30.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.026+0000 7f6f976a8640 1 Processor -- start 2023-11-06T02:13:30.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.030+0000 7f6f976a8640 1 -- start start 2023-11-06T02:13:30.026 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.030+0000 7f6f976a8640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6f9015cd70 0x7f6f9006b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:30.026 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.030+0000 7f6f976a8640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f6f900b0830 0x7f6f9006b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:30.027 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.030+0000 7f6f976a8640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f900aff70 con 0x7f6f900cebe0 2023-11-06T02:13:30.027 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.030+0000 7f6f976a8640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f900b0dd0 con 0x7f6f900b0830 2023-11-06T02:13:30.027 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.030+0000 7f6f976a8640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f900b10b0 con 0x7f6f9015cd70 2023-11-06T02:13:30.027 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.030+0000 7f6f94c43640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f6f900b0830 0x7f6f9006b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:30.028 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.030+0000 7f6f95444640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6f9015cd70 0x7f6f9006b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:30.028 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.030+0000 7f6f95c45640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f6f900cebe0 0x7f6f90160740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.49:42226/0 (socket says 172.21.15.49:42226) 2023-11-06T02:13:30.028 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.030+0000 7f6f95c45640 1 -- 172.21.15.49:0/318490114 learned_addr learned my addr 172.21.15.49:0/318490114 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:13:30.030 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.030+0000 7f6f95444640 1 -- 172.21.15.49:0/318490114 >> v2:172.21.15.150:3300/0 conn(0x7f6f900b0830 msgr2=0x7f6f9006b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:30.030 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.030+0000 7f6f95444640 1 --2- 172.21.15.49:0/318490114 >> v2:172.21.15.150:3300/0 conn(0x7f6f900b0830 0x7f6f9006b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:30.030 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.030+0000 7f6f95444640 1 -- 172.21.15.49:0/318490114 >> v1:172.21.15.150:6789/0 conn(0x7f6f900cebe0 legacy=0x7f6f90160740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:30.031 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.030+0000 7f6f95444640 1 -- 172.21.15.49:0/318490114 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f900b1390 con 0x7f6f9015cd70 2023-11-06T02:13:30.031 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.030+0000 7f6f94c43640 1 --2- 172.21.15.49:0/318490114 >> v2:172.21.15.150:3300/0 conn(0x7f6f900b0830 0x7f6f9006b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T02:13:30.031 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.034+0000 7f6f95444640 1 --2- 172.21.15.49:0/318490114 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6f9015cd70 0x7f6f9006b220 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c009de0 tx=0x7f6f8c01c870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a1d2b10d9797889 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:30.031 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.034+0000 7f6f87fff640 1 -- 172.21.15.49:0/318490114 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f8c0a5950 con 0x7f6f9015cd70 2023-11-06T02:13:30.032 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.034+0000 7f6f87fff640 1 -- 172.21.15.49:0/318490114 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f8c0a5ab0 con 0x7f6f9015cd70 2023-11-06T02:13:30.032 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.034+0000 7f6f87fff640 1 -- 172.21.15.49:0/318490114 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f8c0a5e40 con 0x7f6f9015cd70 2023-11-06T02:13:30.032 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.034+0000 7f6f976a8640 1 -- 172.21.15.49:0/318490114 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6f9015cd70 msgr2=0x7f6f9006b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:30.032 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.034+0000 7f6f976a8640 1 --2- 172.21.15.49:0/318490114 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6f9015cd70 0x7f6f9006b220 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c009de0 tx=0x7f6f8c01c870 comp rx=0 tx=0).stop 2023-11-06T02:13:30.033 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.034+0000 7f6f976a8640 1 -- 172.21.15.49:0/318490114 shutdown_connections 2023-11-06T02:13:30.033 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.034+0000 7f6f976a8640 1 --2- 172.21.15.49:0/318490114 >> v2:172.21.15.150:3300/0 conn(0x7f6f900b0830 0x7f6f9006b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:30.033 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.034+0000 7f6f976a8640 1 --2- 172.21.15.49:0/318490114 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6f9015cd70 0x7f6f9006b220 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:30.033 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.034+0000 7f6f976a8640 1 -- 172.21.15.49:0/318490114 >> 172.21.15.49:0/318490114 conn(0x7f6f90089fd0 msgr2=0x7f6f90099c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:30.034 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.034+0000 7f6f976a8640 1 -- 172.21.15.49:0/318490114 shutdown_connections 2023-11-06T02:13:30.034 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.034+0000 7f6f976a8640 1 -- 172.21.15.49:0/318490114 wait complete. 2023-11-06T02:13:30.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.034+0000 7f6f976a8640 1 Processor -- start 2023-11-06T02:13:30.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.034+0000 7f6f976a8640 1 -- start start 2023-11-06T02:13:30.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.034+0000 7f6f976a8640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6f900b0830 0x7f6f901fd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:30.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.034+0000 7f6f976a8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6f900cebe0 0x7f6f900ae950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:30.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.034+0000 7f6f976a8640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6f9015cd70 0x7f6f900aee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:30.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.034+0000 7f6f976a8640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f901fdbe0 con 0x7f6f900cebe0 2023-11-06T02:13:30.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.034+0000 7f6f976a8640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9015ad90 con 0x7f6f900b0830 2023-11-06T02:13:30.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.034+0000 7f6f976a8640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f900b6240 con 0x7f6f9015cd70 2023-11-06T02:13:30.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.034+0000 7f6f94c43640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6f900cebe0 0x7f6f900ae950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:30.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.038+0000 7f6f94c43640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6f900cebe0 0x7f6f900ae950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51120/0 (socket says 172.21.15.49:51120) 2023-11-06T02:13:30.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.038+0000 7f6f94c43640 1 -- 172.21.15.49:0/916709780 learned_addr learned my addr 172.21.15.49:0/916709780 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:30.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.038+0000 7f6f95c45640 1 --2- 172.21.15.49:0/916709780 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6f9015cd70 0x7f6f900aee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:30.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.038+0000 7f6f95444640 1 --2- 172.21.15.49:0/916709780 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6f900b0830 0x7f6f901fd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:30.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.038+0000 7f6f94c43640 1 -- 172.21.15.49:0/916709780 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6f9015cd70 msgr2=0x7f6f900aee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:30.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.038+0000 7f6f94c43640 1 --2- 172.21.15.49:0/916709780 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6f9015cd70 0x7f6f900aee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:30.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.038+0000 7f6f94c43640 1 -- 172.21.15.49:0/916709780 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6f900b0830 msgr2=0x7f6f901fd6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:30.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.038+0000 7f6f94c43640 1 --2- 172.21.15.49:0/916709780 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6f900b0830 0x7f6f901fd6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:30.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.038+0000 7f6f94c43640 1 -- 172.21.15.49:0/916709780 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f80009420 con 0x7f6f900cebe0 2023-11-06T02:13:30.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.038+0000 7f6f94c43640 1 --2- 172.21.15.49:0/916709780 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6f900cebe0 0x7f6f900ae950 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f6f8000e980 tx=0x7f6f8000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:30.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.038+0000 7f6f85ffb640 1 -- 172.21.15.49:0/916709780 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f8000ca00 con 0x7f6f900cebe0 2023-11-06T02:13:30.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.038+0000 7f6f85ffb640 1 -- 172.21.15.49:0/916709780 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f8000cb60 con 0x7f6f900cebe0 2023-11-06T02:13:30.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.038+0000 7f6f976a8640 1 -- 172.21.15.49:0/916709780 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f8c009a70 con 0x7f6f900cebe0 2023-11-06T02:13:30.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.038+0000 7f6f85ffb640 1 -- 172.21.15.49:0/916709780 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f80013470 con 0x7f6f900cebe0 2023-11-06T02:13:30.042 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.038+0000 7f6f976a8640 1 -- 172.21.15.49:0/916709780 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f9015afe0 con 0x7f6f900cebe0 2023-11-06T02:13:30.042 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.038+0000 7f6f976a8640 1 -- 172.21.15.49:0/916709780 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f900b6760 con 0x7f6f900cebe0 2023-11-06T02:13:30.042 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.038+0000 7f6f85ffb640 1 -- 172.21.15.49:0/916709780 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f6f800135d0 con 0x7f6f900cebe0 2023-11-06T02:13:30.042 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.038+0000 7f6f85ffb640 1 --2- 172.21.15.49:0/916709780 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6f74065a60 0x7f6f74067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:30.043 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.038+0000 7f6f85ffb640 1 -- 172.21.15.49:0/916709780 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f6f800831b0 con 0x7f6f900cebe0 2023-11-06T02:13:30.043 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.038+0000 7f6f95444640 1 --2- 172.21.15.49:0/916709780 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6f74065a60 0x7f6f74067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:30.043 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.042+0000 7f6f95444640 1 --2- 172.21.15.49:0/916709780 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6f74065a60 0x7f6f74067f20 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c0af000 tx=0x7f6f8c0a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:30.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.042+0000 7f6f85ffb640 1 -- 172.21.15.49:0/916709780 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f6f80050df0 con 0x7f6f900cebe0 2023-11-06T02:13:30.199 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.202+0000 7f6f976a8640 1 -- 172.21.15.49:0/916709780 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6f9015c5a0 con 0x7f6f900cebe0 2023-11-06T02:13:30.200 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.206+0000 7f6f85ffb640 1 -- 172.21.15.49:0/916709780 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) v1 ==== 74+0+14172 (secure 0 0 0) 0x7f6f80054410 con 0x7f6f900cebe0 2023-11-06T02:13:30.200 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:13:30.201 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":76,"fsid":"1a8b035c-6ba4-499a-b876-aea47fd9c0d4","created":"2023-11-06T02:08:06.440857+0000","modified":"2023-11-06T02:13:29.464937+0000","last_up_change":"2023-11-06T02:08:17.982372+0000","last_in_change":"2023-11-06T02:08:08.906631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T02:08:19.252222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T02:08:24.776211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ad55af8-0571-4e6a-8722-b5d429b7496d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6809","nonce":38116},{"type":"v1","addr":"172.21.15.49:6811","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6813","nonce":38116},{"type":"v1","addr":"172.21.15.49:6815","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6821","nonce":38116},{"type":"v1","addr":"172.21.15.49:6823","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6817","nonce":38116},{"type":"v1","addr":"172.21.15.49:6819","nonce":38116}]},"public_addr":"172.21.15.49:6811/38116","cluster_addr":"172.21.15.49:6815/38116","heartbeat_back_addr":"172.21.15.49:6823/38116","heartbeat_front_addr":"172.21.15.49:6819/38116","state":["exists","up"]},{"osd":1,"uuid":"c8b31f85-a757-43ed-9419-351acc46611b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":38113},{"type":"v1","addr":"172.21.15.49:6810","nonce":38113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":38113},{"type":"v1","addr":"172.21.15.49:6814","nonce":38113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":38113},{"type":"v1","addr":"172.21.15.49:6822","nonce":38113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":38113},{"type":"v1","addr":"172.21.15.49:6818","nonce":38113}]},"public_addr":"172.21.15.49:6810/38113","cluster_addr":"172.21.15.49:6814/38113","heartbeat_back_addr":"172.21.15.49:6822/38113","heartbeat_front_addr":"172.21.15.49:6818/38113","state":["exists","up"]},{"osd":2,"uuid":"555b3bf3-0a70-4b91-8efd-e4204b5fc26e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":38114},{"type":"v1","addr":"172.21.15.49:6801","nonce":38114}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":38114},{"type":"v1","addr":"172.21.15.49:6803","nonce":38114}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":38114},{"type":"v1","addr":"172.21.15.49:6807","nonce":38114}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":38114},{"type":"v1","addr":"172.21.15.49:6805","nonce":38114}]},"public_addr":"172.21.15.49:6801/38114","cluster_addr":"172.21.15.49:6803/38114","heartbeat_back_addr":"172.21.15.49:6807/38114","heartbeat_front_addr":"172.21.15.49:6805/38114","state":["exists","up"]},{"osd":3,"uuid":"8d4e5d77-03ef-4b9b-bfcb-5ae93a46795c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":38115},{"type":"v1","addr":"172.21.15.49:6825","nonce":38115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":38115},{"type":"v1","addr":"172.21.15.49:6827","nonce":38115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":38115},{"type":"v1","addr":"172.21.15.49:6831","nonce":38115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":38115},{"type":"v1","addr":"172.21.15.49:6829","nonce":38115}]},"public_addr":"172.21.15.49:6825/38115","cluster_addr":"172.21.15.49:6827/38115","heartbeat_back_addr":"172.21.15.49:6831/38115","heartbeat_front_addr":"172.21.15.49:6829/38115","state":["exists","up"]},{"osd":4,"uuid":"dba03ad4-68a3-4cf3-9218-ac611f02eacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":38417},{"type":"v1","addr":"172.21.15.150:6809","nonce":38417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":38417},{"type":"v1","addr":"172.21.15.150:6811","nonce":38417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":38417},{"type":"v1","addr":"172.21.15.150:6815","nonce":38417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":38417},{"type":"v1","addr":"172.21.15.150:6813","nonce":38417}]},"public_addr":"172.21.15.150:6809/38417","cluster_addr":"172.21.15.150:6811/38417","heartbeat_back_addr":"172.21.15.150:6815/38417","heartbeat_front_addr":"172.21.15.150:6813/38417","state":["exists","up"]},{"osd":5,"uuid":"0df9c303-c255-4640-8e91-01f1c3efe18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":38418},{"type":"v1","addr":"172.21.15.150:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":38418},{"type":"v1","addr":"172.21.15.150:6803","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":38418},{"type":"v1","addr":"172.21.15.150:6807","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":38418},{"type":"v1","addr":"172.21.15.150:6805","nonce":38418}]},"public_addr":"172.21.15.150:6801/38418","cluster_addr":"172.21.15.150:6803/38418","heartbeat_back_addr":"172.21.15.150:6807/38418","heartbeat_front_addr":"172.21.15.150:6805/38418","state":["exists","up"]},{"osd":6,"uuid":"e59fa189-4b70-4c06-9722-c02b90828bf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":38428},{"type":"v1","addr":"172.21.15.150:6825","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":38428},{"type":"v1","addr":"172.21.15.150:6827","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":38428},{"type":"v1","addr":"172.21.15.150:6831","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":38428},{"type":"v1","addr":"172.21.15.150:6829","nonce":38428}]},"public_addr":"172.21.15.150:6825/38428","cluster_addr":"172.21.15.150:6827/38428","heartbeat_back_addr":"172.21.15.150:6831/38428","heartbeat_front_addr":"172.21.15.150:6829/38428","state":["exists","up"]},{"osd":7,"uuid":"a64f57e1-ef98-46bd-b304-385759e95898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":38421},{"type":"v1","addr":"172.21.15.150:6817","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":38421},{"type":"v1","addr":"172.21.15.150:6819","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":38421},{"type":"v1","addr":"172.21.15.150:6823","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":38421},{"type":"v1","addr":"172.21.15.150:6821","nonce":38421}]},"public_addr":"172.21.15.150:6817/38421","cluster_addr":"172.21.15.150:6819/38421","heartbeat_back_addr":"172.21.15.150:6823/38421","heartbeat_front_addr":"172.21.15.150:6821/38421","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.438552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.413280+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.296466+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.424966+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.276953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.230984+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.410235+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.480084+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T02:13:30.203 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.206+0000 7f6f976a8640 1 -- 172.21.15.49:0/916709780 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6f74065a60 msgr2=0x7f6f74067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:30.203 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.206+0000 7f6f976a8640 1 --2- 172.21.15.49:0/916709780 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6f74065a60 0x7f6f74067f20 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c0af000 tx=0x7f6f8c0a7070 comp rx=0 tx=0).stop 2023-11-06T02:13:30.203 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.206+0000 7f6f976a8640 1 -- 172.21.15.49:0/916709780 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6f900cebe0 msgr2=0x7f6f900ae950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:30.204 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.206+0000 7f6f976a8640 1 --2- 172.21.15.49:0/916709780 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6f900cebe0 0x7f6f900ae950 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f6f8000e980 tx=0x7f6f8000ee40 comp rx=0 tx=0).stop 2023-11-06T02:13:30.204 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.206+0000 7f6f976a8640 1 -- 172.21.15.49:0/916709780 shutdown_connections 2023-11-06T02:13:30.204 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.206+0000 7f6f976a8640 1 --2- 172.21.15.49:0/916709780 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6f74065a60 0x7f6f74067f20 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:30.204 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.206+0000 7f6f976a8640 1 --2- 172.21.15.49:0/916709780 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6f900cebe0 0x7f6f900ae950 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:30.205 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.206+0000 7f6f976a8640 1 --2- 172.21.15.49:0/916709780 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6f9015cd70 0x7f6f900aee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:30.205 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.206+0000 7f6f976a8640 1 --2- 172.21.15.49:0/916709780 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6f900b0830 0x7f6f901fd6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:30.205 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.206+0000 7f6f976a8640 1 -- 172.21.15.49:0/916709780 >> 172.21.15.49:0/916709780 conn(0x7f6f90089fd0 msgr2=0x7f6f9006e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:30.205 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.206+0000 7f6f976a8640 1 -- 172.21.15.49:0/916709780 shutdown_connections 2023-11-06T02:13:30.205 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.206+0000 7f6f976a8640 1 -- 172.21.15.49:0/916709780 wait complete. 2023-11-06T02:13:30.220 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T02:08:19.252222+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T02:08:24.776211+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T02:13:30.221 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T02:13:30.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.346+0000 7fd9928b9640 1 Processor -- start 2023-11-06T02:13:30.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.346+0000 7fd9928b9640 1 -- start start 2023-11-06T02:13:30.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.346+0000 7fd9928b9640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd98c070870 0x7fd98c0b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:30.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.346+0000 7fd9928b9640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fd98c06d150 0x7fd98c06d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:30.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.346+0000 7fd9928b9640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd98c06db20 con 0x7fd98c15a880 2023-11-06T02:13:30.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.346+0000 7fd9928b9640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd98c0ae360 con 0x7fd98c06d150 2023-11-06T02:13:30.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.346+0000 7fd9928b9640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd98c0ae640 con 0x7fd98c070870 2023-11-06T02:13:30.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.346+0000 7fd98bfff640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fd98c15a880 0x7fd98c070160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:60510/0 (socket says 172.21.15.49:60510) 2023-11-06T02:13:30.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.346+0000 7fd98bfff640 1 -- 172.21.15.49:0/229538976 learned_addr learned my addr 172.21.15.49:0/229538976 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:13:30.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.346+0000 7fd983fff640 1 --2- 172.21.15.49:0/229538976 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd98c070870 0x7fd98c0b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:30.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.346+0000 7fd990e56640 1 --2- 172.21.15.49:0/229538976 >> v2:172.21.15.150:3300/0 conn(0x7fd98c06d150 0x7fd98c06d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:30.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.346+0000 7fd98b7fe640 1 -- 172.21.15.49:0/229538976 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3890773934 0 0) 0x7fd98c06db20 con 0x7fd98c15a880 2023-11-06T02:13:30.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.346+0000 7fd98b7fe640 1 -- 172.21.15.49:0/229538976 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd978003530 con 0x7fd98c15a880 2023-11-06T02:13:30.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.346+0000 7fd98b7fe640 1 -- 172.21.15.49:0/229538976 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2640177743 0 0) 0x7fd978003530 con 0x7fd98c15a880 2023-11-06T02:13:30.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.346+0000 7fd98b7fe640 1 -- 172.21.15.49:0/229538976 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd98c070870 msgr2=0x7fd98c0b15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:30.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.346+0000 7fd98b7fe640 1 --2- 172.21.15.49:0/229538976 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd98c070870 0x7fd98c0b15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:30.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.346+0000 7fd98b7fe640 1 -- 172.21.15.49:0/229538976 >> v2:172.21.15.150:3300/0 conn(0x7fd98c06d150 msgr2=0x7fd98c06d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:30.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.346+0000 7fd98b7fe640 1 --2- 172.21.15.49:0/229538976 >> v2:172.21.15.150:3300/0 conn(0x7fd98c06d150 0x7fd98c06d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:30.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.346+0000 7fd98b7fe640 1 -- 172.21.15.49:0/229538976 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd98c0ae920 con 0x7fd98c15a880 2023-11-06T02:13:30.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.346+0000 7fd98b7fe640 1 -- 172.21.15.49:0/229538976 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7fd974001680 con 0x7fd98c15a880 2023-11-06T02:13:30.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.346+0000 7fd983fff640 1 --2- 172.21.15.49:0/229538976 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd98c070870 0x7fd98c0b15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:13:30.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.346+0000 7fd98b7fe640 1 -- 172.21.15.49:0/229538976 >> v1:172.21.15.49:6789/0 conn(0x7fd98c15a880 legacy=0x7fd98c070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:30.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.346+0000 7fd98b7fe640 1 --2- 172.21.15.49:0/229538976 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd98c070870 0x7fd978003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:30.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.346+0000 7fd98b7fe640 1 --2- 172.21.15.49:0/229538976 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd978003ec0 0x7fd9780047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:30.350 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.346+0000 7fd98b7fe640 1 --2- 172.21.15.49:0/229538976 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd978004d10 0x7fd978005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:30.350 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.346+0000 7fd98b7fe640 1 -- 172.21.15.49:0/229538976 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd98c0ae360 con 0x7fd978004d10 2023-11-06T02:13:30.350 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.346+0000 7fd98b7fe640 1 -- 172.21.15.49:0/229538976 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd98c0ae640 con 0x7fd978003ec0 2023-11-06T02:13:30.350 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.346+0000 7fd98b7fe640 1 -- 172.21.15.49:0/229538976 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd97800b9c0 con 0x7fd98c070870 2023-11-06T02:13:30.351 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.346+0000 7fd990e56640 1 --2- 172.21.15.49:0/229538976 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd978004d10 0x7fd978005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:30.351 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.346+0000 7fd98bfff640 1 --2- 172.21.15.49:0/229538976 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd98c070870 0x7fd978003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:30.351 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.346+0000 7fd983fff640 1 --2- 172.21.15.49:0/229538976 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd978003ec0 0x7fd9780047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:30.351 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.346+0000 7fd990e56640 1 -- 172.21.15.49:0/229538976 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd98c070870 msgr2=0x7fd978003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:30.351 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.346+0000 7fd990e56640 1 --2- 172.21.15.49:0/229538976 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd98c070870 0x7fd978003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:30.352 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.346+0000 7fd990e56640 1 -- 172.21.15.49:0/229538976 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd978003ec0 msgr2=0x7fd9780047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:30.352 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.346+0000 7fd990e56640 1 --2- 172.21.15.49:0/229538976 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd978003ec0 0x7fd9780047d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:30.352 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.346+0000 7fd990e56640 1 -- 172.21.15.49:0/229538976 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd978003530 con 0x7fd978004d10 2023-11-06T02:13:30.352 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.350+0000 7fd990e56640 1 --2- 172.21.15.49:0/229538976 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd978004d10 0x7fd978005190 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fd984009fc0 tx=0x7fd98401cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=921eab23ee7345d5 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:30.352 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.350+0000 7fd98b7fe640 1 -- 172.21.15.49:0/229538976 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9840a5cb0 con 0x7fd978004d10 2023-11-06T02:13:30.352 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.350+0000 7fd98b7fe640 1 -- 172.21.15.49:0/229538976 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd98400f040 con 0x7fd978004d10 2023-11-06T02:13:30.353 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.350+0000 7fd98b7fe640 1 -- 172.21.15.49:0/229538976 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9840ad680 con 0x7fd978004d10 2023-11-06T02:13:30.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.350+0000 7fd9928b9640 1 -- 172.21.15.49:0/229538976 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd978004d10 msgr2=0x7fd978005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:30.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.350+0000 7fd9928b9640 1 --2- 172.21.15.49:0/229538976 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd978004d10 0x7fd978005190 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fd984009fc0 tx=0x7fd98401cca0 comp rx=0 tx=0).stop 2023-11-06T02:13:30.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.350+0000 7fd990e56640 1 -- 172.21.15.49:0/229538976 reap_dead start 2023-11-06T02:13:30.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.350+0000 7fd9928b9640 1 -- 172.21.15.49:0/229538976 shutdown_connections 2023-11-06T02:13:30.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.350+0000 7fd9928b9640 1 -- 172.21.15.49:0/229538976 >> 172.21.15.49:0/229538976 conn(0x7fd98c06a370 msgr2=0x7fd98c0ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:30.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.350+0000 7fd9928b9640 1 -- 172.21.15.49:0/229538976 shutdown_connections 2023-11-06T02:13:30.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.350+0000 7fd9928b9640 1 -- 172.21.15.49:0/229538976 wait complete. 2023-11-06T02:13:30.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.350+0000 7fd9928b9640 1 Processor -- start 2023-11-06T02:13:30.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.350+0000 7fd9928b9640 1 -- start start 2023-11-06T02:13:30.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.350+0000 7fd9928b9640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd98c1511a0 0x7fd98c150090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:30.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.350+0000 7fd9928b9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd98c15b980 0x7fd98c144d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:30.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.350+0000 7fd9928b9640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd98c151580 0x7fd98c145240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:30.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.350+0000 7fd9928b9640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd98c15c990 con 0x7fd98c15b980 2023-11-06T02:13:30.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.350+0000 7fd9928b9640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd98c145780 con 0x7fd98c1511a0 2023-11-06T02:13:30.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.350+0000 7fd9928b9640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd98c15be10 con 0x7fd98c151580 2023-11-06T02:13:30.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.350+0000 7fd983fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd98c15b980 0x7fd98c144d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:30.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.350+0000 7fd983fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd98c15b980 0x7fd98c144d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51132/0 (socket says 172.21.15.49:51132) 2023-11-06T02:13:30.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.350+0000 7fd983fff640 1 -- 172.21.15.49:0/1265235049 learned_addr learned my addr 172.21.15.49:0/1265235049 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:30.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.350+0000 7fd990e56640 1 --2- 172.21.15.49:0/1265235049 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd98c151580 0x7fd98c145240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:30.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.350+0000 7fd98bfff640 1 --2- 172.21.15.49:0/1265235049 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd98c1511a0 0x7fd98c150090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:30.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.354+0000 7fd983fff640 1 -- 172.21.15.49:0/1265235049 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd98c151580 msgr2=0x7fd98c145240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:30.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.354+0000 7fd983fff640 1 --2- 172.21.15.49:0/1265235049 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd98c151580 0x7fd98c145240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:30.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.354+0000 7fd983fff640 1 -- 172.21.15.49:0/1265235049 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd98c1511a0 msgr2=0x7fd98c150090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:30.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.354+0000 7fd983fff640 1 --2- 172.21.15.49:0/1265235049 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd98c1511a0 0x7fd98c150090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:30.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.354+0000 7fd983fff640 1 -- 172.21.15.49:0/1265235049 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd98c15ade0 con 0x7fd98c15b980 2023-11-06T02:13:30.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.354+0000 7fd98bfff640 1 --2- 172.21.15.49:0/1265235049 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd98c1511a0 0x7fd98c150090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:13:30.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.354+0000 7fd990e56640 1 --2- 172.21.15.49:0/1265235049 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd98c151580 0x7fd98c145240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:13:30.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.354+0000 7fd983fff640 1 --2- 172.21.15.49:0/1265235049 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd98c15b980 0x7fd98c144d00 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fd97c00f510 tx=0x7fd97c00f9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:30.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.354+0000 7fd9897fa640 1 -- 172.21.15.49:0/1265235049 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd97c014890 con 0x7fd98c15b980 2023-11-06T02:13:30.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.354+0000 7fd9897fa640 1 -- 172.21.15.49:0/1265235049 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd97c00c030 con 0x7fd98c15b980 2023-11-06T02:13:30.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.354+0000 7fd9897fa640 1 -- 172.21.15.49:0/1265235049 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd97c014c30 con 0x7fd98c15b980 2023-11-06T02:13:30.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.354+0000 7fd9928b9640 1 -- 172.21.15.49:0/1265235049 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd98c15c0f0 con 0x7fd98c15b980 2023-11-06T02:13:30.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.354+0000 7fd9928b9640 1 -- 172.21.15.49:0/1265235049 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd98c15c6b0 con 0x7fd98c15b980 2023-11-06T02:13:30.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.354+0000 7fd9928b9640 1 -- 172.21.15.49:0/1265235049 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd98c074040 con 0x7fd98c15b980 2023-11-06T02:13:30.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.354+0000 7fd9897fa640 1 -- 172.21.15.49:0/1265235049 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fd97c0133e0 con 0x7fd98c15b980 2023-11-06T02:13:30.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.354+0000 7fd9897fa640 1 --2- 172.21.15.49:0/1265235049 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd96c065a60 0x7fd96c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:30.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.354+0000 7fd9897fa640 1 -- 172.21.15.49:0/1265235049 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fd97c083220 con 0x7fd98c15b980 2023-11-06T02:13:30.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.354+0000 7fd98bfff640 1 --2- 172.21.15.49:0/1265235049 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd96c065a60 0x7fd96c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:30.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.354+0000 7fd98bfff640 1 --2- 172.21.15.49:0/1265235049 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd96c065a60 0x7fd96c067f20 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fd974001060 tx=0x7fd974001180 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:30.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.358+0000 7fd9897fa640 1 -- 172.21.15.49:0/1265235049 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fd97c050e60 con 0x7fd98c15b980 2023-11-06T02:13:30.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.514+0000 7fd9928b9640 1 -- 172.21.15.49:0/1265235049 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd98c06c430 con 0x7fd98c15b980 2023-11-06T02:13:30.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.514+0000 7fd9897fa640 1 -- 172.21.15.49:0/1265235049 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v76) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd97c054480 con 0x7fd98c15b980 2023-11-06T02:13:30.510 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2023-11-06T02:13:30.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.514+0000 7fd9928b9640 1 -- 172.21.15.49:0/1265235049 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd96c065a60 msgr2=0x7fd96c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:30.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.514+0000 7fd9928b9640 1 --2- 172.21.15.49:0/1265235049 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd96c065a60 0x7fd96c067f20 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fd974001060 tx=0x7fd974001180 comp rx=0 tx=0).stop 2023-11-06T02:13:30.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.514+0000 7fd9928b9640 1 -- 172.21.15.49:0/1265235049 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd98c15b980 msgr2=0x7fd98c144d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:30.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.514+0000 7fd9928b9640 1 --2- 172.21.15.49:0/1265235049 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd98c15b980 0x7fd98c144d00 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fd97c00f510 tx=0x7fd97c00f9d0 comp rx=0 tx=0).stop 2023-11-06T02:13:30.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.514+0000 7fd9928b9640 1 -- 172.21.15.49:0/1265235049 shutdown_connections 2023-11-06T02:13:30.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.514+0000 7fd9928b9640 1 --2- 172.21.15.49:0/1265235049 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd96c065a60 0x7fd96c067f20 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:30.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.514+0000 7fd9928b9640 1 --2- 172.21.15.49:0/1265235049 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd98c15b980 0x7fd98c144d00 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:30.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.514+0000 7fd9928b9640 1 --2- 172.21.15.49:0/1265235049 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd98c151580 0x7fd98c145240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:30.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.514+0000 7fd9928b9640 1 --2- 172.21.15.49:0/1265235049 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd98c1511a0 0x7fd98c150090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:30.514 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.514+0000 7fd9928b9640 1 -- 172.21.15.49:0/1265235049 >> 172.21.15.49:0/1265235049 conn(0x7fd98c06a370 msgr2=0x7fd98c0b3ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:30.514 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.514+0000 7fd9928b9640 1 -- 172.21.15.49:0/1265235049 shutdown_connections 2023-11-06T02:13:30.514 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.514+0000 7fd9928b9640 1 -- 172.21.15.49:0/1265235049 wait complete. 2023-11-06T02:13:30.527 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T02:13:30.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.650+0000 7fded1877640 1 Processor -- start 2023-11-06T02:13:30.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.650+0000 7fded1877640 1 -- start start 2023-11-06T02:13:30.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.650+0000 7fded1877640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fdecc1595a0 0x7fdecc15dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:30.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.650+0000 7fded1877640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fdecc06d640 0x7fdecc0b0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:30.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.650+0000 7fded1877640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdecc06c5a0 con 0x7fdecc0b0760 2023-11-06T02:13:30.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.650+0000 7fded1877640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdecc15e250 con 0x7fdecc06d640 2023-11-06T02:13:30.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.650+0000 7fded1877640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdecc06c880 con 0x7fdecc1595a0 2023-11-06T02:13:30.648 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.650+0000 7fdeca7fc640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fdecc06d640 0x7fdecc0b0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:30.648 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.650+0000 7fdeca7fc640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fdecc06d640 0x7fdecc0b0220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51144/0 (socket says 172.21.15.49:51144) 2023-11-06T02:13:30.648 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.650+0000 7fdeca7fc640 1 -- 172.21.15.49:0/1678590796 learned_addr learned my addr 172.21.15.49:0/1678590796 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:30.648 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.650+0000 7fdecaffd640 1 --2- 172.21.15.49:0/1678590796 >> v2:172.21.15.150:3300/0 conn(0x7fdecc1595a0 0x7fdecc15dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:30.648 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.650+0000 7fdeca7fc640 1 -- 172.21.15.49:0/1678590796 >> v2:172.21.15.150:3300/0 conn(0x7fdecc1595a0 msgr2=0x7fdecc15dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:30.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.650+0000 7fdeca7fc640 1 --2- 172.21.15.49:0/1678590796 >> v2:172.21.15.150:3300/0 conn(0x7fdecc1595a0 0x7fdecc15dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:30.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.650+0000 7fdeca7fc640 1 -- 172.21.15.49:0/1678590796 >> v1:172.21.15.150:6789/0 conn(0x7fdecc0b0760 legacy=0x7fdecc0c8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:30.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.650+0000 7fdeca7fc640 1 -- 172.21.15.49:0/1678590796 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdecc06cb60 con 0x7fdecc06d640 2023-11-06T02:13:30.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.650+0000 7fdecaffd640 1 --2- 172.21.15.49:0/1678590796 >> v2:172.21.15.150:3300/0 conn(0x7fdecc1595a0 0x7fdecc15dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:13:30.650 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.654+0000 7fdeca7fc640 1 --2- 172.21.15.49:0/1678590796 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdecc06d640 0x7fdecc0b0220 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fdecc071420 tx=0x7fdec001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4165733ad14b8816 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:30.651 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.654+0000 7fdec9ffb640 1 -- 172.21.15.49:0/1678590796 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec00aa5f0 con 0x7fdecc06d640 2023-11-06T02:13:30.651 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.654+0000 7fdec9ffb640 1 -- 172.21.15.49:0/1678590796 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdec00aa750 con 0x7fdecc06d640 2023-11-06T02:13:30.651 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.654+0000 7fdec9ffb640 1 -- 172.21.15.49:0/1678590796 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec00aaae0 con 0x7fdecc06d640 2023-11-06T02:13:30.651 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.654+0000 7fded1877640 1 -- 172.21.15.49:0/1678590796 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdecc06d640 msgr2=0x7fdecc0b0220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:30.651 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.654+0000 7fded1877640 1 --2- 172.21.15.49:0/1678590796 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdecc06d640 0x7fdecc0b0220 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fdecc071420 tx=0x7fdec001c490 comp rx=0 tx=0).stop 2023-11-06T02:13:30.652 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.654+0000 7fded1877640 1 -- 172.21.15.49:0/1678590796 shutdown_connections 2023-11-06T02:13:30.652 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.654+0000 7fded1877640 1 --2- 172.21.15.49:0/1678590796 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdecc06d640 0x7fdecc0b0220 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:30.652 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.654+0000 7fded1877640 1 --2- 172.21.15.49:0/1678590796 >> v2:172.21.15.150:3300/0 conn(0x7fdecc1595a0 0x7fdecc15dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:30.652 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.654+0000 7fded1877640 1 -- 172.21.15.49:0/1678590796 >> 172.21.15.49:0/1678590796 conn(0x7fdecc06a370 msgr2=0x7fdecc0ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:30.652 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.654+0000 7fded1877640 1 -- 172.21.15.49:0/1678590796 shutdown_connections 2023-11-06T02:13:30.653 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.654+0000 7fded1877640 1 -- 172.21.15.49:0/1678590796 wait complete. 2023-11-06T02:13:30.653 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.654+0000 7fded1877640 1 Processor -- start 2023-11-06T02:13:30.653 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.654+0000 7fded1877640 1 -- start start 2023-11-06T02:13:30.655 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.654+0000 7fded1877640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdecc06d640 0x7fdecc14d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:30.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.654+0000 7fded1877640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdecc0b0760 0x7fdecc0cc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:30.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.654+0000 7fded1877640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdecc1595a0 0x7fdecc150060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:30.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.654+0000 7fded1877640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdecc0cda40 con 0x7fdecc1595a0 2023-11-06T02:13:30.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.654+0000 7fded1877640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdecc0cd1a0 con 0x7fdecc06d640 2023-11-06T02:13:30.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.654+0000 7fded1877640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdecc0cd480 con 0x7fdecc0b0760 2023-11-06T02:13:30.657 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.654+0000 7fdecb7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdecc1595a0 0x7fdecc150060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:30.657 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.654+0000 7fdecb7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdecc1595a0 0x7fdecc150060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51150/0 (socket says 172.21.15.49:51150) 2023-11-06T02:13:30.657 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.654+0000 7fdecb7fe640 1 -- 172.21.15.49:0/3723707981 learned_addr learned my addr 172.21.15.49:0/3723707981 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:30.657 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.654+0000 7fdeca7fc640 1 --2- 172.21.15.49:0/3723707981 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdecc0b0760 0x7fdecc0cc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:30.658 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.654+0000 7fdecaffd640 1 --2- 172.21.15.49:0/3723707981 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdecc06d640 0x7fdecc14d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:30.658 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.654+0000 7fdecb7fe640 1 -- 172.21.15.49:0/3723707981 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdecc0b0760 msgr2=0x7fdecc0cc8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:30.658 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.654+0000 7fdecb7fe640 1 --2- 172.21.15.49:0/3723707981 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdecc0b0760 0x7fdecc0cc8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:30.658 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.654+0000 7fdecb7fe640 1 -- 172.21.15.49:0/3723707981 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdecc06d640 msgr2=0x7fdecc14d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:30.658 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.654+0000 7fdecb7fe640 1 --2- 172.21.15.49:0/3723707981 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdecc06d640 0x7fdecc14d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:30.658 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.654+0000 7fdecb7fe640 1 -- 172.21.15.49:0/3723707981 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdec00094d0 con 0x7fdecc1595a0 2023-11-06T02:13:30.659 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.654+0000 7fdecb7fe640 1 --2- 172.21.15.49:0/3723707981 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdecc1595a0 0x7fdecc150060 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fdec400c780 tx=0x7fdec400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:30.659 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.658+0000 7fdeb3fff640 1 -- 172.21.15.49:0/3723707981 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec4004b70 con 0x7fdecc1595a0 2023-11-06T02:13:30.659 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.658+0000 7fdeb3fff640 1 -- 172.21.15.49:0/3723707981 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdec400ce90 con 0x7fdecc1595a0 2023-11-06T02:13:30.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.658+0000 7fded1877640 1 -- 172.21.15.49:0/3723707981 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdecc0cd760 con 0x7fdecc1595a0 2023-11-06T02:13:30.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.658+0000 7fdeb3fff640 1 -- 172.21.15.49:0/3723707981 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec400a850 con 0x7fdecc1595a0 2023-11-06T02:13:30.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.658+0000 7fded1877640 1 -- 172.21.15.49:0/3723707981 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdecc1505a0 con 0x7fdecc1595a0 2023-11-06T02:13:30.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.658+0000 7fded1877640 1 -- 172.21.15.49:0/3723707981 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdecc135450 con 0x7fdecc1595a0 2023-11-06T02:13:30.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.658+0000 7fdeb3fff640 1 -- 172.21.15.49:0/3723707981 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fdec400a9b0 con 0x7fdecc1595a0 2023-11-06T02:13:30.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.658+0000 7fdeb3fff640 1 --2- 172.21.15.49:0/3723707981 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fdeb8065a60 0x7fdeb8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:30.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.658+0000 7fdeb3fff640 1 -- 172.21.15.49:0/3723707981 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fdec4083160 con 0x7fdecc1595a0 2023-11-06T02:13:30.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.658+0000 7fdecaffd640 1 --2- 172.21.15.49:0/3723707981 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fdeb8065a60 0x7fdeb8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:30.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.658+0000 7fdecaffd640 1 --2- 172.21.15.49:0/3723707981 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fdeb8065a60 0x7fdeb8067f20 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fdebc006fd0 tx=0x7fdebc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:30.662 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.662+0000 7fdeb3fff640 1 -- 172.21.15.49:0/3723707981 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fdecc135450 con 0x7fdecc1595a0 2023-11-06T02:13:30.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.810+0000 7fded1877640 1 -- 172.21.15.49:0/3723707981 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fdecc135450 con 0x7fdecc1595a0 2023-11-06T02:13:30.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.810+0000 7fdeb3fff640 1 -- 172.21.15.49:0/3723707981 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v76) v1 ==== 92+0+10 (secure 0 0 0) 0x7fdecc135450 con 0x7fdecc1595a0 2023-11-06T02:13:30.806 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 8 2023-11-06T02:13:30.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.810+0000 7fded1877640 1 -- 172.21.15.49:0/3723707981 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fdeb8065a60 msgr2=0x7fdeb8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:30.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.810+0000 7fded1877640 1 --2- 172.21.15.49:0/3723707981 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fdeb8065a60 0x7fdeb8067f20 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fdebc006fd0 tx=0x7fdebc008040 comp rx=0 tx=0).stop 2023-11-06T02:13:30.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.810+0000 7fded1877640 1 -- 172.21.15.49:0/3723707981 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdecc1595a0 msgr2=0x7fdecc150060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:30.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.810+0000 7fded1877640 1 --2- 172.21.15.49:0/3723707981 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdecc1595a0 0x7fdecc150060 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fdec400c780 tx=0x7fdec400cc40 comp rx=0 tx=0).stop 2023-11-06T02:13:30.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.814+0000 7fded1877640 1 -- 172.21.15.49:0/3723707981 shutdown_connections 2023-11-06T02:13:30.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.814+0000 7fded1877640 1 --2- 172.21.15.49:0/3723707981 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fdeb8065a60 0x7fdeb8067f20 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:30.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.814+0000 7fded1877640 1 --2- 172.21.15.49:0/3723707981 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdecc1595a0 0x7fdecc150060 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:30.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.814+0000 7fded1877640 1 --2- 172.21.15.49:0/3723707981 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdecc0b0760 0x7fdecc0cc8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:30.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.814+0000 7fded1877640 1 --2- 172.21.15.49:0/3723707981 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdecc06d640 0x7fdecc14d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:30.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.814+0000 7fded1877640 1 -- 172.21.15.49:0/3723707981 >> 172.21.15.49:0/3723707981 conn(0x7fdecc06a370 msgr2=0x7fdecc152040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:30.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.814+0000 7fded1877640 1 -- 172.21.15.49:0/3723707981 shutdown_connections 2023-11-06T02:13:30.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.814+0000 7fded1877640 1 -- 172.21.15.49:0/3723707981 wait complete. 2023-11-06T02:13:30.824 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-06T02:13:30.935 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.938+0000 7f0e6b986640 1 Processor -- start 2023-11-06T02:13:30.936 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.938+0000 7f0e6b986640 1 -- start start 2023-11-06T02:13:30.938 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.942+0000 7f0e6b986640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0e6415aae0 0x7f0e64070a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:30.939 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.942+0000 7f0e6b986640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e640cacf0 con 0x7f0e640b0790 2023-11-06T02:13:30.939 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.942+0000 7f0e6b986640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e6415aec0 con 0x7f0e640ca830 2023-11-06T02:13:30.939 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.942+0000 7f0e6b986640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e64070fc0 con 0x7f0e6415aae0 2023-11-06T02:13:30.940 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.942+0000 7f0e69f23640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f0e640b0790 0x7f0e640b1450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:60526/0 (socket says 172.21.15.49:60526) 2023-11-06T02:13:30.940 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.942+0000 7f0e69f23640 1 -- 172.21.15.49:0/1886140700 learned_addr learned my addr 172.21.15.49:0/1886140700 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:13:30.940 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.942+0000 7f0e69722640 1 --2- 172.21.15.49:0/1886140700 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0e6415aae0 0x7f0e64070a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:30.940 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.942+0000 7f0e5bfff640 1 -- 172.21.15.49:0/1886140700 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2345003466 0 0) 0x7f0e640cacf0 con 0x7f0e640b0790 2023-11-06T02:13:30.940 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.942+0000 7f0e5bfff640 1 -- 172.21.15.49:0/1886140700 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e50003530 con 0x7f0e640b0790 2023-11-06T02:13:30.940 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.942+0000 7f0e5bfff640 1 -- 172.21.15.49:0/1886140700 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 781584824 0 0) 0x7f0e50003530 con 0x7f0e640b0790 2023-11-06T02:13:30.941 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.942+0000 7f0e5bfff640 1 -- 172.21.15.49:0/1886140700 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0e6415aae0 msgr2=0x7f0e64070a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:30.941 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.942+0000 7f0e5bfff640 1 --2- 172.21.15.49:0/1886140700 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0e6415aae0 0x7f0e64070a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:30.941 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.942+0000 7f0e5bfff640 1 -- 172.21.15.49:0/1886140700 >> v1:172.21.15.150:6789/0 conn(0x7f0e640ca830 legacy=0x7f0e640b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:30.941 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.942+0000 7f0e5bfff640 1 -- 172.21.15.49:0/1886140700 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e640c8ff0 con 0x7f0e640b0790 2023-11-06T02:13:30.942 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.942+0000 7f0e5bfff640 1 -- 172.21.15.49:0/1886140700 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7f0e5c001a00 con 0x7f0e640b0790 2023-11-06T02:13:30.942 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.942+0000 7f0e5bfff640 1 -- 172.21.15.49:0/1886140700 >> v1:172.21.15.49:6789/0 conn(0x7f0e640b0790 legacy=0x7f0e640b1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:30.942 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.942+0000 7f0e5bfff640 1 --2- 172.21.15.49:0/1886140700 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e500039b0 0x7f0e50003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:30.942 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.942+0000 7f0e5bfff640 1 --2- 172.21.15.49:0/1886140700 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0e6415aae0 0x7f0e500047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:30.942 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.942+0000 7f0e5bfff640 1 --2- 172.21.15.49:0/1886140700 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e50004d10 0x7f0e50005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:30.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.942+0000 7f0e5bfff640 1 -- 172.21.15.49:0/1886140700 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e5000ba90 con 0x7f0e50004d10 2023-11-06T02:13:30.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.942+0000 7f0e5bfff640 1 -- 172.21.15.49:0/1886140700 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e5000bc00 con 0x7f0e500039b0 2023-11-06T02:13:30.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.942+0000 7f0e5bfff640 1 -- 172.21.15.49:0/1886140700 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e5000bdc0 con 0x7f0e6415aae0 2023-11-06T02:13:30.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.942+0000 7f0e69f23640 1 --2- 172.21.15.49:0/1886140700 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e50004d10 0x7f0e50005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:30.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.942+0000 7f0e69722640 1 --2- 172.21.15.49:0/1886140700 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e500039b0 0x7f0e50003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:30.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.942+0000 7f0e68f21640 1 --2- 172.21.15.49:0/1886140700 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0e6415aae0 0x7f0e500047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:30.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.942+0000 7f0e69f23640 1 -- 172.21.15.49:0/1886140700 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0e6415aae0 msgr2=0x7f0e500047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:30.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.942+0000 7f0e69f23640 1 --2- 172.21.15.49:0/1886140700 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0e6415aae0 0x7f0e500047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:30.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.942+0000 7f0e69f23640 1 -- 172.21.15.49:0/1886140700 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e500039b0 msgr2=0x7f0e50003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:30.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.942+0000 7f0e69f23640 1 --2- 172.21.15.49:0/1886140700 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e500039b0 0x7f0e50003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:30.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.942+0000 7f0e69f23640 1 -- 172.21.15.49:0/1886140700 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e50003530 con 0x7f0e50004d10 2023-11-06T02:13:30.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.942+0000 7f0e69722640 1 --2- 172.21.15.49:0/1886140700 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e500039b0 0x7f0e50003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:13:30.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.942+0000 7f0e68f21640 1 --2- 172.21.15.49:0/1886140700 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0e6415aae0 0x7f0e500047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:13:30.948 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.942+0000 7f0e69f23640 1 --2- 172.21.15.49:0/1886140700 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e50004d10 0x7f0e50005190 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f0e640cf700 tx=0x7f0e5c01fa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2869ed446309500e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:30.949 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.942+0000 7f0e5bfff640 1 -- 172.21.15.49:0/1886140700 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e5c0aa670 con 0x7f0e50004d10 2023-11-06T02:13:30.949 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.942+0000 7f0e5bfff640 1 -- 172.21.15.49:0/1886140700 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e5c0aa7f0 con 0x7f0e50004d10 2023-11-06T02:13:30.949 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.942+0000 7f0e5bfff640 1 -- 172.21.15.49:0/1886140700 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e5c0aaba0 con 0x7f0e50004d10 2023-11-06T02:13:30.949 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.946+0000 7f0e6b986640 1 -- 172.21.15.49:0/1886140700 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e50004d10 msgr2=0x7f0e50005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:30.950 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.946+0000 7f0e6b986640 1 --2- 172.21.15.49:0/1886140700 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e50004d10 0x7f0e50005190 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f0e640cf700 tx=0x7f0e5c01fa40 comp rx=0 tx=0).stop 2023-11-06T02:13:30.950 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.946+0000 7f0e69f23640 1 -- 172.21.15.49:0/1886140700 reap_dead start 2023-11-06T02:13:30.950 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.946+0000 7f0e6b986640 1 -- 172.21.15.49:0/1886140700 shutdown_connections 2023-11-06T02:13:30.950 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.946+0000 7f0e6b986640 1 -- 172.21.15.49:0/1886140700 >> 172.21.15.49:0/1886140700 conn(0x7f0e6406c0a0 msgr2=0x7f0e6406c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:30.950 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.946+0000 7f0e6b986640 1 -- 172.21.15.49:0/1886140700 shutdown_connections 2023-11-06T02:13:30.951 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.946+0000 7f0e6b986640 1 -- 172.21.15.49:0/1886140700 wait complete. 2023-11-06T02:13:30.951 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.946+0000 7f0e6b986640 1 Processor -- start 2023-11-06T02:13:30.951 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.946+0000 7f0e6b986640 1 -- start start 2023-11-06T02:13:30.951 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.946+0000 7f0e6b986640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0e6415bb50 0x7f0e641f3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:30.951 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.946+0000 7f0e6b986640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e6415bf30 0x7f0e640b3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:30.951 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.946+0000 7f0e6b986640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e640c5780 0x7f0e640c5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:30.952 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.946+0000 7f0e6b986640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e640b2990 con 0x7f0e640c5780 2023-11-06T02:13:30.952 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.946+0000 7f0e6b986640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e640b2c70 con 0x7f0e6415bf30 2023-11-06T02:13:30.952 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.946+0000 7f0e6b986640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e640b2f50 con 0x7f0e6415bb50 2023-11-06T02:13:30.952 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.946+0000 7f0e69f23640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e640c5780 0x7f0e640c5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:30.953 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.946+0000 7f0e69f23640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e640c5780 0x7f0e640c5c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51172/0 (socket says 172.21.15.49:51172) 2023-11-06T02:13:30.953 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.946+0000 7f0e69f23640 1 -- 172.21.15.49:0/1755470961 learned_addr learned my addr 172.21.15.49:0/1755470961 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:30.953 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.946+0000 7f0e69f23640 1 -- 172.21.15.49:0/1755470961 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0e6415bb50 msgr2=0x7f0e641f3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:30.953 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.946+0000 7f0e69f23640 1 --2- 172.21.15.49:0/1755470961 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0e6415bb50 0x7f0e641f3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:30.953 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.946+0000 7f0e69f23640 1 -- 172.21.15.49:0/1755470961 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e6415bf30 msgr2=0x7f0e640b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:30.954 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.946+0000 7f0e69f23640 1 --2- 172.21.15.49:0/1755470961 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e6415bf30 0x7f0e640b3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:30.954 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.946+0000 7f0e69f23640 1 -- 172.21.15.49:0/1755470961 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e640b3230 con 0x7f0e640c5780 2023-11-06T02:13:30.954 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.946+0000 7f0e69f23640 1 --2- 172.21.15.49:0/1755470961 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e640c5780 0x7f0e640c5c00 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c00a8c0 tx=0x7f0e50005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:30.954 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.946+0000 7f0e59ffb640 1 -- 172.21.15.49:0/1755470961 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e5c001fb0 con 0x7f0e640c5780 2023-11-06T02:13:30.954 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.946+0000 7f0e59ffb640 1 -- 172.21.15.49:0/1755470961 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e5c0aadb0 con 0x7f0e640c5780 2023-11-06T02:13:30.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.946+0000 7f0e6b986640 1 -- 172.21.15.49:0/1755470961 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e640b3510 con 0x7f0e640c5780 2023-11-06T02:13:30.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.946+0000 7f0e59ffb640 1 -- 172.21.15.49:0/1755470961 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e5c0aa690 con 0x7f0e640c5780 2023-11-06T02:13:30.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.946+0000 7f0e6b986640 1 -- 172.21.15.49:0/1755470961 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e640c6140 con 0x7f0e640c5780 2023-11-06T02:13:30.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.950+0000 7f0e6b986640 1 -- 172.21.15.49:0/1755470961 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e64074040 con 0x7f0e640c5780 2023-11-06T02:13:30.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.950+0000 7f0e59ffb640 1 -- 172.21.15.49:0/1755470961 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f0e5c0017c0 con 0x7f0e640c5780 2023-11-06T02:13:30.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.950+0000 7f0e59ffb640 1 --2- 172.21.15.49:0/1755470961 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0e48065a60 0x7f0e48067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:30.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.950+0000 7f0e59ffb640 1 -- 172.21.15.49:0/1755470961 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f0e5c120480 con 0x7f0e640c5780 2023-11-06T02:13:30.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.950+0000 7f0e69722640 1 --2- 172.21.15.49:0/1755470961 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0e48065a60 0x7f0e48067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:30.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.950+0000 7f0e69722640 1 --2- 172.21.15.49:0/1755470961 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0e48065a60 0x7f0e48067f20 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f0e600099e0 tx=0x7f0e60008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:30.957 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:30.954+0000 7f0e59ffb640 1 -- 172.21.15.49:0/1755470961 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f0e5c0ee030 con 0x7f0e640c5780 2023-11-06T02:13:31.099 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:31.102+0000 7f0e6b986640 1 -- 172.21.15.49:0/1755470961 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f0e6415dba0 con 0x7f0e640c5780 2023-11-06T02:13:31.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:31.102+0000 7f0e59ffb640 1 -- 172.21.15.49:0/1755470961 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v76) v1 ==== 84+0+126 (secure 0 0 0) 0x7f0e5c0f1650 con 0x7f0e640c5780 2023-11-06T02:13:31.100 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:13:31.101 INFO:teuthology.orchestra.run.smithi049.stdout:[ 2023-11-06T02:13:31.101 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:13:31.101 INFO:teuthology.orchestra.run.smithi049.stdout: "poolnum": 1, 2023-11-06T02:13:31.101 INFO:teuthology.orchestra.run.smithi049.stdout: "poolname": ".mgr" 2023-11-06T02:13:31.101 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:13:31.102 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-11-06T02:13:31.102 INFO:teuthology.orchestra.run.smithi049.stdout: "poolnum": 2, 2023-11-06T02:13:31.102 INFO:teuthology.orchestra.run.smithi049.stdout: "poolname": "rbd" 2023-11-06T02:13:31.102 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:13:31.102 INFO:teuthology.orchestra.run.smithi049.stdout:] 2023-11-06T02:13:31.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:31.106+0000 7f0e6b986640 1 -- 172.21.15.49:0/1755470961 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0e48065a60 msgr2=0x7f0e48067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:31.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:31.106+0000 7f0e6b986640 1 --2- 172.21.15.49:0/1755470961 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0e48065a60 0x7f0e48067f20 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f0e600099e0 tx=0x7f0e60008040 comp rx=0 tx=0).stop 2023-11-06T02:13:31.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:31.106+0000 7f0e6b986640 1 -- 172.21.15.49:0/1755470961 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e640c5780 msgr2=0x7f0e640c5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:31.105 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:31.106+0000 7f0e6b986640 1 --2- 172.21.15.49:0/1755470961 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e640c5780 0x7f0e640c5c00 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c00a8c0 tx=0x7f0e50005800 comp rx=0 tx=0).stop 2023-11-06T02:13:31.105 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:31.106+0000 7f0e6b986640 1 -- 172.21.15.49:0/1755470961 shutdown_connections 2023-11-06T02:13:31.105 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:31.106+0000 7f0e6b986640 1 --2- 172.21.15.49:0/1755470961 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0e48065a60 0x7f0e48067f20 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:31.105 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:31.106+0000 7f0e6b986640 1 --2- 172.21.15.49:0/1755470961 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e640c5780 0x7f0e640c5c00 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:31.105 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:31.106+0000 7f0e6b986640 1 --2- 172.21.15.49:0/1755470961 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e6415bf30 0x7f0e640b3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:31.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:31.106+0000 7f0e6b986640 1 --2- 172.21.15.49:0/1755470961 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0e6415bb50 0x7f0e641f3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:31.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:31.106+0000 7f0e6b986640 1 -- 172.21.15.49:0/1755470961 >> 172.21.15.49:0/1755470961 conn(0x7f0e6406c0a0 msgr2=0x7f0e6415b150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:31.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:31.106+0000 7f0e6b986640 1 -- 172.21.15.49:0/1755470961 shutdown_connections 2023-11-06T02:13:31.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:31.106+0000 7f0e6b986640 1 -- 172.21.15.49:0/1755470961 wait complete. 2023-11-06T02:13:31.117 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-06T02:13:31.118 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-06T02:13:31.247 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:31.250+0000 7f2b4b416640 1 Processor -- start 2023-11-06T02:13:31.247 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:31.250+0000 7f2b4b416640 1 -- start start 2023-11-06T02:13:31.249 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:31.254+0000 7f2b4b416640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b440b0090 0x7f2b440b0510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:31.249 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:31.254+0000 7f2b4b416640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f2b44153210 0x7f2b441536a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:31.249 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:31.254+0000 7f2b4b416640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b44072150 con 0x7f2b4415c9a0 2023-11-06T02:13:31.249 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:31.254+0000 7f2b4b416640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b440b0a50 con 0x7f2b44153210 2023-11-06T02:13:31.250 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:31.254+0000 7f2b4b416640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b440c8fa0 con 0x7f2b440b0090 2023-11-06T02:13:31.250 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:31.254+0000 7f2b499b3640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f2b44153210 0x7f2b441536a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:31.250 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:31.254+0000 7f2b499b3640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f2b44153210 0x7f2b441536a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51176/0 (socket says 172.21.15.49:51176) 2023-11-06T02:13:31.250 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:31.254+0000 7f2b491b2640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f2b4415c9a0 0x7f2b44070d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:60538/0 (socket says 172.21.15.49:60538) 2023-11-06T02:13:31.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:31.254+0000 7f2b499b3640 1 -- 172.21.15.49:0/3471170673 learned_addr learned my addr 172.21.15.49:0/3471170673 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:31.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:31.254+0000 7f2b489b1640 1 --2- 172.21.15.49:0/3471170673 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b440b0090 0x7f2b440b0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:31.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:31.254+0000 7f2b499b3640 1 -- 172.21.15.49:0/3471170673 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b440b0090 msgr2=0x7f2b440b0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:31.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:31.254+0000 7f2b499b3640 1 --2- 172.21.15.49:0/3471170673 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b440b0090 0x7f2b440b0510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:31.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:31.254+0000 7f2b499b3640 1 -- 172.21.15.49:0/3471170673 >> v1:172.21.15.49:6789/0 conn(0x7f2b4415c9a0 legacy=0x7f2b44070d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:31.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:31.254+0000 7f2b499b3640 1 -- 172.21.15.49:0/3471170673 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b440ca440 con 0x7f2b44153210 2023-11-06T02:13:31.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:31.254+0000 7f2b3bfff640 1 -- 172.21.15.49:0/3471170673 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1750532558 0 0) 0x7f2b44072150 con 0x7f2b4415c9a0 2023-11-06T02:13:31.255 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:31.254+0000 7f2b499b3640 1 --2- 172.21.15.49:0/3471170673 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b44153210 0x7f2b441536a0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c009e50 tx=0x7f2b3c01c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e22d48b80d50e753 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:31.255 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:31.254+0000 7f2b3bfff640 1 -- 172.21.15.49:0/3471170673 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b3c0a57b0 con 0x7f2b44153210 2023-11-06T02:13:31.255 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:31.254+0000 7f2b3bfff640 1 -- 172.21.15.49:0/3471170673 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b3c0a5910 con 0x7f2b44153210 2023-11-06T02:13:31.255 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:31.254+0000 7f2b3bfff640 1 -- 172.21.15.49:0/3471170673 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b3c0a5ca0 con 0x7f2b44153210 2023-11-06T02:13:31.256 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:31.254+0000 7f2b4b416640 1 -- 172.21.15.49:0/3471170673 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b44153210 msgr2=0x7f2b441536a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:31.256 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:31.254+0000 7f2b4b416640 1 --2- 172.21.15.49:0/3471170673 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b44153210 0x7f2b441536a0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c009e50 tx=0x7f2b3c01c8f0 comp rx=0 tx=0).stop 2023-11-06T02:13:31.256 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:31.254+0000 7f2b4b416640 1 -- 172.21.15.49:0/3471170673 shutdown_connections 2023-11-06T02:13:31.256 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:31.254+0000 7f2b4b416640 1 --2- 172.21.15.49:0/3471170673 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b44153210 0x7f2b441536a0 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:31.256 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:31.254+0000 7f2b4b416640 1 --2- 172.21.15.49:0/3471170673 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b440b0090 0x7f2b440b0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:31.257 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:31.254+0000 7f2b4b416640 1 -- 172.21.15.49:0/3471170673 >> 172.21.15.49:0/3471170673 conn(0x7f2b440ce610 msgr2=0x7f2b440cea20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:31.257 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:31.254+0000 7f2b4b416640 1 -- 172.21.15.49:0/3471170673 shutdown_connections 2023-11-06T02:13:31.257 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:31.254+0000 7f2b4b416640 1 -- 172.21.15.49:0/3471170673 wait complete. 2023-11-06T02:13:31.257 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:31.258+0000 7f2b4b416640 1 Processor -- start 2023-11-06T02:13:31.257 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:31.258+0000 7f2b4b416640 1 -- start start 2023-11-06T02:13:31.258 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:31.258+0000 7f2b4b416640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b440b0090 0x7f2b4414c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:31.258 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:31.258+0000 7f2b4b416640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b44153210 0x7f2b440b1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:31.258 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:31.258+0000 7f2b4b416640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b440b1770 0x7f2b440b29a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:31.258 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:31.258+0000 7f2b4b416640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b440c6b70 con 0x7f2b440b0090 2023-11-06T02:13:31.258 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:31.258+0000 7f2b4b416640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b440b1c20 con 0x7f2b44153210 2023-11-06T02:13:31.259 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:31.258+0000 7f2b4b416640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b4414c570 con 0x7f2b440b1770 2023-11-06T02:13:31.259 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:31.258+0000 7f2b491b2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b440b0090 0x7f2b4414c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:31.259 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:31.258+0000 7f2b491b2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b440b0090 0x7f2b4414c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51184/0 (socket says 172.21.15.49:51184) 2023-11-06T02:13:31.259 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:31.258+0000 7f2b491b2640 1 -- 172.21.15.49:0/2694887289 learned_addr learned my addr 172.21.15.49:0/2694887289 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:31.259 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:31.258+0000 7f2b489b1640 1 --2- 172.21.15.49:0/2694887289 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b44153210 0x7f2b440b1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:31.259 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:31.258+0000 7f2b491b2640 1 -- 172.21.15.49:0/2694887289 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b440b1770 msgr2=0x7f2b440b29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:31.260 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:31.258+0000 7f2b499b3640 1 --2- 172.21.15.49:0/2694887289 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b440b1770 0x7f2b440b29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:31.260 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:31.258+0000 7f2b491b2640 1 --2- 172.21.15.49:0/2694887289 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b440b1770 0x7f2b440b29a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:31.260 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:31.258+0000 7f2b491b2640 1 -- 172.21.15.49:0/2694887289 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b44153210 msgr2=0x7f2b440b1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:31.260 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:31.258+0000 7f2b491b2640 1 --2- 172.21.15.49:0/2694887289 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b44153210 0x7f2b440b1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:31.261 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:31.258+0000 7f2b491b2640 1 -- 172.21.15.49:0/2694887289 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b3c009b30 con 0x7f2b440b0090 2023-11-06T02:13:31.261 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:31.258+0000 7f2b491b2640 1 --2- 172.21.15.49:0/2694887289 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b440b0090 0x7f2b4414c030 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f2b4000c4e0 tx=0x7f2b4000c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:31.261 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:31.258+0000 7f2b39ffb640 1 -- 172.21.15.49:0/2694887289 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b400074e0 con 0x7f2b440b0090 2023-11-06T02:13:31.261 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:31.258+0000 7f2b39ffb640 1 -- 172.21.15.49:0/2694887289 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b40007660 con 0x7f2b440b0090 2023-11-06T02:13:31.261 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:31.258+0000 7f2b4b416640 1 -- 172.21.15.49:0/2694887289 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b440b3690 con 0x7f2b440b0090 2023-11-06T02:13:31.261 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:31.258+0000 7f2b39ffb640 1 -- 172.21.15.49:0/2694887289 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b40004a30 con 0x7f2b440b0090 2023-11-06T02:13:31.262 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:31.258+0000 7f2b4b416640 1 -- 172.21.15.49:0/2694887289 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b440c6790 con 0x7f2b440b0090 2023-11-06T02:13:31.263 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:31.258+0000 7f2b4b416640 1 -- 172.21.15.49:0/2694887289 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b440c6cb0 con 0x7f2b440b0090 2023-11-06T02:13:31.263 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:31.258+0000 7f2b39ffb640 1 -- 172.21.15.49:0/2694887289 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f2b40004b90 con 0x7f2b440b0090 2023-11-06T02:13:31.263 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:31.262+0000 7f2b39ffb640 1 --2- 172.21.15.49:0/2694887289 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2b28065a60 0x7f2b28067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:31.264 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:31.262+0000 7f2b39ffb640 1 -- 172.21.15.49:0/2694887289 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f2b40082110 con 0x7f2b440b0090 2023-11-06T02:13:31.264 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:31.262+0000 7f2b489b1640 1 --2- 172.21.15.49:0/2694887289 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2b28065a60 0x7f2b28067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:31.264 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:31.262+0000 7f2b489b1640 1 --2- 172.21.15.49:0/2694887289 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2b28065a60 0x7f2b28067f20 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f2b440cbdd0 tx=0x7f2b34008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:31.264 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:31.266+0000 7f2b39ffb640 1 -- 172.21.15.49:0/2694887289 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f2b440c6cb0 con 0x7f2b440b0090 2023-11-06T02:13:31.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:31.418+0000 7f2b4b416640 1 -- 172.21.15.49:0/2694887289 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f2b440c6cb0 con 0x7f2b440b0090 2023-11-06T02:13:32.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:32.226+0000 7f2b39ffb640 1 -- 172.21.15.49:0/2694887289 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v77) v1 ==== 138+0+0 (secure 0 0 0) 0x7f2b440c6cb0 con 0x7f2b440b0090 2023-11-06T02:13:32.224 INFO:teuthology.orchestra.run.smithi049.stderr:pool 'cephfs_metadata' created 2023-11-06T02:13:32.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:32.234+0000 7f2b4b416640 1 -- 172.21.15.49:0/2694887289 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2b28065a60 msgr2=0x7f2b28067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:32.229 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:32.234+0000 7f2b4b416640 1 --2- 172.21.15.49:0/2694887289 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2b28065a60 0x7f2b28067f20 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f2b440cbdd0 tx=0x7f2b34008040 comp rx=0 tx=0).stop 2023-11-06T02:13:32.229 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:32.234+0000 7f2b4b416640 1 -- 172.21.15.49:0/2694887289 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b440b0090 msgr2=0x7f2b4414c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:32.229 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:32.234+0000 7f2b4b416640 1 --2- 172.21.15.49:0/2694887289 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b440b0090 0x7f2b4414c030 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f2b4000c4e0 tx=0x7f2b4000c9a0 comp rx=0 tx=0).stop 2023-11-06T02:13:32.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:32.234+0000 7f2b4b416640 1 -- 172.21.15.49:0/2694887289 shutdown_connections 2023-11-06T02:13:32.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:32.234+0000 7f2b4b416640 1 --2- 172.21.15.49:0/2694887289 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2b28065a60 0x7f2b28067f20 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:32.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:32.234+0000 7f2b4b416640 1 --2- 172.21.15.49:0/2694887289 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b440b1770 0x7f2b440b29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:32.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:32.234+0000 7f2b4b416640 1 --2- 172.21.15.49:0/2694887289 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b44153210 0x7f2b440b1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:32.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:32.234+0000 7f2b4b416640 1 --2- 172.21.15.49:0/2694887289 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b440b0090 0x7f2b4414c030 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:32.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:32.234+0000 7f2b4b416640 1 -- 172.21.15.49:0/2694887289 >> 172.21.15.49:0/2694887289 conn(0x7f2b440ce610 msgr2=0x7f2b440cb650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:32.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:32.234+0000 7f2b4b416640 1 -- 172.21.15.49:0/2694887289 shutdown_connections 2023-11-06T02:13:32.232 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:32.234+0000 7f2b4b416640 1 -- 172.21.15.49:0/2694887289 wait complete. 2023-11-06T02:13:32.252 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-06T02:13:32.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:32.386+0000 7f8d21b1f640 1 Processor -- start 2023-11-06T02:13:32.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:32.386+0000 7f8d21b1f640 1 -- start start 2023-11-06T02:13:32.383 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:32.386+0000 7f8d21b1f640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f8d1c0b04e0 0x7f8d1c0b08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:32.383 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:32.386+0000 7f8d21b1f640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d1c0cc5c0 con 0x7f8d1c0cb2d0 2023-11-06T02:13:32.383 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:32.386+0000 7f8d21b1f640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d1c0cb760 con 0x7f8d1c15a9e0 2023-11-06T02:13:32.383 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:32.386+0000 7f8d21b1f640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d1c0cd160 con 0x7f8d1c0b04e0 2023-11-06T02:13:32.384 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:32.386+0000 7f8d1affd640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f8d1c0cb2d0 0x7f8d1c0cca50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:60552/0 (socket says 172.21.15.49:60552) 2023-11-06T02:13:32.384 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:32.386+0000 7f8d1affd640 1 -- 172.21.15.49:0/3921504126 learned_addr learned my addr 172.21.15.49:0/3921504126 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:13:32.384 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:32.386+0000 7f8d1bfff640 1 --2- 172.21.15.49:0/3921504126 >> v2:172.21.15.49:3300/0 conn(0x7f8d1c0b04e0 0x7f8d1c0b08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:32.384 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:32.386+0000 7f8d1a7fc640 1 -- 172.21.15.49:0/3921504126 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2161554401 0 0) 0x7f8d1c0cc5c0 con 0x7f8d1c0cb2d0 2023-11-06T02:13:32.384 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:32.386+0000 7f8d1a7fc640 1 -- 172.21.15.49:0/3921504126 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d04003530 con 0x7f8d1c0cb2d0 2023-11-06T02:13:32.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:32.386+0000 7f8d1bfff640 1 -- 172.21.15.49:0/3921504126 >> v1:172.21.15.150:6789/0 conn(0x7f8d1c15a9e0 legacy=0x7f8d1c0afbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:32.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:32.386+0000 7f8d1bfff640 1 -- 172.21.15.49:0/3921504126 >> v1:172.21.15.49:6789/0 conn(0x7f8d1c0cb2d0 legacy=0x7f8d1c0cca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:32.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:32.386+0000 7f8d1bfff640 1 -- 172.21.15.49:0/3921504126 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d1c0b0e30 con 0x7f8d1c0b04e0 2023-11-06T02:13:32.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:32.390+0000 7f8d1bfff640 1 --2- 172.21.15.49:0/3921504126 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d1c0b04e0 0x7f8d1c0b08f0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c0097c0 tx=0x7f8d0c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6c6395624a753b6 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:32.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:32.390+0000 7f8d1a7fc640 1 -- 172.21.15.49:0/3921504126 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d0c0ab650 con 0x7f8d1c0b04e0 2023-11-06T02:13:32.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:32.390+0000 7f8d1a7fc640 1 -- 172.21.15.49:0/3921504126 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d0c0ab7b0 con 0x7f8d1c0b04e0 2023-11-06T02:13:32.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:32.390+0000 7f8d1a7fc640 1 -- 172.21.15.49:0/3921504126 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d0c0abb20 con 0x7f8d1c0b04e0 2023-11-06T02:13:32.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:32.390+0000 7f8d21b1f640 1 -- 172.21.15.49:0/3921504126 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d1c0b04e0 msgr2=0x7f8d1c0b08f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:32.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:32.390+0000 7f8d21b1f640 1 --2- 172.21.15.49:0/3921504126 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d1c0b04e0 0x7f8d1c0b08f0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c0097c0 tx=0x7f8d0c01c450 comp rx=0 tx=0).stop 2023-11-06T02:13:32.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:32.390+0000 7f8d21b1f640 1 -- 172.21.15.49:0/3921504126 shutdown_connections 2023-11-06T02:13:32.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:32.390+0000 7f8d21b1f640 1 --2- 172.21.15.49:0/3921504126 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d1c0b04e0 0x7f8d1c0b08f0 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:32.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:32.390+0000 7f8d21b1f640 1 -- 172.21.15.49:0/3921504126 >> 172.21.15.49:0/3921504126 conn(0x7f8d1c098a40 msgr2=0x7f8d1c0cf860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:32.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:32.390+0000 7f8d21b1f640 1 -- 172.21.15.49:0/3921504126 shutdown_connections 2023-11-06T02:13:32.389 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:32.390+0000 7f8d21b1f640 1 -- 172.21.15.49:0/3921504126 wait complete. 2023-11-06T02:13:32.391 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:32.390+0000 7f8d21b1f640 1 Processor -- start 2023-11-06T02:13:32.391 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:32.390+0000 7f8d21b1f640 1 -- start start 2023-11-06T02:13:32.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:32.390+0000 7f8d21b1f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d1c0b04e0 0x7f8d1c152890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:32.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:32.390+0000 7f8d21b1f640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8d1c0cb2d0 0x7f8d1c0b2450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:32.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:32.390+0000 7f8d21b1f640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d1c15a9e0 0x7f8d1c148cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:32.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:32.390+0000 7f8d21b1f640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1c0b2f10 con 0x7f8d1c0b04e0 2023-11-06T02:13:32.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:32.390+0000 7f8d21b1f640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1c1498e0 con 0x7f8d1c15a9e0 2023-11-06T02:13:32.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:32.390+0000 7f8d21b1f640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1c149bc0 con 0x7f8d1c0cb2d0 2023-11-06T02:13:32.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:32.390+0000 7f8d1b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d1c0b04e0 0x7f8d1c152890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:32.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:32.390+0000 7f8d1b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d1c0b04e0 0x7f8d1c152890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51206/0 (socket says 172.21.15.49:51206) 2023-11-06T02:13:32.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:32.390+0000 7f8d1b7fe640 1 -- 172.21.15.49:0/3276409412 learned_addr learned my addr 172.21.15.49:0/3276409412 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:32.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:32.394+0000 7f8d1bfff640 1 --2- 172.21.15.49:0/3276409412 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d1c15a9e0 0x7f8d1c148cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:32.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:32.394+0000 7f8d1affd640 1 --2- 172.21.15.49:0/3276409412 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8d1c0cb2d0 0x7f8d1c0b2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:32.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:32.394+0000 7f8d1b7fe640 1 -- 172.21.15.49:0/3276409412 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8d1c0cb2d0 msgr2=0x7f8d1c0b2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:32.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:32.394+0000 7f8d1b7fe640 1 --2- 172.21.15.49:0/3276409412 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8d1c0cb2d0 0x7f8d1c0b2450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:32.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:32.394+0000 7f8d1b7fe640 1 -- 172.21.15.49:0/3276409412 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d1c15a9e0 msgr2=0x7f8d1c148cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:32.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:32.394+0000 7f8d1b7fe640 1 --2- 172.21.15.49:0/3276409412 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d1c15a9e0 0x7f8d1c148cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:32.395 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:32.394+0000 7f8d1b7fe640 1 -- 172.21.15.49:0/3276409412 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d0c009420 con 0x7f8d1c0b04e0 2023-11-06T02:13:32.395 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:32.394+0000 7f8d1bfff640 1 --2- 172.21.15.49:0/3276409412 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d1c15a9e0 0x7f8d1c148cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:13:32.395 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:32.394+0000 7f8d1b7fe640 1 --2- 172.21.15.49:0/3276409412 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d1c0b04e0 0x7f8d1c152890 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f8d1400c370 tx=0x7f8d1400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:32.395 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:32.394+0000 7f8d03fff640 1 -- 172.21.15.49:0/3276409412 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d140052d0 con 0x7f8d1c0b04e0 2023-11-06T02:13:32.395 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:32.394+0000 7f8d03fff640 1 -- 172.21.15.49:0/3276409412 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d14005430 con 0x7f8d1c0b04e0 2023-11-06T02:13:32.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:32.394+0000 7f8d21b1f640 1 -- 172.21.15.49:0/3276409412 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d1c149ea0 con 0x7f8d1c0b04e0 2023-11-06T02:13:32.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:32.394+0000 7f8d03fff640 1 -- 172.21.15.49:0/3276409412 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d14007260 con 0x7f8d1c0b04e0 2023-11-06T02:13:32.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:32.394+0000 7f8d21b1f640 1 -- 172.21.15.49:0/3276409412 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d1c14c040 con 0x7f8d1c0b04e0 2023-11-06T02:13:32.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:32.394+0000 7f8d21b1f640 1 -- 172.21.15.49:0/3276409412 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d1c0b21a0 con 0x7f8d1c0b04e0 2023-11-06T02:13:32.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:32.394+0000 7f8d03fff640 1 -- 172.21.15.49:0/3276409412 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f8d140073c0 con 0x7f8d1c0b04e0 2023-11-06T02:13:32.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:32.394+0000 7f8d03fff640 1 --2- 172.21.15.49:0/3276409412 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8d08065a60 0x7f8d08067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:32.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:32.394+0000 7f8d03fff640 1 -- 172.21.15.49:0/3276409412 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5580+0+0 (secure 0 0 0) 0x7f8d140828f0 con 0x7f8d1c0b04e0 2023-11-06T02:13:32.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:32.394+0000 7f8d1affd640 1 --2- 172.21.15.49:0/3276409412 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8d08065a60 0x7f8d08067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:32.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:32.398+0000 7f8d1affd640 1 --2- 172.21.15.49:0/3276409412 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8d08065a60 0x7f8d08067f20 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c0b0ba0 tx=0x7f8d10009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:32.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:32.398+0000 7f8d03fff640 1 -- 172.21.15.49:0/3276409412 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f8d14050410 con 0x7f8d1c0b04e0 2023-11-06T02:13:32.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:32.558+0000 7f8d21b1f640 1 -- 172.21.15.49:0/3276409412 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f8d1c156500 con 0x7f8d1c0b04e0 2023-11-06T02:13:33.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.238+0000 7f8d03fff640 1 -- 172.21.15.49:0/3276409412 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v78) v1 ==== 170+0+0 (secure 0 0 0) 0x7f8d140200d0 con 0x7f8d1c0b04e0 2023-11-06T02:13:33.233 INFO:teuthology.orchestra.run.smithi049.stderr:pool 'cephfs_data' created 2023-11-06T02:13:33.242 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.246+0000 7f8d21b1f640 1 -- 172.21.15.49:0/3276409412 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8d08065a60 msgr2=0x7f8d08067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:33.243 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.246+0000 7f8d21b1f640 1 --2- 172.21.15.49:0/3276409412 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8d08065a60 0x7f8d08067f20 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c0b0ba0 tx=0x7f8d10009040 comp rx=0 tx=0).stop 2023-11-06T02:13:33.243 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.246+0000 7f8d21b1f640 1 -- 172.21.15.49:0/3276409412 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d1c0b04e0 msgr2=0x7f8d1c152890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:33.243 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.246+0000 7f8d21b1f640 1 --2- 172.21.15.49:0/3276409412 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d1c0b04e0 0x7f8d1c152890 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f8d1400c370 tx=0x7f8d1400c830 comp rx=0 tx=0).stop 2023-11-06T02:13:33.250 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.254+0000 7f8d21b1f640 1 -- 172.21.15.49:0/3276409412 shutdown_connections 2023-11-06T02:13:33.250 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.254+0000 7f8d21b1f640 1 --2- 172.21.15.49:0/3276409412 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8d08065a60 0x7f8d08067f20 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:33.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.254+0000 7f8d21b1f640 1 --2- 172.21.15.49:0/3276409412 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d1c15a9e0 0x7f8d1c148cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:33.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.254+0000 7f8d21b1f640 1 --2- 172.21.15.49:0/3276409412 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8d1c0cb2d0 0x7f8d1c0b2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:33.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.254+0000 7f8d21b1f640 1 --2- 172.21.15.49:0/3276409412 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d1c0b04e0 0x7f8d1c152890 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:33.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.254+0000 7f8d21b1f640 1 -- 172.21.15.49:0/3276409412 >> 172.21.15.49:0/3276409412 conn(0x7f8d1c098a40 msgr2=0x7f8d1c0aea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:33.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.254+0000 7f8d21b1f640 1 -- 172.21.15.49:0/3276409412 shutdown_connections 2023-11-06T02:13:33.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.254+0000 7f8d21b1f640 1 -- 172.21.15.49:0/3276409412 wait complete. 2023-11-06T02:13:33.285 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-06T02:13:33.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.414+0000 7fac32998640 1 Processor -- start 2023-11-06T02:13:33.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.418+0000 7fac32998640 1 -- start start 2023-11-06T02:13:33.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.418+0000 7fac32998640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fac2c06b2f0 0x7fac2c0b1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:33.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.418+0000 7fac32998640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fac2c15cf10 0x7fac2c06b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:33.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.418+0000 7fac32998640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac2c0b1990 con 0x7fac2c0b0810 2023-11-06T02:13:33.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.418+0000 7fac32998640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac2c0b0d80 con 0x7fac2c15cf10 2023-11-06T02:13:33.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.418+0000 7fac32998640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac2c0b1060 con 0x7fac2c06b2f0 2023-11-06T02:13:33.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.418+0000 7fac30f35640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fac2c15cf10 0x7fac2c06b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:33.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.418+0000 7fac237fe640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fac2c0b0810 0x7fac2c15c800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:60564/0 (socket says 172.21.15.49:60564) 2023-11-06T02:13:33.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.418+0000 7fac237fe640 1 -- 172.21.15.49:0/2921979326 learned_addr learned my addr 172.21.15.49:0/2921979326 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:13:33.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.418+0000 7fac2bfff640 1 --2- 172.21.15.49:0/2921979326 >> v2:172.21.15.150:3300/0 conn(0x7fac2c06b2f0 0x7fac2c0b1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:33.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.418+0000 7fac30f35640 1 -- 172.21.15.49:0/2921979326 >> v2:172.21.15.150:3300/0 conn(0x7fac2c06b2f0 msgr2=0x7fac2c0b1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:33.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.418+0000 7fac30f35640 1 --2- 172.21.15.49:0/2921979326 >> v2:172.21.15.150:3300/0 conn(0x7fac2c06b2f0 0x7fac2c0b1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:33.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.418+0000 7fac30f35640 1 -- 172.21.15.49:0/2921979326 >> v1:172.21.15.49:6789/0 conn(0x7fac2c0b0810 legacy=0x7fac2c15c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:33.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.418+0000 7fac30f35640 1 -- 172.21.15.49:0/2921979326 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac2c0b1340 con 0x7fac2c15cf10 2023-11-06T02:13:33.418 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.418+0000 7fac30f35640 1 --2- 172.21.15.49:0/2921979326 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fac2c15cf10 0x7fac2c06b6d0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fac2c071a30 tx=0x7fac2401f0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b661e9cb52f37f98 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:33.418 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.422+0000 7fac2b7fe640 1 -- 172.21.15.49:0/2921979326 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac240b0440 con 0x7fac2c15cf10 2023-11-06T02:13:33.418 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.422+0000 7fac2b7fe640 1 -- 172.21.15.49:0/2921979326 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac240b05a0 con 0x7fac2c15cf10 2023-11-06T02:13:33.418 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.422+0000 7fac2b7fe640 1 -- 172.21.15.49:0/2921979326 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac240b0930 con 0x7fac2c15cf10 2023-11-06T02:13:33.419 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.422+0000 7fac32998640 1 -- 172.21.15.49:0/2921979326 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fac2c15cf10 msgr2=0x7fac2c06b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:33.419 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.422+0000 7fac32998640 1 --2- 172.21.15.49:0/2921979326 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fac2c15cf10 0x7fac2c06b6d0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fac2c071a30 tx=0x7fac2401f0f0 comp rx=0 tx=0).stop 2023-11-06T02:13:33.419 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.422+0000 7fac32998640 1 -- 172.21.15.49:0/2921979326 shutdown_connections 2023-11-06T02:13:33.419 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.422+0000 7fac32998640 1 --2- 172.21.15.49:0/2921979326 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fac2c15cf10 0x7fac2c06b6d0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:33.419 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.422+0000 7fac32998640 1 --2- 172.21.15.49:0/2921979326 >> v2:172.21.15.150:3300/0 conn(0x7fac2c06b2f0 0x7fac2c0b1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:33.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.422+0000 7fac32998640 1 -- 172.21.15.49:0/2921979326 >> 172.21.15.49:0/2921979326 conn(0x7fac2c0d08d0 msgr2=0x7fac2c0d0cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:33.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.422+0000 7fac32998640 1 -- 172.21.15.49:0/2921979326 shutdown_connections 2023-11-06T02:13:33.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.422+0000 7fac32998640 1 -- 172.21.15.49:0/2921979326 wait complete. 2023-11-06T02:13:33.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.422+0000 7fac32998640 1 Processor -- start 2023-11-06T02:13:33.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.422+0000 7fac32998640 1 -- start start 2023-11-06T02:13:33.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.422+0000 7fac32998640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac2c06b2f0 0x7fac2c1f31c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:33.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.422+0000 7fac32998640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fac2c0b0810 0x7fac2c158f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:33.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.422+0000 7fac32998640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fac2c15cf10 0x7fac2c0c6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:33.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.422+0000 7fac32998640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac2c1f2700 con 0x7fac2c0b0810 2023-11-06T02:13:33.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.422+0000 7fac32998640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac2c0c6bf0 con 0x7fac2c06b2f0 2023-11-06T02:13:33.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.422+0000 7fac32998640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac2c0c6ed0 con 0x7fac2c15cf10 2023-11-06T02:13:33.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.422+0000 7fac237fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fac2c0b0810 0x7fac2c158f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:33.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.422+0000 7fac237fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fac2c0b0810 0x7fac2c158f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51216/0 (socket says 172.21.15.49:51216) 2023-11-06T02:13:33.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.422+0000 7fac237fe640 1 -- 172.21.15.49:0/338063873 learned_addr learned my addr 172.21.15.49:0/338063873 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:33.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.422+0000 7fac2bfff640 1 --2- 172.21.15.49:0/338063873 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac2c06b2f0 0x7fac2c1f31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:33.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.422+0000 7fac237fe640 1 -- 172.21.15.49:0/338063873 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fac2c15cf10 msgr2=0x7fac2c0c6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:33.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.422+0000 7fac30f35640 1 --2- 172.21.15.49:0/338063873 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fac2c15cf10 0x7fac2c0c6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:33.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.422+0000 7fac237fe640 1 --2- 172.21.15.49:0/338063873 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fac2c15cf10 0x7fac2c0c6210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:33.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.422+0000 7fac237fe640 1 -- 172.21.15.49:0/338063873 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac2c06b2f0 msgr2=0x7fac2c1f31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:33.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.422+0000 7fac237fe640 1 --2- 172.21.15.49:0/338063873 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac2c06b2f0 0x7fac2c1f31c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:33.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.422+0000 7fac237fe640 1 -- 172.21.15.49:0/338063873 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac24009a70 con 0x7fac2c0b0810 2023-11-06T02:13:33.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.422+0000 7fac2bfff640 1 --2- 172.21.15.49:0/338063873 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac2c06b2f0 0x7fac2c1f31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:13:33.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.422+0000 7fac237fe640 1 --2- 172.21.15.49:0/338063873 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fac2c0b0810 0x7fac2c158f70 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fac1400c420 tx=0x7fac1400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:33.427 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.422+0000 7fac297fa640 1 -- 172.21.15.49:0/338063873 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac14005cf0 con 0x7fac2c0b0810 2023-11-06T02:13:33.427 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.422+0000 7fac297fa640 1 -- 172.21.15.49:0/338063873 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac14005e50 con 0x7fac2c0b0810 2023-11-06T02:13:33.427 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.422+0000 7fac297fa640 1 -- 172.21.15.49:0/338063873 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac140072f0 con 0x7fac2c0b0810 2023-11-06T02:13:33.427 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.426+0000 7fac32998640 1 -- 172.21.15.49:0/338063873 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac2c1f29c0 con 0x7fac2c0b0810 2023-11-06T02:13:33.427 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.426+0000 7fac32998640 1 -- 172.21.15.49:0/338063873 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac2c1594b0 con 0x7fac2c0b0810 2023-11-06T02:13:33.427 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.426+0000 7fac32998640 1 -- 172.21.15.49:0/338063873 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac2c0d0740 con 0x7fac2c0b0810 2023-11-06T02:13:33.428 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.426+0000 7fac297fa640 1 -- 172.21.15.49:0/338063873 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fac14007450 con 0x7fac2c0b0810 2023-11-06T02:13:33.428 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.426+0000 7fac297fa640 1 --2- 172.21.15.49:0/338063873 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fac0c065a90 0x7fac0c067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:33.428 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.426+0000 7fac297fa640 1 -- 172.21.15.49:0/338063873 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5962+0+0 (secure 0 0 0) 0x7fac14082830 con 0x7fac2c0b0810 2023-11-06T02:13:33.428 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.426+0000 7fac2bfff640 1 --2- 172.21.15.49:0/338063873 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fac0c065a90 0x7fac0c067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:33.428 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.426+0000 7fac2bfff640 1 --2- 172.21.15.49:0/338063873 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fac0c065a90 0x7fac0c067f50 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fac1c006fd0 tx=0x7fac1c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:33.429 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.430+0000 7fac297fa640 1 -- 172.21.15.49:0/338063873 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fac140500f0 con 0x7fac2c0b0810 2023-11-06T02:13:33.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.622+0000 7fac32998640 1 -- 172.21.15.49:0/338063873 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fac2c074040 con 0x7fac2c0b0810 2023-11-06T02:13:33.631 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.634+0000 7fac297fa640 1 -- 172.21.15.49:0/338063873 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 11 and data pool 12 v42) v1 ==== 174+0+0 (secure 0 0 0) 0x7fac14017070 con 0x7fac2c0b0810 2023-11-06T02:13:33.636 INFO:teuthology.orchestra.run.smithi049.stderr:new fs with metadata pool 11 and data pool 12 2023-11-06T02:13:33.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.642+0000 7fac32998640 1 -- 172.21.15.49:0/338063873 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fac0c065a90 msgr2=0x7fac0c067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:33.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.642+0000 7fac32998640 1 --2- 172.21.15.49:0/338063873 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fac0c065a90 0x7fac0c067f50 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fac1c006fd0 tx=0x7fac1c008040 comp rx=0 tx=0).stop 2023-11-06T02:13:33.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.642+0000 7fac32998640 1 -- 172.21.15.49:0/338063873 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fac2c0b0810 msgr2=0x7fac2c158f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:33.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.642+0000 7fac32998640 1 --2- 172.21.15.49:0/338063873 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fac2c0b0810 0x7fac2c158f70 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fac1400c420 tx=0x7fac1400c8e0 comp rx=0 tx=0).stop 2023-11-06T02:13:33.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.642+0000 7fac32998640 1 -- 172.21.15.49:0/338063873 shutdown_connections 2023-11-06T02:13:33.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.642+0000 7fac32998640 1 --2- 172.21.15.49:0/338063873 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fac0c065a90 0x7fac0c067f50 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:33.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.642+0000 7fac32998640 1 --2- 172.21.15.49:0/338063873 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fac2c0b0810 0x7fac2c158f70 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:33.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.642+0000 7fac32998640 1 --2- 172.21.15.49:0/338063873 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fac2c15cf10 0x7fac2c0c6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:33.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.642+0000 7fac32998640 1 --2- 172.21.15.49:0/338063873 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac2c06b2f0 0x7fac2c1f31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:33.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.642+0000 7fac32998640 1 -- 172.21.15.49:0/338063873 >> 172.21.15.49:0/338063873 conn(0x7fac2c0d08d0 msgr2=0x7fac2c06c510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:33.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.646+0000 7fac32998640 1 -- 172.21.15.49:0/338063873 shutdown_connections 2023-11-06T02:13:33.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.646+0000 7fac32998640 1 -- 172.21.15.49:0/338063873 wait complete. 2023-11-06T02:13:33.673 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T02:13:33.791 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.794+0000 7f422d735640 1 Processor -- start 2023-11-06T02:13:33.791 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.794+0000 7f422d735640 1 -- start start 2023-11-06T02:13:33.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.794+0000 7f422d735640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f42280cb010 0x7f42280ca980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:33.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.794+0000 7f422d735640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4228159b40 0x7f42280cb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:33.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.794+0000 7f422d735640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f42280cdd50 0x7f42280c8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:33.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.794+0000 7f422d735640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42280cb930 con 0x7f42280cb010 2023-11-06T02:13:33.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.794+0000 7f422d735640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42280723a0 con 0x7f42280cdd50 2023-11-06T02:13:33.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.794+0000 7f422d735640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42280715e0 con 0x7f4228159b40 2023-11-06T02:13:33.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.798+0000 7f4226ffd640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f42280cb010 0x7f42280ca980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:33.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.798+0000 7f4226ffd640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f42280cb010 0x7f42280ca980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51226/0 (socket says 172.21.15.49:51226) 2023-11-06T02:13:33.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.798+0000 7f4226ffd640 1 -- 172.21.15.49:0/2951515519 learned_addr learned my addr 172.21.15.49:0/2951515519 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:33.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.798+0000 7f42267fc640 1 --2- 172.21.15.49:0/2951515519 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4228159b40 0x7f42280cb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:33.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.798+0000 7f42277fe640 1 --2- 172.21.15.49:0/2951515519 >> v2:172.21.15.150:3300/0 conn(0x7f42280cdd50 0x7f42280c8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:33.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.798+0000 7f4226ffd640 1 -- 172.21.15.49:0/2951515519 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4228159b40 msgr2=0x7f42280cb3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:33.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.798+0000 7f4226ffd640 1 --2- 172.21.15.49:0/2951515519 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4228159b40 0x7f42280cb3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:33.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.798+0000 7f4226ffd640 1 -- 172.21.15.49:0/2951515519 >> v2:172.21.15.150:3300/0 conn(0x7f42280cdd50 msgr2=0x7f42280c8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:33.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.798+0000 7f4226ffd640 1 --2- 172.21.15.49:0/2951515519 >> v2:172.21.15.150:3300/0 conn(0x7f42280cdd50 0x7f42280c8c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:33.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.798+0000 7f4226ffd640 1 -- 172.21.15.49:0/2951515519 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42280718c0 con 0x7f42280cb010 2023-11-06T02:13:33.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.798+0000 7f42267fc640 1 --2- 172.21.15.49:0/2951515519 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4228159b40 0x7f42280cb3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:13:33.799 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.798+0000 7f4226ffd640 1 --2- 172.21.15.49:0/2951515519 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42280cb010 0x7f42280ca980 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f422806add0 tx=0x7f421801c210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61f97a86fbd22cc4 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:33.799 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.798+0000 7f4225ffb640 1 -- 172.21.15.49:0/2951515519 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42180ab600 con 0x7f42280cb010 2023-11-06T02:13:33.799 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.798+0000 7f4225ffb640 1 -- 172.21.15.49:0/2951515519 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42180a7050 con 0x7f42280cb010 2023-11-06T02:13:33.800 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.798+0000 7f4225ffb640 1 -- 172.21.15.49:0/2951515519 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42180ab980 con 0x7f42280cb010 2023-11-06T02:13:33.800 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.798+0000 7f422d735640 1 -- 172.21.15.49:0/2951515519 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42280cb010 msgr2=0x7f42280ca980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:33.800 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.798+0000 7f422d735640 1 --2- 172.21.15.49:0/2951515519 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42280cb010 0x7f42280ca980 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f422806add0 tx=0x7f421801c210 comp rx=0 tx=0).stop 2023-11-06T02:13:33.800 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.798+0000 7f422d735640 1 -- 172.21.15.49:0/2951515519 shutdown_connections 2023-11-06T02:13:33.800 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.798+0000 7f422d735640 1 --2- 172.21.15.49:0/2951515519 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4228159b40 0x7f42280cb3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:33.801 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.798+0000 7f422d735640 1 --2- 172.21.15.49:0/2951515519 >> v2:172.21.15.150:3300/0 conn(0x7f42280cdd50 0x7f42280c8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:33.801 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.798+0000 7f422d735640 1 --2- 172.21.15.49:0/2951515519 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42280cb010 0x7f42280ca980 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:33.801 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.798+0000 7f422d735640 1 -- 172.21.15.49:0/2951515519 >> 172.21.15.49:0/2951515519 conn(0x7f422806c0a0 msgr2=0x7f422806c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:33.801 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.798+0000 7f422d735640 1 -- 172.21.15.49:0/2951515519 shutdown_connections 2023-11-06T02:13:33.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.798+0000 7f422d735640 1 -- 172.21.15.49:0/2951515519 wait complete. 2023-11-06T02:13:33.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.802+0000 7f422d735640 1 Processor -- start 2023-11-06T02:13:33.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.802+0000 7f422d735640 1 -- start start 2023-11-06T02:13:33.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.802+0000 7f422d735640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f42280cb010 0x7f4228145b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:33.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.802+0000 7f422d735640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f42280cdd50 0x7f4228146400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:33.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.802+0000 7f422d735640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4228159b40 0x7f4228146940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:33.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.802+0000 7f422d735640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4228146e80 con 0x7f4228159b40 2023-11-06T02:13:33.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.802+0000 7f422d735640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42280c2b00 con 0x7f42280cdd50 2023-11-06T02:13:33.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.802+0000 7f422d735640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42281470b0 con 0x7f42280cb010 2023-11-06T02:13:33.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.802+0000 7f42277fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4228159b40 0x7f4228146940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:33.804 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.802+0000 7f42277fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4228159b40 0x7f4228146940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51232/0 (socket says 172.21.15.49:51232) 2023-11-06T02:13:33.804 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.802+0000 7f42277fe640 1 -- 172.21.15.49:0/780730312 learned_addr learned my addr 172.21.15.49:0/780730312 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:33.804 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.802+0000 7f4226ffd640 1 --2- 172.21.15.49:0/780730312 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f42280cb010 0x7f4228145b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:33.804 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.802+0000 7f42267fc640 1 --2- 172.21.15.49:0/780730312 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f42280cdd50 0x7f4228146400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:33.804 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.802+0000 7f42277fe640 1 -- 172.21.15.49:0/780730312 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f42280cb010 msgr2=0x7f4228145b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:33.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.802+0000 7f42277fe640 1 --2- 172.21.15.49:0/780730312 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f42280cb010 0x7f4228145b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:33.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.802+0000 7f42277fe640 1 -- 172.21.15.49:0/780730312 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f42280cdd50 msgr2=0x7f4228146400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:33.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.802+0000 7f42277fe640 1 --2- 172.21.15.49:0/780730312 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f42280cdd50 0x7f4228146400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:33.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.802+0000 7f42277fe640 1 -- 172.21.15.49:0/780730312 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4218009480 con 0x7f4228159b40 2023-11-06T02:13:33.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.802+0000 7f42277fe640 1 --2- 172.21.15.49:0/780730312 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4228159b40 0x7f4228146940 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f422000cb80 tx=0x7f422000a4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:33.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.802+0000 7f420f7fe640 1 -- 172.21.15.49:0/780730312 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f422000ce70 con 0x7f4228159b40 2023-11-06T02:13:33.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.802+0000 7f420f7fe640 1 -- 172.21.15.49:0/780730312 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f422000abd0 con 0x7f4228159b40 2023-11-06T02:13:33.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.802+0000 7f422d735640 1 -- 172.21.15.49:0/780730312 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42280c21d0 con 0x7f4228159b40 2023-11-06T02:13:33.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.802+0000 7f420f7fe640 1 -- 172.21.15.49:0/780730312 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4220011410 con 0x7f4228159b40 2023-11-06T02:13:33.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.802+0000 7f422d735640 1 -- 172.21.15.49:0/780730312 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42280c2790 con 0x7f4228159b40 2023-11-06T02:13:33.807 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.802+0000 7f422d735640 1 -- 172.21.15.49:0/780730312 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42280c2c70 con 0x7f4228159b40 2023-11-06T02:13:33.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.802+0000 7f420f7fe640 1 -- 172.21.15.49:0/780730312 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f4220011570 con 0x7f4228159b40 2023-11-06T02:13:33.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.806+0000 7f420f7fe640 1 --2- 172.21.15.49:0/780730312 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4214065a60 0x7f4214067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:33.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.806+0000 7f420f7fe640 1 -- 172.21.15.49:0/780730312 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4220083080 con 0x7f4228159b40 2023-11-06T02:13:33.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.806+0000 7f4226ffd640 1 --2- 172.21.15.49:0/780730312 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4214065a60 0x7f4214067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:33.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.806+0000 7f4226ffd640 1 --2- 172.21.15.49:0/780730312 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4214065a60 0x7f4214067f20 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f42180ad000 tx=0x7f42180a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:33.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.810+0000 7f420f7fe640 1 -- 172.21.15.49:0/780730312 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f4220050970 con 0x7f4228159b40 2023-11-06T02:13:33.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.966+0000 7f422d735640 1 -- 172.21.15.49:0/780730312 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4228074040 con 0x7f4228159b40 2023-11-06T02:13:33.966 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.970+0000 7f420f7fe640 1 -- 172.21.15.49:0/780730312 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+17899 (secure 0 0 0) 0x7f4220053f90 con 0x7f4228159b40 2023-11-06T02:13:33.966 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:13:33.967 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":79,"fsid":"1a8b035c-6ba4-499a-b876-aea47fd9c0d4","created":"2023-11-06T02:08:06.440857+0000","modified":"2023-11-06T02:13:33.624959+0000","last_up_change":"2023-11-06T02:08:17.982372+0000","last_in_change":"2023-11-06T02:08:08.906631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T02:08:19.252222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T02:08:24.776211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-11-06T02:13:31.459314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-11-06T02:13:32.603710+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ad55af8-0571-4e6a-8722-b5d429b7496d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6809","nonce":38116},{"type":"v1","addr":"172.21.15.49:6811","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6813","nonce":38116},{"type":"v1","addr":"172.21.15.49:6815","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6821","nonce":38116},{"type":"v1","addr":"172.21.15.49:6823","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6817","nonce":38116},{"type":"v1","addr":"172.21.15.49:6819","nonce":38116}]},"public_addr":"172.21.15.49:6811/38116","cluster_addr":"172.21.15.49:6815/38116","heartbeat_back_addr":"172.21.15.49:6823/38116","heartbeat_front_addr":"172.21.15.49:6819/38116","state":["exists","up"]},{"osd":1,"uuid":"c8b31f85-a757-43ed-9419-351acc46611b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":38113},{"type":"v1","addr":"172.21.15.49:6810","nonce":38113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":38113},{"type":"v1","addr":"172.21.15.49:6814","nonce":38113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":38113},{"type":"v1","addr":"172.21.15.49:6822","nonce":38113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":38113},{"type":"v1","addr":"172.21.15.49:6818","nonce":38113}]},"public_addr":"172.21.15.49:6810/38113","cluster_addr":"172.21.15.49:6814/38113","heartbeat_back_addr":"172.21.15.49:6822/38113","heartbeat_front_addr":"172.21.15.49:6818/38113","state":["exists","up"]},{"osd":2,"uuid":"555b3bf3-0a70-4b91-8efd-e4204b5fc26e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":38114},{"type":"v1","addr":"172.21.15.49:6801","nonce":38114}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":38114},{"type":"v1","addr":"172.21.15.49:6803","nonce":38114}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":38114},{"type":"v1","addr":"172.21.15.49:6807","nonce":38114}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":38114},{"type":"v1","addr":"172.21.15.49:6805","nonce":38114}]},"public_addr":"172.21.15.49:6801/38114","cluster_addr":"172.21.15.49:6803/38114","heartbeat_back_addr":"172.21.15.49:6807/38114","heartbeat_front_addr":"172.21.15.49:6805/38114","state":["exists","up"]},{"osd":3,"uuid":"8d4e5d77-03ef-4b9b-bfcb-5ae93a46795c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":38115},{"type":"v1","addr":"172.21.15.49:6825","nonce":38115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":38115},{"type":"v1","addr":"172.21.15.49:6827","nonce":38115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":38115},{"type":"v1","addr":"172.21.15.49:6831","nonce":38115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":38115},{"type":"v1","addr":"172.21.15.49:6829","nonce":38115}]},"public_addr":"172.21.15.49:6825/38115","cluster_addr":"172.21.15.49:6827/38115","heartbeat_back_addr":"172.21.15.49:6831/38115","heartbeat_front_addr":"172.21.15.49:6829/38115","state":["exists","up"]},{"osd":4,"uuid":"dba03ad4-68a3-4cf3-9218-ac611f02eacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":38417},{"type":"v1","addr":"172.21.15.150:6809","nonce":38417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":38417},{"type":"v1","addr":"172.21.15.150:6811","nonce":38417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":38417},{"type":"v1","addr":"172.21.15.150:6815","nonce":38417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":38417},{"type":"v1","addr":"172.21.15.150:6813","nonce":38417}]},"public_addr":"172.21.15.150:6809/38417","cluster_addr":"172.21.15.150:6811/38417","heartbeat_back_addr":"172.21.15.150:6815/38417","heartbeat_front_addr":"172.21.15.150:6813/38417","state":["exists","up"]},{"osd":5,"uuid":"0df9c303-c255-4640-8e91-01f1c3efe18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":38418},{"type":"v1","addr":"172.21.15.150:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":38418},{"type":"v1","addr":"172.21.15.150:6803","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":38418},{"type":"v1","addr":"172.21.15.150:6807","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":38418},{"type":"v1","addr":"172.21.15.150:6805","nonce":38418}]},"public_addr":"172.21.15.150:6801/38418","cluster_addr":"172.21.15.150:6803/38418","heartbeat_back_addr":"172.21.15.150:6807/38418","heartbeat_front_addr":"172.21.15.150:6805/38418","state":["exists","up"]},{"osd":6,"uuid":"e59fa189-4b70-4c06-9722-c02b90828bf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":38428},{"type":"v1","addr":"172.21.15.150:6825","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":38428},{"type":"v1","addr":"172.21.15.150:6827","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":38428},{"type":"v1","addr":"172.21.15.150:6831","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":38428},{"type":"v1","addr":"172.21.15.150:6829","nonce":38428}]},"public_addr":"172.21.15.150:6825/38428","cluster_addr":"172.21.15.150:6827/38428","heartbeat_back_addr":"172.21.15.150:6831/38428","heartbeat_front_addr":"172.21.15.150:6829/38428","state":["exists","up"]},{"osd":7,"uuid":"a64f57e1-ef98-46bd-b304-385759e95898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":38421},{"type":"v1","addr":"172.21.15.150:6817","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":38421},{"type":"v1","addr":"172.21.15.150:6819","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":38421},{"type":"v1","addr":"172.21.15.150:6823","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":38421},{"type":"v1","addr":"172.21.15.150:6821","nonce":38421}]},"public_addr":"172.21.15.150:6817/38421","cluster_addr":"172.21.15.150:6819/38421","heartbeat_back_addr":"172.21.15.150:6823/38421","heartbeat_front_addr":"172.21.15.150:6821/38421","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.438552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.413280+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.296466+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.424966+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.276953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.230984+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.410235+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.480084+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T02:13:33.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.970+0000 7f422d735640 1 -- 172.21.15.49:0/780730312 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4214065a60 msgr2=0x7f4214067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:33.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.970+0000 7f422d735640 1 --2- 172.21.15.49:0/780730312 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4214065a60 0x7f4214067f20 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f42180ad000 tx=0x7f42180a6000 comp rx=0 tx=0).stop 2023-11-06T02:13:33.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.970+0000 7f422d735640 1 -- 172.21.15.49:0/780730312 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4228159b40 msgr2=0x7f4228146940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:33.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.970+0000 7f422d735640 1 --2- 172.21.15.49:0/780730312 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4228159b40 0x7f4228146940 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f422000cb80 tx=0x7f422000a4c0 comp rx=0 tx=0).stop 2023-11-06T02:13:33.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.970+0000 7f422d735640 1 -- 172.21.15.49:0/780730312 shutdown_connections 2023-11-06T02:13:33.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.970+0000 7f422d735640 1 --2- 172.21.15.49:0/780730312 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4214065a60 0x7f4214067f20 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:33.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.970+0000 7f422d735640 1 --2- 172.21.15.49:0/780730312 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4228159b40 0x7f4228146940 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:33.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.970+0000 7f422d735640 1 --2- 172.21.15.49:0/780730312 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f42280cdd50 0x7f4228146400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:33.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.970+0000 7f422d735640 1 --2- 172.21.15.49:0/780730312 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f42280cb010 0x7f4228145b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:33.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.970+0000 7f422d735640 1 -- 172.21.15.49:0/780730312 >> 172.21.15.49:0/780730312 conn(0x7f422806c0a0 msgr2=0x7f42280af800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:33.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.970+0000 7f422d735640 1 -- 172.21.15.49:0/780730312 shutdown_connections 2023-11-06T02:13:33.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:33.970+0000 7f422d735640 1 -- 172.21.15.49:0/780730312 wait complete. 2023-11-06T02:13:33.985 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T02:13:34.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.114+0000 7f76481b2640 1 Processor -- start 2023-11-06T02:13:34.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.114+0000 7f76481b2640 1 -- start start 2023-11-06T02:13:34.113 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.118+0000 7f76481b2640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f764015cd70 0x7f764006b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:34.113 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.118+0000 7f76481b2640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76400aff70 con 0x7f764006bfa0 2023-11-06T02:13:34.113 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.118+0000 7f76481b2640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76400b0dd0 con 0x7f76400b0830 2023-11-06T02:13:34.113 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.118+0000 7f76481b2640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76400b10b0 con 0x7f764015cd70 2023-11-06T02:13:34.114 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.118+0000 7f7645f4e640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f764015cd70 0x7f764006b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:34.114 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.118+0000 7f764674f640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f764006bfa0 0x7f76400b2310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:60578/0 (socket says 172.21.15.49:60578) 2023-11-06T02:13:34.114 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.118+0000 7f764674f640 1 -- 172.21.15.49:0/1354047514 learned_addr learned my addr 172.21.15.49:0/1354047514 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:13:34.114 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.118+0000 7f7644f4c640 1 -- 172.21.15.49:0/1354047514 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3306383987 0 0) 0x7f76400aff70 con 0x7f764006bfa0 2023-11-06T02:13:34.114 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.118+0000 7f7644f4c640 1 -- 172.21.15.49:0/1354047514 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f762c003530 con 0x7f764006bfa0 2023-11-06T02:13:34.115 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.118+0000 7f7645f4e640 1 -- 172.21.15.49:0/1354047514 >> v1:172.21.15.150:6789/0 conn(0x7f76400b0830 legacy=0x7f764006b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:34.115 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.118+0000 7f7645f4e640 1 -- 172.21.15.49:0/1354047514 >> v1:172.21.15.49:6789/0 conn(0x7f764006bfa0 legacy=0x7f76400b2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:34.115 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.118+0000 7f7645f4e640 1 -- 172.21.15.49:0/1354047514 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76400b1390 con 0x7f764015cd70 2023-11-06T02:13:34.116 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.118+0000 7f7645f4e640 1 --2- 172.21.15.49:0/1354047514 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f764015cd70 0x7f764006b220 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f763c0097a0 tx=0x7f763c01c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a4c102fc6470391 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:34.116 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.118+0000 7f7644f4c640 1 -- 172.21.15.49:0/1354047514 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f763c0ad660 con 0x7f764015cd70 2023-11-06T02:13:34.116 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.118+0000 7f7644f4c640 1 -- 172.21.15.49:0/1354047514 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f763c0ad7c0 con 0x7f764015cd70 2023-11-06T02:13:34.117 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.122+0000 7f7644f4c640 1 -- 172.21.15.49:0/1354047514 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f763c0adb50 con 0x7f764015cd70 2023-11-06T02:13:34.118 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.122+0000 7f76481b2640 1 -- 172.21.15.49:0/1354047514 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f764015cd70 msgr2=0x7f764006b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:34.118 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.122+0000 7f76481b2640 1 --2- 172.21.15.49:0/1354047514 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f764015cd70 0x7f764006b220 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f763c0097a0 tx=0x7f763c01c340 comp rx=0 tx=0).stop 2023-11-06T02:13:34.118 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.122+0000 7f76481b2640 1 -- 172.21.15.49:0/1354047514 shutdown_connections 2023-11-06T02:13:34.118 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.122+0000 7f76481b2640 1 --2- 172.21.15.49:0/1354047514 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f764015cd70 0x7f764006b220 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:34.119 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.122+0000 7f76481b2640 1 -- 172.21.15.49:0/1354047514 >> 172.21.15.49:0/1354047514 conn(0x7f7640089fd0 msgr2=0x7f7640099c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:34.119 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.122+0000 7f76481b2640 1 -- 172.21.15.49:0/1354047514 shutdown_connections 2023-11-06T02:13:34.119 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.122+0000 7f76481b2640 1 -- 172.21.15.49:0/1354047514 wait complete. 2023-11-06T02:13:34.119 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.122+0000 7f76481b2640 1 Processor -- start 2023-11-06T02:13:34.120 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.122+0000 7f76481b2640 1 -- start start 2023-11-06T02:13:34.122 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.122+0000 7f76481b2640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f764006bfa0 0x7f76400ae950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:34.122 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.122+0000 7f76481b2640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76400b0830 0x7f764015d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:34.123 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.122+0000 7f76481b2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f764015cd70 0x7f764015db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:34.123 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.122+0000 7f76481b2640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76401600e0 con 0x7f764015cd70 2023-11-06T02:13:34.123 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.122+0000 7f76481b2640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76400cfac0 con 0x7f76400b0830 2023-11-06T02:13:34.123 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.122+0000 7f76481b2640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76400cf190 con 0x7f764006bfa0 2023-11-06T02:13:34.123 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.122+0000 7f764674f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f764015cd70 0x7f764015db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:34.123 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.122+0000 7f764674f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f764015cd70 0x7f764015db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51252/0 (socket says 172.21.15.49:51252) 2023-11-06T02:13:34.124 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.122+0000 7f764674f640 1 -- 172.21.15.49:0/246570330 learned_addr learned my addr 172.21.15.49:0/246570330 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:34.124 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.122+0000 7f764574d640 1 --2- 172.21.15.49:0/246570330 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76400b0830 0x7f764015d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:34.124 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.122+0000 7f7645f4e640 1 --2- 172.21.15.49:0/246570330 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f764006bfa0 0x7f76400ae950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:34.124 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.122+0000 7f764674f640 1 -- 172.21.15.49:0/246570330 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f764006bfa0 msgr2=0x7f76400ae950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:34.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.122+0000 7f764674f640 1 --2- 172.21.15.49:0/246570330 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f764006bfa0 0x7f76400ae950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:34.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.122+0000 7f764674f640 1 -- 172.21.15.49:0/246570330 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76400b0830 msgr2=0x7f764015d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:34.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.122+0000 7f764674f640 1 --2- 172.21.15.49:0/246570330 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76400b0830 0x7f764015d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:34.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.122+0000 7f764674f640 1 -- 172.21.15.49:0/246570330 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f763c0093b0 con 0x7f764015cd70 2023-11-06T02:13:34.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.122+0000 7f764574d640 1 --2- 172.21.15.49:0/246570330 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76400b0830 0x7f764015d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:13:34.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.122+0000 7f764674f640 1 --2- 172.21.15.49:0/246570330 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f764015cd70 0x7f764015db50 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f763800caa0 tx=0x7f763800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:34.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.122+0000 7f76367fc640 1 -- 172.21.15.49:0/246570330 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f763800ac00 con 0x7f764015cd70 2023-11-06T02:13:34.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.122+0000 7f76367fc640 1 -- 172.21.15.49:0/246570330 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f763800ad60 con 0x7f764015cd70 2023-11-06T02:13:34.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.122+0000 7f76481b2640 1 -- 172.21.15.49:0/246570330 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76400cf470 con 0x7f764015cd70 2023-11-06T02:13:34.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.122+0000 7f76367fc640 1 -- 172.21.15.49:0/246570330 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7638012620 con 0x7f764015cd70 2023-11-06T02:13:34.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.122+0000 7f76481b2640 1 -- 172.21.15.49:0/246570330 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f764014e890 con 0x7f764015cd70 2023-11-06T02:13:34.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.126+0000 7f76481b2640 1 -- 172.21.15.49:0/246570330 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76400b03e0 con 0x7f764015cd70 2023-11-06T02:13:34.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.126+0000 7f76367fc640 1 -- 172.21.15.49:0/246570330 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f7638007070 con 0x7f764015cd70 2023-11-06T02:13:34.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.126+0000 7f76367fc640 1 --2- 172.21.15.49:0/246570330 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7624065a60 0x7f7624067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:34.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.126+0000 7f76367fc640 1 -- 172.21.15.49:0/246570330 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f76380832c0 con 0x7f764015cd70 2023-11-06T02:13:34.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.126+0000 7f7645f4e640 1 --2- 172.21.15.49:0/246570330 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7624065a60 0x7f7624067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:34.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.126+0000 7f7645f4e640 1 --2- 172.21.15.49:0/246570330 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7624065a60 0x7f7624067f20 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f763c0af000 tx=0x7f763c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:34.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.130+0000 7f76367fc640 1 -- 172.21.15.49:0/246570330 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f7638050b20 con 0x7f764015cd70 2023-11-06T02:13:34.283 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.286+0000 7f76481b2640 1 -- 172.21.15.49:0/246570330 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f76400cfc00 con 0x7f764015cd70 2023-11-06T02:13:34.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.290+0000 7f76367fc640 1 -- 172.21.15.49:0/246570330 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+17899 (secure 0 0 0) 0x7f7638054140 con 0x7f764015cd70 2023-11-06T02:13:34.285 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:13:34.286 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":79,"fsid":"1a8b035c-6ba4-499a-b876-aea47fd9c0d4","created":"2023-11-06T02:08:06.440857+0000","modified":"2023-11-06T02:13:33.624959+0000","last_up_change":"2023-11-06T02:08:17.982372+0000","last_in_change":"2023-11-06T02:08:08.906631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T02:08:19.252222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T02:08:24.776211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-11-06T02:13:31.459314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-11-06T02:13:32.603710+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ad55af8-0571-4e6a-8722-b5d429b7496d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6809","nonce":38116},{"type":"v1","addr":"172.21.15.49:6811","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6813","nonce":38116},{"type":"v1","addr":"172.21.15.49:6815","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6821","nonce":38116},{"type":"v1","addr":"172.21.15.49:6823","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6817","nonce":38116},{"type":"v1","addr":"172.21.15.49:6819","nonce":38116}]},"public_addr":"172.21.15.49:6811/38116","cluster_addr":"172.21.15.49:6815/38116","heartbeat_back_addr":"172.21.15.49:6823/38116","heartbeat_front_addr":"172.21.15.49:6819/38116","state":["exists","up"]},{"osd":1,"uuid":"c8b31f85-a757-43ed-9419-351acc46611b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":38113},{"type":"v1","addr":"172.21.15.49:6810","nonce":38113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":38113},{"type":"v1","addr":"172.21.15.49:6814","nonce":38113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":38113},{"type":"v1","addr":"172.21.15.49:6822","nonce":38113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":38113},{"type":"v1","addr":"172.21.15.49:6818","nonce":38113}]},"public_addr":"172.21.15.49:6810/38113","cluster_addr":"172.21.15.49:6814/38113","heartbeat_back_addr":"172.21.15.49:6822/38113","heartbeat_front_addr":"172.21.15.49:6818/38113","state":["exists","up"]},{"osd":2,"uuid":"555b3bf3-0a70-4b91-8efd-e4204b5fc26e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":38114},{"type":"v1","addr":"172.21.15.49:6801","nonce":38114}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":38114},{"type":"v1","addr":"172.21.15.49:6803","nonce":38114}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":38114},{"type":"v1","addr":"172.21.15.49:6807","nonce":38114}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":38114},{"type":"v1","addr":"172.21.15.49:6805","nonce":38114}]},"public_addr":"172.21.15.49:6801/38114","cluster_addr":"172.21.15.49:6803/38114","heartbeat_back_addr":"172.21.15.49:6807/38114","heartbeat_front_addr":"172.21.15.49:6805/38114","state":["exists","up"]},{"osd":3,"uuid":"8d4e5d77-03ef-4b9b-bfcb-5ae93a46795c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":38115},{"type":"v1","addr":"172.21.15.49:6825","nonce":38115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":38115},{"type":"v1","addr":"172.21.15.49:6827","nonce":38115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":38115},{"type":"v1","addr":"172.21.15.49:6831","nonce":38115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":38115},{"type":"v1","addr":"172.21.15.49:6829","nonce":38115}]},"public_addr":"172.21.15.49:6825/38115","cluster_addr":"172.21.15.49:6827/38115","heartbeat_back_addr":"172.21.15.49:6831/38115","heartbeat_front_addr":"172.21.15.49:6829/38115","state":["exists","up"]},{"osd":4,"uuid":"dba03ad4-68a3-4cf3-9218-ac611f02eacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":38417},{"type":"v1","addr":"172.21.15.150:6809","nonce":38417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":38417},{"type":"v1","addr":"172.21.15.150:6811","nonce":38417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":38417},{"type":"v1","addr":"172.21.15.150:6815","nonce":38417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":38417},{"type":"v1","addr":"172.21.15.150:6813","nonce":38417}]},"public_addr":"172.21.15.150:6809/38417","cluster_addr":"172.21.15.150:6811/38417","heartbeat_back_addr":"172.21.15.150:6815/38417","heartbeat_front_addr":"172.21.15.150:6813/38417","state":["exists","up"]},{"osd":5,"uuid":"0df9c303-c255-4640-8e91-01f1c3efe18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":38418},{"type":"v1","addr":"172.21.15.150:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":38418},{"type":"v1","addr":"172.21.15.150:6803","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":38418},{"type":"v1","addr":"172.21.15.150:6807","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":38418},{"type":"v1","addr":"172.21.15.150:6805","nonce":38418}]},"public_addr":"172.21.15.150:6801/38418","cluster_addr":"172.21.15.150:6803/38418","heartbeat_back_addr":"172.21.15.150:6807/38418","heartbeat_front_addr":"172.21.15.150:6805/38418","state":["exists","up"]},{"osd":6,"uuid":"e59fa189-4b70-4c06-9722-c02b90828bf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":38428},{"type":"v1","addr":"172.21.15.150:6825","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":38428},{"type":"v1","addr":"172.21.15.150:6827","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":38428},{"type":"v1","addr":"172.21.15.150:6831","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":38428},{"type":"v1","addr":"172.21.15.150:6829","nonce":38428}]},"public_addr":"172.21.15.150:6825/38428","cluster_addr":"172.21.15.150:6827/38428","heartbeat_back_addr":"172.21.15.150:6831/38428","heartbeat_front_addr":"172.21.15.150:6829/38428","state":["exists","up"]},{"osd":7,"uuid":"a64f57e1-ef98-46bd-b304-385759e95898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":38421},{"type":"v1","addr":"172.21.15.150:6817","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":38421},{"type":"v1","addr":"172.21.15.150:6819","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":38421},{"type":"v1","addr":"172.21.15.150:6823","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":38421},{"type":"v1","addr":"172.21.15.150:6821","nonce":38421}]},"public_addr":"172.21.15.150:6817/38421","cluster_addr":"172.21.15.150:6819/38421","heartbeat_back_addr":"172.21.15.150:6823/38421","heartbeat_front_addr":"172.21.15.150:6821/38421","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.438552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.413280+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.296466+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.424966+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.276953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.230984+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.410235+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.480084+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T02:13:34.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.290+0000 7f76481b2640 1 -- 172.21.15.49:0/246570330 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7624065a60 msgr2=0x7f7624067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:34.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.290+0000 7f76481b2640 1 --2- 172.21.15.49:0/246570330 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7624065a60 0x7f7624067f20 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f763c0af000 tx=0x7f763c008040 comp rx=0 tx=0).stop 2023-11-06T02:13:34.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.290+0000 7f76481b2640 1 -- 172.21.15.49:0/246570330 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f764015cd70 msgr2=0x7f764015db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:34.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.290+0000 7f76481b2640 1 --2- 172.21.15.49:0/246570330 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f764015cd70 0x7f764015db50 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f763800caa0 tx=0x7f763800cf60 comp rx=0 tx=0).stop 2023-11-06T02:13:34.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.290+0000 7f76481b2640 1 -- 172.21.15.49:0/246570330 shutdown_connections 2023-11-06T02:13:34.289 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.290+0000 7f76481b2640 1 --2- 172.21.15.49:0/246570330 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7624065a60 0x7f7624067f20 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:34.289 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.290+0000 7f76481b2640 1 --2- 172.21.15.49:0/246570330 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f764015cd70 0x7f764015db50 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:34.289 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.290+0000 7f76481b2640 1 --2- 172.21.15.49:0/246570330 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76400b0830 0x7f764015d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:34.289 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.290+0000 7f76481b2640 1 --2- 172.21.15.49:0/246570330 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f764006bfa0 0x7f76400ae950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:34.289 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.290+0000 7f76481b2640 1 -- 172.21.15.49:0/246570330 >> 172.21.15.49:0/246570330 conn(0x7f7640089fd0 msgr2=0x7f76400cd0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:34.289 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.290+0000 7f76481b2640 1 -- 172.21.15.49:0/246570330 shutdown_connections 2023-11-06T02:13:34.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.290+0000 7f76481b2640 1 -- 172.21.15.49:0/246570330 wait complete. 2023-11-06T02:13:34.307 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-06T02:13:34.434 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.438+0000 7f6c7f4d7640 1 Processor -- start 2023-11-06T02:13:34.434 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.438+0000 7f6c7f4d7640 1 -- start start 2023-11-06T02:13:34.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.438+0000 7f6c7f4d7640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f6c781595a0 0x7f6c7815dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:34.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.438+0000 7f6c7f4d7640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f6c7815a990 0x7f6c780c8e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:34.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.438+0000 7f6c7f4d7640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c7806c5a0 con 0x7f6c7806d640 2023-11-06T02:13:34.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.438+0000 7f6c7f4d7640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c7815e250 con 0x7f6c781595a0 2023-11-06T02:13:34.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.438+0000 7f6c7f4d7640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c7806c880 con 0x7f6c7815a990 2023-11-06T02:13:34.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.438+0000 7f6c7d273640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f6c781595a0 0x7f6c7815dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:34.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.438+0000 7f6c7d273640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f6c781595a0 0x7f6c7815dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51254/0 (socket says 172.21.15.49:51254) 2023-11-06T02:13:34.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.438+0000 7f6c7d273640 1 -- 172.21.15.49:0/3821068535 learned_addr learned my addr 172.21.15.49:0/3821068535 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:34.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.438+0000 7f6c7d273640 1 -- 172.21.15.49:0/3821068535 >> v2:172.21.15.150:3300/0 conn(0x7f6c7815a990 msgr2=0x7f6c780c8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:34.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.438+0000 7f6c7da74640 1 --2- 172.21.15.49:0/3821068535 >> v2:172.21.15.150:3300/0 conn(0x7f6c7815a990 0x7f6c780c8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:34.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.438+0000 7f6c7d273640 1 --2- 172.21.15.49:0/3821068535 >> v2:172.21.15.150:3300/0 conn(0x7f6c7815a990 0x7f6c780c8e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:34.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.438+0000 7f6c7d273640 1 -- 172.21.15.49:0/3821068535 >> v1:172.21.15.150:6789/0 conn(0x7f6c7806d640 legacy=0x7f6c780b0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:34.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.438+0000 7f6c7d273640 1 -- 172.21.15.49:0/3821068535 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c7806cb60 con 0x7f6c781595a0 2023-11-06T02:13:34.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.442+0000 7f6c7d273640 1 --2- 172.21.15.49:0/3821068535 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c781595a0 0x7f6c7815dd10 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f6c7806adb0 tx=0x7f6c7401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94e18aeaf34acd30 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:34.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.442+0000 7f6c6ffff640 1 -- 172.21.15.49:0/3821068535 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c740ac600 con 0x7f6c781595a0 2023-11-06T02:13:34.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.442+0000 7f6c6ffff640 1 -- 172.21.15.49:0/3821068535 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c740a8050 con 0x7f6c781595a0 2023-11-06T02:13:34.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.442+0000 7f6c6ffff640 1 -- 172.21.15.49:0/3821068535 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c740ac980 con 0x7f6c781595a0 2023-11-06T02:13:34.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.442+0000 7f6c7f4d7640 1 -- 172.21.15.49:0/3821068535 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c781595a0 msgr2=0x7f6c7815dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:34.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.442+0000 7f6c7f4d7640 1 --2- 172.21.15.49:0/3821068535 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c781595a0 0x7f6c7815dd10 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f6c7806adb0 tx=0x7f6c7401c490 comp rx=0 tx=0).stop 2023-11-06T02:13:34.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.442+0000 7f6c7f4d7640 1 -- 172.21.15.49:0/3821068535 shutdown_connections 2023-11-06T02:13:34.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.442+0000 7f6c7f4d7640 1 --2- 172.21.15.49:0/3821068535 >> v2:172.21.15.150:3300/0 conn(0x7f6c7815a990 0x7f6c780c8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:34.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.442+0000 7f6c7f4d7640 1 --2- 172.21.15.49:0/3821068535 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c781595a0 0x7f6c7815dd10 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:34.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.442+0000 7f6c7f4d7640 1 -- 172.21.15.49:0/3821068535 >> 172.21.15.49:0/3821068535 conn(0x7f6c7806a370 msgr2=0x7f6c780ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:34.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.442+0000 7f6c7f4d7640 1 -- 172.21.15.49:0/3821068535 shutdown_connections 2023-11-06T02:13:34.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.442+0000 7f6c7f4d7640 1 -- 172.21.15.49:0/3821068535 wait complete. 2023-11-06T02:13:34.444 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.446+0000 7f6c7f4d7640 1 Processor -- start 2023-11-06T02:13:34.444 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.446+0000 7f6c7f4d7640 1 -- start start 2023-11-06T02:13:34.445 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.446+0000 7f6c7f4d7640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6c7806d640 0x7f6c7814c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:34.445 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.446+0000 7f6c7f4d7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c781595a0 0x7f6c780c7940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:34.445 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.446+0000 7f6c7f4d7640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6c7815a990 0x7f6c780c7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:34.445 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.446+0000 7f6c7f4d7640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c780b1f80 con 0x7f6c781595a0 2023-11-06T02:13:34.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.446+0000 7f6c7f4d7640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c780b2260 con 0x7f6c7806d640 2023-11-06T02:13:34.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.446+0000 7f6c7f4d7640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c780b2540 con 0x7f6c7815a990 2023-11-06T02:13:34.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.446+0000 7f6c7ca72640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c781595a0 0x7f6c780c7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:34.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.446+0000 7f6c7ca72640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c781595a0 0x7f6c780c7940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51262/0 (socket says 172.21.15.49:51262) 2023-11-06T02:13:34.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.446+0000 7f6c7ca72640 1 -- 172.21.15.49:0/3211684866 learned_addr learned my addr 172.21.15.49:0/3211684866 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:34.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.446+0000 7f6c7d273640 1 --2- 172.21.15.49:0/3211684866 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6c7806d640 0x7f6c7814c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:34.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.446+0000 7f6c7ca72640 1 -- 172.21.15.49:0/3211684866 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6c7815a990 msgr2=0x7f6c780c7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:34.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.446+0000 7f6c7ca72640 1 --2- 172.21.15.49:0/3211684866 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6c7815a990 0x7f6c780c7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:34.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.446+0000 7f6c7ca72640 1 -- 172.21.15.49:0/3211684866 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6c7806d640 msgr2=0x7f6c7814c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:34.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.446+0000 7f6c7ca72640 1 --2- 172.21.15.49:0/3211684866 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6c7806d640 0x7f6c7814c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:34.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.446+0000 7f6c7ca72640 1 -- 172.21.15.49:0/3211684866 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c740094d0 con 0x7f6c781595a0 2023-11-06T02:13:34.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.446+0000 7f6c7d273640 1 --2- 172.21.15.49:0/3211684866 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6c7806d640 0x7f6c7814c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:13:34.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.446+0000 7f6c7ca72640 1 --2- 172.21.15.49:0/3211684866 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c781595a0 0x7f6c780c7940 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f6c6800c450 tx=0x7f6c6800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:34.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.446+0000 7f6c6dffb640 1 -- 172.21.15.49:0/3211684866 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c68012830 con 0x7f6c781595a0 2023-11-06T02:13:34.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.446+0000 7f6c6dffb640 1 -- 172.21.15.49:0/3211684866 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c680129b0 con 0x7f6c781595a0 2023-11-06T02:13:34.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.446+0000 7f6c7f4d7640 1 -- 172.21.15.49:0/3211684866 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c780b2820 con 0x7f6c781595a0 2023-11-06T02:13:34.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.446+0000 7f6c6dffb640 1 -- 172.21.15.49:0/3211684866 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c68012d60 con 0x7f6c781595a0 2023-11-06T02:13:34.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.446+0000 7f6c7f4d7640 1 -- 172.21.15.49:0/3211684866 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c7814c760 con 0x7f6c781595a0 2023-11-06T02:13:34.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.446+0000 7f6c7f4d7640 1 -- 172.21.15.49:0/3211684866 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c7806c210 con 0x7f6c781595a0 2023-11-06T02:13:34.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.446+0000 7f6c6dffb640 1 -- 172.21.15.49:0/3211684866 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f6c6801c460 con 0x7f6c781595a0 2023-11-06T02:13:34.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.446+0000 7f6c6dffb640 1 --2- 172.21.15.49:0/3211684866 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6c5c065a90 0x7f6c5c067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:34.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.450+0000 7f6c6dffb640 1 -- 172.21.15.49:0/3211684866 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6c68083ad0 con 0x7f6c781595a0 2023-11-06T02:13:34.451 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.450+0000 7f6c7d273640 1 --2- 172.21.15.49:0/3211684866 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6c5c065a90 0x7f6c5c067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:34.451 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.450+0000 7f6c7d273640 1 --2- 172.21.15.49:0/3211684866 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6c5c065a90 0x7f6c5c067f50 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f6c740ae000 tx=0x7f6c74008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:34.451 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.450+0000 7f6c6dffb640 1 -- 172.21.15.49:0/3211684866 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f6c680513a0 con 0x7f6c781595a0 2023-11-06T02:13:34.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:34.642+0000 7f6c7f4d7640 1 -- 172.21.15.49:0/3211684866 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f6c7806c210 con 0x7f6c781595a0 2023-11-06T02:13:35.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:35.646+0000 7f6c6dffb640 1 -- 172.21.15.49:0/3211684866 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v45) v1 ==== 118+0+0 (secure 0 0 0) 0x7f6c7806c210 con 0x7f6c781595a0 2023-11-06T02:13:35.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:35.650+0000 7f6c7f4d7640 1 -- 172.21.15.49:0/3211684866 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6c5c065a90 msgr2=0x7f6c5c067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:35.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:35.650+0000 7f6c7f4d7640 1 --2- 172.21.15.49:0/3211684866 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6c5c065a90 0x7f6c5c067f50 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f6c740ae000 tx=0x7f6c74008040 comp rx=0 tx=0).stop 2023-11-06T02:13:35.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:35.650+0000 7f6c7f4d7640 1 -- 172.21.15.49:0/3211684866 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c781595a0 msgr2=0x7f6c780c7940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:35.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:35.650+0000 7f6c7f4d7640 1 --2- 172.21.15.49:0/3211684866 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c781595a0 0x7f6c780c7940 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f6c6800c450 tx=0x7f6c6800c910 comp rx=0 tx=0).stop 2023-11-06T02:13:35.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:35.650+0000 7f6c7f4d7640 1 -- 172.21.15.49:0/3211684866 shutdown_connections 2023-11-06T02:13:35.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:35.650+0000 7f6c7f4d7640 1 --2- 172.21.15.49:0/3211684866 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6c5c065a90 0x7f6c5c067f50 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:35.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:35.650+0000 7f6c7f4d7640 1 --2- 172.21.15.49:0/3211684866 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c781595a0 0x7f6c780c7940 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:35.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:35.650+0000 7f6c7f4d7640 1 --2- 172.21.15.49:0/3211684866 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6c7815a990 0x7f6c780c7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:35.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:35.650+0000 7f6c7f4d7640 1 --2- 172.21.15.49:0/3211684866 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6c7806d640 0x7f6c7814c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:35.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:35.650+0000 7f6c7f4d7640 1 -- 172.21.15.49:0/3211684866 >> 172.21.15.49:0/3211684866 conn(0x7f6c7806a370 msgr2=0x7f6c780cc170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:35.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:35.650+0000 7f6c7f4d7640 1 -- 172.21.15.49:0/3211684866 shutdown_connections 2023-11-06T02:13:35.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:35.650+0000 7f6c7f4d7640 1 -- 172.21.15.49:0/3211684866 wait complete. 2023-11-06T02:13:35.670 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-06T02:13:35.670 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-06T02:13:35.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:35.810+0000 7f68c8fe7640 1 Processor -- start 2023-11-06T02:13:35.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:35.810+0000 7f68c8fe7640 1 -- start start 2023-11-06T02:13:35.807 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:35.810+0000 7f68c8fe7640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f68c415e0c0 0x7f68c40ce840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:35.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:35.810+0000 7f68c8fe7640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f68c40b16e0 0x7f68c40b1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:35.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:35.810+0000 7f68c8fe7640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68c40d1430 con 0x7f68c415e4a0 2023-11-06T02:13:35.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:35.810+0000 7f68c8fe7640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68c40cd8d0 con 0x7f68c40b16e0 2023-11-06T02:13:35.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:35.810+0000 7f68c8fe7640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68c40b20d0 con 0x7f68c415e0c0 2023-11-06T02:13:35.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:35.810+0000 7f68c2d9d640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f68c40b16e0 0x7f68c40b1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:35.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:35.810+0000 7f68c2d9d640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f68c40b16e0 0x7f68c40b1b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51270/0 (socket says 172.21.15.49:51270) 2023-11-06T02:13:35.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:35.810+0000 7f68c2d9d640 1 -- 172.21.15.49:0/2469348244 learned_addr learned my addr 172.21.15.49:0/2469348244 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:35.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:35.810+0000 7f68c1d9b640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f68c415e4a0 0x7f68c40cdcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:60594/0 (socket says 172.21.15.49:60594) 2023-11-06T02:13:35.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:35.810+0000 7f68c259c640 1 --2- 172.21.15.49:0/2469348244 >> v2:172.21.15.150:3300/0 conn(0x7f68c415e0c0 0x7f68c40ce840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:35.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:35.810+0000 7f68c2d9d640 1 -- 172.21.15.49:0/2469348244 >> v2:172.21.15.150:3300/0 conn(0x7f68c415e0c0 msgr2=0x7f68c40ce840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:35.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:35.810+0000 7f68c2d9d640 1 --2- 172.21.15.49:0/2469348244 >> v2:172.21.15.150:3300/0 conn(0x7f68c415e0c0 0x7f68c40ce840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:35.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:35.810+0000 7f68c2d9d640 1 -- 172.21.15.49:0/2469348244 >> v1:172.21.15.49:6789/0 conn(0x7f68c415e4a0 legacy=0x7f68c40cdcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:35.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:35.810+0000 7f68c2d9d640 1 -- 172.21.15.49:0/2469348244 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68c40d1710 con 0x7f68c40b16e0 2023-11-06T02:13:35.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:35.810+0000 7f68c259c640 1 --2- 172.21.15.49:0/2469348244 >> v2:172.21.15.150:3300/0 conn(0x7f68c415e0c0 0x7f68c40ce840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:13:35.813 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:35.814+0000 7f68c2d9d640 1 --2- 172.21.15.49:0/2469348244 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f68c40b16e0 0x7f68c40b1b90 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f68c408a2e0 tx=0x7f68bc01c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f989dc71adbc8c4 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:35.813 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:35.814+0000 7f68c159a640 1 -- 172.21.15.49:0/2469348244 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68bc0a56f0 con 0x7f68c40b16e0 2023-11-06T02:13:35.813 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:35.814+0000 7f68c159a640 1 -- 172.21.15.49:0/2469348244 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68bc0a5850 con 0x7f68c40b16e0 2023-11-06T02:13:35.814 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:35.814+0000 7f68c159a640 1 -- 172.21.15.49:0/2469348244 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68bc0a5be0 con 0x7f68c40b16e0 2023-11-06T02:13:35.814 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:35.814+0000 7f68c8fe7640 1 -- 172.21.15.49:0/2469348244 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f68c40b16e0 msgr2=0x7f68c40b1b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:35.814 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:35.814+0000 7f68c8fe7640 1 --2- 172.21.15.49:0/2469348244 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f68c40b16e0 0x7f68c40b1b90 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f68c408a2e0 tx=0x7f68bc01c940 comp rx=0 tx=0).stop 2023-11-06T02:13:35.814 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:35.814+0000 7f68c8fe7640 1 -- 172.21.15.49:0/2469348244 shutdown_connections 2023-11-06T02:13:35.814 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:35.814+0000 7f68c8fe7640 1 --2- 172.21.15.49:0/2469348244 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f68c40b16e0 0x7f68c40b1b90 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:35.815 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:35.814+0000 7f68c8fe7640 1 --2- 172.21.15.49:0/2469348244 >> v2:172.21.15.150:3300/0 conn(0x7f68c415e0c0 0x7f68c40ce840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:35.815 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:35.814+0000 7f68c8fe7640 1 -- 172.21.15.49:0/2469348244 >> 172.21.15.49:0/2469348244 conn(0x7f68c4099c50 msgr2=0x7f68c40d0940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:35.815 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:35.814+0000 7f68c8fe7640 1 -- 172.21.15.49:0/2469348244 shutdown_connections 2023-11-06T02:13:35.815 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:35.814+0000 7f68c8fe7640 1 -- 172.21.15.49:0/2469348244 wait complete. 2023-11-06T02:13:35.815 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:35.814+0000 7f68c8fe7640 1 Processor -- start 2023-11-06T02:13:35.816 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:35.814+0000 7f68c8fe7640 1 -- start start 2023-11-06T02:13:35.816 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:35.818+0000 7f68c8fe7640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f68c40b16e0 0x7f68c415c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:35.816 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:35.818+0000 7f68c8fe7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f68c415e0c0 0x7f68c40c7a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:35.816 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:35.818+0000 7f68c8fe7640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68c415e4a0 0x7f68c4157ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:35.816 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:35.818+0000 7f68c8fe7640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c40c9ca0 con 0x7f68c415e0c0 2023-11-06T02:13:35.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:35.818+0000 7f68c8fe7640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c40c8790 con 0x7f68c415e4a0 2023-11-06T02:13:35.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:35.818+0000 7f68c8fe7640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68c40c84b0 con 0x7f68c40b16e0 2023-11-06T02:13:35.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:35.818+0000 7f68c1d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f68c415e0c0 0x7f68c40c7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:35.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:35.818+0000 7f68c1d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f68c415e0c0 0x7f68c40c7a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51272/0 (socket says 172.21.15.49:51272) 2023-11-06T02:13:35.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:35.818+0000 7f68c1d9b640 1 -- 172.21.15.49:0/494183817 learned_addr learned my addr 172.21.15.49:0/494183817 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:35.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:35.818+0000 7f68c259c640 1 --2- 172.21.15.49:0/494183817 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f68c40b16e0 0x7f68c415c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:35.818 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:35.818+0000 7f68c2d9d640 1 --2- 172.21.15.49:0/494183817 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68c415e4a0 0x7f68c4157ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:35.818 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:35.818+0000 7f68c1d9b640 1 -- 172.21.15.49:0/494183817 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f68c40b16e0 msgr2=0x7f68c415c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:35.818 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:35.818+0000 7f68c1d9b640 1 --2- 172.21.15.49:0/494183817 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f68c40b16e0 0x7f68c415c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:35.818 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:35.818+0000 7f68c1d9b640 1 -- 172.21.15.49:0/494183817 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68c415e4a0 msgr2=0x7f68c4157ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:35.818 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:35.818+0000 7f68c1d9b640 1 --2- 172.21.15.49:0/494183817 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68c415e4a0 0x7f68c4157ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:35.819 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:35.818+0000 7f68c1d9b640 1 -- 172.21.15.49:0/494183817 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68bc009b30 con 0x7f68c415e0c0 2023-11-06T02:13:35.819 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:35.818+0000 7f68c259c640 1 --2- 172.21.15.49:0/494183817 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f68c40b16e0 0x7f68c415c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:13:35.819 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:35.818+0000 7f68c2d9d640 1 --2- 172.21.15.49:0/494183817 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68c415e4a0 0x7f68c4157ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:13:35.819 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:35.818+0000 7f68c1d9b640 1 --2- 172.21.15.49:0/494183817 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f68c415e0c0 0x7f68c40c7a10 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f68b800c490 tx=0x7f68b800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:35.819 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:35.818+0000 7f68b2ffd640 1 -- 172.21.15.49:0/494183817 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b8007480 con 0x7f68c415e0c0 2023-11-06T02:13:35.820 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:35.818+0000 7f68b2ffd640 1 -- 172.21.15.49:0/494183817 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68b80075e0 con 0x7f68c415e0c0 2023-11-06T02:13:35.820 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:35.818+0000 7f68c8fe7640 1 -- 172.21.15.49:0/494183817 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68c40c9ea0 con 0x7f68c415e0c0 2023-11-06T02:13:35.820 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:35.818+0000 7f68b2ffd640 1 -- 172.21.15.49:0/494183817 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b8004a10 con 0x7f68c415e0c0 2023-11-06T02:13:35.820 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:35.818+0000 7f68c8fe7640 1 -- 172.21.15.49:0/494183817 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68c40ca460 con 0x7f68c415e0c0 2023-11-06T02:13:35.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:35.818+0000 7f68c8fe7640 1 -- 172.21.15.49:0/494183817 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68c4067140 con 0x7f68c415e0c0 2023-11-06T02:13:35.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:35.818+0000 7f68b2ffd640 1 -- 172.21.15.49:0/494183817 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f68b8004b70 con 0x7f68c415e0c0 2023-11-06T02:13:35.822 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:35.818+0000 7f68b2ffd640 1 --2- 172.21.15.49:0/494183817 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f68a4065a90 0x7f68a4067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:35.822 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:35.818+0000 7f68b2ffd640 1 -- 172.21.15.49:0/494183817 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f68b80823c0 con 0x7f68c415e0c0 2023-11-06T02:13:35.822 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:35.818+0000 7f68c259c640 1 --2- 172.21.15.49:0/494183817 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f68a4065a90 0x7f68a4067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:35.822 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:35.822+0000 7f68c259c640 1 --2- 172.21.15.49:0/494183817 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f68a4065a90 0x7f68a4067f50 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f68b4008890 tx=0x7f68b4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:35.823 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:35.822+0000 7f68b2ffd640 1 -- 172.21.15.49:0/494183817 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f68b804fc20 con 0x7f68c415e0c0 2023-11-06T02:13:36.009 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.014+0000 7f68c8fe7640 1 -- 172.21.15.49:0/494183817 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f68c409a030 con 0x7f68c415e0c0 2023-11-06T02:13:36.651 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.654+0000 7f68b2ffd640 1 -- 172.21.15.49:0/494183817 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v46) v1 ==== 122+0+0 (secure 0 0 0) 0x7f68b8053240 con 0x7f68c415e0c0 2023-11-06T02:13:36.654 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.658+0000 7f68c8fe7640 1 -- 172.21.15.49:0/494183817 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f68a4065a90 msgr2=0x7f68a4067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:36.654 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.658+0000 7f68c8fe7640 1 --2- 172.21.15.49:0/494183817 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f68a4065a90 0x7f68a4067f50 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f68b4008890 tx=0x7f68b4008040 comp rx=0 tx=0).stop 2023-11-06T02:13:36.655 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.658+0000 7f68c8fe7640 1 -- 172.21.15.49:0/494183817 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f68c415e0c0 msgr2=0x7f68c40c7a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:36.655 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.658+0000 7f68c8fe7640 1 --2- 172.21.15.49:0/494183817 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f68c415e0c0 0x7f68c40c7a10 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f68b800c490 tx=0x7f68b800c950 comp rx=0 tx=0).stop 2023-11-06T02:13:36.655 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.658+0000 7f68c8fe7640 1 -- 172.21.15.49:0/494183817 shutdown_connections 2023-11-06T02:13:36.655 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.658+0000 7f68c8fe7640 1 --2- 172.21.15.49:0/494183817 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f68a4065a90 0x7f68a4067f50 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:36.655 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.658+0000 7f68c8fe7640 1 --2- 172.21.15.49:0/494183817 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f68c415e0c0 0x7f68c40c7a10 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:36.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.658+0000 7f68c8fe7640 1 --2- 172.21.15.49:0/494183817 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68c415e4a0 0x7f68c4157ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:36.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.658+0000 7f68c8fe7640 1 --2- 172.21.15.49:0/494183817 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f68c40b16e0 0x7f68c415c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:36.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.658+0000 7f68c8fe7640 1 -- 172.21.15.49:0/494183817 >> 172.21.15.49:0/494183817 conn(0x7f68c4099c50 msgr2=0x7f68c40b0910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:36.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.658+0000 7f68c8fe7640 1 -- 172.21.15.49:0/494183817 shutdown_connections 2023-11-06T02:13:36.657 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.658+0000 7f68c8fe7640 1 -- 172.21.15.49:0/494183817 wait complete. 2023-11-06T02:13:36.673 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-06T02:13:36.674 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T02:13:36.800 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.802+0000 7f5f533cc640 1 Processor -- start 2023-11-06T02:13:36.800 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.802+0000 7f5f533cc640 1 -- start start 2023-11-06T02:13:36.801 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.806+0000 7f5f533cc640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5f4c071120 0x7f5f4c071500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:36.801 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.806+0000 7f5f533cc640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f4c071a40 con 0x7f5f4c0c9c80 2023-11-06T02:13:36.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.806+0000 7f5f533cc640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f4c0ae6e0 con 0x7f5f4c0b5270 2023-11-06T02:13:36.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.806+0000 7f5f533cc640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4c0ae9c0 con 0x7f5f4c071120 2023-11-06T02:13:36.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.806+0000 7f5f50967640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f5f4c0c9c80 0x7f5f4c0b2940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:60608/0 (socket says 172.21.15.49:60608) 2023-11-06T02:13:36.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.806+0000 7f5f50967640 1 -- 172.21.15.49:0/1920917253 learned_addr learned my addr 172.21.15.49:0/1920917253 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:13:36.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.806+0000 7f5f51168640 1 --2- 172.21.15.49:0/1920917253 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5f4c071120 0x7f5f4c071500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:36.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.806+0000 7f5f43fff640 1 -- 172.21.15.49:0/1920917253 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3729789699 0 0) 0x7f5f4c071a40 con 0x7f5f4c0c9c80 2023-11-06T02:13:36.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.806+0000 7f5f43fff640 1 -- 172.21.15.49:0/1920917253 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f38003530 con 0x7f5f4c0c9c80 2023-11-06T02:13:36.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.806+0000 7f5f43fff640 1 -- 172.21.15.49:0/1920917253 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3706033325 0 0) 0x7f5f38003530 con 0x7f5f4c0c9c80 2023-11-06T02:13:36.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.806+0000 7f5f43fff640 1 -- 172.21.15.49:0/1920917253 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5f4c071120 msgr2=0x7f5f4c071500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:36.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.806+0000 7f5f43fff640 1 --2- 172.21.15.49:0/1920917253 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5f4c071120 0x7f5f4c071500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:36.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.806+0000 7f5f43fff640 1 -- 172.21.15.49:0/1920917253 >> v1:172.21.15.150:6789/0 conn(0x7f5f4c0b5270 legacy=0x7f5f4c0ca640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:36.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.806+0000 7f5f43fff640 1 -- 172.21.15.49:0/1920917253 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f4c0aed30 con 0x7f5f4c0c9c80 2023-11-06T02:13:36.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.806+0000 7f5f43fff640 1 -- 172.21.15.49:0/1920917253 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7f5f3c001680 con 0x7f5f4c0c9c80 2023-11-06T02:13:36.807 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.806+0000 7f5f51168640 1 --2- 172.21.15.49:0/1920917253 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5f4c071120 0x7f5f4c071500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:13:36.807 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.806+0000 7f5f43fff640 1 -- 172.21.15.49:0/1920917253 >> v1:172.21.15.49:6789/0 conn(0x7f5f4c0c9c80 legacy=0x7f5f4c0b2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:36.807 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.806+0000 7f5f43fff640 1 --2- 172.21.15.49:0/1920917253 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f380039b0 0x7f5f38003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:36.807 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.806+0000 7f5f43fff640 1 --2- 172.21.15.49:0/1920917253 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5f4c071120 0x7f5f380047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:36.807 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.806+0000 7f5f43fff640 1 --2- 172.21.15.49:0/1920917253 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f38004d10 0x7f5f38005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:36.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.806+0000 7f5f43fff640 1 -- 172.21.15.49:0/1920917253 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4c0ae9c0 con 0x7f5f380039b0 2023-11-06T02:13:36.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.806+0000 7f5f43fff640 1 -- 172.21.15.49:0/1920917253 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f3800bb30 con 0x7f5f38004d10 2023-11-06T02:13:36.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.806+0000 7f5f43fff640 1 -- 172.21.15.49:0/1920917253 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f3800bcf0 con 0x7f5f4c071120 2023-11-06T02:13:36.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.806+0000 7f5f51168640 1 --2- 172.21.15.49:0/1920917253 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f380039b0 0x7f5f38003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:36.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.806+0000 7f5f50967640 1 --2- 172.21.15.49:0/1920917253 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5f4c071120 0x7f5f380047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:36.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.806+0000 7f5f51969640 1 --2- 172.21.15.49:0/1920917253 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f38004d10 0x7f5f38005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:36.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.806+0000 7f5f51168640 1 -- 172.21.15.49:0/1920917253 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5f4c071120 msgr2=0x7f5f380047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:36.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.806+0000 7f5f51168640 1 --2- 172.21.15.49:0/1920917253 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5f4c071120 0x7f5f380047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:36.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.806+0000 7f5f51168640 1 -- 172.21.15.49:0/1920917253 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f38004d10 msgr2=0x7f5f38005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:36.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.806+0000 7f5f51168640 1 --2- 172.21.15.49:0/1920917253 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f38004d10 0x7f5f38005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:36.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.806+0000 7f5f51168640 1 -- 172.21.15.49:0/1920917253 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f38003530 con 0x7f5f380039b0 2023-11-06T02:13:36.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.806+0000 7f5f51969640 1 --2- 172.21.15.49:0/1920917253 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f38004d10 0x7f5f38005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:13:36.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.810+0000 7f5f51168640 1 --2- 172.21.15.49:0/1920917253 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f380039b0 0x7f5f38003dd0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f5f480098a0 tx=0x7f5f4801c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a63032dab418d25 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:36.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.810+0000 7f5f43fff640 1 -- 172.21.15.49:0/1920917253 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f480ab600 con 0x7f5f380039b0 2023-11-06T02:13:36.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.810+0000 7f5f43fff640 1 -- 172.21.15.49:0/1920917253 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f480ab760 con 0x7f5f380039b0 2023-11-06T02:13:36.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.810+0000 7f5f43fff640 1 -- 172.21.15.49:0/1920917253 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f480abad0 con 0x7f5f380039b0 2023-11-06T02:13:36.814 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.810+0000 7f5f533cc640 1 -- 172.21.15.49:0/1920917253 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f380039b0 msgr2=0x7f5f38003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:36.814 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.810+0000 7f5f533cc640 1 --2- 172.21.15.49:0/1920917253 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f380039b0 0x7f5f38003dd0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f5f480098a0 tx=0x7f5f4801c4b0 comp rx=0 tx=0).stop 2023-11-06T02:13:36.815 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.810+0000 7f5f51969640 1 -- 172.21.15.49:0/1920917253 reap_dead start 2023-11-06T02:13:36.815 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.810+0000 7f5f533cc640 1 -- 172.21.15.49:0/1920917253 shutdown_connections 2023-11-06T02:13:36.815 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.810+0000 7f5f533cc640 1 -- 172.21.15.49:0/1920917253 >> 172.21.15.49:0/1920917253 conn(0x7f5f4c0cc140 msgr2=0x7f5f4c0cc550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:36.815 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.810+0000 7f5f533cc640 1 -- 172.21.15.49:0/1920917253 shutdown_connections 2023-11-06T02:13:36.815 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.810+0000 7f5f533cc640 1 -- 172.21.15.49:0/1920917253 wait complete. 2023-11-06T02:13:36.816 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.810+0000 7f5f533cc640 1 Processor -- start 2023-11-06T02:13:36.816 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.810+0000 7f5f533cc640 1 -- start start 2023-11-06T02:13:36.816 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.814+0000 7f5f533cc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f4c0b1b50 0x7f5f4c0c57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:36.816 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.814+0000 7f5f533cc640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5f4c0c5cf0 0x7f5f4c0b3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:36.816 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.814+0000 7f5f533cc640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f4c0b3eb0 0x7f5f4c1e12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:36.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.814+0000 7f5f533cc640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4c0af010 con 0x7f5f4c0b1b50 2023-11-06T02:13:36.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.814+0000 7f5f533cc640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4c0b1f30 con 0x7f5f4c0b3eb0 2023-11-06T02:13:36.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.814+0000 7f5f533cc640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4c0b4330 con 0x7f5f4c0c5cf0 2023-11-06T02:13:36.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.814+0000 7f5f51168640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f4c0b1b50 0x7f5f4c0c57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:36.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.814+0000 7f5f51168640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f4c0b1b50 0x7f5f4c0c57b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51284/0 (socket says 172.21.15.49:51284) 2023-11-06T02:13:36.818 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.814+0000 7f5f51168640 1 -- 172.21.15.49:0/755597531 learned_addr learned my addr 172.21.15.49:0/755597531 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:36.818 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.814+0000 7f5f51969640 1 --2- 172.21.15.49:0/755597531 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f4c0b3eb0 0x7f5f4c1e12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:36.818 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.814+0000 7f5f50967640 1 --2- 172.21.15.49:0/755597531 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5f4c0c5cf0 0x7f5f4c0b3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:36.818 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.814+0000 7f5f51168640 1 -- 172.21.15.49:0/755597531 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5f4c0c5cf0 msgr2=0x7f5f4c0b3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:36.819 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.814+0000 7f5f51168640 1 --2- 172.21.15.49:0/755597531 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5f4c0c5cf0 0x7f5f4c0b3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:36.819 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.814+0000 7f5f51168640 1 -- 172.21.15.49:0/755597531 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f4c0b3eb0 msgr2=0x7f5f4c1e12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:36.819 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.814+0000 7f5f51168640 1 --2- 172.21.15.49:0/755597531 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f4c0b3eb0 0x7f5f4c1e12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:36.819 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.814+0000 7f5f51168640 1 -- 172.21.15.49:0/755597531 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f4c1e35d0 con 0x7f5f4c0b1b50 2023-11-06T02:13:36.819 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.814+0000 7f5f51969640 1 --2- 172.21.15.49:0/755597531 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f4c0b3eb0 0x7f5f4c1e12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:13:36.820 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.814+0000 7f5f50967640 1 --2- 172.21.15.49:0/755597531 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5f4c0c5cf0 0x7f5f4c0b3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:13:36.820 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.814+0000 7f5f51168640 1 --2- 172.21.15.49:0/755597531 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f4c0b1b50 0x7f5f4c0c57b0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f5f4801cd50 tx=0x7f5f38005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:36.820 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.814+0000 7f5f41ffb640 1 -- 172.21.15.49:0/755597531 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f480abae0 con 0x7f5f4c0b1b50 2023-11-06T02:13:36.820 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.814+0000 7f5f41ffb640 1 -- 172.21.15.49:0/755597531 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f480abc40 con 0x7f5f4c0b1b50 2023-11-06T02:13:36.820 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.814+0000 7f5f533cc640 1 -- 172.21.15.49:0/755597531 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f4c1e2a00 con 0x7f5f4c0b1b50 2023-11-06T02:13:36.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.814+0000 7f5f41ffb640 1 -- 172.21.15.49:0/755597531 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f480bd410 con 0x7f5f4c0b1b50 2023-11-06T02:13:36.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.814+0000 7f5f533cc640 1 -- 172.21.15.49:0/755597531 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f4c1e2f80 con 0x7f5f4c0b1b50 2023-11-06T02:13:36.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.814+0000 7f5f533cc640 1 -- 172.21.15.49:0/755597531 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f4c067140 con 0x7f5f4c0b1b50 2023-11-06T02:13:36.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.814+0000 7f5f41ffb640 1 -- 172.21.15.49:0/755597531 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f5f480b43f0 con 0x7f5f4c0b1b50 2023-11-06T02:13:36.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.814+0000 7f5f41ffb640 1 --2- 172.21.15.49:0/755597531 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5f30065a60 0x7f5f30067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:36.822 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.814+0000 7f5f41ffb640 1 -- 172.21.15.49:0/755597531 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5f4811ced0 con 0x7f5f4c0b1b50 2023-11-06T02:13:36.822 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.814+0000 7f5f50967640 1 --2- 172.21.15.49:0/755597531 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5f30065a60 0x7f5f30067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:36.822 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.818+0000 7f5f50967640 1 --2- 172.21.15.49:0/755597531 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5f30065a60 0x7f5f30067f20 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c001ca0 tx=0x7f5f3c001b30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:36.823 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:36.818+0000 7f5f41ffb640 1 -- 172.21.15.49:0/755597531 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f5f480ea8e0 con 0x7f5f4c0b1b50 2023-11-06T02:13:37.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.010+0000 7f5f533cc640 1 -- 172.21.15.49:0/755597531 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5f4c15a7c0 con 0x7f5f4c0b1b50 2023-11-06T02:13:37.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.014+0000 7f5f41ffb640 1 -- 172.21.15.49:0/755597531 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+4650 (secure 0 0 0) 0x7f5f4800f040 con 0x7f5f4c0b1b50 2023-11-06T02:13:37.008 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:13:37.009 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":46,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5714,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/315581318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":315581318},{"type":"v1","addr":"172.21.15.150:6837","nonce":315581318}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41},{"gid":6121,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/136800245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":136800245},{"type":"v1","addr":"172.21.15.150:6835","nonce":136800245}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41},{"gid":7110,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/4259579254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":4259579254},{"type":"v1","addr":"172.21.15.49:6833","nonce":4259579254}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T02:13:33.625701+0000","modified":"2023-11-06T02:13:36.649877+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7116},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7116":{"gid":7116,"name":"c","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.49:6835/1130379293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":1130379293},{"type":"v1","addr":"172.21.15.49:6835","nonce":1130379293}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-06T02:13:37.009 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 46 2023-11-06T02:13:37.010 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.014+0000 7f5f533cc640 1 -- 172.21.15.49:0/755597531 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5f30065a60 msgr2=0x7f5f30067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:37.010 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.014+0000 7f5f533cc640 1 --2- 172.21.15.49:0/755597531 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5f30065a60 0x7f5f30067f20 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c001ca0 tx=0x7f5f3c001b30 comp rx=0 tx=0).stop 2023-11-06T02:13:37.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.014+0000 7f5f533cc640 1 -- 172.21.15.49:0/755597531 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f4c0b1b50 msgr2=0x7f5f4c0c57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:37.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.014+0000 7f5f533cc640 1 --2- 172.21.15.49:0/755597531 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f4c0b1b50 0x7f5f4c0c57b0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f5f4801cd50 tx=0x7f5f38005800 comp rx=0 tx=0).stop 2023-11-06T02:13:37.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.014+0000 7f5f533cc640 1 -- 172.21.15.49:0/755597531 shutdown_connections 2023-11-06T02:13:37.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.014+0000 7f5f533cc640 1 --2- 172.21.15.49:0/755597531 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5f30065a60 0x7f5f30067f20 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:37.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.014+0000 7f5f533cc640 1 --2- 172.21.15.49:0/755597531 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f4c0b3eb0 0x7f5f4c1e12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:37.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.014+0000 7f5f533cc640 1 --2- 172.21.15.49:0/755597531 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5f4c0c5cf0 0x7f5f4c0b3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:37.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.014+0000 7f5f533cc640 1 --2- 172.21.15.49:0/755597531 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f4c0b1b50 0x7f5f4c0c57b0 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:37.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.014+0000 7f5f533cc640 1 -- 172.21.15.49:0/755597531 >> 172.21.15.49:0/755597531 conn(0x7f5f4c0cc140 msgr2=0x7f5f4c15a0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:37.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.014+0000 7f5f533cc640 1 -- 172.21.15.49:0/755597531 shutdown_connections 2023-11-06T02:13:37.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.014+0000 7f5f533cc640 1 -- 172.21.15.49:0/755597531 wait complete. 2023-11-06T02:13:37.028 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T02:13:37.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.150+0000 7fa5ceb0f640 1 Processor -- start 2023-11-06T02:13:37.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.150+0000 7fa5ceb0f640 1 -- start start 2023-11-06T02:13:37.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.150+0000 7fa5ceb0f640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa5c80b0790 0x7fa5c80c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:37.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.150+0000 7fa5ceb0f640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fa5c80cb010 0x7fa5c80c2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:37.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.150+0000 7fa5ceb0f640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5c80cab80 con 0x7fa5c815ab50 2023-11-06T02:13:37.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.150+0000 7fa5ceb0f640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5c815af30 con 0x7fa5c80cb010 2023-11-06T02:13:37.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.150+0000 7fa5ceb0f640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c80cb5b0 con 0x7fa5c80b0790 2023-11-06T02:13:37.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.150+0000 7fa5cc8ab640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fa5c815ab50 0x7fa5c80b0080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:60610/0 (socket says 172.21.15.49:60610) 2023-11-06T02:13:37.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.150+0000 7fa5cc8ab640 1 -- 172.21.15.49:0/2985182366 learned_addr learned my addr 172.21.15.49:0/2985182366 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:13:37.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.150+0000 7fa5cd0ac640 1 --2- 172.21.15.49:0/2985182366 >> v2:172.21.15.150:3300/0 conn(0x7fa5c80cb010 0x7fa5c80c2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:37.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.150+0000 7fa5c7fff640 1 --2- 172.21.15.49:0/2985182366 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa5c80b0790 0x7fa5c80c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:37.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.150+0000 7fa5c77fe640 1 -- 172.21.15.49:0/2985182366 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 862597018 0 0) 0x7fa5c80cab80 con 0x7fa5c815ab50 2023-11-06T02:13:37.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.150+0000 7fa5c77fe640 1 -- 172.21.15.49:0/2985182366 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5b0003530 con 0x7fa5c815ab50 2023-11-06T02:13:37.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.150+0000 7fa5c77fe640 1 -- 172.21.15.49:0/2985182366 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2580705797 0 0) 0x7fa5b0003530 con 0x7fa5c815ab50 2023-11-06T02:13:37.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.150+0000 7fa5c77fe640 1 -- 172.21.15.49:0/2985182366 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa5c80b0790 msgr2=0x7fa5c80c9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:37.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.150+0000 7fa5c77fe640 1 --2- 172.21.15.49:0/2985182366 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa5c80b0790 0x7fa5c80c9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:37.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.150+0000 7fa5c77fe640 1 -- 172.21.15.49:0/2985182366 >> v2:172.21.15.150:3300/0 conn(0x7fa5c80cb010 msgr2=0x7fa5c80c2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:37.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.150+0000 7fa5c77fe640 1 --2- 172.21.15.49:0/2985182366 >> v2:172.21.15.150:3300/0 conn(0x7fa5c80cb010 0x7fa5c80c2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:37.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.150+0000 7fa5c77fe640 1 -- 172.21.15.49:0/2985182366 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5c80cb890 con 0x7fa5c815ab50 2023-11-06T02:13:37.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.150+0000 7fa5c77fe640 1 -- 172.21.15.49:0/2985182366 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7fa5bc001800 con 0x7fa5c815ab50 2023-11-06T02:13:37.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.150+0000 7fa5c77fe640 1 -- 172.21.15.49:0/2985182366 >> v1:172.21.15.49:6789/0 conn(0x7fa5c815ab50 legacy=0x7fa5c80b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:37.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.150+0000 7fa5c77fe640 1 --2- 172.21.15.49:0/2985182366 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5b0003980 0x7fa5b0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:37.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.150+0000 7fa5c77fe640 1 --2- 172.21.15.49:0/2985182366 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa5c80b0790 0x7fa5b00047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:37.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.150+0000 7fa5c77fe640 1 --2- 172.21.15.49:0/2985182366 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa5b0004d10 0x7fa5b0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:37.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.150+0000 7fa5c77fe640 1 -- 172.21.15.49:0/2985182366 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c815af30 con 0x7fa5b0003980 2023-11-06T02:13:37.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.150+0000 7fa5c77fe640 1 -- 172.21.15.49:0/2985182366 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c80cb5b0 con 0x7fa5b0004d10 2023-11-06T02:13:37.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.150+0000 7fa5c77fe640 1 -- 172.21.15.49:0/2985182366 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b000b990 con 0x7fa5c80b0790 2023-11-06T02:13:37.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.150+0000 7fa5cc8ab640 1 --2- 172.21.15.49:0/2985182366 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5b0003980 0x7fa5b0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:37.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.150+0000 7fa5c7fff640 1 --2- 172.21.15.49:0/2985182366 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa5c80b0790 0x7fa5b00047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:37.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.150+0000 7fa5cd0ac640 1 --2- 172.21.15.49:0/2985182366 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa5b0004d10 0x7fa5b0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:37.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.150+0000 7fa5cc8ab640 1 -- 172.21.15.49:0/2985182366 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa5c80b0790 msgr2=0x7fa5b00047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:37.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.150+0000 7fa5cc8ab640 1 --2- 172.21.15.49:0/2985182366 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa5c80b0790 0x7fa5b00047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:37.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.150+0000 7fa5cc8ab640 1 -- 172.21.15.49:0/2985182366 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa5b0004d10 msgr2=0x7fa5b0005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:37.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.150+0000 7fa5cc8ab640 1 --2- 172.21.15.49:0/2985182366 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa5b0004d10 0x7fa5b0005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:37.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.150+0000 7fa5cc8ab640 1 -- 172.21.15.49:0/2985182366 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5b0003530 con 0x7fa5b0003980 2023-11-06T02:13:37.156 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.150+0000 7fa5cd0ac640 1 --2- 172.21.15.49:0/2985182366 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa5b0004d10 0x7fa5b0005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:13:37.159 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.154+0000 7fa5cc8ab640 1 --2- 172.21.15.49:0/2985182366 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5b0003980 0x7fa5b0003da0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc00c210 tx=0x7fa5bc01cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a216807fde4516ce server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:37.159 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.154+0000 7fa5c77fe640 1 -- 172.21.15.49:0/2985182366 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5bc0a5910 con 0x7fa5b0003980 2023-11-06T02:13:37.159 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.154+0000 7fa5c77fe640 1 -- 172.21.15.49:0/2985182366 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5bc0a5a70 con 0x7fa5b0003980 2023-11-06T02:13:37.160 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.154+0000 7fa5c77fe640 1 -- 172.21.15.49:0/2985182366 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5bc0a5e20 con 0x7fa5b0003980 2023-11-06T02:13:37.160 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.154+0000 7fa5ceb0f640 1 -- 172.21.15.49:0/2985182366 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5b0003980 msgr2=0x7fa5b0003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:37.160 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.154+0000 7fa5ceb0f640 1 --2- 172.21.15.49:0/2985182366 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5b0003980 0x7fa5b0003da0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc00c210 tx=0x7fa5bc01cc30 comp rx=0 tx=0).stop 2023-11-06T02:13:37.160 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.154+0000 7fa5cd0ac640 1 -- 172.21.15.49:0/2985182366 reap_dead start 2023-11-06T02:13:37.160 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.154+0000 7fa5ceb0f640 1 -- 172.21.15.49:0/2985182366 shutdown_connections 2023-11-06T02:13:37.161 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.154+0000 7fa5ceb0f640 1 -- 172.21.15.49:0/2985182366 >> 172.21.15.49:0/2985182366 conn(0x7fa5c806c0a0 msgr2=0x7fa5c806c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:37.161 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.154+0000 7fa5ceb0f640 1 -- 172.21.15.49:0/2985182366 shutdown_connections 2023-11-06T02:13:37.161 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.154+0000 7fa5ceb0f640 1 -- 172.21.15.49:0/2985182366 wait complete. 2023-11-06T02:13:37.161 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.158+0000 7fa5ceb0f640 1 Processor -- start 2023-11-06T02:13:37.161 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.158+0000 7fa5ceb0f640 1 -- start start 2023-11-06T02:13:37.162 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.158+0000 7fa5ceb0f640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa5b0003980 0x7fa5c8154af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:37.162 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.158+0000 7fa5ceb0f640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa5c80c68b0 0x7fa5c80c6d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:37.162 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.158+0000 7fa5ceb0f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5c8155fd0 0x7fa5c8156480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:37.162 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.158+0000 7fa5ceb0f640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c81569c0 con 0x7fa5c8155fd0 2023-11-06T02:13:37.162 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.158+0000 7fa5ceb0f640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c80c84a0 con 0x7fa5c80c68b0 2023-11-06T02:13:37.163 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.158+0000 7fa5ceb0f640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c8155030 con 0x7fa5b0003980 2023-11-06T02:13:37.163 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.158+0000 7fa5cd0ac640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5c8155fd0 0x7fa5c8156480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:37.163 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.158+0000 7fa5cd0ac640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5c8155fd0 0x7fa5c8156480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51306/0 (socket says 172.21.15.49:51306) 2023-11-06T02:13:37.163 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.158+0000 7fa5cd0ac640 1 -- 172.21.15.49:0/2588889541 learned_addr learned my addr 172.21.15.49:0/2588889541 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:37.163 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.158+0000 7fa5cd0ac640 1 -- 172.21.15.49:0/2588889541 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa5b0003980 msgr2=0x7fa5c8154af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:37.163 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.158+0000 7fa5c7fff640 1 --2- 172.21.15.49:0/2588889541 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa5c80c68b0 0x7fa5c80c6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:37.164 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.158+0000 7fa5cd0ac640 1 --2- 172.21.15.49:0/2588889541 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa5b0003980 0x7fa5c8154af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:37.164 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.158+0000 7fa5cd0ac640 1 -- 172.21.15.49:0/2588889541 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa5c80c68b0 msgr2=0x7fa5c80c6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:37.164 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.158+0000 7fa5cd0ac640 1 --2- 172.21.15.49:0/2588889541 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa5c80c68b0 0x7fa5c80c6d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:37.164 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.158+0000 7fa5cd0ac640 1 -- 172.21.15.49:0/2588889541 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5c80c7400 con 0x7fa5c8155fd0 2023-11-06T02:13:37.165 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.158+0000 7fa5cd0ac640 1 --2- 172.21.15.49:0/2588889541 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5c8155fd0 0x7fa5c8156480 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fa5c80cae30 tx=0x7fa5c000f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:37.165 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.158+0000 7fa5c57fa640 1 -- 172.21.15.49:0/2588889541 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c0016620 con 0x7fa5c8155fd0 2023-11-06T02:13:37.165 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.158+0000 7fa5c57fa640 1 -- 172.21.15.49:0/2588889541 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5c0016780 con 0x7fa5c8155fd0 2023-11-06T02:13:37.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.158+0000 7fa5ceb0f640 1 -- 172.21.15.49:0/2588889541 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5c80c76e0 con 0x7fa5c8155fd0 2023-11-06T02:13:37.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.158+0000 7fa5c57fa640 1 -- 172.21.15.49:0/2588889541 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c0016b10 con 0x7fa5c8155fd0 2023-11-06T02:13:37.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.158+0000 7fa5ceb0f640 1 -- 172.21.15.49:0/2588889541 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5c80c7ca0 con 0x7fa5c8155fd0 2023-11-06T02:13:37.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.158+0000 7fa5ceb0f640 1 -- 172.21.15.49:0/2588889541 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5c80b0de0 con 0x7fa5c8155fd0 2023-11-06T02:13:37.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.158+0000 7fa5c57fa640 1 -- 172.21.15.49:0/2588889541 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fa5c0016c70 con 0x7fa5c8155fd0 2023-11-06T02:13:37.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.158+0000 7fa5c57fa640 1 --2- 172.21.15.49:0/2588889541 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa5b4065a60 0x7fa5b4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:37.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.162+0000 7fa5c57fa640 1 -- 172.21.15.49:0/2588889541 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa5c0083fb0 con 0x7fa5c8155fd0 2023-11-06T02:13:37.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.162+0000 7fa5cc8ab640 1 --2- 172.21.15.49:0/2588889541 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa5b4065a60 0x7fa5b4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:37.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.162+0000 7fa5cc8ab640 1 --2- 172.21.15.49:0/2588889541 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa5b4065a60 0x7fa5b4067f20 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc0aa000 tx=0x7fa5bc001170 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:37.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.166+0000 7fa5c57fa640 1 -- 172.21.15.49:0/2588889541 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fa5c0051810 con 0x7fa5c8155fd0 2023-11-06T02:13:37.325 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.330+0000 7fa5ceb0f640 1 -- 172.21.15.49:0/2588889541 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa5c80736c0 con 0x7fa5c8155fd0 2023-11-06T02:13:37.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.334+0000 7fa5c57fa640 1 -- 172.21.15.49:0/2588889541 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+17886 (secure 0 0 0) 0x7fa5c0054e30 con 0x7fa5c8155fd0 2023-11-06T02:13:37.328 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:13:37.329 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":80,"fsid":"1a8b035c-6ba4-499a-b876-aea47fd9c0d4","created":"2023-11-06T02:08:06.440857+0000","modified":"2023-11-06T02:13:34.630031+0000","last_up_change":"2023-11-06T02:08:17.982372+0000","last_in_change":"2023-11-06T02:08:08.906631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T02:08:19.252222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T02:08:24.776211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-11-06T02:13:31.459314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-11-06T02:13:32.603710+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ad55af8-0571-4e6a-8722-b5d429b7496d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6809","nonce":38116},{"type":"v1","addr":"172.21.15.49:6811","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6813","nonce":38116},{"type":"v1","addr":"172.21.15.49:6815","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6821","nonce":38116},{"type":"v1","addr":"172.21.15.49:6823","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6817","nonce":38116},{"type":"v1","addr":"172.21.15.49:6819","nonce":38116}]},"public_addr":"172.21.15.49:6811/38116","cluster_addr":"172.21.15.49:6815/38116","heartbeat_back_addr":"172.21.15.49:6823/38116","heartbeat_front_addr":"172.21.15.49:6819/38116","state":["exists","up"]},{"osd":1,"uuid":"c8b31f85-a757-43ed-9419-351acc46611b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":38113},{"type":"v1","addr":"172.21.15.49:6810","nonce":38113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":38113},{"type":"v1","addr":"172.21.15.49:6814","nonce":38113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":38113},{"type":"v1","addr":"172.21.15.49:6822","nonce":38113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":38113},{"type":"v1","addr":"172.21.15.49:6818","nonce":38113}]},"public_addr":"172.21.15.49:6810/38113","cluster_addr":"172.21.15.49:6814/38113","heartbeat_back_addr":"172.21.15.49:6822/38113","heartbeat_front_addr":"172.21.15.49:6818/38113","state":["exists","up"]},{"osd":2,"uuid":"555b3bf3-0a70-4b91-8efd-e4204b5fc26e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":38114},{"type":"v1","addr":"172.21.15.49:6801","nonce":38114}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":38114},{"type":"v1","addr":"172.21.15.49:6803","nonce":38114}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":38114},{"type":"v1","addr":"172.21.15.49:6807","nonce":38114}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":38114},{"type":"v1","addr":"172.21.15.49:6805","nonce":38114}]},"public_addr":"172.21.15.49:6801/38114","cluster_addr":"172.21.15.49:6803/38114","heartbeat_back_addr":"172.21.15.49:6807/38114","heartbeat_front_addr":"172.21.15.49:6805/38114","state":["exists","up"]},{"osd":3,"uuid":"8d4e5d77-03ef-4b9b-bfcb-5ae93a46795c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":38115},{"type":"v1","addr":"172.21.15.49:6825","nonce":38115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":38115},{"type":"v1","addr":"172.21.15.49:6827","nonce":38115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":38115},{"type":"v1","addr":"172.21.15.49:6831","nonce":38115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":38115},{"type":"v1","addr":"172.21.15.49:6829","nonce":38115}]},"public_addr":"172.21.15.49:6825/38115","cluster_addr":"172.21.15.49:6827/38115","heartbeat_back_addr":"172.21.15.49:6831/38115","heartbeat_front_addr":"172.21.15.49:6829/38115","state":["exists","up"]},{"osd":4,"uuid":"dba03ad4-68a3-4cf3-9218-ac611f02eacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":38417},{"type":"v1","addr":"172.21.15.150:6809","nonce":38417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":38417},{"type":"v1","addr":"172.21.15.150:6811","nonce":38417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":38417},{"type":"v1","addr":"172.21.15.150:6815","nonce":38417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":38417},{"type":"v1","addr":"172.21.15.150:6813","nonce":38417}]},"public_addr":"172.21.15.150:6809/38417","cluster_addr":"172.21.15.150:6811/38417","heartbeat_back_addr":"172.21.15.150:6815/38417","heartbeat_front_addr":"172.21.15.150:6813/38417","state":["exists","up"]},{"osd":5,"uuid":"0df9c303-c255-4640-8e91-01f1c3efe18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":38418},{"type":"v1","addr":"172.21.15.150:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":38418},{"type":"v1","addr":"172.21.15.150:6803","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":38418},{"type":"v1","addr":"172.21.15.150:6807","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":38418},{"type":"v1","addr":"172.21.15.150:6805","nonce":38418}]},"public_addr":"172.21.15.150:6801/38418","cluster_addr":"172.21.15.150:6803/38418","heartbeat_back_addr":"172.21.15.150:6807/38418","heartbeat_front_addr":"172.21.15.150:6805/38418","state":["exists","up"]},{"osd":6,"uuid":"e59fa189-4b70-4c06-9722-c02b90828bf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":38428},{"type":"v1","addr":"172.21.15.150:6825","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":38428},{"type":"v1","addr":"172.21.15.150:6827","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":38428},{"type":"v1","addr":"172.21.15.150:6831","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":38428},{"type":"v1","addr":"172.21.15.150:6829","nonce":38428}]},"public_addr":"172.21.15.150:6825/38428","cluster_addr":"172.21.15.150:6827/38428","heartbeat_back_addr":"172.21.15.150:6831/38428","heartbeat_front_addr":"172.21.15.150:6829/38428","state":["exists","up"]},{"osd":7,"uuid":"a64f57e1-ef98-46bd-b304-385759e95898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":38421},{"type":"v1","addr":"172.21.15.150:6817","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":38421},{"type":"v1","addr":"172.21.15.150:6819","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":38421},{"type":"v1","addr":"172.21.15.150:6823","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":38421},{"type":"v1","addr":"172.21.15.150:6821","nonce":38421}]},"public_addr":"172.21.15.150:6817/38421","cluster_addr":"172.21.15.150:6819/38421","heartbeat_back_addr":"172.21.15.150:6823/38421","heartbeat_front_addr":"172.21.15.150:6821/38421","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.438552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.413280+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.296466+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.424966+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.276953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.230984+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.410235+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.480084+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T02:13:37.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.334+0000 7fa5ceb0f640 1 -- 172.21.15.49:0/2588889541 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa5b4065a60 msgr2=0x7fa5b4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:37.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.334+0000 7fa5ceb0f640 1 --2- 172.21.15.49:0/2588889541 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa5b4065a60 0x7fa5b4067f20 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc0aa000 tx=0x7fa5bc001170 comp rx=0 tx=0).stop 2023-11-06T02:13:37.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.334+0000 7fa5ceb0f640 1 -- 172.21.15.49:0/2588889541 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5c8155fd0 msgr2=0x7fa5c8156480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:37.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.334+0000 7fa5ceb0f640 1 --2- 172.21.15.49:0/2588889541 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5c8155fd0 0x7fa5c8156480 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fa5c80cae30 tx=0x7fa5c000f520 comp rx=0 tx=0).stop 2023-11-06T02:13:37.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.334+0000 7fa5ceb0f640 1 -- 172.21.15.49:0/2588889541 shutdown_connections 2023-11-06T02:13:37.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.334+0000 7fa5ceb0f640 1 --2- 172.21.15.49:0/2588889541 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa5b4065a60 0x7fa5b4067f20 secure :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc0aa000 tx=0x7fa5bc001170 comp rx=0 tx=0).stop 2023-11-06T02:13:37.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.334+0000 7fa5ceb0f640 1 --2- 172.21.15.49:0/2588889541 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5c8155fd0 0x7fa5c8156480 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:37.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.334+0000 7fa5ceb0f640 1 --2- 172.21.15.49:0/2588889541 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa5c80c68b0 0x7fa5c80c6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:37.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.334+0000 7fa5ceb0f640 1 --2- 172.21.15.49:0/2588889541 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa5b0003980 0x7fa5c8154af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:37.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.334+0000 7fa5ceb0f640 1 -- 172.21.15.49:0/2588889541 >> 172.21.15.49:0/2588889541 conn(0x7fa5c806c0a0 msgr2=0x7fa5c8158610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:37.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.334+0000 7fa5ceb0f640 1 -- 172.21.15.49:0/2588889541 shutdown_connections 2023-11-06T02:13:37.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.334+0000 7fa5ceb0f640 1 -- 172.21.15.49:0/2588889541 wait complete. 2023-11-06T02:13:37.348 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-06T02:13:37.349 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T02:13:37.469 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.474+0000 7f091f5e0640 1 Processor -- start 2023-11-06T02:13:37.469 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.474+0000 7f091f5e0640 1 -- start start 2023-11-06T02:13:37.471 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.474+0000 7f091f5e0640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f091815ab50 0x7f09180c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:37.471 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.474+0000 7f091f5e0640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f09180b0790 0x7f09180c2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:37.471 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.474+0000 7f091f5e0640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09180cab80 con 0x7f09180cb010 2023-11-06T02:13:37.471 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.474+0000 7f091f5e0640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f091815af30 con 0x7f09180b0790 2023-11-06T02:13:37.472 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.474+0000 7f091f5e0640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09180cb5b0 con 0x7f091815ab50 2023-11-06T02:13:37.472 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.474+0000 7f091db7d640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f09180b0790 0x7f09180c2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:37.472 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.474+0000 7f091db7d640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f09180b0790 0x7f09180c2710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51312/0 (socket says 172.21.15.49:51312) 2023-11-06T02:13:37.472 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.474+0000 7f091db7d640 1 -- 172.21.15.49:0/3180192671 learned_addr learned my addr 172.21.15.49:0/3180192671 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:37.472 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.474+0000 7f091d37c640 1 --2- 172.21.15.49:0/3180192671 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f091815ab50 0x7f09180c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:37.473 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.474+0000 7f091db7d640 1 -- 172.21.15.49:0/3180192671 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f091815ab50 msgr2=0x7f09180c9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:37.473 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.474+0000 7f091db7d640 1 --2- 172.21.15.49:0/3180192671 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f091815ab50 0x7f09180c9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:37.473 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.474+0000 7f091db7d640 1 -- 172.21.15.49:0/3180192671 >> v1:172.21.15.150:6789/0 conn(0x7f09180cb010 legacy=0x7f09180b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:37.473 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.474+0000 7f091db7d640 1 -- 172.21.15.49:0/3180192671 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09180cb890 con 0x7f09180b0790 2023-11-06T02:13:37.474 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.478+0000 7f091db7d640 1 --2- 172.21.15.49:0/3180192671 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09180b0790 0x7f09180c2710 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f0910009e50 tx=0x7f091001c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=837ee97ef42dce23 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:37.474 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.478+0000 7f090ffff640 1 -- 172.21.15.49:0/3180192671 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09100a58d0 con 0x7f09180b0790 2023-11-06T02:13:37.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.478+0000 7f090ffff640 1 -- 172.21.15.49:0/3180192671 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09100a5a30 con 0x7f09180b0790 2023-11-06T02:13:37.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.478+0000 7f090ffff640 1 -- 172.21.15.49:0/3180192671 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09100a5dc0 con 0x7f09180b0790 2023-11-06T02:13:37.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.478+0000 7f091f5e0640 1 -- 172.21.15.49:0/3180192671 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09180b0790 msgr2=0x7f09180c2710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:37.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.478+0000 7f091f5e0640 1 --2- 172.21.15.49:0/3180192671 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09180b0790 0x7f09180c2710 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f0910009e50 tx=0x7f091001c9e0 comp rx=0 tx=0).stop 2023-11-06T02:13:37.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.478+0000 7f091f5e0640 1 -- 172.21.15.49:0/3180192671 shutdown_connections 2023-11-06T02:13:37.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.478+0000 7f091f5e0640 1 --2- 172.21.15.49:0/3180192671 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09180b0790 0x7f09180c2710 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:37.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.478+0000 7f091f5e0640 1 --2- 172.21.15.49:0/3180192671 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f091815ab50 0x7f09180c9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:37.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.478+0000 7f091f5e0640 1 -- 172.21.15.49:0/3180192671 >> 172.21.15.49:0/3180192671 conn(0x7f091806c0a0 msgr2=0x7f091806c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:37.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.478+0000 7f091f5e0640 1 -- 172.21.15.49:0/3180192671 shutdown_connections 2023-11-06T02:13:37.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.478+0000 7f091f5e0640 1 -- 172.21.15.49:0/3180192671 wait complete. 2023-11-06T02:13:37.479 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.478+0000 7f091f5e0640 1 Processor -- start 2023-11-06T02:13:37.480 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.478+0000 7f091f5e0640 1 -- start start 2023-11-06T02:13:37.480 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.478+0000 7f091f5e0640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09180b0790 0x7f09180cdad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:37.480 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.478+0000 7f091f5e0640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f09180cb010 0x7f09181593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:37.480 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.478+0000 7f091f5e0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f091815ab50 0x7f09180ae060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:37.480 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.478+0000 7f091f5e0640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09180c7f80 con 0x7f091815ab50 2023-11-06T02:13:37.480 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.478+0000 7f091f5e0640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09180ac100 con 0x7f09180b0790 2023-11-06T02:13:37.481 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.478+0000 7f091f5e0640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09180ac3e0 con 0x7f09180cb010 2023-11-06T02:13:37.481 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.482+0000 7f091db7d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f091815ab50 0x7f09180ae060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:37.481 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.482+0000 7f091db7d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f091815ab50 0x7f09180ae060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51314/0 (socket says 172.21.15.49:51314) 2023-11-06T02:13:37.481 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.482+0000 7f091db7d640 1 -- 172.21.15.49:0/2699784416 learned_addr learned my addr 172.21.15.49:0/2699784416 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:37.481 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.482+0000 7f091d37c640 1 --2- 172.21.15.49:0/2699784416 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09180b0790 0x7f09180cdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:37.482 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.482+0000 7f091cb7b640 1 --2- 172.21.15.49:0/2699784416 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f09180cb010 0x7f09181593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:37.482 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.482+0000 7f091db7d640 1 -- 172.21.15.49:0/2699784416 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f09180cb010 msgr2=0x7f09181593e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:37.482 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.482+0000 7f091db7d640 1 --2- 172.21.15.49:0/2699784416 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f09180cb010 0x7f09181593e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:37.482 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.482+0000 7f091db7d640 1 -- 172.21.15.49:0/2699784416 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09180b0790 msgr2=0x7f09180cdad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:37.482 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.482+0000 7f091db7d640 1 --2- 172.21.15.49:0/2699784416 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09180b0790 0x7f09180cdad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:37.483 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.482+0000 7f091db7d640 1 -- 172.21.15.49:0/2699784416 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0910009b30 con 0x7f091815ab50 2023-11-06T02:13:37.483 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.482+0000 7f091d37c640 1 --2- 172.21.15.49:0/2699784416 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09180b0790 0x7f09180cdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:13:37.483 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.482+0000 7f091db7d640 1 --2- 172.21.15.49:0/2699784416 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f091815ab50 0x7f09180ae060 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f091001a610 tx=0x7f0910019780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:37.483 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.482+0000 7f090dffb640 1 -- 172.21.15.49:0/2699784416 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09100b8670 con 0x7f091815ab50 2023-11-06T02:13:37.483 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.482+0000 7f090dffb640 1 -- 172.21.15.49:0/2699784416 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09100b87d0 con 0x7f091815ab50 2023-11-06T02:13:37.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.482+0000 7f091f5e0640 1 -- 172.21.15.49:0/2699784416 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09180ac6c0 con 0x7f091815ab50 2023-11-06T02:13:37.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.482+0000 7f090dffb640 1 -- 172.21.15.49:0/2699784416 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09100b8b80 con 0x7f091815ab50 2023-11-06T02:13:37.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.482+0000 7f091f5e0640 1 -- 172.21.15.49:0/2699784416 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09180acc80 con 0x7f091815ab50 2023-11-06T02:13:37.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.482+0000 7f091f5e0640 1 -- 172.21.15.49:0/2699784416 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09180b0de0 con 0x7f091815ab50 2023-11-06T02:13:37.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.482+0000 7f090dffb640 1 -- 172.21.15.49:0/2699784416 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f09100b8ce0 con 0x7f091815ab50 2023-11-06T02:13:37.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.482+0000 7f090dffb640 1 --2- 172.21.15.49:0/2699784416 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f08fc065a60 0x7f08fc067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:37.485 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.482+0000 7f090dffb640 1 -- 172.21.15.49:0/2699784416 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f091011e820 con 0x7f091815ab50 2023-11-06T02:13:37.485 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.482+0000 7f091d37c640 1 --2- 172.21.15.49:0/2699784416 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f08fc065a60 0x7f08fc067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:37.485 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.482+0000 7f091d37c640 1 --2- 172.21.15.49:0/2699784416 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f08fc065a60 0x7f08fc067f20 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f0914006fd0 tx=0x7f0914008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:37.485 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.486+0000 7f090dffb640 1 -- 172.21.15.49:0/2699784416 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f09100ec1d0 con 0x7f091815ab50 2023-11-06T02:13:37.635 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.638+0000 7f091f5e0640 1 -- 172.21.15.49:0/2699784416 --> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f09180cbe50 con 0x7f08fc065a60 2023-11-06T02:13:37.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.666+0000 7f090dffb640 1 -- 172.21.15.49:0/2699784416 <== mgr.4100 v2:172.21.15.150:6832/38104 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+306073 (secure 0 0 0) 0x7f0914004030 con 0x7f08fc065a60 2023-11-06T02:13:37.664 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:13:37.665 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2023-11-06T02:13:37.667 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.670+0000 7f091f5e0640 1 -- 172.21.15.49:0/2699784416 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f08fc065a60 msgr2=0x7f08fc067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:37.667 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.670+0000 7f091f5e0640 1 --2- 172.21.15.49:0/2699784416 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f08fc065a60 0x7f08fc067f20 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f0914006fd0 tx=0x7f0914008040 comp rx=0 tx=0).stop 2023-11-06T02:13:37.667 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.670+0000 7f091f5e0640 1 -- 172.21.15.49:0/2699784416 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f091815ab50 msgr2=0x7f09180ae060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:37.668 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.670+0000 7f091f5e0640 1 --2- 172.21.15.49:0/2699784416 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f091815ab50 0x7f09180ae060 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f091001a610 tx=0x7f0910019780 comp rx=0 tx=0).stop 2023-11-06T02:13:37.668 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.670+0000 7f091f5e0640 1 -- 172.21.15.49:0/2699784416 shutdown_connections 2023-11-06T02:13:37.668 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.670+0000 7f091f5e0640 1 --2- 172.21.15.49:0/2699784416 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f08fc065a60 0x7f08fc067f20 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:37.668 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.670+0000 7f091f5e0640 1 --2- 172.21.15.49:0/2699784416 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f091815ab50 0x7f09180ae060 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:37.668 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.670+0000 7f091f5e0640 1 --2- 172.21.15.49:0/2699784416 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f09180cb010 0x7f09181593e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:37.669 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.670+0000 7f091f5e0640 1 --2- 172.21.15.49:0/2699784416 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09180b0790 0x7f09180cdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:37.669 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.670+0000 7f091f5e0640 1 -- 172.21.15.49:0/2699784416 >> 172.21.15.49:0/2699784416 conn(0x7f091806c0a0 msgr2=0x7f0918159ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:37.669 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.670+0000 7f091f5e0640 1 -- 172.21.15.49:0/2699784416 shutdown_connections 2023-11-06T02:13:37.669 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.670+0000 7f091f5e0640 1 -- 172.21.15.49:0/2699784416 wait complete. 2023-11-06T02:13:37.682 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":233,"stamp":"2023-11-06T02:13:37.286493+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":20,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":248676,"kb_used_data":8484,"kb_used_omap":0,"kb_used_meta":240128,"kb_avail":754726044,"statfs":{"total":773094113280,"available":772839469056,"internally_reserved":0,"allocated":8687616,"data_stored":4531565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":245891072},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":44,"apply_latency_ms":44,"commit_latency_ns":44000000,"apply_latency_ns":44000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.060378"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.255018+0000","last_change":"2023-11-06T02:13:33.255018+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.255018+0000","last_undegraded":"2023-11-06T02:13:33.255018+0000","last_fullsized":"2023-11-06T02:13:33.255018+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:18:07.485044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.252648+0000","last_change":"2023-11-06T02:13:33.252648+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.252648+0000","last_undegraded":"2023-11-06T02:13:33.252648+0000","last_fullsized":"2023-11-06T02:13:33.252648+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:22:01.334447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.257065+0000","last_change":"2023-11-06T02:13:33.257065+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.257065+0000","last_undegraded":"2023-11-06T02:13:33.257065+0000","last_fullsized":"2023-11-06T02:13:33.257065+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:42:42.388106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.256535+0000","last_change":"2023-11-06T02:13:33.256535+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.256535+0000","last_undegraded":"2023-11-06T02:13:33.256535+0000","last_fullsized":"2023-11-06T02:13:33.256535+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:54:33.444383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.246148+0000","last_change":"2023-11-06T02:13:33.246148+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.246148+0000","last_undegraded":"2023-11-06T02:13:33.246148+0000","last_fullsized":"2023-11-06T02:13:33.246148+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:01:06.093323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.258259+0000","last_change":"2023-11-06T02:13:33.258259+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.258259+0000","last_undegraded":"2023-11-06T02:13:33.258259+0000","last_fullsized":"2023-11-06T02:13:33.258259+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:41:55.941458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.239742+0000","last_change":"2023-11-06T02:13:33.239742+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.239742+0000","last_undegraded":"2023-11-06T02:13:33.239742+0000","last_fullsized":"2023-11-06T02:13:33.239742+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:40:54.464756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.246060+0000","last_change":"2023-11-06T02:13:33.246060+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.246060+0000","last_undegraded":"2023-11-06T02:13:33.246060+0000","last_fullsized":"2023-11-06T02:13:33.246060+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:39.514240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.243570+0000","last_change":"2023-11-06T02:13:33.243570+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.243570+0000","last_undegraded":"2023-11-06T02:13:33.243570+0000","last_fullsized":"2023-11-06T02:13:33.243570+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:02:51.697659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.257271+0000","last_change":"2023-11-06T02:13:33.257271+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.257271+0000","last_undegraded":"2023-11-06T02:13:33.257271+0000","last_fullsized":"2023-11-06T02:13:33.257271+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:12:00.717133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.252080+0000","last_change":"2023-11-06T02:13:33.252080+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.252080+0000","last_undegraded":"2023-11-06T02:13:33.252080+0000","last_fullsized":"2023-11-06T02:13:33.252080+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:41:33.827814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.255251+0000","last_change":"2023-11-06T02:13:33.255251+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.255251+0000","last_undegraded":"2023-11-06T02:13:33.255251+0000","last_fullsized":"2023-11-06T02:13:33.255251+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:21:24.032595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.245690+0000","last_change":"2023-11-06T02:13:33.245690+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.245690+0000","last_undegraded":"2023-11-06T02:13:33.245690+0000","last_fullsized":"2023-11-06T02:13:33.245690+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:42:30.360623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.257567+0000","last_change":"2023-11-06T02:13:33.257567+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.257567+0000","last_undegraded":"2023-11-06T02:13:33.257567+0000","last_fullsized":"2023-11-06T02:13:33.257567+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:41:33.827814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.252478+0000","last_change":"2023-11-06T02:13:33.252478+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.252478+0000","last_undegraded":"2023-11-06T02:13:33.252478+0000","last_fullsized":"2023-11-06T02:13:33.252478+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:28:22.791353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.254366+0000","last_change":"2023-11-06T02:13:33.254366+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.254366+0000","last_undegraded":"2023-11-06T02:13:33.254366+0000","last_fullsized":"2023-11-06T02:13:33.254366+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:21:24.032595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.246654+0000","last_change":"2023-11-06T02:13:33.246654+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.246654+0000","last_undegraded":"2023-11-06T02:13:33.246654+0000","last_fullsized":"2023-11-06T02:13:33.246654+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:54:43.806737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.255469+0000","last_change":"2023-11-06T02:13:33.255469+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.255469+0000","last_undegraded":"2023-11-06T02:13:33.255469+0000","last_fullsized":"2023-11-06T02:13:33.255469+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:17:36.098143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.254869+0000","last_change":"2023-11-06T02:13:33.254869+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.254869+0000","last_undegraded":"2023-11-06T02:13:33.254869+0000","last_fullsized":"2023-11-06T02:13:33.254869+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:22:13.884367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.247005+0000","last_change":"2023-11-06T02:13:33.247005+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.247005+0000","last_undegraded":"2023-11-06T02:13:33.247005+0000","last_fullsized":"2023-11-06T02:13:33.247005+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:28:11.270422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.242891+0000","last_change":"2023-11-06T02:13:33.242891+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.242891+0000","last_undegraded":"2023-11-06T02:13:33.242891+0000","last_fullsized":"2023-11-06T02:13:33.242891+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:43:49.529761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.246368+0000","last_change":"2023-11-06T02:13:33.246368+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.246368+0000","last_undegraded":"2023-11-06T02:13:33.246368+0000","last_fullsized":"2023-11-06T02:13:33.246368+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:19:02.943437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.244217+0000","last_change":"2023-11-06T02:13:33.244217+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.244217+0000","last_undegraded":"2023-11-06T02:13:33.244217+0000","last_fullsized":"2023-11-06T02:13:33.244217+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:17:17.217507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.252017+0000","last_change":"2023-11-06T02:13:33.252017+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.252017+0000","last_undegraded":"2023-11-06T02:13:33.252017+0000","last_fullsized":"2023-11-06T02:13:33.252017+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:41:55.941458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.250038+0000","last_change":"2023-11-06T02:13:33.250038+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.250038+0000","last_undegraded":"2023-11-06T02:13:33.250038+0000","last_fullsized":"2023-11-06T02:13:33.250038+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:22:13.884367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.255314+0000","last_change":"2023-11-06T02:13:33.255314+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.255314+0000","last_undegraded":"2023-11-06T02:13:33.255314+0000","last_fullsized":"2023-11-06T02:13:33.255314+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:18:07.485044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.254181+0000","last_change":"2023-11-06T02:13:33.254181+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.254181+0000","last_undegraded":"2023-11-06T02:13:33.254181+0000","last_fullsized":"2023-11-06T02:13:33.254181+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:00:13.166796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.243183+0000","last_change":"2023-11-06T02:13:33.243183+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.243183+0000","last_undegraded":"2023-11-06T02:13:33.243183+0000","last_fullsized":"2023-11-06T02:13:33.243183+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:16:46.930331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.249877+0000","last_change":"2023-11-06T02:13:33.249877+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.249877+0000","last_undegraded":"2023-11-06T02:13:33.249877+0000","last_fullsized":"2023-11-06T02:13:33.249877+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:20:11.090360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.250528+0000","last_change":"2023-11-06T02:13:33.250528+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.250528+0000","last_undegraded":"2023-11-06T02:13:33.250528+0000","last_fullsized":"2023-11-06T02:13:33.250528+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:34:15.876381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.253284+0000","last_change":"2023-11-06T02:13:33.253284+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.253284+0000","last_undegraded":"2023-11-06T02:13:33.253284+0000","last_fullsized":"2023-11-06T02:13:33.253284+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:14:22.392069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.252628+0000","last_change":"2023-11-06T02:13:33.252628+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.252628+0000","last_undegraded":"2023-11-06T02:13:33.252628+0000","last_fullsized":"2023-11-06T02:13:33.252628+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:24:31.881719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.252017+0000","last_change":"2023-11-06T02:13:33.252017+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.252017+0000","last_undegraded":"2023-11-06T02:13:33.252017+0000","last_fullsized":"2023-11-06T02:13:33.252017+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:46:42.636025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":3,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.235656+0000","last_change":"2023-11-06T02:13:33.235656+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.235656+0000","last_undegraded":"2023-11-06T02:13:33.235656+0000","last_fullsized":"2023-11-06T02:13:33.235656+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:07:13.179013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.249040+0000","last_change":"2023-11-06T02:13:33.249040+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.249040+0000","last_undegraded":"2023-11-06T02:13:33.249040+0000","last_fullsized":"2023-11-06T02:13:33.249040+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:23:55.083630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":3,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.256359+0000","last_change":"2023-11-06T02:13:33.256359+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.256359+0000","last_undegraded":"2023-11-06T02:13:33.256359+0000","last_fullsized":"2023-11-06T02:13:33.256359+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:33:14.271873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":["7","6"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.246368+0000","last_change":"2023-11-06T02:13:33.246368+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.246368+0000","last_undegraded":"2023-11-06T02:13:33.246368+0000","last_fullsized":"2023-11-06T02:13:33.246368+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:34:15.876381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":4,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.264127+0000","last_change":"2023-11-06T02:13:33.257252+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.264127+0000","last_undegraded":"2023-11-06T02:13:33.264127+0000","last_fullsized":"2023-11-06T02:13:33.264127+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:35:42.092353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":["7","0"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.241683+0000","last_change":"2023-11-06T02:13:33.241683+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.241683+0000","last_undegraded":"2023-11-06T02:13:33.241683+0000","last_fullsized":"2023-11-06T02:13:33.241683+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:14:21.400766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.277611+0000","last_change":"2023-11-06T02:13:33.267790+0000","last_active":"2023-11-06T02:13:33.277611+0000","last_peered":"2023-11-06T02:13:33.277611+0000","last_clean":"2023-11-06T02:13:33.277611+0000","last_became_active":"2023-11-06T02:13:33.267049+0000","last_became_peered":"2023-11-06T02:13:33.267049+0000","last_unstale":"2023-11-06T02:13:33.277611+0000","last_undegraded":"2023-11-06T02:13:33.277611+0000","last_fullsized":"2023-11-06T02:13:33.277611+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:44:56.393504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.249993+0000","last_change":"2023-11-06T02:13:33.249993+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.249993+0000","last_undegraded":"2023-11-06T02:13:33.249993+0000","last_fullsized":"2023-11-06T02:13:33.249993+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:11:19.807848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":3,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.240593+0000","last_change":"2023-11-06T02:13:33.240593+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.240593+0000","last_undegraded":"2023-11-06T02:13:33.240593+0000","last_fullsized":"2023-11-06T02:13:33.240593+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:33:31.177122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":["4","7"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.249061+0000","last_change":"2023-11-06T02:13:33.249061+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.249061+0000","last_undegraded":"2023-11-06T02:13:33.249061+0000","last_fullsized":"2023-11-06T02:13:33.249061+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:16:52.758810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.277263+0000","last_change":"2023-11-06T02:13:33.272642+0000","last_active":"2023-11-06T02:13:33.277263+0000","last_peered":"2023-11-06T02:13:33.277263+0000","last_clean":"2023-11-06T02:13:33.277263+0000","last_became_active":"2023-11-06T02:13:33.271888+0000","last_became_peered":"2023-11-06T02:13:33.271888+0000","last_unstale":"2023-11-06T02:13:33.277263+0000","last_undegraded":"2023-11-06T02:13:33.277263+0000","last_fullsized":"2023-11-06T02:13:33.277263+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:51:47.363705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.242755+0000","last_change":"2023-11-06T02:13:33.242755+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.242755+0000","last_undegraded":"2023-11-06T02:13:33.242755+0000","last_fullsized":"2023-11-06T02:13:33.242755+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:16:36.406664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":3,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.243828+0000","last_change":"2023-11-06T02:13:33.243828+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.243828+0000","last_undegraded":"2023-11-06T02:13:33.243828+0000","last_fullsized":"2023-11-06T02:13:33.243828+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:54:32.434908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":["0","7"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.245643+0000","last_change":"2023-11-06T02:13:33.245643+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.245643+0000","last_undegraded":"2023-11-06T02:13:33.245643+0000","last_fullsized":"2023-11-06T02:13:33.245643+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:33:50.179908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":3,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.235521+0000","last_change":"2023-11-06T02:13:33.235521+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.235521+0000","last_undegraded":"2023-11-06T02:13:33.235521+0000","last_fullsized":"2023-11-06T02:13:33.235521+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:39:03.018907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":["2","4"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.249061+0000","last_change":"2023-11-06T02:13:33.249061+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.249061+0000","last_undegraded":"2023-11-06T02:13:33.249061+0000","last_fullsized":"2023-11-06T02:13:33.249061+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:32:02.459972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":3,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.259605+0000","last_change":"2023-11-06T02:13:33.259605+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.259605+0000","last_undegraded":"2023-11-06T02:13:33.259605+0000","last_fullsized":"2023-11-06T02:13:33.259605+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:53:46.349156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":["7","6"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.253727+0000","last_change":"2023-11-06T02:13:33.253727+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.253727+0000","last_undegraded":"2023-11-06T02:13:33.253727+0000","last_fullsized":"2023-11-06T02:13:33.253727+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:00:13.166796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.283810+0000","last_change":"2023-11-06T02:13:33.281331+0000","last_active":"2023-11-06T02:13:33.283810+0000","last_peered":"2023-11-06T02:13:33.283810+0000","last_clean":"2023-11-06T02:13:33.283810+0000","last_became_active":"2023-11-06T02:13:33.280630+0000","last_became_peered":"2023-11-06T02:13:33.280630+0000","last_unstale":"2023-11-06T02:13:33.283810+0000","last_undegraded":"2023-11-06T02:13:33.283810+0000","last_fullsized":"2023-11-06T02:13:33.283810+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:17:16.208032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.256899+0000","last_change":"2023-11-06T02:13:33.256899+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.256899+0000","last_undegraded":"2023-11-06T02:13:33.256899+0000","last_fullsized":"2023-11-06T02:13:33.256899+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:45:05.474573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":3,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.235632+0000","last_change":"2023-11-06T02:13:33.235632+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.235632+0000","last_undegraded":"2023-11-06T02:13:33.235632+0000","last_fullsized":"2023-11-06T02:13:33.235632+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:55:36.788472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.251869+0000","last_change":"2023-11-06T02:13:33.251869+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.251869+0000","last_undegraded":"2023-11-06T02:13:33.251869+0000","last_fullsized":"2023-11-06T02:13:33.251869+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:33:50.179908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.277447+0000","last_change":"2023-11-06T02:13:33.265304+0000","last_active":"2023-11-06T02:13:33.277447+0000","last_peered":"2023-11-06T02:13:33.277447+0000","last_clean":"2023-11-06T02:13:33.277447+0000","last_became_active":"2023-11-06T02:13:33.264524+0000","last_became_peered":"2023-11-06T02:13:33.264524+0000","last_unstale":"2023-11-06T02:13:33.277447+0000","last_undegraded":"2023-11-06T02:13:33.277447+0000","last_fullsized":"2023-11-06T02:13:33.277447+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:39:01.059129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.253815+0000","last_change":"2023-11-06T02:13:33.253815+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.253815+0000","last_undegraded":"2023-11-06T02:13:33.253815+0000","last_fullsized":"2023-11-06T02:13:33.253815+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:17:59.025543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.280385+0000","last_change":"2023-11-06T02:13:33.273211+0000","last_active":"2023-11-06T02:13:33.280385+0000","last_peered":"2023-11-06T02:13:33.280385+0000","last_clean":"2023-11-06T02:13:33.280385+0000","last_became_active":"2023-11-06T02:13:33.272067+0000","last_became_peered":"2023-11-06T02:13:33.272067+0000","last_unstale":"2023-11-06T02:13:33.280385+0000","last_undegraded":"2023-11-06T02:13:33.280385+0000","last_fullsized":"2023-11-06T02:13:33.280385+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:16:35.397189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.240661+0000","last_change":"2023-11-06T02:13:33.240661+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.240661+0000","last_undegraded":"2023-11-06T02:13:33.240661+0000","last_fullsized":"2023-11-06T02:13:33.240661+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:17:36.098143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":3,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.256327+0000","last_change":"2023-11-06T02:13:33.256327+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.256327+0000","last_undegraded":"2023-11-06T02:13:33.256327+0000","last_fullsized":"2023-11-06T02:13:33.256327+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:53:59.721435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":["7","2"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.241911+0000","last_change":"2023-11-06T02:13:33.241911+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.241911+0000","last_undegraded":"2023-11-06T02:13:33.241911+0000","last_fullsized":"2023-11-06T02:13:33.241911+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:10:52.936432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":3,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.235562+0000","last_change":"2023-11-06T02:13:33.235562+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.235562+0000","last_undegraded":"2023-11-06T02:13:33.235562+0000","last_fullsized":"2023-11-06T02:13:33.235562+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:30:40.192944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.244479+0000","last_change":"2023-11-06T02:13:33.244479+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.244479+0000","last_undegraded":"2023-11-06T02:13:33.244479+0000","last_fullsized":"2023-11-06T02:13:33.244479+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:21:05.055643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":3,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.247845+0000","last_change":"2023-11-06T02:13:33.247845+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.247845+0000","last_undegraded":"2023-11-06T02:13:33.247845+0000","last_fullsized":"2023-11-06T02:13:33.247845+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:17:58.016068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":["0","6"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.241033+0000","last_change":"2023-11-06T02:13:33.241033+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.241033+0000","last_undegraded":"2023-11-06T02:13:33.241033+0000","last_fullsized":"2023-11-06T02:13:33.241033+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:54:00.730910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.279807+0000","last_change":"2023-11-06T02:13:33.272867+0000","last_active":"2023-11-06T02:13:33.279807+0000","last_peered":"2023-11-06T02:13:33.279807+0000","last_clean":"2023-11-06T02:13:33.279807+0000","last_became_active":"2023-11-06T02:13:33.272136+0000","last_became_peered":"2023-11-06T02:13:33.272136+0000","last_unstale":"2023-11-06T02:13:33.279807+0000","last_undegraded":"2023-11-06T02:13:33.279807+0000","last_fullsized":"2023-11-06T02:13:33.279807+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:02:50.688184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.238294+0000","last_change":"2023-11-06T02:13:33.238294+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.238294+0000","last_undegraded":"2023-11-06T02:13:33.238294+0000","last_fullsized":"2023-11-06T02:13:33.238294+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:02:47.316294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":3,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.244338+0000","last_change":"2023-11-06T02:13:33.244338+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.244338+0000","last_undegraded":"2023-11-06T02:13:33.244338+0000","last_fullsized":"2023-11-06T02:13:33.244338+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:11:59.707658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.245156+0000","last_change":"2023-11-06T02:13:33.245156+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.245156+0000","last_undegraded":"2023-11-06T02:13:33.245156+0000","last_fullsized":"2023-11-06T02:13:33.245156+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:07:57.793830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":3,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.258159+0000","last_change":"2023-11-06T02:13:33.258159+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.258159+0000","last_undegraded":"2023-11-06T02:13:33.258159+0000","last_fullsized":"2023-11-06T02:13:33.258159+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:16:35.397189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":["7","5"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.248850+0000","last_change":"2023-11-06T02:13:33.248850+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.248850+0000","last_undegraded":"2023-11-06T02:13:33.248850+0000","last_fullsized":"2023-11-06T02:13:33.248850+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:22:47.360905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":3,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.256366+0000","last_change":"2023-11-06T02:13:33.256366+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.256366+0000","last_undegraded":"2023-11-06T02:13:33.256366+0000","last_fullsized":"2023-11-06T02:13:33.256366+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:12:35.766676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":["7","6"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.244536+0000","last_change":"2023-11-06T02:13:33.244536+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.244536+0000","last_undegraded":"2023-11-06T02:13:33.244536+0000","last_fullsized":"2023-11-06T02:13:33.244536+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:42:42.388106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.277757+0000","last_change":"2023-11-06T02:13:33.267817+0000","last_active":"2023-11-06T02:13:33.277757+0000","last_peered":"2023-11-06T02:13:33.277757+0000","last_clean":"2023-11-06T02:13:33.277757+0000","last_became_active":"2023-11-06T02:13:33.267091+0000","last_became_peered":"2023-11-06T02:13:33.267091+0000","last_unstale":"2023-11-06T02:13:33.277757+0000","last_undegraded":"2023-11-06T02:13:33.277757+0000","last_fullsized":"2023-11-06T02:13:33.277757+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:36:21.262835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.239368+0000","last_change":"2023-11-06T02:13:33.239368+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.239368+0000","last_undegraded":"2023-11-06T02:13:33.239368+0000","last_fullsized":"2023-11-06T02:13:33.239368+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:31:19.083004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":3,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.235609+0000","last_change":"2023-11-06T02:13:33.235609+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.235609+0000","last_undegraded":"2023-11-06T02:13:33.235609+0000","last_fullsized":"2023-11-06T02:13:33.235609+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:19:20.386051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.254957+0000","last_change":"2023-11-06T02:13:33.254957+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.254957+0000","last_undegraded":"2023-11-06T02:13:33.254957+0000","last_fullsized":"2023-11-06T02:13:33.254957+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:02:47.316294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.277367+0000","last_change":"2023-11-06T02:13:33.264127+0000","last_active":"2023-11-06T02:13:33.277367+0000","last_peered":"2023-11-06T02:13:33.277367+0000","last_clean":"2023-11-06T02:13:33.277367+0000","last_became_active":"2023-11-06T02:13:33.263418+0000","last_became_peered":"2023-11-06T02:13:33.263418+0000","last_unstale":"2023-11-06T02:13:33.277367+0000","last_undegraded":"2023-11-06T02:13:33.277367+0000","last_fullsized":"2023-11-06T02:13:33.277367+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:04:53.601947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.253013+0000","last_change":"2023-11-06T02:13:33.253013+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.253013+0000","last_undegraded":"2023-11-06T02:13:33.253013+0000","last_fullsized":"2023-11-06T02:13:33.253013+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:31:19.083004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":3,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.251425+0000","last_change":"2023-11-06T02:13:33.251425+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.251425+0000","last_undegraded":"2023-11-06T02:13:33.251425+0000","last_fullsized":"2023-11-06T02:13:33.251425+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:13:44.482660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":["1","3"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.252952+0000","last_change":"2023-11-06T02:13:33.252952+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.252952+0000","last_undegraded":"2023-11-06T02:13:33.252952+0000","last_fullsized":"2023-11-06T02:13:33.252952+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:12:00.717133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":3,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.253882+0000","last_change":"2023-11-06T02:13:33.253882+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.253882+0000","last_undegraded":"2023-11-06T02:13:33.253882+0000","last_fullsized":"2023-11-06T02:13:33.253882+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:16:51.749335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":["3","6"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":216,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-06T02:13:32.232683+0000","last_change":"2023-11-06T02:08:20.523427+0000","last_active":"2023-11-06T02:13:32.232683+0000","last_peered":"2023-11-06T02:13:32.232683+0000","last_clean":"2023-11-06T02:13:32.232683+0000","last_became_active":"2023-11-06T02:08:20.521089+0000","last_became_peered":"2023-11-06T02:08:20.521089+0000","last_unstale":"2023-11-06T02:13:32.232683+0000","last_undegraded":"2023-11-06T02:13:32.232683+0000","last_fullsized":"2023-11-06T02:13:32.232683+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:19.252348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:19.252348+0000","last_clean_scrub_stamp":"2023-11-06T02:08:19.252348+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:30:15.566397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":62,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-06T02:13:32.246890+0000","last_change":"2023-11-06T02:08:26.059502+0000","last_active":"2023-11-06T02:13:32.246890+0000","last_peered":"2023-11-06T02:13:32.246890+0000","last_clean":"2023-11-06T02:13:32.246890+0000","last_became_active":"2023-11-06T02:08:26.058612+0000","last_became_peered":"2023-11-06T02:08:26.058612+0000","last_unstale":"2023-11-06T02:13:32.246890+0000","last_undegraded":"2023-11-06T02:13:32.246890+0000","last_fullsized":"2023-11-06T02:13:32.246890+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:10.626021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.250595+0000","last_change":"2023-11-06T02:13:33.250595+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.250595+0000","last_undegraded":"2023-11-06T02:13:33.250595+0000","last_fullsized":"2023-11-06T02:13:33.250595+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:46:01.922659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":3,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.246452+0000","last_change":"2023-11-06T02:13:33.246452+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.246452+0000","last_undegraded":"2023-11-06T02:13:33.246452+0000","last_fullsized":"2023-11-06T02:13:33.246452+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:11:51.109403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":["3","6"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":63,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-06T02:13:32.232564+0000","last_change":"2023-11-06T02:08:26.060406+0000","last_active":"2023-11-06T02:13:32.232564+0000","last_peered":"2023-11-06T02:13:32.232564+0000","last_clean":"2023-11-06T02:13:32.232564+0000","last_became_active":"2023-11-06T02:08:26.059262+0000","last_became_peered":"2023-11-06T02:08:26.059262+0000","last_unstale":"2023-11-06T02:13:32.232564+0000","last_undegraded":"2023-11-06T02:13:32.232564+0000","last_fullsized":"2023-11-06T02:13:32.232564+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:10.626021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.243210+0000","last_change":"2023-11-06T02:13:33.243210+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.243210+0000","last_undegraded":"2023-11-06T02:13:33.243210+0000","last_fullsized":"2023-11-06T02:13:33.243210+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:33:32.186597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":3,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.244333+0000","last_change":"2023-11-06T02:13:33.244333+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.244333+0000","last_undegraded":"2023-11-06T02:13:33.244333+0000","last_fullsized":"2023-11-06T02:13:33.244333+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:51:52.147063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":["3","6"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":63,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-06T02:13:32.236950+0000","last_change":"2023-11-06T02:08:26.559477+0000","last_active":"2023-11-06T02:13:32.236950+0000","last_peered":"2023-11-06T02:13:32.236950+0000","last_clean":"2023-11-06T02:13:32.236950+0000","last_became_active":"2023-11-06T02:08:26.559202+0000","last_became_peered":"2023-11-06T02:08:26.559202+0000","last_unstale":"2023-11-06T02:13:32.236950+0000","last_undegraded":"2023-11-06T02:13:32.236950+0000","last_fullsized":"2023-11-06T02:13:32.236950+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:55:04.552404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.249368+0000","last_change":"2023-11-06T02:13:33.249368+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.249368+0000","last_undegraded":"2023-11-06T02:13:33.249368+0000","last_fullsized":"2023-11-06T02:13:33.249368+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:54:11.389133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":3,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.245263+0000","last_change":"2023-11-06T02:13:33.245263+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.245263+0000","last_undegraded":"2023-11-06T02:13:33.245263+0000","last_fullsized":"2023-11-06T02:13:33.245263+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:38:14.971699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":62,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-06T02:13:32.246949+0000","last_change":"2023-11-06T02:08:26.059546+0000","last_active":"2023-11-06T02:13:32.246949+0000","last_peered":"2023-11-06T02:13:32.246949+0000","last_clean":"2023-11-06T02:13:32.246949+0000","last_became_active":"2023-11-06T02:08:26.058806+0000","last_became_peered":"2023-11-06T02:08:26.058806+0000","last_unstale":"2023-11-06T02:13:32.246949+0000","last_undegraded":"2023-11-06T02:13:32.246949+0000","last_fullsized":"2023-11-06T02:13:32.246949+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:23.468510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.242981+0000","last_change":"2023-11-06T02:13:33.242981+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.242981+0000","last_undegraded":"2023-11-06T02:13:33.242981+0000","last_fullsized":"2023-11-06T02:13:33.242981+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:36:12.474443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":3,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.236266+0000","last_change":"2023-11-06T02:13:33.236266+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.236266+0000","last_undegraded":"2023-11-06T02:13:33.236266+0000","last_fullsized":"2023-11-06T02:13:33.236266+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:30:05.141853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":["2","4"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":62,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-06T02:13:32.229548+0000","last_change":"2023-11-06T02:08:26.436463+0000","last_active":"2023-11-06T02:13:32.229548+0000","last_peered":"2023-11-06T02:13:32.229548+0000","last_clean":"2023-11-06T02:13:32.229548+0000","last_became_active":"2023-11-06T02:08:26.436075+0000","last_became_peered":"2023-11-06T02:08:26.436075+0000","last_unstale":"2023-11-06T02:13:32.229548+0000","last_undegraded":"2023-11-06T02:13:32.229548+0000","last_fullsized":"2023-11-06T02:13:32.229548+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:23.468510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.246317+0000","last_change":"2023-11-06T02:13:33.246317+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.246317+0000","last_undegraded":"2023-11-06T02:13:33.246317+0000","last_fullsized":"2023-11-06T02:13:33.246317+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:25:19.608926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":3,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.251742+0000","last_change":"2023-11-06T02:13:33.251742+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.251742+0000","last_undegraded":"2023-11-06T02:13:33.251742+0000","last_fullsized":"2023-11-06T02:13:33.251742+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:23:42.695735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":63,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-06T02:13:32.239073+0000","last_change":"2023-11-06T02:08:26.436607+0000","last_active":"2023-11-06T02:13:32.239073+0000","last_peered":"2023-11-06T02:13:32.239073+0000","last_clean":"2023-11-06T02:13:32.239073+0000","last_became_active":"2023-11-06T02:08:26.435924+0000","last_became_peered":"2023-11-06T02:08:26.435924+0000","last_unstale":"2023-11-06T02:13:32.239073+0000","last_undegraded":"2023-11-06T02:13:32.239073+0000","last_fullsized":"2023-11-06T02:13:32.239073+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:23.468510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.244818+0000","last_change":"2023-11-06T02:13:33.244818+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.244818+0000","last_undegraded":"2023-11-06T02:13:33.244818+0000","last_fullsized":"2023-11-06T02:13:33.244818+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:22:01.334447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":4,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.269809+0000","last_change":"2023-11-06T02:13:33.260450+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.269809+0000","last_undegraded":"2023-11-06T02:13:33.269809+0000","last_fullsized":"2023-11-06T02:13:33.269809+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:51:47.363705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":["7","1"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":62,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-06T02:13:32.229549+0000","last_change":"2023-11-06T02:08:26.057076+0000","last_active":"2023-11-06T02:13:32.229549+0000","last_peered":"2023-11-06T02:13:32.229549+0000","last_clean":"2023-11-06T02:13:32.229549+0000","last_became_active":"2023-11-06T02:08:26.056419+0000","last_became_peered":"2023-11-06T02:08:26.056419+0000","last_unstale":"2023-11-06T02:13:32.229549+0000","last_undegraded":"2023-11-06T02:13:32.229549+0000","last_fullsized":"2023-11-06T02:13:32.229549+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:10.626021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.242736+0000","last_change":"2023-11-06T02:13:33.242736+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.242736+0000","last_undegraded":"2023-11-06T02:13:33.242736+0000","last_fullsized":"2023-11-06T02:13:33.242736+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:38:31.470172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.265697+0000","last_change":"2023-11-06T02:13:33.265697+0000","last_active":"2023-11-06T02:13:33.265697+0000","last_peered":"2023-11-06T02:13:33.265697+0000","last_clean":"2023-11-06T02:13:33.265697+0000","last_became_active":"2023-11-06T02:13:33.264914+0000","last_became_peered":"2023-11-06T02:13:33.264914+0000","last_unstale":"2023-11-06T02:13:33.265697+0000","last_undegraded":"2023-11-06T02:13:33.265697+0000","last_fullsized":"2023-11-06T02:13:33.265697+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:17:35.088668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":71,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-06T02:13:32.237181+0000","last_change":"2023-11-06T02:08:26.058596+0000","last_active":"2023-11-06T02:13:32.237181+0000","last_peered":"2023-11-06T02:13:32.237181+0000","last_clean":"2023-11-06T02:13:32.237181+0000","last_became_active":"2023-11-06T02:08:26.057015+0000","last_became_peered":"2023-11-06T02:08:26.057015+0000","last_unstale":"2023-11-06T02:13:32.237181+0000","last_undegraded":"2023-11-06T02:13:32.237181+0000","last_fullsized":"2023-11-06T02:13:32.237181+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:55:04.552404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574914,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32544,"kb_used_data":1496,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339296,"statfs":{"total":96636764160,"available":96603439104,"internally_reserved":0,"allocated":1531904,"data_stored":1009349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:13:26 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.86599999999999999,"15min":0.85399999999999998},"min":{"1min":0.59199999999999997,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.2190000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.62},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.91200000000000003,"15min":0.91100000000000003},"min":{"1min":0.57299999999999995,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.214,"5min":1.506,"15min":1.506},"last":1}]},{"osd":1,"last update":"Mon Nov 6 02:13:26 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.83599999999999997,"15min":0.84599999999999997},"min":{"1min":0.66400000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":0.94799999999999995},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.94099999999999995,"15min":0.90600000000000003},"min":{"1min":0.64300000000000002,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.681,"5min":1.681,"15min":1.681},"last":0.75600000000000001}]},{"osd":2,"last update":"Mon Nov 6 02:13:26 2023","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.96599999999999997,"15min":0.95799999999999996},"min":{"1min":0.66200000000000003,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.22,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.94899999999999995,"15min":0.98299999999999998},"min":{"1min":0.71899999999999997,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.2789999999999999,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.70499999999999996}]},{"osd":3,"last update":"Mon Nov 6 02:13:26 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.89400000000000002,"15min":0.88100000000000001},"min":{"1min":0.67500000000000004,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.196,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.78300000000000003},{"interface":"front","average":{"1min":1.069,"5min":1.046,"15min":1.0660000000000001},"min":{"1min":0.86399999999999999,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.2430000000000001,"5min":1.526,"15min":1.526},"last":0.91300000000000003}]},{"osd":4,"last update":"Mon Nov 6 02:13:26 2023","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.92300000000000004,"15min":0.94599999999999995},"min":{"1min":0.59699999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.2350000000000001,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.874},{"interface":"front","average":{"1min":1.0129999999999999,"5min":1.004,"15min":1.0309999999999999},"min":{"1min":0.625,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.377,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":1.052}]},{"osd":5,"last update":"Mon Nov 6 02:13:26 2023","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.038,"15min":1.0649999999999999},"min":{"1min":0.76300000000000001,"5min":0.503,"15min":0.503},"max":{"1min":1.409,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":1.032},{"interface":"front","average":{"1min":1.046,"5min":1.0629999999999999,"15min":1.111},"min":{"1min":0.75900000000000001,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.3109999999999999,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":1.087}]},{"osd":6,"last update":"Mon Nov 6 02:13:26 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.1399999999999999,"15min":1.181},"min":{"1min":0.59399999999999997,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.278,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":1.1639999999999999},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.091,"15min":1.135},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.4570000000000001,"5min":1.74,"15min":1.74},"last":1.133}]}]},{"osd":6,"up_from":13,"seq":55834574913,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31964,"kb_used_data":916,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339876,"statfs":{"total":96636764160,"available":96604033024,"internally_reserved":0,"allocated":937984,"data_stored":418981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:13:28 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.86699999999999999,"15min":0.86099999999999999},"min":{"1min":0.58899999999999997,"5min":0.495,"15min":0.495},"max":{"1min":1.202,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.98099999999999998},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.874,"15min":0.82799999999999996},"min":{"1min":0.63,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.2649999999999999,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.56399999999999995}]},{"osd":1,"last update":"Mon Nov 6 02:13:28 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.84999999999999998,"15min":0.86699999999999999},"min":{"1min":0.63500000000000001,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.244,"5min":1.244,"15min":1.244},"last":1.1639999999999999},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.89600000000000002,"15min":0.88100000000000001},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.624}]},{"osd":2,"last update":"Mon Nov 6 02:13:28 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.88700000000000001,"15min":0.89500000000000002},"min":{"1min":0.52100000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.206,"5min":1.252,"15min":1.252},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.90000000000000002,"15min":0.92000000000000004},"min":{"1min":0.61699999999999999,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.411,"5min":1.411,"15min":1.411},"last":0.82899999999999996}]},{"osd":3,"last update":"Mon Nov 6 02:13:28 2023","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.93899999999999995,"15min":0.96099999999999997},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.212,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.0549999999999999,"5min":0.99299999999999999,"15min":0.998},"min":{"1min":0.77300000000000002,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.381,"5min":1.381,"15min":1.381},"last":1.101}]},{"osd":4,"last update":"Mon Nov 6 02:13:28 2023","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.92600000000000005,"15min":0.97099999999999997},"min":{"1min":0.67500000000000004,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.302,"5min":1.474,"15min":1.474},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.0469999999999999,"5min":0.98199999999999998,"15min":1.02},"min":{"1min":0.77100000000000002,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.556,"5min":1.556,"15min":1.556},"last":1.2050000000000001}]},{"osd":5,"last update":"Mon Nov 6 02:13:28 2023","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":0.97199999999999998,"15min":1.0089999999999999},"min":{"1min":0.73099999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":1.4650000000000001},{"interface":"front","average":{"1min":1.129,"5min":1.0720000000000001,"15min":1.1000000000000001},"min":{"1min":0.94899999999999995,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.587,"5min":1.587,"15min":1.587},"last":1.3999999999999999}]},{"osd":7,"last update":"Mon Nov 6 02:13:28 2023","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.087,"15min":1.107},"min":{"1min":0.95899999999999996,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.615,"5min":1.615,"15min":1.615},"last":1.363},{"interface":"front","average":{"1min":1.109,"5min":1.0629999999999999,"15min":1.131},"min":{"1min":0.83499999999999996,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":1.2969999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574914,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":916,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":937984,"data_stored":418981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:13:30 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.86899999999999999,"15min":0.84199999999999997},"min":{"1min":0.51700000000000002,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.1930000000000001,"5min":1.381,"15min":1.381},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.86599999999999999,"15min":0.84999999999999998},"min":{"1min":0.58199999999999996,"5min":0.503,"15min":0.503},"max":{"1min":1.1399999999999999,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":1.1399999999999999}]},{"osd":1,"last update":"Mon Nov 6 02:13:30 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.89600000000000002,"15min":0.86499999999999999},"min":{"1min":0.69799999999999995,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.177,"5min":1.331,"15min":1.331},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.88100000000000001,"15min":0.83799999999999997},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.139,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.71399999999999997}]},{"osd":2,"last update":"Mon Nov 6 02:13:30 2023","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.93400000000000005,"15min":0.94199999999999995},"min":{"1min":0.68000000000000005,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.298,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.872},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.88400000000000001,"15min":0.82099999999999995},"min":{"1min":0.56899999999999995,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.2869999999999999,"5min":1.377,"15min":1.377},"last":0.98399999999999999}]},{"osd":3,"last update":"Mon Nov 6 02:13:30 2023","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.95399999999999996,"15min":0.93100000000000005},"min":{"1min":0.77100000000000002,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.2210000000000001,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":1.091},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.997,"15min":1.0529999999999999},"min":{"1min":0.67900000000000005,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.224,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":0.78900000000000003}]},{"osd":4,"last update":"Mon Nov 6 02:13:30 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.89500000000000002,"15min":0.90900000000000003},"min":{"1min":0.58099999999999996,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.1779999999999999,"5min":1.276,"15min":1.276},"last":0.63700000000000001},{"interface":"front","average":{"1min":1.006,"5min":0.99299999999999999,"15min":1.016},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.254,"5min":1.554,"15min":1.554},"last":0.94799999999999995}]},{"osd":6,"last update":"Mon Nov 6 02:13:30 2023","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":1.0089999999999999,"15min":1.048},"min":{"1min":0.71399999999999997,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.2569999999999999,"5min":1.454,"15min":1.454},"last":1.016},{"interface":"front","average":{"1min":0.98699999999999999,"5min":1.022,"15min":1.034},"min":{"1min":0.75,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.1930000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.76000000000000001}]},{"osd":7,"last update":"Mon Nov 6 02:13:30 2023","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.0609999999999999,"15min":1.071},"min":{"1min":0.77900000000000003,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.2330000000000001,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":1.048},{"interface":"front","average":{"1min":1.109,"5min":1.1120000000000001,"15min":1.1399999999999999},"min":{"1min":0.84599999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.3799999999999999,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":1.0740000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574914,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":916,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":937984,"data_stored":418981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:13:27 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.877,"15min":0.84199999999999997},"min":{"1min":0.65100000000000002,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.0840000000000001,"5min":1.343,"15min":1.343},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.83299999999999996,"15min":0.79100000000000004},"min":{"1min":0.626,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.1699999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.77000000000000002}]},{"osd":1,"last update":"Mon Nov 6 02:13:27 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.86799999999999999,"15min":0.85199999999999998},"min":{"1min":0.57299999999999995,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.0329999999999999,"5min":1.462,"15min":1.462},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.93300000000000005,"15min":0.90500000000000003},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.224,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.59699999999999998}]},{"osd":2,"last update":"Mon Nov 6 02:13:27 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.93799999999999994,"15min":0.99199999999999999},"min":{"1min":0.70299999999999996,"5min":0.503,"15min":0.503},"max":{"1min":1.173,"5min":1.615,"15min":1.615},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.95499999999999996,"15min":0.93500000000000005},"min":{"1min":0.72499999999999998,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.163,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.97899999999999998}]},{"osd":3,"last update":"Mon Nov 6 02:13:27 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.88200000000000001,"15min":0.90600000000000003},"min":{"1min":0.61499999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.216,"5min":1.52,"15min":1.52},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.997,"15min":1.0449999999999999},"min":{"1min":0.66900000000000004,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.2050000000000001,"5min":1.595,"15min":1.595},"last":0.80900000000000005}]},{"osd":5,"last update":"Mon Nov 6 02:13:27 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.99099999999999999,"15min":1.0309999999999999},"min":{"1min":0.64500000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.339,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":1.0800000000000001},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.97599999999999998,"15min":1.0289999999999999},"min":{"1min":0.68899999999999995,"5min":0.629,"15min":0.629},"max":{"1min":1.161,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":0.86699999999999999}]},{"osd":6,"last update":"Mon Nov 6 02:13:27 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":1.008,"15min":1.0649999999999999},"min":{"1min":0.69599999999999995,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.268,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":1.03},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.1120000000000001,"15min":1.169},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.2789999999999999,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":1.129}]},{"osd":7,"last update":"Mon Nov 6 02:13:27 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.0489999999999999,"15min":1.079},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.294,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.93899999999999995},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.133,"15min":1.204},"min":{"1min":0.77500000000000002,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.3149999999999999,"5min":1.7410000000000001,"15min":1.7410000000000001},"last":1.1060000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574913,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31964,"kb_used_data":916,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339876,"statfs":{"total":96636764160,"available":96604033024,"internally_reserved":0,"allocated":937984,"data_stored":418981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:13:28 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.63200000000000001,"15min":0.64200000000000002},"min":{"1min":0.40799999999999997,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":1.198,"5min":1.198,"15min":1.198},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.64000000000000001,"15min":0.63},"min":{"1min":0.38700000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.72099999999999997}]},{"osd":1,"last update":"Mon Nov 6 02:13:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.61699999999999999,"15min":0.627},"min":{"1min":0.34599999999999997,"5min":0.248,"15min":0.248},"max":{"1min":0.92600000000000005,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.73999999999999999,"15min":0.73599999999999999},"min":{"1min":0.53000000000000003,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.60099999999999998}]},{"osd":2,"last update":"Mon Nov 6 02:13:28 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.71899999999999997,"15min":0.72899999999999998},"min":{"1min":0.45400000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.107,"5min":1.139,"15min":1.139},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.80800000000000005,"15min":0.81799999999999995},"min":{"1min":0.48099999999999998,"5min":0.308,"15min":0.308},"max":{"1min":1.212,"5min":1.405,"15min":1.405},"last":0.75700000000000001}]},{"osd":4,"last update":"Mon Nov 6 02:13:28 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.92000000000000004,"15min":0.92600000000000005},"min":{"1min":0.72599999999999998,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.0820000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.95299999999999996,"15min":0.98299999999999998},"min":{"1min":0.65100000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.1459999999999999,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.97399999999999998}]},{"osd":5,"last update":"Mon Nov 6 02:13:28 2023","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.98899999999999999,"15min":1.0149999999999999},"min":{"1min":0.72799999999999998,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.333,"5min":1.375,"15min":1.375},"last":0.81299999999999994},{"interface":"front","average":{"1min":1.091,"5min":1.0389999999999999,"15min":1.0669999999999999},"min":{"1min":0.627,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.4890000000000001,"5min":2.21,"15min":2.21},"last":1.0269999999999999}]},{"osd":6,"last update":"Mon Nov 6 02:13:28 2023","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.016,"15min":1.022},"min":{"1min":0.61299999999999999,"5min":0.439,"15min":0.439},"max":{"1min":1.3740000000000001,"5min":1.395,"15min":1.395},"last":0.78900000000000003},{"interface":"front","average":{"1min":1.036,"5min":1.04,"15min":1.0609999999999999},"min":{"1min":0.54600000000000004,"5min":0.375,"15min":0.375},"max":{"1min":1.3069999999999999,"5min":1.554,"15min":1.554},"last":1.107}]},{"osd":7,"last update":"Mon Nov 6 02:13:28 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.1080000000000001,"15min":1.1240000000000001},"min":{"1min":0.66800000000000004,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.421,"5min":2.2829999999999999,"15min":2.2829999999999999},"last":0.94099999999999995},{"interface":"front","average":{"1min":1.091,"5min":1.0940000000000001,"15min":1.1020000000000001},"min":{"1min":0.70999999999999996,"5min":0.497,"15min":0.497},"max":{"1min":1.4490000000000001,"5min":1.504,"15min":1.504},"last":1.048}]}]},{"osd":2,"up_from":13,"seq":55834574913,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31964,"kb_used_data":916,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339876,"statfs":{"total":96636764160,"available":96604033024,"internally_reserved":0,"allocated":937984,"data_stored":418981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:13:24 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.57699999999999996,"15min":0.52500000000000002},"min":{"1min":0.33100000000000002,"5min":0.252,"15min":0.252},"max":{"1min":0.85699999999999998,"5min":1.079,"15min":1.079},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.625,"15min":0.56699999999999995},"min":{"1min":0.505,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.1060000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":0.56999999999999995}]},{"osd":1,"last update":"Mon Nov 6 02:13:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.63500000000000001,"15min":0.58499999999999996},"min":{"1min":0.38800000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.88300000000000001,"5min":1.48,"15min":1.48},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.70599999999999996,"15min":0.61399999999999999},"min":{"1min":0.377,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.282,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":0.70499999999999996}]},{"osd":3,"last update":"Mon Nov 6 02:13:24 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.70099999999999996,"15min":0.63200000000000001},"min":{"1min":0.48199999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.98999999999999999,"5min":1.155,"15min":1.155},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.749,"15min":0.67700000000000005},"min":{"1min":0.41999999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.81499999999999995}]},{"osd":4,"last update":"Mon Nov 6 02:13:24 2023","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.90900000000000003,"15min":0.83499999999999996},"min":{"1min":0.58499999999999996,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.2150000000000001,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.93500000000000005,"15min":0.92100000000000004},"min":{"1min":0.73099999999999998,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.3120000000000001,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":1.0589999999999999}]},{"osd":5,"last update":"Mon Nov 6 02:13:24 2023","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":0.96799999999999997,"15min":0.95099999999999996},"min":{"1min":0.75,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.2270000000000001,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":0.93999999999999995},{"interface":"front","average":{"1min":1,"5min":0.97099999999999997,"15min":0.94599999999999995},"min":{"1min":0.628,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.3109999999999999,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.96399999999999997}]},{"osd":6,"last update":"Mon Nov 6 02:13:24 2023","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":0.97099999999999997,"15min":0.94899999999999995},"min":{"1min":0.59899999999999998,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.3129999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.0229999999999999,"15min":1.0049999999999999},"min":{"1min":0.73299999999999998,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.2769999999999999,"5min":1.677,"15min":1.677},"last":1.1479999999999999}]},{"osd":7,"last update":"Mon Nov 6 02:13:24 2023","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.026,"15min":0.99199999999999999},"min":{"1min":0.72199999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.4079999999999999,"5min":1.659,"15min":1.659},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.123,"5min":1.0820000000000001,"15min":1.0549999999999999},"min":{"1min":0.79300000000000004,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.3819999999999999,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":1.087}]}]},{"osd":1,"up_from":13,"seq":55834574914,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31964,"kb_used_data":916,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339876,"statfs":{"total":96636764160,"available":96604033024,"internally_reserved":0,"allocated":937984,"data_stored":418981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:13:27 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.63900000000000001,"15min":0.63},"min":{"1min":0.35899999999999999,"5min":0.224,"15min":0.224},"max":{"1min":1.343,"5min":1.603,"15min":1.603},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.71899999999999997,"15min":0.67400000000000004},"min":{"1min":0.38200000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":2.161,"5min":2.161,"15min":2.161},"last":1.081}]},{"osd":2,"last update":"Mon Nov 6 02:13:27 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.65400000000000003,"15min":0.63200000000000001},"min":{"1min":0.26500000000000001,"5min":0.253,"15min":0.253},"max":{"1min":1.256,"5min":1.649,"15min":1.649},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.753,"15min":0.747},"min":{"1min":0.41099999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":1.5249999999999999,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.88400000000000001}]},{"osd":3,"last update":"Mon Nov 6 02:13:27 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.81299999999999994,"15min":0.80600000000000005},"min":{"1min":0.49099999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.6319999999999999,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":1.131},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.81299999999999994,"15min":0.81100000000000005},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.1799999999999999,"5min":1.8149999999999999,"15min":1.8149999999999999},"last":0.94199999999999995}]},{"osd":4,"last update":"Mon Nov 6 02:13:27 2023","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":0.97899999999999998,"15min":0.98799999999999999},"min":{"1min":0.78500000000000003,"5min":0.504,"15min":0.504},"max":{"1min":2.2759999999999998,"5min":2.2759999999999998,"15min":2.2759999999999998},"last":1.196},{"interface":"front","average":{"1min":1.0740000000000001,"5min":0.97699999999999998,"15min":0.98799999999999999},"min":{"1min":0.64200000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":2.9180000000000001,"5min":2.9180000000000001,"15min":2.9180000000000001},"last":1.0049999999999999}]},{"osd":5,"last update":"Mon Nov 6 02:13:27 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":0.98099999999999998,"15min":0.97099999999999997},"min":{"1min":0.68100000000000005,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":2.7469999999999999,"5min":2.7469999999999999,"15min":2.7469999999999999},"last":1.1659999999999999},{"interface":"front","average":{"1min":1.1579999999999999,"5min":1.069,"15min":1.075},"min":{"1min":0.81100000000000005,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.921,"5min":1.921,"15min":1.921},"last":1.044}]},{"osd":6,"last update":"Mon Nov 6 02:13:27 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.113,"15min":1.1060000000000001},"min":{"1min":0.83099999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":3.5710000000000002,"5min":3.5710000000000002,"15min":3.5710000000000002},"last":1.278},{"interface":"front","average":{"1min":1.157,"5min":1.0629999999999999,"15min":1.0720000000000001},"min":{"1min":0.60299999999999998,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":2.843,"5min":2.843,"15min":2.843},"last":0.83499999999999996}]},{"osd":7,"last update":"Mon Nov 6 02:13:27 2023","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.1180000000000001,"15min":1.1200000000000001},"min":{"1min":0.89600000000000002,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":2.0009999999999999,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":1.2330000000000001},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.1639999999999999,"15min":1.1619999999999999},"min":{"1min":0.91100000000000003,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":3.4790000000000001,"5min":3.4790000000000001,"15min":3.4790000000000001},"last":1.3520000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574914,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32540,"kb_used_data":1492,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339300,"statfs":{"total":96636764160,"available":96603443200,"internally_reserved":0,"allocated":1527808,"data_stored":1008330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 02:13:26 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.61499999999999999,"15min":0.60299999999999998},"min":{"1min":0.53100000000000003,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.0289999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.755,"5min":0.66900000000000004,"15min":0.64700000000000002},"min":{"1min":0.53800000000000003,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.079,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.46300000000000002}]},{"osd":2,"last update":"Mon Nov 6 02:13:26 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.68600000000000005,"15min":0.67200000000000004},"min":{"1min":0.42999999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.1080000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.98299999999999998},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.69799999999999995,"15min":0.68200000000000005},"min":{"1min":0.57299999999999995,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.75700000000000001}]},{"osd":3,"last update":"Mon Nov 6 02:13:26 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.67200000000000004,"15min":0.64300000000000002},"min":{"1min":0.52500000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.1659999999999999,"5min":1.429,"15min":1.429},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.87,"5min":0.78300000000000003,"15min":0.748},"min":{"1min":0.69999999999999996,"5min":0.317,"15min":0.317},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.89000000000000001}]},{"osd":4,"last update":"Mon Nov 6 02:13:26 2023","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.90200000000000002,"15min":0.85199999999999998},"min":{"1min":0.76800000000000002,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.1779999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":1.0589999999999999,"5min":0.96099999999999997,"15min":0.94499999999999995},"min":{"1min":0.78900000000000003,"5min":0.5,"15min":0.5},"max":{"1min":1.292,"5min":1.387,"15min":1.387},"last":1.137}]},{"osd":5,"last update":"Mon Nov 6 02:13:26 2023","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":0.95699999999999996,"15min":0.95099999999999996},"min":{"1min":0.81799999999999995,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.006,"5min":0.93400000000000005,"15min":0.89200000000000002},"min":{"1min":0.81100000000000005,"5min":0.496,"15min":0.496},"max":{"1min":1.2549999999999999,"5min":1.448,"15min":1.448},"last":1.101}]},{"osd":6,"last update":"Mon Nov 6 02:13:26 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.0389999999999999,"15min":1.0369999999999999},"min":{"1min":0.73899999999999999,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.385,"5min":1.589,"15min":1.589},"last":1.236},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.0269999999999999,"15min":1.0089999999999999},"min":{"1min":0.76000000000000001,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.419,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":1.1779999999999999}]},{"osd":7,"last update":"Mon Nov 6 02:13:26 2023","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.026,"15min":1.0209999999999999},"min":{"1min":0.76400000000000001,"5min":0.504,"15min":0.504},"max":{"1min":1.504,"5min":1.504,"15min":1.504},"last":1.022},{"interface":"front","average":{"1min":1.073,"5min":0.998,"15min":1.0049999999999999},"min":{"1min":0.74299999999999999,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.3999999999999999,"5min":1.556,"15min":1.556},"last":0.873}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T02:13:37.689 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T02:13:37.852 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.858+0000 7f5191b54640 1 Processor -- start 2023-11-06T02:13:37.852 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.858+0000 7f5191b54640 1 -- start start 2023-11-06T02:13:37.853 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.858+0000 7f5191b54640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f518c071bb0 0x7f518c161650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:37.854 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.858+0000 7f5191b54640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f518c0d17c0 con 0x7f518c0cda70 2023-11-06T02:13:37.854 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.858+0000 7f5191b54640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f518c0ce6c0 con 0x7f518c0b1f90 2023-11-06T02:13:37.854 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.858+0000 7f5191b54640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f518c0cdf20 con 0x7f518c071bb0 2023-11-06T02:13:37.854 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.858+0000 7f518affd640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f518c0cda70 0x7f518c0714a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:60624/0 (socket says 172.21.15.49:60624) 2023-11-06T02:13:37.854 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.858+0000 7f518affd640 1 -- 172.21.15.49:0/778169634 learned_addr learned my addr 172.21.15.49:0/778169634 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:13:37.855 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.858+0000 7f518bfff640 1 --2- 172.21.15.49:0/778169634 >> v2:172.21.15.150:3300/0 conn(0x7f518c071bb0 0x7f518c161650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:37.855 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.858+0000 7f518a7fc640 1 -- 172.21.15.49:0/778169634 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 355552661 0 0) 0x7f518c0d17c0 con 0x7f518c0cda70 2023-11-06T02:13:37.855 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.858+0000 7f518a7fc640 1 -- 172.21.15.49:0/778169634 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5178003530 con 0x7f518c0cda70 2023-11-06T02:13:37.858 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.858+0000 7f518a7fc640 1 -- 172.21.15.49:0/778169634 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 230714026 0 0) 0x7f5178003530 con 0x7f518c0cda70 2023-11-06T02:13:37.858 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.858+0000 7f518a7fc640 1 -- 172.21.15.49:0/778169634 >> v2:172.21.15.150:3300/0 conn(0x7f518c071bb0 msgr2=0x7f518c161650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:37.858 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.858+0000 7f518a7fc640 1 --2- 172.21.15.49:0/778169634 >> v2:172.21.15.150:3300/0 conn(0x7f518c071bb0 0x7f518c161650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:37.858 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.858+0000 7f518a7fc640 1 -- 172.21.15.49:0/778169634 >> v1:172.21.15.150:6789/0 conn(0x7f518c0b1f90 legacy=0x7f518c0d1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:37.859 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.858+0000 7f518a7fc640 1 -- 172.21.15.49:0/778169634 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f518c0d2360 con 0x7f518c0cda70 2023-11-06T02:13:37.859 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.858+0000 7f518a7fc640 1 -- 172.21.15.49:0/778169634 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7f5174001680 con 0x7f518c0cda70 2023-11-06T02:13:37.859 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.858+0000 7f518a7fc640 1 -- 172.21.15.49:0/778169634 >> v1:172.21.15.49:6789/0 conn(0x7f518c0cda70 legacy=0x7f518c0714a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:37.859 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.858+0000 7f518a7fc640 1 --2- 172.21.15.49:0/778169634 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5178003980 0x7f5178003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:37.859 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.858+0000 7f518a7fc640 1 --2- 172.21.15.49:0/778169634 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51780047d0 0x7f5178004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:37.860 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.858+0000 7f518a7fc640 1 --2- 172.21.15.49:0/778169634 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5178005190 0x7f517800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:37.860 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.858+0000 7f518a7fc640 1 -- 172.21.15.49:0/778169634 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f518c0cdf20 con 0x7f51780047d0 2023-11-06T02:13:37.860 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.858+0000 7f518a7fc640 1 -- 172.21.15.49:0/778169634 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f517800c030 con 0x7f5178003980 2023-11-06T02:13:37.860 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.858+0000 7f518a7fc640 1 -- 172.21.15.49:0/778169634 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f517800c1f0 con 0x7f5178005190 2023-11-06T02:13:37.860 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.858+0000 7f518affd640 1 --2- 172.21.15.49:0/778169634 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51780047d0 0x7f5178004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:37.860 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.858+0000 7f518b7fe640 1 --2- 172.21.15.49:0/778169634 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5178003980 0x7f5178003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:37.861 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.858+0000 7f518bfff640 1 --2- 172.21.15.49:0/778169634 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5178005190 0x7f517800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:37.861 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.858+0000 7f518affd640 1 -- 172.21.15.49:0/778169634 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5178005190 msgr2=0x7f517800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:37.861 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.858+0000 7f518affd640 1 --2- 172.21.15.49:0/778169634 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5178005190 0x7f517800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:37.861 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.858+0000 7f518affd640 1 -- 172.21.15.49:0/778169634 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5178003980 msgr2=0x7f5178003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:37.862 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.858+0000 7f518affd640 1 --2- 172.21.15.49:0/778169634 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5178003980 0x7f5178003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:37.862 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.858+0000 7f518affd640 1 -- 172.21.15.49:0/778169634 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5178003530 con 0x7f51780047d0 2023-11-06T02:13:37.862 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.858+0000 7f518bfff640 1 --2- 172.21.15.49:0/778169634 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5178005190 0x7f517800b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:13:37.862 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.858+0000 7f518bfff640 1 -- 172.21.15.49:0/778169634 reap_dead start 2023-11-06T02:13:37.862 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.862+0000 7f518affd640 1 --2- 172.21.15.49:0/778169634 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51780047d0 0x7f5178004c50 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f517400c240 tx=0x7f517401f2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83f4ef53fb477d2e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:37.863 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.862+0000 7f518a7fc640 1 -- 172.21.15.49:0/778169634 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51740058e0 con 0x7f51780047d0 2023-11-06T02:13:37.863 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.862+0000 7f518a7fc640 1 -- 172.21.15.49:0/778169634 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5174005a60 con 0x7f51780047d0 2023-11-06T02:13:37.863 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.862+0000 7f518a7fc640 1 -- 172.21.15.49:0/778169634 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5174005e10 con 0x7f51780047d0 2023-11-06T02:13:37.863 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.862+0000 7f5191b54640 1 -- 172.21.15.49:0/778169634 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51780047d0 msgr2=0x7f5178004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:37.863 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.862+0000 7f5191b54640 1 --2- 172.21.15.49:0/778169634 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51780047d0 0x7f5178004c50 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f517400c240 tx=0x7f517401f2b0 comp rx=0 tx=0).stop 2023-11-06T02:13:37.864 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.862+0000 7f5191b54640 1 -- 172.21.15.49:0/778169634 shutdown_connections 2023-11-06T02:13:37.864 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.862+0000 7f5191b54640 1 --2- 172.21.15.49:0/778169634 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51780047d0 0x7f5178004c50 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:37.864 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.862+0000 7f5191b54640 1 -- 172.21.15.49:0/778169634 >> 172.21.15.49:0/778169634 conn(0x7f518c0d2aa0 msgr2=0x7f518c0d2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:37.864 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.862+0000 7f5191b54640 1 -- 172.21.15.49:0/778169634 shutdown_connections 2023-11-06T02:13:37.864 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.862+0000 7f5191b54640 1 -- 172.21.15.49:0/778169634 wait complete. 2023-11-06T02:13:37.867 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.862+0000 7f5191b54640 1 Processor -- start 2023-11-06T02:13:37.867 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.862+0000 7f5191b54640 1 -- start start 2023-11-06T02:13:37.868 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.862+0000 7f5191b54640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51780047d0 0x7f518c15d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:37.868 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.862+0000 7f5191b54640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f518c0cb630 0x7f518c150ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:37.868 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.862+0000 7f5191b54640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f518c0cba90 0x7f518c151410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:37.868 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.862+0000 7f5191b54640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f518c0cc8e0 con 0x7f518c0cb630 2023-11-06T02:13:37.869 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.862+0000 7f5191b54640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f518c0b2660 con 0x7f51780047d0 2023-11-06T02:13:37.869 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.862+0000 7f5191b54640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f518c0cc040 con 0x7f518c0cba90 2023-11-06T02:13:37.869 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.862+0000 7f518affd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f518c0cb630 0x7f518c150ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:37.869 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.862+0000 7f518affd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f518c0cb630 0x7f518c150ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51338/0 (socket says 172.21.15.49:51338) 2023-11-06T02:13:37.870 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.862+0000 7f518affd640 1 -- 172.21.15.49:0/3319831731 learned_addr learned my addr 172.21.15.49:0/3319831731 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:37.870 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.866+0000 7f518b7fe640 1 --2- 172.21.15.49:0/3319831731 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51780047d0 0x7f518c15d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:37.870 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.866+0000 7f518bfff640 1 --2- 172.21.15.49:0/3319831731 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f518c0cba90 0x7f518c151410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:37.870 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.866+0000 7f518affd640 1 -- 172.21.15.49:0/3319831731 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f518c0cba90 msgr2=0x7f518c151410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:37.870 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.866+0000 7f518affd640 1 --2- 172.21.15.49:0/3319831731 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f518c0cba90 0x7f518c151410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:37.871 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.866+0000 7f518affd640 1 -- 172.21.15.49:0/3319831731 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51780047d0 msgr2=0x7f518c15d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:37.871 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.866+0000 7f518affd640 1 --2- 172.21.15.49:0/3319831731 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51780047d0 0x7f518c15d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:37.871 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.866+0000 7f518affd640 1 -- 172.21.15.49:0/3319831731 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f518c0cc320 con 0x7f518c0cb630 2023-11-06T02:13:37.871 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.866+0000 7f518affd640 1 --2- 172.21.15.49:0/3319831731 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f518c0cb630 0x7f518c150ed0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f5174001940 tx=0x7f5178005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:37.871 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.866+0000 7f5183fff640 1 -- 172.21.15.49:0/3319831731 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51740b7070 con 0x7f518c0cb630 2023-11-06T02:13:37.872 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.866+0000 7f5183fff640 1 -- 172.21.15.49:0/3319831731 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5174005bd0 con 0x7f518c0cb630 2023-11-06T02:13:37.872 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.866+0000 7f5191b54640 1 -- 172.21.15.49:0/3319831731 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f518c0cc600 con 0x7f518c0cb630 2023-11-06T02:13:37.872 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.866+0000 7f5183fff640 1 -- 172.21.15.49:0/3319831731 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51740070d0 con 0x7f518c0cb630 2023-11-06T02:13:37.872 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.866+0000 7f5191b54640 1 -- 172.21.15.49:0/3319831731 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f518c15d5b0 con 0x7f518c0cb630 2023-11-06T02:13:37.872 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.866+0000 7f5191b54640 1 -- 172.21.15.49:0/3319831731 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f518c0b23c0 con 0x7f518c0cb630 2023-11-06T02:13:37.873 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.866+0000 7f5183fff640 1 -- 172.21.15.49:0/3319831731 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f5174007230 con 0x7f518c0cb630 2023-11-06T02:13:37.873 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.866+0000 7f5183fff640 1 --2- 172.21.15.49:0/3319831731 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f516c065a90 0x7f516c067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:37.873 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.866+0000 7f5183fff640 1 -- 172.21.15.49:0/3319831731 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5174120330 con 0x7f518c0cb630 2023-11-06T02:13:37.873 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.866+0000 7f518b7fe640 1 --2- 172.21.15.49:0/3319831731 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f516c065a90 0x7f516c067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:37.873 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.866+0000 7f518b7fe640 1 --2- 172.21.15.49:0/3319831731 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f516c065a90 0x7f516c067f50 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f517c000c10 tx=0x7f517c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:37.873 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:37.870+0000 7f5183fff640 1 -- 172.21.15.49:0/3319831731 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f518c0b23c0 con 0x7f518c0cb630 2023-11-06T02:13:38.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.022+0000 7f5191b54640 1 -- 172.21.15.49:0/3319831731 --> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f518c0ce3e0 con 0x7f516c065a90 2023-11-06T02:13:38.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.046+0000 7f5183fff640 1 -- 172.21.15.49:0/3319831731 <== mgr.4100 v2:172.21.15.150:6832/38104 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+306073 (secure 0 0 0) 0x7f517c001360 con 0x7f516c065a90 2023-11-06T02:13:38.044 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:13:38.045 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2023-11-06T02:13:38.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.050+0000 7f5191b54640 1 -- 172.21.15.49:0/3319831731 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f516c065a90 msgr2=0x7f516c067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:38.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.050+0000 7f5191b54640 1 --2- 172.21.15.49:0/3319831731 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f516c065a90 0x7f516c067f50 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f517c000c10 tx=0x7f517c009040 comp rx=0 tx=0).stop 2023-11-06T02:13:38.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.050+0000 7f5191b54640 1 -- 172.21.15.49:0/3319831731 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f518c0cb630 msgr2=0x7f518c150ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:38.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.050+0000 7f5191b54640 1 --2- 172.21.15.49:0/3319831731 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f518c0cb630 0x7f518c150ed0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f5174001940 tx=0x7f5178005800 comp rx=0 tx=0).stop 2023-11-06T02:13:38.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.050+0000 7f5191b54640 1 -- 172.21.15.49:0/3319831731 shutdown_connections 2023-11-06T02:13:38.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.050+0000 7f5191b54640 1 --2- 172.21.15.49:0/3319831731 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f516c065a90 0x7f516c067f50 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:38.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.050+0000 7f5191b54640 1 --2- 172.21.15.49:0/3319831731 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f518c0cb630 0x7f518c150ed0 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:38.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.050+0000 7f5191b54640 1 --2- 172.21.15.49:0/3319831731 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f518c0cba90 0x7f518c151410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:38.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.050+0000 7f5191b54640 1 --2- 172.21.15.49:0/3319831731 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51780047d0 0x7f518c15d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:38.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.050+0000 7f5191b54640 1 -- 172.21.15.49:0/3319831731 >> 172.21.15.49:0/3319831731 conn(0x7f518c0d2aa0 msgr2=0x7f518c0b8170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:38.049 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.050+0000 7f5191b54640 1 -- 172.21.15.49:0/3319831731 shutdown_connections 2023-11-06T02:13:38.049 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.050+0000 7f5191b54640 1 -- 172.21.15.49:0/3319831731 wait complete. 2023-11-06T02:13:38.061 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":233,"stamp":"2023-11-06T02:13:37.286493+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":20,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":248676,"kb_used_data":8484,"kb_used_omap":0,"kb_used_meta":240128,"kb_avail":754726044,"statfs":{"total":773094113280,"available":772839469056,"internally_reserved":0,"allocated":8687616,"data_stored":4531565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":245891072},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":44,"apply_latency_ms":44,"commit_latency_ns":44000000,"apply_latency_ns":44000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.060378"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.255018+0000","last_change":"2023-11-06T02:13:33.255018+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.255018+0000","last_undegraded":"2023-11-06T02:13:33.255018+0000","last_fullsized":"2023-11-06T02:13:33.255018+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:18:07.485044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.252648+0000","last_change":"2023-11-06T02:13:33.252648+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.252648+0000","last_undegraded":"2023-11-06T02:13:33.252648+0000","last_fullsized":"2023-11-06T02:13:33.252648+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:22:01.334447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.257065+0000","last_change":"2023-11-06T02:13:33.257065+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.257065+0000","last_undegraded":"2023-11-06T02:13:33.257065+0000","last_fullsized":"2023-11-06T02:13:33.257065+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:42:42.388106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.256535+0000","last_change":"2023-11-06T02:13:33.256535+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.256535+0000","last_undegraded":"2023-11-06T02:13:33.256535+0000","last_fullsized":"2023-11-06T02:13:33.256535+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:54:33.444383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.246148+0000","last_change":"2023-11-06T02:13:33.246148+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.246148+0000","last_undegraded":"2023-11-06T02:13:33.246148+0000","last_fullsized":"2023-11-06T02:13:33.246148+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:01:06.093323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.258259+0000","last_change":"2023-11-06T02:13:33.258259+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.258259+0000","last_undegraded":"2023-11-06T02:13:33.258259+0000","last_fullsized":"2023-11-06T02:13:33.258259+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:41:55.941458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.239742+0000","last_change":"2023-11-06T02:13:33.239742+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.239742+0000","last_undegraded":"2023-11-06T02:13:33.239742+0000","last_fullsized":"2023-11-06T02:13:33.239742+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:40:54.464756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.246060+0000","last_change":"2023-11-06T02:13:33.246060+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.246060+0000","last_undegraded":"2023-11-06T02:13:33.246060+0000","last_fullsized":"2023-11-06T02:13:33.246060+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:39.514240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.243570+0000","last_change":"2023-11-06T02:13:33.243570+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.243570+0000","last_undegraded":"2023-11-06T02:13:33.243570+0000","last_fullsized":"2023-11-06T02:13:33.243570+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:02:51.697659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.257271+0000","last_change":"2023-11-06T02:13:33.257271+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.257271+0000","last_undegraded":"2023-11-06T02:13:33.257271+0000","last_fullsized":"2023-11-06T02:13:33.257271+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:12:00.717133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.252080+0000","last_change":"2023-11-06T02:13:33.252080+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.252080+0000","last_undegraded":"2023-11-06T02:13:33.252080+0000","last_fullsized":"2023-11-06T02:13:33.252080+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:41:33.827814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.255251+0000","last_change":"2023-11-06T02:13:33.255251+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.255251+0000","last_undegraded":"2023-11-06T02:13:33.255251+0000","last_fullsized":"2023-11-06T02:13:33.255251+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:21:24.032595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.245690+0000","last_change":"2023-11-06T02:13:33.245690+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.245690+0000","last_undegraded":"2023-11-06T02:13:33.245690+0000","last_fullsized":"2023-11-06T02:13:33.245690+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:42:30.360623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.257567+0000","last_change":"2023-11-06T02:13:33.257567+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.257567+0000","last_undegraded":"2023-11-06T02:13:33.257567+0000","last_fullsized":"2023-11-06T02:13:33.257567+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:41:33.827814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.252478+0000","last_change":"2023-11-06T02:13:33.252478+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.252478+0000","last_undegraded":"2023-11-06T02:13:33.252478+0000","last_fullsized":"2023-11-06T02:13:33.252478+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:28:22.791353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.254366+0000","last_change":"2023-11-06T02:13:33.254366+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.254366+0000","last_undegraded":"2023-11-06T02:13:33.254366+0000","last_fullsized":"2023-11-06T02:13:33.254366+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:21:24.032595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.246654+0000","last_change":"2023-11-06T02:13:33.246654+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.246654+0000","last_undegraded":"2023-11-06T02:13:33.246654+0000","last_fullsized":"2023-11-06T02:13:33.246654+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:54:43.806737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.255469+0000","last_change":"2023-11-06T02:13:33.255469+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.255469+0000","last_undegraded":"2023-11-06T02:13:33.255469+0000","last_fullsized":"2023-11-06T02:13:33.255469+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:17:36.098143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.254869+0000","last_change":"2023-11-06T02:13:33.254869+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.254869+0000","last_undegraded":"2023-11-06T02:13:33.254869+0000","last_fullsized":"2023-11-06T02:13:33.254869+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:22:13.884367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.247005+0000","last_change":"2023-11-06T02:13:33.247005+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.247005+0000","last_undegraded":"2023-11-06T02:13:33.247005+0000","last_fullsized":"2023-11-06T02:13:33.247005+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:28:11.270422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.242891+0000","last_change":"2023-11-06T02:13:33.242891+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.242891+0000","last_undegraded":"2023-11-06T02:13:33.242891+0000","last_fullsized":"2023-11-06T02:13:33.242891+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:43:49.529761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.246368+0000","last_change":"2023-11-06T02:13:33.246368+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.246368+0000","last_undegraded":"2023-11-06T02:13:33.246368+0000","last_fullsized":"2023-11-06T02:13:33.246368+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:19:02.943437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.244217+0000","last_change":"2023-11-06T02:13:33.244217+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.244217+0000","last_undegraded":"2023-11-06T02:13:33.244217+0000","last_fullsized":"2023-11-06T02:13:33.244217+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:17:17.217507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.252017+0000","last_change":"2023-11-06T02:13:33.252017+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.252017+0000","last_undegraded":"2023-11-06T02:13:33.252017+0000","last_fullsized":"2023-11-06T02:13:33.252017+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:41:55.941458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.250038+0000","last_change":"2023-11-06T02:13:33.250038+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.250038+0000","last_undegraded":"2023-11-06T02:13:33.250038+0000","last_fullsized":"2023-11-06T02:13:33.250038+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:22:13.884367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.255314+0000","last_change":"2023-11-06T02:13:33.255314+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.255314+0000","last_undegraded":"2023-11-06T02:13:33.255314+0000","last_fullsized":"2023-11-06T02:13:33.255314+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:18:07.485044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.254181+0000","last_change":"2023-11-06T02:13:33.254181+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.254181+0000","last_undegraded":"2023-11-06T02:13:33.254181+0000","last_fullsized":"2023-11-06T02:13:33.254181+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:00:13.166796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.243183+0000","last_change":"2023-11-06T02:13:33.243183+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.243183+0000","last_undegraded":"2023-11-06T02:13:33.243183+0000","last_fullsized":"2023-11-06T02:13:33.243183+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:16:46.930331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.249877+0000","last_change":"2023-11-06T02:13:33.249877+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.249877+0000","last_undegraded":"2023-11-06T02:13:33.249877+0000","last_fullsized":"2023-11-06T02:13:33.249877+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:20:11.090360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.250528+0000","last_change":"2023-11-06T02:13:33.250528+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.250528+0000","last_undegraded":"2023-11-06T02:13:33.250528+0000","last_fullsized":"2023-11-06T02:13:33.250528+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:34:15.876381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.253284+0000","last_change":"2023-11-06T02:13:33.253284+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.253284+0000","last_undegraded":"2023-11-06T02:13:33.253284+0000","last_fullsized":"2023-11-06T02:13:33.253284+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:14:22.392069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.252628+0000","last_change":"2023-11-06T02:13:33.252628+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.252628+0000","last_undegraded":"2023-11-06T02:13:33.252628+0000","last_fullsized":"2023-11-06T02:13:33.252628+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:24:31.881719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.252017+0000","last_change":"2023-11-06T02:13:33.252017+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.252017+0000","last_undegraded":"2023-11-06T02:13:33.252017+0000","last_fullsized":"2023-11-06T02:13:33.252017+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:46:42.636025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":3,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.235656+0000","last_change":"2023-11-06T02:13:33.235656+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.235656+0000","last_undegraded":"2023-11-06T02:13:33.235656+0000","last_fullsized":"2023-11-06T02:13:33.235656+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:07:13.179013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.249040+0000","last_change":"2023-11-06T02:13:33.249040+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.249040+0000","last_undegraded":"2023-11-06T02:13:33.249040+0000","last_fullsized":"2023-11-06T02:13:33.249040+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:23:55.083630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":3,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.256359+0000","last_change":"2023-11-06T02:13:33.256359+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.256359+0000","last_undegraded":"2023-11-06T02:13:33.256359+0000","last_fullsized":"2023-11-06T02:13:33.256359+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:33:14.271873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":["7","6"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.246368+0000","last_change":"2023-11-06T02:13:33.246368+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.246368+0000","last_undegraded":"2023-11-06T02:13:33.246368+0000","last_fullsized":"2023-11-06T02:13:33.246368+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:34:15.876381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":4,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.264127+0000","last_change":"2023-11-06T02:13:33.257252+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.264127+0000","last_undegraded":"2023-11-06T02:13:33.264127+0000","last_fullsized":"2023-11-06T02:13:33.264127+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:35:42.092353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":["7","0"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.241683+0000","last_change":"2023-11-06T02:13:33.241683+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.241683+0000","last_undegraded":"2023-11-06T02:13:33.241683+0000","last_fullsized":"2023-11-06T02:13:33.241683+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:14:21.400766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.277611+0000","last_change":"2023-11-06T02:13:33.267790+0000","last_active":"2023-11-06T02:13:33.277611+0000","last_peered":"2023-11-06T02:13:33.277611+0000","last_clean":"2023-11-06T02:13:33.277611+0000","last_became_active":"2023-11-06T02:13:33.267049+0000","last_became_peered":"2023-11-06T02:13:33.267049+0000","last_unstale":"2023-11-06T02:13:33.277611+0000","last_undegraded":"2023-11-06T02:13:33.277611+0000","last_fullsized":"2023-11-06T02:13:33.277611+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:44:56.393504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.249993+0000","last_change":"2023-11-06T02:13:33.249993+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.249993+0000","last_undegraded":"2023-11-06T02:13:33.249993+0000","last_fullsized":"2023-11-06T02:13:33.249993+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:11:19.807848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":3,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.240593+0000","last_change":"2023-11-06T02:13:33.240593+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.240593+0000","last_undegraded":"2023-11-06T02:13:33.240593+0000","last_fullsized":"2023-11-06T02:13:33.240593+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:33:31.177122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":["4","7"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.249061+0000","last_change":"2023-11-06T02:13:33.249061+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.249061+0000","last_undegraded":"2023-11-06T02:13:33.249061+0000","last_fullsized":"2023-11-06T02:13:33.249061+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:16:52.758810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.277263+0000","last_change":"2023-11-06T02:13:33.272642+0000","last_active":"2023-11-06T02:13:33.277263+0000","last_peered":"2023-11-06T02:13:33.277263+0000","last_clean":"2023-11-06T02:13:33.277263+0000","last_became_active":"2023-11-06T02:13:33.271888+0000","last_became_peered":"2023-11-06T02:13:33.271888+0000","last_unstale":"2023-11-06T02:13:33.277263+0000","last_undegraded":"2023-11-06T02:13:33.277263+0000","last_fullsized":"2023-11-06T02:13:33.277263+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:51:47.363705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.242755+0000","last_change":"2023-11-06T02:13:33.242755+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.242755+0000","last_undegraded":"2023-11-06T02:13:33.242755+0000","last_fullsized":"2023-11-06T02:13:33.242755+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:16:36.406664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":3,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.243828+0000","last_change":"2023-11-06T02:13:33.243828+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.243828+0000","last_undegraded":"2023-11-06T02:13:33.243828+0000","last_fullsized":"2023-11-06T02:13:33.243828+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:54:32.434908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":["0","7"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.245643+0000","last_change":"2023-11-06T02:13:33.245643+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.245643+0000","last_undegraded":"2023-11-06T02:13:33.245643+0000","last_fullsized":"2023-11-06T02:13:33.245643+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:33:50.179908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":3,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.235521+0000","last_change":"2023-11-06T02:13:33.235521+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.235521+0000","last_undegraded":"2023-11-06T02:13:33.235521+0000","last_fullsized":"2023-11-06T02:13:33.235521+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:39:03.018907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":["2","4"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.249061+0000","last_change":"2023-11-06T02:13:33.249061+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.249061+0000","last_undegraded":"2023-11-06T02:13:33.249061+0000","last_fullsized":"2023-11-06T02:13:33.249061+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:32:02.459972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":3,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.259605+0000","last_change":"2023-11-06T02:13:33.259605+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.259605+0000","last_undegraded":"2023-11-06T02:13:33.259605+0000","last_fullsized":"2023-11-06T02:13:33.259605+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:53:46.349156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":["7","6"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.253727+0000","last_change":"2023-11-06T02:13:33.253727+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.253727+0000","last_undegraded":"2023-11-06T02:13:33.253727+0000","last_fullsized":"2023-11-06T02:13:33.253727+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:00:13.166796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.283810+0000","last_change":"2023-11-06T02:13:33.281331+0000","last_active":"2023-11-06T02:13:33.283810+0000","last_peered":"2023-11-06T02:13:33.283810+0000","last_clean":"2023-11-06T02:13:33.283810+0000","last_became_active":"2023-11-06T02:13:33.280630+0000","last_became_peered":"2023-11-06T02:13:33.280630+0000","last_unstale":"2023-11-06T02:13:33.283810+0000","last_undegraded":"2023-11-06T02:13:33.283810+0000","last_fullsized":"2023-11-06T02:13:33.283810+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:17:16.208032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.256899+0000","last_change":"2023-11-06T02:13:33.256899+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.256899+0000","last_undegraded":"2023-11-06T02:13:33.256899+0000","last_fullsized":"2023-11-06T02:13:33.256899+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:45:05.474573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":3,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.235632+0000","last_change":"2023-11-06T02:13:33.235632+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.235632+0000","last_undegraded":"2023-11-06T02:13:33.235632+0000","last_fullsized":"2023-11-06T02:13:33.235632+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:55:36.788472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.251869+0000","last_change":"2023-11-06T02:13:33.251869+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.251869+0000","last_undegraded":"2023-11-06T02:13:33.251869+0000","last_fullsized":"2023-11-06T02:13:33.251869+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:33:50.179908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.277447+0000","last_change":"2023-11-06T02:13:33.265304+0000","last_active":"2023-11-06T02:13:33.277447+0000","last_peered":"2023-11-06T02:13:33.277447+0000","last_clean":"2023-11-06T02:13:33.277447+0000","last_became_active":"2023-11-06T02:13:33.264524+0000","last_became_peered":"2023-11-06T02:13:33.264524+0000","last_unstale":"2023-11-06T02:13:33.277447+0000","last_undegraded":"2023-11-06T02:13:33.277447+0000","last_fullsized":"2023-11-06T02:13:33.277447+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:39:01.059129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.253815+0000","last_change":"2023-11-06T02:13:33.253815+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.253815+0000","last_undegraded":"2023-11-06T02:13:33.253815+0000","last_fullsized":"2023-11-06T02:13:33.253815+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:17:59.025543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.280385+0000","last_change":"2023-11-06T02:13:33.273211+0000","last_active":"2023-11-06T02:13:33.280385+0000","last_peered":"2023-11-06T02:13:33.280385+0000","last_clean":"2023-11-06T02:13:33.280385+0000","last_became_active":"2023-11-06T02:13:33.272067+0000","last_became_peered":"2023-11-06T02:13:33.272067+0000","last_unstale":"2023-11-06T02:13:33.280385+0000","last_undegraded":"2023-11-06T02:13:33.280385+0000","last_fullsized":"2023-11-06T02:13:33.280385+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:16:35.397189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.240661+0000","last_change":"2023-11-06T02:13:33.240661+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.240661+0000","last_undegraded":"2023-11-06T02:13:33.240661+0000","last_fullsized":"2023-11-06T02:13:33.240661+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:17:36.098143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":3,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.256327+0000","last_change":"2023-11-06T02:13:33.256327+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.256327+0000","last_undegraded":"2023-11-06T02:13:33.256327+0000","last_fullsized":"2023-11-06T02:13:33.256327+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:53:59.721435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":["7","2"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.241911+0000","last_change":"2023-11-06T02:13:33.241911+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.241911+0000","last_undegraded":"2023-11-06T02:13:33.241911+0000","last_fullsized":"2023-11-06T02:13:33.241911+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:10:52.936432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":3,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.235562+0000","last_change":"2023-11-06T02:13:33.235562+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.235562+0000","last_undegraded":"2023-11-06T02:13:33.235562+0000","last_fullsized":"2023-11-06T02:13:33.235562+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:30:40.192944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.244479+0000","last_change":"2023-11-06T02:13:33.244479+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.244479+0000","last_undegraded":"2023-11-06T02:13:33.244479+0000","last_fullsized":"2023-11-06T02:13:33.244479+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:21:05.055643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":3,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.247845+0000","last_change":"2023-11-06T02:13:33.247845+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.247845+0000","last_undegraded":"2023-11-06T02:13:33.247845+0000","last_fullsized":"2023-11-06T02:13:33.247845+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:17:58.016068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":["0","6"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.241033+0000","last_change":"2023-11-06T02:13:33.241033+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.241033+0000","last_undegraded":"2023-11-06T02:13:33.241033+0000","last_fullsized":"2023-11-06T02:13:33.241033+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:54:00.730910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.279807+0000","last_change":"2023-11-06T02:13:33.272867+0000","last_active":"2023-11-06T02:13:33.279807+0000","last_peered":"2023-11-06T02:13:33.279807+0000","last_clean":"2023-11-06T02:13:33.279807+0000","last_became_active":"2023-11-06T02:13:33.272136+0000","last_became_peered":"2023-11-06T02:13:33.272136+0000","last_unstale":"2023-11-06T02:13:33.279807+0000","last_undegraded":"2023-11-06T02:13:33.279807+0000","last_fullsized":"2023-11-06T02:13:33.279807+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:02:50.688184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.238294+0000","last_change":"2023-11-06T02:13:33.238294+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.238294+0000","last_undegraded":"2023-11-06T02:13:33.238294+0000","last_fullsized":"2023-11-06T02:13:33.238294+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:02:47.316294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":3,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.244338+0000","last_change":"2023-11-06T02:13:33.244338+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.244338+0000","last_undegraded":"2023-11-06T02:13:33.244338+0000","last_fullsized":"2023-11-06T02:13:33.244338+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:11:59.707658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.245156+0000","last_change":"2023-11-06T02:13:33.245156+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.245156+0000","last_undegraded":"2023-11-06T02:13:33.245156+0000","last_fullsized":"2023-11-06T02:13:33.245156+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:07:57.793830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":3,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.258159+0000","last_change":"2023-11-06T02:13:33.258159+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.258159+0000","last_undegraded":"2023-11-06T02:13:33.258159+0000","last_fullsized":"2023-11-06T02:13:33.258159+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:16:35.397189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":["7","5"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.248850+0000","last_change":"2023-11-06T02:13:33.248850+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.248850+0000","last_undegraded":"2023-11-06T02:13:33.248850+0000","last_fullsized":"2023-11-06T02:13:33.248850+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:22:47.360905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":3,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.256366+0000","last_change":"2023-11-06T02:13:33.256366+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.256366+0000","last_undegraded":"2023-11-06T02:13:33.256366+0000","last_fullsized":"2023-11-06T02:13:33.256366+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:12:35.766676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":["7","6"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.244536+0000","last_change":"2023-11-06T02:13:33.244536+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.244536+0000","last_undegraded":"2023-11-06T02:13:33.244536+0000","last_fullsized":"2023-11-06T02:13:33.244536+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:42:42.388106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.277757+0000","last_change":"2023-11-06T02:13:33.267817+0000","last_active":"2023-11-06T02:13:33.277757+0000","last_peered":"2023-11-06T02:13:33.277757+0000","last_clean":"2023-11-06T02:13:33.277757+0000","last_became_active":"2023-11-06T02:13:33.267091+0000","last_became_peered":"2023-11-06T02:13:33.267091+0000","last_unstale":"2023-11-06T02:13:33.277757+0000","last_undegraded":"2023-11-06T02:13:33.277757+0000","last_fullsized":"2023-11-06T02:13:33.277757+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:36:21.262835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.239368+0000","last_change":"2023-11-06T02:13:33.239368+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.239368+0000","last_undegraded":"2023-11-06T02:13:33.239368+0000","last_fullsized":"2023-11-06T02:13:33.239368+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:31:19.083004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":3,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.235609+0000","last_change":"2023-11-06T02:13:33.235609+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.235609+0000","last_undegraded":"2023-11-06T02:13:33.235609+0000","last_fullsized":"2023-11-06T02:13:33.235609+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:19:20.386051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.254957+0000","last_change":"2023-11-06T02:13:33.254957+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.254957+0000","last_undegraded":"2023-11-06T02:13:33.254957+0000","last_fullsized":"2023-11-06T02:13:33.254957+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:02:47.316294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.277367+0000","last_change":"2023-11-06T02:13:33.264127+0000","last_active":"2023-11-06T02:13:33.277367+0000","last_peered":"2023-11-06T02:13:33.277367+0000","last_clean":"2023-11-06T02:13:33.277367+0000","last_became_active":"2023-11-06T02:13:33.263418+0000","last_became_peered":"2023-11-06T02:13:33.263418+0000","last_unstale":"2023-11-06T02:13:33.277367+0000","last_undegraded":"2023-11-06T02:13:33.277367+0000","last_fullsized":"2023-11-06T02:13:33.277367+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:04:53.601947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.253013+0000","last_change":"2023-11-06T02:13:33.253013+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.253013+0000","last_undegraded":"2023-11-06T02:13:33.253013+0000","last_fullsized":"2023-11-06T02:13:33.253013+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:31:19.083004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":3,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.251425+0000","last_change":"2023-11-06T02:13:33.251425+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.251425+0000","last_undegraded":"2023-11-06T02:13:33.251425+0000","last_fullsized":"2023-11-06T02:13:33.251425+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:13:44.482660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":["1","3"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.252952+0000","last_change":"2023-11-06T02:13:33.252952+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.252952+0000","last_undegraded":"2023-11-06T02:13:33.252952+0000","last_fullsized":"2023-11-06T02:13:33.252952+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:12:00.717133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":3,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.253882+0000","last_change":"2023-11-06T02:13:33.253882+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.253882+0000","last_undegraded":"2023-11-06T02:13:33.253882+0000","last_fullsized":"2023-11-06T02:13:33.253882+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:16:51.749335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":["3","6"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":216,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-06T02:13:32.232683+0000","last_change":"2023-11-06T02:08:20.523427+0000","last_active":"2023-11-06T02:13:32.232683+0000","last_peered":"2023-11-06T02:13:32.232683+0000","last_clean":"2023-11-06T02:13:32.232683+0000","last_became_active":"2023-11-06T02:08:20.521089+0000","last_became_peered":"2023-11-06T02:08:20.521089+0000","last_unstale":"2023-11-06T02:13:32.232683+0000","last_undegraded":"2023-11-06T02:13:32.232683+0000","last_fullsized":"2023-11-06T02:13:32.232683+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:19.252348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:19.252348+0000","last_clean_scrub_stamp":"2023-11-06T02:08:19.252348+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:30:15.566397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":62,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-06T02:13:32.246890+0000","last_change":"2023-11-06T02:08:26.059502+0000","last_active":"2023-11-06T02:13:32.246890+0000","last_peered":"2023-11-06T02:13:32.246890+0000","last_clean":"2023-11-06T02:13:32.246890+0000","last_became_active":"2023-11-06T02:08:26.058612+0000","last_became_peered":"2023-11-06T02:08:26.058612+0000","last_unstale":"2023-11-06T02:13:32.246890+0000","last_undegraded":"2023-11-06T02:13:32.246890+0000","last_fullsized":"2023-11-06T02:13:32.246890+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:10.626021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.250595+0000","last_change":"2023-11-06T02:13:33.250595+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.250595+0000","last_undegraded":"2023-11-06T02:13:33.250595+0000","last_fullsized":"2023-11-06T02:13:33.250595+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:46:01.922659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":3,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.246452+0000","last_change":"2023-11-06T02:13:33.246452+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.246452+0000","last_undegraded":"2023-11-06T02:13:33.246452+0000","last_fullsized":"2023-11-06T02:13:33.246452+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:11:51.109403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":["3","6"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":63,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-06T02:13:32.232564+0000","last_change":"2023-11-06T02:08:26.060406+0000","last_active":"2023-11-06T02:13:32.232564+0000","last_peered":"2023-11-06T02:13:32.232564+0000","last_clean":"2023-11-06T02:13:32.232564+0000","last_became_active":"2023-11-06T02:08:26.059262+0000","last_became_peered":"2023-11-06T02:08:26.059262+0000","last_unstale":"2023-11-06T02:13:32.232564+0000","last_undegraded":"2023-11-06T02:13:32.232564+0000","last_fullsized":"2023-11-06T02:13:32.232564+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:10.626021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.243210+0000","last_change":"2023-11-06T02:13:33.243210+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.243210+0000","last_undegraded":"2023-11-06T02:13:33.243210+0000","last_fullsized":"2023-11-06T02:13:33.243210+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:33:32.186597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":3,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.244333+0000","last_change":"2023-11-06T02:13:33.244333+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.244333+0000","last_undegraded":"2023-11-06T02:13:33.244333+0000","last_fullsized":"2023-11-06T02:13:33.244333+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:51:52.147063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":["3","6"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":63,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-06T02:13:32.236950+0000","last_change":"2023-11-06T02:08:26.559477+0000","last_active":"2023-11-06T02:13:32.236950+0000","last_peered":"2023-11-06T02:13:32.236950+0000","last_clean":"2023-11-06T02:13:32.236950+0000","last_became_active":"2023-11-06T02:08:26.559202+0000","last_became_peered":"2023-11-06T02:08:26.559202+0000","last_unstale":"2023-11-06T02:13:32.236950+0000","last_undegraded":"2023-11-06T02:13:32.236950+0000","last_fullsized":"2023-11-06T02:13:32.236950+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:55:04.552404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.249368+0000","last_change":"2023-11-06T02:13:33.249368+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.249368+0000","last_undegraded":"2023-11-06T02:13:33.249368+0000","last_fullsized":"2023-11-06T02:13:33.249368+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:54:11.389133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":3,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.245263+0000","last_change":"2023-11-06T02:13:33.245263+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.245263+0000","last_undegraded":"2023-11-06T02:13:33.245263+0000","last_fullsized":"2023-11-06T02:13:33.245263+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:38:14.971699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":62,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-06T02:13:32.246949+0000","last_change":"2023-11-06T02:08:26.059546+0000","last_active":"2023-11-06T02:13:32.246949+0000","last_peered":"2023-11-06T02:13:32.246949+0000","last_clean":"2023-11-06T02:13:32.246949+0000","last_became_active":"2023-11-06T02:08:26.058806+0000","last_became_peered":"2023-11-06T02:08:26.058806+0000","last_unstale":"2023-11-06T02:13:32.246949+0000","last_undegraded":"2023-11-06T02:13:32.246949+0000","last_fullsized":"2023-11-06T02:13:32.246949+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:23.468510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.242981+0000","last_change":"2023-11-06T02:13:33.242981+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.242981+0000","last_undegraded":"2023-11-06T02:13:33.242981+0000","last_fullsized":"2023-11-06T02:13:33.242981+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:36:12.474443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":3,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.236266+0000","last_change":"2023-11-06T02:13:33.236266+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.236266+0000","last_undegraded":"2023-11-06T02:13:33.236266+0000","last_fullsized":"2023-11-06T02:13:33.236266+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:30:05.141853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":["2","4"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":62,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-06T02:13:32.229548+0000","last_change":"2023-11-06T02:08:26.436463+0000","last_active":"2023-11-06T02:13:32.229548+0000","last_peered":"2023-11-06T02:13:32.229548+0000","last_clean":"2023-11-06T02:13:32.229548+0000","last_became_active":"2023-11-06T02:08:26.436075+0000","last_became_peered":"2023-11-06T02:08:26.436075+0000","last_unstale":"2023-11-06T02:13:32.229548+0000","last_undegraded":"2023-11-06T02:13:32.229548+0000","last_fullsized":"2023-11-06T02:13:32.229548+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:23.468510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.246317+0000","last_change":"2023-11-06T02:13:33.246317+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.246317+0000","last_undegraded":"2023-11-06T02:13:33.246317+0000","last_fullsized":"2023-11-06T02:13:33.246317+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:25:19.608926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":3,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.251742+0000","last_change":"2023-11-06T02:13:33.251742+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.251742+0000","last_undegraded":"2023-11-06T02:13:33.251742+0000","last_fullsized":"2023-11-06T02:13:33.251742+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:23:42.695735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":63,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-06T02:13:32.239073+0000","last_change":"2023-11-06T02:08:26.436607+0000","last_active":"2023-11-06T02:13:32.239073+0000","last_peered":"2023-11-06T02:13:32.239073+0000","last_clean":"2023-11-06T02:13:32.239073+0000","last_became_active":"2023-11-06T02:08:26.435924+0000","last_became_peered":"2023-11-06T02:08:26.435924+0000","last_unstale":"2023-11-06T02:13:32.239073+0000","last_undegraded":"2023-11-06T02:13:32.239073+0000","last_fullsized":"2023-11-06T02:13:32.239073+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:23.468510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.244818+0000","last_change":"2023-11-06T02:13:33.244818+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.244818+0000","last_undegraded":"2023-11-06T02:13:33.244818+0000","last_fullsized":"2023-11-06T02:13:33.244818+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:22:01.334447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":4,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.269809+0000","last_change":"2023-11-06T02:13:33.260450+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.269809+0000","last_undegraded":"2023-11-06T02:13:33.269809+0000","last_fullsized":"2023-11-06T02:13:33.269809+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:51:47.363705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":["7","1"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":62,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-06T02:13:32.229549+0000","last_change":"2023-11-06T02:08:26.057076+0000","last_active":"2023-11-06T02:13:32.229549+0000","last_peered":"2023-11-06T02:13:32.229549+0000","last_clean":"2023-11-06T02:13:32.229549+0000","last_became_active":"2023-11-06T02:08:26.056419+0000","last_became_peered":"2023-11-06T02:08:26.056419+0000","last_unstale":"2023-11-06T02:13:32.229549+0000","last_undegraded":"2023-11-06T02:13:32.229549+0000","last_fullsized":"2023-11-06T02:13:32.229549+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:10.626021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.242736+0000","last_change":"2023-11-06T02:13:33.242736+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.242736+0000","last_undegraded":"2023-11-06T02:13:33.242736+0000","last_fullsized":"2023-11-06T02:13:33.242736+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:38:31.470172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.265697+0000","last_change":"2023-11-06T02:13:33.265697+0000","last_active":"2023-11-06T02:13:33.265697+0000","last_peered":"2023-11-06T02:13:33.265697+0000","last_clean":"2023-11-06T02:13:33.265697+0000","last_became_active":"2023-11-06T02:13:33.264914+0000","last_became_peered":"2023-11-06T02:13:33.264914+0000","last_unstale":"2023-11-06T02:13:33.265697+0000","last_undegraded":"2023-11-06T02:13:33.265697+0000","last_fullsized":"2023-11-06T02:13:33.265697+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:17:35.088668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":71,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-06T02:13:32.237181+0000","last_change":"2023-11-06T02:08:26.058596+0000","last_active":"2023-11-06T02:13:32.237181+0000","last_peered":"2023-11-06T02:13:32.237181+0000","last_clean":"2023-11-06T02:13:32.237181+0000","last_became_active":"2023-11-06T02:08:26.057015+0000","last_became_peered":"2023-11-06T02:08:26.057015+0000","last_unstale":"2023-11-06T02:13:32.237181+0000","last_undegraded":"2023-11-06T02:13:32.237181+0000","last_fullsized":"2023-11-06T02:13:32.237181+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:55:04.552404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574914,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32544,"kb_used_data":1496,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339296,"statfs":{"total":96636764160,"available":96603439104,"internally_reserved":0,"allocated":1531904,"data_stored":1009349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:13:26 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.86599999999999999,"15min":0.85399999999999998},"min":{"1min":0.59199999999999997,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.2190000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.62},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.91200000000000003,"15min":0.91100000000000003},"min":{"1min":0.57299999999999995,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.214,"5min":1.506,"15min":1.506},"last":1}]},{"osd":1,"last update":"Mon Nov 6 02:13:26 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.83599999999999997,"15min":0.84599999999999997},"min":{"1min":0.66400000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":0.94799999999999995},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.94099999999999995,"15min":0.90600000000000003},"min":{"1min":0.64300000000000002,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.681,"5min":1.681,"15min":1.681},"last":0.75600000000000001}]},{"osd":2,"last update":"Mon Nov 6 02:13:26 2023","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.96599999999999997,"15min":0.95799999999999996},"min":{"1min":0.66200000000000003,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.22,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.94899999999999995,"15min":0.98299999999999998},"min":{"1min":0.71899999999999997,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.2789999999999999,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.70499999999999996}]},{"osd":3,"last update":"Mon Nov 6 02:13:26 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.89400000000000002,"15min":0.88100000000000001},"min":{"1min":0.67500000000000004,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.196,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.78300000000000003},{"interface":"front","average":{"1min":1.069,"5min":1.046,"15min":1.0660000000000001},"min":{"1min":0.86399999999999999,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.2430000000000001,"5min":1.526,"15min":1.526},"last":0.91300000000000003}]},{"osd":4,"last update":"Mon Nov 6 02:13:26 2023","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.92300000000000004,"15min":0.94599999999999995},"min":{"1min":0.59699999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.2350000000000001,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.874},{"interface":"front","average":{"1min":1.0129999999999999,"5min":1.004,"15min":1.0309999999999999},"min":{"1min":0.625,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.377,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":1.052}]},{"osd":5,"last update":"Mon Nov 6 02:13:26 2023","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.038,"15min":1.0649999999999999},"min":{"1min":0.76300000000000001,"5min":0.503,"15min":0.503},"max":{"1min":1.409,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":1.032},{"interface":"front","average":{"1min":1.046,"5min":1.0629999999999999,"15min":1.111},"min":{"1min":0.75900000000000001,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.3109999999999999,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":1.087}]},{"osd":6,"last update":"Mon Nov 6 02:13:26 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.1399999999999999,"15min":1.181},"min":{"1min":0.59399999999999997,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.278,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":1.1639999999999999},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.091,"15min":1.135},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.4570000000000001,"5min":1.74,"15min":1.74},"last":1.133}]}]},{"osd":6,"up_from":13,"seq":55834574913,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31964,"kb_used_data":916,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339876,"statfs":{"total":96636764160,"available":96604033024,"internally_reserved":0,"allocated":937984,"data_stored":418981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:13:28 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.86699999999999999,"15min":0.86099999999999999},"min":{"1min":0.58899999999999997,"5min":0.495,"15min":0.495},"max":{"1min":1.202,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.98099999999999998},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.874,"15min":0.82799999999999996},"min":{"1min":0.63,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.2649999999999999,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.56399999999999995}]},{"osd":1,"last update":"Mon Nov 6 02:13:28 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.84999999999999998,"15min":0.86699999999999999},"min":{"1min":0.63500000000000001,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.244,"5min":1.244,"15min":1.244},"last":1.1639999999999999},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.89600000000000002,"15min":0.88100000000000001},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.624}]},{"osd":2,"last update":"Mon Nov 6 02:13:28 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.88700000000000001,"15min":0.89500000000000002},"min":{"1min":0.52100000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.206,"5min":1.252,"15min":1.252},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.90000000000000002,"15min":0.92000000000000004},"min":{"1min":0.61699999999999999,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.411,"5min":1.411,"15min":1.411},"last":0.82899999999999996}]},{"osd":3,"last update":"Mon Nov 6 02:13:28 2023","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.93899999999999995,"15min":0.96099999999999997},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.212,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.0549999999999999,"5min":0.99299999999999999,"15min":0.998},"min":{"1min":0.77300000000000002,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.381,"5min":1.381,"15min":1.381},"last":1.101}]},{"osd":4,"last update":"Mon Nov 6 02:13:28 2023","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.92600000000000005,"15min":0.97099999999999997},"min":{"1min":0.67500000000000004,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.302,"5min":1.474,"15min":1.474},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.0469999999999999,"5min":0.98199999999999998,"15min":1.02},"min":{"1min":0.77100000000000002,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.556,"5min":1.556,"15min":1.556},"last":1.2050000000000001}]},{"osd":5,"last update":"Mon Nov 6 02:13:28 2023","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":0.97199999999999998,"15min":1.0089999999999999},"min":{"1min":0.73099999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":1.4650000000000001},{"interface":"front","average":{"1min":1.129,"5min":1.0720000000000001,"15min":1.1000000000000001},"min":{"1min":0.94899999999999995,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.587,"5min":1.587,"15min":1.587},"last":1.3999999999999999}]},{"osd":7,"last update":"Mon Nov 6 02:13:28 2023","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.087,"15min":1.107},"min":{"1min":0.95899999999999996,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.615,"5min":1.615,"15min":1.615},"last":1.363},{"interface":"front","average":{"1min":1.109,"5min":1.0629999999999999,"15min":1.131},"min":{"1min":0.83499999999999996,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":1.2969999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574914,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":916,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":937984,"data_stored":418981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:13:30 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.86899999999999999,"15min":0.84199999999999997},"min":{"1min":0.51700000000000002,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.1930000000000001,"5min":1.381,"15min":1.381},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.86599999999999999,"15min":0.84999999999999998},"min":{"1min":0.58199999999999996,"5min":0.503,"15min":0.503},"max":{"1min":1.1399999999999999,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":1.1399999999999999}]},{"osd":1,"last update":"Mon Nov 6 02:13:30 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.89600000000000002,"15min":0.86499999999999999},"min":{"1min":0.69799999999999995,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.177,"5min":1.331,"15min":1.331},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.88100000000000001,"15min":0.83799999999999997},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.139,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.71399999999999997}]},{"osd":2,"last update":"Mon Nov 6 02:13:30 2023","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.93400000000000005,"15min":0.94199999999999995},"min":{"1min":0.68000000000000005,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.298,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.872},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.88400000000000001,"15min":0.82099999999999995},"min":{"1min":0.56899999999999995,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.2869999999999999,"5min":1.377,"15min":1.377},"last":0.98399999999999999}]},{"osd":3,"last update":"Mon Nov 6 02:13:30 2023","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.95399999999999996,"15min":0.93100000000000005},"min":{"1min":0.77100000000000002,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.2210000000000001,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":1.091},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.997,"15min":1.0529999999999999},"min":{"1min":0.67900000000000005,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.224,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":0.78900000000000003}]},{"osd":4,"last update":"Mon Nov 6 02:13:30 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.89500000000000002,"15min":0.90900000000000003},"min":{"1min":0.58099999999999996,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.1779999999999999,"5min":1.276,"15min":1.276},"last":0.63700000000000001},{"interface":"front","average":{"1min":1.006,"5min":0.99299999999999999,"15min":1.016},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.254,"5min":1.554,"15min":1.554},"last":0.94799999999999995}]},{"osd":6,"last update":"Mon Nov 6 02:13:30 2023","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":1.0089999999999999,"15min":1.048},"min":{"1min":0.71399999999999997,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.2569999999999999,"5min":1.454,"15min":1.454},"last":1.016},{"interface":"front","average":{"1min":0.98699999999999999,"5min":1.022,"15min":1.034},"min":{"1min":0.75,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.1930000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.76000000000000001}]},{"osd":7,"last update":"Mon Nov 6 02:13:30 2023","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.0609999999999999,"15min":1.071},"min":{"1min":0.77900000000000003,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.2330000000000001,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":1.048},{"interface":"front","average":{"1min":1.109,"5min":1.1120000000000001,"15min":1.1399999999999999},"min":{"1min":0.84599999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.3799999999999999,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":1.0740000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574914,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":916,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":937984,"data_stored":418981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:13:27 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.877,"15min":0.84199999999999997},"min":{"1min":0.65100000000000002,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.0840000000000001,"5min":1.343,"15min":1.343},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.83299999999999996,"15min":0.79100000000000004},"min":{"1min":0.626,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.1699999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.77000000000000002}]},{"osd":1,"last update":"Mon Nov 6 02:13:27 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.86799999999999999,"15min":0.85199999999999998},"min":{"1min":0.57299999999999995,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.0329999999999999,"5min":1.462,"15min":1.462},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.93300000000000005,"15min":0.90500000000000003},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.224,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.59699999999999998}]},{"osd":2,"last update":"Mon Nov 6 02:13:27 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.93799999999999994,"15min":0.99199999999999999},"min":{"1min":0.70299999999999996,"5min":0.503,"15min":0.503},"max":{"1min":1.173,"5min":1.615,"15min":1.615},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.95499999999999996,"15min":0.93500000000000005},"min":{"1min":0.72499999999999998,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.163,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.97899999999999998}]},{"osd":3,"last update":"Mon Nov 6 02:13:27 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.88200000000000001,"15min":0.90600000000000003},"min":{"1min":0.61499999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.216,"5min":1.52,"15min":1.52},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.997,"15min":1.0449999999999999},"min":{"1min":0.66900000000000004,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.2050000000000001,"5min":1.595,"15min":1.595},"last":0.80900000000000005}]},{"osd":5,"last update":"Mon Nov 6 02:13:27 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.99099999999999999,"15min":1.0309999999999999},"min":{"1min":0.64500000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.339,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":1.0800000000000001},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.97599999999999998,"15min":1.0289999999999999},"min":{"1min":0.68899999999999995,"5min":0.629,"15min":0.629},"max":{"1min":1.161,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":0.86699999999999999}]},{"osd":6,"last update":"Mon Nov 6 02:13:27 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":1.008,"15min":1.0649999999999999},"min":{"1min":0.69599999999999995,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.268,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":1.03},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.1120000000000001,"15min":1.169},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.2789999999999999,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":1.129}]},{"osd":7,"last update":"Mon Nov 6 02:13:27 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.0489999999999999,"15min":1.079},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.294,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.93899999999999995},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.133,"15min":1.204},"min":{"1min":0.77500000000000002,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.3149999999999999,"5min":1.7410000000000001,"15min":1.7410000000000001},"last":1.1060000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574913,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31964,"kb_used_data":916,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339876,"statfs":{"total":96636764160,"available":96604033024,"internally_reserved":0,"allocated":937984,"data_stored":418981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:13:28 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.63200000000000001,"15min":0.64200000000000002},"min":{"1min":0.40799999999999997,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":1.198,"5min":1.198,"15min":1.198},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.64000000000000001,"15min":0.63},"min":{"1min":0.38700000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.72099999999999997}]},{"osd":1,"last update":"Mon Nov 6 02:13:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.61699999999999999,"15min":0.627},"min":{"1min":0.34599999999999997,"5min":0.248,"15min":0.248},"max":{"1min":0.92600000000000005,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.73999999999999999,"15min":0.73599999999999999},"min":{"1min":0.53000000000000003,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.60099999999999998}]},{"osd":2,"last update":"Mon Nov 6 02:13:28 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.71899999999999997,"15min":0.72899999999999998},"min":{"1min":0.45400000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.107,"5min":1.139,"15min":1.139},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.80800000000000005,"15min":0.81799999999999995},"min":{"1min":0.48099999999999998,"5min":0.308,"15min":0.308},"max":{"1min":1.212,"5min":1.405,"15min":1.405},"last":0.75700000000000001}]},{"osd":4,"last update":"Mon Nov 6 02:13:28 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.92000000000000004,"15min":0.92600000000000005},"min":{"1min":0.72599999999999998,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.0820000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.95299999999999996,"15min":0.98299999999999998},"min":{"1min":0.65100000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.1459999999999999,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.97399999999999998}]},{"osd":5,"last update":"Mon Nov 6 02:13:28 2023","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.98899999999999999,"15min":1.0149999999999999},"min":{"1min":0.72799999999999998,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.333,"5min":1.375,"15min":1.375},"last":0.81299999999999994},{"interface":"front","average":{"1min":1.091,"5min":1.0389999999999999,"15min":1.0669999999999999},"min":{"1min":0.627,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.4890000000000001,"5min":2.21,"15min":2.21},"last":1.0269999999999999}]},{"osd":6,"last update":"Mon Nov 6 02:13:28 2023","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.016,"15min":1.022},"min":{"1min":0.61299999999999999,"5min":0.439,"15min":0.439},"max":{"1min":1.3740000000000001,"5min":1.395,"15min":1.395},"last":0.78900000000000003},{"interface":"front","average":{"1min":1.036,"5min":1.04,"15min":1.0609999999999999},"min":{"1min":0.54600000000000004,"5min":0.375,"15min":0.375},"max":{"1min":1.3069999999999999,"5min":1.554,"15min":1.554},"last":1.107}]},{"osd":7,"last update":"Mon Nov 6 02:13:28 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.1080000000000001,"15min":1.1240000000000001},"min":{"1min":0.66800000000000004,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.421,"5min":2.2829999999999999,"15min":2.2829999999999999},"last":0.94099999999999995},{"interface":"front","average":{"1min":1.091,"5min":1.0940000000000001,"15min":1.1020000000000001},"min":{"1min":0.70999999999999996,"5min":0.497,"15min":0.497},"max":{"1min":1.4490000000000001,"5min":1.504,"15min":1.504},"last":1.048}]}]},{"osd":2,"up_from":13,"seq":55834574913,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31964,"kb_used_data":916,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339876,"statfs":{"total":96636764160,"available":96604033024,"internally_reserved":0,"allocated":937984,"data_stored":418981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:13:24 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.57699999999999996,"15min":0.52500000000000002},"min":{"1min":0.33100000000000002,"5min":0.252,"15min":0.252},"max":{"1min":0.85699999999999998,"5min":1.079,"15min":1.079},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.625,"15min":0.56699999999999995},"min":{"1min":0.505,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.1060000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":0.56999999999999995}]},{"osd":1,"last update":"Mon Nov 6 02:13:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.63500000000000001,"15min":0.58499999999999996},"min":{"1min":0.38800000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.88300000000000001,"5min":1.48,"15min":1.48},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.70599999999999996,"15min":0.61399999999999999},"min":{"1min":0.377,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.282,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":0.70499999999999996}]},{"osd":3,"last update":"Mon Nov 6 02:13:24 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.70099999999999996,"15min":0.63200000000000001},"min":{"1min":0.48199999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.98999999999999999,"5min":1.155,"15min":1.155},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.749,"15min":0.67700000000000005},"min":{"1min":0.41999999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.81499999999999995}]},{"osd":4,"last update":"Mon Nov 6 02:13:24 2023","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.90900000000000003,"15min":0.83499999999999996},"min":{"1min":0.58499999999999996,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.2150000000000001,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.93500000000000005,"15min":0.92100000000000004},"min":{"1min":0.73099999999999998,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.3120000000000001,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":1.0589999999999999}]},{"osd":5,"last update":"Mon Nov 6 02:13:24 2023","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":0.96799999999999997,"15min":0.95099999999999996},"min":{"1min":0.75,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.2270000000000001,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":0.93999999999999995},{"interface":"front","average":{"1min":1,"5min":0.97099999999999997,"15min":0.94599999999999995},"min":{"1min":0.628,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.3109999999999999,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.96399999999999997}]},{"osd":6,"last update":"Mon Nov 6 02:13:24 2023","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":0.97099999999999997,"15min":0.94899999999999995},"min":{"1min":0.59899999999999998,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.3129999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.0229999999999999,"15min":1.0049999999999999},"min":{"1min":0.73299999999999998,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.2769999999999999,"5min":1.677,"15min":1.677},"last":1.1479999999999999}]},{"osd":7,"last update":"Mon Nov 6 02:13:24 2023","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.026,"15min":0.99199999999999999},"min":{"1min":0.72199999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.4079999999999999,"5min":1.659,"15min":1.659},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.123,"5min":1.0820000000000001,"15min":1.0549999999999999},"min":{"1min":0.79300000000000004,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.3819999999999999,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":1.087}]}]},{"osd":1,"up_from":13,"seq":55834574914,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31964,"kb_used_data":916,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339876,"statfs":{"total":96636764160,"available":96604033024,"internally_reserved":0,"allocated":937984,"data_stored":418981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:13:27 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.63900000000000001,"15min":0.63},"min":{"1min":0.35899999999999999,"5min":0.224,"15min":0.224},"max":{"1min":1.343,"5min":1.603,"15min":1.603},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.71899999999999997,"15min":0.67400000000000004},"min":{"1min":0.38200000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":2.161,"5min":2.161,"15min":2.161},"last":1.081}]},{"osd":2,"last update":"Mon Nov 6 02:13:27 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.65400000000000003,"15min":0.63200000000000001},"min":{"1min":0.26500000000000001,"5min":0.253,"15min":0.253},"max":{"1min":1.256,"5min":1.649,"15min":1.649},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.753,"15min":0.747},"min":{"1min":0.41099999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":1.5249999999999999,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.88400000000000001}]},{"osd":3,"last update":"Mon Nov 6 02:13:27 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.81299999999999994,"15min":0.80600000000000005},"min":{"1min":0.49099999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.6319999999999999,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":1.131},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.81299999999999994,"15min":0.81100000000000005},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.1799999999999999,"5min":1.8149999999999999,"15min":1.8149999999999999},"last":0.94199999999999995}]},{"osd":4,"last update":"Mon Nov 6 02:13:27 2023","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":0.97899999999999998,"15min":0.98799999999999999},"min":{"1min":0.78500000000000003,"5min":0.504,"15min":0.504},"max":{"1min":2.2759999999999998,"5min":2.2759999999999998,"15min":2.2759999999999998},"last":1.196},{"interface":"front","average":{"1min":1.0740000000000001,"5min":0.97699999999999998,"15min":0.98799999999999999},"min":{"1min":0.64200000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":2.9180000000000001,"5min":2.9180000000000001,"15min":2.9180000000000001},"last":1.0049999999999999}]},{"osd":5,"last update":"Mon Nov 6 02:13:27 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":0.98099999999999998,"15min":0.97099999999999997},"min":{"1min":0.68100000000000005,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":2.7469999999999999,"5min":2.7469999999999999,"15min":2.7469999999999999},"last":1.1659999999999999},{"interface":"front","average":{"1min":1.1579999999999999,"5min":1.069,"15min":1.075},"min":{"1min":0.81100000000000005,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.921,"5min":1.921,"15min":1.921},"last":1.044}]},{"osd":6,"last update":"Mon Nov 6 02:13:27 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.113,"15min":1.1060000000000001},"min":{"1min":0.83099999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":3.5710000000000002,"5min":3.5710000000000002,"15min":3.5710000000000002},"last":1.278},{"interface":"front","average":{"1min":1.157,"5min":1.0629999999999999,"15min":1.0720000000000001},"min":{"1min":0.60299999999999998,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":2.843,"5min":2.843,"15min":2.843},"last":0.83499999999999996}]},{"osd":7,"last update":"Mon Nov 6 02:13:27 2023","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.1180000000000001,"15min":1.1200000000000001},"min":{"1min":0.89600000000000002,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":2.0009999999999999,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":1.2330000000000001},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.1639999999999999,"15min":1.1619999999999999},"min":{"1min":0.91100000000000003,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":3.4790000000000001,"5min":3.4790000000000001,"15min":3.4790000000000001},"last":1.3520000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574914,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32540,"kb_used_data":1492,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339300,"statfs":{"total":96636764160,"available":96603443200,"internally_reserved":0,"allocated":1527808,"data_stored":1008330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 02:13:26 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.61499999999999999,"15min":0.60299999999999998},"min":{"1min":0.53100000000000003,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.0289999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.755,"5min":0.66900000000000004,"15min":0.64700000000000002},"min":{"1min":0.53800000000000003,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.079,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.46300000000000002}]},{"osd":2,"last update":"Mon Nov 6 02:13:26 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.68600000000000005,"15min":0.67200000000000004},"min":{"1min":0.42999999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.1080000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.98299999999999998},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.69799999999999995,"15min":0.68200000000000005},"min":{"1min":0.57299999999999995,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.75700000000000001}]},{"osd":3,"last update":"Mon Nov 6 02:13:26 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.67200000000000004,"15min":0.64300000000000002},"min":{"1min":0.52500000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.1659999999999999,"5min":1.429,"15min":1.429},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.87,"5min":0.78300000000000003,"15min":0.748},"min":{"1min":0.69999999999999996,"5min":0.317,"15min":0.317},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.89000000000000001}]},{"osd":4,"last update":"Mon Nov 6 02:13:26 2023","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.90200000000000002,"15min":0.85199999999999998},"min":{"1min":0.76800000000000002,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.1779999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":1.0589999999999999,"5min":0.96099999999999997,"15min":0.94499999999999995},"min":{"1min":0.78900000000000003,"5min":0.5,"15min":0.5},"max":{"1min":1.292,"5min":1.387,"15min":1.387},"last":1.137}]},{"osd":5,"last update":"Mon Nov 6 02:13:26 2023","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":0.95699999999999996,"15min":0.95099999999999996},"min":{"1min":0.81799999999999995,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.006,"5min":0.93400000000000005,"15min":0.89200000000000002},"min":{"1min":0.81100000000000005,"5min":0.496,"15min":0.496},"max":{"1min":1.2549999999999999,"5min":1.448,"15min":1.448},"last":1.101}]},{"osd":6,"last update":"Mon Nov 6 02:13:26 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.0389999999999999,"15min":1.0369999999999999},"min":{"1min":0.73899999999999999,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.385,"5min":1.589,"15min":1.589},"last":1.236},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.0269999999999999,"15min":1.0089999999999999},"min":{"1min":0.76000000000000001,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.419,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":1.1779999999999999}]},{"osd":7,"last update":"Mon Nov 6 02:13:26 2023","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.026,"15min":1.0209999999999999},"min":{"1min":0.76400000000000001,"5min":0.504,"15min":0.504},"max":{"1min":1.504,"5min":1.504,"15min":1.504},"last":1.022},{"interface":"front","average":{"1min":1.073,"5min":0.998,"15min":1.0049999999999999},"min":{"1min":0.74299999999999999,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.3999999999999999,"5min":1.556,"15min":1.556},"last":0.873}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T02:13:38.067 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T02:13:38.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.230+0000 7f1c7432c640 1 Processor -- start 2023-11-06T02:13:38.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.230+0000 7f1c7432c640 1 -- start start 2023-11-06T02:13:38.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.230+0000 7f1c7432c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f1c6c0cb010 0x7f1c6c0c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:38.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.230+0000 7f1c7432c640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c6c0cab80 con 0x7f1c6c15ab50 2023-11-06T02:13:38.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.230+0000 7f1c7432c640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c6c15af30 con 0x7f1c6c0b0790 2023-11-06T02:13:38.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.230+0000 7f1c7432c640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c6c0cb5b0 con 0x7f1c6c0cb010 2023-11-06T02:13:38.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.230+0000 7f1c728c9640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f1c6c0cb010 0x7f1c6c0c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:38.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.230+0000 7f1c720c8640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f1c6c15ab50 0x7f1c6c0b0080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:60638/0 (socket says 172.21.15.49:60638) 2023-11-06T02:13:38.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.230+0000 7f1c720c8640 1 -- 172.21.15.49:0/180902957 learned_addr learned my addr 172.21.15.49:0/180902957 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:13:38.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.230+0000 7f1c728c9640 1 -- 172.21.15.49:0/180902957 >> v1:172.21.15.150:6789/0 conn(0x7f1c6c0b0790 legacy=0x7f1c6c0c2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:38.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.230+0000 7f1c728c9640 1 -- 172.21.15.49:0/180902957 >> v1:172.21.15.49:6789/0 conn(0x7f1c6c15ab50 legacy=0x7f1c6c0b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:38.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.230+0000 7f1c728c9640 1 -- 172.21.15.49:0/180902957 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c6c0cb890 con 0x7f1c6c0cb010 2023-11-06T02:13:38.229 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.230+0000 7f1c728c9640 1 --2- 172.21.15.49:0/180902957 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c6c0cb010 0x7f1c6c0c9bb0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f1c64009d90 tx=0x7f1c6401c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef020c1122c7f736 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:38.229 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.234+0000 7f1c710c6640 1 -- 172.21.15.49:0/180902957 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c640a59a0 con 0x7f1c6c0cb010 2023-11-06T02:13:38.229 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.234+0000 7f1c710c6640 1 -- 172.21.15.49:0/180902957 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c640a5b00 con 0x7f1c6c0cb010 2023-11-06T02:13:38.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.234+0000 7f1c710c6640 1 -- 172.21.15.49:0/180902957 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c640a5e90 con 0x7f1c6c0cb010 2023-11-06T02:13:38.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.234+0000 7f1c7432c640 1 -- 172.21.15.49:0/180902957 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c6c0cb010 msgr2=0x7f1c6c0c9bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:38.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.234+0000 7f1c7432c640 1 --2- 172.21.15.49:0/180902957 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c6c0cb010 0x7f1c6c0c9bb0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f1c64009d90 tx=0x7f1c6401c8e0 comp rx=0 tx=0).stop 2023-11-06T02:13:38.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.234+0000 7f1c7432c640 1 -- 172.21.15.49:0/180902957 shutdown_connections 2023-11-06T02:13:38.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.234+0000 7f1c7432c640 1 --2- 172.21.15.49:0/180902957 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c6c0cb010 0x7f1c6c0c9bb0 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:38.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.234+0000 7f1c7432c640 1 -- 172.21.15.49:0/180902957 >> 172.21.15.49:0/180902957 conn(0x7f1c6c06c0a0 msgr2=0x7f1c6c06c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:38.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.234+0000 7f1c7432c640 1 -- 172.21.15.49:0/180902957 shutdown_connections 2023-11-06T02:13:38.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.234+0000 7f1c7432c640 1 -- 172.21.15.49:0/180902957 wait complete. 2023-11-06T02:13:38.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.234+0000 7f1c7432c640 1 Processor -- start 2023-11-06T02:13:38.234 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.234+0000 7f1c7432c640 1 -- start start 2023-11-06T02:13:38.234 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.234+0000 7f1c7432c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1c6c0b0790 0x7f1c6c06ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:38.234 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.234+0000 7f1c7432c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c6c0cb010 0x7f1c6c0704e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:38.234 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.234+0000 7f1c7432c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c6c15ab50 0x7f1c6c151e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:38.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.234+0000 7f1c7432c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c6c0c6c10 con 0x7f1c6c15ab50 2023-11-06T02:13:38.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.234+0000 7f1c7432c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c6c0c6290 con 0x7f1c6c0cb010 2023-11-06T02:13:38.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.234+0000 7f1c7432c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c6c0c6550 con 0x7f1c6c0b0790 2023-11-06T02:13:38.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.234+0000 7f1c728c9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c6c15ab50 0x7f1c6c151e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:38.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.234+0000 7f1c728c9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c6c15ab50 0x7f1c6c151e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51370/0 (socket says 172.21.15.49:51370) 2023-11-06T02:13:38.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.234+0000 7f1c728c9640 1 -- 172.21.15.49:0/2860432400 learned_addr learned my addr 172.21.15.49:0/2860432400 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:38.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.234+0000 7f1c718c7640 1 --2- 172.21.15.49:0/2860432400 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c6c0cb010 0x7f1c6c0704e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:38.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.234+0000 7f1c720c8640 1 --2- 172.21.15.49:0/2860432400 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1c6c0b0790 0x7f1c6c06ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:38.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.234+0000 7f1c728c9640 1 -- 172.21.15.49:0/2860432400 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1c6c0b0790 msgr2=0x7f1c6c06ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:38.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.234+0000 7f1c728c9640 1 --2- 172.21.15.49:0/2860432400 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1c6c0b0790 0x7f1c6c06ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:38.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.234+0000 7f1c728c9640 1 -- 172.21.15.49:0/2860432400 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c6c0cb010 msgr2=0x7f1c6c0704e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:38.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.234+0000 7f1c728c9640 1 --2- 172.21.15.49:0/2860432400 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c6c0cb010 0x7f1c6c0704e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:38.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.234+0000 7f1c728c9640 1 -- 172.21.15.49:0/2860432400 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c64009a70 con 0x7f1c6c15ab50 2023-11-06T02:13:38.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.234+0000 7f1c720c8640 1 --2- 172.21.15.49:0/2860432400 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1c6c0b0790 0x7f1c6c06ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:13:38.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.234+0000 7f1c728c9640 1 --2- 172.21.15.49:0/2860432400 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c6c15ab50 0x7f1c6c151e30 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f1c6401a510 tx=0x7f1c64019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:38.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.234+0000 7f1c62ffd640 1 -- 172.21.15.49:0/2860432400 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c640ade40 con 0x7f1c6c15ab50 2023-11-06T02:13:38.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.234+0000 7f1c62ffd640 1 -- 172.21.15.49:0/2860432400 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c640b9450 con 0x7f1c6c15ab50 2023-11-06T02:13:38.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.234+0000 7f1c7432c640 1 -- 172.21.15.49:0/2860432400 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c6c0c6830 con 0x7f1c6c15ab50 2023-11-06T02:13:38.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.234+0000 7f1c62ffd640 1 -- 172.21.15.49:0/2860432400 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c640b97e0 con 0x7f1c6c15ab50 2023-11-06T02:13:38.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.234+0000 7f1c7432c640 1 -- 172.21.15.49:0/2860432400 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c6c0a8250 con 0x7f1c6c15ab50 2023-11-06T02:13:38.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.238+0000 7f1c7432c640 1 -- 172.21.15.49:0/2860432400 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c6c0b0de0 con 0x7f1c6c15ab50 2023-11-06T02:13:38.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.238+0000 7f1c62ffd640 1 -- 172.21.15.49:0/2860432400 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f1c640b9940 con 0x7f1c6c15ab50 2023-11-06T02:13:38.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.238+0000 7f1c62ffd640 1 --2- 172.21.15.49:0/2860432400 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1c50065a60 0x7f1c50067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:38.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.238+0000 7f1c62ffd640 1 -- 172.21.15.49:0/2860432400 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1c6411ecc0 con 0x7f1c6c15ab50 2023-11-06T02:13:38.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.238+0000 7f1c720c8640 1 --2- 172.21.15.49:0/2860432400 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1c50065a60 0x7f1c50067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:38.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.238+0000 7f1c720c8640 1 --2- 172.21.15.49:0/2860432400 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1c50065a60 0x7f1c50067f20 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f1c680010f0 tx=0x7f1c68009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:38.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.242+0000 7f1c62ffd640 1 -- 172.21.15.49:0/2860432400 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f1c64125080 con 0x7f1c6c15ab50 2023-11-06T02:13:38.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.390+0000 7f1c7432c640 1 -- 172.21.15.49:0/2860432400 --> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1c6c0cbe50 con 0x7f1c50065a60 2023-11-06T02:13:38.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.414+0000 7f1c62ffd640 1 -- 172.21.15.49:0/2860432400 <== mgr.4100 v2:172.21.15.150:6832/38104 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+306073 (secure 0 0 0) 0x7f1c68001c00 con 0x7f1c50065a60 2023-11-06T02:13:38.412 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:13:38.415 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2023-11-06T02:13:38.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.418+0000 7f1c7432c640 1 -- 172.21.15.49:0/2860432400 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1c50065a60 msgr2=0x7f1c50067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:38.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.418+0000 7f1c7432c640 1 --2- 172.21.15.49:0/2860432400 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1c50065a60 0x7f1c50067f20 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f1c680010f0 tx=0x7f1c68009040 comp rx=0 tx=0).stop 2023-11-06T02:13:38.417 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.418+0000 7f1c7432c640 1 -- 172.21.15.49:0/2860432400 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c6c15ab50 msgr2=0x7f1c6c151e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:38.417 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.418+0000 7f1c7432c640 1 --2- 172.21.15.49:0/2860432400 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c6c15ab50 0x7f1c6c151e30 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f1c6401a510 tx=0x7f1c64019680 comp rx=0 tx=0).stop 2023-11-06T02:13:38.417 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.418+0000 7f1c7432c640 1 -- 172.21.15.49:0/2860432400 shutdown_connections 2023-11-06T02:13:38.417 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.418+0000 7f1c7432c640 1 --2- 172.21.15.49:0/2860432400 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1c50065a60 0x7f1c50067f20 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:38.417 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.418+0000 7f1c7432c640 1 --2- 172.21.15.49:0/2860432400 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c6c15ab50 0x7f1c6c151e30 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:38.417 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.418+0000 7f1c7432c640 1 --2- 172.21.15.49:0/2860432400 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c6c0cb010 0x7f1c6c0704e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:38.418 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.418+0000 7f1c7432c640 1 --2- 172.21.15.49:0/2860432400 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1c6c0b0790 0x7f1c6c06ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:38.418 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.418+0000 7f1c7432c640 1 -- 172.21.15.49:0/2860432400 >> 172.21.15.49:0/2860432400 conn(0x7f1c6c06c0a0 msgr2=0x7f1c6c0c2710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:38.418 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.418+0000 7f1c7432c640 1 -- 172.21.15.49:0/2860432400 shutdown_connections 2023-11-06T02:13:38.418 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.418+0000 7f1c7432c640 1 -- 172.21.15.49:0/2860432400 wait complete. 2023-11-06T02:13:38.430 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":233,"stamp":"2023-11-06T02:13:37.286493+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":20,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":248676,"kb_used_data":8484,"kb_used_omap":0,"kb_used_meta":240128,"kb_avail":754726044,"statfs":{"total":773094113280,"available":772839469056,"internally_reserved":0,"allocated":8687616,"data_stored":4531565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":245891072},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":44,"apply_latency_ms":44,"commit_latency_ns":44000000,"apply_latency_ns":44000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.060378"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.255018+0000","last_change":"2023-11-06T02:13:33.255018+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.255018+0000","last_undegraded":"2023-11-06T02:13:33.255018+0000","last_fullsized":"2023-11-06T02:13:33.255018+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:18:07.485044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.252648+0000","last_change":"2023-11-06T02:13:33.252648+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.252648+0000","last_undegraded":"2023-11-06T02:13:33.252648+0000","last_fullsized":"2023-11-06T02:13:33.252648+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:22:01.334447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.257065+0000","last_change":"2023-11-06T02:13:33.257065+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.257065+0000","last_undegraded":"2023-11-06T02:13:33.257065+0000","last_fullsized":"2023-11-06T02:13:33.257065+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:42:42.388106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.256535+0000","last_change":"2023-11-06T02:13:33.256535+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.256535+0000","last_undegraded":"2023-11-06T02:13:33.256535+0000","last_fullsized":"2023-11-06T02:13:33.256535+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:54:33.444383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.246148+0000","last_change":"2023-11-06T02:13:33.246148+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.246148+0000","last_undegraded":"2023-11-06T02:13:33.246148+0000","last_fullsized":"2023-11-06T02:13:33.246148+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:01:06.093323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.258259+0000","last_change":"2023-11-06T02:13:33.258259+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.258259+0000","last_undegraded":"2023-11-06T02:13:33.258259+0000","last_fullsized":"2023-11-06T02:13:33.258259+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:41:55.941458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.239742+0000","last_change":"2023-11-06T02:13:33.239742+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.239742+0000","last_undegraded":"2023-11-06T02:13:33.239742+0000","last_fullsized":"2023-11-06T02:13:33.239742+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:40:54.464756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.246060+0000","last_change":"2023-11-06T02:13:33.246060+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.246060+0000","last_undegraded":"2023-11-06T02:13:33.246060+0000","last_fullsized":"2023-11-06T02:13:33.246060+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:39.514240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.243570+0000","last_change":"2023-11-06T02:13:33.243570+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.243570+0000","last_undegraded":"2023-11-06T02:13:33.243570+0000","last_fullsized":"2023-11-06T02:13:33.243570+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:02:51.697659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.257271+0000","last_change":"2023-11-06T02:13:33.257271+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.257271+0000","last_undegraded":"2023-11-06T02:13:33.257271+0000","last_fullsized":"2023-11-06T02:13:33.257271+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:12:00.717133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.252080+0000","last_change":"2023-11-06T02:13:33.252080+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.252080+0000","last_undegraded":"2023-11-06T02:13:33.252080+0000","last_fullsized":"2023-11-06T02:13:33.252080+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:41:33.827814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.255251+0000","last_change":"2023-11-06T02:13:33.255251+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.255251+0000","last_undegraded":"2023-11-06T02:13:33.255251+0000","last_fullsized":"2023-11-06T02:13:33.255251+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:21:24.032595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.245690+0000","last_change":"2023-11-06T02:13:33.245690+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.245690+0000","last_undegraded":"2023-11-06T02:13:33.245690+0000","last_fullsized":"2023-11-06T02:13:33.245690+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:42:30.360623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.257567+0000","last_change":"2023-11-06T02:13:33.257567+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.257567+0000","last_undegraded":"2023-11-06T02:13:33.257567+0000","last_fullsized":"2023-11-06T02:13:33.257567+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:41:33.827814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.252478+0000","last_change":"2023-11-06T02:13:33.252478+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.252478+0000","last_undegraded":"2023-11-06T02:13:33.252478+0000","last_fullsized":"2023-11-06T02:13:33.252478+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:28:22.791353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.254366+0000","last_change":"2023-11-06T02:13:33.254366+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.254366+0000","last_undegraded":"2023-11-06T02:13:33.254366+0000","last_fullsized":"2023-11-06T02:13:33.254366+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:21:24.032595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.246654+0000","last_change":"2023-11-06T02:13:33.246654+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.246654+0000","last_undegraded":"2023-11-06T02:13:33.246654+0000","last_fullsized":"2023-11-06T02:13:33.246654+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:54:43.806737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.255469+0000","last_change":"2023-11-06T02:13:33.255469+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.255469+0000","last_undegraded":"2023-11-06T02:13:33.255469+0000","last_fullsized":"2023-11-06T02:13:33.255469+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:17:36.098143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.254869+0000","last_change":"2023-11-06T02:13:33.254869+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.254869+0000","last_undegraded":"2023-11-06T02:13:33.254869+0000","last_fullsized":"2023-11-06T02:13:33.254869+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:22:13.884367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.247005+0000","last_change":"2023-11-06T02:13:33.247005+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.247005+0000","last_undegraded":"2023-11-06T02:13:33.247005+0000","last_fullsized":"2023-11-06T02:13:33.247005+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:28:11.270422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.242891+0000","last_change":"2023-11-06T02:13:33.242891+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.242891+0000","last_undegraded":"2023-11-06T02:13:33.242891+0000","last_fullsized":"2023-11-06T02:13:33.242891+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:43:49.529761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.246368+0000","last_change":"2023-11-06T02:13:33.246368+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.246368+0000","last_undegraded":"2023-11-06T02:13:33.246368+0000","last_fullsized":"2023-11-06T02:13:33.246368+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:19:02.943437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.244217+0000","last_change":"2023-11-06T02:13:33.244217+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.244217+0000","last_undegraded":"2023-11-06T02:13:33.244217+0000","last_fullsized":"2023-11-06T02:13:33.244217+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:17:17.217507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.252017+0000","last_change":"2023-11-06T02:13:33.252017+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.252017+0000","last_undegraded":"2023-11-06T02:13:33.252017+0000","last_fullsized":"2023-11-06T02:13:33.252017+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:41:55.941458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.250038+0000","last_change":"2023-11-06T02:13:33.250038+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.250038+0000","last_undegraded":"2023-11-06T02:13:33.250038+0000","last_fullsized":"2023-11-06T02:13:33.250038+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:22:13.884367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.255314+0000","last_change":"2023-11-06T02:13:33.255314+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.255314+0000","last_undegraded":"2023-11-06T02:13:33.255314+0000","last_fullsized":"2023-11-06T02:13:33.255314+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:18:07.485044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.254181+0000","last_change":"2023-11-06T02:13:33.254181+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.254181+0000","last_undegraded":"2023-11-06T02:13:33.254181+0000","last_fullsized":"2023-11-06T02:13:33.254181+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:00:13.166796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.243183+0000","last_change":"2023-11-06T02:13:33.243183+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.243183+0000","last_undegraded":"2023-11-06T02:13:33.243183+0000","last_fullsized":"2023-11-06T02:13:33.243183+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:16:46.930331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.249877+0000","last_change":"2023-11-06T02:13:33.249877+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.249877+0000","last_undegraded":"2023-11-06T02:13:33.249877+0000","last_fullsized":"2023-11-06T02:13:33.249877+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:20:11.090360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.250528+0000","last_change":"2023-11-06T02:13:33.250528+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.250528+0000","last_undegraded":"2023-11-06T02:13:33.250528+0000","last_fullsized":"2023-11-06T02:13:33.250528+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:34:15.876381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.253284+0000","last_change":"2023-11-06T02:13:33.253284+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.253284+0000","last_undegraded":"2023-11-06T02:13:33.253284+0000","last_fullsized":"2023-11-06T02:13:33.253284+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:14:22.392069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.252628+0000","last_change":"2023-11-06T02:13:33.252628+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.252628+0000","last_undegraded":"2023-11-06T02:13:33.252628+0000","last_fullsized":"2023-11-06T02:13:33.252628+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:24:31.881719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.252017+0000","last_change":"2023-11-06T02:13:33.252017+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.252017+0000","last_undegraded":"2023-11-06T02:13:33.252017+0000","last_fullsized":"2023-11-06T02:13:33.252017+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:46:42.636025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":3,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.235656+0000","last_change":"2023-11-06T02:13:33.235656+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.235656+0000","last_undegraded":"2023-11-06T02:13:33.235656+0000","last_fullsized":"2023-11-06T02:13:33.235656+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:07:13.179013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.249040+0000","last_change":"2023-11-06T02:13:33.249040+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.249040+0000","last_undegraded":"2023-11-06T02:13:33.249040+0000","last_fullsized":"2023-11-06T02:13:33.249040+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:23:55.083630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":3,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.256359+0000","last_change":"2023-11-06T02:13:33.256359+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.256359+0000","last_undegraded":"2023-11-06T02:13:33.256359+0000","last_fullsized":"2023-11-06T02:13:33.256359+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:33:14.271873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":["7","6"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.246368+0000","last_change":"2023-11-06T02:13:33.246368+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.246368+0000","last_undegraded":"2023-11-06T02:13:33.246368+0000","last_fullsized":"2023-11-06T02:13:33.246368+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:34:15.876381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":4,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.264127+0000","last_change":"2023-11-06T02:13:33.257252+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.264127+0000","last_undegraded":"2023-11-06T02:13:33.264127+0000","last_fullsized":"2023-11-06T02:13:33.264127+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:35:42.092353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":["7","0"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.241683+0000","last_change":"2023-11-06T02:13:33.241683+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.241683+0000","last_undegraded":"2023-11-06T02:13:33.241683+0000","last_fullsized":"2023-11-06T02:13:33.241683+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:14:21.400766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.277611+0000","last_change":"2023-11-06T02:13:33.267790+0000","last_active":"2023-11-06T02:13:33.277611+0000","last_peered":"2023-11-06T02:13:33.277611+0000","last_clean":"2023-11-06T02:13:33.277611+0000","last_became_active":"2023-11-06T02:13:33.267049+0000","last_became_peered":"2023-11-06T02:13:33.267049+0000","last_unstale":"2023-11-06T02:13:33.277611+0000","last_undegraded":"2023-11-06T02:13:33.277611+0000","last_fullsized":"2023-11-06T02:13:33.277611+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:44:56.393504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.249993+0000","last_change":"2023-11-06T02:13:33.249993+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.249993+0000","last_undegraded":"2023-11-06T02:13:33.249993+0000","last_fullsized":"2023-11-06T02:13:33.249993+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:11:19.807848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":3,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.240593+0000","last_change":"2023-11-06T02:13:33.240593+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.240593+0000","last_undegraded":"2023-11-06T02:13:33.240593+0000","last_fullsized":"2023-11-06T02:13:33.240593+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:33:31.177122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":["4","7"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.249061+0000","last_change":"2023-11-06T02:13:33.249061+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.249061+0000","last_undegraded":"2023-11-06T02:13:33.249061+0000","last_fullsized":"2023-11-06T02:13:33.249061+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:16:52.758810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.277263+0000","last_change":"2023-11-06T02:13:33.272642+0000","last_active":"2023-11-06T02:13:33.277263+0000","last_peered":"2023-11-06T02:13:33.277263+0000","last_clean":"2023-11-06T02:13:33.277263+0000","last_became_active":"2023-11-06T02:13:33.271888+0000","last_became_peered":"2023-11-06T02:13:33.271888+0000","last_unstale":"2023-11-06T02:13:33.277263+0000","last_undegraded":"2023-11-06T02:13:33.277263+0000","last_fullsized":"2023-11-06T02:13:33.277263+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:51:47.363705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.242755+0000","last_change":"2023-11-06T02:13:33.242755+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.242755+0000","last_undegraded":"2023-11-06T02:13:33.242755+0000","last_fullsized":"2023-11-06T02:13:33.242755+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:16:36.406664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":3,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.243828+0000","last_change":"2023-11-06T02:13:33.243828+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.243828+0000","last_undegraded":"2023-11-06T02:13:33.243828+0000","last_fullsized":"2023-11-06T02:13:33.243828+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:54:32.434908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":["0","7"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.245643+0000","last_change":"2023-11-06T02:13:33.245643+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.245643+0000","last_undegraded":"2023-11-06T02:13:33.245643+0000","last_fullsized":"2023-11-06T02:13:33.245643+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:33:50.179908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":3,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.235521+0000","last_change":"2023-11-06T02:13:33.235521+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.235521+0000","last_undegraded":"2023-11-06T02:13:33.235521+0000","last_fullsized":"2023-11-06T02:13:33.235521+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:39:03.018907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":["2","4"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.249061+0000","last_change":"2023-11-06T02:13:33.249061+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.249061+0000","last_undegraded":"2023-11-06T02:13:33.249061+0000","last_fullsized":"2023-11-06T02:13:33.249061+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:32:02.459972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":3,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.259605+0000","last_change":"2023-11-06T02:13:33.259605+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.259605+0000","last_undegraded":"2023-11-06T02:13:33.259605+0000","last_fullsized":"2023-11-06T02:13:33.259605+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:53:46.349156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":["7","6"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.253727+0000","last_change":"2023-11-06T02:13:33.253727+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.253727+0000","last_undegraded":"2023-11-06T02:13:33.253727+0000","last_fullsized":"2023-11-06T02:13:33.253727+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:00:13.166796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.283810+0000","last_change":"2023-11-06T02:13:33.281331+0000","last_active":"2023-11-06T02:13:33.283810+0000","last_peered":"2023-11-06T02:13:33.283810+0000","last_clean":"2023-11-06T02:13:33.283810+0000","last_became_active":"2023-11-06T02:13:33.280630+0000","last_became_peered":"2023-11-06T02:13:33.280630+0000","last_unstale":"2023-11-06T02:13:33.283810+0000","last_undegraded":"2023-11-06T02:13:33.283810+0000","last_fullsized":"2023-11-06T02:13:33.283810+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:17:16.208032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.256899+0000","last_change":"2023-11-06T02:13:33.256899+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.256899+0000","last_undegraded":"2023-11-06T02:13:33.256899+0000","last_fullsized":"2023-11-06T02:13:33.256899+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:45:05.474573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":3,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.235632+0000","last_change":"2023-11-06T02:13:33.235632+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.235632+0000","last_undegraded":"2023-11-06T02:13:33.235632+0000","last_fullsized":"2023-11-06T02:13:33.235632+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:55:36.788472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.251869+0000","last_change":"2023-11-06T02:13:33.251869+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.251869+0000","last_undegraded":"2023-11-06T02:13:33.251869+0000","last_fullsized":"2023-11-06T02:13:33.251869+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:33:50.179908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.277447+0000","last_change":"2023-11-06T02:13:33.265304+0000","last_active":"2023-11-06T02:13:33.277447+0000","last_peered":"2023-11-06T02:13:33.277447+0000","last_clean":"2023-11-06T02:13:33.277447+0000","last_became_active":"2023-11-06T02:13:33.264524+0000","last_became_peered":"2023-11-06T02:13:33.264524+0000","last_unstale":"2023-11-06T02:13:33.277447+0000","last_undegraded":"2023-11-06T02:13:33.277447+0000","last_fullsized":"2023-11-06T02:13:33.277447+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:39:01.059129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.253815+0000","last_change":"2023-11-06T02:13:33.253815+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.253815+0000","last_undegraded":"2023-11-06T02:13:33.253815+0000","last_fullsized":"2023-11-06T02:13:33.253815+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:17:59.025543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.280385+0000","last_change":"2023-11-06T02:13:33.273211+0000","last_active":"2023-11-06T02:13:33.280385+0000","last_peered":"2023-11-06T02:13:33.280385+0000","last_clean":"2023-11-06T02:13:33.280385+0000","last_became_active":"2023-11-06T02:13:33.272067+0000","last_became_peered":"2023-11-06T02:13:33.272067+0000","last_unstale":"2023-11-06T02:13:33.280385+0000","last_undegraded":"2023-11-06T02:13:33.280385+0000","last_fullsized":"2023-11-06T02:13:33.280385+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:16:35.397189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.240661+0000","last_change":"2023-11-06T02:13:33.240661+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.240661+0000","last_undegraded":"2023-11-06T02:13:33.240661+0000","last_fullsized":"2023-11-06T02:13:33.240661+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:17:36.098143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":3,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.256327+0000","last_change":"2023-11-06T02:13:33.256327+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.256327+0000","last_undegraded":"2023-11-06T02:13:33.256327+0000","last_fullsized":"2023-11-06T02:13:33.256327+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:53:59.721435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":["7","2"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.241911+0000","last_change":"2023-11-06T02:13:33.241911+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.241911+0000","last_undegraded":"2023-11-06T02:13:33.241911+0000","last_fullsized":"2023-11-06T02:13:33.241911+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:10:52.936432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":3,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.235562+0000","last_change":"2023-11-06T02:13:33.235562+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.235562+0000","last_undegraded":"2023-11-06T02:13:33.235562+0000","last_fullsized":"2023-11-06T02:13:33.235562+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:30:40.192944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.244479+0000","last_change":"2023-11-06T02:13:33.244479+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.244479+0000","last_undegraded":"2023-11-06T02:13:33.244479+0000","last_fullsized":"2023-11-06T02:13:33.244479+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:21:05.055643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":3,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.247845+0000","last_change":"2023-11-06T02:13:33.247845+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.247845+0000","last_undegraded":"2023-11-06T02:13:33.247845+0000","last_fullsized":"2023-11-06T02:13:33.247845+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:17:58.016068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":["0","6"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.241033+0000","last_change":"2023-11-06T02:13:33.241033+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.241033+0000","last_undegraded":"2023-11-06T02:13:33.241033+0000","last_fullsized":"2023-11-06T02:13:33.241033+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:54:00.730910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.279807+0000","last_change":"2023-11-06T02:13:33.272867+0000","last_active":"2023-11-06T02:13:33.279807+0000","last_peered":"2023-11-06T02:13:33.279807+0000","last_clean":"2023-11-06T02:13:33.279807+0000","last_became_active":"2023-11-06T02:13:33.272136+0000","last_became_peered":"2023-11-06T02:13:33.272136+0000","last_unstale":"2023-11-06T02:13:33.279807+0000","last_undegraded":"2023-11-06T02:13:33.279807+0000","last_fullsized":"2023-11-06T02:13:33.279807+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:02:50.688184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.238294+0000","last_change":"2023-11-06T02:13:33.238294+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.238294+0000","last_undegraded":"2023-11-06T02:13:33.238294+0000","last_fullsized":"2023-11-06T02:13:33.238294+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:02:47.316294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":3,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.244338+0000","last_change":"2023-11-06T02:13:33.244338+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.244338+0000","last_undegraded":"2023-11-06T02:13:33.244338+0000","last_fullsized":"2023-11-06T02:13:33.244338+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:11:59.707658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.245156+0000","last_change":"2023-11-06T02:13:33.245156+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.245156+0000","last_undegraded":"2023-11-06T02:13:33.245156+0000","last_fullsized":"2023-11-06T02:13:33.245156+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:07:57.793830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":3,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.258159+0000","last_change":"2023-11-06T02:13:33.258159+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.258159+0000","last_undegraded":"2023-11-06T02:13:33.258159+0000","last_fullsized":"2023-11-06T02:13:33.258159+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:16:35.397189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":["7","5"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.248850+0000","last_change":"2023-11-06T02:13:33.248850+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.248850+0000","last_undegraded":"2023-11-06T02:13:33.248850+0000","last_fullsized":"2023-11-06T02:13:33.248850+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:22:47.360905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":3,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.256366+0000","last_change":"2023-11-06T02:13:33.256366+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.256366+0000","last_undegraded":"2023-11-06T02:13:33.256366+0000","last_fullsized":"2023-11-06T02:13:33.256366+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:12:35.766676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":["7","6"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.244536+0000","last_change":"2023-11-06T02:13:33.244536+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.244536+0000","last_undegraded":"2023-11-06T02:13:33.244536+0000","last_fullsized":"2023-11-06T02:13:33.244536+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:42:42.388106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.277757+0000","last_change":"2023-11-06T02:13:33.267817+0000","last_active":"2023-11-06T02:13:33.277757+0000","last_peered":"2023-11-06T02:13:33.277757+0000","last_clean":"2023-11-06T02:13:33.277757+0000","last_became_active":"2023-11-06T02:13:33.267091+0000","last_became_peered":"2023-11-06T02:13:33.267091+0000","last_unstale":"2023-11-06T02:13:33.277757+0000","last_undegraded":"2023-11-06T02:13:33.277757+0000","last_fullsized":"2023-11-06T02:13:33.277757+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:36:21.262835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.239368+0000","last_change":"2023-11-06T02:13:33.239368+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.239368+0000","last_undegraded":"2023-11-06T02:13:33.239368+0000","last_fullsized":"2023-11-06T02:13:33.239368+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:31:19.083004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":3,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.235609+0000","last_change":"2023-11-06T02:13:33.235609+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.235609+0000","last_undegraded":"2023-11-06T02:13:33.235609+0000","last_fullsized":"2023-11-06T02:13:33.235609+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:19:20.386051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.254957+0000","last_change":"2023-11-06T02:13:33.254957+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.254957+0000","last_undegraded":"2023-11-06T02:13:33.254957+0000","last_fullsized":"2023-11-06T02:13:33.254957+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:02:47.316294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.277367+0000","last_change":"2023-11-06T02:13:33.264127+0000","last_active":"2023-11-06T02:13:33.277367+0000","last_peered":"2023-11-06T02:13:33.277367+0000","last_clean":"2023-11-06T02:13:33.277367+0000","last_became_active":"2023-11-06T02:13:33.263418+0000","last_became_peered":"2023-11-06T02:13:33.263418+0000","last_unstale":"2023-11-06T02:13:33.277367+0000","last_undegraded":"2023-11-06T02:13:33.277367+0000","last_fullsized":"2023-11-06T02:13:33.277367+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:04:53.601947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.253013+0000","last_change":"2023-11-06T02:13:33.253013+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.253013+0000","last_undegraded":"2023-11-06T02:13:33.253013+0000","last_fullsized":"2023-11-06T02:13:33.253013+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:31:19.083004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":3,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.251425+0000","last_change":"2023-11-06T02:13:33.251425+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.251425+0000","last_undegraded":"2023-11-06T02:13:33.251425+0000","last_fullsized":"2023-11-06T02:13:33.251425+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:13:44.482660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":["1","3"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.252952+0000","last_change":"2023-11-06T02:13:33.252952+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.252952+0000","last_undegraded":"2023-11-06T02:13:33.252952+0000","last_fullsized":"2023-11-06T02:13:33.252952+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:12:00.717133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":3,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.253882+0000","last_change":"2023-11-06T02:13:33.253882+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.253882+0000","last_undegraded":"2023-11-06T02:13:33.253882+0000","last_fullsized":"2023-11-06T02:13:33.253882+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:16:51.749335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":["3","6"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":216,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-06T02:13:32.232683+0000","last_change":"2023-11-06T02:08:20.523427+0000","last_active":"2023-11-06T02:13:32.232683+0000","last_peered":"2023-11-06T02:13:32.232683+0000","last_clean":"2023-11-06T02:13:32.232683+0000","last_became_active":"2023-11-06T02:08:20.521089+0000","last_became_peered":"2023-11-06T02:08:20.521089+0000","last_unstale":"2023-11-06T02:13:32.232683+0000","last_undegraded":"2023-11-06T02:13:32.232683+0000","last_fullsized":"2023-11-06T02:13:32.232683+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:19.252348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:19.252348+0000","last_clean_scrub_stamp":"2023-11-06T02:08:19.252348+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:30:15.566397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":62,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-06T02:13:32.246890+0000","last_change":"2023-11-06T02:08:26.059502+0000","last_active":"2023-11-06T02:13:32.246890+0000","last_peered":"2023-11-06T02:13:32.246890+0000","last_clean":"2023-11-06T02:13:32.246890+0000","last_became_active":"2023-11-06T02:08:26.058612+0000","last_became_peered":"2023-11-06T02:08:26.058612+0000","last_unstale":"2023-11-06T02:13:32.246890+0000","last_undegraded":"2023-11-06T02:13:32.246890+0000","last_fullsized":"2023-11-06T02:13:32.246890+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:10.626021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.250595+0000","last_change":"2023-11-06T02:13:33.250595+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.250595+0000","last_undegraded":"2023-11-06T02:13:33.250595+0000","last_fullsized":"2023-11-06T02:13:33.250595+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:46:01.922659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":3,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.246452+0000","last_change":"2023-11-06T02:13:33.246452+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.246452+0000","last_undegraded":"2023-11-06T02:13:33.246452+0000","last_fullsized":"2023-11-06T02:13:33.246452+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:11:51.109403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":["3","6"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":63,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-06T02:13:32.232564+0000","last_change":"2023-11-06T02:08:26.060406+0000","last_active":"2023-11-06T02:13:32.232564+0000","last_peered":"2023-11-06T02:13:32.232564+0000","last_clean":"2023-11-06T02:13:32.232564+0000","last_became_active":"2023-11-06T02:08:26.059262+0000","last_became_peered":"2023-11-06T02:08:26.059262+0000","last_unstale":"2023-11-06T02:13:32.232564+0000","last_undegraded":"2023-11-06T02:13:32.232564+0000","last_fullsized":"2023-11-06T02:13:32.232564+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:10.626021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.243210+0000","last_change":"2023-11-06T02:13:33.243210+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.243210+0000","last_undegraded":"2023-11-06T02:13:33.243210+0000","last_fullsized":"2023-11-06T02:13:33.243210+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:33:32.186597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":3,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.244333+0000","last_change":"2023-11-06T02:13:33.244333+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.244333+0000","last_undegraded":"2023-11-06T02:13:33.244333+0000","last_fullsized":"2023-11-06T02:13:33.244333+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:51:52.147063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":["3","6"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":63,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-06T02:13:32.236950+0000","last_change":"2023-11-06T02:08:26.559477+0000","last_active":"2023-11-06T02:13:32.236950+0000","last_peered":"2023-11-06T02:13:32.236950+0000","last_clean":"2023-11-06T02:13:32.236950+0000","last_became_active":"2023-11-06T02:08:26.559202+0000","last_became_peered":"2023-11-06T02:08:26.559202+0000","last_unstale":"2023-11-06T02:13:32.236950+0000","last_undegraded":"2023-11-06T02:13:32.236950+0000","last_fullsized":"2023-11-06T02:13:32.236950+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:55:04.552404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.249368+0000","last_change":"2023-11-06T02:13:33.249368+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.249368+0000","last_undegraded":"2023-11-06T02:13:33.249368+0000","last_fullsized":"2023-11-06T02:13:33.249368+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:54:11.389133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":3,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.245263+0000","last_change":"2023-11-06T02:13:33.245263+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.245263+0000","last_undegraded":"2023-11-06T02:13:33.245263+0000","last_fullsized":"2023-11-06T02:13:33.245263+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:38:14.971699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":62,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-06T02:13:32.246949+0000","last_change":"2023-11-06T02:08:26.059546+0000","last_active":"2023-11-06T02:13:32.246949+0000","last_peered":"2023-11-06T02:13:32.246949+0000","last_clean":"2023-11-06T02:13:32.246949+0000","last_became_active":"2023-11-06T02:08:26.058806+0000","last_became_peered":"2023-11-06T02:08:26.058806+0000","last_unstale":"2023-11-06T02:13:32.246949+0000","last_undegraded":"2023-11-06T02:13:32.246949+0000","last_fullsized":"2023-11-06T02:13:32.246949+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:23.468510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.242981+0000","last_change":"2023-11-06T02:13:33.242981+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.242981+0000","last_undegraded":"2023-11-06T02:13:33.242981+0000","last_fullsized":"2023-11-06T02:13:33.242981+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:36:12.474443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":3,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.236266+0000","last_change":"2023-11-06T02:13:33.236266+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.236266+0000","last_undegraded":"2023-11-06T02:13:33.236266+0000","last_fullsized":"2023-11-06T02:13:33.236266+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:30:05.141853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":["2","4"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":62,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-06T02:13:32.229548+0000","last_change":"2023-11-06T02:08:26.436463+0000","last_active":"2023-11-06T02:13:32.229548+0000","last_peered":"2023-11-06T02:13:32.229548+0000","last_clean":"2023-11-06T02:13:32.229548+0000","last_became_active":"2023-11-06T02:08:26.436075+0000","last_became_peered":"2023-11-06T02:08:26.436075+0000","last_unstale":"2023-11-06T02:13:32.229548+0000","last_undegraded":"2023-11-06T02:13:32.229548+0000","last_fullsized":"2023-11-06T02:13:32.229548+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:23.468510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.246317+0000","last_change":"2023-11-06T02:13:33.246317+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.246317+0000","last_undegraded":"2023-11-06T02:13:33.246317+0000","last_fullsized":"2023-11-06T02:13:33.246317+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:25:19.608926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":3,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.251742+0000","last_change":"2023-11-06T02:13:33.251742+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.251742+0000","last_undegraded":"2023-11-06T02:13:33.251742+0000","last_fullsized":"2023-11-06T02:13:33.251742+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:23:42.695735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":63,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-06T02:13:32.239073+0000","last_change":"2023-11-06T02:08:26.436607+0000","last_active":"2023-11-06T02:13:32.239073+0000","last_peered":"2023-11-06T02:13:32.239073+0000","last_clean":"2023-11-06T02:13:32.239073+0000","last_became_active":"2023-11-06T02:08:26.435924+0000","last_became_peered":"2023-11-06T02:08:26.435924+0000","last_unstale":"2023-11-06T02:13:32.239073+0000","last_undegraded":"2023-11-06T02:13:32.239073+0000","last_fullsized":"2023-11-06T02:13:32.239073+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:23.468510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.244818+0000","last_change":"2023-11-06T02:13:33.244818+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.244818+0000","last_undegraded":"2023-11-06T02:13:33.244818+0000","last_fullsized":"2023-11-06T02:13:33.244818+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:22:01.334447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":4,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.269809+0000","last_change":"2023-11-06T02:13:33.260450+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.269809+0000","last_undegraded":"2023-11-06T02:13:33.269809+0000","last_fullsized":"2023-11-06T02:13:33.269809+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:51:47.363705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":["7","1"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":62,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-06T02:13:32.229549+0000","last_change":"2023-11-06T02:08:26.057076+0000","last_active":"2023-11-06T02:13:32.229549+0000","last_peered":"2023-11-06T02:13:32.229549+0000","last_clean":"2023-11-06T02:13:32.229549+0000","last_became_active":"2023-11-06T02:08:26.056419+0000","last_became_peered":"2023-11-06T02:08:26.056419+0000","last_unstale":"2023-11-06T02:13:32.229549+0000","last_undegraded":"2023-11-06T02:13:32.229549+0000","last_fullsized":"2023-11-06T02:13:32.229549+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:10.626021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.242736+0000","last_change":"2023-11-06T02:13:33.242736+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.242736+0000","last_undegraded":"2023-11-06T02:13:33.242736+0000","last_fullsized":"2023-11-06T02:13:33.242736+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:38:31.470172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.265697+0000","last_change":"2023-11-06T02:13:33.265697+0000","last_active":"2023-11-06T02:13:33.265697+0000","last_peered":"2023-11-06T02:13:33.265697+0000","last_clean":"2023-11-06T02:13:33.265697+0000","last_became_active":"2023-11-06T02:13:33.264914+0000","last_became_peered":"2023-11-06T02:13:33.264914+0000","last_unstale":"2023-11-06T02:13:33.265697+0000","last_undegraded":"2023-11-06T02:13:33.265697+0000","last_fullsized":"2023-11-06T02:13:33.265697+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:17:35.088668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":71,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-06T02:13:32.237181+0000","last_change":"2023-11-06T02:08:26.058596+0000","last_active":"2023-11-06T02:13:32.237181+0000","last_peered":"2023-11-06T02:13:32.237181+0000","last_clean":"2023-11-06T02:13:32.237181+0000","last_became_active":"2023-11-06T02:08:26.057015+0000","last_became_peered":"2023-11-06T02:08:26.057015+0000","last_unstale":"2023-11-06T02:13:32.237181+0000","last_undegraded":"2023-11-06T02:13:32.237181+0000","last_fullsized":"2023-11-06T02:13:32.237181+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:55:04.552404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574914,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32544,"kb_used_data":1496,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339296,"statfs":{"total":96636764160,"available":96603439104,"internally_reserved":0,"allocated":1531904,"data_stored":1009349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:13:26 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.86599999999999999,"15min":0.85399999999999998},"min":{"1min":0.59199999999999997,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.2190000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.62},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.91200000000000003,"15min":0.91100000000000003},"min":{"1min":0.57299999999999995,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.214,"5min":1.506,"15min":1.506},"last":1}]},{"osd":1,"last update":"Mon Nov 6 02:13:26 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.83599999999999997,"15min":0.84599999999999997},"min":{"1min":0.66400000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":0.94799999999999995},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.94099999999999995,"15min":0.90600000000000003},"min":{"1min":0.64300000000000002,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.681,"5min":1.681,"15min":1.681},"last":0.75600000000000001}]},{"osd":2,"last update":"Mon Nov 6 02:13:26 2023","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.96599999999999997,"15min":0.95799999999999996},"min":{"1min":0.66200000000000003,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.22,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.94899999999999995,"15min":0.98299999999999998},"min":{"1min":0.71899999999999997,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.2789999999999999,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.70499999999999996}]},{"osd":3,"last update":"Mon Nov 6 02:13:26 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.89400000000000002,"15min":0.88100000000000001},"min":{"1min":0.67500000000000004,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.196,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.78300000000000003},{"interface":"front","average":{"1min":1.069,"5min":1.046,"15min":1.0660000000000001},"min":{"1min":0.86399999999999999,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.2430000000000001,"5min":1.526,"15min":1.526},"last":0.91300000000000003}]},{"osd":4,"last update":"Mon Nov 6 02:13:26 2023","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.92300000000000004,"15min":0.94599999999999995},"min":{"1min":0.59699999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.2350000000000001,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.874},{"interface":"front","average":{"1min":1.0129999999999999,"5min":1.004,"15min":1.0309999999999999},"min":{"1min":0.625,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.377,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":1.052}]},{"osd":5,"last update":"Mon Nov 6 02:13:26 2023","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.038,"15min":1.0649999999999999},"min":{"1min":0.76300000000000001,"5min":0.503,"15min":0.503},"max":{"1min":1.409,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":1.032},{"interface":"front","average":{"1min":1.046,"5min":1.0629999999999999,"15min":1.111},"min":{"1min":0.75900000000000001,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.3109999999999999,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":1.087}]},{"osd":6,"last update":"Mon Nov 6 02:13:26 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.1399999999999999,"15min":1.181},"min":{"1min":0.59399999999999997,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.278,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":1.1639999999999999},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.091,"15min":1.135},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.4570000000000001,"5min":1.74,"15min":1.74},"last":1.133}]}]},{"osd":6,"up_from":13,"seq":55834574913,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31964,"kb_used_data":916,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339876,"statfs":{"total":96636764160,"available":96604033024,"internally_reserved":0,"allocated":937984,"data_stored":418981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:13:28 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.86699999999999999,"15min":0.86099999999999999},"min":{"1min":0.58899999999999997,"5min":0.495,"15min":0.495},"max":{"1min":1.202,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.98099999999999998},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.874,"15min":0.82799999999999996},"min":{"1min":0.63,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.2649999999999999,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.56399999999999995}]},{"osd":1,"last update":"Mon Nov 6 02:13:28 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.84999999999999998,"15min":0.86699999999999999},"min":{"1min":0.63500000000000001,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.244,"5min":1.244,"15min":1.244},"last":1.1639999999999999},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.89600000000000002,"15min":0.88100000000000001},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.624}]},{"osd":2,"last update":"Mon Nov 6 02:13:28 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.88700000000000001,"15min":0.89500000000000002},"min":{"1min":0.52100000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.206,"5min":1.252,"15min":1.252},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.90000000000000002,"15min":0.92000000000000004},"min":{"1min":0.61699999999999999,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.411,"5min":1.411,"15min":1.411},"last":0.82899999999999996}]},{"osd":3,"last update":"Mon Nov 6 02:13:28 2023","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.93899999999999995,"15min":0.96099999999999997},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.212,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.0549999999999999,"5min":0.99299999999999999,"15min":0.998},"min":{"1min":0.77300000000000002,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.381,"5min":1.381,"15min":1.381},"last":1.101}]},{"osd":4,"last update":"Mon Nov 6 02:13:28 2023","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.92600000000000005,"15min":0.97099999999999997},"min":{"1min":0.67500000000000004,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.302,"5min":1.474,"15min":1.474},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.0469999999999999,"5min":0.98199999999999998,"15min":1.02},"min":{"1min":0.77100000000000002,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.556,"5min":1.556,"15min":1.556},"last":1.2050000000000001}]},{"osd":5,"last update":"Mon Nov 6 02:13:28 2023","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":0.97199999999999998,"15min":1.0089999999999999},"min":{"1min":0.73099999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":1.4650000000000001},{"interface":"front","average":{"1min":1.129,"5min":1.0720000000000001,"15min":1.1000000000000001},"min":{"1min":0.94899999999999995,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.587,"5min":1.587,"15min":1.587},"last":1.3999999999999999}]},{"osd":7,"last update":"Mon Nov 6 02:13:28 2023","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.087,"15min":1.107},"min":{"1min":0.95899999999999996,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.615,"5min":1.615,"15min":1.615},"last":1.363},{"interface":"front","average":{"1min":1.109,"5min":1.0629999999999999,"15min":1.131},"min":{"1min":0.83499999999999996,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":1.2969999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574914,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":916,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":937984,"data_stored":418981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:13:30 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.86899999999999999,"15min":0.84199999999999997},"min":{"1min":0.51700000000000002,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.1930000000000001,"5min":1.381,"15min":1.381},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.86599999999999999,"15min":0.84999999999999998},"min":{"1min":0.58199999999999996,"5min":0.503,"15min":0.503},"max":{"1min":1.1399999999999999,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":1.1399999999999999}]},{"osd":1,"last update":"Mon Nov 6 02:13:30 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.89600000000000002,"15min":0.86499999999999999},"min":{"1min":0.69799999999999995,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.177,"5min":1.331,"15min":1.331},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.88100000000000001,"15min":0.83799999999999997},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.139,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.71399999999999997}]},{"osd":2,"last update":"Mon Nov 6 02:13:30 2023","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.93400000000000005,"15min":0.94199999999999995},"min":{"1min":0.68000000000000005,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.298,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.872},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.88400000000000001,"15min":0.82099999999999995},"min":{"1min":0.56899999999999995,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.2869999999999999,"5min":1.377,"15min":1.377},"last":0.98399999999999999}]},{"osd":3,"last update":"Mon Nov 6 02:13:30 2023","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.95399999999999996,"15min":0.93100000000000005},"min":{"1min":0.77100000000000002,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.2210000000000001,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":1.091},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.997,"15min":1.0529999999999999},"min":{"1min":0.67900000000000005,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.224,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":0.78900000000000003}]},{"osd":4,"last update":"Mon Nov 6 02:13:30 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.89500000000000002,"15min":0.90900000000000003},"min":{"1min":0.58099999999999996,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.1779999999999999,"5min":1.276,"15min":1.276},"last":0.63700000000000001},{"interface":"front","average":{"1min":1.006,"5min":0.99299999999999999,"15min":1.016},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.254,"5min":1.554,"15min":1.554},"last":0.94799999999999995}]},{"osd":6,"last update":"Mon Nov 6 02:13:30 2023","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":1.0089999999999999,"15min":1.048},"min":{"1min":0.71399999999999997,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.2569999999999999,"5min":1.454,"15min":1.454},"last":1.016},{"interface":"front","average":{"1min":0.98699999999999999,"5min":1.022,"15min":1.034},"min":{"1min":0.75,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.1930000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.76000000000000001}]},{"osd":7,"last update":"Mon Nov 6 02:13:30 2023","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.0609999999999999,"15min":1.071},"min":{"1min":0.77900000000000003,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.2330000000000001,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":1.048},{"interface":"front","average":{"1min":1.109,"5min":1.1120000000000001,"15min":1.1399999999999999},"min":{"1min":0.84599999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.3799999999999999,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":1.0740000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574914,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":916,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":937984,"data_stored":418981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:13:27 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.877,"15min":0.84199999999999997},"min":{"1min":0.65100000000000002,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.0840000000000001,"5min":1.343,"15min":1.343},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.83299999999999996,"15min":0.79100000000000004},"min":{"1min":0.626,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.1699999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.77000000000000002}]},{"osd":1,"last update":"Mon Nov 6 02:13:27 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.86799999999999999,"15min":0.85199999999999998},"min":{"1min":0.57299999999999995,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.0329999999999999,"5min":1.462,"15min":1.462},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.93300000000000005,"15min":0.90500000000000003},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.224,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.59699999999999998}]},{"osd":2,"last update":"Mon Nov 6 02:13:27 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.93799999999999994,"15min":0.99199999999999999},"min":{"1min":0.70299999999999996,"5min":0.503,"15min":0.503},"max":{"1min":1.173,"5min":1.615,"15min":1.615},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.95499999999999996,"15min":0.93500000000000005},"min":{"1min":0.72499999999999998,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.163,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.97899999999999998}]},{"osd":3,"last update":"Mon Nov 6 02:13:27 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.88200000000000001,"15min":0.90600000000000003},"min":{"1min":0.61499999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.216,"5min":1.52,"15min":1.52},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.997,"15min":1.0449999999999999},"min":{"1min":0.66900000000000004,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.2050000000000001,"5min":1.595,"15min":1.595},"last":0.80900000000000005}]},{"osd":5,"last update":"Mon Nov 6 02:13:27 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.99099999999999999,"15min":1.0309999999999999},"min":{"1min":0.64500000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.339,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":1.0800000000000001},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.97599999999999998,"15min":1.0289999999999999},"min":{"1min":0.68899999999999995,"5min":0.629,"15min":0.629},"max":{"1min":1.161,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":0.86699999999999999}]},{"osd":6,"last update":"Mon Nov 6 02:13:27 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":1.008,"15min":1.0649999999999999},"min":{"1min":0.69599999999999995,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.268,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":1.03},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.1120000000000001,"15min":1.169},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.2789999999999999,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":1.129}]},{"osd":7,"last update":"Mon Nov 6 02:13:27 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.0489999999999999,"15min":1.079},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.294,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.93899999999999995},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.133,"15min":1.204},"min":{"1min":0.77500000000000002,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.3149999999999999,"5min":1.7410000000000001,"15min":1.7410000000000001},"last":1.1060000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574913,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31964,"kb_used_data":916,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339876,"statfs":{"total":96636764160,"available":96604033024,"internally_reserved":0,"allocated":937984,"data_stored":418981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:13:28 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.63200000000000001,"15min":0.64200000000000002},"min":{"1min":0.40799999999999997,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":1.198,"5min":1.198,"15min":1.198},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.64000000000000001,"15min":0.63},"min":{"1min":0.38700000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.72099999999999997}]},{"osd":1,"last update":"Mon Nov 6 02:13:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.61699999999999999,"15min":0.627},"min":{"1min":0.34599999999999997,"5min":0.248,"15min":0.248},"max":{"1min":0.92600000000000005,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.73999999999999999,"15min":0.73599999999999999},"min":{"1min":0.53000000000000003,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.60099999999999998}]},{"osd":2,"last update":"Mon Nov 6 02:13:28 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.71899999999999997,"15min":0.72899999999999998},"min":{"1min":0.45400000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.107,"5min":1.139,"15min":1.139},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.80800000000000005,"15min":0.81799999999999995},"min":{"1min":0.48099999999999998,"5min":0.308,"15min":0.308},"max":{"1min":1.212,"5min":1.405,"15min":1.405},"last":0.75700000000000001}]},{"osd":4,"last update":"Mon Nov 6 02:13:28 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.92000000000000004,"15min":0.92600000000000005},"min":{"1min":0.72599999999999998,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.0820000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.95299999999999996,"15min":0.98299999999999998},"min":{"1min":0.65100000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.1459999999999999,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.97399999999999998}]},{"osd":5,"last update":"Mon Nov 6 02:13:28 2023","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.98899999999999999,"15min":1.0149999999999999},"min":{"1min":0.72799999999999998,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.333,"5min":1.375,"15min":1.375},"last":0.81299999999999994},{"interface":"front","average":{"1min":1.091,"5min":1.0389999999999999,"15min":1.0669999999999999},"min":{"1min":0.627,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.4890000000000001,"5min":2.21,"15min":2.21},"last":1.0269999999999999}]},{"osd":6,"last update":"Mon Nov 6 02:13:28 2023","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.016,"15min":1.022},"min":{"1min":0.61299999999999999,"5min":0.439,"15min":0.439},"max":{"1min":1.3740000000000001,"5min":1.395,"15min":1.395},"last":0.78900000000000003},{"interface":"front","average":{"1min":1.036,"5min":1.04,"15min":1.0609999999999999},"min":{"1min":0.54600000000000004,"5min":0.375,"15min":0.375},"max":{"1min":1.3069999999999999,"5min":1.554,"15min":1.554},"last":1.107}]},{"osd":7,"last update":"Mon Nov 6 02:13:28 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.1080000000000001,"15min":1.1240000000000001},"min":{"1min":0.66800000000000004,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.421,"5min":2.2829999999999999,"15min":2.2829999999999999},"last":0.94099999999999995},{"interface":"front","average":{"1min":1.091,"5min":1.0940000000000001,"15min":1.1020000000000001},"min":{"1min":0.70999999999999996,"5min":0.497,"15min":0.497},"max":{"1min":1.4490000000000001,"5min":1.504,"15min":1.504},"last":1.048}]}]},{"osd":2,"up_from":13,"seq":55834574913,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31964,"kb_used_data":916,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339876,"statfs":{"total":96636764160,"available":96604033024,"internally_reserved":0,"allocated":937984,"data_stored":418981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:13:24 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.57699999999999996,"15min":0.52500000000000002},"min":{"1min":0.33100000000000002,"5min":0.252,"15min":0.252},"max":{"1min":0.85699999999999998,"5min":1.079,"15min":1.079},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.625,"15min":0.56699999999999995},"min":{"1min":0.505,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.1060000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":0.56999999999999995}]},{"osd":1,"last update":"Mon Nov 6 02:13:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.63500000000000001,"15min":0.58499999999999996},"min":{"1min":0.38800000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.88300000000000001,"5min":1.48,"15min":1.48},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.70599999999999996,"15min":0.61399999999999999},"min":{"1min":0.377,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.282,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":0.70499999999999996}]},{"osd":3,"last update":"Mon Nov 6 02:13:24 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.70099999999999996,"15min":0.63200000000000001},"min":{"1min":0.48199999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.98999999999999999,"5min":1.155,"15min":1.155},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.749,"15min":0.67700000000000005},"min":{"1min":0.41999999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.81499999999999995}]},{"osd":4,"last update":"Mon Nov 6 02:13:24 2023","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.90900000000000003,"15min":0.83499999999999996},"min":{"1min":0.58499999999999996,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.2150000000000001,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.93500000000000005,"15min":0.92100000000000004},"min":{"1min":0.73099999999999998,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.3120000000000001,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":1.0589999999999999}]},{"osd":5,"last update":"Mon Nov 6 02:13:24 2023","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":0.96799999999999997,"15min":0.95099999999999996},"min":{"1min":0.75,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.2270000000000001,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":0.93999999999999995},{"interface":"front","average":{"1min":1,"5min":0.97099999999999997,"15min":0.94599999999999995},"min":{"1min":0.628,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.3109999999999999,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.96399999999999997}]},{"osd":6,"last update":"Mon Nov 6 02:13:24 2023","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":0.97099999999999997,"15min":0.94899999999999995},"min":{"1min":0.59899999999999998,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.3129999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.0229999999999999,"15min":1.0049999999999999},"min":{"1min":0.73299999999999998,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.2769999999999999,"5min":1.677,"15min":1.677},"last":1.1479999999999999}]},{"osd":7,"last update":"Mon Nov 6 02:13:24 2023","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.026,"15min":0.99199999999999999},"min":{"1min":0.72199999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.4079999999999999,"5min":1.659,"15min":1.659},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.123,"5min":1.0820000000000001,"15min":1.0549999999999999},"min":{"1min":0.79300000000000004,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.3819999999999999,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":1.087}]}]},{"osd":1,"up_from":13,"seq":55834574914,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31964,"kb_used_data":916,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339876,"statfs":{"total":96636764160,"available":96604033024,"internally_reserved":0,"allocated":937984,"data_stored":418981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:13:27 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.63900000000000001,"15min":0.63},"min":{"1min":0.35899999999999999,"5min":0.224,"15min":0.224},"max":{"1min":1.343,"5min":1.603,"15min":1.603},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.71899999999999997,"15min":0.67400000000000004},"min":{"1min":0.38200000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":2.161,"5min":2.161,"15min":2.161},"last":1.081}]},{"osd":2,"last update":"Mon Nov 6 02:13:27 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.65400000000000003,"15min":0.63200000000000001},"min":{"1min":0.26500000000000001,"5min":0.253,"15min":0.253},"max":{"1min":1.256,"5min":1.649,"15min":1.649},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.753,"15min":0.747},"min":{"1min":0.41099999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":1.5249999999999999,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.88400000000000001}]},{"osd":3,"last update":"Mon Nov 6 02:13:27 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.81299999999999994,"15min":0.80600000000000005},"min":{"1min":0.49099999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.6319999999999999,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":1.131},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.81299999999999994,"15min":0.81100000000000005},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.1799999999999999,"5min":1.8149999999999999,"15min":1.8149999999999999},"last":0.94199999999999995}]},{"osd":4,"last update":"Mon Nov 6 02:13:27 2023","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":0.97899999999999998,"15min":0.98799999999999999},"min":{"1min":0.78500000000000003,"5min":0.504,"15min":0.504},"max":{"1min":2.2759999999999998,"5min":2.2759999999999998,"15min":2.2759999999999998},"last":1.196},{"interface":"front","average":{"1min":1.0740000000000001,"5min":0.97699999999999998,"15min":0.98799999999999999},"min":{"1min":0.64200000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":2.9180000000000001,"5min":2.9180000000000001,"15min":2.9180000000000001},"last":1.0049999999999999}]},{"osd":5,"last update":"Mon Nov 6 02:13:27 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":0.98099999999999998,"15min":0.97099999999999997},"min":{"1min":0.68100000000000005,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":2.7469999999999999,"5min":2.7469999999999999,"15min":2.7469999999999999},"last":1.1659999999999999},{"interface":"front","average":{"1min":1.1579999999999999,"5min":1.069,"15min":1.075},"min":{"1min":0.81100000000000005,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.921,"5min":1.921,"15min":1.921},"last":1.044}]},{"osd":6,"last update":"Mon Nov 6 02:13:27 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.113,"15min":1.1060000000000001},"min":{"1min":0.83099999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":3.5710000000000002,"5min":3.5710000000000002,"15min":3.5710000000000002},"last":1.278},{"interface":"front","average":{"1min":1.157,"5min":1.0629999999999999,"15min":1.0720000000000001},"min":{"1min":0.60299999999999998,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":2.843,"5min":2.843,"15min":2.843},"last":0.83499999999999996}]},{"osd":7,"last update":"Mon Nov 6 02:13:27 2023","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.1180000000000001,"15min":1.1200000000000001},"min":{"1min":0.89600000000000002,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":2.0009999999999999,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":1.2330000000000001},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.1639999999999999,"15min":1.1619999999999999},"min":{"1min":0.91100000000000003,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":3.4790000000000001,"5min":3.4790000000000001,"15min":3.4790000000000001},"last":1.3520000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574914,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32540,"kb_used_data":1492,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339300,"statfs":{"total":96636764160,"available":96603443200,"internally_reserved":0,"allocated":1527808,"data_stored":1008330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 02:13:26 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.61499999999999999,"15min":0.60299999999999998},"min":{"1min":0.53100000000000003,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.0289999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.755,"5min":0.66900000000000004,"15min":0.64700000000000002},"min":{"1min":0.53800000000000003,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.079,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.46300000000000002}]},{"osd":2,"last update":"Mon Nov 6 02:13:26 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.68600000000000005,"15min":0.67200000000000004},"min":{"1min":0.42999999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.1080000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.98299999999999998},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.69799999999999995,"15min":0.68200000000000005},"min":{"1min":0.57299999999999995,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.75700000000000001}]},{"osd":3,"last update":"Mon Nov 6 02:13:26 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.67200000000000004,"15min":0.64300000000000002},"min":{"1min":0.52500000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.1659999999999999,"5min":1.429,"15min":1.429},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.87,"5min":0.78300000000000003,"15min":0.748},"min":{"1min":0.69999999999999996,"5min":0.317,"15min":0.317},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.89000000000000001}]},{"osd":4,"last update":"Mon Nov 6 02:13:26 2023","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.90200000000000002,"15min":0.85199999999999998},"min":{"1min":0.76800000000000002,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.1779999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":1.0589999999999999,"5min":0.96099999999999997,"15min":0.94499999999999995},"min":{"1min":0.78900000000000003,"5min":0.5,"15min":0.5},"max":{"1min":1.292,"5min":1.387,"15min":1.387},"last":1.137}]},{"osd":5,"last update":"Mon Nov 6 02:13:26 2023","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":0.95699999999999996,"15min":0.95099999999999996},"min":{"1min":0.81799999999999995,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.006,"5min":0.93400000000000005,"15min":0.89200000000000002},"min":{"1min":0.81100000000000005,"5min":0.496,"15min":0.496},"max":{"1min":1.2549999999999999,"5min":1.448,"15min":1.448},"last":1.101}]},{"osd":6,"last update":"Mon Nov 6 02:13:26 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.0389999999999999,"15min":1.0369999999999999},"min":{"1min":0.73899999999999999,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.385,"5min":1.589,"15min":1.589},"last":1.236},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.0269999999999999,"15min":1.0089999999999999},"min":{"1min":0.76000000000000001,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.419,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":1.1779999999999999}]},{"osd":7,"last update":"Mon Nov 6 02:13:26 2023","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.026,"15min":1.0209999999999999},"min":{"1min":0.76400000000000001,"5min":0.504,"15min":0.504},"max":{"1min":1.504,"5min":1.504,"15min":1.504},"last":1.022},{"interface":"front","average":{"1min":1.073,"5min":0.998,"15min":1.0049999999999999},"min":{"1min":0.74299999999999999,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.3999999999999999,"5min":1.556,"15min":1.556},"last":0.873}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T02:13:38.437 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.3f is not active+clean 2023-11-06T02:13:38.437 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-06T02:13:33.255018+0000', 'last_change': '2023-11-06T02:13:33.255018+0000', 'last_active': '2023-11-06T02:13:33.224257+0000', 'last_peered': '2023-11-06T02:13:33.224257+0000', 'last_clean': '2023-11-06T02:13:33.224257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:13:33.255018+0000', 'last_undegraded': '2023-11-06T02:13:33.255018+0000', 'last_fullsized': '2023-11-06T02:13:33.255018+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_clean_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T06:18:07.485044+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-06T02:13:38.438 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.3e is not active+clean 2023-11-06T02:13:38.438 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-06T02:13:33.252648+0000', 'last_change': '2023-11-06T02:13:33.252648+0000', 'last_active': '2023-11-06T02:13:33.224257+0000', 'last_peered': '2023-11-06T02:13:33.224257+0000', 'last_clean': '2023-11-06T02:13:33.224257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:13:33.252648+0000', 'last_undegraded': '2023-11-06T02:13:33.252648+0000', 'last_fullsized': '2023-11-06T02:13:33.252648+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_clean_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T08:22:01.334447+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-06T02:13:38.439 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.3d is not active+clean 2023-11-06T02:13:38.439 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-06T02:13:33.257065+0000', 'last_change': '2023-11-06T02:13:33.257065+0000', 'last_active': '2023-11-06T02:13:33.224257+0000', 'last_peered': '2023-11-06T02:13:33.224257+0000', 'last_clean': '2023-11-06T02:13:33.224257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:13:33.257065+0000', 'last_undegraded': '2023-11-06T02:13:33.257065+0000', 'last_fullsized': '2023-11-06T02:13:33.257065+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_clean_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T08:42:42.388106+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-06T02:13:38.439 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.3c is not active+clean 2023-11-06T02:13:38.440 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-06T02:13:33.256535+0000', 'last_change': '2023-11-06T02:13:33.256535+0000', 'last_active': '2023-11-06T02:13:33.224257+0000', 'last_peered': '2023-11-06T02:13:33.224257+0000', 'last_clean': '2023-11-06T02:13:33.224257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:13:33.256535+0000', 'last_undegraded': '2023-11-06T02:13:33.256535+0000', 'last_fullsized': '2023-11-06T02:13:33.256535+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_clean_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T11:54:33.444383+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-06T02:13:38.440 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.3b is not active+clean 2023-11-06T02:13:38.440 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-06T02:13:33.246148+0000', 'last_change': '2023-11-06T02:13:33.246148+0000', 'last_active': '2023-11-06T02:13:33.224257+0000', 'last_peered': '2023-11-06T02:13:33.224257+0000', 'last_clean': '2023-11-06T02:13:33.224257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:13:33.246148+0000', 'last_undegraded': '2023-11-06T02:13:33.246148+0000', 'last_fullsized': '2023-11-06T02:13:33.246148+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_clean_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T04:01:06.093323+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-06T02:13:38.441 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.3a is not active+clean 2023-11-06T02:13:38.441 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-06T02:13:33.258259+0000', 'last_change': '2023-11-06T02:13:33.258259+0000', 'last_active': '2023-11-06T02:13:33.224257+0000', 'last_peered': '2023-11-06T02:13:33.224257+0000', 'last_clean': '2023-11-06T02:13:33.224257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:13:33.258259+0000', 'last_undegraded': '2023-11-06T02:13:33.258259+0000', 'last_fullsized': '2023-11-06T02:13:33.258259+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_clean_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T07:41:55.941458+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-06T02:13:38.442 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.39 is not active+clean 2023-11-06T02:13:38.442 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-06T02:13:33.239742+0000', 'last_change': '2023-11-06T02:13:33.239742+0000', 'last_active': '2023-11-06T02:13:33.224257+0000', 'last_peered': '2023-11-06T02:13:33.224257+0000', 'last_clean': '2023-11-06T02:13:33.224257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:13:33.239742+0000', 'last_undegraded': '2023-11-06T02:13:33.239742+0000', 'last_fullsized': '2023-11-06T02:13:33.239742+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_clean_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T06:40:54.464756+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-06T02:13:38.442 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.38 is not active+clean 2023-11-06T02:13:38.443 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-06T02:13:33.246060+0000', 'last_change': '2023-11-06T02:13:33.246060+0000', 'last_active': '2023-11-06T02:13:33.224257+0000', 'last_peered': '2023-11-06T02:13:33.224257+0000', 'last_clean': '2023-11-06T02:13:33.224257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:13:33.246060+0000', 'last_undegraded': '2023-11-06T02:13:33.246060+0000', 'last_fullsized': '2023-11-06T02:13:33.246060+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_clean_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T12:38:39.514240+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-06T02:13:38.443 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.37 is not active+clean 2023-11-06T02:13:38.443 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-06T02:13:33.243570+0000', 'last_change': '2023-11-06T02:13:33.243570+0000', 'last_active': '2023-11-06T02:13:33.224257+0000', 'last_peered': '2023-11-06T02:13:33.224257+0000', 'last_clean': '2023-11-06T02:13:33.224257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:13:33.243570+0000', 'last_undegraded': '2023-11-06T02:13:33.243570+0000', 'last_fullsized': '2023-11-06T02:13:33.243570+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_clean_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T04:02:51.697659+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-06T02:13:38.444 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.36 is not active+clean 2023-11-06T02:13:38.444 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-06T02:13:33.257271+0000', 'last_change': '2023-11-06T02:13:33.257271+0000', 'last_active': '2023-11-06T02:13:33.224257+0000', 'last_peered': '2023-11-06T02:13:33.224257+0000', 'last_clean': '2023-11-06T02:13:33.224257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:13:33.257271+0000', 'last_undegraded': '2023-11-06T02:13:33.257271+0000', 'last_fullsized': '2023-11-06T02:13:33.257271+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_clean_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T09:12:00.717133+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-06T02:13:38.445 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.35 is not active+clean 2023-11-06T02:13:38.445 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-06T02:13:33.252080+0000', 'last_change': '2023-11-06T02:13:33.252080+0000', 'last_active': '2023-11-06T02:13:33.224257+0000', 'last_peered': '2023-11-06T02:13:33.224257+0000', 'last_clean': '2023-11-06T02:13:33.224257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:13:33.252080+0000', 'last_undegraded': '2023-11-06T02:13:33.252080+0000', 'last_fullsized': '2023-11-06T02:13:33.252080+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_clean_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T09:41:33.827814+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-06T02:13:38.445 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.34 is not active+clean 2023-11-06T02:13:38.446 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-06T02:13:33.255251+0000', 'last_change': '2023-11-06T02:13:33.255251+0000', 'last_active': '2023-11-06T02:13:33.224257+0000', 'last_peered': '2023-11-06T02:13:33.224257+0000', 'last_clean': '2023-11-06T02:13:33.224257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:13:33.255251+0000', 'last_undegraded': '2023-11-06T02:13:33.255251+0000', 'last_fullsized': '2023-11-06T02:13:33.255251+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_clean_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T03:21:24.032595+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-06T02:13:38.446 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.33 is not active+clean 2023-11-06T02:13:38.447 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-06T02:13:33.245690+0000', 'last_change': '2023-11-06T02:13:33.245690+0000', 'last_active': '2023-11-06T02:13:33.224257+0000', 'last_peered': '2023-11-06T02:13:33.224257+0000', 'last_clean': '2023-11-06T02:13:33.224257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:13:33.245690+0000', 'last_undegraded': '2023-11-06T02:13:33.245690+0000', 'last_fullsized': '2023-11-06T02:13:33.245690+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_clean_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T07:42:30.360623+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-06T02:13:38.447 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.32 is not active+clean 2023-11-06T02:13:38.447 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-06T02:13:33.257567+0000', 'last_change': '2023-11-06T02:13:33.257567+0000', 'last_active': '2023-11-06T02:13:33.224257+0000', 'last_peered': '2023-11-06T02:13:33.224257+0000', 'last_clean': '2023-11-06T02:13:33.224257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:13:33.257567+0000', 'last_undegraded': '2023-11-06T02:13:33.257567+0000', 'last_fullsized': '2023-11-06T02:13:33.257567+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_clean_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T09:41:33.827814+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-06T02:13:38.448 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.31 is not active+clean 2023-11-06T02:13:38.448 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-06T02:13:33.252478+0000', 'last_change': '2023-11-06T02:13:33.252478+0000', 'last_active': '2023-11-06T02:13:33.224257+0000', 'last_peered': '2023-11-06T02:13:33.224257+0000', 'last_clean': '2023-11-06T02:13:33.224257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:13:33.252478+0000', 'last_undegraded': '2023-11-06T02:13:33.252478+0000', 'last_fullsized': '2023-11-06T02:13:33.252478+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_clean_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T12:28:22.791353+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-06T02:13:38.448 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.30 is not active+clean 2023-11-06T02:13:38.449 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-06T02:13:33.254366+0000', 'last_change': '2023-11-06T02:13:33.254366+0000', 'last_active': '2023-11-06T02:13:33.224257+0000', 'last_peered': '2023-11-06T02:13:33.224257+0000', 'last_clean': '2023-11-06T02:13:33.224257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:13:33.254366+0000', 'last_undegraded': '2023-11-06T02:13:33.254366+0000', 'last_fullsized': '2023-11-06T02:13:33.254366+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_clean_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T03:21:24.032595+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-06T02:13:38.449 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.2f is not active+clean 2023-11-06T02:13:38.450 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-06T02:13:33.246654+0000', 'last_change': '2023-11-06T02:13:33.246654+0000', 'last_active': '2023-11-06T02:13:33.224257+0000', 'last_peered': '2023-11-06T02:13:33.224257+0000', 'last_clean': '2023-11-06T02:13:33.224257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:13:33.246654+0000', 'last_undegraded': '2023-11-06T02:13:33.246654+0000', 'last_fullsized': '2023-11-06T02:13:33.246654+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_clean_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T12:54:43.806737+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-06T02:13:38.450 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.2e is not active+clean 2023-11-06T02:13:38.450 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-06T02:13:33.255469+0000', 'last_change': '2023-11-06T02:13:33.255469+0000', 'last_active': '2023-11-06T02:13:33.224257+0000', 'last_peered': '2023-11-06T02:13:33.224257+0000', 'last_clean': '2023-11-06T02:13:33.224257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:13:33.255469+0000', 'last_undegraded': '2023-11-06T02:13:33.255469+0000', 'last_fullsized': '2023-11-06T02:13:33.255469+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_clean_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T10:17:36.098143+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-06T02:13:38.451 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.2d is not active+clean 2023-11-06T02:13:38.451 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-06T02:13:33.254869+0000', 'last_change': '2023-11-06T02:13:33.254869+0000', 'last_active': '2023-11-06T02:13:33.224257+0000', 'last_peered': '2023-11-06T02:13:33.224257+0000', 'last_clean': '2023-11-06T02:13:33.224257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:13:33.254869+0000', 'last_undegraded': '2023-11-06T02:13:33.254869+0000', 'last_fullsized': '2023-11-06T02:13:33.254869+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_clean_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T03:22:13.884367+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-06T02:13:38.452 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.2c is not active+clean 2023-11-06T02:13:38.452 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-06T02:13:33.247005+0000', 'last_change': '2023-11-06T02:13:33.247005+0000', 'last_active': '2023-11-06T02:13:33.224257+0000', 'last_peered': '2023-11-06T02:13:33.224257+0000', 'last_clean': '2023-11-06T02:13:33.224257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:13:33.247005+0000', 'last_undegraded': '2023-11-06T02:13:33.247005+0000', 'last_fullsized': '2023-11-06T02:13:33.247005+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_clean_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T13:28:11.270422+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-06T02:13:38.452 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.2b is not active+clean 2023-11-06T02:13:38.453 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-06T02:13:33.242891+0000', 'last_change': '2023-11-06T02:13:33.242891+0000', 'last_active': '2023-11-06T02:13:33.224257+0000', 'last_peered': '2023-11-06T02:13:33.224257+0000', 'last_clean': '2023-11-06T02:13:33.224257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:13:33.242891+0000', 'last_undegraded': '2023-11-06T02:13:33.242891+0000', 'last_fullsized': '2023-11-06T02:13:33.242891+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_clean_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T05:43:49.529761+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-06T02:13:38.453 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.2a is not active+clean 2023-11-06T02:13:38.453 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-06T02:13:33.246368+0000', 'last_change': '2023-11-06T02:13:33.246368+0000', 'last_active': '2023-11-06T02:13:33.224257+0000', 'last_peered': '2023-11-06T02:13:33.224257+0000', 'last_clean': '2023-11-06T02:13:33.224257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:13:33.246368+0000', 'last_undegraded': '2023-11-06T02:13:33.246368+0000', 'last_fullsized': '2023-11-06T02:13:33.246368+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_clean_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T10:19:02.943437+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-06T02:13:38.454 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.29 is not active+clean 2023-11-06T02:13:38.454 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-06T02:13:33.244217+0000', 'last_change': '2023-11-06T02:13:33.244217+0000', 'last_active': '2023-11-06T02:13:33.224257+0000', 'last_peered': '2023-11-06T02:13:33.224257+0000', 'last_clean': '2023-11-06T02:13:33.224257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:13:33.244217+0000', 'last_undegraded': '2023-11-06T02:13:33.244217+0000', 'last_fullsized': '2023-11-06T02:13:33.244217+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_clean_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T04:17:17.217507+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-06T02:13:38.455 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.28 is not active+clean 2023-11-06T02:13:38.455 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-06T02:13:33.252017+0000', 'last_change': '2023-11-06T02:13:33.252017+0000', 'last_active': '2023-11-06T02:13:33.224257+0000', 'last_peered': '2023-11-06T02:13:33.224257+0000', 'last_clean': '2023-11-06T02:13:33.224257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:13:33.252017+0000', 'last_undegraded': '2023-11-06T02:13:33.252017+0000', 'last_fullsized': '2023-11-06T02:13:33.252017+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_clean_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T07:41:55.941458+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-06T02:13:38.456 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.27 is not active+clean 2023-11-06T02:13:38.456 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-06T02:13:33.250038+0000', 'last_change': '2023-11-06T02:13:33.250038+0000', 'last_active': '2023-11-06T02:13:33.224257+0000', 'last_peered': '2023-11-06T02:13:33.224257+0000', 'last_clean': '2023-11-06T02:13:33.224257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:13:33.250038+0000', 'last_undegraded': '2023-11-06T02:13:33.250038+0000', 'last_fullsized': '2023-11-06T02:13:33.250038+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_clean_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T03:22:13.884367+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-06T02:13:38.456 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.26 is not active+clean 2023-11-06T02:13:38.457 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-06T02:13:33.255314+0000', 'last_change': '2023-11-06T02:13:33.255314+0000', 'last_active': '2023-11-06T02:13:33.224257+0000', 'last_peered': '2023-11-06T02:13:33.224257+0000', 'last_clean': '2023-11-06T02:13:33.224257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:13:33.255314+0000', 'last_undegraded': '2023-11-06T02:13:33.255314+0000', 'last_fullsized': '2023-11-06T02:13:33.255314+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_clean_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T06:18:07.485044+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-06T02:13:38.457 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.25 is not active+clean 2023-11-06T02:13:38.457 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-06T02:13:33.254181+0000', 'last_change': '2023-11-06T02:13:33.254181+0000', 'last_active': '2023-11-06T02:13:33.224257+0000', 'last_peered': '2023-11-06T02:13:33.224257+0000', 'last_clean': '2023-11-06T02:13:33.224257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:13:33.254181+0000', 'last_undegraded': '2023-11-06T02:13:33.254181+0000', 'last_fullsized': '2023-11-06T02:13:33.254181+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_clean_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T09:00:13.166796+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-06T02:13:38.458 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.24 is not active+clean 2023-11-06T02:13:38.458 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-06T02:13:33.243183+0000', 'last_change': '2023-11-06T02:13:33.243183+0000', 'last_active': '2023-11-06T02:13:33.224257+0000', 'last_peered': '2023-11-06T02:13:33.224257+0000', 'last_clean': '2023-11-06T02:13:33.224257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:13:33.243183+0000', 'last_undegraded': '2023-11-06T02:13:33.243183+0000', 'last_fullsized': '2023-11-06T02:13:33.243183+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_clean_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T12:16:46.930331+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-06T02:13:38.459 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.23 is not active+clean 2023-11-06T02:13:38.459 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-06T02:13:33.249877+0000', 'last_change': '2023-11-06T02:13:33.249877+0000', 'last_active': '2023-11-06T02:13:33.224257+0000', 'last_peered': '2023-11-06T02:13:33.224257+0000', 'last_clean': '2023-11-06T02:13:33.224257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:13:33.249877+0000', 'last_undegraded': '2023-11-06T02:13:33.249877+0000', 'last_fullsized': '2023-11-06T02:13:33.249877+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_clean_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T05:20:11.090360+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-06T02:13:38.460 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.22 is not active+clean 2023-11-06T02:13:38.460 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-06T02:13:33.250528+0000', 'last_change': '2023-11-06T02:13:33.250528+0000', 'last_active': '2023-11-06T02:13:33.224257+0000', 'last_peered': '2023-11-06T02:13:33.224257+0000', 'last_clean': '2023-11-06T02:13:33.224257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:13:33.250528+0000', 'last_undegraded': '2023-11-06T02:13:33.250528+0000', 'last_fullsized': '2023-11-06T02:13:33.250528+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_clean_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T07:34:15.876381+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-06T02:13:38.460 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.21 is not active+clean 2023-11-06T02:13:38.461 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-06T02:13:33.253284+0000', 'last_change': '2023-11-06T02:13:33.253284+0000', 'last_active': '2023-11-06T02:13:33.224257+0000', 'last_peered': '2023-11-06T02:13:33.224257+0000', 'last_clean': '2023-11-06T02:13:33.224257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:13:33.253284+0000', 'last_undegraded': '2023-11-06T02:13:33.253284+0000', 'last_fullsized': '2023-11-06T02:13:33.253284+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_clean_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T12:14:22.392069+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-06T02:13:38.461 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.20 is not active+clean 2023-11-06T02:13:38.462 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-06T02:13:33.252628+0000', 'last_change': '2023-11-06T02:13:33.252628+0000', 'last_active': '2023-11-06T02:13:33.224257+0000', 'last_peered': '2023-11-06T02:13:33.224257+0000', 'last_clean': '2023-11-06T02:13:33.224257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:13:33.252628+0000', 'last_undegraded': '2023-11-06T02:13:33.252628+0000', 'last_fullsized': '2023-11-06T02:13:33.252628+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_clean_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T02:24:31.881719+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-06T02:13:38.462 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.18 is not active+clean 2023-11-06T02:13:38.462 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-06T02:13:33.252017+0000', 'last_change': '2023-11-06T02:13:33.252017+0000', 'last_active': '2023-11-06T02:13:33.224257+0000', 'last_peered': '2023-11-06T02:13:33.224257+0000', 'last_clean': '2023-11-06T02:13:33.224257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:13:33.252017+0000', 'last_undegraded': '2023-11-06T02:13:33.252017+0000', 'last_fullsized': '2023-11-06T02:13:33.252017+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_clean_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T10:46:42.636025+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-06T02:13:38.463 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1f is not active+clean 2023-11-06T02:13:38.463 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1f', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 78, 'state': 'creating+activating', 'last_fresh': '2023-11-06T02:13:33.235656+0000', 'last_change': '2023-11-06T02:13:33.235656+0000', 'last_active': '2023-11-06T02:13:32.214782+0000', 'last_peered': '2023-11-06T02:13:32.214782+0000', 'last_clean': '2023-11-06T02:13:32.214782+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:13:33.235656+0000', 'last_undegraded': '2023-11-06T02:13:33.235656+0000', 'last_fullsized': '2023-11-06T02:13:33.235656+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:13:32.214782+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:13:32.214782+0000', 'last_clean_scrub_stamp': '2023-11-06T02:13:32.214782+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T11:07:13.179013+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-06T02:13:38.464 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.19 is not active+clean 2023-11-06T02:13:38.464 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-06T02:13:33.249040+0000', 'last_change': '2023-11-06T02:13:33.249040+0000', 'last_active': '2023-11-06T02:13:33.224257+0000', 'last_peered': '2023-11-06T02:13:33.224257+0000', 'last_clean': '2023-11-06T02:13:33.224257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:13:33.249040+0000', 'last_undegraded': '2023-11-06T02:13:33.249040+0000', 'last_fullsized': '2023-11-06T02:13:33.249040+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_clean_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T07:23:55.083630+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-06T02:13:38.464 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1e is not active+clean 2023-11-06T02:13:38.465 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1e', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 78, 'state': 'creating+activating', 'last_fresh': '2023-11-06T02:13:33.256359+0000', 'last_change': '2023-11-06T02:13:33.256359+0000', 'last_active': '2023-11-06T02:13:32.214782+0000', 'last_peered': '2023-11-06T02:13:32.214782+0000', 'last_clean': '2023-11-06T02:13:32.214782+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:13:33.256359+0000', 'last_undegraded': '2023-11-06T02:13:33.256359+0000', 'last_fullsized': '2023-11-06T02:13:33.256359+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:13:32.214782+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:13:32.214782+0000', 'last_clean_scrub_stamp': '2023-11-06T02:13:32.214782+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T06:33:14.271873+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': ['7', '6'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-06T02:13:38.465 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1a is not active+clean 2023-11-06T02:13:38.466 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-06T02:13:33.246368+0000', 'last_change': '2023-11-06T02:13:33.246368+0000', 'last_active': '2023-11-06T02:13:33.224257+0000', 'last_peered': '2023-11-06T02:13:33.224257+0000', 'last_clean': '2023-11-06T02:13:33.224257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:13:33.246368+0000', 'last_undegraded': '2023-11-06T02:13:33.246368+0000', 'last_fullsized': '2023-11-06T02:13:33.246368+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_clean_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T07:34:15.876381+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-06T02:13:38.466 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1d is not active+clean 2023-11-06T02:13:38.466 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1d', 'version': "0'0", 'reported_seq': 4, 'reported_epoch': 78, 'state': 'creating+activating', 'last_fresh': '2023-11-06T02:13:33.264127+0000', 'last_change': '2023-11-06T02:13:33.257252+0000', 'last_active': '2023-11-06T02:13:32.214782+0000', 'last_peered': '2023-11-06T02:13:32.214782+0000', 'last_clean': '2023-11-06T02:13:32.214782+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:13:33.264127+0000', 'last_undegraded': '2023-11-06T02:13:33.264127+0000', 'last_fullsized': '2023-11-06T02:13:33.264127+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:13:32.214782+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:13:32.214782+0000', 'last_clean_scrub_stamp': '2023-11-06T02:13:32.214782+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T13:35:42.092353+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': ['7', '0'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-06T02:13:38.467 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1b is not active+clean 2023-11-06T02:13:38.467 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-06T02:13:33.241683+0000', 'last_change': '2023-11-06T02:13:33.241683+0000', 'last_active': '2023-11-06T02:13:33.224257+0000', 'last_peered': '2023-11-06T02:13:33.224257+0000', 'last_clean': '2023-11-06T02:13:33.224257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:13:33.241683+0000', 'last_undegraded': '2023-11-06T02:13:33.241683+0000', 'last_fullsized': '2023-11-06T02:13:33.241683+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_clean_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T05:14:21.400766+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-06T02:13:38.468 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1c is not active+clean 2023-11-06T02:13:38.468 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-06T02:13:33.249993+0000', 'last_change': '2023-11-06T02:13:33.249993+0000', 'last_active': '2023-11-06T02:13:33.224257+0000', 'last_peered': '2023-11-06T02:13:33.224257+0000', 'last_clean': '2023-11-06T02:13:33.224257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:13:33.249993+0000', 'last_undegraded': '2023-11-06T02:13:33.249993+0000', 'last_fullsized': '2023-11-06T02:13:33.249993+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_clean_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T10:11:19.807848+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-06T02:13:38.469 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1b is not active+clean 2023-11-06T02:13:38.469 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1b', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 78, 'state': 'creating+activating', 'last_fresh': '2023-11-06T02:13:33.240593+0000', 'last_change': '2023-11-06T02:13:33.240593+0000', 'last_active': '2023-11-06T02:13:32.214782+0000', 'last_peered': '2023-11-06T02:13:32.214782+0000', 'last_clean': '2023-11-06T02:13:32.214782+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:13:33.240593+0000', 'last_undegraded': '2023-11-06T02:13:33.240593+0000', 'last_fullsized': '2023-11-06T02:13:33.240593+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:13:32.214782+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:13:32.214782+0000', 'last_clean_scrub_stamp': '2023-11-06T02:13:32.214782+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T12:33:31.177122+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': ['4', '7'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-06T02:13:38.469 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1d is not active+clean 2023-11-06T02:13:38.470 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-06T02:13:33.249061+0000', 'last_change': '2023-11-06T02:13:33.249061+0000', 'last_active': '2023-11-06T02:13:33.224257+0000', 'last_peered': '2023-11-06T02:13:33.224257+0000', 'last_clean': '2023-11-06T02:13:33.224257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:13:33.249061+0000', 'last_undegraded': '2023-11-06T02:13:33.249061+0000', 'last_fullsized': '2023-11-06T02:13:33.249061+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_clean_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T03:16:52.758810+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-06T02:13:38.470 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1e is not active+clean 2023-11-06T02:13:38.470 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-06T02:13:33.242755+0000', 'last_change': '2023-11-06T02:13:33.242755+0000', 'last_active': '2023-11-06T02:13:33.224257+0000', 'last_peered': '2023-11-06T02:13:33.224257+0000', 'last_clean': '2023-11-06T02:13:33.224257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:13:33.242755+0000', 'last_undegraded': '2023-11-06T02:13:33.242755+0000', 'last_fullsized': '2023-11-06T02:13:33.242755+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_clean_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T04:16:36.406664+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-06T02:13:38.471 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.19 is not active+clean 2023-11-06T02:13:38.471 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.19', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 78, 'state': 'creating+activating', 'last_fresh': '2023-11-06T02:13:33.243828+0000', 'last_change': '2023-11-06T02:13:33.243828+0000', 'last_active': '2023-11-06T02:13:32.214782+0000', 'last_peered': '2023-11-06T02:13:32.214782+0000', 'last_clean': '2023-11-06T02:13:32.214782+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:13:33.243828+0000', 'last_undegraded': '2023-11-06T02:13:33.243828+0000', 'last_fullsized': '2023-11-06T02:13:33.243828+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:13:32.214782+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:13:32.214782+0000', 'last_clean_scrub_stamp': '2023-11-06T02:13:32.214782+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T11:54:32.434908+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': ['0', '7'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-06T02:13:38.472 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1f is not active+clean 2023-11-06T02:13:38.472 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-06T02:13:33.245643+0000', 'last_change': '2023-11-06T02:13:33.245643+0000', 'last_active': '2023-11-06T02:13:33.224257+0000', 'last_peered': '2023-11-06T02:13:33.224257+0000', 'last_clean': '2023-11-06T02:13:33.224257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:13:33.245643+0000', 'last_undegraded': '2023-11-06T02:13:33.245643+0000', 'last_fullsized': '2023-11-06T02:13:33.245643+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_clean_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T08:33:50.179908+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-06T02:13:38.472 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.18 is not active+clean 2023-11-06T02:13:38.473 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.18', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 78, 'state': 'creating+activating', 'last_fresh': '2023-11-06T02:13:33.235521+0000', 'last_change': '2023-11-06T02:13:33.235521+0000', 'last_active': '2023-11-06T02:13:32.214782+0000', 'last_peered': '2023-11-06T02:13:32.214782+0000', 'last_clean': '2023-11-06T02:13:32.214782+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:13:33.235521+0000', 'last_undegraded': '2023-11-06T02:13:33.235521+0000', 'last_fullsized': '2023-11-06T02:13:33.235521+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:13:32.214782+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:13:32.214782+0000', 'last_clean_scrub_stamp': '2023-11-06T02:13:32.214782+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T11:39:03.018907+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': ['2', '4'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-06T02:13:38.473 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.10 is not active+clean 2023-11-06T02:13:38.473 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-06T02:13:33.249061+0000', 'last_change': '2023-11-06T02:13:33.249061+0000', 'last_active': '2023-11-06T02:13:33.224257+0000', 'last_peered': '2023-11-06T02:13:33.224257+0000', 'last_clean': '2023-11-06T02:13:33.224257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:13:33.249061+0000', 'last_undegraded': '2023-11-06T02:13:33.249061+0000', 'last_fullsized': '2023-11-06T02:13:33.249061+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_clean_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T03:32:02.459972+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-06T02:13:38.474 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.17 is not active+clean 2023-11-06T02:13:38.474 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.17', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 78, 'state': 'creating+activating', 'last_fresh': '2023-11-06T02:13:33.259605+0000', 'last_change': '2023-11-06T02:13:33.259605+0000', 'last_active': '2023-11-06T02:13:32.214782+0000', 'last_peered': '2023-11-06T02:13:32.214782+0000', 'last_clean': '2023-11-06T02:13:32.214782+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:13:33.259605+0000', 'last_undegraded': '2023-11-06T02:13:33.259605+0000', 'last_fullsized': '2023-11-06T02:13:33.259605+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:13:32.214782+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:13:32.214782+0000', 'last_clean_scrub_stamp': '2023-11-06T02:13:32.214782+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T13:53:46.349156+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': ['7', '6'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-06T02:13:38.475 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.11 is not active+clean 2023-11-06T02:13:38.475 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-06T02:13:33.253727+0000', 'last_change': '2023-11-06T02:13:33.253727+0000', 'last_active': '2023-11-06T02:13:33.224257+0000', 'last_peered': '2023-11-06T02:13:33.224257+0000', 'last_clean': '2023-11-06T02:13:33.224257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:13:33.253727+0000', 'last_undegraded': '2023-11-06T02:13:33.253727+0000', 'last_fullsized': '2023-11-06T02:13:33.253727+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_clean_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T09:00:13.166796+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-06T02:13:38.475 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.12 is not active+clean 2023-11-06T02:13:38.476 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-06T02:13:33.256899+0000', 'last_change': '2023-11-06T02:13:33.256899+0000', 'last_active': '2023-11-06T02:13:33.224257+0000', 'last_peered': '2023-11-06T02:13:33.224257+0000', 'last_clean': '2023-11-06T02:13:33.224257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:13:33.256899+0000', 'last_undegraded': '2023-11-06T02:13:33.256899+0000', 'last_fullsized': '2023-11-06T02:13:33.256899+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_clean_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T10:45:05.474573+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-06T02:13:38.476 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.15 is not active+clean 2023-11-06T02:13:38.476 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.15', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 78, 'state': 'creating+activating', 'last_fresh': '2023-11-06T02:13:33.235632+0000', 'last_change': '2023-11-06T02:13:33.235632+0000', 'last_active': '2023-11-06T02:13:32.214782+0000', 'last_peered': '2023-11-06T02:13:32.214782+0000', 'last_clean': '2023-11-06T02:13:32.214782+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:13:33.235632+0000', 'last_undegraded': '2023-11-06T02:13:33.235632+0000', 'last_fullsized': '2023-11-06T02:13:33.235632+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:13:32.214782+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:13:32.214782+0000', 'last_clean_scrub_stamp': '2023-11-06T02:13:32.214782+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T11:55:36.788472+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': ['2', '1'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-06T02:13:38.477 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.13 is not active+clean 2023-11-06T02:13:38.477 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-06T02:13:33.251869+0000', 'last_change': '2023-11-06T02:13:33.251869+0000', 'last_active': '2023-11-06T02:13:33.224257+0000', 'last_peered': '2023-11-06T02:13:33.224257+0000', 'last_clean': '2023-11-06T02:13:33.224257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:13:33.251869+0000', 'last_undegraded': '2023-11-06T02:13:33.251869+0000', 'last_fullsized': '2023-11-06T02:13:33.251869+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_clean_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T08:33:50.179908+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-06T02:13:38.478 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.14 is not active+clean 2023-11-06T02:13:38.478 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-06T02:13:33.253815+0000', 'last_change': '2023-11-06T02:13:33.253815+0000', 'last_active': '2023-11-06T02:13:33.224257+0000', 'last_peered': '2023-11-06T02:13:33.224257+0000', 'last_clean': '2023-11-06T02:13:33.224257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:13:33.253815+0000', 'last_undegraded': '2023-11-06T02:13:33.253815+0000', 'last_fullsized': '2023-11-06T02:13:33.253815+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_clean_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T13:17:59.025543+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-06T02:13:38.479 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.15 is not active+clean 2023-11-06T02:13:38.479 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-06T02:13:33.240661+0000', 'last_change': '2023-11-06T02:13:33.240661+0000', 'last_active': '2023-11-06T02:13:33.224257+0000', 'last_peered': '2023-11-06T02:13:33.224257+0000', 'last_clean': '2023-11-06T02:13:33.224257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:13:33.240661+0000', 'last_undegraded': '2023-11-06T02:13:33.240661+0000', 'last_fullsized': '2023-11-06T02:13:33.240661+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_clean_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T10:17:36.098143+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-06T02:13:38.479 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.12 is not active+clean 2023-11-06T02:13:38.480 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.12', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 78, 'state': 'creating+activating', 'last_fresh': '2023-11-06T02:13:33.256327+0000', 'last_change': '2023-11-06T02:13:33.256327+0000', 'last_active': '2023-11-06T02:13:32.214782+0000', 'last_peered': '2023-11-06T02:13:32.214782+0000', 'last_clean': '2023-11-06T02:13:32.214782+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:13:33.256327+0000', 'last_undegraded': '2023-11-06T02:13:33.256327+0000', 'last_fullsized': '2023-11-06T02:13:33.256327+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:13:32.214782+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:13:32.214782+0000', 'last_clean_scrub_stamp': '2023-11-06T02:13:32.214782+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T12:53:59.721435+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': ['7', '2'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-06T02:13:38.480 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.16 is not active+clean 2023-11-06T02:13:38.480 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-06T02:13:33.241911+0000', 'last_change': '2023-11-06T02:13:33.241911+0000', 'last_active': '2023-11-06T02:13:33.224257+0000', 'last_peered': '2023-11-06T02:13:33.224257+0000', 'last_clean': '2023-11-06T02:13:33.224257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:13:33.241911+0000', 'last_undegraded': '2023-11-06T02:13:33.241911+0000', 'last_fullsized': '2023-11-06T02:13:33.241911+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_clean_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T14:10:52.936432+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-06T02:13:38.481 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.11 is not active+clean 2023-11-06T02:13:38.481 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.11', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 78, 'state': 'creating+activating', 'last_fresh': '2023-11-06T02:13:33.235562+0000', 'last_change': '2023-11-06T02:13:33.235562+0000', 'last_active': '2023-11-06T02:13:32.214782+0000', 'last_peered': '2023-11-06T02:13:32.214782+0000', 'last_clean': '2023-11-06T02:13:32.214782+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:13:33.235562+0000', 'last_undegraded': '2023-11-06T02:13:33.235562+0000', 'last_fullsized': '2023-11-06T02:13:33.235562+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:13:32.214782+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:13:32.214782+0000', 'last_clean_scrub_stamp': '2023-11-06T02:13:32.214782+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T13:30:40.192944+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-06T02:13:38.482 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.17 is not active+clean 2023-11-06T02:13:38.482 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-06T02:13:33.244479+0000', 'last_change': '2023-11-06T02:13:33.244479+0000', 'last_active': '2023-11-06T02:13:33.224257+0000', 'last_peered': '2023-11-06T02:13:33.224257+0000', 'last_clean': '2023-11-06T02:13:33.224257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:13:33.244479+0000', 'last_undegraded': '2023-11-06T02:13:33.244479+0000', 'last_fullsized': '2023-11-06T02:13:33.244479+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_clean_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T02:21:05.055643+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-06T02:13:38.482 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.10 is not active+clean 2023-11-06T02:13:38.483 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.10', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 78, 'state': 'creating+activating', 'last_fresh': '2023-11-06T02:13:33.247845+0000', 'last_change': '2023-11-06T02:13:33.247845+0000', 'last_active': '2023-11-06T02:13:32.214782+0000', 'last_peered': '2023-11-06T02:13:32.214782+0000', 'last_clean': '2023-11-06T02:13:32.214782+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:13:33.247845+0000', 'last_undegraded': '2023-11-06T02:13:33.247845+0000', 'last_fullsized': '2023-11-06T02:13:33.247845+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:13:32.214782+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:13:32.214782+0000', 'last_clean_scrub_stamp': '2023-11-06T02:13:32.214782+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T13:17:58.016068+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': ['0', '6'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-06T02:13:38.483 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.0 is not active+clean 2023-11-06T02:13:38.483 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-06T02:13:33.241033+0000', 'last_change': '2023-11-06T02:13:33.241033+0000', 'last_active': '2023-11-06T02:13:33.224257+0000', 'last_peered': '2023-11-06T02:13:33.224257+0000', 'last_clean': '2023-11-06T02:13:33.224257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:13:33.241033+0000', 'last_undegraded': '2023-11-06T02:13:33.241033+0000', 'last_fullsized': '2023-11-06T02:13:33.241033+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_clean_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T12:54:00.730910+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-06T02:13:38.484 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1 is not active+clean 2023-11-06T02:13:38.484 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-06T02:13:33.238294+0000', 'last_change': '2023-11-06T02:13:33.238294+0000', 'last_active': '2023-11-06T02:13:33.224257+0000', 'last_peered': '2023-11-06T02:13:33.224257+0000', 'last_clean': '2023-11-06T02:13:33.224257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:13:33.238294+0000', 'last_undegraded': '2023-11-06T02:13:33.238294+0000', 'last_fullsized': '2023-11-06T02:13:33.238294+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_clean_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T14:02:47.316294+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-06T02:13:38.485 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.6 is not active+clean 2023-11-06T02:13:38.485 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.6', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 78, 'state': 'creating+activating', 'last_fresh': '2023-11-06T02:13:33.244338+0000', 'last_change': '2023-11-06T02:13:33.244338+0000', 'last_active': '2023-11-06T02:13:32.214782+0000', 'last_peered': '2023-11-06T02:13:32.214782+0000', 'last_clean': '2023-11-06T02:13:32.214782+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:13:33.244338+0000', 'last_undegraded': '2023-11-06T02:13:33.244338+0000', 'last_fullsized': '2023-11-06T02:13:33.244338+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:13:32.214782+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:13:32.214782+0000', 'last_clean_scrub_stamp': '2023-11-06T02:13:32.214782+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T09:11:59.707658+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': ['0', '2'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-06T02:13:38.485 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.2 is not active+clean 2023-11-06T02:13:38.486 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-06T02:13:33.245156+0000', 'last_change': '2023-11-06T02:13:33.245156+0000', 'last_active': '2023-11-06T02:13:33.224257+0000', 'last_peered': '2023-11-06T02:13:33.224257+0000', 'last_clean': '2023-11-06T02:13:33.224257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:13:33.245156+0000', 'last_undegraded': '2023-11-06T02:13:33.245156+0000', 'last_fullsized': '2023-11-06T02:13:33.245156+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_clean_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T14:07:57.793830+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-06T02:13:38.486 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.5 is not active+clean 2023-11-06T02:13:38.487 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.5', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 78, 'state': 'creating+activating', 'last_fresh': '2023-11-06T02:13:33.258159+0000', 'last_change': '2023-11-06T02:13:33.258159+0000', 'last_active': '2023-11-06T02:13:32.214782+0000', 'last_peered': '2023-11-06T02:13:32.214782+0000', 'last_clean': '2023-11-06T02:13:32.214782+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:13:33.258159+0000', 'last_undegraded': '2023-11-06T02:13:33.258159+0000', 'last_fullsized': '2023-11-06T02:13:33.258159+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:13:32.214782+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:13:32.214782+0000', 'last_clean_scrub_stamp': '2023-11-06T02:13:32.214782+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T04:16:35.397189+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': ['7', '5'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-06T02:13:38.487 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.3 is not active+clean 2023-11-06T02:13:38.487 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-06T02:13:33.248850+0000', 'last_change': '2023-11-06T02:13:33.248850+0000', 'last_active': '2023-11-06T02:13:33.224257+0000', 'last_peered': '2023-11-06T02:13:33.224257+0000', 'last_clean': '2023-11-06T02:13:33.224257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:13:33.248850+0000', 'last_undegraded': '2023-11-06T02:13:33.248850+0000', 'last_fullsized': '2023-11-06T02:13:33.248850+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_clean_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T12:22:47.360905+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-06T02:13:38.488 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.4 is not active+clean 2023-11-06T02:13:38.488 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.4', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 78, 'state': 'creating+activating', 'last_fresh': '2023-11-06T02:13:33.256366+0000', 'last_change': '2023-11-06T02:13:33.256366+0000', 'last_active': '2023-11-06T02:13:32.214782+0000', 'last_peered': '2023-11-06T02:13:32.214782+0000', 'last_clean': '2023-11-06T02:13:32.214782+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:13:33.256366+0000', 'last_undegraded': '2023-11-06T02:13:33.256366+0000', 'last_fullsized': '2023-11-06T02:13:33.256366+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:13:32.214782+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:13:32.214782+0000', 'last_clean_scrub_stamp': '2023-11-06T02:13:32.214782+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T03:12:35.766676+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': ['7', '6'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-06T02:13:38.489 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.4 is not active+clean 2023-11-06T02:13:38.489 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-06T02:13:33.244536+0000', 'last_change': '2023-11-06T02:13:33.244536+0000', 'last_active': '2023-11-06T02:13:33.224257+0000', 'last_peered': '2023-11-06T02:13:33.224257+0000', 'last_clean': '2023-11-06T02:13:33.224257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:13:33.244536+0000', 'last_undegraded': '2023-11-06T02:13:33.244536+0000', 'last_fullsized': '2023-11-06T02:13:33.244536+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_clean_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T08:42:42.388106+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-06T02:13:38.489 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.5 is not active+clean 2023-11-06T02:13:38.490 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-06T02:13:33.239368+0000', 'last_change': '2023-11-06T02:13:33.239368+0000', 'last_active': '2023-11-06T02:13:33.224257+0000', 'last_peered': '2023-11-06T02:13:33.224257+0000', 'last_clean': '2023-11-06T02:13:33.224257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:13:33.239368+0000', 'last_undegraded': '2023-11-06T02:13:33.239368+0000', 'last_fullsized': '2023-11-06T02:13:33.239368+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_clean_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T06:31:19.083004+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-06T02:13:38.490 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2 is not active+clean 2023-11-06T02:13:38.491 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 78, 'state': 'creating+activating', 'last_fresh': '2023-11-06T02:13:33.235609+0000', 'last_change': '2023-11-06T02:13:33.235609+0000', 'last_active': '2023-11-06T02:13:32.214782+0000', 'last_peered': '2023-11-06T02:13:32.214782+0000', 'last_clean': '2023-11-06T02:13:32.214782+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:13:33.235609+0000', 'last_undegraded': '2023-11-06T02:13:33.235609+0000', 'last_fullsized': '2023-11-06T02:13:33.235609+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:13:32.214782+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:13:32.214782+0000', 'last_clean_scrub_stamp': '2023-11-06T02:13:32.214782+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T06:19:20.386051+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-06T02:13:38.491 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.6 is not active+clean 2023-11-06T02:13:38.491 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-06T02:13:33.254957+0000', 'last_change': '2023-11-06T02:13:33.254957+0000', 'last_active': '2023-11-06T02:13:33.224257+0000', 'last_peered': '2023-11-06T02:13:33.224257+0000', 'last_clean': '2023-11-06T02:13:33.224257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:13:33.254957+0000', 'last_undegraded': '2023-11-06T02:13:33.254957+0000', 'last_fullsized': '2023-11-06T02:13:33.254957+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_clean_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T14:02:47.316294+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-06T02:13:38.492 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.7 is not active+clean 2023-11-06T02:13:38.492 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-06T02:13:33.253013+0000', 'last_change': '2023-11-06T02:13:33.253013+0000', 'last_active': '2023-11-06T02:13:33.224257+0000', 'last_peered': '2023-11-06T02:13:33.224257+0000', 'last_clean': '2023-11-06T02:13:33.224257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:13:33.253013+0000', 'last_undegraded': '2023-11-06T02:13:33.253013+0000', 'last_fullsized': '2023-11-06T02:13:33.253013+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_clean_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T06:31:19.083004+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-06T02:13:38.492 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.0 is not active+clean 2023-11-06T02:13:38.493 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.0', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 78, 'state': 'creating+activating', 'last_fresh': '2023-11-06T02:13:33.251425+0000', 'last_change': '2023-11-06T02:13:33.251425+0000', 'last_active': '2023-11-06T02:13:32.214782+0000', 'last_peered': '2023-11-06T02:13:32.214782+0000', 'last_clean': '2023-11-06T02:13:32.214782+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:13:33.251425+0000', 'last_undegraded': '2023-11-06T02:13:33.251425+0000', 'last_fullsized': '2023-11-06T02:13:33.251425+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:13:32.214782+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:13:32.214782+0000', 'last_clean_scrub_stamp': '2023-11-06T02:13:32.214782+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T08:13:44.482660+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': ['1', '3'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-06T02:13:38.493 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.d is not active+clean 2023-11-06T02:13:38.494 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-06T02:13:33.252952+0000', 'last_change': '2023-11-06T02:13:33.252952+0000', 'last_active': '2023-11-06T02:13:33.224257+0000', 'last_peered': '2023-11-06T02:13:33.224257+0000', 'last_clean': '2023-11-06T02:13:33.224257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:13:33.252952+0000', 'last_undegraded': '2023-11-06T02:13:33.252952+0000', 'last_fullsized': '2023-11-06T02:13:33.252952+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_clean_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T09:12:00.717133+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-06T02:13:38.494 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.a is not active+clean 2023-11-06T02:13:38.494 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.a', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 78, 'state': 'creating+activating', 'last_fresh': '2023-11-06T02:13:33.253882+0000', 'last_change': '2023-11-06T02:13:33.253882+0000', 'last_active': '2023-11-06T02:13:32.214782+0000', 'last_peered': '2023-11-06T02:13:32.214782+0000', 'last_clean': '2023-11-06T02:13:32.214782+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:13:33.253882+0000', 'last_undegraded': '2023-11-06T02:13:33.253882+0000', 'last_fullsized': '2023-11-06T02:13:33.253882+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:13:32.214782+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:13:32.214782+0000', 'last_clean_scrub_stamp': '2023-11-06T02:13:32.214782+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T03:16:51.749335+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': ['3', '6'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-06T02:13:38.495 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.e is not active+clean 2023-11-06T02:13:38.495 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-06T02:13:33.250595+0000', 'last_change': '2023-11-06T02:13:33.250595+0000', 'last_active': '2023-11-06T02:13:33.224257+0000', 'last_peered': '2023-11-06T02:13:33.224257+0000', 'last_clean': '2023-11-06T02:13:33.224257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:13:33.250595+0000', 'last_undegraded': '2023-11-06T02:13:33.250595+0000', 'last_fullsized': '2023-11-06T02:13:33.250595+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_clean_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T09:46:01.922659+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-06T02:13:38.495 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.9 is not active+clean 2023-11-06T02:13:38.496 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.9', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 78, 'state': 'creating+activating', 'last_fresh': '2023-11-06T02:13:33.246452+0000', 'last_change': '2023-11-06T02:13:33.246452+0000', 'last_active': '2023-11-06T02:13:32.214782+0000', 'last_peered': '2023-11-06T02:13:32.214782+0000', 'last_clean': '2023-11-06T02:13:32.214782+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:13:33.246452+0000', 'last_undegraded': '2023-11-06T02:13:33.246452+0000', 'last_fullsized': '2023-11-06T02:13:33.246452+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:13:32.214782+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:13:32.214782+0000', 'last_clean_scrub_stamp': '2023-11-06T02:13:32.214782+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T10:11:51.109403+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': ['3', '6'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-06T02:13:38.496 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.f is not active+clean 2023-11-06T02:13:38.496 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-06T02:13:33.243210+0000', 'last_change': '2023-11-06T02:13:33.243210+0000', 'last_active': '2023-11-06T02:13:33.224257+0000', 'last_peered': '2023-11-06T02:13:33.224257+0000', 'last_clean': '2023-11-06T02:13:33.224257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:13:33.243210+0000', 'last_undegraded': '2023-11-06T02:13:33.243210+0000', 'last_fullsized': '2023-11-06T02:13:33.243210+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_clean_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T12:33:32.186597+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-06T02:13:38.497 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.8 is not active+clean 2023-11-06T02:13:38.497 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.8', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 78, 'state': 'creating+activating', 'last_fresh': '2023-11-06T02:13:33.244333+0000', 'last_change': '2023-11-06T02:13:33.244333+0000', 'last_active': '2023-11-06T02:13:32.214782+0000', 'last_peered': '2023-11-06T02:13:32.214782+0000', 'last_clean': '2023-11-06T02:13:32.214782+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:13:33.244333+0000', 'last_undegraded': '2023-11-06T02:13:33.244333+0000', 'last_fullsized': '2023-11-06T02:13:33.244333+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:13:32.214782+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:13:32.214782+0000', 'last_clean_scrub_stamp': '2023-11-06T02:13:32.214782+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T11:51:52.147063+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': ['3', '6'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-06T02:13:38.497 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.c is not active+clean 2023-11-06T02:13:38.498 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-06T02:13:33.249368+0000', 'last_change': '2023-11-06T02:13:33.249368+0000', 'last_active': '2023-11-06T02:13:33.224257+0000', 'last_peered': '2023-11-06T02:13:33.224257+0000', 'last_clean': '2023-11-06T02:13:33.224257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:13:33.249368+0000', 'last_undegraded': '2023-11-06T02:13:33.249368+0000', 'last_fullsized': '2023-11-06T02:13:33.249368+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_clean_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T10:54:11.389133+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-06T02:13:38.498 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.b is not active+clean 2023-11-06T02:13:38.499 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.b', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 78, 'state': 'creating+activating', 'last_fresh': '2023-11-06T02:13:33.245263+0000', 'last_change': '2023-11-06T02:13:33.245263+0000', 'last_active': '2023-11-06T02:13:32.214782+0000', 'last_peered': '2023-11-06T02:13:32.214782+0000', 'last_clean': '2023-11-06T02:13:32.214782+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:13:33.245263+0000', 'last_undegraded': '2023-11-06T02:13:33.245263+0000', 'last_fullsized': '2023-11-06T02:13:33.245263+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:13:32.214782+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:13:32.214782+0000', 'last_clean_scrub_stamp': '2023-11-06T02:13:32.214782+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T07:38:14.971699+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': ['3', '1'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-06T02:13:38.499 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.a is not active+clean 2023-11-06T02:13:38.499 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-06T02:13:33.242981+0000', 'last_change': '2023-11-06T02:13:33.242981+0000', 'last_active': '2023-11-06T02:13:33.224257+0000', 'last_peered': '2023-11-06T02:13:33.224257+0000', 'last_clean': '2023-11-06T02:13:33.224257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:13:33.242981+0000', 'last_undegraded': '2023-11-06T02:13:33.242981+0000', 'last_fullsized': '2023-11-06T02:13:33.242981+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_clean_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T12:36:12.474443+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-06T02:13:38.500 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.d is not active+clean 2023-11-06T02:13:38.500 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.d', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 78, 'state': 'creating+activating', 'last_fresh': '2023-11-06T02:13:33.236266+0000', 'last_change': '2023-11-06T02:13:33.236266+0000', 'last_active': '2023-11-06T02:13:32.214782+0000', 'last_peered': '2023-11-06T02:13:32.214782+0000', 'last_clean': '2023-11-06T02:13:32.214782+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:13:33.236266+0000', 'last_undegraded': '2023-11-06T02:13:33.236266+0000', 'last_fullsized': '2023-11-06T02:13:33.236266+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:13:32.214782+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:13:32.214782+0000', 'last_clean_scrub_stamp': '2023-11-06T02:13:32.214782+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T03:30:05.141853+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': ['2', '4'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-06T02:13:38.501 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.b is not active+clean 2023-11-06T02:13:38.501 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-06T02:13:33.246317+0000', 'last_change': '2023-11-06T02:13:33.246317+0000', 'last_active': '2023-11-06T02:13:33.224257+0000', 'last_peered': '2023-11-06T02:13:33.224257+0000', 'last_clean': '2023-11-06T02:13:33.224257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:13:33.246317+0000', 'last_undegraded': '2023-11-06T02:13:33.246317+0000', 'last_fullsized': '2023-11-06T02:13:33.246317+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_clean_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T10:25:19.608926+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-06T02:13:38.501 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.c is not active+clean 2023-11-06T02:13:38.502 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.c', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 78, 'state': 'creating+activating', 'last_fresh': '2023-11-06T02:13:33.251742+0000', 'last_change': '2023-11-06T02:13:33.251742+0000', 'last_active': '2023-11-06T02:13:32.214782+0000', 'last_peered': '2023-11-06T02:13:32.214782+0000', 'last_clean': '2023-11-06T02:13:32.214782+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:13:33.251742+0000', 'last_undegraded': '2023-11-06T02:13:33.251742+0000', 'last_fullsized': '2023-11-06T02:13:33.251742+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:13:32.214782+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:13:32.214782+0000', 'last_clean_scrub_stamp': '2023-11-06T02:13:32.214782+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T11:23:42.695735+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': ['1', '2'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-06T02:13:38.502 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.8 is not active+clean 2023-11-06T02:13:38.502 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-06T02:13:33.244818+0000', 'last_change': '2023-11-06T02:13:33.244818+0000', 'last_active': '2023-11-06T02:13:33.224257+0000', 'last_peered': '2023-11-06T02:13:33.224257+0000', 'last_clean': '2023-11-06T02:13:33.224257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:13:33.244818+0000', 'last_undegraded': '2023-11-06T02:13:33.244818+0000', 'last_fullsized': '2023-11-06T02:13:33.244818+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_clean_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T08:22:01.334447+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-06T02:13:38.503 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.f is not active+clean 2023-11-06T02:13:38.503 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.f', 'version': "0'0", 'reported_seq': 4, 'reported_epoch': 78, 'state': 'creating+activating', 'last_fresh': '2023-11-06T02:13:33.269809+0000', 'last_change': '2023-11-06T02:13:33.260450+0000', 'last_active': '2023-11-06T02:13:32.214782+0000', 'last_peered': '2023-11-06T02:13:32.214782+0000', 'last_clean': '2023-11-06T02:13:32.214782+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:13:33.269809+0000', 'last_undegraded': '2023-11-06T02:13:33.269809+0000', 'last_fullsized': '2023-11-06T02:13:33.269809+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:13:32.214782+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:13:32.214782+0000', 'last_clean_scrub_stamp': '2023-11-06T02:13:32.214782+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T11:51:47.363705+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': ['7', '1'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-06T02:13:38.504 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.9 is not active+clean 2023-11-06T02:13:38.504 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-06T02:13:33.242736+0000', 'last_change': '2023-11-06T02:13:33.242736+0000', 'last_active': '2023-11-06T02:13:33.224257+0000', 'last_peered': '2023-11-06T02:13:33.224257+0000', 'last_clean': '2023-11-06T02:13:33.224257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T02:13:33.242736+0000', 'last_undegraded': '2023-11-06T02:13:33.242736+0000', 'last_fullsized': '2023-11-06T02:13:33.242736+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'last_clean_scrub_stamp': '2023-11-06T02:13:33.224257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T10:38:31.470172+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-06T02:13:38.505 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-11-06T02:13:38.630 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.634+0000 7fb7bc719640 1 Processor -- start 2023-11-06T02:13:38.630 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.634+0000 7fb7bc719640 1 -- start start 2023-11-06T02:13:38.633 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.634+0000 7fb7bc719640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7b40cb010 0x7fb7b40ca980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:38.633 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.634+0000 7fb7bc719640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fb7b40cdd50 0x7fb7b40cc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:38.633 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.634+0000 7fb7bc719640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7b40723a0 con 0x7fb7b4159b40 2023-11-06T02:13:38.634 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.634+0000 7fb7bc719640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7b40715e0 con 0x7fb7b40cdd50 2023-11-06T02:13:38.634 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.634+0000 7fb7bc719640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b40718c0 con 0x7fb7b40cb010 2023-11-06T02:13:38.634 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.634+0000 7fb7b9cb4640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fb7b4159b40 0x7fb7b40cb3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:60844/0 (socket says 172.21.15.49:60844) 2023-11-06T02:13:38.634 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.634+0000 7fb7b9cb4640 1 -- 172.21.15.49:0/2858787432 learned_addr learned my addr 172.21.15.49:0/2858787432 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:13:38.634 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.634+0000 7fb7bacb6640 1 --2- 172.21.15.49:0/2858787432 >> v2:172.21.15.150:3300/0 conn(0x7fb7b40cdd50 0x7fb7b40cc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:38.635 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.634+0000 7fb7ba4b5640 1 --2- 172.21.15.49:0/2858787432 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7b40cb010 0x7fb7b40ca980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:38.635 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.634+0000 7fb7b94b3640 1 -- 172.21.15.49:0/2858787432 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2969722371 0 0) 0x7fb7b40723a0 con 0x7fb7b4159b40 2023-11-06T02:13:38.635 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.634+0000 7fb7b94b3640 1 -- 172.21.15.49:0/2858787432 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb79c003530 con 0x7fb7b4159b40 2023-11-06T02:13:38.635 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.638+0000 7fb7b94b3640 1 -- 172.21.15.49:0/2858787432 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2246430147 0 0) 0x7fb79c003530 con 0x7fb7b4159b40 2023-11-06T02:13:38.635 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.638+0000 7fb7b94b3640 1 -- 172.21.15.49:0/2858787432 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7b40cb010 msgr2=0x7fb7b40ca980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:38.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.638+0000 7fb7b94b3640 1 --2- 172.21.15.49:0/2858787432 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7b40cb010 0x7fb7b40ca980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:38.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.638+0000 7fb7b94b3640 1 -- 172.21.15.49:0/2858787432 >> v2:172.21.15.150:3300/0 conn(0x7fb7b40cdd50 msgr2=0x7fb7b40cc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:38.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.638+0000 7fb7b94b3640 1 --2- 172.21.15.49:0/2858787432 >> v2:172.21.15.150:3300/0 conn(0x7fb7b40cdd50 0x7fb7b40cc1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:38.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.638+0000 7fb7b94b3640 1 -- 172.21.15.49:0/2858787432 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7b4071ba0 con 0x7fb7b4159b40 2023-11-06T02:13:38.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.638+0000 7fb7b94b3640 1 -- 172.21.15.49:0/2858787432 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7fb7a8001a00 con 0x7fb7b4159b40 2023-11-06T02:13:38.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.638+0000 7fb7b94b3640 1 -- 172.21.15.49:0/2858787432 >> v1:172.21.15.49:6789/0 conn(0x7fb7b4159b40 legacy=0x7fb7b40cb3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:38.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.638+0000 7fb7b94b3640 1 --2- 172.21.15.49:0/2858787432 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb79c003980 0x7fb79c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:38.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.638+0000 7fb7b94b3640 1 --2- 172.21.15.49:0/2858787432 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb79c0047d0 0x7fb79c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:38.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.638+0000 7fb7b94b3640 1 --2- 172.21.15.49:0/2858787432 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7b40cb010 0x7fb79c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:38.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.638+0000 7fb7b94b3640 1 -- 172.21.15.49:0/2858787432 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b40715e0 con 0x7fb79c0047d0 2023-11-06T02:13:38.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.638+0000 7fb7b94b3640 1 -- 172.21.15.49:0/2858787432 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b40718c0 con 0x7fb79c003980 2023-11-06T02:13:38.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.638+0000 7fb7b94b3640 1 -- 172.21.15.49:0/2858787432 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb79c00dba0 con 0x7fb7b40cb010 2023-11-06T02:13:38.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.638+0000 7fb7b9cb4640 1 --2- 172.21.15.49:0/2858787432 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb79c0047d0 0x7fb79c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:38.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.638+0000 7fb7ba4b5640 1 --2- 172.21.15.49:0/2858787432 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb79c003980 0x7fb79c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:38.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.638+0000 7fb7b9cb4640 1 -- 172.21.15.49:0/2858787432 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7b40cb010 msgr2=0x7fb79c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:38.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.638+0000 7fb7b9cb4640 1 --2- 172.21.15.49:0/2858787432 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7b40cb010 0x7fb79c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:38.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.638+0000 7fb7b9cb4640 1 -- 172.21.15.49:0/2858787432 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb79c003980 msgr2=0x7fb79c003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:38.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.638+0000 7fb7b9cb4640 1 --2- 172.21.15.49:0/2858787432 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb79c003980 0x7fb79c003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:38.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.638+0000 7fb7b9cb4640 1 -- 172.21.15.49:0/2858787432 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb79c003530 con 0x7fb79c0047d0 2023-11-06T02:13:38.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.638+0000 7fb7b9cb4640 1 --2- 172.21.15.49:0/2858787432 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb79c0047d0 0x7fb79c004c50 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fb7a800a850 tx=0x7fb7a801fac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=941e12bab4497e88 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:38.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.638+0000 7fb7b94b3640 1 -- 172.21.15.49:0/2858787432 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7a80ac670 con 0x7fb79c0047d0 2023-11-06T02:13:38.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.638+0000 7fb7b94b3640 1 -- 172.21.15.49:0/2858787432 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7a80ac7f0 con 0x7fb79c0047d0 2023-11-06T02:13:38.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.638+0000 7fb7b94b3640 1 -- 172.21.15.49:0/2858787432 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7a80acba0 con 0x7fb79c0047d0 2023-11-06T02:13:38.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.638+0000 7fb7bc719640 1 -- 172.21.15.49:0/2858787432 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb79c0047d0 msgr2=0x7fb79c004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:38.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.638+0000 7fb7bc719640 1 --2- 172.21.15.49:0/2858787432 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb79c0047d0 0x7fb79c004c50 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fb7a800a850 tx=0x7fb7a801fac0 comp rx=0 tx=0).stop 2023-11-06T02:13:38.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.638+0000 7fb7bacb6640 1 -- 172.21.15.49:0/2858787432 reap_dead start 2023-11-06T02:13:38.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.638+0000 7fb7bc719640 1 -- 172.21.15.49:0/2858787432 shutdown_connections 2023-11-06T02:13:38.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.638+0000 7fb7bc719640 1 -- 172.21.15.49:0/2858787432 >> 172.21.15.49:0/2858787432 conn(0x7fb7b406c0a0 msgr2=0x7fb7b406c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:38.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.638+0000 7fb7bc719640 1 -- 172.21.15.49:0/2858787432 shutdown_connections 2023-11-06T02:13:38.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.638+0000 7fb7bc719640 1 -- 172.21.15.49:0/2858787432 wait complete. 2023-11-06T02:13:38.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.642+0000 7fb7bc719640 1 Processor -- start 2023-11-06T02:13:38.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.642+0000 7fb7bc719640 1 -- start start 2023-11-06T02:13:38.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.642+0000 7fb7bc719640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb79c0047d0 0x7fb7b40b0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:38.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.642+0000 7fb7bc719640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7b40b28a0 0x7fb7b4157f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:38.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.642+0000 7fb7bc719640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb7b40af030 0x7fb7b40af4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:38.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.642+0000 7fb7bc719640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b40af9f0 con 0x7fb79c0047d0 2023-11-06T02:13:38.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.642+0000 7fb7bc719640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b4158480 con 0x7fb7b40af030 2023-11-06T02:13:38.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.642+0000 7fb7bc719640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b40b3030 con 0x7fb7b40b28a0 2023-11-06T02:13:38.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.642+0000 7fb7ba4b5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb79c0047d0 0x7fb7b40b0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:38.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.642+0000 7fb7ba4b5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb79c0047d0 0x7fb7b40b0220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47878/0 (socket says 172.21.15.49:47878) 2023-11-06T02:13:38.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.642+0000 7fb7ba4b5640 1 -- 172.21.15.49:0/2383362638 learned_addr learned my addr 172.21.15.49:0/2383362638 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:38.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.642+0000 7fb7b9cb4640 1 --2- 172.21.15.49:0/2383362638 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7b40b28a0 0x7fb7b4157f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:38.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.642+0000 7fb7bacb6640 1 --2- 172.21.15.49:0/2383362638 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb7b40af030 0x7fb7b40af4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:38.648 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.642+0000 7fb7ba4b5640 1 -- 172.21.15.49:0/2383362638 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7b40b28a0 msgr2=0x7fb7b4157f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:38.648 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.642+0000 7fb7ba4b5640 1 --2- 172.21.15.49:0/2383362638 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7b40b28a0 0x7fb7b4157f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:38.648 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.642+0000 7fb7ba4b5640 1 -- 172.21.15.49:0/2383362638 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb7b40af030 msgr2=0x7fb7b40af4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:38.648 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.642+0000 7fb7ba4b5640 1 --2- 172.21.15.49:0/2383362638 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb7b40af030 0x7fb7b40af4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:38.648 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.642+0000 7fb7ba4b5640 1 -- 172.21.15.49:0/2383362638 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7b40b3310 con 0x7fb79c0047d0 2023-11-06T02:13:38.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.642+0000 7fb7bacb6640 1 --2- 172.21.15.49:0/2383362638 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb7b40af030 0x7fb7b40af4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:13:38.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.642+0000 7fb7b9cb4640 1 --2- 172.21.15.49:0/2383362638 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7b40b28a0 0x7fb7b4157f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:13:38.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.642+0000 7fb7ba4b5640 1 --2- 172.21.15.49:0/2383362638 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb79c0047d0 0x7fb7b40b0220 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac011590 tx=0x7fb7ac011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:38.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.642+0000 7fb7b2ffd640 1 -- 172.21.15.49:0/2383362638 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7ac015870 con 0x7fb79c0047d0 2023-11-06T02:13:38.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.642+0000 7fb7b2ffd640 1 -- 172.21.15.49:0/2383362638 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7ac0159d0 con 0x7fb79c0047d0 2023-11-06T02:13:38.650 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.642+0000 7fb7bc719640 1 -- 172.21.15.49:0/2383362638 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7b40b35f0 con 0x7fb79c0047d0 2023-11-06T02:13:38.650 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.642+0000 7fb7b2ffd640 1 -- 172.21.15.49:0/2383362638 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7ac015d40 con 0x7fb79c0047d0 2023-11-06T02:13:38.650 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.642+0000 7fb7bc719640 1 -- 172.21.15.49:0/2383362638 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7b40b38d0 con 0x7fb79c0047d0 2023-11-06T02:13:38.650 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.642+0000 7fb7bc719640 1 -- 172.21.15.49:0/2383362638 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7b4074040 con 0x7fb79c0047d0 2023-11-06T02:13:38.650 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.646+0000 7fb7b2ffd640 1 -- 172.21.15.49:0/2383362638 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fb7ac00f460 con 0x7fb79c0047d0 2023-11-06T02:13:38.651 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.646+0000 7fb7b2ffd640 1 --2- 172.21.15.49:0/2383362638 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb7a0065a60 0x7fb7a0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:38.651 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.646+0000 7fb7b2ffd640 1 -- 172.21.15.49:0/2383362638 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb7ac083a20 con 0x7fb79c0047d0 2023-11-06T02:13:38.651 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.646+0000 7fb7b9cb4640 1 --2- 172.21.15.49:0/2383362638 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb7a0065a60 0x7fb7a0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:38.651 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.646+0000 7fb7b9cb4640 1 --2- 172.21.15.49:0/2383362638 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb7a0065a60 0x7fb7a0067f20 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fb7a80b0000 tx=0x7fb7a8001d90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:38.652 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.650+0000 7fb7b2ffd640 1 -- 172.21.15.49:0/2383362638 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fb7ac051280 con 0x7fb79c0047d0 2023-11-06T02:13:38.861 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.866+0000 7fb7bc719640 1 -- 172.21.15.49:0/2383362638 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fb7b40c6f20 con 0x7fb79c0047d0 2023-11-06T02:13:38.861 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.866+0000 7fb7b2ffd640 1 -- 172.21.15.49:0/2383362638 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1082 (secure 0 0 0) 0x7fb7ac022080 con 0x7fb79c0047d0 2023-11-06T02:13:38.862 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:13:38.862 INFO:teuthology.orchestra.run.smithi049.stdout:{"fsid":"1a8b035c-6ba4-499a-b876-aea47fd9c0d4","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":332,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":80,"num_osds":8,"num_up_osds":8,"osd_up_since":1699236497,"num_in_osds":8,"osd_in_since":1699236488,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"creating+activating","count":23},{"state_name":"active+clean","count":18}],"num_pgs":105,"num_pools":4,"num_objects":2,"data_bytes":590368,"bytes_used":254644224,"bytes_avail":772839469056,"bytes_total":773094113280,"inactive_pgs_ratio":0.82857143878936768},"fsmap":{"epoch":46,"id":5,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":5,"rank":0,"name":"c","status":"up:active","gid":7116}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2023-11-06T02:12:37.264581+0000","services":{}},"progress_events":{}} 2023-11-06T02:13:38.865 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.870+0000 7fb7bc719640 1 -- 172.21.15.49:0/2383362638 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb7a0065a60 msgr2=0x7fb7a0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:38.865 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.870+0000 7fb7bc719640 1 --2- 172.21.15.49:0/2383362638 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb7a0065a60 0x7fb7a0067f20 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fb7a80b0000 tx=0x7fb7a8001d90 comp rx=0 tx=0).stop 2023-11-06T02:13:38.865 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.870+0000 7fb7bc719640 1 -- 172.21.15.49:0/2383362638 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb79c0047d0 msgr2=0x7fb7b40b0220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:38.865 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.870+0000 7fb7bc719640 1 --2- 172.21.15.49:0/2383362638 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb79c0047d0 0x7fb7b40b0220 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac011590 tx=0x7fb7ac011a50 comp rx=0 tx=0).stop 2023-11-06T02:13:38.866 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.870+0000 7fb7bc719640 1 -- 172.21.15.49:0/2383362638 shutdown_connections 2023-11-06T02:13:38.866 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.870+0000 7fb7bc719640 1 --2- 172.21.15.49:0/2383362638 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb7a0065a60 0x7fb7a0067f20 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:38.866 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.870+0000 7fb7bc719640 1 --2- 172.21.15.49:0/2383362638 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb7b40af030 0x7fb7b40af4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:38.866 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.870+0000 7fb7bc719640 1 --2- 172.21.15.49:0/2383362638 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7b40b28a0 0x7fb7b4157f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:38.867 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.870+0000 7fb7bc719640 1 --2- 172.21.15.49:0/2383362638 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb79c0047d0 0x7fb7b40b0220 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:38.867 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.870+0000 7fb7bc719640 1 -- 172.21.15.49:0/2383362638 >> 172.21.15.49:0/2383362638 conn(0x7fb7b406c0a0 msgr2=0x7fb7b40c8c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:38.867 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.870+0000 7fb7bc719640 1 -- 172.21.15.49:0/2383362638 shutdown_connections 2023-11-06T02:13:38.867 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:38.870+0000 7fb7bc719640 1 -- 172.21.15.49:0/2383362638 wait complete. 2023-11-06T02:13:38.882 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-11-06T02:13:38.882 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T02:13:39.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:39.010+0000 7f5ceb601640 1 Processor -- start 2023-11-06T02:13:39.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:39.010+0000 7f5ceb601640 1 -- start start 2023-11-06T02:13:39.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:39.014+0000 7f5ceb601640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ce415aab0 0x7f5ce40cb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:39.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:39.014+0000 7f5ceb601640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ce40ce3f0 con 0x7f5ce40cb970 2023-11-06T02:13:39.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:39.014+0000 7f5ceb601640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ce40cbdf0 con 0x7f5ce40cd990 2023-11-06T02:13:39.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:39.014+0000 7f5ceb601640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce415ae90 con 0x7f5ce415aab0 2023-11-06T02:13:39.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:39.014+0000 7f5ce8b9c640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f5ce40cb970 0x7f5ce40cd280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:60854/0 (socket says 172.21.15.49:60854) 2023-11-06T02:13:39.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:39.014+0000 7f5ce8b9c640 1 -- 172.21.15.49:0/1638719162 learned_addr learned my addr 172.21.15.49:0/1638719162 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:13:39.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:39.014+0000 7f5ce939d640 1 --2- 172.21.15.49:0/1638719162 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ce415aab0 0x7f5ce40cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:39.013 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:39.014+0000 7f5cdbfff640 1 -- 172.21.15.49:0/1638719162 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 516407244 0 0) 0x7f5ce40ce3f0 con 0x7f5ce40cb970 2023-11-06T02:13:39.013 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:39.014+0000 7f5cdbfff640 1 -- 172.21.15.49:0/1638719162 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5cd0003530 con 0x7f5ce40cb970 2023-11-06T02:13:39.013 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:39.014+0000 7f5cdbfff640 1 -- 172.21.15.49:0/1638719162 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4148025625 0 0) 0x7f5cd0003530 con 0x7f5ce40cb970 2023-11-06T02:13:39.013 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:39.014+0000 7f5cdbfff640 1 -- 172.21.15.49:0/1638719162 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ce415aab0 msgr2=0x7f5ce40cb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:39.013 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:39.014+0000 7f5cdbfff640 1 --2- 172.21.15.49:0/1638719162 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ce415aab0 0x7f5ce40cb430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:39.014 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:39.014+0000 7f5cdbfff640 1 -- 172.21.15.49:0/1638719162 >> v1:172.21.15.150:6789/0 conn(0x7f5ce40cd990 legacy=0x7f5ce40ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:39.014 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:39.014+0000 7f5cdbfff640 1 -- 172.21.15.49:0/1638719162 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ce40cc0d0 con 0x7f5ce40cb970 2023-11-06T02:13:39.014 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:39.014+0000 7f5cdbfff640 1 -- 172.21.15.49:0/1638719162 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7f5cd4001680 con 0x7f5ce40cb970 2023-11-06T02:13:39.014 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:39.014+0000 7f5cdbfff640 1 -- 172.21.15.49:0/1638719162 >> v1:172.21.15.49:6789/0 conn(0x7f5ce40cb970 legacy=0x7f5ce40cd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:39.014 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:39.014+0000 7f5cdbfff640 1 --2- 172.21.15.49:0/1638719162 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5cd00039b0 0x7f5cd0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:39.015 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:39.014+0000 7f5cdbfff640 1 --2- 172.21.15.49:0/1638719162 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ce415aab0 0x7f5cd00047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:39.015 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:39.014+0000 7f5cdbfff640 1 --2- 172.21.15.49:0/1638719162 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5cd0004d10 0x7f5cd0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:39.015 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:39.014+0000 7f5cdbfff640 1 -- 172.21.15.49:0/1638719162 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce415ae90 con 0x7f5cd00039b0 2023-11-06T02:13:39.015 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:39.014+0000 7f5cdbfff640 1 -- 172.21.15.49:0/1638719162 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd000bac0 con 0x7f5cd0004d10 2023-11-06T02:13:39.015 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:39.014+0000 7f5cdbfff640 1 -- 172.21.15.49:0/1638719162 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd000bc80 con 0x7f5ce415aab0 2023-11-06T02:13:39.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:39.014+0000 7f5ce939d640 1 --2- 172.21.15.49:0/1638719162 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5cd00039b0 0x7f5cd0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:39.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:39.014+0000 7f5ce8b9c640 1 --2- 172.21.15.49:0/1638719162 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ce415aab0 0x7f5cd00047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:39.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:39.014+0000 7f5ce9b9e640 1 --2- 172.21.15.49:0/1638719162 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5cd0004d10 0x7f5cd0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:39.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:39.014+0000 7f5ce939d640 1 -- 172.21.15.49:0/1638719162 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ce415aab0 msgr2=0x7f5cd00047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:39.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:39.014+0000 7f5ce939d640 1 --2- 172.21.15.49:0/1638719162 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ce415aab0 0x7f5cd00047d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:39.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:39.014+0000 7f5ce939d640 1 -- 172.21.15.49:0/1638719162 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5cd0004d10 msgr2=0x7f5cd0005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:39.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:39.014+0000 7f5ce939d640 1 --2- 172.21.15.49:0/1638719162 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5cd0004d10 0x7f5cd0005190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:39.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:39.014+0000 7f5ce939d640 1 -- 172.21.15.49:0/1638719162 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cd0003530 con 0x7f5cd00039b0 2023-11-06T02:13:39.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:39.018+0000 7f5ce939d640 1 --2- 172.21.15.49:0/1638719162 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5cd00039b0 0x7f5cd0003dd0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f5ce000e170 tx=0x7f5ce001cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25d76a7c2bc2e5f5 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:39.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:39.018+0000 7f5cdbfff640 1 -- 172.21.15.49:0/1638719162 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce00a5ba0 con 0x7f5cd00039b0 2023-11-06T02:13:39.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:39.018+0000 7f5cdbfff640 1 -- 172.21.15.49:0/1638719162 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ce000f040 con 0x7f5cd00039b0 2023-11-06T02:13:39.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:39.018+0000 7f5cdbfff640 1 -- 172.21.15.49:0/1638719162 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce00ad440 con 0x7f5cd00039b0 2023-11-06T02:13:39.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:39.018+0000 7f5ceb601640 1 -- 172.21.15.49:0/1638719162 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5cd00039b0 msgr2=0x7f5cd0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:39.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:39.018+0000 7f5ceb601640 1 --2- 172.21.15.49:0/1638719162 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5cd00039b0 0x7f5cd0003dd0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f5ce000e170 tx=0x7f5ce001cbb0 comp rx=0 tx=0).stop 2023-11-06T02:13:39.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:39.018+0000 7f5ce9b9e640 1 -- 172.21.15.49:0/1638719162 reap_dead start 2023-11-06T02:13:39.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:39.018+0000 7f5ceb601640 1 -- 172.21.15.49:0/1638719162 shutdown_connections 2023-11-06T02:13:39.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:39.018+0000 7f5ceb601640 1 -- 172.21.15.49:0/1638719162 >> 172.21.15.49:0/1638719162 conn(0x7f5ce4089fd0 msgr2=0x7f5ce4099c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:39.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:39.018+0000 7f5ceb601640 1 -- 172.21.15.49:0/1638719162 shutdown_connections 2023-11-06T02:13:39.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:39.018+0000 7f5ceb601640 1 -- 172.21.15.49:0/1638719162 wait complete. 2023-11-06T02:13:39.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:39.018+0000 7f5ceb601640 1 Processor -- start 2023-11-06T02:13:39.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:39.018+0000 7f5ceb601640 1 -- start start 2023-11-06T02:13:39.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:39.018+0000 7f5ceb601640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5cd00039b0 0x7f5ce40ad660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:39.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:39.018+0000 7f5ceb601640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ce40c79f0 0x7f5ce40c8290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:39.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:39.018+0000 7f5ceb601640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ce41f3880 0x7f5ce4157a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:39.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:39.018+0000 7f5ceb601640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce40b4510 con 0x7f5cd00039b0 2023-11-06T02:13:39.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:39.018+0000 7f5ceb601640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce40b47f0 con 0x7f5ce40c79f0 2023-11-06T02:13:39.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:39.018+0000 7f5ceb601640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce40b4ad0 con 0x7f5ce41f3880 2023-11-06T02:13:39.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:39.018+0000 7f5ce939d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5cd00039b0 0x7f5ce40ad660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:39.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:39.018+0000 7f5ce939d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5cd00039b0 0x7f5ce40ad660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47886/0 (socket says 172.21.15.49:47886) 2023-11-06T02:13:39.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:39.018+0000 7f5ce939d640 1 -- 172.21.15.49:0/3890618339 learned_addr learned my addr 172.21.15.49:0/3890618339 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:39.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:39.018+0000 7f5ce9b9e640 1 --2- 172.21.15.49:0/3890618339 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ce41f3880 0x7f5ce4157a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:39.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:39.018+0000 7f5ce8b9c640 1 --2- 172.21.15.49:0/3890618339 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ce40c79f0 0x7f5ce40c8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:39.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:39.018+0000 7f5ce939d640 1 -- 172.21.15.49:0/3890618339 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ce41f3880 msgr2=0x7f5ce4157a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:39.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:39.018+0000 7f5ce939d640 1 --2- 172.21.15.49:0/3890618339 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ce41f3880 0x7f5ce4157a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:39.026 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:39.018+0000 7f5ce939d640 1 -- 172.21.15.49:0/3890618339 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ce40c79f0 msgr2=0x7f5ce40c8290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:39.026 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:39.018+0000 7f5ce939d640 1 --2- 172.21.15.49:0/3890618339 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ce40c79f0 0x7f5ce40c8290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:39.026 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:39.018+0000 7f5ce939d640 1 -- 172.21.15.49:0/3890618339 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ce40b4db0 con 0x7f5cd00039b0 2023-11-06T02:13:39.026 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:39.018+0000 7f5ce939d640 1 --2- 172.21.15.49:0/3890618339 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5cd00039b0 0x7f5ce40ad660 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f5ce0009fb0 tx=0x7f5ce0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:39.026 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:39.018+0000 7f5cd9ffb640 1 -- 172.21.15.49:0/3890618339 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce00bd070 con 0x7f5cd00039b0 2023-11-06T02:13:39.027 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:39.022+0000 7f5cd9ffb640 1 -- 172.21.15.49:0/3890618339 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ce00039a0 con 0x7f5cd00039b0 2023-11-06T02:13:39.027 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:39.022+0000 7f5ceb601640 1 -- 172.21.15.49:0/3890618339 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ce40b5090 con 0x7f5cd00039b0 2023-11-06T02:13:39.027 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:39.022+0000 7f5cd9ffb640 1 -- 172.21.15.49:0/3890618339 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce0003d70 con 0x7f5cd00039b0 2023-11-06T02:13:39.027 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:39.022+0000 7f5ceb601640 1 -- 172.21.15.49:0/3890618339 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ce40add70 con 0x7f5cd00039b0 2023-11-06T02:13:39.027 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:39.022+0000 7f5ceb601640 1 -- 172.21.15.49:0/3890618339 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ce40b5220 con 0x7f5cd00039b0 2023-11-06T02:13:39.028 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:39.022+0000 7f5cd9ffb640 1 -- 172.21.15.49:0/3890618339 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f5ce00b4040 con 0x7f5cd00039b0 2023-11-06T02:13:39.028 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:39.022+0000 7f5cd9ffb640 1 --2- 172.21.15.49:0/3890618339 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5cc8065a60 0x7f5cc8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:39.028 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:39.022+0000 7f5cd9ffb640 1 -- 172.21.15.49:0/3890618339 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5ce011d520 con 0x7f5cd00039b0 2023-11-06T02:13:39.028 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:39.022+0000 7f5ce8b9c640 1 --2- 172.21.15.49:0/3890618339 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5cc8065a60 0x7f5cc8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:39.028 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:39.022+0000 7f5ce8b9c640 1 --2- 172.21.15.49:0/3890618339 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5cc8065a60 0x7f5cc8067f20 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f5ce40c7ff0 tx=0x7f5cd4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:39.029 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:39.026+0000 7f5cd9ffb640 1 -- 172.21.15.49:0/3890618339 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f5ce00eaea0 con 0x7f5cd00039b0 2023-11-06T02:13:39.174 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:39.178+0000 7f5ceb601640 1 -- 172.21.15.49:0/3890618339 --> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5ce40b6730 con 0x7f5cc8065a60 2023-11-06T02:13:39.197 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:39.202+0000 7f5cd9ffb640 1 -- 172.21.15.49:0/3890618339 <== mgr.4100 v2:172.21.15.150:6832/38104 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+306073 (secure 0 0 0) 0x7f5ce40b6730 con 0x7f5cc8065a60 2023-11-06T02:13:39.201 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:13:39.202 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2023-11-06T02:13:39.203 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:39.206+0000 7f5ceb601640 1 -- 172.21.15.49:0/3890618339 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5cc8065a60 msgr2=0x7f5cc8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:39.204 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:39.206+0000 7f5ceb601640 1 --2- 172.21.15.49:0/3890618339 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5cc8065a60 0x7f5cc8067f20 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f5ce40c7ff0 tx=0x7f5cd4009040 comp rx=0 tx=0).stop 2023-11-06T02:13:39.204 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:39.206+0000 7f5ceb601640 1 -- 172.21.15.49:0/3890618339 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5cd00039b0 msgr2=0x7f5ce40ad660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:39.204 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:39.206+0000 7f5ceb601640 1 --2- 172.21.15.49:0/3890618339 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5cd00039b0 0x7f5ce40ad660 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f5ce0009fb0 tx=0x7f5ce0009750 comp rx=0 tx=0).stop 2023-11-06T02:13:39.204 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:39.206+0000 7f5ceb601640 1 -- 172.21.15.49:0/3890618339 shutdown_connections 2023-11-06T02:13:39.205 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:39.206+0000 7f5ceb601640 1 --2- 172.21.15.49:0/3890618339 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5cc8065a60 0x7f5cc8067f20 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:39.205 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:39.206+0000 7f5ceb601640 1 --2- 172.21.15.49:0/3890618339 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ce41f3880 0x7f5ce4157a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:39.205 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:39.206+0000 7f5ceb601640 1 --2- 172.21.15.49:0/3890618339 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ce40c79f0 0x7f5ce40c8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:39.205 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:39.206+0000 7f5ceb601640 1 --2- 172.21.15.49:0/3890618339 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5cd00039b0 0x7f5ce40ad660 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:39.205 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:39.206+0000 7f5ceb601640 1 -- 172.21.15.49:0/3890618339 >> 172.21.15.49:0/3890618339 conn(0x7f5ce4089fd0 msgr2=0x7f5ce415eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:39.205 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:39.206+0000 7f5ceb601640 1 -- 172.21.15.49:0/3890618339 shutdown_connections 2023-11-06T02:13:39.206 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:39.206+0000 7f5ceb601640 1 -- 172.21.15.49:0/3890618339 wait complete. 2023-11-06T02:13:39.221 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":233,"stamp":"2023-11-06T02:13:37.286493+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":20,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":248676,"kb_used_data":8484,"kb_used_omap":0,"kb_used_meta":240128,"kb_avail":754726044,"statfs":{"total":773094113280,"available":772839469056,"internally_reserved":0,"allocated":8687616,"data_stored":4531565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":245891072},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":44,"apply_latency_ms":44,"commit_latency_ns":44000000,"apply_latency_ns":44000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.060378"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.255018+0000","last_change":"2023-11-06T02:13:33.255018+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.255018+0000","last_undegraded":"2023-11-06T02:13:33.255018+0000","last_fullsized":"2023-11-06T02:13:33.255018+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:18:07.485044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.252648+0000","last_change":"2023-11-06T02:13:33.252648+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.252648+0000","last_undegraded":"2023-11-06T02:13:33.252648+0000","last_fullsized":"2023-11-06T02:13:33.252648+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:22:01.334447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.257065+0000","last_change":"2023-11-06T02:13:33.257065+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.257065+0000","last_undegraded":"2023-11-06T02:13:33.257065+0000","last_fullsized":"2023-11-06T02:13:33.257065+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:42:42.388106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.256535+0000","last_change":"2023-11-06T02:13:33.256535+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.256535+0000","last_undegraded":"2023-11-06T02:13:33.256535+0000","last_fullsized":"2023-11-06T02:13:33.256535+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:54:33.444383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.246148+0000","last_change":"2023-11-06T02:13:33.246148+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.246148+0000","last_undegraded":"2023-11-06T02:13:33.246148+0000","last_fullsized":"2023-11-06T02:13:33.246148+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:01:06.093323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.258259+0000","last_change":"2023-11-06T02:13:33.258259+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.258259+0000","last_undegraded":"2023-11-06T02:13:33.258259+0000","last_fullsized":"2023-11-06T02:13:33.258259+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:41:55.941458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.239742+0000","last_change":"2023-11-06T02:13:33.239742+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.239742+0000","last_undegraded":"2023-11-06T02:13:33.239742+0000","last_fullsized":"2023-11-06T02:13:33.239742+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:40:54.464756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.246060+0000","last_change":"2023-11-06T02:13:33.246060+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.246060+0000","last_undegraded":"2023-11-06T02:13:33.246060+0000","last_fullsized":"2023-11-06T02:13:33.246060+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:39.514240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.243570+0000","last_change":"2023-11-06T02:13:33.243570+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.243570+0000","last_undegraded":"2023-11-06T02:13:33.243570+0000","last_fullsized":"2023-11-06T02:13:33.243570+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:02:51.697659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.257271+0000","last_change":"2023-11-06T02:13:33.257271+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.257271+0000","last_undegraded":"2023-11-06T02:13:33.257271+0000","last_fullsized":"2023-11-06T02:13:33.257271+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:12:00.717133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.252080+0000","last_change":"2023-11-06T02:13:33.252080+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.252080+0000","last_undegraded":"2023-11-06T02:13:33.252080+0000","last_fullsized":"2023-11-06T02:13:33.252080+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:41:33.827814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.255251+0000","last_change":"2023-11-06T02:13:33.255251+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.255251+0000","last_undegraded":"2023-11-06T02:13:33.255251+0000","last_fullsized":"2023-11-06T02:13:33.255251+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:21:24.032595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.245690+0000","last_change":"2023-11-06T02:13:33.245690+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.245690+0000","last_undegraded":"2023-11-06T02:13:33.245690+0000","last_fullsized":"2023-11-06T02:13:33.245690+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:42:30.360623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.257567+0000","last_change":"2023-11-06T02:13:33.257567+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.257567+0000","last_undegraded":"2023-11-06T02:13:33.257567+0000","last_fullsized":"2023-11-06T02:13:33.257567+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:41:33.827814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.252478+0000","last_change":"2023-11-06T02:13:33.252478+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.252478+0000","last_undegraded":"2023-11-06T02:13:33.252478+0000","last_fullsized":"2023-11-06T02:13:33.252478+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:28:22.791353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.254366+0000","last_change":"2023-11-06T02:13:33.254366+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.254366+0000","last_undegraded":"2023-11-06T02:13:33.254366+0000","last_fullsized":"2023-11-06T02:13:33.254366+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:21:24.032595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.246654+0000","last_change":"2023-11-06T02:13:33.246654+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.246654+0000","last_undegraded":"2023-11-06T02:13:33.246654+0000","last_fullsized":"2023-11-06T02:13:33.246654+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:54:43.806737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.255469+0000","last_change":"2023-11-06T02:13:33.255469+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.255469+0000","last_undegraded":"2023-11-06T02:13:33.255469+0000","last_fullsized":"2023-11-06T02:13:33.255469+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:17:36.098143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.254869+0000","last_change":"2023-11-06T02:13:33.254869+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.254869+0000","last_undegraded":"2023-11-06T02:13:33.254869+0000","last_fullsized":"2023-11-06T02:13:33.254869+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:22:13.884367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.247005+0000","last_change":"2023-11-06T02:13:33.247005+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.247005+0000","last_undegraded":"2023-11-06T02:13:33.247005+0000","last_fullsized":"2023-11-06T02:13:33.247005+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:28:11.270422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.242891+0000","last_change":"2023-11-06T02:13:33.242891+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.242891+0000","last_undegraded":"2023-11-06T02:13:33.242891+0000","last_fullsized":"2023-11-06T02:13:33.242891+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:43:49.529761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.246368+0000","last_change":"2023-11-06T02:13:33.246368+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.246368+0000","last_undegraded":"2023-11-06T02:13:33.246368+0000","last_fullsized":"2023-11-06T02:13:33.246368+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:19:02.943437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.244217+0000","last_change":"2023-11-06T02:13:33.244217+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.244217+0000","last_undegraded":"2023-11-06T02:13:33.244217+0000","last_fullsized":"2023-11-06T02:13:33.244217+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:17:17.217507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.252017+0000","last_change":"2023-11-06T02:13:33.252017+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.252017+0000","last_undegraded":"2023-11-06T02:13:33.252017+0000","last_fullsized":"2023-11-06T02:13:33.252017+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:41:55.941458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.250038+0000","last_change":"2023-11-06T02:13:33.250038+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.250038+0000","last_undegraded":"2023-11-06T02:13:33.250038+0000","last_fullsized":"2023-11-06T02:13:33.250038+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:22:13.884367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.255314+0000","last_change":"2023-11-06T02:13:33.255314+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.255314+0000","last_undegraded":"2023-11-06T02:13:33.255314+0000","last_fullsized":"2023-11-06T02:13:33.255314+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:18:07.485044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.254181+0000","last_change":"2023-11-06T02:13:33.254181+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.254181+0000","last_undegraded":"2023-11-06T02:13:33.254181+0000","last_fullsized":"2023-11-06T02:13:33.254181+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:00:13.166796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.243183+0000","last_change":"2023-11-06T02:13:33.243183+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.243183+0000","last_undegraded":"2023-11-06T02:13:33.243183+0000","last_fullsized":"2023-11-06T02:13:33.243183+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:16:46.930331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.249877+0000","last_change":"2023-11-06T02:13:33.249877+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.249877+0000","last_undegraded":"2023-11-06T02:13:33.249877+0000","last_fullsized":"2023-11-06T02:13:33.249877+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:20:11.090360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.250528+0000","last_change":"2023-11-06T02:13:33.250528+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.250528+0000","last_undegraded":"2023-11-06T02:13:33.250528+0000","last_fullsized":"2023-11-06T02:13:33.250528+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:34:15.876381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.253284+0000","last_change":"2023-11-06T02:13:33.253284+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.253284+0000","last_undegraded":"2023-11-06T02:13:33.253284+0000","last_fullsized":"2023-11-06T02:13:33.253284+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:14:22.392069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.252628+0000","last_change":"2023-11-06T02:13:33.252628+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.252628+0000","last_undegraded":"2023-11-06T02:13:33.252628+0000","last_fullsized":"2023-11-06T02:13:33.252628+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:24:31.881719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.252017+0000","last_change":"2023-11-06T02:13:33.252017+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.252017+0000","last_undegraded":"2023-11-06T02:13:33.252017+0000","last_fullsized":"2023-11-06T02:13:33.252017+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:46:42.636025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":3,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.235656+0000","last_change":"2023-11-06T02:13:33.235656+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.235656+0000","last_undegraded":"2023-11-06T02:13:33.235656+0000","last_fullsized":"2023-11-06T02:13:33.235656+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:07:13.179013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.249040+0000","last_change":"2023-11-06T02:13:33.249040+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.249040+0000","last_undegraded":"2023-11-06T02:13:33.249040+0000","last_fullsized":"2023-11-06T02:13:33.249040+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:23:55.083630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":3,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.256359+0000","last_change":"2023-11-06T02:13:33.256359+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.256359+0000","last_undegraded":"2023-11-06T02:13:33.256359+0000","last_fullsized":"2023-11-06T02:13:33.256359+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:33:14.271873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":["7","6"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.246368+0000","last_change":"2023-11-06T02:13:33.246368+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.246368+0000","last_undegraded":"2023-11-06T02:13:33.246368+0000","last_fullsized":"2023-11-06T02:13:33.246368+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:34:15.876381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":4,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.264127+0000","last_change":"2023-11-06T02:13:33.257252+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.264127+0000","last_undegraded":"2023-11-06T02:13:33.264127+0000","last_fullsized":"2023-11-06T02:13:33.264127+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:35:42.092353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":["7","0"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.241683+0000","last_change":"2023-11-06T02:13:33.241683+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.241683+0000","last_undegraded":"2023-11-06T02:13:33.241683+0000","last_fullsized":"2023-11-06T02:13:33.241683+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:14:21.400766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.277611+0000","last_change":"2023-11-06T02:13:33.267790+0000","last_active":"2023-11-06T02:13:33.277611+0000","last_peered":"2023-11-06T02:13:33.277611+0000","last_clean":"2023-11-06T02:13:33.277611+0000","last_became_active":"2023-11-06T02:13:33.267049+0000","last_became_peered":"2023-11-06T02:13:33.267049+0000","last_unstale":"2023-11-06T02:13:33.277611+0000","last_undegraded":"2023-11-06T02:13:33.277611+0000","last_fullsized":"2023-11-06T02:13:33.277611+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:44:56.393504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.249993+0000","last_change":"2023-11-06T02:13:33.249993+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.249993+0000","last_undegraded":"2023-11-06T02:13:33.249993+0000","last_fullsized":"2023-11-06T02:13:33.249993+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:11:19.807848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":3,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.240593+0000","last_change":"2023-11-06T02:13:33.240593+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.240593+0000","last_undegraded":"2023-11-06T02:13:33.240593+0000","last_fullsized":"2023-11-06T02:13:33.240593+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:33:31.177122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":["4","7"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.249061+0000","last_change":"2023-11-06T02:13:33.249061+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.249061+0000","last_undegraded":"2023-11-06T02:13:33.249061+0000","last_fullsized":"2023-11-06T02:13:33.249061+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:16:52.758810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.277263+0000","last_change":"2023-11-06T02:13:33.272642+0000","last_active":"2023-11-06T02:13:33.277263+0000","last_peered":"2023-11-06T02:13:33.277263+0000","last_clean":"2023-11-06T02:13:33.277263+0000","last_became_active":"2023-11-06T02:13:33.271888+0000","last_became_peered":"2023-11-06T02:13:33.271888+0000","last_unstale":"2023-11-06T02:13:33.277263+0000","last_undegraded":"2023-11-06T02:13:33.277263+0000","last_fullsized":"2023-11-06T02:13:33.277263+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:51:47.363705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.242755+0000","last_change":"2023-11-06T02:13:33.242755+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.242755+0000","last_undegraded":"2023-11-06T02:13:33.242755+0000","last_fullsized":"2023-11-06T02:13:33.242755+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:16:36.406664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":3,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.243828+0000","last_change":"2023-11-06T02:13:33.243828+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.243828+0000","last_undegraded":"2023-11-06T02:13:33.243828+0000","last_fullsized":"2023-11-06T02:13:33.243828+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:54:32.434908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":["0","7"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.245643+0000","last_change":"2023-11-06T02:13:33.245643+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.245643+0000","last_undegraded":"2023-11-06T02:13:33.245643+0000","last_fullsized":"2023-11-06T02:13:33.245643+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:33:50.179908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":3,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.235521+0000","last_change":"2023-11-06T02:13:33.235521+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.235521+0000","last_undegraded":"2023-11-06T02:13:33.235521+0000","last_fullsized":"2023-11-06T02:13:33.235521+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:39:03.018907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":["2","4"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.249061+0000","last_change":"2023-11-06T02:13:33.249061+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.249061+0000","last_undegraded":"2023-11-06T02:13:33.249061+0000","last_fullsized":"2023-11-06T02:13:33.249061+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:32:02.459972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":3,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.259605+0000","last_change":"2023-11-06T02:13:33.259605+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.259605+0000","last_undegraded":"2023-11-06T02:13:33.259605+0000","last_fullsized":"2023-11-06T02:13:33.259605+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:53:46.349156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":["7","6"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.253727+0000","last_change":"2023-11-06T02:13:33.253727+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.253727+0000","last_undegraded":"2023-11-06T02:13:33.253727+0000","last_fullsized":"2023-11-06T02:13:33.253727+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:00:13.166796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.283810+0000","last_change":"2023-11-06T02:13:33.281331+0000","last_active":"2023-11-06T02:13:33.283810+0000","last_peered":"2023-11-06T02:13:33.283810+0000","last_clean":"2023-11-06T02:13:33.283810+0000","last_became_active":"2023-11-06T02:13:33.280630+0000","last_became_peered":"2023-11-06T02:13:33.280630+0000","last_unstale":"2023-11-06T02:13:33.283810+0000","last_undegraded":"2023-11-06T02:13:33.283810+0000","last_fullsized":"2023-11-06T02:13:33.283810+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:17:16.208032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.256899+0000","last_change":"2023-11-06T02:13:33.256899+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.256899+0000","last_undegraded":"2023-11-06T02:13:33.256899+0000","last_fullsized":"2023-11-06T02:13:33.256899+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:45:05.474573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":3,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.235632+0000","last_change":"2023-11-06T02:13:33.235632+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.235632+0000","last_undegraded":"2023-11-06T02:13:33.235632+0000","last_fullsized":"2023-11-06T02:13:33.235632+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:55:36.788472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.251869+0000","last_change":"2023-11-06T02:13:33.251869+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.251869+0000","last_undegraded":"2023-11-06T02:13:33.251869+0000","last_fullsized":"2023-11-06T02:13:33.251869+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:33:50.179908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.277447+0000","last_change":"2023-11-06T02:13:33.265304+0000","last_active":"2023-11-06T02:13:33.277447+0000","last_peered":"2023-11-06T02:13:33.277447+0000","last_clean":"2023-11-06T02:13:33.277447+0000","last_became_active":"2023-11-06T02:13:33.264524+0000","last_became_peered":"2023-11-06T02:13:33.264524+0000","last_unstale":"2023-11-06T02:13:33.277447+0000","last_undegraded":"2023-11-06T02:13:33.277447+0000","last_fullsized":"2023-11-06T02:13:33.277447+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:39:01.059129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.253815+0000","last_change":"2023-11-06T02:13:33.253815+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.253815+0000","last_undegraded":"2023-11-06T02:13:33.253815+0000","last_fullsized":"2023-11-06T02:13:33.253815+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:17:59.025543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.280385+0000","last_change":"2023-11-06T02:13:33.273211+0000","last_active":"2023-11-06T02:13:33.280385+0000","last_peered":"2023-11-06T02:13:33.280385+0000","last_clean":"2023-11-06T02:13:33.280385+0000","last_became_active":"2023-11-06T02:13:33.272067+0000","last_became_peered":"2023-11-06T02:13:33.272067+0000","last_unstale":"2023-11-06T02:13:33.280385+0000","last_undegraded":"2023-11-06T02:13:33.280385+0000","last_fullsized":"2023-11-06T02:13:33.280385+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:16:35.397189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.240661+0000","last_change":"2023-11-06T02:13:33.240661+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.240661+0000","last_undegraded":"2023-11-06T02:13:33.240661+0000","last_fullsized":"2023-11-06T02:13:33.240661+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:17:36.098143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":3,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.256327+0000","last_change":"2023-11-06T02:13:33.256327+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.256327+0000","last_undegraded":"2023-11-06T02:13:33.256327+0000","last_fullsized":"2023-11-06T02:13:33.256327+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:53:59.721435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":["7","2"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.241911+0000","last_change":"2023-11-06T02:13:33.241911+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.241911+0000","last_undegraded":"2023-11-06T02:13:33.241911+0000","last_fullsized":"2023-11-06T02:13:33.241911+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:10:52.936432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":3,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.235562+0000","last_change":"2023-11-06T02:13:33.235562+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.235562+0000","last_undegraded":"2023-11-06T02:13:33.235562+0000","last_fullsized":"2023-11-06T02:13:33.235562+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:30:40.192944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.244479+0000","last_change":"2023-11-06T02:13:33.244479+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.244479+0000","last_undegraded":"2023-11-06T02:13:33.244479+0000","last_fullsized":"2023-11-06T02:13:33.244479+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:21:05.055643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":3,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.247845+0000","last_change":"2023-11-06T02:13:33.247845+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.247845+0000","last_undegraded":"2023-11-06T02:13:33.247845+0000","last_fullsized":"2023-11-06T02:13:33.247845+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:17:58.016068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":["0","6"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.241033+0000","last_change":"2023-11-06T02:13:33.241033+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.241033+0000","last_undegraded":"2023-11-06T02:13:33.241033+0000","last_fullsized":"2023-11-06T02:13:33.241033+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:54:00.730910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.279807+0000","last_change":"2023-11-06T02:13:33.272867+0000","last_active":"2023-11-06T02:13:33.279807+0000","last_peered":"2023-11-06T02:13:33.279807+0000","last_clean":"2023-11-06T02:13:33.279807+0000","last_became_active":"2023-11-06T02:13:33.272136+0000","last_became_peered":"2023-11-06T02:13:33.272136+0000","last_unstale":"2023-11-06T02:13:33.279807+0000","last_undegraded":"2023-11-06T02:13:33.279807+0000","last_fullsized":"2023-11-06T02:13:33.279807+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:02:50.688184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.238294+0000","last_change":"2023-11-06T02:13:33.238294+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.238294+0000","last_undegraded":"2023-11-06T02:13:33.238294+0000","last_fullsized":"2023-11-06T02:13:33.238294+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:02:47.316294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":3,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.244338+0000","last_change":"2023-11-06T02:13:33.244338+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.244338+0000","last_undegraded":"2023-11-06T02:13:33.244338+0000","last_fullsized":"2023-11-06T02:13:33.244338+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:11:59.707658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.245156+0000","last_change":"2023-11-06T02:13:33.245156+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.245156+0000","last_undegraded":"2023-11-06T02:13:33.245156+0000","last_fullsized":"2023-11-06T02:13:33.245156+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:07:57.793830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":3,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.258159+0000","last_change":"2023-11-06T02:13:33.258159+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.258159+0000","last_undegraded":"2023-11-06T02:13:33.258159+0000","last_fullsized":"2023-11-06T02:13:33.258159+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:16:35.397189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":["7","5"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.248850+0000","last_change":"2023-11-06T02:13:33.248850+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.248850+0000","last_undegraded":"2023-11-06T02:13:33.248850+0000","last_fullsized":"2023-11-06T02:13:33.248850+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:22:47.360905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":3,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.256366+0000","last_change":"2023-11-06T02:13:33.256366+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.256366+0000","last_undegraded":"2023-11-06T02:13:33.256366+0000","last_fullsized":"2023-11-06T02:13:33.256366+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:12:35.766676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":["7","6"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.244536+0000","last_change":"2023-11-06T02:13:33.244536+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.244536+0000","last_undegraded":"2023-11-06T02:13:33.244536+0000","last_fullsized":"2023-11-06T02:13:33.244536+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:42:42.388106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.277757+0000","last_change":"2023-11-06T02:13:33.267817+0000","last_active":"2023-11-06T02:13:33.277757+0000","last_peered":"2023-11-06T02:13:33.277757+0000","last_clean":"2023-11-06T02:13:33.277757+0000","last_became_active":"2023-11-06T02:13:33.267091+0000","last_became_peered":"2023-11-06T02:13:33.267091+0000","last_unstale":"2023-11-06T02:13:33.277757+0000","last_undegraded":"2023-11-06T02:13:33.277757+0000","last_fullsized":"2023-11-06T02:13:33.277757+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:36:21.262835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.239368+0000","last_change":"2023-11-06T02:13:33.239368+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.239368+0000","last_undegraded":"2023-11-06T02:13:33.239368+0000","last_fullsized":"2023-11-06T02:13:33.239368+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:31:19.083004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":3,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.235609+0000","last_change":"2023-11-06T02:13:33.235609+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.235609+0000","last_undegraded":"2023-11-06T02:13:33.235609+0000","last_fullsized":"2023-11-06T02:13:33.235609+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:19:20.386051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.254957+0000","last_change":"2023-11-06T02:13:33.254957+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.254957+0000","last_undegraded":"2023-11-06T02:13:33.254957+0000","last_fullsized":"2023-11-06T02:13:33.254957+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:02:47.316294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.277367+0000","last_change":"2023-11-06T02:13:33.264127+0000","last_active":"2023-11-06T02:13:33.277367+0000","last_peered":"2023-11-06T02:13:33.277367+0000","last_clean":"2023-11-06T02:13:33.277367+0000","last_became_active":"2023-11-06T02:13:33.263418+0000","last_became_peered":"2023-11-06T02:13:33.263418+0000","last_unstale":"2023-11-06T02:13:33.277367+0000","last_undegraded":"2023-11-06T02:13:33.277367+0000","last_fullsized":"2023-11-06T02:13:33.277367+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:04:53.601947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.253013+0000","last_change":"2023-11-06T02:13:33.253013+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.253013+0000","last_undegraded":"2023-11-06T02:13:33.253013+0000","last_fullsized":"2023-11-06T02:13:33.253013+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:31:19.083004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":3,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.251425+0000","last_change":"2023-11-06T02:13:33.251425+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.251425+0000","last_undegraded":"2023-11-06T02:13:33.251425+0000","last_fullsized":"2023-11-06T02:13:33.251425+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:13:44.482660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":["1","3"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.252952+0000","last_change":"2023-11-06T02:13:33.252952+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.252952+0000","last_undegraded":"2023-11-06T02:13:33.252952+0000","last_fullsized":"2023-11-06T02:13:33.252952+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:12:00.717133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":3,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.253882+0000","last_change":"2023-11-06T02:13:33.253882+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.253882+0000","last_undegraded":"2023-11-06T02:13:33.253882+0000","last_fullsized":"2023-11-06T02:13:33.253882+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:16:51.749335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":["3","6"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":216,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-06T02:13:32.232683+0000","last_change":"2023-11-06T02:08:20.523427+0000","last_active":"2023-11-06T02:13:32.232683+0000","last_peered":"2023-11-06T02:13:32.232683+0000","last_clean":"2023-11-06T02:13:32.232683+0000","last_became_active":"2023-11-06T02:08:20.521089+0000","last_became_peered":"2023-11-06T02:08:20.521089+0000","last_unstale":"2023-11-06T02:13:32.232683+0000","last_undegraded":"2023-11-06T02:13:32.232683+0000","last_fullsized":"2023-11-06T02:13:32.232683+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:19.252348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:19.252348+0000","last_clean_scrub_stamp":"2023-11-06T02:08:19.252348+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:30:15.566397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":62,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-06T02:13:32.246890+0000","last_change":"2023-11-06T02:08:26.059502+0000","last_active":"2023-11-06T02:13:32.246890+0000","last_peered":"2023-11-06T02:13:32.246890+0000","last_clean":"2023-11-06T02:13:32.246890+0000","last_became_active":"2023-11-06T02:08:26.058612+0000","last_became_peered":"2023-11-06T02:08:26.058612+0000","last_unstale":"2023-11-06T02:13:32.246890+0000","last_undegraded":"2023-11-06T02:13:32.246890+0000","last_fullsized":"2023-11-06T02:13:32.246890+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:10.626021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.250595+0000","last_change":"2023-11-06T02:13:33.250595+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.250595+0000","last_undegraded":"2023-11-06T02:13:33.250595+0000","last_fullsized":"2023-11-06T02:13:33.250595+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:46:01.922659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":3,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.246452+0000","last_change":"2023-11-06T02:13:33.246452+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.246452+0000","last_undegraded":"2023-11-06T02:13:33.246452+0000","last_fullsized":"2023-11-06T02:13:33.246452+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:11:51.109403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":["3","6"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":63,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-06T02:13:32.232564+0000","last_change":"2023-11-06T02:08:26.060406+0000","last_active":"2023-11-06T02:13:32.232564+0000","last_peered":"2023-11-06T02:13:32.232564+0000","last_clean":"2023-11-06T02:13:32.232564+0000","last_became_active":"2023-11-06T02:08:26.059262+0000","last_became_peered":"2023-11-06T02:08:26.059262+0000","last_unstale":"2023-11-06T02:13:32.232564+0000","last_undegraded":"2023-11-06T02:13:32.232564+0000","last_fullsized":"2023-11-06T02:13:32.232564+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:10.626021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.243210+0000","last_change":"2023-11-06T02:13:33.243210+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.243210+0000","last_undegraded":"2023-11-06T02:13:33.243210+0000","last_fullsized":"2023-11-06T02:13:33.243210+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:33:32.186597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":3,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.244333+0000","last_change":"2023-11-06T02:13:33.244333+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.244333+0000","last_undegraded":"2023-11-06T02:13:33.244333+0000","last_fullsized":"2023-11-06T02:13:33.244333+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:51:52.147063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":["3","6"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":63,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-06T02:13:32.236950+0000","last_change":"2023-11-06T02:08:26.559477+0000","last_active":"2023-11-06T02:13:32.236950+0000","last_peered":"2023-11-06T02:13:32.236950+0000","last_clean":"2023-11-06T02:13:32.236950+0000","last_became_active":"2023-11-06T02:08:26.559202+0000","last_became_peered":"2023-11-06T02:08:26.559202+0000","last_unstale":"2023-11-06T02:13:32.236950+0000","last_undegraded":"2023-11-06T02:13:32.236950+0000","last_fullsized":"2023-11-06T02:13:32.236950+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:55:04.552404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.249368+0000","last_change":"2023-11-06T02:13:33.249368+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.249368+0000","last_undegraded":"2023-11-06T02:13:33.249368+0000","last_fullsized":"2023-11-06T02:13:33.249368+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:54:11.389133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":3,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.245263+0000","last_change":"2023-11-06T02:13:33.245263+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.245263+0000","last_undegraded":"2023-11-06T02:13:33.245263+0000","last_fullsized":"2023-11-06T02:13:33.245263+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:38:14.971699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":62,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-06T02:13:32.246949+0000","last_change":"2023-11-06T02:08:26.059546+0000","last_active":"2023-11-06T02:13:32.246949+0000","last_peered":"2023-11-06T02:13:32.246949+0000","last_clean":"2023-11-06T02:13:32.246949+0000","last_became_active":"2023-11-06T02:08:26.058806+0000","last_became_peered":"2023-11-06T02:08:26.058806+0000","last_unstale":"2023-11-06T02:13:32.246949+0000","last_undegraded":"2023-11-06T02:13:32.246949+0000","last_fullsized":"2023-11-06T02:13:32.246949+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:23.468510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.242981+0000","last_change":"2023-11-06T02:13:33.242981+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.242981+0000","last_undegraded":"2023-11-06T02:13:33.242981+0000","last_fullsized":"2023-11-06T02:13:33.242981+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:36:12.474443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":3,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.236266+0000","last_change":"2023-11-06T02:13:33.236266+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.236266+0000","last_undegraded":"2023-11-06T02:13:33.236266+0000","last_fullsized":"2023-11-06T02:13:33.236266+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:30:05.141853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":["2","4"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":62,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-06T02:13:32.229548+0000","last_change":"2023-11-06T02:08:26.436463+0000","last_active":"2023-11-06T02:13:32.229548+0000","last_peered":"2023-11-06T02:13:32.229548+0000","last_clean":"2023-11-06T02:13:32.229548+0000","last_became_active":"2023-11-06T02:08:26.436075+0000","last_became_peered":"2023-11-06T02:08:26.436075+0000","last_unstale":"2023-11-06T02:13:32.229548+0000","last_undegraded":"2023-11-06T02:13:32.229548+0000","last_fullsized":"2023-11-06T02:13:32.229548+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:23.468510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.246317+0000","last_change":"2023-11-06T02:13:33.246317+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.246317+0000","last_undegraded":"2023-11-06T02:13:33.246317+0000","last_fullsized":"2023-11-06T02:13:33.246317+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:25:19.608926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":3,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.251742+0000","last_change":"2023-11-06T02:13:33.251742+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.251742+0000","last_undegraded":"2023-11-06T02:13:33.251742+0000","last_fullsized":"2023-11-06T02:13:33.251742+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:23:42.695735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":63,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-06T02:13:32.239073+0000","last_change":"2023-11-06T02:08:26.436607+0000","last_active":"2023-11-06T02:13:32.239073+0000","last_peered":"2023-11-06T02:13:32.239073+0000","last_clean":"2023-11-06T02:13:32.239073+0000","last_became_active":"2023-11-06T02:08:26.435924+0000","last_became_peered":"2023-11-06T02:08:26.435924+0000","last_unstale":"2023-11-06T02:13:32.239073+0000","last_undegraded":"2023-11-06T02:13:32.239073+0000","last_fullsized":"2023-11-06T02:13:32.239073+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:23.468510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.244818+0000","last_change":"2023-11-06T02:13:33.244818+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.244818+0000","last_undegraded":"2023-11-06T02:13:33.244818+0000","last_fullsized":"2023-11-06T02:13:33.244818+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:22:01.334447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":4,"reported_epoch":78,"state":"creating+activating","last_fresh":"2023-11-06T02:13:33.269809+0000","last_change":"2023-11-06T02:13:33.260450+0000","last_active":"2023-11-06T02:13:32.214782+0000","last_peered":"2023-11-06T02:13:32.214782+0000","last_clean":"2023-11-06T02:13:32.214782+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.269809+0000","last_undegraded":"2023-11-06T02:13:33.269809+0000","last_fullsized":"2023-11-06T02:13:33.269809+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:51:47.363705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":["7","1"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":62,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-06T02:13:32.229549+0000","last_change":"2023-11-06T02:08:26.057076+0000","last_active":"2023-11-06T02:13:32.229549+0000","last_peered":"2023-11-06T02:13:32.229549+0000","last_clean":"2023-11-06T02:13:32.229549+0000","last_became_active":"2023-11-06T02:08:26.056419+0000","last_became_peered":"2023-11-06T02:08:26.056419+0000","last_unstale":"2023-11-06T02:13:32.229549+0000","last_undegraded":"2023-11-06T02:13:32.229549+0000","last_fullsized":"2023-11-06T02:13:32.229549+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:10.626021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-06T02:13:33.242736+0000","last_change":"2023-11-06T02:13:33.242736+0000","last_active":"2023-11-06T02:13:33.224257+0000","last_peered":"2023-11-06T02:13:33.224257+0000","last_clean":"2023-11-06T02:13:33.224257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T02:13:33.242736+0000","last_undegraded":"2023-11-06T02:13:33.242736+0000","last_fullsized":"2023-11-06T02:13:33.242736+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:38:31.470172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":8,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.265697+0000","last_change":"2023-11-06T02:13:33.265697+0000","last_active":"2023-11-06T02:13:33.265697+0000","last_peered":"2023-11-06T02:13:33.265697+0000","last_clean":"2023-11-06T02:13:33.265697+0000","last_became_active":"2023-11-06T02:13:33.264914+0000","last_became_peered":"2023-11-06T02:13:33.264914+0000","last_unstale":"2023-11-06T02:13:33.265697+0000","last_undegraded":"2023-11-06T02:13:33.265697+0000","last_fullsized":"2023-11-06T02:13:33.265697+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:17:35.088668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":71,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-06T02:13:32.237181+0000","last_change":"2023-11-06T02:08:26.058596+0000","last_active":"2023-11-06T02:13:32.237181+0000","last_peered":"2023-11-06T02:13:32.237181+0000","last_clean":"2023-11-06T02:13:32.237181+0000","last_became_active":"2023-11-06T02:08:26.057015+0000","last_became_peered":"2023-11-06T02:08:26.057015+0000","last_unstale":"2023-11-06T02:13:32.237181+0000","last_undegraded":"2023-11-06T02:13:32.237181+0000","last_fullsized":"2023-11-06T02:13:32.237181+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:55:04.552404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574914,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32544,"kb_used_data":1496,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339296,"statfs":{"total":96636764160,"available":96603439104,"internally_reserved":0,"allocated":1531904,"data_stored":1009349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:13:26 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.86599999999999999,"15min":0.85399999999999998},"min":{"1min":0.59199999999999997,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.2190000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.62},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.91200000000000003,"15min":0.91100000000000003},"min":{"1min":0.57299999999999995,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.214,"5min":1.506,"15min":1.506},"last":1}]},{"osd":1,"last update":"Mon Nov 6 02:13:26 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.83599999999999997,"15min":0.84599999999999997},"min":{"1min":0.66400000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":0.94799999999999995},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.94099999999999995,"15min":0.90600000000000003},"min":{"1min":0.64300000000000002,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.681,"5min":1.681,"15min":1.681},"last":0.75600000000000001}]},{"osd":2,"last update":"Mon Nov 6 02:13:26 2023","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.96599999999999997,"15min":0.95799999999999996},"min":{"1min":0.66200000000000003,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.22,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.94899999999999995,"15min":0.98299999999999998},"min":{"1min":0.71899999999999997,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.2789999999999999,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.70499999999999996}]},{"osd":3,"last update":"Mon Nov 6 02:13:26 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.89400000000000002,"15min":0.88100000000000001},"min":{"1min":0.67500000000000004,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.196,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.78300000000000003},{"interface":"front","average":{"1min":1.069,"5min":1.046,"15min":1.0660000000000001},"min":{"1min":0.86399999999999999,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.2430000000000001,"5min":1.526,"15min":1.526},"last":0.91300000000000003}]},{"osd":4,"last update":"Mon Nov 6 02:13:26 2023","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.92300000000000004,"15min":0.94599999999999995},"min":{"1min":0.59699999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.2350000000000001,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.874},{"interface":"front","average":{"1min":1.0129999999999999,"5min":1.004,"15min":1.0309999999999999},"min":{"1min":0.625,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.377,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":1.052}]},{"osd":5,"last update":"Mon Nov 6 02:13:26 2023","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.038,"15min":1.0649999999999999},"min":{"1min":0.76300000000000001,"5min":0.503,"15min":0.503},"max":{"1min":1.409,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":1.032},{"interface":"front","average":{"1min":1.046,"5min":1.0629999999999999,"15min":1.111},"min":{"1min":0.75900000000000001,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.3109999999999999,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":1.087}]},{"osd":6,"last update":"Mon Nov 6 02:13:26 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.1399999999999999,"15min":1.181},"min":{"1min":0.59399999999999997,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.278,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":1.1639999999999999},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.091,"15min":1.135},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.4570000000000001,"5min":1.74,"15min":1.74},"last":1.133}]}]},{"osd":6,"up_from":13,"seq":55834574913,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31964,"kb_used_data":916,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339876,"statfs":{"total":96636764160,"available":96604033024,"internally_reserved":0,"allocated":937984,"data_stored":418981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:13:28 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.86699999999999999,"15min":0.86099999999999999},"min":{"1min":0.58899999999999997,"5min":0.495,"15min":0.495},"max":{"1min":1.202,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.98099999999999998},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.874,"15min":0.82799999999999996},"min":{"1min":0.63,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.2649999999999999,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.56399999999999995}]},{"osd":1,"last update":"Mon Nov 6 02:13:28 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.84999999999999998,"15min":0.86699999999999999},"min":{"1min":0.63500000000000001,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.244,"5min":1.244,"15min":1.244},"last":1.1639999999999999},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.89600000000000002,"15min":0.88100000000000001},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.624}]},{"osd":2,"last update":"Mon Nov 6 02:13:28 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.88700000000000001,"15min":0.89500000000000002},"min":{"1min":0.52100000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.206,"5min":1.252,"15min":1.252},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.90000000000000002,"15min":0.92000000000000004},"min":{"1min":0.61699999999999999,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.411,"5min":1.411,"15min":1.411},"last":0.82899999999999996}]},{"osd":3,"last update":"Mon Nov 6 02:13:28 2023","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.93899999999999995,"15min":0.96099999999999997},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.212,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.0549999999999999,"5min":0.99299999999999999,"15min":0.998},"min":{"1min":0.77300000000000002,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.381,"5min":1.381,"15min":1.381},"last":1.101}]},{"osd":4,"last update":"Mon Nov 6 02:13:28 2023","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.92600000000000005,"15min":0.97099999999999997},"min":{"1min":0.67500000000000004,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.302,"5min":1.474,"15min":1.474},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.0469999999999999,"5min":0.98199999999999998,"15min":1.02},"min":{"1min":0.77100000000000002,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.556,"5min":1.556,"15min":1.556},"last":1.2050000000000001}]},{"osd":5,"last update":"Mon Nov 6 02:13:28 2023","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":0.97199999999999998,"15min":1.0089999999999999},"min":{"1min":0.73099999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":1.4650000000000001},{"interface":"front","average":{"1min":1.129,"5min":1.0720000000000001,"15min":1.1000000000000001},"min":{"1min":0.94899999999999995,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.587,"5min":1.587,"15min":1.587},"last":1.3999999999999999}]},{"osd":7,"last update":"Mon Nov 6 02:13:28 2023","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.087,"15min":1.107},"min":{"1min":0.95899999999999996,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.615,"5min":1.615,"15min":1.615},"last":1.363},{"interface":"front","average":{"1min":1.109,"5min":1.0629999999999999,"15min":1.131},"min":{"1min":0.83499999999999996,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":1.2969999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574914,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":916,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":937984,"data_stored":418981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:13:30 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.86899999999999999,"15min":0.84199999999999997},"min":{"1min":0.51700000000000002,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.1930000000000001,"5min":1.381,"15min":1.381},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.86599999999999999,"15min":0.84999999999999998},"min":{"1min":0.58199999999999996,"5min":0.503,"15min":0.503},"max":{"1min":1.1399999999999999,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":1.1399999999999999}]},{"osd":1,"last update":"Mon Nov 6 02:13:30 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.89600000000000002,"15min":0.86499999999999999},"min":{"1min":0.69799999999999995,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.177,"5min":1.331,"15min":1.331},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.88100000000000001,"15min":0.83799999999999997},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.139,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.71399999999999997}]},{"osd":2,"last update":"Mon Nov 6 02:13:30 2023","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.93400000000000005,"15min":0.94199999999999995},"min":{"1min":0.68000000000000005,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.298,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.872},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.88400000000000001,"15min":0.82099999999999995},"min":{"1min":0.56899999999999995,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.2869999999999999,"5min":1.377,"15min":1.377},"last":0.98399999999999999}]},{"osd":3,"last update":"Mon Nov 6 02:13:30 2023","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.95399999999999996,"15min":0.93100000000000005},"min":{"1min":0.77100000000000002,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.2210000000000001,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":1.091},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.997,"15min":1.0529999999999999},"min":{"1min":0.67900000000000005,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.224,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":0.78900000000000003}]},{"osd":4,"last update":"Mon Nov 6 02:13:30 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.89500000000000002,"15min":0.90900000000000003},"min":{"1min":0.58099999999999996,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.1779999999999999,"5min":1.276,"15min":1.276},"last":0.63700000000000001},{"interface":"front","average":{"1min":1.006,"5min":0.99299999999999999,"15min":1.016},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.254,"5min":1.554,"15min":1.554},"last":0.94799999999999995}]},{"osd":6,"last update":"Mon Nov 6 02:13:30 2023","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":1.0089999999999999,"15min":1.048},"min":{"1min":0.71399999999999997,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.2569999999999999,"5min":1.454,"15min":1.454},"last":1.016},{"interface":"front","average":{"1min":0.98699999999999999,"5min":1.022,"15min":1.034},"min":{"1min":0.75,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.1930000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.76000000000000001}]},{"osd":7,"last update":"Mon Nov 6 02:13:30 2023","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.0609999999999999,"15min":1.071},"min":{"1min":0.77900000000000003,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.2330000000000001,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":1.048},{"interface":"front","average":{"1min":1.109,"5min":1.1120000000000001,"15min":1.1399999999999999},"min":{"1min":0.84599999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.3799999999999999,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":1.0740000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574914,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":916,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":937984,"data_stored":418981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:13:27 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.877,"15min":0.84199999999999997},"min":{"1min":0.65100000000000002,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.0840000000000001,"5min":1.343,"15min":1.343},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.83299999999999996,"15min":0.79100000000000004},"min":{"1min":0.626,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.1699999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.77000000000000002}]},{"osd":1,"last update":"Mon Nov 6 02:13:27 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.86799999999999999,"15min":0.85199999999999998},"min":{"1min":0.57299999999999995,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.0329999999999999,"5min":1.462,"15min":1.462},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.93300000000000005,"15min":0.90500000000000003},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.224,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.59699999999999998}]},{"osd":2,"last update":"Mon Nov 6 02:13:27 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.93799999999999994,"15min":0.99199999999999999},"min":{"1min":0.70299999999999996,"5min":0.503,"15min":0.503},"max":{"1min":1.173,"5min":1.615,"15min":1.615},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.95499999999999996,"15min":0.93500000000000005},"min":{"1min":0.72499999999999998,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.163,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.97899999999999998}]},{"osd":3,"last update":"Mon Nov 6 02:13:27 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.88200000000000001,"15min":0.90600000000000003},"min":{"1min":0.61499999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.216,"5min":1.52,"15min":1.52},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.997,"15min":1.0449999999999999},"min":{"1min":0.66900000000000004,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.2050000000000001,"5min":1.595,"15min":1.595},"last":0.80900000000000005}]},{"osd":5,"last update":"Mon Nov 6 02:13:27 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.99099999999999999,"15min":1.0309999999999999},"min":{"1min":0.64500000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.339,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":1.0800000000000001},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.97599999999999998,"15min":1.0289999999999999},"min":{"1min":0.68899999999999995,"5min":0.629,"15min":0.629},"max":{"1min":1.161,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":0.86699999999999999}]},{"osd":6,"last update":"Mon Nov 6 02:13:27 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":1.008,"15min":1.0649999999999999},"min":{"1min":0.69599999999999995,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.268,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":1.03},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.1120000000000001,"15min":1.169},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.2789999999999999,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":1.129}]},{"osd":7,"last update":"Mon Nov 6 02:13:27 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.0489999999999999,"15min":1.079},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.294,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.93899999999999995},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.133,"15min":1.204},"min":{"1min":0.77500000000000002,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.3149999999999999,"5min":1.7410000000000001,"15min":1.7410000000000001},"last":1.1060000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574913,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31964,"kb_used_data":916,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339876,"statfs":{"total":96636764160,"available":96604033024,"internally_reserved":0,"allocated":937984,"data_stored":418981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:13:28 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.63200000000000001,"15min":0.64200000000000002},"min":{"1min":0.40799999999999997,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":1.198,"5min":1.198,"15min":1.198},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.64000000000000001,"15min":0.63},"min":{"1min":0.38700000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.72099999999999997}]},{"osd":1,"last update":"Mon Nov 6 02:13:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.61699999999999999,"15min":0.627},"min":{"1min":0.34599999999999997,"5min":0.248,"15min":0.248},"max":{"1min":0.92600000000000005,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.73999999999999999,"15min":0.73599999999999999},"min":{"1min":0.53000000000000003,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.60099999999999998}]},{"osd":2,"last update":"Mon Nov 6 02:13:28 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.71899999999999997,"15min":0.72899999999999998},"min":{"1min":0.45400000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.107,"5min":1.139,"15min":1.139},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.80800000000000005,"15min":0.81799999999999995},"min":{"1min":0.48099999999999998,"5min":0.308,"15min":0.308},"max":{"1min":1.212,"5min":1.405,"15min":1.405},"last":0.75700000000000001}]},{"osd":4,"last update":"Mon Nov 6 02:13:28 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.92000000000000004,"15min":0.92600000000000005},"min":{"1min":0.72599999999999998,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.0820000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.95299999999999996,"15min":0.98299999999999998},"min":{"1min":0.65100000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.1459999999999999,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.97399999999999998}]},{"osd":5,"last update":"Mon Nov 6 02:13:28 2023","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.98899999999999999,"15min":1.0149999999999999},"min":{"1min":0.72799999999999998,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.333,"5min":1.375,"15min":1.375},"last":0.81299999999999994},{"interface":"front","average":{"1min":1.091,"5min":1.0389999999999999,"15min":1.0669999999999999},"min":{"1min":0.627,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.4890000000000001,"5min":2.21,"15min":2.21},"last":1.0269999999999999}]},{"osd":6,"last update":"Mon Nov 6 02:13:28 2023","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.016,"15min":1.022},"min":{"1min":0.61299999999999999,"5min":0.439,"15min":0.439},"max":{"1min":1.3740000000000001,"5min":1.395,"15min":1.395},"last":0.78900000000000003},{"interface":"front","average":{"1min":1.036,"5min":1.04,"15min":1.0609999999999999},"min":{"1min":0.54600000000000004,"5min":0.375,"15min":0.375},"max":{"1min":1.3069999999999999,"5min":1.554,"15min":1.554},"last":1.107}]},{"osd":7,"last update":"Mon Nov 6 02:13:28 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.1080000000000001,"15min":1.1240000000000001},"min":{"1min":0.66800000000000004,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.421,"5min":2.2829999999999999,"15min":2.2829999999999999},"last":0.94099999999999995},{"interface":"front","average":{"1min":1.091,"5min":1.0940000000000001,"15min":1.1020000000000001},"min":{"1min":0.70999999999999996,"5min":0.497,"15min":0.497},"max":{"1min":1.4490000000000001,"5min":1.504,"15min":1.504},"last":1.048}]}]},{"osd":2,"up_from":13,"seq":55834574913,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31964,"kb_used_data":916,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339876,"statfs":{"total":96636764160,"available":96604033024,"internally_reserved":0,"allocated":937984,"data_stored":418981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:13:24 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.57699999999999996,"15min":0.52500000000000002},"min":{"1min":0.33100000000000002,"5min":0.252,"15min":0.252},"max":{"1min":0.85699999999999998,"5min":1.079,"15min":1.079},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.625,"15min":0.56699999999999995},"min":{"1min":0.505,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.1060000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":0.56999999999999995}]},{"osd":1,"last update":"Mon Nov 6 02:13:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.63500000000000001,"15min":0.58499999999999996},"min":{"1min":0.38800000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.88300000000000001,"5min":1.48,"15min":1.48},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.70599999999999996,"15min":0.61399999999999999},"min":{"1min":0.377,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.282,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":0.70499999999999996}]},{"osd":3,"last update":"Mon Nov 6 02:13:24 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.70099999999999996,"15min":0.63200000000000001},"min":{"1min":0.48199999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.98999999999999999,"5min":1.155,"15min":1.155},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.749,"15min":0.67700000000000005},"min":{"1min":0.41999999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.81499999999999995}]},{"osd":4,"last update":"Mon Nov 6 02:13:24 2023","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.90900000000000003,"15min":0.83499999999999996},"min":{"1min":0.58499999999999996,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.2150000000000001,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.93500000000000005,"15min":0.92100000000000004},"min":{"1min":0.73099999999999998,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.3120000000000001,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":1.0589999999999999}]},{"osd":5,"last update":"Mon Nov 6 02:13:24 2023","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":0.96799999999999997,"15min":0.95099999999999996},"min":{"1min":0.75,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.2270000000000001,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":0.93999999999999995},{"interface":"front","average":{"1min":1,"5min":0.97099999999999997,"15min":0.94599999999999995},"min":{"1min":0.628,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.3109999999999999,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.96399999999999997}]},{"osd":6,"last update":"Mon Nov 6 02:13:24 2023","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":0.97099999999999997,"15min":0.94899999999999995},"min":{"1min":0.59899999999999998,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.3129999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.0229999999999999,"15min":1.0049999999999999},"min":{"1min":0.73299999999999998,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.2769999999999999,"5min":1.677,"15min":1.677},"last":1.1479999999999999}]},{"osd":7,"last update":"Mon Nov 6 02:13:24 2023","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.026,"15min":0.99199999999999999},"min":{"1min":0.72199999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.4079999999999999,"5min":1.659,"15min":1.659},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.123,"5min":1.0820000000000001,"15min":1.0549999999999999},"min":{"1min":0.79300000000000004,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.3819999999999999,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":1.087}]}]},{"osd":1,"up_from":13,"seq":55834574914,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31964,"kb_used_data":916,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339876,"statfs":{"total":96636764160,"available":96604033024,"internally_reserved":0,"allocated":937984,"data_stored":418981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:13:27 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.63900000000000001,"15min":0.63},"min":{"1min":0.35899999999999999,"5min":0.224,"15min":0.224},"max":{"1min":1.343,"5min":1.603,"15min":1.603},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.71899999999999997,"15min":0.67400000000000004},"min":{"1min":0.38200000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":2.161,"5min":2.161,"15min":2.161},"last":1.081}]},{"osd":2,"last update":"Mon Nov 6 02:13:27 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.65400000000000003,"15min":0.63200000000000001},"min":{"1min":0.26500000000000001,"5min":0.253,"15min":0.253},"max":{"1min":1.256,"5min":1.649,"15min":1.649},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.753,"15min":0.747},"min":{"1min":0.41099999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":1.5249999999999999,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.88400000000000001}]},{"osd":3,"last update":"Mon Nov 6 02:13:27 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.81299999999999994,"15min":0.80600000000000005},"min":{"1min":0.49099999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.6319999999999999,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":1.131},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.81299999999999994,"15min":0.81100000000000005},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.1799999999999999,"5min":1.8149999999999999,"15min":1.8149999999999999},"last":0.94199999999999995}]},{"osd":4,"last update":"Mon Nov 6 02:13:27 2023","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":0.97899999999999998,"15min":0.98799999999999999},"min":{"1min":0.78500000000000003,"5min":0.504,"15min":0.504},"max":{"1min":2.2759999999999998,"5min":2.2759999999999998,"15min":2.2759999999999998},"last":1.196},{"interface":"front","average":{"1min":1.0740000000000001,"5min":0.97699999999999998,"15min":0.98799999999999999},"min":{"1min":0.64200000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":2.9180000000000001,"5min":2.9180000000000001,"15min":2.9180000000000001},"last":1.0049999999999999}]},{"osd":5,"last update":"Mon Nov 6 02:13:27 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":0.98099999999999998,"15min":0.97099999999999997},"min":{"1min":0.68100000000000005,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":2.7469999999999999,"5min":2.7469999999999999,"15min":2.7469999999999999},"last":1.1659999999999999},{"interface":"front","average":{"1min":1.1579999999999999,"5min":1.069,"15min":1.075},"min":{"1min":0.81100000000000005,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.921,"5min":1.921,"15min":1.921},"last":1.044}]},{"osd":6,"last update":"Mon Nov 6 02:13:27 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.113,"15min":1.1060000000000001},"min":{"1min":0.83099999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":3.5710000000000002,"5min":3.5710000000000002,"15min":3.5710000000000002},"last":1.278},{"interface":"front","average":{"1min":1.157,"5min":1.0629999999999999,"15min":1.0720000000000001},"min":{"1min":0.60299999999999998,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":2.843,"5min":2.843,"15min":2.843},"last":0.83499999999999996}]},{"osd":7,"last update":"Mon Nov 6 02:13:27 2023","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.1180000000000001,"15min":1.1200000000000001},"min":{"1min":0.89600000000000002,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":2.0009999999999999,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":1.2330000000000001},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.1639999999999999,"15min":1.1619999999999999},"min":{"1min":0.91100000000000003,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":3.4790000000000001,"5min":3.4790000000000001,"15min":3.4790000000000001},"last":1.3520000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574914,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32540,"kb_used_data":1492,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339300,"statfs":{"total":96636764160,"available":96603443200,"internally_reserved":0,"allocated":1527808,"data_stored":1008330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 02:13:26 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.61499999999999999,"15min":0.60299999999999998},"min":{"1min":0.53100000000000003,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.0289999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.755,"5min":0.66900000000000004,"15min":0.64700000000000002},"min":{"1min":0.53800000000000003,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.079,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.46300000000000002}]},{"osd":2,"last update":"Mon Nov 6 02:13:26 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.68600000000000005,"15min":0.67200000000000004},"min":{"1min":0.42999999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.1080000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.98299999999999998},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.69799999999999995,"15min":0.68200000000000005},"min":{"1min":0.57299999999999995,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.75700000000000001}]},{"osd":3,"last update":"Mon Nov 6 02:13:26 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.67200000000000004,"15min":0.64300000000000002},"min":{"1min":0.52500000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.1659999999999999,"5min":1.429,"15min":1.429},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.87,"5min":0.78300000000000003,"15min":0.748},"min":{"1min":0.69999999999999996,"5min":0.317,"15min":0.317},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.89000000000000001}]},{"osd":4,"last update":"Mon Nov 6 02:13:26 2023","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.90200000000000002,"15min":0.85199999999999998},"min":{"1min":0.76800000000000002,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.1779999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":1.0589999999999999,"5min":0.96099999999999997,"15min":0.94499999999999995},"min":{"1min":0.78900000000000003,"5min":0.5,"15min":0.5},"max":{"1min":1.292,"5min":1.387,"15min":1.387},"last":1.137}]},{"osd":5,"last update":"Mon Nov 6 02:13:26 2023","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":0.95699999999999996,"15min":0.95099999999999996},"min":{"1min":0.81799999999999995,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.006,"5min":0.93400000000000005,"15min":0.89200000000000002},"min":{"1min":0.81100000000000005,"5min":0.496,"15min":0.496},"max":{"1min":1.2549999999999999,"5min":1.448,"15min":1.448},"last":1.101}]},{"osd":6,"last update":"Mon Nov 6 02:13:26 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.0389999999999999,"15min":1.0369999999999999},"min":{"1min":0.73899999999999999,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.385,"5min":1.589,"15min":1.589},"last":1.236},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.0269999999999999,"15min":1.0089999999999999},"min":{"1min":0.76000000000000001,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.419,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":1.1779999999999999}]},{"osd":7,"last update":"Mon Nov 6 02:13:26 2023","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.026,"15min":1.0209999999999999},"min":{"1min":0.76400000000000001,"5min":0.504,"15min":0.504},"max":{"1min":1.504,"5min":1.504,"15min":1.504},"last":1.022},{"interface":"front","average":{"1min":1.073,"5min":0.998,"15min":1.0049999999999999},"min":{"1min":0.74299999999999999,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.3999999999999999,"5min":1.556,"15min":1.556},"last":0.873}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T02:13:39.434 INFO:tasks.ceph.mds.c.smithi049.stderr:2023-11-06T02:13:39.438+0000 7f0988600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-06T02:13:42.228 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T02:13:42.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.370+0000 7fb4d5db9640 1 Processor -- start 2023-11-06T02:13:42.365 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.370+0000 7fb4d5db9640 1 -- start start 2023-11-06T02:13:42.366 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.370+0000 7fb4d5db9640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fb4d015ab50 0x7fb4d00c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:42.366 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.370+0000 7fb4d5db9640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb4d00b0790 0x7fb4d00c2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:42.366 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.370+0000 7fb4d5db9640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4d00cab80 con 0x7fb4d00cb010 2023-11-06T02:13:42.367 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.370+0000 7fb4d5db9640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4d015af30 con 0x7fb4d015ab50 2023-11-06T02:13:42.367 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.370+0000 7fb4d5db9640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d00cb5b0 con 0x7fb4d00b0790 2023-11-06T02:13:42.367 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.370+0000 7fb4cffff640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb4d00b0790 0x7fb4d00c2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:42.367 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.370+0000 7fb4cf7fe640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fb4d015ab50 0x7fb4d00c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:42.367 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.370+0000 7fb4cf7fe640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fb4d015ab50 0x7fb4d00c9bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.49:57540/0 (socket says 172.21.15.49:57540) 2023-11-06T02:13:42.369 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.370+0000 7fb4cffff640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb4d00b0790 0x7fb4d00c2710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.49:59332/0 (socket says 172.21.15.49:59332) 2023-11-06T02:13:42.369 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.370+0000 7fb4cf7fe640 1 -- 172.21.15.49:0/2655107452 learned_addr learned my addr 172.21.15.49:0/2655107452 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:42.370 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.370+0000 7fb4ceffd640 1 --1- 172.21.15.49:0/2655107452 >> v1:172.21.15.150:6789/0 conn(0x7fb4d00cb010 0x7fb4d00b0080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.49:39224/0 (socket says 172.21.15.49:39224) 2023-11-06T02:13:42.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.370+0000 7fb4cf7fe640 1 -- 172.21.15.49:0/2655107452 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb4d00b0790 msgr2=0x7fb4d00c2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:42.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.370+0000 7fb4cf7fe640 1 --2- 172.21.15.49:0/2655107452 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb4d00b0790 0x7fb4d00c2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:42.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.370+0000 7fb4cf7fe640 1 -- 172.21.15.49:0/2655107452 >> v1:172.21.15.150:6789/0 conn(0x7fb4d00cb010 legacy=0x7fb4d00b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:42.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.370+0000 7fb4cf7fe640 1 -- 172.21.15.49:0/2655107452 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4d00cb890 con 0x7fb4d015ab50 2023-11-06T02:13:42.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.374+0000 7fb4cf7fe640 1 --2- 172.21.15.49:0/2655107452 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb4d015ab50 0x7fb4d00c9bb0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb4c0009ea0 tx=0x7fb4c001ca20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a4ca9ed151f2b33 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:42.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.374+0000 7fb4ce7fc640 1 -- 172.21.15.49:0/2655107452 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c00a58f0 con 0x7fb4d015ab50 2023-11-06T02:13:42.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.374+0000 7fb4ce7fc640 1 -- 172.21.15.49:0/2655107452 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4c00a5a50 con 0x7fb4d015ab50 2023-11-06T02:13:42.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.374+0000 7fb4ce7fc640 1 -- 172.21.15.49:0/2655107452 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c00a5de0 con 0x7fb4d015ab50 2023-11-06T02:13:42.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.374+0000 7fb4d5db9640 1 -- 172.21.15.49:0/2655107452 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb4d015ab50 msgr2=0x7fb4d00c9bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:42.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.374+0000 7fb4d5db9640 1 --2- 172.21.15.49:0/2655107452 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb4d015ab50 0x7fb4d00c9bb0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb4c0009ea0 tx=0x7fb4c001ca20 comp rx=0 tx=0).stop 2023-11-06T02:13:42.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.374+0000 7fb4d5db9640 1 -- 172.21.15.49:0/2655107452 shutdown_connections 2023-11-06T02:13:42.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.374+0000 7fb4d5db9640 1 --2- 172.21.15.49:0/2655107452 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb4d00b0790 0x7fb4d00c2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:42.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.374+0000 7fb4d5db9640 1 --2- 172.21.15.49:0/2655107452 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb4d015ab50 0x7fb4d00c9bb0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:42.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.374+0000 7fb4d5db9640 1 -- 172.21.15.49:0/2655107452 >> 172.21.15.49:0/2655107452 conn(0x7fb4d006c0a0 msgr2=0x7fb4d006c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:42.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.374+0000 7fb4d5db9640 1 -- 172.21.15.49:0/2655107452 shutdown_connections 2023-11-06T02:13:42.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.374+0000 7fb4d5db9640 1 -- 172.21.15.49:0/2655107452 wait complete. 2023-11-06T02:13:42.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.374+0000 7fb4d5db9640 1 Processor -- start 2023-11-06T02:13:42.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.374+0000 7fb4d5db9640 1 -- start start 2023-11-06T02:13:42.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.378+0000 7fb4d5db9640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb4d00b0790 0x7fb4d00c6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:42.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.378+0000 7fb4d5db9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb4d00cb010 0x7fb4d0154ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:42.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.378+0000 7fb4d5db9640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb4d015ab50 0x7fb4d0155730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:42.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.378+0000 7fb4d5db9640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d00c5770 con 0x7fb4d00cb010 2023-11-06T02:13:42.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.378+0000 7fb4d5db9640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d00c5a50 con 0x7fb4d015ab50 2023-11-06T02:13:42.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.378+0000 7fb4d5db9640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d00c5d30 con 0x7fb4d00b0790 2023-11-06T02:13:42.378 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.378+0000 7fb4ceffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb4d00cb010 0x7fb4d0154ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:42.378 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.378+0000 7fb4ceffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb4d00cb010 0x7fb4d0154ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47900/0 (socket says 172.21.15.49:47900) 2023-11-06T02:13:42.378 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.378+0000 7fb4ceffd640 1 -- 172.21.15.49:0/3144083046 learned_addr learned my addr 172.21.15.49:0/3144083046 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:42.378 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.378+0000 7fb4cf7fe640 1 --2- 172.21.15.49:0/3144083046 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb4d00b0790 0x7fb4d00c6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:42.378 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.378+0000 7fb4cffff640 1 --2- 172.21.15.49:0/3144083046 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb4d015ab50 0x7fb4d0155730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:42.379 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.378+0000 7fb4ceffd640 1 -- 172.21.15.49:0/3144083046 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb4d00b0790 msgr2=0x7fb4d00c6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:42.379 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.378+0000 7fb4ceffd640 1 --2- 172.21.15.49:0/3144083046 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb4d00b0790 0x7fb4d00c6810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:42.379 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.378+0000 7fb4ceffd640 1 -- 172.21.15.49:0/3144083046 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb4d015ab50 msgr2=0x7fb4d0155730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:42.379 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.378+0000 7fb4ceffd640 1 --2- 172.21.15.49:0/3144083046 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb4d015ab50 0x7fb4d0155730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:42.379 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.378+0000 7fb4ceffd640 1 -- 172.21.15.49:0/3144083046 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4c0009b30 con 0x7fb4d00cb010 2023-11-06T02:13:42.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.378+0000 7fb4cffff640 1 --2- 172.21.15.49:0/3144083046 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb4d015ab50 0x7fb4d0155730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:13:42.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.378+0000 7fb4ceffd640 1 --2- 172.21.15.49:0/3144083046 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb4d00cb010 0x7fb4d0154ef0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fb4b800c460 tx=0x7fb4b800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:42.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.378+0000 7fb4c7fff640 1 -- 172.21.15.49:0/3144083046 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b8007480 con 0x7fb4d00cb010 2023-11-06T02:13:42.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.378+0000 7fb4c7fff640 1 -- 172.21.15.49:0/3144083046 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4b8007600 con 0x7fb4d00cb010 2023-11-06T02:13:42.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.378+0000 7fb4c7fff640 1 -- 172.21.15.49:0/3144083046 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b8004a30 con 0x7fb4d00cb010 2023-11-06T02:13:42.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.378+0000 7fb4d5db9640 1 -- 172.21.15.49:0/3144083046 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4d00c6010 con 0x7fb4d00cb010 2023-11-06T02:13:42.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.378+0000 7fb4d5db9640 1 -- 172.21.15.49:0/3144083046 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4d00c6d50 con 0x7fb4d00cb010 2023-11-06T02:13:42.382 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.378+0000 7fb4d5db9640 1 -- 172.21.15.49:0/3144083046 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4d00b0de0 con 0x7fb4d00cb010 2023-11-06T02:13:42.382 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.378+0000 7fb4c7fff640 1 -- 172.21.15.49:0/3144083046 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fb4b8004b90 con 0x7fb4d00cb010 2023-11-06T02:13:42.382 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.378+0000 7fb4c7fff640 1 --2- 172.21.15.49:0/3144083046 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb4b0065a60 0x7fb4b0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:42.383 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.378+0000 7fb4c7fff640 1 -- 172.21.15.49:0/3144083046 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb4b80824e0 con 0x7fb4d00cb010 2023-11-06T02:13:42.383 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.378+0000 7fb4cf7fe640 1 --2- 172.21.15.49:0/3144083046 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb4b0065a60 0x7fb4b0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:42.383 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.382+0000 7fb4cf7fe640 1 --2- 172.21.15.49:0/3144083046 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb4b0065a60 0x7fb4b0067f20 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fb4c00ae000 tx=0x7fb4c00a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:42.383 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.382+0000 7fb4c7fff640 1 -- 172.21.15.49:0/3144083046 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fb4b804fd40 con 0x7fb4d00cb010 2023-11-06T02:13:42.535 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.538+0000 7fb4d5db9640 1 -- 172.21.15.49:0/3144083046 --> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb4d00cbe50 con 0x7fb4b0065a60 2023-11-06T02:13:42.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.562+0000 7fb4c7fff640 1 -- 172.21.15.49:0/3144083046 <== mgr.4100 v2:172.21.15.150:6832/38104 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+309860 (secure 0 0 0) 0x7fb4c00bc090 con 0x7fb4b0065a60 2023-11-06T02:13:42.561 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:13:42.563 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2023-11-06T02:13:42.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.566+0000 7fb4d5db9640 1 -- 172.21.15.49:0/3144083046 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb4b0065a60 msgr2=0x7fb4b0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:42.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.566+0000 7fb4d5db9640 1 --2- 172.21.15.49:0/3144083046 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb4b0065a60 0x7fb4b0067f20 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fb4c00ae000 tx=0x7fb4c00a7070 comp rx=0 tx=0).stop 2023-11-06T02:13:42.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.566+0000 7fb4d5db9640 1 -- 172.21.15.49:0/3144083046 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb4d00cb010 msgr2=0x7fb4d0154ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:42.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.566+0000 7fb4d5db9640 1 --2- 172.21.15.49:0/3144083046 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb4d00cb010 0x7fb4d0154ef0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fb4b800c460 tx=0x7fb4b800c920 comp rx=0 tx=0).stop 2023-11-06T02:13:42.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.566+0000 7fb4d5db9640 1 -- 172.21.15.49:0/3144083046 shutdown_connections 2023-11-06T02:13:42.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.566+0000 7fb4d5db9640 1 --2- 172.21.15.49:0/3144083046 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb4b0065a60 0x7fb4b0067f20 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:42.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.566+0000 7fb4d5db9640 1 --2- 172.21.15.49:0/3144083046 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb4d00cb010 0x7fb4d0154ef0 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:42.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.566+0000 7fb4d5db9640 1 --2- 172.21.15.49:0/3144083046 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb4d015ab50 0x7fb4d0155730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:42.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.566+0000 7fb4d5db9640 1 --2- 172.21.15.49:0/3144083046 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb4d00b0790 0x7fb4d00c6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:42.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.566+0000 7fb4d5db9640 1 -- 172.21.15.49:0/3144083046 >> 172.21.15.49:0/3144083046 conn(0x7fb4d006c0a0 msgr2=0x7fb4d00731f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:42.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.566+0000 7fb4d5db9640 1 -- 172.21.15.49:0/3144083046 shutdown_connections 2023-11-06T02:13:42.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.566+0000 7fb4d5db9640 1 -- 172.21.15.49:0/3144083046 wait complete. 2023-11-06T02:13:42.581 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":235,"stamp":"2023-11-06T02:13:41.288132+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":191,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":249028,"kb_used_data":8836,"kb_used_omap":0,"kb_used_meta":240128,"kb_avail":754725692,"statfs":{"total":773094113280,"available":772839108608,"internally_reserved":0,"allocated":9048064,"data_stored":4699561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":245891072},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,4],"upper_bound":32},"perf_stat":{"commit_latency_ms":69,"apply_latency_ms":69,"commit_latency_ns":69000000,"apply_latency_ns":69000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003481"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.690204+0000","last_change":"2023-11-06T02:13:33.686136+0000","last_active":"2023-11-06T02:13:33.690204+0000","last_peered":"2023-11-06T02:13:33.690204+0000","last_clean":"2023-11-06T02:13:33.690204+0000","last_became_active":"2023-11-06T02:13:33.685465+0000","last_became_peered":"2023-11-06T02:13:33.685465+0000","last_unstale":"2023-11-06T02:13:33.690204+0000","last_undegraded":"2023-11-06T02:13:33.690204+0000","last_fullsized":"2023-11-06T02:13:33.690204+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:18:07.485044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.655058+0000","last_change":"2023-11-06T02:13:33.653384+0000","last_active":"2023-11-06T02:13:33.655058+0000","last_peered":"2023-11-06T02:13:33.655058+0000","last_clean":"2023-11-06T02:13:33.655058+0000","last_became_active":"2023-11-06T02:13:33.653110+0000","last_became_peered":"2023-11-06T02:13:33.653110+0000","last_unstale":"2023-11-06T02:13:33.655058+0000","last_undegraded":"2023-11-06T02:13:33.655058+0000","last_fullsized":"2023-11-06T02:13:33.655058+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:40:58.713688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.659312+0000","last_change":"2023-11-06T02:13:33.654336+0000","last_active":"2023-11-06T02:13:33.659312+0000","last_peered":"2023-11-06T02:13:33.659312+0000","last_clean":"2023-11-06T02:13:33.659312+0000","last_became_active":"2023-11-06T02:13:33.654042+0000","last_became_peered":"2023-11-06T02:13:33.654042+0000","last_unstale":"2023-11-06T02:13:33.659312+0000","last_undegraded":"2023-11-06T02:13:33.659312+0000","last_fullsized":"2023-11-06T02:13:33.659312+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:42:42.388106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.693694+0000","last_change":"2023-11-06T02:13:33.689067+0000","last_active":"2023-11-06T02:13:33.693694+0000","last_peered":"2023-11-06T02:13:33.693694+0000","last_clean":"2023-11-06T02:13:33.693694+0000","last_became_active":"2023-11-06T02:13:33.688553+0000","last_became_peered":"2023-11-06T02:13:33.688553+0000","last_unstale":"2023-11-06T02:13:33.693694+0000","last_undegraded":"2023-11-06T02:13:33.693694+0000","last_fullsized":"2023-11-06T02:13:33.693694+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:54:33.444383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.665816+0000","last_change":"2023-11-06T02:13:33.662627+0000","last_active":"2023-11-06T02:13:33.665816+0000","last_peered":"2023-11-06T02:13:33.665816+0000","last_clean":"2023-11-06T02:13:33.665816+0000","last_became_active":"2023-11-06T02:13:33.662358+0000","last_became_peered":"2023-11-06T02:13:33.662358+0000","last_unstale":"2023-11-06T02:13:33.665816+0000","last_undegraded":"2023-11-06T02:13:33.665816+0000","last_fullsized":"2023-11-06T02:13:33.665816+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:22:24.191027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.688691+0000","last_change":"2023-11-06T02:13:33.686083+0000","last_active":"2023-11-06T02:13:33.688691+0000","last_peered":"2023-11-06T02:13:33.688691+0000","last_clean":"2023-11-06T02:13:33.688691+0000","last_became_active":"2023-11-06T02:13:33.685735+0000","last_became_peered":"2023-11-06T02:13:33.685735+0000","last_unstale":"2023-11-06T02:13:33.688691+0000","last_undegraded":"2023-11-06T02:13:33.688691+0000","last_fullsized":"2023-11-06T02:13:33.688691+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:23:55.083630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.659232+0000","last_change":"2023-11-06T02:13:33.651410+0000","last_active":"2023-11-06T02:13:33.659232+0000","last_peered":"2023-11-06T02:13:33.659232+0000","last_clean":"2023-11-06T02:13:33.659232+0000","last_became_active":"2023-11-06T02:13:33.650693+0000","last_became_peered":"2023-11-06T02:13:33.650693+0000","last_unstale":"2023-11-06T02:13:33.659232+0000","last_undegraded":"2023-11-06T02:13:33.659232+0000","last_fullsized":"2023-11-06T02:13:33.659232+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:40:54.464756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.667011+0000","last_change":"2023-11-06T02:13:33.666241+0000","last_active":"2023-11-06T02:13:33.667011+0000","last_peered":"2023-11-06T02:13:33.667011+0000","last_clean":"2023-11-06T02:13:33.667011+0000","last_became_active":"2023-11-06T02:13:33.665947+0000","last_became_peered":"2023-11-06T02:13:33.665947+0000","last_unstale":"2023-11-06T02:13:33.667011+0000","last_undegraded":"2023-11-06T02:13:33.667011+0000","last_fullsized":"2023-11-06T02:13:33.667011+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:39:02.068604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.692538+0000","last_change":"2023-11-06T02:13:33.669488+0000","last_active":"2023-11-06T02:13:33.692538+0000","last_peered":"2023-11-06T02:13:33.692538+0000","last_clean":"2023-11-06T02:13:33.692538+0000","last_became_active":"2023-11-06T02:13:33.668802+0000","last_became_peered":"2023-11-06T02:13:33.668802+0000","last_unstale":"2023-11-06T02:13:33.692538+0000","last_undegraded":"2023-11-06T02:13:33.692538+0000","last_fullsized":"2023-11-06T02:13:33.692538+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:02:51.697659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.693412+0000","last_change":"2023-11-06T02:13:33.680722+0000","last_active":"2023-11-06T02:13:33.693412+0000","last_peered":"2023-11-06T02:13:33.693412+0000","last_clean":"2023-11-06T02:13:33.693412+0000","last_became_active":"2023-11-06T02:13:33.680037+0000","last_became_peered":"2023-11-06T02:13:33.680037+0000","last_unstale":"2023-11-06T02:13:33.693412+0000","last_undegraded":"2023-11-06T02:13:33.693412+0000","last_fullsized":"2023-11-06T02:13:33.693412+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:12:00.717133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.659139+0000","last_change":"2023-11-06T02:13:33.657522+0000","last_active":"2023-11-06T02:13:33.659139+0000","last_peered":"2023-11-06T02:13:33.659139+0000","last_clean":"2023-11-06T02:13:33.659139+0000","last_became_active":"2023-11-06T02:13:33.657120+0000","last_became_peered":"2023-11-06T02:13:33.657120+0000","last_unstale":"2023-11-06T02:13:33.659139+0000","last_undegraded":"2023-11-06T02:13:33.659139+0000","last_fullsized":"2023-11-06T02:13:33.659139+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:42:31.765331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.693545+0000","last_change":"2023-11-06T02:13:33.682264+0000","last_active":"2023-11-06T02:13:33.693545+0000","last_peered":"2023-11-06T02:13:33.693545+0000","last_clean":"2023-11-06T02:13:33.693545+0000","last_became_active":"2023-11-06T02:13:33.681170+0000","last_became_peered":"2023-11-06T02:13:33.681170+0000","last_unstale":"2023-11-06T02:13:33.693545+0000","last_undegraded":"2023-11-06T02:13:33.693545+0000","last_fullsized":"2023-11-06T02:13:33.693545+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:21:24.032595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.657894+0000","last_change":"2023-11-06T02:13:33.652084+0000","last_active":"2023-11-06T02:13:33.657894+0000","last_peered":"2023-11-06T02:13:33.657894+0000","last_clean":"2023-11-06T02:13:33.657894+0000","last_became_active":"2023-11-06T02:13:33.651629+0000","last_became_peered":"2023-11-06T02:13:33.651629+0000","last_unstale":"2023-11-06T02:13:33.657894+0000","last_undegraded":"2023-11-06T02:13:33.657894+0000","last_fullsized":"2023-11-06T02:13:33.657894+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:32:06.316090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.688701+0000","last_change":"2023-11-06T02:13:33.686183+0000","last_active":"2023-11-06T02:13:33.688701+0000","last_peered":"2023-11-06T02:13:33.688701+0000","last_clean":"2023-11-06T02:13:33.688701+0000","last_became_active":"2023-11-06T02:13:33.685800+0000","last_became_peered":"2023-11-06T02:13:33.685800+0000","last_unstale":"2023-11-06T02:13:33.688701+0000","last_undegraded":"2023-11-06T02:13:33.688701+0000","last_fullsized":"2023-11-06T02:13:33.688701+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:11:19.807848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.654500+0000","last_change":"2023-11-06T02:13:33.651516+0000","last_active":"2023-11-06T02:13:33.654500+0000","last_peered":"2023-11-06T02:13:33.654500+0000","last_clean":"2023-11-06T02:13:33.654500+0000","last_became_active":"2023-11-06T02:13:33.651204+0000","last_became_peered":"2023-11-06T02:13:33.651204+0000","last_unstale":"2023-11-06T02:13:33.654500+0000","last_undegraded":"2023-11-06T02:13:33.654500+0000","last_fullsized":"2023-11-06T02:13:33.654500+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:28:22.791353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.685867+0000","last_change":"2023-11-06T02:13:33.682181+0000","last_active":"2023-11-06T02:13:33.685867+0000","last_peered":"2023-11-06T02:13:33.685867+0000","last_clean":"2023-11-06T02:13:33.685867+0000","last_became_active":"2023-11-06T02:13:33.681740+0000","last_became_peered":"2023-11-06T02:13:33.681740+0000","last_unstale":"2023-11-06T02:13:33.685867+0000","last_undegraded":"2023-11-06T02:13:33.685867+0000","last_fullsized":"2023-11-06T02:13:33.685867+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:21:24.032595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.674274+0000","last_change":"2023-11-06T02:13:33.673178+0000","last_active":"2023-11-06T02:13:33.674274+0000","last_peered":"2023-11-06T02:13:33.674274+0000","last_clean":"2023-11-06T02:13:33.674274+0000","last_became_active":"2023-11-06T02:13:33.672781+0000","last_became_peered":"2023-11-06T02:13:33.672781+0000","last_unstale":"2023-11-06T02:13:33.674274+0000","last_undegraded":"2023-11-06T02:13:33.674274+0000","last_fullsized":"2023-11-06T02:13:33.674274+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:54:43.806737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.688319+0000","last_change":"2023-11-06T02:13:33.687066+0000","last_active":"2023-11-06T02:13:33.688319+0000","last_peered":"2023-11-06T02:13:33.688319+0000","last_clean":"2023-11-06T02:13:33.688319+0000","last_became_active":"2023-11-06T02:13:33.686729+0000","last_became_peered":"2023-11-06T02:13:33.686729+0000","last_unstale":"2023-11-06T02:13:33.688319+0000","last_undegraded":"2023-11-06T02:13:33.688319+0000","last_fullsized":"2023-11-06T02:13:33.688319+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:35:43.101828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.663426+0000","last_change":"2023-11-06T02:13:33.656330+0000","last_active":"2023-11-06T02:13:33.663426+0000","last_peered":"2023-11-06T02:13:33.663426+0000","last_clean":"2023-11-06T02:13:33.663426+0000","last_became_active":"2023-11-06T02:13:33.656010+0000","last_became_peered":"2023-11-06T02:13:33.656010+0000","last_unstale":"2023-11-06T02:13:33.663426+0000","last_undegraded":"2023-11-06T02:13:33.663426+0000","last_fullsized":"2023-11-06T02:13:33.663426+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:20:11.090360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.658032+0000","last_change":"2023-11-06T02:13:33.652788+0000","last_active":"2023-11-06T02:13:33.658032+0000","last_peered":"2023-11-06T02:13:33.658032+0000","last_clean":"2023-11-06T02:13:33.658032+0000","last_became_active":"2023-11-06T02:13:33.651573+0000","last_became_peered":"2023-11-06T02:13:33.651573+0000","last_unstale":"2023-11-06T02:13:33.658032+0000","last_undegraded":"2023-11-06T02:13:33.658032+0000","last_fullsized":"2023-11-06T02:13:33.658032+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:04:54.611423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.696210+0000","last_change":"2023-11-06T02:13:33.685129+0000","last_active":"2023-11-06T02:13:33.696210+0000","last_peered":"2023-11-06T02:13:33.696210+0000","last_clean":"2023-11-06T02:13:33.696210+0000","last_became_active":"2023-11-06T02:13:33.684306+0000","last_became_peered":"2023-11-06T02:13:33.684306+0000","last_unstale":"2023-11-06T02:13:33.696210+0000","last_undegraded":"2023-11-06T02:13:33.696210+0000","last_fullsized":"2023-11-06T02:13:33.696210+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:43:49.529761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.658965+0000","last_change":"2023-11-06T02:13:33.657261+0000","last_active":"2023-11-06T02:13:33.658965+0000","last_peered":"2023-11-06T02:13:33.658965+0000","last_clean":"2023-11-06T02:13:33.658965+0000","last_became_active":"2023-11-06T02:13:33.656893+0000","last_became_peered":"2023-11-06T02:13:33.656893+0000","last_unstale":"2023-11-06T02:13:33.658965+0000","last_undegraded":"2023-11-06T02:13:33.658965+0000","last_fullsized":"2023-11-06T02:13:33.658965+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:36:22.272311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.692831+0000","last_change":"2023-11-06T02:13:33.681064+0000","last_active":"2023-11-06T02:13:33.692831+0000","last_peered":"2023-11-06T02:13:33.692831+0000","last_clean":"2023-11-06T02:13:33.692831+0000","last_became_active":"2023-11-06T02:13:33.680644+0000","last_became_peered":"2023-11-06T02:13:33.680644+0000","last_unstale":"2023-11-06T02:13:33.692831+0000","last_undegraded":"2023-11-06T02:13:33.692831+0000","last_fullsized":"2023-11-06T02:13:33.692831+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:17:17.217507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.655009+0000","last_change":"2023-11-06T02:13:33.652060+0000","last_active":"2023-11-06T02:13:33.655009+0000","last_peered":"2023-11-06T02:13:33.655009+0000","last_clean":"2023-11-06T02:13:33.655009+0000","last_became_active":"2023-11-06T02:13:33.651804+0000","last_became_peered":"2023-11-06T02:13:33.651804+0000","last_unstale":"2023-11-06T02:13:33.655009+0000","last_undegraded":"2023-11-06T02:13:33.655009+0000","last_fullsized":"2023-11-06T02:13:33.655009+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:36:24.452901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.668376+0000","last_change":"2023-11-06T02:13:33.661732+0000","last_active":"2023-11-06T02:13:33.668376+0000","last_peered":"2023-11-06T02:13:33.668376+0000","last_clean":"2023-11-06T02:13:33.668376+0000","last_became_active":"2023-11-06T02:13:33.661445+0000","last_became_peered":"2023-11-06T02:13:33.661445+0000","last_unstale":"2023-11-06T02:13:33.668376+0000","last_undegraded":"2023-11-06T02:13:33.668376+0000","last_fullsized":"2023-11-06T02:13:33.668376+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:23:35.537414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.691324+0000","last_change":"2023-11-06T02:13:33.671648+0000","last_active":"2023-11-06T02:13:33.691324+0000","last_peered":"2023-11-06T02:13:33.691324+0000","last_clean":"2023-11-06T02:13:33.691324+0000","last_became_active":"2023-11-06T02:13:33.670474+0000","last_became_peered":"2023-11-06T02:13:33.670474+0000","last_unstale":"2023-11-06T02:13:33.691324+0000","last_undegraded":"2023-11-06T02:13:33.691324+0000","last_fullsized":"2023-11-06T02:13:33.691324+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:18:07.485044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.686384+0000","last_change":"2023-11-06T02:13:33.682685+0000","last_active":"2023-11-06T02:13:33.686384+0000","last_peered":"2023-11-06T02:13:33.686384+0000","last_clean":"2023-11-06T02:13:33.686384+0000","last_became_active":"2023-11-06T02:13:33.682013+0000","last_became_peered":"2023-11-06T02:13:33.682013+0000","last_unstale":"2023-11-06T02:13:33.686384+0000","last_undegraded":"2023-11-06T02:13:33.686384+0000","last_fullsized":"2023-11-06T02:13:33.686384+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:00:13.166796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.684677+0000","last_change":"2023-11-06T02:13:33.684066+0000","last_active":"2023-11-06T02:13:33.684677+0000","last_peered":"2023-11-06T02:13:33.684677+0000","last_clean":"2023-11-06T02:13:33.684677+0000","last_became_active":"2023-11-06T02:13:33.683798+0000","last_became_peered":"2023-11-06T02:13:33.683798+0000","last_unstale":"2023-11-06T02:13:33.684677+0000","last_undegraded":"2023-11-06T02:13:33.684677+0000","last_fullsized":"2023-11-06T02:13:33.684677+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:31:50.853359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.668690+0000","last_change":"2023-11-06T02:13:33.664572+0000","last_active":"2023-11-06T02:13:33.668690+0000","last_peered":"2023-11-06T02:13:33.668690+0000","last_clean":"2023-11-06T02:13:33.668690+0000","last_became_active":"2023-11-06T02:13:33.664296+0000","last_became_peered":"2023-11-06T02:13:33.664296+0000","last_unstale":"2023-11-06T02:13:33.668690+0000","last_undegraded":"2023-11-06T02:13:33.668690+0000","last_fullsized":"2023-11-06T02:13:33.668690+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:20:11.090360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.678996+0000","last_change":"2023-11-06T02:13:33.670726+0000","last_active":"2023-11-06T02:13:33.678996+0000","last_peered":"2023-11-06T02:13:33.678996+0000","last_clean":"2023-11-06T02:13:33.678996+0000","last_became_active":"2023-11-06T02:13:33.669369+0000","last_became_peered":"2023-11-06T02:13:33.669369+0000","last_unstale":"2023-11-06T02:13:33.678996+0000","last_undegraded":"2023-11-06T02:13:33.678996+0000","last_fullsized":"2023-11-06T02:13:33.678996+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:34:15.876381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.662973+0000","last_change":"2023-11-06T02:13:33.660099+0000","last_active":"2023-11-06T02:13:33.662973+0000","last_peered":"2023-11-06T02:13:33.662973+0000","last_clean":"2023-11-06T02:13:33.662973+0000","last_became_active":"2023-11-06T02:13:33.659716+0000","last_became_peered":"2023-11-06T02:13:33.659716+0000","last_unstale":"2023-11-06T02:13:33.662973+0000","last_undegraded":"2023-11-06T02:13:33.662973+0000","last_fullsized":"2023-11-06T02:13:33.662973+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:21:05.055643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.690154+0000","last_change":"2023-11-06T02:13:33.686040+0000","last_active":"2023-11-06T02:13:33.690154+0000","last_peered":"2023-11-06T02:13:33.690154+0000","last_clean":"2023-11-06T02:13:33.690154+0000","last_became_active":"2023-11-06T02:13:33.685539+0000","last_became_peered":"2023-11-06T02:13:33.685539+0000","last_unstale":"2023-11-06T02:13:33.690154+0000","last_undegraded":"2023-11-06T02:13:33.690154+0000","last_fullsized":"2023-11-06T02:13:33.690154+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:24:31.881719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.688311+0000","last_change":"2023-11-06T02:13:33.686846+0000","last_active":"2023-11-06T02:13:33.688311+0000","last_peered":"2023-11-06T02:13:33.688311+0000","last_clean":"2023-11-06T02:13:33.688311+0000","last_became_active":"2023-11-06T02:13:33.686556+0000","last_became_peered":"2023-11-06T02:13:33.686556+0000","last_unstale":"2023-11-06T02:13:33.688311+0000","last_undegraded":"2023-11-06T02:13:33.688311+0000","last_fullsized":"2023-11-06T02:13:33.688311+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:08:17.035454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"79'4","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.664006+0000","last_change":"2023-11-06T02:13:33.263624+0000","last_active":"2023-11-06T02:13:33.664006+0000","last_peered":"2023-11-06T02:13:33.664006+0000","last_clean":"2023-11-06T02:13:33.664006+0000","last_became_active":"2023-11-06T02:13:33.263309+0000","last_became_peered":"2023-11-06T02:13:33.263309+0000","last_unstale":"2023-11-06T02:13:33.664006+0000","last_undegraded":"2023-11-06T02:13:33.664006+0000","last_fullsized":"2023-11-06T02:13:33.664006+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:07:13.179013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.683019+0000","last_change":"2023-11-06T02:13:33.681901+0000","last_active":"2023-11-06T02:13:33.683019+0000","last_peered":"2023-11-06T02:13:33.683019+0000","last_clean":"2023-11-06T02:13:33.683019+0000","last_became_active":"2023-11-06T02:13:33.681614+0000","last_became_peered":"2023-11-06T02:13:33.681614+0000","last_unstale":"2023-11-06T02:13:33.683019+0000","last_undegraded":"2023-11-06T02:13:33.683019+0000","last_fullsized":"2023-11-06T02:13:33.683019+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:23:55.083630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"79'1","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.683428+0000","last_change":"2023-11-06T02:13:33.280567+0000","last_active":"2023-11-06T02:13:33.683428+0000","last_peered":"2023-11-06T02:13:33.683428+0000","last_clean":"2023-11-06T02:13:33.683428+0000","last_became_active":"2023-11-06T02:13:33.279882+0000","last_became_peered":"2023-11-06T02:13:33.279882+0000","last_unstale":"2023-11-06T02:13:33.683428+0000","last_undegraded":"2023-11-06T02:13:33.683428+0000","last_fullsized":"2023-11-06T02:13:33.683428+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:33:14.271873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.696174+0000","last_change":"2023-11-06T02:13:33.684925+0000","last_active":"2023-11-06T02:13:33.696174+0000","last_peered":"2023-11-06T02:13:33.696174+0000","last_clean":"2023-11-06T02:13:33.696174+0000","last_became_active":"2023-11-06T02:13:33.684452+0000","last_became_peered":"2023-11-06T02:13:33.684452+0000","last_unstale":"2023-11-06T02:13:33.696174+0000","last_undegraded":"2023-11-06T02:13:33.696174+0000","last_fullsized":"2023-11-06T02:13:33.696174+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:34:15.876381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"79'2","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.670623+0000","last_change":"2023-11-06T02:13:33.279088+0000","last_active":"2023-11-06T02:13:33.670623+0000","last_peered":"2023-11-06T02:13:33.670623+0000","last_clean":"2023-11-06T02:13:33.670623+0000","last_became_active":"2023-11-06T02:13:33.278288+0000","last_became_peered":"2023-11-06T02:13:33.278288+0000","last_unstale":"2023-11-06T02:13:33.670623+0000","last_undegraded":"2023-11-06T02:13:33.670623+0000","last_fullsized":"2023-11-06T02:13:33.670623+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:35:42.092353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.692650+0000","last_change":"2023-11-06T02:13:33.670098+0000","last_active":"2023-11-06T02:13:33.692650+0000","last_peered":"2023-11-06T02:13:33.692650+0000","last_clean":"2023-11-06T02:13:33.692650+0000","last_became_active":"2023-11-06T02:13:33.665807+0000","last_became_peered":"2023-11-06T02:13:33.665807+0000","last_unstale":"2023-11-06T02:13:33.692650+0000","last_undegraded":"2023-11-06T02:13:33.692650+0000","last_fullsized":"2023-11-06T02:13:33.692650+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:14:21.400766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"79'2","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.677753+0000","last_change":"2023-11-06T02:13:33.267790+0000","last_active":"2023-11-06T02:13:33.677753+0000","last_peered":"2023-11-06T02:13:33.677753+0000","last_clean":"2023-11-06T02:13:33.677753+0000","last_became_active":"2023-11-06T02:13:33.267049+0000","last_became_peered":"2023-11-06T02:13:33.267049+0000","last_unstale":"2023-11-06T02:13:33.677753+0000","last_undegraded":"2023-11-06T02:13:33.677753+0000","last_fullsized":"2023-11-06T02:13:33.677753+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:44:56.393504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.657846+0000","last_change":"2023-11-06T02:13:33.656103+0000","last_active":"2023-11-06T02:13:33.657846+0000","last_peered":"2023-11-06T02:13:33.657846+0000","last_clean":"2023-11-06T02:13:33.657846+0000","last_became_active":"2023-11-06T02:13:33.655726+0000","last_became_peered":"2023-11-06T02:13:33.655726+0000","last_unstale":"2023-11-06T02:13:33.657846+0000","last_undegraded":"2023-11-06T02:13:33.657846+0000","last_fullsized":"2023-11-06T02:13:33.657846+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:11:19.807848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"79'1","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.684788+0000","last_change":"2023-11-06T02:13:33.270300+0000","last_active":"2023-11-06T02:13:33.684788+0000","last_peered":"2023-11-06T02:13:33.684788+0000","last_clean":"2023-11-06T02:13:33.684788+0000","last_became_active":"2023-11-06T02:13:33.269591+0000","last_became_peered":"2023-11-06T02:13:33.269591+0000","last_unstale":"2023-11-06T02:13:33.684788+0000","last_undegraded":"2023-11-06T02:13:33.684788+0000","last_fullsized":"2023-11-06T02:13:33.684788+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:33:31.177122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.658660+0000","last_change":"2023-11-06T02:13:33.657357+0000","last_active":"2023-11-06T02:13:33.658660+0000","last_peered":"2023-11-06T02:13:33.658660+0000","last_clean":"2023-11-06T02:13:33.658660+0000","last_became_active":"2023-11-06T02:13:33.656718+0000","last_became_peered":"2023-11-06T02:13:33.656718+0000","last_unstale":"2023-11-06T02:13:33.658660+0000","last_undegraded":"2023-11-06T02:13:33.658660+0000","last_fullsized":"2023-11-06T02:13:33.658660+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:34:34.194876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"79'1","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.691088+0000","last_change":"2023-11-06T02:13:33.272642+0000","last_active":"2023-11-06T02:13:33.691088+0000","last_peered":"2023-11-06T02:13:33.691088+0000","last_clean":"2023-11-06T02:13:33.691088+0000","last_became_active":"2023-11-06T02:13:33.271888+0000","last_became_peered":"2023-11-06T02:13:33.271888+0000","last_unstale":"2023-11-06T02:13:33.691088+0000","last_undegraded":"2023-11-06T02:13:33.691088+0000","last_fullsized":"2023-11-06T02:13:33.691088+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:51:47.363705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.692440+0000","last_change":"2023-11-06T02:13:33.666664+0000","last_active":"2023-11-06T02:13:33.692440+0000","last_peered":"2023-11-06T02:13:33.692440+0000","last_clean":"2023-11-06T02:13:33.692440+0000","last_became_active":"2023-11-06T02:13:33.665936+0000","last_became_peered":"2023-11-06T02:13:33.665936+0000","last_unstale":"2023-11-06T02:13:33.692440+0000","last_undegraded":"2023-11-06T02:13:33.692440+0000","last_fullsized":"2023-11-06T02:13:33.692440+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:16:36.406664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.643907+0000","last_change":"2023-11-06T02:13:33.269015+0000","last_active":"2023-11-06T02:13:33.643907+0000","last_peered":"2023-11-06T02:13:33.643907+0000","last_clean":"2023-11-06T02:13:33.643907+0000","last_became_active":"2023-11-06T02:13:33.268546+0000","last_became_peered":"2023-11-06T02:13:33.268546+0000","last_unstale":"2023-11-06T02:13:33.643907+0000","last_undegraded":"2023-11-06T02:13:33.643907+0000","last_fullsized":"2023-11-06T02:13:33.643907+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:54:32.434908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.676989+0000","last_change":"2023-11-06T02:13:33.670964+0000","last_active":"2023-11-06T02:13:33.676989+0000","last_peered":"2023-11-06T02:13:33.676989+0000","last_clean":"2023-11-06T02:13:33.676989+0000","last_became_active":"2023-11-06T02:13:33.662388+0000","last_became_peered":"2023-11-06T02:13:33.662388+0000","last_unstale":"2023-11-06T02:13:33.676989+0000","last_undegraded":"2023-11-06T02:13:33.676989+0000","last_fullsized":"2023-11-06T02:13:33.676989+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:33:50.179908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.646067+0000","last_change":"2023-11-06T02:13:33.265532+0000","last_active":"2023-11-06T02:13:33.646067+0000","last_peered":"2023-11-06T02:13:33.646067+0000","last_clean":"2023-11-06T02:13:33.646067+0000","last_became_active":"2023-11-06T02:13:33.265209+0000","last_became_peered":"2023-11-06T02:13:33.265209+0000","last_unstale":"2023-11-06T02:13:33.646067+0000","last_undegraded":"2023-11-06T02:13:33.646067+0000","last_fullsized":"2023-11-06T02:13:33.646067+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:39:03.018907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.668722+0000","last_change":"2023-11-06T02:13:33.665214+0000","last_active":"2023-11-06T02:13:33.668722+0000","last_peered":"2023-11-06T02:13:33.668722+0000","last_clean":"2023-11-06T02:13:33.668722+0000","last_became_active":"2023-11-06T02:13:33.664989+0000","last_became_peered":"2023-11-06T02:13:33.664989+0000","last_unstale":"2023-11-06T02:13:33.668722+0000","last_undegraded":"2023-11-06T02:13:33.668722+0000","last_fullsized":"2023-11-06T02:13:33.668722+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:32:02.459972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.650994+0000","last_change":"2023-11-06T02:13:33.282852+0000","last_active":"2023-11-06T02:13:33.650994+0000","last_peered":"2023-11-06T02:13:33.650994+0000","last_clean":"2023-11-06T02:13:33.650994+0000","last_became_active":"2023-11-06T02:13:33.281487+0000","last_became_peered":"2023-11-06T02:13:33.281487+0000","last_unstale":"2023-11-06T02:13:33.650994+0000","last_undegraded":"2023-11-06T02:13:33.650994+0000","last_fullsized":"2023-11-06T02:13:33.650994+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:53:46.349156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.693745+0000","last_change":"2023-11-06T02:13:33.689127+0000","last_active":"2023-11-06T02:13:33.693745+0000","last_peered":"2023-11-06T02:13:33.693745+0000","last_clean":"2023-11-06T02:13:33.693745+0000","last_became_active":"2023-11-06T02:13:33.688518+0000","last_became_peered":"2023-11-06T02:13:33.688518+0000","last_unstale":"2023-11-06T02:13:33.693745+0000","last_undegraded":"2023-11-06T02:13:33.693745+0000","last_fullsized":"2023-11-06T02:13:33.693745+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:00:13.166796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.660143+0000","last_change":"2023-11-06T02:13:33.281331+0000","last_active":"2023-11-06T02:13:33.660143+0000","last_peered":"2023-11-06T02:13:33.660143+0000","last_clean":"2023-11-06T02:13:33.660143+0000","last_became_active":"2023-11-06T02:13:33.280630+0000","last_became_peered":"2023-11-06T02:13:33.280630+0000","last_unstale":"2023-11-06T02:13:33.660143+0000","last_undegraded":"2023-11-06T02:13:33.660143+0000","last_fullsized":"2023-11-06T02:13:33.660143+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:17:16.208032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.661079+0000","last_change":"2023-11-06T02:13:33.649612+0000","last_active":"2023-11-06T02:13:33.661079+0000","last_peered":"2023-11-06T02:13:33.661079+0000","last_clean":"2023-11-06T02:13:33.661079+0000","last_became_active":"2023-11-06T02:13:33.649236+0000","last_became_peered":"2023-11-06T02:13:33.649236+0000","last_unstale":"2023-11-06T02:13:33.661079+0000","last_undegraded":"2023-11-06T02:13:33.661079+0000","last_fullsized":"2023-11-06T02:13:33.661079+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:28:22.791353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"79'3","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.660690+0000","last_change":"2023-11-06T02:13:33.265047+0000","last_active":"2023-11-06T02:13:33.660690+0000","last_peered":"2023-11-06T02:13:33.660690+0000","last_clean":"2023-11-06T02:13:33.660690+0000","last_became_active":"2023-11-06T02:13:33.264385+0000","last_became_peered":"2023-11-06T02:13:33.264385+0000","last_unstale":"2023-11-06T02:13:33.660690+0000","last_undegraded":"2023-11-06T02:13:33.660690+0000","last_fullsized":"2023-11-06T02:13:33.660690+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:55:36.788472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.659369+0000","last_change":"2023-11-06T02:13:33.654270+0000","last_active":"2023-11-06T02:13:33.659369+0000","last_peered":"2023-11-06T02:13:33.659369+0000","last_clean":"2023-11-06T02:13:33.659369+0000","last_became_active":"2023-11-06T02:13:33.653974+0000","last_became_peered":"2023-11-06T02:13:33.653974+0000","last_unstale":"2023-11-06T02:13:33.659369+0000","last_undegraded":"2023-11-06T02:13:33.659369+0000","last_fullsized":"2023-11-06T02:13:33.659369+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:33:50.179908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"79'6","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.698016+0000","last_change":"2023-11-06T02:13:33.265304+0000","last_active":"2023-11-06T02:13:33.698016+0000","last_peered":"2023-11-06T02:13:33.698016+0000","last_clean":"2023-11-06T02:13:33.698016+0000","last_became_active":"2023-11-06T02:13:33.264524+0000","last_became_peered":"2023-11-06T02:13:33.264524+0000","last_unstale":"2023-11-06T02:13:33.698016+0000","last_undegraded":"2023-11-06T02:13:33.698016+0000","last_fullsized":"2023-11-06T02:13:33.698016+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:39:01.059129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.693479+0000","last_change":"2023-11-06T02:13:33.681794+0000","last_active":"2023-11-06T02:13:33.693479+0000","last_peered":"2023-11-06T02:13:33.693479+0000","last_clean":"2023-11-06T02:13:33.693479+0000","last_became_active":"2023-11-06T02:13:33.680433+0000","last_became_peered":"2023-11-06T02:13:33.680433+0000","last_unstale":"2023-11-06T02:13:33.693479+0000","last_undegraded":"2023-11-06T02:13:33.693479+0000","last_fullsized":"2023-11-06T02:13:33.693479+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:17:59.025543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"79'2","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.693079+0000","last_change":"2023-11-06T02:13:33.273211+0000","last_active":"2023-11-06T02:13:33.693079+0000","last_peered":"2023-11-06T02:13:33.693079+0000","last_clean":"2023-11-06T02:13:33.693079+0000","last_became_active":"2023-11-06T02:13:33.272067+0000","last_became_peered":"2023-11-06T02:13:33.272067+0000","last_unstale":"2023-11-06T02:13:33.693079+0000","last_undegraded":"2023-11-06T02:13:33.693079+0000","last_fullsized":"2023-11-06T02:13:33.693079+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:16:35.397189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.672340+0000","last_change":"2023-11-06T02:13:33.653278+0000","last_active":"2023-11-06T02:13:33.672340+0000","last_peered":"2023-11-06T02:13:33.672340+0000","last_clean":"2023-11-06T02:13:33.672340+0000","last_became_active":"2023-11-06T02:13:33.652577+0000","last_became_peered":"2023-11-06T02:13:33.652577+0000","last_unstale":"2023-11-06T02:13:33.672340+0000","last_undegraded":"2023-11-06T02:13:33.672340+0000","last_fullsized":"2023-11-06T02:13:33.672340+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:17:36.098143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.649363+0000","last_change":"2023-11-06T02:13:33.279003+0000","last_active":"2023-11-06T02:13:33.649363+0000","last_peered":"2023-11-06T02:13:33.649363+0000","last_clean":"2023-11-06T02:13:33.649363+0000","last_became_active":"2023-11-06T02:13:33.278190+0000","last_became_peered":"2023-11-06T02:13:33.278190+0000","last_unstale":"2023-11-06T02:13:33.649363+0000","last_undegraded":"2023-11-06T02:13:33.649363+0000","last_fullsized":"2023-11-06T02:13:33.649363+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:53:59.721435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.690241+0000","last_change":"2023-11-06T02:13:33.686928+0000","last_active":"2023-11-06T02:13:33.690241+0000","last_peered":"2023-11-06T02:13:33.690241+0000","last_clean":"2023-11-06T02:13:33.690241+0000","last_became_active":"2023-11-06T02:13:33.686477+0000","last_became_peered":"2023-11-06T02:13:33.686477+0000","last_unstale":"2023-11-06T02:13:33.690241+0000","last_undegraded":"2023-11-06T02:13:33.690241+0000","last_fullsized":"2023-11-06T02:13:33.690241+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:10:52.936432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.645828+0000","last_change":"2023-11-06T02:13:33.265543+0000","last_active":"2023-11-06T02:13:33.645828+0000","last_peered":"2023-11-06T02:13:33.645828+0000","last_clean":"2023-11-06T02:13:33.645828+0000","last_became_active":"2023-11-06T02:13:33.265118+0000","last_became_peered":"2023-11-06T02:13:33.265118+0000","last_unstale":"2023-11-06T02:13:33.645828+0000","last_undegraded":"2023-11-06T02:13:33.645828+0000","last_fullsized":"2023-11-06T02:13:33.645828+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:30:40.192944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.677030+0000","last_change":"2023-11-06T02:13:33.671009+0000","last_active":"2023-11-06T02:13:33.677030+0000","last_peered":"2023-11-06T02:13:33.677030+0000","last_clean":"2023-11-06T02:13:33.677030+0000","last_became_active":"2023-11-06T02:13:33.663369+0000","last_became_peered":"2023-11-06T02:13:33.663369+0000","last_unstale":"2023-11-06T02:13:33.677030+0000","last_undegraded":"2023-11-06T02:13:33.677030+0000","last_fullsized":"2023-11-06T02:13:33.677030+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:21:05.055643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"79'8","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.682532+0000","last_change":"2023-11-06T02:13:33.278600+0000","last_active":"2023-11-06T02:13:33.682532+0000","last_peered":"2023-11-06T02:13:33.682532+0000","last_clean":"2023-11-06T02:13:33.682532+0000","last_became_active":"2023-11-06T02:13:33.278136+0000","last_became_peered":"2023-11-06T02:13:33.278136+0000","last_unstale":"2023-11-06T02:13:33.682532+0000","last_undegraded":"2023-11-06T02:13:33.682532+0000","last_fullsized":"2023-11-06T02:13:33.682532+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:17:58.016068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.696250+0000","last_change":"2023-11-06T02:13:33.685866+0000","last_active":"2023-11-06T02:13:33.696250+0000","last_peered":"2023-11-06T02:13:33.696250+0000","last_clean":"2023-11-06T02:13:33.696250+0000","last_became_active":"2023-11-06T02:13:33.685233+0000","last_became_peered":"2023-11-06T02:13:33.685233+0000","last_unstale":"2023-11-06T02:13:33.696250+0000","last_undegraded":"2023-11-06T02:13:33.696250+0000","last_fullsized":"2023-11-06T02:13:33.696250+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:54:00.730910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"79'2","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.691023+0000","last_change":"2023-11-06T02:13:33.272867+0000","last_active":"2023-11-06T02:13:33.691023+0000","last_peered":"2023-11-06T02:13:33.691023+0000","last_clean":"2023-11-06T02:13:33.691023+0000","last_became_active":"2023-11-06T02:13:33.272136+0000","last_became_peered":"2023-11-06T02:13:33.272136+0000","last_unstale":"2023-11-06T02:13:33.691023+0000","last_undegraded":"2023-11-06T02:13:33.691023+0000","last_fullsized":"2023-11-06T02:13:33.691023+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:02:50.688184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.659177+0000","last_change":"2023-11-06T02:13:33.646192+0000","last_active":"2023-11-06T02:13:33.659177+0000","last_peered":"2023-11-06T02:13:33.659177+0000","last_clean":"2023-11-06T02:13:33.659177+0000","last_became_active":"2023-11-06T02:13:33.643963+0000","last_became_peered":"2023-11-06T02:13:33.643963+0000","last_unstale":"2023-11-06T02:13:33.659177+0000","last_undegraded":"2023-11-06T02:13:33.659177+0000","last_fullsized":"2023-11-06T02:13:33.659177+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:02:47.316294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.648838+0000","last_change":"2023-11-06T02:13:33.267168+0000","last_active":"2023-11-06T02:13:33.648838+0000","last_peered":"2023-11-06T02:13:33.648838+0000","last_clean":"2023-11-06T02:13:33.648838+0000","last_became_active":"2023-11-06T02:13:33.266509+0000","last_became_peered":"2023-11-06T02:13:33.266509+0000","last_unstale":"2023-11-06T02:13:33.648838+0000","last_undegraded":"2023-11-06T02:13:33.648838+0000","last_fullsized":"2023-11-06T02:13:33.648838+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:11:59.707658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.692478+0000","last_change":"2023-11-06T02:13:33.668105+0000","last_active":"2023-11-06T02:13:33.692478+0000","last_peered":"2023-11-06T02:13:33.692478+0000","last_clean":"2023-11-06T02:13:33.692478+0000","last_became_active":"2023-11-06T02:13:33.667389+0000","last_became_peered":"2023-11-06T02:13:33.667389+0000","last_unstale":"2023-11-06T02:13:33.692478+0000","last_undegraded":"2023-11-06T02:13:33.692478+0000","last_fullsized":"2023-11-06T02:13:33.692478+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:07:57.793830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.651289+0000","last_change":"2023-11-06T02:13:33.279035+0000","last_active":"2023-11-06T02:13:33.651289+0000","last_peered":"2023-11-06T02:13:33.651289+0000","last_clean":"2023-11-06T02:13:33.651289+0000","last_became_active":"2023-11-06T02:13:33.278260+0000","last_became_peered":"2023-11-06T02:13:33.278260+0000","last_unstale":"2023-11-06T02:13:33.651289+0000","last_undegraded":"2023-11-06T02:13:33.651289+0000","last_fullsized":"2023-11-06T02:13:33.651289+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:16:35.397189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.668712+0000","last_change":"2023-11-06T02:13:33.664890+0000","last_active":"2023-11-06T02:13:33.668712+0000","last_peered":"2023-11-06T02:13:33.668712+0000","last_clean":"2023-11-06T02:13:33.668712+0000","last_became_active":"2023-11-06T02:13:33.664608+0000","last_became_peered":"2023-11-06T02:13:33.664608+0000","last_unstale":"2023-11-06T02:13:33.668712+0000","last_undegraded":"2023-11-06T02:13:33.668712+0000","last_fullsized":"2023-11-06T02:13:33.668712+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:22:47.360905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.650313+0000","last_change":"2023-11-06T02:13:33.281882+0000","last_active":"2023-11-06T02:13:33.650313+0000","last_peered":"2023-11-06T02:13:33.650313+0000","last_clean":"2023-11-06T02:13:33.650313+0000","last_became_active":"2023-11-06T02:13:33.281241+0000","last_became_peered":"2023-11-06T02:13:33.281241+0000","last_unstale":"2023-11-06T02:13:33.650313+0000","last_undegraded":"2023-11-06T02:13:33.650313+0000","last_fullsized":"2023-11-06T02:13:33.650313+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:12:35.766676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.674502+0000","last_change":"2023-11-06T02:13:33.644604+0000","last_active":"2023-11-06T02:13:33.674502+0000","last_peered":"2023-11-06T02:13:33.674502+0000","last_clean":"2023-11-06T02:13:33.674502+0000","last_became_active":"2023-11-06T02:13:33.643768+0000","last_became_peered":"2023-11-06T02:13:33.643768+0000","last_unstale":"2023-11-06T02:13:33.674502+0000","last_undegraded":"2023-11-06T02:13:33.674502+0000","last_fullsized":"2023-11-06T02:13:33.674502+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:42:42.388106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.636063+0000","last_change":"2023-11-06T02:13:33.267817+0000","last_active":"2023-11-06T02:13:33.636063+0000","last_peered":"2023-11-06T02:13:33.636063+0000","last_clean":"2023-11-06T02:13:33.636063+0000","last_became_active":"2023-11-06T02:13:33.267091+0000","last_became_peered":"2023-11-06T02:13:33.267091+0000","last_unstale":"2023-11-06T02:13:33.636063+0000","last_undegraded":"2023-11-06T02:13:33.636063+0000","last_fullsized":"2023-11-06T02:13:33.636063+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:36:21.262835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.676544+0000","last_change":"2023-11-06T02:13:33.663511+0000","last_active":"2023-11-06T02:13:33.676544+0000","last_peered":"2023-11-06T02:13:33.676544+0000","last_clean":"2023-11-06T02:13:33.676544+0000","last_became_active":"2023-11-06T02:13:33.662822+0000","last_became_peered":"2023-11-06T02:13:33.662822+0000","last_unstale":"2023-11-06T02:13:33.676544+0000","last_undegraded":"2023-11-06T02:13:33.676544+0000","last_fullsized":"2023-11-06T02:13:33.676544+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:31:19.083004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.646094+0000","last_change":"2023-11-06T02:13:33.263669+0000","last_active":"2023-11-06T02:13:33.646094+0000","last_peered":"2023-11-06T02:13:33.646094+0000","last_clean":"2023-11-06T02:13:33.646094+0000","last_became_active":"2023-11-06T02:13:33.263355+0000","last_became_peered":"2023-11-06T02:13:33.263355+0000","last_unstale":"2023-11-06T02:13:33.646094+0000","last_undegraded":"2023-11-06T02:13:33.646094+0000","last_fullsized":"2023-11-06T02:13:33.646094+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:19:20.386051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.662904+0000","last_change":"2023-11-06T02:13:33.659767+0000","last_active":"2023-11-06T02:13:33.662904+0000","last_peered":"2023-11-06T02:13:33.662904+0000","last_clean":"2023-11-06T02:13:33.662904+0000","last_became_active":"2023-11-06T02:13:33.659458+0000","last_became_peered":"2023-11-06T02:13:33.659458+0000","last_unstale":"2023-11-06T02:13:33.662904+0000","last_undegraded":"2023-11-06T02:13:33.662904+0000","last_fullsized":"2023-11-06T02:13:33.662904+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:02:47.316294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.638171+0000","last_change":"2023-11-06T02:13:33.264127+0000","last_active":"2023-11-06T02:13:33.638171+0000","last_peered":"2023-11-06T02:13:33.638171+0000","last_clean":"2023-11-06T02:13:33.638171+0000","last_became_active":"2023-11-06T02:13:33.263418+0000","last_became_peered":"2023-11-06T02:13:33.263418+0000","last_unstale":"2023-11-06T02:13:33.638171+0000","last_undegraded":"2023-11-06T02:13:33.638171+0000","last_fullsized":"2023-11-06T02:13:33.638171+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:04:53.601947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.667542+0000","last_change":"2023-11-06T02:13:33.666567+0000","last_active":"2023-11-06T02:13:33.667542+0000","last_peered":"2023-11-06T02:13:33.667542+0000","last_clean":"2023-11-06T02:13:33.667542+0000","last_became_active":"2023-11-06T02:13:33.666291+0000","last_became_peered":"2023-11-06T02:13:33.666291+0000","last_unstale":"2023-11-06T02:13:33.667542+0000","last_undegraded":"2023-11-06T02:13:33.667542+0000","last_fullsized":"2023-11-06T02:13:33.667542+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:12:36.776151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.632865+0000","last_change":"2023-11-06T02:13:33.267967+0000","last_active":"2023-11-06T02:13:33.632865+0000","last_peered":"2023-11-06T02:13:33.632865+0000","last_clean":"2023-11-06T02:13:33.632865+0000","last_became_active":"2023-11-06T02:13:33.267674+0000","last_became_peered":"2023-11-06T02:13:33.267674+0000","last_unstale":"2023-11-06T02:13:33.632865+0000","last_undegraded":"2023-11-06T02:13:33.632865+0000","last_fullsized":"2023-11-06T02:13:33.632865+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:13:44.482660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.678951+0000","last_change":"2023-11-06T02:13:33.670094+0000","last_active":"2023-11-06T02:13:33.678951+0000","last_peered":"2023-11-06T02:13:33.678951+0000","last_clean":"2023-11-06T02:13:33.678951+0000","last_became_active":"2023-11-06T02:13:33.669424+0000","last_became_peered":"2023-11-06T02:13:33.669424+0000","last_unstale":"2023-11-06T02:13:33.678951+0000","last_undegraded":"2023-11-06T02:13:33.678951+0000","last_fullsized":"2023-11-06T02:13:33.678951+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:12:00.717133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.641189+0000","last_change":"2023-11-06T02:13:33.278729+0000","last_active":"2023-11-06T02:13:33.641189+0000","last_peered":"2023-11-06T02:13:33.641189+0000","last_clean":"2023-11-06T02:13:33.641189+0000","last_became_active":"2023-11-06T02:13:33.278240+0000","last_became_peered":"2023-11-06T02:13:33.278240+0000","last_unstale":"2023-11-06T02:13:33.641189+0000","last_undegraded":"2023-11-06T02:13:33.641189+0000","last_fullsized":"2023-11-06T02:13:33.641189+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:16:51.749335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":218,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.650498+0000","last_change":"2023-11-06T02:08:20.523427+0000","last_active":"2023-11-06T02:13:33.650498+0000","last_peered":"2023-11-06T02:13:33.650498+0000","last_clean":"2023-11-06T02:13:33.650498+0000","last_became_active":"2023-11-06T02:08:20.521089+0000","last_became_peered":"2023-11-06T02:08:20.521089+0000","last_unstale":"2023-11-06T02:13:33.650498+0000","last_undegraded":"2023-11-06T02:13:33.650498+0000","last_fullsized":"2023-11-06T02:13:33.650498+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:19.252348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:19.252348+0000","last_clean_scrub_stamp":"2023-11-06T02:08:19.252348+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:30:15.566397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":64,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.652143+0000","last_change":"2023-11-06T02:08:26.059502+0000","last_active":"2023-11-06T02:13:33.652143+0000","last_peered":"2023-11-06T02:13:33.652143+0000","last_clean":"2023-11-06T02:13:33.652143+0000","last_became_active":"2023-11-06T02:08:26.058612+0000","last_became_peered":"2023-11-06T02:08:26.058612+0000","last_unstale":"2023-11-06T02:13:33.652143+0000","last_undegraded":"2023-11-06T02:13:33.652143+0000","last_fullsized":"2023-11-06T02:13:33.652143+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:10.626021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.688617+0000","last_change":"2023-11-06T02:13:33.686968+0000","last_active":"2023-11-06T02:13:33.688617+0000","last_peered":"2023-11-06T02:13:33.688617+0000","last_clean":"2023-11-06T02:13:33.688617+0000","last_became_active":"2023-11-06T02:13:33.686722+0000","last_became_peered":"2023-11-06T02:13:33.686722+0000","last_unstale":"2023-11-06T02:13:33.688617+0000","last_undegraded":"2023-11-06T02:13:33.688617+0000","last_fullsized":"2023-11-06T02:13:33.688617+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:34:40.956993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.641653+0000","last_change":"2023-11-06T02:13:33.278720+0000","last_active":"2023-11-06T02:13:33.641653+0000","last_peered":"2023-11-06T02:13:33.641653+0000","last_clean":"2023-11-06T02:13:33.641653+0000","last_became_active":"2023-11-06T02:13:33.278233+0000","last_became_peered":"2023-11-06T02:13:33.278233+0000","last_unstale":"2023-11-06T02:13:33.641653+0000","last_undegraded":"2023-11-06T02:13:33.641653+0000","last_fullsized":"2023-11-06T02:13:33.641653+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:11:51.109403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":65,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.650253+0000","last_change":"2023-11-06T02:08:26.060406+0000","last_active":"2023-11-06T02:13:33.650253+0000","last_peered":"2023-11-06T02:13:33.650253+0000","last_clean":"2023-11-06T02:13:33.650253+0000","last_became_active":"2023-11-06T02:08:26.059262+0000","last_became_peered":"2023-11-06T02:08:26.059262+0000","last_unstale":"2023-11-06T02:13:33.650253+0000","last_undegraded":"2023-11-06T02:13:33.650253+0000","last_fullsized":"2023-11-06T02:13:33.650253+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:10.626021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.686767+0000","last_change":"2023-11-06T02:13:33.683644+0000","last_active":"2023-11-06T02:13:33.686767+0000","last_peered":"2023-11-06T02:13:33.686767+0000","last_clean":"2023-11-06T02:13:33.686767+0000","last_became_active":"2023-11-06T02:13:33.683059+0000","last_became_peered":"2023-11-06T02:13:33.683059+0000","last_unstale":"2023-11-06T02:13:33.686767+0000","last_undegraded":"2023-11-06T02:13:33.686767+0000","last_fullsized":"2023-11-06T02:13:33.686767+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:33:32.186597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.642538+0000","last_change":"2023-11-06T02:13:33.278293+0000","last_active":"2023-11-06T02:13:33.642538+0000","last_peered":"2023-11-06T02:13:33.642538+0000","last_clean":"2023-11-06T02:13:33.642538+0000","last_became_active":"2023-11-06T02:13:33.277952+0000","last_became_peered":"2023-11-06T02:13:33.277952+0000","last_unstale":"2023-11-06T02:13:33.642538+0000","last_undegraded":"2023-11-06T02:13:33.642538+0000","last_fullsized":"2023-11-06T02:13:33.642538+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:51:52.147063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":65,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.645920+0000","last_change":"2023-11-06T02:08:26.559477+0000","last_active":"2023-11-06T02:13:33.645920+0000","last_peered":"2023-11-06T02:13:33.645920+0000","last_clean":"2023-11-06T02:13:33.645920+0000","last_became_active":"2023-11-06T02:08:26.559202+0000","last_became_peered":"2023-11-06T02:08:26.559202+0000","last_unstale":"2023-11-06T02:13:33.645920+0000","last_undegraded":"2023-11-06T02:13:33.645920+0000","last_fullsized":"2023-11-06T02:13:33.645920+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:55:04.552404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.693070+0000","last_change":"2023-11-06T02:13:33.692457+0000","last_active":"2023-11-06T02:13:33.693070+0000","last_peered":"2023-11-06T02:13:33.693070+0000","last_clean":"2023-11-06T02:13:33.693070+0000","last_became_active":"2023-11-06T02:13:33.692061+0000","last_became_peered":"2023-11-06T02:13:33.692061+0000","last_unstale":"2023-11-06T02:13:33.693070+0000","last_undegraded":"2023-11-06T02:13:33.693070+0000","last_fullsized":"2023-11-06T02:13:33.693070+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:54:11.389133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"79'2","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.664608+0000","last_change":"2023-11-06T02:13:33.266996+0000","last_active":"2023-11-06T02:13:33.664608+0000","last_peered":"2023-11-06T02:13:33.664608+0000","last_clean":"2023-11-06T02:13:33.664608+0000","last_became_active":"2023-11-06T02:13:33.265873+0000","last_became_peered":"2023-11-06T02:13:33.265873+0000","last_unstale":"2023-11-06T02:13:33.664608+0000","last_undegraded":"2023-11-06T02:13:33.664608+0000","last_fullsized":"2023-11-06T02:13:33.664608+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:38:14.971699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":64,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.651758+0000","last_change":"2023-11-06T02:08:26.059546+0000","last_active":"2023-11-06T02:13:33.651758+0000","last_peered":"2023-11-06T02:13:33.651758+0000","last_clean":"2023-11-06T02:13:33.651758+0000","last_became_active":"2023-11-06T02:08:26.058806+0000","last_became_peered":"2023-11-06T02:08:26.058806+0000","last_unstale":"2023-11-06T02:13:33.651758+0000","last_undegraded":"2023-11-06T02:13:33.651758+0000","last_fullsized":"2023-11-06T02:13:33.651758+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:23.468510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.658554+0000","last_change":"2023-11-06T02:13:33.653074+0000","last_active":"2023-11-06T02:13:33.658554+0000","last_peered":"2023-11-06T02:13:33.658554+0000","last_clean":"2023-11-06T02:13:33.658554+0000","last_became_active":"2023-11-06T02:13:33.652247+0000","last_became_peered":"2023-11-06T02:13:33.652247+0000","last_unstale":"2023-11-06T02:13:33.658554+0000","last_undegraded":"2023-11-06T02:13:33.658554+0000","last_fullsized":"2023-11-06T02:13:33.658554+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:36:12.474443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"79'1","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.685492+0000","last_change":"2023-11-06T02:13:33.265582+0000","last_active":"2023-11-06T02:13:33.685492+0000","last_peered":"2023-11-06T02:13:33.685492+0000","last_clean":"2023-11-06T02:13:33.685492+0000","last_became_active":"2023-11-06T02:13:33.264851+0000","last_became_peered":"2023-11-06T02:13:33.264851+0000","last_unstale":"2023-11-06T02:13:33.685492+0000","last_undegraded":"2023-11-06T02:13:33.685492+0000","last_fullsized":"2023-11-06T02:13:33.685492+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:30:05.141853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":64,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.633068+0000","last_change":"2023-11-06T02:08:26.436463+0000","last_active":"2023-11-06T02:13:33.633068+0000","last_peered":"2023-11-06T02:13:33.633068+0000","last_clean":"2023-11-06T02:13:33.633068+0000","last_became_active":"2023-11-06T02:08:26.436075+0000","last_became_peered":"2023-11-06T02:08:26.436075+0000","last_unstale":"2023-11-06T02:13:33.633068+0000","last_undegraded":"2023-11-06T02:13:33.633068+0000","last_fullsized":"2023-11-06T02:13:33.633068+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:23.468510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.664486+0000","last_change":"2023-11-06T02:13:33.656141+0000","last_active":"2023-11-06T02:13:33.664486+0000","last_peered":"2023-11-06T02:13:33.664486+0000","last_clean":"2023-11-06T02:13:33.664486+0000","last_became_active":"2023-11-06T02:13:33.655681+0000","last_became_peered":"2023-11-06T02:13:33.655681+0000","last_unstale":"2023-11-06T02:13:33.664486+0000","last_undegraded":"2023-11-06T02:13:33.664486+0000","last_fullsized":"2023-11-06T02:13:33.664486+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:25:19.608926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.632982+0000","last_change":"2023-11-06T02:13:33.272972+0000","last_active":"2023-11-06T02:13:33.632982+0000","last_peered":"2023-11-06T02:13:33.632982+0000","last_clean":"2023-11-06T02:13:33.632982+0000","last_became_active":"2023-11-06T02:13:33.272567+0000","last_became_peered":"2023-11-06T02:13:33.272567+0000","last_unstale":"2023-11-06T02:13:33.632982+0000","last_undegraded":"2023-11-06T02:13:33.632982+0000","last_fullsized":"2023-11-06T02:13:33.632982+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:23:42.695735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":65,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.651532+0000","last_change":"2023-11-06T02:08:26.436607+0000","last_active":"2023-11-06T02:13:33.651532+0000","last_peered":"2023-11-06T02:13:33.651532+0000","last_clean":"2023-11-06T02:13:33.651532+0000","last_became_active":"2023-11-06T02:08:26.435924+0000","last_became_peered":"2023-11-06T02:08:26.435924+0000","last_unstale":"2023-11-06T02:13:33.651532+0000","last_undegraded":"2023-11-06T02:13:33.651532+0000","last_fullsized":"2023-11-06T02:13:33.651532+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:23.468510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.661118+0000","last_change":"2023-11-06T02:13:33.651065+0000","last_active":"2023-11-06T02:13:33.661118+0000","last_peered":"2023-11-06T02:13:33.661118+0000","last_clean":"2023-11-06T02:13:33.661118+0000","last_became_active":"2023-11-06T02:13:33.650694+0000","last_became_peered":"2023-11-06T02:13:33.650694+0000","last_unstale":"2023-11-06T02:13:33.661118+0000","last_undegraded":"2023-11-06T02:13:33.661118+0000","last_fullsized":"2023-11-06T02:13:33.661118+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:22:01.334447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"79'1","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.677238+0000","last_change":"2023-11-06T02:13:33.282240+0000","last_active":"2023-11-06T02:13:33.677238+0000","last_peered":"2023-11-06T02:13:33.677238+0000","last_clean":"2023-11-06T02:13:33.677238+0000","last_became_active":"2023-11-06T02:13:33.280471+0000","last_became_peered":"2023-11-06T02:13:33.280471+0000","last_unstale":"2023-11-06T02:13:33.677238+0000","last_undegraded":"2023-11-06T02:13:33.677238+0000","last_fullsized":"2023-11-06T02:13:33.677238+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:51:47.363705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":64,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.633683+0000","last_change":"2023-11-06T02:08:26.057076+0000","last_active":"2023-11-06T02:13:33.633683+0000","last_peered":"2023-11-06T02:13:33.633683+0000","last_clean":"2023-11-06T02:13:33.633683+0000","last_became_active":"2023-11-06T02:08:26.056419+0000","last_became_peered":"2023-11-06T02:08:26.056419+0000","last_unstale":"2023-11-06T02:13:33.633683+0000","last_undegraded":"2023-11-06T02:13:33.633683+0000","last_fullsized":"2023-11-06T02:13:33.633683+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:10.626021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.665911+0000","last_change":"2023-11-06T02:13:33.664075+0000","last_active":"2023-11-06T02:13:33.665911+0000","last_peered":"2023-11-06T02:13:33.665911+0000","last_clean":"2023-11-06T02:13:33.665911+0000","last_became_active":"2023-11-06T02:13:33.663640+0000","last_became_peered":"2023-11-06T02:13:33.663640+0000","last_unstale":"2023-11-06T02:13:33.665911+0000","last_undegraded":"2023-11-06T02:13:33.665911+0000","last_fullsized":"2023-11-06T02:13:33.665911+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:33.224257+0000","last_clean_scrub_stamp":"2023-11-06T02:13:33.224257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:38:31.470172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"79'2","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.684931+0000","last_change":"2023-11-06T02:13:33.265697+0000","last_active":"2023-11-06T02:13:33.684931+0000","last_peered":"2023-11-06T02:13:33.684931+0000","last_clean":"2023-11-06T02:13:33.684931+0000","last_became_active":"2023-11-06T02:13:33.264914+0000","last_became_peered":"2023-11-06T02:13:33.264914+0000","last_unstale":"2023-11-06T02:13:33.684931+0000","last_undegraded":"2023-11-06T02:13:33.684931+0000","last_fullsized":"2023-11-06T02:13:33.684931+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:13:32.214782+0000","last_clean_scrub_stamp":"2023-11-06T02:13:32.214782+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:17:35.088668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":73,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-06T02:13:33.639596+0000","last_change":"2023-11-06T02:08:26.058596+0000","last_active":"2023-11-06T02:13:33.639596+0000","last_peered":"2023-11-06T02:13:33.639596+0000","last_clean":"2023-11-06T02:13:33.639596+0000","last_became_active":"2023-11-06T02:08:26.057015+0000","last_became_peered":"2023-11-06T02:08:26.057015+0000","last_unstale":"2023-11-06T02:13:33.639596+0000","last_undegraded":"2023-11-06T02:13:33.639596+0000","last_fullsized":"2023-11-06T02:13:33.639596+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:55:04.552404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574915,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32584,"kb_used_data":1536,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339256,"statfs":{"total":96636764160,"available":96603398144,"internally_reserved":0,"allocated":1572864,"data_stored":1029780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:13:26 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.86599999999999999,"15min":0.85399999999999998},"min":{"1min":0.59199999999999997,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.2190000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.91200000000000003,"15min":0.91100000000000003},"min":{"1min":0.57299999999999995,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.214,"5min":1.506,"15min":1.506},"last":1.1299999999999999}]},{"osd":1,"last update":"Mon Nov 6 02:13:26 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.83599999999999997,"15min":0.84599999999999997},"min":{"1min":0.66400000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.94099999999999995,"15min":0.90600000000000003},"min":{"1min":0.64300000000000002,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.681,"5min":1.681,"15min":1.681},"last":1.056}]},{"osd":2,"last update":"Mon Nov 6 02:13:26 2023","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.96599999999999997,"15min":0.95799999999999996},"min":{"1min":0.66200000000000003,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.22,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":1.1659999999999999},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.94899999999999995,"15min":0.98299999999999998},"min":{"1min":0.71899999999999997,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.2789999999999999,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":1.0309999999999999}]},{"osd":3,"last update":"Mon Nov 6 02:13:26 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.89400000000000002,"15min":0.88100000000000001},"min":{"1min":0.67500000000000004,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.196,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.96399999999999997},{"interface":"front","average":{"1min":1.069,"5min":1.046,"15min":1.0660000000000001},"min":{"1min":0.86399999999999999,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.2430000000000001,"5min":1.526,"15min":1.526},"last":0.98799999999999999}]},{"osd":4,"last update":"Mon Nov 6 02:13:26 2023","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.92300000000000004,"15min":0.94599999999999995},"min":{"1min":0.59699999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.2350000000000001,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":1.1930000000000001},{"interface":"front","average":{"1min":1.0129999999999999,"5min":1.004,"15min":1.0309999999999999},"min":{"1min":0.625,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.377,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.89300000000000002}]},{"osd":5,"last update":"Mon Nov 6 02:13:26 2023","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.038,"15min":1.0649999999999999},"min":{"1min":0.76300000000000001,"5min":0.503,"15min":0.503},"max":{"1min":1.409,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":0.92000000000000004},{"interface":"front","average":{"1min":1.046,"5min":1.0629999999999999,"15min":1.111},"min":{"1min":0.75900000000000001,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.3109999999999999,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":1.288}]},{"osd":6,"last update":"Mon Nov 6 02:13:26 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.1399999999999999,"15min":1.181},"min":{"1min":0.59399999999999997,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.278,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":1.24},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.091,"15min":1.135},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.4570000000000001,"5min":1.74,"15min":1.74},"last":1.0900000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574914,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32012,"kb_used_data":964,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339828,"statfs":{"total":96636764160,"available":96603983872,"internally_reserved":0,"allocated":987136,"data_stored":440954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:13:28 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.86699999999999999,"15min":0.86099999999999999},"min":{"1min":0.58899999999999997,"5min":0.495,"15min":0.495},"max":{"1min":1.202,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.98699999999999999},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.874,"15min":0.82799999999999996},"min":{"1min":0.63,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.2649999999999999,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":1.3280000000000001}]},{"osd":1,"last update":"Mon Nov 6 02:13:28 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.84999999999999998,"15min":0.86699999999999999},"min":{"1min":0.63500000000000001,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.244,"5min":1.244,"15min":1.244},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.89600000000000002,"15min":0.88100000000000001},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":1.5760000000000001}]},{"osd":2,"last update":"Mon Nov 6 02:13:28 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.88700000000000001,"15min":0.89500000000000002},"min":{"1min":0.52100000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.206,"5min":1.252,"15min":1.252},"last":1.266},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.90000000000000002,"15min":0.92000000000000004},"min":{"1min":0.61699999999999999,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.411,"5min":1.411,"15min":1.411},"last":0.83799999999999997}]},{"osd":3,"last update":"Mon Nov 6 02:13:28 2023","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.93899999999999995,"15min":0.96099999999999997},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.212,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":1.4299999999999999},{"interface":"front","average":{"1min":1.0549999999999999,"5min":0.99299999999999999,"15min":0.998},"min":{"1min":0.77300000000000002,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.381,"5min":1.381,"15min":1.381},"last":1.0800000000000001}]},{"osd":4,"last update":"Mon Nov 6 02:13:28 2023","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.92600000000000005,"15min":0.97099999999999997},"min":{"1min":0.67500000000000004,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.302,"5min":1.474,"15min":1.474},"last":1.2390000000000001},{"interface":"front","average":{"1min":1.0469999999999999,"5min":0.98199999999999998,"15min":1.02},"min":{"1min":0.77100000000000002,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.556,"5min":1.556,"15min":1.556},"last":1.4039999999999999}]},{"osd":5,"last update":"Mon Nov 6 02:13:28 2023","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":0.97199999999999998,"15min":1.0089999999999999},"min":{"1min":0.73099999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":1.1579999999999999},{"interface":"front","average":{"1min":1.129,"5min":1.0720000000000001,"15min":1.1000000000000001},"min":{"1min":0.94899999999999995,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.587,"5min":1.587,"15min":1.587},"last":1.3720000000000001}]},{"osd":7,"last update":"Mon Nov 6 02:13:28 2023","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.087,"15min":1.107},"min":{"1min":0.95899999999999996,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.615,"5min":1.615,"15min":1.615},"last":1.5369999999999999},{"interface":"front","average":{"1min":1.109,"5min":1.0629999999999999,"15min":1.131},"min":{"1min":0.83499999999999996,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":1.4610000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574915,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27908,"kb_used_data":956,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343932,"statfs":{"total":96636764160,"available":96608186368,"internally_reserved":0,"allocated":978944,"data_stored":439424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,4],"upper_bound":32},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:13:30 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.86899999999999999,"15min":0.84199999999999997},"min":{"1min":0.51700000000000002,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.1930000000000001,"5min":1.381,"15min":1.381},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.86599999999999999,"15min":0.84999999999999998},"min":{"1min":0.58199999999999996,"5min":0.503,"15min":0.503},"max":{"1min":1.1399999999999999,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":0.433}]},{"osd":1,"last update":"Mon Nov 6 02:13:30 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.89600000000000002,"15min":0.86499999999999999},"min":{"1min":0.69799999999999995,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.177,"5min":1.331,"15min":1.331},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.88100000000000001,"15min":0.83799999999999997},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.139,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":1.093}]},{"osd":2,"last update":"Mon Nov 6 02:13:30 2023","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.93400000000000005,"15min":0.94199999999999995},"min":{"1min":0.68000000000000005,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.298,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":1.242},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.88400000000000001,"15min":0.82099999999999995},"min":{"1min":0.56899999999999995,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.2869999999999999,"5min":1.377,"15min":1.377},"last":0.66100000000000003}]},{"osd":3,"last update":"Mon Nov 6 02:13:30 2023","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.95399999999999996,"15min":0.93100000000000005},"min":{"1min":0.77100000000000002,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.2210000000000001,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.997,"15min":1.0529999999999999},"min":{"1min":0.67900000000000005,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.224,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":1.1759999999999999}]},{"osd":4,"last update":"Mon Nov 6 02:13:30 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.89500000000000002,"15min":0.90900000000000003},"min":{"1min":0.58099999999999996,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.1779999999999999,"5min":1.276,"15min":1.276},"last":1.0129999999999999},{"interface":"front","average":{"1min":1.006,"5min":0.99299999999999999,"15min":1.016},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.254,"5min":1.554,"15min":1.554},"last":1.496}]},{"osd":6,"last update":"Mon Nov 6 02:13:30 2023","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":1.0089999999999999,"15min":1.048},"min":{"1min":0.71399999999999997,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.2569999999999999,"5min":1.454,"15min":1.454},"last":1.3440000000000001},{"interface":"front","average":{"1min":0.98699999999999999,"5min":1.022,"15min":1.034},"min":{"1min":0.75,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.1930000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":2.2309999999999999}]},{"osd":7,"last update":"Mon Nov 6 02:13:30 2023","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.0609999999999999,"15min":1.071},"min":{"1min":0.77900000000000003,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.2330000000000001,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":1.4059999999999999},{"interface":"front","average":{"1min":1.109,"5min":1.1120000000000001,"15min":1.1399999999999999},"min":{"1min":0.84599999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.3799999999999999,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":2.3759999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574915,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27924,"kb_used_data":972,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343916,"statfs":{"total":96636764160,"available":96608169984,"internally_reserved":0,"allocated":995328,"data_stored":441058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:13:27 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.877,"15min":0.84199999999999997},"min":{"1min":0.65100000000000002,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.0840000000000001,"5min":1.343,"15min":1.343},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.83299999999999996,"15min":0.79100000000000004},"min":{"1min":0.626,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.1699999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.622}]},{"osd":1,"last update":"Mon Nov 6 02:13:27 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.86799999999999999,"15min":0.85199999999999998},"min":{"1min":0.57299999999999995,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.0329999999999999,"5min":1.462,"15min":1.462},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.93300000000000005,"15min":0.90500000000000003},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.224,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.81599999999999995}]},{"osd":2,"last update":"Mon Nov 6 02:13:27 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.93799999999999994,"15min":0.99199999999999999},"min":{"1min":0.70299999999999996,"5min":0.503,"15min":0.503},"max":{"1min":1.173,"5min":1.615,"15min":1.615},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.95499999999999996,"15min":0.93500000000000005},"min":{"1min":0.72499999999999998,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.163,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.93999999999999995}]},{"osd":3,"last update":"Mon Nov 6 02:13:27 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.88200000000000001,"15min":0.90600000000000003},"min":{"1min":0.61499999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.216,"5min":1.52,"15min":1.52},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.997,"15min":1.0449999999999999},"min":{"1min":0.66900000000000004,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.2050000000000001,"5min":1.595,"15min":1.595},"last":0.68100000000000005}]},{"osd":5,"last update":"Mon Nov 6 02:13:27 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.99099999999999999,"15min":1.0309999999999999},"min":{"1min":0.64500000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.339,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":0.995},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.97599999999999998,"15min":1.0289999999999999},"min":{"1min":0.68899999999999995,"5min":0.629,"15min":0.629},"max":{"1min":1.161,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":0.96999999999999997}]},{"osd":6,"last update":"Mon Nov 6 02:13:27 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":1.008,"15min":1.0649999999999999},"min":{"1min":0.69599999999999995,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.268,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":0.86799999999999999},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.1120000000000001,"15min":1.169},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.2789999999999999,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":1.0409999999999999}]},{"osd":7,"last update":"Mon Nov 6 02:13:27 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.0489999999999999,"15min":1.079},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.294,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.89400000000000002},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.133,"15min":1.204},"min":{"1min":0.77500000000000002,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.3149999999999999,"5min":1.7410000000000001,"15min":1.7410000000000001},"last":0.91600000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574914,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32004,"kb_used_data":956,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339836,"statfs":{"total":96636764160,"available":96603992064,"internally_reserved":0,"allocated":978944,"data_stored":439972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:13:28 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.63200000000000001,"15min":0.64200000000000002},"min":{"1min":0.40799999999999997,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":1.198,"5min":1.198,"15min":1.198},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.64000000000000001,"15min":0.63},"min":{"1min":0.38700000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.64400000000000002}]},{"osd":1,"last update":"Mon Nov 6 02:13:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.61699999999999999,"15min":0.627},"min":{"1min":0.34599999999999997,"5min":0.248,"15min":0.248},"max":{"1min":0.92600000000000005,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.73999999999999999,"15min":0.73599999999999999},"min":{"1min":0.53000000000000003,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.73699999999999999}]},{"osd":2,"last update":"Mon Nov 6 02:13:28 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.71899999999999997,"15min":0.72899999999999998},"min":{"1min":0.45400000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.107,"5min":1.139,"15min":1.139},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.80800000000000005,"15min":0.81799999999999995},"min":{"1min":0.48099999999999998,"5min":0.308,"15min":0.308},"max":{"1min":1.212,"5min":1.405,"15min":1.405},"last":0.95799999999999996}]},{"osd":4,"last update":"Mon Nov 6 02:13:28 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.92000000000000004,"15min":0.92600000000000005},"min":{"1min":0.72599999999999998,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.0820000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.95299999999999996,"15min":0.98299999999999998},"min":{"1min":0.65100000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.1459999999999999,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.99399999999999999}]},{"osd":5,"last update":"Mon Nov 6 02:13:28 2023","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.98899999999999999,"15min":1.0149999999999999},"min":{"1min":0.72799999999999998,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.333,"5min":1.375,"15min":1.375},"last":0.93999999999999995},{"interface":"front","average":{"1min":1.091,"5min":1.0389999999999999,"15min":1.0669999999999999},"min":{"1min":0.627,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.4890000000000001,"5min":2.21,"15min":2.21},"last":1.032}]},{"osd":6,"last update":"Mon Nov 6 02:13:28 2023","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.016,"15min":1.022},"min":{"1min":0.61299999999999999,"5min":0.439,"15min":0.439},"max":{"1min":1.3740000000000001,"5min":1.395,"15min":1.395},"last":0.91500000000000004},{"interface":"front","average":{"1min":1.036,"5min":1.04,"15min":1.0609999999999999},"min":{"1min":0.54600000000000004,"5min":0.375,"15min":0.375},"max":{"1min":1.3069999999999999,"5min":1.554,"15min":1.554},"last":0.77800000000000002}]},{"osd":7,"last update":"Mon Nov 6 02:13:28 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.1080000000000001,"15min":1.1240000000000001},"min":{"1min":0.66800000000000004,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.421,"5min":2.2829999999999999,"15min":2.2829999999999999},"last":1.0960000000000001},{"interface":"front","average":{"1min":1.091,"5min":1.0940000000000001,"15min":1.1020000000000001},"min":{"1min":0.70999999999999996,"5min":0.497,"15min":0.497},"max":{"1min":1.4490000000000001,"5min":1.504,"15min":1.504},"last":1.0609999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574914,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32012,"kb_used_data":964,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339828,"statfs":{"total":96636764160,"available":96603983872,"internally_reserved":0,"allocated":987136,"data_stored":440132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:13:24 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.57699999999999996,"15min":0.52500000000000002},"min":{"1min":0.33100000000000002,"5min":0.252,"15min":0.252},"max":{"1min":0.85699999999999998,"5min":1.079,"15min":1.079},"last":0.621},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.625,"15min":0.56699999999999995},"min":{"1min":0.505,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.1060000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":0.67100000000000004}]},{"osd":1,"last update":"Mon Nov 6 02:13:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.63500000000000001,"15min":0.58499999999999996},"min":{"1min":0.38800000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.88300000000000001,"5min":1.48,"15min":1.48},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.70599999999999996,"15min":0.61399999999999999},"min":{"1min":0.377,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.282,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":0.78500000000000003}]},{"osd":3,"last update":"Mon Nov 6 02:13:24 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.70099999999999996,"15min":0.63200000000000001},"min":{"1min":0.48199999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.98999999999999999,"5min":1.155,"15min":1.155},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.749,"15min":0.67700000000000005},"min":{"1min":0.41999999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.97999999999999998}]},{"osd":4,"last update":"Mon Nov 6 02:13:24 2023","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.90900000000000003,"15min":0.83499999999999996},"min":{"1min":0.58499999999999996,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.2150000000000001,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":1.117},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.93500000000000005,"15min":0.92100000000000004},"min":{"1min":0.73099999999999998,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.3120000000000001,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":0.93700000000000006}]},{"osd":5,"last update":"Mon Nov 6 02:13:24 2023","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":0.96799999999999997,"15min":0.95099999999999996},"min":{"1min":0.75,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.2270000000000001,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":1.1850000000000001},{"interface":"front","average":{"1min":1,"5min":0.97099999999999997,"15min":0.94599999999999995},"min":{"1min":0.628,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.3109999999999999,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":1.2310000000000001}]},{"osd":6,"last update":"Mon Nov 6 02:13:24 2023","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":0.97099999999999997,"15min":0.94899999999999995},"min":{"1min":0.59899999999999998,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.3129999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":0.82699999999999996},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.0229999999999999,"15min":1.0049999999999999},"min":{"1min":0.73299999999999998,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.2769999999999999,"5min":1.677,"15min":1.677},"last":1.274}]},{"osd":7,"last update":"Mon Nov 6 02:13:24 2023","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.026,"15min":0.99199999999999999},"min":{"1min":0.72199999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.4079999999999999,"5min":1.659,"15min":1.659},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.123,"5min":1.0820000000000001,"15min":1.0549999999999999},"min":{"1min":0.79300000000000004,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.3819999999999999,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":1.0369999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574915,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32008,"kb_used_data":960,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339832,"statfs":{"total":96636764160,"available":96603987968,"internally_reserved":0,"allocated":983040,"data_stored":439502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:13:27 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.63900000000000001,"15min":0.63},"min":{"1min":0.35899999999999999,"5min":0.224,"15min":0.224},"max":{"1min":1.343,"5min":1.603,"15min":1.603},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.71899999999999997,"15min":0.67400000000000004},"min":{"1min":0.38200000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":2.161,"5min":2.161,"15min":2.161},"last":0.499}]},{"osd":2,"last update":"Mon Nov 6 02:13:27 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.65400000000000003,"15min":0.63200000000000001},"min":{"1min":0.26500000000000001,"5min":0.253,"15min":0.253},"max":{"1min":1.256,"5min":1.649,"15min":1.649},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.753,"15min":0.747},"min":{"1min":0.41099999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":1.5249999999999999,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.89300000000000002}]},{"osd":3,"last update":"Mon Nov 6 02:13:27 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.81299999999999994,"15min":0.80600000000000005},"min":{"1min":0.49099999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.6319999999999999,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.81299999999999994,"15min":0.81100000000000005},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.1799999999999999,"5min":1.8149999999999999,"15min":1.8149999999999999},"last":0.58399999999999996}]},{"osd":4,"last update":"Mon Nov 6 02:13:27 2023","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":0.97899999999999998,"15min":0.98799999999999999},"min":{"1min":0.78500000000000003,"5min":0.504,"15min":0.504},"max":{"1min":2.2759999999999998,"5min":2.2759999999999998,"15min":2.2759999999999998},"last":1.1930000000000001},{"interface":"front","average":{"1min":1.0740000000000001,"5min":0.97699999999999998,"15min":0.98799999999999999},"min":{"1min":0.64200000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":2.9180000000000001,"5min":2.9180000000000001,"15min":2.9180000000000001},"last":1.046}]},{"osd":5,"last update":"Mon Nov 6 02:13:27 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":0.98099999999999998,"15min":0.97099999999999997},"min":{"1min":0.68100000000000005,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":2.7469999999999999,"5min":2.7469999999999999,"15min":2.7469999999999999},"last":1.111},{"interface":"front","average":{"1min":1.1579999999999999,"5min":1.069,"15min":1.075},"min":{"1min":0.81100000000000005,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.921,"5min":1.921,"15min":1.921},"last":1.3720000000000001}]},{"osd":6,"last update":"Mon Nov 6 02:13:27 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.113,"15min":1.1060000000000001},"min":{"1min":0.83099999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":3.5710000000000002,"5min":3.5710000000000002,"15min":3.5710000000000002},"last":1.0620000000000001},{"interface":"front","average":{"1min":1.157,"5min":1.0629999999999999,"15min":1.0720000000000001},"min":{"1min":0.60299999999999998,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":2.843,"5min":2.843,"15min":2.843},"last":1.087}]},{"osd":7,"last update":"Mon Nov 6 02:13:27 2023","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.1180000000000001,"15min":1.1200000000000001},"min":{"1min":0.89600000000000002,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":2.0009999999999999,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":1.4159999999999999},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.1639999999999999,"15min":1.1619999999999999},"min":{"1min":0.91100000000000003,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":3.4790000000000001,"5min":3.4790000000000001,"15min":3.4790000000000001},"last":1.018}]}]},{"osd":0,"up_from":13,"seq":55834574915,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32576,"kb_used_data":1528,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339264,"statfs":{"total":96636764160,"available":96603406336,"internally_reserved":0,"allocated":1564672,"data_stored":1028739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 02:13:26 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.61499999999999999,"15min":0.60299999999999998},"min":{"1min":0.53100000000000003,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.0289999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.755,"5min":0.66900000000000004,"15min":0.64700000000000002},"min":{"1min":0.53800000000000003,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.079,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.498}]},{"osd":2,"last update":"Mon Nov 6 02:13:26 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.68600000000000005,"15min":0.67200000000000004},"min":{"1min":0.42999999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.1080000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.69799999999999995,"15min":0.68200000000000005},"min":{"1min":0.57299999999999995,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.77500000000000002}]},{"osd":3,"last update":"Mon Nov 6 02:13:26 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.67200000000000004,"15min":0.64300000000000002},"min":{"1min":0.52500000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.1659999999999999,"5min":1.429,"15min":1.429},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.87,"5min":0.78300000000000003,"15min":0.748},"min":{"1min":0.69999999999999996,"5min":0.317,"15min":0.317},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.65200000000000002}]},{"osd":4,"last update":"Mon Nov 6 02:13:26 2023","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.90200000000000002,"15min":0.85199999999999998},"min":{"1min":0.76800000000000002,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.1779999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.751},{"interface":"front","average":{"1min":1.0589999999999999,"5min":0.96099999999999997,"15min":0.94499999999999995},"min":{"1min":0.78900000000000003,"5min":0.5,"15min":0.5},"max":{"1min":1.292,"5min":1.387,"15min":1.387},"last":1.018}]},{"osd":5,"last update":"Mon Nov 6 02:13:26 2023","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":0.95699999999999996,"15min":0.95099999999999996},"min":{"1min":0.81799999999999995,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.871},{"interface":"front","average":{"1min":1.006,"5min":0.93400000000000005,"15min":0.89200000000000002},"min":{"1min":0.81100000000000005,"5min":0.496,"15min":0.496},"max":{"1min":1.2549999999999999,"5min":1.448,"15min":1.448},"last":0.90100000000000002}]},{"osd":6,"last update":"Mon Nov 6 02:13:26 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.0389999999999999,"15min":1.0369999999999999},"min":{"1min":0.73899999999999999,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.385,"5min":1.589,"15min":1.589},"last":1.111},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.0269999999999999,"15min":1.0089999999999999},"min":{"1min":0.76000000000000001,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.419,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":1.0609999999999999}]},{"osd":7,"last update":"Mon Nov 6 02:13:26 2023","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.026,"15min":1.0209999999999999},"min":{"1min":0.76400000000000001,"5min":0.504,"15min":0.504},"max":{"1min":1.504,"5min":1.504,"15min":1.504},"last":0.95799999999999996},{"interface":"front","average":{"1min":1.073,"5min":0.998,"15min":1.0049999999999999},"min":{"1min":0.74299999999999999,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.3999999999999999,"5min":1.556,"15min":1.556},"last":0.80800000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T02:13:42.587 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-06T02:13:42.588 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-06T02:13:42.762 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.766+0000 7f1a6cbb2640 1 Processor -- start 2023-11-06T02:13:42.762 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.766+0000 7f1a6cbb2640 1 -- start start 2023-11-06T02:13:42.764 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.766+0000 7f1a6cbb2640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1a6815a880 0x7f1a680b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:42.764 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.766+0000 7f1a6cbb2640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f1a68070160 0x7f1a680705e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:42.764 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.766+0000 7f1a6cbb2640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f1a6806d150 0x7f1a6806d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:42.764 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.766+0000 7f1a6cbb2640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a68070b20 con 0x7f1a68070160 2023-11-06T02:13:42.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.766+0000 7f1a6cbb2640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a6806db20 con 0x7f1a6806d150 2023-11-06T02:13:42.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.766+0000 7f1a6cbb2640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a680ae360 con 0x7f1a6815a880 2023-11-06T02:13:42.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.766+0000 7f1a65d9b640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f1a68070160 0x7f1a680705e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:42.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.766+0000 7f1a65d9b640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f1a68070160 0x7f1a680705e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47910/0 (socket says 172.21.15.49:47910) 2023-11-06T02:13:42.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.766+0000 7f1a65d9b640 1 -- 172.21.15.49:0/2270473467 learned_addr learned my addr 172.21.15.49:0/2270473467 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:42.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.766+0000 7f1a66d9d640 1 --2- 172.21.15.49:0/2270473467 >> v2:172.21.15.150:3300/0 conn(0x7f1a6806d150 0x7f1a6806d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:42.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.766+0000 7f1a6659c640 1 --2- 172.21.15.49:0/2270473467 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1a6815a880 0x7f1a680b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:42.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.766+0000 7f1a65d9b640 1 -- 172.21.15.49:0/2270473467 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1a6815a880 msgr2=0x7f1a680b15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:42.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.766+0000 7f1a65d9b640 1 --2- 172.21.15.49:0/2270473467 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1a6815a880 0x7f1a680b15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:42.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.766+0000 7f1a65d9b640 1 -- 172.21.15.49:0/2270473467 >> v2:172.21.15.150:3300/0 conn(0x7f1a6806d150 msgr2=0x7f1a6806d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:42.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.766+0000 7f1a65d9b640 1 --2- 172.21.15.49:0/2270473467 >> v2:172.21.15.150:3300/0 conn(0x7f1a6806d150 0x7f1a6806d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:42.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.766+0000 7f1a65d9b640 1 -- 172.21.15.49:0/2270473467 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a680ae640 con 0x7f1a68070160 2023-11-06T02:13:42.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.766+0000 7f1a66d9d640 1 --2- 172.21.15.49:0/2270473467 >> v2:172.21.15.150:3300/0 conn(0x7f1a6806d150 0x7f1a6806d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:13:42.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.770+0000 7f1a65d9b640 1 --2- 172.21.15.49:0/2270473467 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1a68070160 0x7f1a680705e0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f1a680cf1b0 tx=0x7f1a5801c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f05c516cb881be6 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:42.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.770+0000 7f1a6559a640 1 -- 172.21.15.49:0/2270473467 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a580ac630 con 0x7f1a68070160 2023-11-06T02:13:42.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.770+0000 7f1a6559a640 1 -- 172.21.15.49:0/2270473467 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a580ac790 con 0x7f1a68070160 2023-11-06T02:13:42.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.770+0000 7f1a6559a640 1 -- 172.21.15.49:0/2270473467 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a580acb20 con 0x7f1a68070160 2023-11-06T02:13:42.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.770+0000 7f1a6cbb2640 1 -- 172.21.15.49:0/2270473467 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1a68070160 msgr2=0x7f1a680705e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:42.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.770+0000 7f1a6cbb2640 1 --2- 172.21.15.49:0/2270473467 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1a68070160 0x7f1a680705e0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f1a680cf1b0 tx=0x7f1a5801c500 comp rx=0 tx=0).stop 2023-11-06T02:13:42.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.770+0000 7f1a6cbb2640 1 -- 172.21.15.49:0/2270473467 shutdown_connections 2023-11-06T02:13:42.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.770+0000 7f1a6cbb2640 1 --2- 172.21.15.49:0/2270473467 >> v2:172.21.15.150:3300/0 conn(0x7f1a6806d150 0x7f1a6806d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:42.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.770+0000 7f1a6cbb2640 1 --2- 172.21.15.49:0/2270473467 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1a68070160 0x7f1a680705e0 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:42.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.770+0000 7f1a6cbb2640 1 --2- 172.21.15.49:0/2270473467 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1a6815a880 0x7f1a680b15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:42.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.770+0000 7f1a6cbb2640 1 -- 172.21.15.49:0/2270473467 >> 172.21.15.49:0/2270473467 conn(0x7f1a6806a370 msgr2=0x7f1a680ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:42.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.770+0000 7f1a6cbb2640 1 -- 172.21.15.49:0/2270473467 shutdown_connections 2023-11-06T02:13:42.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.770+0000 7f1a6cbb2640 1 -- 172.21.15.49:0/2270473467 wait complete. 2023-11-06T02:13:42.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.774+0000 7f1a6cbb2640 1 Processor -- start 2023-11-06T02:13:42.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.774+0000 7f1a6cbb2640 1 -- start start 2023-11-06T02:13:42.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.774+0000 7f1a6cbb2640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1a6806d150 0x7f1a680cb020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:42.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.774+0000 7f1a6cbb2640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1a68070160 0x7f1a680ca750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:42.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.774+0000 7f1a6cbb2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1a6815a880 0x7f1a68142080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:42.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.774+0000 7f1a6cbb2640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a681425c0 con 0x7f1a6815a880 2023-11-06T02:13:42.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.774+0000 7f1a6cbb2640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a680cb600 con 0x7f1a6806d150 2023-11-06T02:13:42.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.774+0000 7f1a6cbb2640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a680cb8e0 con 0x7f1a68070160 2023-11-06T02:13:42.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.774+0000 7f1a66d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1a6815a880 0x7f1a68142080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:42.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.774+0000 7f1a66d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1a6815a880 0x7f1a68142080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47920/0 (socket says 172.21.15.49:47920) 2023-11-06T02:13:42.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.774+0000 7f1a66d9d640 1 -- 172.21.15.49:0/3734591927 learned_addr learned my addr 172.21.15.49:0/3734591927 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:42.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.774+0000 7f1a65d9b640 1 --2- 172.21.15.49:0/3734591927 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1a68070160 0x7f1a680ca750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:42.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.774+0000 7f1a6659c640 1 --2- 172.21.15.49:0/3734591927 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1a6806d150 0x7f1a680cb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:42.776 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.774+0000 7f1a66d9d640 1 -- 172.21.15.49:0/3734591927 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1a68070160 msgr2=0x7f1a680ca750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:42.776 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.774+0000 7f1a66d9d640 1 --2- 172.21.15.49:0/3734591927 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1a68070160 0x7f1a680ca750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:42.776 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.774+0000 7f1a66d9d640 1 -- 172.21.15.49:0/3734591927 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1a6806d150 msgr2=0x7f1a680cb020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:42.776 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.774+0000 7f1a66d9d640 1 --2- 172.21.15.49:0/3734591927 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1a6806d150 0x7f1a680cb020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:42.776 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.774+0000 7f1a66d9d640 1 -- 172.21.15.49:0/3734591927 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a580094d0 con 0x7f1a6815a880 2023-11-06T02:13:42.776 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.774+0000 7f1a66d9d640 1 --2- 172.21.15.49:0/3734591927 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1a6815a880 0x7f1a68142080 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f1a6000c7f0 tx=0x7f1a6000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:42.777 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.774+0000 7f1a5effd640 1 -- 172.21.15.49:0/3734591927 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a60007820 con 0x7f1a6815a880 2023-11-06T02:13:42.777 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.774+0000 7f1a5effd640 1 -- 172.21.15.49:0/3734591927 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a60007980 con 0x7f1a6815a880 2023-11-06T02:13:42.777 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.774+0000 7f1a6cbb2640 1 -- 172.21.15.49:0/3734591927 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a6815ade0 con 0x7f1a6815a880 2023-11-06T02:13:42.777 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.774+0000 7f1a5effd640 1 -- 172.21.15.49:0/3734591927 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a60007d30 con 0x7f1a6815a880 2023-11-06T02:13:42.777 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.774+0000 7f1a6cbb2640 1 -- 172.21.15.49:0/3734591927 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a680cbea0 con 0x7f1a6815a880 2023-11-06T02:13:42.778 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.774+0000 7f1a6cbb2640 1 -- 172.21.15.49:0/3734591927 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a680c3d10 con 0x7f1a6815a880 2023-11-06T02:13:42.779 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.774+0000 7f1a5effd640 1 -- 172.21.15.49:0/3734591927 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f1a600113f0 con 0x7f1a6815a880 2023-11-06T02:13:42.779 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.774+0000 7f1a5effd640 1 --2- 172.21.15.49:0/3734591927 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1a48065a60 0x7f1a48067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:42.779 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.774+0000 7f1a5effd640 1 -- 172.21.15.49:0/3734591927 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1a60083580 con 0x7f1a6815a880 2023-11-06T02:13:42.779 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.778+0000 7f1a6659c640 1 --2- 172.21.15.49:0/3734591927 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1a48065a60 0x7f1a48067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:42.780 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.778+0000 7f1a6659c640 1 --2- 172.21.15.49:0/3734591927 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1a48065a60 0x7f1a48067f20 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f1a68151f10 tx=0x7f1a500078e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:42.780 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.782+0000 7f1a5effd640 1 -- 172.21.15.49:0/3734591927 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f1a60050f30 con 0x7f1a6815a880 2023-11-06T02:13:42.990 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:42.994+0000 7f1a6cbb2640 1 -- 172.21.15.49:0/3734591927 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f1a68074040 con 0x7f1a6815a880 2023-11-06T02:13:42.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.002+0000 7f1a5effd640 1 -- 172.21.15.49:0/3734591927 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) v1 ==== 190+0+0 (secure 0 0 0) 0x7f1a60054550 con 0x7f1a6815a880 2023-11-06T02:13:42.998 INFO:teuthology.orchestra.run.smithi049.stderr:updated caps for client.0 2023-11-06T02:13:43.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.006+0000 7f1a6cbb2640 1 -- 172.21.15.49:0/3734591927 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1a48065a60 msgr2=0x7f1a48067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:43.001 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.006+0000 7f1a6cbb2640 1 --2- 172.21.15.49:0/3734591927 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1a48065a60 0x7f1a48067f20 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f1a68151f10 tx=0x7f1a500078e0 comp rx=0 tx=0).stop 2023-11-06T02:13:43.001 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.006+0000 7f1a6cbb2640 1 -- 172.21.15.49:0/3734591927 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1a6815a880 msgr2=0x7f1a68142080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:43.001 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.006+0000 7f1a6cbb2640 1 --2- 172.21.15.49:0/3734591927 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1a6815a880 0x7f1a68142080 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f1a6000c7f0 tx=0x7f1a6000ccb0 comp rx=0 tx=0).stop 2023-11-06T02:13:43.001 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.006+0000 7f1a6cbb2640 1 -- 172.21.15.49:0/3734591927 shutdown_connections 2023-11-06T02:13:43.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.006+0000 7f1a6cbb2640 1 --2- 172.21.15.49:0/3734591927 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1a48065a60 0x7f1a48067f20 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:43.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.006+0000 7f1a6cbb2640 1 --2- 172.21.15.49:0/3734591927 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1a6815a880 0x7f1a68142080 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:43.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.006+0000 7f1a6cbb2640 1 --2- 172.21.15.49:0/3734591927 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1a68070160 0x7f1a680ca750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:43.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.006+0000 7f1a6cbb2640 1 --2- 172.21.15.49:0/3734591927 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1a6806d150 0x7f1a680cb020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:43.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.006+0000 7f1a6cbb2640 1 -- 172.21.15.49:0/3734591927 >> 172.21.15.49:0/3734591927 conn(0x7f1a6806a370 msgr2=0x7f1a681543d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:43.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.006+0000 7f1a6cbb2640 1 -- 172.21.15.49:0/3734591927 shutdown_connections 2023-11-06T02:13:43.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.006+0000 7f1a6cbb2640 1 -- 172.21.15.49:0/3734591927 wait complete. 2023-11-06T02:13:43.018 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T02:13:43.144 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.146+0000 7f302dffd640 1 Processor -- start 2023-11-06T02:13:43.144 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.146+0000 7f302dffd640 1 -- start start 2023-11-06T02:13:43.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.150+0000 7f302dffd640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f30280b0790 0x7f30280c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:43.146 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.150+0000 7f302dffd640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f30280cb010 0x7f30280c2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:43.146 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.150+0000 7f302dffd640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30280cab80 con 0x7f302815ab50 2023-11-06T02:13:43.146 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.150+0000 7f302dffd640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f302815af30 con 0x7f30280b0790 2023-11-06T02:13:43.146 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.150+0000 7f302dffd640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30280cb5b0 con 0x7f30280cb010 2023-11-06T02:13:43.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.150+0000 7f3026ffd640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f30280b0790 0x7f30280c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:43.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.150+0000 7f3026ffd640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f30280b0790 0x7f30280c9bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47924/0 (socket says 172.21.15.49:47924) 2023-11-06T02:13:43.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.150+0000 7f3026ffd640 1 -- 172.21.15.49:0/1219820851 learned_addr learned my addr 172.21.15.49:0/1219820851 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:43.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.150+0000 7f3027fff640 1 --2- 172.21.15.49:0/1219820851 >> v2:172.21.15.150:3300/0 conn(0x7f30280cb010 0x7f30280c2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:43.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.150+0000 7f3026ffd640 1 -- 172.21.15.49:0/1219820851 >> v2:172.21.15.150:3300/0 conn(0x7f30280cb010 msgr2=0x7f30280c2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:43.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.150+0000 7f3026ffd640 1 --2- 172.21.15.49:0/1219820851 >> v2:172.21.15.150:3300/0 conn(0x7f30280cb010 0x7f30280c2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:43.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.150+0000 7f3026ffd640 1 -- 172.21.15.49:0/1219820851 >> v1:172.21.15.150:6789/0 conn(0x7f302815ab50 legacy=0x7f30280b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:43.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.150+0000 7f3026ffd640 1 -- 172.21.15.49:0/1219820851 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30280cb890 con 0x7f30280b0790 2023-11-06T02:13:43.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.150+0000 7f3027fff640 1 --2- 172.21.15.49:0/1219820851 >> v2:172.21.15.150:3300/0 conn(0x7f30280cb010 0x7f30280c2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:13:43.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.154+0000 7f3026ffd640 1 --2- 172.21.15.49:0/1219820851 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30280b0790 0x7f30280c9bb0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f30280cae30 tx=0x7f301c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d4cff303ed9b05b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:43.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.154+0000 7f30267fc640 1 -- 172.21.15.49:0/1219820851 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f301c0aa610 con 0x7f30280b0790 2023-11-06T02:13:43.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.154+0000 7f30267fc640 1 -- 172.21.15.49:0/1219820851 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f301c0a6040 con 0x7f30280b0790 2023-11-06T02:13:43.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.154+0000 7f30267fc640 1 -- 172.21.15.49:0/1219820851 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f301c0aa990 con 0x7f30280b0790 2023-11-06T02:13:43.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.154+0000 7f302dffd640 1 -- 172.21.15.49:0/1219820851 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30280b0790 msgr2=0x7f30280c9bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:43.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.154+0000 7f302dffd640 1 --2- 172.21.15.49:0/1219820851 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30280b0790 0x7f30280c9bb0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f30280cae30 tx=0x7f301c01c490 comp rx=0 tx=0).stop 2023-11-06T02:13:43.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.154+0000 7f302dffd640 1 -- 172.21.15.49:0/1219820851 shutdown_connections 2023-11-06T02:13:43.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.154+0000 7f302dffd640 1 --2- 172.21.15.49:0/1219820851 >> v2:172.21.15.150:3300/0 conn(0x7f30280cb010 0x7f30280c2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:43.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.154+0000 7f302dffd640 1 --2- 172.21.15.49:0/1219820851 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30280b0790 0x7f30280c9bb0 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:43.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.154+0000 7f302dffd640 1 -- 172.21.15.49:0/1219820851 >> 172.21.15.49:0/1219820851 conn(0x7f302806c0a0 msgr2=0x7f302806c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:43.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.154+0000 7f302dffd640 1 -- 172.21.15.49:0/1219820851 shutdown_connections 2023-11-06T02:13:43.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.154+0000 7f302dffd640 1 -- 172.21.15.49:0/1219820851 wait complete. 2023-11-06T02:13:43.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.154+0000 7f302dffd640 1 Processor -- start 2023-11-06T02:13:43.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.154+0000 7f302dffd640 1 -- start start 2023-11-06T02:13:43.157 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.154+0000 7f302dffd640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f30280b0790 0x7f30280c8370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:43.157 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.154+0000 7f302dffd640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f30280cb010 0x7f30280c08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:43.158 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.154+0000 7f302dffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f302815ab50 0x7f30281547d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:43.158 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.154+0000 7f302dffd640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30280c6b50 con 0x7f302815ab50 2023-11-06T02:13:43.158 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.154+0000 7f302dffd640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30280c88b0 con 0x7f30280b0790 2023-11-06T02:13:43.159 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.154+0000 7f302dffd640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30281539a0 con 0x7f30280cb010 2023-11-06T02:13:43.159 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.154+0000 7f3027fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f302815ab50 0x7f30281547d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:43.160 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.158+0000 7f3027fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f302815ab50 0x7f30281547d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47930/0 (socket says 172.21.15.49:47930) 2023-11-06T02:13:43.160 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.158+0000 7f3027fff640 1 -- 172.21.15.49:0/4152208471 learned_addr learned my addr 172.21.15.49:0/4152208471 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:43.160 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.158+0000 7f3026ffd640 1 --2- 172.21.15.49:0/4152208471 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f30280cb010 0x7f30280c08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:43.160 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.158+0000 7f30277fe640 1 --2- 172.21.15.49:0/4152208471 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f30280b0790 0x7f30280c8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:43.161 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.158+0000 7f3027fff640 1 -- 172.21.15.49:0/4152208471 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f30280cb010 msgr2=0x7f30280c08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:43.161 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.158+0000 7f3027fff640 1 --2- 172.21.15.49:0/4152208471 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f30280cb010 0x7f30280c08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:43.161 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.158+0000 7f3027fff640 1 -- 172.21.15.49:0/4152208471 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f30280b0790 msgr2=0x7f30280c8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:43.162 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.158+0000 7f3027fff640 1 --2- 172.21.15.49:0/4152208471 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f30280b0790 0x7f30280c8370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:43.162 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.158+0000 7f3027fff640 1 -- 172.21.15.49:0/4152208471 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f301c0094d0 con 0x7f302815ab50 2023-11-06T02:13:43.162 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.158+0000 7f3027fff640 1 --2- 172.21.15.49:0/4152208471 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f302815ab50 0x7f30281547d0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f302000cf20 tx=0x7f302000a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:43.162 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.158+0000 7f300ffff640 1 -- 172.21.15.49:0/4152208471 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f302000d840 con 0x7f302815ab50 2023-11-06T02:13:43.162 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.158+0000 7f300ffff640 1 -- 172.21.15.49:0/4152208471 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f302000d9a0 con 0x7f302815ab50 2023-11-06T02:13:43.163 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.158+0000 7f302dffd640 1 -- 172.21.15.49:0/4152208471 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f302815ca90 con 0x7f302815ab50 2023-11-06T02:13:43.163 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.158+0000 7f300ffff640 1 -- 172.21.15.49:0/4152208471 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f302000dd50 con 0x7f302815ab50 2023-11-06T02:13:43.163 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.158+0000 7f302dffd640 1 -- 172.21.15.49:0/4152208471 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30280c67e0 con 0x7f302815ab50 2023-11-06T02:13:43.164 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.158+0000 7f302dffd640 1 -- 172.21.15.49:0/4152208471 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30280c6d00 con 0x7f302815ab50 2023-11-06T02:13:43.164 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.158+0000 7f300ffff640 1 -- 172.21.15.49:0/4152208471 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f302001e3b0 con 0x7f302815ab50 2023-11-06T02:13:43.164 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.158+0000 7f300ffff640 1 --2- 172.21.15.49:0/4152208471 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3014065a90 0x7f3014067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:43.164 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.158+0000 7f300ffff640 1 -- 172.21.15.49:0/4152208471 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3020083d60 con 0x7f302815ab50 2023-11-06T02:13:43.165 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.158+0000 7f30277fe640 1 --2- 172.21.15.49:0/4152208471 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3014065a90 0x7f3014067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:43.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.158+0000 7f30277fe640 1 --2- 172.21.15.49:0/4152208471 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3014065a90 0x7f3014067f50 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f3018006fd0 tx=0x7f3018008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:43.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.162+0000 7f300ffff640 1 -- 172.21.15.49:0/4152208471 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f30280c6d00 con 0x7f302815ab50 2023-11-06T02:13:43.350 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.354+0000 7f302dffd640 1 -- 172.21.15.49:0/4152208471 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f30280c6d00 con 0x7f302815ab50 2023-11-06T02:13:43.351 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.354+0000 7f300ffff640 1 -- 172.21.15.49:0/4152208471 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+4650 (secure 0 0 0) 0x7f30280c6d00 con 0x7f302815ab50 2023-11-06T02:13:43.351 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:13:43.352 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":46,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5714,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/315581318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":315581318},{"type":"v1","addr":"172.21.15.150:6837","nonce":315581318}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41},{"gid":6121,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/136800245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":136800245},{"type":"v1","addr":"172.21.15.150:6835","nonce":136800245}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41},{"gid":7110,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/4259579254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":4259579254},{"type":"v1","addr":"172.21.15.49:6833","nonce":4259579254}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T02:13:33.625701+0000","modified":"2023-11-06T02:13:36.649877+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7116},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7116":{"gid":7116,"name":"c","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.49:6835/1130379293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":1130379293},{"type":"v1","addr":"172.21.15.49:6835","nonce":1130379293}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-06T02:13:43.352 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 46 2023-11-06T02:13:43.353 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.358+0000 7f302dffd640 1 -- 172.21.15.49:0/4152208471 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3014065a90 msgr2=0x7f3014067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:43.353 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.358+0000 7f302dffd640 1 --2- 172.21.15.49:0/4152208471 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3014065a90 0x7f3014067f50 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f3018006fd0 tx=0x7f3018008040 comp rx=0 tx=0).stop 2023-11-06T02:13:43.353 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.358+0000 7f302dffd640 1 -- 172.21.15.49:0/4152208471 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f302815ab50 msgr2=0x7f30281547d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:43.353 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.358+0000 7f302dffd640 1 --2- 172.21.15.49:0/4152208471 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f302815ab50 0x7f30281547d0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f302000cf20 tx=0x7f302000a7d0 comp rx=0 tx=0).stop 2023-11-06T02:13:43.354 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.358+0000 7f302dffd640 1 -- 172.21.15.49:0/4152208471 shutdown_connections 2023-11-06T02:13:43.354 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.358+0000 7f302dffd640 1 --2- 172.21.15.49:0/4152208471 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3014065a90 0x7f3014067f50 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:43.354 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.358+0000 7f302dffd640 1 --2- 172.21.15.49:0/4152208471 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f302815ab50 0x7f30281547d0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:43.354 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.358+0000 7f302dffd640 1 --2- 172.21.15.49:0/4152208471 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f30280cb010 0x7f30280c08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:43.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.358+0000 7f302dffd640 1 --2- 172.21.15.49:0/4152208471 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f30280b0790 0x7f30280c8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:43.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.358+0000 7f302dffd640 1 -- 172.21.15.49:0/4152208471 >> 172.21.15.49:0/4152208471 conn(0x7f302806c0a0 msgr2=0x7f3028159a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:43.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.358+0000 7f302dffd640 1 -- 172.21.15.49:0/4152208471 shutdown_connections 2023-11-06T02:13:43.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.358+0000 7f302dffd640 1 -- 172.21.15.49:0/4152208471 wait complete. 2023-11-06T02:13:43.771 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 46, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-06T02:13:33.625701+0000', 'modified': '2023-11-06T02:13:36.649877+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7116}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7116': {'gid': 7116, 'name': 'c', 'rank': 0, 'incarnation': 42, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.49:6835/1130379293', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6834', 'nonce': 1130379293}, {'type': 'v1', 'addr': '172.21.15.49:6835', 'nonce': 1130379293}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-06T02:13:43.771 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-06T02:13:43.772 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-06T02:13:43.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.914+0000 7fb59c31b640 1 Processor -- start 2023-11-06T02:13:43.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.914+0000 7fb59c31b640 1 -- start start 2023-11-06T02:13:43.911 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.914+0000 7fb59c31b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb5940ca110 0x7fb59415aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:43.911 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.914+0000 7fb59c31b640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb59415b1e0 0x7fb5940cb830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:43.911 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.914+0000 7fb59c31b640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb59415b650 con 0x7fb59415a8c0 2023-11-06T02:13:43.911 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.914+0000 7fb59c31b640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb59415b8c0 con 0x7fb59415b1e0 2023-11-06T02:13:43.911 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.914+0000 7fb59c31b640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb59415bba0 con 0x7fb5940ca110 2023-11-06T02:13:43.912 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.914+0000 7fb59a8b8640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb59415b1e0 0x7fb5940cb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:43.912 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.914+0000 7fb59a8b8640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb59415b1e0 0x7fb5940cb830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47936/0 (socket says 172.21.15.49:47936) 2023-11-06T02:13:43.912 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.914+0000 7fb59a8b8640 1 -- 172.21.15.49:0/947432897 learned_addr learned my addr 172.21.15.49:0/947432897 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:43.912 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.914+0000 7fb5998b6640 1 --2- 172.21.15.49:0/947432897 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb5940ca110 0x7fb59415aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:43.912 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.914+0000 7fb59a8b8640 1 -- 172.21.15.49:0/947432897 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb5940ca110 msgr2=0x7fb59415aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:43.913 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.914+0000 7fb59a8b8640 1 --2- 172.21.15.49:0/947432897 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb5940ca110 0x7fb59415aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:43.913 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.914+0000 7fb59a8b8640 1 -- 172.21.15.49:0/947432897 >> v1:172.21.15.150:6789/0 conn(0x7fb59415a8c0 legacy=0x7fb59415ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:43.913 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.914+0000 7fb59a8b8640 1 -- 172.21.15.49:0/947432897 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb59415e650 con 0x7fb59415b1e0 2023-11-06T02:13:43.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.918+0000 7fb59a8b8640 1 --2- 172.21.15.49:0/947432897 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb59415b1e0 0x7fb5940cb830 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fb584009870 tx=0x7fb58401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea6144fb866e65c0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:43.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.918+0000 7fb5990b5640 1 -- 172.21.15.49:0/947432897 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5840a6040 con 0x7fb59415b1e0 2023-11-06T02:13:43.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.918+0000 7fb5990b5640 1 -- 172.21.15.49:0/947432897 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5840aa600 con 0x7fb59415b1e0 2023-11-06T02:13:43.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.918+0000 7fb5990b5640 1 -- 172.21.15.49:0/947432897 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5840aa990 con 0x7fb59415b1e0 2023-11-06T02:13:43.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.918+0000 7fb59c31b640 1 -- 172.21.15.49:0/947432897 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb59415b1e0 msgr2=0x7fb5940cb830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:43.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.918+0000 7fb59c31b640 1 --2- 172.21.15.49:0/947432897 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb59415b1e0 0x7fb5940cb830 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fb584009870 tx=0x7fb58401c480 comp rx=0 tx=0).stop 2023-11-06T02:13:43.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.918+0000 7fb59c31b640 1 -- 172.21.15.49:0/947432897 shutdown_connections 2023-11-06T02:13:43.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.918+0000 7fb59c31b640 1 --2- 172.21.15.49:0/947432897 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb5940ca110 0x7fb59415aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:43.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.918+0000 7fb59c31b640 1 --2- 172.21.15.49:0/947432897 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb59415b1e0 0x7fb5940cb830 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:43.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.918+0000 7fb59c31b640 1 -- 172.21.15.49:0/947432897 >> 172.21.15.49:0/947432897 conn(0x7fb5940b0bd0 msgr2=0x7fb5940b0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:43.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.918+0000 7fb59c31b640 1 -- 172.21.15.49:0/947432897 shutdown_connections 2023-11-06T02:13:43.917 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.918+0000 7fb59c31b640 1 -- 172.21.15.49:0/947432897 wait complete. 2023-11-06T02:13:43.919 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.918+0000 7fb59c31b640 1 Processor -- start 2023-11-06T02:13:43.919 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.918+0000 7fb59c31b640 1 -- start start 2023-11-06T02:13:43.919 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.918+0000 7fb59c31b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb5940ca110 0x7fb5940cd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:43.919 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.918+0000 7fb59c31b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb59415a8c0 0x7fb594151b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:43.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.918+0000 7fb59c31b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb59415b1e0 0x7fb59415cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:43.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.918+0000 7fb59c31b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5940ccaf0 con 0x7fb59415b1e0 2023-11-06T02:13:43.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.918+0000 7fb59c31b640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5941f93b0 con 0x7fb59415a8c0 2023-11-06T02:13:43.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.918+0000 7fb59c31b640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5940ccd00 con 0x7fb5940ca110 2023-11-06T02:13:43.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.922+0000 7fb59a8b8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb59415b1e0 0x7fb59415cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:43.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.922+0000 7fb59a8b8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb59415b1e0 0x7fb59415cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47946/0 (socket says 172.21.15.49:47946) 2023-11-06T02:13:43.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.922+0000 7fb59a8b8640 1 -- 172.21.15.49:0/3377969331 learned_addr learned my addr 172.21.15.49:0/3377969331 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:43.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.922+0000 7fb59a0b7640 1 --2- 172.21.15.49:0/3377969331 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb5940ca110 0x7fb5940cd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:43.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.922+0000 7fb59a8b8640 1 -- 172.21.15.49:0/3377969331 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb5940ca110 msgr2=0x7fb5940cd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:43.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.922+0000 7fb59a8b8640 1 --2- 172.21.15.49:0/3377969331 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb5940ca110 0x7fb5940cd7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:43.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.922+0000 7fb59a8b8640 1 -- 172.21.15.49:0/3377969331 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb59415a8c0 msgr2=0x7fb594151b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:43.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.922+0000 7fb59a8b8640 1 --2- 172.21.15.49:0/3377969331 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb59415a8c0 0x7fb594151b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:43.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.922+0000 7fb59a8b8640 1 -- 172.21.15.49:0/3377969331 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5840094d0 con 0x7fb59415b1e0 2023-11-06T02:13:43.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.922+0000 7fb59a8b8640 1 --2- 172.21.15.49:0/3377969331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb59415b1e0 0x7fb59415cbb0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fb58401a0b0 tx=0x7fb58401acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:43.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.922+0000 7fb592ffd640 1 -- 172.21.15.49:0/3377969331 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5840b3600 con 0x7fb59415b1e0 2023-11-06T02:13:43.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.922+0000 7fb592ffd640 1 -- 172.21.15.49:0/3377969331 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5840b3780 con 0x7fb59415b1e0 2023-11-06T02:13:43.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.922+0000 7fb592ffd640 1 -- 172.21.15.49:0/3377969331 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5840b3b30 con 0x7fb59415b1e0 2023-11-06T02:13:43.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.922+0000 7fb59c31b640 1 -- 172.21.15.49:0/3377969331 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5940ccfe0 con 0x7fb59415b1e0 2023-11-06T02:13:43.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.922+0000 7fb59c31b640 1 -- 172.21.15.49:0/3377969331 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5941520a0 con 0x7fb59415b1e0 2023-11-06T02:13:43.924 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.922+0000 7fb592ffd640 1 -- 172.21.15.49:0/3377969331 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fb5840b3c90 con 0x7fb59415b1e0 2023-11-06T02:13:43.924 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.922+0000 7fb592ffd640 1 --2- 172.21.15.49:0/3377969331 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb580065a30 0x7fb580067ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:43.924 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.922+0000 7fb592ffd640 1 -- 172.21.15.49:0/3377969331 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb58411d670 con 0x7fb59415b1e0 2023-11-06T02:13:43.924 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.922+0000 7fb59a0b7640 1 --2- 172.21.15.49:0/3377969331 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb580065a30 0x7fb580067ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:43.930 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.922+0000 7fb59f443000 1 Processor -- start 2023-11-06T02:13:43.930 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.922+0000 7fb59f443000 1 -- start start 2023-11-06T02:13:43.930 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.922+0000 7fb59a0b7640 1 --2- 172.21.15.49:0/3377969331 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb580065a30 0x7fb580067ef0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fb58c0012f0 tx=0x7fb58c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:43.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.926+0000 7fb59f443000 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b9be3086e0 0x55b9be308ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:43.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.926+0000 7fb59f443000 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9be309000 0x55b9be2be550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:43.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.926+0000 7fb59f443000 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b9be2bea90 0x55b9be2bee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:43.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.926+0000 7fb59f443000 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9be186fc0 con 0x55b9be309000 2023-11-06T02:13:43.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.926+0000 7fb59f443000 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9be1151a0 con 0x55b9be2bea90 2023-11-06T02:13:43.932 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.926+0000 7fb59f443000 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b9be112770 con 0x55b9be3086e0 2023-11-06T02:13:43.932 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.926+0000 7fb5998b6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9be309000 0x55b9be2be550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:43.932 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.926+0000 7fb5998b6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9be309000 0x55b9be2be550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47954/0 (socket says 172.21.15.49:47954) 2023-11-06T02:13:43.932 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.926+0000 7fb5998b6640 1 -- 172.21.15.49:0/4068980294 learned_addr learned my addr 172.21.15.49:0/4068980294 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:43.932 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.926+0000 7fb59a0b7640 1 --2- 172.21.15.49:0/4068980294 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b9be3086e0 0x55b9be308ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:43.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.926+0000 7fb59a8b8640 1 --2- 172.21.15.49:0/4068980294 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b9be2bea90 0x55b9be2bee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:43.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.926+0000 7fb5998b6640 1 -- 172.21.15.49:0/4068980294 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b9be3086e0 msgr2=0x55b9be308ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:43.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.926+0000 7fb5998b6640 1 --2- 172.21.15.49:0/4068980294 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b9be3086e0 0x55b9be308ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:43.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.926+0000 7fb5998b6640 1 -- 172.21.15.49:0/4068980294 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b9be2bea90 msgr2=0x55b9be2bee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:43.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.926+0000 7fb5998b6640 1 --2- 172.21.15.49:0/4068980294 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b9be2bea90 0x55b9be2bee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:43.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.926+0000 7fb5998b6640 1 -- 172.21.15.49:0/4068980294 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b9bde0a890 con 0x55b9be309000 2023-11-06T02:13:43.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.926+0000 7fb5998b6640 1 --2- 172.21.15.49:0/4068980294 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9be309000 0x55b9be2be550 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fb588007780 tx=0x7fb588007c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3893dd7a8514723e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:43.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.926+0000 7fb57bfff640 1 -- 172.21.15.49:0/4068980294 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb588007e90 con 0x55b9be309000 2023-11-06T02:13:43.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.926+0000 7fb57bfff640 1 -- 172.21.15.49:0/4068980294 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb58800d950 con 0x55b9be309000 2023-11-06T02:13:43.935 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.926+0000 7fb57bfff640 1 -- 172.21.15.49:0/4068980294 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb58800dd00 con 0x55b9be309000 2023-11-06T02:13:43.935 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.926+0000 7fb59f443000 1 -- 172.21.15.49:0/4068980294 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9be309000 msgr2=0x55b9be2be550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:43.935 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.926+0000 7fb59f443000 1 --2- 172.21.15.49:0/4068980294 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9be309000 0x55b9be2be550 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fb588007780 tx=0x7fb588007c40 comp rx=0 tx=0).stop 2023-11-06T02:13:43.935 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.926+0000 7fb59f443000 1 -- 172.21.15.49:0/4068980294 shutdown_connections 2023-11-06T02:13:43.935 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.926+0000 7fb59f443000 1 --2- 172.21.15.49:0/4068980294 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9be309000 0x55b9be2be550 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:43.935 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.926+0000 7fb59f443000 1 --2- 172.21.15.49:0/4068980294 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b9be2bea90 0x55b9be2bee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:43.936 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.926+0000 7fb59f443000 1 --2- 172.21.15.49:0/4068980294 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b9be3086e0 0x55b9be308ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:43.936 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.926+0000 7fb59f443000 1 -- 172.21.15.49:0/4068980294 >> 172.21.15.49:0/4068980294 conn(0x55b9be2c4bf0 msgr2=0x55b9be2bce30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:43.936 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.926+0000 7fb59f443000 1 -- 172.21.15.49:0/4068980294 shutdown_connections 2023-11-06T02:13:43.936 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.926+0000 7fb59f443000 1 -- 172.21.15.49:0/4068980294 wait complete. 2023-11-06T02:13:43.936 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.926+0000 7fb59f443000 1 -- start start 2023-11-06T02:13:43.937 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.926+0000 7fb59f443000 1 Processor -- start 2023-11-06T02:13:43.937 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.926+0000 7fb59f443000 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b9be2bea90 0x55b9be2c9a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:43.937 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.926+0000 7fb59f443000 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9be3086e0 0x55b9be2c9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:43.937 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.926+0000 7fb59f443000 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b9be309000 0x55b9be2ca510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:43.937 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.926+0000 7fb59f443000 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9be10e060 con 0x55b9be3086e0 2023-11-06T02:13:43.938 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.926+0000 7fb59f443000 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9be0ad290 con 0x55b9be309000 2023-11-06T02:13:43.938 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.926+0000 7fb59f443000 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b9be0cb6f0 con 0x55b9be2bea90 2023-11-06T02:13:43.938 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.926+0000 7fb59f443000 10 client.0 _ll_register_callbacks cb 0x55b9be2aa1d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T02:13:43.938 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.926+0000 7fb5998b6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9be3086e0 0x55b9be2c9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:43.938 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.926+0000 7fb5998b6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9be3086e0 0x55b9be2c9fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47956/0 (socket says 172.21.15.49:47956) 2023-11-06T02:13:43.939 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.926+0000 7fb5998b6640 1 -- 172.21.15.49:0/2213389362 learned_addr learned my addr 172.21.15.49:0/2213389362 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:43.939 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.926+0000 7fb59a8b8640 1 --2- 172.21.15.49:0/2213389362 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b9be309000 0x55b9be2ca510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:43.939 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.926+0000 7fb59a0b7640 1 --2- 172.21.15.49:0/2213389362 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b9be2bea90 0x55b9be2c9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:43.939 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.926+0000 7fb5998b6640 1 -- 172.21.15.49:0/2213389362 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b9be2bea90 msgr2=0x55b9be2c9a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:43.939 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.926+0000 7fb5998b6640 1 --2- 172.21.15.49:0/2213389362 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b9be2bea90 0x55b9be2c9a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:43.939 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.926+0000 7fb5998b6640 1 -- 172.21.15.49:0/2213389362 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b9be309000 msgr2=0x55b9be2ca510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:43.940 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.926+0000 7fb5998b6640 1 --2- 172.21.15.49:0/2213389362 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b9be309000 0x55b9be2ca510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:43.940 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.926+0000 7fb5998b6640 1 -- 172.21.15.49:0/2213389362 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb58800e040 con 0x55b9be3086e0 2023-11-06T02:13:43.940 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.926+0000 7fb59a0b7640 1 --2- 172.21.15.49:0/2213389362 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b9be2bea90 0x55b9be2c9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:13:43.940 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.926+0000 7fb59a8b8640 1 --2- 172.21.15.49:0/2213389362 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b9be309000 0x55b9be2ca510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:13:43.940 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.930+0000 7fb5998b6640 1 --2- 172.21.15.49:0/2213389362 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9be3086e0 0x55b9be2c9fd0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x55b9be18ce40 tx=0x7fb588006e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=546f8366787acc56 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:43.941 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.930+0000 7fb5797fa640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-11-06T02:13:43.941 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.930+0000 7fb5797fa640 1 -- 172.21.15.49:0/2213389362 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb58800d950 con 0x55b9be3086e0 2023-11-06T02:13:43.941 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.930+0000 7fb59f443000 1 -- 172.21.15.49:0/2213389362 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b9be10c9b0 con 0x55b9be3086e0 2023-11-06T02:13:43.941 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.930+0000 7fb5797fa640 1 -- 172.21.15.49:0/2213389362 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb58800dab0 con 0x55b9be3086e0 2023-11-06T02:13:43.942 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.930+0000 7fb5797fa640 1 -- 172.21.15.49:0/2213389362 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb58800dc70 con 0x55b9be3086e0 2023-11-06T02:13:43.942 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.930+0000 7fb59f443000 10 client.7293 fetch_fsmap learned FSMap version 46 2023-11-06T02:13:43.942 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.930+0000 7fb59f443000 1 -- 172.21.15.49:0/2213389362 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) v3 -- 0x55b9be0d31e0 con 0x55b9be3086e0 2023-11-06T02:13:43.942 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.930+0000 7fb5797fa640 1 -- 172.21.15.49:0/2213389362 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b9be10c9b0 con 0x55b9be3086e0 2023-11-06T02:13:43.942 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.930+0000 7fb5797fa640 1 -- 172.21.15.49:0/2213389362 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 46) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fb588003620 con 0x55b9be3086e0 2023-11-06T02:13:43.942 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.930+0000 7fb59f443000 10 client.7293 fetch_fsmap finished waiting for FSMap version 46 2023-11-06T02:13:43.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.930+0000 7fb59f443000 10 client.7293 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-06T02:13:43.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.930+0000 7fb59f443000 20 client.7293 populate_metadata read hostname 'smithi049' 2023-11-06T02:13:43.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.930+0000 7fb59f443000 1 --2- 172.21.15.49:0/2213389362 >> [v2:172.21.15.49:6834/1130379293,v1:172.21.15.49:6835/1130379293] conn(0x55b9be2cadb0 0x55b9be319c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:43.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.930+0000 7fb59f443000 4 client.7293 mds_command: new command op to 7116 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T02:13:43.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.930+0000 7fb59f443000 1 -- 172.21.15.49:0/2213389362 --> [v2:172.21.15.49:6834/1130379293,v1:172.21.15.49:6835/1130379293] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b9be13d4d0 con 0x55b9be2cadb0 2023-11-06T02:13:43.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.930+0000 7fb59a0b7640 1 --2- 172.21.15.49:0/2213389362 >> [v2:172.21.15.49:6834/1130379293,v1:172.21.15.49:6835/1130379293] conn(0x55b9be2cadb0 0x55b9be319c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:43.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.930+0000 7fb5797fa640 10 client.7293 ms_handle_connect on v2:172.21.15.49:6834/1130379293 2023-11-06T02:13:43.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.930+0000 7fb59a0b7640 1 --2- 172.21.15.49:0/2213389362 >> [v2:172.21.15.49:6834/1130379293,v1:172.21.15.49:6835/1130379293] conn(0x55b9be2cadb0 0x55b9be319c50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=832f37cd95369c02 server_cookie=f6b9a4ce51031e68 in_seq=0 out_seq=0 2023-11-06T02:13:43.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.930+0000 7fb5797fa640 1 -- 172.21.15.49:0/2213389362 <== mds.0 v2:172.21.15.49:6834/1130379293 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55b9be13d4d0 con 0x55b9be2cadb0 2023-11-06T02:13:43.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.930+0000 7fb5797fa640 10 client.7293 handle_command_reply: tid=0 2023-11-06T02:13:43.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.930+0000 7fb59f443000 1 client.7293 shutdown 2023-11-06T02:13:43.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.930+0000 7fb59f443000 1 -- 172.21.15.49:0/2213389362 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9be3086e0 msgr2=0x55b9be2c9fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:43.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.930+0000 7fb59f443000 1 --2- 172.21.15.49:0/2213389362 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9be3086e0 0x55b9be2c9fd0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x55b9be18ce40 tx=0x7fb588006e20 comp rx=0 tx=0).stop 2023-11-06T02:13:43.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.930+0000 7fb59f443000 1 -- 172.21.15.49:0/2213389362 shutdown_connections 2023-11-06T02:13:43.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.930+0000 7fb59f443000 1 --2- 172.21.15.49:0/2213389362 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9be3086e0 0x55b9be2c9fd0 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:43.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.930+0000 7fb59f443000 1 --2- 172.21.15.49:0/2213389362 >> [v2:172.21.15.49:6834/1130379293,v1:172.21.15.49:6835/1130379293] conn(0x55b9be2cadb0 0x55b9be319c50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:43.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.930+0000 7fb59f443000 1 --2- 172.21.15.49:0/2213389362 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b9be309000 0x55b9be2ca510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:43.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.930+0000 7fb59f443000 1 --2- 172.21.15.49:0/2213389362 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b9be2bea90 0x55b9be2c9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:43.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.930+0000 7fb5797fa640 0 client.7293 ms_handle_reset on v2:172.21.15.49:6834/1130379293 2023-11-06T02:13:43.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.930+0000 7fb59f443000 1 -- 172.21.15.49:0/2213389362 >> 172.21.15.49:0/2213389362 conn(0x55b9be2c4bf0 msgr2=0x55b9be2bd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:43.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.930+0000 7fb59f443000 1 -- 172.21.15.49:0/2213389362 shutdown_connections 2023-11-06T02:13:43.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.930+0000 7fb59f443000 1 -- 172.21.15.49:0/2213389362 wait complete. 2023-11-06T02:13:43.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.930+0000 7fb59f443000 20 client.7293 trim_cache size 0 max 16384 2023-11-06T02:13:43.952 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.946+0000 7fb59f443000 1 Processor -- start 2023-11-06T02:13:43.953 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.946+0000 7fb59f443000 1 -- start start 2023-11-06T02:13:43.953 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.946+0000 7fb59f443000 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9be2bea90 0x55b9be31b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:43.953 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.946+0000 7fb59f443000 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b9be3086e0 0x55b9be31b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:43.953 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.946+0000 7fb59f443000 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b9be309000 0x55b9be301340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:43.953 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.946+0000 7fb59f443000 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9bddcb940 con 0x55b9be2bea90 2023-11-06T02:13:43.953 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.946+0000 7fb59f443000 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9be2b62a0 con 0x55b9be3086e0 2023-11-06T02:13:43.954 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.946+0000 7fb59f443000 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b9be151560 con 0x55b9be309000 2023-11-06T02:13:43.954 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.946+0000 7fb59a0b7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9be2bea90 0x55b9be31b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:43.954 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.946+0000 7fb59a0b7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9be2bea90 0x55b9be31b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47970/0 (socket says 172.21.15.49:47970) 2023-11-06T02:13:43.954 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.946+0000 7fb59a0b7640 1 -- 172.21.15.49:0/1038173271 learned_addr learned my addr 172.21.15.49:0/1038173271 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:43.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.946+0000 7fb5998b6640 1 --2- 172.21.15.49:0/1038173271 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b9be3086e0 0x55b9be31b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:43.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.946+0000 7fb59a0b7640 1 -- 172.21.15.49:0/1038173271 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b9be309000 msgr2=0x55b9be301340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:43.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.946+0000 7fb59a0b7640 1 --2- 172.21.15.49:0/1038173271 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b9be309000 0x55b9be301340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:43.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.946+0000 7fb59a0b7640 1 -- 172.21.15.49:0/1038173271 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b9be3086e0 msgr2=0x55b9be31b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:43.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.946+0000 7fb59a0b7640 1 --2- 172.21.15.49:0/1038173271 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b9be3086e0 0x55b9be31b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:43.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.946+0000 7fb59a0b7640 1 -- 172.21.15.49:0/1038173271 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b9be0b9330 con 0x55b9be2bea90 2023-11-06T02:13:43.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.946+0000 7fb5998b6640 1 --2- 172.21.15.49:0/1038173271 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b9be3086e0 0x55b9be31b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:13:43.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.946+0000 7fb59a0b7640 1 --2- 172.21.15.49:0/1038173271 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9be2bea90 0x55b9be31b370 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fb58c003b10 tx=0x7fb58c017040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ed33091dae91e40 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:43.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.946+0000 7fb590ff9640 1 -- 172.21.15.49:0/1038173271 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb58c003ea0 con 0x55b9be2bea90 2023-11-06T02:13:43.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.946+0000 7fb590ff9640 1 -- 172.21.15.49:0/1038173271 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb58c027070 con 0x55b9be2bea90 2023-11-06T02:13:43.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.946+0000 7fb590ff9640 1 -- 172.21.15.49:0/1038173271 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb58c0055d0 con 0x55b9be2bea90 2023-11-06T02:13:43.957 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.946+0000 7fb59f443000 1 -- 172.21.15.49:0/1038173271 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9be2bea90 msgr2=0x55b9be31b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:43.957 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.946+0000 7fb59f443000 1 --2- 172.21.15.49:0/1038173271 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9be2bea90 0x55b9be31b370 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fb58c003b10 tx=0x7fb58c017040 comp rx=0 tx=0).stop 2023-11-06T02:13:43.957 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.946+0000 7fb59f443000 1 -- 172.21.15.49:0/1038173271 shutdown_connections 2023-11-06T02:13:43.957 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.946+0000 7fb59f443000 1 --2- 172.21.15.49:0/1038173271 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b9be309000 0x55b9be301340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:43.957 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.946+0000 7fb59f443000 1 --2- 172.21.15.49:0/1038173271 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b9be3086e0 0x55b9be31b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:43.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.946+0000 7fb59f443000 1 --2- 172.21.15.49:0/1038173271 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9be2bea90 0x55b9be31b370 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:43.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.946+0000 7fb59f443000 1 -- 172.21.15.49:0/1038173271 >> 172.21.15.49:0/1038173271 conn(0x55b9be2c4bf0 msgr2=0x55b9be319c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:43.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.946+0000 7fb59f443000 1 -- 172.21.15.49:0/1038173271 shutdown_connections 2023-11-06T02:13:43.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.946+0000 7fb59f443000 1 -- 172.21.15.49:0/1038173271 wait complete. 2023-11-06T02:13:43.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.950+0000 7fb59f443000 1 -- start start 2023-11-06T02:13:43.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.950+0000 7fb59f443000 1 Processor -- start 2023-11-06T02:13:43.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.950+0000 7fb59f443000 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b9be2bea90 0x55b9be365140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:43.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.950+0000 7fb59f443000 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b9be3086e0 0x55b9be365680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:43.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.950+0000 7fb59f443000 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9be309000 0x55b9be304520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:43.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.950+0000 7fb59f443000 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9be2c1a80 con 0x55b9be309000 2023-11-06T02:13:43.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.950+0000 7fb59f443000 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9be046a60 con 0x55b9be3086e0 2023-11-06T02:13:43.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.950+0000 7fb59f443000 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b9be2c2120 con 0x55b9be2bea90 2023-11-06T02:13:43.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.950+0000 7fb59f443000 10 client.0 _ll_register_callbacks cb 0x55b9be28a9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T02:13:43.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.950+0000 7fb59a8b8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9be309000 0x55b9be304520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:43.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.950+0000 7fb59a8b8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9be309000 0x55b9be304520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47980/0 (socket says 172.21.15.49:47980) 2023-11-06T02:13:43.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.950+0000 7fb59a8b8640 1 -- 172.21.15.49:0/1031416696 learned_addr learned my addr 172.21.15.49:0/1031416696 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:43.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.950+0000 7fb59a0b7640 1 --2- 172.21.15.49:0/1031416696 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b9be2bea90 0x55b9be365140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:43.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.950+0000 7fb5998b6640 1 --2- 172.21.15.49:0/1031416696 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b9be3086e0 0x55b9be365680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:43.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.950+0000 7fb59a8b8640 1 -- 172.21.15.49:0/1031416696 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b9be2bea90 msgr2=0x55b9be365140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:43.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.950+0000 7fb59a8b8640 1 --2- 172.21.15.49:0/1031416696 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b9be2bea90 0x55b9be365140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:43.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.950+0000 7fb59a8b8640 1 -- 172.21.15.49:0/1031416696 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b9be3086e0 msgr2=0x55b9be365680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:43.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.950+0000 7fb59a8b8640 1 --2- 172.21.15.49:0/1031416696 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b9be3086e0 0x55b9be365680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:43.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.950+0000 7fb59a8b8640 1 -- 172.21.15.49:0/1031416696 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb59415b650 con 0x55b9be309000 2023-11-06T02:13:43.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.950+0000 7fb5998b6640 1 --2- 172.21.15.49:0/1031416696 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b9be3086e0 0x55b9be365680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:13:43.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.950+0000 7fb59a8b8640 1 --2- 172.21.15.49:0/1031416696 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9be309000 0x55b9be304520 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fb5840eeb00 tx=0x7fb5840eeb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e849e3ccf4f180aa server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:43.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.950+0000 7fb5797fa640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-11-06T02:13:43.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.950+0000 7fb5797fa640 1 -- 172.21.15.49:0/1031416696 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5840a6020 con 0x55b9be309000 2023-11-06T02:13:43.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.950+0000 7fb5797fa640 1 -- 172.21.15.49:0/1031416696 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb58411be40 con 0x55b9be309000 2023-11-06T02:13:43.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.950+0000 7fb59f443000 1 -- 172.21.15.49:0/1031416696 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b9be0ddb90 con 0x55b9be309000 2023-11-06T02:13:43.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.950+0000 7fb5797fa640 1 -- 172.21.15.49:0/1031416696 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb5840f6b50 con 0x55b9be309000 2023-11-06T02:13:43.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.950+0000 7fb59f443000 10 client.7299 fetch_fsmap learned FSMap version 46 2023-11-06T02:13:43.964 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.950+0000 7fb59f443000 1 -- 172.21.15.49:0/1031416696 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) v3 -- 0x55b9bde0acc0 con 0x55b9be309000 2023-11-06T02:13:43.964 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.950+0000 7fb5797fa640 1 -- 172.21.15.49:0/1031416696 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b9be0ddb90 con 0x55b9be309000 2023-11-06T02:13:43.964 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.950+0000 7fb5797fa640 1 -- 172.21.15.49:0/1031416696 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 46) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fb5840efb80 con 0x55b9be309000 2023-11-06T02:13:43.964 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.950+0000 7fb59f443000 10 client.7299 fetch_fsmap finished waiting for FSMap version 46 2023-11-06T02:13:43.964 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.950+0000 7fb59f443000 10 client.7299 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-06T02:13:43.965 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.950+0000 7fb59f443000 20 client.7299 populate_metadata read hostname 'smithi049' 2023-11-06T02:13:43.965 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.950+0000 7fb59f443000 1 --2- 172.21.15.49:0/1031416696 >> [v2:172.21.15.49:6834/1130379293,v1:172.21.15.49:6835/1130379293] conn(0x55b9be304a60 0x55b9be304e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:43.965 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.950+0000 7fb59f443000 4 client.7299 mds_command: new command op to 7116 tid=0[{"prefix": "status"}] 2023-11-06T02:13:43.965 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.950+0000 7fb59f443000 1 -- 172.21.15.49:0/1031416696 --> [v2:172.21.15.49:6834/1130379293,v1:172.21.15.49:6835/1130379293] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b9be1070e0 con 0x55b9be304a60 2023-11-06T02:13:43.965 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.950+0000 7fb59a0b7640 1 --2- 172.21.15.49:0/1031416696 >> [v2:172.21.15.49:6834/1130379293,v1:172.21.15.49:6835/1130379293] conn(0x55b9be304a60 0x55b9be304e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:43.966 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.950+0000 7fb5797fa640 10 client.7299 ms_handle_connect on v2:172.21.15.49:6834/1130379293 2023-11-06T02:13:43.966 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.950+0000 7fb59a0b7640 1 --2- 172.21.15.49:0/1031416696 >> [v2:172.21.15.49:6834/1130379293,v1:172.21.15.49:6835/1130379293] conn(0x55b9be304a60 0x55b9be304e40 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c7277f46c7a81a9 server_cookie=df1fb3fbb8477034 in_seq=0 out_seq=0 2023-11-06T02:13:43.966 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.950+0000 7fb5797fa640 1 -- 172.21.15.49:0/1031416696 <== mds.0 v2:172.21.15.49:6834/1130379293 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55b9be1070e0 con 0x55b9be304a60 2023-11-06T02:13:43.966 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.950+0000 7fb5797fa640 10 client.7299 handle_command_reply: tid=0 2023-11-06T02:13:43.966 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.950+0000 7fb59f443000 1 client.7299 shutdown 2023-11-06T02:13:43.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.950+0000 7fb59f443000 1 -- 172.21.15.49:0/1031416696 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9be309000 msgr2=0x55b9be304520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:43.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.950+0000 7fb59f443000 1 --2- 172.21.15.49:0/1031416696 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9be309000 0x55b9be304520 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fb5840eeb00 tx=0x7fb5840eeb70 comp rx=0 tx=0).stop 2023-11-06T02:13:43.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.950+0000 7fb59f443000 1 -- 172.21.15.49:0/1031416696 shutdown_connections 2023-11-06T02:13:43.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.950+0000 7fb59f443000 1 --2- 172.21.15.49:0/1031416696 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9be309000 0x55b9be304520 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:43.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.950+0000 7fb59f443000 1 --2- 172.21.15.49:0/1031416696 >> [v2:172.21.15.49:6834/1130379293,v1:172.21.15.49:6835/1130379293] conn(0x55b9be304a60 0x55b9be304e40 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:43.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.950+0000 7fb59f443000 1 --2- 172.21.15.49:0/1031416696 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b9be3086e0 0x55b9be365680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:43.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.950+0000 7fb59f443000 1 --2- 172.21.15.49:0/1031416696 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b9be2bea90 0x55b9be365140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:43.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.950+0000 7fb5797fa640 0 client.7299 ms_handle_reset on v2:172.21.15.49:6834/1130379293 2023-11-06T02:13:43.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.950+0000 7fb59f443000 1 -- 172.21.15.49:0/1031416696 >> 172.21.15.49:0/1031416696 conn(0x55b9be2c4bf0 msgr2=0x55b9be319c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:43.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.950+0000 7fb59f443000 1 -- 172.21.15.49:0/1031416696 shutdown_connections 2023-11-06T02:13:43.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.950+0000 7fb59f443000 1 -- 172.21.15.49:0/1031416696 wait complete. 2023-11-06T02:13:43.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.950+0000 7fb59f443000 20 client.7299 trim_cache size 0 max 16384 2023-11-06T02:13:43.969 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-11-06T02:13:43.969 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_fsid": "1a8b035c-6ba4-499a-b876-aea47fd9c0d4", 2023-11-06T02:13:43.970 INFO:teuthology.orchestra.run.smithi049.stdout: "whoami": 0, 2023-11-06T02:13:43.970 INFO:teuthology.orchestra.run.smithi049.stdout: "id": 7116, 2023-11-06T02:13:43.970 INFO:teuthology.orchestra.run.smithi049.stdout: "want_state": "up:active", 2023-11-06T02:13:43.970 INFO:teuthology.orchestra.run.smithi049.stdout: "state": "up:active", 2023-11-06T02:13:43.971 INFO:teuthology.orchestra.run.smithi049.stdout: "fs_name": "cephfs", 2023-11-06T02:13:43.971 INFO:teuthology.orchestra.run.smithi049.stdout: "rank_uptime": 10.324390953, 2023-11-06T02:13:43.971 INFO:teuthology.orchestra.run.smithi049.stdout: "mdsmap_epoch": 46, 2023-11-06T02:13:43.971 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch": 80, 2023-11-06T02:13:43.971 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch_barrier": 80, 2023-11-06T02:13:43.971 INFO:teuthology.orchestra.run.smithi049.stdout: "uptime": 15.521770163999999 2023-11-06T02:13:43.972 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-11-06T02:13:43.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.954+0000 7fb59c31b640 1 -- 172.21.15.49:0/3377969331 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb580065a30 msgr2=0x7fb580067ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:43.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.954+0000 7fb59c31b640 1 --2- 172.21.15.49:0/3377969331 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb580065a30 0x7fb580067ef0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fb58c0012f0 tx=0x7fb58c009040 comp rx=0 tx=0).stop 2023-11-06T02:13:43.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.954+0000 7fb59c31b640 1 -- 172.21.15.49:0/3377969331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb59415b1e0 msgr2=0x7fb59415cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:43.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.954+0000 7fb59c31b640 1 --2- 172.21.15.49:0/3377969331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb59415b1e0 0x7fb59415cbb0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fb58401a0b0 tx=0x7fb58401acc0 comp rx=0 tx=0).stop 2023-11-06T02:13:43.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.954+0000 7fb59c31b640 1 -- 172.21.15.49:0/3377969331 shutdown_connections 2023-11-06T02:13:43.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.954+0000 7fb59c31b640 1 --2- 172.21.15.49:0/3377969331 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb580065a30 0x7fb580067ef0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:43.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.954+0000 7fb59c31b640 1 --2- 172.21.15.49:0/3377969331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb59415b1e0 0x7fb59415cbb0 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:43.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.954+0000 7fb59c31b640 1 --2- 172.21.15.49:0/3377969331 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb59415a8c0 0x7fb594151b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:43.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.954+0000 7fb59c31b640 1 --2- 172.21.15.49:0/3377969331 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb5940ca110 0x7fb5940cd7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:43.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.954+0000 7fb59c31b640 1 -- 172.21.15.49:0/3377969331 >> 172.21.15.49:0/3377969331 conn(0x7fb5940b0bd0 msgr2=0x7fb5940c9110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:43.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.954+0000 7fb59c31b640 1 -- 172.21.15.49:0/3377969331 shutdown_connections 2023-11-06T02:13:43.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:43.954+0000 7fb59c31b640 1 -- 172.21.15.49:0/3377969331 wait complete. 2023-11-06T02:13:43.976 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-06T02:13:44.091 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:44.094+0000 7f617d8f1640 1 Processor -- start 2023-11-06T02:13:44.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:44.094+0000 7f617d8f1640 1 -- start start 2023-11-06T02:13:44.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:44.098+0000 7f617d8f1640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f61780ca980 0x7f61781644c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:44.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:44.098+0000 7f617d8f1640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61780cad90 con 0x7f6178159b40 2023-11-06T02:13:44.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:44.098+0000 7f617d8f1640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61780723a0 con 0x7f61780cb010 2023-11-06T02:13:44.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:44.098+0000 7f617d8f1640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61780715e0 con 0x7f61780ca980 2023-11-06T02:13:44.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:44.098+0000 7f61777fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f61780ca980 0x7f61781644c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:44.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:44.098+0000 7f61767fc640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f6178159b40 0x7f61780cdd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:60860/0 (socket says 172.21.15.49:60860) 2023-11-06T02:13:44.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:44.098+0000 7f61767fc640 1 -- 172.21.15.49:0/778244274 learned_addr learned my addr 172.21.15.49:0/778244274 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:13:44.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:44.098+0000 7f61777fe640 1 -- 172.21.15.49:0/778244274 >> v1:172.21.15.150:6789/0 conn(0x7f61780cb010 legacy=0x7f61780cb3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:44.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:44.098+0000 7f61777fe640 1 -- 172.21.15.49:0/778244274 >> v1:172.21.15.49:6789/0 conn(0x7f6178159b40 legacy=0x7f61780cdd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:44.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:44.098+0000 7f6175ffb640 1 -- 172.21.15.49:0/778244274 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3079424989 0 0) 0x7f61780cad90 con 0x7f6178159b40 2023-11-06T02:13:44.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:44.098+0000 7f61777fe640 1 -- 172.21.15.49:0/778244274 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61780718c0 con 0x7f61780ca980 2023-11-06T02:13:44.096 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:44.102+0000 7f61777fe640 1 --2- 172.21.15.49:0/778244274 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61780ca980 0x7f61781644c0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f61700097c0 tx=0x7f617001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fac1a6a494442c66 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:44.096 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:44.102+0000 7f6175ffb640 1 -- 172.21.15.49:0/778244274 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61700ab650 con 0x7f61780ca980 2023-11-06T02:13:44.097 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:44.102+0000 7f6175ffb640 1 -- 172.21.15.49:0/778244274 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61700ab7b0 con 0x7f61780ca980 2023-11-06T02:13:44.097 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:44.102+0000 7f6175ffb640 1 -- 172.21.15.49:0/778244274 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61700abb40 con 0x7f61780ca980 2023-11-06T02:13:44.098 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:44.102+0000 7f617d8f1640 1 -- 172.21.15.49:0/778244274 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61780ca980 msgr2=0x7f61781644c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:44.098 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:44.102+0000 7f617d8f1640 1 --2- 172.21.15.49:0/778244274 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61780ca980 0x7f61781644c0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f61700097c0 tx=0x7f617001c450 comp rx=0 tx=0).stop 2023-11-06T02:13:44.098 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:44.102+0000 7f617d8f1640 1 -- 172.21.15.49:0/778244274 shutdown_connections 2023-11-06T02:13:44.098 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:44.102+0000 7f617d8f1640 1 --2- 172.21.15.49:0/778244274 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61780ca980 0x7f61781644c0 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:44.099 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:44.102+0000 7f617d8f1640 1 -- 172.21.15.49:0/778244274 >> 172.21.15.49:0/778244274 conn(0x7f617806c0a0 msgr2=0x7f617806c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:44.099 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:44.102+0000 7f617d8f1640 1 -- 172.21.15.49:0/778244274 shutdown_connections 2023-11-06T02:13:44.099 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:44.102+0000 7f617d8f1640 1 -- 172.21.15.49:0/778244274 wait complete. 2023-11-06T02:13:44.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:44.102+0000 7f617d8f1640 1 Processor -- start 2023-11-06T02:13:44.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:44.102+0000 7f617d8f1640 1 -- start start 2023-11-06T02:13:44.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:44.102+0000 7f617d8f1640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61780ca980 0x7f61780c4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:44.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:44.102+0000 7f617d8f1640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61780cb010 0x7f6178158530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:44.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:44.102+0000 7f617d8f1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6178159b40 0x7f61780c56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:44.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:44.102+0000 7f617d8f1640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6178158a70 con 0x7f6178159b40 2023-11-06T02:13:44.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:44.102+0000 7f617d8f1640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61780c8a10 con 0x7f61780cb010 2023-11-06T02:13:44.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:44.102+0000 7f617d8f1640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61780c5c10 con 0x7f61780ca980 2023-11-06T02:13:44.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:44.102+0000 7f61777fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6178159b40 0x7f61780c56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:44.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:44.102+0000 7f61777fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6178159b40 0x7f61780c56d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47998/0 (socket says 172.21.15.49:47998) 2023-11-06T02:13:44.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:44.102+0000 7f61777fe640 1 -- 172.21.15.49:0/65239838 learned_addr learned my addr 172.21.15.49:0/65239838 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:44.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:44.102+0000 7f61767fc640 1 --2- 172.21.15.49:0/65239838 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61780cb010 0x7f6178158530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:44.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:44.102+0000 7f6176ffd640 1 --2- 172.21.15.49:0/65239838 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61780ca980 0x7f61780c4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:44.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:44.102+0000 7f61777fe640 1 -- 172.21.15.49:0/65239838 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61780ca980 msgr2=0x7f61780c4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:44.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:44.102+0000 7f61777fe640 1 --2- 172.21.15.49:0/65239838 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61780ca980 0x7f61780c4cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:44.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:44.102+0000 7f61777fe640 1 -- 172.21.15.49:0/65239838 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61780cb010 msgr2=0x7f6178158530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:44.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:44.102+0000 7f61777fe640 1 --2- 172.21.15.49:0/65239838 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61780cb010 0x7f6178158530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:44.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:44.102+0000 7f61777fe640 1 -- 172.21.15.49:0/65239838 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6170009420 con 0x7f6178159b40 2023-11-06T02:13:44.105 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:44.106+0000 7f61777fe640 1 --2- 172.21.15.49:0/65239838 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6178159b40 0x7f61780c56d0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f61700ad040 tx=0x7f617001ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:44.105 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:44.106+0000 7f616ffff640 1 -- 172.21.15.49:0/65239838 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61700b5810 con 0x7f6178159b40 2023-11-06T02:13:44.105 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:44.106+0000 7f616ffff640 1 -- 172.21.15.49:0/65239838 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61700b5970 con 0x7f6178159b40 2023-11-06T02:13:44.105 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:44.106+0000 7f617d8f1640 1 -- 172.21.15.49:0/65239838 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61780c80e0 con 0x7f6178159b40 2023-11-06T02:13:44.105 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:44.106+0000 7f616ffff640 1 -- 172.21.15.49:0/65239838 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61700b5d00 con 0x7f6178159b40 2023-11-06T02:13:44.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:44.106+0000 7f617d8f1640 1 -- 172.21.15.49:0/65239838 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61780c86a0 con 0x7f6178159b40 2023-11-06T02:13:44.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:44.106+0000 7f617d8f1640 1 -- 172.21.15.49:0/65239838 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6178159f80 con 0x7f6178159b40 2023-11-06T02:13:44.107 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:44.106+0000 7f616ffff640 1 -- 172.21.15.49:0/65239838 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f61700b4390 con 0x7f6178159b40 2023-11-06T02:13:44.107 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:44.106+0000 7f616ffff640 1 --2- 172.21.15.49:0/65239838 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6158065a60 0x7f6158067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:44.107 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:44.106+0000 7f616ffff640 1 -- 172.21.15.49:0/65239838 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f617011df30 con 0x7f6178159b40 2023-11-06T02:13:44.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:44.106+0000 7f6176ffd640 1 --2- 172.21.15.49:0/65239838 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6158065a60 0x7f6158067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:44.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:44.106+0000 7f6176ffd640 1 --2- 172.21.15.49:0/65239838 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6158065a60 0x7f6158067f20 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f6168007fd0 tx=0x7f6168009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:44.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:44.110+0000 7f616ffff640 1 -- 172.21.15.49:0/65239838 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f6178159f80 con 0x7f6178159b40 2023-11-06T02:13:44.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:44.298+0000 7f617d8f1640 1 -- 172.21.15.49:0/65239838 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f6178159f80 con 0x7f6178159b40 2023-11-06T02:13:44.293 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:44.298+0000 7f616ffff640 1 -- 172.21.15.49:0/65239838 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v46) v1 ==== 53+0+73 (secure 0 0 0) 0x7f6178159f80 con 0x7f6178159b40 2023-11-06T02:13:44.293 INFO:teuthology.orchestra.run.smithi049.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-06T02:13:44.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:44.298+0000 7f617d8f1640 1 -- 172.21.15.49:0/65239838 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6158065a60 msgr2=0x7f6158067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:44.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:44.298+0000 7f617d8f1640 1 --2- 172.21.15.49:0/65239838 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6158065a60 0x7f6158067f20 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f6168007fd0 tx=0x7f6168009040 comp rx=0 tx=0).stop 2023-11-06T02:13:44.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:44.298+0000 7f617d8f1640 1 -- 172.21.15.49:0/65239838 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6178159b40 msgr2=0x7f61780c56d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:44.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:44.298+0000 7f617d8f1640 1 --2- 172.21.15.49:0/65239838 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6178159b40 0x7f61780c56d0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f61700ad040 tx=0x7f617001ac90 comp rx=0 tx=0).stop 2023-11-06T02:13:44.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:44.298+0000 7f617d8f1640 1 -- 172.21.15.49:0/65239838 shutdown_connections 2023-11-06T02:13:44.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:44.298+0000 7f617d8f1640 1 --2- 172.21.15.49:0/65239838 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6158065a60 0x7f6158067f20 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:44.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:44.298+0000 7f617d8f1640 1 --2- 172.21.15.49:0/65239838 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6178159b40 0x7f61780c56d0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:44.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:44.298+0000 7f617d8f1640 1 --2- 172.21.15.49:0/65239838 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61780cb010 0x7f6178158530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:44.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:44.298+0000 7f617d8f1640 1 --2- 172.21.15.49:0/65239838 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61780ca980 0x7f61780c4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:44.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:44.298+0000 7f617d8f1640 1 -- 172.21.15.49:0/65239838 >> 172.21.15.49:0/65239838 conn(0x7f617806c0a0 msgr2=0x7f61780ce600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:44.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:44.298+0000 7f617d8f1640 1 -- 172.21.15.49:0/65239838 shutdown_connections 2023-11-06T02:13:44.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:44.298+0000 7f617d8f1640 1 -- 172.21.15.49:0/65239838 wait complete. 2023-11-06T02:13:44.311 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-06T02:13:44.311 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-06T02:13:44.312 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi049.front.sepia.ceph.com 2023-11-06T02:13:44.312 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-06T02:13:44.312 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-06T02:13:44.313 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-06T02:13:44.313 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-06T02:13:44.313 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-06T02:13:44.314 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-06T02:13:44.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:13:44.314 DEBUG:teuthology.orchestra.run.smithi049:> ip addr 2023-11-06T02:13:44.322 INFO:teuthology.orchestra.run.smithi049.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-06T02:13:44.323 INFO:teuthology.orchestra.run.smithi049.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-06T02:13:44.323 INFO:teuthology.orchestra.run.smithi049.stdout: inet 127.0.0.1/8 scope host lo 2023-11-06T02:13:44.323 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:13:44.323 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 ::1/128 scope host 2023-11-06T02:13:44.324 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:13:44.324 INFO:teuthology.orchestra.run.smithi049.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-06T02:13:44.324 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:6b:ff:84 brd ff:ff:ff:ff:ff:ff 2023-11-06T02:13:44.324 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp7s0f0 2023-11-06T02:13:44.324 INFO:teuthology.orchestra.run.smithi049.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-06T02:13:44.325 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:6b:ff:85 brd ff:ff:ff:ff:ff:ff 2023-11-06T02:13:44.325 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp7s0f1 2023-11-06T02:13:44.325 INFO:teuthology.orchestra.run.smithi049.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-06T02:13:44.325 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:bd:15:66 brd ff:ff:ff:ff:ff:ff 2023-11-06T02:13:44.326 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp1s0f0 2023-11-06T02:13:44.326 INFO:teuthology.orchestra.run.smithi049.stdout: inet 172.21.15.49/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-11-06T02:13:44.326 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft 42173sec preferred_lft 42173sec 2023-11-06T02:13:44.326 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 fe80::ec4:7aff:febd:1566/64 scope link 2023-11-06T02:13:44.326 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:13:44.327 INFO:teuthology.orchestra.run.smithi049.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-06T02:13:44.327 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:bd:15:67 brd ff:ff:ff:ff:ff:ff 2023-11-06T02:13:44.327 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp1s0f1 2023-11-06T02:13:44.327 INFO:teuthology.orchestra.run.smithi049.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-06T02:13:44.327 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 02:42:47:49:33:69 brd ff:ff:ff:ff:ff:ff 2023-11-06T02:13:44.328 INFO:teuthology.orchestra.run.smithi049.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-06T02:13:44.328 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:13:44.328 INFO:teuthology.orchestra.run.smithi049.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-06T02:13:44.328 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-11-06T02:13:44.328 INFO:teuthology.orchestra.run.smithi049.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-06T02:13:44.329 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:13:44.329 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-11-06T02:13:44.329 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:13:44.329 INFO:teuthology.orchestra.run.smithi049.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-06T02:13:44.330 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-06T02:13:44.330 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-11-06T02:13:44.330 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:13:44.330 INFO:teuthology.orchestra.run.smithi049.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-06T02:13:44.330 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-06T02:13:44.331 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-11-06T02:13:44.331 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:13:44.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:13:44.332 DEBUG:teuthology.orchestra.run.smithi049:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-06T02:13:44.378 INFO:teuthology.orchestra.run.smithi049.stdout:1 2023-11-06T02:13:44.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:13:44.380 DEBUG:teuthology.orchestra.run.smithi049:> route 2023-11-06T02:13:44.430 INFO:teuthology.orchestra.run.smithi049.stdout:Kernel IP routing table 2023-11-06T02:13:44.430 INFO:teuthology.orchestra.run.smithi049.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-06T02:13:44.430 INFO:teuthology.orchestra.run.smithi049.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-11-06T02:13:44.430 INFO:teuthology.orchestra.run.smithi049.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-06T02:13:44.431 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-11-06T02:13:44.431 INFO:teuthology.orchestra.run.smithi049.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-06T02:13:44.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:13:44.432 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec bash -c ' 2023-11-06T02:13:44.432 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2023-11-06T02:13:44.432 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-06T02:13:44.433 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-06T02:13:44.433 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-06T02:13:44.434 DEBUG:teuthology.orchestra.run.smithi049:> ') 2023-11-06T02:13:44.503 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:13:44.504 DEBUG:teuthology.orchestra.run.smithi049:> ip netns list 2023-11-06T02:13:44.553 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-06T02:13:44.553 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-06T02:13:44.554 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-06T02:13:44.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:13:44.555 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link set brx.0 up 2023-11-06T02:13:44.609 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-06T02:13:44.609 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-06T02:13:44.609 DEBUG:teuthology.orchestra.run.smithi049:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-06T02:13:44.657 INFO:teuthology.orchestra.run.smithi049.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-06T02:13:44.658 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-06T02:13:44.658 DEBUG:teuthology.orchestra.run.smithi049:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-06T02:13:44.704 DEBUG:teuthology.orchestra.run.smithi049:> sudo modprobe fuse 2023-11-06T02:13:44.762 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-06T02:13:44.763 DEBUG:teuthology.orchestra.run.smithi049:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-06T02:13:44.815 INFO:teuthology.orchestra.run.smithi049.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-06T02:13:44.816 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-06T02:13:44.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:13:44.818 DEBUG:teuthology.orchestra.run.smithi049:> ls /sys/fs/fuse/connections 2023-11-06T02:13:44.865 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-06T02:13:44.865 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-06T02:13:44.910 DEBUG:teuthology.orchestra.run.smithi049:> sudo modprobe fuse 2023-11-06T02:13:44.925 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-06T02:13:44.925 DEBUG:teuthology.orchestra.run.smithi049:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-06T02:13:44.978 INFO:teuthology.orchestra.run.smithi049.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-06T02:13:44.979 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-06T02:13:44.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:13:44.980 DEBUG:teuthology.orchestra.run.smithi049:> ls /sys/fs/fuse/connections 2023-11-06T02:13:44.987 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi049.stderr:2023-11-06T02:13:44.990+0000 7fb448b99580 -1 init, newargv = 0x563a3f244c70 newargc=15 2023-11-06T02:13:44.988 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi049.stderr:ceph-fuse[54957]: starting ceph client 2023-11-06T02:13:44.995 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi049.stderr:ceph-fuse[54957]: starting fuse 2023-11-06T02:13:46.093 DEBUG:teuthology.orchestra.run.smithi049:> sudo modprobe fuse 2023-11-06T02:13:46.110 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-06T02:13:46.111 DEBUG:teuthology.orchestra.run.smithi049:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-06T02:13:46.164 INFO:teuthology.orchestra.run.smithi049.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-06T02:13:46.165 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-06T02:13:46.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:13:46.166 DEBUG:teuthology.orchestra.run.smithi049:> ls /sys/fs/fuse/connections 2023-11-06T02:13:46.214 INFO:teuthology.orchestra.run.smithi049.stdout:45 2023-11-06T02:13:46.214 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2023-11-06T02:13:46.215 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-11-06T02:13:46.216 DEBUG:teuthology.orchestra.run.smithi049:> import glob 2023-11-06T02:13:46.217 DEBUG:teuthology.orchestra.run.smithi049:> import re 2023-11-06T02:13:46.217 DEBUG:teuthology.orchestra.run.smithi049:> import os 2023-11-06T02:13:46.218 DEBUG:teuthology.orchestra.run.smithi049:> import subprocess 2023-11-06T02:13:46.219 DEBUG:teuthology.orchestra.run.smithi049:> 2023-11-06T02:13:46.219 DEBUG:teuthology.orchestra.run.smithi049:> def _find_admin_socket(client_name): 2023-11-06T02:13:46.220 DEBUG:teuthology.orchestra.run.smithi049:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-06T02:13:46.221 DEBUG:teuthology.orchestra.run.smithi049:> files = glob.glob(asok_path) 2023-11-06T02:13:46.221 DEBUG:teuthology.orchestra.run.smithi049:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-06T02:13:46.222 DEBUG:teuthology.orchestra.run.smithi049:> 2023-11-06T02:13:46.223 DEBUG:teuthology.orchestra.run.smithi049:> # Given a non-glob path, it better be there 2023-11-06T02:13:46.223 DEBUG:teuthology.orchestra.run.smithi049:> if "*" not in asok_path: 2023-11-06T02:13:46.224 DEBUG:teuthology.orchestra.run.smithi049:> assert(len(files) == 1) 2023-11-06T02:13:46.224 DEBUG:teuthology.orchestra.run.smithi049:> return files[0] 2023-11-06T02:13:46.225 DEBUG:teuthology.orchestra.run.smithi049:> 2023-11-06T02:13:46.225 DEBUG:teuthology.orchestra.run.smithi049:> for f in files: 2023-11-06T02:13:46.226 DEBUG:teuthology.orchestra.run.smithi049:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-06T02:13:46.227 DEBUG:teuthology.orchestra.run.smithi049:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-06T02:13:46.228 DEBUG:teuthology.orchestra.run.smithi049:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-06T02:13:46.228 DEBUG:teuthology.orchestra.run.smithi049:> contents = proc_f.read() 2023-11-06T02:13:46.229 DEBUG:teuthology.orchestra.run.smithi049:> if mountpoint in contents: 2023-11-06T02:13:46.229 DEBUG:teuthology.orchestra.run.smithi049:> return f 2023-11-06T02:13:46.229 DEBUG:teuthology.orchestra.run.smithi049:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-06T02:13:46.230 DEBUG:teuthology.orchestra.run.smithi049:> 2023-11-06T02:13:46.230 DEBUG:teuthology.orchestra.run.smithi049:> print(_find_admin_socket("client.0")) 2023-11-06T02:13:46.231 DEBUG:teuthology.orchestra.run.smithi049:> ' 2023-11-06T02:13:46.336 INFO:teuthology.orchestra.run.smithi049.stdout:/var/run/ceph/ceph-client.0.54957.asok 2023-11-06T02:13:46.519 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.54957.asok 2023-11-06T02:13:46.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:13:46.520 DEBUG:teuthology.orchestra.run.smithi049:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.54957.asok status 2023-11-06T02:13:46.678 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-11-06T02:13:46.679 INFO:teuthology.orchestra.run.smithi049.stdout: "metadata": { 2023-11-06T02:13:46.679 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_sha1": "f7e9f9af51acf98e0ccc3037487018e4ae40caa5", 2023-11-06T02:13:46.679 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_version": "ceph version 18.2.0-1181-gf7e9f9af (f7e9f9af51acf98e0ccc3037487018e4ae40caa5) reef (stable)", 2023-11-06T02:13:46.679 INFO:teuthology.orchestra.run.smithi049.stdout: "entity_id": "0", 2023-11-06T02:13:46.680 INFO:teuthology.orchestra.run.smithi049.stdout: "hostname": "smithi049", 2023-11-06T02:13:46.680 INFO:teuthology.orchestra.run.smithi049.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-06T02:13:46.680 INFO:teuthology.orchestra.run.smithi049.stdout: "pid": "54957", 2023-11-06T02:13:46.680 INFO:teuthology.orchestra.run.smithi049.stdout: "root": "/" 2023-11-06T02:13:46.680 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:13:46.680 INFO:teuthology.orchestra.run.smithi049.stdout: "dentry_count": 0, 2023-11-06T02:13:46.681 INFO:teuthology.orchestra.run.smithi049.stdout: "dentry_pinned_count": 0, 2023-11-06T02:13:46.681 INFO:teuthology.orchestra.run.smithi049.stdout: "id": 7314, 2023-11-06T02:13:46.681 INFO:teuthology.orchestra.run.smithi049.stdout: "inst": { 2023-11-06T02:13:46.681 INFO:teuthology.orchestra.run.smithi049.stdout: "name": { 2023-11-06T02:13:46.681 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "client", 2023-11-06T02:13:46.682 INFO:teuthology.orchestra.run.smithi049.stdout: "num": 7314 2023-11-06T02:13:46.682 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:13:46.682 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": { 2023-11-06T02:13:46.682 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2023-11-06T02:13:46.682 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "192.168.0.1:0", 2023-11-06T02:13:46.683 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 3684989087 2023-11-06T02:13:46.683 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:13:46.683 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:13:46.683 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": { 2023-11-06T02:13:46.683 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2023-11-06T02:13:46.683 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "192.168.0.1:0", 2023-11-06T02:13:46.684 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 3684989087 2023-11-06T02:13:46.684 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:13:46.684 INFO:teuthology.orchestra.run.smithi049.stdout: "inst_str": "client.7314 192.168.0.1:0/3684989087", 2023-11-06T02:13:46.684 INFO:teuthology.orchestra.run.smithi049.stdout: "addr_str": "192.168.0.1:0/3684989087", 2023-11-06T02:13:46.684 INFO:teuthology.orchestra.run.smithi049.stdout: "inode_count": 1, 2023-11-06T02:13:46.685 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_epoch": 46, 2023-11-06T02:13:46.685 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_epoch": 80, 2023-11-06T02:13:46.685 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_epoch_barrier": 0, 2023-11-06T02:13:46.685 INFO:teuthology.orchestra.run.smithi049.stdout: "blocklisted": false, 2023-11-06T02:13:46.685 INFO:teuthology.orchestra.run.smithi049.stdout: "fs_name": "cephfs" 2023-11-06T02:13:46.686 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-11-06T02:13:46.692 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:13:46.692 DEBUG:teuthology.orchestra.run.smithi049:> stat --file-system '--printf=%T 2023-11-06T02:13:46.693 DEBUG:teuthology.orchestra.run.smithi049:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-06T02:13:46.698 INFO:teuthology.orchestra.run.smithi049.stdout:fuseblk 2023-11-06T02:13:46.699 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-06T02:13:46.699 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:13:46.700 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-06T02:13:46.752 DEBUG:teuthology.orchestra.run.smithi049:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-06T02:13:46.752 DEBUG:teuthology.orchestra.run.smithi049:> import os 2023-11-06T02:13:46.753 DEBUG:teuthology.orchestra.run.smithi049:> mount_point = "/home/ubuntu/cephtest/mnt.0" 2023-11-06T02:13:46.753 DEBUG:teuthology.orchestra.run.smithi049:> parent = os.path.join(mount_point, "parent") 2023-11-06T02:13:46.753 DEBUG:teuthology.orchestra.run.smithi049:> os.mkdir(parent) 2023-11-06T02:13:46.753 DEBUG:teuthology.orchestra.run.smithi049:> flushed = os.path.join(parent, "flushed") 2023-11-06T02:13:46.754 DEBUG:teuthology.orchestra.run.smithi049:> os.mkdir(flushed) 2023-11-06T02:13:46.754 DEBUG:teuthology.orchestra.run.smithi049:> for f in ["alpha", "bravo", "charlie"]: 2023-11-06T02:13:46.755 DEBUG:teuthology.orchestra.run.smithi049:> open(os.path.join(flushed, f), '"'"'w'"'"').write(f) 2023-11-06T02:13:46.755 DEBUG:teuthology.orchestra.run.smithi049:> ' 2023-11-06T02:13:47.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:13:47.048 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2023-11-06T02:13:47.119 INFO:teuthology.orchestra.run.smithi049.stdout:./ 2023-11-06T02:13:47.120 INFO:teuthology.orchestra.run.smithi049.stdout:./parent 2023-11-06T02:13:47.120 INFO:teuthology.orchestra.run.smithi049.stdout:./parent/flushed 2023-11-06T02:13:47.120 INFO:teuthology.orchestra.run.smithi049.stdout:./parent/flushed/charlie 2023-11-06T02:13:47.120 INFO:teuthology.orchestra.run.smithi049.stdout:./parent/flushed/alpha 2023-11-06T02:13:47.120 INFO:teuthology.orchestra.run.smithi049.stdout:./parent/flushed/bravo 2023-11-06T02:13:47.121 DEBUG:teuthology.orchestra.run.smithi049:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-06T02:13:47.122 DEBUG:teuthology.orchestra.run.smithi049:> import os 2023-11-06T02:13:47.122 DEBUG:teuthology.orchestra.run.smithi049:> import stat 2023-11-06T02:13:47.123 DEBUG:teuthology.orchestra.run.smithi049:> 2023-11-06T02:13:47.123 DEBUG:teuthology.orchestra.run.smithi049:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2023-11-06T02:13:47.123 DEBUG:teuthology.orchestra.run.smithi049:> ' 2023-11-06T02:13:47.229 INFO:teuthology.orchestra.run.smithi049.stdout:1 2023-11-06T02:13:47.417 DEBUG:teuthology.orchestra.run.smithi049:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-06T02:13:47.418 DEBUG:teuthology.orchestra.run.smithi049:> import os 2023-11-06T02:13:47.418 DEBUG:teuthology.orchestra.run.smithi049:> import stat 2023-11-06T02:13:47.418 DEBUG:teuthology.orchestra.run.smithi049:> 2023-11-06T02:13:47.419 DEBUG:teuthology.orchestra.run.smithi049:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2023-11-06T02:13:47.419 DEBUG:teuthology.orchestra.run.smithi049:> ' 2023-11-06T02:13:47.527 INFO:teuthology.orchestra.run.smithi049.stdout:1099511627776 2023-11-06T02:13:47.714 DEBUG:teuthology.orchestra.run.smithi049:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-06T02:13:47.715 DEBUG:teuthology.orchestra.run.smithi049:> import os 2023-11-06T02:13:47.715 DEBUG:teuthology.orchestra.run.smithi049:> import stat 2023-11-06T02:13:47.715 DEBUG:teuthology.orchestra.run.smithi049:> 2023-11-06T02:13:47.716 DEBUG:teuthology.orchestra.run.smithi049:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2023-11-06T02:13:47.716 DEBUG:teuthology.orchestra.run.smithi049:> ' 2023-11-06T02:13:47.824 INFO:teuthology.orchestra.run.smithi049.stdout:1099511627777 2023-11-06T02:13:48.011 DEBUG:teuthology.orchestra.run.smithi049:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-06T02:13:48.012 DEBUG:teuthology.orchestra.run.smithi049:> import os 2023-11-06T02:13:48.012 DEBUG:teuthology.orchestra.run.smithi049:> import stat 2023-11-06T02:13:48.012 DEBUG:teuthology.orchestra.run.smithi049:> 2023-11-06T02:13:48.013 DEBUG:teuthology.orchestra.run.smithi049:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2023-11-06T02:13:48.013 DEBUG:teuthology.orchestra.run.smithi049:> ' 2023-11-06T02:13:48.119 INFO:teuthology.orchestra.run.smithi049.stdout:1099511628280 2023-11-06T02:13:48.307 DEBUG:teuthology.orchestra.run.smithi049:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-06T02:13:48.308 DEBUG:teuthology.orchestra.run.smithi049:> import os 2023-11-06T02:13:48.308 DEBUG:teuthology.orchestra.run.smithi049:> import stat 2023-11-06T02:13:48.308 DEBUG:teuthology.orchestra.run.smithi049:> 2023-11-06T02:13:48.309 DEBUG:teuthology.orchestra.run.smithi049:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2023-11-06T02:13:48.309 DEBUG:teuthology.orchestra.run.smithi049:> ' 2023-11-06T02:13:48.414 INFO:teuthology.orchestra.run.smithi049.stdout:1099511627778 2023-11-06T02:13:48.602 DEBUG:teuthology.orchestra.run.smithi049:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-06T02:13:48.602 DEBUG:teuthology.orchestra.run.smithi049:> import os 2023-11-06T02:13:48.603 DEBUG:teuthology.orchestra.run.smithi049:> import stat 2023-11-06T02:13:48.603 DEBUG:teuthology.orchestra.run.smithi049:> 2023-11-06T02:13:48.603 DEBUG:teuthology.orchestra.run.smithi049:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/bravo").st_ino) 2023-11-06T02:13:48.604 DEBUG:teuthology.orchestra.run.smithi049:> ' 2023-11-06T02:13:48.711 INFO:teuthology.orchestra.run.smithi049.stdout:1099511628279 2023-11-06T02:13:48.898 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-11-06T02:13:48.898 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T02:13:48.945 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T02:13:49.117 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.122+0000 7f5b1e9d8640 1 Processor -- start 2023-11-06T02:13:49.117 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.122+0000 7f5b1e9d8640 1 -- start start 2023-11-06T02:13:49.119 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.122+0000 7f5b1e9d8640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f5b1815cc50 0x7f5b180701e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:49.119 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.122+0000 7f5b1e9d8640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f5b180c9690 0x7f5b180b04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:49.120 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.122+0000 7f5b1e9d8640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b180b0a20 con 0x7f5b180b0080 2023-11-06T02:13:49.120 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.122+0000 7f5b1e9d8640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b1815d030 con 0x7f5b1815cc50 2023-11-06T02:13:49.120 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.122+0000 7f5b1e9d8640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b180cc9c0 con 0x7f5b180c9690 2023-11-06T02:13:49.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.122+0000 7f5b17fff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f5b1815cc50 0x7f5b180701e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:49.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.122+0000 7f5b17fff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f5b1815cc50 0x7f5b180701e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:32902/0 (socket says 172.21.15.49:32902) 2023-11-06T02:13:49.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.122+0000 7f5b177fe640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f5b180b0080 0x7f5b180c8f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:55372/0 (socket says 172.21.15.49:55372) 2023-11-06T02:13:49.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.122+0000 7f5b17fff640 1 -- 172.21.15.49:0/2916913776 learned_addr learned my addr 172.21.15.49:0/2916913776 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:49.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.122+0000 7f5b1cf75640 1 --2- 172.21.15.49:0/2916913776 >> v2:172.21.15.150:3300/0 conn(0x7f5b180c9690 0x7f5b180b04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:49.122 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.122+0000 7f5b17fff640 1 -- 172.21.15.49:0/2916913776 >> v2:172.21.15.150:3300/0 conn(0x7f5b180c9690 msgr2=0x7f5b180b04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:49.122 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.122+0000 7f5b17fff640 1 --2- 172.21.15.49:0/2916913776 >> v2:172.21.15.150:3300/0 conn(0x7f5b180c9690 0x7f5b180b04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:49.122 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.122+0000 7f5b17fff640 1 -- 172.21.15.49:0/2916913776 >> v1:172.21.15.49:6789/0 conn(0x7f5b180b0080 legacy=0x7f5b180c8f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:49.122 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.122+0000 7f5b16ffd640 1 -- 172.21.15.49:0/2916913776 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3201334696 0 0) 0x7f5b180b0a20 con 0x7f5b180b0080 2023-11-06T02:13:49.123 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.122+0000 7f5b17fff640 1 -- 172.21.15.49:0/2916913776 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b180ccc60 con 0x7f5b1815cc50 2023-11-06T02:13:49.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.126+0000 7f5b17fff640 1 --2- 172.21.15.49:0/2916913776 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b1815cc50 0x7f5b180701e0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f5b180c8ce0 tx=0x7f5b1001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74a5c6138c48fc08 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:49.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.126+0000 7f5b16ffd640 1 -- 172.21.15.49:0/2916913776 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b100a5830 con 0x7f5b1815cc50 2023-11-06T02:13:49.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.126+0000 7f5b16ffd640 1 -- 172.21.15.49:0/2916913776 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b100a5990 con 0x7f5b1815cc50 2023-11-06T02:13:49.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.126+0000 7f5b16ffd640 1 -- 172.21.15.49:0/2916913776 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b100a5d20 con 0x7f5b1815cc50 2023-11-06T02:13:49.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.126+0000 7f5b1e9d8640 1 -- 172.21.15.49:0/2916913776 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b1815cc50 msgr2=0x7f5b180701e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:49.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.126+0000 7f5b1e9d8640 1 --2- 172.21.15.49:0/2916913776 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b1815cc50 0x7f5b180701e0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f5b180c8ce0 tx=0x7f5b1001c970 comp rx=0 tx=0).stop 2023-11-06T02:13:49.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.126+0000 7f5b1e9d8640 1 -- 172.21.15.49:0/2916913776 shutdown_connections 2023-11-06T02:13:49.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.126+0000 7f5b1e9d8640 1 --2- 172.21.15.49:0/2916913776 >> v2:172.21.15.150:3300/0 conn(0x7f5b180c9690 0x7f5b180b04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:49.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.126+0000 7f5b1e9d8640 1 --2- 172.21.15.49:0/2916913776 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b1815cc50 0x7f5b180701e0 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:49.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.126+0000 7f5b1e9d8640 1 -- 172.21.15.49:0/2916913776 >> 172.21.15.49:0/2916913776 conn(0x7f5b1806c0a0 msgr2=0x7f5b1806c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:49.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.126+0000 7f5b1e9d8640 1 -- 172.21.15.49:0/2916913776 shutdown_connections 2023-11-06T02:13:49.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.126+0000 7f5b1e9d8640 1 -- 172.21.15.49:0/2916913776 wait complete. 2023-11-06T02:13:49.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.126+0000 7f5b1e9d8640 1 Processor -- start 2023-11-06T02:13:49.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.126+0000 7f5b1e9d8640 1 -- start start 2023-11-06T02:13:49.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.126+0000 7f5b1e9d8640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5b180b0080 0x7f5b18149310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:49.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.126+0000 7f5b1e9d8640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b180c9690 0x7f5b180c3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:49.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.126+0000 7f5b1e9d8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b1815cc50 0x7f5b18152a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:49.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.126+0000 7f5b1e9d8640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b180c47d0 con 0x7f5b1815cc50 2023-11-06T02:13:49.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.126+0000 7f5b1e9d8640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b180c4ab0 con 0x7f5b180c9690 2023-11-06T02:13:49.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.126+0000 7f5b1e9d8640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b180c4d90 con 0x7f5b180b0080 2023-11-06T02:13:49.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.126+0000 7f5b1cf75640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b1815cc50 0x7f5b18152a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:49.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.126+0000 7f5b1cf75640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b1815cc50 0x7f5b18152a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:32914/0 (socket says 172.21.15.49:32914) 2023-11-06T02:13:49.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.126+0000 7f5b1cf75640 1 -- 172.21.15.49:0/944733522 learned_addr learned my addr 172.21.15.49:0/944733522 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:49.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.126+0000 7f5b17fff640 1 --2- 172.21.15.49:0/944733522 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5b180b0080 0x7f5b18149310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:49.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.126+0000 7f5b177fe640 1 --2- 172.21.15.49:0/944733522 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b180c9690 0x7f5b180c3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:49.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.126+0000 7f5b1cf75640 1 -- 172.21.15.49:0/944733522 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5b180b0080 msgr2=0x7f5b18149310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:49.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.126+0000 7f5b1cf75640 1 --2- 172.21.15.49:0/944733522 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5b180b0080 0x7f5b18149310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:49.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.126+0000 7f5b1cf75640 1 -- 172.21.15.49:0/944733522 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b180c9690 msgr2=0x7f5b180c3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:49.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.126+0000 7f5b1cf75640 1 --2- 172.21.15.49:0/944733522 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b180c9690 0x7f5b180c3cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:49.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.126+0000 7f5b1cf75640 1 -- 172.21.15.49:0/944733522 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b10009b30 con 0x7f5b1815cc50 2023-11-06T02:13:49.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.130+0000 7f5b1cf75640 1 --2- 172.21.15.49:0/944733522 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b1815cc50 0x7f5b18152a80 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f5b0800d800 tx=0x7f5b0800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:49.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.130+0000 7f5b14ff9640 1 -- 172.21.15.49:0/944733522 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0800b6c0 con 0x7f5b1815cc50 2023-11-06T02:13:49.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.130+0000 7f5b14ff9640 1 -- 172.21.15.49:0/944733522 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b0800b820 con 0x7f5b1815cc50 2023-11-06T02:13:49.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.130+0000 7f5b1e9d8640 1 -- 172.21.15.49:0/944733522 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b180c5af0 con 0x7f5b1815cc50 2023-11-06T02:13:49.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.130+0000 7f5b14ff9640 1 -- 172.21.15.49:0/944733522 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0800bbd0 con 0x7f5b1815cc50 2023-11-06T02:13:49.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.130+0000 7f5b1e9d8640 1 -- 172.21.15.49:0/944733522 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b180c2240 con 0x7f5b1815cc50 2023-11-06T02:13:49.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.130+0000 7f5b1e9d8640 1 -- 172.21.15.49:0/944733522 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b180c5950 con 0x7f5b1815cc50 2023-11-06T02:13:49.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.130+0000 7f5b14ff9640 1 -- 172.21.15.49:0/944733522 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f5b0800bd30 con 0x7f5b1815cc50 2023-11-06T02:13:49.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.130+0000 7f5b14ff9640 1 --2- 172.21.15.49:0/944733522 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5af8065a60 0x7f5af8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:49.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.130+0000 7f5b14ff9640 1 -- 172.21.15.49:0/944733522 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5b080837b0 con 0x7f5b1815cc50 2023-11-06T02:13:49.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.130+0000 7f5b17fff640 1 --2- 172.21.15.49:0/944733522 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5af8065a60 0x7f5af8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:49.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.130+0000 7f5b17fff640 1 --2- 172.21.15.49:0/944733522 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5af8065a60 0x7f5af8067f20 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f5b100ae000 tx=0x7f5b100a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:49.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.134+0000 7f5b14ff9640 1 -- 172.21.15.49:0/944733522 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f5b180c5950 con 0x7f5b1815cc50 2023-11-06T02:13:49.293 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.298+0000 7f5b1e9d8640 1 -- 172.21.15.49:0/944733522 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5b180c5950 con 0x7f5b1815cc50 2023-11-06T02:13:49.294 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.298+0000 7f5b14ff9640 1 -- 172.21.15.49:0/944733522 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+17886 (secure 0 0 0) 0x7f5b180c5950 con 0x7f5b1815cc50 2023-11-06T02:13:49.295 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:13:49.295 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":80,"fsid":"1a8b035c-6ba4-499a-b876-aea47fd9c0d4","created":"2023-11-06T02:08:06.440857+0000","modified":"2023-11-06T02:13:34.630031+0000","last_up_change":"2023-11-06T02:08:17.982372+0000","last_in_change":"2023-11-06T02:08:08.906631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T02:08:19.252222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T02:08:24.776211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-11-06T02:13:31.459314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-11-06T02:13:32.603710+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ad55af8-0571-4e6a-8722-b5d429b7496d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6809","nonce":38116},{"type":"v1","addr":"172.21.15.49:6811","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6813","nonce":38116},{"type":"v1","addr":"172.21.15.49:6815","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6821","nonce":38116},{"type":"v1","addr":"172.21.15.49:6823","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6817","nonce":38116},{"type":"v1","addr":"172.21.15.49:6819","nonce":38116}]},"public_addr":"172.21.15.49:6811/38116","cluster_addr":"172.21.15.49:6815/38116","heartbeat_back_addr":"172.21.15.49:6823/38116","heartbeat_front_addr":"172.21.15.49:6819/38116","state":["exists","up"]},{"osd":1,"uuid":"c8b31f85-a757-43ed-9419-351acc46611b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":38113},{"type":"v1","addr":"172.21.15.49:6810","nonce":38113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":38113},{"type":"v1","addr":"172.21.15.49:6814","nonce":38113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":38113},{"type":"v1","addr":"172.21.15.49:6822","nonce":38113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":38113},{"type":"v1","addr":"172.21.15.49:6818","nonce":38113}]},"public_addr":"172.21.15.49:6810/38113","cluster_addr":"172.21.15.49:6814/38113","heartbeat_back_addr":"172.21.15.49:6822/38113","heartbeat_front_addr":"172.21.15.49:6818/38113","state":["exists","up"]},{"osd":2,"uuid":"555b3bf3-0a70-4b91-8efd-e4204b5fc26e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":38114},{"type":"v1","addr":"172.21.15.49:6801","nonce":38114}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":38114},{"type":"v1","addr":"172.21.15.49:6803","nonce":38114}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":38114},{"type":"v1","addr":"172.21.15.49:6807","nonce":38114}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":38114},{"type":"v1","addr":"172.21.15.49:6805","nonce":38114}]},"public_addr":"172.21.15.49:6801/38114","cluster_addr":"172.21.15.49:6803/38114","heartbeat_back_addr":"172.21.15.49:6807/38114","heartbeat_front_addr":"172.21.15.49:6805/38114","state":["exists","up"]},{"osd":3,"uuid":"8d4e5d77-03ef-4b9b-bfcb-5ae93a46795c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":38115},{"type":"v1","addr":"172.21.15.49:6825","nonce":38115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":38115},{"type":"v1","addr":"172.21.15.49:6827","nonce":38115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":38115},{"type":"v1","addr":"172.21.15.49:6831","nonce":38115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":38115},{"type":"v1","addr":"172.21.15.49:6829","nonce":38115}]},"public_addr":"172.21.15.49:6825/38115","cluster_addr":"172.21.15.49:6827/38115","heartbeat_back_addr":"172.21.15.49:6831/38115","heartbeat_front_addr":"172.21.15.49:6829/38115","state":["exists","up"]},{"osd":4,"uuid":"dba03ad4-68a3-4cf3-9218-ac611f02eacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":38417},{"type":"v1","addr":"172.21.15.150:6809","nonce":38417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":38417},{"type":"v1","addr":"172.21.15.150:6811","nonce":38417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":38417},{"type":"v1","addr":"172.21.15.150:6815","nonce":38417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":38417},{"type":"v1","addr":"172.21.15.150:6813","nonce":38417}]},"public_addr":"172.21.15.150:6809/38417","cluster_addr":"172.21.15.150:6811/38417","heartbeat_back_addr":"172.21.15.150:6815/38417","heartbeat_front_addr":"172.21.15.150:6813/38417","state":["exists","up"]},{"osd":5,"uuid":"0df9c303-c255-4640-8e91-01f1c3efe18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":38418},{"type":"v1","addr":"172.21.15.150:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":38418},{"type":"v1","addr":"172.21.15.150:6803","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":38418},{"type":"v1","addr":"172.21.15.150:6807","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":38418},{"type":"v1","addr":"172.21.15.150:6805","nonce":38418}]},"public_addr":"172.21.15.150:6801/38418","cluster_addr":"172.21.15.150:6803/38418","heartbeat_back_addr":"172.21.15.150:6807/38418","heartbeat_front_addr":"172.21.15.150:6805/38418","state":["exists","up"]},{"osd":6,"uuid":"e59fa189-4b70-4c06-9722-c02b90828bf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":38428},{"type":"v1","addr":"172.21.15.150:6825","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":38428},{"type":"v1","addr":"172.21.15.150:6827","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":38428},{"type":"v1","addr":"172.21.15.150:6831","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":38428},{"type":"v1","addr":"172.21.15.150:6829","nonce":38428}]},"public_addr":"172.21.15.150:6825/38428","cluster_addr":"172.21.15.150:6827/38428","heartbeat_back_addr":"172.21.15.150:6831/38428","heartbeat_front_addr":"172.21.15.150:6829/38428","state":["exists","up"]},{"osd":7,"uuid":"a64f57e1-ef98-46bd-b304-385759e95898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":38421},{"type":"v1","addr":"172.21.15.150:6817","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":38421},{"type":"v1","addr":"172.21.15.150:6819","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":38421},{"type":"v1","addr":"172.21.15.150:6823","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":38421},{"type":"v1","addr":"172.21.15.150:6821","nonce":38421}]},"public_addr":"172.21.15.150:6817/38421","cluster_addr":"172.21.15.150:6819/38421","heartbeat_back_addr":"172.21.15.150:6823/38421","heartbeat_front_addr":"172.21.15.150:6821/38421","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.438552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.413280+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.296466+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.424966+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.276953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.230984+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.410235+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.480084+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T02:13:49.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.302+0000 7f5b1e9d8640 1 -- 172.21.15.49:0/944733522 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5af8065a60 msgr2=0x7f5af8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:49.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.302+0000 7f5b1e9d8640 1 --2- 172.21.15.49:0/944733522 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5af8065a60 0x7f5af8067f20 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f5b100ae000 tx=0x7f5b100a7070 comp rx=0 tx=0).stop 2023-11-06T02:13:49.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.302+0000 7f5b1e9d8640 1 -- 172.21.15.49:0/944733522 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b1815cc50 msgr2=0x7f5b18152a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:49.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.302+0000 7f5b1e9d8640 1 --2- 172.21.15.49:0/944733522 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b1815cc50 0x7f5b18152a80 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f5b0800d800 tx=0x7f5b0800dcc0 comp rx=0 tx=0).stop 2023-11-06T02:13:49.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.302+0000 7f5b1e9d8640 1 -- 172.21.15.49:0/944733522 shutdown_connections 2023-11-06T02:13:49.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.302+0000 7f5b1e9d8640 1 --2- 172.21.15.49:0/944733522 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5af8065a60 0x7f5af8067f20 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:49.300 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.302+0000 7f5b1e9d8640 1 --2- 172.21.15.49:0/944733522 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b1815cc50 0x7f5b18152a80 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:49.300 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.302+0000 7f5b1e9d8640 1 --2- 172.21.15.49:0/944733522 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b180c9690 0x7f5b180c3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:49.301 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.302+0000 7f5b1e9d8640 1 --2- 172.21.15.49:0/944733522 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5b180b0080 0x7f5b18149310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:49.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.302+0000 7f5b1e9d8640 1 -- 172.21.15.49:0/944733522 >> 172.21.15.49:0/944733522 conn(0x7f5b1806c0a0 msgr2=0x7f5b180af1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:49.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.302+0000 7f5b1e9d8640 1 -- 172.21.15.49:0/944733522 shutdown_connections 2023-11-06T02:13:49.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.302+0000 7f5b1e9d8640 1 -- 172.21.15.49:0/944733522 wait complete. 2023-11-06T02:13:49.314 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T02:08:19.252222+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T02:08:24.776211+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T02:13:31.459314+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T02:13:32.603710+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T02:13:49.315 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T02:13:49.444 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.450+0000 7f37e0de4640 1 Processor -- start 2023-11-06T02:13:49.445 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.450+0000 7f37e0de4640 1 -- start start 2023-11-06T02:13:49.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.450+0000 7f37e0de4640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f37dc15a880 0x7f37dc0b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:49.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.450+0000 7f37e0de4640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f37dc06d860 0x7f37dc070610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:49.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.450+0000 7f37e0de4640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37dc0ae360 con 0x7f37dc070160 2023-11-06T02:13:49.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.450+0000 7f37e0de4640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37dc070b50 con 0x7f37dc06d860 2023-11-06T02:13:49.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.450+0000 7f37e0de4640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37dc0ae640 con 0x7f37dc15a880 2023-11-06T02:13:49.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.450+0000 7f37d9d9b640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f37dc070160 0x7f37dc06d150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:55378/0 (socket says 172.21.15.49:55378) 2023-11-06T02:13:49.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.450+0000 7f37d9d9b640 1 -- 172.21.15.49:0/3855671462 learned_addr learned my addr 172.21.15.49:0/3855671462 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:13:49.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.450+0000 7f37dad9d640 1 --2- 172.21.15.49:0/3855671462 >> v2:172.21.15.49:3300/0 conn(0x7f37dc06d860 0x7f37dc070610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:49.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.450+0000 7f37da59c640 1 --2- 172.21.15.49:0/3855671462 >> v2:172.21.15.150:3300/0 conn(0x7f37dc15a880 0x7f37dc0b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:49.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.450+0000 7f37d959a640 1 -- 172.21.15.49:0/3855671462 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3568239004 0 0) 0x7f37dc0ae360 con 0x7f37dc070160 2023-11-06T02:13:49.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.450+0000 7f37d959a640 1 -- 172.21.15.49:0/3855671462 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37c4003530 con 0x7f37dc070160 2023-11-06T02:13:49.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.450+0000 7f37dad9d640 1 -- 172.21.15.49:0/3855671462 >> v2:172.21.15.150:3300/0 conn(0x7f37dc15a880 msgr2=0x7f37dc0b15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:49.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.450+0000 7f37dad9d640 1 --2- 172.21.15.49:0/3855671462 >> v2:172.21.15.150:3300/0 conn(0x7f37dc15a880 0x7f37dc0b15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:49.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.450+0000 7f37dad9d640 1 -- 172.21.15.49:0/3855671462 >> v1:172.21.15.49:6789/0 conn(0x7f37dc070160 legacy=0x7f37dc06d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:49.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.450+0000 7f37dad9d640 1 -- 172.21.15.49:0/3855671462 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37dc0ae920 con 0x7f37dc06d860 2023-11-06T02:13:49.451 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.454+0000 7f37dad9d640 1 --2- 172.21.15.49:0/3855671462 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37dc06d860 0x7f37dc070610 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f37dc0cafc0 tx=0x7f37d401c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5b58e5a2e56e15d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:49.451 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.454+0000 7f37d959a640 1 -- 172.21.15.49:0/3855671462 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d40a58e0 con 0x7f37dc06d860 2023-11-06T02:13:49.451 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.454+0000 7f37d959a640 1 -- 172.21.15.49:0/3855671462 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37d40a5a40 con 0x7f37dc06d860 2023-11-06T02:13:49.451 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.454+0000 7f37d959a640 1 -- 172.21.15.49:0/3855671462 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d40a5dd0 con 0x7f37dc06d860 2023-11-06T02:13:49.451 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.454+0000 7f37e0de4640 1 -- 172.21.15.49:0/3855671462 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37dc06d860 msgr2=0x7f37dc070610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:49.452 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.454+0000 7f37e0de4640 1 --2- 172.21.15.49:0/3855671462 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37dc06d860 0x7f37dc070610 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f37dc0cafc0 tx=0x7f37d401c910 comp rx=0 tx=0).stop 2023-11-06T02:13:49.452 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.454+0000 7f37e0de4640 1 -- 172.21.15.49:0/3855671462 shutdown_connections 2023-11-06T02:13:49.452 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.454+0000 7f37e0de4640 1 --2- 172.21.15.49:0/3855671462 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37dc06d860 0x7f37dc070610 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:49.452 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.454+0000 7f37e0de4640 1 --2- 172.21.15.49:0/3855671462 >> v2:172.21.15.150:3300/0 conn(0x7f37dc15a880 0x7f37dc0b15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:49.452 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.454+0000 7f37e0de4640 1 -- 172.21.15.49:0/3855671462 >> 172.21.15.49:0/3855671462 conn(0x7f37dc06a370 msgr2=0x7f37dc0ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:49.453 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.454+0000 7f37e0de4640 1 -- 172.21.15.49:0/3855671462 shutdown_connections 2023-11-06T02:13:49.453 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.454+0000 7f37e0de4640 1 -- 172.21.15.49:0/3855671462 wait complete. 2023-11-06T02:13:49.453 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.454+0000 7f37e0de4640 1 Processor -- start 2023-11-06T02:13:49.453 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.454+0000 7f37e0de4640 1 -- start start 2023-11-06T02:13:49.455 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.454+0000 7f37e0de4640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f37dc06d860 0x7f37dc147be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:49.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.454+0000 7f37e0de4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37dc070160 0x7f37dc14aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:49.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.454+0000 7f37e0de4640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f37dc15a880 0x7f37dc149540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:49.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.454+0000 7f37e0de4640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37dc14b6e0 con 0x7f37dc070160 2023-11-06T02:13:49.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.454+0000 7f37e0de4640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37dc1511d0 con 0x7f37dc15a880 2023-11-06T02:13:49.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.454+0000 7f37e0de4640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37dc149a80 con 0x7f37dc06d860 2023-11-06T02:13:49.457 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.454+0000 7f37d9d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37dc070160 0x7f37dc14aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:49.457 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.454+0000 7f37d9d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37dc070160 0x7f37dc14aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:32932/0 (socket says 172.21.15.49:32932) 2023-11-06T02:13:49.457 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.454+0000 7f37d9d9b640 1 -- 172.21.15.49:0/596704610 learned_addr learned my addr 172.21.15.49:0/596704610 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:49.457 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.458+0000 7f37d9d9b640 1 -- 172.21.15.49:0/596704610 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f37dc06d860 msgr2=0x7f37dc147be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:49.457 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.458+0000 7f37d9d9b640 1 --2- 172.21.15.49:0/596704610 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f37dc06d860 0x7f37dc147be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:49.457 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.458+0000 7f37d9d9b640 1 -- 172.21.15.49:0/596704610 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f37dc15a880 msgr2=0x7f37dc149540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:49.458 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.458+0000 7f37dad9d640 1 --2- 172.21.15.49:0/596704610 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f37dc15a880 0x7f37dc149540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:49.458 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.458+0000 7f37d9d9b640 1 --2- 172.21.15.49:0/596704610 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f37dc15a880 0x7f37dc149540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:49.458 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.458+0000 7f37d9d9b640 1 -- 172.21.15.49:0/596704610 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37d4009a70 con 0x7f37dc070160 2023-11-06T02:13:49.458 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.458+0000 7f37d9d9b640 1 --2- 172.21.15.49:0/596704610 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37dc070160 0x7f37dc14aa40 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f37d000c450 tx=0x7f37d000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:49.458 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.458+0000 7f37caffd640 1 -- 172.21.15.49:0/596704610 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d0005cf0 con 0x7f37dc070160 2023-11-06T02:13:49.459 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.458+0000 7f37caffd640 1 -- 172.21.15.49:0/596704610 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37d0005e50 con 0x7f37dc070160 2023-11-06T02:13:49.459 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.458+0000 7f37e0de4640 1 -- 172.21.15.49:0/596704610 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37dc14b120 con 0x7f37dc070160 2023-11-06T02:13:49.459 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.458+0000 7f37caffd640 1 -- 172.21.15.49:0/596704610 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d0007290 con 0x7f37dc070160 2023-11-06T02:13:49.459 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.458+0000 7f37e0de4640 1 -- 172.21.15.49:0/596704610 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37dc151420 con 0x7f37dc070160 2023-11-06T02:13:49.459 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.458+0000 7f37e0de4640 1 -- 172.21.15.49:0/596704610 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37dc074040 con 0x7f37dc070160 2023-11-06T02:13:49.460 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.458+0000 7f37caffd640 1 -- 172.21.15.49:0/596704610 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f37d00073f0 con 0x7f37dc070160 2023-11-06T02:13:49.460 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.458+0000 7f37caffd640 1 --2- 172.21.15.49:0/596704610 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f37bc065a60 0x7f37bc067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:49.460 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.458+0000 7f37caffd640 1 -- 172.21.15.49:0/596704610 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f37d0082c70 con 0x7f37dc070160 2023-11-06T02:13:49.460 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.458+0000 7f37da59c640 1 --2- 172.21.15.49:0/596704610 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f37bc065a60 0x7f37bc067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:49.460 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.458+0000 7f37da59c640 1 --2- 172.21.15.49:0/596704610 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f37bc065a60 0x7f37bc067f20 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f37cc006aa0 tx=0x7f37cc006a30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:49.461 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.462+0000 7f37caffd640 1 -- 172.21.15.49:0/596704610 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f37d0050540 con 0x7f37dc070160 2023-11-06T02:13:49.613 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.618+0000 7f37e0de4640 1 -- 172.21.15.49:0/596704610 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f37dc0cdff0 con 0x7f37dc070160 2023-11-06T02:13:49.614 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.618+0000 7f37caffd640 1 -- 172.21.15.49:0/596704610 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) v1 ==== 93+0+10 (secure 0 0 0) 0x7f37d0017020 con 0x7f37dc070160 2023-11-06T02:13:49.614 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2023-11-06T02:13:49.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.622+0000 7f37e0de4640 1 -- 172.21.15.49:0/596704610 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f37bc065a60 msgr2=0x7f37bc067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:49.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.622+0000 7f37e0de4640 1 --2- 172.21.15.49:0/596704610 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f37bc065a60 0x7f37bc067f20 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f37cc006aa0 tx=0x7f37cc006a30 comp rx=0 tx=0).stop 2023-11-06T02:13:49.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.622+0000 7f37e0de4640 1 -- 172.21.15.49:0/596704610 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37dc070160 msgr2=0x7f37dc14aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:49.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.622+0000 7f37e0de4640 1 --2- 172.21.15.49:0/596704610 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37dc070160 0x7f37dc14aa40 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f37d000c450 tx=0x7f37d000c910 comp rx=0 tx=0).stop 2023-11-06T02:13:49.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.622+0000 7f37e0de4640 1 -- 172.21.15.49:0/596704610 shutdown_connections 2023-11-06T02:13:49.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.622+0000 7f37e0de4640 1 --2- 172.21.15.49:0/596704610 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f37bc065a60 0x7f37bc067f20 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:49.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.622+0000 7f37e0de4640 1 --2- 172.21.15.49:0/596704610 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37dc070160 0x7f37dc14aa40 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:49.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.622+0000 7f37e0de4640 1 --2- 172.21.15.49:0/596704610 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f37dc15a880 0x7f37dc149540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:49.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.622+0000 7f37e0de4640 1 --2- 172.21.15.49:0/596704610 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f37dc06d860 0x7f37dc147be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:49.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.622+0000 7f37e0de4640 1 -- 172.21.15.49:0/596704610 >> 172.21.15.49:0/596704610 conn(0x7f37dc06a370 msgr2=0x7f37dc1531c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:49.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.622+0000 7f37e0de4640 1 -- 172.21.15.49:0/596704610 shutdown_connections 2023-11-06T02:13:49.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.622+0000 7f37e0de4640 1 -- 172.21.15.49:0/596704610 wait complete. 2023-11-06T02:13:49.633 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T02:13:49.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.766+0000 7fa74e431640 1 Processor -- start 2023-11-06T02:13:49.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.766+0000 7fa74e431640 1 -- start start 2023-11-06T02:13:49.762 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.766+0000 7fa74e431640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa74815a9b0 0x7fa7480cde40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:49.763 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.766+0000 7fa74e431640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fa74815ad90 0x7fa7480cee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:49.763 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.766+0000 7fa74e431640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7480b3720 con 0x7fa7480cd2a0 2023-11-06T02:13:49.763 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.766+0000 7fa74e431640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7480cf3c0 con 0x7fa74815a9b0 2023-11-06T02:13:49.763 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.766+0000 7fa74e431640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7480cf690 con 0x7fa74815ad90 2023-11-06T02:13:49.763 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.766+0000 7fa747fff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa74815a9b0 0x7fa7480cde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:49.763 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.766+0000 7fa74c9ce640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fa7480cd2a0 0x7fa7480cb3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:55380/0 (socket says 172.21.15.49:55380) 2023-11-06T02:13:49.764 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.766+0000 7fa74c9ce640 1 -- 172.21.15.49:0/1189487905 learned_addr learned my addr 172.21.15.49:0/1189487905 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:13:49.764 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.766+0000 7fa7477fe640 1 --2- 172.21.15.49:0/1189487905 >> v2:172.21.15.150:3300/0 conn(0x7fa74815ad90 0x7fa7480cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:49.764 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.766+0000 7fa747fff640 1 -- 172.21.15.49:0/1189487905 >> v2:172.21.15.150:3300/0 conn(0x7fa74815ad90 msgr2=0x7fa7480cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:49.764 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.766+0000 7fa747fff640 1 --2- 172.21.15.49:0/1189487905 >> v2:172.21.15.150:3300/0 conn(0x7fa74815ad90 0x7fa7480cee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:49.764 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.766+0000 7fa747fff640 1 -- 172.21.15.49:0/1189487905 >> v1:172.21.15.49:6789/0 conn(0x7fa7480cd2a0 legacy=0x7fa7480cb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:49.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.766+0000 7fa747fff640 1 -- 172.21.15.49:0/1189487905 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7480b3970 con 0x7fa74815a9b0 2023-11-06T02:13:49.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.766+0000 7fa7477fe640 1 --2- 172.21.15.49:0/1189487905 >> v2:172.21.15.150:3300/0 conn(0x7fa74815ad90 0x7fa7480cee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:13:49.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.770+0000 7fa747fff640 1 --2- 172.21.15.49:0/1189487905 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa74815a9b0 0x7fa7480cde40 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fa7480ccad0 tx=0x7fa73801c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b34186a4d8da1fc2 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:49.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.770+0000 7fa746ffd640 1 -- 172.21.15.49:0/1189487905 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7380ab600 con 0x7fa74815a9b0 2023-11-06T02:13:49.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.770+0000 7fa746ffd640 1 -- 172.21.15.49:0/1189487905 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7380a7050 con 0x7fa74815a9b0 2023-11-06T02:13:49.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.770+0000 7fa746ffd640 1 -- 172.21.15.49:0/1189487905 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7380ab980 con 0x7fa74815a9b0 2023-11-06T02:13:49.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.770+0000 7fa74e431640 1 -- 172.21.15.49:0/1189487905 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa74815a9b0 msgr2=0x7fa7480cde40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:49.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.770+0000 7fa74e431640 1 --2- 172.21.15.49:0/1189487905 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa74815a9b0 0x7fa7480cde40 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fa7480ccad0 tx=0x7fa73801c440 comp rx=0 tx=0).stop 2023-11-06T02:13:49.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.770+0000 7fa74e431640 1 -- 172.21.15.49:0/1189487905 shutdown_connections 2023-11-06T02:13:49.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.770+0000 7fa74e431640 1 --2- 172.21.15.49:0/1189487905 >> v2:172.21.15.150:3300/0 conn(0x7fa74815ad90 0x7fa7480cee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:49.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.770+0000 7fa74e431640 1 --2- 172.21.15.49:0/1189487905 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa74815a9b0 0x7fa7480cde40 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:49.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.770+0000 7fa74e431640 1 -- 172.21.15.49:0/1189487905 >> 172.21.15.49:0/1189487905 conn(0x7fa7480d08d0 msgr2=0x7fa7480d0cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:49.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.770+0000 7fa74e431640 1 -- 172.21.15.49:0/1189487905 shutdown_connections 2023-11-06T02:13:49.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.770+0000 7fa74e431640 1 -- 172.21.15.49:0/1189487905 wait complete. 2023-11-06T02:13:49.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.770+0000 7fa74e431640 1 Processor -- start 2023-11-06T02:13:49.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.770+0000 7fa74e431640 1 -- start start 2023-11-06T02:13:49.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.770+0000 7fa74e431640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa7480cd2a0 0x7fa748150780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:49.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.770+0000 7fa74e431640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa74815a9b0 0x7fa74806eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:49.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.770+0000 7fa74e431640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa74815ad90 0x7fa74806f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:49.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.770+0000 7fa74e431640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa748151260 con 0x7fa74815ad90 2023-11-06T02:13:49.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.770+0000 7fa74e431640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa748150cc0 con 0x7fa7480cd2a0 2023-11-06T02:13:49.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.770+0000 7fa74e431640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7480ca000 con 0x7fa74815a9b0 2023-11-06T02:13:49.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.774+0000 7fa74c9ce640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa74815ad90 0x7fa74806f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:49.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.774+0000 7fa74c9ce640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa74815ad90 0x7fa74806f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:32958/0 (socket says 172.21.15.49:32958) 2023-11-06T02:13:49.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.774+0000 7fa74c9ce640 1 -- 172.21.15.49:0/2037993664 learned_addr learned my addr 172.21.15.49:0/2037993664 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:49.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.774+0000 7fa7477fe640 1 --2- 172.21.15.49:0/2037993664 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa74815a9b0 0x7fa74806eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:49.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.774+0000 7fa747fff640 1 --2- 172.21.15.49:0/2037993664 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa7480cd2a0 0x7fa748150780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:49.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.774+0000 7fa74c9ce640 1 -- 172.21.15.49:0/2037993664 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa74815a9b0 msgr2=0x7fa74806eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:49.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.774+0000 7fa74c9ce640 1 --2- 172.21.15.49:0/2037993664 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa74815a9b0 0x7fa74806eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:49.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.774+0000 7fa74c9ce640 1 -- 172.21.15.49:0/2037993664 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa7480cd2a0 msgr2=0x7fa748150780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:49.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.774+0000 7fa74c9ce640 1 --2- 172.21.15.49:0/2037993664 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa7480cd2a0 0x7fa748150780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:49.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.774+0000 7fa74c9ce640 1 -- 172.21.15.49:0/2037993664 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7380093b0 con 0x7fa74815ad90 2023-11-06T02:13:49.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.774+0000 7fa7477fe640 1 --2- 172.21.15.49:0/2037993664 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa74815a9b0 0x7fa74806eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:13:49.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.774+0000 7fa747fff640 1 --2- 172.21.15.49:0/2037993664 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa7480cd2a0 0x7fa748150780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:13:49.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.774+0000 7fa74c9ce640 1 --2- 172.21.15.49:0/2037993664 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa74815ad90 0x7fa74806f030 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fa74000ca70 tx=0x7fa74000cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:49.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.774+0000 7fa744ff9640 1 -- 172.21.15.49:0/2037993664 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa74000abd0 con 0x7fa74815ad90 2023-11-06T02:13:49.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.774+0000 7fa744ff9640 1 -- 172.21.15.49:0/2037993664 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa74000ad30 con 0x7fa74815ad90 2023-11-06T02:13:49.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.774+0000 7fa744ff9640 1 -- 172.21.15.49:0/2037993664 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa740012600 con 0x7fa74815ad90 2023-11-06T02:13:49.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.774+0000 7fa74e431640 1 -- 172.21.15.49:0/2037993664 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7480cdc40 con 0x7fa74815ad90 2023-11-06T02:13:49.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.774+0000 7fa74e431640 1 -- 172.21.15.49:0/2037993664 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7481593a0 con 0x7fa74815ad90 2023-11-06T02:13:49.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.774+0000 7fa74e431640 1 -- 172.21.15.49:0/2037993664 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa748159200 con 0x7fa74815ad90 2023-11-06T02:13:49.776 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.774+0000 7fa744ff9640 1 -- 172.21.15.49:0/2037993664 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fa740007070 con 0x7fa74815ad90 2023-11-06T02:13:49.776 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.774+0000 7fa744ff9640 1 --2- 172.21.15.49:0/2037993664 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa734065a60 0x7fa734067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:49.776 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.774+0000 7fa744ff9640 1 -- 172.21.15.49:0/2037993664 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa740083320 con 0x7fa74815ad90 2023-11-06T02:13:49.776 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.774+0000 7fa747fff640 1 --2- 172.21.15.49:0/2037993664 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa734065a60 0x7fa734067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:49.776 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.778+0000 7fa747fff640 1 --2- 172.21.15.49:0/2037993664 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa734065a60 0x7fa734067f20 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fa7380ad000 tx=0x7fa738008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:49.777 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.778+0000 7fa744ff9640 1 -- 172.21.15.49:0/2037993664 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fa740050d10 con 0x7fa74815ad90 2023-11-06T02:13:49.930 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.934+0000 7fa74e431640 1 -- 172.21.15.49:0/2037993664 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa748159200 con 0x7fa74815ad90 2023-11-06T02:13:49.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.934+0000 7fa744ff9640 1 -- 172.21.15.49:0/2037993664 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v80) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa748159200 con 0x7fa74815ad90 2023-11-06T02:13:49.931 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 8 2023-11-06T02:13:49.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.938+0000 7fa74e431640 1 -- 172.21.15.49:0/2037993664 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa734065a60 msgr2=0x7fa734067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:49.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.938+0000 7fa74e431640 1 --2- 172.21.15.49:0/2037993664 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa734065a60 0x7fa734067f20 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fa7380ad000 tx=0x7fa738008040 comp rx=0 tx=0).stop 2023-11-06T02:13:49.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.938+0000 7fa74e431640 1 -- 172.21.15.49:0/2037993664 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa74815ad90 msgr2=0x7fa74806f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:49.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.938+0000 7fa74e431640 1 --2- 172.21.15.49:0/2037993664 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa74815ad90 0x7fa74806f030 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fa74000ca70 tx=0x7fa74000cf30 comp rx=0 tx=0).stop 2023-11-06T02:13:49.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.938+0000 7fa74e431640 1 -- 172.21.15.49:0/2037993664 shutdown_connections 2023-11-06T02:13:49.935 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.938+0000 7fa74e431640 1 --2- 172.21.15.49:0/2037993664 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa734065a60 0x7fa734067f20 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:49.935 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.938+0000 7fa74e431640 1 --2- 172.21.15.49:0/2037993664 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa74815ad90 0x7fa74806f030 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:49.935 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.938+0000 7fa74e431640 1 --2- 172.21.15.49:0/2037993664 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa74815a9b0 0x7fa74806eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:49.935 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.938+0000 7fa74e431640 1 --2- 172.21.15.49:0/2037993664 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa7480cd2a0 0x7fa748150780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:49.935 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.938+0000 7fa74e431640 1 -- 172.21.15.49:0/2037993664 >> 172.21.15.49:0/2037993664 conn(0x7fa7480d08d0 msgr2=0x7fa7480b2cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:49.935 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.938+0000 7fa74e431640 1 -- 172.21.15.49:0/2037993664 shutdown_connections 2023-11-06T02:13:49.936 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:49.938+0000 7fa74e431640 1 -- 172.21.15.49:0/2037993664 wait complete. 2023-11-06T02:13:49.951 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T02:13:50.073 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.078+0000 7fb3100e6640 1 Processor -- start 2023-11-06T02:13:50.073 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.078+0000 7fb3100e6640 1 -- start start 2023-11-06T02:13:50.074 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.078+0000 7fb3100e6640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fb3080d2a60 0x7fb3080d2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:50.074 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.078+0000 7fb3100e6640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb30815cf90 con 0x7fb30815cbb0 2023-11-06T02:13:50.074 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.078+0000 7fb3100e6640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3080d0b80 con 0x7fb3080b5d40 2023-11-06T02:13:50.075 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.078+0000 7fb3100e6640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3080d3410 con 0x7fb3080d2a60 2023-11-06T02:13:50.075 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.078+0000 7fb30de82640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fb30815cbb0 0x7fb3080d1fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:55390/0 (socket says 172.21.15.49:55390) 2023-11-06T02:13:50.075 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.078+0000 7fb30de82640 1 -- 172.21.15.49:0/1439482986 learned_addr learned my addr 172.21.15.49:0/1439482986 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:13:50.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.078+0000 7fb30d681640 1 --2- 172.21.15.49:0/1439482986 >> v2:172.21.15.150:3300/0 conn(0x7fb3080d2a60 0x7fb3080d2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:50.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.078+0000 7fb30ce80640 1 -- 172.21.15.49:0/1439482986 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 208712561 0 0) 0x7fb30815cf90 con 0x7fb30815cbb0 2023-11-06T02:13:50.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.078+0000 7fb30ce80640 1 -- 172.21.15.49:0/1439482986 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2f4003530 con 0x7fb30815cbb0 2023-11-06T02:13:50.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.078+0000 7fb30ce80640 1 -- 172.21.15.49:0/1439482986 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3070602331 0 0) 0x7fb2f4003530 con 0x7fb30815cbb0 2023-11-06T02:13:50.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.078+0000 7fb30ce80640 1 -- 172.21.15.49:0/1439482986 >> v2:172.21.15.150:3300/0 conn(0x7fb3080d2a60 msgr2=0x7fb3080d2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:50.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.078+0000 7fb30ce80640 1 --2- 172.21.15.49:0/1439482986 >> v2:172.21.15.150:3300/0 conn(0x7fb3080d2a60 0x7fb3080d2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:50.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.078+0000 7fb30ce80640 1 -- 172.21.15.49:0/1439482986 >> v1:172.21.15.150:6789/0 conn(0x7fb3080b5d40 legacy=0x7fb3080b61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:50.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.078+0000 7fb30ce80640 1 -- 172.21.15.49:0/1439482986 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3080d0e60 con 0x7fb30815cbb0 2023-11-06T02:13:50.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.078+0000 7fb30ce80640 1 -- 172.21.15.49:0/1439482986 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7fb2f8001680 con 0x7fb30815cbb0 2023-11-06T02:13:50.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.078+0000 7fb30ce80640 1 -- 172.21.15.49:0/1439482986 >> v1:172.21.15.49:6789/0 conn(0x7fb30815cbb0 legacy=0x7fb3080d1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:50.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.078+0000 7fb30ce80640 1 --2- 172.21.15.49:0/1439482986 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2f4003980 0x7fb2f4003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:50.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.078+0000 7fb30ce80640 1 --2- 172.21.15.49:0/1439482986 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb2f40047d0 0x7fb2f4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:50.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.078+0000 7fb30ce80640 1 --2- 172.21.15.49:0/1439482986 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2f4005190 0x7fb2f400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:50.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.078+0000 7fb30ce80640 1 -- 172.21.15.49:0/1439482986 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3080d3410 con 0x7fb2f40047d0 2023-11-06T02:13:50.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.078+0000 7fb30ce80640 1 -- 172.21.15.49:0/1439482986 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f400c040 con 0x7fb2f4005190 2023-11-06T02:13:50.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.078+0000 7fb30ce80640 1 -- 172.21.15.49:0/1439482986 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f400c200 con 0x7fb2f4003980 2023-11-06T02:13:50.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.078+0000 7fb30d681640 1 --2- 172.21.15.49:0/1439482986 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb2f40047d0 0x7fb2f4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:50.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.078+0000 7fb30de82640 1 --2- 172.21.15.49:0/1439482986 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2f4003980 0x7fb2f4003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:50.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.078+0000 7fb30e683640 1 --2- 172.21.15.49:0/1439482986 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2f4005190 0x7fb2f400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:50.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.078+0000 7fb30d681640 1 -- 172.21.15.49:0/1439482986 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2f4003980 msgr2=0x7fb2f4003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:50.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.078+0000 7fb30d681640 1 --2- 172.21.15.49:0/1439482986 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2f4003980 0x7fb2f4003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:50.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.078+0000 7fb30d681640 1 -- 172.21.15.49:0/1439482986 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2f4005190 msgr2=0x7fb2f400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:50.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.078+0000 7fb30d681640 1 --2- 172.21.15.49:0/1439482986 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2f4005190 0x7fb2f400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:50.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.078+0000 7fb30d681640 1 -- 172.21.15.49:0/1439482986 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2f4003530 con 0x7fb2f40047d0 2023-11-06T02:13:50.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.078+0000 7fb30e683640 1 --2- 172.21.15.49:0/1439482986 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2f4005190 0x7fb2f400b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:13:50.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.078+0000 7fb30de82640 1 --2- 172.21.15.49:0/1439482986 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2f4003980 0x7fb2f4003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:13:50.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.078+0000 7fb30e683640 1 -- 172.21.15.49:0/1439482986 reap_dead start 2023-11-06T02:13:50.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.082+0000 7fb30d681640 1 --2- 172.21.15.49:0/1439482986 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb2f40047d0 0x7fb2f4004c50 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fb304009ae0 tx=0x7fb30401ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3bf7e69f30e94cd server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:50.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.082+0000 7fb30ce80640 1 -- 172.21.15.49:0/1439482986 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3040a5640 con 0x7fb2f40047d0 2023-11-06T02:13:50.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.082+0000 7fb30ce80640 1 -- 172.21.15.49:0/1439482986 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3040a57a0 con 0x7fb2f40047d0 2023-11-06T02:13:50.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.082+0000 7fb30ce80640 1 -- 172.21.15.49:0/1439482986 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3040a5b50 con 0x7fb2f40047d0 2023-11-06T02:13:50.087 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.082+0000 7fb3100e6640 1 -- 172.21.15.49:0/1439482986 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb2f40047d0 msgr2=0x7fb2f4004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:50.088 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.082+0000 7fb3100e6640 1 --2- 172.21.15.49:0/1439482986 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb2f40047d0 0x7fb2f4004c50 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fb304009ae0 tx=0x7fb30401ca50 comp rx=0 tx=0).stop 2023-11-06T02:13:50.088 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.082+0000 7fb3100e6640 1 -- 172.21.15.49:0/1439482986 shutdown_connections 2023-11-06T02:13:50.088 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.082+0000 7fb3100e6640 1 --2- 172.21.15.49:0/1439482986 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb2f40047d0 0x7fb2f4004c50 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:50.088 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.082+0000 7fb3100e6640 1 -- 172.21.15.49:0/1439482986 >> 172.21.15.49:0/1439482986 conn(0x7fb3080718a0 msgr2=0x7fb308071cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:50.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.082+0000 7fb3100e6640 1 -- 172.21.15.49:0/1439482986 shutdown_connections 2023-11-06T02:13:50.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.082+0000 7fb3100e6640 1 -- 172.21.15.49:0/1439482986 wait complete. 2023-11-06T02:13:50.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.082+0000 7fb3100e6640 1 Processor -- start 2023-11-06T02:13:50.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.082+0000 7fb3100e6640 1 -- start start 2023-11-06T02:13:50.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.086+0000 7fb3100e6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb2f40047d0 0x7fb3080c8de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:50.090 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.086+0000 7fb3100e6640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb3080b4910 0x7fb30806b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:50.090 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.086+0000 7fb3100e6640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb30806b540 0x7fb3080c7010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:50.090 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.086+0000 7fb3100e6640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb30806b9c0 con 0x7fb2f40047d0 2023-11-06T02:13:50.090 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.086+0000 7fb3100e6640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3080c7d40 con 0x7fb30806b540 2023-11-06T02:13:50.090 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.086+0000 7fb3100e6640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3080c8020 con 0x7fb3080b4910 2023-11-06T02:13:50.091 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.086+0000 7fb30de82640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb2f40047d0 0x7fb3080c8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:50.091 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.086+0000 7fb30de82640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb2f40047d0 0x7fb3080c8de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:32978/0 (socket says 172.21.15.49:32978) 2023-11-06T02:13:50.091 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.086+0000 7fb30de82640 1 -- 172.21.15.49:0/255851129 learned_addr learned my addr 172.21.15.49:0/255851129 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:50.091 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.086+0000 7fb30d681640 1 --2- 172.21.15.49:0/255851129 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb3080b4910 0x7fb30806b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:50.091 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.086+0000 7fb30e683640 1 --2- 172.21.15.49:0/255851129 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb30806b540 0x7fb3080c7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:50.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.086+0000 7fb30de82640 1 -- 172.21.15.49:0/255851129 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb3080b4910 msgr2=0x7fb30806b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:50.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.086+0000 7fb30de82640 1 --2- 172.21.15.49:0/255851129 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb3080b4910 0x7fb30806b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:50.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.086+0000 7fb30de82640 1 -- 172.21.15.49:0/255851129 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb30806b540 msgr2=0x7fb3080c7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:50.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.086+0000 7fb30de82640 1 --2- 172.21.15.49:0/255851129 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb30806b540 0x7fb3080c7010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:50.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.086+0000 7fb30de82640 1 -- 172.21.15.49:0/255851129 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3080c8300 con 0x7fb2f40047d0 2023-11-06T02:13:50.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.086+0000 7fb30e683640 1 --2- 172.21.15.49:0/255851129 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb30806b540 0x7fb3080c7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:13:50.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.086+0000 7fb30de82640 1 --2- 172.21.15.49:0/255851129 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb2f40047d0 0x7fb3080c8de0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fb2f800d7e0 tx=0x7fb2f800dca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:50.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.086+0000 7fb2fe7fc640 1 -- 172.21.15.49:0/255851129 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f80074e0 con 0x7fb2f40047d0 2023-11-06T02:13:50.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.086+0000 7fb2fe7fc640 1 -- 172.21.15.49:0/255851129 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2f8007640 con 0x7fb2f40047d0 2023-11-06T02:13:50.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.086+0000 7fb3100e6640 1 -- 172.21.15.49:0/255851129 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3080c85e0 con 0x7fb2f40047d0 2023-11-06T02:13:50.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.086+0000 7fb2fe7fc640 1 -- 172.21.15.49:0/255851129 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f8004a10 con 0x7fb2f40047d0 2023-11-06T02:13:50.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.086+0000 7fb3100e6640 1 -- 172.21.15.49:0/255851129 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3080c9320 con 0x7fb2f40047d0 2023-11-06T02:13:50.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.086+0000 7fb3100e6640 1 -- 172.21.15.49:0/255851129 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb308074040 con 0x7fb2f40047d0 2023-11-06T02:13:50.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.086+0000 7fb2fe7fc640 1 -- 172.21.15.49:0/255851129 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fb2f8004b70 con 0x7fb2f40047d0 2023-11-06T02:13:50.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.086+0000 7fb2fe7fc640 1 --2- 172.21.15.49:0/255851129 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb2ec065a90 0x7fb2ec067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:50.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.086+0000 7fb2fe7fc640 1 -- 172.21.15.49:0/255851129 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb2f8082fe0 con 0x7fb2f40047d0 2023-11-06T02:13:50.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.086+0000 7fb30d681640 1 --2- 172.21.15.49:0/255851129 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb2ec065a90 0x7fb2ec067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:50.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.090+0000 7fb30d681640 1 --2- 172.21.15.49:0/255851129 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb2ec065a90 0x7fb2ec067f50 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fb3040af000 tx=0x7fb304008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:50.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.090+0000 7fb2fe7fc640 1 -- 172.21.15.49:0/255851129 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fb2f80507e0 con 0x7fb2f40047d0 2023-11-06T02:13:50.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.246+0000 7fb3100e6640 1 -- 172.21.15.49:0/255851129 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb3080d13f0 con 0x7fb2f40047d0 2023-11-06T02:13:50.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.246+0000 7fb2fe7fc640 1 -- 172.21.15.49:0/255851129 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v80) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb2f80190a0 con 0x7fb2f40047d0 2023-11-06T02:13:50.241 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2023-11-06T02:13:50.243 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.246+0000 7fb3100e6640 1 -- 172.21.15.49:0/255851129 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb2ec065a90 msgr2=0x7fb2ec067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:50.243 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.246+0000 7fb3100e6640 1 --2- 172.21.15.49:0/255851129 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb2ec065a90 0x7fb2ec067f50 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fb3040af000 tx=0x7fb304008040 comp rx=0 tx=0).stop 2023-11-06T02:13:50.243 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.246+0000 7fb3100e6640 1 -- 172.21.15.49:0/255851129 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb2f40047d0 msgr2=0x7fb3080c8de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:50.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.246+0000 7fb3100e6640 1 --2- 172.21.15.49:0/255851129 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb2f40047d0 0x7fb3080c8de0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fb2f800d7e0 tx=0x7fb2f800dca0 comp rx=0 tx=0).stop 2023-11-06T02:13:50.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.246+0000 7fb3100e6640 1 -- 172.21.15.49:0/255851129 shutdown_connections 2023-11-06T02:13:50.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.246+0000 7fb3100e6640 1 --2- 172.21.15.49:0/255851129 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb2ec065a90 0x7fb2ec067f50 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:50.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.246+0000 7fb3100e6640 1 --2- 172.21.15.49:0/255851129 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb30806b540 0x7fb3080c7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:50.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.246+0000 7fb3100e6640 1 --2- 172.21.15.49:0/255851129 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb3080b4910 0x7fb30806b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:50.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.246+0000 7fb3100e6640 1 --2- 172.21.15.49:0/255851129 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb2f40047d0 0x7fb3080c8de0 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:50.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.246+0000 7fb3100e6640 1 -- 172.21.15.49:0/255851129 >> 172.21.15.49:0/255851129 conn(0x7fb3080718a0 msgr2=0x7fb30815d6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:50.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.246+0000 7fb3100e6640 1 -- 172.21.15.49:0/255851129 shutdown_connections 2023-11-06T02:13:50.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.246+0000 7fb3100e6640 1 -- 172.21.15.49:0/255851129 wait complete. 2023-11-06T02:13:50.260 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T02:13:50.389 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.394+0000 7f2f68bb9640 1 Processor -- start 2023-11-06T02:13:50.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.394+0000 7f2f68bb9640 1 -- start start 2023-11-06T02:13:50.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.394+0000 7f2f68bb9640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f2f6415a880 0x7f2f640b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:50.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.394+0000 7f2f68bb9640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2f6406d860 0x7f2f64070610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:50.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.394+0000 7f2f68bb9640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f640ae360 con 0x7f2f64070160 2023-11-06T02:13:50.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.394+0000 7f2f68bb9640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f64070b50 con 0x7f2f6415a880 2023-11-06T02:13:50.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.394+0000 7f2f68bb9640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f640ae640 con 0x7f2f6406d860 2023-11-06T02:13:50.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.394+0000 7f2f61d9b640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f2f64070160 0x7f2f6406d150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:55398/0 (socket says 172.21.15.49:55398) 2023-11-06T02:13:50.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.394+0000 7f2f61d9b640 1 -- 172.21.15.49:0/3996204081 learned_addr learned my addr 172.21.15.49:0/3996204081 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:13:50.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.394+0000 7f2f6259c640 1 --2- 172.21.15.49:0/3996204081 >> v2:172.21.15.150:3300/0 conn(0x7f2f6415a880 0x7f2f640b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:50.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.394+0000 7f2f62d9d640 1 --2- 172.21.15.49:0/3996204081 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2f6406d860 0x7f2f64070610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:50.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.394+0000 7f2f6159a640 1 -- 172.21.15.49:0/3996204081 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2806124451 0 0) 0x7f2f640ae360 con 0x7f2f64070160 2023-11-06T02:13:50.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.394+0000 7f2f6159a640 1 -- 172.21.15.49:0/3996204081 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f50003530 con 0x7f2f64070160 2023-11-06T02:13:50.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.394+0000 7f2f6159a640 1 -- 172.21.15.49:0/3996204081 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2333183620 0 0) 0x7f2f50003530 con 0x7f2f64070160 2023-11-06T02:13:50.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.394+0000 7f2f6159a640 1 -- 172.21.15.49:0/3996204081 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2f6406d860 msgr2=0x7f2f64070610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:50.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.394+0000 7f2f6159a640 1 --2- 172.21.15.49:0/3996204081 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2f6406d860 0x7f2f64070610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:50.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.394+0000 7f2f6159a640 1 -- 172.21.15.49:0/3996204081 >> v2:172.21.15.150:3300/0 conn(0x7f2f6415a880 msgr2=0x7f2f640b15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:50.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.394+0000 7f2f6159a640 1 --2- 172.21.15.49:0/3996204081 >> v2:172.21.15.150:3300/0 conn(0x7f2f6415a880 0x7f2f640b15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:50.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.394+0000 7f2f6159a640 1 -- 172.21.15.49:0/3996204081 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f640ae920 con 0x7f2f64070160 2023-11-06T02:13:50.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.394+0000 7f2f6159a640 1 -- 172.21.15.49:0/3996204081 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7f2f54001a00 con 0x7f2f64070160 2023-11-06T02:13:50.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.394+0000 7f2f6159a640 1 -- 172.21.15.49:0/3996204081 >> v1:172.21.15.49:6789/0 conn(0x7f2f64070160 legacy=0x7f2f6406d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:50.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.394+0000 7f2f6159a640 1 --2- 172.21.15.49:0/3996204081 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2f6406d860 0x7f2f50003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:50.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.394+0000 7f2f6159a640 1 --2- 172.21.15.49:0/3996204081 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f50003f00 0x7f2f500047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:50.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.394+0000 7f2f6159a640 1 --2- 172.21.15.49:0/3996204081 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f50004d10 0x7f2f50005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:50.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.394+0000 7f2f6159a640 1 -- 172.21.15.49:0/3996204081 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f64070b50 con 0x7f2f50003f00 2023-11-06T02:13:50.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.394+0000 7f2f6159a640 1 -- 172.21.15.49:0/3996204081 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f640ae640 con 0x7f2f50004d10 2023-11-06T02:13:50.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.394+0000 7f2f6159a640 1 -- 172.21.15.49:0/3996204081 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f5000b990 con 0x7f2f6406d860 2023-11-06T02:13:50.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.394+0000 7f2f61d9b640 1 --2- 172.21.15.49:0/3996204081 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f50003f00 0x7f2f500047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:50.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.394+0000 7f2f6259c640 1 --2- 172.21.15.49:0/3996204081 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2f6406d860 0x7f2f50003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:50.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.394+0000 7f2f61d9b640 1 -- 172.21.15.49:0/3996204081 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2f6406d860 msgr2=0x7f2f50003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:50.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.394+0000 7f2f61d9b640 1 --2- 172.21.15.49:0/3996204081 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2f6406d860 0x7f2f50003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:50.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.394+0000 7f2f61d9b640 1 -- 172.21.15.49:0/3996204081 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f50004d10 msgr2=0x7f2f50005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:50.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.394+0000 7f2f61d9b640 1 --2- 172.21.15.49:0/3996204081 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f50004d10 0x7f2f50005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:50.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.394+0000 7f2f61d9b640 1 -- 172.21.15.49:0/3996204081 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f50003530 con 0x7f2f50003f00 2023-11-06T02:13:50.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.398+0000 7f2f61d9b640 1 --2- 172.21.15.49:0/3996204081 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f50003f00 0x7f2f500047d0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f2f5400a8d0 tx=0x7f2f5401fae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25a9fad8de7ff603 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:50.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.398+0000 7f2f6159a640 1 -- 172.21.15.49:0/3996204081 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f540aa6e0 con 0x7f2f50003f00 2023-11-06T02:13:50.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.398+0000 7f2f6159a640 1 -- 172.21.15.49:0/3996204081 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f540aa840 con 0x7f2f50003f00 2023-11-06T02:13:50.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.398+0000 7f2f6159a640 1 -- 172.21.15.49:0/3996204081 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f540aabf0 con 0x7f2f50003f00 2023-11-06T02:13:50.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.398+0000 7f2f68bb9640 1 -- 172.21.15.49:0/3996204081 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f50003f00 msgr2=0x7f2f500047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:50.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.398+0000 7f2f68bb9640 1 --2- 172.21.15.49:0/3996204081 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f50003f00 0x7f2f500047d0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f2f5400a8d0 tx=0x7f2f5401fae0 comp rx=0 tx=0).stop 2023-11-06T02:13:50.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.398+0000 7f2f62d9d640 1 -- 172.21.15.49:0/3996204081 reap_dead start 2023-11-06T02:13:50.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.398+0000 7f2f68bb9640 1 -- 172.21.15.49:0/3996204081 shutdown_connections 2023-11-06T02:13:50.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.398+0000 7f2f68bb9640 1 -- 172.21.15.49:0/3996204081 >> 172.21.15.49:0/3996204081 conn(0x7f2f6406a370 msgr2=0x7f2f640ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:50.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.398+0000 7f2f68bb9640 1 -- 172.21.15.49:0/3996204081 shutdown_connections 2023-11-06T02:13:50.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.398+0000 7f2f68bb9640 1 -- 172.21.15.49:0/3996204081 wait complete. 2023-11-06T02:13:50.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.398+0000 7f2f68bb9640 1 Processor -- start 2023-11-06T02:13:50.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.398+0000 7f2f68bb9640 1 -- start start 2023-11-06T02:13:50.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.402+0000 7f2f68bb9640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f50003f00 0x7f2f640adc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:50.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.402+0000 7f2f68bb9640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2f641f0930 0x7f2f640c7b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:50.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.402+0000 7f2f68bb9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f640c8040 0x7f2f640c3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:50.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.402+0000 7f2f68bb9640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f640ae160 con 0x7f2f640c8040 2023-11-06T02:13:50.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.402+0000 7f2f68bb9640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f64157660 con 0x7f2f50003f00 2023-11-06T02:13:50.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.402+0000 7f2f68bb9640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f64157940 con 0x7f2f641f0930 2023-11-06T02:13:50.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.402+0000 7f2f62d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f640c8040 0x7f2f640c3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:50.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.402+0000 7f2f62d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f640c8040 0x7f2f640c3530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33000/0 (socket says 172.21.15.49:33000) 2023-11-06T02:13:50.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.402+0000 7f2f62d9d640 1 -- 172.21.15.49:0/2096802326 learned_addr learned my addr 172.21.15.49:0/2096802326 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:50.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.402+0000 7f2f62d9d640 1 -- 172.21.15.49:0/2096802326 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2f641f0930 msgr2=0x7f2f640c7b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:50.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.402+0000 7f2f62d9d640 1 --2- 172.21.15.49:0/2096802326 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2f641f0930 0x7f2f640c7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:50.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.402+0000 7f2f62d9d640 1 -- 172.21.15.49:0/2096802326 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f50003f00 msgr2=0x7f2f640adc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:50.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.402+0000 7f2f62d9d640 1 --2- 172.21.15.49:0/2096802326 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f50003f00 0x7f2f640adc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:50.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.402+0000 7f2f62d9d640 1 -- 172.21.15.49:0/2096802326 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f6415ade0 con 0x7f2f640c8040 2023-11-06T02:13:50.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.402+0000 7f2f62d9d640 1 --2- 172.21.15.49:0/2096802326 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f640c8040 0x7f2f640c3530 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f2f6406ae30 tx=0x7f2f5c00ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:50.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.402+0000 7f2f5affd640 1 -- 172.21.15.49:0/2096802326 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f5c00c680 con 0x7f2f640c8040 2023-11-06T02:13:50.411 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.402+0000 7f2f5affd640 1 -- 172.21.15.49:0/2096802326 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f5c00c800 con 0x7f2f640c8040 2023-11-06T02:13:50.411 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.402+0000 7f2f68bb9640 1 -- 172.21.15.49:0/2096802326 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f64157c20 con 0x7f2f640c8040 2023-11-06T02:13:50.411 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.402+0000 7f2f5affd640 1 -- 172.21.15.49:0/2096802326 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f5c00cbb0 con 0x7f2f640c8040 2023-11-06T02:13:50.411 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.402+0000 7f2f68bb9640 1 -- 172.21.15.49:0/2096802326 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f641581e0 con 0x7f2f640c8040 2023-11-06T02:13:50.411 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.402+0000 7f2f68bb9640 1 -- 172.21.15.49:0/2096802326 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f64074040 con 0x7f2f640c8040 2023-11-06T02:13:50.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.402+0000 7f2f5affd640 1 -- 172.21.15.49:0/2096802326 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f2f5c00cd10 con 0x7f2f640c8040 2023-11-06T02:13:50.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.402+0000 7f2f5affd640 1 --2- 172.21.15.49:0/2096802326 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2f44065a60 0x7f2f44067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:50.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.402+0000 7f2f5affd640 1 -- 172.21.15.49:0/2096802326 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2f5c082b90 con 0x7f2f640c8040 2023-11-06T02:13:50.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.402+0000 7f2f6259c640 1 --2- 172.21.15.49:0/2096802326 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2f44065a60 0x7f2f44067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:50.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.406+0000 7f2f6259c640 1 --2- 172.21.15.49:0/2096802326 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2f44065a60 0x7f2f44067f20 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f2f640b4440 tx=0x7f2f4c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:50.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.406+0000 7f2f5affd640 1 -- 172.21.15.49:0/2096802326 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f2f5c0503f0 con 0x7f2f640c8040 2023-11-06T02:13:50.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.562+0000 7f2f68bb9640 1 -- 172.21.15.49:0/2096802326 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2f6406c430 con 0x7f2f640c8040 2023-11-06T02:13:50.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.562+0000 7f2f5affd640 1 -- 172.21.15.49:0/2096802326 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v80) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2f5c053a10 con 0x7f2f640c8040 2023-11-06T02:13:50.559 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2023-11-06T02:13:50.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.566+0000 7f2f68bb9640 1 -- 172.21.15.49:0/2096802326 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2f44065a60 msgr2=0x7f2f44067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:50.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.566+0000 7f2f68bb9640 1 --2- 172.21.15.49:0/2096802326 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2f44065a60 0x7f2f44067f20 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f2f640b4440 tx=0x7f2f4c008040 comp rx=0 tx=0).stop 2023-11-06T02:13:50.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.566+0000 7f2f68bb9640 1 -- 172.21.15.49:0/2096802326 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f640c8040 msgr2=0x7f2f640c3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:50.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.566+0000 7f2f68bb9640 1 --2- 172.21.15.49:0/2096802326 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f640c8040 0x7f2f640c3530 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f2f6406ae30 tx=0x7f2f5c00ec00 comp rx=0 tx=0).stop 2023-11-06T02:13:50.563 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.566+0000 7f2f68bb9640 1 -- 172.21.15.49:0/2096802326 shutdown_connections 2023-11-06T02:13:50.563 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.566+0000 7f2f68bb9640 1 --2- 172.21.15.49:0/2096802326 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2f44065a60 0x7f2f44067f20 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:50.563 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.566+0000 7f2f68bb9640 1 --2- 172.21.15.49:0/2096802326 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f640c8040 0x7f2f640c3530 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:50.563 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.566+0000 7f2f68bb9640 1 --2- 172.21.15.49:0/2096802326 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2f641f0930 0x7f2f640c7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:50.563 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.566+0000 7f2f68bb9640 1 --2- 172.21.15.49:0/2096802326 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f50003f00 0x7f2f640adc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:50.563 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.566+0000 7f2f68bb9640 1 -- 172.21.15.49:0/2096802326 >> 172.21.15.49:0/2096802326 conn(0x7f2f6406a370 msgr2=0x7f2f6415da30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:50.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.566+0000 7f2f68bb9640 1 -- 172.21.15.49:0/2096802326 shutdown_connections 2023-11-06T02:13:50.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.566+0000 7f2f68bb9640 1 -- 172.21.15.49:0/2096802326 wait complete. 2023-11-06T02:13:50.579 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-06T02:13:50.705 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.710+0000 7fb83a179640 1 Processor -- start 2023-11-06T02:13:50.706 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.710+0000 7fb83a179640 1 -- start start 2023-11-06T02:13:50.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.710+0000 7fb83a179640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fb83415ab50 0x7fb8340c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:50.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.710+0000 7fb83a179640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb8340cb010 0x7fb8340b0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:50.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.710+0000 7fb83a179640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb8340b05c0 0x7fb8340700a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:50.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.710+0000 7fb83a179640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8340b0a50 con 0x7fb8340cb010 2023-11-06T02:13:50.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.710+0000 7fb83a179640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8340cab80 con 0x7fb83415ab50 2023-11-06T02:13:50.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.710+0000 7fb83a179640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb83415af30 con 0x7fb8340b05c0 2023-11-06T02:13:50.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.710+0000 7fb832ffd640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb8340cb010 0x7fb8340b0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:50.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.710+0000 7fb832ffd640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb8340cb010 0x7fb8340b0080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33006/0 (socket says 172.21.15.49:33006) 2023-11-06T02:13:50.709 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.710+0000 7fb832ffd640 1 -- 172.21.15.49:0/474102999 learned_addr learned my addr 172.21.15.49:0/474102999 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:50.709 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.710+0000 7fb833fff640 1 --2- 172.21.15.49:0/474102999 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb8340b05c0 0x7fb8340700a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:50.709 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.710+0000 7fb8337fe640 1 --2- 172.21.15.49:0/474102999 >> v2:172.21.15.150:3300/0 conn(0x7fb83415ab50 0x7fb8340c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:50.709 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.710+0000 7fb832ffd640 1 -- 172.21.15.49:0/474102999 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb8340b05c0 msgr2=0x7fb8340700a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:50.709 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.710+0000 7fb832ffd640 1 --2- 172.21.15.49:0/474102999 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb8340b05c0 0x7fb8340700a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:50.710 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.710+0000 7fb832ffd640 1 -- 172.21.15.49:0/474102999 >> v2:172.21.15.150:3300/0 conn(0x7fb83415ab50 msgr2=0x7fb8340c9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:50.710 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.710+0000 7fb832ffd640 1 --2- 172.21.15.49:0/474102999 >> v2:172.21.15.150:3300/0 conn(0x7fb83415ab50 0x7fb8340c9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:50.710 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.710+0000 7fb832ffd640 1 -- 172.21.15.49:0/474102999 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8340cb5b0 con 0x7fb8340cb010 2023-11-06T02:13:50.710 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.710+0000 7fb8337fe640 1 --2- 172.21.15.49:0/474102999 >> v2:172.21.15.150:3300/0 conn(0x7fb83415ab50 0x7fb8340c9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:13:50.710 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.710+0000 7fb833fff640 1 --2- 172.21.15.49:0/474102999 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb8340b05c0 0x7fb8340700a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:13:50.713 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.714+0000 7fb832ffd640 1 --2- 172.21.15.49:0/474102999 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb8340cb010 0x7fb8340b0080 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fb8340725a0 tx=0x7fb81c01c210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a908b1fc4fa0490 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:50.713 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.714+0000 7fb8327fc640 1 -- 172.21.15.49:0/474102999 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81c0ab600 con 0x7fb8340cb010 2023-11-06T02:13:50.713 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.714+0000 7fb8327fc640 1 -- 172.21.15.49:0/474102999 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb81c0a7050 con 0x7fb8340cb010 2023-11-06T02:13:50.714 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.714+0000 7fb8327fc640 1 -- 172.21.15.49:0/474102999 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81c0ab980 con 0x7fb8340cb010 2023-11-06T02:13:50.714 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.714+0000 7fb83a179640 1 -- 172.21.15.49:0/474102999 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb8340cb010 msgr2=0x7fb8340b0080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:50.714 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.714+0000 7fb83a179640 1 --2- 172.21.15.49:0/474102999 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb8340cb010 0x7fb8340b0080 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fb8340725a0 tx=0x7fb81c01c210 comp rx=0 tx=0).stop 2023-11-06T02:13:50.714 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.714+0000 7fb83a179640 1 -- 172.21.15.49:0/474102999 shutdown_connections 2023-11-06T02:13:50.714 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.714+0000 7fb83a179640 1 --2- 172.21.15.49:0/474102999 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb8340b05c0 0x7fb8340700a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:50.715 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.714+0000 7fb83a179640 1 --2- 172.21.15.49:0/474102999 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb8340cb010 0x7fb8340b0080 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:50.715 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.714+0000 7fb83a179640 1 --2- 172.21.15.49:0/474102999 >> v2:172.21.15.150:3300/0 conn(0x7fb83415ab50 0x7fb8340c9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:50.715 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.714+0000 7fb83a179640 1 -- 172.21.15.49:0/474102999 >> 172.21.15.49:0/474102999 conn(0x7fb83406c0a0 msgr2=0x7fb83406c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:50.715 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.714+0000 7fb83a179640 1 -- 172.21.15.49:0/474102999 shutdown_connections 2023-11-06T02:13:50.715 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.714+0000 7fb83a179640 1 -- 172.21.15.49:0/474102999 wait complete. 2023-11-06T02:13:50.716 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.714+0000 7fb83a179640 1 Processor -- start 2023-11-06T02:13:50.716 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.714+0000 7fb83a179640 1 -- start start 2023-11-06T02:13:50.716 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.718+0000 7fb83a179640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8340b05c0 0x7fb8340b0040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:50.716 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.718+0000 7fb83a179640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb8340cb010 0x7fb8340ac990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:50.716 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.718+0000 7fb83a179640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb83415ab50 0x7fb834150eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:50.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.718+0000 7fb83a179640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8340aced0 con 0x7fb8340cb010 2023-11-06T02:13:50.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.718+0000 7fb83a179640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb834151700 con 0x7fb8340b05c0 2023-11-06T02:13:50.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.718+0000 7fb83a179640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8341519e0 con 0x7fb83415ab50 2023-11-06T02:13:50.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.718+0000 7fb832ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb8340cb010 0x7fb8340ac990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:50.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.718+0000 7fb832ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb8340cb010 0x7fb8340ac990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33014/0 (socket says 172.21.15.49:33014) 2023-11-06T02:13:50.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.718+0000 7fb832ffd640 1 -- 172.21.15.49:0/355655116 learned_addr learned my addr 172.21.15.49:0/355655116 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:50.718 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.718+0000 7fb8337fe640 1 --2- 172.21.15.49:0/355655116 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8340b05c0 0x7fb8340b0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:50.718 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.718+0000 7fb833fff640 1 --2- 172.21.15.49:0/355655116 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb83415ab50 0x7fb834150eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:50.718 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.718+0000 7fb832ffd640 1 -- 172.21.15.49:0/355655116 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb83415ab50 msgr2=0x7fb834150eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:50.718 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.718+0000 7fb832ffd640 1 --2- 172.21.15.49:0/355655116 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb83415ab50 0x7fb834150eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:50.718 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.718+0000 7fb832ffd640 1 -- 172.21.15.49:0/355655116 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8340b05c0 msgr2=0x7fb8340b0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:50.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.718+0000 7fb832ffd640 1 --2- 172.21.15.49:0/355655116 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8340b05c0 0x7fb8340b0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:50.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.718+0000 7fb832ffd640 1 -- 172.21.15.49:0/355655116 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb81c0094d0 con 0x7fb8340cb010 2023-11-06T02:13:50.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.718+0000 7fb8337fe640 1 --2- 172.21.15.49:0/355655116 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8340b05c0 0x7fb8340b0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:13:50.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.718+0000 7fb832ffd640 1 --2- 172.21.15.49:0/355655116 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb8340cb010 0x7fb8340ac990 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fb81c01a8c0 tx=0x7fb81c018fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:50.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.718+0000 7fb82bfff640 1 -- 172.21.15.49:0/355655116 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81c0b37d0 con 0x7fb8340cb010 2023-11-06T02:13:50.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.718+0000 7fb82bfff640 1 -- 172.21.15.49:0/355655116 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb81c0b3930 con 0x7fb8340cb010 2023-11-06T02:13:50.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.718+0000 7fb83a179640 1 -- 172.21.15.49:0/355655116 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb834151cc0 con 0x7fb8340cb010 2023-11-06T02:13:50.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.718+0000 7fb82bfff640 1 -- 172.21.15.49:0/355655116 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81c0b3ca0 con 0x7fb8340cb010 2023-11-06T02:13:50.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.718+0000 7fb83a179640 1 -- 172.21.15.49:0/355655116 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb834152280 con 0x7fb8340cb010 2023-11-06T02:13:50.721 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.718+0000 7fb83a179640 1 -- 172.21.15.49:0/355655116 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8340b0de0 con 0x7fb8340cb010 2023-11-06T02:13:50.722 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.718+0000 7fb82bfff640 1 -- 172.21.15.49:0/355655116 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fb81c0b2070 con 0x7fb8340cb010 2023-11-06T02:13:50.722 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.718+0000 7fb82bfff640 1 --2- 172.21.15.49:0/355655116 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb814065a60 0x7fb814067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:50.722 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.718+0000 7fb82bfff640 1 -- 172.21.15.49:0/355655116 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb81c11d570 con 0x7fb8340cb010 2023-11-06T02:13:50.722 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.718+0000 7fb8337fe640 1 --2- 172.21.15.49:0/355655116 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb814065a60 0x7fb814067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:50.722 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.722+0000 7fb8337fe640 1 --2- 172.21.15.49:0/355655116 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb814065a60 0x7fb814067f20 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fb82c009af0 tx=0x7fb82c003a40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:50.723 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.722+0000 7fb82bfff640 1 -- 172.21.15.49:0/355655116 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fb81c0eadd0 con 0x7fb8340cb010 2023-11-06T02:13:50.888 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.894+0000 7fb83a179640 1 -- 172.21.15.49:0/355655116 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb834074040 con 0x7fb8340cb010 2023-11-06T02:13:50.889 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.894+0000 7fb82bfff640 1 -- 172.21.15.49:0/355655116 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v80) v1 ==== 80+0+0 (secure 0 0 0) 0x7fb81c0ee3f0 con 0x7fb8340cb010 2023-11-06T02:13:50.889 INFO:teuthology.orchestra.run.smithi049.stderr:listed 0 entries 2023-11-06T02:13:50.891 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.894+0000 7fb83a179640 1 -- 172.21.15.49:0/355655116 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb814065a60 msgr2=0x7fb814067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:50.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.894+0000 7fb83a179640 1 --2- 172.21.15.49:0/355655116 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb814065a60 0x7fb814067f20 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fb82c009af0 tx=0x7fb82c003a40 comp rx=0 tx=0).stop 2023-11-06T02:13:50.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.894+0000 7fb83a179640 1 -- 172.21.15.49:0/355655116 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb8340cb010 msgr2=0x7fb8340ac990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:50.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.894+0000 7fb83a179640 1 --2- 172.21.15.49:0/355655116 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb8340cb010 0x7fb8340ac990 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fb81c01a8c0 tx=0x7fb81c018fb0 comp rx=0 tx=0).stop 2023-11-06T02:13:50.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.894+0000 7fb83a179640 1 -- 172.21.15.49:0/355655116 shutdown_connections 2023-11-06T02:13:50.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.894+0000 7fb83a179640 1 --2- 172.21.15.49:0/355655116 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb814065a60 0x7fb814067f20 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:50.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.894+0000 7fb83a179640 1 --2- 172.21.15.49:0/355655116 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb8340cb010 0x7fb8340ac990 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:50.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.894+0000 7fb83a179640 1 --2- 172.21.15.49:0/355655116 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb83415ab50 0x7fb834150eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:50.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.894+0000 7fb83a179640 1 --2- 172.21.15.49:0/355655116 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8340b05c0 0x7fb8340b0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:50.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.894+0000 7fb83a179640 1 -- 172.21.15.49:0/355655116 >> 172.21.15.49:0/355655116 conn(0x7fb83406c0a0 msgr2=0x7fb8340c3d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:50.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.894+0000 7fb83a179640 1 -- 172.21.15.49:0/355655116 shutdown_connections 2023-11-06T02:13:50.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:50.894+0000 7fb83a179640 1 -- 172.21.15.49:0/355655116 wait complete. 2023-11-06T02:13:50.909 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-11-06T02:13:50.909 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T02:13:50.915 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T02:13:51.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.086+0000 7fafdcde0640 1 Processor -- start 2023-11-06T02:13:51.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.086+0000 7fafdcde0640 1 -- start start 2023-11-06T02:13:51.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.086+0000 7fafdcde0640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fafd80b1f90 0x7fafd80cda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:51.085 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.086+0000 7fafdcde0640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafd80d17c0 con 0x7fafd8161650 2023-11-06T02:13:51.085 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.086+0000 7fafdcde0640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafd80ce6c0 con 0x7fafd80714a0 2023-11-06T02:13:51.085 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.086+0000 7fafdcde0640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafd8161ad0 con 0x7fafd80b1f90 2023-11-06T02:13:51.085 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.086+0000 7fafd5d9b640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fafd8161650 0x7fafd80d1c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:55400/0 (socket says 172.21.15.49:55400) 2023-11-06T02:13:51.085 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.086+0000 7fafd5d9b640 1 -- 172.21.15.49:0/3183762098 learned_addr learned my addr 172.21.15.49:0/3183762098 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:13:51.086 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.086+0000 7fafd659c640 1 --2- 172.21.15.49:0/3183762098 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fafd80b1f90 0x7fafd80cda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:51.086 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.086+0000 7fafd559a640 1 -- 172.21.15.49:0/3183762098 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 545183962 0 0) 0x7fafd80d17c0 con 0x7fafd8161650 2023-11-06T02:13:51.086 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.086+0000 7fafd559a640 1 -- 172.21.15.49:0/3183762098 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fafc0003530 con 0x7fafd8161650 2023-11-06T02:13:51.086 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.086+0000 7fafd559a640 1 -- 172.21.15.49:0/3183762098 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1774904746 0 0) 0x7fafc0003530 con 0x7fafd8161650 2023-11-06T02:13:51.086 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.086+0000 7fafd559a640 1 -- 172.21.15.49:0/3183762098 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fafd80b1f90 msgr2=0x7fafd80cda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:51.086 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.086+0000 7fafd559a640 1 --2- 172.21.15.49:0/3183762098 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fafd80b1f90 0x7fafd80cda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:51.087 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.086+0000 7fafd559a640 1 -- 172.21.15.49:0/3183762098 >> v1:172.21.15.150:6789/0 conn(0x7fafd80714a0 legacy=0x7fafd80b6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:51.087 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.086+0000 7fafd559a640 1 -- 172.21.15.49:0/3183762098 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafd80d2360 con 0x7fafd8161650 2023-11-06T02:13:51.087 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.086+0000 7fafd559a640 1 -- 172.21.15.49:0/3183762098 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7fafcc001680 con 0x7fafd8161650 2023-11-06T02:13:51.087 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.086+0000 7fafd559a640 1 -- 172.21.15.49:0/3183762098 >> v1:172.21.15.49:6789/0 conn(0x7fafd8161650 legacy=0x7fafd80d1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:51.087 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.090+0000 7fafd559a640 1 --2- 172.21.15.49:0/3183762098 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fafd80b1f90 0x7fafc00039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:51.088 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.090+0000 7fafd559a640 1 --2- 172.21.15.49:0/3183762098 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fafc0003f30 0x7fafc00047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:51.088 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.090+0000 7fafd559a640 1 --2- 172.21.15.49:0/3183762098 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fafc0004d10 0x7fafc0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:51.088 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.090+0000 7fafd559a640 1 -- 172.21.15.49:0/3183762098 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafd8161ad0 con 0x7fafc0004d10 2023-11-06T02:13:51.088 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.090+0000 7fafd559a640 1 -- 172.21.15.49:0/3183762098 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafc000ba70 con 0x7fafc0003f30 2023-11-06T02:13:51.088 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.090+0000 7fafd559a640 1 -- 172.21.15.49:0/3183762098 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafc000bc30 con 0x7fafd80b1f90 2023-11-06T02:13:51.088 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.090+0000 7fafd6d9d640 1 --2- 172.21.15.49:0/3183762098 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fafc0004d10 0x7fafc0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:51.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.090+0000 7fafd659c640 1 --2- 172.21.15.49:0/3183762098 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fafd80b1f90 0x7fafc00039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:51.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.090+0000 7fafd5d9b640 1 --2- 172.21.15.49:0/3183762098 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fafc0003f30 0x7fafc00047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:51.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.090+0000 7fafd6d9d640 1 -- 172.21.15.49:0/3183762098 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fafd80b1f90 msgr2=0x7fafc00039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:51.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.090+0000 7fafd6d9d640 1 --2- 172.21.15.49:0/3183762098 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fafd80b1f90 0x7fafc00039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:51.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.090+0000 7fafd6d9d640 1 -- 172.21.15.49:0/3183762098 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fafc0003f30 msgr2=0x7fafc00047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:51.090 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.090+0000 7fafd6d9d640 1 --2- 172.21.15.49:0/3183762098 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fafc0003f30 0x7fafc00047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:51.090 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.090+0000 7fafd6d9d640 1 -- 172.21.15.49:0/3183762098 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafc0003530 con 0x7fafc0004d10 2023-11-06T02:13:51.090 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.090+0000 7fafd5d9b640 1 --2- 172.21.15.49:0/3183762098 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fafc0003f30 0x7fafc00047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:13:51.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.090+0000 7fafd6d9d640 1 --2- 172.21.15.49:0/3183762098 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fafc0004d10 0x7fafc0005190 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fafd0001780 tx=0x7fafd001c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=147d51c18d0ffd1a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:51.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.090+0000 7fafd559a640 1 -- 172.21.15.49:0/3183762098 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd00052d0 con 0x7fafc0004d10 2023-11-06T02:13:51.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.090+0000 7fafd559a640 1 -- 172.21.15.49:0/3183762098 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafd00a7050 con 0x7fafc0004d10 2023-11-06T02:13:51.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.090+0000 7fafd559a640 1 -- 172.21.15.49:0/3183762098 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd0003600 con 0x7fafc0004d10 2023-11-06T02:13:51.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.090+0000 7fafdcde0640 1 -- 172.21.15.49:0/3183762098 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fafc0004d10 msgr2=0x7fafc0005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:51.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.090+0000 7fafdcde0640 1 --2- 172.21.15.49:0/3183762098 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fafc0004d10 0x7fafc0005190 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fafd0001780 tx=0x7fafd001c400 comp rx=0 tx=0).stop 2023-11-06T02:13:51.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.090+0000 7fafd6d9d640 1 -- 172.21.15.49:0/3183762098 reap_dead start 2023-11-06T02:13:51.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.090+0000 7fafdcde0640 1 -- 172.21.15.49:0/3183762098 shutdown_connections 2023-11-06T02:13:51.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.090+0000 7fafdcde0640 1 -- 172.21.15.49:0/3183762098 >> 172.21.15.49:0/3183762098 conn(0x7fafd80d2aa0 msgr2=0x7fafd80d2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:51.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.090+0000 7fafdcde0640 1 -- 172.21.15.49:0/3183762098 shutdown_connections 2023-11-06T02:13:51.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.090+0000 7fafdcde0640 1 -- 172.21.15.49:0/3183762098 wait complete. 2023-11-06T02:13:51.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.094+0000 7fafdcde0640 1 Processor -- start 2023-11-06T02:13:51.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.094+0000 7fafdcde0640 1 -- start start 2023-11-06T02:13:51.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.094+0000 7fafdcde0640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fafd815f0f0 0x7fafd80ab3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:51.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.094+0000 7fafdcde0640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fafd80b12e0 0x7fafd80ab8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:51.096 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.094+0000 7fafdcde0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fafd80b23c0 0x7fafd81f5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:51.096 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.094+0000 7fafdcde0640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafd80a9640 con 0x7fafd80b23c0 2023-11-06T02:13:51.096 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.094+0000 7fafdcde0640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafd80b3280 con 0x7fafd815f0f0 2023-11-06T02:13:51.096 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.094+0000 7fafdcde0640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafd80b53d0 con 0x7fafd80b12e0 2023-11-06T02:13:51.096 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.094+0000 7fafd6d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fafd80b23c0 0x7fafd81f5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:51.097 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.094+0000 7fafd6d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fafd80b23c0 0x7fafd81f5f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33032/0 (socket says 172.21.15.49:33032) 2023-11-06T02:13:51.097 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.094+0000 7fafd6d9d640 1 -- 172.21.15.49:0/432416449 learned_addr learned my addr 172.21.15.49:0/432416449 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:51.097 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.094+0000 7fafd5d9b640 1 --2- 172.21.15.49:0/432416449 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fafd80b12e0 0x7fafd80ab8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:51.097 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.094+0000 7fafd659c640 1 --2- 172.21.15.49:0/432416449 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fafd815f0f0 0x7fafd80ab3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:51.097 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.094+0000 7fafd6d9d640 1 -- 172.21.15.49:0/432416449 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fafd80b12e0 msgr2=0x7fafd80ab8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:51.097 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.094+0000 7fafd6d9d640 1 --2- 172.21.15.49:0/432416449 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fafd80b12e0 0x7fafd80ab8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:51.098 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.094+0000 7fafd6d9d640 1 -- 172.21.15.49:0/432416449 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fafd815f0f0 msgr2=0x7fafd80ab3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:51.098 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.094+0000 7fafd6d9d640 1 --2- 172.21.15.49:0/432416449 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fafd815f0f0 0x7fafd80ab3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:51.098 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.094+0000 7fafd6d9d640 1 -- 172.21.15.49:0/432416449 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafd80b3530 con 0x7fafd80b23c0 2023-11-06T02:13:51.098 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.094+0000 7fafd6d9d640 1 --2- 172.21.15.49:0/432416449 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fafd80b23c0 0x7fafd81f5f00 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fafd0005040 tx=0x7fafc0005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:51.098 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.094+0000 7fafc6ffd640 1 -- 172.21.15.49:0/432416449 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd0005500 con 0x7fafd80b23c0 2023-11-06T02:13:51.099 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.094+0000 7fafc6ffd640 1 -- 172.21.15.49:0/432416449 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafd0003cc0 con 0x7fafd80b23c0 2023-11-06T02:13:51.099 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.094+0000 7fafdcde0640 1 -- 172.21.15.49:0/432416449 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafd80b3810 con 0x7fafd80b23c0 2023-11-06T02:13:51.099 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.094+0000 7fafc6ffd640 1 -- 172.21.15.49:0/432416449 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd0007230 con 0x7fafd80b23c0 2023-11-06T02:13:51.099 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.094+0000 7fafdcde0640 1 -- 172.21.15.49:0/432416449 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafd815f4d0 con 0x7fafd80b23c0 2023-11-06T02:13:51.099 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.094+0000 7fafdcde0640 1 -- 172.21.15.49:0/432416449 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafd80a93a0 con 0x7fafd80b23c0 2023-11-06T02:13:51.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.098+0000 7fafc6ffd640 1 -- 172.21.15.49:0/432416449 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fafd0007390 con 0x7fafd80b23c0 2023-11-06T02:13:51.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.098+0000 7fafc6ffd640 1 --2- 172.21.15.49:0/432416449 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fafb8065a60 0x7fafb8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:51.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.098+0000 7fafc6ffd640 1 -- 172.21.15.49:0/432416449 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fafd011dae0 con 0x7fafd80b23c0 2023-11-06T02:13:51.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.098+0000 7fafd659c640 1 --2- 172.21.15.49:0/432416449 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fafb8065a60 0x7fafb8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:51.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.098+0000 7fafd659c640 1 --2- 172.21.15.49:0/432416449 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fafb8065a60 0x7fafb8067f20 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fafc8007520 tx=0x7fafc800b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:51.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.102+0000 7fafc6ffd640 1 -- 172.21.15.49:0/432416449 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fafd00eb4b0 con 0x7fafd80b23c0 2023-11-06T02:13:51.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.258+0000 7fafdcde0640 1 -- 172.21.15.49:0/432416449 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fafd80a9780 con 0x7fafd80b23c0 2023-11-06T02:13:51.255 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.258+0000 7fafc6ffd640 1 -- 172.21.15.49:0/432416449 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+17886 (secure 0 0 0) 0x7fafd80a9780 con 0x7fafd80b23c0 2023-11-06T02:13:51.255 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:13:51.255 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":80,"fsid":"1a8b035c-6ba4-499a-b876-aea47fd9c0d4","created":"2023-11-06T02:08:06.440857+0000","modified":"2023-11-06T02:13:34.630031+0000","last_up_change":"2023-11-06T02:08:17.982372+0000","last_in_change":"2023-11-06T02:08:08.906631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T02:08:19.252222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T02:08:24.776211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-11-06T02:13:31.459314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-11-06T02:13:32.603710+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ad55af8-0571-4e6a-8722-b5d429b7496d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6809","nonce":38116},{"type":"v1","addr":"172.21.15.49:6811","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6813","nonce":38116},{"type":"v1","addr":"172.21.15.49:6815","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6821","nonce":38116},{"type":"v1","addr":"172.21.15.49:6823","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6817","nonce":38116},{"type":"v1","addr":"172.21.15.49:6819","nonce":38116}]},"public_addr":"172.21.15.49:6811/38116","cluster_addr":"172.21.15.49:6815/38116","heartbeat_back_addr":"172.21.15.49:6823/38116","heartbeat_front_addr":"172.21.15.49:6819/38116","state":["exists","up"]},{"osd":1,"uuid":"c8b31f85-a757-43ed-9419-351acc46611b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":38113},{"type":"v1","addr":"172.21.15.49:6810","nonce":38113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":38113},{"type":"v1","addr":"172.21.15.49:6814","nonce":38113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":38113},{"type":"v1","addr":"172.21.15.49:6822","nonce":38113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":38113},{"type":"v1","addr":"172.21.15.49:6818","nonce":38113}]},"public_addr":"172.21.15.49:6810/38113","cluster_addr":"172.21.15.49:6814/38113","heartbeat_back_addr":"172.21.15.49:6822/38113","heartbeat_front_addr":"172.21.15.49:6818/38113","state":["exists","up"]},{"osd":2,"uuid":"555b3bf3-0a70-4b91-8efd-e4204b5fc26e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":38114},{"type":"v1","addr":"172.21.15.49:6801","nonce":38114}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":38114},{"type":"v1","addr":"172.21.15.49:6803","nonce":38114}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":38114},{"type":"v1","addr":"172.21.15.49:6807","nonce":38114}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":38114},{"type":"v1","addr":"172.21.15.49:6805","nonce":38114}]},"public_addr":"172.21.15.49:6801/38114","cluster_addr":"172.21.15.49:6803/38114","heartbeat_back_addr":"172.21.15.49:6807/38114","heartbeat_front_addr":"172.21.15.49:6805/38114","state":["exists","up"]},{"osd":3,"uuid":"8d4e5d77-03ef-4b9b-bfcb-5ae93a46795c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":38115},{"type":"v1","addr":"172.21.15.49:6825","nonce":38115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":38115},{"type":"v1","addr":"172.21.15.49:6827","nonce":38115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":38115},{"type":"v1","addr":"172.21.15.49:6831","nonce":38115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":38115},{"type":"v1","addr":"172.21.15.49:6829","nonce":38115}]},"public_addr":"172.21.15.49:6825/38115","cluster_addr":"172.21.15.49:6827/38115","heartbeat_back_addr":"172.21.15.49:6831/38115","heartbeat_front_addr":"172.21.15.49:6829/38115","state":["exists","up"]},{"osd":4,"uuid":"dba03ad4-68a3-4cf3-9218-ac611f02eacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":38417},{"type":"v1","addr":"172.21.15.150:6809","nonce":38417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":38417},{"type":"v1","addr":"172.21.15.150:6811","nonce":38417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":38417},{"type":"v1","addr":"172.21.15.150:6815","nonce":38417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":38417},{"type":"v1","addr":"172.21.15.150:6813","nonce":38417}]},"public_addr":"172.21.15.150:6809/38417","cluster_addr":"172.21.15.150:6811/38417","heartbeat_back_addr":"172.21.15.150:6815/38417","heartbeat_front_addr":"172.21.15.150:6813/38417","state":["exists","up"]},{"osd":5,"uuid":"0df9c303-c255-4640-8e91-01f1c3efe18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":38418},{"type":"v1","addr":"172.21.15.150:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":38418},{"type":"v1","addr":"172.21.15.150:6803","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":38418},{"type":"v1","addr":"172.21.15.150:6807","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":38418},{"type":"v1","addr":"172.21.15.150:6805","nonce":38418}]},"public_addr":"172.21.15.150:6801/38418","cluster_addr":"172.21.15.150:6803/38418","heartbeat_back_addr":"172.21.15.150:6807/38418","heartbeat_front_addr":"172.21.15.150:6805/38418","state":["exists","up"]},{"osd":6,"uuid":"e59fa189-4b70-4c06-9722-c02b90828bf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":38428},{"type":"v1","addr":"172.21.15.150:6825","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":38428},{"type":"v1","addr":"172.21.15.150:6827","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":38428},{"type":"v1","addr":"172.21.15.150:6831","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":38428},{"type":"v1","addr":"172.21.15.150:6829","nonce":38428}]},"public_addr":"172.21.15.150:6825/38428","cluster_addr":"172.21.15.150:6827/38428","heartbeat_back_addr":"172.21.15.150:6831/38428","heartbeat_front_addr":"172.21.15.150:6829/38428","state":["exists","up"]},{"osd":7,"uuid":"a64f57e1-ef98-46bd-b304-385759e95898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":38421},{"type":"v1","addr":"172.21.15.150:6817","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":38421},{"type":"v1","addr":"172.21.15.150:6819","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":38421},{"type":"v1","addr":"172.21.15.150:6823","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":38421},{"type":"v1","addr":"172.21.15.150:6821","nonce":38421}]},"public_addr":"172.21.15.150:6817/38421","cluster_addr":"172.21.15.150:6819/38421","heartbeat_back_addr":"172.21.15.150:6823/38421","heartbeat_front_addr":"172.21.15.150:6821/38421","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.438552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.413280+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.296466+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.424966+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.276953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.230984+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.410235+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.480084+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T02:13:51.257 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.262+0000 7fafdcde0640 1 -- 172.21.15.49:0/432416449 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fafb8065a60 msgr2=0x7fafb8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:51.257 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.262+0000 7fafdcde0640 1 --2- 172.21.15.49:0/432416449 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fafb8065a60 0x7fafb8067f20 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fafc8007520 tx=0x7fafc800b040 comp rx=0 tx=0).stop 2023-11-06T02:13:51.257 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.262+0000 7fafdcde0640 1 -- 172.21.15.49:0/432416449 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fafd80b23c0 msgr2=0x7fafd81f5f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:51.258 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.262+0000 7fafdcde0640 1 --2- 172.21.15.49:0/432416449 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fafd80b23c0 0x7fafd81f5f00 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fafd0005040 tx=0x7fafc0005800 comp rx=0 tx=0).stop 2023-11-06T02:13:51.258 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.262+0000 7fafdcde0640 1 -- 172.21.15.49:0/432416449 shutdown_connections 2023-11-06T02:13:51.258 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.262+0000 7fafdcde0640 1 --2- 172.21.15.49:0/432416449 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fafb8065a60 0x7fafb8067f20 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:51.258 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.262+0000 7fafdcde0640 1 --2- 172.21.15.49:0/432416449 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fafd80b23c0 0x7fafd81f5f00 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:51.258 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.262+0000 7fafdcde0640 1 --2- 172.21.15.49:0/432416449 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fafd80b12e0 0x7fafd80ab8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:51.259 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.262+0000 7fafdcde0640 1 --2- 172.21.15.49:0/432416449 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fafd815f0f0 0x7fafd80ab3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:51.259 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.262+0000 7fafdcde0640 1 -- 172.21.15.49:0/432416449 >> 172.21.15.49:0/432416449 conn(0x7fafd80d2aa0 msgr2=0x7fafd80b6060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:51.259 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.262+0000 7fafdcde0640 1 -- 172.21.15.49:0/432416449 shutdown_connections 2023-11-06T02:13:51.259 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.262+0000 7fafdcde0640 1 -- 172.21.15.49:0/432416449 wait complete. 2023-11-06T02:13:51.274 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T02:08:19.252222+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T02:08:24.776211+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T02:13:31.459314+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T02:13:32.603710+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T02:13:51.278 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T02:13:51.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.410+0000 7fd2bc74b640 1 Processor -- start 2023-11-06T02:13:51.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.410+0000 7fd2bc74b640 1 -- start start 2023-11-06T02:13:51.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.410+0000 7fd2bc74b640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd2b40d08b0 0x7fd2b40d0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:51.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.410+0000 7fd2bc74b640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fd2b40aeff0 0x7fd2b40af440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:51.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.410+0000 7fd2bc74b640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2b40d1230 con 0x7fd2b415ab60 2023-11-06T02:13:51.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.410+0000 7fd2bc74b640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2b40cb340 con 0x7fd2b40d08b0 2023-11-06T02:13:51.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.410+0000 7fd2bc74b640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2b40adf50 con 0x7fd2b40aeff0 2023-11-06T02:13:51.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.410+0000 7fd2b9ce6640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd2b40d08b0 0x7fd2b40d0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:51.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.410+0000 7fd2b9ce6640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd2b40d08b0 0x7fd2b40d0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33040/0 (socket says 172.21.15.49:33040) 2023-11-06T02:13:51.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.410+0000 7fd2b9ce6640 1 -- 172.21.15.49:0/510332572 learned_addr learned my addr 172.21.15.49:0/510332572 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:51.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.410+0000 7fd2bace8640 1 --2- 172.21.15.49:0/510332572 >> v2:172.21.15.150:3300/0 conn(0x7fd2b40aeff0 0x7fd2b40af440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:51.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.410+0000 7fd2b9ce6640 1 -- 172.21.15.49:0/510332572 >> v2:172.21.15.150:3300/0 conn(0x7fd2b40aeff0 msgr2=0x7fd2b40af440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:51.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.410+0000 7fd2b9ce6640 1 --2- 172.21.15.49:0/510332572 >> v2:172.21.15.150:3300/0 conn(0x7fd2b40aeff0 0x7fd2b40af440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:51.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.410+0000 7fd2b9ce6640 1 -- 172.21.15.49:0/510332572 >> v1:172.21.15.150:6789/0 conn(0x7fd2b415ab60 legacy=0x7fd2b40ad520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:51.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.410+0000 7fd2b9ce6640 1 -- 172.21.15.49:0/510332572 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2b40ae230 con 0x7fd2b40d08b0 2023-11-06T02:13:51.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.414+0000 7fd2b9ce6640 1 --2- 172.21.15.49:0/510332572 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2b40d08b0 0x7fd2b40d0cf0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fd2b40b34d0 tx=0x7fd2a401c210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d65a5ac3b293f505 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:51.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.414+0000 7fd2b94e5640 1 -- 172.21.15.49:0/510332572 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a40a4ea0 con 0x7fd2b40d08b0 2023-11-06T02:13:51.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.414+0000 7fd2b94e5640 1 -- 172.21.15.49:0/510332572 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2a40a7050 con 0x7fd2b40d08b0 2023-11-06T02:13:51.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.414+0000 7fd2b94e5640 1 -- 172.21.15.49:0/510332572 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a40ab820 con 0x7fd2b40d08b0 2023-11-06T02:13:51.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.414+0000 7fd2bc74b640 1 -- 172.21.15.49:0/510332572 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2b40d08b0 msgr2=0x7fd2b40d0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:51.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.414+0000 7fd2bc74b640 1 --2- 172.21.15.49:0/510332572 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2b40d08b0 0x7fd2b40d0cf0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fd2b40b34d0 tx=0x7fd2a401c210 comp rx=0 tx=0).stop 2023-11-06T02:13:51.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.414+0000 7fd2bc74b640 1 -- 172.21.15.49:0/510332572 shutdown_connections 2023-11-06T02:13:51.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.414+0000 7fd2bc74b640 1 --2- 172.21.15.49:0/510332572 >> v2:172.21.15.150:3300/0 conn(0x7fd2b40aeff0 0x7fd2b40af440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:51.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.414+0000 7fd2bc74b640 1 --2- 172.21.15.49:0/510332572 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2b40d08b0 0x7fd2b40d0cf0 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:51.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.414+0000 7fd2bc74b640 1 -- 172.21.15.49:0/510332572 >> 172.21.15.49:0/510332572 conn(0x7fd2b4089fd0 msgr2=0x7fd2b4099c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:51.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.414+0000 7fd2bc74b640 1 -- 172.21.15.49:0/510332572 shutdown_connections 2023-11-06T02:13:51.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.414+0000 7fd2bc74b640 1 -- 172.21.15.49:0/510332572 wait complete. 2023-11-06T02:13:51.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.414+0000 7fd2bc74b640 1 Processor -- start 2023-11-06T02:13:51.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.414+0000 7fd2bc74b640 1 -- start start 2023-11-06T02:13:51.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.418+0000 7fd2bc74b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd2b40aeff0 0x7fd2b4159b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:51.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.418+0000 7fd2bc74b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2b40d08b0 0x7fd2b40c4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:51.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.418+0000 7fd2bc74b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd2b414d520 0x7fd2b4152d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:51.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.418+0000 7fd2bc74b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b40b4e80 con 0x7fd2b40d08b0 2023-11-06T02:13:51.417 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.418+0000 7fd2bc74b640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b414d9d0 con 0x7fd2b414d520 2023-11-06T02:13:51.417 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.418+0000 7fd2bc74b640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b415a040 con 0x7fd2b40aeff0 2023-11-06T02:13:51.417 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.418+0000 7fd2b9ce6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2b40d08b0 0x7fd2b40c4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:51.417 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.418+0000 7fd2b9ce6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2b40d08b0 0x7fd2b40c4470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33048/0 (socket says 172.21.15.49:33048) 2023-11-06T02:13:51.417 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.418+0000 7fd2b9ce6640 1 -- 172.21.15.49:0/1438407762 learned_addr learned my addr 172.21.15.49:0/1438407762 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:51.418 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.418+0000 7fd2ba4e7640 1 --2- 172.21.15.49:0/1438407762 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd2b40aeff0 0x7fd2b4159b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:51.418 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.418+0000 7fd2bace8640 1 --2- 172.21.15.49:0/1438407762 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd2b414d520 0x7fd2b4152d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:51.418 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.418+0000 7fd2b9ce6640 1 -- 172.21.15.49:0/1438407762 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd2b40aeff0 msgr2=0x7fd2b4159b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:51.418 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.418+0000 7fd2b9ce6640 1 --2- 172.21.15.49:0/1438407762 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd2b40aeff0 0x7fd2b4159b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:51.418 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.418+0000 7fd2b9ce6640 1 -- 172.21.15.49:0/1438407762 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd2b414d520 msgr2=0x7fd2b4152d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:51.419 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.418+0000 7fd2b9ce6640 1 --2- 172.21.15.49:0/1438407762 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd2b414d520 0x7fd2b4152d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:51.419 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.418+0000 7fd2b9ce6640 1 -- 172.21.15.49:0/1438407762 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2a40094d0 con 0x7fd2b40d08b0 2023-11-06T02:13:51.419 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.418+0000 7fd2bace8640 1 --2- 172.21.15.49:0/1438407762 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd2b414d520 0x7fd2b4152d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:13:51.419 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.418+0000 7fd2b9ce6640 1 --2- 172.21.15.49:0/1438407762 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2b40d08b0 0x7fd2b40c4470 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fd2a4018e20 tx=0x7fd2a4018fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:51.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.418+0000 7fd2aaffd640 1 -- 172.21.15.49:0/1438407762 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a40a4ea0 con 0x7fd2b40d08b0 2023-11-06T02:13:51.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.418+0000 7fd2aaffd640 1 -- 172.21.15.49:0/1438407762 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2a40b4440 con 0x7fd2b40d08b0 2023-11-06T02:13:51.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.418+0000 7fd2bc74b640 1 -- 172.21.15.49:0/1438407762 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2b40b50d0 con 0x7fd2b40d08b0 2023-11-06T02:13:51.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.418+0000 7fd2aaffd640 1 -- 172.21.15.49:0/1438407762 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a40b47d0 con 0x7fd2b40d08b0 2023-11-06T02:13:51.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.418+0000 7fd2bc74b640 1 -- 172.21.15.49:0/1438407762 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2b414dc40 con 0x7fd2b40d08b0 2023-11-06T02:13:51.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.418+0000 7fd2bc74b640 1 -- 172.21.15.49:0/1438407762 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2b4067140 con 0x7fd2b40d08b0 2023-11-06T02:13:51.422 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.418+0000 7fd2aaffd640 1 -- 172.21.15.49:0/1438407762 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fd2a40b2070 con 0x7fd2b40d08b0 2023-11-06T02:13:51.422 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.418+0000 7fd2aaffd640 1 --2- 172.21.15.49:0/1438407762 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd298065a60 0x7fd298067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:51.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.418+0000 7fd2aaffd640 1 -- 172.21.15.49:0/1438407762 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd2a411d110 con 0x7fd2b40d08b0 2023-11-06T02:13:51.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.422+0000 7fd2ba4e7640 1 --2- 172.21.15.49:0/1438407762 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd298065a60 0x7fd298067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:51.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.422+0000 7fd2ba4e7640 1 --2- 172.21.15.49:0/1438407762 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd298065a60 0x7fd298067f20 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fd2b0000f70 tx=0x7fd2b0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:51.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.426+0000 7fd2aaffd640 1 -- 172.21.15.49:0/1438407762 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fd2a40ea970 con 0x7fd2b40d08b0 2023-11-06T02:13:51.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.582+0000 7fd2bc74b640 1 -- 172.21.15.49:0/1438407762 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd2b40cb480 con 0x7fd2b40d08b0 2023-11-06T02:13:51.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.582+0000 7fd2aaffd640 1 -- 172.21.15.49:0/1438407762 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd2a40edf90 con 0x7fd2b40d08b0 2023-11-06T02:13:51.577 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2023-11-06T02:13:51.579 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.582+0000 7fd2bc74b640 1 -- 172.21.15.49:0/1438407762 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd298065a60 msgr2=0x7fd298067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:51.579 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.582+0000 7fd2bc74b640 1 --2- 172.21.15.49:0/1438407762 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd298065a60 0x7fd298067f20 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fd2b0000f70 tx=0x7fd2b0009040 comp rx=0 tx=0).stop 2023-11-06T02:13:51.580 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.582+0000 7fd2bc74b640 1 -- 172.21.15.49:0/1438407762 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2b40d08b0 msgr2=0x7fd2b40c4470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:51.580 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.582+0000 7fd2bc74b640 1 --2- 172.21.15.49:0/1438407762 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2b40d08b0 0x7fd2b40c4470 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fd2a4018e20 tx=0x7fd2a4018fb0 comp rx=0 tx=0).stop 2023-11-06T02:13:51.580 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.582+0000 7fd2bc74b640 1 -- 172.21.15.49:0/1438407762 shutdown_connections 2023-11-06T02:13:51.580 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.582+0000 7fd2bc74b640 1 --2- 172.21.15.49:0/1438407762 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd298065a60 0x7fd298067f20 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:51.581 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.582+0000 7fd2bc74b640 1 --2- 172.21.15.49:0/1438407762 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2b40d08b0 0x7fd2b40c4470 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:51.581 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.582+0000 7fd2bc74b640 1 --2- 172.21.15.49:0/1438407762 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd2b414d520 0x7fd2b4152d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:51.581 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.582+0000 7fd2bc74b640 1 --2- 172.21.15.49:0/1438407762 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd2b40aeff0 0x7fd2b4159b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:51.581 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.582+0000 7fd2bc74b640 1 -- 172.21.15.49:0/1438407762 >> 172.21.15.49:0/1438407762 conn(0x7fd2b4089fd0 msgr2=0x7fd2b40b1160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:51.581 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.582+0000 7fd2bc74b640 1 -- 172.21.15.49:0/1438407762 shutdown_connections 2023-11-06T02:13:51.582 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.582+0000 7fd2bc74b640 1 -- 172.21.15.49:0/1438407762 wait complete. 2023-11-06T02:13:51.596 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T02:13:51.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.722+0000 7f623983d640 1 Processor -- start 2023-11-06T02:13:51.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.722+0000 7f623983d640 1 -- start start 2023-11-06T02:13:51.718 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.722+0000 7f623983d640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f623415a880 0x7f62340b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:51.718 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.722+0000 7f623983d640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f6234070160 0x7f6234070610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:51.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.722+0000 7f623983d640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62340ae360 con 0x7f623406d150 2023-11-06T02:13:51.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.722+0000 7f623983d640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6234070b50 con 0x7f6234070160 2023-11-06T02:13:51.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.722+0000 7f623983d640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62340ae640 con 0x7f623415a880 2023-11-06T02:13:51.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.722+0000 7f62327fc640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f6234070160 0x7f6234070610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:51.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.722+0000 7f62337fe640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f623406d150 0x7f623406d600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:55406/0 (socket says 172.21.15.49:55406) 2023-11-06T02:13:51.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.722+0000 7f62337fe640 1 -- 172.21.15.49:0/1800717671 learned_addr learned my addr 172.21.15.49:0/1800717671 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:13:51.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.722+0000 7f6232ffd640 1 --2- 172.21.15.49:0/1800717671 >> v2:172.21.15.150:3300/0 conn(0x7f623415a880 0x7f62340b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:51.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.722+0000 7f62327fc640 1 -- 172.21.15.49:0/1800717671 >> v2:172.21.15.150:3300/0 conn(0x7f623415a880 msgr2=0x7f62340b15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:51.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.722+0000 7f62327fc640 1 --2- 172.21.15.49:0/1800717671 >> v2:172.21.15.150:3300/0 conn(0x7f623415a880 0x7f62340b15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:51.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.722+0000 7f62327fc640 1 -- 172.21.15.49:0/1800717671 >> v1:172.21.15.49:6789/0 conn(0x7f623406d150 legacy=0x7f623406d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:51.721 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.722+0000 7f6231ffb640 1 -- 172.21.15.49:0/1800717671 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 483759818 0 0) 0x7f62340ae360 con 0x7f623406d150 2023-11-06T02:13:51.721 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.722+0000 7f62327fc640 1 -- 172.21.15.49:0/1800717671 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62340ae920 con 0x7f6234070160 2023-11-06T02:13:51.722 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.726+0000 7f62327fc640 1 --2- 172.21.15.49:0/1800717671 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6234070160 0x7f6234070610 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f62340cafc0 tx=0x7f622801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=218b90551d33dd47 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:51.722 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.726+0000 7f6231ffb640 1 -- 172.21.15.49:0/1800717671 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62280ab600 con 0x7f6234070160 2023-11-06T02:13:51.722 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.726+0000 7f6231ffb640 1 -- 172.21.15.49:0/1800717671 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62280a7050 con 0x7f6234070160 2023-11-06T02:13:51.723 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.726+0000 7f6231ffb640 1 -- 172.21.15.49:0/1800717671 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62280ab980 con 0x7f6234070160 2023-11-06T02:13:51.723 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.726+0000 7f623983d640 1 -- 172.21.15.49:0/1800717671 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6234070160 msgr2=0x7f6234070610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:51.723 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.726+0000 7f623983d640 1 --2- 172.21.15.49:0/1800717671 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6234070160 0x7f6234070610 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f62340cafc0 tx=0x7f622801c460 comp rx=0 tx=0).stop 2023-11-06T02:13:51.723 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.726+0000 7f623983d640 1 -- 172.21.15.49:0/1800717671 shutdown_connections 2023-11-06T02:13:51.723 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.726+0000 7f623983d640 1 --2- 172.21.15.49:0/1800717671 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6234070160 0x7f6234070610 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:51.723 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.726+0000 7f623983d640 1 --2- 172.21.15.49:0/1800717671 >> v2:172.21.15.150:3300/0 conn(0x7f623415a880 0x7f62340b15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:51.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.726+0000 7f623983d640 1 -- 172.21.15.49:0/1800717671 >> 172.21.15.49:0/1800717671 conn(0x7f623406a370 msgr2=0x7f62340ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:51.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.726+0000 7f623983d640 1 -- 172.21.15.49:0/1800717671 shutdown_connections 2023-11-06T02:13:51.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.726+0000 7f623983d640 1 -- 172.21.15.49:0/1800717671 wait complete. 2023-11-06T02:13:51.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.726+0000 7f623983d640 1 Processor -- start 2023-11-06T02:13:51.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.726+0000 7f623983d640 1 -- start start 2023-11-06T02:13:51.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.726+0000 7f623983d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f623406d150 0x7f623414d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:51.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.726+0000 7f623983d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6234070160 0x7f62340ca7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:51.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.726+0000 7f623983d640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f623415a880 0x7f62340ccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:51.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.726+0000 7f623983d640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62340cad20 con 0x7f623406d150 2023-11-06T02:13:51.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.726+0000 7f623983d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62340cd300 con 0x7f6234070160 2023-11-06T02:13:51.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.726+0000 7f623983d640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62340cd550 con 0x7f623415a880 2023-11-06T02:13:51.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.726+0000 7f6232ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f623406d150 0x7f623414d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:51.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.730+0000 7f6232ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f623406d150 0x7f623414d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33074/0 (socket says 172.21.15.49:33074) 2023-11-06T02:13:51.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.730+0000 7f6232ffd640 1 -- 172.21.15.49:0/729379327 learned_addr learned my addr 172.21.15.49:0/729379327 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:51.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.730+0000 7f62337fe640 1 --2- 172.21.15.49:0/729379327 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f623415a880 0x7f62340ccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:51.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.730+0000 7f62327fc640 1 --2- 172.21.15.49:0/729379327 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6234070160 0x7f62340ca7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:51.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.730+0000 7f6232ffd640 1 -- 172.21.15.49:0/729379327 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f623415a880 msgr2=0x7f62340ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:51.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.730+0000 7f6232ffd640 1 --2- 172.21.15.49:0/729379327 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f623415a880 0x7f62340ccdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:51.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.730+0000 7f6232ffd640 1 -- 172.21.15.49:0/729379327 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6234070160 msgr2=0x7f62340ca7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:51.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.730+0000 7f6232ffd640 1 --2- 172.21.15.49:0/729379327 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6234070160 0x7f62340ca7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:51.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.730+0000 7f6232ffd640 1 -- 172.21.15.49:0/729379327 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6228009420 con 0x7f623406d150 2023-11-06T02:13:51.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.730+0000 7f62337fe640 1 --2- 172.21.15.49:0/729379327 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f623415a880 0x7f62340ccdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:13:51.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.730+0000 7f6232ffd640 1 --2- 172.21.15.49:0/729379327 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f623406d150 0x7f623414d130 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f622400d800 tx=0x7f622400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:51.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.730+0000 7f621bfff640 1 -- 172.21.15.49:0/729379327 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f622400b6c0 con 0x7f623406d150 2023-11-06T02:13:51.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.730+0000 7f621bfff640 1 -- 172.21.15.49:0/729379327 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f622400b820 con 0x7f623406d150 2023-11-06T02:13:51.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.730+0000 7f623983d640 1 -- 172.21.15.49:0/729379327 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62340cd830 con 0x7f623406d150 2023-11-06T02:13:51.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.730+0000 7f621bfff640 1 -- 172.21.15.49:0/729379327 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f622400bb90 con 0x7f623406d150 2023-11-06T02:13:51.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.730+0000 7f623983d640 1 -- 172.21.15.49:0/729379327 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62340c8ea0 con 0x7f623406d150 2023-11-06T02:13:51.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.730+0000 7f623983d640 1 -- 172.21.15.49:0/729379327 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62340c8a20 con 0x7f623406d150 2023-11-06T02:13:51.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.730+0000 7f621bfff640 1 -- 172.21.15.49:0/729379327 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f622400bcf0 con 0x7f623406d150 2023-11-06T02:13:51.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.730+0000 7f621bfff640 1 --2- 172.21.15.49:0/729379327 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6220065a60 0x7f6220067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:51.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.730+0000 7f621bfff640 1 -- 172.21.15.49:0/729379327 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6224083880 con 0x7f623406d150 2023-11-06T02:13:51.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.730+0000 7f62327fc640 1 --2- 172.21.15.49:0/729379327 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6220065a60 0x7f6220067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:51.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.730+0000 7f62327fc640 1 --2- 172.21.15.49:0/729379327 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6220065a60 0x7f6220067f20 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f62280ad000 tx=0x7f6228008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:51.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.734+0000 7f621bfff640 1 -- 172.21.15.49:0/729379327 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f62340c8a20 con 0x7f623406d150 2023-11-06T02:13:51.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.886+0000 7f623983d640 1 -- 172.21.15.49:0/729379327 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f62340c8a20 con 0x7f623406d150 2023-11-06T02:13:51.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.886+0000 7f621bfff640 1 -- 172.21.15.49:0/729379327 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v80) v1 ==== 92+0+10 (secure 0 0 0) 0x7f62340c8a20 con 0x7f623406d150 2023-11-06T02:13:51.884 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 8 2023-11-06T02:13:51.885 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.890+0000 7f623983d640 1 -- 172.21.15.49:0/729379327 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6220065a60 msgr2=0x7f6220067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:51.885 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.890+0000 7f623983d640 1 --2- 172.21.15.49:0/729379327 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6220065a60 0x7f6220067f20 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f62280ad000 tx=0x7f6228008040 comp rx=0 tx=0).stop 2023-11-06T02:13:51.886 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.890+0000 7f623983d640 1 -- 172.21.15.49:0/729379327 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f623406d150 msgr2=0x7f623414d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:51.886 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.890+0000 7f623983d640 1 --2- 172.21.15.49:0/729379327 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f623406d150 0x7f623414d130 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f622400d800 tx=0x7f622400dcc0 comp rx=0 tx=0).stop 2023-11-06T02:13:51.886 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.890+0000 7f623983d640 1 -- 172.21.15.49:0/729379327 shutdown_connections 2023-11-06T02:13:51.886 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.890+0000 7f623983d640 1 --2- 172.21.15.49:0/729379327 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6220065a60 0x7f6220067f20 secure :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f62280ad000 tx=0x7f6228008040 comp rx=0 tx=0).stop 2023-11-06T02:13:51.887 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.890+0000 7f623983d640 1 --2- 172.21.15.49:0/729379327 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f623415a880 0x7f62340ccdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:51.887 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.890+0000 7f623983d640 1 --2- 172.21.15.49:0/729379327 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6234070160 0x7f62340ca7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:51.887 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.890+0000 7f623983d640 1 --2- 172.21.15.49:0/729379327 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f623406d150 0x7f623414d130 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:51.888 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.890+0000 7f623983d640 1 -- 172.21.15.49:0/729379327 >> 172.21.15.49:0/729379327 conn(0x7f623406a370 msgr2=0x7f623415d720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:51.888 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.890+0000 7f623983d640 1 -- 172.21.15.49:0/729379327 shutdown_connections 2023-11-06T02:13:51.888 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:51.890+0000 7f623983d640 1 -- 172.21.15.49:0/729379327 wait complete. 2023-11-06T02:13:51.906 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T02:13:52.033 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.038+0000 7f36ef44c640 1 Processor -- start 2023-11-06T02:13:52.033 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.038+0000 7f36ef44c640 1 -- start start 2023-11-06T02:13:52.035 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.038+0000 7f36ef44c640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f36e815ca90 0x7f36e80afba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:52.036 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.038+0000 7f36ef44c640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36e80b00e0 con 0x7f36e815ce70 2023-11-06T02:13:52.036 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.038+0000 7f36ef44c640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36e80cf640 con 0x7f36e80cf190 2023-11-06T02:13:52.036 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.038+0000 7f36ef44c640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36e80aede0 con 0x7f36e815ca90 2023-11-06T02:13:52.036 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.038+0000 7f36ec9e7640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f36e815ce70 0x7f36e80cea80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:55412/0 (socket says 172.21.15.49:55412) 2023-11-06T02:13:52.036 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.038+0000 7f36ec9e7640 1 -- 172.21.15.49:0/897034985 learned_addr learned my addr 172.21.15.49:0/897034985 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:13:52.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.038+0000 7f36ed1e8640 1 --2- 172.21.15.49:0/897034985 >> v2:172.21.15.150:3300/0 conn(0x7f36e815ca90 0x7f36e80afba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:52.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.038+0000 7f36e7fff640 1 -- 172.21.15.49:0/897034985 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2955030375 0 0) 0x7f36e80b00e0 con 0x7f36e815ce70 2023-11-06T02:13:52.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.038+0000 7f36e7fff640 1 -- 172.21.15.49:0/897034985 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36d0003530 con 0x7f36e815ce70 2023-11-06T02:13:52.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.038+0000 7f36e7fff640 1 -- 172.21.15.49:0/897034985 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1270640543 0 0) 0x7f36d0003530 con 0x7f36e815ce70 2023-11-06T02:13:52.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.038+0000 7f36e7fff640 1 -- 172.21.15.49:0/897034985 >> v2:172.21.15.150:3300/0 conn(0x7f36e815ca90 msgr2=0x7f36e80afba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:52.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.038+0000 7f36e7fff640 1 --2- 172.21.15.49:0/897034985 >> v2:172.21.15.150:3300/0 conn(0x7f36e815ca90 0x7f36e80afba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:52.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.038+0000 7f36e7fff640 1 -- 172.21.15.49:0/897034985 >> v1:172.21.15.150:6789/0 conn(0x7f36e80cf190 legacy=0x7f36e80b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:52.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.038+0000 7f36e7fff640 1 -- 172.21.15.49:0/897034985 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36e80af0c0 con 0x7f36e815ce70 2023-11-06T02:13:52.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.038+0000 7f36e7fff640 1 -- 172.21.15.49:0/897034985 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7f36dc001680 con 0x7f36e815ce70 2023-11-06T02:13:52.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.038+0000 7f36e7fff640 1 -- 172.21.15.49:0/897034985 >> v1:172.21.15.49:6789/0 conn(0x7f36e815ce70 legacy=0x7f36e80cea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:52.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.038+0000 7f36e7fff640 1 --2- 172.21.15.49:0/897034985 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f36d0003980 0x7f36d0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:52.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.038+0000 7f36e7fff640 1 --2- 172.21.15.49:0/897034985 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f36d00047d0 0x7f36d0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:52.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.038+0000 7f36e7fff640 1 --2- 172.21.15.49:0/897034985 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36d0005190 0x7f36d000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:52.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.038+0000 7f36e7fff640 1 -- 172.21.15.49:0/897034985 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e80aede0 con 0x7f36d0005190 2023-11-06T02:13:52.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.038+0000 7f36e7fff640 1 -- 172.21.15.49:0/897034985 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36d000c030 con 0x7f36d00047d0 2023-11-06T02:13:52.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.038+0000 7f36e7fff640 1 -- 172.21.15.49:0/897034985 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36d000c1f0 con 0x7f36d0003980 2023-11-06T02:13:52.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.038+0000 7f36ed9e9640 1 --2- 172.21.15.49:0/897034985 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36d0005190 0x7f36d000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:52.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.038+0000 7f36ec9e7640 1 --2- 172.21.15.49:0/897034985 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f36d00047d0 0x7f36d0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:52.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.038+0000 7f36ed1e8640 1 --2- 172.21.15.49:0/897034985 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f36d0003980 0x7f36d0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:52.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.038+0000 7f36ed9e9640 1 -- 172.21.15.49:0/897034985 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f36d0003980 msgr2=0x7f36d0003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:52.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.038+0000 7f36ed9e9640 1 --2- 172.21.15.49:0/897034985 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f36d0003980 0x7f36d0003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:52.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.038+0000 7f36ed9e9640 1 -- 172.21.15.49:0/897034985 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f36d00047d0 msgr2=0x7f36d0004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:52.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.038+0000 7f36ed9e9640 1 --2- 172.21.15.49:0/897034985 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f36d00047d0 0x7f36d0004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:52.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.038+0000 7f36ed9e9640 1 -- 172.21.15.49:0/897034985 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36d0003530 con 0x7f36d0005190 2023-11-06T02:13:52.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.042+0000 7f36ed9e9640 1 -- 172.21.15.49:0/897034985 reap_dead start 2023-11-06T02:13:52.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.042+0000 7f36ed9e9640 1 --2- 172.21.15.49:0/897034985 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36d0005190 0x7f36d000b990 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f36d8001e40 tx=0x7f36d801c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=872bfe2e15cb597e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:52.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.042+0000 7f36e7fff640 1 -- 172.21.15.49:0/897034985 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d8003b80 con 0x7f36d0005190 2023-11-06T02:13:52.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.042+0000 7f36e7fff640 1 -- 172.21.15.49:0/897034985 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36d8003ce0 con 0x7f36d0005190 2023-11-06T02:13:52.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.042+0000 7f36e7fff640 1 -- 172.21.15.49:0/897034985 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d8005260 con 0x7f36d0005190 2023-11-06T02:13:52.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.042+0000 7f36ef44c640 1 -- 172.21.15.49:0/897034985 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36d0005190 msgr2=0x7f36d000b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:52.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.042+0000 7f36ef44c640 1 --2- 172.21.15.49:0/897034985 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36d0005190 0x7f36d000b990 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f36d8001e40 tx=0x7f36d801c740 comp rx=0 tx=0).stop 2023-11-06T02:13:52.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.042+0000 7f36ef44c640 1 -- 172.21.15.49:0/897034985 shutdown_connections 2023-11-06T02:13:52.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.042+0000 7f36ef44c640 1 --2- 172.21.15.49:0/897034985 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36d0005190 0x7f36d000b990 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:52.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.042+0000 7f36ef44c640 1 -- 172.21.15.49:0/897034985 >> 172.21.15.49:0/897034985 conn(0x7f36e8098a40 msgr2=0x7f36e80cf860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:52.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.042+0000 7f36ef44c640 1 -- 172.21.15.49:0/897034985 shutdown_connections 2023-11-06T02:13:52.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.042+0000 7f36ef44c640 1 -- 172.21.15.49:0/897034985 wait complete. 2023-11-06T02:13:52.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.042+0000 7f36ef44c640 1 Processor -- start 2023-11-06T02:13:52.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.042+0000 7f36ef44c640 1 -- start start 2023-11-06T02:13:52.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.046+0000 7f36ef44c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f36d0005190 0x7f36e81550f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:52.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.046+0000 7f36ef44c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36e8156770 0x7f36e8157e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:52.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.046+0000 7f36ef44c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f36e8158370 0x7f36e81e59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:52.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.046+0000 7f36ef44c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e8158800 con 0x7f36e8156770 2023-11-06T02:13:52.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.046+0000 7f36ef44c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e8155630 con 0x7f36d0005190 2023-11-06T02:13:52.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.046+0000 7f36ef44c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36e8159320 con 0x7f36e8158370 2023-11-06T02:13:52.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.046+0000 7f36ec9e7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36e8156770 0x7f36e8157e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:52.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.046+0000 7f36ec9e7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36e8156770 0x7f36e8157e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33080/0 (socket says 172.21.15.49:33080) 2023-11-06T02:13:52.049 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.046+0000 7f36ec9e7640 1 -- 172.21.15.49:0/3279340613 learned_addr learned my addr 172.21.15.49:0/3279340613 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:52.049 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.046+0000 7f36ed1e8640 1 --2- 172.21.15.49:0/3279340613 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f36d0005190 0x7f36e81550f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:52.049 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.046+0000 7f36ed9e9640 1 --2- 172.21.15.49:0/3279340613 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f36e8158370 0x7f36e81e59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:52.049 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.046+0000 7f36ec9e7640 1 -- 172.21.15.49:0/3279340613 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f36e8158370 msgr2=0x7f36e81e59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:52.049 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.046+0000 7f36ec9e7640 1 --2- 172.21.15.49:0/3279340613 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f36e8158370 0x7f36e81e59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:52.050 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.046+0000 7f36ec9e7640 1 -- 172.21.15.49:0/3279340613 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f36d0005190 msgr2=0x7f36e81550f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:52.050 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.046+0000 7f36ec9e7640 1 --2- 172.21.15.49:0/3279340613 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f36d0005190 0x7f36e81550f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:52.050 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.046+0000 7f36ec9e7640 1 -- 172.21.15.49:0/3279340613 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36e8157510 con 0x7f36e8156770 2023-11-06T02:13:52.050 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.046+0000 7f36ed9e9640 1 --2- 172.21.15.49:0/3279340613 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f36e8158370 0x7f36e81e59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:13:52.050 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.046+0000 7f36ec9e7640 1 --2- 172.21.15.49:0/3279340613 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36e8156770 0x7f36e8157e30 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f36dc00e780 tx=0x7f36dc00ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:52.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.046+0000 7f36e5ffb640 1 -- 172.21.15.49:0/3279340613 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36dc00ee90 con 0x7f36e8156770 2023-11-06T02:13:52.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.046+0000 7f36e5ffb640 1 -- 172.21.15.49:0/3279340613 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36dc00c870 con 0x7f36e8156770 2023-11-06T02:13:52.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.046+0000 7f36ef44c640 1 -- 172.21.15.49:0/3279340613 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36e8156c70 con 0x7f36e8156770 2023-11-06T02:13:52.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.046+0000 7f36e5ffb640 1 -- 172.21.15.49:0/3279340613 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36dc00cc20 con 0x7f36e8156770 2023-11-06T02:13:52.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.046+0000 7f36ef44c640 1 -- 172.21.15.49:0/3279340613 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36e8157230 con 0x7f36e8156770 2023-11-06T02:13:52.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.046+0000 7f36ef44c640 1 -- 172.21.15.49:0/3279340613 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36e8099040 con 0x7f36e8156770 2023-11-06T02:13:52.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.046+0000 7f36e5ffb640 1 -- 172.21.15.49:0/3279340613 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f36dc013420 con 0x7f36e8156770 2023-11-06T02:13:52.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.046+0000 7f36e5ffb640 1 --2- 172.21.15.49:0/3279340613 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f36d4065a90 0x7f36d4067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:52.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.046+0000 7f36e5ffb640 1 -- 172.21.15.49:0/3279340613 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f36dc082ef0 con 0x7f36e8156770 2023-11-06T02:13:52.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.046+0000 7f36ed1e8640 1 --2- 172.21.15.49:0/3279340613 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f36d4065a90 0x7f36d4067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:52.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.050+0000 7f36ed1e8640 1 --2- 172.21.15.49:0/3279340613 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f36d4065a90 0x7f36d4067f50 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f36e806ac00 tx=0x7f36e0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:52.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.050+0000 7f36e5ffb640 1 -- 172.21.15.49:0/3279340613 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f36dc0507c0 con 0x7f36e8156770 2023-11-06T02:13:52.177 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T02:13:52.180 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T02:13:52.204 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.210+0000 7f36ef44c640 1 -- 172.21.15.49:0/3279340613 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f36e80cd1f0 con 0x7f36e8156770 2023-11-06T02:13:52.204 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.210+0000 7f36e5ffb640 1 -- 172.21.15.49:0/3279340613 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v80) v1 ==== 104+0+11 (secure 0 0 0) 0x7f36dc053de0 con 0x7f36e8156770 2023-11-06T02:13:52.205 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2023-11-06T02:13:52.207 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.210+0000 7f36ef44c640 1 -- 172.21.15.49:0/3279340613 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f36d4065a90 msgr2=0x7f36d4067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:52.207 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.210+0000 7f36ef44c640 1 --2- 172.21.15.49:0/3279340613 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f36d4065a90 0x7f36d4067f50 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f36e806ac00 tx=0x7f36e0008040 comp rx=0 tx=0).stop 2023-11-06T02:13:52.207 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.210+0000 7f36ef44c640 1 -- 172.21.15.49:0/3279340613 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36e8156770 msgr2=0x7f36e8157e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:52.207 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.210+0000 7f36ef44c640 1 --2- 172.21.15.49:0/3279340613 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36e8156770 0x7f36e8157e30 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f36dc00e780 tx=0x7f36dc00ec40 comp rx=0 tx=0).stop 2023-11-06T02:13:52.207 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.210+0000 7f36ef44c640 1 -- 172.21.15.49:0/3279340613 shutdown_connections 2023-11-06T02:13:52.208 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.210+0000 7f36ef44c640 1 --2- 172.21.15.49:0/3279340613 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f36d4065a90 0x7f36d4067f50 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:52.208 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.210+0000 7f36ef44c640 1 --2- 172.21.15.49:0/3279340613 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36e8156770 0x7f36e8157e30 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:52.208 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.210+0000 7f36ef44c640 1 --2- 172.21.15.49:0/3279340613 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f36e8158370 0x7f36e81e59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:52.208 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.210+0000 7f36ef44c640 1 --2- 172.21.15.49:0/3279340613 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f36d0005190 0x7f36e81550f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:52.208 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.210+0000 7f36ef44c640 1 -- 172.21.15.49:0/3279340613 >> 172.21.15.49:0/3279340613 conn(0x7f36e8098a40 msgr2=0x7f36e81e2140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:52.209 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.210+0000 7f36ef44c640 1 -- 172.21.15.49:0/3279340613 shutdown_connections 2023-11-06T02:13:52.209 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.210+0000 7f36ef44c640 1 -- 172.21.15.49:0/3279340613 wait complete. 2023-11-06T02:13:52.224 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T02:13:52.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.394+0000 7f1266aaf640 1 Processor -- start 2023-11-06T02:13:52.389 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.394+0000 7f1266aaf640 1 -- start start 2023-11-06T02:13:52.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.394+0000 7f1266aaf640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f126015a880 0x7f12600b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:52.391 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.394+0000 7f1266aaf640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f1260070160 0x7f12600705e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:52.391 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.394+0000 7f1266aaf640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12600ae360 con 0x7f126006d150 2023-11-06T02:13:52.391 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.394+0000 7f1266aaf640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1260070b20 con 0x7f1260070160 2023-11-06T02:13:52.391 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.394+0000 7f1266aaf640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12600ae640 con 0x7f126015a880 2023-11-06T02:13:52.391 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.394+0000 7f125ffff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f1260070160 0x7f12600705e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:52.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.394+0000 7f125ffff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f1260070160 0x7f12600705e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33092/0 (socket says 172.21.15.49:33092) 2023-11-06T02:13:52.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.394+0000 7f125ffff640 1 -- 172.21.15.49:0/1551137651 learned_addr learned my addr 172.21.15.49:0/1551137651 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:52.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.394+0000 7f126504c640 1 --1- 172.21.15.49:0/1551137651 >> v1:172.21.15.49:6789/0 conn(0x7f126006d150 0x7f126006d5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:55428/0 (socket says 172.21.15.49:55428) 2023-11-06T02:13:52.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.394+0000 7f126484b640 1 --2- 172.21.15.49:0/1551137651 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f126015a880 0x7f12600b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:52.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.394+0000 7f125ffff640 1 -- 172.21.15.49:0/1551137651 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f126015a880 msgr2=0x7f12600b15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:52.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.394+0000 7f125ffff640 1 --2- 172.21.15.49:0/1551137651 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f126015a880 0x7f12600b15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:52.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.394+0000 7f125ffff640 1 -- 172.21.15.49:0/1551137651 >> v1:172.21.15.49:6789/0 conn(0x7f126006d150 legacy=0x7f126006d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:52.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.394+0000 7f125ffff640 1 -- 172.21.15.49:0/1551137651 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12600ae920 con 0x7f1260070160 2023-11-06T02:13:52.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.394+0000 7f125f7fe640 1 -- 172.21.15.49:0/1551137651 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4158102237 0 0) 0x7f12600ae360 con 0x7f126006d150 2023-11-06T02:13:52.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.398+0000 7f125ffff640 1 --2- 172.21.15.49:0/1551137651 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1260070160 0x7f12600705e0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f124c009870 tx=0x7f124c01c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6578b7c81b2d64f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:52.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.398+0000 7f125f7fe640 1 -- 172.21.15.49:0/1551137651 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f124c0ab650 con 0x7f1260070160 2023-11-06T02:13:52.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.398+0000 7f125f7fe640 1 -- 172.21.15.49:0/1551137651 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f124c0ab7b0 con 0x7f1260070160 2023-11-06T02:13:52.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.398+0000 7f125f7fe640 1 -- 172.21.15.49:0/1551137651 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f124c0abb40 con 0x7f1260070160 2023-11-06T02:13:52.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.398+0000 7f1266aaf640 1 -- 172.21.15.49:0/1551137651 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1260070160 msgr2=0x7f12600705e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:52.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.398+0000 7f1266aaf640 1 --2- 172.21.15.49:0/1551137651 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1260070160 0x7f12600705e0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f124c009870 tx=0x7f124c01c540 comp rx=0 tx=0).stop 2023-11-06T02:13:52.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.398+0000 7f1266aaf640 1 -- 172.21.15.49:0/1551137651 shutdown_connections 2023-11-06T02:13:52.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.398+0000 7f1266aaf640 1 --2- 172.21.15.49:0/1551137651 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1260070160 0x7f12600705e0 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:52.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.398+0000 7f1266aaf640 1 --2- 172.21.15.49:0/1551137651 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f126015a880 0x7f12600b15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:52.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.398+0000 7f1266aaf640 1 -- 172.21.15.49:0/1551137651 >> 172.21.15.49:0/1551137651 conn(0x7f126006a370 msgr2=0x7f12600ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:52.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.398+0000 7f1266aaf640 1 -- 172.21.15.49:0/1551137651 shutdown_connections 2023-11-06T02:13:52.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.398+0000 7f1266aaf640 1 -- 172.21.15.49:0/1551137651 wait complete. 2023-11-06T02:13:52.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.398+0000 7f1266aaf640 1 Processor -- start 2023-11-06T02:13:52.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.398+0000 7f1266aaf640 1 -- start start 2023-11-06T02:13:52.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.398+0000 7f1266aaf640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f126006d150 0x7f12600cdc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:52.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.398+0000 7f1266aaf640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1260070160 0x7f126014d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:52.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.398+0000 7f1266aaf640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f126015a880 0x7f126015b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:52.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.398+0000 7f1266aaf640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f126014d900 con 0x7f126015a880 2023-11-06T02:13:52.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.398+0000 7f1266aaf640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12600c9e30 con 0x7f1260070160 2023-11-06T02:13:52.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.398+0000 7f1266aaf640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12600ca0d0 con 0x7f126006d150 2023-11-06T02:13:52.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.398+0000 7f126504c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f126015a880 0x7f126015b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:52.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.398+0000 7f126504c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f126015a880 0x7f126015b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33100/0 (socket says 172.21.15.49:33100) 2023-11-06T02:13:52.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.398+0000 7f126504c640 1 -- 172.21.15.49:0/89102021 learned_addr learned my addr 172.21.15.49:0/89102021 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:52.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.398+0000 7f125ffff640 1 --2- 172.21.15.49:0/89102021 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1260070160 0x7f126014d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:52.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.398+0000 7f126484b640 1 --2- 172.21.15.49:0/89102021 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f126006d150 0x7f12600cdc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:52.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.398+0000 7f126504c640 1 -- 172.21.15.49:0/89102021 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f126006d150 msgr2=0x7f12600cdc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:52.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.398+0000 7f126504c640 1 --2- 172.21.15.49:0/89102021 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f126006d150 0x7f12600cdc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:52.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.398+0000 7f126504c640 1 -- 172.21.15.49:0/89102021 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1260070160 msgr2=0x7f126014d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:52.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.398+0000 7f126504c640 1 --2- 172.21.15.49:0/89102021 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1260070160 0x7f126014d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:52.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.398+0000 7f126504c640 1 -- 172.21.15.49:0/89102021 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f124c0094d0 con 0x7f126015a880 2023-11-06T02:13:52.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.398+0000 7f125ffff640 1 --2- 172.21.15.49:0/89102021 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1260070160 0x7f126014d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:13:52.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.402+0000 7f126504c640 1 --2- 172.21.15.49:0/89102021 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f126015a880 0x7f126015b070 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f1258001570 tx=0x7f125800cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:52.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.402+0000 7f125d7fa640 1 -- 172.21.15.49:0/89102021 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f125800a640 con 0x7f126015a880 2023-11-06T02:13:52.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.402+0000 7f125d7fa640 1 -- 172.21.15.49:0/89102021 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f125800a7c0 con 0x7f126015a880 2023-11-06T02:13:52.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.402+0000 7f1266aaf640 1 -- 172.21.15.49:0/89102021 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12600ca3b0 con 0x7f126015a880 2023-11-06T02:13:52.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.402+0000 7f125d7fa640 1 -- 172.21.15.49:0/89102021 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f125800ab70 con 0x7f126015a880 2023-11-06T02:13:52.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.402+0000 7f1266aaf640 1 -- 172.21.15.49:0/89102021 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12600ca970 con 0x7f126015a880 2023-11-06T02:13:52.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.402+0000 7f1266aaf640 1 -- 172.21.15.49:0/89102021 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1260074040 con 0x7f126015a880 2023-11-06T02:13:52.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.402+0000 7f125d7fa640 1 -- 172.21.15.49:0/89102021 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f125800acd0 con 0x7f126015a880 2023-11-06T02:13:52.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.402+0000 7f125d7fa640 1 --2- 172.21.15.49:0/89102021 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1244065a90 0x7f1244067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:52.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.402+0000 7f125d7fa640 1 -- 172.21.15.49:0/89102021 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1258082e00 con 0x7f126015a880 2023-11-06T02:13:52.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.402+0000 7f126484b640 1 --2- 172.21.15.49:0/89102021 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1244065a90 0x7f1244067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:52.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.406+0000 7f126484b640 1 --2- 172.21.15.49:0/89102021 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1244065a90 0x7f1244067f50 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f125400b7c0 tx=0x7f1254008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:52.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.406+0000 7f125d7fa640 1 -- 172.21.15.49:0/89102021 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f12580506c0 con 0x7f126015a880 2023-11-06T02:13:52.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.554+0000 7f1266aaf640 1 -- 172.21.15.49:0/89102021 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f126006c430 con 0x7f126015a880 2023-11-06T02:13:52.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.554+0000 7f125d7fa640 1 -- 172.21.15.49:0/89102021 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v80) v1 ==== 100+0+11 (secure 0 0 0) 0x7f12580180a0 con 0x7f126015a880 2023-11-06T02:13:52.550 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2023-11-06T02:13:52.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.554+0000 7f1266aaf640 1 -- 172.21.15.49:0/89102021 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1244065a90 msgr2=0x7f1244067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:52.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.554+0000 7f1266aaf640 1 --2- 172.21.15.49:0/89102021 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1244065a90 0x7f1244067f50 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f125400b7c0 tx=0x7f1254008040 comp rx=0 tx=0).stop 2023-11-06T02:13:52.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.554+0000 7f1266aaf640 1 -- 172.21.15.49:0/89102021 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f126015a880 msgr2=0x7f126015b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:52.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.554+0000 7f1266aaf640 1 --2- 172.21.15.49:0/89102021 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f126015a880 0x7f126015b070 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f1258001570 tx=0x7f125800cca0 comp rx=0 tx=0).stop 2023-11-06T02:13:52.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.558+0000 7f1266aaf640 1 -- 172.21.15.49:0/89102021 shutdown_connections 2023-11-06T02:13:52.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.558+0000 7f1266aaf640 1 --2- 172.21.15.49:0/89102021 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1244065a90 0x7f1244067f50 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:52.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.558+0000 7f1266aaf640 1 --2- 172.21.15.49:0/89102021 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f126015a880 0x7f126015b070 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:52.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.558+0000 7f1266aaf640 1 --2- 172.21.15.49:0/89102021 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1260070160 0x7f126014d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:52.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.558+0000 7f1266aaf640 1 --2- 172.21.15.49:0/89102021 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f126006d150 0x7f12600cdc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:52.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.558+0000 7f1266aaf640 1 -- 172.21.15.49:0/89102021 >> 172.21.15.49:0/89102021 conn(0x7f126006a370 msgr2=0x7f12600b3830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:52.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.558+0000 7f1266aaf640 1 -- 172.21.15.49:0/89102021 shutdown_connections 2023-11-06T02:13:52.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.558+0000 7f1266aaf640 1 -- 172.21.15.49:0/89102021 wait complete. 2023-11-06T02:13:52.567 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-06T02:13:52.680 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.686+0000 7f37b3c68640 1 Processor -- start 2023-11-06T02:13:52.681 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.686+0000 7f37b3c68640 1 -- start start 2023-11-06T02:13:52.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.686+0000 7f37b3c68640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f37ac15ab50 0x7f37ac0c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:52.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.686+0000 7f37b3c68640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f37ac0cb010 0x7f37ac0b0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:52.682 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.686+0000 7f37b3c68640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37ac0cab80 con 0x7f37ac0b05c0 2023-11-06T02:13:52.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.686+0000 7f37b3c68640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37ac0b0a10 con 0x7f37ac0cb010 2023-11-06T02:13:52.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.686+0000 7f37b3c68640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37ac15af30 con 0x7f37ac15ab50 2023-11-06T02:13:52.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.686+0000 7f37b1203640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f37ac0cb010 0x7f37ac0b0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:52.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.686+0000 7f37b2205640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f37ac0b05c0 0x7f37ac0700a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:55430/0 (socket says 172.21.15.49:55430) 2023-11-06T02:13:52.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.686+0000 7f37b2205640 1 -- 172.21.15.49:0/2275796947 learned_addr learned my addr 172.21.15.49:0/2275796947 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:13:52.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.686+0000 7f37b1a04640 1 --2- 172.21.15.49:0/2275796947 >> v2:172.21.15.150:3300/0 conn(0x7f37ac15ab50 0x7f37ac0c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:52.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.686+0000 7f37b1203640 1 -- 172.21.15.49:0/2275796947 >> v2:172.21.15.150:3300/0 conn(0x7f37ac15ab50 msgr2=0x7f37ac0c9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:52.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.686+0000 7f37b1203640 1 --2- 172.21.15.49:0/2275796947 >> v2:172.21.15.150:3300/0 conn(0x7f37ac15ab50 0x7f37ac0c9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:52.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.686+0000 7f37b1203640 1 -- 172.21.15.49:0/2275796947 >> v1:172.21.15.49:6789/0 conn(0x7f37ac0b05c0 legacy=0x7f37ac0700a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:52.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.686+0000 7f37b1203640 1 -- 172.21.15.49:0/2275796947 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37ac0cb5b0 con 0x7f37ac0cb010 2023-11-06T02:13:52.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.686+0000 7f37b1a04640 1 --2- 172.21.15.49:0/2275796947 >> v2:172.21.15.150:3300/0 conn(0x7f37ac15ab50 0x7f37ac0c9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:13:52.686 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.690+0000 7f37b1203640 1 --2- 172.21.15.49:0/2275796947 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37ac0cb010 0x7f37ac0b0080 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f37ac0725a0 tx=0x7f379c01c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40175883790be734 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:52.686 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.690+0000 7f37b0a02640 1 -- 172.21.15.49:0/2275796947 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379c0a7070 con 0x7f37ac0cb010 2023-11-06T02:13:52.686 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.690+0000 7f37b0a02640 1 -- 172.21.15.49:0/2275796947 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f379c0ab430 con 0x7f37ac0cb010 2023-11-06T02:13:52.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.690+0000 7f37b0a02640 1 -- 172.21.15.49:0/2275796947 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379c0ab7c0 con 0x7f37ac0cb010 2023-11-06T02:13:52.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.690+0000 7f37b3c68640 1 -- 172.21.15.49:0/2275796947 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37ac0cb010 msgr2=0x7f37ac0b0080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:52.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.690+0000 7f37b3c68640 1 --2- 172.21.15.49:0/2275796947 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37ac0cb010 0x7f37ac0b0080 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f37ac0725a0 tx=0x7f379c01c430 comp rx=0 tx=0).stop 2023-11-06T02:13:52.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.690+0000 7f37b3c68640 1 -- 172.21.15.49:0/2275796947 shutdown_connections 2023-11-06T02:13:52.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.690+0000 7f37b3c68640 1 --2- 172.21.15.49:0/2275796947 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37ac0cb010 0x7f37ac0b0080 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:52.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.690+0000 7f37b3c68640 1 --2- 172.21.15.49:0/2275796947 >> v2:172.21.15.150:3300/0 conn(0x7f37ac15ab50 0x7f37ac0c9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:52.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.690+0000 7f37b3c68640 1 -- 172.21.15.49:0/2275796947 >> 172.21.15.49:0/2275796947 conn(0x7f37ac06c0a0 msgr2=0x7f37ac06c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:52.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.690+0000 7f37b3c68640 1 -- 172.21.15.49:0/2275796947 shutdown_connections 2023-11-06T02:13:52.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.690+0000 7f37b3c68640 1 -- 172.21.15.49:0/2275796947 wait complete. 2023-11-06T02:13:52.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.690+0000 7f37b3c68640 1 Processor -- start 2023-11-06T02:13:52.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.690+0000 7f37b3c68640 1 -- start start 2023-11-06T02:13:52.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.690+0000 7f37b3c68640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f37ac0b05c0 0x7f37ac0acf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:52.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.690+0000 7f37b3c68640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f37ac0cb010 0x7f37ac0cc240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:52.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.690+0000 7f37b3c68640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37ac15ab50 0x7f37ac1513e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:52.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.690+0000 7f37b3c68640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37ac1f60d0 con 0x7f37ac15ab50 2023-11-06T02:13:52.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.690+0000 7f37b3c68640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37ac14f180 con 0x7f37ac0b05c0 2023-11-06T02:13:52.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.690+0000 7f37b3c68640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37ac0ad4d0 con 0x7f37ac0cb010 2023-11-06T02:13:52.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.690+0000 7f37b2205640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37ac15ab50 0x7f37ac1513e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:52.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.690+0000 7f37b2205640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37ac15ab50 0x7f37ac1513e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33116/0 (socket says 172.21.15.49:33116) 2023-11-06T02:13:52.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.690+0000 7f37b2205640 1 -- 172.21.15.49:0/2312301759 learned_addr learned my addr 172.21.15.49:0/2312301759 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:52.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.690+0000 7f37b1203640 1 --2- 172.21.15.49:0/2312301759 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f37ac0cb010 0x7f37ac0cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:52.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.690+0000 7f37b2205640 1 -- 172.21.15.49:0/2312301759 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f37ac0cb010 msgr2=0x7f37ac0cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:52.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.690+0000 7f37b2205640 1 --2- 172.21.15.49:0/2312301759 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f37ac0cb010 0x7f37ac0cc240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:52.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.690+0000 7f37b2205640 1 -- 172.21.15.49:0/2312301759 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f37ac0b05c0 msgr2=0x7f37ac0acf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:52.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.690+0000 7f37b2205640 1 --2- 172.21.15.49:0/2312301759 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f37ac0b05c0 0x7f37ac0acf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:52.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.690+0000 7f37b2205640 1 -- 172.21.15.49:0/2312301759 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f379c009420 con 0x7f37ac15ab50 2023-11-06T02:13:52.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.690+0000 7f37b2205640 1 --2- 172.21.15.49:0/2312301759 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37ac15ab50 0x7f37ac1513e0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f37a400c3c0 tx=0x7f37a400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:52.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.690+0000 7f37a27fc640 1 -- 172.21.15.49:0/2312301759 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a4005cf0 con 0x7f37ac15ab50 2023-11-06T02:13:52.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.690+0000 7f37a27fc640 1 -- 172.21.15.49:0/2312301759 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37a4005e50 con 0x7f37ac15ab50 2023-11-06T02:13:52.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.690+0000 7f37b3c68640 1 -- 172.21.15.49:0/2312301759 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37ac1f5550 con 0x7f37ac15ab50 2023-11-06T02:13:52.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.690+0000 7f37a27fc640 1 -- 172.21.15.49:0/2312301759 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a40072f0 con 0x7f37ac15ab50 2023-11-06T02:13:52.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.690+0000 7f37b3c68640 1 -- 172.21.15.49:0/2312301759 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37ac1f5b10 con 0x7f37ac15ab50 2023-11-06T02:13:52.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.694+0000 7f37b3c68640 1 -- 172.21.15.49:0/2312301759 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37ac0b0de0 con 0x7f37ac15ab50 2023-11-06T02:13:52.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.694+0000 7f37a27fc640 1 -- 172.21.15.49:0/2312301759 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f37a4007450 con 0x7f37ac15ab50 2023-11-06T02:13:52.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.694+0000 7f37a27fc640 1 --2- 172.21.15.49:0/2312301759 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3790065a60 0x7f3790067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:52.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.694+0000 7f37a27fc640 1 -- 172.21.15.49:0/2312301759 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f37a40827d0 con 0x7f37ac15ab50 2023-11-06T02:13:52.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.694+0000 7f37b1a04640 1 --2- 172.21.15.49:0/2312301759 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3790065a60 0x7f3790067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:52.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.694+0000 7f37b1a04640 1 --2- 172.21.15.49:0/2312301759 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3790065a60 0x7f3790067f20 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f37a8008ef0 tx=0x7f37a8008420 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:52.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.698+0000 7f37a27fc640 1 -- 172.21.15.49:0/2312301759 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f37a4050030 con 0x7f37ac15ab50 2023-11-06T02:13:52.855 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.858+0000 7f37b3c68640 1 -- 172.21.15.49:0/2312301759 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f37ac14f340 con 0x7f37ac15ab50 2023-11-06T02:13:52.856 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.858+0000 7f37a27fc640 1 -- 172.21.15.49:0/2312301759 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v80) v1 ==== 80+0+0 (secure 0 0 0) 0x7f37a4017080 con 0x7f37ac15ab50 2023-11-06T02:13:52.856 INFO:teuthology.orchestra.run.smithi049.stderr:listed 0 entries 2023-11-06T02:13:52.857 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.862+0000 7f37b3c68640 1 -- 172.21.15.49:0/2312301759 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3790065a60 msgr2=0x7f3790067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:52.858 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.862+0000 7f37b3c68640 1 --2- 172.21.15.49:0/2312301759 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3790065a60 0x7f3790067f20 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f37a8008ef0 tx=0x7f37a8008420 comp rx=0 tx=0).stop 2023-11-06T02:13:52.858 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.862+0000 7f37b3c68640 1 -- 172.21.15.49:0/2312301759 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37ac15ab50 msgr2=0x7f37ac1513e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:52.858 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.862+0000 7f37b3c68640 1 --2- 172.21.15.49:0/2312301759 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37ac15ab50 0x7f37ac1513e0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f37a400c3c0 tx=0x7f37a400c880 comp rx=0 tx=0).stop 2023-11-06T02:13:52.858 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.862+0000 7f37b3c68640 1 -- 172.21.15.49:0/2312301759 shutdown_connections 2023-11-06T02:13:52.859 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.862+0000 7f37b3c68640 1 --2- 172.21.15.49:0/2312301759 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3790065a60 0x7f3790067f20 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:52.859 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.862+0000 7f37b3c68640 1 --2- 172.21.15.49:0/2312301759 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f37ac15ab50 0x7f37ac1513e0 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:52.859 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.862+0000 7f37b3c68640 1 --2- 172.21.15.49:0/2312301759 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f37ac0cb010 0x7f37ac0cc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:52.859 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.862+0000 7f37b3c68640 1 --2- 172.21.15.49:0/2312301759 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f37ac0b05c0 0x7f37ac0acf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:52.859 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.862+0000 7f37b3c68640 1 -- 172.21.15.49:0/2312301759 >> 172.21.15.49:0/2312301759 conn(0x7f37ac06c0a0 msgr2=0x7f37ac0ce570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:52.860 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.862+0000 7f37b3c68640 1 -- 172.21.15.49:0/2312301759 shutdown_connections 2023-11-06T02:13:52.860 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:52.862+0000 7f37b3c68640 1 -- 172.21.15.49:0/2312301759 wait complete. 2023-11-06T02:13:52.873 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi049.front.sepia.ceph.com... 2023-11-06T02:13:52.873 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:13:52.874 DEBUG:teuthology.orchestra.run.smithi049:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-06T02:13:52.884 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi049.stderr:ceph-fuse[54957]: fuse finished with error 0 and tester_r 0 2023-11-06T02:13:52.885 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T02:13:58.987 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi049.front.sepia.ceph.com 2023-11-06T02:13:58.987 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:13:58.988 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-06T02:13:58.996 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:0 flush journal 2023-11-06T02:13:59.174 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.178+0000 7f364aa02640 1 Processor -- start 2023-11-06T02:13:59.174 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.178+0000 7f364aa02640 1 -- start start 2023-11-06T02:13:59.175 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.178+0000 7f364aa02640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f364415d160 0x7f364415e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:59.175 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.178+0000 7f364aa02640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f364415e860 0x7f36440707b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:59.175 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.178+0000 7f364aa02640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f364415e100 con 0x7f3644153d50 2023-11-06T02:13:59.176 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.178+0000 7f364aa02640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f364406e0a0 con 0x7f364415d160 2023-11-06T02:13:59.176 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.178+0000 7f364aa02640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3644154130 con 0x7f364415e860 2023-11-06T02:13:59.177 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.182+0000 7f36437fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f364415d160 0x7f364415e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:59.177 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.182+0000 7f3648f9f640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f364415e860 0x7f36440707b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:59.177 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.182+0000 7f3643fff640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f3644153d50 0x7f36440c9100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.49:38822/0 (socket says 172.21.15.49:38822) 2023-11-06T02:13:59.177 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.182+0000 7f3643fff640 1 -- 172.21.15.49:0/577110990 learned_addr learned my addr 172.21.15.49:0/577110990 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:13:59.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.182+0000 7f3642ffd640 1 -- 172.21.15.49:0/577110990 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2526626271 0 0) 0x7f364415e100 con 0x7f3644153d50 2023-11-06T02:13:59.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.182+0000 7f36437fe640 1 -- 172.21.15.49:0/577110990 >> v2:172.21.15.150:3300/0 conn(0x7f364415e860 msgr2=0x7f36440707b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:59.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.182+0000 7f36437fe640 1 --2- 172.21.15.49:0/577110990 >> v2:172.21.15.150:3300/0 conn(0x7f364415e860 0x7f36440707b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:59.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.182+0000 7f36437fe640 1 -- 172.21.15.49:0/577110990 >> v1:172.21.15.150:6789/0 conn(0x7f3644153d50 legacy=0x7f36440c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:59.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.182+0000 7f36437fe640 1 -- 172.21.15.49:0/577110990 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f364415d5f0 con 0x7f364415d160 2023-11-06T02:13:59.180 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.182+0000 7f36437fe640 1 --2- 172.21.15.49:0/577110990 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f364415d160 0x7f364415e320 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f36440b1360 tx=0x7f363801c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97fd1e9df5d81b1e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:59.180 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.182+0000 7f3642ffd640 1 -- 172.21.15.49:0/577110990 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36380ab600 con 0x7f364415d160 2023-11-06T02:13:59.180 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.182+0000 7f3642ffd640 1 -- 172.21.15.49:0/577110990 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36380a7050 con 0x7f364415d160 2023-11-06T02:13:59.180 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.182+0000 7f3642ffd640 1 -- 172.21.15.49:0/577110990 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36380ab980 con 0x7f364415d160 2023-11-06T02:13:59.181 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.182+0000 7f364aa02640 1 -- 172.21.15.49:0/577110990 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f364415d160 msgr2=0x7f364415e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:59.181 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.182+0000 7f364aa02640 1 --2- 172.21.15.49:0/577110990 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f364415d160 0x7f364415e320 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f36440b1360 tx=0x7f363801c3a0 comp rx=0 tx=0).stop 2023-11-06T02:13:59.181 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.186+0000 7f364aa02640 1 -- 172.21.15.49:0/577110990 shutdown_connections 2023-11-06T02:13:59.181 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.186+0000 7f364aa02640 1 --2- 172.21.15.49:0/577110990 >> v2:172.21.15.150:3300/0 conn(0x7f364415e860 0x7f36440707b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:59.181 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.186+0000 7f364aa02640 1 --2- 172.21.15.49:0/577110990 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f364415d160 0x7f364415e320 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:59.182 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.186+0000 7f364aa02640 1 -- 172.21.15.49:0/577110990 >> 172.21.15.49:0/577110990 conn(0x7f364406a370 msgr2=0x7f36440ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:59.182 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.186+0000 7f364aa02640 1 -- 172.21.15.49:0/577110990 shutdown_connections 2023-11-06T02:13:59.182 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.186+0000 7f364aa02640 1 -- 172.21.15.49:0/577110990 wait complete. 2023-11-06T02:13:59.184 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.186+0000 7f364aa02640 1 Processor -- start 2023-11-06T02:13:59.184 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.186+0000 7f364aa02640 1 -- start start 2023-11-06T02:13:59.185 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.186+0000 7f364aa02640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3644153d50 0x7f364414b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:59.185 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.186+0000 7f364aa02640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f364415d160 0x7f3644149380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:59.185 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.186+0000 7f364aa02640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f364415e860 0x7f3644154c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:59.185 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.186+0000 7f364aa02640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3644151e00 con 0x7f364415d160 2023-11-06T02:13:59.185 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.186+0000 7f364aa02640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36441520e0 con 0x7f364415e860 2023-11-06T02:13:59.186 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.186+0000 7f364aa02640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f364414a740 con 0x7f3644153d50 2023-11-06T02:13:59.186 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.186+0000 7f36437fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f364415d160 0x7f3644149380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:59.186 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.186+0000 7f36437fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f364415d160 0x7f3644149380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51466/0 (socket says 172.21.15.49:51466) 2023-11-06T02:13:59.186 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.186+0000 7f36437fe640 1 -- 172.21.15.49:0/4215016683 learned_addr learned my addr 172.21.15.49:0/4215016683 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:59.186 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.186+0000 7f3648f9f640 1 --2- 172.21.15.49:0/4215016683 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f364415e860 0x7f3644154c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:59.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.186+0000 7f36437fe640 1 -- 172.21.15.49:0/4215016683 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3644153d50 msgr2=0x7f364414b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:59.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.186+0000 7f3643fff640 1 --2- 172.21.15.49:0/4215016683 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3644153d50 0x7f364414b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:59.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.186+0000 7f36437fe640 1 --2- 172.21.15.49:0/4215016683 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3644153d50 0x7f364414b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:59.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.186+0000 7f36437fe640 1 -- 172.21.15.49:0/4215016683 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f364415e860 msgr2=0x7f3644154c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:59.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.186+0000 7f36437fe640 1 --2- 172.21.15.49:0/4215016683 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f364415e860 0x7f3644154c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:59.188 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.186+0000 7f36437fe640 1 -- 172.21.15.49:0/4215016683 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3638009420 con 0x7f364415d160 2023-11-06T02:13:59.188 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.186+0000 7f3648f9f640 1 --2- 172.21.15.49:0/4215016683 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f364415e860 0x7f3644154c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:13:59.188 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.186+0000 7f36437fe640 1 --2- 172.21.15.49:0/4215016683 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f364415d160 0x7f3644149380 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f36380ae040 tx=0x7f3638019140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:59.188 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.186+0000 7f3640ff9640 1 -- 172.21.15.49:0/4215016683 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36380b4430 con 0x7f364415d160 2023-11-06T02:13:59.188 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.186+0000 7f3640ff9640 1 -- 172.21.15.49:0/4215016683 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36380b45b0 con 0x7f364415d160 2023-11-06T02:13:59.189 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.186+0000 7f364aa02640 1 -- 172.21.15.49:0/4215016683 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f364414aa20 con 0x7f364415d160 2023-11-06T02:13:59.189 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.186+0000 7f3640ff9640 1 -- 172.21.15.49:0/4215016683 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36380b4960 con 0x7f364415d160 2023-11-06T02:13:59.189 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.186+0000 7f364aa02640 1 -- 172.21.15.49:0/4215016683 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3644151a40 con 0x7f364415d160 2023-11-06T02:13:59.189 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.190+0000 7f3640ff9640 1 -- 172.21.15.49:0/4215016683 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f36380b2040 con 0x7f364415d160 2023-11-06T02:13:59.195 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.190+0000 7f3640ff9640 1 --2- 172.21.15.49:0/4215016683 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3630065a60 0x7f3630067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:59.195 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.190+0000 7f3640ff9640 1 -- 172.21.15.49:0/4215016683 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f363811d1e0 con 0x7f364415d160 2023-11-06T02:13:59.195 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.190+0000 7f3643fff640 1 --2- 172.21.15.49:0/4215016683 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3630065a60 0x7f3630067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:59.196 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.190+0000 7f364db2a000 1 Processor -- start 2023-11-06T02:13:59.196 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.190+0000 7f364db2a000 1 -- start start 2023-11-06T02:13:59.196 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.190+0000 7f3643fff640 1 --2- 172.21.15.49:0/4215016683 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3630065a60 0x7f3630067f20 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f3634000f70 tx=0x7f3634009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:59.196 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.190+0000 7f364db2a000 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563f799c3130 0x563f799c3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:59.196 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.190+0000 7f364db2a000 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563f799c3a50 0x563f79978d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:59.197 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.190+0000 7f364db2a000 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f79979260 0x563f79979640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:59.197 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.190+0000 7f364db2a000 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f79840c10 con 0x563f79979260 2023-11-06T02:13:59.197 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.190+0000 7f364db2a000 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f797cf3e0 con 0x563f799c3a50 2023-11-06T02:13:59.197 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.190+0000 7f364db2a000 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x563f797cc9b0 con 0x563f799c3130 2023-11-06T02:13:59.197 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.190+0000 7f3648f9f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f79979260 0x563f79979640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:59.198 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.190+0000 7f3648f9f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f79979260 0x563f79979640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51474/0 (socket says 172.21.15.49:51474) 2023-11-06T02:13:59.198 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.190+0000 7f3648f9f640 1 -- 172.21.15.49:0/3838244258 learned_addr learned my addr 172.21.15.49:0/3838244258 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:59.198 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.190+0000 7f3643fff640 1 --2- 172.21.15.49:0/3838244258 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563f799c3130 0x563f799c3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:59.198 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.190+0000 7f36437fe640 1 --2- 172.21.15.49:0/3838244258 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563f799c3a50 0x563f79978d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:59.198 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.190+0000 7f3648f9f640 1 -- 172.21.15.49:0/3838244258 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563f799c3130 msgr2=0x563f799c3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:59.199 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.190+0000 7f3648f9f640 1 --2- 172.21.15.49:0/3838244258 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563f799c3130 0x563f799c3510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:59.199 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.190+0000 7f3648f9f640 1 -- 172.21.15.49:0/3838244258 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563f799c3a50 msgr2=0x563f79978d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:59.199 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.190+0000 7f3648f9f640 1 --2- 172.21.15.49:0/3838244258 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563f799c3a50 0x563f79978d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:59.199 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.190+0000 7f3648f9f640 1 -- 172.21.15.49:0/3838244258 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563f794c4e20 con 0x563f79979260 2023-11-06T02:13:59.199 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.190+0000 7f3643fff640 1 --2- 172.21.15.49:0/3838244258 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563f799c3130 0x563f799c3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:13:59.200 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.190+0000 7f36437fe640 1 --2- 172.21.15.49:0/3838244258 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563f799c3a50 0x563f79978d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:13:59.200 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.190+0000 7f3648f9f640 1 --2- 172.21.15.49:0/3838244258 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f79979260 0x563f79979640 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f363c0074c0 tx=0x7f363c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddf3afe8db788882 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:59.200 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.194+0000 7f3629ffb640 1 -- 172.21.15.49:0/3838244258 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363c0147e0 con 0x563f79979260 2023-11-06T02:13:59.200 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.194+0000 7f3629ffb640 1 -- 172.21.15.49:0/3838244258 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f363c00f050 con 0x563f79979260 2023-11-06T02:13:59.200 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.194+0000 7f3629ffb640 1 -- 172.21.15.49:0/3838244258 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363c014b80 con 0x563f79979260 2023-11-06T02:13:59.200 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.194+0000 7f364db2a000 1 -- 172.21.15.49:0/3838244258 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f79979260 msgr2=0x563f79979640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:59.201 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.194+0000 7f364db2a000 1 --2- 172.21.15.49:0/3838244258 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f79979260 0x563f79979640 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f363c0074c0 tx=0x7f363c009750 comp rx=0 tx=0).stop 2023-11-06T02:13:59.201 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.194+0000 7f364db2a000 1 -- 172.21.15.49:0/3838244258 shutdown_connections 2023-11-06T02:13:59.201 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.194+0000 7f364db2a000 1 --2- 172.21.15.49:0/3838244258 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f79979260 0x563f79979640 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:59.201 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.194+0000 7f364db2a000 1 --2- 172.21.15.49:0/3838244258 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563f799c3a50 0x563f79978d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:59.201 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.194+0000 7f364db2a000 1 --2- 172.21.15.49:0/3838244258 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563f799c3130 0x563f799c3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:59.202 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.194+0000 7f364db2a000 1 -- 172.21.15.49:0/3838244258 >> 172.21.15.49:0/3838244258 conn(0x563f7997f9f0 msgr2=0x563f79977600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:59.202 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.194+0000 7f364db2a000 1 -- 172.21.15.49:0/3838244258 shutdown_connections 2023-11-06T02:13:59.202 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.194+0000 7f364db2a000 1 -- 172.21.15.49:0/3838244258 wait complete. 2023-11-06T02:13:59.202 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.194+0000 7f364db2a000 1 -- start start 2023-11-06T02:13:59.202 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.194+0000 7f364db2a000 1 Processor -- start 2023-11-06T02:13:59.203 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.194+0000 7f364db2a000 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563f79979260 0x563f799c26b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:59.203 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.194+0000 7f364db2a000 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f799c3130 0x563f799c3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:59.203 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.194+0000 7f364db2a000 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563f799c3a50 0x563f799c4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:59.203 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.194+0000 7f364db2a000 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f797c82a0 con 0x563f799c3130 2023-11-06T02:13:59.203 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.194+0000 7f364db2a000 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f79767470 con 0x563f799c3a50 2023-11-06T02:13:59.204 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.194+0000 7f364db2a000 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x563f79785660 con 0x563f79979260 2023-11-06T02:13:59.204 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.194+0000 7f364db2a000 10 client.0 _ll_register_callbacks cb 0x563f79964c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T02:13:59.204 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.194+0000 7f36437fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f799c3130 0x563f799c3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:59.205 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.194+0000 7f36437fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f799c3130 0x563f799c3e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51482/0 (socket says 172.21.15.49:51482) 2023-11-06T02:13:59.205 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.194+0000 7f36437fe640 1 -- 172.21.15.49:0/2543659310 learned_addr learned my addr 172.21.15.49:0/2543659310 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:59.205 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.194+0000 7f36437fe640 1 -- 172.21.15.49:0/2543659310 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563f79979260 msgr2=0x563f799c26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:59.206 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.194+0000 7f36437fe640 1 --2- 172.21.15.49:0/2543659310 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563f79979260 0x563f799c26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:59.206 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.194+0000 7f36437fe640 1 -- 172.21.15.49:0/2543659310 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563f799c3a50 msgr2=0x563f799c4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:59.207 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.194+0000 7f36437fe640 1 --2- 172.21.15.49:0/2543659310 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563f799c3a50 0x563f799c4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:59.207 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.194+0000 7f36437fe640 1 -- 172.21.15.49:0/2543659310 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f363c009df0 con 0x563f799c3130 2023-11-06T02:13:59.207 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.194+0000 7f36437fe640 1 --2- 172.21.15.49:0/2543659310 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f799c3130 0x563f799c3e30 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f36380eabf0 tx=0x7f36380ee7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce24ce49ec36e355 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:59.207 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.194+0000 7f3622ffd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-11-06T02:13:59.207 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.194+0000 7f3622ffd640 1 -- 172.21.15.49:0/2543659310 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36380f66d0 con 0x563f799c3130 2023-11-06T02:13:59.208 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.194+0000 7f3622ffd640 1 -- 172.21.15.49:0/2543659310 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f36380f2120 con 0x563f799c3130 2023-11-06T02:13:59.208 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.194+0000 7f364db2a000 1 -- 172.21.15.49:0/2543659310 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563f797850a0 con 0x563f799c3130 2023-11-06T02:13:59.208 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.194+0000 7f3622ffd640 1 -- 172.21.15.49:0/2543659310 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f36380f2320 con 0x563f799c3130 2023-11-06T02:13:59.208 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.194+0000 7f364db2a000 10 client.7416 fetch_fsmap learned FSMap version 46 2023-11-06T02:13:59.209 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.194+0000 7f364db2a000 1 -- 172.21.15.49:0/2543659310 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) v3 -- 0x563f7978d1d0 con 0x563f799c3130 2023-11-06T02:13:59.209 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.194+0000 7f3622ffd640 1 -- 172.21.15.49:0/2543659310 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563f797850a0 con 0x563f799c3130 2023-11-06T02:13:59.209 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.194+0000 7f3622ffd640 1 -- 172.21.15.49:0/2543659310 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 46) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f36380edbe0 con 0x563f799c3130 2023-11-06T02:13:59.209 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.194+0000 7f364db2a000 10 client.7416 fetch_fsmap finished waiting for FSMap version 46 2023-11-06T02:13:59.209 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.194+0000 7f364db2a000 10 client.7416 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2023-11-06T02:13:59.210 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.194+0000 7f364db2a000 20 client.7416 populate_metadata read hostname 'smithi049' 2023-11-06T02:13:59.210 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.194+0000 7f364db2a000 1 --2- 172.21.15.49:0/2543659310 >> [v2:172.21.15.49:6834/1130379293,v1:172.21.15.49:6835/1130379293] conn(0x563f799c4d80 0x563f799c5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:59.210 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.194+0000 7f364db2a000 4 client.7416 mds_command: new command op to 7116 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T02:13:59.210 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.194+0000 7f364db2a000 1 -- 172.21.15.49:0/2543659310 --> [v2:172.21.15.49:6834/1130379293,v1:172.21.15.49:6835/1130379293] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563f797c6bf0 con 0x563f799c4d80 2023-11-06T02:13:59.210 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.194+0000 7f3643fff640 1 --2- 172.21.15.49:0/2543659310 >> [v2:172.21.15.49:6834/1130379293,v1:172.21.15.49:6835/1130379293] conn(0x563f799c4d80 0x563f799c5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:59.211 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.198+0000 7f3643fff640 1 --2- 172.21.15.49:0/2543659310 >> [v2:172.21.15.49:6834/1130379293,v1:172.21.15.49:6835/1130379293] conn(0x563f799c4d80 0x563f799c5190 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2705bff7390c885 server_cookie=d4ee5cfd067fc309 in_seq=0 out_seq=0 2023-11-06T02:13:59.211 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.198+0000 7f3622ffd640 10 client.7416 ms_handle_connect on v2:172.21.15.49:6834/1130379293 2023-11-06T02:13:59.211 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.198+0000 7f3622ffd640 1 -- 172.21.15.49:0/2543659310 <== mds.0 v2:172.21.15.49:6834/1130379293 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x563f797c6bf0 con 0x563f799c4d80 2023-11-06T02:13:59.211 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.198+0000 7f3622ffd640 10 client.7416 handle_command_reply: tid=0 2023-11-06T02:13:59.211 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.198+0000 7f364db2a000 1 client.7416 shutdown 2023-11-06T02:13:59.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.198+0000 7f364db2a000 1 -- 172.21.15.49:0/2543659310 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f799c3130 msgr2=0x563f799c3e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:59.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.198+0000 7f364db2a000 1 --2- 172.21.15.49:0/2543659310 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f799c3130 0x563f799c3e30 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f36380eabf0 tx=0x7f36380ee7b0 comp rx=0 tx=0).stop 2023-11-06T02:13:59.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.198+0000 7f364db2a000 1 -- 172.21.15.49:0/2543659310 shutdown_connections 2023-11-06T02:13:59.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.198+0000 7f364db2a000 1 --2- 172.21.15.49:0/2543659310 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f799c3130 0x563f799c3e30 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:59.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.198+0000 7f364db2a000 1 --2- 172.21.15.49:0/2543659310 >> [v2:172.21.15.49:6834/1130379293,v1:172.21.15.49:6835/1130379293] conn(0x563f799c4d80 0x563f799c5190 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:59.213 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.198+0000 7f364db2a000 1 --2- 172.21.15.49:0/2543659310 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563f799c3a50 0x563f799c4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:59.213 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.198+0000 7f364db2a000 1 --2- 172.21.15.49:0/2543659310 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563f79979260 0x563f799c26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:59.213 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.198+0000 7f3622ffd640 0 client.7416 ms_handle_reset on v2:172.21.15.49:6834/1130379293 2023-11-06T02:13:59.213 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.198+0000 7f364db2a000 1 -- 172.21.15.49:0/2543659310 >> 172.21.15.49:0/2543659310 conn(0x563f7997f9f0 msgr2=0x563f79977d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:59.213 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.198+0000 7f364db2a000 1 -- 172.21.15.49:0/2543659310 shutdown_connections 2023-11-06T02:13:59.214 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.198+0000 7f364db2a000 1 -- 172.21.15.49:0/2543659310 wait complete. 2023-11-06T02:13:59.214 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.198+0000 7f364db2a000 20 client.7416 trim_cache size 0 max 16384 2023-11-06T02:13:59.218 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.210+0000 7f364db2a000 1 Processor -- start 2023-11-06T02:13:59.218 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.210+0000 7f364db2a000 1 -- start start 2023-11-06T02:13:59.219 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.210+0000 7f364db2a000 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563f79979260 0x563f799c1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:59.219 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.210+0000 7f364db2a000 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f799c3130 0x563f799c1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:59.219 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.210+0000 7f364db2a000 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563f799c3a50 0x563f799c71a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:59.219 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.210+0000 7f364db2a000 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f79485aa0 con 0x563f799c3130 2023-11-06T02:13:59.219 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.210+0000 7f364db2a000 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f797c5ab0 con 0x563f79979260 2023-11-06T02:13:59.220 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.210+0000 7f364db2a000 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x563f79786400 con 0x563f799c3a50 2023-11-06T02:13:59.220 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.210+0000 7f36437fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f799c3130 0x563f799c1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:59.220 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.210+0000 7f36437fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f799c3130 0x563f799c1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51490/0 (socket says 172.21.15.49:51490) 2023-11-06T02:13:59.220 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.210+0000 7f36437fe640 1 -- 172.21.15.49:0/1950927464 learned_addr learned my addr 172.21.15.49:0/1950927464 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:59.220 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.210+0000 7f36437fe640 1 -- 172.21.15.49:0/1950927464 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563f799c3a50 msgr2=0x563f799c71a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:59.221 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.210+0000 7f36437fe640 1 --2- 172.21.15.49:0/1950927464 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563f799c3a50 0x563f799c71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:59.221 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.210+0000 7f36437fe640 1 -- 172.21.15.49:0/1950927464 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563f79979260 msgr2=0x563f799c1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:59.221 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.210+0000 7f36437fe640 1 --2- 172.21.15.49:0/1950927464 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563f79979260 0x563f799c1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:59.221 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.210+0000 7f36437fe640 1 -- 172.21.15.49:0/1950927464 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563f79773520 con 0x563f799c3130 2023-11-06T02:13:59.221 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.210+0000 7f36437fe640 1 --2- 172.21.15.49:0/1950927464 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f799c3130 0x563f799c1a50 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f36380f26e0 tx=0x7f36380b0000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b6d8ae75bc0f7fb server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:59.222 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.210+0000 7f362a7fc640 1 -- 172.21.15.49:0/1950927464 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36380ee4c0 con 0x563f799c3130 2023-11-06T02:13:59.222 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.210+0000 7f362a7fc640 1 -- 172.21.15.49:0/1950927464 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36380eedf0 con 0x563f799c3130 2023-11-06T02:13:59.222 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.210+0000 7f362a7fc640 1 -- 172.21.15.49:0/1950927464 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36380f66d0 con 0x563f799c3130 2023-11-06T02:13:59.222 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.210+0000 7f364db2a000 1 -- 172.21.15.49:0/1950927464 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f799c3130 msgr2=0x563f799c1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:59.222 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.210+0000 7f364db2a000 1 --2- 172.21.15.49:0/1950927464 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f799c3130 0x563f799c1a50 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f36380f26e0 tx=0x7f36380b0000 comp rx=0 tx=0).stop 2023-11-06T02:13:59.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.210+0000 7f364db2a000 1 -- 172.21.15.49:0/1950927464 shutdown_connections 2023-11-06T02:13:59.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.210+0000 7f364db2a000 1 --2- 172.21.15.49:0/1950927464 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f799c3130 0x563f799c1a50 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:59.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.210+0000 7f364db2a000 1 --2- 172.21.15.49:0/1950927464 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563f799c3a50 0x563f799c71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:59.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.210+0000 7f364db2a000 1 --2- 172.21.15.49:0/1950927464 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563f79979260 0x563f799c1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:59.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.210+0000 7f364db2a000 1 -- 172.21.15.49:0/1950927464 >> 172.21.15.49:0/1950927464 conn(0x563f7997f9f0 msgr2=0x563f7997daf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:59.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.214+0000 7f364db2a000 1 -- 172.21.15.49:0/1950927464 shutdown_connections 2023-11-06T02:13:59.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.214+0000 7f364db2a000 1 -- 172.21.15.49:0/1950927464 wait complete. 2023-11-06T02:13:59.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.214+0000 7f364db2a000 1 -- start start 2023-11-06T02:13:59.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.214+0000 7f364db2a000 1 Processor -- start 2023-11-06T02:13:59.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.214+0000 7f364db2a000 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563f79979260 0x563f79a20100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:59.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.214+0000 7f364db2a000 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f799c3130 0x563f79a20640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:59.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.214+0000 7f364db2a000 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563f799c3a50 0x563f799bd300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:59.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.214+0000 7f364db2a000 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f797c2640 con 0x563f799c3130 2023-11-06T02:13:59.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.214+0000 7f364db2a000 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f797cc2a0 con 0x563f79979260 2023-11-06T02:13:59.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.214+0000 7f364db2a000 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x563f797c56a0 con 0x563f799c3a50 2023-11-06T02:13:59.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.214+0000 7f364db2a000 10 client.0 _ll_register_callbacks cb 0x563f7978ccb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T02:13:59.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.214+0000 7f36437fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f799c3130 0x563f79a20640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:59.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.214+0000 7f36437fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f799c3130 0x563f79a20640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51492/0 (socket says 172.21.15.49:51492) 2023-11-06T02:13:59.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.214+0000 7f36437fe640 1 -- 172.21.15.49:0/1606410371 learned_addr learned my addr 172.21.15.49:0/1606410371 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:59.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.214+0000 7f3643fff640 1 --2- 172.21.15.49:0/1606410371 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563f79979260 0x563f79a20100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:59.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.214+0000 7f3648f9f640 1 --2- 172.21.15.49:0/1606410371 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563f799c3a50 0x563f799bd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:59.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.214+0000 7f36437fe640 1 -- 172.21.15.49:0/1606410371 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563f799c3a50 msgr2=0x563f799bd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:59.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.214+0000 7f36437fe640 1 --2- 172.21.15.49:0/1606410371 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563f799c3a50 0x563f799bd300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:59.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.214+0000 7f36437fe640 1 -- 172.21.15.49:0/1606410371 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563f79979260 msgr2=0x563f79a20100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:59.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.214+0000 7f36437fe640 1 --2- 172.21.15.49:0/1606410371 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563f79979260 0x563f79a20100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:59.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.214+0000 7f36437fe640 1 -- 172.21.15.49:0/1606410371 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563f7978d1d0 con 0x563f799c3130 2023-11-06T02:13:59.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.214+0000 7f3648f9f640 1 --2- 172.21.15.49:0/1606410371 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563f799c3a50 0x563f799bd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:13:59.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.214+0000 7f36437fe640 1 --2- 172.21.15.49:0/1606410371 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f799c3130 0x563f79a20640 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f363801c910 tx=0x7f363811b470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23fcf4ded86d4c07 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:59.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.214+0000 7f3622ffd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-11-06T02:13:59.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.214+0000 7f3622ffd640 1 -- 172.21.15.49:0/1606410371 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36380f66d0 con 0x563f799c3130 2023-11-06T02:13:59.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.214+0000 7f3622ffd640 1 -- 172.21.15.49:0/1606410371 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f363810ddb0 con 0x563f799c3130 2023-11-06T02:13:59.229 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.214+0000 7f364db2a000 1 -- 172.21.15.49:0/1606410371 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563f79791620 con 0x563f799c3130 2023-11-06T02:13:59.229 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.214+0000 7f3622ffd640 1 -- 172.21.15.49:0/1606410371 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f36380edbe0 con 0x563f799c3130 2023-11-06T02:13:59.229 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.214+0000 7f364db2a000 10 client.7422 fetch_fsmap learned FSMap version 46 2023-11-06T02:13:59.229 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.214+0000 7f364db2a000 1 -- 172.21.15.49:0/1606410371 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) v3 -- 0x563f79767090 con 0x563f799c3130 2023-11-06T02:13:59.229 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.214+0000 7f3622ffd640 1 -- 172.21.15.49:0/1606410371 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563f79791620 con 0x563f799c3130 2023-11-06T02:13:59.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.214+0000 7f3622ffd640 1 -- 172.21.15.49:0/1606410371 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 46) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f363810b070 con 0x563f799c3130 2023-11-06T02:13:59.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.214+0000 7f364db2a000 10 client.7422 fetch_fsmap finished waiting for FSMap version 46 2023-11-06T02:13:59.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.214+0000 7f364db2a000 10 client.7422 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2023-11-06T02:13:59.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.214+0000 7f364db2a000 20 client.7422 populate_metadata read hostname 'smithi049' 2023-11-06T02:13:59.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.214+0000 7f364db2a000 1 --2- 172.21.15.49:0/1606410371 >> [v2:172.21.15.49:6834/1130379293,v1:172.21.15.49:6835/1130379293] conn(0x563f799bd840 0x563f799bdc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:59.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.214+0000 7f364db2a000 4 client.7422 mds_command: new command op to 7116 tid=0[{"prefix": "flush journal"}] 2023-11-06T02:13:59.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.214+0000 7f364db2a000 1 -- 172.21.15.49:0/1606410371 --> [v2:172.21.15.49:6834/1130379293,v1:172.21.15.49:6835/1130379293] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563f797c1320 con 0x563f799bd840 2023-11-06T02:13:59.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.214+0000 7f3643fff640 1 --2- 172.21.15.49:0/1606410371 >> [v2:172.21.15.49:6834/1130379293,v1:172.21.15.49:6835/1130379293] conn(0x563f799bd840 0x563f799bdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:59.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.214+0000 7f3622ffd640 10 client.7422 ms_handle_connect on v2:172.21.15.49:6834/1130379293 2023-11-06T02:13:59.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.214+0000 7f3643fff640 1 --2- 172.21.15.49:0/1606410371 >> [v2:172.21.15.49:6834/1130379293,v1:172.21.15.49:6835/1130379293] conn(0x563f799bd840 0x563f799bdc20 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ff25f89a2f452d9 server_cookie=c1dc80c0dfa3107c in_seq=0 out_seq=0 2023-11-06T02:13:59.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.230+0000 7f3622ffd640 1 -- 172.21.15.49:0/1606410371 <== mds.0 v2:172.21.15.49:6834/1130379293 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7f363401a040 con 0x563f799bd840 2023-11-06T02:13:59.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.230+0000 7f3622ffd640 10 client.7422 handle_command_reply: tid=0 2023-11-06T02:13:59.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.230+0000 7f364db2a000 1 client.7422 shutdown 2023-11-06T02:13:59.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.230+0000 7f364db2a000 1 -- 172.21.15.49:0/1606410371 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f799c3130 msgr2=0x563f79a20640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:59.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.230+0000 7f364db2a000 1 --2- 172.21.15.49:0/1606410371 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f799c3130 0x563f79a20640 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f363801c910 tx=0x7f363811b470 comp rx=0 tx=0).stop 2023-11-06T02:13:59.234 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.230+0000 7f364db2a000 1 -- 172.21.15.49:0/1606410371 shutdown_connections 2023-11-06T02:13:59.234 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.230+0000 7f364db2a000 1 --2- 172.21.15.49:0/1606410371 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563f799c3130 0x563f79a20640 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:59.234 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.230+0000 7f364db2a000 1 --2- 172.21.15.49:0/1606410371 >> [v2:172.21.15.49:6834/1130379293,v1:172.21.15.49:6835/1130379293] conn(0x563f799bd840 0x563f799bdc20 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:59.234 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.230+0000 7f364db2a000 1 --2- 172.21.15.49:0/1606410371 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563f799c3a50 0x563f799bd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:59.234 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.230+0000 7f3622ffd640 0 client.7422 ms_handle_reset on v2:172.21.15.49:6834/1130379293 2023-11-06T02:13:59.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.230+0000 7f364db2a000 1 --2- 172.21.15.49:0/1606410371 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563f79979260 0x563f79a20100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:59.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.230+0000 7f364db2a000 1 -- 172.21.15.49:0/1606410371 >> 172.21.15.49:0/1606410371 conn(0x563f7997f9f0 msgr2=0x563f7997daf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:59.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.230+0000 7f364db2a000 1 -- 172.21.15.49:0/1606410371 shutdown_connections 2023-11-06T02:13:59.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.230+0000 7f364db2a000 1 -- 172.21.15.49:0/1606410371 wait complete. 2023-11-06T02:13:59.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.230+0000 7f364db2a000 20 client.7422 trim_cache size 0 max 16384 2023-11-06T02:13:59.236 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-11-06T02:13:59.236 INFO:teuthology.orchestra.run.smithi049.stdout: "message": "", 2023-11-06T02:13:59.236 INFO:teuthology.orchestra.run.smithi049.stdout: "return_code": 0 2023-11-06T02:13:59.236 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-11-06T02:13:59.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.234+0000 7f364aa02640 1 -- 172.21.15.49:0/4215016683 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3630065a60 msgr2=0x7f3630067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:59.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.234+0000 7f364aa02640 1 --2- 172.21.15.49:0/4215016683 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3630065a60 0x7f3630067f20 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f3634000f70 tx=0x7f3634009040 comp rx=0 tx=0).stop 2023-11-06T02:13:59.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.234+0000 7f364aa02640 1 -- 172.21.15.49:0/4215016683 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f364415d160 msgr2=0x7f3644149380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:59.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.234+0000 7f364aa02640 1 --2- 172.21.15.49:0/4215016683 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f364415d160 0x7f3644149380 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f36380ae040 tx=0x7f3638019140 comp rx=0 tx=0).stop 2023-11-06T02:13:59.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.234+0000 7f364aa02640 1 -- 172.21.15.49:0/4215016683 shutdown_connections 2023-11-06T02:13:59.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.234+0000 7f364aa02640 1 --2- 172.21.15.49:0/4215016683 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3630065a60 0x7f3630067f20 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:59.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.234+0000 7f364aa02640 1 --2- 172.21.15.49:0/4215016683 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f364415d160 0x7f3644149380 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:59.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.234+0000 7f364aa02640 1 --2- 172.21.15.49:0/4215016683 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f364415e860 0x7f3644154c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:59.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.234+0000 7f364aa02640 1 --2- 172.21.15.49:0/4215016683 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3644153d50 0x7f364414b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:59.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.234+0000 7f364aa02640 1 -- 172.21.15.49:0/4215016683 >> 172.21.15.49:0/4215016683 conn(0x7f364406a370 msgr2=0x7f36440cb560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:59.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.234+0000 7f364aa02640 1 -- 172.21.15.49:0/4215016683 shutdown_connections 2023-11-06T02:13:59.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.234+0000 7f364aa02640 1 -- 172.21.15.49:0/4215016683 wait complete. 2023-11-06T02:13:59.245 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-06T02:13:59.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.362+0000 7f4b22ab4640 1 Processor -- start 2023-11-06T02:13:59.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.362+0000 7f4b22ab4640 1 -- start start 2023-11-06T02:13:59.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.366+0000 7f4b22ab4640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4b1c0cb010 0x7f4b1c0c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:59.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.366+0000 7f4b22ab4640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b1c0cab80 con 0x7f4b1c0b0790 2023-11-06T02:13:59.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.366+0000 7f4b22ab4640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b1c15af30 con 0x7f4b1c15ab50 2023-11-06T02:13:59.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.366+0000 7f4b22ab4640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b1c0cb5b0 con 0x7f4b1c0cb010 2023-11-06T02:13:59.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.366+0000 7f4b1bfff640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f4b1c0b0790 0x7f4b1c0c2710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:59278/0 (socket says 172.21.15.49:59278) 2023-11-06T02:13:59.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.366+0000 7f4b1bfff640 1 -- 172.21.15.49:0/1477443971 learned_addr learned my addr 172.21.15.49:0/1477443971 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:13:59.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.366+0000 7f4b1b7fe640 1 -- 172.21.15.49:0/1477443971 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2336074152 0 0) 0x7f4b1c0cab80 con 0x7f4b1c0b0790 2023-11-06T02:13:59.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.366+0000 7f4b1b7fe640 1 -- 172.21.15.49:0/1477443971 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b08003530 con 0x7f4b1c0b0790 2023-11-06T02:13:59.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.366+0000 7f4b21051640 1 --2- 172.21.15.49:0/1477443971 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4b1c0cb010 0x7f4b1c0c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:59.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.366+0000 7f4b1b7fe640 1 -- 172.21.15.49:0/1477443971 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 253096347 0 0) 0x7f4b08003530 con 0x7f4b1c0b0790 2023-11-06T02:13:59.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.366+0000 7f4b1b7fe640 1 -- 172.21.15.49:0/1477443971 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4b1c0cb010 msgr2=0x7f4b1c0c9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:59.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.366+0000 7f4b1b7fe640 1 --2- 172.21.15.49:0/1477443971 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4b1c0cb010 0x7f4b1c0c9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:59.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.366+0000 7f4b1b7fe640 1 -- 172.21.15.49:0/1477443971 >> v1:172.21.15.150:6789/0 conn(0x7f4b1c15ab50 legacy=0x7f4b1c0b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:59.365 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.366+0000 7f4b1b7fe640 1 -- 172.21.15.49:0/1477443971 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b1c0cb890 con 0x7f4b1c0b0790 2023-11-06T02:13:59.365 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.366+0000 7f4b1b7fe640 1 -- 172.21.15.49:0/1477443971 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7f4b040016a0 con 0x7f4b1c0b0790 2023-11-06T02:13:59.365 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.366+0000 7f4b1b7fe640 1 -- 172.21.15.49:0/1477443971 >> v1:172.21.15.49:6789/0 conn(0x7f4b1c0b0790 legacy=0x7f4b1c0c2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:59.365 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.366+0000 7f4b1b7fe640 1 --2- 172.21.15.49:0/1477443971 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b080039b0 0x7f4b08003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:59.365 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.366+0000 7f4b1b7fe640 1 --2- 172.21.15.49:0/1477443971 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4b1c0cb010 0x7f4b080047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:59.366 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.366+0000 7f4b1b7fe640 1 --2- 172.21.15.49:0/1477443971 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4b08004d10 0x7f4b08005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:59.366 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.366+0000 7f4b1b7fe640 1 -- 172.21.15.49:0/1477443971 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b1c0cb5b0 con 0x7f4b080039b0 2023-11-06T02:13:59.366 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.366+0000 7f4b1b7fe640 1 -- 172.21.15.49:0/1477443971 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b0800bb20 con 0x7f4b08004d10 2023-11-06T02:13:59.366 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.366+0000 7f4b1b7fe640 1 -- 172.21.15.49:0/1477443971 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b0800bcb0 con 0x7f4b1c0cb010 2023-11-06T02:13:59.366 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.366+0000 7f4b20850640 1 --2- 172.21.15.49:0/1477443971 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b080039b0 0x7f4b08003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:59.367 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.366+0000 7f4b1bfff640 1 --2- 172.21.15.49:0/1477443971 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4b1c0cb010 0x7f4b080047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:59.367 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.366+0000 7f4b21051640 1 --2- 172.21.15.49:0/1477443971 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4b08004d10 0x7f4b08005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:59.367 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.366+0000 7f4b20850640 1 -- 172.21.15.49:0/1477443971 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4b1c0cb010 msgr2=0x7f4b080047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:59.367 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.366+0000 7f4b20850640 1 --2- 172.21.15.49:0/1477443971 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4b1c0cb010 0x7f4b080047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:59.367 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.366+0000 7f4b20850640 1 -- 172.21.15.49:0/1477443971 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4b08004d10 msgr2=0x7f4b08005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:13:59.367 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.366+0000 7f4b20850640 1 --2- 172.21.15.49:0/1477443971 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4b08004d10 0x7f4b08005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:59.368 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.366+0000 7f4b20850640 1 -- 172.21.15.49:0/1477443971 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b08003530 con 0x7f4b080039b0 2023-11-06T02:13:59.368 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.366+0000 7f4b21051640 1 --2- 172.21.15.49:0/1477443971 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4b08004d10 0x7f4b08005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:13:59.371 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.370+0000 7f4b20850640 1 --2- 172.21.15.49:0/1477443971 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b080039b0 0x7f4b08003dd0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f4b14001ee0 tx=0x7f4b1401c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47c5e05100fd9aa2 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:59.371 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.370+0000 7f4b1b7fe640 1 -- 172.21.15.49:0/1477443971 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b14003bd0 con 0x7f4b080039b0 2023-11-06T02:13:59.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.370+0000 7f4b1b7fe640 1 -- 172.21.15.49:0/1477443971 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b14003d30 con 0x7f4b080039b0 2023-11-06T02:13:59.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.370+0000 7f4b1b7fe640 1 -- 172.21.15.49:0/1477443971 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b14003ea0 con 0x7f4b080039b0 2023-11-06T02:13:59.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.370+0000 7f4b22ab4640 1 -- 172.21.15.49:0/1477443971 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b080039b0 msgr2=0x7f4b08003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:59.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.370+0000 7f4b22ab4640 1 --2- 172.21.15.49:0/1477443971 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b080039b0 0x7f4b08003dd0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f4b14001ee0 tx=0x7f4b1401c710 comp rx=0 tx=0).stop 2023-11-06T02:13:59.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.370+0000 7f4b21051640 1 -- 172.21.15.49:0/1477443971 reap_dead start 2023-11-06T02:13:59.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.370+0000 7f4b22ab4640 1 -- 172.21.15.49:0/1477443971 shutdown_connections 2023-11-06T02:13:59.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.370+0000 7f4b22ab4640 1 -- 172.21.15.49:0/1477443971 >> 172.21.15.49:0/1477443971 conn(0x7f4b1c06c0a0 msgr2=0x7f4b1c06c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:59.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.370+0000 7f4b22ab4640 1 -- 172.21.15.49:0/1477443971 shutdown_connections 2023-11-06T02:13:59.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.370+0000 7f4b22ab4640 1 -- 172.21.15.49:0/1477443971 wait complete. 2023-11-06T02:13:59.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.370+0000 7f4b22ab4640 1 Processor -- start 2023-11-06T02:13:59.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.370+0000 7f4b22ab4640 1 -- start start 2023-11-06T02:13:59.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.370+0000 7f4b22ab4640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4b080039b0 0x7f4b1c153600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:59.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.370+0000 7f4b22ab4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b1c15b150 0x7f4b1c0c7f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:59.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.370+0000 7f4b22ab4640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4b1c152d60 0x7f4b1c0c5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:59.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.370+0000 7f4b22ab4640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b1c153f00 con 0x7f4b1c15b150 2023-11-06T02:13:59.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.370+0000 7f4b22ab4640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b1c0ae5a0 con 0x7f4b080039b0 2023-11-06T02:13:59.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.370+0000 7f4b22ab4640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b1c15b5d0 con 0x7f4b1c152d60 2023-11-06T02:13:59.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.370+0000 7f4b1bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b1c15b150 0x7f4b1c0c7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:59.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.370+0000 7f4b1bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b1c15b150 0x7f4b1c0c7f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51506/0 (socket says 172.21.15.49:51506) 2023-11-06T02:13:59.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.370+0000 7f4b1bfff640 1 -- 172.21.15.49:0/903888302 learned_addr learned my addr 172.21.15.49:0/903888302 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:13:59.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.370+0000 7f4b20850640 1 --2- 172.21.15.49:0/903888302 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4b080039b0 0x7f4b1c153600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:59.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.370+0000 7f4b21051640 1 --2- 172.21.15.49:0/903888302 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4b1c152d60 0x7f4b1c0c5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:59.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.370+0000 7f4b1bfff640 1 -- 172.21.15.49:0/903888302 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4b1c152d60 msgr2=0x7f4b1c0c5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:59.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.370+0000 7f4b1bfff640 1 --2- 172.21.15.49:0/903888302 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4b1c152d60 0x7f4b1c0c5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:59.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.370+0000 7f4b1bfff640 1 -- 172.21.15.49:0/903888302 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4b080039b0 msgr2=0x7f4b1c153600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:59.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.370+0000 7f4b1bfff640 1 --2- 172.21.15.49:0/903888302 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4b080039b0 0x7f4b1c153600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:59.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.370+0000 7f4b1bfff640 1 -- 172.21.15.49:0/903888302 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b1c0c8490 con 0x7f4b1c15b150 2023-11-06T02:13:59.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.374+0000 7f4b1bfff640 1 --2- 172.21.15.49:0/903888302 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b1c15b150 0x7f4b1c0c7f50 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f4b04001230 tx=0x7f4b0400ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:59.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.374+0000 7f4b197fa640 1 -- 172.21.15.49:0/903888302 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b0400c8d0 con 0x7f4b1c15b150 2023-11-06T02:13:59.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.374+0000 7f4b197fa640 1 -- 172.21.15.49:0/903888302 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b0400ca50 con 0x7f4b1c15b150 2023-11-06T02:13:59.378 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.374+0000 7f4b22ab4640 1 -- 172.21.15.49:0/903888302 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b1c0ae7b0 con 0x7f4b1c15b150 2023-11-06T02:13:59.378 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.374+0000 7f4b197fa640 1 -- 172.21.15.49:0/903888302 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b0400ce00 con 0x7f4b1c15b150 2023-11-06T02:13:59.378 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.374+0000 7f4b22ab4640 1 -- 172.21.15.49:0/903888302 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b1c0aed70 con 0x7f4b1c15b150 2023-11-06T02:13:59.378 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.374+0000 7f4b22ab4640 1 -- 172.21.15.49:0/903888302 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b1c0b0de0 con 0x7f4b1c15b150 2023-11-06T02:13:59.378 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.374+0000 7f4b197fa640 1 -- 172.21.15.49:0/903888302 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f4b04013420 con 0x7f4b1c15b150 2023-11-06T02:13:59.379 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.374+0000 7f4b197fa640 1 --2- 172.21.15.49:0/903888302 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4afc065a90 0x7f4afc067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:13:59.379 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.374+0000 7f4b197fa640 1 -- 172.21.15.49:0/903888302 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4b04082ed0 con 0x7f4b1c15b150 2023-11-06T02:13:59.379 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.374+0000 7f4b20850640 1 --2- 172.21.15.49:0/903888302 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4afc065a90 0x7f4afc067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:13:59.379 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.374+0000 7f4b20850640 1 --2- 172.21.15.49:0/903888302 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4afc065a90 0x7f4afc067f50 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f4b140ad000 tx=0x7f4b14001570 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:13:59.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.378+0000 7f4b197fa640 1 -- 172.21.15.49:0/903888302 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f4b040507c0 con 0x7f4b1c15b150 2023-11-06T02:13:59.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.570+0000 7f4b22ab4640 1 -- 172.21.15.49:0/903888302 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4b1c074040 con 0x7f4b1c15b150 2023-11-06T02:13:59.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.570+0000 7f4b197fa640 1 -- 172.21.15.49:0/903888302 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v46) v1 ==== 53+0+73 (secure 0 0 0) 0x7f4b04053de0 con 0x7f4b1c15b150 2023-11-06T02:13:59.567 INFO:teuthology.orchestra.run.smithi049.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-06T02:13:59.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.574+0000 7f4b22ab4640 1 -- 172.21.15.49:0/903888302 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4afc065a90 msgr2=0x7f4afc067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:59.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.574+0000 7f4b22ab4640 1 --2- 172.21.15.49:0/903888302 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4afc065a90 0x7f4afc067f50 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f4b140ad000 tx=0x7f4b14001570 comp rx=0 tx=0).stop 2023-11-06T02:13:59.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.574+0000 7f4b22ab4640 1 -- 172.21.15.49:0/903888302 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b1c15b150 msgr2=0x7f4b1c0c7f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:13:59.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.574+0000 7f4b22ab4640 1 --2- 172.21.15.49:0/903888302 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b1c15b150 0x7f4b1c0c7f50 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f4b04001230 tx=0x7f4b0400ec60 comp rx=0 tx=0).stop 2023-11-06T02:13:59.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.574+0000 7f4b22ab4640 1 -- 172.21.15.49:0/903888302 shutdown_connections 2023-11-06T02:13:59.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.574+0000 7f4b22ab4640 1 --2- 172.21.15.49:0/903888302 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4afc065a90 0x7f4afc067f50 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:59.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.574+0000 7f4b22ab4640 1 --2- 172.21.15.49:0/903888302 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b1c15b150 0x7f4b1c0c7f50 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:59.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.574+0000 7f4b22ab4640 1 --2- 172.21.15.49:0/903888302 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4b1c152d60 0x7f4b1c0c5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:59.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.574+0000 7f4b22ab4640 1 --2- 172.21.15.49:0/903888302 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4b080039b0 0x7f4b1c153600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:13:59.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.574+0000 7f4b22ab4640 1 -- 172.21.15.49:0/903888302 >> 172.21.15.49:0/903888302 conn(0x7f4b1c06c0a0 msgr2=0x7f4b1c0ce570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:13:59.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.574+0000 7f4b22ab4640 1 -- 172.21.15.49:0/903888302 shutdown_connections 2023-11-06T02:13:59.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:13:59.574+0000 7f4b22ab4640 1 -- 172.21.15.49:0/903888302 wait complete. 2023-11-06T02:13:59.584 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-06T02:13:59.584 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-06T02:13:59.585 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi049.front.sepia.ceph.com 2023-11-06T02:13:59.585 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-06T02:13:59.585 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-06T02:13:59.586 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-06T02:13:59.586 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-06T02:13:59.586 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-06T02:13:59.587 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-06T02:13:59.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:13:59.588 DEBUG:teuthology.orchestra.run.smithi049:> ip addr 2023-11-06T02:13:59.594 INFO:teuthology.orchestra.run.smithi049.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-06T02:13:59.594 INFO:teuthology.orchestra.run.smithi049.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-06T02:13:59.594 INFO:teuthology.orchestra.run.smithi049.stdout: inet 127.0.0.1/8 scope host lo 2023-11-06T02:13:59.594 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:13:59.595 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 ::1/128 scope host 2023-11-06T02:13:59.595 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:13:59.595 INFO:teuthology.orchestra.run.smithi049.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-06T02:13:59.595 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:6b:ff:84 brd ff:ff:ff:ff:ff:ff 2023-11-06T02:13:59.595 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp7s0f0 2023-11-06T02:13:59.596 INFO:teuthology.orchestra.run.smithi049.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-06T02:13:59.596 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:6b:ff:85 brd ff:ff:ff:ff:ff:ff 2023-11-06T02:13:59.596 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp7s0f1 2023-11-06T02:13:59.596 INFO:teuthology.orchestra.run.smithi049.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-06T02:13:59.596 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:bd:15:66 brd ff:ff:ff:ff:ff:ff 2023-11-06T02:13:59.597 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp1s0f0 2023-11-06T02:13:59.597 INFO:teuthology.orchestra.run.smithi049.stdout: inet 172.21.15.49/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-11-06T02:13:59.597 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft 42158sec preferred_lft 42158sec 2023-11-06T02:13:59.597 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 fe80::ec4:7aff:febd:1566/64 scope link 2023-11-06T02:13:59.597 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:13:59.598 INFO:teuthology.orchestra.run.smithi049.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-06T02:13:59.598 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:bd:15:67 brd ff:ff:ff:ff:ff:ff 2023-11-06T02:13:59.598 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp1s0f1 2023-11-06T02:13:59.598 INFO:teuthology.orchestra.run.smithi049.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-06T02:13:59.598 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 02:42:47:49:33:69 brd ff:ff:ff:ff:ff:ff 2023-11-06T02:13:59.599 INFO:teuthology.orchestra.run.smithi049.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-06T02:13:59.599 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:13:59.599 INFO:teuthology.orchestra.run.smithi049.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-06T02:13:59.599 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-11-06T02:13:59.599 INFO:teuthology.orchestra.run.smithi049.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-06T02:13:59.600 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:13:59.600 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-11-06T02:13:59.600 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:13:59.600 INFO:teuthology.orchestra.run.smithi049.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-06T02:13:59.600 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-06T02:13:59.600 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-11-06T02:13:59.601 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:13:59.601 INFO:teuthology.orchestra.run.smithi049.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-06T02:13:59.601 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-06T02:13:59.601 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-11-06T02:13:59.601 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:13:59.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:13:59.602 DEBUG:teuthology.orchestra.run.smithi049:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-06T02:13:59.646 INFO:teuthology.orchestra.run.smithi049.stdout:1 2023-11-06T02:13:59.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:13:59.648 DEBUG:teuthology.orchestra.run.smithi049:> route 2023-11-06T02:13:59.697 INFO:teuthology.orchestra.run.smithi049.stdout:Kernel IP routing table 2023-11-06T02:13:59.697 INFO:teuthology.orchestra.run.smithi049.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-06T02:13:59.698 INFO:teuthology.orchestra.run.smithi049.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-11-06T02:13:59.698 INFO:teuthology.orchestra.run.smithi049.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-06T02:13:59.698 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-11-06T02:13:59.698 INFO:teuthology.orchestra.run.smithi049.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-06T02:13:59.699 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:13:59.699 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec bash -c ' 2023-11-06T02:13:59.699 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2023-11-06T02:13:59.700 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-06T02:13:59.700 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-06T02:13:59.700 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-06T02:13:59.701 DEBUG:teuthology.orchestra.run.smithi049:> ') 2023-11-06T02:13:59.771 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:13:59.772 DEBUG:teuthology.orchestra.run.smithi049:> ip netns list 2023-11-06T02:13:59.820 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-06T02:13:59.820 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-06T02:13:59.820 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-06T02:13:59.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:13:59.821 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link set brx.0 up 2023-11-06T02:13:59.876 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-06T02:13:59.876 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-06T02:13:59.877 DEBUG:teuthology.orchestra.run.smithi049:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-06T02:13:59.924 INFO:teuthology.orchestra.run.smithi049.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-06T02:13:59.924 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-06T02:13:59.925 DEBUG:teuthology.orchestra.run.smithi049:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-06T02:13:59.972 DEBUG:teuthology.orchestra.run.smithi049:> sudo modprobe fuse 2023-11-06T02:14:00.029 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-06T02:14:00.030 DEBUG:teuthology.orchestra.run.smithi049:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-06T02:14:00.083 INFO:teuthology.orchestra.run.smithi049.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-06T02:14:00.084 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-06T02:14:00.085 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:14:00.085 DEBUG:teuthology.orchestra.run.smithi049:> ls /sys/fs/fuse/connections 2023-11-06T02:14:00.132 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-06T02:14:00.133 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-06T02:14:00.177 DEBUG:teuthology.orchestra.run.smithi049:> sudo modprobe fuse 2023-11-06T02:14:00.193 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-06T02:14:00.193 DEBUG:teuthology.orchestra.run.smithi049:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-06T02:14:00.247 INFO:teuthology.orchestra.run.smithi049.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-06T02:14:00.248 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-06T02:14:00.249 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:14:00.249 DEBUG:teuthology.orchestra.run.smithi049:> ls /sys/fs/fuse/connections 2023-11-06T02:14:00.259 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi049.stderr:2023-11-06T02:14:00.262+0000 7f9c82274580 -1 init, newargv = 0x5590115acfd0 newargc=15 2023-11-06T02:14:00.259 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi049.stderr:ceph-fuse[55460]: starting ceph client 2023-11-06T02:14:00.265 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi049.stderr:ceph-fuse[55460]: starting fuse 2023-11-06T02:14:01.365 DEBUG:teuthology.orchestra.run.smithi049:> sudo modprobe fuse 2023-11-06T02:14:01.382 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-06T02:14:01.382 DEBUG:teuthology.orchestra.run.smithi049:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-06T02:14:01.439 INFO:teuthology.orchestra.run.smithi049.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-06T02:14:01.440 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-06T02:14:01.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:14:01.441 DEBUG:teuthology.orchestra.run.smithi049:> ls /sys/fs/fuse/connections 2023-11-06T02:14:01.488 INFO:teuthology.orchestra.run.smithi049.stdout:45 2023-11-06T02:14:01.489 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2023-11-06T02:14:01.489 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-11-06T02:14:01.490 DEBUG:teuthology.orchestra.run.smithi049:> import glob 2023-11-06T02:14:01.490 DEBUG:teuthology.orchestra.run.smithi049:> import re 2023-11-06T02:14:01.490 DEBUG:teuthology.orchestra.run.smithi049:> import os 2023-11-06T02:14:01.491 DEBUG:teuthology.orchestra.run.smithi049:> import subprocess 2023-11-06T02:14:01.491 DEBUG:teuthology.orchestra.run.smithi049:> 2023-11-06T02:14:01.491 DEBUG:teuthology.orchestra.run.smithi049:> def _find_admin_socket(client_name): 2023-11-06T02:14:01.492 DEBUG:teuthology.orchestra.run.smithi049:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-06T02:14:01.492 DEBUG:teuthology.orchestra.run.smithi049:> files = glob.glob(asok_path) 2023-11-06T02:14:01.492 DEBUG:teuthology.orchestra.run.smithi049:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-06T02:14:01.493 DEBUG:teuthology.orchestra.run.smithi049:> 2023-11-06T02:14:01.493 DEBUG:teuthology.orchestra.run.smithi049:> # Given a non-glob path, it better be there 2023-11-06T02:14:01.494 DEBUG:teuthology.orchestra.run.smithi049:> if "*" not in asok_path: 2023-11-06T02:14:01.494 DEBUG:teuthology.orchestra.run.smithi049:> assert(len(files) == 1) 2023-11-06T02:14:01.494 DEBUG:teuthology.orchestra.run.smithi049:> return files[0] 2023-11-06T02:14:01.495 DEBUG:teuthology.orchestra.run.smithi049:> 2023-11-06T02:14:01.495 DEBUG:teuthology.orchestra.run.smithi049:> for f in files: 2023-11-06T02:14:01.495 DEBUG:teuthology.orchestra.run.smithi049:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-06T02:14:01.496 DEBUG:teuthology.orchestra.run.smithi049:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-06T02:14:01.496 DEBUG:teuthology.orchestra.run.smithi049:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-06T02:14:01.496 DEBUG:teuthology.orchestra.run.smithi049:> contents = proc_f.read() 2023-11-06T02:14:01.497 DEBUG:teuthology.orchestra.run.smithi049:> if mountpoint in contents: 2023-11-06T02:14:01.497 DEBUG:teuthology.orchestra.run.smithi049:> return f 2023-11-06T02:14:01.498 DEBUG:teuthology.orchestra.run.smithi049:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-06T02:14:01.498 DEBUG:teuthology.orchestra.run.smithi049:> 2023-11-06T02:14:01.498 DEBUG:teuthology.orchestra.run.smithi049:> print(_find_admin_socket("client.0")) 2023-11-06T02:14:01.499 DEBUG:teuthology.orchestra.run.smithi049:> ' 2023-11-06T02:14:01.613 INFO:teuthology.orchestra.run.smithi049.stdout:/var/run/ceph/ceph-client.0.55460.asok 2023-11-06T02:14:01.794 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.55460.asok 2023-11-06T02:14:01.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:14:01.795 DEBUG:teuthology.orchestra.run.smithi049:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.55460.asok status 2023-11-06T02:14:01.952 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-11-06T02:14:01.952 INFO:teuthology.orchestra.run.smithi049.stdout: "metadata": { 2023-11-06T02:14:01.952 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_sha1": "f7e9f9af51acf98e0ccc3037487018e4ae40caa5", 2023-11-06T02:14:01.953 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_version": "ceph version 18.2.0-1181-gf7e9f9af (f7e9f9af51acf98e0ccc3037487018e4ae40caa5) reef (stable)", 2023-11-06T02:14:01.953 INFO:teuthology.orchestra.run.smithi049.stdout: "entity_id": "0", 2023-11-06T02:14:01.953 INFO:teuthology.orchestra.run.smithi049.stdout: "hostname": "smithi049", 2023-11-06T02:14:01.953 INFO:teuthology.orchestra.run.smithi049.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-06T02:14:01.953 INFO:teuthology.orchestra.run.smithi049.stdout: "pid": "55460", 2023-11-06T02:14:01.954 INFO:teuthology.orchestra.run.smithi049.stdout: "root": "/" 2023-11-06T02:14:01.954 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:14:01.954 INFO:teuthology.orchestra.run.smithi049.stdout: "dentry_count": 0, 2023-11-06T02:14:01.954 INFO:teuthology.orchestra.run.smithi049.stdout: "dentry_pinned_count": 0, 2023-11-06T02:14:01.954 INFO:teuthology.orchestra.run.smithi049.stdout: "id": 7434, 2023-11-06T02:14:01.955 INFO:teuthology.orchestra.run.smithi049.stdout: "inst": { 2023-11-06T02:14:01.955 INFO:teuthology.orchestra.run.smithi049.stdout: "name": { 2023-11-06T02:14:01.955 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "client", 2023-11-06T02:14:01.955 INFO:teuthology.orchestra.run.smithi049.stdout: "num": 7434 2023-11-06T02:14:01.955 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:14:01.956 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": { 2023-11-06T02:14:01.956 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2023-11-06T02:14:01.956 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "192.168.0.1:0", 2023-11-06T02:14:01.956 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 2478798501 2023-11-06T02:14:01.956 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:14:01.956 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:14:01.957 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": { 2023-11-06T02:14:01.957 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2023-11-06T02:14:01.957 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "192.168.0.1:0", 2023-11-06T02:14:01.957 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 2478798501 2023-11-06T02:14:01.957 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:14:01.958 INFO:teuthology.orchestra.run.smithi049.stdout: "inst_str": "client.7434 192.168.0.1:0/2478798501", 2023-11-06T02:14:01.958 INFO:teuthology.orchestra.run.smithi049.stdout: "addr_str": "192.168.0.1:0/2478798501", 2023-11-06T02:14:01.958 INFO:teuthology.orchestra.run.smithi049.stdout: "inode_count": 1, 2023-11-06T02:14:01.958 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_epoch": 46, 2023-11-06T02:14:01.958 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_epoch": 80, 2023-11-06T02:14:01.959 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_epoch_barrier": 0, 2023-11-06T02:14:01.959 INFO:teuthology.orchestra.run.smithi049.stdout: "blocklisted": false, 2023-11-06T02:14:01.959 INFO:teuthology.orchestra.run.smithi049.stdout: "fs_name": "cephfs" 2023-11-06T02:14:01.959 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-11-06T02:14:01.965 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:14:01.966 DEBUG:teuthology.orchestra.run.smithi049:> stat --file-system '--printf=%T 2023-11-06T02:14:01.966 DEBUG:teuthology.orchestra.run.smithi049:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-06T02:14:01.973 INFO:teuthology.orchestra.run.smithi049.stdout:fuseblk 2023-11-06T02:14:01.973 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-06T02:14:01.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:14:01.974 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-06T02:14:02.027 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:14:02.027 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent/unflushed) 2023-11-06T02:14:02.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:14:02.083 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=./parent/unflushed/jfile bs=1M count=8) 2023-11-06T02:14:02.197 INFO:teuthology.orchestra.run.smithi049.stderr:8+0 records in 2023-11-06T02:14:02.198 INFO:teuthology.orchestra.run.smithi049.stderr:8+0 records out 2023-11-06T02:14:02.198 INFO:teuthology.orchestra.run.smithi049.stderr:8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.0651295 s, 129 MB/s 2023-11-06T02:14:02.199 DEBUG:teuthology.orchestra.run.smithi049:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-06T02:14:02.199 DEBUG:teuthology.orchestra.run.smithi049:> import os 2023-11-06T02:14:02.200 DEBUG:teuthology.orchestra.run.smithi049:> import stat 2023-11-06T02:14:02.200 DEBUG:teuthology.orchestra.run.smithi049:> 2023-11-06T02:14:02.200 DEBUG:teuthology.orchestra.run.smithi049:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2023-11-06T02:14:02.201 DEBUG:teuthology.orchestra.run.smithi049:> ' 2023-11-06T02:14:02.271 INFO:teuthology.orchestra.run.smithi049.stdout:1099511627779 2023-11-06T02:14:02.459 DEBUG:teuthology.orchestra.run.smithi049:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-06T02:14:02.460 DEBUG:teuthology.orchestra.run.smithi049:> import os 2023-11-06T02:14:02.460 DEBUG:teuthology.orchestra.run.smithi049:> import stat 2023-11-06T02:14:02.460 DEBUG:teuthology.orchestra.run.smithi049:> 2023-11-06T02:14:02.461 DEBUG:teuthology.orchestra.run.smithi049:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2023-11-06T02:14:02.461 DEBUG:teuthology.orchestra.run.smithi049:> ' 2023-11-06T02:14:02.575 INFO:teuthology.orchestra.run.smithi049.stdout:1099511627780 2023-11-06T02:14:02.761 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-11-06T02:14:02.762 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T02:14:02.809 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T02:14:02.983 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:02.986+0000 7f267d67d640 1 Processor -- start 2023-11-06T02:14:02.983 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:02.986+0000 7f267d67d640 1 -- start start 2023-11-06T02:14:02.985 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:02.990+0000 7f267d67d640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f26780b0790 0x7f26780c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:02.985 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:02.990+0000 7f267d67d640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f26780cb010 0x7f26780c2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:02.985 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:02.990+0000 7f267d67d640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26780cab80 con 0x7f267815ab50 2023-11-06T02:14:02.985 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:02.990+0000 7f267d67d640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f267815af30 con 0x7f26780cb010 2023-11-06T02:14:02.986 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:02.990+0000 7f267d67d640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26780cb5b0 con 0x7f26780b0790 2023-11-06T02:14:02.986 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:02.990+0000 7f26777fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f26780cb010 0x7f26780c2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:02.986 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:02.990+0000 7f26777fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f26780cb010 0x7f26780c2710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51516/0 (socket says 172.21.15.49:51516) 2023-11-06T02:14:02.986 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:02.990+0000 7f26777fe640 1 -- 172.21.15.49:0/2022572351 learned_addr learned my addr 172.21.15.49:0/2022572351 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:02.986 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:02.990+0000 7f26767fc640 1 --2- 172.21.15.49:0/2022572351 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f26780b0790 0x7f26780c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:02.987 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:02.990+0000 7f26777fe640 1 -- 172.21.15.49:0/2022572351 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f26780b0790 msgr2=0x7f26780c9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:02.987 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:02.990+0000 7f26777fe640 1 --2- 172.21.15.49:0/2022572351 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f26780b0790 0x7f26780c9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:02.987 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:02.990+0000 7f26777fe640 1 -- 172.21.15.49:0/2022572351 >> v1:172.21.15.49:6789/0 conn(0x7f267815ab50 legacy=0x7f26780b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:02.987 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:02.990+0000 7f26777fe640 1 -- 172.21.15.49:0/2022572351 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26780cb890 con 0x7f26780cb010 2023-11-06T02:14:02.987 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:02.990+0000 7f2675ffb640 1 -- 172.21.15.49:0/2022572351 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1641593810 0 0) 0x7f26780cab80 con 0x7f267815ab50 2023-11-06T02:14:02.989 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:02.994+0000 7f26777fe640 1 --2- 172.21.15.49:0/2022572351 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26780cb010 0x7f26780c2710 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f2670009870 tx=0x7f267001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ae1e6be9e612fe9 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:02.989 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:02.994+0000 7f2675ffb640 1 -- 172.21.15.49:0/2022572351 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26700ab680 con 0x7f26780cb010 2023-11-06T02:14:02.989 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:02.994+0000 7f2675ffb640 1 -- 172.21.15.49:0/2022572351 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26700ab7e0 con 0x7f26780cb010 2023-11-06T02:14:02.989 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:02.994+0000 7f2675ffb640 1 -- 172.21.15.49:0/2022572351 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26700abb70 con 0x7f26780cb010 2023-11-06T02:14:02.990 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:02.994+0000 7f267d67d640 1 -- 172.21.15.49:0/2022572351 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26780cb010 msgr2=0x7f26780c2710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:02.990 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:02.994+0000 7f267d67d640 1 --2- 172.21.15.49:0/2022572351 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26780cb010 0x7f26780c2710 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f2670009870 tx=0x7f267001c480 comp rx=0 tx=0).stop 2023-11-06T02:14:02.991 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:02.994+0000 7f267d67d640 1 -- 172.21.15.49:0/2022572351 shutdown_connections 2023-11-06T02:14:02.991 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:02.994+0000 7f267d67d640 1 --2- 172.21.15.49:0/2022572351 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26780cb010 0x7f26780c2710 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:02.991 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:02.994+0000 7f267d67d640 1 --2- 172.21.15.49:0/2022572351 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f26780b0790 0x7f26780c9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:02.991 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:02.994+0000 7f267d67d640 1 -- 172.21.15.49:0/2022572351 >> 172.21.15.49:0/2022572351 conn(0x7f267806c0a0 msgr2=0x7f267806c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:02.991 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:02.994+0000 7f267d67d640 1 -- 172.21.15.49:0/2022572351 shutdown_connections 2023-11-06T02:14:02.992 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:02.994+0000 7f267d67d640 1 -- 172.21.15.49:0/2022572351 wait complete. 2023-11-06T02:14:02.994 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:02.994+0000 7f267d67d640 1 Processor -- start 2023-11-06T02:14:02.994 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:02.994+0000 7f267d67d640 1 -- start start 2023-11-06T02:14:02.994 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:02.998+0000 7f267d67d640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f26780b0790 0x7f26780c0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:02.994 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:02.998+0000 7f267d67d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26780cb010 0x7f26780c5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:02.995 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:02.998+0000 7f267d67d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f267815ab50 0x7f2678153330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:02.995 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:02.998+0000 7f267d67d640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f267814bb60 con 0x7f267815ab50 2023-11-06T02:14:02.995 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:02.998+0000 7f267d67d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2678071a50 con 0x7f26780cb010 2023-11-06T02:14:02.995 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:02.998+0000 7f267d67d640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f267814ac70 con 0x7f26780b0790 2023-11-06T02:14:02.995 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:02.998+0000 7f26777fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f267815ab50 0x7f2678153330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:02.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:02.998+0000 7f26777fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f267815ab50 0x7f2678153330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51520/0 (socket says 172.21.15.49:51520) 2023-11-06T02:14:02.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:02.998+0000 7f26777fe640 1 -- 172.21.15.49:0/2924113225 learned_addr learned my addr 172.21.15.49:0/2924113225 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:02.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:02.998+0000 7f2676ffd640 1 --2- 172.21.15.49:0/2924113225 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f26780b0790 0x7f26780c0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:02.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:02.998+0000 7f26767fc640 1 --2- 172.21.15.49:0/2924113225 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26780cb010 0x7f26780c5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:02.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:02.998+0000 7f26777fe640 1 -- 172.21.15.49:0/2924113225 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f26780b0790 msgr2=0x7f26780c0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:02.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:02.998+0000 7f26777fe640 1 --2- 172.21.15.49:0/2924113225 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f26780b0790 0x7f26780c0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:02.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:02.998+0000 7f26777fe640 1 -- 172.21.15.49:0/2924113225 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26780cb010 msgr2=0x7f26780c5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:02.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:02.998+0000 7f26777fe640 1 --2- 172.21.15.49:0/2924113225 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26780cb010 0x7f26780c5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:02.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:02.998+0000 7f26777fe640 1 -- 172.21.15.49:0/2924113225 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26700094d0 con 0x7f267815ab50 2023-11-06T02:14:02.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:02.998+0000 7f26777fe640 1 --2- 172.21.15.49:0/2924113225 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f267815ab50 0x7f2678153330 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f267001a0b0 tx=0x7f267001acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:02.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:02.998+0000 7f266f7fe640 1 -- 172.21.15.49:0/2924113225 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26700abe20 con 0x7f267815ab50 2023-11-06T02:14:02.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:02.998+0000 7f266f7fe640 1 -- 172.21.15.49:0/2924113225 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26700b8070 con 0x7f267815ab50 2023-11-06T02:14:02.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:02.998+0000 7f267d67d640 1 -- 172.21.15.49:0/2924113225 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f267814af50 con 0x7f267815ab50 2023-11-06T02:14:02.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:02.998+0000 7f266f7fe640 1 -- 172.21.15.49:0/2924113225 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2670004250 con 0x7f267815ab50 2023-11-06T02:14:02.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:02.998+0000 7f267d67d640 1 -- 172.21.15.49:0/2924113225 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f267814b510 con 0x7f267815ab50 2023-11-06T02:14:02.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:02.998+0000 7f267d67d640 1 -- 172.21.15.49:0/2924113225 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26780b0de0 con 0x7f267815ab50 2023-11-06T02:14:02.999 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:02.998+0000 7f266f7fe640 1 -- 172.21.15.49:0/2924113225 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f26700043b0 con 0x7f267815ab50 2023-11-06T02:14:03.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:02.998+0000 7f266f7fe640 1 --2- 172.21.15.49:0/2924113225 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2658065a90 0x7f2658067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:03.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:02.998+0000 7f266f7fe640 1 -- 172.21.15.49:0/2924113225 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f267011dab0 con 0x7f267815ab50 2023-11-06T02:14:03.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:02.998+0000 7f2676ffd640 1 --2- 172.21.15.49:0/2924113225 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2658065a90 0x7f2658067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:03.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.002+0000 7f2676ffd640 1 --2- 172.21.15.49:0/2924113225 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2658065a90 0x7f2658067f50 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f2668000fe0 tx=0x7f2668009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:03.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.002+0000 7f266f7fe640 1 -- 172.21.15.49:0/2924113225 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f26700eb300 con 0x7f267815ab50 2023-11-06T02:14:03.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.158+0000 7f267d67d640 1 -- 172.21.15.49:0/2924113225 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2678074040 con 0x7f267815ab50 2023-11-06T02:14:03.157 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.162+0000 7f266f7fe640 1 -- 172.21.15.49:0/2924113225 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+17886 (secure 0 0 0) 0x7f26700ba090 con 0x7f267815ab50 2023-11-06T02:14:03.158 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:03.158 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":80,"fsid":"1a8b035c-6ba4-499a-b876-aea47fd9c0d4","created":"2023-11-06T02:08:06.440857+0000","modified":"2023-11-06T02:13:34.630031+0000","last_up_change":"2023-11-06T02:08:17.982372+0000","last_in_change":"2023-11-06T02:08:08.906631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T02:08:19.252222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T02:08:24.776211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-11-06T02:13:31.459314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-11-06T02:13:32.603710+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ad55af8-0571-4e6a-8722-b5d429b7496d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6809","nonce":38116},{"type":"v1","addr":"172.21.15.49:6811","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6813","nonce":38116},{"type":"v1","addr":"172.21.15.49:6815","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6821","nonce":38116},{"type":"v1","addr":"172.21.15.49:6823","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6817","nonce":38116},{"type":"v1","addr":"172.21.15.49:6819","nonce":38116}]},"public_addr":"172.21.15.49:6811/38116","cluster_addr":"172.21.15.49:6815/38116","heartbeat_back_addr":"172.21.15.49:6823/38116","heartbeat_front_addr":"172.21.15.49:6819/38116","state":["exists","up"]},{"osd":1,"uuid":"c8b31f85-a757-43ed-9419-351acc46611b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":38113},{"type":"v1","addr":"172.21.15.49:6810","nonce":38113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":38113},{"type":"v1","addr":"172.21.15.49:6814","nonce":38113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":38113},{"type":"v1","addr":"172.21.15.49:6822","nonce":38113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":38113},{"type":"v1","addr":"172.21.15.49:6818","nonce":38113}]},"public_addr":"172.21.15.49:6810/38113","cluster_addr":"172.21.15.49:6814/38113","heartbeat_back_addr":"172.21.15.49:6822/38113","heartbeat_front_addr":"172.21.15.49:6818/38113","state":["exists","up"]},{"osd":2,"uuid":"555b3bf3-0a70-4b91-8efd-e4204b5fc26e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":38114},{"type":"v1","addr":"172.21.15.49:6801","nonce":38114}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":38114},{"type":"v1","addr":"172.21.15.49:6803","nonce":38114}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":38114},{"type":"v1","addr":"172.21.15.49:6807","nonce":38114}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":38114},{"type":"v1","addr":"172.21.15.49:6805","nonce":38114}]},"public_addr":"172.21.15.49:6801/38114","cluster_addr":"172.21.15.49:6803/38114","heartbeat_back_addr":"172.21.15.49:6807/38114","heartbeat_front_addr":"172.21.15.49:6805/38114","state":["exists","up"]},{"osd":3,"uuid":"8d4e5d77-03ef-4b9b-bfcb-5ae93a46795c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":38115},{"type":"v1","addr":"172.21.15.49:6825","nonce":38115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":38115},{"type":"v1","addr":"172.21.15.49:6827","nonce":38115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":38115},{"type":"v1","addr":"172.21.15.49:6831","nonce":38115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":38115},{"type":"v1","addr":"172.21.15.49:6829","nonce":38115}]},"public_addr":"172.21.15.49:6825/38115","cluster_addr":"172.21.15.49:6827/38115","heartbeat_back_addr":"172.21.15.49:6831/38115","heartbeat_front_addr":"172.21.15.49:6829/38115","state":["exists","up"]},{"osd":4,"uuid":"dba03ad4-68a3-4cf3-9218-ac611f02eacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":38417},{"type":"v1","addr":"172.21.15.150:6809","nonce":38417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":38417},{"type":"v1","addr":"172.21.15.150:6811","nonce":38417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":38417},{"type":"v1","addr":"172.21.15.150:6815","nonce":38417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":38417},{"type":"v1","addr":"172.21.15.150:6813","nonce":38417}]},"public_addr":"172.21.15.150:6809/38417","cluster_addr":"172.21.15.150:6811/38417","heartbeat_back_addr":"172.21.15.150:6815/38417","heartbeat_front_addr":"172.21.15.150:6813/38417","state":["exists","up"]},{"osd":5,"uuid":"0df9c303-c255-4640-8e91-01f1c3efe18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":38418},{"type":"v1","addr":"172.21.15.150:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":38418},{"type":"v1","addr":"172.21.15.150:6803","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":38418},{"type":"v1","addr":"172.21.15.150:6807","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":38418},{"type":"v1","addr":"172.21.15.150:6805","nonce":38418}]},"public_addr":"172.21.15.150:6801/38418","cluster_addr":"172.21.15.150:6803/38418","heartbeat_back_addr":"172.21.15.150:6807/38418","heartbeat_front_addr":"172.21.15.150:6805/38418","state":["exists","up"]},{"osd":6,"uuid":"e59fa189-4b70-4c06-9722-c02b90828bf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":38428},{"type":"v1","addr":"172.21.15.150:6825","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":38428},{"type":"v1","addr":"172.21.15.150:6827","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":38428},{"type":"v1","addr":"172.21.15.150:6831","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":38428},{"type":"v1","addr":"172.21.15.150:6829","nonce":38428}]},"public_addr":"172.21.15.150:6825/38428","cluster_addr":"172.21.15.150:6827/38428","heartbeat_back_addr":"172.21.15.150:6831/38428","heartbeat_front_addr":"172.21.15.150:6829/38428","state":["exists","up"]},{"osd":7,"uuid":"a64f57e1-ef98-46bd-b304-385759e95898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":38421},{"type":"v1","addr":"172.21.15.150:6817","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":38421},{"type":"v1","addr":"172.21.15.150:6819","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":38421},{"type":"v1","addr":"172.21.15.150:6823","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":38421},{"type":"v1","addr":"172.21.15.150:6821","nonce":38421}]},"public_addr":"172.21.15.150:6817/38421","cluster_addr":"172.21.15.150:6819/38421","heartbeat_back_addr":"172.21.15.150:6823/38421","heartbeat_front_addr":"172.21.15.150:6821/38421","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.438552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.413280+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.296466+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.424966+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.276953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.230984+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.410235+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.480084+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T02:14:03.160 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.166+0000 7f267d67d640 1 -- 172.21.15.49:0/2924113225 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2658065a90 msgr2=0x7f2658067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:03.160 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.166+0000 7f267d67d640 1 --2- 172.21.15.49:0/2924113225 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2658065a90 0x7f2658067f50 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f2668000fe0 tx=0x7f2668009040 comp rx=0 tx=0).stop 2023-11-06T02:14:03.160 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.166+0000 7f267d67d640 1 -- 172.21.15.49:0/2924113225 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f267815ab50 msgr2=0x7f2678153330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:03.161 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.166+0000 7f267d67d640 1 --2- 172.21.15.49:0/2924113225 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f267815ab50 0x7f2678153330 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f267001a0b0 tx=0x7f267001acc0 comp rx=0 tx=0).stop 2023-11-06T02:14:03.161 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.166+0000 7f267d67d640 1 -- 172.21.15.49:0/2924113225 shutdown_connections 2023-11-06T02:14:03.161 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.166+0000 7f267d67d640 1 --2- 172.21.15.49:0/2924113225 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2658065a90 0x7f2658067f50 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:03.161 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.166+0000 7f267d67d640 1 --2- 172.21.15.49:0/2924113225 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f267815ab50 0x7f2678153330 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:03.161 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.166+0000 7f267d67d640 1 --2- 172.21.15.49:0/2924113225 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26780cb010 0x7f26780c5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:03.161 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.166+0000 7f267d67d640 1 --2- 172.21.15.49:0/2924113225 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f26780b0790 0x7f26780c0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:03.162 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.166+0000 7f267d67d640 1 -- 172.21.15.49:0/2924113225 >> 172.21.15.49:0/2924113225 conn(0x7f267806c0a0 msgr2=0x7f2678156540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:03.162 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.166+0000 7f267d67d640 1 -- 172.21.15.49:0/2924113225 shutdown_connections 2023-11-06T02:14:03.162 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.166+0000 7f267d67d640 1 -- 172.21.15.49:0/2924113225 wait complete. 2023-11-06T02:14:03.177 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T02:08:19.252222+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T02:08:24.776211+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T02:13:31.459314+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T02:13:32.603710+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T02:14:03.179 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T02:14:03.311 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.314+0000 7f04e5800640 1 Processor -- start 2023-11-06T02:14:03.312 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.318+0000 7f04e5800640 1 -- start start 2023-11-06T02:14:03.313 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.318+0000 7f04e5800640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f04e015cbb0 0x7f04e00d1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:03.313 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.318+0000 7f04e5800640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f04e00d24e0 0x7f04e00d2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:03.314 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.318+0000 7f04e5800640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04e00d3400 con 0x7f04e00d2fa0 2023-11-06T02:14:03.314 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.318+0000 7f04e5800640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04e015cf90 con 0x7f04e00d24e0 2023-11-06T02:14:03.314 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.318+0000 7f04e5800640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04e00d0b80 con 0x7f04e015cbb0 2023-11-06T02:14:03.314 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.318+0000 7f04de7fc640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f04e00d24e0 0x7f04e00d2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:03.315 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.318+0000 7f04de7fc640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f04e00d24e0 0x7f04e00d2a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51536/0 (socket says 172.21.15.49:51536) 2023-11-06T02:14:03.315 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.318+0000 7f04de7fc640 1 -- 172.21.15.49:0/3798304038 learned_addr learned my addr 172.21.15.49:0/3798304038 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:03.315 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.318+0000 7f04deffd640 1 --2- 172.21.15.49:0/3798304038 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f04e015cbb0 0x7f04e00d1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:03.315 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.318+0000 7f04de7fc640 1 -- 172.21.15.49:0/3798304038 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f04e015cbb0 msgr2=0x7f04e00d1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:03.315 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.318+0000 7f04de7fc640 1 --2- 172.21.15.49:0/3798304038 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f04e015cbb0 0x7f04e00d1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:03.316 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.318+0000 7f04de7fc640 1 -- 172.21.15.49:0/3798304038 >> v1:172.21.15.150:6789/0 conn(0x7f04e00d2fa0 legacy=0x7f04e00b4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:03.316 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.318+0000 7f04de7fc640 1 -- 172.21.15.49:0/3798304038 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04e00d0e60 con 0x7f04e00d24e0 2023-11-06T02:14:03.316 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.318+0000 7f04deffd640 1 --2- 172.21.15.49:0/3798304038 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f04e015cbb0 0x7f04e00d1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:14:03.317 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.318+0000 7f04de7fc640 1 --2- 172.21.15.49:0/3798304038 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f04e00d24e0 0x7f04e00d2a60 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f04e00b55d0 tx=0x7f04d401ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d21f7c0cd997eac8 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:03.318 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.322+0000 7f04ddffb640 1 -- 172.21.15.49:0/3798304038 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d40a8970 con 0x7f04e00d24e0 2023-11-06T02:14:03.318 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.322+0000 7f04ddffb640 1 -- 172.21.15.49:0/3798304038 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04d40a8ad0 con 0x7f04e00d24e0 2023-11-06T02:14:03.318 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.322+0000 7f04ddffb640 1 -- 172.21.15.49:0/3798304038 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d40a8e60 con 0x7f04e00d24e0 2023-11-06T02:14:03.318 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.322+0000 7f04e5800640 1 -- 172.21.15.49:0/3798304038 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f04e00d24e0 msgr2=0x7f04e00d2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:03.319 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.322+0000 7f04e5800640 1 --2- 172.21.15.49:0/3798304038 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f04e00d24e0 0x7f04e00d2a60 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f04e00b55d0 tx=0x7f04d401ecb0 comp rx=0 tx=0).stop 2023-11-06T02:14:03.319 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.322+0000 7f04e5800640 1 -- 172.21.15.49:0/3798304038 shutdown_connections 2023-11-06T02:14:03.319 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.322+0000 7f04e5800640 1 --2- 172.21.15.49:0/3798304038 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f04e00d24e0 0x7f04e00d2a60 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:03.319 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.322+0000 7f04e5800640 1 --2- 172.21.15.49:0/3798304038 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f04e015cbb0 0x7f04e00d1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:03.319 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.322+0000 7f04e5800640 1 -- 172.21.15.49:0/3798304038 >> 172.21.15.49:0/3798304038 conn(0x7f04e00718a0 msgr2=0x7f04e0071cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:03.320 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.322+0000 7f04e5800640 1 -- 172.21.15.49:0/3798304038 shutdown_connections 2023-11-06T02:14:03.320 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.322+0000 7f04e5800640 1 -- 172.21.15.49:0/3798304038 wait complete. 2023-11-06T02:14:03.320 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.322+0000 7f04e5800640 1 Processor -- start 2023-11-06T02:14:03.320 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.322+0000 7f04e5800640 1 -- start start 2023-11-06T02:14:03.324 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.322+0000 7f04e5800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f04e00d24e0 0x7f04e01f3500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:03.324 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.322+0000 7f04e5800640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f04e00d2fa0 0x7f04e014ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:03.324 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.322+0000 7f04e5800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f04e015cbb0 0x7f04e00acc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:03.324 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.322+0000 7f04e5800640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04e01f3a40 con 0x7f04e00d24e0 2023-11-06T02:14:03.325 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.322+0000 7f04e5800640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04e00a9d30 con 0x7f04e015cbb0 2023-11-06T02:14:03.325 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.322+0000 7f04e5800640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04e014f460 con 0x7f04e00d2fa0 2023-11-06T02:14:03.325 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.322+0000 7f04deffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f04e00d24e0 0x7f04e01f3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:03.325 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.322+0000 7f04deffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f04e00d24e0 0x7f04e01f3500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51552/0 (socket says 172.21.15.49:51552) 2023-11-06T02:14:03.325 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.322+0000 7f04deffd640 1 -- 172.21.15.49:0/1401171294 learned_addr learned my addr 172.21.15.49:0/1401171294 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:03.326 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.322+0000 7f04de7fc640 1 --2- 172.21.15.49:0/1401171294 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f04e00d2fa0 0x7f04e014ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:03.326 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.322+0000 7f04df7fe640 1 --2- 172.21.15.49:0/1401171294 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f04e015cbb0 0x7f04e00acc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:03.326 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.322+0000 7f04deffd640 1 -- 172.21.15.49:0/1401171294 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f04e00d2fa0 msgr2=0x7f04e014ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:03.326 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.322+0000 7f04deffd640 1 --2- 172.21.15.49:0/1401171294 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f04e00d2fa0 0x7f04e014ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:03.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.322+0000 7f04deffd640 1 -- 172.21.15.49:0/1401171294 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f04e015cbb0 msgr2=0x7f04e00acc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:03.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.322+0000 7f04deffd640 1 --2- 172.21.15.49:0/1401171294 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f04e015cbb0 0x7f04e00acc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:03.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.322+0000 7f04deffd640 1 -- 172.21.15.49:0/1401171294 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04d40094d0 con 0x7f04e00d24e0 2023-11-06T02:14:03.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.322+0000 7f04deffd640 1 --2- 172.21.15.49:0/1401171294 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f04e00d24e0 0x7f04e01f3500 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f04d000c7f0 tx=0x7f04d000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:03.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.326+0000 7f04c77fe640 1 -- 172.21.15.49:0/1401171294 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d0007820 con 0x7f04e00d24e0 2023-11-06T02:14:03.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.326+0000 7f04e5800640 1 -- 172.21.15.49:0/1401171294 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04e00ad170 con 0x7f04e00d24e0 2023-11-06T02:14:03.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.326+0000 7f04c77fe640 1 -- 172.21.15.49:0/1401171294 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04d0007980 con 0x7f04e00d24e0 2023-11-06T02:14:03.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.326+0000 7f04c77fe640 1 -- 172.21.15.49:0/1401171294 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d0007d30 con 0x7f04e00d24e0 2023-11-06T02:14:03.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.326+0000 7f04e5800640 1 -- 172.21.15.49:0/1401171294 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04e00aa260 con 0x7f04e00d24e0 2023-11-06T02:14:03.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.326+0000 7f04e5800640 1 -- 172.21.15.49:0/1401171294 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04e0074040 con 0x7f04e00d24e0 2023-11-06T02:14:03.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.326+0000 7f04c77fe640 1 -- 172.21.15.49:0/1401171294 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f04d00113f0 con 0x7f04e00d24e0 2023-11-06T02:14:03.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.326+0000 7f04c77fe640 1 --2- 172.21.15.49:0/1401171294 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f04cc065a60 0x7f04cc067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:03.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.326+0000 7f04c77fe640 1 -- 172.21.15.49:0/1401171294 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f04d0083620 con 0x7f04e00d24e0 2023-11-06T02:14:03.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.326+0000 7f04de7fc640 1 --2- 172.21.15.49:0/1401171294 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f04cc065a60 0x7f04cc067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:03.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.326+0000 7f04de7fc640 1 --2- 172.21.15.49:0/1401171294 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f04cc065a60 0x7f04cc067f20 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f04d40b2000 tx=0x7f04d40aa070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:03.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.330+0000 7f04c77fe640 1 -- 172.21.15.49:0/1401171294 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f04d0050e90 con 0x7f04e00d24e0 2023-11-06T02:14:03.480 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.486+0000 7f04e5800640 1 -- 172.21.15.49:0/1401171294 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f04e015b820 con 0x7f04e00d24e0 2023-11-06T02:14:03.481 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.486+0000 7f04c77fe640 1 -- 172.21.15.49:0/1401171294 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) v1 ==== 93+0+10 (secure 0 0 0) 0x7f04d00544b0 con 0x7f04e00d24e0 2023-11-06T02:14:03.481 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2023-11-06T02:14:03.483 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.486+0000 7f04e5800640 1 -- 172.21.15.49:0/1401171294 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f04cc065a60 msgr2=0x7f04cc067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:03.483 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.486+0000 7f04e5800640 1 --2- 172.21.15.49:0/1401171294 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f04cc065a60 0x7f04cc067f20 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f04d40b2000 tx=0x7f04d40aa070 comp rx=0 tx=0).stop 2023-11-06T02:14:03.483 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.486+0000 7f04e5800640 1 -- 172.21.15.49:0/1401171294 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f04e00d24e0 msgr2=0x7f04e01f3500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:03.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.486+0000 7f04e5800640 1 --2- 172.21.15.49:0/1401171294 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f04e00d24e0 0x7f04e01f3500 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f04d000c7f0 tx=0x7f04d000ccb0 comp rx=0 tx=0).stop 2023-11-06T02:14:03.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.486+0000 7f04e5800640 1 -- 172.21.15.49:0/1401171294 shutdown_connections 2023-11-06T02:14:03.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.486+0000 7f04e5800640 1 --2- 172.21.15.49:0/1401171294 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f04cc065a60 0x7f04cc067f20 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:03.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.486+0000 7f04e5800640 1 --2- 172.21.15.49:0/1401171294 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f04e015cbb0 0x7f04e00acc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:03.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.486+0000 7f04e5800640 1 --2- 172.21.15.49:0/1401171294 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f04e00d2fa0 0x7f04e014ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:03.485 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.486+0000 7f04e5800640 1 --2- 172.21.15.49:0/1401171294 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f04e00d24e0 0x7f04e01f3500 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:03.485 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.486+0000 7f04e5800640 1 -- 172.21.15.49:0/1401171294 >> 172.21.15.49:0/1401171294 conn(0x7f04e00718a0 msgr2=0x7f04e00cbd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:03.485 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.486+0000 7f04e5800640 1 -- 172.21.15.49:0/1401171294 shutdown_connections 2023-11-06T02:14:03.485 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.486+0000 7f04e5800640 1 -- 172.21.15.49:0/1401171294 wait complete. 2023-11-06T02:14:03.501 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T02:14:03.627 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.630+0000 7fa710b2b640 1 Processor -- start 2023-11-06T02:14:03.628 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.630+0000 7fa710b2b640 1 -- start start 2023-11-06T02:14:03.628 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.634+0000 7fa710b2b640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa70c0d2a60 0x7fa70c0d2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:03.628 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.634+0000 7fa710b2b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa70c0b5d40 0x7fa70c0b61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:03.629 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.634+0000 7fa710b2b640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa70c15cf90 con 0x7fa70c15cbb0 2023-11-06T02:14:03.629 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.634+0000 7fa710b2b640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa70c0d3410 con 0x7fa70c0d2a60 2023-11-06T02:14:03.629 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.634+0000 7fa710b2b640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa70c0d0b80 con 0x7fa70c0b5d40 2023-11-06T02:14:03.630 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.634+0000 7fa70a59c640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fa70c15cbb0 0x7fa70c0d1fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:59300/0 (socket says 172.21.15.49:59300) 2023-11-06T02:14:03.630 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.634+0000 7fa70a59c640 1 -- 172.21.15.49:0/4158627057 learned_addr learned my addr 172.21.15.49:0/4158627057 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:14:03.630 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.634+0000 7fa701d9b640 1 --2- 172.21.15.49:0/4158627057 >> v2:172.21.15.49:3300/0 conn(0x7fa70c0d2a60 0x7fa70c0d2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:03.631 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.634+0000 7fa70ad9d640 1 --2- 172.21.15.49:0/4158627057 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa70c0b5d40 0x7fa70c0b61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:03.631 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.634+0000 7fa709d9b640 1 -- 172.21.15.49:0/4158627057 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2538546338 0 0) 0x7fa70c15cf90 con 0x7fa70c15cbb0 2023-11-06T02:14:03.631 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.634+0000 7fa709d9b640 1 -- 172.21.15.49:0/4158627057 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6f8003530 con 0x7fa70c15cbb0 2023-11-06T02:14:03.631 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.634+0000 7fa701d9b640 1 -- 172.21.15.49:0/4158627057 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa70c0b5d40 msgr2=0x7fa70c0b61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:03.631 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.634+0000 7fa701d9b640 1 --2- 172.21.15.49:0/4158627057 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa70c0b5d40 0x7fa70c0b61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:03.631 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.634+0000 7fa701d9b640 1 -- 172.21.15.49:0/4158627057 >> v1:172.21.15.49:6789/0 conn(0x7fa70c15cbb0 legacy=0x7fa70c0d1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:03.632 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.634+0000 7fa701d9b640 1 -- 172.21.15.49:0/4158627057 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa70c0d0e60 con 0x7fa70c0d2a60 2023-11-06T02:14:03.633 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.638+0000 7fa701d9b640 1 --2- 172.21.15.49:0/4158627057 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa70c0d2a60 0x7fa70c0d2ed0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fa6f40097c0 tx=0x7fa6f401ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec98eaa8a44bd308 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:03.633 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.638+0000 7fa709d9b640 1 -- 172.21.15.49:0/4158627057 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f40a8970 con 0x7fa70c0d2a60 2023-11-06T02:14:03.633 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.638+0000 7fa709d9b640 1 -- 172.21.15.49:0/4158627057 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6f40a8ad0 con 0x7fa70c0d2a60 2023-11-06T02:14:03.634 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.638+0000 7fa709d9b640 1 -- 172.21.15.49:0/4158627057 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f40a8e60 con 0x7fa70c0d2a60 2023-11-06T02:14:03.634 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.638+0000 7fa710b2b640 1 -- 172.21.15.49:0/4158627057 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa70c0d2a60 msgr2=0x7fa70c0d2ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:03.634 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.638+0000 7fa710b2b640 1 --2- 172.21.15.49:0/4158627057 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa70c0d2a60 0x7fa70c0d2ed0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fa6f40097c0 tx=0x7fa6f401ec70 comp rx=0 tx=0).stop 2023-11-06T02:14:03.634 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.638+0000 7fa710b2b640 1 -- 172.21.15.49:0/4158627057 shutdown_connections 2023-11-06T02:14:03.634 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.638+0000 7fa710b2b640 1 --2- 172.21.15.49:0/4158627057 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa70c0b5d40 0x7fa70c0b61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:03.635 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.638+0000 7fa710b2b640 1 --2- 172.21.15.49:0/4158627057 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa70c0d2a60 0x7fa70c0d2ed0 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:03.635 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.638+0000 7fa710b2b640 1 -- 172.21.15.49:0/4158627057 >> 172.21.15.49:0/4158627057 conn(0x7fa70c0718a0 msgr2=0x7fa70c071cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:03.635 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.638+0000 7fa710b2b640 1 -- 172.21.15.49:0/4158627057 shutdown_connections 2023-11-06T02:14:03.635 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.638+0000 7fa710b2b640 1 -- 172.21.15.49:0/4158627057 wait complete. 2023-11-06T02:14:03.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.638+0000 7fa710b2b640 1 Processor -- start 2023-11-06T02:14:03.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.638+0000 7fa710b2b640 1 -- start start 2023-11-06T02:14:03.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.638+0000 7fa710b2b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa70c0b5d40 0x7fa70c1508a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:03.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.638+0000 7fa710b2b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa70c0d2a60 0x7fa70c0ae930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:03.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.638+0000 7fa710b2b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa70c15cbb0 0x7fa70c0aee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:03.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.638+0000 7fa710b2b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa70c0aadf0 con 0x7fa70c0d2a60 2023-11-06T02:14:03.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.638+0000 7fa710b2b640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa70c0aab10 con 0x7fa70c0b5d40 2023-11-06T02:14:03.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.638+0000 7fa710b2b640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa70c0ac170 con 0x7fa70c15cbb0 2023-11-06T02:14:03.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.638+0000 7fa701d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa70c0d2a60 0x7fa70c0ae930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:03.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.638+0000 7fa701d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa70c0d2a60 0x7fa70c0ae930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51564/0 (socket says 172.21.15.49:51564) 2023-11-06T02:14:03.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.638+0000 7fa701d9b640 1 -- 172.21.15.49:0/3985770289 learned_addr learned my addr 172.21.15.49:0/3985770289 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:03.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.638+0000 7fa70ad9d640 1 --2- 172.21.15.49:0/3985770289 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa70c15cbb0 0x7fa70c0aee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:03.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.638+0000 7fa70a59c640 1 --2- 172.21.15.49:0/3985770289 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa70c0b5d40 0x7fa70c1508a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:03.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.638+0000 7fa701d9b640 1 -- 172.21.15.49:0/3985770289 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa70c15cbb0 msgr2=0x7fa70c0aee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:03.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.638+0000 7fa701d9b640 1 --2- 172.21.15.49:0/3985770289 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa70c15cbb0 0x7fa70c0aee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:03.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.638+0000 7fa701d9b640 1 -- 172.21.15.49:0/3985770289 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa70c0b5d40 msgr2=0x7fa70c1508a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:03.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.638+0000 7fa701d9b640 1 --2- 172.21.15.49:0/3985770289 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa70c0b5d40 0x7fa70c1508a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:03.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.638+0000 7fa701d9b640 1 -- 172.21.15.49:0/3985770289 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6f4009420 con 0x7fa70c0d2a60 2023-11-06T02:14:03.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.638+0000 7fa70a59c640 1 --2- 172.21.15.49:0/3985770289 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa70c0b5d40 0x7fa70c1508a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:14:03.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.638+0000 7fa701d9b640 1 --2- 172.21.15.49:0/3985770289 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa70c0d2a60 0x7fa70c0ae930 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fa6f40b3040 tx=0x7fa6f401ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:03.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.642+0000 7fa7037fe640 1 -- 172.21.15.49:0/3985770289 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f40b0d60 con 0x7fa70c0d2a60 2023-11-06T02:14:03.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.642+0000 7fa7037fe640 1 -- 172.21.15.49:0/3985770289 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6f40b9430 con 0x7fa70c0d2a60 2023-11-06T02:14:03.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.642+0000 7fa710b2b640 1 -- 172.21.15.49:0/3985770289 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa70c0ac450 con 0x7fa70c0d2a60 2023-11-06T02:14:03.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.642+0000 7fa7037fe640 1 -- 172.21.15.49:0/3985770289 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f40b97e0 con 0x7fa70c0d2a60 2023-11-06T02:14:03.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.642+0000 7fa710b2b640 1 -- 172.21.15.49:0/3985770289 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa70c150de0 con 0x7fa70c0d2a60 2023-11-06T02:14:03.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.642+0000 7fa710b2b640 1 -- 172.21.15.49:0/3985770289 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa70c074040 con 0x7fa70c0d2a60 2023-11-06T02:14:03.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.642+0000 7fa7037fe640 1 -- 172.21.15.49:0/3985770289 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fa6f40b7040 con 0x7fa70c0d2a60 2023-11-06T02:14:03.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.642+0000 7fa7037fe640 1 --2- 172.21.15.49:0/3985770289 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa6ec065a60 0x7fa6ec067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:03.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.642+0000 7fa7037fe640 1 -- 172.21.15.49:0/3985770289 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa6f4120d90 con 0x7fa70c0d2a60 2023-11-06T02:14:03.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.642+0000 7fa70a59c640 1 --2- 172.21.15.49:0/3985770289 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa6ec065a60 0x7fa6ec067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:03.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.642+0000 7fa70a59c640 1 --2- 172.21.15.49:0/3985770289 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa6ec065a60 0x7fa6ec067f20 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc005890 tx=0x7fa6fc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:03.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.646+0000 7fa7037fe640 1 -- 172.21.15.49:0/3985770289 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fa6f40ee680 con 0x7fa70c0d2a60 2023-11-06T02:14:03.792 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.798+0000 7fa710b2b640 1 -- 172.21.15.49:0/3985770289 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa70c0ac900 con 0x7fa70c0d2a60 2023-11-06T02:14:03.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.798+0000 7fa7037fe640 1 -- 172.21.15.49:0/3985770289 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v80) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa6f40b50a0 con 0x7fa70c0d2a60 2023-11-06T02:14:03.793 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 8 2023-11-06T02:14:03.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.798+0000 7fa710b2b640 1 -- 172.21.15.49:0/3985770289 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa6ec065a60 msgr2=0x7fa6ec067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:03.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.798+0000 7fa710b2b640 1 --2- 172.21.15.49:0/3985770289 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa6ec065a60 0x7fa6ec067f20 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc005890 tx=0x7fa6fc009040 comp rx=0 tx=0).stop 2023-11-06T02:14:03.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.798+0000 7fa710b2b640 1 -- 172.21.15.49:0/3985770289 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa70c0d2a60 msgr2=0x7fa70c0ae930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:03.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.798+0000 7fa710b2b640 1 --2- 172.21.15.49:0/3985770289 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa70c0d2a60 0x7fa70c0ae930 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fa6f40b3040 tx=0x7fa6f401ba10 comp rx=0 tx=0).stop 2023-11-06T02:14:03.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.798+0000 7fa710b2b640 1 -- 172.21.15.49:0/3985770289 shutdown_connections 2023-11-06T02:14:03.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.798+0000 7fa710b2b640 1 --2- 172.21.15.49:0/3985770289 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa6ec065a60 0x7fa6ec067f20 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:03.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.798+0000 7fa710b2b640 1 --2- 172.21.15.49:0/3985770289 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa70c0d2a60 0x7fa70c0ae930 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:03.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.798+0000 7fa710b2b640 1 --2- 172.21.15.49:0/3985770289 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa70c15cbb0 0x7fa70c0aee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:03.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.798+0000 7fa710b2b640 1 --2- 172.21.15.49:0/3985770289 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa70c0b5d40 0x7fa70c1508a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:03.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.798+0000 7fa710b2b640 1 -- 172.21.15.49:0/3985770289 >> 172.21.15.49:0/3985770289 conn(0x7fa70c0718a0 msgr2=0x7fa70c0cbbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:03.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.798+0000 7fa710b2b640 1 -- 172.21.15.49:0/3985770289 shutdown_connections 2023-11-06T02:14:03.798 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.802+0000 7fa710b2b640 1 -- 172.21.15.49:0/3985770289 wait complete. 2023-11-06T02:14:03.811 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T02:14:03.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.934+0000 7efed1a68640 1 Processor -- start 2023-11-06T02:14:03.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.934+0000 7efed1a68640 1 -- start start 2023-11-06T02:14:03.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.934+0000 7efed1a68640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7efecc070870 0x7efecc0b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:03.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.934+0000 7efed1a68640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7efecc06d150 0x7efecc06d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:03.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.934+0000 7efed1a68640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efecc06db40 con 0x7efecc15a880 2023-11-06T02:14:03.930 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.934+0000 7efed1a68640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efecc0ae360 con 0x7efecc070870 2023-11-06T02:14:03.930 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.934+0000 7efed1a68640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efecc0ae640 con 0x7efecc06d150 2023-11-06T02:14:03.930 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.934+0000 7efecaffd640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7efecc070870 0x7efecc0b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:03.930 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.934+0000 7efecaffd640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7efecc070870 0x7efecc0b15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51580/0 (socket says 172.21.15.49:51580) 2023-11-06T02:14:03.930 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.934+0000 7efecaffd640 1 -- 172.21.15.49:0/842897946 learned_addr learned my addr 172.21.15.49:0/842897946 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:03.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.934+0000 7efecb7fe640 1 --1- 172.21.15.49:0/842897946 >> v1:172.21.15.49:6789/0 conn(0x7efecc15a880 0x7efecc070160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:59312/0 (socket says 172.21.15.49:59312) 2023-11-06T02:14:03.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.934+0000 7efecaffd640 1 -- 172.21.15.49:0/842897946 >> v2:172.21.15.150:3300/0 conn(0x7efecc06d150 msgr2=0x7efecc06d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:03.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.934+0000 7efecaffd640 1 --2- 172.21.15.49:0/842897946 >> v2:172.21.15.150:3300/0 conn(0x7efecc06d150 0x7efecc06d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:03.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.934+0000 7efecaffd640 1 -- 172.21.15.49:0/842897946 >> v1:172.21.15.49:6789/0 conn(0x7efecc15a880 legacy=0x7efecc070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:03.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.934+0000 7efecaffd640 1 -- 172.21.15.49:0/842897946 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efecc0ae920 con 0x7efecc070870 2023-11-06T02:14:03.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.934+0000 7efecaffd640 1 --2- 172.21.15.49:0/842897946 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efecc070870 0x7efecc0b15c0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7efecc0b02a0 tx=0x7efebc01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce232b6e15673bbb server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:03.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.934+0000 7efeca7fc640 1 -- 172.21.15.49:0/842897946 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efebc0ad680 con 0x7efecc070870 2023-11-06T02:14:03.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.934+0000 7efeca7fc640 1 -- 172.21.15.49:0/842897946 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efebc0ad7e0 con 0x7efecc070870 2023-11-06T02:14:03.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.938+0000 7efeca7fc640 1 -- 172.21.15.49:0/842897946 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efebc0adb70 con 0x7efecc070870 2023-11-06T02:14:03.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.938+0000 7efed1a68640 1 -- 172.21.15.49:0/842897946 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efecc070870 msgr2=0x7efecc0b15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:03.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.938+0000 7efed1a68640 1 --2- 172.21.15.49:0/842897946 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efecc070870 0x7efecc0b15c0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7efecc0b02a0 tx=0x7efebc01c470 comp rx=0 tx=0).stop 2023-11-06T02:14:03.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.938+0000 7efed1a68640 1 -- 172.21.15.49:0/842897946 shutdown_connections 2023-11-06T02:14:03.935 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.938+0000 7efed1a68640 1 --2- 172.21.15.49:0/842897946 >> v2:172.21.15.150:3300/0 conn(0x7efecc06d150 0x7efecc06d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:03.935 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.938+0000 7efed1a68640 1 --2- 172.21.15.49:0/842897946 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efecc070870 0x7efecc0b15c0 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:03.935 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.938+0000 7efed1a68640 1 -- 172.21.15.49:0/842897946 >> 172.21.15.49:0/842897946 conn(0x7efecc06a370 msgr2=0x7efecc0ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:03.935 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.938+0000 7efed1a68640 1 -- 172.21.15.49:0/842897946 shutdown_connections 2023-11-06T02:14:03.935 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.938+0000 7efed1a68640 1 -- 172.21.15.49:0/842897946 wait complete. 2023-11-06T02:14:03.938 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.938+0000 7efed1a68640 1 Processor -- start 2023-11-06T02:14:03.939 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.938+0000 7efed1a68640 1 -- start start 2023-11-06T02:14:03.939 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.938+0000 7efed1a68640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efecc06d150 0x7efecc155090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:03.939 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.938+0000 7efed1a68640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efecc070870 0x7efecc14d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:03.939 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.938+0000 7efed1a68640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efecc15a880 0x7efecc0cc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:03.939 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.938+0000 7efed1a68640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efecc157790 con 0x7efecc15a880 2023-11-06T02:14:03.940 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.938+0000 7efed1a68640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efecc1563a0 con 0x7efecc06d150 2023-11-06T02:14:03.940 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.938+0000 7efed1a68640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efecc14d6e0 con 0x7efecc070870 2023-11-06T02:14:03.940 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.938+0000 7efecbfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efecc15a880 0x7efecc0cc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:03.940 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.938+0000 7efecbfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efecc15a880 0x7efecc0cc3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51586/0 (socket says 172.21.15.49:51586) 2023-11-06T02:14:03.940 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.938+0000 7efecbfff640 1 -- 172.21.15.49:0/396935958 learned_addr learned my addr 172.21.15.49:0/396935958 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:03.940 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.938+0000 7efecb7fe640 1 --2- 172.21.15.49:0/396935958 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efecc06d150 0x7efecc155090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:03.941 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.938+0000 7efecaffd640 1 --2- 172.21.15.49:0/396935958 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efecc070870 0x7efecc14d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:03.941 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.938+0000 7efecbfff640 1 -- 172.21.15.49:0/396935958 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efecc070870 msgr2=0x7efecc14d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:03.941 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.938+0000 7efecbfff640 1 --2- 172.21.15.49:0/396935958 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efecc070870 0x7efecc14d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:03.941 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.938+0000 7efecbfff640 1 -- 172.21.15.49:0/396935958 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efecc06d150 msgr2=0x7efecc155090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:03.941 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.938+0000 7efecbfff640 1 --2- 172.21.15.49:0/396935958 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efecc06d150 0x7efecc155090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:03.942 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.938+0000 7efecbfff640 1 -- 172.21.15.49:0/396935958 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efebc0094d0 con 0x7efecc15a880 2023-11-06T02:14:03.942 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.938+0000 7efecb7fe640 1 --2- 172.21.15.49:0/396935958 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efecc06d150 0x7efecc155090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:14:03.942 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.938+0000 7efecaffd640 1 --2- 172.21.15.49:0/396935958 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efecc070870 0x7efecc14d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:14:03.942 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.938+0000 7efecbfff640 1 --2- 172.21.15.49:0/396935958 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efecc15a880 0x7efecc0cc3c0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7efec400b970 tx=0x7efec400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:03.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.938+0000 7efeb3fff640 1 -- 172.21.15.49:0/396935958 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec400cb40 con 0x7efecc15a880 2023-11-06T02:14:03.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.938+0000 7efeb3fff640 1 -- 172.21.15.49:0/396935958 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efec400ccc0 con 0x7efecc15a880 2023-11-06T02:14:03.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.938+0000 7efed1a68640 1 -- 172.21.15.49:0/396935958 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efecc0cc900 con 0x7efecc15a880 2023-11-06T02:14:03.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.938+0000 7efeb3fff640 1 -- 172.21.15.49:0/396935958 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec4014680 con 0x7efecc15a880 2023-11-06T02:14:03.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.938+0000 7efed1a68640 1 -- 172.21.15.49:0/396935958 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efecc155a00 con 0x7efecc15a880 2023-11-06T02:14:03.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.942+0000 7efed1a68640 1 -- 172.21.15.49:0/396935958 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efecc157900 con 0x7efecc15a880 2023-11-06T02:14:03.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.942+0000 7efeb3fff640 1 -- 172.21.15.49:0/396935958 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7efec401d3b0 con 0x7efecc15a880 2023-11-06T02:14:03.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.942+0000 7efeb3fff640 1 --2- 172.21.15.49:0/396935958 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7efeb8065a60 0x7efeb8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:03.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.942+0000 7efeb3fff640 1 -- 172.21.15.49:0/396935958 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6062+0+0 (secure 0 0 0) 0x7efec4083300 con 0x7efecc15a880 2023-11-06T02:14:03.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.942+0000 7efecb7fe640 1 --2- 172.21.15.49:0/396935958 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7efeb8065a60 0x7efeb8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:03.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.942+0000 7efecb7fe640 1 --2- 172.21.15.49:0/396935958 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7efeb8065a60 0x7efeb8067f20 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7efec0000f70 tx=0x7efec0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:03.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:03.946+0000 7efeb3fff640 1 -- 172.21.15.49:0/396935958 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7efecc157900 con 0x7efecc15a880 2023-11-06T02:14:04.087 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.090+0000 7efed1a68640 1 -- 172.21.15.49:0/396935958 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7efecc1578d0 con 0x7efecc15a880 2023-11-06T02:14:04.088 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.094+0000 7efeb3fff640 1 -- 172.21.15.49:0/396935958 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v80) v1 ==== 104+0+11 (secure 0 0 0) 0x7efec4054550 con 0x7efecc15a880 2023-11-06T02:14:04.088 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2023-11-06T02:14:04.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.094+0000 7efed1a68640 1 -- 172.21.15.49:0/396935958 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7efeb8065a60 msgr2=0x7efeb8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:04.090 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.094+0000 7efed1a68640 1 --2- 172.21.15.49:0/396935958 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7efeb8065a60 0x7efeb8067f20 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7efec0000f70 tx=0x7efec0009040 comp rx=0 tx=0).stop 2023-11-06T02:14:04.090 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.094+0000 7efed1a68640 1 -- 172.21.15.49:0/396935958 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efecc15a880 msgr2=0x7efecc0cc3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:04.090 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.094+0000 7efed1a68640 1 --2- 172.21.15.49:0/396935958 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efecc15a880 0x7efecc0cc3c0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7efec400b970 tx=0x7efec400be30 comp rx=0 tx=0).stop 2023-11-06T02:14:04.090 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.094+0000 7efed1a68640 1 -- 172.21.15.49:0/396935958 shutdown_connections 2023-11-06T02:14:04.091 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.094+0000 7efed1a68640 1 --2- 172.21.15.49:0/396935958 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7efeb8065a60 0x7efeb8067f20 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:04.091 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.094+0000 7efed1a68640 1 --2- 172.21.15.49:0/396935958 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efecc15a880 0x7efecc0cc3c0 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:04.091 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.094+0000 7efed1a68640 1 --2- 172.21.15.49:0/396935958 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efecc070870 0x7efecc14d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:04.091 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.094+0000 7efed1a68640 1 --2- 172.21.15.49:0/396935958 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efecc06d150 0x7efecc155090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:04.091 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.094+0000 7efed1a68640 1 -- 172.21.15.49:0/396935958 >> 172.21.15.49:0/396935958 conn(0x7efecc06a370 msgr2=0x7efecc15c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:04.091 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.094+0000 7efed1a68640 1 -- 172.21.15.49:0/396935958 shutdown_connections 2023-11-06T02:14:04.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.094+0000 7efed1a68640 1 -- 172.21.15.49:0/396935958 wait complete. 2023-11-06T02:14:04.105 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T02:14:04.232 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.238+0000 7f319f7e2640 1 Processor -- start 2023-11-06T02:14:04.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.238+0000 7f319f7e2640 1 -- start start 2023-11-06T02:14:04.234 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.238+0000 7f319f7e2640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3198070870 0x7f31980b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:04.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.238+0000 7f319f7e2640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f319806d150 0x7f319806d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:04.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.238+0000 7f319f7e2640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f319806db20 con 0x7f319815a880 2023-11-06T02:14:04.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.238+0000 7f319f7e2640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31980ae360 con 0x7f319806d150 2023-11-06T02:14:04.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.238+0000 7f319f7e2640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31980ae640 con 0x7f3198070870 2023-11-06T02:14:04.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.238+0000 7f319dd7f640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f319806d150 0x7f319806d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:04.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.238+0000 7f319dd7f640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f319806d150 0x7f319806d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51598/0 (socket says 172.21.15.49:51598) 2023-11-06T02:14:04.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.238+0000 7f319dd7f640 1 -- 172.21.15.49:0/2520568380 learned_addr learned my addr 172.21.15.49:0/2520568380 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:04.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.238+0000 7f319cd7d640 1 --2- 172.21.15.49:0/2520568380 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3198070870 0x7f31980b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:04.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.238+0000 7f319dd7f640 1 -- 172.21.15.49:0/2520568380 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3198070870 msgr2=0x7f31980b15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:04.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.238+0000 7f319dd7f640 1 --2- 172.21.15.49:0/2520568380 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3198070870 0x7f31980b15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:04.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.238+0000 7f319dd7f640 1 -- 172.21.15.49:0/2520568380 >> v1:172.21.15.150:6789/0 conn(0x7f319815a880 legacy=0x7f3198070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:04.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.238+0000 7f319dd7f640 1 -- 172.21.15.49:0/2520568380 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31980ae920 con 0x7f319806d150 2023-11-06T02:14:04.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.238+0000 7f319cd7d640 1 --2- 172.21.15.49:0/2520568380 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3198070870 0x7f31980b15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:14:04.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.242+0000 7f319dd7f640 1 --2- 172.21.15.49:0/2520568380 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f319806d150 0x7f319806d5e0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f3190009e50 tx=0x7f319001c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afdf05eef683c5ad server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:04.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.242+0000 7f318ffff640 1 -- 172.21.15.49:0/2520568380 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31900a5760 con 0x7f319806d150 2023-11-06T02:14:04.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.246+0000 7f318ffff640 1 -- 172.21.15.49:0/2520568380 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31900a58c0 con 0x7f319806d150 2023-11-06T02:14:04.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.246+0000 7f318ffff640 1 -- 172.21.15.49:0/2520568380 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31900a5c50 con 0x7f319806d150 2023-11-06T02:14:04.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.246+0000 7f319f7e2640 1 -- 172.21.15.49:0/2520568380 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f319806d150 msgr2=0x7f319806d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:04.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.246+0000 7f319f7e2640 1 --2- 172.21.15.49:0/2520568380 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f319806d150 0x7f319806d5e0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f3190009e50 tx=0x7f319001c9d0 comp rx=0 tx=0).stop 2023-11-06T02:14:04.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.246+0000 7f319f7e2640 1 -- 172.21.15.49:0/2520568380 shutdown_connections 2023-11-06T02:14:04.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.246+0000 7f319f7e2640 1 --2- 172.21.15.49:0/2520568380 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3198070870 0x7f31980b15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:04.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.246+0000 7f319f7e2640 1 --2- 172.21.15.49:0/2520568380 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f319806d150 0x7f319806d5e0 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:04.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.246+0000 7f319f7e2640 1 -- 172.21.15.49:0/2520568380 >> 172.21.15.49:0/2520568380 conn(0x7f319806a370 msgr2=0x7f31980ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:04.242 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.246+0000 7f319f7e2640 1 -- 172.21.15.49:0/2520568380 shutdown_connections 2023-11-06T02:14:04.242 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.246+0000 7f319f7e2640 1 -- 172.21.15.49:0/2520568380 wait complete. 2023-11-06T02:14:04.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.246+0000 7f319f7e2640 1 Processor -- start 2023-11-06T02:14:04.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.246+0000 7f319f7e2640 1 -- start start 2023-11-06T02:14:04.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.246+0000 7f319f7e2640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f319806d150 0x7f31980c5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:04.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.246+0000 7f319f7e2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3198070870 0x7f31981528f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:04.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.246+0000 7f319f7e2640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f319815a880 0x7f319814d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:04.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.246+0000 7f319f7e2640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3198150bd0 con 0x7f3198070870 2023-11-06T02:14:04.246 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.246+0000 7f319f7e2640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31980c5740 con 0x7f319806d150 2023-11-06T02:14:04.246 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.246+0000 7f319f7e2640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3198152e30 con 0x7f319815a880 2023-11-06T02:14:04.246 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.246+0000 7f319cd7d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3198070870 0x7f31981528f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:04.246 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.246+0000 7f319cd7d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3198070870 0x7f31981528f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51614/0 (socket says 172.21.15.49:51614) 2023-11-06T02:14:04.246 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.246+0000 7f319cd7d640 1 -- 172.21.15.49:0/2900994897 learned_addr learned my addr 172.21.15.49:0/2900994897 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:04.247 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.246+0000 7f319d57e640 1 --2- 172.21.15.49:0/2900994897 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f319806d150 0x7f31980c5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:04.247 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.246+0000 7f319dd7f640 1 --2- 172.21.15.49:0/2900994897 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f319815a880 0x7f319814d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:04.247 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.246+0000 7f319cd7d640 1 -- 172.21.15.49:0/2900994897 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f319815a880 msgr2=0x7f319814d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:04.247 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.246+0000 7f319cd7d640 1 --2- 172.21.15.49:0/2900994897 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f319815a880 0x7f319814d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:04.248 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.246+0000 7f319cd7d640 1 -- 172.21.15.49:0/2900994897 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f319806d150 msgr2=0x7f31980c5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:04.248 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.246+0000 7f319cd7d640 1 --2- 172.21.15.49:0/2900994897 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f319806d150 0x7f31980c5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:04.248 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.246+0000 7f319cd7d640 1 -- 172.21.15.49:0/2900994897 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3190009b30 con 0x7f3198070870 2023-11-06T02:14:04.248 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.246+0000 7f319cd7d640 1 --2- 172.21.15.49:0/2900994897 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3198070870 0x7f31981528f0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f318800c7f0 tx=0x7f318800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:04.248 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.246+0000 7f318dffb640 1 -- 172.21.15.49:0/2900994897 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3188007820 con 0x7f3198070870 2023-11-06T02:14:04.249 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.246+0000 7f318dffb640 1 -- 172.21.15.49:0/2900994897 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3188007980 con 0x7f3198070870 2023-11-06T02:14:04.249 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.246+0000 7f319f7e2640 1 -- 172.21.15.49:0/2900994897 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3198150230 con 0x7f3198070870 2023-11-06T02:14:04.249 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.246+0000 7f318dffb640 1 -- 172.21.15.49:0/2900994897 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3188007d30 con 0x7f3198070870 2023-11-06T02:14:04.249 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.246+0000 7f319f7e2640 1 -- 172.21.15.49:0/2900994897 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31981507f0 con 0x7f3198070870 2023-11-06T02:14:04.249 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.250+0000 7f319f7e2640 1 -- 172.21.15.49:0/2900994897 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3198074040 con 0x7f3198070870 2023-11-06T02:14:04.250 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.250+0000 7f318dffb640 1 -- 172.21.15.49:0/2900994897 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f31880113f0 con 0x7f3198070870 2023-11-06T02:14:04.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.250+0000 7f318dffb640 1 --2- 172.21.15.49:0/2900994897 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f317c065a60 0x7f317c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:04.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.250+0000 7f318dffb640 1 -- 172.21.15.49:0/2900994897 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3188083680 con 0x7f3198070870 2023-11-06T02:14:04.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.250+0000 7f319d57e640 1 --2- 172.21.15.49:0/2900994897 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f317c065a60 0x7f317c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:04.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.250+0000 7f319d57e640 1 --2- 172.21.15.49:0/2900994897 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f317c065a60 0x7f317c067f20 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f3194004e30 tx=0x7f3194009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:04.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.254+0000 7f318dffb640 1 -- 172.21.15.49:0/2900994897 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f3188050e50 con 0x7f3198070870 2023-11-06T02:14:04.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.410+0000 7f319f7e2640 1 -- 172.21.15.49:0/2900994897 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f319806c430 con 0x7f3198070870 2023-11-06T02:14:04.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.410+0000 7f318dffb640 1 -- 172.21.15.49:0/2900994897 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v80) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3188054470 con 0x7f3198070870 2023-11-06T02:14:04.406 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2023-11-06T02:14:04.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.414+0000 7f319f7e2640 1 -- 172.21.15.49:0/2900994897 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f317c065a60 msgr2=0x7f317c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:04.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.414+0000 7f319f7e2640 1 --2- 172.21.15.49:0/2900994897 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f317c065a60 0x7f317c067f20 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f3194004e30 tx=0x7f3194009040 comp rx=0 tx=0).stop 2023-11-06T02:14:04.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.414+0000 7f319f7e2640 1 -- 172.21.15.49:0/2900994897 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3198070870 msgr2=0x7f31981528f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:04.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.414+0000 7f319f7e2640 1 --2- 172.21.15.49:0/2900994897 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3198070870 0x7f31981528f0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f318800c7f0 tx=0x7f318800ccb0 comp rx=0 tx=0).stop 2023-11-06T02:14:04.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.414+0000 7f319f7e2640 1 -- 172.21.15.49:0/2900994897 shutdown_connections 2023-11-06T02:14:04.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.414+0000 7f319f7e2640 1 --2- 172.21.15.49:0/2900994897 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f317c065a60 0x7f317c067f20 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:04.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.414+0000 7f319f7e2640 1 --2- 172.21.15.49:0/2900994897 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3198070870 0x7f31981528f0 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:04.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.414+0000 7f319f7e2640 1 --2- 172.21.15.49:0/2900994897 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f319815a880 0x7f319814d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:04.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.414+0000 7f319f7e2640 1 --2- 172.21.15.49:0/2900994897 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f319806d150 0x7f31980c5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:04.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.414+0000 7f319f7e2640 1 -- 172.21.15.49:0/2900994897 >> 172.21.15.49:0/2900994897 conn(0x7f319806a370 msgr2=0x7f319815ae70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:04.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.414+0000 7f319f7e2640 1 -- 172.21.15.49:0/2900994897 shutdown_connections 2023-11-06T02:14:04.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.414+0000 7f319f7e2640 1 -- 172.21.15.49:0/2900994897 wait complete. 2023-11-06T02:14:04.425 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-06T02:14:04.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.558+0000 7fc48fc90640 1 Processor -- start 2023-11-06T02:14:04.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.558+0000 7fc48fc90640 1 -- start start 2023-11-06T02:14:04.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.558+0000 7fc48fc90640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc48815a940 0x7fc4880b3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:04.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.558+0000 7fc48fc90640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc4880b0110 0x7fc4880b05a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:04.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.558+0000 7fc48fc90640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc48815e520 0x7fc48815e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:04.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.558+0000 7fc48fc90640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4880b0ae0 con 0x7fc4880b0110 2023-11-06T02:14:04.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.558+0000 7fc48fc90640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4880cafc0 con 0x7fc48815a940 2023-11-06T02:14:04.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.558+0000 7fc48fc90640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4880cb210 con 0x7fc48815e520 2023-11-06T02:14:04.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.558+0000 7fc48d22b640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc4880b0110 0x7fc4880b05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:04.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.558+0000 7fc48d22b640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc4880b0110 0x7fc4880b05a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51626/0 (socket says 172.21.15.49:51626) 2023-11-06T02:14:04.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.558+0000 7fc48d22b640 1 -- 172.21.15.49:0/2561413390 learned_addr learned my addr 172.21.15.49:0/2561413390 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:04.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.558+0000 7fc48da2c640 1 --2- 172.21.15.49:0/2561413390 >> v2:172.21.15.150:3300/0 conn(0x7fc48815a940 0x7fc4880b3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:04.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.558+0000 7fc48e22d640 1 --2- 172.21.15.49:0/2561413390 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc48815e520 0x7fc48815e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:04.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.558+0000 7fc48d22b640 1 -- 172.21.15.49:0/2561413390 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc48815e520 msgr2=0x7fc48815e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:04.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.558+0000 7fc48d22b640 1 --2- 172.21.15.49:0/2561413390 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc48815e520 0x7fc48815e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:04.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.558+0000 7fc48d22b640 1 -- 172.21.15.49:0/2561413390 >> v2:172.21.15.150:3300/0 conn(0x7fc48815a940 msgr2=0x7fc4880b3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:04.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.558+0000 7fc48d22b640 1 --2- 172.21.15.49:0/2561413390 >> v2:172.21.15.150:3300/0 conn(0x7fc48815a940 0x7fc4880b3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:04.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.558+0000 7fc48d22b640 1 -- 172.21.15.49:0/2561413390 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4880cb4f0 con 0x7fc4880b0110 2023-11-06T02:14:04.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.558+0000 7fc48e22d640 1 --2- 172.21.15.49:0/2561413390 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc48815e520 0x7fc48815e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:14:04.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.562+0000 7fc48d22b640 1 --2- 172.21.15.49:0/2561413390 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4880b0110 0x7fc4880b05a0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fc48806ce80 tx=0x7fc47801c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=575174846c2cdbf7 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:04.563 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.562+0000 7fc48ca2a640 1 -- 172.21.15.49:0/2561413390 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4780ab600 con 0x7fc4880b0110 2023-11-06T02:14:04.563 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.562+0000 7fc48ca2a640 1 -- 172.21.15.49:0/2561413390 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4780ab760 con 0x7fc4880b0110 2023-11-06T02:14:04.563 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.562+0000 7fc48ca2a640 1 -- 172.21.15.49:0/2561413390 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4780abaf0 con 0x7fc4880b0110 2023-11-06T02:14:04.563 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.562+0000 7fc48fc90640 1 -- 172.21.15.49:0/2561413390 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4880b0110 msgr2=0x7fc4880b05a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:04.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.562+0000 7fc48fc90640 1 --2- 172.21.15.49:0/2561413390 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4880b0110 0x7fc4880b05a0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fc48806ce80 tx=0x7fc47801c440 comp rx=0 tx=0).stop 2023-11-06T02:14:04.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.562+0000 7fc48fc90640 1 -- 172.21.15.49:0/2561413390 shutdown_connections 2023-11-06T02:14:04.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.562+0000 7fc48fc90640 1 --2- 172.21.15.49:0/2561413390 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc48815e520 0x7fc48815e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:04.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.562+0000 7fc48fc90640 1 --2- 172.21.15.49:0/2561413390 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4880b0110 0x7fc4880b05a0 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:04.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.562+0000 7fc48fc90640 1 --2- 172.21.15.49:0/2561413390 >> v2:172.21.15.150:3300/0 conn(0x7fc48815a940 0x7fc4880b3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:04.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.562+0000 7fc48fc90640 1 -- 172.21.15.49:0/2561413390 >> 172.21.15.49:0/2561413390 conn(0x7fc48806c160 msgr2=0x7fc48806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:04.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.562+0000 7fc48fc90640 1 -- 172.21.15.49:0/2561413390 shutdown_connections 2023-11-06T02:14:04.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.562+0000 7fc48fc90640 1 -- 172.21.15.49:0/2561413390 wait complete. 2023-11-06T02:14:04.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.562+0000 7fc48fc90640 1 Processor -- start 2023-11-06T02:14:04.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.562+0000 7fc48fc90640 1 -- start start 2023-11-06T02:14:04.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.566+0000 7fc48fc90640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4880b0110 0x7fc4880c4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:04.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.566+0000 7fc48fc90640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc48815a940 0x7fc488153590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:04.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.566+0000 7fc48fc90640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc48815e520 0x7fc4880c2770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:04.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.566+0000 7fc48fc90640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc488153ad0 con 0x7fc48815a940 2023-11-06T02:14:04.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.566+0000 7fc48fc90640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4880c3070 con 0x7fc48815e520 2023-11-06T02:14:04.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.566+0000 7fc48fc90640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4880c3350 con 0x7fc4880b0110 2023-11-06T02:14:04.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.566+0000 7fc48d22b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc48815a940 0x7fc488153590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:04.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.566+0000 7fc48d22b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc48815a940 0x7fc488153590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51636/0 (socket says 172.21.15.49:51636) 2023-11-06T02:14:04.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.566+0000 7fc48d22b640 1 -- 172.21.15.49:0/3251212423 learned_addr learned my addr 172.21.15.49:0/3251212423 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:04.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.566+0000 7fc48da2c640 1 --2- 172.21.15.49:0/3251212423 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4880b0110 0x7fc4880c4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:04.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.566+0000 7fc48d22b640 1 -- 172.21.15.49:0/3251212423 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4880b0110 msgr2=0x7fc4880c4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:04.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.566+0000 7fc48e22d640 1 --2- 172.21.15.49:0/3251212423 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc48815e520 0x7fc4880c2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:04.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.566+0000 7fc48d22b640 1 --2- 172.21.15.49:0/3251212423 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4880b0110 0x7fc4880c4110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:04.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.566+0000 7fc48d22b640 1 -- 172.21.15.49:0/3251212423 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc48815e520 msgr2=0x7fc4880c2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:04.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.566+0000 7fc48d22b640 1 --2- 172.21.15.49:0/3251212423 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc48815e520 0x7fc4880c2770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:04.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.566+0000 7fc48d22b640 1 -- 172.21.15.49:0/3251212423 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4780094d0 con 0x7fc48815a940 2023-11-06T02:14:04.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.566+0000 7fc48d22b640 1 --2- 172.21.15.49:0/3251212423 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc48815a940 0x7fc488153590 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fc47801aaf0 tx=0x7fc47801ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:04.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.566+0000 7fc47e7fc640 1 -- 172.21.15.49:0/3251212423 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4780ab600 con 0x7fc48815a940 2023-11-06T02:14:04.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.566+0000 7fc47e7fc640 1 -- 172.21.15.49:0/3251212423 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4780b4800 con 0x7fc48815a940 2023-11-06T02:14:04.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.566+0000 7fc47e7fc640 1 -- 172.21.15.49:0/3251212423 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4780b4bb0 con 0x7fc48815a940 2023-11-06T02:14:04.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.566+0000 7fc48fc90640 1 -- 172.21.15.49:0/3251212423 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4880c3630 con 0x7fc48815a940 2023-11-06T02:14:04.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.566+0000 7fc48fc90640 1 -- 172.21.15.49:0/3251212423 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4880c3bf0 con 0x7fc48815a940 2023-11-06T02:14:04.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.566+0000 7fc48fc90640 1 -- 172.21.15.49:0/3251212423 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc488074040 con 0x7fc48815a940 2023-11-06T02:14:04.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.566+0000 7fc47e7fc640 1 -- 172.21.15.49:0/3251212423 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fc4780b4d10 con 0x7fc48815a940 2023-11-06T02:14:04.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.566+0000 7fc47e7fc640 1 --2- 172.21.15.49:0/3251212423 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc46c065a60 0x7fc46c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:04.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.566+0000 7fc47e7fc640 1 -- 172.21.15.49:0/3251212423 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc47811d8e0 con 0x7fc48815a940 2023-11-06T02:14:04.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.570+0000 7fc48da2c640 1 --2- 172.21.15.49:0/3251212423 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc46c065a60 0x7fc46c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:04.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.570+0000 7fc48da2c640 1 --2- 172.21.15.49:0/3251212423 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc46c065a60 0x7fc46c067f20 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fc4880ca600 tx=0x7fc484008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:04.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.570+0000 7fc47e7fc640 1 -- 172.21.15.49:0/3251212423 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fc4780eb1f0 con 0x7fc48815a940 2023-11-06T02:14:04.736 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.742+0000 7fc48fc90640 1 -- 172.21.15.49:0/3251212423 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc4880c3db0 con 0x7fc48815a940 2023-11-06T02:14:04.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.742+0000 7fc47e7fc640 1 -- 172.21.15.49:0/3251212423 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v80) v1 ==== 80+0+0 (secure 0 0 0) 0x7fc4780c4080 con 0x7fc48815a940 2023-11-06T02:14:04.737 INFO:teuthology.orchestra.run.smithi049.stderr:listed 0 entries 2023-11-06T02:14:04.739 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.742+0000 7fc48fc90640 1 -- 172.21.15.49:0/3251212423 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc46c065a60 msgr2=0x7fc46c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:04.739 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.742+0000 7fc48fc90640 1 --2- 172.21.15.49:0/3251212423 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc46c065a60 0x7fc46c067f20 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fc4880ca600 tx=0x7fc484008040 comp rx=0 tx=0).stop 2023-11-06T02:14:04.739 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.742+0000 7fc48fc90640 1 -- 172.21.15.49:0/3251212423 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc48815a940 msgr2=0x7fc488153590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:04.740 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.742+0000 7fc48fc90640 1 --2- 172.21.15.49:0/3251212423 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc48815a940 0x7fc488153590 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fc47801aaf0 tx=0x7fc47801ac80 comp rx=0 tx=0).stop 2023-11-06T02:14:04.740 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.742+0000 7fc48fc90640 1 -- 172.21.15.49:0/3251212423 shutdown_connections 2023-11-06T02:14:04.740 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.742+0000 7fc48fc90640 1 --2- 172.21.15.49:0/3251212423 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc46c065a60 0x7fc46c067f20 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:04.740 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.742+0000 7fc48fc90640 1 --2- 172.21.15.49:0/3251212423 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc48815a940 0x7fc488153590 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:04.740 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.742+0000 7fc48fc90640 1 --2- 172.21.15.49:0/3251212423 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc48815e520 0x7fc4880c2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:04.741 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.742+0000 7fc48fc90640 1 --2- 172.21.15.49:0/3251212423 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4880b0110 0x7fc4880c4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:04.741 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.742+0000 7fc48fc90640 1 -- 172.21.15.49:0/3251212423 >> 172.21.15.49:0/3251212423 conn(0x7fc48806c160 msgr2=0x7fc48815d850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:04.741 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.742+0000 7fc48fc90640 1 -- 172.21.15.49:0/3251212423 shutdown_connections 2023-11-06T02:14:04.741 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.742+0000 7fc48fc90640 1 -- 172.21.15.49:0/3251212423 wait complete. 2023-11-06T02:14:04.756 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-11-06T02:14:04.756 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T02:14:04.763 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T02:14:04.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.926+0000 7f8728c8c640 1 Processor -- start 2023-11-06T02:14:04.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.926+0000 7f8728c8c640 1 -- start start 2023-11-06T02:14:04.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.926+0000 7f8728c8c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f87240b1f90 0x7f87240cda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:04.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.926+0000 7f8728c8c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f8724161650 0x7f87240d1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:04.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.926+0000 7f8728c8c640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87240d17c0 con 0x7f87240d2190 2023-11-06T02:14:04.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.926+0000 7f8728c8c640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87240ce6c0 con 0x7f8724161650 2023-11-06T02:14:04.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.926+0000 7f8728c8c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8724161ad0 con 0x7f87240b1f90 2023-11-06T02:14:04.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.926+0000 7f871bfff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f8724161650 0x7f87240d1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:04.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.926+0000 7f871bfff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f8724161650 0x7f87240d1c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51648/0 (socket says 172.21.15.49:51648) 2023-11-06T02:14:04.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.926+0000 7f871bfff640 1 -- 172.21.15.49:0/1154695868 learned_addr learned my addr 172.21.15.49:0/1154695868 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:04.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.926+0000 7f872259c640 1 --2- 172.21.15.49:0/1154695868 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f87240b1f90 0x7f87240cda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:04.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.926+0000 7f871bfff640 1 -- 172.21.15.49:0/1154695868 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f87240b1f90 msgr2=0x7f87240cda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:04.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.926+0000 7f871bfff640 1 --2- 172.21.15.49:0/1154695868 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f87240b1f90 0x7f87240cda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:04.924 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.926+0000 7f8721d9b640 1 -- 172.21.15.49:0/1154695868 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1319552956 0 0) 0x7f87240d17c0 con 0x7f87240d2190 2023-11-06T02:14:04.924 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.926+0000 7f871bfff640 1 -- 172.21.15.49:0/1154695868 >> v1:172.21.15.49:6789/0 conn(0x7f87240d2190 legacy=0x7f87240714a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:04.924 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.926+0000 7f871bfff640 1 -- 172.21.15.49:0/1154695868 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87240ce100 con 0x7f8724161650 2023-11-06T02:14:04.925 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.930+0000 7f871bfff640 1 --2- 172.21.15.49:0/1154695868 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8724161650 0x7f87240d1c50 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f870c009870 tx=0x7f870c01c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d6722b78694b1d3 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:04.926 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.930+0000 7f8721d9b640 1 -- 172.21.15.49:0/1154695868 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f870c0aa670 con 0x7f8724161650 2023-11-06T02:14:04.926 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.930+0000 7f8721d9b640 1 -- 172.21.15.49:0/1154695868 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f870c0aa7d0 con 0x7f8724161650 2023-11-06T02:14:04.926 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.930+0000 7f8721d9b640 1 -- 172.21.15.49:0/1154695868 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f870c0aab60 con 0x7f8724161650 2023-11-06T02:14:04.926 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.930+0000 7f8728c8c640 1 -- 172.21.15.49:0/1154695868 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8724161650 msgr2=0x7f87240d1c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:04.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.930+0000 7f8728c8c640 1 --2- 172.21.15.49:0/1154695868 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8724161650 0x7f87240d1c50 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f870c009870 tx=0x7f870c01c510 comp rx=0 tx=0).stop 2023-11-06T02:14:04.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.930+0000 7f8728c8c640 1 -- 172.21.15.49:0/1154695868 shutdown_connections 2023-11-06T02:14:04.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.930+0000 7f8728c8c640 1 --2- 172.21.15.49:0/1154695868 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8724161650 0x7f87240d1c50 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:04.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.930+0000 7f8728c8c640 1 --2- 172.21.15.49:0/1154695868 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f87240b1f90 0x7f87240cda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:04.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.930+0000 7f8728c8c640 1 -- 172.21.15.49:0/1154695868 >> 172.21.15.49:0/1154695868 conn(0x7f87240d2aa0 msgr2=0x7f87240d2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:04.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.930+0000 7f8728c8c640 1 -- 172.21.15.49:0/1154695868 shutdown_connections 2023-11-06T02:14:04.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.930+0000 7f8728c8c640 1 -- 172.21.15.49:0/1154695868 wait complete. 2023-11-06T02:14:04.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.930+0000 7f8728c8c640 1 Processor -- start 2023-11-06T02:14:04.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.930+0000 7f8728c8c640 1 -- start start 2023-11-06T02:14:04.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.930+0000 7f8728c8c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f87240b1f90 0x7f8724150c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:04.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.930+0000 7f8728c8c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f87240d2190 0x7f87240cc230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:04.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.930+0000 7f8728c8c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8724161650 0x7f8724159280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:04.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.930+0000 7f8728c8c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87240cce60 con 0x7f8724161650 2023-11-06T02:14:04.932 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.930+0000 7f8728c8c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87241511b0 con 0x7f87240b1f90 2023-11-06T02:14:04.932 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.930+0000 7f8728c8c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87240ccb80 con 0x7f87240d2190 2023-11-06T02:14:04.932 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.930+0000 7f8722d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8724161650 0x7f8724159280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:04.932 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.930+0000 7f8722d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8724161650 0x7f8724159280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51660/0 (socket says 172.21.15.49:51660) 2023-11-06T02:14:04.932 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.930+0000 7f8722d9d640 1 -- 172.21.15.49:0/2384990175 learned_addr learned my addr 172.21.15.49:0/2384990175 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:04.932 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.930+0000 7f872259c640 1 --2- 172.21.15.49:0/2384990175 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f87240b1f90 0x7f8724150c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:04.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.930+0000 7f871bfff640 1 --2- 172.21.15.49:0/2384990175 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f87240d2190 0x7f87240cc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:04.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.930+0000 7f8722d9d640 1 -- 172.21.15.49:0/2384990175 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f87240d2190 msgr2=0x7f87240cc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:04.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.930+0000 7f8722d9d640 1 --2- 172.21.15.49:0/2384990175 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f87240d2190 0x7f87240cc230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:04.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.930+0000 7f8722d9d640 1 -- 172.21.15.49:0/2384990175 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f87240b1f90 msgr2=0x7f8724150c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:04.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.930+0000 7f8722d9d640 1 --2- 172.21.15.49:0/2384990175 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f87240b1f90 0x7f8724150c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:04.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.930+0000 7f8722d9d640 1 -- 172.21.15.49:0/2384990175 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f870c0094d0 con 0x7f8724161650 2023-11-06T02:14:04.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.930+0000 7f8722d9d640 1 --2- 172.21.15.49:0/2384990175 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8724161650 0x7f8724159280 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f871c001570 tx=0x7f871c00cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:04.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.930+0000 7f871affd640 1 -- 172.21.15.49:0/2384990175 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f871c003db0 con 0x7f8724161650 2023-11-06T02:14:04.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.930+0000 7f871affd640 1 -- 172.21.15.49:0/2384990175 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f871c005070 con 0x7f8724161650 2023-11-06T02:14:04.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.930+0000 7f8728c8c640 1 -- 172.21.15.49:0/2384990175 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87241587a0 con 0x7f8724161650 2023-11-06T02:14:04.935 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.930+0000 7f871affd640 1 -- 172.21.15.49:0/2384990175 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f871c005420 con 0x7f8724161650 2023-11-06T02:14:04.935 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.930+0000 7f8728c8c640 1 -- 172.21.15.49:0/2384990175 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8724158d60 con 0x7f8724161650 2023-11-06T02:14:04.935 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.934+0000 7f8728c8c640 1 -- 172.21.15.49:0/2384990175 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8724158f50 con 0x7f8724161650 2023-11-06T02:14:04.935 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.934+0000 7f871affd640 1 -- 172.21.15.49:0/2384990175 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f871c005580 con 0x7f8724161650 2023-11-06T02:14:04.935 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.934+0000 7f871affd640 1 --2- 172.21.15.49:0/2384990175 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8704065a60 0x7f8704067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:04.935 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.934+0000 7f871affd640 1 -- 172.21.15.49:0/2384990175 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f871c082ba0 con 0x7f8724161650 2023-11-06T02:14:04.936 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.934+0000 7f872259c640 1 --2- 172.21.15.49:0/2384990175 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8704065a60 0x7f8704067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:04.936 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.934+0000 7f872259c640 1 --2- 172.21.15.49:0/2384990175 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8704065a60 0x7f8704067f20 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f871400b9d0 tx=0x7f8714008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:04.936 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:04.938+0000 7f871affd640 1 -- 172.21.15.49:0/2384990175 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f871c050400 con 0x7f8724161650 2023-11-06T02:14:05.086 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.090+0000 7f8728c8c640 1 -- 172.21.15.49:0/2384990175 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8724074040 con 0x7f8724161650 2023-11-06T02:14:05.088 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.094+0000 7f871affd640 1 -- 172.21.15.49:0/2384990175 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+17886 (secure 0 0 0) 0x7f871c017020 con 0x7f8724161650 2023-11-06T02:14:05.089 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:05.089 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":80,"fsid":"1a8b035c-6ba4-499a-b876-aea47fd9c0d4","created":"2023-11-06T02:08:06.440857+0000","modified":"2023-11-06T02:13:34.630031+0000","last_up_change":"2023-11-06T02:08:17.982372+0000","last_in_change":"2023-11-06T02:08:08.906631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T02:08:19.252222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T02:08:24.776211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-11-06T02:13:31.459314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-11-06T02:13:32.603710+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ad55af8-0571-4e6a-8722-b5d429b7496d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6809","nonce":38116},{"type":"v1","addr":"172.21.15.49:6811","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6813","nonce":38116},{"type":"v1","addr":"172.21.15.49:6815","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6821","nonce":38116},{"type":"v1","addr":"172.21.15.49:6823","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6817","nonce":38116},{"type":"v1","addr":"172.21.15.49:6819","nonce":38116}]},"public_addr":"172.21.15.49:6811/38116","cluster_addr":"172.21.15.49:6815/38116","heartbeat_back_addr":"172.21.15.49:6823/38116","heartbeat_front_addr":"172.21.15.49:6819/38116","state":["exists","up"]},{"osd":1,"uuid":"c8b31f85-a757-43ed-9419-351acc46611b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":38113},{"type":"v1","addr":"172.21.15.49:6810","nonce":38113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":38113},{"type":"v1","addr":"172.21.15.49:6814","nonce":38113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":38113},{"type":"v1","addr":"172.21.15.49:6822","nonce":38113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":38113},{"type":"v1","addr":"172.21.15.49:6818","nonce":38113}]},"public_addr":"172.21.15.49:6810/38113","cluster_addr":"172.21.15.49:6814/38113","heartbeat_back_addr":"172.21.15.49:6822/38113","heartbeat_front_addr":"172.21.15.49:6818/38113","state":["exists","up"]},{"osd":2,"uuid":"555b3bf3-0a70-4b91-8efd-e4204b5fc26e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":38114},{"type":"v1","addr":"172.21.15.49:6801","nonce":38114}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":38114},{"type":"v1","addr":"172.21.15.49:6803","nonce":38114}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":38114},{"type":"v1","addr":"172.21.15.49:6807","nonce":38114}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":38114},{"type":"v1","addr":"172.21.15.49:6805","nonce":38114}]},"public_addr":"172.21.15.49:6801/38114","cluster_addr":"172.21.15.49:6803/38114","heartbeat_back_addr":"172.21.15.49:6807/38114","heartbeat_front_addr":"172.21.15.49:6805/38114","state":["exists","up"]},{"osd":3,"uuid":"8d4e5d77-03ef-4b9b-bfcb-5ae93a46795c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":38115},{"type":"v1","addr":"172.21.15.49:6825","nonce":38115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":38115},{"type":"v1","addr":"172.21.15.49:6827","nonce":38115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":38115},{"type":"v1","addr":"172.21.15.49:6831","nonce":38115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":38115},{"type":"v1","addr":"172.21.15.49:6829","nonce":38115}]},"public_addr":"172.21.15.49:6825/38115","cluster_addr":"172.21.15.49:6827/38115","heartbeat_back_addr":"172.21.15.49:6831/38115","heartbeat_front_addr":"172.21.15.49:6829/38115","state":["exists","up"]},{"osd":4,"uuid":"dba03ad4-68a3-4cf3-9218-ac611f02eacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":38417},{"type":"v1","addr":"172.21.15.150:6809","nonce":38417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":38417},{"type":"v1","addr":"172.21.15.150:6811","nonce":38417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":38417},{"type":"v1","addr":"172.21.15.150:6815","nonce":38417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":38417},{"type":"v1","addr":"172.21.15.150:6813","nonce":38417}]},"public_addr":"172.21.15.150:6809/38417","cluster_addr":"172.21.15.150:6811/38417","heartbeat_back_addr":"172.21.15.150:6815/38417","heartbeat_front_addr":"172.21.15.150:6813/38417","state":["exists","up"]},{"osd":5,"uuid":"0df9c303-c255-4640-8e91-01f1c3efe18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":38418},{"type":"v1","addr":"172.21.15.150:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":38418},{"type":"v1","addr":"172.21.15.150:6803","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":38418},{"type":"v1","addr":"172.21.15.150:6807","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":38418},{"type":"v1","addr":"172.21.15.150:6805","nonce":38418}]},"public_addr":"172.21.15.150:6801/38418","cluster_addr":"172.21.15.150:6803/38418","heartbeat_back_addr":"172.21.15.150:6807/38418","heartbeat_front_addr":"172.21.15.150:6805/38418","state":["exists","up"]},{"osd":6,"uuid":"e59fa189-4b70-4c06-9722-c02b90828bf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":38428},{"type":"v1","addr":"172.21.15.150:6825","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":38428},{"type":"v1","addr":"172.21.15.150:6827","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":38428},{"type":"v1","addr":"172.21.15.150:6831","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":38428},{"type":"v1","addr":"172.21.15.150:6829","nonce":38428}]},"public_addr":"172.21.15.150:6825/38428","cluster_addr":"172.21.15.150:6827/38428","heartbeat_back_addr":"172.21.15.150:6831/38428","heartbeat_front_addr":"172.21.15.150:6829/38428","state":["exists","up"]},{"osd":7,"uuid":"a64f57e1-ef98-46bd-b304-385759e95898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":38421},{"type":"v1","addr":"172.21.15.150:6817","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":38421},{"type":"v1","addr":"172.21.15.150:6819","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":38421},{"type":"v1","addr":"172.21.15.150:6823","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":38421},{"type":"v1","addr":"172.21.15.150:6821","nonce":38421}]},"public_addr":"172.21.15.150:6817/38421","cluster_addr":"172.21.15.150:6819/38421","heartbeat_back_addr":"172.21.15.150:6823/38421","heartbeat_front_addr":"172.21.15.150:6821/38421","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.438552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.413280+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.296466+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.424966+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.276953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.230984+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.410235+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.480084+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T02:14:05.091 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.094+0000 7f8728c8c640 1 -- 172.21.15.49:0/2384990175 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8704065a60 msgr2=0x7f8704067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:05.091 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.094+0000 7f8728c8c640 1 --2- 172.21.15.49:0/2384990175 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8704065a60 0x7f8704067f20 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f871400b9d0 tx=0x7f8714008040 comp rx=0 tx=0).stop 2023-11-06T02:14:05.091 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.094+0000 7f8728c8c640 1 -- 172.21.15.49:0/2384990175 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8724161650 msgr2=0x7f8724159280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:05.091 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.094+0000 7f8728c8c640 1 --2- 172.21.15.49:0/2384990175 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8724161650 0x7f8724159280 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f871c001570 tx=0x7f871c00cd20 comp rx=0 tx=0).stop 2023-11-06T02:14:05.091 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.094+0000 7f8728c8c640 1 -- 172.21.15.49:0/2384990175 shutdown_connections 2023-11-06T02:14:05.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.094+0000 7f8728c8c640 1 --2- 172.21.15.49:0/2384990175 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8704065a60 0x7f8704067f20 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:05.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.094+0000 7f8728c8c640 1 --2- 172.21.15.49:0/2384990175 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8724161650 0x7f8724159280 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:05.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.094+0000 7f8728c8c640 1 --2- 172.21.15.49:0/2384990175 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f87240d2190 0x7f87240cc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:05.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.094+0000 7f8728c8c640 1 --2- 172.21.15.49:0/2384990175 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f87240b1f90 0x7f8724150c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:05.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.094+0000 7f8728c8c640 1 -- 172.21.15.49:0/2384990175 >> 172.21.15.49:0/2384990175 conn(0x7f87240d2aa0 msgr2=0x7f872415bb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:05.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.094+0000 7f8728c8c640 1 -- 172.21.15.49:0/2384990175 shutdown_connections 2023-11-06T02:14:05.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.094+0000 7f8728c8c640 1 -- 172.21.15.49:0/2384990175 wait complete. 2023-11-06T02:14:05.107 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T02:08:19.252222+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T02:08:24.776211+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T02:13:31.459314+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T02:13:32.603710+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T02:14:05.108 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T02:14:05.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.238+0000 7fbcd883b640 1 Processor -- start 2023-11-06T02:14:05.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.238+0000 7fbcd883b640 1 -- start start 2023-11-06T02:14:05.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.242+0000 7fbcd883b640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fbcd415a990 0x7fbcd415dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:05.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.242+0000 7fbcd883b640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fbcd40b0220 0x7fbcd40b0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:05.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.242+0000 7fbcd883b640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcd406c5a0 con 0x7fbcd41595a0 2023-11-06T02:14:05.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.242+0000 7fbcd883b640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcd415e250 con 0x7fbcd415a990 2023-11-06T02:14:05.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.242+0000 7fbcd883b640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcd406c880 con 0x7fbcd40b0220 2023-11-06T02:14:05.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.242+0000 7fbccbfff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fbcd415a990 0x7fbcd415dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:05.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.242+0000 7fbccbfff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fbcd415a990 0x7fbcd415dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51676/0 (socket says 172.21.15.49:51676) 2023-11-06T02:14:05.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.242+0000 7fbccbfff640 1 -- 172.21.15.49:0/1147479198 learned_addr learned my addr 172.21.15.49:0/1147479198 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:05.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.242+0000 7fbcd2d9d640 1 --2- 172.21.15.49:0/1147479198 >> v2:172.21.15.150:3300/0 conn(0x7fbcd40b0220 0x7fbcd40b0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:05.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.242+0000 7fbccbfff640 1 -- 172.21.15.49:0/1147479198 >> v2:172.21.15.150:3300/0 conn(0x7fbcd40b0220 msgr2=0x7fbcd40b0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:05.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.242+0000 7fbccbfff640 1 --2- 172.21.15.49:0/1147479198 >> v2:172.21.15.150:3300/0 conn(0x7fbcd40b0220 0x7fbcd40b0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:05.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.242+0000 7fbccbfff640 1 -- 172.21.15.49:0/1147479198 >> v1:172.21.15.150:6789/0 conn(0x7fbcd41595a0 legacy=0x7fbcd406d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:05.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.242+0000 7fbccbfff640 1 -- 172.21.15.49:0/1147479198 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcd406cb60 con 0x7fbcd415a990 2023-11-06T02:14:05.243 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.242+0000 7fbccbfff640 1 --2- 172.21.15.49:0/1147479198 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbcd415a990 0x7fbcd415dd10 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fbcd40afa80 tx=0x7fbcbc01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90f441fce461ea5d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:05.243 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.242+0000 7fbcd1d9b640 1 -- 172.21.15.49:0/1147479198 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcbc0aa430 con 0x7fbcd415a990 2023-11-06T02:14:05.243 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.242+0000 7fbcd1d9b640 1 -- 172.21.15.49:0/1147479198 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcbc0aa590 con 0x7fbcd415a990 2023-11-06T02:14:05.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.242+0000 7fbcd1d9b640 1 -- 172.21.15.49:0/1147479198 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcbc0aa920 con 0x7fbcd415a990 2023-11-06T02:14:05.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.242+0000 7fbcd883b640 1 -- 172.21.15.49:0/1147479198 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbcd415a990 msgr2=0x7fbcd415dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:05.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.242+0000 7fbcd883b640 1 --2- 172.21.15.49:0/1147479198 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbcd415a990 0x7fbcd415dd10 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fbcd40afa80 tx=0x7fbcbc01c4d0 comp rx=0 tx=0).stop 2023-11-06T02:14:05.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.242+0000 7fbcd883b640 1 -- 172.21.15.49:0/1147479198 shutdown_connections 2023-11-06T02:14:05.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.242+0000 7fbcd883b640 1 --2- 172.21.15.49:0/1147479198 >> v2:172.21.15.150:3300/0 conn(0x7fbcd40b0220 0x7fbcd40b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:05.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.242+0000 7fbcd883b640 1 --2- 172.21.15.49:0/1147479198 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbcd415a990 0x7fbcd415dd10 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:05.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.242+0000 7fbcd883b640 1 -- 172.21.15.49:0/1147479198 >> 172.21.15.49:0/1147479198 conn(0x7fbcd406a370 msgr2=0x7fbcd40ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:05.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.242+0000 7fbcd883b640 1 -- 172.21.15.49:0/1147479198 shutdown_connections 2023-11-06T02:14:05.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.242+0000 7fbcd883b640 1 -- 172.21.15.49:0/1147479198 wait complete. 2023-11-06T02:14:05.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.246+0000 7fbcd883b640 1 Processor -- start 2023-11-06T02:14:05.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.246+0000 7fbcd883b640 1 -- start start 2023-11-06T02:14:05.246 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.246+0000 7fbcd883b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbcd40b0220 0x7fbcd40b13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:05.246 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.246+0000 7fbcd883b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbcd41595a0 0x7fbcd40ca100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:05.246 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.246+0000 7fbcd883b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbcd415a990 0x7fbcd414a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:05.246 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.246+0000 7fbcd883b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd4150760 con 0x7fbcd40b0220 2023-11-06T02:14:05.246 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.246+0000 7fbcd883b640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd41502d0 con 0x7fbcd415a990 2023-11-06T02:14:05.247 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.246+0000 7fbcd883b640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd414a420 con 0x7fbcd41595a0 2023-11-06T02:14:05.247 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.246+0000 7fbcd259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbcd40b0220 0x7fbcd40b13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:05.247 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.246+0000 7fbcd259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbcd40b0220 0x7fbcd40b13b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51682/0 (socket says 172.21.15.49:51682) 2023-11-06T02:14:05.247 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.246+0000 7fbcd259c640 1 -- 172.21.15.49:0/1825516508 learned_addr learned my addr 172.21.15.49:0/1825516508 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:05.247 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.246+0000 7fbccbfff640 1 --2- 172.21.15.49:0/1825516508 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbcd41595a0 0x7fbcd40ca100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:05.248 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.246+0000 7fbcd2d9d640 1 --2- 172.21.15.49:0/1825516508 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbcd415a990 0x7fbcd414a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:05.248 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.246+0000 7fbcd259c640 1 -- 172.21.15.49:0/1825516508 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbcd41595a0 msgr2=0x7fbcd40ca100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:05.248 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.246+0000 7fbcd259c640 1 --2- 172.21.15.49:0/1825516508 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbcd41595a0 0x7fbcd40ca100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:05.248 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.246+0000 7fbcd259c640 1 -- 172.21.15.49:0/1825516508 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbcd415a990 msgr2=0x7fbcd414a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:05.248 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.246+0000 7fbcd259c640 1 --2- 172.21.15.49:0/1825516508 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbcd415a990 0x7fbcd414a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:05.248 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.246+0000 7fbcd259c640 1 -- 172.21.15.49:0/1825516508 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcbc0094d0 con 0x7fbcd40b0220 2023-11-06T02:14:05.249 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.246+0000 7fbcd2d9d640 1 --2- 172.21.15.49:0/1825516508 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbcd415a990 0x7fbcd414a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:14:05.249 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.246+0000 7fbccbfff640 1 --2- 172.21.15.49:0/1825516508 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbcd41595a0 0x7fbcd40ca100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:14:05.249 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.246+0000 7fbcd259c640 1 --2- 172.21.15.49:0/1825516508 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbcd40b0220 0x7fbcd40b13b0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7fbcc400c530 tx=0x7fbcc400c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:05.249 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.246+0000 7fbccaffd640 1 -- 172.21.15.49:0/1825516508 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc4005220 con 0x7fbcd40b0220 2023-11-06T02:14:05.249 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.246+0000 7fbccaffd640 1 -- 172.21.15.49:0/1825516508 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcc4005380 con 0x7fbcd40b0220 2023-11-06T02:14:05.250 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.246+0000 7fbcd883b640 1 -- 172.21.15.49:0/1825516508 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcd414ffa0 con 0x7fbcd40b0220 2023-11-06T02:14:05.250 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.246+0000 7fbccaffd640 1 -- 172.21.15.49:0/1825516508 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc4007270 con 0x7fbcd40b0220 2023-11-06T02:14:05.250 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.246+0000 7fbcd883b640 1 -- 172.21.15.49:0/1825516508 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcd40b2130 con 0x7fbcd40b0220 2023-11-06T02:14:05.250 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.246+0000 7fbcd883b640 1 -- 172.21.15.49:0/1825516508 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcd406c210 con 0x7fbcd40b0220 2023-11-06T02:14:05.250 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.246+0000 7fbccaffd640 1 -- 172.21.15.49:0/1825516508 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fbcc40073d0 con 0x7fbcd40b0220 2023-11-06T02:14:05.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.250+0000 7fbccaffd640 1 --2- 172.21.15.49:0/1825516508 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fbcb4065a60 0x7fbcb4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:05.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.250+0000 7fbccaffd640 1 -- 172.21.15.49:0/1825516508 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbcc4082b10 con 0x7fbcd40b0220 2023-11-06T02:14:05.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.250+0000 7fbccbfff640 1 --2- 172.21.15.49:0/1825516508 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fbcb4065a60 0x7fbcb4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:05.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.250+0000 7fbccbfff640 1 --2- 172.21.15.49:0/1825516508 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fbcb4065a60 0x7fbcb4067f20 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc0ae000 tx=0x7fbcbc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:05.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.254+0000 7fbccaffd640 1 -- 172.21.15.49:0/1825516508 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fbcd406c210 con 0x7fbcd40b0220 2023-11-06T02:14:05.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.410+0000 7fbcd883b640 1 -- 172.21.15.49:0/1825516508 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbcd406c210 con 0x7fbcd40b0220 2023-11-06T02:14:05.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.410+0000 7fbccaffd640 1 -- 172.21.15.49:0/1825516508 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbcd406c210 con 0x7fbcd40b0220 2023-11-06T02:14:05.406 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2023-11-06T02:14:05.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.414+0000 7fbcd883b640 1 -- 172.21.15.49:0/1825516508 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fbcb4065a60 msgr2=0x7fbcb4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:05.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.414+0000 7fbcd883b640 1 --2- 172.21.15.49:0/1825516508 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fbcb4065a60 0x7fbcb4067f20 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc0ae000 tx=0x7fbcbc008040 comp rx=0 tx=0).stop 2023-11-06T02:14:05.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.414+0000 7fbcd883b640 1 -- 172.21.15.49:0/1825516508 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbcd40b0220 msgr2=0x7fbcd40b13b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:05.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.414+0000 7fbcd883b640 1 --2- 172.21.15.49:0/1825516508 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbcd40b0220 0x7fbcd40b13b0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7fbcc400c530 tx=0x7fbcc400c9f0 comp rx=0 tx=0).stop 2023-11-06T02:14:05.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.414+0000 7fbcd883b640 1 -- 172.21.15.49:0/1825516508 shutdown_connections 2023-11-06T02:14:05.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.414+0000 7fbcd883b640 1 --2- 172.21.15.49:0/1825516508 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fbcb4065a60 0x7fbcb4067f20 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:05.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.414+0000 7fbcd883b640 1 --2- 172.21.15.49:0/1825516508 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbcd415a990 0x7fbcd414a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:05.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.414+0000 7fbcd883b640 1 --2- 172.21.15.49:0/1825516508 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbcd41595a0 0x7fbcd40ca100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:05.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.414+0000 7fbcd883b640 1 --2- 172.21.15.49:0/1825516508 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbcd40b0220 0x7fbcd40b13b0 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:05.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.414+0000 7fbcd883b640 1 -- 172.21.15.49:0/1825516508 >> 172.21.15.49:0/1825516508 conn(0x7fbcd406a370 msgr2=0x7fbcd4152690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:05.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.414+0000 7fbcd883b640 1 -- 172.21.15.49:0/1825516508 shutdown_connections 2023-11-06T02:14:05.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.414+0000 7fbcd883b640 1 -- 172.21.15.49:0/1825516508 wait complete. 2023-11-06T02:14:05.424 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T02:14:05.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.554+0000 7fbd0e772640 1 Processor -- start 2023-11-06T02:14:05.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.554+0000 7fbd0e772640 1 -- start start 2023-11-06T02:14:05.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.554+0000 7fbd0e772640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fbd0815a880 0x7fbd080b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:05.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.554+0000 7fbd0e772640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fbd0806d860 0x7fbd08070610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:05.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.554+0000 7fbd0e772640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd080ae360 con 0x7fbd08070160 2023-11-06T02:14:05.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.554+0000 7fbd0e772640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd08070b50 con 0x7fbd0806d860 2023-11-06T02:14:05.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.554+0000 7fbd0e772640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd080ae640 con 0x7fbd0815a880 2023-11-06T02:14:05.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.554+0000 7fbd0cd0f640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fbd0806d860 0x7fbd08070610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:05.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.554+0000 7fbd077fe640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fbd08070160 0x7fbd0806d150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:59330/0 (socket says 172.21.15.49:59330) 2023-11-06T02:14:05.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.554+0000 7fbd077fe640 1 -- 172.21.15.49:0/2847519972 learned_addr learned my addr 172.21.15.49:0/2847519972 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:14:05.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.554+0000 7fbd07fff640 1 --2- 172.21.15.49:0/2847519972 >> v2:172.21.15.150:3300/0 conn(0x7fbd0815a880 0x7fbd080b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:05.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.554+0000 7fbd0cd0f640 1 -- 172.21.15.49:0/2847519972 >> v2:172.21.15.150:3300/0 conn(0x7fbd0815a880 msgr2=0x7fbd080b15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:05.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.554+0000 7fbd0cd0f640 1 --2- 172.21.15.49:0/2847519972 >> v2:172.21.15.150:3300/0 conn(0x7fbd0815a880 0x7fbd080b15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:05.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.554+0000 7fbd0cd0f640 1 -- 172.21.15.49:0/2847519972 >> v1:172.21.15.49:6789/0 conn(0x7fbd08070160 legacy=0x7fbd0806d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:05.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.554+0000 7fbd0cd0f640 1 -- 172.21.15.49:0/2847519972 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd080ae920 con 0x7fbd0806d860 2023-11-06T02:14:05.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.554+0000 7fbd07fff640 1 --2- 172.21.15.49:0/2847519972 >> v2:172.21.15.150:3300/0 conn(0x7fbd0815a880 0x7fbd080b15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:14:05.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.558+0000 7fbd0cd0f640 1 --2- 172.21.15.49:0/2847519972 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd0806d860 0x7fbd08070610 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fbd080cafc0 tx=0x7fbd0001c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9246a1e56d459e6d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:05.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.558+0000 7fbd06ffd640 1 -- 172.21.15.49:0/2847519972 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd000a58e0 con 0x7fbd0806d860 2023-11-06T02:14:05.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.558+0000 7fbd06ffd640 1 -- 172.21.15.49:0/2847519972 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd000a5a40 con 0x7fbd0806d860 2023-11-06T02:14:05.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.558+0000 7fbd06ffd640 1 -- 172.21.15.49:0/2847519972 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd000a5dd0 con 0x7fbd0806d860 2023-11-06T02:14:05.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.558+0000 7fbd0e772640 1 -- 172.21.15.49:0/2847519972 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd0806d860 msgr2=0x7fbd08070610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:05.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.558+0000 7fbd0e772640 1 --2- 172.21.15.49:0/2847519972 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd0806d860 0x7fbd08070610 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fbd080cafc0 tx=0x7fbd0001c910 comp rx=0 tx=0).stop 2023-11-06T02:14:05.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.558+0000 7fbd0e772640 1 -- 172.21.15.49:0/2847519972 shutdown_connections 2023-11-06T02:14:05.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.558+0000 7fbd0e772640 1 --2- 172.21.15.49:0/2847519972 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd0806d860 0x7fbd08070610 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:05.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.558+0000 7fbd0e772640 1 --2- 172.21.15.49:0/2847519972 >> v2:172.21.15.150:3300/0 conn(0x7fbd0815a880 0x7fbd080b15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:05.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.558+0000 7fbd0e772640 1 -- 172.21.15.49:0/2847519972 >> 172.21.15.49:0/2847519972 conn(0x7fbd0806a370 msgr2=0x7fbd080ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:05.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.558+0000 7fbd0e772640 1 -- 172.21.15.49:0/2847519972 shutdown_connections 2023-11-06T02:14:05.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.558+0000 7fbd0e772640 1 -- 172.21.15.49:0/2847519972 wait complete. 2023-11-06T02:14:05.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.558+0000 7fbd0e772640 1 Processor -- start 2023-11-06T02:14:05.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.558+0000 7fbd0e772640 1 -- start start 2023-11-06T02:14:05.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.558+0000 7fbd0e772640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd0806d860 0x7fbd080ca5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:05.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.558+0000 7fbd0e772640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbd08070160 0x7fbd080c7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:05.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.558+0000 7fbd0e772640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbd0815a880 0x7fbd080c7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:05.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.558+0000 7fbd0e772640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd0815c3b0 con 0x7fbd0806d860 2023-11-06T02:14:05.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.558+0000 7fbd0e772640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd0815c650 con 0x7fbd0815a880 2023-11-06T02:14:05.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.558+0000 7fbd0e772640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd0815c930 con 0x7fbd08070160 2023-11-06T02:14:05.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.558+0000 7fbd07fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd0806d860 0x7fbd080ca5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:05.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.558+0000 7fbd07fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd0806d860 0x7fbd080ca5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51698/0 (socket says 172.21.15.49:51698) 2023-11-06T02:14:05.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.558+0000 7fbd07fff640 1 -- 172.21.15.49:0/690152903 learned_addr learned my addr 172.21.15.49:0/690152903 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:05.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.558+0000 7fbd077fe640 1 --2- 172.21.15.49:0/690152903 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbd08070160 0x7fbd080c7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:05.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.558+0000 7fbd0cd0f640 1 --2- 172.21.15.49:0/690152903 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbd0815a880 0x7fbd080c7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:05.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.558+0000 7fbd07fff640 1 -- 172.21.15.49:0/690152903 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbd08070160 msgr2=0x7fbd080c7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:05.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.558+0000 7fbd07fff640 1 --2- 172.21.15.49:0/690152903 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbd08070160 0x7fbd080c7970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:05.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.558+0000 7fbd07fff640 1 -- 172.21.15.49:0/690152903 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbd0815a880 msgr2=0x7fbd080c7eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:05.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.558+0000 7fbd07fff640 1 --2- 172.21.15.49:0/690152903 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbd0815a880 0x7fbd080c7eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:05.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.558+0000 7fbd07fff640 1 -- 172.21.15.49:0/690152903 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd00009a70 con 0x7fbd0806d860 2023-11-06T02:14:05.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.562+0000 7fbd07fff640 1 --2- 172.21.15.49:0/690152903 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd0806d860 0x7fbd080ca5f0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc00c7f0 tx=0x7fbcfc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:05.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.562+0000 7fbd04ff9640 1 -- 172.21.15.49:0/690152903 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcfc007820 con 0x7fbd0806d860 2023-11-06T02:14:05.563 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.562+0000 7fbd04ff9640 1 -- 172.21.15.49:0/690152903 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcfc007980 con 0x7fbd0806d860 2023-11-06T02:14:05.563 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.562+0000 7fbd0e772640 1 -- 172.21.15.49:0/690152903 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd0815cc10 con 0x7fbd0806d860 2023-11-06T02:14:05.563 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.562+0000 7fbd04ff9640 1 -- 172.21.15.49:0/690152903 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcfc007d30 con 0x7fbd0806d860 2023-11-06T02:14:05.563 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.562+0000 7fbd0e772640 1 -- 172.21.15.49:0/690152903 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd080cab30 con 0x7fbd0806d860 2023-11-06T02:14:05.563 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.562+0000 7fbd0e772640 1 -- 172.21.15.49:0/690152903 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd080cad80 con 0x7fbd0806d860 2023-11-06T02:14:05.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.562+0000 7fbd04ff9640 1 -- 172.21.15.49:0/690152903 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fbcfc0113f0 con 0x7fbd0806d860 2023-11-06T02:14:05.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.562+0000 7fbd04ff9640 1 --2- 172.21.15.49:0/690152903 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fbcf4065a90 0x7fbcf4067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:05.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.562+0000 7fbd04ff9640 1 -- 172.21.15.49:0/690152903 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbcfc083400 con 0x7fbd0806d860 2023-11-06T02:14:05.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.562+0000 7fbd077fe640 1 --2- 172.21.15.49:0/690152903 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fbcf4065a90 0x7fbcf4067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:05.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.562+0000 7fbd077fe640 1 --2- 172.21.15.49:0/690152903 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fbcf4065a90 0x7fbcf4067f50 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fbd08071090 tx=0x7fbcf8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:05.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.566+0000 7fbd04ff9640 1 -- 172.21.15.49:0/690152903 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fbcfc050c60 con 0x7fbd0806d860 2023-11-06T02:14:05.722 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.726+0000 7fbd0e772640 1 -- 172.21.15.49:0/690152903 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fbd08074040 con 0x7fbd0806d860 2023-11-06T02:14:05.723 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.726+0000 7fbd04ff9640 1 -- 172.21.15.49:0/690152903 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v80) v1 ==== 92+0+10 (secure 0 0 0) 0x7fbcfc054280 con 0x7fbd0806d860 2023-11-06T02:14:05.723 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 8 2023-11-06T02:14:05.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.730+0000 7fbd0e772640 1 -- 172.21.15.49:0/690152903 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fbcf4065a90 msgr2=0x7fbcf4067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:05.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.730+0000 7fbd0e772640 1 --2- 172.21.15.49:0/690152903 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fbcf4065a90 0x7fbcf4067f50 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fbd08071090 tx=0x7fbcf8009040 comp rx=0 tx=0).stop 2023-11-06T02:14:05.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.730+0000 7fbd0e772640 1 -- 172.21.15.49:0/690152903 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd0806d860 msgr2=0x7fbd080ca5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:05.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.730+0000 7fbd0e772640 1 --2- 172.21.15.49:0/690152903 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd0806d860 0x7fbd080ca5f0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc00c7f0 tx=0x7fbcfc00ccb0 comp rx=0 tx=0).stop 2023-11-06T02:14:05.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.730+0000 7fbd0e772640 1 -- 172.21.15.49:0/690152903 shutdown_connections 2023-11-06T02:14:05.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.730+0000 7fbd0e772640 1 --2- 172.21.15.49:0/690152903 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fbcf4065a90 0x7fbcf4067f50 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:05.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.730+0000 7fbd0e772640 1 --2- 172.21.15.49:0/690152903 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbd0815a880 0x7fbd080c7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:05.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.730+0000 7fbd0e772640 1 --2- 172.21.15.49:0/690152903 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbd08070160 0x7fbd080c7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:05.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.730+0000 7fbd0e772640 1 --2- 172.21.15.49:0/690152903 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd0806d860 0x7fbd080ca5f0 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:05.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.730+0000 7fbd0e772640 1 -- 172.21.15.49:0/690152903 >> 172.21.15.49:0/690152903 conn(0x7fbd0806a370 msgr2=0x7fbd080b3ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:05.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.730+0000 7fbd0e772640 1 -- 172.21.15.49:0/690152903 shutdown_connections 2023-11-06T02:14:05.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.730+0000 7fbd0e772640 1 -- 172.21.15.49:0/690152903 wait complete. 2023-11-06T02:14:05.741 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T02:14:05.856 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.862+0000 7f9105223640 1 Processor -- start 2023-11-06T02:14:05.857 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.862+0000 7f9105223640 1 -- start start 2023-11-06T02:14:05.859 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.862+0000 7f9105223640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f91001595a0 0x7f910015dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:05.860 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.862+0000 7f9105223640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f910006d640 0x7f91000b0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:05.860 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.862+0000 7f9105223640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f91000b0760 0x7f91000c8e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:05.860 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.862+0000 7f9105223640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f910015e250 con 0x7f91000b0760 2023-11-06T02:14:05.861 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.862+0000 7f9105223640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f910006c5a0 con 0x7f910006d640 2023-11-06T02:14:05.861 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.862+0000 7f9105223640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f910006c880 con 0x7f91001595a0 2023-11-06T02:14:05.861 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.862+0000 7f90ff59e640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f91000b0760 0x7f91000c8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:05.862 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.862+0000 7f90ff59e640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f91000b0760 0x7f91000c8e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51708/0 (socket says 172.21.15.49:51708) 2023-11-06T02:14:05.862 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.862+0000 7f90ff59e640 1 -- 172.21.15.49:0/3070691986 learned_addr learned my addr 172.21.15.49:0/3070691986 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:05.862 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.862+0000 7f90fe59c640 1 --2- 172.21.15.49:0/3070691986 >> v2:172.21.15.150:3300/0 conn(0x7f910006d640 0x7f91000b0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:05.863 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.862+0000 7f90fed9d640 1 --2- 172.21.15.49:0/3070691986 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f91001595a0 0x7f910015dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:05.863 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.862+0000 7f90ff59e640 1 -- 172.21.15.49:0/3070691986 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f91001595a0 msgr2=0x7f910015dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:05.864 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.862+0000 7f90ff59e640 1 --2- 172.21.15.49:0/3070691986 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f91001595a0 0x7f910015dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:05.864 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.862+0000 7f90ff59e640 1 -- 172.21.15.49:0/3070691986 >> v2:172.21.15.150:3300/0 conn(0x7f910006d640 msgr2=0x7f91000b0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:05.864 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.862+0000 7f90ff59e640 1 --2- 172.21.15.49:0/3070691986 >> v2:172.21.15.150:3300/0 conn(0x7f910006d640 0x7f91000b0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:05.865 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.862+0000 7f90ff59e640 1 -- 172.21.15.49:0/3070691986 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f910006cb60 con 0x7f91000b0760 2023-11-06T02:14:05.865 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.862+0000 7f90fe59c640 1 --2- 172.21.15.49:0/3070691986 >> v2:172.21.15.150:3300/0 conn(0x7f910006d640 0x7f91000b0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:14:05.871 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.866+0000 7f90ff59e640 1 --2- 172.21.15.49:0/3070691986 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f91000b0760 0x7f91000c8e90 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f9100071540 tx=0x7f90f801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8c9c337693b4367 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:05.871 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.866+0000 7f90fdd9b640 1 -- 172.21.15.49:0/3070691986 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f80ab650 con 0x7f91000b0760 2023-11-06T02:14:05.872 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.866+0000 7f90fdd9b640 1 -- 172.21.15.49:0/3070691986 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90f80ab7b0 con 0x7f91000b0760 2023-11-06T02:14:05.872 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.866+0000 7f90fdd9b640 1 -- 172.21.15.49:0/3070691986 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f80abb40 con 0x7f91000b0760 2023-11-06T02:14:05.872 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.866+0000 7f9105223640 1 -- 172.21.15.49:0/3070691986 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f91000b0760 msgr2=0x7f91000c8e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:05.872 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.866+0000 7f9105223640 1 --2- 172.21.15.49:0/3070691986 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f91000b0760 0x7f91000c8e90 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f9100071540 tx=0x7f90f801c4d0 comp rx=0 tx=0).stop 2023-11-06T02:14:05.872 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.866+0000 7f9105223640 1 -- 172.21.15.49:0/3070691986 shutdown_connections 2023-11-06T02:14:05.873 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.866+0000 7f9105223640 1 --2- 172.21.15.49:0/3070691986 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f91000b0760 0x7f91000c8e90 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:05.873 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.866+0000 7f9105223640 1 --2- 172.21.15.49:0/3070691986 >> v2:172.21.15.150:3300/0 conn(0x7f910006d640 0x7f91000b0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:05.873 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.866+0000 7f9105223640 1 --2- 172.21.15.49:0/3070691986 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f91001595a0 0x7f910015dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:05.873 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.866+0000 7f9105223640 1 -- 172.21.15.49:0/3070691986 >> 172.21.15.49:0/3070691986 conn(0x7f910006a370 msgr2=0x7f91000ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:05.873 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.866+0000 7f9105223640 1 -- 172.21.15.49:0/3070691986 shutdown_connections 2023-11-06T02:14:05.874 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.866+0000 7f9105223640 1 -- 172.21.15.49:0/3070691986 wait complete. 2023-11-06T02:14:05.874 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.866+0000 7f9105223640 1 Processor -- start 2023-11-06T02:14:05.874 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.866+0000 7f9105223640 1 -- start start 2023-11-06T02:14:05.874 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.866+0000 7f9105223640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f910006d640 0x7f91000cd000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:05.874 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.866+0000 7f9105223640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f91000b0760 0x7f9100150060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:05.875 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.866+0000 7f9105223640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f91001595a0 0x7f910014b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:05.875 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.866+0000 7f9105223640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f910014c1f0 con 0x7f910006d640 2023-11-06T02:14:05.875 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.866+0000 7f9105223640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91001f81d0 con 0x7f91001595a0 2023-11-06T02:14:05.875 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.866+0000 7f9105223640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91000cc480 con 0x7f91000b0760 2023-11-06T02:14:05.875 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.866+0000 7f90fed9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f910006d640 0x7f91000cd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:05.876 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.866+0000 7f90fed9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f910006d640 0x7f91000cd000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51720/0 (socket says 172.21.15.49:51720) 2023-11-06T02:14:05.876 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.866+0000 7f90fed9d640 1 -- 172.21.15.49:0/2954878658 learned_addr learned my addr 172.21.15.49:0/2954878658 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:05.876 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.866+0000 7f90ff59e640 1 --2- 172.21.15.49:0/2954878658 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f91001595a0 0x7f910014b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:05.877 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.866+0000 7f90fe59c640 1 --2- 172.21.15.49:0/2954878658 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f91000b0760 0x7f9100150060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:05.877 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.866+0000 7f90fed9d640 1 -- 172.21.15.49:0/2954878658 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f91000b0760 msgr2=0x7f9100150060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:05.878 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.866+0000 7f90fed9d640 1 --2- 172.21.15.49:0/2954878658 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f91000b0760 0x7f9100150060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:05.878 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.866+0000 7f90fed9d640 1 -- 172.21.15.49:0/2954878658 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f91001595a0 msgr2=0x7f910014b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:05.878 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.866+0000 7f90fed9d640 1 --2- 172.21.15.49:0/2954878658 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f91001595a0 0x7f910014b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:05.879 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.866+0000 7f90fed9d640 1 -- 172.21.15.49:0/2954878658 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90f80094d0 con 0x7f910006d640 2023-11-06T02:14:05.879 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.866+0000 7f90ff59e640 1 --2- 172.21.15.49:0/2954878658 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f91001595a0 0x7f910014b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:14:05.879 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.866+0000 7f90fed9d640 1 --2- 172.21.15.49:0/2954878658 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f910006d640 0x7f91000cd000 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f90f000cd00 tx=0x7f90f000a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:05.880 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.870+0000 7f90f77fe640 1 -- 172.21.15.49:0/2954878658 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f00039a0 con 0x7f910006d640 2023-11-06T02:14:05.880 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.870+0000 7f90f77fe640 1 -- 172.21.15.49:0/2954878658 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90f0003b00 con 0x7f910006d640 2023-11-06T02:14:05.880 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.870+0000 7f9105223640 1 -- 172.21.15.49:0/2954878658 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f910014c440 con 0x7f910006d640 2023-11-06T02:14:05.881 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.870+0000 7f90f77fe640 1 -- 172.21.15.49:0/2954878658 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f0003e90 con 0x7f910006d640 2023-11-06T02:14:05.881 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.870+0000 7f9105223640 1 -- 172.21.15.49:0/2954878658 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91001f8420 con 0x7f910006d640 2023-11-06T02:14:05.881 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.870+0000 7f9105223640 1 -- 172.21.15.49:0/2954878658 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f910006c210 con 0x7f910006d640 2023-11-06T02:14:05.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.870+0000 7f90f77fe640 1 -- 172.21.15.49:0/2954878658 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f90f0004020 con 0x7f910006d640 2023-11-06T02:14:05.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.870+0000 7f90f77fe640 1 --2- 172.21.15.49:0/2954878658 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f90e0065a60 0x7f90e0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:05.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.870+0000 7f90f77fe640 1 -- 172.21.15.49:0/2954878658 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f90f00835a0 con 0x7f910006d640 2023-11-06T02:14:05.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.870+0000 7f90fe59c640 1 --2- 172.21.15.49:0/2954878658 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f90e0065a60 0x7f90e0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:05.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.870+0000 7f90fe59c640 1 --2- 172.21.15.49:0/2954878658 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f90e0065a60 0x7f90e0067f20 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f90e8006fd0 tx=0x7f90e8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:05.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:05.874+0000 7f90f77fe640 1 -- 172.21.15.49:0/2954878658 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f910006c210 con 0x7f910006d640 2023-11-06T02:14:06.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.022+0000 7f9105223640 1 -- 172.21.15.49:0/2954878658 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f910006c210 con 0x7f910006d640 2023-11-06T02:14:06.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.022+0000 7f90f77fe640 1 -- 172.21.15.49:0/2954878658 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v80) v1 ==== 104+0+11 (secure 0 0 0) 0x7f90f001d020 con 0x7f910006d640 2023-11-06T02:14:06.017 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2023-11-06T02:14:06.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.022+0000 7f9105223640 1 -- 172.21.15.49:0/2954878658 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f90e0065a60 msgr2=0x7f90e0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:06.019 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.022+0000 7f9105223640 1 --2- 172.21.15.49:0/2954878658 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f90e0065a60 0x7f90e0067f20 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f90e8006fd0 tx=0x7f90e8008040 comp rx=0 tx=0).stop 2023-11-06T02:14:06.019 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.022+0000 7f9105223640 1 -- 172.21.15.49:0/2954878658 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f910006d640 msgr2=0x7f91000cd000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:06.019 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.022+0000 7f9105223640 1 --2- 172.21.15.49:0/2954878658 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f910006d640 0x7f91000cd000 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f90f000cd00 tx=0x7f90f000a5c0 comp rx=0 tx=0).stop 2023-11-06T02:14:06.019 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.022+0000 7f9105223640 1 -- 172.21.15.49:0/2954878658 shutdown_connections 2023-11-06T02:14:06.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.022+0000 7f9105223640 1 --2- 172.21.15.49:0/2954878658 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f90e0065a60 0x7f90e0067f20 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:06.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.022+0000 7f9105223640 1 --2- 172.21.15.49:0/2954878658 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f91001595a0 0x7f910014b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:06.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.022+0000 7f9105223640 1 --2- 172.21.15.49:0/2954878658 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f91000b0760 0x7f9100150060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:06.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.022+0000 7f9105223640 1 --2- 172.21.15.49:0/2954878658 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f910006d640 0x7f91000cd000 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:06.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.022+0000 7f9105223640 1 -- 172.21.15.49:0/2954878658 >> 172.21.15.49:0/2954878658 conn(0x7f910006a370 msgr2=0x7f9100072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:06.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.022+0000 7f9105223640 1 -- 172.21.15.49:0/2954878658 shutdown_connections 2023-11-06T02:14:06.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.022+0000 7f9105223640 1 -- 172.21.15.49:0/2954878658 wait complete. 2023-11-06T02:14:06.034 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T02:14:06.159 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.162+0000 7fbe3a789640 1 Processor -- start 2023-11-06T02:14:06.159 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.162+0000 7fbe3a789640 1 -- start start 2023-11-06T02:14:06.160 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.166+0000 7fbe3a789640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fbe3415a820 0x7fbe340c9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:06.161 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.166+0000 7fbe3a789640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fbe340cdae0 0x7fbe340cdf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:06.161 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.166+0000 7fbe3a789640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbe34069d70 0x7fbe3406a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:06.161 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.166+0000 7fbe3a789640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe3415dec0 con 0x7fbe340cdae0 2023-11-06T02:14:06.161 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.166+0000 7fbe3a789640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe3406a740 con 0x7fbe3415a820 2023-11-06T02:14:06.161 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.166+0000 7fbe3a789640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3415d240 con 0x7fbe34069d70 2023-11-06T02:14:06.162 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.166+0000 7fbe337fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fbe340cdae0 0x7fbe340cdf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:06.162 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.166+0000 7fbe337fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fbe340cdae0 0x7fbe340cdf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51726/0 (socket says 172.21.15.49:51726) 2023-11-06T02:14:06.162 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.166+0000 7fbe337fe640 1 -- 172.21.15.49:0/1285063439 learned_addr learned my addr 172.21.15.49:0/1285063439 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:06.162 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.166+0000 7fbe38d26640 1 --2- 172.21.15.49:0/1285063439 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbe34069d70 0x7fbe3406a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:06.163 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.166+0000 7fbe33fff640 1 --2- 172.21.15.49:0/1285063439 >> v2:172.21.15.150:3300/0 conn(0x7fbe3415a820 0x7fbe340c9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:06.163 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.166+0000 7fbe337fe640 1 -- 172.21.15.49:0/1285063439 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbe34069d70 msgr2=0x7fbe3406a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:06.163 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.166+0000 7fbe337fe640 1 --2- 172.21.15.49:0/1285063439 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbe34069d70 0x7fbe3406a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:06.163 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.166+0000 7fbe337fe640 1 -- 172.21.15.49:0/1285063439 >> v2:172.21.15.150:3300/0 conn(0x7fbe3415a820 msgr2=0x7fbe340c9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:06.163 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.166+0000 7fbe337fe640 1 --2- 172.21.15.49:0/1285063439 >> v2:172.21.15.150:3300/0 conn(0x7fbe3415a820 0x7fbe340c9fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:06.163 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.166+0000 7fbe337fe640 1 -- 172.21.15.49:0/1285063439 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe3415d520 con 0x7fbe340cdae0 2023-11-06T02:14:06.164 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.166+0000 7fbe38d26640 1 --2- 172.21.15.49:0/1285063439 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbe34069d70 0x7fbe3406a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:14:06.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.166+0000 7fbe337fe640 1 --2- 172.21.15.49:0/1285063439 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe340cdae0 0x7fbe340cdf20 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fbe340d0a00 tx=0x7fbe2801c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e733726a286f0fee server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:06.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.166+0000 7fbe32ffd640 1 -- 172.21.15.49:0/1285063439 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe280a5730 con 0x7fbe340cdae0 2023-11-06T02:14:06.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.166+0000 7fbe32ffd640 1 -- 172.21.15.49:0/1285063439 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe280a5890 con 0x7fbe340cdae0 2023-11-06T02:14:06.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.166+0000 7fbe32ffd640 1 -- 172.21.15.49:0/1285063439 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe280a5c20 con 0x7fbe340cdae0 2023-11-06T02:14:06.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.170+0000 7fbe3a789640 1 -- 172.21.15.49:0/1285063439 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe340cdae0 msgr2=0x7fbe340cdf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:06.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.170+0000 7fbe3a789640 1 --2- 172.21.15.49:0/1285063439 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe340cdae0 0x7fbe340cdf20 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fbe340d0a00 tx=0x7fbe2801c8b0 comp rx=0 tx=0).stop 2023-11-06T02:14:06.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.170+0000 7fbe3a789640 1 -- 172.21.15.49:0/1285063439 shutdown_connections 2023-11-06T02:14:06.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.170+0000 7fbe3a789640 1 --2- 172.21.15.49:0/1285063439 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbe34069d70 0x7fbe3406a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:06.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.170+0000 7fbe3a789640 1 --2- 172.21.15.49:0/1285063439 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe340cdae0 0x7fbe340cdf20 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:06.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.170+0000 7fbe3a789640 1 --2- 172.21.15.49:0/1285063439 >> v2:172.21.15.150:3300/0 conn(0x7fbe3415a820 0x7fbe340c9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:06.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.170+0000 7fbe3a789640 1 -- 172.21.15.49:0/1285063439 >> 172.21.15.49:0/1285063439 conn(0x7fbe3406d4b0 msgr2=0x7fbe3406d890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:06.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.170+0000 7fbe3a789640 1 -- 172.21.15.49:0/1285063439 shutdown_connections 2023-11-06T02:14:06.169 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.170+0000 7fbe3a789640 1 -- 172.21.15.49:0/1285063439 wait complete. 2023-11-06T02:14:06.169 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.170+0000 7fbe3a789640 1 Processor -- start 2023-11-06T02:14:06.169 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.170+0000 7fbe3a789640 1 -- start start 2023-11-06T02:14:06.169 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.170+0000 7fbe3a789640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe34069d70 0x7fbe3414fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:06.169 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.170+0000 7fbe3a789640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbe340cdae0 0x7fbe341528f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:06.169 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.170+0000 7fbe3a789640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe3415a820 0x7fbe341608c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:06.170 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.170+0000 7fbe3a789640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe34155a40 con 0x7fbe3415a820 2023-11-06T02:14:06.170 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.170+0000 7fbe3a789640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe34155480 con 0x7fbe34069d70 2023-11-06T02:14:06.170 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.170+0000 7fbe3a789640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe34155760 con 0x7fbe340cdae0 2023-11-06T02:14:06.170 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.170+0000 7fbe38d26640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe3415a820 0x7fbe341608c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:06.170 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.170+0000 7fbe38d26640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe3415a820 0x7fbe341608c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51740/0 (socket says 172.21.15.49:51740) 2023-11-06T02:14:06.171 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.170+0000 7fbe38d26640 1 -- 172.21.15.49:0/2641141109 learned_addr learned my addr 172.21.15.49:0/2641141109 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:06.171 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.170+0000 7fbe33fff640 1 --2- 172.21.15.49:0/2641141109 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe34069d70 0x7fbe3414fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:06.171 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.170+0000 7fbe337fe640 1 --2- 172.21.15.49:0/2641141109 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbe340cdae0 0x7fbe341528f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:06.171 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.170+0000 7fbe38d26640 1 -- 172.21.15.49:0/2641141109 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbe340cdae0 msgr2=0x7fbe341528f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:06.171 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.170+0000 7fbe38d26640 1 --2- 172.21.15.49:0/2641141109 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbe340cdae0 0x7fbe341528f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:06.172 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.170+0000 7fbe38d26640 1 -- 172.21.15.49:0/2641141109 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe34069d70 msgr2=0x7fbe3414fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:06.172 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.170+0000 7fbe38d26640 1 --2- 172.21.15.49:0/2641141109 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe34069d70 0x7fbe3414fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:06.172 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.170+0000 7fbe38d26640 1 -- 172.21.15.49:0/2641141109 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe28009b30 con 0x7fbe3415a820 2023-11-06T02:14:06.172 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.170+0000 7fbe337fe640 1 --2- 172.21.15.49:0/2641141109 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbe340cdae0 0x7fbe341528f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:14:06.172 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.170+0000 7fbe33fff640 1 --2- 172.21.15.49:0/2641141109 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe34069d70 0x7fbe3414fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:14:06.173 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.170+0000 7fbe38d26640 1 --2- 172.21.15.49:0/2641141109 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe3415a820 0x7fbe341608c0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fbe2400c570 tx=0x7fbe2400ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:06.173 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.170+0000 7fbe30ff9640 1 -- 172.21.15.49:0/2641141109 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe24004440 con 0x7fbe3415a820 2023-11-06T02:14:06.173 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.170+0000 7fbe30ff9640 1 -- 172.21.15.49:0/2641141109 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe24003620 con 0x7fbe3415a820 2023-11-06T02:14:06.173 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.170+0000 7fbe3a789640 1 -- 172.21.15.49:0/2641141109 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe341504a0 con 0x7fbe3415a820 2023-11-06T02:14:06.173 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.170+0000 7fbe30ff9640 1 -- 172.21.15.49:0/2641141109 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe240039d0 con 0x7fbe3415a820 2023-11-06T02:14:06.173 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.170+0000 7fbe3a789640 1 -- 172.21.15.49:0/2641141109 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe34150080 con 0x7fbe3415a820 2023-11-06T02:14:06.174 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.170+0000 7fbe3a789640 1 -- 172.21.15.49:0/2641141109 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe34150300 con 0x7fbe3415a820 2023-11-06T02:14:06.175 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.174+0000 7fbe30ff9640 1 -- 172.21.15.49:0/2641141109 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fbe24003b30 con 0x7fbe3415a820 2023-11-06T02:14:06.175 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.174+0000 7fbe30ff9640 1 --2- 172.21.15.49:0/2641141109 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fbe20065a90 0x7fbe20067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:06.175 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.174+0000 7fbe30ff9640 1 -- 172.21.15.49:0/2641141109 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbe24083420 con 0x7fbe3415a820 2023-11-06T02:14:06.175 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.174+0000 7fbe33fff640 1 --2- 172.21.15.49:0/2641141109 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fbe20065a90 0x7fbe20067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:06.176 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.174+0000 7fbe33fff640 1 --2- 172.21.15.49:0/2641141109 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fbe20065a90 0x7fbe20067f50 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c006fd0 tx=0x7fbe2c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:06.176 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.178+0000 7fbe30ff9640 1 -- 172.21.15.49:0/2641141109 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fbe34150300 con 0x7fbe3415a820 2023-11-06T02:14:06.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.342+0000 7fbe3a789640 1 -- 172.21.15.49:0/2641141109 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbe341509c0 con 0x7fbe3415a820 2023-11-06T02:14:06.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.342+0000 7fbe30ff9640 1 -- 172.21.15.49:0/2641141109 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v80) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbe341509c0 con 0x7fbe3415a820 2023-11-06T02:14:06.337 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2023-11-06T02:14:06.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.342+0000 7fbe3a789640 1 -- 172.21.15.49:0/2641141109 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fbe20065a90 msgr2=0x7fbe20067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:06.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.342+0000 7fbe3a789640 1 --2- 172.21.15.49:0/2641141109 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fbe20065a90 0x7fbe20067f50 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c006fd0 tx=0x7fbe2c008040 comp rx=0 tx=0).stop 2023-11-06T02:14:06.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.342+0000 7fbe3a789640 1 -- 172.21.15.49:0/2641141109 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe3415a820 msgr2=0x7fbe341608c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:06.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.342+0000 7fbe3a789640 1 --2- 172.21.15.49:0/2641141109 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe3415a820 0x7fbe341608c0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fbe2400c570 tx=0x7fbe2400ca30 comp rx=0 tx=0).stop 2023-11-06T02:14:06.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.342+0000 7fbe3a789640 1 -- 172.21.15.49:0/2641141109 shutdown_connections 2023-11-06T02:14:06.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.342+0000 7fbe3a789640 1 --2- 172.21.15.49:0/2641141109 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fbe20065a90 0x7fbe20067f50 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:06.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.342+0000 7fbe3a789640 1 --2- 172.21.15.49:0/2641141109 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe3415a820 0x7fbe341608c0 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:06.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.342+0000 7fbe3a789640 1 --2- 172.21.15.49:0/2641141109 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbe340cdae0 0x7fbe341528f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:06.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.342+0000 7fbe3a789640 1 --2- 172.21.15.49:0/2641141109 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe34069d70 0x7fbe3414fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:06.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.342+0000 7fbe3a789640 1 -- 172.21.15.49:0/2641141109 >> 172.21.15.49:0/2641141109 conn(0x7fbe3406d4b0 msgr2=0x7fbe3415c660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:06.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.342+0000 7fbe3a789640 1 -- 172.21.15.49:0/2641141109 shutdown_connections 2023-11-06T02:14:06.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.342+0000 7fbe3a789640 1 -- 172.21.15.49:0/2641141109 wait complete. 2023-11-06T02:14:06.355 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-06T02:14:06.468 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.474+0000 7f028f825640 1 Processor -- start 2023-11-06T02:14:06.469 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.474+0000 7f028f825640 1 -- start start 2023-11-06T02:14:06.471 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.474+0000 7f028f825640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f0288071bb0 0x7f0288161650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:06.471 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.474+0000 7f028f825640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02880d17c0 con 0x7f02880b1f90 2023-11-06T02:14:06.472 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.474+0000 7f028f825640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02880ce6c0 con 0x7f02880cda70 2023-11-06T02:14:06.472 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.474+0000 7f028f825640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02880cdf20 con 0x7f0288071bb0 2023-11-06T02:14:06.472 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.474+0000 7f028d5c1640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f02880b1f90 0x7f02880d1c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:59344/0 (socket says 172.21.15.49:59344) 2023-11-06T02:14:06.472 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.474+0000 7f028d5c1640 1 -- 172.21.15.49:0/2261149468 learned_addr learned my addr 172.21.15.49:0/2261149468 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:14:06.473 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.474+0000 7f028ddc2640 1 --2- 172.21.15.49:0/2261149468 >> v2:172.21.15.150:3300/0 conn(0x7f0288071bb0 0x7f0288161650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:06.473 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.474+0000 7f027bfff640 1 -- 172.21.15.49:0/2261149468 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4175531164 0 0) 0x7f02880d17c0 con 0x7f02880b1f90 2023-11-06T02:14:06.473 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.474+0000 7f027bfff640 1 -- 172.21.15.49:0/2261149468 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0274003530 con 0x7f02880b1f90 2023-11-06T02:14:06.473 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.474+0000 7f027bfff640 1 -- 172.21.15.49:0/2261149468 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 48118734 0 0) 0x7f0274003530 con 0x7f02880b1f90 2023-11-06T02:14:06.473 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.474+0000 7f027bfff640 1 -- 172.21.15.49:0/2261149468 >> v2:172.21.15.150:3300/0 conn(0x7f0288071bb0 msgr2=0x7f0288161650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:06.474 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.474+0000 7f027bfff640 1 --2- 172.21.15.49:0/2261149468 >> v2:172.21.15.150:3300/0 conn(0x7f0288071bb0 0x7f0288161650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:06.474 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.474+0000 7f027bfff640 1 -- 172.21.15.49:0/2261149468 >> v1:172.21.15.150:6789/0 conn(0x7f02880cda70 legacy=0x7f02880714a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:06.474 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.474+0000 7f027bfff640 1 -- 172.21.15.49:0/2261149468 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02880d2360 con 0x7f02880b1f90 2023-11-06T02:14:06.474 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.474+0000 7f027bfff640 1 -- 172.21.15.49:0/2261149468 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7f0284001680 con 0x7f02880b1f90 2023-11-06T02:14:06.474 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.474+0000 7f027bfff640 1 -- 172.21.15.49:0/2261149468 >> v1:172.21.15.49:6789/0 conn(0x7f02880b1f90 legacy=0x7f02880d1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:06.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.474+0000 7f027bfff640 1 --2- 172.21.15.49:0/2261149468 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0274003980 0x7f0274003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:06.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.474+0000 7f027bfff640 1 --2- 172.21.15.49:0/2261149468 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02740047d0 0x7f0274004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:06.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.474+0000 7f027bfff640 1 --2- 172.21.15.49:0/2261149468 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0274005190 0x7f027400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:06.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.474+0000 7f027bfff640 1 -- 172.21.15.49:0/2261149468 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02880cdf20 con 0x7f02740047d0 2023-11-06T02:14:06.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.474+0000 7f027bfff640 1 -- 172.21.15.49:0/2261149468 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f027400c030 con 0x7f0274005190 2023-11-06T02:14:06.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.474+0000 7f027bfff640 1 -- 172.21.15.49:0/2261149468 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f027400c1f0 con 0x7f0274003980 2023-11-06T02:14:06.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.474+0000 7f028cdc0640 1 --2- 172.21.15.49:0/2261149468 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02740047d0 0x7f0274004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:06.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.474+0000 7f028d5c1640 1 --2- 172.21.15.49:0/2261149468 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0274003980 0x7f0274003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:06.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.474+0000 7f028ddc2640 1 --2- 172.21.15.49:0/2261149468 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0274005190 0x7f027400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:06.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.474+0000 7f028cdc0640 1 -- 172.21.15.49:0/2261149468 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0274003980 msgr2=0x7f0274003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:06.477 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.474+0000 7f028cdc0640 1 --2- 172.21.15.49:0/2261149468 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0274003980 0x7f0274003da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:06.477 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.474+0000 7f028cdc0640 1 -- 172.21.15.49:0/2261149468 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0274005190 msgr2=0x7f027400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:06.477 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.474+0000 7f028cdc0640 1 --2- 172.21.15.49:0/2261149468 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0274005190 0x7f027400b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:06.477 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.474+0000 7f028cdc0640 1 -- 172.21.15.49:0/2261149468 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0274003530 con 0x7f02740047d0 2023-11-06T02:14:06.477 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.474+0000 7f028ddc2640 1 -- 172.21.15.49:0/2261149468 reap_dead start 2023-11-06T02:14:06.481 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.478+0000 7f028cdc0640 1 --2- 172.21.15.49:0/2261149468 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02740047d0 0x7f0274004c50 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f027c001780 tx=0x7f027c01c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad51fad68d11047a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:06.481 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.478+0000 7f027bfff640 1 -- 172.21.15.49:0/2261149468 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f027c0052d0 con 0x7f02740047d0 2023-11-06T02:14:06.481 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.478+0000 7f027bfff640 1 -- 172.21.15.49:0/2261149468 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f027c0a7050 con 0x7f02740047d0 2023-11-06T02:14:06.481 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.478+0000 7f027bfff640 1 -- 172.21.15.49:0/2261149468 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f027c003600 con 0x7f02740047d0 2023-11-06T02:14:06.482 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.478+0000 7f028f825640 1 -- 172.21.15.49:0/2261149468 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02740047d0 msgr2=0x7f0274004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:06.482 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.478+0000 7f028f825640 1 --2- 172.21.15.49:0/2261149468 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02740047d0 0x7f0274004c50 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f027c001780 tx=0x7f027c01c3e0 comp rx=0 tx=0).stop 2023-11-06T02:14:06.482 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.478+0000 7f028f825640 1 -- 172.21.15.49:0/2261149468 shutdown_connections 2023-11-06T02:14:06.482 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.478+0000 7f028f825640 1 --2- 172.21.15.49:0/2261149468 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02740047d0 0x7f0274004c50 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:06.482 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.478+0000 7f028f825640 1 -- 172.21.15.49:0/2261149468 >> 172.21.15.49:0/2261149468 conn(0x7f02880d2aa0 msgr2=0x7f02880d2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:06.483 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.478+0000 7f028f825640 1 -- 172.21.15.49:0/2261149468 shutdown_connections 2023-11-06T02:14:06.483 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.478+0000 7f028f825640 1 -- 172.21.15.49:0/2261149468 wait complete. 2023-11-06T02:14:06.483 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.478+0000 7f028f825640 1 Processor -- start 2023-11-06T02:14:06.483 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.478+0000 7f028f825640 1 -- start start 2023-11-06T02:14:06.483 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.482+0000 7f028f825640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02740047d0 0x7f02880b4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:06.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.482+0000 7f028f825640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f02881f2b00 0x7f02881f2f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:06.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.482+0000 7f028f825640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f02881509d0 0x7f0288150e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:06.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.482+0000 7f028f825640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02880b23c0 con 0x7f02740047d0 2023-11-06T02:14:06.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.482+0000 7f028f825640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02880b2f40 con 0x7f02881509d0 2023-11-06T02:14:06.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.482+0000 7f028f825640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0288151390 con 0x7f02881f2b00 2023-11-06T02:14:06.485 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.482+0000 7f028d5c1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02740047d0 0x7f02880b4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:06.485 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.482+0000 7f028d5c1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02740047d0 0x7f02880b4b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51758/0 (socket says 172.21.15.49:51758) 2023-11-06T02:14:06.485 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.482+0000 7f028d5c1640 1 -- 172.21.15.49:0/3960076710 learned_addr learned my addr 172.21.15.49:0/3960076710 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:06.485 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.482+0000 7f028cdc0640 1 --2- 172.21.15.49:0/3960076710 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f02881f2b00 0x7f02881f2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:06.485 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.482+0000 7f028ddc2640 1 --2- 172.21.15.49:0/3960076710 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f02881509d0 0x7f0288150e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:06.486 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.482+0000 7f028d5c1640 1 -- 172.21.15.49:0/3960076710 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f02881f2b00 msgr2=0x7f02881f2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:06.486 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.482+0000 7f028d5c1640 1 --2- 172.21.15.49:0/3960076710 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f02881f2b00 0x7f02881f2f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:06.486 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.482+0000 7f028d5c1640 1 -- 172.21.15.49:0/3960076710 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f02881509d0 msgr2=0x7f0288150e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:06.486 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.482+0000 7f028d5c1640 1 --2- 172.21.15.49:0/3960076710 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f02881509d0 0x7f0288150e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:06.486 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.482+0000 7f028d5c1640 1 -- 172.21.15.49:0/3960076710 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02880b2610 con 0x7f02740047d0 2023-11-06T02:14:06.487 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.482+0000 7f028d5c1640 1 --2- 172.21.15.49:0/3960076710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02740047d0 0x7f02880b4b50 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f0284010b10 tx=0x7f028400e100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:06.487 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.482+0000 7f0279ffb640 1 -- 172.21.15.49:0/3960076710 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f028400ed10 con 0x7f02740047d0 2023-11-06T02:14:06.487 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.482+0000 7f0279ffb640 1 -- 172.21.15.49:0/3960076710 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f028400ee70 con 0x7f02740047d0 2023-11-06T02:14:06.487 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.482+0000 7f028f825640 1 -- 172.21.15.49:0/3960076710 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02880b28f0 con 0x7f02740047d0 2023-11-06T02:14:06.487 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.482+0000 7f0279ffb640 1 -- 172.21.15.49:0/3960076710 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02840145c0 con 0x7f02740047d0 2023-11-06T02:14:06.488 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.482+0000 7f028f825640 1 -- 172.21.15.49:0/3960076710 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02880b5090 con 0x7f02740047d0 2023-11-06T02:14:06.488 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.482+0000 7f028f825640 1 -- 172.21.15.49:0/3960076710 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0288074040 con 0x7f02740047d0 2023-11-06T02:14:06.488 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.482+0000 7f0279ffb640 1 -- 172.21.15.49:0/3960076710 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f0284014720 con 0x7f02740047d0 2023-11-06T02:14:06.488 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.482+0000 7f0279ffb640 1 --2- 172.21.15.49:0/3960076710 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f026c065a90 0x7f026c067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:06.488 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.482+0000 7f0279ffb640 1 -- 172.21.15.49:0/3960076710 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0284086250 con 0x7f02740047d0 2023-11-06T02:14:06.489 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.482+0000 7f028cdc0640 1 --2- 172.21.15.49:0/3960076710 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f026c065a90 0x7f026c067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:06.489 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.486+0000 7f028cdc0640 1 --2- 172.21.15.49:0/3960076710 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f026c065a90 0x7f026c067f50 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f027c001750 tx=0x7f027c0011a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:06.489 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.486+0000 7f0279ffb640 1 -- 172.21.15.49:0/3960076710 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f0284053ab0 con 0x7f02740047d0 2023-11-06T02:14:06.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.650+0000 7f028f825640 1 -- 172.21.15.49:0/3960076710 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f02880b56c0 con 0x7f02740047d0 2023-11-06T02:14:06.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.650+0000 7f0279ffb640 1 -- 172.21.15.49:0/3960076710 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v80) v1 ==== 80+0+0 (secure 0 0 0) 0x7f02840570d0 con 0x7f02740047d0 2023-11-06T02:14:06.646 INFO:teuthology.orchestra.run.smithi049.stderr:listed 0 entries 2023-11-06T02:14:06.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.650+0000 7f028f825640 1 -- 172.21.15.49:0/3960076710 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f026c065a90 msgr2=0x7f026c067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:06.648 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.650+0000 7f028f825640 1 --2- 172.21.15.49:0/3960076710 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f026c065a90 0x7f026c067f50 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f027c001750 tx=0x7f027c0011a0 comp rx=0 tx=0).stop 2023-11-06T02:14:06.648 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.650+0000 7f028f825640 1 -- 172.21.15.49:0/3960076710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02740047d0 msgr2=0x7f02880b4b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:06.648 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.650+0000 7f028f825640 1 --2- 172.21.15.49:0/3960076710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02740047d0 0x7f02880b4b50 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f0284010b10 tx=0x7f028400e100 comp rx=0 tx=0).stop 2023-11-06T02:14:06.648 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.650+0000 7f028f825640 1 -- 172.21.15.49:0/3960076710 shutdown_connections 2023-11-06T02:14:06.648 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.650+0000 7f028f825640 1 --2- 172.21.15.49:0/3960076710 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f026c065a90 0x7f026c067f50 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:06.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.650+0000 7f028f825640 1 --2- 172.21.15.49:0/3960076710 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f02881509d0 0x7f0288150e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:06.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.650+0000 7f028f825640 1 --2- 172.21.15.49:0/3960076710 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f02881f2b00 0x7f02881f2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:06.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.650+0000 7f028f825640 1 --2- 172.21.15.49:0/3960076710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02740047d0 0x7f02880b4b50 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:06.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.650+0000 7f028f825640 1 -- 172.21.15.49:0/3960076710 >> 172.21.15.49:0/3960076710 conn(0x7f02880d2aa0 msgr2=0x7f02880c67c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:06.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.654+0000 7f028f825640 1 -- 172.21.15.49:0/3960076710 shutdown_connections 2023-11-06T02:14:06.650 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:06.654+0000 7f028f825640 1 -- 172.21.15.49:0/3960076710 wait complete. 2023-11-06T02:14:06.663 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi049.front.sepia.ceph.com... 2023-11-06T02:14:06.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:14:06.666 DEBUG:teuthology.orchestra.run.smithi049:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-06T02:14:06.677 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi049.stderr:ceph-fuse[55460]: fuse finished with error 0 and tester_r 0 2023-11-06T02:14:06.680 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T02:14:12.782 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi049.front.sepia.ceph.com 2023-11-06T02:14:12.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:14:12.783 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-06T02:14:12.791 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-11-06T02:14:12.791 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T02:14:12.841 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336476k,nr_inodes=4084119,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16105 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-06T02:14:12.841 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse client.0 is not mounted at ubuntu@smithi049.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0 2023-11-06T02:14:12.842 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi049.front.sepia.ceph.com 2023-11-06T02:14:12.842 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:14:12.842 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-06T02:14:12.888 INFO:teuthology.orchestra.run.smithi049.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-11-06T02:14:12.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T02:14:12.890 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-06T02:14:13.065 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.070+0000 7fb7571b6640 1 Processor -- start 2023-11-06T02:14:13.066 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.070+0000 7fb7571b6640 1 -- start start 2023-11-06T02:14:13.067 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.070+0000 7fb7571b6640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fb75015cc50 0x7fb7500701e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:13.067 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.070+0000 7fb7571b6640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7500b0080 0x7fb7500b04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:13.067 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.070+0000 7fb7571b6640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb7500c8f80 0x7fb7500c9410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:13.068 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.070+0000 7fb7571b6640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7500b0a20 con 0x7fb7500c8f80 2023-11-06T02:14:13.068 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.070+0000 7fb7571b6640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb75015d030 con 0x7fb75015cc50 2023-11-06T02:14:13.068 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.070+0000 7fb7571b6640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7500cc9c0 con 0x7fb7500b0080 2023-11-06T02:14:13.068 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.070+0000 7fb755753640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb7500c8f80 0x7fb7500c9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:13.068 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.070+0000 7fb755753640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb7500c8f80 0x7fb7500c9410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55362/0 (socket says 172.21.15.49:55362) 2023-11-06T02:14:13.069 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.070+0000 7fb755753640 1 -- 172.21.15.49:0/4179209122 learned_addr learned my addr 172.21.15.49:0/4179209122 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:13.069 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.070+0000 7fb754f52640 1 --2- 172.21.15.49:0/4179209122 >> v2:172.21.15.150:3300/0 conn(0x7fb75015cc50 0x7fb7500701e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:13.069 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.070+0000 7fb74ffff640 1 --2- 172.21.15.49:0/4179209122 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7500b0080 0x7fb7500b04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:13.069 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.070+0000 7fb755753640 1 -- 172.21.15.49:0/4179209122 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7500b0080 msgr2=0x7fb7500b04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:13.069 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.070+0000 7fb755753640 1 --2- 172.21.15.49:0/4179209122 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7500b0080 0x7fb7500b04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:13.070 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.070+0000 7fb755753640 1 -- 172.21.15.49:0/4179209122 >> v2:172.21.15.150:3300/0 conn(0x7fb75015cc50 msgr2=0x7fb7500701e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:13.070 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.070+0000 7fb755753640 1 --2- 172.21.15.49:0/4179209122 >> v2:172.21.15.150:3300/0 conn(0x7fb75015cc50 0x7fb7500701e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:13.070 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.070+0000 7fb755753640 1 -- 172.21.15.49:0/4179209122 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7500ccc60 con 0x7fb7500c8f80 2023-11-06T02:14:13.071 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.074+0000 7fb755753640 1 --2- 172.21.15.49:0/4179209122 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7500c8f80 0x7fb7500c9410 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fb748009e50 tx=0x7fb74801c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8475e5029e6eced server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:13.071 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.074+0000 7fb74f7fe640 1 -- 172.21.15.49:0/4179209122 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7480a5790 con 0x7fb7500c8f80 2023-11-06T02:14:13.072 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.074+0000 7fb74f7fe640 1 -- 172.21.15.49:0/4179209122 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7480a58f0 con 0x7fb7500c8f80 2023-11-06T02:14:13.072 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.074+0000 7fb74f7fe640 1 -- 172.21.15.49:0/4179209122 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7480a5c80 con 0x7fb7500c8f80 2023-11-06T02:14:13.072 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.074+0000 7fb7571b6640 1 -- 172.21.15.49:0/4179209122 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7500c8f80 msgr2=0x7fb7500c9410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:13.072 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.074+0000 7fb7571b6640 1 --2- 172.21.15.49:0/4179209122 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7500c8f80 0x7fb7500c9410 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fb748009e50 tx=0x7fb74801c910 comp rx=0 tx=0).stop 2023-11-06T02:14:13.072 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.074+0000 7fb7571b6640 1 -- 172.21.15.49:0/4179209122 shutdown_connections 2023-11-06T02:14:13.073 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.074+0000 7fb7571b6640 1 --2- 172.21.15.49:0/4179209122 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7500b0080 0x7fb7500b04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:13.073 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.074+0000 7fb7571b6640 1 --2- 172.21.15.49:0/4179209122 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7500c8f80 0x7fb7500c9410 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:13.073 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.074+0000 7fb7571b6640 1 --2- 172.21.15.49:0/4179209122 >> v2:172.21.15.150:3300/0 conn(0x7fb75015cc50 0x7fb7500701e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:13.073 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.074+0000 7fb7571b6640 1 -- 172.21.15.49:0/4179209122 >> 172.21.15.49:0/4179209122 conn(0x7fb75006c0a0 msgr2=0x7fb75006c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:13.073 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.074+0000 7fb7571b6640 1 -- 172.21.15.49:0/4179209122 shutdown_connections 2023-11-06T02:14:13.074 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.074+0000 7fb7571b6640 1 -- 172.21.15.49:0/4179209122 wait complete. 2023-11-06T02:14:13.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.078+0000 7fb7571b6640 1 Processor -- start 2023-11-06T02:14:13.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.078+0000 7fb7571b6640 1 -- start start 2023-11-06T02:14:13.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.078+0000 7fb7571b6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7500b0080 0x7fb750149d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:13.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.078+0000 7fb7571b6640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb7500c8f80 0x7fb75014b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:13.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.078+0000 7fb7571b6640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb75015cc50 0x7fb7500ad4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:13.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.078+0000 7fb7571b6640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7500abca0 con 0x7fb7500b0080 2023-11-06T02:14:13.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.078+0000 7fb7571b6640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb750150d90 con 0x7fb7500c8f80 2023-11-06T02:14:13.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.078+0000 7fb7571b6640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb750151070 con 0x7fb75015cc50 2023-11-06T02:14:13.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.078+0000 7fb754f52640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7500b0080 0x7fb750149d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:13.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.078+0000 7fb754f52640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7500b0080 0x7fb750149d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55374/0 (socket says 172.21.15.49:55374) 2023-11-06T02:14:13.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.078+0000 7fb754f52640 1 -- 172.21.15.49:0/1927099361 learned_addr learned my addr 172.21.15.49:0/1927099361 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:13.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.078+0000 7fb74ffff640 1 --2- 172.21.15.49:0/1927099361 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb7500c8f80 0x7fb75014b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:13.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.078+0000 7fb755753640 1 --2- 172.21.15.49:0/1927099361 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb75015cc50 0x7fb7500ad4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:13.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.078+0000 7fb754f52640 1 -- 172.21.15.49:0/1927099361 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb75015cc50 msgr2=0x7fb7500ad4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:13.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.078+0000 7fb754f52640 1 --2- 172.21.15.49:0/1927099361 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb75015cc50 0x7fb7500ad4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:13.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.078+0000 7fb754f52640 1 -- 172.21.15.49:0/1927099361 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb7500c8f80 msgr2=0x7fb75014b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:13.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.078+0000 7fb754f52640 1 --2- 172.21.15.49:0/1927099361 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb7500c8f80 0x7fb75014b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:13.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.078+0000 7fb754f52640 1 -- 172.21.15.49:0/1927099361 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb748009b30 con 0x7fb7500b0080 2023-11-06T02:14:13.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.078+0000 7fb754f52640 1 --2- 172.21.15.49:0/1927099361 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7500b0080 0x7fb750149d20 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fb74000d800 tx=0x7fb74000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:13.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.078+0000 7fb74d7fa640 1 -- 172.21.15.49:0/1927099361 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb740018070 con 0x7fb7500b0080 2023-11-06T02:14:13.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.078+0000 7fb74d7fa640 1 -- 172.21.15.49:0/1927099361 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7400059a0 con 0x7fb7500b0080 2023-11-06T02:14:13.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.078+0000 7fb7571b6640 1 -- 172.21.15.49:0/1927099361 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb750151350 con 0x7fb7500b0080 2023-11-06T02:14:13.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.078+0000 7fb74d7fa640 1 -- 172.21.15.49:0/1927099361 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb740005b50 con 0x7fb7500b0080 2023-11-06T02:14:13.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.078+0000 7fb7571b6640 1 -- 172.21.15.49:0/1927099361 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb75014a260 con 0x7fb7500b0080 2023-11-06T02:14:13.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.078+0000 7fb7571b6640 1 -- 172.21.15.49:0/1927099361 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb750074040 con 0x7fb7500b0080 2023-11-06T02:14:13.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.082+0000 7fb74d7fa640 1 -- 172.21.15.49:0/1927099361 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fb740005cb0 con 0x7fb7500b0080 2023-11-06T02:14:13.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.082+0000 7fb74d7fa640 1 --2- 172.21.15.49:0/1927099361 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb730065a90 0x7fb730067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:13.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.082+0000 7fb74d7fa640 1 -- 172.21.15.49:0/1927099361 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb7400831f0 con 0x7fb7500b0080 2023-11-06T02:14:13.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.082+0000 7fb74ffff640 1 --2- 172.21.15.49:0/1927099361 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb730065a90 0x7fb730067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:13.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.082+0000 7fb74ffff640 1 --2- 172.21.15.49:0/1927099361 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb730065a90 0x7fb730067f50 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fb7501679e0 tx=0x7fb738008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:13.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.086+0000 7fb74d7fa640 1 -- 172.21.15.49:0/1927099361 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fb7400509b0 con 0x7fb7500b0080 2023-11-06T02:14:13.276 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.282+0000 7fb7571b6640 1 -- 172.21.15.49:0/1927099361 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fb7500b4580 con 0x7fb7500b0080 2023-11-06T02:14:13.278 INFO:tasks.ceph.mon.a.smithi049.stderr:2023-11-06T02:14:13.282+0000 7f0164e5a640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-06T02:14:13.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.298+0000 7fb74d7fa640 1 -- 172.21.15.49:0/1927099361 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v47) v1 ==== 161+0+0 (secure 0 0 0) 0x7fb740053fd0 con 0x7fb7500b0080 2023-11-06T02:14:13.295 INFO:teuthology.orchestra.run.smithi049.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-06T02:14:13.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.302+0000 7fb7571b6640 1 -- 172.21.15.49:0/1927099361 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb730065a90 msgr2=0x7fb730067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:13.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.302+0000 7fb7571b6640 1 --2- 172.21.15.49:0/1927099361 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb730065a90 0x7fb730067f50 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fb7501679e0 tx=0x7fb738008040 comp rx=0 tx=0).stop 2023-11-06T02:14:13.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.302+0000 7fb7571b6640 1 -- 172.21.15.49:0/1927099361 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7500b0080 msgr2=0x7fb750149d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:13.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.302+0000 7fb7571b6640 1 --2- 172.21.15.49:0/1927099361 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7500b0080 0x7fb750149d20 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fb74000d800 tx=0x7fb74000dcc0 comp rx=0 tx=0).stop 2023-11-06T02:14:13.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.302+0000 7fb7571b6640 1 -- 172.21.15.49:0/1927099361 shutdown_connections 2023-11-06T02:14:13.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.302+0000 7fb7571b6640 1 --2- 172.21.15.49:0/1927099361 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb730065a90 0x7fb730067f50 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:13.300 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.302+0000 7fb7571b6640 1 --2- 172.21.15.49:0/1927099361 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb75015cc50 0x7fb7500ad4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:13.300 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.302+0000 7fb7571b6640 1 --2- 172.21.15.49:0/1927099361 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb7500c8f80 0x7fb75014b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:13.300 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.302+0000 7fb7571b6640 1 --2- 172.21.15.49:0/1927099361 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7500b0080 0x7fb750149d20 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:13.300 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.302+0000 7fb7571b6640 1 -- 172.21.15.49:0/1927099361 >> 172.21.15.49:0/1927099361 conn(0x7fb75006c0a0 msgr2=0x7fb750071a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:13.300 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.302+0000 7fb7571b6640 1 -- 172.21.15.49:0/1927099361 shutdown_connections 2023-11-06T02:14:13.301 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.302+0000 7fb7571b6640 1 -- 172.21.15.49:0/1927099361 wait complete. 2023-11-06T02:14:13.307 INFO:tasks.ceph.mds.c.smithi049.stderr: -7930> 2023-11-06T02:13:39.438+0000 7f0988600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-06T02:14:13.318 DEBUG:teuthology.orchestra.run.smithi049:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-06T02:14:13.321 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-06T02:14:13.345 INFO:teuthology.orchestra.run.smithi049.stdout:[global] 2023-11-06T02:14:13.345 INFO:teuthology.orchestra.run.smithi049.stdout: chdir = "" 2023-11-06T02:14:13.345 INFO:teuthology.orchestra.run.smithi049.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-06T02:14:13.346 INFO:teuthology.orchestra.run.smithi049.stdout: auth supported = cephx 2023-11-06T02:14:13.346 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:13.346 INFO:teuthology.orchestra.run.smithi049.stdout: filestore xattr use omap = true 2023-11-06T02:14:13.346 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:13.347 INFO:teuthology.orchestra.run.smithi049.stdout: mon clock drift allowed = 1.000 2023-11-06T02:14:13.347 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:13.347 INFO:teuthology.orchestra.run.smithi049.stdout: osd crush chooseleaf type = 0 2023-11-06T02:14:13.347 INFO:teuthology.orchestra.run.smithi049.stdout: auth debug = true 2023-11-06T02:14:13.347 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:13.347 INFO:teuthology.orchestra.run.smithi049.stdout: ms die on old message = true 2023-11-06T02:14:13.348 INFO:teuthology.orchestra.run.smithi049.stdout: ms die on bug = true 2023-11-06T02:14:13.348 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:13.348 INFO:teuthology.orchestra.run.smithi049.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-06T02:14:13.348 INFO:teuthology.orchestra.run.smithi049.stdout: mon pg warn max object skew = 0 2023-11-06T02:14:13.348 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:13.349 INFO:teuthology.orchestra.run.smithi049.stdout: # disable pg_autoscaler by default for new pools 2023-11-06T02:14:13.349 INFO:teuthology.orchestra.run.smithi049.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-06T02:14:13.349 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:13.349 INFO:teuthology.orchestra.run.smithi049.stdout: osd pool default size = 2 2023-11-06T02:14:13.349 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:13.350 INFO:teuthology.orchestra.run.smithi049.stdout: mon osd allow primary affinity = true 2023-11-06T02:14:13.350 INFO:teuthology.orchestra.run.smithi049.stdout: mon osd allow pg remap = true 2023-11-06T02:14:13.350 INFO:teuthology.orchestra.run.smithi049.stdout: mon warn on legacy crush tunables = false 2023-11-06T02:14:13.350 INFO:teuthology.orchestra.run.smithi049.stdout: mon warn on crush straw calc version zero = false 2023-11-06T02:14:13.351 INFO:teuthology.orchestra.run.smithi049.stdout: mon warn on no sortbitwise = false 2023-11-06T02:14:13.351 INFO:teuthology.orchestra.run.smithi049.stdout: mon warn on osd down out interval zero = false 2023-11-06T02:14:13.351 INFO:teuthology.orchestra.run.smithi049.stdout: mon warn on too few osds = false 2023-11-06T02:14:13.351 INFO:teuthology.orchestra.run.smithi049.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-06T02:14:13.351 INFO:teuthology.orchestra.run.smithi049.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-06T02:14:13.351 INFO:teuthology.orchestra.run.smithi049.stdout: mon_allow_pool_size_one = true 2023-11-06T02:14:13.352 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:13.352 INFO:teuthology.orchestra.run.smithi049.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-06T02:14:13.352 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:13.352 INFO:teuthology.orchestra.run.smithi049.stdout: osd default data pool replay window = 5 2023-11-06T02:14:13.352 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:13.353 INFO:teuthology.orchestra.run.smithi049.stdout: mon allow pool delete = true 2023-11-06T02:14:13.353 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:13.353 INFO:teuthology.orchestra.run.smithi049.stdout: mon cluster log file level = debug 2023-11-06T02:14:13.353 INFO:teuthology.orchestra.run.smithi049.stdout: debug asserts on shutdown = true 2023-11-06T02:14:13.353 INFO:teuthology.orchestra.run.smithi049.stdout: mon health detail to clog = false 2023-11-06T02:14:13.353 INFO:teuthology.orchestra.run.smithi049.stdout: mon host = "172.21.15.49,172.21.15.150,[v2:172.21.15.150:3301,v1:172.21.15.150:6790]" 2023-11-06T02:14:13.354 INFO:teuthology.orchestra.run.smithi049.stdout: fsid = 1a8b035c-6ba4-499a-b876-aea47fd9c0d4 2023-11-06T02:14:13.354 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:13.354 INFO:teuthology.orchestra.run.smithi049.stdout:[osd] 2023-11-06T02:14:13.354 INFO:teuthology.orchestra.run.smithi049.stdout: osd journal size = 100 2023-11-06T02:14:13.354 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:13.355 INFO:teuthology.orchestra.run.smithi049.stdout: osd scrub load threshold = 5.0 2023-11-06T02:14:13.355 INFO:teuthology.orchestra.run.smithi049.stdout: osd scrub max interval = 600 2023-11-06T02:14:13.355 INFO:teuthology.orchestra.run.smithi049.stdout: osd mclock profile = high_recovery_ops 2023-11-06T02:14:13.355 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:13.355 INFO:teuthology.orchestra.run.smithi049.stdout: osd recover clone overlap = true 2023-11-06T02:14:13.356 INFO:teuthology.orchestra.run.smithi049.stdout: osd recovery max chunk = 1048576 2023-11-06T02:14:13.356 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:13.356 INFO:teuthology.orchestra.run.smithi049.stdout: osd debug shutdown = true 2023-11-06T02:14:13.356 INFO:teuthology.orchestra.run.smithi049.stdout: osd debug op order = true 2023-11-06T02:14:13.356 INFO:teuthology.orchestra.run.smithi049.stdout: osd debug verify stray on activate = true 2023-11-06T02:14:13.357 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:13.357 INFO:teuthology.orchestra.run.smithi049.stdout: osd open classes on start = true 2023-11-06T02:14:13.357 INFO:teuthology.orchestra.run.smithi049.stdout: osd debug pg log writeout = true 2023-11-06T02:14:13.357 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:13.357 INFO:teuthology.orchestra.run.smithi049.stdout: osd deep scrub update digest min age = 30 2023-11-06T02:14:13.358 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:13.358 INFO:teuthology.orchestra.run.smithi049.stdout: osd map max advance = 10 2023-11-06T02:14:13.358 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:13.358 INFO:teuthology.orchestra.run.smithi049.stdout: journal zero on create = true 2023-11-06T02:14:13.358 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:13.359 INFO:teuthology.orchestra.run.smithi049.stdout: filestore ondisk finisher threads = 3 2023-11-06T02:14:13.359 INFO:teuthology.orchestra.run.smithi049.stdout: filestore apply finisher threads = 3 2023-11-06T02:14:13.359 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:13.359 INFO:teuthology.orchestra.run.smithi049.stdout: bdev debug aio = true 2023-11-06T02:14:13.359 INFO:teuthology.orchestra.run.smithi049.stdout: osd debug misdirected ops = true 2023-11-06T02:14:13.359 INFO:teuthology.orchestra.run.smithi049.stdout: bdev async discard = True 2023-11-06T02:14:13.360 INFO:teuthology.orchestra.run.smithi049.stdout: bdev enable discard = True 2023-11-06T02:14:13.360 INFO:teuthology.orchestra.run.smithi049.stdout: bluestore allocator = bitmap 2023-11-06T02:14:13.360 INFO:teuthology.orchestra.run.smithi049.stdout: bluestore block size = 96636764160 2023-11-06T02:14:13.360 INFO:teuthology.orchestra.run.smithi049.stdout: bluestore fsck on mount = True 2023-11-06T02:14:13.360 INFO:teuthology.orchestra.run.smithi049.stdout: debug bluefs = 1/20 2023-11-06T02:14:13.361 INFO:teuthology.orchestra.run.smithi049.stdout: debug bluestore = 1/20 2023-11-06T02:14:13.361 INFO:teuthology.orchestra.run.smithi049.stdout: debug ms = 1 2023-11-06T02:14:13.361 INFO:teuthology.orchestra.run.smithi049.stdout: debug osd = 20 2023-11-06T02:14:13.361 INFO:teuthology.orchestra.run.smithi049.stdout: debug rocksdb = 4/10 2023-11-06T02:14:13.361 INFO:teuthology.orchestra.run.smithi049.stdout: mon osd backfillfull_ratio = 0.85 2023-11-06T02:14:13.361 INFO:teuthology.orchestra.run.smithi049.stdout: mon osd full ratio = 0.9 2023-11-06T02:14:13.362 INFO:teuthology.orchestra.run.smithi049.stdout: mon osd nearfull ratio = 0.8 2023-11-06T02:14:13.362 INFO:teuthology.orchestra.run.smithi049.stdout: osd failsafe full ratio = 0.95 2023-11-06T02:14:13.362 INFO:teuthology.orchestra.run.smithi049.stdout: osd objectstore = bluestore 2023-11-06T02:14:13.362 INFO:teuthology.orchestra.run.smithi049.stdout: osd op complaint time = 180 2023-11-06T02:14:13.362 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:13.363 INFO:teuthology.orchestra.run.smithi049.stdout:[mgr] 2023-11-06T02:14:13.363 INFO:teuthology.orchestra.run.smithi049.stdout: debug ms = 1 2023-11-06T02:14:13.363 INFO:teuthology.orchestra.run.smithi049.stdout: debug mgr = 20 2023-11-06T02:14:13.363 INFO:teuthology.orchestra.run.smithi049.stdout: debug mon = 20 2023-11-06T02:14:13.364 INFO:teuthology.orchestra.run.smithi049.stdout: debug auth = 20 2023-11-06T02:14:13.364 INFO:teuthology.orchestra.run.smithi049.stdout: mon reweight min pgs per osd = 4 2023-11-06T02:14:13.364 INFO:teuthology.orchestra.run.smithi049.stdout: mon reweight min bytes per osd = 10 2023-11-06T02:14:13.364 INFO:teuthology.orchestra.run.smithi049.stdout: mgr/telemetry/nag = false 2023-11-06T02:14:13.364 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:13.365 INFO:teuthology.orchestra.run.smithi049.stdout:[mon] 2023-11-06T02:14:13.365 INFO:teuthology.orchestra.run.smithi049.stdout: debug ms = 1 2023-11-06T02:14:13.365 INFO:teuthology.orchestra.run.smithi049.stdout: debug mon = 20 2023-11-06T02:14:13.365 INFO:teuthology.orchestra.run.smithi049.stdout: debug paxos = 20 2023-11-06T02:14:13.365 INFO:teuthology.orchestra.run.smithi049.stdout: debug auth = 20 2023-11-06T02:14:13.365 INFO:teuthology.orchestra.run.smithi049.stdout: mon data avail warn = 5 2023-11-06T02:14:13.366 INFO:teuthology.orchestra.run.smithi049.stdout: mon mgr mkfs grace = 240 2023-11-06T02:14:13.366 INFO:teuthology.orchestra.run.smithi049.stdout: mon reweight min pgs per osd = 4 2023-11-06T02:14:13.366 INFO:teuthology.orchestra.run.smithi049.stdout: mon osd reporter subtree level = osd 2023-11-06T02:14:13.366 INFO:teuthology.orchestra.run.smithi049.stdout: mon osd prime pg temp = true 2023-11-06T02:14:13.366 INFO:teuthology.orchestra.run.smithi049.stdout: mon reweight min bytes per osd = 10 2023-11-06T02:14:13.367 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:13.367 INFO:teuthology.orchestra.run.smithi049.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-06T02:14:13.367 INFO:teuthology.orchestra.run.smithi049.stdout: auth mon ticket ttl = 660 # 11m 2023-11-06T02:14:13.367 INFO:teuthology.orchestra.run.smithi049.stdout: auth service ticket ttl = 240 # 4m 2023-11-06T02:14:13.367 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:13.368 INFO:teuthology.orchestra.run.smithi049.stdout: # don't complain about insecure global_id in the test suite 2023-11-06T02:14:13.368 INFO:teuthology.orchestra.run.smithi049.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-06T02:14:13.368 INFO:teuthology.orchestra.run.smithi049.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-06T02:14:13.368 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:13.368 INFO:teuthology.orchestra.run.smithi049.stdout: # 1m isn't quite enough 2023-11-06T02:14:13.368 INFO:teuthology.orchestra.run.smithi049.stdout: mon_down_mkfs_grace = 2m 2023-11-06T02:14:13.369 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:13.369 INFO:teuthology.orchestra.run.smithi049.stdout: mon_warn_on_filestore_osds = false 2023-11-06T02:14:13.369 INFO:teuthology.orchestra.run.smithi049.stdout: mon op complaint time = 120 2023-11-06T02:14:13.369 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:13.369 INFO:teuthology.orchestra.run.smithi049.stdout:[client] 2023-11-06T02:14:13.370 INFO:teuthology.orchestra.run.smithi049.stdout: rgw cache enabled = true 2023-11-06T02:14:13.370 INFO:teuthology.orchestra.run.smithi049.stdout: rgw enable ops log = true 2023-11-06T02:14:13.370 INFO:teuthology.orchestra.run.smithi049.stdout: rgw enable usage log = true 2023-11-06T02:14:13.370 INFO:teuthology.orchestra.run.smithi049.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-06T02:14:13.370 INFO:teuthology.orchestra.run.smithi049.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-06T02:14:13.371 INFO:teuthology.orchestra.run.smithi049.stdout: client mount timeout = 600 2023-11-06T02:14:13.371 INFO:teuthology.orchestra.run.smithi049.stdout: debug client = 20 2023-11-06T02:14:13.371 INFO:teuthology.orchestra.run.smithi049.stdout: debug ms = 1 2023-11-06T02:14:13.371 INFO:teuthology.orchestra.run.smithi049.stdout: pid file = "" 2023-11-06T02:14:13.371 INFO:teuthology.orchestra.run.smithi049.stdout: rados mon op timeout = 900 2023-11-06T02:14:13.371 INFO:teuthology.orchestra.run.smithi049.stdout: rados osd op timeout = 900 2023-11-06T02:14:13.372 INFO:teuthology.orchestra.run.smithi049.stdout:[mon.a] 2023-11-06T02:14:13.372 INFO:teuthology.orchestra.run.smithi049.stdout:[mon.b] 2023-11-06T02:14:13.372 INFO:teuthology.orchestra.run.smithi049.stdout:[mon.c] 2023-11-06T02:14:13.372 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.a] 2023-11-06T02:14:13.372 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.c] 2023-11-06T02:14:13.373 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.b] 2023-11-06T02:14:13.373 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.d] 2023-11-06T02:14:13.373 INFO:teuthology.orchestra.run.smithi049.stdout:[mds] 2023-11-06T02:14:13.373 INFO:teuthology.orchestra.run.smithi049.stdout: debug mds = 20 2023-11-06T02:14:13.373 INFO:teuthology.orchestra.run.smithi049.stdout: debug mds balancer = 20 2023-11-06T02:14:13.373 INFO:teuthology.orchestra.run.smithi049.stdout: debug ms = 1 2023-11-06T02:14:13.374 INFO:teuthology.orchestra.run.smithi049.stdout: mds debug frag = True 2023-11-06T02:14:13.374 INFO:teuthology.orchestra.run.smithi049.stdout: mds debug scatterstat = True 2023-11-06T02:14:13.374 INFO:teuthology.orchestra.run.smithi049.stdout: mds op complaint time = 180 2023-11-06T02:14:13.374 INFO:teuthology.orchestra.run.smithi049.stdout: mds verify scatter = False 2023-11-06T02:14:13.374 INFO:teuthology.orchestra.run.smithi049.stdout: osd op complaint time = 180 2023-11-06T02:14:13.375 INFO:teuthology.orchestra.run.smithi049.stdout: rados mon op timeout = 900 2023-11-06T02:14:13.375 INFO:teuthology.orchestra.run.smithi049.stdout: rados osd op timeout = 900 2023-11-06T02:14:13.376 INFO:teuthology.orchestra.run.smithi150.stdout:[global] 2023-11-06T02:14:13.377 INFO:teuthology.orchestra.run.smithi150.stdout: chdir = "" 2023-11-06T02:14:13.377 INFO:teuthology.orchestra.run.smithi150.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-06T02:14:13.377 INFO:teuthology.orchestra.run.smithi150.stdout: auth supported = cephx 2023-11-06T02:14:13.377 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:13.377 INFO:teuthology.orchestra.run.smithi150.stdout: filestore xattr use omap = true 2023-11-06T02:14:13.378 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:13.378 INFO:teuthology.orchestra.run.smithi150.stdout: mon clock drift allowed = 1.000 2023-11-06T02:14:13.378 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:13.378 INFO:teuthology.orchestra.run.smithi150.stdout: osd crush chooseleaf type = 0 2023-11-06T02:14:13.379 INFO:teuthology.orchestra.run.smithi150.stdout: auth debug = true 2023-11-06T02:14:13.379 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:13.379 INFO:teuthology.orchestra.run.smithi150.stdout: ms die on old message = true 2023-11-06T02:14:13.379 INFO:teuthology.orchestra.run.smithi150.stdout: ms die on bug = true 2023-11-06T02:14:13.379 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:13.379 INFO:teuthology.orchestra.run.smithi150.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-06T02:14:13.380 INFO:teuthology.orchestra.run.smithi150.stdout: mon pg warn max object skew = 0 2023-11-06T02:14:13.380 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:13.380 INFO:teuthology.orchestra.run.smithi150.stdout: # disable pg_autoscaler by default for new pools 2023-11-06T02:14:13.380 INFO:teuthology.orchestra.run.smithi150.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-06T02:14:13.380 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:13.381 INFO:teuthology.orchestra.run.smithi150.stdout: osd pool default size = 2 2023-11-06T02:14:13.381 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:13.381 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd allow primary affinity = true 2023-11-06T02:14:13.381 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd allow pg remap = true 2023-11-06T02:14:13.381 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on legacy crush tunables = false 2023-11-06T02:14:13.382 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on crush straw calc version zero = false 2023-11-06T02:14:13.382 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on no sortbitwise = false 2023-11-06T02:14:13.382 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on osd down out interval zero = false 2023-11-06T02:14:13.382 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on too few osds = false 2023-11-06T02:14:13.382 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-06T02:14:13.382 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-06T02:14:13.383 INFO:teuthology.orchestra.run.smithi150.stdout: mon_allow_pool_size_one = true 2023-11-06T02:14:13.383 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:13.383 INFO:teuthology.orchestra.run.smithi150.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-06T02:14:13.383 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:13.383 INFO:teuthology.orchestra.run.smithi150.stdout: osd default data pool replay window = 5 2023-11-06T02:14:13.384 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:13.384 INFO:teuthology.orchestra.run.smithi150.stdout: mon allow pool delete = true 2023-11-06T02:14:13.384 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:13.384 INFO:teuthology.orchestra.run.smithi150.stdout: mon cluster log file level = debug 2023-11-06T02:14:13.384 INFO:teuthology.orchestra.run.smithi150.stdout: debug asserts on shutdown = true 2023-11-06T02:14:13.385 INFO:teuthology.orchestra.run.smithi150.stdout: mon health detail to clog = false 2023-11-06T02:14:13.385 INFO:teuthology.orchestra.run.smithi150.stdout: mon host = "172.21.15.49,172.21.15.150,[v2:172.21.15.150:3301,v1:172.21.15.150:6790]" 2023-11-06T02:14:13.385 INFO:teuthology.orchestra.run.smithi150.stdout: fsid = 1a8b035c-6ba4-499a-b876-aea47fd9c0d4 2023-11-06T02:14:13.385 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:13.385 INFO:teuthology.orchestra.run.smithi150.stdout:[osd] 2023-11-06T02:14:13.385 INFO:teuthology.orchestra.run.smithi150.stdout: osd journal size = 100 2023-11-06T02:14:13.386 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:13.386 INFO:teuthology.orchestra.run.smithi150.stdout: osd scrub load threshold = 5.0 2023-11-06T02:14:13.386 INFO:teuthology.orchestra.run.smithi150.stdout: osd scrub max interval = 600 2023-11-06T02:14:13.386 INFO:teuthology.orchestra.run.smithi150.stdout: osd mclock profile = high_recovery_ops 2023-11-06T02:14:13.386 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:13.387 INFO:teuthology.orchestra.run.smithi150.stdout: osd recover clone overlap = true 2023-11-06T02:14:13.387 INFO:teuthology.orchestra.run.smithi150.stdout: osd recovery max chunk = 1048576 2023-11-06T02:14:13.387 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:13.387 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug shutdown = true 2023-11-06T02:14:13.387 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug op order = true 2023-11-06T02:14:13.388 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug verify stray on activate = true 2023-11-06T02:14:13.388 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:13.388 INFO:teuthology.orchestra.run.smithi150.stdout: osd open classes on start = true 2023-11-06T02:14:13.388 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug pg log writeout = true 2023-11-06T02:14:13.388 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:13.388 INFO:teuthology.orchestra.run.smithi150.stdout: osd deep scrub update digest min age = 30 2023-11-06T02:14:13.389 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:13.389 INFO:teuthology.orchestra.run.smithi150.stdout: osd map max advance = 10 2023-11-06T02:14:13.389 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:13.389 INFO:teuthology.orchestra.run.smithi150.stdout: journal zero on create = true 2023-11-06T02:14:13.389 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:13.390 INFO:teuthology.orchestra.run.smithi150.stdout: filestore ondisk finisher threads = 3 2023-11-06T02:14:13.390 INFO:teuthology.orchestra.run.smithi150.stdout: filestore apply finisher threads = 3 2023-11-06T02:14:13.390 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:13.390 INFO:teuthology.orchestra.run.smithi150.stdout: bdev debug aio = true 2023-11-06T02:14:13.390 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug misdirected ops = true 2023-11-06T02:14:13.390 INFO:teuthology.orchestra.run.smithi150.stdout: bdev async discard = True 2023-11-06T02:14:13.391 INFO:teuthology.orchestra.run.smithi150.stdout: bdev enable discard = True 2023-11-06T02:14:13.391 INFO:teuthology.orchestra.run.smithi150.stdout: bluestore allocator = bitmap 2023-11-06T02:14:13.391 INFO:teuthology.orchestra.run.smithi150.stdout: bluestore block size = 96636764160 2023-11-06T02:14:13.391 INFO:teuthology.orchestra.run.smithi150.stdout: bluestore fsck on mount = True 2023-11-06T02:14:13.391 INFO:teuthology.orchestra.run.smithi150.stdout: debug bluefs = 1/20 2023-11-06T02:14:13.392 INFO:teuthology.orchestra.run.smithi150.stdout: debug bluestore = 1/20 2023-11-06T02:14:13.392 INFO:teuthology.orchestra.run.smithi150.stdout: debug ms = 1 2023-11-06T02:14:13.392 INFO:teuthology.orchestra.run.smithi150.stdout: debug osd = 20 2023-11-06T02:14:13.392 INFO:teuthology.orchestra.run.smithi150.stdout: debug rocksdb = 4/10 2023-11-06T02:14:13.392 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd backfillfull_ratio = 0.85 2023-11-06T02:14:13.392 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd full ratio = 0.9 2023-11-06T02:14:13.393 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd nearfull ratio = 0.8 2023-11-06T02:14:13.393 INFO:teuthology.orchestra.run.smithi150.stdout: osd failsafe full ratio = 0.95 2023-11-06T02:14:13.393 INFO:teuthology.orchestra.run.smithi150.stdout: osd objectstore = bluestore 2023-11-06T02:14:13.393 INFO:teuthology.orchestra.run.smithi150.stdout: osd op complaint time = 180 2023-11-06T02:14:13.393 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:13.394 INFO:teuthology.orchestra.run.smithi150.stdout:[mgr] 2023-11-06T02:14:13.394 INFO:teuthology.orchestra.run.smithi150.stdout: debug ms = 1 2023-11-06T02:14:13.394 INFO:teuthology.orchestra.run.smithi150.stdout: debug mgr = 20 2023-11-06T02:14:13.394 INFO:teuthology.orchestra.run.smithi150.stdout: debug mon = 20 2023-11-06T02:14:13.394 INFO:teuthology.orchestra.run.smithi150.stdout: debug auth = 20 2023-11-06T02:14:13.395 INFO:teuthology.orchestra.run.smithi150.stdout: mon reweight min pgs per osd = 4 2023-11-06T02:14:13.395 INFO:teuthology.orchestra.run.smithi150.stdout: mon reweight min bytes per osd = 10 2023-11-06T02:14:13.395 INFO:teuthology.orchestra.run.smithi150.stdout: mgr/telemetry/nag = false 2023-11-06T02:14:13.395 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:13.395 INFO:teuthology.orchestra.run.smithi150.stdout:[mon] 2023-11-06T02:14:13.395 INFO:teuthology.orchestra.run.smithi150.stdout: debug ms = 1 2023-11-06T02:14:13.396 INFO:teuthology.orchestra.run.smithi150.stdout: debug mon = 20 2023-11-06T02:14:13.396 INFO:teuthology.orchestra.run.smithi150.stdout: debug paxos = 20 2023-11-06T02:14:13.396 INFO:teuthology.orchestra.run.smithi150.stdout: debug auth = 20 2023-11-06T02:14:13.396 INFO:teuthology.orchestra.run.smithi150.stdout: mon data avail warn = 5 2023-11-06T02:14:13.396 INFO:teuthology.orchestra.run.smithi150.stdout: mon mgr mkfs grace = 240 2023-11-06T02:14:13.397 INFO:teuthology.orchestra.run.smithi150.stdout: mon reweight min pgs per osd = 4 2023-11-06T02:14:13.397 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd reporter subtree level = osd 2023-11-06T02:14:13.397 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd prime pg temp = true 2023-11-06T02:14:13.397 INFO:teuthology.orchestra.run.smithi150.stdout: mon reweight min bytes per osd = 10 2023-11-06T02:14:13.397 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:13.397 INFO:teuthology.orchestra.run.smithi150.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-06T02:14:13.398 INFO:teuthology.orchestra.run.smithi150.stdout: auth mon ticket ttl = 660 # 11m 2023-11-06T02:14:13.398 INFO:teuthology.orchestra.run.smithi150.stdout: auth service ticket ttl = 240 # 4m 2023-11-06T02:14:13.398 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:13.398 INFO:teuthology.orchestra.run.smithi150.stdout: # don't complain about insecure global_id in the test suite 2023-11-06T02:14:13.398 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-06T02:14:13.399 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-06T02:14:13.399 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:13.399 INFO:teuthology.orchestra.run.smithi150.stdout: # 1m isn't quite enough 2023-11-06T02:14:13.399 INFO:teuthology.orchestra.run.smithi150.stdout: mon_down_mkfs_grace = 2m 2023-11-06T02:14:13.399 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:13.399 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_filestore_osds = false 2023-11-06T02:14:13.400 INFO:teuthology.orchestra.run.smithi150.stdout: mon op complaint time = 120 2023-11-06T02:14:13.400 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:13.400 INFO:teuthology.orchestra.run.smithi150.stdout:[client] 2023-11-06T02:14:13.400 INFO:teuthology.orchestra.run.smithi150.stdout: rgw cache enabled = true 2023-11-06T02:14:13.400 INFO:teuthology.orchestra.run.smithi150.stdout: rgw enable ops log = true 2023-11-06T02:14:13.401 INFO:teuthology.orchestra.run.smithi150.stdout: rgw enable usage log = true 2023-11-06T02:14:13.401 INFO:teuthology.orchestra.run.smithi150.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-06T02:14:13.401 INFO:teuthology.orchestra.run.smithi150.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-06T02:14:13.401 INFO:teuthology.orchestra.run.smithi150.stdout: client mount timeout = 600 2023-11-06T02:14:13.401 INFO:teuthology.orchestra.run.smithi150.stdout: debug client = 20 2023-11-06T02:14:13.401 INFO:teuthology.orchestra.run.smithi150.stdout: debug ms = 1 2023-11-06T02:14:13.402 INFO:teuthology.orchestra.run.smithi150.stdout: pid file = "" 2023-11-06T02:14:13.402 INFO:teuthology.orchestra.run.smithi150.stdout: rados mon op timeout = 900 2023-11-06T02:14:13.402 INFO:teuthology.orchestra.run.smithi150.stdout: rados osd op timeout = 900 2023-11-06T02:14:13.402 INFO:teuthology.orchestra.run.smithi150.stdout:[mon.a] 2023-11-06T02:14:13.402 INFO:teuthology.orchestra.run.smithi150.stdout:[mon.b] 2023-11-06T02:14:13.403 INFO:teuthology.orchestra.run.smithi150.stdout:[mon.c] 2023-11-06T02:14:13.403 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.a] 2023-11-06T02:14:13.403 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.c] 2023-11-06T02:14:13.403 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.b] 2023-11-06T02:14:13.403 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.d] 2023-11-06T02:14:13.404 INFO:teuthology.orchestra.run.smithi150.stdout:[mds] 2023-11-06T02:14:13.404 INFO:teuthology.orchestra.run.smithi150.stdout: debug mds = 20 2023-11-06T02:14:13.404 INFO:teuthology.orchestra.run.smithi150.stdout: debug mds balancer = 20 2023-11-06T02:14:13.404 INFO:teuthology.orchestra.run.smithi150.stdout: debug ms = 1 2023-11-06T02:14:13.404 INFO:teuthology.orchestra.run.smithi150.stdout: mds debug frag = True 2023-11-06T02:14:13.404 INFO:teuthology.orchestra.run.smithi150.stdout: mds debug scatterstat = True 2023-11-06T02:14:13.405 INFO:teuthology.orchestra.run.smithi150.stdout: mds op complaint time = 180 2023-11-06T02:14:13.405 INFO:teuthology.orchestra.run.smithi150.stdout: mds verify scatter = False 2023-11-06T02:14:13.405 INFO:teuthology.orchestra.run.smithi150.stdout: osd op complaint time = 180 2023-11-06T02:14:13.405 INFO:teuthology.orchestra.run.smithi150.stdout: rados mon op timeout = 900 2023-11-06T02:14:13.405 INFO:teuthology.orchestra.run.smithi150.stdout: rados osd op timeout = 900 2023-11-06T02:14:13.406 DEBUG:teuthology.orchestra.run.smithi049:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-06T02:14:13.408 INFO:tasks.ceph.mds.c.smithi049.stdout:starting mds.c at 2023-11-06T02:14:13.409 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-06T02:14:13.433 INFO:teuthology.orchestra.run.smithi049.stdout:[global] 2023-11-06T02:14:13.433 INFO:teuthology.orchestra.run.smithi049.stdout: chdir = "" 2023-11-06T02:14:13.434 INFO:teuthology.orchestra.run.smithi049.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-06T02:14:13.434 INFO:teuthology.orchestra.run.smithi049.stdout: auth supported = cephx 2023-11-06T02:14:13.434 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:13.434 INFO:teuthology.orchestra.run.smithi049.stdout: filestore xattr use omap = true 2023-11-06T02:14:13.434 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:13.435 INFO:teuthology.orchestra.run.smithi049.stdout: mon clock drift allowed = 1.000 2023-11-06T02:14:13.435 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:13.435 INFO:teuthology.orchestra.run.smithi049.stdout: osd crush chooseleaf type = 0 2023-11-06T02:14:13.435 INFO:teuthology.orchestra.run.smithi049.stdout: auth debug = true 2023-11-06T02:14:13.435 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:13.436 INFO:teuthology.orchestra.run.smithi049.stdout: ms die on old message = true 2023-11-06T02:14:13.436 INFO:teuthology.orchestra.run.smithi049.stdout: ms die on bug = true 2023-11-06T02:14:13.436 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:13.436 INFO:teuthology.orchestra.run.smithi049.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-06T02:14:13.436 INFO:teuthology.orchestra.run.smithi049.stdout: mon pg warn max object skew = 0 2023-11-06T02:14:13.436 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:13.437 INFO:teuthology.orchestra.run.smithi049.stdout: # disable pg_autoscaler by default for new pools 2023-11-06T02:14:13.437 INFO:teuthology.orchestra.run.smithi049.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-06T02:14:13.437 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:13.437 INFO:teuthology.orchestra.run.smithi049.stdout: osd pool default size = 2 2023-11-06T02:14:13.437 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:13.438 INFO:teuthology.orchestra.run.smithi049.stdout: mon osd allow primary affinity = true 2023-11-06T02:14:13.438 INFO:teuthology.orchestra.run.smithi049.stdout: mon osd allow pg remap = true 2023-11-06T02:14:13.438 INFO:teuthology.orchestra.run.smithi049.stdout: mon warn on legacy crush tunables = false 2023-11-06T02:14:13.438 INFO:teuthology.orchestra.run.smithi049.stdout: mon warn on crush straw calc version zero = false 2023-11-06T02:14:13.438 INFO:teuthology.orchestra.run.smithi049.stdout: mon warn on no sortbitwise = false 2023-11-06T02:14:13.439 INFO:teuthology.orchestra.run.smithi049.stdout: mon warn on osd down out interval zero = false 2023-11-06T02:14:13.439 INFO:teuthology.orchestra.run.smithi049.stdout: mon warn on too few osds = false 2023-11-06T02:14:13.439 INFO:teuthology.orchestra.run.smithi049.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-06T02:14:13.439 INFO:teuthology.orchestra.run.smithi049.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-06T02:14:13.440 INFO:teuthology.orchestra.run.smithi049.stdout: mon_allow_pool_size_one = true 2023-11-06T02:14:13.440 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:13.440 INFO:teuthology.orchestra.run.smithi049.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-06T02:14:13.441 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:13.441 INFO:teuthology.orchestra.run.smithi049.stdout: osd default data pool replay window = 5 2023-11-06T02:14:13.441 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:13.441 INFO:teuthology.orchestra.run.smithi049.stdout: mon allow pool delete = true 2023-11-06T02:14:13.441 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:13.442 INFO:teuthology.orchestra.run.smithi049.stdout: mon cluster log file level = debug 2023-11-06T02:14:13.442 INFO:teuthology.orchestra.run.smithi049.stdout: debug asserts on shutdown = true 2023-11-06T02:14:13.442 INFO:teuthology.orchestra.run.smithi049.stdout: mon health detail to clog = false 2023-11-06T02:14:13.442 INFO:teuthology.orchestra.run.smithi049.stdout: mon host = "172.21.15.49,172.21.15.150,[v2:172.21.15.150:3301,v1:172.21.15.150:6790]" 2023-11-06T02:14:13.442 INFO:teuthology.orchestra.run.smithi049.stdout: fsid = 1a8b035c-6ba4-499a-b876-aea47fd9c0d4 2023-11-06T02:14:13.443 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:13.443 INFO:teuthology.orchestra.run.smithi049.stdout:[osd] 2023-11-06T02:14:13.443 INFO:teuthology.orchestra.run.smithi049.stdout: osd journal size = 100 2023-11-06T02:14:13.443 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:13.443 INFO:teuthology.orchestra.run.smithi049.stdout: osd scrub load threshold = 5.0 2023-11-06T02:14:13.444 INFO:teuthology.orchestra.run.smithi049.stdout: osd scrub max interval = 600 2023-11-06T02:14:13.444 INFO:teuthology.orchestra.run.smithi049.stdout: osd mclock profile = high_recovery_ops 2023-11-06T02:14:13.444 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:13.444 INFO:teuthology.orchestra.run.smithi049.stdout: osd recover clone overlap = true 2023-11-06T02:14:13.444 INFO:teuthology.orchestra.run.smithi049.stdout: osd recovery max chunk = 1048576 2023-11-06T02:14:13.444 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:13.445 INFO:teuthology.orchestra.run.smithi049.stdout: osd debug shutdown = true 2023-11-06T02:14:13.445 INFO:teuthology.orchestra.run.smithi049.stdout: osd debug op order = true 2023-11-06T02:14:13.445 INFO:teuthology.orchestra.run.smithi049.stdout: osd debug verify stray on activate = true 2023-11-06T02:14:13.445 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:13.445 INFO:teuthology.orchestra.run.smithi049.stdout: osd open classes on start = true 2023-11-06T02:14:13.446 INFO:teuthology.orchestra.run.smithi049.stdout: osd debug pg log writeout = true 2023-11-06T02:14:13.446 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:13.446 INFO:teuthology.orchestra.run.smithi049.stdout: osd deep scrub update digest min age = 30 2023-11-06T02:14:13.446 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:13.446 INFO:teuthology.orchestra.run.smithi049.stdout: osd map max advance = 10 2023-11-06T02:14:13.447 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:13.447 INFO:teuthology.orchestra.run.smithi049.stdout: journal zero on create = true 2023-11-06T02:14:13.447 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:13.447 INFO:teuthology.orchestra.run.smithi049.stdout: filestore ondisk finisher threads = 3 2023-11-06T02:14:13.447 INFO:teuthology.orchestra.run.smithi049.stdout: filestore apply finisher threads = 3 2023-11-06T02:14:13.447 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:13.448 INFO:teuthology.orchestra.run.smithi049.stdout: bdev debug aio = true 2023-11-06T02:14:13.448 INFO:teuthology.orchestra.run.smithi049.stdout: osd debug misdirected ops = true 2023-11-06T02:14:13.448 INFO:teuthology.orchestra.run.smithi049.stdout: bdev async discard = True 2023-11-06T02:14:13.448 INFO:teuthology.orchestra.run.smithi049.stdout: bdev enable discard = True 2023-11-06T02:14:13.448 INFO:teuthology.orchestra.run.smithi049.stdout: bluestore allocator = bitmap 2023-11-06T02:14:13.449 INFO:teuthology.orchestra.run.smithi049.stdout: bluestore block size = 96636764160 2023-11-06T02:14:13.449 INFO:teuthology.orchestra.run.smithi049.stdout: bluestore fsck on mount = True 2023-11-06T02:14:13.449 INFO:teuthology.orchestra.run.smithi049.stdout: debug bluefs = 1/20 2023-11-06T02:14:13.449 INFO:teuthology.orchestra.run.smithi049.stdout: debug bluestore = 1/20 2023-11-06T02:14:13.449 INFO:teuthology.orchestra.run.smithi049.stdout: debug ms = 1 2023-11-06T02:14:13.450 INFO:teuthology.orchestra.run.smithi049.stdout: debug osd = 20 2023-11-06T02:14:13.450 INFO:teuthology.orchestra.run.smithi049.stdout: debug rocksdb = 4/10 2023-11-06T02:14:13.450 INFO:teuthology.orchestra.run.smithi049.stdout: mon osd backfillfull_ratio = 0.85 2023-11-06T02:14:13.450 INFO:teuthology.orchestra.run.smithi049.stdout: mon osd full ratio = 0.9 2023-11-06T02:14:13.450 INFO:teuthology.orchestra.run.smithi049.stdout: mon osd nearfull ratio = 0.8 2023-11-06T02:14:13.450 INFO:teuthology.orchestra.run.smithi049.stdout: osd failsafe full ratio = 0.95 2023-11-06T02:14:13.451 INFO:teuthology.orchestra.run.smithi049.stdout: osd objectstore = bluestore 2023-11-06T02:14:13.451 INFO:teuthology.orchestra.run.smithi049.stdout: osd op complaint time = 180 2023-11-06T02:14:13.451 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:13.451 INFO:teuthology.orchestra.run.smithi049.stdout:[mgr] 2023-11-06T02:14:13.451 INFO:teuthology.orchestra.run.smithi049.stdout: debug ms = 1 2023-11-06T02:14:13.452 INFO:teuthology.orchestra.run.smithi049.stdout: debug mgr = 20 2023-11-06T02:14:13.452 INFO:teuthology.orchestra.run.smithi049.stdout: debug mon = 20 2023-11-06T02:14:13.452 INFO:teuthology.orchestra.run.smithi049.stdout: debug auth = 20 2023-11-06T02:14:13.452 INFO:teuthology.orchestra.run.smithi049.stdout: mon reweight min pgs per osd = 4 2023-11-06T02:14:13.452 INFO:teuthology.orchestra.run.smithi049.stdout: mon reweight min bytes per osd = 10 2023-11-06T02:14:13.452 INFO:teuthology.orchestra.run.smithi049.stdout: mgr/telemetry/nag = false 2023-11-06T02:14:13.453 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:13.453 INFO:teuthology.orchestra.run.smithi049.stdout:[mon] 2023-11-06T02:14:13.453 INFO:teuthology.orchestra.run.smithi049.stdout: debug ms = 1 2023-11-06T02:14:13.453 INFO:teuthology.orchestra.run.smithi049.stdout: debug mon = 20 2023-11-06T02:14:13.453 INFO:teuthology.orchestra.run.smithi049.stdout: debug paxos = 20 2023-11-06T02:14:13.454 INFO:teuthology.orchestra.run.smithi049.stdout: debug auth = 20 2023-11-06T02:14:13.454 INFO:teuthology.orchestra.run.smithi049.stdout: mon data avail warn = 5 2023-11-06T02:14:13.454 INFO:teuthology.orchestra.run.smithi049.stdout: mon mgr mkfs grace = 240 2023-11-06T02:14:13.454 INFO:teuthology.orchestra.run.smithi049.stdout: mon reweight min pgs per osd = 4 2023-11-06T02:14:13.454 INFO:teuthology.orchestra.run.smithi049.stdout: mon osd reporter subtree level = osd 2023-11-06T02:14:13.455 INFO:teuthology.orchestra.run.smithi049.stdout: mon osd prime pg temp = true 2023-11-06T02:14:13.455 INFO:teuthology.orchestra.run.smithi049.stdout: mon reweight min bytes per osd = 10 2023-11-06T02:14:13.455 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:13.455 INFO:teuthology.orchestra.run.smithi049.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-06T02:14:13.455 INFO:teuthology.orchestra.run.smithi049.stdout: auth mon ticket ttl = 660 # 11m 2023-11-06T02:14:13.455 INFO:teuthology.orchestra.run.smithi049.stdout: auth service ticket ttl = 240 # 4m 2023-11-06T02:14:13.456 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:13.456 INFO:teuthology.orchestra.run.smithi049.stdout: # don't complain about insecure global_id in the test suite 2023-11-06T02:14:13.456 INFO:teuthology.orchestra.run.smithi049.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-06T02:14:13.456 INFO:teuthology.orchestra.run.smithi049.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-06T02:14:13.456 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:13.457 INFO:teuthology.orchestra.run.smithi049.stdout: # 1m isn't quite enough 2023-11-06T02:14:13.457 INFO:teuthology.orchestra.run.smithi049.stdout: mon_down_mkfs_grace = 2m 2023-11-06T02:14:13.457 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:13.457 INFO:teuthology.orchestra.run.smithi049.stdout: mon_warn_on_filestore_osds = false 2023-11-06T02:14:13.457 INFO:teuthology.orchestra.run.smithi049.stdout: mon op complaint time = 120 2023-11-06T02:14:13.458 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:13.458 INFO:teuthology.orchestra.run.smithi049.stdout:[client] 2023-11-06T02:14:13.458 INFO:teuthology.orchestra.run.smithi049.stdout: rgw cache enabled = true 2023-11-06T02:14:13.458 INFO:teuthology.orchestra.run.smithi049.stdout: rgw enable ops log = true 2023-11-06T02:14:13.458 INFO:teuthology.orchestra.run.smithi049.stdout: rgw enable usage log = true 2023-11-06T02:14:13.458 INFO:teuthology.orchestra.run.smithi049.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-06T02:14:13.459 INFO:teuthology.orchestra.run.smithi049.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-06T02:14:13.459 INFO:teuthology.orchestra.run.smithi049.stdout: client mount timeout = 600 2023-11-06T02:14:13.459 INFO:teuthology.orchestra.run.smithi049.stdout: debug client = 20 2023-11-06T02:14:13.459 INFO:teuthology.orchestra.run.smithi049.stdout: debug ms = 1 2023-11-06T02:14:13.459 INFO:teuthology.orchestra.run.smithi049.stdout: pid file = "" 2023-11-06T02:14:13.460 INFO:teuthology.orchestra.run.smithi049.stdout: rados mon op timeout = 900 2023-11-06T02:14:13.460 INFO:teuthology.orchestra.run.smithi049.stdout: rados osd op timeout = 900 2023-11-06T02:14:13.460 INFO:teuthology.orchestra.run.smithi049.stdout:[mon.a] 2023-11-06T02:14:13.460 INFO:teuthology.orchestra.run.smithi049.stdout:[mon.b] 2023-11-06T02:14:13.460 INFO:teuthology.orchestra.run.smithi049.stdout:[mon.c] 2023-11-06T02:14:13.461 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.a] 2023-11-06T02:14:13.461 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.c] 2023-11-06T02:14:13.461 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.b] 2023-11-06T02:14:13.461 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.d] 2023-11-06T02:14:13.461 INFO:teuthology.orchestra.run.smithi049.stdout:[mds] 2023-11-06T02:14:13.461 INFO:teuthology.orchestra.run.smithi049.stdout: debug mds = 20 2023-11-06T02:14:13.462 INFO:teuthology.orchestra.run.smithi049.stdout: debug mds balancer = 20 2023-11-06T02:14:13.462 INFO:teuthology.orchestra.run.smithi049.stdout: debug ms = 1 2023-11-06T02:14:13.462 INFO:teuthology.orchestra.run.smithi049.stdout: mds debug frag = True 2023-11-06T02:14:13.462 INFO:teuthology.orchestra.run.smithi049.stdout: mds debug scatterstat = False 2023-11-06T02:14:13.462 INFO:teuthology.orchestra.run.smithi049.stdout: mds op complaint time = 180 2023-11-06T02:14:13.463 INFO:teuthology.orchestra.run.smithi049.stdout: mds verify scatter = False 2023-11-06T02:14:13.463 INFO:teuthology.orchestra.run.smithi049.stdout: osd op complaint time = 180 2023-11-06T02:14:13.463 INFO:teuthology.orchestra.run.smithi049.stdout: rados mon op timeout = 900 2023-11-06T02:14:13.463 INFO:teuthology.orchestra.run.smithi049.stdout: rados osd op timeout = 900 2023-11-06T02:14:13.465 INFO:teuthology.orchestra.run.smithi150.stdout:[global] 2023-11-06T02:14:13.465 INFO:teuthology.orchestra.run.smithi150.stdout: chdir = "" 2023-11-06T02:14:13.465 INFO:teuthology.orchestra.run.smithi150.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-06T02:14:13.465 INFO:teuthology.orchestra.run.smithi150.stdout: auth supported = cephx 2023-11-06T02:14:13.466 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:13.466 INFO:teuthology.orchestra.run.smithi150.stdout: filestore xattr use omap = true 2023-11-06T02:14:13.466 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:13.466 INFO:teuthology.orchestra.run.smithi150.stdout: mon clock drift allowed = 1.000 2023-11-06T02:14:13.466 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:13.467 INFO:teuthology.orchestra.run.smithi150.stdout: osd crush chooseleaf type = 0 2023-11-06T02:14:13.467 INFO:teuthology.orchestra.run.smithi150.stdout: auth debug = true 2023-11-06T02:14:13.467 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:13.467 INFO:teuthology.orchestra.run.smithi150.stdout: ms die on old message = true 2023-11-06T02:14:13.467 INFO:teuthology.orchestra.run.smithi150.stdout: ms die on bug = true 2023-11-06T02:14:13.468 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:13.468 INFO:teuthology.orchestra.run.smithi150.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-06T02:14:13.468 INFO:teuthology.orchestra.run.smithi150.stdout: mon pg warn max object skew = 0 2023-11-06T02:14:13.468 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:13.468 INFO:teuthology.orchestra.run.smithi150.stdout: # disable pg_autoscaler by default for new pools 2023-11-06T02:14:13.469 INFO:teuthology.orchestra.run.smithi150.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-06T02:14:13.469 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:13.469 INFO:teuthology.orchestra.run.smithi150.stdout: osd pool default size = 2 2023-11-06T02:14:13.469 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:13.469 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd allow primary affinity = true 2023-11-06T02:14:13.469 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd allow pg remap = true 2023-11-06T02:14:13.470 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on legacy crush tunables = false 2023-11-06T02:14:13.470 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on crush straw calc version zero = false 2023-11-06T02:14:13.470 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on no sortbitwise = false 2023-11-06T02:14:13.470 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on osd down out interval zero = false 2023-11-06T02:14:13.470 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on too few osds = false 2023-11-06T02:14:13.471 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-06T02:14:13.471 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-06T02:14:13.471 INFO:teuthology.orchestra.run.smithi150.stdout: mon_allow_pool_size_one = true 2023-11-06T02:14:13.471 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:13.471 INFO:teuthology.orchestra.run.smithi150.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-06T02:14:13.472 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:13.472 INFO:teuthology.orchestra.run.smithi150.stdout: osd default data pool replay window = 5 2023-11-06T02:14:13.472 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:13.472 INFO:teuthology.orchestra.run.smithi150.stdout: mon allow pool delete = true 2023-11-06T02:14:13.472 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:13.472 INFO:teuthology.orchestra.run.smithi150.stdout: mon cluster log file level = debug 2023-11-06T02:14:13.473 INFO:teuthology.orchestra.run.smithi150.stdout: debug asserts on shutdown = true 2023-11-06T02:14:13.473 INFO:teuthology.orchestra.run.smithi150.stdout: mon health detail to clog = false 2023-11-06T02:14:13.473 INFO:teuthology.orchestra.run.smithi150.stdout: mon host = "172.21.15.49,172.21.15.150,[v2:172.21.15.150:3301,v1:172.21.15.150:6790]" 2023-11-06T02:14:13.473 INFO:teuthology.orchestra.run.smithi150.stdout: fsid = 1a8b035c-6ba4-499a-b876-aea47fd9c0d4 2023-11-06T02:14:13.473 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:13.474 INFO:teuthology.orchestra.run.smithi150.stdout:[osd] 2023-11-06T02:14:13.474 INFO:teuthology.orchestra.run.smithi150.stdout: osd journal size = 100 2023-11-06T02:14:13.474 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:13.474 INFO:teuthology.orchestra.run.smithi150.stdout: osd scrub load threshold = 5.0 2023-11-06T02:14:13.475 INFO:teuthology.orchestra.run.smithi150.stdout: osd scrub max interval = 600 2023-11-06T02:14:13.475 INFO:teuthology.orchestra.run.smithi150.stdout: osd mclock profile = high_recovery_ops 2023-11-06T02:14:13.475 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:13.475 INFO:teuthology.orchestra.run.smithi150.stdout: osd recover clone overlap = true 2023-11-06T02:14:13.475 INFO:teuthology.orchestra.run.smithi150.stdout: osd recovery max chunk = 1048576 2023-11-06T02:14:13.475 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:13.476 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug shutdown = true 2023-11-06T02:14:13.476 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug op order = true 2023-11-06T02:14:13.476 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug verify stray on activate = true 2023-11-06T02:14:13.476 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:13.476 INFO:teuthology.orchestra.run.smithi150.stdout: osd open classes on start = true 2023-11-06T02:14:13.477 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug pg log writeout = true 2023-11-06T02:14:13.477 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:13.477 INFO:teuthology.orchestra.run.smithi150.stdout: osd deep scrub update digest min age = 30 2023-11-06T02:14:13.477 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:13.477 INFO:teuthology.orchestra.run.smithi150.stdout: osd map max advance = 10 2023-11-06T02:14:13.477 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:13.478 INFO:teuthology.orchestra.run.smithi150.stdout: journal zero on create = true 2023-11-06T02:14:13.478 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:13.478 INFO:teuthology.orchestra.run.smithi150.stdout: filestore ondisk finisher threads = 3 2023-11-06T02:14:13.478 INFO:teuthology.orchestra.run.smithi150.stdout: filestore apply finisher threads = 3 2023-11-06T02:14:13.478 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:13.479 INFO:teuthology.orchestra.run.smithi150.stdout: bdev debug aio = true 2023-11-06T02:14:13.479 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug misdirected ops = true 2023-11-06T02:14:13.479 INFO:teuthology.orchestra.run.smithi150.stdout: bdev async discard = True 2023-11-06T02:14:13.479 INFO:teuthology.orchestra.run.smithi150.stdout: bdev enable discard = True 2023-11-06T02:14:13.479 INFO:teuthology.orchestra.run.smithi150.stdout: bluestore allocator = bitmap 2023-11-06T02:14:13.479 INFO:teuthology.orchestra.run.smithi150.stdout: bluestore block size = 96636764160 2023-11-06T02:14:13.480 INFO:teuthology.orchestra.run.smithi150.stdout: bluestore fsck on mount = True 2023-11-06T02:14:13.480 INFO:teuthology.orchestra.run.smithi150.stdout: debug bluefs = 1/20 2023-11-06T02:14:13.480 INFO:teuthology.orchestra.run.smithi150.stdout: debug bluestore = 1/20 2023-11-06T02:14:13.480 INFO:teuthology.orchestra.run.smithi150.stdout: debug ms = 1 2023-11-06T02:14:13.480 INFO:teuthology.orchestra.run.smithi150.stdout: debug osd = 20 2023-11-06T02:14:13.481 INFO:teuthology.orchestra.run.smithi150.stdout: debug rocksdb = 4/10 2023-11-06T02:14:13.481 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd backfillfull_ratio = 0.85 2023-11-06T02:14:13.481 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd full ratio = 0.9 2023-11-06T02:14:13.481 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd nearfull ratio = 0.8 2023-11-06T02:14:13.481 INFO:teuthology.orchestra.run.smithi150.stdout: osd failsafe full ratio = 0.95 2023-11-06T02:14:13.482 INFO:teuthology.orchestra.run.smithi150.stdout: osd objectstore = bluestore 2023-11-06T02:14:13.482 INFO:teuthology.orchestra.run.smithi150.stdout: osd op complaint time = 180 2023-11-06T02:14:13.482 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:13.482 INFO:teuthology.orchestra.run.smithi150.stdout:[mgr] 2023-11-06T02:14:13.482 INFO:teuthology.orchestra.run.smithi150.stdout: debug ms = 1 2023-11-06T02:14:13.482 INFO:teuthology.orchestra.run.smithi150.stdout: debug mgr = 20 2023-11-06T02:14:13.483 INFO:teuthology.orchestra.run.smithi150.stdout: debug mon = 20 2023-11-06T02:14:13.483 INFO:teuthology.orchestra.run.smithi150.stdout: debug auth = 20 2023-11-06T02:14:13.483 INFO:teuthology.orchestra.run.smithi150.stdout: mon reweight min pgs per osd = 4 2023-11-06T02:14:13.483 INFO:teuthology.orchestra.run.smithi150.stdout: mon reweight min bytes per osd = 10 2023-11-06T02:14:13.483 INFO:teuthology.orchestra.run.smithi150.stdout: mgr/telemetry/nag = false 2023-11-06T02:14:13.484 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:13.484 INFO:teuthology.orchestra.run.smithi150.stdout:[mon] 2023-11-06T02:14:13.484 INFO:teuthology.orchestra.run.smithi150.stdout: debug ms = 1 2023-11-06T02:14:13.484 INFO:teuthology.orchestra.run.smithi150.stdout: debug mon = 20 2023-11-06T02:14:13.484 INFO:teuthology.orchestra.run.smithi150.stdout: debug paxos = 20 2023-11-06T02:14:13.485 INFO:teuthology.orchestra.run.smithi150.stdout: debug auth = 20 2023-11-06T02:14:13.485 INFO:teuthology.orchestra.run.smithi150.stdout: mon data avail warn = 5 2023-11-06T02:14:13.485 INFO:teuthology.orchestra.run.smithi150.stdout: mon mgr mkfs grace = 240 2023-11-06T02:14:13.485 INFO:teuthology.orchestra.run.smithi150.stdout: mon reweight min pgs per osd = 4 2023-11-06T02:14:13.485 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd reporter subtree level = osd 2023-11-06T02:14:13.486 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd prime pg temp = true 2023-11-06T02:14:13.486 INFO:teuthology.orchestra.run.smithi150.stdout: mon reweight min bytes per osd = 10 2023-11-06T02:14:13.486 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:13.486 INFO:teuthology.orchestra.run.smithi150.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-06T02:14:13.486 INFO:teuthology.orchestra.run.smithi150.stdout: auth mon ticket ttl = 660 # 11m 2023-11-06T02:14:13.487 INFO:teuthology.orchestra.run.smithi150.stdout: auth service ticket ttl = 240 # 4m 2023-11-06T02:14:13.487 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:13.487 INFO:teuthology.orchestra.run.smithi150.stdout: # don't complain about insecure global_id in the test suite 2023-11-06T02:14:13.487 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-06T02:14:13.487 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-06T02:14:13.487 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:13.488 INFO:teuthology.orchestra.run.smithi150.stdout: # 1m isn't quite enough 2023-11-06T02:14:13.488 INFO:teuthology.orchestra.run.smithi150.stdout: mon_down_mkfs_grace = 2m 2023-11-06T02:14:13.488 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:13.488 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_filestore_osds = false 2023-11-06T02:14:13.488 INFO:teuthology.orchestra.run.smithi150.stdout: mon op complaint time = 120 2023-11-06T02:14:13.489 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:13.489 INFO:teuthology.orchestra.run.smithi150.stdout:[client] 2023-11-06T02:14:13.489 INFO:teuthology.orchestra.run.smithi150.stdout: rgw cache enabled = true 2023-11-06T02:14:13.489 INFO:teuthology.orchestra.run.smithi150.stdout: rgw enable ops log = true 2023-11-06T02:14:13.489 INFO:teuthology.orchestra.run.smithi150.stdout: rgw enable usage log = true 2023-11-06T02:14:13.489 INFO:teuthology.orchestra.run.smithi150.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-06T02:14:13.490 INFO:teuthology.orchestra.run.smithi150.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-06T02:14:13.490 INFO:teuthology.orchestra.run.smithi150.stdout: client mount timeout = 600 2023-11-06T02:14:13.490 INFO:teuthology.orchestra.run.smithi150.stdout: debug client = 20 2023-11-06T02:14:13.490 INFO:teuthology.orchestra.run.smithi150.stdout: debug ms = 1 2023-11-06T02:14:13.490 INFO:teuthology.orchestra.run.smithi150.stdout: pid file = "" 2023-11-06T02:14:13.491 INFO:teuthology.orchestra.run.smithi150.stdout: rados mon op timeout = 900 2023-11-06T02:14:13.491 INFO:teuthology.orchestra.run.smithi150.stdout: rados osd op timeout = 900 2023-11-06T02:14:13.491 INFO:teuthology.orchestra.run.smithi150.stdout:[mon.a] 2023-11-06T02:14:13.491 INFO:teuthology.orchestra.run.smithi150.stdout:[mon.b] 2023-11-06T02:14:13.491 INFO:teuthology.orchestra.run.smithi150.stdout:[mon.c] 2023-11-06T02:14:13.492 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.a] 2023-11-06T02:14:13.492 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.c] 2023-11-06T02:14:13.492 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.b] 2023-11-06T02:14:13.492 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.d] 2023-11-06T02:14:13.492 INFO:teuthology.orchestra.run.smithi150.stdout:[mds] 2023-11-06T02:14:13.492 INFO:teuthology.orchestra.run.smithi150.stdout: debug mds = 20 2023-11-06T02:14:13.493 INFO:teuthology.orchestra.run.smithi150.stdout: debug mds balancer = 20 2023-11-06T02:14:13.493 INFO:teuthology.orchestra.run.smithi150.stdout: debug ms = 1 2023-11-06T02:14:13.493 INFO:teuthology.orchestra.run.smithi150.stdout: mds debug frag = True 2023-11-06T02:14:13.493 INFO:teuthology.orchestra.run.smithi150.stdout: mds debug scatterstat = False 2023-11-06T02:14:13.493 INFO:teuthology.orchestra.run.smithi150.stdout: mds op complaint time = 180 2023-11-06T02:14:13.493 INFO:teuthology.orchestra.run.smithi150.stdout: mds verify scatter = False 2023-11-06T02:14:13.494 INFO:teuthology.orchestra.run.smithi150.stdout: osd op complaint time = 180 2023-11-06T02:14:13.494 INFO:teuthology.orchestra.run.smithi150.stdout: rados mon op timeout = 900 2023-11-06T02:14:13.494 INFO:teuthology.orchestra.run.smithi150.stdout: rados osd op timeout = 900 2023-11-06T02:14:13.495 DEBUG:teuthology.orchestra.run.smithi049:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmomapkey 10000000001.00000000 bravo_head 2023-11-06T02:14:13.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.554+0000 7f937b008080 1 Processor -- start 2023-11-06T02:14:13.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.554+0000 7f937b008080 1 -- start start 2023-11-06T02:14:13.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.554+0000 7f937b008080 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e34d2cfa10 0x55e34d374db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:13.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.554+0000 7f937b008080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e34d2ebe20 0x55e34d3752f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:13.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.554+0000 7f937b008080 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e34d3784c0 0x55e34d372e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:13.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.554+0000 7f937b008080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e34d1ea450 con 0x55e34d2ebe20 2023-11-06T02:14:13.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.554+0000 7f937b008080 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e34d1ea150 con 0x55e34d2cfa10 2023-11-06T02:14:13.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.554+0000 7f937b008080 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e34d1e0dc0 con 0x55e34d3784c0 2023-11-06T02:14:13.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.554+0000 7f9378321640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e34d2ebe20 0x55e34d3752f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:13.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.554+0000 7f9378321640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e34d2ebe20 0x55e34d3752f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55412/0 (socket says 172.21.15.49:55412) 2023-11-06T02:14:13.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.554+0000 7f9378321640 1 -- 172.21.15.49:0/2684436404 learned_addr learned my addr 172.21.15.49:0/2684436404 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:13.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.554+0000 7f9378b22640 1 --2- 172.21.15.49:0/2684436404 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e34d2cfa10 0x55e34d374db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:13.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.554+0000 7f9379323640 1 --2- 172.21.15.49:0/2684436404 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e34d3784c0 0x55e34d372e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:13.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.554+0000 7f9378321640 1 -- 172.21.15.49:0/2684436404 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e34d3784c0 msgr2=0x55e34d372e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:13.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.554+0000 7f9378321640 1 --2- 172.21.15.49:0/2684436404 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e34d3784c0 0x55e34d372e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:13.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.554+0000 7f9378321640 1 -- 172.21.15.49:0/2684436404 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e34d2cfa10 msgr2=0x55e34d374db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:13.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.554+0000 7f9378321640 1 --2- 172.21.15.49:0/2684436404 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e34d2cfa10 0x55e34d374db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:13.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.554+0000 7f9378321640 1 -- 172.21.15.49:0/2684436404 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f937000a1c0 con 0x55e34d2ebe20 2023-11-06T02:14:13.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.554+0000 7f9379323640 1 --2- 172.21.15.49:0/2684436404 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e34d3784c0 0x55e34d372e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:14:13.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.554+0000 7f9378b22640 1 --2- 172.21.15.49:0/2684436404 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e34d2cfa10 0x55e34d374db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:14:13.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.554+0000 7f9378321640 1 --2- 172.21.15.49:0/2684436404 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e34d2ebe20 0x55e34d3752f0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f936c009950 tx=0x7f936c005a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c82dc7b2ca6f6eeb server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:13.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.554+0000 7f937731f640 1 -- 172.21.15.49:0/2684436404 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f936c00f040 con 0x55e34d2ebe20 2023-11-06T02:14:13.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.554+0000 7f937731f640 1 -- 172.21.15.49:0/2684436404 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f936c008da0 con 0x55e34d2ebe20 2023-11-06T02:14:13.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.554+0000 7f937731f640 1 -- 172.21.15.49:0/2684436404 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f936c013640 con 0x55e34d2ebe20 2023-11-06T02:14:13.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.554+0000 7f937b008080 1 -- 172.21.15.49:0/2684436404 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e34d2ebe20 msgr2=0x55e34d3752f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:13.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.554+0000 7f937b008080 1 --2- 172.21.15.49:0/2684436404 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e34d2ebe20 0x55e34d3752f0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f936c009950 tx=0x7f936c005a10 comp rx=0 tx=0).stop 2023-11-06T02:14:13.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.554+0000 7f937b008080 1 -- 172.21.15.49:0/2684436404 shutdown_connections 2023-11-06T02:14:13.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.554+0000 7f937b008080 1 --2- 172.21.15.49:0/2684436404 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e34d2ebe20 0x55e34d3752f0 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:13.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.554+0000 7f937b008080 1 --2- 172.21.15.49:0/2684436404 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e34d3784c0 0x55e34d372e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:13.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.554+0000 7f937b008080 1 --2- 172.21.15.49:0/2684436404 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e34d2cfa10 0x55e34d374db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:13.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.554+0000 7f937b008080 1 -- 172.21.15.49:0/2684436404 >> 172.21.15.49:0/2684436404 conn(0x55e34d2cdee0 msgr2=0x55e34d372540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:13.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.554+0000 7f937b008080 1 -- 172.21.15.49:0/2684436404 shutdown_connections 2023-11-06T02:14:13.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.554+0000 7f937b008080 1 -- 172.21.15.49:0/2684436404 wait complete. 2023-11-06T02:14:13.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.558+0000 7f937b008080 1 Processor -- start 2023-11-06T02:14:13.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.558+0000 7f937b008080 1 -- start start 2023-11-06T02:14:13.563 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.558+0000 7f937b008080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e34d2cfa10 0x55e34d373010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:13.563 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.558+0000 7f937b008080 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e34d2ebe20 0x55e34d2c5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:13.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.558+0000 7f937b008080 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e34d3784c0 0x55e34d2c5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:13.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.558+0000 7f937b008080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e34d381390 con 0x55e34d2cfa10 2023-11-06T02:14:13.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.558+0000 7f937b008080 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e34d2d2710 con 0x55e34d2ebe20 2023-11-06T02:14:13.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.558+0000 7f937b008080 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e34d2d29b0 con 0x55e34d3784c0 2023-11-06T02:14:13.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.558+0000 7f9378b22640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e34d2cfa10 0x55e34d373010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:13.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.558+0000 7f9378b22640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e34d2cfa10 0x55e34d373010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55418/0 (socket says 172.21.15.49:55418) 2023-11-06T02:14:13.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.558+0000 7f9378b22640 1 -- 172.21.15.49:0/859945150 learned_addr learned my addr 172.21.15.49:0/859945150 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:13.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.558+0000 7f9378321640 1 --2- 172.21.15.49:0/859945150 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e34d2ebe20 0x55e34d2c5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:13.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.558+0000 7f9379323640 1 --2- 172.21.15.49:0/859945150 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e34d3784c0 0x55e34d2c5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:13.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.558+0000 7f9378b22640 1 -- 172.21.15.49:0/859945150 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e34d3784c0 msgr2=0x55e34d2c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:13.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.558+0000 7f9378b22640 1 --2- 172.21.15.49:0/859945150 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e34d3784c0 0x55e34d2c5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:13.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.558+0000 7f9378b22640 1 -- 172.21.15.49:0/859945150 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e34d2ebe20 msgr2=0x55e34d2c5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:13.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.558+0000 7f9378b22640 1 --2- 172.21.15.49:0/859945150 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e34d2ebe20 0x55e34d2c5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:13.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.558+0000 7f9378b22640 1 -- 172.21.15.49:0/859945150 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f936c009520 con 0x55e34d2cfa10 2023-11-06T02:14:13.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.558+0000 7f9379323640 1 --2- 172.21.15.49:0/859945150 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55e34d3784c0 0x55e34d2c5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:14:13.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.558+0000 7f9378b22640 1 --2- 172.21.15.49:0/859945150 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e34d2cfa10 0x55e34d373010 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f936800cf70 tx=0x7f936800a5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:13.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.558+0000 7f937631d640 1 -- 172.21.15.49:0/859945150 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9368015030 con 0x55e34d2cfa10 2023-11-06T02:14:13.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.558+0000 7f937631d640 1 -- 172.21.15.49:0/859945150 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9368010680 con 0x55e34d2cfa10 2023-11-06T02:14:13.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.558+0000 7f937b008080 1 -- 172.21.15.49:0/859945150 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55e34d1f8ff0 con 0x55e34d2cfa10 2023-11-06T02:14:13.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.558+0000 7f937631d640 1 -- 172.21.15.49:0/859945150 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9368010a30 con 0x55e34d2cfa10 2023-11-06T02:14:13.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.558+0000 7f937b008080 1 -- 172.21.15.49:0/859945150 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e34d1ec430 con 0x55e34d2cfa10 2023-11-06T02:14:13.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.558+0000 7f937631d640 1 -- 172.21.15.49:0/859945150 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f9368018070 con 0x55e34d2cfa10 2023-11-06T02:14:13.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.562+0000 7f937631d640 1 --2- 172.21.15.49:0/859945150 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9364064490 0x7f9364066950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:13.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.562+0000 7f937631d640 1 -- 172.21.15.49:0/859945150 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f9368016070 con 0x55e34d2cfa10 2023-11-06T02:14:13.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.562+0000 7f937b008080 1 --2- 172.21.15.49:0/859945150 >> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] conn(0x55e34d373550 0x55e34d2d1370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:13.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.562+0000 7f937b008080 1 -- 172.21.15.49:0/859945150 --> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] -- osd_op(unknown.0.0:1 11.7 11:e2e07930:::10000000001.00000000:head [omap-rm-keys in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81) v8 -- 0x55e34d2d30f0 con 0x55e34d373550 2023-11-06T02:14:13.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.562+0000 7f9378321640 1 --2- 172.21.15.49:0/859945150 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9364064490 0x7f9364066950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:13.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.562+0000 7f9379323640 1 --2- 172.21.15.49:0/859945150 >> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] conn(0x55e34d373550 0x55e34d2d1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:13.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.562+0000 7f9378321640 1 --2- 172.21.15.49:0/859945150 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9364064490 0x7f9364066950 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f936c015040 tx=0x7f936c012000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:13.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.562+0000 7f9379323640 1 --2- 172.21.15.49:0/859945150 >> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] conn(0x55e34d373550 0x55e34d2d1370 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:13.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.566+0000 7f9379323640 1 -- 172.21.15.49:0/859945150 <== osd.5 v2:172.21.15.150:6800/38418 1 ==== osd_op_reply(1 10000000001.00000000 [omap-rm-keys] v81'5 uv5 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f93700a8070 con 0x55e34d373550 2023-11-06T02:14:13.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.566+0000 7f937b008080 1 -- 172.21.15.49:0/859945150 >> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] conn(0x55e34d373550 msgr2=0x55e34d2d1370 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:13.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.566+0000 7f937b008080 1 --2- 172.21.15.49:0/859945150 >> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] conn(0x55e34d373550 0x55e34d2d1370 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:13.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.566+0000 7f937b008080 1 -- 172.21.15.49:0/859945150 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9364064490 msgr2=0x7f9364066950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:13.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.566+0000 7f937b008080 1 --2- 172.21.15.49:0/859945150 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9364064490 0x7f9364066950 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f936c015040 tx=0x7f936c012000 comp rx=0 tx=0).stop 2023-11-06T02:14:13.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.566+0000 7f937b008080 1 -- 172.21.15.49:0/859945150 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e34d2cfa10 msgr2=0x55e34d373010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:13.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.566+0000 7f937b008080 1 --2- 172.21.15.49:0/859945150 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e34d2cfa10 0x55e34d373010 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f936800cf70 tx=0x7f936800a5b0 comp rx=0 tx=0).stop 2023-11-06T02:14:13.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.566+0000 7f9379323640 1 -- 172.21.15.49:0/859945150 reap_dead start 2023-11-06T02:14:13.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.566+0000 7f937b008080 1 -- 172.21.15.49:0/859945150 shutdown_connections 2023-11-06T02:14:13.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.566+0000 7f937b008080 1 -- 172.21.15.49:0/859945150 >> 172.21.15.49:0/859945150 conn(0x55e34d2cdee0 msgr2=0x55e34d377410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:13.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.566+0000 7f937b008080 1 -- 172.21.15.49:0/859945150 shutdown_connections 2023-11-06T02:14:13.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.566+0000 7f937b008080 1 -- 172.21.15.49:0/859945150 wait complete. 2023-11-06T02:14:13.573 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-11-06T02:14:13.740 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.746+0000 7f0633f9a640 1 Processor -- start 2023-11-06T02:14:13.741 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.746+0000 7f0633f9a640 1 -- start start 2023-11-06T02:14:13.743 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.746+0000 7f0633f9a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f062c15e0c0 0x7f062c0ce840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:13.744 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.746+0000 7f0633f9a640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f062c15e4a0 0x7f062c0cdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:13.744 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.746+0000 7f0633f9a640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f062c0ce230 con 0x7f062c0b16e0 2023-11-06T02:14:13.744 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.746+0000 7f0633f9a640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f062c0cd8d0 con 0x7f062c15e4a0 2023-11-06T02:14:13.744 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.746+0000 7f0633f9a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f062c0d1430 con 0x7f062c15e0c0 2023-11-06T02:14:13.744 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.746+0000 7f0632537640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f062c0b16e0 0x7f062c0b0100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:50490/0 (socket says 172.21.15.49:50490) 2023-11-06T02:14:13.745 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.746+0000 7f0632537640 1 -- 172.21.15.49:0/3744295925 learned_addr learned my addr 172.21.15.49:0/3744295925 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:14:13.745 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.746+0000 7f0631d36640 1 --2- 172.21.15.49:0/3744295925 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f062c15e0c0 0x7f062c0ce840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:13.745 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.746+0000 7f0631535640 1 --2- 172.21.15.49:0/3744295925 >> v2:172.21.15.150:3300/0 conn(0x7f062c15e4a0 0x7f062c0cdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:13.745 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.746+0000 7f0630d34640 1 -- 172.21.15.49:0/3744295925 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2887934969 0 0) 0x7f062c0ce230 con 0x7f062c0b16e0 2023-11-06T02:14:13.745 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.746+0000 7f0630d34640 1 -- 172.21.15.49:0/3744295925 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0614003530 con 0x7f062c0b16e0 2023-11-06T02:14:13.746 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.746+0000 7f0630d34640 1 -- 172.21.15.49:0/3744295925 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2014752000 0 0) 0x7f0614003530 con 0x7f062c0b16e0 2023-11-06T02:14:13.746 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.746+0000 7f0630d34640 1 -- 172.21.15.49:0/3744295925 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f062c15e0c0 msgr2=0x7f062c0ce840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:13.746 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.746+0000 7f0630d34640 1 --2- 172.21.15.49:0/3744295925 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f062c15e0c0 0x7f062c0ce840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:13.746 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.746+0000 7f0630d34640 1 -- 172.21.15.49:0/3744295925 >> v2:172.21.15.150:3300/0 conn(0x7f062c15e4a0 msgr2=0x7f062c0cdcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:13.746 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.746+0000 7f0630d34640 1 --2- 172.21.15.49:0/3744295925 >> v2:172.21.15.150:3300/0 conn(0x7f062c15e4a0 0x7f062c0cdcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:13.747 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.746+0000 7f0630d34640 1 -- 172.21.15.49:0/3744295925 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f062c0d1710 con 0x7f062c0b16e0 2023-11-06T02:14:13.747 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.746+0000 7f0630d34640 1 -- 172.21.15.49:0/3744295925 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7f061c001680 con 0x7f062c0b16e0 2023-11-06T02:14:13.747 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.746+0000 7f0630d34640 1 -- 172.21.15.49:0/3744295925 >> v1:172.21.15.49:6789/0 conn(0x7f062c0b16e0 legacy=0x7f062c0b0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:13.747 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.746+0000 7f0630d34640 1 --2- 172.21.15.49:0/3744295925 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0614003980 0x7f0614003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:13.747 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.746+0000 7f0630d34640 1 --2- 172.21.15.49:0/3744295925 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06140047d0 0x7f0614004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:13.748 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.746+0000 7f0630d34640 1 --2- 172.21.15.49:0/3744295925 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f062c15e0c0 0x7f0614005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:13.748 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.746+0000 7f0630d34640 1 -- 172.21.15.49:0/3744295925 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f062c0cd8d0 con 0x7f0614003980 2023-11-06T02:14:13.748 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.746+0000 7f0630d34640 1 -- 172.21.15.49:0/3744295925 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f062c0d1430 con 0x7f06140047d0 2023-11-06T02:14:13.748 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.746+0000 7f0630d34640 1 -- 172.21.15.49:0/3744295925 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f061400b990 con 0x7f062c15e0c0 2023-11-06T02:14:13.748 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.746+0000 7f0631d36640 1 --2- 172.21.15.49:0/3744295925 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0614003980 0x7f0614003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:13.749 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.746+0000 7f0631535640 1 --2- 172.21.15.49:0/3744295925 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06140047d0 0x7f0614004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:13.749 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.746+0000 7f0632537640 1 --2- 172.21.15.49:0/3744295925 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f062c15e0c0 0x7f0614005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:13.749 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.746+0000 7f0631d36640 1 -- 172.21.15.49:0/3744295925 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f062c15e0c0 msgr2=0x7f0614005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:13.750 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.746+0000 7f0631d36640 1 --2- 172.21.15.49:0/3744295925 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f062c15e0c0 0x7f0614005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:13.750 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.746+0000 7f0631d36640 1 -- 172.21.15.49:0/3744295925 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06140047d0 msgr2=0x7f0614004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:13.750 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.746+0000 7f0631d36640 1 --2- 172.21.15.49:0/3744295925 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06140047d0 0x7f0614004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:13.750 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.746+0000 7f0631d36640 1 -- 172.21.15.49:0/3744295925 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0614003530 con 0x7f0614003980 2023-11-06T02:14:13.751 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.746+0000 7f0631535640 1 --2- 172.21.15.49:0/3744295925 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06140047d0 0x7f0614004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:14:13.751 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.746+0000 7f0632537640 1 --2- 172.21.15.49:0/3744295925 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f062c15e0c0 0x7f0614005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:14:13.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.750+0000 7f0631d36640 1 --2- 172.21.15.49:0/3744295925 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0614003980 0x7f0614003da0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f0624009b10 tx=0x7f062401ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=780481baffb1f468 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:13.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.750+0000 7f0630d34640 1 -- 172.21.15.49:0/3744295925 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06240a5510 con 0x7f0614003980 2023-11-06T02:14:13.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.750+0000 7f0630d34640 1 -- 172.21.15.49:0/3744295925 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06240a5670 con 0x7f0614003980 2023-11-06T02:14:13.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.750+0000 7f0630d34640 1 -- 172.21.15.49:0/3744295925 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06240a5a20 con 0x7f0614003980 2023-11-06T02:14:13.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.750+0000 7f0633f9a640 1 -- 172.21.15.49:0/3744295925 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0614003980 msgr2=0x7f0614003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:13.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.750+0000 7f0633f9a640 1 --2- 172.21.15.49:0/3744295925 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0614003980 0x7f0614003da0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f0624009b10 tx=0x7f062401ca20 comp rx=0 tx=0).stop 2023-11-06T02:14:13.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.750+0000 7f0632537640 1 -- 172.21.15.49:0/3744295925 reap_dead start 2023-11-06T02:14:13.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.750+0000 7f0633f9a640 1 -- 172.21.15.49:0/3744295925 shutdown_connections 2023-11-06T02:14:13.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.750+0000 7f0633f9a640 1 -- 172.21.15.49:0/3744295925 >> 172.21.15.49:0/3744295925 conn(0x7f062c099c50 msgr2=0x7f062c0d0940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:13.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.750+0000 7f0633f9a640 1 -- 172.21.15.49:0/3744295925 shutdown_connections 2023-11-06T02:14:13.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.750+0000 7f0633f9a640 1 -- 172.21.15.49:0/3744295925 wait complete. 2023-11-06T02:14:13.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.750+0000 7f0633f9a640 1 Processor -- start 2023-11-06T02:14:13.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.750+0000 7f0633f9a640 1 -- start start 2023-11-06T02:14:13.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.754+0000 7f0633f9a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0614003980 0x7f062c0b3560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:13.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.754+0000 7f0633f9a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f062c0b6350 0x7f062c0c6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:13.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.754+0000 7f0633f9a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f062c14a6d0 0x7f062c15c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:13.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.754+0000 7f0633f9a640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f062c0b5ec0 con 0x7f062c0b6350 2023-11-06T02:14:13.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.754+0000 7f0633f9a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f062c0b5590 con 0x7f0614003980 2023-11-06T02:14:13.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.754+0000 7f0633f9a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f062c0b5870 con 0x7f062c14a6d0 2023-11-06T02:14:13.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.754+0000 7f0631535640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f062c0b6350 0x7f062c0c6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:13.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.754+0000 7f0631535640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f062c0b6350 0x7f062c0c6bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55422/0 (socket says 172.21.15.49:55422) 2023-11-06T02:14:13.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.754+0000 7f0631535640 1 -- 172.21.15.49:0/1151822190 learned_addr learned my addr 172.21.15.49:0/1151822190 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:13.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.754+0000 7f0632537640 1 --2- 172.21.15.49:0/1151822190 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f062c14a6d0 0x7f062c15c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:13.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.754+0000 7f0631535640 1 -- 172.21.15.49:0/1151822190 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f062c14a6d0 msgr2=0x7f062c15c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:13.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.754+0000 7f0631535640 1 --2- 172.21.15.49:0/1151822190 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f062c14a6d0 0x7f062c15c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:13.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.754+0000 7f0631d36640 1 --2- 172.21.15.49:0/1151822190 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0614003980 0x7f062c0b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:13.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.754+0000 7f0631535640 1 -- 172.21.15.49:0/1151822190 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0614003980 msgr2=0x7f062c0b3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:13.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.754+0000 7f0631535640 1 --2- 172.21.15.49:0/1151822190 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0614003980 0x7f062c0b3560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:13.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.754+0000 7f0631535640 1 -- 172.21.15.49:0/1151822190 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f062c0b5b50 con 0x7f062c0b6350 2023-11-06T02:14:13.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.754+0000 7f0631535640 1 --2- 172.21.15.49:0/1151822190 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f062c0b6350 0x7f062c0c6bf0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f062000f3d0 tx=0x7f062000f890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:13.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.754+0000 7f062a7fc640 1 -- 172.21.15.49:0/1151822190 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062001a070 con 0x7f062c0b6350 2023-11-06T02:14:13.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.754+0000 7f062a7fc640 1 -- 172.21.15.49:0/1151822190 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06200039a0 con 0x7f062c0b6350 2023-11-06T02:14:13.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.754+0000 7f0633f9a640 1 -- 172.21.15.49:0/1151822190 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f062c0b5240 con 0x7f062c0b6350 2023-11-06T02:14:13.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.754+0000 7f062a7fc640 1 -- 172.21.15.49:0/1151822190 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0620003d10 con 0x7f062c0b6350 2023-11-06T02:14:13.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.754+0000 7f0633f9a640 1 -- 172.21.15.49:0/1151822190 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f062c0c7130 con 0x7f062c0b6350 2023-11-06T02:14:13.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.754+0000 7f0633f9a640 1 -- 172.21.15.49:0/1151822190 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f062c0b6030 con 0x7f062c0b6350 2023-11-06T02:14:13.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.754+0000 7f062a7fc640 1 -- 172.21.15.49:0/1151822190 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f0620004020 con 0x7f062c0b6350 2023-11-06T02:14:13.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.754+0000 7f062a7fc640 1 --2- 172.21.15.49:0/1151822190 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0618065a90 0x7f0618067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:13.762 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.754+0000 7f062a7fc640 1 -- 172.21.15.49:0/1151822190 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f0620083e30 con 0x7f062c0b6350 2023-11-06T02:14:13.762 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.754+0000 7f0631d36640 1 --2- 172.21.15.49:0/1151822190 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0618065a90 0x7f0618067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:13.762 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.758+0000 7f0631d36640 1 --2- 172.21.15.49:0/1151822190 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0618065a90 0x7f0618067f50 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f06240af000 tx=0x7f06240046d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:13.763 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.758+0000 7f062a7fc640 1 -- 172.21.15.49:0/1151822190 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f06200516c0 con 0x7f062c0b6350 2023-11-06T02:14:13.948 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:13.954+0000 7f0633f9a640 1 -- 172.21.15.49:0/1151822190 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f062c0b5ce0 con 0x7f062c0b6350 2023-11-06T02:14:14.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.342+0000 7f062a7fc640 1 -- 172.21.15.49:0/1151822190 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v48) v1 ==== 162+0+0 (secure 0 0 0) 0x7f0620054ce0 con 0x7f062c0b6350 2023-11-06T02:14:14.339 INFO:teuthology.orchestra.run.smithi049.stderr:cephfs marked joinable; MDS may join as newly active. 2023-11-06T02:14:14.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.346+0000 7f0633f9a640 1 -- 172.21.15.49:0/1151822190 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0618065a90 msgr2=0x7f0618067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:14.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.346+0000 7f0633f9a640 1 --2- 172.21.15.49:0/1151822190 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0618065a90 0x7f0618067f50 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f06240af000 tx=0x7f06240046d0 comp rx=0 tx=0).stop 2023-11-06T02:14:14.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.346+0000 7f0633f9a640 1 -- 172.21.15.49:0/1151822190 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f062c0b6350 msgr2=0x7f062c0c6bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:14.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.346+0000 7f0633f9a640 1 --2- 172.21.15.49:0/1151822190 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f062c0b6350 0x7f062c0c6bf0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f062000f3d0 tx=0x7f062000f890 comp rx=0 tx=0).stop 2023-11-06T02:14:14.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.346+0000 7f0633f9a640 1 -- 172.21.15.49:0/1151822190 shutdown_connections 2023-11-06T02:14:14.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.346+0000 7f0633f9a640 1 --2- 172.21.15.49:0/1151822190 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0618065a90 0x7f0618067f50 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:14.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.346+0000 7f0633f9a640 1 --2- 172.21.15.49:0/1151822190 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f062c0b6350 0x7f062c0c6bf0 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:14.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.346+0000 7f0633f9a640 1 --2- 172.21.15.49:0/1151822190 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f062c14a6d0 0x7f062c15c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:14.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.346+0000 7f0633f9a640 1 --2- 172.21.15.49:0/1151822190 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0614003980 0x7f062c0b3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:14.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.346+0000 7f0633f9a640 1 -- 172.21.15.49:0/1151822190 >> 172.21.15.49:0/1151822190 conn(0x7f062c099c50 msgr2=0x7f062c06e6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:14.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.346+0000 7f0633f9a640 1 -- 172.21.15.49:0/1151822190 shutdown_connections 2023-11-06T02:14:14.344 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.346+0000 7f0633f9a640 1 -- 172.21.15.49:0/1151822190 wait complete. 2023-11-06T02:14:14.368 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T02:14:14.487 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.490+0000 7f50364f6640 1 Processor -- start 2023-11-06T02:14:14.487 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.490+0000 7f50364f6640 1 -- start start 2023-11-06T02:14:14.488 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.494+0000 7f50364f6640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f5030071bb0 0x7f5030161650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:14.488 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.494+0000 7f50364f6640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50300d17c0 con 0x7f50300cda70 2023-11-06T02:14:14.488 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.494+0000 7f50364f6640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50300ce6c0 con 0x7f50300b1f90 2023-11-06T02:14:14.489 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.494+0000 7f50364f6640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50300cdf20 con 0x7f5030071bb0 2023-11-06T02:14:14.489 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.494+0000 7f502f7fe640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f50300cda70 0x7f50300714a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:50498/0 (socket says 172.21.15.49:50498) 2023-11-06T02:14:14.489 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.494+0000 7f502f7fe640 1 -- 172.21.15.49:0/595620754 learned_addr learned my addr 172.21.15.49:0/595620754 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:14:14.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.494+0000 7f5034a93640 1 --2- 172.21.15.49:0/595620754 >> v2:172.21.15.150:3300/0 conn(0x7f5030071bb0 0x7f5030161650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:14.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.494+0000 7f502effd640 1 -- 172.21.15.49:0/595620754 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 756240303 0 0) 0x7f50300d17c0 con 0x7f50300cda70 2023-11-06T02:14:14.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.494+0000 7f502effd640 1 -- 172.21.15.49:0/595620754 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5018003530 con 0x7f50300cda70 2023-11-06T02:14:14.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.494+0000 7f502effd640 1 -- 172.21.15.49:0/595620754 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2967610373 0 0) 0x7f5018003530 con 0x7f50300cda70 2023-11-06T02:14:14.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.494+0000 7f502effd640 1 -- 172.21.15.49:0/595620754 >> v2:172.21.15.150:3300/0 conn(0x7f5030071bb0 msgr2=0x7f5030161650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:14.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.494+0000 7f502effd640 1 --2- 172.21.15.49:0/595620754 >> v2:172.21.15.150:3300/0 conn(0x7f5030071bb0 0x7f5030161650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:14.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.494+0000 7f502effd640 1 -- 172.21.15.49:0/595620754 >> v1:172.21.15.150:6789/0 conn(0x7f50300b1f90 legacy=0x7f50300d1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:14.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.494+0000 7f502effd640 1 -- 172.21.15.49:0/595620754 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50300d2360 con 0x7f50300cda70 2023-11-06T02:14:14.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.494+0000 7f502effd640 1 -- 172.21.15.49:0/595620754 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7f5024001a00 con 0x7f50300cda70 2023-11-06T02:14:14.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.494+0000 7f502effd640 1 -- 172.21.15.49:0/595620754 >> v1:172.21.15.49:6789/0 conn(0x7f50300cda70 legacy=0x7f50300714a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:14.493 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.494+0000 7f502effd640 1 --2- 172.21.15.49:0/595620754 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5018003980 0x7f5018003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:14.493 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.494+0000 7f502effd640 1 --2- 172.21.15.49:0/595620754 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f50180047d0 0x7f5018004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:14.493 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.494+0000 7f502effd640 1 --2- 172.21.15.49:0/595620754 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5018005190 0x7f501800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:14.493 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.494+0000 7f502effd640 1 -- 172.21.15.49:0/595620754 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50300cdf20 con 0x7f5018003980 2023-11-06T02:14:14.493 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.494+0000 7f502effd640 1 -- 172.21.15.49:0/595620754 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f501800c030 con 0x7f5018005190 2023-11-06T02:14:14.494 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.494+0000 7f502effd640 1 -- 172.21.15.49:0/595620754 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f501800c1f0 con 0x7f50180047d0 2023-11-06T02:14:14.494 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.494+0000 7f502ffff640 1 --2- 172.21.15.49:0/595620754 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5018003980 0x7f5018003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:14.494 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.494+0000 7f502f7fe640 1 --2- 172.21.15.49:0/595620754 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f50180047d0 0x7f5018004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:14.494 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.494+0000 7f5034a93640 1 --2- 172.21.15.49:0/595620754 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5018005190 0x7f501800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:14.494 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.494+0000 7f502ffff640 1 -- 172.21.15.49:0/595620754 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f50180047d0 msgr2=0x7f5018004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:14.495 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.494+0000 7f502ffff640 1 --2- 172.21.15.49:0/595620754 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f50180047d0 0x7f5018004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:14.495 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.494+0000 7f502ffff640 1 -- 172.21.15.49:0/595620754 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5018005190 msgr2=0x7f501800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:14.495 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.494+0000 7f502ffff640 1 --2- 172.21.15.49:0/595620754 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5018005190 0x7f501800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:14.495 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.494+0000 7f502ffff640 1 -- 172.21.15.49:0/595620754 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5018003530 con 0x7f5018003980 2023-11-06T02:14:14.495 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.494+0000 7f5034a93640 1 --2- 172.21.15.49:0/595620754 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5018005190 0x7f501800b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:14:14.496 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.494+0000 7f5034a93640 1 -- 172.21.15.49:0/595620754 reap_dead start 2023-11-06T02:14:14.498 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.498+0000 7f502ffff640 1 --2- 172.21.15.49:0/595620754 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5018003980 0x7f5018003da0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f5020001780 tx=0x7f502001c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=911cbfeea888e6d5 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:14.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.498+0000 7f502effd640 1 -- 172.21.15.49:0/595620754 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50200052d0 con 0x7f5018003980 2023-11-06T02:14:14.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.498+0000 7f502effd640 1 -- 172.21.15.49:0/595620754 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5020005450 con 0x7f5018003980 2023-11-06T02:14:14.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.498+0000 7f502effd640 1 -- 172.21.15.49:0/595620754 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5020003840 con 0x7f5018003980 2023-11-06T02:14:14.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.498+0000 7f50364f6640 1 -- 172.21.15.49:0/595620754 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5018003980 msgr2=0x7f5018003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:14.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.498+0000 7f50364f6640 1 --2- 172.21.15.49:0/595620754 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5018003980 0x7f5018003da0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f5020001780 tx=0x7f502001c3b0 comp rx=0 tx=0).stop 2023-11-06T02:14:14.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.498+0000 7f50364f6640 1 -- 172.21.15.49:0/595620754 shutdown_connections 2023-11-06T02:14:14.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.498+0000 7f50364f6640 1 --2- 172.21.15.49:0/595620754 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5018003980 0x7f5018003da0 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:14.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.498+0000 7f50364f6640 1 -- 172.21.15.49:0/595620754 >> 172.21.15.49:0/595620754 conn(0x7f50300d2aa0 msgr2=0x7f50300d2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:14.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.498+0000 7f50364f6640 1 -- 172.21.15.49:0/595620754 shutdown_connections 2023-11-06T02:14:14.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.498+0000 7f50364f6640 1 -- 172.21.15.49:0/595620754 wait complete. 2023-11-06T02:14:14.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.498+0000 7f50364f6640 1 Processor -- start 2023-11-06T02:14:14.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.498+0000 7f50364f6640 1 -- start start 2023-11-06T02:14:14.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.498+0000 7f50364f6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5018003980 0x7f50301f54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:14.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.498+0000 7f50364f6640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f50301f8150 0x7f50300b3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:14.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.498+0000 7f50364f6640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f50300b4190 0x7f503015fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:14.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.498+0000 7f50364f6640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50300aa560 con 0x7f5018003980 2023-11-06T02:14:14.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.498+0000 7f50364f6640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50300aa840 con 0x7f50300b4190 2023-11-06T02:14:14.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.498+0000 7f50364f6640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50300aab20 con 0x7f50301f8150 2023-11-06T02:14:14.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.498+0000 7f502ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5018003980 0x7f50301f54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:14.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.498+0000 7f502ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5018003980 0x7f50301f54b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55444/0 (socket says 172.21.15.49:55444) 2023-11-06T02:14:14.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.498+0000 7f502ffff640 1 -- 172.21.15.49:0/2049111762 learned_addr learned my addr 172.21.15.49:0/2049111762 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:14.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.502+0000 7f5034a93640 1 --2- 172.21.15.49:0/2049111762 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f50300b4190 0x7f503015fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:14.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.502+0000 7f502f7fe640 1 --2- 172.21.15.49:0/2049111762 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f50301f8150 0x7f50300b3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:14.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.502+0000 7f502ffff640 1 -- 172.21.15.49:0/2049111762 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f50301f8150 msgr2=0x7f50300b3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:14.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.502+0000 7f502ffff640 1 --2- 172.21.15.49:0/2049111762 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f50301f8150 0x7f50300b3c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:14.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.502+0000 7f502ffff640 1 -- 172.21.15.49:0/2049111762 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f50300b4190 msgr2=0x7f503015fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:14.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.502+0000 7f502ffff640 1 --2- 172.21.15.49:0/2049111762 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f50300b4190 0x7f503015fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:14.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.502+0000 7f502ffff640 1 -- 172.21.15.49:0/2049111762 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50300aae00 con 0x7f5018003980 2023-11-06T02:14:14.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.502+0000 7f502ffff640 1 --2- 172.21.15.49:0/2049111762 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5018003980 0x7f50301f54b0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f502001c1e0 tx=0x7f5018005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:14.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.502+0000 7f502cff9640 1 -- 172.21.15.49:0/2049111762 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50200b9070 con 0x7f5018003980 2023-11-06T02:14:14.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.502+0000 7f502cff9640 1 -- 172.21.15.49:0/2049111762 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5020007250 con 0x7f5018003980 2023-11-06T02:14:14.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.502+0000 7f502cff9640 1 -- 172.21.15.49:0/2049111762 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5020007400 con 0x7f5018003980 2023-11-06T02:14:14.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.502+0000 7f50364f6640 1 -- 172.21.15.49:0/2049111762 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50300aa260 con 0x7f5018003980 2023-11-06T02:14:14.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.502+0000 7f50364f6640 1 -- 172.21.15.49:0/2049111762 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50300aa050 con 0x7f5018003980 2023-11-06T02:14:14.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.502+0000 7f50364f6640 1 -- 172.21.15.49:0/2049111762 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5030074040 con 0x7f5018003980 2023-11-06T02:14:14.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.502+0000 7f502cff9640 1 -- 172.21.15.49:0/2049111762 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f5020007560 con 0x7f5018003980 2023-11-06T02:14:14.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.502+0000 7f502cff9640 1 --2- 172.21.15.49:0/2049111762 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f501c065a60 0x7f501c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:14.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.502+0000 7f502cff9640 1 -- 172.21.15.49:0/2049111762 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f502011cbd0 con 0x7f5018003980 2023-11-06T02:14:14.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.502+0000 7f502f7fe640 1 --2- 172.21.15.49:0/2049111762 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f501c065a60 0x7f501c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:14.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.506+0000 7f502f7fe640 1 --2- 172.21.15.49:0/2049111762 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f501c065a60 0x7f501c067f20 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f5024002240 tx=0x7f5024009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:14.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.506+0000 7f502cff9640 1 -- 172.21.15.49:0/2049111762 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f50200ea3d0 con 0x7f5018003980 2023-11-06T02:14:14.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.698+0000 7f50364f6640 1 -- 172.21.15.49:0/2049111762 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5030074040 con 0x7f5018003980 2023-11-06T02:14:14.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.698+0000 7f502cff9640 1 -- 172.21.15.49:0/2049111762 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4651 (secure 0 0 0) 0x7f50200ed9f0 con 0x7f5018003980 2023-11-06T02:14:14.694 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:14.695 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":49,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5714,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/315581318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":315581318},{"type":"v1","addr":"172.21.15.150:6837","nonce":315581318}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41},{"gid":6121,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/136800245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":136800245},{"type":"v1","addr":"172.21.15.150:6835","nonce":136800245}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41},{"gid":7110,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/4259579254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":4259579254},{"type":"v1","addr":"172.21.15.49:6833","nonce":4259579254}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T02:13:33.625701+0000","modified":"2023-11-06T02:14:14.346124+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7533":{"gid":7533,"name":"c","rank":0,"incarnation":49,"state":"up:replay","state_seq":1,"addr":"172.21.15.49:6835/4037104163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":4037104163},{"type":"v1","addr":"172.21.15.49:6835","nonce":4037104163}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-06T02:14:14.695 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 49 2023-11-06T02:14:14.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.702+0000 7f50364f6640 1 -- 172.21.15.49:0/2049111762 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f501c065a60 msgr2=0x7f501c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:14.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.702+0000 7f50364f6640 1 --2- 172.21.15.49:0/2049111762 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f501c065a60 0x7f501c067f20 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f5024002240 tx=0x7f5024009040 comp rx=0 tx=0).stop 2023-11-06T02:14:14.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.702+0000 7f50364f6640 1 -- 172.21.15.49:0/2049111762 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5018003980 msgr2=0x7f50301f54b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:14.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.702+0000 7f50364f6640 1 --2- 172.21.15.49:0/2049111762 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5018003980 0x7f50301f54b0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f502001c1e0 tx=0x7f5018005800 comp rx=0 tx=0).stop 2023-11-06T02:14:14.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.702+0000 7f50364f6640 1 -- 172.21.15.49:0/2049111762 shutdown_connections 2023-11-06T02:14:14.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.702+0000 7f50364f6640 1 --2- 172.21.15.49:0/2049111762 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f501c065a60 0x7f501c067f20 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:14.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.702+0000 7f50364f6640 1 --2- 172.21.15.49:0/2049111762 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f50300b4190 0x7f503015fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:14.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.702+0000 7f50364f6640 1 --2- 172.21.15.49:0/2049111762 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f50301f8150 0x7f50300b3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:14.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.702+0000 7f50364f6640 1 --2- 172.21.15.49:0/2049111762 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5018003980 0x7f50301f54b0 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:14.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.702+0000 7f50364f6640 1 -- 172.21.15.49:0/2049111762 >> 172.21.15.49:0/2049111762 conn(0x7f50300d2aa0 msgr2=0x7f5030072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:14.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.702+0000 7f50364f6640 1 -- 172.21.15.49:0/2049111762 shutdown_connections 2023-11-06T02:14:14.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:14.702+0000 7f50364f6640 1 -- 172.21.15.49:0/2049111762 wait complete. 2023-11-06T02:14:15.117 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 49, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-06T02:13:33.625701+0000', 'modified': '2023-11-06T02:14:14.346124+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 81, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7533}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7533': {'gid': 7533, 'name': 'c', 'rank': 0, 'incarnation': 49, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.49:6835/4037104163', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6834', 'nonce': 4037104163}, {'type': 'v1', 'addr': '172.21.15.49:6835', 'nonce': 4037104163}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-06T02:14:15.118 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7533:up:replay 2023-11-06T02:14:16.119 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T02:14:16.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:16.258+0000 7f6c03992640 1 Processor -- start 2023-11-06T02:14:16.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:16.258+0000 7f6c03992640 1 -- start start 2023-11-06T02:14:16.253 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:16.258+0000 7f6c03992640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f6bfc159980 0x7f6bfc071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:16.254 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:16.258+0000 7f6c03992640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bfc071890 con 0x7f6bfc0b1450 2023-11-06T02:14:16.254 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:16.258+0000 7f6c03992640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bfc071b70 con 0x7f6bfc0cb010 2023-11-06T02:14:16.254 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:16.258+0000 7f6c03992640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bfc072410 con 0x7f6bfc159980 2023-11-06T02:14:16.254 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:16.258+0000 7f6c0172e640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f6bfc159980 0x7f6bfc071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:16.255 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:16.258+0000 7f6c00f2d640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f6bfc0b1450 0x7f6bfc0cdd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:50512/0 (socket says 172.21.15.49:50512) 2023-11-06T02:14:16.255 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:16.258+0000 7f6c00f2d640 1 -- 172.21.15.49:0/4114488153 learned_addr learned my addr 172.21.15.49:0/4114488153 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:14:16.255 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:16.258+0000 7f6c0172e640 1 -- 172.21.15.49:0/4114488153 >> v1:172.21.15.150:6789/0 conn(0x7f6bfc0cb010 legacy=0x7f6bfc0cc540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:16.255 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:16.258+0000 7f6c0172e640 1 -- 172.21.15.49:0/4114488153 >> v1:172.21.15.49:6789/0 conn(0x7f6bfc0b1450 legacy=0x7f6bfc0cdd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:16.255 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:16.258+0000 7f6c0172e640 1 -- 172.21.15.49:0/4114488153 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bfc071e50 con 0x7f6bfc159980 2023-11-06T02:14:16.255 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:16.258+0000 7f6bf3fff640 1 -- 172.21.15.49:0/4114488153 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2407905070 0 0) 0x7f6bfc071890 con 0x7f6bfc0b1450 2023-11-06T02:14:16.257 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:16.262+0000 7f6c0172e640 1 --2- 172.21.15.49:0/4114488153 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6bfc159980 0x7f6bfc071290 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f6bf80097a0 tx=0x7f6bf801c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa9b76f1fed20b26 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:16.257 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:16.262+0000 7f6bf3fff640 1 -- 172.21.15.49:0/4114488153 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf80ab600 con 0x7f6bfc159980 2023-11-06T02:14:16.257 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:16.262+0000 7f6bf3fff640 1 -- 172.21.15.49:0/4114488153 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bf80a7050 con 0x7f6bfc159980 2023-11-06T02:14:16.257 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:16.262+0000 7f6bf3fff640 1 -- 172.21.15.49:0/4114488153 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf80ab980 con 0x7f6bfc159980 2023-11-06T02:14:16.257 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:16.262+0000 7f6c03992640 1 -- 172.21.15.49:0/4114488153 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6bfc159980 msgr2=0x7f6bfc071290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:16.258 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:16.262+0000 7f6c03992640 1 --2- 172.21.15.49:0/4114488153 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6bfc159980 0x7f6bfc071290 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f6bf80097a0 tx=0x7f6bf801c420 comp rx=0 tx=0).stop 2023-11-06T02:14:16.258 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:16.262+0000 7f6c03992640 1 -- 172.21.15.49:0/4114488153 shutdown_connections 2023-11-06T02:14:16.258 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:16.262+0000 7f6c03992640 1 --2- 172.21.15.49:0/4114488153 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6bfc159980 0x7f6bfc071290 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:16.258 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:16.262+0000 7f6c03992640 1 -- 172.21.15.49:0/4114488153 >> 172.21.15.49:0/4114488153 conn(0x7f6bfc06c0a0 msgr2=0x7f6bfc06c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:16.258 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:16.262+0000 7f6c03992640 1 -- 172.21.15.49:0/4114488153 shutdown_connections 2023-11-06T02:14:16.259 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:16.262+0000 7f6c03992640 1 -- 172.21.15.49:0/4114488153 wait complete. 2023-11-06T02:14:16.261 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:16.262+0000 7f6c03992640 1 Processor -- start 2023-11-06T02:14:16.261 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:16.262+0000 7f6c03992640 1 -- start start 2023-11-06T02:14:16.261 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:16.262+0000 7f6c03992640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6bfc0b1450 0x7f6bfc14cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:16.262 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:16.262+0000 7f6c03992640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6bfc0cb010 0x7f6bfc0c0f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:16.262 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:16.262+0000 7f6c03992640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6bfc159980 0x7f6bfc1558a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:16.262 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:16.262+0000 7f6c03992640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bfc0ce440 con 0x7f6bfc0cb010 2023-11-06T02:14:16.262 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:16.262+0000 7f6c03992640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bfc0b1890 con 0x7f6bfc0b1450 2023-11-06T02:14:16.262 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:16.262+0000 7f6c03992640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bfc14d300 con 0x7f6bfc159980 2023-11-06T02:14:16.263 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:16.262+0000 7f6c00f2d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6bfc0cb010 0x7f6bfc0c0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:16.263 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:16.262+0000 7f6c00f2d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6bfc0cb010 0x7f6bfc0c0f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55458/0 (socket says 172.21.15.49:55458) 2023-11-06T02:14:16.263 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:16.262+0000 7f6c00f2d640 1 -- 172.21.15.49:0/809109011 learned_addr learned my addr 172.21.15.49:0/809109011 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:16.263 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:16.266+0000 7f6c0172e640 1 --2- 172.21.15.49:0/809109011 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6bfc0b1450 0x7f6bfc14cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:16.263 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:16.266+0000 7f6c01f2f640 1 --2- 172.21.15.49:0/809109011 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6bfc159980 0x7f6bfc1558a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:16.264 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:16.266+0000 7f6c00f2d640 1 -- 172.21.15.49:0/809109011 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6bfc159980 msgr2=0x7f6bfc1558a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:16.264 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:16.266+0000 7f6c00f2d640 1 --2- 172.21.15.49:0/809109011 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6bfc159980 0x7f6bfc1558a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:16.264 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:16.266+0000 7f6c00f2d640 1 -- 172.21.15.49:0/809109011 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6bfc0b1450 msgr2=0x7f6bfc14cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:16.264 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:16.266+0000 7f6c00f2d640 1 --2- 172.21.15.49:0/809109011 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6bfc0b1450 0x7f6bfc14cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:16.264 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:16.266+0000 7f6c00f2d640 1 -- 172.21.15.49:0/809109011 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bf80093b0 con 0x7f6bfc0cb010 2023-11-06T02:14:16.265 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:16.266+0000 7f6c01f2f640 1 --2- 172.21.15.49:0/809109011 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6bfc159980 0x7f6bfc1558a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:14:16.265 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:16.266+0000 7f6c00f2d640 1 --2- 172.21.15.49:0/809109011 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6bfc0cb010 0x7f6bfc0c0f50 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f6bec00c520 tx=0x7f6bec00c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:16.265 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:16.266+0000 7f6bf1ffb640 1 -- 172.21.15.49:0/809109011 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bec0074e0 con 0x7f6bfc0cb010 2023-11-06T02:14:16.265 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:16.266+0000 7f6bf1ffb640 1 -- 172.21.15.49:0/809109011 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bec007640 con 0x7f6bfc0cb010 2023-11-06T02:14:16.266 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:16.266+0000 7f6c03992640 1 -- 172.21.15.49:0/809109011 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bfc0c1490 con 0x7f6bfc0cb010 2023-11-06T02:14:16.266 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:16.266+0000 7f6bf1ffb640 1 -- 172.21.15.49:0/809109011 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bec0049f0 con 0x7f6bfc0cb010 2023-11-06T02:14:16.266 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:16.266+0000 7f6c03992640 1 -- 172.21.15.49:0/809109011 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bfc14a9e0 con 0x7f6bfc0cb010 2023-11-06T02:14:16.266 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:16.266+0000 7f6c03992640 1 -- 172.21.15.49:0/809109011 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bfc074040 con 0x7f6bfc0cb010 2023-11-06T02:14:16.266 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:16.266+0000 7f6bf1ffb640 1 -- 172.21.15.49:0/809109011 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f6bec004b50 con 0x7f6bfc0cb010 2023-11-06T02:14:16.267 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:16.266+0000 7f6bf1ffb640 1 --2- 172.21.15.49:0/809109011 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6be0065a60 0x7f6be0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:16.268 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:16.266+0000 7f6bf1ffb640 1 -- 172.21.15.49:0/809109011 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f6bec082570 con 0x7f6bfc0cb010 2023-11-06T02:14:16.268 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:16.266+0000 7f6c0172e640 1 --2- 172.21.15.49:0/809109011 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6be0065a60 0x7f6be0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:16.268 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:16.266+0000 7f6c0172e640 1 --2- 172.21.15.49:0/809109011 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6be0065a60 0x7f6be0067f20 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f6bf80ad000 tx=0x7f6bf80a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:16.268 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:16.270+0000 7f6bf1ffb640 1 -- 172.21.15.49:0/809109011 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f6bec04fe10 con 0x7f6bfc0cb010 2023-11-06T02:14:16.460 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:16.466+0000 7f6c03992640 1 -- 172.21.15.49:0/809109011 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6bfc0c1b00 con 0x7f6bfc0cb010 2023-11-06T02:14:16.461 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:16.466+0000 7f6bf1ffb640 1 -- 172.21.15.49:0/809109011 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+4651 (secure 0 0 0) 0x7f6bec053430 con 0x7f6bfc0cb010 2023-11-06T02:14:16.462 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:16.462 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":51,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5714,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/315581318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":315581318},{"type":"v1","addr":"172.21.15.150:6837","nonce":315581318}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41},{"gid":6121,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/136800245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":136800245},{"type":"v1","addr":"172.21.15.150:6835","nonce":136800245}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41},{"gid":7110,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/4259579254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":4259579254},{"type":"v1","addr":"172.21.15.49:6833","nonce":4259579254}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T02:13:33.625701+0000","modified":"2023-11-06T02:14:16.365112+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7533":{"gid":7533,"name":"c","rank":0,"incarnation":49,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.49:6835/4037104163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":4037104163},{"type":"v1","addr":"172.21.15.49:6835","nonce":4037104163}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-06T02:14:16.462 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 51 2023-11-06T02:14:16.464 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:16.470+0000 7f6c03992640 1 -- 172.21.15.49:0/809109011 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6be0065a60 msgr2=0x7f6be0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:16.465 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:16.470+0000 7f6c03992640 1 --2- 172.21.15.49:0/809109011 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6be0065a60 0x7f6be0067f20 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f6bf80ad000 tx=0x7f6bf80a6000 comp rx=0 tx=0).stop 2023-11-06T02:14:16.465 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:16.470+0000 7f6c03992640 1 -- 172.21.15.49:0/809109011 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6bfc0cb010 msgr2=0x7f6bfc0c0f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:16.465 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:16.470+0000 7f6c03992640 1 --2- 172.21.15.49:0/809109011 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6bfc0cb010 0x7f6bfc0c0f50 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f6bec00c520 tx=0x7f6bec00c9e0 comp rx=0 tx=0).stop 2023-11-06T02:14:16.465 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:16.470+0000 7f6c03992640 1 -- 172.21.15.49:0/809109011 shutdown_connections 2023-11-06T02:14:16.465 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:16.470+0000 7f6c03992640 1 --2- 172.21.15.49:0/809109011 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6be0065a60 0x7f6be0067f20 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:16.466 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:16.470+0000 7f6c03992640 1 --2- 172.21.15.49:0/809109011 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6bfc0cb010 0x7f6bfc0c0f50 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:16.466 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:16.470+0000 7f6c03992640 1 --2- 172.21.15.49:0/809109011 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6bfc159980 0x7f6bfc1558a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:16.466 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:16.470+0000 7f6c03992640 1 --2- 172.21.15.49:0/809109011 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6bfc0b1450 0x7f6bfc14cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:16.466 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:16.470+0000 7f6c03992640 1 -- 172.21.15.49:0/809109011 >> 172.21.15.49:0/809109011 conn(0x7f6bfc06c0a0 msgr2=0x7f6bfc0b1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:16.466 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:16.470+0000 7f6c03992640 1 -- 172.21.15.49:0/809109011 shutdown_connections 2023-11-06T02:14:16.467 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:16.470+0000 7f6c03992640 1 -- 172.21.15.49:0/809109011 wait complete. 2023-11-06T02:14:16.884 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 51, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-06T02:13:33.625701+0000', 'modified': '2023-11-06T02:14:16.365112+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 81, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7533}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7533': {'gid': 7533, 'name': 'c', 'rank': 0, 'incarnation': 49, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.49:6835/4037104163', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6834', 'nonce': 4037104163}, {'type': 'v1', 'addr': '172.21.15.49:6835', 'nonce': 4037104163}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-06T02:14:16.885 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7533:up:rejoin 2023-11-06T02:14:17.886 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T02:14:18.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.026+0000 7f82b12fa640 1 Processor -- start 2023-11-06T02:14:18.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.026+0000 7f82b12fa640 1 -- start start 2023-11-06T02:14:18.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.026+0000 7f82b12fa640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f82ac071290 0x7f82ac0b1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:18.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.026+0000 7f82b12fa640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f82ac0cb010 0x7f82ac0cb480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:18.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.026+0000 7f82b12fa640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82ac071890 con 0x7f82ac159980 2023-11-06T02:14:18.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.026+0000 7f82b12fa640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82ac072410 con 0x7f82ac0cb010 2023-11-06T02:14:18.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.026+0000 7f82b12fa640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82ac0ce440 con 0x7f82ac071290 2023-11-06T02:14:18.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.026+0000 7f82ab7fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f82ac0cb010 0x7f82ac0cb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:18.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.026+0000 7f82ab7fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f82ac0cb010 0x7f82ac0cb480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55464/0 (socket says 172.21.15.49:55464) 2023-11-06T02:14:18.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.026+0000 7f82ab7fe640 1 -- 172.21.15.49:0/1154747573 learned_addr learned my addr 172.21.15.49:0/1154747573 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:18.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.026+0000 7f82aa7fc640 1 --2- 172.21.15.49:0/1154747573 >> v2:172.21.15.150:3300/0 conn(0x7f82ac071290 0x7f82ac0b1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:18.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.026+0000 7f82ab7fe640 1 -- 172.21.15.49:0/1154747573 >> v2:172.21.15.150:3300/0 conn(0x7f82ac071290 msgr2=0x7f82ac0b1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:18.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.026+0000 7f82ab7fe640 1 --2- 172.21.15.49:0/1154747573 >> v2:172.21.15.150:3300/0 conn(0x7f82ac071290 0x7f82ac0b1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:18.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.026+0000 7f82ab7fe640 1 -- 172.21.15.49:0/1154747573 >> v1:172.21.15.150:6789/0 conn(0x7f82ac159980 legacy=0x7f82ac0cdd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:18.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.026+0000 7f82ab7fe640 1 -- 172.21.15.49:0/1154747573 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82ac071b70 con 0x7f82ac0cb010 2023-11-06T02:14:18.026 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.030+0000 7f82ab7fe640 1 --2- 172.21.15.49:0/1154747573 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82ac0cb010 0x7f82ac0cb480 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f82ac06d090 tx=0x7f82a401c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8128a3fea1acf318 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:18.027 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.030+0000 7f82a9ffb640 1 -- 172.21.15.49:0/1154747573 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a40a5720 con 0x7f82ac0cb010 2023-11-06T02:14:18.027 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.030+0000 7f82a9ffb640 1 -- 172.21.15.49:0/1154747573 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82a40a5880 con 0x7f82ac0cb010 2023-11-06T02:14:18.027 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.030+0000 7f82a9ffb640 1 -- 172.21.15.49:0/1154747573 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a40a5c10 con 0x7f82ac0cb010 2023-11-06T02:14:18.028 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.030+0000 7f82b12fa640 1 -- 172.21.15.49:0/1154747573 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82ac0cb010 msgr2=0x7f82ac0cb480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:18.028 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.030+0000 7f82b12fa640 1 --2- 172.21.15.49:0/1154747573 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82ac0cb010 0x7f82ac0cb480 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f82ac06d090 tx=0x7f82a401c950 comp rx=0 tx=0).stop 2023-11-06T02:14:18.028 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.030+0000 7f82b12fa640 1 -- 172.21.15.49:0/1154747573 shutdown_connections 2023-11-06T02:14:18.028 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.030+0000 7f82b12fa640 1 --2- 172.21.15.49:0/1154747573 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82ac0cb010 0x7f82ac0cb480 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:18.028 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.030+0000 7f82b12fa640 1 --2- 172.21.15.49:0/1154747573 >> v2:172.21.15.150:3300/0 conn(0x7f82ac071290 0x7f82ac0b1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:18.029 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.030+0000 7f82b12fa640 1 -- 172.21.15.49:0/1154747573 >> 172.21.15.49:0/1154747573 conn(0x7f82ac06c0a0 msgr2=0x7f82ac06c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:18.029 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.030+0000 7f82b12fa640 1 -- 172.21.15.49:0/1154747573 shutdown_connections 2023-11-06T02:14:18.029 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.030+0000 7f82b12fa640 1 -- 172.21.15.49:0/1154747573 wait complete. 2023-11-06T02:14:18.029 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.030+0000 7f82b12fa640 1 Processor -- start 2023-11-06T02:14:18.029 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.030+0000 7f82b12fa640 1 -- start start 2023-11-06T02:14:18.032 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.034+0000 7f82b12fa640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f82ac071290 0x7f82ac159e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:18.033 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.034+0000 7f82b12fa640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82ac0cb010 0x7f82ac1581f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:18.033 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.034+0000 7f82b12fa640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82ac159980 0x7f82ac0c0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:18.033 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.034+0000 7f82b12fa640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82ac152860 con 0x7f82ac159980 2023-11-06T02:14:18.033 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.034+0000 7f82b12fa640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82ac149ae0 con 0x7f82ac0cb010 2023-11-06T02:14:18.033 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.034+0000 7f82b12fa640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82ac149dc0 con 0x7f82ac071290 2023-11-06T02:14:18.034 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.034+0000 7f82ab7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82ac159980 0x7f82ac0c0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:18.034 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.034+0000 7f82ab7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82ac159980 0x7f82ac0c0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55468/0 (socket says 172.21.15.49:55468) 2023-11-06T02:14:18.034 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.034+0000 7f82ab7fe640 1 -- 172.21.15.49:0/2994761374 learned_addr learned my addr 172.21.15.49:0/2994761374 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:18.034 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.034+0000 7f82aaffd640 1 --2- 172.21.15.49:0/2994761374 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f82ac071290 0x7f82ac159e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:18.034 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.034+0000 7f82ab7fe640 1 -- 172.21.15.49:0/2994761374 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f82ac071290 msgr2=0x7f82ac159e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:18.035 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.034+0000 7f82aa7fc640 1 --2- 172.21.15.49:0/2994761374 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82ac0cb010 0x7f82ac1581f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:18.035 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.034+0000 7f82ab7fe640 1 --2- 172.21.15.49:0/2994761374 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f82ac071290 0x7f82ac159e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:18.035 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.034+0000 7f82ab7fe640 1 -- 172.21.15.49:0/2994761374 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82ac0cb010 msgr2=0x7f82ac1581f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:18.035 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.034+0000 7f82ab7fe640 1 --2- 172.21.15.49:0/2994761374 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82ac0cb010 0x7f82ac1581f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:18.036 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.034+0000 7f82ab7fe640 1 -- 172.21.15.49:0/2994761374 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82a4009b30 con 0x7f82ac159980 2023-11-06T02:14:18.036 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.034+0000 7f82aaffd640 1 --2- 172.21.15.49:0/2994761374 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f82ac071290 0x7f82ac159e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:14:18.036 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.034+0000 7f82ab7fe640 1 --2- 172.21.15.49:0/2994761374 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82ac159980 0x7f82ac0c0ae0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f82a40af040 tx=0x7f82a40196f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:18.036 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.034+0000 7f82937fe640 1 -- 172.21.15.49:0/2994761374 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a40b8670 con 0x7f82ac159980 2023-11-06T02:14:18.036 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.034+0000 7f82937fe640 1 -- 172.21.15.49:0/2994761374 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82a40b87f0 con 0x7f82ac159980 2023-11-06T02:14:18.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.034+0000 7f82b12fa640 1 -- 172.21.15.49:0/2994761374 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82ac14a0a0 con 0x7f82ac159980 2023-11-06T02:14:18.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.034+0000 7f82937fe640 1 -- 172.21.15.49:0/2994761374 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a40b8ba0 con 0x7f82ac159980 2023-11-06T02:14:18.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.034+0000 7f82b12fa640 1 -- 172.21.15.49:0/2994761374 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82ac15a3c0 con 0x7f82ac159980 2023-11-06T02:14:18.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.034+0000 7f82b12fa640 1 -- 172.21.15.49:0/2994761374 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82ac14eb40 con 0x7f82ac159980 2023-11-06T02:14:18.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.034+0000 7f82937fe640 1 -- 172.21.15.49:0/2994761374 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f82a40b8d00 con 0x7f82ac159980 2023-11-06T02:14:18.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.034+0000 7f82937fe640 1 --2- 172.21.15.49:0/2994761374 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8298065a60 0x7f8298067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:18.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.034+0000 7f82937fe640 1 -- 172.21.15.49:0/2994761374 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f82a411e780 con 0x7f82ac159980 2023-11-06T02:14:18.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.034+0000 7f82aaffd640 1 --2- 172.21.15.49:0/2994761374 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8298065a60 0x7f8298067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:18.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.038+0000 7f82aaffd640 1 --2- 172.21.15.49:0/2994761374 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8298065a60 0x7f8298067f20 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f82a000a830 tx=0x7f82a0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:18.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.038+0000 7f82937fe640 1 -- 172.21.15.49:0/2994761374 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f82a40ec030 con 0x7f82ac159980 2023-11-06T02:14:18.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.230+0000 7f82b12fa640 1 -- 172.21.15.49:0/2994761374 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f82ac14eb40 con 0x7f82ac159980 2023-11-06T02:14:18.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.230+0000 7f82937fe640 1 -- 172.21.15.49:0/2994761374 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+4651 (secure 0 0 0) 0x7f82a40ef650 con 0x7f82ac159980 2023-11-06T02:14:18.225 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:18.225 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":52,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5714,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/315581318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":315581318},{"type":"v1","addr":"172.21.15.150:6837","nonce":315581318}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41},{"gid":6121,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/136800245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":136800245},{"type":"v1","addr":"172.21.15.150:6835","nonce":136800245}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41},{"gid":7110,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/4259579254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":4259579254},{"type":"v1","addr":"172.21.15.49:6833","nonce":4259579254}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T02:13:33.625701+0000","modified":"2023-11-06T02:14:17.377756+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7533":{"gid":7533,"name":"c","rank":0,"incarnation":49,"state":"up:active","state_seq":4,"addr":"172.21.15.49:6835/4037104163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":4037104163},{"type":"v1","addr":"172.21.15.49:6835","nonce":4037104163}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-06T02:14:18.225 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 52 2023-11-06T02:14:18.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.230+0000 7f82b12fa640 1 -- 172.21.15.49:0/2994761374 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8298065a60 msgr2=0x7f8298067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:18.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.230+0000 7f82b12fa640 1 --2- 172.21.15.49:0/2994761374 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8298065a60 0x7f8298067f20 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f82a000a830 tx=0x7f82a0008040 comp rx=0 tx=0).stop 2023-11-06T02:14:18.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.230+0000 7f82b12fa640 1 -- 172.21.15.49:0/2994761374 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82ac159980 msgr2=0x7f82ac0c0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:18.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.230+0000 7f82b12fa640 1 --2- 172.21.15.49:0/2994761374 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82ac159980 0x7f82ac0c0ae0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f82a40af040 tx=0x7f82a40196f0 comp rx=0 tx=0).stop 2023-11-06T02:14:18.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.230+0000 7f82b12fa640 1 -- 172.21.15.49:0/2994761374 shutdown_connections 2023-11-06T02:14:18.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.230+0000 7f82b12fa640 1 --2- 172.21.15.49:0/2994761374 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8298065a60 0x7f8298067f20 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:18.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.230+0000 7f82b12fa640 1 --2- 172.21.15.49:0/2994761374 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82ac159980 0x7f82ac0c0ae0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:18.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.230+0000 7f82b12fa640 1 --2- 172.21.15.49:0/2994761374 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82ac0cb010 0x7f82ac1581f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:18.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.230+0000 7f82b12fa640 1 --2- 172.21.15.49:0/2994761374 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f82ac071290 0x7f82ac159e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:18.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.230+0000 7f82b12fa640 1 -- 172.21.15.49:0/2994761374 >> 172.21.15.49:0/2994761374 conn(0x7f82ac06c0a0 msgr2=0x7f82ac0c9db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:18.229 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.230+0000 7f82b12fa640 1 -- 172.21.15.49:0/2994761374 shutdown_connections 2023-11-06T02:14:18.229 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.230+0000 7f82b12fa640 1 -- 172.21.15.49:0/2994761374 wait complete. 2023-11-06T02:14:18.647 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 52, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-06T02:13:33.625701+0000', 'modified': '2023-11-06T02:14:17.377756+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 81, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7533}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7533': {'gid': 7533, 'name': 'c', 'rank': 0, 'incarnation': 49, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.49:6835/4037104163', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6834', 'nonce': 4037104163}, {'type': 'v1', 'addr': '172.21.15.49:6835', 'nonce': 4037104163}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-06T02:14:18.648 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-06T02:14:18.648 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-06T02:14:18.785 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.790+0000 7f87e0078640 1 Processor -- start 2023-11-06T02:14:18.785 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.790+0000 7f87e0078640 1 -- start start 2023-11-06T02:14:18.786 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.790+0000 7f87e0078640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f87d815ab50 0x7f87d80c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:18.786 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.790+0000 7f87e0078640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f87d80b0790 0x7f87d80c2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:18.787 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.790+0000 7f87e0078640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87d80cab80 con 0x7f87d80cb010 2023-11-06T02:14:18.787 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.790+0000 7f87e0078640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87d815af30 con 0x7f87d815ab50 2023-11-06T02:14:18.787 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.790+0000 7f87e0078640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87d80cb5b0 con 0x7f87d80b0790 2023-11-06T02:14:18.787 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.790+0000 7f87dde14640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f87d815ab50 0x7f87d80c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:18.787 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.790+0000 7f87de615640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f87d80b0790 0x7f87d80c2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:18.788 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.790+0000 7f87dd613640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f87d80cb010 0x7f87d80b0080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.49:50554/0 (socket says 172.21.15.49:50554) 2023-11-06T02:14:18.788 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.790+0000 7f87dd613640 1 -- 172.21.15.49:0/2244971282 learned_addr learned my addr 172.21.15.49:0/2244971282 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:14:18.788 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.790+0000 7f87dde14640 1 -- 172.21.15.49:0/2244971282 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f87d80b0790 msgr2=0x7f87d80c2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:18.789 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.790+0000 7f87dde14640 1 --2- 172.21.15.49:0/2244971282 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f87d80b0790 0x7f87d80c2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:18.789 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.790+0000 7f87dde14640 1 -- 172.21.15.49:0/2244971282 >> v1:172.21.15.150:6789/0 conn(0x7f87d80cb010 legacy=0x7f87d80b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:18.789 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.790+0000 7f87dde14640 1 -- 172.21.15.49:0/2244971282 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87d80cb890 con 0x7f87d815ab50 2023-11-06T02:14:18.789 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.790+0000 7f87de615640 1 --2- 172.21.15.49:0/2244971282 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f87d80b0790 0x7f87d80c2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T02:14:18.791 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.794+0000 7f87dde14640 1 --2- 172.21.15.49:0/2244971282 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f87d815ab50 0x7f87d80c9bb0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f87d0009de0 tx=0x7f87d001c960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83a675ed3c78b017 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:18.791 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.794+0000 7f87dce12640 1 -- 172.21.15.49:0/2244971282 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d00a5a20 con 0x7f87d815ab50 2023-11-06T02:14:18.791 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.794+0000 7f87dce12640 1 -- 172.21.15.49:0/2244971282 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87d00a5b80 con 0x7f87d815ab50 2023-11-06T02:14:18.791 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.794+0000 7f87dce12640 1 -- 172.21.15.49:0/2244971282 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d00ad440 con 0x7f87d815ab50 2023-11-06T02:14:18.791 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.794+0000 7f87e0078640 1 -- 172.21.15.49:0/2244971282 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f87d815ab50 msgr2=0x7f87d80c9bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:18.792 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.794+0000 7f87e0078640 1 --2- 172.21.15.49:0/2244971282 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f87d815ab50 0x7f87d80c9bb0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f87d0009de0 tx=0x7f87d001c960 comp rx=0 tx=0).stop 2023-11-06T02:14:18.792 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.794+0000 7f87e0078640 1 -- 172.21.15.49:0/2244971282 shutdown_connections 2023-11-06T02:14:18.792 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.794+0000 7f87e0078640 1 --2- 172.21.15.49:0/2244971282 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f87d80b0790 0x7f87d80c2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:18.792 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.794+0000 7f87e0078640 1 --2- 172.21.15.49:0/2244971282 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f87d815ab50 0x7f87d80c9bb0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:18.792 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.794+0000 7f87e0078640 1 -- 172.21.15.49:0/2244971282 >> 172.21.15.49:0/2244971282 conn(0x7f87d806c0a0 msgr2=0x7f87d806c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:18.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.794+0000 7f87e0078640 1 -- 172.21.15.49:0/2244971282 shutdown_connections 2023-11-06T02:14:18.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.794+0000 7f87e0078640 1 -- 172.21.15.49:0/2244971282 wait complete. 2023-11-06T02:14:18.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.798+0000 7f87e0078640 1 Processor -- start 2023-11-06T02:14:18.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.798+0000 7f87e0078640 1 -- start start 2023-11-06T02:14:18.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.798+0000 7f87e0078640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f87d80b0790 0x7f87d81548c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:18.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.798+0000 7f87e0078640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f87d80cb010 0x7f87d8155260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:18.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.798+0000 7f87e0078640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87d815ab50 0x7f87d81f8440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:18.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.798+0000 7f87e0078640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87d8158760 con 0x7f87d815ab50 2023-11-06T02:14:18.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.798+0000 7f87e0078640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87d8157b90 con 0x7f87d80cb010 2023-11-06T02:14:18.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.798+0000 7f87e0078640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87d8157e30 con 0x7f87d80b0790 2023-11-06T02:14:18.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.798+0000 7f87de615640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87d815ab50 0x7f87d81f8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:18.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.798+0000 7f87de615640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87d815ab50 0x7f87d81f8440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54206/0 (socket says 172.21.15.49:54206) 2023-11-06T02:14:18.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.798+0000 7f87de615640 1 -- 172.21.15.49:0/197493786 learned_addr learned my addr 172.21.15.49:0/197493786 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:18.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.798+0000 7f87dde14640 1 --2- 172.21.15.49:0/197493786 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f87d80b0790 0x7f87d81548c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:18.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.798+0000 7f87dd613640 1 --2- 172.21.15.49:0/197493786 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f87d80cb010 0x7f87d8155260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:18.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.798+0000 7f87de615640 1 -- 172.21.15.49:0/197493786 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f87d80b0790 msgr2=0x7f87d81548c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:18.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.798+0000 7f87de615640 1 --2- 172.21.15.49:0/197493786 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f87d80b0790 0x7f87d81548c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:18.798 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.798+0000 7f87de615640 1 -- 172.21.15.49:0/197493786 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f87d80cb010 msgr2=0x7f87d8155260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:18.798 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.798+0000 7f87de615640 1 --2- 172.21.15.49:0/197493786 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f87d80cb010 0x7f87d8155260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:18.798 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.798+0000 7f87de615640 1 -- 172.21.15.49:0/197493786 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87d4009420 con 0x7f87d815ab50 2023-11-06T02:14:18.798 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.798+0000 7f87dd613640 1 --2- 172.21.15.49:0/197493786 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f87d80cb010 0x7f87d8155260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:14:18.798 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.798+0000 7f87dde14640 1 --2- 172.21.15.49:0/197493786 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f87d80b0790 0x7f87d81548c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:14:18.799 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.798+0000 7f87de615640 1 --2- 172.21.15.49:0/197493786 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87d815ab50 0x7f87d81f8440 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f87d400e8e0 tx=0x7f87d400eda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:18.799 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.798+0000 7f87ce7fc640 1 -- 172.21.15.49:0/197493786 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d400ca00 con 0x7f87d815ab50 2023-11-06T02:14:18.799 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.798+0000 7f87e0078640 1 -- 172.21.15.49:0/197493786 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87d0009a70 con 0x7f87d815ab50 2023-11-06T02:14:18.799 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.798+0000 7f87ce7fc640 1 -- 172.21.15.49:0/197493786 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87d400cb80 con 0x7f87d815ab50 2023-11-06T02:14:18.799 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.798+0000 7f87ce7fc640 1 -- 172.21.15.49:0/197493786 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d4004020 con 0x7f87d815ab50 2023-11-06T02:14:18.800 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.798+0000 7f87e0078640 1 -- 172.21.15.49:0/197493786 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87d81583f0 con 0x7f87d815ab50 2023-11-06T02:14:18.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.802+0000 7f87ce7fc640 1 -- 172.21.15.49:0/197493786 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f87d4004180 con 0x7f87d815ab50 2023-11-06T02:14:18.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.802+0000 7f87ce7fc640 1 --2- 172.21.15.49:0/197493786 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f87bc065a90 0x7f87bc067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:18.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.802+0000 7f87ce7fc640 1 -- 172.21.15.49:0/197493786 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f87d40832a0 con 0x7f87d815ab50 2023-11-06T02:14:18.807 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.802+0000 7f87dde14640 1 --2- 172.21.15.49:0/197493786 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f87bc065a90 0x7f87bc067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:18.807 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.802+0000 7f87e31a0000 1 Processor -- start 2023-11-06T02:14:18.807 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.802+0000 7f87e31a0000 1 -- start start 2023-11-06T02:14:18.807 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.802+0000 7f87dde14640 1 --2- 172.21.15.49:0/197493786 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f87bc065a90 0x7f87bc067f50 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f87d00af000 tx=0x7f87d00a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:18.807 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.802+0000 7f87e31a0000 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560fe12816e0 0x560fe1281ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:18.807 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.802+0000 7f87e31a0000 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560fe1282000 0x560fe1237550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:18.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.802+0000 7f87e31a0000 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fe1237a90 0x560fe1237e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:18.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.802+0000 7f87e31a0000 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fe10fffc0 con 0x560fe1282000 2023-11-06T02:14:18.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.802+0000 7f87e31a0000 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fe108e1a0 con 0x560fe1237a90 2023-11-06T02:14:18.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.802+0000 7f87e31a0000 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x560fe108b770 con 0x560fe12816e0 2023-11-06T02:14:18.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.802+0000 7f87dd613640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560fe1282000 0x560fe1237550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:18.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.802+0000 7f87dd613640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560fe1282000 0x560fe1237550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54222/0 (socket says 172.21.15.49:54222) 2023-11-06T02:14:18.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.802+0000 7f87dd613640 1 -- 172.21.15.49:0/622352772 learned_addr learned my addr 172.21.15.49:0/622352772 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:18.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.802+0000 7f87dde14640 1 --2- 172.21.15.49:0/622352772 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560fe12816e0 0x560fe1281ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:18.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.802+0000 7f87de615640 1 --2- 172.21.15.49:0/622352772 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fe1237a90 0x560fe1237e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:18.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.802+0000 7f87dd613640 1 -- 172.21.15.49:0/622352772 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560fe12816e0 msgr2=0x560fe1281ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:18.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.802+0000 7f87dd613640 1 --2- 172.21.15.49:0/622352772 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560fe12816e0 0x560fe1281ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:18.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.802+0000 7f87dd613640 1 -- 172.21.15.49:0/622352772 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fe1237a90 msgr2=0x560fe1237e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:18.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.802+0000 7f87dd613640 1 --2- 172.21.15.49:0/622352772 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fe1237a90 0x560fe1237e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:18.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.802+0000 7f87dd613640 1 -- 172.21.15.49:0/622352772 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560fe0d83890 con 0x560fe1282000 2023-11-06T02:14:18.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.802+0000 7f87dd613640 1 --2- 172.21.15.49:0/622352772 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560fe1282000 0x560fe1237550 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f87d80cae60 tx=0x7f87c8001af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d4a324a8f2f5aca server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:18.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.802+0000 7f87bb7fe640 1 -- 172.21.15.49:0/622352772 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c8001d40 con 0x560fe1282000 2023-11-06T02:14:18.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.802+0000 7f87bb7fe640 1 -- 172.21.15.49:0/622352772 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87c8005ad0 con 0x560fe1282000 2023-11-06T02:14:18.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.802+0000 7f87bb7fe640 1 -- 172.21.15.49:0/622352772 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c8005e40 con 0x560fe1282000 2023-11-06T02:14:18.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.806+0000 7f87e31a0000 1 -- 172.21.15.49:0/622352772 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560fe1282000 msgr2=0x560fe1237550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:18.812 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.806+0000 7f87e31a0000 1 --2- 172.21.15.49:0/622352772 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560fe1282000 0x560fe1237550 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f87d80cae60 tx=0x7f87c8001af0 comp rx=0 tx=0).stop 2023-11-06T02:14:18.812 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.806+0000 7f87e31a0000 1 -- 172.21.15.49:0/622352772 shutdown_connections 2023-11-06T02:14:18.812 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.806+0000 7f87e31a0000 1 --2- 172.21.15.49:0/622352772 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560fe1282000 0x560fe1237550 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:18.812 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.806+0000 7f87e31a0000 1 --2- 172.21.15.49:0/622352772 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fe1237a90 0x560fe1237e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:18.812 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.806+0000 7f87e31a0000 1 --2- 172.21.15.49:0/622352772 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560fe12816e0 0x560fe1281ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:18.813 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.806+0000 7f87e31a0000 1 -- 172.21.15.49:0/622352772 >> 172.21.15.49:0/622352772 conn(0x560fe123dbf0 msgr2=0x560fe1235e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:18.813 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.806+0000 7f87e31a0000 1 -- 172.21.15.49:0/622352772 shutdown_connections 2023-11-06T02:14:18.813 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.806+0000 7f87e31a0000 1 -- 172.21.15.49:0/622352772 wait complete. 2023-11-06T02:14:18.813 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.806+0000 7f87e31a0000 1 -- start start 2023-11-06T02:14:18.813 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.806+0000 7f87e31a0000 1 Processor -- start 2023-11-06T02:14:18.814 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.806+0000 7f87e31a0000 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560fe1237a90 0x560fe1242a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:18.814 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.806+0000 7f87e31a0000 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fe12816e0 0x560fe1242fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:18.814 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.806+0000 7f87e31a0000 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560fe1282000 0x560fe1243510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:18.814 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.806+0000 7f87e31a0000 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fe1087060 con 0x560fe1282000 2023-11-06T02:14:18.814 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.806+0000 7f87e31a0000 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fe1026290 con 0x560fe12816e0 2023-11-06T02:14:18.815 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.806+0000 7f87e31a0000 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x560fe10446f0 con 0x560fe1237a90 2023-11-06T02:14:18.815 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.806+0000 7f87e31a0000 10 client.0 _ll_register_callbacks cb 0x560fe12231d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T02:14:18.815 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.806+0000 7f87de615640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560fe1282000 0x560fe1243510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:18.815 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.806+0000 7f87de615640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560fe1282000 0x560fe1243510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54232/0 (socket says 172.21.15.49:54232) 2023-11-06T02:14:18.815 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.806+0000 7f87de615640 1 -- 172.21.15.49:0/3305441382 learned_addr learned my addr 172.21.15.49:0/3305441382 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:18.815 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.806+0000 7f87dde14640 1 --2- 172.21.15.49:0/3305441382 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560fe1237a90 0x560fe1242a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:18.816 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.806+0000 7f87dd613640 1 --2- 172.21.15.49:0/3305441382 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fe12816e0 0x560fe1242fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:18.816 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.806+0000 7f87de615640 1 -- 172.21.15.49:0/3305441382 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560fe1237a90 msgr2=0x560fe1242a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:18.816 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.806+0000 7f87de615640 1 --2- 172.21.15.49:0/3305441382 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560fe1237a90 0x560fe1242a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:18.816 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.806+0000 7f87de615640 1 -- 172.21.15.49:0/3305441382 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fe12816e0 msgr2=0x560fe1242fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:18.816 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.806+0000 7f87de615640 1 --2- 172.21.15.49:0/3305441382 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fe12816e0 0x560fe1242fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:18.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.806+0000 7f87de615640 1 -- 172.21.15.49:0/3305441382 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87c800f040 con 0x560fe1282000 2023-11-06T02:14:18.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.806+0000 7f87de615640 1 --2- 172.21.15.49:0/3305441382 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560fe1282000 0x560fe1243510 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f87d4054700 tx=0x7f87d4054f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a40e849eda1226c9 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:18.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.806+0000 7f87b8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-11-06T02:14:18.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.806+0000 7f87b8ff9640 1 -- 172.21.15.49:0/3305441382 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d40830d0 con 0x560fe1282000 2023-11-06T02:14:18.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.806+0000 7f87b8ff9640 1 -- 172.21.15.49:0/3305441382 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f87d405c750 con 0x560fe1282000 2023-11-06T02:14:18.818 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.806+0000 7f87e31a0000 1 -- 172.21.15.49:0/3305441382 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560fe10859b0 con 0x560fe1282000 2023-11-06T02:14:18.818 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.806+0000 7f87b8ff9640 1 -- 172.21.15.49:0/3305441382 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87d4058120 con 0x560fe1282000 2023-11-06T02:14:18.818 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.806+0000 7f87e31a0000 10 client.7578 fetch_fsmap learned FSMap version 52 2023-11-06T02:14:18.818 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.806+0000 7f87e31a0000 1 -- 172.21.15.49:0/3305441382 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) v3 -- 0x560fe104c1e0 con 0x560fe1282000 2023-11-06T02:14:18.818 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.806+0000 7f87b8ff9640 1 -- 172.21.15.49:0/3305441382 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560fe10859b0 con 0x560fe1282000 2023-11-06T02:14:18.819 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.806+0000 7f87b8ff9640 1 -- 172.21.15.49:0/3305441382 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 52) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f87d4011070 con 0x560fe1282000 2023-11-06T02:14:18.819 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.806+0000 7f87e31a0000 10 client.7578 fetch_fsmap finished waiting for FSMap version 52 2023-11-06T02:14:18.819 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.806+0000 7f87e31a0000 10 client.7578 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-06T02:14:18.819 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.806+0000 7f87e31a0000 20 client.7578 populate_metadata read hostname 'smithi049' 2023-11-06T02:14:18.819 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.806+0000 7f87e31a0000 1 --2- 172.21.15.49:0/3305441382 >> [v2:172.21.15.49:6834/4037104163,v1:172.21.15.49:6835/4037104163] conn(0x560fe1243de0 0x560fe1292e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:18.819 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.806+0000 7f87e31a0000 4 client.7578 mds_command: new command op to 7533 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T02:14:18.820 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.806+0000 7f87e31a0000 1 -- 172.21.15.49:0/3305441382 --> [v2:172.21.15.49:6834/4037104163,v1:172.21.15.49:6835/4037104163] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560fe10b64d0 con 0x560fe1243de0 2023-11-06T02:14:18.820 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.806+0000 7f87dde14640 1 --2- 172.21.15.49:0/3305441382 >> [v2:172.21.15.49:6834/4037104163,v1:172.21.15.49:6835/4037104163] conn(0x560fe1243de0 0x560fe1292e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:18.820 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.810+0000 7f87dde14640 1 --2- 172.21.15.49:0/3305441382 >> [v2:172.21.15.49:6834/4037104163,v1:172.21.15.49:6835/4037104163] conn(0x560fe1243de0 0x560fe1292e60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51bcd1123f92414a server_cookie=5ab9a9a3bb7b5911 in_seq=0 out_seq=0 2023-11-06T02:14:18.820 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.810+0000 7f87b8ff9640 10 client.7578 ms_handle_connect on v2:172.21.15.49:6834/4037104163 2023-11-06T02:14:18.820 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.810+0000 7f87b8ff9640 1 -- 172.21.15.49:0/3305441382 <== mds.0 v2:172.21.15.49:6834/4037104163 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x560fe10b64d0 con 0x560fe1243de0 2023-11-06T02:14:18.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.810+0000 7f87b8ff9640 10 client.7578 handle_command_reply: tid=0 2023-11-06T02:14:18.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.810+0000 7f87e31a0000 1 client.7578 shutdown 2023-11-06T02:14:18.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.810+0000 7f87e31a0000 1 -- 172.21.15.49:0/3305441382 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560fe1282000 msgr2=0x560fe1243510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:18.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.810+0000 7f87e31a0000 1 --2- 172.21.15.49:0/3305441382 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560fe1282000 0x560fe1243510 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f87d4054700 tx=0x7f87d4054f30 comp rx=0 tx=0).stop 2023-11-06T02:14:18.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.810+0000 7f87e31a0000 1 -- 172.21.15.49:0/3305441382 shutdown_connections 2023-11-06T02:14:18.822 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.810+0000 7f87e31a0000 1 --2- 172.21.15.49:0/3305441382 >> [v2:172.21.15.49:6834/4037104163,v1:172.21.15.49:6835/4037104163] conn(0x560fe1243de0 0x560fe1292e60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:18.822 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.810+0000 7f87e31a0000 1 --2- 172.21.15.49:0/3305441382 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560fe1282000 0x560fe1243510 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:18.822 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.810+0000 7f87e31a0000 1 --2- 172.21.15.49:0/3305441382 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fe12816e0 0x560fe1242fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:18.822 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.810+0000 7f87b8ff9640 0 client.7578 ms_handle_reset on v2:172.21.15.49:6834/4037104163 2023-11-06T02:14:18.822 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.810+0000 7f87e31a0000 1 --2- 172.21.15.49:0/3305441382 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560fe1237a90 0x560fe1242a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:18.823 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.810+0000 7f87e31a0000 1 -- 172.21.15.49:0/3305441382 >> 172.21.15.49:0/3305441382 conn(0x560fe123dbf0 msgr2=0x560fe1236530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:18.823 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.810+0000 7f87e31a0000 1 -- 172.21.15.49:0/3305441382 shutdown_connections 2023-11-06T02:14:18.823 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.810+0000 7f87e31a0000 1 -- 172.21.15.49:0/3305441382 wait complete. 2023-11-06T02:14:18.823 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.810+0000 7f87e31a0000 20 client.7578 trim_cache size 0 max 16384 2023-11-06T02:14:18.829 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.826+0000 7f87e31a0000 1 Processor -- start 2023-11-06T02:14:18.829 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.826+0000 7f87e31a0000 1 -- start start 2023-11-06T02:14:18.829 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.826+0000 7f87e31a0000 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560fe1237a90 0x560fe1280a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:18.829 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.826+0000 7f87e31a0000 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560fe12816e0 0x560fe1293e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:18.830 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.826+0000 7f87e31a0000 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fe1282000 0x560fe12943b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:18.830 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.826+0000 7f87e31a0000 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fe0d44940 con 0x560fe1237a90 2023-11-06T02:14:18.830 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.826+0000 7f87e31a0000 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fe1085030 con 0x560fe1282000 2023-11-06T02:14:18.830 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.826+0000 7f87e31a0000 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x560fe1082fd0 con 0x560fe12816e0 2023-11-06T02:14:18.830 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.826+0000 7f87dde14640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560fe1237a90 0x560fe1280a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:18.831 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.826+0000 7f87dde14640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560fe1237a90 0x560fe1280a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54242/0 (socket says 172.21.15.49:54242) 2023-11-06T02:14:18.831 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.826+0000 7f87dde14640 1 -- 172.21.15.49:0/1952650986 learned_addr learned my addr 172.21.15.49:0/1952650986 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:18.831 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.826+0000 7f87dde14640 1 -- 172.21.15.49:0/1952650986 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560fe12816e0 msgr2=0x560fe1293e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:18.831 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.826+0000 7f87dde14640 1 --2- 172.21.15.49:0/1952650986 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560fe12816e0 0x560fe1293e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:18.831 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.826+0000 7f87dde14640 1 -- 172.21.15.49:0/1952650986 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fe1282000 msgr2=0x560fe12943b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:18.832 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.826+0000 7f87de615640 1 --2- 172.21.15.49:0/1952650986 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fe1282000 0x560fe12943b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:18.832 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.826+0000 7f87dde14640 1 --2- 172.21.15.49:0/1952650986 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fe1282000 0x560fe12943b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:18.832 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.826+0000 7f87dde14640 1 -- 172.21.15.49:0/1952650986 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560fe1032330 con 0x560fe1237a90 2023-11-06T02:14:18.832 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.826+0000 7f87dde14640 1 --2- 172.21.15.49:0/1952650986 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560fe1237a90 0x560fe1280a00 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f87d0009de0 tx=0x7f87d0004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bef7b4a9f53702d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:18.832 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.826+0000 7f87bbfff640 1 -- 172.21.15.49:0/1952650986 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d00d0070 con 0x560fe1237a90 2023-11-06T02:14:18.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.826+0000 7f87bbfff640 1 -- 172.21.15.49:0/1952650986 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87d00cc310 con 0x560fe1237a90 2023-11-06T02:14:18.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.826+0000 7f87bbfff640 1 -- 172.21.15.49:0/1952650986 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d00cb070 con 0x560fe1237a90 2023-11-06T02:14:18.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.826+0000 7f87e31a0000 1 -- 172.21.15.49:0/1952650986 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560fe1237a90 msgr2=0x560fe1280a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:18.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.826+0000 7f87e31a0000 1 --2- 172.21.15.49:0/1952650986 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560fe1237a90 0x560fe1280a00 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f87d0009de0 tx=0x7f87d0004ac0 comp rx=0 tx=0).stop 2023-11-06T02:14:18.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.826+0000 7f87e31a0000 1 -- 172.21.15.49:0/1952650986 shutdown_connections 2023-11-06T02:14:18.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.826+0000 7f87e31a0000 1 --2- 172.21.15.49:0/1952650986 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fe1282000 0x560fe12943b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:18.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.826+0000 7f87e31a0000 1 --2- 172.21.15.49:0/1952650986 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560fe12816e0 0x560fe1293e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:18.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.826+0000 7f87e31a0000 1 --2- 172.21.15.49:0/1952650986 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560fe1237a90 0x560fe1280a00 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:18.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.826+0000 7f87e31a0000 1 -- 172.21.15.49:0/1952650986 >> 172.21.15.49:0/1952650986 conn(0x560fe123dbf0 msgr2=0x560fe123cac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:18.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.826+0000 7f87e31a0000 1 -- 172.21.15.49:0/1952650986 shutdown_connections 2023-11-06T02:14:18.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.826+0000 7f87e31a0000 1 -- 172.21.15.49:0/1952650986 wait complete. 2023-11-06T02:14:18.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.826+0000 7f87e31a0000 1 -- start start 2023-11-06T02:14:18.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.826+0000 7f87e31a0000 1 Processor -- start 2023-11-06T02:14:18.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.826+0000 7f87e31a0000 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fe1237a90 0x560fe12de130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:18.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.826+0000 7f87e31a0000 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560fe12816e0 0x560fe12de670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:18.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.826+0000 7f87e31a0000 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560fe1282000 0x560fe12da6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:18.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.826+0000 7f87e31a0000 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fe11505b0 con 0x560fe12816e0 2023-11-06T02:14:18.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.826+0000 7f87e31a0000 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fe1049960 con 0x560fe1237a90 2023-11-06T02:14:18.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.826+0000 7f87e31a0000 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x560fe110ee30 con 0x560fe1282000 2023-11-06T02:14:18.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.830+0000 7f87e31a0000 10 client.0 _ll_register_callbacks cb 0x560fe12039e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T02:14:18.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.830+0000 7f87dd613640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560fe12816e0 0x560fe12de670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:18.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.830+0000 7f87dd613640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560fe12816e0 0x560fe12de670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54244/0 (socket says 172.21.15.49:54244) 2023-11-06T02:14:18.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.830+0000 7f87dd613640 1 -- 172.21.15.49:0/2353801162 learned_addr learned my addr 172.21.15.49:0/2353801162 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:18.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.830+0000 7f87de615640 1 --2- 172.21.15.49:0/2353801162 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560fe1282000 0x560fe12da6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:18.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.830+0000 7f87dde14640 1 --2- 172.21.15.49:0/2353801162 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fe1237a90 0x560fe12de130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:18.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.830+0000 7f87dd613640 1 -- 172.21.15.49:0/2353801162 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560fe1282000 msgr2=0x560fe12da6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:18.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.830+0000 7f87dd613640 1 --2- 172.21.15.49:0/2353801162 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560fe1282000 0x560fe12da6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:18.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.830+0000 7f87dd613640 1 -- 172.21.15.49:0/2353801162 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fe1237a90 msgr2=0x560fe12de130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:18.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.830+0000 7f87dd613640 1 --2- 172.21.15.49:0/2353801162 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fe1237a90 0x560fe12de130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:18.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.830+0000 7f87dd613640 1 -- 172.21.15.49:0/2353801162 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87d80cb890 con 0x560fe12816e0 2023-11-06T02:14:18.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.830+0000 7f87dd613640 1 --2- 172.21.15.49:0/2353801162 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560fe12816e0 0x560fe12de670 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x560fe1105e40 tx=0x7f87c8005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84aaf276d37f0d61 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:18.839 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.830+0000 7f87b8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-11-06T02:14:18.839 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.830+0000 7f87b8ff9640 1 -- 172.21.15.49:0/2353801162 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c8005ce0 con 0x560fe12816e0 2023-11-06T02:14:18.839 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.830+0000 7f87b8ff9640 1 -- 172.21.15.49:0/2353801162 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f87c8005e40 con 0x560fe12816e0 2023-11-06T02:14:18.839 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.830+0000 7f87e31a0000 1 -- 172.21.15.49:0/2353801162 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560fe1281350 con 0x560fe12816e0 2023-11-06T02:14:18.839 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.830+0000 7f87b8ff9640 1 -- 172.21.15.49:0/2353801162 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87c80070d0 con 0x560fe12816e0 2023-11-06T02:14:18.840 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.830+0000 7f87e31a0000 10 client.7584 fetch_fsmap learned FSMap version 52 2023-11-06T02:14:18.840 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.830+0000 7f87e31a0000 1 -- 172.21.15.49:0/2353801162 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) v3 -- 0x560fe0d83cc0 con 0x560fe12816e0 2023-11-06T02:14:18.840 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.830+0000 7f87b8ff9640 1 -- 172.21.15.49:0/2353801162 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560fe1281350 con 0x560fe12816e0 2023-11-06T02:14:18.840 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.830+0000 7f87b8ff9640 1 -- 172.21.15.49:0/2353801162 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 52) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f87c8007230 con 0x560fe12816e0 2023-11-06T02:14:18.840 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.830+0000 7f87e31a0000 10 client.7584 fetch_fsmap finished waiting for FSMap version 52 2023-11-06T02:14:18.841 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.830+0000 7f87e31a0000 10 client.7584 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-06T02:14:18.841 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.830+0000 7f87e31a0000 20 client.7584 populate_metadata read hostname 'smithi049' 2023-11-06T02:14:18.841 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.830+0000 7f87e31a0000 1 --2- 172.21.15.49:0/2353801162 >> [v2:172.21.15.49:6834/4037104163,v1:172.21.15.49:6835/4037104163] conn(0x560fe12dac30 0x560fe12db010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:18.841 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.830+0000 7f87e31a0000 4 client.7584 mds_command: new command op to 7533 tid=0[{"prefix": "status"}] 2023-11-06T02:14:18.841 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.830+0000 7f87e31a0000 1 -- 172.21.15.49:0/2353801162 --> [v2:172.21.15.49:6834/4037104163,v1:172.21.15.49:6835/4037104163] -- command(tid 0: {"prefix": "status"}) v1 -- 0x560fe1056b90 con 0x560fe12dac30 2023-11-06T02:14:18.841 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.830+0000 7f87dde14640 1 --2- 172.21.15.49:0/2353801162 >> [v2:172.21.15.49:6834/4037104163,v1:172.21.15.49:6835/4037104163] conn(0x560fe12dac30 0x560fe12db010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:18.842 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.830+0000 7f87dde14640 1 --2- 172.21.15.49:0/2353801162 >> [v2:172.21.15.49:6834/4037104163,v1:172.21.15.49:6835/4037104163] conn(0x560fe12dac30 0x560fe12db010 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fc922f59d0a6c8e server_cookie=4ba9af8eb3e3fee6 in_seq=0 out_seq=0 2023-11-06T02:14:18.842 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.830+0000 7f87b8ff9640 10 client.7584 ms_handle_connect on v2:172.21.15.49:6834/4037104163 2023-11-06T02:14:18.842 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.830+0000 7f87b8ff9640 1 -- 172.21.15.49:0/2353801162 <== mds.0 v2:172.21.15.49:6834/4037104163 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x560fe1056b90 con 0x560fe12dac30 2023-11-06T02:14:18.842 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.830+0000 7f87b8ff9640 10 client.7584 handle_command_reply: tid=0 2023-11-06T02:14:18.842 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.830+0000 7f87e31a0000 1 client.7584 shutdown 2023-11-06T02:14:18.843 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.830+0000 7f87e31a0000 1 -- 172.21.15.49:0/2353801162 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560fe12816e0 msgr2=0x560fe12de670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:18.843 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.830+0000 7f87e31a0000 1 --2- 172.21.15.49:0/2353801162 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560fe12816e0 0x560fe12de670 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x560fe1105e40 tx=0x7f87c8005800 comp rx=0 tx=0).stop 2023-11-06T02:14:18.843 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.830+0000 7f87e31a0000 1 -- 172.21.15.49:0/2353801162 shutdown_connections 2023-11-06T02:14:18.843 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.830+0000 7f87e31a0000 1 --2- 172.21.15.49:0/2353801162 >> [v2:172.21.15.49:6834/4037104163,v1:172.21.15.49:6835/4037104163] conn(0x560fe12dac30 0x560fe12db010 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:18.843 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.830+0000 7f87e31a0000 1 --2- 172.21.15.49:0/2353801162 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560fe12816e0 0x560fe12de670 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:18.844 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.830+0000 7f87e31a0000 1 --2- 172.21.15.49:0/2353801162 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560fe1282000 0x560fe12da6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:18.844 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.830+0000 7f87b8ff9640 0 client.7584 ms_handle_reset on v2:172.21.15.49:6834/4037104163 2023-11-06T02:14:18.844 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.830+0000 7f87e31a0000 1 --2- 172.21.15.49:0/2353801162 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fe1237a90 0x560fe12de130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:18.844 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.830+0000 7f87e31a0000 1 -- 172.21.15.49:0/2353801162 >> 172.21.15.49:0/2353801162 conn(0x560fe123dbf0 msgr2=0x560fe1292e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:18.844 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.830+0000 7f87e31a0000 1 -- 172.21.15.49:0/2353801162 shutdown_connections 2023-11-06T02:14:18.845 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.830+0000 7f87e31a0000 1 -- 172.21.15.49:0/2353801162 wait complete. 2023-11-06T02:14:18.845 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.830+0000 7f87e31a0000 20 client.7584 trim_cache size 0 max 16384 2023-11-06T02:14:18.845 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-11-06T02:14:18.845 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_fsid": "1a8b035c-6ba4-499a-b876-aea47fd9c0d4", 2023-11-06T02:14:18.846 INFO:teuthology.orchestra.run.smithi049.stdout: "whoami": 0, 2023-11-06T02:14:18.846 INFO:teuthology.orchestra.run.smithi049.stdout: "id": 7533, 2023-11-06T02:14:18.846 INFO:teuthology.orchestra.run.smithi049.stdout: "want_state": "up:active", 2023-11-06T02:14:18.846 INFO:teuthology.orchestra.run.smithi049.stdout: "state": "up:active", 2023-11-06T02:14:18.846 INFO:teuthology.orchestra.run.smithi049.stdout: "fs_name": "cephfs", 2023-11-06T02:14:18.847 INFO:teuthology.orchestra.run.smithi049.stdout: "rank_uptime": 4.4805819930000004, 2023-11-06T02:14:18.847 INFO:teuthology.orchestra.run.smithi049.stdout: "mdsmap_epoch": 52, 2023-11-06T02:14:18.847 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch": 81, 2023-11-06T02:14:18.847 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch_barrier": 81, 2023-11-06T02:14:18.847 INFO:teuthology.orchestra.run.smithi049.stdout: "uptime": 5.4438936780000002 2023-11-06T02:14:18.848 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-11-06T02:14:18.849 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.830+0000 7f87e0078640 1 -- 172.21.15.49:0/197493786 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f87bc065a90 msgr2=0x7f87bc067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:18.849 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.830+0000 7f87e0078640 1 --2- 172.21.15.49:0/197493786 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f87bc065a90 0x7f87bc067f50 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f87d00af000 tx=0x7f87d00a7070 comp rx=0 tx=0).stop 2023-11-06T02:14:18.849 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.830+0000 7f87e0078640 1 -- 172.21.15.49:0/197493786 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87d815ab50 msgr2=0x7f87d81f8440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:18.850 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.830+0000 7f87e0078640 1 --2- 172.21.15.49:0/197493786 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87d815ab50 0x7f87d81f8440 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f87d400e8e0 tx=0x7f87d400eda0 comp rx=0 tx=0).stop 2023-11-06T02:14:18.850 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.830+0000 7f87e0078640 1 -- 172.21.15.49:0/197493786 shutdown_connections 2023-11-06T02:14:18.850 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.830+0000 7f87e0078640 1 --2- 172.21.15.49:0/197493786 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f87bc065a90 0x7f87bc067f50 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:18.850 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.830+0000 7f87e0078640 1 --2- 172.21.15.49:0/197493786 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87d815ab50 0x7f87d81f8440 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:18.850 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.830+0000 7f87e0078640 1 --2- 172.21.15.49:0/197493786 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f87d80cb010 0x7f87d8155260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:18.851 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.830+0000 7f87e0078640 1 --2- 172.21.15.49:0/197493786 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f87d80b0790 0x7f87d81548c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:18.851 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.830+0000 7f87e0078640 1 -- 172.21.15.49:0/197493786 >> 172.21.15.49:0/197493786 conn(0x7f87d806c0a0 msgr2=0x7f87d80c9bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:18.851 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.830+0000 7f87e0078640 1 -- 172.21.15.49:0/197493786 shutdown_connections 2023-11-06T02:14:18.851 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.830+0000 7f87e0078640 1 -- 172.21.15.49:0/197493786 wait complete. 2023-11-06T02:14:18.852 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-06T02:14:18.966 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.970+0000 7f5358663640 1 Processor -- start 2023-11-06T02:14:18.966 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.970+0000 7f5358663640 1 -- start start 2023-11-06T02:14:18.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.974+0000 7f5358663640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f535015ab50 0x7f53500c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:18.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.974+0000 7f5358663640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f53500b0790 0x7f53500c2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:18.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.974+0000 7f5358663640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53500cab80 con 0x7f53500cb010 2023-11-06T02:14:18.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.974+0000 7f5358663640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f535015af30 con 0x7f535015ab50 2023-11-06T02:14:18.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.974+0000 7f5358663640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53500cb5b0 con 0x7f53500b0790 2023-11-06T02:14:18.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.974+0000 7f53563ff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f535015ab50 0x7f53500c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:18.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.974+0000 7f53563ff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f535015ab50 0x7f53500c9bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54248/0 (socket says 172.21.15.49:54248) 2023-11-06T02:14:18.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.974+0000 7f53563ff640 1 -- 172.21.15.49:0/2420644249 learned_addr learned my addr 172.21.15.49:0/2420644249 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:18.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.974+0000 7f53563ff640 1 -- 172.21.15.49:0/2420644249 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f53500b0790 msgr2=0x7f53500c2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:18.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.974+0000 7f5356c00640 1 --2- 172.21.15.49:0/2420644249 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f53500b0790 0x7f53500c2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:18.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.974+0000 7f53563ff640 1 --2- 172.21.15.49:0/2420644249 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f53500b0790 0x7f53500c2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:18.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.974+0000 7f53563ff640 1 -- 172.21.15.49:0/2420644249 >> v1:172.21.15.150:6789/0 conn(0x7f53500cb010 legacy=0x7f53500b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:18.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.974+0000 7f53563ff640 1 -- 172.21.15.49:0/2420644249 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53500cb890 con 0x7f535015ab50 2023-11-06T02:14:18.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.974+0000 7f5356c00640 1 --2- 172.21.15.49:0/2420644249 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f53500b0790 0x7f53500c2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:14:18.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.974+0000 7f53563ff640 1 --2- 172.21.15.49:0/2420644249 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f535015ab50 0x7f53500c9bb0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f534c006fd0 tx=0x7f534c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caa7484e2e5e30e8 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:18.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.974+0000 7f53553fd640 1 -- 172.21.15.49:0/2420644249 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f534c0aa650 con 0x7f535015ab50 2023-11-06T02:14:18.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.974+0000 7f53553fd640 1 -- 172.21.15.49:0/2420644249 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f534c0aa7b0 con 0x7f535015ab50 2023-11-06T02:14:18.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.974+0000 7f53553fd640 1 -- 172.21.15.49:0/2420644249 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f534c0aab40 con 0x7f535015ab50 2023-11-06T02:14:18.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.974+0000 7f5358663640 1 -- 172.21.15.49:0/2420644249 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f535015ab50 msgr2=0x7f53500c9bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:18.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.974+0000 7f5358663640 1 --2- 172.21.15.49:0/2420644249 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f535015ab50 0x7f53500c9bb0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f534c006fd0 tx=0x7f534c01c4f0 comp rx=0 tx=0).stop 2023-11-06T02:14:18.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.974+0000 7f5358663640 1 -- 172.21.15.49:0/2420644249 shutdown_connections 2023-11-06T02:14:18.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.974+0000 7f5358663640 1 --2- 172.21.15.49:0/2420644249 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f53500b0790 0x7f53500c2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:18.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.974+0000 7f5358663640 1 --2- 172.21.15.49:0/2420644249 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f535015ab50 0x7f53500c9bb0 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:18.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.974+0000 7f5358663640 1 -- 172.21.15.49:0/2420644249 >> 172.21.15.49:0/2420644249 conn(0x7f535006c0a0 msgr2=0x7f535006c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:18.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.974+0000 7f5358663640 1 -- 172.21.15.49:0/2420644249 shutdown_connections 2023-11-06T02:14:18.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.974+0000 7f5358663640 1 -- 172.21.15.49:0/2420644249 wait complete. 2023-11-06T02:14:18.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.978+0000 7f5358663640 1 Processor -- start 2023-11-06T02:14:18.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.978+0000 7f5358663640 1 -- start start 2023-11-06T02:14:18.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.978+0000 7f5358663640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f53500b0790 0x7f53500acd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:18.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.978+0000 7f5358663640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f53500cb010 0x7f53501f5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:18.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.978+0000 7f5358663640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f535015ab50 0x7f53500cda10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:18.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.978+0000 7f5358663640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53500ccdd0 con 0x7f535015ab50 2023-11-06T02:14:18.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.978+0000 7f5358663640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53500ad270 con 0x7f53500cb010 2023-11-06T02:14:18.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.978+0000 7f5358663640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53501f60c0 con 0x7f53500b0790 2023-11-06T02:14:18.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.978+0000 7f5356c00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f535015ab50 0x7f53500cda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:18.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.978+0000 7f5356c00640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f535015ab50 0x7f53500cda10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54262/0 (socket says 172.21.15.49:54262) 2023-11-06T02:14:18.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.978+0000 7f5356c00640 1 -- 172.21.15.49:0/2913481798 learned_addr learned my addr 172.21.15.49:0/2913481798 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:18.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.978+0000 7f53563ff640 1 --2- 172.21.15.49:0/2913481798 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f53500b0790 0x7f53500acd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:18.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.978+0000 7f5355bfe640 1 --2- 172.21.15.49:0/2913481798 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f53500cb010 0x7f53501f5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:18.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.978+0000 7f5356c00640 1 -- 172.21.15.49:0/2913481798 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f53500b0790 msgr2=0x7f53500acd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:18.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.978+0000 7f5356c00640 1 --2- 172.21.15.49:0/2913481798 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f53500b0790 0x7f53500acd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:18.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.978+0000 7f5356c00640 1 -- 172.21.15.49:0/2913481798 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f53500cb010 msgr2=0x7f53501f5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:18.980 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.978+0000 7f5356c00640 1 --2- 172.21.15.49:0/2913481798 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f53500cb010 0x7f53501f5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:18.980 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.978+0000 7f5356c00640 1 -- 172.21.15.49:0/2913481798 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f534c009480 con 0x7f535015ab50 2023-11-06T02:14:18.980 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.978+0000 7f5355bfe640 1 --2- 172.21.15.49:0/2913481798 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f53500cb010 0x7f53501f5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:14:18.980 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.978+0000 7f53563ff640 1 --2- 172.21.15.49:0/2913481798 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f53500b0790 0x7f53500acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:14:18.980 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.978+0000 7f5356c00640 1 --2- 172.21.15.49:0/2913481798 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f535015ab50 0x7f53500cda10 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f534800cf20 tx=0x7f534800a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:18.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.978+0000 7f5346ffd640 1 -- 172.21.15.49:0/2913481798 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f534800d840 con 0x7f535015ab50 2023-11-06T02:14:18.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.978+0000 7f5346ffd640 1 -- 172.21.15.49:0/2913481798 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f534800d9a0 con 0x7f535015ab50 2023-11-06T02:14:18.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.978+0000 7f5346ffd640 1 -- 172.21.15.49:0/2913481798 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f534800dd50 con 0x7f535015ab50 2023-11-06T02:14:18.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.978+0000 7f5358663640 1 -- 172.21.15.49:0/2913481798 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53500cc810 con 0x7f535015ab50 2023-11-06T02:14:18.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.978+0000 7f5358663640 1 -- 172.21.15.49:0/2913481798 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53500cd210 con 0x7f535015ab50 2023-11-06T02:14:18.982 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.982+0000 7f5358663640 1 -- 172.21.15.49:0/2913481798 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53500b0de0 con 0x7f535015ab50 2023-11-06T02:14:18.982 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.982+0000 7f5346ffd640 1 -- 172.21.15.49:0/2913481798 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f534801e3b0 con 0x7f535015ab50 2023-11-06T02:14:18.982 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.982+0000 7f5346ffd640 1 --2- 172.21.15.49:0/2913481798 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5334065a60 0x7f5334067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:18.982 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.982+0000 7f5346ffd640 1 -- 172.21.15.49:0/2913481798 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f5348083d80 con 0x7f535015ab50 2023-11-06T02:14:18.982 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.982+0000 7f53563ff640 1 --2- 172.21.15.49:0/2913481798 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5334065a60 0x7f5334067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:18.983 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.982+0000 7f53563ff640 1 --2- 172.21.15.49:0/2913481798 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5334065a60 0x7f5334067f20 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f534c0ae000 tx=0x7f534c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:18.983 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:18.986+0000 7f5346ffd640 1 -- 172.21.15.49:0/2913481798 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f5348051570 con 0x7f535015ab50 2023-11-06T02:14:19.163 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:19.166+0000 7f5358663640 1 -- 172.21.15.49:0/2913481798 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f5350074040 con 0x7f535015ab50 2023-11-06T02:14:19.163 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:19.170+0000 7f5346ffd640 1 -- 172.21.15.49:0/2913481798 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v52) v1 ==== 53+0+73 (secure 0 0 0) 0x7f534801a070 con 0x7f535015ab50 2023-11-06T02:14:19.164 INFO:teuthology.orchestra.run.smithi049.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-06T02:14:19.165 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:19.170+0000 7f5358663640 1 -- 172.21.15.49:0/2913481798 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5334065a60 msgr2=0x7f5334067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:19.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:19.170+0000 7f5358663640 1 --2- 172.21.15.49:0/2913481798 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5334065a60 0x7f5334067f20 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f534c0ae000 tx=0x7f534c009040 comp rx=0 tx=0).stop 2023-11-06T02:14:19.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:19.170+0000 7f5358663640 1 -- 172.21.15.49:0/2913481798 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f535015ab50 msgr2=0x7f53500cda10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:19.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:19.170+0000 7f5358663640 1 --2- 172.21.15.49:0/2913481798 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f535015ab50 0x7f53500cda10 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f534800cf20 tx=0x7f534800a7d0 comp rx=0 tx=0).stop 2023-11-06T02:14:19.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:19.170+0000 7f5358663640 1 -- 172.21.15.49:0/2913481798 shutdown_connections 2023-11-06T02:14:19.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:19.170+0000 7f5358663640 1 --2- 172.21.15.49:0/2913481798 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5334065a60 0x7f5334067f20 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:19.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:19.170+0000 7f5358663640 1 --2- 172.21.15.49:0/2913481798 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f535015ab50 0x7f53500cda10 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:19.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:19.170+0000 7f5358663640 1 --2- 172.21.15.49:0/2913481798 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f53500cb010 0x7f53501f5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:19.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:19.170+0000 7f5358663640 1 --2- 172.21.15.49:0/2913481798 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f53500b0790 0x7f53500acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:19.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:19.170+0000 7f5358663640 1 -- 172.21.15.49:0/2913481798 >> 172.21.15.49:0/2913481798 conn(0x7f535006c0a0 msgr2=0x7f53500afde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:19.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:19.170+0000 7f5358663640 1 -- 172.21.15.49:0/2913481798 shutdown_connections 2023-11-06T02:14:19.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:19.170+0000 7f5358663640 1 -- 172.21.15.49:0/2913481798 wait complete. 2023-11-06T02:14:19.181 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-06T02:14:19.181 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-06T02:14:19.182 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi049.front.sepia.ceph.com 2023-11-06T02:14:19.182 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-06T02:14:19.182 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-06T02:14:19.183 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-06T02:14:19.183 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-06T02:14:19.183 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-06T02:14:19.184 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-06T02:14:19.184 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:14:19.185 DEBUG:teuthology.orchestra.run.smithi049:> ip addr 2023-11-06T02:14:19.190 INFO:teuthology.orchestra.run.smithi049.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-06T02:14:19.190 INFO:teuthology.orchestra.run.smithi049.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-06T02:14:19.191 INFO:teuthology.orchestra.run.smithi049.stdout: inet 127.0.0.1/8 scope host lo 2023-11-06T02:14:19.191 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:14:19.191 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 ::1/128 scope host 2023-11-06T02:14:19.191 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:14:19.192 INFO:teuthology.orchestra.run.smithi049.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-06T02:14:19.192 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:6b:ff:84 brd ff:ff:ff:ff:ff:ff 2023-11-06T02:14:19.192 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp7s0f0 2023-11-06T02:14:19.192 INFO:teuthology.orchestra.run.smithi049.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-06T02:14:19.192 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:6b:ff:85 brd ff:ff:ff:ff:ff:ff 2023-11-06T02:14:19.193 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp7s0f1 2023-11-06T02:14:19.193 INFO:teuthology.orchestra.run.smithi049.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-06T02:14:19.193 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:bd:15:66 brd ff:ff:ff:ff:ff:ff 2023-11-06T02:14:19.193 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp1s0f0 2023-11-06T02:14:19.193 INFO:teuthology.orchestra.run.smithi049.stdout: inet 172.21.15.49/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-11-06T02:14:19.194 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft 42138sec preferred_lft 42138sec 2023-11-06T02:14:19.194 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 fe80::ec4:7aff:febd:1566/64 scope link 2023-11-06T02:14:19.194 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:14:19.194 INFO:teuthology.orchestra.run.smithi049.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-06T02:14:19.195 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:bd:15:67 brd ff:ff:ff:ff:ff:ff 2023-11-06T02:14:19.195 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp1s0f1 2023-11-06T02:14:19.195 INFO:teuthology.orchestra.run.smithi049.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-06T02:14:19.195 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 02:42:47:49:33:69 brd ff:ff:ff:ff:ff:ff 2023-11-06T02:14:19.195 INFO:teuthology.orchestra.run.smithi049.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-06T02:14:19.196 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:14:19.196 INFO:teuthology.orchestra.run.smithi049.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-06T02:14:19.196 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-11-06T02:14:19.196 INFO:teuthology.orchestra.run.smithi049.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-06T02:14:19.196 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:14:19.197 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-11-06T02:14:19.197 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:14:19.197 INFO:teuthology.orchestra.run.smithi049.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-06T02:14:19.197 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-06T02:14:19.198 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-11-06T02:14:19.198 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:14:19.198 INFO:teuthology.orchestra.run.smithi049.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-06T02:14:19.198 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-06T02:14:19.198 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-11-06T02:14:19.199 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:14:19.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:14:19.199 DEBUG:teuthology.orchestra.run.smithi049:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-06T02:14:19.240 INFO:teuthology.orchestra.run.smithi049.stdout:1 2023-11-06T02:14:19.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:14:19.242 DEBUG:teuthology.orchestra.run.smithi049:> route 2023-11-06T02:14:19.289 INFO:teuthology.orchestra.run.smithi049.stdout:Kernel IP routing table 2023-11-06T02:14:19.289 INFO:teuthology.orchestra.run.smithi049.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-06T02:14:19.289 INFO:teuthology.orchestra.run.smithi049.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-11-06T02:14:19.289 INFO:teuthology.orchestra.run.smithi049.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-06T02:14:19.290 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-11-06T02:14:19.290 INFO:teuthology.orchestra.run.smithi049.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-06T02:14:19.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:14:19.291 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec bash -c ' 2023-11-06T02:14:19.291 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2023-11-06T02:14:19.292 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-06T02:14:19.292 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-06T02:14:19.292 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-06T02:14:19.293 DEBUG:teuthology.orchestra.run.smithi049:> ') 2023-11-06T02:14:19.364 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:14:19.364 DEBUG:teuthology.orchestra.run.smithi049:> ip netns list 2023-11-06T02:14:19.412 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-06T02:14:19.413 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-06T02:14:19.413 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-06T02:14:19.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:14:19.414 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link set brx.0 up 2023-11-06T02:14:19.469 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-06T02:14:19.469 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-06T02:14:19.470 DEBUG:teuthology.orchestra.run.smithi049:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-06T02:14:19.517 INFO:teuthology.orchestra.run.smithi049.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-06T02:14:19.517 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-06T02:14:19.517 DEBUG:teuthology.orchestra.run.smithi049:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-06T02:14:19.564 DEBUG:teuthology.orchestra.run.smithi049:> sudo modprobe fuse 2023-11-06T02:14:19.621 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-06T02:14:19.622 DEBUG:teuthology.orchestra.run.smithi049:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-06T02:14:19.676 INFO:teuthology.orchestra.run.smithi049.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-06T02:14:19.677 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-06T02:14:19.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:14:19.678 DEBUG:teuthology.orchestra.run.smithi049:> ls /sys/fs/fuse/connections 2023-11-06T02:14:19.724 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-06T02:14:19.725 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-06T02:14:19.769 DEBUG:teuthology.orchestra.run.smithi049:> sudo modprobe fuse 2023-11-06T02:14:19.784 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-06T02:14:19.784 DEBUG:teuthology.orchestra.run.smithi049:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-06T02:14:19.839 INFO:teuthology.orchestra.run.smithi049.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-06T02:14:19.840 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-06T02:14:19.841 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:14:19.842 DEBUG:teuthology.orchestra.run.smithi049:> ls /sys/fs/fuse/connections 2023-11-06T02:14:19.853 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi049.stderr:2023-11-06T02:14:19.858+0000 7f87e7677580 -1 init, newargv = 0x55f403914c60 newargc=15 2023-11-06T02:14:19.853 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi049.stderr:ceph-fuse[56153]: starting ceph client 2023-11-06T02:14:19.861 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi049.stderr:ceph-fuse[56153]: starting fuse 2023-11-06T02:14:20.346 INFO:tasks.ceph.mds.c.smithi049.stderr:2023-11-06T02:14:20.350+0000 7f1efeaa9640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-06T02:14:20.959 DEBUG:teuthology.orchestra.run.smithi049:> sudo modprobe fuse 2023-11-06T02:14:20.977 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-06T02:14:20.978 DEBUG:teuthology.orchestra.run.smithi049:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-06T02:14:21.031 INFO:teuthology.orchestra.run.smithi049.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-06T02:14:21.032 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-06T02:14:21.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:14:21.033 DEBUG:teuthology.orchestra.run.smithi049:> ls /sys/fs/fuse/connections 2023-11-06T02:14:21.084 INFO:teuthology.orchestra.run.smithi049.stdout:45 2023-11-06T02:14:21.084 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2023-11-06T02:14:21.085 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-11-06T02:14:21.085 DEBUG:teuthology.orchestra.run.smithi049:> import glob 2023-11-06T02:14:21.086 DEBUG:teuthology.orchestra.run.smithi049:> import re 2023-11-06T02:14:21.086 DEBUG:teuthology.orchestra.run.smithi049:> import os 2023-11-06T02:14:21.086 DEBUG:teuthology.orchestra.run.smithi049:> import subprocess 2023-11-06T02:14:21.087 DEBUG:teuthology.orchestra.run.smithi049:> 2023-11-06T02:14:21.087 DEBUG:teuthology.orchestra.run.smithi049:> def _find_admin_socket(client_name): 2023-11-06T02:14:21.088 DEBUG:teuthology.orchestra.run.smithi049:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-06T02:14:21.088 DEBUG:teuthology.orchestra.run.smithi049:> files = glob.glob(asok_path) 2023-11-06T02:14:21.088 DEBUG:teuthology.orchestra.run.smithi049:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-06T02:14:21.089 DEBUG:teuthology.orchestra.run.smithi049:> 2023-11-06T02:14:21.089 DEBUG:teuthology.orchestra.run.smithi049:> # Given a non-glob path, it better be there 2023-11-06T02:14:21.089 DEBUG:teuthology.orchestra.run.smithi049:> if "*" not in asok_path: 2023-11-06T02:14:21.090 DEBUG:teuthology.orchestra.run.smithi049:> assert(len(files) == 1) 2023-11-06T02:14:21.090 DEBUG:teuthology.orchestra.run.smithi049:> return files[0] 2023-11-06T02:14:21.091 DEBUG:teuthology.orchestra.run.smithi049:> 2023-11-06T02:14:21.091 DEBUG:teuthology.orchestra.run.smithi049:> for f in files: 2023-11-06T02:14:21.091 DEBUG:teuthology.orchestra.run.smithi049:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-06T02:14:21.092 DEBUG:teuthology.orchestra.run.smithi049:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-06T02:14:21.092 DEBUG:teuthology.orchestra.run.smithi049:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-06T02:14:21.092 DEBUG:teuthology.orchestra.run.smithi049:> contents = proc_f.read() 2023-11-06T02:14:21.093 DEBUG:teuthology.orchestra.run.smithi049:> if mountpoint in contents: 2023-11-06T02:14:21.093 DEBUG:teuthology.orchestra.run.smithi049:> return f 2023-11-06T02:14:21.093 DEBUG:teuthology.orchestra.run.smithi049:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-06T02:14:21.094 DEBUG:teuthology.orchestra.run.smithi049:> 2023-11-06T02:14:21.094 DEBUG:teuthology.orchestra.run.smithi049:> print(_find_admin_socket("client.0")) 2023-11-06T02:14:21.095 DEBUG:teuthology.orchestra.run.smithi049:> ' 2023-11-06T02:14:21.209 INFO:teuthology.orchestra.run.smithi049.stdout:/var/run/ceph/ceph-client.0.56153.asok 2023-11-06T02:14:21.391 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.56153.asok 2023-11-06T02:14:21.392 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:14:21.392 DEBUG:teuthology.orchestra.run.smithi049:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.56153.asok status 2023-11-06T02:14:21.554 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-11-06T02:14:21.555 INFO:teuthology.orchestra.run.smithi049.stdout: "metadata": { 2023-11-06T02:14:21.555 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_sha1": "f7e9f9af51acf98e0ccc3037487018e4ae40caa5", 2023-11-06T02:14:21.555 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_version": "ceph version 18.2.0-1181-gf7e9f9af (f7e9f9af51acf98e0ccc3037487018e4ae40caa5) reef (stable)", 2023-11-06T02:14:21.555 INFO:teuthology.orchestra.run.smithi049.stdout: "entity_id": "0", 2023-11-06T02:14:21.556 INFO:teuthology.orchestra.run.smithi049.stdout: "hostname": "smithi049", 2023-11-06T02:14:21.556 INFO:teuthology.orchestra.run.smithi049.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-06T02:14:21.556 INFO:teuthology.orchestra.run.smithi049.stdout: "pid": "56153", 2023-11-06T02:14:21.556 INFO:teuthology.orchestra.run.smithi049.stdout: "root": "/" 2023-11-06T02:14:21.556 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:14:21.556 INFO:teuthology.orchestra.run.smithi049.stdout: "dentry_count": 0, 2023-11-06T02:14:21.557 INFO:teuthology.orchestra.run.smithi049.stdout: "dentry_pinned_count": 0, 2023-11-06T02:14:21.557 INFO:teuthology.orchestra.run.smithi049.stdout: "id": 7599, 2023-11-06T02:14:21.557 INFO:teuthology.orchestra.run.smithi049.stdout: "inst": { 2023-11-06T02:14:21.557 INFO:teuthology.orchestra.run.smithi049.stdout: "name": { 2023-11-06T02:14:21.557 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "client", 2023-11-06T02:14:21.558 INFO:teuthology.orchestra.run.smithi049.stdout: "num": 7599 2023-11-06T02:14:21.558 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:14:21.558 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": { 2023-11-06T02:14:21.558 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2023-11-06T02:14:21.558 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "192.168.0.1:0", 2023-11-06T02:14:21.559 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 70983300 2023-11-06T02:14:21.559 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:14:21.559 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:14:21.559 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": { 2023-11-06T02:14:21.560 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2023-11-06T02:14:21.560 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "192.168.0.1:0", 2023-11-06T02:14:21.560 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 70983300 2023-11-06T02:14:21.560 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:14:21.560 INFO:teuthology.orchestra.run.smithi049.stdout: "inst_str": "client.7599 192.168.0.1:0/70983300", 2023-11-06T02:14:21.560 INFO:teuthology.orchestra.run.smithi049.stdout: "addr_str": "192.168.0.1:0/70983300", 2023-11-06T02:14:21.561 INFO:teuthology.orchestra.run.smithi049.stdout: "inode_count": 1, 2023-11-06T02:14:21.561 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_epoch": 52, 2023-11-06T02:14:21.561 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_epoch": 81, 2023-11-06T02:14:21.561 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_epoch_barrier": 0, 2023-11-06T02:14:21.562 INFO:teuthology.orchestra.run.smithi049.stdout: "blocklisted": false, 2023-11-06T02:14:21.562 INFO:teuthology.orchestra.run.smithi049.stdout: "fs_name": "cephfs" 2023-11-06T02:14:21.562 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-11-06T02:14:21.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:14:21.570 DEBUG:teuthology.orchestra.run.smithi049:> stat --file-system '--printf=%T 2023-11-06T02:14:21.570 DEBUG:teuthology.orchestra.run.smithi049:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-06T02:14:21.577 INFO:teuthology.orchestra.run.smithi049.stdout:fuseblk 2023-11-06T02:14:21.578 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-06T02:14:21.578 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:14:21.579 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-06T02:14:21.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:14:21.636 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2023-11-06T02:14:21.700 INFO:teuthology.orchestra.run.smithi049.stdout:./ 2023-11-06T02:14:21.700 INFO:teuthology.orchestra.run.smithi049.stdout:./parent 2023-11-06T02:14:21.700 INFO:teuthology.orchestra.run.smithi049.stdout:./parent/flushed 2023-11-06T02:14:21.701 INFO:teuthology.orchestra.run.smithi049.stdout:./parent/flushed/charlie 2023-11-06T02:14:21.701 INFO:teuthology.orchestra.run.smithi049.stdout:./parent/flushed/alpha 2023-11-06T02:14:21.701 INFO:teuthology.orchestra.run.smithi049.stdout:./parent/unflushed 2023-11-06T02:14:21.701 INFO:teuthology.orchestra.run.smithi049.stdout:./parent/unflushed/jfile 2023-11-06T02:14:21.702 DEBUG:teuthology.orchestra.run.smithi049:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-06T02:14:21.702 DEBUG:teuthology.orchestra.run.smithi049:> import os 2023-11-06T02:14:21.702 DEBUG:teuthology.orchestra.run.smithi049:> import stat 2023-11-06T02:14:21.703 DEBUG:teuthology.orchestra.run.smithi049:> 2023-11-06T02:14:21.703 DEBUG:teuthology.orchestra.run.smithi049:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2023-11-06T02:14:21.704 DEBUG:teuthology.orchestra.run.smithi049:> ' 2023-11-06T02:14:21.810 INFO:teuthology.orchestra.run.smithi049.stdout:1 2023-11-06T02:14:21.998 DEBUG:teuthology.orchestra.run.smithi049:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-06T02:14:21.998 DEBUG:teuthology.orchestra.run.smithi049:> import os 2023-11-06T02:14:21.999 DEBUG:teuthology.orchestra.run.smithi049:> import stat 2023-11-06T02:14:21.999 DEBUG:teuthology.orchestra.run.smithi049:> 2023-11-06T02:14:21.999 DEBUG:teuthology.orchestra.run.smithi049:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2023-11-06T02:14:22.000 DEBUG:teuthology.orchestra.run.smithi049:> ' 2023-11-06T02:14:22.103 INFO:teuthology.orchestra.run.smithi049.stdout:1099511627776 2023-11-06T02:14:22.226 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T02:14:22.230 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T02:14:22.290 DEBUG:teuthology.orchestra.run.smithi049:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-06T02:14:22.290 DEBUG:teuthology.orchestra.run.smithi049:> import os 2023-11-06T02:14:22.291 DEBUG:teuthology.orchestra.run.smithi049:> import stat 2023-11-06T02:14:22.291 DEBUG:teuthology.orchestra.run.smithi049:> 2023-11-06T02:14:22.291 DEBUG:teuthology.orchestra.run.smithi049:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2023-11-06T02:14:22.292 DEBUG:teuthology.orchestra.run.smithi049:> ' 2023-11-06T02:14:22.398 INFO:teuthology.orchestra.run.smithi049.stdout:1099511627777 2023-11-06T02:14:22.586 DEBUG:teuthology.orchestra.run.smithi049:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-06T02:14:22.587 DEBUG:teuthology.orchestra.run.smithi049:> import os 2023-11-06T02:14:22.587 DEBUG:teuthology.orchestra.run.smithi049:> import stat 2023-11-06T02:14:22.587 DEBUG:teuthology.orchestra.run.smithi049:> 2023-11-06T02:14:22.588 DEBUG:teuthology.orchestra.run.smithi049:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2023-11-06T02:14:22.588 DEBUG:teuthology.orchestra.run.smithi049:> ' 2023-11-06T02:14:22.695 INFO:teuthology.orchestra.run.smithi049.stdout:1099511628280 2023-11-06T02:14:22.882 DEBUG:teuthology.orchestra.run.smithi049:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-06T02:14:22.882 DEBUG:teuthology.orchestra.run.smithi049:> import os 2023-11-06T02:14:22.883 DEBUG:teuthology.orchestra.run.smithi049:> import stat 2023-11-06T02:14:22.883 DEBUG:teuthology.orchestra.run.smithi049:> 2023-11-06T02:14:22.883 DEBUG:teuthology.orchestra.run.smithi049:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2023-11-06T02:14:22.884 DEBUG:teuthology.orchestra.run.smithi049:> ' 2023-11-06T02:14:22.985 INFO:teuthology.orchestra.run.smithi049.stdout:1099511627778 2023-11-06T02:14:23.173 DEBUG:teuthology.orchestra.run.smithi049:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-06T02:14:23.173 DEBUG:teuthology.orchestra.run.smithi049:> import os 2023-11-06T02:14:23.174 DEBUG:teuthology.orchestra.run.smithi049:> import stat 2023-11-06T02:14:23.174 DEBUG:teuthology.orchestra.run.smithi049:> 2023-11-06T02:14:23.174 DEBUG:teuthology.orchestra.run.smithi049:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2023-11-06T02:14:23.175 DEBUG:teuthology.orchestra.run.smithi049:> ' 2023-11-06T02:14:23.278 INFO:teuthology.orchestra.run.smithi049.stdout:1099511627779 2023-11-06T02:14:23.470 DEBUG:teuthology.orchestra.run.smithi049:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-06T02:14:23.470 DEBUG:teuthology.orchestra.run.smithi049:> import os 2023-11-06T02:14:23.471 DEBUG:teuthology.orchestra.run.smithi049:> import stat 2023-11-06T02:14:23.471 DEBUG:teuthology.orchestra.run.smithi049:> 2023-11-06T02:14:23.471 DEBUG:teuthology.orchestra.run.smithi049:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2023-11-06T02:14:23.472 DEBUG:teuthology.orchestra.run.smithi049:> ' 2023-11-06T02:14:23.578 INFO:teuthology.orchestra.run.smithi049.stdout:1099511627780 2023-11-06T02:14:23.765 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-11-06T02:14:23.765 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T02:14:23.813 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T02:14:23.988 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:23.994+0000 7fbb5685c640 1 Processor -- start 2023-11-06T02:14:23.988 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:23.994+0000 7fbb5685c640 1 -- start start 2023-11-06T02:14:23.991 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:23.994+0000 7fbb5685c640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fbb500cb010 0x7fbb500ca980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:23.991 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:23.994+0000 7fbb5685c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbb50159b40 0x7fbb500cb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:23.991 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:23.994+0000 7fbb5685c640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb500cb930 con 0x7fbb500cdd50 2023-11-06T02:14:23.992 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:23.994+0000 7fbb5685c640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb500723a0 con 0x7fbb500cb010 2023-11-06T02:14:23.992 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:23.994+0000 7fbb5685c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb500715e0 con 0x7fbb50159b40 2023-11-06T02:14:23.992 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:23.994+0000 7fbb54df9640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fbb500cdd50 0x7fbb500c8c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:54216/0 (socket says 172.21.15.49:54216) 2023-11-06T02:14:23.992 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:23.994+0000 7fbb54df9640 1 -- 172.21.15.49:0/1596618830 learned_addr learned my addr 172.21.15.49:0/1596618830 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:14:23.992 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:23.994+0000 7fbb4ffff640 1 --2- 172.21.15.49:0/1596618830 >> v2:172.21.15.150:3300/0 conn(0x7fbb500cb010 0x7fbb500ca980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:23.993 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:23.994+0000 7fbb4f7fe640 1 --2- 172.21.15.49:0/1596618830 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbb50159b40 0x7fbb500cb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:23.993 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:23.994+0000 7fbb4effd640 1 -- 172.21.15.49:0/1596618830 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4099553575 0 0) 0x7fbb500cb930 con 0x7fbb500cdd50 2023-11-06T02:14:23.993 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:23.994+0000 7fbb4effd640 1 -- 172.21.15.49:0/1596618830 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb3c005740 con 0x7fbb500cdd50 2023-11-06T02:14:23.993 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:23.994+0000 7fbb4effd640 1 -- 172.21.15.49:0/1596618830 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 976757498 0 0) 0x7fbb3c005740 con 0x7fbb500cdd50 2023-11-06T02:14:23.993 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:23.994+0000 7fbb4effd640 1 -- 172.21.15.49:0/1596618830 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbb50159b40 msgr2=0x7fbb500cb3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:23.994 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:23.994+0000 7fbb4effd640 1 --2- 172.21.15.49:0/1596618830 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbb50159b40 0x7fbb500cb3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:23.994 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:23.994+0000 7fbb4effd640 1 -- 172.21.15.49:0/1596618830 >> v2:172.21.15.150:3300/0 conn(0x7fbb500cb010 msgr2=0x7fbb500ca980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:23.994 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:23.994+0000 7fbb4effd640 1 --2- 172.21.15.49:0/1596618830 >> v2:172.21.15.150:3300/0 conn(0x7fbb500cb010 0x7fbb500ca980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:23.994 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:23.994+0000 7fbb4effd640 1 -- 172.21.15.49:0/1596618830 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb500718c0 con 0x7fbb500cdd50 2023-11-06T02:14:23.994 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:23.994+0000 7fbb4effd640 1 -- 172.21.15.49:0/1596618830 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7fbb48001a00 con 0x7fbb500cdd50 2023-11-06T02:14:23.995 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:23.994+0000 7fbb4effd640 1 -- 172.21.15.49:0/1596618830 >> v1:172.21.15.49:6789/0 conn(0x7fbb500cdd50 legacy=0x7fbb500c8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:23.995 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:23.994+0000 7fbb4effd640 1 --2- 172.21.15.49:0/1596618830 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb3c005b90 0x7fbb3c005fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:23.995 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:23.994+0000 7fbb4effd640 1 --2- 172.21.15.49:0/1596618830 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbb50159b40 0x7fbb3c0069e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:23.995 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:23.994+0000 7fbb4effd640 1 --2- 172.21.15.49:0/1596618830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb3c006f20 0x7fbb3c0073a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:23.995 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:23.994+0000 7fbb4effd640 1 -- 172.21.15.49:0/1596618830 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb500723a0 con 0x7fbb3c006f20 2023-11-06T02:14:23.995 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:23.994+0000 7fbb4effd640 1 -- 172.21.15.49:0/1596618830 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb500715e0 con 0x7fbb3c005b90 2023-11-06T02:14:23.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:23.994+0000 7fbb4effd640 1 -- 172.21.15.49:0/1596618830 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3c00dba0 con 0x7fbb50159b40 2023-11-06T02:14:23.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:23.994+0000 7fbb54df9640 1 --2- 172.21.15.49:0/1596618830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb3c006f20 0x7fbb3c0073a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:23.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:23.994+0000 7fbb4ffff640 1 --2- 172.21.15.49:0/1596618830 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb3c005b90 0x7fbb3c005fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:23.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:23.994+0000 7fbb4f7fe640 1 --2- 172.21.15.49:0/1596618830 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbb50159b40 0x7fbb3c0069e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:23.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:23.994+0000 7fbb54df9640 1 -- 172.21.15.49:0/1596618830 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbb50159b40 msgr2=0x7fbb3c0069e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:23.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:23.994+0000 7fbb54df9640 1 --2- 172.21.15.49:0/1596618830 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbb50159b40 0x7fbb3c0069e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:23.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:23.994+0000 7fbb54df9640 1 -- 172.21.15.49:0/1596618830 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb3c005b90 msgr2=0x7fbb3c005fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:23.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:23.994+0000 7fbb54df9640 1 --2- 172.21.15.49:0/1596618830 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb3c005b90 0x7fbb3c005fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:23.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:23.994+0000 7fbb54df9640 1 -- 172.21.15.49:0/1596618830 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb3c005740 con 0x7fbb3c006f20 2023-11-06T02:14:24.001 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:23.998+0000 7fbb54df9640 1 --2- 172.21.15.49:0/1596618830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb3c006f20 0x7fbb3c0073a0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fbb4800a890 tx=0x7fbb4801d360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e16371a909ec0f2e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:24.001 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:23.998+0000 7fbb4effd640 1 -- 172.21.15.49:0/1596618830 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb480ad630 con 0x7fbb3c006f20 2023-11-06T02:14:24.001 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:23.998+0000 7fbb4effd640 1 -- 172.21.15.49:0/1596618830 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb480a8070 con 0x7fbb3c006f20 2023-11-06T02:14:24.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:23.998+0000 7fbb4effd640 1 -- 172.21.15.49:0/1596618830 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb480ada10 con 0x7fbb3c006f20 2023-11-06T02:14:24.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:23.998+0000 7fbb5685c640 1 -- 172.21.15.49:0/1596618830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb3c006f20 msgr2=0x7fbb3c0073a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:24.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:23.998+0000 7fbb5685c640 1 --2- 172.21.15.49:0/1596618830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb3c006f20 0x7fbb3c0073a0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fbb4800a890 tx=0x7fbb4801d360 comp rx=0 tx=0).stop 2023-11-06T02:14:24.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:23.998+0000 7fbb54df9640 1 -- 172.21.15.49:0/1596618830 reap_dead start 2023-11-06T02:14:24.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:23.998+0000 7fbb5685c640 1 -- 172.21.15.49:0/1596618830 shutdown_connections 2023-11-06T02:14:24.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:23.998+0000 7fbb5685c640 1 -- 172.21.15.49:0/1596618830 >> 172.21.15.49:0/1596618830 conn(0x7fbb5006c0a0 msgr2=0x7fbb5006c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:24.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:23.998+0000 7fbb5685c640 1 -- 172.21.15.49:0/1596618830 shutdown_connections 2023-11-06T02:14:24.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:23.998+0000 7fbb5685c640 1 -- 172.21.15.49:0/1596618830 wait complete. 2023-11-06T02:14:24.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:23.998+0000 7fbb5685c640 1 Processor -- start 2023-11-06T02:14:24.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:23.998+0000 7fbb5685c640 1 -- start start 2023-11-06T02:14:24.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.002+0000 7fbb5685c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbb3c006f20 0x7fbb501579f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:24.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.002+0000 7fbb5685c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb501ede50 0x7fbb501ee2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:24.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.002+0000 7fbb5685c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb500c4300 0x7fbb500c4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:24.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.002+0000 7fbb5685c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb500c4cc0 con 0x7fbb500c4300 2023-11-06T02:14:24.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.002+0000 7fbb5685c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb50157f30 con 0x7fbb501ede50 2023-11-06T02:14:24.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.002+0000 7fbb5685c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb500b12f0 con 0x7fbb3c006f20 2023-11-06T02:14:24.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.002+0000 7fbb54df9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb500c4300 0x7fbb500c4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:24.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.002+0000 7fbb54df9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb500c4300 0x7fbb500c4780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54290/0 (socket says 172.21.15.49:54290) 2023-11-06T02:14:24.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.002+0000 7fbb54df9640 1 -- 172.21.15.49:0/2047832553 learned_addr learned my addr 172.21.15.49:0/2047832553 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:24.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.002+0000 7fbb4f7fe640 1 --2- 172.21.15.49:0/2047832553 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb501ede50 0x7fbb501ee2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:24.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.002+0000 7fbb4ffff640 1 --2- 172.21.15.49:0/2047832553 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbb3c006f20 0x7fbb501579f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:24.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.002+0000 7fbb54df9640 1 -- 172.21.15.49:0/2047832553 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbb3c006f20 msgr2=0x7fbb501579f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:24.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.002+0000 7fbb54df9640 1 --2- 172.21.15.49:0/2047832553 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbb3c006f20 0x7fbb501579f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:24.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.002+0000 7fbb54df9640 1 -- 172.21.15.49:0/2047832553 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb501ede50 msgr2=0x7fbb501ee2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:24.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.002+0000 7fbb54df9640 1 --2- 172.21.15.49:0/2047832553 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb501ede50 0x7fbb501ee2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:24.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.002+0000 7fbb54df9640 1 -- 172.21.15.49:0/2047832553 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb500b15d0 con 0x7fbb500c4300 2023-11-06T02:14:24.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.002+0000 7fbb4ffff640 1 --2- 172.21.15.49:0/2047832553 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbb3c006f20 0x7fbb501579f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:14:24.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.002+0000 7fbb4f7fe640 1 --2- 172.21.15.49:0/2047832553 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb501ede50 0x7fbb501ee2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:14:24.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.002+0000 7fbb54df9640 1 --2- 172.21.15.49:0/2047832553 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb500c4300 0x7fbb500c4780 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fbb4800a890 tx=0x7fbb3c007a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:24.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.002+0000 7fbb4cff9640 1 -- 172.21.15.49:0/2047832553 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb480b9070 con 0x7fbb500c4300 2023-11-06T02:14:24.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.002+0000 7fbb4cff9640 1 -- 172.21.15.49:0/2047832553 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb480019d0 con 0x7fbb500c4300 2023-11-06T02:14:24.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.002+0000 7fbb5685c640 1 -- 172.21.15.49:0/2047832553 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb500b18b0 con 0x7fbb500c4300 2023-11-06T02:14:24.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.002+0000 7fbb4cff9640 1 -- 172.21.15.49:0/2047832553 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb480011d0 con 0x7fbb500c4300 2023-11-06T02:14:24.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.002+0000 7fbb5685c640 1 -- 172.21.15.49:0/2047832553 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb500b1e70 con 0x7fbb500c4300 2023-11-06T02:14:24.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.002+0000 7fbb5685c640 1 -- 172.21.15.49:0/2047832553 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb500b2060 con 0x7fbb500c4300 2023-11-06T02:14:24.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.002+0000 7fbb4cff9640 1 -- 172.21.15.49:0/2047832553 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fbb480aa030 con 0x7fbb500c4300 2023-11-06T02:14:24.009 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.002+0000 7fbb4cff9640 1 --2- 172.21.15.49:0/2047832553 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fbb30065a60 0x7fbb30067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:24.009 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.002+0000 7fbb4cff9640 1 -- 172.21.15.49:0/2047832553 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6148+0+0 (secure 0 0 0) 0x7fbb4811cbe0 con 0x7fbb500c4300 2023-11-06T02:14:24.009 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.002+0000 7fbb4ffff640 1 --2- 172.21.15.49:0/2047832553 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fbb30065a60 0x7fbb30067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:24.009 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.006+0000 7fbb4ffff640 1 --2- 172.21.15.49:0/2047832553 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fbb30065a60 0x7fbb30067f20 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fbb500b37e0 tx=0x7fbb40008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:24.010 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.006+0000 7fbb4cff9640 1 -- 172.21.15.49:0/2047832553 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fbb480ea470 con 0x7fbb500c4300 2023-11-06T02:14:24.162 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.166+0000 7fbb5685c640 1 -- 172.21.15.49:0/2047832553 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbb50074040 con 0x7fbb500c4300 2023-11-06T02:14:24.163 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.170+0000 7fbb4cff9640 1 -- 172.21.15.49:0/2047832553 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+18015 (secure 0 0 0) 0x7fbb480b4380 con 0x7fbb500c4300 2023-11-06T02:14:24.164 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:24.164 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":81,"fsid":"1a8b035c-6ba4-499a-b876-aea47fd9c0d4","created":"2023-11-06T02:08:06.440857+0000","modified":"2023-11-06T02:14:13.284476+0000","last_up_change":"2023-11-06T02:08:17.982372+0000","last_in_change":"2023-11-06T02:08:08.906631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T02:08:19.252222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T02:08:24.776211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-11-06T02:13:31.459314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-11-06T02:13:32.603710+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ad55af8-0571-4e6a-8722-b5d429b7496d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6809","nonce":38116},{"type":"v1","addr":"172.21.15.49:6811","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6813","nonce":38116},{"type":"v1","addr":"172.21.15.49:6815","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6821","nonce":38116},{"type":"v1","addr":"172.21.15.49:6823","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6817","nonce":38116},{"type":"v1","addr":"172.21.15.49:6819","nonce":38116}]},"public_addr":"172.21.15.49:6811/38116","cluster_addr":"172.21.15.49:6815/38116","heartbeat_back_addr":"172.21.15.49:6823/38116","heartbeat_front_addr":"172.21.15.49:6819/38116","state":["exists","up"]},{"osd":1,"uuid":"c8b31f85-a757-43ed-9419-351acc46611b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":38113},{"type":"v1","addr":"172.21.15.49:6810","nonce":38113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":38113},{"type":"v1","addr":"172.21.15.49:6814","nonce":38113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":38113},{"type":"v1","addr":"172.21.15.49:6822","nonce":38113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":38113},{"type":"v1","addr":"172.21.15.49:6818","nonce":38113}]},"public_addr":"172.21.15.49:6810/38113","cluster_addr":"172.21.15.49:6814/38113","heartbeat_back_addr":"172.21.15.49:6822/38113","heartbeat_front_addr":"172.21.15.49:6818/38113","state":["exists","up"]},{"osd":2,"uuid":"555b3bf3-0a70-4b91-8efd-e4204b5fc26e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":38114},{"type":"v1","addr":"172.21.15.49:6801","nonce":38114}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":38114},{"type":"v1","addr":"172.21.15.49:6803","nonce":38114}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":38114},{"type":"v1","addr":"172.21.15.49:6807","nonce":38114}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":38114},{"type":"v1","addr":"172.21.15.49:6805","nonce":38114}]},"public_addr":"172.21.15.49:6801/38114","cluster_addr":"172.21.15.49:6803/38114","heartbeat_back_addr":"172.21.15.49:6807/38114","heartbeat_front_addr":"172.21.15.49:6805/38114","state":["exists","up"]},{"osd":3,"uuid":"8d4e5d77-03ef-4b9b-bfcb-5ae93a46795c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":38115},{"type":"v1","addr":"172.21.15.49:6825","nonce":38115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":38115},{"type":"v1","addr":"172.21.15.49:6827","nonce":38115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":38115},{"type":"v1","addr":"172.21.15.49:6831","nonce":38115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":38115},{"type":"v1","addr":"172.21.15.49:6829","nonce":38115}]},"public_addr":"172.21.15.49:6825/38115","cluster_addr":"172.21.15.49:6827/38115","heartbeat_back_addr":"172.21.15.49:6831/38115","heartbeat_front_addr":"172.21.15.49:6829/38115","state":["exists","up"]},{"osd":4,"uuid":"dba03ad4-68a3-4cf3-9218-ac611f02eacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":38417},{"type":"v1","addr":"172.21.15.150:6809","nonce":38417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":38417},{"type":"v1","addr":"172.21.15.150:6811","nonce":38417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":38417},{"type":"v1","addr":"172.21.15.150:6815","nonce":38417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":38417},{"type":"v1","addr":"172.21.15.150:6813","nonce":38417}]},"public_addr":"172.21.15.150:6809/38417","cluster_addr":"172.21.15.150:6811/38417","heartbeat_back_addr":"172.21.15.150:6815/38417","heartbeat_front_addr":"172.21.15.150:6813/38417","state":["exists","up"]},{"osd":5,"uuid":"0df9c303-c255-4640-8e91-01f1c3efe18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":38418},{"type":"v1","addr":"172.21.15.150:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":38418},{"type":"v1","addr":"172.21.15.150:6803","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":38418},{"type":"v1","addr":"172.21.15.150:6807","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":38418},{"type":"v1","addr":"172.21.15.150:6805","nonce":38418}]},"public_addr":"172.21.15.150:6801/38418","cluster_addr":"172.21.15.150:6803/38418","heartbeat_back_addr":"172.21.15.150:6807/38418","heartbeat_front_addr":"172.21.15.150:6805/38418","state":["exists","up"]},{"osd":6,"uuid":"e59fa189-4b70-4c06-9722-c02b90828bf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":38428},{"type":"v1","addr":"172.21.15.150:6825","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":38428},{"type":"v1","addr":"172.21.15.150:6827","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":38428},{"type":"v1","addr":"172.21.15.150:6831","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":38428},{"type":"v1","addr":"172.21.15.150:6829","nonce":38428}]},"public_addr":"172.21.15.150:6825/38428","cluster_addr":"172.21.15.150:6827/38428","heartbeat_back_addr":"172.21.15.150:6831/38428","heartbeat_front_addr":"172.21.15.150:6829/38428","state":["exists","up"]},{"osd":7,"uuid":"a64f57e1-ef98-46bd-b304-385759e95898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":38421},{"type":"v1","addr":"172.21.15.150:6817","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":38421},{"type":"v1","addr":"172.21.15.150:6819","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":38421},{"type":"v1","addr":"172.21.15.150:6823","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":38421},{"type":"v1","addr":"172.21.15.150:6821","nonce":38421}]},"public_addr":"172.21.15.150:6817/38421","cluster_addr":"172.21.15.150:6819/38421","heartbeat_back_addr":"172.21.15.150:6823/38421","heartbeat_front_addr":"172.21.15.150:6821/38421","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.438552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.413280+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.296466+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.424966+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.276953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.230984+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.410235+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.480084+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:6835/1130379293":"2023-11-07T02:14:13.284419+0000","172.21.15.49:6834/1130379293":"2023-11-07T02:14:13.284419+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T02:14:24.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.170+0000 7fbb5685c640 1 -- 172.21.15.49:0/2047832553 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fbb30065a60 msgr2=0x7fbb30067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:24.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.170+0000 7fbb5685c640 1 --2- 172.21.15.49:0/2047832553 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fbb30065a60 0x7fbb30067f20 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fbb500b37e0 tx=0x7fbb40008040 comp rx=0 tx=0).stop 2023-11-06T02:14:24.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.170+0000 7fbb5685c640 1 -- 172.21.15.49:0/2047832553 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb500c4300 msgr2=0x7fbb500c4780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:24.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.170+0000 7fbb5685c640 1 --2- 172.21.15.49:0/2047832553 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb500c4300 0x7fbb500c4780 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fbb4800a890 tx=0x7fbb3c007a10 comp rx=0 tx=0).stop 2023-11-06T02:14:24.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.170+0000 7fbb5685c640 1 -- 172.21.15.49:0/2047832553 shutdown_connections 2023-11-06T02:14:24.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.170+0000 7fbb5685c640 1 --2- 172.21.15.49:0/2047832553 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fbb30065a60 0x7fbb30067f20 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:24.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.170+0000 7fbb5685c640 1 --2- 172.21.15.49:0/2047832553 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb500c4300 0x7fbb500c4780 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:24.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.170+0000 7fbb5685c640 1 --2- 172.21.15.49:0/2047832553 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb501ede50 0x7fbb501ee2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:24.169 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.170+0000 7fbb5685c640 1 --2- 172.21.15.49:0/2047832553 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbb3c006f20 0x7fbb501579f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:24.169 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.170+0000 7fbb5685c640 1 -- 172.21.15.49:0/2047832553 >> 172.21.15.49:0/2047832553 conn(0x7fbb5006c0a0 msgr2=0x7fbb500ae8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:24.169 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.170+0000 7fbb5685c640 1 -- 172.21.15.49:0/2047832553 shutdown_connections 2023-11-06T02:14:24.169 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.174+0000 7fbb5685c640 1 -- 172.21.15.49:0/2047832553 wait complete. 2023-11-06T02:14:24.184 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T02:08:19.252222+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T02:08:24.776211+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T02:13:31.459314+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T02:13:32.603710+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T02:14:24.186 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T02:14:24.316 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.322+0000 7fad7e821640 1 Processor -- start 2023-11-06T02:14:24.316 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.322+0000 7fad7e821640 1 -- start start 2023-11-06T02:14:24.317 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.322+0000 7fad7e821640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fad7815a880 0x7fad780b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:24.318 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.322+0000 7fad7e821640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fad78070160 0x7fad78070610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:24.318 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.322+0000 7fad7e821640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad780ae360 con 0x7fad7806d150 2023-11-06T02:14:24.318 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.322+0000 7fad7e821640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad78070b50 con 0x7fad7815a880 2023-11-06T02:14:24.318 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.322+0000 7fad7e821640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad780ae640 con 0x7fad78070160 2023-11-06T02:14:24.318 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.322+0000 7fad77fff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fad7815a880 0x7fad780b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:24.319 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.322+0000 7fad77fff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fad7815a880 0x7fad780b15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54300/0 (socket says 172.21.15.49:54300) 2023-11-06T02:14:24.319 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.322+0000 7fad7cdbe640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fad7806d150 0x7fad7806d600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:54222/0 (socket says 172.21.15.49:54222) 2023-11-06T02:14:24.319 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.322+0000 7fad77fff640 1 -- 172.21.15.49:0/1904377326 learned_addr learned my addr 172.21.15.49:0/1904377326 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:24.319 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.322+0000 7fad77fff640 1 -- 172.21.15.49:0/1904377326 >> v2:172.21.15.150:3300/0 conn(0x7fad78070160 msgr2=0x7fad78070610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:24.320 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.322+0000 7fad77fff640 1 --2- 172.21.15.49:0/1904377326 >> v2:172.21.15.150:3300/0 conn(0x7fad78070160 0x7fad78070610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:24.320 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.322+0000 7fad77fff640 1 -- 172.21.15.49:0/1904377326 >> v1:172.21.15.49:6789/0 conn(0x7fad7806d150 legacy=0x7fad7806d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:24.320 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.322+0000 7fad77fff640 1 -- 172.21.15.49:0/1904377326 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad780ae920 con 0x7fad7815a880 2023-11-06T02:14:24.320 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.322+0000 7fad76ffd640 1 -- 172.21.15.49:0/1904377326 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 803705833 0 0) 0x7fad780ae360 con 0x7fad7806d150 2023-11-06T02:14:24.321 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.326+0000 7fad77fff640 1 --2- 172.21.15.49:0/1904377326 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad7815a880 0x7fad780b15c0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7fad7815e3a0 tx=0x7fad6801c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc1054d93827a810 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:24.322 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.326+0000 7fad76ffd640 1 -- 172.21.15.49:0/1904377326 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad680a57a0 con 0x7fad7815a880 2023-11-06T02:14:24.322 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.326+0000 7fad76ffd640 1 -- 172.21.15.49:0/1904377326 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad680a5900 con 0x7fad7815a880 2023-11-06T02:14:24.322 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.326+0000 7fad76ffd640 1 -- 172.21.15.49:0/1904377326 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad680a5c90 con 0x7fad7815a880 2023-11-06T02:14:24.322 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.326+0000 7fad7e821640 1 -- 172.21.15.49:0/1904377326 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad7815a880 msgr2=0x7fad780b15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:24.323 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.326+0000 7fad7e821640 1 --2- 172.21.15.49:0/1904377326 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad7815a880 0x7fad780b15c0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7fad7815e3a0 tx=0x7fad6801c9d0 comp rx=0 tx=0).stop 2023-11-06T02:14:24.323 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.326+0000 7fad7e821640 1 -- 172.21.15.49:0/1904377326 shutdown_connections 2023-11-06T02:14:24.323 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.326+0000 7fad7e821640 1 --2- 172.21.15.49:0/1904377326 >> v2:172.21.15.150:3300/0 conn(0x7fad78070160 0x7fad78070610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:24.323 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.326+0000 7fad7e821640 1 --2- 172.21.15.49:0/1904377326 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad7815a880 0x7fad780b15c0 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:24.324 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.326+0000 7fad7e821640 1 -- 172.21.15.49:0/1904377326 >> 172.21.15.49:0/1904377326 conn(0x7fad7806a370 msgr2=0x7fad780ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:24.324 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.326+0000 7fad7e821640 1 -- 172.21.15.49:0/1904377326 shutdown_connections 2023-11-06T02:14:24.324 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.326+0000 7fad7e821640 1 -- 172.21.15.49:0/1904377326 wait complete. 2023-11-06T02:14:24.324 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.326+0000 7fad7e821640 1 Processor -- start 2023-11-06T02:14:24.324 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.326+0000 7fad7e821640 1 -- start start 2023-11-06T02:14:24.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.326+0000 7fad7e821640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad7806d150 0x7fad780c8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:24.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.326+0000 7fad7e821640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fad78070160 0x7fad780cc120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:24.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.326+0000 7fad7e821640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad7815a880 0x7fad780ca710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:24.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.326+0000 7fad7e821640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad780ca2f0 con 0x7fad7806d150 2023-11-06T02:14:24.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.326+0000 7fad7e821640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad780cac50 con 0x7fad7815a880 2023-11-06T02:14:24.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.326+0000 7fad7e821640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad780c9650 con 0x7fad78070160 2023-11-06T02:14:24.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.326+0000 7fad77fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad7806d150 0x7fad780c8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:24.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.326+0000 7fad77fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad7806d150 0x7fad780c8fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54312/0 (socket says 172.21.15.49:54312) 2023-11-06T02:14:24.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.326+0000 7fad77fff640 1 -- 172.21.15.49:0/1463482808 learned_addr learned my addr 172.21.15.49:0/1463482808 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:24.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.326+0000 7fad77fff640 1 -- 172.21.15.49:0/1463482808 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fad78070160 msgr2=0x7fad780cc120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:24.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.326+0000 7fad77fff640 1 --2- 172.21.15.49:0/1463482808 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fad78070160 0x7fad780cc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:24.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.326+0000 7fad7cdbe640 1 --2- 172.21.15.49:0/1463482808 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad7815a880 0x7fad780ca710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:24.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.326+0000 7fad77fff640 1 -- 172.21.15.49:0/1463482808 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad7815a880 msgr2=0x7fad780ca710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:24.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.326+0000 7fad77fff640 1 --2- 172.21.15.49:0/1463482808 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad7815a880 0x7fad780ca710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:24.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.326+0000 7fad77fff640 1 -- 172.21.15.49:0/1463482808 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad68009b30 con 0x7fad7806d150 2023-11-06T02:14:24.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.326+0000 7fad77fff640 1 --2- 172.21.15.49:0/1463482808 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad7806d150 0x7fad780c8fc0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7fad6801b080 tx=0x7fad68019770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:24.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.330+0000 7fad74ff9640 1 -- 172.21.15.49:0/1463482808 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad680bd070 con 0x7fad7806d150 2023-11-06T02:14:24.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.330+0000 7fad74ff9640 1 -- 172.21.15.49:0/1463482808 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad680b8470 con 0x7fad7806d150 2023-11-06T02:14:24.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.330+0000 7fad7e821640 1 -- 172.21.15.49:0/1463482808 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad780c9950 con 0x7fad7806d150 2023-11-06T02:14:24.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.330+0000 7fad74ff9640 1 -- 172.21.15.49:0/1463482808 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad680b8800 con 0x7fad7806d150 2023-11-06T02:14:24.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.330+0000 7fad7e821640 1 -- 172.21.15.49:0/1463482808 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad780c9f10 con 0x7fad7806d150 2023-11-06T02:14:24.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.330+0000 7fad7e821640 1 -- 172.21.15.49:0/1463482808 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad78074040 con 0x7fad7806d150 2023-11-06T02:14:24.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.330+0000 7fad74ff9640 1 -- 172.21.15.49:0/1463482808 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fad680b8960 con 0x7fad7806d150 2023-11-06T02:14:24.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.330+0000 7fad74ff9640 1 --2- 172.21.15.49:0/1463482808 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fad64065a60 0x7fad64067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:24.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.330+0000 7fad74ff9640 1 -- 172.21.15.49:0/1463482808 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6148+0+0 (secure 0 0 0) 0x7fad6811e130 con 0x7fad7806d150 2023-11-06T02:14:24.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.330+0000 7fad777fe640 1 --2- 172.21.15.49:0/1463482808 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fad64065a60 0x7fad64067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:24.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.330+0000 7fad777fe640 1 --2- 172.21.15.49:0/1463482808 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fad64065a60 0x7fad64067f20 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fad7814d490 tx=0x7fad6c006ca0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:24.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.334+0000 7fad74ff9640 1 -- 172.21.15.49:0/1463482808 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fad680eb890 con 0x7fad7806d150 2023-11-06T02:14:24.487 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.490+0000 7fad7e821640 1 -- 172.21.15.49:0/1463482808 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fad780c8a60 con 0x7fad7806d150 2023-11-06T02:14:24.487 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.494+0000 7fad74ff9640 1 -- 172.21.15.49:0/1463482808 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7fad680ace60 con 0x7fad7806d150 2023-11-06T02:14:24.488 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2023-11-06T02:14:24.490 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.494+0000 7fad7e821640 1 -- 172.21.15.49:0/1463482808 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fad64065a60 msgr2=0x7fad64067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:24.490 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.494+0000 7fad7e821640 1 --2- 172.21.15.49:0/1463482808 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fad64065a60 0x7fad64067f20 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fad7814d490 tx=0x7fad6c006ca0 comp rx=0 tx=0).stop 2023-11-06T02:14:24.490 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.494+0000 7fad7e821640 1 -- 172.21.15.49:0/1463482808 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad7806d150 msgr2=0x7fad780c8fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:24.490 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.494+0000 7fad7e821640 1 --2- 172.21.15.49:0/1463482808 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad7806d150 0x7fad780c8fc0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7fad6801b080 tx=0x7fad68019770 comp rx=0 tx=0).stop 2023-11-06T02:14:24.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.494+0000 7fad7e821640 1 -- 172.21.15.49:0/1463482808 shutdown_connections 2023-11-06T02:14:24.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.494+0000 7fad7e821640 1 --2- 172.21.15.49:0/1463482808 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fad64065a60 0x7fad64067f20 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:24.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.494+0000 7fad7e821640 1 --2- 172.21.15.49:0/1463482808 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad7815a880 0x7fad780ca710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:24.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.494+0000 7fad7e821640 1 --2- 172.21.15.49:0/1463482808 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fad78070160 0x7fad780cc120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:24.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.494+0000 7fad7e821640 1 --2- 172.21.15.49:0/1463482808 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad7806d150 0x7fad780c8fc0 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:24.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.494+0000 7fad7e821640 1 -- 172.21.15.49:0/1463482808 >> 172.21.15.49:0/1463482808 conn(0x7fad7806a370 msgr2=0x7fad7814fd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:24.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.494+0000 7fad7e821640 1 -- 172.21.15.49:0/1463482808 shutdown_connections 2023-11-06T02:14:24.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.494+0000 7fad7e821640 1 -- 172.21.15.49:0/1463482808 wait complete. 2023-11-06T02:14:24.507 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T02:14:24.635 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.638+0000 7f96fc9ea640 1 Processor -- start 2023-11-06T02:14:24.635 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.638+0000 7f96fc9ea640 1 -- start start 2023-11-06T02:14:24.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.642+0000 7f96fc9ea640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f96f81595a0 0x7f96f815dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:24.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.642+0000 7f96fc9ea640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f96f806d640 0x7f96f80b0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:24.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.642+0000 7f96fc9ea640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96f806c5a0 con 0x7f96f80b0760 2023-11-06T02:14:24.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.642+0000 7f96fc9ea640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96f815e250 con 0x7f96f806d640 2023-11-06T02:14:24.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.642+0000 7f96fc9ea640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96f806c880 con 0x7f96f81595a0 2023-11-06T02:14:24.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.642+0000 7f96f659c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f96f81595a0 0x7f96f815dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:24.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.642+0000 7f96f5d9b640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f96f806d640 0x7f96f80b0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:24.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.642+0000 7f96f659c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f96f81595a0 0x7f96f815dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.49:52444/0 (socket says 172.21.15.49:52444) 2023-11-06T02:14:24.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.642+0000 7f96f659c640 1 -- 172.21.15.49:0/103880392 learned_addr learned my addr 172.21.15.49:0/103880392 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:24.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.642+0000 7f96f5d9b640 1 -- 172.21.15.49:0/103880392 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f96f81595a0 msgr2=0x7f96f815dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:24.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.642+0000 7f96f5d9b640 1 --2- 172.21.15.49:0/103880392 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f96f81595a0 0x7f96f815dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:24.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.642+0000 7f96f5d9b640 1 -- 172.21.15.49:0/103880392 >> v1:172.21.15.150:6789/0 conn(0x7f96f80b0760 legacy=0x7f96f80c8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:24.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.642+0000 7f96f5d9b640 1 -- 172.21.15.49:0/103880392 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96f806cb60 con 0x7f96f806d640 2023-11-06T02:14:24.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.642+0000 7f96f659c640 1 --2- 172.21.15.49:0/103880392 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f96f81595a0 0x7f96f815dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T02:14:24.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.646+0000 7f96f5d9b640 1 --2- 172.21.15.49:0/103880392 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f96f806d640 0x7f96f80b0220 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f96e40097c0 tx=0x7f96e401c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6edb9bc2ec318953 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:24.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.646+0000 7f96f559a640 1 -- 172.21.15.49:0/103880392 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e40ab650 con 0x7f96f806d640 2023-11-06T02:14:24.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.646+0000 7f96f559a640 1 -- 172.21.15.49:0/103880392 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96e40ab7b0 con 0x7f96f806d640 2023-11-06T02:14:24.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.646+0000 7f96f559a640 1 -- 172.21.15.49:0/103880392 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e40abb40 con 0x7f96f806d640 2023-11-06T02:14:24.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.646+0000 7f96fc9ea640 1 -- 172.21.15.49:0/103880392 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f96f806d640 msgr2=0x7f96f80b0220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:24.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.646+0000 7f96fc9ea640 1 --2- 172.21.15.49:0/103880392 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f96f806d640 0x7f96f80b0220 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f96e40097c0 tx=0x7f96e401c450 comp rx=0 tx=0).stop 2023-11-06T02:14:24.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.646+0000 7f96fc9ea640 1 -- 172.21.15.49:0/103880392 shutdown_connections 2023-11-06T02:14:24.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.646+0000 7f96fc9ea640 1 --2- 172.21.15.49:0/103880392 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f96f806d640 0x7f96f80b0220 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:24.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.646+0000 7f96fc9ea640 1 --2- 172.21.15.49:0/103880392 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f96f81595a0 0x7f96f815dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:24.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.646+0000 7f96fc9ea640 1 -- 172.21.15.49:0/103880392 >> 172.21.15.49:0/103880392 conn(0x7f96f806a370 msgr2=0x7f96f80ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:24.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.646+0000 7f96fc9ea640 1 -- 172.21.15.49:0/103880392 shutdown_connections 2023-11-06T02:14:24.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.646+0000 7f96fc9ea640 1 -- 172.21.15.49:0/103880392 wait complete. 2023-11-06T02:14:24.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.646+0000 7f96fc9ea640 1 Processor -- start 2023-11-06T02:14:24.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.646+0000 7f96fc9ea640 1 -- start start 2023-11-06T02:14:24.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.646+0000 7f96fc9ea640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f96f806d640 0x7f96f80cca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:24.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.646+0000 7f96fc9ea640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f96f80b0760 0x7f96f814c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:24.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.646+0000 7f96fc9ea640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96f81595a0 0x7f96f814fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:24.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.646+0000 7f96fc9ea640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96f80cde60 con 0x7f96f81595a0 2023-11-06T02:14:24.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.646+0000 7f96fc9ea640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96f80cd250 con 0x7f96f806d640 2023-11-06T02:14:24.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.646+0000 7f96fc9ea640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96f80cd530 con 0x7f96f80b0760 2023-11-06T02:14:24.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.650+0000 7f96f6d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96f81595a0 0x7f96f814fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:24.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.650+0000 7f96f6d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96f81595a0 0x7f96f814fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54326/0 (socket says 172.21.15.49:54326) 2023-11-06T02:14:24.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.650+0000 7f96f659c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f96f806d640 0x7f96f80cca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:24.648 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.650+0000 7f96f5d9b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f96f80b0760 0x7f96f814c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:24.648 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.650+0000 7f96f6d9d640 1 -- 172.21.15.49:0/171285759 learned_addr learned my addr 172.21.15.49:0/171285759 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:24.648 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.650+0000 7f96f6d9d640 1 -- 172.21.15.49:0/171285759 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f96f80b0760 msgr2=0x7f96f814c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:24.648 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.650+0000 7f96f6d9d640 1 --2- 172.21.15.49:0/171285759 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f96f80b0760 0x7f96f814c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:24.648 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.650+0000 7f96f6d9d640 1 -- 172.21.15.49:0/171285759 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f96f806d640 msgr2=0x7f96f80cca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:24.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.650+0000 7f96f6d9d640 1 --2- 172.21.15.49:0/171285759 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f96f806d640 0x7f96f80cca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:24.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.650+0000 7f96f6d9d640 1 -- 172.21.15.49:0/171285759 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96e80094d0 con 0x7f96f81595a0 2023-11-06T02:14:24.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.650+0000 7f96f6d9d640 1 --2- 172.21.15.49:0/171285759 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96f81595a0 0x7f96f814fff0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f96f000c770 tx=0x7f96f000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:24.650 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.650+0000 7f96eeffd640 1 -- 172.21.15.49:0/171285759 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f0003b80 con 0x7f96f81595a0 2023-11-06T02:14:24.650 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.650+0000 7f96eeffd640 1 -- 172.21.15.49:0/171285759 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96f0003ce0 con 0x7f96f81595a0 2023-11-06T02:14:24.650 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.650+0000 7f96fc9ea640 1 -- 172.21.15.49:0/171285759 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96e4009420 con 0x7f96f81595a0 2023-11-06T02:14:24.655 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.650+0000 7f96eeffd640 1 -- 172.21.15.49:0/171285759 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f000aa90 con 0x7f96f81595a0 2023-11-06T02:14:24.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.650+0000 7f96fc9ea640 1 -- 172.21.15.49:0/171285759 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96f80cdaf0 con 0x7f96f81595a0 2023-11-06T02:14:24.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.650+0000 7f96fc9ea640 1 -- 172.21.15.49:0/171285759 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96f8074040 con 0x7f96f81595a0 2023-11-06T02:14:24.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.650+0000 7f96eeffd640 1 -- 172.21.15.49:0/171285759 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f96f000abf0 con 0x7f96f81595a0 2023-11-06T02:14:24.657 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.650+0000 7f96eeffd640 1 --2- 172.21.15.49:0/171285759 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f96e0065a60 0x7f96e0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:24.657 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.650+0000 7f96eeffd640 1 -- 172.21.15.49:0/171285759 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f96f0083030 con 0x7f96f81595a0 2023-11-06T02:14:24.657 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.650+0000 7f96f659c640 1 --2- 172.21.15.49:0/171285759 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f96e0065a60 0x7f96e0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:24.658 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.654+0000 7f96f659c640 1 --2- 172.21.15.49:0/171285759 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f96e0065a60 0x7f96e0067f20 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f96e8006cb0 tx=0x7f96e8009240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:24.658 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.654+0000 7f96eeffd640 1 -- 172.21.15.49:0/171285759 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f96f00508e0 con 0x7f96f81595a0 2023-11-06T02:14:24.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.810+0000 7f96fc9ea640 1 -- 172.21.15.49:0/171285759 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f96f80cf150 con 0x7f96f81595a0 2023-11-06T02:14:24.807 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.810+0000 7f96eeffd640 1 -- 172.21.15.49:0/171285759 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v81) v1 ==== 92+0+10 (secure 0 0 0) 0x7f96f0053f00 con 0x7f96f81595a0 2023-11-06T02:14:24.807 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 8 2023-11-06T02:14:24.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.814+0000 7f96fc9ea640 1 -- 172.21.15.49:0/171285759 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f96e0065a60 msgr2=0x7f96e0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:24.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.814+0000 7f96fc9ea640 1 --2- 172.21.15.49:0/171285759 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f96e0065a60 0x7f96e0067f20 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f96e8006cb0 tx=0x7f96e8009240 comp rx=0 tx=0).stop 2023-11-06T02:14:24.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.814+0000 7f96fc9ea640 1 -- 172.21.15.49:0/171285759 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96f81595a0 msgr2=0x7f96f814fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:24.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.814+0000 7f96fc9ea640 1 --2- 172.21.15.49:0/171285759 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96f81595a0 0x7f96f814fff0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f96f000c770 tx=0x7f96f000cc30 comp rx=0 tx=0).stop 2023-11-06T02:14:24.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.814+0000 7f96fc9ea640 1 -- 172.21.15.49:0/171285759 shutdown_connections 2023-11-06T02:14:24.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.814+0000 7f96fc9ea640 1 --2- 172.21.15.49:0/171285759 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f96e0065a60 0x7f96e0067f20 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:24.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.814+0000 7f96fc9ea640 1 --2- 172.21.15.49:0/171285759 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96f81595a0 0x7f96f814fff0 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:24.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.814+0000 7f96fc9ea640 1 --2- 172.21.15.49:0/171285759 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f96f80b0760 0x7f96f814c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:24.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.814+0000 7f96fc9ea640 1 --2- 172.21.15.49:0/171285759 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f96f806d640 0x7f96f80cca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:24.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.814+0000 7f96fc9ea640 1 -- 172.21.15.49:0/171285759 >> 172.21.15.49:0/171285759 conn(0x7f96f806a370 msgr2=0x7f96f80c7940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:24.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.814+0000 7f96fc9ea640 1 -- 172.21.15.49:0/171285759 shutdown_connections 2023-11-06T02:14:24.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.814+0000 7f96fc9ea640 1 -- 172.21.15.49:0/171285759 wait complete. 2023-11-06T02:14:24.826 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T02:14:24.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.962+0000 7f1195962640 1 Processor -- start 2023-11-06T02:14:24.957 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.962+0000 7f1195962640 1 -- start start 2023-11-06T02:14:24.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.962+0000 7f1195962640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f119015a990 0x7f119015dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:24.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.962+0000 7f1195962640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f11900b0220 0x7f11900b0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:24.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.962+0000 7f1195962640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f119006c5a0 con 0x7f11901595a0 2023-11-06T02:14:24.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.962+0000 7f1195962640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f119015e250 con 0x7f11900b0220 2023-11-06T02:14:24.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.962+0000 7f1195962640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f119006c880 con 0x7f119015a990 2023-11-06T02:14:24.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.962+0000 7f118f7fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f11900b0220 0x7f11900b0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:24.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.962+0000 7f118f7fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f11900b0220 0x7f11900b0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54342/0 (socket says 172.21.15.49:54342) 2023-11-06T02:14:24.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.962+0000 7f118f7fe640 1 -- 172.21.15.49:0/560312303 learned_addr learned my addr 172.21.15.49:0/560312303 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:24.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.962+0000 7f118e7fc640 1 --2- 172.21.15.49:0/560312303 >> v2:172.21.15.150:3300/0 conn(0x7f119015a990 0x7f119015dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:24.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.962+0000 7f118f7fe640 1 -- 172.21.15.49:0/560312303 >> v2:172.21.15.150:3300/0 conn(0x7f119015a990 msgr2=0x7f119015dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:24.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.962+0000 7f118f7fe640 1 --2- 172.21.15.49:0/560312303 >> v2:172.21.15.150:3300/0 conn(0x7f119015a990 0x7f119015dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:24.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.962+0000 7f118f7fe640 1 -- 172.21.15.49:0/560312303 >> v1:172.21.15.150:6789/0 conn(0x7f11901595a0 legacy=0x7f119006d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:24.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.962+0000 7f118f7fe640 1 -- 172.21.15.49:0/560312303 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f119006cb60 con 0x7f11900b0220 2023-11-06T02:14:24.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.966+0000 7f118f7fe640 1 --2- 172.21.15.49:0/560312303 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11900b0220 0x7f11900b0650 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f119006ad80 tx=0x7f118801c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d5c64cf244c66c9 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:24.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.966+0000 7f118dffb640 1 -- 172.21.15.49:0/560312303 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11880a5820 con 0x7f11900b0220 2023-11-06T02:14:24.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.966+0000 7f118dffb640 1 -- 172.21.15.49:0/560312303 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11880a5980 con 0x7f11900b0220 2023-11-06T02:14:24.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.966+0000 7f118dffb640 1 -- 172.21.15.49:0/560312303 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11880a5d10 con 0x7f11900b0220 2023-11-06T02:14:24.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.966+0000 7f1195962640 1 -- 172.21.15.49:0/560312303 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11900b0220 msgr2=0x7f11900b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:24.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.966+0000 7f1195962640 1 --2- 172.21.15.49:0/560312303 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11900b0220 0x7f11900b0650 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f119006ad80 tx=0x7f118801c960 comp rx=0 tx=0).stop 2023-11-06T02:14:24.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.966+0000 7f1195962640 1 -- 172.21.15.49:0/560312303 shutdown_connections 2023-11-06T02:14:24.964 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.966+0000 7f1195962640 1 --2- 172.21.15.49:0/560312303 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11900b0220 0x7f11900b0650 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:24.964 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.966+0000 7f1195962640 1 --2- 172.21.15.49:0/560312303 >> v2:172.21.15.150:3300/0 conn(0x7f119015a990 0x7f119015dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:24.964 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.966+0000 7f1195962640 1 -- 172.21.15.49:0/560312303 >> 172.21.15.49:0/560312303 conn(0x7f119006a370 msgr2=0x7f11900ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:24.964 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.966+0000 7f1195962640 1 -- 172.21.15.49:0/560312303 shutdown_connections 2023-11-06T02:14:24.964 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.966+0000 7f1195962640 1 -- 172.21.15.49:0/560312303 wait complete. 2023-11-06T02:14:24.965 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.966+0000 7f1195962640 1 Processor -- start 2023-11-06T02:14:24.965 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.966+0000 7f1195962640 1 -- start start 2023-11-06T02:14:24.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.966+0000 7f1195962640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11900b0220 0x7f119015ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:24.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.966+0000 7f1195962640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f11901595a0 0x7f11901564e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:24.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.966+0000 7f1195962640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f119015a990 0x7f1190166930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:24.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.966+0000 7f1195962640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f119015b2e0 con 0x7f11900b0220 2023-11-06T02:14:24.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.966+0000 7f1195962640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f119014ac80 con 0x7f119015a990 2023-11-06T02:14:24.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.966+0000 7f1195962640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11901676d0 con 0x7f11901595a0 2023-11-06T02:14:24.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.966+0000 7f118effd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11900b0220 0x7f119015ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:24.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.966+0000 7f118effd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11900b0220 0x7f119015ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54354/0 (socket says 172.21.15.49:54354) 2023-11-06T02:14:24.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.966+0000 7f118effd640 1 -- 172.21.15.49:0/2066856726 learned_addr learned my addr 172.21.15.49:0/2066856726 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:24.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.970+0000 7f118e7fc640 1 --2- 172.21.15.49:0/2066856726 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f11901595a0 0x7f11901564e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:24.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.970+0000 7f118f7fe640 1 --2- 172.21.15.49:0/2066856726 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f119015a990 0x7f1190166930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:24.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.970+0000 7f118effd640 1 -- 172.21.15.49:0/2066856726 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f11901595a0 msgr2=0x7f11901564e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:24.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.970+0000 7f118effd640 1 --2- 172.21.15.49:0/2066856726 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f11901595a0 0x7f11901564e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:24.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.970+0000 7f118effd640 1 -- 172.21.15.49:0/2066856726 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f119015a990 msgr2=0x7f1190166930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:24.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.970+0000 7f118effd640 1 --2- 172.21.15.49:0/2066856726 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f119015a990 0x7f1190166930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:24.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.970+0000 7f118effd640 1 -- 172.21.15.49:0/2066856726 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1188009b30 con 0x7f11900b0220 2023-11-06T02:14:24.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.970+0000 7f118e7fc640 1 --2- 172.21.15.49:0/2066856726 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f11901595a0 0x7f11901564e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:14:24.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.970+0000 7f118f7fe640 1 --2- 172.21.15.49:0/2066856726 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f119015a990 0x7f1190166930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:14:24.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.970+0000 7f118effd640 1 --2- 172.21.15.49:0/2066856726 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11900b0220 0x7f119015ca60 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f118000c450 tx=0x7f118000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:24.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.970+0000 7f1177fff640 1 -- 172.21.15.49:0/2066856726 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1180012830 con 0x7f11900b0220 2023-11-06T02:14:24.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.970+0000 7f1177fff640 1 -- 172.21.15.49:0/2066856726 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11800129b0 con 0x7f11900b0220 2023-11-06T02:14:24.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.970+0000 7f1195962640 1 -- 172.21.15.49:0/2066856726 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1190166e70 con 0x7f11900b0220 2023-11-06T02:14:24.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.970+0000 7f1177fff640 1 -- 172.21.15.49:0/2066856726 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1180012d60 con 0x7f11900b0220 2023-11-06T02:14:24.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.970+0000 7f1195962640 1 -- 172.21.15.49:0/2066856726 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11901673f0 con 0x7f11900b0220 2023-11-06T02:14:24.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.970+0000 7f1195962640 1 -- 172.21.15.49:0/2066856726 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f119006c210 con 0x7f11900b0220 2023-11-06T02:14:24.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.970+0000 7f1177fff640 1 -- 172.21.15.49:0/2066856726 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f118001c460 con 0x7f11900b0220 2023-11-06T02:14:24.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.970+0000 7f1177fff640 1 --2- 172.21.15.49:0/2066856726 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f117c065a60 0x7f117c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:24.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.970+0000 7f1177fff640 1 -- 172.21.15.49:0/2066856726 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f11800830c0 con 0x7f11900b0220 2023-11-06T02:14:24.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.970+0000 7f118e7fc640 1 --2- 172.21.15.49:0/2066856726 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f117c065a60 0x7f117c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:24.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.970+0000 7f118e7fc640 1 --2- 172.21.15.49:0/2066856726 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f117c065a60 0x7f117c067f20 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f11900c8500 tx=0x7f1184008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:24.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:24.974+0000 7f1177fff640 1 -- 172.21.15.49:0/2066856726 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f119006c210 con 0x7f11900b0220 2023-11-06T02:14:25.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.130+0000 7f1195962640 1 -- 172.21.15.49:0/2066856726 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f119006c210 con 0x7f11900b0220 2023-11-06T02:14:25.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.130+0000 7f1177fff640 1 -- 172.21.15.49:0/2066856726 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v81) v1 ==== 104+0+11 (secure 0 0 0) 0x7f119006c210 con 0x7f11900b0220 2023-11-06T02:14:25.126 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2023-11-06T02:14:25.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.134+0000 7f1195962640 1 -- 172.21.15.49:0/2066856726 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f117c065a60 msgr2=0x7f117c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:25.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.134+0000 7f1195962640 1 --2- 172.21.15.49:0/2066856726 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f117c065a60 0x7f117c067f20 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f11900c8500 tx=0x7f1184008040 comp rx=0 tx=0).stop 2023-11-06T02:14:25.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.134+0000 7f1195962640 1 -- 172.21.15.49:0/2066856726 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11900b0220 msgr2=0x7f119015ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:25.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.134+0000 7f1195962640 1 --2- 172.21.15.49:0/2066856726 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11900b0220 0x7f119015ca60 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f118000c450 tx=0x7f118000c910 comp rx=0 tx=0).stop 2023-11-06T02:14:25.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.134+0000 7f1195962640 1 -- 172.21.15.49:0/2066856726 shutdown_connections 2023-11-06T02:14:25.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.134+0000 7f1195962640 1 --2- 172.21.15.49:0/2066856726 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f117c065a60 0x7f117c067f20 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:25.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.134+0000 7f1195962640 1 --2- 172.21.15.49:0/2066856726 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f119015a990 0x7f1190166930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:25.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.134+0000 7f1195962640 1 --2- 172.21.15.49:0/2066856726 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f11901595a0 0x7f11901564e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:25.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.134+0000 7f1195962640 1 --2- 172.21.15.49:0/2066856726 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11900b0220 0x7f119015ca60 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:25.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.134+0000 7f1195962640 1 -- 172.21.15.49:0/2066856726 >> 172.21.15.49:0/2066856726 conn(0x7f119006a370 msgr2=0x7f11900cae50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:25.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.134+0000 7f1195962640 1 -- 172.21.15.49:0/2066856726 shutdown_connections 2023-11-06T02:14:25.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.134+0000 7f1195962640 1 -- 172.21.15.49:0/2066856726 wait complete. 2023-11-06T02:14:25.145 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T02:14:25.267 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.270+0000 7f1aa9d80640 1 Processor -- start 2023-11-06T02:14:25.267 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.270+0000 7f1aa9d80640 1 -- start start 2023-11-06T02:14:25.268 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.274+0000 7f1aa9d80640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f1aa415a880 0x7f1aa40b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:25.268 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.274+0000 7f1aa9d80640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1aa4070160 0x7f1aa4070610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:25.268 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.274+0000 7f1aa9d80640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1aa40ae360 con 0x7f1aa406d150 2023-11-06T02:14:25.269 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.274+0000 7f1aa9d80640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1aa4070b50 con 0x7f1aa415a880 2023-11-06T02:14:25.269 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.274+0000 7f1aa9d80640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa40ae640 con 0x7f1aa4070160 2023-11-06T02:14:25.269 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.274+0000 7f1aa3fff640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f1aa406d150 0x7f1aa406d5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:54238/0 (socket says 172.21.15.49:54238) 2023-11-06T02:14:25.269 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.274+0000 7f1aa3fff640 1 -- 172.21.15.49:0/1557978505 learned_addr learned my addr 172.21.15.49:0/1557978505 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:14:25.271 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.274+0000 7f1a9affd640 1 --2- 172.21.15.49:0/1557978505 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1aa4070160 0x7f1aa4070610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:25.272 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.274+0000 7f1aa37fe640 1 --2- 172.21.15.49:0/1557978505 >> v2:172.21.15.150:3300/0 conn(0x7f1aa415a880 0x7f1aa40b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:25.272 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.274+0000 7f1aa2ffd640 1 -- 172.21.15.49:0/1557978505 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3782992085 0 0) 0x7f1aa40ae360 con 0x7f1aa406d150 2023-11-06T02:14:25.272 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.274+0000 7f1aa2ffd640 1 -- 172.21.15.49:0/1557978505 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a90003530 con 0x7f1aa406d150 2023-11-06T02:14:25.272 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.274+0000 7f1aa2ffd640 1 -- 172.21.15.49:0/1557978505 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1247218235 0 0) 0x7f1a90003530 con 0x7f1aa406d150 2023-11-06T02:14:25.273 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.274+0000 7f1aa2ffd640 1 -- 172.21.15.49:0/1557978505 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1aa4070160 msgr2=0x7f1aa4070610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:25.273 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.274+0000 7f1aa2ffd640 1 --2- 172.21.15.49:0/1557978505 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1aa4070160 0x7f1aa4070610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:25.273 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.274+0000 7f1aa2ffd640 1 -- 172.21.15.49:0/1557978505 >> v2:172.21.15.150:3300/0 conn(0x7f1aa415a880 msgr2=0x7f1aa40b15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:25.273 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.274+0000 7f1aa2ffd640 1 --2- 172.21.15.49:0/1557978505 >> v2:172.21.15.150:3300/0 conn(0x7f1aa415a880 0x7f1aa40b15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:25.273 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.274+0000 7f1aa2ffd640 1 -- 172.21.15.49:0/1557978505 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aa40ae920 con 0x7f1aa406d150 2023-11-06T02:14:25.274 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.274+0000 7f1aa2ffd640 1 -- 172.21.15.49:0/1557978505 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7f1a9c001a00 con 0x7f1aa406d150 2023-11-06T02:14:25.274 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.274+0000 7f1aa2ffd640 1 -- 172.21.15.49:0/1557978505 >> v1:172.21.15.49:6789/0 conn(0x7f1aa406d150 legacy=0x7f1aa406d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:25.274 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.274+0000 7f1aa2ffd640 1 --2- 172.21.15.49:0/1557978505 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1a90003980 0x7f1a90003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:25.274 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.274+0000 7f1aa2ffd640 1 --2- 172.21.15.49:0/1557978505 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1aa4070160 0x7f1a900047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:25.275 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.274+0000 7f1aa2ffd640 1 --2- 172.21.15.49:0/1557978505 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1a90004d10 0x7f1a90005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:25.275 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.274+0000 7f1aa2ffd640 1 -- 172.21.15.49:0/1557978505 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa4070b50 con 0x7f1a90003980 2023-11-06T02:14:25.275 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.274+0000 7f1aa2ffd640 1 -- 172.21.15.49:0/1557978505 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa40ae640 con 0x7f1a90004d10 2023-11-06T02:14:25.276 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.274+0000 7f1aa2ffd640 1 -- 172.21.15.49:0/1557978505 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a9000b990 con 0x7f1aa4070160 2023-11-06T02:14:25.276 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.274+0000 7f1aa37fe640 1 --2- 172.21.15.49:0/1557978505 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1a90003980 0x7f1a90003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:25.276 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.274+0000 7f1aa37fe640 1 -- 172.21.15.49:0/1557978505 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1aa4070160 msgr2=0x7f1a900047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:25.277 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.274+0000 7f1a9affd640 1 --2- 172.21.15.49:0/1557978505 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1aa4070160 0x7f1a900047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:25.277 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.274+0000 7f1aa3fff640 1 --2- 172.21.15.49:0/1557978505 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1a90004d10 0x7f1a90005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:25.277 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.274+0000 7f1aa37fe640 1 --2- 172.21.15.49:0/1557978505 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1aa4070160 0x7f1a900047d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:25.277 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.274+0000 7f1aa37fe640 1 -- 172.21.15.49:0/1557978505 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1a90004d10 msgr2=0x7f1a90005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:25.278 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.274+0000 7f1aa37fe640 1 --2- 172.21.15.49:0/1557978505 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1a90004d10 0x7f1a90005190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:25.278 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.274+0000 7f1aa37fe640 1 -- 172.21.15.49:0/1557978505 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a90003530 con 0x7f1a90003980 2023-11-06T02:14:25.278 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.278+0000 7f1aa37fe640 1 --2- 172.21.15.49:0/1557978505 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1a90003980 0x7f1a90003da0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f1a94009b10 tx=0x7f1a9401ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8344ce61a781dee0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:25.278 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.278+0000 7f1aa2ffd640 1 -- 172.21.15.49:0/1557978505 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a940a5540 con 0x7f1a90003980 2023-11-06T02:14:25.278 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.278+0000 7f1aa2ffd640 1 -- 172.21.15.49:0/1557978505 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a940a56a0 con 0x7f1a90003980 2023-11-06T02:14:25.279 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.278+0000 7f1aa2ffd640 1 -- 172.21.15.49:0/1557978505 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a940a5a50 con 0x7f1a90003980 2023-11-06T02:14:25.282 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.278+0000 7f1aa9d80640 1 -- 172.21.15.49:0/1557978505 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1a90003980 msgr2=0x7f1a90003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:25.282 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.278+0000 7f1aa9d80640 1 --2- 172.21.15.49:0/1557978505 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1a90003980 0x7f1a90003da0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f1a94009b10 tx=0x7f1a9401ca90 comp rx=0 tx=0).stop 2023-11-06T02:14:25.282 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.278+0000 7f1aa3fff640 1 -- 172.21.15.49:0/1557978505 reap_dead start 2023-11-06T02:14:25.282 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.278+0000 7f1aa9d80640 1 -- 172.21.15.49:0/1557978505 shutdown_connections 2023-11-06T02:14:25.283 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.278+0000 7f1aa9d80640 1 -- 172.21.15.49:0/1557978505 >> 172.21.15.49:0/1557978505 conn(0x7f1aa406a370 msgr2=0x7f1aa40ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:25.283 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.278+0000 7f1aa9d80640 1 -- 172.21.15.49:0/1557978505 shutdown_connections 2023-11-06T02:14:25.283 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.278+0000 7f1aa9d80640 1 -- 172.21.15.49:0/1557978505 wait complete. 2023-11-06T02:14:25.283 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.278+0000 7f1aa9d80640 1 Processor -- start 2023-11-06T02:14:25.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.278+0000 7f1aa9d80640 1 -- start start 2023-11-06T02:14:25.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.278+0000 7f1aa9d80640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1a90003980 0x7f1aa4144d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:25.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.278+0000 7f1aa9d80640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1aa4145490 0x7f1aa415ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:25.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.278+0000 7f1aa9d80640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1aa4145910 0x7f1aa415d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:25.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.278+0000 7f1aa9d80640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa415bec0 con 0x7f1aa4145490 2023-11-06T02:14:25.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.278+0000 7f1aa9d80640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa415c110 con 0x7f1aa4145910 2023-11-06T02:14:25.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.278+0000 7f1aa9d80640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa415c3f0 con 0x7f1a90003980 2023-11-06T02:14:25.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.278+0000 7f1a9affd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1aa4145490 0x7f1aa415ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:25.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.278+0000 7f1a9affd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1aa4145490 0x7f1aa415ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54370/0 (socket says 172.21.15.49:54370) 2023-11-06T02:14:25.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.278+0000 7f1a9affd640 1 -- 172.21.15.49:0/4100365235 learned_addr learned my addr 172.21.15.49:0/4100365235 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:25.286 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.278+0000 7f1aa37fe640 1 --2- 172.21.15.49:0/4100365235 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1a90003980 0x7f1aa4144d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:25.286 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.278+0000 7f1aa3fff640 1 --2- 172.21.15.49:0/4100365235 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1aa4145910 0x7f1aa415d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:25.286 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.278+0000 7f1a9affd640 1 -- 172.21.15.49:0/4100365235 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1a90003980 msgr2=0x7f1aa4144d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:25.286 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.278+0000 7f1a9affd640 1 --2- 172.21.15.49:0/4100365235 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1a90003980 0x7f1aa4144d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:25.286 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.278+0000 7f1a9affd640 1 -- 172.21.15.49:0/4100365235 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1aa4145910 msgr2=0x7f1aa415d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:25.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.282+0000 7f1a9affd640 1 --2- 172.21.15.49:0/4100365235 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1aa4145910 0x7f1aa415d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:25.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.282+0000 7f1a9affd640 1 -- 172.21.15.49:0/4100365235 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aa415c6d0 con 0x7f1aa4145490 2023-11-06T02:14:25.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.282+0000 7f1a9affd640 1 --2- 172.21.15.49:0/4100365235 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1aa4145490 0x7f1aa415ced0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c010890 tx=0x7f1a8c010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:25.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.282+0000 7f1aa0ff9640 1 -- 172.21.15.49:0/4100365235 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a8c009630 con 0x7f1aa4145490 2023-11-06T02:14:25.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.282+0000 7f1aa0ff9640 1 -- 172.21.15.49:0/4100365235 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a8c00eb60 con 0x7f1aa4145490 2023-11-06T02:14:25.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.282+0000 7f1aa9d80640 1 -- 172.21.15.49:0/4100365235 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1aa415c9b0 con 0x7f1aa4145490 2023-11-06T02:14:25.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.282+0000 7f1aa0ff9640 1 -- 172.21.15.49:0/4100365235 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a8c01c400 con 0x7f1aa4145490 2023-11-06T02:14:25.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.282+0000 7f1aa9d80640 1 -- 172.21.15.49:0/4100365235 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1aa4156a10 con 0x7f1aa4145490 2023-11-06T02:14:25.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.282+0000 7f1aa9d80640 1 -- 172.21.15.49:0/4100365235 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1aa4074040 con 0x7f1aa4145490 2023-11-06T02:14:25.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.282+0000 7f1aa0ff9640 1 -- 172.21.15.49:0/4100365235 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f1a8c01c560 con 0x7f1aa4145490 2023-11-06T02:14:25.289 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.282+0000 7f1aa0ff9640 1 --2- 172.21.15.49:0/4100365235 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1a84065a90 0x7f1a84067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:25.289 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.282+0000 7f1aa0ff9640 1 -- 172.21.15.49:0/4100365235 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f1a8c083950 con 0x7f1aa4145490 2023-11-06T02:14:25.289 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.282+0000 7f1aa37fe640 1 --2- 172.21.15.49:0/4100365235 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1a84065a90 0x7f1a84067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:25.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.282+0000 7f1aa37fe640 1 --2- 172.21.15.49:0/4100365235 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1a84065a90 0x7f1a84067f50 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f1a940af000 tx=0x7f1a940046d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:25.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.286+0000 7f1aa0ff9640 1 -- 172.21.15.49:0/4100365235 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f1a8c051370 con 0x7f1aa4145490 2023-11-06T02:14:25.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.442+0000 7f1aa9d80640 1 -- 172.21.15.49:0/4100365235 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1aa406c430 con 0x7f1aa4145490 2023-11-06T02:14:25.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.442+0000 7f1aa0ff9640 1 -- 172.21.15.49:0/4100365235 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v81) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1a8c054990 con 0x7f1aa4145490 2023-11-06T02:14:25.438 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2023-11-06T02:14:25.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.446+0000 7f1aa9d80640 1 -- 172.21.15.49:0/4100365235 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1a84065a90 msgr2=0x7f1a84067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:25.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.446+0000 7f1aa9d80640 1 --2- 172.21.15.49:0/4100365235 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1a84065a90 0x7f1a84067f50 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f1a940af000 tx=0x7f1a940046d0 comp rx=0 tx=0).stop 2023-11-06T02:14:25.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.446+0000 7f1aa9d80640 1 -- 172.21.15.49:0/4100365235 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1aa4145490 msgr2=0x7f1aa415ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:25.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.446+0000 7f1aa9d80640 1 --2- 172.21.15.49:0/4100365235 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1aa4145490 0x7f1aa415ced0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c010890 tx=0x7f1a8c010d50 comp rx=0 tx=0).stop 2023-11-06T02:14:25.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.446+0000 7f1aa9d80640 1 -- 172.21.15.49:0/4100365235 shutdown_connections 2023-11-06T02:14:25.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.446+0000 7f1aa9d80640 1 --2- 172.21.15.49:0/4100365235 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1a84065a90 0x7f1a84067f50 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:25.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.446+0000 7f1aa9d80640 1 --2- 172.21.15.49:0/4100365235 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1aa4145490 0x7f1aa415ced0 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:25.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.446+0000 7f1aa9d80640 1 --2- 172.21.15.49:0/4100365235 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1aa4145910 0x7f1aa415d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:25.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.446+0000 7f1aa9d80640 1 --2- 172.21.15.49:0/4100365235 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1a90003980 0x7f1aa4144d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:25.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.446+0000 7f1aa9d80640 1 -- 172.21.15.49:0/4100365235 >> 172.21.15.49:0/4100365235 conn(0x7f1aa406a370 msgr2=0x7f1aa40cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:25.443 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.446+0000 7f1aa9d80640 1 -- 172.21.15.49:0/4100365235 shutdown_connections 2023-11-06T02:14:25.443 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.446+0000 7f1aa9d80640 1 -- 172.21.15.49:0/4100365235 wait complete. 2023-11-06T02:14:25.458 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-06T02:14:25.585 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.590+0000 7fd0e0f93640 1 Processor -- start 2023-11-06T02:14:25.586 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.590+0000 7fd0e0f93640 1 -- start start 2023-11-06T02:14:25.588 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.590+0000 7fd0e0f93640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fd0dc0cfc10 0x7fd0dc15e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:25.588 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.590+0000 7fd0e0f93640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd0dc0ce1b0 0x7fd0dc0d2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:25.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.590+0000 7fd0e0f93640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0dc15e5d0 con 0x7fd0dc0cf1e0 2023-11-06T02:14:25.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.590+0000 7fd0e0f93640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0dc161bd0 con 0x7fd0dc0cfc10 2023-11-06T02:14:25.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.590+0000 7fd0e0f93640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0dc0cf6c0 con 0x7fd0dc0ce1b0 2023-11-06T02:14:25.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.590+0000 7fd0d9d9b640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fd0dc0cf1e0 0x7fd0dc0cdaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:54254/0 (socket says 172.21.15.49:54254) 2023-11-06T02:14:25.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.590+0000 7fd0d9d9b640 1 -- 172.21.15.49:0/703430033 learned_addr learned my addr 172.21.15.49:0/703430033 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:14:25.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.590+0000 7fd0d959a640 1 -- 172.21.15.49:0/703430033 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1725445202 0 0) 0x7fd0dc15e5d0 con 0x7fd0dc0cf1e0 2023-11-06T02:14:25.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.590+0000 7fd0d959a640 1 -- 172.21.15.49:0/703430033 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0c4003530 con 0x7fd0dc0cf1e0 2023-11-06T02:14:25.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.590+0000 7fd0da59c640 1 --2- 172.21.15.49:0/703430033 >> v2:172.21.15.150:3300/0 conn(0x7fd0dc0cfc10 0x7fd0dc15e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:25.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.590+0000 7fd0dad9d640 1 --2- 172.21.15.49:0/703430033 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd0dc0ce1b0 0x7fd0dc0d2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:25.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.590+0000 7fd0d959a640 1 -- 172.21.15.49:0/703430033 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2456848456 0 0) 0x7fd0c4003530 con 0x7fd0dc0cf1e0 2023-11-06T02:14:25.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.594+0000 7fd0d959a640 1 -- 172.21.15.49:0/703430033 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd0dc0ce1b0 msgr2=0x7fd0dc0d2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:25.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.594+0000 7fd0d959a640 1 --2- 172.21.15.49:0/703430033 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd0dc0ce1b0 0x7fd0dc0d2890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:25.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.594+0000 7fd0d959a640 1 -- 172.21.15.49:0/703430033 >> v2:172.21.15.150:3300/0 conn(0x7fd0dc0cfc10 msgr2=0x7fd0dc15e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:25.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.594+0000 7fd0d959a640 1 --2- 172.21.15.49:0/703430033 >> v2:172.21.15.150:3300/0 conn(0x7fd0dc0cfc10 0x7fd0dc15e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:25.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.594+0000 7fd0d959a640 1 -- 172.21.15.49:0/703430033 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0dc161050 con 0x7fd0dc0cf1e0 2023-11-06T02:14:25.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.594+0000 7fd0d959a640 1 -- 172.21.15.49:0/703430033 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7fd0d0001680 con 0x7fd0dc0cf1e0 2023-11-06T02:14:25.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.594+0000 7fd0d959a640 1 -- 172.21.15.49:0/703430033 >> v1:172.21.15.49:6789/0 conn(0x7fd0dc0cf1e0 legacy=0x7fd0dc0cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:25.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.594+0000 7fd0d959a640 1 --2- 172.21.15.49:0/703430033 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0c4003980 0x7fd0c4003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:25.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.594+0000 7fd0d959a640 1 --2- 172.21.15.49:0/703430033 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0c40047d0 0x7fd0c4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:25.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.594+0000 7fd0d959a640 1 --2- 172.21.15.49:0/703430033 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd0dc0ce1b0 0x7fd0c4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:25.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.594+0000 7fd0d959a640 1 -- 172.21.15.49:0/703430033 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0dc161bd0 con 0x7fd0c4003980 2023-11-06T02:14:25.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.594+0000 7fd0d959a640 1 -- 172.21.15.49:0/703430033 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0dc0cf6c0 con 0x7fd0c40047d0 2023-11-06T02:14:25.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.594+0000 7fd0d959a640 1 -- 172.21.15.49:0/703430033 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c400bb40 con 0x7fd0dc0ce1b0 2023-11-06T02:14:25.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.594+0000 7fd0da59c640 1 --2- 172.21.15.49:0/703430033 >> v2:172.21.15.150:3300/0 conn(0x7fd0dc0cfc10 0x7fd0dc15e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:14:25.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.594+0000 7fd0da59c640 1 --2- 172.21.15.49:0/703430033 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0c4003980 0x7fd0c4003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:25.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.594+0000 7fd0d9d9b640 1 --2- 172.21.15.49:0/703430033 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0c40047d0 0x7fd0c4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:25.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.594+0000 7fd0dad9d640 1 --2- 172.21.15.49:0/703430033 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd0dc0ce1b0 0x7fd0c4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:25.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.594+0000 7fd0da59c640 1 -- 172.21.15.49:0/703430033 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd0dc0ce1b0 msgr2=0x7fd0c4005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:25.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.594+0000 7fd0da59c640 1 --2- 172.21.15.49:0/703430033 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd0dc0ce1b0 0x7fd0c4005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:25.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.594+0000 7fd0da59c640 1 -- 172.21.15.49:0/703430033 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0c40047d0 msgr2=0x7fd0c4004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:25.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.594+0000 7fd0da59c640 1 --2- 172.21.15.49:0/703430033 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0c40047d0 0x7fd0c4004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:25.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.594+0000 7fd0da59c640 1 -- 172.21.15.49:0/703430033 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0c4003530 con 0x7fd0c4003980 2023-11-06T02:14:25.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.594+0000 7fd0da59c640 1 --2- 172.21.15.49:0/703430033 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0c4003980 0x7fd0c4003da0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc0098b0 tx=0x7fd0cc01c5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32a1ae3ddee6cf28 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:25.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.594+0000 7fd0d959a640 1 -- 172.21.15.49:0/703430033 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0cc004220 con 0x7fd0c4003980 2023-11-06T02:14:25.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.594+0000 7fd0d959a640 1 -- 172.21.15.49:0/703430033 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0cc004380 con 0x7fd0c4003980 2023-11-06T02:14:25.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.594+0000 7fd0d959a640 1 -- 172.21.15.49:0/703430033 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0cc003840 con 0x7fd0c4003980 2023-11-06T02:14:25.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.594+0000 7fd0e0f93640 1 -- 172.21.15.49:0/703430033 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0c4003980 msgr2=0x7fd0c4003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:25.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.594+0000 7fd0e0f93640 1 --2- 172.21.15.49:0/703430033 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0c4003980 0x7fd0c4003da0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc0098b0 tx=0x7fd0cc01c5e0 comp rx=0 tx=0).stop 2023-11-06T02:14:25.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.594+0000 7fd0dad9d640 1 -- 172.21.15.49:0/703430033 reap_dead start 2023-11-06T02:14:25.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.594+0000 7fd0e0f93640 1 -- 172.21.15.49:0/703430033 shutdown_connections 2023-11-06T02:14:25.600 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.594+0000 7fd0e0f93640 1 -- 172.21.15.49:0/703430033 >> 172.21.15.49:0/703430033 conn(0x7fd0dc0714a0 msgr2=0x7fd0dc0718b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:25.600 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.594+0000 7fd0e0f93640 1 -- 172.21.15.49:0/703430033 shutdown_connections 2023-11-06T02:14:25.600 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.594+0000 7fd0e0f93640 1 -- 172.21.15.49:0/703430033 wait complete. 2023-11-06T02:14:25.600 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.598+0000 7fd0e0f93640 1 Processor -- start 2023-11-06T02:14:25.600 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.598+0000 7fd0e0f93640 1 -- start start 2023-11-06T02:14:25.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.598+0000 7fd0e0f93640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0c4003980 0x7fd0dc151770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:25.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.598+0000 7fd0e0f93640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd0dc155130 0x7fd0dc155590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:25.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.598+0000 7fd0e0f93640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0dc0c7600 0x7fd0dc0c7a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:25.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.598+0000 7fd0e0f93640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0dc152240 con 0x7fd0dc0c7600 2023-11-06T02:14:25.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.598+0000 7fd0e0f93640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0dc155ad0 con 0x7fd0c4003980 2023-11-06T02:14:25.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.598+0000 7fd0e0f93640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0dc0b6180 con 0x7fd0dc155130 2023-11-06T02:14:25.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.598+0000 7fd0dad9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0dc0c7600 0x7fd0dc0c7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:25.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.598+0000 7fd0dad9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0dc0c7600 0x7fd0dc0c7a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54392/0 (socket says 172.21.15.49:54392) 2023-11-06T02:14:25.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.598+0000 7fd0dad9d640 1 -- 172.21.15.49:0/4136439968 learned_addr learned my addr 172.21.15.49:0/4136439968 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:25.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.598+0000 7fd0da59c640 1 --2- 172.21.15.49:0/4136439968 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0c4003980 0x7fd0dc151770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:25.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.598+0000 7fd0d9d9b640 1 --2- 172.21.15.49:0/4136439968 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd0dc155130 0x7fd0dc155590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:25.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.598+0000 7fd0dad9d640 1 -- 172.21.15.49:0/4136439968 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd0dc155130 msgr2=0x7fd0dc155590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:25.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.598+0000 7fd0dad9d640 1 --2- 172.21.15.49:0/4136439968 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd0dc155130 0x7fd0dc155590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:25.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.598+0000 7fd0dad9d640 1 -- 172.21.15.49:0/4136439968 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0c4003980 msgr2=0x7fd0dc151770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:25.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.598+0000 7fd0dad9d640 1 --2- 172.21.15.49:0/4136439968 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0c4003980 0x7fd0dc151770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:25.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.598+0000 7fd0dad9d640 1 -- 172.21.15.49:0/4136439968 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0dc0b6d00 con 0x7fd0dc0c7600 2023-11-06T02:14:25.604 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.598+0000 7fd0da59c640 1 --2- 172.21.15.49:0/4136439968 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0c4003980 0x7fd0dc151770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:14:25.604 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.598+0000 7fd0d9d9b640 1 --2- 172.21.15.49:0/4136439968 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd0dc155130 0x7fd0dc155590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:14:25.604 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.598+0000 7fd0dad9d640 1 --2- 172.21.15.49:0/4136439968 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0dc0c7600 0x7fd0dc0c7a80 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fd0c4004750 tx=0x7fd0d4009d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:25.604 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.598+0000 7fd0caffd640 1 -- 172.21.15.49:0/4136439968 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d4007c10 con 0x7fd0dc0c7600 2023-11-06T02:14:25.604 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.598+0000 7fd0caffd640 1 -- 172.21.15.49:0/4136439968 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0d400f050 con 0x7fd0dc0c7600 2023-11-06T02:14:25.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.598+0000 7fd0e0f93640 1 -- 172.21.15.49:0/4136439968 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0dc0c7fc0 con 0x7fd0dc0c7600 2023-11-06T02:14:25.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.598+0000 7fd0caffd640 1 -- 172.21.15.49:0/4136439968 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d4014410 con 0x7fd0dc0c7600 2023-11-06T02:14:25.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.598+0000 7fd0e0f93640 1 -- 172.21.15.49:0/4136439968 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0dc0b66b0 con 0x7fd0dc0c7600 2023-11-06T02:14:25.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.598+0000 7fd0e0f93640 1 -- 172.21.15.49:0/4136439968 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0dc0b6e90 con 0x7fd0dc0c7600 2023-11-06T02:14:25.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.602+0000 7fd0caffd640 1 -- 172.21.15.49:0/4136439968 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fd0d4014570 con 0x7fd0dc0c7600 2023-11-06T02:14:25.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.602+0000 7fd0caffd640 1 --2- 172.21.15.49:0/4136439968 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd0bc065a60 0x7fd0bc067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:25.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.602+0000 7fd0caffd640 1 -- 172.21.15.49:0/4136439968 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6148+0+0 (secure 0 0 0) 0x7fd0d40837c0 con 0x7fd0dc0c7600 2023-11-06T02:14:25.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.602+0000 7fd0da59c640 1 --2- 172.21.15.49:0/4136439968 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd0bc065a60 0x7fd0bc067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:25.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.602+0000 7fd0da59c640 1 --2- 172.21.15.49:0/4136439968 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd0bc065a60 0x7fd0bc067f20 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc0ae000 tx=0x7fd0cc0046d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:25.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.606+0000 7fd0caffd640 1 -- 172.21.15.49:0/4136439968 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fd0d4050f20 con 0x7fd0dc0c7600 2023-11-06T02:14:25.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.774+0000 7fd0e0f93640 1 -- 172.21.15.49:0/4136439968 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd0dc067140 con 0x7fd0dc0c7600 2023-11-06T02:14:25.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.774+0000 7fd0caffd640 1 -- 172.21.15.49:0/4136439968 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v81) v1 ==== 80+0+122 (secure 0 0 0) 0x7fd0d4054540 con 0x7fd0dc0c7600 2023-11-06T02:14:25.769 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:6835/1130379293 2023-11-07T02:14:13.284419+0000 2023-11-06T02:14:25.769 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:6834/1130379293 2023-11-07T02:14:13.284419+0000 2023-11-06T02:14:25.770 INFO:teuthology.orchestra.run.smithi049.stderr:listed 2 entries 2023-11-06T02:14:25.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.774+0000 7fd0e0f93640 1 -- 172.21.15.49:0/4136439968 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd0bc065a60 msgr2=0x7fd0bc067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:25.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.774+0000 7fd0e0f93640 1 --2- 172.21.15.49:0/4136439968 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd0bc065a60 0x7fd0bc067f20 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc0ae000 tx=0x7fd0cc0046d0 comp rx=0 tx=0).stop 2023-11-06T02:14:25.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.774+0000 7fd0e0f93640 1 -- 172.21.15.49:0/4136439968 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0dc0c7600 msgr2=0x7fd0dc0c7a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:25.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.774+0000 7fd0e0f93640 1 --2- 172.21.15.49:0/4136439968 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0dc0c7600 0x7fd0dc0c7a80 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fd0c4004750 tx=0x7fd0d4009d40 comp rx=0 tx=0).stop 2023-11-06T02:14:25.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.774+0000 7fd0e0f93640 1 -- 172.21.15.49:0/4136439968 shutdown_connections 2023-11-06T02:14:25.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.774+0000 7fd0e0f93640 1 --2- 172.21.15.49:0/4136439968 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd0bc065a60 0x7fd0bc067f20 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:25.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.774+0000 7fd0e0f93640 1 --2- 172.21.15.49:0/4136439968 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0dc0c7600 0x7fd0dc0c7a80 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:25.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.774+0000 7fd0e0f93640 1 --2- 172.21.15.49:0/4136439968 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd0dc155130 0x7fd0dc155590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:25.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.774+0000 7fd0e0f93640 1 --2- 172.21.15.49:0/4136439968 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0c4003980 0x7fd0dc151770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:25.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.778+0000 7fd0e0f93640 1 -- 172.21.15.49:0/4136439968 >> 172.21.15.49:0/4136439968 conn(0x7fd0dc0714a0 msgr2=0x7fd0dc0cd520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:25.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.778+0000 7fd0e0f93640 1 -- 172.21.15.49:0/4136439968 shutdown_connections 2023-11-06T02:14:25.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.778+0000 7fd0e0f93640 1 -- 172.21.15.49:0/4136439968 wait complete. 2023-11-06T02:14:25.789 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-11-06T02:14:25.790 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T02:14:25.795 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T02:14:25.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.966+0000 7f2156beb640 1 Processor -- start 2023-11-06T02:14:25.964 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.970+0000 7f2156beb640 1 -- start start 2023-11-06T02:14:25.965 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.970+0000 7f2156beb640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f21500b1f90 0x7f21500cda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:25.965 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.970+0000 7f2156beb640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21500d17c0 con 0x7f2150161650 2023-11-06T02:14:25.965 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.970+0000 7f2156beb640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2150161b00 con 0x7f21500714a0 2023-11-06T02:14:25.966 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.970+0000 7f2156beb640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21500ce6c0 con 0x7f21500b1f90 2023-11-06T02:14:25.966 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.970+0000 7f2154987640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f21500b1f90 0x7f21500cda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:25.966 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.970+0000 7f2154987640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f21500b1f90 0x7f21500cda70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54396/0 (socket says 172.21.15.49:54396) 2023-11-06T02:14:25.966 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.970+0000 7f214ffff640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f2150161650 0x7f21500d1c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:54268/0 (socket says 172.21.15.49:54268) 2023-11-06T02:14:25.966 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.970+0000 7f2154987640 1 -- 172.21.15.49:0/883237272 learned_addr learned my addr 172.21.15.49:0/883237272 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:25.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.970+0000 7f2154987640 1 -- 172.21.15.49:0/883237272 >> v1:172.21.15.150:6789/0 conn(0x7f21500714a0 legacy=0x7f21500718b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:25.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.970+0000 7f2154987640 1 -- 172.21.15.49:0/883237272 >> v1:172.21.15.49:6789/0 conn(0x7f2150161650 legacy=0x7f21500d1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:25.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.970+0000 7f2154987640 1 -- 172.21.15.49:0/883237272 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21500d2360 con 0x7f21500b1f90 2023-11-06T02:14:25.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.970+0000 7f214f7fe640 1 -- 172.21.15.49:0/883237272 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3969277952 0 0) 0x7f21500d17c0 con 0x7f2150161650 2023-11-06T02:14:25.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.970+0000 7f2154987640 1 --2- 172.21.15.49:0/883237272 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f21500b1f90 0x7f21500cda70 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f2140009870 tx=0x7f214001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e2a9e440750f970 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:25.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.974+0000 7f214f7fe640 1 -- 172.21.15.49:0/883237272 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21400aa430 con 0x7f21500b1f90 2023-11-06T02:14:25.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.974+0000 7f214f7fe640 1 -- 172.21.15.49:0/883237272 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21400aa590 con 0x7f21500b1f90 2023-11-06T02:14:25.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.974+0000 7f214f7fe640 1 -- 172.21.15.49:0/883237272 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21400aa920 con 0x7f21500b1f90 2023-11-06T02:14:25.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.974+0000 7f2156beb640 1 -- 172.21.15.49:0/883237272 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f21500b1f90 msgr2=0x7f21500cda70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:25.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.974+0000 7f2156beb640 1 --2- 172.21.15.49:0/883237272 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f21500b1f90 0x7f21500cda70 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f2140009870 tx=0x7f214001c4d0 comp rx=0 tx=0).stop 2023-11-06T02:14:25.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.974+0000 7f2156beb640 1 -- 172.21.15.49:0/883237272 shutdown_connections 2023-11-06T02:14:25.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.974+0000 7f2156beb640 1 --2- 172.21.15.49:0/883237272 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f21500b1f90 0x7f21500cda70 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:25.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.974+0000 7f2156beb640 1 -- 172.21.15.49:0/883237272 >> 172.21.15.49:0/883237272 conn(0x7f21500d2aa0 msgr2=0x7f21500d2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:25.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.974+0000 7f2156beb640 1 -- 172.21.15.49:0/883237272 shutdown_connections 2023-11-06T02:14:25.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.974+0000 7f2156beb640 1 -- 172.21.15.49:0/883237272 wait complete. 2023-11-06T02:14:25.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.974+0000 7f2156beb640 1 Processor -- start 2023-11-06T02:14:25.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.974+0000 7f2156beb640 1 -- start start 2023-11-06T02:14:25.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.974+0000 7f2156beb640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f21500714a0 0x7f21500b4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:25.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.974+0000 7f2156beb640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f21500b1f90 0x7f215015dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:25.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.974+0000 7f2156beb640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2150161650 0x7f215015e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:25.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.974+0000 7f2156beb640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f215014f8c0 con 0x7f21500b1f90 2023-11-06T02:14:25.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.974+0000 7f2156beb640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f215014f590 con 0x7f2150161650 2023-11-06T02:14:25.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.974+0000 7f2156beb640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21500c9c80 con 0x7f21500714a0 2023-11-06T02:14:25.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.974+0000 7f214ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f21500b1f90 0x7f215015dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:25.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.974+0000 7f214ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f21500b1f90 0x7f215015dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54400/0 (socket says 172.21.15.49:54400) 2023-11-06T02:14:25.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.974+0000 7f214ffff640 1 -- 172.21.15.49:0/2518061289 learned_addr learned my addr 172.21.15.49:0/2518061289 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:25.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.974+0000 7f2154987640 1 --2- 172.21.15.49:0/2518061289 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f21500714a0 0x7f21500b4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:25.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.974+0000 7f2155188640 1 --2- 172.21.15.49:0/2518061289 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2150161650 0x7f215015e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:25.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.974+0000 7f214ffff640 1 -- 172.21.15.49:0/2518061289 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f21500714a0 msgr2=0x7f21500b4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:25.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.974+0000 7f214ffff640 1 --2- 172.21.15.49:0/2518061289 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f21500714a0 0x7f21500b4c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:25.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.974+0000 7f214ffff640 1 -- 172.21.15.49:0/2518061289 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2150161650 msgr2=0x7f215015e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:25.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.974+0000 7f214ffff640 1 --2- 172.21.15.49:0/2518061289 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2150161650 0x7f215015e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:25.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.974+0000 7f214ffff640 1 -- 172.21.15.49:0/2518061289 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21400094d0 con 0x7f21500b1f90 2023-11-06T02:14:25.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.974+0000 7f214ffff640 1 --2- 172.21.15.49:0/2518061289 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f21500b1f90 0x7f215015dae0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f214400c480 tx=0x7f214400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:25.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.974+0000 7f214d7fa640 1 -- 172.21.15.49:0/2518061289 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21440074e0 con 0x7f21500b1f90 2023-11-06T02:14:25.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.974+0000 7f214d7fa640 1 -- 172.21.15.49:0/2518061289 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2144007660 con 0x7f21500b1f90 2023-11-06T02:14:25.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.974+0000 7f2156beb640 1 -- 172.21.15.49:0/2518061289 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21500c9f60 con 0x7f21500b1f90 2023-11-06T02:14:25.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.974+0000 7f214d7fa640 1 -- 172.21.15.49:0/2518061289 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2144004a30 con 0x7f21500b1f90 2023-11-06T02:14:25.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.974+0000 7f2156beb640 1 -- 172.21.15.49:0/2518061289 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21500ca240 con 0x7f21500b1f90 2023-11-06T02:14:25.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.978+0000 7f2156beb640 1 -- 172.21.15.49:0/2518061289 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2150074040 con 0x7f21500b1f90 2023-11-06T02:14:25.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.978+0000 7f214d7fa640 1 -- 172.21.15.49:0/2518061289 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f2144004b90 con 0x7f21500b1f90 2023-11-06T02:14:25.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.978+0000 7f214d7fa640 1 --2- 172.21.15.49:0/2518061289 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f213c065a90 0x7f213c067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:25.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.978+0000 7f214d7fa640 1 -- 172.21.15.49:0/2518061289 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f2144082880 con 0x7f21500b1f90 2023-11-06T02:14:25.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.978+0000 7f2154987640 1 --2- 172.21.15.49:0/2518061289 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f213c065a90 0x7f213c067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:25.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.978+0000 7f2154987640 1 --2- 172.21.15.49:0/2518061289 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f213c065a90 0x7f213c067f50 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f21400ae000 tx=0x7f21400a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:25.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:25.982+0000 7f214d7fa640 1 -- 172.21.15.49:0/2518061289 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f2144053b00 con 0x7f21500b1f90 2023-11-06T02:14:26.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.138+0000 7f2156beb640 1 -- 172.21.15.49:0/2518061289 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f21500d0c30 con 0x7f21500b1f90 2023-11-06T02:14:26.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.142+0000 7f214d7fa640 1 -- 172.21.15.49:0/2518061289 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+18015 (secure 0 0 0) 0x7f2144053690 con 0x7f21500b1f90 2023-11-06T02:14:26.138 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:26.138 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":81,"fsid":"1a8b035c-6ba4-499a-b876-aea47fd9c0d4","created":"2023-11-06T02:08:06.440857+0000","modified":"2023-11-06T02:14:13.284476+0000","last_up_change":"2023-11-06T02:08:17.982372+0000","last_in_change":"2023-11-06T02:08:08.906631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T02:08:19.252222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T02:08:24.776211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-11-06T02:13:31.459314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-11-06T02:13:32.603710+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ad55af8-0571-4e6a-8722-b5d429b7496d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6809","nonce":38116},{"type":"v1","addr":"172.21.15.49:6811","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6813","nonce":38116},{"type":"v1","addr":"172.21.15.49:6815","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6821","nonce":38116},{"type":"v1","addr":"172.21.15.49:6823","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6817","nonce":38116},{"type":"v1","addr":"172.21.15.49:6819","nonce":38116}]},"public_addr":"172.21.15.49:6811/38116","cluster_addr":"172.21.15.49:6815/38116","heartbeat_back_addr":"172.21.15.49:6823/38116","heartbeat_front_addr":"172.21.15.49:6819/38116","state":["exists","up"]},{"osd":1,"uuid":"c8b31f85-a757-43ed-9419-351acc46611b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":38113},{"type":"v1","addr":"172.21.15.49:6810","nonce":38113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":38113},{"type":"v1","addr":"172.21.15.49:6814","nonce":38113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":38113},{"type":"v1","addr":"172.21.15.49:6822","nonce":38113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":38113},{"type":"v1","addr":"172.21.15.49:6818","nonce":38113}]},"public_addr":"172.21.15.49:6810/38113","cluster_addr":"172.21.15.49:6814/38113","heartbeat_back_addr":"172.21.15.49:6822/38113","heartbeat_front_addr":"172.21.15.49:6818/38113","state":["exists","up"]},{"osd":2,"uuid":"555b3bf3-0a70-4b91-8efd-e4204b5fc26e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":38114},{"type":"v1","addr":"172.21.15.49:6801","nonce":38114}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":38114},{"type":"v1","addr":"172.21.15.49:6803","nonce":38114}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":38114},{"type":"v1","addr":"172.21.15.49:6807","nonce":38114}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":38114},{"type":"v1","addr":"172.21.15.49:6805","nonce":38114}]},"public_addr":"172.21.15.49:6801/38114","cluster_addr":"172.21.15.49:6803/38114","heartbeat_back_addr":"172.21.15.49:6807/38114","heartbeat_front_addr":"172.21.15.49:6805/38114","state":["exists","up"]},{"osd":3,"uuid":"8d4e5d77-03ef-4b9b-bfcb-5ae93a46795c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":38115},{"type":"v1","addr":"172.21.15.49:6825","nonce":38115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":38115},{"type":"v1","addr":"172.21.15.49:6827","nonce":38115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":38115},{"type":"v1","addr":"172.21.15.49:6831","nonce":38115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":38115},{"type":"v1","addr":"172.21.15.49:6829","nonce":38115}]},"public_addr":"172.21.15.49:6825/38115","cluster_addr":"172.21.15.49:6827/38115","heartbeat_back_addr":"172.21.15.49:6831/38115","heartbeat_front_addr":"172.21.15.49:6829/38115","state":["exists","up"]},{"osd":4,"uuid":"dba03ad4-68a3-4cf3-9218-ac611f02eacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":38417},{"type":"v1","addr":"172.21.15.150:6809","nonce":38417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":38417},{"type":"v1","addr":"172.21.15.150:6811","nonce":38417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":38417},{"type":"v1","addr":"172.21.15.150:6815","nonce":38417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":38417},{"type":"v1","addr":"172.21.15.150:6813","nonce":38417}]},"public_addr":"172.21.15.150:6809/38417","cluster_addr":"172.21.15.150:6811/38417","heartbeat_back_addr":"172.21.15.150:6815/38417","heartbeat_front_addr":"172.21.15.150:6813/38417","state":["exists","up"]},{"osd":5,"uuid":"0df9c303-c255-4640-8e91-01f1c3efe18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":38418},{"type":"v1","addr":"172.21.15.150:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":38418},{"type":"v1","addr":"172.21.15.150:6803","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":38418},{"type":"v1","addr":"172.21.15.150:6807","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":38418},{"type":"v1","addr":"172.21.15.150:6805","nonce":38418}]},"public_addr":"172.21.15.150:6801/38418","cluster_addr":"172.21.15.150:6803/38418","heartbeat_back_addr":"172.21.15.150:6807/38418","heartbeat_front_addr":"172.21.15.150:6805/38418","state":["exists","up"]},{"osd":6,"uuid":"e59fa189-4b70-4c06-9722-c02b90828bf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":38428},{"type":"v1","addr":"172.21.15.150:6825","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":38428},{"type":"v1","addr":"172.21.15.150:6827","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":38428},{"type":"v1","addr":"172.21.15.150:6831","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":38428},{"type":"v1","addr":"172.21.15.150:6829","nonce":38428}]},"public_addr":"172.21.15.150:6825/38428","cluster_addr":"172.21.15.150:6827/38428","heartbeat_back_addr":"172.21.15.150:6831/38428","heartbeat_front_addr":"172.21.15.150:6829/38428","state":["exists","up"]},{"osd":7,"uuid":"a64f57e1-ef98-46bd-b304-385759e95898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":38421},{"type":"v1","addr":"172.21.15.150:6817","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":38421},{"type":"v1","addr":"172.21.15.150:6819","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":38421},{"type":"v1","addr":"172.21.15.150:6823","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":38421},{"type":"v1","addr":"172.21.15.150:6821","nonce":38421}]},"public_addr":"172.21.15.150:6817/38421","cluster_addr":"172.21.15.150:6819/38421","heartbeat_back_addr":"172.21.15.150:6823/38421","heartbeat_front_addr":"172.21.15.150:6821/38421","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.438552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.413280+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.296466+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.424966+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.276953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.230984+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.410235+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.480084+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:6835/1130379293":"2023-11-07T02:14:13.284419+0000","172.21.15.49:6834/1130379293":"2023-11-07T02:14:13.284419+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T02:14:26.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.142+0000 7f2156beb640 1 -- 172.21.15.49:0/2518061289 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f213c065a90 msgr2=0x7f213c067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:26.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.142+0000 7f2156beb640 1 --2- 172.21.15.49:0/2518061289 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f213c065a90 0x7f213c067f50 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f21400ae000 tx=0x7f21400a7000 comp rx=0 tx=0).stop 2023-11-06T02:14:26.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.142+0000 7f2156beb640 1 -- 172.21.15.49:0/2518061289 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f21500b1f90 msgr2=0x7f215015dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:26.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.142+0000 7f2156beb640 1 --2- 172.21.15.49:0/2518061289 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f21500b1f90 0x7f215015dae0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f214400c480 tx=0x7f214400c940 comp rx=0 tx=0).stop 2023-11-06T02:14:26.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.142+0000 7f2156beb640 1 -- 172.21.15.49:0/2518061289 shutdown_connections 2023-11-06T02:14:26.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.142+0000 7f2156beb640 1 --2- 172.21.15.49:0/2518061289 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f213c065a90 0x7f213c067f50 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:26.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.142+0000 7f2156beb640 1 --2- 172.21.15.49:0/2518061289 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f21500b1f90 0x7f215015dae0 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:26.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.142+0000 7f2156beb640 1 --2- 172.21.15.49:0/2518061289 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2150161650 0x7f215015e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:26.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.142+0000 7f2156beb640 1 --2- 172.21.15.49:0/2518061289 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f21500714a0 0x7f21500b4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:26.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.142+0000 7f2156beb640 1 -- 172.21.15.49:0/2518061289 >> 172.21.15.49:0/2518061289 conn(0x7f21500d2aa0 msgr2=0x7f21500c6dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:26.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.142+0000 7f2156beb640 1 -- 172.21.15.49:0/2518061289 shutdown_connections 2023-11-06T02:14:26.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.142+0000 7f2156beb640 1 -- 172.21.15.49:0/2518061289 wait complete. 2023-11-06T02:14:26.156 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T02:08:19.252222+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T02:08:24.776211+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T02:13:31.459314+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T02:13:32.603710+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T02:14:26.157 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T02:14:26.280 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.286+0000 7f98cfae0640 1 Processor -- start 2023-11-06T02:14:26.280 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.286+0000 7f98cfae0640 1 -- start start 2023-11-06T02:14:26.281 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.286+0000 7f98cfae0640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f98c815e0c0 0x7f98c80ce840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:26.281 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.286+0000 7f98cfae0640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f98c815e4a0 0x7f98c80cdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:26.281 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.286+0000 7f98cfae0640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98c80ce230 con 0x7f98c80b16e0 2023-11-06T02:14:26.281 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.286+0000 7f98cfae0640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98c80cd8d0 con 0x7f98c815e4a0 2023-11-06T02:14:26.281 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.286+0000 7f98cfae0640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98c80d1430 con 0x7f98c815e0c0 2023-11-06T02:14:26.282 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.286+0000 7f98cd07b640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f98c815e4a0 0x7f98c80cdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:26.283 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.286+0000 7f98ce07d640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f98c80b16e0 0x7f98c80b0100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:54284/0 (socket says 172.21.15.49:54284) 2023-11-06T02:14:26.283 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.286+0000 7f98cd07b640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f98c815e4a0 0x7f98c80cdcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54408/0 (socket says 172.21.15.49:54408) 2023-11-06T02:14:26.283 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.286+0000 7f98ce07d640 1 -- 172.21.15.49:0/194861475 learned_addr learned my addr 172.21.15.49:0/194861475 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:14:26.283 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.286+0000 7f98cd87c640 1 --2- 172.21.15.49:0/194861475 >> v2:172.21.15.150:3300/0 conn(0x7f98c815e0c0 0x7f98c80ce840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:26.283 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.286+0000 7f98cd07b640 1 -- 172.21.15.49:0/194861475 >> v2:172.21.15.150:3300/0 conn(0x7f98c815e0c0 msgr2=0x7f98c80ce840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:26.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.286+0000 7f98cd07b640 1 --2- 172.21.15.49:0/194861475 >> v2:172.21.15.150:3300/0 conn(0x7f98c815e0c0 0x7f98c80ce840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:26.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.286+0000 7f98cd07b640 1 -- 172.21.15.49:0/194861475 >> v1:172.21.15.49:6789/0 conn(0x7f98c80b16e0 legacy=0x7f98c80b0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:26.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.286+0000 7f98cd07b640 1 -- 172.21.15.49:0/194861475 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98c80d1710 con 0x7f98c815e4a0 2023-11-06T02:14:26.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.290+0000 7f98cd07b640 1 --2- 172.21.15.49:0/194861475 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98c815e4a0 0x7f98c80cdcf0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f98c808a2e0 tx=0x7f98b801c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=612c37adb7478c92 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:26.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.290+0000 7f98cc87a640 1 -- 172.21.15.49:0/194861475 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b80ab600 con 0x7f98c815e4a0 2023-11-06T02:14:26.286 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.290+0000 7f98cc87a640 1 -- 172.21.15.49:0/194861475 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98b80a7050 con 0x7f98c815e4a0 2023-11-06T02:14:26.286 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.290+0000 7f98cc87a640 1 -- 172.21.15.49:0/194861475 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b80ab980 con 0x7f98c815e4a0 2023-11-06T02:14:26.286 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.290+0000 7f98cfae0640 1 -- 172.21.15.49:0/194861475 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98c815e4a0 msgr2=0x7f98c80cdcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:26.286 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.290+0000 7f98cfae0640 1 --2- 172.21.15.49:0/194861475 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98c815e4a0 0x7f98c80cdcf0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f98c808a2e0 tx=0x7f98b801c3d0 comp rx=0 tx=0).stop 2023-11-06T02:14:26.286 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.290+0000 7f98cfae0640 1 -- 172.21.15.49:0/194861475 shutdown_connections 2023-11-06T02:14:26.286 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.290+0000 7f98cfae0640 1 --2- 172.21.15.49:0/194861475 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98c815e4a0 0x7f98c80cdcf0 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:26.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.290+0000 7f98cfae0640 1 --2- 172.21.15.49:0/194861475 >> v2:172.21.15.150:3300/0 conn(0x7f98c815e0c0 0x7f98c80ce840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:26.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.290+0000 7f98cfae0640 1 -- 172.21.15.49:0/194861475 >> 172.21.15.49:0/194861475 conn(0x7f98c8099c50 msgr2=0x7f98c80d0940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:26.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.290+0000 7f98cfae0640 1 -- 172.21.15.49:0/194861475 shutdown_connections 2023-11-06T02:14:26.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.290+0000 7f98cfae0640 1 -- 172.21.15.49:0/194861475 wait complete. 2023-11-06T02:14:26.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.290+0000 7f98cfae0640 1 Processor -- start 2023-11-06T02:14:26.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.290+0000 7f98cfae0640 1 -- start start 2023-11-06T02:14:26.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.290+0000 7f98cfae0640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98c80b16e0 0x7f98c80b3560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:26.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.290+0000 7f98cfae0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98c815e0c0 0x7f98c8142b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:26.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.290+0000 7f98cfae0640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f98c815e4a0 0x7f98c8143060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:26.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.290+0000 7f98cfae0640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98c8140d00 con 0x7f98c815e0c0 2023-11-06T02:14:26.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.290+0000 7f98cfae0640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98c80af010 con 0x7f98c80b16e0 2023-11-06T02:14:26.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.290+0000 7f98cfae0640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98c80afb00 con 0x7f98c815e4a0 2023-11-06T02:14:26.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.290+0000 7f98cd07b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98c815e0c0 0x7f98c8142b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:26.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.290+0000 7f98cd07b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98c815e0c0 0x7f98c8142b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54412/0 (socket says 172.21.15.49:54412) 2023-11-06T02:14:26.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.290+0000 7f98cd07b640 1 -- 172.21.15.49:0/96038039 learned_addr learned my addr 172.21.15.49:0/96038039 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:26.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.294+0000 7f98cd07b640 1 -- 172.21.15.49:0/96038039 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f98c815e4a0 msgr2=0x7f98c8143060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:26.293 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.294+0000 7f98cd07b640 1 --2- 172.21.15.49:0/96038039 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f98c815e4a0 0x7f98c8143060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:26.293 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.294+0000 7f98cd87c640 1 --2- 172.21.15.49:0/96038039 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98c80b16e0 0x7f98c80b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:26.293 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.294+0000 7f98cd07b640 1 -- 172.21.15.49:0/96038039 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98c80b16e0 msgr2=0x7f98c80b3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:26.293 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.294+0000 7f98cd07b640 1 --2- 172.21.15.49:0/96038039 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98c80b16e0 0x7f98c80b3560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:26.293 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.294+0000 7f98cd07b640 1 -- 172.21.15.49:0/96038039 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98b8009420 con 0x7f98c815e0c0 2023-11-06T02:14:26.294 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.294+0000 7f98cd07b640 1 --2- 172.21.15.49:0/96038039 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98c815e0c0 0x7f98c8142b20 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f98b8018fe0 tx=0x7f98b8019170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:26.294 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.294+0000 7f98be7fc640 1 -- 172.21.15.49:0/96038039 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b80b4400 con 0x7f98c815e0c0 2023-11-06T02:14:26.294 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.294+0000 7f98be7fc640 1 -- 172.21.15.49:0/96038039 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98b80b4560 con 0x7f98c815e0c0 2023-11-06T02:14:26.294 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.294+0000 7f98be7fc640 1 -- 172.21.15.49:0/96038039 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b80b48d0 con 0x7f98c815e0c0 2023-11-06T02:14:26.294 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.294+0000 7f98cfae0640 1 -- 172.21.15.49:0/96038039 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98c80b2080 con 0x7f98c815e0c0 2023-11-06T02:14:26.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.294+0000 7f98cfae0640 1 -- 172.21.15.49:0/96038039 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98c80af260 con 0x7f98c815e0c0 2023-11-06T02:14:26.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.294+0000 7f98cfae0640 1 -- 172.21.15.49:0/96038039 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98c8140e40 con 0x7f98c815e0c0 2023-11-06T02:14:26.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.294+0000 7f98be7fc640 1 -- 172.21.15.49:0/96038039 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f98b80b2040 con 0x7f98c815e0c0 2023-11-06T02:14:26.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.294+0000 7f98be7fc640 1 --2- 172.21.15.49:0/96038039 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f98ac065a60 0x7f98ac067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:26.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.294+0000 7f98be7fc640 1 -- 172.21.15.49:0/96038039 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f98b811d310 con 0x7f98c815e0c0 2023-11-06T02:14:26.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.294+0000 7f98cd87c640 1 --2- 172.21.15.49:0/96038039 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f98ac065a60 0x7f98ac067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:26.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.298+0000 7f98cd87c640 1 --2- 172.21.15.49:0/96038039 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f98ac065a60 0x7f98ac067f20 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f98c815b220 tx=0x7f98c4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:26.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.298+0000 7f98be7fc640 1 -- 172.21.15.49:0/96038039 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f98c8140e40 con 0x7f98c815e0c0 2023-11-06T02:14:26.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.450+0000 7f98cfae0640 1 -- 172.21.15.49:0/96038039 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f98c8140e40 con 0x7f98c815e0c0 2023-11-06T02:14:26.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.454+0000 7f98be7fc640 1 -- 172.21.15.49:0/96038039 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7f98c8140e40 con 0x7f98c815e0c0 2023-11-06T02:14:26.448 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2023-11-06T02:14:26.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.454+0000 7f98cfae0640 1 -- 172.21.15.49:0/96038039 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f98ac065a60 msgr2=0x7f98ac067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:26.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.454+0000 7f98cfae0640 1 --2- 172.21.15.49:0/96038039 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f98ac065a60 0x7f98ac067f20 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f98c815b220 tx=0x7f98c4008040 comp rx=0 tx=0).stop 2023-11-06T02:14:26.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.454+0000 7f98cfae0640 1 -- 172.21.15.49:0/96038039 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98c815e0c0 msgr2=0x7f98c8142b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:26.451 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.454+0000 7f98cfae0640 1 --2- 172.21.15.49:0/96038039 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98c815e0c0 0x7f98c8142b20 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f98b8018fe0 tx=0x7f98b8019170 comp rx=0 tx=0).stop 2023-11-06T02:14:26.451 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.454+0000 7f98cfae0640 1 -- 172.21.15.49:0/96038039 shutdown_connections 2023-11-06T02:14:26.451 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.454+0000 7f98cfae0640 1 --2- 172.21.15.49:0/96038039 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f98ac065a60 0x7f98ac067f20 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:26.451 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.454+0000 7f98cfae0640 1 --2- 172.21.15.49:0/96038039 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98c815e0c0 0x7f98c8142b20 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:26.452 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.454+0000 7f98cfae0640 1 --2- 172.21.15.49:0/96038039 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f98c815e4a0 0x7f98c8143060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:26.452 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.454+0000 7f98cfae0640 1 --2- 172.21.15.49:0/96038039 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98c80b16e0 0x7f98c80b3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:26.452 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.454+0000 7f98cfae0640 1 -- 172.21.15.49:0/96038039 >> 172.21.15.49:0/96038039 conn(0x7f98c8099c50 msgr2=0x7f98c815c200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:26.452 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.454+0000 7f98cfae0640 1 -- 172.21.15.49:0/96038039 shutdown_connections 2023-11-06T02:14:26.452 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.454+0000 7f98cfae0640 1 -- 172.21.15.49:0/96038039 wait complete. 2023-11-06T02:14:26.466 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T02:14:26.587 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.594+0000 7f45fde68640 1 Processor -- start 2023-11-06T02:14:26.588 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.594+0000 7f45fde68640 1 -- start start 2023-11-06T02:14:26.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.594+0000 7f45fde68640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f45f81595a0 0x7f45f815dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:26.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.594+0000 7f45fde68640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f45f815a990 0x7f45f80c8e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:26.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.594+0000 7f45fde68640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45f806c5a0 con 0x7f45f806d640 2023-11-06T02:14:26.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.594+0000 7f45fde68640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45f815e250 con 0x7f45f81595a0 2023-11-06T02:14:26.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.594+0000 7f45fde68640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45f806c880 con 0x7f45f815a990 2023-11-06T02:14:26.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.594+0000 7f45f77fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f45f81595a0 0x7f45f815dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:26.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.594+0000 7f45effff640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f45f806d640 0x7f45f80b0220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:54292/0 (socket says 172.21.15.49:54292) 2023-11-06T02:14:26.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.594+0000 7f45effff640 1 -- 172.21.15.49:0/4115167441 learned_addr learned my addr 172.21.15.49:0/4115167441 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:14:26.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.594+0000 7f45f7fff640 1 --2- 172.21.15.49:0/4115167441 >> v2:172.21.15.150:3300/0 conn(0x7f45f815a990 0x7f45f80c8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:26.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.594+0000 7f45f77fe640 1 -- 172.21.15.49:0/4115167441 >> v2:172.21.15.150:3300/0 conn(0x7f45f815a990 msgr2=0x7f45f80c8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:26.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.594+0000 7f45f77fe640 1 --2- 172.21.15.49:0/4115167441 >> v2:172.21.15.150:3300/0 conn(0x7f45f815a990 0x7f45f80c8e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:26.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.594+0000 7f45f77fe640 1 -- 172.21.15.49:0/4115167441 >> v1:172.21.15.49:6789/0 conn(0x7f45f806d640 legacy=0x7f45f80b0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:26.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.594+0000 7f45f77fe640 1 -- 172.21.15.49:0/4115167441 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45f806cb60 con 0x7f45f81595a0 2023-11-06T02:14:26.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.594+0000 7f45f7fff640 1 --2- 172.21.15.49:0/4115167441 >> v2:172.21.15.150:3300/0 conn(0x7f45f815a990 0x7f45f80c8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:14:26.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.598+0000 7f45f77fe640 1 --2- 172.21.15.49:0/4115167441 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45f81595a0 0x7f45f815dd10 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f45f806adb0 tx=0x7f45e801e9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0e4b13a08cfa196 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:26.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.598+0000 7f45f6ffd640 1 -- 172.21.15.49:0/4115167441 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45e80a7a30 con 0x7f45f81595a0 2023-11-06T02:14:26.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.598+0000 7f45f6ffd640 1 -- 172.21.15.49:0/4115167441 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45e80a7b90 con 0x7f45f81595a0 2023-11-06T02:14:26.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.598+0000 7f45f6ffd640 1 -- 172.21.15.49:0/4115167441 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45e80af440 con 0x7f45f81595a0 2023-11-06T02:14:26.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.598+0000 7f45fde68640 1 -- 172.21.15.49:0/4115167441 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45f81595a0 msgr2=0x7f45f815dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:26.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.598+0000 7f45fde68640 1 --2- 172.21.15.49:0/4115167441 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45f81595a0 0x7f45f815dd10 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f45f806adb0 tx=0x7f45e801e9c0 comp rx=0 tx=0).stop 2023-11-06T02:14:26.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.598+0000 7f45fde68640 1 -- 172.21.15.49:0/4115167441 shutdown_connections 2023-11-06T02:14:26.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.598+0000 7f45fde68640 1 --2- 172.21.15.49:0/4115167441 >> v2:172.21.15.150:3300/0 conn(0x7f45f815a990 0x7f45f80c8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:26.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.598+0000 7f45fde68640 1 --2- 172.21.15.49:0/4115167441 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45f81595a0 0x7f45f815dd10 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:26.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.598+0000 7f45fde68640 1 -- 172.21.15.49:0/4115167441 >> 172.21.15.49:0/4115167441 conn(0x7f45f806a370 msgr2=0x7f45f80ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:26.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.598+0000 7f45fde68640 1 -- 172.21.15.49:0/4115167441 shutdown_connections 2023-11-06T02:14:26.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.598+0000 7f45fde68640 1 -- 172.21.15.49:0/4115167441 wait complete. 2023-11-06T02:14:26.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.598+0000 7f45fde68640 1 Processor -- start 2023-11-06T02:14:26.600 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.598+0000 7f45fde68640 1 -- start start 2023-11-06T02:14:26.600 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.598+0000 7f45fde68640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45f806d640 0x7f45f81f1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:26.600 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.598+0000 7f45fde68640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f45f81595a0 0x7f45f814d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:26.600 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.598+0000 7f45fde68640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f45f815a990 0x7f45f8149180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:26.600 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.598+0000 7f45fde68640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45f814d590 con 0x7f45f806d640 2023-11-06T02:14:26.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.598+0000 7f45fde68640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45f81496c0 con 0x7f45f81595a0 2023-11-06T02:14:26.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.598+0000 7f45fde68640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45f814ee70 con 0x7f45f815a990 2023-11-06T02:14:26.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.598+0000 7f45f77fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45f806d640 0x7f45f81f1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:26.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.598+0000 7f45f77fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45f806d640 0x7f45f81f1f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54418/0 (socket says 172.21.15.49:54418) 2023-11-06T02:14:26.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.598+0000 7f45f77fe640 1 -- 172.21.15.49:0/2230306331 learned_addr learned my addr 172.21.15.49:0/2230306331 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:26.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.598+0000 7f45effff640 1 --2- 172.21.15.49:0/2230306331 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f45f81595a0 0x7f45f814d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:26.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.598+0000 7f45f7fff640 1 --2- 172.21.15.49:0/2230306331 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f45f815a990 0x7f45f8149180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:26.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.598+0000 7f45f77fe640 1 -- 172.21.15.49:0/2230306331 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f45f815a990 msgr2=0x7f45f8149180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:26.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.598+0000 7f45f77fe640 1 --2- 172.21.15.49:0/2230306331 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f45f815a990 0x7f45f8149180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:26.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.598+0000 7f45f77fe640 1 -- 172.21.15.49:0/2230306331 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f45f81595a0 msgr2=0x7f45f814d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:26.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.598+0000 7f45f77fe640 1 --2- 172.21.15.49:0/2230306331 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f45f81595a0 0x7f45f814d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:26.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.598+0000 7f45f77fe640 1 -- 172.21.15.49:0/2230306331 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45e800b6c0 con 0x7f45f806d640 2023-11-06T02:14:26.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.602+0000 7f45f77fe640 1 --2- 172.21.15.49:0/2230306331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45f806d640 0x7f45f81f1f10 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f45e801c5f0 tx=0x7f45e801b760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:26.604 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.602+0000 7f45f4ff9640 1 -- 172.21.15.49:0/2230306331 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45e80bb650 con 0x7f45f806d640 2023-11-06T02:14:26.604 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.602+0000 7f45f4ff9640 1 -- 172.21.15.49:0/2230306331 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45e80bb7b0 con 0x7f45f806d640 2023-11-06T02:14:26.604 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.602+0000 7f45f4ff9640 1 -- 172.21.15.49:0/2230306331 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45e80bbb20 con 0x7f45f806d640 2023-11-06T02:14:26.604 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.602+0000 7f45fde68640 1 -- 172.21.15.49:0/2230306331 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45f814f150 con 0x7f45f806d640 2023-11-06T02:14:26.604 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.602+0000 7f45fde68640 1 -- 172.21.15.49:0/2230306331 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45f814f710 con 0x7f45f806d640 2023-11-06T02:14:26.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.602+0000 7f45fde68640 1 -- 172.21.15.49:0/2230306331 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45f8074040 con 0x7f45f806d640 2023-11-06T02:14:26.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.602+0000 7f45f4ff9640 1 -- 172.21.15.49:0/2230306331 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f45e80bbc80 con 0x7f45f806d640 2023-11-06T02:14:26.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.602+0000 7f45f4ff9640 1 --2- 172.21.15.49:0/2230306331 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f45dc065a60 0x7f45dc067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:26.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.602+0000 7f45f4ff9640 1 -- 172.21.15.49:0/2230306331 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f45e8120e70 con 0x7f45f806d640 2023-11-06T02:14:26.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.602+0000 7f45effff640 1 --2- 172.21.15.49:0/2230306331 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f45dc065a60 0x7f45dc067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:26.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.602+0000 7f45effff640 1 --2- 172.21.15.49:0/2230306331 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f45dc065a60 0x7f45dc067f20 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f45f814d000 tx=0x7f45e000b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:26.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.606+0000 7f45f4ff9640 1 -- 172.21.15.49:0/2230306331 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f45e80ee700 con 0x7f45f806d640 2023-11-06T02:14:26.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.762+0000 7f45fde68640 1 -- 172.21.15.49:0/2230306331 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f45f80cb4b0 con 0x7f45f806d640 2023-11-06T02:14:26.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.762+0000 7f45f4ff9640 1 -- 172.21.15.49:0/2230306331 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v81) v1 ==== 92+0+10 (secure 0 0 0) 0x7f45e80f1d20 con 0x7f45f806d640 2023-11-06T02:14:26.757 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 8 2023-11-06T02:14:26.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.766+0000 7f45fde68640 1 -- 172.21.15.49:0/2230306331 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f45dc065a60 msgr2=0x7f45dc067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:26.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.766+0000 7f45fde68640 1 --2- 172.21.15.49:0/2230306331 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f45dc065a60 0x7f45dc067f20 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f45f814d000 tx=0x7f45e000b040 comp rx=0 tx=0).stop 2023-11-06T02:14:26.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.766+0000 7f45fde68640 1 -- 172.21.15.49:0/2230306331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45f806d640 msgr2=0x7f45f81f1f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:26.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.766+0000 7f45fde68640 1 --2- 172.21.15.49:0/2230306331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45f806d640 0x7f45f81f1f10 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f45e801c5f0 tx=0x7f45e801b760 comp rx=0 tx=0).stop 2023-11-06T02:14:26.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.766+0000 7f45fde68640 1 -- 172.21.15.49:0/2230306331 shutdown_connections 2023-11-06T02:14:26.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.766+0000 7f45fde68640 1 --2- 172.21.15.49:0/2230306331 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f45dc065a60 0x7f45dc067f20 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:26.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.766+0000 7f45fde68640 1 --2- 172.21.15.49:0/2230306331 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f45f815a990 0x7f45f8149180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:26.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.766+0000 7f45fde68640 1 --2- 172.21.15.49:0/2230306331 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f45f81595a0 0x7f45f814d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:26.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.766+0000 7f45fde68640 1 --2- 172.21.15.49:0/2230306331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45f806d640 0x7f45f81f1f10 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:26.762 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.766+0000 7f45fde68640 1 -- 172.21.15.49:0/2230306331 >> 172.21.15.49:0/2230306331 conn(0x7f45f806a370 msgr2=0x7f45f80cc170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:26.762 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.766+0000 7f45fde68640 1 -- 172.21.15.49:0/2230306331 shutdown_connections 2023-11-06T02:14:26.762 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.766+0000 7f45fde68640 1 -- 172.21.15.49:0/2230306331 wait complete. 2023-11-06T02:14:26.776 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T02:14:26.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.906+0000 7f9ceb30c640 1 Processor -- start 2023-11-06T02:14:26.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.906+0000 7f9ceb30c640 1 -- start start 2023-11-06T02:14:26.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.906+0000 7f9ceb30c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9ce4065930 0x7f9ce40c99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:26.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.906+0000 7f9ceb30c640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f9ce40c9f20 0x7f9ce406e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:26.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.906+0000 7f9ceb30c640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ce40cabf0 con 0x7f9ce40c9600 2023-11-06T02:14:26.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.906+0000 7f9ceb30c640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ce406def0 con 0x7f9ce40c9f20 2023-11-06T02:14:26.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.906+0000 7f9ceb30c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce415b970 con 0x7f9ce4065930 2023-11-06T02:14:26.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.906+0000 7f9ce88a7640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9ce4065930 0x7f9ce40c99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:26.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.906+0000 7f9ce98a9640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f9ce40c9f20 0x7f9ce406e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:26.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.906+0000 7f9ce88a7640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9ce4065930 0x7f9ce40c99e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.49:52606/0 (socket says 172.21.15.49:52606) 2023-11-06T02:14:26.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.906+0000 7f9ce88a7640 1 -- 172.21.15.49:0/456818510 learned_addr learned my addr 172.21.15.49:0/456818510 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:26.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.906+0000 7f9ce88a7640 1 -- 172.21.15.49:0/456818510 >> v2:172.21.15.150:3300/0 conn(0x7f9ce40c9f20 msgr2=0x7f9ce406e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:26.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.906+0000 7f9ce88a7640 1 --2- 172.21.15.49:0/456818510 >> v2:172.21.15.150:3300/0 conn(0x7f9ce40c9f20 0x7f9ce406e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:26.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.906+0000 7f9ce88a7640 1 -- 172.21.15.49:0/456818510 >> v1:172.21.15.150:6789/0 conn(0x7f9ce40c9600 legacy=0x7f9ce40cb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:26.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.906+0000 7f9ce88a7640 1 -- 172.21.15.49:0/456818510 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ce40ca8c0 con 0x7f9ce4065930 2023-11-06T02:14:26.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.906+0000 7f9ce98a9640 1 --2- 172.21.15.49:0/456818510 >> v2:172.21.15.150:3300/0 conn(0x7f9ce40c9f20 0x7f9ce406e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T02:14:26.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.910+0000 7f9ce88a7640 1 --2- 172.21.15.49:0/456818510 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9ce4065930 0x7f9ce40c99e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9cd4009870 tx=0x7f9cd401c540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d999478455f04c2 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:26.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.910+0000 7f9cdbfff640 1 -- 172.21.15.49:0/456818510 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cd40a7070 con 0x7f9ce4065930 2023-11-06T02:14:26.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.910+0000 7f9cdbfff640 1 -- 172.21.15.49:0/456818510 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cd40ab430 con 0x7f9ce4065930 2023-11-06T02:14:26.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.910+0000 7f9cdbfff640 1 -- 172.21.15.49:0/456818510 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cd40ab7c0 con 0x7f9ce4065930 2023-11-06T02:14:26.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.910+0000 7f9ceb30c640 1 -- 172.21.15.49:0/456818510 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9ce4065930 msgr2=0x7f9ce40c99e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:26.908 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.910+0000 7f9ceb30c640 1 --2- 172.21.15.49:0/456818510 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9ce4065930 0x7f9ce40c99e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9cd4009870 tx=0x7f9cd401c540 comp rx=0 tx=0).stop 2023-11-06T02:14:26.908 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.910+0000 7f9ceb30c640 1 -- 172.21.15.49:0/456818510 shutdown_connections 2023-11-06T02:14:26.908 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.910+0000 7f9ceb30c640 1 --2- 172.21.15.49:0/456818510 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9ce4065930 0x7f9ce40c99e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:26.908 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.910+0000 7f9ceb30c640 1 --2- 172.21.15.49:0/456818510 >> v2:172.21.15.150:3300/0 conn(0x7f9ce40c9f20 0x7f9ce406e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:26.908 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.910+0000 7f9ceb30c640 1 -- 172.21.15.49:0/456818510 >> 172.21.15.49:0/456818510 conn(0x7f9ce40b0aa0 msgr2=0x7f9ce40b0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:26.908 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.910+0000 7f9ceb30c640 1 -- 172.21.15.49:0/456818510 shutdown_connections 2023-11-06T02:14:26.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.910+0000 7f9ceb30c640 1 -- 172.21.15.49:0/456818510 wait complete. 2023-11-06T02:14:26.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.910+0000 7f9ceb30c640 1 Processor -- start 2023-11-06T02:14:26.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.910+0000 7f9ceb30c640 1 -- start start 2023-11-06T02:14:26.912 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.914+0000 7f9ceb30c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ce4065930 0x7f9ce4154d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:26.912 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.914+0000 7f9ceb30c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9ce40c9600 0x7f9ce40c2a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:26.912 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.914+0000 7f9ceb30c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ce40c9f20 0x7f9ce40c2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:26.913 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.914+0000 7f9ceb30c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce40b4dd0 con 0x7f9ce4065930 2023-11-06T02:14:26.913 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.914+0000 7f9ceb30c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce41f6cc0 con 0x7f9ce40c9f20 2023-11-06T02:14:26.913 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.914+0000 7f9ceb30c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce409d750 con 0x7f9ce40c9600 2023-11-06T02:14:26.913 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.914+0000 7f9ce90a8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ce4065930 0x7f9ce4154d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:26.913 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.914+0000 7f9ce90a8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ce4065930 0x7f9ce4154d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54432/0 (socket says 172.21.15.49:54432) 2023-11-06T02:14:26.913 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.914+0000 7f9ce90a8640 1 -- 172.21.15.49:0/3815086879 learned_addr learned my addr 172.21.15.49:0/3815086879 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:26.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.914+0000 7f9ce98a9640 1 --2- 172.21.15.49:0/3815086879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ce40c9f20 0x7f9ce40c2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:26.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.914+0000 7f9ce88a7640 1 --2- 172.21.15.49:0/3815086879 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9ce40c9600 0x7f9ce40c2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:26.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.914+0000 7f9ce90a8640 1 -- 172.21.15.49:0/3815086879 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9ce40c9600 msgr2=0x7f9ce40c2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:26.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.914+0000 7f9ce90a8640 1 --2- 172.21.15.49:0/3815086879 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9ce40c9600 0x7f9ce40c2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:26.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.914+0000 7f9ce90a8640 1 -- 172.21.15.49:0/3815086879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ce40c9f20 msgr2=0x7f9ce40c2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:26.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.914+0000 7f9ce90a8640 1 --2- 172.21.15.49:0/3815086879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ce40c9f20 0x7f9ce40c2fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:26.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.914+0000 7f9ce90a8640 1 -- 172.21.15.49:0/3815086879 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cdc009a70 con 0x7f9ce4065930 2023-11-06T02:14:26.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.914+0000 7f9ce90a8640 1 --2- 172.21.15.49:0/3815086879 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ce4065930 0x7f9ce4154d00 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f9ce000c3f0 tx=0x7f9ce000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:26.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.914+0000 7f9cd9ffb640 1 -- 172.21.15.49:0/3815086879 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce00052d0 con 0x7f9ce4065930 2023-11-06T02:14:26.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.914+0000 7f9cd9ffb640 1 -- 172.21.15.49:0/3815086879 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ce0005430 con 0x7f9ce4065930 2023-11-06T02:14:26.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.914+0000 7f9ceb30c640 1 -- 172.21.15.49:0/3815086879 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cd40094d0 con 0x7f9ce4065930 2023-11-06T02:14:26.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.914+0000 7f9cd9ffb640 1 -- 172.21.15.49:0/3815086879 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce0003840 con 0x7f9ce4065930 2023-11-06T02:14:26.917 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.914+0000 7f9ceb30c640 1 -- 172.21.15.49:0/3815086879 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ce41f6fc0 con 0x7f9ce4065930 2023-11-06T02:14:26.917 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.914+0000 7f9ceb30c640 1 -- 172.21.15.49:0/3815086879 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ce40b17e0 con 0x7f9ce4065930 2023-11-06T02:14:26.917 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.914+0000 7f9cd9ffb640 1 -- 172.21.15.49:0/3815086879 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f9ce00039a0 con 0x7f9ce4065930 2023-11-06T02:14:26.917 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.914+0000 7f9cd9ffb640 1 --2- 172.21.15.49:0/3815086879 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9cc8065a60 0x7f9cc8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:26.917 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.914+0000 7f9cd9ffb640 1 -- 172.21.15.49:0/3815086879 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f9ce0082840 con 0x7f9ce4065930 2023-11-06T02:14:26.918 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.914+0000 7f9ce88a7640 1 --2- 172.21.15.49:0/3815086879 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9cc8065a60 0x7f9cc8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:26.918 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.918+0000 7f9ce88a7640 1 --2- 172.21.15.49:0/3815086879 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9cc8065a60 0x7f9cc8067f20 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f9cd40af000 tx=0x7f9cd4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:26.918 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:26.918+0000 7f9cd9ffb640 1 -- 172.21.15.49:0/3815086879 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f9ce00500d0 con 0x7f9ce4065930 2023-11-06T02:14:27.070 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.074+0000 7f9ceb30c640 1 -- 172.21.15.49:0/3815086879 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9ce40b4f10 con 0x7f9ce4065930 2023-11-06T02:14:27.070 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.074+0000 7f9cd9ffb640 1 -- 172.21.15.49:0/3815086879 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v81) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9ce00536f0 con 0x7f9ce4065930 2023-11-06T02:14:27.071 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2023-11-06T02:14:27.072 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.078+0000 7f9ceb30c640 1 -- 172.21.15.49:0/3815086879 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9cc8065a60 msgr2=0x7f9cc8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:27.073 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.078+0000 7f9ceb30c640 1 --2- 172.21.15.49:0/3815086879 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9cc8065a60 0x7f9cc8067f20 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f9cd40af000 tx=0x7f9cd4008040 comp rx=0 tx=0).stop 2023-11-06T02:14:27.073 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.078+0000 7f9ceb30c640 1 -- 172.21.15.49:0/3815086879 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ce4065930 msgr2=0x7f9ce4154d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:27.073 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.078+0000 7f9ceb30c640 1 --2- 172.21.15.49:0/3815086879 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ce4065930 0x7f9ce4154d00 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f9ce000c3f0 tx=0x7f9ce000c8b0 comp rx=0 tx=0).stop 2023-11-06T02:14:27.073 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.078+0000 7f9ceb30c640 1 -- 172.21.15.49:0/3815086879 shutdown_connections 2023-11-06T02:14:27.074 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.078+0000 7f9ceb30c640 1 --2- 172.21.15.49:0/3815086879 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9cc8065a60 0x7f9cc8067f20 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:27.074 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.078+0000 7f9ceb30c640 1 --2- 172.21.15.49:0/3815086879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ce40c9f20 0x7f9ce40c2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:27.074 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.078+0000 7f9ceb30c640 1 --2- 172.21.15.49:0/3815086879 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9ce40c9600 0x7f9ce40c2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:27.074 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.078+0000 7f9ceb30c640 1 --2- 172.21.15.49:0/3815086879 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ce4065930 0x7f9ce4154d00 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:27.074 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.078+0000 7f9ceb30c640 1 -- 172.21.15.49:0/3815086879 >> 172.21.15.49:0/3815086879 conn(0x7f9ce40b0aa0 msgr2=0x7f9ce409ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:27.075 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.078+0000 7f9ceb30c640 1 -- 172.21.15.49:0/3815086879 shutdown_connections 2023-11-06T02:14:27.075 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.078+0000 7f9ceb30c640 1 -- 172.21.15.49:0/3815086879 wait complete. 2023-11-06T02:14:27.089 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T02:14:27.219 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.226+0000 7f8d27b49640 1 Processor -- start 2023-11-06T02:14:27.220 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.226+0000 7f8d27b49640 1 -- start start 2023-11-06T02:14:27.222 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.226+0000 7f8d27b49640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f8d201595a0 0x7f8d2015dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:27.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.226+0000 7f8d27b49640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8d2006d640 0x7f8d200b0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:27.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.226+0000 7f8d27b49640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d2006c5a0 con 0x7f8d200b0760 2023-11-06T02:14:27.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.226+0000 7f8d27b49640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d2015e250 con 0x7f8d201595a0 2023-11-06T02:14:27.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.226+0000 7f8d27b49640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2006c880 con 0x7f8d2006d640 2023-11-06T02:14:27.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.226+0000 7f8d260e6640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f8d200b0760 0x7f8d200c8e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:54294/0 (socket says 172.21.15.49:54294) 2023-11-06T02:14:27.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.226+0000 7f8d260e6640 1 -- 172.21.15.49:0/295406148 learned_addr learned my addr 172.21.15.49:0/295406148 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:14:27.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.226+0000 7f8d258e5640 1 --2- 172.21.15.49:0/295406148 >> v2:172.21.15.150:3300/0 conn(0x7f8d201595a0 0x7f8d2015dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:27.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.226+0000 7f8d250e4640 1 --2- 172.21.15.49:0/295406148 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8d2006d640 0x7f8d200b0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:27.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.226+0000 7f8d248e3640 1 -- 172.21.15.49:0/295406148 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4281859719 0 0) 0x7f8d2006c5a0 con 0x7f8d200b0760 2023-11-06T02:14:27.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.226+0000 7f8d248e3640 1 -- 172.21.15.49:0/295406148 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d0c003530 con 0x7f8d200b0760 2023-11-06T02:14:27.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.226+0000 7f8d248e3640 1 -- 172.21.15.49:0/295406148 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4121301753 0 0) 0x7f8d0c003530 con 0x7f8d200b0760 2023-11-06T02:14:27.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.226+0000 7f8d248e3640 1 -- 172.21.15.49:0/295406148 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8d2006d640 msgr2=0x7f8d200b0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:27.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.226+0000 7f8d248e3640 1 --2- 172.21.15.49:0/295406148 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8d2006d640 0x7f8d200b0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:27.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.226+0000 7f8d248e3640 1 -- 172.21.15.49:0/295406148 >> v2:172.21.15.150:3300/0 conn(0x7f8d201595a0 msgr2=0x7f8d2015dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:27.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.226+0000 7f8d248e3640 1 --2- 172.21.15.49:0/295406148 >> v2:172.21.15.150:3300/0 conn(0x7f8d201595a0 0x7f8d2015dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:27.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.226+0000 7f8d248e3640 1 -- 172.21.15.49:0/295406148 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d2006cb60 con 0x7f8d200b0760 2023-11-06T02:14:27.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.226+0000 7f8d248e3640 1 -- 172.21.15.49:0/295406148 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7f8d18001a00 con 0x7f8d200b0760 2023-11-06T02:14:27.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.226+0000 7f8d248e3640 1 -- 172.21.15.49:0/295406148 >> v1:172.21.15.49:6789/0 conn(0x7f8d200b0760 legacy=0x7f8d200c8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:27.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.226+0000 7f8d248e3640 1 --2- 172.21.15.49:0/295406148 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d0c003980 0x7f8d0c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:27.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.226+0000 7f8d248e3640 1 --2- 172.21.15.49:0/295406148 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8d2006d640 0x7f8d0c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:27.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.226+0000 7f8d248e3640 1 --2- 172.21.15.49:0/295406148 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d0c004d10 0x7f8d0c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:27.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.226+0000 7f8d248e3640 1 -- 172.21.15.49:0/295406148 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2015e250 con 0x7f8d0c003980 2023-11-06T02:14:27.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.226+0000 7f8d248e3640 1 -- 172.21.15.49:0/295406148 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2006c880 con 0x7f8d0c004d10 2023-11-06T02:14:27.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.226+0000 7f8d248e3640 1 -- 172.21.15.49:0/295406148 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d0c00b990 con 0x7f8d2006d640 2023-11-06T02:14:27.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.226+0000 7f8d258e5640 1 --2- 172.21.15.49:0/295406148 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d0c003980 0x7f8d0c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:27.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.226+0000 7f8d250e4640 1 --2- 172.21.15.49:0/295406148 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8d2006d640 0x7f8d0c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:27.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.226+0000 7f8d260e6640 1 --2- 172.21.15.49:0/295406148 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d0c004d10 0x7f8d0c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:27.229 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.226+0000 7f8d258e5640 1 -- 172.21.15.49:0/295406148 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8d2006d640 msgr2=0x7f8d0c0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:27.229 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.226+0000 7f8d258e5640 1 --2- 172.21.15.49:0/295406148 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8d2006d640 0x7f8d0c0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:27.229 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.226+0000 7f8d258e5640 1 -- 172.21.15.49:0/295406148 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d0c004d10 msgr2=0x7f8d0c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:27.229 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.226+0000 7f8d258e5640 1 --2- 172.21.15.49:0/295406148 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d0c004d10 0x7f8d0c005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:27.229 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.226+0000 7f8d258e5640 1 -- 172.21.15.49:0/295406148 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d0c003530 con 0x7f8d0c003980 2023-11-06T02:14:27.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.226+0000 7f8d260e6640 1 --2- 172.21.15.49:0/295406148 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d0c004d10 0x7f8d0c005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:14:27.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.230+0000 7f8d258e5640 1 --2- 172.21.15.49:0/295406148 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d0c003980 0x7f8d0c003da0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c009b10 tx=0x7f8d1c01ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18ec28e68d322158 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:27.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.230+0000 7f8d248e3640 1 -- 172.21.15.49:0/295406148 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c0a56e0 con 0x7f8d0c003980 2023-11-06T02:14:27.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.230+0000 7f8d248e3640 1 -- 172.21.15.49:0/295406148 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d1c0a5840 con 0x7f8d0c003980 2023-11-06T02:14:27.234 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.230+0000 7f8d248e3640 1 -- 172.21.15.49:0/295406148 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c0a5bf0 con 0x7f8d0c003980 2023-11-06T02:14:27.234 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.230+0000 7f8d27b49640 1 -- 172.21.15.49:0/295406148 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d0c003980 msgr2=0x7f8d0c003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:27.234 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.230+0000 7f8d27b49640 1 --2- 172.21.15.49:0/295406148 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d0c003980 0x7f8d0c003da0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c009b10 tx=0x7f8d1c01ca60 comp rx=0 tx=0).stop 2023-11-06T02:14:27.234 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.230+0000 7f8d260e6640 1 -- 172.21.15.49:0/295406148 reap_dead start 2023-11-06T02:14:27.234 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.230+0000 7f8d27b49640 1 -- 172.21.15.49:0/295406148 shutdown_connections 2023-11-06T02:14:27.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.230+0000 7f8d27b49640 1 -- 172.21.15.49:0/295406148 >> 172.21.15.49:0/295406148 conn(0x7f8d2006a370 msgr2=0x7f8d200ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:27.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.230+0000 7f8d27b49640 1 -- 172.21.15.49:0/295406148 shutdown_connections 2023-11-06T02:14:27.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.230+0000 7f8d27b49640 1 -- 172.21.15.49:0/295406148 wait complete. 2023-11-06T02:14:27.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.230+0000 7f8d27b49640 1 Processor -- start 2023-11-06T02:14:27.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.230+0000 7f8d27b49640 1 -- start start 2023-11-06T02:14:27.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.230+0000 7f8d27b49640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d0c003980 0x7f8d200b3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:27.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.230+0000 7f8d27b49640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d201f54d0 0x7f8d201f5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:27.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.230+0000 7f8d27b49640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8d200c7940 0x7f8d200c7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:27.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.234+0000 7f8d27b49640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d200c8300 con 0x7f8d201f54d0 2023-11-06T02:14:27.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.234+0000 7f8d27b49640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d201f5f60 con 0x7f8d0c003980 2023-11-06T02:14:27.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.234+0000 7f8d27b49640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d201f6240 con 0x7f8d200c7940 2023-11-06T02:14:27.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.234+0000 7f8d250e4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d201f54d0 0x7f8d201f5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:27.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.234+0000 7f8d250e4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d201f54d0 0x7f8d201f5950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54440/0 (socket says 172.21.15.49:54440) 2023-11-06T02:14:27.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.234+0000 7f8d250e4640 1 -- 172.21.15.49:0/2551053751 learned_addr learned my addr 172.21.15.49:0/2551053751 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:27.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.234+0000 7f8d258e5640 1 --2- 172.21.15.49:0/2551053751 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d0c003980 0x7f8d200b3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:27.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.234+0000 7f8d260e6640 1 --2- 172.21.15.49:0/2551053751 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8d200c7940 0x7f8d200c7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:27.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.234+0000 7f8d250e4640 1 -- 172.21.15.49:0/2551053751 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8d200c7940 msgr2=0x7f8d200c7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:27.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.234+0000 7f8d250e4640 1 --2- 172.21.15.49:0/2551053751 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8d200c7940 0x7f8d200c7dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:27.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.234+0000 7f8d250e4640 1 -- 172.21.15.49:0/2551053751 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d0c003980 msgr2=0x7f8d200b3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:27.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.234+0000 7f8d250e4640 1 --2- 172.21.15.49:0/2551053751 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d0c003980 0x7f8d200b3530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:27.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.234+0000 7f8d250e4640 1 -- 172.21.15.49:0/2551053751 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d201f6520 con 0x7f8d201f54d0 2023-11-06T02:14:27.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.234+0000 7f8d250e4640 1 --2- 172.21.15.49:0/2551053751 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d201f54d0 0x7f8d201f5950 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f8d100115e0 tx=0x7f8d10011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:27.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.234+0000 7f8d167fc640 1 -- 172.21.15.49:0/2551053751 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1000f8f0 con 0x7f8d201f54d0 2023-11-06T02:14:27.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.234+0000 7f8d167fc640 1 -- 172.21.15.49:0/2551053751 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d1000fa50 con 0x7f8d201f54d0 2023-11-06T02:14:27.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.234+0000 7f8d27b49640 1 -- 172.21.15.49:0/2551053751 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d201f6800 con 0x7f8d201f54d0 2023-11-06T02:14:27.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.234+0000 7f8d167fc640 1 -- 172.21.15.49:0/2551053751 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1000fdc0 con 0x7f8d201f54d0 2023-11-06T02:14:27.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.234+0000 7f8d27b49640 1 -- 172.21.15.49:0/2551053751 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d200b3a70 con 0x7f8d201f54d0 2023-11-06T02:14:27.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.234+0000 7f8d27b49640 1 -- 172.21.15.49:0/2551053751 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d200c9410 con 0x7f8d201f54d0 2023-11-06T02:14:27.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.234+0000 7f8d167fc640 1 -- 172.21.15.49:0/2551053751 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f8d10015460 con 0x7f8d201f54d0 2023-11-06T02:14:27.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.234+0000 7f8d167fc640 1 --2- 172.21.15.49:0/2551053751 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8d04065a90 0x7f8d04067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:27.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.234+0000 7f8d167fc640 1 -- 172.21.15.49:0/2551053751 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f8d10083c10 con 0x7f8d201f54d0 2023-11-06T02:14:27.242 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.234+0000 7f8d258e5640 1 --2- 172.21.15.49:0/2551053751 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8d04065a90 0x7f8d04067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:27.242 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.234+0000 7f8d258e5640 1 --2- 172.21.15.49:0/2551053751 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8d04065a90 0x7f8d04067f50 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c0af000 tx=0x7f8d1c0046d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:27.242 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.238+0000 7f8d167fc640 1 -- 172.21.15.49:0/2551053751 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f8d10051410 con 0x7f8d201f54d0 2023-11-06T02:14:27.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.394+0000 7f8d27b49640 1 -- 172.21.15.49:0/2551053751 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8d2006c210 con 0x7f8d201f54d0 2023-11-06T02:14:27.391 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.394+0000 7f8d167fc640 1 -- 172.21.15.49:0/2551053751 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v81) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8d2006c210 con 0x7f8d201f54d0 2023-11-06T02:14:27.391 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2023-11-06T02:14:27.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.398+0000 7f8d27b49640 1 -- 172.21.15.49:0/2551053751 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8d04065a90 msgr2=0x7f8d04067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:27.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.398+0000 7f8d27b49640 1 --2- 172.21.15.49:0/2551053751 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8d04065a90 0x7f8d04067f50 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c0af000 tx=0x7f8d1c0046d0 comp rx=0 tx=0).stop 2023-11-06T02:14:27.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.398+0000 7f8d27b49640 1 -- 172.21.15.49:0/2551053751 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d201f54d0 msgr2=0x7f8d201f5950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:27.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.398+0000 7f8d27b49640 1 --2- 172.21.15.49:0/2551053751 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d201f54d0 0x7f8d201f5950 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f8d100115e0 tx=0x7f8d10011aa0 comp rx=0 tx=0).stop 2023-11-06T02:14:27.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.398+0000 7f8d27b49640 1 -- 172.21.15.49:0/2551053751 shutdown_connections 2023-11-06T02:14:27.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.398+0000 7f8d27b49640 1 --2- 172.21.15.49:0/2551053751 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8d04065a90 0x7f8d04067f50 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:27.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.398+0000 7f8d27b49640 1 --2- 172.21.15.49:0/2551053751 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d201f54d0 0x7f8d201f5950 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:27.395 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.398+0000 7f8d27b49640 1 --2- 172.21.15.49:0/2551053751 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8d200c7940 0x7f8d200c7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:27.395 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.398+0000 7f8d27b49640 1 --2- 172.21.15.49:0/2551053751 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d0c003980 0x7f8d200b3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:27.395 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.398+0000 7f8d27b49640 1 -- 172.21.15.49:0/2551053751 >> 172.21.15.49:0/2551053751 conn(0x7f8d2006a370 msgr2=0x7f8d200cb540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:27.395 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.398+0000 7f8d27b49640 1 -- 172.21.15.49:0/2551053751 shutdown_connections 2023-11-06T02:14:27.395 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.398+0000 7f8d27b49640 1 -- 172.21.15.49:0/2551053751 wait complete. 2023-11-06T02:14:27.410 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-06T02:14:27.536 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.542+0000 7f806e7ca640 1 Processor -- start 2023-11-06T02:14:27.537 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.542+0000 7f806e7ca640 1 -- start start 2023-11-06T02:14:27.538 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.542+0000 7f806e7ca640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f806815ab50 0x7f80680c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:27.538 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.542+0000 7f806e7ca640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f80680cb010 0x7f80680b0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:27.538 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.542+0000 7f806e7ca640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80680cab80 con 0x7f80680b05c0 2023-11-06T02:14:27.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.542+0000 7f806e7ca640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80680b0a10 con 0x7f806815ab50 2023-11-06T02:14:27.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.542+0000 7f806e7ca640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f806815af30 con 0x7f80680cb010 2023-11-06T02:14:27.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.542+0000 7f8067fff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f806815ab50 0x7f80680c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:27.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.542+0000 7f806cd67640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f80680b05c0 0x7f80680700a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:54296/0 (socket says 172.21.15.49:54296) 2023-11-06T02:14:27.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.542+0000 7f806cd67640 1 -- 172.21.15.49:0/1010852367 learned_addr learned my addr 172.21.15.49:0/1010852367 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:14:27.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.542+0000 7f80677fe640 1 --2- 172.21.15.49:0/1010852367 >> v2:172.21.15.150:3300/0 conn(0x7f80680cb010 0x7f80680b0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:27.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.542+0000 7f8067fff640 1 -- 172.21.15.49:0/1010852367 >> v2:172.21.15.150:3300/0 conn(0x7f80680cb010 msgr2=0x7f80680b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:27.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.542+0000 7f8067fff640 1 --2- 172.21.15.49:0/1010852367 >> v2:172.21.15.150:3300/0 conn(0x7f80680cb010 0x7f80680b0080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:27.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.542+0000 7f8067fff640 1 -- 172.21.15.49:0/1010852367 >> v1:172.21.15.49:6789/0 conn(0x7f80680b05c0 legacy=0x7f80680700a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:27.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.542+0000 7f8067fff640 1 -- 172.21.15.49:0/1010852367 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80680cb5b0 con 0x7f806815ab50 2023-11-06T02:14:27.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.542+0000 7f80677fe640 1 --2- 172.21.15.49:0/1010852367 >> v2:172.21.15.150:3300/0 conn(0x7f80680cb010 0x7f80680b0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:14:27.542 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.546+0000 7f8067fff640 1 --2- 172.21.15.49:0/1010852367 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f806815ab50 0x7f80680c9bb0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f806806dde0 tx=0x7f805801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a9d8c5b705f1a8e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:27.542 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.546+0000 7f8066ffd640 1 -- 172.21.15.49:0/1010852367 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80580ab610 con 0x7f806815ab50 2023-11-06T02:14:27.542 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.546+0000 7f8066ffd640 1 -- 172.21.15.49:0/1010852367 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80580ab770 con 0x7f806815ab50 2023-11-06T02:14:27.542 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.546+0000 7f8066ffd640 1 -- 172.21.15.49:0/1010852367 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80580abb00 con 0x7f806815ab50 2023-11-06T02:14:27.542 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.546+0000 7f806e7ca640 1 -- 172.21.15.49:0/1010852367 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f806815ab50 msgr2=0x7f80680c9bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:27.543 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.546+0000 7f806e7ca640 1 --2- 172.21.15.49:0/1010852367 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f806815ab50 0x7f80680c9bb0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f806806dde0 tx=0x7f805801c410 comp rx=0 tx=0).stop 2023-11-06T02:14:27.543 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.546+0000 7f806e7ca640 1 -- 172.21.15.49:0/1010852367 shutdown_connections 2023-11-06T02:14:27.543 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.546+0000 7f806e7ca640 1 --2- 172.21.15.49:0/1010852367 >> v2:172.21.15.150:3300/0 conn(0x7f80680cb010 0x7f80680b0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:27.543 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.546+0000 7f806e7ca640 1 --2- 172.21.15.49:0/1010852367 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f806815ab50 0x7f80680c9bb0 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:27.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.546+0000 7f806e7ca640 1 -- 172.21.15.49:0/1010852367 >> 172.21.15.49:0/1010852367 conn(0x7f806806c0a0 msgr2=0x7f806806c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:27.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.546+0000 7f806e7ca640 1 -- 172.21.15.49:0/1010852367 shutdown_connections 2023-11-06T02:14:27.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.546+0000 7f806e7ca640 1 -- 172.21.15.49:0/1010852367 wait complete. 2023-11-06T02:14:27.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.546+0000 7f806e7ca640 1 Processor -- start 2023-11-06T02:14:27.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.546+0000 7f806e7ca640 1 -- start start 2023-11-06T02:14:27.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.550+0000 7f806e7ca640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f80680b05c0 0x7f80680c0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:27.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.550+0000 7f806e7ca640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80680cb010 0x7f80680a8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:27.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.550+0000 7f806e7ca640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f806815ab50 0x7f80680ae060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:27.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.550+0000 7f806e7ca640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80681416c0 con 0x7f80680cb010 2023-11-06T02:14:27.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.550+0000 7f806e7ca640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8068141910 con 0x7f806815ab50 2023-11-06T02:14:27.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.550+0000 7f806e7ca640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8068141bf0 con 0x7f80680b05c0 2023-11-06T02:14:27.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.550+0000 7f80677fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80680cb010 0x7f80680a8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:27.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.550+0000 7f80677fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80680cb010 0x7f80680a8ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54470/0 (socket says 172.21.15.49:54470) 2023-11-06T02:14:27.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.550+0000 7f80677fe640 1 -- 172.21.15.49:0/2463138905 learned_addr learned my addr 172.21.15.49:0/2463138905 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:27.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.550+0000 7f806cd67640 1 --2- 172.21.15.49:0/2463138905 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f806815ab50 0x7f80680ae060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:27.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.550+0000 7f8067fff640 1 --2- 172.21.15.49:0/2463138905 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f80680b05c0 0x7f80680c0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:27.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.550+0000 7f80677fe640 1 -- 172.21.15.49:0/2463138905 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f80680b05c0 msgr2=0x7f80680c0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:27.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.550+0000 7f80677fe640 1 --2- 172.21.15.49:0/2463138905 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f80680b05c0 0x7f80680c0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:27.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.550+0000 7f80677fe640 1 -- 172.21.15.49:0/2463138905 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f806815ab50 msgr2=0x7f80680ae060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:27.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.550+0000 7f80677fe640 1 --2- 172.21.15.49:0/2463138905 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f806815ab50 0x7f80680ae060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:27.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.550+0000 7f80677fe640 1 -- 172.21.15.49:0/2463138905 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80580093b0 con 0x7f80680cb010 2023-11-06T02:14:27.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.550+0000 7f806cd67640 1 --2- 172.21.15.49:0/2463138905 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f806815ab50 0x7f80680ae060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:14:27.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.550+0000 7f80677fe640 1 --2- 172.21.15.49:0/2463138905 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80680cb010 0x7f80680a8ab0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f805c00c7f0 tx=0x7f805c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:27.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.550+0000 7f8064ff9640 1 -- 172.21.15.49:0/2463138905 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f805c007820 con 0x7f80680cb010 2023-11-06T02:14:27.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.550+0000 7f8064ff9640 1 -- 172.21.15.49:0/2463138905 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f805c007980 con 0x7f80680cb010 2023-11-06T02:14:27.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.550+0000 7f806e7ca640 1 -- 172.21.15.49:0/2463138905 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8068141ed0 con 0x7f80680cb010 2023-11-06T02:14:27.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.550+0000 7f8064ff9640 1 -- 172.21.15.49:0/2463138905 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f805c007d30 con 0x7f80680cb010 2023-11-06T02:14:27.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.550+0000 7f806e7ca640 1 -- 172.21.15.49:0/2463138905 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8068142490 con 0x7f80680cb010 2023-11-06T02:14:27.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.550+0000 7f806e7ca640 1 -- 172.21.15.49:0/2463138905 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80680b0de0 con 0x7f80680cb010 2023-11-06T02:14:27.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.550+0000 7f8064ff9640 1 -- 172.21.15.49:0/2463138905 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f805c0113f0 con 0x7f80680cb010 2023-11-06T02:14:27.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.550+0000 7f8064ff9640 1 --2- 172.21.15.49:0/2463138905 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8054065a60 0x7f8054067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:27.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.550+0000 7f8064ff9640 1 -- 172.21.15.49:0/2463138905 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f805c083790 con 0x7f80680cb010 2023-11-06T02:14:27.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.550+0000 7f8067fff640 1 --2- 172.21.15.49:0/2463138905 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8054065a60 0x7f8054067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:27.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.550+0000 7f8067fff640 1 --2- 172.21.15.49:0/2463138905 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8054065a60 0x7f8054067f20 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f80580af000 tx=0x7f8058008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:27.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.554+0000 7f8064ff9640 1 -- 172.21.15.49:0/2463138905 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f805c050ef0 con 0x7f80680cb010 2023-11-06T02:14:27.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.722+0000 7f806e7ca640 1 -- 172.21.15.49:0/2463138905 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f8068074040 con 0x7f80680cb010 2023-11-06T02:14:27.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.726+0000 7f8064ff9640 1 -- 172.21.15.49:0/2463138905 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v81) v1 ==== 80+0+122 (secure 0 0 0) 0x7f805c054510 con 0x7f80680cb010 2023-11-06T02:14:27.720 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:6835/1130379293 2023-11-07T02:14:13.284419+0000 2023-11-06T02:14:27.720 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:6834/1130379293 2023-11-07T02:14:13.284419+0000 2023-11-06T02:14:27.722 INFO:teuthology.orchestra.run.smithi049.stderr:listed 2 entries 2023-11-06T02:14:27.722 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.726+0000 7f806e7ca640 1 -- 172.21.15.49:0/2463138905 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8054065a60 msgr2=0x7f8054067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:27.722 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.726+0000 7f806e7ca640 1 --2- 172.21.15.49:0/2463138905 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8054065a60 0x7f8054067f20 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f80580af000 tx=0x7f8058008040 comp rx=0 tx=0).stop 2023-11-06T02:14:27.722 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.726+0000 7f806e7ca640 1 -- 172.21.15.49:0/2463138905 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80680cb010 msgr2=0x7f80680a8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:27.723 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.726+0000 7f806e7ca640 1 --2- 172.21.15.49:0/2463138905 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80680cb010 0x7f80680a8ab0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f805c00c7f0 tx=0x7f805c00ccb0 comp rx=0 tx=0).stop 2023-11-06T02:14:27.723 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.726+0000 7f806e7ca640 1 -- 172.21.15.49:0/2463138905 shutdown_connections 2023-11-06T02:14:27.723 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.726+0000 7f806e7ca640 1 --2- 172.21.15.49:0/2463138905 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8054065a60 0x7f8054067f20 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:27.723 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.726+0000 7f806e7ca640 1 --2- 172.21.15.49:0/2463138905 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80680cb010 0x7f80680a8ab0 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:27.723 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.726+0000 7f806e7ca640 1 --2- 172.21.15.49:0/2463138905 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f806815ab50 0x7f80680ae060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:27.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.726+0000 7f806e7ca640 1 --2- 172.21.15.49:0/2463138905 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f80680b05c0 0x7f80680c0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:27.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.726+0000 7f806e7ca640 1 -- 172.21.15.49:0/2463138905 >> 172.21.15.49:0/2463138905 conn(0x7f806806c0a0 msgr2=0x7f80680aee90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:27.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.726+0000 7f806e7ca640 1 -- 172.21.15.49:0/2463138905 shutdown_connections 2023-11-06T02:14:27.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:27.726+0000 7f806e7ca640 1 -- 172.21.15.49:0/2463138905 wait complete. 2023-11-06T02:14:27.739 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi049.front.sepia.ceph.com... 2023-11-06T02:14:27.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:14:27.740 DEBUG:teuthology.orchestra.run.smithi049:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-06T02:14:27.751 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi049.stderr:ceph-fuse[56153]: fuse finished with error 0 and tester_r 0 2023-11-06T02:14:27.753 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T02:14:33.855 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi049.front.sepia.ceph.com 2023-11-06T02:14:33.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:14:33.856 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-06T02:14:33.864 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T02:14:34.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.046+0000 7fe30c732640 1 Processor -- start 2023-11-06T02:14:34.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.046+0000 7fe30c732640 1 -- start start 2023-11-06T02:14:34.043 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.046+0000 7fe30c732640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe3040cda70 0x7fe304161650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:34.043 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.046+0000 7fe30c732640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fe3040714a0 0x7fe3040718b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:34.043 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.046+0000 7fe30c732640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3040d17c0 con 0x7fe3040b1f90 2023-11-06T02:14:34.043 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.046+0000 7fe30c732640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3040cdf20 con 0x7fe3040cda70 2023-11-06T02:14:34.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.046+0000 7fe30c732640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe304161b90 con 0x7fe3040714a0 2023-11-06T02:14:34.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.046+0000 7fe309ccd640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe3040cda70 0x7fe304161650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:34.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.046+0000 7fe30a4ce640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fe3040b1f90 0x7fe3040d1c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:60616/0 (socket says 172.21.15.49:60616) 2023-11-06T02:14:34.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.046+0000 7fe30a4ce640 1 -- 172.21.15.49:0/1418994027 learned_addr learned my addr 172.21.15.49:0/1418994027 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:14:34.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.046+0000 7fe30accf640 1 --2- 172.21.15.49:0/1418994027 >> v2:172.21.15.150:3300/0 conn(0x7fe3040714a0 0x7fe3040718b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:34.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.046+0000 7fe309ccd640 1 -- 172.21.15.49:0/1418994027 >> v2:172.21.15.150:3300/0 conn(0x7fe3040714a0 msgr2=0x7fe3040718b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:34.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.046+0000 7fe309ccd640 1 --2- 172.21.15.49:0/1418994027 >> v2:172.21.15.150:3300/0 conn(0x7fe3040714a0 0x7fe3040718b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:34.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.046+0000 7fe309ccd640 1 -- 172.21.15.49:0/1418994027 >> v1:172.21.15.49:6789/0 conn(0x7fe3040b1f90 legacy=0x7fe3040d1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:34.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.046+0000 7fe309ccd640 1 -- 172.21.15.49:0/1418994027 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe304071df0 con 0x7fe3040cda70 2023-11-06T02:14:34.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.050+0000 7fe309ccd640 1 --2- 172.21.15.49:0/1418994027 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3040cda70 0x7fe304161650 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fe3040b2370 tx=0x7fe2f801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b78dfe123811b252 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:34.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.050+0000 7fe3094cc640 1 -- 172.21.15.49:0/1418994027 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f8004230 con 0x7fe3040cda70 2023-11-06T02:14:34.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.050+0000 7fe3094cc640 1 -- 172.21.15.49:0/1418994027 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2f80a7050 con 0x7fe3040cda70 2023-11-06T02:14:34.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.050+0000 7fe3094cc640 1 -- 172.21.15.49:0/1418994027 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f8003620 con 0x7fe3040cda70 2023-11-06T02:14:34.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.050+0000 7fe30c732640 1 -- 172.21.15.49:0/1418994027 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3040cda70 msgr2=0x7fe304161650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:34.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.050+0000 7fe30c732640 1 --2- 172.21.15.49:0/1418994027 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3040cda70 0x7fe304161650 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fe3040b2370 tx=0x7fe2f801c460 comp rx=0 tx=0).stop 2023-11-06T02:14:34.054 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.050+0000 7fe30c732640 1 -- 172.21.15.49:0/1418994027 shutdown_connections 2023-11-06T02:14:34.054 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.050+0000 7fe30c732640 1 --2- 172.21.15.49:0/1418994027 >> v2:172.21.15.150:3300/0 conn(0x7fe3040714a0 0x7fe3040718b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:34.054 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.050+0000 7fe30c732640 1 --2- 172.21.15.49:0/1418994027 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3040cda70 0x7fe304161650 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:34.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.050+0000 7fe30c732640 1 -- 172.21.15.49:0/1418994027 >> 172.21.15.49:0/1418994027 conn(0x7fe3040d2aa0 msgr2=0x7fe3040d2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:34.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.050+0000 7fe30c732640 1 -- 172.21.15.49:0/1418994027 shutdown_connections 2023-11-06T02:14:34.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.050+0000 7fe30c732640 1 -- 172.21.15.49:0/1418994027 wait complete. 2023-11-06T02:14:34.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.050+0000 7fe30c732640 1 Processor -- start 2023-11-06T02:14:34.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.050+0000 7fe30c732640 1 -- start start 2023-11-06T02:14:34.057 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.054+0000 7fe30c732640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe3040714a0 0x7fe3041f5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:34.057 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.054+0000 7fe30c732640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3040b1f90 0x7fe304157b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:34.057 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.054+0000 7fe30c732640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe3040cda70 0x7fe3041685f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:34.058 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.054+0000 7fe30c732640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3040ca900 con 0x7fe3040b1f90 2023-11-06T02:14:34.058 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.054+0000 7fe30c732640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3040c9c80 con 0x7fe3040714a0 2023-11-06T02:14:34.058 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.054+0000 7fe30c732640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3040c9f60 con 0x7fe3040cda70 2023-11-06T02:14:34.058 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.054+0000 7fe309ccd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3040b1f90 0x7fe304157b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:34.058 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.054+0000 7fe309ccd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3040b1f90 0x7fe304157b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37940/0 (socket says 172.21.15.49:37940) 2023-11-06T02:14:34.059 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.054+0000 7fe309ccd640 1 -- 172.21.15.49:0/2595357165 learned_addr learned my addr 172.21.15.49:0/2595357165 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:34.059 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.054+0000 7fe30a4ce640 1 --2- 172.21.15.49:0/2595357165 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe3040714a0 0x7fe3041f5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:34.059 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.054+0000 7fe30accf640 1 --2- 172.21.15.49:0/2595357165 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe3040cda70 0x7fe3041685f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:34.059 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.054+0000 7fe309ccd640 1 -- 172.21.15.49:0/2595357165 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe3040cda70 msgr2=0x7fe3041685f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:34.059 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.054+0000 7fe309ccd640 1 --2- 172.21.15.49:0/2595357165 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe3040cda70 0x7fe3041685f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:34.060 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.054+0000 7fe309ccd640 1 -- 172.21.15.49:0/2595357165 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe3040714a0 msgr2=0x7fe3041f5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:34.060 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.054+0000 7fe309ccd640 1 --2- 172.21.15.49:0/2595357165 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe3040714a0 0x7fe3041f5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:34.060 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.054+0000 7fe309ccd640 1 -- 172.21.15.49:0/2595357165 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2f8009420 con 0x7fe3040b1f90 2023-11-06T02:14:34.060 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.054+0000 7fe30accf640 1 --2- 172.21.15.49:0/2595357165 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe3040cda70 0x7fe3041685f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:14:34.060 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.054+0000 7fe30a4ce640 1 --2- 172.21.15.49:0/2595357165 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe3040714a0 0x7fe3041f5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:14:34.061 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.054+0000 7fe309ccd640 1 --2- 172.21.15.49:0/2595357165 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3040b1f90 0x7fe304157b10 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7fe2f80ae040 tx=0x7fe2f8019200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:34.061 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.054+0000 7fe2f6ffd640 1 -- 172.21.15.49:0/2595357165 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f8003b70 con 0x7fe3040b1f90 2023-11-06T02:14:34.062 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.054+0000 7fe2f6ffd640 1 -- 172.21.15.49:0/2595357165 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2f8003cd0 con 0x7fe3040b1f90 2023-11-06T02:14:34.062 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.054+0000 7fe30c732640 1 -- 172.21.15.49:0/2595357165 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3040ca240 con 0x7fe3040b1f90 2023-11-06T02:14:34.062 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.054+0000 7fe2f6ffd640 1 -- 172.21.15.49:0/2595357165 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f80ab680 con 0x7fe3040b1f90 2023-11-06T02:14:34.063 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.054+0000 7fe30c732640 1 -- 172.21.15.49:0/2595357165 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3040cc580 con 0x7fe3040b1f90 2023-11-06T02:14:34.063 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.054+0000 7fe30c732640 1 -- 172.21.15.49:0/2595357165 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3040caa40 con 0x7fe3040b1f90 2023-11-06T02:14:34.064 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.054+0000 7fe2f6ffd640 1 -- 172.21.15.49:0/2595357165 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fe2f80b2040 con 0x7fe3040b1f90 2023-11-06T02:14:34.064 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.054+0000 7fe2f6ffd640 1 --2- 172.21.15.49:0/2595357165 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe2e8065a60 0x7fe2e8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:34.064 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.054+0000 7fe2f6ffd640 1 -- 172.21.15.49:0/2595357165 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6148+0+0 (secure 0 0 0) 0x7fe2f811d230 con 0x7fe3040b1f90 2023-11-06T02:14:34.065 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.054+0000 7fe30a4ce640 1 --2- 172.21.15.49:0/2595357165 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe2e8065a60 0x7fe2e8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:34.065 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.058+0000 7fe30a4ce640 1 --2- 172.21.15.49:0/2595357165 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe2e8065a60 0x7fe2e8067f20 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fe300005c10 tx=0x7fe300009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:34.066 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.058+0000 7fe2f6ffd640 1 -- 172.21.15.49:0/2595357165 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fe3040caa40 con 0x7fe3040b1f90 2023-11-06T02:14:34.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.250+0000 7fe30c732640 1 -- 172.21.15.49:0/2595357165 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe3040caa40 con 0x7fe3040b1f90 2023-11-06T02:14:34.246 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.250+0000 7fe2f6ffd640 1 -- 172.21.15.49:0/2595357165 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+4651 (secure 0 0 0) 0x7fe3040caa40 con 0x7fe3040b1f90 2023-11-06T02:14:34.247 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:34.247 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":52,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5714,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/315581318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":315581318},{"type":"v1","addr":"172.21.15.150:6837","nonce":315581318}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41},{"gid":6121,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/136800245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":136800245},{"type":"v1","addr":"172.21.15.150:6835","nonce":136800245}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41},{"gid":7110,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/4259579254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":4259579254},{"type":"v1","addr":"172.21.15.49:6833","nonce":4259579254}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T02:13:33.625701+0000","modified":"2023-11-06T02:14:17.377756+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7533":{"gid":7533,"name":"c","rank":0,"incarnation":49,"state":"up:active","state_seq":4,"addr":"172.21.15.49:6835/4037104163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":4037104163},{"type":"v1","addr":"172.21.15.49:6835","nonce":4037104163}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-06T02:14:34.248 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 52 2023-11-06T02:14:34.248 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.254+0000 7fe30c732640 1 -- 172.21.15.49:0/2595357165 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe2e8065a60 msgr2=0x7fe2e8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:34.249 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.254+0000 7fe30c732640 1 --2- 172.21.15.49:0/2595357165 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe2e8065a60 0x7fe2e8067f20 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fe300005c10 tx=0x7fe300009040 comp rx=0 tx=0).stop 2023-11-06T02:14:34.249 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.254+0000 7fe30c732640 1 -- 172.21.15.49:0/2595357165 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3040b1f90 msgr2=0x7fe304157b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:34.249 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.254+0000 7fe30c732640 1 --2- 172.21.15.49:0/2595357165 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3040b1f90 0x7fe304157b10 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7fe2f80ae040 tx=0x7fe2f8019200 comp rx=0 tx=0).stop 2023-11-06T02:14:34.249 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.254+0000 7fe30c732640 1 -- 172.21.15.49:0/2595357165 shutdown_connections 2023-11-06T02:14:34.249 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.254+0000 7fe30c732640 1 --2- 172.21.15.49:0/2595357165 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe2e8065a60 0x7fe2e8067f20 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:34.250 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.254+0000 7fe30c732640 1 --2- 172.21.15.49:0/2595357165 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3040b1f90 0x7fe304157b10 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:34.250 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.254+0000 7fe30c732640 1 --2- 172.21.15.49:0/2595357165 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe3040cda70 0x7fe3041685f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:34.250 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.254+0000 7fe30c732640 1 --2- 172.21.15.49:0/2595357165 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe3040714a0 0x7fe3041f5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:34.250 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.254+0000 7fe30c732640 1 -- 172.21.15.49:0/2595357165 >> 172.21.15.49:0/2595357165 conn(0x7fe3040d2aa0 msgr2=0x7fe3040c82e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:34.250 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.254+0000 7fe30c732640 1 -- 172.21.15.49:0/2595357165 shutdown_connections 2023-11-06T02:14:34.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.254+0000 7fe30c732640 1 -- 172.21.15.49:0/2595357165 wait complete. 2023-11-06T02:14:34.265 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json tag path /parent mytag123 2023-11-06T02:14:34.406 INFO:teuthology.orchestra.run.smithi049.stdout:{"return_code":0,"scrub_tag":"mytag123","mode":"asynchronous"} 2023-11-06T02:14:34.407 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mode": "asynchronous", "return_code": 0, "scrub_tag": "mytag123" } 2023-11-06T02:14:34.407 DEBUG:teuthology.orchestra.run.smithi049:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f7.00000000 scrub_tag 2023-11-06T02:14:34.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.450+0000 7fefc133e080 1 Processor -- start 2023-11-06T02:14:34.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.450+0000 7fefc133e080 1 -- start start 2023-11-06T02:14:34.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.450+0000 7fefc133e080 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f5d0a45e20 0x55f5d0ad45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:34.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.450+0000 7fefc133e080 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f5d0a46200 0x55f5d0adac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:34.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.450+0000 7fefc133e080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f5d0adc9a0 0x55f5d0ad7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:34.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.450+0000 7fefc133e080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5d094c450 con 0x55f5d0adc9a0 2023-11-06T02:14:34.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.450+0000 7fefc133e080 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5d094c150 con 0x55f5d0a46200 2023-11-06T02:14:34.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.450+0000 7fefc133e080 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f5d0942dc0 con 0x55f5d0a45e20 2023-11-06T02:14:34.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.454+0000 7fefbf659640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f5d0adc9a0 0x55f5d0ad7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:34.451 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.454+0000 7fefbf659640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f5d0adc9a0 0x55f5d0ad7ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37946/0 (socket says 172.21.15.49:37946) 2023-11-06T02:14:34.451 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.454+0000 7fefbf659640 1 -- 172.21.15.49:0/3655470957 learned_addr learned my addr 172.21.15.49:0/3655470957 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:34.451 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.454+0000 7fefbe657640 1 --2- 172.21.15.49:0/3655470957 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f5d0a46200 0x55f5d0adac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:34.451 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.454+0000 7fefbee58640 1 --2- 172.21.15.49:0/3655470957 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f5d0a45e20 0x55f5d0ad45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:34.451 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.454+0000 7fefbf659640 1 -- 172.21.15.49:0/3655470957 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f5d0a45e20 msgr2=0x55f5d0ad45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:34.451 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.454+0000 7fefbf659640 1 --2- 172.21.15.49:0/3655470957 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f5d0a45e20 0x55f5d0ad45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:34.452 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.454+0000 7fefbf659640 1 -- 172.21.15.49:0/3655470957 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f5d0a46200 msgr2=0x55f5d0adac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:34.452 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.454+0000 7fefbf659640 1 --2- 172.21.15.49:0/3655470957 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f5d0a46200 0x55f5d0adac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:34.452 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.454+0000 7fefbf659640 1 -- 172.21.15.49:0/3655470957 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefb000a1c0 con 0x55f5d0adc9a0 2023-11-06T02:14:34.452 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.454+0000 7fefbf659640 1 --2- 172.21.15.49:0/3655470957 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f5d0adc9a0 0x55f5d0ad7ef0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fefb000a910 tx=0x7fefb000add0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14e584dbfd5552ae server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:34.452 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.454+0000 7fefbd655640 1 -- 172.21.15.49:0/3655470957 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefb000a400 con 0x55f5d0adc9a0 2023-11-06T02:14:34.453 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.454+0000 7fefbd655640 1 -- 172.21.15.49:0/3655470957 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefb000d960 con 0x55f5d0adc9a0 2023-11-06T02:14:34.453 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.454+0000 7fefbd655640 1 -- 172.21.15.49:0/3655470957 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefb000dcf0 con 0x55f5d0adc9a0 2023-11-06T02:14:34.453 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.454+0000 7fefc133e080 1 -- 172.21.15.49:0/3655470957 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f5d0adc9a0 msgr2=0x55f5d0ad7ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:34.453 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.454+0000 7fefc133e080 1 --2- 172.21.15.49:0/3655470957 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f5d0adc9a0 0x55f5d0ad7ef0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fefb000a910 tx=0x7fefb000add0 comp rx=0 tx=0).stop 2023-11-06T02:14:34.453 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.454+0000 7fefc133e080 1 -- 172.21.15.49:0/3655470957 shutdown_connections 2023-11-06T02:14:34.454 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.454+0000 7fefc133e080 1 --2- 172.21.15.49:0/3655470957 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f5d0adc9a0 0x55f5d0ad7ef0 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:34.454 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.454+0000 7fefc133e080 1 --2- 172.21.15.49:0/3655470957 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f5d0a46200 0x55f5d0adac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:34.454 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.454+0000 7fefc133e080 1 --2- 172.21.15.49:0/3655470957 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f5d0a45e20 0x55f5d0ad45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:34.454 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.454+0000 7fefc133e080 1 -- 172.21.15.49:0/3655470957 >> 172.21.15.49:0/3655470957 conn(0x55f5d0a30cf0 msgr2=0x55f5d0adfb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:34.454 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.454+0000 7fefc133e080 1 -- 172.21.15.49:0/3655470957 shutdown_connections 2023-11-06T02:14:34.455 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.454+0000 7fefc133e080 1 -- 172.21.15.49:0/3655470957 wait complete. 2023-11-06T02:14:34.455 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.454+0000 7fefc133e080 1 Processor -- start 2023-11-06T02:14:34.455 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.454+0000 7fefc133e080 1 -- start start 2023-11-06T02:14:34.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T02:14:34.459 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.454+0000 7fefc133e080 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f5d0a45e20 0x55f5d0ae6f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:34.460 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.454+0000 7fefc133e080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f5d0a46200 0x55f5d0ae0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:34.460 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.454+0000 7fefc133e080 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f5d0adc9a0 0x55f5d0ad2f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:34.460 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.454+0000 7fefc133e080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5d0a494e0 con 0x55f5d0a46200 2023-11-06T02:14:34.460 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.454+0000 7fefc133e080 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5d0adfc00 con 0x55f5d0a45e20 2023-11-06T02:14:34.460 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.454+0000 7fefc133e080 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f5d0a49760 con 0x55f5d0adc9a0 2023-11-06T02:14:34.461 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.454+0000 7fefbe657640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f5d0a46200 0x55f5d0ae0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:34.461 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.454+0000 7fefbe657640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f5d0a46200 0x55f5d0ae0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37954/0 (socket says 172.21.15.49:37954) 2023-11-06T02:14:34.461 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.454+0000 7fefbe657640 1 -- 172.21.15.49:0/3390155606 learned_addr learned my addr 172.21.15.49:0/3390155606 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:34.461 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.454+0000 7fefbf659640 1 --2- 172.21.15.49:0/3390155606 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f5d0adc9a0 0x55f5d0ad2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:34.461 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.454+0000 7fefbee58640 1 --2- 172.21.15.49:0/3390155606 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f5d0a45e20 0x55f5d0ae6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:34.462 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.454+0000 7fefbe657640 1 -- 172.21.15.49:0/3390155606 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f5d0adc9a0 msgr2=0x55f5d0ad2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:34.462 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.454+0000 7fefbe657640 1 --2- 172.21.15.49:0/3390155606 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f5d0adc9a0 0x55f5d0ad2f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:34.462 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.454+0000 7fefbe657640 1 -- 172.21.15.49:0/3390155606 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f5d0a45e20 msgr2=0x55f5d0ae6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:34.462 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.454+0000 7fefbe657640 1 --2- 172.21.15.49:0/3390155606 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f5d0a45e20 0x55f5d0ae6f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:34.462 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.454+0000 7fefbe657640 1 -- 172.21.15.49:0/3390155606 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefb000a560 con 0x55f5d0a46200 2023-11-06T02:14:34.463 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.454+0000 7fefbf659640 1 --2- 172.21.15.49:0/3390155606 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55f5d0adc9a0 0x55f5d0ad2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:14:34.463 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.454+0000 7fefbee58640 1 --2- 172.21.15.49:0/3390155606 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f5d0a45e20 0x55f5d0ae6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:14:34.463 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.454+0000 7fefbe657640 1 --2- 172.21.15.49:0/3390155606 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f5d0a46200 0x55f5d0ae0460 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fefb4019920 tx=0x7fefb4018010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:34.463 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.454+0000 7fefa7fff640 1 -- 172.21.15.49:0/3390155606 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefb40b5680 con 0x55f5d0a46200 2023-11-06T02:14:34.463 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.458+0000 7fefa7fff640 1 -- 172.21.15.49:0/3390155606 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefb40b57e0 con 0x55f5d0a46200 2023-11-06T02:14:34.464 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.458+0000 7fefc133e080 1 -- 172.21.15.49:0/3390155606 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefb4005d30 con 0x55f5d0a46200 2023-11-06T02:14:34.464 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.458+0000 7fefa7fff640 1 -- 172.21.15.49:0/3390155606 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefb40b5b70 con 0x55f5d0a46200 2023-11-06T02:14:34.464 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.458+0000 7fefc133e080 1 -- 172.21.15.49:0/3390155606 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f5d095aff0 con 0x55f5d0a46200 2023-11-06T02:14:34.464 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.458+0000 7fefa7fff640 1 -- 172.21.15.49:0/3390155606 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fefb40b1040 con 0x55f5d0a46200 2023-11-06T02:14:34.464 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.458+0000 7fefa7fff640 1 --2- 172.21.15.49:0/3390155606 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fefac064440 0x7fefac066900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:34.465 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.458+0000 7fefa7fff640 1 -- 172.21.15.49:0/3390155606 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6148+0+0 (secure 0 0 0) 0x7fefb411cda0 con 0x55f5d0a46200 2023-11-06T02:14:34.465 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.458+0000 7fefc133e080 1 --2- 172.21.15.49:0/3390155606 >> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] conn(0x55f5d0ae09a0 0x55f5d0a4a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:34.465 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.458+0000 7fefc133e080 1 -- 172.21.15.49:0/3390155606 --> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] -- osd_op(unknown.0.0:1 12.36 12:6fe38dc2:::100000001f7.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81) v8 -- 0x55f5d0aded30 con 0x55f5d0ae09a0 2023-11-06T02:14:34.465 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.458+0000 7fefbee58640 1 --2- 172.21.15.49:0/3390155606 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fefac064440 0x7fefac066900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:34.465 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.458+0000 7fefbf659640 1 --2- 172.21.15.49:0/3390155606 >> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] conn(0x55f5d0ae09a0 0x55f5d0a4a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:34.466 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.458+0000 7fefbee58640 1 --2- 172.21.15.49:0/3390155606 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fefac064440 0x7fefac066900 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fefb8002fc0 tx=0x7fefb80025c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:34.466 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.458+0000 7fefbf659640 1 --2- 172.21.15.49:0/3390155606 >> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] conn(0x55f5d0ae09a0 0x55f5d0a4a650 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:34.466 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.458+0000 7fefbf659640 1 -- 172.21.15.49:0/3390155606 <== osd.5 v2:172.21.15.150:6800/38418 1 ==== osd_op_reply(1 100000001f7.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x7fefb0018070 con 0x55f5d0ae09a0 2023-11-06T02:14:34.466 INFO:teuthology.orchestra.run.smithi049.stderr:error getting xattr cephfs_data/100000001f7.00000000/scrub_tag: (61) No data available 2023-11-06T02:14:34.467 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.462+0000 7fefc133e080 1 -- 172.21.15.49:0/3390155606 >> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] conn(0x55f5d0ae09a0 msgr2=0x55f5d0a4a650 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:34.467 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.462+0000 7fefc133e080 1 --2- 172.21.15.49:0/3390155606 >> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] conn(0x55f5d0ae09a0 0x55f5d0a4a650 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:34.467 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.462+0000 7fefc133e080 1 -- 172.21.15.49:0/3390155606 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fefac064440 msgr2=0x7fefac066900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:34.467 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.462+0000 7fefc133e080 1 --2- 172.21.15.49:0/3390155606 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fefac064440 0x7fefac066900 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fefb8002fc0 tx=0x7fefb80025c0 comp rx=0 tx=0).stop 2023-11-06T02:14:34.467 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.462+0000 7fefc133e080 1 -- 172.21.15.49:0/3390155606 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f5d0a46200 msgr2=0x55f5d0ae0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:34.468 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.462+0000 7fefc133e080 1 --2- 172.21.15.49:0/3390155606 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f5d0a46200 0x55f5d0ae0460 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fefb4019920 tx=0x7fefb4018010 comp rx=0 tx=0).stop 2023-11-06T02:14:34.468 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.462+0000 7fefbf659640 1 -- 172.21.15.49:0/3390155606 reap_dead start 2023-11-06T02:14:34.468 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.462+0000 7fefc133e080 1 -- 172.21.15.49:0/3390155606 shutdown_connections 2023-11-06T02:14:34.468 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.462+0000 7fefc133e080 1 -- 172.21.15.49:0/3390155606 >> 172.21.15.49:0/3390155606 conn(0x55f5d0a30cf0 msgr2=0x55f5d0adaff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:34.468 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.462+0000 7fefc133e080 1 -- 172.21.15.49:0/3390155606 shutdown_connections 2023-11-06T02:14:34.469 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.462+0000 7fefc133e080 1 -- 172.21.15.49:0/3390155606 wait complete. 2023-11-06T02:14:34.469 DEBUG:teuthology.orchestra.run.smithi049:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000002.00000000 scrub_tag 2023-11-06T02:14:34.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.550+0000 7f73ecab5080 1 Processor -- start 2023-11-06T02:14:34.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.550+0000 7f73ecab5080 1 -- start start 2023-11-06T02:14:34.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.550+0000 7f73ecab5080 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a5a2b75a80 0x55a5a2c27670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:34.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.550+0000 7f73ecab5080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a5a2b93830 0x55a5a2b92d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:34.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.550+0000 7f73ecab5080 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a5a2c02780 0x55a5a2b78b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:34.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.550+0000 7f73ecab5080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5a2a91450 con 0x55a5a2b93830 2023-11-06T02:14:34.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.550+0000 7f73ecab5080 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5a2a91150 con 0x55a5a2b75a80 2023-11-06T02:14:34.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.550+0000 7f73ecab5080 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a5a2a87dc0 con 0x55a5a2c02780 2023-11-06T02:14:34.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.550+0000 7f73e9dce640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a5a2b93830 0x55a5a2b92d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:34.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.550+0000 7f73e9dce640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a5a2b93830 0x55a5a2b92d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37984/0 (socket says 172.21.15.49:37984) 2023-11-06T02:14:34.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.550+0000 7f73e9dce640 1 -- 172.21.15.49:0/2820661489 learned_addr learned my addr 172.21.15.49:0/2820661489 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:34.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.550+0000 7f73e9dce640 1 -- 172.21.15.49:0/2820661489 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a5a2c02780 msgr2=0x55a5a2b78b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:34.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.550+0000 7f73e9dce640 1 --2- 172.21.15.49:0/2820661489 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a5a2c02780 0x55a5a2b78b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:34.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.550+0000 7f73e9dce640 1 -- 172.21.15.49:0/2820661489 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a5a2b75a80 msgr2=0x55a5a2c27670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:34.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.550+0000 7f73e9dce640 1 --2- 172.21.15.49:0/2820661489 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a5a2b75a80 0x55a5a2c27670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:34.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.550+0000 7f73e9dce640 1 -- 172.21.15.49:0/2820661489 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73dc005df0 con 0x55a5a2b93830 2023-11-06T02:14:34.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.550+0000 7f73e9dce640 1 --2- 172.21.15.49:0/2820661489 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a5a2b93830 0x55a5a2b92d30 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f73dc0ad040 tx=0x7f73dc003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=102d0b0a39aba122 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:34.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.550+0000 7f73e8dcc640 1 -- 172.21.15.49:0/2820661489 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73dc0aa680 con 0x55a5a2b93830 2023-11-06T02:14:34.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.550+0000 7f73e8dcc640 1 -- 172.21.15.49:0/2820661489 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73dc0aa7e0 con 0x55a5a2b93830 2023-11-06T02:14:34.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.550+0000 7f73e8dcc640 1 -- 172.21.15.49:0/2820661489 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73dc0aab50 con 0x55a5a2b93830 2023-11-06T02:14:34.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.550+0000 7f73ecab5080 1 -- 172.21.15.49:0/2820661489 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a5a2b93830 msgr2=0x55a5a2b92d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:34.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.550+0000 7f73ecab5080 1 --2- 172.21.15.49:0/2820661489 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a5a2b93830 0x55a5a2b92d30 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f73dc0ad040 tx=0x7f73dc003030 comp rx=0 tx=0).stop 2023-11-06T02:14:34.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.550+0000 7f73ecab5080 1 -- 172.21.15.49:0/2820661489 shutdown_connections 2023-11-06T02:14:34.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.550+0000 7f73ecab5080 1 --2- 172.21.15.49:0/2820661489 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a5a2b93830 0x55a5a2b92d30 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:34.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.550+0000 7f73ecab5080 1 --2- 172.21.15.49:0/2820661489 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a5a2c02780 0x55a5a2b78b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:34.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.550+0000 7f73ecab5080 1 --2- 172.21.15.49:0/2820661489 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a5a2b75a80 0x55a5a2c27670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:34.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.550+0000 7f73ecab5080 1 -- 172.21.15.49:0/2820661489 >> 172.21.15.49:0/2820661489 conn(0x55a5a2b74660 msgr2=0x55a5a2b4cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:34.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.550+0000 7f73ecab5080 1 -- 172.21.15.49:0/2820661489 shutdown_connections 2023-11-06T02:14:34.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.550+0000 7f73ecab5080 1 -- 172.21.15.49:0/2820661489 wait complete. 2023-11-06T02:14:34.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.550+0000 7f73ecab5080 1 Processor -- start 2023-11-06T02:14:34.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.550+0000 7f73ecab5080 1 -- start start 2023-11-06T02:14:34.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.554+0000 7f73ecab5080 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a5a2b75a80 0x55a5a2c29800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:34.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.554+0000 7f73ecab5080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a5a2b93830 0x55a5a2c21830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:34.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.554+0000 7f73ecab5080 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a5a2c02780 0x55a5a2b90610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:34.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.554+0000 7f73ecab5080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5a2c29d40 con 0x55a5a2b93830 2023-11-06T02:14:34.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.554+0000 7f73ecab5080 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5a2c22710 con 0x55a5a2c02780 2023-11-06T02:14:34.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.554+0000 7f73ecab5080 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a5a2b6c430 con 0x55a5a2b75a80 2023-11-06T02:14:34.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.554+0000 7f73e9dce640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a5a2b93830 0x55a5a2c21830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:34.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.554+0000 7f73e9dce640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a5a2b93830 0x55a5a2c21830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:37992/0 (socket says 172.21.15.49:37992) 2023-11-06T02:14:34.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.554+0000 7f73e9dce640 1 -- 172.21.15.49:0/3479981495 learned_addr learned my addr 172.21.15.49:0/3479981495 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:34.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.554+0000 7f73ea5cf640 1 --2- 172.21.15.49:0/3479981495 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a5a2b75a80 0x55a5a2c29800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:34.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.554+0000 7f73eadd0640 1 --2- 172.21.15.49:0/3479981495 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a5a2c02780 0x55a5a2b90610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:34.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.554+0000 7f73e9dce640 1 -- 172.21.15.49:0/3479981495 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a5a2b75a80 msgr2=0x55a5a2c29800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:34.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.554+0000 7f73e9dce640 1 --2- 172.21.15.49:0/3479981495 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a5a2b75a80 0x55a5a2c29800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:34.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.554+0000 7f73e9dce640 1 -- 172.21.15.49:0/3479981495 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a5a2c02780 msgr2=0x55a5a2b90610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:34.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.554+0000 7f73e9dce640 1 --2- 172.21.15.49:0/3479981495 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a5a2c02780 0x55a5a2b90610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:34.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.554+0000 7f73e9dce640 1 -- 172.21.15.49:0/3479981495 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a5a2aba190 con 0x55a5a2b93830 2023-11-06T02:14:34.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.554+0000 7f73eadd0640 1 --2- 172.21.15.49:0/3479981495 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a5a2c02780 0x55a5a2b90610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:14:34.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.554+0000 7f73ea5cf640 1 --2- 172.21.15.49:0/3479981495 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55a5a2b75a80 0x55a5a2c29800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:14:34.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.554+0000 7f73e9dce640 1 --2- 172.21.15.49:0/3479981495 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a5a2b93830 0x55a5a2c21830 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f73dc018f10 tx=0x7f73dc018080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:34.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.554+0000 7f73e37fe640 1 -- 172.21.15.49:0/3479981495 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73dc0b4c30 con 0x55a5a2b93830 2023-11-06T02:14:34.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.554+0000 7f73e37fe640 1 -- 172.21.15.49:0/3479981495 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73dc0b4d90 con 0x55a5a2b93830 2023-11-06T02:14:34.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.554+0000 7f73ecab5080 1 -- 172.21.15.49:0/3479981495 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a5a2a9fff0 con 0x55a5a2b93830 2023-11-06T02:14:34.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.554+0000 7f73e37fe640 1 -- 172.21.15.49:0/3479981495 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73dc0c5640 con 0x55a5a2b93830 2023-11-06T02:14:34.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.554+0000 7f73ecab5080 1 -- 172.21.15.49:0/3479981495 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a5a2a93430 con 0x55a5a2b93830 2023-11-06T02:14:34.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.554+0000 7f73e37fe640 1 -- 172.21.15.49:0/3479981495 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f73dc0b1070 con 0x55a5a2b93830 2023-11-06T02:14:34.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.554+0000 7f73e37fe640 1 --2- 172.21.15.49:0/3479981495 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f73cc064410 0x7f73cc0668d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:34.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.554+0000 7f73e37fe640 1 -- 172.21.15.49:0/3479981495 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f73dc11dbb0 con 0x55a5a2b93830 2023-11-06T02:14:34.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.554+0000 7f73ecab5080 1 --2- 172.21.15.49:0/3479981495 >> [v2:172.21.15.49:6824/38115,v1:172.21.15.49:6825/38115] conn(0x55a5a2b6bf90 0x55a5a2cba210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:34.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.554+0000 7f73ecab5080 1 -- 172.21.15.49:0/3479981495 --> [v2:172.21.15.49:6824/38115,v1:172.21.15.49:6825/38115] -- osd_op(unknown.0.0:1 12.12 12:4bec3886:::10000000002.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81) v8 -- 0x55a5a2cba750 con 0x55a5a2b6bf90 2023-11-06T02:14:34.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.554+0000 7f73ea5cf640 1 --2- 172.21.15.49:0/3479981495 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f73cc064410 0x7f73cc0668d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:34.563 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.554+0000 7f73eadd0640 1 --2- 172.21.15.49:0/3479981495 >> [v2:172.21.15.49:6824/38115,v1:172.21.15.49:6825/38115] conn(0x55a5a2b6bf90 0x55a5a2cba210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:34.563 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.558+0000 7f73eadd0640 1 --2- 172.21.15.49:0/3479981495 >> [v2:172.21.15.49:6824/38115,v1:172.21.15.49:6825/38115] conn(0x55a5a2b6bf90 0x55a5a2cba210 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:34.563 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.558+0000 7f73ea5cf640 1 --2- 172.21.15.49:0/3479981495 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f73cc064410 0x7f73cc0668d0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f73e400d380 tx=0x7f73e4007040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:34.563 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.558+0000 7f73eadd0640 1 -- 172.21.15.49:0/3479981495 <== osd.3 v2:172.21.15.49:6824/38115 1 ==== osd_op_reply(1 10000000002.00000000 [getxattr (12) out=12b] v0'0 uv22 ondisk = 0) v8 ==== 164+0+12 (crc 0 0 0) 0x7f73d400f3b0 con 0x55a5a2b6bf90 2023-11-06T02:14:34.563 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.558+0000 7f73ecab5080 1 -- 172.21.15.49:0/3479981495 >> [v2:172.21.15.49:6824/38115,v1:172.21.15.49:6825/38115] conn(0x55a5a2b6bf90 msgr2=0x55a5a2cba210 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:34.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.558+0000 7f73ecab5080 1 --2- 172.21.15.49:0/3479981495 >> [v2:172.21.15.49:6824/38115,v1:172.21.15.49:6825/38115] conn(0x55a5a2b6bf90 0x55a5a2cba210 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:34.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.558+0000 7f73ecab5080 1 -- 172.21.15.49:0/3479981495 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f73cc064410 msgr2=0x7f73cc0668d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:34.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.558+0000 7f73ecab5080 1 --2- 172.21.15.49:0/3479981495 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f73cc064410 0x7f73cc0668d0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f73e400d380 tx=0x7f73e4007040 comp rx=0 tx=0).stop 2023-11-06T02:14:34.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.558+0000 7f73ecab5080 1 -- 172.21.15.49:0/3479981495 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a5a2b93830 msgr2=0x55a5a2c21830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:34.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.558+0000 7f73ecab5080 1 --2- 172.21.15.49:0/3479981495 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a5a2b93830 0x55a5a2c21830 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f73dc018f10 tx=0x7f73dc018080 comp rx=0 tx=0).stop 2023-11-06T02:14:34.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.558+0000 7f73eadd0640 1 -- 172.21.15.49:0/3479981495 reap_dead start 2023-11-06T02:14:34.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.558+0000 7f73ecab5080 1 -- 172.21.15.49:0/3479981495 shutdown_connections 2023-11-06T02:14:34.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.558+0000 7f73ecab5080 1 -- 172.21.15.49:0/3479981495 >> 172.21.15.49:0/3479981495 conn(0x55a5a2b74660 msgr2=0x55a5a2b4cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:34.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.558+0000 7f73ecab5080 1 -- 172.21.15.49:0/3479981495 shutdown_connections 2023-11-06T02:14:34.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.558+0000 7f73ecab5080 1 -- 172.21.15.49:0/3479981495 wait complete. 2023-11-06T02:14:34.566 INFO:teuthology.orchestra.run.smithi049.stdout:mytag123 2023-11-06T02:14:34.567 DEBUG:teuthology.orchestra.run.smithi049:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f8.00000000 scrub_tag 2023-11-06T02:14:34.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.646+0000 7f1cf94c2080 1 Processor -- start 2023-11-06T02:14:34.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.646+0000 7f1cf94c2080 1 -- start start 2023-11-06T02:14:34.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.646+0000 7f1cf94c2080 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559aaf8a5af0 0x559aaf92e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:34.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.646+0000 7f1cf94c2080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559aaf8a5ed0 0x559aaf92f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:34.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.646+0000 7f1cf94c2080 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x559aaf932120 0x559aaf929480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:34.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.646+0000 7f1cf94c2080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x559aaf7a1450 con 0x559aaf8a5ed0 2023-11-06T02:14:34.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.646+0000 7f1cf94c2080 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x559aaf7a1150 con 0x559aaf8a5af0 2023-11-06T02:14:34.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.646+0000 7f1cf94c2080 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x559aaf797dc0 con 0x559aaf932120 2023-11-06T02:14:34.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.646+0000 7f1cf67db640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559aaf8a5ed0 0x559aaf92f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:34.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.646+0000 7f1cf67db640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559aaf8a5ed0 0x559aaf92f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38016/0 (socket says 172.21.15.49:38016) 2023-11-06T02:14:34.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.646+0000 7f1cf67db640 1 -- 172.21.15.49:0/2914808984 learned_addr learned my addr 172.21.15.49:0/2914808984 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:34.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.646+0000 7f1cf77dd640 1 --2- 172.21.15.49:0/2914808984 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x559aaf932120 0x559aaf929480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:34.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.646+0000 7f1cf67db640 1 -- 172.21.15.49:0/2914808984 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x559aaf932120 msgr2=0x559aaf929480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:34.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.646+0000 7f1cf6fdc640 1 --2- 172.21.15.49:0/2914808984 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559aaf8a5af0 0x559aaf92e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:34.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.646+0000 7f1cf67db640 1 --2- 172.21.15.49:0/2914808984 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x559aaf932120 0x559aaf929480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:34.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.646+0000 7f1cf67db640 1 -- 172.21.15.49:0/2914808984 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559aaf8a5af0 msgr2=0x559aaf92e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:34.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.646+0000 7f1cf67db640 1 --2- 172.21.15.49:0/2914808984 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559aaf8a5af0 0x559aaf92e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:34.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.646+0000 7f1cf67db640 1 -- 172.21.15.49:0/2914808984 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ce8005df0 con 0x559aaf8a5ed0 2023-11-06T02:14:34.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.646+0000 7f1cf6fdc640 1 --2- 172.21.15.49:0/2914808984 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559aaf8a5af0 0x559aaf92e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:14:34.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.646+0000 7f1cf67db640 1 --2- 172.21.15.49:0/2914808984 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559aaf8a5ed0 0x559aaf92f1d0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f1cec00a7b0 tx=0x7f1cec00ac70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fd475fcebcd019f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:34.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.646+0000 7f1cf57d9640 1 -- 172.21.15.49:0/2914808984 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cec00f070 con 0x559aaf8a5ed0 2023-11-06T02:14:34.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.646+0000 7f1cf57d9640 1 -- 172.21.15.49:0/2914808984 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cec00ada0 con 0x559aaf8a5ed0 2023-11-06T02:14:34.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.646+0000 7f1cf57d9640 1 -- 172.21.15.49:0/2914808984 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cec008c10 con 0x559aaf8a5ed0 2023-11-06T02:14:34.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.646+0000 7f1cf94c2080 1 -- 172.21.15.49:0/2914808984 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559aaf8a5ed0 msgr2=0x559aaf92f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:34.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.646+0000 7f1cf94c2080 1 --2- 172.21.15.49:0/2914808984 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559aaf8a5ed0 0x559aaf92f1d0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f1cec00a7b0 tx=0x7f1cec00ac70 comp rx=0 tx=0).stop 2023-11-06T02:14:34.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.646+0000 7f1cf94c2080 1 -- 172.21.15.49:0/2914808984 shutdown_connections 2023-11-06T02:14:34.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.646+0000 7f1cf94c2080 1 --2- 172.21.15.49:0/2914808984 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559aaf8a5ed0 0x559aaf92f1d0 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:34.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.646+0000 7f1cf94c2080 1 --2- 172.21.15.49:0/2914808984 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x559aaf932120 0x559aaf929480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:34.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.646+0000 7f1cf94c2080 1 --2- 172.21.15.49:0/2914808984 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559aaf8a5af0 0x559aaf92e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:34.648 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.646+0000 7f1cf94c2080 1 -- 172.21.15.49:0/2914808984 >> 172.21.15.49:0/2914808984 conn(0x559aaf8890e0 msgr2=0x559aaf934520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:34.648 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.646+0000 7f1cf94c2080 1 -- 172.21.15.49:0/2914808984 shutdown_connections 2023-11-06T02:14:34.648 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.646+0000 7f1cf94c2080 1 -- 172.21.15.49:0/2914808984 wait complete. 2023-11-06T02:14:34.648 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.646+0000 7f1cf94c2080 1 Processor -- start 2023-11-06T02:14:34.648 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.646+0000 7f1cf94c2080 1 -- start start 2023-11-06T02:14:34.652 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.650+0000 7f1cf94c2080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559aaf8a5af0 0x559aaf9277d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:34.652 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.650+0000 7f1cf94c2080 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559aaf8a5ed0 0x559aaf929130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:34.652 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.650+0000 7f1cf94c2080 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x559aaf932120 0x559aaf92a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:34.652 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.650+0000 7f1cf94c2080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x559aaf929920 con 0x559aaf8a5af0 2023-11-06T02:14:34.653 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.650+0000 7f1cf94c2080 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x559aaf929bc0 con 0x559aaf8a5ed0 2023-11-06T02:14:34.653 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.650+0000 7f1cf94c2080 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x559aaf929e60 con 0x559aaf932120 2023-11-06T02:14:34.653 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.650+0000 7f1cf6fdc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559aaf8a5af0 0x559aaf9277d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:34.653 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.650+0000 7f1cf6fdc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559aaf8a5af0 0x559aaf9277d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38022/0 (socket says 172.21.15.49:38022) 2023-11-06T02:14:34.653 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.650+0000 7f1cf6fdc640 1 -- 172.21.15.49:0/2106867878 learned_addr learned my addr 172.21.15.49:0/2106867878 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:34.654 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.650+0000 7f1cf67db640 1 --2- 172.21.15.49:0/2106867878 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559aaf8a5ed0 0x559aaf929130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:34.654 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.650+0000 7f1cf77dd640 1 --2- 172.21.15.49:0/2106867878 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x559aaf932120 0x559aaf92a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:34.654 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.650+0000 7f1cf6fdc640 1 -- 172.21.15.49:0/2106867878 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x559aaf932120 msgr2=0x559aaf92a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:34.654 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.650+0000 7f1cf6fdc640 1 --2- 172.21.15.49:0/2106867878 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x559aaf932120 0x559aaf92a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:34.654 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.650+0000 7f1cf6fdc640 1 -- 172.21.15.49:0/2106867878 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559aaf8a5ed0 msgr2=0x559aaf929130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:34.655 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.650+0000 7f1cf6fdc640 1 --2- 172.21.15.49:0/2106867878 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559aaf8a5ed0 0x559aaf929130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:34.655 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.650+0000 7f1cf6fdc640 1 -- 172.21.15.49:0/2106867878 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cec00a430 con 0x559aaf8a5af0 2023-11-06T02:14:34.655 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.650+0000 7f1cf67db640 1 --2- 172.21.15.49:0/2106867878 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559aaf8a5ed0 0x559aaf929130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:14:34.655 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.650+0000 7f1cf6fdc640 1 --2- 172.21.15.49:0/2106867878 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559aaf8a5af0 0x559aaf9277d0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f1cf000c430 tx=0x7f1cf000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:34.655 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.650+0000 7f1cdffff640 1 -- 172.21.15.49:0/2106867878 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf000a070 con 0x559aaf8a5af0 2023-11-06T02:14:34.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.650+0000 7f1cdffff640 1 -- 172.21.15.49:0/2106867878 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cf0010780 con 0x559aaf8a5af0 2023-11-06T02:14:34.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.650+0000 7f1cf94c2080 1 -- 172.21.15.49:0/2106867878 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x559aaf7afff0 con 0x559aaf8a5af0 2023-11-06T02:14:34.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.650+0000 7f1cdffff640 1 -- 172.21.15.49:0/2106867878 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf0010b30 con 0x559aaf8a5af0 2023-11-06T02:14:34.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.650+0000 7f1cf94c2080 1 -- 172.21.15.49:0/2106867878 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559aaf7a3430 con 0x559aaf8a5af0 2023-11-06T02:14:34.657 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.650+0000 7f1cdffff640 1 -- 172.21.15.49:0/2106867878 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f1cf0010c90 con 0x559aaf8a5af0 2023-11-06T02:14:34.657 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.650+0000 7f1cdffff640 1 --2- 172.21.15.49:0/2106867878 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1ce4064440 0x7f1ce4066900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:34.657 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.650+0000 7f1cdffff640 1 -- 172.21.15.49:0/2106867878 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f1cf0082160 con 0x559aaf8a5af0 2023-11-06T02:14:34.657 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.650+0000 7f1cf94c2080 1 --2- 172.21.15.49:0/2106867878 >> [v2:172.21.15.49:6809/38116,v1:172.21.15.49:6811/38116] conn(0x559aaf926d50 0x559aaf921fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:34.658 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.650+0000 7f1cf94c2080 1 -- 172.21.15.49:0/2106867878 --> [v2:172.21.15.49:6809/38116,v1:172.21.15.49:6811/38116] -- osd_op(unknown.0.0:1 12.19 12:997b7129:::100000001f8.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81) v8 -- 0x559aaf9cdb40 con 0x559aaf926d50 2023-11-06T02:14:34.658 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.650+0000 7f1cf67db640 1 --2- 172.21.15.49:0/2106867878 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1ce4064440 0x7f1ce4066900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:34.658 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.650+0000 7f1cf77dd640 1 --2- 172.21.15.49:0/2106867878 >> [v2:172.21.15.49:6809/38116,v1:172.21.15.49:6811/38116] conn(0x559aaf926d50 0x559aaf921fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:34.658 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.650+0000 7f1cf77dd640 1 --2- 172.21.15.49:0/2106867878 >> [v2:172.21.15.49:6809/38116,v1:172.21.15.49:6811/38116] conn(0x559aaf926d50 0x559aaf921fc0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:34.658 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.654+0000 7f1cf67db640 1 --2- 172.21.15.49:0/2106867878 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1ce4064440 0x7f1ce4066900 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f1cec018040 tx=0x7f1cec012000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:34.659 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.654+0000 7f1cf77dd640 1 -- 172.21.15.49:0/2106867878 <== osd.0 v2:172.21.15.49:6809/38116 1 ==== osd_op_reply(1 100000001f8.00000000 [getxattr (12) out=12b] v0'0 uv23 ondisk = 0) v8 ==== 164+0+12 (crc 0 0 0) 0x7f1ce80aa5d0 con 0x559aaf926d50 2023-11-06T02:14:34.659 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.654+0000 7f1cf94c2080 1 -- 172.21.15.49:0/2106867878 >> [v2:172.21.15.49:6809/38116,v1:172.21.15.49:6811/38116] conn(0x559aaf926d50 msgr2=0x559aaf921fc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:34.659 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.654+0000 7f1cf94c2080 1 --2- 172.21.15.49:0/2106867878 >> [v2:172.21.15.49:6809/38116,v1:172.21.15.49:6811/38116] conn(0x559aaf926d50 0x559aaf921fc0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:34.659 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.654+0000 7f1cf94c2080 1 -- 172.21.15.49:0/2106867878 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1ce4064440 msgr2=0x7f1ce4066900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:34.659 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.654+0000 7f1cf94c2080 1 --2- 172.21.15.49:0/2106867878 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1ce4064440 0x7f1ce4066900 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f1cec018040 tx=0x7f1cec012000 comp rx=0 tx=0).stop 2023-11-06T02:14:34.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.654+0000 7f1cf94c2080 1 -- 172.21.15.49:0/2106867878 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559aaf8a5af0 msgr2=0x559aaf9277d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:34.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.654+0000 7f1cf94c2080 1 --2- 172.21.15.49:0/2106867878 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559aaf8a5af0 0x559aaf9277d0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f1cf000c430 tx=0x7f1cf000c8f0 comp rx=0 tx=0).stop 2023-11-06T02:14:34.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.654+0000 7f1cf77dd640 1 -- 172.21.15.49:0/2106867878 reap_dead start 2023-11-06T02:14:34.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.654+0000 7f1cf94c2080 1 -- 172.21.15.49:0/2106867878 shutdown_connections 2023-11-06T02:14:34.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.654+0000 7f1cf94c2080 1 -- 172.21.15.49:0/2106867878 >> 172.21.15.49:0/2106867878 conn(0x559aaf8890e0 msgr2=0x559aaf927d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:34.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.654+0000 7f1cf94c2080 1 -- 172.21.15.49:0/2106867878 shutdown_connections 2023-11-06T02:14:34.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.654+0000 7f1cf94c2080 1 -- 172.21.15.49:0/2106867878 wait complete. 2023-11-06T02:14:34.662 INFO:teuthology.orchestra.run.smithi049.stdout:mytag123 2023-11-06T02:14:34.662 DEBUG:teuthology.orchestra.run.smithi049:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000004.00000000 scrub_tag 2023-11-06T02:14:34.745 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.750+0000 7f863e647080 1 Processor -- start 2023-11-06T02:14:34.745 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.750+0000 7f863e647080 1 -- start start 2023-11-06T02:14:34.747 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.750+0000 7f863e647080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5578441f3b00 0x5578441c2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:34.747 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.750+0000 7f863e647080 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5578441f6480 0x557844298160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:34.748 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.750+0000 7f863e647080 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55784429b8a0 0x5578442986a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:34.748 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.750+0000 7f863e647080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55784410b450 con 0x5578441f3b00 2023-11-06T02:14:34.748 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.750+0000 7f863e647080 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55784410b150 con 0x5578441f6480 2023-11-06T02:14:34.748 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.750+0000 7f863e647080 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x557844101dc0 con 0x55784429b8a0 2023-11-06T02:14:34.748 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.750+0000 7f863c161640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5578441f3b00 0x5578441c2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:34.749 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.750+0000 7f863c161640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5578441f3b00 0x5578441c2900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38028/0 (socket says 172.21.15.49:38028) 2023-11-06T02:14:34.749 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.750+0000 7f863c161640 1 -- 172.21.15.49:0/718316681 learned_addr learned my addr 172.21.15.49:0/718316681 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:34.749 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.750+0000 7f863c962640 1 --2- 172.21.15.49:0/718316681 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55784429b8a0 0x5578442986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:34.749 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.750+0000 7f863b960640 1 --2- 172.21.15.49:0/718316681 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5578441f6480 0x557844298160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:34.749 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.750+0000 7f863c161640 1 -- 172.21.15.49:0/718316681 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55784429b8a0 msgr2=0x5578442986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:34.750 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.750+0000 7f863c161640 1 --2- 172.21.15.49:0/718316681 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55784429b8a0 0x5578442986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:34.750 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.750+0000 7f863c161640 1 -- 172.21.15.49:0/718316681 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5578441f6480 msgr2=0x557844298160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:34.750 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.750+0000 7f863c161640 1 --2- 172.21.15.49:0/718316681 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5578441f6480 0x557844298160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:34.750 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.750+0000 7f863c161640 1 -- 172.21.15.49:0/718316681 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8634005d30 con 0x5578441f3b00 2023-11-06T02:14:34.750 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.750+0000 7f863b960640 1 --2- 172.21.15.49:0/718316681 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5578441f6480 0x557844298160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:14:34.753 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.750+0000 7f863c161640 1 --2- 172.21.15.49:0/718316681 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5578441f3b00 0x5578441c2900 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f863400a500 tx=0x7f8634003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7b540f03aa3552f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:34.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.750+0000 7f863a95e640 1 -- 172.21.15.49:0/718316681 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86340b4610 con 0x5578441f3b00 2023-11-06T02:14:34.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.750+0000 7f863a95e640 1 -- 172.21.15.49:0/718316681 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86340b4770 con 0x5578441f3b00 2023-11-06T02:14:34.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.750+0000 7f863a95e640 1 -- 172.21.15.49:0/718316681 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86340b4b20 con 0x5578441f3b00 2023-11-06T02:14:34.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.754+0000 7f863e647080 1 -- 172.21.15.49:0/718316681 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5578441f3b00 msgr2=0x5578441c2900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:34.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.754+0000 7f863e647080 1 --2- 172.21.15.49:0/718316681 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5578441f3b00 0x5578441c2900 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f863400a500 tx=0x7f8634003030 comp rx=0 tx=0).stop 2023-11-06T02:14:34.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.754+0000 7f863e647080 1 -- 172.21.15.49:0/718316681 shutdown_connections 2023-11-06T02:14:34.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.754+0000 7f863e647080 1 --2- 172.21.15.49:0/718316681 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55784429b8a0 0x5578442986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:34.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.754+0000 7f863e647080 1 --2- 172.21.15.49:0/718316681 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5578441f6480 0x557844298160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:34.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.754+0000 7f863e647080 1 --2- 172.21.15.49:0/718316681 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5578441f3b00 0x5578441c2900 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:34.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.754+0000 7f863e647080 1 -- 172.21.15.49:0/718316681 >> 172.21.15.49:0/718316681 conn(0x5578441f21a0 msgr2=0x5578441f5300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:34.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.754+0000 7f863e647080 1 -- 172.21.15.49:0/718316681 shutdown_connections 2023-11-06T02:14:34.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.754+0000 7f863e647080 1 -- 172.21.15.49:0/718316681 wait complete. 2023-11-06T02:14:34.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.754+0000 7f863e647080 1 Processor -- start 2023-11-06T02:14:34.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.754+0000 7f863e647080 1 -- start start 2023-11-06T02:14:34.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.754+0000 7f863e647080 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5578441f3b00 0x55784420d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:34.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.754+0000 7f863e647080 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5578441f6480 0x557844209d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:34.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.754+0000 7f863e647080 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55784429b8a0 0x55784420bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:34.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.754+0000 7f863e647080 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55784420e390 con 0x5578441f3b00 2023-11-06T02:14:34.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.754+0000 7f863e647080 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55784420e0f0 con 0x55784429b8a0 2023-11-06T02:14:34.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.754+0000 7f863e647080 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x557844296010 con 0x5578441f6480 2023-11-06T02:14:34.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.754+0000 7f863c161640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5578441f3b00 0x55784420d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:34.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.754+0000 7f863c161640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5578441f3b00 0x55784420d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38030/0 (socket says 172.21.15.49:38030) 2023-11-06T02:14:34.762 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.754+0000 7f863c161640 1 -- 172.21.15.49:0/1447106034 learned_addr learned my addr 172.21.15.49:0/1447106034 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:34.762 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.754+0000 7f863b960640 1 --2- 172.21.15.49:0/1447106034 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5578441f6480 0x557844209d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:34.762 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.754+0000 7f863c962640 1 --2- 172.21.15.49:0/1447106034 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55784429b8a0 0x55784420bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:34.762 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.754+0000 7f863c161640 1 -- 172.21.15.49:0/1447106034 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5578441f6480 msgr2=0x557844209d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:34.762 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.754+0000 7f863c161640 1 --2- 172.21.15.49:0/1447106034 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5578441f6480 0x557844209d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:34.763 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.754+0000 7f863c161640 1 -- 172.21.15.49:0/1447106034 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55784429b8a0 msgr2=0x55784420bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:34.763 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.754+0000 7f863c161640 1 --2- 172.21.15.49:0/1447106034 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55784429b8a0 0x55784420bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:34.763 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.754+0000 7f863c161640 1 -- 172.21.15.49:0/1447106034 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557844134190 con 0x5578441f3b00 2023-11-06T02:14:34.763 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.754+0000 7f863b960640 1 --2- 172.21.15.49:0/1447106034 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5578441f6480 0x557844209d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:14:34.763 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.754+0000 7f863c962640 1 --2- 172.21.15.49:0/1447106034 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55784429b8a0 0x55784420bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:14:34.764 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.754+0000 7f863c161640 1 --2- 172.21.15.49:0/1447106034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5578441f3b00 0x55784420d770 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f86340199b0 tx=0x7f86340180a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:34.764 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.754+0000 7f863995c640 1 -- 172.21.15.49:0/1447106034 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86340bd660 con 0x5578441f3b00 2023-11-06T02:14:34.764 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.754+0000 7f863995c640 1 -- 172.21.15.49:0/1447106034 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86340bd7c0 con 0x5578441f3b00 2023-11-06T02:14:34.764 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.754+0000 7f863e647080 1 -- 172.21.15.49:0/1447106034 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x557844119ff0 con 0x5578441f3b00 2023-11-06T02:14:34.764 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.754+0000 7f863995c640 1 -- 172.21.15.49:0/1447106034 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86340bdb70 con 0x5578441f3b00 2023-11-06T02:14:34.764 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.754+0000 7f863e647080 1 -- 172.21.15.49:0/1447106034 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55784410d430 con 0x5578441f3b00 2023-11-06T02:14:34.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.758+0000 7f863995c640 1 -- 172.21.15.49:0/1447106034 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f86340b1070 con 0x5578441f3b00 2023-11-06T02:14:34.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.758+0000 7f863995c640 1 --2- 172.21.15.49:0/1447106034 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8628064410 0x7f86280668d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:34.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.758+0000 7f863995c640 1 -- 172.21.15.49:0/1447106034 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f863411ddd0 con 0x5578441f3b00 2023-11-06T02:14:34.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.758+0000 7f863e647080 1 --2- 172.21.15.49:0/1447106034 >> [v2:172.21.15.150:6824/38428,v1:172.21.15.150:6825/38428] conn(0x55784420a290 0x5578442969f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:34.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.758+0000 7f863e647080 1 -- 172.21.15.49:0/1447106034 --> [v2:172.21.15.150:6824/38428,v1:172.21.15.150:6825/38428] -- osd_op(unknown.0.0:1 12.5 12:a367a3e8:::10000000004.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81) v8 -- 0x557844296f30 con 0x55784420a290 2023-11-06T02:14:34.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.758+0000 7f863b960640 1 --2- 172.21.15.49:0/1447106034 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8628064410 0x7f86280668d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:34.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.758+0000 7f863c962640 1 --2- 172.21.15.49:0/1447106034 >> [v2:172.21.15.150:6824/38428,v1:172.21.15.150:6825/38428] conn(0x55784420a290 0x5578442969f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:34.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.758+0000 7f863b960640 1 --2- 172.21.15.49:0/1447106034 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8628064410 0x7f86280668d0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f8630005fd0 tx=0x7f8630005a10 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:34.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.758+0000 7f863c962640 1 --2- 172.21.15.49:0/1447106034 >> [v2:172.21.15.150:6824/38428,v1:172.21.15.150:6825/38428] conn(0x55784420a290 0x5578442969f0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:34.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.758+0000 7f863c962640 1 -- 172.21.15.49:0/1447106034 <== osd.6 v2:172.21.15.150:6824/38428 1 ==== osd_op_reply(1 10000000004.00000000 [getxattr (12) out=12b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+12 (crc 0 0 0) 0x7f862c001800 con 0x55784420a290 2023-11-06T02:14:34.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.758+0000 7f863e647080 1 -- 172.21.15.49:0/1447106034 >> [v2:172.21.15.150:6824/38428,v1:172.21.15.150:6825/38428] conn(0x55784420a290 msgr2=0x5578442969f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:34.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.758+0000 7f863e647080 1 --2- 172.21.15.49:0/1447106034 >> [v2:172.21.15.150:6824/38428,v1:172.21.15.150:6825/38428] conn(0x55784420a290 0x5578442969f0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:34.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.758+0000 7f863e647080 1 -- 172.21.15.49:0/1447106034 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8628064410 msgr2=0x7f86280668d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:34.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.758+0000 7f863e647080 1 --2- 172.21.15.49:0/1447106034 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8628064410 0x7f86280668d0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f8630005fd0 tx=0x7f8630005a10 comp rx=0 tx=0).stop 2023-11-06T02:14:34.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.758+0000 7f863e647080 1 -- 172.21.15.49:0/1447106034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5578441f3b00 msgr2=0x55784420d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:34.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.758+0000 7f863e647080 1 --2- 172.21.15.49:0/1447106034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5578441f3b00 0x55784420d770 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f86340199b0 tx=0x7f86340180a0 comp rx=0 tx=0).stop 2023-11-06T02:14:34.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.758+0000 7f863c962640 1 -- 172.21.15.49:0/1447106034 reap_dead start 2023-11-06T02:14:34.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.758+0000 7f863e647080 1 -- 172.21.15.49:0/1447106034 shutdown_connections 2023-11-06T02:14:34.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.758+0000 7f863e647080 1 -- 172.21.15.49:0/1447106034 >> 172.21.15.49:0/1447106034 conn(0x5578441f21a0 msgr2=0x55784432f5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:34.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.758+0000 7f863e647080 1 -- 172.21.15.49:0/1447106034 shutdown_connections 2023-11-06T02:14:34.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.758+0000 7f863e647080 1 -- 172.21.15.49:0/1447106034 wait complete. 2023-11-06T02:14:34.770 INFO:teuthology.orchestra.run.smithi049.stdout:mytag123 2023-11-06T02:14:34.770 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:0 flush journal 2023-11-06T02:14:34.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.950+0000 7fa167d0d640 1 Processor -- start 2023-11-06T02:14:34.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.950+0000 7fa167d0d640 1 -- start start 2023-11-06T02:14:34.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.950+0000 7fa167d0d640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fa160070160 0x7fa1600705d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:34.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.950+0000 7fa167d0d640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1600c8c80 con 0x7fa16006c230 2023-11-06T02:14:34.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.950+0000 7fa167d0d640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa160159700 con 0x7fa16014ffc0 2023-11-06T02:14:34.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.950+0000 7fa167d0d640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1600ae4d0 con 0x7fa160070160 2023-11-06T02:14:34.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.950+0000 7fa1662aa640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fa16006c230 0x7fa16006c680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:60634/0 (socket says 172.21.15.49:60634) 2023-11-06T02:14:34.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.950+0000 7fa1662aa640 1 -- 172.21.15.49:0/49756449 learned_addr learned my addr 172.21.15.49:0/49756449 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:14:34.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.950+0000 7fa1652a8640 1 --2- 172.21.15.49:0/49756449 >> v2:172.21.15.150:3300/0 conn(0x7fa160070160 0x7fa1600705d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:34.948 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.950+0000 7fa164aa7640 1 -- 172.21.15.49:0/49756449 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1254358687 0 0) 0x7fa1600c8c80 con 0x7fa16006c230 2023-11-06T02:14:34.948 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.950+0000 7fa164aa7640 1 -- 172.21.15.49:0/49756449 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa14c003530 con 0x7fa16006c230 2023-11-06T02:14:34.948 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.950+0000 7fa164aa7640 1 -- 172.21.15.49:0/49756449 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3867270965 0 0) 0x7fa14c003530 con 0x7fa16006c230 2023-11-06T02:14:34.948 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.950+0000 7fa164aa7640 1 -- 172.21.15.49:0/49756449 >> v2:172.21.15.150:3300/0 conn(0x7fa160070160 msgr2=0x7fa1600705d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:34.948 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.950+0000 7fa164aa7640 1 --2- 172.21.15.49:0/49756449 >> v2:172.21.15.150:3300/0 conn(0x7fa160070160 0x7fa1600705d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:34.949 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.950+0000 7fa164aa7640 1 -- 172.21.15.49:0/49756449 >> v1:172.21.15.150:6789/0 conn(0x7fa16014ffc0 legacy=0x7fa1600ada30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:34.949 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.950+0000 7fa164aa7640 1 -- 172.21.15.49:0/49756449 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1601599e0 con 0x7fa16006c230 2023-11-06T02:14:34.949 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.950+0000 7fa164aa7640 1 -- 172.21.15.49:0/49756449 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7fa158001a00 con 0x7fa16006c230 2023-11-06T02:14:34.949 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.950+0000 7fa164aa7640 1 -- 172.21.15.49:0/49756449 >> v1:172.21.15.49:6789/0 conn(0x7fa16006c230 legacy=0x7fa16006c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:34.949 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.950+0000 7fa164aa7640 1 --2- 172.21.15.49:0/49756449 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa14c003980 0x7fa14c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:34.950 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.950+0000 7fa164aa7640 1 --2- 172.21.15.49:0/49756449 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa14c0047d0 0x7fa14c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:34.950 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.950+0000 7fa164aa7640 1 --2- 172.21.15.49:0/49756449 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa14c005190 0x7fa14c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:34.950 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.950+0000 7fa164aa7640 1 -- 172.21.15.49:0/49756449 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1600ae4d0 con 0x7fa14c0047d0 2023-11-06T02:14:34.950 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.950+0000 7fa164aa7640 1 -- 172.21.15.49:0/49756449 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa14c00d140 con 0x7fa14c003980 2023-11-06T02:14:34.950 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.950+0000 7fa164aa7640 1 -- 172.21.15.49:0/49756449 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa14c00d300 con 0x7fa14c005190 2023-11-06T02:14:34.951 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.950+0000 7fa1652a8640 1 --2- 172.21.15.49:0/49756449 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa14c0047d0 0x7fa14c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:34.951 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.950+0000 7fa165aa9640 1 --2- 172.21.15.49:0/49756449 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa14c003980 0x7fa14c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:34.951 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.950+0000 7fa1662aa640 1 --2- 172.21.15.49:0/49756449 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa14c005190 0x7fa14c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:34.951 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.950+0000 7fa1652a8640 1 -- 172.21.15.49:0/49756449 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa14c005190 msgr2=0x7fa14c00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:34.951 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.950+0000 7fa1652a8640 1 --2- 172.21.15.49:0/49756449 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa14c005190 0x7fa14c00b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:34.952 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.950+0000 7fa1652a8640 1 -- 172.21.15.49:0/49756449 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa14c003980 msgr2=0x7fa14c003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:34.953 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.950+0000 7fa1652a8640 1 --2- 172.21.15.49:0/49756449 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa14c003980 0x7fa14c003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:34.953 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.950+0000 7fa1652a8640 1 -- 172.21.15.49:0/49756449 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa14c003530 con 0x7fa14c0047d0 2023-11-06T02:14:34.953 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.950+0000 7fa1662aa640 1 -- 172.21.15.49:0/49756449 reap_dead start 2023-11-06T02:14:34.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.954+0000 7fa1652a8640 1 --2- 172.21.15.49:0/49756449 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa14c0047d0 0x7fa14c004c50 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7fa150009ae0 tx=0x7fa15001ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36abaa1aa5abb82f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:34.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.954+0000 7fa164aa7640 1 -- 172.21.15.49:0/49756449 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1500a5640 con 0x7fa14c0047d0 2023-11-06T02:14:34.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.954+0000 7fa164aa7640 1 -- 172.21.15.49:0/49756449 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1500a57a0 con 0x7fa14c0047d0 2023-11-06T02:14:34.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.954+0000 7fa164aa7640 1 -- 172.21.15.49:0/49756449 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1500a5b50 con 0x7fa14c0047d0 2023-11-06T02:14:34.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.954+0000 7fa167d0d640 1 -- 172.21.15.49:0/49756449 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa14c0047d0 msgr2=0x7fa14c004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:34.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.954+0000 7fa167d0d640 1 --2- 172.21.15.49:0/49756449 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa14c0047d0 0x7fa14c004c50 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7fa150009ae0 tx=0x7fa15001ca50 comp rx=0 tx=0).stop 2023-11-06T02:14:34.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.954+0000 7fa167d0d640 1 -- 172.21.15.49:0/49756449 shutdown_connections 2023-11-06T02:14:34.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.954+0000 7fa167d0d640 1 --2- 172.21.15.49:0/49756449 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa14c0047d0 0x7fa14c004c50 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:34.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.954+0000 7fa167d0d640 1 -- 172.21.15.49:0/49756449 >> 172.21.15.49:0/49756449 conn(0x7fa16006a370 msgr2=0x7fa1600ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:34.964 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.954+0000 7fa167d0d640 1 -- 172.21.15.49:0/49756449 shutdown_connections 2023-11-06T02:14:34.964 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.954+0000 7fa167d0d640 1 -- 172.21.15.49:0/49756449 wait complete. 2023-11-06T02:14:34.964 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.954+0000 7fa167d0d640 1 Processor -- start 2023-11-06T02:14:34.964 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.954+0000 7fa167d0d640 1 -- start start 2023-11-06T02:14:34.964 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.954+0000 7fa167d0d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa14c0047d0 0x7fa160147040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:34.965 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.954+0000 7fa167d0d640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa1601e3220 0x7fa1601e0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:34.965 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.954+0000 7fa167d0d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1600c7940 0x7fa1600c7df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:34.965 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.954+0000 7fa167d0d640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1601e17b0 con 0x7fa1600c7940 2023-11-06T02:14:34.965 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.954+0000 7fa167d0d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1600c8330 con 0x7fa14c0047d0 2023-11-06T02:14:34.965 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.954+0000 7fa167d0d640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1601e0b30 con 0x7fa1601e3220 2023-11-06T02:14:34.966 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.954+0000 7fa1662aa640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1600c7940 0x7fa1600c7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:34.966 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.954+0000 7fa1662aa640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1600c7940 0x7fa1600c7df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38056/0 (socket says 172.21.15.49:38056) 2023-11-06T02:14:34.966 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.954+0000 7fa1662aa640 1 -- 172.21.15.49:0/213569895 learned_addr learned my addr 172.21.15.49:0/213569895 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:34.966 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.958+0000 7fa1652a8640 1 --2- 172.21.15.49:0/213569895 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa1601e3220 0x7fa1601e0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:34.966 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.958+0000 7fa165aa9640 1 --2- 172.21.15.49:0/213569895 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa14c0047d0 0x7fa160147040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:34.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.958+0000 7fa1662aa640 1 -- 172.21.15.49:0/213569895 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa1601e3220 msgr2=0x7fa1601e0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:34.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.958+0000 7fa1662aa640 1 --2- 172.21.15.49:0/213569895 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa1601e3220 0x7fa1601e0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:34.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.958+0000 7fa1662aa640 1 -- 172.21.15.49:0/213569895 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa14c0047d0 msgr2=0x7fa160147040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:34.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.958+0000 7fa1662aa640 1 --2- 172.21.15.49:0/213569895 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa14c0047d0 0x7fa160147040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:34.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.958+0000 7fa1662aa640 1 -- 172.21.15.49:0/213569895 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1601e0e10 con 0x7fa1600c7940 2023-11-06T02:14:34.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.958+0000 7fa1662aa640 1 --2- 172.21.15.49:0/213569895 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1600c7940 0x7fa1600c7df0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7fa1600cf730 tx=0x7fa158010cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:34.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.958+0000 7fa1567fc640 1 -- 172.21.15.49:0/213569895 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15800eac0 con 0x7fa1600c7940 2023-11-06T02:14:34.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.958+0000 7fa1567fc640 1 -- 172.21.15.49:0/213569895 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa15800ec60 con 0x7fa1600c7940 2023-11-06T02:14:34.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.958+0000 7fa167d0d640 1 -- 172.21.15.49:0/213569895 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1601e10f0 con 0x7fa1600c7940 2023-11-06T02:14:34.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.958+0000 7fa1567fc640 1 -- 172.21.15.49:0/213569895 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa158015400 con 0x7fa1600c7940 2023-11-06T02:14:34.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.958+0000 7fa167d0d640 1 -- 172.21.15.49:0/213569895 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa160147580 con 0x7fa1600c7940 2023-11-06T02:14:34.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.958+0000 7fa1567fc640 1 -- 172.21.15.49:0/213569895 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fa158015560 con 0x7fa1600c7940 2023-11-06T02:14:34.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.958+0000 7fa1567fc640 1 --2- 172.21.15.49:0/213569895 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa144065a90 0x7fa144067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:34.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.958+0000 7fa1567fc640 1 -- 172.21.15.49:0/213569895 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6148+0+0 (secure 0 0 0) 0x7fa158085d40 con 0x7fa1600c7940 2023-11-06T02:14:34.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.958+0000 7fa165aa9640 1 --2- 172.21.15.49:0/213569895 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa144065a90 0x7fa144067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:34.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.958+0000 7fa16ae35000 1 Processor -- start 2023-11-06T02:14:34.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.958+0000 7fa165aa9640 1 --2- 172.21.15.49:0/213569895 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa144065a90 0x7fa144067f50 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fa15c000c10 tx=0x7fa15c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:34.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.958+0000 7fa16ae35000 1 -- start start 2023-11-06T02:14:34.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.962+0000 7fa16ae35000 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557cd3f08130 0x557cd3f08510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:34.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.962+0000 7fa16ae35000 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557cd3f08a50 0x557cd3ebdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:34.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.962+0000 7fa16ae35000 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557cd3ebe260 0x557cd3ebe640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:34.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.962+0000 7fa16ae35000 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cd3d85c10 con 0x557cd3ebe260 2023-11-06T02:14:34.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.962+0000 7fa16ae35000 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cd3d143e0 con 0x557cd3f08a50 2023-11-06T02:14:34.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.962+0000 7fa16ae35000 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x557cd3d119b0 con 0x557cd3f08130 2023-11-06T02:14:34.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.962+0000 7fa1662aa640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557cd3ebe260 0x557cd3ebe640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:34.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.962+0000 7fa1662aa640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557cd3ebe260 0x557cd3ebe640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38066/0 (socket says 172.21.15.49:38066) 2023-11-06T02:14:34.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.962+0000 7fa1662aa640 1 -- 172.21.15.49:0/2954915829 learned_addr learned my addr 172.21.15.49:0/2954915829 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:34.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.962+0000 7fa165aa9640 1 --2- 172.21.15.49:0/2954915829 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557cd3f08130 0x557cd3f08510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:34.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.962+0000 7fa1652a8640 1 --2- 172.21.15.49:0/2954915829 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557cd3f08a50 0x557cd3ebdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:34.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.962+0000 7fa1662aa640 1 -- 172.21.15.49:0/2954915829 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557cd3f08130 msgr2=0x557cd3f08510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:34.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.962+0000 7fa1662aa640 1 --2- 172.21.15.49:0/2954915829 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557cd3f08130 0x557cd3f08510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:34.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.962+0000 7fa1662aa640 1 -- 172.21.15.49:0/2954915829 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557cd3f08a50 msgr2=0x557cd3ebdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:34.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.962+0000 7fa1662aa640 1 --2- 172.21.15.49:0/2954915829 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557cd3f08a50 0x557cd3ebdd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:34.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.962+0000 7fa1662aa640 1 -- 172.21.15.49:0/2954915829 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557cd3a09e20 con 0x557cd3ebe260 2023-11-06T02:14:34.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.962+0000 7fa1652a8640 1 --2- 172.21.15.49:0/2954915829 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557cd3f08a50 0x557cd3ebdd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:14:34.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.962+0000 7fa1662aa640 1 --2- 172.21.15.49:0/2954915829 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557cd3ebe260 0x557cd3ebe640 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7fa15805ed40 tx=0x7fa158057340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4d900d9322c0002 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:34.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.962+0000 7fa1437fe640 1 -- 172.21.15.49:0/2954915829 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15806e070 con 0x557cd3ebe260 2023-11-06T02:14:34.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.962+0000 7fa1437fe640 1 -- 172.21.15.49:0/2954915829 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa15805f270 con 0x557cd3ebe260 2023-11-06T02:14:34.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.962+0000 7fa1437fe640 1 -- 172.21.15.49:0/2954915829 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1580556e0 con 0x557cd3ebe260 2023-11-06T02:14:34.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.962+0000 7fa16ae35000 1 -- 172.21.15.49:0/2954915829 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557cd3ebe260 msgr2=0x557cd3ebe640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:34.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.962+0000 7fa16ae35000 1 --2- 172.21.15.49:0/2954915829 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557cd3ebe260 0x557cd3ebe640 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7fa15805ed40 tx=0x7fa158057340 comp rx=0 tx=0).stop 2023-11-06T02:14:34.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.962+0000 7fa16ae35000 1 -- 172.21.15.49:0/2954915829 shutdown_connections 2023-11-06T02:14:34.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.962+0000 7fa16ae35000 1 --2- 172.21.15.49:0/2954915829 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557cd3ebe260 0x557cd3ebe640 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:34.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.962+0000 7fa16ae35000 1 --2- 172.21.15.49:0/2954915829 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557cd3f08a50 0x557cd3ebdd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:34.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.962+0000 7fa16ae35000 1 --2- 172.21.15.49:0/2954915829 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557cd3f08130 0x557cd3f08510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:34.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.962+0000 7fa16ae35000 1 -- 172.21.15.49:0/2954915829 >> 172.21.15.49:0/2954915829 conn(0x557cd3ec49f0 msgr2=0x557cd3ebc600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:34.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.962+0000 7fa16ae35000 1 -- 172.21.15.49:0/2954915829 shutdown_connections 2023-11-06T02:14:34.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.962+0000 7fa16ae35000 1 -- 172.21.15.49:0/2954915829 wait complete. 2023-11-06T02:14:34.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.962+0000 7fa16ae35000 1 -- start start 2023-11-06T02:14:34.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.962+0000 7fa16ae35000 1 Processor -- start 2023-11-06T02:14:34.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.962+0000 7fa16ae35000 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557cd3ebe260 0x557cd3f0f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:34.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.962+0000 7fa16ae35000 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557cd3f08130 0x557cd3f0fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:34.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.962+0000 7fa16ae35000 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557cd3f08a50 0x557cd3f09040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:34.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.962+0000 7fa16ae35000 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cd3d0d2a0 con 0x557cd3ebe260 2023-11-06T02:14:34.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.962+0000 7fa16ae35000 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cd3cac470 con 0x557cd3f08130 2023-11-06T02:14:34.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.962+0000 7fa16ae35000 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x557cd3cca660 con 0x557cd3f08a50 2023-11-06T02:14:34.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.962+0000 7fa16ae35000 10 client.0 _ll_register_callbacks cb 0x557cd3ea9c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T02:14:34.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.962+0000 7fa165aa9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557cd3ebe260 0x557cd3f0f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:34.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.962+0000 7fa165aa9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557cd3ebe260 0x557cd3f0f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38074/0 (socket says 172.21.15.49:38074) 2023-11-06T02:14:34.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.962+0000 7fa165aa9640 1 -- 172.21.15.49:0/3739867382 learned_addr learned my addr 172.21.15.49:0/3739867382 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:34.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.962+0000 7fa1652a8640 1 --2- 172.21.15.49:0/3739867382 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557cd3f08130 0x557cd3f0fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:34.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.962+0000 7fa165aa9640 1 -- 172.21.15.49:0/3739867382 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557cd3f08a50 msgr2=0x557cd3f09040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:34.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.962+0000 7fa1662aa640 1 --2- 172.21.15.49:0/3739867382 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557cd3f08a50 0x557cd3f09040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:34.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.962+0000 7fa165aa9640 1 --2- 172.21.15.49:0/3739867382 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557cd3f08a50 0x557cd3f09040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:34.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.962+0000 7fa165aa9640 1 -- 172.21.15.49:0/3739867382 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557cd3f08130 msgr2=0x557cd3f0fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:34.980 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.962+0000 7fa165aa9640 1 --2- 172.21.15.49:0/3739867382 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557cd3f08130 0x557cd3f0fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:34.980 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.962+0000 7fa165aa9640 1 -- 172.21.15.49:0/3739867382 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa160147580 con 0x557cd3ebe260 2023-11-06T02:14:34.980 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.962+0000 7fa1652a8640 1 --2- 172.21.15.49:0/3739867382 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557cd3f08130 0x557cd3f0fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:14:34.980 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.966+0000 7fa165aa9640 1 --2- 172.21.15.49:0/3739867382 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557cd3ebe260 0x557cd3f0f600 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7fa15c001b60 tx=0x7fa15c0011a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c1c6824e981caa7 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:34.980 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.966+0000 7fa140ff9640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-11-06T02:14:34.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.966+0000 7fa140ff9640 1 -- 172.21.15.49:0/3739867382 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15c0215a0 con 0x557cd3ebe260 2023-11-06T02:14:34.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.966+0000 7fa140ff9640 1 -- 172.21.15.49:0/3739867382 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 6148+0+0 (secure 0 0 0) 0x7fa15c01c070 con 0x557cd3ebe260 2023-11-06T02:14:34.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.966+0000 7fa16ae35000 1 -- 172.21.15.49:0/3739867382 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557cd3cca0a0 con 0x557cd3ebe260 2023-11-06T02:14:34.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.966+0000 7fa140ff9640 1 -- 172.21.15.49:0/3739867382 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa15c014e70 con 0x557cd3ebe260 2023-11-06T02:14:34.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.966+0000 7fa16ae35000 10 client.7734 fetch_fsmap learned FSMap version 52 2023-11-06T02:14:34.982 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.966+0000 7fa16ae35000 1 -- 172.21.15.49:0/3739867382 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) v3 -- 0x557cd3cd21d0 con 0x557cd3ebe260 2023-11-06T02:14:34.982 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.966+0000 7fa140ff9640 1 -- 172.21.15.49:0/3739867382 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557cd3cca0a0 con 0x557cd3ebe260 2023-11-06T02:14:34.982 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.966+0000 7fa140ff9640 1 -- 172.21.15.49:0/3739867382 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 52) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fa15c021700 con 0x557cd3ebe260 2023-11-06T02:14:34.982 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.966+0000 7fa16ae35000 10 client.7734 fetch_fsmap finished waiting for FSMap version 52 2023-11-06T02:14:34.982 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.966+0000 7fa16ae35000 10 client.7734 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2023-11-06T02:14:34.983 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.966+0000 7fa16ae35000 20 client.7734 populate_metadata read hostname 'smithi049' 2023-11-06T02:14:34.983 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.966+0000 7fa16ae35000 1 --2- 172.21.15.49:0/3739867382 >> [v2:172.21.15.49:6834/4037104163,v1:172.21.15.49:6835/4037104163] conn(0x557cd3f09db0 0x557cd3f0a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:34.983 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.966+0000 7fa16ae35000 4 client.7734 mds_command: new command op to 7533 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T02:14:34.983 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.966+0000 7fa16ae35000 1 -- 172.21.15.49:0/3739867382 --> [v2:172.21.15.49:6834/4037104163,v1:172.21.15.49:6835/4037104163] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557cd3d0bbf0 con 0x557cd3f09db0 2023-11-06T02:14:34.983 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.966+0000 7fa1652a8640 1 --2- 172.21.15.49:0/3739867382 >> [v2:172.21.15.49:6834/4037104163,v1:172.21.15.49:6835/4037104163] conn(0x557cd3f09db0 0x557cd3f0a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:34.983 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.966+0000 7fa1652a8640 1 --2- 172.21.15.49:0/3739867382 >> [v2:172.21.15.49:6834/4037104163,v1:172.21.15.49:6835/4037104163] conn(0x557cd3f09db0 0x557cd3f0a1c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7121348eb17fd8fa server_cookie=9806618e35c6862d in_seq=0 out_seq=0 2023-11-06T02:14:34.984 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.966+0000 7fa140ff9640 10 client.7734 ms_handle_connect on v2:172.21.15.49:6834/4037104163 2023-11-06T02:14:34.984 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.966+0000 7fa140ff9640 1 -- 172.21.15.49:0/3739867382 <== mds.0 v2:172.21.15.49:6834/4037104163 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x557cd3d0bbf0 con 0x557cd3f09db0 2023-11-06T02:14:34.984 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.966+0000 7fa140ff9640 10 client.7734 handle_command_reply: tid=0 2023-11-06T02:14:34.984 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.966+0000 7fa16ae35000 1 client.7734 shutdown 2023-11-06T02:14:34.984 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.966+0000 7fa16ae35000 1 -- 172.21.15.49:0/3739867382 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557cd3ebe260 msgr2=0x557cd3f0f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:34.985 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.966+0000 7fa16ae35000 1 --2- 172.21.15.49:0/3739867382 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557cd3ebe260 0x557cd3f0f600 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7fa15c001b60 tx=0x7fa15c0011a0 comp rx=0 tx=0).stop 2023-11-06T02:14:34.985 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.966+0000 7fa16ae35000 1 -- 172.21.15.49:0/3739867382 shutdown_connections 2023-11-06T02:14:34.985 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.966+0000 7fa16ae35000 1 --2- 172.21.15.49:0/3739867382 >> [v2:172.21.15.49:6834/4037104163,v1:172.21.15.49:6835/4037104163] conn(0x557cd3f09db0 0x557cd3f0a1c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:34.985 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.966+0000 7fa16ae35000 1 --2- 172.21.15.49:0/3739867382 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557cd3f08a50 0x557cd3f09040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:34.985 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.966+0000 7fa16ae35000 1 --2- 172.21.15.49:0/3739867382 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557cd3f08130 0x557cd3f0fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:34.986 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.966+0000 7fa140ff9640 0 client.7734 ms_handle_reset on v2:172.21.15.49:6834/4037104163 2023-11-06T02:14:34.986 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.966+0000 7fa16ae35000 1 --2- 172.21.15.49:0/3739867382 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557cd3ebe260 0x557cd3f0f600 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:34.986 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.966+0000 7fa16ae35000 1 -- 172.21.15.49:0/3739867382 >> 172.21.15.49:0/3739867382 conn(0x557cd3ec49f0 msgr2=0x557cd3ebcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:34.986 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.966+0000 7fa16ae35000 1 -- 172.21.15.49:0/3739867382 shutdown_connections 2023-11-06T02:14:34.986 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.966+0000 7fa16ae35000 1 -- 172.21.15.49:0/3739867382 wait complete. 2023-11-06T02:14:34.986 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.966+0000 7fa16ae35000 20 client.7734 trim_cache size 0 max 16384 2023-11-06T02:14:34.991 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.978+0000 7fa16ae35000 1 Processor -- start 2023-11-06T02:14:34.991 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.978+0000 7fa16ae35000 1 -- start start 2023-11-06T02:14:34.991 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.978+0000 7fa16ae35000 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557cd3ebe260 0x557cd3ec3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:34.991 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.978+0000 7fa16ae35000 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557cd3f08130 0x557cd3ec3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:34.992 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.978+0000 7fa16ae35000 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557cd3f08a50 0x557cd3f0d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:34.992 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.978+0000 7fa16ae35000 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cd39caaa0 con 0x557cd3f08130 2023-11-06T02:14:34.992 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.978+0000 7fa16ae35000 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cd3d0a6a0 con 0x557cd3ebe260 2023-11-06T02:14:34.992 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.978+0000 7fa16ae35000 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x557cd3d0c130 con 0x557cd3f08a50 2023-11-06T02:14:34.992 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.978+0000 7fa1652a8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557cd3f08130 0x557cd3ec3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:34.993 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.978+0000 7fa1652a8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557cd3f08130 0x557cd3ec3dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38088/0 (socket says 172.21.15.49:38088) 2023-11-06T02:14:34.993 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.978+0000 7fa1652a8640 1 -- 172.21.15.49:0/114074890 learned_addr learned my addr 172.21.15.49:0/114074890 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:34.993 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.978+0000 7fa1652a8640 1 -- 172.21.15.49:0/114074890 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557cd3f08a50 msgr2=0x557cd3f0d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:34.993 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.978+0000 7fa1652a8640 1 --2- 172.21.15.49:0/114074890 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557cd3f08a50 0x557cd3f0d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:34.993 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.978+0000 7fa1652a8640 1 -- 172.21.15.49:0/114074890 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557cd3ebe260 msgr2=0x557cd3ec3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:34.994 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.978+0000 7fa1652a8640 1 --2- 172.21.15.49:0/114074890 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557cd3ebe260 0x557cd3ec3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:34.994 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.978+0000 7fa1652a8640 1 -- 172.21.15.49:0/114074890 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557cd3cb8520 con 0x557cd3f08130 2023-11-06T02:14:34.994 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.982+0000 7fa1652a8640 1 --2- 172.21.15.49:0/114074890 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557cd3f08130 0x557cd3ec3dc0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fa15001c880 tx=0x7fa1500046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=724cfe697073447c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:34.994 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.982+0000 7fa143fff640 1 -- 172.21.15.49:0/114074890 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1500042c0 con 0x557cd3f08130 2023-11-06T02:14:34.994 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.982+0000 7fa143fff640 1 -- 172.21.15.49:0/114074890 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa150004420 con 0x557cd3f08130 2023-11-06T02:14:34.994 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.982+0000 7fa143fff640 1 -- 172.21.15.49:0/114074890 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1500a5b50 con 0x557cd3f08130 2023-11-06T02:14:34.995 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.982+0000 7fa16ae35000 1 -- 172.21.15.49:0/114074890 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557cd3f08130 msgr2=0x557cd3ec3dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:34.995 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.982+0000 7fa16ae35000 1 --2- 172.21.15.49:0/114074890 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557cd3f08130 0x557cd3ec3dc0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fa15001c880 tx=0x7fa1500046d0 comp rx=0 tx=0).stop 2023-11-06T02:14:34.995 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.982+0000 7fa16ae35000 1 -- 172.21.15.49:0/114074890 shutdown_connections 2023-11-06T02:14:34.995 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.982+0000 7fa16ae35000 1 --2- 172.21.15.49:0/114074890 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557cd3f08130 0x557cd3ec3dc0 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:34.995 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.982+0000 7fa16ae35000 1 --2- 172.21.15.49:0/114074890 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557cd3f08a50 0x557cd3f0d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:34.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.982+0000 7fa16ae35000 1 --2- 172.21.15.49:0/114074890 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557cd3ebe260 0x557cd3ec3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:34.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.982+0000 7fa16ae35000 1 -- 172.21.15.49:0/114074890 >> 172.21.15.49:0/114074890 conn(0x557cd3ec49f0 msgr2=0x557cd3ec3170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:34.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.982+0000 7fa16ae35000 1 -- 172.21.15.49:0/114074890 shutdown_connections 2023-11-06T02:14:34.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.982+0000 7fa16ae35000 1 -- 172.21.15.49:0/114074890 wait complete. 2023-11-06T02:14:34.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.982+0000 7fa16ae35000 1 -- start start 2023-11-06T02:14:34.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.982+0000 7fa16ae35000 1 Processor -- start 2023-11-06T02:14:34.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.982+0000 7fa16ae35000 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557cd3ebe260 0x557cd3f64680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:34.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.982+0000 7fa16ae35000 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557cd3f08130 0x557cd3f037b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:34.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.982+0000 7fa16ae35000 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557cd3f08a50 0x557cd3f03cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:34.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.982+0000 7fa16ae35000 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cd3ec0880 con 0x557cd3f08a50 2023-11-06T02:14:34.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.982+0000 7fa16ae35000 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cd3d07640 con 0x557cd3f08130 2023-11-06T02:14:34.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.982+0000 7fa16ae35000 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x557cd3d0aab0 con 0x557cd3ebe260 2023-11-06T02:14:34.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.982+0000 7fa16ae35000 10 client.0 _ll_register_callbacks cb 0x557cd3cd1cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T02:14:34.999 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.982+0000 7fa1662aa640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557cd3f08a50 0x557cd3f03cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:34.999 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.982+0000 7fa1662aa640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557cd3f08a50 0x557cd3f03cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38098/0 (socket says 172.21.15.49:38098) 2023-11-06T02:14:34.999 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.982+0000 7fa1662aa640 1 -- 172.21.15.49:0/3080218401 learned_addr learned my addr 172.21.15.49:0/3080218401 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:34.999 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.982+0000 7fa165aa9640 1 --2- 172.21.15.49:0/3080218401 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557cd3ebe260 0x557cd3f64680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:34.999 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.982+0000 7fa1652a8640 1 --2- 172.21.15.49:0/3080218401 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557cd3f08130 0x557cd3f037b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:35.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.982+0000 7fa1662aa640 1 -- 172.21.15.49:0/3080218401 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557cd3ebe260 msgr2=0x557cd3f64680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:35.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.982+0000 7fa1662aa640 1 --2- 172.21.15.49:0/3080218401 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557cd3ebe260 0x557cd3f64680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.982+0000 7fa1662aa640 1 -- 172.21.15.49:0/3080218401 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557cd3f08130 msgr2=0x557cd3f037b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:35.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.982+0000 7fa1662aa640 1 --2- 172.21.15.49:0/3080218401 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557cd3f08130 0x557cd3f037b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.982+0000 7fa1662aa640 1 -- 172.21.15.49:0/3080218401 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa14c003530 con 0x557cd3f08a50 2023-11-06T02:14:35.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.982+0000 7fa1662aa640 1 --2- 172.21.15.49:0/3080218401 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557cd3f08a50 0x557cd3f03cf0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fa15805ed40 tx=0x7fa158084010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3f065ee67a9fcbd server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:35.001 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.982+0000 7fa140ff9640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-11-06T02:14:35.001 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.982+0000 7fa140ff9640 1 -- 172.21.15.49:0/3080218401 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15806e040 con 0x557cd3f08a50 2023-11-06T02:14:35.001 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.982+0000 7fa140ff9640 1 -- 172.21.15.49:0/3080218401 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 6148+0+0 (secure 0 0 0) 0x7fa15805d0c0 con 0x557cd3f08a50 2023-11-06T02:14:35.001 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.982+0000 7fa16ae35000 1 -- 172.21.15.49:0/3080218401 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557cd3cd6620 con 0x557cd3f08a50 2023-11-06T02:14:35.001 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.982+0000 7fa140ff9640 1 -- 172.21.15.49:0/3080218401 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa158056830 con 0x557cd3f08a50 2023-11-06T02:14:35.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.982+0000 7fa16ae35000 10 client.7740 fetch_fsmap learned FSMap version 52 2023-11-06T02:14:35.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.982+0000 7fa16ae35000 1 -- 172.21.15.49:0/3080218401 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) v3 -- 0x557cd3cac090 con 0x557cd3f08a50 2023-11-06T02:14:35.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.982+0000 7fa140ff9640 1 -- 172.21.15.49:0/3080218401 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557cd3cd6620 con 0x557cd3f08a50 2023-11-06T02:14:35.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.982+0000 7fa140ff9640 1 -- 172.21.15.49:0/3080218401 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 52) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fa15805ede0 con 0x557cd3f08a50 2023-11-06T02:14:35.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.982+0000 7fa16ae35000 10 client.7740 fetch_fsmap finished waiting for FSMap version 52 2023-11-06T02:14:35.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.982+0000 7fa16ae35000 10 client.7740 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2023-11-06T02:14:35.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.982+0000 7fa16ae35000 20 client.7740 populate_metadata read hostname 'smithi049' 2023-11-06T02:14:35.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.982+0000 7fa16ae35000 1 --2- 172.21.15.49:0/3080218401 >> [v2:172.21.15.49:6834/4037104163,v1:172.21.15.49:6835/4037104163] conn(0x557cd3f04230 0x557cd3f04610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:35.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.982+0000 7fa16ae35000 4 client.7740 mds_command: new command op to 7533 tid=0[{"prefix": "flush journal"}] 2023-11-06T02:14:35.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.982+0000 7fa16ae35000 1 -- 172.21.15.49:0/3080218401 --> [v2:172.21.15.49:6834/4037104163,v1:172.21.15.49:6835/4037104163] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557cd3d06320 con 0x557cd3f04230 2023-11-06T02:14:35.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.982+0000 7fa165aa9640 1 --2- 172.21.15.49:0/3080218401 >> [v2:172.21.15.49:6834/4037104163,v1:172.21.15.49:6835/4037104163] conn(0x557cd3f04230 0x557cd3f04610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:35.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.986+0000 7fa165aa9640 1 --2- 172.21.15.49:0/3080218401 >> [v2:172.21.15.49:6834/4037104163,v1:172.21.15.49:6835/4037104163] conn(0x557cd3f04230 0x557cd3f04610 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c2c7dac292c5a63 server_cookie=5933bc57425699e4 in_seq=0 out_seq=0 2023-11-06T02:14:35.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:34.986+0000 7fa140ff9640 10 client.7740 ms_handle_connect on v2:172.21.15.49:6834/4037104163 2023-11-06T02:14:35.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.002+0000 7fa140ff9640 1 -- 172.21.15.49:0/3080218401 <== mds.0 v2:172.21.15.49:6834/4037104163 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557cd3d06320 con 0x557cd3f04230 2023-11-06T02:14:35.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.002+0000 7fa140ff9640 10 client.7740 handle_command_reply: tid=0 2023-11-06T02:14:35.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.002+0000 7fa16ae35000 1 client.7740 shutdown 2023-11-06T02:14:35.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.002+0000 7fa16ae35000 1 -- 172.21.15.49:0/3080218401 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557cd3f08a50 msgr2=0x557cd3f03cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:35.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.002+0000 7fa16ae35000 1 --2- 172.21.15.49:0/3080218401 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557cd3f08a50 0x557cd3f03cf0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fa15805ed40 tx=0x7fa158084010 comp rx=0 tx=0).stop 2023-11-06T02:14:35.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.002+0000 7fa16ae35000 1 -- 172.21.15.49:0/3080218401 shutdown_connections 2023-11-06T02:14:35.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.002+0000 7fa16ae35000 1 --2- 172.21.15.49:0/3080218401 >> [v2:172.21.15.49:6834/4037104163,v1:172.21.15.49:6835/4037104163] conn(0x557cd3f04230 0x557cd3f04610 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.002+0000 7fa16ae35000 1 --2- 172.21.15.49:0/3080218401 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557cd3f08a50 0x557cd3f03cf0 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.002+0000 7fa16ae35000 1 --2- 172.21.15.49:0/3080218401 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557cd3f08130 0x557cd3f037b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.002+0000 7fa140ff9640 0 client.7740 ms_handle_reset on v2:172.21.15.49:6834/4037104163 2023-11-06T02:14:35.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.002+0000 7fa16ae35000 1 --2- 172.21.15.49:0/3080218401 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557cd3ebe260 0x557cd3f64680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.002+0000 7fa16ae35000 1 -- 172.21.15.49:0/3080218401 >> 172.21.15.49:0/3080218401 conn(0x557cd3ec49f0 msgr2=0x557cd3ec3170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:35.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.002+0000 7fa16ae35000 1 -- 172.21.15.49:0/3080218401 shutdown_connections 2023-11-06T02:14:35.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.002+0000 7fa16ae35000 1 -- 172.21.15.49:0/3080218401 wait complete. 2023-11-06T02:14:35.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.002+0000 7fa16ae35000 20 client.7740 trim_cache size 0 max 16384 2023-11-06T02:14:35.009 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-11-06T02:14:35.009 INFO:teuthology.orchestra.run.smithi049.stdout: "message": "", 2023-11-06T02:14:35.009 INFO:teuthology.orchestra.run.smithi049.stdout: "return_code": 0 2023-11-06T02:14:35.009 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-11-06T02:14:35.010 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.006+0000 7fa167d0d640 1 -- 172.21.15.49:0/213569895 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa144065a90 msgr2=0x7fa144067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:35.010 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.006+0000 7fa167d0d640 1 --2- 172.21.15.49:0/213569895 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa144065a90 0x7fa144067f50 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fa15c000c10 tx=0x7fa15c009040 comp rx=0 tx=0).stop 2023-11-06T02:14:35.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.006+0000 7fa167d0d640 1 -- 172.21.15.49:0/213569895 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1600c7940 msgr2=0x7fa1600c7df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:35.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.006+0000 7fa167d0d640 1 --2- 172.21.15.49:0/213569895 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1600c7940 0x7fa1600c7df0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7fa1600cf730 tx=0x7fa158010cf0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.006+0000 7fa167d0d640 1 -- 172.21.15.49:0/213569895 shutdown_connections 2023-11-06T02:14:35.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.006+0000 7fa167d0d640 1 --2- 172.21.15.49:0/213569895 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa144065a90 0x7fa144067f50 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.006+0000 7fa167d0d640 1 --2- 172.21.15.49:0/213569895 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1600c7940 0x7fa1600c7df0 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.006+0000 7fa167d0d640 1 --2- 172.21.15.49:0/213569895 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa1601e3220 0x7fa1601e0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.006+0000 7fa167d0d640 1 --2- 172.21.15.49:0/213569895 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa14c0047d0 0x7fa160147040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.006+0000 7fa167d0d640 1 -- 172.21.15.49:0/213569895 >> 172.21.15.49:0/213569895 conn(0x7fa16006a370 msgr2=0x7fa16014ef00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:35.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.006+0000 7fa167d0d640 1 -- 172.21.15.49:0/213569895 shutdown_connections 2023-11-06T02:14:35.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.006+0000 7fa167d0d640 1 -- 172.21.15.49:0/213569895 wait complete. 2023-11-06T02:14:35.016 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-06T02:14:35.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.138+0000 7f3d61d74640 1 Processor -- start 2023-11-06T02:14:35.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.138+0000 7f3d61d74640 1 -- start start 2023-11-06T02:14:35.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.138+0000 7f3d61d74640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f3d5c0cda70 0x7f3d5c161650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:35.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.138+0000 7f3d61d74640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f3d5c0714a0 0x7f3d5c0718b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:35.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.138+0000 7f3d61d74640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d5c0d17c0 con 0x7f3d5c0b1f90 2023-11-06T02:14:35.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.138+0000 7f3d61d74640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d5c0cdf20 con 0x7f3d5c0cda70 2023-11-06T02:14:35.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.138+0000 7f3d61d74640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d5c161b90 con 0x7f3d5c0714a0 2023-11-06T02:14:35.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.138+0000 7f3d53fff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f3d5c0cda70 0x7f3d5c161650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:35.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.138+0000 7f3d53fff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f3d5c0cda70 0x7f3d5c161650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38112/0 (socket says 172.21.15.49:38112) 2023-11-06T02:14:35.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.138+0000 7f3d53fff640 1 -- 172.21.15.49:0/1241053770 learned_addr learned my addr 172.21.15.49:0/1241053770 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:35.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.138+0000 7f3d5bfff640 1 --2- 172.21.15.49:0/1241053770 >> v2:172.21.15.150:3300/0 conn(0x7f3d5c0714a0 0x7f3d5c0718b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:35.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.142+0000 7f3d53fff640 1 -- 172.21.15.49:0/1241053770 >> v2:172.21.15.150:3300/0 conn(0x7f3d5c0714a0 msgr2=0x7f3d5c0718b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:35.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.142+0000 7f3d53fff640 1 --2- 172.21.15.49:0/1241053770 >> v2:172.21.15.150:3300/0 conn(0x7f3d5c0714a0 0x7f3d5c0718b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.142+0000 7f3d53fff640 1 -- 172.21.15.49:0/1241053770 >> v1:172.21.15.150:6789/0 conn(0x7f3d5c0b1f90 legacy=0x7f3d5c0d1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:35.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.142+0000 7f3d53fff640 1 -- 172.21.15.49:0/1241053770 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d5c071df0 con 0x7f3d5c0cda70 2023-11-06T02:14:35.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.142+0000 7f3d53fff640 1 --2- 172.21.15.49:0/1241053770 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d5c0cda70 0x7f3d5c161650 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c0b2370 tx=0x7f3d4401c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eceb1d60cc03ab3 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:35.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.142+0000 7f3d5affd640 1 -- 172.21.15.49:0/1241053770 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d440ab5f0 con 0x7f3d5c0cda70 2023-11-06T02:14:35.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.142+0000 7f3d5affd640 1 -- 172.21.15.49:0/1241053770 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d440ab750 con 0x7f3d5c0cda70 2023-11-06T02:14:35.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.142+0000 7f3d5affd640 1 -- 172.21.15.49:0/1241053770 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d440abae0 con 0x7f3d5c0cda70 2023-11-06T02:14:35.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.142+0000 7f3d61d74640 1 -- 172.21.15.49:0/1241053770 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d5c0cda70 msgr2=0x7f3d5c161650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:35.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.142+0000 7f3d61d74640 1 --2- 172.21.15.49:0/1241053770 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d5c0cda70 0x7f3d5c161650 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c0b2370 tx=0x7f3d4401c500 comp rx=0 tx=0).stop 2023-11-06T02:14:35.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.142+0000 7f3d61d74640 1 -- 172.21.15.49:0/1241053770 shutdown_connections 2023-11-06T02:14:35.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.142+0000 7f3d61d74640 1 --2- 172.21.15.49:0/1241053770 >> v2:172.21.15.150:3300/0 conn(0x7f3d5c0714a0 0x7f3d5c0718b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.142+0000 7f3d61d74640 1 --2- 172.21.15.49:0/1241053770 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d5c0cda70 0x7f3d5c161650 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.142+0000 7f3d61d74640 1 -- 172.21.15.49:0/1241053770 >> 172.21.15.49:0/1241053770 conn(0x7f3d5c0d2aa0 msgr2=0x7f3d5c0d2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:35.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.142+0000 7f3d61d74640 1 -- 172.21.15.49:0/1241053770 shutdown_connections 2023-11-06T02:14:35.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.142+0000 7f3d61d74640 1 -- 172.21.15.49:0/1241053770 wait complete. 2023-11-06T02:14:35.143 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.146+0000 7f3d61d74640 1 Processor -- start 2023-11-06T02:14:35.143 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.146+0000 7f3d61d74640 1 -- start start 2023-11-06T02:14:35.144 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.146+0000 7f3d61d74640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d5c0714a0 0x7f3d5c15cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:35.144 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.146+0000 7f3d61d74640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d5c0b1f90 0x7f3d5c0b7c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:35.144 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.146+0000 7f3d61d74640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3d5c0cda70 0x7f3d5c15a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:35.144 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.146+0000 7f3d61d74640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d5c151a20 con 0x7f3d5c0714a0 2023-11-06T02:14:35.144 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.146+0000 7f3d61d74640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d5c0b8740 con 0x7f3d5c0b1f90 2023-11-06T02:14:35.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.146+0000 7f3d61d74640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d5c0b8460 con 0x7f3d5c0cda70 2023-11-06T02:14:35.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.146+0000 7f3d5b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d5c0714a0 0x7f3d5c15cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:35.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.146+0000 7f3d5b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d5c0714a0 0x7f3d5c15cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38114/0 (socket says 172.21.15.49:38114) 2023-11-06T02:14:35.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.146+0000 7f3d5b7fe640 1 -- 172.21.15.49:0/2613706443 learned_addr learned my addr 172.21.15.49:0/2613706443 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:35.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.146+0000 7f3d53fff640 1 --2- 172.21.15.49:0/2613706443 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d5c0b1f90 0x7f3d5c0b7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:35.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.146+0000 7f3d5bfff640 1 --2- 172.21.15.49:0/2613706443 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3d5c0cda70 0x7f3d5c15a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:35.146 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.146+0000 7f3d5b7fe640 1 -- 172.21.15.49:0/2613706443 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3d5c0cda70 msgr2=0x7f3d5c15a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:35.146 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.146+0000 7f3d5b7fe640 1 --2- 172.21.15.49:0/2613706443 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3d5c0cda70 0x7f3d5c15a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.146 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.146+0000 7f3d5b7fe640 1 -- 172.21.15.49:0/2613706443 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d5c0b1f90 msgr2=0x7f3d5c0b7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:35.146 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.146+0000 7f3d5b7fe640 1 --2- 172.21.15.49:0/2613706443 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d5c0b1f90 0x7f3d5c0b7c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.146+0000 7f3d5b7fe640 1 -- 172.21.15.49:0/2613706443 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d440094d0 con 0x7f3d5c0714a0 2023-11-06T02:14:35.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.146+0000 7f3d53fff640 1 --2- 172.21.15.49:0/2613706443 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d5c0b1f90 0x7f3d5c0b7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:14:35.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.146+0000 7f3d5bfff640 1 --2- 172.21.15.49:0/2613706443 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3d5c0cda70 0x7f3d5c15a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:14:35.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.146+0000 7f3d5b7fe640 1 --2- 172.21.15.49:0/2613706443 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d5c0714a0 0x7f3d5c15cdf0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c00c500 tx=0x7f3d4c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:35.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.146+0000 7f3d58ff9640 1 -- 172.21.15.49:0/2613706443 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c005220 con 0x7f3d5c0714a0 2023-11-06T02:14:35.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.146+0000 7f3d58ff9640 1 -- 172.21.15.49:0/2613706443 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d4c005380 con 0x7f3d5c0714a0 2023-11-06T02:14:35.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.146+0000 7f3d61d74640 1 -- 172.21.15.49:0/2613706443 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d5c0b2410 con 0x7f3d5c0714a0 2023-11-06T02:14:35.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.146+0000 7f3d58ff9640 1 -- 172.21.15.49:0/2613706443 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c007270 con 0x7f3d5c0714a0 2023-11-06T02:14:35.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.146+0000 7f3d61d74640 1 -- 172.21.15.49:0/2613706443 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d5c0b29d0 con 0x7f3d5c0714a0 2023-11-06T02:14:35.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.146+0000 7f3d61d74640 1 -- 172.21.15.49:0/2613706443 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d5c074040 con 0x7f3d5c0714a0 2023-11-06T02:14:35.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.146+0000 7f3d58ff9640 1 -- 172.21.15.49:0/2613706443 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f3d4c0073d0 con 0x7f3d5c0714a0 2023-11-06T02:14:35.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.150+0000 7f3d58ff9640 1 --2- 172.21.15.49:0/2613706443 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3d3c065a60 0x7f3d3c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:35.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.150+0000 7f3d58ff9640 1 -- 172.21.15.49:0/2613706443 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f3d4c082cb0 con 0x7f3d5c0714a0 2023-11-06T02:14:35.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.150+0000 7f3d53fff640 1 --2- 172.21.15.49:0/2613706443 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3d3c065a60 0x7f3d3c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:35.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.150+0000 7f3d53fff640 1 --2- 172.21.15.49:0/2613706443 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3d3c065a60 0x7f3d3c067f20 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f3d440af000 tx=0x7f3d44008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:35.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.154+0000 7f3d58ff9640 1 -- 172.21.15.49:0/2613706443 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f3d4c0504c0 con 0x7f3d5c0714a0 2023-11-06T02:14:35.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.342+0000 7f3d61d74640 1 -- 172.21.15.49:0/2613706443 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f3d5c0d0c30 con 0x7f3d5c0714a0 2023-11-06T02:14:35.341 INFO:tasks.ceph.mon.a.smithi049.stderr:2023-11-06T02:14:35.346+0000 7f0164e5a640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-06T02:14:35.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.362+0000 7f3d58ff9640 1 -- 172.21.15.49:0/2613706443 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v53) v1 ==== 161+0+0 (secure 0 0 0) 0x7f3d4c053ae0 con 0x7f3d5c0714a0 2023-11-06T02:14:35.359 INFO:teuthology.orchestra.run.smithi049.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-06T02:14:35.359 INFO:tasks.ceph.mds.c.smithi049.stderr: -2150> 2023-11-06T02:14:20.350+0000 7f1efeaa9640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-06T02:14:35.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.366+0000 7f3d61d74640 1 -- 172.21.15.49:0/2613706443 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3d3c065a60 msgr2=0x7f3d3c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:35.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.366+0000 7f3d61d74640 1 --2- 172.21.15.49:0/2613706443 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3d3c065a60 0x7f3d3c067f20 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f3d440af000 tx=0x7f3d44008040 comp rx=0 tx=0).stop 2023-11-06T02:14:35.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.366+0000 7f3d61d74640 1 -- 172.21.15.49:0/2613706443 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d5c0714a0 msgr2=0x7f3d5c15cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:35.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.366+0000 7f3d61d74640 1 --2- 172.21.15.49:0/2613706443 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d5c0714a0 0x7f3d5c15cdf0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c00c500 tx=0x7f3d4c00c9c0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.365 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.370+0000 7f3d61d74640 1 -- 172.21.15.49:0/2613706443 shutdown_connections 2023-11-06T02:14:35.366 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.370+0000 7f3d61d74640 1 --2- 172.21.15.49:0/2613706443 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3d3c065a60 0x7f3d3c067f20 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.366 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.370+0000 7f3d61d74640 1 --2- 172.21.15.49:0/2613706443 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3d5c0cda70 0x7f3d5c15a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.366 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.370+0000 7f3d61d74640 1 --2- 172.21.15.49:0/2613706443 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d5c0b1f90 0x7f3d5c0b7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.366 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.370+0000 7f3d61d74640 1 --2- 172.21.15.49:0/2613706443 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d5c0714a0 0x7f3d5c15cdf0 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.366 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.370+0000 7f3d61d74640 1 -- 172.21.15.49:0/2613706443 >> 172.21.15.49:0/2613706443 conn(0x7f3d5c0d2aa0 msgr2=0x7f3d5c0c71d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:35.367 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.370+0000 7f3d61d74640 1 -- 172.21.15.49:0/2613706443 shutdown_connections 2023-11-06T02:14:35.367 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.370+0000 7f3d61d74640 1 -- 172.21.15.49:0/2613706443 wait complete. 2023-11-06T02:14:35.383 DEBUG:teuthology.orchestra.run.smithi049:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 journal reset --force 2023-11-06T02:14:35.389 INFO:tasks.ceph.mds.c.smithi049.stdout:starting mds.c at 2023-11-06T02:14:35.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.422+0000 7f5ec7ebf280 1 -- start start 2023-11-06T02:14:35.417 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.422+0000 7f5ec7ebf280 1 Processor -- start 2023-11-06T02:14:35.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.422+0000 7f5ec7ebf280 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563b47b8c000 0x563b46e45080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:35.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.422+0000 7f5ec7ebf280 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563b46db9c00 0x563b46e45b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:35.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.422+0000 7f5ec7ebf280 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b46db9800 0x563b46e45600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:35.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.422+0000 7f5ec7ebf280 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b46e86280 con 0x563b46db9800 2023-11-06T02:14:35.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.422+0000 7f5ec7ebf280 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b46e863c0 con 0x563b47b8c000 2023-11-06T02:14:35.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.422+0000 7f5ec7ebf280 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x563b46e86500 con 0x563b46db9c00 2023-11-06T02:14:35.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.422+0000 7f5ec6e80640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b46db9800 0x563b46e45600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:35.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.422+0000 7f5ec6e80640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b46db9800 0x563b46e45600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38154/0 (socket says 172.21.15.49:38154) 2023-11-06T02:14:35.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.422+0000 7f5ec6e80640 1 -- 172.21.15.49:0/13810490 learned_addr learned my addr 172.21.15.49:0/13810490 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:35.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.422+0000 7f5ec667f640 1 --2- 172.21.15.49:0/13810490 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563b47b8c000 0x563b46e45080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:35.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.422+0000 7f5ec5e7e640 1 --2- 172.21.15.49:0/13810490 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563b46db9c00 0x563b46e45b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:35.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.422+0000 7f5ec6e80640 1 -- 172.21.15.49:0/13810490 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563b46db9c00 msgr2=0x563b46e45b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:35.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.422+0000 7f5ec6e80640 1 --2- 172.21.15.49:0/13810490 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563b46db9c00 0x563b46e45b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.427 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.422+0000 7f5ec6e80640 1 -- 172.21.15.49:0/13810490 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563b47b8c000 msgr2=0x563b46e45080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:35.427 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.422+0000 7f5ec6e80640 1 --2- 172.21.15.49:0/13810490 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563b47b8c000 0x563b46e45080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.427 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.422+0000 7f5ec6e80640 1 -- 172.21.15.49:0/13810490 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b46deeea0 con 0x563b46db9800 2023-11-06T02:14:35.428 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.422+0000 7f5ec6e80640 1 --2- 172.21.15.49:0/13810490 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b46db9800 0x563b46e45600 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x563b46e715c0 tx=0x563b47b91140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b276465e1c38494 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:35.428 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.422+0000 7f5ec3e7a640 1 -- 172.21.15.49:0/13810490 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563b46e1d080 con 0x563b46db9800 2023-11-06T02:14:35.428 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.422+0000 7f5ec3e7a640 1 -- 172.21.15.49:0/13810490 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563b46e14480 con 0x563b46db9800 2023-11-06T02:14:35.428 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.422+0000 7f5ec3e7a640 1 -- 172.21.15.49:0/13810490 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563b46e1d600 con 0x563b46db9800 2023-11-06T02:14:35.429 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.422+0000 7f5ec7ebf280 1 -- 172.21.15.49:0/13810490 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563b46def520 con 0x563b46db9800 2023-11-06T02:14:35.429 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.422+0000 7f5ec7ebf280 1 -- 172.21.15.49:0/13810490 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x563b46def6c0 con 0x563b46db9800 2023-11-06T02:14:35.430 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.422+0000 7f5ec7ebf280 4 waiting for MDS map... 2023-11-06T02:14:35.430 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.422+0000 7f5ec3e7a640 1 -- 172.21.15.49:0/13810490 <== mon.0 v2:172.21.15.49:3300/0 4 ==== osd_map(82..82 src has 1..82) v4 ==== 6234+0+0 (secure 0 0 0) 0x563b46df8c40 con 0x563b46db9800 2023-11-06T02:14:35.430 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.422+0000 7f5ec3e7a640 1 -- 172.21.15.49:0/13810490 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 53) v1 ==== 2565+0+0 (secure 0 0 0) 0x563b46e06f00 con 0x563b46db9800 2023-11-06T02:14:35.431 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.422+0000 7f5ec7ebf280 4 Got MDS map 53 2023-11-06T02:14:35.431 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.422+0000 7f5ec7ebf280 10 main: JournalTool::main 2023-11-06T02:14:35.431 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.422+0000 7f5ec7ebf280 4 main: JournalTool: connecting to RADOS... 2023-11-06T02:14:35.432 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.426+0000 7f5ec7ebf280 1 Processor -- start 2023-11-06T02:14:35.432 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.426+0000 7f5ec7ebf280 1 -- start start 2023-11-06T02:14:35.432 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.426+0000 7f5ec7ebf280 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563b47b8c400 0x563b46e44b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:35.433 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.426+0000 7f5ec7ebf280 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b47cde800 0x563b46e46100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:35.433 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.426+0000 7f5ec7ebf280 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563b47cdec00 0x563b46e46680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:35.433 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.426+0000 7f5ec7ebf280 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b46dffe00 con 0x563b47cde800 2023-11-06T02:14:35.433 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.426+0000 7f5ec7ebf280 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b46e863c0 con 0x563b47b8c400 2023-11-06T02:14:35.434 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.426+0000 7f5ec7ebf280 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x563b46e86280 con 0x563b47cdec00 2023-11-06T02:14:35.434 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.426+0000 7f5ec667f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b47cde800 0x563b46e46100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:35.434 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.426+0000 7f5ec667f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b47cde800 0x563b46e46100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38160/0 (socket says 172.21.15.49:38160) 2023-11-06T02:14:35.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.426+0000 7f5ec667f640 1 -- 172.21.15.49:0/4274600493 learned_addr learned my addr 172.21.15.49:0/4274600493 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:35.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.426+0000 7f5ec5e7e640 1 --2- 172.21.15.49:0/4274600493 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563b47cdec00 0x563b46e46680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:35.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.426+0000 7f5ec5e7e640 1 --2- 172.21.15.49:0/4274600493 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563b47b8c400 0x563b46e44b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:35.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.426+0000 7f5ec667f640 1 -- 172.21.15.49:0/4274600493 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563b47cdec00 msgr2=0x563b46e46680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:35.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.426+0000 7f5ec667f640 1 --2- 172.21.15.49:0/4274600493 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563b47cdec00 0x563b46e46680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.426+0000 7f5ec667f640 1 -- 172.21.15.49:0/4274600493 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563b47b8c400 msgr2=0x563b46e44b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:35.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.426+0000 7f5ec667f640 1 --2- 172.21.15.49:0/4274600493 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563b47b8c400 0x563b46e44b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.426+0000 7f5ec667f640 1 -- 172.21.15.49:0/4274600493 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b46def860 con 0x563b47cde800 2023-11-06T02:14:35.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.426+0000 7f5ec667f640 1 --2- 172.21.15.49:0/4274600493 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b47cde800 0x563b46e46100 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x563b4709e000 tx=0x563b47cf2c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afbddd99f54a9df9 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:35.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.426+0000 7f5ec1e76640 1 -- 172.21.15.49:0/4274600493 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563b46e1d080 con 0x563b47cde800 2023-11-06T02:14:35.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.426+0000 7f5ec1e76640 1 -- 172.21.15.49:0/4274600493 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563b47cc6300 con 0x563b47cde800 2023-11-06T02:14:35.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.426+0000 7f5ec1e76640 1 -- 172.21.15.49:0/4274600493 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563b46e1d600 con 0x563b47cde800 2023-11-06T02:14:35.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.426+0000 7f5ec7ebf280 1 -- 172.21.15.49:0/4274600493 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b47cde800 msgr2=0x563b46e46100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:35.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.426+0000 7f5ec7ebf280 1 --2- 172.21.15.49:0/4274600493 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b47cde800 0x563b46e46100 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x563b4709e000 tx=0x563b47cf2c00 comp rx=0 tx=0).stop 2023-11-06T02:14:35.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.426+0000 7f5ec7ebf280 1 -- 172.21.15.49:0/4274600493 shutdown_connections 2023-11-06T02:14:35.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.426+0000 7f5ec7ebf280 1 --2- 172.21.15.49:0/4274600493 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b47cde800 0x563b46e46100 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.426+0000 7f5ec7ebf280 1 --2- 172.21.15.49:0/4274600493 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563b47cdec00 0x563b46e46680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.426+0000 7f5ec7ebf280 1 --2- 172.21.15.49:0/4274600493 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563b47b8c400 0x563b46e44b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.426+0000 7f5ec7ebf280 1 -- 172.21.15.49:0/4274600493 >> 172.21.15.49:0/4274600493 conn(0x563b47b8c800 msgr2=0x563b47d36000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:35.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.426+0000 7f5ec7ebf280 1 -- 172.21.15.49:0/4274600493 shutdown_connections 2023-11-06T02:14:35.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.426+0000 7f5ec7ebf280 1 -- 172.21.15.49:0/4274600493 wait complete. 2023-11-06T02:14:35.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.426+0000 7f5ec7ebf280 1 Processor -- start 2023-11-06T02:14:35.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.426+0000 7f5ec7ebf280 1 -- start start 2023-11-06T02:14:35.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.426+0000 7f5ec7ebf280 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b47b8c800 0x563b46e44b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:35.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.426+0000 7f5ec7ebf280 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563b47cdf800 0x563b46e46100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:35.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.426+0000 7f5ec7ebf280 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563b47cdf400 0x563b46e46680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:35.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.426+0000 7f5ec7ebf280 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b46e86780 con 0x563b47b8c800 2023-11-06T02:14:35.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.426+0000 7f5ec7ebf280 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b46e86140 con 0x563b47cdf800 2023-11-06T02:14:35.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.426+0000 7f5ec7ebf280 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x563b46e86000 con 0x563b47cdf400 2023-11-06T02:14:35.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.426+0000 7f5ec5e7e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b47b8c800 0x563b46e44b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:35.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.426+0000 7f5ec5e7e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b47b8c800 0x563b46e44b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38176/0 (socket says 172.21.15.49:38176) 2023-11-06T02:14:35.443 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.426+0000 7f5ec5e7e640 1 -- 172.21.15.49:0/3079992012 learned_addr learned my addr 172.21.15.49:0/3079992012 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:35.443 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.426+0000 7f5ec5e7e640 1 --2- 172.21.15.49:0/3079992012 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563b47cdf400 0x563b46e46680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:35.443 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.426+0000 7f5ec667f640 1 --2- 172.21.15.49:0/3079992012 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563b47cdf800 0x563b46e46100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:35.444 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.426+0000 7f5ec5e7e640 1 -- 172.21.15.49:0/3079992012 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563b47cdf400 msgr2=0x563b46e46680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:35.444 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.426+0000 7f5ec5e7e640 1 --2- 172.21.15.49:0/3079992012 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563b47cdf400 0x563b46e46680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.444 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.426+0000 7f5ec5e7e640 1 -- 172.21.15.49:0/3079992012 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563b47cdf800 msgr2=0x563b46e46100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:35.445 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.426+0000 7f5ec5e7e640 1 --2- 172.21.15.49:0/3079992012 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563b47cdf800 0x563b46e46100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.445 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.426+0000 7f5ec5e7e640 1 -- 172.21.15.49:0/3079992012 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b46dee340 con 0x563b47b8c800 2023-11-06T02:14:35.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.426+0000 7f5ec667f640 1 --2- 172.21.15.49:0/3079992012 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563b47cdf800 0x563b46e46100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:14:35.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.426+0000 7f5ec5e7e640 1 --2- 172.21.15.49:0/3079992012 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b47b8c800 0x563b46e44b00 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x563b4709f020 tx=0x563b47b90420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:35.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.426+0000 7f5ec0673640 1 -- 172.21.15.49:0/3079992012 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563b46e1cf20 con 0x563b47b8c800 2023-11-06T02:14:35.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.430+0000 7f5ec0673640 1 -- 172.21.15.49:0/3079992012 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563b47cc7680 con 0x563b47b8c800 2023-11-06T02:14:35.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.430+0000 7f5ec7ebf280 1 -- 172.21.15.49:0/3079992012 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x563b46def1e0 con 0x563b47b8c800 2023-11-06T02:14:35.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.430+0000 7f5ec0673640 1 -- 172.21.15.49:0/3079992012 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563b46e1d080 con 0x563b47b8c800 2023-11-06T02:14:35.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.430+0000 7f5ec7ebf280 1 -- 172.21.15.49:0/3079992012 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563b46deeea0 con 0x563b47b8c800 2023-11-06T02:14:35.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.430+0000 7f5ec7ebf280 4 main: JournalTool: resolving pool 11 2023-11-06T02:14:35.453 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.430+0000 7f5ec0673640 1 -- 172.21.15.49:0/3079992012 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x563b46e83080 con 0x563b47b8c800 2023-11-06T02:14:35.454 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.430+0000 7f5ec0673640 1 --2- 172.21.15.49:0/3079992012 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x563b47b92800 0x563b46e47700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:35.454 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.430+0000 7f5ec0673640 1 -- 172.21.15.49:0/3079992012 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6234+0+0 (secure 0 0 0) 0x563b46df8a80 con 0x563b47b8c800 2023-11-06T02:14:35.454 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.430+0000 7f5ec7ebf280 4 main: JournalTool: creating IoCtx.. 2023-11-06T02:14:35.455 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.430+0000 7f5ec7ebf280 4 main: Executing for rank 0 2023-11-06T02:14:35.455 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.430+0000 7f5ec667f640 1 --2- 172.21.15.49:0/3079992012 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x563b47b92800 0x563b46e47700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:35.455 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.430+0000 7f5ec7ebf280 1 -- start start 2023-11-06T02:14:35.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.430+0000 7f5ec7ebf280 1 Processor -- start 2023-11-06T02:14:35.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.430+0000 7f5ec667f640 1 --2- 172.21.15.49:0/3079992012 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x563b47b92800 0x563b46e47700 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x563b4709e000 tx=0x563b47b90120 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:35.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.430+0000 7f5ec7ebf280 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b47cdec00 0x563b46e46c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:35.457 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.430+0000 7f5ec7ebf280 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563b47cde800 0x563b46e47180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:35.457 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.430+0000 7f5ec7ebf280 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563b47b92c00 0x563b471ea000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:35.457 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.430+0000 7f5ec7ebf280 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b46e86a00 con 0x563b47cdec00 2023-11-06T02:14:35.458 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.430+0000 7f5ec7ebf280 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b46e86000 con 0x563b47cde800 2023-11-06T02:14:35.458 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.430+0000 7f5ec7ebf280 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x563b46e86280 con 0x563b47b92c00 2023-11-06T02:14:35.458 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.430+0000 7f5ec6e80640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b47cdec00 0x563b46e46c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:35.459 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.430+0000 7f5ec6e80640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b47cdec00 0x563b46e46c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38184/0 (socket says 172.21.15.49:38184) 2023-11-06T02:14:35.459 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.430+0000 7f5ec6e80640 1 -- 172.21.15.49:0/1160813852 learned_addr learned my addr 172.21.15.49:0/1160813852 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:35.459 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.430+0000 7f5ec667f640 1 --2- 172.21.15.49:0/1160813852 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563b47cde800 0x563b46e47180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:35.460 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.430+0000 7f5ec5e7e640 1 --2- 172.21.15.49:0/1160813852 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563b47b92c00 0x563b471ea000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:35.460 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.430+0000 7f5ec6e80640 1 -- 172.21.15.49:0/1160813852 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563b47b92c00 msgr2=0x563b471ea000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:35.461 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.430+0000 7f5ec6e80640 1 --2- 172.21.15.49:0/1160813852 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563b47b92c00 0x563b471ea000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.461 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.430+0000 7f5ec6e80640 1 -- 172.21.15.49:0/1160813852 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563b47cde800 msgr2=0x563b46e47180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:35.461 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.430+0000 7f5ec6e80640 1 --2- 172.21.15.49:0/1160813852 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563b47cde800 0x563b46e47180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.461 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.430+0000 7f5ec6e80640 1 -- 172.21.15.49:0/1160813852 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b470d2d00 con 0x563b47cdec00 2023-11-06T02:14:35.462 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.430+0000 7f5ec6e80640 1 --2- 172.21.15.49:0/1160813852 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b47cdec00 0x563b46e46c00 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x563b47156ed0 tx=0x563b47b91080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=151deff2ec710c8e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:35.462 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.430+0000 7f5ebd66d640 1 -- 172.21.15.49:0/1160813852 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563b46e1c2c0 con 0x563b47cdec00 2023-11-06T02:14:35.462 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.434+0000 7f5ebd66d640 1 -- 172.21.15.49:0/1160813852 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563b47cc6780 con 0x563b47cdec00 2023-11-06T02:14:35.462 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.434+0000 7f5ebd66d640 1 -- 172.21.15.49:0/1160813852 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563b46e1c6e0 con 0x563b47cdec00 2023-11-06T02:14:35.463 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.434+0000 7f5ec7ebf280 1 -- 172.21.15.49:0/1160813852 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563b470d2b60 con 0x563b47cdec00 2023-11-06T02:14:35.463 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.434+0000 7f5ec7ebf280 1 -- 172.21.15.49:0/1160813852 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x563b470d29c0 con 0x563b47cdec00 2023-11-06T02:14:35.463 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.434+0000 7f5ec7ebf280 4 waiting for MDS map... 2023-11-06T02:14:35.463 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.434+0000 7f5ebd66d640 1 -- 172.21.15.49:0/1160813852 <== mon.0 v2:172.21.15.49:3300/0 4 ==== osd_map(82..82 src has 1..82) v4 ==== 6234+0+0 (secure 0 0 0) 0x563b46df88c0 con 0x563b47cdec00 2023-11-06T02:14:35.463 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.434+0000 7f5ebd66d640 1 -- 172.21.15.49:0/1160813852 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 53) v1 ==== 2565+0+0 (secure 0 0 0) 0x563b471db800 con 0x563b47cdec00 2023-11-06T02:14:35.463 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.434+0000 7f5ec7ebf280 4 Got MDS map 53 2023-11-06T02:14:35.464 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.434+0000 7f5ec7ebf280 1 --2- 172.21.15.49:0/1160813852 >> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] conn(0x563b47b93000 0x563b471ea580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:35.464 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.434+0000 7f5ec7ebf280 1 -- 172.21.15.49:0/1160813852 --> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] -- osd_op(unknown.0.0:1 11.f 11:f1f69726:::400.00000000:head [writefull 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82) v8 -- 0x563b47b93400 con 0x563b47b93000 2023-11-06T02:14:35.464 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.434+0000 7f5ec667f640 1 --2- 172.21.15.49:0/1160813852 >> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] conn(0x563b47b93000 0x563b471ea580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:35.464 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.434+0000 7f5ec667f640 1 --2- 172.21.15.49:0/1160813852 >> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] conn(0x563b47b93000 0x563b471ea580 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=f99397e1ab665f78 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:35.465 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.438+0000 7f5ec667f640 1 -- 172.21.15.49:0/1160813852 <== osd.7 v2:172.21.15.150:6816/38421 1 ==== osd_op_reply(1 400.00000000 [writefull 0~22] v82'2 uv2 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x563b47b9d200 con 0x563b47b93000 2023-11-06T02:14:35.465 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.438+0000 7f5ec7ebf280 1 --2- 172.21.15.49:0/1160813852 >> [v2:172.21.15.150:6824/38428,v1:172.21.15.150:6825/38428] conn(0x563b47b93800 0x563b471eab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:35.465 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.438+0000 7f5ec7ebf280 1 -- 172.21.15.49:0/1160813852 --> [v2:172.21.15.150:6824/38428,v1:172.21.15.150:6825/38428] -- osd_op(unknown.0.0:2 11.14 11:292cf221:::200.00000000:head [writefull 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82) v8 -- 0x563b47b93c00 con 0x563b47b93800 2023-11-06T02:14:35.465 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.438+0000 7f5ec5e7e640 1 --2- 172.21.15.49:0/1160813852 >> [v2:172.21.15.150:6824/38428,v1:172.21.15.150:6825/38428] conn(0x563b47b93800 0x563b471eab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:35.466 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.438+0000 7f5ec5e7e640 1 --2- 172.21.15.49:0/1160813852 >> [v2:172.21.15.150:6824/38428,v1:172.21.15.150:6825/38428] conn(0x563b47b93800 0x563b471eab00 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=e157034329d5f3a5 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:35.466 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.442+0000 7f5ec5e7e640 1 -- 172.21.15.49:0/1160813852 <== osd.6 v2:172.21.15.150:6824/38428 1 ==== osd_op_reply(2 200.00000000 [writefull 0~90] v82'42 uv42 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x563b47b9d440 con 0x563b47b93800 2023-11-06T02:14:35.467 INFO:teuthology.orchestra.run.smithi049.stdout:writing EResetJournal entry 2023-11-06T02:14:35.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.442+0000 7f5ec7ebf280 1 -- 172.21.15.49:0/1160813852 --> [v2:172.21.15.150:6824/38428,v1:172.21.15.150:6825/38428] -- osd_op(unknown.0.0:3 11.14 11:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82) v8 -- 0x563b471f4c00 con 0x563b47b93800 2023-11-06T02:14:35.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.442+0000 7f5ec7ebf280 1 --2- 172.21.15.49:0/1160813852 >> [v2:172.21.15.150:6808/38417,v1:172.21.15.150:6809/38417] conn(0x563b471f5000 0x563b471eb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:35.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.442+0000 7f5ec7ebf280 1 -- 172.21.15.49:0/1160813852 --> [v2:172.21.15.150:6808/38417,v1:172.21.15.150:6809/38417] -- osd_op(unknown.0.0:4 11.1b 11:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82) v8 -- 0x563b471f5400 con 0x563b471f5000 2023-11-06T02:14:35.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.442+0000 7f5ec7ebf280 1 --2- 172.21.15.49:0/1160813852 >> [v2:172.21.15.49:6824/38115,v1:172.21.15.49:6825/38115] conn(0x563b471f5800 0x563b471eb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:35.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.442+0000 7f5ec7ebf280 1 -- 172.21.15.49:0/1160813852 --> [v2:172.21.15.49:6824/38115,v1:172.21.15.49:6825/38115] -- osd_op(unknown.0.0:5 11.b 11:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82) v8 -- 0x563b471f5c00 con 0x563b471f5800 2023-11-06T02:14:35.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.442+0000 7f5ec7ebf280 1 --2- 172.21.15.49:0/1160813852 >> [v2:172.21.15.49:6800/38114,v1:172.21.15.49:6801/38114] conn(0x563b47216000 0x563b471ebb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:35.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.442+0000 7f5ec7ebf280 1 -- 172.21.15.49:0/1160813852 --> [v2:172.21.15.49:6800/38114,v1:172.21.15.49:6801/38114] -- osd_op(unknown.0.0:6 11.1f 11:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82) v8 -- 0x563b47216400 con 0x563b47216000 2023-11-06T02:14:35.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.442+0000 7f5ec7ebf280 1 --2- 172.21.15.49:0/1160813852 >> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] conn(0x563b47216800 0x563b471ec100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:35.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.442+0000 7f5ec7ebf280 1 -- 172.21.15.49:0/1160813852 --> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] -- osd_op(unknown.0.0:7 11.1a 11:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82) v8 -- 0x563b47216c00 con 0x563b47216800 2023-11-06T02:14:35.477 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.442+0000 7f5ec7ebf280 1 -- 172.21.15.49:0/1160813852 --> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] -- osd_op(unknown.0.0:8 11.1e 11:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82) v8 -- 0x563b47217000 con 0x563b47b93000 2023-11-06T02:14:35.477 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.442+0000 7f5ec5e7e640 1 --2- 172.21.15.49:0/1160813852 >> [v2:172.21.15.49:6800/38114,v1:172.21.15.49:6801/38114] conn(0x563b47216000 0x563b471ebb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:35.477 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.442+0000 7f5ec667f640 1 --2- 172.21.15.49:0/1160813852 >> [v2:172.21.15.49:6824/38115,v1:172.21.15.49:6825/38115] conn(0x563b471f5800 0x563b471eb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:35.478 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.442+0000 7f5ec6e80640 1 --2- 172.21.15.49:0/1160813852 >> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] conn(0x563b47216800 0x563b471ec100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:35.478 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.442+0000 7f5ec6e80640 1 --2- 172.21.15.49:0/1160813852 >> [v2:172.21.15.150:6808/38417,v1:172.21.15.150:6809/38417] conn(0x563b471f5000 0x563b471eb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:35.478 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.442+0000 7f5ec5e7e640 1 --2- 172.21.15.49:0/1160813852 >> [v2:172.21.15.49:6800/38114,v1:172.21.15.49:6801/38114] conn(0x563b47216000 0x563b471ebb80 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=fc76be673dc5d585 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:35.478 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.442+0000 7f5ec667f640 1 --2- 172.21.15.49:0/1160813852 >> [v2:172.21.15.49:6824/38115,v1:172.21.15.49:6825/38115] conn(0x563b471f5800 0x563b471eb600 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=d4d0790ab23708f7 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:35.479 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.442+0000 7f5ec6e80640 1 --2- 172.21.15.49:0/1160813852 >> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] conn(0x563b47216800 0x563b471ec100 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=121b586eba28b150 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:35.479 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.442+0000 7f5ec6e80640 1 --2- 172.21.15.49:0/1160813852 >> [v2:172.21.15.150:6808/38417,v1:172.21.15.150:6809/38417] conn(0x563b471f5000 0x563b471eb080 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=8dfbde5e6d9b0000 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:35.479 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.446+0000 7f5ec5e7e640 1 -- 172.21.15.49:0/1160813852 <== osd.2 v2:172.21.15.49:6800/38114 1 ==== osd_op_reply(6 200.00000004 [delete] v82'9 uv8 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x563b47b9d440 con 0x563b47216000 2023-11-06T02:14:35.479 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.446+0000 7f5ec5e7e640 1 -- 172.21.15.49:0/1160813852 <== osd.6 v2:172.21.15.150:6824/38428 2 ==== osd_op_reply(3 200.00000001 [delete] v82'43 uv43 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x563b47b9d440 con 0x563b47b93800 2023-11-06T02:14:35.480 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.446+0000 7f5ec667f640 1 -- 172.21.15.49:0/1160813852 <== osd.3 v2:172.21.15.49:6824/38115 1 ==== osd_op_reply(5 200.00000003 [delete] v82'6 uv5 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x563b47b9d200 con 0x563b471f5800 2023-11-06T02:14:35.480 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.446+0000 7f5ec667f640 1 -- 172.21.15.49:0/1160813852 <== osd.7 v2:172.21.15.150:6816/38421 2 ==== osd_op_reply(8 200.00000006 [delete] v82'3 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x563b47b9d200 con 0x563b47b93000 2023-11-06T02:14:35.480 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.450+0000 7f5ec6e80640 1 -- 172.21.15.49:0/1160813852 <== osd.5 v2:172.21.15.150:6800/38418 1 ==== osd_op_reply(7 200.00000005 [delete] v82'3 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x563b47cce000 con 0x563b47216800 2023-11-06T02:14:35.481 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.450+0000 7f5ec6e80640 1 -- 172.21.15.49:0/1160813852 <== osd.4 v2:172.21.15.150:6808/38417 1 ==== osd_op_reply(4 200.00000002 [delete] v82'3 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x563b47cce000 con 0x563b471f5000 2023-11-06T02:14:35.481 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.450+0000 7f5ebbe6a640 1 -- 172.21.15.49:0/1160813852 --> [v2:172.21.15.150:6824/38428,v1:172.21.15.150:6825/38428] -- osd_op(unknown.0.0:9 11.14 11:2e2fa760:::200.00000001:head [write 0~48 in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82) v8 -- 0x563b47b92400 con 0x563b47b93800 2023-11-06T02:14:35.482 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.454+0000 7f5ec5e7e640 1 -- 172.21.15.49:0/1160813852 <== osd.6 v2:172.21.15.150:6824/38428 3 ==== osd_op_reply(9 200.00000001 [write 0~48] v82'44 uv44 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x563b47b9d440 con 0x563b47b93800 2023-11-06T02:14:35.482 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.454+0000 7f5ec7ebf280 4 Successfully wrote new journal pointer and header for rank 5:0 2023-11-06T02:14:35.482 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.454+0000 7f5ec7ebf280 1 -- 172.21.15.49:0/1160813852 >> [v2:172.21.15.49:6800/38114,v1:172.21.15.49:6801/38114] conn(0x563b47216000 msgr2=0x563b471ebb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:35.483 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.454+0000 7f5ec7ebf280 1 --2- 172.21.15.49:0/1160813852 >> [v2:172.21.15.49:6800/38114,v1:172.21.15.49:6801/38114] conn(0x563b47216000 0x563b471ebb80 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.483 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.454+0000 7f5ec7ebf280 1 -- 172.21.15.49:0/1160813852 >> [v2:172.21.15.49:6824/38115,v1:172.21.15.49:6825/38115] conn(0x563b471f5800 msgr2=0x563b471eb600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:35.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.454+0000 7f5ec7ebf280 1 --2- 172.21.15.49:0/1160813852 >> [v2:172.21.15.49:6824/38115,v1:172.21.15.49:6825/38115] conn(0x563b471f5800 0x563b471eb600 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.454+0000 7f5ec7ebf280 1 -- 172.21.15.49:0/1160813852 >> [v2:172.21.15.150:6808/38417,v1:172.21.15.150:6809/38417] conn(0x563b471f5000 msgr2=0x563b471eb080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:35.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.454+0000 7f5ec7ebf280 1 --2- 172.21.15.49:0/1160813852 >> [v2:172.21.15.150:6808/38417,v1:172.21.15.150:6809/38417] conn(0x563b471f5000 0x563b471eb080 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.454+0000 7f5ec7ebf280 1 -- 172.21.15.49:0/1160813852 >> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] conn(0x563b47216800 msgr2=0x563b471ec100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:35.485 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.454+0000 7f5ec7ebf280 1 --2- 172.21.15.49:0/1160813852 >> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] conn(0x563b47216800 0x563b471ec100 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.485 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.454+0000 7f5ec5e7e640 1 -- 172.21.15.49:0/1160813852 reap_dead start 2023-11-06T02:14:35.486 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.454+0000 7f5ec7ebf280 1 -- 172.21.15.49:0/1160813852 >> [v2:172.21.15.150:6824/38428,v1:172.21.15.150:6825/38428] conn(0x563b47b93800 msgr2=0x563b471eab00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:35.486 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.454+0000 7f5ec7ebf280 1 --2- 172.21.15.49:0/1160813852 >> [v2:172.21.15.150:6824/38428,v1:172.21.15.150:6825/38428] conn(0x563b47b93800 0x563b471eab00 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.486 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.454+0000 7f5ec7ebf280 1 -- 172.21.15.49:0/1160813852 >> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] conn(0x563b47b93000 msgr2=0x563b471ea580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:35.487 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.454+0000 7f5ec7ebf280 1 --2- 172.21.15.49:0/1160813852 >> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] conn(0x563b47b93000 0x563b471ea580 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.487 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.454+0000 7f5ec7ebf280 1 -- 172.21.15.49:0/1160813852 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b47cdec00 msgr2=0x563b46e46c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:35.488 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.454+0000 7f5ec7ebf280 1 --2- 172.21.15.49:0/1160813852 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b47cdec00 0x563b46e46c00 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x563b47156ed0 tx=0x563b47b91080 comp rx=0 tx=0).stop 2023-11-06T02:14:35.488 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.454+0000 7f5ec7ebf280 1 -- 172.21.15.49:0/1160813852 shutdown_connections 2023-11-06T02:14:35.488 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.454+0000 7f5ec7ebf280 1 --2- 172.21.15.49:0/1160813852 >> [v2:172.21.15.150:6824/38428,v1:172.21.15.150:6825/38428] conn(0x563b47b93800 0x563b471eab00 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.488 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.454+0000 7f5ec7ebf280 1 --2- 172.21.15.49:0/1160813852 >> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] conn(0x563b47b93000 0x563b471ea580 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.489 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.454+0000 7f5ec7ebf280 1 --2- 172.21.15.49:0/1160813852 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b47cdec00 0x563b46e46c00 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.489 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.454+0000 7f5ec7ebf280 1 -- 172.21.15.49:0/1160813852 >> 172.21.15.49:0/1160813852 conn(0x563b47cdf000 msgr2=0x563b47d36800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:35.489 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.454+0000 7f5ec7ebf280 1 -- 172.21.15.49:0/1160813852 shutdown_connections 2023-11-06T02:14:35.490 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.454+0000 7f5ec7ebf280 1 -- 172.21.15.49:0/1160813852 wait complete. 2023-11-06T02:14:35.490 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.454+0000 7f5ec7ebf280 1 -- 172.21.15.49:0/3079992012 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x563b47b92800 msgr2=0x563b46e47700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:35.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.454+0000 7f5ec7ebf280 1 --2- 172.21.15.49:0/3079992012 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x563b47b92800 0x563b46e47700 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x563b4709e000 tx=0x563b47b90120 comp rx=0 tx=0).stop 2023-11-06T02:14:35.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.454+0000 7f5ec7ebf280 1 -- 172.21.15.49:0/3079992012 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b47b8c800 msgr2=0x563b46e44b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:35.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.454+0000 7f5ec7ebf280 1 --2- 172.21.15.49:0/3079992012 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b47b8c800 0x563b46e44b00 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x563b4709f020 tx=0x563b47b90420 comp rx=0 tx=0).stop 2023-11-06T02:14:35.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.454+0000 7f5ec7ebf280 1 -- 172.21.15.49:0/3079992012 shutdown_connections 2023-11-06T02:14:35.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.454+0000 7f5ec7ebf280 1 --2- 172.21.15.49:0/3079992012 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x563b47b92800 0x563b46e47700 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.454+0000 7f5ec7ebf280 1 --2- 172.21.15.49:0/3079992012 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563b47cdf400 0x563b46e46680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.454+0000 7f5ec7ebf280 1 --2- 172.21.15.49:0/3079992012 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563b47cdf800 0x563b46e46100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.454+0000 7f5ec7ebf280 1 --2- 172.21.15.49:0/3079992012 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b47b8c800 0x563b46e44b00 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.454+0000 7f5ec7ebf280 1 -- 172.21.15.49:0/3079992012 >> 172.21.15.49:0/3079992012 conn(0x563b47b8c400 msgr2=0x563b47d36000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:35.493 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.454+0000 7f5ec7ebf280 1 -- 172.21.15.49:0/3079992012 shutdown_connections 2023-11-06T02:14:35.493 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.454+0000 7f5ec7ebf280 1 -- 172.21.15.49:0/3079992012 wait complete. 2023-11-06T02:14:35.493 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.454+0000 7f5ec7ebf280 1 -- 172.21.15.49:0/13810490 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b46db9800 msgr2=0x563b46e45600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:35.493 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.454+0000 7f5ec7ebf280 1 --2- 172.21.15.49:0/13810490 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b46db9800 0x563b46e45600 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x563b46e715c0 tx=0x563b47b91140 comp rx=0 tx=0).stop 2023-11-06T02:14:35.493 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.454+0000 7f5ec7ebf280 1 -- 172.21.15.49:0/13810490 shutdown_connections 2023-11-06T02:14:35.494 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.454+0000 7f5ec7ebf280 1 --2- 172.21.15.49:0/13810490 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b46db9800 0x563b46e45600 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.494 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.454+0000 7f5ec7ebf280 1 --2- 172.21.15.49:0/13810490 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x563b46db9c00 0x563b46e45b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.494 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.454+0000 7f5ec7ebf280 1 --2- 172.21.15.49:0/13810490 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563b47b8c000 0x563b46e45080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.494 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.454+0000 7f5ec7ebf280 1 -- 172.21.15.49:0/13810490 >> 172.21.15.49:0/13810490 conn(0x563b46db8c00 msgr2=0x563b46de7800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:35.494 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.454+0000 7f5ec7ebf280 1 -- 172.21.15.49:0/13810490 shutdown_connections 2023-11-06T02:14:35.495 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.454+0000 7f5ec7ebf280 1 -- 172.21.15.49:0/13810490 wait complete. 2023-11-06T02:14:35.495 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'journal', 'reset', '--force'] in time 0:00:00.111653, result: writing EResetJournal entry 2023-11-06T02:14:35.496 DEBUG:teuthology.orchestra.run.smithi049:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents cephfs_data 2023-11-06T02:14:35.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.558+0000 7fc31420c740 1 -- start start 2023-11-06T02:14:35.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.558+0000 7fc31420c740 1 Processor -- start 2023-11-06T02:14:35.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.562+0000 7fc31420c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562de53b1800 0x562de543c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:35.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.562+0000 7fc31420c740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562de53b1400 0x562de543cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:35.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.562+0000 7fc31420c740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562de53b1000 0x562de543d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:35.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.562+0000 7fc31420c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x562de53f7e00 con 0x562de53b1800 2023-11-06T02:14:35.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.562+0000 7fc31420c740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x562de547ea00 con 0x562de53b1400 2023-11-06T02:14:35.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.562+0000 7fc31420c740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x562de547e640 con 0x562de53b1000 2023-11-06T02:14:35.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.562+0000 7fc3129cc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562de53b1800 0x562de543c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:35.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.562+0000 7fc3129cc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562de53b1800 0x562de543c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38194/0 (socket says 172.21.15.49:38194) 2023-11-06T02:14:35.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.562+0000 7fc3129cc640 1 -- 172.21.15.49:0/353727567 learned_addr learned my addr 172.21.15.49:0/353727567 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:35.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.562+0000 7fc3131cd640 1 --2- 172.21.15.49:0/353727567 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562de53b1000 0x562de543d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:35.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.562+0000 7fc3121cb640 1 --2- 172.21.15.49:0/353727567 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562de53b1400 0x562de543cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:35.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.562+0000 7fc3129cc640 1 -- 172.21.15.49:0/353727567 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562de53b1000 msgr2=0x562de543d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:35.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.562+0000 7fc3129cc640 1 --2- 172.21.15.49:0/353727567 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562de53b1000 0x562de543d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.562+0000 7fc3129cc640 1 -- 172.21.15.49:0/353727567 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562de53b1400 msgr2=0x562de543cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:35.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.562+0000 7fc3129cc640 1 --2- 172.21.15.49:0/353727567 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562de53b1400 0x562de543cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.562+0000 7fc3129cc640 1 -- 172.21.15.49:0/353727567 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562de53e51e0 con 0x562de53b1800 2023-11-06T02:14:35.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.562+0000 7fc3121cb640 1 --2- 172.21.15.49:0/353727567 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562de53b1400 0x562de543cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:14:35.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.562+0000 7fc3129cc640 1 --2- 172.21.15.49:0/353727567 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562de53b1800 0x562de543c580 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x562de6207170 tx=0x562de620cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=783f13338910aaf9 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:35.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.562+0000 7fc3101c7640 1 -- 172.21.15.49:0/353727567 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562de5415600 con 0x562de53b1800 2023-11-06T02:14:35.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.562+0000 7fc3101c7640 1 -- 172.21.15.49:0/353727567 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562de540c900 con 0x562de53b1800 2023-11-06T02:14:35.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.562+0000 7fc3101c7640 1 -- 172.21.15.49:0/353727567 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562de54151e0 con 0x562de53b1800 2023-11-06T02:14:35.563 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.562+0000 7fc31420c740 1 -- 172.21.15.49:0/353727567 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562de53e49c0 con 0x562de53b1800 2023-11-06T02:14:35.563 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.562+0000 7fc31420c740 1 -- 172.21.15.49:0/353727567 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x562de53e4b60 con 0x562de53b1800 2023-11-06T02:14:35.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.562+0000 7fc31420c740 4 waiting for MDS map... 2023-11-06T02:14:35.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.562+0000 7fc3101c7640 1 -- 172.21.15.49:0/353727567 <== mon.0 v2:172.21.15.49:3300/0 4 ==== osd_map(82..82 src has 1..82) v4 ==== 6234+0+0 (secure 0 0 0) 0x562de53f0c40 con 0x562de53b1800 2023-11-06T02:14:35.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.562+0000 7fc3101c7640 1 -- 172.21.15.49:0/353727567 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 53) v1 ==== 2565+0+0 (secure 0 0 0) 0x562de53ff200 con 0x562de53b1800 2023-11-06T02:14:35.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.562+0000 7fc31420c740 4 Got MDS map 53 2023-11-06T02:14:35.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.562+0000 7fc31420c740 4 datascan.main: Using metadata pool output 2023-11-06T02:14:35.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.562+0000 7fc31420c740 4 datascan.main: connecting to RADOS... 2023-11-06T02:14:35.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.562+0000 7fc31420c740 1 Processor -- start 2023-11-06T02:14:35.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.562+0000 7fc31420c740 1 -- start start 2023-11-06T02:14:35.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.562+0000 7fc31420c740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562de62b8c00 0x562de543d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:35.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.562+0000 7fc31420c740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562de62b9000 0x562de543db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:35.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.562+0000 7fc31420c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562de62b9400 0x562de543e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:35.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.562+0000 7fc31420c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x562de547e500 con 0x562de62b9400 2023-11-06T02:14:35.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.562+0000 7fc31420c740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x562de547e3c0 con 0x562de62b8c00 2023-11-06T02:14:35.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.562+0000 7fc31420c740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x562de547e640 con 0x562de62b9000 2023-11-06T02:14:35.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.562+0000 7fc3121cb640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562de62b9400 0x562de543e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:35.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.562+0000 7fc3121cb640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562de62b9400 0x562de543e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38198/0 (socket says 172.21.15.49:38198) 2023-11-06T02:14:35.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.562+0000 7fc3121cb640 1 -- 172.21.15.49:0/3026730067 learned_addr learned my addr 172.21.15.49:0/3026730067 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:35.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.562+0000 7fc3129cc640 1 --2- 172.21.15.49:0/3026730067 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562de62b9000 0x562de543db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:35.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.562+0000 7fc3131cd640 1 --2- 172.21.15.49:0/3026730067 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562de62b8c00 0x562de543d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:35.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.562+0000 7fc3121cb640 1 -- 172.21.15.49:0/3026730067 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562de62b9000 msgr2=0x562de543db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:35.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.562+0000 7fc3121cb640 1 --2- 172.21.15.49:0/3026730067 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562de62b9000 0x562de543db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.579 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.562+0000 7fc3121cb640 1 -- 172.21.15.49:0/3026730067 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562de62b8c00 msgr2=0x562de543d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:35.579 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.562+0000 7fc3121cb640 1 --2- 172.21.15.49:0/3026730067 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562de62b8c00 0x562de543d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.579 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.562+0000 7fc3121cb640 1 -- 172.21.15.49:0/3026730067 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562de53e4ea0 con 0x562de62b9400 2023-11-06T02:14:35.579 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.562+0000 7fc3129cc640 1 --2- 172.21.15.49:0/3026730067 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562de62b9000 0x562de543db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:14:35.579 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.562+0000 7fc3131cd640 1 --2- 172.21.15.49:0/3026730067 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562de62b8c00 0x562de543d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:14:35.580 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.566+0000 7fc3121cb640 1 --2- 172.21.15.49:0/3026730067 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562de62b9400 0x562de543e100 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x562de62069f0 tx=0x562de620c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=908a24e6f8d33f94 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:35.580 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.566+0000 7fc30e1c3640 1 -- 172.21.15.49:0/3026730067 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562de5415600 con 0x562de62b9400 2023-11-06T02:14:35.580 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.566+0000 7fc30e1c3640 1 -- 172.21.15.49:0/3026730067 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562de6168600 con 0x562de62b9400 2023-11-06T02:14:35.580 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.566+0000 7fc30e1c3640 1 -- 172.21.15.49:0/3026730067 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562de54151e0 con 0x562de62b9400 2023-11-06T02:14:35.581 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.566+0000 7fc31420c740 1 -- 172.21.15.49:0/3026730067 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562de62b9400 msgr2=0x562de543e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:35.581 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.566+0000 7fc31420c740 1 --2- 172.21.15.49:0/3026730067 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562de62b9400 0x562de543e100 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x562de62069f0 tx=0x562de620c0c0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.581 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.566+0000 7fc31420c740 1 -- 172.21.15.49:0/3026730067 shutdown_connections 2023-11-06T02:14:35.581 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.566+0000 7fc31420c740 1 --2- 172.21.15.49:0/3026730067 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562de62b9400 0x562de543e100 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.581 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.566+0000 7fc31420c740 1 --2- 172.21.15.49:0/3026730067 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562de62b9000 0x562de543db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.582 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.566+0000 7fc31420c740 1 --2- 172.21.15.49:0/3026730067 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562de62b8c00 0x562de543d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.582 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.566+0000 7fc31420c740 1 -- 172.21.15.49:0/3026730067 >> 172.21.15.49:0/3026730067 conn(0x562de53b0c00 msgr2=0x562de53db000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:35.582 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.566+0000 7fc31420c740 1 -- 172.21.15.49:0/3026730067 shutdown_connections 2023-11-06T02:14:35.582 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.566+0000 7fc31420c740 1 -- 172.21.15.49:0/3026730067 wait complete. 2023-11-06T02:14:35.583 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.566+0000 7fc31420c740 1 Processor -- start 2023-11-06T02:14:35.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.566+0000 7fc31420c740 1 -- start start 2023-11-06T02:14:35.585 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.566+0000 7fc31420c740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562de62b8c00 0x562de543f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:35.585 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.566+0000 7fc31420c740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562de62b9000 0x562de543ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:35.585 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.566+0000 7fc31420c740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562de62b9400 0x562de543e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:35.586 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.566+0000 7fc31420c740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x562de547ea00 con 0x562de62b8c00 2023-11-06T02:14:35.586 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.566+0000 7fc31420c740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x562de547e280 con 0x562de62b9000 2023-11-06T02:14:35.586 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.566+0000 7fc31420c740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x562de547e140 con 0x562de62b9400 2023-11-06T02:14:35.586 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.566+0000 7fc3131cd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562de62b8c00 0x562de543f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:35.586 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.566+0000 7fc3131cd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562de62b8c00 0x562de543f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38206/0 (socket says 172.21.15.49:38206) 2023-11-06T02:14:35.587 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.566+0000 7fc3131cd640 1 -- 172.21.15.49:0/3018643979 learned_addr learned my addr 172.21.15.49:0/3018643979 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:35.587 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.566+0000 7fc3129cc640 1 --2- 172.21.15.49:0/3018643979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562de62b9000 0x562de543ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:35.587 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.566+0000 7fc3121cb640 1 --2- 172.21.15.49:0/3018643979 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562de62b9400 0x562de543e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:35.587 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.566+0000 7fc3131cd640 1 -- 172.21.15.49:0/3018643979 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562de62b9400 msgr2=0x562de543e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:35.587 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.566+0000 7fc3131cd640 1 --2- 172.21.15.49:0/3018643979 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562de62b9400 0x562de543e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.588 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.566+0000 7fc3131cd640 1 -- 172.21.15.49:0/3018643979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562de62b9000 msgr2=0x562de543ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:35.588 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.566+0000 7fc3131cd640 1 --2- 172.21.15.49:0/3018643979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562de62b9000 0x562de543ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.588 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.566+0000 7fc3131cd640 1 -- 172.21.15.49:0/3018643979 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562de53e4340 con 0x562de62b8c00 2023-11-06T02:14:35.588 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.566+0000 7fc3131cd640 1 --2- 172.21.15.49:0/3018643979 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562de62b8c00 0x562de543f180 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x562de61732f0 tx=0x562de62dac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:35.588 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.566+0000 7fc30c9c0640 1 -- 172.21.15.49:0/3018643979 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562de5415600 con 0x562de62b8c00 2023-11-06T02:14:35.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.566+0000 7fc30c9c0640 1 -- 172.21.15.49:0/3018643979 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562de540c480 con 0x562de62b8c00 2023-11-06T02:14:35.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.566+0000 7fc31420c740 1 -- 172.21.15.49:0/3018643979 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x562de53e51e0 con 0x562de62b8c00 2023-11-06T02:14:35.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.566+0000 7fc30c9c0640 1 -- 172.21.15.49:0/3018643979 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562de54151e0 con 0x562de62b8c00 2023-11-06T02:14:35.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.566+0000 7fc31420c740 1 -- 172.21.15.49:0/3018643979 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562de53e49c0 con 0x562de62b8c00 2023-11-06T02:14:35.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.566+0000 7fc31420c740 4 datascan.init: resolving metadata pool 11 2023-11-06T02:14:35.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.570+0000 7fc30c9c0640 1 -- 172.21.15.49:0/3018643979 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x562de547a840 con 0x562de62b8c00 2023-11-06T02:14:35.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.570+0000 7fc30c9c0640 1 --2- 172.21.15.49:0/3018643979 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x562de5692c00 0x562de543f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:35.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.570+0000 7fc30c9c0640 1 -- 172.21.15.49:0/3018643979 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6234+0+0 (secure 0 0 0) 0x562de53f0a80 con 0x562de62b8c00 2023-11-06T02:14:35.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.570+0000 7fc31420c740 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-11-06T02:14:35.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.570+0000 7fc31420c740 4 datascan.main: data pool 'cephfs_data' has ID 12 2023-11-06T02:14:35.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.570+0000 7fc31420c740 4 datascan.main: opening data pool 'cephfs_data' 2023-11-06T02:14:35.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.570+0000 7fc31420c740 1 --2- 172.21.15.49:0/3018643979 >> [v2:172.21.15.150:6808/38417,v1:172.21.15.150:6809/38417] conn(0x562de62b9800 0x562de543e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:35.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.570+0000 7fc31420c740 1 -- 172.21.15.49:0/3018643979 --> [v2:172.21.15.150:6808/38417,v1:172.21.15.150:6809/38417] -- osd_op(unknown.0.0:1 12.0 12:00000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x562de62b9c00 con 0x562de62b9800 2023-11-06T02:14:35.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.570+0000 7fc3129cc640 1 --2- 172.21.15.49:0/3018643979 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x562de5692c00 0x562de543f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:35.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.570+0000 7fc3121cb640 1 --2- 172.21.15.49:0/3018643979 >> [v2:172.21.15.150:6808/38417,v1:172.21.15.150:6809/38417] conn(0x562de62b9800 0x562de543e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:35.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.570+0000 7fc3129cc640 1 --2- 172.21.15.49:0/3018643979 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x562de5692c00 0x562de543f700 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x562de6207650 tx=0x562de620cd80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:35.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.570+0000 7fc3121cb640 1 --2- 172.21.15.49:0/3018643979 >> [v2:172.21.15.150:6808/38417,v1:172.21.15.150:6809/38417] conn(0x562de62b9800 0x562de543e100 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:35.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.570+0000 7fc3121cb640 1 -- 172.21.15.49:0/3018643979 <== osd.4 v2:172.21.15.150:6808/38417 1 ==== osd_op_reply(1 [pgnls start_epoch 82 out=49b] v80'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562de62a8fc0 con 0x562de62b9800 2023-11-06T02:14:35.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.570+0000 7fc30d9c2640 1 --2- 172.21.15.49:0/3018643979 >> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] conn(0x562de6175400 0x562de578e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:35.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.570+0000 7fc30d9c2640 1 -- 172.21.15.49:0/3018643979 --> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] -- osd_op(unknown.0.0:2 12.20 12:04000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x562de6175000 con 0x562de6175400 2023-11-06T02:14:35.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.570+0000 7fc3131cd640 1 --2- 172.21.15.49:0/3018643979 >> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] conn(0x562de6175400 0x562de578e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:35.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.574+0000 7fc3131cd640 1 --2- 172.21.15.49:0/3018643979 >> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] conn(0x562de6175400 0x562de578e000 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:35.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.574+0000 7fc3131cd640 1 -- 172.21.15.49:0/3018643979 <== osd.7 v2:172.21.15.150:6816/38421 1 ==== osd_op_reply(2 [pgnls start_epoch 82 out=49b] v80'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562de62a9200 con 0x562de6175400 2023-11-06T02:14:35.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.574+0000 7fc30e1c3640 1 --2- 172.21.15.49:0/3018643979 >> [v2:172.21.15.49:6809/38116,v1:172.21.15.49:6811/38116] conn(0x562de6174800 0x562de578e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:35.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.574+0000 7fc30e1c3640 1 -- 172.21.15.49:0/3018643979 --> [v2:172.21.15.49:6809/38116,v1:172.21.15.49:6811/38116] -- osd_op(unknown.0.0:3 12.10 12:08000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x562de5693000 con 0x562de6174800 2023-11-06T02:14:35.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.574+0000 7fc3129cc640 1 --2- 172.21.15.49:0/3018643979 >> [v2:172.21.15.49:6809/38116,v1:172.21.15.49:6811/38116] conn(0x562de6174800 0x562de578e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:35.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.574+0000 7fc3129cc640 1 --2- 172.21.15.49:0/3018643979 >> [v2:172.21.15.49:6809/38116,v1:172.21.15.49:6811/38116] conn(0x562de6174800 0x562de578e580 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:35.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.574+0000 7fc3129cc640 1 -- 172.21.15.49:0/3018643979 <== osd.0 v2:172.21.15.49:6809/38116 1 ==== osd_op_reply(3 [pgnls start_epoch 82 out=49b] v80'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562de62a9440 con 0x562de6174800 2023-11-06T02:14:35.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.574+0000 7fc30d9c2640 1 -- 172.21.15.49:0/3018643979 --> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] -- osd_op(unknown.0.0:4 12.30 12:0c000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x562de6174c00 con 0x562de6175400 2023-11-06T02:14:35.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.574+0000 7fc3131cd640 1 -- 172.21.15.49:0/3018643979 <== osd.7 v2:172.21.15.150:6816/38421 2 ==== osd_op_reply(4 [pgnls start_epoch 82 out=49b] v80'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562de62a9200 con 0x562de6175400 2023-11-06T02:14:35.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.574+0000 7fc30e1c3640 1 --2- 172.21.15.49:0/3018643979 >> [v2:172.21.15.49:6824/38115,v1:172.21.15.49:6825/38115] conn(0x562de62b9c00 0x562de578eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:35.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.574+0000 7fc30e1c3640 1 -- 172.21.15.49:0/3018643979 --> [v2:172.21.15.49:6824/38115,v1:172.21.15.49:6825/38115] -- osd_op(unknown.0.0:5 12.8 12:10000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x562de5693400 con 0x562de62b9c00 2023-11-06T02:14:35.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.574+0000 7fc3121cb640 1 --2- 172.21.15.49:0/3018643979 >> [v2:172.21.15.49:6824/38115,v1:172.21.15.49:6825/38115] conn(0x562de62b9c00 0x562de578eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:35.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.574+0000 7fc3121cb640 1 --2- 172.21.15.49:0/3018643979 >> [v2:172.21.15.49:6824/38115,v1:172.21.15.49:6825/38115] conn(0x562de62b9c00 0x562de578eb00 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:35.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.578+0000 7fc3121cb640 1 -- 172.21.15.49:0/3018643979 <== osd.3 v2:172.21.15.49:6824/38115 1 ==== osd_op_reply(5 [pgnls start_epoch 82 out=49b] v80'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562de62a8fc0 con 0x562de62b9c00 2023-11-06T02:14:35.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.578+0000 7fc30d9c2640 1 -- 172.21.15.49:0/3018643979 --> [v2:172.21.15.49:6809/38116,v1:172.21.15.49:6811/38116] -- osd_op(unknown.0.0:6 12.28 12:14000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x562de57a0000 con 0x562de6174800 2023-11-06T02:14:35.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.578+0000 7fc3129cc640 1 -- 172.21.15.49:0/3018643979 <== osd.0 v2:172.21.15.49:6809/38116 2 ==== osd_op_reply(6 [pgnls start_epoch 82 out=49b] v80'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562de62a9440 con 0x562de6174800 2023-11-06T02:14:35.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.578+0000 7fc30e1c3640 1 --2- 172.21.15.49:0/3018643979 >> [v2:172.21.15.49:6808/38113,v1:172.21.15.49:6810/38113] conn(0x562de5693800 0x562de578f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:35.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.578+0000 7fc30e1c3640 1 -- 172.21.15.49:0/3018643979 --> [v2:172.21.15.49:6808/38113,v1:172.21.15.49:6810/38113] -- osd_op(unknown.0.0:7 12.18 12:18000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x562de5693c00 con 0x562de5693800 2023-11-06T02:14:35.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.578+0000 7fc3131cd640 1 --2- 172.21.15.49:0/3018643979 >> [v2:172.21.15.49:6808/38113,v1:172.21.15.49:6810/38113] conn(0x562de5693800 0x562de578f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:35.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.578+0000 7fc3131cd640 1 --2- 172.21.15.49:0/3018643979 >> [v2:172.21.15.49:6808/38113,v1:172.21.15.49:6810/38113] conn(0x562de5693800 0x562de578f080 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:35.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.578+0000 7fc3131cd640 1 -- 172.21.15.49:0/3018643979 <== osd.1 v2:172.21.15.49:6808/38113 1 ==== osd_op_reply(7 [pgnls start_epoch 82 out=49b] v80'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562de62a9200 con 0x562de5693800 2023-11-06T02:14:35.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.578+0000 7fc30d9c2640 1 --2- 172.21.15.49:0/3018643979 >> [v2:172.21.15.49:6800/38114,v1:172.21.15.49:6801/38114] conn(0x562de57a0400 0x562de578f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:35.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.578+0000 7fc30d9c2640 1 -- 172.21.15.49:0/3018643979 --> [v2:172.21.15.49:6800/38114,v1:172.21.15.49:6801/38114] -- osd_op(unknown.0.0:8 12.38 12:1c000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x562de57a0800 con 0x562de57a0400 2023-11-06T02:14:35.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.578+0000 7fc3129cc640 1 --2- 172.21.15.49:0/3018643979 >> [v2:172.21.15.49:6800/38114,v1:172.21.15.49:6801/38114] conn(0x562de57a0400 0x562de578f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:35.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.578+0000 7fc3129cc640 1 --2- 172.21.15.49:0/3018643979 >> [v2:172.21.15.49:6800/38114,v1:172.21.15.49:6801/38114] conn(0x562de57a0400 0x562de578f600 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:35.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.578+0000 7fc3129cc640 1 -- 172.21.15.49:0/3018643979 <== osd.2 v2:172.21.15.49:6800/38114 1 ==== osd_op_reply(8 [pgnls start_epoch 82 out=49b] v80'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562de62a9440 con 0x562de57a0400 2023-11-06T02:14:35.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.578+0000 7fc30e1c3640 1 --2- 172.21.15.49:0/3018643979 >> [v2:172.21.15.150:6824/38428,v1:172.21.15.150:6825/38428] conn(0x562de57a0c00 0x562de5790100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:35.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.578+0000 7fc30e1c3640 1 -- 172.21.15.49:0/3018643979 --> [v2:172.21.15.150:6824/38428,v1:172.21.15.150:6825/38428] -- osd_op(unknown.0.0:9 12.4 12:20000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x562de57a1000 con 0x562de57a0c00 2023-11-06T02:14:35.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.578+0000 7fc3121cb640 1 --2- 172.21.15.49:0/3018643979 >> [v2:172.21.15.150:6824/38428,v1:172.21.15.150:6825/38428] conn(0x562de57a0c00 0x562de5790100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:35.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.578+0000 7fc3121cb640 1 --2- 172.21.15.49:0/3018643979 >> [v2:172.21.15.150:6824/38428,v1:172.21.15.150:6825/38428] conn(0x562de57a0c00 0x562de5790100 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:35.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.582+0000 7fc3121cb640 1 -- 172.21.15.49:0/3018643979 <== osd.6 v2:172.21.15.150:6824/38428 1 ==== osd_op_reply(9 [pgnls start_epoch 82 out=49b] v80'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562de62a8fc0 con 0x562de57a0c00 2023-11-06T02:14:35.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.582+0000 7fc30d9c2640 1 -- 172.21.15.49:0/3018643979 --> [v2:172.21.15.49:6800/38114,v1:172.21.15.49:6801/38114] -- osd_op(unknown.0.0:10 12.24 12:24000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x562de57a1c00 con 0x562de57a0400 2023-11-06T02:14:35.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.582+0000 7fc3129cc640 1 -- 172.21.15.49:0/3018643979 <== osd.2 v2:172.21.15.49:6800/38114 2 ==== osd_op_reply(10 [pgnls start_epoch 82 out=49b] v80'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562de62a9440 con 0x562de57a0400 2023-11-06T02:14:35.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.582+0000 7fc30e1c3640 1 --2- 172.21.15.49:0/3018643979 >> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] conn(0x562de57a1400 0x562de5790680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:35.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.582+0000 7fc30e1c3640 1 -- 172.21.15.49:0/3018643979 --> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] -- osd_op(unknown.0.0:11 12.14 12:28000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x562de57a1800 con 0x562de57a1400 2023-11-06T02:14:35.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.582+0000 7fc3131cd640 1 --2- 172.21.15.49:0/3018643979 >> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] conn(0x562de57a1400 0x562de5790680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:35.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.582+0000 7fc3131cd640 1 --2- 172.21.15.49:0/3018643979 >> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] conn(0x562de57a1400 0x562de5790680 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:35.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.582+0000 7fc3131cd640 1 -- 172.21.15.49:0/3018643979 <== osd.5 v2:172.21.15.150:6800/38418 1 ==== osd_op_reply(11 [pgnls start_epoch 82 out=49b] v80'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562de62a9200 con 0x562de57a1400 2023-11-06T02:14:35.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.582+0000 7fc30d9c2640 1 -- 172.21.15.49:0/3018643979 --> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] -- osd_op(unknown.0.0:12 12.34 12:2c000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x562de57a1800 con 0x562de57a1400 2023-11-06T02:14:35.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.582+0000 7fc3131cd640 1 -- 172.21.15.49:0/3018643979 <== osd.5 v2:172.21.15.150:6800/38418 2 ==== osd_op_reply(12 [pgnls start_epoch 82 out=49b] v80'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562de62a9200 con 0x562de57a1400 2023-11-06T02:14:35.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.582+0000 7fc30e1c3640 1 -- 172.21.15.49:0/3018643979 --> [v2:172.21.15.49:6809/38116,v1:172.21.15.49:6811/38116] -- osd_op(unknown.0.0:13 12.c 12:30000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x562de57b4000 con 0x562de6174800 2023-11-06T02:14:35.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.582+0000 7fc3129cc640 1 -- 172.21.15.49:0/3018643979 <== osd.0 v2:172.21.15.49:6809/38116 3 ==== osd_op_reply(13 [pgnls start_epoch 82 out=49b] v80'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562de62a9440 con 0x562de6174800 2023-11-06T02:14:35.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.582+0000 7fc30d9c2640 1 -- 172.21.15.49:0/3018643979 --> [v2:172.21.15.49:6800/38114,v1:172.21.15.49:6801/38114] -- osd_op(unknown.0.0:14 12.2c 12:34000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x562de5693c00 con 0x562de57a0400 2023-11-06T02:14:35.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.582+0000 7fc3129cc640 1 -- 172.21.15.49:0/3018643979 <== osd.2 v2:172.21.15.49:6800/38114 3 ==== osd_op_reply(14 [pgnls start_epoch 82 out=49b] v80'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562de62a9440 con 0x562de57a0400 2023-11-06T02:14:35.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.582+0000 7fc30e1c3640 1 -- 172.21.15.49:0/3018643979 --> [v2:172.21.15.49:6809/38116,v1:172.21.15.49:6811/38116] -- osd_op(unknown.0.0:15 12.1c 12:38000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x562de57b4400 con 0x562de6174800 2023-11-06T02:14:35.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.586+0000 7fc3129cc640 1 -- 172.21.15.49:0/3018643979 <== osd.0 v2:172.21.15.49:6809/38116 4 ==== osd_op_reply(15 [pgnls start_epoch 82 out=49b] v80'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562de62a9440 con 0x562de6174800 2023-11-06T02:14:35.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.586+0000 7fc30d9c2640 1 -- 172.21.15.49:0/3018643979 --> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] -- osd_op(unknown.0.0:16 12.3c 12:3c000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x562de6175000 con 0x562de57a1400 2023-11-06T02:14:35.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.586+0000 7fc3131cd640 1 -- 172.21.15.49:0/3018643979 <== osd.5 v2:172.21.15.150:6800/38418 3 ==== osd_op_reply(16 [pgnls start_epoch 82 out=49b] v80'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562de62a9200 con 0x562de57a1400 2023-11-06T02:14:35.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.586+0000 7fc30e1c3640 1 -- 172.21.15.49:0/3018643979 --> [v2:172.21.15.150:6808/38417,v1:172.21.15.150:6809/38417] -- osd_op(unknown.0.0:17 12.2 12:40000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x562de57b4800 con 0x562de62b9800 2023-11-06T02:14:35.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.586+0000 7fc3121cb640 1 -- 172.21.15.49:0/3018643979 <== osd.4 v2:172.21.15.150:6808/38417 2 ==== osd_op_reply(17 [pgnls start_epoch 82 out=49b] v80'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562de62a8fc0 con 0x562de62b9800 2023-11-06T02:14:35.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.586+0000 7fc30d9c2640 1 -- 172.21.15.49:0/3018643979 --> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] -- osd_op(unknown.0.0:18 12.22 12:44000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x562de6174c00 con 0x562de6175400 2023-11-06T02:14:35.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.586+0000 7fc3131cd640 1 -- 172.21.15.49:0/3018643979 <== osd.7 v2:172.21.15.150:6816/38421 3 ==== osd_op_reply(18 [pgnls start_epoch 82 out=49b] v80'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562de62a9200 con 0x562de6175400 2023-11-06T02:14:35.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.586+0000 7fc30e1c3640 1 -- 172.21.15.49:0/3018643979 --> [v2:172.21.15.49:6824/38115,v1:172.21.15.49:6825/38115] -- osd_op(unknown.0.0:19 12.12 12:48000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x562de57b4c00 con 0x562de62b9c00 2023-11-06T02:14:35.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.586+0000 7fc3121cb640 1 -- 172.21.15.49:0/3018643979 <== osd.3 v2:172.21.15.49:6824/38115 2 ==== osd_op_reply(19 [pgnls start_epoch 82 out=81b] v81'22 uv22 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x562de62a8fc0 con 0x562de62b9c00 2023-11-06T02:14:35.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.586+0000 7fc31420c740 1 -- 172.21.15.49:0/3018643979 --> [v2:172.21.15.49:6824/38115,v1:172.21.15.49:6825/38115] -- osd_op(unknown.0.0:20 12.12 12:4bec3886:::10000000002.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82) v8 -- 0x562de57b4000 con 0x562de62b9c00 2023-11-06T02:14:35.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.586+0000 7fc3121cb640 1 -- 172.21.15.49:0/3018643979 <== osd.3 v2:172.21.15.49:6824/38115 3 ==== osd_op_reply(20 10000000002.00000000 [stat out=16b] v0'0 uv22 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x562de62a8fc0 con 0x562de62b9c00 2023-11-06T02:14:35.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.586+0000 7fc31420c740 10 datascan.operator(): handling object 1099511627778.0 2023-11-06T02:14:35.612 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.586+0000 7fc31420c740 1 -- 172.21.15.49:0/3018643979 --> [v2:172.21.15.49:6824/38115,v1:172.21.15.49:6825/38115] -- osd_op(unknown.0.0:21 12.12 12:4bec3886:::10000000002.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82) v8 -- 0x562de57b5400 con 0x562de62b9c00 2023-11-06T02:14:35.612 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.590+0000 7fc3121cb640 1 -- 172.21.15.49:0/3018643979 <== osd.3 v2:172.21.15.49:6824/38115 4 ==== osd_op_reply(21 10000000002.00000000 [call] v82'23 uv23 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562de62a8fc0 con 0x562de62b9c00 2023-11-06T02:14:35.612 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.590+0000 7fc31420c740 1 -- 172.21.15.49:0/3018643979 --> [v2:172.21.15.49:6824/38115,v1:172.21.15.49:6825/38115] -- osd_op(unknown.0.0:22 12.32 12:4c000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x562de57b5800 con 0x562de62b9c00 2023-11-06T02:14:35.612 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.590+0000 7fc3121cb640 1 -- 172.21.15.49:0/3018643979 <== osd.3 v2:172.21.15.49:6824/38115 5 ==== osd_op_reply(22 [pgnls start_epoch 82 out=49b] v80'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562de62a8fc0 con 0x562de62b9c00 2023-11-06T02:14:35.612 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.590+0000 7fc30e1c3640 1 -- 172.21.15.49:0/3018643979 --> [v2:172.21.15.49:6800/38114,v1:172.21.15.49:6801/38114] -- osd_op(unknown.0.0:23 12.a 12:50000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x562de57b5000 con 0x562de57a0400 2023-11-06T02:14:35.613 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.590+0000 7fc3129cc640 1 -- 172.21.15.49:0/3018643979 <== osd.2 v2:172.21.15.49:6800/38114 4 ==== osd_op_reply(23 [pgnls start_epoch 82 out=49b] v80'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562de62a9440 con 0x562de57a0400 2023-11-06T02:14:35.613 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.590+0000 7fc30d9c2640 1 -- 172.21.15.49:0/3018643979 --> [v2:172.21.15.49:6800/38114,v1:172.21.15.49:6801/38114] -- osd_op(unknown.0.0:24 12.2a 12:54000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x562de57b4800 con 0x562de57a0400 2023-11-06T02:14:35.613 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.590+0000 7fc3129cc640 1 -- 172.21.15.49:0/3018643979 <== osd.2 v2:172.21.15.49:6800/38114 5 ==== osd_op_reply(24 [pgnls start_epoch 82 out=49b] v80'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562de62a9440 con 0x562de57a0400 2023-11-06T02:14:35.613 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.590+0000 7fc30e1c3640 1 -- 172.21.15.49:0/3018643979 --> [v2:172.21.15.150:6808/38417,v1:172.21.15.150:6809/38417] -- osd_op(unknown.0.0:25 12.1a 12:58000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x562de57b5c00 con 0x562de62b9800 2023-11-06T02:14:35.613 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.590+0000 7fc3121cb640 1 -- 172.21.15.49:0/3018643979 <== osd.4 v2:172.21.15.150:6808/38417 3 ==== osd_op_reply(25 [pgnls start_epoch 82 out=49b] v80'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562de62a8fc0 con 0x562de62b9800 2023-11-06T02:14:35.614 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.590+0000 7fc30d9c2640 1 -- 172.21.15.49:0/3018643979 --> [v2:172.21.15.49:6824/38115,v1:172.21.15.49:6825/38115] -- osd_op(unknown.0.0:26 12.3a 12:5c000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x562de57b4c00 con 0x562de62b9c00 2023-11-06T02:14:35.614 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.590+0000 7fc3121cb640 1 -- 172.21.15.49:0/3018643979 <== osd.3 v2:172.21.15.49:6824/38115 6 ==== osd_op_reply(26 [pgnls start_epoch 82 out=49b] v80'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562de62a8fc0 con 0x562de62b9c00 2023-11-06T02:14:35.614 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.590+0000 7fc30e1c3640 1 -- 172.21.15.49:0/3018643979 --> [v2:172.21.15.49:6808/38113,v1:172.21.15.49:6810/38113] -- osd_op(unknown.0.0:27 12.6 12:60000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x562de57b6000 con 0x562de5693800 2023-11-06T02:14:35.614 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.590+0000 7fc3131cd640 1 -- 172.21.15.49:0/3018643979 <== osd.1 v2:172.21.15.49:6808/38113 2 ==== osd_op_reply(27 [pgnls start_epoch 82 out=49b] v80'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562de62a9200 con 0x562de5693800 2023-11-06T02:14:35.614 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.590+0000 7fc30d9c2640 1 -- 172.21.15.49:0/3018643979 --> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] -- osd_op(unknown.0.0:28 12.26 12:64000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x562de57b4000 con 0x562de57a1400 2023-11-06T02:14:35.615 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.594+0000 7fc3131cd640 1 -- 172.21.15.49:0/3018643979 <== osd.5 v2:172.21.15.150:6800/38418 4 ==== osd_op_reply(28 [pgnls start_epoch 82 out=49b] v80'30 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562de62a9200 con 0x562de57a1400 2023-11-06T02:14:35.615 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.594+0000 7fc30e1c3640 1 -- 172.21.15.49:0/3018643979 --> [v2:172.21.15.150:6824/38428,v1:172.21.15.150:6825/38428] -- osd_op(unknown.0.0:29 12.16 12:68000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x562de57b6400 con 0x562de57a0c00 2023-11-06T02:14:35.615 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.594+0000 7fc3121cb640 1 -- 172.21.15.49:0/3018643979 <== osd.6 v2:172.21.15.150:6824/38428 2 ==== osd_op_reply(29 [pgnls start_epoch 82 out=49b] v80'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562de62a8fc0 con 0x562de57a0c00 2023-11-06T02:14:35.615 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.594+0000 7fc30d9c2640 1 -- 172.21.15.49:0/3018643979 --> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] -- osd_op(unknown.0.0:30 12.36 12:6c000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x562de57b5400 con 0x562de57a1400 2023-11-06T02:14:35.615 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.594+0000 7fc3131cd640 1 -- 172.21.15.49:0/3018643979 <== osd.5 v2:172.21.15.150:6800/38418 5 ==== osd_op_reply(30 [pgnls start_epoch 82 out=81b] v80'33 uv17 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x562de62a9200 con 0x562de57a1400 2023-11-06T02:14:35.616 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.594+0000 7fc31420c740 1 -- 172.21.15.49:0/3018643979 --> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] -- osd_op(unknown.0.0:31 12.36 12:6fe38dc2:::100000001f7.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82) v8 -- 0x562de57a0000 con 0x562de57a1400 2023-11-06T02:14:35.616 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.594+0000 7fc3131cd640 1 -- 172.21.15.49:0/3018643979 <== osd.5 v2:172.21.15.150:6800/38418 6 ==== osd_op_reply(31 100000001f7.00000000 [stat out=16b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x562de62a9200 con 0x562de57a1400 2023-11-06T02:14:35.616 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.594+0000 7fc31420c740 10 datascan.operator(): handling object 1099511628279.0 2023-11-06T02:14:35.616 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.594+0000 7fc31420c740 1 -- 172.21.15.49:0/3018643979 --> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] -- osd_op(unknown.0.0:32 12.36 12:6fe38dc2:::100000001f7.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82) v8 -- 0x562de57a0800 con 0x562de57a1400 2023-11-06T02:14:35.616 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.598+0000 7fc3131cd640 1 -- 172.21.15.49:0/3018643979 <== osd.5 v2:172.21.15.150:6800/38418 7 ==== osd_op_reply(32 100000001f7.00000000 [call] v82'34 uv34 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562de62a9200 con 0x562de57a1400 2023-11-06T02:14:35.616 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.598+0000 7fc31420c740 1 -- 172.21.15.49:0/3018643979 --> [v2:172.21.15.49:6809/38116,v1:172.21.15.49:6811/38116] -- osd_op(unknown.0.0:33 12.e 12:70000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x562de57a1c00 con 0x562de6174800 2023-11-06T02:14:35.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.598+0000 7fc3129cc640 1 -- 172.21.15.49:0/3018643979 <== osd.0 v2:172.21.15.49:6809/38116 5 ==== osd_op_reply(33 [pgnls start_epoch 82 out=49b] v80'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562de62a9440 con 0x562de6174800 2023-11-06T02:14:35.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.598+0000 7fc30d9c2640 1 -- 172.21.15.49:0/3018643979 --> [v2:172.21.15.49:6808/38113,v1:172.21.15.49:6810/38113] -- osd_op(unknown.0.0:34 12.2e 12:74000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x562de57a1000 con 0x562de5693800 2023-11-06T02:14:35.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.598+0000 7fc3131cd640 1 -- 172.21.15.49:0/3018643979 <== osd.1 v2:172.21.15.49:6808/38113 3 ==== osd_op_reply(34 [pgnls start_epoch 82 out=49b] v80'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562de62a9200 con 0x562de5693800 2023-11-06T02:14:35.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.598+0000 7fc30e1c3640 1 -- 172.21.15.49:0/3018643979 --> [v2:172.21.15.150:6808/38417,v1:172.21.15.150:6809/38417] -- osd_op(unknown.0.0:35 12.1e 12:78000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x562de57b6800 con 0x562de62b9800 2023-11-06T02:14:35.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.602+0000 7fc3121cb640 1 -- 172.21.15.49:0/3018643979 <== osd.4 v2:172.21.15.150:6808/38417 4 ==== osd_op_reply(35 [pgnls start_epoch 82 out=49b] v80'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562de62a8fc0 con 0x562de62b9800 2023-11-06T02:14:35.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.602+0000 7fc30d9c2640 1 -- 172.21.15.49:0/3018643979 --> [v2:172.21.15.49:6809/38116,v1:172.21.15.49:6811/38116] -- osd_op(unknown.0.0:36 12.3e 12:7c000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x562de5693400 con 0x562de6174800 2023-11-06T02:14:35.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.602+0000 7fc3129cc640 1 -- 172.21.15.49:0/3018643979 <== osd.0 v2:172.21.15.49:6809/38116 6 ==== osd_op_reply(36 [pgnls start_epoch 82 out=49b] v80'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562de62a9440 con 0x562de6174800 2023-11-06T02:14:35.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.602+0000 7fc30e1c3640 1 -- 172.21.15.49:0/3018643979 --> [v2:172.21.15.150:6824/38428,v1:172.21.15.150:6825/38428] -- osd_op(unknown.0.0:37 12.1 12:80000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x562de57b6c00 con 0x562de57a0c00 2023-11-06T02:14:35.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.602+0000 7fc3121cb640 1 -- 172.21.15.49:0/3018643979 <== osd.6 v2:172.21.15.150:6824/38428 3 ==== osd_op_reply(37 [pgnls start_epoch 82 out=49b] v80'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562de62a8fc0 con 0x562de57a0c00 2023-11-06T02:14:35.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.602+0000 7fc30d9c2640 1 -- 172.21.15.49:0/3018643979 --> [v2:172.21.15.49:6808/38113,v1:172.21.15.49:6810/38113] -- osd_op(unknown.0.0:38 12.21 12:84000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x562de57b4400 con 0x562de5693800 2023-11-06T02:14:35.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.602+0000 7fc3131cd640 1 -- 172.21.15.49:0/3018643979 <== osd.1 v2:172.21.15.49:6808/38113 4 ==== osd_op_reply(38 [pgnls start_epoch 82 out=49b] v80'24 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562de62a9200 con 0x562de5693800 2023-11-06T02:14:35.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.602+0000 7fc30e1c3640 1 -- 172.21.15.49:0/3018643979 --> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] -- osd_op(unknown.0.0:39 12.11 12:88000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x562de57b7000 con 0x562de57a1400 2023-11-06T02:14:35.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.602+0000 7fc3131cd640 1 -- 172.21.15.49:0/3018643979 <== osd.5 v2:172.21.15.150:6800/38418 8 ==== osd_op_reply(39 [pgnls start_epoch 82 out=49b] v80'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562de62a9200 con 0x562de57a1400 2023-11-06T02:14:35.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.602+0000 7fc30d9c2640 1 -- 172.21.15.49:0/3018643979 --> [v2:172.21.15.49:6809/38116,v1:172.21.15.49:6811/38116] -- osd_op(unknown.0.0:40 12.31 12:8c000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x562de6175000 con 0x562de6174800 2023-11-06T02:14:35.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.602+0000 7fc3129cc640 1 -- 172.21.15.49:0/3018643979 <== osd.0 v2:172.21.15.49:6809/38116 7 ==== osd_op_reply(40 [pgnls start_epoch 82 out=81b] v80'15 uv8 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x562de62a9440 con 0x562de6174800 2023-11-06T02:14:35.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.602+0000 7fc31420c740 1 -- 172.21.15.49:0/3018643979 --> [v2:172.21.15.49:6809/38116,v1:172.21.15.49:6811/38116] -- osd_op(unknown.0.0:41 12.31 12:8de29a03:::10000000004.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82) v8 -- 0x562de5693000 con 0x562de6174800 2023-11-06T02:14:35.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.602+0000 7fc3129cc640 1 -- 172.21.15.49:0/3018643979 <== osd.0 v2:172.21.15.49:6809/38116 8 ==== osd_op_reply(41 10000000004.00000001 [stat out=16b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x562de62a9440 con 0x562de6174800 2023-11-06T02:14:35.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.602+0000 7fc31420c740 10 datascan.operator(): handling object 1099511627780.1 2023-11-06T02:14:35.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.602+0000 7fc31420c740 1 -- 172.21.15.49:0/3018643979 --> [v2:172.21.15.150:6824/38428,v1:172.21.15.150:6825/38428] -- osd_op(unknown.0.0:42 12.5 12:a367a3e8:::10000000004.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82) v8 -- 0x562de57b5400 con 0x562de57a0c00 2023-11-06T02:14:35.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.606+0000 7fc3121cb640 1 -- 172.21.15.49:0/3018643979 <== osd.6 v2:172.21.15.150:6824/38428 4 ==== osd_op_reply(42 10000000004.00000000 [call] v82'14 uv14 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562de62a8fc0 con 0x562de57a0c00 2023-11-06T02:14:35.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.606+0000 7fc31420c740 1 -- 172.21.15.49:0/3018643979 --> [v2:172.21.15.49:6800/38114,v1:172.21.15.49:6801/38114] -- osd_op(unknown.0.0:43 12.9 12:90000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x562de57b4400 con 0x562de57a0400 2023-11-06T02:14:35.621 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.606+0000 7fc3129cc640 1 -- 172.21.15.49:0/3018643979 <== osd.2 v2:172.21.15.49:6800/38114 6 ==== osd_op_reply(43 [pgnls start_epoch 82 out=49b] v80'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562de62a9440 con 0x562de57a0400 2023-11-06T02:14:35.621 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.606+0000 7fc30d9c2640 1 -- 172.21.15.49:0/3018643979 --> [v2:172.21.15.150:6808/38417,v1:172.21.15.150:6809/38417] -- osd_op(unknown.0.0:44 12.29 12:94000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x562de6174c00 con 0x562de62b9800 2023-11-06T02:14:35.621 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.606+0000 7fc3121cb640 1 -- 172.21.15.49:0/3018643979 <== osd.4 v2:172.21.15.150:6808/38417 5 ==== osd_op_reply(44 [pgnls start_epoch 82 out=49b] v80'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562de62a8fc0 con 0x562de62b9800 2023-11-06T02:14:35.621 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.606+0000 7fc30e1c3640 1 -- 172.21.15.49:0/3018643979 --> [v2:172.21.15.49:6809/38116,v1:172.21.15.49:6811/38116] -- osd_op(unknown.0.0:45 12.19 12:98000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x562de57b7400 con 0x562de6174800 2023-11-06T02:14:35.621 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.606+0000 7fc3129cc640 1 -- 172.21.15.49:0/3018643979 <== osd.0 v2:172.21.15.49:6809/38116 9 ==== osd_op_reply(45 [pgnls start_epoch 82 out=81b] v81'23 uv23 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x562de62a9440 con 0x562de6174800 2023-11-06T02:14:35.622 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.606+0000 7fc31420c740 1 -- 172.21.15.49:0/3018643979 --> [v2:172.21.15.49:6809/38116,v1:172.21.15.49:6811/38116] -- osd_op(unknown.0.0:46 12.19 12:997b7129:::100000001f8.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82) v8 -- 0x562de57b5000 con 0x562de6174800 2023-11-06T02:14:35.622 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.610+0000 7fc3129cc640 1 -- 172.21.15.49:0/3018643979 <== osd.0 v2:172.21.15.49:6809/38116 10 ==== osd_op_reply(46 100000001f8.00000000 [stat out=16b] v0'0 uv23 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x562de62a9440 con 0x562de6174800 2023-11-06T02:14:35.622 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.610+0000 7fc31420c740 10 datascan.operator(): handling object 1099511628280.0 2023-11-06T02:14:35.622 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.610+0000 7fc31420c740 1 -- 172.21.15.49:0/3018643979 --> [v2:172.21.15.49:6809/38116,v1:172.21.15.49:6811/38116] -- osd_op(unknown.0.0:47 12.19 12:997b7129:::100000001f8.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82) v8 -- 0x562de57b4800 con 0x562de6174800 2023-11-06T02:14:35.622 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.610+0000 7fc3129cc640 1 -- 172.21.15.49:0/3018643979 <== osd.0 v2:172.21.15.49:6809/38116 11 ==== osd_op_reply(47 100000001f8.00000000 [call] v82'24 uv24 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562de62a9440 con 0x562de6174800 2023-11-06T02:14:35.623 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.610+0000 7fc31420c740 1 -- 172.21.15.49:0/3018643979 --> [v2:172.21.15.150:6824/38428,v1:172.21.15.150:6825/38428] -- osd_op(unknown.0.0:48 12.39 12:9c000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x562de57b4000 con 0x562de57a0c00 2023-11-06T02:14:35.623 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.610+0000 7fc3121cb640 1 -- 172.21.15.49:0/3018643979 <== osd.6 v2:172.21.15.150:6824/38428 5 ==== osd_op_reply(48 [pgnls start_epoch 82 out=49b] v80'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562de62a8fc0 con 0x562de57a0c00 2023-11-06T02:14:35.623 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.610+0000 7fc30e1c3640 1 -- 172.21.15.49:0/3018643979 --> [v2:172.21.15.150:6824/38428,v1:172.21.15.150:6825/38428] -- osd_op(unknown.0.0:49 12.5 12:a0000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x562de57b7800 con 0x562de57a0c00 2023-11-06T02:14:35.623 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.614+0000 7fc3121cb640 1 -- 172.21.15.49:0/3018643979 <== osd.6 v2:172.21.15.150:6824/38428 6 ==== osd_op_reply(49 [pgnls start_epoch 82 out=81b] v82'14 uv14 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x562de62a8fc0 con 0x562de57a0c00 2023-11-06T02:14:35.623 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.614+0000 7fc31420c740 1 -- 172.21.15.49:0/3018643979 --> [v2:172.21.15.150:6824/38428,v1:172.21.15.150:6825/38428] -- osd_op(unknown.0.0:50 12.5 12:a367a3e8:::10000000004.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82) v8 -- 0x562de57a0000 con 0x562de57a0c00 2023-11-06T02:14:35.630 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.614+0000 7fc3121cb640 1 -- 172.21.15.49:0/3018643979 <== osd.6 v2:172.21.15.150:6824/38428 7 ==== osd_op_reply(50 10000000004.00000000 [stat out=16b] v0'0 uv14 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x562de62a8fc0 con 0x562de57a0c00 2023-11-06T02:14:35.630 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.614+0000 7fc31420c740 10 datascan.operator(): handling object 1099511627780.0 2023-11-06T02:14:35.630 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.614+0000 7fc31420c740 1 -- 172.21.15.49:0/3018643979 --> [v2:172.21.15.150:6824/38428,v1:172.21.15.150:6825/38428] -- osd_op(unknown.0.0:51 12.5 12:a367a3e8:::10000000004.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82) v8 -- 0x562de57a0800 con 0x562de57a0c00 2023-11-06T02:14:35.630 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.614+0000 7fc3121cb640 1 -- 172.21.15.49:0/3018643979 <== osd.6 v2:172.21.15.150:6824/38428 8 ==== osd_op_reply(51 10000000004.00000000 [call] v82'15 uv15 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562de62a8fc0 con 0x562de57a0c00 2023-11-06T02:14:35.631 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.614+0000 7fc31420c740 1 -- 172.21.15.49:0/3018643979 --> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] -- osd_op(unknown.0.0:52 12.25 12:a4000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x562de57a1000 con 0x562de6175400 2023-11-06T02:14:35.631 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.618+0000 7fc3131cd640 1 -- 172.21.15.49:0/3018643979 <== osd.7 v2:172.21.15.150:6816/38421 4 ==== osd_op_reply(52 [pgnls start_epoch 82 out=49b] v80'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562de62a9200 con 0x562de6175400 2023-11-06T02:14:35.631 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.618+0000 7fc30e1c3640 1 -- 172.21.15.49:0/3018643979 --> [v2:172.21.15.150:6824/38428,v1:172.21.15.150:6825/38428] -- osd_op(unknown.0.0:53 12.15 12:a8000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x562de57b6400 con 0x562de57a0c00 2023-11-06T02:14:35.631 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.618+0000 7fc3121cb640 1 -- 172.21.15.49:0/3018643979 <== osd.6 v2:172.21.15.150:6824/38428 9 ==== osd_op_reply(53 [pgnls start_epoch 82 out=49b] v80'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562de62a8fc0 con 0x562de57a0c00 2023-11-06T02:14:35.631 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.618+0000 7fc30d9c2640 1 -- 172.21.15.49:0/3018643979 --> [v2:172.21.15.49:6809/38116,v1:172.21.15.49:6811/38116] -- osd_op(unknown.0.0:54 12.35 12:ac000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x562de57a1c00 con 0x562de6174800 2023-11-06T02:14:35.632 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.618+0000 7fc3129cc640 1 -- 172.21.15.49:0/3018643979 <== osd.0 v2:172.21.15.49:6809/38116 12 ==== osd_op_reply(54 [pgnls start_epoch 82 out=49b] v80'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562de62a9440 con 0x562de6174800 2023-11-06T02:14:35.632 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.618+0000 7fc30e1c3640 1 -- 172.21.15.49:0/3018643979 --> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] -- osd_op(unknown.0.0:55 12.d 12:b0000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x562de57b6800 con 0x562de6175400 2023-11-06T02:14:35.632 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.618+0000 7fc3131cd640 1 -- 172.21.15.49:0/3018643979 <== osd.7 v2:172.21.15.150:6816/38421 5 ==== osd_op_reply(55 [pgnls start_epoch 82 out=49b] v80'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562de62a9200 con 0x562de6175400 2023-11-06T02:14:35.632 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.618+0000 7fc30d9c2640 1 -- 172.21.15.49:0/3018643979 --> [v2:172.21.15.49:6824/38115,v1:172.21.15.49:6825/38115] -- osd_op(unknown.0.0:56 12.2d 12:b4000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x562de57a1800 con 0x562de62b9c00 2023-11-06T02:14:35.632 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.618+0000 7fc3121cb640 1 -- 172.21.15.49:0/3018643979 <== osd.3 v2:172.21.15.49:6824/38115 7 ==== osd_op_reply(56 [pgnls start_epoch 82 out=49b] v80'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562de62a8fc0 con 0x562de62b9c00 2023-11-06T02:14:35.633 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.618+0000 7fc30e1c3640 1 -- 172.21.15.49:0/3018643979 --> [v2:172.21.15.49:6809/38116,v1:172.21.15.49:6811/38116] -- osd_op(unknown.0.0:57 12.1d 12:b8000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x562de57b6c00 con 0x562de6174800 2023-11-06T02:14:35.633 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.618+0000 7fc3129cc640 1 -- 172.21.15.49:0/3018643979 <== osd.0 v2:172.21.15.49:6809/38116 13 ==== osd_op_reply(57 [pgnls start_epoch 82 out=49b] v80'30 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562de62a9440 con 0x562de6174800 2023-11-06T02:14:35.633 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.618+0000 7fc30d9c2640 1 -- 172.21.15.49:0/3018643979 --> [v2:172.21.15.49:6808/38113,v1:172.21.15.49:6810/38113] -- osd_op(unknown.0.0:58 12.3d 12:bc000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x562de5693c00 con 0x562de5693800 2023-11-06T02:14:35.633 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.622+0000 7fc3131cd640 1 -- 172.21.15.49:0/3018643979 <== osd.1 v2:172.21.15.49:6808/38113 5 ==== osd_op_reply(58 [pgnls start_epoch 82 out=49b] v80'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562de62a9200 con 0x562de5693800 2023-11-06T02:14:35.633 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.622+0000 7fc30e1c3640 1 -- 172.21.15.49:0/3018643979 --> [v2:172.21.15.49:6809/38116,v1:172.21.15.49:6811/38116] -- osd_op(unknown.0.0:59 12.3 12:c0000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x562de57b7000 con 0x562de6174800 2023-11-06T02:14:35.634 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.622+0000 7fc3129cc640 1 -- 172.21.15.49:0/3018643979 <== osd.0 v2:172.21.15.49:6809/38116 14 ==== osd_op_reply(59 [pgnls start_epoch 82 out=49b] v80'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562de62a9440 con 0x562de6174800 2023-11-06T02:14:35.634 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.622+0000 7fc30d9c2640 1 -- 172.21.15.49:0/3018643979 --> [v2:172.21.15.49:6809/38116,v1:172.21.15.49:6811/38116] -- osd_op(unknown.0.0:60 12.23 12:c4000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x562de5693400 con 0x562de6174800 2023-11-06T02:14:35.634 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.622+0000 7fc3129cc640 1 -- 172.21.15.49:0/3018643979 <== osd.0 v2:172.21.15.49:6809/38116 15 ==== osd_op_reply(60 [pgnls start_epoch 82 out=49b] v80'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562de62a9440 con 0x562de6174800 2023-11-06T02:14:35.634 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.622+0000 7fc30e1c3640 1 -- 172.21.15.49:0/3018643979 --> [v2:172.21.15.49:6808/38113,v1:172.21.15.49:6810/38113] -- osd_op(unknown.0.0:61 12.13 12:c8000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x562de57b5800 con 0x562de5693800 2023-11-06T02:14:35.634 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.622+0000 7fc3131cd640 1 -- 172.21.15.49:0/3018643979 <== osd.1 v2:172.21.15.49:6808/38113 6 ==== osd_op_reply(61 [pgnls start_epoch 82 out=49b] v80'30 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562de62a9200 con 0x562de5693800 2023-11-06T02:14:35.635 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.622+0000 7fc30d9c2640 1 -- 172.21.15.49:0/3018643979 --> [v2:172.21.15.49:6800/38114,v1:172.21.15.49:6801/38114] -- osd_op(unknown.0.0:62 12.33 12:cc000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x562de57b7400 con 0x562de57a0400 2023-11-06T02:14:35.635 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.622+0000 7fc3129cc640 1 -- 172.21.15.49:0/3018643979 <== osd.2 v2:172.21.15.49:6800/38114 7 ==== osd_op_reply(62 [pgnls start_epoch 82 out=49b] v80'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562de62a9440 con 0x562de57a0400 2023-11-06T02:14:35.635 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.622+0000 7fc30e1c3640 1 -- 172.21.15.49:0/3018643979 --> [v2:172.21.15.49:6824/38115,v1:172.21.15.49:6825/38115] -- osd_op(unknown.0.0:63 12.b 12:d0000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x562de57b5c00 con 0x562de62b9c00 2023-11-06T02:14:35.635 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.622+0000 7fc3121cb640 1 -- 172.21.15.49:0/3018643979 <== osd.3 v2:172.21.15.49:6824/38115 8 ==== osd_op_reply(63 [pgnls start_epoch 82 out=49b] v80'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562de62a8fc0 con 0x562de62b9c00 2023-11-06T02:14:35.635 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.622+0000 7fc30d9c2640 1 -- 172.21.15.49:0/3018643979 --> [v2:172.21.15.150:6808/38417,v1:172.21.15.150:6809/38417] -- osd_op(unknown.0.0:64 12.2b 12:d4000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x562de57b6000 con 0x562de62b9800 2023-11-06T02:14:35.635 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.622+0000 7fc3121cb640 1 -- 172.21.15.49:0/3018643979 <== osd.4 v2:172.21.15.150:6808/38417 6 ==== osd_op_reply(64 [pgnls start_epoch 82 out=49b] v80'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562de62a8fc0 con 0x562de62b9800 2023-11-06T02:14:35.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.622+0000 7fc30e1c3640 1 -- 172.21.15.49:0/3018643979 --> [v2:172.21.15.150:6808/38417,v1:172.21.15.150:6809/38417] -- osd_op(unknown.0.0:65 12.1b 12:d8000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x562de57b4c00 con 0x562de62b9800 2023-11-06T02:14:35.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.626+0000 7fc3121cb640 1 -- 172.21.15.49:0/3018643979 <== osd.4 v2:172.21.15.150:6808/38417 7 ==== osd_op_reply(65 [pgnls start_epoch 82 out=49b] v80'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562de62a8fc0 con 0x562de62b9800 2023-11-06T02:14:35.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.626+0000 7fc30d9c2640 1 -- 172.21.15.49:0/3018643979 --> [v2:172.21.15.49:6800/38114,v1:172.21.15.49:6801/38114] -- osd_op(unknown.0.0:66 12.3b 12:dc000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x562de57a1c00 con 0x562de57a0400 2023-11-06T02:14:35.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.626+0000 7fc3129cc640 1 -- 172.21.15.49:0/3018643979 <== osd.2 v2:172.21.15.49:6800/38114 8 ==== osd_op_reply(66 [pgnls start_epoch 82 out=49b] v80'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562de62a9440 con 0x562de57a0400 2023-11-06T02:14:35.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.626+0000 7fc30e1c3640 1 -- 172.21.15.49:0/3018643979 --> [v2:172.21.15.49:6808/38113,v1:172.21.15.49:6810/38113] -- osd_op(unknown.0.0:67 12.7 12:e0000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x562de57b5400 con 0x562de5693800 2023-11-06T02:14:35.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.626+0000 7fc3131cd640 1 -- 172.21.15.49:0/3018643979 <== osd.1 v2:172.21.15.49:6808/38113 7 ==== osd_op_reply(67 [pgnls start_epoch 82 out=49b] v80'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562de62a9200 con 0x562de5693800 2023-11-06T02:14:35.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.626+0000 7fc30d9c2640 1 -- 172.21.15.49:0/3018643979 --> [v2:172.21.15.49:6809/38116,v1:172.21.15.49:6811/38116] -- osd_op(unknown.0.0:68 12.27 12:e4000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x562de57b4400 con 0x562de6174800 2023-11-06T02:14:35.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.626+0000 7fc3129cc640 1 -- 172.21.15.49:0/3018643979 <== osd.0 v2:172.21.15.49:6809/38116 16 ==== osd_op_reply(68 [pgnls start_epoch 82 out=49b] v80'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562de62a9440 con 0x562de6174800 2023-11-06T02:14:35.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.626+0000 7fc30e1c3640 1 -- 172.21.15.49:0/3018643979 --> [v2:172.21.15.150:6824/38428,v1:172.21.15.150:6825/38428] -- osd_op(unknown.0.0:69 12.17 12:e8000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x562de57b7800 con 0x562de57a0c00 2023-11-06T02:14:35.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.626+0000 7fc3121cb640 1 -- 172.21.15.49:0/3018643979 <== osd.6 v2:172.21.15.150:6824/38428 10 ==== osd_op_reply(69 [pgnls start_epoch 82 out=49b] v80'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562de62a8fc0 con 0x562de57a0c00 2023-11-06T02:14:35.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.626+0000 7fc30d9c2640 1 -- 172.21.15.49:0/3018643979 --> [v2:172.21.15.150:6808/38417,v1:172.21.15.150:6809/38417] -- osd_op(unknown.0.0:70 12.37 12:ec000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x562de57b5000 con 0x562de62b9800 2023-11-06T02:14:35.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.626+0000 7fc3121cb640 1 -- 172.21.15.49:0/3018643979 <== osd.4 v2:172.21.15.150:6808/38417 8 ==== osd_op_reply(70 [pgnls start_epoch 82 out=49b] v80'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562de62a8fc0 con 0x562de62b9800 2023-11-06T02:14:35.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.630+0000 7fc30e1c3640 1 -- 172.21.15.49:0/3018643979 --> [v2:172.21.15.150:6824/38428,v1:172.21.15.150:6825/38428] -- osd_op(unknown.0.0:71 12.f 12:f0000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x562de57b6400 con 0x562de57a0c00 2023-11-06T02:14:35.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.630+0000 7fc3121cb640 1 -- 172.21.15.49:0/3018643979 <== osd.6 v2:172.21.15.150:6824/38428 11 ==== osd_op_reply(71 [pgnls start_epoch 82 out=49b] v80'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562de62a8fc0 con 0x562de57a0c00 2023-11-06T02:14:35.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.630+0000 7fc30d9c2640 1 -- 172.21.15.49:0/3018643979 --> [v2:172.21.15.49:6800/38114,v1:172.21.15.49:6801/38114] -- osd_op(unknown.0.0:72 12.2f 12:f4000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x562de57b4800 con 0x562de57a0400 2023-11-06T02:14:35.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.630+0000 7fc3129cc640 1 -- 172.21.15.49:0/3018643979 <== osd.2 v2:172.21.15.49:6800/38114 9 ==== osd_op_reply(72 [pgnls start_epoch 82 out=49b] v80'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562de62a9440 con 0x562de57a0400 2023-11-06T02:14:35.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.630+0000 7fc30e1c3640 1 -- 172.21.15.49:0/3018643979 --> [v2:172.21.15.150:6824/38428,v1:172.21.15.150:6825/38428] -- osd_op(unknown.0.0:73 12.1f 12:f8000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x562de57b7c00 con 0x562de57a0c00 2023-11-06T02:14:35.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.630+0000 7fc3121cb640 1 -- 172.21.15.49:0/3018643979 <== osd.6 v2:172.21.15.150:6824/38428 12 ==== osd_op_reply(73 [pgnls start_epoch 82 out=49b] v80'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562de62a8fc0 con 0x562de57a0c00 2023-11-06T02:14:35.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.630+0000 7fc30d9c2640 1 -- 172.21.15.49:0/3018643979 --> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] -- osd_op(unknown.0.0:74 12.3f 12:fc000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x562de5693000 con 0x562de6175400 2023-11-06T02:14:35.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.630+0000 7fc3131cd640 1 -- 172.21.15.49:0/3018643979 <== osd.7 v2:172.21.15.150:6816/38421 6 ==== osd_op_reply(74 [pgnls start_epoch 82 out=49b] v80'17 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562de62a9200 con 0x562de6175400 2023-11-06T02:14:35.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.630+0000 7fc31420c740 1 -- 172.21.15.49:0/3018643979 >> [v2:172.21.15.49:6809/38116,v1:172.21.15.49:6811/38116] conn(0x562de6174800 msgr2=0x562de578e580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:35.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.630+0000 7fc31420c740 1 --2- 172.21.15.49:0/3018643979 >> [v2:172.21.15.49:6809/38116,v1:172.21.15.49:6811/38116] conn(0x562de6174800 0x562de578e580 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.630+0000 7fc31420c740 1 -- 172.21.15.49:0/3018643979 >> [v2:172.21.15.49:6808/38113,v1:172.21.15.49:6810/38113] conn(0x562de5693800 msgr2=0x562de578f080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:35.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.630+0000 7fc31420c740 1 --2- 172.21.15.49:0/3018643979 >> [v2:172.21.15.49:6808/38113,v1:172.21.15.49:6810/38113] conn(0x562de5693800 0x562de578f080 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.630+0000 7fc31420c740 1 -- 172.21.15.49:0/3018643979 >> [v2:172.21.15.49:6800/38114,v1:172.21.15.49:6801/38114] conn(0x562de57a0400 msgr2=0x562de578f600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:35.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.630+0000 7fc31420c740 1 --2- 172.21.15.49:0/3018643979 >> [v2:172.21.15.49:6800/38114,v1:172.21.15.49:6801/38114] conn(0x562de57a0400 0x562de578f600 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.630+0000 7fc31420c740 1 -- 172.21.15.49:0/3018643979 >> [v2:172.21.15.49:6824/38115,v1:172.21.15.49:6825/38115] conn(0x562de62b9c00 msgr2=0x562de578eb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:35.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.630+0000 7fc31420c740 1 --2- 172.21.15.49:0/3018643979 >> [v2:172.21.15.49:6824/38115,v1:172.21.15.49:6825/38115] conn(0x562de62b9c00 0x562de578eb00 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.630+0000 7fc3121cb640 1 -- 172.21.15.49:0/3018643979 reap_dead start 2023-11-06T02:14:35.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.630+0000 7fc31420c740 1 -- 172.21.15.49:0/3018643979 >> [v2:172.21.15.150:6808/38417,v1:172.21.15.150:6809/38417] conn(0x562de62b9800 msgr2=0x562de543e100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:35.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.630+0000 7fc31420c740 1 --2- 172.21.15.49:0/3018643979 >> [v2:172.21.15.150:6808/38417,v1:172.21.15.150:6809/38417] conn(0x562de62b9800 0x562de543e100 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.630+0000 7fc3121cb640 1 -- 172.21.15.49:0/3018643979 reap_dead start 2023-11-06T02:14:35.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.630+0000 7fc31420c740 1 -- 172.21.15.49:0/3018643979 >> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] conn(0x562de57a1400 msgr2=0x562de5790680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:35.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.630+0000 7fc31420c740 1 --2- 172.21.15.49:0/3018643979 >> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] conn(0x562de57a1400 0x562de5790680 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.630+0000 7fc31420c740 1 -- 172.21.15.49:0/3018643979 >> [v2:172.21.15.150:6824/38428,v1:172.21.15.150:6825/38428] conn(0x562de57a0c00 msgr2=0x562de5790100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:35.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.630+0000 7fc31420c740 1 --2- 172.21.15.49:0/3018643979 >> [v2:172.21.15.150:6824/38428,v1:172.21.15.150:6825/38428] conn(0x562de57a0c00 0x562de5790100 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.630+0000 7fc31420c740 1 -- 172.21.15.49:0/3018643979 >> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] conn(0x562de6175400 msgr2=0x562de578e000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:35.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.630+0000 7fc31420c740 1 --2- 172.21.15.49:0/3018643979 >> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] conn(0x562de6175400 0x562de578e000 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.630+0000 7fc31420c740 1 -- 172.21.15.49:0/3018643979 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x562de5692c00 msgr2=0x562de543f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:35.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.630+0000 7fc31420c740 1 --2- 172.21.15.49:0/3018643979 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x562de5692c00 0x562de543f700 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x562de6207650 tx=0x562de620cd80 comp rx=0 tx=0).stop 2023-11-06T02:14:35.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.630+0000 7fc31420c740 1 -- 172.21.15.49:0/3018643979 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562de62b8c00 msgr2=0x562de543f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:35.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.630+0000 7fc31420c740 1 --2- 172.21.15.49:0/3018643979 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562de62b8c00 0x562de543f180 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x562de61732f0 tx=0x562de62dac00 comp rx=0 tx=0).stop 2023-11-06T02:14:35.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.630+0000 7fc3121cb640 1 -- 172.21.15.49:0/3018643979 reap_dead start 2023-11-06T02:14:35.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.630+0000 7fc31420c740 1 -- 172.21.15.49:0/3018643979 shutdown_connections 2023-11-06T02:14:35.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.630+0000 7fc31420c740 1 -- 172.21.15.49:0/3018643979 >> 172.21.15.49:0/3018643979 conn(0x562de53b0c00 msgr2=0x562de53db000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:35.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.634+0000 7fc31420c740 1 -- 172.21.15.49:0/3018643979 shutdown_connections 2023-11-06T02:14:35.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.634+0000 7fc31420c740 1 -- 172.21.15.49:0/3018643979 wait complete. 2023-11-06T02:14:35.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.634+0000 7fc31420c740 1 -- 172.21.15.49:0/353727567 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562de53b1800 msgr2=0x562de543c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:35.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.634+0000 7fc31420c740 1 --2- 172.21.15.49:0/353727567 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562de53b1800 0x562de543c580 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x562de6207170 tx=0x562de620cf60 comp rx=0 tx=0).stop 2023-11-06T02:14:35.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.634+0000 7fc31420c740 1 -- 172.21.15.49:0/353727567 shutdown_connections 2023-11-06T02:14:35.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.634+0000 7fc31420c740 1 --2- 172.21.15.49:0/353727567 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562de53b1000 0x562de543d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.634+0000 7fc31420c740 1 --2- 172.21.15.49:0/353727567 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562de53b1400 0x562de543cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.634+0000 7fc31420c740 1 --2- 172.21.15.49:0/353727567 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562de53b1800 0x562de543c580 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.634+0000 7fc31420c740 1 -- 172.21.15.49:0/353727567 >> 172.21.15.49:0/353727567 conn(0x562de53b1c00 msgr2=0x562de53db800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:35.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.634+0000 7fc31420c740 1 -- 172.21.15.49:0/353727567 shutdown_connections 2023-11-06T02:14:35.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.634+0000 7fc31420c740 1 -- 172.21.15.49:0/353727567 wait complete. 2023-11-06T02:14:35.647 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', 'cephfs_data'] in time 0:00:00.151345, result: 2023-11-06T02:14:35.647 DEBUG:teuthology.orchestra.run.smithi049:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes --filter-tag mytag123 cephfs_data 2023-11-06T02:14:35.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.738+0000 7f6c6d2ff740 1 -- start start 2023-11-06T02:14:35.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.738+0000 7f6c6d2ff740 1 Processor -- start 2023-11-06T02:14:35.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.738+0000 7f6c6d2ff740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5594de84f800 0x5594de8dbb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:35.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.738+0000 7f6c6d2ff740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5594de84f400 0x5594de8db600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:35.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.738+0000 7f6c6d2ff740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5594de84f000 0x5594de8db080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:35.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.738+0000 7f6c6d2ff740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594de91ca00 con 0x5594de84f000 2023-11-06T02:14:35.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.738+0000 7f6c6d2ff740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594de91c640 con 0x5594de84f400 2023-11-06T02:14:35.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.738+0000 7f6c6d2ff740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x5594de91c500 con 0x5594de84f800 2023-11-06T02:14:35.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.738+0000 7f6c6c2c0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5594de84f000 0x5594de8db080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:35.736 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.738+0000 7f6c6c2c0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5594de84f000 0x5594de8db080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38226/0 (socket says 172.21.15.49:38226) 2023-11-06T02:14:35.736 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.738+0000 7f6c6c2c0640 1 -- 172.21.15.49:0/2085664153 learned_addr learned my addr 172.21.15.49:0/2085664153 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:35.736 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.738+0000 7f6c6b2be640 1 --2- 172.21.15.49:0/2085664153 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5594de84f400 0x5594de8db600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:35.736 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.738+0000 7f6c6babf640 1 --2- 172.21.15.49:0/2085664153 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5594de84f800 0x5594de8dbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:35.736 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.738+0000 7f6c6c2c0640 1 -- 172.21.15.49:0/2085664153 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5594de84f800 msgr2=0x5594de8dbb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:35.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.738+0000 7f6c6c2c0640 1 --2- 172.21.15.49:0/2085664153 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5594de84f800 0x5594de8dbb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.738+0000 7f6c6c2c0640 1 -- 172.21.15.49:0/2085664153 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5594de84f400 msgr2=0x5594de8db600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:35.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.738+0000 7f6c6c2c0640 1 --2- 172.21.15.49:0/2085664153 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5594de84f400 0x5594de8db600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.738+0000 7f6c6c2c0640 1 -- 172.21.15.49:0/2085664153 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5594de882b60 con 0x5594de84f000 2023-11-06T02:14:35.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.738+0000 7f6c6b2be640 1 --2- 172.21.15.49:0/2085664153 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5594de84f400 0x5594de8db600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:14:35.738 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.738+0000 7f6c6babf640 1 --2- 172.21.15.49:0/2085664153 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5594de84f800 0x5594de8dbb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:14:35.738 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.738+0000 7f6c6c2c0640 1 --2- 172.21.15.49:0/2085664153 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5594de84f000 0x5594de8db080 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x5594df6d2d50 tx=0x5594df633020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=436d3dec6444f5fa server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:35.738 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.738+0000 7f6c692ba640 1 -- 172.21.15.49:0/2085664153 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5594de8b3600 con 0x5594de84f000 2023-11-06T02:14:35.738 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.738+0000 7f6c692ba640 1 -- 172.21.15.49:0/2085664153 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5594df768000 con 0x5594de84f000 2023-11-06T02:14:35.738 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.738+0000 7f6c692ba640 1 -- 172.21.15.49:0/2085664153 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5594de8b31e0 con 0x5594de84f000 2023-11-06T02:14:35.739 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.738+0000 7f6c6d2ff740 1 -- 172.21.15.49:0/2085664153 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5594de882340 con 0x5594de84f000 2023-11-06T02:14:35.739 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.738+0000 7f6c6d2ff740 1 -- 172.21.15.49:0/2085664153 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x5594de882000 con 0x5594de84f000 2023-11-06T02:14:35.739 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.738+0000 7f6c6d2ff740 4 waiting for MDS map... 2023-11-06T02:14:35.739 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.738+0000 7f6c692ba640 1 -- 172.21.15.49:0/2085664153 <== mon.0 v2:172.21.15.49:3300/0 4 ==== osd_map(82..82 src has 1..82) v4 ==== 6234+0+0 (secure 0 0 0) 0x5594de88ec40 con 0x5594de84f000 2023-11-06T02:14:35.739 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.738+0000 7f6c692ba640 1 -- 172.21.15.49:0/2085664153 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 53) v1 ==== 2565+0+0 (secure 0 0 0) 0x5594de89d800 con 0x5594de84f000 2023-11-06T02:14:35.739 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.738+0000 7f6c6d2ff740 4 Got MDS map 53 2023-11-06T02:14:35.740 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.738+0000 7f6c6d2ff740 10 datascan.parse_kwarg: Applying tag filter: 'mytag123' 2023-11-06T02:14:35.740 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.738+0000 7f6c6d2ff740 4 datascan.main: Using metadata pool output 2023-11-06T02:14:35.740 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.738+0000 7f6c6d2ff740 4 datascan.main: connecting to RADOS... 2023-11-06T02:14:35.740 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.738+0000 7f6c6d2ff740 1 Processor -- start 2023-11-06T02:14:35.740 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.738+0000 7f6c6d2ff740 1 -- start start 2023-11-06T02:14:35.746 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.738+0000 7f6c6d2ff740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5594df762c00 0x5594de8dab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:35.746 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.738+0000 7f6c6d2ff740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5594df763000 0x5594de8da580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:35.747 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.738+0000 7f6c6d2ff740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5594df763400 0x5594de8dc100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:35.747 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.738+0000 7f6c6d2ff740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594de895e00 con 0x5594df763400 2023-11-06T02:14:35.747 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.738+0000 7f6c6d2ff740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594de91c640 con 0x5594df762c00 2023-11-06T02:14:35.747 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.738+0000 7f6c6d2ff740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x5594de91c3c0 con 0x5594df763000 2023-11-06T02:14:35.747 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.742+0000 7f6c6b2be640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5594df763400 0x5594de8dc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:35.748 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.742+0000 7f6c6b2be640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5594df763400 0x5594de8dc100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38230/0 (socket says 172.21.15.49:38230) 2023-11-06T02:14:35.748 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.742+0000 7f6c6b2be640 1 -- 172.21.15.49:0/1684838678 learned_addr learned my addr 172.21.15.49:0/1684838678 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:35.748 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.742+0000 7f6c6b2be640 1 --2- 172.21.15.49:0/1684838678 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5594df762c00 0x5594de8dab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:35.748 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.742+0000 7f6c6babf640 1 --2- 172.21.15.49:0/1684838678 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5594df763000 0x5594de8da580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:35.748 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.742+0000 7f6c6b2be640 1 -- 172.21.15.49:0/1684838678 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5594df763000 msgr2=0x5594de8da580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:35.748 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.742+0000 7f6c6b2be640 1 --2- 172.21.15.49:0/1684838678 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5594df763000 0x5594de8da580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.749 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.742+0000 7f6c6b2be640 1 -- 172.21.15.49:0/1684838678 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5594df762c00 msgr2=0x5594de8dab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:35.749 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.742+0000 7f6c6b2be640 1 --2- 172.21.15.49:0/1684838678 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5594df762c00 0x5594de8dab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.749 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.742+0000 7f6c6b2be640 1 -- 172.21.15.49:0/1684838678 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5594de882ea0 con 0x5594df763400 2023-11-06T02:14:35.749 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.742+0000 7f6c6b2be640 1 --2- 172.21.15.49:0/1684838678 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5594df763400 0x5594de8dc100 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x5594df7d4090 tx=0x5594df632300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf12b27a4cbf1818 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:35.749 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.742+0000 7f6c672b6640 1 -- 172.21.15.49:0/1684838678 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5594de8b3080 con 0x5594df763400 2023-11-06T02:14:35.750 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.742+0000 7f6c672b6640 1 -- 172.21.15.49:0/1684838678 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5594df768480 con 0x5594df763400 2023-11-06T02:14:35.750 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.742+0000 7f6c672b6640 1 -- 172.21.15.49:0/1684838678 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5594de8b3600 con 0x5594df763400 2023-11-06T02:14:35.750 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.742+0000 7f6c6d2ff740 1 -- 172.21.15.49:0/1684838678 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5594df763400 msgr2=0x5594de8dc100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:35.750 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.742+0000 7f6c6d2ff740 1 --2- 172.21.15.49:0/1684838678 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5594df763400 0x5594de8dc100 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x5594df7d4090 tx=0x5594df632300 comp rx=0 tx=0).stop 2023-11-06T02:14:35.750 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.742+0000 7f6c6d2ff740 1 -- 172.21.15.49:0/1684838678 shutdown_connections 2023-11-06T02:14:35.751 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.742+0000 7f6c6d2ff740 1 --2- 172.21.15.49:0/1684838678 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5594df763400 0x5594de8dc100 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.751 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.742+0000 7f6c6d2ff740 1 --2- 172.21.15.49:0/1684838678 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5594df763000 0x5594de8da580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.751 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.742+0000 7f6c6d2ff740 1 --2- 172.21.15.49:0/1684838678 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5594df762c00 0x5594de8dab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.751 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.742+0000 7f6c6d2ff740 1 -- 172.21.15.49:0/1684838678 >> 172.21.15.49:0/1684838678 conn(0x5594de84ec00 msgr2=0x5594de87b800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:35.751 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.742+0000 7f6c6d2ff740 1 -- 172.21.15.49:0/1684838678 shutdown_connections 2023-11-06T02:14:35.752 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.742+0000 7f6c6d2ff740 1 -- 172.21.15.49:0/1684838678 wait complete. 2023-11-06T02:14:35.752 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.742+0000 7f6c6d2ff740 1 Processor -- start 2023-11-06T02:14:35.752 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.742+0000 7f6c6d2ff740 1 -- start start 2023-11-06T02:14:35.752 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.742+0000 7f6c6d2ff740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5594df762c00 0x5594de8dab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:35.752 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.742+0000 7f6c6d2ff740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5594df763000 0x5594de8da580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:35.753 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.742+0000 7f6c6d2ff740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5594df763800 0x5594de8dc680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:35.753 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.742+0000 7f6c6d2ff740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594de91c280 con 0x5594df762c00 2023-11-06T02:14:35.753 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.742+0000 7f6c6d2ff740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594de91c140 con 0x5594df763800 2023-11-06T02:14:35.753 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.742+0000 7f6c6d2ff740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x5594de91c000 con 0x5594df763000 2023-11-06T02:14:35.753 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.742+0000 7f6c6b2be640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5594df762c00 0x5594de8dab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:35.753 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.742+0000 7f6c6b2be640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5594df762c00 0x5594de8dab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38236/0 (socket says 172.21.15.49:38236) 2023-11-06T02:14:35.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.742+0000 7f6c6b2be640 1 -- 172.21.15.49:0/1260454981 learned_addr learned my addr 172.21.15.49:0/1260454981 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:35.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.742+0000 7f6c6babf640 1 --2- 172.21.15.49:0/1260454981 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5594df763000 0x5594de8da580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:35.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.742+0000 7f6c6b2be640 1 --2- 172.21.15.49:0/1260454981 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5594df763800 0x5594de8dc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:35.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.742+0000 7f6c6b2be640 1 -- 172.21.15.49:0/1260454981 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5594df763000 msgr2=0x5594de8da580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:35.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.742+0000 7f6c6b2be640 1 --2- 172.21.15.49:0/1260454981 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5594df763000 0x5594de8da580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.742+0000 7f6c6b2be640 1 -- 172.21.15.49:0/1260454981 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5594df763800 msgr2=0x5594de8dc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:35.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.742+0000 7f6c6b2be640 1 --2- 172.21.15.49:0/1260454981 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5594df763800 0x5594de8dc680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.742+0000 7f6c6b2be640 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5594de8831e0 con 0x5594df762c00 2023-11-06T02:14:35.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.742+0000 7f6c6b2be640 1 --2- 172.21.15.49:0/1260454981 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5594df762c00 0x5594de8dab00 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x5594df7d4330 tx=0x5594df6321e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:35.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.742+0000 7f6c65ab3640 1 -- 172.21.15.49:0/1260454981 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5594de8b31e0 con 0x5594df762c00 2023-11-06T02:14:35.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.742+0000 7f6c65ab3640 1 -- 172.21.15.49:0/1260454981 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5594df768180 con 0x5594df762c00 2023-11-06T02:14:35.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.742+0000 7f6c65ab3640 1 -- 172.21.15.49:0/1260454981 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5594de8b2dc0 con 0x5594df762c00 2023-11-06T02:14:35.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.746+0000 7f6c6d2ff740 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5594de882ea0 con 0x5594df762c00 2023-11-06T02:14:35.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.746+0000 7f6c6d2ff740 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5594de882b60 con 0x5594df762c00 2023-11-06T02:14:35.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.746+0000 7f6c6d2ff740 4 datascan.init: resolving metadata pool 11 2023-11-06T02:14:35.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.746+0000 7f6c65ab3640 1 -- 172.21.15.49:0/1260454981 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x5594de919080 con 0x5594df762c00 2023-11-06T02:14:35.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.746+0000 7f6c65ab3640 1 --2- 172.21.15.49:0/1260454981 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x5594dec14000 0x5594dec1a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:35.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.746+0000 7f6c65ab3640 1 -- 172.21.15.49:0/1260454981 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6234+0+0 (secure 0 0 0) 0x5594de88ea80 con 0x5594df762c00 2023-11-06T02:14:35.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.746+0000 7f6c6d2ff740 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-11-06T02:14:35.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.746+0000 7f6c6d2ff740 4 datascan.main: data pool 'cephfs_data' has ID 12 2023-11-06T02:14:35.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.746+0000 7f6c6d2ff740 4 datascan.main: opening data pool 'cephfs_data' 2023-11-06T02:14:35.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.746+0000 7f6c6d2ff740 1 --2- 172.21.15.49:0/1260454981 >> [v2:172.21.15.150:6824/38428,v1:172.21.15.150:6825/38428] conn(0x5594df763c00 0x5594de8dcc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:35.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.746+0000 7f6c6d2ff740 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.150:6824/38428,v1:172.21.15.150:6825/38428] -- osd_op(unknown.0.0:1 11.14 11:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594dec14400 con 0x5594df763c00 2023-11-06T02:14:35.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.746+0000 7f6c6babf640 1 --2- 172.21.15.49:0/1260454981 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x5594dec14000 0x5594dec1a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:35.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.746+0000 7f6c6b2be640 1 --2- 172.21.15.49:0/1260454981 >> [v2:172.21.15.150:6824/38428,v1:172.21.15.150:6825/38428] conn(0x5594df763c00 0x5594de8dcc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:35.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.746+0000 7f6c6b2be640 1 --2- 172.21.15.49:0/1260454981 >> [v2:172.21.15.150:6824/38428,v1:172.21.15.150:6825/38428] conn(0x5594df763c00 0x5594de8dcc00 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:35.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.746+0000 7f6c6babf640 1 --2- 172.21.15.49:0/1260454981 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x5594dec14000 0x5594dec1a000 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x5594df618150 tx=0x5594df780f00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:35.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.746+0000 7f6c6b2be640 1 -- 172.21.15.49:0/1260454981 <== osd.6 v2:172.21.15.150:6824/38428 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv38 ondisk = 0) v8 ==== 160+0+16 (crc 0 0 0) 0x5594df5acfc0 con 0x5594df763c00 2023-11-06T02:14:35.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.746+0000 7f6c6d2ff740 1 --2- 172.21.15.49:0/1260454981 >> [v2:172.21.15.49:6800/38114,v1:172.21.15.49:6801/38114] conn(0x5594dec14800 0x5594de8dd180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:35.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.746+0000 7f6c6d2ff740 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.49:6800/38114,v1:172.21.15.49:6801/38114] -- osd_op(unknown.0.0:2 11.1f 11:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594dec14c00 con 0x5594dec14800 2023-11-06T02:14:35.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.746+0000 7f6c6c2c0640 1 --2- 172.21.15.49:0/1260454981 >> [v2:172.21.15.49:6800/38114,v1:172.21.15.49:6801/38114] conn(0x5594dec14800 0x5594de8dd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:35.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.750+0000 7f6c6c2c0640 1 --2- 172.21.15.49:0/1260454981 >> [v2:172.21.15.49:6800/38114,v1:172.21.15.49:6801/38114] conn(0x5594dec14800 0x5594de8dd180 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:35.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.750+0000 7f6c6c2c0640 1 -- 172.21.15.49:0/1260454981 <== osd.2 v2:172.21.15.49:6800/38114 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 162+0+16 (crc 0 0 0) 0x5594df5ad200 con 0x5594dec14800 2023-11-06T02:14:35.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.750+0000 7f6c6d2ff740 1 --2- 172.21.15.49:0/1260454981 >> [v2:172.21.15.150:6808/38417,v1:172.21.15.150:6809/38417] conn(0x5594dec15000 0x5594de8dd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:35.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.750+0000 7f6c6d2ff740 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.150:6808/38417,v1:172.21.15.150:6809/38417] -- osd_op(unknown.0.0:3 12.0 12:00000000::::head [pgnls-filter start_epoch 82 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594dec15400 con 0x5594dec15000 2023-11-06T02:14:35.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.750+0000 7f6c6babf640 1 --2- 172.21.15.49:0/1260454981 >> [v2:172.21.15.150:6808/38417,v1:172.21.15.150:6809/38417] conn(0x5594dec15000 0x5594de8dd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:35.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.750+0000 7f6c6babf640 1 --2- 172.21.15.49:0/1260454981 >> [v2:172.21.15.150:6808/38417,v1:172.21.15.150:6809/38417] conn(0x5594dec15000 0x5594de8dd700 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:35.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.750+0000 7f6c6babf640 1 -- 172.21.15.49:0/1260454981 <== osd.4 v2:172.21.15.150:6808/38417 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 82 out=49b] v80'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594df5ad440 con 0x5594dec15000 2023-11-06T02:14:35.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.750+0000 7f6c66ab5640 1 --2- 172.21.15.49:0/1260454981 >> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] conn(0x5594dec22c00 0x5594dec1a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:35.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.750+0000 7f6c66ab5640 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] -- osd_op(unknown.0.0:4 12.20 12:04000000::::head [pgnls-filter start_epoch 82 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594dec23000 con 0x5594dec22c00 2023-11-06T02:14:35.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.750+0000 7f6c6b2be640 1 --2- 172.21.15.49:0/1260454981 >> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] conn(0x5594dec22c00 0x5594dec1a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:35.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.750+0000 7f6c6b2be640 1 --2- 172.21.15.49:0/1260454981 >> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] conn(0x5594dec22c00 0x5594dec1a580 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:35.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.754+0000 7f6c6b2be640 1 -- 172.21.15.49:0/1260454981 <== osd.7 v2:172.21.15.150:6816/38421 1 ==== osd_op_reply(4 [pgnls-filter start_epoch 82 out=49b] v80'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594df5acfc0 con 0x5594dec22c00 2023-11-06T02:14:35.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.754+0000 7f6c672b6640 1 --2- 172.21.15.49:0/1260454981 >> [v2:172.21.15.49:6809/38116,v1:172.21.15.49:6811/38116] conn(0x5594dec23800 0x5594dec1ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:35.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.754+0000 7f6c672b6640 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.49:6809/38116,v1:172.21.15.49:6811/38116] -- osd_op(unknown.0.0:5 12.10 12:08000000::::head [pgnls-filter start_epoch 82 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594dec23c00 con 0x5594dec23800 2023-11-06T02:14:35.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.754+0000 7f6c6c2c0640 1 --2- 172.21.15.49:0/1260454981 >> [v2:172.21.15.49:6809/38116,v1:172.21.15.49:6811/38116] conn(0x5594dec23800 0x5594dec1ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:35.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.754+0000 7f6c6c2c0640 1 --2- 172.21.15.49:0/1260454981 >> [v2:172.21.15.49:6809/38116,v1:172.21.15.49:6811/38116] conn(0x5594dec23800 0x5594dec1ab00 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:35.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.754+0000 7f6c6c2c0640 1 -- 172.21.15.49:0/1260454981 <== osd.0 v2:172.21.15.49:6809/38116 1 ==== osd_op_reply(5 [pgnls-filter start_epoch 82 out=49b] v80'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594df5ad200 con 0x5594dec23800 2023-11-06T02:14:35.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.754+0000 7f6c66ab5640 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] -- osd_op(unknown.0.0:6 12.30 12:0c000000::::head [pgnls-filter start_epoch 82 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594dec23400 con 0x5594dec22c00 2023-11-06T02:14:35.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.754+0000 7f6c6b2be640 1 -- 172.21.15.49:0/1260454981 <== osd.7 v2:172.21.15.150:6816/38421 2 ==== osd_op_reply(6 [pgnls-filter start_epoch 82 out=49b] v80'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594df5acfc0 con 0x5594dec22c00 2023-11-06T02:14:35.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.754+0000 7f6c672b6640 1 --2- 172.21.15.49:0/1260454981 >> [v2:172.21.15.49:6824/38115,v1:172.21.15.49:6825/38115] conn(0x5594dec46000 0x5594dec1b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:35.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.754+0000 7f6c672b6640 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.49:6824/38115,v1:172.21.15.49:6825/38115] -- osd_op(unknown.0.0:7 12.8 12:10000000::::head [pgnls-filter start_epoch 82 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594dec46400 con 0x5594dec46000 2023-11-06T02:14:35.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.754+0000 7f6c6babf640 1 --2- 172.21.15.49:0/1260454981 >> [v2:172.21.15.49:6824/38115,v1:172.21.15.49:6825/38115] conn(0x5594dec46000 0x5594dec1b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:35.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.754+0000 7f6c6babf640 1 --2- 172.21.15.49:0/1260454981 >> [v2:172.21.15.49:6824/38115,v1:172.21.15.49:6825/38115] conn(0x5594dec46000 0x5594dec1b080 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:35.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.754+0000 7f6c6babf640 1 -- 172.21.15.49:0/1260454981 <== osd.3 v2:172.21.15.49:6824/38115 1 ==== osd_op_reply(7 [pgnls-filter start_epoch 82 out=49b] v80'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594df5ad440 con 0x5594dec46000 2023-11-06T02:14:35.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.754+0000 7f6c66ab5640 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.49:6809/38116,v1:172.21.15.49:6811/38116] -- osd_op(unknown.0.0:8 12.28 12:14000000::::head [pgnls-filter start_epoch 82 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594dec46c00 con 0x5594dec23800 2023-11-06T02:14:35.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.754+0000 7f6c6c2c0640 1 -- 172.21.15.49:0/1260454981 <== osd.0 v2:172.21.15.49:6809/38116 2 ==== osd_op_reply(8 [pgnls-filter start_epoch 82 out=49b] v80'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594df5ad200 con 0x5594dec23800 2023-11-06T02:14:35.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.754+0000 7f6c672b6640 1 --2- 172.21.15.49:0/1260454981 >> [v2:172.21.15.49:6808/38113,v1:172.21.15.49:6810/38113] conn(0x5594dec46800 0x5594dec1b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:35.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.754+0000 7f6c672b6640 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.49:6808/38113,v1:172.21.15.49:6810/38113] -- osd_op(unknown.0.0:9 12.18 12:18000000::::head [pgnls-filter start_epoch 82 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594dec23c00 con 0x5594dec46800 2023-11-06T02:14:35.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.758+0000 7f6c6b2be640 1 --2- 172.21.15.49:0/1260454981 >> [v2:172.21.15.49:6808/38113,v1:172.21.15.49:6810/38113] conn(0x5594dec46800 0x5594dec1b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:35.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.758+0000 7f6c6b2be640 1 --2- 172.21.15.49:0/1260454981 >> [v2:172.21.15.49:6808/38113,v1:172.21.15.49:6810/38113] conn(0x5594dec46800 0x5594dec1b600 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:35.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.758+0000 7f6c6b2be640 1 -- 172.21.15.49:0/1260454981 <== osd.1 v2:172.21.15.49:6808/38113 1 ==== osd_op_reply(9 [pgnls-filter start_epoch 82 out=49b] v80'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594df5acfc0 con 0x5594dec46800 2023-11-06T02:14:35.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.758+0000 7f6c66ab5640 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.49:6800/38114,v1:172.21.15.49:6801/38114] -- osd_op(unknown.0.0:10 12.38 12:1c000000::::head [pgnls-filter start_epoch 82 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594dec47000 con 0x5594dec14800 2023-11-06T02:14:35.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.758+0000 7f6c6c2c0640 1 -- 172.21.15.49:0/1260454981 <== osd.2 v2:172.21.15.49:6800/38114 2 ==== osd_op_reply(10 [pgnls-filter start_epoch 82 out=49b] v80'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594df5ad200 con 0x5594dec14800 2023-11-06T02:14:35.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.758+0000 7f6c672b6640 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.150:6824/38428,v1:172.21.15.150:6825/38428] -- osd_op(unknown.0.0:11 12.4 12:20000000::::head [pgnls-filter start_epoch 82 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594dec14c00 con 0x5594df763c00 2023-11-06T02:14:35.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.758+0000 7f6c6b2be640 1 -- 172.21.15.49:0/1260454981 <== osd.6 v2:172.21.15.150:6824/38428 2 ==== osd_op_reply(11 [pgnls-filter start_epoch 82 out=49b] v80'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594df5acfc0 con 0x5594df763c00 2023-11-06T02:14:35.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.758+0000 7f6c66ab5640 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.49:6800/38114,v1:172.21.15.49:6801/38114] -- osd_op(unknown.0.0:12 12.24 12:24000000::::head [pgnls-filter start_epoch 82 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594dec47400 con 0x5594dec14800 2023-11-06T02:14:35.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.758+0000 7f6c6c2c0640 1 -- 172.21.15.49:0/1260454981 <== osd.2 v2:172.21.15.49:6800/38114 3 ==== osd_op_reply(12 [pgnls-filter start_epoch 82 out=49b] v80'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594df5ad200 con 0x5594dec14800 2023-11-06T02:14:35.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.758+0000 7f6c672b6640 1 --2- 172.21.15.49:0/1260454981 >> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] conn(0x5594df762800 0x5594dec1bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:35.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.758+0000 7f6c672b6640 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] -- osd_op(unknown.0.0:13 12.14 12:28000000::::head [pgnls-filter start_epoch 82 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594df762400 con 0x5594df762800 2023-11-06T02:14:35.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.758+0000 7f6c6c2c0640 1 --2- 172.21.15.49:0/1260454981 >> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] conn(0x5594df762800 0x5594dec1bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:35.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.758+0000 7f6c6c2c0640 1 --2- 172.21.15.49:0/1260454981 >> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] conn(0x5594df762800 0x5594dec1bb80 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:35.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.762+0000 7f6c6c2c0640 1 -- 172.21.15.49:0/1260454981 <== osd.5 v2:172.21.15.150:6800/38418 1 ==== osd_op_reply(13 [pgnls-filter start_epoch 82 out=49b] v80'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594df5ad200 con 0x5594df762800 2023-11-06T02:14:35.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.762+0000 7f6c66ab5640 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] -- osd_op(unknown.0.0:14 12.34 12:2c000000::::head [pgnls-filter start_epoch 82 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594dec23000 con 0x5594df762800 2023-11-06T02:14:35.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.762+0000 7f6c6c2c0640 1 -- 172.21.15.49:0/1260454981 <== osd.5 v2:172.21.15.150:6800/38418 2 ==== osd_op_reply(14 [pgnls-filter start_epoch 82 out=49b] v80'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594df5ad200 con 0x5594df762800 2023-11-06T02:14:35.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.762+0000 7f6c672b6640 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.49:6809/38116,v1:172.21.15.49:6811/38116] -- osd_op(unknown.0.0:15 12.c 12:30000000::::head [pgnls-filter start_epoch 82 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594dec46c00 con 0x5594dec23800 2023-11-06T02:14:35.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.762+0000 7f6c6c2c0640 1 -- 172.21.15.49:0/1260454981 <== osd.0 v2:172.21.15.49:6809/38116 3 ==== osd_op_reply(15 [pgnls-filter start_epoch 82 out=49b] v80'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594df5ad200 con 0x5594dec23800 2023-11-06T02:14:35.778 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.762+0000 7f6c66ab5640 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.49:6800/38114,v1:172.21.15.49:6801/38114] -- osd_op(unknown.0.0:16 12.2c 12:34000000::::head [pgnls-filter start_epoch 82 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594dec23400 con 0x5594dec14800 2023-11-06T02:14:35.778 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.762+0000 7f6c6c2c0640 1 -- 172.21.15.49:0/1260454981 <== osd.2 v2:172.21.15.49:6800/38114 4 ==== osd_op_reply(16 [pgnls-filter start_epoch 82 out=49b] v80'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594df5ad200 con 0x5594dec14800 2023-11-06T02:14:35.779 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.762+0000 7f6c672b6640 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.49:6809/38116,v1:172.21.15.49:6811/38116] -- osd_op(unknown.0.0:17 12.1c 12:38000000::::head [pgnls-filter start_epoch 82 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594dec47800 con 0x5594dec23800 2023-11-06T02:14:35.779 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.762+0000 7f6c6c2c0640 1 -- 172.21.15.49:0/1260454981 <== osd.0 v2:172.21.15.49:6809/38116 4 ==== osd_op_reply(17 [pgnls-filter start_epoch 82 out=49b] v80'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594df5ad200 con 0x5594dec23800 2023-11-06T02:14:35.779 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.762+0000 7f6c66ab5640 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] -- osd_op(unknown.0.0:18 12.3c 12:3c000000::::head [pgnls-filter start_epoch 82 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594df61e800 con 0x5594df762800 2023-11-06T02:14:35.779 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.762+0000 7f6c6c2c0640 1 -- 172.21.15.49:0/1260454981 <== osd.5 v2:172.21.15.150:6800/38418 3 ==== osd_op_reply(18 [pgnls-filter start_epoch 82 out=49b] v80'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594df5ad200 con 0x5594df762800 2023-11-06T02:14:35.779 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.762+0000 7f6c672b6640 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.150:6808/38417,v1:172.21.15.150:6809/38417] -- osd_op(unknown.0.0:19 12.2 12:40000000::::head [pgnls-filter start_epoch 82 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594dec14400 con 0x5594dec15000 2023-11-06T02:14:35.780 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.766+0000 7f6c6babf640 1 -- 172.21.15.49:0/1260454981 <== osd.4 v2:172.21.15.150:6808/38417 2 ==== osd_op_reply(19 [pgnls-filter start_epoch 82 out=49b] v80'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594df5ad440 con 0x5594dec15000 2023-11-06T02:14:35.780 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.766+0000 7f6c66ab5640 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] -- osd_op(unknown.0.0:20 12.22 12:44000000::::head [pgnls-filter start_epoch 82 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594df61ec00 con 0x5594dec22c00 2023-11-06T02:14:35.780 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.766+0000 7f6c6b2be640 1 -- 172.21.15.49:0/1260454981 <== osd.7 v2:172.21.15.150:6816/38421 3 ==== osd_op_reply(20 [pgnls-filter start_epoch 82 out=49b] v80'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594df5acfc0 con 0x5594dec22c00 2023-11-06T02:14:35.780 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.766+0000 7f6c672b6640 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.49:6824/38115,v1:172.21.15.49:6825/38115] -- osd_op(unknown.0.0:21 12.12 12:48000000::::head [pgnls-filter start_epoch 82 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594dec14c00 con 0x5594dec46000 2023-11-06T02:14:35.780 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.766+0000 7f6c6babf640 1 -- 172.21.15.49:0/1260454981 <== osd.3 v2:172.21.15.49:6824/38115 2 ==== osd_op_reply(21 [pgnls-filter start_epoch 82 out=81b] v82'23 uv23 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5594df5ad440 con 0x5594dec46000 2023-11-06T02:14:35.781 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.766+0000 7f6c6d2ff740 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.150:6808/38417,v1:172.21.15.150:6809/38417] -- osd_op(unknown.0.0:22 12.0 12:00000000::::head [pgnls-filter start_epoch 82 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594dec15800 con 0x5594dec15000 2023-11-06T02:14:35.781 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.766+0000 7f6c6babf640 1 -- 172.21.15.49:0/1260454981 <== osd.4 v2:172.21.15.150:6808/38417 3 ==== osd_op_reply(22 [pgnls-filter start_epoch 82 out=49b] v80'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594df5ad440 con 0x5594dec15000 2023-11-06T02:14:35.781 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.766+0000 7f6c672b6640 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] -- osd_op(unknown.0.0:23 12.20 12:04000000::::head [pgnls-filter start_epoch 82 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594dec23c00 con 0x5594dec22c00 2023-11-06T02:14:35.781 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.770+0000 7f6c6b2be640 1 -- 172.21.15.49:0/1260454981 <== osd.7 v2:172.21.15.150:6816/38421 4 ==== osd_op_reply(23 [pgnls-filter start_epoch 82 out=49b] v80'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594df5acfc0 con 0x5594dec22c00 2023-11-06T02:14:35.781 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.770+0000 7f6c66ab5640 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.49:6809/38116,v1:172.21.15.49:6811/38116] -- osd_op(unknown.0.0:24 12.10 12:08000000::::head [pgnls-filter start_epoch 82 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594dec22800 con 0x5594dec23800 2023-11-06T02:14:35.781 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.770+0000 7f6c6c2c0640 1 -- 172.21.15.49:0/1260454981 <== osd.0 v2:172.21.15.49:6809/38116 5 ==== osd_op_reply(24 [pgnls-filter start_epoch 82 out=49b] v80'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594df5ad200 con 0x5594dec23800 2023-11-06T02:14:35.782 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.770+0000 7f6c672b6640 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] -- osd_op(unknown.0.0:25 12.30 12:0c000000::::head [pgnls-filter start_epoch 82 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594dec46400 con 0x5594dec22c00 2023-11-06T02:14:35.782 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.770+0000 7f6c6b2be640 1 -- 172.21.15.49:0/1260454981 <== osd.7 v2:172.21.15.150:6816/38421 5 ==== osd_op_reply(25 [pgnls-filter start_epoch 82 out=49b] v80'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594df5acfc0 con 0x5594dec22c00 2023-11-06T02:14:35.782 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.770+0000 7f6c66ab5640 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.49:6824/38115,v1:172.21.15.49:6825/38115] -- osd_op(unknown.0.0:26 12.8 12:10000000::::head [pgnls-filter start_epoch 82 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594dec23000 con 0x5594dec46000 2023-11-06T02:14:35.782 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.770+0000 7f6c6babf640 1 -- 172.21.15.49:0/1260454981 <== osd.3 v2:172.21.15.49:6824/38115 3 ==== osd_op_reply(26 [pgnls-filter start_epoch 82 out=49b] v80'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594df5ad440 con 0x5594dec46000 2023-11-06T02:14:35.782 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.770+0000 7f6c672b6640 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.49:6809/38116,v1:172.21.15.49:6811/38116] -- osd_op(unknown.0.0:27 12.28 12:14000000::::head [pgnls-filter start_epoch 82 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594dec47000 con 0x5594dec23800 2023-11-06T02:14:35.783 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.770+0000 7f6c6c2c0640 1 -- 172.21.15.49:0/1260454981 <== osd.0 v2:172.21.15.49:6809/38116 6 ==== osd_op_reply(27 [pgnls-filter start_epoch 82 out=49b] v80'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594df5ad200 con 0x5594dec23800 2023-11-06T02:14:35.783 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.770+0000 7f6c66ab5640 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.49:6808/38113,v1:172.21.15.49:6810/38113] -- osd_op(unknown.0.0:28 12.18 12:18000000::::head [pgnls-filter start_epoch 82 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594df762400 con 0x5594dec46800 2023-11-06T02:14:35.783 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.770+0000 7f6c6b2be640 1 -- 172.21.15.49:0/1260454981 <== osd.1 v2:172.21.15.49:6808/38113 2 ==== osd_op_reply(28 [pgnls-filter start_epoch 82 out=49b] v80'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594df5acfc0 con 0x5594dec46800 2023-11-06T02:14:35.783 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.770+0000 7f6c672b6640 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.49:6800/38114,v1:172.21.15.49:6801/38114] -- osd_op(unknown.0.0:29 12.38 12:1c000000::::head [pgnls-filter start_epoch 82 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594dec47400 con 0x5594dec14800 2023-11-06T02:14:35.783 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.770+0000 7f6c6c2c0640 1 -- 172.21.15.49:0/1260454981 <== osd.2 v2:172.21.15.49:6800/38114 5 ==== osd_op_reply(29 [pgnls-filter start_epoch 82 out=49b] v80'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594df5ad200 con 0x5594dec14800 2023-11-06T02:14:35.784 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.770+0000 7f6c66ab5640 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.150:6824/38428,v1:172.21.15.150:6825/38428] -- osd_op(unknown.0.0:30 12.4 12:20000000::::head [pgnls-filter start_epoch 82 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594dec15400 con 0x5594df763c00 2023-11-06T02:14:35.784 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.770+0000 7f6c6b2be640 1 -- 172.21.15.49:0/1260454981 <== osd.6 v2:172.21.15.150:6824/38428 3 ==== osd_op_reply(30 [pgnls-filter start_epoch 82 out=49b] v80'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594df5acfc0 con 0x5594df763c00 2023-11-06T02:14:35.784 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.770+0000 7f6c672b6640 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.49:6800/38114,v1:172.21.15.49:6801/38114] -- osd_op(unknown.0.0:31 12.24 12:24000000::::head [pgnls-filter start_epoch 82 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594dec46c00 con 0x5594dec14800 2023-11-06T02:14:35.784 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.774+0000 7f6c6c2c0640 1 -- 172.21.15.49:0/1260454981 <== osd.2 v2:172.21.15.49:6800/38114 6 ==== osd_op_reply(31 [pgnls-filter start_epoch 82 out=49b] v80'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594df5ad200 con 0x5594dec14800 2023-11-06T02:14:35.784 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.774+0000 7f6c66ab5640 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] -- osd_op(unknown.0.0:32 12.14 12:28000000::::head [pgnls-filter start_epoch 82 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594dec14400 con 0x5594df762800 2023-11-06T02:14:35.784 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.774+0000 7f6c6c2c0640 1 -- 172.21.15.49:0/1260454981 <== osd.5 v2:172.21.15.150:6800/38418 4 ==== osd_op_reply(32 [pgnls-filter start_epoch 82 out=49b] v80'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594df5ad200 con 0x5594df762800 2023-11-06T02:14:35.785 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.774+0000 7f6c672b6640 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] -- osd_op(unknown.0.0:33 12.34 12:2c000000::::head [pgnls-filter start_epoch 82 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594dec47c00 con 0x5594df762800 2023-11-06T02:14:35.792 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.774+0000 7f6c6c2c0640 1 -- 172.21.15.49:0/1260454981 <== osd.5 v2:172.21.15.150:6800/38418 5 ==== osd_op_reply(33 [pgnls-filter start_epoch 82 out=49b] v80'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594df5ad200 con 0x5594df762800 2023-11-06T02:14:35.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.774+0000 7f6c66ab5640 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.49:6809/38116,v1:172.21.15.49:6811/38116] -- osd_op(unknown.0.0:34 12.c 12:30000000::::head [pgnls-filter start_epoch 82 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594dec47800 con 0x5594dec23800 2023-11-06T02:14:35.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.774+0000 7f6c6c2c0640 1 -- 172.21.15.49:0/1260454981 <== osd.0 v2:172.21.15.49:6809/38116 7 ==== osd_op_reply(34 [pgnls-filter start_epoch 82 out=49b] v80'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594df5ad200 con 0x5594dec23800 2023-11-06T02:14:35.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.774+0000 7f6c672b6640 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.49:6800/38114,v1:172.21.15.49:6801/38114] -- osd_op(unknown.0.0:35 12.2c 12:34000000::::head [pgnls-filter start_epoch 82 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594dec23400 con 0x5594dec14800 2023-11-06T02:14:35.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.774+0000 7f6c6c2c0640 1 -- 172.21.15.49:0/1260454981 <== osd.2 v2:172.21.15.49:6800/38114 7 ==== osd_op_reply(35 [pgnls-filter start_epoch 82 out=49b] v80'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594df5ad200 con 0x5594dec14800 2023-11-06T02:14:35.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.774+0000 7f6c66ab5640 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.49:6809/38116,v1:172.21.15.49:6811/38116] -- osd_op(unknown.0.0:36 12.1c 12:38000000::::head [pgnls-filter start_epoch 82 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594dec47000 con 0x5594dec23800 2023-11-06T02:14:35.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.774+0000 7f6c6c2c0640 1 -- 172.21.15.49:0/1260454981 <== osd.0 v2:172.21.15.49:6809/38116 8 ==== osd_op_reply(36 [pgnls-filter start_epoch 82 out=49b] v80'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594df5ad200 con 0x5594dec23800 2023-11-06T02:14:35.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.774+0000 7f6c672b6640 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] -- osd_op(unknown.0.0:37 12.3c 12:3c000000::::head [pgnls-filter start_epoch 82 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594dec14400 con 0x5594df762800 2023-11-06T02:14:35.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.774+0000 7f6c6c2c0640 1 -- 172.21.15.49:0/1260454981 <== osd.5 v2:172.21.15.150:6800/38418 6 ==== osd_op_reply(37 [pgnls-filter start_epoch 82 out=49b] v80'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594df5ad200 con 0x5594df762800 2023-11-06T02:14:35.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.774+0000 7f6c66ab5640 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.150:6808/38417,v1:172.21.15.150:6809/38417] -- osd_op(unknown.0.0:38 12.2 12:40000000::::head [pgnls-filter start_epoch 82 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594dec47400 con 0x5594dec15000 2023-11-06T02:14:35.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.778+0000 7f6c6babf640 1 -- 172.21.15.49:0/1260454981 <== osd.4 v2:172.21.15.150:6808/38417 4 ==== osd_op_reply(38 [pgnls-filter start_epoch 82 out=49b] v80'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594df5ad440 con 0x5594dec15000 2023-11-06T02:14:35.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.778+0000 7f6c672b6640 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] -- osd_op(unknown.0.0:39 12.22 12:44000000::::head [pgnls-filter start_epoch 82 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594dec5c000 con 0x5594dec22c00 2023-11-06T02:14:35.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.778+0000 7f6c6b2be640 1 -- 172.21.15.49:0/1260454981 <== osd.7 v2:172.21.15.150:6816/38421 6 ==== osd_op_reply(39 [pgnls-filter start_epoch 82 out=49b] v80'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594df5acfc0 con 0x5594dec22c00 2023-11-06T02:14:35.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.778+0000 7f6c66ab5640 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.49:6824/38115,v1:172.21.15.49:6825/38115] -- osd_op(unknown.0.0:40 12.12 12:48000000::::head [pgnls-filter start_epoch 82 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594dec46c00 con 0x5594dec46000 2023-11-06T02:14:35.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.778+0000 7f6c6babf640 1 -- 172.21.15.49:0/1260454981 <== osd.3 v2:172.21.15.49:6824/38115 4 ==== osd_op_reply(40 [pgnls-filter start_epoch 82 out=49b] v82'23 uv23 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594df5ad440 con 0x5594dec46000 2023-11-06T02:14:35.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.778+0000 7f6c672b6640 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.49:6824/38115,v1:172.21.15.49:6825/38115] -- osd_op(unknown.0.0:41 12.32 12:4c000000::::head [pgnls-filter start_epoch 82 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594dec5c400 con 0x5594dec46000 2023-11-06T02:14:35.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.778+0000 7f6c6babf640 1 -- 172.21.15.49:0/1260454981 <== osd.3 v2:172.21.15.49:6824/38115 5 ==== osd_op_reply(41 [pgnls-filter start_epoch 82 out=49b] v80'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594df5ad440 con 0x5594dec46000 2023-11-06T02:14:35.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.778+0000 7f6c66ab5640 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.49:6800/38114,v1:172.21.15.49:6801/38114] -- osd_op(unknown.0.0:42 12.a 12:50000000::::head [pgnls-filter start_epoch 82 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594dec22800 con 0x5594dec14800 2023-11-06T02:14:35.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.778+0000 7f6c6c2c0640 1 -- 172.21.15.49:0/1260454981 <== osd.2 v2:172.21.15.49:6800/38114 8 ==== osd_op_reply(42 [pgnls-filter start_epoch 82 out=49b] v80'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594df5ad200 con 0x5594dec14800 2023-11-06T02:14:35.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.778+0000 7f6c672b6640 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.49:6800/38114,v1:172.21.15.49:6801/38114] -- osd_op(unknown.0.0:43 12.2a 12:54000000::::head [pgnls-filter start_epoch 82 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594dec5c800 con 0x5594dec14800 2023-11-06T02:14:35.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.778+0000 7f6c6c2c0640 1 -- 172.21.15.49:0/1260454981 <== osd.2 v2:172.21.15.49:6800/38114 9 ==== osd_op_reply(43 [pgnls-filter start_epoch 82 out=49b] v80'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594df5ad200 con 0x5594dec14800 2023-11-06T02:14:35.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.778+0000 7f6c66ab5640 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.150:6808/38417,v1:172.21.15.150:6809/38417] -- osd_op(unknown.0.0:44 12.1a 12:58000000::::head [pgnls-filter start_epoch 82 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594df61e800 con 0x5594dec15000 2023-11-06T02:14:35.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.778+0000 7f6c6babf640 1 -- 172.21.15.49:0/1260454981 <== osd.4 v2:172.21.15.150:6808/38417 5 ==== osd_op_reply(44 [pgnls-filter start_epoch 82 out=49b] v80'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594df5ad440 con 0x5594dec15000 2023-11-06T02:14:35.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.778+0000 7f6c672b6640 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.49:6824/38115,v1:172.21.15.49:6825/38115] -- osd_op(unknown.0.0:45 12.3a 12:5c000000::::head [pgnls-filter start_epoch 82 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594dec5cc00 con 0x5594dec46000 2023-11-06T02:14:35.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.782+0000 7f6c6babf640 1 -- 172.21.15.49:0/1260454981 <== osd.3 v2:172.21.15.49:6824/38115 6 ==== osd_op_reply(45 [pgnls-filter start_epoch 82 out=49b] v80'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594df5ad440 con 0x5594dec46000 2023-11-06T02:14:35.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.782+0000 7f6c66ab5640 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.49:6808/38113,v1:172.21.15.49:6810/38113] -- osd_op(unknown.0.0:46 12.6 12:60000000::::head [pgnls-filter start_epoch 82 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594dec47400 con 0x5594dec46800 2023-11-06T02:14:35.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.782+0000 7f6c6b2be640 1 -- 172.21.15.49:0/1260454981 <== osd.1 v2:172.21.15.49:6808/38113 3 ==== osd_op_reply(46 [pgnls-filter start_epoch 82 out=49b] v80'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594df5acfc0 con 0x5594dec46800 2023-11-06T02:14:35.798 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.782+0000 7f6c672b6640 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] -- osd_op(unknown.0.0:47 12.26 12:64000000::::head [pgnls-filter start_epoch 82 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594dec5d000 con 0x5594df762800 2023-11-06T02:14:35.798 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.782+0000 7f6c6c2c0640 1 -- 172.21.15.49:0/1260454981 <== osd.5 v2:172.21.15.150:6800/38418 7 ==== osd_op_reply(47 [pgnls-filter start_epoch 82 out=49b] v80'30 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594df5ad200 con 0x5594df762800 2023-11-06T02:14:35.798 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.782+0000 7f6c66ab5640 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.150:6824/38428,v1:172.21.15.150:6825/38428] -- osd_op(unknown.0.0:48 12.16 12:68000000::::head [pgnls-filter start_epoch 82 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594dec46c00 con 0x5594df763c00 2023-11-06T02:14:35.798 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.782+0000 7f6c6b2be640 1 -- 172.21.15.49:0/1260454981 <== osd.6 v2:172.21.15.150:6824/38428 4 ==== osd_op_reply(48 [pgnls-filter start_epoch 82 out=49b] v80'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594df5acfc0 con 0x5594df763c00 2023-11-06T02:14:35.798 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.782+0000 7f6c672b6640 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] -- osd_op(unknown.0.0:49 12.36 12:6c000000::::head [pgnls-filter start_epoch 82 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594dec5d400 con 0x5594df762800 2023-11-06T02:14:35.799 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.782+0000 7f6c6c2c0640 1 -- 172.21.15.49:0/1260454981 <== osd.5 v2:172.21.15.150:6800/38418 8 ==== osd_op_reply(49 [pgnls-filter start_epoch 82 out=81b] v82'34 uv34 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5594df5ad200 con 0x5594df762800 2023-11-06T02:14:35.799 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.782+0000 7f6c6d2ff740 20 datascan.forall_objects: OSD matched oid 100000001f7.00000000 2023-11-06T02:14:35.799 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.782+0000 7f6c6d2ff740 10 datascan.operator(): handling object 100000001f7.0 2023-11-06T02:14:35.799 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.782+0000 7f6c6d2ff740 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] -- osd_op(unknown.0.0:50 12.36 12:6fe38dc2:::100000001f7.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594dec15c00 con 0x5594df762800 2023-11-06T02:14:35.799 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.786+0000 7f6c6c2c0640 1 -- 172.21.15.49:0/1260454981 <== osd.5 v2:172.21.15.150:6800/38418 9 ==== osd_op_reply(50 100000001f7.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (126) out=126b,getxattr (30) out=30b,getxattr] v0'0 uv34 ondisk = 0) v8 ==== 416+0+194 (crc 0 0 0) 0x5594df5ad200 con 0x5594df762800 2023-11-06T02:14:35.800 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.786+0000 7f6c6d2ff740 10 datascan.inject_with_backtrace: inode: 0x0x100000001f7 2023-11-06T02:14:35.800 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.786+0000 7f6c6d2ff740 10 datascan.inject_with_backtrace: backptr: 0x0x10000000001/bravo 2023-11-06T02:14:35.800 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.786+0000 7f6c6d2ff740 20 datascan.get_frag_of: dirino=0x10000000001 target_dname=bravo 2023-11-06T02:14:35.800 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.786+0000 7f6c6d2ff740 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] -- osd_op(unknown.0.0:51 11.7 11:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594dec46400 con 0x5594df762800 2023-11-06T02:14:35.800 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.786+0000 7f6c6c2c0640 1 -- 172.21.15.49:0/1260454981 <== osd.5 v2:172.21.15.150:6800/38418 10 ==== osd_op_reply(51 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+95 (crc 0 0 0) 0x5594df5ad200 con 0x5594df762800 2023-11-06T02:14:35.800 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.786+0000 7f6c6d2ff740 20 datascan.get_frag_of: got backtrace parent 0x10000000000/flushed 2023-11-06T02:14:35.801 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.786+0000 7f6c6d2ff740 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.49:6809/38116,v1:172.21.15.49:6811/38116] -- osd_op(unknown.0.0:52 11.10 11:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594dec47400 con 0x5594dec23800 2023-11-06T02:14:35.801 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.786+0000 7f6c6c2c0640 1 -- 172.21.15.49:0/1260454981 <== osd.0 v2:172.21.15.49:6809/38116 9 ==== osd_op_reply(52 10000000000.00000000 [omap-get-vals-by-keys out=524b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+524 (crc 0 0 0) 0x5594df5ad200 con 0x5594dec23800 2023-11-06T02:14:35.801 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.786+0000 7f6c6d2ff740 20 datascan.read_dentry: oid=10000000000.00000000 dname=flushed frag=*, r=0 2023-11-06T02:14:35.801 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.786+0000 7f6c6d2ff740 20 datascan.get_frag_of: fast path, fragtree is fragtree_t() 2023-11-06T02:14:35.801 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.786+0000 7f6c6d2ff740 20 datascan.get_frag_of: frag is * 2023-11-06T02:14:35.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.786+0000 7f6c6d2ff740 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] -- osd_op(unknown.0.0:53 11.7 11:e2e07930:::10000000001.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594dec46c00 con 0x5594df762800 2023-11-06T02:14:35.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.786+0000 7f6c6c2c0640 1 -- 172.21.15.49:0/1260454981 <== osd.5 v2:172.21.15.150:6800/38418 11 ==== osd_op_reply(53 10000000001.00000000 [omap-get-header out=274b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5594df5ad200 con 0x5594df762800 2023-11-06T02:14:35.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.786+0000 7f6c6d2ff740 10 datascan.find_or_create_dirfrag: read_version = 8 2023-11-06T02:14:35.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.786+0000 7f6c6d2ff740 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000001 * 2023-11-06T02:14:35.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.786+0000 7f6c6d2ff740 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] -- osd_op(unknown.0.0:54 11.7 11:e2e07930:::10000000001.00000000:head [omap-get-vals-by-keys in=18b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594dec47c00 con 0x5594df762800 2023-11-06T02:14:35.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.786+0000 7f6c6c2c0640 1 -- 172.21.15.49:0/1260454981 <== osd.5 v2:172.21.15.150:6800/38418 12 ==== osd_op_reply(54 10000000001.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x5594df5ad200 con 0x5594df762800 2023-11-06T02:14:35.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.786+0000 7f6c6d2ff740 20 datascan.read_dentry: oid=10000000001.00000000 dname=bravo frag=*, r=0 2023-11-06T02:14:35.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.786+0000 7f6c6d2ff740 20 datascan.read_dentry: bravo_head not found in result 2023-11-06T02:14:35.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.786+0000 7f6c6d2ff740 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001f7 at 0x0x10000000001/bravo with size=5 bytes 2023-11-06T02:14:35.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.786+0000 7f6c6d2ff740 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] -- osd_op(unknown.0.0:55 11.7 11:e2e07930:::10000000001.00000000:head [omap-set-vals in=506b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594dec47800 con 0x5594df762800 2023-11-06T02:14:35.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.790+0000 7f6c6c2c0640 1 -- 172.21.15.49:0/1260454981 <== osd.5 v2:172.21.15.150:6800/38418 13 ==== osd_op_reply(55 10000000001.00000000 [omap-set-vals] v82'9 uv9 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5594df5ad200 con 0x5594df762800 2023-11-06T02:14:35.804 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.790+0000 7f6c6d2ff740 20 datascan.inject_linkage: Injected dentry 0x0x10000000001/bravo pointing to 0x0x100000001f7 2023-11-06T02:14:35.804 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.790+0000 7f6c6d2ff740 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.49:6809/38116,v1:172.21.15.49:6811/38116] -- osd_op(unknown.0.0:56 12.e 12:70000000::::head [pgnls-filter start_epoch 82 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594dec47000 con 0x5594dec23800 2023-11-06T02:14:35.804 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.790+0000 7f6c6c2c0640 1 -- 172.21.15.49:0/1260454981 <== osd.0 v2:172.21.15.49:6809/38116 10 ==== osd_op_reply(56 [pgnls-filter start_epoch 82 out=49b] v80'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594df5ad200 con 0x5594dec23800 2023-11-06T02:14:35.804 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.790+0000 7f6c672b6640 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.49:6808/38113,v1:172.21.15.49:6810/38113] -- osd_op(unknown.0.0:57 12.2e 12:74000000::::head [pgnls-filter start_epoch 82 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594dec5d800 con 0x5594dec46800 2023-11-06T02:14:35.804 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.790+0000 7f6c6b2be640 1 -- 172.21.15.49:0/1260454981 <== osd.1 v2:172.21.15.49:6808/38113 4 ==== osd_op_reply(57 [pgnls-filter start_epoch 82 out=49b] v80'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594df5acfc0 con 0x5594dec46800 2023-11-06T02:14:35.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.790+0000 7f6c66ab5640 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.150:6808/38417,v1:172.21.15.150:6809/38417] -- osd_op(unknown.0.0:58 12.1e 12:78000000::::head [pgnls-filter start_epoch 82 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594dec23000 con 0x5594dec15000 2023-11-06T02:14:35.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.790+0000 7f6c6babf640 1 -- 172.21.15.49:0/1260454981 <== osd.4 v2:172.21.15.150:6808/38417 6 ==== osd_op_reply(58 [pgnls-filter start_epoch 82 out=49b] v80'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594df5ad440 con 0x5594dec15000 2023-11-06T02:14:35.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.790+0000 7f6c672b6640 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.49:6809/38116,v1:172.21.15.49:6811/38116] -- osd_op(unknown.0.0:59 12.3e 12:7c000000::::head [pgnls-filter start_epoch 82 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594dec5dc00 con 0x5594dec23800 2023-11-06T02:14:35.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.790+0000 7f6c6c2c0640 1 -- 172.21.15.49:0/1260454981 <== osd.0 v2:172.21.15.49:6809/38116 11 ==== osd_op_reply(59 [pgnls-filter start_epoch 82 out=49b] v80'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594df5ad200 con 0x5594dec23800 2023-11-06T02:14:35.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.790+0000 7f6c66ab5640 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.150:6824/38428,v1:172.21.15.150:6825/38428] -- osd_op(unknown.0.0:60 12.1 12:80000000::::head [pgnls-filter start_epoch 82 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594dec14c00 con 0x5594df763c00 2023-11-06T02:14:35.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.794+0000 7f6c6b2be640 1 -- 172.21.15.49:0/1260454981 <== osd.6 v2:172.21.15.150:6824/38428 5 ==== osd_op_reply(60 [pgnls-filter start_epoch 82 out=49b] v80'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594df5acfc0 con 0x5594df763c00 2023-11-06T02:14:35.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.794+0000 7f6c672b6640 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.49:6808/38113,v1:172.21.15.49:6810/38113] -- osd_op(unknown.0.0:61 12.21 12:84000000::::head [pgnls-filter start_epoch 82 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594dec15c00 con 0x5594dec46800 2023-11-06T02:14:35.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.794+0000 7f6c6b2be640 1 -- 172.21.15.49:0/1260454981 <== osd.1 v2:172.21.15.49:6808/38113 5 ==== osd_op_reply(61 [pgnls-filter start_epoch 82 out=49b] v80'24 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594df5acfc0 con 0x5594dec46800 2023-11-06T02:14:35.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.794+0000 7f6c66ab5640 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] -- osd_op(unknown.0.0:62 12.11 12:88000000::::head [pgnls-filter start_epoch 82 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594dec15800 con 0x5594df762800 2023-11-06T02:14:35.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.794+0000 7f6c6c2c0640 1 -- 172.21.15.49:0/1260454981 <== osd.5 v2:172.21.15.150:6800/38418 14 ==== osd_op_reply(62 [pgnls-filter start_epoch 82 out=49b] v80'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594df5ad200 con 0x5594df762800 2023-11-06T02:14:35.807 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.794+0000 7f6c672b6640 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.49:6809/38116,v1:172.21.15.49:6811/38116] -- osd_op(unknown.0.0:63 12.31 12:8c000000::::head [pgnls-filter start_epoch 82 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594dec15400 con 0x5594dec23800 2023-11-06T02:14:35.807 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.794+0000 7f6c6c2c0640 1 -- 172.21.15.49:0/1260454981 <== osd.0 v2:172.21.15.49:6809/38116 12 ==== osd_op_reply(63 [pgnls-filter start_epoch 82 out=49b] v80'15 uv8 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594df5ad200 con 0x5594dec23800 2023-11-06T02:14:35.807 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.794+0000 7f6c66ab5640 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.49:6800/38114,v1:172.21.15.49:6801/38114] -- osd_op(unknown.0.0:64 12.9 12:90000000::::head [pgnls-filter start_epoch 82 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594dec5d400 con 0x5594dec14800 2023-11-06T02:14:35.807 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.794+0000 7f6c6c2c0640 1 -- 172.21.15.49:0/1260454981 <== osd.2 v2:172.21.15.49:6800/38114 10 ==== osd_op_reply(64 [pgnls-filter start_epoch 82 out=49b] v80'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594df5ad200 con 0x5594dec14800 2023-11-06T02:14:35.807 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.794+0000 7f6c672b6640 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.150:6808/38417,v1:172.21.15.150:6809/38417] -- osd_op(unknown.0.0:65 12.29 12:94000000::::head [pgnls-filter start_epoch 82 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594dec14400 con 0x5594dec15000 2023-11-06T02:14:35.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.794+0000 7f6c6babf640 1 -- 172.21.15.49:0/1260454981 <== osd.4 v2:172.21.15.150:6808/38417 7 ==== osd_op_reply(65 [pgnls-filter start_epoch 82 out=49b] v80'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594df5ad440 con 0x5594dec15000 2023-11-06T02:14:35.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.794+0000 7f6c66ab5640 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.49:6809/38116,v1:172.21.15.49:6811/38116] -- osd_op(unknown.0.0:66 12.19 12:98000000::::head [pgnls-filter start_epoch 82 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594dec5dc00 con 0x5594dec23800 2023-11-06T02:14:35.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.798+0000 7f6c6c2c0640 1 -- 172.21.15.49:0/1260454981 <== osd.0 v2:172.21.15.49:6809/38116 13 ==== osd_op_reply(66 [pgnls-filter start_epoch 82 out=49b] v82'24 uv24 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594df5ad200 con 0x5594dec23800 2023-11-06T02:14:35.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.798+0000 7f6c672b6640 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.150:6824/38428,v1:172.21.15.150:6825/38428] -- osd_op(unknown.0.0:67 12.39 12:9c000000::::head [pgnls-filter start_epoch 82 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594dec46400 con 0x5594df763c00 2023-11-06T02:14:35.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.798+0000 7f6c6b2be640 1 -- 172.21.15.49:0/1260454981 <== osd.6 v2:172.21.15.150:6824/38428 6 ==== osd_op_reply(67 [pgnls-filter start_epoch 82 out=49b] v80'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594df5acfc0 con 0x5594df763c00 2023-11-06T02:14:35.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.798+0000 7f6c66ab5640 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.150:6824/38428,v1:172.21.15.150:6825/38428] -- osd_op(unknown.0.0:68 12.5 12:a0000000::::head [pgnls-filter start_epoch 82 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594dec5c000 con 0x5594df763c00 2023-11-06T02:14:35.816 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.798+0000 7f6c6b2be640 1 -- 172.21.15.49:0/1260454981 <== osd.6 v2:172.21.15.150:6824/38428 7 ==== osd_op_reply(68 [pgnls-filter start_epoch 82 out=49b] v82'15 uv15 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594df5acfc0 con 0x5594df763c00 2023-11-06T02:14:35.816 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.798+0000 7f6c672b6640 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] -- osd_op(unknown.0.0:69 12.25 12:a4000000::::head [pgnls-filter start_epoch 82 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594dec47400 con 0x5594dec22c00 2023-11-06T02:14:35.816 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.798+0000 7f6c6b2be640 1 -- 172.21.15.49:0/1260454981 <== osd.7 v2:172.21.15.150:6816/38421 7 ==== osd_op_reply(69 [pgnls-filter start_epoch 82 out=49b] v80'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594df5acfc0 con 0x5594dec22c00 2023-11-06T02:14:35.816 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.798+0000 7f6c66ab5640 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.150:6824/38428,v1:172.21.15.150:6825/38428] -- osd_op(unknown.0.0:70 12.15 12:a8000000::::head [pgnls-filter start_epoch 82 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594dec5c800 con 0x5594df763c00 2023-11-06T02:14:35.816 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.802+0000 7f6c6b2be640 1 -- 172.21.15.49:0/1260454981 <== osd.6 v2:172.21.15.150:6824/38428 8 ==== osd_op_reply(70 [pgnls-filter start_epoch 82 out=49b] v80'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594df5acfc0 con 0x5594df763c00 2023-11-06T02:14:35.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.802+0000 7f6c672b6640 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.49:6809/38116,v1:172.21.15.49:6811/38116] -- osd_op(unknown.0.0:71 12.35 12:ac000000::::head [pgnls-filter start_epoch 82 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594dec46c00 con 0x5594dec23800 2023-11-06T02:14:35.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.802+0000 7f6c6c2c0640 1 -- 172.21.15.49:0/1260454981 <== osd.0 v2:172.21.15.49:6809/38116 14 ==== osd_op_reply(71 [pgnls-filter start_epoch 82 out=49b] v80'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594df5ad200 con 0x5594dec23800 2023-11-06T02:14:35.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.802+0000 7f6c66ab5640 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] -- osd_op(unknown.0.0:72 12.d 12:b0000000::::head [pgnls-filter start_epoch 82 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594dec5d000 con 0x5594dec22c00 2023-11-06T02:14:35.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.802+0000 7f6c6b2be640 1 -- 172.21.15.49:0/1260454981 <== osd.7 v2:172.21.15.150:6816/38421 8 ==== osd_op_reply(72 [pgnls-filter start_epoch 82 out=49b] v80'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594df5acfc0 con 0x5594dec22c00 2023-11-06T02:14:35.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.802+0000 7f6c672b6640 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.49:6824/38115,v1:172.21.15.49:6825/38115] -- osd_op(unknown.0.0:73 12.2d 12:b4000000::::head [pgnls-filter start_epoch 82 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594dec47c00 con 0x5594dec46000 2023-11-06T02:14:35.818 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.802+0000 7f6c6babf640 1 -- 172.21.15.49:0/1260454981 <== osd.3 v2:172.21.15.49:6824/38115 7 ==== osd_op_reply(73 [pgnls-filter start_epoch 82 out=49b] v80'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594df5ad440 con 0x5594dec46000 2023-11-06T02:14:35.818 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.802+0000 7f6c66ab5640 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.49:6809/38116,v1:172.21.15.49:6811/38116] -- osd_op(unknown.0.0:74 12.1d 12:b8000000::::head [pgnls-filter start_epoch 82 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594dec23c00 con 0x5594dec23800 2023-11-06T02:14:35.818 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.802+0000 7f6c6c2c0640 1 -- 172.21.15.49:0/1260454981 <== osd.0 v2:172.21.15.49:6809/38116 15 ==== osd_op_reply(74 [pgnls-filter start_epoch 82 out=49b] v80'30 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594df5ad200 con 0x5594dec23800 2023-11-06T02:14:35.818 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.802+0000 7f6c672b6640 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.49:6808/38113,v1:172.21.15.49:6810/38113] -- osd_op(unknown.0.0:75 12.3d 12:bc000000::::head [pgnls-filter start_epoch 82 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594dec47800 con 0x5594dec46800 2023-11-06T02:14:35.818 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.802+0000 7f6c6b2be640 1 -- 172.21.15.49:0/1260454981 <== osd.1 v2:172.21.15.49:6808/38113 6 ==== osd_op_reply(75 [pgnls-filter start_epoch 82 out=49b] v80'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594df5acfc0 con 0x5594dec46800 2023-11-06T02:14:35.819 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.802+0000 7f6c66ab5640 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.49:6809/38116,v1:172.21.15.49:6811/38116] -- osd_op(unknown.0.0:76 12.3 12:c0000000::::head [pgnls-filter start_epoch 82 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594dec23400 con 0x5594dec23800 2023-11-06T02:14:35.819 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.802+0000 7f6c6c2c0640 1 -- 172.21.15.49:0/1260454981 <== osd.0 v2:172.21.15.49:6809/38116 16 ==== osd_op_reply(76 [pgnls-filter start_epoch 82 out=49b] v80'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594df5ad200 con 0x5594dec23800 2023-11-06T02:14:35.819 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.802+0000 7f6c672b6640 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.49:6809/38116,v1:172.21.15.49:6811/38116] -- osd_op(unknown.0.0:77 12.23 12:c4000000::::head [pgnls-filter start_epoch 82 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594dec47000 con 0x5594dec23800 2023-11-06T02:14:35.819 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.806+0000 7f6c6c2c0640 1 -- 172.21.15.49:0/1260454981 <== osd.0 v2:172.21.15.49:6809/38116 17 ==== osd_op_reply(77 [pgnls-filter start_epoch 82 out=49b] v80'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594df5ad200 con 0x5594dec23800 2023-11-06T02:14:35.819 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.806+0000 7f6c66ab5640 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.49:6808/38113,v1:172.21.15.49:6810/38113] -- osd_op(unknown.0.0:78 12.13 12:c8000000::::head [pgnls-filter start_epoch 82 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594dec22800 con 0x5594dec46800 2023-11-06T02:14:35.820 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.806+0000 7f6c6b2be640 1 -- 172.21.15.49:0/1260454981 <== osd.1 v2:172.21.15.49:6808/38113 7 ==== osd_op_reply(78 [pgnls-filter start_epoch 82 out=49b] v80'30 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594df5acfc0 con 0x5594dec46800 2023-11-06T02:14:35.820 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.806+0000 7f6c672b6640 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.49:6800/38114,v1:172.21.15.49:6801/38114] -- osd_op(unknown.0.0:79 12.33 12:cc000000::::head [pgnls-filter start_epoch 82 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594dec14c00 con 0x5594dec14800 2023-11-06T02:14:35.820 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.806+0000 7f6c6c2c0640 1 -- 172.21.15.49:0/1260454981 <== osd.2 v2:172.21.15.49:6800/38114 11 ==== osd_op_reply(79 [pgnls-filter start_epoch 82 out=49b] v80'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594df5ad200 con 0x5594dec14800 2023-11-06T02:14:35.820 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.806+0000 7f6c66ab5640 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.49:6824/38115,v1:172.21.15.49:6825/38115] -- osd_op(unknown.0.0:80 12.b 12:d0000000::::head [pgnls-filter start_epoch 82 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594dec46400 con 0x5594dec46000 2023-11-06T02:14:35.820 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.806+0000 7f6c6babf640 1 -- 172.21.15.49:0/1260454981 <== osd.3 v2:172.21.15.49:6824/38115 8 ==== osd_op_reply(80 [pgnls-filter start_epoch 82 out=49b] v80'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594df5ad440 con 0x5594dec46000 2023-11-06T02:14:35.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.806+0000 7f6c672b6640 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.150:6808/38417,v1:172.21.15.150:6809/38417] -- osd_op(unknown.0.0:81 12.2b 12:d4000000::::head [pgnls-filter start_epoch 82 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594dec15c00 con 0x5594dec15000 2023-11-06T02:14:35.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.806+0000 7f6c6babf640 1 -- 172.21.15.49:0/1260454981 <== osd.4 v2:172.21.15.150:6808/38417 8 ==== osd_op_reply(81 [pgnls-filter start_epoch 82 out=49b] v80'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594df5ad440 con 0x5594dec15000 2023-11-06T02:14:35.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.806+0000 7f6c66ab5640 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.150:6808/38417,v1:172.21.15.150:6809/38417] -- osd_op(unknown.0.0:82 12.1b 12:d8000000::::head [pgnls-filter start_epoch 82 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594dec47400 con 0x5594dec15000 2023-11-06T02:14:35.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.806+0000 7f6c6babf640 1 -- 172.21.15.49:0/1260454981 <== osd.4 v2:172.21.15.150:6808/38417 9 ==== osd_op_reply(82 [pgnls-filter start_epoch 82 out=49b] v80'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594df5ad440 con 0x5594dec15000 2023-11-06T02:14:35.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.806+0000 7f6c672b6640 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.49:6800/38114,v1:172.21.15.49:6801/38114] -- osd_op(unknown.0.0:83 12.3b 12:dc000000::::head [pgnls-filter start_epoch 82 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594dec23000 con 0x5594dec14800 2023-11-06T02:14:35.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.810+0000 7f6c6c2c0640 1 -- 172.21.15.49:0/1260454981 <== osd.2 v2:172.21.15.49:6800/38114 12 ==== osd_op_reply(83 [pgnls-filter start_epoch 82 out=49b] v80'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594df5ad200 con 0x5594dec14800 2023-11-06T02:14:35.822 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.810+0000 7f6c66ab5640 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.49:6808/38113,v1:172.21.15.49:6810/38113] -- osd_op(unknown.0.0:84 12.7 12:e0000000::::head [pgnls-filter start_epoch 82 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594dec47800 con 0x5594dec46800 2023-11-06T02:14:35.822 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.810+0000 7f6c6b2be640 1 -- 172.21.15.49:0/1260454981 <== osd.1 v2:172.21.15.49:6808/38113 8 ==== osd_op_reply(84 [pgnls-filter start_epoch 82 out=49b] v80'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594df5acfc0 con 0x5594dec46800 2023-11-06T02:14:35.822 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.810+0000 7f6c672b6640 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.49:6809/38116,v1:172.21.15.49:6811/38116] -- osd_op(unknown.0.0:85 12.27 12:e4000000::::head [pgnls-filter start_epoch 82 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594dec5d800 con 0x5594dec23800 2023-11-06T02:14:35.822 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.810+0000 7f6c6c2c0640 1 -- 172.21.15.49:0/1260454981 <== osd.0 v2:172.21.15.49:6809/38116 18 ==== osd_op_reply(85 [pgnls-filter start_epoch 82 out=49b] v80'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594df5ad200 con 0x5594dec23800 2023-11-06T02:14:35.822 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.810+0000 7f6c66ab5640 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.150:6824/38428,v1:172.21.15.150:6825/38428] -- osd_op(unknown.0.0:86 12.17 12:e8000000::::head [pgnls-filter start_epoch 82 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594dec47c00 con 0x5594df763c00 2023-11-06T02:14:35.823 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.810+0000 7f6c6b2be640 1 -- 172.21.15.49:0/1260454981 <== osd.6 v2:172.21.15.150:6824/38428 9 ==== osd_op_reply(86 [pgnls-filter start_epoch 82 out=49b] v80'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594df5acfc0 con 0x5594df763c00 2023-11-06T02:14:35.823 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.810+0000 7f6c672b6640 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.150:6808/38417,v1:172.21.15.150:6809/38417] -- osd_op(unknown.0.0:87 12.37 12:ec000000::::head [pgnls-filter start_epoch 82 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594dec5c000 con 0x5594dec15000 2023-11-06T02:14:35.823 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.810+0000 7f6c6babf640 1 -- 172.21.15.49:0/1260454981 <== osd.4 v2:172.21.15.150:6808/38417 10 ==== osd_op_reply(87 [pgnls-filter start_epoch 82 out=49b] v80'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594df5ad440 con 0x5594dec15000 2023-11-06T02:14:35.823 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.810+0000 7f6c66ab5640 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.150:6824/38428,v1:172.21.15.150:6825/38428] -- osd_op(unknown.0.0:88 12.f 12:f0000000::::head [pgnls-filter start_epoch 82 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594dec14400 con 0x5594df763c00 2023-11-06T02:14:35.823 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.810+0000 7f6c6b2be640 1 -- 172.21.15.49:0/1260454981 <== osd.6 v2:172.21.15.150:6824/38428 10 ==== osd_op_reply(88 [pgnls-filter start_epoch 82 out=49b] v80'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594df5acfc0 con 0x5594df763c00 2023-11-06T02:14:35.824 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.810+0000 7f6c672b6640 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.49:6800/38114,v1:172.21.15.49:6801/38114] -- osd_op(unknown.0.0:89 12.2f 12:f4000000::::head [pgnls-filter start_epoch 82 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594dec5c800 con 0x5594dec14800 2023-11-06T02:14:35.824 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.814+0000 7f6c6c2c0640 1 -- 172.21.15.49:0/1260454981 <== osd.2 v2:172.21.15.49:6800/38114 13 ==== osd_op_reply(89 [pgnls-filter start_epoch 82 out=49b] v80'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594df5ad200 con 0x5594dec14800 2023-11-06T02:14:35.824 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.814+0000 7f6c66ab5640 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.150:6824/38428,v1:172.21.15.150:6825/38428] -- osd_op(unknown.0.0:90 12.1f 12:f8000000::::head [pgnls-filter start_epoch 82 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594dec23c00 con 0x5594df763c00 2023-11-06T02:14:35.824 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.814+0000 7f6c6b2be640 1 -- 172.21.15.49:0/1260454981 <== osd.6 v2:172.21.15.150:6824/38428 11 ==== osd_op_reply(90 [pgnls-filter start_epoch 82 out=49b] v80'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594df5acfc0 con 0x5594df763c00 2023-11-06T02:14:35.824 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.814+0000 7f6c672b6640 1 -- 172.21.15.49:0/1260454981 --> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] -- osd_op(unknown.0.0:91 12.3f 12:fc000000::::head [pgnls-filter start_epoch 82 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x5594dec5d000 con 0x5594dec22c00 2023-11-06T02:14:35.824 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.814+0000 7f6c6b2be640 1 -- 172.21.15.49:0/1260454981 <== osd.7 v2:172.21.15.150:6816/38421 9 ==== osd_op_reply(91 [pgnls-filter start_epoch 82 out=49b] v80'17 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5594df5acfc0 con 0x5594dec22c00 2023-11-06T02:14:35.825 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.814+0000 7f6c6d2ff740 1 -- 172.21.15.49:0/1260454981 >> [v2:172.21.15.49:6809/38116,v1:172.21.15.49:6811/38116] conn(0x5594dec23800 msgr2=0x5594dec1ab00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:35.825 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.814+0000 7f6c6d2ff740 1 --2- 172.21.15.49:0/1260454981 >> [v2:172.21.15.49:6809/38116,v1:172.21.15.49:6811/38116] conn(0x5594dec23800 0x5594dec1ab00 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.825 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.814+0000 7f6c6d2ff740 1 -- 172.21.15.49:0/1260454981 >> [v2:172.21.15.49:6808/38113,v1:172.21.15.49:6810/38113] conn(0x5594dec46800 msgr2=0x5594dec1b600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:35.825 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.814+0000 7f6c6d2ff740 1 --2- 172.21.15.49:0/1260454981 >> [v2:172.21.15.49:6808/38113,v1:172.21.15.49:6810/38113] conn(0x5594dec46800 0x5594dec1b600 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.825 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.814+0000 7f6c6d2ff740 1 -- 172.21.15.49:0/1260454981 >> [v2:172.21.15.49:6800/38114,v1:172.21.15.49:6801/38114] conn(0x5594dec14800 msgr2=0x5594de8dd180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:35.826 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.814+0000 7f6c6d2ff740 1 --2- 172.21.15.49:0/1260454981 >> [v2:172.21.15.49:6800/38114,v1:172.21.15.49:6801/38114] conn(0x5594dec14800 0x5594de8dd180 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.826 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.814+0000 7f6c6d2ff740 1 -- 172.21.15.49:0/1260454981 >> [v2:172.21.15.49:6824/38115,v1:172.21.15.49:6825/38115] conn(0x5594dec46000 msgr2=0x5594dec1b080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:35.826 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.814+0000 7f6c6d2ff740 1 --2- 172.21.15.49:0/1260454981 >> [v2:172.21.15.49:6824/38115,v1:172.21.15.49:6825/38115] conn(0x5594dec46000 0x5594dec1b080 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.826 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.814+0000 7f6c6babf640 1 -- 172.21.15.49:0/1260454981 reap_dead start 2023-11-06T02:14:35.826 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.814+0000 7f6c6d2ff740 1 -- 172.21.15.49:0/1260454981 >> [v2:172.21.15.150:6808/38417,v1:172.21.15.150:6809/38417] conn(0x5594dec15000 msgr2=0x5594de8dd700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:35.827 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.814+0000 7f6c6d2ff740 1 --2- 172.21.15.49:0/1260454981 >> [v2:172.21.15.150:6808/38417,v1:172.21.15.150:6809/38417] conn(0x5594dec15000 0x5594de8dd700 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.827 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.814+0000 7f6c6babf640 1 -- 172.21.15.49:0/1260454981 reap_dead start 2023-11-06T02:14:35.827 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.814+0000 7f6c6d2ff740 1 -- 172.21.15.49:0/1260454981 >> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] conn(0x5594df762800 msgr2=0x5594dec1bb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:35.827 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.814+0000 7f6c6d2ff740 1 --2- 172.21.15.49:0/1260454981 >> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] conn(0x5594df762800 0x5594dec1bb80 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.827 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.814+0000 7f6c6d2ff740 1 -- 172.21.15.49:0/1260454981 >> [v2:172.21.15.150:6824/38428,v1:172.21.15.150:6825/38428] conn(0x5594df763c00 msgr2=0x5594de8dcc00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:35.827 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.814+0000 7f6c6d2ff740 1 --2- 172.21.15.49:0/1260454981 >> [v2:172.21.15.150:6824/38428,v1:172.21.15.150:6825/38428] conn(0x5594df763c00 0x5594de8dcc00 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.828 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.814+0000 7f6c6d2ff740 1 -- 172.21.15.49:0/1260454981 >> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] conn(0x5594dec22c00 msgr2=0x5594dec1a580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:35.828 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.814+0000 7f6c6d2ff740 1 --2- 172.21.15.49:0/1260454981 >> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] conn(0x5594dec22c00 0x5594dec1a580 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.828 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.814+0000 7f6c6d2ff740 1 -- 172.21.15.49:0/1260454981 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x5594dec14000 msgr2=0x5594dec1a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:35.828 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.814+0000 7f6c6d2ff740 1 --2- 172.21.15.49:0/1260454981 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x5594dec14000 0x5594dec1a000 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x5594df618150 tx=0x5594df780f00 comp rx=0 tx=0).stop 2023-11-06T02:14:35.828 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.814+0000 7f6c6d2ff740 1 -- 172.21.15.49:0/1260454981 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5594df762c00 msgr2=0x5594de8dab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:35.829 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.814+0000 7f6c6d2ff740 1 --2- 172.21.15.49:0/1260454981 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5594df762c00 0x5594de8dab00 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x5594df7d4330 tx=0x5594df6321e0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.829 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.814+0000 7f6c6babf640 1 -- 172.21.15.49:0/1260454981 reap_dead start 2023-11-06T02:14:35.829 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.814+0000 7f6c6d2ff740 1 -- 172.21.15.49:0/1260454981 shutdown_connections 2023-11-06T02:14:35.829 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.814+0000 7f6c6d2ff740 1 -- 172.21.15.49:0/1260454981 >> 172.21.15.49:0/1260454981 conn(0x5594de84ec00 msgr2=0x5594de87b800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:35.829 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.814+0000 7f6c6d2ff740 1 -- 172.21.15.49:0/1260454981 shutdown_connections 2023-11-06T02:14:35.830 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.814+0000 7f6c6d2ff740 1 -- 172.21.15.49:0/1260454981 wait complete. 2023-11-06T02:14:35.830 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.814+0000 7f6c6d2ff740 1 -- 172.21.15.49:0/2085664153 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5594de84f000 msgr2=0x5594de8db080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:35.830 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.814+0000 7f6c6d2ff740 1 --2- 172.21.15.49:0/2085664153 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5594de84f000 0x5594de8db080 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x5594df6d2d50 tx=0x5594df633020 comp rx=0 tx=0).stop 2023-11-06T02:14:35.830 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.814+0000 7f6c6d2ff740 1 -- 172.21.15.49:0/2085664153 shutdown_connections 2023-11-06T02:14:35.830 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.814+0000 7f6c6d2ff740 1 --2- 172.21.15.49:0/2085664153 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5594de84f000 0x5594de8db080 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.830 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.814+0000 7f6c6d2ff740 1 --2- 172.21.15.49:0/2085664153 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5594de84f400 0x5594de8db600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.831 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.814+0000 7f6c6d2ff740 1 --2- 172.21.15.49:0/2085664153 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5594de84f800 0x5594de8dbb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.831 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.814+0000 7f6c6d2ff740 1 -- 172.21.15.49:0/2085664153 >> 172.21.15.49:0/2085664153 conn(0x5594de84fc00 msgr2=0x5594de87b000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:35.831 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.814+0000 7f6c6d2ff740 1 -- 172.21.15.49:0/2085664153 shutdown_connections 2023-11-06T02:14:35.831 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.814+0000 7f6c6d2ff740 1 -- 172.21.15.49:0/2085664153 wait complete. 2023-11-06T02:14:35.832 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', '--filter-tag', 'mytag123', 'cephfs_data'] in time 0:00:00.184358, result: 2023-11-06T02:14:35.832 DEBUG:teuthology.orchestra.run.smithi049:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_links 2023-11-06T02:14:35.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.902+0000 7fb68fd36740 1 -- start start 2023-11-06T02:14:35.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.902+0000 7fb68fd36740 1 Processor -- start 2023-11-06T02:14:35.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.902+0000 7fb68fd36740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55eaf2769000 0x55eaf27f4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:35.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.902+0000 7fb68fd36740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eaf2769400 0x55eaf27f4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:35.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.902+0000 7fb68fd36740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eaf2769800 0x55eaf27f5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:35.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.902+0000 7fb68fd36740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaf2836a00 con 0x55eaf2769400 2023-11-06T02:14:35.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.902+0000 7fb68fd36740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaf2836780 con 0x55eaf2769800 2023-11-06T02:14:35.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.902+0000 7fb68fd36740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eaf2836640 con 0x55eaf2769000 2023-11-06T02:14:35.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.902+0000 7fb68dcf5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eaf2769400 0x55eaf27f4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:35.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.902+0000 7fb68dcf5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eaf2769400 0x55eaf27f4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38258/0 (socket says 172.21.15.49:38258) 2023-11-06T02:14:35.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.902+0000 7fb68dcf5640 1 -- 172.21.15.49:0/4152712064 learned_addr learned my addr 172.21.15.49:0/4152712064 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:35.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.902+0000 7fb68e4f6640 1 --2- 172.21.15.49:0/4152712064 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55eaf2769000 0x55eaf27f4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:35.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.902+0000 7fb68ecf7640 1 --2- 172.21.15.49:0/4152712064 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eaf2769800 0x55eaf27f5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:35.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.902+0000 7fb68dcf5640 1 -- 172.21.15.49:0/4152712064 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55eaf2769000 msgr2=0x55eaf27f4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:35.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.902+0000 7fb68dcf5640 1 --2- 172.21.15.49:0/4152712064 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55eaf2769000 0x55eaf27f4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.902+0000 7fb68dcf5640 1 -- 172.21.15.49:0/4152712064 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eaf2769800 msgr2=0x55eaf27f5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:35.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.902+0000 7fb68dcf5640 1 --2- 172.21.15.49:0/4152712064 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eaf2769800 0x55eaf27f5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.902+0000 7fb68dcf5640 1 -- 172.21.15.49:0/4152712064 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eaf279cb60 con 0x55eaf2769400 2023-11-06T02:14:35.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.902+0000 7fb68ecf7640 1 --2- 172.21.15.49:0/4152712064 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eaf2769800 0x55eaf27f5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:14:35.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.902+0000 7fb68e4f6640 1 --2- 172.21.15.49:0/4152712064 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55eaf2769000 0x55eaf27f4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:14:35.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.902+0000 7fb68dcf5640 1 --2- 172.21.15.49:0/4152712064 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eaf2769400 0x55eaf27f4b00 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x55eaf35e8d80 tx=0x55eaf3540600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53ed02970ad76155 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:35.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.902+0000 7fb68bcf1640 1 -- 172.21.15.49:0/4152712064 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55eaf27cd600 con 0x55eaf2769400 2023-11-06T02:14:35.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.902+0000 7fb68bcf1640 1 -- 172.21.15.49:0/4152712064 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55eaf27c4180 con 0x55eaf2769400 2023-11-06T02:14:35.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.902+0000 7fb68bcf1640 1 -- 172.21.15.49:0/4152712064 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55eaf27cd1e0 con 0x55eaf2769400 2023-11-06T02:14:35.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.906+0000 7fb68fd36740 1 -- 172.21.15.49:0/4152712064 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eaf279c340 con 0x55eaf2769400 2023-11-06T02:14:35.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.906+0000 7fb68fd36740 1 -- 172.21.15.49:0/4152712064 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55eaf279c000 con 0x55eaf2769400 2023-11-06T02:14:35.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.906+0000 7fb68fd36740 4 waiting for MDS map... 2023-11-06T02:14:35.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.906+0000 7fb68bcf1640 1 -- 172.21.15.49:0/4152712064 <== mon.0 v2:172.21.15.49:3300/0 4 ==== osd_map(82..82 src has 1..82) v4 ==== 6234+0+0 (secure 0 0 0) 0x55eaf27a8c40 con 0x55eaf2769400 2023-11-06T02:14:35.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.906+0000 7fb68bcf1640 1 -- 172.21.15.49:0/4152712064 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 53) v1 ==== 2565+0+0 (secure 0 0 0) 0x55eaf3698000 con 0x55eaf2769400 2023-11-06T02:14:35.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.906+0000 7fb68fd36740 4 Got MDS map 53 2023-11-06T02:14:35.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.906+0000 7fb68fd36740 4 datascan.main: Using metadata pool output 2023-11-06T02:14:35.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.906+0000 7fb68fd36740 4 datascan.main: connecting to RADOS... 2023-11-06T02:14:35.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.906+0000 7fb68fd36740 1 Processor -- start 2023-11-06T02:14:35.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.906+0000 7fb68fd36740 1 -- start start 2023-11-06T02:14:35.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.906+0000 7fb68fd36740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eaf36ea400 0x55eaf27f5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:35.917 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.906+0000 7fb68fd36740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55eaf36ea800 0x55eaf27f5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:35.917 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.906+0000 7fb68fd36740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eaf36eac00 0x55eaf27f6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:35.917 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.906+0000 7fb68fd36740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaf27afe00 con 0x55eaf36eac00 2023-11-06T02:14:35.917 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.906+0000 7fb68fd36740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaf2836780 con 0x55eaf36ea400 2023-11-06T02:14:35.918 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.906+0000 7fb68fd36740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eaf2836500 con 0x55eaf36ea800 2023-11-06T02:14:35.918 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.906+0000 7fb68dcf5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eaf36eac00 0x55eaf27f6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:35.918 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.906+0000 7fb68dcf5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eaf36eac00 0x55eaf27f6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38262/0 (socket says 172.21.15.49:38262) 2023-11-06T02:14:35.918 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.906+0000 7fb68dcf5640 1 -- 172.21.15.49:0/3892709653 learned_addr learned my addr 172.21.15.49:0/3892709653 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:35.919 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.906+0000 7fb68ecf7640 1 --2- 172.21.15.49:0/3892709653 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eaf36ea400 0x55eaf27f5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:35.919 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.906+0000 7fb68e4f6640 1 --2- 172.21.15.49:0/3892709653 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55eaf36ea800 0x55eaf27f5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:35.919 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.906+0000 7fb68dcf5640 1 -- 172.21.15.49:0/3892709653 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55eaf36ea800 msgr2=0x55eaf27f5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:35.919 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.906+0000 7fb68dcf5640 1 --2- 172.21.15.49:0/3892709653 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55eaf36ea800 0x55eaf27f5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.919 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.906+0000 7fb68dcf5640 1 -- 172.21.15.49:0/3892709653 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eaf36ea400 msgr2=0x55eaf27f5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:35.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.906+0000 7fb68dcf5640 1 --2- 172.21.15.49:0/3892709653 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eaf36ea400 0x55eaf27f5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.906+0000 7fb68dcf5640 1 -- 172.21.15.49:0/3892709653 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eaf279cea0 con 0x55eaf36eac00 2023-11-06T02:14:35.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.906+0000 7fb68dcf5640 1 --2- 172.21.15.49:0/3892709653 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eaf36eac00 0x55eaf27f6100 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x55eaf2a480f0 tx=0x55eaf3540540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc165f1a71256efe server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:35.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.906+0000 7fb689ced640 1 -- 172.21.15.49:0/3892709653 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55eaf27ccf20 con 0x55eaf36eac00 2023-11-06T02:14:35.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.906+0000 7fb689ced640 1 -- 172.21.15.49:0/3892709653 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55eaf27c4480 con 0x55eaf36eac00 2023-11-06T02:14:35.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.906+0000 7fb689ced640 1 -- 172.21.15.49:0/3892709653 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55eaf27cd600 con 0x55eaf36eac00 2023-11-06T02:14:35.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.906+0000 7fb68fd36740 1 -- 172.21.15.49:0/3892709653 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eaf36eac00 msgr2=0x55eaf27f6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:35.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.906+0000 7fb68fd36740 1 --2- 172.21.15.49:0/3892709653 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eaf36eac00 0x55eaf27f6100 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x55eaf2a480f0 tx=0x55eaf3540540 comp rx=0 tx=0).stop 2023-11-06T02:14:35.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.906+0000 7fb68fd36740 1 -- 172.21.15.49:0/3892709653 shutdown_connections 2023-11-06T02:14:35.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.906+0000 7fb68fd36740 1 --2- 172.21.15.49:0/3892709653 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eaf36eac00 0x55eaf27f6100 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.906+0000 7fb68fd36740 1 --2- 172.21.15.49:0/3892709653 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55eaf36ea800 0x55eaf27f5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.906+0000 7fb68fd36740 1 --2- 172.21.15.49:0/3892709653 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eaf36ea400 0x55eaf27f5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.906+0000 7fb68fd36740 1 -- 172.21.15.49:0/3892709653 >> 172.21.15.49:0/3892709653 conn(0x55eaf36ea000 msgr2=0x55eaf2794000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:35.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.906+0000 7fb68fd36740 1 -- 172.21.15.49:0/3892709653 shutdown_connections 2023-11-06T02:14:35.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.906+0000 7fb68fd36740 1 -- 172.21.15.49:0/3892709653 wait complete. 2023-11-06T02:14:35.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.906+0000 7fb68fd36740 1 Processor -- start 2023-11-06T02:14:35.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.910+0000 7fb68fd36740 1 -- start start 2023-11-06T02:14:35.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.910+0000 7fb68fd36740 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55eaf36ea400 0x55eaf27f7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:35.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.910+0000 7fb68fd36740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eaf36ea800 0x55eaf27f6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:35.924 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.910+0000 7fb68fd36740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eaf36eac00 0x55eaf27f6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:35.924 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.910+0000 7fb68fd36740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaf28363c0 con 0x55eaf36eac00 2023-11-06T02:14:35.924 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.910+0000 7fb68fd36740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaf2836280 con 0x55eaf36ea800 2023-11-06T02:14:35.924 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.910+0000 7fb68fd36740 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eaf2836140 con 0x55eaf36ea400 2023-11-06T02:14:35.924 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.910+0000 7fb68dcf5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eaf36eac00 0x55eaf27f6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:35.925 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.910+0000 7fb68dcf5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eaf36eac00 0x55eaf27f6680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38266/0 (socket says 172.21.15.49:38266) 2023-11-06T02:14:35.925 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.910+0000 7fb68dcf5640 1 -- 172.21.15.49:0/119336422 learned_addr learned my addr 172.21.15.49:0/119336422 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:35.925 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.910+0000 7fb68e4f6640 1 --2- 172.21.15.49:0/119336422 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eaf36ea800 0x55eaf27f6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:35.925 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.910+0000 7fb68ecf7640 1 --2- 172.21.15.49:0/119336422 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55eaf36ea400 0x55eaf27f7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:35.926 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.910+0000 7fb68dcf5640 1 -- 172.21.15.49:0/119336422 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55eaf36ea400 msgr2=0x55eaf27f7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:35.926 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.910+0000 7fb68dcf5640 1 --2- 172.21.15.49:0/119336422 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55eaf36ea400 0x55eaf27f7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.926 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.910+0000 7fb68dcf5640 1 -- 172.21.15.49:0/119336422 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eaf36ea800 msgr2=0x55eaf27f6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:35.926 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.910+0000 7fb68dcf5640 1 --2- 172.21.15.49:0/119336422 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eaf36ea800 0x55eaf27f6c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:35.926 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.910+0000 7fb68dcf5640 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eaf279cb60 con 0x55eaf36eac00 2023-11-06T02:14:35.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.910+0000 7fb68dcf5640 1 --2- 172.21.15.49:0/119336422 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eaf36eac00 0x55eaf27f6680 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x55eaf2a493b0 tx=0x55eaf369ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:35.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.910+0000 7fb6884ea640 1 -- 172.21.15.49:0/119336422 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55eaf27cd1e0 con 0x55eaf36eac00 2023-11-06T02:14:35.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.910+0000 7fb6884ea640 1 -- 172.21.15.49:0/119336422 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55eaf27c4180 con 0x55eaf36eac00 2023-11-06T02:14:35.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.910+0000 7fb68fd36740 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55eaf279c340 con 0x55eaf36eac00 2023-11-06T02:14:35.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.910+0000 7fb6884ea640 1 -- 172.21.15.49:0/119336422 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55eaf27ccdc0 con 0x55eaf36eac00 2023-11-06T02:14:35.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.910+0000 7fb68fd36740 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eaf279d520 con 0x55eaf36eac00 2023-11-06T02:14:35.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.910+0000 7fb68fd36740 4 datascan.init: resolving metadata pool 11 2023-11-06T02:14:35.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.910+0000 7fb6884ea640 1 -- 172.21.15.49:0/119336422 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x55eaf2833340 con 0x55eaf36eac00 2023-11-06T02:14:35.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.910+0000 7fb6884ea640 1 --2- 172.21.15.49:0/119336422 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x55eaf36ebc00 0x55eaf27f7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:35.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.910+0000 7fb6884ea640 1 -- 172.21.15.49:0/119336422 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6234+0+0 (secure 0 0 0) 0x55eaf27a8a80 con 0x55eaf36eac00 2023-11-06T02:14:35.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.910+0000 7fb68fd36740 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-11-06T02:14:35.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.910+0000 7fb68fd36740 4 datascan.main: resolving metadata pool 11 2023-11-06T02:14:35.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.910+0000 7fb68fd36740 1 --2- 172.21.15.49:0/119336422 >> [v2:172.21.15.49:6808/38113,v1:172.21.15.49:6810/38113] conn(0x55eaf2b44000 0x55eaf27f6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:35.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.910+0000 7fb68fd36740 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.49:6808/38113,v1:172.21.15.49:6810/38113] -- osd_op(unknown.0.0:1 11.0 11:00000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b44400 con 0x55eaf2b44000 2023-11-06T02:14:35.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.910+0000 7fb68ecf7640 1 --2- 172.21.15.49:0/119336422 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x55eaf36ebc00 0x55eaf27f7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:35.930 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.910+0000 7fb68e4f6640 1 --2- 172.21.15.49:0/119336422 >> [v2:172.21.15.49:6808/38113,v1:172.21.15.49:6810/38113] conn(0x55eaf2b44000 0x55eaf27f6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:35.930 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.914+0000 7fb68e4f6640 1 --2- 172.21.15.49:0/119336422 >> [v2:172.21.15.49:6808/38113,v1:172.21.15.49:6810/38113] conn(0x55eaf2b44000 0x55eaf27f6100 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:35.930 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.914+0000 7fb68ecf7640 1 --2- 172.21.15.49:0/119336422 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x55eaf36ebc00 0x55eaf27f7700 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x55eaf34c80f0 tx=0x55eaf369ecc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:35.930 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.914+0000 7fb68e4f6640 1 -- 172.21.15.49:0/119336422 <== osd.1 v2:172.21.15.49:6808/38113 1 ==== osd_op_reply(1 [pgnls start_epoch 82 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eaf3672fc0 con 0x55eaf2b44000 2023-11-06T02:14:35.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.914+0000 7fb68e4f6640 1 --2- 172.21.15.49:0/119336422 >> [v2:172.21.15.49:6809/38116,v1:172.21.15.49:6811/38116] conn(0x55eaf2b44400 0x55eaf3680000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:35.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.914+0000 7fb68e4f6640 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.49:6809/38116,v1:172.21.15.49:6811/38116] -- osd_op(unknown.0.0:2 11.10 11:08000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf3551000 con 0x55eaf2b44400 2023-11-06T02:14:35.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.914+0000 7fb68ecf7640 1 --2- 172.21.15.49:0/119336422 >> [v2:172.21.15.49:6809/38116,v1:172.21.15.49:6811/38116] conn(0x55eaf2b44400 0x55eaf3680000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:35.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.914+0000 7fb68ecf7640 1 --2- 172.21.15.49:0/119336422 >> [v2:172.21.15.49:6809/38116,v1:172.21.15.49:6811/38116] conn(0x55eaf2b44400 0x55eaf3680000 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:35.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.914+0000 7fb68ecf7640 1 -- 172.21.15.49:0/119336422 <== osd.0 v2:172.21.15.49:6809/38116 1 ==== osd_op_reply(2 [pgnls start_epoch 82 out=177b] v81'16 uv16 ondisk = 1) v8 ==== 144+0+177 (crc 0 0 0) 0x55eaf3673200 con 0x55eaf2b44400 2023-11-06T02:14:35.932 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.914+0000 7fb68ecf7640 1 --2- 172.21.15.49:0/119336422 >> [v2:172.21.15.49:6824/38115,v1:172.21.15.49:6825/38115] conn(0x55eaf3551000 0x55eaf3680580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:35.932 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.914+0000 7fb68ecf7640 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.49:6824/38115,v1:172.21.15.49:6825/38115] -- osd_op(unknown.0.0:3 11.8 11:10000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b45c00 con 0x55eaf3551000 2023-11-06T02:14:35.932 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.914+0000 7fb68e4f6640 1 --2- 172.21.15.49:0/119336422 >> [v2:172.21.15.49:6824/38115,v1:172.21.15.49:6825/38115] conn(0x55eaf3551000 0x55eaf3680580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:35.932 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.914+0000 7fb68e4f6640 1 --2- 172.21.15.49:0/119336422 >> [v2:172.21.15.49:6824/38115,v1:172.21.15.49:6825/38115] conn(0x55eaf3551000 0x55eaf3680580 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:35.932 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.914+0000 7fb68e4f6640 1 -- 172.21.15.49:0/119336422 <== osd.3 v2:172.21.15.49:6824/38115 1 ==== osd_op_reply(3 [pgnls start_epoch 82 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eaf3672fc0 con 0x55eaf3551000 2023-11-06T02:14:35.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.914+0000 7fb68e4f6640 1 --2- 172.21.15.49:0/119336422 >> [v2:172.21.15.49:6800/38114,v1:172.21.15.49:6801/38114] conn(0x55eaf2b45c00 0x55eaf3680b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:35.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.914+0000 7fb68e4f6640 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.49:6800/38114,v1:172.21.15.49:6801/38114] -- osd_op(unknown.0.0:4 11.18 11:18000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf3550c00 con 0x55eaf2b45c00 2023-11-06T02:14:35.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.914+0000 7fb68dcf5640 1 --2- 172.21.15.49:0/119336422 >> [v2:172.21.15.49:6800/38114,v1:172.21.15.49:6801/38114] conn(0x55eaf2b45c00 0x55eaf3680b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:35.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.914+0000 7fb68dcf5640 1 --2- 172.21.15.49:0/119336422 >> [v2:172.21.15.49:6800/38114,v1:172.21.15.49:6801/38114] conn(0x55eaf2b45c00 0x55eaf3680b00 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:35.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.914+0000 7fb68dcf5640 1 -- 172.21.15.49:0/119336422 <== osd.2 v2:172.21.15.49:6800/38114 1 ==== osd_op_reply(4 [pgnls start_epoch 82 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eaf3673440 con 0x55eaf2b45c00 2023-11-06T02:14:35.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.914+0000 7fb68dcf5640 1 --2- 172.21.15.49:0/119336422 >> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] conn(0x55eaf3550c00 0x55eaf3681600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:35.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.914+0000 7fb68dcf5640 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] -- osd_op(unknown.0.0:5 11.4 11:20000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b5a000 con 0x55eaf3550c00 2023-11-06T02:14:35.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.918+0000 7fb68ecf7640 1 --2- 172.21.15.49:0/119336422 >> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] conn(0x55eaf3550c00 0x55eaf3681600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:35.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.918+0000 7fb68ecf7640 1 --2- 172.21.15.49:0/119336422 >> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] conn(0x55eaf3550c00 0x55eaf3681600 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:35.935 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.918+0000 7fb68ecf7640 1 -- 172.21.15.49:0/119336422 <== osd.7 v2:172.21.15.150:6816/38421 1 ==== osd_op_reply(5 [pgnls start_epoch 82 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eaf3673200 con 0x55eaf3550c00 2023-11-06T02:14:35.935 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.918+0000 7fb68ecf7640 1 --2- 172.21.15.49:0/119336422 >> [v2:172.21.15.150:6824/38428,v1:172.21.15.150:6825/38428] conn(0x55eaf2b5a000 0x55eaf3681b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:35.935 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.918+0000 7fb68ecf7640 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.150:6824/38428,v1:172.21.15.150:6825/38428] -- osd_op(unknown.0.0:6 11.14 11:28000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b5b000 con 0x55eaf2b5a000 2023-11-06T02:14:35.936 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.918+0000 7fb68e4f6640 1 --2- 172.21.15.49:0/119336422 >> [v2:172.21.15.150:6824/38428,v1:172.21.15.150:6825/38428] conn(0x55eaf2b5a000 0x55eaf3681b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:35.936 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.918+0000 7fb68e4f6640 1 --2- 172.21.15.49:0/119336422 >> [v2:172.21.15.150:6824/38428,v1:172.21.15.150:6825/38428] conn(0x55eaf2b5a000 0x55eaf3681b80 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:35.936 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.918+0000 7fb68e4f6640 1 -- 172.21.15.49:0/119336422 <== osd.6 v2:172.21.15.150:6824/38428 1 ==== osd_op_reply(6 [pgnls start_epoch 82 out=149b] v82'44 uv44 ondisk = 1) v8 ==== 144+0+149 (crc 0 0 0) 0x55eaf3672fc0 con 0x55eaf2b5a000 2023-11-06T02:14:35.936 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.918+0000 7fb68e4f6640 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.49:6808/38113,v1:172.21.15.49:6810/38113] -- osd_op(unknown.0.0:7 11.c 11:30000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b5b000 con 0x55eaf2b44000 2023-11-06T02:14:35.936 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.918+0000 7fb68e4f6640 1 -- 172.21.15.49:0/119336422 <== osd.1 v2:172.21.15.49:6808/38113 2 ==== osd_op_reply(7 [pgnls start_epoch 82 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eaf3672fc0 con 0x55eaf2b44000 2023-11-06T02:14:35.937 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.922+0000 7fb68e4f6640 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.150:6824/38428,v1:172.21.15.150:6825/38428] -- osd_op(unknown.0.0:8 11.1c 11:38000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b5b000 con 0x55eaf2b5a000 2023-11-06T02:14:35.937 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.922+0000 7fb68e4f6640 1 -- 172.21.15.49:0/119336422 <== osd.6 v2:172.21.15.150:6824/38428 2 ==== osd_op_reply(8 [pgnls start_epoch 82 out=73b] v81'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55eaf3672fc0 con 0x55eaf2b5a000 2023-11-06T02:14:35.937 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.922+0000 7fb68e4f6640 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.49:6800/38114,v1:172.21.15.49:6801/38114] -- osd_op(unknown.0.0:9 11.2 11:40000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b5b000 con 0x55eaf2b45c00 2023-11-06T02:14:35.937 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.922+0000 7fb68dcf5640 1 -- 172.21.15.49:0/119336422 <== osd.2 v2:172.21.15.49:6800/38114 2 ==== osd_op_reply(9 [pgnls start_epoch 82 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eaf3673440 con 0x55eaf2b45c00 2023-11-06T02:14:35.938 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.922+0000 7fb68dcf5640 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] -- osd_op(unknown.0.0:10 11.12 11:48000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b5b000 con 0x55eaf3550c00 2023-11-06T02:14:35.952 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.922+0000 7fb68ecf7640 1 -- 172.21.15.49:0/119336422 <== osd.7 v2:172.21.15.150:6816/38421 2 ==== osd_op_reply(10 [pgnls start_epoch 82 out=81b] v81'3 uv3 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55eaf3673200 con 0x55eaf3550c00 2023-11-06T02:14:35.952 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.922+0000 7fb68ecf7640 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.49:6824/38115,v1:172.21.15.49:6825/38115] -- osd_op(unknown.0.0:11 11.a 11:50000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b5b000 con 0x55eaf3551000 2023-11-06T02:14:35.952 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.922+0000 7fb68e4f6640 1 -- 172.21.15.49:0/119336422 <== osd.3 v2:172.21.15.49:6824/38115 2 ==== osd_op_reply(11 [pgnls start_epoch 82 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eaf3672fc0 con 0x55eaf3551000 2023-11-06T02:14:35.953 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.922+0000 7fb68e4f6640 1 --2- 172.21.15.49:0/119336422 >> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] conn(0x55eaf2b5b000 0x55eaf3681080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:35.953 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.922+0000 7fb68e4f6640 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] -- osd_op(unknown.0.0:12 11.1a 11:58000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf3550400 con 0x55eaf2b5b000 2023-11-06T02:14:35.953 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.922+0000 7fb68dcf5640 1 --2- 172.21.15.49:0/119336422 >> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] conn(0x55eaf2b5b000 0x55eaf3681080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:35.953 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.922+0000 7fb68dcf5640 1 --2- 172.21.15.49:0/119336422 >> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] conn(0x55eaf2b5b000 0x55eaf3681080 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:35.954 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.926+0000 7fb68dcf5640 1 -- 172.21.15.49:0/119336422 <== osd.5 v2:172.21.15.150:6800/38418 1 ==== osd_op_reply(12 [pgnls start_epoch 82 out=49b] v82'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eaf3673440 con 0x55eaf2b5b000 2023-11-06T02:14:35.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.926+0000 7fb68dcf5640 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.49:6809/38116,v1:172.21.15.49:6811/38116] -- osd_op(unknown.0.0:13 11.6 11:60000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf3550400 con 0x55eaf2b44400 2023-11-06T02:14:35.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.926+0000 7fb68ecf7640 1 -- 172.21.15.49:0/119336422 <== osd.0 v2:172.21.15.49:6809/38116 2 ==== osd_op_reply(13 [pgnls start_epoch 82 out=77b] v81'11 uv11 ondisk = 1) v8 ==== 144+0+77 (crc 0 0 0) 0x55eaf3673200 con 0x55eaf2b44400 2023-11-06T02:14:35.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.926+0000 7fb68ecf7640 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] -- osd_op(unknown.0.0:14 11.16 11:68000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf3550400 con 0x55eaf2b5b000 2023-11-06T02:14:35.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.926+0000 7fb68dcf5640 1 -- 172.21.15.49:0/119336422 <== osd.5 v2:172.21.15.150:6800/38418 2 ==== osd_op_reply(14 [pgnls start_epoch 82 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eaf3673440 con 0x55eaf2b5b000 2023-11-06T02:14:35.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.926+0000 7fb68dcf5640 1 --2- 172.21.15.49:0/119336422 >> [v2:172.21.15.150:6808/38417,v1:172.21.15.150:6809/38417] conn(0x55eaf3550400 0x55eaf3682680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:35.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.926+0000 7fb68dcf5640 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.150:6808/38417,v1:172.21.15.150:6809/38417] -- osd_op(unknown.0.0:15 11.e 11:70000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b5a400 con 0x55eaf3550400 2023-11-06T02:14:35.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.926+0000 7fb68ecf7640 1 --2- 172.21.15.49:0/119336422 >> [v2:172.21.15.150:6808/38417,v1:172.21.15.150:6809/38417] conn(0x55eaf3550400 0x55eaf3682680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:35.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.926+0000 7fb68ecf7640 1 --2- 172.21.15.49:0/119336422 >> [v2:172.21.15.150:6808/38417,v1:172.21.15.150:6809/38417] conn(0x55eaf3550400 0x55eaf3682680 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:35.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.926+0000 7fb68ecf7640 1 -- 172.21.15.49:0/119336422 <== osd.4 v2:172.21.15.150:6808/38417 1 ==== osd_op_reply(15 [pgnls start_epoch 82 out=73b] v81'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55eaf3673200 con 0x55eaf3550400 2023-11-06T02:14:35.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.926+0000 7fb68ecf7640 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] -- osd_op(unknown.0.0:16 11.1e 11:78000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b5a400 con 0x55eaf3550c00 2023-11-06T02:14:35.957 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.930+0000 7fb68ecf7640 1 -- 172.21.15.49:0/119336422 <== osd.7 v2:172.21.15.150:6816/38421 3 ==== osd_op_reply(16 [pgnls start_epoch 82 out=49b] v82'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eaf3673200 con 0x55eaf3550c00 2023-11-06T02:14:35.957 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.930+0000 7fb68ecf7640 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.150:6824/38428,v1:172.21.15.150:6825/38428] -- osd_op(unknown.0.0:17 11.1 11:80000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b5a400 con 0x55eaf2b5a000 2023-11-06T02:14:35.957 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.930+0000 7fb68e4f6640 1 -- 172.21.15.49:0/119336422 <== osd.6 v2:172.21.15.150:6824/38428 3 ==== osd_op_reply(17 [pgnls start_epoch 82 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eaf3672fc0 con 0x55eaf2b5a000 2023-11-06T02:14:35.957 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.930+0000 7fb68e4f6640 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.49:6800/38114,v1:172.21.15.49:6801/38114] -- osd_op(unknown.0.0:18 11.11 11:88000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b5a400 con 0x55eaf2b45c00 2023-11-06T02:14:35.957 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.930+0000 7fb68dcf5640 1 -- 172.21.15.49:0/119336422 <== osd.2 v2:172.21.15.49:6800/38114 3 ==== osd_op_reply(18 [pgnls start_epoch 82 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eaf3673440 con 0x55eaf2b45c00 2023-11-06T02:14:35.957 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.930+0000 7fb68dcf5640 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.49:6824/38115,v1:172.21.15.49:6825/38115] -- osd_op(unknown.0.0:19 11.9 11:90000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b5a400 con 0x55eaf3551000 2023-11-06T02:14:35.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.930+0000 7fb68e4f6640 1 -- 172.21.15.49:0/119336422 <== osd.3 v2:172.21.15.49:6824/38115 3 ==== osd_op_reply(19 [pgnls start_epoch 82 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eaf3672fc0 con 0x55eaf3551000 2023-11-06T02:14:35.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.930+0000 7fb68e4f6640 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.49:6809/38116,v1:172.21.15.49:6811/38116] -- osd_op(unknown.0.0:20 11.19 11:98000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b5a400 con 0x55eaf2b44400 2023-11-06T02:14:35.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.930+0000 7fb68ecf7640 1 -- 172.21.15.49:0/119336422 <== osd.0 v2:172.21.15.49:6809/38116 3 ==== osd_op_reply(20 [pgnls start_epoch 82 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eaf3673200 con 0x55eaf2b44400 2023-11-06T02:14:35.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.930+0000 7fb68ecf7640 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] -- osd_op(unknown.0.0:21 11.5 11:a0000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b5a400 con 0x55eaf3550c00 2023-11-06T02:14:35.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.930+0000 7fb68ecf7640 1 -- 172.21.15.49:0/119336422 <== osd.7 v2:172.21.15.150:6816/38421 4 ==== osd_op_reply(21 [pgnls start_epoch 82 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eaf3673200 con 0x55eaf3550c00 2023-11-06T02:14:35.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.930+0000 7fb68ecf7640 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.49:6800/38114,v1:172.21.15.49:6801/38114] -- osd_op(unknown.0.0:22 11.15 11:a8000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b5a400 con 0x55eaf2b45c00 2023-11-06T02:14:35.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.934+0000 7fb68dcf5640 1 -- 172.21.15.49:0/119336422 <== osd.2 v2:172.21.15.49:6800/38114 4 ==== osd_op_reply(22 [pgnls start_epoch 82 out=97b] v81'4 uv4 ondisk = 1) v8 ==== 144+0+97 (crc 0 0 0) 0x55eaf3673440 con 0x55eaf2b45c00 2023-11-06T02:14:35.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.934+0000 7fb68dcf5640 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.49:6800/38114,v1:172.21.15.49:6801/38114] -- osd_op(unknown.0.0:23 11.d 11:b0000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b5a400 con 0x55eaf2b45c00 2023-11-06T02:14:35.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.934+0000 7fb68dcf5640 1 -- 172.21.15.49:0/119336422 <== osd.2 v2:172.21.15.49:6800/38114 5 ==== osd_op_reply(23 [pgnls start_epoch 82 out=74b] v79'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x55eaf3673440 con 0x55eaf2b45c00 2023-11-06T02:14:35.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.934+0000 7fb68dcf5640 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] -- osd_op(unknown.0.0:24 11.1d 11:b8000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b5a400 con 0x55eaf3550c00 2023-11-06T02:14:35.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.934+0000 7fb68ecf7640 1 -- 172.21.15.49:0/119336422 <== osd.7 v2:172.21.15.150:6816/38421 5 ==== osd_op_reply(24 [pgnls start_epoch 82 out=73b] v81'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55eaf3673200 con 0x55eaf3550c00 2023-11-06T02:14:35.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.934+0000 7fb68ecf7640 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.150:6824/38428,v1:172.21.15.150:6825/38428] -- osd_op(unknown.0.0:25 11.3 11:c0000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b5a400 con 0x55eaf2b5a000 2023-11-06T02:14:35.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.934+0000 7fb68e4f6640 1 -- 172.21.15.49:0/119336422 <== osd.6 v2:172.21.15.150:6824/38428 4 ==== osd_op_reply(25 [pgnls start_epoch 82 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eaf3672fc0 con 0x55eaf2b5a000 2023-11-06T02:14:35.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.934+0000 7fb68e4f6640 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] -- osd_op(unknown.0.0:26 11.13 11:c8000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b5a400 con 0x55eaf2b5b000 2023-11-06T02:14:35.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.934+0000 7fb68dcf5640 1 -- 172.21.15.49:0/119336422 <== osd.5 v2:172.21.15.150:6800/38418 3 ==== osd_op_reply(26 [pgnls start_epoch 82 out=98b] v81'5 uv5 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x55eaf3673440 con 0x55eaf2b5b000 2023-11-06T02:14:35.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.934+0000 7fb68dcf5640 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.49:6824/38115,v1:172.21.15.49:6825/38115] -- osd_op(unknown.0.0:27 11.b 11:d0000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b5a400 con 0x55eaf3551000 2023-11-06T02:14:35.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.934+0000 7fb68e4f6640 1 -- 172.21.15.49:0/119336422 <== osd.3 v2:172.21.15.49:6824/38115 4 ==== osd_op_reply(27 [pgnls start_epoch 82 out=76b] v82'6 uv5 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x55eaf3672fc0 con 0x55eaf3551000 2023-11-06T02:14:35.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.934+0000 7fb68e4f6640 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.150:6808/38417,v1:172.21.15.150:6809/38417] -- osd_op(unknown.0.0:28 11.1b 11:d8000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b5a400 con 0x55eaf3550400 2023-11-06T02:14:35.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.938+0000 7fb68ecf7640 1 -- 172.21.15.49:0/119336422 <== osd.4 v2:172.21.15.150:6808/38417 2 ==== osd_op_reply(28 [pgnls start_epoch 82 out=49b] v82'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eaf3673200 con 0x55eaf3550400 2023-11-06T02:14:35.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.938+0000 7fb68ecf7640 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] -- osd_op(unknown.0.0:29 11.7 11:e0000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b5a400 con 0x55eaf2b5b000 2023-11-06T02:14:35.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.938+0000 7fb68dcf5640 1 -- 172.21.15.49:0/119336422 <== osd.5 v2:172.21.15.150:6800/38418 4 ==== osd_op_reply(29 [pgnls start_epoch 82 out=105b] v82'9 uv9 ondisk = 1) v8 ==== 144+0+105 (crc 0 0 0) 0x55eaf3673440 con 0x55eaf2b5b000 2023-11-06T02:14:35.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.938+0000 7fb68dcf5640 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] -- osd_op(unknown.0.0:30 11.17 11:e8000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b5a400 con 0x55eaf3550c00 2023-11-06T02:14:35.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.938+0000 7fb68ecf7640 1 -- 172.21.15.49:0/119336422 <== osd.7 v2:172.21.15.150:6816/38421 6 ==== osd_op_reply(30 [pgnls start_epoch 82 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eaf3673200 con 0x55eaf3550c00 2023-11-06T02:14:35.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.938+0000 7fb68ecf7640 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] -- osd_op(unknown.0.0:31 11.f 11:f0000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b5a400 con 0x55eaf3550c00 2023-11-06T02:14:35.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.938+0000 7fb68ecf7640 1 -- 172.21.15.49:0/119336422 <== osd.7 v2:172.21.15.150:6816/38421 7 ==== osd_op_reply(31 [pgnls start_epoch 82 out=73b] v82'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55eaf3673200 con 0x55eaf3550c00 2023-11-06T02:14:35.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.938+0000 7fb68ecf7640 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.49:6800/38114,v1:172.21.15.49:6801/38114] -- osd_op(unknown.0.0:32 11.1f 11:f8000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b5a400 con 0x55eaf2b45c00 2023-11-06T02:14:35.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.938+0000 7fb68dcf5640 1 -- 172.21.15.49:0/119336422 <== osd.2 v2:172.21.15.49:6800/38114 6 ==== osd_op_reply(32 [pgnls start_epoch 82 out=101b] v82'9 uv8 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x55eaf3673440 con 0x55eaf2b45c00 2023-11-06T02:14:35.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.942+0000 7fb68fd36740 10 datascan.scan_links: step 1: handling object 601.00000000 2023-11-06T02:14:35.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.942+0000 7fb68fd36740 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.49:6809/38116,v1:172.21.15.49:6811/38116] -- osd_op(unknown.0.0:33 11.10 11:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b44800 con 0x55eaf2b44400 2023-11-06T02:14:35.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.942+0000 7fb68ecf7640 1 -- 172.21.15.49:0/119336422 <== osd.0 v2:172.21.15.49:6809/38116 4 ==== osd_op_reply(33 601.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55eaf3673200 con 0x55eaf2b44400 2023-11-06T02:14:35.964 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.942+0000 7fb68fd36740 10 datascan.scan_links: step 1: handling object 602.00000000 2023-11-06T02:14:35.964 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.942+0000 7fb68fd36740 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.49:6809/38116,v1:172.21.15.49:6811/38116] -- osd_op(unknown.0.0:34 11.10 11:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b44c00 con 0x55eaf2b44400 2023-11-06T02:14:35.964 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.942+0000 7fb68ecf7640 1 -- 172.21.15.49:0/119336422 <== osd.0 v2:172.21.15.49:6809/38116 5 ==== osd_op_reply(34 602.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55eaf3673200 con 0x55eaf2b44400 2023-11-06T02:14:35.964 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.942+0000 7fb68fd36740 10 datascan.scan_links: step 1: handling object 10000000000.00000000 2023-11-06T02:14:35.964 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.942+0000 7fb68fd36740 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.49:6809/38116,v1:172.21.15.49:6811/38116] -- osd_op(unknown.0.0:35 11.10 11:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b45000 con 0x55eaf2b44400 2023-11-06T02:14:35.964 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.942+0000 7fb68ecf7640 1 -- 172.21.15.49:0/119336422 <== osd.0 v2:172.21.15.49:6809/38116 6 ==== osd_op_reply(35 10000000000.00000000 [omap-get-vals out=1047b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+1047 (crc 0 0 0) 0x55eaf3673200 con 0x55eaf2b44400 2023-11-06T02:14:35.965 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.942+0000 7fb68fd36740 10 datascan.scan_links: step 1: handling object 600.00000000 2023-11-06T02:14:35.965 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.942+0000 7fb68fd36740 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.49:6809/38116,v1:172.21.15.49:6811/38116] -- osd_op(unknown.0.0:36 11.10 11:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b5b800 con 0x55eaf2b44400 2023-11-06T02:14:35.965 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.942+0000 7fb68ecf7640 1 -- 172.21.15.49:0/119336422 <== osd.0 v2:172.21.15.49:6809/38116 7 ==== osd_op_reply(36 600.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55eaf3673200 con 0x55eaf2b44400 2023-11-06T02:14:35.965 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.942+0000 7fb68fd36740 10 datascan.scan_links: step 1: handling object 603.00000000 2023-11-06T02:14:35.965 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.942+0000 7fb68fd36740 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.49:6809/38116,v1:172.21.15.49:6811/38116] -- osd_op(unknown.0.0:37 11.10 11:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b5b400 con 0x55eaf2b44400 2023-11-06T02:14:35.966 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.942+0000 7fb68ecf7640 1 -- 172.21.15.49:0/119336422 <== osd.0 v2:172.21.15.49:6809/38116 8 ==== osd_op_reply(37 603.00000000 [omap-get-vals out=5b] v0'0 uv16 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55eaf3673200 con 0x55eaf2b44400 2023-11-06T02:14:35.966 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.942+0000 7fb68fd36740 10 datascan.scan_links: step 1: handling object 1.00000000.inode 2023-11-06T02:14:35.966 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.942+0000 7fb68fd36740 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2023-11-06T02:14:35.966 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.942+0000 7fb68fd36740 10 datascan.scan_links: step 1: handling object 200.00000000 2023-11-06T02:14:35.966 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.942+0000 7fb68fd36740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2023-11-06T02:14:35.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.942+0000 7fb68fd36740 10 datascan.scan_links: step 1: handling object 200.00000001 2023-11-06T02:14:35.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.942+0000 7fb68fd36740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2023-11-06T02:14:35.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.942+0000 7fb68fd36740 10 datascan.scan_links: step 1: handling object 606.00000000 2023-11-06T02:14:35.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.942+0000 7fb68fd36740 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.150:6824/38428,v1:172.21.15.150:6825/38428] -- osd_op(unknown.0.0:38 11.14 11:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b5bc00 con 0x55eaf2b5a000 2023-11-06T02:14:35.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.942+0000 7fb68e4f6640 1 -- 172.21.15.49:0/119336422 <== osd.6 v2:172.21.15.150:6824/38428 5 ==== osd_op_reply(38 606.00000000 [omap-get-vals out=5b] v0'0 uv39 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55eaf3672fc0 con 0x55eaf2b5a000 2023-11-06T02:14:35.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.942+0000 7fb68fd36740 10 datascan.scan_links: step 1: handling object 607.00000000 2023-11-06T02:14:35.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.942+0000 7fb68fd36740 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.150:6824/38428,v1:172.21.15.150:6825/38428] -- osd_op(unknown.0.0:39 11.1c 11:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b44800 con 0x55eaf2b5a000 2023-11-06T02:14:35.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.942+0000 7fb68e4f6640 1 -- 172.21.15.49:0/119336422 <== osd.6 v2:172.21.15.150:6824/38428 6 ==== osd_op_reply(39 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55eaf3672fc0 con 0x55eaf2b5a000 2023-11-06T02:14:35.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.942+0000 7fb68fd36740 10 datascan.scan_links: step 1: handling object 10000000003.00000000 2023-11-06T02:14:35.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.942+0000 7fb68fd36740 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] -- osd_op(unknown.0.0:40 11.12 11:4def72d4:::10000000003.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b44c00 con 0x55eaf3550c00 2023-11-06T02:14:35.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.946+0000 7fb68ecf7640 1 -- 172.21.15.49:0/119336422 <== osd.7 v2:172.21.15.150:6816/38421 8 ==== osd_op_reply(40 10000000003.00000000 [omap-get-vals out=523b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+523 (crc 0 0 0) 0x55eaf3673200 con 0x55eaf3550c00 2023-11-06T02:14:35.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.946+0000 7fb68fd36740 10 datascan.scan_links: step 1: handling object mds0_openfiles.0 2023-11-06T02:14:35.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.946+0000 7fb68fd36740 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2023-11-06T02:14:35.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.946+0000 7fb68fd36740 10 datascan.scan_links: step 1: handling object 608.00000000 2023-11-06T02:14:35.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.946+0000 7fb68fd36740 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.150:6808/38417,v1:172.21.15.150:6809/38417] -- osd_op(unknown.0.0:41 11.e 11:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b45000 con 0x55eaf3550400 2023-11-06T02:14:35.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.946+0000 7fb68ecf7640 1 -- 172.21.15.49:0/119336422 <== osd.4 v2:172.21.15.150:6808/38417 3 ==== osd_op_reply(41 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55eaf3673200 con 0x55eaf3550400 2023-11-06T02:14:35.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.946+0000 7fb68fd36740 10 datascan.scan_links: step 1: handling object 604.00000000 2023-11-06T02:14:35.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.946+0000 7fb68fd36740 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.49:6800/38114,v1:172.21.15.49:6801/38114] -- osd_op(unknown.0.0:42 11.15 11:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b76000 con 0x55eaf2b45c00 2023-11-06T02:14:35.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.946+0000 7fb68dcf5640 1 -- 172.21.15.49:0/119336422 <== osd.2 v2:172.21.15.49:6800/38114 7 ==== osd_op_reply(42 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55eaf3673440 con 0x55eaf2b45c00 2023-11-06T02:14:35.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.946+0000 7fb68fd36740 10 datascan.scan_links: step 1: handling object 500.00000000 2023-11-06T02:14:35.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.946+0000 7fb68fd36740 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2023-11-06T02:14:35.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.946+0000 7fb68fd36740 10 datascan.scan_links: step 1: handling object mds_snaptable 2023-11-06T02:14:35.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.946+0000 7fb68fd36740 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2023-11-06T02:14:35.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.946+0000 7fb68fd36740 10 datascan.scan_links: step 1: handling object 605.00000000 2023-11-06T02:14:35.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.946+0000 7fb68fd36740 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] -- osd_op(unknown.0.0:43 11.1d 11:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b76400 con 0x55eaf3550c00 2023-11-06T02:14:35.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.946+0000 7fb68ecf7640 1 -- 172.21.15.49:0/119336422 <== osd.7 v2:172.21.15.150:6816/38421 9 ==== osd_op_reply(43 605.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55eaf3673200 con 0x55eaf3550c00 2023-11-06T02:14:35.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.946+0000 7fb68fd36740 10 datascan.scan_links: step 1: handling object mds0_inotable 2023-11-06T02:14:35.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.946+0000 7fb68fd36740 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2023-11-06T02:14:35.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.946+0000 7fb68fd36740 10 datascan.scan_links: step 1: handling object 100.00000000 2023-11-06T02:14:35.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.946+0000 7fb68fd36740 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] -- osd_op(unknown.0.0:44 11.13 11:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b76800 con 0x55eaf2b5b000 2023-11-06T02:14:35.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.946+0000 7fb68dcf5640 1 -- 172.21.15.49:0/119336422 <== osd.5 v2:172.21.15.150:6800/38418 5 ==== osd_op_reply(44 100.00000000 [omap-get-vals out=5195b] v0'0 uv5 ondisk = 0) v8 ==== 156+0+5195 (crc 0 0 0) 0x55eaf3673440 con 0x55eaf2b5b000 2023-11-06T02:14:35.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.946+0000 7fb68fd36740 10 datascan.scan_links: step 1: handling object mds0_sessionmap 2023-11-06T02:14:35.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.946+0000 7fb68fd36740 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2023-11-06T02:14:35.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.946+0000 7fb68fd36740 10 datascan.scan_links: step 1: handling object 609.00000000 2023-11-06T02:14:35.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.946+0000 7fb68fd36740 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] -- osd_op(unknown.0.0:45 11.7 11:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b76c00 con 0x55eaf2b5b000 2023-11-06T02:14:35.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.950+0000 7fb68dcf5640 1 -- 172.21.15.49:0/119336422 <== osd.5 v2:172.21.15.150:6800/38418 6 ==== osd_op_reply(45 609.00000000 [omap-get-vals out=5b] v0'0 uv7 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55eaf3673440 con 0x55eaf2b5b000 2023-11-06T02:14:35.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.950+0000 7fb68fd36740 10 datascan.scan_links: step 1: handling object 10000000001.00000000 2023-11-06T02:14:35.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.950+0000 7fb68fd36740 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] -- osd_op(unknown.0.0:46 11.7 11:e2e07930:::10000000001.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b77000 con 0x55eaf2b5b000 2023-11-06T02:14:35.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.950+0000 7fb68dcf5640 1 -- 172.21.15.49:0/119336422 <== osd.5 v2:172.21.15.150:6800/38418 7 ==== osd_op_reply(46 10000000001.00000000 [omap-get-vals out=1545b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+1545 (crc 0 0 0) 0x55eaf3673440 con 0x55eaf2b5b000 2023-11-06T02:14:35.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.950+0000 7fb68fd36740 20 datascan.scan_links: injected ino detected 2023-11-06T02:14:35.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.950+0000 7fb68fd36740 10 datascan.scan_links: step 1: handling object 400.00000000 2023-11-06T02:14:35.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.950+0000 7fb68fd36740 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2023-11-06T02:14:35.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.950+0000 7fb68fd36740 10 datascan.scan_links: step 1: handling object 100.00000000.inode 2023-11-06T02:14:35.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.950+0000 7fb68fd36740 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2023-11-06T02:14:35.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.950+0000 7fb68fd36740 10 datascan.scan_links: step 1: handling object 1.00000000 2023-11-06T02:14:35.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.950+0000 7fb68fd36740 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.49:6800/38114,v1:172.21.15.49:6801/38114] -- osd_op(unknown.0.0:47 11.1f 11:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b77400 con 0x55eaf2b45c00 2023-11-06T02:14:35.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.950+0000 7fb68dcf5640 1 -- 172.21.15.49:0/119336422 <== osd.2 v2:172.21.15.49:6800/38114 8 ==== osd_op_reply(47 1.00000000 [omap-get-vals out=524b] v0'0 uv8 ondisk = 0) v8 ==== 154+0+524 (crc 0 0 0) 0x55eaf3673440 con 0x55eaf2b45c00 2023-11-06T02:14:35.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.950+0000 7fb68fd36740 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.49:6808/38113,v1:172.21.15.49:6810/38113] -- osd_op(unknown.0.0:48 11.0 11:00000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b77800 con 0x55eaf2b44000 2023-11-06T02:14:35.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.950+0000 7fb68e4f6640 1 -- 172.21.15.49:0/119336422 <== osd.1 v2:172.21.15.49:6808/38113 3 ==== osd_op_reply(48 [pgnls start_epoch 82 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eaf3672fc0 con 0x55eaf2b44000 2023-11-06T02:14:35.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.950+0000 7fb68e4f6640 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.49:6809/38116,v1:172.21.15.49:6811/38116] -- osd_op(unknown.0.0:49 11.10 11:08000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b77800 con 0x55eaf2b44400 2023-11-06T02:14:35.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.950+0000 7fb68ecf7640 1 -- 172.21.15.49:0/119336422 <== osd.0 v2:172.21.15.49:6809/38116 9 ==== osd_op_reply(49 [pgnls start_epoch 82 out=177b] v81'16 uv16 ondisk = 1) v8 ==== 144+0+177 (crc 0 0 0) 0x55eaf3673200 con 0x55eaf2b44400 2023-11-06T02:14:35.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.950+0000 7fb68ecf7640 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.49:6824/38115,v1:172.21.15.49:6825/38115] -- osd_op(unknown.0.0:50 11.8 11:10000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b5b800 con 0x55eaf3551000 2023-11-06T02:14:35.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.950+0000 7fb68e4f6640 1 -- 172.21.15.49:0/119336422 <== osd.3 v2:172.21.15.49:6824/38115 5 ==== osd_op_reply(50 [pgnls start_epoch 82 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eaf3672fc0 con 0x55eaf3551000 2023-11-06T02:14:35.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.950+0000 7fb68e4f6640 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.49:6800/38114,v1:172.21.15.49:6801/38114] -- osd_op(unknown.0.0:51 11.18 11:18000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b5b800 con 0x55eaf2b45c00 2023-11-06T02:14:35.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.950+0000 7fb68dcf5640 1 -- 172.21.15.49:0/119336422 <== osd.2 v2:172.21.15.49:6800/38114 9 ==== osd_op_reply(51 [pgnls start_epoch 82 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eaf3673440 con 0x55eaf2b45c00 2023-11-06T02:14:35.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.950+0000 7fb68dcf5640 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] -- osd_op(unknown.0.0:52 11.4 11:20000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b5b800 con 0x55eaf3550c00 2023-11-06T02:14:35.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.954+0000 7fb68ecf7640 1 -- 172.21.15.49:0/119336422 <== osd.7 v2:172.21.15.150:6816/38421 10 ==== osd_op_reply(52 [pgnls start_epoch 82 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eaf3673200 con 0x55eaf3550c00 2023-11-06T02:14:35.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.954+0000 7fb68ecf7640 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.150:6824/38428,v1:172.21.15.150:6825/38428] -- osd_op(unknown.0.0:53 11.14 11:28000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b5b800 con 0x55eaf2b5a000 2023-11-06T02:14:35.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.954+0000 7fb68e4f6640 1 -- 172.21.15.49:0/119336422 <== osd.6 v2:172.21.15.150:6824/38428 7 ==== osd_op_reply(53 [pgnls start_epoch 82 out=149b] v82'44 uv44 ondisk = 1) v8 ==== 144+0+149 (crc 0 0 0) 0x55eaf3672fc0 con 0x55eaf2b5a000 2023-11-06T02:14:35.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.954+0000 7fb68e4f6640 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.49:6808/38113,v1:172.21.15.49:6810/38113] -- osd_op(unknown.0.0:54 11.c 11:30000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b5b800 con 0x55eaf2b44000 2023-11-06T02:14:35.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.954+0000 7fb68e4f6640 1 -- 172.21.15.49:0/119336422 <== osd.1 v2:172.21.15.49:6808/38113 4 ==== osd_op_reply(54 [pgnls start_epoch 82 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eaf3672fc0 con 0x55eaf2b44000 2023-11-06T02:14:35.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.954+0000 7fb68e4f6640 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.150:6824/38428,v1:172.21.15.150:6825/38428] -- osd_op(unknown.0.0:55 11.1c 11:38000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b5b800 con 0x55eaf2b5a000 2023-11-06T02:14:35.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.954+0000 7fb68e4f6640 1 -- 172.21.15.49:0/119336422 <== osd.6 v2:172.21.15.150:6824/38428 8 ==== osd_op_reply(55 [pgnls start_epoch 82 out=73b] v81'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55eaf3672fc0 con 0x55eaf2b5a000 2023-11-06T02:14:35.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.954+0000 7fb68e4f6640 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.49:6800/38114,v1:172.21.15.49:6801/38114] -- osd_op(unknown.0.0:56 11.2 11:40000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b5b800 con 0x55eaf2b45c00 2023-11-06T02:14:35.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.954+0000 7fb68dcf5640 1 -- 172.21.15.49:0/119336422 <== osd.2 v2:172.21.15.49:6800/38114 10 ==== osd_op_reply(56 [pgnls start_epoch 82 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eaf3673440 con 0x55eaf2b45c00 2023-11-06T02:14:35.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.954+0000 7fb68dcf5640 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] -- osd_op(unknown.0.0:57 11.12 11:48000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b5b800 con 0x55eaf3550c00 2023-11-06T02:14:35.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.958+0000 7fb68ecf7640 1 -- 172.21.15.49:0/119336422 <== osd.7 v2:172.21.15.150:6816/38421 11 ==== osd_op_reply(57 [pgnls start_epoch 82 out=81b] v81'3 uv3 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55eaf3673200 con 0x55eaf3550c00 2023-11-06T02:14:35.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.958+0000 7fb68ecf7640 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.49:6824/38115,v1:172.21.15.49:6825/38115] -- osd_op(unknown.0.0:58 11.a 11:50000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b5b800 con 0x55eaf3551000 2023-11-06T02:14:35.980 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.958+0000 7fb68e4f6640 1 -- 172.21.15.49:0/119336422 <== osd.3 v2:172.21.15.49:6824/38115 6 ==== osd_op_reply(58 [pgnls start_epoch 82 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eaf3672fc0 con 0x55eaf3551000 2023-11-06T02:14:35.980 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.958+0000 7fb68e4f6640 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] -- osd_op(unknown.0.0:59 11.1a 11:58000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b5b800 con 0x55eaf2b5b000 2023-11-06T02:14:35.989 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.958+0000 7fb68dcf5640 1 -- 172.21.15.49:0/119336422 <== osd.5 v2:172.21.15.150:6800/38418 8 ==== osd_op_reply(59 [pgnls start_epoch 82 out=49b] v82'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eaf3673440 con 0x55eaf2b5b000 2023-11-06T02:14:35.989 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.958+0000 7fb68dcf5640 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.49:6809/38116,v1:172.21.15.49:6811/38116] -- osd_op(unknown.0.0:60 11.6 11:60000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b5b800 con 0x55eaf2b44400 2023-11-06T02:14:35.989 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.958+0000 7fb68ecf7640 1 -- 172.21.15.49:0/119336422 <== osd.0 v2:172.21.15.49:6809/38116 10 ==== osd_op_reply(60 [pgnls start_epoch 82 out=77b] v81'11 uv11 ondisk = 1) v8 ==== 144+0+77 (crc 0 0 0) 0x55eaf3673200 con 0x55eaf2b44400 2023-11-06T02:14:35.989 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.958+0000 7fb68ecf7640 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] -- osd_op(unknown.0.0:61 11.16 11:68000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b5b800 con 0x55eaf2b5b000 2023-11-06T02:14:35.990 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.958+0000 7fb68dcf5640 1 -- 172.21.15.49:0/119336422 <== osd.5 v2:172.21.15.150:6800/38418 9 ==== osd_op_reply(61 [pgnls start_epoch 82 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eaf3673440 con 0x55eaf2b5b000 2023-11-06T02:14:35.990 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.958+0000 7fb68dcf5640 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.150:6808/38417,v1:172.21.15.150:6809/38417] -- osd_op(unknown.0.0:62 11.e 11:70000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b5b800 con 0x55eaf3550400 2023-11-06T02:14:35.990 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.962+0000 7fb68ecf7640 1 -- 172.21.15.49:0/119336422 <== osd.4 v2:172.21.15.150:6808/38417 4 ==== osd_op_reply(62 [pgnls start_epoch 82 out=73b] v81'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55eaf3673200 con 0x55eaf3550400 2023-11-06T02:14:35.990 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.962+0000 7fb68ecf7640 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] -- osd_op(unknown.0.0:63 11.1e 11:78000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b5b800 con 0x55eaf3550c00 2023-11-06T02:14:35.990 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.962+0000 7fb68ecf7640 1 -- 172.21.15.49:0/119336422 <== osd.7 v2:172.21.15.150:6816/38421 12 ==== osd_op_reply(63 [pgnls start_epoch 82 out=49b] v82'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eaf3673200 con 0x55eaf3550c00 2023-11-06T02:14:35.991 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.962+0000 7fb68ecf7640 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.150:6824/38428,v1:172.21.15.150:6825/38428] -- osd_op(unknown.0.0:64 11.1 11:80000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b5b800 con 0x55eaf2b5a000 2023-11-06T02:14:35.991 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.962+0000 7fb68e4f6640 1 -- 172.21.15.49:0/119336422 <== osd.6 v2:172.21.15.150:6824/38428 9 ==== osd_op_reply(64 [pgnls start_epoch 82 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eaf3672fc0 con 0x55eaf2b5a000 2023-11-06T02:14:35.991 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.962+0000 7fb68e4f6640 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.49:6800/38114,v1:172.21.15.49:6801/38114] -- osd_op(unknown.0.0:65 11.11 11:88000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b5b800 con 0x55eaf2b45c00 2023-11-06T02:14:35.991 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.962+0000 7fb68dcf5640 1 -- 172.21.15.49:0/119336422 <== osd.2 v2:172.21.15.49:6800/38114 11 ==== osd_op_reply(65 [pgnls start_epoch 82 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eaf3673440 con 0x55eaf2b45c00 2023-11-06T02:14:35.991 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.962+0000 7fb68dcf5640 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.49:6824/38115,v1:172.21.15.49:6825/38115] -- osd_op(unknown.0.0:66 11.9 11:90000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b5b800 con 0x55eaf3551000 2023-11-06T02:14:35.992 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.962+0000 7fb68e4f6640 1 -- 172.21.15.49:0/119336422 <== osd.3 v2:172.21.15.49:6824/38115 7 ==== osd_op_reply(66 [pgnls start_epoch 82 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eaf3672fc0 con 0x55eaf3551000 2023-11-06T02:14:35.992 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.962+0000 7fb68e4f6640 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.49:6809/38116,v1:172.21.15.49:6811/38116] -- osd_op(unknown.0.0:67 11.19 11:98000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b5b800 con 0x55eaf2b44400 2023-11-06T02:14:35.992 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.966+0000 7fb68ecf7640 1 -- 172.21.15.49:0/119336422 <== osd.0 v2:172.21.15.49:6809/38116 11 ==== osd_op_reply(67 [pgnls start_epoch 82 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eaf3673200 con 0x55eaf2b44400 2023-11-06T02:14:35.992 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.966+0000 7fb68ecf7640 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] -- osd_op(unknown.0.0:68 11.5 11:a0000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b5b800 con 0x55eaf3550c00 2023-11-06T02:14:35.992 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.966+0000 7fb68ecf7640 1 -- 172.21.15.49:0/119336422 <== osd.7 v2:172.21.15.150:6816/38421 13 ==== osd_op_reply(68 [pgnls start_epoch 82 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eaf3673200 con 0x55eaf3550c00 2023-11-06T02:14:35.992 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.966+0000 7fb68ecf7640 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.49:6800/38114,v1:172.21.15.49:6801/38114] -- osd_op(unknown.0.0:69 11.15 11:a8000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b5b800 con 0x55eaf2b45c00 2023-11-06T02:14:35.993 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.966+0000 7fb68dcf5640 1 -- 172.21.15.49:0/119336422 <== osd.2 v2:172.21.15.49:6800/38114 12 ==== osd_op_reply(69 [pgnls start_epoch 82 out=97b] v81'4 uv4 ondisk = 1) v8 ==== 144+0+97 (crc 0 0 0) 0x55eaf3673440 con 0x55eaf2b45c00 2023-11-06T02:14:35.993 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.966+0000 7fb68dcf5640 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.49:6800/38114,v1:172.21.15.49:6801/38114] -- osd_op(unknown.0.0:70 11.d 11:b0000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b5b800 con 0x55eaf2b45c00 2023-11-06T02:14:35.993 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.966+0000 7fb68dcf5640 1 -- 172.21.15.49:0/119336422 <== osd.2 v2:172.21.15.49:6800/38114 13 ==== osd_op_reply(70 [pgnls start_epoch 82 out=74b] v79'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x55eaf3673440 con 0x55eaf2b45c00 2023-11-06T02:14:35.993 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.966+0000 7fb68dcf5640 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] -- osd_op(unknown.0.0:71 11.1d 11:b8000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b5b800 con 0x55eaf3550c00 2023-11-06T02:14:35.993 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.966+0000 7fb68ecf7640 1 -- 172.21.15.49:0/119336422 <== osd.7 v2:172.21.15.150:6816/38421 14 ==== osd_op_reply(71 [pgnls start_epoch 82 out=73b] v81'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55eaf3673200 con 0x55eaf3550c00 2023-11-06T02:14:35.994 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.966+0000 7fb68ecf7640 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.150:6824/38428,v1:172.21.15.150:6825/38428] -- osd_op(unknown.0.0:72 11.3 11:c0000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b5b800 con 0x55eaf2b5a000 2023-11-06T02:14:35.994 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.966+0000 7fb68e4f6640 1 -- 172.21.15.49:0/119336422 <== osd.6 v2:172.21.15.150:6824/38428 10 ==== osd_op_reply(72 [pgnls start_epoch 82 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eaf3672fc0 con 0x55eaf2b5a000 2023-11-06T02:14:35.994 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.966+0000 7fb68e4f6640 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] -- osd_op(unknown.0.0:73 11.13 11:c8000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b5b800 con 0x55eaf2b5b000 2023-11-06T02:14:35.994 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.970+0000 7fb68dcf5640 1 -- 172.21.15.49:0/119336422 <== osd.5 v2:172.21.15.150:6800/38418 10 ==== osd_op_reply(73 [pgnls start_epoch 82 out=98b] v81'5 uv5 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x55eaf3673440 con 0x55eaf2b5b000 2023-11-06T02:14:35.994 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.970+0000 7fb68dcf5640 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.49:6824/38115,v1:172.21.15.49:6825/38115] -- osd_op(unknown.0.0:74 11.b 11:d0000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b5b800 con 0x55eaf3551000 2023-11-06T02:14:35.995 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.970+0000 7fb68e4f6640 1 -- 172.21.15.49:0/119336422 <== osd.3 v2:172.21.15.49:6824/38115 8 ==== osd_op_reply(74 [pgnls start_epoch 82 out=76b] v82'6 uv5 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x55eaf3672fc0 con 0x55eaf3551000 2023-11-06T02:14:35.995 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.970+0000 7fb68e4f6640 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.150:6808/38417,v1:172.21.15.150:6809/38417] -- osd_op(unknown.0.0:75 11.1b 11:d8000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b5b800 con 0x55eaf3550400 2023-11-06T02:14:35.995 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.970+0000 7fb68ecf7640 1 -- 172.21.15.49:0/119336422 <== osd.4 v2:172.21.15.150:6808/38417 5 ==== osd_op_reply(75 [pgnls start_epoch 82 out=49b] v82'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eaf3673200 con 0x55eaf3550400 2023-11-06T02:14:35.995 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.970+0000 7fb68ecf7640 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] -- osd_op(unknown.0.0:76 11.7 11:e0000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b5b800 con 0x55eaf2b5b000 2023-11-06T02:14:35.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.970+0000 7fb68dcf5640 1 -- 172.21.15.49:0/119336422 <== osd.5 v2:172.21.15.150:6800/38418 11 ==== osd_op_reply(76 [pgnls start_epoch 82 out=105b] v82'9 uv9 ondisk = 1) v8 ==== 144+0+105 (crc 0 0 0) 0x55eaf3673440 con 0x55eaf2b5b000 2023-11-06T02:14:35.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.970+0000 7fb68dcf5640 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] -- osd_op(unknown.0.0:77 11.17 11:e8000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b5b800 con 0x55eaf3550c00 2023-11-06T02:14:35.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.974+0000 7fb68ecf7640 1 -- 172.21.15.49:0/119336422 <== osd.7 v2:172.21.15.150:6816/38421 15 ==== osd_op_reply(77 [pgnls start_epoch 82 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eaf3673200 con 0x55eaf3550c00 2023-11-06T02:14:35.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.974+0000 7fb68ecf7640 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] -- osd_op(unknown.0.0:78 11.f 11:f0000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b5b800 con 0x55eaf3550c00 2023-11-06T02:14:35.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.974+0000 7fb68ecf7640 1 -- 172.21.15.49:0/119336422 <== osd.7 v2:172.21.15.150:6816/38421 16 ==== osd_op_reply(78 [pgnls start_epoch 82 out=73b] v82'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55eaf3673200 con 0x55eaf3550c00 2023-11-06T02:14:35.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.974+0000 7fb68ecf7640 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.49:6800/38114,v1:172.21.15.49:6801/38114] -- osd_op(unknown.0.0:79 11.1f 11:f8000000::::head [pgnls start_epoch 82 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b5b800 con 0x55eaf2b45c00 2023-11-06T02:14:35.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.974+0000 7fb68dcf5640 1 -- 172.21.15.49:0/119336422 <== osd.2 v2:172.21.15.49:6800/38114 14 ==== osd_op_reply(79 [pgnls start_epoch 82 out=101b] v82'9 uv8 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x55eaf3673440 con 0x55eaf2b45c00 2023-11-06T02:14:35.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.974+0000 7fb68fd36740 10 datascan.scan_links: step 2: handling object 601.00000000 2023-11-06T02:14:35.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.974+0000 7fb68fd36740 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.49:6809/38116,v1:172.21.15.49:6811/38116] -- osd_op(unknown.0.0:80 11.10 11:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b76400 con 0x55eaf2b44400 2023-11-06T02:14:35.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.974+0000 7fb68ecf7640 1 -- 172.21.15.49:0/119336422 <== osd.0 v2:172.21.15.49:6809/38116 12 ==== osd_op_reply(80 601.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55eaf3673200 con 0x55eaf2b44400 2023-11-06T02:14:35.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.974+0000 7fb68fd36740 10 datascan.scan_links: step 2: handling object 602.00000000 2023-11-06T02:14:35.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.974+0000 7fb68fd36740 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.49:6809/38116,v1:172.21.15.49:6811/38116] -- osd_op(unknown.0.0:81 11.10 11:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b77800 con 0x55eaf2b44400 2023-11-06T02:14:35.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.978+0000 7fb68ecf7640 1 -- 172.21.15.49:0/119336422 <== osd.0 v2:172.21.15.49:6809/38116 13 ==== osd_op_reply(81 602.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55eaf3673200 con 0x55eaf2b44400 2023-11-06T02:14:35.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.978+0000 7fb68fd36740 10 datascan.scan_links: step 2: handling object 10000000000.00000000 2023-11-06T02:14:35.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.978+0000 7fb68fd36740 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.49:6809/38116,v1:172.21.15.49:6811/38116] -- osd_op(unknown.0.0:82 11.10 11:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b76000 con 0x55eaf2b44400 2023-11-06T02:14:35.999 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.978+0000 7fb68ecf7640 1 -- 172.21.15.49:0/119336422 <== osd.0 v2:172.21.15.49:6809/38116 14 ==== osd_op_reply(82 10000000000.00000000 [omap-get-vals out=1047b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+1047 (crc 0 0 0) 0x55eaf3673200 con 0x55eaf2b44400 2023-11-06T02:14:35.999 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.978+0000 7fb68fd36740 10 datascan.scan_links: step 2: handling object 600.00000000 2023-11-06T02:14:35.999 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.978+0000 7fb68fd36740 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.49:6809/38116,v1:172.21.15.49:6811/38116] -- osd_op(unknown.0.0:83 11.10 11:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b76800 con 0x55eaf2b44400 2023-11-06T02:14:35.999 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.978+0000 7fb68ecf7640 1 -- 172.21.15.49:0/119336422 <== osd.0 v2:172.21.15.49:6809/38116 15 ==== osd_op_reply(83 600.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55eaf3673200 con 0x55eaf2b44400 2023-11-06T02:14:35.999 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.978+0000 7fb68fd36740 10 datascan.scan_links: step 2: handling object 603.00000000 2023-11-06T02:14:36.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.978+0000 7fb68fd36740 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.49:6809/38116,v1:172.21.15.49:6811/38116] -- osd_op(unknown.0.0:84 11.10 11:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b76c00 con 0x55eaf2b44400 2023-11-06T02:14:36.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.978+0000 7fb68ecf7640 1 -- 172.21.15.49:0/119336422 <== osd.0 v2:172.21.15.49:6809/38116 16 ==== osd_op_reply(84 603.00000000 [omap-get-vals out=5b] v0'0 uv16 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55eaf3673200 con 0x55eaf2b44400 2023-11-06T02:14:36.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.978+0000 7fb68fd36740 10 datascan.scan_links: step 2: handling object 1.00000000.inode 2023-11-06T02:14:36.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.978+0000 7fb68fd36740 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2023-11-06T02:14:36.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.978+0000 7fb68fd36740 10 datascan.scan_links: step 2: handling object 200.00000000 2023-11-06T02:14:36.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.978+0000 7fb68fd36740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2023-11-06T02:14:36.001 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.978+0000 7fb68fd36740 10 datascan.scan_links: step 2: handling object 200.00000001 2023-11-06T02:14:36.001 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.978+0000 7fb68fd36740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2023-11-06T02:14:36.001 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.978+0000 7fb68fd36740 10 datascan.scan_links: step 2: handling object 606.00000000 2023-11-06T02:14:36.001 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.978+0000 7fb68fd36740 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.150:6824/38428,v1:172.21.15.150:6825/38428] -- osd_op(unknown.0.0:85 11.14 11:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b77c00 con 0x55eaf2b5a000 2023-11-06T02:14:36.001 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.978+0000 7fb68e4f6640 1 -- 172.21.15.49:0/119336422 <== osd.6 v2:172.21.15.150:6824/38428 11 ==== osd_op_reply(85 606.00000000 [omap-get-vals out=5b] v0'0 uv39 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55eaf3672fc0 con 0x55eaf2b5a000 2023-11-06T02:14:36.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.978+0000 7fb68fd36740 10 datascan.scan_links: step 2: handling object 607.00000000 2023-11-06T02:14:36.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.978+0000 7fb68fd36740 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.150:6824/38428,v1:172.21.15.150:6825/38428] -- osd_op(unknown.0.0:86 11.1c 11:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b5a400 con 0x55eaf2b5a000 2023-11-06T02:14:36.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.978+0000 7fb68e4f6640 1 -- 172.21.15.49:0/119336422 <== osd.6 v2:172.21.15.150:6824/38428 12 ==== osd_op_reply(86 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55eaf3672fc0 con 0x55eaf2b5a000 2023-11-06T02:14:36.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.978+0000 7fb68fd36740 10 datascan.scan_links: step 2: handling object 10000000003.00000000 2023-11-06T02:14:36.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.978+0000 7fb68fd36740 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] -- osd_op(unknown.0.0:87 11.12 11:4def72d4:::10000000003.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b76400 con 0x55eaf3550c00 2023-11-06T02:14:36.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.982+0000 7fb68ecf7640 1 -- 172.21.15.49:0/119336422 <== osd.7 v2:172.21.15.150:6816/38421 17 ==== osd_op_reply(87 10000000003.00000000 [omap-get-vals out=523b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+523 (crc 0 0 0) 0x55eaf3673200 con 0x55eaf3550c00 2023-11-06T02:14:36.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.982+0000 7fb68fd36740 10 datascan.scan_links: step 2: handling object mds0_openfiles.0 2023-11-06T02:14:36.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.982+0000 7fb68fd36740 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2023-11-06T02:14:36.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.982+0000 7fb68fd36740 10 datascan.scan_links: step 2: handling object 608.00000000 2023-11-06T02:14:36.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.982+0000 7fb68fd36740 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.150:6808/38417,v1:172.21.15.150:6809/38417] -- osd_op(unknown.0.0:88 11.e 11:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b77800 con 0x55eaf3550400 2023-11-06T02:14:36.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.982+0000 7fb68ecf7640 1 -- 172.21.15.49:0/119336422 <== osd.4 v2:172.21.15.150:6808/38417 6 ==== osd_op_reply(88 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55eaf3673200 con 0x55eaf3550400 2023-11-06T02:14:36.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.982+0000 7fb68fd36740 10 datascan.scan_links: step 2: handling object 604.00000000 2023-11-06T02:14:36.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.982+0000 7fb68fd36740 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.49:6800/38114,v1:172.21.15.49:6801/38114] -- osd_op(unknown.0.0:89 11.15 11:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b76000 con 0x55eaf2b45c00 2023-11-06T02:14:36.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.982+0000 7fb68dcf5640 1 -- 172.21.15.49:0/119336422 <== osd.2 v2:172.21.15.49:6800/38114 15 ==== osd_op_reply(89 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55eaf3673440 con 0x55eaf2b45c00 2023-11-06T02:14:36.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.982+0000 7fb68fd36740 10 datascan.scan_links: step 2: handling object 500.00000000 2023-11-06T02:14:36.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.982+0000 7fb68fd36740 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2023-11-06T02:14:36.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.982+0000 7fb68fd36740 10 datascan.scan_links: step 2: handling object mds_snaptable 2023-11-06T02:14:36.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.982+0000 7fb68fd36740 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2023-11-06T02:14:36.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.982+0000 7fb68fd36740 10 datascan.scan_links: step 2: handling object 605.00000000 2023-11-06T02:14:36.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.982+0000 7fb68fd36740 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] -- osd_op(unknown.0.0:90 11.1d 11:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b5a800 con 0x55eaf3550c00 2023-11-06T02:14:36.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.982+0000 7fb68ecf7640 1 -- 172.21.15.49:0/119336422 <== osd.7 v2:172.21.15.150:6816/38421 18 ==== osd_op_reply(90 605.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55eaf3673200 con 0x55eaf3550c00 2023-11-06T02:14:36.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.982+0000 7fb68fd36740 10 datascan.scan_links: step 2: handling object mds0_inotable 2023-11-06T02:14:36.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.982+0000 7fb68fd36740 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2023-11-06T02:14:36.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.982+0000 7fb68fd36740 10 datascan.scan_links: step 2: handling object 100.00000000 2023-11-06T02:14:36.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.982+0000 7fb68fd36740 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] -- osd_op(unknown.0.0:91 11.13 11:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b5ac00 con 0x55eaf2b5b000 2023-11-06T02:14:36.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.986+0000 7fb68dcf5640 1 -- 172.21.15.49:0/119336422 <== osd.5 v2:172.21.15.150:6800/38418 12 ==== osd_op_reply(91 100.00000000 [omap-get-vals out=5195b] v0'0 uv5 ondisk = 0) v8 ==== 156+0+5195 (crc 0 0 0) 0x55eaf3673440 con 0x55eaf2b5b000 2023-11-06T02:14:36.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.986+0000 7fb68fd36740 10 datascan.scan_links: step 2: handling object mds0_sessionmap 2023-11-06T02:14:36.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.986+0000 7fb68fd36740 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2023-11-06T02:14:36.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.986+0000 7fb68fd36740 10 datascan.scan_links: step 2: handling object 609.00000000 2023-11-06T02:14:36.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.986+0000 7fb68fd36740 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] -- osd_op(unknown.0.0:92 11.7 11:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b5b400 con 0x55eaf2b5b000 2023-11-06T02:14:36.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.986+0000 7fb68dcf5640 1 -- 172.21.15.49:0/119336422 <== osd.5 v2:172.21.15.150:6800/38418 13 ==== osd_op_reply(92 609.00000000 [omap-get-vals out=5b] v0'0 uv7 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55eaf3673440 con 0x55eaf2b5b000 2023-11-06T02:14:36.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.986+0000 7fb68fd36740 10 datascan.scan_links: step 2: handling object 10000000001.00000000 2023-11-06T02:14:36.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.986+0000 7fb68fd36740 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] -- osd_op(unknown.0.0:93 11.7 11:e2e07930:::10000000001.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b45000 con 0x55eaf2b5b000 2023-11-06T02:14:36.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.986+0000 7fb68dcf5640 1 -- 172.21.15.49:0/119336422 <== osd.5 v2:172.21.15.150:6800/38418 14 ==== osd_op_reply(93 10000000001.00000000 [omap-get-vals out=1545b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+1545 (crc 0 0 0) 0x55eaf3673440 con 0x55eaf2b5b000 2023-11-06T02:14:36.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.986+0000 7fb68fd36740 20 datascan.scan_links: injected ino detected 2023-11-06T02:14:36.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.986+0000 7fb68fd36740 20 datascan.scan_links: adding 0x100000001f7 for future processing to fix dnfirst 2023-11-06T02:14:36.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.986+0000 7fb68fd36740 10 datascan.scan_links: step 2: handling object 400.00000000 2023-11-06T02:14:36.009 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.986+0000 7fb68fd36740 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2023-11-06T02:14:36.009 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.986+0000 7fb68fd36740 10 datascan.scan_links: step 2: handling object 100.00000000.inode 2023-11-06T02:14:36.009 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.986+0000 7fb68fd36740 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2023-11-06T02:14:36.009 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.986+0000 7fb68fd36740 10 datascan.scan_links: step 2: handling object 1.00000000 2023-11-06T02:14:36.009 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.986+0000 7fb68fd36740 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.49:6800/38114,v1:172.21.15.49:6801/38114] -- osd_op(unknown.0.0:94 11.1f 11:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b44c00 con 0x55eaf2b45c00 2023-11-06T02:14:36.010 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.986+0000 7fb68dcf5640 1 -- 172.21.15.49:0/119336422 <== osd.2 v2:172.21.15.49:6800/38114 16 ==== osd_op_reply(94 1.00000000 [omap-get-vals out=524b] v0'0 uv8 ondisk = 0) v8 ==== 154+0+524 (crc 0 0 0) 0x55eaf3673440 con 0x55eaf2b45c00 2023-11-06T02:14:36.010 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.986+0000 7fb68fd36740 10 datascan.scan_links: processing 0 dup_primaries, 0 remote_links 2023-11-06T02:14:36.010 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.986+0000 7fb68fd36740 10 datascan.scan_links: removing dup dentries from 0 objects 2023-11-06T02:14:36.010 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.986+0000 7fb68fd36740 10 datascan.scan_links: processing 0 bad_nlink_inos 2023-11-06T02:14:36.010 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.986+0000 7fb68fd36740 10 datascan.scan_links: processing 1 injected_inos 2023-11-06T02:14:36.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.986+0000 7fb68fd36740 10 datascan.scan_links: handling injected_ino 0x100000001f7 2023-11-06T02:14:36.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.986+0000 7fb68fd36740 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000001:bravo 2023-11-06T02:14:36.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.986+0000 7fb68fd36740 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] -- osd_op(unknown.0.0:95 11.7 11:e2e07930:::10000000001.00000000:head [omap-get-vals-by-keys in=18b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b45000 con 0x55eaf2b5b000 2023-11-06T02:14:36.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.986+0000 7fb68dcf5640 1 -- 172.21.15.49:0/119336422 <== osd.5 v2:172.21.15.150:6800/38418 15 ==== osd_op_reply(95 10000000001.00000000 [omap-get-vals-by-keys out=506b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+506 (crc 0 0 0) 0x55eaf3673440 con 0x55eaf2b5b000 2023-11-06T02:14:36.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.986+0000 7fb68fd36740 20 datascan.read_dentry: oid=10000000001.00000000 dname=bravo frag=*, r=0 2023-11-06T02:14:36.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.986+0000 7fb68fd36740 20 datascan.scan_links: first is now 2 2023-11-06T02:14:36.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.986+0000 7fb68fd36740 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] -- osd_op(unknown.0.0:96 11.7 11:e2e07930:::10000000001.00000000:head [omap-set-vals in=506b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b5b800 con 0x55eaf2b5b000 2023-11-06T02:14:36.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.990+0000 7fb68dcf5640 1 -- 172.21.15.49:0/119336422 <== osd.5 v2:172.21.15.150:6800/38418 16 ==== osd_op_reply(96 10000000001.00000000 [omap-set-vals] v82'10 uv10 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55eaf3673440 con 0x55eaf2b5b000 2023-11-06T02:14:36.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.990+0000 7fb68fd36740 20 datascan.inject_linkage: Injected dentry 0x0x10000000001/bravo pointing to 0x0x100000001f7 2023-11-06T02:14:36.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.990+0000 7fb68fd36740 10 datascan.scan_links: updating inotable 2023-11-06T02:14:36.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.990+0000 7fb68fd36740 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] -- osd_op(unknown.0.0:97 11.13 11:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b5ac00 con 0x55eaf2b5b000 2023-11-06T02:14:36.013 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.990+0000 7fb68dcf5640 1 -- 172.21.15.49:0/119336422 <== osd.5 v2:172.21.15.150:6800/38418 17 ==== osd_op_reply(97 mds0_inotable [read 0~50 out=50b] v0'0 uv4 ondisk = 0) v8 ==== 157+0+50 (crc 0 0 0) 0x55eaf3673440 con 0x55eaf2b5b000 2023-11-06T02:14:36.013 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.990+0000 7fb68fd36740 10 mds.0.inotable: skip_inos was [0x10000000005~0x1f2,0x100000001f9~0xfffffffe07] 2023-11-06T02:14:36.013 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.990+0000 7fb68fd36740 10 mds.0.inotable: skip_inos now [0x100000001f9~0xfffffffe07] 2023-11-06T02:14:36.013 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.990+0000 7fb68fd36740 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] -- osd_op(unknown.0.0:98 11.13 11:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b5b400 con 0x55eaf2b5b000 2023-11-06T02:14:36.013 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.994+0000 7fb68dcf5640 1 -- 172.21.15.49:0/119336422 <== osd.5 v2:172.21.15.150:6800/38418 18 ==== osd_op_reply(98 mds0_inotable [writefull 0~34] v82'6 uv6 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x55eaf3673440 con 0x55eaf2b5b000 2023-11-06T02:14:36.014 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.994+0000 7fb68fd36740 10 datascan.scan_links: updating snaptable 2023-11-06T02:14:36.014 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.994+0000 7fb68fd36740 1 -- 172.21.15.49:0/119336422 --> [v2:172.21.15.49:6800/38114,v1:172.21.15.49:6801/38114] -- osd_op(unknown.0.0:99 11.d 11:b50e409b:::mds_snaptable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82) v8 -- 0x55eaf2b77000 con 0x55eaf2b45c00 2023-11-06T02:14:36.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.994+0000 7fb68dcf5640 1 -- 172.21.15.49:0/119336422 <== osd.2 v2:172.21.15.49:6800/38114 17 ==== osd_op_reply(99 mds_snaptable [read 0~70 out=70b] v0'0 uv1 ondisk = 0) v8 ==== 157+0+70 (crc 0 0 0) 0x55eaf3673440 con 0x55eaf2b45c00 2023-11-06T02:14:36.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.994+0000 7fb68fd36740 1 -- 172.21.15.49:0/119336422 >> [v2:172.21.15.49:6809/38116,v1:172.21.15.49:6811/38116] conn(0x55eaf2b44400 msgr2=0x55eaf3680000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:36.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.994+0000 7fb68fd36740 1 --2- 172.21.15.49:0/119336422 >> [v2:172.21.15.49:6809/38116,v1:172.21.15.49:6811/38116] conn(0x55eaf2b44400 0x55eaf3680000 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:36.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.994+0000 7fb68fd36740 1 -- 172.21.15.49:0/119336422 >> [v2:172.21.15.49:6808/38113,v1:172.21.15.49:6810/38113] conn(0x55eaf2b44000 msgr2=0x55eaf27f6100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:36.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.994+0000 7fb68fd36740 1 --2- 172.21.15.49:0/119336422 >> [v2:172.21.15.49:6808/38113,v1:172.21.15.49:6810/38113] conn(0x55eaf2b44000 0x55eaf27f6100 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:36.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.994+0000 7fb68fd36740 1 -- 172.21.15.49:0/119336422 >> [v2:172.21.15.49:6800/38114,v1:172.21.15.49:6801/38114] conn(0x55eaf2b45c00 msgr2=0x55eaf3680b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:36.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.994+0000 7fb68fd36740 1 --2- 172.21.15.49:0/119336422 >> [v2:172.21.15.49:6800/38114,v1:172.21.15.49:6801/38114] conn(0x55eaf2b45c00 0x55eaf3680b00 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:36.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.994+0000 7fb68fd36740 1 -- 172.21.15.49:0/119336422 >> [v2:172.21.15.49:6824/38115,v1:172.21.15.49:6825/38115] conn(0x55eaf3551000 msgr2=0x55eaf3680580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:36.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.994+0000 7fb68fd36740 1 --2- 172.21.15.49:0/119336422 >> [v2:172.21.15.49:6824/38115,v1:172.21.15.49:6825/38115] conn(0x55eaf3551000 0x55eaf3680580 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:36.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.994+0000 7fb68fd36740 1 -- 172.21.15.49:0/119336422 >> [v2:172.21.15.150:6808/38417,v1:172.21.15.150:6809/38417] conn(0x55eaf3550400 msgr2=0x55eaf3682680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:36.019 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.994+0000 7fb68fd36740 1 --2- 172.21.15.49:0/119336422 >> [v2:172.21.15.150:6808/38417,v1:172.21.15.150:6809/38417] conn(0x55eaf3550400 0x55eaf3682680 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:36.019 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.994+0000 7fb68fd36740 1 -- 172.21.15.49:0/119336422 >> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] conn(0x55eaf2b5b000 msgr2=0x55eaf3681080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:36.019 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.994+0000 7fb68fd36740 1 --2- 172.21.15.49:0/119336422 >> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] conn(0x55eaf2b5b000 0x55eaf3681080 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:36.019 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.994+0000 7fb68fd36740 1 -- 172.21.15.49:0/119336422 >> [v2:172.21.15.150:6824/38428,v1:172.21.15.150:6825/38428] conn(0x55eaf2b5a000 msgr2=0x55eaf3681b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:36.019 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.994+0000 7fb68fd36740 1 --2- 172.21.15.49:0/119336422 >> [v2:172.21.15.150:6824/38428,v1:172.21.15.150:6825/38428] conn(0x55eaf2b5a000 0x55eaf3681b80 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:36.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.994+0000 7fb68fd36740 1 -- 172.21.15.49:0/119336422 >> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] conn(0x55eaf3550c00 msgr2=0x55eaf3681600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:36.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.994+0000 7fb68fd36740 1 --2- 172.21.15.49:0/119336422 >> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] conn(0x55eaf3550c00 0x55eaf3681600 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:36.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.994+0000 7fb68e4f6640 1 -- 172.21.15.49:0/119336422 reap_dead start 2023-11-06T02:14:36.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.994+0000 7fb68e4f6640 1 -- 172.21.15.49:0/119336422 reap_dead start 2023-11-06T02:14:36.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.994+0000 7fb68fd36740 1 -- 172.21.15.49:0/119336422 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x55eaf36ebc00 msgr2=0x55eaf27f7700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:36.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.994+0000 7fb68fd36740 1 --2- 172.21.15.49:0/119336422 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x55eaf36ebc00 0x55eaf27f7700 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x55eaf34c80f0 tx=0x55eaf369ecc0 comp rx=0 tx=0).stop 2023-11-06T02:14:36.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.994+0000 7fb68fd36740 1 -- 172.21.15.49:0/119336422 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eaf36eac00 msgr2=0x55eaf27f6680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:36.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.994+0000 7fb68fd36740 1 --2- 172.21.15.49:0/119336422 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eaf36eac00 0x55eaf27f6680 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x55eaf2a493b0 tx=0x55eaf369ec00 comp rx=0 tx=0).stop 2023-11-06T02:14:36.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.994+0000 7fb68e4f6640 1 -- 172.21.15.49:0/119336422 reap_dead start 2023-11-06T02:14:36.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.994+0000 7fb68e4f6640 1 -- 172.21.15.49:0/119336422 reap_dead start 2023-11-06T02:14:36.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.994+0000 7fb68e4f6640 1 -- 172.21.15.49:0/119336422 reap_dead start 2023-11-06T02:14:36.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.994+0000 7fb68fd36740 1 -- 172.21.15.49:0/119336422 shutdown_connections 2023-11-06T02:14:36.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.994+0000 7fb68fd36740 1 -- 172.21.15.49:0/119336422 >> 172.21.15.49:0/119336422 conn(0x55eaf36ea000 msgr2=0x55eaf2794000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:36.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.994+0000 7fb68fd36740 1 -- 172.21.15.49:0/119336422 shutdown_connections 2023-11-06T02:14:36.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.994+0000 7fb68fd36740 1 -- 172.21.15.49:0/119336422 wait complete. 2023-11-06T02:14:36.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.994+0000 7fb68fd36740 1 -- 172.21.15.49:0/4152712064 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eaf2769400 msgr2=0x55eaf27f4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:36.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.994+0000 7fb68fd36740 1 --2- 172.21.15.49:0/4152712064 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eaf2769400 0x55eaf27f4b00 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x55eaf35e8d80 tx=0x55eaf3540600 comp rx=0 tx=0).stop 2023-11-06T02:14:36.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.994+0000 7fb68fd36740 1 -- 172.21.15.49:0/4152712064 shutdown_connections 2023-11-06T02:14:36.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.994+0000 7fb68fd36740 1 --2- 172.21.15.49:0/4152712064 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eaf2769400 0x55eaf27f4b00 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:36.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.994+0000 7fb68fd36740 1 --2- 172.21.15.49:0/4152712064 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eaf2769800 0x55eaf27f5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:36.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.994+0000 7fb68fd36740 1 --2- 172.21.15.49:0/4152712064 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55eaf2769000 0x55eaf27f4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:36.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.994+0000 7fb68fd36740 1 -- 172.21.15.49:0/4152712064 >> 172.21.15.49:0/4152712064 conn(0x55eaf2768c00 msgr2=0x55eaf2793800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:36.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.994+0000 7fb68fd36740 1 -- 172.21.15.49:0/4152712064 shutdown_connections 2023-11-06T02:14:36.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:35.994+0000 7fb68fd36740 1 -- 172.21.15.49:0/4152712064 wait complete. 2023-11-06T02:14:36.024 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_links'] in time 0:00:00.192233, result: 2023-11-06T02:14:36.025 DEBUG:teuthology.orchestra.run.smithi049:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-06T02:14:36.061 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-06T02:14:36.091 INFO:teuthology.orchestra.run.smithi049.stdout:[global] 2023-11-06T02:14:36.091 INFO:teuthology.orchestra.run.smithi049.stdout: chdir = "" 2023-11-06T02:14:36.092 INFO:teuthology.orchestra.run.smithi049.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-06T02:14:36.092 INFO:teuthology.orchestra.run.smithi049.stdout: auth supported = cephx 2023-11-06T02:14:36.092 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:36.092 INFO:teuthology.orchestra.run.smithi049.stdout: filestore xattr use omap = true 2023-11-06T02:14:36.092 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:36.092 INFO:teuthology.orchestra.run.smithi049.stdout: mon clock drift allowed = 1.000 2023-11-06T02:14:36.093 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:36.093 INFO:teuthology.orchestra.run.smithi049.stdout: osd crush chooseleaf type = 0 2023-11-06T02:14:36.093 INFO:teuthology.orchestra.run.smithi049.stdout: auth debug = true 2023-11-06T02:14:36.093 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:36.093 INFO:teuthology.orchestra.run.smithi049.stdout: ms die on old message = true 2023-11-06T02:14:36.094 INFO:teuthology.orchestra.run.smithi049.stdout: ms die on bug = true 2023-11-06T02:14:36.094 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:36.094 INFO:teuthology.orchestra.run.smithi049.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-06T02:14:36.094 INFO:teuthology.orchestra.run.smithi049.stdout: mon pg warn max object skew = 0 2023-11-06T02:14:36.094 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:36.095 INFO:teuthology.orchestra.run.smithi049.stdout: # disable pg_autoscaler by default for new pools 2023-11-06T02:14:36.095 INFO:teuthology.orchestra.run.smithi049.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-06T02:14:36.095 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:36.095 INFO:teuthology.orchestra.run.smithi049.stdout: osd pool default size = 2 2023-11-06T02:14:36.095 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:36.096 INFO:teuthology.orchestra.run.smithi049.stdout: mon osd allow primary affinity = true 2023-11-06T02:14:36.096 INFO:teuthology.orchestra.run.smithi049.stdout: mon osd allow pg remap = true 2023-11-06T02:14:36.096 INFO:teuthology.orchestra.run.smithi049.stdout: mon warn on legacy crush tunables = false 2023-11-06T02:14:36.096 INFO:teuthology.orchestra.run.smithi049.stdout: mon warn on crush straw calc version zero = false 2023-11-06T02:14:36.096 INFO:teuthology.orchestra.run.smithi049.stdout: mon warn on no sortbitwise = false 2023-11-06T02:14:36.097 INFO:teuthology.orchestra.run.smithi049.stdout: mon warn on osd down out interval zero = false 2023-11-06T02:14:36.097 INFO:teuthology.orchestra.run.smithi049.stdout: mon warn on too few osds = false 2023-11-06T02:14:36.097 INFO:teuthology.orchestra.run.smithi049.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-06T02:14:36.097 INFO:teuthology.orchestra.run.smithi049.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-06T02:14:36.097 INFO:teuthology.orchestra.run.smithi049.stdout: mon_allow_pool_size_one = true 2023-11-06T02:14:36.098 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:36.098 INFO:teuthology.orchestra.run.smithi049.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-06T02:14:36.098 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:36.098 INFO:teuthology.orchestra.run.smithi049.stdout: osd default data pool replay window = 5 2023-11-06T02:14:36.098 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:36.099 INFO:teuthology.orchestra.run.smithi049.stdout: mon allow pool delete = true 2023-11-06T02:14:36.099 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:36.099 INFO:teuthology.orchestra.run.smithi049.stdout: mon cluster log file level = debug 2023-11-06T02:14:36.099 INFO:teuthology.orchestra.run.smithi049.stdout: debug asserts on shutdown = true 2023-11-06T02:14:36.099 INFO:teuthology.orchestra.run.smithi049.stdout: mon health detail to clog = false 2023-11-06T02:14:36.100 INFO:teuthology.orchestra.run.smithi049.stdout: mon host = "172.21.15.49,172.21.15.150,[v2:172.21.15.150:3301,v1:172.21.15.150:6790]" 2023-11-06T02:14:36.100 INFO:teuthology.orchestra.run.smithi049.stdout: fsid = 1a8b035c-6ba4-499a-b876-aea47fd9c0d4 2023-11-06T02:14:36.100 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:36.100 INFO:teuthology.orchestra.run.smithi049.stdout:[osd] 2023-11-06T02:14:36.100 INFO:teuthology.orchestra.run.smithi049.stdout: osd journal size = 100 2023-11-06T02:14:36.101 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:36.101 INFO:teuthology.orchestra.run.smithi049.stdout: osd scrub load threshold = 5.0 2023-11-06T02:14:36.101 INFO:teuthology.orchestra.run.smithi049.stdout: osd scrub max interval = 600 2023-11-06T02:14:36.101 INFO:teuthology.orchestra.run.smithi049.stdout: osd mclock profile = high_recovery_ops 2023-11-06T02:14:36.101 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:36.102 INFO:teuthology.orchestra.run.smithi049.stdout: osd recover clone overlap = true 2023-11-06T02:14:36.102 INFO:teuthology.orchestra.run.smithi049.stdout: osd recovery max chunk = 1048576 2023-11-06T02:14:36.102 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:36.102 INFO:teuthology.orchestra.run.smithi049.stdout: osd debug shutdown = true 2023-11-06T02:14:36.102 INFO:teuthology.orchestra.run.smithi049.stdout: osd debug op order = true 2023-11-06T02:14:36.103 INFO:teuthology.orchestra.run.smithi049.stdout: osd debug verify stray on activate = true 2023-11-06T02:14:36.103 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:36.103 INFO:teuthology.orchestra.run.smithi049.stdout: osd open classes on start = true 2023-11-06T02:14:36.103 INFO:teuthology.orchestra.run.smithi049.stdout: osd debug pg log writeout = true 2023-11-06T02:14:36.103 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:36.103 INFO:teuthology.orchestra.run.smithi049.stdout: osd deep scrub update digest min age = 30 2023-11-06T02:14:36.104 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:36.104 INFO:teuthology.orchestra.run.smithi049.stdout: osd map max advance = 10 2023-11-06T02:14:36.104 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:36.104 INFO:teuthology.orchestra.run.smithi049.stdout: journal zero on create = true 2023-11-06T02:14:36.104 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:36.105 INFO:teuthology.orchestra.run.smithi049.stdout: filestore ondisk finisher threads = 3 2023-11-06T02:14:36.105 INFO:teuthology.orchestra.run.smithi049.stdout: filestore apply finisher threads = 3 2023-11-06T02:14:36.105 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:36.105 INFO:teuthology.orchestra.run.smithi049.stdout: bdev debug aio = true 2023-11-06T02:14:36.105 INFO:teuthology.orchestra.run.smithi049.stdout: osd debug misdirected ops = true 2023-11-06T02:14:36.106 INFO:teuthology.orchestra.run.smithi049.stdout: bdev async discard = True 2023-11-06T02:14:36.106 INFO:teuthology.orchestra.run.smithi049.stdout: bdev enable discard = True 2023-11-06T02:14:36.106 INFO:teuthology.orchestra.run.smithi049.stdout: bluestore allocator = bitmap 2023-11-06T02:14:36.106 INFO:teuthology.orchestra.run.smithi049.stdout: bluestore block size = 96636764160 2023-11-06T02:14:36.106 INFO:teuthology.orchestra.run.smithi049.stdout: bluestore fsck on mount = True 2023-11-06T02:14:36.107 INFO:teuthology.orchestra.run.smithi049.stdout: debug bluefs = 1/20 2023-11-06T02:14:36.107 INFO:teuthology.orchestra.run.smithi049.stdout: debug bluestore = 1/20 2023-11-06T02:14:36.107 INFO:teuthology.orchestra.run.smithi049.stdout: debug ms = 1 2023-11-06T02:14:36.107 INFO:teuthology.orchestra.run.smithi049.stdout: debug osd = 20 2023-11-06T02:14:36.107 INFO:teuthology.orchestra.run.smithi049.stdout: debug rocksdb = 4/10 2023-11-06T02:14:36.108 INFO:teuthology.orchestra.run.smithi049.stdout: mon osd backfillfull_ratio = 0.85 2023-11-06T02:14:36.108 INFO:teuthology.orchestra.run.smithi049.stdout: mon osd full ratio = 0.9 2023-11-06T02:14:36.108 INFO:teuthology.orchestra.run.smithi049.stdout: mon osd nearfull ratio = 0.8 2023-11-06T02:14:36.108 INFO:teuthology.orchestra.run.smithi049.stdout: osd failsafe full ratio = 0.95 2023-11-06T02:14:36.108 INFO:teuthology.orchestra.run.smithi049.stdout: osd objectstore = bluestore 2023-11-06T02:14:36.108 INFO:teuthology.orchestra.run.smithi049.stdout: osd op complaint time = 180 2023-11-06T02:14:36.109 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:36.109 INFO:teuthology.orchestra.run.smithi049.stdout:[mgr] 2023-11-06T02:14:36.109 INFO:teuthology.orchestra.run.smithi049.stdout: debug ms = 1 2023-11-06T02:14:36.109 INFO:teuthology.orchestra.run.smithi049.stdout: debug mgr = 20 2023-11-06T02:14:36.109 INFO:teuthology.orchestra.run.smithi049.stdout: debug mon = 20 2023-11-06T02:14:36.110 INFO:teuthology.orchestra.run.smithi049.stdout: debug auth = 20 2023-11-06T02:14:36.110 INFO:teuthology.orchestra.run.smithi049.stdout: mon reweight min pgs per osd = 4 2023-11-06T02:14:36.110 INFO:teuthology.orchestra.run.smithi049.stdout: mon reweight min bytes per osd = 10 2023-11-06T02:14:36.110 INFO:teuthology.orchestra.run.smithi049.stdout: mgr/telemetry/nag = false 2023-11-06T02:14:36.110 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:36.111 INFO:teuthology.orchestra.run.smithi049.stdout:[mon] 2023-11-06T02:14:36.111 INFO:teuthology.orchestra.run.smithi049.stdout: debug ms = 1 2023-11-06T02:14:36.111 INFO:teuthology.orchestra.run.smithi049.stdout: debug mon = 20 2023-11-06T02:14:36.111 INFO:teuthology.orchestra.run.smithi049.stdout: debug paxos = 20 2023-11-06T02:14:36.111 INFO:teuthology.orchestra.run.smithi049.stdout: debug auth = 20 2023-11-06T02:14:36.112 INFO:teuthology.orchestra.run.smithi049.stdout: mon data avail warn = 5 2023-11-06T02:14:36.112 INFO:teuthology.orchestra.run.smithi049.stdout: mon mgr mkfs grace = 240 2023-11-06T02:14:36.112 INFO:teuthology.orchestra.run.smithi049.stdout: mon reweight min pgs per osd = 4 2023-11-06T02:14:36.112 INFO:teuthology.orchestra.run.smithi049.stdout: mon osd reporter subtree level = osd 2023-11-06T02:14:36.112 INFO:teuthology.orchestra.run.smithi049.stdout: mon osd prime pg temp = true 2023-11-06T02:14:36.112 INFO:teuthology.orchestra.run.smithi049.stdout: mon reweight min bytes per osd = 10 2023-11-06T02:14:36.113 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:36.113 INFO:teuthology.orchestra.run.smithi049.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-06T02:14:36.113 INFO:teuthology.orchestra.run.smithi049.stdout: auth mon ticket ttl = 660 # 11m 2023-11-06T02:14:36.113 INFO:teuthology.orchestra.run.smithi049.stdout: auth service ticket ttl = 240 # 4m 2023-11-06T02:14:36.113 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:36.114 INFO:teuthology.orchestra.run.smithi049.stdout: # don't complain about insecure global_id in the test suite 2023-11-06T02:14:36.114 INFO:teuthology.orchestra.run.smithi049.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-06T02:14:36.114 INFO:teuthology.orchestra.run.smithi049.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-06T02:14:36.114 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:36.114 INFO:teuthology.orchestra.run.smithi049.stdout: # 1m isn't quite enough 2023-11-06T02:14:36.114 INFO:teuthology.orchestra.run.smithi049.stdout: mon_down_mkfs_grace = 2m 2023-11-06T02:14:36.115 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:36.115 INFO:teuthology.orchestra.run.smithi049.stdout: mon_warn_on_filestore_osds = false 2023-11-06T02:14:36.115 INFO:teuthology.orchestra.run.smithi049.stdout: mon op complaint time = 120 2023-11-06T02:14:36.115 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:36.115 INFO:teuthology.orchestra.run.smithi049.stdout:[client] 2023-11-06T02:14:36.116 INFO:teuthology.orchestra.run.smithi049.stdout: rgw cache enabled = true 2023-11-06T02:14:36.116 INFO:teuthology.orchestra.run.smithi049.stdout: rgw enable ops log = true 2023-11-06T02:14:36.116 INFO:teuthology.orchestra.run.smithi049.stdout: rgw enable usage log = true 2023-11-06T02:14:36.116 INFO:teuthology.orchestra.run.smithi049.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-06T02:14:36.116 INFO:teuthology.orchestra.run.smithi049.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-06T02:14:36.117 INFO:teuthology.orchestra.run.smithi049.stdout: client mount timeout = 600 2023-11-06T02:14:36.117 INFO:teuthology.orchestra.run.smithi049.stdout: debug client = 20 2023-11-06T02:14:36.117 INFO:teuthology.orchestra.run.smithi049.stdout: debug ms = 1 2023-11-06T02:14:36.117 INFO:teuthology.orchestra.run.smithi049.stdout: pid file = "" 2023-11-06T02:14:36.117 INFO:teuthology.orchestra.run.smithi049.stdout: rados mon op timeout = 900 2023-11-06T02:14:36.117 INFO:teuthology.orchestra.run.smithi049.stdout: rados osd op timeout = 900 2023-11-06T02:14:36.118 INFO:teuthology.orchestra.run.smithi049.stdout:[mon.a] 2023-11-06T02:14:36.118 INFO:teuthology.orchestra.run.smithi049.stdout:[mon.b] 2023-11-06T02:14:36.118 INFO:teuthology.orchestra.run.smithi049.stdout:[mon.c] 2023-11-06T02:14:36.118 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.a] 2023-11-06T02:14:36.118 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.c] 2023-11-06T02:14:36.119 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.b] 2023-11-06T02:14:36.119 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.d] 2023-11-06T02:14:36.119 INFO:teuthology.orchestra.run.smithi049.stdout:[mds] 2023-11-06T02:14:36.119 INFO:teuthology.orchestra.run.smithi049.stdout: debug mds = 20 2023-11-06T02:14:36.120 INFO:teuthology.orchestra.run.smithi049.stdout: debug mds balancer = 20 2023-11-06T02:14:36.120 INFO:teuthology.orchestra.run.smithi049.stdout: debug ms = 1 2023-11-06T02:14:36.120 INFO:teuthology.orchestra.run.smithi049.stdout: mds debug frag = True 2023-11-06T02:14:36.120 INFO:teuthology.orchestra.run.smithi049.stdout: mds debug scatterstat = False 2023-11-06T02:14:36.120 INFO:teuthology.orchestra.run.smithi049.stdout: mds op complaint time = 180 2023-11-06T02:14:36.121 INFO:teuthology.orchestra.run.smithi049.stdout: mds verify scatter = True 2023-11-06T02:14:36.121 INFO:teuthology.orchestra.run.smithi049.stdout: osd op complaint time = 180 2023-11-06T02:14:36.121 INFO:teuthology.orchestra.run.smithi049.stdout: rados mon op timeout = 900 2023-11-06T02:14:36.121 INFO:teuthology.orchestra.run.smithi049.stdout: rados osd op timeout = 900 2023-11-06T02:14:36.123 INFO:teuthology.orchestra.run.smithi150.stdout:[global] 2023-11-06T02:14:36.123 INFO:teuthology.orchestra.run.smithi150.stdout: chdir = "" 2023-11-06T02:14:36.123 INFO:teuthology.orchestra.run.smithi150.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-06T02:14:36.123 INFO:teuthology.orchestra.run.smithi150.stdout: auth supported = cephx 2023-11-06T02:14:36.124 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:36.124 INFO:teuthology.orchestra.run.smithi150.stdout: filestore xattr use omap = true 2023-11-06T02:14:36.124 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:36.124 INFO:teuthology.orchestra.run.smithi150.stdout: mon clock drift allowed = 1.000 2023-11-06T02:14:36.124 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:36.125 INFO:teuthology.orchestra.run.smithi150.stdout: osd crush chooseleaf type = 0 2023-11-06T02:14:36.125 INFO:teuthology.orchestra.run.smithi150.stdout: auth debug = true 2023-11-06T02:14:36.125 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:36.125 INFO:teuthology.orchestra.run.smithi150.stdout: ms die on old message = true 2023-11-06T02:14:36.125 INFO:teuthology.orchestra.run.smithi150.stdout: ms die on bug = true 2023-11-06T02:14:36.126 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:36.126 INFO:teuthology.orchestra.run.smithi150.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-06T02:14:36.126 INFO:teuthology.orchestra.run.smithi150.stdout: mon pg warn max object skew = 0 2023-11-06T02:14:36.126 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:36.126 INFO:teuthology.orchestra.run.smithi150.stdout: # disable pg_autoscaler by default for new pools 2023-11-06T02:14:36.127 INFO:teuthology.orchestra.run.smithi150.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-06T02:14:36.127 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:36.127 INFO:teuthology.orchestra.run.smithi150.stdout: osd pool default size = 2 2023-11-06T02:14:36.127 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:36.127 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd allow primary affinity = true 2023-11-06T02:14:36.127 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd allow pg remap = true 2023-11-06T02:14:36.128 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on legacy crush tunables = false 2023-11-06T02:14:36.128 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on crush straw calc version zero = false 2023-11-06T02:14:36.128 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on no sortbitwise = false 2023-11-06T02:14:36.128 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on osd down out interval zero = false 2023-11-06T02:14:36.128 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on too few osds = false 2023-11-06T02:14:36.129 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-06T02:14:36.129 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-06T02:14:36.129 INFO:teuthology.orchestra.run.smithi150.stdout: mon_allow_pool_size_one = true 2023-11-06T02:14:36.129 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:36.129 INFO:teuthology.orchestra.run.smithi150.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-06T02:14:36.129 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:36.130 INFO:teuthology.orchestra.run.smithi150.stdout: osd default data pool replay window = 5 2023-11-06T02:14:36.130 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:36.130 INFO:teuthology.orchestra.run.smithi150.stdout: mon allow pool delete = true 2023-11-06T02:14:36.130 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:36.130 INFO:teuthology.orchestra.run.smithi150.stdout: mon cluster log file level = debug 2023-11-06T02:14:36.131 INFO:teuthology.orchestra.run.smithi150.stdout: debug asserts on shutdown = true 2023-11-06T02:14:36.131 INFO:teuthology.orchestra.run.smithi150.stdout: mon health detail to clog = false 2023-11-06T02:14:36.131 INFO:teuthology.orchestra.run.smithi150.stdout: mon host = "172.21.15.49,172.21.15.150,[v2:172.21.15.150:3301,v1:172.21.15.150:6790]" 2023-11-06T02:14:36.131 INFO:teuthology.orchestra.run.smithi150.stdout: fsid = 1a8b035c-6ba4-499a-b876-aea47fd9c0d4 2023-11-06T02:14:36.131 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:36.132 INFO:teuthology.orchestra.run.smithi150.stdout:[osd] 2023-11-06T02:14:36.132 INFO:teuthology.orchestra.run.smithi150.stdout: osd journal size = 100 2023-11-06T02:14:36.132 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:36.132 INFO:teuthology.orchestra.run.smithi150.stdout: osd scrub load threshold = 5.0 2023-11-06T02:14:36.132 INFO:teuthology.orchestra.run.smithi150.stdout: osd scrub max interval = 600 2023-11-06T02:14:36.132 INFO:teuthology.orchestra.run.smithi150.stdout: osd mclock profile = high_recovery_ops 2023-11-06T02:14:36.133 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:36.133 INFO:teuthology.orchestra.run.smithi150.stdout: osd recover clone overlap = true 2023-11-06T02:14:36.133 INFO:teuthology.orchestra.run.smithi150.stdout: osd recovery max chunk = 1048576 2023-11-06T02:14:36.133 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:36.133 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug shutdown = true 2023-11-06T02:14:36.134 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug op order = true 2023-11-06T02:14:36.134 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug verify stray on activate = true 2023-11-06T02:14:36.134 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:36.134 INFO:teuthology.orchestra.run.smithi150.stdout: osd open classes on start = true 2023-11-06T02:14:36.134 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug pg log writeout = true 2023-11-06T02:14:36.135 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:36.135 INFO:teuthology.orchestra.run.smithi150.stdout: osd deep scrub update digest min age = 30 2023-11-06T02:14:36.135 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:36.135 INFO:teuthology.orchestra.run.smithi150.stdout: osd map max advance = 10 2023-11-06T02:14:36.136 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:36.136 INFO:teuthology.orchestra.run.smithi150.stdout: journal zero on create = true 2023-11-06T02:14:36.136 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:36.136 INFO:teuthology.orchestra.run.smithi150.stdout: filestore ondisk finisher threads = 3 2023-11-06T02:14:36.136 INFO:teuthology.orchestra.run.smithi150.stdout: filestore apply finisher threads = 3 2023-11-06T02:14:36.137 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:36.137 INFO:teuthology.orchestra.run.smithi150.stdout: bdev debug aio = true 2023-11-06T02:14:36.137 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug misdirected ops = true 2023-11-06T02:14:36.137 INFO:teuthology.orchestra.run.smithi150.stdout: bdev async discard = True 2023-11-06T02:14:36.137 INFO:teuthology.orchestra.run.smithi150.stdout: bdev enable discard = True 2023-11-06T02:14:36.137 INFO:teuthology.orchestra.run.smithi150.stdout: bluestore allocator = bitmap 2023-11-06T02:14:36.138 INFO:teuthology.orchestra.run.smithi150.stdout: bluestore block size = 96636764160 2023-11-06T02:14:36.138 INFO:teuthology.orchestra.run.smithi150.stdout: bluestore fsck on mount = True 2023-11-06T02:14:36.138 INFO:teuthology.orchestra.run.smithi150.stdout: debug bluefs = 1/20 2023-11-06T02:14:36.138 INFO:teuthology.orchestra.run.smithi150.stdout: debug bluestore = 1/20 2023-11-06T02:14:36.138 INFO:teuthology.orchestra.run.smithi150.stdout: debug ms = 1 2023-11-06T02:14:36.139 INFO:teuthology.orchestra.run.smithi150.stdout: debug osd = 20 2023-11-06T02:14:36.139 INFO:teuthology.orchestra.run.smithi150.stdout: debug rocksdb = 4/10 2023-11-06T02:14:36.139 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd backfillfull_ratio = 0.85 2023-11-06T02:14:36.139 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd full ratio = 0.9 2023-11-06T02:14:36.139 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd nearfull ratio = 0.8 2023-11-06T02:14:36.140 INFO:teuthology.orchestra.run.smithi150.stdout: osd failsafe full ratio = 0.95 2023-11-06T02:14:36.140 INFO:teuthology.orchestra.run.smithi150.stdout: osd objectstore = bluestore 2023-11-06T02:14:36.140 INFO:teuthology.orchestra.run.smithi150.stdout: osd op complaint time = 180 2023-11-06T02:14:36.140 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:36.140 INFO:teuthology.orchestra.run.smithi150.stdout:[mgr] 2023-11-06T02:14:36.140 INFO:teuthology.orchestra.run.smithi150.stdout: debug ms = 1 2023-11-06T02:14:36.141 INFO:teuthology.orchestra.run.smithi150.stdout: debug mgr = 20 2023-11-06T02:14:36.141 INFO:teuthology.orchestra.run.smithi150.stdout: debug mon = 20 2023-11-06T02:14:36.141 INFO:teuthology.orchestra.run.smithi150.stdout: debug auth = 20 2023-11-06T02:14:36.141 INFO:teuthology.orchestra.run.smithi150.stdout: mon reweight min pgs per osd = 4 2023-11-06T02:14:36.141 INFO:teuthology.orchestra.run.smithi150.stdout: mon reweight min bytes per osd = 10 2023-11-06T02:14:36.142 INFO:teuthology.orchestra.run.smithi150.stdout: mgr/telemetry/nag = false 2023-11-06T02:14:36.142 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:36.142 INFO:teuthology.orchestra.run.smithi150.stdout:[mon] 2023-11-06T02:14:36.142 INFO:teuthology.orchestra.run.smithi150.stdout: debug ms = 1 2023-11-06T02:14:36.142 INFO:teuthology.orchestra.run.smithi150.stdout: debug mon = 20 2023-11-06T02:14:36.143 INFO:teuthology.orchestra.run.smithi150.stdout: debug paxos = 20 2023-11-06T02:14:36.143 INFO:teuthology.orchestra.run.smithi150.stdout: debug auth = 20 2023-11-06T02:14:36.143 INFO:teuthology.orchestra.run.smithi150.stdout: mon data avail warn = 5 2023-11-06T02:14:36.143 INFO:teuthology.orchestra.run.smithi150.stdout: mon mgr mkfs grace = 240 2023-11-06T02:14:36.143 INFO:teuthology.orchestra.run.smithi150.stdout: mon reweight min pgs per osd = 4 2023-11-06T02:14:36.143 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd reporter subtree level = osd 2023-11-06T02:14:36.144 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd prime pg temp = true 2023-11-06T02:14:36.144 INFO:teuthology.orchestra.run.smithi150.stdout: mon reweight min bytes per osd = 10 2023-11-06T02:14:36.144 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:36.144 INFO:teuthology.orchestra.run.smithi150.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-06T02:14:36.144 INFO:teuthology.orchestra.run.smithi150.stdout: auth mon ticket ttl = 660 # 11m 2023-11-06T02:14:36.145 INFO:teuthology.orchestra.run.smithi150.stdout: auth service ticket ttl = 240 # 4m 2023-11-06T02:14:36.145 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:36.145 INFO:teuthology.orchestra.run.smithi150.stdout: # don't complain about insecure global_id in the test suite 2023-11-06T02:14:36.145 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-06T02:14:36.145 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-06T02:14:36.146 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:36.146 INFO:teuthology.orchestra.run.smithi150.stdout: # 1m isn't quite enough 2023-11-06T02:14:36.146 INFO:teuthology.orchestra.run.smithi150.stdout: mon_down_mkfs_grace = 2m 2023-11-06T02:14:36.146 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:36.146 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_filestore_osds = false 2023-11-06T02:14:36.147 INFO:teuthology.orchestra.run.smithi150.stdout: mon op complaint time = 120 2023-11-06T02:14:36.147 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:36.147 INFO:teuthology.orchestra.run.smithi150.stdout:[client] 2023-11-06T02:14:36.147 INFO:teuthology.orchestra.run.smithi150.stdout: rgw cache enabled = true 2023-11-06T02:14:36.147 INFO:teuthology.orchestra.run.smithi150.stdout: rgw enable ops log = true 2023-11-06T02:14:36.148 INFO:teuthology.orchestra.run.smithi150.stdout: rgw enable usage log = true 2023-11-06T02:14:36.148 INFO:teuthology.orchestra.run.smithi150.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-06T02:14:36.148 INFO:teuthology.orchestra.run.smithi150.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-06T02:14:36.148 INFO:teuthology.orchestra.run.smithi150.stdout: client mount timeout = 600 2023-11-06T02:14:36.148 INFO:teuthology.orchestra.run.smithi150.stdout: debug client = 20 2023-11-06T02:14:36.149 INFO:teuthology.orchestra.run.smithi150.stdout: debug ms = 1 2023-11-06T02:14:36.149 INFO:teuthology.orchestra.run.smithi150.stdout: pid file = "" 2023-11-06T02:14:36.149 INFO:teuthology.orchestra.run.smithi150.stdout: rados mon op timeout = 900 2023-11-06T02:14:36.149 INFO:teuthology.orchestra.run.smithi150.stdout: rados osd op timeout = 900 2023-11-06T02:14:36.149 INFO:teuthology.orchestra.run.smithi150.stdout:[mon.a] 2023-11-06T02:14:36.150 INFO:teuthology.orchestra.run.smithi150.stdout:[mon.b] 2023-11-06T02:14:36.150 INFO:teuthology.orchestra.run.smithi150.stdout:[mon.c] 2023-11-06T02:14:36.150 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.a] 2023-11-06T02:14:36.150 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.c] 2023-11-06T02:14:36.150 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.b] 2023-11-06T02:14:36.151 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.d] 2023-11-06T02:14:36.151 INFO:teuthology.orchestra.run.smithi150.stdout:[mds] 2023-11-06T02:14:36.151 INFO:teuthology.orchestra.run.smithi150.stdout: debug mds = 20 2023-11-06T02:14:36.151 INFO:teuthology.orchestra.run.smithi150.stdout: debug mds balancer = 20 2023-11-06T02:14:36.151 INFO:teuthology.orchestra.run.smithi150.stdout: debug ms = 1 2023-11-06T02:14:36.151 INFO:teuthology.orchestra.run.smithi150.stdout: mds debug frag = True 2023-11-06T02:14:36.152 INFO:teuthology.orchestra.run.smithi150.stdout: mds debug scatterstat = False 2023-11-06T02:14:36.152 INFO:teuthology.orchestra.run.smithi150.stdout: mds op complaint time = 180 2023-11-06T02:14:36.152 INFO:teuthology.orchestra.run.smithi150.stdout: mds verify scatter = True 2023-11-06T02:14:36.152 INFO:teuthology.orchestra.run.smithi150.stdout: osd op complaint time = 180 2023-11-06T02:14:36.152 INFO:teuthology.orchestra.run.smithi150.stdout: rados mon op timeout = 900 2023-11-06T02:14:36.153 INFO:teuthology.orchestra.run.smithi150.stdout: rados osd op timeout = 900 2023-11-06T02:14:36.154 DEBUG:teuthology.orchestra.run.smithi049:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-06T02:14:36.169 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-06T02:14:36.195 INFO:teuthology.orchestra.run.smithi049.stdout:[global] 2023-11-06T02:14:36.196 INFO:teuthology.orchestra.run.smithi049.stdout: chdir = "" 2023-11-06T02:14:36.196 INFO:teuthology.orchestra.run.smithi049.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-06T02:14:36.196 INFO:teuthology.orchestra.run.smithi049.stdout: auth supported = cephx 2023-11-06T02:14:36.196 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:36.196 INFO:teuthology.orchestra.run.smithi049.stdout: filestore xattr use omap = true 2023-11-06T02:14:36.197 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:36.197 INFO:teuthology.orchestra.run.smithi049.stdout: mon clock drift allowed = 1.000 2023-11-06T02:14:36.197 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:36.197 INFO:teuthology.orchestra.run.smithi049.stdout: osd crush chooseleaf type = 0 2023-11-06T02:14:36.197 INFO:teuthology.orchestra.run.smithi049.stdout: auth debug = true 2023-11-06T02:14:36.198 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:36.198 INFO:teuthology.orchestra.run.smithi049.stdout: ms die on old message = true 2023-11-06T02:14:36.198 INFO:teuthology.orchestra.run.smithi049.stdout: ms die on bug = true 2023-11-06T02:14:36.198 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:36.198 INFO:teuthology.orchestra.run.smithi049.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-06T02:14:36.199 INFO:teuthology.orchestra.run.smithi049.stdout: mon pg warn max object skew = 0 2023-11-06T02:14:36.199 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:36.199 INFO:teuthology.orchestra.run.smithi049.stdout: # disable pg_autoscaler by default for new pools 2023-11-06T02:14:36.199 INFO:teuthology.orchestra.run.smithi049.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-06T02:14:36.199 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:36.200 INFO:teuthology.orchestra.run.smithi049.stdout: osd pool default size = 2 2023-11-06T02:14:36.200 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:36.200 INFO:teuthology.orchestra.run.smithi049.stdout: mon osd allow primary affinity = true 2023-11-06T02:14:36.200 INFO:teuthology.orchestra.run.smithi049.stdout: mon osd allow pg remap = true 2023-11-06T02:14:36.200 INFO:teuthology.orchestra.run.smithi049.stdout: mon warn on legacy crush tunables = false 2023-11-06T02:14:36.200 INFO:teuthology.orchestra.run.smithi049.stdout: mon warn on crush straw calc version zero = false 2023-11-06T02:14:36.201 INFO:teuthology.orchestra.run.smithi049.stdout: mon warn on no sortbitwise = false 2023-11-06T02:14:36.201 INFO:teuthology.orchestra.run.smithi049.stdout: mon warn on osd down out interval zero = false 2023-11-06T02:14:36.201 INFO:teuthology.orchestra.run.smithi049.stdout: mon warn on too few osds = false 2023-11-06T02:14:36.201 INFO:teuthology.orchestra.run.smithi049.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-06T02:14:36.201 INFO:teuthology.orchestra.run.smithi049.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-06T02:14:36.202 INFO:teuthology.orchestra.run.smithi049.stdout: mon_allow_pool_size_one = true 2023-11-06T02:14:36.202 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:36.202 INFO:teuthology.orchestra.run.smithi049.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-06T02:14:36.202 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:36.202 INFO:teuthology.orchestra.run.smithi049.stdout: osd default data pool replay window = 5 2023-11-06T02:14:36.203 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:36.203 INFO:teuthology.orchestra.run.smithi049.stdout: mon allow pool delete = true 2023-11-06T02:14:36.203 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:36.203 INFO:teuthology.orchestra.run.smithi049.stdout: mon cluster log file level = debug 2023-11-06T02:14:36.203 INFO:teuthology.orchestra.run.smithi049.stdout: debug asserts on shutdown = true 2023-11-06T02:14:36.203 INFO:teuthology.orchestra.run.smithi049.stdout: mon health detail to clog = false 2023-11-06T02:14:36.204 INFO:teuthology.orchestra.run.smithi049.stdout: mon host = "172.21.15.49,172.21.15.150,[v2:172.21.15.150:3301,v1:172.21.15.150:6790]" 2023-11-06T02:14:36.204 INFO:teuthology.orchestra.run.smithi049.stdout: fsid = 1a8b035c-6ba4-499a-b876-aea47fd9c0d4 2023-11-06T02:14:36.204 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:36.204 INFO:teuthology.orchestra.run.smithi049.stdout:[osd] 2023-11-06T02:14:36.204 INFO:teuthology.orchestra.run.smithi049.stdout: osd journal size = 100 2023-11-06T02:14:36.205 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:36.205 INFO:teuthology.orchestra.run.smithi049.stdout: osd scrub load threshold = 5.0 2023-11-06T02:14:36.205 INFO:teuthology.orchestra.run.smithi049.stdout: osd scrub max interval = 600 2023-11-06T02:14:36.205 INFO:teuthology.orchestra.run.smithi049.stdout: osd mclock profile = high_recovery_ops 2023-11-06T02:14:36.205 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:36.206 INFO:teuthology.orchestra.run.smithi049.stdout: osd recover clone overlap = true 2023-11-06T02:14:36.206 INFO:teuthology.orchestra.run.smithi049.stdout: osd recovery max chunk = 1048576 2023-11-06T02:14:36.206 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:36.206 INFO:teuthology.orchestra.run.smithi049.stdout: osd debug shutdown = true 2023-11-06T02:14:36.206 INFO:teuthology.orchestra.run.smithi049.stdout: osd debug op order = true 2023-11-06T02:14:36.206 INFO:teuthology.orchestra.run.smithi049.stdout: osd debug verify stray on activate = true 2023-11-06T02:14:36.207 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:36.207 INFO:teuthology.orchestra.run.smithi049.stdout: osd open classes on start = true 2023-11-06T02:14:36.207 INFO:teuthology.orchestra.run.smithi049.stdout: osd debug pg log writeout = true 2023-11-06T02:14:36.207 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:36.207 INFO:teuthology.orchestra.run.smithi049.stdout: osd deep scrub update digest min age = 30 2023-11-06T02:14:36.208 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:36.208 INFO:teuthology.orchestra.run.smithi049.stdout: osd map max advance = 10 2023-11-06T02:14:36.208 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:36.208 INFO:teuthology.orchestra.run.smithi049.stdout: journal zero on create = true 2023-11-06T02:14:36.209 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:36.209 INFO:teuthology.orchestra.run.smithi049.stdout: filestore ondisk finisher threads = 3 2023-11-06T02:14:36.209 INFO:teuthology.orchestra.run.smithi049.stdout: filestore apply finisher threads = 3 2023-11-06T02:14:36.209 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:36.209 INFO:teuthology.orchestra.run.smithi049.stdout: bdev debug aio = true 2023-11-06T02:14:36.209 INFO:teuthology.orchestra.run.smithi049.stdout: osd debug misdirected ops = true 2023-11-06T02:14:36.210 INFO:teuthology.orchestra.run.smithi049.stdout: bdev async discard = True 2023-11-06T02:14:36.210 INFO:teuthology.orchestra.run.smithi049.stdout: bdev enable discard = True 2023-11-06T02:14:36.210 INFO:teuthology.orchestra.run.smithi049.stdout: bluestore allocator = bitmap 2023-11-06T02:14:36.210 INFO:teuthology.orchestra.run.smithi049.stdout: bluestore block size = 96636764160 2023-11-06T02:14:36.211 INFO:teuthology.orchestra.run.smithi049.stdout: bluestore fsck on mount = True 2023-11-06T02:14:36.211 INFO:teuthology.orchestra.run.smithi049.stdout: debug bluefs = 1/20 2023-11-06T02:14:36.211 INFO:teuthology.orchestra.run.smithi049.stdout: debug bluestore = 1/20 2023-11-06T02:14:36.211 INFO:teuthology.orchestra.run.smithi049.stdout: debug ms = 1 2023-11-06T02:14:36.211 INFO:teuthology.orchestra.run.smithi049.stdout: debug osd = 20 2023-11-06T02:14:36.212 INFO:teuthology.orchestra.run.smithi049.stdout: debug rocksdb = 4/10 2023-11-06T02:14:36.212 INFO:teuthology.orchestra.run.smithi049.stdout: mon osd backfillfull_ratio = 0.85 2023-11-06T02:14:36.212 INFO:teuthology.orchestra.run.smithi049.stdout: mon osd full ratio = 0.9 2023-11-06T02:14:36.212 INFO:teuthology.orchestra.run.smithi049.stdout: mon osd nearfull ratio = 0.8 2023-11-06T02:14:36.212 INFO:teuthology.orchestra.run.smithi049.stdout: osd failsafe full ratio = 0.95 2023-11-06T02:14:36.213 INFO:teuthology.orchestra.run.smithi049.stdout: osd objectstore = bluestore 2023-11-06T02:14:36.213 INFO:teuthology.orchestra.run.smithi049.stdout: osd op complaint time = 180 2023-11-06T02:14:36.213 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:36.213 INFO:teuthology.orchestra.run.smithi049.stdout:[mgr] 2023-11-06T02:14:36.213 INFO:teuthology.orchestra.run.smithi049.stdout: debug ms = 1 2023-11-06T02:14:36.213 INFO:teuthology.orchestra.run.smithi049.stdout: debug mgr = 20 2023-11-06T02:14:36.214 INFO:teuthology.orchestra.run.smithi049.stdout: debug mon = 20 2023-11-06T02:14:36.214 INFO:teuthology.orchestra.run.smithi049.stdout: debug auth = 20 2023-11-06T02:14:36.214 INFO:teuthology.orchestra.run.smithi049.stdout: mon reweight min pgs per osd = 4 2023-11-06T02:14:36.215 INFO:teuthology.orchestra.run.smithi049.stdout: mon reweight min bytes per osd = 10 2023-11-06T02:14:36.215 INFO:teuthology.orchestra.run.smithi049.stdout: mgr/telemetry/nag = false 2023-11-06T02:14:36.219 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:36.220 INFO:teuthology.orchestra.run.smithi049.stdout:[mon] 2023-11-06T02:14:36.220 INFO:teuthology.orchestra.run.smithi049.stdout: debug ms = 1 2023-11-06T02:14:36.220 INFO:teuthology.orchestra.run.smithi049.stdout: debug mon = 20 2023-11-06T02:14:36.220 INFO:teuthology.orchestra.run.smithi049.stdout: debug paxos = 20 2023-11-06T02:14:36.220 INFO:teuthology.orchestra.run.smithi049.stdout: debug auth = 20 2023-11-06T02:14:36.221 INFO:teuthology.orchestra.run.smithi049.stdout: mon data avail warn = 5 2023-11-06T02:14:36.221 INFO:teuthology.orchestra.run.smithi049.stdout: mon mgr mkfs grace = 240 2023-11-06T02:14:36.221 INFO:teuthology.orchestra.run.smithi049.stdout: mon reweight min pgs per osd = 4 2023-11-06T02:14:36.221 INFO:teuthology.orchestra.run.smithi049.stdout: mon osd reporter subtree level = osd 2023-11-06T02:14:36.221 INFO:teuthology.orchestra.run.smithi049.stdout: mon osd prime pg temp = true 2023-11-06T02:14:36.222 INFO:teuthology.orchestra.run.smithi049.stdout: mon reweight min bytes per osd = 10 2023-11-06T02:14:36.222 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:36.222 INFO:teuthology.orchestra.run.smithi049.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-06T02:14:36.222 INFO:teuthology.orchestra.run.smithi049.stdout: auth mon ticket ttl = 660 # 11m 2023-11-06T02:14:36.222 INFO:teuthology.orchestra.run.smithi049.stdout: auth service ticket ttl = 240 # 4m 2023-11-06T02:14:36.223 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:36.223 INFO:teuthology.orchestra.run.smithi049.stdout: # don't complain about insecure global_id in the test suite 2023-11-06T02:14:36.223 INFO:teuthology.orchestra.run.smithi049.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-06T02:14:36.223 INFO:teuthology.orchestra.run.smithi049.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-06T02:14:36.223 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:36.224 INFO:teuthology.orchestra.run.smithi049.stdout: # 1m isn't quite enough 2023-11-06T02:14:36.224 INFO:teuthology.orchestra.run.smithi049.stdout: mon_down_mkfs_grace = 2m 2023-11-06T02:14:36.224 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:36.224 INFO:teuthology.orchestra.run.smithi049.stdout: mon_warn_on_filestore_osds = false 2023-11-06T02:14:36.224 INFO:teuthology.orchestra.run.smithi049.stdout: mon op complaint time = 120 2023-11-06T02:14:36.224 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:36.225 INFO:teuthology.orchestra.run.smithi049.stdout:[client] 2023-11-06T02:14:36.225 INFO:teuthology.orchestra.run.smithi049.stdout: rgw cache enabled = true 2023-11-06T02:14:36.225 INFO:teuthology.orchestra.run.smithi049.stdout: rgw enable ops log = true 2023-11-06T02:14:36.225 INFO:teuthology.orchestra.run.smithi049.stdout: rgw enable usage log = true 2023-11-06T02:14:36.225 INFO:teuthology.orchestra.run.smithi049.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-06T02:14:36.226 INFO:teuthology.orchestra.run.smithi049.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-06T02:14:36.226 INFO:teuthology.orchestra.run.smithi049.stdout: client mount timeout = 600 2023-11-06T02:14:36.226 INFO:teuthology.orchestra.run.smithi049.stdout: debug client = 20 2023-11-06T02:14:36.226 INFO:teuthology.orchestra.run.smithi049.stdout: debug ms = 1 2023-11-06T02:14:36.226 INFO:teuthology.orchestra.run.smithi049.stdout: pid file = "" 2023-11-06T02:14:36.227 INFO:teuthology.orchestra.run.smithi049.stdout: rados mon op timeout = 900 2023-11-06T02:14:36.227 INFO:teuthology.orchestra.run.smithi049.stdout: rados osd op timeout = 900 2023-11-06T02:14:36.227 INFO:teuthology.orchestra.run.smithi049.stdout:[mon.a] 2023-11-06T02:14:36.227 INFO:teuthology.orchestra.run.smithi049.stdout:[mon.b] 2023-11-06T02:14:36.227 INFO:teuthology.orchestra.run.smithi049.stdout:[mon.c] 2023-11-06T02:14:36.227 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.a] 2023-11-06T02:14:36.228 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.c] 2023-11-06T02:14:36.228 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.b] 2023-11-06T02:14:36.228 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.d] 2023-11-06T02:14:36.228 INFO:teuthology.orchestra.run.smithi049.stdout:[mds] 2023-11-06T02:14:36.228 INFO:teuthology.orchestra.run.smithi049.stdout: debug mds = 20 2023-11-06T02:14:36.229 INFO:teuthology.orchestra.run.smithi049.stdout: debug mds balancer = 20 2023-11-06T02:14:36.229 INFO:teuthology.orchestra.run.smithi049.stdout: debug ms = 1 2023-11-06T02:14:36.229 INFO:teuthology.orchestra.run.smithi049.stdout: mds debug frag = True 2023-11-06T02:14:36.229 INFO:teuthology.orchestra.run.smithi049.stdout: mds debug scatterstat = True 2023-11-06T02:14:36.229 INFO:teuthology.orchestra.run.smithi049.stdout: mds op complaint time = 180 2023-11-06T02:14:36.229 INFO:teuthology.orchestra.run.smithi049.stdout: mds verify scatter = True 2023-11-06T02:14:36.230 INFO:teuthology.orchestra.run.smithi049.stdout: osd op complaint time = 180 2023-11-06T02:14:36.230 INFO:teuthology.orchestra.run.smithi049.stdout: rados mon op timeout = 900 2023-11-06T02:14:36.230 INFO:teuthology.orchestra.run.smithi049.stdout: rados osd op timeout = 900 2023-11-06T02:14:36.232 INFO:teuthology.orchestra.run.smithi150.stdout:[global] 2023-11-06T02:14:36.232 INFO:teuthology.orchestra.run.smithi150.stdout: chdir = "" 2023-11-06T02:14:36.232 INFO:teuthology.orchestra.run.smithi150.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-06T02:14:36.232 INFO:teuthology.orchestra.run.smithi150.stdout: auth supported = cephx 2023-11-06T02:14:36.233 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:36.233 INFO:teuthology.orchestra.run.smithi150.stdout: filestore xattr use omap = true 2023-11-06T02:14:36.233 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:36.233 INFO:teuthology.orchestra.run.smithi150.stdout: mon clock drift allowed = 1.000 2023-11-06T02:14:36.233 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:36.233 INFO:teuthology.orchestra.run.smithi150.stdout: osd crush chooseleaf type = 0 2023-11-06T02:14:36.234 INFO:teuthology.orchestra.run.smithi150.stdout: auth debug = true 2023-11-06T02:14:36.234 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:36.234 INFO:teuthology.orchestra.run.smithi150.stdout: ms die on old message = true 2023-11-06T02:14:36.234 INFO:teuthology.orchestra.run.smithi150.stdout: ms die on bug = true 2023-11-06T02:14:36.234 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:36.235 INFO:teuthology.orchestra.run.smithi150.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-06T02:14:36.235 INFO:teuthology.orchestra.run.smithi150.stdout: mon pg warn max object skew = 0 2023-11-06T02:14:36.235 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:36.235 INFO:teuthology.orchestra.run.smithi150.stdout: # disable pg_autoscaler by default for new pools 2023-11-06T02:14:36.235 INFO:teuthology.orchestra.run.smithi150.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-06T02:14:36.236 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:36.236 INFO:teuthology.orchestra.run.smithi150.stdout: osd pool default size = 2 2023-11-06T02:14:36.236 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:36.236 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd allow primary affinity = true 2023-11-06T02:14:36.236 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd allow pg remap = true 2023-11-06T02:14:36.237 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on legacy crush tunables = false 2023-11-06T02:14:36.237 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on crush straw calc version zero = false 2023-11-06T02:14:36.237 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on no sortbitwise = false 2023-11-06T02:14:36.237 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on osd down out interval zero = false 2023-11-06T02:14:36.237 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on too few osds = false 2023-11-06T02:14:36.237 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-06T02:14:36.238 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-06T02:14:36.238 INFO:teuthology.orchestra.run.smithi150.stdout: mon_allow_pool_size_one = true 2023-11-06T02:14:36.238 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:36.238 INFO:teuthology.orchestra.run.smithi150.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-06T02:14:36.238 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:36.239 INFO:teuthology.orchestra.run.smithi150.stdout: osd default data pool replay window = 5 2023-11-06T02:14:36.239 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:36.239 INFO:teuthology.orchestra.run.smithi150.stdout: mon allow pool delete = true 2023-11-06T02:14:36.239 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:36.239 INFO:teuthology.orchestra.run.smithi150.stdout: mon cluster log file level = debug 2023-11-06T02:14:36.240 INFO:teuthology.orchestra.run.smithi150.stdout: debug asserts on shutdown = true 2023-11-06T02:14:36.240 INFO:teuthology.orchestra.run.smithi150.stdout: mon health detail to clog = false 2023-11-06T02:14:36.240 INFO:teuthology.orchestra.run.smithi150.stdout: mon host = "172.21.15.49,172.21.15.150,[v2:172.21.15.150:3301,v1:172.21.15.150:6790]" 2023-11-06T02:14:36.240 INFO:teuthology.orchestra.run.smithi150.stdout: fsid = 1a8b035c-6ba4-499a-b876-aea47fd9c0d4 2023-11-06T02:14:36.240 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:36.240 INFO:teuthology.orchestra.run.smithi150.stdout:[osd] 2023-11-06T02:14:36.241 INFO:teuthology.orchestra.run.smithi150.stdout: osd journal size = 100 2023-11-06T02:14:36.241 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:36.241 INFO:teuthology.orchestra.run.smithi150.stdout: osd scrub load threshold = 5.0 2023-11-06T02:14:36.241 INFO:teuthology.orchestra.run.smithi150.stdout: osd scrub max interval = 600 2023-11-06T02:14:36.241 INFO:teuthology.orchestra.run.smithi150.stdout: osd mclock profile = high_recovery_ops 2023-11-06T02:14:36.242 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:36.242 INFO:teuthology.orchestra.run.smithi150.stdout: osd recover clone overlap = true 2023-11-06T02:14:36.242 INFO:teuthology.orchestra.run.smithi150.stdout: osd recovery max chunk = 1048576 2023-11-06T02:14:36.242 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:36.242 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug shutdown = true 2023-11-06T02:14:36.243 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug op order = true 2023-11-06T02:14:36.243 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug verify stray on activate = true 2023-11-06T02:14:36.243 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:36.243 INFO:teuthology.orchestra.run.smithi150.stdout: osd open classes on start = true 2023-11-06T02:14:36.243 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug pg log writeout = true 2023-11-06T02:14:36.243 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:36.244 INFO:teuthology.orchestra.run.smithi150.stdout: osd deep scrub update digest min age = 30 2023-11-06T02:14:36.244 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:36.244 INFO:teuthology.orchestra.run.smithi150.stdout: osd map max advance = 10 2023-11-06T02:14:36.244 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:36.244 INFO:teuthology.orchestra.run.smithi150.stdout: journal zero on create = true 2023-11-06T02:14:36.245 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:36.245 INFO:teuthology.orchestra.run.smithi150.stdout: filestore ondisk finisher threads = 3 2023-11-06T02:14:36.245 INFO:teuthology.orchestra.run.smithi150.stdout: filestore apply finisher threads = 3 2023-11-06T02:14:36.245 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:36.245 INFO:teuthology.orchestra.run.smithi150.stdout: bdev debug aio = true 2023-11-06T02:14:36.245 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug misdirected ops = true 2023-11-06T02:14:36.246 INFO:teuthology.orchestra.run.smithi150.stdout: bdev async discard = True 2023-11-06T02:14:36.246 INFO:teuthology.orchestra.run.smithi150.stdout: bdev enable discard = True 2023-11-06T02:14:36.246 INFO:teuthology.orchestra.run.smithi150.stdout: bluestore allocator = bitmap 2023-11-06T02:14:36.246 INFO:teuthology.orchestra.run.smithi150.stdout: bluestore block size = 96636764160 2023-11-06T02:14:36.246 INFO:teuthology.orchestra.run.smithi150.stdout: bluestore fsck on mount = True 2023-11-06T02:14:36.247 INFO:teuthology.orchestra.run.smithi150.stdout: debug bluefs = 1/20 2023-11-06T02:14:36.247 INFO:teuthology.orchestra.run.smithi150.stdout: debug bluestore = 1/20 2023-11-06T02:14:36.247 INFO:teuthology.orchestra.run.smithi150.stdout: debug ms = 1 2023-11-06T02:14:36.247 INFO:teuthology.orchestra.run.smithi150.stdout: debug osd = 20 2023-11-06T02:14:36.247 INFO:teuthology.orchestra.run.smithi150.stdout: debug rocksdb = 4/10 2023-11-06T02:14:36.247 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd backfillfull_ratio = 0.85 2023-11-06T02:14:36.248 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd full ratio = 0.9 2023-11-06T02:14:36.248 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd nearfull ratio = 0.8 2023-11-06T02:14:36.248 INFO:teuthology.orchestra.run.smithi150.stdout: osd failsafe full ratio = 0.95 2023-11-06T02:14:36.248 INFO:teuthology.orchestra.run.smithi150.stdout: osd objectstore = bluestore 2023-11-06T02:14:36.248 INFO:teuthology.orchestra.run.smithi150.stdout: osd op complaint time = 180 2023-11-06T02:14:36.249 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:36.249 INFO:teuthology.orchestra.run.smithi150.stdout:[mgr] 2023-11-06T02:14:36.249 INFO:teuthology.orchestra.run.smithi150.stdout: debug ms = 1 2023-11-06T02:14:36.249 INFO:teuthology.orchestra.run.smithi150.stdout: debug mgr = 20 2023-11-06T02:14:36.249 INFO:teuthology.orchestra.run.smithi150.stdout: debug mon = 20 2023-11-06T02:14:36.250 INFO:teuthology.orchestra.run.smithi150.stdout: debug auth = 20 2023-11-06T02:14:36.250 INFO:teuthology.orchestra.run.smithi150.stdout: mon reweight min pgs per osd = 4 2023-11-06T02:14:36.250 INFO:teuthology.orchestra.run.smithi150.stdout: mon reweight min bytes per osd = 10 2023-11-06T02:14:36.250 INFO:teuthology.orchestra.run.smithi150.stdout: mgr/telemetry/nag = false 2023-11-06T02:14:36.250 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:36.250 INFO:teuthology.orchestra.run.smithi150.stdout:[mon] 2023-11-06T02:14:36.251 INFO:teuthology.orchestra.run.smithi150.stdout: debug ms = 1 2023-11-06T02:14:36.251 INFO:teuthology.orchestra.run.smithi150.stdout: debug mon = 20 2023-11-06T02:14:36.251 INFO:teuthology.orchestra.run.smithi150.stdout: debug paxos = 20 2023-11-06T02:14:36.251 INFO:teuthology.orchestra.run.smithi150.stdout: debug auth = 20 2023-11-06T02:14:36.251 INFO:teuthology.orchestra.run.smithi150.stdout: mon data avail warn = 5 2023-11-06T02:14:36.252 INFO:teuthology.orchestra.run.smithi150.stdout: mon mgr mkfs grace = 240 2023-11-06T02:14:36.252 INFO:teuthology.orchestra.run.smithi150.stdout: mon reweight min pgs per osd = 4 2023-11-06T02:14:36.252 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd reporter subtree level = osd 2023-11-06T02:14:36.252 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd prime pg temp = true 2023-11-06T02:14:36.252 INFO:teuthology.orchestra.run.smithi150.stdout: mon reweight min bytes per osd = 10 2023-11-06T02:14:36.253 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:36.253 INFO:teuthology.orchestra.run.smithi150.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-06T02:14:36.253 INFO:teuthology.orchestra.run.smithi150.stdout: auth mon ticket ttl = 660 # 11m 2023-11-06T02:14:36.253 INFO:teuthology.orchestra.run.smithi150.stdout: auth service ticket ttl = 240 # 4m 2023-11-06T02:14:36.253 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:36.254 INFO:teuthology.orchestra.run.smithi150.stdout: # don't complain about insecure global_id in the test suite 2023-11-06T02:14:36.254 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-06T02:14:36.254 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-06T02:14:36.254 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:36.254 INFO:teuthology.orchestra.run.smithi150.stdout: # 1m isn't quite enough 2023-11-06T02:14:36.254 INFO:teuthology.orchestra.run.smithi150.stdout: mon_down_mkfs_grace = 2m 2023-11-06T02:14:36.255 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:36.255 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_filestore_osds = false 2023-11-06T02:14:36.255 INFO:teuthology.orchestra.run.smithi150.stdout: mon op complaint time = 120 2023-11-06T02:14:36.255 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-06T02:14:36.255 INFO:teuthology.orchestra.run.smithi150.stdout:[client] 2023-11-06T02:14:36.256 INFO:teuthology.orchestra.run.smithi150.stdout: rgw cache enabled = true 2023-11-06T02:14:36.256 INFO:teuthology.orchestra.run.smithi150.stdout: rgw enable ops log = true 2023-11-06T02:14:36.256 INFO:teuthology.orchestra.run.smithi150.stdout: rgw enable usage log = true 2023-11-06T02:14:36.256 INFO:teuthology.orchestra.run.smithi150.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-06T02:14:36.256 INFO:teuthology.orchestra.run.smithi150.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-06T02:14:36.257 INFO:teuthology.orchestra.run.smithi150.stdout: client mount timeout = 600 2023-11-06T02:14:36.257 INFO:teuthology.orchestra.run.smithi150.stdout: debug client = 20 2023-11-06T02:14:36.257 INFO:teuthology.orchestra.run.smithi150.stdout: debug ms = 1 2023-11-06T02:14:36.257 INFO:teuthology.orchestra.run.smithi150.stdout: pid file = "" 2023-11-06T02:14:36.257 INFO:teuthology.orchestra.run.smithi150.stdout: rados mon op timeout = 900 2023-11-06T02:14:36.257 INFO:teuthology.orchestra.run.smithi150.stdout: rados osd op timeout = 900 2023-11-06T02:14:36.258 INFO:teuthology.orchestra.run.smithi150.stdout:[mon.a] 2023-11-06T02:14:36.258 INFO:teuthology.orchestra.run.smithi150.stdout:[mon.b] 2023-11-06T02:14:36.258 INFO:teuthology.orchestra.run.smithi150.stdout:[mon.c] 2023-11-06T02:14:36.258 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.a] 2023-11-06T02:14:36.258 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.c] 2023-11-06T02:14:36.259 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.b] 2023-11-06T02:14:36.259 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.d] 2023-11-06T02:14:36.259 INFO:teuthology.orchestra.run.smithi150.stdout:[mds] 2023-11-06T02:14:36.259 INFO:teuthology.orchestra.run.smithi150.stdout: debug mds = 20 2023-11-06T02:14:36.259 INFO:teuthology.orchestra.run.smithi150.stdout: debug mds balancer = 20 2023-11-06T02:14:36.259 INFO:teuthology.orchestra.run.smithi150.stdout: debug ms = 1 2023-11-06T02:14:36.260 INFO:teuthology.orchestra.run.smithi150.stdout: mds debug frag = True 2023-11-06T02:14:36.260 INFO:teuthology.orchestra.run.smithi150.stdout: mds debug scatterstat = True 2023-11-06T02:14:36.260 INFO:teuthology.orchestra.run.smithi150.stdout: mds op complaint time = 180 2023-11-06T02:14:36.260 INFO:teuthology.orchestra.run.smithi150.stdout: mds verify scatter = True 2023-11-06T02:14:36.260 INFO:teuthology.orchestra.run.smithi150.stdout: osd op complaint time = 180 2023-11-06T02:14:36.261 INFO:teuthology.orchestra.run.smithi150.stdout: rados mon op timeout = 900 2023-11-06T02:14:36.261 INFO:teuthology.orchestra.run.smithi150.stdout: rados osd op timeout = 900 2023-11-06T02:14:36.261 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-11-06T02:14:36.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:36.418+0000 7f5d966aa640 1 Processor -- start 2023-11-06T02:14:36.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:36.418+0000 7f5d966aa640 1 -- start start 2023-11-06T02:14:36.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:36.418+0000 7f5d966aa640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5d9015cf30 0x7f5d9006e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:36.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:36.418+0000 7f5d966aa640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f5d9006e9d0 0x7f5d900ccb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:36.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:36.418+0000 7f5d966aa640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f5d900cd0c0 0x7f5d900c9be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:36.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:36.418+0000 7f5d966aa640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d900c9600 con 0x7f5d900cd0c0 2023-11-06T02:14:36.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:36.418+0000 7f5d966aa640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d900c98e0 con 0x7f5d9006e9d0 2023-11-06T02:14:36.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:36.418+0000 7f5d966aa640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9006ae80 con 0x7f5d9015cf30 2023-11-06T02:14:36.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:36.418+0000 7f5d94c47640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f5d900cd0c0 0x7f5d900c9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:36.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:36.418+0000 7f5d94c47640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f5d900cd0c0 0x7f5d900c9be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38278/0 (socket says 172.21.15.49:38278) 2023-11-06T02:14:36.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:36.418+0000 7f5d94c47640 1 -- 172.21.15.49:0/2331983051 learned_addr learned my addr 172.21.15.49:0/2331983051 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:36.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:36.418+0000 7f5d8f7fe640 1 --2- 172.21.15.49:0/2331983051 >> v2:172.21.15.150:3300/0 conn(0x7f5d9006e9d0 0x7f5d900ccb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:36.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:36.418+0000 7f5d8ffff640 1 --2- 172.21.15.49:0/2331983051 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5d9015cf30 0x7f5d9006e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:36.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:36.418+0000 7f5d94c47640 1 -- 172.21.15.49:0/2331983051 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5d9015cf30 msgr2=0x7f5d9006e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:36.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:36.418+0000 7f5d94c47640 1 --2- 172.21.15.49:0/2331983051 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5d9015cf30 0x7f5d9006e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:36.417 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:36.418+0000 7f5d94c47640 1 -- 172.21.15.49:0/2331983051 >> v2:172.21.15.150:3300/0 conn(0x7f5d9006e9d0 msgr2=0x7f5d900ccb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:36.417 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:36.418+0000 7f5d94c47640 1 --2- 172.21.15.49:0/2331983051 >> v2:172.21.15.150:3300/0 conn(0x7f5d9006e9d0 0x7f5d900ccb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:36.417 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:36.418+0000 7f5d94c47640 1 -- 172.21.15.49:0/2331983051 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d9006b160 con 0x7f5d900cd0c0 2023-11-06T02:14:36.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:36.422+0000 7f5d94c47640 1 --2- 172.21.15.49:0/2331983051 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d900cd0c0 0x7f5d900c9be0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f5d90070470 tx=0x7f5d8801c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71fdab4d41fdea13 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:36.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:36.422+0000 7f5d8effd640 1 -- 172.21.15.49:0/2331983051 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d880a6500 con 0x7f5d900cd0c0 2023-11-06T02:14:36.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:36.422+0000 7f5d8effd640 1 -- 172.21.15.49:0/2331983051 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d880a6660 con 0x7f5d900cd0c0 2023-11-06T02:14:36.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:36.422+0000 7f5d8effd640 1 -- 172.21.15.49:0/2331983051 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d880a69f0 con 0x7f5d900cd0c0 2023-11-06T02:14:36.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:36.422+0000 7f5d966aa640 1 -- 172.21.15.49:0/2331983051 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d900cd0c0 msgr2=0x7f5d900c9be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:36.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:36.422+0000 7f5d966aa640 1 --2- 172.21.15.49:0/2331983051 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d900cd0c0 0x7f5d900c9be0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f5d90070470 tx=0x7f5d8801c6e0 comp rx=0 tx=0).stop 2023-11-06T02:14:36.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:36.422+0000 7f5d966aa640 1 -- 172.21.15.49:0/2331983051 shutdown_connections 2023-11-06T02:14:36.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:36.422+0000 7f5d966aa640 1 --2- 172.21.15.49:0/2331983051 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d900cd0c0 0x7f5d900c9be0 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:36.422 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:36.422+0000 7f5d966aa640 1 --2- 172.21.15.49:0/2331983051 >> v2:172.21.15.150:3300/0 conn(0x7f5d9006e9d0 0x7f5d900ccb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:36.422 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:36.422+0000 7f5d966aa640 1 --2- 172.21.15.49:0/2331983051 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5d9015cf30 0x7f5d9006e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:36.422 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:36.422+0000 7f5d966aa640 1 -- 172.21.15.49:0/2331983051 >> 172.21.15.49:0/2331983051 conn(0x7f5d900b0aa0 msgr2=0x7f5d900b0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:36.422 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:36.422+0000 7f5d966aa640 1 -- 172.21.15.49:0/2331983051 shutdown_connections 2023-11-06T02:14:36.422 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:36.422+0000 7f5d966aa640 1 -- 172.21.15.49:0/2331983051 wait complete. 2023-11-06T02:14:36.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:36.422+0000 7f5d966aa640 1 Processor -- start 2023-11-06T02:14:36.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:36.422+0000 7f5d966aa640 1 -- start start 2023-11-06T02:14:36.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:36.422+0000 7f5d966aa640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d9006e9d0 0x7f5d900b2710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:36.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:36.422+0000 7f5d966aa640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d900cd0c0 0x7f5d900c6c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:36.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:36.422+0000 7f5d966aa640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5d9015cf30 0x7f5d90163b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:36.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:36.422+0000 7f5d966aa640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d900c7180 con 0x7f5d9006e9d0 2023-11-06T02:14:36.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:36.422+0000 7f5d966aa640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d900c7400 con 0x7f5d900cd0c0 2023-11-06T02:14:36.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:36.422+0000 7f5d966aa640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d900c76e0 con 0x7f5d9015cf30 2023-11-06T02:14:36.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:36.422+0000 7f5d8ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d9006e9d0 0x7f5d900b2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:36.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:36.422+0000 7f5d8ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d9006e9d0 0x7f5d900b2710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38288/0 (socket says 172.21.15.49:38288) 2023-11-06T02:14:36.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:36.422+0000 7f5d8ffff640 1 -- 172.21.15.49:0/724551612 learned_addr learned my addr 172.21.15.49:0/724551612 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:36.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:36.426+0000 7f5d8f7fe640 1 --2- 172.21.15.49:0/724551612 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d900cd0c0 0x7f5d900c6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:36.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:36.426+0000 7f5d8ffff640 1 -- 172.21.15.49:0/724551612 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5d9015cf30 msgr2=0x7f5d90163b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:36.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:36.426+0000 7f5d94c47640 1 --2- 172.21.15.49:0/724551612 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5d9015cf30 0x7f5d90163b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:36.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:36.426+0000 7f5d8ffff640 1 --2- 172.21.15.49:0/724551612 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5d9015cf30 0x7f5d90163b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:36.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:36.426+0000 7f5d8ffff640 1 -- 172.21.15.49:0/724551612 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d900cd0c0 msgr2=0x7f5d900c6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:36.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:36.426+0000 7f5d8ffff640 1 --2- 172.21.15.49:0/724551612 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d900cd0c0 0x7f5d900c6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:36.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:36.426+0000 7f5d8ffff640 1 -- 172.21.15.49:0/724551612 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d88009b30 con 0x7f5d9006e9d0 2023-11-06T02:14:36.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:36.426+0000 7f5d8f7fe640 1 --2- 172.21.15.49:0/724551612 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d900cd0c0 0x7f5d900c6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:14:36.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:36.426+0000 7f5d8ffff640 1 --2- 172.21.15.49:0/724551612 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d9006e9d0 0x7f5d900b2710 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f5d8000d800 tx=0x7f5d8000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:36.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:36.426+0000 7f5d8cff9640 1 -- 172.21.15.49:0/724551612 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d8000b6c0 con 0x7f5d9006e9d0 2023-11-06T02:14:36.427 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:36.426+0000 7f5d8cff9640 1 -- 172.21.15.49:0/724551612 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d8000b820 con 0x7f5d9006e9d0 2023-11-06T02:14:36.427 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:36.426+0000 7f5d966aa640 1 -- 172.21.15.49:0/724551612 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d900c79c0 con 0x7f5d9006e9d0 2023-11-06T02:14:36.427 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:36.426+0000 7f5d8cff9640 1 -- 172.21.15.49:0/724551612 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d8000bb90 con 0x7f5d9006e9d0 2023-11-06T02:14:36.427 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:36.426+0000 7f5d966aa640 1 -- 172.21.15.49:0/724551612 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d900c7f80 con 0x7f5d9006e9d0 2023-11-06T02:14:36.428 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:36.426+0000 7f5d966aa640 1 -- 172.21.15.49:0/724551612 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d900b18d0 con 0x7f5d9006e9d0 2023-11-06T02:14:36.429 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:36.426+0000 7f5d8cff9640 1 -- 172.21.15.49:0/724551612 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f5d8000bcf0 con 0x7f5d9006e9d0 2023-11-06T02:14:36.429 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:36.426+0000 7f5d8cff9640 1 --2- 172.21.15.49:0/724551612 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5d7c065a60 0x7f5d7c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:36.429 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:36.426+0000 7f5d8cff9640 1 -- 172.21.15.49:0/724551612 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6234+0+0 (secure 0 0 0) 0x7f5d80084330 con 0x7f5d9006e9d0 2023-11-06T02:14:36.429 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:36.426+0000 7f5d8f7fe640 1 --2- 172.21.15.49:0/724551612 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5d7c065a60 0x7f5d7c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:36.429 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:36.430+0000 7f5d8f7fe640 1 --2- 172.21.15.49:0/724551612 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5d7c065a60 0x7f5d7c067f20 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f5d901f8790 tx=0x7f5d84008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:36.430 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:36.430+0000 7f5d8cff9640 1 -- 172.21.15.49:0/724551612 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f5d80051bc0 con 0x7f5d9006e9d0 2023-11-06T02:14:36.616 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:36.622+0000 7f5d966aa640 1 -- 172.21.15.49:0/724551612 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f5d9006d3d0 con 0x7f5d9006e9d0 2023-11-06T02:14:37.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:37.402+0000 7f5d8cff9640 1 -- 172.21.15.49:0/724551612 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v55) v1 ==== 162+0+0 (secure 0 0 0) 0x7f5d800551e0 con 0x7f5d9006e9d0 2023-11-06T02:14:37.397 INFO:teuthology.orchestra.run.smithi049.stderr:cephfs marked joinable; MDS may join as newly active. 2023-11-06T02:14:37.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:37.406+0000 7f5d966aa640 1 -- 172.21.15.49:0/724551612 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5d7c065a60 msgr2=0x7f5d7c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:37.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:37.406+0000 7f5d966aa640 1 --2- 172.21.15.49:0/724551612 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5d7c065a60 0x7f5d7c067f20 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f5d901f8790 tx=0x7f5d84008040 comp rx=0 tx=0).stop 2023-11-06T02:14:37.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:37.406+0000 7f5d966aa640 1 -- 172.21.15.49:0/724551612 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d9006e9d0 msgr2=0x7f5d900b2710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:37.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:37.406+0000 7f5d966aa640 1 --2- 172.21.15.49:0/724551612 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d9006e9d0 0x7f5d900b2710 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f5d8000d800 tx=0x7f5d8000dcc0 comp rx=0 tx=0).stop 2023-11-06T02:14:37.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:37.406+0000 7f5d966aa640 1 -- 172.21.15.49:0/724551612 shutdown_connections 2023-11-06T02:14:37.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:37.406+0000 7f5d966aa640 1 --2- 172.21.15.49:0/724551612 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5d7c065a60 0x7f5d7c067f20 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:37.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:37.406+0000 7f5d966aa640 1 --2- 172.21.15.49:0/724551612 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5d9015cf30 0x7f5d90163b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:37.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:37.406+0000 7f5d966aa640 1 --2- 172.21.15.49:0/724551612 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d900cd0c0 0x7f5d900c6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:37.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:37.406+0000 7f5d966aa640 1 --2- 172.21.15.49:0/724551612 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d9006e9d0 0x7f5d900b2710 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:37.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:37.406+0000 7f5d966aa640 1 -- 172.21.15.49:0/724551612 >> 172.21.15.49:0/724551612 conn(0x7f5d900b0aa0 msgr2=0x7f5d9015be20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:37.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:37.406+0000 7f5d966aa640 1 -- 172.21.15.49:0/724551612 shutdown_connections 2023-11-06T02:14:37.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:37.406+0000 7f5d966aa640 1 -- 172.21.15.49:0/724551612 wait complete. 2023-11-06T02:14:37.426 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T02:14:37.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:37.550+0000 7fc17ccc8640 1 Processor -- start 2023-11-06T02:14:37.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:37.554+0000 7fc17ccc8640 1 -- start start 2023-11-06T02:14:37.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:37.554+0000 7fc17ccc8640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc17815ab50 0x7fc1780c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:37.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:37.554+0000 7fc17ccc8640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc1780cb010 0x7fc1780b0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:37.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:37.554+0000 7fc17ccc8640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1780cab80 con 0x7fc1780b05c0 2023-11-06T02:14:37.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:37.554+0000 7fc17ccc8640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1780b0a10 con 0x7fc1780cb010 2023-11-06T02:14:37.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:37.554+0000 7fc17ccc8640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc17815af30 con 0x7fc17815ab50 2023-11-06T02:14:37.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:37.554+0000 7fc175d9b640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc1780cb010 0x7fc1780b0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:37.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:37.554+0000 7fc176d9d640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fc1780b05c0 0x7fc1780700a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:60654/0 (socket says 172.21.15.49:60654) 2023-11-06T02:14:37.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:37.554+0000 7fc176d9d640 1 -- 172.21.15.49:0/3912568230 learned_addr learned my addr 172.21.15.49:0/3912568230 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:14:37.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:37.554+0000 7fc17659c640 1 --2- 172.21.15.49:0/3912568230 >> v2:172.21.15.150:3300/0 conn(0x7fc17815ab50 0x7fc1780c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:37.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:37.554+0000 7fc175d9b640 1 -- 172.21.15.49:0/3912568230 >> v2:172.21.15.150:3300/0 conn(0x7fc17815ab50 msgr2=0x7fc1780c9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:37.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:37.554+0000 7fc175d9b640 1 --2- 172.21.15.49:0/3912568230 >> v2:172.21.15.150:3300/0 conn(0x7fc17815ab50 0x7fc1780c9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:37.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:37.554+0000 7fc175d9b640 1 -- 172.21.15.49:0/3912568230 >> v1:172.21.15.49:6789/0 conn(0x7fc1780b05c0 legacy=0x7fc1780700a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:37.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:37.554+0000 7fc175d9b640 1 -- 172.21.15.49:0/3912568230 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1780cb5b0 con 0x7fc1780cb010 2023-11-06T02:14:37.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:37.554+0000 7fc17559a640 1 -- 172.21.15.49:0/3912568230 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 943843639 0 0) 0x7fc1780cab80 con 0x7fc1780b05c0 2023-11-06T02:14:37.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:37.554+0000 7fc175d9b640 1 --2- 172.21.15.49:0/3912568230 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc1780cb010 0x7fc1780b0080 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fc1780725a0 tx=0x7fc16001c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=852fcc89757dd068 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:37.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:37.558+0000 7fc17559a640 1 -- 172.21.15.49:0/3912568230 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1600a7070 con 0x7fc1780cb010 2023-11-06T02:14:37.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:37.558+0000 7fc17559a640 1 -- 172.21.15.49:0/3912568230 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1600ab430 con 0x7fc1780cb010 2023-11-06T02:14:37.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:37.558+0000 7fc17559a640 1 -- 172.21.15.49:0/3912568230 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1600ab7c0 con 0x7fc1780cb010 2023-11-06T02:14:37.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:37.558+0000 7fc17ccc8640 1 -- 172.21.15.49:0/3912568230 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc1780cb010 msgr2=0x7fc1780b0080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:37.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:37.558+0000 7fc17ccc8640 1 --2- 172.21.15.49:0/3912568230 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc1780cb010 0x7fc1780b0080 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fc1780725a0 tx=0x7fc16001c430 comp rx=0 tx=0).stop 2023-11-06T02:14:37.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:37.558+0000 7fc17ccc8640 1 -- 172.21.15.49:0/3912568230 shutdown_connections 2023-11-06T02:14:37.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:37.558+0000 7fc17ccc8640 1 --2- 172.21.15.49:0/3912568230 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc1780cb010 0x7fc1780b0080 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:37.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:37.558+0000 7fc17ccc8640 1 --2- 172.21.15.49:0/3912568230 >> v2:172.21.15.150:3300/0 conn(0x7fc17815ab50 0x7fc1780c9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:37.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:37.558+0000 7fc17ccc8640 1 -- 172.21.15.49:0/3912568230 >> 172.21.15.49:0/3912568230 conn(0x7fc17806c0a0 msgr2=0x7fc17806c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:37.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:37.558+0000 7fc17ccc8640 1 -- 172.21.15.49:0/3912568230 shutdown_connections 2023-11-06T02:14:37.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:37.558+0000 7fc17ccc8640 1 -- 172.21.15.49:0/3912568230 wait complete. 2023-11-06T02:14:37.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:37.558+0000 7fc17ccc8640 1 Processor -- start 2023-11-06T02:14:37.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:37.558+0000 7fc17ccc8640 1 -- start start 2023-11-06T02:14:37.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:37.558+0000 7fc17ccc8640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1780b05c0 0x7fc1780acc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:37.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:37.558+0000 7fc17ccc8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc1780cb010 0x7fc1780cd940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:37.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:37.558+0000 7fc17ccc8640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc17815ab50 0x7fc17815b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:37.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:37.558+0000 7fc17ccc8640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1781549f0 con 0x7fc1780cb010 2023-11-06T02:14:37.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:37.558+0000 7fc17ccc8640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1780ad190 con 0x7fc1780b05c0 2023-11-06T02:14:37.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:37.558+0000 7fc17ccc8640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1781540c0 con 0x7fc17815ab50 2023-11-06T02:14:37.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:37.558+0000 7fc175d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc1780cb010 0x7fc1780cd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:37.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:37.558+0000 7fc175d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc1780cb010 0x7fc1780cd940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38306/0 (socket says 172.21.15.49:38306) 2023-11-06T02:14:37.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:37.558+0000 7fc175d9b640 1 -- 172.21.15.49:0/2345507726 learned_addr learned my addr 172.21.15.49:0/2345507726 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:37.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:37.558+0000 7fc17659c640 1 --2- 172.21.15.49:0/2345507726 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1780b05c0 0x7fc1780acc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:37.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:37.558+0000 7fc176d9d640 1 --2- 172.21.15.49:0/2345507726 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc17815ab50 0x7fc17815b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:37.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:37.558+0000 7fc175d9b640 1 -- 172.21.15.49:0/2345507726 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc17815ab50 msgr2=0x7fc17815b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:37.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:37.558+0000 7fc175d9b640 1 --2- 172.21.15.49:0/2345507726 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc17815ab50 0x7fc17815b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:37.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:37.558+0000 7fc175d9b640 1 -- 172.21.15.49:0/2345507726 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1780b05c0 msgr2=0x7fc1780acc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:37.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:37.558+0000 7fc175d9b640 1 --2- 172.21.15.49:0/2345507726 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1780b05c0 0x7fc1780acc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:37.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:37.558+0000 7fc175d9b640 1 -- 172.21.15.49:0/2345507726 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc160009420 con 0x7fc1780cb010 2023-11-06T02:14:37.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:37.558+0000 7fc176d9d640 1 --2- 172.21.15.49:0/2345507726 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc17815ab50 0x7fc17815b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:14:37.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:37.562+0000 7fc175d9b640 1 --2- 172.21.15.49:0/2345507726 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc1780cb010 0x7fc1780cd940 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fc1600ad040 tx=0x7fc16001ac70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:37.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:37.562+0000 7fc16effd640 1 -- 172.21.15.49:0/2345507726 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1600b5810 con 0x7fc1780cb010 2023-11-06T02:14:37.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:37.562+0000 7fc16effd640 1 -- 172.21.15.49:0/2345507726 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1600b5990 con 0x7fc1780cb010 2023-11-06T02:14:37.563 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:37.562+0000 7fc16effd640 1 -- 172.21.15.49:0/2345507726 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1600b5d40 con 0x7fc1780cb010 2023-11-06T02:14:37.563 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:37.562+0000 7fc17ccc8640 1 -- 172.21.15.49:0/2345507726 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1781543a0 con 0x7fc1780cb010 2023-11-06T02:14:37.563 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:37.562+0000 7fc17ccc8640 1 -- 172.21.15.49:0/2345507726 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc178153d70 con 0x7fc1780cb010 2023-11-06T02:14:37.563 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:37.562+0000 7fc17ccc8640 1 -- 172.21.15.49:0/2345507726 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1780b0de0 con 0x7fc1780cb010 2023-11-06T02:14:37.563 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:37.562+0000 7fc16effd640 1 -- 172.21.15.49:0/2345507726 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fc1600b4390 con 0x7fc1780cb010 2023-11-06T02:14:37.563 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:37.562+0000 7fc16effd640 1 --2- 172.21.15.49:0/2345507726 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc158065a60 0x7fc158067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:37.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:37.562+0000 7fc16effd640 1 -- 172.21.15.49:0/2345507726 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6234+0+0 (secure 0 0 0) 0x7fc16011da80 con 0x7fc1780cb010 2023-11-06T02:14:37.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:37.562+0000 7fc17659c640 1 --2- 172.21.15.49:0/2345507726 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc158065a60 0x7fc158067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:37.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:37.562+0000 7fc17659c640 1 --2- 172.21.15.49:0/2345507726 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc158065a60 0x7fc158067f20 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fc168006fd0 tx=0x7fc168008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:37.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:37.566+0000 7fc16effd640 1 -- 172.21.15.49:0/2345507726 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fc1600eb400 con 0x7fc1780cb010 2023-11-06T02:14:37.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:37.758+0000 7fc17ccc8640 1 -- 172.21.15.49:0/2345507726 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc178154200 con 0x7fc1780cb010 2023-11-06T02:14:37.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:37.762+0000 7fc16effd640 1 -- 172.21.15.49:0/2345507726 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+4651 (secure 0 0 0) 0x7fc178154200 con 0x7fc1780cb010 2023-11-06T02:14:37.756 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:37.757 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":56,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5714,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/315581318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":315581318},{"type":"v1","addr":"172.21.15.150:6837","nonce":315581318}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41},{"gid":6121,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/136800245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":136800245},{"type":"v1","addr":"172.21.15.150:6835","nonce":136800245}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41},{"gid":7110,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/4259579254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":4259579254},{"type":"v1","addr":"172.21.15.49:6833","nonce":4259579254}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T02:13:33.625701+0000","modified":"2023-11-06T02:14:37.404717+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7752},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7752":{"gid":7752,"name":"c","rank":0,"incarnation":56,"state":"up:replay","state_seq":1,"addr":"172.21.15.49:6835/2241629386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2241629386},{"type":"v1","addr":"172.21.15.49:6835","nonce":2241629386}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-06T02:14:37.757 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 56 2023-11-06T02:14:37.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:37.762+0000 7fc17ccc8640 1 -- 172.21.15.49:0/2345507726 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc158065a60 msgr2=0x7fc158067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:37.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:37.762+0000 7fc17ccc8640 1 --2- 172.21.15.49:0/2345507726 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc158065a60 0x7fc158067f20 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fc168006fd0 tx=0x7fc168008040 comp rx=0 tx=0).stop 2023-11-06T02:14:37.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:37.762+0000 7fc17ccc8640 1 -- 172.21.15.49:0/2345507726 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc1780cb010 msgr2=0x7fc1780cd940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:37.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:37.762+0000 7fc17ccc8640 1 --2- 172.21.15.49:0/2345507726 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc1780cb010 0x7fc1780cd940 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fc1600ad040 tx=0x7fc16001ac70 comp rx=0 tx=0).stop 2023-11-06T02:14:37.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:37.762+0000 7fc17ccc8640 1 -- 172.21.15.49:0/2345507726 shutdown_connections 2023-11-06T02:14:37.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:37.762+0000 7fc17ccc8640 1 --2- 172.21.15.49:0/2345507726 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc158065a60 0x7fc158067f20 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:37.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:37.762+0000 7fc17ccc8640 1 --2- 172.21.15.49:0/2345507726 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc1780cb010 0x7fc1780cd940 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:37.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:37.762+0000 7fc17ccc8640 1 --2- 172.21.15.49:0/2345507726 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc17815ab50 0x7fc17815b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:37.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:37.762+0000 7fc17ccc8640 1 --2- 172.21.15.49:0/2345507726 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1780b05c0 0x7fc1780acc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:37.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:37.762+0000 7fc17ccc8640 1 -- 172.21.15.49:0/2345507726 >> 172.21.15.49:0/2345507726 conn(0x7fc17806c0a0 msgr2=0x7fc178072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:37.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:37.762+0000 7fc17ccc8640 1 -- 172.21.15.49:0/2345507726 shutdown_connections 2023-11-06T02:14:37.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:37.762+0000 7fc17ccc8640 1 -- 172.21.15.49:0/2345507726 wait complete. 2023-11-06T02:14:38.178 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 56, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-06T02:13:33.625701+0000', 'modified': '2023-11-06T02:14:37.404717+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 82, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7752}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7752': {'gid': 7752, 'name': 'c', 'rank': 0, 'incarnation': 56, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.49:6835/2241629386', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6834', 'nonce': 2241629386}, {'type': 'v1', 'addr': '172.21.15.49:6835', 'nonce': 2241629386}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-06T02:14:38.178 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7752:up:replay 2023-11-06T02:14:39.180 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T02:14:39.315 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:39.322+0000 7f1b5043d640 1 Processor -- start 2023-11-06T02:14:39.316 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:39.322+0000 7f1b5043d640 1 -- start start 2023-11-06T02:14:39.317 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:39.322+0000 7f1b5043d640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f1b4815aae0 0x7f1b48070a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:39.318 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:39.322+0000 7f1b5043d640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1b480ca830 0x7f1b480b0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:39.318 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:39.322+0000 7f1b5043d640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f1b480c9db0 0x7f1b480b05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:39.318 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:39.322+0000 7f1b5043d640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b48070fc0 con 0x7f1b4815aae0 2023-11-06T02:14:39.318 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:39.322+0000 7f1b5043d640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b480cacf0 con 0x7f1b480c9db0 2023-11-06T02:14:39.319 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:39.322+0000 7f1b5043d640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b4815aec0 con 0x7f1b480ca830 2023-11-06T02:14:39.319 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:39.322+0000 7f1b4e1d9640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f1b4815aae0 0x7f1b48070a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:39.319 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:39.322+0000 7f1b4e1d9640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f1b4815aae0 0x7f1b48070a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47904/0 (socket says 172.21.15.49:47904) 2023-11-06T02:14:39.319 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:39.322+0000 7f1b4e1d9640 1 -- 172.21.15.49:0/2909245725 learned_addr learned my addr 172.21.15.49:0/2909245725 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:39.319 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:39.322+0000 7f1b4e9da640 1 --2- 172.21.15.49:0/2909245725 >> v2:172.21.15.150:3300/0 conn(0x7f1b480c9db0 0x7f1b480b05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:39.320 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:39.322+0000 7f1b4d9d8640 1 --2- 172.21.15.49:0/2909245725 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1b480ca830 0x7f1b480b0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:39.320 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:39.322+0000 7f1b4e1d9640 1 -- 172.21.15.49:0/2909245725 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1b480ca830 msgr2=0x7f1b480b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:39.320 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:39.322+0000 7f1b4e1d9640 1 --2- 172.21.15.49:0/2909245725 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1b480ca830 0x7f1b480b0080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:39.320 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:39.322+0000 7f1b4e1d9640 1 -- 172.21.15.49:0/2909245725 >> v2:172.21.15.150:3300/0 conn(0x7f1b480c9db0 msgr2=0x7f1b480b05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:39.321 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:39.322+0000 7f1b4e1d9640 1 --2- 172.21.15.49:0/2909245725 >> v2:172.21.15.150:3300/0 conn(0x7f1b480c9db0 0x7f1b480b05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:39.321 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:39.322+0000 7f1b4e1d9640 1 -- 172.21.15.49:0/2909245725 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b480c8ff0 con 0x7f1b4815aae0 2023-11-06T02:14:39.321 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:39.322+0000 7f1b4e9da640 1 --2- 172.21.15.49:0/2909245725 >> v2:172.21.15.150:3300/0 conn(0x7f1b480c9db0 0x7f1b480b05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:14:39.322 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:39.326+0000 7f1b4e1d9640 1 --2- 172.21.15.49:0/2909245725 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b4815aae0 0x7f1b48070a80 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f1b4806acb0 tx=0x7f1b4401c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba23425e592a56f0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:39.323 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:39.326+0000 7f1b4d1d7640 1 -- 172.21.15.49:0/2909245725 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b440ab600 con 0x7f1b4815aae0 2023-11-06T02:14:39.323 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:39.326+0000 7f1b4d1d7640 1 -- 172.21.15.49:0/2909245725 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b440ab760 con 0x7f1b4815aae0 2023-11-06T02:14:39.323 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:39.326+0000 7f1b4d1d7640 1 -- 172.21.15.49:0/2909245725 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b440abaf0 con 0x7f1b4815aae0 2023-11-06T02:14:39.323 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:39.326+0000 7f1b5043d640 1 -- 172.21.15.49:0/2909245725 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b4815aae0 msgr2=0x7f1b48070a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:39.323 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:39.326+0000 7f1b5043d640 1 --2- 172.21.15.49:0/2909245725 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b4815aae0 0x7f1b48070a80 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f1b4806acb0 tx=0x7f1b4401c440 comp rx=0 tx=0).stop 2023-11-06T02:14:39.324 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:39.326+0000 7f1b5043d640 1 -- 172.21.15.49:0/2909245725 shutdown_connections 2023-11-06T02:14:39.324 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:39.326+0000 7f1b5043d640 1 --2- 172.21.15.49:0/2909245725 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1b480ca830 0x7f1b480b0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:39.324 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:39.326+0000 7f1b5043d640 1 --2- 172.21.15.49:0/2909245725 >> v2:172.21.15.150:3300/0 conn(0x7f1b480c9db0 0x7f1b480b05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:39.324 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:39.326+0000 7f1b5043d640 1 --2- 172.21.15.49:0/2909245725 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b4815aae0 0x7f1b48070a80 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:39.324 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:39.326+0000 7f1b5043d640 1 -- 172.21.15.49:0/2909245725 >> 172.21.15.49:0/2909245725 conn(0x7f1b4806c0a0 msgr2=0x7f1b4806c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:39.325 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:39.326+0000 7f1b5043d640 1 -- 172.21.15.49:0/2909245725 shutdown_connections 2023-11-06T02:14:39.325 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:39.326+0000 7f1b5043d640 1 -- 172.21.15.49:0/2909245725 wait complete. 2023-11-06T02:14:39.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:39.330+0000 7f1b5043d640 1 Processor -- start 2023-11-06T02:14:39.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:39.330+0000 7f1b5043d640 1 -- start start 2023-11-06T02:14:39.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:39.330+0000 7f1b5043d640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1b480c9db0 0x7f1b48153120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:39.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:39.330+0000 7f1b5043d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b480ca830 0x7f1b480c56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:39.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:39.330+0000 7f1b5043d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b4815aae0 0x7f1b480c6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:39.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:39.330+0000 7f1b5043d640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b480c6ba0 con 0x7f1b480ca830 2023-11-06T02:14:39.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:39.330+0000 7f1b5043d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b48153660 con 0x7f1b4815aae0 2023-11-06T02:14:39.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:39.330+0000 7f1b5043d640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b480c5f20 con 0x7f1b480c9db0 2023-11-06T02:14:39.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:39.330+0000 7f1b4d9d8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b480ca830 0x7f1b480c56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:39.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:39.330+0000 7f1b4d9d8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b480ca830 0x7f1b480c56d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47906/0 (socket says 172.21.15.49:47906) 2023-11-06T02:14:39.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:39.330+0000 7f1b4d9d8640 1 -- 172.21.15.49:0/2397304866 learned_addr learned my addr 172.21.15.49:0/2397304866 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:39.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:39.330+0000 7f1b4e1d9640 1 --2- 172.21.15.49:0/2397304866 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1b480c9db0 0x7f1b48153120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:39.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:39.330+0000 7f1b4e9da640 1 --2- 172.21.15.49:0/2397304866 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b4815aae0 0x7f1b480c6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:39.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:39.330+0000 7f1b4d9d8640 1 -- 172.21.15.49:0/2397304866 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1b480c9db0 msgr2=0x7f1b48153120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:39.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:39.330+0000 7f1b4d9d8640 1 --2- 172.21.15.49:0/2397304866 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1b480c9db0 0x7f1b48153120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:39.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:39.330+0000 7f1b4d9d8640 1 -- 172.21.15.49:0/2397304866 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b4815aae0 msgr2=0x7f1b480c6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:39.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:39.330+0000 7f1b4d9d8640 1 --2- 172.21.15.49:0/2397304866 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b4815aae0 0x7f1b480c6fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:39.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:39.330+0000 7f1b4d9d8640 1 -- 172.21.15.49:0/2397304866 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b44009480 con 0x7f1b480ca830 2023-11-06T02:14:39.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:39.330+0000 7f1b4e9da640 1 --2- 172.21.15.49:0/2397304866 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b4815aae0 0x7f1b480c6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:14:39.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:39.330+0000 7f1b4d9d8640 1 --2- 172.21.15.49:0/2397304866 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b480ca830 0x7f1b480c56d0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f1b3800c7f0 tx=0x7f1b3800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:39.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:39.330+0000 7f1b3effd640 1 -- 172.21.15.49:0/2397304866 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b38007820 con 0x7f1b480ca830 2023-11-06T02:14:39.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:39.330+0000 7f1b3effd640 1 -- 172.21.15.49:0/2397304866 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b38007980 con 0x7f1b480ca830 2023-11-06T02:14:39.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:39.330+0000 7f1b3effd640 1 -- 172.21.15.49:0/2397304866 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b38007d70 con 0x7f1b480ca830 2023-11-06T02:14:39.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:39.330+0000 7f1b5043d640 1 -- 172.21.15.49:0/2397304866 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b480c6200 con 0x7f1b480ca830 2023-11-06T02:14:39.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:39.330+0000 7f1b5043d640 1 -- 172.21.15.49:0/2397304866 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b480c67c0 con 0x7f1b480ca830 2023-11-06T02:14:39.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:39.330+0000 7f1b5043d640 1 -- 172.21.15.49:0/2397304866 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b480c6ce0 con 0x7f1b480ca830 2023-11-06T02:14:39.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:39.330+0000 7f1b3effd640 1 -- 172.21.15.49:0/2397304866 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f1b380113f0 con 0x7f1b480ca830 2023-11-06T02:14:39.334 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:39.334+0000 7f1b3effd640 1 --2- 172.21.15.49:0/2397304866 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1b2c065a60 0x7f1b2c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:39.334 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:39.334+0000 7f1b3effd640 1 -- 172.21.15.49:0/2397304866 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6234+0+0 (secure 0 0 0) 0x7f1b38083630 con 0x7f1b480ca830 2023-11-06T02:14:39.334 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:39.334+0000 7f1b4e1d9640 1 --2- 172.21.15.49:0/2397304866 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1b2c065a60 0x7f1b2c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:39.335 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:39.334+0000 7f1b4e1d9640 1 --2- 172.21.15.49:0/2397304866 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1b2c065a60 0x7f1b2c067f20 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f1b440af000 tx=0x7f1b440a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:39.335 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:39.338+0000 7f1b3effd640 1 -- 172.21.15.49:0/2397304866 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f1b480c6ce0 con 0x7f1b480ca830 2023-11-06T02:14:39.523 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:39.526+0000 7f1b5043d640 1 -- 172.21.15.49:0/2397304866 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1b480c6ce0 con 0x7f1b480ca830 2023-11-06T02:14:39.524 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:39.530+0000 7f1b3effd640 1 -- 172.21.15.49:0/2397304866 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+4651 (secure 0 0 0) 0x7f1b480c6ce0 con 0x7f1b480ca830 2023-11-06T02:14:39.524 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:39.524 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":58,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5714,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/315581318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":315581318},{"type":"v1","addr":"172.21.15.150:6837","nonce":315581318}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41},{"gid":6121,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/136800245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":136800245},{"type":"v1","addr":"172.21.15.150:6835","nonce":136800245}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41},{"gid":7110,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/4259579254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":4259579254},{"type":"v1","addr":"172.21.15.49:6833","nonce":4259579254}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T02:13:33.625701+0000","modified":"2023-11-06T02:14:39.411625+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7752},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7752":{"gid":7752,"name":"c","rank":0,"incarnation":56,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.49:6835/2241629386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2241629386},{"type":"v1","addr":"172.21.15.49:6835","nonce":2241629386}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-06T02:14:39.525 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 58 2023-11-06T02:14:39.526 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:39.530+0000 7f1b5043d640 1 -- 172.21.15.49:0/2397304866 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1b2c065a60 msgr2=0x7f1b2c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:39.527 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:39.530+0000 7f1b5043d640 1 --2- 172.21.15.49:0/2397304866 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1b2c065a60 0x7f1b2c067f20 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f1b440af000 tx=0x7f1b440a7000 comp rx=0 tx=0).stop 2023-11-06T02:14:39.527 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:39.530+0000 7f1b5043d640 1 -- 172.21.15.49:0/2397304866 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b480ca830 msgr2=0x7f1b480c56d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:39.527 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:39.530+0000 7f1b5043d640 1 --2- 172.21.15.49:0/2397304866 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b480ca830 0x7f1b480c56d0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f1b3800c7f0 tx=0x7f1b3800ccb0 comp rx=0 tx=0).stop 2023-11-06T02:14:39.527 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:39.530+0000 7f1b5043d640 1 -- 172.21.15.49:0/2397304866 shutdown_connections 2023-11-06T02:14:39.528 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:39.530+0000 7f1b5043d640 1 --2- 172.21.15.49:0/2397304866 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1b2c065a60 0x7f1b2c067f20 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:39.528 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:39.530+0000 7f1b5043d640 1 --2- 172.21.15.49:0/2397304866 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b480ca830 0x7f1b480c56d0 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:39.528 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:39.530+0000 7f1b5043d640 1 --2- 172.21.15.49:0/2397304866 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b4815aae0 0x7f1b480c6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:39.528 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:39.530+0000 7f1b5043d640 1 --2- 172.21.15.49:0/2397304866 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1b480c9db0 0x7f1b48153120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:39.528 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:39.530+0000 7f1b5043d640 1 -- 172.21.15.49:0/2397304866 >> 172.21.15.49:0/2397304866 conn(0x7f1b4806c0a0 msgr2=0x7f1b480b2850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:39.529 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:39.530+0000 7f1b5043d640 1 -- 172.21.15.49:0/2397304866 shutdown_connections 2023-11-06T02:14:39.529 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:39.530+0000 7f1b5043d640 1 -- 172.21.15.49:0/2397304866 wait complete. 2023-11-06T02:14:39.947 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 58, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-06T02:13:33.625701+0000', 'modified': '2023-11-06T02:14:39.411625+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 82, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7752}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7752': {'gid': 7752, 'name': 'c', 'rank': 0, 'incarnation': 56, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.49:6835/2241629386', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6834', 'nonce': 2241629386}, {'type': 'v1', 'addr': '172.21.15.49:6835', 'nonce': 2241629386}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-06T02:14:39.947 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7752:up:rejoin 2023-11-06T02:14:40.948 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T02:14:41.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.090+0000 7f269e337640 1 Processor -- start 2023-11-06T02:14:41.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.090+0000 7f269e337640 1 -- start start 2023-11-06T02:14:41.085 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.090+0000 7f269e337640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f269815ab50 0x7f26980c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:41.086 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.090+0000 7f269e337640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f26980b0790 0x7f26980c2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:41.086 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.090+0000 7f269e337640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26980cab80 con 0x7f26980cb010 2023-11-06T02:14:41.086 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.090+0000 7f269e337640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f269815af30 con 0x7f26980b0790 2023-11-06T02:14:41.086 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.090+0000 7f269e337640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26980cb5b0 con 0x7f269815ab50 2023-11-06T02:14:41.086 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.090+0000 7f269c8d4640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f26980b0790 0x7f26980c2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:41.087 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.090+0000 7f26977fe640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f26980cb010 0x7f26980b0080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:33570/0 (socket says 172.21.15.49:33570) 2023-11-06T02:14:41.087 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.090+0000 7f26977fe640 1 -- 172.21.15.49:0/4214157504 learned_addr learned my addr 172.21.15.49:0/4214157504 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:14:41.087 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.090+0000 7f2697fff640 1 --2- 172.21.15.49:0/4214157504 >> v2:172.21.15.150:3300/0 conn(0x7f269815ab50 0x7f26980c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:41.087 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.090+0000 7f269c8d4640 1 -- 172.21.15.49:0/4214157504 >> v2:172.21.15.150:3300/0 conn(0x7f269815ab50 msgr2=0x7f26980c9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:41.087 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.090+0000 7f269c8d4640 1 --2- 172.21.15.49:0/4214157504 >> v2:172.21.15.150:3300/0 conn(0x7f269815ab50 0x7f26980c9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:41.088 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.090+0000 7f269c8d4640 1 -- 172.21.15.49:0/4214157504 >> v1:172.21.15.49:6789/0 conn(0x7f26980cb010 legacy=0x7f26980b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:41.088 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.090+0000 7f2696ffd640 1 -- 172.21.15.49:0/4214157504 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2279869939 0 0) 0x7f26980cab80 con 0x7f26980cb010 2023-11-06T02:14:41.088 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.090+0000 7f269c8d4640 1 -- 172.21.15.49:0/4214157504 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26980cb890 con 0x7f26980b0790 2023-11-06T02:14:41.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.094+0000 7f269c8d4640 1 --2- 172.21.15.49:0/4214157504 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26980b0790 0x7f26980c2710 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f26980725a0 tx=0x7f269001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c82bd440ca4bb9f5 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:41.090 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.094+0000 7f2696ffd640 1 -- 172.21.15.49:0/4214157504 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26900a59a0 con 0x7f26980b0790 2023-11-06T02:14:41.090 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.094+0000 7f2696ffd640 1 -- 172.21.15.49:0/4214157504 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26900a5b00 con 0x7f26980b0790 2023-11-06T02:14:41.090 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.094+0000 7f2696ffd640 1 -- 172.21.15.49:0/4214157504 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26900a5e90 con 0x7f26980b0790 2023-11-06T02:14:41.090 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.094+0000 7f269e337640 1 -- 172.21.15.49:0/4214157504 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26980b0790 msgr2=0x7f26980c2710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:41.090 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.094+0000 7f269e337640 1 --2- 172.21.15.49:0/4214157504 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26980b0790 0x7f26980c2710 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f26980725a0 tx=0x7f269001c8e0 comp rx=0 tx=0).stop 2023-11-06T02:14:41.091 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.094+0000 7f269e337640 1 -- 172.21.15.49:0/4214157504 shutdown_connections 2023-11-06T02:14:41.091 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.094+0000 7f269e337640 1 --2- 172.21.15.49:0/4214157504 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26980b0790 0x7f26980c2710 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:41.091 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.094+0000 7f269e337640 1 --2- 172.21.15.49:0/4214157504 >> v2:172.21.15.150:3300/0 conn(0x7f269815ab50 0x7f26980c9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:41.091 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.094+0000 7f269e337640 1 -- 172.21.15.49:0/4214157504 >> 172.21.15.49:0/4214157504 conn(0x7f269806c0a0 msgr2=0x7f269806c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:41.091 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.094+0000 7f269e337640 1 -- 172.21.15.49:0/4214157504 shutdown_connections 2023-11-06T02:14:41.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.094+0000 7f269e337640 1 -- 172.21.15.49:0/4214157504 wait complete. 2023-11-06T02:14:41.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.094+0000 7f269e337640 1 Processor -- start 2023-11-06T02:14:41.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.094+0000 7f269e337640 1 -- start start 2023-11-06T02:14:41.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.094+0000 7f269e337640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f26980b0790 0x7f26980c17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:41.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.094+0000 7f269e337640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26980cb010 0x7f26981552d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:41.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.094+0000 7f269e337640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f269815ab50 0x7f2698155810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:41.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.094+0000 7f269e337640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2698158b00 con 0x7f26980cb010 2023-11-06T02:14:41.096 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.094+0000 7f269e337640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2698158de0 con 0x7f269815ab50 2023-11-06T02:14:41.096 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.094+0000 7f269e337640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26981590c0 con 0x7f26980b0790 2023-11-06T02:14:41.096 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.094+0000 7f26977fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26980cb010 0x7f26981552d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:41.096 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.094+0000 7f26977fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26980cb010 0x7f26981552d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47928/0 (socket says 172.21.15.49:47928) 2023-11-06T02:14:41.096 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.094+0000 7f26977fe640 1 -- 172.21.15.49:0/3319353566 learned_addr learned my addr 172.21.15.49:0/3319353566 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:41.097 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.094+0000 7f269c8d4640 1 --2- 172.21.15.49:0/3319353566 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f269815ab50 0x7f2698155810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:41.097 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.094+0000 7f2697fff640 1 --2- 172.21.15.49:0/3319353566 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f26980b0790 0x7f26980c17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:41.097 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.094+0000 7f26977fe640 1 -- 172.21.15.49:0/3319353566 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f26980b0790 msgr2=0x7f26980c17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:41.097 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.094+0000 7f26977fe640 1 --2- 172.21.15.49:0/3319353566 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f26980b0790 0x7f26980c17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:41.097 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.094+0000 7f26977fe640 1 -- 172.21.15.49:0/3319353566 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f269815ab50 msgr2=0x7f2698155810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:41.098 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.094+0000 7f26977fe640 1 --2- 172.21.15.49:0/3319353566 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f269815ab50 0x7f2698155810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:41.098 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.094+0000 7f26977fe640 1 -- 172.21.15.49:0/3319353566 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2690009a70 con 0x7f26980cb010 2023-11-06T02:14:41.098 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.098+0000 7f26977fe640 1 --2- 172.21.15.49:0/3319353566 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26980cb010 0x7f26981552d0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f268c0011f0 tx=0x7f268c00c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:41.098 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.098+0000 7f2694ff9640 1 -- 172.21.15.49:0/3319353566 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f268c0074e0 con 0x7f26980cb010 2023-11-06T02:14:41.099 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.098+0000 7f2694ff9640 1 -- 172.21.15.49:0/3319353566 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f268c007640 con 0x7f26980cb010 2023-11-06T02:14:41.099 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.098+0000 7f269e337640 1 -- 172.21.15.49:0/3319353566 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26981593a0 con 0x7f26980cb010 2023-11-06T02:14:41.099 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.098+0000 7f2694ff9640 1 -- 172.21.15.49:0/3319353566 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f268c004a10 con 0x7f26980cb010 2023-11-06T02:14:41.099 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.098+0000 7f269e337640 1 -- 172.21.15.49:0/3319353566 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26980c1ce0 con 0x7f26980cb010 2023-11-06T02:14:41.099 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.098+0000 7f269e337640 1 -- 172.21.15.49:0/3319353566 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26980b0de0 con 0x7f26980cb010 2023-11-06T02:14:41.099 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.098+0000 7f2694ff9640 1 -- 172.21.15.49:0/3319353566 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f268c004b70 con 0x7f26980cb010 2023-11-06T02:14:41.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.098+0000 7f2694ff9640 1 --2- 172.21.15.49:0/3319353566 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2684065a90 0x7f2684067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:41.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.098+0000 7f2694ff9640 1 -- 172.21.15.49:0/3319353566 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6234+0+0 (secure 0 0 0) 0x7f268c082810 con 0x7f26980cb010 2023-11-06T02:14:41.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.098+0000 7f2697fff640 1 --2- 172.21.15.49:0/3319353566 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2684065a90 0x7f2684067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:41.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.098+0000 7f2697fff640 1 --2- 172.21.15.49:0/3319353566 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2684065a90 0x7f2684067f50 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f2688006fd0 tx=0x7f2688008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:41.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.102+0000 7f2694ff9640 1 -- 172.21.15.49:0/3319353566 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f268c050050 con 0x7f26980cb010 2023-11-06T02:14:41.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.294+0000 7f269e337640 1 -- 172.21.15.49:0/3319353566 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2698074040 con 0x7f26980cb010 2023-11-06T02:14:41.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.294+0000 7f2694ff9640 1 -- 172.21.15.49:0/3319353566 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+4651 (secure 0 0 0) 0x7f268c018090 con 0x7f26980cb010 2023-11-06T02:14:41.288 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:41.288 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":59,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5714,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/315581318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":315581318},{"type":"v1","addr":"172.21.15.150:6837","nonce":315581318}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41},{"gid":6121,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/136800245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":136800245},{"type":"v1","addr":"172.21.15.150:6835","nonce":136800245}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41},{"gid":7110,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/4259579254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":4259579254},{"type":"v1","addr":"172.21.15.49:6833","nonce":4259579254}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T02:13:33.625701+0000","modified":"2023-11-06T02:14:40.422212+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7752},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7752":{"gid":7752,"name":"c","rank":0,"incarnation":56,"state":"up:active","state_seq":4,"addr":"172.21.15.49:6835/2241629386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2241629386},{"type":"v1","addr":"172.21.15.49:6835","nonce":2241629386}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-06T02:14:41.289 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 59 2023-11-06T02:14:41.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.294+0000 7f269e337640 1 -- 172.21.15.49:0/3319353566 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2684065a90 msgr2=0x7f2684067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:41.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.294+0000 7f269e337640 1 --2- 172.21.15.49:0/3319353566 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2684065a90 0x7f2684067f50 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f2688006fd0 tx=0x7f2688008040 comp rx=0 tx=0).stop 2023-11-06T02:14:41.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.294+0000 7f269e337640 1 -- 172.21.15.49:0/3319353566 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26980cb010 msgr2=0x7f26981552d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:41.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.294+0000 7f269e337640 1 --2- 172.21.15.49:0/3319353566 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26980cb010 0x7f26981552d0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f268c0011f0 tx=0x7f268c00c9b0 comp rx=0 tx=0).stop 2023-11-06T02:14:41.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.294+0000 7f269e337640 1 -- 172.21.15.49:0/3319353566 shutdown_connections 2023-11-06T02:14:41.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.294+0000 7f269e337640 1 --2- 172.21.15.49:0/3319353566 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f2684065a90 0x7f2684067f50 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:41.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.294+0000 7f269e337640 1 --2- 172.21.15.49:0/3319353566 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26980cb010 0x7f26981552d0 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:41.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.294+0000 7f269e337640 1 --2- 172.21.15.49:0/3319353566 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f269815ab50 0x7f2698155810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:41.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.294+0000 7f269e337640 1 --2- 172.21.15.49:0/3319353566 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f26980b0790 0x7f26980c17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:41.293 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.294+0000 7f269e337640 1 -- 172.21.15.49:0/3319353566 >> 172.21.15.49:0/3319353566 conn(0x7f269806c0a0 msgr2=0x7f26981537a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:41.293 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.294+0000 7f269e337640 1 -- 172.21.15.49:0/3319353566 shutdown_connections 2023-11-06T02:14:41.293 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.294+0000 7f269e337640 1 -- 172.21.15.49:0/3319353566 wait complete. 2023-11-06T02:14:41.711 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 59, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-06T02:13:33.625701+0000', 'modified': '2023-11-06T02:14:40.422212+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 82, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7752}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7752': {'gid': 7752, 'name': 'c', 'rank': 0, 'incarnation': 56, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.49:6835/2241629386', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6834', 'nonce': 2241629386}, {'type': 'v1', 'addr': '172.21.15.49:6835', 'nonce': 2241629386}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-06T02:14:41.711 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-06T02:14:41.712 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-06T02:14:41.849 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.854+0000 7f4b0c372640 1 Processor -- start 2023-11-06T02:14:41.849 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.854+0000 7f4b0c372640 1 -- start start 2023-11-06T02:14:41.851 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.854+0000 7f4b0c372640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f4b0415cc50 0x7f4b040701e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:41.851 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.854+0000 7f4b0c372640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f4b040b0080 0x7f4b040b04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:41.852 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.854+0000 7f4b0c372640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b040b0a20 con 0x7f4b040c8f80 2023-11-06T02:14:41.852 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.854+0000 7f4b0c372640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b0415d030 con 0x7f4b0415cc50 2023-11-06T02:14:41.852 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.854+0000 7f4b0c372640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b040cc9c0 con 0x7f4b040b0080 2023-11-06T02:14:41.852 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.854+0000 7f4b0a10e640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f4b0415cc50 0x7f4b040701e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:41.852 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.854+0000 7f4b0a10e640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f4b0415cc50 0x7f4b040701e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47942/0 (socket says 172.21.15.49:47942) 2023-11-06T02:14:41.853 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.854+0000 7f4b0a90f640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f4b040c8f80 0x7f4b040c9410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:33586/0 (socket says 172.21.15.49:33586) 2023-11-06T02:14:41.853 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.854+0000 7f4b0a10e640 1 -- 172.21.15.49:0/1572976833 learned_addr learned my addr 172.21.15.49:0/1572976833 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:41.853 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.854+0000 7f4b0990d640 1 --2- 172.21.15.49:0/1572976833 >> v2:172.21.15.150:3300/0 conn(0x7f4b040b0080 0x7f4b040b04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:41.853 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.854+0000 7f4b0a10e640 1 -- 172.21.15.49:0/1572976833 >> v2:172.21.15.150:3300/0 conn(0x7f4b040b0080 msgr2=0x7f4b040b04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:41.853 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.854+0000 7f4b0a10e640 1 --2- 172.21.15.49:0/1572976833 >> v2:172.21.15.150:3300/0 conn(0x7f4b040b0080 0x7f4b040b04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:41.854 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.854+0000 7f4b0a10e640 1 -- 172.21.15.49:0/1572976833 >> v1:172.21.15.49:6789/0 conn(0x7f4b040c8f80 legacy=0x7f4b040c9410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:41.854 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.854+0000 7f4b0910c640 1 -- 172.21.15.49:0/1572976833 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1049860825 0 0) 0x7f4b040b0a20 con 0x7f4b040c8f80 2023-11-06T02:14:41.854 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.854+0000 7f4b0a10e640 1 -- 172.21.15.49:0/1572976833 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b040ccc60 con 0x7f4b0415cc50 2023-11-06T02:14:41.855 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.858+0000 7f4b0a10e640 1 --2- 172.21.15.49:0/1572976833 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b0415cc50 0x7f4b040701e0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f4b0406dde0 tx=0x7f4afc01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7beaedcd899aad41 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:41.856 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.858+0000 7f4b0910c640 1 -- 172.21.15.49:0/1572976833 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4afc0a5830 con 0x7f4b0415cc50 2023-11-06T02:14:41.856 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.858+0000 7f4b0910c640 1 -- 172.21.15.49:0/1572976833 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4afc0a5990 con 0x7f4b0415cc50 2023-11-06T02:14:41.856 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.858+0000 7f4b0910c640 1 -- 172.21.15.49:0/1572976833 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4afc0a5d20 con 0x7f4b0415cc50 2023-11-06T02:14:41.856 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.858+0000 7f4b0c372640 1 -- 172.21.15.49:0/1572976833 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b0415cc50 msgr2=0x7f4b040701e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:41.856 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.858+0000 7f4b0c372640 1 --2- 172.21.15.49:0/1572976833 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b0415cc50 0x7f4b040701e0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f4b0406dde0 tx=0x7f4afc01c970 comp rx=0 tx=0).stop 2023-11-06T02:14:41.857 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.858+0000 7f4b0c372640 1 -- 172.21.15.49:0/1572976833 shutdown_connections 2023-11-06T02:14:41.857 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.858+0000 7f4b0c372640 1 --2- 172.21.15.49:0/1572976833 >> v2:172.21.15.150:3300/0 conn(0x7f4b040b0080 0x7f4b040b04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:41.857 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.858+0000 7f4b0c372640 1 --2- 172.21.15.49:0/1572976833 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b0415cc50 0x7f4b040701e0 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:41.857 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.858+0000 7f4b0c372640 1 -- 172.21.15.49:0/1572976833 >> 172.21.15.49:0/1572976833 conn(0x7f4b0406c0a0 msgr2=0x7f4b0406c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:41.857 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.858+0000 7f4b0c372640 1 -- 172.21.15.49:0/1572976833 shutdown_connections 2023-11-06T02:14:41.858 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.858+0000 7f4b0c372640 1 -- 172.21.15.49:0/1572976833 wait complete. 2023-11-06T02:14:41.858 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.862+0000 7f4b0c372640 1 Processor -- start 2023-11-06T02:14:41.858 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.862+0000 7f4b0c372640 1 -- start start 2023-11-06T02:14:41.862 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.862+0000 7f4b0c372640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b040b0080 0x7f4b040c4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:41.862 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.862+0000 7f4b0c372640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4b040c8f80 0x7f4b040c7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:41.862 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.862+0000 7f4b0c372640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4b0415cc50 0x7f4b040ae190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:41.863 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.862+0000 7f4b0c372640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b040c6960 con 0x7f4b040b0080 2023-11-06T02:14:41.863 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.862+0000 7f4b0c372640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b040c6680 con 0x7f4b040c8f80 2023-11-06T02:14:41.863 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.862+0000 7f4b0c372640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b040c6eb0 con 0x7f4b0415cc50 2023-11-06T02:14:41.863 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.862+0000 7f4b0a10e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b040b0080 0x7f4b040c4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:41.863 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.862+0000 7f4b0a10e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b040b0080 0x7f4b040c4950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47944/0 (socket says 172.21.15.49:47944) 2023-11-06T02:14:41.864 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.862+0000 7f4b0a10e640 1 -- 172.21.15.49:0/2946372775 learned_addr learned my addr 172.21.15.49:0/2946372775 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:41.864 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.862+0000 7f4b0990d640 1 --2- 172.21.15.49:0/2946372775 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4b040c8f80 0x7f4b040c7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:41.864 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.862+0000 7f4b0a90f640 1 --2- 172.21.15.49:0/2946372775 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4b0415cc50 0x7f4b040ae190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:41.864 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.862+0000 7f4b0a10e640 1 -- 172.21.15.49:0/2946372775 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4b0415cc50 msgr2=0x7f4b040ae190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:41.864 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.862+0000 7f4b0a10e640 1 --2- 172.21.15.49:0/2946372775 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4b0415cc50 0x7f4b040ae190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:41.865 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.862+0000 7f4b0a10e640 1 -- 172.21.15.49:0/2946372775 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4b040c8f80 msgr2=0x7f4b040c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:41.865 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.862+0000 7f4b0a10e640 1 --2- 172.21.15.49:0/2946372775 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4b040c8f80 0x7f4b040c7990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:41.865 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.862+0000 7f4b0a10e640 1 -- 172.21.15.49:0/2946372775 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4afc009b30 con 0x7f4b040b0080 2023-11-06T02:14:41.865 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.862+0000 7f4b0a10e640 1 --2- 172.21.15.49:0/2946372775 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b040b0080 0x7f4b040c4950 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f4afc0af040 tx=0x7f4afc019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:41.865 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.862+0000 7f4b02ffd640 1 -- 172.21.15.49:0/2946372775 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4afc0b8670 con 0x7f4b040b0080 2023-11-06T02:14:41.866 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.862+0000 7f4b02ffd640 1 -- 172.21.15.49:0/2946372775 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4afc0b87d0 con 0x7f4b040b0080 2023-11-06T02:14:41.866 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.862+0000 7f4b0c372640 1 -- 172.21.15.49:0/2946372775 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b040c7190 con 0x7f4b040b0080 2023-11-06T02:14:41.866 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.862+0000 7f4b02ffd640 1 -- 172.21.15.49:0/2946372775 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4afc0b8b80 con 0x7f4b040b0080 2023-11-06T02:14:41.866 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.862+0000 7f4b0c372640 1 -- 172.21.15.49:0/2946372775 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b040c4e90 con 0x7f4b040b0080 2023-11-06T02:14:41.866 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.862+0000 7f4b02ffd640 1 -- 172.21.15.49:0/2946372775 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f4afc006d50 con 0x7f4b040b0080 2023-11-06T02:14:41.867 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.866+0000 7f4b02ffd640 1 --2- 172.21.15.49:0/2946372775 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4af0065a60 0x7f4af0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:41.867 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.866+0000 7f4b02ffd640 1 -- 172.21.15.49:0/2946372775 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6234+0+0 (secure 0 0 0) 0x7f4afc11e7b0 con 0x7f4b040b0080 2023-11-06T02:14:41.867 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.866+0000 7f4b0990d640 1 --2- 172.21.15.49:0/2946372775 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4af0065a60 0x7f4af0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:41.867 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.866+0000 7f4b0f49a000 1 Processor -- start 2023-11-06T02:14:41.867 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.866+0000 7f4b0f49a000 1 -- start start 2023-11-06T02:14:41.868 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.866+0000 7f4b0990d640 1 --2- 172.21.15.49:0/2946372775 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4af0065a60 0x7f4af0067f20 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f4b040724e0 tx=0x7f4af8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:41.868 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.866+0000 7f4b0f49a000 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56236011d6e0 0x56236011dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:41.868 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.866+0000 7f4b0f49a000 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56236011e000 0x5623600d3550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:41.868 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.866+0000 7f4b0f49a000 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5623600d3a90 0x5623600d3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:41.868 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.866+0000 7f4b0f49a000 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56235ff9bfc0 con 0x56236011e000 2023-11-06T02:14:41.868 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.866+0000 7f4b0f49a000 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x56235ff2a1a0 con 0x5623600d3a90 2023-11-06T02:14:41.869 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.866+0000 7f4b0f49a000 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x56235ff27770 con 0x56236011d6e0 2023-11-06T02:14:41.869 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.866+0000 7f4b0990d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56236011e000 0x5623600d3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:41.869 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.866+0000 7f4b0990d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56236011e000 0x5623600d3550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47948/0 (socket says 172.21.15.49:47948) 2023-11-06T02:14:41.869 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.866+0000 7f4b0990d640 1 -- 172.21.15.49:0/664446990 learned_addr learned my addr 172.21.15.49:0/664446990 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:41.869 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.866+0000 7f4b0a90f640 1 --2- 172.21.15.49:0/664446990 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5623600d3a90 0x5623600d3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:41.870 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.866+0000 7f4b0a10e640 1 --2- 172.21.15.49:0/664446990 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56236011d6e0 0x56236011dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:41.870 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.866+0000 7f4b0990d640 1 -- 172.21.15.49:0/664446990 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56236011d6e0 msgr2=0x56236011dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:41.870 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.866+0000 7f4b0990d640 1 --2- 172.21.15.49:0/664446990 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56236011d6e0 0x56236011dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:41.870 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.866+0000 7f4b0990d640 1 -- 172.21.15.49:0/664446990 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5623600d3a90 msgr2=0x5623600d3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:41.870 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.866+0000 7f4b0990d640 1 --2- 172.21.15.49:0/664446990 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5623600d3a90 0x5623600d3e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:41.871 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.866+0000 7f4b0990d640 1 -- 172.21.15.49:0/664446990 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56235fc1f890 con 0x56236011e000 2023-11-06T02:14:41.871 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.866+0000 7f4b0990d640 1 --2- 172.21.15.49:0/664446990 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56236011e000 0x5623600d3550 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f4af800fa50 tx=0x7f4af8012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=effd75e09299bcb3 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:41.871 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.866+0000 7f4aebfff640 1 -- 172.21.15.49:0/664446990 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af8014be0 con 0x56236011e000 2023-11-06T02:14:41.871 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.866+0000 7f4aebfff640 1 -- 172.21.15.49:0/664446990 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4af8019070 con 0x56236011e000 2023-11-06T02:14:41.871 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.866+0000 7f4aebfff640 1 -- 172.21.15.49:0/664446990 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af8020440 con 0x56236011e000 2023-11-06T02:14:41.876 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.866+0000 7f4b0f49a000 1 -- 172.21.15.49:0/664446990 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56236011e000 msgr2=0x5623600d3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:41.876 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.866+0000 7f4b0f49a000 1 --2- 172.21.15.49:0/664446990 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56236011e000 0x5623600d3550 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f4af800fa50 tx=0x7f4af8012000 comp rx=0 tx=0).stop 2023-11-06T02:14:41.876 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.866+0000 7f4b0f49a000 1 -- 172.21.15.49:0/664446990 shutdown_connections 2023-11-06T02:14:41.876 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.866+0000 7f4b0f49a000 1 --2- 172.21.15.49:0/664446990 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56236011e000 0x5623600d3550 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:41.876 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.866+0000 7f4b0f49a000 1 --2- 172.21.15.49:0/664446990 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5623600d3a90 0x5623600d3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:41.877 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.866+0000 7f4b0f49a000 1 --2- 172.21.15.49:0/664446990 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56236011d6e0 0x56236011dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:41.877 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.866+0000 7f4b0f49a000 1 -- 172.21.15.49:0/664446990 >> 172.21.15.49:0/664446990 conn(0x5623600d9bf0 msgr2=0x5623600d1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:41.877 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.866+0000 7f4b0f49a000 1 -- 172.21.15.49:0/664446990 shutdown_connections 2023-11-06T02:14:41.877 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.866+0000 7f4b0f49a000 1 -- 172.21.15.49:0/664446990 wait complete. 2023-11-06T02:14:41.877 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.870+0000 7f4b0f49a000 1 -- start start 2023-11-06T02:14:41.878 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.870+0000 7f4b0f49a000 1 Processor -- start 2023-11-06T02:14:41.878 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.870+0000 7f4b0f49a000 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5623600d3a90 0x5623600dea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:41.878 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.870+0000 7f4b0f49a000 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56236011d6e0 0x5623600defd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:41.878 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.870+0000 7f4b0f49a000 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56236011e000 0x5623600df510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:41.878 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.870+0000 7f4b0f49a000 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56235ff23060 con 0x56236011d6e0 2023-11-06T02:14:41.879 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.870+0000 7f4b0f49a000 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x56235fec2290 con 0x56236011e000 2023-11-06T02:14:41.879 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.870+0000 7f4b0f49a000 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x56235fee06f0 con 0x5623600d3a90 2023-11-06T02:14:41.879 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.870+0000 7f4b0f49a000 10 client.0 _ll_register_callbacks cb 0x5623600bf1d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T02:14:41.879 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.870+0000 7f4b0990d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56236011d6e0 0x5623600defd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:41.879 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.870+0000 7f4b0990d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56236011d6e0 0x5623600defd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47958/0 (socket says 172.21.15.49:47958) 2023-11-06T02:14:41.879 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.870+0000 7f4b0990d640 1 -- 172.21.15.49:0/820329121 learned_addr learned my addr 172.21.15.49:0/820329121 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:41.880 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.870+0000 7f4b0a90f640 1 --2- 172.21.15.49:0/820329121 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56236011e000 0x5623600df510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:41.880 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.870+0000 7f4b0a10e640 1 --2- 172.21.15.49:0/820329121 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5623600d3a90 0x5623600dea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:41.880 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.870+0000 7f4b0990d640 1 -- 172.21.15.49:0/820329121 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5623600d3a90 msgr2=0x5623600dea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:41.880 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.870+0000 7f4b0990d640 1 --2- 172.21.15.49:0/820329121 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5623600d3a90 0x5623600dea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:41.880 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.870+0000 7f4b0990d640 1 -- 172.21.15.49:0/820329121 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56236011e000 msgr2=0x5623600df510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:41.881 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.870+0000 7f4b0990d640 1 --2- 172.21.15.49:0/820329121 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56236011e000 0x5623600df510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:41.881 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.870+0000 7f4b0990d640 1 -- 172.21.15.49:0/820329121 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4af800e040 con 0x56236011d6e0 2023-11-06T02:14:41.881 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.870+0000 7f4b0a10e640 1 --2- 172.21.15.49:0/820329121 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5623600d3a90 0x5623600dea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:14:41.881 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.870+0000 7f4b0a90f640 1 --2- 172.21.15.49:0/820329121 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56236011e000 0x5623600df510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:14:41.881 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.870+0000 7f4b0990d640 1 --2- 172.21.15.49:0/820329121 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56236011d6e0 0x5623600defd0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f4af8022040 tx=0x7f4af8004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6449c999bf7f81fc server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:41.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.870+0000 7f4ae97fa640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-11-06T02:14:41.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.870+0000 7f4ae97fa640 1 -- 172.21.15.49:0/820329121 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af8020c00 con 0x56236011d6e0 2023-11-06T02:14:41.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.870+0000 7f4ae97fa640 1 -- 172.21.15.49:0/820329121 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 6234+0+0 (secure 0 0 0) 0x7f4af800b1c0 con 0x56236011d6e0 2023-11-06T02:14:41.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.870+0000 7f4b0f49a000 1 -- 172.21.15.49:0/820329121 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56235ff219b0 con 0x56236011d6e0 2023-11-06T02:14:41.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.870+0000 7f4ae97fa640 1 -- 172.21.15.49:0/820329121 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4af8020d60 con 0x56236011d6e0 2023-11-06T02:14:41.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.870+0000 7f4b0f49a000 10 client.7848 fetch_fsmap learned FSMap version 59 2023-11-06T02:14:41.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.870+0000 7f4b0f49a000 1 -- 172.21.15.49:0/820329121 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=2+}) v3 -- 0x56235fee81e0 con 0x56236011d6e0 2023-11-06T02:14:41.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.870+0000 7f4ae97fa640 1 -- 172.21.15.49:0/820329121 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56235ff219b0 con 0x56236011d6e0 2023-11-06T02:14:41.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.870+0000 7f4ae97fa640 1 -- 172.21.15.49:0/820329121 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 59) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f4af8033470 con 0x56236011d6e0 2023-11-06T02:14:41.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.870+0000 7f4b0f49a000 10 client.7848 fetch_fsmap finished waiting for FSMap version 59 2023-11-06T02:14:41.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.870+0000 7f4b0f49a000 10 client.7848 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-06T02:14:41.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.870+0000 7f4b0f49a000 20 client.7848 populate_metadata read hostname 'smithi049' 2023-11-06T02:14:41.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.870+0000 7f4b0f49a000 1 --2- 172.21.15.49:0/820329121 >> [v2:172.21.15.49:6834/2241629386,v1:172.21.15.49:6835/2241629386] conn(0x5623600dfdb0 0x56236012ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:41.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.870+0000 7f4b0f49a000 4 client.7848 mds_command: new command op to 7752 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T02:14:41.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.870+0000 7f4b0f49a000 1 -- 172.21.15.49:0/820329121 --> [v2:172.21.15.49:6834/2241629386,v1:172.21.15.49:6835/2241629386] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56235ff524d0 con 0x5623600dfdb0 2023-11-06T02:14:41.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.870+0000 7f4b0a10e640 1 --2- 172.21.15.49:0/820329121 >> [v2:172.21.15.49:6834/2241629386,v1:172.21.15.49:6835/2241629386] conn(0x5623600dfdb0 0x56236012ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:41.885 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.870+0000 7f4b0a10e640 1 --2- 172.21.15.49:0/820329121 >> [v2:172.21.15.49:6834/2241629386,v1:172.21.15.49:6835/2241629386] conn(0x5623600dfdb0 0x56236012ec50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4a3d81aeef60578 server_cookie=c56fb9fdbe7bd6c9 in_seq=0 out_seq=0 2023-11-06T02:14:41.885 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.870+0000 7f4ae97fa640 10 client.7848 ms_handle_connect on v2:172.21.15.49:6834/2241629386 2023-11-06T02:14:41.885 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.874+0000 7f4ae97fa640 1 -- 172.21.15.49:0/820329121 <== mds.0 v2:172.21.15.49:6834/2241629386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x56235ff524d0 con 0x5623600dfdb0 2023-11-06T02:14:41.885 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.874+0000 7f4ae97fa640 10 client.7848 handle_command_reply: tid=0 2023-11-06T02:14:41.885 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.874+0000 7f4b0f49a000 1 client.7848 shutdown 2023-11-06T02:14:41.886 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.874+0000 7f4b0f49a000 1 -- 172.21.15.49:0/820329121 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56236011d6e0 msgr2=0x5623600defd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:41.886 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.874+0000 7f4b0f49a000 1 --2- 172.21.15.49:0/820329121 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56236011d6e0 0x5623600defd0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f4af8022040 tx=0x7f4af8004740 comp rx=0 tx=0).stop 2023-11-06T02:14:41.886 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.874+0000 7f4b0f49a000 1 -- 172.21.15.49:0/820329121 shutdown_connections 2023-11-06T02:14:41.886 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.874+0000 7f4b0f49a000 1 --2- 172.21.15.49:0/820329121 >> [v2:172.21.15.49:6834/2241629386,v1:172.21.15.49:6835/2241629386] conn(0x5623600dfdb0 0x56236012ec50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:41.886 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.874+0000 7f4b0f49a000 1 --2- 172.21.15.49:0/820329121 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56236011d6e0 0x5623600defd0 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:41.887 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.874+0000 7f4b0f49a000 1 --2- 172.21.15.49:0/820329121 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56236011e000 0x5623600df510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:41.887 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.874+0000 7f4ae97fa640 0 client.7848 ms_handle_reset on v2:172.21.15.49:6834/2241629386 2023-11-06T02:14:41.887 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.874+0000 7f4b0f49a000 1 --2- 172.21.15.49:0/820329121 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5623600d3a90 0x5623600dea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:41.887 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.874+0000 7f4b0f49a000 1 -- 172.21.15.49:0/820329121 >> 172.21.15.49:0/820329121 conn(0x5623600d9bf0 msgr2=0x5623600d2530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:41.887 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.874+0000 7f4b0f49a000 1 -- 172.21.15.49:0/820329121 shutdown_connections 2023-11-06T02:14:41.887 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.874+0000 7f4b0f49a000 1 -- 172.21.15.49:0/820329121 wait complete. 2023-11-06T02:14:41.888 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.874+0000 7f4b0f49a000 20 client.7848 trim_cache size 0 max 16384 2023-11-06T02:14:41.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.886+0000 7f4b0f49a000 1 Processor -- start 2023-11-06T02:14:41.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.886+0000 7f4b0f49a000 1 -- start start 2023-11-06T02:14:41.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.890+0000 7f4b0f49a000 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623600d3a90 0x562360130370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:41.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.890+0000 7f4b0f49a000 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56236011d6e0 0x5623601308b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:41.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.890+0000 7f4b0f49a000 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56236011e000 0x562360116340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:41.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.890+0000 7f4b0f49a000 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56235fbe0940 con 0x5623600d3a90 2023-11-06T02:14:41.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.890+0000 7f4b0f49a000 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x56235ff66560 con 0x56236011e000 2023-11-06T02:14:41.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.890+0000 7f4b0f49a000 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x56235ff17d30 con 0x56236011d6e0 2023-11-06T02:14:41.895 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.890+0000 7f4b0a10e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623600d3a90 0x562360130370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:41.895 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.890+0000 7f4b0a10e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623600d3a90 0x562360130370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47966/0 (socket says 172.21.15.49:47966) 2023-11-06T02:14:41.895 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.890+0000 7f4b0a10e640 1 -- 172.21.15.49:0/479146942 learned_addr learned my addr 172.21.15.49:0/479146942 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:41.895 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.890+0000 7f4b0a10e640 1 -- 172.21.15.49:0/479146942 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56236011d6e0 msgr2=0x5623601308b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:41.895 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.890+0000 7f4b0990d640 1 --2- 172.21.15.49:0/479146942 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56236011d6e0 0x5623601308b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:41.896 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.890+0000 7f4b0a90f640 1 --2- 172.21.15.49:0/479146942 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56236011e000 0x562360116340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:41.896 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.890+0000 7f4b0a10e640 1 --2- 172.21.15.49:0/479146942 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56236011d6e0 0x5623601308b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:41.896 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.890+0000 7f4b0a10e640 1 -- 172.21.15.49:0/479146942 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56236011e000 msgr2=0x562360116340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:41.896 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.890+0000 7f4b0a10e640 1 --2- 172.21.15.49:0/479146942 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56236011e000 0x562360116340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:41.896 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.890+0000 7f4b0a10e640 1 -- 172.21.15.49:0/479146942 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56235fece330 con 0x5623600d3a90 2023-11-06T02:14:41.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.890+0000 7f4b0a90f640 1 --2- 172.21.15.49:0/479146942 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56236011e000 0x562360116340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:14:41.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.890+0000 7f4b0a10e640 1 --2- 172.21.15.49:0/479146942 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623600d3a90 0x562360130370 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f4afc0ec120 tx=0x7f4afc0ba040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=787348ec4fa5dda9 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:41.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.890+0000 7f4b00ff9640 1 -- 172.21.15.49:0/479146942 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4afc11cbe0 con 0x5623600d3a90 2023-11-06T02:14:41.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.890+0000 7f4b00ff9640 1 -- 172.21.15.49:0/479146942 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4afc0edd60 con 0x5623600d3a90 2023-11-06T02:14:41.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.890+0000 7f4b00ff9640 1 -- 172.21.15.49:0/479146942 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4afc0ec330 con 0x5623600d3a90 2023-11-06T02:14:41.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.890+0000 7f4b0f49a000 1 -- 172.21.15.49:0/479146942 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623600d3a90 msgr2=0x562360130370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:41.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.890+0000 7f4b0f49a000 1 --2- 172.21.15.49:0/479146942 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623600d3a90 0x562360130370 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f4afc0ec120 tx=0x7f4afc0ba040 comp rx=0 tx=0).stop 2023-11-06T02:14:41.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.890+0000 7f4b0f49a000 1 -- 172.21.15.49:0/479146942 shutdown_connections 2023-11-06T02:14:41.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.890+0000 7f4b0f49a000 1 --2- 172.21.15.49:0/479146942 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56236011e000 0x562360116340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:41.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.890+0000 7f4b0f49a000 1 --2- 172.21.15.49:0/479146942 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56236011d6e0 0x5623601308b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:41.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.890+0000 7f4b0f49a000 1 --2- 172.21.15.49:0/479146942 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623600d3a90 0x562360130370 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:41.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.890+0000 7f4b0f49a000 1 -- 172.21.15.49:0/479146942 >> 172.21.15.49:0/479146942 conn(0x5623600d9bf0 msgr2=0x56236012ec50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:41.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.890+0000 7f4b0f49a000 1 -- 172.21.15.49:0/479146942 shutdown_connections 2023-11-06T02:14:41.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.890+0000 7f4b0f49a000 1 -- 172.21.15.49:0/479146942 wait complete. 2023-11-06T02:14:41.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.890+0000 7f4b0f49a000 1 -- start start 2023-11-06T02:14:41.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.890+0000 7f4b0f49a000 1 Processor -- start 2023-11-06T02:14:41.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.890+0000 7f4b0f49a000 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5623600d3a90 0x5623601799a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:41.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.890+0000 7f4b0f49a000 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56236011d6e0 0x562360179ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:41.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.890+0000 7f4b0f49a000 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56236011e000 0x5623601190c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:41.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.890+0000 7f4b0f49a000 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56235fee4e40 con 0x56236011e000 2023-11-06T02:14:41.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.890+0000 7f4b0f49a000 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x56235fef1ec0 con 0x56236011d6e0 2023-11-06T02:14:41.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.890+0000 7f4b0f49a000 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x56235fee6ff0 con 0x5623600d3a90 2023-11-06T02:14:41.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.890+0000 7f4b0f49a000 10 client.0 _ll_register_callbacks cb 0x56236009f9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T02:14:41.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.890+0000 7f4b0a90f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56236011e000 0x5623601190c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:41.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.890+0000 7f4b0a90f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56236011e000 0x5623601190c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47968/0 (socket says 172.21.15.49:47968) 2023-11-06T02:14:41.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.890+0000 7f4b0a90f640 1 -- 172.21.15.49:0/3577290536 learned_addr learned my addr 172.21.15.49:0/3577290536 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:41.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.890+0000 7f4b0990d640 1 --2- 172.21.15.49:0/3577290536 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56236011d6e0 0x562360179ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:41.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.890+0000 7f4b0a10e640 1 --2- 172.21.15.49:0/3577290536 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5623600d3a90 0x5623601799a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:41.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.890+0000 7f4b0a90f640 1 -- 172.21.15.49:0/3577290536 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5623600d3a90 msgr2=0x5623601799a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:41.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.890+0000 7f4b0a90f640 1 --2- 172.21.15.49:0/3577290536 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5623600d3a90 0x5623601799a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:41.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.894+0000 7f4b0a90f640 1 -- 172.21.15.49:0/3577290536 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56236011d6e0 msgr2=0x562360179ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:41.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.894+0000 7f4b0a90f640 1 --2- 172.21.15.49:0/3577290536 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56236011d6e0 0x562360179ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:41.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.894+0000 7f4b0a90f640 1 -- 172.21.15.49:0/3577290536 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b040c4e90 con 0x56236011e000 2023-11-06T02:14:41.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.894+0000 7f4b0a90f640 1 --2- 172.21.15.49:0/3577290536 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56236011e000 0x5623601190c0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f4af4005b30 tx=0x7f4af4007670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c413635499724d8a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:41.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.894+0000 7f4ae97fa640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-11-06T02:14:41.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.894+0000 7f4ae97fa640 1 -- 172.21.15.49:0/3577290536 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af4001cf0 con 0x56236011e000 2023-11-06T02:14:41.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.894+0000 7f4b0f49a000 1 -- 172.21.15.49:0/3577290536 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56235fef2b90 con 0x56236011e000 2023-11-06T02:14:41.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.894+0000 7f4ae97fa640 1 -- 172.21.15.49:0/3577290536 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 6234+0+0 (secure 0 0 0) 0x7f4af4016480 con 0x56236011e000 2023-11-06T02:14:41.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.894+0000 7f4ae97fa640 1 -- 172.21.15.49:0/3577290536 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4af4016640 con 0x56236011e000 2023-11-06T02:14:41.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.894+0000 7f4b0f49a000 10 client.7854 fetch_fsmap learned FSMap version 59 2023-11-06T02:14:41.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.894+0000 7f4b0f49a000 1 -- 172.21.15.49:0/3577290536 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=2+}) v3 -- 0x56235fc1fcc0 con 0x56236011e000 2023-11-06T02:14:41.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.894+0000 7f4ae97fa640 1 -- 172.21.15.49:0/3577290536 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56235fef2b90 con 0x56236011e000 2023-11-06T02:14:41.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.894+0000 7f4ae97fa640 1 -- 172.21.15.49:0/3577290536 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 59) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f4af40167a0 con 0x56236011e000 2023-11-06T02:14:41.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.894+0000 7f4b0f49a000 10 client.7854 fetch_fsmap finished waiting for FSMap version 59 2023-11-06T02:14:41.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.894+0000 7f4b0f49a000 10 client.7854 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-06T02:14:41.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.894+0000 7f4b0f49a000 20 client.7854 populate_metadata read hostname 'smithi049' 2023-11-06T02:14:41.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.894+0000 7f4b0f49a000 1 --2- 172.21.15.49:0/3577290536 >> [v2:172.21.15.49:6834/2241629386,v1:172.21.15.49:6835/2241629386] conn(0x562360119600 0x5623601199e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:41.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.894+0000 7f4b0f49a000 4 client.7854 mds_command: new command op to 7752 tid=0[{"prefix": "status"}] 2023-11-06T02:14:41.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.894+0000 7f4b0f49a000 1 -- 172.21.15.49:0/3577290536 --> [v2:172.21.15.49:6834/2241629386,v1:172.21.15.49:6835/2241629386] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56235ff1c0e0 con 0x562360119600 2023-11-06T02:14:41.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.894+0000 7f4b0a10e640 1 --2- 172.21.15.49:0/3577290536 >> [v2:172.21.15.49:6834/2241629386,v1:172.21.15.49:6835/2241629386] conn(0x562360119600 0x5623601199e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:41.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.894+0000 7f4ae97fa640 10 client.7854 ms_handle_connect on v2:172.21.15.49:6834/2241629386 2023-11-06T02:14:41.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.894+0000 7f4b0a10e640 1 --2- 172.21.15.49:0/3577290536 >> [v2:172.21.15.49:6834/2241629386,v1:172.21.15.49:6835/2241629386] conn(0x562360119600 0x5623601199e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4598e9ec5de8b034 server_cookie=e9a6f5f2fa22330c in_seq=0 out_seq=0 2023-11-06T02:14:41.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.894+0000 7f4ae97fa640 1 -- 172.21.15.49:0/3577290536 <== mds.0 v2:172.21.15.49:6834/2241629386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x56235ff1c0e0 con 0x562360119600 2023-11-06T02:14:41.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.894+0000 7f4ae97fa640 10 client.7854 handle_command_reply: tid=0 2023-11-06T02:14:41.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.894+0000 7f4b0f49a000 1 client.7854 shutdown 2023-11-06T02:14:41.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.894+0000 7f4b0f49a000 1 -- 172.21.15.49:0/3577290536 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56236011e000 msgr2=0x5623601190c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:41.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.894+0000 7f4b0f49a000 1 --2- 172.21.15.49:0/3577290536 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56236011e000 0x5623601190c0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f4af4005b30 tx=0x7f4af4007670 comp rx=0 tx=0).stop 2023-11-06T02:14:41.908 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.894+0000 7f4b0f49a000 1 -- 172.21.15.49:0/3577290536 shutdown_connections 2023-11-06T02:14:41.908 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.894+0000 7f4b0f49a000 1 --2- 172.21.15.49:0/3577290536 >> [v2:172.21.15.49:6834/2241629386,v1:172.21.15.49:6835/2241629386] conn(0x562360119600 0x5623601199e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:41.908 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.894+0000 7f4b0f49a000 1 --2- 172.21.15.49:0/3577290536 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56236011e000 0x5623601190c0 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:41.908 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.894+0000 7f4b0f49a000 1 --2- 172.21.15.49:0/3577290536 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56236011d6e0 0x562360179ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:41.908 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.894+0000 7f4ae97fa640 0 client.7854 ms_handle_reset on v2:172.21.15.49:6834/2241629386 2023-11-06T02:14:41.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.894+0000 7f4b0f49a000 1 --2- 172.21.15.49:0/3577290536 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5623600d3a90 0x5623601799a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:41.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.894+0000 7f4b0f49a000 1 -- 172.21.15.49:0/3577290536 >> 172.21.15.49:0/3577290536 conn(0x5623600d9bf0 msgr2=0x56236012ec50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:41.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.894+0000 7f4b0f49a000 1 -- 172.21.15.49:0/3577290536 shutdown_connections 2023-11-06T02:14:41.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.894+0000 7f4b0f49a000 1 -- 172.21.15.49:0/3577290536 wait complete. 2023-11-06T02:14:41.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.894+0000 7f4b0f49a000 20 client.7854 trim_cache size 0 max 16384 2023-11-06T02:14:41.910 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-11-06T02:14:41.910 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_fsid": "1a8b035c-6ba4-499a-b876-aea47fd9c0d4", 2023-11-06T02:14:41.910 INFO:teuthology.orchestra.run.smithi049.stdout: "whoami": 0, 2023-11-06T02:14:41.910 INFO:teuthology.orchestra.run.smithi049.stdout: "id": 7752, 2023-11-06T02:14:41.911 INFO:teuthology.orchestra.run.smithi049.stdout: "want_state": "up:active", 2023-11-06T02:14:41.911 INFO:teuthology.orchestra.run.smithi049.stdout: "state": "up:active", 2023-11-06T02:14:41.911 INFO:teuthology.orchestra.run.smithi049.stdout: "fs_name": "cephfs", 2023-11-06T02:14:41.911 INFO:teuthology.orchestra.run.smithi049.stdout: "rank_uptime": 4.4881010830000001, 2023-11-06T02:14:41.911 INFO:teuthology.orchestra.run.smithi049.stdout: "mdsmap_epoch": 59, 2023-11-06T02:14:41.912 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch": 82, 2023-11-06T02:14:41.912 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch_barrier": 82, 2023-11-06T02:14:41.912 INFO:teuthology.orchestra.run.smithi049.stdout: "uptime": 6.4980919979999996 2023-11-06T02:14:41.912 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-11-06T02:14:41.913 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.894+0000 7f4b0c372640 1 -- 172.21.15.49:0/2946372775 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4af0065a60 msgr2=0x7f4af0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:41.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.894+0000 7f4b0c372640 1 --2- 172.21.15.49:0/2946372775 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4af0065a60 0x7f4af0067f20 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f4b040724e0 tx=0x7f4af8008040 comp rx=0 tx=0).stop 2023-11-06T02:14:41.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.894+0000 7f4b0c372640 1 -- 172.21.15.49:0/2946372775 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b040b0080 msgr2=0x7f4b040c4950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:41.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.894+0000 7f4b0c372640 1 --2- 172.21.15.49:0/2946372775 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b040b0080 0x7f4b040c4950 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f4afc0af040 tx=0x7f4afc019710 comp rx=0 tx=0).stop 2023-11-06T02:14:41.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.894+0000 7f4b0c372640 1 -- 172.21.15.49:0/2946372775 shutdown_connections 2023-11-06T02:14:41.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.894+0000 7f4b0c372640 1 --2- 172.21.15.49:0/2946372775 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4af0065a60 0x7f4af0067f20 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:41.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.894+0000 7f4b0c372640 1 --2- 172.21.15.49:0/2946372775 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4b0415cc50 0x7f4b040ae190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:41.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.894+0000 7f4b0c372640 1 --2- 172.21.15.49:0/2946372775 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4b040c8f80 0x7f4b040c7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:41.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.894+0000 7f4b0c372640 1 --2- 172.21.15.49:0/2946372775 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b040b0080 0x7f4b040c4950 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:41.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.894+0000 7f4b0c372640 1 -- 172.21.15.49:0/2946372775 >> 172.21.15.49:0/2946372775 conn(0x7f4b0406c0a0 msgr2=0x7f4b0414fdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:41.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.894+0000 7f4b0c372640 1 -- 172.21.15.49:0/2946372775 shutdown_connections 2023-11-06T02:14:41.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:41.894+0000 7f4b0c372640 1 -- 172.21.15.49:0/2946372775 wait complete. 2023-11-06T02:14:41.916 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-06T02:14:42.032 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:42.038+0000 7f014c62a640 1 Processor -- start 2023-11-06T02:14:42.032 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:42.038+0000 7f014c62a640 1 -- start start 2023-11-06T02:14:42.033 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:42.038+0000 7f014c62a640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f014415ad50 0x7f01440ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:42.034 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:42.038+0000 7f014c62a640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f01440cebb0 0x7f01440b1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:42.034 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:42.038+0000 7f014c62a640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01440b0d70 con 0x7f014415a970 2023-11-06T02:14:42.034 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:42.038+0000 7f014c62a640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01440cf020 con 0x7f014415ad50 2023-11-06T02:14:42.034 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:42.038+0000 7f014c62a640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01440b01f0 con 0x7f01440cebb0 2023-11-06T02:14:42.035 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:42.038+0000 7f0149bc5640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f014415ad50 0x7f01440ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:42.035 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:42.038+0000 7f0149bc5640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f014415ad50 0x7f01440ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47978/0 (socket says 172.21.15.49:47978) 2023-11-06T02:14:42.035 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:42.038+0000 7f0149bc5640 1 -- 172.21.15.49:0/2341775335 learned_addr learned my addr 172.21.15.49:0/2341775335 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:42.035 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:42.038+0000 7f014abc7640 1 --2- 172.21.15.49:0/2341775335 >> v2:172.21.15.150:3300/0 conn(0x7f01440cebb0 0x7f01440b1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:42.035 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:42.038+0000 7f0149bc5640 1 -- 172.21.15.49:0/2341775335 >> v2:172.21.15.150:3300/0 conn(0x7f01440cebb0 msgr2=0x7f01440b1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:42.035 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:42.038+0000 7f0149bc5640 1 --2- 172.21.15.49:0/2341775335 >> v2:172.21.15.150:3300/0 conn(0x7f01440cebb0 0x7f01440b1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:42.036 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:42.038+0000 7f0149bc5640 1 -- 172.21.15.49:0/2341775335 >> v1:172.21.15.150:6789/0 conn(0x7f014415a970 legacy=0x7f01440cd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:42.036 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:42.038+0000 7f0149bc5640 1 -- 172.21.15.49:0/2341775335 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01440b0440 con 0x7f014415ad50 2023-11-06T02:14:42.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:42.042+0000 7f0149bc5640 1 --2- 172.21.15.49:0/2341775335 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f014415ad50 0x7f01440ce670 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f0144071d70 tx=0x7f013801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f050dffb5a0239d7 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:42.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:42.042+0000 7f01493c4640 1 -- 172.21.15.49:0/2341775335 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01380aa5f0 con 0x7f014415ad50 2023-11-06T02:14:42.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:42.042+0000 7f01493c4640 1 -- 172.21.15.49:0/2341775335 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01380aa750 con 0x7f014415ad50 2023-11-06T02:14:42.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:42.042+0000 7f01493c4640 1 -- 172.21.15.49:0/2341775335 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01380aaae0 con 0x7f014415ad50 2023-11-06T02:14:42.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:42.042+0000 7f014c62a640 1 -- 172.21.15.49:0/2341775335 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f014415ad50 msgr2=0x7f01440ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:42.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:42.042+0000 7f014c62a640 1 --2- 172.21.15.49:0/2341775335 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f014415ad50 0x7f01440ce670 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f0144071d70 tx=0x7f013801c490 comp rx=0 tx=0).stop 2023-11-06T02:14:42.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:42.042+0000 7f014c62a640 1 -- 172.21.15.49:0/2341775335 shutdown_connections 2023-11-06T02:14:42.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:42.042+0000 7f014c62a640 1 --2- 172.21.15.49:0/2341775335 >> v2:172.21.15.150:3300/0 conn(0x7f01440cebb0 0x7f01440b1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:42.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:42.042+0000 7f014c62a640 1 --2- 172.21.15.49:0/2341775335 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f014415ad50 0x7f01440ce670 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:42.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:42.042+0000 7f014c62a640 1 -- 172.21.15.49:0/2341775335 >> 172.21.15.49:0/2341775335 conn(0x7f01440d0890 msgr2=0x7f01440d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:42.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:42.042+0000 7f014c62a640 1 -- 172.21.15.49:0/2341775335 shutdown_connections 2023-11-06T02:14:42.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:42.042+0000 7f014c62a640 1 -- 172.21.15.49:0/2341775335 wait complete. 2023-11-06T02:14:42.042 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:42.042+0000 7f014c62a640 1 Processor -- start 2023-11-06T02:14:42.042 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:42.042+0000 7f014c62a640 1 -- start start 2023-11-06T02:14:42.043 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:42.046+0000 7f014c62a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f01440cebb0 0x7f014414b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:42.043 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:42.046+0000 7f014c62a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f014415a970 0x7f01440c89f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:42.043 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:42.046+0000 7f014c62a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f014415ad50 0x7f014406f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:42.043 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:42.046+0000 7f014c62a640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01440ca640 con 0x7f01440cebb0 2023-11-06T02:14:42.043 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:42.046+0000 7f014c62a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01440ae510 con 0x7f014415ad50 2023-11-06T02:14:42.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:42.046+0000 7f014c62a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f014414be90 con 0x7f014415a970 2023-11-06T02:14:42.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:42.046+0000 7f014a3c6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f01440cebb0 0x7f014414b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:42.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:42.046+0000 7f014a3c6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f01440cebb0 0x7f014414b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47984/0 (socket says 172.21.15.49:47984) 2023-11-06T02:14:42.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:42.046+0000 7f014a3c6640 1 -- 172.21.15.49:0/2527538944 learned_addr learned my addr 172.21.15.49:0/2527538944 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:42.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:42.046+0000 7f0149bc5640 1 --2- 172.21.15.49:0/2527538944 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f014415a970 0x7f01440c89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:42.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:42.046+0000 7f014abc7640 1 --2- 172.21.15.49:0/2527538944 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f014415ad50 0x7f014406f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:42.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:42.046+0000 7f014a3c6640 1 -- 172.21.15.49:0/2527538944 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f014415a970 msgr2=0x7f01440c89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:42.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:42.046+0000 7f014a3c6640 1 --2- 172.21.15.49:0/2527538944 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f014415a970 0x7f01440c89f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:42.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:42.046+0000 7f014a3c6640 1 -- 172.21.15.49:0/2527538944 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f014415ad50 msgr2=0x7f014406f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:42.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:42.046+0000 7f014a3c6640 1 --2- 172.21.15.49:0/2527538944 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f014415ad50 0x7f014406f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:42.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:42.046+0000 7f014a3c6640 1 -- 172.21.15.49:0/2527538944 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01380094d0 con 0x7f01440cebb0 2023-11-06T02:14:42.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:42.046+0000 7f014abc7640 1 --2- 172.21.15.49:0/2527538944 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f014415ad50 0x7f014406f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:14:42.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:42.046+0000 7f014a3c6640 1 --2- 172.21.15.49:0/2527538944 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f01440cebb0 0x7f014414b680 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f013400c7c0 tx=0x7f013400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:42.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:42.046+0000 7f0142ffd640 1 -- 172.21.15.49:0/2527538944 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f013400aa50 con 0x7f01440cebb0 2023-11-06T02:14:42.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:42.046+0000 7f0142ffd640 1 -- 172.21.15.49:0/2527538944 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f013400abb0 con 0x7f01440cebb0 2023-11-06T02:14:42.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:42.046+0000 7f014c62a640 1 -- 172.21.15.49:0/2527538944 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01440c90d0 con 0x7f01440cebb0 2023-11-06T02:14:42.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:42.046+0000 7f0142ffd640 1 -- 172.21.15.49:0/2527538944 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0134012420 con 0x7f01440cebb0 2023-11-06T02:14:42.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:42.046+0000 7f014c62a640 1 -- 172.21.15.49:0/2527538944 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01440adec0 con 0x7f01440cebb0 2023-11-06T02:14:42.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:42.046+0000 7f014c62a640 1 -- 172.21.15.49:0/2527538944 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0144074040 con 0x7f01440cebb0 2023-11-06T02:14:42.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:42.046+0000 7f0142ffd640 1 -- 172.21.15.49:0/2527538944 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f0134006bb0 con 0x7f01440cebb0 2023-11-06T02:14:42.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:42.046+0000 7f0142ffd640 1 --2- 172.21.15.49:0/2527538944 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0130065a60 0x7f0130067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:42.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:42.046+0000 7f0142ffd640 1 -- 172.21.15.49:0/2527538944 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6234+0+0 (secure 0 0 0) 0x7f0134084110 con 0x7f01440cebb0 2023-11-06T02:14:42.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:42.046+0000 7f0149bc5640 1 --2- 172.21.15.49:0/2527538944 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0130065a60 0x7f0130067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:42.049 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:42.050+0000 7f0149bc5640 1 --2- 172.21.15.49:0/2527538944 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0130065a60 0x7f0130067f20 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f01380ae000 tx=0x7f01380a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:42.049 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:42.050+0000 7f0142ffd640 1 -- 172.21.15.49:0/2527538944 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f01340518a0 con 0x7f01440cebb0 2023-11-06T02:14:42.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:42.234+0000 7f014c62a640 1 -- 172.21.15.49:0/2527538944 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f01440b14d0 con 0x7f01440cebb0 2023-11-06T02:14:42.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:42.234+0000 7f0142ffd640 1 -- 172.21.15.49:0/2527538944 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v59) v1 ==== 53+0+73 (secure 0 0 0) 0x7f0134016070 con 0x7f01440cebb0 2023-11-06T02:14:42.231 INFO:teuthology.orchestra.run.smithi049.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-06T02:14:42.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:42.238+0000 7f014c62a640 1 -- 172.21.15.49:0/2527538944 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0130065a60 msgr2=0x7f0130067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:42.234 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:42.238+0000 7f014c62a640 1 --2- 172.21.15.49:0/2527538944 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0130065a60 0x7f0130067f20 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f01380ae000 tx=0x7f01380a7000 comp rx=0 tx=0).stop 2023-11-06T02:14:42.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:42.238+0000 7f014c62a640 1 -- 172.21.15.49:0/2527538944 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f01440cebb0 msgr2=0x7f014414b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:42.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:42.238+0000 7f014c62a640 1 --2- 172.21.15.49:0/2527538944 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f01440cebb0 0x7f014414b680 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f013400c7c0 tx=0x7f013400cc80 comp rx=0 tx=0).stop 2023-11-06T02:14:42.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:42.238+0000 7f014c62a640 1 -- 172.21.15.49:0/2527538944 shutdown_connections 2023-11-06T02:14:42.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:42.238+0000 7f014c62a640 1 --2- 172.21.15.49:0/2527538944 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0130065a60 0x7f0130067f20 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:42.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:42.238+0000 7f014c62a640 1 --2- 172.21.15.49:0/2527538944 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f014415ad50 0x7f014406f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:42.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:42.238+0000 7f014c62a640 1 --2- 172.21.15.49:0/2527538944 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f014415a970 0x7f01440c89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:42.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:42.238+0000 7f014c62a640 1 --2- 172.21.15.49:0/2527538944 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f01440cebb0 0x7f014414b680 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:42.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:42.238+0000 7f014c62a640 1 -- 172.21.15.49:0/2527538944 >> 172.21.15.49:0/2527538944 conn(0x7f01440d0890 msgr2=0x7f01440cc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:42.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:42.238+0000 7f014c62a640 1 -- 172.21.15.49:0/2527538944 shutdown_connections 2023-11-06T02:14:42.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:42.238+0000 7f014c62a640 1 -- 172.21.15.49:0/2527538944 wait complete. 2023-11-06T02:14:42.249 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-06T02:14:42.249 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-06T02:14:42.250 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi049.front.sepia.ceph.com 2023-11-06T02:14:42.250 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-06T02:14:42.250 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-06T02:14:42.251 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-06T02:14:42.251 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-06T02:14:42.251 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-06T02:14:42.252 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-06T02:14:42.252 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:14:42.253 DEBUG:teuthology.orchestra.run.smithi049:> ip addr 2023-11-06T02:14:42.259 INFO:teuthology.orchestra.run.smithi049.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-06T02:14:42.259 INFO:teuthology.orchestra.run.smithi049.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-06T02:14:42.260 INFO:teuthology.orchestra.run.smithi049.stdout: inet 127.0.0.1/8 scope host lo 2023-11-06T02:14:42.260 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:14:42.260 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 ::1/128 scope host 2023-11-06T02:14:42.260 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:14:42.260 INFO:teuthology.orchestra.run.smithi049.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-06T02:14:42.260 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:6b:ff:84 brd ff:ff:ff:ff:ff:ff 2023-11-06T02:14:42.261 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp7s0f0 2023-11-06T02:14:42.261 INFO:teuthology.orchestra.run.smithi049.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-06T02:14:42.261 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:6b:ff:85 brd ff:ff:ff:ff:ff:ff 2023-11-06T02:14:42.261 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp7s0f1 2023-11-06T02:14:42.261 INFO:teuthology.orchestra.run.smithi049.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-06T02:14:42.262 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:bd:15:66 brd ff:ff:ff:ff:ff:ff 2023-11-06T02:14:42.262 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp1s0f0 2023-11-06T02:14:42.262 INFO:teuthology.orchestra.run.smithi049.stdout: inet 172.21.15.49/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-11-06T02:14:42.262 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft 42115sec preferred_lft 42115sec 2023-11-06T02:14:42.263 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 fe80::ec4:7aff:febd:1566/64 scope link 2023-11-06T02:14:42.263 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:14:42.263 INFO:teuthology.orchestra.run.smithi049.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-06T02:14:42.263 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:bd:15:67 brd ff:ff:ff:ff:ff:ff 2023-11-06T02:14:42.263 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp1s0f1 2023-11-06T02:14:42.264 INFO:teuthology.orchestra.run.smithi049.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-06T02:14:42.264 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 02:42:47:49:33:69 brd ff:ff:ff:ff:ff:ff 2023-11-06T02:14:42.264 INFO:teuthology.orchestra.run.smithi049.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-06T02:14:42.264 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:14:42.264 INFO:teuthology.orchestra.run.smithi049.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-06T02:14:42.265 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-11-06T02:14:42.265 INFO:teuthology.orchestra.run.smithi049.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-06T02:14:42.265 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:14:42.265 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-11-06T02:14:42.265 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:14:42.266 INFO:teuthology.orchestra.run.smithi049.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-06T02:14:42.266 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-06T02:14:42.266 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-11-06T02:14:42.266 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:14:42.266 INFO:teuthology.orchestra.run.smithi049.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-06T02:14:42.266 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-06T02:14:42.267 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-11-06T02:14:42.267 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-11-06T02:14:42.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:14:42.268 DEBUG:teuthology.orchestra.run.smithi049:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-06T02:14:42.314 INFO:teuthology.orchestra.run.smithi049.stdout:1 2023-11-06T02:14:42.315 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:14:42.315 DEBUG:teuthology.orchestra.run.smithi049:> route 2023-11-06T02:14:42.365 INFO:teuthology.orchestra.run.smithi049.stdout:Kernel IP routing table 2023-11-06T02:14:42.365 INFO:teuthology.orchestra.run.smithi049.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-06T02:14:42.365 INFO:teuthology.orchestra.run.smithi049.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-11-06T02:14:42.366 INFO:teuthology.orchestra.run.smithi049.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-06T02:14:42.366 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-11-06T02:14:42.366 INFO:teuthology.orchestra.run.smithi049.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-06T02:14:42.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:14:42.367 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec bash -c ' 2023-11-06T02:14:42.367 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2023-11-06T02:14:42.368 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-06T02:14:42.368 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-06T02:14:42.368 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-06T02:14:42.369 DEBUG:teuthology.orchestra.run.smithi049:> ') 2023-11-06T02:14:42.383 INFO:tasks.ceph.mds.c.smithi049.stderr:2023-11-06T02:14:42.390+0000 7fabcf0f2640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-06T02:14:42.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:14:42.417 DEBUG:teuthology.orchestra.run.smithi049:> ip netns list 2023-11-06T02:14:42.465 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-06T02:14:42.465 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-06T02:14:42.466 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-06T02:14:42.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:14:42.466 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link set brx.0 up 2023-11-06T02:14:42.520 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-06T02:14:42.521 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-06T02:14:42.521 DEBUG:teuthology.orchestra.run.smithi049:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-06T02:14:42.569 INFO:teuthology.orchestra.run.smithi049.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-06T02:14:42.569 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-06T02:14:42.570 DEBUG:teuthology.orchestra.run.smithi049:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-06T02:14:42.615 DEBUG:teuthology.orchestra.run.smithi049:> sudo modprobe fuse 2023-11-06T02:14:42.673 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-06T02:14:42.674 DEBUG:teuthology.orchestra.run.smithi049:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-06T02:14:42.726 INFO:teuthology.orchestra.run.smithi049.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-06T02:14:42.728 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-06T02:14:42.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:14:42.729 DEBUG:teuthology.orchestra.run.smithi049:> ls /sys/fs/fuse/connections 2023-11-06T02:14:42.776 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-06T02:14:42.776 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-06T02:14:42.821 DEBUG:teuthology.orchestra.run.smithi049:> sudo modprobe fuse 2023-11-06T02:14:42.836 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-06T02:14:42.836 DEBUG:teuthology.orchestra.run.smithi049:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-06T02:14:42.891 INFO:teuthology.orchestra.run.smithi049.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-06T02:14:42.892 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-06T02:14:42.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:14:42.893 DEBUG:teuthology.orchestra.run.smithi049:> ls /sys/fs/fuse/connections 2023-11-06T02:14:42.899 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi049.stderr:2023-11-06T02:14:42.906+0000 7f465b3d5580 -1 init, newargv = 0x55cfd9ca7bd0 newargc=15 2023-11-06T02:14:42.900 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi049.stderr:ceph-fuse[57117]: starting ceph client 2023-11-06T02:14:42.907 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi049.stderr:ceph-fuse[57117]: starting fuse 2023-11-06T02:14:44.005 DEBUG:teuthology.orchestra.run.smithi049:> sudo modprobe fuse 2023-11-06T02:14:44.022 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-06T02:14:44.022 DEBUG:teuthology.orchestra.run.smithi049:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-06T02:14:44.075 INFO:teuthology.orchestra.run.smithi049.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-06T02:14:44.076 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-06T02:14:44.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:14:44.077 DEBUG:teuthology.orchestra.run.smithi049:> ls /sys/fs/fuse/connections 2023-11-06T02:14:44.124 INFO:teuthology.orchestra.run.smithi049.stdout:45 2023-11-06T02:14:44.125 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2023-11-06T02:14:44.125 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-11-06T02:14:44.126 DEBUG:teuthology.orchestra.run.smithi049:> import glob 2023-11-06T02:14:44.126 DEBUG:teuthology.orchestra.run.smithi049:> import re 2023-11-06T02:14:44.126 DEBUG:teuthology.orchestra.run.smithi049:> import os 2023-11-06T02:14:44.127 DEBUG:teuthology.orchestra.run.smithi049:> import subprocess 2023-11-06T02:14:44.127 DEBUG:teuthology.orchestra.run.smithi049:> 2023-11-06T02:14:44.128 DEBUG:teuthology.orchestra.run.smithi049:> def _find_admin_socket(client_name): 2023-11-06T02:14:44.128 DEBUG:teuthology.orchestra.run.smithi049:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-06T02:14:44.129 DEBUG:teuthology.orchestra.run.smithi049:> files = glob.glob(asok_path) 2023-11-06T02:14:44.129 DEBUG:teuthology.orchestra.run.smithi049:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-06T02:14:44.130 DEBUG:teuthology.orchestra.run.smithi049:> 2023-11-06T02:14:44.130 DEBUG:teuthology.orchestra.run.smithi049:> # Given a non-glob path, it better be there 2023-11-06T02:14:44.130 DEBUG:teuthology.orchestra.run.smithi049:> if "*" not in asok_path: 2023-11-06T02:14:44.131 DEBUG:teuthology.orchestra.run.smithi049:> assert(len(files) == 1) 2023-11-06T02:14:44.131 DEBUG:teuthology.orchestra.run.smithi049:> return files[0] 2023-11-06T02:14:44.131 DEBUG:teuthology.orchestra.run.smithi049:> 2023-11-06T02:14:44.132 DEBUG:teuthology.orchestra.run.smithi049:> for f in files: 2023-11-06T02:14:44.132 DEBUG:teuthology.orchestra.run.smithi049:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-06T02:14:44.133 DEBUG:teuthology.orchestra.run.smithi049:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-06T02:14:44.133 DEBUG:teuthology.orchestra.run.smithi049:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-06T02:14:44.133 DEBUG:teuthology.orchestra.run.smithi049:> contents = proc_f.read() 2023-11-06T02:14:44.134 DEBUG:teuthology.orchestra.run.smithi049:> if mountpoint in contents: 2023-11-06T02:14:44.134 DEBUG:teuthology.orchestra.run.smithi049:> return f 2023-11-06T02:14:44.134 DEBUG:teuthology.orchestra.run.smithi049:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-06T02:14:44.135 DEBUG:teuthology.orchestra.run.smithi049:> 2023-11-06T02:14:44.135 DEBUG:teuthology.orchestra.run.smithi049:> print(_find_admin_socket("client.0")) 2023-11-06T02:14:44.136 DEBUG:teuthology.orchestra.run.smithi049:> ' 2023-11-06T02:14:44.247 INFO:teuthology.orchestra.run.smithi049.stdout:/var/run/ceph/ceph-client.0.57117.asok 2023-11-06T02:14:44.429 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.57117.asok 2023-11-06T02:14:44.430 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:14:44.430 DEBUG:teuthology.orchestra.run.smithi049:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.57117.asok status 2023-11-06T02:14:44.590 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-11-06T02:14:44.591 INFO:teuthology.orchestra.run.smithi049.stdout: "metadata": { 2023-11-06T02:14:44.591 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_sha1": "f7e9f9af51acf98e0ccc3037487018e4ae40caa5", 2023-11-06T02:14:44.592 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_version": "ceph version 18.2.0-1181-gf7e9f9af (f7e9f9af51acf98e0ccc3037487018e4ae40caa5) reef (stable)", 2023-11-06T02:14:44.592 INFO:teuthology.orchestra.run.smithi049.stdout: "entity_id": "0", 2023-11-06T02:14:44.592 INFO:teuthology.orchestra.run.smithi049.stdout: "hostname": "smithi049", 2023-11-06T02:14:44.593 INFO:teuthology.orchestra.run.smithi049.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-06T02:14:44.593 INFO:teuthology.orchestra.run.smithi049.stdout: "pid": "57117", 2023-11-06T02:14:44.593 INFO:teuthology.orchestra.run.smithi049.stdout: "root": "/" 2023-11-06T02:14:44.594 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:14:44.594 INFO:teuthology.orchestra.run.smithi049.stdout: "dentry_count": 0, 2023-11-06T02:14:44.595 INFO:teuthology.orchestra.run.smithi049.stdout: "dentry_pinned_count": 0, 2023-11-06T02:14:44.595 INFO:teuthology.orchestra.run.smithi049.stdout: "id": 7869, 2023-11-06T02:14:44.595 INFO:teuthology.orchestra.run.smithi049.stdout: "inst": { 2023-11-06T02:14:44.595 INFO:teuthology.orchestra.run.smithi049.stdout: "name": { 2023-11-06T02:14:44.596 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "client", 2023-11-06T02:14:44.596 INFO:teuthology.orchestra.run.smithi049.stdout: "num": 7869 2023-11-06T02:14:44.596 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:14:44.597 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": { 2023-11-06T02:14:44.597 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2023-11-06T02:14:44.597 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "192.168.0.1:0", 2023-11-06T02:14:44.598 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 939153826 2023-11-06T02:14:44.598 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-11-06T02:14:44.598 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:14:44.598 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": { 2023-11-06T02:14:44.599 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2023-11-06T02:14:44.599 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "192.168.0.1:0", 2023-11-06T02:14:44.599 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 939153826 2023-11-06T02:14:44.600 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-11-06T02:14:44.600 INFO:teuthology.orchestra.run.smithi049.stdout: "inst_str": "client.7869 192.168.0.1:0/939153826", 2023-11-06T02:14:44.600 INFO:teuthology.orchestra.run.smithi049.stdout: "addr_str": "192.168.0.1:0/939153826", 2023-11-06T02:14:44.600 INFO:teuthology.orchestra.run.smithi049.stdout: "inode_count": 1, 2023-11-06T02:14:44.600 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_epoch": 59, 2023-11-06T02:14:44.601 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_epoch": 82, 2023-11-06T02:14:44.601 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_epoch_barrier": 0, 2023-11-06T02:14:44.601 INFO:teuthology.orchestra.run.smithi049.stdout: "blocklisted": false, 2023-11-06T02:14:44.601 INFO:teuthology.orchestra.run.smithi049.stdout: "fs_name": "cephfs" 2023-11-06T02:14:44.602 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-11-06T02:14:44.605 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:14:44.605 DEBUG:teuthology.orchestra.run.smithi049:> stat --file-system '--printf=%T 2023-11-06T02:14:44.606 DEBUG:teuthology.orchestra.run.smithi049:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-06T02:14:44.612 INFO:teuthology.orchestra.run.smithi049.stdout:fuseblk 2023-11-06T02:14:44.612 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-06T02:14:44.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:14:44.613 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-06T02:14:44.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:14:44.665 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2023-11-06T02:14:44.738 INFO:teuthology.orchestra.run.smithi049.stdout:./ 2023-11-06T02:14:44.739 INFO:teuthology.orchestra.run.smithi049.stdout:./parent 2023-11-06T02:14:44.739 INFO:teuthology.orchestra.run.smithi049.stdout:./parent/flushed 2023-11-06T02:14:44.739 INFO:teuthology.orchestra.run.smithi049.stdout:./parent/flushed/charlie 2023-11-06T02:14:44.739 INFO:teuthology.orchestra.run.smithi049.stdout:./parent/flushed/alpha 2023-11-06T02:14:44.740 INFO:teuthology.orchestra.run.smithi049.stdout:./parent/flushed/bravo 2023-11-06T02:14:44.740 INFO:teuthology.orchestra.run.smithi049.stdout:./parent/unflushed 2023-11-06T02:14:44.740 INFO:teuthology.orchestra.run.smithi049.stdout:./parent/unflushed/jfile 2023-11-06T02:14:44.740 DEBUG:teuthology.orchestra.run.smithi049:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-06T02:14:44.741 DEBUG:teuthology.orchestra.run.smithi049:> import os 2023-11-06T02:14:44.741 DEBUG:teuthology.orchestra.run.smithi049:> import stat 2023-11-06T02:14:44.741 DEBUG:teuthology.orchestra.run.smithi049:> 2023-11-06T02:14:44.742 DEBUG:teuthology.orchestra.run.smithi049:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2023-11-06T02:14:44.742 DEBUG:teuthology.orchestra.run.smithi049:> ' 2023-11-06T02:14:44.848 INFO:teuthology.orchestra.run.smithi049.stdout:1 2023-11-06T02:14:45.037 DEBUG:teuthology.orchestra.run.smithi049:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-06T02:14:45.038 DEBUG:teuthology.orchestra.run.smithi049:> import os 2023-11-06T02:14:45.038 DEBUG:teuthology.orchestra.run.smithi049:> import stat 2023-11-06T02:14:45.038 DEBUG:teuthology.orchestra.run.smithi049:> 2023-11-06T02:14:45.039 DEBUG:teuthology.orchestra.run.smithi049:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2023-11-06T02:14:45.039 DEBUG:teuthology.orchestra.run.smithi049:> ' 2023-11-06T02:14:45.142 INFO:teuthology.orchestra.run.smithi049.stdout:1099511627776 2023-11-06T02:14:45.330 DEBUG:teuthology.orchestra.run.smithi049:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-06T02:14:45.331 DEBUG:teuthology.orchestra.run.smithi049:> import os 2023-11-06T02:14:45.331 DEBUG:teuthology.orchestra.run.smithi049:> import stat 2023-11-06T02:14:45.332 DEBUG:teuthology.orchestra.run.smithi049:> 2023-11-06T02:14:45.333 DEBUG:teuthology.orchestra.run.smithi049:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2023-11-06T02:14:45.333 DEBUG:teuthology.orchestra.run.smithi049:> ' 2023-11-06T02:14:45.438 INFO:teuthology.orchestra.run.smithi049.stdout:1099511627777 2023-11-06T02:14:45.625 DEBUG:teuthology.orchestra.run.smithi049:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-06T02:14:45.626 DEBUG:teuthology.orchestra.run.smithi049:> import os 2023-11-06T02:14:45.626 DEBUG:teuthology.orchestra.run.smithi049:> import stat 2023-11-06T02:14:45.626 DEBUG:teuthology.orchestra.run.smithi049:> 2023-11-06T02:14:45.627 DEBUG:teuthology.orchestra.run.smithi049:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2023-11-06T02:14:45.627 DEBUG:teuthology.orchestra.run.smithi049:> ' 2023-11-06T02:14:45.735 INFO:teuthology.orchestra.run.smithi049.stdout:1099511628280 2023-11-06T02:14:45.923 DEBUG:teuthology.orchestra.run.smithi049:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-06T02:14:45.924 DEBUG:teuthology.orchestra.run.smithi049:> import os 2023-11-06T02:14:45.924 DEBUG:teuthology.orchestra.run.smithi049:> import stat 2023-11-06T02:14:45.924 DEBUG:teuthology.orchestra.run.smithi049:> 2023-11-06T02:14:45.925 DEBUG:teuthology.orchestra.run.smithi049:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2023-11-06T02:14:45.925 DEBUG:teuthology.orchestra.run.smithi049:> ' 2023-11-06T02:14:46.031 INFO:teuthology.orchestra.run.smithi049.stdout:1099511627778 2023-11-06T02:14:46.218 DEBUG:teuthology.orchestra.run.smithi049:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-06T02:14:46.219 DEBUG:teuthology.orchestra.run.smithi049:> import os 2023-11-06T02:14:46.219 DEBUG:teuthology.orchestra.run.smithi049:> import stat 2023-11-06T02:14:46.219 DEBUG:teuthology.orchestra.run.smithi049:> 2023-11-06T02:14:46.220 DEBUG:teuthology.orchestra.run.smithi049:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/bravo").st_ino) 2023-11-06T02:14:46.220 DEBUG:teuthology.orchestra.run.smithi049:> ' 2023-11-06T02:14:46.327 INFO:teuthology.orchestra.run.smithi049.stdout:1099511628279 2023-11-06T02:14:46.514 DEBUG:teuthology.orchestra.run.smithi049:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-06T02:14:46.515 DEBUG:teuthology.orchestra.run.smithi049:> import os 2023-11-06T02:14:46.515 DEBUG:teuthology.orchestra.run.smithi049:> import stat 2023-11-06T02:14:46.515 DEBUG:teuthology.orchestra.run.smithi049:> 2023-11-06T02:14:46.516 DEBUG:teuthology.orchestra.run.smithi049:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2023-11-06T02:14:46.516 DEBUG:teuthology.orchestra.run.smithi049:> ' 2023-11-06T02:14:46.623 INFO:teuthology.orchestra.run.smithi049.stdout:1099511627779 2023-11-06T02:14:46.812 DEBUG:teuthology.orchestra.run.smithi049:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-06T02:14:46.812 DEBUG:teuthology.orchestra.run.smithi049:> import os 2023-11-06T02:14:46.813 DEBUG:teuthology.orchestra.run.smithi049:> import stat 2023-11-06T02:14:46.813 DEBUG:teuthology.orchestra.run.smithi049:> 2023-11-06T02:14:46.814 DEBUG:teuthology.orchestra.run.smithi049:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2023-11-06T02:14:46.814 DEBUG:teuthology.orchestra.run.smithi049:> ' 2023-11-06T02:14:46.919 INFO:teuthology.orchestra.run.smithi049.stdout:1099511627780 2023-11-06T02:14:47.106 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-06T02:14:47.107 DEBUG:teuthology.orchestra.run.smithi049:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T02:14:47.197 DEBUG:teuthology.orchestra.run.smithi150:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T02:14:47.249 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-06T02:14:47.250 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-11-06T02:14:47.255 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T02:14:47.262 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T02:14:47.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.442+0000 7f95b5709640 1 Processor -- start 2023-11-06T02:14:47.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.442+0000 7f95b5709640 1 -- start start 2023-11-06T02:14:47.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.442+0000 7f95b5709640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f95b00b4b80 0x7f95b0069d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:47.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.442+0000 7f95b5709640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95b00ca8b0 con 0x7f95b0159820 2023-11-06T02:14:47.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.442+0000 7f95b5709640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95b006b580 con 0x7f95b00709f0 2023-11-06T02:14:47.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.442+0000 7f95b5709640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95b015a460 con 0x7f95b00b4b80 2023-11-06T02:14:47.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.442+0000 7f95aeffd640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f95b0159820 0x7f95b0159c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:33602/0 (socket says 172.21.15.49:33602) 2023-11-06T02:14:47.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.442+0000 7f95aeffd640 1 -- 172.21.15.49:0/3938652074 learned_addr learned my addr 172.21.15.49:0/3938652074 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:14:47.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.442+0000 7f95ae7fc640 1 --2- 172.21.15.49:0/3938652074 >> v2:172.21.15.49:3300/0 conn(0x7f95b00b4b80 0x7f95b0069d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:47.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.446+0000 7f95adffb640 1 -- 172.21.15.49:0/3938652074 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4141287413 0 0) 0x7f95b00ca8b0 con 0x7f95b0159820 2023-11-06T02:14:47.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.446+0000 7f95adffb640 1 -- 172.21.15.49:0/3938652074 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9598003530 con 0x7f95b0159820 2023-11-06T02:14:47.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.446+0000 7f95ae7fc640 1 -- 172.21.15.49:0/3938652074 >> v1:172.21.15.150:6789/0 conn(0x7f95b00709f0 legacy=0x7f95b00cad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:47.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.446+0000 7f95ae7fc640 1 -- 172.21.15.49:0/3938652074 >> v1:172.21.15.49:6789/0 conn(0x7f95b0159820 legacy=0x7f95b0159c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:47.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.446+0000 7f95ae7fc640 1 -- 172.21.15.49:0/3938652074 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95b015a740 con 0x7f95b00b4b80 2023-11-06T02:14:47.443 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.446+0000 7f95ae7fc640 1 --2- 172.21.15.49:0/3938652074 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95b00b4b80 0x7f95b0069d70 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f95a40097c0 tx=0x7f95a401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25bd6f21d8a0ee31 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:47.443 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.446+0000 7f95adffb640 1 -- 172.21.15.49:0/3938652074 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a40ab600 con 0x7f95b00b4b80 2023-11-06T02:14:47.443 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.446+0000 7f95adffb640 1 -- 172.21.15.49:0/3938652074 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95a40a7050 con 0x7f95b00b4b80 2023-11-06T02:14:47.443 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.446+0000 7f95adffb640 1 -- 172.21.15.49:0/3938652074 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a40ab980 con 0x7f95b00b4b80 2023-11-06T02:14:47.443 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.446+0000 7f95b5709640 1 -- 172.21.15.49:0/3938652074 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95b00b4b80 msgr2=0x7f95b0069d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:47.444 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.446+0000 7f95b5709640 1 --2- 172.21.15.49:0/3938652074 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95b00b4b80 0x7f95b0069d70 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f95a40097c0 tx=0x7f95a401c480 comp rx=0 tx=0).stop 2023-11-06T02:14:47.444 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.446+0000 7f95b5709640 1 -- 172.21.15.49:0/3938652074 shutdown_connections 2023-11-06T02:14:47.444 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.446+0000 7f95b5709640 1 --2- 172.21.15.49:0/3938652074 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95b00b4b80 0x7f95b0069d70 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:47.444 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.446+0000 7f95b5709640 1 -- 172.21.15.49:0/3938652074 >> 172.21.15.49:0/3938652074 conn(0x7f95b00b0bd0 msgr2=0x7f95b00b0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:47.444 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.446+0000 7f95b5709640 1 -- 172.21.15.49:0/3938652074 shutdown_connections 2023-11-06T02:14:47.445 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.446+0000 7f95b5709640 1 -- 172.21.15.49:0/3938652074 wait complete. 2023-11-06T02:14:47.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.450+0000 7f95b5709640 1 Processor -- start 2023-11-06T02:14:47.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.450+0000 7f95b5709640 1 -- start start 2023-11-06T02:14:47.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.450+0000 7f95b5709640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95b00709f0 0x7f95b01531f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:47.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.450+0000 7f95b5709640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f95b00b4b80 0x7f95b00b0020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:47.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.450+0000 7f95b5709640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f95b0159820 0x7f95b00b24c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:47.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.450+0000 7f95b5709640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95b00b35e0 con 0x7f95b00709f0 2023-11-06T02:14:47.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.450+0000 7f95b5709640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95b00b38c0 con 0x7f95b00b4b80 2023-11-06T02:14:47.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.450+0000 7f95b5709640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95b00b3ba0 con 0x7f95b0159820 2023-11-06T02:14:47.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.450+0000 7f95aeffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95b00709f0 0x7f95b01531f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:47.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.450+0000 7f95aeffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95b00709f0 0x7f95b01531f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48014/0 (socket says 172.21.15.49:48014) 2023-11-06T02:14:47.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.450+0000 7f95aeffd640 1 -- 172.21.15.49:0/2361520345 learned_addr learned my addr 172.21.15.49:0/2361520345 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:47.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.450+0000 7f95ae7fc640 1 --2- 172.21.15.49:0/2361520345 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f95b00b4b80 0x7f95b00b0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:47.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.450+0000 7f95af7fe640 1 --2- 172.21.15.49:0/2361520345 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f95b0159820 0x7f95b00b24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:47.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.450+0000 7f95aeffd640 1 -- 172.21.15.49:0/2361520345 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f95b0159820 msgr2=0x7f95b00b24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:47.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.450+0000 7f95aeffd640 1 --2- 172.21.15.49:0/2361520345 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f95b0159820 0x7f95b00b24c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:47.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.450+0000 7f95aeffd640 1 -- 172.21.15.49:0/2361520345 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f95b00b4b80 msgr2=0x7f95b00b0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:47.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.450+0000 7f95aeffd640 1 --2- 172.21.15.49:0/2361520345 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f95b00b4b80 0x7f95b00b0020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:47.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.450+0000 7f95aeffd640 1 -- 172.21.15.49:0/2361520345 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95a4009420 con 0x7f95b00709f0 2023-11-06T02:14:47.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.450+0000 7f95aeffd640 1 --2- 172.21.15.49:0/2361520345 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95b00709f0 0x7f95b01531f0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f95a000c4d0 tx=0x7f95a000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:47.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.450+0000 7f95977fe640 1 -- 172.21.15.49:0/2361520345 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a0005cf0 con 0x7f95b00709f0 2023-11-06T02:14:47.451 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.450+0000 7f95977fe640 1 -- 172.21.15.49:0/2361520345 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95a0005e50 con 0x7f95b00709f0 2023-11-06T02:14:47.451 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.450+0000 7f95b5709640 1 -- 172.21.15.49:0/2361520345 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95b0071390 con 0x7f95b00709f0 2023-11-06T02:14:47.451 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.450+0000 7f95977fe640 1 -- 172.21.15.49:0/2361520345 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a0007270 con 0x7f95b00709f0 2023-11-06T02:14:47.451 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.450+0000 7f95b5709640 1 -- 172.21.15.49:0/2361520345 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95b00b1fa0 con 0x7f95b00709f0 2023-11-06T02:14:47.452 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.454+0000 7f95b5709640 1 -- 172.21.15.49:0/2361520345 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95b00b2190 con 0x7f95b00709f0 2023-11-06T02:14:47.452 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.454+0000 7f95977fe640 1 -- 172.21.15.49:0/2361520345 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f95a00073d0 con 0x7f95b00709f0 2023-11-06T02:14:47.453 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.454+0000 7f95977fe640 1 --2- 172.21.15.49:0/2361520345 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f959c065a60 0x7f959c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:47.453 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.454+0000 7f95977fe640 1 -- 172.21.15.49:0/2361520345 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6234+0+0 (secure 0 0 0) 0x7f95a0082940 con 0x7f95b00709f0 2023-11-06T02:14:47.454 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.454+0000 7f95ae7fc640 1 --2- 172.21.15.49:0/2361520345 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f959c065a60 0x7f959c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:47.455 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.454+0000 7f95ae7fc640 1 --2- 172.21.15.49:0/2361520345 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f959c065a60 0x7f959c067f20 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f95a40ad000 tx=0x7f95a4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:47.457 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.462+0000 7f95977fe640 1 -- 172.21.15.49:0/2361520345 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f95a0050110 con 0x7f95b00709f0 2023-11-06T02:14:47.616 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.622+0000 7f95b5709640 1 -- 172.21.15.49:0/2361520345 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f95b0067140 con 0x7f95b00709f0 2023-11-06T02:14:47.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.626+0000 7f95977fe640 1 -- 172.21.15.49:0/2361520345 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+18145 (secure 0 0 0) 0x7f95a0017070 con 0x7f95b00709f0 2023-11-06T02:14:47.620 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:47.620 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":82,"fsid":"1a8b035c-6ba4-499a-b876-aea47fd9c0d4","created":"2023-11-06T02:08:06.440857+0000","modified":"2023-11-06T02:14:35.347605+0000","last_up_change":"2023-11-06T02:08:17.982372+0000","last_in_change":"2023-11-06T02:08:08.906631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T02:08:19.252222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T02:08:24.776211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-11-06T02:13:31.459314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-11-06T02:13:32.603710+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ad55af8-0571-4e6a-8722-b5d429b7496d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6809","nonce":38116},{"type":"v1","addr":"172.21.15.49:6811","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6813","nonce":38116},{"type":"v1","addr":"172.21.15.49:6815","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6821","nonce":38116},{"type":"v1","addr":"172.21.15.49:6823","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6817","nonce":38116},{"type":"v1","addr":"172.21.15.49:6819","nonce":38116}]},"public_addr":"172.21.15.49:6811/38116","cluster_addr":"172.21.15.49:6815/38116","heartbeat_back_addr":"172.21.15.49:6823/38116","heartbeat_front_addr":"172.21.15.49:6819/38116","state":["exists","up"]},{"osd":1,"uuid":"c8b31f85-a757-43ed-9419-351acc46611b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":38113},{"type":"v1","addr":"172.21.15.49:6810","nonce":38113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":38113},{"type":"v1","addr":"172.21.15.49:6814","nonce":38113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":38113},{"type":"v1","addr":"172.21.15.49:6822","nonce":38113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":38113},{"type":"v1","addr":"172.21.15.49:6818","nonce":38113}]},"public_addr":"172.21.15.49:6810/38113","cluster_addr":"172.21.15.49:6814/38113","heartbeat_back_addr":"172.21.15.49:6822/38113","heartbeat_front_addr":"172.21.15.49:6818/38113","state":["exists","up"]},{"osd":2,"uuid":"555b3bf3-0a70-4b91-8efd-e4204b5fc26e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":38114},{"type":"v1","addr":"172.21.15.49:6801","nonce":38114}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":38114},{"type":"v1","addr":"172.21.15.49:6803","nonce":38114}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":38114},{"type":"v1","addr":"172.21.15.49:6807","nonce":38114}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":38114},{"type":"v1","addr":"172.21.15.49:6805","nonce":38114}]},"public_addr":"172.21.15.49:6801/38114","cluster_addr":"172.21.15.49:6803/38114","heartbeat_back_addr":"172.21.15.49:6807/38114","heartbeat_front_addr":"172.21.15.49:6805/38114","state":["exists","up"]},{"osd":3,"uuid":"8d4e5d77-03ef-4b9b-bfcb-5ae93a46795c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":38115},{"type":"v1","addr":"172.21.15.49:6825","nonce":38115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":38115},{"type":"v1","addr":"172.21.15.49:6827","nonce":38115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":38115},{"type":"v1","addr":"172.21.15.49:6831","nonce":38115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":38115},{"type":"v1","addr":"172.21.15.49:6829","nonce":38115}]},"public_addr":"172.21.15.49:6825/38115","cluster_addr":"172.21.15.49:6827/38115","heartbeat_back_addr":"172.21.15.49:6831/38115","heartbeat_front_addr":"172.21.15.49:6829/38115","state":["exists","up"]},{"osd":4,"uuid":"dba03ad4-68a3-4cf3-9218-ac611f02eacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":38417},{"type":"v1","addr":"172.21.15.150:6809","nonce":38417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":38417},{"type":"v1","addr":"172.21.15.150:6811","nonce":38417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":38417},{"type":"v1","addr":"172.21.15.150:6815","nonce":38417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":38417},{"type":"v1","addr":"172.21.15.150:6813","nonce":38417}]},"public_addr":"172.21.15.150:6809/38417","cluster_addr":"172.21.15.150:6811/38417","heartbeat_back_addr":"172.21.15.150:6815/38417","heartbeat_front_addr":"172.21.15.150:6813/38417","state":["exists","up"]},{"osd":5,"uuid":"0df9c303-c255-4640-8e91-01f1c3efe18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":38418},{"type":"v1","addr":"172.21.15.150:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":38418},{"type":"v1","addr":"172.21.15.150:6803","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":38418},{"type":"v1","addr":"172.21.15.150:6807","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":38418},{"type":"v1","addr":"172.21.15.150:6805","nonce":38418}]},"public_addr":"172.21.15.150:6801/38418","cluster_addr":"172.21.15.150:6803/38418","heartbeat_back_addr":"172.21.15.150:6807/38418","heartbeat_front_addr":"172.21.15.150:6805/38418","state":["exists","up"]},{"osd":6,"uuid":"e59fa189-4b70-4c06-9722-c02b90828bf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":38428},{"type":"v1","addr":"172.21.15.150:6825","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":38428},{"type":"v1","addr":"172.21.15.150:6827","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":38428},{"type":"v1","addr":"172.21.15.150:6831","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":38428},{"type":"v1","addr":"172.21.15.150:6829","nonce":38428}]},"public_addr":"172.21.15.150:6825/38428","cluster_addr":"172.21.15.150:6827/38428","heartbeat_back_addr":"172.21.15.150:6831/38428","heartbeat_front_addr":"172.21.15.150:6829/38428","state":["exists","up"]},{"osd":7,"uuid":"a64f57e1-ef98-46bd-b304-385759e95898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":38421},{"type":"v1","addr":"172.21.15.150:6817","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":38421},{"type":"v1","addr":"172.21.15.150:6819","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":38421},{"type":"v1","addr":"172.21.15.150:6823","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":38421},{"type":"v1","addr":"172.21.15.150:6821","nonce":38421}]},"public_addr":"172.21.15.150:6817/38421","cluster_addr":"172.21.15.150:6819/38421","heartbeat_back_addr":"172.21.15.150:6823/38421","heartbeat_front_addr":"172.21.15.150:6821/38421","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.438552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.413280+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.296466+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.424966+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.276953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.230984+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.410235+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.480084+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:6835/4037104163":"2023-11-07T02:14:35.347551+0000","172.21.15.49:6834/4037104163":"2023-11-07T02:14:35.347551+0000","172.21.15.49:6835/1130379293":"2023-11-07T02:14:13.284419+0000","172.21.15.49:6834/1130379293":"2023-11-07T02:14:13.284419+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T02:14:47.621 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.626+0000 7f95b5709640 1 -- 172.21.15.49:0/2361520345 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f959c065a60 msgr2=0x7f959c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:47.621 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.626+0000 7f95b5709640 1 --2- 172.21.15.49:0/2361520345 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f959c065a60 0x7f959c067f20 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f95a40ad000 tx=0x7f95a4008040 comp rx=0 tx=0).stop 2023-11-06T02:14:47.622 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.626+0000 7f95b5709640 1 -- 172.21.15.49:0/2361520345 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95b00709f0 msgr2=0x7f95b01531f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:47.622 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.626+0000 7f95b5709640 1 --2- 172.21.15.49:0/2361520345 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95b00709f0 0x7f95b01531f0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f95a000c4d0 tx=0x7f95a000c990 comp rx=0 tx=0).stop 2023-11-06T02:14:47.622 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.626+0000 7f95b5709640 1 -- 172.21.15.49:0/2361520345 shutdown_connections 2023-11-06T02:14:47.622 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.626+0000 7f95b5709640 1 --2- 172.21.15.49:0/2361520345 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f959c065a60 0x7f959c067f20 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:47.622 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.626+0000 7f95b5709640 1 --2- 172.21.15.49:0/2361520345 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f95b0159820 0x7f95b00b24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:47.623 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.626+0000 7f95b5709640 1 --2- 172.21.15.49:0/2361520345 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f95b00b4b80 0x7f95b00b0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:47.623 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.626+0000 7f95b5709640 1 --2- 172.21.15.49:0/2361520345 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95b00709f0 0x7f95b01531f0 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:47.623 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.626+0000 7f95b5709640 1 -- 172.21.15.49:0/2361520345 >> 172.21.15.49:0/2361520345 conn(0x7f95b00b0bd0 msgr2=0x7f95b0152a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:47.623 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.626+0000 7f95b5709640 1 -- 172.21.15.49:0/2361520345 shutdown_connections 2023-11-06T02:14:47.623 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.626+0000 7f95b5709640 1 -- 172.21.15.49:0/2361520345 wait complete. 2023-11-06T02:14:47.638 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T02:08:19.252222+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T02:08:24.776211+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T02:13:31.459314+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T02:13:32.603710+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T02:14:47.639 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T02:14:47.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.778+0000 7f5e8e15b640 1 Processor -- start 2023-11-06T02:14:47.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.778+0000 7f5e8e15b640 1 -- start start 2023-11-06T02:14:47.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.778+0000 7f5e8e15b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5e88070870 0x7f5e880b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:47.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.778+0000 7f5e8e15b640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f5e8806d150 0x7f5e8806d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:47.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.778+0000 7f5e8e15b640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e8806db20 con 0x7f5e8815a880 2023-11-06T02:14:47.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.778+0000 7f5e8e15b640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e880ae360 con 0x7f5e8806d150 2023-11-06T02:14:47.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.778+0000 7f5e8e15b640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e880ae640 con 0x7f5e88070870 2023-11-06T02:14:47.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.778+0000 7f5e87fff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f5e8806d150 0x7f5e8806d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:47.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.778+0000 7f5e87fff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f5e8806d150 0x7f5e8806d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48016/0 (socket says 172.21.15.49:48016) 2023-11-06T02:14:47.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.778+0000 7f5e87fff640 1 -- 172.21.15.49:0/3740404421 learned_addr learned my addr 172.21.15.49:0/3740404421 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:47.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.778+0000 7f5e86ffd640 1 --2- 172.21.15.49:0/3740404421 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5e88070870 0x7f5e880b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:47.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.778+0000 7f5e87fff640 1 -- 172.21.15.49:0/3740404421 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5e88070870 msgr2=0x7f5e880b15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:47.776 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.778+0000 7f5e87fff640 1 --2- 172.21.15.49:0/3740404421 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5e88070870 0x7f5e880b15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:47.776 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.778+0000 7f5e87fff640 1 -- 172.21.15.49:0/3740404421 >> v1:172.21.15.150:6789/0 conn(0x7f5e8815a880 legacy=0x7f5e88070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:47.776 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.778+0000 7f5e87fff640 1 -- 172.21.15.49:0/3740404421 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e880ae920 con 0x7f5e8806d150 2023-11-06T02:14:47.777 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.782+0000 7f5e87fff640 1 --2- 172.21.15.49:0/3740404421 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e8806d150 0x7f5e8806d5e0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f5e80009870 tx=0x7f5e8001c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec8ccec16e848d36 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:47.777 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.782+0000 7f5e867fc640 1 -- 172.21.15.49:0/3740404421 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e800ab650 con 0x7f5e8806d150 2023-11-06T02:14:47.778 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.782+0000 7f5e867fc640 1 -- 172.21.15.49:0/3740404421 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e800ab7b0 con 0x7f5e8806d150 2023-11-06T02:14:47.778 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.782+0000 7f5e867fc640 1 -- 172.21.15.49:0/3740404421 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e800abb40 con 0x7f5e8806d150 2023-11-06T02:14:47.778 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.782+0000 7f5e8e15b640 1 -- 172.21.15.49:0/3740404421 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e8806d150 msgr2=0x7f5e8806d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:47.778 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.782+0000 7f5e8e15b640 1 --2- 172.21.15.49:0/3740404421 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e8806d150 0x7f5e8806d5e0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f5e80009870 tx=0x7f5e8001c540 comp rx=0 tx=0).stop 2023-11-06T02:14:47.778 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.782+0000 7f5e8e15b640 1 -- 172.21.15.49:0/3740404421 shutdown_connections 2023-11-06T02:14:47.779 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.782+0000 7f5e8e15b640 1 --2- 172.21.15.49:0/3740404421 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5e88070870 0x7f5e880b15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:47.779 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.782+0000 7f5e8e15b640 1 --2- 172.21.15.49:0/3740404421 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e8806d150 0x7f5e8806d5e0 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:47.779 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.782+0000 7f5e8e15b640 1 -- 172.21.15.49:0/3740404421 >> 172.21.15.49:0/3740404421 conn(0x7f5e8806a370 msgr2=0x7f5e880ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:47.779 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.782+0000 7f5e8e15b640 1 -- 172.21.15.49:0/3740404421 shutdown_connections 2023-11-06T02:14:47.779 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.782+0000 7f5e8e15b640 1 -- 172.21.15.49:0/3740404421 wait complete. 2023-11-06T02:14:47.782 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.782+0000 7f5e8e15b640 1 Processor -- start 2023-11-06T02:14:47.782 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.782+0000 7f5e8e15b640 1 -- start start 2023-11-06T02:14:47.782 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.782+0000 7f5e8e15b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e8806d150 0x7f5e88164f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:47.783 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.782+0000 7f5e8e15b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5e88070870 0x7f5e8814c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:47.783 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.786+0000 7f5e8e15b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e8815a880 0x7f5e880c7940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:47.783 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.786+0000 7f5e8e15b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e880c9540 con 0x7f5e8806d150 2023-11-06T02:14:47.783 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.786+0000 7f5e8e15b640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e880c91a0 con 0x7f5e8815a880 2023-11-06T02:14:47.783 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.786+0000 7f5e8e15b640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e880cc120 con 0x7f5e88070870 2023-11-06T02:14:47.784 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.786+0000 7f5e877fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e8806d150 0x7f5e88164f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:47.784 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.786+0000 7f5e877fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e8806d150 0x7f5e88164f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48020/0 (socket says 172.21.15.49:48020) 2023-11-06T02:14:47.784 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.786+0000 7f5e877fe640 1 -- 172.21.15.49:0/864610725 learned_addr learned my addr 172.21.15.49:0/864610725 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:47.784 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.786+0000 7f5e86ffd640 1 --2- 172.21.15.49:0/864610725 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5e88070870 0x7f5e8814c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:47.784 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.786+0000 7f5e87fff640 1 --2- 172.21.15.49:0/864610725 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e8815a880 0x7f5e880c7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:47.784 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.786+0000 7f5e877fe640 1 -- 172.21.15.49:0/864610725 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5e88070870 msgr2=0x7f5e8814c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:47.785 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.786+0000 7f5e877fe640 1 --2- 172.21.15.49:0/864610725 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5e88070870 0x7f5e8814c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:47.785 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.786+0000 7f5e877fe640 1 -- 172.21.15.49:0/864610725 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e8815a880 msgr2=0x7f5e880c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:47.785 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.786+0000 7f5e877fe640 1 --2- 172.21.15.49:0/864610725 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e8815a880 0x7f5e880c7940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:47.785 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.786+0000 7f5e877fe640 1 -- 172.21.15.49:0/864610725 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e800094d0 con 0x7f5e8806d150 2023-11-06T02:14:47.785 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.786+0000 7f5e87fff640 1 --2- 172.21.15.49:0/864610725 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e8815a880 0x7f5e880c7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:14:47.786 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.786+0000 7f5e86ffd640 1 --2- 172.21.15.49:0/864610725 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5e88070870 0x7f5e8814c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:14:47.786 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.786+0000 7f5e877fe640 1 --2- 172.21.15.49:0/864610725 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e8806d150 0x7f5e88164f30 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f5e7000c3c0 tx=0x7f5e7000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:47.786 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.786+0000 7f5e7ffff640 1 -- 172.21.15.49:0/864610725 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e700052d0 con 0x7f5e8806d150 2023-11-06T02:14:47.786 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.786+0000 7f5e7ffff640 1 -- 172.21.15.49:0/864610725 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e70005430 con 0x7f5e8806d150 2023-11-06T02:14:47.786 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.786+0000 7f5e7ffff640 1 -- 172.21.15.49:0/864610725 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e70003840 con 0x7f5e8806d150 2023-11-06T02:14:47.787 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.786+0000 7f5e8e15b640 1 -- 172.21.15.49:0/864610725 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e880cc400 con 0x7f5e8806d150 2023-11-06T02:14:47.787 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.786+0000 7f5e8e15b640 1 -- 172.21.15.49:0/864610725 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e880cc6e0 con 0x7f5e8806d150 2023-11-06T02:14:47.787 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.786+0000 7f5e8e15b640 1 -- 172.21.15.49:0/864610725 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e880c9680 con 0x7f5e8806d150 2023-11-06T02:14:47.787 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.786+0000 7f5e7ffff640 1 -- 172.21.15.49:0/864610725 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f5e700039a0 con 0x7f5e8806d150 2023-11-06T02:14:47.787 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.786+0000 7f5e7ffff640 1 --2- 172.21.15.49:0/864610725 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5e68065a60 0x7f5e68067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:47.788 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.786+0000 7f5e7ffff640 1 -- 172.21.15.49:0/864610725 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6234+0+0 (secure 0 0 0) 0x7f5e70082850 con 0x7f5e8806d150 2023-11-06T02:14:47.788 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.786+0000 7f5e86ffd640 1 --2- 172.21.15.49:0/864610725 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5e68065a60 0x7f5e68067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:47.788 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.790+0000 7f5e86ffd640 1 --2- 172.21.15.49:0/864610725 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5e68065a60 0x7f5e68067f20 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f5e8814d4c0 tx=0x7f5e7800b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:47.789 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.790+0000 7f5e7ffff640 1 -- 172.21.15.49:0/864610725 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f5e880c9680 con 0x7f5e8806d150 2023-11-06T02:14:47.941 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.946+0000 7f5e8e15b640 1 -- 172.21.15.49:0/864610725 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5e880c9680 con 0x7f5e8806d150 2023-11-06T02:14:47.942 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.946+0000 7f5e7ffff640 1 -- 172.21.15.49:0/864610725 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5e70017020 con 0x7f5e8806d150 2023-11-06T02:14:47.942 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2023-11-06T02:14:47.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.950+0000 7f5e8e15b640 1 -- 172.21.15.49:0/864610725 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5e68065a60 msgr2=0x7f5e68067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:47.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.950+0000 7f5e8e15b640 1 --2- 172.21.15.49:0/864610725 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5e68065a60 0x7f5e68067f20 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f5e8814d4c0 tx=0x7f5e7800b040 comp rx=0 tx=0).stop 2023-11-06T02:14:47.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.950+0000 7f5e8e15b640 1 -- 172.21.15.49:0/864610725 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e8806d150 msgr2=0x7f5e88164f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:47.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.950+0000 7f5e8e15b640 1 --2- 172.21.15.49:0/864610725 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e8806d150 0x7f5e88164f30 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f5e7000c3c0 tx=0x7f5e7000c880 comp rx=0 tx=0).stop 2023-11-06T02:14:47.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.950+0000 7f5e8e15b640 1 -- 172.21.15.49:0/864610725 shutdown_connections 2023-11-06T02:14:47.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.950+0000 7f5e8e15b640 1 --2- 172.21.15.49:0/864610725 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5e68065a60 0x7f5e68067f20 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:47.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.950+0000 7f5e8e15b640 1 --2- 172.21.15.49:0/864610725 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e8815a880 0x7f5e880c7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:47.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.950+0000 7f5e8e15b640 1 --2- 172.21.15.49:0/864610725 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5e88070870 0x7f5e8814c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:47.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.950+0000 7f5e8e15b640 1 --2- 172.21.15.49:0/864610725 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e8806d150 0x7f5e88164f30 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:47.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.950+0000 7f5e8e15b640 1 -- 172.21.15.49:0/864610725 >> 172.21.15.49:0/864610725 conn(0x7f5e8806a370 msgr2=0x7f5e8806c650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:47.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.950+0000 7f5e8e15b640 1 -- 172.21.15.49:0/864610725 shutdown_connections 2023-11-06T02:14:47.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:47.950+0000 7f5e8e15b640 1 -- 172.21.15.49:0/864610725 wait complete. 2023-11-06T02:14:47.961 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T02:14:48.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.090+0000 7f61ac567640 1 Processor -- start 2023-11-06T02:14:48.085 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.090+0000 7f61ac567640 1 -- start start 2023-11-06T02:14:48.086 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.090+0000 7f61ac567640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f61a415a880 0x7f61a40b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:48.086 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.090+0000 7f61ac567640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61a406d860 0x7f61a4070610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:48.087 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.090+0000 7f61ac567640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61a40ae360 con 0x7f61a4070160 2023-11-06T02:14:48.087 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.090+0000 7f61ac567640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61a4070b50 con 0x7f61a415a880 2023-11-06T02:14:48.087 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.090+0000 7f61ac567640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61a40ae640 con 0x7f61a406d860 2023-11-06T02:14:48.087 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.090+0000 7f61aa303640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f61a415a880 0x7f61a40b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:48.088 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.090+0000 7f61a9b02640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f61a4070160 0x7f61a406d150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:33604/0 (socket says 172.21.15.49:33604) 2023-11-06T02:14:48.088 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.090+0000 7f61a9b02640 1 -- 172.21.15.49:0/123425059 learned_addr learned my addr 172.21.15.49:0/123425059 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:14:48.088 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.090+0000 7f61aab04640 1 --2- 172.21.15.49:0/123425059 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61a406d860 0x7f61a4070610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:48.088 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.090+0000 7f61aa303640 1 -- 172.21.15.49:0/123425059 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61a406d860 msgr2=0x7f61a4070610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:48.088 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.090+0000 7f61aa303640 1 --2- 172.21.15.49:0/123425059 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61a406d860 0x7f61a4070610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:48.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.090+0000 7f61aa303640 1 -- 172.21.15.49:0/123425059 >> v1:172.21.15.49:6789/0 conn(0x7f61a4070160 legacy=0x7f61a406d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:48.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.090+0000 7f61aa303640 1 -- 172.21.15.49:0/123425059 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61a40ae920 con 0x7f61a415a880 2023-11-06T02:14:48.090 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.094+0000 7f61aa303640 1 --2- 172.21.15.49:0/123425059 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61a415a880 0x7f61a40b15c0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f61a00097c0 tx=0x7f61a001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24f5ff05909c314d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:48.091 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.094+0000 7f61a9301640 1 -- 172.21.15.49:0/123425059 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a00ad640 con 0x7f61a415a880 2023-11-06T02:14:48.091 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.094+0000 7f61a9301640 1 -- 172.21.15.49:0/123425059 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61a00ad7a0 con 0x7f61a415a880 2023-11-06T02:14:48.091 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.094+0000 7f61a9301640 1 -- 172.21.15.49:0/123425059 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a00adb30 con 0x7f61a415a880 2023-11-06T02:14:48.091 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.094+0000 7f61ac567640 1 -- 172.21.15.49:0/123425059 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61a415a880 msgr2=0x7f61a40b15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:48.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.094+0000 7f61ac567640 1 --2- 172.21.15.49:0/123425059 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61a415a880 0x7f61a40b15c0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f61a00097c0 tx=0x7f61a001c480 comp rx=0 tx=0).stop 2023-11-06T02:14:48.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.094+0000 7f61ac567640 1 -- 172.21.15.49:0/123425059 shutdown_connections 2023-11-06T02:14:48.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.094+0000 7f61ac567640 1 --2- 172.21.15.49:0/123425059 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61a406d860 0x7f61a4070610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:48.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.094+0000 7f61ac567640 1 --2- 172.21.15.49:0/123425059 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61a415a880 0x7f61a40b15c0 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:48.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.094+0000 7f61ac567640 1 -- 172.21.15.49:0/123425059 >> 172.21.15.49:0/123425059 conn(0x7f61a406a370 msgr2=0x7f61a40ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:48.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.094+0000 7f61ac567640 1 -- 172.21.15.49:0/123425059 shutdown_connections 2023-11-06T02:14:48.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.094+0000 7f61ac567640 1 -- 172.21.15.49:0/123425059 wait complete. 2023-11-06T02:14:48.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.094+0000 7f61ac567640 1 Processor -- start 2023-11-06T02:14:48.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.094+0000 7f61ac567640 1 -- start start 2023-11-06T02:14:48.096 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.098+0000 7f61ac567640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61a406d860 0x7f61a40ca150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:48.096 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.098+0000 7f61ac567640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61a4070160 0x7f61a414e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:48.096 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.098+0000 7f61ac567640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61a415a880 0x7f61a40b2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:48.097 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.098+0000 7f61ac567640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61a40c7fc0 con 0x7f61a415a880 2023-11-06T02:14:48.097 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.098+0000 7f61ac567640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61a40c7ce0 con 0x7f61a4070160 2023-11-06T02:14:48.097 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.098+0000 7f61ac567640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61a40cbe50 con 0x7f61a406d860 2023-11-06T02:14:48.097 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.098+0000 7f61aab04640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61a415a880 0x7f61a40b2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:48.098 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.098+0000 7f61aab04640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61a415a880 0x7f61a40b2ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48040/0 (socket says 172.21.15.49:48040) 2023-11-06T02:14:48.098 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.098+0000 7f61aab04640 1 -- 172.21.15.49:0/1395363581 learned_addr learned my addr 172.21.15.49:0/1395363581 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:48.098 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.098+0000 7f61aa303640 1 --2- 172.21.15.49:0/1395363581 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61a406d860 0x7f61a40ca150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:48.098 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.098+0000 7f61a9b02640 1 --2- 172.21.15.49:0/1395363581 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61a4070160 0x7f61a414e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:48.099 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.098+0000 7f61aab04640 1 -- 172.21.15.49:0/1395363581 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61a406d860 msgr2=0x7f61a40ca150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:48.099 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.098+0000 7f61aab04640 1 --2- 172.21.15.49:0/1395363581 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61a406d860 0x7f61a40ca150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:48.099 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.098+0000 7f61aab04640 1 -- 172.21.15.49:0/1395363581 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61a4070160 msgr2=0x7f61a414e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:48.099 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.098+0000 7f61aab04640 1 --2- 172.21.15.49:0/1395363581 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61a4070160 0x7f61a414e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:48.099 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.098+0000 7f61aab04640 1 -- 172.21.15.49:0/1395363581 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61a0009420 con 0x7f61a415a880 2023-11-06T02:14:48.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.098+0000 7f61a9b02640 1 --2- 172.21.15.49:0/1395363581 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61a4070160 0x7f61a414e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:14:48.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.098+0000 7f61aab04640 1 --2- 172.21.15.49:0/1395363581 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61a415a880 0x7f61a40b2ea0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f619c007e40 tx=0x7f619c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:48.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.098+0000 7f619affd640 1 -- 172.21.15.49:0/1395363581 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f619c014620 con 0x7f61a415a880 2023-11-06T02:14:48.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.098+0000 7f619affd640 1 -- 172.21.15.49:0/1395363581 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f619c00ee90 con 0x7f61a415a880 2023-11-06T02:14:48.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.098+0000 7f61ac567640 1 -- 172.21.15.49:0/1395363581 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61a40cc130 con 0x7f61a415a880 2023-11-06T02:14:48.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.098+0000 7f619affd640 1 -- 172.21.15.49:0/1395363581 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f619c0149e0 con 0x7f61a415a880 2023-11-06T02:14:48.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.098+0000 7f61ac567640 1 -- 172.21.15.49:0/1395363581 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61a40ca690 con 0x7f61a415a880 2023-11-06T02:14:48.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.098+0000 7f61ac567640 1 -- 172.21.15.49:0/1395363581 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61a4074040 con 0x7f61a415a880 2023-11-06T02:14:48.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.098+0000 7f619affd640 1 -- 172.21.15.49:0/1395363581 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f619c014b40 con 0x7f61a415a880 2023-11-06T02:14:48.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.098+0000 7f619affd640 1 --2- 172.21.15.49:0/1395363581 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6188065a60 0x7f6188067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:48.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.098+0000 7f619affd640 1 -- 172.21.15.49:0/1395363581 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6234+0+0 (secure 0 0 0) 0x7f619c083d50 con 0x7f61a415a880 2023-11-06T02:14:48.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.098+0000 7f61aa303640 1 --2- 172.21.15.49:0/1395363581 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6188065a60 0x7f6188067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:48.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.102+0000 7f61aa303640 1 --2- 172.21.15.49:0/1395363581 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6188065a60 0x7f6188067f20 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f61a00af000 tx=0x7f61a0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:48.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.102+0000 7f619affd640 1 -- 172.21.15.49:0/1395363581 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f619c087030 con 0x7f61a415a880 2023-11-06T02:14:48.256 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.262+0000 7f61ac567640 1 -- 172.21.15.49:0/1395363581 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f61a40cc5e0 con 0x7f61a415a880 2023-11-06T02:14:48.257 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.262+0000 7f619affd640 1 -- 172.21.15.49:0/1395363581 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v82) v1 ==== 92+0+10 (secure 0 0 0) 0x7f619c051500 con 0x7f61a415a880 2023-11-06T02:14:48.257 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 8 2023-11-06T02:14:48.259 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.262+0000 7f61ac567640 1 -- 172.21.15.49:0/1395363581 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6188065a60 msgr2=0x7f6188067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:48.259 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.262+0000 7f61ac567640 1 --2- 172.21.15.49:0/1395363581 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6188065a60 0x7f6188067f20 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f61a00af000 tx=0x7f61a0008040 comp rx=0 tx=0).stop 2023-11-06T02:14:48.259 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.262+0000 7f61ac567640 1 -- 172.21.15.49:0/1395363581 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61a415a880 msgr2=0x7f61a40b2ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:48.259 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.262+0000 7f61ac567640 1 --2- 172.21.15.49:0/1395363581 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61a415a880 0x7f61a40b2ea0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f619c007e40 tx=0x7f619c00e4b0 comp rx=0 tx=0).stop 2023-11-06T02:14:48.260 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.262+0000 7f61ac567640 1 -- 172.21.15.49:0/1395363581 shutdown_connections 2023-11-06T02:14:48.260 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.262+0000 7f61ac567640 1 --2- 172.21.15.49:0/1395363581 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6188065a60 0x7f6188067f20 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:48.260 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.262+0000 7f61ac567640 1 --2- 172.21.15.49:0/1395363581 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61a415a880 0x7f61a40b2ea0 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:48.260 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.262+0000 7f61ac567640 1 --2- 172.21.15.49:0/1395363581 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61a4070160 0x7f61a414e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:48.261 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.262+0000 7f61ac567640 1 --2- 172.21.15.49:0/1395363581 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61a406d860 0x7f61a40ca150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:48.261 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.262+0000 7f61ac567640 1 -- 172.21.15.49:0/1395363581 >> 172.21.15.49:0/1395363581 conn(0x7f61a406a370 msgr2=0x7f61a415cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:48.261 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.262+0000 7f61ac567640 1 -- 172.21.15.49:0/1395363581 shutdown_connections 2023-11-06T02:14:48.262 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.262+0000 7f61ac567640 1 -- 172.21.15.49:0/1395363581 wait complete. 2023-11-06T02:14:48.278 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T02:14:48.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.410+0000 7feb28297640 1 Processor -- start 2023-11-06T02:14:48.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.410+0000 7feb28297640 1 -- start start 2023-11-06T02:14:48.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.410+0000 7feb28297640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7feb20070870 0x7feb200b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:48.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.410+0000 7feb28297640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb200ae360 con 0x7feb2006d150 2023-11-06T02:14:48.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.410+0000 7feb28297640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb200ae640 con 0x7feb2015a880 2023-11-06T02:14:48.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.410+0000 7feb28297640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb200ae920 con 0x7feb20070870 2023-11-06T02:14:48.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.410+0000 7feb25832640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7feb20070870 0x7feb200b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:48.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.410+0000 7feb26834640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7feb2006d150 0x7feb2006d600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:33608/0 (socket says 172.21.15.49:33608) 2023-11-06T02:14:48.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.410+0000 7feb25832640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7feb20070870 0x7feb200b15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48050/0 (socket says 172.21.15.49:48050) 2023-11-06T02:14:48.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.410+0000 7feb26834640 1 -- 172.21.15.49:0/1324406685 learned_addr learned my addr 172.21.15.49:0/1324406685 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:14:48.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.410+0000 7feb25832640 1 -- 172.21.15.49:0/1324406685 >> v1:172.21.15.150:6789/0 conn(0x7feb2015a880 legacy=0x7feb20070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:48.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.410+0000 7feb25832640 1 -- 172.21.15.49:0/1324406685 >> v1:172.21.15.49:6789/0 conn(0x7feb2006d150 legacy=0x7feb2006d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:48.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.410+0000 7feb25832640 1 -- 172.21.15.49:0/1324406685 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb200aec00 con 0x7feb20070870 2023-11-06T02:14:48.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.410+0000 7feb25031640 1 -- 172.21.15.49:0/1324406685 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3143563751 0 0) 0x7feb200ae360 con 0x7feb2006d150 2023-11-06T02:14:48.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.414+0000 7feb25832640 1 --2- 172.21.15.49:0/1324406685 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb20070870 0x7feb200b15c0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7feb100097c0 tx=0x7feb1001c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95af56f67e31abd6 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:48.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.414+0000 7feb25031640 1 -- 172.21.15.49:0/1324406685 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb100ab600 con 0x7feb20070870 2023-11-06T02:14:48.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.414+0000 7feb25031640 1 -- 172.21.15.49:0/1324406685 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb100a7050 con 0x7feb20070870 2023-11-06T02:14:48.411 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.414+0000 7feb25031640 1 -- 172.21.15.49:0/1324406685 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb100ab980 con 0x7feb20070870 2023-11-06T02:14:48.411 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.414+0000 7feb28297640 1 -- 172.21.15.49:0/1324406685 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb20070870 msgr2=0x7feb200b15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:48.411 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.414+0000 7feb28297640 1 --2- 172.21.15.49:0/1324406685 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb20070870 0x7feb200b15c0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7feb100097c0 tx=0x7feb1001c3b0 comp rx=0 tx=0).stop 2023-11-06T02:14:48.411 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.414+0000 7feb28297640 1 -- 172.21.15.49:0/1324406685 shutdown_connections 2023-11-06T02:14:48.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.414+0000 7feb28297640 1 --2- 172.21.15.49:0/1324406685 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb20070870 0x7feb200b15c0 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:48.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.414+0000 7feb28297640 1 -- 172.21.15.49:0/1324406685 >> 172.21.15.49:0/1324406685 conn(0x7feb2006a370 msgr2=0x7feb200ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:48.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.414+0000 7feb28297640 1 -- 172.21.15.49:0/1324406685 shutdown_connections 2023-11-06T02:14:48.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.414+0000 7feb28297640 1 -- 172.21.15.49:0/1324406685 wait complete. 2023-11-06T02:14:48.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.418+0000 7feb28297640 1 Processor -- start 2023-11-06T02:14:48.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.418+0000 7feb28297640 1 -- start start 2023-11-06T02:14:48.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.418+0000 7feb28297640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb2006d150 0x7feb200c9520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:48.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.418+0000 7feb28297640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb20070870 0x7feb200a80a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:48.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.418+0000 7feb28297640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feb2015a880 0x7feb2014dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:48.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.418+0000 7feb28297640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb200c9b50 con 0x7feb20070870 2023-11-06T02:14:48.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.418+0000 7feb28297640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb200c9e30 con 0x7feb2006d150 2023-11-06T02:14:48.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.418+0000 7feb28297640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb200ca110 con 0x7feb2015a880 2023-11-06T02:14:48.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.418+0000 7feb25832640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb20070870 0x7feb200a80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:48.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.418+0000 7feb25832640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb20070870 0x7feb200a80a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48062/0 (socket says 172.21.15.49:48062) 2023-11-06T02:14:48.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.418+0000 7feb25832640 1 -- 172.21.15.49:0/3489708896 learned_addr learned my addr 172.21.15.49:0/3489708896 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:48.417 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.418+0000 7feb26834640 1 --2- 172.21.15.49:0/3489708896 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feb2015a880 0x7feb2014dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:48.417 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.418+0000 7feb26033640 1 --2- 172.21.15.49:0/3489708896 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb2006d150 0x7feb200c9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:48.417 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.418+0000 7feb25832640 1 -- 172.21.15.49:0/3489708896 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feb2015a880 msgr2=0x7feb2014dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:48.417 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.418+0000 7feb25832640 1 --2- 172.21.15.49:0/3489708896 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feb2015a880 0x7feb2014dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:48.417 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.418+0000 7feb25832640 1 -- 172.21.15.49:0/3489708896 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb2006d150 msgr2=0x7feb200c9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:48.418 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.418+0000 7feb25832640 1 --2- 172.21.15.49:0/3489708896 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb2006d150 0x7feb200c9520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:48.418 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.418+0000 7feb25832640 1 -- 172.21.15.49:0/3489708896 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb10009420 con 0x7feb20070870 2023-11-06T02:14:48.418 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.418+0000 7feb26033640 1 --2- 172.21.15.49:0/3489708896 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb2006d150 0x7feb200c9520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:14:48.418 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.418+0000 7feb25832640 1 --2- 172.21.15.49:0/3489708896 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb20070870 0x7feb200a80a0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7feb10018fc0 tx=0x7feb10019150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:48.418 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.418+0000 7feb16ffd640 1 -- 172.21.15.49:0/3489708896 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb100b4610 con 0x7feb20070870 2023-11-06T02:14:48.419 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.418+0000 7feb16ffd640 1 -- 172.21.15.49:0/3489708896 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb100b4790 con 0x7feb20070870 2023-11-06T02:14:48.419 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.418+0000 7feb16ffd640 1 -- 172.21.15.49:0/3489708896 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb100b4b40 con 0x7feb20070870 2023-11-06T02:14:48.419 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.418+0000 7feb28297640 1 -- 172.21.15.49:0/3489708896 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb200ca3f0 con 0x7feb20070870 2023-11-06T02:14:48.419 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.418+0000 7feb28297640 1 -- 172.21.15.49:0/3489708896 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb200cabf0 con 0x7feb20070870 2023-11-06T02:14:48.419 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.418+0000 7feb28297640 1 -- 172.21.15.49:0/3489708896 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb20074040 con 0x7feb20070870 2023-11-06T02:14:48.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.418+0000 7feb16ffd640 1 -- 172.21.15.49:0/3489708896 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7feb100b2040 con 0x7feb20070870 2023-11-06T02:14:48.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.422+0000 7feb16ffd640 1 --2- 172.21.15.49:0/3489708896 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7feb04065a60 0x7feb04067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:48.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.422+0000 7feb16ffd640 1 -- 172.21.15.49:0/3489708896 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6234+0+0 (secure 0 0 0) 0x7feb1011d7f0 con 0x7feb20070870 2023-11-06T02:14:48.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.422+0000 7feb26033640 1 --2- 172.21.15.49:0/3489708896 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7feb04065a60 0x7feb04067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:48.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.422+0000 7feb26033640 1 --2- 172.21.15.49:0/3489708896 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7feb04065a60 0x7feb04067f20 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7feb1c00a850 tx=0x7feb1c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:48.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.426+0000 7feb16ffd640 1 -- 172.21.15.49:0/3489708896 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7feb100eb030 con 0x7feb20070870 2023-11-06T02:14:48.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.578+0000 7feb28297640 1 -- 172.21.15.49:0/3489708896 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7feb2006c430 con 0x7feb20070870 2023-11-06T02:14:48.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.578+0000 7feb16ffd640 1 -- 172.21.15.49:0/3489708896 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v82) v1 ==== 104+0+11 (secure 0 0 0) 0x7feb100ee650 con 0x7feb20070870 2023-11-06T02:14:48.572 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2023-11-06T02:14:48.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.578+0000 7feb28297640 1 -- 172.21.15.49:0/3489708896 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7feb04065a60 msgr2=0x7feb04067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:48.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.578+0000 7feb28297640 1 --2- 172.21.15.49:0/3489708896 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7feb04065a60 0x7feb04067f20 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7feb1c00a850 tx=0x7feb1c008040 comp rx=0 tx=0).stop 2023-11-06T02:14:48.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.578+0000 7feb28297640 1 -- 172.21.15.49:0/3489708896 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb20070870 msgr2=0x7feb200a80a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:48.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.578+0000 7feb28297640 1 --2- 172.21.15.49:0/3489708896 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb20070870 0x7feb200a80a0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7feb10018fc0 tx=0x7feb10019150 comp rx=0 tx=0).stop 2023-11-06T02:14:48.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.578+0000 7feb28297640 1 -- 172.21.15.49:0/3489708896 shutdown_connections 2023-11-06T02:14:48.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.578+0000 7feb28297640 1 --2- 172.21.15.49:0/3489708896 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7feb04065a60 0x7feb04067f20 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:48.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.578+0000 7feb28297640 1 --2- 172.21.15.49:0/3489708896 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb20070870 0x7feb200a80a0 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:48.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.578+0000 7feb28297640 1 --2- 172.21.15.49:0/3489708896 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feb2015a880 0x7feb2014dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:48.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.578+0000 7feb28297640 1 --2- 172.21.15.49:0/3489708896 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb2006d150 0x7feb200c9520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:48.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.578+0000 7feb28297640 1 -- 172.21.15.49:0/3489708896 >> 172.21.15.49:0/3489708896 conn(0x7feb2006a370 msgr2=0x7feb2006c650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:48.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.578+0000 7feb28297640 1 -- 172.21.15.49:0/3489708896 shutdown_connections 2023-11-06T02:14:48.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.578+0000 7feb28297640 1 -- 172.21.15.49:0/3489708896 wait complete. 2023-11-06T02:14:48.590 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T02:14:48.706 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.710+0000 7f4c9ad34640 1 Processor -- start 2023-11-06T02:14:48.706 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.710+0000 7f4c9ad34640 1 -- start start 2023-11-06T02:14:48.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.714+0000 7f4c9ad34640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f4c9415a880 0x7f4c940b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:48.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.714+0000 7f4c9ad34640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4c94070160 0x7f4c94070610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:48.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.714+0000 7f4c9ad34640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c940ae360 con 0x7f4c9406d150 2023-11-06T02:14:48.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.714+0000 7f4c9ad34640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c94070b50 con 0x7f4c9415a880 2023-11-06T02:14:48.709 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.714+0000 7f4c9ad34640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c940ae640 con 0x7f4c94070160 2023-11-06T02:14:48.709 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.714+0000 7f4c98ad0640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f4c9415a880 0x7f4c940b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:48.709 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.714+0000 7f4c98ad0640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f4c9415a880 0x7f4c940b15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44294/0 (socket says 172.21.15.49:44294) 2023-11-06T02:14:48.709 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.714+0000 7f4c98ad0640 1 -- 172.21.15.49:0/3456566970 learned_addr learned my addr 172.21.15.49:0/3456566970 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:48.709 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.714+0000 7f4c93fff640 1 --2- 172.21.15.49:0/3456566970 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4c94070160 0x7f4c94070610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:48.710 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.714+0000 7f4c98ad0640 1 -- 172.21.15.49:0/3456566970 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4c94070160 msgr2=0x7f4c94070610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:48.710 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.714+0000 7f4c98ad0640 1 --2- 172.21.15.49:0/3456566970 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4c94070160 0x7f4c94070610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:48.710 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.714+0000 7f4c98ad0640 1 -- 172.21.15.49:0/3456566970 >> v1:172.21.15.150:6789/0 conn(0x7f4c9406d150 legacy=0x7f4c9406d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:48.710 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.714+0000 7f4c98ad0640 1 -- 172.21.15.49:0/3456566970 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c940ae920 con 0x7f4c9415a880 2023-11-06T02:14:48.710 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.714+0000 7f4c93fff640 1 --2- 172.21.15.49:0/3456566970 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4c94070160 0x7f4c94070610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:14:48.712 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.714+0000 7f4c98ad0640 1 --2- 172.21.15.49:0/3456566970 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c9415a880 0x7f4c940b15c0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f4c88009870 tx=0x7f4c8801c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25335a2e07fe44c1 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:48.712 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.714+0000 7f4c937fe640 1 -- 172.21.15.49:0/3456566970 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c880ac630 con 0x7f4c9415a880 2023-11-06T02:14:48.712 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.714+0000 7f4c937fe640 1 -- 172.21.15.49:0/3456566970 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c880ac790 con 0x7f4c9415a880 2023-11-06T02:14:48.712 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.714+0000 7f4c937fe640 1 -- 172.21.15.49:0/3456566970 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c880acb20 con 0x7f4c9415a880 2023-11-06T02:14:48.713 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.718+0000 7f4c9ad34640 1 -- 172.21.15.49:0/3456566970 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c9415a880 msgr2=0x7f4c940b15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:48.713 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.718+0000 7f4c9ad34640 1 --2- 172.21.15.49:0/3456566970 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c9415a880 0x7f4c940b15c0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f4c88009870 tx=0x7f4c8801c500 comp rx=0 tx=0).stop 2023-11-06T02:14:48.713 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.718+0000 7f4c9ad34640 1 -- 172.21.15.49:0/3456566970 shutdown_connections 2023-11-06T02:14:48.713 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.718+0000 7f4c9ad34640 1 --2- 172.21.15.49:0/3456566970 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4c94070160 0x7f4c94070610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:48.713 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.718+0000 7f4c9ad34640 1 --2- 172.21.15.49:0/3456566970 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c9415a880 0x7f4c940b15c0 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:48.713 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.718+0000 7f4c9ad34640 1 -- 172.21.15.49:0/3456566970 >> 172.21.15.49:0/3456566970 conn(0x7f4c9406a370 msgr2=0x7f4c940ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:48.714 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.718+0000 7f4c9ad34640 1 -- 172.21.15.49:0/3456566970 shutdown_connections 2023-11-06T02:14:48.714 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.718+0000 7f4c9ad34640 1 -- 172.21.15.49:0/3456566970 wait complete. 2023-11-06T02:14:48.716 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.718+0000 7f4c9ad34640 1 Processor -- start 2023-11-06T02:14:48.716 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.718+0000 7f4c9ad34640 1 -- start start 2023-11-06T02:14:48.716 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.718+0000 7f4c9ad34640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c9406d150 0x7f4c941f36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:48.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.718+0000 7f4c9ad34640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4c94070160 0x7f4c9414ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:48.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.718+0000 7f4c9ad34640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c9415a880 0x7f4c9414ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:48.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.718+0000 7f4c9ad34640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c940c3a90 con 0x7f4c9415a880 2023-11-06T02:14:48.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.718+0000 7f4c9ad34640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c941ed0d0 con 0x7f4c9406d150 2023-11-06T02:14:48.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.718+0000 7f4c9ad34640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c941f3bf0 con 0x7f4c94070160 2023-11-06T02:14:48.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.718+0000 7f4c992d1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c9415a880 0x7f4c9414ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:48.718 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.718+0000 7f4c992d1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c9415a880 0x7f4c9414ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44304/0 (socket says 172.21.15.49:44304) 2023-11-06T02:14:48.718 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.718+0000 7f4c992d1640 1 -- 172.21.15.49:0/4269832906 learned_addr learned my addr 172.21.15.49:0/4269832906 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:48.718 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.718+0000 7f4c98ad0640 1 --2- 172.21.15.49:0/4269832906 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c9406d150 0x7f4c941f36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:48.718 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.718+0000 7f4c93fff640 1 --2- 172.21.15.49:0/4269832906 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4c94070160 0x7f4c9414ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:48.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.718+0000 7f4c992d1640 1 -- 172.21.15.49:0/4269832906 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4c94070160 msgr2=0x7f4c9414ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:48.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.718+0000 7f4c992d1640 1 --2- 172.21.15.49:0/4269832906 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4c94070160 0x7f4c9414ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:48.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.718+0000 7f4c992d1640 1 -- 172.21.15.49:0/4269832906 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c9406d150 msgr2=0x7f4c941f36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:48.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.718+0000 7f4c992d1640 1 --2- 172.21.15.49:0/4269832906 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c9406d150 0x7f4c941f36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:48.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.718+0000 7f4c992d1640 1 -- 172.21.15.49:0/4269832906 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c880094d0 con 0x7f4c9415a880 2023-11-06T02:14:48.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.718+0000 7f4c992d1640 1 --2- 172.21.15.49:0/4269832906 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c9415a880 0x7f4c9414ea60 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c00c780 tx=0x7f4c8c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:48.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.718+0000 7f4c917fa640 1 -- 172.21.15.49:0/4269832906 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c004b70 con 0x7f4c9415a880 2023-11-06T02:14:48.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.718+0000 7f4c917fa640 1 -- 172.21.15.49:0/4269832906 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c8c00ce90 con 0x7f4c9415a880 2023-11-06T02:14:48.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.718+0000 7f4c9ad34640 1 -- 172.21.15.49:0/4269832906 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c941ec370 con 0x7f4c9415a880 2023-11-06T02:14:48.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.718+0000 7f4c917fa640 1 -- 172.21.15.49:0/4269832906 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c00a7f0 con 0x7f4c9415a880 2023-11-06T02:14:48.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.718+0000 7f4c9ad34640 1 -- 172.21.15.49:0/4269832906 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c941ec610 con 0x7f4c9415a880 2023-11-06T02:14:48.721 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.722+0000 7f4c9ad34640 1 -- 172.21.15.49:0/4269832906 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c94074040 con 0x7f4c9415a880 2023-11-06T02:14:48.721 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.722+0000 7f4c917fa640 1 -- 172.21.15.49:0/4269832906 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f4c8c00a950 con 0x7f4c9415a880 2023-11-06T02:14:48.721 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.722+0000 7f4c917fa640 1 --2- 172.21.15.49:0/4269832906 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4c78065a60 0x7f4c78067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:48.722 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.722+0000 7f4c917fa640 1 -- 172.21.15.49:0/4269832906 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6234+0+0 (secure 0 0 0) 0x7f4c8c083460 con 0x7f4c9415a880 2023-11-06T02:14:48.722 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.722+0000 7f4c98ad0640 1 --2- 172.21.15.49:0/4269832906 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4c78065a60 0x7f4c78067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:48.722 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.722+0000 7f4c98ad0640 1 --2- 172.21.15.49:0/4269832906 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4c78065a60 0x7f4c78067f20 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f4c880ae000 tx=0x7f4c880a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:48.722 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.726+0000 7f4c917fa640 1 -- 172.21.15.49:0/4269832906 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f4c8c050ca0 con 0x7f4c9415a880 2023-11-06T02:14:48.868 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.874+0000 7f4c9ad34640 1 -- 172.21.15.49:0/4269832906 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4c941ec9b0 con 0x7f4c9415a880 2023-11-06T02:14:48.869 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.874+0000 7f4c917fa640 1 -- 172.21.15.49:0/4269832906 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v82) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4c8c0542c0 con 0x7f4c9415a880 2023-11-06T02:14:48.870 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2023-11-06T02:14:48.871 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.874+0000 7f4c9ad34640 1 -- 172.21.15.49:0/4269832906 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4c78065a60 msgr2=0x7f4c78067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:48.871 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.874+0000 7f4c9ad34640 1 --2- 172.21.15.49:0/4269832906 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4c78065a60 0x7f4c78067f20 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f4c880ae000 tx=0x7f4c880a7000 comp rx=0 tx=0).stop 2023-11-06T02:14:48.871 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.874+0000 7f4c9ad34640 1 -- 172.21.15.49:0/4269832906 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c9415a880 msgr2=0x7f4c9414ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:48.872 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.874+0000 7f4c9ad34640 1 --2- 172.21.15.49:0/4269832906 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c9415a880 0x7f4c9414ea60 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c00c780 tx=0x7f4c8c00cc40 comp rx=0 tx=0).stop 2023-11-06T02:14:48.872 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.874+0000 7f4c9ad34640 1 -- 172.21.15.49:0/4269832906 shutdown_connections 2023-11-06T02:14:48.872 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.874+0000 7f4c9ad34640 1 --2- 172.21.15.49:0/4269832906 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4c78065a60 0x7f4c78067f20 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:48.872 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.874+0000 7f4c9ad34640 1 --2- 172.21.15.49:0/4269832906 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c9415a880 0x7f4c9414ea60 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:48.873 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.874+0000 7f4c9ad34640 1 --2- 172.21.15.49:0/4269832906 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4c94070160 0x7f4c9414ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:48.873 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.874+0000 7f4c9ad34640 1 --2- 172.21.15.49:0/4269832906 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c9406d150 0x7f4c941f36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:48.873 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.874+0000 7f4c9ad34640 1 -- 172.21.15.49:0/4269832906 >> 172.21.15.49:0/4269832906 conn(0x7f4c9406a370 msgr2=0x7f4c940b37a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:48.873 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.874+0000 7f4c9ad34640 1 -- 172.21.15.49:0/4269832906 shutdown_connections 2023-11-06T02:14:48.873 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:48.874+0000 7f4c9ad34640 1 -- 172.21.15.49:0/4269832906 wait complete. 2023-11-06T02:14:48.887 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-06T02:14:49.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.018+0000 7fa006d5a640 1 Processor -- start 2023-11-06T02:14:49.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.018+0000 7fa006d5a640 1 -- start start 2023-11-06T02:14:49.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.018+0000 7fa006d5a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa0000cb010 0x7fa0000ca980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:49.013 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.018+0000 7fa006d5a640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fa0000cdd50 0x7fa0000cc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:49.013 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.018+0000 7fa006d5a640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0000723a0 con 0x7fa000159b40 2023-11-06T02:14:49.013 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.018+0000 7fa006d5a640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0000715e0 con 0x7fa0000cdd50 2023-11-06T02:14:49.013 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.018+0000 7fa006d5a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0000718c0 con 0x7fa0000cb010 2023-11-06T02:14:49.013 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.018+0000 7f9ffffff640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fa000159b40 0x7fa0000cb3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:41930/0 (socket says 172.21.15.49:41930) 2023-11-06T02:14:49.014 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.018+0000 7f9ffffff640 1 -- 172.21.15.49:0/3857287534 learned_addr learned my addr 172.21.15.49:0/3857287534 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:14:49.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.018+0000 7f9fff7fe640 1 -- 172.21.15.49:0/3857287534 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4089978566 0 0) 0x7fa0000723a0 con 0x7fa000159b40 2023-11-06T02:14:49.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.018+0000 7f9fff7fe640 1 -- 172.21.15.49:0/3857287534 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ff0003530 con 0x7fa000159b40 2023-11-06T02:14:49.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.018+0000 7fa0052f7640 1 --2- 172.21.15.49:0/3857287534 >> v2:172.21.15.150:3300/0 conn(0x7fa0000cdd50 0x7fa0000cc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:49.019 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.018+0000 7fa004af6640 1 --2- 172.21.15.49:0/3857287534 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa0000cb010 0x7fa0000ca980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:49.019 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.018+0000 7f9fff7fe640 1 -- 172.21.15.49:0/3857287534 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2607342955 0 0) 0x7f9ff0003530 con 0x7fa000159b40 2023-11-06T02:14:49.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.018+0000 7f9fff7fe640 1 -- 172.21.15.49:0/3857287534 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa0000cb010 msgr2=0x7fa0000ca980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:49.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.018+0000 7f9fff7fe640 1 --2- 172.21.15.49:0/3857287534 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa0000cb010 0x7fa0000ca980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:49.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.018+0000 7f9fff7fe640 1 -- 172.21.15.49:0/3857287534 >> v2:172.21.15.150:3300/0 conn(0x7fa0000cdd50 msgr2=0x7fa0000cc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:49.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.018+0000 7f9fff7fe640 1 --2- 172.21.15.49:0/3857287534 >> v2:172.21.15.150:3300/0 conn(0x7fa0000cdd50 0x7fa0000cc1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:49.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.018+0000 7f9fff7fe640 1 -- 172.21.15.49:0/3857287534 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa000071ba0 con 0x7fa000159b40 2023-11-06T02:14:49.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.018+0000 7f9fff7fe640 1 -- 172.21.15.49:0/3857287534 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7f9fec001680 con 0x7fa000159b40 2023-11-06T02:14:49.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.018+0000 7fa004af6640 1 --2- 172.21.15.49:0/3857287534 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa0000cb010 0x7fa0000ca980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:14:49.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.018+0000 7fa0052f7640 1 --2- 172.21.15.49:0/3857287534 >> v2:172.21.15.150:3300/0 conn(0x7fa0000cdd50 0x7fa0000cc1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:14:49.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.018+0000 7f9fff7fe640 1 -- 172.21.15.49:0/3857287534 >> v1:172.21.15.49:6789/0 conn(0x7fa000159b40 legacy=0x7fa0000cb3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:49.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.018+0000 7f9fff7fe640 1 --2- 172.21.15.49:0/3857287534 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ff0003980 0x7f9ff0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:49.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.018+0000 7f9fff7fe640 1 --2- 172.21.15.49:0/3857287534 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ff00047d0 0x7f9ff0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:49.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.018+0000 7f9fff7fe640 1 --2- 172.21.15.49:0/3857287534 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa0000cb010 0x7f9ff0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:49.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.018+0000 7f9fff7fe640 1 -- 172.21.15.49:0/3857287534 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0000715e0 con 0x7f9ff00047d0 2023-11-06T02:14:49.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.018+0000 7f9fff7fe640 1 -- 172.21.15.49:0/3857287534 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0000718c0 con 0x7f9ff0003980 2023-11-06T02:14:49.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.018+0000 7f9fff7fe640 1 -- 172.21.15.49:0/3857287534 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff000ba10 con 0x7fa0000cb010 2023-11-06T02:14:49.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.018+0000 7f9ffffff640 1 --2- 172.21.15.49:0/3857287534 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ff00047d0 0x7f9ff0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:49.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.018+0000 7fa004af6640 1 --2- 172.21.15.49:0/3857287534 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ff0003980 0x7f9ff0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:49.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.018+0000 7fa0052f7640 1 --2- 172.21.15.49:0/3857287534 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa0000cb010 0x7f9ff0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:49.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.018+0000 7f9ffffff640 1 -- 172.21.15.49:0/3857287534 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa0000cb010 msgr2=0x7f9ff0005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:49.026 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.018+0000 7f9ffffff640 1 --2- 172.21.15.49:0/3857287534 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa0000cb010 0x7f9ff0005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:49.026 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.018+0000 7f9ffffff640 1 -- 172.21.15.49:0/3857287534 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ff0003980 msgr2=0x7f9ff0003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:49.026 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.018+0000 7f9ffffff640 1 --2- 172.21.15.49:0/3857287534 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ff0003980 0x7f9ff0003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:49.026 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.018+0000 7f9ffffff640 1 -- 172.21.15.49:0/3857287534 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ff0003530 con 0x7f9ff00047d0 2023-11-06T02:14:49.026 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.022+0000 7f9ffffff640 1 --2- 172.21.15.49:0/3857287534 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ff00047d0 0x7f9ff0004c50 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f9fec00c240 tx=0x7f9fec01f3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b1be2bd1b18eded server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:49.027 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.022+0000 7f9fff7fe640 1 -- 172.21.15.49:0/3857287534 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fec0a8d00 con 0x7f9ff00047d0 2023-11-06T02:14:49.027 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.022+0000 7f9fff7fe640 1 -- 172.21.15.49:0/3857287534 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fec0a8e60 con 0x7f9ff00047d0 2023-11-06T02:14:49.027 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.022+0000 7f9fff7fe640 1 -- 172.21.15.49:0/3857287534 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fec005ae0 con 0x7f9ff00047d0 2023-11-06T02:14:49.027 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.022+0000 7fa006d5a640 1 -- 172.21.15.49:0/3857287534 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ff00047d0 msgr2=0x7f9ff0004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:49.027 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.022+0000 7fa006d5a640 1 --2- 172.21.15.49:0/3857287534 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ff00047d0 0x7f9ff0004c50 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f9fec00c240 tx=0x7f9fec01f3d0 comp rx=0 tx=0).stop 2023-11-06T02:14:49.028 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.022+0000 7fa0052f7640 1 -- 172.21.15.49:0/3857287534 reap_dead start 2023-11-06T02:14:49.028 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.022+0000 7fa006d5a640 1 -- 172.21.15.49:0/3857287534 shutdown_connections 2023-11-06T02:14:49.028 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.022+0000 7fa006d5a640 1 -- 172.21.15.49:0/3857287534 >> 172.21.15.49:0/3857287534 conn(0x7fa00006c0a0 msgr2=0x7fa00006c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:49.028 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.022+0000 7fa006d5a640 1 -- 172.21.15.49:0/3857287534 shutdown_connections 2023-11-06T02:14:49.028 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.022+0000 7fa006d5a640 1 -- 172.21.15.49:0/3857287534 wait complete. 2023-11-06T02:14:49.028 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.022+0000 7fa006d5a640 1 Processor -- start 2023-11-06T02:14:49.029 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.022+0000 7fa006d5a640 1 -- start start 2023-11-06T02:14:49.032 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.022+0000 7fa006d5a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ff00047d0 0x7fa000150e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:49.032 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.022+0000 7fa006d5a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa00015ca60 0x7fa0000c3f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:49.032 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.022+0000 7fa006d5a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa0000b0be0 0x7fa0000c76e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:49.033 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.022+0000 7fa006d5a640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0000c44a0 con 0x7f9ff00047d0 2023-11-06T02:14:49.033 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.022+0000 7fa006d5a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0000c25d0 con 0x7fa00015ca60 2023-11-06T02:14:49.033 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.022+0000 7fa006d5a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa00015cee0 con 0x7fa0000b0be0 2023-11-06T02:14:49.033 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.022+0000 7fa004af6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ff00047d0 0x7fa000150e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:49.033 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.022+0000 7fa004af6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ff00047d0 0x7fa000150e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44328/0 (socket says 172.21.15.49:44328) 2023-11-06T02:14:49.034 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.022+0000 7fa004af6640 1 -- 172.21.15.49:0/1798023710 learned_addr learned my addr 172.21.15.49:0/1798023710 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:49.034 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.026+0000 7f9ffffff640 1 --2- 172.21.15.49:0/1798023710 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa00015ca60 0x7fa0000c3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:49.034 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.026+0000 7fa0052f7640 1 --2- 172.21.15.49:0/1798023710 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa0000b0be0 0x7fa0000c76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:49.035 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.026+0000 7fa004af6640 1 -- 172.21.15.49:0/1798023710 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa0000b0be0 msgr2=0x7fa0000c76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:49.035 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.026+0000 7fa004af6640 1 --2- 172.21.15.49:0/1798023710 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa0000b0be0 0x7fa0000c76e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:49.035 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.026+0000 7fa004af6640 1 -- 172.21.15.49:0/1798023710 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa00015ca60 msgr2=0x7fa0000c3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:49.035 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.026+0000 7fa004af6640 1 --2- 172.21.15.49:0/1798023710 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa00015ca60 0x7fa0000c3f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:49.036 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.026+0000 7fa004af6640 1 -- 172.21.15.49:0/1798023710 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0000c46d0 con 0x7f9ff00047d0 2023-11-06T02:14:49.036 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.026+0000 7fa004af6640 1 --2- 172.21.15.49:0/1798023710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ff00047d0 0x7fa000150e40 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f9ff4010860 tx=0x7f9ff4010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:49.036 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.026+0000 7f9ffd7fa640 1 -- 172.21.15.49:0/1798023710 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff4005ba0 con 0x7f9ff00047d0 2023-11-06T02:14:49.036 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.026+0000 7f9ffd7fa640 1 -- 172.21.15.49:0/1798023710 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ff4005d00 con 0x7f9ff00047d0 2023-11-06T02:14:49.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.026+0000 7fa006d5a640 1 -- 172.21.15.49:0/1798023710 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0000c49b0 con 0x7f9ff00047d0 2023-11-06T02:14:49.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.026+0000 7f9ffd7fa640 1 -- 172.21.15.49:0/1798023710 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff400cb60 con 0x7f9ff00047d0 2023-11-06T02:14:49.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.026+0000 7fa006d5a640 1 -- 172.21.15.49:0/1798023710 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0000c4c90 con 0x7f9ff00047d0 2023-11-06T02:14:49.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.026+0000 7fa006d5a640 1 -- 172.21.15.49:0/1798023710 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa000074040 con 0x7f9ff00047d0 2023-11-06T02:14:49.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.026+0000 7f9ffd7fa640 1 -- 172.21.15.49:0/1798023710 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f9ff400ccc0 con 0x7f9ff00047d0 2023-11-06T02:14:49.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.026+0000 7f9ffd7fa640 1 --2- 172.21.15.49:0/1798023710 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9fe4065a50 0x7f9fe4067f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:49.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.026+0000 7f9ffd7fa640 1 -- 172.21.15.49:0/1798023710 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6234+0+0 (secure 0 0 0) 0x7f9ff4083aa0 con 0x7f9ff00047d0 2023-11-06T02:14:49.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.026+0000 7f9ffffff640 1 --2- 172.21.15.49:0/1798023710 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9fe4065a50 0x7f9fe4067f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:49.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.026+0000 7f9ffffff640 1 --2- 172.21.15.49:0/1798023710 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9fe4065a50 0x7f9fe4067f10 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f9fec00c240 tx=0x7f9fec001a80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:49.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.030+0000 7f9ffd7fa640 1 -- 172.21.15.49:0/1798023710 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f9ff40512e0 con 0x7f9ff00047d0 2023-11-06T02:14:49.190 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.194+0000 7fa006d5a640 1 -- 172.21.15.49:0/1798023710 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa0000cb8f0 con 0x7f9ff00047d0 2023-11-06T02:14:49.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.194+0000 7f9ffd7fa640 1 -- 172.21.15.49:0/1798023710 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v82) v1 ==== 80+0+244 (secure 0 0 0) 0x7f9ff4054900 con 0x7f9ff00047d0 2023-11-06T02:14:49.191 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:6835/4037104163 2023-11-07T02:14:35.347551+0000 2023-11-06T02:14:49.192 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:6834/4037104163 2023-11-07T02:14:35.347551+0000 2023-11-06T02:14:49.192 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:6835/1130379293 2023-11-07T02:14:13.284419+0000 2023-11-06T02:14:49.192 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:6834/1130379293 2023-11-07T02:14:13.284419+0000 2023-11-06T02:14:49.193 INFO:teuthology.orchestra.run.smithi049.stderr:listed 4 entries 2023-11-06T02:14:49.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.198+0000 7fa006d5a640 1 -- 172.21.15.49:0/1798023710 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9fe4065a50 msgr2=0x7f9fe4067f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:49.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.198+0000 7fa006d5a640 1 --2- 172.21.15.49:0/1798023710 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9fe4065a50 0x7f9fe4067f10 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f9fec00c240 tx=0x7f9fec001a80 comp rx=0 tx=0).stop 2023-11-06T02:14:49.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.198+0000 7fa006d5a640 1 -- 172.21.15.49:0/1798023710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ff00047d0 msgr2=0x7fa000150e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:49.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.198+0000 7fa006d5a640 1 --2- 172.21.15.49:0/1798023710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ff00047d0 0x7fa000150e40 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f9ff4010860 tx=0x7f9ff4010d20 comp rx=0 tx=0).stop 2023-11-06T02:14:49.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.198+0000 7fa006d5a640 1 -- 172.21.15.49:0/1798023710 shutdown_connections 2023-11-06T02:14:49.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.198+0000 7fa006d5a640 1 --2- 172.21.15.49:0/1798023710 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9fe4065a50 0x7f9fe4067f10 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:49.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.198+0000 7fa006d5a640 1 --2- 172.21.15.49:0/1798023710 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa0000b0be0 0x7fa0000c76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:49.195 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.198+0000 7fa006d5a640 1 --2- 172.21.15.49:0/1798023710 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa00015ca60 0x7fa0000c3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:49.195 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.198+0000 7fa006d5a640 1 --2- 172.21.15.49:0/1798023710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ff00047d0 0x7fa000150e40 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:49.195 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.198+0000 7fa006d5a640 1 -- 172.21.15.49:0/1798023710 >> 172.21.15.49:0/1798023710 conn(0x7fa00006c0a0 msgr2=0x7fa0000ab8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:49.195 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.198+0000 7fa006d5a640 1 -- 172.21.15.49:0/1798023710 shutdown_connections 2023-11-06T02:14:49.195 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.198+0000 7fa006d5a640 1 -- 172.21.15.49:0/1798023710 wait complete. 2023-11-06T02:14:49.208 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi049.front.sepia.ceph.com... 2023-11-06T02:14:49.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:14:49.209 DEBUG:teuthology.orchestra.run.smithi049:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-06T02:14:49.218 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi049.stderr:ceph-fuse[57117]: fuse finished with error 0 and tester_r 0 2023-11-06T02:14:49.220 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi049.front.sepia.ceph.com 2023-11-06T02:14:49.220 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:14:49.221 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-06T02:14:49.295 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-11-06T02:14:49.296 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T02:14:49.304 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336552k,nr_inodes=4084138,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19862 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274692k,nr_inodes=818673,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-06T02:14:49.305 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi150.front.sepia.ceph.com 2023-11-06T02:14:49.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:14:49.306 DEBUG:teuthology.orchestra.run.smithi150:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-06T02:14:49.352 INFO:teuthology.orchestra.run.smithi150.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-11-06T02:14:49.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T02:14:49.354 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-11-06T02:14:49.354 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T02:14:49.360 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336476k,nr_inodes=4084119,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16105 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-06T02:14:49.360 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi049.front.sepia.ceph.com 2023-11-06T02:14:49.361 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:14:49.361 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-06T02:14:49.407 INFO:teuthology.orchestra.run.smithi049.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-11-06T02:14:49.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T02:14:49.409 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-11-06T02:14:49.409 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T02:14:49.415 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336552k,nr_inodes=4084138,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19862 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274692k,nr_inodes=818673,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-06T02:14:49.415 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi150.front.sepia.ceph.com 2023-11-06T02:14:49.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:14:49.416 DEBUG:teuthology.orchestra.run.smithi150:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-06T02:14:49.465 INFO:teuthology.orchestra.run.smithi150.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-11-06T02:14:49.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T02:14:49.466 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T02:14:49.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.598+0000 7f6a8da1f640 1 Processor -- start 2023-11-06T02:14:49.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.598+0000 7f6a8da1f640 1 -- start start 2023-11-06T02:14:49.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.598+0000 7f6a8da1f640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f6a8815ab50 0x7f6a880c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:49.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.598+0000 7f6a8da1f640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f6a880b0790 0x7f6a880c2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:49.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.598+0000 7f6a8da1f640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a880cab80 con 0x7f6a880cb010 2023-11-06T02:14:49.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.598+0000 7f6a8da1f640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a8815af30 con 0x7f6a880b0790 2023-11-06T02:14:49.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.598+0000 7f6a8da1f640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a880cb5b0 con 0x7f6a8815ab50 2023-11-06T02:14:49.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.598+0000 7f6a867fc640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f6a880cb010 0x7f6a880b0080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:41944/0 (socket says 172.21.15.49:41944) 2023-11-06T02:14:49.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.598+0000 7f6a867fc640 1 -- 172.21.15.49:0/1972381084 learned_addr learned my addr 172.21.15.49:0/1972381084 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:14:49.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.598+0000 7f6a877fe640 1 --2- 172.21.15.49:0/1972381084 >> v2:172.21.15.49:3300/0 conn(0x7f6a880b0790 0x7f6a880c2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:49.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.598+0000 7f6a85ffb640 1 -- 172.21.15.49:0/1972381084 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1732095701 0 0) 0x7f6a880cab80 con 0x7f6a880cb010 2023-11-06T02:14:49.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.598+0000 7f6a86ffd640 1 --2- 172.21.15.49:0/1972381084 >> v2:172.21.15.150:3300/0 conn(0x7f6a8815ab50 0x7f6a880c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:49.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.598+0000 7f6a85ffb640 1 -- 172.21.15.49:0/1972381084 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a74003530 con 0x7f6a880cb010 2023-11-06T02:14:49.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.598+0000 7f6a85ffb640 1 -- 172.21.15.49:0/1972381084 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1742058624 0 0) 0x7f6a74003530 con 0x7f6a880cb010 2023-11-06T02:14:49.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.598+0000 7f6a877fe640 1 -- 172.21.15.49:0/1972381084 >> v2:172.21.15.150:3300/0 conn(0x7f6a8815ab50 msgr2=0x7f6a880c9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:49.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.598+0000 7f6a877fe640 1 --2- 172.21.15.49:0/1972381084 >> v2:172.21.15.150:3300/0 conn(0x7f6a8815ab50 0x7f6a880c9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:49.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.598+0000 7f6a877fe640 1 -- 172.21.15.49:0/1972381084 >> v1:172.21.15.49:6789/0 conn(0x7f6a880cb010 legacy=0x7f6a880b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:49.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.598+0000 7f6a877fe640 1 -- 172.21.15.49:0/1972381084 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a880cb890 con 0x7f6a880b0790 2023-11-06T02:14:49.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.602+0000 7f6a877fe640 1 --2- 172.21.15.49:0/1972381084 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a880b0790 0x7f6a880c2710 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f6a880725a0 tx=0x7f6a8001c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f324a40e08a7418 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:49.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.602+0000 7f6a85ffb640 1 -- 172.21.15.49:0/1972381084 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a70001680 con 0x7f6a880b0790 2023-11-06T02:14:49.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.602+0000 7f6a85ffb640 1 -- 172.21.15.49:0/1972381084 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a800a5640 con 0x7f6a880b0790 2023-11-06T02:14:49.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.602+0000 7f6a85ffb640 1 -- 172.21.15.49:0/1972381084 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a800a59d0 con 0x7f6a880b0790 2023-11-06T02:14:49.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.602+0000 7f6a8da1f640 1 -- 172.21.15.49:0/1972381084 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a880b0790 msgr2=0x7f6a880c2710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:49.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.602+0000 7f6a8da1f640 1 --2- 172.21.15.49:0/1972381084 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a880b0790 0x7f6a880c2710 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f6a880725a0 tx=0x7f6a8001c800 comp rx=0 tx=0).stop 2023-11-06T02:14:49.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.602+0000 7f6a8da1f640 1 -- 172.21.15.49:0/1972381084 shutdown_connections 2023-11-06T02:14:49.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.602+0000 7f6a8da1f640 1 --2- 172.21.15.49:0/1972381084 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a880b0790 0x7f6a880c2710 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:49.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.602+0000 7f6a8da1f640 1 --2- 172.21.15.49:0/1972381084 >> v2:172.21.15.150:3300/0 conn(0x7f6a8815ab50 0x7f6a880c9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:49.600 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.602+0000 7f6a8da1f640 1 -- 172.21.15.49:0/1972381084 >> 172.21.15.49:0/1972381084 conn(0x7f6a8806c0a0 msgr2=0x7f6a8806c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:49.600 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.602+0000 7f6a8da1f640 1 -- 172.21.15.49:0/1972381084 shutdown_connections 2023-11-06T02:14:49.600 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.602+0000 7f6a8da1f640 1 -- 172.21.15.49:0/1972381084 wait complete. 2023-11-06T02:14:49.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.602+0000 7f6a8da1f640 1 Processor -- start 2023-11-06T02:14:49.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.602+0000 7f6a8da1f640 1 -- start start 2023-11-06T02:14:49.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.606+0000 7f6a8da1f640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6a880b0790 0x7f6a881f92c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:49.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.606+0000 7f6a8da1f640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a880cb010 0x7f6a88154460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:49.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.606+0000 7f6a8da1f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a8815ab50 0x7f6a881578a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:49.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.606+0000 7f6a8da1f640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a881f8e30 con 0x7f6a8815ab50 2023-11-06T02:14:49.604 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.606+0000 7f6a8da1f640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a88157de0 con 0x7f6a880cb010 2023-11-06T02:14:49.604 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.606+0000 7f6a8da1f640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a88158080 con 0x7f6a880b0790 2023-11-06T02:14:49.604 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.606+0000 7f6a877fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a8815ab50 0x7f6a881578a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:49.604 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.606+0000 7f6a877fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a8815ab50 0x7f6a881578a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44356/0 (socket says 172.21.15.49:44356) 2023-11-06T02:14:49.604 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.606+0000 7f6a877fe640 1 -- 172.21.15.49:0/1956691916 learned_addr learned my addr 172.21.15.49:0/1956691916 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:49.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.606+0000 7f6a867fc640 1 --2- 172.21.15.49:0/1956691916 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a880cb010 0x7f6a88154460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:49.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.606+0000 7f6a877fe640 1 -- 172.21.15.49:0/1956691916 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6a880b0790 msgr2=0x7f6a881f92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:49.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.606+0000 7f6a877fe640 1 --2- 172.21.15.49:0/1956691916 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6a880b0790 0x7f6a881f92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:49.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.606+0000 7f6a877fe640 1 -- 172.21.15.49:0/1956691916 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a880cb010 msgr2=0x7f6a88154460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:49.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.606+0000 7f6a877fe640 1 --2- 172.21.15.49:0/1956691916 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a880cb010 0x7f6a88154460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:49.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.606+0000 7f6a877fe640 1 -- 172.21.15.49:0/1956691916 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a80009a70 con 0x7f6a8815ab50 2023-11-06T02:14:49.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.606+0000 7f6a877fe640 1 --2- 172.21.15.49:0/1956691916 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a8815ab50 0x7f6a881578a0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f6a800b0040 tx=0x7f6a800195a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:49.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.606+0000 7f6a7ffff640 1 -- 172.21.15.49:0/1956691916 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a800b9860 con 0x7f6a8815ab50 2023-11-06T02:14:49.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.606+0000 7f6a7ffff640 1 -- 172.21.15.49:0/1956691916 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a800b99c0 con 0x7f6a8815ab50 2023-11-06T02:14:49.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.606+0000 7f6a8da1f640 1 -- 172.21.15.49:0/1956691916 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a881f87e0 con 0x7f6a8815ab50 2023-11-06T02:14:49.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.606+0000 7f6a7ffff640 1 -- 172.21.15.49:0/1956691916 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a800b9db0 con 0x7f6a8815ab50 2023-11-06T02:14:49.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.606+0000 7f6a8da1f640 1 -- 172.21.15.49:0/1956691916 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a88158360 con 0x7f6a8815ab50 2023-11-06T02:14:49.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.606+0000 7f6a8da1f640 1 -- 172.21.15.49:0/1956691916 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a880b0de0 con 0x7f6a8815ab50 2023-11-06T02:14:49.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.606+0000 7f6a7ffff640 1 -- 172.21.15.49:0/1956691916 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f6a800c4460 con 0x7f6a8815ab50 2023-11-06T02:14:49.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.606+0000 7f6a7ffff640 1 --2- 172.21.15.49:0/1956691916 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6a68065a60 0x7f6a68067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:49.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.606+0000 7f6a7ffff640 1 -- 172.21.15.49:0/1956691916 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6234+0+0 (secure 0 0 0) 0x7f6a8011e970 con 0x7f6a8815ab50 2023-11-06T02:14:49.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.606+0000 7f6a86ffd640 1 --2- 172.21.15.49:0/1956691916 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6a68065a60 0x7f6a68067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:49.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.610+0000 7f6a86ffd640 1 --2- 172.21.15.49:0/1956691916 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6a68065a60 0x7f6a68067f20 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f6a78006fd0 tx=0x7f6a78008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:49.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.610+0000 7f6a7ffff640 1 -- 172.21.15.49:0/1956691916 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f6a800ec240 con 0x7f6a8815ab50 2023-11-06T02:14:49.790 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.794+0000 7f6a8da1f640 1 -- 172.21.15.49:0/1956691916 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6a881f8f70 con 0x7f6a8815ab50 2023-11-06T02:14:49.791 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.798+0000 7f6a7ffff640 1 -- 172.21.15.49:0/1956691916 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+4651 (secure 0 0 0) 0x7f6a881f8f70 con 0x7f6a8815ab50 2023-11-06T02:14:49.792 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:49.792 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":59,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5714,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/315581318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":315581318},{"type":"v1","addr":"172.21.15.150:6837","nonce":315581318}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41},{"gid":6121,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/136800245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":136800245},{"type":"v1","addr":"172.21.15.150:6835","nonce":136800245}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41},{"gid":7110,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/4259579254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":4259579254},{"type":"v1","addr":"172.21.15.49:6833","nonce":4259579254}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T02:13:33.625701+0000","modified":"2023-11-06T02:14:40.422212+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7752},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7752":{"gid":7752,"name":"c","rank":0,"incarnation":56,"state":"up:active","state_seq":4,"addr":"172.21.15.49:6835/2241629386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2241629386},{"type":"v1","addr":"172.21.15.49:6835","nonce":2241629386}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-06T02:14:49.792 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 59 2023-11-06T02:14:49.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.798+0000 7f6a8da1f640 1 -- 172.21.15.49:0/1956691916 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6a68065a60 msgr2=0x7f6a68067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:49.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.798+0000 7f6a8da1f640 1 --2- 172.21.15.49:0/1956691916 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6a68065a60 0x7f6a68067f20 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f6a78006fd0 tx=0x7f6a78008040 comp rx=0 tx=0).stop 2023-11-06T02:14:49.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.798+0000 7f6a8da1f640 1 -- 172.21.15.49:0/1956691916 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a8815ab50 msgr2=0x7f6a881578a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:49.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.798+0000 7f6a8da1f640 1 --2- 172.21.15.49:0/1956691916 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a8815ab50 0x7f6a881578a0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f6a800b0040 tx=0x7f6a800195a0 comp rx=0 tx=0).stop 2023-11-06T02:14:49.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.798+0000 7f6a8da1f640 1 -- 172.21.15.49:0/1956691916 shutdown_connections 2023-11-06T02:14:49.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.798+0000 7f6a8da1f640 1 --2- 172.21.15.49:0/1956691916 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f6a68065a60 0x7f6a68067f20 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:49.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.798+0000 7f6a8da1f640 1 --2- 172.21.15.49:0/1956691916 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a8815ab50 0x7f6a881578a0 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:49.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.798+0000 7f6a8da1f640 1 --2- 172.21.15.49:0/1956691916 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a880cb010 0x7f6a88154460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:49.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.798+0000 7f6a8da1f640 1 --2- 172.21.15.49:0/1956691916 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6a880b0790 0x7f6a881f92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:49.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.798+0000 7f6a8da1f640 1 -- 172.21.15.49:0/1956691916 >> 172.21.15.49:0/1956691916 conn(0x7f6a8806c0a0 msgr2=0x7f6a88154b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:49.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.798+0000 7f6a8da1f640 1 -- 172.21.15.49:0/1956691916 shutdown_connections 2023-11-06T02:14:49.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.798+0000 7f6a8da1f640 1 -- 172.21.15.49:0/1956691916 wait complete. 2023-11-06T02:14:49.810 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T02:14:49.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.938+0000 7f4d9c08d640 1 Processor -- start 2023-11-06T02:14:49.932 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.938+0000 7f4d9c08d640 1 -- start start 2023-11-06T02:14:49.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.938+0000 7f4d9c08d640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f4d940c9690 0x7f4d940b04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:49.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.938+0000 7f4d9c08d640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d940b0a20 con 0x7f4d9415cc50 2023-11-06T02:14:49.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.938+0000 7f4d9c08d640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d9415d030 con 0x7f4d940b0080 2023-11-06T02:14:49.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.938+0000 7f4d9c08d640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d940cc9c0 con 0x7f4d940c9690 2023-11-06T02:14:49.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.938+0000 7f4d9a62a640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f4d940c9690 0x7f4d940b04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:49.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.938+0000 7f4d99e29640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f4d9415cc50 0x7f4d940701e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:41948/0 (socket says 172.21.15.49:41948) 2023-11-06T02:14:49.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.938+0000 7f4d9a62a640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f4d940c9690 0x7f4d940b04e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44364/0 (socket says 172.21.15.49:44364) 2023-11-06T02:14:49.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.938+0000 7f4d99e29640 1 -- 172.21.15.49:0/2452470669 learned_addr learned my addr 172.21.15.49:0/2452470669 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:14:49.935 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.938+0000 7f4d9a62a640 1 -- 172.21.15.49:0/2452470669 >> v1:172.21.15.150:6789/0 conn(0x7f4d940b0080 legacy=0x7f4d940c8f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:49.935 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.938+0000 7f4d98e27640 1 -- 172.21.15.49:0/2452470669 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1191915428 0 0) 0x7f4d940b0a20 con 0x7f4d9415cc50 2023-11-06T02:14:49.935 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.938+0000 7f4d9a62a640 1 -- 172.21.15.49:0/2452470669 >> v1:172.21.15.49:6789/0 conn(0x7f4d9415cc50 legacy=0x7f4d940701e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:49.935 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.938+0000 7f4d9a62a640 1 -- 172.21.15.49:0/2452470669 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d940ccc60 con 0x7f4d940c9690 2023-11-06T02:14:49.936 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.942+0000 7f4d9a62a640 1 --2- 172.21.15.49:0/2452470669 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d940c9690 0x7f4d940b04e0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c009d90 tx=0x7f4d8c01c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5a12cce7a886d48 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:49.936 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.942+0000 7f4d98e27640 1 -- 172.21.15.49:0/2452470669 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d8c0a5950 con 0x7f4d940c9690 2023-11-06T02:14:49.936 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.942+0000 7f4d98e27640 1 -- 172.21.15.49:0/2452470669 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d8c0a5ab0 con 0x7f4d940c9690 2023-11-06T02:14:49.936 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.942+0000 7f4d98e27640 1 -- 172.21.15.49:0/2452470669 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d8c0a5e40 con 0x7f4d940c9690 2023-11-06T02:14:49.937 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.942+0000 7f4d9c08d640 1 -- 172.21.15.49:0/2452470669 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d940c9690 msgr2=0x7f4d940b04e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:49.938 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.942+0000 7f4d9c08d640 1 --2- 172.21.15.49:0/2452470669 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d940c9690 0x7f4d940b04e0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c009d90 tx=0x7f4d8c01c890 comp rx=0 tx=0).stop 2023-11-06T02:14:49.938 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.942+0000 7f4d9c08d640 1 -- 172.21.15.49:0/2452470669 shutdown_connections 2023-11-06T02:14:49.938 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.942+0000 7f4d9c08d640 1 --2- 172.21.15.49:0/2452470669 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d940c9690 0x7f4d940b04e0 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:49.938 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.942+0000 7f4d9c08d640 1 -- 172.21.15.49:0/2452470669 >> 172.21.15.49:0/2452470669 conn(0x7f4d9406c0a0 msgr2=0x7f4d9406c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:49.938 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.942+0000 7f4d9c08d640 1 -- 172.21.15.49:0/2452470669 shutdown_connections 2023-11-06T02:14:49.939 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.942+0000 7f4d9c08d640 1 -- 172.21.15.49:0/2452470669 wait complete. 2023-11-06T02:14:49.941 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.942+0000 7f4d9c08d640 1 Processor -- start 2023-11-06T02:14:49.941 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.942+0000 7f4d9c08d640 1 -- start start 2023-11-06T02:14:49.941 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.942+0000 7f4d9c08d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d940b0080 0x7f4d940c68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:49.941 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.942+0000 7f4d9c08d640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4d940c9690 0x7f4d94151c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:49.941 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.942+0000 7f4d9c08d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d9415cc50 0x7f4d940b3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:49.942 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.942+0000 7f4d9c08d640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d940c7b50 con 0x7f4d9415cc50 2023-11-06T02:14:49.942 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.942+0000 7f4d9c08d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d941521a0 con 0x7f4d940b0080 2023-11-06T02:14:49.942 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.942+0000 7f4d9c08d640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d940c6f40 con 0x7f4d940c9690 2023-11-06T02:14:49.942 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.946+0000 7f4d9a62a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d9415cc50 0x7f4d940b3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:49.942 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.946+0000 7f4d9a62a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d9415cc50 0x7f4d940b3960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44372/0 (socket says 172.21.15.49:44372) 2023-11-06T02:14:49.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.946+0000 7f4d9a62a640 1 -- 172.21.15.49:0/1100377164 learned_addr learned my addr 172.21.15.49:0/1100377164 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:49.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.946+0000 7f4d99e29640 1 --2- 172.21.15.49:0/1100377164 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d940b0080 0x7f4d940c68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:49.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.946+0000 7f4d99628640 1 --2- 172.21.15.49:0/1100377164 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4d940c9690 0x7f4d94151c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:49.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.946+0000 7f4d9a62a640 1 -- 172.21.15.49:0/1100377164 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4d940c9690 msgr2=0x7f4d94151c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:49.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.946+0000 7f4d9a62a640 1 --2- 172.21.15.49:0/1100377164 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4d940c9690 0x7f4d94151c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:49.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.946+0000 7f4d9a62a640 1 -- 172.21.15.49:0/1100377164 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d940b0080 msgr2=0x7f4d940c68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:49.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.946+0000 7f4d9a62a640 1 --2- 172.21.15.49:0/1100377164 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d940b0080 0x7f4d940c68f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:49.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.946+0000 7f4d9a62a640 1 -- 172.21.15.49:0/1100377164 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d8c009a70 con 0x7f4d9415cc50 2023-11-06T02:14:49.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.946+0000 7f4d99628640 1 --2- 172.21.15.49:0/1100377164 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4d940c9690 0x7f4d94151c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:14:49.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.946+0000 7f4d9a62a640 1 --2- 172.21.15.49:0/1100377164 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d9415cc50 0x7f4d940b3960 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c0b0040 tx=0x7f4d8c019630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:49.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.946+0000 7f4d8a7fc640 1 -- 172.21.15.49:0/1100377164 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d8c0ade40 con 0x7f4d9415cc50 2023-11-06T02:14:49.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.946+0000 7f4d8a7fc640 1 -- 172.21.15.49:0/1100377164 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d8c0b9450 con 0x7f4d9415cc50 2023-11-06T02:14:49.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.946+0000 7f4d9c08d640 1 -- 172.21.15.49:0/1100377164 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d940c7220 con 0x7f4d9415cc50 2023-11-06T02:14:49.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.946+0000 7f4d8a7fc640 1 -- 172.21.15.49:0/1100377164 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d8c0b97e0 con 0x7f4d9415cc50 2023-11-06T02:14:49.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.946+0000 7f4d9c08d640 1 -- 172.21.15.49:0/1100377164 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d940c77e0 con 0x7f4d9415cc50 2023-11-06T02:14:49.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.946+0000 7f4d9c08d640 1 -- 172.21.15.49:0/1100377164 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d940c7cc0 con 0x7f4d9415cc50 2023-11-06T02:14:49.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.946+0000 7f4d8a7fc640 1 -- 172.21.15.49:0/1100377164 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f4d8c0b9940 con 0x7f4d9415cc50 2023-11-06T02:14:49.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.946+0000 7f4d8a7fc640 1 --2- 172.21.15.49:0/1100377164 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4d78065a60 0x7f4d78067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:49.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.946+0000 7f4d8a7fc640 1 -- 172.21.15.49:0/1100377164 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6234+0+0 (secure 0 0 0) 0x7f4d8c11edc0 con 0x7f4d9415cc50 2023-11-06T02:14:49.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.946+0000 7f4d99e29640 1 --2- 172.21.15.49:0/1100377164 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4d78065a60 0x7f4d78067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:49.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.946+0000 7f4d99e29640 1 --2- 172.21.15.49:0/1100377164 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4d78065a60 0x7f4d78067f20 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f4d90000c10 tx=0x7f4d90009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:49.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:49.950+0000 7f4d8a7fc640 1 -- 172.21.15.49:0/1100377164 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f4d8c0ec500 con 0x7f4d9415cc50 2023-11-06T02:14:50.105 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.110+0000 7f4d9c08d640 1 -- 172.21.15.49:0/1100377164 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4d94074040 con 0x7f4d9415cc50 2023-11-06T02:14:50.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.114+0000 7f4d8a7fc640 1 -- 172.21.15.49:0/1100377164 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+18145 (secure 0 0 0) 0x7f4d8c0b2020 con 0x7f4d9415cc50 2023-11-06T02:14:50.108 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:50.109 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":82,"fsid":"1a8b035c-6ba4-499a-b876-aea47fd9c0d4","created":"2023-11-06T02:08:06.440857+0000","modified":"2023-11-06T02:14:35.347605+0000","last_up_change":"2023-11-06T02:08:17.982372+0000","last_in_change":"2023-11-06T02:08:08.906631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T02:08:19.252222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T02:08:24.776211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-11-06T02:13:31.459314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-11-06T02:13:32.603710+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ad55af8-0571-4e6a-8722-b5d429b7496d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6809","nonce":38116},{"type":"v1","addr":"172.21.15.49:6811","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6813","nonce":38116},{"type":"v1","addr":"172.21.15.49:6815","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6821","nonce":38116},{"type":"v1","addr":"172.21.15.49:6823","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6817","nonce":38116},{"type":"v1","addr":"172.21.15.49:6819","nonce":38116}]},"public_addr":"172.21.15.49:6811/38116","cluster_addr":"172.21.15.49:6815/38116","heartbeat_back_addr":"172.21.15.49:6823/38116","heartbeat_front_addr":"172.21.15.49:6819/38116","state":["exists","up"]},{"osd":1,"uuid":"c8b31f85-a757-43ed-9419-351acc46611b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":38113},{"type":"v1","addr":"172.21.15.49:6810","nonce":38113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":38113},{"type":"v1","addr":"172.21.15.49:6814","nonce":38113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":38113},{"type":"v1","addr":"172.21.15.49:6822","nonce":38113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":38113},{"type":"v1","addr":"172.21.15.49:6818","nonce":38113}]},"public_addr":"172.21.15.49:6810/38113","cluster_addr":"172.21.15.49:6814/38113","heartbeat_back_addr":"172.21.15.49:6822/38113","heartbeat_front_addr":"172.21.15.49:6818/38113","state":["exists","up"]},{"osd":2,"uuid":"555b3bf3-0a70-4b91-8efd-e4204b5fc26e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":38114},{"type":"v1","addr":"172.21.15.49:6801","nonce":38114}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":38114},{"type":"v1","addr":"172.21.15.49:6803","nonce":38114}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":38114},{"type":"v1","addr":"172.21.15.49:6807","nonce":38114}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":38114},{"type":"v1","addr":"172.21.15.49:6805","nonce":38114}]},"public_addr":"172.21.15.49:6801/38114","cluster_addr":"172.21.15.49:6803/38114","heartbeat_back_addr":"172.21.15.49:6807/38114","heartbeat_front_addr":"172.21.15.49:6805/38114","state":["exists","up"]},{"osd":3,"uuid":"8d4e5d77-03ef-4b9b-bfcb-5ae93a46795c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":38115},{"type":"v1","addr":"172.21.15.49:6825","nonce":38115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":38115},{"type":"v1","addr":"172.21.15.49:6827","nonce":38115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":38115},{"type":"v1","addr":"172.21.15.49:6831","nonce":38115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":38115},{"type":"v1","addr":"172.21.15.49:6829","nonce":38115}]},"public_addr":"172.21.15.49:6825/38115","cluster_addr":"172.21.15.49:6827/38115","heartbeat_back_addr":"172.21.15.49:6831/38115","heartbeat_front_addr":"172.21.15.49:6829/38115","state":["exists","up"]},{"osd":4,"uuid":"dba03ad4-68a3-4cf3-9218-ac611f02eacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":38417},{"type":"v1","addr":"172.21.15.150:6809","nonce":38417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":38417},{"type":"v1","addr":"172.21.15.150:6811","nonce":38417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":38417},{"type":"v1","addr":"172.21.15.150:6815","nonce":38417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":38417},{"type":"v1","addr":"172.21.15.150:6813","nonce":38417}]},"public_addr":"172.21.15.150:6809/38417","cluster_addr":"172.21.15.150:6811/38417","heartbeat_back_addr":"172.21.15.150:6815/38417","heartbeat_front_addr":"172.21.15.150:6813/38417","state":["exists","up"]},{"osd":5,"uuid":"0df9c303-c255-4640-8e91-01f1c3efe18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":38418},{"type":"v1","addr":"172.21.15.150:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":38418},{"type":"v1","addr":"172.21.15.150:6803","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":38418},{"type":"v1","addr":"172.21.15.150:6807","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":38418},{"type":"v1","addr":"172.21.15.150:6805","nonce":38418}]},"public_addr":"172.21.15.150:6801/38418","cluster_addr":"172.21.15.150:6803/38418","heartbeat_back_addr":"172.21.15.150:6807/38418","heartbeat_front_addr":"172.21.15.150:6805/38418","state":["exists","up"]},{"osd":6,"uuid":"e59fa189-4b70-4c06-9722-c02b90828bf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":38428},{"type":"v1","addr":"172.21.15.150:6825","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":38428},{"type":"v1","addr":"172.21.15.150:6827","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":38428},{"type":"v1","addr":"172.21.15.150:6831","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":38428},{"type":"v1","addr":"172.21.15.150:6829","nonce":38428}]},"public_addr":"172.21.15.150:6825/38428","cluster_addr":"172.21.15.150:6827/38428","heartbeat_back_addr":"172.21.15.150:6831/38428","heartbeat_front_addr":"172.21.15.150:6829/38428","state":["exists","up"]},{"osd":7,"uuid":"a64f57e1-ef98-46bd-b304-385759e95898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":38421},{"type":"v1","addr":"172.21.15.150:6817","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":38421},{"type":"v1","addr":"172.21.15.150:6819","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":38421},{"type":"v1","addr":"172.21.15.150:6823","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":38421},{"type":"v1","addr":"172.21.15.150:6821","nonce":38421}]},"public_addr":"172.21.15.150:6817/38421","cluster_addr":"172.21.15.150:6819/38421","heartbeat_back_addr":"172.21.15.150:6823/38421","heartbeat_front_addr":"172.21.15.150:6821/38421","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.438552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.413280+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.296466+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.424966+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.276953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.230984+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.410235+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.480084+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:6835/4037104163":"2023-11-07T02:14:35.347551+0000","172.21.15.49:6834/4037104163":"2023-11-07T02:14:35.347551+0000","172.21.15.49:6835/1130379293":"2023-11-07T02:14:13.284419+0000","172.21.15.49:6834/1130379293":"2023-11-07T02:14:13.284419+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T02:14:50.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.114+0000 7f4d9c08d640 1 -- 172.21.15.49:0/1100377164 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4d78065a60 msgr2=0x7f4d78067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:50.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.114+0000 7f4d9c08d640 1 --2- 172.21.15.49:0/1100377164 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4d78065a60 0x7f4d78067f20 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f4d90000c10 tx=0x7f4d90009040 comp rx=0 tx=0).stop 2023-11-06T02:14:50.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.114+0000 7f4d9c08d640 1 -- 172.21.15.49:0/1100377164 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d9415cc50 msgr2=0x7f4d940b3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:50.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.114+0000 7f4d9c08d640 1 --2- 172.21.15.49:0/1100377164 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d9415cc50 0x7f4d940b3960 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c0b0040 tx=0x7f4d8c019630 comp rx=0 tx=0).stop 2023-11-06T02:14:50.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.114+0000 7f4d9c08d640 1 -- 172.21.15.49:0/1100377164 shutdown_connections 2023-11-06T02:14:50.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.114+0000 7f4d9c08d640 1 --2- 172.21.15.49:0/1100377164 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4d78065a60 0x7f4d78067f20 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:50.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.114+0000 7f4d9c08d640 1 --2- 172.21.15.49:0/1100377164 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d9415cc50 0x7f4d940b3960 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:50.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.114+0000 7f4d9c08d640 1 --2- 172.21.15.49:0/1100377164 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4d940c9690 0x7f4d94151c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:50.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.114+0000 7f4d9c08d640 1 --2- 172.21.15.49:0/1100377164 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d940b0080 0x7f4d940c68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:50.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.114+0000 7f4d9c08d640 1 -- 172.21.15.49:0/1100377164 >> 172.21.15.49:0/1100377164 conn(0x7f4d9406c0a0 msgr2=0x7f4d940ce590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:50.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.114+0000 7f4d9c08d640 1 -- 172.21.15.49:0/1100377164 shutdown_connections 2023-11-06T02:14:50.113 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.114+0000 7f4d9c08d640 1 -- 172.21.15.49:0/1100377164 wait complete. 2023-11-06T02:14:50.128 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T02:08:19.252222+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T02:08:24.776211+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T02:13:31.459314+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T02:13:32.603710+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T02:14:50.129 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T02:14:50.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.258+0000 7f120927d640 1 Processor -- start 2023-11-06T02:14:50.253 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.258+0000 7f120927d640 1 -- start start 2023-11-06T02:14:50.255 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.258+0000 7f120927d640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f12041595a0 0x7f120415dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:50.255 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.258+0000 7f120927d640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f120415a990 0x7f12040c8e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:50.255 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.258+0000 7f120927d640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f120406c5a0 con 0x7f120406d640 2023-11-06T02:14:50.256 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.258+0000 7f120927d640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f120415e250 con 0x7f120415a990 2023-11-06T02:14:50.256 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.258+0000 7f120927d640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f120406c880 con 0x7f12041595a0 2023-11-06T02:14:50.256 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.258+0000 7f12037fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f120415a990 0x7f12040c8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:50.257 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.258+0000 7f12037fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f120415a990 0x7f12040c8e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44380/0 (socket says 172.21.15.49:44380) 2023-11-06T02:14:50.257 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.258+0000 7f12037fe640 1 -- 172.21.15.49:0/3153581160 learned_addr learned my addr 172.21.15.49:0/3153581160 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:50.257 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.258+0000 7f1202ffd640 1 --2- 172.21.15.49:0/3153581160 >> v2:172.21.15.150:3300/0 conn(0x7f12041595a0 0x7f120415dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:50.258 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.258+0000 7f12037fe640 1 -- 172.21.15.49:0/3153581160 >> v2:172.21.15.150:3300/0 conn(0x7f12041595a0 msgr2=0x7f120415dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:50.258 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.258+0000 7f12037fe640 1 --2- 172.21.15.49:0/3153581160 >> v2:172.21.15.150:3300/0 conn(0x7f12041595a0 0x7f120415dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:50.259 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.258+0000 7f12037fe640 1 -- 172.21.15.49:0/3153581160 >> v1:172.21.15.150:6789/0 conn(0x7f120406d640 legacy=0x7f12040b0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:50.259 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.258+0000 7f12037fe640 1 -- 172.21.15.49:0/3153581160 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f120406cb60 con 0x7f120415a990 2023-11-06T02:14:50.263 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.262+0000 7f12037fe640 1 --2- 172.21.15.49:0/3153581160 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f120415a990 0x7f12040c8e90 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f1204071420 tx=0x7f11fc01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c3c539548377165 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:50.263 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.262+0000 7f1201ffb640 1 -- 172.21.15.49:0/3153581160 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11fc0ac640 con 0x7f120415a990 2023-11-06T02:14:50.264 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.262+0000 7f1201ffb640 1 -- 172.21.15.49:0/3153581160 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11fc0ac7a0 con 0x7f120415a990 2023-11-06T02:14:50.264 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.262+0000 7f1201ffb640 1 -- 172.21.15.49:0/3153581160 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11fc0acb30 con 0x7f120415a990 2023-11-06T02:14:50.264 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.262+0000 7f120927d640 1 -- 172.21.15.49:0/3153581160 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f120415a990 msgr2=0x7f12040c8e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:50.264 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.262+0000 7f120927d640 1 --2- 172.21.15.49:0/3153581160 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f120415a990 0x7f12040c8e90 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f1204071420 tx=0x7f11fc01c4d0 comp rx=0 tx=0).stop 2023-11-06T02:14:50.264 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.262+0000 7f120927d640 1 -- 172.21.15.49:0/3153581160 shutdown_connections 2023-11-06T02:14:50.265 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.262+0000 7f120927d640 1 --2- 172.21.15.49:0/3153581160 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f120415a990 0x7f12040c8e90 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:50.265 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.262+0000 7f120927d640 1 --2- 172.21.15.49:0/3153581160 >> v2:172.21.15.150:3300/0 conn(0x7f12041595a0 0x7f120415dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:50.265 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.262+0000 7f120927d640 1 -- 172.21.15.49:0/3153581160 >> 172.21.15.49:0/3153581160 conn(0x7f120406a370 msgr2=0x7f12040ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:50.265 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.262+0000 7f120927d640 1 -- 172.21.15.49:0/3153581160 shutdown_connections 2023-11-06T02:14:50.265 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.262+0000 7f120927d640 1 -- 172.21.15.49:0/3153581160 wait complete. 2023-11-06T02:14:50.266 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.262+0000 7f120927d640 1 Processor -- start 2023-11-06T02:14:50.266 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.262+0000 7f120927d640 1 -- start start 2023-11-06T02:14:50.266 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.262+0000 7f120927d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f120406d640 0x7f12041453a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:50.266 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.262+0000 7f120927d640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f12041595a0 0x7f12040cd830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:50.266 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.262+0000 7f120927d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f120415a990 0x7f1204150fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:50.267 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.262+0000 7f120927d640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f120415ada0 con 0x7f120415a990 2023-11-06T02:14:50.267 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.262+0000 7f120927d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f120415b920 con 0x7f120406d640 2023-11-06T02:14:50.267 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.262+0000 7f120927d640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f120415aff0 con 0x7f12041595a0 2023-11-06T02:14:50.267 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.262+0000 7f12037fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f120415a990 0x7f1204150fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:50.267 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.266+0000 7f12037fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f120415a990 0x7f1204150fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44394/0 (socket says 172.21.15.49:44394) 2023-11-06T02:14:50.268 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.266+0000 7f12037fe640 1 -- 172.21.15.49:0/235198177 learned_addr learned my addr 172.21.15.49:0/235198177 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:50.268 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.266+0000 7f1202ffd640 1 --2- 172.21.15.49:0/235198177 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f120406d640 0x7f12041453a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:50.268 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.266+0000 7f12027fc640 1 --2- 172.21.15.49:0/235198177 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f12041595a0 0x7f12040cd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:50.268 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.266+0000 7f12037fe640 1 -- 172.21.15.49:0/235198177 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f12041595a0 msgr2=0x7f12040cd830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:50.268 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.266+0000 7f12037fe640 1 --2- 172.21.15.49:0/235198177 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f12041595a0 0x7f12040cd830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:50.269 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.266+0000 7f12037fe640 1 -- 172.21.15.49:0/235198177 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f120406d640 msgr2=0x7f12041453a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:50.269 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.266+0000 7f12037fe640 1 --2- 172.21.15.49:0/235198177 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f120406d640 0x7f12041453a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:50.269 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.266+0000 7f12037fe640 1 -- 172.21.15.49:0/235198177 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11fc0094d0 con 0x7f120415a990 2023-11-06T02:14:50.270 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.266+0000 7f12037fe640 1 --2- 172.21.15.49:0/235198177 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f120415a990 0x7f1204150fe0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f11fc01a100 tx=0x7f11fc01ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:50.270 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.266+0000 7f11fb7fe640 1 -- 172.21.15.49:0/235198177 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11fc0b7660 con 0x7f120415a990 2023-11-06T02:14:50.270 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.266+0000 7f11fb7fe640 1 -- 172.21.15.49:0/235198177 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11fc0b77c0 con 0x7f120415a990 2023-11-06T02:14:50.270 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.266+0000 7f11fb7fe640 1 -- 172.21.15.49:0/235198177 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11fc0b7b70 con 0x7f120415a990 2023-11-06T02:14:50.271 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.266+0000 7f120927d640 1 -- 172.21.15.49:0/235198177 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f120415b2d0 con 0x7f120415a990 2023-11-06T02:14:50.271 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.266+0000 7f120927d640 1 -- 172.21.15.49:0/235198177 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12041458e0 con 0x7f120415a990 2023-11-06T02:14:50.271 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.266+0000 7f120927d640 1 -- 172.21.15.49:0/235198177 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f120415ba90 con 0x7f120415a990 2023-11-06T02:14:50.271 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.266+0000 7f11fb7fe640 1 -- 172.21.15.49:0/235198177 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f11fc0b3070 con 0x7f120415a990 2023-11-06T02:14:50.271 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.266+0000 7f11fb7fe640 1 --2- 172.21.15.49:0/235198177 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f11e4065a90 0x7f11e4067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:50.272 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.266+0000 7f11fb7fe640 1 -- 172.21.15.49:0/235198177 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6234+0+0 (secure 0 0 0) 0x7f11fc11dd00 con 0x7f120415a990 2023-11-06T02:14:50.272 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.266+0000 7f1202ffd640 1 --2- 172.21.15.49:0/235198177 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f11e4065a90 0x7f11e4067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:50.272 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.266+0000 7f1202ffd640 1 --2- 172.21.15.49:0/235198177 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f11e4065a90 0x7f11e4067f50 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f11f4003fd0 tx=0x7f11f40039a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:50.273 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.270+0000 7f11fb7fe640 1 -- 172.21.15.49:0/235198177 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f11fc0eb4b0 con 0x7f120415a990 2023-11-06T02:14:50.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.426+0000 7f120927d640 1 -- 172.21.15.49:0/235198177 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1204074040 con 0x7f120415a990 2023-11-06T02:14:50.422 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.426+0000 7f11fb7fe640 1 -- 172.21.15.49:0/235198177 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7f11fc0eead0 con 0x7f120415a990 2023-11-06T02:14:50.422 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2023-11-06T02:14:50.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.430+0000 7f120927d640 1 -- 172.21.15.49:0/235198177 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f11e4065a90 msgr2=0x7f11e4067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:50.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.430+0000 7f120927d640 1 --2- 172.21.15.49:0/235198177 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f11e4065a90 0x7f11e4067f50 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f11f4003fd0 tx=0x7f11f40039a0 comp rx=0 tx=0).stop 2023-11-06T02:14:50.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.430+0000 7f120927d640 1 -- 172.21.15.49:0/235198177 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f120415a990 msgr2=0x7f1204150fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:50.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.430+0000 7f120927d640 1 --2- 172.21.15.49:0/235198177 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f120415a990 0x7f1204150fe0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f11fc01a100 tx=0x7f11fc01ad10 comp rx=0 tx=0).stop 2023-11-06T02:14:50.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.430+0000 7f120927d640 1 -- 172.21.15.49:0/235198177 shutdown_connections 2023-11-06T02:14:50.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.430+0000 7f120927d640 1 --2- 172.21.15.49:0/235198177 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f11e4065a90 0x7f11e4067f50 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:50.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.430+0000 7f120927d640 1 --2- 172.21.15.49:0/235198177 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f120415a990 0x7f1204150fe0 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:50.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.430+0000 7f120927d640 1 --2- 172.21.15.49:0/235198177 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f12041595a0 0x7f12040cd830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:50.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.430+0000 7f120927d640 1 --2- 172.21.15.49:0/235198177 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f120406d640 0x7f12041453a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:50.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.430+0000 7f120927d640 1 -- 172.21.15.49:0/235198177 >> 172.21.15.49:0/235198177 conn(0x7f120406a370 msgr2=0x7f12040b3360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:50.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.430+0000 7f120927d640 1 -- 172.21.15.49:0/235198177 shutdown_connections 2023-11-06T02:14:50.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.430+0000 7f120927d640 1 -- 172.21.15.49:0/235198177 wait complete. 2023-11-06T02:14:50.442 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T02:14:50.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.578+0000 7f4bf1e91640 1 Processor -- start 2023-11-06T02:14:50.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.578+0000 7f4bf1e91640 1 -- start start 2023-11-06T02:14:50.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.578+0000 7f4bf1e91640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f4bec14ffc0 0x7f4bec0ada30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:50.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.578+0000 7f4bf1e91640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4bec070870 0x7f4bec06c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:50.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.578+0000 7f4bf1e91640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bec0c8c80 con 0x7f4bec0adf70 2023-11-06T02:14:50.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.578+0000 7f4bf1e91640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bec0ae4d0 con 0x7f4bec14ffc0 2023-11-06T02:14:50.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.578+0000 7f4bf1e91640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bec1594f0 con 0x7f4bec070870 2023-11-06T02:14:50.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.578+0000 7f4beb7fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f4bec14ffc0 0x7f4bec0ada30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:50.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.578+0000 7f4beb7fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f4bec14ffc0 0x7f4bec0ada30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44400/0 (socket says 172.21.15.49:44400) 2023-11-06T02:14:50.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.578+0000 7f4beb7fe640 1 -- 172.21.15.49:0/1976888710 learned_addr learned my addr 172.21.15.49:0/1976888710 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:50.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.578+0000 7f4bebfff640 1 --2- 172.21.15.49:0/1976888710 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4bec070870 0x7f4bec06c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:50.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.578+0000 7f4beb7fe640 1 -- 172.21.15.49:0/1976888710 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4bec070870 msgr2=0x7f4bec06c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:50.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.578+0000 7f4beb7fe640 1 --2- 172.21.15.49:0/1976888710 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4bec070870 0x7f4bec06c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:50.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.578+0000 7f4beb7fe640 1 -- 172.21.15.49:0/1976888710 >> v1:172.21.15.150:6789/0 conn(0x7f4bec0adf70 legacy=0x7f4bec070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:50.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.578+0000 7f4beb7fe640 1 -- 172.21.15.49:0/1976888710 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bec159700 con 0x7f4bec14ffc0 2023-11-06T02:14:50.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.582+0000 7f4beb7fe640 1 --2- 172.21.15.49:0/1976888710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bec14ffc0 0x7f4bec0ada30 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f4be400d150 tx=0x7f4be401ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d423a2710e7031c1 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:50.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.582+0000 7f4bea7fc640 1 -- 172.21.15.49:0/1976888710 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be40a7b50 con 0x7f4bec14ffc0 2023-11-06T02:14:50.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.582+0000 7f4bea7fc640 1 -- 172.21.15.49:0/1976888710 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4be40ab070 con 0x7f4bec14ffc0 2023-11-06T02:14:50.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.582+0000 7f4bea7fc640 1 -- 172.21.15.49:0/1976888710 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be40b1400 con 0x7f4bec14ffc0 2023-11-06T02:14:50.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.582+0000 7f4bf1e91640 1 -- 172.21.15.49:0/1976888710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bec14ffc0 msgr2=0x7f4bec0ada30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:50.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.582+0000 7f4bf1e91640 1 --2- 172.21.15.49:0/1976888710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bec14ffc0 0x7f4bec0ada30 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f4be400d150 tx=0x7f4be401ec10 comp rx=0 tx=0).stop 2023-11-06T02:14:50.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.582+0000 7f4bf1e91640 1 -- 172.21.15.49:0/1976888710 shutdown_connections 2023-11-06T02:14:50.579 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.582+0000 7f4bf1e91640 1 --2- 172.21.15.49:0/1976888710 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4bec070870 0x7f4bec06c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:50.579 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.582+0000 7f4bf1e91640 1 --2- 172.21.15.49:0/1976888710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bec14ffc0 0x7f4bec0ada30 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:50.579 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.582+0000 7f4bf1e91640 1 -- 172.21.15.49:0/1976888710 >> 172.21.15.49:0/1976888710 conn(0x7f4bec06a370 msgr2=0x7f4bec0ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:50.579 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.582+0000 7f4bf1e91640 1 -- 172.21.15.49:0/1976888710 shutdown_connections 2023-11-06T02:14:50.579 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.582+0000 7f4bf1e91640 1 -- 172.21.15.49:0/1976888710 wait complete. 2023-11-06T02:14:50.582 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.582+0000 7f4bf1e91640 1 Processor -- start 2023-11-06T02:14:50.582 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.582+0000 7f4bf1e91640 1 -- start start 2023-11-06T02:14:50.583 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.582+0000 7f4bf1e91640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4bec070870 0x7f4bec143190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:50.583 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.582+0000 7f4bf1e91640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bec0adf70 0x7f4bec148d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:50.583 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.582+0000 7f4bf1e91640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4bec14ffc0 0x7f4bec0b20f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:50.583 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.582+0000 7f4bf1e91640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bec147cb0 con 0x7f4bec0adf70 2023-11-06T02:14:50.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.582+0000 7f4bf1e91640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bec147f90 con 0x7f4bec14ffc0 2023-11-06T02:14:50.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.582+0000 7f4bf1e91640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bec148270 con 0x7f4bec070870 2023-11-06T02:14:50.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.582+0000 7f4beaffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bec0adf70 0x7f4bec148d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:50.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.582+0000 7f4beaffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bec0adf70 0x7f4bec148d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44404/0 (socket says 172.21.15.49:44404) 2023-11-06T02:14:50.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.582+0000 7f4beaffd640 1 -- 172.21.15.49:0/182104670 learned_addr learned my addr 172.21.15.49:0/182104670 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:50.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.582+0000 7f4beb7fe640 1 --2- 172.21.15.49:0/182104670 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4bec070870 0x7f4bec143190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:50.585 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.582+0000 7f4bebfff640 1 --2- 172.21.15.49:0/182104670 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4bec14ffc0 0x7f4bec0b20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:50.585 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.582+0000 7f4beaffd640 1 -- 172.21.15.49:0/182104670 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4bec070870 msgr2=0x7f4bec143190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:50.585 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.582+0000 7f4beaffd640 1 --2- 172.21.15.49:0/182104670 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4bec070870 0x7f4bec143190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:50.585 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.582+0000 7f4beaffd640 1 -- 172.21.15.49:0/182104670 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4bec14ffc0 msgr2=0x7f4bec0b20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:50.586 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.582+0000 7f4beaffd640 1 --2- 172.21.15.49:0/182104670 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4bec14ffc0 0x7f4bec0b20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:50.586 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.582+0000 7f4beaffd640 1 -- 172.21.15.49:0/182104670 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4be400bd40 con 0x7f4bec0adf70 2023-11-06T02:14:50.586 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.582+0000 7f4bebfff640 1 --2- 172.21.15.49:0/182104670 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4bec14ffc0 0x7f4bec0b20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:14:50.586 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.586+0000 7f4beaffd640 1 --2- 172.21.15.49:0/182104670 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bec0adf70 0x7f4bec148d50 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f4be0007fd0 tx=0x7f4be000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:50.586 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.586+0000 7f4bd3fff640 1 -- 172.21.15.49:0/182104670 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be00052d0 con 0x7f4bec0adf70 2023-11-06T02:14:50.587 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.586+0000 7f4bd3fff640 1 -- 172.21.15.49:0/182104670 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4be0005430 con 0x7f4bec0adf70 2023-11-06T02:14:50.587 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.586+0000 7f4bf1e91640 1 -- 172.21.15.49:0/182104670 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bec148550 con 0x7f4bec0adf70 2023-11-06T02:14:50.588 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.586+0000 7f4bd3fff640 1 -- 172.21.15.49:0/182104670 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be0003840 con 0x7f4bec0adf70 2023-11-06T02:14:50.588 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.586+0000 7f4bf1e91640 1 -- 172.21.15.49:0/182104670 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bec147940 con 0x7f4bec0adf70 2023-11-06T02:14:50.588 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.586+0000 7f4bf1e91640 1 -- 172.21.15.49:0/182104670 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bec074040 con 0x7f4bec0adf70 2023-11-06T02:14:50.588 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.586+0000 7f4bd3fff640 1 -- 172.21.15.49:0/182104670 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f4be00039a0 con 0x7f4bec0adf70 2023-11-06T02:14:50.588 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.586+0000 7f4bd3fff640 1 --2- 172.21.15.49:0/182104670 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4bd8065a60 0x7f4bd8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:50.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.586+0000 7f4bd3fff640 1 -- 172.21.15.49:0/182104670 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6234+0+0 (secure 0 0 0) 0x7f4be0082a20 con 0x7f4bec0adf70 2023-11-06T02:14:50.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.586+0000 7f4beb7fe640 1 --2- 172.21.15.49:0/182104670 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4bd8065a60 0x7f4bd8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:50.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.586+0000 7f4beb7fe640 1 --2- 172.21.15.49:0/182104670 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4bd8065a60 0x7f4bd8067f20 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f4be40b2000 tx=0x7f4be40a9070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:50.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.590+0000 7f4bd3fff640 1 -- 172.21.15.49:0/182104670 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f4be0050130 con 0x7f4bec0adf70 2023-11-06T02:14:50.742 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.746+0000 7f4bf1e91640 1 -- 172.21.15.49:0/182104670 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f4bec148a00 con 0x7f4bec0adf70 2023-11-06T02:14:50.743 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.746+0000 7f4bd3fff640 1 -- 172.21.15.49:0/182104670 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v82) v1 ==== 92+0+10 (secure 0 0 0) 0x7f4be00170a0 con 0x7f4bec0adf70 2023-11-06T02:14:50.744 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 8 2023-11-06T02:14:50.745 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.750+0000 7f4bf1e91640 1 -- 172.21.15.49:0/182104670 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4bd8065a60 msgr2=0x7f4bd8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:50.746 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.750+0000 7f4bf1e91640 1 --2- 172.21.15.49:0/182104670 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4bd8065a60 0x7f4bd8067f20 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f4be40b2000 tx=0x7f4be40a9070 comp rx=0 tx=0).stop 2023-11-06T02:14:50.746 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.750+0000 7f4bf1e91640 1 -- 172.21.15.49:0/182104670 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bec0adf70 msgr2=0x7f4bec148d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:50.746 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.750+0000 7f4bf1e91640 1 --2- 172.21.15.49:0/182104670 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bec0adf70 0x7f4bec148d50 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f4be0007fd0 tx=0x7f4be000c890 comp rx=0 tx=0).stop 2023-11-06T02:14:50.747 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.750+0000 7f4bf1e91640 1 -- 172.21.15.49:0/182104670 shutdown_connections 2023-11-06T02:14:50.747 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.750+0000 7f4bf1e91640 1 --2- 172.21.15.49:0/182104670 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4bd8065a60 0x7f4bd8067f20 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:50.747 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.750+0000 7f4bf1e91640 1 --2- 172.21.15.49:0/182104670 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bec0adf70 0x7f4bec148d50 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:50.747 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.750+0000 7f4bf1e91640 1 --2- 172.21.15.49:0/182104670 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4bec14ffc0 0x7f4bec0b20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:50.748 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.750+0000 7f4bf1e91640 1 --2- 172.21.15.49:0/182104670 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4bec070870 0x7f4bec143190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:50.748 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.750+0000 7f4bf1e91640 1 -- 172.21.15.49:0/182104670 >> 172.21.15.49:0/182104670 conn(0x7f4bec06a370 msgr2=0x7f4bec0c9a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:50.748 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.750+0000 7f4bf1e91640 1 -- 172.21.15.49:0/182104670 shutdown_connections 2023-11-06T02:14:50.749 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.750+0000 7f4bf1e91640 1 -- 172.21.15.49:0/182104670 wait complete. 2023-11-06T02:14:50.762 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T02:14:50.895 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.902+0000 7ffa5748c640 1 Processor -- start 2023-11-06T02:14:50.896 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.902+0000 7ffa5748c640 1 -- start start 2023-11-06T02:14:50.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.902+0000 7ffa5748c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ffa50070870 0x7ffa500b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:50.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.902+0000 7ffa5748c640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7ffa5006d150 0x7ffa5006d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:50.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.902+0000 7ffa5748c640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa5006db40 con 0x7ffa5015a880 2023-11-06T02:14:50.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.902+0000 7ffa5748c640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa500ae360 con 0x7ffa50070870 2023-11-06T02:14:50.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.902+0000 7ffa5748c640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa500ae640 con 0x7ffa5006d150 2023-11-06T02:14:50.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.902+0000 7ffa54a27640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ffa50070870 0x7ffa500b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:50.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.902+0000 7ffa54a27640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ffa50070870 0x7ffa500b15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44406/0 (socket says 172.21.15.49:44406) 2023-11-06T02:14:50.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.902+0000 7ffa54a27640 1 -- 172.21.15.49:0/318065416 learned_addr learned my addr 172.21.15.49:0/318065416 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:50.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.902+0000 7ffa55a29640 1 --2- 172.21.15.49:0/318065416 >> v2:172.21.15.150:3300/0 conn(0x7ffa5006d150 0x7ffa5006d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:50.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.902+0000 7ffa54a27640 1 -- 172.21.15.49:0/318065416 >> v2:172.21.15.150:3300/0 conn(0x7ffa5006d150 msgr2=0x7ffa5006d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:50.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.902+0000 7ffa54a27640 1 --2- 172.21.15.49:0/318065416 >> v2:172.21.15.150:3300/0 conn(0x7ffa5006d150 0x7ffa5006d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:50.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.902+0000 7ffa54a27640 1 -- 172.21.15.49:0/318065416 >> v1:172.21.15.150:6789/0 conn(0x7ffa5015a880 legacy=0x7ffa50070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:50.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.902+0000 7ffa54a27640 1 -- 172.21.15.49:0/318065416 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa500ae920 con 0x7ffa50070870 2023-11-06T02:14:50.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.906+0000 7ffa54a27640 1 --2- 172.21.15.49:0/318065416 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa50070870 0x7ffa500b15c0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7ffa500b02a0 tx=0x7ffa4001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83adfec319abbffb server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:50.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.906+0000 7ffa47fff640 1 -- 172.21.15.49:0/318065416 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa400ad680 con 0x7ffa50070870 2023-11-06T02:14:50.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.906+0000 7ffa47fff640 1 -- 172.21.15.49:0/318065416 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa400ad7e0 con 0x7ffa50070870 2023-11-06T02:14:50.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.906+0000 7ffa47fff640 1 -- 172.21.15.49:0/318065416 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa400adb70 con 0x7ffa50070870 2023-11-06T02:14:50.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.906+0000 7ffa5748c640 1 -- 172.21.15.49:0/318065416 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa50070870 msgr2=0x7ffa500b15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:50.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.906+0000 7ffa5748c640 1 --2- 172.21.15.49:0/318065416 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa50070870 0x7ffa500b15c0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7ffa500b02a0 tx=0x7ffa4001c470 comp rx=0 tx=0).stop 2023-11-06T02:14:50.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.906+0000 7ffa5748c640 1 -- 172.21.15.49:0/318065416 shutdown_connections 2023-11-06T02:14:50.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.906+0000 7ffa5748c640 1 --2- 172.21.15.49:0/318065416 >> v2:172.21.15.150:3300/0 conn(0x7ffa5006d150 0x7ffa5006d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:50.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.906+0000 7ffa5748c640 1 --2- 172.21.15.49:0/318065416 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa50070870 0x7ffa500b15c0 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:50.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.906+0000 7ffa5748c640 1 -- 172.21.15.49:0/318065416 >> 172.21.15.49:0/318065416 conn(0x7ffa5006a370 msgr2=0x7ffa500ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:50.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.906+0000 7ffa5748c640 1 -- 172.21.15.49:0/318065416 shutdown_connections 2023-11-06T02:14:50.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.906+0000 7ffa5748c640 1 -- 172.21.15.49:0/318065416 wait complete. 2023-11-06T02:14:50.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.906+0000 7ffa5748c640 1 Processor -- start 2023-11-06T02:14:50.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.906+0000 7ffa5748c640 1 -- start start 2023-11-06T02:14:50.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.906+0000 7ffa5748c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffa5006d150 0x7ffa5014b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:50.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.906+0000 7ffa5748c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa50070870 0x7ffa5014fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:50.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.906+0000 7ffa5748c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffa5015a880 0x7ffa500c4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:50.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.906+0000 7ffa5748c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa50150c50 con 0x7ffa50070870 2023-11-06T02:14:50.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.906+0000 7ffa5748c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5014bca0 con 0x7ffa5015a880 2023-11-06T02:14:50.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.906+0000 7ffa5748c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa501500e0 con 0x7ffa5006d150 2023-11-06T02:14:50.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.906+0000 7ffa54a27640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa50070870 0x7ffa5014fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:50.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.906+0000 7ffa54a27640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa50070870 0x7ffa5014fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44416/0 (socket says 172.21.15.49:44416) 2023-11-06T02:14:50.908 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.906+0000 7ffa54a27640 1 -- 172.21.15.49:0/4013563862 learned_addr learned my addr 172.21.15.49:0/4013563862 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:50.908 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.906+0000 7ffa54a27640 1 -- 172.21.15.49:0/4013563862 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffa5006d150 msgr2=0x7ffa5014b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:50.908 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.906+0000 7ffa54a27640 1 --2- 172.21.15.49:0/4013563862 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffa5006d150 0x7ffa5014b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:50.908 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.906+0000 7ffa55a29640 1 --2- 172.21.15.49:0/4013563862 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffa5015a880 0x7ffa500c4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:50.908 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.906+0000 7ffa54a27640 1 -- 172.21.15.49:0/4013563862 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffa5015a880 msgr2=0x7ffa500c4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:50.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.906+0000 7ffa54a27640 1 --2- 172.21.15.49:0/4013563862 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffa5015a880 0x7ffa500c4050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:50.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.906+0000 7ffa54a27640 1 -- 172.21.15.49:0/4013563862 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa400094d0 con 0x7ffa50070870 2023-11-06T02:14:50.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.910+0000 7ffa54a27640 1 --2- 172.21.15.49:0/4013563862 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa50070870 0x7ffa5014fba0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7ffa40019080 tx=0x7ffa40019210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:50.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.910+0000 7ffa45ffb640 1 -- 172.21.15.49:0/4013563862 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa400ade20 con 0x7ffa50070870 2023-11-06T02:14:50.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.910+0000 7ffa45ffb640 1 -- 172.21.15.49:0/4013563862 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa400b8470 con 0x7ffa50070870 2023-11-06T02:14:50.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.910+0000 7ffa5748c640 1 -- 172.21.15.49:0/4013563862 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa501503b0 con 0x7ffa50070870 2023-11-06T02:14:50.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.910+0000 7ffa45ffb640 1 -- 172.21.15.49:0/4013563862 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa400b87e0 con 0x7ffa50070870 2023-11-06T02:14:50.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.910+0000 7ffa5748c640 1 -- 172.21.15.49:0/4013563862 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa50150970 con 0x7ffa50070870 2023-11-06T02:14:50.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.910+0000 7ffa5748c640 1 -- 172.21.15.49:0/4013563862 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa50074040 con 0x7ffa50070870 2023-11-06T02:14:50.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.910+0000 7ffa45ffb640 1 -- 172.21.15.49:0/4013563862 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7ffa400b4070 con 0x7ffa50070870 2023-11-06T02:14:50.911 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.910+0000 7ffa45ffb640 1 --2- 172.21.15.49:0/4013563862 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ffa34065a90 0x7ffa34067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:50.911 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.910+0000 7ffa45ffb640 1 -- 172.21.15.49:0/4013563862 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6234+0+0 (secure 0 0 0) 0x7ffa4011d720 con 0x7ffa50070870 2023-11-06T02:14:50.911 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.910+0000 7ffa55228640 1 --2- 172.21.15.49:0/4013563862 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ffa34065a90 0x7ffa34067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:50.913 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.910+0000 7ffa55228640 1 --2- 172.21.15.49:0/4013563862 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ffa34065a90 0x7ffa34067f50 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c000f70 tx=0x7ffa4c0079e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:50.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:50.914+0000 7ffa45ffb640 1 -- 172.21.15.49:0/4013563862 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7ffa400eaf60 con 0x7ffa50070870 2023-11-06T02:14:51.063 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.070+0000 7ffa5748c640 1 -- 172.21.15.49:0/4013563862 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ffa5006cbf0 con 0x7ffa50070870 2023-11-06T02:14:51.064 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.070+0000 7ffa45ffb640 1 -- 172.21.15.49:0/4013563862 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v82) v1 ==== 104+0+11 (secure 0 0 0) 0x7ffa4011be60 con 0x7ffa50070870 2023-11-06T02:14:51.064 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2023-11-06T02:14:51.066 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.070+0000 7ffa5748c640 1 -- 172.21.15.49:0/4013563862 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ffa34065a90 msgr2=0x7ffa34067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:51.066 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.070+0000 7ffa5748c640 1 --2- 172.21.15.49:0/4013563862 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ffa34065a90 0x7ffa34067f50 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c000f70 tx=0x7ffa4c0079e0 comp rx=0 tx=0).stop 2023-11-06T02:14:51.067 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.070+0000 7ffa5748c640 1 -- 172.21.15.49:0/4013563862 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa50070870 msgr2=0x7ffa5014fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:51.067 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.070+0000 7ffa5748c640 1 --2- 172.21.15.49:0/4013563862 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa50070870 0x7ffa5014fba0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7ffa40019080 tx=0x7ffa40019210 comp rx=0 tx=0).stop 2023-11-06T02:14:51.067 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.070+0000 7ffa5748c640 1 -- 172.21.15.49:0/4013563862 shutdown_connections 2023-11-06T02:14:51.067 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.070+0000 7ffa5748c640 1 --2- 172.21.15.49:0/4013563862 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ffa34065a90 0x7ffa34067f50 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:51.067 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.070+0000 7ffa5748c640 1 --2- 172.21.15.49:0/4013563862 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa50070870 0x7ffa5014fba0 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:51.068 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.070+0000 7ffa5748c640 1 --2- 172.21.15.49:0/4013563862 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffa5015a880 0x7ffa500c4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:51.068 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.070+0000 7ffa5748c640 1 --2- 172.21.15.49:0/4013563862 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffa5006d150 0x7ffa5014b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:51.068 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.070+0000 7ffa5748c640 1 -- 172.21.15.49:0/4013563862 >> 172.21.15.49:0/4013563862 conn(0x7ffa5006a370 msgr2=0x7ffa500b36c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:51.068 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.070+0000 7ffa5748c640 1 -- 172.21.15.49:0/4013563862 shutdown_connections 2023-11-06T02:14:51.068 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.070+0000 7ffa5748c640 1 -- 172.21.15.49:0/4013563862 wait complete. 2023-11-06T02:14:51.082 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T02:14:51.211 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.218+0000 7f8c181dd640 1 Processor -- start 2023-11-06T02:14:51.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.218+0000 7f8c181dd640 1 -- start start 2023-11-06T02:14:51.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.218+0000 7f8c181dd640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8c101595a0 0x7f8c1015dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:51.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.218+0000 7f8c181dd640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c1006c5a0 con 0x7f8c1015a990 2023-11-06T02:14:51.213 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.218+0000 7f8c181dd640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c1006c880 con 0x7f8c1006d640 2023-11-06T02:14:51.213 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.218+0000 7f8c181dd640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1015e250 con 0x7f8c101595a0 2023-11-06T02:14:51.215 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.218+0000 7f8c1677a640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f8c1015a990 0x7f8c100c8e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:41954/0 (socket says 172.21.15.49:41954) 2023-11-06T02:14:51.215 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.218+0000 7f8c1677a640 1 -- 172.21.15.49:0/2634215 learned_addr learned my addr 172.21.15.49:0/2634215 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:14:51.215 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.218+0000 7f8c15f79640 1 --2- 172.21.15.49:0/2634215 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8c101595a0 0x7f8c1015dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:51.216 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.218+0000 7f8c14f77640 1 -- 172.21.15.49:0/2634215 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1716240718 0 0) 0x7f8c1006c5a0 con 0x7f8c1015a990 2023-11-06T02:14:51.216 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.218+0000 7f8c14f77640 1 -- 172.21.15.49:0/2634215 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8bf8003530 con 0x7f8c1015a990 2023-11-06T02:14:51.216 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.218+0000 7f8c14f77640 1 -- 172.21.15.49:0/2634215 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1662333186 0 0) 0x7f8bf8003530 con 0x7f8c1015a990 2023-11-06T02:14:51.216 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.218+0000 7f8c14f77640 1 -- 172.21.15.49:0/2634215 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8c101595a0 msgr2=0x7f8c1015dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:51.217 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.218+0000 7f8c14f77640 1 --2- 172.21.15.49:0/2634215 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8c101595a0 0x7f8c1015dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:51.217 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.218+0000 7f8c14f77640 1 -- 172.21.15.49:0/2634215 >> v1:172.21.15.150:6789/0 conn(0x7f8c1006d640 legacy=0x7f8c100b0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:51.217 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.218+0000 7f8c14f77640 1 -- 172.21.15.49:0/2634215 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c1006cb60 con 0x7f8c1015a990 2023-11-06T02:14:51.217 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.218+0000 7f8c14f77640 1 -- 172.21.15.49:0/2634215 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7f8c08001a00 con 0x7f8c1015a990 2023-11-06T02:14:51.217 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.218+0000 7f8c14f77640 1 -- 172.21.15.49:0/2634215 >> v1:172.21.15.49:6789/0 conn(0x7f8c1015a990 legacy=0x7f8c100c8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:51.217 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.218+0000 7f8c14f77640 1 --2- 172.21.15.49:0/2634215 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8c101595a0 0x7f8bf80039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:51.218 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.218+0000 7f8c14f77640 1 --2- 172.21.15.49:0/2634215 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8bf8003ef0 0x7f8bf80047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:51.218 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.218+0000 7f8c14f77640 1 --2- 172.21.15.49:0/2634215 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8bf8004d10 0x7f8bf8005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:51.218 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.218+0000 7f8c14f77640 1 -- 172.21.15.49:0/2634215 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1015e250 con 0x7f8bf8004d10 2023-11-06T02:14:51.218 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.218+0000 7f8c14f77640 1 -- 172.21.15.49:0/2634215 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf800b9f0 con 0x7f8bf8003ef0 2023-11-06T02:14:51.219 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.218+0000 7f8c14f77640 1 -- 172.21.15.49:0/2634215 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf800ddc0 con 0x7f8c101595a0 2023-11-06T02:14:51.219 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.218+0000 7f8c1677a640 1 --2- 172.21.15.49:0/2634215 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8bf8004d10 0x7f8bf8005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:51.219 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.218+0000 7f8c15778640 1 --2- 172.21.15.49:0/2634215 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8bf8003ef0 0x7f8bf80047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:51.219 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.218+0000 7f8c15f79640 1 --2- 172.21.15.49:0/2634215 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8c101595a0 0x7f8bf80039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:51.219 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.218+0000 7f8c1677a640 1 -- 172.21.15.49:0/2634215 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8c101595a0 msgr2=0x7f8bf80039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:51.219 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.218+0000 7f8c1677a640 1 --2- 172.21.15.49:0/2634215 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8c101595a0 0x7f8bf80039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:51.220 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.218+0000 7f8c1677a640 1 -- 172.21.15.49:0/2634215 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8bf8003ef0 msgr2=0x7f8bf80047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:51.220 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.218+0000 7f8c1677a640 1 --2- 172.21.15.49:0/2634215 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8bf8003ef0 0x7f8bf80047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:51.220 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.218+0000 7f8c1677a640 1 -- 172.21.15.49:0/2634215 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bf8003530 con 0x7f8bf8004d10 2023-11-06T02:14:51.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.222+0000 7f8c1677a640 1 --2- 172.21.15.49:0/2634215 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8bf8004d10 0x7f8bf8005190 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f8c1015ada0 tx=0x7f8c0801fa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64fb7eb9628cfc4d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:51.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.222+0000 7f8c14f77640 1 -- 172.21.15.49:0/2634215 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c080aa670 con 0x7f8bf8004d10 2023-11-06T02:14:51.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.222+0000 7f8c14f77640 1 -- 172.21.15.49:0/2634215 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c080aa7f0 con 0x7f8bf8004d10 2023-11-06T02:14:51.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.222+0000 7f8c14f77640 1 -- 172.21.15.49:0/2634215 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c080aabe0 con 0x7f8bf8004d10 2023-11-06T02:14:51.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.222+0000 7f8c181dd640 1 -- 172.21.15.49:0/2634215 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8bf8004d10 msgr2=0x7f8bf8005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:51.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.222+0000 7f8c181dd640 1 --2- 172.21.15.49:0/2634215 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8bf8004d10 0x7f8bf8005190 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f8c1015ada0 tx=0x7f8c0801fa40 comp rx=0 tx=0).stop 2023-11-06T02:14:51.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.222+0000 7f8c1677a640 1 -- 172.21.15.49:0/2634215 reap_dead start 2023-11-06T02:14:51.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.222+0000 7f8c181dd640 1 -- 172.21.15.49:0/2634215 shutdown_connections 2023-11-06T02:14:51.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.222+0000 7f8c181dd640 1 -- 172.21.15.49:0/2634215 >> 172.21.15.49:0/2634215 conn(0x7f8c1006a370 msgr2=0x7f8c100ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:51.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.222+0000 7f8c181dd640 1 -- 172.21.15.49:0/2634215 shutdown_connections 2023-11-06T02:14:51.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.222+0000 7f8c181dd640 1 -- 172.21.15.49:0/2634215 wait complete. 2023-11-06T02:14:51.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.222+0000 7f8c181dd640 1 Processor -- start 2023-11-06T02:14:51.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.222+0000 7f8c181dd640 1 -- start start 2023-11-06T02:14:51.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.222+0000 7f8c181dd640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8bf8004d10 0x7f8c10150a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:51.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.226+0000 7f8c181dd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8c101561d0 0x7f8c10156650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:51.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.226+0000 7f8c181dd640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8c10158220 0x7f8c101586a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:51.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.226+0000 7f8c181dd640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c10156b90 con 0x7f8c101561d0 2023-11-06T02:14:51.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.226+0000 7f8c181dd640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c10151a70 con 0x7f8bf8004d10 2023-11-06T02:14:51.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.226+0000 7f8c181dd640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c10151d50 con 0x7f8c10158220 2023-11-06T02:14:51.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.226+0000 7f8c15778640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8c101561d0 0x7f8c10156650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:51.229 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.226+0000 7f8c15778640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8c101561d0 0x7f8c10156650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44422/0 (socket says 172.21.15.49:44422) 2023-11-06T02:14:51.229 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.226+0000 7f8c15778640 1 -- 172.21.15.49:0/2926151227 learned_addr learned my addr 172.21.15.49:0/2926151227 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:51.229 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.226+0000 7f8c15f79640 1 --2- 172.21.15.49:0/2926151227 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8bf8004d10 0x7f8c10150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:51.229 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.226+0000 7f8c1677a640 1 --2- 172.21.15.49:0/2926151227 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8c10158220 0x7f8c101586a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:51.229 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.226+0000 7f8c15778640 1 -- 172.21.15.49:0/2926151227 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8c10158220 msgr2=0x7f8c101586a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:51.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.226+0000 7f8c15778640 1 --2- 172.21.15.49:0/2926151227 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8c10158220 0x7f8c101586a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:51.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.226+0000 7f8c15778640 1 -- 172.21.15.49:0/2926151227 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8bf8004d10 msgr2=0x7f8c10150a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:51.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.226+0000 7f8c15778640 1 --2- 172.21.15.49:0/2926151227 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8bf8004d10 0x7f8c10150a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:51.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.226+0000 7f8c15778640 1 -- 172.21.15.49:0/2926151227 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c10152030 con 0x7f8c101561d0 2023-11-06T02:14:51.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.226+0000 7f8c1677a640 1 --2- 172.21.15.49:0/2926151227 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8c10158220 0x7f8c101586a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:14:51.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.226+0000 7f8c15778640 1 --2- 172.21.15.49:0/2926151227 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8c101561d0 0x7f8c10156650 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c00e500 tx=0x7f8c0c00e9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:51.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.226+0000 7f8c067fc640 1 -- 172.21.15.49:0/2926151227 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c0052e0 con 0x7f8c101561d0 2023-11-06T02:14:51.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.226+0000 7f8c067fc640 1 -- 172.21.15.49:0/2926151227 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c0c005460 con 0x7f8c101561d0 2023-11-06T02:14:51.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.226+0000 7f8c067fc640 1 -- 172.21.15.49:0/2926151227 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c003840 con 0x7f8c101561d0 2023-11-06T02:14:51.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.226+0000 7f8c181dd640 1 -- 172.21.15.49:0/2926151227 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c10152310 con 0x7f8c101561d0 2023-11-06T02:14:51.232 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.226+0000 7f8c181dd640 1 -- 172.21.15.49:0/2926151227 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c10150fa0 con 0x7f8c101561d0 2023-11-06T02:14:51.232 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.226+0000 7f8c181dd640 1 -- 172.21.15.49:0/2926151227 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c1006c210 con 0x7f8c101561d0 2023-11-06T02:14:51.232 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.226+0000 7f8c067fc640 1 -- 172.21.15.49:0/2926151227 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f8c0c0039a0 con 0x7f8c101561d0 2023-11-06T02:14:51.232 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.226+0000 7f8c067fc640 1 --2- 172.21.15.49:0/2926151227 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8bfc065a90 0x7f8bfc067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:51.232 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.226+0000 7f8c067fc640 1 -- 172.21.15.49:0/2926151227 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6234+0+0 (secure 0 0 0) 0x7f8c0c083180 con 0x7f8c101561d0 2023-11-06T02:14:51.232 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.226+0000 7f8c15f79640 1 --2- 172.21.15.49:0/2926151227 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8bfc065a90 0x7f8bfc067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:51.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.226+0000 7f8c15f79640 1 --2- 172.21.15.49:0/2926151227 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8bfc065a90 0x7f8bfc067f50 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f8c101f18d0 tx=0x7f8c00008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:51.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.230+0000 7f8c067fc640 1 -- 172.21.15.49:0/2926151227 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f8c1006c210 con 0x7f8c101561d0 2023-11-06T02:14:51.382 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.386+0000 7f8c181dd640 1 -- 172.21.15.49:0/2926151227 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8c1006c210 con 0x7f8c101561d0 2023-11-06T02:14:51.383 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.386+0000 7f8c067fc640 1 -- 172.21.15.49:0/2926151227 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v82) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8c1006c210 con 0x7f8c101561d0 2023-11-06T02:14:51.383 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2023-11-06T02:14:51.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.390+0000 7f8c181dd640 1 -- 172.21.15.49:0/2926151227 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8bfc065a90 msgr2=0x7f8bfc067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:51.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.390+0000 7f8c181dd640 1 --2- 172.21.15.49:0/2926151227 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8bfc065a90 0x7f8bfc067f50 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f8c101f18d0 tx=0x7f8c00008040 comp rx=0 tx=0).stop 2023-11-06T02:14:51.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.390+0000 7f8c181dd640 1 -- 172.21.15.49:0/2926151227 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8c101561d0 msgr2=0x7f8c10156650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:51.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.390+0000 7f8c181dd640 1 --2- 172.21.15.49:0/2926151227 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8c101561d0 0x7f8c10156650 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c00e500 tx=0x7f8c0c00e9c0 comp rx=0 tx=0).stop 2023-11-06T02:14:51.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.390+0000 7f8c181dd640 1 -- 172.21.15.49:0/2926151227 shutdown_connections 2023-11-06T02:14:51.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.390+0000 7f8c181dd640 1 --2- 172.21.15.49:0/2926151227 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f8bfc065a90 0x7f8bfc067f50 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:51.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.390+0000 7f8c181dd640 1 --2- 172.21.15.49:0/2926151227 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8c101561d0 0x7f8c10156650 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:51.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.390+0000 7f8c181dd640 1 --2- 172.21.15.49:0/2926151227 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8c10158220 0x7f8c101586a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:51.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.390+0000 7f8c181dd640 1 --2- 172.21.15.49:0/2926151227 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8bf8004d10 0x7f8c10150a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:51.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.390+0000 7f8c181dd640 1 -- 172.21.15.49:0/2926151227 >> 172.21.15.49:0/2926151227 conn(0x7f8c1006a370 msgr2=0x7f8c100cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:51.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.390+0000 7f8c181dd640 1 -- 172.21.15.49:0/2926151227 shutdown_connections 2023-11-06T02:14:51.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.390+0000 7f8c181dd640 1 -- 172.21.15.49:0/2926151227 wait complete. 2023-11-06T02:14:51.403 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T02:14:51.526 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.530+0000 7f0925631640 1 Processor -- start 2023-11-06T02:14:51.526 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.530+0000 7f0925631640 1 -- start start 2023-11-06T02:14:51.527 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.530+0000 7f0925631640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f092015abd0 0x7f09200b43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:51.528 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.530+0000 7f0925631640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f09200710c0 0x7f09200b1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:51.528 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.530+0000 7f0925631640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0920069d70 con 0x7f09200b4900 2023-11-06T02:14:51.528 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.534+0000 7f0925631640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09200b4d80 con 0x7f092015abd0 2023-11-06T02:14:51.528 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.534+0000 7f0925631640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09200ca160 con 0x7f09200710c0 2023-11-06T02:14:51.528 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.534+0000 7f091effd640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f092015abd0 0x7f09200b43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:51.529 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.534+0000 7f091effd640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f092015abd0 0x7f09200b43c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44430/0 (socket says 172.21.15.49:44430) 2023-11-06T02:14:51.529 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.534+0000 7f091effd640 1 -- 172.21.15.49:0/2169852615 learned_addr learned my addr 172.21.15.49:0/2169852615 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:51.529 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.534+0000 7f091f7fe640 1 --2- 172.21.15.49:0/2169852615 >> v2:172.21.15.150:3300/0 conn(0x7f09200710c0 0x7f09200b1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:51.529 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.534+0000 7f091effd640 1 -- 172.21.15.49:0/2169852615 >> v2:172.21.15.150:3300/0 conn(0x7f09200710c0 msgr2=0x7f09200b1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:51.529 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.534+0000 7f091effd640 1 --2- 172.21.15.49:0/2169852615 >> v2:172.21.15.150:3300/0 conn(0x7f09200710c0 0x7f09200b1c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:51.530 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.534+0000 7f091effd640 1 -- 172.21.15.49:0/2169852615 >> v1:172.21.15.150:6789/0 conn(0x7f09200b4900 legacy=0x7f09200709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:51.530 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.534+0000 7f091effd640 1 -- 172.21.15.49:0/2169852615 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f092006ad80 con 0x7f092015abd0 2023-11-06T02:14:51.531 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.534+0000 7f091effd640 1 --2- 172.21.15.49:0/2169852615 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f092015abd0 0x7f09200b43c0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f0920066020 tx=0x7f091001c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=174479b04410d80a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:51.531 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.534+0000 7f091dffb640 1 -- 172.21.15.49:0/2169852615 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09100a5790 con 0x7f092015abd0 2023-11-06T02:14:51.531 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.534+0000 7f091dffb640 1 -- 172.21.15.49:0/2169852615 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09100a58f0 con 0x7f092015abd0 2023-11-06T02:14:51.532 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.534+0000 7f091dffb640 1 -- 172.21.15.49:0/2169852615 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09100a5c80 con 0x7f092015abd0 2023-11-06T02:14:51.532 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.534+0000 7f0925631640 1 -- 172.21.15.49:0/2169852615 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f092015abd0 msgr2=0x7f09200b43c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:51.532 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.534+0000 7f0925631640 1 --2- 172.21.15.49:0/2169852615 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f092015abd0 0x7f09200b43c0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f0920066020 tx=0x7f091001c8d0 comp rx=0 tx=0).stop 2023-11-06T02:14:51.532 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.534+0000 7f0925631640 1 -- 172.21.15.49:0/2169852615 shutdown_connections 2023-11-06T02:14:51.533 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.534+0000 7f0925631640 1 --2- 172.21.15.49:0/2169852615 >> v2:172.21.15.150:3300/0 conn(0x7f09200710c0 0x7f09200b1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:51.533 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.534+0000 7f0925631640 1 --2- 172.21.15.49:0/2169852615 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f092015abd0 0x7f09200b43c0 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:51.533 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.534+0000 7f0925631640 1 -- 172.21.15.49:0/2169852615 >> 172.21.15.49:0/2169852615 conn(0x7f09200cee20 msgr2=0x7f09200cf230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:51.533 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.534+0000 7f0925631640 1 -- 172.21.15.49:0/2169852615 shutdown_connections 2023-11-06T02:14:51.533 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.534+0000 7f0925631640 1 -- 172.21.15.49:0/2169852615 wait complete. 2023-11-06T02:14:51.535 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.538+0000 7f0925631640 1 Processor -- start 2023-11-06T02:14:51.536 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.538+0000 7f0925631640 1 -- start start 2023-11-06T02:14:51.536 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.538+0000 7f0925631640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09200710c0 0x7f0920153900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:51.536 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.538+0000 7f0925631640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09200b4900 0x7f09200c7740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:51.536 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.538+0000 7f0925631640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f092015abd0 0x7f09200c4990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:51.536 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.538+0000 7f0925631640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0920154bd0 con 0x7f09200710c0 2023-11-06T02:14:51.537 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.538+0000 7f0925631640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0920153fc0 con 0x7f09200b4900 2023-11-06T02:14:51.537 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.538+0000 7f0925631640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09201542a0 con 0x7f092015abd0 2023-11-06T02:14:51.537 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.538+0000 7f091effd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09200710c0 0x7f0920153900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:51.537 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.538+0000 7f091effd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09200710c0 0x7f0920153900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44434/0 (socket says 172.21.15.49:44434) 2023-11-06T02:14:51.537 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.538+0000 7f091effd640 1 -- 172.21.15.49:0/3734857303 learned_addr learned my addr 172.21.15.49:0/3734857303 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:51.538 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.538+0000 7f091e7fc640 1 --2- 172.21.15.49:0/3734857303 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09200b4900 0x7f09200c7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:51.538 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.538+0000 7f091f7fe640 1 --2- 172.21.15.49:0/3734857303 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f092015abd0 0x7f09200c4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:51.538 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.538+0000 7f091effd640 1 -- 172.21.15.49:0/3734857303 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f092015abd0 msgr2=0x7f09200c4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:51.538 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.538+0000 7f091effd640 1 --2- 172.21.15.49:0/3734857303 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f092015abd0 0x7f09200c4990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:51.538 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.538+0000 7f091effd640 1 -- 172.21.15.49:0/3734857303 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09200b4900 msgr2=0x7f09200c7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:51.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.538+0000 7f091effd640 1 --2- 172.21.15.49:0/3734857303 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09200b4900 0x7f09200c7740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:51.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.538+0000 7f091effd640 1 -- 172.21.15.49:0/3734857303 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0910009b30 con 0x7f09200710c0 2023-11-06T02:14:51.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.538+0000 7f091e7fc640 1 --2- 172.21.15.49:0/3734857303 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09200b4900 0x7f09200c7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:14:51.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.538+0000 7f091f7fe640 1 --2- 172.21.15.49:0/3734857303 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f092015abd0 0x7f09200c4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:14:51.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.538+0000 7f091effd640 1 --2- 172.21.15.49:0/3734857303 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09200710c0 0x7f0920153900 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f09100af040 tx=0x7f0910019670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:51.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.538+0000 7f09077fe640 1 -- 172.21.15.49:0/3734857303 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09100b8670 con 0x7f09200710c0 2023-11-06T02:14:51.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.538+0000 7f09077fe640 1 -- 172.21.15.49:0/3734857303 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09100b87d0 con 0x7f09200710c0 2023-11-06T02:14:51.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.538+0000 7f09077fe640 1 -- 172.21.15.49:0/3734857303 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09100b8b80 con 0x7f09200710c0 2023-11-06T02:14:51.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.538+0000 7f0925631640 1 -- 172.21.15.49:0/3734857303 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0920154580 con 0x7f09200710c0 2023-11-06T02:14:51.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.538+0000 7f0925631640 1 -- 172.21.15.49:0/3734857303 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09200c4ed0 con 0x7f09200710c0 2023-11-06T02:14:51.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.542+0000 7f0925631640 1 -- 172.21.15.49:0/3734857303 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0920074040 con 0x7f09200710c0 2023-11-06T02:14:51.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.542+0000 7f09077fe640 1 -- 172.21.15.49:0/3734857303 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f09100b8ce0 con 0x7f09200710c0 2023-11-06T02:14:51.542 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.542+0000 7f09077fe640 1 --2- 172.21.15.49:0/3734857303 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f090c065a60 0x7f090c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:51.542 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.542+0000 7f09077fe640 1 -- 172.21.15.49:0/3734857303 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6234+0+0 (secure 0 0 0) 0x7f091011e960 con 0x7f09200710c0 2023-11-06T02:14:51.542 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.542+0000 7f091e7fc640 1 --2- 172.21.15.49:0/3734857303 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f090c065a60 0x7f090c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:51.542 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.542+0000 7f091e7fc640 1 --2- 172.21.15.49:0/3734857303 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f090c065a60 0x7f090c067f20 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f09200c9c10 tx=0x7f0914009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:51.543 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.546+0000 7f09077fe640 1 -- 172.21.15.49:0/3734857303 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f09100ec370 con 0x7f09200710c0 2023-11-06T02:14:51.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.738+0000 7f0925631640 1 -- 172.21.15.49:0/3734857303 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f09200c9960 con 0x7f09200710c0 2023-11-06T02:14:51.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.738+0000 7f09077fe640 1 -- 172.21.15.49:0/3734857303 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+4651 (secure 0 0 0) 0x7f09100ef990 con 0x7f09200710c0 2023-11-06T02:14:51.735 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:51.735 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":59,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5714,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/315581318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":315581318},{"type":"v1","addr":"172.21.15.150:6837","nonce":315581318}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41},{"gid":6121,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/136800245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":136800245},{"type":"v1","addr":"172.21.15.150:6835","nonce":136800245}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41},{"gid":7110,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/4259579254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":4259579254},{"type":"v1","addr":"172.21.15.49:6833","nonce":4259579254}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T02:13:33.625701+0000","modified":"2023-11-06T02:14:40.422212+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7752},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7752":{"gid":7752,"name":"c","rank":0,"incarnation":56,"state":"up:active","state_seq":4,"addr":"172.21.15.49:6835/2241629386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2241629386},{"type":"v1","addr":"172.21.15.49:6835","nonce":2241629386}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-06T02:14:51.735 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 59 2023-11-06T02:14:51.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.742+0000 7f0925631640 1 -- 172.21.15.49:0/3734857303 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f090c065a60 msgr2=0x7f090c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:51.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.742+0000 7f0925631640 1 --2- 172.21.15.49:0/3734857303 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f090c065a60 0x7f090c067f20 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f09200c9c10 tx=0x7f0914009040 comp rx=0 tx=0).stop 2023-11-06T02:14:51.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.742+0000 7f0925631640 1 -- 172.21.15.49:0/3734857303 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09200710c0 msgr2=0x7f0920153900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:51.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.742+0000 7f0925631640 1 --2- 172.21.15.49:0/3734857303 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09200710c0 0x7f0920153900 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f09100af040 tx=0x7f0910019670 comp rx=0 tx=0).stop 2023-11-06T02:14:51.738 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.742+0000 7f0925631640 1 -- 172.21.15.49:0/3734857303 shutdown_connections 2023-11-06T02:14:51.738 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.742+0000 7f0925631640 1 --2- 172.21.15.49:0/3734857303 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f090c065a60 0x7f090c067f20 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:51.738 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.742+0000 7f0925631640 1 --2- 172.21.15.49:0/3734857303 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f092015abd0 0x7f09200c4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:51.738 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.742+0000 7f0925631640 1 --2- 172.21.15.49:0/3734857303 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09200b4900 0x7f09200c7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:51.738 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.742+0000 7f0925631640 1 --2- 172.21.15.49:0/3734857303 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09200710c0 0x7f0920153900 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:51.739 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.742+0000 7f0925631640 1 -- 172.21.15.49:0/3734857303 >> 172.21.15.49:0/3734857303 conn(0x7f09200cee20 msgr2=0x7f09200b2db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:51.739 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.742+0000 7f0925631640 1 -- 172.21.15.49:0/3734857303 shutdown_connections 2023-11-06T02:14:51.739 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.742+0000 7f0925631640 1 -- 172.21.15.49:0/3734857303 wait complete. 2023-11-06T02:14:51.754 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T02:14:51.871 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.874+0000 7f7dbc2bf640 1 Processor -- start 2023-11-06T02:14:51.871 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.874+0000 7f7dbc2bf640 1 -- start start 2023-11-06T02:14:51.872 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.878+0000 7f7dbc2bf640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f7db415ab50 0x7f7db40c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:51.872 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.878+0000 7f7dbc2bf640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f7db40cb010 0x7f7db40b0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:51.873 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.878+0000 7f7dbc2bf640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7db40cab80 con 0x7f7db40b05c0 2023-11-06T02:14:51.873 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.878+0000 7f7dbc2bf640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7db40b0a10 con 0x7f7db40cb010 2023-11-06T02:14:51.873 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.878+0000 7f7dbc2bf640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7db415af30 con 0x7f7db415ab50 2023-11-06T02:14:51.873 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.878+0000 7f7db985a640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f7db40cb010 0x7f7db40b0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:51.874 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.878+0000 7f7dba85c640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f7db40b05c0 0x7f7db40700a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:41956/0 (socket says 172.21.15.49:41956) 2023-11-06T02:14:51.874 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.878+0000 7f7dba85c640 1 -- 172.21.15.49:0/72215815 learned_addr learned my addr 172.21.15.49:0/72215815 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:14:51.874 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.878+0000 7f7dba05b640 1 --2- 172.21.15.49:0/72215815 >> v2:172.21.15.150:3300/0 conn(0x7f7db415ab50 0x7f7db40c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:51.874 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.878+0000 7f7db985a640 1 -- 172.21.15.49:0/72215815 >> v2:172.21.15.150:3300/0 conn(0x7f7db415ab50 msgr2=0x7f7db40c9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:51.874 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.878+0000 7f7db985a640 1 --2- 172.21.15.49:0/72215815 >> v2:172.21.15.150:3300/0 conn(0x7f7db415ab50 0x7f7db40c9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:51.875 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.878+0000 7f7db985a640 1 -- 172.21.15.49:0/72215815 >> v1:172.21.15.49:6789/0 conn(0x7f7db40b05c0 legacy=0x7f7db40700a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:51.875 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.878+0000 7f7db985a640 1 -- 172.21.15.49:0/72215815 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7db40cb5b0 con 0x7f7db40cb010 2023-11-06T02:14:51.875 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.878+0000 7f7db9059640 1 -- 172.21.15.49:0/72215815 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 917707604 0 0) 0x7f7db40cab80 con 0x7f7db40b05c0 2023-11-06T02:14:51.876 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.882+0000 7f7db985a640 1 --2- 172.21.15.49:0/72215815 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7db40cb010 0x7f7db40b0080 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f7db40725a0 tx=0x7f7db001c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5788ead9ddc2d167 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:51.876 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.882+0000 7f7db9059640 1 -- 172.21.15.49:0/72215815 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7db00a7070 con 0x7f7db40cb010 2023-11-06T02:14:51.877 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.882+0000 7f7db9059640 1 -- 172.21.15.49:0/72215815 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7db00ab430 con 0x7f7db40cb010 2023-11-06T02:14:51.877 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.882+0000 7f7db9059640 1 -- 172.21.15.49:0/72215815 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7db00ab7c0 con 0x7f7db40cb010 2023-11-06T02:14:51.877 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.882+0000 7f7dbc2bf640 1 -- 172.21.15.49:0/72215815 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7db40cb010 msgr2=0x7f7db40b0080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:51.877 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.882+0000 7f7dbc2bf640 1 --2- 172.21.15.49:0/72215815 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7db40cb010 0x7f7db40b0080 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f7db40725a0 tx=0x7f7db001c430 comp rx=0 tx=0).stop 2023-11-06T02:14:51.877 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.882+0000 7f7dbc2bf640 1 -- 172.21.15.49:0/72215815 shutdown_connections 2023-11-06T02:14:51.878 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.882+0000 7f7dbc2bf640 1 --2- 172.21.15.49:0/72215815 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7db40cb010 0x7f7db40b0080 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:51.878 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.882+0000 7f7dbc2bf640 1 --2- 172.21.15.49:0/72215815 >> v2:172.21.15.150:3300/0 conn(0x7f7db415ab50 0x7f7db40c9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:51.878 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.882+0000 7f7dbc2bf640 1 -- 172.21.15.49:0/72215815 >> 172.21.15.49:0/72215815 conn(0x7f7db406c0a0 msgr2=0x7f7db406c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:51.878 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.882+0000 7f7dbc2bf640 1 -- 172.21.15.49:0/72215815 shutdown_connections 2023-11-06T02:14:51.879 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.882+0000 7f7dbc2bf640 1 -- 172.21.15.49:0/72215815 wait complete. 2023-11-06T02:14:51.881 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.882+0000 7f7dbc2bf640 1 Processor -- start 2023-11-06T02:14:51.881 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.882+0000 7f7dbc2bf640 1 -- start start 2023-11-06T02:14:51.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.882+0000 7f7dbc2bf640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7db40b05c0 0x7f7db4155380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:51.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.882+0000 7f7dbc2bf640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7db40cb010 0x7f7db414b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:51.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.882+0000 7f7dbc2bf640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7db415ab50 0x7f7db415b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:51.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.882+0000 7f7dbc2bf640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7db415c500 con 0x7f7db40b05c0 2023-11-06T02:14:51.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.882+0000 7f7dbc2bf640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7db414b920 con 0x7f7db40cb010 2023-11-06T02:14:51.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.882+0000 7f7dbc2bf640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7db415b670 con 0x7f7db415ab50 2023-11-06T02:14:51.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.882+0000 7f7dba05b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7db40b05c0 0x7f7db4155380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:51.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.882+0000 7f7dba05b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7db40b05c0 0x7f7db4155380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44452/0 (socket says 172.21.15.49:44452) 2023-11-06T02:14:51.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.882+0000 7f7dba05b640 1 -- 172.21.15.49:0/725382889 learned_addr learned my addr 172.21.15.49:0/725382889 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:51.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.882+0000 7f7dba05b640 1 -- 172.21.15.49:0/725382889 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7db415ab50 msgr2=0x7f7db415b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:51.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.882+0000 7f7dba85c640 1 --2- 172.21.15.49:0/725382889 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7db415ab50 0x7f7db415b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:51.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.882+0000 7f7db985a640 1 --2- 172.21.15.49:0/725382889 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7db40cb010 0x7f7db414b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:51.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.882+0000 7f7dba05b640 1 --2- 172.21.15.49:0/725382889 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7db415ab50 0x7f7db415b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:51.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.882+0000 7f7dba05b640 1 -- 172.21.15.49:0/725382889 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7db40cb010 msgr2=0x7f7db414b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:51.885 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.882+0000 7f7dba05b640 1 --2- 172.21.15.49:0/725382889 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7db40cb010 0x7f7db414b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:51.885 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.882+0000 7f7dba05b640 1 -- 172.21.15.49:0/725382889 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7db0009420 con 0x7f7db40b05c0 2023-11-06T02:14:51.885 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.882+0000 7f7dba05b640 1 --2- 172.21.15.49:0/725382889 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7db40b05c0 0x7f7db4155380 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f7da400d800 tx=0x7f7da400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:51.885 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.882+0000 7f7daaffd640 1 -- 172.21.15.49:0/725382889 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7da4018070 con 0x7f7db40b05c0 2023-11-06T02:14:51.885 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.882+0000 7f7daaffd640 1 -- 172.21.15.49:0/725382889 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7da40059a0 con 0x7f7db40b05c0 2023-11-06T02:14:51.886 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.882+0000 7f7dbc2bf640 1 -- 172.21.15.49:0/725382889 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7db415b880 con 0x7f7db40b05c0 2023-11-06T02:14:51.886 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.886+0000 7f7daaffd640 1 -- 172.21.15.49:0/725382889 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7da4005b50 con 0x7f7db40b05c0 2023-11-06T02:14:51.886 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.886+0000 7f7dbc2bf640 1 -- 172.21.15.49:0/725382889 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7db415be40 con 0x7f7db40b05c0 2023-11-06T02:14:51.886 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.886+0000 7f7dbc2bf640 1 -- 172.21.15.49:0/725382889 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7db40b0de0 con 0x7f7db40b05c0 2023-11-06T02:14:51.886 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.886+0000 7f7daaffd640 1 -- 172.21.15.49:0/725382889 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f7da4005cb0 con 0x7f7db40b05c0 2023-11-06T02:14:51.886 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.886+0000 7f7daaffd640 1 --2- 172.21.15.49:0/725382889 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7d98065a60 0x7f7d98067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:51.887 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.886+0000 7f7daaffd640 1 -- 172.21.15.49:0/725382889 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6234+0+0 (secure 0 0 0) 0x7f7da40831e0 con 0x7f7db40b05c0 2023-11-06T02:14:51.887 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.886+0000 7f7db985a640 1 --2- 172.21.15.49:0/725382889 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7d98065a60 0x7f7d98067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:51.887 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.886+0000 7f7db985a640 1 --2- 172.21.15.49:0/725382889 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7d98065a60 0x7f7d98067f20 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f7db00af000 tx=0x7f7db0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:51.888 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:51.890+0000 7f7daaffd640 1 -- 172.21.15.49:0/725382889 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f7da4050a20 con 0x7f7db40b05c0 2023-11-06T02:14:52.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.046+0000 7f7dbc2bf640 1 -- 172.21.15.49:0/725382889 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7db40cbe20 con 0x7f7db40b05c0 2023-11-06T02:14:52.042 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.046+0000 7f7daaffd640 1 -- 172.21.15.49:0/725382889 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+18145 (secure 0 0 0) 0x7f7da4054040 con 0x7f7db40b05c0 2023-11-06T02:14:52.043 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:52.043 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":82,"fsid":"1a8b035c-6ba4-499a-b876-aea47fd9c0d4","created":"2023-11-06T02:08:06.440857+0000","modified":"2023-11-06T02:14:35.347605+0000","last_up_change":"2023-11-06T02:08:17.982372+0000","last_in_change":"2023-11-06T02:08:08.906631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T02:08:19.252222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T02:08:24.776211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-11-06T02:13:31.459314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-11-06T02:13:32.603710+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ad55af8-0571-4e6a-8722-b5d429b7496d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6809","nonce":38116},{"type":"v1","addr":"172.21.15.49:6811","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6813","nonce":38116},{"type":"v1","addr":"172.21.15.49:6815","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6821","nonce":38116},{"type":"v1","addr":"172.21.15.49:6823","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6817","nonce":38116},{"type":"v1","addr":"172.21.15.49:6819","nonce":38116}]},"public_addr":"172.21.15.49:6811/38116","cluster_addr":"172.21.15.49:6815/38116","heartbeat_back_addr":"172.21.15.49:6823/38116","heartbeat_front_addr":"172.21.15.49:6819/38116","state":["exists","up"]},{"osd":1,"uuid":"c8b31f85-a757-43ed-9419-351acc46611b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":38113},{"type":"v1","addr":"172.21.15.49:6810","nonce":38113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":38113},{"type":"v1","addr":"172.21.15.49:6814","nonce":38113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":38113},{"type":"v1","addr":"172.21.15.49:6822","nonce":38113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":38113},{"type":"v1","addr":"172.21.15.49:6818","nonce":38113}]},"public_addr":"172.21.15.49:6810/38113","cluster_addr":"172.21.15.49:6814/38113","heartbeat_back_addr":"172.21.15.49:6822/38113","heartbeat_front_addr":"172.21.15.49:6818/38113","state":["exists","up"]},{"osd":2,"uuid":"555b3bf3-0a70-4b91-8efd-e4204b5fc26e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":38114},{"type":"v1","addr":"172.21.15.49:6801","nonce":38114}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":38114},{"type":"v1","addr":"172.21.15.49:6803","nonce":38114}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":38114},{"type":"v1","addr":"172.21.15.49:6807","nonce":38114}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":38114},{"type":"v1","addr":"172.21.15.49:6805","nonce":38114}]},"public_addr":"172.21.15.49:6801/38114","cluster_addr":"172.21.15.49:6803/38114","heartbeat_back_addr":"172.21.15.49:6807/38114","heartbeat_front_addr":"172.21.15.49:6805/38114","state":["exists","up"]},{"osd":3,"uuid":"8d4e5d77-03ef-4b9b-bfcb-5ae93a46795c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":38115},{"type":"v1","addr":"172.21.15.49:6825","nonce":38115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":38115},{"type":"v1","addr":"172.21.15.49:6827","nonce":38115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":38115},{"type":"v1","addr":"172.21.15.49:6831","nonce":38115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":38115},{"type":"v1","addr":"172.21.15.49:6829","nonce":38115}]},"public_addr":"172.21.15.49:6825/38115","cluster_addr":"172.21.15.49:6827/38115","heartbeat_back_addr":"172.21.15.49:6831/38115","heartbeat_front_addr":"172.21.15.49:6829/38115","state":["exists","up"]},{"osd":4,"uuid":"dba03ad4-68a3-4cf3-9218-ac611f02eacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":38417},{"type":"v1","addr":"172.21.15.150:6809","nonce":38417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":38417},{"type":"v1","addr":"172.21.15.150:6811","nonce":38417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":38417},{"type":"v1","addr":"172.21.15.150:6815","nonce":38417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":38417},{"type":"v1","addr":"172.21.15.150:6813","nonce":38417}]},"public_addr":"172.21.15.150:6809/38417","cluster_addr":"172.21.15.150:6811/38417","heartbeat_back_addr":"172.21.15.150:6815/38417","heartbeat_front_addr":"172.21.15.150:6813/38417","state":["exists","up"]},{"osd":5,"uuid":"0df9c303-c255-4640-8e91-01f1c3efe18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":38418},{"type":"v1","addr":"172.21.15.150:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":38418},{"type":"v1","addr":"172.21.15.150:6803","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":38418},{"type":"v1","addr":"172.21.15.150:6807","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":38418},{"type":"v1","addr":"172.21.15.150:6805","nonce":38418}]},"public_addr":"172.21.15.150:6801/38418","cluster_addr":"172.21.15.150:6803/38418","heartbeat_back_addr":"172.21.15.150:6807/38418","heartbeat_front_addr":"172.21.15.150:6805/38418","state":["exists","up"]},{"osd":6,"uuid":"e59fa189-4b70-4c06-9722-c02b90828bf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":38428},{"type":"v1","addr":"172.21.15.150:6825","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":38428},{"type":"v1","addr":"172.21.15.150:6827","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":38428},{"type":"v1","addr":"172.21.15.150:6831","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":38428},{"type":"v1","addr":"172.21.15.150:6829","nonce":38428}]},"public_addr":"172.21.15.150:6825/38428","cluster_addr":"172.21.15.150:6827/38428","heartbeat_back_addr":"172.21.15.150:6831/38428","heartbeat_front_addr":"172.21.15.150:6829/38428","state":["exists","up"]},{"osd":7,"uuid":"a64f57e1-ef98-46bd-b304-385759e95898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":38421},{"type":"v1","addr":"172.21.15.150:6817","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":38421},{"type":"v1","addr":"172.21.15.150:6819","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":38421},{"type":"v1","addr":"172.21.15.150:6823","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":38421},{"type":"v1","addr":"172.21.15.150:6821","nonce":38421}]},"public_addr":"172.21.15.150:6817/38421","cluster_addr":"172.21.15.150:6819/38421","heartbeat_back_addr":"172.21.15.150:6823/38421","heartbeat_front_addr":"172.21.15.150:6821/38421","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.438552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.413280+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.296466+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.424966+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.276953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.230984+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.410235+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.480084+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:6835/4037104163":"2023-11-07T02:14:35.347551+0000","172.21.15.49:6834/4037104163":"2023-11-07T02:14:35.347551+0000","172.21.15.49:6835/1130379293":"2023-11-07T02:14:13.284419+0000","172.21.15.49:6834/1130379293":"2023-11-07T02:14:13.284419+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T02:14:52.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.050+0000 7f7dbc2bf640 1 -- 172.21.15.49:0/725382889 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7d98065a60 msgr2=0x7f7d98067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:52.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.050+0000 7f7dbc2bf640 1 --2- 172.21.15.49:0/725382889 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7d98065a60 0x7f7d98067f20 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f7db00af000 tx=0x7f7db0008040 comp rx=0 tx=0).stop 2023-11-06T02:14:52.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.050+0000 7f7dbc2bf640 1 -- 172.21.15.49:0/725382889 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7db40b05c0 msgr2=0x7f7db4155380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:52.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.050+0000 7f7dbc2bf640 1 --2- 172.21.15.49:0/725382889 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7db40b05c0 0x7f7db4155380 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f7da400d800 tx=0x7f7da400dcc0 comp rx=0 tx=0).stop 2023-11-06T02:14:52.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.050+0000 7f7dbc2bf640 1 -- 172.21.15.49:0/725382889 shutdown_connections 2023-11-06T02:14:52.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.050+0000 7f7dbc2bf640 1 --2- 172.21.15.49:0/725382889 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7d98065a60 0x7f7d98067f20 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:52.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.050+0000 7f7dbc2bf640 1 --2- 172.21.15.49:0/725382889 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7db415ab50 0x7f7db415b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:52.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.050+0000 7f7dbc2bf640 1 --2- 172.21.15.49:0/725382889 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7db40cb010 0x7f7db414b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:52.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.050+0000 7f7dbc2bf640 1 --2- 172.21.15.49:0/725382889 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7db40b05c0 0x7f7db4155380 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:52.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.050+0000 7f7dbc2bf640 1 -- 172.21.15.49:0/725382889 >> 172.21.15.49:0/725382889 conn(0x7f7db406c0a0 msgr2=0x7f7db4072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:52.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.050+0000 7f7dbc2bf640 1 -- 172.21.15.49:0/725382889 shutdown_connections 2023-11-06T02:14:52.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.050+0000 7f7dbc2bf640 1 -- 172.21.15.49:0/725382889 wait complete. 2023-11-06T02:14:52.062 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-06T02:14:52.062 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T02:14:52.172 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.178+0000 7f1da878c640 1 Processor -- start 2023-11-06T02:14:52.172 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.178+0000 7f1da878c640 1 -- start start 2023-11-06T02:14:52.173 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.178+0000 7f1da878c640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f1da015ab80 0x7f1da006f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:52.173 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.178+0000 7f1da878c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f1da0071240 0x7f1da015edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:52.173 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.178+0000 7f1da878c640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1da00719e0 con 0x7f1da00cd5d0 2023-11-06T02:14:52.173 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.178+0000 7f1da878c640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1da015af60 con 0x7f1da0071240 2023-11-06T02:14:52.174 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.178+0000 7f1da878c640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1da0071cc0 con 0x7f1da015ab80 2023-11-06T02:14:52.174 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.178+0000 7f1da5d27640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f1da0071240 0x7f1da015edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:52.175 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.178+0000 7f1da5d27640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f1da0071240 0x7f1da015edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44454/0 (socket says 172.21.15.49:44454) 2023-11-06T02:14:52.175 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.178+0000 7f1da5d27640 1 -- 172.21.15.49:0/243235438 learned_addr learned my addr 172.21.15.49:0/243235438 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:52.175 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.178+0000 7f1da6528640 1 --2- 172.21.15.49:0/243235438 >> v2:172.21.15.150:3300/0 conn(0x7f1da015ab80 0x7f1da006f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:52.175 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.178+0000 7f1da5d27640 1 -- 172.21.15.49:0/243235438 >> v2:172.21.15.150:3300/0 conn(0x7f1da015ab80 msgr2=0x7f1da006f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:52.175 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.178+0000 7f1da5d27640 1 --2- 172.21.15.49:0/243235438 >> v2:172.21.15.150:3300/0 conn(0x7f1da015ab80 0x7f1da006f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:52.176 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.178+0000 7f1da5d27640 1 -- 172.21.15.49:0/243235438 >> v1:172.21.15.150:6789/0 conn(0x7f1da00cd5d0 legacy=0x7f1da00b2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:52.176 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.178+0000 7f1da5d27640 1 -- 172.21.15.49:0/243235438 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1da0071fa0 con 0x7f1da0071240 2023-11-06T02:14:52.177 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.182+0000 7f1da5d27640 1 --2- 172.21.15.49:0/243235438 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1da0071240 0x7f1da015edc0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f1da00b2290 tx=0x7f1d9c01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80090ea5a51f3b37 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:52.177 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.182+0000 7f1da5526640 1 -- 172.21.15.49:0/243235438 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d9c0ad660 con 0x7f1da0071240 2023-11-06T02:14:52.177 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.182+0000 7f1da5526640 1 -- 172.21.15.49:0/243235438 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d9c0ad7c0 con 0x7f1da0071240 2023-11-06T02:14:52.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.182+0000 7f1da5526640 1 -- 172.21.15.49:0/243235438 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d9c0adb50 con 0x7f1da0071240 2023-11-06T02:14:52.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.182+0000 7f1da878c640 1 -- 172.21.15.49:0/243235438 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1da0071240 msgr2=0x7f1da015edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:52.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.182+0000 7f1da878c640 1 --2- 172.21.15.49:0/243235438 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1da0071240 0x7f1da015edc0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f1da00b2290 tx=0x7f1d9c01c470 comp rx=0 tx=0).stop 2023-11-06T02:14:52.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.182+0000 7f1da878c640 1 -- 172.21.15.49:0/243235438 shutdown_connections 2023-11-06T02:14:52.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.182+0000 7f1da878c640 1 --2- 172.21.15.49:0/243235438 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1da0071240 0x7f1da015edc0 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:52.179 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.182+0000 7f1da878c640 1 --2- 172.21.15.49:0/243235438 >> v2:172.21.15.150:3300/0 conn(0x7f1da015ab80 0x7f1da006f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:52.179 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.182+0000 7f1da878c640 1 -- 172.21.15.49:0/243235438 >> 172.21.15.49:0/243235438 conn(0x7f1da00cc150 msgr2=0x7f1da00cc560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:52.179 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.182+0000 7f1da878c640 1 -- 172.21.15.49:0/243235438 shutdown_connections 2023-11-06T02:14:52.179 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.182+0000 7f1da878c640 1 -- 172.21.15.49:0/243235438 wait complete. 2023-11-06T02:14:52.183 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.182+0000 7f1da878c640 1 Processor -- start 2023-11-06T02:14:52.184 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.182+0000 7f1da878c640 1 -- start start 2023-11-06T02:14:52.184 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.182+0000 7f1da878c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1da0071240 0x7f1da01630d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:52.184 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.182+0000 7f1da878c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1da00cd5d0 0x7f1da0155180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:52.184 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.182+0000 7f1da878c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1da015ab80 0x7f1da015d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:52.185 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.182+0000 7f1da878c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1da00afc40 con 0x7f1da00cd5d0 2023-11-06T02:14:52.185 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.182+0000 7f1da878c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1da0163fe0 con 0x7f1da0071240 2023-11-06T02:14:52.185 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.182+0000 7f1da878c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1da0163740 con 0x7f1da015ab80 2023-11-06T02:14:52.185 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.182+0000 7f1da5d27640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1da00cd5d0 0x7f1da0155180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:52.185 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.182+0000 7f1da5d27640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1da00cd5d0 0x7f1da0155180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44468/0 (socket says 172.21.15.49:44468) 2023-11-06T02:14:52.186 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.182+0000 7f1da5d27640 1 -- 172.21.15.49:0/2121776952 learned_addr learned my addr 172.21.15.49:0/2121776952 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:52.186 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.182+0000 7f1da6d29640 1 --2- 172.21.15.49:0/2121776952 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1da015ab80 0x7f1da015d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:52.186 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.182+0000 7f1da6528640 1 --2- 172.21.15.49:0/2121776952 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1da0071240 0x7f1da01630d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:52.186 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.182+0000 7f1da5d27640 1 -- 172.21.15.49:0/2121776952 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1da015ab80 msgr2=0x7f1da015d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:52.186 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.182+0000 7f1da5d27640 1 --2- 172.21.15.49:0/2121776952 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1da015ab80 0x7f1da015d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:52.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.182+0000 7f1da5d27640 1 -- 172.21.15.49:0/2121776952 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1da0071240 msgr2=0x7f1da01630d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:52.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.182+0000 7f1da5d27640 1 --2- 172.21.15.49:0/2121776952 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1da0071240 0x7f1da01630d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:52.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.182+0000 7f1da5d27640 1 -- 172.21.15.49:0/2121776952 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d9c0094d0 con 0x7f1da00cd5d0 2023-11-06T02:14:52.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.182+0000 7f1da6d29640 1 --2- 172.21.15.49:0/2121776952 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1da015ab80 0x7f1da015d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:14:52.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.186+0000 7f1da5d27640 1 --2- 172.21.15.49:0/2121776952 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1da00cd5d0 0x7f1da0155180 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c000c10 tx=0x7f1d9c019210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:52.188 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.186+0000 7f1d96ffd640 1 -- 172.21.15.49:0/2121776952 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d9c0b8620 con 0x7f1da00cd5d0 2023-11-06T02:14:52.188 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.186+0000 7f1d96ffd640 1 -- 172.21.15.49:0/2121776952 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d9c0b87a0 con 0x7f1da00cd5d0 2023-11-06T02:14:52.188 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.186+0000 7f1da878c640 1 -- 172.21.15.49:0/2121776952 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1da00cdf70 con 0x7f1da00cd5d0 2023-11-06T02:14:52.188 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.186+0000 7f1d96ffd640 1 -- 172.21.15.49:0/2121776952 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d9c0b8b50 con 0x7f1da00cd5d0 2023-11-06T02:14:52.188 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.186+0000 7f1da878c640 1 -- 172.21.15.49:0/2121776952 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1da0163d00 con 0x7f1da00cd5d0 2023-11-06T02:14:52.189 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.186+0000 7f1da878c640 1 -- 172.21.15.49:0/2121776952 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1da00afa10 con 0x7f1da00cd5d0 2023-11-06T02:14:52.189 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.186+0000 7f1d96ffd640 1 -- 172.21.15.49:0/2121776952 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f1d9c0b4040 con 0x7f1da00cd5d0 2023-11-06T02:14:52.189 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.186+0000 7f1d96ffd640 1 --2- 172.21.15.49:0/2121776952 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1d84065a60 0x7f1d84067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:52.189 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.186+0000 7f1d96ffd640 1 -- 172.21.15.49:0/2121776952 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6234+0+0 (secure 0 0 0) 0x7f1d9c11d7d0 con 0x7f1da00cd5d0 2023-11-06T02:14:52.189 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.186+0000 7f1da6528640 1 --2- 172.21.15.49:0/2121776952 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1d84065a60 0x7f1d84067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:52.190 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.186+0000 7f1da6528640 1 --2- 172.21.15.49:0/2121776952 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1d84065a60 0x7f1d84067f20 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f1d90009790 tx=0x7f1d90008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:52.190 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.190+0000 7f1d96ffd640 1 -- 172.21.15.49:0/2121776952 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f1d9c0eb210 con 0x7f1da00cd5d0 2023-11-06T02:14:52.315 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T02:14:52.318 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T02:14:52.368 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.374+0000 7f1da878c640 1 -- 172.21.15.49:0/2121776952 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1da0156670 con 0x7f1da00cd5d0 2023-11-06T02:14:52.369 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.374+0000 7f1d96ffd640 1 -- 172.21.15.49:0/2121776952 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+4651 (secure 0 0 0) 0x7f1d9c0ee830 con 0x7f1da00cd5d0 2023-11-06T02:14:52.369 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:52.369 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":59,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5714,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/315581318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":315581318},{"type":"v1","addr":"172.21.15.150:6837","nonce":315581318}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41},{"gid":6121,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/136800245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":136800245},{"type":"v1","addr":"172.21.15.150:6835","nonce":136800245}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41},{"gid":7110,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/4259579254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":4259579254},{"type":"v1","addr":"172.21.15.49:6833","nonce":4259579254}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T02:13:33.625701+0000","modified":"2023-11-06T02:14:40.422212+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7752},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7752":{"gid":7752,"name":"c","rank":0,"incarnation":56,"state":"up:active","state_seq":4,"addr":"172.21.15.49:6835/2241629386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2241629386},{"type":"v1","addr":"172.21.15.49:6835","nonce":2241629386}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-06T02:14:52.370 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 59 2023-11-06T02:14:52.371 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.374+0000 7f1da878c640 1 -- 172.21.15.49:0/2121776952 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1d84065a60 msgr2=0x7f1d84067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:52.371 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.374+0000 7f1da878c640 1 --2- 172.21.15.49:0/2121776952 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1d84065a60 0x7f1d84067f20 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f1d90009790 tx=0x7f1d90008040 comp rx=0 tx=0).stop 2023-11-06T02:14:52.371 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.374+0000 7f1da878c640 1 -- 172.21.15.49:0/2121776952 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1da00cd5d0 msgr2=0x7f1da0155180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:52.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.374+0000 7f1da878c640 1 --2- 172.21.15.49:0/2121776952 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1da00cd5d0 0x7f1da0155180 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c000c10 tx=0x7f1d9c019210 comp rx=0 tx=0).stop 2023-11-06T02:14:52.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.374+0000 7f1da878c640 1 -- 172.21.15.49:0/2121776952 shutdown_connections 2023-11-06T02:14:52.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.374+0000 7f1da878c640 1 --2- 172.21.15.49:0/2121776952 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1d84065a60 0x7f1d84067f20 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:52.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.374+0000 7f1da878c640 1 --2- 172.21.15.49:0/2121776952 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1da00cd5d0 0x7f1da0155180 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:52.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.374+0000 7f1da878c640 1 --2- 172.21.15.49:0/2121776952 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1da015ab80 0x7f1da015d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:52.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.374+0000 7f1da878c640 1 --2- 172.21.15.49:0/2121776952 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1da0071240 0x7f1da01630d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:52.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.374+0000 7f1da878c640 1 -- 172.21.15.49:0/2121776952 >> 172.21.15.49:0/2121776952 conn(0x7f1da00cc150 msgr2=0x7f1da00b3040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:52.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.378+0000 7f1da878c640 1 -- 172.21.15.49:0/2121776952 shutdown_connections 2023-11-06T02:14:52.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.378+0000 7f1da878c640 1 -- 172.21.15.49:0/2121776952 wait complete. 2023-11-06T02:14:52.387 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T02:14:52.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.554+0000 7f340919d640 1 Processor -- start 2023-11-06T02:14:52.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.554+0000 7f340919d640 1 -- start start 2023-11-06T02:14:52.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.554+0000 7f340919d640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f340415ab50 0x7f34040c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:52.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.554+0000 7f340919d640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f34040b0790 0x7f34040c2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:52.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.554+0000 7f340919d640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34040cab80 con 0x7f34040cb010 2023-11-06T02:14:52.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.554+0000 7f340919d640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f340415af30 con 0x7f340415ab50 2023-11-06T02:14:52.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.554+0000 7f340919d640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34040cb5b0 con 0x7f34040b0790 2023-11-06T02:14:52.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.554+0000 7f3402d9d640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f340415ab50 0x7f34040c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:52.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.554+0000 7f3402d9d640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f340415ab50 0x7f34040c9bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44484/0 (socket says 172.21.15.49:44484) 2023-11-06T02:14:52.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.554+0000 7f3402d9d640 1 -- 172.21.15.49:0/1567269884 learned_addr learned my addr 172.21.15.49:0/1567269884 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:52.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.554+0000 7f340359e640 1 --2- 172.21.15.49:0/1567269884 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f34040b0790 0x7f34040c2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:52.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.554+0000 7f3402d9d640 1 -- 172.21.15.49:0/1567269884 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f34040b0790 msgr2=0x7f34040c2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:52.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.554+0000 7f3402d9d640 1 --2- 172.21.15.49:0/1567269884 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f34040b0790 0x7f34040c2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:52.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.554+0000 7f3402d9d640 1 -- 172.21.15.49:0/1567269884 >> v1:172.21.15.150:6789/0 conn(0x7f34040cb010 legacy=0x7f34040b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:52.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.554+0000 7f3402d9d640 1 -- 172.21.15.49:0/1567269884 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34040cb890 con 0x7f340415ab50 2023-11-06T02:14:52.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.558+0000 7f3402d9d640 1 --2- 172.21.15.49:0/1567269884 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f340415ab50 0x7f34040c9bb0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f34040725a0 tx=0x7f33f401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6371aae3e987e93 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:52.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.558+0000 7f340259c640 1 -- 172.21.15.49:0/1567269884 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f40ab650 con 0x7f340415ab50 2023-11-06T02:14:52.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.558+0000 7f340259c640 1 -- 172.21.15.49:0/1567269884 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33f40ab7b0 con 0x7f340415ab50 2023-11-06T02:14:52.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.558+0000 7f340259c640 1 -- 172.21.15.49:0/1567269884 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f40abb40 con 0x7f340415ab50 2023-11-06T02:14:52.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.558+0000 7f340919d640 1 -- 172.21.15.49:0/1567269884 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f340415ab50 msgr2=0x7f34040c9bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:52.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.558+0000 7f340919d640 1 --2- 172.21.15.49:0/1567269884 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f340415ab50 0x7f34040c9bb0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f34040725a0 tx=0x7f33f401c490 comp rx=0 tx=0).stop 2023-11-06T02:14:52.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.558+0000 7f340919d640 1 -- 172.21.15.49:0/1567269884 shutdown_connections 2023-11-06T02:14:52.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.558+0000 7f340919d640 1 --2- 172.21.15.49:0/1567269884 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f34040b0790 0x7f34040c2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:52.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.558+0000 7f340919d640 1 --2- 172.21.15.49:0/1567269884 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f340415ab50 0x7f34040c9bb0 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:52.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.558+0000 7f340919d640 1 -- 172.21.15.49:0/1567269884 >> 172.21.15.49:0/1567269884 conn(0x7f340406c0a0 msgr2=0x7f340406c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:52.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.558+0000 7f340919d640 1 -- 172.21.15.49:0/1567269884 shutdown_connections 2023-11-06T02:14:52.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.558+0000 7f340919d640 1 -- 172.21.15.49:0/1567269884 wait complete. 2023-11-06T02:14:52.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.558+0000 7f340919d640 1 Processor -- start 2023-11-06T02:14:52.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.558+0000 7f340919d640 1 -- start start 2023-11-06T02:14:52.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.558+0000 7f340919d640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f34040b0790 0x7f34041520a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:52.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.558+0000 7f340919d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f34040cb010 0x7f34041f4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:52.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.558+0000 7f340919d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f340415ab50 0x7f34040a82d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:52.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.558+0000 7f340919d640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3404152a60 con 0x7f34040cb010 2023-11-06T02:14:52.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.558+0000 7f340919d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34040c69a0 con 0x7f340415ab50 2023-11-06T02:14:52.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.558+0000 7f340919d640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34040c6bf0 con 0x7f34040b0790 2023-11-06T02:14:52.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.558+0000 7f33fbfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f34040cb010 0x7f34041f4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:52.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.558+0000 7f33fbfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f34040cb010 0x7f34041f4740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44496/0 (socket says 172.21.15.49:44496) 2023-11-06T02:14:52.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.558+0000 7f33fbfff640 1 -- 172.21.15.49:0/3005567403 learned_addr learned my addr 172.21.15.49:0/3005567403 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:52.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.558+0000 7f3402d9d640 1 --2- 172.21.15.49:0/3005567403 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f34040b0790 0x7f34041520a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:52.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.562+0000 7f340359e640 1 --2- 172.21.15.49:0/3005567403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f340415ab50 0x7f34040a82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:52.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.562+0000 7f33fbfff640 1 -- 172.21.15.49:0/3005567403 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f34040b0790 msgr2=0x7f34041520a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:52.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.562+0000 7f33fbfff640 1 --2- 172.21.15.49:0/3005567403 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f34040b0790 0x7f34041520a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:52.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.562+0000 7f33fbfff640 1 -- 172.21.15.49:0/3005567403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f340415ab50 msgr2=0x7f34040a82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:52.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.562+0000 7f33fbfff640 1 --2- 172.21.15.49:0/3005567403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f340415ab50 0x7f34040a82d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:52.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.562+0000 7f33fbfff640 1 -- 172.21.15.49:0/3005567403 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33f4009480 con 0x7f34040cb010 2023-11-06T02:14:52.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.562+0000 7f33fbfff640 1 --2- 172.21.15.49:0/3005567403 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f34040cb010 0x7f34041f4740 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f33ec00c450 tx=0x7f33ec00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:52.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.562+0000 7f33fb7fe640 1 -- 172.21.15.49:0/3005567403 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33ec012830 con 0x7f34040cb010 2023-11-06T02:14:52.563 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.562+0000 7f33fb7fe640 1 -- 172.21.15.49:0/3005567403 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33ec0129b0 con 0x7f34040cb010 2023-11-06T02:14:52.563 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.562+0000 7f340919d640 1 -- 172.21.15.49:0/3005567403 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34040c6ed0 con 0x7f34040cb010 2023-11-06T02:14:52.563 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.562+0000 7f33fb7fe640 1 -- 172.21.15.49:0/3005567403 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33ec012d60 con 0x7f34040cb010 2023-11-06T02:14:52.563 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.562+0000 7f340919d640 1 -- 172.21.15.49:0/3005567403 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34040c7490 con 0x7f34040cb010 2023-11-06T02:14:52.563 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.562+0000 7f340919d640 1 -- 172.21.15.49:0/3005567403 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34040b0de0 con 0x7f34040cb010 2023-11-06T02:14:52.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.562+0000 7f33fb7fe640 1 -- 172.21.15.49:0/3005567403 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f33ec01c460 con 0x7f34040cb010 2023-11-06T02:14:52.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.562+0000 7f33fb7fe640 1 --2- 172.21.15.49:0/3005567403 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f33e4065a60 0x7f33e4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:52.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.562+0000 7f33fb7fe640 1 -- 172.21.15.49:0/3005567403 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6234+0+0 (secure 0 0 0) 0x7f33ec083280 con 0x7f34040cb010 2023-11-06T02:14:52.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.562+0000 7f3402d9d640 1 --2- 172.21.15.49:0/3005567403 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f33e4065a60 0x7f33e4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:52.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.562+0000 7f3402d9d640 1 --2- 172.21.15.49:0/3005567403 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f33e4065a60 0x7f33e4067f20 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f33f40af000 tx=0x7f33f40a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:52.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.566+0000 7f33fb7fe640 1 -- 172.21.15.49:0/3005567403 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f33ec050990 con 0x7f34040cb010 2023-11-06T02:14:52.746 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.750+0000 7f340919d640 1 -- 172.21.15.49:0/3005567403 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3404074040 con 0x7f34040cb010 2023-11-06T02:14:52.746 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.750+0000 7f33fb7fe640 1 -- 172.21.15.49:0/3005567403 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+4651 (secure 0 0 0) 0x7f33ec053fb0 con 0x7f34040cb010 2023-11-06T02:14:52.747 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:52.747 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":59,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5714,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/315581318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":315581318},{"type":"v1","addr":"172.21.15.150:6837","nonce":315581318}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41},{"gid":6121,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/136800245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":136800245},{"type":"v1","addr":"172.21.15.150:6835","nonce":136800245}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41},{"gid":7110,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/4259579254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":4259579254},{"type":"v1","addr":"172.21.15.49:6833","nonce":4259579254}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T02:13:33.625701+0000","modified":"2023-11-06T02:14:40.422212+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7752},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7752":{"gid":7752,"name":"c","rank":0,"incarnation":56,"state":"up:active","state_seq":4,"addr":"172.21.15.49:6835/2241629386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":2241629386},{"type":"v1","addr":"172.21.15.49:6835","nonce":2241629386}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-06T02:14:52.747 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 59 2023-11-06T02:14:52.750 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.754+0000 7f340919d640 1 -- 172.21.15.49:0/3005567403 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f33e4065a60 msgr2=0x7f33e4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:52.751 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.754+0000 7f340919d640 1 --2- 172.21.15.49:0/3005567403 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f33e4065a60 0x7f33e4067f20 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f33f40af000 tx=0x7f33f40a7000 comp rx=0 tx=0).stop 2023-11-06T02:14:52.751 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.754+0000 7f340919d640 1 -- 172.21.15.49:0/3005567403 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f34040cb010 msgr2=0x7f34041f4740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:52.751 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.754+0000 7f340919d640 1 --2- 172.21.15.49:0/3005567403 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f34040cb010 0x7f34041f4740 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f33ec00c450 tx=0x7f33ec00c910 comp rx=0 tx=0).stop 2023-11-06T02:14:52.752 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.754+0000 7f340919d640 1 -- 172.21.15.49:0/3005567403 shutdown_connections 2023-11-06T02:14:52.752 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.754+0000 7f340919d640 1 --2- 172.21.15.49:0/3005567403 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f33e4065a60 0x7f33e4067f20 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:52.752 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.754+0000 7f340919d640 1 --2- 172.21.15.49:0/3005567403 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f34040cb010 0x7f34041f4740 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:52.753 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.754+0000 7f340919d640 1 --2- 172.21.15.49:0/3005567403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f340415ab50 0x7f34040a82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:52.753 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.754+0000 7f340919d640 1 --2- 172.21.15.49:0/3005567403 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f34040b0790 0x7f34041520a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:52.753 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.754+0000 7f340919d640 1 -- 172.21.15.49:0/3005567403 >> 172.21.15.49:0/3005567403 conn(0x7f340406c0a0 msgr2=0x7f3404159d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:52.753 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.754+0000 7f340919d640 1 -- 172.21.15.49:0/3005567403 shutdown_connections 2023-11-06T02:14:52.753 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.754+0000 7f340919d640 1 -- 172.21.15.49:0/3005567403 wait complete. 2023-11-06T02:14:52.766 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T02:14:52.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.886+0000 7fe647a9e640 1 Processor -- start 2023-11-06T02:14:52.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.886+0000 7fe647a9e640 1 -- start start 2023-11-06T02:14:52.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.890+0000 7fe647a9e640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe64015a9a0 0x7fe64015eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:52.885 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.890+0000 7fe647a9e640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fe6400c9690 0x7fe6400cb830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:52.885 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.890+0000 7fe647a9e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe6400cbd70 0x7fe6400709f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:52.885 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.890+0000 7fe647a9e640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe64015ad80 con 0x7fe64015a9a0 2023-11-06T02:14:52.885 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.890+0000 7fe647a9e640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6400ca420 con 0x7fe6400c9690 2023-11-06T02:14:52.886 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.890+0000 7fe647a9e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe640069d70 con 0x7fe6400cbd70 2023-11-06T02:14:52.886 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.890+0000 7fe64583a640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe64015a9a0 0x7fe64015eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:52.886 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.890+0000 7fe64583a640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe64015a9a0 0x7fe64015eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44512/0 (socket says 172.21.15.49:44512) 2023-11-06T02:14:52.886 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.890+0000 7fe64583a640 1 -- 172.21.15.49:0/493143351 learned_addr learned my addr 172.21.15.49:0/493143351 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:52.886 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.890+0000 7fe645039640 1 --2- 172.21.15.49:0/493143351 >> v2:172.21.15.150:3300/0 conn(0x7fe6400c9690 0x7fe6400cb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:52.887 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.890+0000 7fe64603b640 1 --2- 172.21.15.49:0/493143351 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe6400cbd70 0x7fe6400709f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:52.887 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.890+0000 7fe64583a640 1 -- 172.21.15.49:0/493143351 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe6400cbd70 msgr2=0x7fe6400709f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:52.887 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.890+0000 7fe64583a640 1 --2- 172.21.15.49:0/493143351 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe6400cbd70 0x7fe6400709f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:52.887 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.890+0000 7fe64583a640 1 -- 172.21.15.49:0/493143351 >> v2:172.21.15.150:3300/0 conn(0x7fe6400c9690 msgr2=0x7fe6400cb830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:52.887 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.890+0000 7fe64583a640 1 --2- 172.21.15.49:0/493143351 >> v2:172.21.15.150:3300/0 conn(0x7fe6400c9690 0x7fe6400cb830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:52.888 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.890+0000 7fe64583a640 1 -- 172.21.15.49:0/493143351 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe64006a050 con 0x7fe64015a9a0 2023-11-06T02:14:52.888 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.890+0000 7fe64603b640 1 --2- 172.21.15.49:0/493143351 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe6400cbd70 0x7fe6400709f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:14:52.891 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.890+0000 7fe64583a640 1 --2- 172.21.15.49:0/493143351 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe64015a9a0 0x7fe64015eea0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fe6400cffc0 tx=0x7fe63c01c1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94e71ec54f9059df server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:52.891 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.890+0000 7fe644838640 1 -- 172.21.15.49:0/493143351 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe63c0ab600 con 0x7fe64015a9a0 2023-11-06T02:14:52.891 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.890+0000 7fe644838640 1 -- 172.21.15.49:0/493143351 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe63c0a7050 con 0x7fe64015a9a0 2023-11-06T02:14:52.891 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.890+0000 7fe644838640 1 -- 172.21.15.49:0/493143351 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe63c0ab980 con 0x7fe64015a9a0 2023-11-06T02:14:52.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.894+0000 7fe647a9e640 1 -- 172.21.15.49:0/493143351 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe64015a9a0 msgr2=0x7fe64015eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:52.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.894+0000 7fe647a9e640 1 --2- 172.21.15.49:0/493143351 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe64015a9a0 0x7fe64015eea0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fe6400cffc0 tx=0x7fe63c01c1b0 comp rx=0 tx=0).stop 2023-11-06T02:14:52.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.894+0000 7fe647a9e640 1 -- 172.21.15.49:0/493143351 shutdown_connections 2023-11-06T02:14:52.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.894+0000 7fe647a9e640 1 --2- 172.21.15.49:0/493143351 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe6400cbd70 0x7fe6400709f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:52.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.894+0000 7fe647a9e640 1 --2- 172.21.15.49:0/493143351 >> v2:172.21.15.150:3300/0 conn(0x7fe6400c9690 0x7fe6400cb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:52.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.894+0000 7fe647a9e640 1 --2- 172.21.15.49:0/493143351 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe64015a9a0 0x7fe64015eea0 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:52.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.894+0000 7fe647a9e640 1 -- 172.21.15.49:0/493143351 >> 172.21.15.49:0/493143351 conn(0x7fe6400b0bd0 msgr2=0x7fe6400b0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:52.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.894+0000 7fe647a9e640 1 -- 172.21.15.49:0/493143351 shutdown_connections 2023-11-06T02:14:52.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.894+0000 7fe647a9e640 1 -- 172.21.15.49:0/493143351 wait complete. 2023-11-06T02:14:52.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.894+0000 7fe647a9e640 1 Processor -- start 2023-11-06T02:14:52.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.894+0000 7fe647a9e640 1 -- start start 2023-11-06T02:14:52.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.894+0000 7fe647a9e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe6400c9690 0x7fe6401467f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:52.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.894+0000 7fe647a9e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6400cbd70 0x7fe640150250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:52.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.894+0000 7fe647a9e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe64015a9a0 0x7fe640150790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:52.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.894+0000 7fe647a9e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6400aeca0 con 0x7fe6400c9690 2023-11-06T02:14:52.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.894+0000 7fe647a9e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6400aef80 con 0x7fe6400cbd70 2023-11-06T02:14:52.895 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.894+0000 7fe647a9e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6400af260 con 0x7fe64015a9a0 2023-11-06T02:14:52.895 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.894+0000 7fe64583a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe6400c9690 0x7fe6401467f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:52.895 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.894+0000 7fe64583a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe6400c9690 0x7fe6401467f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44526/0 (socket says 172.21.15.49:44526) 2023-11-06T02:14:52.896 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.894+0000 7fe64583a640 1 -- 172.21.15.49:0/2334446642 learned_addr learned my addr 172.21.15.49:0/2334446642 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:52.896 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.894+0000 7fe64603b640 1 --2- 172.21.15.49:0/2334446642 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe64015a9a0 0x7fe640150790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:52.896 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.894+0000 7fe645039640 1 --2- 172.21.15.49:0/2334446642 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6400cbd70 0x7fe640150250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:52.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.894+0000 7fe64583a640 1 -- 172.21.15.49:0/2334446642 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe64015a9a0 msgr2=0x7fe640150790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:52.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.894+0000 7fe64583a640 1 --2- 172.21.15.49:0/2334446642 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe64015a9a0 0x7fe640150790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:52.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.894+0000 7fe64583a640 1 -- 172.21.15.49:0/2334446642 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6400cbd70 msgr2=0x7fe640150250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:52.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.894+0000 7fe64583a640 1 --2- 172.21.15.49:0/2334446642 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6400cbd70 0x7fe640150250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:52.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.894+0000 7fe64583a640 1 -- 172.21.15.49:0/2334446642 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe63c0094d0 con 0x7fe6400c9690 2023-11-06T02:14:52.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.894+0000 7fe64603b640 1 --2- 172.21.15.49:0/2334446642 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe64015a9a0 0x7fe640150790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:14:52.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.894+0000 7fe64583a640 1 --2- 172.21.15.49:0/2334446642 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe6400c9690 0x7fe6401467f0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fe63c0ae040 tx=0x7fe63c01a9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:52.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.894+0000 7fe6367fc640 1 -- 172.21.15.49:0/2334446642 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe63c0b6620 con 0x7fe6400c9690 2023-11-06T02:14:52.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.894+0000 7fe6367fc640 1 -- 172.21.15.49:0/2334446642 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe63c0b67a0 con 0x7fe6400c9690 2023-11-06T02:14:52.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.894+0000 7fe647a9e640 1 -- 172.21.15.49:0/2334446642 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6400af540 con 0x7fe6400c9690 2023-11-06T02:14:52.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.894+0000 7fe6367fc640 1 -- 172.21.15.49:0/2334446642 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe63c0b6b50 con 0x7fe6400c9690 2023-11-06T02:14:52.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.894+0000 7fe647a9e640 1 -- 172.21.15.49:0/2334446642 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6400ce0d0 con 0x7fe6400c9690 2023-11-06T02:14:52.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.894+0000 7fe647a9e640 1 -- 172.21.15.49:0/2334446642 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe640067140 con 0x7fe6400c9690 2023-11-06T02:14:52.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.898+0000 7fe6367fc640 1 -- 172.21.15.49:0/2334446642 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fe63c0b2040 con 0x7fe6400c9690 2023-11-06T02:14:52.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.898+0000 7fe6367fc640 1 --2- 172.21.15.49:0/2334446642 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe624065a60 0x7fe624067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:52.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.898+0000 7fe6367fc640 1 -- 172.21.15.49:0/2334446642 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6234+0+0 (secure 0 0 0) 0x7fe63c11db70 con 0x7fe6400c9690 2023-11-06T02:14:52.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.898+0000 7fe645039640 1 --2- 172.21.15.49:0/2334446642 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe624065a60 0x7fe624067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:52.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.898+0000 7fe645039640 1 --2- 172.21.15.49:0/2334446642 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe624065a60 0x7fe624067f20 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fe6401f8530 tx=0x7fe630008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:52.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:52.902+0000 7fe6367fc640 1 -- 172.21.15.49:0/2334446642 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fe63c0eb400 con 0x7fe6400c9690 2023-11-06T02:14:53.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.058+0000 7fe647a9e640 1 -- 172.21.15.49:0/2334446642 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe6400cc180 con 0x7fe6400c9690 2023-11-06T02:14:53.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.058+0000 7fe6367fc640 1 -- 172.21.15.49:0/2334446642 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+18145 (secure 0 0 0) 0x7fe63c0eea20 con 0x7fe6400c9690 2023-11-06T02:14:53.054 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:53.054 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":82,"fsid":"1a8b035c-6ba4-499a-b876-aea47fd9c0d4","created":"2023-11-06T02:08:06.440857+0000","modified":"2023-11-06T02:14:35.347605+0000","last_up_change":"2023-11-06T02:08:17.982372+0000","last_in_change":"2023-11-06T02:08:08.906631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T02:08:19.252222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T02:08:24.776211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-11-06T02:13:31.459314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-11-06T02:13:32.603710+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ad55af8-0571-4e6a-8722-b5d429b7496d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6809","nonce":38116},{"type":"v1","addr":"172.21.15.49:6811","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6813","nonce":38116},{"type":"v1","addr":"172.21.15.49:6815","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6821","nonce":38116},{"type":"v1","addr":"172.21.15.49:6823","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6817","nonce":38116},{"type":"v1","addr":"172.21.15.49:6819","nonce":38116}]},"public_addr":"172.21.15.49:6811/38116","cluster_addr":"172.21.15.49:6815/38116","heartbeat_back_addr":"172.21.15.49:6823/38116","heartbeat_front_addr":"172.21.15.49:6819/38116","state":["exists","up"]},{"osd":1,"uuid":"c8b31f85-a757-43ed-9419-351acc46611b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":38113},{"type":"v1","addr":"172.21.15.49:6810","nonce":38113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":38113},{"type":"v1","addr":"172.21.15.49:6814","nonce":38113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":38113},{"type":"v1","addr":"172.21.15.49:6822","nonce":38113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":38113},{"type":"v1","addr":"172.21.15.49:6818","nonce":38113}]},"public_addr":"172.21.15.49:6810/38113","cluster_addr":"172.21.15.49:6814/38113","heartbeat_back_addr":"172.21.15.49:6822/38113","heartbeat_front_addr":"172.21.15.49:6818/38113","state":["exists","up"]},{"osd":2,"uuid":"555b3bf3-0a70-4b91-8efd-e4204b5fc26e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":38114},{"type":"v1","addr":"172.21.15.49:6801","nonce":38114}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":38114},{"type":"v1","addr":"172.21.15.49:6803","nonce":38114}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":38114},{"type":"v1","addr":"172.21.15.49:6807","nonce":38114}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":38114},{"type":"v1","addr":"172.21.15.49:6805","nonce":38114}]},"public_addr":"172.21.15.49:6801/38114","cluster_addr":"172.21.15.49:6803/38114","heartbeat_back_addr":"172.21.15.49:6807/38114","heartbeat_front_addr":"172.21.15.49:6805/38114","state":["exists","up"]},{"osd":3,"uuid":"8d4e5d77-03ef-4b9b-bfcb-5ae93a46795c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":38115},{"type":"v1","addr":"172.21.15.49:6825","nonce":38115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":38115},{"type":"v1","addr":"172.21.15.49:6827","nonce":38115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":38115},{"type":"v1","addr":"172.21.15.49:6831","nonce":38115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":38115},{"type":"v1","addr":"172.21.15.49:6829","nonce":38115}]},"public_addr":"172.21.15.49:6825/38115","cluster_addr":"172.21.15.49:6827/38115","heartbeat_back_addr":"172.21.15.49:6831/38115","heartbeat_front_addr":"172.21.15.49:6829/38115","state":["exists","up"]},{"osd":4,"uuid":"dba03ad4-68a3-4cf3-9218-ac611f02eacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":38417},{"type":"v1","addr":"172.21.15.150:6809","nonce":38417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":38417},{"type":"v1","addr":"172.21.15.150:6811","nonce":38417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":38417},{"type":"v1","addr":"172.21.15.150:6815","nonce":38417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":38417},{"type":"v1","addr":"172.21.15.150:6813","nonce":38417}]},"public_addr":"172.21.15.150:6809/38417","cluster_addr":"172.21.15.150:6811/38417","heartbeat_back_addr":"172.21.15.150:6815/38417","heartbeat_front_addr":"172.21.15.150:6813/38417","state":["exists","up"]},{"osd":5,"uuid":"0df9c303-c255-4640-8e91-01f1c3efe18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":38418},{"type":"v1","addr":"172.21.15.150:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":38418},{"type":"v1","addr":"172.21.15.150:6803","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":38418},{"type":"v1","addr":"172.21.15.150:6807","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":38418},{"type":"v1","addr":"172.21.15.150:6805","nonce":38418}]},"public_addr":"172.21.15.150:6801/38418","cluster_addr":"172.21.15.150:6803/38418","heartbeat_back_addr":"172.21.15.150:6807/38418","heartbeat_front_addr":"172.21.15.150:6805/38418","state":["exists","up"]},{"osd":6,"uuid":"e59fa189-4b70-4c06-9722-c02b90828bf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":38428},{"type":"v1","addr":"172.21.15.150:6825","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":38428},{"type":"v1","addr":"172.21.15.150:6827","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":38428},{"type":"v1","addr":"172.21.15.150:6831","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":38428},{"type":"v1","addr":"172.21.15.150:6829","nonce":38428}]},"public_addr":"172.21.15.150:6825/38428","cluster_addr":"172.21.15.150:6827/38428","heartbeat_back_addr":"172.21.15.150:6831/38428","heartbeat_front_addr":"172.21.15.150:6829/38428","state":["exists","up"]},{"osd":7,"uuid":"a64f57e1-ef98-46bd-b304-385759e95898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":38421},{"type":"v1","addr":"172.21.15.150:6817","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":38421},{"type":"v1","addr":"172.21.15.150:6819","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":38421},{"type":"v1","addr":"172.21.15.150:6823","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":38421},{"type":"v1","addr":"172.21.15.150:6821","nonce":38421}]},"public_addr":"172.21.15.150:6817/38421","cluster_addr":"172.21.15.150:6819/38421","heartbeat_back_addr":"172.21.15.150:6823/38421","heartbeat_front_addr":"172.21.15.150:6821/38421","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.438552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.413280+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.296466+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.424966+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.276953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.230984+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.410235+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.480084+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:6835/4037104163":"2023-11-07T02:14:35.347551+0000","172.21.15.49:6834/4037104163":"2023-11-07T02:14:35.347551+0000","172.21.15.49:6835/1130379293":"2023-11-07T02:14:13.284419+0000","172.21.15.49:6834/1130379293":"2023-11-07T02:14:13.284419+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T02:14:53.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.062+0000 7fe647a9e640 1 -- 172.21.15.49:0/2334446642 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe624065a60 msgr2=0x7fe624067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:53.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.062+0000 7fe647a9e640 1 --2- 172.21.15.49:0/2334446642 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe624065a60 0x7fe624067f20 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fe6401f8530 tx=0x7fe630008040 comp rx=0 tx=0).stop 2023-11-06T02:14:53.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.062+0000 7fe647a9e640 1 -- 172.21.15.49:0/2334446642 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe6400c9690 msgr2=0x7fe6401467f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:53.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.062+0000 7fe647a9e640 1 --2- 172.21.15.49:0/2334446642 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe6400c9690 0x7fe6401467f0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fe63c0ae040 tx=0x7fe63c01a9f0 comp rx=0 tx=0).stop 2023-11-06T02:14:53.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.062+0000 7fe647a9e640 1 -- 172.21.15.49:0/2334446642 shutdown_connections 2023-11-06T02:14:53.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.062+0000 7fe647a9e640 1 --2- 172.21.15.49:0/2334446642 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe624065a60 0x7fe624067f20 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:53.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.062+0000 7fe647a9e640 1 --2- 172.21.15.49:0/2334446642 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe64015a9a0 0x7fe640150790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:53.057 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.062+0000 7fe647a9e640 1 --2- 172.21.15.49:0/2334446642 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6400cbd70 0x7fe640150250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:53.057 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.062+0000 7fe647a9e640 1 --2- 172.21.15.49:0/2334446642 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe6400c9690 0x7fe6401467f0 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:53.057 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.062+0000 7fe647a9e640 1 -- 172.21.15.49:0/2334446642 >> 172.21.15.49:0/2334446642 conn(0x7fe6400b0bd0 msgr2=0x7fe6400b43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:53.057 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.062+0000 7fe647a9e640 1 -- 172.21.15.49:0/2334446642 shutdown_connections 2023-11-06T02:14:53.057 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.062+0000 7fe647a9e640 1 -- 172.21.15.49:0/2334446642 wait complete. 2023-11-06T02:14:53.073 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-06T02:14:53.188 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.194+0000 7f7d3d458640 1 Processor -- start 2023-11-06T02:14:53.188 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.194+0000 7f7d3d458640 1 -- start start 2023-11-06T02:14:53.189 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.194+0000 7f7d3d458640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7d3806a9b0 0x7f7d380ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:53.190 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.194+0000 7f7d3d458640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f7d38069d70 0x7f7d3806a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:53.190 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.194+0000 7f7d3d458640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d3815cfd0 con 0x7f7d3815cbf0 2023-11-06T02:14:53.190 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.194+0000 7f7d3d458640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d3806a740 con 0x7f7d38069d70 2023-11-06T02:14:53.190 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.194+0000 7f7d3d458640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d380ce0d0 con 0x7f7d3806a9b0 2023-11-06T02:14:53.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.194+0000 7f7d377fe640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f7d38069d70 0x7f7d3806a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:53.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.194+0000 7f7d367fc640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7d3806a9b0 0x7f7d380ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:53.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.194+0000 7f7d36ffd640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f7d3815cbf0 0x7f7d38161100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.49:33064/0 (socket says 172.21.15.49:33064) 2023-11-06T02:14:53.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.194+0000 7f7d36ffd640 1 -- 172.21.15.49:0/3901280656 learned_addr learned my addr 172.21.15.49:0/3901280656 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:14:53.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.194+0000 7f7d367fc640 1 --2- 172.21.15.49:0/3901280656 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7d3806a9b0 0x7f7d380ce560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.49:56756/0 (socket says 172.21.15.49:56756) 2023-11-06T02:14:53.192 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.194+0000 7f7d377fe640 1 -- 172.21.15.49:0/3901280656 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7d3806a9b0 msgr2=0x7f7d380ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:53.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.194+0000 7f7d377fe640 1 --2- 172.21.15.49:0/3901280656 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7d3806a9b0 0x7f7d380ce560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:53.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.194+0000 7f7d377fe640 1 -- 172.21.15.49:0/3901280656 >> v1:172.21.15.150:6789/0 conn(0x7f7d3815cbf0 legacy=0x7f7d38161100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:53.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.194+0000 7f7d377fe640 1 -- 172.21.15.49:0/3901280656 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d380cddd0 con 0x7f7d38069d70 2023-11-06T02:14:53.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.198+0000 7f7d377fe640 1 --2- 172.21.15.49:0/3901280656 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d38069d70 0x7f7d3806a200 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f7d30009d90 tx=0x7f7d3001c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8da5589a0e24c75a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:53.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.198+0000 7f7d35ffb640 1 -- 172.21.15.49:0/3901280656 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d300a5860 con 0x7f7d38069d70 2023-11-06T02:14:53.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.198+0000 7f7d35ffb640 1 -- 172.21.15.49:0/3901280656 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d300a59c0 con 0x7f7d38069d70 2023-11-06T02:14:53.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.198+0000 7f7d35ffb640 1 -- 172.21.15.49:0/3901280656 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d300a5d50 con 0x7f7d38069d70 2023-11-06T02:14:53.195 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.198+0000 7f7d3d458640 1 -- 172.21.15.49:0/3901280656 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d38069d70 msgr2=0x7f7d3806a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:53.195 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.198+0000 7f7d3d458640 1 --2- 172.21.15.49:0/3901280656 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d38069d70 0x7f7d3806a200 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f7d30009d90 tx=0x7f7d3001c890 comp rx=0 tx=0).stop 2023-11-06T02:14:53.195 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.198+0000 7f7d3d458640 1 -- 172.21.15.49:0/3901280656 shutdown_connections 2023-11-06T02:14:53.196 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.198+0000 7f7d3d458640 1 --2- 172.21.15.49:0/3901280656 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7d3806a9b0 0x7f7d380ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:53.196 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.198+0000 7f7d3d458640 1 --2- 172.21.15.49:0/3901280656 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d38069d70 0x7f7d3806a200 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:53.196 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.198+0000 7f7d3d458640 1 -- 172.21.15.49:0/3901280656 >> 172.21.15.49:0/3901280656 conn(0x7f7d380b0bd0 msgr2=0x7f7d380b0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:53.196 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.198+0000 7f7d3d458640 1 -- 172.21.15.49:0/3901280656 shutdown_connections 2023-11-06T02:14:53.196 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.198+0000 7f7d3d458640 1 -- 172.21.15.49:0/3901280656 wait complete. 2023-11-06T02:14:53.197 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.198+0000 7f7d3d458640 1 Processor -- start 2023-11-06T02:14:53.197 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.202+0000 7f7d3d458640 1 -- start start 2023-11-06T02:14:53.199 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.202+0000 7f7d3d458640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d38069d70 0x7f7d380af200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:53.199 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.202+0000 7f7d3d458640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d3806a9b0 0x7f7d3815e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:53.200 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.202+0000 7f7d3d458640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7d3815cbf0 0x7f7d3813f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:53.200 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.202+0000 7f7d3d458640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d3815dd20 con 0x7f7d3806a9b0 2023-11-06T02:14:53.200 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.202+0000 7f7d3d458640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d3815fa30 con 0x7f7d38069d70 2023-11-06T02:14:53.200 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.202+0000 7f7d3d458640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d3815d170 con 0x7f7d3815cbf0 2023-11-06T02:14:53.200 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.202+0000 7f7d367fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d3806a9b0 0x7f7d3815e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:53.201 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.202+0000 7f7d367fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d3806a9b0 0x7f7d3815e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44532/0 (socket says 172.21.15.49:44532) 2023-11-06T02:14:53.201 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.202+0000 7f7d367fc640 1 -- 172.21.15.49:0/3730790314 learned_addr learned my addr 172.21.15.49:0/3730790314 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:53.201 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.202+0000 7f7d377fe640 1 --2- 172.21.15.49:0/3730790314 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7d3815cbf0 0x7f7d3813f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:53.201 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.202+0000 7f7d36ffd640 1 --2- 172.21.15.49:0/3730790314 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d38069d70 0x7f7d380af200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:53.202 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.202+0000 7f7d367fc640 1 -- 172.21.15.49:0/3730790314 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7d3815cbf0 msgr2=0x7f7d3813f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:53.202 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.202+0000 7f7d367fc640 1 --2- 172.21.15.49:0/3730790314 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7d3815cbf0 0x7f7d3813f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:53.202 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.202+0000 7f7d367fc640 1 -- 172.21.15.49:0/3730790314 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d38069d70 msgr2=0x7f7d380af200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:53.202 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.202+0000 7f7d367fc640 1 --2- 172.21.15.49:0/3730790314 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d38069d70 0x7f7d380af200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:53.202 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.202+0000 7f7d367fc640 1 -- 172.21.15.49:0/3730790314 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d30009a70 con 0x7f7d3806a9b0 2023-11-06T02:14:53.203 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.202+0000 7f7d367fc640 1 --2- 172.21.15.49:0/3730790314 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d3806a9b0 0x7f7d3815e140 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c00e930 tx=0x7f7d2c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:53.203 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.202+0000 7f7d1f7fe640 1 -- 172.21.15.49:0/3730790314 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c009690 con 0x7f7d3806a9b0 2023-11-06T02:14:53.203 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.202+0000 7f7d1f7fe640 1 -- 172.21.15.49:0/3730790314 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d2c00ca00 con 0x7f7d3806a9b0 2023-11-06T02:14:53.203 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.202+0000 7f7d3d458640 1 -- 172.21.15.49:0/3730790314 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d3815d380 con 0x7f7d3806a9b0 2023-11-06T02:14:53.204 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.202+0000 7f7d1f7fe640 1 -- 172.21.15.49:0/3730790314 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c00cd70 con 0x7f7d3806a9b0 2023-11-06T02:14:53.205 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.202+0000 7f7d3d458640 1 -- 172.21.15.49:0/3730790314 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d3815d940 con 0x7f7d3806a9b0 2023-11-06T02:14:53.205 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.202+0000 7f7d3d458640 1 -- 172.21.15.49:0/3730790314 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d38067140 con 0x7f7d3806a9b0 2023-11-06T02:14:53.205 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.202+0000 7f7d1f7fe640 1 -- 172.21.15.49:0/3730790314 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f7d2c013470 con 0x7f7d3806a9b0 2023-11-06T02:14:53.206 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.202+0000 7f7d1f7fe640 1 --2- 172.21.15.49:0/3730790314 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7d24065a60 0x7f7d24067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:53.206 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.202+0000 7f7d1f7fe640 1 -- 172.21.15.49:0/3730790314 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6234+0+0 (secure 0 0 0) 0x7f7d2c083630 con 0x7f7d3806a9b0 2023-11-06T02:14:53.206 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.206+0000 7f7d36ffd640 1 --2- 172.21.15.49:0/3730790314 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7d24065a60 0x7f7d24067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:53.207 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.206+0000 7f7d36ffd640 1 --2- 172.21.15.49:0/3730790314 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7d24065a60 0x7f7d24067f20 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f7d28000f70 tx=0x7f7d28009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:53.207 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.206+0000 7f7d1f7fe640 1 -- 172.21.15.49:0/3730790314 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f7d2c050eb0 con 0x7f7d3806a9b0 2023-11-06T02:14:53.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.390+0000 7f7d3d458640 1 -- 172.21.15.49:0/3730790314 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f7d380b4860 con 0x7f7d3806a9b0 2023-11-06T02:14:53.389 INFO:tasks.ceph.mon.a.smithi049.stderr:2023-11-06T02:14:53.394+0000 7f0164e5a640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-06T02:14:53.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.410+0000 7f7d1f7fe640 1 -- 172.21.15.49:0/3730790314 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v60) v1 ==== 161+0+0 (secure 0 0 0) 0x7f7d2c0544d0 con 0x7f7d3806a9b0 2023-11-06T02:14:53.409 INFO:teuthology.orchestra.run.smithi049.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-06T02:14:53.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.414+0000 7f7d3d458640 1 -- 172.21.15.49:0/3730790314 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7d24065a60 msgr2=0x7f7d24067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:53.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.414+0000 7f7d3d458640 1 --2- 172.21.15.49:0/3730790314 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7d24065a60 0x7f7d24067f20 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f7d28000f70 tx=0x7f7d28009040 comp rx=0 tx=0).stop 2023-11-06T02:14:53.411 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.414+0000 7f7d3d458640 1 -- 172.21.15.49:0/3730790314 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d3806a9b0 msgr2=0x7f7d3815e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:53.411 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.414+0000 7f7d3d458640 1 --2- 172.21.15.49:0/3730790314 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d3806a9b0 0x7f7d3815e140 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c00e930 tx=0x7f7d2c00edf0 comp rx=0 tx=0).stop 2023-11-06T02:14:53.411 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.414+0000 7f7d3d458640 1 -- 172.21.15.49:0/3730790314 shutdown_connections 2023-11-06T02:14:53.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.414+0000 7f7d3d458640 1 --2- 172.21.15.49:0/3730790314 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7d24065a60 0x7f7d24067f20 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:53.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.414+0000 7f7d3d458640 1 --2- 172.21.15.49:0/3730790314 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d3806a9b0 0x7f7d3815e140 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:53.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.414+0000 7f7d3d458640 1 --2- 172.21.15.49:0/3730790314 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7d3815cbf0 0x7f7d3813f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:53.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.414+0000 7f7d3d458640 1 --2- 172.21.15.49:0/3730790314 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d38069d70 0x7f7d380af200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:53.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.414+0000 7f7d3d458640 1 -- 172.21.15.49:0/3730790314 >> 172.21.15.49:0/3730790314 conn(0x7f7d380b0bd0 msgr2=0x7f7d380b0fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:53.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.414+0000 7f7d3d458640 1 -- 172.21.15.49:0/3730790314 shutdown_connections 2023-11-06T02:14:53.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.414+0000 7f7d3d458640 1 -- 172.21.15.49:0/3730790314 wait complete. 2023-11-06T02:14:53.413 INFO:tasks.ceph.mds.c.smithi049.stderr: -1340> 2023-11-06T02:14:42.390+0000 7fabcf0f2640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-06T02:14:53.425 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-06T02:14:53.454 INFO:tasks.ceph.mds.c.smithi049.stdout:starting mds.c at 2023-11-06T02:14:53.545 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.550+0000 7fb9f0e20640 1 Processor -- start 2023-11-06T02:14:53.545 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.550+0000 7fb9f0e20640 1 -- start start 2023-11-06T02:14:53.546 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.550+0000 7fb9f0e20640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb9ec1595a0 0x7fb9ec15dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:53.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.550+0000 7fb9f0e20640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9ec06c5a0 con 0x7fb9ec06d640 2023-11-06T02:14:53.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.550+0000 7fb9f0e20640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9ec06c880 con 0x7fb9ec15a990 2023-11-06T02:14:53.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.550+0000 7fb9f0e20640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9ec15e250 con 0x7fb9ec1595a0 2023-11-06T02:14:53.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.550+0000 7fb9ea59c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb9ec1595a0 0x7fb9ec15dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:53.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.550+0000 7fb9e9d9b640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fb9ec06d640 0x7fb9ec0b0220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:41974/0 (socket says 172.21.15.49:41974) 2023-11-06T02:14:53.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.550+0000 7fb9e9d9b640 1 -- 172.21.15.49:0/3114199284 learned_addr learned my addr 172.21.15.49:0/3114199284 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:14:53.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.550+0000 7fb9ea59c640 1 -- 172.21.15.49:0/3114199284 >> v1:172.21.15.150:6789/0 conn(0x7fb9ec15a990 legacy=0x7fb9ec0c8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:53.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.550+0000 7fb9ea59c640 1 -- 172.21.15.49:0/3114199284 >> v1:172.21.15.49:6789/0 conn(0x7fb9ec06d640 legacy=0x7fb9ec0b0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:53.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.550+0000 7fb9e959a640 1 -- 172.21.15.49:0/3114199284 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2541607006 0 0) 0x7fb9ec06c5a0 con 0x7fb9ec06d640 2023-11-06T02:14:53.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.550+0000 7fb9ea59c640 1 -- 172.21.15.49:0/3114199284 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9ec06cb60 con 0x7fb9ec1595a0 2023-11-06T02:14:53.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.554+0000 7fb9ea59c640 1 --2- 172.21.15.49:0/3114199284 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb9ec1595a0 0x7fb9ec15dd10 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7fb9e4009d90 tx=0x7fb9e401c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18614aded5701c1f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:53.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.554+0000 7fb9e959a640 1 -- 172.21.15.49:0/3114199284 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9e40a57f0 con 0x7fb9ec1595a0 2023-11-06T02:14:53.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.554+0000 7fb9e959a640 1 -- 172.21.15.49:0/3114199284 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9e40a5950 con 0x7fb9ec1595a0 2023-11-06T02:14:53.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.554+0000 7fb9e959a640 1 -- 172.21.15.49:0/3114199284 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9e40a5ce0 con 0x7fb9ec1595a0 2023-11-06T02:14:53.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.554+0000 7fb9f0e20640 1 -- 172.21.15.49:0/3114199284 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb9ec1595a0 msgr2=0x7fb9ec15dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:53.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.554+0000 7fb9f0e20640 1 --2- 172.21.15.49:0/3114199284 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb9ec1595a0 0x7fb9ec15dd10 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7fb9e4009d90 tx=0x7fb9e401c920 comp rx=0 tx=0).stop 2023-11-06T02:14:53.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.554+0000 7fb9f0e20640 1 -- 172.21.15.49:0/3114199284 shutdown_connections 2023-11-06T02:14:53.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.554+0000 7fb9f0e20640 1 --2- 172.21.15.49:0/3114199284 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb9ec1595a0 0x7fb9ec15dd10 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:53.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.554+0000 7fb9f0e20640 1 -- 172.21.15.49:0/3114199284 >> 172.21.15.49:0/3114199284 conn(0x7fb9ec06a370 msgr2=0x7fb9ec0ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:53.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.554+0000 7fb9f0e20640 1 -- 172.21.15.49:0/3114199284 shutdown_connections 2023-11-06T02:14:53.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.554+0000 7fb9f0e20640 1 -- 172.21.15.49:0/3114199284 wait complete. 2023-11-06T02:14:53.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.554+0000 7fb9f0e20640 1 Processor -- start 2023-11-06T02:14:53.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.554+0000 7fb9f0e20640 1 -- start start 2023-11-06T02:14:53.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.558+0000 7fb9f0e20640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb9ec06d640 0x7fb9ec14bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:53.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.558+0000 7fb9f0e20640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb9ec1595a0 0x7fb9ec1676c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:53.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.558+0000 7fb9f0e20640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb9ec15a990 0x7fb9ec0c7940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:53.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.558+0000 7fb9f0e20640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ec14cb80 con 0x7fb9ec06d640 2023-11-06T02:14:53.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.558+0000 7fb9f0e20640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ec1472c0 con 0x7fb9ec1595a0 2023-11-06T02:14:53.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.558+0000 7fb9f0e20640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ec14c5c0 con 0x7fb9ec15a990 2023-11-06T02:14:53.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.558+0000 7fb9ea59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb9ec06d640 0x7fb9ec14bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:53.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.558+0000 7fb9ea59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb9ec06d640 0x7fb9ec14bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44592/0 (socket says 172.21.15.49:44592) 2023-11-06T02:14:53.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.558+0000 7fb9ea59c640 1 -- 172.21.15.49:0/2837940897 learned_addr learned my addr 172.21.15.49:0/2837940897 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:53.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.558+0000 7fb9ead9d640 1 --2- 172.21.15.49:0/2837940897 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb9ec15a990 0x7fb9ec0c7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:53.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.558+0000 7fb9ea59c640 1 -- 172.21.15.49:0/2837940897 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb9ec15a990 msgr2=0x7fb9ec0c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:53.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.558+0000 7fb9ea59c640 1 --2- 172.21.15.49:0/2837940897 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb9ec15a990 0x7fb9ec0c7940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:53.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.558+0000 7fb9ea59c640 1 -- 172.21.15.49:0/2837940897 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb9ec1595a0 msgr2=0x7fb9ec1676c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:53.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.558+0000 7fb9e9d9b640 1 --2- 172.21.15.49:0/2837940897 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb9ec1595a0 0x7fb9ec1676c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:53.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.558+0000 7fb9ea59c640 1 --2- 172.21.15.49:0/2837940897 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb9ec1595a0 0x7fb9ec1676c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:53.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.558+0000 7fb9ea59c640 1 -- 172.21.15.49:0/2837940897 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9e4009a70 con 0x7fb9ec06d640 2023-11-06T02:14:53.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.558+0000 7fb9ead9d640 1 --2- 172.21.15.49:0/2837940897 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb9ec15a990 0x7fb9ec0c7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:14:53.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.558+0000 7fb9ea59c640 1 --2- 172.21.15.49:0/2837940897 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb9ec06d640 0x7fb9ec14bfc0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fb9e4019530 tx=0x7fb9e40196c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:53.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.558+0000 7fb9e2ffd640 1 -- 172.21.15.49:0/2837940897 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9e40ad850 con 0x7fb9ec06d640 2023-11-06T02:14:53.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.558+0000 7fb9e2ffd640 1 -- 172.21.15.49:0/2837940897 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9e40ad9b0 con 0x7fb9ec06d640 2023-11-06T02:14:53.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.558+0000 7fb9f0e20640 1 -- 172.21.15.49:0/2837940897 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9ec14c8a0 con 0x7fb9ec06d640 2023-11-06T02:14:53.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.558+0000 7fb9e2ffd640 1 -- 172.21.15.49:0/2837940897 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9e40add20 con 0x7fb9ec06d640 2023-11-06T02:14:53.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.558+0000 7fb9f0e20640 1 -- 172.21.15.49:0/2837940897 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9ec167c00 con 0x7fb9ec06d640 2023-11-06T02:14:53.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.558+0000 7fb9f0e20640 1 -- 172.21.15.49:0/2837940897 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9ec167e20 con 0x7fb9ec06d640 2023-11-06T02:14:53.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.558+0000 7fb9e2ffd640 1 -- 172.21.15.49:0/2837940897 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fb9e40b4070 con 0x7fb9ec06d640 2023-11-06T02:14:53.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.558+0000 7fb9e2ffd640 1 --2- 172.21.15.49:0/2837940897 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb9cc065a60 0x7fb9cc067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:53.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.558+0000 7fb9e2ffd640 1 -- 172.21.15.49:0/2837940897 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6320+0+0 (secure 0 0 0) 0x7fb9e411d870 con 0x7fb9ec06d640 2023-11-06T02:14:53.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.558+0000 7fb9e9d9b640 1 --2- 172.21.15.49:0/2837940897 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb9cc065a60 0x7fb9cc067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:53.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.562+0000 7fb9e9d9b640 1 --2- 172.21.15.49:0/2837940897 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb9cc065a60 0x7fb9cc067f20 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec1698a0 tx=0x7fb9d4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:53.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.562+0000 7fb9e2ffd640 1 -- 172.21.15.49:0/2837940897 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fb9ec167e20 con 0x7fb9ec06d640 2023-11-06T02:14:53.748 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:53.754+0000 7fb9f0e20640 1 -- 172.21.15.49:0/2837940897 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fb9ec147120 con 0x7fb9ec06d640 2023-11-06T02:14:54.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:54.414+0000 7fb9e2ffd640 1 -- 172.21.15.49:0/2837940897 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v61) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb9ec147120 con 0x7fb9ec06d640 2023-11-06T02:14:54.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:54.418+0000 7fb9f0e20640 1 -- 172.21.15.49:0/2837940897 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb9cc065a60 msgr2=0x7fb9cc067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:54.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:54.418+0000 7fb9f0e20640 1 --2- 172.21.15.49:0/2837940897 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb9cc065a60 0x7fb9cc067f20 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec1698a0 tx=0x7fb9d4009040 comp rx=0 tx=0).stop 2023-11-06T02:14:54.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:54.418+0000 7fb9f0e20640 1 -- 172.21.15.49:0/2837940897 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb9ec06d640 msgr2=0x7fb9ec14bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:54.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:54.418+0000 7fb9f0e20640 1 --2- 172.21.15.49:0/2837940897 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb9ec06d640 0x7fb9ec14bfc0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fb9e4019530 tx=0x7fb9e40196c0 comp rx=0 tx=0).stop 2023-11-06T02:14:54.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:54.418+0000 7fb9f0e20640 1 -- 172.21.15.49:0/2837940897 shutdown_connections 2023-11-06T02:14:54.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:54.418+0000 7fb9f0e20640 1 --2- 172.21.15.49:0/2837940897 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fb9cc065a60 0x7fb9cc067f20 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:54.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:54.418+0000 7fb9f0e20640 1 --2- 172.21.15.49:0/2837940897 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb9ec15a990 0x7fb9ec0c7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:54.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:54.418+0000 7fb9f0e20640 1 --2- 172.21.15.49:0/2837940897 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb9ec1595a0 0x7fb9ec1676c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:54.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:54.418+0000 7fb9f0e20640 1 --2- 172.21.15.49:0/2837940897 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb9ec06d640 0x7fb9ec14bfc0 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:54.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:54.418+0000 7fb9f0e20640 1 -- 172.21.15.49:0/2837940897 >> 172.21.15.49:0/2837940897 conn(0x7fb9ec06a370 msgr2=0x7fb9ec0b3580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:54.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:54.418+0000 7fb9f0e20640 1 -- 172.21.15.49:0/2837940897 shutdown_connections 2023-11-06T02:14:54.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:54.418+0000 7fb9f0e20640 1 -- 172.21.15.49:0/2837940897 wait complete. 2023-11-06T02:14:54.436 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-06T02:14:54.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:54.622+0000 7fd9a4b87640 1 Processor -- start 2023-11-06T02:14:54.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:54.622+0000 7fd9a4b87640 1 -- start start 2023-11-06T02:14:54.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:54.622+0000 7fd9a4b87640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fd9a006a480 0x7fd9a00c9940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:54.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:54.622+0000 7fd9a4b87640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd9a015e780 0x7fd9a00c9e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:54.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:54.622+0000 7fd9a4b87640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9a00c94b0 con 0x7fd9a015abd0 2023-11-06T02:14:54.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:54.622+0000 7fd9a4b87640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9a015ecd0 con 0x7fd9a015e780 2023-11-06T02:14:54.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:54.622+0000 7fd9a4b87640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9a015efb0 con 0x7fd9a006a480 2023-11-06T02:14:54.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:54.622+0000 7fd99ed9d640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd9a015e780 0x7fd9a00c9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:54.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:54.622+0000 7fd99e59c640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fd9a015abd0 0x7fd9a0069d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:41988/0 (socket says 172.21.15.49:41988) 2023-11-06T02:14:54.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:54.622+0000 7fd99ed9d640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd9a015e780 0x7fd9a00c9e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44598/0 (socket says 172.21.15.49:44598) 2023-11-06T02:14:54.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:54.622+0000 7fd99ed9d640 1 -- 172.21.15.49:0/200838182 learned_addr learned my addr 172.21.15.49:0/200838182 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:54.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:54.622+0000 7fd99dd9b640 1 --2- 172.21.15.49:0/200838182 >> v2:172.21.15.150:3300/0 conn(0x7fd9a006a480 0x7fd9a00c9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:54.621 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:54.622+0000 7fd99ed9d640 1 -- 172.21.15.49:0/200838182 >> v2:172.21.15.150:3300/0 conn(0x7fd9a006a480 msgr2=0x7fd9a00c9940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:54.621 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:54.622+0000 7fd99ed9d640 1 --2- 172.21.15.49:0/200838182 >> v2:172.21.15.150:3300/0 conn(0x7fd9a006a480 0x7fd9a00c9940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:54.621 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:54.622+0000 7fd99ed9d640 1 -- 172.21.15.49:0/200838182 >> v1:172.21.15.49:6789/0 conn(0x7fd9a015abd0 legacy=0x7fd9a0069d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:54.621 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:54.622+0000 7fd99d59a640 1 -- 172.21.15.49:0/200838182 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3131349947 0 0) 0x7fd9a00c94b0 con 0x7fd9a015abd0 2023-11-06T02:14:54.621 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:54.622+0000 7fd99ed9d640 1 -- 172.21.15.49:0/200838182 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9a015f290 con 0x7fd9a015e780 2023-11-06T02:14:54.623 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:54.626+0000 7fd99ed9d640 1 --2- 172.21.15.49:0/200838182 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9a015e780 0x7fd9a00c9e80 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7fd9a00c93d0 tx=0x7fd99001c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74b99ba1b84ca2ce server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:54.623 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:54.626+0000 7fd99d59a640 1 -- 172.21.15.49:0/200838182 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9900ab660 con 0x7fd9a015e780 2023-11-06T02:14:54.623 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:54.626+0000 7fd99d59a640 1 -- 172.21.15.49:0/200838182 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9900ab7c0 con 0x7fd9a015e780 2023-11-06T02:14:54.623 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:54.626+0000 7fd99d59a640 1 -- 172.21.15.49:0/200838182 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9900abb50 con 0x7fd9a015e780 2023-11-06T02:14:54.624 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:54.626+0000 7fd9a4b87640 1 -- 172.21.15.49:0/200838182 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9a015e780 msgr2=0x7fd9a00c9e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:54.624 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:54.626+0000 7fd9a4b87640 1 --2- 172.21.15.49:0/200838182 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9a015e780 0x7fd9a00c9e80 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7fd9a00c93d0 tx=0x7fd99001c440 comp rx=0 tx=0).stop 2023-11-06T02:14:54.624 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:54.626+0000 7fd9a4b87640 1 -- 172.21.15.49:0/200838182 shutdown_connections 2023-11-06T02:14:54.624 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:54.626+0000 7fd9a4b87640 1 --2- 172.21.15.49:0/200838182 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9a015e780 0x7fd9a00c9e80 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:54.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:54.626+0000 7fd9a4b87640 1 --2- 172.21.15.49:0/200838182 >> v2:172.21.15.150:3300/0 conn(0x7fd9a006a480 0x7fd9a00c9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:54.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:54.626+0000 7fd9a4b87640 1 -- 172.21.15.49:0/200838182 >> 172.21.15.49:0/200838182 conn(0x7fd9a00ce610 msgr2=0x7fd9a00cea20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:54.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:54.626+0000 7fd9a4b87640 1 -- 172.21.15.49:0/200838182 shutdown_connections 2023-11-06T02:14:54.626 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:54.626+0000 7fd9a4b87640 1 -- 172.21.15.49:0/200838182 wait complete. 2023-11-06T02:14:54.631 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:54.630+0000 7fd9a4b87640 1 Processor -- start 2023-11-06T02:14:54.631 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:54.630+0000 7fd9a4b87640 1 -- start start 2023-11-06T02:14:54.632 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:54.630+0000 7fd9a4b87640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9a006a480 0x7fd9a01f9350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:54.632 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:54.630+0000 7fd9a4b87640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9a015abd0 0x7fd9a00b00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:54.632 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:54.630+0000 7fd9a4b87640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd9a015e780 0x7fd9a00b0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:54.633 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:54.630+0000 7fd9a4b87640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a01fdea0 con 0x7fd9a006a480 2023-11-06T02:14:54.633 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:54.630+0000 7fd9a4b87640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a01fdbc0 con 0x7fd9a015abd0 2023-11-06T02:14:54.633 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:54.630+0000 7fd9a4b87640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a0151da0 con 0x7fd9a015e780 2023-11-06T02:14:54.634 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:54.630+0000 7fd99e59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9a006a480 0x7fd9a01f9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:54.634 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:54.630+0000 7fd99e59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9a006a480 0x7fd9a01f9350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44600/0 (socket says 172.21.15.49:44600) 2023-11-06T02:14:54.634 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:54.630+0000 7fd99e59c640 1 -- 172.21.15.49:0/1520341287 learned_addr learned my addr 172.21.15.49:0/1520341287 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:54.634 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:54.630+0000 7fd99ed9d640 1 --2- 172.21.15.49:0/1520341287 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd9a015e780 0x7fd9a00b0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:54.635 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:54.630+0000 7fd99e59c640 1 -- 172.21.15.49:0/1520341287 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd9a015e780 msgr2=0x7fd9a00b0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:54.635 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:54.630+0000 7fd99dd9b640 1 --2- 172.21.15.49:0/1520341287 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9a015abd0 0x7fd9a00b00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:54.635 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:54.630+0000 7fd99e59c640 1 --2- 172.21.15.49:0/1520341287 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd9a015e780 0x7fd9a00b0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:54.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:54.630+0000 7fd99e59c640 1 -- 172.21.15.49:0/1520341287 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9a015abd0 msgr2=0x7fd9a00b00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:54.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:54.630+0000 7fd99e59c640 1 --2- 172.21.15.49:0/1520341287 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9a015abd0 0x7fd9a00b00f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:54.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:54.630+0000 7fd99e59c640 1 -- 172.21.15.49:0/1520341287 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9900094d0 con 0x7fd9a006a480 2023-11-06T02:14:54.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:54.630+0000 7fd99e59c640 1 --2- 172.21.15.49:0/1520341287 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9a006a480 0x7fd9a01f9350 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fd998001570 tx=0x7fd99800cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:54.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:54.630+0000 7fd98effd640 1 -- 172.21.15.49:0/1520341287 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd99800a640 con 0x7fd9a006a480 2023-11-06T02:14:54.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:54.630+0000 7fd98effd640 1 -- 172.21.15.49:0/1520341287 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd99800a7c0 con 0x7fd9a006a480 2023-11-06T02:14:54.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:54.630+0000 7fd98effd640 1 -- 172.21.15.49:0/1520341287 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd99800ab70 con 0x7fd9a006a480 2023-11-06T02:14:54.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:54.630+0000 7fd9a4b87640 1 -- 172.21.15.49:0/1520341287 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9a0152080 con 0x7fd9a006a480 2023-11-06T02:14:54.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:54.630+0000 7fd9a4b87640 1 -- 172.21.15.49:0/1520341287 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9a00c44f0 con 0x7fd9a006a480 2023-11-06T02:14:54.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:54.630+0000 7fd9a4b87640 1 -- 172.21.15.49:0/1520341287 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9a0151c00 con 0x7fd9a006a480 2023-11-06T02:14:54.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:54.634+0000 7fd98effd640 1 -- 172.21.15.49:0/1520341287 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fd99800acd0 con 0x7fd9a006a480 2023-11-06T02:14:54.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:54.634+0000 7fd98effd640 1 --2- 172.21.15.49:0/1520341287 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd980065a60 0x7fd980067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:54.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:54.634+0000 7fd98effd640 1 -- 172.21.15.49:0/1520341287 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6320+0+0 (secure 0 0 0) 0x7fd9980828a0 con 0x7fd9a006a480 2023-11-06T02:14:54.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:54.634+0000 7fd99dd9b640 1 --2- 172.21.15.49:0/1520341287 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd980065a60 0x7fd980067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:54.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:54.634+0000 7fd99dd9b640 1 --2- 172.21.15.49:0/1520341287 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd980065a60 0x7fd980067f20 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fd9a0153630 tx=0x7fd994008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:54.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:54.638+0000 7fd98effd640 1 -- 172.21.15.49:0/1520341287 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fd9a0151c00 con 0x7fd9a006a480 2023-11-06T02:14:54.782 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:54.786+0000 7fd9a4b87640 1 -- 172.21.15.49:0/1520341287 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd9a0151c00 con 0x7fd9a006a480 2023-11-06T02:14:55.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:55.498+0000 7fd98effd640 1 -- 172.21.15.49:0/1520341287 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v84) v1 ==== 181+0+0 (secure 0 0 0) 0x7fd9a0151c00 con 0x7fd9a006a480 2023-11-06T02:14:55.495 INFO:teuthology.orchestra.run.smithi049.stderr:pool 'cephfs_metadata' removed 2023-11-06T02:14:55.498 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:55.502+0000 7fd9a4b87640 1 -- 172.21.15.49:0/1520341287 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd980065a60 msgr2=0x7fd980067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:55.498 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:55.502+0000 7fd9a4b87640 1 --2- 172.21.15.49:0/1520341287 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd980065a60 0x7fd980067f20 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fd9a0153630 tx=0x7fd994008040 comp rx=0 tx=0).stop 2023-11-06T02:14:55.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:55.502+0000 7fd9a4b87640 1 -- 172.21.15.49:0/1520341287 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9a006a480 msgr2=0x7fd9a01f9350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:55.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:55.502+0000 7fd9a4b87640 1 --2- 172.21.15.49:0/1520341287 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9a006a480 0x7fd9a01f9350 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fd998001570 tx=0x7fd99800cd20 comp rx=0 tx=0).stop 2023-11-06T02:14:55.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:55.506+0000 7fd9a4b87640 1 -- 172.21.15.49:0/1520341287 shutdown_connections 2023-11-06T02:14:55.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:55.506+0000 7fd9a4b87640 1 --2- 172.21.15.49:0/1520341287 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd980065a60 0x7fd980067f20 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:55.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:55.506+0000 7fd9a4b87640 1 --2- 172.21.15.49:0/1520341287 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd9a015e780 0x7fd9a00b0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:55.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:55.506+0000 7fd9a4b87640 1 --2- 172.21.15.49:0/1520341287 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9a015abd0 0x7fd9a00b00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:55.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:55.506+0000 7fd9a4b87640 1 --2- 172.21.15.49:0/1520341287 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9a006a480 0x7fd9a01f9350 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:55.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:55.506+0000 7fd9a4b87640 1 -- 172.21.15.49:0/1520341287 >> 172.21.15.49:0/1520341287 conn(0x7fd9a00ce610 msgr2=0x7fd9a00cb090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:55.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:55.506+0000 7fd9a4b87640 1 -- 172.21.15.49:0/1520341287 shutdown_connections 2023-11-06T02:14:55.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:55.506+0000 7fd9a4b87640 1 -- 172.21.15.49:0/1520341287 wait complete. 2023-11-06T02:14:55.521 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-06T02:14:55.657 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:55.662+0000 7f43de912640 1 Processor -- start 2023-11-06T02:14:55.657 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:55.662+0000 7f43de912640 1 -- start start 2023-11-06T02:14:55.658 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:55.662+0000 7f43de912640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f43d815abd0 0x7f43d8069d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:55.658 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:55.662+0000 7f43de912640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f43d80ca950 0x7f43d806a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:55.659 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:55.662+0000 7f43de912640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43d80c94b0 con 0x7f43d815e780 2023-11-06T02:14:55.659 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:55.662+0000 7f43de912640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43d815ecd0 con 0x7f43d815abd0 2023-11-06T02:14:55.659 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:55.662+0000 7f43de912640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43d815efb0 con 0x7f43d80ca950 2023-11-06T02:14:55.659 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:55.662+0000 7f43d7fff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f43d815abd0 0x7f43d8069d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:55.659 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:55.662+0000 7f43d77fe640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f43d815e780 0x7f43d80b0090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:41992/0 (socket says 172.21.15.49:41992) 2023-11-06T02:14:55.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:55.662+0000 7f43d77fe640 1 -- 172.21.15.49:0/935618120 learned_addr learned my addr 172.21.15.49:0/935618120 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:14:55.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:55.662+0000 7f43dceaf640 1 --2- 172.21.15.49:0/935618120 >> v2:172.21.15.150:3300/0 conn(0x7f43d80ca950 0x7f43d806a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:55.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:55.662+0000 7f43d7fff640 1 -- 172.21.15.49:0/935618120 >> v2:172.21.15.150:3300/0 conn(0x7f43d80ca950 msgr2=0x7f43d806a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:55.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:55.662+0000 7f43d7fff640 1 --2- 172.21.15.49:0/935618120 >> v2:172.21.15.150:3300/0 conn(0x7f43d80ca950 0x7f43d806a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:55.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:55.662+0000 7f43d7fff640 1 -- 172.21.15.49:0/935618120 >> v1:172.21.15.49:6789/0 conn(0x7f43d815e780 legacy=0x7f43d80b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:55.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:55.662+0000 7f43d7fff640 1 -- 172.21.15.49:0/935618120 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43d815f290 con 0x7f43d815abd0 2023-11-06T02:14:55.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:55.662+0000 7f43dceaf640 1 --2- 172.21.15.49:0/935618120 >> v2:172.21.15.150:3300/0 conn(0x7f43d80ca950 0x7f43d806a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:14:55.662 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:55.666+0000 7f43d7fff640 1 --2- 172.21.15.49:0/935618120 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f43d815abd0 0x7f43d8069d70 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f43d80c93d0 tx=0x7f43c801c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3005ac6648dcfdb0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:55.662 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:55.666+0000 7f43d6ffd640 1 -- 172.21.15.49:0/935618120 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c80ac620 con 0x7f43d815abd0 2023-11-06T02:14:55.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:55.666+0000 7f43d6ffd640 1 -- 172.21.15.49:0/935618120 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43c80ac780 con 0x7f43d815abd0 2023-11-06T02:14:55.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:55.666+0000 7f43d6ffd640 1 -- 172.21.15.49:0/935618120 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c80acb10 con 0x7f43d815abd0 2023-11-06T02:14:55.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:55.666+0000 7f43de912640 1 -- 172.21.15.49:0/935618120 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f43d815abd0 msgr2=0x7f43d8069d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:55.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:55.666+0000 7f43de912640 1 --2- 172.21.15.49:0/935618120 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f43d815abd0 0x7f43d8069d70 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f43d80c93d0 tx=0x7f43c801c3a0 comp rx=0 tx=0).stop 2023-11-06T02:14:55.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:55.666+0000 7f43de912640 1 -- 172.21.15.49:0/935618120 shutdown_connections 2023-11-06T02:14:55.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:55.666+0000 7f43de912640 1 --2- 172.21.15.49:0/935618120 >> v2:172.21.15.150:3300/0 conn(0x7f43d80ca950 0x7f43d806a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:55.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:55.666+0000 7f43de912640 1 --2- 172.21.15.49:0/935618120 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f43d815abd0 0x7f43d8069d70 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:55.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:55.666+0000 7f43de912640 1 -- 172.21.15.49:0/935618120 >> 172.21.15.49:0/935618120 conn(0x7f43d80ce610 msgr2=0x7f43d80cea20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:55.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:55.666+0000 7f43de912640 1 -- 172.21.15.49:0/935618120 shutdown_connections 2023-11-06T02:14:55.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:55.666+0000 7f43de912640 1 -- 172.21.15.49:0/935618120 wait complete. 2023-11-06T02:14:55.669 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:55.670+0000 7f43de912640 1 Processor -- start 2023-11-06T02:14:55.669 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:55.670+0000 7f43de912640 1 -- start start 2023-11-06T02:14:55.669 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:55.670+0000 7f43de912640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f43d80ca950 0x7f43d80cd370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:55.669 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:55.670+0000 7f43de912640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f43d815abd0 0x7f43d8153120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:55.670 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:55.670+0000 7f43de912640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43d815e780 0x7f43d814dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:55.670 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:55.670+0000 7f43de912640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43d814e280 con 0x7f43d80ca950 2023-11-06T02:14:55.670 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:55.670+0000 7f43de912640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43d8152960 con 0x7f43d815e780 2023-11-06T02:14:55.670 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:55.670+0000 7f43de912640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43d80cda00 con 0x7f43d815abd0 2023-11-06T02:14:55.671 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:55.670+0000 7f43d7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f43d80ca950 0x7f43d80cd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:55.671 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:55.670+0000 7f43d7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f43d80ca950 0x7f43d80cd370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44628/0 (socket says 172.21.15.49:44628) 2023-11-06T02:14:55.671 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:55.670+0000 7f43d7fff640 1 -- 172.21.15.49:0/308155209 learned_addr learned my addr 172.21.15.49:0/308155209 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:55.671 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:55.670+0000 7f43d77fe640 1 --2- 172.21.15.49:0/308155209 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f43d815abd0 0x7f43d8153120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:55.671 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:55.670+0000 7f43dceaf640 1 --2- 172.21.15.49:0/308155209 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43d815e780 0x7f43d814dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:55.672 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:55.670+0000 7f43d7fff640 1 -- 172.21.15.49:0/308155209 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f43d815abd0 msgr2=0x7f43d8153120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:55.672 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:55.670+0000 7f43d7fff640 1 --2- 172.21.15.49:0/308155209 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f43d815abd0 0x7f43d8153120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:55.672 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:55.670+0000 7f43d7fff640 1 -- 172.21.15.49:0/308155209 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43d815e780 msgr2=0x7f43d814dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:55.672 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:55.670+0000 7f43d7fff640 1 --2- 172.21.15.49:0/308155209 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43d815e780 0x7f43d814dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:55.672 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:55.670+0000 7f43d7fff640 1 -- 172.21.15.49:0/308155209 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43c8009420 con 0x7f43d80ca950 2023-11-06T02:14:55.673 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:55.670+0000 7f43d77fe640 1 --2- 172.21.15.49:0/308155209 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f43d815abd0 0x7f43d8153120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:14:55.673 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:55.670+0000 7f43dceaf640 1 --2- 172.21.15.49:0/308155209 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43d815e780 0x7f43d814dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:14:55.673 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:55.670+0000 7f43d7fff640 1 --2- 172.21.15.49:0/308155209 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f43d80ca950 0x7f43d80cd370 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f43c8000c10 tx=0x7f43c8019140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:55.673 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:55.670+0000 7f43d4ff9640 1 -- 172.21.15.49:0/308155209 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c80b5600 con 0x7f43d80ca950 2023-11-06T02:14:55.673 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:55.670+0000 7f43d4ff9640 1 -- 172.21.15.49:0/308155209 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43c80b5760 con 0x7f43d80ca950 2023-11-06T02:14:55.674 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:55.670+0000 7f43de912640 1 -- 172.21.15.49:0/308155209 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43d80cdce0 con 0x7f43d80ca950 2023-11-06T02:14:55.674 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:55.670+0000 7f43d4ff9640 1 -- 172.21.15.49:0/308155209 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c80b5b10 con 0x7f43d80ca950 2023-11-06T02:14:55.674 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:55.670+0000 7f43de912640 1 -- 172.21.15.49:0/308155209 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43d815a200 con 0x7f43d80ca950 2023-11-06T02:14:55.674 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:55.670+0000 7f43de912640 1 -- 172.21.15.49:0/308155209 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43d8074040 con 0x7f43d80ca950 2023-11-06T02:14:55.675 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:55.670+0000 7f43d4ff9640 1 -- 172.21.15.49:0/308155209 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f43c80b3040 con 0x7f43d80ca950 2023-11-06T02:14:55.675 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:55.674+0000 7f43d4ff9640 1 --2- 172.21.15.49:0/308155209 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f43c4065a60 0x7f43c4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:55.675 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:55.674+0000 7f43d4ff9640 1 -- 172.21.15.49:0/308155209 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5882+0+0 (secure 0 0 0) 0x7f43c811d340 con 0x7f43d80ca950 2023-11-06T02:14:55.675 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:55.674+0000 7f43d77fe640 1 --2- 172.21.15.49:0/308155209 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f43c4065a60 0x7f43c4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:55.676 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:55.674+0000 7f43d77fe640 1 --2- 172.21.15.49:0/308155209 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f43c4065a60 0x7f43c4067f20 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f43cc007fd0 tx=0x7f43cc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:55.676 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:55.678+0000 7f43d4ff9640 1 -- 172.21.15.49:0/308155209 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f43c80eac50 con 0x7f43d80ca950 2023-11-06T02:14:55.826 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:55.830+0000 7f43de912640 1 -- 172.21.15.49:0/308155209 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f43d80c8d80 con 0x7f43d80ca950 2023-11-06T02:14:56.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.510+0000 7f43d4ff9640 1 -- 172.21.15.49:0/308155209 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v85) v1 ==== 169+0+0 (secure 0 0 0) 0x7f43c80ee270 con 0x7f43d80ca950 2023-11-06T02:14:56.504 INFO:teuthology.orchestra.run.smithi049.stderr:pool 'cephfs_data' removed 2023-11-06T02:14:56.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.514+0000 7f43de912640 1 -- 172.21.15.49:0/308155209 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f43c4065a60 msgr2=0x7f43c4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:56.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.514+0000 7f43de912640 1 --2- 172.21.15.49:0/308155209 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f43c4065a60 0x7f43c4067f20 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f43cc007fd0 tx=0x7f43cc009040 comp rx=0 tx=0).stop 2023-11-06T02:14:56.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.514+0000 7f43de912640 1 -- 172.21.15.49:0/308155209 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f43d80ca950 msgr2=0x7f43d80cd370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:56.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.514+0000 7f43de912640 1 --2- 172.21.15.49:0/308155209 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f43d80ca950 0x7f43d80cd370 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f43c8000c10 tx=0x7f43c8019140 comp rx=0 tx=0).stop 2023-11-06T02:14:56.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.514+0000 7f43de912640 1 -- 172.21.15.49:0/308155209 shutdown_connections 2023-11-06T02:14:56.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.514+0000 7f43de912640 1 --2- 172.21.15.49:0/308155209 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f43c4065a60 0x7f43c4067f20 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:56.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.514+0000 7f43de912640 1 --2- 172.21.15.49:0/308155209 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43d815e780 0x7f43d814dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:56.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.514+0000 7f43de912640 1 --2- 172.21.15.49:0/308155209 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f43d815abd0 0x7f43d8153120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:56.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.514+0000 7f43de912640 1 --2- 172.21.15.49:0/308155209 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f43d80ca950 0x7f43d80cd370 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:56.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.514+0000 7f43de912640 1 -- 172.21.15.49:0/308155209 >> 172.21.15.49:0/308155209 conn(0x7f43d80ce610 msgr2=0x7f43d80cb050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:56.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.514+0000 7f43de912640 1 -- 172.21.15.49:0/308155209 shutdown_connections 2023-11-06T02:14:56.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.514+0000 7f43de912640 1 -- 172.21.15.49:0/308155209 wait complete. 2023-11-06T02:14:56.563 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-06T02:14:56.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.694+0000 7fe425b2c640 1 Processor -- start 2023-11-06T02:14:56.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.694+0000 7fe425b2c640 1 -- start start 2023-11-06T02:14:56.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.694+0000 7fe425b2c640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fe4200caec0 0x7fe4200cb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:56.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.694+0000 7fe425b2c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe42015dd10 0x7fe4200cd190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:56.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.694+0000 7fe425b2c640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe42015ad50 con 0x7fe4200c9600 2023-11-06T02:14:56.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.694+0000 7fe425b2c640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe42015e280 con 0x7fe4200caec0 2023-11-06T02:14:56.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.694+0000 7fe425b2c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4200cecf0 con 0x7fe42015dd10 2023-11-06T02:14:56.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.698+0000 7fe416ffd640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fe4200c9600 0x7fe420069d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:41998/0 (socket says 172.21.15.49:41998) 2023-11-06T02:14:56.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.698+0000 7fe416ffd640 1 -- 172.21.15.49:0/2323170536 learned_addr learned my addr 172.21.15.49:0/2323170536 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:14:56.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.698+0000 7fe41f7fe640 1 --2- 172.21.15.49:0/2323170536 >> v2:172.21.15.150:3300/0 conn(0x7fe4200caec0 0x7fe4200cb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:56.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.698+0000 7fe41ffff640 1 --2- 172.21.15.49:0/2323170536 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe42015dd10 0x7fe4200cd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:56.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.698+0000 7fe41effd640 1 -- 172.21.15.49:0/2323170536 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 778607528 0 0) 0x7fe410001120 con 0x7fe4200c9600 2023-11-06T02:14:56.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.698+0000 7fe41effd640 1 -- 172.21.15.49:0/2323170536 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe400003530 con 0x7fe4200c9600 2023-11-06T02:14:56.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.698+0000 7fe41effd640 1 -- 172.21.15.49:0/2323170536 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 297341249 0 0) 0x7fe400003530 con 0x7fe4200c9600 2023-11-06T02:14:56.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.698+0000 7fe41effd640 1 -- 172.21.15.49:0/2323170536 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe42015dd10 msgr2=0x7fe4200cd190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:56.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.698+0000 7fe41effd640 1 --2- 172.21.15.49:0/2323170536 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe42015dd10 0x7fe4200cd190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:56.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.698+0000 7fe41effd640 1 -- 172.21.15.49:0/2323170536 >> v2:172.21.15.150:3300/0 conn(0x7fe4200caec0 msgr2=0x7fe4200cb2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:56.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.698+0000 7fe41effd640 1 --2- 172.21.15.49:0/2323170536 >> v2:172.21.15.150:3300/0 conn(0x7fe4200caec0 0x7fe4200cb2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:56.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.698+0000 7fe41effd640 1 -- 172.21.15.49:0/2323170536 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4200cb7e0 con 0x7fe4200c9600 2023-11-06T02:14:56.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.698+0000 7fe41effd640 1 -- 172.21.15.49:0/2323170536 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7fe410001810 con 0x7fe4200c9600 2023-11-06T02:14:56.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.698+0000 7fe41effd640 1 -- 172.21.15.49:0/2323170536 >> v1:172.21.15.49:6789/0 conn(0x7fe4200c9600 legacy=0x7fe420069d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:56.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.698+0000 7fe41effd640 1 --2- 172.21.15.49:0/2323170536 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe400003980 0x7fe400003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:56.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.698+0000 7fe41effd640 1 --2- 172.21.15.49:0/2323170536 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe42015dd10 0x7fe4000047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:56.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.698+0000 7fe41effd640 1 --2- 172.21.15.49:0/2323170536 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe400004d10 0x7fe400005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:56.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.698+0000 7fe41effd640 1 -- 172.21.15.49:0/2323170536 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe42015e280 con 0x7fe400004d10 2023-11-06T02:14:56.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.698+0000 7fe41effd640 1 -- 172.21.15.49:0/2323170536 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4200cecf0 con 0x7fe400003980 2023-11-06T02:14:56.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.698+0000 7fe41effd640 1 -- 172.21.15.49:0/2323170536 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe40000b990 con 0x7fe42015dd10 2023-11-06T02:14:56.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.698+0000 7fe41ffff640 1 --2- 172.21.15.49:0/2323170536 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe400004d10 0x7fe400005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:56.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.698+0000 7fe41f7fe640 1 --2- 172.21.15.49:0/2323170536 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe400003980 0x7fe400003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:56.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.698+0000 7fe41ffff640 1 -- 172.21.15.49:0/2323170536 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe42015dd10 msgr2=0x7fe4000047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:56.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.698+0000 7fe41ffff640 1 --2- 172.21.15.49:0/2323170536 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe42015dd10 0x7fe4000047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:56.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.698+0000 7fe41ffff640 1 -- 172.21.15.49:0/2323170536 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe400003980 msgr2=0x7fe400003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:56.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.698+0000 7fe41ffff640 1 --2- 172.21.15.49:0/2323170536 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe400003980 0x7fe400003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:56.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.698+0000 7fe41ffff640 1 -- 172.21.15.49:0/2323170536 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe400003530 con 0x7fe400004d10 2023-11-06T02:14:56.702 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.698+0000 7fe41ffff640 1 --2- 172.21.15.49:0/2323170536 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe400004d10 0x7fe400005190 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fe418009fc0 tx=0x7fe41801cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bc4418894447b26 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:56.702 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.698+0000 7fe41effd640 1 -- 172.21.15.49:0/2323170536 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4180a5cb0 con 0x7fe400004d10 2023-11-06T02:14:56.702 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.698+0000 7fe41effd640 1 -- 172.21.15.49:0/2323170536 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe41800f040 con 0x7fe400004d10 2023-11-06T02:14:56.702 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.698+0000 7fe41effd640 1 -- 172.21.15.49:0/2323170536 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4180ad680 con 0x7fe400004d10 2023-11-06T02:14:56.702 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.702+0000 7fe425b2c640 1 -- 172.21.15.49:0/2323170536 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe400004d10 msgr2=0x7fe400005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:56.703 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.702+0000 7fe425b2c640 1 --2- 172.21.15.49:0/2323170536 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe400004d10 0x7fe400005190 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fe418009fc0 tx=0x7fe41801cca0 comp rx=0 tx=0).stop 2023-11-06T02:14:56.703 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.702+0000 7fe41ffff640 1 -- 172.21.15.49:0/2323170536 reap_dead start 2023-11-06T02:14:56.703 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.702+0000 7fe425b2c640 1 -- 172.21.15.49:0/2323170536 shutdown_connections 2023-11-06T02:14:56.703 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.702+0000 7fe425b2c640 1 -- 172.21.15.49:0/2323170536 >> 172.21.15.49:0/2323170536 conn(0x7fe4200b0aa0 msgr2=0x7fe4200b0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:56.703 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.702+0000 7fe425b2c640 1 -- 172.21.15.49:0/2323170536 shutdown_connections 2023-11-06T02:14:56.704 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.702+0000 7fe425b2c640 1 -- 172.21.15.49:0/2323170536 wait complete. 2023-11-06T02:14:56.704 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.702+0000 7fe425b2c640 1 Processor -- start 2023-11-06T02:14:56.704 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.702+0000 7fe425b2c640 1 -- start start 2023-11-06T02:14:56.704 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.702+0000 7fe425b2c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe4201561e0 0x7fe42009f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:56.704 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.702+0000 7fe425b2c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe420156b00 0x7fe4200b2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:56.705 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.702+0000 7fe425b2c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4200b2ff0 0x7fe420158de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:56.705 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.702+0000 7fe425b2c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe42015d460 con 0x7fe4200b2ff0 2023-11-06T02:14:56.705 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.702+0000 7fe425b2c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe42015cc90 con 0x7fe420156b00 2023-11-06T02:14:56.705 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.702+0000 7fe425b2c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe42015cea0 con 0x7fe4201561e0 2023-11-06T02:14:56.705 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.702+0000 7fe41ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4200b2ff0 0x7fe420158de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:56.706 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.702+0000 7fe41ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4200b2ff0 0x7fe420158de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44654/0 (socket says 172.21.15.49:44654) 2023-11-06T02:14:56.706 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.702+0000 7fe41ffff640 1 -- 172.21.15.49:0/121610928 learned_addr learned my addr 172.21.15.49:0/121610928 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:56.706 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.702+0000 7fe41f7fe640 1 --2- 172.21.15.49:0/121610928 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe4201561e0 0x7fe42009f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:56.706 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.702+0000 7fe416ffd640 1 --2- 172.21.15.49:0/121610928 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe420156b00 0x7fe4200b2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:56.706 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.702+0000 7fe41ffff640 1 -- 172.21.15.49:0/121610928 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe4201561e0 msgr2=0x7fe42009f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:56.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.702+0000 7fe41ffff640 1 --2- 172.21.15.49:0/121610928 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe4201561e0 0x7fe42009f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:56.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.702+0000 7fe41ffff640 1 -- 172.21.15.49:0/121610928 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe420156b00 msgr2=0x7fe4200b2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:56.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.702+0000 7fe41ffff640 1 --2- 172.21.15.49:0/121610928 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe420156b00 0x7fe4200b2ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:56.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.702+0000 7fe41ffff640 1 -- 172.21.15.49:0/121610928 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe42015d180 con 0x7fe4200b2ff0 2023-11-06T02:14:56.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.702+0000 7fe41f7fe640 1 --2- 172.21.15.49:0/121610928 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe4201561e0 0x7fe42009f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:14:56.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.702+0000 7fe416ffd640 1 --2- 172.21.15.49:0/121610928 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe420156b00 0x7fe4200b2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:14:56.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.702+0000 7fe41ffff640 1 --2- 172.21.15.49:0/121610928 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4200b2ff0 0x7fe420158de0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fe418002240 tx=0x7fe418009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:56.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.702+0000 7fe41cff9640 1 -- 172.21.15.49:0/121610928 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4180039a0 con 0x7fe4200b2ff0 2023-11-06T02:14:56.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.702+0000 7fe41cff9640 1 -- 172.21.15.49:0/121610928 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe418003b00 con 0x7fe4200b2ff0 2023-11-06T02:14:56.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.702+0000 7fe425b2c640 1 -- 172.21.15.49:0/121610928 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe42009d4b0 con 0x7fe4200b2ff0 2023-11-06T02:14:56.709 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.702+0000 7fe41cff9640 1 -- 172.21.15.49:0/121610928 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe418003e70 con 0x7fe4200b2ff0 2023-11-06T02:14:56.709 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.702+0000 7fe425b2c640 1 -- 172.21.15.49:0/121610928 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe42009fde0 con 0x7fe4200b2ff0 2023-11-06T02:14:56.709 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.706+0000 7fe425b2c640 1 -- 172.21.15.49:0/121610928 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe42009d310 con 0x7fe4200b2ff0 2023-11-06T02:14:56.709 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.706+0000 7fe41cff9640 1 -- 172.21.15.49:0/121610928 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fe4180b4040 con 0x7fe4200b2ff0 2023-11-06T02:14:56.709 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.706+0000 7fe41cff9640 1 --2- 172.21.15.49:0/121610928 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe404065a90 0x7fe404067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:56.710 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.706+0000 7fe41cff9640 1 -- 172.21.15.49:0/121610928 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fe41811cd90 con 0x7fe4200b2ff0 2023-11-06T02:14:56.710 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.706+0000 7fe41f7fe640 1 --2- 172.21.15.49:0/121610928 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe404065a90 0x7fe404067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:56.710 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.706+0000 7fe41f7fe640 1 --2- 172.21.15.49:0/121610928 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe404065a90 0x7fe404067f50 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fe408009aa0 tx=0x7fe408008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:56.711 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.710+0000 7fe41cff9640 1 -- 172.21.15.49:0/121610928 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fe42009d310 con 0x7fe4200b2ff0 2023-11-06T02:14:56.860 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.866+0000 7fe425b2c640 1 -- 172.21.15.49:0/121610928 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fe42009d310 con 0x7fe4200b2ff0 2023-11-06T02:14:56.866 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.870+0000 7fe41cff9640 1 -- 172.21.15.49:0/121610928 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe42009d310 con 0x7fe4200b2ff0 2023-11-06T02:14:56.869 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.874+0000 7fe425b2c640 1 -- 172.21.15.49:0/121610928 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe404065a90 msgr2=0x7fe404067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:56.869 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.874+0000 7fe425b2c640 1 --2- 172.21.15.49:0/121610928 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe404065a90 0x7fe404067f50 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fe408009aa0 tx=0x7fe408008040 comp rx=0 tx=0).stop 2023-11-06T02:14:56.869 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.874+0000 7fe425b2c640 1 -- 172.21.15.49:0/121610928 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4200b2ff0 msgr2=0x7fe420158de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:56.870 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.874+0000 7fe425b2c640 1 --2- 172.21.15.49:0/121610928 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4200b2ff0 0x7fe420158de0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fe418002240 tx=0x7fe418009750 comp rx=0 tx=0).stop 2023-11-06T02:14:56.870 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.874+0000 7fe425b2c640 1 -- 172.21.15.49:0/121610928 shutdown_connections 2023-11-06T02:14:56.870 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.874+0000 7fe425b2c640 1 --2- 172.21.15.49:0/121610928 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fe404065a90 0x7fe404067f50 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:56.870 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.874+0000 7fe425b2c640 1 --2- 172.21.15.49:0/121610928 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4200b2ff0 0x7fe420158de0 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:56.870 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.874+0000 7fe425b2c640 1 --2- 172.21.15.49:0/121610928 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe420156b00 0x7fe4200b2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:56.871 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.874+0000 7fe425b2c640 1 --2- 172.21.15.49:0/121610928 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe4201561e0 0x7fe42009f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:56.871 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.874+0000 7fe425b2c640 1 -- 172.21.15.49:0/121610928 >> 172.21.15.49:0/121610928 conn(0x7fe4200b0aa0 msgr2=0x7fe4200ae9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:56.871 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.874+0000 7fe425b2c640 1 -- 172.21.15.49:0/121610928 shutdown_connections 2023-11-06T02:14:56.871 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:56.874+0000 7fe425b2c640 1 -- 172.21.15.49:0/121610928 wait complete. 2023-11-06T02:14:56.887 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan' 2023-11-06T02:14:57.014 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:57.018+0000 7f76adfce640 1 Processor -- start 2023-11-06T02:14:57.015 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:57.018+0000 7f76adfce640 1 -- start start 2023-11-06T02:14:57.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:57.022+0000 7f76adfce640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f76a80b1f90 0x7f76a80cda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:57.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:57.022+0000 7f76adfce640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f76a8161650 0x7f76a80d1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:57.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:57.022+0000 7f76adfce640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76a80d17c0 con 0x7f76a80d2190 2023-11-06T02:14:57.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:57.022+0000 7f76adfce640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76a80ce6c0 con 0x7f76a8161650 2023-11-06T02:14:57.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:57.022+0000 7f76adfce640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76a8161ad0 con 0x7f76a80b1f90 2023-11-06T02:14:57.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:57.022+0000 7f769ffff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f76a8161650 0x7f76a80d1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:57.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:57.022+0000 7f76a7fff640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f76a80d2190 0x7f76a80714a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:42012/0 (socket says 172.21.15.49:42012) 2023-11-06T02:14:57.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:57.022+0000 7f76a7fff640 1 -- 172.21.15.49:0/4089401085 learned_addr learned my addr 172.21.15.49:0/4089401085 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:14:57.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:57.022+0000 7f76a77fe640 1 --2- 172.21.15.49:0/4089401085 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f76a80b1f90 0x7f76a80cda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:57.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:57.022+0000 7f769ffff640 1 -- 172.21.15.49:0/4089401085 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f76a80b1f90 msgr2=0x7f76a80cda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:57.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:57.022+0000 7f769ffff640 1 --2- 172.21.15.49:0/4089401085 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f76a80b1f90 0x7f76a80cda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:57.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:57.022+0000 7f769ffff640 1 -- 172.21.15.49:0/4089401085 >> v1:172.21.15.49:6789/0 conn(0x7f76a80d2190 legacy=0x7f76a80714a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:57.019 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:57.022+0000 7f769ffff640 1 -- 172.21.15.49:0/4089401085 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76a80ce100 con 0x7f76a8161650 2023-11-06T02:14:57.019 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:57.022+0000 7f76a6ffd640 1 -- 172.21.15.49:0/4089401085 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 817822536 0 0) 0x7f76a80d17c0 con 0x7f76a80d2190 2023-11-06T02:14:57.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:57.022+0000 7f769ffff640 1 --2- 172.21.15.49:0/4089401085 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f76a8161650 0x7f76a80d1c50 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f76900097c0 tx=0x7f769001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df830e062c12d723 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:57.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:57.022+0000 7f76a6ffd640 1 -- 172.21.15.49:0/4089401085 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76900ab650 con 0x7f76a8161650 2023-11-06T02:14:57.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:57.022+0000 7f76a6ffd640 1 -- 172.21.15.49:0/4089401085 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76900ab7b0 con 0x7f76a8161650 2023-11-06T02:14:57.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:57.022+0000 7f76a6ffd640 1 -- 172.21.15.49:0/4089401085 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76900abb40 con 0x7f76a8161650 2023-11-06T02:14:57.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:57.026+0000 7f76adfce640 1 -- 172.21.15.49:0/4089401085 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f76a8161650 msgr2=0x7f76a80d1c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:57.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:57.026+0000 7f76adfce640 1 --2- 172.21.15.49:0/4089401085 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f76a8161650 0x7f76a80d1c50 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f76900097c0 tx=0x7f769001c450 comp rx=0 tx=0).stop 2023-11-06T02:14:57.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:57.026+0000 7f76adfce640 1 -- 172.21.15.49:0/4089401085 shutdown_connections 2023-11-06T02:14:57.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:57.026+0000 7f76adfce640 1 --2- 172.21.15.49:0/4089401085 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f76a8161650 0x7f76a80d1c50 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:57.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:57.026+0000 7f76adfce640 1 --2- 172.21.15.49:0/4089401085 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f76a80b1f90 0x7f76a80cda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:57.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:57.026+0000 7f76adfce640 1 -- 172.21.15.49:0/4089401085 >> 172.21.15.49:0/4089401085 conn(0x7f76a80d2aa0 msgr2=0x7f76a80d2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:57.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:57.026+0000 7f76adfce640 1 -- 172.21.15.49:0/4089401085 shutdown_connections 2023-11-06T02:14:57.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:57.026+0000 7f76adfce640 1 -- 172.21.15.49:0/4089401085 wait complete. 2023-11-06T02:14:57.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:57.026+0000 7f76adfce640 1 Processor -- start 2023-11-06T02:14:57.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:57.026+0000 7f76adfce640 1 -- start start 2023-11-06T02:14:57.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:57.026+0000 7f76adfce640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76a80b1f90 0x7f76a8157990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:57.026 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:57.026+0000 7f76adfce640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f76a80d2190 0x7f76a80b2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:57.026 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:57.026+0000 7f76adfce640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f76a8161650 0x7f76a80b33c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:57.026 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:57.026+0000 7f76adfce640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a8157ed0 con 0x7f76a8161650 2023-11-06T02:14:57.026 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:57.026+0000 7f76adfce640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a815cbb0 con 0x7f76a80b1f90 2023-11-06T02:14:57.027 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:57.026+0000 7f76adfce640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76a80cc2e0 con 0x7f76a80d2190 2023-11-06T02:14:57.027 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:57.026+0000 7f76a7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f76a8161650 0x7f76a80b33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:57.027 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:57.026+0000 7f76a7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f76a8161650 0x7f76a80b33c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44682/0 (socket says 172.21.15.49:44682) 2023-11-06T02:14:57.027 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:57.026+0000 7f76a7fff640 1 -- 172.21.15.49:0/4005782414 learned_addr learned my addr 172.21.15.49:0/4005782414 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:57.027 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:57.026+0000 7f76a77fe640 1 --2- 172.21.15.49:0/4005782414 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76a80b1f90 0x7f76a8157990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:57.028 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:57.026+0000 7f769ffff640 1 --2- 172.21.15.49:0/4005782414 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f76a80d2190 0x7f76a80b2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:57.028 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:57.026+0000 7f76a7fff640 1 -- 172.21.15.49:0/4005782414 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f76a80d2190 msgr2=0x7f76a80b2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:57.028 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:57.026+0000 7f76a7fff640 1 --2- 172.21.15.49:0/4005782414 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f76a80d2190 0x7f76a80b2e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:57.028 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:57.026+0000 7f76a7fff640 1 -- 172.21.15.49:0/4005782414 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76a80b1f90 msgr2=0x7f76a8157990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:57.028 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:57.026+0000 7f76a7fff640 1 --2- 172.21.15.49:0/4005782414 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76a80b1f90 0x7f76a8157990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:57.029 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:57.026+0000 7f76a7fff640 1 -- 172.21.15.49:0/4005782414 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7690009420 con 0x7f76a8161650 2023-11-06T02:14:57.029 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:57.026+0000 7f76a77fe640 1 --2- 172.21.15.49:0/4005782414 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76a80b1f90 0x7f76a8157990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:14:57.029 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:57.026+0000 7f76a7fff640 1 --2- 172.21.15.49:0/4005782414 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f76a8161650 0x7f76a80b33c0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f76a000c7a0 tx=0x7f76a000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:57.029 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:57.026+0000 7f76a4ff9640 1 -- 172.21.15.49:0/4005782414 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a000a640 con 0x7f76a8161650 2023-11-06T02:14:57.029 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:57.026+0000 7f76a4ff9640 1 -- 172.21.15.49:0/4005782414 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76a000a7a0 con 0x7f76a8161650 2023-11-06T02:14:57.029 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:57.026+0000 7f76adfce640 1 -- 172.21.15.49:0/4005782414 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76a80cc5c0 con 0x7f76a8161650 2023-11-06T02:14:57.030 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:57.026+0000 7f76a4ff9640 1 -- 172.21.15.49:0/4005782414 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a000abd0 con 0x7f76a8161650 2023-11-06T02:14:57.030 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:57.026+0000 7f76adfce640 1 -- 172.21.15.49:0/4005782414 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76a80ccb80 con 0x7f76a8161650 2023-11-06T02:14:57.030 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:57.030+0000 7f76adfce640 1 -- 172.21.15.49:0/4005782414 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76a8074040 con 0x7f76a8161650 2023-11-06T02:14:57.030 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:57.030+0000 7f76a4ff9640 1 -- 172.21.15.49:0/4005782414 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f76a000ad30 con 0x7f76a8161650 2023-11-06T02:14:57.030 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:57.030+0000 7f76a4ff9640 1 --2- 172.21.15.49:0/4005782414 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7688065a60 0x7f7688067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:57.031 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:57.030+0000 7f76a4ff9640 1 -- 172.21.15.49:0/4005782414 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f76a00828e0 con 0x7f76a8161650 2023-11-06T02:14:57.031 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:57.030+0000 7f76a77fe640 1 --2- 172.21.15.49:0/4005782414 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7688065a60 0x7f7688067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:57.031 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:57.030+0000 7f76a77fe640 1 --2- 172.21.15.49:0/4005782414 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7688065a60 0x7f7688067f20 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f7698009840 tx=0x7f7698008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:57.031 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:57.034+0000 7f76a4ff9640 1 -- 172.21.15.49:0/4005782414 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f76a0050390 con 0x7f76a8161650 2023-11-06T02:14:57.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:57.246+0000 7f76adfce640 1 -- 172.21.15.49:0/4005782414 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]} v 0) v1 -- 0x7f76a8071640 con 0x7f76a8161650 2023-11-06T02:14:57.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:57.510+0000 7f76a4ff9640 1 -- 172.21.15.49:0/4005782414 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]}]=0 v323) v1 ==== 144+0+0 (secure 0 0 0) 0x7f76a00180d0 con 0x7f76a8161650 2023-11-06T02:14:57.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:57.514+0000 7f76adfce640 1 -- 172.21.15.49:0/4005782414 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7688065a60 msgr2=0x7f7688067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:57.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:57.514+0000 7f76adfce640 1 --2- 172.21.15.49:0/4005782414 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7688065a60 0x7f7688067f20 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f7698009840 tx=0x7f7698008040 comp rx=0 tx=0).stop 2023-11-06T02:14:57.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:57.514+0000 7f76adfce640 1 -- 172.21.15.49:0/4005782414 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f76a8161650 msgr2=0x7f76a80b33c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:57.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:57.514+0000 7f76adfce640 1 --2- 172.21.15.49:0/4005782414 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f76a8161650 0x7f76a80b33c0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f76a000c7a0 tx=0x7f76a000cc60 comp rx=0 tx=0).stop 2023-11-06T02:14:57.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:57.514+0000 7f76adfce640 1 -- 172.21.15.49:0/4005782414 shutdown_connections 2023-11-06T02:14:57.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:57.514+0000 7f76adfce640 1 --2- 172.21.15.49:0/4005782414 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7688065a60 0x7f7688067f20 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:57.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:57.514+0000 7f76adfce640 1 --2- 172.21.15.49:0/4005782414 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f76a8161650 0x7f76a80b33c0 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:57.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:57.514+0000 7f76adfce640 1 --2- 172.21.15.49:0/4005782414 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f76a80d2190 0x7f76a80b2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:57.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:57.514+0000 7f76adfce640 1 --2- 172.21.15.49:0/4005782414 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76a80b1f90 0x7f76a8157990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:57.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:57.514+0000 7f76adfce640 1 -- 172.21.15.49:0/4005782414 >> 172.21.15.49:0/4005782414 conn(0x7f76a80d2aa0 msgr2=0x7f76a80b4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:57.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:57.514+0000 7f76adfce640 1 -- 172.21.15.49:0/4005782414 shutdown_connections 2023-11-06T02:14:57.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:57.514+0000 7f76adfce640 1 -- 172.21.15.49:0/4005782414 wait complete. 2023-11-06T02:14:57.528 INFO:tasks.cephfs_test_runner:test_orphan_scan (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2023-11-06T02:14:57.529 INFO:tasks.cephfs_test_runner: 2023-11-06T02:14:57.529 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-06T02:14:57.530 INFO:tasks.cephfs_test_runner:Ran 4 tests in 349.691s 2023-11-06T02:14:57.530 INFO:tasks.cephfs_test_runner: 2023-11-06T02:14:57.530 INFO:tasks.cephfs_test_runner:OK 2023-11-06T02:14:57.531 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-11-06T02:14:57.540 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2023-11-06T02:14:57.548 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2023-11-06T02:14:57.549 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-11-06T02:14:57.549 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T02:14:57.555 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336476k,nr_inodes=4084119,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16105 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-06T02:14:57.556 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-11-06T02:14:57.556 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T02:14:57.604 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336476k,nr_inodes=4084119,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16105 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-06T02:14:57.604 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-11-06T02:14:57.604 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T02:14:57.611 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336552k,nr_inodes=4084138,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19862 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274692k,nr_inodes=818673,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-06T02:14:57.612 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-11-06T02:14:57.612 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T02:14:57.661 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336552k,nr_inodes=4084138,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19862 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274692k,nr_inodes=818673,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-06T02:14:57.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:14:57.662 DEBUG:teuthology.orchestra.run.smithi150:> ip netns list 2023-11-06T02:14:57.709 INFO:teuthology.orchestra.run.smithi150.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-11-06T02:14:57.709 INFO:teuthology.orchestra.run.smithi150.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-06T02:14:57.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:14:57.710 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-06T02:14:57.765 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:14:57.766 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-06T02:14:57.826 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:14:57.826 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link delete ceph-brx 2023-11-06T02:14:57.997 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:14:57.998 DEBUG:teuthology.orchestra.run.smithi049:> ip netns list 2023-11-06T02:14:58.005 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-06T02:14:58.005 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-06T02:14:58.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:14:58.005 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-06T02:14:58.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:14:58.061 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-06T02:14:58.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T02:14:58.130 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link delete ceph-brx 2023-11-06T02:14:58.301 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-11-06T02:14:58.311 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-11-06T02:14:58.312 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T02:14:58.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.442+0000 7f51c7372640 1 Processor -- start 2023-11-06T02:14:58.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.442+0000 7f51c7372640 1 -- start start 2023-11-06T02:14:58.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.442+0000 7f51c7372640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f51c00b0d80 0x7f51c006b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:58.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.442+0000 7f51c7372640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51c00b1970 0x7f51c00cb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:58.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.442+0000 7f51c7372640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51c006bce0 con 0x7f51c00b1420 2023-11-06T02:14:58.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.442+0000 7f51c7372640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51c00cffb0 con 0x7f51c00b0d80 2023-11-06T02:14:58.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.442+0000 7f51c7372640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51c00cf710 con 0x7f51c00b1970 2023-11-06T02:14:58.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.442+0000 7f51c510e640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f51c00b0d80 0x7f51c006b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:58.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.442+0000 7f51c590f640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51c00b1970 0x7f51c00cb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:58.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.442+0000 7f51c490d640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f51c00b1420 0x7f51c006b5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.49:33090/0 (socket says 172.21.15.49:33090) 2023-11-06T02:14:58.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.442+0000 7f51c490d640 1 -- 172.21.15.49:0/4259318320 learned_addr learned my addr 172.21.15.49:0/4259318320 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:14:58.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.442+0000 7f51c510e640 1 -- 172.21.15.49:0/4259318320 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51c00b1970 msgr2=0x7f51c00cb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:58.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.442+0000 7f51c510e640 1 --2- 172.21.15.49:0/4259318320 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51c00b1970 0x7f51c00cb280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:58.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.442+0000 7f51c510e640 1 -- 172.21.15.49:0/4259318320 >> v1:172.21.15.150:6789/0 conn(0x7f51c00b1420 legacy=0x7f51c006b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:58.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.442+0000 7f51c510e640 1 -- 172.21.15.49:0/4259318320 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51c00cf9f0 con 0x7f51c00b0d80 2023-11-06T02:14:58.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.446+0000 7f51c510e640 1 --2- 172.21.15.49:0/4259318320 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51c00b0d80 0x7f51c006b090 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f51bc0097a0 tx=0x7f51bc01c430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=500db5bfa7f8e091 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:58.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.446+0000 7f51b7fff640 1 -- 172.21.15.49:0/4259318320 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51bc0ab650 con 0x7f51c00b0d80 2023-11-06T02:14:58.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.446+0000 7f51b7fff640 1 -- 172.21.15.49:0/4259318320 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51bc0ab7b0 con 0x7f51c00b0d80 2023-11-06T02:14:58.443 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.446+0000 7f51b7fff640 1 -- 172.21.15.49:0/4259318320 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51bc0abb40 con 0x7f51c00b0d80 2023-11-06T02:14:58.443 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.446+0000 7f51c7372640 1 -- 172.21.15.49:0/4259318320 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51c00b0d80 msgr2=0x7f51c006b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:58.443 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.446+0000 7f51c7372640 1 --2- 172.21.15.49:0/4259318320 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51c00b0d80 0x7f51c006b090 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f51bc0097a0 tx=0x7f51bc01c430 comp rx=0 tx=0).stop 2023-11-06T02:14:58.443 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.446+0000 7f51c7372640 1 -- 172.21.15.49:0/4259318320 shutdown_connections 2023-11-06T02:14:58.443 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.446+0000 7f51c7372640 1 --2- 172.21.15.49:0/4259318320 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51c00b1970 0x7f51c00cb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:58.444 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.446+0000 7f51c7372640 1 --2- 172.21.15.49:0/4259318320 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51c00b0d80 0x7f51c006b090 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:58.444 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.446+0000 7f51c7372640 1 -- 172.21.15.49:0/4259318320 >> 172.21.15.49:0/4259318320 conn(0x7f51c00d0890 msgr2=0x7f51c00d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:58.444 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.446+0000 7f51c7372640 1 -- 172.21.15.49:0/4259318320 shutdown_connections 2023-11-06T02:14:58.444 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.446+0000 7f51c7372640 1 -- 172.21.15.49:0/4259318320 wait complete. 2023-11-06T02:14:58.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.450+0000 7f51c7372640 1 Processor -- start 2023-11-06T02:14:58.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.450+0000 7f51c7372640 1 -- start start 2023-11-06T02:14:58.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.450+0000 7f51c7372640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51c00b0d80 0x7f51c01518e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:58.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.450+0000 7f51c7372640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51c00b1420 0x7f51c00c5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:58.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.450+0000 7f51c7372640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51c00b1970 0x7f51c0156f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:58.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.450+0000 7f51c7372640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c00c5f20 con 0x7f51c00b1970 2023-11-06T02:14:58.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.450+0000 7f51c7372640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c00c6200 con 0x7f51c00b0d80 2023-11-06T02:14:58.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.450+0000 7f51c7372640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51c00c64e0 con 0x7f51c00b1420 2023-11-06T02:14:58.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.450+0000 7f51c590f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51c00b1970 0x7f51c0156f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:58.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.450+0000 7f51c590f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51c00b1970 0x7f51c0156f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44696/0 (socket says 172.21.15.49:44696) 2023-11-06T02:14:58.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.450+0000 7f51c590f640 1 -- 172.21.15.49:0/484408712 learned_addr learned my addr 172.21.15.49:0/484408712 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:58.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.450+0000 7f51c510e640 1 --2- 172.21.15.49:0/484408712 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51c00b0d80 0x7f51c01518e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:58.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.450+0000 7f51c490d640 1 --2- 172.21.15.49:0/484408712 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51c00b1420 0x7f51c00c5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:58.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.450+0000 7f51c590f640 1 -- 172.21.15.49:0/484408712 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51c00b1420 msgr2=0x7f51c00c5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:58.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.450+0000 7f51c590f640 1 --2- 172.21.15.49:0/484408712 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51c00b1420 0x7f51c00c5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:58.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.450+0000 7f51c590f640 1 -- 172.21.15.49:0/484408712 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51c00b0d80 msgr2=0x7f51c01518e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:58.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.450+0000 7f51c590f640 1 --2- 172.21.15.49:0/484408712 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51c00b0d80 0x7f51c01518e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:58.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.450+0000 7f51c590f640 1 -- 172.21.15.49:0/484408712 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51bc0093b0 con 0x7f51c00b1970 2023-11-06T02:14:58.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.450+0000 7f51c510e640 1 --2- 172.21.15.49:0/484408712 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51c00b0d80 0x7f51c01518e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:14:58.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.450+0000 7f51c590f640 1 --2- 172.21.15.49:0/484408712 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51c00b1970 0x7f51c0156f90 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f51b800efc0 tx=0x7f51b800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:58.451 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.450+0000 7f51b5ffb640 1 -- 172.21.15.49:0/484408712 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b8009ba0 con 0x7f51c00b1970 2023-11-06T02:14:58.451 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.450+0000 7f51b5ffb640 1 -- 172.21.15.49:0/484408712 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51b800f040 con 0x7f51c00b1970 2023-11-06T02:14:58.451 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.450+0000 7f51b5ffb640 1 -- 172.21.15.49:0/484408712 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b80146c0 con 0x7f51c00b1970 2023-11-06T02:14:58.451 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.450+0000 7f51c7372640 1 -- 172.21.15.49:0/484408712 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51c00c67c0 con 0x7f51c00b1970 2023-11-06T02:14:58.451 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.450+0000 7f51c7372640 1 -- 172.21.15.49:0/484408712 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51c00c5b50 con 0x7f51c00b1970 2023-11-06T02:14:58.452 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.450+0000 7f51c7372640 1 -- 172.21.15.49:0/484408712 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51c00d06c0 con 0x7f51c00b1970 2023-11-06T02:14:58.452 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.450+0000 7f51b5ffb640 1 -- 172.21.15.49:0/484408712 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f51b8014820 con 0x7f51c00b1970 2023-11-06T02:14:58.452 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.454+0000 7f51b5ffb640 1 --2- 172.21.15.49:0/484408712 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f51a4065a60 0x7f51a4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:58.452 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.454+0000 7f51b5ffb640 1 -- 172.21.15.49:0/484408712 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f51b8083dc0 con 0x7f51c00b1970 2023-11-06T02:14:58.453 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.454+0000 7f51c510e640 1 --2- 172.21.15.49:0/484408712 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f51a4065a60 0x7f51a4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:58.453 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.454+0000 7f51c510e640 1 --2- 172.21.15.49:0/484408712 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f51a4065a60 0x7f51a4067f20 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f51bc0af000 tx=0x7f51bc0a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:58.453 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.458+0000 7f51b5ffb640 1 -- 172.21.15.49:0/484408712 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f51b8051900 con 0x7f51c00b1970 2023-11-06T02:14:58.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.606+0000 7f51c7372640 1 -- 172.21.15.49:0/484408712 --> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f51c00c6370 con 0x7f51a4065a60 2023-11-06T02:14:58.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.614+0000 7f51b5ffb640 1 -- 172.21.15.49:0/484408712 <== mgr.4100 v2:172.21.15.150:6832/38104 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+71133 (secure 0 0 0) 0x7f51c00c6370 con 0x7f51a4065a60 2023-11-06T02:14:58.612 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:58.612 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2023-11-06T02:14:58.613 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.618+0000 7f51c7372640 1 -- 172.21.15.49:0/484408712 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f51a4065a60 msgr2=0x7f51a4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:58.614 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.618+0000 7f51c7372640 1 --2- 172.21.15.49:0/484408712 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f51a4065a60 0x7f51a4067f20 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f51bc0af000 tx=0x7f51bc0a7000 comp rx=0 tx=0).stop 2023-11-06T02:14:58.614 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.618+0000 7f51c7372640 1 -- 172.21.15.49:0/484408712 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51c00b1970 msgr2=0x7f51c0156f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:58.614 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.618+0000 7f51c7372640 1 --2- 172.21.15.49:0/484408712 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51c00b1970 0x7f51c0156f90 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f51b800efc0 tx=0x7f51b800c490 comp rx=0 tx=0).stop 2023-11-06T02:14:58.614 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.618+0000 7f51c7372640 1 -- 172.21.15.49:0/484408712 shutdown_connections 2023-11-06T02:14:58.615 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.618+0000 7f51c7372640 1 --2- 172.21.15.49:0/484408712 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f51a4065a60 0x7f51a4067f20 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:58.615 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.618+0000 7f51c7372640 1 --2- 172.21.15.49:0/484408712 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51c00b1970 0x7f51c0156f90 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:58.615 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.618+0000 7f51c7372640 1 --2- 172.21.15.49:0/484408712 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51c00b1420 0x7f51c00c5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:58.615 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.618+0000 7f51c7372640 1 --2- 172.21.15.49:0/484408712 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51c00b0d80 0x7f51c01518e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:58.615 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.618+0000 7f51c7372640 1 -- 172.21.15.49:0/484408712 >> 172.21.15.49:0/484408712 conn(0x7f51c00d0890 msgr2=0x7f51c00cd9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:58.616 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.618+0000 7f51c7372640 1 -- 172.21.15.49:0/484408712 shutdown_connections 2023-11-06T02:14:58.616 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.618+0000 7f51c7372640 1 -- 172.21.15.49:0/484408712 wait complete. 2023-11-06T02:14:58.629 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":278,"stamp":"2023-11-06T02:14:57.316013+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":269724,"kb_used_data":25436,"kb_used_omap":0,"kb_used_meta":244224,"kb_avail":754704996,"statfs":{"total":773094113280,"available":772817915904,"internally_reserved":0,"allocated":26046464,"data_stored":21579923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":250085376},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8390586,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-258,"num_read_kb":-264,"num_write":-177,"num_write_kb":-8295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002028"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":221,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T02:14:23.281538+0000","last_change":"2023-11-06T02:08:20.523427+0000","last_active":"2023-11-06T02:14:23.281538+0000","last_peered":"2023-11-06T02:14:23.281538+0000","last_clean":"2023-11-06T02:14:23.281538+0000","last_became_active":"2023-11-06T02:08:20.521089+0000","last_became_peered":"2023-11-06T02:08:20.521089+0000","last_unstale":"2023-11-06T02:14:23.281538+0000","last_undegraded":"2023-11-06T02:14:23.281538+0000","last_fullsized":"2023-11-06T02:14:23.281538+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:19.252348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:19.252348+0000","last_clean_scrub_stamp":"2023-11-06T02:08:19.252348+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:30:15.566397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":67,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T02:14:35.449337+0000","last_change":"2023-11-06T02:08:26.059502+0000","last_active":"2023-11-06T02:14:35.449337+0000","last_peered":"2023-11-06T02:14:35.449337+0000","last_clean":"2023-11-06T02:14:35.449337+0000","last_became_active":"2023-11-06T02:08:26.058612+0000","last_became_peered":"2023-11-06T02:08:26.058612+0000","last_unstale":"2023-11-06T02:14:35.449337+0000","last_undegraded":"2023-11-06T02:14:35.449337+0000","last_fullsized":"2023-11-06T02:14:35.449337+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:10.626021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":68,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T02:14:28.282679+0000","last_change":"2023-11-06T02:08:26.060406+0000","last_active":"2023-11-06T02:14:28.282679+0000","last_peered":"2023-11-06T02:14:28.282679+0000","last_clean":"2023-11-06T02:14:28.282679+0000","last_became_active":"2023-11-06T02:08:26.059262+0000","last_became_peered":"2023-11-06T02:08:26.059262+0000","last_unstale":"2023-11-06T02:14:28.282679+0000","last_undegraded":"2023-11-06T02:14:28.282679+0000","last_fullsized":"2023-11-06T02:14:28.282679+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:10.626021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":68,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T02:14:35.447393+0000","last_change":"2023-11-06T02:08:26.559477+0000","last_active":"2023-11-06T02:14:35.447393+0000","last_peered":"2023-11-06T02:14:35.447393+0000","last_clean":"2023-11-06T02:14:35.447393+0000","last_became_active":"2023-11-06T02:08:26.559202+0000","last_became_peered":"2023-11-06T02:08:26.559202+0000","last_unstale":"2023-11-06T02:14:35.447393+0000","last_undegraded":"2023-11-06T02:14:35.447393+0000","last_fullsized":"2023-11-06T02:14:35.447393+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:55:04.552404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":67,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T02:14:35.448967+0000","last_change":"2023-11-06T02:08:26.059546+0000","last_active":"2023-11-06T02:14:35.448967+0000","last_peered":"2023-11-06T02:14:35.448967+0000","last_clean":"2023-11-06T02:14:35.448967+0000","last_became_active":"2023-11-06T02:08:26.058806+0000","last_became_peered":"2023-11-06T02:08:26.058806+0000","last_unstale":"2023-11-06T02:14:35.448967+0000","last_undegraded":"2023-11-06T02:14:35.448967+0000","last_fullsized":"2023-11-06T02:14:35.448967+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:23.468510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":67,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T02:14:35.439334+0000","last_change":"2023-11-06T02:08:26.436463+0000","last_active":"2023-11-06T02:14:35.439334+0000","last_peered":"2023-11-06T02:14:35.439334+0000","last_clean":"2023-11-06T02:14:35.439334+0000","last_became_active":"2023-11-06T02:08:26.436075+0000","last_became_peered":"2023-11-06T02:08:26.436075+0000","last_unstale":"2023-11-06T02:14:35.439334+0000","last_undegraded":"2023-11-06T02:14:35.439334+0000","last_fullsized":"2023-11-06T02:14:35.439334+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:23.468510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":68,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T02:14:35.357267+0000","last_change":"2023-11-06T02:08:26.436607+0000","last_active":"2023-11-06T02:14:35.357267+0000","last_peered":"2023-11-06T02:14:35.357267+0000","last_clean":"2023-11-06T02:14:35.357267+0000","last_became_active":"2023-11-06T02:08:26.435924+0000","last_became_peered":"2023-11-06T02:08:26.435924+0000","last_unstale":"2023-11-06T02:14:35.357267+0000","last_undegraded":"2023-11-06T02:14:35.357267+0000","last_fullsized":"2023-11-06T02:14:35.357267+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:23.468510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":66,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T02:14:13.290705+0000","last_change":"2023-11-06T02:08:26.057076+0000","last_active":"2023-11-06T02:14:13.290705+0000","last_peered":"2023-11-06T02:14:13.290705+0000","last_clean":"2023-11-06T02:14:13.290705+0000","last_became_active":"2023-11-06T02:08:26.056419+0000","last_became_peered":"2023-11-06T02:08:26.056419+0000","last_unstale":"2023-11-06T02:14:13.290705+0000","last_undegraded":"2023-11-06T02:14:13.290705+0000","last_fullsized":"2023-11-06T02:14:13.290705+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:10.626021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":75,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T02:14:13.297179+0000","last_change":"2023-11-06T02:08:26.058596+0000","last_active":"2023-11-06T02:14:13.297179+0000","last_peered":"2023-11-06T02:14:13.297179+0000","last_clean":"2023-11-06T02:14:13.297179+0000","last_became_active":"2023-11-06T02:08:26.057015+0000","last_became_peered":"2023-11-06T02:08:26.057015+0000","last_unstale":"2023-11-06T02:14:13.297179+0000","last_undegraded":"2023-11-06T02:14:13.297179+0000","last_fullsized":"2023-11-06T02:14:13.297179+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:55:04.552404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574930,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32608,"kb_used_data":1560,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339232,"statfs":{"total":96636764160,"available":96603373568,"internally_reserved":0,"allocated":1597440,"data_stored":1042734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.87,"15min":0.85599999999999998},"min":{"1min":0.62,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.147,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.52700000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.2230000000000001,"5min":1.421,"15min":1.506},"last":1.0089999999999999}]},{"osd":1,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.83399999999999996,"15min":0.84499999999999997},"min":{"1min":0.498,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.3320000000000001,"5min":1.345,"15min":1.345},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.92900000000000005,"15min":0.90200000000000002},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.54900000000000004},"max":{"1min":1.131,"5min":1.681,"15min":1.681},"last":0.97399999999999998}]},{"osd":2,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.96399999999999997,"15min":0.95699999999999996},"min":{"1min":0.65300000000000002,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.1919999999999999,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":1.133},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.92200000000000004,"15min":0.97399999999999998},"min":{"1min":0.57399999999999995,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.1919999999999999,"5min":1.5,"15min":1.5920000000000001},"last":0.74199999999999999}]},{"osd":3,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.89200000000000002,"15min":0.88},"min":{"1min":0.66100000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.1519999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":1.234},{"interface":"front","average":{"1min":0.995,"5min":1.03,"15min":1.0609999999999999},"min":{"1min":0.74399999999999999,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.2110000000000001,"5min":1.526,"15min":1.526},"last":0.92200000000000004}]},{"osd":4,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.91700000000000004,"15min":0.94399999999999995},"min":{"1min":0.73399999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.1930000000000001,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.97999999999999998,"15min":1.0229999999999999},"min":{"1min":0.65000000000000002,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.1519999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":1.0640000000000001}]},{"osd":5,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":1.026,"15min":1.0609999999999999},"min":{"1min":0.69199999999999995,"5min":0.503,"15min":0.503},"max":{"1min":1.2629999999999999,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":1.0389999999999999},{"interface":"front","average":{"1min":1.044,"5min":1.044,"15min":1.105},"min":{"1min":0.71299999999999997,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.288,"5min":1.3700000000000001,"15min":1.7809999999999999},"last":0.94999999999999996}]},{"osd":6,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":1.1180000000000001,"15min":1.1739999999999999},"min":{"1min":0.89500000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.274,"5min":1.5009999999999999,"15min":1.6679999999999999},"last":1.113},{"interface":"front","average":{"1min":1.028,"5min":1.0649999999999999,"15min":1.127},"min":{"1min":0.72599999999999998,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.2509999999999999,"5min":1.4570000000000001,"15min":1.74},"last":1.1579999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574929,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36136,"kb_used_data":5088,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94335704,"statfs":{"total":96636764160,"available":96599760896,"internally_reserved":0,"allocated":5210112,"data_stored":4647957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.872,"15min":0.86199999999999999},"min":{"1min":0.57199999999999995,"5min":0.495,"15min":0.495},"max":{"1min":1.226,"5min":1.226,"15min":1.2609999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.88600000000000001,"15min":0.83199999999999996},"min":{"1min":0.5,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.3280000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.68899999999999995}]},{"osd":1,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.85099999999999998,"15min":0.86699999999999999},"min":{"1min":0.54400000000000004,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.1639999999999999,"5min":1.244,"15min":1.244},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.90500000000000003,"15min":0.88400000000000001},"min":{"1min":0.56599999999999995,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":0.82599999999999996}]},{"osd":2,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.89200000000000002,"15min":0.89600000000000002},"min":{"1min":0.61499999999999999,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.877,"5min":0.88900000000000001,"15min":0.91700000000000004},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.075,"5min":1.411,"15min":1.411},"last":0.96199999999999997}]},{"osd":3,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.93500000000000005,"15min":0.95899999999999996},"min":{"1min":0.74099999999999999,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.92300000000000004},{"interface":"front","average":{"1min":1.0309999999999999,"5min":0.999,"15min":1},"min":{"1min":0.81599999999999995,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.2609999999999999,"5min":1.381,"15min":1.381},"last":1.0149999999999999}]},{"osd":4,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.92200000000000004,"15min":0.96999999999999997},"min":{"1min":0.627,"5min":0.56799999999999995,"15min":0.52900000000000003},"max":{"1min":1.2390000000000001,"5min":1.302,"15min":1.474},"last":0.995},{"interface":"front","average":{"1min":0.997,"5min":0.97399999999999998,"15min":1.018},"min":{"1min":0.66700000000000004,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.4039999999999999,"5min":1.556,"15min":1.556},"last":1.232}]},{"osd":5,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":0.97199999999999998,"15min":1.0089999999999999},"min":{"1min":0.68899999999999995,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":1.1950000000000001},{"interface":"front","average":{"1min":1.1319999999999999,"5min":1.075,"15min":1.101},"min":{"1min":0.82699999999999996,"5min":0.66900000000000004,"15min":0.58699999999999997},"max":{"1min":1.3999999999999999,"5min":1.587,"15min":1.587},"last":1.069}]},{"osd":7,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.087,"15min":1.107},"min":{"1min":0.88300000000000001,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.5369999999999999,"5min":1.615,"15min":1.615},"last":1.377},{"interface":"front","average":{"1min":1.103,"5min":1.0509999999999999,"15min":1.127},"min":{"1min":0.85599999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.4610000000000001,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":1.284}]}]},{"osd":5,"up_from":13,"seq":55834574930,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27940,"kb_used_data":988,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343900,"statfs":{"total":96636764160,"available":96608153600,"internally_reserved":0,"allocated":1011712,"data_stored":452390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:14:33 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.86199999999999999,"15min":0.83999999999999997},"min":{"1min":0.54400000000000004,"5min":0.51700000000000002,"15min":0.47299999999999998},"max":{"1min":1.079,"5min":1.381,"15min":1.381},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.86899999999999999,"15min":0.85099999999999998},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.159,"5min":1.3200000000000001,"15min":1.6579999999999999},"last":0.56000000000000005}]},{"osd":1,"last update":"Mon Nov 6 02:14:33 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.90500000000000003,"15min":0.86799999999999999},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.60299999999999998},"max":{"1min":1.369,"5min":1.369,"15min":1.369},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.90000000000000002,"15min":0.84399999999999997},"min":{"1min":0.53700000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.1639999999999999,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.69199999999999995}]},{"osd":2,"last update":"Mon Nov 6 02:14:33 2023","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.93899999999999995,"15min":0.94299999999999995},"min":{"1min":0.65100000000000002,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.242,"5min":1.298,"15min":1.3560000000000001},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.89700000000000002,"15min":0.82499999999999996},"min":{"1min":0.58699999999999997,"5min":0.55800000000000005,"15min":0.47899999999999998},"max":{"1min":1.218,"5min":1.323,"15min":1.377},"last":0.76500000000000001}]},{"osd":3,"last update":"Mon Nov 6 02:14:33 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.95199999999999996,"15min":0.93000000000000005},"min":{"1min":0.66400000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.292,"5min":1.3580000000000001,"15min":1.4399999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":1.0369999999999999,"5min":0.98799999999999999,"15min":1.05},"min":{"1min":0.76700000000000002,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.2569999999999999,"5min":1.349,"15min":1.7250000000000001},"last":0.99199999999999999}]},{"osd":4,"last update":"Mon Nov 6 02:14:33 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.87,"15min":0.90000000000000002},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.0129999999999999,"5min":1.276,"15min":1.276},"last":0.94799999999999995},{"interface":"front","average":{"1min":0.995,"5min":0.98599999999999999,"15min":1.014},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.496,"5min":1.496,"15min":1.554},"last":0.90300000000000002}]},{"osd":6,"last update":"Mon Nov 6 02:14:33 2023","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.98699999999999999,"15min":1.0409999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.3440000000000001,"5min":1.454,"15min":1.454},"last":1.0640000000000001},{"interface":"front","average":{"1min":1.046,"5min":1.0229999999999999,"15min":1.0349999999999999},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":2.2309999999999999,"5min":2.2309999999999999,"15min":2.2309999999999999},"last":1.02}]},{"osd":7,"last update":"Mon Nov 6 02:14:33 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.0640000000000001,"15min":1.0720000000000001},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.5209999999999999,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":0.83799999999999997},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.117,"15min":1.1419999999999999},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":2.3759999999999999,"5min":2.3759999999999999,"15min":2.3759999999999999},"last":1.153}]}]},{"osd":4,"up_from":13,"seq":55834574930,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32044,"kb_used_data":996,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339796,"statfs":{"total":96636764160,"available":96603951104,"internally_reserved":0,"allocated":1019904,"data_stored":453752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:14:27 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.86399999999999999,"15min":0.83799999999999997},"min":{"1min":0.53500000000000003,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.2450000000000001,"5min":1.343,"15min":1.343},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.84499999999999997,"15min":0.79500000000000004},"min":{"1min":0.51600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.175,"5min":1.28,"15min":1.3049999999999999},"last":0.85899999999999999}]},{"osd":1,"last update":"Mon Nov 6 02:14:27 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.86399999999999999,"15min":0.84999999999999998},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.508,"5min":1.508,"15min":1.508},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.91800000000000004,"15min":0.90000000000000002},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.1080000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.80100000000000005}]},{"osd":2,"last update":"Mon Nov 6 02:14:27 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.90800000000000003,"15min":0.98199999999999998},"min":{"1min":0.57999999999999996,"5min":0.503,"15min":0.503},"max":{"1min":1.0880000000000001,"5min":1.389,"15min":1.615},"last":1.159},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.95499999999999996,"15min":0.93500000000000005},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.57899999999999996},"max":{"1min":1.1559999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.92400000000000004}]},{"osd":3,"last update":"Mon Nov 6 02:14:27 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.86199999999999999,"15min":0.90000000000000002},"min":{"1min":0.59899999999999998,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.22,"5min":1.22,"15min":1.52},"last":0.745},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.97299999999999998,"15min":1.0369999999999999},"min":{"1min":0.68100000000000005,"5min":0.63900000000000001,"15min":0.59999999999999998},"max":{"1min":1.589,"5min":1.595,"15min":1.595},"last":1.0509999999999999}]},{"osd":5,"last update":"Mon Nov 6 02:14:27 2023","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":1.026},"min":{"1min":0.72599999999999998,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.212,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":1.1200000000000001},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.96099999999999997,"15min":1.024},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.629},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.7869999999999999},"last":1.0960000000000001}]},{"osd":6,"last update":"Mon Nov 6 02:14:27 2023","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.008,"15min":1.0649999999999999},"min":{"1min":0.77600000000000002,"5min":0.68000000000000005,"15min":0.67100000000000004},"max":{"1min":1.6679999999999999,"5min":1.6679999999999999,"15min":1.6699999999999999},"last":1.216},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.085,"15min":1.1599999999999999},"min":{"1min":0.71999999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.2210000000000001,"5min":1.5900000000000001,"15min":2.1190000000000002},"last":1.004}]},{"osd":7,"last update":"Mon Nov 6 02:14:27 2023","interfaces":[{"interface":"back","average":{"1min":1.056,"5min":1.042,"15min":1.0760000000000001},"min":{"1min":0.77100000000000002,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.7110000000000001},"last":0.94699999999999995},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.117,"15min":1.1990000000000001},"min":{"1min":0.90500000000000003,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.754,"5min":1.754,"15min":1.754},"last":1.2629999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574929,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32032,"kb_used_data":984,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339808,"statfs":{"total":96636764160,"available":96603963392,"internally_reserved":0,"allocated":1007616,"data_stored":452931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63,"15min":0.64200000000000002},"min":{"1min":0.45300000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":1.171,"5min":1.198,"15min":1.198},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.65400000000000003,"15min":0.63500000000000001},"min":{"1min":0.49099999999999999,"5min":0.29899999999999999,"15min":0.29799999999999999},"max":{"1min":1.1819999999999999,"5min":1.21,"15min":1.21},"last":0.35199999999999998}]},{"osd":1,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.61599999999999999,"15min":0.626},"min":{"1min":0.42499999999999999,"5min":0.248,"15min":0.248},"max":{"1min":0.86799999999999999,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.373},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.75800000000000001,"15min":0.74199999999999999},"min":{"1min":0.46899999999999997,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.175,"5min":1.204,"15min":1.204},"last":0.53000000000000003}]},{"osd":2,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.72499999999999998,"15min":0.73099999999999998},"min":{"1min":0.499,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.438},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.81299999999999994,"15min":0.81999999999999995},"min":{"1min":0.62,"5min":0.308,"15min":0.308},"max":{"1min":1.0329999999999999,"5min":1.405,"15min":1.405},"last":0.501}]},{"osd":4,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.62,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.262,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.93700000000000006,"15min":0.97799999999999998},"min":{"1min":0.70799999999999996,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.212,"5min":1.292,"15min":1.4410000000000001},"last":0.66300000000000003}]},{"osd":5,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.97499999999999998,"15min":1.0109999999999999},"min":{"1min":0.72699999999999998,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.21,"5min":1.375,"15min":1.375},"last":0.71999999999999997},{"interface":"front","average":{"1min":1.024,"5min":1.0269999999999999,"15min":1.0629999999999999},"min":{"1min":0.65200000000000002,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.323,"5min":1.4890000000000001,"15min":2.21},"last":0.81799999999999995}]},{"osd":6,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.0209999999999999,"15min":1.024},"min":{"1min":0.79800000000000004,"5min":0.439,"15min":0.439},"max":{"1min":1.3080000000000001,"5min":1.395,"15min":1.395},"last":0.90000000000000002},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.0269999999999999,"15min":1.0569999999999999},"min":{"1min":0.74099999999999999,"5min":0.375,"15min":0.375},"max":{"1min":1.2709999999999999,"5min":1.4319999999999999,"15min":1.554},"last":0.83099999999999996}]},{"osd":7,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.1020000000000001,"15min":1.1220000000000001},"min":{"1min":0.68200000000000005,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.4410000000000001,"5min":1.5549999999999999,"15min":2.2829999999999999},"last":0.93100000000000005},{"interface":"front","average":{"1min":1.123,"5min":1.0980000000000001,"15min":1.1040000000000001},"min":{"1min":0.91100000000000003,"5min":0.497,"15min":0.497},"max":{"1min":1.3620000000000001,"5min":1.504,"15min":1.504},"last":0.96799999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574929,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36132,"kb_used_data":5084,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94335708,"statfs":{"total":96636764160,"available":96599764992,"internally_reserved":0,"allocated":5206016,"data_stored":4647390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:14:28 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.59099999999999997,"15min":0.52900000000000003},"min":{"1min":0.28499999999999998,"5min":0.252,"15min":0.252},"max":{"1min":0.90300000000000002,"5min":1.079,"15min":1.079},"last":0.752},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.63400000000000001,"15min":0.56999999999999995},"min":{"1min":0.374,"5min":0.316,"15min":0.27300000000000002},"max":{"1min":0.86699999999999999,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":0.64500000000000002}]},{"osd":1,"last update":"Mon Nov 6 02:14:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.65000000000000002,"15min":0.58999999999999997},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.26400000000000001},"max":{"1min":0.97699999999999998,"5min":1.48,"15min":1.48},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.72799999999999998,"15min":0.621},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.26500000000000001},"max":{"1min":1.04,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":0.42599999999999999}]},{"osd":3,"last update":"Mon Nov 6 02:14:28 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.71399999999999997,"15min":0.63600000000000001},"min":{"1min":0.316,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.0580000000000001,"5min":1.121,"15min":1.155},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.76400000000000001,"15min":0.68200000000000005},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.069,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.94099999999999995}]},{"osd":4,"last update":"Mon Nov 6 02:14:28 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.93100000000000005,"15min":0.84299999999999997},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.1539999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.92900000000000005,"15min":0.91900000000000004},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.234,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":0.96999999999999997}]},{"osd":5,"last update":"Mon Nov 6 02:14:28 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.96299999999999997,"15min":0.94899999999999995},"min":{"1min":0.56299999999999994,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.258,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.97299999999999998,"15min":0.94699999999999995},"min":{"1min":0.626,"5min":0.60999999999999999,"15min":0.52600000000000002},"max":{"1min":1.2310000000000001,"5min":1.615,"15min":1.7110000000000001},"last":1.038}]},{"osd":6,"last update":"Mon Nov 6 02:14:28 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.96899999999999997,"15min":0.94799999999999995},"min":{"1min":0.60199999999999998,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.218,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.93600000000000005,"5min":1.01,"15min":1.0009999999999999},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.274,"5min":1.675,"15min":1.677},"last":0.78200000000000003}]},{"osd":7,"last update":"Mon Nov 6 02:14:28 2023","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":1.0309999999999999,"15min":0.99299999999999999},"min":{"1min":0.67500000000000004,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.2869999999999999,"5min":1.659,"15min":1.659},"last":1.0649999999999999},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.0760000000000001,"15min":1.0529999999999999},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.1879999999999999,"5min":1.4650000000000001,"15min":1.5980000000000001},"last":1.0169999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574930,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36132,"kb_used_data":5084,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94335708,"statfs":{"total":96636764160,"available":96599764992,"internally_reserved":0,"allocated":5206016,"data_stored":4646765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:14:28 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.628,"15min":0.626},"min":{"1min":0.29599999999999999,"5min":0.28699999999999998,"15min":0.224},"max":{"1min":0.95099999999999996,"5min":1.603,"15min":1.603},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.71999999999999997,"15min":0.67400000000000004},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.081,"5min":2.161,"15min":2.161},"last":0.58699999999999997}]},{"osd":2,"last update":"Mon Nov 6 02:14:28 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.64700000000000002,"15min":0.63},"min":{"1min":0.34399999999999997,"5min":0.253,"15min":0.253},"max":{"1min":1.036,"5min":1.649,"15min":1.649},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.74199999999999999,"15min":0.74399999999999999},"min":{"1min":0.373,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.91700000000000004,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.90700000000000003}]},{"osd":3,"last update":"Mon Nov 6 02:14:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.80400000000000005,"15min":0.80300000000000005},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.131,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":0.97999999999999998},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.79100000000000004,"15min":0.80400000000000005},"min":{"1min":0.35499999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.94199999999999995,"5min":1.5,"15min":1.8149999999999999},"last":0.77800000000000002}]},{"osd":4,"last update":"Mon Nov 6 02:14:28 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.96199999999999997,"15min":0.98199999999999998},"min":{"1min":0.55600000000000005,"5min":0.504,"15min":0.504},"max":{"1min":1.2110000000000001,"5min":2.2759999999999998,"15min":2.2759999999999998},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.95599999999999996,"15min":0.98099999999999998},"min":{"1min":0.63300000000000001,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.0700000000000001,"5min":2.9180000000000001,"15min":2.9180000000000001},"last":1.0329999999999999}]},{"osd":5,"last update":"Mon Nov 6 02:14:28 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.97199999999999998,"15min":0.96799999999999997},"min":{"1min":0.65800000000000003,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.177,"5min":2.7469999999999999,"15min":2.7469999999999999},"last":1.2030000000000001},{"interface":"front","average":{"1min":0.98399999999999999,"5min":1.05,"15min":1.069},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.52100000000000002},"max":{"1min":1.3720000000000001,"5min":1.921,"15min":1.921},"last":1.1870000000000001}]},{"osd":6,"last update":"Mon Nov 6 02:14:28 2023","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":1.0960000000000001,"15min":1.1000000000000001},"min":{"1min":0.61899999999999999,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.304,"5min":3.5710000000000002,"15min":3.5710000000000002},"last":1.085},{"interface":"front","average":{"1min":0.94999999999999996,"5min":1.0369999999999999,"15min":1.0629999999999999},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.268,"5min":2.843,"15min":2.843},"last":1.304}]},{"osd":7,"last update":"Mon Nov 6 02:14:28 2023","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":1.0980000000000001,"15min":1.1140000000000001},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.4159999999999999,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":1.133},{"interface":"front","average":{"1min":1.052,"5min":1.1419999999999999,"15min":1.1539999999999999},"min":{"1min":0.61799999999999999,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.3520000000000001,"5min":3.4790000000000001,"15min":3.4790000000000001},"last":1.242}]}]},{"osd":0,"up_from":13,"seq":55834574930,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36700,"kb_used_data":5652,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94335140,"statfs":{"total":96636764160,"available":96599183360,"internally_reserved":0,"allocated":5787648,"data_stored":5236004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 02:14:27 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.62,"15min":0.60499999999999998},"min":{"1min":0.32500000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.94899999999999995,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.62,"5min":0.66600000000000004,"15min":0.64600000000000002},"min":{"1min":0.26100000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.96599999999999997,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.27700000000000002}]},{"osd":2,"last update":"Mon Nov 6 02:14:27 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.68799999999999994,"15min":0.67200000000000004},"min":{"1min":0.36599999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.0900000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.37},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.69699999999999995,"15min":0.68200000000000005},"min":{"1min":0.315,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.98199999999999998,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.315}]},{"osd":3,"last update":"Mon Nov 6 02:14:27 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.69499999999999995,"15min":0.65100000000000002},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":2.1150000000000002,"5min":2.1150000000000002,"15min":2.1150000000000002},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.78500000000000003,"15min":0.749},"min":{"1min":0.39300000000000002,"5min":0.317,"15min":0.317},"max":{"1min":1.0860000000000001,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.34300000000000003}]},{"osd":4,"last update":"Mon Nov 6 02:14:27 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.93000000000000005,"15min":0.86099999999999999},"min":{"1min":0.60099999999999998,"5min":0.55400000000000005,"15min":0.54300000000000004},"max":{"1min":2.3130000000000002,"5min":2.3130000000000002,"15min":2.3130000000000002},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.96299999999999997,"15min":0.94499999999999995},"min":{"1min":0.59599999999999997,"5min":0.5,"15min":0.5},"max":{"1min":1.3140000000000001,"5min":1.3839999999999999,"15min":1.387},"last":0.55300000000000005}]},{"osd":5,"last update":"Mon Nov 6 02:14:27 2023","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.95999999999999996,"15min":0.95199999999999996},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.52400000000000002},{"interface":"front","average":{"1min":1.006,"5min":0.96099999999999997,"15min":0.90100000000000002},"min":{"1min":0.53800000000000003,"5min":0.496,"15min":0.496},"max":{"1min":2.3530000000000002,"5min":2.3530000000000002,"15min":2.3530000000000002},"last":0.68500000000000005}]},{"osd":6,"last update":"Mon Nov 6 02:14:27 2023","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":1.0349999999999999,"15min":1.036},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.4550000000000001,"5min":1.581,"15min":1.589},"last":0.78300000000000003},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.0309999999999999,"15min":1.01},"min":{"1min":0.68100000000000005,"5min":0.65800000000000003,"15min":0.57099999999999995},"max":{"1min":1.4990000000000001,"5min":1.5529999999999999,"15min":1.6220000000000001},"last":0.79100000000000004}]},{"osd":7,"last update":"Mon Nov 6 02:14:27 2023","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":1.024,"15min":1.02},"min":{"1min":0.55300000000000005,"5min":0.504,"15min":0.504},"max":{"1min":1.357,"5min":1.504,"15min":1.504},"last":0.54400000000000004},{"interface":"front","average":{"1min":1.034,"5min":1.0029999999999999,"15min":1.0069999999999999},"min":{"1min":0.64700000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":2.387,"5min":2.387,"15min":2.387},"last":0.753}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T02:14:58.632 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T02:14:58.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.762+0000 7fd446c42640 1 Processor -- start 2023-11-06T02:14:58.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.762+0000 7fd446c42640 1 -- start start 2023-11-06T02:14:58.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.762+0000 7fd446c42640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4401646f0 0x7fd4400b0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:58.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.762+0000 7fd446c42640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd4400c8c50 0x7fd4400b05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:58.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.762+0000 7fd446c42640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd440165b80 con 0x7fd4400adb00 2023-11-06T02:14:58.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.762+0000 7fd446c42640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd440165600 con 0x7fd4400c8c50 2023-11-06T02:14:58.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.762+0000 7fd446c42640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4401658a0 con 0x7fd4401646f0 2023-11-06T02:14:58.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.762+0000 7fd43ffff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd4400c8c50 0x7fd4400b05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:58.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.762+0000 7fd43ffff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd4400c8c50 0x7fd4400b05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56310/0 (socket says 172.21.15.49:56310) 2023-11-06T02:14:58.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.762+0000 7fd43ffff640 1 -- 172.21.15.49:0/3019203609 learned_addr learned my addr 172.21.15.49:0/3019203609 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:58.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.762+0000 7fd4449de640 1 --2- 172.21.15.49:0/3019203609 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4401646f0 0x7fd4400b0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:58.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.762+0000 7fd43ffff640 1 -- 172.21.15.49:0/3019203609 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4401646f0 msgr2=0x7fd4400b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:58.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.762+0000 7fd43ffff640 1 --2- 172.21.15.49:0/3019203609 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4401646f0 0x7fd4400b0080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:58.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.762+0000 7fd43ffff640 1 -- 172.21.15.49:0/3019203609 >> v1:172.21.15.150:6789/0 conn(0x7fd4400adb00 legacy=0x7fd4400cb010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:58.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.762+0000 7fd43ffff640 1 -- 172.21.15.49:0/3019203609 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4400c96f0 con 0x7fd4400c8c50 2023-11-06T02:14:58.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.762+0000 7fd4449de640 1 --2- 172.21.15.49:0/3019203609 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4401646f0 0x7fd4400b0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:14:58.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.766+0000 7fd43ffff640 1 --2- 172.21.15.49:0/3019203609 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4400c8c50 0x7fd4400b05c0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7fd42c009870 tx=0x7fd42c01c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f27242cc0d397dcb server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:58.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.766+0000 7fd43f7fe640 1 -- 172.21.15.49:0/3019203609 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd42c0aa640 con 0x7fd4400c8c50 2023-11-06T02:14:58.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.766+0000 7fd43f7fe640 1 -- 172.21.15.49:0/3019203609 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd42c0aa7a0 con 0x7fd4400c8c50 2023-11-06T02:14:58.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.766+0000 7fd43f7fe640 1 -- 172.21.15.49:0/3019203609 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd42c0aab30 con 0x7fd4400c8c50 2023-11-06T02:14:58.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.766+0000 7fd446c42640 1 -- 172.21.15.49:0/3019203609 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4400c8c50 msgr2=0x7fd4400b05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:58.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.766+0000 7fd446c42640 1 --2- 172.21.15.49:0/3019203609 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4400c8c50 0x7fd4400b05c0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7fd42c009870 tx=0x7fd42c01c500 comp rx=0 tx=0).stop 2023-11-06T02:14:58.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.766+0000 7fd446c42640 1 -- 172.21.15.49:0/3019203609 shutdown_connections 2023-11-06T02:14:58.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.766+0000 7fd446c42640 1 --2- 172.21.15.49:0/3019203609 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4400c8c50 0x7fd4400b05c0 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:58.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.766+0000 7fd446c42640 1 --2- 172.21.15.49:0/3019203609 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4401646f0 0x7fd4400b0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:58.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.766+0000 7fd446c42640 1 -- 172.21.15.49:0/3019203609 >> 172.21.15.49:0/3019203609 conn(0x7fd44006c0a0 msgr2=0x7fd44006c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:58.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.766+0000 7fd446c42640 1 -- 172.21.15.49:0/3019203609 shutdown_connections 2023-11-06T02:14:58.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.766+0000 7fd446c42640 1 -- 172.21.15.49:0/3019203609 wait complete. 2023-11-06T02:14:58.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.766+0000 7fd446c42640 1 Processor -- start 2023-11-06T02:14:58.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.766+0000 7fd446c42640 1 -- start start 2023-11-06T02:14:58.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.766+0000 7fd446c42640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4400adb00 0x7fd4400c56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:58.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.766+0000 7fd446c42640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4400c8c50 0x7fd4401400d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:58.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.766+0000 7fd446c42640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4401646f0 0x7fd440140610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:58.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.766+0000 7fd446c42640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd440154cd0 con 0x7fd4400adb00 2023-11-06T02:14:58.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.766+0000 7fd446c42640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd440158f90 con 0x7fd4401646f0 2023-11-06T02:14:58.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.766+0000 7fd446c42640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd440158310 con 0x7fd4400c8c50 2023-11-06T02:14:58.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.766+0000 7fd4449de640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4400adb00 0x7fd4400c56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:58.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.766+0000 7fd4449de640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4400adb00 0x7fd4400c56d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56316/0 (socket says 172.21.15.49:56316) 2023-11-06T02:14:58.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.766+0000 7fd4449de640 1 -- 172.21.15.49:0/3042495332 learned_addr learned my addr 172.21.15.49:0/3042495332 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:58.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.770+0000 7fd43ffff640 1 --2- 172.21.15.49:0/3042495332 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4400c8c50 0x7fd4401400d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:58.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.770+0000 7fd4451df640 1 --2- 172.21.15.49:0/3042495332 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4401646f0 0x7fd440140610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:58.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.770+0000 7fd4449de640 1 -- 172.21.15.49:0/3042495332 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4400c8c50 msgr2=0x7fd4401400d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:58.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.770+0000 7fd4449de640 1 --2- 172.21.15.49:0/3042495332 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4400c8c50 0x7fd4401400d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:58.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.770+0000 7fd4449de640 1 -- 172.21.15.49:0/3042495332 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4401646f0 msgr2=0x7fd440140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:58.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.770+0000 7fd4449de640 1 --2- 172.21.15.49:0/3042495332 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4401646f0 0x7fd440140610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:58.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.770+0000 7fd4449de640 1 -- 172.21.15.49:0/3042495332 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd42c0094d0 con 0x7fd4400adb00 2023-11-06T02:14:58.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.770+0000 7fd4451df640 1 --2- 172.21.15.49:0/3042495332 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4401646f0 0x7fd440140610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:14:58.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.770+0000 7fd4449de640 1 --2- 172.21.15.49:0/3042495332 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4400adb00 0x7fd4400c56d0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7fd43400c7f0 tx=0x7fd43400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:58.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.770+0000 7fd43d7fa640 1 -- 172.21.15.49:0/3042495332 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd434005ba0 con 0x7fd4400adb00 2023-11-06T02:14:58.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.770+0000 7fd43d7fa640 1 -- 172.21.15.49:0/3042495332 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd434005d00 con 0x7fd4400adb00 2023-11-06T02:14:58.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.770+0000 7fd446c42640 1 -- 172.21.15.49:0/3042495332 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4401585f0 con 0x7fd4400adb00 2023-11-06T02:14:58.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.770+0000 7fd43d7fa640 1 -- 172.21.15.49:0/3042495332 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd434007a50 con 0x7fd4400adb00 2023-11-06T02:14:58.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.770+0000 7fd446c42640 1 -- 172.21.15.49:0/3042495332 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd440158bb0 con 0x7fd4400adb00 2023-11-06T02:14:58.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.770+0000 7fd446c42640 1 -- 172.21.15.49:0/3042495332 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4401590d0 con 0x7fd4400adb00 2023-11-06T02:14:58.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.770+0000 7fd43d7fa640 1 -- 172.21.15.49:0/3042495332 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fd434007bb0 con 0x7fd4400adb00 2023-11-06T02:14:58.776 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.770+0000 7fd43d7fa640 1 --2- 172.21.15.49:0/3042495332 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd424065a60 0x7fd424067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:58.776 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.770+0000 7fd43d7fa640 1 -- 172.21.15.49:0/3042495332 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fd434083090 con 0x7fd4400adb00 2023-11-06T02:14:58.776 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.770+0000 7fd43ffff640 1 --2- 172.21.15.49:0/3042495332 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd424065a60 0x7fd424067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:58.777 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.770+0000 7fd43ffff640 1 --2- 172.21.15.49:0/3042495332 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd424065a60 0x7fd424067f20 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fd42c0ae000 tx=0x7fd42c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:58.777 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.774+0000 7fd43d7fa640 1 -- 172.21.15.49:0/3042495332 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fd4401590d0 con 0x7fd4400adb00 2023-11-06T02:14:58.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.926+0000 7fd446c42640 1 -- 172.21.15.49:0/3042495332 --> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd440074040 con 0x7fd424065a60 2023-11-06T02:14:58.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.934+0000 7fd43d7fa640 1 -- 172.21.15.49:0/3042495332 <== mgr.4100 v2:172.21.15.150:6832/38104 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+71133 (secure 0 0 0) 0x7fd440074040 con 0x7fd424065a60 2023-11-06T02:14:58.929 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:58.930 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2023-11-06T02:14:58.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.934+0000 7fd446c42640 1 -- 172.21.15.49:0/3042495332 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd424065a60 msgr2=0x7fd424067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:58.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.934+0000 7fd446c42640 1 --2- 172.21.15.49:0/3042495332 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd424065a60 0x7fd424067f20 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fd42c0ae000 tx=0x7fd42c008040 comp rx=0 tx=0).stop 2023-11-06T02:14:58.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.934+0000 7fd446c42640 1 -- 172.21.15.49:0/3042495332 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4400adb00 msgr2=0x7fd4400c56d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:58.932 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.934+0000 7fd446c42640 1 --2- 172.21.15.49:0/3042495332 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4400adb00 0x7fd4400c56d0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7fd43400c7f0 tx=0x7fd43400cbf0 comp rx=0 tx=0).stop 2023-11-06T02:14:58.932 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.934+0000 7fd446c42640 1 -- 172.21.15.49:0/3042495332 shutdown_connections 2023-11-06T02:14:58.932 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.934+0000 7fd446c42640 1 --2- 172.21.15.49:0/3042495332 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fd424065a60 0x7fd424067f20 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:58.932 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.934+0000 7fd446c42640 1 --2- 172.21.15.49:0/3042495332 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4401646f0 0x7fd440140610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:58.932 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.934+0000 7fd446c42640 1 --2- 172.21.15.49:0/3042495332 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4400c8c50 0x7fd4401400d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:58.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.934+0000 7fd446c42640 1 --2- 172.21.15.49:0/3042495332 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4400adb00 0x7fd4400c56d0 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:58.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.934+0000 7fd446c42640 1 -- 172.21.15.49:0/3042495332 >> 172.21.15.49:0/3042495332 conn(0x7fd44006c0a0 msgr2=0x7fd4400ce590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:58.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.938+0000 7fd446c42640 1 -- 172.21.15.49:0/3042495332 shutdown_connections 2023-11-06T02:14:58.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:58.938+0000 7fd446c42640 1 -- 172.21.15.49:0/3042495332 wait complete. 2023-11-06T02:14:58.948 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":278,"stamp":"2023-11-06T02:14:57.316013+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":269724,"kb_used_data":25436,"kb_used_omap":0,"kb_used_meta":244224,"kb_avail":754704996,"statfs":{"total":773094113280,"available":772817915904,"internally_reserved":0,"allocated":26046464,"data_stored":21579923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":250085376},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8390586,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-258,"num_read_kb":-264,"num_write":-177,"num_write_kb":-8295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002028"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":221,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T02:14:23.281538+0000","last_change":"2023-11-06T02:08:20.523427+0000","last_active":"2023-11-06T02:14:23.281538+0000","last_peered":"2023-11-06T02:14:23.281538+0000","last_clean":"2023-11-06T02:14:23.281538+0000","last_became_active":"2023-11-06T02:08:20.521089+0000","last_became_peered":"2023-11-06T02:08:20.521089+0000","last_unstale":"2023-11-06T02:14:23.281538+0000","last_undegraded":"2023-11-06T02:14:23.281538+0000","last_fullsized":"2023-11-06T02:14:23.281538+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:19.252348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:19.252348+0000","last_clean_scrub_stamp":"2023-11-06T02:08:19.252348+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:30:15.566397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":67,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T02:14:35.449337+0000","last_change":"2023-11-06T02:08:26.059502+0000","last_active":"2023-11-06T02:14:35.449337+0000","last_peered":"2023-11-06T02:14:35.449337+0000","last_clean":"2023-11-06T02:14:35.449337+0000","last_became_active":"2023-11-06T02:08:26.058612+0000","last_became_peered":"2023-11-06T02:08:26.058612+0000","last_unstale":"2023-11-06T02:14:35.449337+0000","last_undegraded":"2023-11-06T02:14:35.449337+0000","last_fullsized":"2023-11-06T02:14:35.449337+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:10.626021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":68,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T02:14:28.282679+0000","last_change":"2023-11-06T02:08:26.060406+0000","last_active":"2023-11-06T02:14:28.282679+0000","last_peered":"2023-11-06T02:14:28.282679+0000","last_clean":"2023-11-06T02:14:28.282679+0000","last_became_active":"2023-11-06T02:08:26.059262+0000","last_became_peered":"2023-11-06T02:08:26.059262+0000","last_unstale":"2023-11-06T02:14:28.282679+0000","last_undegraded":"2023-11-06T02:14:28.282679+0000","last_fullsized":"2023-11-06T02:14:28.282679+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:10.626021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":68,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T02:14:35.447393+0000","last_change":"2023-11-06T02:08:26.559477+0000","last_active":"2023-11-06T02:14:35.447393+0000","last_peered":"2023-11-06T02:14:35.447393+0000","last_clean":"2023-11-06T02:14:35.447393+0000","last_became_active":"2023-11-06T02:08:26.559202+0000","last_became_peered":"2023-11-06T02:08:26.559202+0000","last_unstale":"2023-11-06T02:14:35.447393+0000","last_undegraded":"2023-11-06T02:14:35.447393+0000","last_fullsized":"2023-11-06T02:14:35.447393+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:55:04.552404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":67,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T02:14:35.448967+0000","last_change":"2023-11-06T02:08:26.059546+0000","last_active":"2023-11-06T02:14:35.448967+0000","last_peered":"2023-11-06T02:14:35.448967+0000","last_clean":"2023-11-06T02:14:35.448967+0000","last_became_active":"2023-11-06T02:08:26.058806+0000","last_became_peered":"2023-11-06T02:08:26.058806+0000","last_unstale":"2023-11-06T02:14:35.448967+0000","last_undegraded":"2023-11-06T02:14:35.448967+0000","last_fullsized":"2023-11-06T02:14:35.448967+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:23.468510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":67,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T02:14:35.439334+0000","last_change":"2023-11-06T02:08:26.436463+0000","last_active":"2023-11-06T02:14:35.439334+0000","last_peered":"2023-11-06T02:14:35.439334+0000","last_clean":"2023-11-06T02:14:35.439334+0000","last_became_active":"2023-11-06T02:08:26.436075+0000","last_became_peered":"2023-11-06T02:08:26.436075+0000","last_unstale":"2023-11-06T02:14:35.439334+0000","last_undegraded":"2023-11-06T02:14:35.439334+0000","last_fullsized":"2023-11-06T02:14:35.439334+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:23.468510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":68,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T02:14:35.357267+0000","last_change":"2023-11-06T02:08:26.436607+0000","last_active":"2023-11-06T02:14:35.357267+0000","last_peered":"2023-11-06T02:14:35.357267+0000","last_clean":"2023-11-06T02:14:35.357267+0000","last_became_active":"2023-11-06T02:08:26.435924+0000","last_became_peered":"2023-11-06T02:08:26.435924+0000","last_unstale":"2023-11-06T02:14:35.357267+0000","last_undegraded":"2023-11-06T02:14:35.357267+0000","last_fullsized":"2023-11-06T02:14:35.357267+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:23.468510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":66,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T02:14:13.290705+0000","last_change":"2023-11-06T02:08:26.057076+0000","last_active":"2023-11-06T02:14:13.290705+0000","last_peered":"2023-11-06T02:14:13.290705+0000","last_clean":"2023-11-06T02:14:13.290705+0000","last_became_active":"2023-11-06T02:08:26.056419+0000","last_became_peered":"2023-11-06T02:08:26.056419+0000","last_unstale":"2023-11-06T02:14:13.290705+0000","last_undegraded":"2023-11-06T02:14:13.290705+0000","last_fullsized":"2023-11-06T02:14:13.290705+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:10.626021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":75,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T02:14:13.297179+0000","last_change":"2023-11-06T02:08:26.058596+0000","last_active":"2023-11-06T02:14:13.297179+0000","last_peered":"2023-11-06T02:14:13.297179+0000","last_clean":"2023-11-06T02:14:13.297179+0000","last_became_active":"2023-11-06T02:08:26.057015+0000","last_became_peered":"2023-11-06T02:08:26.057015+0000","last_unstale":"2023-11-06T02:14:13.297179+0000","last_undegraded":"2023-11-06T02:14:13.297179+0000","last_fullsized":"2023-11-06T02:14:13.297179+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:55:04.552404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574930,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32608,"kb_used_data":1560,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339232,"statfs":{"total":96636764160,"available":96603373568,"internally_reserved":0,"allocated":1597440,"data_stored":1042734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.87,"15min":0.85599999999999998},"min":{"1min":0.62,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.147,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.52700000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.2230000000000001,"5min":1.421,"15min":1.506},"last":1.0089999999999999}]},{"osd":1,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.83399999999999996,"15min":0.84499999999999997},"min":{"1min":0.498,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.3320000000000001,"5min":1.345,"15min":1.345},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.92900000000000005,"15min":0.90200000000000002},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.54900000000000004},"max":{"1min":1.131,"5min":1.681,"15min":1.681},"last":0.97399999999999998}]},{"osd":2,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.96399999999999997,"15min":0.95699999999999996},"min":{"1min":0.65300000000000002,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.1919999999999999,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":1.133},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.92200000000000004,"15min":0.97399999999999998},"min":{"1min":0.57399999999999995,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.1919999999999999,"5min":1.5,"15min":1.5920000000000001},"last":0.74199999999999999}]},{"osd":3,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.89200000000000002,"15min":0.88},"min":{"1min":0.66100000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.1519999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":1.234},{"interface":"front","average":{"1min":0.995,"5min":1.03,"15min":1.0609999999999999},"min":{"1min":0.74399999999999999,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.2110000000000001,"5min":1.526,"15min":1.526},"last":0.92200000000000004}]},{"osd":4,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.91700000000000004,"15min":0.94399999999999995},"min":{"1min":0.73399999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.1930000000000001,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.97999999999999998,"15min":1.0229999999999999},"min":{"1min":0.65000000000000002,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.1519999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":1.0640000000000001}]},{"osd":5,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":1.026,"15min":1.0609999999999999},"min":{"1min":0.69199999999999995,"5min":0.503,"15min":0.503},"max":{"1min":1.2629999999999999,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":1.0389999999999999},{"interface":"front","average":{"1min":1.044,"5min":1.044,"15min":1.105},"min":{"1min":0.71299999999999997,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.288,"5min":1.3700000000000001,"15min":1.7809999999999999},"last":0.94999999999999996}]},{"osd":6,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":1.1180000000000001,"15min":1.1739999999999999},"min":{"1min":0.89500000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.274,"5min":1.5009999999999999,"15min":1.6679999999999999},"last":1.113},{"interface":"front","average":{"1min":1.028,"5min":1.0649999999999999,"15min":1.127},"min":{"1min":0.72599999999999998,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.2509999999999999,"5min":1.4570000000000001,"15min":1.74},"last":1.1579999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574929,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36136,"kb_used_data":5088,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94335704,"statfs":{"total":96636764160,"available":96599760896,"internally_reserved":0,"allocated":5210112,"data_stored":4647957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.872,"15min":0.86199999999999999},"min":{"1min":0.57199999999999995,"5min":0.495,"15min":0.495},"max":{"1min":1.226,"5min":1.226,"15min":1.2609999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.88600000000000001,"15min":0.83199999999999996},"min":{"1min":0.5,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.3280000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.68899999999999995}]},{"osd":1,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.85099999999999998,"15min":0.86699999999999999},"min":{"1min":0.54400000000000004,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.1639999999999999,"5min":1.244,"15min":1.244},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.90500000000000003,"15min":0.88400000000000001},"min":{"1min":0.56599999999999995,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":0.82599999999999996}]},{"osd":2,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.89200000000000002,"15min":0.89600000000000002},"min":{"1min":0.61499999999999999,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.877,"5min":0.88900000000000001,"15min":0.91700000000000004},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.075,"5min":1.411,"15min":1.411},"last":0.96199999999999997}]},{"osd":3,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.93500000000000005,"15min":0.95899999999999996},"min":{"1min":0.74099999999999999,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.92300000000000004},{"interface":"front","average":{"1min":1.0309999999999999,"5min":0.999,"15min":1},"min":{"1min":0.81599999999999995,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.2609999999999999,"5min":1.381,"15min":1.381},"last":1.0149999999999999}]},{"osd":4,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.92200000000000004,"15min":0.96999999999999997},"min":{"1min":0.627,"5min":0.56799999999999995,"15min":0.52900000000000003},"max":{"1min":1.2390000000000001,"5min":1.302,"15min":1.474},"last":0.995},{"interface":"front","average":{"1min":0.997,"5min":0.97399999999999998,"15min":1.018},"min":{"1min":0.66700000000000004,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.4039999999999999,"5min":1.556,"15min":1.556},"last":1.232}]},{"osd":5,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":0.97199999999999998,"15min":1.0089999999999999},"min":{"1min":0.68899999999999995,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":1.1950000000000001},{"interface":"front","average":{"1min":1.1319999999999999,"5min":1.075,"15min":1.101},"min":{"1min":0.82699999999999996,"5min":0.66900000000000004,"15min":0.58699999999999997},"max":{"1min":1.3999999999999999,"5min":1.587,"15min":1.587},"last":1.069}]},{"osd":7,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.087,"15min":1.107},"min":{"1min":0.88300000000000001,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.5369999999999999,"5min":1.615,"15min":1.615},"last":1.377},{"interface":"front","average":{"1min":1.103,"5min":1.0509999999999999,"15min":1.127},"min":{"1min":0.85599999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.4610000000000001,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":1.284}]}]},{"osd":5,"up_from":13,"seq":55834574930,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27940,"kb_used_data":988,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343900,"statfs":{"total":96636764160,"available":96608153600,"internally_reserved":0,"allocated":1011712,"data_stored":452390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:14:33 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.86199999999999999,"15min":0.83999999999999997},"min":{"1min":0.54400000000000004,"5min":0.51700000000000002,"15min":0.47299999999999998},"max":{"1min":1.079,"5min":1.381,"15min":1.381},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.86899999999999999,"15min":0.85099999999999998},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.159,"5min":1.3200000000000001,"15min":1.6579999999999999},"last":0.56000000000000005}]},{"osd":1,"last update":"Mon Nov 6 02:14:33 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.90500000000000003,"15min":0.86799999999999999},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.60299999999999998},"max":{"1min":1.369,"5min":1.369,"15min":1.369},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.90000000000000002,"15min":0.84399999999999997},"min":{"1min":0.53700000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.1639999999999999,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.69199999999999995}]},{"osd":2,"last update":"Mon Nov 6 02:14:33 2023","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.93899999999999995,"15min":0.94299999999999995},"min":{"1min":0.65100000000000002,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.242,"5min":1.298,"15min":1.3560000000000001},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.89700000000000002,"15min":0.82499999999999996},"min":{"1min":0.58699999999999997,"5min":0.55800000000000005,"15min":0.47899999999999998},"max":{"1min":1.218,"5min":1.323,"15min":1.377},"last":0.76500000000000001}]},{"osd":3,"last update":"Mon Nov 6 02:14:33 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.95199999999999996,"15min":0.93000000000000005},"min":{"1min":0.66400000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.292,"5min":1.3580000000000001,"15min":1.4399999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":1.0369999999999999,"5min":0.98799999999999999,"15min":1.05},"min":{"1min":0.76700000000000002,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.2569999999999999,"5min":1.349,"15min":1.7250000000000001},"last":0.99199999999999999}]},{"osd":4,"last update":"Mon Nov 6 02:14:33 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.87,"15min":0.90000000000000002},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.0129999999999999,"5min":1.276,"15min":1.276},"last":0.94799999999999995},{"interface":"front","average":{"1min":0.995,"5min":0.98599999999999999,"15min":1.014},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.496,"5min":1.496,"15min":1.554},"last":0.90300000000000002}]},{"osd":6,"last update":"Mon Nov 6 02:14:33 2023","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.98699999999999999,"15min":1.0409999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.3440000000000001,"5min":1.454,"15min":1.454},"last":1.0640000000000001},{"interface":"front","average":{"1min":1.046,"5min":1.0229999999999999,"15min":1.0349999999999999},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":2.2309999999999999,"5min":2.2309999999999999,"15min":2.2309999999999999},"last":1.02}]},{"osd":7,"last update":"Mon Nov 6 02:14:33 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.0640000000000001,"15min":1.0720000000000001},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.5209999999999999,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":0.83799999999999997},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.117,"15min":1.1419999999999999},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":2.3759999999999999,"5min":2.3759999999999999,"15min":2.3759999999999999},"last":1.153}]}]},{"osd":4,"up_from":13,"seq":55834574930,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32044,"kb_used_data":996,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339796,"statfs":{"total":96636764160,"available":96603951104,"internally_reserved":0,"allocated":1019904,"data_stored":453752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:14:27 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.86399999999999999,"15min":0.83799999999999997},"min":{"1min":0.53500000000000003,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.2450000000000001,"5min":1.343,"15min":1.343},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.84499999999999997,"15min":0.79500000000000004},"min":{"1min":0.51600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.175,"5min":1.28,"15min":1.3049999999999999},"last":0.85899999999999999}]},{"osd":1,"last update":"Mon Nov 6 02:14:27 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.86399999999999999,"15min":0.84999999999999998},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.508,"5min":1.508,"15min":1.508},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.91800000000000004,"15min":0.90000000000000002},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.1080000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.80100000000000005}]},{"osd":2,"last update":"Mon Nov 6 02:14:27 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.90800000000000003,"15min":0.98199999999999998},"min":{"1min":0.57999999999999996,"5min":0.503,"15min":0.503},"max":{"1min":1.0880000000000001,"5min":1.389,"15min":1.615},"last":1.159},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.95499999999999996,"15min":0.93500000000000005},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.57899999999999996},"max":{"1min":1.1559999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.92400000000000004}]},{"osd":3,"last update":"Mon Nov 6 02:14:27 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.86199999999999999,"15min":0.90000000000000002},"min":{"1min":0.59899999999999998,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.22,"5min":1.22,"15min":1.52},"last":0.745},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.97299999999999998,"15min":1.0369999999999999},"min":{"1min":0.68100000000000005,"5min":0.63900000000000001,"15min":0.59999999999999998},"max":{"1min":1.589,"5min":1.595,"15min":1.595},"last":1.0509999999999999}]},{"osd":5,"last update":"Mon Nov 6 02:14:27 2023","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":1.026},"min":{"1min":0.72599999999999998,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.212,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":1.1200000000000001},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.96099999999999997,"15min":1.024},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.629},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.7869999999999999},"last":1.0960000000000001}]},{"osd":6,"last update":"Mon Nov 6 02:14:27 2023","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.008,"15min":1.0649999999999999},"min":{"1min":0.77600000000000002,"5min":0.68000000000000005,"15min":0.67100000000000004},"max":{"1min":1.6679999999999999,"5min":1.6679999999999999,"15min":1.6699999999999999},"last":1.216},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.085,"15min":1.1599999999999999},"min":{"1min":0.71999999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.2210000000000001,"5min":1.5900000000000001,"15min":2.1190000000000002},"last":1.004}]},{"osd":7,"last update":"Mon Nov 6 02:14:27 2023","interfaces":[{"interface":"back","average":{"1min":1.056,"5min":1.042,"15min":1.0760000000000001},"min":{"1min":0.77100000000000002,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.7110000000000001},"last":0.94699999999999995},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.117,"15min":1.1990000000000001},"min":{"1min":0.90500000000000003,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.754,"5min":1.754,"15min":1.754},"last":1.2629999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574929,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32032,"kb_used_data":984,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339808,"statfs":{"total":96636764160,"available":96603963392,"internally_reserved":0,"allocated":1007616,"data_stored":452931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63,"15min":0.64200000000000002},"min":{"1min":0.45300000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":1.171,"5min":1.198,"15min":1.198},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.65400000000000003,"15min":0.63500000000000001},"min":{"1min":0.49099999999999999,"5min":0.29899999999999999,"15min":0.29799999999999999},"max":{"1min":1.1819999999999999,"5min":1.21,"15min":1.21},"last":0.35199999999999998}]},{"osd":1,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.61599999999999999,"15min":0.626},"min":{"1min":0.42499999999999999,"5min":0.248,"15min":0.248},"max":{"1min":0.86799999999999999,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.373},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.75800000000000001,"15min":0.74199999999999999},"min":{"1min":0.46899999999999997,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.175,"5min":1.204,"15min":1.204},"last":0.53000000000000003}]},{"osd":2,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.72499999999999998,"15min":0.73099999999999998},"min":{"1min":0.499,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.438},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.81299999999999994,"15min":0.81999999999999995},"min":{"1min":0.62,"5min":0.308,"15min":0.308},"max":{"1min":1.0329999999999999,"5min":1.405,"15min":1.405},"last":0.501}]},{"osd":4,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.62,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.262,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.93700000000000006,"15min":0.97799999999999998},"min":{"1min":0.70799999999999996,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.212,"5min":1.292,"15min":1.4410000000000001},"last":0.66300000000000003}]},{"osd":5,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.97499999999999998,"15min":1.0109999999999999},"min":{"1min":0.72699999999999998,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.21,"5min":1.375,"15min":1.375},"last":0.71999999999999997},{"interface":"front","average":{"1min":1.024,"5min":1.0269999999999999,"15min":1.0629999999999999},"min":{"1min":0.65200000000000002,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.323,"5min":1.4890000000000001,"15min":2.21},"last":0.81799999999999995}]},{"osd":6,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.0209999999999999,"15min":1.024},"min":{"1min":0.79800000000000004,"5min":0.439,"15min":0.439},"max":{"1min":1.3080000000000001,"5min":1.395,"15min":1.395},"last":0.90000000000000002},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.0269999999999999,"15min":1.0569999999999999},"min":{"1min":0.74099999999999999,"5min":0.375,"15min":0.375},"max":{"1min":1.2709999999999999,"5min":1.4319999999999999,"15min":1.554},"last":0.83099999999999996}]},{"osd":7,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.1020000000000001,"15min":1.1220000000000001},"min":{"1min":0.68200000000000005,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.4410000000000001,"5min":1.5549999999999999,"15min":2.2829999999999999},"last":0.93100000000000005},{"interface":"front","average":{"1min":1.123,"5min":1.0980000000000001,"15min":1.1040000000000001},"min":{"1min":0.91100000000000003,"5min":0.497,"15min":0.497},"max":{"1min":1.3620000000000001,"5min":1.504,"15min":1.504},"last":0.96799999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574929,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36132,"kb_used_data":5084,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94335708,"statfs":{"total":96636764160,"available":96599764992,"internally_reserved":0,"allocated":5206016,"data_stored":4647390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:14:28 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.59099999999999997,"15min":0.52900000000000003},"min":{"1min":0.28499999999999998,"5min":0.252,"15min":0.252},"max":{"1min":0.90300000000000002,"5min":1.079,"15min":1.079},"last":0.752},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.63400000000000001,"15min":0.56999999999999995},"min":{"1min":0.374,"5min":0.316,"15min":0.27300000000000002},"max":{"1min":0.86699999999999999,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":0.64500000000000002}]},{"osd":1,"last update":"Mon Nov 6 02:14:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.65000000000000002,"15min":0.58999999999999997},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.26400000000000001},"max":{"1min":0.97699999999999998,"5min":1.48,"15min":1.48},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.72799999999999998,"15min":0.621},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.26500000000000001},"max":{"1min":1.04,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":0.42599999999999999}]},{"osd":3,"last update":"Mon Nov 6 02:14:28 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.71399999999999997,"15min":0.63600000000000001},"min":{"1min":0.316,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.0580000000000001,"5min":1.121,"15min":1.155},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.76400000000000001,"15min":0.68200000000000005},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.069,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.94099999999999995}]},{"osd":4,"last update":"Mon Nov 6 02:14:28 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.93100000000000005,"15min":0.84299999999999997},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.1539999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.92900000000000005,"15min":0.91900000000000004},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.234,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":0.96999999999999997}]},{"osd":5,"last update":"Mon Nov 6 02:14:28 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.96299999999999997,"15min":0.94899999999999995},"min":{"1min":0.56299999999999994,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.258,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.97299999999999998,"15min":0.94699999999999995},"min":{"1min":0.626,"5min":0.60999999999999999,"15min":0.52600000000000002},"max":{"1min":1.2310000000000001,"5min":1.615,"15min":1.7110000000000001},"last":1.038}]},{"osd":6,"last update":"Mon Nov 6 02:14:28 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.96899999999999997,"15min":0.94799999999999995},"min":{"1min":0.60199999999999998,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.218,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.93600000000000005,"5min":1.01,"15min":1.0009999999999999},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.274,"5min":1.675,"15min":1.677},"last":0.78200000000000003}]},{"osd":7,"last update":"Mon Nov 6 02:14:28 2023","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":1.0309999999999999,"15min":0.99299999999999999},"min":{"1min":0.67500000000000004,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.2869999999999999,"5min":1.659,"15min":1.659},"last":1.0649999999999999},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.0760000000000001,"15min":1.0529999999999999},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.1879999999999999,"5min":1.4650000000000001,"15min":1.5980000000000001},"last":1.0169999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574930,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36132,"kb_used_data":5084,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94335708,"statfs":{"total":96636764160,"available":96599764992,"internally_reserved":0,"allocated":5206016,"data_stored":4646765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:14:28 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.628,"15min":0.626},"min":{"1min":0.29599999999999999,"5min":0.28699999999999998,"15min":0.224},"max":{"1min":0.95099999999999996,"5min":1.603,"15min":1.603},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.71999999999999997,"15min":0.67400000000000004},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.081,"5min":2.161,"15min":2.161},"last":0.58699999999999997}]},{"osd":2,"last update":"Mon Nov 6 02:14:28 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.64700000000000002,"15min":0.63},"min":{"1min":0.34399999999999997,"5min":0.253,"15min":0.253},"max":{"1min":1.036,"5min":1.649,"15min":1.649},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.74199999999999999,"15min":0.74399999999999999},"min":{"1min":0.373,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.91700000000000004,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.90700000000000003}]},{"osd":3,"last update":"Mon Nov 6 02:14:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.80400000000000005,"15min":0.80300000000000005},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.131,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":0.97999999999999998},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.79100000000000004,"15min":0.80400000000000005},"min":{"1min":0.35499999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.94199999999999995,"5min":1.5,"15min":1.8149999999999999},"last":0.77800000000000002}]},{"osd":4,"last update":"Mon Nov 6 02:14:28 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.96199999999999997,"15min":0.98199999999999998},"min":{"1min":0.55600000000000005,"5min":0.504,"15min":0.504},"max":{"1min":1.2110000000000001,"5min":2.2759999999999998,"15min":2.2759999999999998},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.95599999999999996,"15min":0.98099999999999998},"min":{"1min":0.63300000000000001,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.0700000000000001,"5min":2.9180000000000001,"15min":2.9180000000000001},"last":1.0329999999999999}]},{"osd":5,"last update":"Mon Nov 6 02:14:28 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.97199999999999998,"15min":0.96799999999999997},"min":{"1min":0.65800000000000003,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.177,"5min":2.7469999999999999,"15min":2.7469999999999999},"last":1.2030000000000001},{"interface":"front","average":{"1min":0.98399999999999999,"5min":1.05,"15min":1.069},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.52100000000000002},"max":{"1min":1.3720000000000001,"5min":1.921,"15min":1.921},"last":1.1870000000000001}]},{"osd":6,"last update":"Mon Nov 6 02:14:28 2023","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":1.0960000000000001,"15min":1.1000000000000001},"min":{"1min":0.61899999999999999,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.304,"5min":3.5710000000000002,"15min":3.5710000000000002},"last":1.085},{"interface":"front","average":{"1min":0.94999999999999996,"5min":1.0369999999999999,"15min":1.0629999999999999},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.268,"5min":2.843,"15min":2.843},"last":1.304}]},{"osd":7,"last update":"Mon Nov 6 02:14:28 2023","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":1.0980000000000001,"15min":1.1140000000000001},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.4159999999999999,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":1.133},{"interface":"front","average":{"1min":1.052,"5min":1.1419999999999999,"15min":1.1539999999999999},"min":{"1min":0.61799999999999999,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.3520000000000001,"5min":3.4790000000000001,"15min":3.4790000000000001},"last":1.242}]}]},{"osd":0,"up_from":13,"seq":55834574930,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36700,"kb_used_data":5652,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94335140,"statfs":{"total":96636764160,"available":96599183360,"internally_reserved":0,"allocated":5787648,"data_stored":5236004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 02:14:27 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.62,"15min":0.60499999999999998},"min":{"1min":0.32500000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.94899999999999995,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.62,"5min":0.66600000000000004,"15min":0.64600000000000002},"min":{"1min":0.26100000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.96599999999999997,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.27700000000000002}]},{"osd":2,"last update":"Mon Nov 6 02:14:27 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.68799999999999994,"15min":0.67200000000000004},"min":{"1min":0.36599999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.0900000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.37},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.69699999999999995,"15min":0.68200000000000005},"min":{"1min":0.315,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.98199999999999998,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.315}]},{"osd":3,"last update":"Mon Nov 6 02:14:27 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.69499999999999995,"15min":0.65100000000000002},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":2.1150000000000002,"5min":2.1150000000000002,"15min":2.1150000000000002},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.78500000000000003,"15min":0.749},"min":{"1min":0.39300000000000002,"5min":0.317,"15min":0.317},"max":{"1min":1.0860000000000001,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.34300000000000003}]},{"osd":4,"last update":"Mon Nov 6 02:14:27 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.93000000000000005,"15min":0.86099999999999999},"min":{"1min":0.60099999999999998,"5min":0.55400000000000005,"15min":0.54300000000000004},"max":{"1min":2.3130000000000002,"5min":2.3130000000000002,"15min":2.3130000000000002},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.96299999999999997,"15min":0.94499999999999995},"min":{"1min":0.59599999999999997,"5min":0.5,"15min":0.5},"max":{"1min":1.3140000000000001,"5min":1.3839999999999999,"15min":1.387},"last":0.55300000000000005}]},{"osd":5,"last update":"Mon Nov 6 02:14:27 2023","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.95999999999999996,"15min":0.95199999999999996},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.52400000000000002},{"interface":"front","average":{"1min":1.006,"5min":0.96099999999999997,"15min":0.90100000000000002},"min":{"1min":0.53800000000000003,"5min":0.496,"15min":0.496},"max":{"1min":2.3530000000000002,"5min":2.3530000000000002,"15min":2.3530000000000002},"last":0.68500000000000005}]},{"osd":6,"last update":"Mon Nov 6 02:14:27 2023","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":1.0349999999999999,"15min":1.036},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.4550000000000001,"5min":1.581,"15min":1.589},"last":0.78300000000000003},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.0309999999999999,"15min":1.01},"min":{"1min":0.68100000000000005,"5min":0.65800000000000003,"15min":0.57099999999999995},"max":{"1min":1.4990000000000001,"5min":1.5529999999999999,"15min":1.6220000000000001},"last":0.79100000000000004}]},{"osd":7,"last update":"Mon Nov 6 02:14:27 2023","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":1.024,"15min":1.02},"min":{"1min":0.55300000000000005,"5min":0.504,"15min":0.504},"max":{"1min":1.357,"5min":1.504,"15min":1.504},"last":0.54400000000000004},{"interface":"front","average":{"1min":1.034,"5min":1.0029999999999999,"15min":1.0069999999999999},"min":{"1min":0.64700000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":2.387,"5min":2.387,"15min":2.387},"last":0.753}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T02:14:58.950 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-11-06T02:14:58.950 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T02:14:59.067 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.074+0000 7f69045a5640 1 Processor -- start 2023-11-06T02:14:59.068 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.074+0000 7f69045a5640 1 -- start start 2023-11-06T02:14:59.070 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.074+0000 7f69045a5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f68fc0b0790 0x7f68fc0c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:59.071 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.074+0000 7f69045a5640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68fc0cab80 con 0x7f68fc0cb010 2023-11-06T02:14:59.071 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.074+0000 7f69045a5640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68fc15af30 con 0x7f68fc15ab50 2023-11-06T02:14:59.071 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.074+0000 7f69045a5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68fc0cb5b0 con 0x7f68fc0b0790 2023-11-06T02:14:59.071 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.074+0000 7f6902b42640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f68fc0cb010 0x7f68fc0c2710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:35090/0 (socket says 172.21.15.49:35090) 2023-11-06T02:14:59.072 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.074+0000 7f6902b42640 1 -- 172.21.15.49:0/39269390 learned_addr learned my addr 172.21.15.49:0/39269390 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:14:59.072 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.074+0000 7f6901b40640 1 --2- 172.21.15.49:0/39269390 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f68fc0b0790 0x7f68fc0c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:59.072 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.074+0000 7f690133f640 1 -- 172.21.15.49:0/39269390 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1969464531 0 0) 0x7f68fc0cab80 con 0x7f68fc0cb010 2023-11-06T02:14:59.072 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.074+0000 7f690133f640 1 -- 172.21.15.49:0/39269390 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68e8003530 con 0x7f68fc0cb010 2023-11-06T02:14:59.072 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.074+0000 7f690133f640 1 -- 172.21.15.49:0/39269390 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2997374815 0 0) 0x7f68e8003530 con 0x7f68fc0cb010 2023-11-06T02:14:59.073 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.074+0000 7f690133f640 1 -- 172.21.15.49:0/39269390 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f68fc0b0790 msgr2=0x7f68fc0c9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:59.073 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.074+0000 7f690133f640 1 --2- 172.21.15.49:0/39269390 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f68fc0b0790 0x7f68fc0c9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:59.073 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.074+0000 7f690133f640 1 -- 172.21.15.49:0/39269390 >> v1:172.21.15.150:6789/0 conn(0x7f68fc15ab50 legacy=0x7f68fc0b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:59.073 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.074+0000 7f690133f640 1 -- 172.21.15.49:0/39269390 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68fc0cb890 con 0x7f68fc0cb010 2023-11-06T02:14:59.073 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.074+0000 7f690133f640 1 -- 172.21.15.49:0/39269390 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7f68f4001a00 con 0x7f68fc0cb010 2023-11-06T02:14:59.074 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.074+0000 7f690133f640 1 -- 172.21.15.49:0/39269390 >> v1:172.21.15.49:6789/0 conn(0x7f68fc0cb010 legacy=0x7f68fc0c2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:59.074 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.074+0000 7f690133f640 1 --2- 172.21.15.49:0/39269390 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f68fc0b0790 0x7f68e80039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:59.074 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.074+0000 7f690133f640 1 --2- 172.21.15.49:0/39269390 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f68e8003f30 0x7f68e80047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:59.074 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.074+0000 7f690133f640 1 --2- 172.21.15.49:0/39269390 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68e8004d10 0x7f68e8005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:59.075 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.074+0000 7f690133f640 1 -- 172.21.15.49:0/39269390 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68fc0cb5b0 con 0x7f68e8003f30 2023-11-06T02:14:59.075 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.074+0000 7f690133f640 1 -- 172.21.15.49:0/39269390 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e800bb20 con 0x7f68e8004d10 2023-11-06T02:14:59.075 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.074+0000 7f690133f640 1 -- 172.21.15.49:0/39269390 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68e800bce0 con 0x7f68fc0b0790 2023-11-06T02:14:59.075 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.074+0000 7f6901b40640 1 --2- 172.21.15.49:0/39269390 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f68e8003f30 0x7f68e80047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:59.076 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.074+0000 7f6902341640 1 --2- 172.21.15.49:0/39269390 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f68fc0b0790 0x7f68e80039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:59.076 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.074+0000 7f6902b42640 1 --2- 172.21.15.49:0/39269390 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68e8004d10 0x7f68e8005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:59.076 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.074+0000 7f6901b40640 1 -- 172.21.15.49:0/39269390 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f68fc0b0790 msgr2=0x7f68e80039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:59.076 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.074+0000 7f6901b40640 1 --2- 172.21.15.49:0/39269390 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f68fc0b0790 0x7f68e80039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:59.076 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.074+0000 7f6901b40640 1 -- 172.21.15.49:0/39269390 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68e8004d10 msgr2=0x7f68e8005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:59.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.074+0000 7f6901b40640 1 --2- 172.21.15.49:0/39269390 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68e8004d10 0x7f68e8005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:59.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.074+0000 7f6901b40640 1 -- 172.21.15.49:0/39269390 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68e8003530 con 0x7f68e8003f30 2023-11-06T02:14:59.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.074+0000 7f6902b42640 1 --2- 172.21.15.49:0/39269390 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68e8004d10 0x7f68e8005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:14:59.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.074+0000 7f6902341640 1 --2- 172.21.15.49:0/39269390 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f68fc0b0790 0x7f68e80039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:14:59.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.078+0000 7f6901b40640 1 --2- 172.21.15.49:0/39269390 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f68e8003f30 0x7f68e80047d0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f68ec009ae0 tx=0x7f68ec01ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=526fd09b0c12f8f6 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:59.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.078+0000 7f690133f640 1 -- 172.21.15.49:0/39269390 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68ec0a55f0 con 0x7f68e8003f30 2023-11-06T02:14:59.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.078+0000 7f690133f640 1 -- 172.21.15.49:0/39269390 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68ec0a5750 con 0x7f68e8003f30 2023-11-06T02:14:59.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.078+0000 7f690133f640 1 -- 172.21.15.49:0/39269390 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68ec0a5b00 con 0x7f68e8003f30 2023-11-06T02:14:59.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.078+0000 7f69045a5640 1 -- 172.21.15.49:0/39269390 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f68e8003f30 msgr2=0x7f68e80047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:59.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.078+0000 7f69045a5640 1 --2- 172.21.15.49:0/39269390 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f68e8003f30 0x7f68e80047d0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f68ec009ae0 tx=0x7f68ec01ca20 comp rx=0 tx=0).stop 2023-11-06T02:14:59.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.078+0000 7f6902b42640 1 -- 172.21.15.49:0/39269390 reap_dead start 2023-11-06T02:14:59.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.078+0000 7f69045a5640 1 -- 172.21.15.49:0/39269390 shutdown_connections 2023-11-06T02:14:59.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.078+0000 7f69045a5640 1 -- 172.21.15.49:0/39269390 >> 172.21.15.49:0/39269390 conn(0x7f68fc06c0a0 msgr2=0x7f68fc06c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:59.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.078+0000 7f69045a5640 1 -- 172.21.15.49:0/39269390 shutdown_connections 2023-11-06T02:14:59.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.078+0000 7f69045a5640 1 -- 172.21.15.49:0/39269390 wait complete. 2023-11-06T02:14:59.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.078+0000 7f69045a5640 1 Processor -- start 2023-11-06T02:14:59.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.078+0000 7f69045a5640 1 -- start start 2023-11-06T02:14:59.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.078+0000 7f69045a5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f68e8003f30 0x7f68fc149680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:59.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.082+0000 7f69045a5640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68fc15d320 0x7f68fc15d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:59.085 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.082+0000 7f69045a5640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f68fc14c2a0 0x7f68fc14c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:59.085 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.082+0000 7f69045a5640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68fc0ccd80 con 0x7f68e8003f30 2023-11-06T02:14:59.085 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.082+0000 7f69045a5640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68fc0cc1d0 con 0x7f68fc15d320 2023-11-06T02:14:59.085 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.082+0000 7f69045a5640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68fc15dce0 con 0x7f68fc14c2a0 2023-11-06T02:14:59.085 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.082+0000 7f6902341640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f68e8003f30 0x7f68fc149680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:59.086 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.082+0000 7f6902341640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f68e8003f30 0x7f68fc149680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56338/0 (socket says 172.21.15.49:56338) 2023-11-06T02:14:59.086 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.082+0000 7f6902341640 1 -- 172.21.15.49:0/1228665100 learned_addr learned my addr 172.21.15.49:0/1228665100 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:59.086 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.082+0000 7f6901b40640 1 --2- 172.21.15.49:0/1228665100 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68fc15d320 0x7f68fc15d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:59.086 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.082+0000 7f6902b42640 1 --2- 172.21.15.49:0/1228665100 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f68fc14c2a0 0x7f68fc14c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:59.086 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.082+0000 7f6902341640 1 -- 172.21.15.49:0/1228665100 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f68fc14c2a0 msgr2=0x7f68fc14c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:59.087 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.082+0000 7f6902341640 1 --2- 172.21.15.49:0/1228665100 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f68fc14c2a0 0x7f68fc14c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:59.087 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.082+0000 7f6902341640 1 -- 172.21.15.49:0/1228665100 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68fc15d320 msgr2=0x7f68fc15d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:59.087 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.082+0000 7f6902341640 1 --2- 172.21.15.49:0/1228665100 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68fc15d320 0x7f68fc15d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:59.087 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.082+0000 7f6902341640 1 -- 172.21.15.49:0/1228665100 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68fc0cc3e0 con 0x7f68e8003f30 2023-11-06T02:14:59.087 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.082+0000 7f6901b40640 1 --2- 172.21.15.49:0/1228665100 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68fc15d320 0x7f68fc15d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:14:59.088 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.082+0000 7f6902b42640 1 --2- 172.21.15.49:0/1228665100 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f68fc14c2a0 0x7f68fc14c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:14:59.088 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.082+0000 7f6902341640 1 --2- 172.21.15.49:0/1228665100 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f68e8003f30 0x7f68fc149680 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f68f800d7a0 tx=0x7f68f800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:59.088 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.082+0000 7f68f2ffd640 1 -- 172.21.15.49:0/1228665100 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f8017070 con 0x7f68e8003f30 2023-11-06T02:14:59.088 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.082+0000 7f68f2ffd640 1 -- 172.21.15.49:0/1228665100 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68f8005070 con 0x7f68e8003f30 2023-11-06T02:14:59.088 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.082+0000 7f69045a5640 1 -- 172.21.15.49:0/1228665100 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68fc0cc6c0 con 0x7f68e8003f30 2023-11-06T02:14:59.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.082+0000 7f68f2ffd640 1 -- 172.21.15.49:0/1228665100 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f8005420 con 0x7f68e8003f30 2023-11-06T02:14:59.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.082+0000 7f69045a5640 1 -- 172.21.15.49:0/1228665100 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68fc0cd1a0 con 0x7f68e8003f30 2023-11-06T02:14:59.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.082+0000 7f69045a5640 1 -- 172.21.15.49:0/1228665100 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68fc0ccff0 con 0x7f68e8003f30 2023-11-06T02:14:59.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.082+0000 7f68f2ffd640 1 -- 172.21.15.49:0/1228665100 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f68f8006830 con 0x7f68e8003f30 2023-11-06T02:14:59.090 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.082+0000 7f68f2ffd640 1 --2- 172.21.15.49:0/1228665100 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f68e0065a60 0x7f68e0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:59.090 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.082+0000 7f68f2ffd640 1 -- 172.21.15.49:0/1228665100 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f68f8082900 con 0x7f68e8003f30 2023-11-06T02:14:59.090 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.082+0000 7f6901b40640 1 --2- 172.21.15.49:0/1228665100 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f68e0065a60 0x7f68e0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:59.090 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.086+0000 7f6901b40640 1 --2- 172.21.15.49:0/1228665100 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f68e0065a60 0x7f68e0067f20 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f68ec009ae0 tx=0x7f68ec0046d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:59.090 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.086+0000 7f68f2ffd640 1 -- 172.21.15.49:0/1228665100 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f68f8050440 con 0x7f68e8003f30 2023-11-06T02:14:59.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.242+0000 7f69045a5640 1 -- 172.21.15.49:0/1228665100 --> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f68fc0cbe50 con 0x7f68e0065a60 2023-11-06T02:14:59.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.250+0000 7f68f2ffd640 1 -- 172.21.15.49:0/1228665100 <== mgr.4100 v2:172.21.15.150:6832/38104 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+71133 (secure 0 0 0) 0x7f68ec004100 con 0x7f68e0065a60 2023-11-06T02:14:59.246 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:59.246 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2023-11-06T02:14:59.248 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.254+0000 7f69045a5640 1 -- 172.21.15.49:0/1228665100 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f68e0065a60 msgr2=0x7f68e0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:59.248 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.254+0000 7f69045a5640 1 --2- 172.21.15.49:0/1228665100 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f68e0065a60 0x7f68e0067f20 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f68ec009ae0 tx=0x7f68ec0046d0 comp rx=0 tx=0).stop 2023-11-06T02:14:59.248 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.254+0000 7f69045a5640 1 -- 172.21.15.49:0/1228665100 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f68e8003f30 msgr2=0x7f68fc149680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:59.248 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.254+0000 7f69045a5640 1 --2- 172.21.15.49:0/1228665100 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f68e8003f30 0x7f68fc149680 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f68f800d7a0 tx=0x7f68f800dc60 comp rx=0 tx=0).stop 2023-11-06T02:14:59.248 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.254+0000 7f69045a5640 1 -- 172.21.15.49:0/1228665100 shutdown_connections 2023-11-06T02:14:59.249 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.254+0000 7f69045a5640 1 --2- 172.21.15.49:0/1228665100 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f68e0065a60 0x7f68e0067f20 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:59.249 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.254+0000 7f69045a5640 1 --2- 172.21.15.49:0/1228665100 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f68fc14c2a0 0x7f68fc14c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:59.249 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.254+0000 7f69045a5640 1 --2- 172.21.15.49:0/1228665100 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68fc15d320 0x7f68fc15d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:59.249 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.254+0000 7f69045a5640 1 --2- 172.21.15.49:0/1228665100 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f68e8003f30 0x7f68fc149680 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:59.249 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.254+0000 7f69045a5640 1 -- 172.21.15.49:0/1228665100 >> 172.21.15.49:0/1228665100 conn(0x7f68fc06c0a0 msgr2=0x7f68fc0afa50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:59.250 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.254+0000 7f69045a5640 1 -- 172.21.15.49:0/1228665100 shutdown_connections 2023-11-06T02:14:59.250 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.254+0000 7f69045a5640 1 -- 172.21.15.49:0/1228665100 wait complete. 2023-11-06T02:14:59.264 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":278,"stamp":"2023-11-06T02:14:57.316013+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":269724,"kb_used_data":25436,"kb_used_omap":0,"kb_used_meta":244224,"kb_avail":754704996,"statfs":{"total":773094113280,"available":772817915904,"internally_reserved":0,"allocated":26046464,"data_stored":21579923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":250085376},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8390586,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-258,"num_read_kb":-264,"num_write":-177,"num_write_kb":-8295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002028"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":221,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T02:14:23.281538+0000","last_change":"2023-11-06T02:08:20.523427+0000","last_active":"2023-11-06T02:14:23.281538+0000","last_peered":"2023-11-06T02:14:23.281538+0000","last_clean":"2023-11-06T02:14:23.281538+0000","last_became_active":"2023-11-06T02:08:20.521089+0000","last_became_peered":"2023-11-06T02:08:20.521089+0000","last_unstale":"2023-11-06T02:14:23.281538+0000","last_undegraded":"2023-11-06T02:14:23.281538+0000","last_fullsized":"2023-11-06T02:14:23.281538+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:19.252348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:19.252348+0000","last_clean_scrub_stamp":"2023-11-06T02:08:19.252348+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:30:15.566397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":67,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T02:14:35.449337+0000","last_change":"2023-11-06T02:08:26.059502+0000","last_active":"2023-11-06T02:14:35.449337+0000","last_peered":"2023-11-06T02:14:35.449337+0000","last_clean":"2023-11-06T02:14:35.449337+0000","last_became_active":"2023-11-06T02:08:26.058612+0000","last_became_peered":"2023-11-06T02:08:26.058612+0000","last_unstale":"2023-11-06T02:14:35.449337+0000","last_undegraded":"2023-11-06T02:14:35.449337+0000","last_fullsized":"2023-11-06T02:14:35.449337+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:10.626021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":68,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T02:14:28.282679+0000","last_change":"2023-11-06T02:08:26.060406+0000","last_active":"2023-11-06T02:14:28.282679+0000","last_peered":"2023-11-06T02:14:28.282679+0000","last_clean":"2023-11-06T02:14:28.282679+0000","last_became_active":"2023-11-06T02:08:26.059262+0000","last_became_peered":"2023-11-06T02:08:26.059262+0000","last_unstale":"2023-11-06T02:14:28.282679+0000","last_undegraded":"2023-11-06T02:14:28.282679+0000","last_fullsized":"2023-11-06T02:14:28.282679+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:10.626021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":68,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T02:14:35.447393+0000","last_change":"2023-11-06T02:08:26.559477+0000","last_active":"2023-11-06T02:14:35.447393+0000","last_peered":"2023-11-06T02:14:35.447393+0000","last_clean":"2023-11-06T02:14:35.447393+0000","last_became_active":"2023-11-06T02:08:26.559202+0000","last_became_peered":"2023-11-06T02:08:26.559202+0000","last_unstale":"2023-11-06T02:14:35.447393+0000","last_undegraded":"2023-11-06T02:14:35.447393+0000","last_fullsized":"2023-11-06T02:14:35.447393+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:55:04.552404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":67,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T02:14:35.448967+0000","last_change":"2023-11-06T02:08:26.059546+0000","last_active":"2023-11-06T02:14:35.448967+0000","last_peered":"2023-11-06T02:14:35.448967+0000","last_clean":"2023-11-06T02:14:35.448967+0000","last_became_active":"2023-11-06T02:08:26.058806+0000","last_became_peered":"2023-11-06T02:08:26.058806+0000","last_unstale":"2023-11-06T02:14:35.448967+0000","last_undegraded":"2023-11-06T02:14:35.448967+0000","last_fullsized":"2023-11-06T02:14:35.448967+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:23.468510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":67,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T02:14:35.439334+0000","last_change":"2023-11-06T02:08:26.436463+0000","last_active":"2023-11-06T02:14:35.439334+0000","last_peered":"2023-11-06T02:14:35.439334+0000","last_clean":"2023-11-06T02:14:35.439334+0000","last_became_active":"2023-11-06T02:08:26.436075+0000","last_became_peered":"2023-11-06T02:08:26.436075+0000","last_unstale":"2023-11-06T02:14:35.439334+0000","last_undegraded":"2023-11-06T02:14:35.439334+0000","last_fullsized":"2023-11-06T02:14:35.439334+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:23.468510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":68,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-06T02:14:35.357267+0000","last_change":"2023-11-06T02:08:26.436607+0000","last_active":"2023-11-06T02:14:35.357267+0000","last_peered":"2023-11-06T02:14:35.357267+0000","last_clean":"2023-11-06T02:14:35.357267+0000","last_became_active":"2023-11-06T02:08:26.435924+0000","last_became_peered":"2023-11-06T02:08:26.435924+0000","last_unstale":"2023-11-06T02:14:35.357267+0000","last_undegraded":"2023-11-06T02:14:35.357267+0000","last_fullsized":"2023-11-06T02:14:35.357267+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:23.468510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":66,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T02:14:13.290705+0000","last_change":"2023-11-06T02:08:26.057076+0000","last_active":"2023-11-06T02:14:13.290705+0000","last_peered":"2023-11-06T02:14:13.290705+0000","last_clean":"2023-11-06T02:14:13.290705+0000","last_became_active":"2023-11-06T02:08:26.056419+0000","last_became_peered":"2023-11-06T02:08:26.056419+0000","last_unstale":"2023-11-06T02:14:13.290705+0000","last_undegraded":"2023-11-06T02:14:13.290705+0000","last_fullsized":"2023-11-06T02:14:13.290705+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:10.626021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":75,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T02:14:13.297179+0000","last_change":"2023-11-06T02:08:26.058596+0000","last_active":"2023-11-06T02:14:13.297179+0000","last_peered":"2023-11-06T02:14:13.297179+0000","last_clean":"2023-11-06T02:14:13.297179+0000","last_became_active":"2023-11-06T02:08:26.057015+0000","last_became_peered":"2023-11-06T02:08:26.057015+0000","last_unstale":"2023-11-06T02:14:13.297179+0000","last_undegraded":"2023-11-06T02:14:13.297179+0000","last_fullsized":"2023-11-06T02:14:13.297179+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:55:04.552404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574930,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32608,"kb_used_data":1560,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339232,"statfs":{"total":96636764160,"available":96603373568,"internally_reserved":0,"allocated":1597440,"data_stored":1042734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.87,"15min":0.85599999999999998},"min":{"1min":0.62,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.147,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.52700000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.2230000000000001,"5min":1.421,"15min":1.506},"last":1.0089999999999999}]},{"osd":1,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.83399999999999996,"15min":0.84499999999999997},"min":{"1min":0.498,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.3320000000000001,"5min":1.345,"15min":1.345},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.92900000000000005,"15min":0.90200000000000002},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.54900000000000004},"max":{"1min":1.131,"5min":1.681,"15min":1.681},"last":0.97399999999999998}]},{"osd":2,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.96399999999999997,"15min":0.95699999999999996},"min":{"1min":0.65300000000000002,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.1919999999999999,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":1.133},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.92200000000000004,"15min":0.97399999999999998},"min":{"1min":0.57399999999999995,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.1919999999999999,"5min":1.5,"15min":1.5920000000000001},"last":0.74199999999999999}]},{"osd":3,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.89200000000000002,"15min":0.88},"min":{"1min":0.66100000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.1519999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":1.234},{"interface":"front","average":{"1min":0.995,"5min":1.03,"15min":1.0609999999999999},"min":{"1min":0.74399999999999999,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.2110000000000001,"5min":1.526,"15min":1.526},"last":0.92200000000000004}]},{"osd":4,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.91700000000000004,"15min":0.94399999999999995},"min":{"1min":0.73399999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.1930000000000001,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.97999999999999998,"15min":1.0229999999999999},"min":{"1min":0.65000000000000002,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.1519999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":1.0640000000000001}]},{"osd":5,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":1.026,"15min":1.0609999999999999},"min":{"1min":0.69199999999999995,"5min":0.503,"15min":0.503},"max":{"1min":1.2629999999999999,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":1.0389999999999999},{"interface":"front","average":{"1min":1.044,"5min":1.044,"15min":1.105},"min":{"1min":0.71299999999999997,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.288,"5min":1.3700000000000001,"15min":1.7809999999999999},"last":0.94999999999999996}]},{"osd":6,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":1.1180000000000001,"15min":1.1739999999999999},"min":{"1min":0.89500000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.274,"5min":1.5009999999999999,"15min":1.6679999999999999},"last":1.113},{"interface":"front","average":{"1min":1.028,"5min":1.0649999999999999,"15min":1.127},"min":{"1min":0.72599999999999998,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.2509999999999999,"5min":1.4570000000000001,"15min":1.74},"last":1.1579999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574929,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36136,"kb_used_data":5088,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94335704,"statfs":{"total":96636764160,"available":96599760896,"internally_reserved":0,"allocated":5210112,"data_stored":4647957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.872,"15min":0.86199999999999999},"min":{"1min":0.57199999999999995,"5min":0.495,"15min":0.495},"max":{"1min":1.226,"5min":1.226,"15min":1.2609999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.88600000000000001,"15min":0.83199999999999996},"min":{"1min":0.5,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.3280000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.68899999999999995}]},{"osd":1,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.85099999999999998,"15min":0.86699999999999999},"min":{"1min":0.54400000000000004,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.1639999999999999,"5min":1.244,"15min":1.244},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.90500000000000003,"15min":0.88400000000000001},"min":{"1min":0.56599999999999995,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":0.82599999999999996}]},{"osd":2,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.89200000000000002,"15min":0.89600000000000002},"min":{"1min":0.61499999999999999,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.877,"5min":0.88900000000000001,"15min":0.91700000000000004},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.075,"5min":1.411,"15min":1.411},"last":0.96199999999999997}]},{"osd":3,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.93500000000000005,"15min":0.95899999999999996},"min":{"1min":0.74099999999999999,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.92300000000000004},{"interface":"front","average":{"1min":1.0309999999999999,"5min":0.999,"15min":1},"min":{"1min":0.81599999999999995,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.2609999999999999,"5min":1.381,"15min":1.381},"last":1.0149999999999999}]},{"osd":4,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.92200000000000004,"15min":0.96999999999999997},"min":{"1min":0.627,"5min":0.56799999999999995,"15min":0.52900000000000003},"max":{"1min":1.2390000000000001,"5min":1.302,"15min":1.474},"last":0.995},{"interface":"front","average":{"1min":0.997,"5min":0.97399999999999998,"15min":1.018},"min":{"1min":0.66700000000000004,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.4039999999999999,"5min":1.556,"15min":1.556},"last":1.232}]},{"osd":5,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":0.97199999999999998,"15min":1.0089999999999999},"min":{"1min":0.68899999999999995,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":1.1950000000000001},{"interface":"front","average":{"1min":1.1319999999999999,"5min":1.075,"15min":1.101},"min":{"1min":0.82699999999999996,"5min":0.66900000000000004,"15min":0.58699999999999997},"max":{"1min":1.3999999999999999,"5min":1.587,"15min":1.587},"last":1.069}]},{"osd":7,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.087,"15min":1.107},"min":{"1min":0.88300000000000001,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.5369999999999999,"5min":1.615,"15min":1.615},"last":1.377},{"interface":"front","average":{"1min":1.103,"5min":1.0509999999999999,"15min":1.127},"min":{"1min":0.85599999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.4610000000000001,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":1.284}]}]},{"osd":5,"up_from":13,"seq":55834574930,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27940,"kb_used_data":988,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343900,"statfs":{"total":96636764160,"available":96608153600,"internally_reserved":0,"allocated":1011712,"data_stored":452390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:14:33 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.86199999999999999,"15min":0.83999999999999997},"min":{"1min":0.54400000000000004,"5min":0.51700000000000002,"15min":0.47299999999999998},"max":{"1min":1.079,"5min":1.381,"15min":1.381},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.86899999999999999,"15min":0.85099999999999998},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.159,"5min":1.3200000000000001,"15min":1.6579999999999999},"last":0.56000000000000005}]},{"osd":1,"last update":"Mon Nov 6 02:14:33 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.90500000000000003,"15min":0.86799999999999999},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.60299999999999998},"max":{"1min":1.369,"5min":1.369,"15min":1.369},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.90000000000000002,"15min":0.84399999999999997},"min":{"1min":0.53700000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.1639999999999999,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.69199999999999995}]},{"osd":2,"last update":"Mon Nov 6 02:14:33 2023","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.93899999999999995,"15min":0.94299999999999995},"min":{"1min":0.65100000000000002,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.242,"5min":1.298,"15min":1.3560000000000001},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.89700000000000002,"15min":0.82499999999999996},"min":{"1min":0.58699999999999997,"5min":0.55800000000000005,"15min":0.47899999999999998},"max":{"1min":1.218,"5min":1.323,"15min":1.377},"last":0.76500000000000001}]},{"osd":3,"last update":"Mon Nov 6 02:14:33 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.95199999999999996,"15min":0.93000000000000005},"min":{"1min":0.66400000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.292,"5min":1.3580000000000001,"15min":1.4399999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":1.0369999999999999,"5min":0.98799999999999999,"15min":1.05},"min":{"1min":0.76700000000000002,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.2569999999999999,"5min":1.349,"15min":1.7250000000000001},"last":0.99199999999999999}]},{"osd":4,"last update":"Mon Nov 6 02:14:33 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.87,"15min":0.90000000000000002},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.0129999999999999,"5min":1.276,"15min":1.276},"last":0.94799999999999995},{"interface":"front","average":{"1min":0.995,"5min":0.98599999999999999,"15min":1.014},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.496,"5min":1.496,"15min":1.554},"last":0.90300000000000002}]},{"osd":6,"last update":"Mon Nov 6 02:14:33 2023","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.98699999999999999,"15min":1.0409999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.3440000000000001,"5min":1.454,"15min":1.454},"last":1.0640000000000001},{"interface":"front","average":{"1min":1.046,"5min":1.0229999999999999,"15min":1.0349999999999999},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":2.2309999999999999,"5min":2.2309999999999999,"15min":2.2309999999999999},"last":1.02}]},{"osd":7,"last update":"Mon Nov 6 02:14:33 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.0640000000000001,"15min":1.0720000000000001},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.5209999999999999,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":0.83799999999999997},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.117,"15min":1.1419999999999999},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":2.3759999999999999,"5min":2.3759999999999999,"15min":2.3759999999999999},"last":1.153}]}]},{"osd":4,"up_from":13,"seq":55834574930,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32044,"kb_used_data":996,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339796,"statfs":{"total":96636764160,"available":96603951104,"internally_reserved":0,"allocated":1019904,"data_stored":453752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:14:27 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.86399999999999999,"15min":0.83799999999999997},"min":{"1min":0.53500000000000003,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.2450000000000001,"5min":1.343,"15min":1.343},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.84499999999999997,"15min":0.79500000000000004},"min":{"1min":0.51600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.175,"5min":1.28,"15min":1.3049999999999999},"last":0.85899999999999999}]},{"osd":1,"last update":"Mon Nov 6 02:14:27 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.86399999999999999,"15min":0.84999999999999998},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.508,"5min":1.508,"15min":1.508},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.91800000000000004,"15min":0.90000000000000002},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.1080000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.80100000000000005}]},{"osd":2,"last update":"Mon Nov 6 02:14:27 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.90800000000000003,"15min":0.98199999999999998},"min":{"1min":0.57999999999999996,"5min":0.503,"15min":0.503},"max":{"1min":1.0880000000000001,"5min":1.389,"15min":1.615},"last":1.159},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.95499999999999996,"15min":0.93500000000000005},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.57899999999999996},"max":{"1min":1.1559999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.92400000000000004}]},{"osd":3,"last update":"Mon Nov 6 02:14:27 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.86199999999999999,"15min":0.90000000000000002},"min":{"1min":0.59899999999999998,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.22,"5min":1.22,"15min":1.52},"last":0.745},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.97299999999999998,"15min":1.0369999999999999},"min":{"1min":0.68100000000000005,"5min":0.63900000000000001,"15min":0.59999999999999998},"max":{"1min":1.589,"5min":1.595,"15min":1.595},"last":1.0509999999999999}]},{"osd":5,"last update":"Mon Nov 6 02:14:27 2023","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":1.026},"min":{"1min":0.72599999999999998,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.212,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":1.1200000000000001},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.96099999999999997,"15min":1.024},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.629},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.7869999999999999},"last":1.0960000000000001}]},{"osd":6,"last update":"Mon Nov 6 02:14:27 2023","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.008,"15min":1.0649999999999999},"min":{"1min":0.77600000000000002,"5min":0.68000000000000005,"15min":0.67100000000000004},"max":{"1min":1.6679999999999999,"5min":1.6679999999999999,"15min":1.6699999999999999},"last":1.216},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.085,"15min":1.1599999999999999},"min":{"1min":0.71999999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.2210000000000001,"5min":1.5900000000000001,"15min":2.1190000000000002},"last":1.004}]},{"osd":7,"last update":"Mon Nov 6 02:14:27 2023","interfaces":[{"interface":"back","average":{"1min":1.056,"5min":1.042,"15min":1.0760000000000001},"min":{"1min":0.77100000000000002,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.7110000000000001},"last":0.94699999999999995},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.117,"15min":1.1990000000000001},"min":{"1min":0.90500000000000003,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.754,"5min":1.754,"15min":1.754},"last":1.2629999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574929,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32032,"kb_used_data":984,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339808,"statfs":{"total":96636764160,"available":96603963392,"internally_reserved":0,"allocated":1007616,"data_stored":452931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63,"15min":0.64200000000000002},"min":{"1min":0.45300000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":1.171,"5min":1.198,"15min":1.198},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.65400000000000003,"15min":0.63500000000000001},"min":{"1min":0.49099999999999999,"5min":0.29899999999999999,"15min":0.29799999999999999},"max":{"1min":1.1819999999999999,"5min":1.21,"15min":1.21},"last":0.35199999999999998}]},{"osd":1,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.61599999999999999,"15min":0.626},"min":{"1min":0.42499999999999999,"5min":0.248,"15min":0.248},"max":{"1min":0.86799999999999999,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.373},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.75800000000000001,"15min":0.74199999999999999},"min":{"1min":0.46899999999999997,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.175,"5min":1.204,"15min":1.204},"last":0.53000000000000003}]},{"osd":2,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.72499999999999998,"15min":0.73099999999999998},"min":{"1min":0.499,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.438},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.81299999999999994,"15min":0.81999999999999995},"min":{"1min":0.62,"5min":0.308,"15min":0.308},"max":{"1min":1.0329999999999999,"5min":1.405,"15min":1.405},"last":0.501}]},{"osd":4,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.62,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.262,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.93700000000000006,"15min":0.97799999999999998},"min":{"1min":0.70799999999999996,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.212,"5min":1.292,"15min":1.4410000000000001},"last":0.66300000000000003}]},{"osd":5,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.97499999999999998,"15min":1.0109999999999999},"min":{"1min":0.72699999999999998,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.21,"5min":1.375,"15min":1.375},"last":0.71999999999999997},{"interface":"front","average":{"1min":1.024,"5min":1.0269999999999999,"15min":1.0629999999999999},"min":{"1min":0.65200000000000002,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.323,"5min":1.4890000000000001,"15min":2.21},"last":0.81799999999999995}]},{"osd":6,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.0209999999999999,"15min":1.024},"min":{"1min":0.79800000000000004,"5min":0.439,"15min":0.439},"max":{"1min":1.3080000000000001,"5min":1.395,"15min":1.395},"last":0.90000000000000002},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.0269999999999999,"15min":1.0569999999999999},"min":{"1min":0.74099999999999999,"5min":0.375,"15min":0.375},"max":{"1min":1.2709999999999999,"5min":1.4319999999999999,"15min":1.554},"last":0.83099999999999996}]},{"osd":7,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.1020000000000001,"15min":1.1220000000000001},"min":{"1min":0.68200000000000005,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.4410000000000001,"5min":1.5549999999999999,"15min":2.2829999999999999},"last":0.93100000000000005},{"interface":"front","average":{"1min":1.123,"5min":1.0980000000000001,"15min":1.1040000000000001},"min":{"1min":0.91100000000000003,"5min":0.497,"15min":0.497},"max":{"1min":1.3620000000000001,"5min":1.504,"15min":1.504},"last":0.96799999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574929,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36132,"kb_used_data":5084,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94335708,"statfs":{"total":96636764160,"available":96599764992,"internally_reserved":0,"allocated":5206016,"data_stored":4647390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:14:28 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.59099999999999997,"15min":0.52900000000000003},"min":{"1min":0.28499999999999998,"5min":0.252,"15min":0.252},"max":{"1min":0.90300000000000002,"5min":1.079,"15min":1.079},"last":0.752},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.63400000000000001,"15min":0.56999999999999995},"min":{"1min":0.374,"5min":0.316,"15min":0.27300000000000002},"max":{"1min":0.86699999999999999,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":0.64500000000000002}]},{"osd":1,"last update":"Mon Nov 6 02:14:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.65000000000000002,"15min":0.58999999999999997},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.26400000000000001},"max":{"1min":0.97699999999999998,"5min":1.48,"15min":1.48},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.72799999999999998,"15min":0.621},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.26500000000000001},"max":{"1min":1.04,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":0.42599999999999999}]},{"osd":3,"last update":"Mon Nov 6 02:14:28 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.71399999999999997,"15min":0.63600000000000001},"min":{"1min":0.316,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.0580000000000001,"5min":1.121,"15min":1.155},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.76400000000000001,"15min":0.68200000000000005},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.069,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.94099999999999995}]},{"osd":4,"last update":"Mon Nov 6 02:14:28 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.93100000000000005,"15min":0.84299999999999997},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.1539999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.92900000000000005,"15min":0.91900000000000004},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.234,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":0.96999999999999997}]},{"osd":5,"last update":"Mon Nov 6 02:14:28 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.96299999999999997,"15min":0.94899999999999995},"min":{"1min":0.56299999999999994,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.258,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.97299999999999998,"15min":0.94699999999999995},"min":{"1min":0.626,"5min":0.60999999999999999,"15min":0.52600000000000002},"max":{"1min":1.2310000000000001,"5min":1.615,"15min":1.7110000000000001},"last":1.038}]},{"osd":6,"last update":"Mon Nov 6 02:14:28 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.96899999999999997,"15min":0.94799999999999995},"min":{"1min":0.60199999999999998,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.218,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.93600000000000005,"5min":1.01,"15min":1.0009999999999999},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.274,"5min":1.675,"15min":1.677},"last":0.78200000000000003}]},{"osd":7,"last update":"Mon Nov 6 02:14:28 2023","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":1.0309999999999999,"15min":0.99299999999999999},"min":{"1min":0.67500000000000004,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.2869999999999999,"5min":1.659,"15min":1.659},"last":1.0649999999999999},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.0760000000000001,"15min":1.0529999999999999},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.1879999999999999,"5min":1.4650000000000001,"15min":1.5980000000000001},"last":1.0169999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574930,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36132,"kb_used_data":5084,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94335708,"statfs":{"total":96636764160,"available":96599764992,"internally_reserved":0,"allocated":5206016,"data_stored":4646765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:14:28 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.628,"15min":0.626},"min":{"1min":0.29599999999999999,"5min":0.28699999999999998,"15min":0.224},"max":{"1min":0.95099999999999996,"5min":1.603,"15min":1.603},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.71999999999999997,"15min":0.67400000000000004},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.081,"5min":2.161,"15min":2.161},"last":0.58699999999999997}]},{"osd":2,"last update":"Mon Nov 6 02:14:28 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.64700000000000002,"15min":0.63},"min":{"1min":0.34399999999999997,"5min":0.253,"15min":0.253},"max":{"1min":1.036,"5min":1.649,"15min":1.649},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.74199999999999999,"15min":0.74399999999999999},"min":{"1min":0.373,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.91700000000000004,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.90700000000000003}]},{"osd":3,"last update":"Mon Nov 6 02:14:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.80400000000000005,"15min":0.80300000000000005},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.131,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":0.97999999999999998},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.79100000000000004,"15min":0.80400000000000005},"min":{"1min":0.35499999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.94199999999999995,"5min":1.5,"15min":1.8149999999999999},"last":0.77800000000000002}]},{"osd":4,"last update":"Mon Nov 6 02:14:28 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.96199999999999997,"15min":0.98199999999999998},"min":{"1min":0.55600000000000005,"5min":0.504,"15min":0.504},"max":{"1min":1.2110000000000001,"5min":2.2759999999999998,"15min":2.2759999999999998},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.95599999999999996,"15min":0.98099999999999998},"min":{"1min":0.63300000000000001,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.0700000000000001,"5min":2.9180000000000001,"15min":2.9180000000000001},"last":1.0329999999999999}]},{"osd":5,"last update":"Mon Nov 6 02:14:28 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.97199999999999998,"15min":0.96799999999999997},"min":{"1min":0.65800000000000003,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.177,"5min":2.7469999999999999,"15min":2.7469999999999999},"last":1.2030000000000001},{"interface":"front","average":{"1min":0.98399999999999999,"5min":1.05,"15min":1.069},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.52100000000000002},"max":{"1min":1.3720000000000001,"5min":1.921,"15min":1.921},"last":1.1870000000000001}]},{"osd":6,"last update":"Mon Nov 6 02:14:28 2023","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":1.0960000000000001,"15min":1.1000000000000001},"min":{"1min":0.61899999999999999,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.304,"5min":3.5710000000000002,"15min":3.5710000000000002},"last":1.085},{"interface":"front","average":{"1min":0.94999999999999996,"5min":1.0369999999999999,"15min":1.0629999999999999},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.268,"5min":2.843,"15min":2.843},"last":1.304}]},{"osd":7,"last update":"Mon Nov 6 02:14:28 2023","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":1.0980000000000001,"15min":1.1140000000000001},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.4159999999999999,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":1.133},{"interface":"front","average":{"1min":1.052,"5min":1.1419999999999999,"15min":1.1539999999999999},"min":{"1min":0.61799999999999999,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.3520000000000001,"5min":3.4790000000000001,"15min":3.4790000000000001},"last":1.242}]}]},{"osd":0,"up_from":13,"seq":55834574930,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36700,"kb_used_data":5652,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94335140,"statfs":{"total":96636764160,"available":96599183360,"internally_reserved":0,"allocated":5787648,"data_stored":5236004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 02:14:27 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.62,"15min":0.60499999999999998},"min":{"1min":0.32500000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.94899999999999995,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.62,"5min":0.66600000000000004,"15min":0.64600000000000002},"min":{"1min":0.26100000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.96599999999999997,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.27700000000000002}]},{"osd":2,"last update":"Mon Nov 6 02:14:27 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.68799999999999994,"15min":0.67200000000000004},"min":{"1min":0.36599999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.0900000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.37},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.69699999999999995,"15min":0.68200000000000005},"min":{"1min":0.315,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.98199999999999998,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.315}]},{"osd":3,"last update":"Mon Nov 6 02:14:27 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.69499999999999995,"15min":0.65100000000000002},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":2.1150000000000002,"5min":2.1150000000000002,"15min":2.1150000000000002},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.78500000000000003,"15min":0.749},"min":{"1min":0.39300000000000002,"5min":0.317,"15min":0.317},"max":{"1min":1.0860000000000001,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.34300000000000003}]},{"osd":4,"last update":"Mon Nov 6 02:14:27 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.93000000000000005,"15min":0.86099999999999999},"min":{"1min":0.60099999999999998,"5min":0.55400000000000005,"15min":0.54300000000000004},"max":{"1min":2.3130000000000002,"5min":2.3130000000000002,"15min":2.3130000000000002},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.96299999999999997,"15min":0.94499999999999995},"min":{"1min":0.59599999999999997,"5min":0.5,"15min":0.5},"max":{"1min":1.3140000000000001,"5min":1.3839999999999999,"15min":1.387},"last":0.55300000000000005}]},{"osd":5,"last update":"Mon Nov 6 02:14:27 2023","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.95999999999999996,"15min":0.95199999999999996},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.52400000000000002},{"interface":"front","average":{"1min":1.006,"5min":0.96099999999999997,"15min":0.90100000000000002},"min":{"1min":0.53800000000000003,"5min":0.496,"15min":0.496},"max":{"1min":2.3530000000000002,"5min":2.3530000000000002,"15min":2.3530000000000002},"last":0.68500000000000005}]},{"osd":6,"last update":"Mon Nov 6 02:14:27 2023","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":1.0349999999999999,"15min":1.036},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.4550000000000001,"5min":1.581,"15min":1.589},"last":0.78300000000000003},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.0309999999999999,"15min":1.01},"min":{"1min":0.68100000000000005,"5min":0.65800000000000003,"15min":0.57099999999999995},"max":{"1min":1.4990000000000001,"5min":1.5529999999999999,"15min":1.6220000000000001},"last":0.79100000000000004}]},{"osd":7,"last update":"Mon Nov 6 02:14:27 2023","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":1.024,"15min":1.02},"min":{"1min":0.55300000000000005,"5min":0.504,"15min":0.504},"max":{"1min":1.357,"5min":1.504,"15min":1.504},"last":0.54400000000000004},{"interface":"front","average":{"1min":1.034,"5min":1.0029999999999999,"15min":1.0069999999999999},"min":{"1min":0.64700000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":2.387,"5min":2.387,"15min":2.387},"last":0.753}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T02:14:59.267 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T02:14:59.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.398+0000 7fc210352640 1 Processor -- start 2023-11-06T02:14:59.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.398+0000 7fc210352640 1 -- start start 2023-11-06T02:14:59.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.398+0000 7fc210352640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc208159b40 0x7fc2080ca980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:59.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.398+0000 7fc210352640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc2080cdd50 0x7fc2080cc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:59.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.398+0000 7fc210352640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2080723a0 con 0x7fc2080cb010 2023-11-06T02:14:59.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.398+0000 7fc210352640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2080715e0 con 0x7fc208159b40 2023-11-06T02:14:59.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.398+0000 7fc210352640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2080718c0 con 0x7fc2080cdd50 2023-11-06T02:14:59.395 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.398+0000 7fc20d8ed640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc208159b40 0x7fc2080ca980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:59.395 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.398+0000 7fc20e8ef640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc2080cdd50 0x7fc2080cc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:59.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.398+0000 7fc20d8ed640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc208159b40 0x7fc2080ca980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.49:58072/0 (socket says 172.21.15.49:58072) 2023-11-06T02:14:59.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.398+0000 7fc20d8ed640 1 -- 172.21.15.49:0/1663202822 learned_addr learned my addr 172.21.15.49:0/1663202822 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:59.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.398+0000 7fc20e0ee640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7fc2080cb010 0x7fc2080cb3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.49:33022/0 (socket says 172.21.15.49:33022) 2023-11-06T02:14:59.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.398+0000 7fc20e8ef640 1 -- 172.21.15.49:0/1663202822 >> v2:172.21.15.150:3300/0 conn(0x7fc208159b40 msgr2=0x7fc2080ca980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:14:59.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.398+0000 7fc20e8ef640 1 --2- 172.21.15.49:0/1663202822 >> v2:172.21.15.150:3300/0 conn(0x7fc208159b40 0x7fc2080ca980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:59.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.398+0000 7fc20e8ef640 1 -- 172.21.15.49:0/1663202822 >> v1:172.21.15.150:6789/0 conn(0x7fc2080cb010 legacy=0x7fc2080cb3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:59.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.398+0000 7fc20e8ef640 1 -- 172.21.15.49:0/1663202822 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc208071ba0 con 0x7fc2080cdd50 2023-11-06T02:14:59.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.398+0000 7fc20d8ed640 1 --2- 172.21.15.49:0/1663202822 >> v2:172.21.15.150:3300/0 conn(0x7fc208159b40 0x7fc2080ca980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T02:14:59.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.402+0000 7fc20e8ef640 1 --2- 172.21.15.49:0/1663202822 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc2080cdd50 0x7fc2080cc1d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc20000f370 tx=0x7fc200020d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ade5f372f015a2b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:59.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.402+0000 7fc20d0ec640 1 -- 172.21.15.49:0/1663202822 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2000a9e10 con 0x7fc2080cdd50 2023-11-06T02:14:59.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.402+0000 7fc20d0ec640 1 -- 172.21.15.49:0/1663202822 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc20000d0d0 con 0x7fc2080cdd50 2023-11-06T02:14:59.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.402+0000 7fc20d0ec640 1 -- 172.21.15.49:0/1663202822 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc20000d460 con 0x7fc2080cdd50 2023-11-06T02:14:59.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.402+0000 7fc210352640 1 -- 172.21.15.49:0/1663202822 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc2080cdd50 msgr2=0x7fc2080cc1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:59.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.402+0000 7fc210352640 1 --2- 172.21.15.49:0/1663202822 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc2080cdd50 0x7fc2080cc1d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc20000f370 tx=0x7fc200020d20 comp rx=0 tx=0).stop 2023-11-06T02:14:59.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.402+0000 7fc210352640 1 -- 172.21.15.49:0/1663202822 shutdown_connections 2023-11-06T02:14:59.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.402+0000 7fc210352640 1 --2- 172.21.15.49:0/1663202822 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc2080cdd50 0x7fc2080cc1d0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:59.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.402+0000 7fc210352640 1 --2- 172.21.15.49:0/1663202822 >> v2:172.21.15.150:3300/0 conn(0x7fc208159b40 0x7fc2080ca980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:59.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.402+0000 7fc210352640 1 -- 172.21.15.49:0/1663202822 >> 172.21.15.49:0/1663202822 conn(0x7fc20806c0a0 msgr2=0x7fc20806c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:59.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.402+0000 7fc210352640 1 -- 172.21.15.49:0/1663202822 shutdown_connections 2023-11-06T02:14:59.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.402+0000 7fc210352640 1 -- 172.21.15.49:0/1663202822 wait complete. 2023-11-06T02:14:59.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.402+0000 7fc210352640 1 Processor -- start 2023-11-06T02:14:59.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.402+0000 7fc210352640 1 -- start start 2023-11-06T02:14:59.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.406+0000 7fc210352640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc2080cb010 0x7fc2080c4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:59.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.406+0000 7fc210352640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2080cdd50 0x7fc2080c2130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:59.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.406+0000 7fc210352640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc208159b40 0x7fc2080c93a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:59.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.406+0000 7fc210352640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc208156920 con 0x7fc2080cdd50 2023-11-06T02:14:59.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.406+0000 7fc210352640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc208156c00 con 0x7fc208159b40 2023-11-06T02:14:59.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.406+0000 7fc210352640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc208156ee0 con 0x7fc2080cb010 2023-11-06T02:14:59.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.406+0000 7fc20d8ed640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2080cdd50 0x7fc2080c2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:59.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.406+0000 7fc20d8ed640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2080cdd50 0x7fc2080c2130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56350/0 (socket says 172.21.15.49:56350) 2023-11-06T02:14:59.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.406+0000 7fc20d8ed640 1 -- 172.21.15.49:0/1415978921 learned_addr learned my addr 172.21.15.49:0/1415978921 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:14:59.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.406+0000 7fc20e8ef640 1 --2- 172.21.15.49:0/1415978921 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc208159b40 0x7fc2080c93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:59.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.406+0000 7fc20e0ee640 1 --2- 172.21.15.49:0/1415978921 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc2080cb010 0x7fc2080c4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:59.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.406+0000 7fc20d8ed640 1 -- 172.21.15.49:0/1415978921 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc2080cb010 msgr2=0x7fc2080c4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:59.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.406+0000 7fc20d8ed640 1 --2- 172.21.15.49:0/1415978921 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc2080cb010 0x7fc2080c4d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:59.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.406+0000 7fc20d8ed640 1 -- 172.21.15.49:0/1415978921 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc208159b40 msgr2=0x7fc2080c93a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:59.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.406+0000 7fc20d8ed640 1 --2- 172.21.15.49:0/1415978921 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc208159b40 0x7fc2080c93a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:59.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.406+0000 7fc20d8ed640 1 -- 172.21.15.49:0/1415978921 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1f800b580 con 0x7fc2080cdd50 2023-11-06T02:14:59.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.406+0000 7fc20d8ed640 1 --2- 172.21.15.49:0/1415978921 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2080cdd50 0x7fc2080c2130 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fc1f800fb80 tx=0x7fc1f800d720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:59.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.406+0000 7fc1feffd640 1 -- 172.21.15.49:0/1415978921 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f8018640 con 0x7fc2080cdd50 2023-11-06T02:14:59.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.406+0000 7fc1feffd640 1 -- 172.21.15.49:0/1415978921 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1f80187a0 con 0x7fc2080cdd50 2023-11-06T02:14:59.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.406+0000 7fc1feffd640 1 -- 172.21.15.49:0/1415978921 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f8018b10 con 0x7fc2080cdd50 2023-11-06T02:14:59.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.406+0000 7fc210352640 1 -- 172.21.15.49:0/1415978921 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc20000f050 con 0x7fc2080cdd50 2023-11-06T02:14:59.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.406+0000 7fc210352640 1 -- 172.21.15.49:0/1415978921 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2080b11c0 con 0x7fc2080cdd50 2023-11-06T02:14:59.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.406+0000 7fc210352640 1 -- 172.21.15.49:0/1415978921 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc208074040 con 0x7fc2080cdd50 2023-11-06T02:14:59.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.406+0000 7fc1feffd640 1 -- 172.21.15.49:0/1415978921 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fc1f8018c70 con 0x7fc2080cdd50 2023-11-06T02:14:59.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.406+0000 7fc1feffd640 1 --2- 172.21.15.49:0/1415978921 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc1ec065a60 0x7fc1ec067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:14:59.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.410+0000 7fc1feffd640 1 -- 172.21.15.49:0/1415978921 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fc1f8083c90 con 0x7fc2080cdd50 2023-11-06T02:14:59.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.410+0000 7fc20e0ee640 1 --2- 172.21.15.49:0/1415978921 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc1ec065a60 0x7fc1ec067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:14:59.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.410+0000 7fc20e0ee640 1 --2- 172.21.15.49:0/1415978921 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc1ec065a60 0x7fc1ec067f20 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fc204000fc0 tx=0x7fc204007c60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:14:59.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.410+0000 7fc1feffd640 1 -- 172.21.15.49:0/1415978921 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fc1f80518e0 con 0x7fc2080cdd50 2023-11-06T02:14:59.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.570+0000 7fc210352640 1 -- 172.21.15.49:0/1415978921 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc2080c8e80 con 0x7fc2080cdd50 2023-11-06T02:14:59.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.570+0000 7fc1feffd640 1 -- 172.21.15.49:0/1415978921 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+14561 (secure 0 0 0) 0x7fc1f801d020 con 0x7fc2080cdd50 2023-11-06T02:14:59.567 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:14:59.567 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":85,"fsid":"1a8b035c-6ba4-499a-b876-aea47fd9c0d4","created":"2023-11-06T02:08:06.440857+0000","modified":"2023-11-06T02:14:56.493461+0000","last_up_change":"2023-11-06T02:08:17.982372+0000","last_in_change":"2023-11-06T02:08:08.906631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T02:08:19.252222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T02:08:24.776211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ad55af8-0571-4e6a-8722-b5d429b7496d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6809","nonce":38116},{"type":"v1","addr":"172.21.15.49:6811","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6813","nonce":38116},{"type":"v1","addr":"172.21.15.49:6815","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6821","nonce":38116},{"type":"v1","addr":"172.21.15.49:6823","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6817","nonce":38116},{"type":"v1","addr":"172.21.15.49:6819","nonce":38116}]},"public_addr":"172.21.15.49:6811/38116","cluster_addr":"172.21.15.49:6815/38116","heartbeat_back_addr":"172.21.15.49:6823/38116","heartbeat_front_addr":"172.21.15.49:6819/38116","state":["exists","up"]},{"osd":1,"uuid":"c8b31f85-a757-43ed-9419-351acc46611b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":38113},{"type":"v1","addr":"172.21.15.49:6810","nonce":38113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":38113},{"type":"v1","addr":"172.21.15.49:6814","nonce":38113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":38113},{"type":"v1","addr":"172.21.15.49:6822","nonce":38113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":38113},{"type":"v1","addr":"172.21.15.49:6818","nonce":38113}]},"public_addr":"172.21.15.49:6810/38113","cluster_addr":"172.21.15.49:6814/38113","heartbeat_back_addr":"172.21.15.49:6822/38113","heartbeat_front_addr":"172.21.15.49:6818/38113","state":["exists","up"]},{"osd":2,"uuid":"555b3bf3-0a70-4b91-8efd-e4204b5fc26e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":38114},{"type":"v1","addr":"172.21.15.49:6801","nonce":38114}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":38114},{"type":"v1","addr":"172.21.15.49:6803","nonce":38114}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":38114},{"type":"v1","addr":"172.21.15.49:6807","nonce":38114}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":38114},{"type":"v1","addr":"172.21.15.49:6805","nonce":38114}]},"public_addr":"172.21.15.49:6801/38114","cluster_addr":"172.21.15.49:6803/38114","heartbeat_back_addr":"172.21.15.49:6807/38114","heartbeat_front_addr":"172.21.15.49:6805/38114","state":["exists","up"]},{"osd":3,"uuid":"8d4e5d77-03ef-4b9b-bfcb-5ae93a46795c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":38115},{"type":"v1","addr":"172.21.15.49:6825","nonce":38115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":38115},{"type":"v1","addr":"172.21.15.49:6827","nonce":38115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":38115},{"type":"v1","addr":"172.21.15.49:6831","nonce":38115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":38115},{"type":"v1","addr":"172.21.15.49:6829","nonce":38115}]},"public_addr":"172.21.15.49:6825/38115","cluster_addr":"172.21.15.49:6827/38115","heartbeat_back_addr":"172.21.15.49:6831/38115","heartbeat_front_addr":"172.21.15.49:6829/38115","state":["exists","up"]},{"osd":4,"uuid":"dba03ad4-68a3-4cf3-9218-ac611f02eacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":38417},{"type":"v1","addr":"172.21.15.150:6809","nonce":38417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":38417},{"type":"v1","addr":"172.21.15.150:6811","nonce":38417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":38417},{"type":"v1","addr":"172.21.15.150:6815","nonce":38417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":38417},{"type":"v1","addr":"172.21.15.150:6813","nonce":38417}]},"public_addr":"172.21.15.150:6809/38417","cluster_addr":"172.21.15.150:6811/38417","heartbeat_back_addr":"172.21.15.150:6815/38417","heartbeat_front_addr":"172.21.15.150:6813/38417","state":["exists","up"]},{"osd":5,"uuid":"0df9c303-c255-4640-8e91-01f1c3efe18e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":38418},{"type":"v1","addr":"172.21.15.150:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":38418},{"type":"v1","addr":"172.21.15.150:6803","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":38418},{"type":"v1","addr":"172.21.15.150:6807","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":38418},{"type":"v1","addr":"172.21.15.150:6805","nonce":38418}]},"public_addr":"172.21.15.150:6801/38418","cluster_addr":"172.21.15.150:6803/38418","heartbeat_back_addr":"172.21.15.150:6807/38418","heartbeat_front_addr":"172.21.15.150:6805/38418","state":["exists","up"]},{"osd":6,"uuid":"e59fa189-4b70-4c06-9722-c02b90828bf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":38428},{"type":"v1","addr":"172.21.15.150:6825","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":38428},{"type":"v1","addr":"172.21.15.150:6827","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":38428},{"type":"v1","addr":"172.21.15.150:6831","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":38428},{"type":"v1","addr":"172.21.15.150:6829","nonce":38428}]},"public_addr":"172.21.15.150:6825/38428","cluster_addr":"172.21.15.150:6827/38428","heartbeat_back_addr":"172.21.15.150:6831/38428","heartbeat_front_addr":"172.21.15.150:6829/38428","state":["exists","up"]},{"osd":7,"uuid":"a64f57e1-ef98-46bd-b304-385759e95898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":38421},{"type":"v1","addr":"172.21.15.150:6817","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":38421},{"type":"v1","addr":"172.21.15.150:6819","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":38421},{"type":"v1","addr":"172.21.15.150:6823","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":38421},{"type":"v1","addr":"172.21.15.150:6821","nonce":38421}]},"public_addr":"172.21.15.150:6817/38421","cluster_addr":"172.21.15.150:6819/38421","heartbeat_back_addr":"172.21.15.150:6823/38421","heartbeat_front_addr":"172.21.15.150:6821/38421","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.438552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.413280+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.296466+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.424966+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.276953+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.230984+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.410235+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T02:08:14.480084+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:6835/2241629386":"2023-11-07T02:14:53.395711+0000","172.21.15.49:6834/2241629386":"2023-11-07T02:14:53.395711+0000","172.21.15.49:6835/4037104163":"2023-11-07T02:14:35.347551+0000","172.21.15.49:6834/4037104163":"2023-11-07T02:14:35.347551+0000","172.21.15.49:6835/1130379293":"2023-11-07T02:14:13.284419+0000","172.21.15.49:6834/1130379293":"2023-11-07T02:14:13.284419+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T02:14:59.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.574+0000 7fc210352640 1 -- 172.21.15.49:0/1415978921 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc1ec065a60 msgr2=0x7fc1ec067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:59.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.574+0000 7fc210352640 1 --2- 172.21.15.49:0/1415978921 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc1ec065a60 0x7fc1ec067f20 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fc204000fc0 tx=0x7fc204007c60 comp rx=0 tx=0).stop 2023-11-06T02:14:59.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.574+0000 7fc210352640 1 -- 172.21.15.49:0/1415978921 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2080cdd50 msgr2=0x7fc2080c2130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:14:59.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.574+0000 7fc210352640 1 --2- 172.21.15.49:0/1415978921 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2080cdd50 0x7fc2080c2130 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fc1f800fb80 tx=0x7fc1f800d720 comp rx=0 tx=0).stop 2023-11-06T02:14:59.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.574+0000 7fc210352640 1 -- 172.21.15.49:0/1415978921 shutdown_connections 2023-11-06T02:14:59.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.574+0000 7fc210352640 1 --2- 172.21.15.49:0/1415978921 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc1ec065a60 0x7fc1ec067f20 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:59.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.574+0000 7fc210352640 1 --2- 172.21.15.49:0/1415978921 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2080cdd50 0x7fc2080c2130 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:59.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.574+0000 7fc210352640 1 --2- 172.21.15.49:0/1415978921 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc208159b40 0x7fc2080c93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:59.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.574+0000 7fc210352640 1 --2- 172.21.15.49:0/1415978921 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc2080cb010 0x7fc2080c4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:14:59.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.574+0000 7fc210352640 1 -- 172.21.15.49:0/1415978921 >> 172.21.15.49:0/1415978921 conn(0x7fc20806c0a0 msgr2=0x7fc2080aec50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:14:59.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.574+0000 7fc210352640 1 -- 172.21.15.49:0/1415978921 shutdown_connections 2023-11-06T02:14:59.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:14:59.574+0000 7fc210352640 1 -- 172.21.15.49:0/1415978921 wait complete. 2023-11-06T02:15:00.586 INFO:tasks.ceph:Scrubbing osd.0 2023-11-06T02:15:00.587 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-11-06T02:15:00.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.734+0000 7f4188e50640 1 Processor -- start 2023-11-06T02:15:00.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.734+0000 7f4188e50640 1 -- start start 2023-11-06T02:15:00.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.734+0000 7f4188e50640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f4184153ce0 0x7f4184160f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:00.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.734+0000 7f4188e50640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4184160790 con 0x7f4184153900 2023-11-06T02:15:00.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.734+0000 7f4188e50640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f418415d9a0 con 0x7f41840b8710 2023-11-06T02:15:00.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.734+0000 7f4188e50640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4184160b00 con 0x7f4184153ce0 2023-11-06T02:15:00.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.734+0000 7f418259c640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f4184153900 0x7f418415d100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:35094/0 (socket says 172.21.15.49:35094) 2023-11-06T02:15:00.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.734+0000 7f418259c640 1 -- 172.21.15.49:0/3304727225 learned_addr learned my addr 172.21.15.49:0/3304727225 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:15:00.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.734+0000 7f4181d9b640 1 --2- 172.21.15.49:0/3304727225 >> v2:172.21.15.49:3300/0 conn(0x7f4184153ce0 0x7f4184160f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:00.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.734+0000 7f418159a640 1 -- 172.21.15.49:0/3304727225 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1290692334 0 0) 0x7f4184160790 con 0x7f4184153900 2023-11-06T02:15:00.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.734+0000 7f418159a640 1 -- 172.21.15.49:0/3304727225 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4164003530 con 0x7f4184153900 2023-11-06T02:15:00.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.738+0000 7f4181d9b640 1 -- 172.21.15.49:0/3304727225 >> v1:172.21.15.150:6789/0 conn(0x7f41840b8710 legacy=0x7f41840cd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:00.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.738+0000 7f4181d9b640 1 -- 172.21.15.49:0/3304727225 >> v1:172.21.15.49:6789/0 conn(0x7f4184153900 legacy=0x7f418415d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:00.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.738+0000 7f4181d9b640 1 -- 172.21.15.49:0/3304727225 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f418415dc80 con 0x7f4184153ce0 2023-11-06T02:15:00.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.738+0000 7f4181d9b640 1 --2- 172.21.15.49:0/3304727225 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4184153ce0 0x7f4184160f20 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f416c0097f0 tx=0x7f416c01c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1679245afc9f6c02 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:15:00.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.738+0000 7f418159a640 1 -- 172.21.15.49:0/3304727225 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f416c004230 con 0x7f4184153ce0 2023-11-06T02:15:00.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.738+0000 7f418159a640 1 -- 172.21.15.49:0/3304727225 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f416c0a7050 con 0x7f4184153ce0 2023-11-06T02:15:00.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.738+0000 7f418159a640 1 -- 172.21.15.49:0/3304727225 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f416c003620 con 0x7f4184153ce0 2023-11-06T02:15:00.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.738+0000 7f4188e50640 1 -- 172.21.15.49:0/3304727225 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4184153ce0 msgr2=0x7f4184160f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:00.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.738+0000 7f4188e50640 1 --2- 172.21.15.49:0/3304727225 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4184153ce0 0x7f4184160f20 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f416c0097f0 tx=0x7f416c01c330 comp rx=0 tx=0).stop 2023-11-06T02:15:00.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.738+0000 7f4188e50640 1 -- 172.21.15.49:0/3304727225 shutdown_connections 2023-11-06T02:15:00.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.738+0000 7f4188e50640 1 --2- 172.21.15.49:0/3304727225 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4184153ce0 0x7f4184160f20 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:00.736 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.738+0000 7f4188e50640 1 -- 172.21.15.49:0/3304727225 >> 172.21.15.49:0/3304727225 conn(0x7f41840714a0 msgr2=0x7f41840718b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:15:00.736 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.738+0000 7f4188e50640 1 -- 172.21.15.49:0/3304727225 shutdown_connections 2023-11-06T02:15:00.736 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.738+0000 7f4188e50640 1 -- 172.21.15.49:0/3304727225 wait complete. 2023-11-06T02:15:00.739 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.742+0000 7f4188e50640 1 Processor -- start 2023-11-06T02:15:00.739 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.742+0000 7f4188e50640 1 -- start start 2023-11-06T02:15:00.739 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.742+0000 7f4188e50640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f41840b8710 0x7f41840c8f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:00.740 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.742+0000 7f4188e50640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4184153900 0x7f4184155030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:00.740 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.742+0000 7f4188e50640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4184153ce0 0x7f418414d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:00.740 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.742+0000 7f4188e50640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41840c94d0 con 0x7f4184153ce0 2023-11-06T02:15:00.740 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.742+0000 7f4188e50640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4184156500 con 0x7f41840b8710 2023-11-06T02:15:00.740 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.742+0000 7f4188e50640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4184154290 con 0x7f4184153900 2023-11-06T02:15:00.741 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.742+0000 7f4182d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4184153ce0 0x7f418414d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:00.741 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.742+0000 7f4182d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4184153ce0 0x7f418414d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56360/0 (socket says 172.21.15.49:56360) 2023-11-06T02:15:00.741 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.742+0000 7f4182d9d640 1 -- 172.21.15.49:0/1267878170 learned_addr learned my addr 172.21.15.49:0/1267878170 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:15:00.741 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.742+0000 7f418259c640 1 --2- 172.21.15.49:0/1267878170 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f41840b8710 0x7f41840c8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:00.741 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.742+0000 7f4181d9b640 1 --2- 172.21.15.49:0/1267878170 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4184153900 0x7f4184155030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:00.742 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.742+0000 7f4182d9d640 1 -- 172.21.15.49:0/1267878170 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4184153900 msgr2=0x7f4184155030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:00.742 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.742+0000 7f4182d9d640 1 --2- 172.21.15.49:0/1267878170 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4184153900 0x7f4184155030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:00.742 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.742+0000 7f4182d9d640 1 -- 172.21.15.49:0/1267878170 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f41840b8710 msgr2=0x7f41840c8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:00.742 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.742+0000 7f4182d9d640 1 --2- 172.21.15.49:0/1267878170 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f41840b8710 0x7f41840c8f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:00.742 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.742+0000 7f4182d9d640 1 -- 172.21.15.49:0/1267878170 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f416c009420 con 0x7f4184153ce0 2023-11-06T02:15:00.743 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.742+0000 7f4182d9d640 1 --2- 172.21.15.49:0/1267878170 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4184153ce0 0x7f418414d830 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f417c00c6c0 tx=0x7f417c00cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:15:00.743 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.742+0000 7f417affd640 1 -- 172.21.15.49:0/1267878170 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f417c003b80 con 0x7f4184153ce0 2023-11-06T02:15:00.743 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.742+0000 7f417affd640 1 -- 172.21.15.49:0/1267878170 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f417c003ce0 con 0x7f4184153ce0 2023-11-06T02:15:00.743 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.742+0000 7f4188e50640 1 -- 172.21.15.49:0/1267878170 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4184154550 con 0x7f4184153ce0 2023-11-06T02:15:00.743 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.742+0000 7f417affd640 1 -- 172.21.15.49:0/1267878170 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f417c0052f0 con 0x7f4184153ce0 2023-11-06T02:15:00.744 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.742+0000 7f4188e50640 1 -- 172.21.15.49:0/1267878170 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4184154b10 con 0x7f4184153ce0 2023-11-06T02:15:00.744 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.742+0000 7f4188e50640 1 -- 172.21.15.49:0/1267878170 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f41840d13e0 con 0x7f4184153ce0 2023-11-06T02:15:00.744 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.742+0000 7f417affd640 1 -- 172.21.15.49:0/1267878170 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f417c005450 con 0x7f4184153ce0 2023-11-06T02:15:00.744 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.746+0000 7f417affd640 1 --2- 172.21.15.49:0/1267878170 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4168065a90 0x7f4168067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:00.744 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.746+0000 7f417affd640 1 -- 172.21.15.49:0/1267878170 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f417c0837f0 con 0x7f4184153ce0 2023-11-06T02:15:00.744 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.746+0000 7f417affd640 1 --2- 172.21.15.49:0/1267878170 >> [v2:172.21.15.49:6809/38116,v1:172.21.15.49:6811/38116] conn(0x7f416806c050 0x7f416806e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:00.745 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.746+0000 7f417affd640 1 -- 172.21.15.49:0/1267878170 --> [v2:172.21.15.49:6809/38116,v1:172.21.15.49:6811/38116] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f416806eb20 con 0x7f416806c050 2023-11-06T02:15:00.745 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.746+0000 7f417affd640 1 -- 172.21.15.49:0/1267878170 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f417c083c40 con 0x7f4184153ce0 2023-11-06T02:15:00.745 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.746+0000 7f4181d9b640 1 --2- 172.21.15.49:0/1267878170 >> [v2:172.21.15.49:6809/38116,v1:172.21.15.49:6811/38116] conn(0x7f416806c050 0x7f416806e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:00.746 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.746+0000 7f418259c640 1 --2- 172.21.15.49:0/1267878170 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4168065a90 0x7f4168067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:00.746 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.746+0000 7f4181d9b640 1 --2- 172.21.15.49:0/1267878170 >> [v2:172.21.15.49:6809/38116,v1:172.21.15.49:6811/38116] conn(0x7f416806c050 0x7f416806e470 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:15:00.746 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.746+0000 7f418259c640 1 --2- 172.21.15.49:0/1267878170 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4168065a90 0x7f4168067f50 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f41740011f0 tx=0x7f4174009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:15:00.747 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.746+0000 7f417affd640 1 -- 172.21.15.49:0/1267878170 <== osd.0 v2:172.21.15.49:6809/38116 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f416806eb20 con 0x7f416806c050 2023-11-06T02:15:00.753 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.758+0000 7f4188e50640 1 -- 172.21.15.49:0/1267878170 --> [v2:172.21.15.49:6809/38116,v1:172.21.15.49:6811/38116] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f4184154d00 con 0x7f416806c050 2023-11-06T02:15:00.753 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.758+0000 7f417affd640 1 -- 172.21.15.49:0/1267878170 <== osd.0 v2:172.21.15.49:6809/38116 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f4184154d00 con 0x7f416806c050 2023-11-06T02:15:00.754 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-11-06T02:15:00.754 INFO:teuthology.orchestra.run.smithi049.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-06T02:15:00.754 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-11-06T02:15:00.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.758+0000 7f4188e50640 1 -- 172.21.15.49:0/1267878170 >> [v2:172.21.15.49:6809/38116,v1:172.21.15.49:6811/38116] conn(0x7f416806c050 msgr2=0x7f416806e470 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:00.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.758+0000 7f4188e50640 1 --2- 172.21.15.49:0/1267878170 >> [v2:172.21.15.49:6809/38116,v1:172.21.15.49:6811/38116] conn(0x7f416806c050 0x7f416806e470 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:00.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.758+0000 7f4188e50640 1 -- 172.21.15.49:0/1267878170 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4168065a90 msgr2=0x7f4168067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:00.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.758+0000 7f4188e50640 1 --2- 172.21.15.49:0/1267878170 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4168065a90 0x7f4168067f50 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f41740011f0 tx=0x7f4174009040 comp rx=0 tx=0).stop 2023-11-06T02:15:00.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.758+0000 7f4188e50640 1 -- 172.21.15.49:0/1267878170 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4184153ce0 msgr2=0x7f418414d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:00.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.758+0000 7f4188e50640 1 --2- 172.21.15.49:0/1267878170 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4184153ce0 0x7f418414d830 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f417c00c6c0 tx=0x7f417c00cb80 comp rx=0 tx=0).stop 2023-11-06T02:15:00.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.758+0000 7f4182d9d640 1 -- 172.21.15.49:0/1267878170 reap_dead start 2023-11-06T02:15:00.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.758+0000 7f4188e50640 1 -- 172.21.15.49:0/1267878170 shutdown_connections 2023-11-06T02:15:00.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.758+0000 7f4188e50640 1 -- 172.21.15.49:0/1267878170 >> 172.21.15.49:0/1267878170 conn(0x7f41840714a0 msgr2=0x7f41840b7c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:15:00.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.758+0000 7f4188e50640 1 -- 172.21.15.49:0/1267878170 shutdown_connections 2023-11-06T02:15:00.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.758+0000 7f4188e50640 1 -- 172.21.15.49:0/1267878170 wait complete. 2023-11-06T02:15:00.771 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-11-06T02:15:00.888 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.894+0000 7f61ca03e640 1 Processor -- start 2023-11-06T02:15:00.888 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.894+0000 7f61ca03e640 1 -- start start 2023-11-06T02:15:00.889 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.894+0000 7f61ca03e640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f61c40cda70 0x7f61c4161650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:00.889 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.894+0000 7f61ca03e640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61c40d17c0 con 0x7f61c40714a0 2023-11-06T02:15:00.890 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.894+0000 7f61ca03e640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61c40ce6c0 con 0x7f61c40b1f90 2023-11-06T02:15:00.890 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.894+0000 7f61ca03e640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61c40cdf20 con 0x7f61c40cda70 2023-11-06T02:15:00.890 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.894+0000 7f61c3fff640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f61c40714a0 0x7f61c40718b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:35102/0 (socket says 172.21.15.49:35102) 2023-11-06T02:15:00.890 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.894+0000 7f61c3fff640 1 -- 172.21.15.49:0/1587476537 learned_addr learned my addr 172.21.15.49:0/1587476537 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:15:00.890 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.894+0000 7f61c2ffd640 1 --2- 172.21.15.49:0/1587476537 >> v2:172.21.15.150:3300/0 conn(0x7f61c40cda70 0x7f61c4161650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:00.891 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.894+0000 7f61c27fc640 1 -- 172.21.15.49:0/1587476537 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4158404296 0 0) 0x7f61c40d17c0 con 0x7f61c40714a0 2023-11-06T02:15:00.891 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.894+0000 7f61c27fc640 1 -- 172.21.15.49:0/1587476537 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61ac003530 con 0x7f61c40714a0 2023-11-06T02:15:00.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.894+0000 7f61c2ffd640 1 -- 172.21.15.49:0/1587476537 >> v1:172.21.15.150:6789/0 conn(0x7f61c40b1f90 legacy=0x7f61c40d1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:00.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.894+0000 7f61c2ffd640 1 -- 172.21.15.49:0/1587476537 >> v1:172.21.15.49:6789/0 conn(0x7f61c40714a0 legacy=0x7f61c40718b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:00.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.894+0000 7f61c27fc640 1 -- 172.21.15.49:0/1587476537 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2237579294 0 0) 0x7f61ac003530 con 0x7f61c40714a0 2023-11-06T02:15:00.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.894+0000 7f61c2ffd640 1 -- 172.21.15.49:0/1587476537 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61c40d2360 con 0x7f61c40cda70 2023-11-06T02:15:00.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.898+0000 7f61c2ffd640 1 --2- 172.21.15.49:0/1587476537 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61c40cda70 0x7f61c4161650 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f61b80097c0 tx=0x7f61b801c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc0663ec1efdb5ec server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:15:00.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.898+0000 7f61c27fc640 1 -- 172.21.15.49:0/1587476537 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b80a4ea0 con 0x7f61c40cda70 2023-11-06T02:15:00.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.898+0000 7f61c27fc640 1 -- 172.21.15.49:0/1587476537 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61b80a7050 con 0x7f61c40cda70 2023-11-06T02:15:00.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.898+0000 7f61c27fc640 1 -- 172.21.15.49:0/1587476537 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61bc001a00 con 0x7f61c40cda70 2023-11-06T02:15:00.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.898+0000 7f61ca03e640 1 -- 172.21.15.49:0/1587476537 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61c40cda70 msgr2=0x7f61c4161650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:00.895 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.898+0000 7f61ca03e640 1 --2- 172.21.15.49:0/1587476537 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61c40cda70 0x7f61c4161650 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f61b80097c0 tx=0x7f61b801c3a0 comp rx=0 tx=0).stop 2023-11-06T02:15:00.895 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.898+0000 7f61ca03e640 1 -- 172.21.15.49:0/1587476537 shutdown_connections 2023-11-06T02:15:00.895 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.898+0000 7f61ca03e640 1 --2- 172.21.15.49:0/1587476537 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61c40cda70 0x7f61c4161650 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:00.895 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.898+0000 7f61ca03e640 1 -- 172.21.15.49:0/1587476537 >> 172.21.15.49:0/1587476537 conn(0x7f61c40d2aa0 msgr2=0x7f61c40d2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:15:00.895 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.898+0000 7f61ca03e640 1 -- 172.21.15.49:0/1587476537 shutdown_connections 2023-11-06T02:15:00.896 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.898+0000 7f61ca03e640 1 -- 172.21.15.49:0/1587476537 wait complete. 2023-11-06T02:15:00.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.902+0000 7f61ca03e640 1 Processor -- start 2023-11-06T02:15:00.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.902+0000 7f61ca03e640 1 -- start start 2023-11-06T02:15:00.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.902+0000 7f61ca03e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61c40714a0 0x7f61c4157fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:00.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.902+0000 7f61ca03e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61c40b1f90 0x7f61c4158610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:00.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.902+0000 7f61ca03e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61c40cda70 0x7f61c40cab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:00.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.902+0000 7f61ca03e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c415d1a0 con 0x7f61c40714a0 2023-11-06T02:15:00.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.902+0000 7f61ca03e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c4158b50 con 0x7f61c40cda70 2023-11-06T02:15:00.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.902+0000 7f61ca03e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61c415d3f0 con 0x7f61c40b1f90 2023-11-06T02:15:00.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.902+0000 7f61c37fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61c40714a0 0x7f61c4157fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:00.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.902+0000 7f61c37fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61c40714a0 0x7f61c4157fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56366/0 (socket says 172.21.15.49:56366) 2023-11-06T02:15:00.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.902+0000 7f61c37fe640 1 -- 172.21.15.49:0/177422255 learned_addr learned my addr 172.21.15.49:0/177422255 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:15:00.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.902+0000 7f61c2ffd640 1 --2- 172.21.15.49:0/177422255 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61c40b1f90 0x7f61c4158610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:00.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.902+0000 7f61c37fe640 1 -- 172.21.15.49:0/177422255 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61c40b1f90 msgr2=0x7f61c4158610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:00.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.902+0000 7f61c3fff640 1 --2- 172.21.15.49:0/177422255 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61c40cda70 0x7f61c40cab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:00.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.902+0000 7f61c37fe640 1 --2- 172.21.15.49:0/177422255 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61c40b1f90 0x7f61c4158610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:00.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.902+0000 7f61c37fe640 1 -- 172.21.15.49:0/177422255 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61c40cda70 msgr2=0x7f61c40cab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:00.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.902+0000 7f61c37fe640 1 --2- 172.21.15.49:0/177422255 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61c40cda70 0x7f61c40cab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:00.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.902+0000 7f61c37fe640 1 -- 172.21.15.49:0/177422255 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61b8009420 con 0x7f61c40714a0 2023-11-06T02:15:00.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.902+0000 7f61c37fe640 1 --2- 172.21.15.49:0/177422255 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61c40714a0 0x7f61c4157fd0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f61b400c500 tx=0x7f61b400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:15:00.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.902+0000 7f61abfff640 1 -- 172.21.15.49:0/177422255 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b4005220 con 0x7f61c40714a0 2023-11-06T02:15:00.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.902+0000 7f61abfff640 1 -- 172.21.15.49:0/177422255 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61b4005380 con 0x7f61c40714a0 2023-11-06T02:15:00.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.902+0000 7f61abfff640 1 -- 172.21.15.49:0/177422255 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b4007270 con 0x7f61c40714a0 2023-11-06T02:15:00.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.902+0000 7f61ca03e640 1 -- 172.21.15.49:0/177422255 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61c415d6d0 con 0x7f61c40714a0 2023-11-06T02:15:00.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.902+0000 7f61ca03e640 1 -- 172.21.15.49:0/177422255 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61c415dc90 con 0x7f61c40714a0 2023-11-06T02:15:00.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.902+0000 7f61ca03e640 1 -- 172.21.15.49:0/177422255 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61c4074040 con 0x7f61c40714a0 2023-11-06T02:15:00.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.902+0000 7f61abfff640 1 -- 172.21.15.49:0/177422255 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f61b40073d0 con 0x7f61c40714a0 2023-11-06T02:15:00.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.902+0000 7f61abfff640 1 --2- 172.21.15.49:0/177422255 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f61b0065a90 0x7f61b0067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:00.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.906+0000 7f61abfff640 1 -- 172.21.15.49:0/177422255 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f61b4082950 con 0x7f61c40714a0 2023-11-06T02:15:00.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.906+0000 7f61c2ffd640 1 --2- 172.21.15.49:0/177422255 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f61b0065a90 0x7f61b0067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:00.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.906+0000 7f61c2ffd640 1 --2- 172.21.15.49:0/177422255 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f61b0065a90 0x7f61b0067f50 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f61b80ad000 tx=0x7f61b80a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:15:00.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:00.910+0000 7f61abfff640 1 -- 172.21.15.49:0/177422255 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f61b4050410 con 0x7f61c40714a0 2023-11-06T02:15:01.059 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.066+0000 7f61ca03e640 1 -- 172.21.15.49:0/177422255 --> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f61c40d1450 con 0x7f61b0065a90 2023-11-06T02:15:01.060 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.066+0000 7f61abfff640 1 -- 172.21.15.49:0/177422255 <== mgr.4100 v2:172.21.15.150:6832/38104 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f61b80ba090 con 0x7f61b0065a90 2023-11-06T02:15:01.060 INFO:teuthology.orchestra.run.smithi049.stderr:instructed osd(s) 0 to deep-scrub 2023-11-06T02:15:01.062 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.066+0000 7f61ca03e640 1 -- 172.21.15.49:0/177422255 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f61b0065a90 msgr2=0x7f61b0067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:01.062 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.066+0000 7f61ca03e640 1 --2- 172.21.15.49:0/177422255 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f61b0065a90 0x7f61b0067f50 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f61b80ad000 tx=0x7f61b80a6000 comp rx=0 tx=0).stop 2023-11-06T02:15:01.062 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.066+0000 7f61ca03e640 1 -- 172.21.15.49:0/177422255 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61c40714a0 msgr2=0x7f61c4157fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:01.063 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.066+0000 7f61ca03e640 1 --2- 172.21.15.49:0/177422255 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61c40714a0 0x7f61c4157fd0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f61b400c500 tx=0x7f61b400c9c0 comp rx=0 tx=0).stop 2023-11-06T02:15:01.063 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.066+0000 7f61ca03e640 1 -- 172.21.15.49:0/177422255 shutdown_connections 2023-11-06T02:15:01.063 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.066+0000 7f61ca03e640 1 --2- 172.21.15.49:0/177422255 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f61b0065a90 0x7f61b0067f50 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:01.063 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.066+0000 7f61ca03e640 1 --2- 172.21.15.49:0/177422255 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61c40cda70 0x7f61c40cab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:01.063 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.066+0000 7f61ca03e640 1 --2- 172.21.15.49:0/177422255 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61c40b1f90 0x7f61c4158610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:01.064 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.066+0000 7f61ca03e640 1 --2- 172.21.15.49:0/177422255 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61c40714a0 0x7f61c4157fd0 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:01.064 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.066+0000 7f61ca03e640 1 -- 172.21.15.49:0/177422255 >> 172.21.15.49:0/177422255 conn(0x7f61c40d2aa0 msgr2=0x7f61c4160a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:15:01.064 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.066+0000 7f61ca03e640 1 -- 172.21.15.49:0/177422255 shutdown_connections 2023-11-06T02:15:01.064 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.066+0000 7f61ca03e640 1 -- 172.21.15.49:0/177422255 wait complete. 2023-11-06T02:15:01.079 INFO:tasks.ceph:Scrubbing osd.1 2023-11-06T02:15:01.079 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-11-06T02:15:01.209 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.214+0000 7fa701800640 1 Processor -- start 2023-11-06T02:15:01.209 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.214+0000 7fa701800640 1 -- start start 2023-11-06T02:15:01.210 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.214+0000 7fa701800640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa6fc153900 0x7fa6fc15d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:01.211 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.214+0000 7fa701800640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6fc153ce0 0x7fa6fc160f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:01.211 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.214+0000 7fa701800640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6fc160790 con 0x7fa6fc0b8710 2023-11-06T02:15:01.211 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.214+0000 7fa701800640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6fc160b00 con 0x7fa6fc153900 2023-11-06T02:15:01.211 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.214+0000 7fa701800640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6fc15d9a0 con 0x7fa6fc153ce0 2023-11-06T02:15:01.211 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.214+0000 7fa6faffd640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa6fc153900 0x7fa6fc15d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:01.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.214+0000 7fa6fb7fe640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fa6fc0b8710 0x7fa6fc0cd790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:35110/0 (socket says 172.21.15.49:35110) 2023-11-06T02:15:01.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.214+0000 7fa6faffd640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa6fc153900 0x7fa6fc15d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56378/0 (socket says 172.21.15.49:56378) 2023-11-06T02:15:01.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.214+0000 7fa6fb7fe640 1 -- 172.21.15.49:0/1313125390 learned_addr learned my addr 172.21.15.49:0/1313125390 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:15:01.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.214+0000 7fa6fa7fc640 1 --2- 172.21.15.49:0/1313125390 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6fc153ce0 0x7fa6fc160f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:01.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.214+0000 7fa6faffd640 1 -- 172.21.15.49:0/1313125390 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6fc153ce0 msgr2=0x7fa6fc160f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:15:01.213 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.214+0000 7fa6faffd640 1 --2- 172.21.15.49:0/1313125390 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6fc153ce0 0x7fa6fc160f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:01.213 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.214+0000 7fa6f9ffb640 1 -- 172.21.15.49:0/1313125390 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3842374281 0 0) 0x7fa6fc160790 con 0x7fa6fc0b8710 2023-11-06T02:15:01.213 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.214+0000 7fa6faffd640 1 -- 172.21.15.49:0/1313125390 >> v1:172.21.15.49:6789/0 conn(0x7fa6fc0b8710 legacy=0x7fa6fc0cd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:01.213 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.214+0000 7fa6faffd640 1 -- 172.21.15.49:0/1313125390 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6fc15dc80 con 0x7fa6fc153900 2023-11-06T02:15:01.215 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.218+0000 7fa6faffd640 1 --2- 172.21.15.49:0/1313125390 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6fc153900 0x7fa6fc15d100 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec009d90 tx=0x7fa6ec01c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df48de3ed4cdf0bc server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:15:01.215 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.218+0000 7fa6f9ffb640 1 -- 172.21.15.49:0/1313125390 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec0a58e0 con 0x7fa6fc153900 2023-11-06T02:15:01.215 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.218+0000 7fa6f9ffb640 1 -- 172.21.15.49:0/1313125390 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6ec0a5a40 con 0x7fa6fc153900 2023-11-06T02:15:01.215 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.218+0000 7fa6f9ffb640 1 -- 172.21.15.49:0/1313125390 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec0a5dd0 con 0x7fa6fc153900 2023-11-06T02:15:01.215 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.218+0000 7fa701800640 1 -- 172.21.15.49:0/1313125390 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6fc153900 msgr2=0x7fa6fc15d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:01.216 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.218+0000 7fa701800640 1 --2- 172.21.15.49:0/1313125390 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6fc153900 0x7fa6fc15d100 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec009d90 tx=0x7fa6ec01c8c0 comp rx=0 tx=0).stop 2023-11-06T02:15:01.216 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.218+0000 7fa701800640 1 -- 172.21.15.49:0/1313125390 shutdown_connections 2023-11-06T02:15:01.216 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.218+0000 7fa701800640 1 --2- 172.21.15.49:0/1313125390 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6fc153ce0 0x7fa6fc160f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:01.216 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.218+0000 7fa701800640 1 --2- 172.21.15.49:0/1313125390 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6fc153900 0x7fa6fc15d100 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:01.216 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.218+0000 7fa701800640 1 -- 172.21.15.49:0/1313125390 >> 172.21.15.49:0/1313125390 conn(0x7fa6fc0714a0 msgr2=0x7fa6fc0718b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:15:01.217 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.218+0000 7fa701800640 1 -- 172.21.15.49:0/1313125390 shutdown_connections 2023-11-06T02:15:01.217 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.218+0000 7fa701800640 1 -- 172.21.15.49:0/1313125390 wait complete. 2023-11-06T02:15:01.217 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.218+0000 7fa701800640 1 Processor -- start 2023-11-06T02:15:01.217 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.218+0000 7fa701800640 1 -- start start 2023-11-06T02:15:01.220 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.222+0000 7fa701800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa6fc0b8710 0x7fa6fc0b2b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:01.220 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.222+0000 7fa701800640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6fc153900 0x7fa6fc0d1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:01.221 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.222+0000 7fa701800640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6fc153ce0 0x7fa6fc14d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:01.221 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.222+0000 7fa701800640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6fc14d800 con 0x7fa6fc153ce0 2023-11-06T02:15:01.221 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.222+0000 7fa701800640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6fc0c67a0 con 0x7fa6fc0b8710 2023-11-06T02:15:01.221 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.222+0000 7fa701800640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6fc0c6a80 con 0x7fa6fc153900 2023-11-06T02:15:01.221 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.222+0000 7fa6fb7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6fc153ce0 0x7fa6fc14d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:01.222 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.222+0000 7fa6fb7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6fc153ce0 0x7fa6fc14d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56384/0 (socket says 172.21.15.49:56384) 2023-11-06T02:15:01.222 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.222+0000 7fa6fb7fe640 1 -- 172.21.15.49:0/2371232996 learned_addr learned my addr 172.21.15.49:0/2371232996 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:15:01.222 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.222+0000 7fa6fa7fc640 1 --2- 172.21.15.49:0/2371232996 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6fc153900 0x7fa6fc0d1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:01.222 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.222+0000 7fa6faffd640 1 --2- 172.21.15.49:0/2371232996 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa6fc0b8710 0x7fa6fc0b2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:01.222 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.222+0000 7fa6fb7fe640 1 -- 172.21.15.49:0/2371232996 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6fc153900 msgr2=0x7fa6fc0d1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:01.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.222+0000 7fa6fb7fe640 1 --2- 172.21.15.49:0/2371232996 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6fc153900 0x7fa6fc0d1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:01.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.222+0000 7fa6fb7fe640 1 -- 172.21.15.49:0/2371232996 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa6fc0b8710 msgr2=0x7fa6fc0b2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:01.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.222+0000 7fa6fb7fe640 1 --2- 172.21.15.49:0/2371232996 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa6fc0b8710 0x7fa6fc0b2b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:01.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.222+0000 7fa6fb7fe640 1 -- 172.21.15.49:0/2371232996 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6ec009a70 con 0x7fa6fc153ce0 2023-11-06T02:15:01.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.222+0000 7fa6fb7fe640 1 --2- 172.21.15.49:0/2371232996 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6fc153ce0 0x7fa6fc14d2c0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7fa6f400c410 tx=0x7fa6f400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:15:01.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.222+0000 7fa6f37fe640 1 -- 172.21.15.49:0/2371232996 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f40074e0 con 0x7fa6fc153ce0 2023-11-06T02:15:01.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.222+0000 7fa6f37fe640 1 -- 172.21.15.49:0/2371232996 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6f4007640 con 0x7fa6fc153ce0 2023-11-06T02:15:01.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.222+0000 7fa701800640 1 -- 172.21.15.49:0/2371232996 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6fc0c6d60 con 0x7fa6fc153ce0 2023-11-06T02:15:01.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.222+0000 7fa6f37fe640 1 -- 172.21.15.49:0/2371232996 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f4004a30 con 0x7fa6fc153ce0 2023-11-06T02:15:01.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.222+0000 7fa701800640 1 -- 172.21.15.49:0/2371232996 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6fc0c7320 con 0x7fa6fc153ce0 2023-11-06T02:15:01.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.222+0000 7fa701800640 1 -- 172.21.15.49:0/2371232996 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa6fc0cdea0 con 0x7fa6fc153ce0 2023-11-06T02:15:01.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.222+0000 7fa6f37fe640 1 -- 172.21.15.49:0/2371232996 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fa6f4004b90 con 0x7fa6fc153ce0 2023-11-06T02:15:01.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.222+0000 7fa6f37fe640 1 --2- 172.21.15.49:0/2371232996 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa6dc065a60 0x7fa6dc067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:01.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.222+0000 7fa6f37fe640 1 -- 172.21.15.49:0/2371232996 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fa6f40835a0 con 0x7fa6fc153ce0 2023-11-06T02:15:01.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.222+0000 7fa6f37fe640 1 --2- 172.21.15.49:0/2371232996 >> [v2:172.21.15.49:6808/38113,v1:172.21.15.49:6810/38113] conn(0x7fa6dc06c020 0x7fa6dc06e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:01.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.222+0000 7fa6f37fe640 1 -- 172.21.15.49:0/2371232996 --> [v2:172.21.15.49:6808/38113,v1:172.21.15.49:6810/38113] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa6dc06eaf0 con 0x7fa6dc06c020 2023-11-06T02:15:01.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.222+0000 7fa6f37fe640 1 -- 172.21.15.49:0/2371232996 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa6f40839f0 con 0x7fa6fc153ce0 2023-11-06T02:15:01.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.222+0000 7fa6fa7fc640 1 --2- 172.21.15.49:0/2371232996 >> [v2:172.21.15.49:6808/38113,v1:172.21.15.49:6810/38113] conn(0x7fa6dc06c020 0x7fa6dc06e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:01.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.222+0000 7fa6faffd640 1 --2- 172.21.15.49:0/2371232996 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa6dc065a60 0x7fa6dc067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:01.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.226+0000 7fa6fa7fc640 1 --2- 172.21.15.49:0/2371232996 >> [v2:172.21.15.49:6808/38113,v1:172.21.15.49:6810/38113] conn(0x7fa6dc06c020 0x7fa6dc06e440 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:15:01.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.226+0000 7fa6faffd640 1 --2- 172.21.15.49:0/2371232996 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa6dc065a60 0x7fa6dc067f20 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec0af000 tx=0x7fa6ec0a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:15:01.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.226+0000 7fa6f37fe640 1 -- 172.21.15.49:0/2371232996 <== osd.1 v2:172.21.15.49:6808/38113 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fa6dc06eaf0 con 0x7fa6dc06c020 2023-11-06T02:15:01.232 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.238+0000 7fa701800640 1 -- 172.21.15.49:0/2371232996 --> [v2:172.21.15.49:6808/38113,v1:172.21.15.49:6810/38113] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fa6fc0c6bf0 con 0x7fa6dc06c020 2023-11-06T02:15:01.232 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.238+0000 7fa6f37fe640 1 -- 172.21.15.49:0/2371232996 <== osd.1 v2:172.21.15.49:6808/38113 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fa6fc0c6bf0 con 0x7fa6dc06c020 2023-11-06T02:15:01.232 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-11-06T02:15:01.233 INFO:teuthology.orchestra.run.smithi049.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-06T02:15:01.233 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-11-06T02:15:01.234 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.238+0000 7fa701800640 1 -- 172.21.15.49:0/2371232996 >> [v2:172.21.15.49:6808/38113,v1:172.21.15.49:6810/38113] conn(0x7fa6dc06c020 msgr2=0x7fa6dc06e440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:01.234 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.238+0000 7fa701800640 1 --2- 172.21.15.49:0/2371232996 >> [v2:172.21.15.49:6808/38113,v1:172.21.15.49:6810/38113] conn(0x7fa6dc06c020 0x7fa6dc06e440 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:01.234 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.238+0000 7fa701800640 1 -- 172.21.15.49:0/2371232996 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa6dc065a60 msgr2=0x7fa6dc067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:01.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.238+0000 7fa701800640 1 --2- 172.21.15.49:0/2371232996 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fa6dc065a60 0x7fa6dc067f20 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec0af000 tx=0x7fa6ec0a7070 comp rx=0 tx=0).stop 2023-11-06T02:15:01.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.238+0000 7fa701800640 1 -- 172.21.15.49:0/2371232996 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6fc153ce0 msgr2=0x7fa6fc14d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:01.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.238+0000 7fa701800640 1 --2- 172.21.15.49:0/2371232996 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6fc153ce0 0x7fa6fc14d2c0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7fa6f400c410 tx=0x7fa6f400c8d0 comp rx=0 tx=0).stop 2023-11-06T02:15:01.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.238+0000 7fa6fb7fe640 1 -- 172.21.15.49:0/2371232996 reap_dead start 2023-11-06T02:15:01.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.238+0000 7fa701800640 1 -- 172.21.15.49:0/2371232996 shutdown_connections 2023-11-06T02:15:01.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.238+0000 7fa701800640 1 -- 172.21.15.49:0/2371232996 >> 172.21.15.49:0/2371232996 conn(0x7fa6fc0714a0 msgr2=0x7fa6fc15e230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:15:01.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.238+0000 7fa701800640 1 -- 172.21.15.49:0/2371232996 shutdown_connections 2023-11-06T02:15:01.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.238+0000 7fa701800640 1 -- 172.21.15.49:0/2371232996 wait complete. 2023-11-06T02:15:01.249 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-11-06T02:15:01.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.362+0000 7fc60b36e640 1 Processor -- start 2023-11-06T02:15:01.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.362+0000 7fc60b36e640 1 -- start start 2023-11-06T02:15:01.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.362+0000 7fc60b36e640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc60415ab50 0x7fc6040c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:01.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.362+0000 7fc60b36e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc6040cb010 0x7fc6040b0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:01.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.362+0000 7fc60b36e640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc6040b05c0 0x7fc6040700a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:01.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.362+0000 7fc60b36e640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6040b0a50 con 0x7fc60415ab50 2023-11-06T02:15:01.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.362+0000 7fc60b36e640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6040cab80 con 0x7fc6040b05c0 2023-11-06T02:15:01.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.362+0000 7fc60b36e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc60415af30 con 0x7fc6040cb010 2023-11-06T02:15:01.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.362+0000 7fc60910a640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc60415ab50 0x7fc6040c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:01.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.362+0000 7fc60910a640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc60415ab50 0x7fc6040c9bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56386/0 (socket says 172.21.15.49:56386) 2023-11-06T02:15:01.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.362+0000 7fc60910a640 1 -- 172.21.15.49:0/926013913 learned_addr learned my addr 172.21.15.49:0/926013913 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:15:01.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.362+0000 7fc60910a640 1 -- 172.21.15.49:0/926013913 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc6040cb010 msgr2=0x7fc6040b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:15:01.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.362+0000 7fc608909640 1 --2- 172.21.15.49:0/926013913 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc6040cb010 0x7fc6040b0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:01.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.362+0000 7fc60910a640 1 --2- 172.21.15.49:0/926013913 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc6040cb010 0x7fc6040b0080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:01.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.362+0000 7fc60910a640 1 -- 172.21.15.49:0/926013913 >> v2:172.21.15.150:3300/0 conn(0x7fc6040b05c0 msgr2=0x7fc6040700a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:15:01.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.362+0000 7fc60990b640 1 --2- 172.21.15.49:0/926013913 >> v2:172.21.15.150:3300/0 conn(0x7fc6040b05c0 0x7fc6040700a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:01.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.362+0000 7fc60910a640 1 --2- 172.21.15.49:0/926013913 >> v2:172.21.15.150:3300/0 conn(0x7fc6040b05c0 0x7fc6040700a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:01.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.362+0000 7fc60910a640 1 -- 172.21.15.49:0/926013913 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6040cb5b0 con 0x7fc60415ab50 2023-11-06T02:15:01.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.362+0000 7fc60910a640 1 --2- 172.21.15.49:0/926013913 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc60415ab50 0x7fc6040c9bb0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7fc6040cae30 tx=0x7fc60001c210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=575797ca91ffe039 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:15:01.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.362+0000 7fc5fbfff640 1 -- 172.21.15.49:0/926013913 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6000ab600 con 0x7fc60415ab50 2023-11-06T02:15:01.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.362+0000 7fc5fbfff640 1 -- 172.21.15.49:0/926013913 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6000a7050 con 0x7fc60415ab50 2023-11-06T02:15:01.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.362+0000 7fc5fbfff640 1 -- 172.21.15.49:0/926013913 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6000ab980 con 0x7fc60415ab50 2023-11-06T02:15:01.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.366+0000 7fc60b36e640 1 -- 172.21.15.49:0/926013913 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc60415ab50 msgr2=0x7fc6040c9bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:01.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.366+0000 7fc60b36e640 1 --2- 172.21.15.49:0/926013913 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc60415ab50 0x7fc6040c9bb0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7fc6040cae30 tx=0x7fc60001c210 comp rx=0 tx=0).stop 2023-11-06T02:15:01.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.366+0000 7fc60b36e640 1 -- 172.21.15.49:0/926013913 shutdown_connections 2023-11-06T02:15:01.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.366+0000 7fc60b36e640 1 --2- 172.21.15.49:0/926013913 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc6040cb010 0x7fc6040b0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:01.365 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.366+0000 7fc60b36e640 1 --2- 172.21.15.49:0/926013913 >> v2:172.21.15.150:3300/0 conn(0x7fc6040b05c0 0x7fc6040700a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:01.365 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.366+0000 7fc60b36e640 1 --2- 172.21.15.49:0/926013913 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc60415ab50 0x7fc6040c9bb0 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:01.365 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.366+0000 7fc60b36e640 1 -- 172.21.15.49:0/926013913 >> 172.21.15.49:0/926013913 conn(0x7fc60406c0a0 msgr2=0x7fc60406c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:15:01.365 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.366+0000 7fc60b36e640 1 -- 172.21.15.49:0/926013913 shutdown_connections 2023-11-06T02:15:01.365 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.366+0000 7fc60b36e640 1 -- 172.21.15.49:0/926013913 wait complete. 2023-11-06T02:15:01.366 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.366+0000 7fc60b36e640 1 Processor -- start 2023-11-06T02:15:01.366 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.366+0000 7fc60b36e640 1 -- start start 2023-11-06T02:15:01.366 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.366+0000 7fc60b36e640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc6040b05c0 0x7fc6040c56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:01.366 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.366+0000 7fc60b36e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc6040cb010 0x7fc604158c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:01.366 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.366+0000 7fc60b36e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc60415ab50 0x7fc604159170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:01.367 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.366+0000 7fc60b36e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6040c68c0 con 0x7fc60415ab50 2023-11-06T02:15:01.367 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.366+0000 7fc60b36e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6040c5cb0 con 0x7fc6040cb010 2023-11-06T02:15:01.367 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.366+0000 7fc60b36e640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6040c5f90 con 0x7fc6040b05c0 2023-11-06T02:15:01.367 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.366+0000 7fc60990b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc60415ab50 0x7fc604159170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:01.367 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.366+0000 7fc60990b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc60415ab50 0x7fc604159170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56390/0 (socket says 172.21.15.49:56390) 2023-11-06T02:15:01.368 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.366+0000 7fc60990b640 1 -- 172.21.15.49:0/1365673118 learned_addr learned my addr 172.21.15.49:0/1365673118 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:15:01.368 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.366+0000 7fc60910a640 1 --2- 172.21.15.49:0/1365673118 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc6040b05c0 0x7fc6040c56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:01.368 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.366+0000 7fc608909640 1 --2- 172.21.15.49:0/1365673118 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc6040cb010 0x7fc604158c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:01.368 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.366+0000 7fc60990b640 1 -- 172.21.15.49:0/1365673118 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc6040b05c0 msgr2=0x7fc6040c56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:01.368 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.366+0000 7fc60990b640 1 --2- 172.21.15.49:0/1365673118 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc6040b05c0 0x7fc6040c56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:01.368 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.366+0000 7fc60990b640 1 -- 172.21.15.49:0/1365673118 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc6040cb010 msgr2=0x7fc604158c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:01.369 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.366+0000 7fc60990b640 1 --2- 172.21.15.49:0/1365673118 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc6040cb010 0x7fc604158c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:01.369 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.366+0000 7fc60990b640 1 -- 172.21.15.49:0/1365673118 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc600009480 con 0x7fc60415ab50 2023-11-06T02:15:01.369 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.366+0000 7fc60990b640 1 --2- 172.21.15.49:0/1365673118 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc60415ab50 0x7fc604159170 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc00ccf0 tx=0x7fc5fc00a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:15:01.369 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.366+0000 7fc5f9ffb640 1 -- 172.21.15.49:0/1365673118 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5fc003ba0 con 0x7fc60415ab50 2023-11-06T02:15:01.369 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.366+0000 7fc5f9ffb640 1 -- 172.21.15.49:0/1365673118 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5fc003d00 con 0x7fc60415ab50 2023-11-06T02:15:01.370 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.366+0000 7fc60b36e640 1 -- 172.21.15.49:0/1365673118 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6040c6270 con 0x7fc60415ab50 2023-11-06T02:15:01.370 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.366+0000 7fc5f9ffb640 1 -- 172.21.15.49:0/1365673118 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5fc004240 con 0x7fc60415ab50 2023-11-06T02:15:01.370 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.366+0000 7fc60b36e640 1 -- 172.21.15.49:0/1365673118 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc604152cc0 con 0x7fc60415ab50 2023-11-06T02:15:01.370 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.366+0000 7fc60b36e640 1 -- 172.21.15.49:0/1365673118 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6040b0de0 con 0x7fc60415ab50 2023-11-06T02:15:01.370 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.370+0000 7fc5f9ffb640 1 -- 172.21.15.49:0/1365673118 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fc5fc0043a0 con 0x7fc60415ab50 2023-11-06T02:15:01.371 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.370+0000 7fc5f9ffb640 1 --2- 172.21.15.49:0/1365673118 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc5e8065a60 0x7fc5e8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:01.371 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.370+0000 7fc5f9ffb640 1 -- 172.21.15.49:0/1365673118 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fc5fc012070 con 0x7fc60415ab50 2023-11-06T02:15:01.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.370+0000 7fc60910a640 1 --2- 172.21.15.49:0/1365673118 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc5e8065a60 0x7fc5e8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:01.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.370+0000 7fc60910a640 1 --2- 172.21.15.49:0/1365673118 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc5e8065a60 0x7fc5e8067f20 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fc6000ad000 tx=0x7fc6000a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:15:01.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.374+0000 7fc5f9ffb640 1 -- 172.21.15.49:0/1365673118 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fc5fc051510 con 0x7fc60415ab50 2023-11-06T02:15:01.522 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.526+0000 7fc60b36e640 1 -- 172.21.15.49:0/1365673118 --> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7fc6040cbe50 con 0x7fc5e8065a60 2023-11-06T02:15:01.523 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.530+0000 7fc5f9ffb640 1 -- 172.21.15.49:0/1365673118 <== mgr.4100 v2:172.21.15.150:6832/38104 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fc6000042e0 con 0x7fc5e8065a60 2023-11-06T02:15:01.524 INFO:teuthology.orchestra.run.smithi049.stderr:instructed osd(s) 1 to deep-scrub 2023-11-06T02:15:01.525 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.530+0000 7fc60b36e640 1 -- 172.21.15.49:0/1365673118 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc5e8065a60 msgr2=0x7fc5e8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:01.526 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.530+0000 7fc60b36e640 1 --2- 172.21.15.49:0/1365673118 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc5e8065a60 0x7fc5e8067f20 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fc6000ad000 tx=0x7fc6000a6000 comp rx=0 tx=0).stop 2023-11-06T02:15:01.526 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.530+0000 7fc60b36e640 1 -- 172.21.15.49:0/1365673118 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc60415ab50 msgr2=0x7fc604159170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:01.526 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.530+0000 7fc60b36e640 1 --2- 172.21.15.49:0/1365673118 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc60415ab50 0x7fc604159170 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc00ccf0 tx=0x7fc5fc00a660 comp rx=0 tx=0).stop 2023-11-06T02:15:01.526 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.530+0000 7fc60b36e640 1 -- 172.21.15.49:0/1365673118 shutdown_connections 2023-11-06T02:15:01.527 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.530+0000 7fc60b36e640 1 --2- 172.21.15.49:0/1365673118 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fc5e8065a60 0x7fc5e8067f20 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:01.527 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.530+0000 7fc60b36e640 1 --2- 172.21.15.49:0/1365673118 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc60415ab50 0x7fc604159170 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:01.527 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.530+0000 7fc60b36e640 1 --2- 172.21.15.49:0/1365673118 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc6040cb010 0x7fc604158c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:01.527 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.530+0000 7fc60b36e640 1 --2- 172.21.15.49:0/1365673118 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc6040b05c0 0x7fc6040c56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:01.528 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.530+0000 7fc60b36e640 1 -- 172.21.15.49:0/1365673118 >> 172.21.15.49:0/1365673118 conn(0x7fc60406c0a0 msgr2=0x7fc6040c2b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:15:01.528 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.530+0000 7fc60b36e640 1 -- 172.21.15.49:0/1365673118 shutdown_connections 2023-11-06T02:15:01.528 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.530+0000 7fc60b36e640 1 -- 172.21.15.49:0/1365673118 wait complete. 2023-11-06T02:15:01.542 INFO:tasks.ceph:Scrubbing osd.2 2023-11-06T02:15:01.543 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-11-06T02:15:01.669 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.674+0000 7f9e02a45640 1 Processor -- start 2023-11-06T02:15:01.669 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.674+0000 7f9e02a45640 1 -- start start 2023-11-06T02:15:01.673 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.674+0000 7f9e02a45640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f9dfc069fd0 0x7f9dfc0caa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:01.673 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.674+0000 7f9e02a45640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9dfc0ad720 0x7f9dfc159530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:01.673 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.674+0000 7f9e02a45640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dfc071e80 con 0x7f9dfc06a3b0 2023-11-06T02:15:01.674 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.674+0000 7f9e02a45640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dfc072160 con 0x7f9dfc069fd0 2023-11-06T02:15:01.674 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.674+0000 7f9e02a45640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dfc0ca220 con 0x7f9dfc0ad720 2023-11-06T02:15:01.674 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.674+0000 7f9df37fe640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f9dfc06a3b0 0x7f9dfc0715c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:35112/0 (socket says 172.21.15.49:35112) 2023-11-06T02:15:01.674 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.674+0000 7f9df37fe640 1 -- 172.21.15.49:0/1076698229 learned_addr learned my addr 172.21.15.49:0/1076698229 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:15:01.675 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.674+0000 7f9e00fe2640 1 --2- 172.21.15.49:0/1076698229 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9dfc0ad720 0x7f9dfc159530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:01.675 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.674+0000 7f9dfbfff640 1 --2- 172.21.15.49:0/1076698229 >> v2:172.21.15.150:3300/0 conn(0x7f9dfc069fd0 0x7f9dfc0caa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:01.675 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.674+0000 7f9dfb7fe640 1 -- 172.21.15.49:0/1076698229 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3307899067 0 0) 0x7f9dfc071e80 con 0x7f9dfc06a3b0 2023-11-06T02:15:01.675 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.674+0000 7f9dfb7fe640 1 -- 172.21.15.49:0/1076698229 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9de8003530 con 0x7f9dfc06a3b0 2023-11-06T02:15:01.676 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.678+0000 7f9dfb7fe640 1 -- 172.21.15.49:0/1076698229 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1733244781 0 0) 0x7f9de8003530 con 0x7f9dfc06a3b0 2023-11-06T02:15:01.676 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.678+0000 7f9dfb7fe640 1 -- 172.21.15.49:0/1076698229 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9dfc0ad720 msgr2=0x7f9dfc159530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:15:01.676 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.678+0000 7f9dfb7fe640 1 --2- 172.21.15.49:0/1076698229 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9dfc0ad720 0x7f9dfc159530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:01.676 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.678+0000 7f9dfb7fe640 1 -- 172.21.15.49:0/1076698229 >> v2:172.21.15.150:3300/0 conn(0x7f9dfc069fd0 msgr2=0x7f9dfc0caa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:15:01.676 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.678+0000 7f9dfb7fe640 1 --2- 172.21.15.49:0/1076698229 >> v2:172.21.15.150:3300/0 conn(0x7f9dfc069fd0 0x7f9dfc0caa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:01.677 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.678+0000 7f9dfb7fe640 1 -- 172.21.15.49:0/1076698229 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dfc0ca500 con 0x7f9dfc06a3b0 2023-11-06T02:15:01.677 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.678+0000 7f9dfb7fe640 1 -- 172.21.15.49:0/1076698229 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7f9de4001680 con 0x7f9dfc06a3b0 2023-11-06T02:15:01.677 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.678+0000 7f9dfb7fe640 1 -- 172.21.15.49:0/1076698229 >> v1:172.21.15.49:6789/0 conn(0x7f9dfc06a3b0 legacy=0x7f9dfc0715c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:01.677 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.678+0000 7f9dfb7fe640 1 --2- 172.21.15.49:0/1076698229 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9dfc0ad720 0x7f9de8003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:01.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.678+0000 7f9dfb7fe640 1 --2- 172.21.15.49:0/1076698229 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9de8003f00 0x7f9de80047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:01.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.678+0000 7f9dfb7fe640 1 --2- 172.21.15.49:0/1076698229 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9de8004d10 0x7f9de8005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:01.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.678+0000 7f9dfb7fe640 1 -- 172.21.15.49:0/1076698229 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dfc072160 con 0x7f9de8004d10 2023-11-06T02:15:01.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.678+0000 7f9dfb7fe640 1 -- 172.21.15.49:0/1076698229 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dfc0ca220 con 0x7f9de8003f00 2023-11-06T02:15:01.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.678+0000 7f9dfb7fe640 1 -- 172.21.15.49:0/1076698229 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9de800b990 con 0x7f9dfc0ad720 2023-11-06T02:15:01.679 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.678+0000 7f9e00fe2640 1 --2- 172.21.15.49:0/1076698229 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9de8004d10 0x7f9de8005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:01.679 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.678+0000 7f9df37fe640 1 --2- 172.21.15.49:0/1076698229 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9de8003f00 0x7f9de80047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:01.679 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.678+0000 7f9dfbfff640 1 --2- 172.21.15.49:0/1076698229 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9dfc0ad720 0x7f9de8003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:01.679 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.678+0000 7f9e00fe2640 1 -- 172.21.15.49:0/1076698229 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9dfc0ad720 msgr2=0x7f9de8003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:15:01.679 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.678+0000 7f9e00fe2640 1 --2- 172.21.15.49:0/1076698229 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9dfc0ad720 0x7f9de8003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:01.680 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.678+0000 7f9e00fe2640 1 -- 172.21.15.49:0/1076698229 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9de8003f00 msgr2=0x7f9de80047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:15:01.680 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.678+0000 7f9e00fe2640 1 --2- 172.21.15.49:0/1076698229 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9de8003f00 0x7f9de80047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:01.680 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.678+0000 7f9e00fe2640 1 -- 172.21.15.49:0/1076698229 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9de8003530 con 0x7f9de8004d10 2023-11-06T02:15:01.680 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.678+0000 7f9df37fe640 1 --2- 172.21.15.49:0/1076698229 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9de8003f00 0x7f9de80047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:15:01.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.678+0000 7f9e00fe2640 1 --2- 172.21.15.49:0/1076698229 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9de8004d10 0x7f9de8005190 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f9df4009fc0 tx=0x7f9df401cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=393c6f5bf875029a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:15:01.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.678+0000 7f9dfb7fe640 1 -- 172.21.15.49:0/1076698229 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df40a5c70 con 0x7f9de8004d10 2023-11-06T02:15:01.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.678+0000 7f9dfb7fe640 1 -- 172.21.15.49:0/1076698229 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9df400f040 con 0x7f9de8004d10 2023-11-06T02:15:01.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.678+0000 7f9dfb7fe640 1 -- 172.21.15.49:0/1076698229 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df40ad440 con 0x7f9de8004d10 2023-11-06T02:15:01.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.678+0000 7f9e02a45640 1 -- 172.21.15.49:0/1076698229 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9de8004d10 msgr2=0x7f9de8005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:01.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.678+0000 7f9e02a45640 1 --2- 172.21.15.49:0/1076698229 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9de8004d10 0x7f9de8005190 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f9df4009fc0 tx=0x7f9df401cc00 comp rx=0 tx=0).stop 2023-11-06T02:15:01.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.678+0000 7f9e00fe2640 1 -- 172.21.15.49:0/1076698229 reap_dead start 2023-11-06T02:15:01.686 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.678+0000 7f9e02a45640 1 -- 172.21.15.49:0/1076698229 shutdown_connections 2023-11-06T02:15:01.686 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.678+0000 7f9e02a45640 1 -- 172.21.15.49:0/1076698229 >> 172.21.15.49:0/1076698229 conn(0x7f9dfc0ce610 msgr2=0x7f9dfc0cea20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:15:01.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.682+0000 7f9e02a45640 1 -- 172.21.15.49:0/1076698229 shutdown_connections 2023-11-06T02:15:01.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.682+0000 7f9e02a45640 1 -- 172.21.15.49:0/1076698229 wait complete. 2023-11-06T02:15:01.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.682+0000 7f9e02a45640 1 Processor -- start 2023-11-06T02:15:01.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.682+0000 7f9e02a45640 1 -- start start 2023-11-06T02:15:01.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.682+0000 7f9e02a45640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9de8004d10 0x7f9dfc1f01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:01.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.682+0000 7f9e02a45640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9dfc0c30e0 0x7f9dfc14ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:01.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.682+0000 7f9e02a45640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9dfc0c8460 0x7f9dfc0b3220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:01.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.682+0000 7f9e02a45640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dfc0c3560 con 0x7f9dfc0c8460 2023-11-06T02:15:01.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.682+0000 7f9e02a45640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dfc0c3750 con 0x7f9de8004d10 2023-11-06T02:15:01.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.682+0000 7f9e02a45640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dfc0c3a30 con 0x7f9dfc0c30e0 2023-11-06T02:15:01.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.682+0000 7f9e00fe2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9dfc0c8460 0x7f9dfc0b3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:01.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.682+0000 7f9e00fe2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9dfc0c8460 0x7f9dfc0b3220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56416/0 (socket says 172.21.15.49:56416) 2023-11-06T02:15:01.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.682+0000 7f9e00fe2640 1 -- 172.21.15.49:0/3595560756 learned_addr learned my addr 172.21.15.49:0/3595560756 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:15:01.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.682+0000 7f9df37fe640 1 --2- 172.21.15.49:0/3595560756 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9dfc0c30e0 0x7f9dfc14ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:01.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.682+0000 7f9dfbfff640 1 --2- 172.21.15.49:0/3595560756 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9de8004d10 0x7f9dfc1f01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:01.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.682+0000 7f9e00fe2640 1 -- 172.21.15.49:0/3595560756 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9dfc0c30e0 msgr2=0x7f9dfc14ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:01.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.682+0000 7f9e00fe2640 1 --2- 172.21.15.49:0/3595560756 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9dfc0c30e0 0x7f9dfc14ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:01.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.682+0000 7f9e00fe2640 1 -- 172.21.15.49:0/3595560756 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9de8004d10 msgr2=0x7f9dfc1f01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:01.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.682+0000 7f9e00fe2640 1 --2- 172.21.15.49:0/3595560756 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9de8004d10 0x7f9dfc1f01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:01.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.682+0000 7f9e00fe2640 1 -- 172.21.15.49:0/3595560756 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dfc0c3d10 con 0x7f9dfc0c8460 2023-11-06T02:15:01.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.682+0000 7f9e00fe2640 1 --2- 172.21.15.49:0/3595560756 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9dfc0c8460 0x7f9dfc0b3220 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f9df40b0040 tx=0x7f9df4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:15:01.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.682+0000 7f9df97fa640 1 -- 172.21.15.49:0/3595560756 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df40bc070 con 0x7f9dfc0c8460 2023-11-06T02:15:01.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.682+0000 7f9df97fa640 1 -- 172.21.15.49:0/3595560756 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9df40039a0 con 0x7f9dfc0c8460 2023-11-06T02:15:01.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.682+0000 7f9df97fa640 1 -- 172.21.15.49:0/3595560756 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df4003b30 con 0x7f9dfc0c8460 2023-11-06T02:15:01.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.682+0000 7f9e02a45640 1 -- 172.21.15.49:0/3595560756 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dfc0c3ff0 con 0x7f9dfc0c8460 2023-11-06T02:15:01.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.682+0000 7f9e02a45640 1 -- 172.21.15.49:0/3595560756 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dfc15bbb0 con 0x7f9dfc0c8460 2023-11-06T02:15:01.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.686+0000 7f9e02a45640 1 -- 172.21.15.49:0/3595560756 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9dfc074040 con 0x7f9dfc0c8460 2023-11-06T02:15:01.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.686+0000 7f9df97fa640 1 -- 172.21.15.49:0/3595560756 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f9df40b4040 con 0x7f9dfc0c8460 2023-11-06T02:15:01.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.686+0000 7f9df97fa640 1 --2- 172.21.15.49:0/3595560756 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9ddc065a60 0x7f9ddc067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:01.692 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.686+0000 7f9df97fa640 1 -- 172.21.15.49:0/3595560756 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f9df411d310 con 0x7f9dfc0c8460 2023-11-06T02:15:01.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.686+0000 7f9df97fa640 1 --2- 172.21.15.49:0/3595560756 >> [v2:172.21.15.49:6800/38114,v1:172.21.15.49:6801/38114] conn(0x7f9ddc06c020 0x7f9ddc06e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:01.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.686+0000 7f9df97fa640 1 -- 172.21.15.49:0/3595560756 --> [v2:172.21.15.49:6800/38114,v1:172.21.15.49:6801/38114] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9ddc06eaf0 con 0x7f9ddc06c020 2023-11-06T02:15:01.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.686+0000 7f9df97fa640 1 -- 172.21.15.49:0/3595560756 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9df411f9d0 con 0x7f9dfc0c8460 2023-11-06T02:15:01.693 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.686+0000 7f9df37fe640 1 --2- 172.21.15.49:0/3595560756 >> [v2:172.21.15.49:6800/38114,v1:172.21.15.49:6801/38114] conn(0x7f9ddc06c020 0x7f9ddc06e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:01.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.686+0000 7f9dfbfff640 1 --2- 172.21.15.49:0/3595560756 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9ddc065a60 0x7f9ddc067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:01.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.686+0000 7f9df37fe640 1 --2- 172.21.15.49:0/3595560756 >> [v2:172.21.15.49:6800/38114,v1:172.21.15.49:6801/38114] conn(0x7f9ddc06c020 0x7f9ddc06e440 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:15:01.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.686+0000 7f9dfbfff640 1 --2- 172.21.15.49:0/3595560756 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9ddc065a60 0x7f9ddc067f20 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f9dec009ab0 tx=0x7f9dec008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:15:01.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.686+0000 7f9df97fa640 1 -- 172.21.15.49:0/3595560756 <== osd.2 v2:172.21.15.49:6800/38114 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f9ddc06eaf0 con 0x7f9ddc06c020 2023-11-06T02:15:01.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.698+0000 7f9e02a45640 1 -- 172.21.15.49:0/3595560756 --> [v2:172.21.15.49:6800/38114,v1:172.21.15.49:6801/38114] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f9dfc0aeac0 con 0x7f9ddc06c020 2023-11-06T02:15:01.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.698+0000 7f9df97fa640 1 -- 172.21.15.49:0/3595560756 <== osd.2 v2:172.21.15.49:6800/38114 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f9dfc0aeac0 con 0x7f9ddc06c020 2023-11-06T02:15:01.696 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-11-06T02:15:01.697 INFO:teuthology.orchestra.run.smithi049.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-06T02:15:01.697 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-11-06T02:15:01.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.702+0000 7f9e02a45640 1 -- 172.21.15.49:0/3595560756 >> [v2:172.21.15.49:6800/38114,v1:172.21.15.49:6801/38114] conn(0x7f9ddc06c020 msgr2=0x7f9ddc06e440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:01.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.702+0000 7f9e02a45640 1 --2- 172.21.15.49:0/3595560756 >> [v2:172.21.15.49:6800/38114,v1:172.21.15.49:6801/38114] conn(0x7f9ddc06c020 0x7f9ddc06e440 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:01.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.702+0000 7f9e02a45640 1 -- 172.21.15.49:0/3595560756 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9ddc065a60 msgr2=0x7f9ddc067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:01.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.702+0000 7f9e02a45640 1 --2- 172.21.15.49:0/3595560756 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9ddc065a60 0x7f9ddc067f20 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f9dec009ab0 tx=0x7f9dec008040 comp rx=0 tx=0).stop 2023-11-06T02:15:01.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.702+0000 7f9e02a45640 1 -- 172.21.15.49:0/3595560756 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9dfc0c8460 msgr2=0x7f9dfc0b3220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:01.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.702+0000 7f9e02a45640 1 --2- 172.21.15.49:0/3595560756 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9dfc0c8460 0x7f9dfc0b3220 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f9df40b0040 tx=0x7f9df4009750 comp rx=0 tx=0).stop 2023-11-06T02:15:01.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.702+0000 7f9e00fe2640 1 -- 172.21.15.49:0/3595560756 reap_dead start 2023-11-06T02:15:01.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.702+0000 7f9e02a45640 1 -- 172.21.15.49:0/3595560756 shutdown_connections 2023-11-06T02:15:01.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.702+0000 7f9e02a45640 1 -- 172.21.15.49:0/3595560756 >> 172.21.15.49:0/3595560756 conn(0x7f9dfc0ce610 msgr2=0x7f9dfc157a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:15:01.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.702+0000 7f9e02a45640 1 -- 172.21.15.49:0/3595560756 shutdown_connections 2023-11-06T02:15:01.701 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.702+0000 7f9e02a45640 1 -- 172.21.15.49:0/3595560756 wait complete. 2023-11-06T02:15:01.713 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-11-06T02:15:01.828 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.834+0000 7f4329b73640 1 Processor -- start 2023-11-06T02:15:01.829 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.834+0000 7f4329b73640 1 -- start start 2023-11-06T02:15:01.830 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.834+0000 7f4329b73640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f432415aa20 0x7f43240c21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:01.830 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.834+0000 7f4329b73640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f43240b0820 0x7f43240cb240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:01.830 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.834+0000 7f4329b73640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43240ca7f0 con 0x7f432415e520 2023-11-06T02:15:01.830 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.834+0000 7f4329b73640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f432415ae00 con 0x7f432415aa20 2023-11-06T02:15:01.831 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.834+0000 7f4329b73640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f432415ef70 con 0x7f43240b0820 2023-11-06T02:15:01.831 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.834+0000 7f4323fff640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f43240b0820 0x7f43240cb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:01.831 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.834+0000 7f43237fe640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f432415aa20 0x7f43240c21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:01.831 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.834+0000 7f4323fff640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f43240b0820 0x7f43240cb240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.49:36638/0 (socket says 172.21.15.49:36638) 2023-11-06T02:15:01.831 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.834+0000 7f4323fff640 1 -- 172.21.15.49:0/2366139358 learned_addr learned my addr 172.21.15.49:0/2366139358 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:15:01.832 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.834+0000 7f4323fff640 1 -- 172.21.15.49:0/2366139358 >> v2:172.21.15.150:3300/0 conn(0x7f432415aa20 msgr2=0x7f43240c21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:15:01.832 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.834+0000 7f4323fff640 1 --2- 172.21.15.49:0/2366139358 >> v2:172.21.15.150:3300/0 conn(0x7f432415aa20 0x7f43240c21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:01.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.834+0000 7f4323fff640 1 -- 172.21.15.49:0/2366139358 >> v1:172.21.15.150:6789/0 conn(0x7f432415e520 legacy=0x7f43240b0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:01.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.834+0000 7f4323fff640 1 -- 172.21.15.49:0/2366139358 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f432415f250 con 0x7f43240b0820 2023-11-06T02:15:01.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.838+0000 7f4323fff640 1 --2- 172.21.15.49:0/2366139358 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f43240b0820 0x7f43240cb240 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f431c009e50 tx=0x7f431c01c950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=936e904331ca5f23 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:15:01.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.838+0000 7f43227fc640 1 -- 172.21.15.49:0/2366139358 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431c0a5720 con 0x7f43240b0820 2023-11-06T02:15:01.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.838+0000 7f43227fc640 1 -- 172.21.15.49:0/2366139358 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f431c0a5880 con 0x7f43240b0820 2023-11-06T02:15:01.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.838+0000 7f43227fc640 1 -- 172.21.15.49:0/2366139358 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431c0a5c10 con 0x7f43240b0820 2023-11-06T02:15:01.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.838+0000 7f4329b73640 1 -- 172.21.15.49:0/2366139358 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f43240b0820 msgr2=0x7f43240cb240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:01.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.838+0000 7f4329b73640 1 --2- 172.21.15.49:0/2366139358 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f43240b0820 0x7f43240cb240 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f431c009e50 tx=0x7f431c01c950 comp rx=0 tx=0).stop 2023-11-06T02:15:01.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.838+0000 7f4329b73640 1 -- 172.21.15.49:0/2366139358 shutdown_connections 2023-11-06T02:15:01.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.838+0000 7f4329b73640 1 --2- 172.21.15.49:0/2366139358 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f43240b0820 0x7f43240cb240 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:01.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.838+0000 7f4329b73640 1 --2- 172.21.15.49:0/2366139358 >> v2:172.21.15.150:3300/0 conn(0x7f432415aa20 0x7f43240c21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:01.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.838+0000 7f4329b73640 1 -- 172.21.15.49:0/2366139358 >> 172.21.15.49:0/2366139358 conn(0x7f432406c160 msgr2=0x7f432406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:15:01.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.838+0000 7f4329b73640 1 -- 172.21.15.49:0/2366139358 shutdown_connections 2023-11-06T02:15:01.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.838+0000 7f4329b73640 1 -- 172.21.15.49:0/2366139358 wait complete. 2023-11-06T02:15:01.839 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.842+0000 7f4329b73640 1 Processor -- start 2023-11-06T02:15:01.839 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.842+0000 7f4329b73640 1 -- start start 2023-11-06T02:15:01.839 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.842+0000 7f4329b73640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43240b0820 0x7f43240c1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:01.839 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.842+0000 7f4329b73640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f432415aa20 0x7f4324156ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:01.840 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.842+0000 7f4329b73640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f432415e520 0x7f4324157010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:01.840 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.842+0000 7f4329b73640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4324154960 con 0x7f432415e520 2023-11-06T02:15:01.840 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.842+0000 7f4329b73640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43240c73f0 con 0x7f43240b0820 2023-11-06T02:15:01.840 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.842+0000 7f4329b73640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43240c6e30 con 0x7f432415aa20 2023-11-06T02:15:01.840 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.842+0000 7f4323fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f432415e520 0x7f4324157010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:01.841 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.842+0000 7f4323fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f432415e520 0x7f4324157010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56428/0 (socket says 172.21.15.49:56428) 2023-11-06T02:15:01.841 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.842+0000 7f4323fff640 1 -- 172.21.15.49:0/2367289556 learned_addr learned my addr 172.21.15.49:0/2367289556 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:15:01.841 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.842+0000 7f43237fe640 1 --2- 172.21.15.49:0/2367289556 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43240b0820 0x7f43240c1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:01.841 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.842+0000 7f4322ffd640 1 --2- 172.21.15.49:0/2367289556 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f432415aa20 0x7f4324156ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:01.841 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.842+0000 7f4323fff640 1 -- 172.21.15.49:0/2367289556 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f432415aa20 msgr2=0x7f4324156ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:01.842 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.842+0000 7f4323fff640 1 --2- 172.21.15.49:0/2367289556 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f432415aa20 0x7f4324156ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:01.842 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.842+0000 7f4323fff640 1 -- 172.21.15.49:0/2367289556 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43240b0820 msgr2=0x7f43240c1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:01.842 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.842+0000 7f4323fff640 1 --2- 172.21.15.49:0/2367289556 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43240b0820 0x7f43240c1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:01.842 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.842+0000 7f4323fff640 1 -- 172.21.15.49:0/2367289556 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f431c009b30 con 0x7f432415e520 2023-11-06T02:15:01.842 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.842+0000 7f4323fff640 1 --2- 172.21.15.49:0/2367289556 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f432415e520 0x7f4324157010 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f431c0af040 tx=0x7f431c0196f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:15:01.843 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.842+0000 7f430bfff640 1 -- 172.21.15.49:0/2367289556 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431c0b8670 con 0x7f432415e520 2023-11-06T02:15:01.843 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.842+0000 7f430bfff640 1 -- 172.21.15.49:0/2367289556 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f431c0b87d0 con 0x7f432415e520 2023-11-06T02:15:01.843 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.842+0000 7f430bfff640 1 -- 172.21.15.49:0/2367289556 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431c0b8b40 con 0x7f432415e520 2023-11-06T02:15:01.843 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.842+0000 7f4329b73640 1 -- 172.21.15.49:0/2367289556 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43240c7110 con 0x7f432415e520 2023-11-06T02:15:01.843 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.842+0000 7f4329b73640 1 -- 172.21.15.49:0/2367289556 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43240c6170 con 0x7f432415e520 2023-11-06T02:15:01.843 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.842+0000 7f4329b73640 1 -- 172.21.15.49:0/2367289556 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4324074040 con 0x7f432415e520 2023-11-06T02:15:01.844 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.842+0000 7f430bfff640 1 -- 172.21.15.49:0/2367289556 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f431c0b8ca0 con 0x7f432415e520 2023-11-06T02:15:01.844 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.846+0000 7f430bfff640 1 --2- 172.21.15.49:0/2367289556 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4310065a60 0x7f4310067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:01.845 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.846+0000 7f430bfff640 1 -- 172.21.15.49:0/2367289556 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f431c11e4b0 con 0x7f432415e520 2023-11-06T02:15:01.845 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.846+0000 7f43237fe640 1 --2- 172.21.15.49:0/2367289556 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4310065a60 0x7f4310067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:01.845 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.846+0000 7f43237fe640 1 --2- 172.21.15.49:0/2367289556 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4310065a60 0x7f4310067f20 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f4314009af0 tx=0x7f4314008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:15:01.845 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:01.850+0000 7f430bfff640 1 -- 172.21.15.49:0/2367289556 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f431c0ec130 con 0x7f432415e520 2023-11-06T02:15:02.001 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.006+0000 7f4329b73640 1 -- 172.21.15.49:0/2367289556 --> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f4324167860 con 0x7f4310065a60 2023-11-06T02:15:02.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.006+0000 7f430bfff640 1 -- 172.21.15.49:0/2367289556 <== mgr.4100 v2:172.21.15.150:6832/38104 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f43140119b0 con 0x7f4310065a60 2023-11-06T02:15:02.003 INFO:teuthology.orchestra.run.smithi049.stderr:instructed osd(s) 2 to deep-scrub 2023-11-06T02:15:02.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.010+0000 7f4329b73640 1 -- 172.21.15.49:0/2367289556 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4310065a60 msgr2=0x7f4310067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:02.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.010+0000 7f4329b73640 1 --2- 172.21.15.49:0/2367289556 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4310065a60 0x7f4310067f20 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f4314009af0 tx=0x7f4314008040 comp rx=0 tx=0).stop 2023-11-06T02:15:02.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.010+0000 7f4329b73640 1 -- 172.21.15.49:0/2367289556 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f432415e520 msgr2=0x7f4324157010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:02.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.010+0000 7f4329b73640 1 --2- 172.21.15.49:0/2367289556 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f432415e520 0x7f4324157010 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f431c0af040 tx=0x7f431c0196f0 comp rx=0 tx=0).stop 2023-11-06T02:15:02.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.010+0000 7f4329b73640 1 -- 172.21.15.49:0/2367289556 shutdown_connections 2023-11-06T02:15:02.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.010+0000 7f4329b73640 1 --2- 172.21.15.49:0/2367289556 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4310065a60 0x7f4310067f20 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:02.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.010+0000 7f4329b73640 1 --2- 172.21.15.49:0/2367289556 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f432415e520 0x7f4324157010 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:02.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.010+0000 7f4329b73640 1 --2- 172.21.15.49:0/2367289556 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f432415aa20 0x7f4324156ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:02.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.010+0000 7f4329b73640 1 --2- 172.21.15.49:0/2367289556 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43240b0820 0x7f43240c1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:02.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.010+0000 7f4329b73640 1 -- 172.21.15.49:0/2367289556 >> 172.21.15.49:0/2367289556 conn(0x7f432406c160 msgr2=0x7f43240c44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:15:02.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.010+0000 7f4329b73640 1 -- 172.21.15.49:0/2367289556 shutdown_connections 2023-11-06T02:15:02.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.010+0000 7f4329b73640 1 -- 172.21.15.49:0/2367289556 wait complete. 2023-11-06T02:15:02.022 INFO:tasks.ceph:Scrubbing osd.3 2023-11-06T02:15:02.023 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-11-06T02:15:02.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.154+0000 7f1e5d0dc640 1 Processor -- start 2023-11-06T02:15:02.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.154+0000 7f1e5d0dc640 1 -- start start 2023-11-06T02:15:02.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.154+0000 7f1e5d0dc640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f1e580cd060 0x7f1e5815d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:02.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.154+0000 7f1e5d0dc640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e580cd4a0 con 0x7f1e580d25c0 2023-11-06T02:15:02.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.154+0000 7f1e5d0dc640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e5815da20 con 0x7f1e580714a0 2023-11-06T02:15:02.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.154+0000 7f1e5d0dc640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e5815d830 con 0x7f1e580cd060 2023-11-06T02:15:02.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.154+0000 7f1e5659c640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f1e580d25c0 0x7f1e580d0d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:35118/0 (socket says 172.21.15.49:35118) 2023-11-06T02:15:02.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.154+0000 7f1e5659c640 1 -- 172.21.15.49:0/1280192925 learned_addr learned my addr 172.21.15.49:0/1280192925 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:15:02.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.158+0000 7f1e5759e640 1 --2- 172.21.15.49:0/1280192925 >> v2:172.21.15.150:3300/0 conn(0x7f1e580cd060 0x7f1e5815d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:02.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.158+0000 7f1e55d9b640 1 -- 172.21.15.49:0/1280192925 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2349093589 0 0) 0x7f1e4c001120 con 0x7f1e580d25c0 2023-11-06T02:15:02.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.158+0000 7f1e55d9b640 1 -- 172.21.15.49:0/1280192925 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e40003530 con 0x7f1e580d25c0 2023-11-06T02:15:02.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.158+0000 7f1e55d9b640 1 -- 172.21.15.49:0/1280192925 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1367448658 0 0) 0x7f1e40003530 con 0x7f1e580d25c0 2023-11-06T02:15:02.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.158+0000 7f1e55d9b640 1 -- 172.21.15.49:0/1280192925 >> v2:172.21.15.150:3300/0 conn(0x7f1e580cd060 msgr2=0x7f1e5815d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:15:02.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.158+0000 7f1e55d9b640 1 --2- 172.21.15.49:0/1280192925 >> v2:172.21.15.150:3300/0 conn(0x7f1e580cd060 0x7f1e5815d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:02.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.158+0000 7f1e55d9b640 1 -- 172.21.15.49:0/1280192925 >> v1:172.21.15.150:6789/0 conn(0x7f1e580714a0 legacy=0x7f1e58071880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:02.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.158+0000 7f1e55d9b640 1 -- 172.21.15.49:0/1280192925 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e5815dd00 con 0x7f1e580d25c0 2023-11-06T02:15:02.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.158+0000 7f1e55d9b640 1 -- 172.21.15.49:0/1280192925 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7f1e4c001810 con 0x7f1e580d25c0 2023-11-06T02:15:02.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.158+0000 7f1e55d9b640 1 -- 172.21.15.49:0/1280192925 >> v1:172.21.15.49:6789/0 conn(0x7f1e580d25c0 legacy=0x7f1e580d0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:02.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.158+0000 7f1e55d9b640 1 --2- 172.21.15.49:0/1280192925 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e40003980 0x7f1e40003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:02.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.158+0000 7f1e55d9b640 1 --2- 172.21.15.49:0/1280192925 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1e400047d0 0x7f1e40004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:02.156 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.158+0000 7f1e55d9b640 1 --2- 172.21.15.49:0/1280192925 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e400052b0 0x7f1e4000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:02.156 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.158+0000 7f1e55d9b640 1 -- 172.21.15.49:0/1280192925 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4000c150 con 0x7f1e400052b0 2023-11-06T02:15:02.156 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.158+0000 7f1e55d9b640 1 -- 172.21.15.49:0/1280192925 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4000c2c0 con 0x7f1e400047d0 2023-11-06T02:15:02.156 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.158+0000 7f1e55d9b640 1 -- 172.21.15.49:0/1280192925 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4000c480 con 0x7f1e40003980 2023-11-06T02:15:02.156 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.158+0000 7f1e5759e640 1 --2- 172.21.15.49:0/1280192925 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e400052b0 0x7f1e4000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:02.157 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.158+0000 7f1e5659c640 1 --2- 172.21.15.49:0/1280192925 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1e400047d0 0x7f1e40004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:02.157 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.158+0000 7f1e56d9d640 1 --2- 172.21.15.49:0/1280192925 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e40003980 0x7f1e40003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:02.157 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.158+0000 7f1e5759e640 1 -- 172.21.15.49:0/1280192925 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e40003980 msgr2=0x7f1e40003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:15:02.157 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.158+0000 7f1e5759e640 1 --2- 172.21.15.49:0/1280192925 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e40003980 0x7f1e40003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:02.157 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.158+0000 7f1e5759e640 1 -- 172.21.15.49:0/1280192925 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1e400047d0 msgr2=0x7f1e40004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:15:02.157 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.158+0000 7f1e5759e640 1 --2- 172.21.15.49:0/1280192925 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1e400047d0 0x7f1e40004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:02.158 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.158+0000 7f1e5759e640 1 -- 172.21.15.49:0/1280192925 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e40003530 con 0x7f1e400052b0 2023-11-06T02:15:02.158 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.158+0000 7f1e56d9d640 1 --2- 172.21.15.49:0/1280192925 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e40003980 0x7f1e40003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:15:02.162 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.158+0000 7f1e5759e640 1 -- 172.21.15.49:0/1280192925 reap_dead start 2023-11-06T02:15:02.162 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.158+0000 7f1e5759e640 1 --2- 172.21.15.49:0/1280192925 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e400052b0 0x7f1e4000b990 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f1e50009f90 tx=0x7f1e5001cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=146d43afd138f77b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:15:02.162 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.158+0000 7f1e55d9b640 1 -- 172.21.15.49:0/1280192925 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e500a5c20 con 0x7f1e400052b0 2023-11-06T02:15:02.163 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.158+0000 7f1e55d9b640 1 -- 172.21.15.49:0/1280192925 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e5000f040 con 0x7f1e400052b0 2023-11-06T02:15:02.163 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.158+0000 7f1e55d9b640 1 -- 172.21.15.49:0/1280192925 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e500039a0 con 0x7f1e400052b0 2023-11-06T02:15:02.163 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.158+0000 7f1e5d0dc640 1 -- 172.21.15.49:0/1280192925 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e400052b0 msgr2=0x7f1e4000b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:02.163 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.158+0000 7f1e5d0dc640 1 --2- 172.21.15.49:0/1280192925 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e400052b0 0x7f1e4000b990 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f1e50009f90 tx=0x7f1e5001cc30 comp rx=0 tx=0).stop 2023-11-06T02:15:02.163 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.158+0000 7f1e5d0dc640 1 -- 172.21.15.49:0/1280192925 shutdown_connections 2023-11-06T02:15:02.164 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.158+0000 7f1e5d0dc640 1 --2- 172.21.15.49:0/1280192925 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e400052b0 0x7f1e4000b990 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:02.164 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.158+0000 7f1e5d0dc640 1 -- 172.21.15.49:0/1280192925 >> 172.21.15.49:0/1280192925 conn(0x7f1e580d2aa0 msgr2=0x7f1e580d2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:15:02.164 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.162+0000 7f1e5d0dc640 1 -- 172.21.15.49:0/1280192925 shutdown_connections 2023-11-06T02:15:02.164 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.162+0000 7f1e5d0dc640 1 -- 172.21.15.49:0/1280192925 wait complete. 2023-11-06T02:15:02.164 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.162+0000 7f1e5d0dc640 1 Processor -- start 2023-11-06T02:15:02.165 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.162+0000 7f1e5d0dc640 1 -- start start 2023-11-06T02:15:02.165 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.162+0000 7f1e5d0dc640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1e400052b0 0x7f1e580b2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:02.165 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.162+0000 7f1e5d0dc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e580c70b0 0x7f1e581527e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:02.165 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.162+0000 7f1e5d0dc640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e58152d20 0x7f1e58151960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:02.165 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.162+0000 7f1e5d0dc640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e580b25b0 con 0x7f1e580c70b0 2023-11-06T02:15:02.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.162+0000 7f1e5d0dc640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e581514d0 con 0x7f1e400052b0 2023-11-06T02:15:02.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.162+0000 7f1e5d0dc640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e580caed0 con 0x7f1e58152d20 2023-11-06T02:15:02.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.162+0000 7f1e5659c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e580c70b0 0x7f1e581527e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:02.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.162+0000 7f1e5659c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e580c70b0 0x7f1e581527e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56440/0 (socket says 172.21.15.49:56440) 2023-11-06T02:15:02.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.162+0000 7f1e5659c640 1 -- 172.21.15.49:0/1596778259 learned_addr learned my addr 172.21.15.49:0/1596778259 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:15:02.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.162+0000 7f1e5759e640 1 --2- 172.21.15.49:0/1596778259 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e58152d20 0x7f1e58151960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:02.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.162+0000 7f1e56d9d640 1 --2- 172.21.15.49:0/1596778259 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1e400052b0 0x7f1e580b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:02.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.162+0000 7f1e5659c640 1 -- 172.21.15.49:0/1596778259 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e58152d20 msgr2=0x7f1e58151960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:02.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.162+0000 7f1e5659c640 1 --2- 172.21.15.49:0/1596778259 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e58152d20 0x7f1e58151960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:02.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.162+0000 7f1e5659c640 1 -- 172.21.15.49:0/1596778259 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1e400052b0 msgr2=0x7f1e580b2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:02.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.162+0000 7f1e5659c640 1 --2- 172.21.15.49:0/1596778259 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1e400052b0 0x7f1e580b2070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:02.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.162+0000 7f1e5659c640 1 -- 172.21.15.49:0/1596778259 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e580cb1b0 con 0x7f1e580c70b0 2023-11-06T02:15:02.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.162+0000 7f1e5659c640 1 --2- 172.21.15.49:0/1596778259 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e580c70b0 0x7f1e581527e0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c00e780 tx=0x7f1e4c00ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:15:02.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.162+0000 7f1e3f7fe640 1 -- 172.21.15.49:0/1596778259 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e4c00c6c0 con 0x7f1e580c70b0 2023-11-06T02:15:02.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.162+0000 7f1e3f7fe640 1 -- 172.21.15.49:0/1596778259 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e4c00c840 con 0x7f1e580c70b0 2023-11-06T02:15:02.169 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.162+0000 7f1e5d0dc640 1 -- 172.21.15.49:0/1596778259 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e580cb490 con 0x7f1e580c70b0 2023-11-06T02:15:02.169 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.162+0000 7f1e3f7fe640 1 -- 172.21.15.49:0/1596778259 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e4c00cbf0 con 0x7f1e580c70b0 2023-11-06T02:15:02.169 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.162+0000 7f1e5d0dc640 1 -- 172.21.15.49:0/1596778259 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e580c7510 con 0x7f1e580c70b0 2023-11-06T02:15:02.169 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.162+0000 7f1e5d0dc640 1 -- 172.21.15.49:0/1596778259 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1e5815d2f0 con 0x7f1e580c70b0 2023-11-06T02:15:02.169 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.166+0000 7f1e3f7fe640 1 -- 172.21.15.49:0/1596778259 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f1e4c00cd50 con 0x7f1e580c70b0 2023-11-06T02:15:02.170 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.166+0000 7f1e3f7fe640 1 --2- 172.21.15.49:0/1596778259 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1e44065a60 0x7f1e44067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:02.170 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.166+0000 7f1e3f7fe640 1 -- 172.21.15.49:0/1596778259 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f1e4c083810 con 0x7f1e580c70b0 2023-11-06T02:15:02.170 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.166+0000 7f1e3f7fe640 1 --2- 172.21.15.49:0/1596778259 >> [v2:172.21.15.49:6824/38115,v1:172.21.15.49:6825/38115] conn(0x7f1e4406c020 0x7f1e4406e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:02.170 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.166+0000 7f1e3f7fe640 1 -- 172.21.15.49:0/1596778259 --> [v2:172.21.15.49:6824/38115,v1:172.21.15.49:6825/38115] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1e4406eaf0 con 0x7f1e4406c020 2023-11-06T02:15:02.170 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.166+0000 7f1e3f7fe640 1 -- 172.21.15.49:0/1596778259 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1e4c083c60 con 0x7f1e580c70b0 2023-11-06T02:15:02.171 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.166+0000 7f1e56d9d640 1 --2- 172.21.15.49:0/1596778259 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1e44065a60 0x7f1e44067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:02.171 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.166+0000 7f1e5759e640 1 --2- 172.21.15.49:0/1596778259 >> [v2:172.21.15.49:6824/38115,v1:172.21.15.49:6825/38115] conn(0x7f1e4406c020 0x7f1e4406e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:02.171 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.166+0000 7f1e5759e640 1 --2- 172.21.15.49:0/1596778259 >> [v2:172.21.15.49:6824/38115,v1:172.21.15.49:6825/38115] conn(0x7f1e4406c020 0x7f1e4406e440 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:15:02.171 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.166+0000 7f1e56d9d640 1 --2- 172.21.15.49:0/1596778259 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1e44065a60 0x7f1e44067f20 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f1e48007fd0 tx=0x7f1e48009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:15:02.171 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.166+0000 7f1e3f7fe640 1 -- 172.21.15.49:0/1596778259 <== osd.3 v2:172.21.15.49:6824/38115 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f1e4406eaf0 con 0x7f1e4406c020 2023-11-06T02:15:02.174 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.178+0000 7f1e5d0dc640 1 -- 172.21.15.49:0/1596778259 --> [v2:172.21.15.49:6824/38115,v1:172.21.15.49:6825/38115] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f1e580c77f0 con 0x7f1e4406c020 2023-11-06T02:15:02.174 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.178+0000 7f1e3f7fe640 1 -- 172.21.15.49:0/1596778259 <== osd.3 v2:172.21.15.49:6824/38115 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f1e580c77f0 con 0x7f1e4406c020 2023-11-06T02:15:02.174 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-11-06T02:15:02.175 INFO:teuthology.orchestra.run.smithi049.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-06T02:15:02.175 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-11-06T02:15:02.176 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.178+0000 7f1e5d0dc640 1 -- 172.21.15.49:0/1596778259 >> [v2:172.21.15.49:6824/38115,v1:172.21.15.49:6825/38115] conn(0x7f1e4406c020 msgr2=0x7f1e4406e440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:02.176 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.178+0000 7f1e5d0dc640 1 --2- 172.21.15.49:0/1596778259 >> [v2:172.21.15.49:6824/38115,v1:172.21.15.49:6825/38115] conn(0x7f1e4406c020 0x7f1e4406e440 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:02.176 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.178+0000 7f1e5d0dc640 1 -- 172.21.15.49:0/1596778259 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1e44065a60 msgr2=0x7f1e44067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:02.176 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.178+0000 7f1e5d0dc640 1 --2- 172.21.15.49:0/1596778259 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f1e44065a60 0x7f1e44067f20 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f1e48007fd0 tx=0x7f1e48009040 comp rx=0 tx=0).stop 2023-11-06T02:15:02.177 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.178+0000 7f1e5d0dc640 1 -- 172.21.15.49:0/1596778259 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e580c70b0 msgr2=0x7f1e581527e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:02.177 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.178+0000 7f1e5d0dc640 1 --2- 172.21.15.49:0/1596778259 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e580c70b0 0x7f1e581527e0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c00e780 tx=0x7f1e4c00ec40 comp rx=0 tx=0).stop 2023-11-06T02:15:02.177 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.178+0000 7f1e5759e640 1 -- 172.21.15.49:0/1596778259 reap_dead start 2023-11-06T02:15:02.177 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.178+0000 7f1e5d0dc640 1 -- 172.21.15.49:0/1596778259 shutdown_connections 2023-11-06T02:15:02.177 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.178+0000 7f1e5d0dc640 1 -- 172.21.15.49:0/1596778259 >> 172.21.15.49:0/1596778259 conn(0x7f1e580d2aa0 msgr2=0x7f1e58156ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:15:02.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.178+0000 7f1e5d0dc640 1 -- 172.21.15.49:0/1596778259 shutdown_connections 2023-11-06T02:15:02.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.178+0000 7f1e5d0dc640 1 -- 172.21.15.49:0/1596778259 wait complete. 2023-11-06T02:15:02.191 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-11-06T02:15:02.316 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.322+0000 7f4656ee4640 1 Processor -- start 2023-11-06T02:15:02.316 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.322+0000 7f4656ee4640 1 -- start start 2023-11-06T02:15:02.317 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.322+0000 7f4656ee4640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f46500b6100 0x7f46500cb960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:02.318 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.322+0000 7f4656ee4640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f465015e440 0x7f465015b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:02.318 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.322+0000 7f4656ee4640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f465015f3c0 con 0x7f46500cb580 2023-11-06T02:15:02.318 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.322+0000 7f4656ee4640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f465015e890 con 0x7f465015e440 2023-11-06T02:15:02.318 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.322+0000 7f4656ee4640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f465015eb20 con 0x7f46500b6100 2023-11-06T02:15:02.319 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.322+0000 7f4655481640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f465015e440 0x7f465015b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:02.319 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.322+0000 7f4655481640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f465015e440 0x7f465015b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56448/0 (socket says 172.21.15.49:56448) 2023-11-06T02:15:02.319 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.322+0000 7f4655481640 1 -- 172.21.15.49:0/1653954856 learned_addr learned my addr 172.21.15.49:0/1653954856 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:15:02.319 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.322+0000 7f464ffff640 1 --2- 172.21.15.49:0/1653954856 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f46500b6100 0x7f46500cb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:02.319 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.322+0000 7f4655481640 1 -- 172.21.15.49:0/1653954856 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f46500b6100 msgr2=0x7f46500cb960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:15:02.320 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.322+0000 7f4655481640 1 --2- 172.21.15.49:0/1653954856 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f46500b6100 0x7f46500cb960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:02.320 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.322+0000 7f4655481640 1 -- 172.21.15.49:0/1653954856 >> v1:172.21.15.150:6789/0 conn(0x7f46500cb580 legacy=0x7f46500ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:15:02.320 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.322+0000 7f4655481640 1 -- 172.21.15.49:0/1653954856 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f465015ee00 con 0x7f465015e440 2023-11-06T02:15:02.321 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.326+0000 7f4655481640 1 --2- 172.21.15.49:0/1653954856 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f465015e440 0x7f465015b510 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f4640009870 tx=0x7f464001c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32be789d22c7ebbb server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:15:02.321 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.326+0000 7f464f7fe640 1 -- 172.21.15.49:0/1653954856 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46400ac640 con 0x7f465015e440 2023-11-06T02:15:02.321 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.326+0000 7f464f7fe640 1 -- 172.21.15.49:0/1653954856 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46400ac7a0 con 0x7f465015e440 2023-11-06T02:15:02.321 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.326+0000 7f464f7fe640 1 -- 172.21.15.49:0/1653954856 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46400acb30 con 0x7f465015e440 2023-11-06T02:15:02.322 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.326+0000 7f4656ee4640 1 -- 172.21.15.49:0/1653954856 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f465015e440 msgr2=0x7f465015b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:02.323 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.326+0000 7f4656ee4640 1 --2- 172.21.15.49:0/1653954856 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f465015e440 0x7f465015b510 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f4640009870 tx=0x7f464001c440 comp rx=0 tx=0).stop 2023-11-06T02:15:02.323 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.326+0000 7f4656ee4640 1 -- 172.21.15.49:0/1653954856 shutdown_connections 2023-11-06T02:15:02.323 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.326+0000 7f4656ee4640 1 --2- 172.21.15.49:0/1653954856 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f46500b6100 0x7f46500cb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:02.323 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.326+0000 7f4656ee4640 1 --2- 172.21.15.49:0/1653954856 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f465015e440 0x7f465015b510 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:02.323 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.326+0000 7f4656ee4640 1 -- 172.21.15.49:0/1653954856 >> 172.21.15.49:0/1653954856 conn(0x7f46500cd2d0 msgr2=0x7f46500cd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:15:02.324 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.326+0000 7f4656ee4640 1 -- 172.21.15.49:0/1653954856 shutdown_connections 2023-11-06T02:15:02.324 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.326+0000 7f4656ee4640 1 -- 172.21.15.49:0/1653954856 wait complete. 2023-11-06T02:15:02.324 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.326+0000 7f4656ee4640 1 Processor -- start 2023-11-06T02:15:02.324 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.330+0000 7f4656ee4640 1 -- start start 2023-11-06T02:15:02.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.330+0000 7f4656ee4640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46500b6100 0x7f4650159d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:02.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.330+0000 7f4656ee4640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f46500cb580 0x7f46501555c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:02.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.330+0000 7f4656ee4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f465015e440 0x7f46500ca1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:02.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.330+0000 7f4656ee4640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46500c7920 con 0x7f465015e440 2023-11-06T02:15:02.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.330+0000 7f4656ee4640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46500c8180 con 0x7f46500b6100 2023-11-06T02:15:02.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.330+0000 7f4656ee4640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4650155b00 con 0x7f46500cb580 2023-11-06T02:15:02.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.330+0000 7f4655481640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f465015e440 0x7f46500ca1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:02.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.330+0000 7f4655481640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f465015e440 0x7f46500ca1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56454/0 (socket says 172.21.15.49:56454) 2023-11-06T02:15:02.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.330+0000 7f4655481640 1 -- 172.21.15.49:0/1234121348 learned_addr learned my addr 172.21.15.49:0/1234121348 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:15:02.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.330+0000 7f4654c80640 1 --2- 172.21.15.49:0/1234121348 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46500b6100 0x7f4650159d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:02.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.330+0000 7f464ffff640 1 --2- 172.21.15.49:0/1234121348 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f46500cb580 0x7f46501555c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:02.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.330+0000 7f4655481640 1 -- 172.21.15.49:0/1234121348 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f46500cb580 msgr2=0x7f46501555c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:02.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.330+0000 7f4655481640 1 --2- 172.21.15.49:0/1234121348 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f46500cb580 0x7f46501555c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:02.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.330+0000 7f4655481640 1 -- 172.21.15.49:0/1234121348 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46500b6100 msgr2=0x7f4650159d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:02.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.330+0000 7f4655481640 1 --2- 172.21.15.49:0/1234121348 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46500b6100 0x7f4650159d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:02.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.330+0000 7f4655481640 1 -- 172.21.15.49:0/1234121348 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46400094d0 con 0x7f465015e440 2023-11-06T02:15:02.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.330+0000 7f4654c80640 1 --2- 172.21.15.49:0/1234121348 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46500b6100 0x7f4650159d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:15:02.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.330+0000 7f4655481640 1 --2- 172.21.15.49:0/1234121348 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f465015e440 0x7f46500ca1d0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f464001aaf0 tx=0x7f464001ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:15:02.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.330+0000 7f464d7fa640 1 -- 172.21.15.49:0/1234121348 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46400b7620 con 0x7f465015e440 2023-11-06T02:15:02.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.330+0000 7f464d7fa640 1 -- 172.21.15.49:0/1234121348 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46400b7780 con 0x7f465015e440 2023-11-06T02:15:02.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.330+0000 7f464d7fa640 1 -- 172.21.15.49:0/1234121348 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46400b7b30 con 0x7f465015e440 2023-11-06T02:15:02.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.330+0000 7f4656ee4640 1 -- 172.21.15.49:0/1234121348 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46500cecb0 con 0x7f465015e440 2023-11-06T02:15:02.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.330+0000 7f4656ee4640 1 -- 172.21.15.49:0/1234121348 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46500c7b30 con 0x7f465015e440 2023-11-06T02:15:02.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.330+0000 7f4656ee4640 1 -- 172.21.15.49:0/1234121348 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46500c82f0 con 0x7f465015e440 2023-11-06T02:15:02.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.330+0000 7f464d7fa640 1 -- 172.21.15.49:0/1234121348 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f46400b3040 con 0x7f465015e440 2023-11-06T02:15:02.334 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.330+0000 7f464d7fa640 1 --2- 172.21.15.49:0/1234121348 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4630065a90 0x7f4630067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:02.334 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.330+0000 7f464d7fa640 1 -- 172.21.15.49:0/1234121348 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f464011d990 con 0x7f465015e440 2023-11-06T02:15:02.334 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.330+0000 7f4654c80640 1 --2- 172.21.15.49:0/1234121348 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4630065a90 0x7f4630067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:02.335 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.334+0000 7f4654c80640 1 --2- 172.21.15.49:0/1234121348 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4630065a90 0x7f4630067f50 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f46501f6f30 tx=0x7f4648009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:15:02.335 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.338+0000 7f464d7fa640 1 -- 172.21.15.49:0/1234121348 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f46400eb450 con 0x7f465015e440 2023-11-06T02:15:02.489 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.494+0000 7f4656ee4640 1 -- 172.21.15.49:0/1234121348 --> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f46500ceb30 con 0x7f4630065a90 2023-11-06T02:15:02.490 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.494+0000 7f464d7fa640 1 -- 172.21.15.49:0/1234121348 <== mgr.4100 v2:172.21.15.150:6832/38104 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f4648004280 con 0x7f4630065a90 2023-11-06T02:15:02.490 INFO:teuthology.orchestra.run.smithi049.stderr:instructed osd(s) 3 to deep-scrub 2023-11-06T02:15:02.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.498+0000 7f4656ee4640 1 -- 172.21.15.49:0/1234121348 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4630065a90 msgr2=0x7f4630067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:02.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.498+0000 7f4656ee4640 1 --2- 172.21.15.49:0/1234121348 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4630065a90 0x7f4630067f50 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f46501f6f30 tx=0x7f4648009040 comp rx=0 tx=0).stop 2023-11-06T02:15:02.492 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.498+0000 7f4656ee4640 1 -- 172.21.15.49:0/1234121348 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f465015e440 msgr2=0x7f46500ca1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:02.493 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.498+0000 7f4656ee4640 1 --2- 172.21.15.49:0/1234121348 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f465015e440 0x7f46500ca1d0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f464001aaf0 tx=0x7f464001ac80 comp rx=0 tx=0).stop 2023-11-06T02:15:02.493 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.498+0000 7f4656ee4640 1 -- 172.21.15.49:0/1234121348 shutdown_connections 2023-11-06T02:15:02.493 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.498+0000 7f4656ee4640 1 --2- 172.21.15.49:0/1234121348 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f4630065a90 0x7f4630067f50 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:02.493 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.498+0000 7f4656ee4640 1 --2- 172.21.15.49:0/1234121348 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f465015e440 0x7f46500ca1d0 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:02.493 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.498+0000 7f4656ee4640 1 --2- 172.21.15.49:0/1234121348 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f46500cb580 0x7f46501555c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:02.494 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.498+0000 7f4656ee4640 1 --2- 172.21.15.49:0/1234121348 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46500b6100 0x7f4650159d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:02.494 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.498+0000 7f4656ee4640 1 -- 172.21.15.49:0/1234121348 >> 172.21.15.49:0/1234121348 conn(0x7f46500cd2d0 msgr2=0x7f46500b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:15:02.494 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.498+0000 7f4656ee4640 1 -- 172.21.15.49:0/1234121348 shutdown_connections 2023-11-06T02:15:02.494 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.498+0000 7f4656ee4640 1 -- 172.21.15.49:0/1234121348 wait complete. 2023-11-06T02:15:02.510 INFO:tasks.ceph:Scrubbing osd.4 2023-11-06T02:15:02.510 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-11-06T02:15:02.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.646+0000 7f3cec0cd640 1 Processor -- start 2023-11-06T02:15:02.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.646+0000 7f3cec0cd640 1 -- start start 2023-11-06T02:15:02.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.650+0000 7f3cec0cd640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f3ce40714a0 0x7f3ce40cd060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:02.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.650+0000 7f3cec0cd640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f3ce40d25c0 0x7f3ce415d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:02.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.650+0000 7f3cec0cd640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ce415da20 con 0x7f3ce4071880 2023-11-06T02:15:02.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.650+0000 7f3cec0cd640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ce415d830 con 0x7f3ce40714a0 2023-11-06T02:15:02.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.650+0000 7f3cec0cd640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ce40d1d60 con 0x7f3ce40d25c0 2023-11-06T02:15:02.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.650+0000 7f3ce9e69640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f3ce40714a0 0x7f3ce40cd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:02.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.650+0000 7f3cea66a640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f3ce4071880 0x7f3ce40b4520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:35130/0 (socket says 172.21.15.49:35130) 2023-11-06T02:15:02.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.650+0000 7f3cea66a640 1 -- 172.21.15.49:0/1830208381 learned_addr learned my addr 172.21.15.49:0/1830208381 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:15:02.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.650+0000 7f3ce9668640 1 --2- 172.21.15.49:0/1830208381 >> v2:172.21.15.150:3300/0 conn(0x7f3ce40d25c0 0x7f3ce415d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:02.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.650+0000 7f3ce9e69640 1 -- 172.21.15.49:0/1830208381 >> v2:172.21.15.150:3300/0 conn(0x7f3ce40d25c0 msgr2=0x7f3ce415d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:15:02.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.650+0000 7f3ce9e69640 1 --2- 172.21.15.49:0/1830208381 >> v2:172.21.15.150:3300/0 conn(0x7f3ce40d25c0 0x7f3ce415d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:02.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.650+0000 7f3ce9e69640 1 -- 172.21.15.49:0/1830208381 >> v1:172.21.15.49:6789/0 conn(0x7f3ce4071880 legacy=0x7f3ce40b4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:02.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.650+0000 7f3ce9e69640 1 -- 172.21.15.49:0/1830208381 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ce415dd00 con 0x7f3ce40714a0 2023-11-06T02:15:02.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.650+0000 7f3ce9e69640 1 --2- 172.21.15.49:0/1830208381 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ce40714a0 0x7f3ce40cd060 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f3ce4072090 tx=0x7f3ce001c310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=168c9075f28e7ce4 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:15:02.648 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.650+0000 7f3ce8e67640 1 -- 172.21.15.49:0/1830208381 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ce00ab600 con 0x7f3ce40714a0 2023-11-06T02:15:02.648 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.650+0000 7f3ce8e67640 1 -- 172.21.15.49:0/1830208381 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ce00a7050 con 0x7f3ce40714a0 2023-11-06T02:15:02.648 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.650+0000 7f3ce8e67640 1 -- 172.21.15.49:0/1830208381 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ce00ab980 con 0x7f3ce40714a0 2023-11-06T02:15:02.648 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.654+0000 7f3cec0cd640 1 -- 172.21.15.49:0/1830208381 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ce40714a0 msgr2=0x7f3ce40cd060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:02.648 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.654+0000 7f3cec0cd640 1 --2- 172.21.15.49:0/1830208381 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ce40714a0 0x7f3ce40cd060 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f3ce4072090 tx=0x7f3ce001c310 comp rx=0 tx=0).stop 2023-11-06T02:15:02.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.654+0000 7f3cec0cd640 1 -- 172.21.15.49:0/1830208381 shutdown_connections 2023-11-06T02:15:02.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.654+0000 7f3cec0cd640 1 --2- 172.21.15.49:0/1830208381 >> v2:172.21.15.150:3300/0 conn(0x7f3ce40d25c0 0x7f3ce415d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:02.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.654+0000 7f3cec0cd640 1 --2- 172.21.15.49:0/1830208381 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ce40714a0 0x7f3ce40cd060 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:02.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.654+0000 7f3cec0cd640 1 -- 172.21.15.49:0/1830208381 >> 172.21.15.49:0/1830208381 conn(0x7f3ce40d2aa0 msgr2=0x7f3ce40d2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:15:02.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.654+0000 7f3cec0cd640 1 -- 172.21.15.49:0/1830208381 shutdown_connections 2023-11-06T02:15:02.650 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.654+0000 7f3cec0cd640 1 -- 172.21.15.49:0/1830208381 wait complete. 2023-11-06T02:15:02.653 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.654+0000 7f3cec0cd640 1 Processor -- start 2023-11-06T02:15:02.653 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.654+0000 7f3cec0cd640 1 -- start start 2023-11-06T02:15:02.653 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.654+0000 7f3cec0cd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ce40714a0 0x7f3ce4152dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:02.654 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.654+0000 7f3cec0cd640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ce4071880 0x7f3ce40c6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:02.654 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.654+0000 7f3cec0cd640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3ce40d25c0 0x7f3ce40c9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:02.654 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.654+0000 7f3cec0cd640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce415a6d0 con 0x7f3ce40714a0 2023-11-06T02:15:02.654 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.654+0000 7f3cec0cd640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce4153300 con 0x7f3ce4071880 2023-11-06T02:15:02.654 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.654+0000 7f3cec0cd640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce40ca710 con 0x7f3ce40d25c0 2023-11-06T02:15:02.655 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.654+0000 7f3ce9e69640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ce40714a0 0x7f3ce4152dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:02.655 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.654+0000 7f3ce9e69640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ce40714a0 0x7f3ce4152dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56462/0 (socket says 172.21.15.49:56462) 2023-11-06T02:15:02.655 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.654+0000 7f3ce9e69640 1 -- 172.21.15.49:0/3980365043 learned_addr learned my addr 172.21.15.49:0/3980365043 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:15:02.655 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.654+0000 7f3cea66a640 1 --2- 172.21.15.49:0/3980365043 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3ce40d25c0 0x7f3ce40c9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:02.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.654+0000 7f3ce9668640 1 --2- 172.21.15.49:0/3980365043 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ce4071880 0x7f3ce40c6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:02.657 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.654+0000 7f3ce9e69640 1 -- 172.21.15.49:0/3980365043 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3ce40d25c0 msgr2=0x7f3ce40c9ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:02.657 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.654+0000 7f3ce9e69640 1 --2- 172.21.15.49:0/3980365043 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3ce40d25c0 0x7f3ce40c9ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:02.657 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.654+0000 7f3ce9e69640 1 -- 172.21.15.49:0/3980365043 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ce4071880 msgr2=0x7f3ce40c6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:02.657 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.654+0000 7f3ce9e69640 1 --2- 172.21.15.49:0/3980365043 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ce4071880 0x7f3ce40c6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:02.657 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.654+0000 7f3ce9e69640 1 -- 172.21.15.49:0/3980365043 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ce0009420 con 0x7f3ce40714a0 2023-11-06T02:15:02.658 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.654+0000 7f3ce9668640 1 --2- 172.21.15.49:0/3980365043 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ce4071880 0x7f3ce40c6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:15:02.658 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.654+0000 7f3ce9e69640 1 --2- 172.21.15.49:0/3980365043 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ce40714a0 0x7f3ce4152dc0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f3ce00ae040 tx=0x7f3ce001ab50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:15:02.658 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.654+0000 7f3cda7fc640 1 -- 172.21.15.49:0/3980365043 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ce00b7650 con 0x7f3ce40714a0 2023-11-06T02:15:02.658 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.654+0000 7f3cda7fc640 1 -- 172.21.15.49:0/3980365043 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ce00b77d0 con 0x7f3ce40714a0 2023-11-06T02:15:02.658 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.654+0000 7f3cec0cd640 1 -- 172.21.15.49:0/3980365043 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ce40ca430 con 0x7f3ce40714a0 2023-11-06T02:15:02.659 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.654+0000 7f3cda7fc640 1 -- 172.21.15.49:0/3980365043 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ce00b7b80 con 0x7f3ce40714a0 2023-11-06T02:15:02.659 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.654+0000 7f3cec0cd640 1 -- 172.21.15.49:0/3980365043 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ce415ac50 con 0x7f3ce40714a0 2023-11-06T02:15:02.659 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.658+0000 7f3cec0cd640 1 -- 172.21.15.49:0/3980365043 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3ce415e7b0 con 0x7f3ce40714a0 2023-11-06T02:15:02.659 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.658+0000 7f3cda7fc640 1 -- 172.21.15.49:0/3980365043 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f3ce00b7ce0 con 0x7f3ce40714a0 2023-11-06T02:15:02.659 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.658+0000 7f3cda7fc640 1 --2- 172.21.15.49:0/3980365043 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3cc8065a60 0x7f3cc8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:02.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.658+0000 7f3cda7fc640 1 -- 172.21.15.49:0/3980365043 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f3ce011d970 con 0x7f3ce40714a0 2023-11-06T02:15:02.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.658+0000 7f3cda7fc640 1 --2- 172.21.15.49:0/3980365043 >> [v2:172.21.15.150:6808/38417,v1:172.21.15.150:6809/38417] conn(0x7f3cc806c020 0x7f3cc806e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:02.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.658+0000 7f3cda7fc640 1 -- 172.21.15.49:0/3980365043 --> [v2:172.21.15.150:6808/38417,v1:172.21.15.150:6809/38417] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3cc806eaf0 con 0x7f3cc806c020 2023-11-06T02:15:02.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.658+0000 7f3cda7fc640 1 -- 172.21.15.49:0/3980365043 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ce01222c0 con 0x7f3ce40714a0 2023-11-06T02:15:02.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.658+0000 7f3ce9668640 1 --2- 172.21.15.49:0/3980365043 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3cc8065a60 0x7f3cc8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:02.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.658+0000 7f3cea66a640 1 --2- 172.21.15.49:0/3980365043 >> [v2:172.21.15.150:6808/38417,v1:172.21.15.150:6809/38417] conn(0x7f3cc806c020 0x7f3cc806e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:02.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.658+0000 7f3cea66a640 1 --2- 172.21.15.49:0/3980365043 >> [v2:172.21.15.150:6808/38417,v1:172.21.15.150:6809/38417] conn(0x7f3cc806c020 0x7f3cc806e440 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:15:02.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.658+0000 7f3ce9668640 1 --2- 172.21.15.49:0/3980365043 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3cc8065a60 0x7f3cc8067f20 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f3ce40b7530 tx=0x7f3cd4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:15:02.662 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.662+0000 7f3cda7fc640 1 -- 172.21.15.49:0/3980365043 <== osd.4 v2:172.21.15.150:6808/38417 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f3cc806eaf0 con 0x7f3cc806c020 2023-11-06T02:15:02.667 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.674+0000 7f3cec0cd640 1 -- 172.21.15.49:0/3980365043 --> [v2:172.21.15.150:6808/38417,v1:172.21.15.150:6809/38417] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f3ce415a530 con 0x7f3cc806c020 2023-11-06T02:15:02.668 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.674+0000 7f3cda7fc640 1 -- 172.21.15.49:0/3980365043 <== osd.4 v2:172.21.15.150:6808/38417 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f3ce415a530 con 0x7f3cc806c020 2023-11-06T02:15:02.668 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-11-06T02:15:02.669 INFO:teuthology.orchestra.run.smithi049.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-06T02:15:02.669 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-11-06T02:15:02.670 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.674+0000 7f3cec0cd640 1 -- 172.21.15.49:0/3980365043 >> [v2:172.21.15.150:6808/38417,v1:172.21.15.150:6809/38417] conn(0x7f3cc806c020 msgr2=0x7f3cc806e440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:02.670 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.674+0000 7f3cec0cd640 1 --2- 172.21.15.49:0/3980365043 >> [v2:172.21.15.150:6808/38417,v1:172.21.15.150:6809/38417] conn(0x7f3cc806c020 0x7f3cc806e440 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:02.670 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.674+0000 7f3cec0cd640 1 -- 172.21.15.49:0/3980365043 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3cc8065a60 msgr2=0x7f3cc8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:02.670 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.674+0000 7f3cec0cd640 1 --2- 172.21.15.49:0/3980365043 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f3cc8065a60 0x7f3cc8067f20 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f3ce40b7530 tx=0x7f3cd4008040 comp rx=0 tx=0).stop 2023-11-06T02:15:02.671 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.674+0000 7f3cec0cd640 1 -- 172.21.15.49:0/3980365043 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ce40714a0 msgr2=0x7f3ce4152dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:02.671 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.674+0000 7f3cec0cd640 1 --2- 172.21.15.49:0/3980365043 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ce40714a0 0x7f3ce4152dc0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f3ce00ae040 tx=0x7f3ce001ab50 comp rx=0 tx=0).stop 2023-11-06T02:15:02.671 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.674+0000 7f3cea66a640 1 -- 172.21.15.49:0/3980365043 reap_dead start 2023-11-06T02:15:02.671 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.674+0000 7f3cec0cd640 1 -- 172.21.15.49:0/3980365043 shutdown_connections 2023-11-06T02:15:02.671 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.674+0000 7f3cec0cd640 1 -- 172.21.15.49:0/3980365043 >> 172.21.15.49:0/3980365043 conn(0x7f3ce40d2aa0 msgr2=0x7f3ce40b7b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:15:02.672 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.674+0000 7f3cec0cd640 1 -- 172.21.15.49:0/3980365043 shutdown_connections 2023-11-06T02:15:02.672 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.674+0000 7f3cec0cd640 1 -- 172.21.15.49:0/3980365043 wait complete. 2023-11-06T02:15:02.685 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-11-06T02:15:02.800 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.806+0000 7ff49af24640 1 Processor -- start 2023-11-06T02:15:02.801 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.806+0000 7ff49af24640 1 -- start start 2023-11-06T02:15:02.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.806+0000 7ff49af24640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff49415ab50 0x7ff4940c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:02.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.806+0000 7ff49af24640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4940cab80 con 0x7ff4940b0790 2023-11-06T02:15:02.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.806+0000 7ff49af24640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff49415af30 con 0x7ff4940cb010 2023-11-06T02:15:02.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.806+0000 7ff49af24640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4940cb5b0 con 0x7ff49415ab50 2023-11-06T02:15:02.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.806+0000 7ff498cc0640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff49415ab50 0x7ff4940c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:02.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.806+0000 7ff4994c1640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7ff4940b0790 0x7ff4940c2710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:35142/0 (socket says 172.21.15.49:35142) 2023-11-06T02:15:02.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.806+0000 7ff4994c1640 1 -- 172.21.15.49:0/3018679369 learned_addr learned my addr 172.21.15.49:0/3018679369 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:15:02.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.806+0000 7ff498cc0640 1 -- 172.21.15.49:0/3018679369 >> v1:172.21.15.150:6789/0 conn(0x7ff4940cb010 legacy=0x7ff4940b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:15:02.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.806+0000 7ff498cc0640 1 -- 172.21.15.49:0/3018679369 >> v1:172.21.15.49:6789/0 conn(0x7ff4940b0790 legacy=0x7ff4940c2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:02.804 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.806+0000 7ff498cc0640 1 -- 172.21.15.49:0/3018679369 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4940cb890 con 0x7ff49415ab50 2023-11-06T02:15:02.804 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.810+0000 7ff498cc0640 1 --2- 172.21.15.49:0/3018679369 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff49415ab50 0x7ff4940c9bb0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7ff48c0097a0 tx=0x7ff48c01c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eebf4e29e84fdd2c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:15:02.804 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.810+0000 7ff4937fe640 1 -- 172.21.15.49:0/3018679369 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff48c0ab5f0 con 0x7ff49415ab50 2023-11-06T02:15:02.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.810+0000 7ff4937fe640 1 -- 172.21.15.49:0/3018679369 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff48c0ab750 con 0x7ff49415ab50 2023-11-06T02:15:02.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.810+0000 7ff4937fe640 1 -- 172.21.15.49:0/3018679369 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff48c0abae0 con 0x7ff49415ab50 2023-11-06T02:15:02.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.810+0000 7ff49af24640 1 -- 172.21.15.49:0/3018679369 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff49415ab50 msgr2=0x7ff4940c9bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:02.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.810+0000 7ff49af24640 1 --2- 172.21.15.49:0/3018679369 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff49415ab50 0x7ff4940c9bb0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7ff48c0097a0 tx=0x7ff48c01c3f0 comp rx=0 tx=0).stop 2023-11-06T02:15:02.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.810+0000 7ff49af24640 1 -- 172.21.15.49:0/3018679369 shutdown_connections 2023-11-06T02:15:02.807 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.810+0000 7ff49af24640 1 --2- 172.21.15.49:0/3018679369 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff49415ab50 0x7ff4940c9bb0 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:02.807 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.810+0000 7ff49af24640 1 -- 172.21.15.49:0/3018679369 >> 172.21.15.49:0/3018679369 conn(0x7ff49406c0a0 msgr2=0x7ff49406c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:15:02.807 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.810+0000 7ff49af24640 1 -- 172.21.15.49:0/3018679369 shutdown_connections 2023-11-06T02:15:02.807 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.810+0000 7ff49af24640 1 -- 172.21.15.49:0/3018679369 wait complete. 2023-11-06T02:15:02.807 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.810+0000 7ff49af24640 1 Processor -- start 2023-11-06T02:15:02.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.810+0000 7ff49af24640 1 -- start start 2023-11-06T02:15:02.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.814+0000 7ff49af24640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4940b0790 0x7ff49415ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:02.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.814+0000 7ff49af24640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4940cb010 0x7ff494147740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:02.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.814+0000 7ff49af24640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff49415ab50 0x7ff4940cd230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:02.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.814+0000 7ff49af24640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff49415bf50 con 0x7ff49415ab50 2023-11-06T02:15:02.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.814+0000 7ff49af24640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff494147c80 con 0x7ff4940b0790 2023-11-06T02:15:02.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.814+0000 7ff49af24640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff494156720 con 0x7ff4940cb010 2023-11-06T02:15:02.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.814+0000 7ff4994c1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff49415ab50 0x7ff4940cd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:02.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.814+0000 7ff4994c1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff49415ab50 0x7ff4940cd230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56478/0 (socket says 172.21.15.49:56478) 2023-11-06T02:15:02.812 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.814+0000 7ff4994c1640 1 -- 172.21.15.49:0/1943657504 learned_addr learned my addr 172.21.15.49:0/1943657504 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:15:02.812 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.814+0000 7ff498cc0640 1 --2- 172.21.15.49:0/1943657504 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4940b0790 0x7ff49415ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:02.812 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.814+0000 7ff493fff640 1 --2- 172.21.15.49:0/1943657504 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4940cb010 0x7ff494147740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:02.812 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.814+0000 7ff4994c1640 1 -- 172.21.15.49:0/1943657504 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4940cb010 msgr2=0x7ff494147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:02.812 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.814+0000 7ff4994c1640 1 --2- 172.21.15.49:0/1943657504 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4940cb010 0x7ff494147740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:02.812 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.814+0000 7ff4994c1640 1 -- 172.21.15.49:0/1943657504 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4940b0790 msgr2=0x7ff49415ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:02.813 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.814+0000 7ff4994c1640 1 --2- 172.21.15.49:0/1943657504 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4940b0790 0x7ff49415ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:02.813 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.814+0000 7ff4994c1640 1 -- 172.21.15.49:0/1943657504 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff48c0093b0 con 0x7ff49415ab50 2023-11-06T02:15:02.813 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.814+0000 7ff498cc0640 1 --2- 172.21.15.49:0/1943657504 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4940b0790 0x7ff49415ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:15:02.813 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.814+0000 7ff493fff640 1 --2- 172.21.15.49:0/1943657504 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4940cb010 0x7ff494147740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:15:02.813 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.814+0000 7ff4994c1640 1 --2- 172.21.15.49:0/1943657504 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff49415ab50 0x7ff4940cd230 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7ff48400c400 tx=0x7ff48400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:15:02.814 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.814+0000 7ff4917fa640 1 -- 172.21.15.49:0/1943657504 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff484005cf0 con 0x7ff49415ab50 2023-11-06T02:15:02.814 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.814+0000 7ff4917fa640 1 -- 172.21.15.49:0/1943657504 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff484005e50 con 0x7ff49415ab50 2023-11-06T02:15:02.814 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.814+0000 7ff49af24640 1 -- 172.21.15.49:0/1943657504 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4941571b0 con 0x7ff49415ab50 2023-11-06T02:15:02.814 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.814+0000 7ff4917fa640 1 -- 172.21.15.49:0/1943657504 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4840072f0 con 0x7ff49415ab50 2023-11-06T02:15:02.814 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.814+0000 7ff49af24640 1 -- 172.21.15.49:0/1943657504 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff494157770 con 0x7ff49415ab50 2023-11-06T02:15:02.815 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.814+0000 7ff49af24640 1 -- 172.21.15.49:0/1943657504 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4940b0de0 con 0x7ff49415ab50 2023-11-06T02:15:02.815 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.814+0000 7ff4917fa640 1 -- 172.21.15.49:0/1943657504 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7ff484007450 con 0x7ff49415ab50 2023-11-06T02:15:02.815 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.814+0000 7ff4917fa640 1 --2- 172.21.15.49:0/1943657504 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff474065a60 0x7ff474067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:02.815 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.814+0000 7ff4917fa640 1 -- 172.21.15.49:0/1943657504 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5468+0+0 (secure 0 0 0) 0x7ff484082580 con 0x7ff49415ab50 2023-11-06T02:15:02.816 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.814+0000 7ff498cc0640 1 --2- 172.21.15.49:0/1943657504 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff474065a60 0x7ff474067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:02.816 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.818+0000 7ff498cc0640 1 --2- 172.21.15.49:0/1943657504 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff474065a60 0x7ff474067f20 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7ff48c0af000 tx=0x7ff48c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:15:02.816 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.818+0000 7ff4917fa640 1 -- 172.21.15.49:0/1943657504 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7ff4840500c0 con 0x7ff49415ab50 2023-11-06T02:15:02.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.974+0000 7ff49af24640 1 -- 172.21.15.49:0/1943657504 --> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7ff4940cbe50 con 0x7ff474065a60 2023-11-06T02:15:02.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.978+0000 7ff4917fa640 1 -- 172.21.15.49:0/1943657504 <== mgr.4100 v2:172.21.15.150:6832/38104 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7ff48c0bc090 con 0x7ff474065a60 2023-11-06T02:15:02.971 INFO:teuthology.orchestra.run.smithi049.stderr:instructed osd(s) 4 to deep-scrub 2023-11-06T02:15:02.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.978+0000 7ff49af24640 1 -- 172.21.15.49:0/1943657504 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff474065a60 msgr2=0x7ff474067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:02.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.978+0000 7ff49af24640 1 --2- 172.21.15.49:0/1943657504 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff474065a60 0x7ff474067f20 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7ff48c0af000 tx=0x7ff48c008040 comp rx=0 tx=0).stop 2023-11-06T02:15:02.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.978+0000 7ff49af24640 1 -- 172.21.15.49:0/1943657504 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff49415ab50 msgr2=0x7ff4940cd230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:02.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.978+0000 7ff49af24640 1 --2- 172.21.15.49:0/1943657504 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff49415ab50 0x7ff4940cd230 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7ff48400c400 tx=0x7ff48400c8c0 comp rx=0 tx=0).stop 2023-11-06T02:15:02.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.978+0000 7ff49af24640 1 -- 172.21.15.49:0/1943657504 shutdown_connections 2023-11-06T02:15:02.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.978+0000 7ff49af24640 1 --2- 172.21.15.49:0/1943657504 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7ff474065a60 0x7ff474067f20 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:02.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.978+0000 7ff49af24640 1 --2- 172.21.15.49:0/1943657504 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff49415ab50 0x7ff4940cd230 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:02.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.978+0000 7ff49af24640 1 --2- 172.21.15.49:0/1943657504 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4940cb010 0x7ff494147740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:02.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.978+0000 7ff49af24640 1 --2- 172.21.15.49:0/1943657504 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4940b0790 0x7ff49415ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:02.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.978+0000 7ff49af24640 1 -- 172.21.15.49:0/1943657504 >> 172.21.15.49:0/1943657504 conn(0x7ff49406c0a0 msgr2=0x7ff4940730c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:15:02.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.978+0000 7ff49af24640 1 -- 172.21.15.49:0/1943657504 shutdown_connections 2023-11-06T02:15:02.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:02.978+0000 7ff49af24640 1 -- 172.21.15.49:0/1943657504 wait complete. 2023-11-06T02:15:02.990 INFO:tasks.ceph:Scrubbing osd.5 2023-11-06T02:15:02.990 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-11-06T02:15:03.120 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.126+0000 7f5e6d1be640 1 Processor -- start 2023-11-06T02:15:03.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.126+0000 7f5e6d1be640 1 -- start start 2023-11-06T02:15:03.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.126+0000 7f5e6d1be640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f5e6806a3b0 0x7f5e680caa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:03.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.126+0000 7f5e6d1be640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e68071e80 con 0x7f5e68069fd0 2023-11-06T02:15:03.122 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.126+0000 7f5e6d1be640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e68072160 con 0x7f5e680ad720 2023-11-06T02:15:03.122 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.126+0000 7f5e6d1be640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e680ca220 con 0x7f5e6806a3b0 2023-11-06T02:15:03.122 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.126+0000 7f5e66d9d640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f5e68069fd0 0x7f5e680715c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:35144/0 (socket says 172.21.15.49:35144) 2023-11-06T02:15:03.122 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.126+0000 7f5e66d9d640 1 -- 172.21.15.49:0/4175604489 learned_addr learned my addr 172.21.15.49:0/4175604489 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:15:03.124 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.126+0000 7f5e65d9b640 1 -- 172.21.15.49:0/4175604489 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 81333049 0 0) 0x7f5e68071e80 con 0x7f5e68069fd0 2023-11-06T02:15:03.124 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.126+0000 7f5e6659c640 1 --2- 172.21.15.49:0/4175604489 >> v2:172.21.15.150:3300/0 conn(0x7f5e6806a3b0 0x7f5e680caa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:03.124 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.126+0000 7f5e65d9b640 1 -- 172.21.15.49:0/4175604489 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e50003530 con 0x7f5e68069fd0 2023-11-06T02:15:03.124 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.126+0000 7f5e65d9b640 1 -- 172.21.15.49:0/4175604489 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3240680846 0 0) 0x7f5e50003530 con 0x7f5e68069fd0 2023-11-06T02:15:03.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.126+0000 7f5e65d9b640 1 -- 172.21.15.49:0/4175604489 >> v2:172.21.15.150:3300/0 conn(0x7f5e6806a3b0 msgr2=0x7f5e680caa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:15:03.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.126+0000 7f5e65d9b640 1 --2- 172.21.15.49:0/4175604489 >> v2:172.21.15.150:3300/0 conn(0x7f5e6806a3b0 0x7f5e680caa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:03.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.126+0000 7f5e65d9b640 1 -- 172.21.15.49:0/4175604489 >> v1:172.21.15.150:6789/0 conn(0x7f5e680ad720 legacy=0x7f5e68159530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:03.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.126+0000 7f5e65d9b640 1 -- 172.21.15.49:0/4175604489 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e680ca500 con 0x7f5e68069fd0 2023-11-06T02:15:03.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.126+0000 7f5e65d9b640 1 -- 172.21.15.49:0/4175604489 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7f5e5c001680 con 0x7f5e68069fd0 2023-11-06T02:15:03.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.126+0000 7f5e65d9b640 1 -- 172.21.15.49:0/4175604489 >> v1:172.21.15.49:6789/0 conn(0x7f5e68069fd0 legacy=0x7f5e680715c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:03.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.126+0000 7f5e65d9b640 1 --2- 172.21.15.49:0/4175604489 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e50003980 0x7f5e50003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:03.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.126+0000 7f5e65d9b640 1 --2- 172.21.15.49:0/4175604489 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5e500047d0 0x7f5e50004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:03.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.126+0000 7f5e65d9b640 1 --2- 172.21.15.49:0/4175604489 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e50005190 0x7f5e5000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:03.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.126+0000 7f5e65d9b640 1 -- 172.21.15.49:0/4175604489 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e680ca220 con 0x7f5e50003980 2023-11-06T02:15:03.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.126+0000 7f5e65d9b640 1 -- 172.21.15.49:0/4175604489 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5000c030 con 0x7f5e50005190 2023-11-06T02:15:03.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.126+0000 7f5e65d9b640 1 -- 172.21.15.49:0/4175604489 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5000c1f0 con 0x7f5e500047d0 2023-11-06T02:15:03.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.126+0000 7f5e66d9d640 1 --2- 172.21.15.49:0/4175604489 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e50003980 0x7f5e50003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:03.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.126+0000 7f5e6759e640 1 --2- 172.21.15.49:0/4175604489 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e50005190 0x7f5e5000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:03.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.126+0000 7f5e66d9d640 1 -- 172.21.15.49:0/4175604489 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5e500047d0 msgr2=0x7f5e50004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:15:03.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.126+0000 7f5e66d9d640 1 --2- 172.21.15.49:0/4175604489 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5e500047d0 0x7f5e50004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:03.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.126+0000 7f5e66d9d640 1 -- 172.21.15.49:0/4175604489 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e50005190 msgr2=0x7f5e5000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:15:03.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.126+0000 7f5e66d9d640 1 --2- 172.21.15.49:0/4175604489 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e50005190 0x7f5e5000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:03.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.126+0000 7f5e66d9d640 1 -- 172.21.15.49:0/4175604489 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e50003530 con 0x7f5e50003980 2023-11-06T02:15:03.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.130+0000 7f5e6759e640 1 --2- 172.21.15.49:0/4175604489 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e50005190 0x7f5e5000b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:15:03.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.130+0000 7f5e6759e640 1 -- 172.21.15.49:0/4175604489 reap_dead start 2023-11-06T02:15:03.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.130+0000 7f5e66d9d640 1 --2- 172.21.15.49:0/4175604489 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e50003980 0x7f5e50003da0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c00c240 tx=0x7f5e5c01cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed8858feeef7fc7f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:15:03.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.130+0000 7f5e65d9b640 1 -- 172.21.15.49:0/4175604489 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e5c005ae0 con 0x7f5e50003980 2023-11-06T02:15:03.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.130+0000 7f5e65d9b640 1 -- 172.21.15.49:0/4175604489 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e5c005c40 con 0x7f5e50003980 2023-11-06T02:15:03.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.130+0000 7f5e65d9b640 1 -- 172.21.15.49:0/4175604489 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e5c0a5a70 con 0x7f5e50003980 2023-11-06T02:15:03.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.130+0000 7f5e6d1be640 1 -- 172.21.15.49:0/4175604489 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e50003980 msgr2=0x7f5e50003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:03.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.130+0000 7f5e6d1be640 1 --2- 172.21.15.49:0/4175604489 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e50003980 0x7f5e50003da0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c00c240 tx=0x7f5e5c01cc40 comp rx=0 tx=0).stop 2023-11-06T02:15:03.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.130+0000 7f5e6d1be640 1 -- 172.21.15.49:0/4175604489 shutdown_connections 2023-11-06T02:15:03.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.130+0000 7f5e6d1be640 1 --2- 172.21.15.49:0/4175604489 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e50003980 0x7f5e50003da0 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:03.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.130+0000 7f5e6d1be640 1 -- 172.21.15.49:0/4175604489 >> 172.21.15.49:0/4175604489 conn(0x7f5e680ce610 msgr2=0x7f5e680cea20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:15:03.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.130+0000 7f5e6d1be640 1 -- 172.21.15.49:0/4175604489 shutdown_connections 2023-11-06T02:15:03.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.130+0000 7f5e6d1be640 1 -- 172.21.15.49:0/4175604489 wait complete. 2023-11-06T02:15:03.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.134+0000 7f5e6d1be640 1 Processor -- start 2023-11-06T02:15:03.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.134+0000 7f5e6d1be640 1 -- start start 2023-11-06T02:15:03.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.134+0000 7f5e6d1be640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e50003980 0x7f5e6814ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:03.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.134+0000 7f5e6d1be640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5e6814f8b0 0x7f5e680c7d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:03.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.134+0000 7f5e6d1be640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e6814fd30 0x7f5e6815c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:03.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.134+0000 7f5e6d1be640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e680c3e00 con 0x7f5e50003980 2023-11-06T02:15:03.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.134+0000 7f5e6d1be640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e680c3230 con 0x7f5e6814fd30 2023-11-06T02:15:03.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.134+0000 7f5e6d1be640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e680c34d0 con 0x7f5e6814f8b0 2023-11-06T02:15:03.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.134+0000 7f5e66d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e50003980 0x7f5e6814ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:03.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.134+0000 7f5e66d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e50003980 0x7f5e6814ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56506/0 (socket says 172.21.15.49:56506) 2023-11-06T02:15:03.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.134+0000 7f5e66d9d640 1 -- 172.21.15.49:0/2057747902 learned_addr learned my addr 172.21.15.49:0/2057747902 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:15:03.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.134+0000 7f5e6759e640 1 --2- 172.21.15.49:0/2057747902 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e6814fd30 0x7f5e6815c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:03.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.134+0000 7f5e6659c640 1 --2- 172.21.15.49:0/2057747902 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5e6814f8b0 0x7f5e680c7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:03.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.134+0000 7f5e66d9d640 1 -- 172.21.15.49:0/2057747902 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5e6814f8b0 msgr2=0x7f5e680c7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:03.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.134+0000 7f5e66d9d640 1 --2- 172.21.15.49:0/2057747902 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5e6814f8b0 0x7f5e680c7d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:03.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.134+0000 7f5e66d9d640 1 -- 172.21.15.49:0/2057747902 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e6814fd30 msgr2=0x7f5e6815c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:03.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.134+0000 7f5e66d9d640 1 --2- 172.21.15.49:0/2057747902 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e6814fd30 0x7f5e6815c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:03.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.134+0000 7f5e66d9d640 1 -- 172.21.15.49:0/2057747902 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e680c37b0 con 0x7f5e50003980 2023-11-06T02:15:03.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.134+0000 7f5e6659c640 1 --2- 172.21.15.49:0/2057747902 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5e6814f8b0 0x7f5e680c7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:15:03.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.134+0000 7f5e66d9d640 1 --2- 172.21.15.49:0/2057747902 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e50003980 0x7f5e6814ecc0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c00c240 tx=0x7f5e50005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:15:03.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.134+0000 7f5e4f7fe640 1 -- 172.21.15.49:0/2057747902 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e5c0b4070 con 0x7f5e50003980 2023-11-06T02:15:03.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.134+0000 7f5e4f7fe640 1 -- 172.21.15.49:0/2057747902 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e5c0a5830 con 0x7f5e50003980 2023-11-06T02:15:03.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.134+0000 7f5e6d1be640 1 -- 172.21.15.49:0/2057747902 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e680c3a90 con 0x7f5e50003980 2023-11-06T02:15:03.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.134+0000 7f5e4f7fe640 1 -- 172.21.15.49:0/2057747902 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e5c0a5be0 con 0x7f5e50003980 2023-11-06T02:15:03.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.134+0000 7f5e6d1be640 1 -- 172.21.15.49:0/2057747902 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e680c8280 con 0x7f5e50003980 2023-11-06T02:15:03.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.134+0000 7f5e6d1be640 1 -- 172.21.15.49:0/2057747902 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5e680c6870 con 0x7f5e50003980 2023-11-06T02:15:03.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.134+0000 7f5e4f7fe640 1 -- 172.21.15.49:0/2057747902 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f5e5c0a5d40 con 0x7f5e50003980 2023-11-06T02:15:03.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.138+0000 7f5e4f7fe640 1 --2- 172.21.15.49:0/2057747902 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5e54065a90 0x7f5e54067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:03.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.138+0000 7f5e4f7fe640 1 -- 172.21.15.49:0/2057747902 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f5e5c11d140 con 0x7f5e50003980 2023-11-06T02:15:03.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.138+0000 7f5e4f7fe640 1 --2- 172.21.15.49:0/2057747902 >> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] conn(0x7f5e5406c050 0x7f5e5406e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:03.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.138+0000 7f5e4f7fe640 1 -- 172.21.15.49:0/2057747902 --> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5e5406eb20 con 0x7f5e5406c050 2023-11-06T02:15:03.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.138+0000 7f5e4f7fe640 1 -- 172.21.15.49:0/2057747902 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5e5c11d590 con 0x7f5e50003980 2023-11-06T02:15:03.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.138+0000 7f5e6659c640 1 --2- 172.21.15.49:0/2057747902 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5e54065a90 0x7f5e54067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:03.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.138+0000 7f5e6759e640 1 --2- 172.21.15.49:0/2057747902 >> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] conn(0x7f5e5406c050 0x7f5e5406e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:03.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.138+0000 7f5e6759e640 1 --2- 172.21.15.49:0/2057747902 >> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] conn(0x7f5e5406c050 0x7f5e5406e470 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:15:03.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.138+0000 7f5e6659c640 1 --2- 172.21.15.49:0/2057747902 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5e54065a90 0x7f5e54067f50 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f5e681589b0 tx=0x7f5e58008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:15:03.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.138+0000 7f5e4f7fe640 1 -- 172.21.15.49:0/2057747902 <== osd.5 v2:172.21.15.150:6800/38418 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f5e5406eb20 con 0x7f5e5406c050 2023-11-06T02:15:03.146 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.150+0000 7f5e6d1be640 1 -- 172.21.15.49:0/2057747902 --> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f5e680c3f40 con 0x7f5e5406c050 2023-11-06T02:15:03.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.154+0000 7f5e4f7fe640 1 -- 172.21.15.49:0/2057747902 <== osd.5 v2:172.21.15.150:6800/38418 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f5e6000d040 con 0x7f5e5406c050 2023-11-06T02:15:03.147 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-11-06T02:15:03.148 INFO:teuthology.orchestra.run.smithi049.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-06T02:15:03.148 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-11-06T02:15:03.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.154+0000 7f5e6d1be640 1 -- 172.21.15.49:0/2057747902 >> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] conn(0x7f5e5406c050 msgr2=0x7f5e5406e470 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:03.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.154+0000 7f5e6d1be640 1 --2- 172.21.15.49:0/2057747902 >> [v2:172.21.15.150:6800/38418,v1:172.21.15.150:6801/38418] conn(0x7f5e5406c050 0x7f5e5406e470 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:03.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.154+0000 7f5e6d1be640 1 -- 172.21.15.49:0/2057747902 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5e54065a90 msgr2=0x7f5e54067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:03.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.154+0000 7f5e6d1be640 1 --2- 172.21.15.49:0/2057747902 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5e54065a90 0x7f5e54067f50 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f5e681589b0 tx=0x7f5e58008040 comp rx=0 tx=0).stop 2023-11-06T02:15:03.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.154+0000 7f5e6d1be640 1 -- 172.21.15.49:0/2057747902 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e50003980 msgr2=0x7f5e6814ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:03.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.154+0000 7f5e6d1be640 1 --2- 172.21.15.49:0/2057747902 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5e50003980 0x7f5e6814ecc0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c00c240 tx=0x7f5e50005800 comp rx=0 tx=0).stop 2023-11-06T02:15:03.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.154+0000 7f5e6759e640 1 -- 172.21.15.49:0/2057747902 reap_dead start 2023-11-06T02:15:03.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.154+0000 7f5e6d1be640 1 -- 172.21.15.49:0/2057747902 shutdown_connections 2023-11-06T02:15:03.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.154+0000 7f5e6d1be640 1 -- 172.21.15.49:0/2057747902 >> 172.21.15.49:0/2057747902 conn(0x7f5e680ce610 msgr2=0x7f5e680af2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:15:03.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.154+0000 7f5e6d1be640 1 -- 172.21.15.49:0/2057747902 shutdown_connections 2023-11-06T02:15:03.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.154+0000 7f5e6d1be640 1 -- 172.21.15.49:0/2057747902 wait complete. 2023-11-06T02:15:03.163 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-11-06T02:15:03.281 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.285+0000 7f78b9ba6640 1 Processor -- start 2023-11-06T02:15:03.282 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.285+0000 7f78b9ba6640 1 -- start start 2023-11-06T02:15:03.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.285+0000 7f78b9ba6640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f78b40adae0 0x7f78b40c8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:03.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.285+0000 7f78b9ba6640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78b41594d0 con 0x7f78b40b0790 2023-11-06T02:15:03.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.285+0000 7f78b9ba6640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78b41597a0 con 0x7f78b41590f0 2023-11-06T02:15:03.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.285+0000 7f78b9ba6640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78b415a140 con 0x7f78b40adae0 2023-11-06T02:15:03.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.285+0000 7f78abfff640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f78b40b0790 0x7f78b4070b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:35154/0 (socket says 172.21.15.49:35154) 2023-11-06T02:15:03.286 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.285+0000 7f78abfff640 1 -- 172.21.15.49:0/788271470 learned_addr learned my addr 172.21.15.49:0/788271470 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:15:03.286 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.289+0000 7f78b3fff640 1 --2- 172.21.15.49:0/788271470 >> v2:172.21.15.150:3300/0 conn(0x7f78b40adae0 0x7f78b40c8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:03.286 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.289+0000 7f78b2ffd640 1 -- 172.21.15.49:0/788271470 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3045636084 0 0) 0x7f78b41594d0 con 0x7f78b40b0790 2023-11-06T02:15:03.286 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.289+0000 7f78b2ffd640 1 -- 172.21.15.49:0/788271470 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78a0004640 con 0x7f78b40b0790 2023-11-06T02:15:03.286 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.289+0000 7f78b2ffd640 1 -- 172.21.15.49:0/788271470 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3678936738 0 0) 0x7f78a0004640 con 0x7f78b40b0790 2023-11-06T02:15:03.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.289+0000 7f78b2ffd640 1 -- 172.21.15.49:0/788271470 >> v2:172.21.15.150:3300/0 conn(0x7f78b40adae0 msgr2=0x7f78b40c8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:15:03.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.289+0000 7f78b2ffd640 1 --2- 172.21.15.49:0/788271470 >> v2:172.21.15.150:3300/0 conn(0x7f78b40adae0 0x7f78b40c8c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:03.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.289+0000 7f78b2ffd640 1 -- 172.21.15.49:0/788271470 >> v1:172.21.15.150:6789/0 conn(0x7f78b41590f0 legacy=0x7f78b40b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:03.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.289+0000 7f78b2ffd640 1 -- 172.21.15.49:0/788271470 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78b4159a80 con 0x7f78b40b0790 2023-11-06T02:15:03.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.289+0000 7f78b2ffd640 1 -- 172.21.15.49:0/788271470 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7f789c001680 con 0x7f78b40b0790 2023-11-06T02:15:03.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.289+0000 7f78b2ffd640 1 -- 172.21.15.49:0/788271470 >> v1:172.21.15.49:6789/0 conn(0x7f78b40b0790 legacy=0x7f78b4070b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:03.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.289+0000 7f78b2ffd640 1 --2- 172.21.15.49:0/788271470 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78a0004a90 0x7f78a0004eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:03.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.289+0000 7f78b2ffd640 1 --2- 172.21.15.49:0/788271470 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f78a00058e0 0x7f78a0005d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:03.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.289+0000 7f78b2ffd640 1 --2- 172.21.15.49:0/788271470 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f78a00062a0 0x7f78a000caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:03.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.289+0000 7f78b2ffd640 1 -- 172.21.15.49:0/788271470 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78b415a140 con 0x7f78a0004a90 2023-11-06T02:15:03.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.289+0000 7f78b2ffd640 1 -- 172.21.15.49:0/788271470 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a000d240 con 0x7f78a00062a0 2023-11-06T02:15:03.289 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.289+0000 7f78b2ffd640 1 -- 172.21.15.49:0/788271470 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78a000d400 con 0x7f78a00058e0 2023-11-06T02:15:03.289 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.289+0000 7f78b37fe640 1 --2- 172.21.15.49:0/788271470 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78a0004a90 0x7f78a0004eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:03.289 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.289+0000 7f78b3fff640 1 --2- 172.21.15.49:0/788271470 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f78a00062a0 0x7f78a000caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:03.289 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.289+0000 7f78abfff640 1 --2- 172.21.15.49:0/788271470 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f78a00058e0 0x7f78a0005d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:03.289 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.289+0000 7f78b37fe640 1 -- 172.21.15.49:0/788271470 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f78a00058e0 msgr2=0x7f78a0005d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:15:03.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.289+0000 7f78b37fe640 1 --2- 172.21.15.49:0/788271470 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f78a00058e0 0x7f78a0005d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:03.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.289+0000 7f78b37fe640 1 -- 172.21.15.49:0/788271470 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f78a00062a0 msgr2=0x7f78a000caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:15:03.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.289+0000 7f78b37fe640 1 --2- 172.21.15.49:0/788271470 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f78a00062a0 0x7f78a000caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:03.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.289+0000 7f78b37fe640 1 -- 172.21.15.49:0/788271470 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78a0004640 con 0x7f78a0004a90 2023-11-06T02:15:03.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.289+0000 7f78b3fff640 1 --2- 172.21.15.49:0/788271470 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f78a00062a0 0x7f78a000caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:15:03.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.289+0000 7f78b3fff640 1 -- 172.21.15.49:0/788271470 reap_dead start 2023-11-06T02:15:03.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.289+0000 7f78b37fe640 1 --2- 172.21.15.49:0/788271470 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78a0004a90 0x7f78a0004eb0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f78a40017b0 tx=0x7f78a401c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7c77f0555ada3db server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:15:03.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.289+0000 7f78b2ffd640 1 -- 172.21.15.49:0/788271470 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a40052d0 con 0x7f78a0004a90 2023-11-06T02:15:03.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.289+0000 7f78b2ffd640 1 -- 172.21.15.49:0/788271470 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78a40a7050 con 0x7f78a0004a90 2023-11-06T02:15:03.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.289+0000 7f78b2ffd640 1 -- 172.21.15.49:0/788271470 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a4003650 con 0x7f78a0004a90 2023-11-06T02:15:03.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.289+0000 7f78b9ba6640 1 -- 172.21.15.49:0/788271470 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78a0004a90 msgr2=0x7f78a0004eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:03.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.289+0000 7f78b9ba6640 1 --2- 172.21.15.49:0/788271470 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78a0004a90 0x7f78a0004eb0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f78a40017b0 tx=0x7f78a401c3c0 comp rx=0 tx=0).stop 2023-11-06T02:15:03.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.289+0000 7f78b9ba6640 1 -- 172.21.15.49:0/788271470 shutdown_connections 2023-11-06T02:15:03.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.289+0000 7f78b9ba6640 1 --2- 172.21.15.49:0/788271470 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78a0004a90 0x7f78a0004eb0 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:03.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.289+0000 7f78b9ba6640 1 -- 172.21.15.49:0/788271470 >> 172.21.15.49:0/788271470 conn(0x7f78b406c0a0 msgr2=0x7f78b406c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:15:03.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.293+0000 7f78b9ba6640 1 -- 172.21.15.49:0/788271470 shutdown_connections 2023-11-06T02:15:03.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.293+0000 7f78b9ba6640 1 -- 172.21.15.49:0/788271470 wait complete. 2023-11-06T02:15:03.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.293+0000 7f78b9ba6640 1 Processor -- start 2023-11-06T02:15:03.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.293+0000 7f78b9ba6640 1 -- start start 2023-11-06T02:15:03.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.293+0000 7f78b9ba6640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f78a0004a90 0x7f78b40acf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:03.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.293+0000 7f78b9ba6640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f78b40af810 0x7f78b415aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:03.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.293+0000 7f78b9ba6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78b415b530 0x7f78b40ab8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:03.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.293+0000 7f78b9ba6640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78b415b9b0 con 0x7f78b415b530 2023-11-06T02:15:03.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.293+0000 7f78b9ba6640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78b40ad4c0 con 0x7f78a0004a90 2023-11-06T02:15:03.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.293+0000 7f78b9ba6640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78b40afd40 con 0x7f78b40af810 2023-11-06T02:15:03.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.293+0000 7f78b3fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78b415b530 0x7f78b40ab8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:03.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.293+0000 7f78b3fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78b415b530 0x7f78b40ab8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56524/0 (socket says 172.21.15.49:56524) 2023-11-06T02:15:03.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.293+0000 7f78b3fff640 1 -- 172.21.15.49:0/619497200 learned_addr learned my addr 172.21.15.49:0/619497200 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:15:03.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.293+0000 7f78abfff640 1 --2- 172.21.15.49:0/619497200 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f78b40af810 0x7f78b415aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:03.300 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.293+0000 7f78b37fe640 1 --2- 172.21.15.49:0/619497200 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f78a0004a90 0x7f78b40acf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:03.300 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.293+0000 7f78b3fff640 1 -- 172.21.15.49:0/619497200 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f78b40af810 msgr2=0x7f78b415aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:03.300 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.293+0000 7f78b3fff640 1 --2- 172.21.15.49:0/619497200 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f78b40af810 0x7f78b415aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:03.300 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.293+0000 7f78b3fff640 1 -- 172.21.15.49:0/619497200 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f78a0004a90 msgr2=0x7f78b40acf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:03.301 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.293+0000 7f78b3fff640 1 --2- 172.21.15.49:0/619497200 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f78a0004a90 0x7f78b40acf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:03.301 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.293+0000 7f78b3fff640 1 -- 172.21.15.49:0/619497200 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78b40c56d0 con 0x7f78b415b530 2023-11-06T02:15:03.301 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.293+0000 7f78abfff640 1 --2- 172.21.15.49:0/619497200 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f78b40af810 0x7f78b415aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:15:03.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.293+0000 7f78b37fe640 1 --2- 172.21.15.49:0/619497200 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f78a0004a90 0x7f78b40acf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:15:03.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.293+0000 7f78b3fff640 1 --2- 172.21.15.49:0/619497200 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78b415b530 0x7f78b40ab8c0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f78b415dc20 tx=0x7f78ac00f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:15:03.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.293+0000 7f78b0ff9640 1 -- 172.21.15.49:0/619497200 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78ac016420 con 0x7f78b415b530 2023-11-06T02:15:03.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.293+0000 7f78b0ff9640 1 -- 172.21.15.49:0/619497200 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78ac016580 con 0x7f78b415b530 2023-11-06T02:15:03.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.293+0000 7f78b0ff9640 1 -- 172.21.15.49:0/619497200 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78ac0168f0 con 0x7f78b415b530 2023-11-06T02:15:03.303 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.293+0000 7f78b9ba6640 1 -- 172.21.15.49:0/619497200 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78b40c59b0 con 0x7f78b415b530 2023-11-06T02:15:03.303 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.293+0000 7f78b9ba6640 1 -- 172.21.15.49:0/619497200 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78b40c5f70 con 0x7f78b415b530 2023-11-06T02:15:03.303 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.293+0000 7f78b9ba6640 1 -- 172.21.15.49:0/619497200 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78b4074040 con 0x7f78b415b530 2023-11-06T02:15:03.303 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.297+0000 7f78b0ff9640 1 -- 172.21.15.49:0/619497200 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f78ac016a50 con 0x7f78b415b530 2023-11-06T02:15:03.304 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.297+0000 7f78b0ff9640 1 --2- 172.21.15.49:0/619497200 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7894065a60 0x7f7894067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:03.304 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.297+0000 7f78b0ff9640 1 -- 172.21.15.49:0/619497200 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f78ac083e60 con 0x7f78b415b530 2023-11-06T02:15:03.304 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.297+0000 7f78b37fe640 1 --2- 172.21.15.49:0/619497200 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7894065a60 0x7f7894067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:03.304 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.297+0000 7f78b37fe640 1 --2- 172.21.15.49:0/619497200 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7894065a60 0x7f7894067f20 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f78a40ac000 tx=0x7f78a40011f0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:15:03.305 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.301+0000 7f78b0ff9640 1 -- 172.21.15.49:0/619497200 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f78ac0519a0 con 0x7f78b415b530 2023-11-06T02:15:03.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.453+0000 7f78b9ba6640 1 -- 172.21.15.49:0/619497200 --> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f78b40c9060 con 0x7f7894065a60 2023-11-06T02:15:03.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.453+0000 7f78b0ff9640 1 -- 172.21.15.49:0/619497200 <== mgr.4100 v2:172.21.15.150:6832/38104 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f78a4005430 con 0x7f7894065a60 2023-11-06T02:15:03.450 INFO:teuthology.orchestra.run.smithi049.stderr:instructed osd(s) 5 to deep-scrub 2023-11-06T02:15:03.453 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.457+0000 7f78b9ba6640 1 -- 172.21.15.49:0/619497200 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7894065a60 msgr2=0x7f7894067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:03.453 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.457+0000 7f78b9ba6640 1 --2- 172.21.15.49:0/619497200 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7894065a60 0x7f7894067f20 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f78a40ac000 tx=0x7f78a40011f0 comp rx=0 tx=0).stop 2023-11-06T02:15:03.453 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.457+0000 7f78b9ba6640 1 -- 172.21.15.49:0/619497200 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78b415b530 msgr2=0x7f78b40ab8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:03.453 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.457+0000 7f78b9ba6640 1 --2- 172.21.15.49:0/619497200 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78b415b530 0x7f78b40ab8c0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f78b415dc20 tx=0x7f78ac00f520 comp rx=0 tx=0).stop 2023-11-06T02:15:03.453 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.457+0000 7f78b9ba6640 1 -- 172.21.15.49:0/619497200 shutdown_connections 2023-11-06T02:15:03.454 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.457+0000 7f78b9ba6640 1 --2- 172.21.15.49:0/619497200 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f7894065a60 0x7f7894067f20 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:03.454 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.457+0000 7f78b9ba6640 1 --2- 172.21.15.49:0/619497200 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f78b415b530 0x7f78b40ab8c0 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:03.454 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.457+0000 7f78b9ba6640 1 --2- 172.21.15.49:0/619497200 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f78b40af810 0x7f78b415aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:03.454 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.457+0000 7f78b9ba6640 1 --2- 172.21.15.49:0/619497200 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f78a0004a90 0x7f78b40acf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:03.454 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.457+0000 7f78b9ba6640 1 -- 172.21.15.49:0/619497200 >> 172.21.15.49:0/619497200 conn(0x7f78b406c0a0 msgr2=0x7f78b40cc1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:15:03.455 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.457+0000 7f78b9ba6640 1 -- 172.21.15.49:0/619497200 shutdown_connections 2023-11-06T02:15:03.455 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.457+0000 7f78b9ba6640 1 -- 172.21.15.49:0/619497200 wait complete. 2023-11-06T02:15:03.468 INFO:tasks.ceph:Scrubbing osd.6 2023-11-06T02:15:03.469 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-11-06T02:15:03.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.601+0000 7f0a0fef7640 1 Processor -- start 2023-11-06T02:15:03.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.601+0000 7f0a0fef7640 1 -- start start 2023-11-06T02:15:03.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.605+0000 7f0a0fef7640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f0a0815e780 0x7f0a080ca050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:03.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.605+0000 7f0a0fef7640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a080c94b0 con 0x7f0a0806a480 2023-11-06T02:15:03.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.605+0000 7f0a0fef7640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a0815ecd0 con 0x7f0a0815abd0 2023-11-06T02:15:03.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.605+0000 7f0a0fef7640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a0815efb0 con 0x7f0a0815e780 2023-11-06T02:15:03.600 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.605+0000 7f0a0d492640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f0a0806a480 0x7f0a080c9940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:35168/0 (socket says 172.21.15.49:35168) 2023-11-06T02:15:03.600 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.605+0000 7f0a0d492640 1 -- 172.21.15.49:0/1967703343 learned_addr learned my addr 172.21.15.49:0/1967703343 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:15:03.600 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.605+0000 7f0a0e494640 1 --2- 172.21.15.49:0/1967703343 >> v2:172.21.15.150:3300/0 conn(0x7f0a0815e780 0x7f0a080ca050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:03.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.605+0000 7f0a0cc91640 1 -- 172.21.15.49:0/1967703343 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 303458051 0 0) 0x7f0a080c94b0 con 0x7f0a0806a480 2023-11-06T02:15:03.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.605+0000 7f0a0cc91640 1 -- 172.21.15.49:0/1967703343 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09f4003530 con 0x7f0a0806a480 2023-11-06T02:15:03.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.605+0000 7f0a0cc91640 1 -- 172.21.15.49:0/1967703343 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1334249130 0 0) 0x7f09f4003530 con 0x7f0a0806a480 2023-11-06T02:15:03.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.605+0000 7f0a0cc91640 1 -- 172.21.15.49:0/1967703343 >> v2:172.21.15.150:3300/0 conn(0x7f0a0815e780 msgr2=0x7f0a080ca050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:15:03.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.605+0000 7f0a0cc91640 1 --2- 172.21.15.49:0/1967703343 >> v2:172.21.15.150:3300/0 conn(0x7f0a0815e780 0x7f0a080ca050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:03.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.605+0000 7f0a0cc91640 1 -- 172.21.15.49:0/1967703343 >> v1:172.21.15.150:6789/0 conn(0x7f0a0815abd0 legacy=0x7f0a08069d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:03.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.605+0000 7f0a0cc91640 1 -- 172.21.15.49:0/1967703343 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a0815f290 con 0x7f0a0806a480 2023-11-06T02:15:03.604 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.605+0000 7f0a0cc91640 1 -- 172.21.15.49:0/1967703343 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7f09f8001680 con 0x7f0a0806a480 2023-11-06T02:15:03.604 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.605+0000 7f0a0cc91640 1 -- 172.21.15.49:0/1967703343 >> v1:172.21.15.49:6789/0 conn(0x7f0a0806a480 legacy=0x7f0a080c9940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:03.604 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.605+0000 7f0a0cc91640 1 --2- 172.21.15.49:0/1967703343 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f09f4003980 0x7f09f4003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:03.604 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.605+0000 7f0a0cc91640 1 --2- 172.21.15.49:0/1967703343 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09f40047d0 0x7f09f4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:03.604 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.605+0000 7f0a0cc91640 1 --2- 172.21.15.49:0/1967703343 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09f4005190 0x7f09f400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:03.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.605+0000 7f0a0cc91640 1 -- 172.21.15.49:0/1967703343 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0815efb0 con 0x7f09f40047d0 2023-11-06T02:15:03.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.605+0000 7f0a0cc91640 1 -- 172.21.15.49:0/1967703343 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09f400c130 con 0x7f09f4005190 2023-11-06T02:15:03.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.605+0000 7f0a0cc91640 1 -- 172.21.15.49:0/1967703343 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09f400c2f0 con 0x7f09f4003980 2023-11-06T02:15:03.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.605+0000 7f0a0d492640 1 --2- 172.21.15.49:0/1967703343 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09f40047d0 0x7f09f4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:03.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.605+0000 7f0a0e494640 1 --2- 172.21.15.49:0/1967703343 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09f4005190 0x7f09f400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:03.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.605+0000 7f0a0dc93640 1 --2- 172.21.15.49:0/1967703343 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f09f4003980 0x7f09f4003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:03.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.605+0000 7f0a0d492640 1 -- 172.21.15.49:0/1967703343 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f09f4003980 msgr2=0x7f09f4003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:15:03.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.605+0000 7f0a0d492640 1 --2- 172.21.15.49:0/1967703343 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f09f4003980 0x7f09f4003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:03.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.605+0000 7f0a0d492640 1 -- 172.21.15.49:0/1967703343 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09f4005190 msgr2=0x7f09f400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:15:03.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.605+0000 7f0a0d492640 1 --2- 172.21.15.49:0/1967703343 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09f4005190 0x7f09f400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:03.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.605+0000 7f0a0d492640 1 -- 172.21.15.49:0/1967703343 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09f4003530 con 0x7f09f40047d0 2023-11-06T02:15:03.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.605+0000 7f0a0e494640 1 -- 172.21.15.49:0/1967703343 reap_dead start 2023-11-06T02:15:03.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.609+0000 7f0a0d492640 1 --2- 172.21.15.49:0/1967703343 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09f40047d0 0x7f09f4004c50 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f09f800c240 tx=0x7f09f801cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae7c370c0015cdb server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:15:03.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.609+0000 7f0a0cc91640 1 -- 172.21.15.49:0/1967703343 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f8005ae0 con 0x7f09f40047d0 2023-11-06T02:15:03.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.609+0000 7f0a0cc91640 1 -- 172.21.15.49:0/1967703343 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09f8005c40 con 0x7f09f40047d0 2023-11-06T02:15:03.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.609+0000 7f0a0cc91640 1 -- 172.21.15.49:0/1967703343 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f80a57c0 con 0x7f09f40047d0 2023-11-06T02:15:03.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.609+0000 7f0a0fef7640 1 -- 172.21.15.49:0/1967703343 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09f40047d0 msgr2=0x7f09f4004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:03.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.609+0000 7f0a0fef7640 1 --2- 172.21.15.49:0/1967703343 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09f40047d0 0x7f09f4004c50 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f09f800c240 tx=0x7f09f801cbd0 comp rx=0 tx=0).stop 2023-11-06T02:15:03.612 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.609+0000 7f0a0fef7640 1 -- 172.21.15.49:0/1967703343 shutdown_connections 2023-11-06T02:15:03.612 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.609+0000 7f0a0fef7640 1 --2- 172.21.15.49:0/1967703343 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09f40047d0 0x7f09f4004c50 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:03.612 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.609+0000 7f0a0fef7640 1 -- 172.21.15.49:0/1967703343 >> 172.21.15.49:0/1967703343 conn(0x7f0a080ce610 msgr2=0x7f0a080cea20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:15:03.612 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.609+0000 7f0a0fef7640 1 -- 172.21.15.49:0/1967703343 shutdown_connections 2023-11-06T02:15:03.612 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.609+0000 7f0a0fef7640 1 -- 172.21.15.49:0/1967703343 wait complete. 2023-11-06T02:15:03.613 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.609+0000 7f0a0fef7640 1 Processor -- start 2023-11-06T02:15:03.613 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.609+0000 7f0a0fef7640 1 -- start start 2023-11-06T02:15:03.613 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.613+0000 7f0a0fef7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09f40047d0 0x7f0a080c6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:03.613 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.613+0000 7f0a0fef7640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0a0815c840 0x7f0a0814eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:03.614 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.613+0000 7f0a0fef7640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a080cb330 0x7f0a080cb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:03.614 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.613+0000 7f0a0fef7640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a080cbd20 con 0x7f09f40047d0 2023-11-06T02:15:03.614 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.613+0000 7f0a0fef7640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a081512d0 con 0x7f0a080cb330 2023-11-06T02:15:03.614 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.613+0000 7f0a0fef7640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0814f0c0 con 0x7f0a0815c840 2023-11-06T02:15:03.615 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.613+0000 7f0a0dc93640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09f40047d0 0x7f0a080c6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:03.615 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.613+0000 7f0a0dc93640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09f40047d0 0x7f0a080c6060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56540/0 (socket says 172.21.15.49:56540) 2023-11-06T02:15:03.615 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.613+0000 7f0a0dc93640 1 -- 172.21.15.49:0/1966463150 learned_addr learned my addr 172.21.15.49:0/1966463150 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:15:03.615 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.613+0000 7f0a0d492640 1 --2- 172.21.15.49:0/1966463150 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0a0815c840 0x7f0a0814eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:03.616 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.613+0000 7f0a0dc93640 1 -- 172.21.15.49:0/1966463150 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0a0815c840 msgr2=0x7f0a0814eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:03.616 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.613+0000 7f0a0dc93640 1 --2- 172.21.15.49:0/1966463150 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0a0815c840 0x7f0a0814eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:03.616 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.613+0000 7f0a0dc93640 1 -- 172.21.15.49:0/1966463150 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a080cb330 msgr2=0x7f0a080cb7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:03.616 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.613+0000 7f0a0e494640 1 --2- 172.21.15.49:0/1966463150 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a080cb330 0x7f0a080cb7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:03.616 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.613+0000 7f0a0dc93640 1 --2- 172.21.15.49:0/1966463150 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a080cb330 0x7f0a080cb7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:03.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.613+0000 7f0a0dc93640 1 -- 172.21.15.49:0/1966463150 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a080c7a10 con 0x7f09f40047d0 2023-11-06T02:15:03.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.613+0000 7f0a0dc93640 1 --2- 172.21.15.49:0/1966463150 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09f40047d0 0x7f0a080c6060 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f0a04001ec0 tx=0x7f0a0400e8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:15:03.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.613+0000 7f09fe7fc640 1 -- 172.21.15.49:0/1966463150 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a040052e0 con 0x7f09f40047d0 2023-11-06T02:15:03.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.613+0000 7f09fe7fc640 1 -- 172.21.15.49:0/1966463150 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a04005440 con 0x7f09f40047d0 2023-11-06T02:15:03.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.613+0000 7f09fe7fc640 1 -- 172.21.15.49:0/1966463150 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a04003820 con 0x7f09f40047d0 2023-11-06T02:15:03.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.613+0000 7f0a0fef7640 1 -- 172.21.15.49:0/1966463150 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a0815cd40 con 0x7f09f40047d0 2023-11-06T02:15:03.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.613+0000 7f0a0fef7640 1 -- 172.21.15.49:0/1966463150 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a0815d300 con 0x7f09f40047d0 2023-11-06T02:15:03.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.613+0000 7f0a0fef7640 1 -- 172.21.15.49:0/1966463150 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0a080add50 con 0x7f09f40047d0 2023-11-06T02:15:03.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.613+0000 7f09fe7fc640 1 -- 172.21.15.49:0/1966463150 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f0a04003980 con 0x7f09f40047d0 2023-11-06T02:15:03.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.613+0000 7f09fe7fc640 1 --2- 172.21.15.49:0/1966463150 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f09ec065a60 0x7f09ec067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:03.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.613+0000 7f09fe7fc640 1 -- 172.21.15.49:0/1966463150 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f0a04083490 con 0x7f09f40047d0 2023-11-06T02:15:03.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.613+0000 7f09fe7fc640 1 --2- 172.21.15.49:0/1966463150 >> [v2:172.21.15.150:6824/38428,v1:172.21.15.150:6825/38428] conn(0x7f09ec06c020 0x7f09ec06e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:03.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.613+0000 7f09fe7fc640 1 -- 172.21.15.49:0/1966463150 --> [v2:172.21.15.150:6824/38428,v1:172.21.15.150:6825/38428] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f09ec06eaf0 con 0x7f09ec06c020 2023-11-06T02:15:03.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.613+0000 7f09fe7fc640 1 -- 172.21.15.49:0/1966463150 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a04083800 con 0x7f09f40047d0 2023-11-06T02:15:03.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.613+0000 7f0a0d492640 1 --2- 172.21.15.49:0/1966463150 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f09ec065a60 0x7f09ec067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:03.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.613+0000 7f0a0e494640 1 --2- 172.21.15.49:0/1966463150 >> [v2:172.21.15.150:6824/38428,v1:172.21.15.150:6825/38428] conn(0x7f09ec06c020 0x7f09ec06e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:03.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.617+0000 7f0a0d492640 1 --2- 172.21.15.49:0/1966463150 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f09ec065a60 0x7f09ec067f20 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f09f80ab000 tx=0x7f09f80a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:15:03.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.617+0000 7f0a0e494640 1 --2- 172.21.15.49:0/1966463150 >> [v2:172.21.15.150:6824/38428,v1:172.21.15.150:6825/38428] conn(0x7f09ec06c020 0x7f09ec06e440 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:15:03.621 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.617+0000 7f09fe7fc640 1 -- 172.21.15.49:0/1966463150 <== osd.6 v2:172.21.15.150:6824/38428 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f09ec06eaf0 con 0x7f09ec06c020 2023-11-06T02:15:03.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.629+0000 7f0a0fef7640 1 -- 172.21.15.49:0/1966463150 --> [v2:172.21.15.150:6824/38428,v1:172.21.15.150:6825/38428] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f0a08074040 con 0x7f09ec06c020 2023-11-06T02:15:03.626 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.629+0000 7f09fe7fc640 1 -- 172.21.15.49:0/1966463150 <== osd.6 v2:172.21.15.150:6824/38428 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f0a08074040 con 0x7f09ec06c020 2023-11-06T02:15:03.627 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-11-06T02:15:03.627 INFO:teuthology.orchestra.run.smithi049.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-06T02:15:03.627 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-11-06T02:15:03.628 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.633+0000 7f0a0fef7640 1 -- 172.21.15.49:0/1966463150 >> [v2:172.21.15.150:6824/38428,v1:172.21.15.150:6825/38428] conn(0x7f09ec06c020 msgr2=0x7f09ec06e440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:03.628 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.633+0000 7f0a0fef7640 1 --2- 172.21.15.49:0/1966463150 >> [v2:172.21.15.150:6824/38428,v1:172.21.15.150:6825/38428] conn(0x7f09ec06c020 0x7f09ec06e440 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:03.628 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.633+0000 7f0a0fef7640 1 -- 172.21.15.49:0/1966463150 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f09ec065a60 msgr2=0x7f09ec067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:03.629 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.633+0000 7f0a0fef7640 1 --2- 172.21.15.49:0/1966463150 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f09ec065a60 0x7f09ec067f20 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f09f80ab000 tx=0x7f09f80a7070 comp rx=0 tx=0).stop 2023-11-06T02:15:03.629 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.633+0000 7f0a0fef7640 1 -- 172.21.15.49:0/1966463150 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09f40047d0 msgr2=0x7f0a080c6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:03.629 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.633+0000 7f0a0fef7640 1 --2- 172.21.15.49:0/1966463150 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f09f40047d0 0x7f0a080c6060 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f0a04001ec0 tx=0x7f0a0400e8a0 comp rx=0 tx=0).stop 2023-11-06T02:15:03.629 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.633+0000 7f0a0e494640 1 -- 172.21.15.49:0/1966463150 reap_dead start 2023-11-06T02:15:03.629 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.633+0000 7f0a0fef7640 1 -- 172.21.15.49:0/1966463150 shutdown_connections 2023-11-06T02:15:03.630 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.633+0000 7f0a0fef7640 1 -- 172.21.15.49:0/1966463150 >> 172.21.15.49:0/1966463150 conn(0x7f0a080ce610 msgr2=0x7f0a080b2c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:15:03.630 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.633+0000 7f0a0fef7640 1 -- 172.21.15.49:0/1966463150 shutdown_connections 2023-11-06T02:15:03.630 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.633+0000 7f0a0fef7640 1 -- 172.21.15.49:0/1966463150 wait complete. 2023-11-06T02:15:03.644 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-11-06T02:15:03.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.765+0000 7fac3bfb3640 1 Processor -- start 2023-11-06T02:15:03.762 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.765+0000 7fac3bfb3640 1 -- start start 2023-11-06T02:15:03.764 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.765+0000 7fac3bfb3640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fac340cda70 0x7fac34161650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:03.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.765+0000 7fac3bfb3640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fac340714a0 0x7fac340718f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:03.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.765+0000 7fac3bfb3640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac34071e30 con 0x7fac340b1f90 2023-11-06T02:15:03.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.765+0000 7fac3bfb3640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac34161b90 con 0x7fac340714a0 2023-11-06T02:15:03.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.765+0000 7fac3bfb3640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac340cdef0 con 0x7fac340cda70 2023-11-06T02:15:03.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.765+0000 7fac39d4f640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fac340b1f90 0x7fac340d1c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:35182/0 (socket says 172.21.15.49:35182) 2023-11-06T02:15:03.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.765+0000 7fac39d4f640 1 -- 172.21.15.49:0/2509186619 learned_addr learned my addr 172.21.15.49:0/2509186619 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:15:03.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.769+0000 7fac38d4d640 1 -- 172.21.15.49:0/2509186619 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3398424833 0 0) 0x7fac30001120 con 0x7fac340b1f90 2023-11-06T02:15:03.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.769+0000 7fac38d4d640 1 -- 172.21.15.49:0/2509186619 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac20003530 con 0x7fac340b1f90 2023-11-06T02:15:03.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.769+0000 7fac3a550640 1 --2- 172.21.15.49:0/2509186619 >> v2:172.21.15.150:3300/0 conn(0x7fac340714a0 0x7fac340718f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:03.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.769+0000 7fac3954e640 1 --2- 172.21.15.49:0/2509186619 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fac340cda70 0x7fac34161650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:03.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.769+0000 7fac38d4d640 1 -- 172.21.15.49:0/2509186619 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3476300481 0 0) 0x7fac20003530 con 0x7fac340b1f90 2023-11-06T02:15:03.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.769+0000 7fac38d4d640 1 -- 172.21.15.49:0/2509186619 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fac340cda70 msgr2=0x7fac34161650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:15:03.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.769+0000 7fac38d4d640 1 --2- 172.21.15.49:0/2509186619 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fac340cda70 0x7fac34161650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:03.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.769+0000 7fac38d4d640 1 -- 172.21.15.49:0/2509186619 >> v2:172.21.15.150:3300/0 conn(0x7fac340714a0 msgr2=0x7fac340718f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:15:03.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.769+0000 7fac38d4d640 1 --2- 172.21.15.49:0/2509186619 >> v2:172.21.15.150:3300/0 conn(0x7fac340714a0 0x7fac340718f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:03.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.769+0000 7fac38d4d640 1 -- 172.21.15.49:0/2509186619 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac340d17c0 con 0x7fac340b1f90 2023-11-06T02:15:03.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.769+0000 7fac38d4d640 1 -- 172.21.15.49:0/2509186619 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7fac30001810 con 0x7fac340b1f90 2023-11-06T02:15:03.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.769+0000 7fac3954e640 1 --2- 172.21.15.49:0/2509186619 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fac340cda70 0x7fac34161650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T02:15:03.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.769+0000 7fac38d4d640 1 -- 172.21.15.49:0/2509186619 >> v1:172.21.15.49:6789/0 conn(0x7fac340b1f90 legacy=0x7fac340d1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:03.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.769+0000 7fac38d4d640 1 --2- 172.21.15.49:0/2509186619 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac20003980 0x7fac20003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:03.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.769+0000 7fac38d4d640 1 --2- 172.21.15.49:0/2509186619 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fac340cda70 0x7fac200047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:03.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.769+0000 7fac38d4d640 1 --2- 172.21.15.49:0/2509186619 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fac20004d10 0x7fac20005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:03.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.769+0000 7fac38d4d640 1 -- 172.21.15.49:0/2509186619 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac340cdef0 con 0x7fac20004d10 2023-11-06T02:15:03.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.769+0000 7fac38d4d640 1 -- 172.21.15.49:0/2509186619 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac2000b990 con 0x7fac20003980 2023-11-06T02:15:03.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.769+0000 7fac38d4d640 1 -- 172.21.15.49:0/2509186619 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac2000bb00 con 0x7fac340cda70 2023-11-06T02:15:03.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.769+0000 7fac3a550640 1 --2- 172.21.15.49:0/2509186619 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fac20004d10 0x7fac20005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:03.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.769+0000 7fac39d4f640 1 --2- 172.21.15.49:0/2509186619 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac20003980 0x7fac20003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:03.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.769+0000 7fac3954e640 1 --2- 172.21.15.49:0/2509186619 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fac340cda70 0x7fac200047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:03.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.769+0000 7fac3a550640 1 -- 172.21.15.49:0/2509186619 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fac340cda70 msgr2=0x7fac200047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:15:03.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.769+0000 7fac3a550640 1 --2- 172.21.15.49:0/2509186619 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fac340cda70 0x7fac200047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:03.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.769+0000 7fac3a550640 1 -- 172.21.15.49:0/2509186619 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac20003980 msgr2=0x7fac20003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:15:03.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.769+0000 7fac3a550640 1 --2- 172.21.15.49:0/2509186619 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac20003980 0x7fac20003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:03.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.769+0000 7fac3a550640 1 -- 172.21.15.49:0/2509186619 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac20003530 con 0x7fac20004d10 2023-11-06T02:15:03.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.769+0000 7fac3a550640 1 --2- 172.21.15.49:0/2509186619 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fac20004d10 0x7fac20005190 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7fac2c009fc0 tx=0x7fac2c01cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6076874c89b8d43 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:15:03.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.769+0000 7fac38d4d640 1 -- 172.21.15.49:0/2509186619 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac2c0a5cb0 con 0x7fac20004d10 2023-11-06T02:15:03.776 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.769+0000 7fac38d4d640 1 -- 172.21.15.49:0/2509186619 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac2c00f040 con 0x7fac20004d10 2023-11-06T02:15:03.776 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.769+0000 7fac38d4d640 1 -- 172.21.15.49:0/2509186619 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac2c0ad680 con 0x7fac20004d10 2023-11-06T02:15:03.776 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.773+0000 7fac3bfb3640 1 -- 172.21.15.49:0/2509186619 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fac20004d10 msgr2=0x7fac20005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:03.776 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.773+0000 7fac3bfb3640 1 --2- 172.21.15.49:0/2509186619 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fac20004d10 0x7fac20005190 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7fac2c009fc0 tx=0x7fac2c01cca0 comp rx=0 tx=0).stop 2023-11-06T02:15:03.776 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.773+0000 7fac3a550640 1 -- 172.21.15.49:0/2509186619 reap_dead start 2023-11-06T02:15:03.777 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.773+0000 7fac3bfb3640 1 -- 172.21.15.49:0/2509186619 shutdown_connections 2023-11-06T02:15:03.777 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.773+0000 7fac3bfb3640 1 -- 172.21.15.49:0/2509186619 >> 172.21.15.49:0/2509186619 conn(0x7fac340d2aa0 msgr2=0x7fac340d2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:15:03.777 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.773+0000 7fac3bfb3640 1 -- 172.21.15.49:0/2509186619 shutdown_connections 2023-11-06T02:15:03.777 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.773+0000 7fac3bfb3640 1 -- 172.21.15.49:0/2509186619 wait complete. 2023-11-06T02:15:03.777 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.773+0000 7fac3bfb3640 1 Processor -- start 2023-11-06T02:15:03.778 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.773+0000 7fac3bfb3640 1 -- start start 2023-11-06T02:15:03.778 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.773+0000 7fac3bfb3640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac20004d10 0x7fac3415d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:03.778 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.773+0000 7fac3bfb3640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fac3415a9f0 0x7fac3415ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:03.778 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.773+0000 7fac3bfb3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fac3415b3b0 0x7fac340b3180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:03.778 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.773+0000 7fac3bfb3640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac34157ba0 con 0x7fac3415b3b0 2023-11-06T02:15:03.779 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.773+0000 7fac3bfb3640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac3415b830 con 0x7fac20004d10 2023-11-06T02:15:03.779 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.773+0000 7fac3bfb3640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac3415dd60 con 0x7fac3415a9f0 2023-11-06T02:15:03.779 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.773+0000 7fac3a550640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fac3415b3b0 0x7fac340b3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:03.779 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.773+0000 7fac3a550640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fac3415b3b0 0x7fac340b3180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56564/0 (socket says 172.21.15.49:56564) 2023-11-06T02:15:03.779 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.773+0000 7fac3a550640 1 -- 172.21.15.49:0/4208225647 learned_addr learned my addr 172.21.15.49:0/4208225647 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:15:03.780 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.773+0000 7fac39d4f640 1 --2- 172.21.15.49:0/4208225647 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac20004d10 0x7fac3415d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:03.780 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.773+0000 7fac3954e640 1 --2- 172.21.15.49:0/4208225647 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fac3415a9f0 0x7fac3415ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:03.780 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.773+0000 7fac3a550640 1 -- 172.21.15.49:0/4208225647 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fac3415a9f0 msgr2=0x7fac3415ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:03.780 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.773+0000 7fac3a550640 1 --2- 172.21.15.49:0/4208225647 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fac3415a9f0 0x7fac3415ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:03.780 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.773+0000 7fac3a550640 1 -- 172.21.15.49:0/4208225647 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac20004d10 msgr2=0x7fac3415d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:03.780 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.773+0000 7fac3a550640 1 --2- 172.21.15.49:0/4208225647 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac20004d10 0x7fac3415d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:03.781 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.773+0000 7fac3a550640 1 -- 172.21.15.49:0/4208225647 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac340b8740 con 0x7fac3415b3b0 2023-11-06T02:15:03.781 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.773+0000 7fac39d4f640 1 --2- 172.21.15.49:0/4208225647 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac20004d10 0x7fac3415d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:15:03.781 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.773+0000 7fac3a550640 1 --2- 172.21.15.49:0/4208225647 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fac3415b3b0 0x7fac340b3180 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7fac2c0b0040 tx=0x7fac2c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:15:03.781 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.773+0000 7fac2a7fc640 1 -- 172.21.15.49:0/4208225647 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac2c0039a0 con 0x7fac3415b3b0 2023-11-06T02:15:03.781 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.773+0000 7fac2a7fc640 1 -- 172.21.15.49:0/4208225647 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac2c003b20 con 0x7fac3415b3b0 2023-11-06T02:15:03.782 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.773+0000 7fac2a7fc640 1 -- 172.21.15.49:0/4208225647 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac2c0ae950 con 0x7fac3415b3b0 2023-11-06T02:15:03.782 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.773+0000 7fac3bfb3640 1 -- 172.21.15.49:0/4208225647 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac340b23c0 con 0x7fac3415b3b0 2023-11-06T02:15:03.782 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.773+0000 7fac3bfb3640 1 -- 172.21.15.49:0/4208225647 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac340b2980 con 0x7fac3415b3b0 2023-11-06T02:15:03.782 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.777+0000 7fac3bfb3640 1 -- 172.21.15.49:0/4208225647 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac34074040 con 0x7fac3415b3b0 2023-11-06T02:15:03.782 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.777+0000 7fac2a7fc640 1 -- 172.21.15.49:0/4208225647 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fac2c0b4040 con 0x7fac3415b3b0 2023-11-06T02:15:03.783 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.777+0000 7fac2a7fc640 1 --2- 172.21.15.49:0/4208225647 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fac18065a90 0x7fac18067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:03.783 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.777+0000 7fac2a7fc640 1 -- 172.21.15.49:0/4208225647 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fac2c11cff0 con 0x7fac3415b3b0 2023-11-06T02:15:03.783 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.777+0000 7fac39d4f640 1 --2- 172.21.15.49:0/4208225647 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fac18065a90 0x7fac18067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:03.783 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.777+0000 7fac39d4f640 1 --2- 172.21.15.49:0/4208225647 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fac18065a90 0x7fac18067f50 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fac30001060 tx=0x7fac30001120 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:15:03.784 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.781+0000 7fac2a7fc640 1 -- 172.21.15.49:0/4208225647 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fac2c0eaa50 con 0x7fac3415b3b0 2023-11-06T02:15:03.930 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.933+0000 7fac3bfb3640 1 -- 172.21.15.49:0/4208225647 --> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7fac340d2360 con 0x7fac18065a90 2023-11-06T02:15:03.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.937+0000 7fac2a7fc640 1 -- 172.21.15.49:0/4208225647 <== mgr.4100 v2:172.21.15.150:6832/38104 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fac3000f7c0 con 0x7fac18065a90 2023-11-06T02:15:03.931 INFO:teuthology.orchestra.run.smithi049.stderr:instructed osd(s) 6 to deep-scrub 2023-11-06T02:15:03.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.937+0000 7fac3bfb3640 1 -- 172.21.15.49:0/4208225647 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fac18065a90 msgr2=0x7fac18067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:03.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.937+0000 7fac3bfb3640 1 --2- 172.21.15.49:0/4208225647 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fac18065a90 0x7fac18067f50 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fac30001060 tx=0x7fac30001120 comp rx=0 tx=0).stop 2023-11-06T02:15:03.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.937+0000 7fac3bfb3640 1 -- 172.21.15.49:0/4208225647 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fac3415b3b0 msgr2=0x7fac340b3180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:03.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.937+0000 7fac3bfb3640 1 --2- 172.21.15.49:0/4208225647 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fac3415b3b0 0x7fac340b3180 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7fac2c0b0040 tx=0x7fac2c009750 comp rx=0 tx=0).stop 2023-11-06T02:15:03.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.937+0000 7fac3bfb3640 1 -- 172.21.15.49:0/4208225647 shutdown_connections 2023-11-06T02:15:03.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.937+0000 7fac3bfb3640 1 --2- 172.21.15.49:0/4208225647 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7fac18065a90 0x7fac18067f50 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:03.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.937+0000 7fac3bfb3640 1 --2- 172.21.15.49:0/4208225647 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fac3415b3b0 0x7fac340b3180 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:03.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.937+0000 7fac3bfb3640 1 --2- 172.21.15.49:0/4208225647 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fac3415a9f0 0x7fac3415ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:03.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.937+0000 7fac3bfb3640 1 --2- 172.21.15.49:0/4208225647 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac20004d10 0x7fac3415d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:03.935 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.937+0000 7fac3bfb3640 1 -- 172.21.15.49:0/4208225647 >> 172.21.15.49:0/4208225647 conn(0x7fac340d2aa0 msgr2=0x7fac340c8aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:15:03.935 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.937+0000 7fac3bfb3640 1 -- 172.21.15.49:0/4208225647 shutdown_connections 2023-11-06T02:15:03.935 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:03.937+0000 7fac3bfb3640 1 -- 172.21.15.49:0/4208225647 wait complete. 2023-11-06T02:15:03.949 INFO:tasks.ceph:Scrubbing osd.7 2023-11-06T02:15:03.949 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-11-06T02:15:04.067 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.069+0000 7f5744fd0640 1 Processor -- start 2023-11-06T02:15:04.067 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.069+0000 7f5744fd0640 1 -- start start 2023-11-06T02:15:04.069 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.073+0000 7f5744fd0640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f5740069fd0 0x7f57400caa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:04.069 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.073+0000 7f5744fd0640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f574006a3b0 0x7f57400715c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:04.069 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.073+0000 7f5744fd0640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f57400ad720 0x7f5740159530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:04.069 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.073+0000 7f5744fd0640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5740071e80 con 0x7f5740069fd0 2023-11-06T02:15:04.069 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.073+0000 7f5744fd0640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5740072160 con 0x7f57400ad720 2023-11-06T02:15:04.070 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.073+0000 7f5744fd0640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57400ca220 con 0x7f574006a3b0 2023-11-06T02:15:04.070 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.073+0000 7f573e59c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f5740069fd0 0x7f57400caa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:04.070 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.073+0000 7f573e59c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f5740069fd0 0x7f57400caa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56578/0 (socket says 172.21.15.49:56578) 2023-11-06T02:15:04.070 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.073+0000 7f573e59c640 1 -- 172.21.15.49:0/501860667 learned_addr learned my addr 172.21.15.49:0/501860667 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:15:04.071 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.073+0000 7f573ed9d640 1 --2- 172.21.15.49:0/501860667 >> v2:172.21.15.150:3300/0 conn(0x7f57400ad720 0x7f5740159530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:04.071 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.073+0000 7f573dd9b640 1 --2- 172.21.15.49:0/501860667 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f574006a3b0 0x7f57400715c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:04.071 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.073+0000 7f573e59c640 1 -- 172.21.15.49:0/501860667 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f574006a3b0 msgr2=0x7f57400715c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:15:04.071 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.073+0000 7f573e59c640 1 --2- 172.21.15.49:0/501860667 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f574006a3b0 0x7f57400715c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:04.072 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.073+0000 7f573e59c640 1 -- 172.21.15.49:0/501860667 >> v2:172.21.15.150:3300/0 conn(0x7f57400ad720 msgr2=0x7f5740159530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:15:04.072 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.073+0000 7f573e59c640 1 --2- 172.21.15.49:0/501860667 >> v2:172.21.15.150:3300/0 conn(0x7f57400ad720 0x7f5740159530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:04.073 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.073+0000 7f573e59c640 1 -- 172.21.15.49:0/501860667 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57400ca500 con 0x7f5740069fd0 2023-11-06T02:15:04.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.077+0000 7f573e59c640 1 --2- 172.21.15.49:0/501860667 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5740069fd0 0x7f57400caa20 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f57400b4370 tx=0x7f573001ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ce9699e0c856519 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:15:04.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.077+0000 7f573d59a640 1 -- 172.21.15.49:0/501860667 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57300a7b70 con 0x7f5740069fd0 2023-11-06T02:15:04.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.077+0000 7f573d59a640 1 -- 172.21.15.49:0/501860667 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57300a7cd0 con 0x7f5740069fd0 2023-11-06T02:15:04.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.077+0000 7f573d59a640 1 -- 172.21.15.49:0/501860667 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57300af660 con 0x7f5740069fd0 2023-11-06T02:15:04.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.077+0000 7f5744fd0640 1 -- 172.21.15.49:0/501860667 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5740069fd0 msgr2=0x7f57400caa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:04.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.077+0000 7f5744fd0640 1 --2- 172.21.15.49:0/501860667 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5740069fd0 0x7f57400caa20 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f57400b4370 tx=0x7f573001ebd0 comp rx=0 tx=0).stop 2023-11-06T02:15:04.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.077+0000 7f5744fd0640 1 -- 172.21.15.49:0/501860667 shutdown_connections 2023-11-06T02:15:04.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.077+0000 7f5744fd0640 1 --2- 172.21.15.49:0/501860667 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f574006a3b0 0x7f57400715c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:04.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.077+0000 7f5744fd0640 1 --2- 172.21.15.49:0/501860667 >> v2:172.21.15.150:3300/0 conn(0x7f57400ad720 0x7f5740159530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:04.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.077+0000 7f5744fd0640 1 --2- 172.21.15.49:0/501860667 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5740069fd0 0x7f57400caa20 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:04.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.077+0000 7f5744fd0640 1 -- 172.21.15.49:0/501860667 >> 172.21.15.49:0/501860667 conn(0x7f57400ce610 msgr2=0x7f57400cea20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:15:04.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.077+0000 7f5744fd0640 1 -- 172.21.15.49:0/501860667 shutdown_connections 2023-11-06T02:15:04.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.077+0000 7f5744fd0640 1 -- 172.21.15.49:0/501860667 wait complete. 2023-11-06T02:15:04.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.077+0000 7f5744fd0640 1 Processor -- start 2023-11-06T02:15:04.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.077+0000 7f5744fd0640 1 -- start start 2023-11-06T02:15:04.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.077+0000 7f5744fd0640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5740069fd0 0x7f57401475e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:04.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.077+0000 7f5744fd0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f574006a3b0 0x7f574014a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:04.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.077+0000 7f5744fd0640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57400ad720 0x7f574014fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:04.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.077+0000 7f5744fd0640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5740156590 con 0x7f574006a3b0 2023-11-06T02:15:04.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.077+0000 7f5744fd0640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5740158920 con 0x7f57400ad720 2023-11-06T02:15:04.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.077+0000 7f5744fd0640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5740147b20 con 0x7f5740069fd0 2023-11-06T02:15:04.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.077+0000 7f573dd9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f574006a3b0 0x7f574014a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:04.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.077+0000 7f573dd9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f574006a3b0 0x7f574014a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56592/0 (socket says 172.21.15.49:56592) 2023-11-06T02:15:04.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.077+0000 7f573dd9b640 1 -- 172.21.15.49:0/3021670063 learned_addr learned my addr 172.21.15.49:0/3021670063 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:15:04.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.077+0000 7f573e59c640 1 --2- 172.21.15.49:0/3021670063 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5740069fd0 0x7f57401475e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:04.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.077+0000 7f573ed9d640 1 --2- 172.21.15.49:0/3021670063 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57400ad720 0x7f574014fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:04.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.077+0000 7f573dd9b640 1 -- 172.21.15.49:0/3021670063 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5740069fd0 msgr2=0x7f57401475e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:04.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.077+0000 7f573dd9b640 1 --2- 172.21.15.49:0/3021670063 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5740069fd0 0x7f57401475e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:04.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.077+0000 7f573dd9b640 1 -- 172.21.15.49:0/3021670063 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57400ad720 msgr2=0x7f574014fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:04.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.077+0000 7f573dd9b640 1 --2- 172.21.15.49:0/3021670063 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57400ad720 0x7f574014fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:04.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.077+0000 7f573dd9b640 1 -- 172.21.15.49:0/3021670063 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f573000b770 con 0x7f574006a3b0 2023-11-06T02:15:04.085 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.077+0000 7f573e59c640 1 --2- 172.21.15.49:0/3021670063 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5740069fd0 0x7f57401475e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:15:04.085 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.077+0000 7f573dd9b640 1 --2- 172.21.15.49:0/3021670063 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f574006a3b0 0x7f574014a1d0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f572c00d830 tx=0x7f572c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:15:04.085 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.077+0000 7f5736ffd640 1 -- 172.21.15.49:0/3021670063 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f572c00b6c0 con 0x7f574006a3b0 2023-11-06T02:15:04.085 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.077+0000 7f5736ffd640 1 -- 172.21.15.49:0/3021670063 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f572c00b820 con 0x7f574006a3b0 2023-11-06T02:15:04.085 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.077+0000 7f5744fd0640 1 -- 172.21.15.49:0/3021670063 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5740155910 con 0x7f574006a3b0 2023-11-06T02:15:04.086 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.077+0000 7f5736ffd640 1 -- 172.21.15.49:0/3021670063 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f572c00bb90 con 0x7f574006a3b0 2023-11-06T02:15:04.086 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.077+0000 7f5744fd0640 1 -- 172.21.15.49:0/3021670063 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5740155ed0 con 0x7f574006a3b0 2023-11-06T02:15:04.086 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.081+0000 7f5744fd0640 1 -- 172.21.15.49:0/3021670063 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5740074040 con 0x7f574006a3b0 2023-11-06T02:15:04.086 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.081+0000 7f5736ffd640 1 -- 172.21.15.49:0/3021670063 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f572c01e460 con 0x7f574006a3b0 2023-11-06T02:15:04.086 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.081+0000 7f5736ffd640 1 --2- 172.21.15.49:0/3021670063 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5728065a60 0x7f5728067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:04.087 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.081+0000 7f5736ffd640 1 -- 172.21.15.49:0/3021670063 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f572c0842c0 con 0x7f574006a3b0 2023-11-06T02:15:04.087 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.081+0000 7f5736ffd640 1 --2- 172.21.15.49:0/3021670063 >> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] conn(0x7f572806c020 0x7f572806e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:04.087 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.081+0000 7f5736ffd640 1 -- 172.21.15.49:0/3021670063 --> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f572806eaf0 con 0x7f572806c020 2023-11-06T02:15:04.087 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.081+0000 7f5736ffd640 1 -- 172.21.15.49:0/3021670063 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f572c084630 con 0x7f574006a3b0 2023-11-06T02:15:04.087 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.081+0000 7f573e59c640 1 --2- 172.21.15.49:0/3021670063 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5728065a60 0x7f5728067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:04.088 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.081+0000 7f573ed9d640 1 --2- 172.21.15.49:0/3021670063 >> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] conn(0x7f572806c020 0x7f572806e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:04.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.081+0000 7f573ed9d640 1 --2- 172.21.15.49:0/3021670063 >> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] conn(0x7f572806c020 0x7f572806e440 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:15:04.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.081+0000 7f573e59c640 1 --2- 172.21.15.49:0/3021670063 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5728065a60 0x7f5728067f20 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f57300b1000 tx=0x7f5730008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:15:04.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.085+0000 7f5736ffd640 1 -- 172.21.15.49:0/3021670063 <== osd.7 v2:172.21.15.150:6816/38421 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f572806eaf0 con 0x7f572806c020 2023-11-06T02:15:04.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.097+0000 7f5744fd0640 1 -- 172.21.15.49:0/3021670063 --> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f57400cd640 con 0x7f572806c020 2023-11-06T02:15:04.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.097+0000 7f5736ffd640 1 -- 172.21.15.49:0/3021670063 <== osd.7 v2:172.21.15.150:6816/38421 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f57400cd640 con 0x7f572806c020 2023-11-06T02:15:04.093 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-11-06T02:15:04.093 INFO:teuthology.orchestra.run.smithi049.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-06T02:15:04.093 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-11-06T02:15:04.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.097+0000 7f5744fd0640 1 -- 172.21.15.49:0/3021670063 >> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] conn(0x7f572806c020 msgr2=0x7f572806e440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:04.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.097+0000 7f5744fd0640 1 --2- 172.21.15.49:0/3021670063 >> [v2:172.21.15.150:6816/38421,v1:172.21.15.150:6817/38421] conn(0x7f572806c020 0x7f572806e440 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:04.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.097+0000 7f5744fd0640 1 -- 172.21.15.49:0/3021670063 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5728065a60 msgr2=0x7f5728067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:04.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.097+0000 7f5744fd0640 1 --2- 172.21.15.49:0/3021670063 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f5728065a60 0x7f5728067f20 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f57300b1000 tx=0x7f5730008040 comp rx=0 tx=0).stop 2023-11-06T02:15:04.096 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.097+0000 7f5744fd0640 1 -- 172.21.15.49:0/3021670063 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f574006a3b0 msgr2=0x7f574014a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:04.096 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.097+0000 7f5744fd0640 1 --2- 172.21.15.49:0/3021670063 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f574006a3b0 0x7f574014a1d0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f572c00d830 tx=0x7f572c00dcf0 comp rx=0 tx=0).stop 2023-11-06T02:15:04.096 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.097+0000 7f573ed9d640 1 -- 172.21.15.49:0/3021670063 reap_dead start 2023-11-06T02:15:04.096 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.097+0000 7f5744fd0640 1 -- 172.21.15.49:0/3021670063 shutdown_connections 2023-11-06T02:15:04.096 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.097+0000 7f5744fd0640 1 -- 172.21.15.49:0/3021670063 >> 172.21.15.49:0/3021670063 conn(0x7f57400ce610 msgr2=0x7f57400cd860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:15:04.097 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.097+0000 7f5744fd0640 1 -- 172.21.15.49:0/3021670063 shutdown_connections 2023-11-06T02:15:04.097 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.097+0000 7f5744fd0640 1 -- 172.21.15.49:0/3021670063 wait complete. 2023-11-06T02:15:04.107 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-11-06T02:15:04.219 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.225+0000 7f9c95157640 1 Processor -- start 2023-11-06T02:15:04.220 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.225+0000 7f9c95157640 1 -- start start 2023-11-06T02:15:04.222 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.225+0000 7f9c95157640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9c9015a970 0x7f9c900cd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:04.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.225+0000 7f9c95157640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c900b0d70 con 0x7f9c900cde10 2023-11-06T02:15:04.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.225+0000 7f9c95157640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c900b01f0 con 0x7f9c900ced80 2023-11-06T02:15:04.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.225+0000 7f9c95157640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c900b0440 con 0x7f9c9015a970 2023-11-06T02:15:04.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.225+0000 7f9c8e59c640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f9c900cde10 0x7f9c900ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:35198/0 (socket says 172.21.15.49:35198) 2023-11-06T02:15:04.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.225+0000 7f9c8e59c640 1 -- 172.21.15.49:0/3877738026 learned_addr learned my addr 172.21.15.49:0/3877738026 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:15:04.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.225+0000 7f9c8dd9b640 1 -- 172.21.15.49:0/3877738026 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2298291918 0 0) 0x7f9c900b0d70 con 0x7f9c900cde10 2023-11-06T02:15:04.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.225+0000 7f9c8dd9b640 1 -- 172.21.15.49:0/3877738026 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c78003530 con 0x7f9c900cde10 2023-11-06T02:15:04.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.225+0000 7f9c8ed9d640 1 --2- 172.21.15.49:0/3877738026 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9c9015a970 0x7f9c900cd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:04.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.225+0000 7f9c8dd9b640 1 -- 172.21.15.49:0/3877738026 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 208784548 0 0) 0x7f9c78003530 con 0x7f9c900cde10 2023-11-06T02:15:04.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.225+0000 7f9c8dd9b640 1 -- 172.21.15.49:0/3877738026 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9c9015a970 msgr2=0x7f9c900cd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:15:04.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.225+0000 7f9c8dd9b640 1 --2- 172.21.15.49:0/3877738026 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9c9015a970 0x7f9c900cd8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:04.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.225+0000 7f9c8dd9b640 1 -- 172.21.15.49:0/3877738026 >> v1:172.21.15.150:6789/0 conn(0x7f9c900ced80 legacy=0x7f9c900b1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:04.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.225+0000 7f9c8dd9b640 1 -- 172.21.15.49:0/3877738026 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c900b0720 con 0x7f9c900cde10 2023-11-06T02:15:04.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.225+0000 7f9c8dd9b640 1 -- 172.21.15.49:0/3877738026 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7f9c84001680 con 0x7f9c900cde10 2023-11-06T02:15:04.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.225+0000 7f9c8dd9b640 1 -- 172.21.15.49:0/3877738026 >> v1:172.21.15.49:6789/0 conn(0x7f9c900cde10 legacy=0x7f9c900ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:04.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.225+0000 7f9c8dd9b640 1 --2- 172.21.15.49:0/3877738026 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c780039b0 0x7f9c78003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:04.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.225+0000 7f9c8dd9b640 1 --2- 172.21.15.49:0/3877738026 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9c9015a970 0x7f9c780047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:04.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.225+0000 7f9c8dd9b640 1 --2- 172.21.15.49:0/3877738026 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c78004d10 0x7f9c78005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:04.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.225+0000 7f9c8dd9b640 1 -- 172.21.15.49:0/3877738026 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c900b0440 con 0x7f9c78004d10 2023-11-06T02:15:04.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.225+0000 7f9c8dd9b640 1 -- 172.21.15.49:0/3877738026 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c7800bb20 con 0x7f9c780039b0 2023-11-06T02:15:04.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.225+0000 7f9c8dd9b640 1 -- 172.21.15.49:0/3877738026 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c7800bcb0 con 0x7f9c9015a970 2023-11-06T02:15:04.229 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.225+0000 7f9c8f59e640 1 --2- 172.21.15.49:0/3877738026 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c78004d10 0x7f9c78005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:04.229 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.225+0000 7f9c8ed9d640 1 --2- 172.21.15.49:0/3877738026 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c780039b0 0x7f9c78003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:04.229 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.225+0000 7f9c8e59c640 1 --2- 172.21.15.49:0/3877738026 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9c9015a970 0x7f9c780047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:04.229 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.225+0000 7f9c8f59e640 1 -- 172.21.15.49:0/3877738026 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9c9015a970 msgr2=0x7f9c780047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:15:04.229 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.225+0000 7f9c8f59e640 1 --2- 172.21.15.49:0/3877738026 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9c9015a970 0x7f9c780047d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:04.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.225+0000 7f9c8f59e640 1 -- 172.21.15.49:0/3877738026 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c780039b0 msgr2=0x7f9c78003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:15:04.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.225+0000 7f9c8f59e640 1 --2- 172.21.15.49:0/3877738026 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c780039b0 0x7f9c78003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:04.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.225+0000 7f9c8f59e640 1 -- 172.21.15.49:0/3877738026 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c78003530 con 0x7f9c78004d10 2023-11-06T02:15:04.234 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.229+0000 7f9c8f59e640 1 --2- 172.21.15.49:0/3877738026 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c78004d10 0x7f9c78005190 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f9c88001ea0 tx=0x7f9c8801c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de385e26947ca0b5 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:15:04.234 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.229+0000 7f9c8dd9b640 1 -- 172.21.15.49:0/3877738026 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c88003b80 con 0x7f9c78004d10 2023-11-06T02:15:04.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.229+0000 7f9c8dd9b640 1 -- 172.21.15.49:0/3877738026 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c88003ce0 con 0x7f9c78004d10 2023-11-06T02:15:04.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.229+0000 7f9c8dd9b640 1 -- 172.21.15.49:0/3877738026 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c880052c0 con 0x7f9c78004d10 2023-11-06T02:15:04.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.229+0000 7f9c95157640 1 -- 172.21.15.49:0/3877738026 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c78004d10 msgr2=0x7f9c78005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:04.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.229+0000 7f9c95157640 1 --2- 172.21.15.49:0/3877738026 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c78004d10 0x7f9c78005190 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f9c88001ea0 tx=0x7f9c8801c760 comp rx=0 tx=0).stop 2023-11-06T02:15:04.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.229+0000 7f9c8f59e640 1 -- 172.21.15.49:0/3877738026 reap_dead start 2023-11-06T02:15:04.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.229+0000 7f9c95157640 1 -- 172.21.15.49:0/3877738026 shutdown_connections 2023-11-06T02:15:04.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.229+0000 7f9c95157640 1 -- 172.21.15.49:0/3877738026 >> 172.21.15.49:0/3877738026 conn(0x7f9c900d0890 msgr2=0x7f9c900d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:15:04.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.229+0000 7f9c95157640 1 -- 172.21.15.49:0/3877738026 shutdown_connections 2023-11-06T02:15:04.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.229+0000 7f9c95157640 1 -- 172.21.15.49:0/3877738026 wait complete. 2023-11-06T02:15:04.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.229+0000 7f9c95157640 1 Processor -- start 2023-11-06T02:15:04.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.229+0000 7f9c95157640 1 -- start start 2023-11-06T02:15:04.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.229+0000 7f9c95157640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c90157ad0 0x7f9c9006be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:04.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.229+0000 7f9c95157640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9c900c43c0 0x7f9c9006c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:04.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.229+0000 7f9c95157640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c900b3ec0 0x7f9c900b4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:04.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.229+0000 7f9c95157640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c900c9530 con 0x7f9c90157ad0 2023-11-06T02:15:04.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.229+0000 7f9c95157640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c900b4880 con 0x7f9c900b3ec0 2023-11-06T02:15:04.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.229+0000 7f9c95157640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c90158570 con 0x7f9c900c43c0 2023-11-06T02:15:04.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.229+0000 7f9c8ed9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c90157ad0 0x7f9c9006be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:04.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.229+0000 7f9c8ed9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c90157ad0 0x7f9c9006be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56616/0 (socket says 172.21.15.49:56616) 2023-11-06T02:15:04.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.229+0000 7f9c8ed9d640 1 -- 172.21.15.49:0/272583490 learned_addr learned my addr 172.21.15.49:0/272583490 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:15:04.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.229+0000 7f9c8e59c640 1 --2- 172.21.15.49:0/272583490 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9c900c43c0 0x7f9c9006c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:04.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.229+0000 7f9c8f59e640 1 --2- 172.21.15.49:0/272583490 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c900b3ec0 0x7f9c900b4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:04.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.229+0000 7f9c8ed9d640 1 -- 172.21.15.49:0/272583490 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9c900c43c0 msgr2=0x7f9c9006c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:04.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.229+0000 7f9c8ed9d640 1 --2- 172.21.15.49:0/272583490 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9c900c43c0 0x7f9c9006c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:04.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.229+0000 7f9c8ed9d640 1 -- 172.21.15.49:0/272583490 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c900b3ec0 msgr2=0x7f9c900b4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:04.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.229+0000 7f9c8ed9d640 1 --2- 172.21.15.49:0/272583490 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c900b3ec0 0x7f9c900b4340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:04.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.229+0000 7f9c8ed9d640 1 -- 172.21.15.49:0/272583490 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c900c8bd0 con 0x7f9c90157ad0 2023-11-06T02:15:04.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.229+0000 7f9c8f59e640 1 --2- 172.21.15.49:0/272583490 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c900b3ec0 0x7f9c900b4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:15:04.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.233+0000 7f9c8ed9d640 1 --2- 172.21.15.49:0/272583490 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c90157ad0 0x7f9c9006be00 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f9c8000f5b0 tx=0x7f9c8000fa70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:15:04.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.233+0000 7f9c777fe640 1 -- 172.21.15.49:0/272583490 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c80005ba0 con 0x7f9c90157ad0 2023-11-06T02:15:04.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.233+0000 7f9c777fe640 1 -- 172.21.15.49:0/272583490 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c80005d00 con 0x7f9c90157ad0 2023-11-06T02:15:04.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.233+0000 7f9c95157640 1 -- 172.21.15.49:0/272583490 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c900b4aa0 con 0x7f9c90157ad0 2023-11-06T02:15:04.242 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.233+0000 7f9c777fe640 1 -- 172.21.15.49:0/272583490 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c80014890 con 0x7f9c90157ad0 2023-11-06T02:15:04.242 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.233+0000 7f9c95157640 1 -- 172.21.15.49:0/272583490 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c900c9150 con 0x7f9c90157ad0 2023-11-06T02:15:04.242 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.233+0000 7f9c95157640 1 -- 172.21.15.49:0/272583490 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c90074040 con 0x7f9c90157ad0 2023-11-06T02:15:04.242 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.233+0000 7f9c777fe640 1 -- 172.21.15.49:0/272583490 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f9c800149f0 con 0x7f9c90157ad0 2023-11-06T02:15:04.243 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.233+0000 7f9c777fe640 1 --2- 172.21.15.49:0/272583490 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9c7c065a60 0x7f9c7c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:04.243 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.233+0000 7f9c777fe640 1 -- 172.21.15.49:0/272583490 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f9c80083910 con 0x7f9c90157ad0 2023-11-06T02:15:04.243 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.233+0000 7f9c8e59c640 1 --2- 172.21.15.49:0/272583490 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9c7c065a60 0x7f9c7c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:04.243 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.233+0000 7f9c8e59c640 1 --2- 172.21.15.49:0/272583490 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9c7c065a60 0x7f9c7c067f20 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f9c9015a4a0 tx=0x7f9c84009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:15:04.243 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.237+0000 7f9c777fe640 1 -- 172.21.15.49:0/272583490 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f9c80051450 con 0x7f9c90157ad0 2023-11-06T02:15:04.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.393+0000 7f9c95157640 1 -- 172.21.15.49:0/272583490 --> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f9c900b0a00 con 0x7f9c7c065a60 2023-11-06T02:15:04.389 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.393+0000 7f9c777fe640 1 -- 172.21.15.49:0/272583490 <== mgr.4100 v2:172.21.15.150:6832/38104 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f9c8400d070 con 0x7f9c7c065a60 2023-11-06T02:15:04.389 INFO:teuthology.orchestra.run.smithi049.stderr:instructed osd(s) 7 to deep-scrub 2023-11-06T02:15:04.391 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.397+0000 7f9c95157640 1 -- 172.21.15.49:0/272583490 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9c7c065a60 msgr2=0x7f9c7c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:04.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.397+0000 7f9c95157640 1 --2- 172.21.15.49:0/272583490 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9c7c065a60 0x7f9c7c067f20 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f9c9015a4a0 tx=0x7f9c84009040 comp rx=0 tx=0).stop 2023-11-06T02:15:04.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.397+0000 7f9c95157640 1 -- 172.21.15.49:0/272583490 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c90157ad0 msgr2=0x7f9c9006be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:04.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.397+0000 7f9c95157640 1 --2- 172.21.15.49:0/272583490 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c90157ad0 0x7f9c9006be00 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f9c8000f5b0 tx=0x7f9c8000fa70 comp rx=0 tx=0).stop 2023-11-06T02:15:04.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.397+0000 7f9c95157640 1 -- 172.21.15.49:0/272583490 shutdown_connections 2023-11-06T02:15:04.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.397+0000 7f9c95157640 1 --2- 172.21.15.49:0/272583490 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9c7c065a60 0x7f9c7c067f20 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:04.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.397+0000 7f9c95157640 1 --2- 172.21.15.49:0/272583490 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c900b3ec0 0x7f9c900b4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:04.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.397+0000 7f9c95157640 1 --2- 172.21.15.49:0/272583490 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9c900c43c0 0x7f9c9006c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:04.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.397+0000 7f9c95157640 1 --2- 172.21.15.49:0/272583490 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c90157ad0 0x7f9c9006be00 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:04.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.397+0000 7f9c95157640 1 -- 172.21.15.49:0/272583490 >> 172.21.15.49:0/272583490 conn(0x7f9c900d0890 msgr2=0x7f9c9006ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:15:04.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.397+0000 7f9c95157640 1 -- 172.21.15.49:0/272583490 shutdown_connections 2023-11-06T02:15:04.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.397+0000 7f9c95157640 1 -- 172.21.15.49:0/272583490 wait complete. 2023-11-06T02:15:04.408 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T02:15:04.531 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.533+0000 7f70765ff640 1 Processor -- start 2023-11-06T02:15:04.531 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.537+0000 7f70765ff640 1 -- start start 2023-11-06T02:15:04.533 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.537+0000 7f70765ff640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f707015ab50 0x7f70700c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:04.533 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.537+0000 7f70765ff640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f70700cb010 0x7f70700b0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:04.533 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.537+0000 7f70765ff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f70700b05c0 0x7f70700700a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:04.534 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.537+0000 7f70765ff640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70700b0a50 con 0x7f70700b05c0 2023-11-06T02:15:04.534 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.537+0000 7f70765ff640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70700cab80 con 0x7f707015ab50 2023-11-06T02:15:04.534 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.537+0000 7f70765ff640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f707015af30 con 0x7f70700cb010 2023-11-06T02:15:04.534 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.537+0000 7f7074b9c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f70700b05c0 0x7f70700700a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:04.534 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.537+0000 7f7074b9c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f70700b05c0 0x7f70700700a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56632/0 (socket says 172.21.15.49:56632) 2023-11-06T02:15:04.535 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.537+0000 7f7074b9c640 1 -- 172.21.15.49:0/331620404 learned_addr learned my addr 172.21.15.49:0/331620404 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:15:04.535 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.537+0000 7f706f7fe640 1 --2- 172.21.15.49:0/331620404 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f70700cb010 0x7f70700b0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:04.535 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.537+0000 7f7074b9c640 1 -- 172.21.15.49:0/331620404 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f70700cb010 msgr2=0x7f70700b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:15:04.535 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.537+0000 7f7074b9c640 1 --2- 172.21.15.49:0/331620404 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f70700cb010 0x7f70700b0080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:04.535 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.537+0000 7f706ffff640 1 --2- 172.21.15.49:0/331620404 >> v2:172.21.15.150:3300/0 conn(0x7f707015ab50 0x7f70700c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:04.536 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.537+0000 7f7074b9c640 1 -- 172.21.15.49:0/331620404 >> v2:172.21.15.150:3300/0 conn(0x7f707015ab50 msgr2=0x7f70700c9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:15:04.536 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.537+0000 7f7074b9c640 1 --2- 172.21.15.49:0/331620404 >> v2:172.21.15.150:3300/0 conn(0x7f707015ab50 0x7f70700c9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:04.536 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.537+0000 7f7074b9c640 1 -- 172.21.15.49:0/331620404 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70700cb5b0 con 0x7f70700b05c0 2023-11-06T02:15:04.536 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.537+0000 7f706ffff640 1 --2- 172.21.15.49:0/331620404 >> v2:172.21.15.150:3300/0 conn(0x7f707015ab50 0x7f70700c9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:15:04.537 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.541+0000 7f7074b9c640 1 --2- 172.21.15.49:0/331620404 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f70700b05c0 0x7f70700700a0 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f70700725a0 tx=0x7f706801c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c616cec6ee6d037 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:15:04.538 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.541+0000 7f706effd640 1 -- 172.21.15.49:0/331620404 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70680a5580 con 0x7f70700b05c0 2023-11-06T02:15:04.538 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.541+0000 7f706effd640 1 -- 172.21.15.49:0/331620404 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70680a56e0 con 0x7f70700b05c0 2023-11-06T02:15:04.538 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.541+0000 7f706effd640 1 -- 172.21.15.49:0/331620404 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70680a5a70 con 0x7f70700b05c0 2023-11-06T02:15:04.538 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.541+0000 7f70765ff640 1 -- 172.21.15.49:0/331620404 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f70700b05c0 msgr2=0x7f70700700a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:04.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.541+0000 7f70765ff640 1 --2- 172.21.15.49:0/331620404 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f70700b05c0 0x7f70700700a0 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f70700725a0 tx=0x7f706801c6e0 comp rx=0 tx=0).stop 2023-11-06T02:15:04.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.541+0000 7f70765ff640 1 -- 172.21.15.49:0/331620404 shutdown_connections 2023-11-06T02:15:04.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.541+0000 7f70765ff640 1 --2- 172.21.15.49:0/331620404 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f70700cb010 0x7f70700b0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:04.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.541+0000 7f70765ff640 1 --2- 172.21.15.49:0/331620404 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f70700b05c0 0x7f70700700a0 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:04.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.541+0000 7f70765ff640 1 --2- 172.21.15.49:0/331620404 >> v2:172.21.15.150:3300/0 conn(0x7f707015ab50 0x7f70700c9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:04.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.541+0000 7f70765ff640 1 -- 172.21.15.49:0/331620404 >> 172.21.15.49:0/331620404 conn(0x7f707006c0a0 msgr2=0x7f707006c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:15:04.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.541+0000 7f70765ff640 1 -- 172.21.15.49:0/331620404 shutdown_connections 2023-11-06T02:15:04.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.541+0000 7f70765ff640 1 -- 172.21.15.49:0/331620404 wait complete. 2023-11-06T02:15:04.543 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.541+0000 7f70765ff640 1 Processor -- start 2023-11-06T02:15:04.543 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.541+0000 7f70765ff640 1 -- start start 2023-11-06T02:15:04.543 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.545+0000 7f70765ff640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70700b05c0 0x7f707015df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:04.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.545+0000 7f70765ff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f70700cb010 0x7f70701506d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:04.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.545+0000 7f70765ff640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f707015ab50 0x7f70700ae840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:04.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.545+0000 7f70765ff640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f707015bd20 con 0x7f70700cb010 2023-11-06T02:15:04.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.545+0000 7f70765ff640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70700ace00 con 0x7f70700b05c0 2023-11-06T02:15:04.545 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.545+0000 7f70765ff640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70700ad0e0 con 0x7f707015ab50 2023-11-06T02:15:04.545 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.545+0000 7f706f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f70700cb010 0x7f70701506d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:04.545 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.545+0000 7f706f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f70700cb010 0x7f70701506d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56648/0 (socket says 172.21.15.49:56648) 2023-11-06T02:15:04.545 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.545+0000 7f706f7fe640 1 -- 172.21.15.49:0/3125150842 learned_addr learned my addr 172.21.15.49:0/3125150842 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:15:04.545 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.545+0000 7f706ffff640 1 --2- 172.21.15.49:0/3125150842 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70700b05c0 0x7f707015df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:04.546 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.545+0000 7f7074b9c640 1 --2- 172.21.15.49:0/3125150842 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f707015ab50 0x7f70700ae840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:04.546 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.545+0000 7f706f7fe640 1 -- 172.21.15.49:0/3125150842 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f707015ab50 msgr2=0x7f70700ae840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:04.546 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.545+0000 7f706f7fe640 1 --2- 172.21.15.49:0/3125150842 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f707015ab50 0x7f70700ae840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:04.546 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.545+0000 7f706f7fe640 1 -- 172.21.15.49:0/3125150842 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70700b05c0 msgr2=0x7f707015df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:04.546 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.545+0000 7f706f7fe640 1 --2- 172.21.15.49:0/3125150842 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70700b05c0 0x7f707015df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:04.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.545+0000 7f706f7fe640 1 -- 172.21.15.49:0/3125150842 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7068009b30 con 0x7f70700cb010 2023-11-06T02:15:04.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.545+0000 7f706f7fe640 1 --2- 172.21.15.49:0/3125150842 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f70700cb010 0x7f70701506d0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f706000c7f0 tx=0x7f706000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:15:04.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.545+0000 7f706cff9640 1 -- 172.21.15.49:0/3125150842 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7060004220 con 0x7f70700cb010 2023-11-06T02:15:04.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.545+0000 7f706cff9640 1 -- 172.21.15.49:0/3125150842 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7060004380 con 0x7f70700cb010 2023-11-06T02:15:04.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.545+0000 7f70765ff640 1 -- 172.21.15.49:0/3125150842 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70701526a0 con 0x7f70700cb010 2023-11-06T02:15:04.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.545+0000 7f706cff9640 1 -- 172.21.15.49:0/3125150842 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7060003860 con 0x7f70700cb010 2023-11-06T02:15:04.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.545+0000 7f70765ff640 1 -- 172.21.15.49:0/3125150842 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7070150c10 con 0x7f70700cb010 2023-11-06T02:15:04.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.545+0000 7f70765ff640 1 -- 172.21.15.49:0/3125150842 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70700b0de0 con 0x7f70700cb010 2023-11-06T02:15:04.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.545+0000 7f706cff9640 1 -- 172.21.15.49:0/3125150842 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f7060003bd0 con 0x7f70700cb010 2023-11-06T02:15:04.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.545+0000 7f706cff9640 1 --2- 172.21.15.49:0/3125150842 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f705c065a90 0x7f705c067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:04.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.545+0000 7f706cff9640 1 -- 172.21.15.49:0/3125150842 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f7060082fb0 con 0x7f70700cb010 2023-11-06T02:15:04.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.545+0000 7f706ffff640 1 --2- 172.21.15.49:0/3125150842 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f705c065a90 0x7f705c067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:04.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.549+0000 7f706ffff640 1 --2- 172.21.15.49:0/3125150842 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f705c065a90 0x7f705c067f50 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f706400a950 tx=0x7f7064008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:15:04.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.553+0000 7f706cff9640 1 -- 172.21.15.49:0/3125150842 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f70600b4070 con 0x7f70700cb010 2023-11-06T02:15:04.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.701+0000 7f70765ff640 1 -- 172.21.15.49:0/3125150842 --> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f70700acf70 con 0x7f705c065a90 2023-11-06T02:15:04.704 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.709+0000 7f706cff9640 1 -- 172.21.15.49:0/3125150842 <== mgr.4100 v2:172.21.15.150:6832/38104 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+71091 (secure 0 0 0) 0x7f70700acf70 con 0x7f705c065a90 2023-11-06T02:15:04.705 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:15:04.706 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2023-11-06T02:15:04.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.709+0000 7f70765ff640 1 -- 172.21.15.49:0/3125150842 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f705c065a90 msgr2=0x7f705c067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:04.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.709+0000 7f70765ff640 1 --2- 172.21.15.49:0/3125150842 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f705c065a90 0x7f705c067f50 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f706400a950 tx=0x7f7064008040 comp rx=0 tx=0).stop 2023-11-06T02:15:04.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.709+0000 7f70765ff640 1 -- 172.21.15.49:0/3125150842 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f70700cb010 msgr2=0x7f70701506d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:04.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.709+0000 7f70765ff640 1 --2- 172.21.15.49:0/3125150842 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f70700cb010 0x7f70701506d0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f706000c7f0 tx=0x7f706000ccb0 comp rx=0 tx=0).stop 2023-11-06T02:15:04.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.709+0000 7f70765ff640 1 -- 172.21.15.49:0/3125150842 shutdown_connections 2023-11-06T02:15:04.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.709+0000 7f70765ff640 1 --2- 172.21.15.49:0/3125150842 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f705c065a90 0x7f705c067f50 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:04.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.709+0000 7f70765ff640 1 --2- 172.21.15.49:0/3125150842 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f70700cb010 0x7f70701506d0 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:04.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.709+0000 7f70765ff640 1 --2- 172.21.15.49:0/3125150842 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f707015ab50 0x7f70700ae840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:04.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.709+0000 7f70765ff640 1 --2- 172.21.15.49:0/3125150842 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70700b05c0 0x7f707015df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:04.709 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.709+0000 7f70765ff640 1 -- 172.21.15.49:0/3125150842 >> 172.21.15.49:0/3125150842 conn(0x7f707006c0a0 msgr2=0x7f70700ae060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:15:04.709 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.709+0000 7f70765ff640 1 -- 172.21.15.49:0/3125150842 shutdown_connections 2023-11-06T02:15:04.709 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:04.709+0000 7f70765ff640 1 -- 172.21.15.49:0/3125150842 wait complete. 2023-11-06T02:15:04.723 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":281,"stamp":"2023-11-06T02:15:03.317701+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":261796,"kb_used_data":9316,"kb_used_omap":0,"kb_used_meta":252416,"kb_avail":754712924,"statfs":{"total":773094113280,"available":772826034176,"internally_reserved":0,"allocated":9539584,"data_stored":4952261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":258473984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8390727,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-258,"num_read_kb":-264,"num_write":-180,"num_write_kb":-8296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002247"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":224,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T02:14:56.128156+0000","last_change":"2023-11-06T02:08:20.523427+0000","last_active":"2023-11-06T02:14:56.128156+0000","last_peered":"2023-11-06T02:14:56.128156+0000","last_clean":"2023-11-06T02:14:56.128156+0000","last_became_active":"2023-11-06T02:08:20.521089+0000","last_became_peered":"2023-11-06T02:08:20.521089+0000","last_unstale":"2023-11-06T02:14:56.128156+0000","last_undegraded":"2023-11-06T02:14:56.128156+0000","last_fullsized":"2023-11-06T02:14:56.128156+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:19.252348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:19.252348+0000","last_clean_scrub_stamp":"2023-11-06T02:08:19.252348+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:30:15.566397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":70,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T02:14:56.126616+0000","last_change":"2023-11-06T02:08:26.059502+0000","last_active":"2023-11-06T02:14:56.126616+0000","last_peered":"2023-11-06T02:14:56.126616+0000","last_clean":"2023-11-06T02:14:56.126616+0000","last_became_active":"2023-11-06T02:08:26.058612+0000","last_became_peered":"2023-11-06T02:08:26.058612+0000","last_unstale":"2023-11-06T02:14:56.126616+0000","last_undegraded":"2023-11-06T02:14:56.126616+0000","last_fullsized":"2023-11-06T02:14:56.126616+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:10.626021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":71,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T02:14:56.127930+0000","last_change":"2023-11-06T02:08:26.060406+0000","last_active":"2023-11-06T02:14:56.127930+0000","last_peered":"2023-11-06T02:14:56.127930+0000","last_clean":"2023-11-06T02:14:56.127930+0000","last_became_active":"2023-11-06T02:08:26.059262+0000","last_became_peered":"2023-11-06T02:08:26.059262+0000","last_unstale":"2023-11-06T02:14:56.127930+0000","last_undegraded":"2023-11-06T02:14:56.127930+0000","last_fullsized":"2023-11-06T02:14:56.127930+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:10.626021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":72,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T02:14:56.534822+0000","last_change":"2023-11-06T02:08:26.559477+0000","last_active":"2023-11-06T02:14:56.534822+0000","last_peered":"2023-11-06T02:14:56.534822+0000","last_clean":"2023-11-06T02:14:56.534822+0000","last_became_active":"2023-11-06T02:08:26.559202+0000","last_became_peered":"2023-11-06T02:08:26.559202+0000","last_unstale":"2023-11-06T02:14:56.534822+0000","last_undegraded":"2023-11-06T02:14:56.534822+0000","last_fullsized":"2023-11-06T02:14:56.534822+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":70,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T02:14:56.126347+0000","last_change":"2023-11-06T02:08:26.059546+0000","last_active":"2023-11-06T02:14:56.126347+0000","last_peered":"2023-11-06T02:14:56.126347+0000","last_clean":"2023-11-06T02:14:56.126347+0000","last_became_active":"2023-11-06T02:08:26.058806+0000","last_became_peered":"2023-11-06T02:08:26.058806+0000","last_unstale":"2023-11-06T02:14:56.126347+0000","last_undegraded":"2023-11-06T02:14:56.126347+0000","last_fullsized":"2023-11-06T02:14:56.126347+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:23.468510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":71,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T02:14:56.508894+0000","last_change":"2023-11-06T02:08:26.436463+0000","last_active":"2023-11-06T02:14:56.508894+0000","last_peered":"2023-11-06T02:14:56.508894+0000","last_clean":"2023-11-06T02:14:56.508894+0000","last_became_active":"2023-11-06T02:08:26.436075+0000","last_became_peered":"2023-11-06T02:08:26.436075+0000","last_unstale":"2023-11-06T02:14:56.508894+0000","last_undegraded":"2023-11-06T02:14:56.508894+0000","last_fullsized":"2023-11-06T02:14:56.508894+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":71,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T02:14:56.131140+0000","last_change":"2023-11-06T02:08:26.436607+0000","last_active":"2023-11-06T02:14:56.131140+0000","last_peered":"2023-11-06T02:14:56.131140+0000","last_clean":"2023-11-06T02:14:56.131140+0000","last_became_active":"2023-11-06T02:08:26.435924+0000","last_became_peered":"2023-11-06T02:08:26.435924+0000","last_unstale":"2023-11-06T02:14:56.131140+0000","last_undegraded":"2023-11-06T02:14:56.131140+0000","last_fullsized":"2023-11-06T02:14:56.131140+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:23.468510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":71,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T02:14:58.274138+0000","last_change":"2023-11-06T02:08:26.057076+0000","last_active":"2023-11-06T02:14:58.274138+0000","last_peered":"2023-11-06T02:14:58.274138+0000","last_clean":"2023-11-06T02:14:58.274138+0000","last_became_active":"2023-11-06T02:08:26.056419+0000","last_became_peered":"2023-11-06T02:08:26.056419+0000","last_unstale":"2023-11-06T02:14:58.274138+0000","last_undegraded":"2023-11-06T02:14:58.274138+0000","last_fullsized":"2023-11-06T02:14:58.274138+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":79,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T02:14:56.520224+0000","last_change":"2023-11-06T02:08:26.058596+0000","last_active":"2023-11-06T02:14:56.520224+0000","last_peered":"2023-11-06T02:14:56.520224+0000","last_clean":"2023-11-06T02:14:56.520224+0000","last_became_active":"2023-11-06T02:08:26.057015+0000","last_became_peered":"2023-11-06T02:08:26.057015+0000","last_unstale":"2023-11-06T02:14:56.520224+0000","last_undegraded":"2023-11-06T02:14:56.520224+0000","last_fullsized":"2023-11-06T02:14:56.520224+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:08:25.026811+0000","last_clean_scrub_stamp":"2023-11-06T02:08:25.026811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T02:55:04.552404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574932,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32648,"kb_used_data":1600,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339192,"statfs":{"total":96636764160,"available":96603332608,"internally_reserved":0,"allocated":1638400,"data_stored":1061936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.87,"15min":0.85599999999999998},"min":{"1min":0.62,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.147,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.52700000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.2230000000000001,"5min":1.421,"15min":1.506},"last":0.70899999999999996}]},{"osd":1,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.83399999999999996,"15min":0.84499999999999997},"min":{"1min":0.498,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.3320000000000001,"5min":1.345,"15min":1.345},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.92900000000000005,"15min":0.90200000000000002},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.54900000000000004},"max":{"1min":1.131,"5min":1.681,"15min":1.681},"last":0.998}]},{"osd":2,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.96399999999999997,"15min":0.95699999999999996},"min":{"1min":0.65300000000000002,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.1919999999999999,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.745},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.92200000000000004,"15min":0.97399999999999998},"min":{"1min":0.57399999999999995,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.1919999999999999,"5min":1.5,"15min":1.5920000000000001},"last":1.0800000000000001}]},{"osd":3,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.89200000000000002,"15min":0.88},"min":{"1min":0.66100000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.1519999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.872},{"interface":"front","average":{"1min":0.995,"5min":1.03,"15min":1.0609999999999999},"min":{"1min":0.74399999999999999,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.2110000000000001,"5min":1.526,"15min":1.526},"last":1.155}]},{"osd":4,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.91700000000000004,"15min":0.94399999999999995},"min":{"1min":0.73399999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.1930000000000001,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":1.2330000000000001},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.97999999999999998,"15min":1.0229999999999999},"min":{"1min":0.65000000000000002,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.1519999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":1.1180000000000001}]},{"osd":5,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":1.026,"15min":1.0609999999999999},"min":{"1min":0.69199999999999995,"5min":0.503,"15min":0.503},"max":{"1min":1.2629999999999999,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":1.28},{"interface":"front","average":{"1min":1.044,"5min":1.044,"15min":1.105},"min":{"1min":0.71299999999999997,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.288,"5min":1.3700000000000001,"15min":1.7809999999999999},"last":1.3}]},{"osd":6,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":1.1180000000000001,"15min":1.1739999999999999},"min":{"1min":0.89500000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.274,"5min":1.5009999999999999,"15min":1.6679999999999999},"last":1.349},{"interface":"front","average":{"1min":1.028,"5min":1.0649999999999999,"15min":1.127},"min":{"1min":0.72599999999999998,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.2509999999999999,"5min":1.4570000000000001,"15min":1.74},"last":1.2070000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574930,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32068,"kb_used_data":1020,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339772,"statfs":{"total":96636764160,"available":96603926528,"internally_reserved":0,"allocated":1044480,"data_stored":471568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.872,"15min":0.86199999999999999},"min":{"1min":0.57199999999999995,"5min":0.495,"15min":0.495},"max":{"1min":1.226,"5min":1.226,"15min":1.2609999999999999},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.88600000000000001,"15min":0.83199999999999996},"min":{"1min":0.5,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.3280000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.66700000000000004}]},{"osd":1,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.85099999999999998,"15min":0.86699999999999999},"min":{"1min":0.54400000000000004,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.1639999999999999,"5min":1.244,"15min":1.244},"last":0.95299999999999996},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.90500000000000003,"15min":0.88400000000000001},"min":{"1min":0.56599999999999995,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":0.68400000000000005}]},{"osd":2,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.89200000000000002,"15min":0.89600000000000002},"min":{"1min":0.61499999999999999,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.877,"5min":0.88900000000000001,"15min":0.91700000000000004},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.075,"5min":1.411,"15min":1.411},"last":0.89600000000000002}]},{"osd":3,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.93500000000000005,"15min":0.95899999999999996},"min":{"1min":0.74099999999999999,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.875},{"interface":"front","average":{"1min":1.0309999999999999,"5min":0.999,"15min":1},"min":{"1min":0.81599999999999995,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.2609999999999999,"5min":1.381,"15min":1.381},"last":0.85899999999999999}]},{"osd":4,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.92200000000000004,"15min":0.96999999999999997},"min":{"1min":0.627,"5min":0.56799999999999995,"15min":0.52900000000000003},"max":{"1min":1.2390000000000001,"5min":1.302,"15min":1.474},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.997,"5min":0.97399999999999998,"15min":1.018},"min":{"1min":0.66700000000000004,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.4039999999999999,"5min":1.556,"15min":1.556},"last":0.58499999999999996}]},{"osd":5,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":0.97199999999999998,"15min":1.0089999999999999},"min":{"1min":0.68899999999999995,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.71499999999999997},{"interface":"front","average":{"1min":1.1319999999999999,"5min":1.075,"15min":1.101},"min":{"1min":0.82699999999999996,"5min":0.66900000000000004,"15min":0.58699999999999997},"max":{"1min":1.3999999999999999,"5min":1.587,"15min":1.587},"last":0.61699999999999999}]},{"osd":7,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.087,"15min":1.107},"min":{"1min":0.88300000000000001,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.5369999999999999,"5min":1.615,"15min":1.615},"last":0.76900000000000002},{"interface":"front","average":{"1min":1.103,"5min":1.0509999999999999,"15min":1.127},"min":{"1min":0.85599999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.4610000000000001,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.629}]}]},{"osd":5,"up_from":13,"seq":55834574932,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32068,"kb_used_data":1020,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339772,"statfs":{"total":96636764160,"available":96603926528,"internally_reserved":0,"allocated":1044480,"data_stored":471568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:14:33 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.86199999999999999,"15min":0.83999999999999997},"min":{"1min":0.54400000000000004,"5min":0.51700000000000002,"15min":0.47299999999999998},"max":{"1min":1.079,"5min":1.381,"15min":1.381},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.86899999999999999,"15min":0.85099999999999998},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.159,"5min":1.3200000000000001,"15min":1.6579999999999999},"last":0.81499999999999995}]},{"osd":1,"last update":"Mon Nov 6 02:14:33 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.90500000000000003,"15min":0.86799999999999999},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.60299999999999998},"max":{"1min":1.369,"5min":1.369,"15min":1.369},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.90000000000000002,"15min":0.84399999999999997},"min":{"1min":0.53700000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.1639999999999999,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.94899999999999995}]},{"osd":2,"last update":"Mon Nov 6 02:14:33 2023","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.93899999999999995,"15min":0.94299999999999995},"min":{"1min":0.65100000000000002,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.242,"5min":1.298,"15min":1.3560000000000001},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.89700000000000002,"15min":0.82499999999999996},"min":{"1min":0.58699999999999997,"5min":0.55800000000000005,"15min":0.47899999999999998},"max":{"1min":1.218,"5min":1.323,"15min":1.377},"last":0.875}]},{"osd":3,"last update":"Mon Nov 6 02:14:33 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.95199999999999996,"15min":0.93000000000000005},"min":{"1min":0.66400000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.292,"5min":1.3580000000000001,"15min":1.4399999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":1.0369999999999999,"5min":0.98799999999999999,"15min":1.05},"min":{"1min":0.76700000000000002,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.2569999999999999,"5min":1.349,"15min":1.7250000000000001},"last":0.76700000000000002}]},{"osd":4,"last update":"Mon Nov 6 02:14:33 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.87,"15min":0.90000000000000002},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.0129999999999999,"5min":1.276,"15min":1.276},"last":0.99299999999999999},{"interface":"front","average":{"1min":0.995,"5min":0.98599999999999999,"15min":1.014},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.496,"5min":1.496,"15min":1.554},"last":0.92400000000000004}]},{"osd":6,"last update":"Mon Nov 6 02:14:33 2023","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.98699999999999999,"15min":1.0409999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.3440000000000001,"5min":1.454,"15min":1.454},"last":0.85699999999999998},{"interface":"front","average":{"1min":1.046,"5min":1.0229999999999999,"15min":1.0349999999999999},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":2.2309999999999999,"5min":2.2309999999999999,"15min":2.2309999999999999},"last":1.073}]},{"osd":7,"last update":"Mon Nov 6 02:14:33 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.0640000000000001,"15min":1.0720000000000001},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.5209999999999999,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":1.044},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.117,"15min":1.1419999999999999},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":2.3759999999999999,"5min":2.3759999999999999,"15min":2.3759999999999999},"last":1.0189999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574932,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32068,"kb_used_data":1020,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339772,"statfs":{"total":96636764160,"available":96603926528,"internally_reserved":0,"allocated":1044480,"data_stored":471568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:14:27 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.86399999999999999,"15min":0.83799999999999997},"min":{"1min":0.53500000000000003,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.2450000000000001,"5min":1.343,"15min":1.343},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.84499999999999997,"15min":0.79500000000000004},"min":{"1min":0.51600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.175,"5min":1.28,"15min":1.3049999999999999},"last":0.77800000000000002}]},{"osd":1,"last update":"Mon Nov 6 02:14:27 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.86399999999999999,"15min":0.84999999999999998},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.508,"5min":1.508,"15min":1.508},"last":0.96199999999999997},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.91800000000000004,"15min":0.90000000000000002},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.1080000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.66900000000000004}]},{"osd":2,"last update":"Mon Nov 6 02:14:27 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.90800000000000003,"15min":0.98199999999999998},"min":{"1min":0.57999999999999996,"5min":0.503,"15min":0.503},"max":{"1min":1.0880000000000001,"5min":1.389,"15min":1.615},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.95499999999999996,"15min":0.93500000000000005},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.57899999999999996},"max":{"1min":1.1559999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.74099999999999999}]},{"osd":3,"last update":"Mon Nov 6 02:14:27 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.86199999999999999,"15min":0.90000000000000002},"min":{"1min":0.59899999999999998,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.22,"5min":1.22,"15min":1.52},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.97299999999999998,"15min":1.0369999999999999},"min":{"1min":0.68100000000000005,"5min":0.63900000000000001,"15min":0.59999999999999998},"max":{"1min":1.589,"5min":1.595,"15min":1.595},"last":0.63400000000000001}]},{"osd":5,"last update":"Mon Nov 6 02:14:27 2023","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":1.026},"min":{"1min":0.72599999999999998,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.212,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":0.878},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.96099999999999997,"15min":1.024},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.629},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.7869999999999999},"last":0.99099999999999999}]},{"osd":6,"last update":"Mon Nov 6 02:14:27 2023","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.008,"15min":1.0649999999999999},"min":{"1min":0.77600000000000002,"5min":0.68000000000000005,"15min":0.67100000000000004},"max":{"1min":1.6679999999999999,"5min":1.6679999999999999,"15min":1.6699999999999999},"last":1.0609999999999999},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.085,"15min":1.1599999999999999},"min":{"1min":0.71999999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.2210000000000001,"5min":1.5900000000000001,"15min":2.1190000000000002},"last":0.93899999999999995}]},{"osd":7,"last update":"Mon Nov 6 02:14:27 2023","interfaces":[{"interface":"back","average":{"1min":1.056,"5min":1.042,"15min":1.0760000000000001},"min":{"1min":0.77100000000000002,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.7110000000000001},"last":1.0389999999999999},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.117,"15min":1.1990000000000001},"min":{"1min":0.90500000000000003,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.754,"5min":1.754,"15min":1.754},"last":1.0209999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574931,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32068,"kb_used_data":1020,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339772,"statfs":{"total":96636764160,"available":96603926528,"internally_reserved":0,"allocated":1044480,"data_stored":471568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63,"15min":0.64200000000000002},"min":{"1min":0.45300000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":1.171,"5min":1.198,"15min":1.198},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.65400000000000003,"15min":0.63500000000000001},"min":{"1min":0.49099999999999999,"5min":0.29899999999999999,"15min":0.29799999999999999},"max":{"1min":1.1819999999999999,"5min":1.21,"15min":1.21},"last":0.47599999999999998}]},{"osd":1,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.61599999999999999,"15min":0.626},"min":{"1min":0.42499999999999999,"5min":0.248,"15min":0.248},"max":{"1min":0.86799999999999999,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.75800000000000001,"15min":0.74199999999999999},"min":{"1min":0.46899999999999997,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.175,"5min":1.204,"15min":1.204},"last":0.38}]},{"osd":2,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.72499999999999998,"15min":0.73099999999999998},"min":{"1min":0.499,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.81299999999999994,"15min":0.81999999999999995},"min":{"1min":0.62,"5min":0.308,"15min":0.308},"max":{"1min":1.0329999999999999,"5min":1.405,"15min":1.405},"last":0.34000000000000002}]},{"osd":4,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.62,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.262,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.93700000000000006,"15min":0.97799999999999998},"min":{"1min":0.70799999999999996,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.212,"5min":1.292,"15min":1.4410000000000001},"last":0.72099999999999997}]},{"osd":5,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.97499999999999998,"15min":1.0109999999999999},"min":{"1min":0.72699999999999998,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.21,"5min":1.375,"15min":1.375},"last":0.65500000000000003},{"interface":"front","average":{"1min":1.024,"5min":1.0269999999999999,"15min":1.0629999999999999},"min":{"1min":0.65200000000000002,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.323,"5min":1.4890000000000001,"15min":2.21},"last":0.56999999999999995}]},{"osd":6,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.0209999999999999,"15min":1.024},"min":{"1min":0.79800000000000004,"5min":0.439,"15min":0.439},"max":{"1min":1.3080000000000001,"5min":1.395,"15min":1.395},"last":0.57999999999999996},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.0269999999999999,"15min":1.0569999999999999},"min":{"1min":0.74099999999999999,"5min":0.375,"15min":0.375},"max":{"1min":1.2709999999999999,"5min":1.4319999999999999,"15min":1.554},"last":0.69599999999999995}]},{"osd":7,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.1020000000000001,"15min":1.1220000000000001},"min":{"1min":0.68200000000000005,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.4410000000000001,"5min":1.5549999999999999,"15min":2.2829999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":1.123,"5min":1.0980000000000001,"15min":1.1040000000000001},"min":{"1min":0.91100000000000003,"5min":0.497,"15min":0.497},"max":{"1min":1.3620000000000001,"5min":1.504,"15min":1.504},"last":0.72999999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574931,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32068,"kb_used_data":1020,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339772,"statfs":{"total":96636764160,"available":96603926528,"internally_reserved":0,"allocated":1044480,"data_stored":471568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:14:28 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.59099999999999997,"15min":0.52900000000000003},"min":{"1min":0.28499999999999998,"5min":0.252,"15min":0.252},"max":{"1min":0.90300000000000002,"5min":1.079,"15min":1.079},"last":0.38},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.63400000000000001,"15min":0.56999999999999995},"min":{"1min":0.374,"5min":0.316,"15min":0.27300000000000002},"max":{"1min":0.86699999999999999,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":0.64300000000000002}]},{"osd":1,"last update":"Mon Nov 6 02:14:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.65000000000000002,"15min":0.58999999999999997},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.26400000000000001},"max":{"1min":0.97699999999999998,"5min":1.48,"15min":1.48},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.72799999999999998,"15min":0.621},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.26500000000000001},"max":{"1min":1.04,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":0.59199999999999997}]},{"osd":3,"last update":"Mon Nov 6 02:14:28 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.71399999999999997,"15min":0.63600000000000001},"min":{"1min":0.316,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.0580000000000001,"5min":1.121,"15min":1.155},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.76400000000000001,"15min":0.68200000000000005},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.069,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.52900000000000003}]},{"osd":4,"last update":"Mon Nov 6 02:14:28 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.93100000000000005,"15min":0.84299999999999997},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.1539999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.92900000000000005,"15min":0.91900000000000004},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.234,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":0.78700000000000003}]},{"osd":5,"last update":"Mon Nov 6 02:14:28 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.96299999999999997,"15min":0.94899999999999995},"min":{"1min":0.56299999999999994,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.258,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.97299999999999998,"15min":0.94699999999999995},"min":{"1min":0.626,"5min":0.60999999999999999,"15min":0.52600000000000002},"max":{"1min":1.2310000000000001,"5min":1.615,"15min":1.7110000000000001},"last":0.88600000000000001}]},{"osd":6,"last update":"Mon Nov 6 02:14:28 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.96899999999999997,"15min":0.94799999999999995},"min":{"1min":0.60199999999999998,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.218,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.93600000000000005,"5min":1.01,"15min":1.0009999999999999},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.274,"5min":1.675,"15min":1.677},"last":0.84899999999999998}]},{"osd":7,"last update":"Mon Nov 6 02:14:28 2023","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":1.0309999999999999,"15min":0.99299999999999999},"min":{"1min":0.67500000000000004,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.2869999999999999,"5min":1.659,"15min":1.659},"last":0.72099999999999997},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.0760000000000001,"15min":1.0529999999999999},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.1879999999999999,"5min":1.4650000000000001,"15min":1.5980000000000001},"last":0.92900000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574932,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36164,"kb_used_data":1020,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335676,"statfs":{"total":96636764160,"available":96599732224,"internally_reserved":0,"allocated":1044480,"data_stored":471568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:14:28 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.628,"15min":0.626},"min":{"1min":0.29599999999999999,"5min":0.28699999999999998,"15min":0.224},"max":{"1min":0.95099999999999996,"5min":1.603,"15min":1.603},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.71999999999999997,"15min":0.67400000000000004},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.081,"5min":2.161,"15min":2.161},"last":0.30399999999999999}]},{"osd":2,"last update":"Mon Nov 6 02:14:28 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.64700000000000002,"15min":0.63},"min":{"1min":0.34399999999999997,"5min":0.253,"15min":0.253},"max":{"1min":1.036,"5min":1.649,"15min":1.649},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.74199999999999999,"15min":0.74399999999999999},"min":{"1min":0.373,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.91700000000000004,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.49099999999999999}]},{"osd":3,"last update":"Mon Nov 6 02:14:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.80400000000000005,"15min":0.80300000000000005},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.131,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":0.318},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.79100000000000004,"15min":0.80400000000000005},"min":{"1min":0.35499999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.94199999999999995,"5min":1.5,"15min":1.8149999999999999},"last":0.41399999999999998}]},{"osd":4,"last update":"Mon Nov 6 02:14:28 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.96199999999999997,"15min":0.98199999999999998},"min":{"1min":0.55600000000000005,"5min":0.504,"15min":0.504},"max":{"1min":1.2110000000000001,"5min":2.2759999999999998,"15min":2.2759999999999998},"last":0.624},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.95599999999999996,"15min":0.98099999999999998},"min":{"1min":0.63300000000000001,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.0700000000000001,"5min":2.9180000000000001,"15min":2.9180000000000001},"last":0.80700000000000005}]},{"osd":5,"last update":"Mon Nov 6 02:14:28 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.97199999999999998,"15min":0.96799999999999997},"min":{"1min":0.65800000000000003,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.177,"5min":2.7469999999999999,"15min":2.7469999999999999},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.98399999999999999,"5min":1.05,"15min":1.069},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.52100000000000002},"max":{"1min":1.3720000000000001,"5min":1.921,"15min":1.921},"last":0.61499999999999999}]},{"osd":6,"last update":"Mon Nov 6 02:14:28 2023","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":1.0960000000000001,"15min":1.1000000000000001},"min":{"1min":0.61899999999999999,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.304,"5min":3.5710000000000002,"15min":3.5710000000000002},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.94999999999999996,"5min":1.0369999999999999,"15min":1.0629999999999999},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.268,"5min":2.843,"15min":2.843},"last":0.69199999999999995}]},{"osd":7,"last update":"Mon Nov 6 02:14:28 2023","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":1.0980000000000001,"15min":1.1140000000000001},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.4159999999999999,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":1.052,"5min":1.1419999999999999,"15min":1.1539999999999999},"min":{"1min":0.61799999999999999,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.3520000000000001,"5min":3.4790000000000001,"15min":3.4790000000000001},"last":0.752}]}]},{"osd":0,"up_from":13,"seq":55834574932,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32644,"kb_used_data":1596,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339196,"statfs":{"total":96636764160,"available":96603336704,"internally_reserved":0,"allocated":1634304,"data_stored":1060917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 02:14:27 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.62,"15min":0.60499999999999998},"min":{"1min":0.32500000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.94899999999999995,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.62,"5min":0.66600000000000004,"15min":0.64600000000000002},"min":{"1min":0.26100000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.96599999999999997,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.51800000000000002}]},{"osd":2,"last update":"Mon Nov 6 02:14:27 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.68799999999999994,"15min":0.67200000000000004},"min":{"1min":0.36599999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.0900000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.69699999999999995,"15min":0.68200000000000005},"min":{"1min":0.315,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.98199999999999998,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.45700000000000002}]},{"osd":3,"last update":"Mon Nov 6 02:14:27 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.69499999999999995,"15min":0.65100000000000002},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":2.1150000000000002,"5min":2.1150000000000002,"15min":2.1150000000000002},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.78500000000000003,"15min":0.749},"min":{"1min":0.39300000000000002,"5min":0.317,"15min":0.317},"max":{"1min":1.0860000000000001,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.42199999999999999}]},{"osd":4,"last update":"Mon Nov 6 02:14:27 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.93000000000000005,"15min":0.86099999999999999},"min":{"1min":0.60099999999999998,"5min":0.55400000000000005,"15min":0.54300000000000004},"max":{"1min":2.3130000000000002,"5min":2.3130000000000002,"15min":2.3130000000000002},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.96299999999999997,"15min":0.94499999999999995},"min":{"1min":0.59599999999999997,"5min":0.5,"15min":0.5},"max":{"1min":1.3140000000000001,"5min":1.3839999999999999,"15min":1.387},"last":0.60699999999999998}]},{"osd":5,"last update":"Mon Nov 6 02:14:27 2023","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.95999999999999996,"15min":0.95199999999999996},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.89800000000000002},{"interface":"front","average":{"1min":1.006,"5min":0.96099999999999997,"15min":0.90100000000000002},"min":{"1min":0.53800000000000003,"5min":0.496,"15min":0.496},"max":{"1min":2.3530000000000002,"5min":2.3530000000000002,"15min":2.3530000000000002},"last":0.74399999999999999}]},{"osd":6,"last update":"Mon Nov 6 02:14:27 2023","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":1.0349999999999999,"15min":1.036},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.4550000000000001,"5min":1.581,"15min":1.589},"last":0.69099999999999995},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.0309999999999999,"15min":1.01},"min":{"1min":0.68100000000000005,"5min":0.65800000000000003,"15min":0.57099999999999995},"max":{"1min":1.4990000000000001,"5min":1.5529999999999999,"15min":1.6220000000000001},"last":0.92700000000000005}]},{"osd":7,"last update":"Mon Nov 6 02:14:27 2023","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":1.024,"15min":1.02},"min":{"1min":0.55300000000000005,"5min":0.504,"15min":0.504},"max":{"1min":1.357,"5min":1.504,"15min":1.504},"last":0.80500000000000005},{"interface":"front","average":{"1min":1.034,"5min":1.0029999999999999,"15min":1.0069999999999999},"min":{"1min":0.64700000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":2.387,"5min":2.387,"15min":2.387},"last":0.85099999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T02:15:04.727 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-11-06T02:08:19.252348+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=6, tm_hour=2, tm_min=8, tm_sec=19, tm_wday=0, tm_yday=310, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=6, tm_hour=2, tm_min=14, tm_sec=59, tm_wday=0, tm_yday=310, tm_isdst=0) 2023-11-06T02:15:04.727 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-11-06T02:08:25.026811+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=6, tm_hour=2, tm_min=8, tm_sec=25, tm_wday=0, tm_yday=310, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=6, tm_hour=2, tm_min=14, tm_sec=59, tm_wday=0, tm_yday=310, tm_isdst=0) 2023-11-06T02:15:04.728 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-11-06T02:08:25.026811+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=6, tm_hour=2, tm_min=8, tm_sec=25, tm_wday=0, tm_yday=310, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=6, tm_hour=2, tm_min=14, tm_sec=59, tm_wday=0, tm_yday=310, tm_isdst=0) 2023-11-06T02:15:04.728 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2023-11-06T02:08:25.026811+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=6, tm_hour=2, tm_min=8, tm_sec=25, tm_wday=0, tm_yday=310, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=6, tm_hour=2, tm_min=14, tm_sec=59, tm_wday=0, tm_yday=310, tm_isdst=0) 2023-11-06T02:15:04.728 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-11-06T02:08:25.026811+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=6, tm_hour=2, tm_min=8, tm_sec=25, tm_wday=0, tm_yday=310, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=6, tm_hour=2, tm_min=14, tm_sec=59, tm_wday=0, tm_yday=310, tm_isdst=0) 2023-11-06T02:15:04.729 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-11-06T02:08:25.026811+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=6, tm_hour=2, tm_min=8, tm_sec=25, tm_wday=0, tm_yday=310, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=6, tm_hour=2, tm_min=14, tm_sec=59, tm_wday=0, tm_yday=310, tm_isdst=0) 2023-11-06T02:15:04.729 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-11-06T02:08:25.026811+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=6, tm_hour=2, tm_min=8, tm_sec=25, tm_wday=0, tm_yday=310, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=6, tm_hour=2, tm_min=14, tm_sec=59, tm_wday=0, tm_yday=310, tm_isdst=0) 2023-11-06T02:15:04.729 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-11-06T02:08:25.026811+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=6, tm_hour=2, tm_min=8, tm_sec=25, tm_wday=0, tm_yday=310, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=6, tm_hour=2, tm_min=14, tm_sec=59, tm_wday=0, tm_yday=310, tm_isdst=0) 2023-11-06T02:15:04.730 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-11-06T02:08:25.026811+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=6, tm_hour=2, tm_min=8, tm_sec=25, tm_wday=0, tm_yday=310, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=6, tm_hour=2, tm_min=14, tm_sec=59, tm_wday=0, tm_yday=310, tm_isdst=0) 2023-11-06T02:15:04.730 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-11-06T02:15:22.447 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T02:15:22.452 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T02:15:24.731 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T02:15:24.863 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:24.869+0000 7f5136c8c640 1 Processor -- start 2023-11-06T02:15:24.864 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:24.869+0000 7f5136c8c640 1 -- start start 2023-11-06T02:15:24.866 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:24.869+0000 7f5136c8c640 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f51300b1f90 0x7f51300cda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:24.866 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:24.869+0000 7f5136c8c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5130161650 0x7f51300d1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:24.866 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:24.869+0000 7f5136c8c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f51300d2190 0x7f51300714a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:24.866 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:24.869+0000 7f5136c8c640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51300d17c0 con 0x7f51300d2190 2023-11-06T02:15:24.866 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:24.869+0000 7f5136c8c640 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51300ce6c0 con 0x7f51300b1f90 2023-11-06T02:15:24.867 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:24.869+0000 7f5136c8c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5130161ad0 con 0x7f5130161650 2023-11-06T02:15:24.867 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:24.869+0000 7f5135229640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f51300d2190 0x7f51300714a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:24.867 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:24.869+0000 7f5135229640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f51300d2190 0x7f51300714a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35428/0 (socket says 172.21.15.49:35428) 2023-11-06T02:15:24.867 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:24.869+0000 7f5135229640 1 -- 172.21.15.49:0/180761130 learned_addr learned my addr 172.21.15.49:0/180761130 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:15:24.868 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:24.869+0000 7f5134a28640 1 --2- 172.21.15.49:0/180761130 >> v2:172.21.15.150:3300/0 conn(0x7f51300b1f90 0x7f51300cda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:24.868 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:24.869+0000 7f512ffff640 1 --2- 172.21.15.49:0/180761130 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5130161650 0x7f51300d1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:24.868 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:24.869+0000 7f5135229640 1 -- 172.21.15.49:0/180761130 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5130161650 msgr2=0x7f51300d1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:15:24.868 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:24.869+0000 7f5135229640 1 --2- 172.21.15.49:0/180761130 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5130161650 0x7f51300d1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:24.868 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:24.869+0000 7f5135229640 1 -- 172.21.15.49:0/180761130 >> v2:172.21.15.150:3300/0 conn(0x7f51300b1f90 msgr2=0x7f51300cda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:15:24.869 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:24.869+0000 7f5135229640 1 --2- 172.21.15.49:0/180761130 >> v2:172.21.15.150:3300/0 conn(0x7f51300b1f90 0x7f51300cda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:24.869 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:24.869+0000 7f5135229640 1 -- 172.21.15.49:0/180761130 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51300ce100 con 0x7f51300d2190 2023-11-06T02:15:24.869 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:24.869+0000 7f5134a28640 1 --2- 172.21.15.49:0/180761130 >> v2:172.21.15.150:3300/0 conn(0x7f51300b1f90 0x7f51300cda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:15:24.869 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:24.869+0000 7f512ffff640 1 --2- 172.21.15.49:0/180761130 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5130161650 0x7f51300d1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:15:24.872 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:24.873+0000 7f5135229640 1 --2- 172.21.15.49:0/180761130 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51300d2190 0x7f51300714a0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f5130072180 tx=0x7f512801c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b99c3fea9be82ad server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:15:24.873 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:24.873+0000 7f512f7fe640 1 -- 172.21.15.49:0/180761130 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51280a5730 con 0x7f51300d2190 2023-11-06T02:15:24.873 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:24.873+0000 7f512f7fe640 1 -- 172.21.15.49:0/180761130 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51280a5890 con 0x7f51300d2190 2023-11-06T02:15:24.873 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:24.873+0000 7f512f7fe640 1 -- 172.21.15.49:0/180761130 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51280a5c20 con 0x7f51300d2190 2023-11-06T02:15:24.873 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:24.873+0000 7f5136c8c640 1 -- 172.21.15.49:0/180761130 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51300d2190 msgr2=0x7f51300714a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:24.874 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:24.873+0000 7f5136c8c640 1 --2- 172.21.15.49:0/180761130 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51300d2190 0x7f51300714a0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f5130072180 tx=0x7f512801c9a0 comp rx=0 tx=0).stop 2023-11-06T02:15:24.874 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:24.873+0000 7f5136c8c640 1 -- 172.21.15.49:0/180761130 shutdown_connections 2023-11-06T02:15:24.874 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:24.873+0000 7f5136c8c640 1 --2- 172.21.15.49:0/180761130 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5130161650 0x7f51300d1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:24.874 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:24.873+0000 7f5136c8c640 1 --2- 172.21.15.49:0/180761130 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51300d2190 0x7f51300714a0 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:24.875 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:24.873+0000 7f5136c8c640 1 --2- 172.21.15.49:0/180761130 >> v2:172.21.15.150:3300/0 conn(0x7f51300b1f90 0x7f51300cda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:24.875 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:24.873+0000 7f5136c8c640 1 -- 172.21.15.49:0/180761130 >> 172.21.15.49:0/180761130 conn(0x7f51300d2aa0 msgr2=0x7f51300d2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:15:24.875 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:24.873+0000 7f5136c8c640 1 -- 172.21.15.49:0/180761130 shutdown_connections 2023-11-06T02:15:24.875 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:24.873+0000 7f5136c8c640 1 -- 172.21.15.49:0/180761130 wait complete. 2023-11-06T02:15:24.875 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:24.873+0000 7f5136c8c640 1 Processor -- start 2023-11-06T02:15:24.876 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:24.873+0000 7f5136c8c640 1 -- start start 2023-11-06T02:15:24.876 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:24.877+0000 7f5136c8c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51300b1f90 0x7f513015b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:24.876 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:24.877+0000 7f5136c8c640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51300d2190 0x7f5130158c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:24.876 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:24.877+0000 7f5136c8c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5130161650 0x7f51301591b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:24.876 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:24.877+0000 7f5136c8c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f513015b900 con 0x7f51300b1f90 2023-11-06T02:15:24.877 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:24.877+0000 7f5136c8c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f513015ab40 con 0x7f5130161650 2023-11-06T02:15:24.877 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:24.877+0000 7f5136c8c640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f513015e1f0 con 0x7f51300d2190 2023-11-06T02:15:24.877 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:24.877+0000 7f5134a28640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51300b1f90 0x7f513015b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:24.877 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:24.877+0000 7f5134a28640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51300b1f90 0x7f513015b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35430/0 (socket says 172.21.15.49:35430) 2023-11-06T02:15:24.878 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:24.877+0000 7f5134a28640 1 -- 172.21.15.49:0/634611015 learned_addr learned my addr 172.21.15.49:0/634611015 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:15:24.878 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:24.877+0000 7f5135229640 1 --2- 172.21.15.49:0/634611015 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5130161650 0x7f51301591b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:24.878 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:24.877+0000 7f512ffff640 1 --2- 172.21.15.49:0/634611015 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51300d2190 0x7f5130158c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:24.878 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:24.877+0000 7f5134a28640 1 -- 172.21.15.49:0/634611015 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51300d2190 msgr2=0x7f5130158c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:24.879 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:24.877+0000 7f5134a28640 1 --2- 172.21.15.49:0/634611015 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51300d2190 0x7f5130158c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:24.879 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:24.877+0000 7f5134a28640 1 -- 172.21.15.49:0/634611015 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5130161650 msgr2=0x7f51301591b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:24.879 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:24.877+0000 7f5134a28640 1 --2- 172.21.15.49:0/634611015 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5130161650 0x7f51301591b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:24.879 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:24.877+0000 7f5134a28640 1 -- 172.21.15.49:0/634611015 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5128009b30 con 0x7f51300b1f90 2023-11-06T02:15:24.879 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:24.877+0000 7f5134a28640 1 --2- 172.21.15.49:0/634611015 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51300b1f90 0x7f513015b3c0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f512000c7c0 tx=0x7f512000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:15:24.880 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:24.877+0000 7f512d7fa640 1 -- 172.21.15.49:0/634611015 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5120007a30 con 0x7f51300b1f90 2023-11-06T02:15:24.880 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:24.877+0000 7f512d7fa640 1 -- 172.21.15.49:0/634611015 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5120007b90 con 0x7f51300b1f90 2023-11-06T02:15:24.880 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:24.877+0000 7f5136c8c640 1 -- 172.21.15.49:0/634611015 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51300cc7b0 con 0x7f51300b1f90 2023-11-06T02:15:24.880 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:24.877+0000 7f512d7fa640 1 -- 172.21.15.49:0/634611015 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512000ce70 con 0x7f51300b1f90 2023-11-06T02:15:24.880 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:24.877+0000 7f5136c8c640 1 -- 172.21.15.49:0/634611015 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f513015ff50 con 0x7f51300b1f90 2023-11-06T02:15:24.881 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:24.877+0000 7f5136c8c640 1 -- 172.21.15.49:0/634611015 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5130074040 con 0x7f51300b1f90 2023-11-06T02:15:24.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:24.877+0000 7f512d7fa640 1 -- 172.21.15.49:0/634611015 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f51200113f0 con 0x7f51300b1f90 2023-11-06T02:15:24.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:24.877+0000 7f512d7fa640 1 --2- 172.21.15.49:0/634611015 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f511c065a60 0x7f511c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:24.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:24.877+0000 7f512d7fa640 1 -- 172.21.15.49:0/634611015 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f5120083220 con 0x7f51300b1f90 2023-11-06T02:15:24.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:24.877+0000 7f512ffff640 1 --2- 172.21.15.49:0/634611015 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f511c065a60 0x7f511c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:24.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:24.881+0000 7f512ffff640 1 --2- 172.21.15.49:0/634611015 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f511c065a60 0x7f511c067f20 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f5124006fd0 tx=0x7f5124008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:15:24.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:24.881+0000 7f512d7fa640 1 -- 172.21.15.49:0/634611015 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f5120050cd0 con 0x7f51300b1f90 2023-11-06T02:15:25.031 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.037+0000 7f5136c8c640 1 -- 172.21.15.49:0/634611015 --> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f51300b4520 con 0x7f511c065a60 2023-11-06T02:15:25.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.041+0000 7f512d7fa640 1 -- 172.21.15.49:0/634611015 <== mgr.4100 v2:172.21.15.150:6832/38104 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+71124 (secure 0 0 0) 0x7f512400c050 con 0x7f511c065a60 2023-11-06T02:15:25.039 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:15:25.040 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2023-11-06T02:15:25.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.045+0000 7f5136c8c640 1 -- 172.21.15.49:0/634611015 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f511c065a60 msgr2=0x7f511c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:25.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.045+0000 7f5136c8c640 1 --2- 172.21.15.49:0/634611015 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f511c065a60 0x7f511c067f20 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f5124006fd0 tx=0x7f5124008040 comp rx=0 tx=0).stop 2023-11-06T02:15:25.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.045+0000 7f5136c8c640 1 -- 172.21.15.49:0/634611015 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51300b1f90 msgr2=0x7f513015b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:25.042 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.045+0000 7f5136c8c640 1 --2- 172.21.15.49:0/634611015 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51300b1f90 0x7f513015b3c0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f512000c7c0 tx=0x7f512000cc80 comp rx=0 tx=0).stop 2023-11-06T02:15:25.042 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.045+0000 7f5136c8c640 1 -- 172.21.15.49:0/634611015 shutdown_connections 2023-11-06T02:15:25.042 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.045+0000 7f5136c8c640 1 --2- 172.21.15.49:0/634611015 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f511c065a60 0x7f511c067f20 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:25.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.045+0000 7f5136c8c640 1 --2- 172.21.15.49:0/634611015 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5130161650 0x7f51301591b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:25.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.045+0000 7f5136c8c640 1 --2- 172.21.15.49:0/634611015 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51300d2190 0x7f5130158c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:25.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.045+0000 7f5136c8c640 1 --2- 172.21.15.49:0/634611015 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51300b1f90 0x7f513015b3c0 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:25.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.045+0000 7f5136c8c640 1 -- 172.21.15.49:0/634611015 >> 172.21.15.49:0/634611015 conn(0x7f51300d2aa0 msgr2=0x7f51300b2980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:15:25.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.045+0000 7f5136c8c640 1 -- 172.21.15.49:0/634611015 shutdown_connections 2023-11-06T02:15:25.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.045+0000 7f5136c8c640 1 -- 172.21.15.49:0/634611015 wait complete. 2023-11-06T02:15:25.058 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":291,"stamp":"2023-11-06T02:15:23.323800+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":261796,"kb_used_data":9316,"kb_used_omap":0,"kb_used_meta":252416,"kb_avail":754712924,"statfs":{"total":773094113280,"available":772826034176,"internally_reserved":0,"allocated":9539584,"data_stored":4952261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":258473984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003949"},"pg_stats":[{"pgid":"1.0","version":"85'119","reported_seq":238,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T02:15:06.880310+0000","last_change":"2023-11-06T02:15:06.880591+0000","last_active":"2023-11-06T02:15:06.880310+0000","last_peered":"2023-11-06T02:15:06.880310+0000","last_clean":"2023-11-06T02:15:06.880310+0000","last_became_active":"2023-11-06T02:08:20.521089+0000","last_became_peered":"2023-11-06T02:08:20.521089+0000","last_unstale":"2023-11-06T02:15:06.880310+0000","last_undegraded":"2023-11-06T02:15:06.880310+0000","last_fullsized":"2023-11-06T02:15:06.880310+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'119","last_scrub_stamp":"2023-11-06T02:15:06.880258+0000","last_deep_scrub":"85'119","last_deep_scrub_stamp":"2023-11-06T02:15:06.880258+0000","last_clean_scrub_stamp":"2023-11-06T02:15:06.880258+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:53:30.766452+0000","scrub_duration":0.011430617000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":82,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T02:15:13.340822+0000","last_change":"2023-11-06T02:15:13.341075+0000","last_active":"2023-11-06T02:15:13.340822+0000","last_peered":"2023-11-06T02:15:13.340822+0000","last_clean":"2023-11-06T02:15:13.340822+0000","last_became_active":"2023-11-06T02:08:26.058612+0000","last_became_peered":"2023-11-06T02:08:26.058612+0000","last_unstale":"2023-11-06T02:15:13.340822+0000","last_undegraded":"2023-11-06T02:15:13.340822+0000","last_fullsized":"2023-11-06T02:15:13.340822+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:15:13.340766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:15:13.340766+0000","last_clean_scrub_stamp":"2023-11-06T02:15:13.340766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:19:00.069381+0000","scrub_duration":0.0053619740000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":83,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T02:15:11.954992+0000","last_change":"2023-11-06T02:15:11.955252+0000","last_active":"2023-11-06T02:15:11.954992+0000","last_peered":"2023-11-06T02:15:11.954992+0000","last_clean":"2023-11-06T02:15:11.954992+0000","last_became_active":"2023-11-06T02:08:26.059262+0000","last_became_peered":"2023-11-06T02:08:26.059262+0000","last_unstale":"2023-11-06T02:15:11.954992+0000","last_undegraded":"2023-11-06T02:15:11.954992+0000","last_fullsized":"2023-11-06T02:15:11.954992+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:15:11.954937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:15:11.954937+0000","last_clean_scrub_stamp":"2023-11-06T02:15:11.954937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:11:00.333600+0000","scrub_duration":0.0054395679999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":83,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T02:15:03.340958+0000","last_change":"2023-11-06T02:15:03.341048+0000","last_active":"2023-11-06T02:15:03.340958+0000","last_peered":"2023-11-06T02:15:03.340958+0000","last_clean":"2023-11-06T02:15:03.340958+0000","last_became_active":"2023-11-06T02:08:26.559202+0000","last_became_peered":"2023-11-06T02:08:26.559202+0000","last_unstale":"2023-11-06T02:15:03.340958+0000","last_undegraded":"2023-11-06T02:15:03.340958+0000","last_fullsized":"2023-11-06T02:15:03.340958+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:15:03.340941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:15:03.340941+0000","last_clean_scrub_stamp":"2023-11-06T02:15:03.340941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:09:06.936451+0000","scrub_duration":0.0023469540000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":82,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T02:15:11.379482+0000","last_change":"2023-11-06T02:15:11.379740+0000","last_active":"2023-11-06T02:15:11.379482+0000","last_peered":"2023-11-06T02:15:11.379482+0000","last_clean":"2023-11-06T02:15:11.379482+0000","last_became_active":"2023-11-06T02:08:26.058806+0000","last_became_peered":"2023-11-06T02:08:26.058806+0000","last_unstale":"2023-11-06T02:15:11.379482+0000","last_undegraded":"2023-11-06T02:15:11.379482+0000","last_fullsized":"2023-11-06T02:15:11.379482+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:15:11.379433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:15:11.379433+0000","last_clean_scrub_stamp":"2023-11-06T02:15:11.379433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:10:59.758096+0000","scrub_duration":0.0056893539999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":82,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T02:15:06.764843+0000","last_change":"2023-11-06T02:15:06.765137+0000","last_active":"2023-11-06T02:15:06.764843+0000","last_peered":"2023-11-06T02:15:06.764843+0000","last_clean":"2023-11-06T02:15:06.764843+0000","last_became_active":"2023-11-06T02:08:26.436075+0000","last_became_peered":"2023-11-06T02:08:26.436075+0000","last_unstale":"2023-11-06T02:15:06.764843+0000","last_undegraded":"2023-11-06T02:15:06.764843+0000","last_fullsized":"2023-11-06T02:15:06.764843+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:15:06.764789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:15:06.764789+0000","last_clean_scrub_stamp":"2023-11-06T02:15:06.764789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:33:50.254917+0000","scrub_duration":0.004982149,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":83,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T02:15:12.909185+0000","last_change":"2023-11-06T02:15:12.909494+0000","last_active":"2023-11-06T02:15:12.909185+0000","last_peered":"2023-11-06T02:15:12.909185+0000","last_clean":"2023-11-06T02:15:12.909185+0000","last_became_active":"2023-11-06T02:08:26.435924+0000","last_became_peered":"2023-11-06T02:08:26.435924+0000","last_unstale":"2023-11-06T02:15:12.909185+0000","last_undegraded":"2023-11-06T02:15:12.909185+0000","last_fullsized":"2023-11-06T02:15:12.909185+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:15:12.909127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:15:12.909127+0000","last_clean_scrub_stamp":"2023-11-06T02:15:12.909127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:18:59.637742+0000","scrub_duration":0.0057751670000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":82,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T02:15:05.775735+0000","last_change":"2023-11-06T02:15:05.776028+0000","last_active":"2023-11-06T02:15:05.775735+0000","last_peered":"2023-11-06T02:15:05.775735+0000","last_clean":"2023-11-06T02:15:05.775735+0000","last_became_active":"2023-11-06T02:08:26.056419+0000","last_became_peered":"2023-11-06T02:08:26.056419+0000","last_unstale":"2023-11-06T02:15:05.775735+0000","last_undegraded":"2023-11-06T02:15:05.775735+0000","last_fullsized":"2023-11-06T02:15:05.775735+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:15:05.775685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:15:05.775685+0000","last_clean_scrub_stamp":"2023-11-06T02:15:05.775685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:24:42.490480+0000","scrub_duration":0.0059659070000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":91,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T02:15:07.328421+0000","last_change":"2023-11-06T02:15:07.328668+0000","last_active":"2023-11-06T02:15:07.328421+0000","last_peered":"2023-11-06T02:15:07.328421+0000","last_clean":"2023-11-06T02:15:07.328421+0000","last_became_active":"2023-11-06T02:08:26.057015+0000","last_became_peered":"2023-11-06T02:08:26.057015+0000","last_unstale":"2023-11-06T02:15:07.328421+0000","last_undegraded":"2023-11-06T02:15:07.328421+0000","last_fullsized":"2023-11-06T02:15:07.328421+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T02:15:07.328368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T02:15:07.328368+0000","last_clean_scrub_stamp":"2023-11-06T02:15:07.328368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:24:44.043163+0000","scrub_duration":0.0053440069999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574936,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32648,"kb_used_data":1600,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339192,"statfs":{"total":96636764160,"available":96603332608,"internally_reserved":0,"allocated":1638400,"data_stored":1061936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.87,"15min":0.85599999999999998},"min":{"1min":0.62,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.147,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.52700000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.2230000000000001,"5min":1.421,"15min":1.506},"last":0.89700000000000002}]},{"osd":1,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.83399999999999996,"15min":0.84499999999999997},"min":{"1min":0.498,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.3320000000000001,"5min":1.345,"15min":1.345},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.92900000000000005,"15min":0.90200000000000002},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.54900000000000004},"max":{"1min":1.131,"5min":1.681,"15min":1.681},"last":0.98399999999999999}]},{"osd":2,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.96399999999999997,"15min":0.95699999999999996},"min":{"1min":0.65300000000000002,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.1919999999999999,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":1.056},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.92200000000000004,"15min":0.97399999999999998},"min":{"1min":0.57399999999999995,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.1919999999999999,"5min":1.5,"15min":1.5920000000000001},"last":0.70899999999999996}]},{"osd":3,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.89200000000000002,"15min":0.88},"min":{"1min":0.66100000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.1519999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.995,"5min":1.03,"15min":1.0609999999999999},"min":{"1min":0.74399999999999999,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.2110000000000001,"5min":1.526,"15min":1.526},"last":1.103}]},{"osd":4,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.91700000000000004,"15min":0.94399999999999995},"min":{"1min":0.73399999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.1930000000000001,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.97999999999999998,"15min":1.0229999999999999},"min":{"1min":0.65000000000000002,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.1519999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.83099999999999996}]},{"osd":5,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":1.026,"15min":1.0609999999999999},"min":{"1min":0.69199999999999995,"5min":0.503,"15min":0.503},"max":{"1min":1.2629999999999999,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":0.85099999999999998},{"interface":"front","average":{"1min":1.044,"5min":1.044,"15min":1.105},"min":{"1min":0.71299999999999997,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.288,"5min":1.3700000000000001,"15min":1.7809999999999999},"last":0.79800000000000004}]},{"osd":6,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":1.1180000000000001,"15min":1.1739999999999999},"min":{"1min":0.89500000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.274,"5min":1.5009999999999999,"15min":1.6679999999999999},"last":1.008},{"interface":"front","average":{"1min":1.028,"5min":1.0649999999999999,"15min":1.127},"min":{"1min":0.72599999999999998,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.2509999999999999,"5min":1.4570000000000001,"15min":1.74},"last":1.02}]}]},{"osd":6,"up_from":13,"seq":55834574934,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32068,"kb_used_data":1020,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339772,"statfs":{"total":96636764160,"available":96603926528,"internally_reserved":0,"allocated":1044480,"data_stored":471568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.872,"15min":0.86199999999999999},"min":{"1min":0.57199999999999995,"5min":0.495,"15min":0.495},"max":{"1min":1.226,"5min":1.226,"15min":1.2609999999999999},"last":1.0529999999999999},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.88600000000000001,"15min":0.83199999999999996},"min":{"1min":0.5,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.3280000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.64800000000000002}]},{"osd":1,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.85099999999999998,"15min":0.86699999999999999},"min":{"1min":0.54400000000000004,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.1639999999999999,"5min":1.244,"15min":1.244},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.90500000000000003,"15min":0.88400000000000001},"min":{"1min":0.56599999999999995,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":1.1379999999999999}]},{"osd":2,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.89200000000000002,"15min":0.89600000000000002},"min":{"1min":0.61499999999999999,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.877,"5min":0.88900000000000001,"15min":0.91700000000000004},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.075,"5min":1.411,"15min":1.411},"last":0.96799999999999997}]},{"osd":3,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.93500000000000005,"15min":0.95899999999999996},"min":{"1min":0.74099999999999999,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":1.1879999999999999},{"interface":"front","average":{"1min":1.0309999999999999,"5min":0.999,"15min":1},"min":{"1min":0.81599999999999995,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.2609999999999999,"5min":1.381,"15min":1.381},"last":0.93600000000000005}]},{"osd":4,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.92200000000000004,"15min":0.96999999999999997},"min":{"1min":0.627,"5min":0.56799999999999995,"15min":0.52900000000000003},"max":{"1min":1.2390000000000001,"5min":1.302,"15min":1.474},"last":1.258},{"interface":"front","average":{"1min":0.997,"5min":0.97399999999999998,"15min":1.018},"min":{"1min":0.66700000000000004,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.4039999999999999,"5min":1.556,"15min":1.556},"last":1.0049999999999999}]},{"osd":5,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":0.97199999999999998,"15min":1.0089999999999999},"min":{"1min":0.68899999999999995,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.89000000000000001},{"interface":"front","average":{"1min":1.1319999999999999,"5min":1.075,"15min":1.101},"min":{"1min":0.82699999999999996,"5min":0.66900000000000004,"15min":0.58699999999999997},"max":{"1min":1.3999999999999999,"5min":1.587,"15min":1.587},"last":1.3100000000000001}]},{"osd":7,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.087,"15min":1.107},"min":{"1min":0.88300000000000001,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.5369999999999999,"5min":1.615,"15min":1.615},"last":1.216},{"interface":"front","average":{"1min":1.103,"5min":1.0509999999999999,"15min":1.127},"min":{"1min":0.85599999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.4610000000000001,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":1.0980000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574936,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32068,"kb_used_data":1020,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339772,"statfs":{"total":96636764160,"available":96603926528,"internally_reserved":0,"allocated":1044480,"data_stored":471568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:14:33 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.86199999999999999,"15min":0.83999999999999997},"min":{"1min":0.54400000000000004,"5min":0.51700000000000002,"15min":0.47299999999999998},"max":{"1min":1.079,"5min":1.381,"15min":1.381},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.86899999999999999,"15min":0.85099999999999998},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.159,"5min":1.3200000000000001,"15min":1.6579999999999999},"last":0.66400000000000003}]},{"osd":1,"last update":"Mon Nov 6 02:14:33 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.90500000000000003,"15min":0.86799999999999999},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.60299999999999998},"max":{"1min":1.369,"5min":1.369,"15min":1.369},"last":1.069},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.90000000000000002,"15min":0.84399999999999997},"min":{"1min":0.53700000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.1639999999999999,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.68799999999999994}]},{"osd":2,"last update":"Mon Nov 6 02:14:33 2023","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.93899999999999995,"15min":0.94299999999999995},"min":{"1min":0.65100000000000002,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.242,"5min":1.298,"15min":1.3560000000000001},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.89700000000000002,"15min":0.82499999999999996},"min":{"1min":0.58699999999999997,"5min":0.55800000000000005,"15min":0.47899999999999998},"max":{"1min":1.218,"5min":1.323,"15min":1.377},"last":1.044}]},{"osd":3,"last update":"Mon Nov 6 02:14:33 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.95199999999999996,"15min":0.93000000000000005},"min":{"1min":0.66400000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.292,"5min":1.3580000000000001,"15min":1.4399999999999999},"last":0.82099999999999995},{"interface":"front","average":{"1min":1.0369999999999999,"5min":0.98799999999999999,"15min":1.05},"min":{"1min":0.76700000000000002,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.2569999999999999,"5min":1.349,"15min":1.7250000000000001},"last":1.016}]},{"osd":4,"last update":"Mon Nov 6 02:14:33 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.87,"15min":0.90000000000000002},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.0129999999999999,"5min":1.276,"15min":1.276},"last":1.105},{"interface":"front","average":{"1min":0.995,"5min":0.98599999999999999,"15min":1.014},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.496,"5min":1.496,"15min":1.554},"last":0.75}]},{"osd":6,"last update":"Mon Nov 6 02:14:33 2023","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.98699999999999999,"15min":1.0409999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.3440000000000001,"5min":1.454,"15min":1.454},"last":0.85599999999999998},{"interface":"front","average":{"1min":1.046,"5min":1.0229999999999999,"15min":1.0349999999999999},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":2.2309999999999999,"5min":2.2309999999999999,"15min":2.2309999999999999},"last":1.1539999999999999}]},{"osd":7,"last update":"Mon Nov 6 02:14:33 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.0640000000000001,"15min":1.0720000000000001},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.5209999999999999,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":0.98899999999999999},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.117,"15min":1.1419999999999999},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":2.3759999999999999,"5min":2.3759999999999999,"15min":2.3759999999999999},"last":1.125}]}]},{"osd":4,"up_from":13,"seq":55834574936,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32068,"kb_used_data":1020,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339772,"statfs":{"total":96636764160,"available":96603926528,"internally_reserved":0,"allocated":1044480,"data_stored":471568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:14:27 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.86399999999999999,"15min":0.83799999999999997},"min":{"1min":0.53500000000000003,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.2450000000000001,"5min":1.343,"15min":1.343},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.84499999999999997,"15min":0.79500000000000004},"min":{"1min":0.51600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.175,"5min":1.28,"15min":1.3049999999999999},"last":1.002}]},{"osd":1,"last update":"Mon Nov 6 02:14:27 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.86399999999999999,"15min":0.84999999999999998},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.508,"5min":1.508,"15min":1.508},"last":0.94799999999999995},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.91800000000000004,"15min":0.90000000000000002},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.1080000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.82499999999999996}]},{"osd":2,"last update":"Mon Nov 6 02:14:27 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.90800000000000003,"15min":0.98199999999999998},"min":{"1min":0.57999999999999996,"5min":0.503,"15min":0.503},"max":{"1min":1.0880000000000001,"5min":1.389,"15min":1.615},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.95499999999999996,"15min":0.93500000000000005},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.57899999999999996},"max":{"1min":1.1559999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":1.113}]},{"osd":3,"last update":"Mon Nov 6 02:14:27 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.86199999999999999,"15min":0.90000000000000002},"min":{"1min":0.59899999999999998,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.22,"5min":1.22,"15min":1.52},"last":1.0820000000000001},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.97299999999999998,"15min":1.0369999999999999},"min":{"1min":0.68100000000000005,"5min":0.63900000000000001,"15min":0.59999999999999998},"max":{"1min":1.589,"5min":1.595,"15min":1.595},"last":1.1679999999999999}]},{"osd":5,"last update":"Mon Nov 6 02:14:27 2023","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":1.026},"min":{"1min":0.72599999999999998,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.212,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.96099999999999997,"15min":1.024},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.629},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.7869999999999999},"last":1.141}]},{"osd":6,"last update":"Mon Nov 6 02:14:27 2023","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.008,"15min":1.0649999999999999},"min":{"1min":0.77600000000000002,"5min":0.68000000000000005,"15min":0.67100000000000004},"max":{"1min":1.6679999999999999,"5min":1.6679999999999999,"15min":1.6699999999999999},"last":1.0660000000000001},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.085,"15min":1.1599999999999999},"min":{"1min":0.71999999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.2210000000000001,"5min":1.5900000000000001,"15min":2.1190000000000002},"last":1.04}]},{"osd":7,"last update":"Mon Nov 6 02:14:27 2023","interfaces":[{"interface":"back","average":{"1min":1.056,"5min":1.042,"15min":1.0760000000000001},"min":{"1min":0.77100000000000002,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.7110000000000001},"last":1.198},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.117,"15min":1.1990000000000001},"min":{"1min":0.90500000000000003,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.754,"5min":1.754,"15min":1.754},"last":1.0269999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574935,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32068,"kb_used_data":1020,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339772,"statfs":{"total":96636764160,"available":96603926528,"internally_reserved":0,"allocated":1044480,"data_stored":471568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63,"15min":0.64200000000000002},"min":{"1min":0.45300000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":1.171,"5min":1.198,"15min":1.198},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.65400000000000003,"15min":0.63500000000000001},"min":{"1min":0.49099999999999999,"5min":0.29899999999999999,"15min":0.29799999999999999},"max":{"1min":1.1819999999999999,"5min":1.21,"15min":1.21},"last":0.81100000000000005}]},{"osd":1,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.61599999999999999,"15min":0.626},"min":{"1min":0.42499999999999999,"5min":0.248,"15min":0.248},"max":{"1min":0.86799999999999999,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.75800000000000001,"15min":0.74199999999999999},"min":{"1min":0.46899999999999997,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.175,"5min":1.204,"15min":1.204},"last":0.90700000000000003}]},{"osd":2,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.72499999999999998,"15min":0.73099999999999998},"min":{"1min":0.499,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.98999999999999999},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.81299999999999994,"15min":0.81999999999999995},"min":{"1min":0.62,"5min":0.308,"15min":0.308},"max":{"1min":1.0329999999999999,"5min":1.405,"15min":1.405},"last":1.1220000000000001}]},{"osd":4,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.62,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.262,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":1.1419999999999999},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.93700000000000006,"15min":0.97799999999999998},"min":{"1min":0.70799999999999996,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.212,"5min":1.292,"15min":1.4410000000000001},"last":0.96299999999999997}]},{"osd":5,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.97499999999999998,"15min":1.0109999999999999},"min":{"1min":0.72699999999999998,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.21,"5min":1.375,"15min":1.375},"last":0.88300000000000001},{"interface":"front","average":{"1min":1.024,"5min":1.0269999999999999,"15min":1.0629999999999999},"min":{"1min":0.65200000000000002,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.323,"5min":1.4890000000000001,"15min":2.21},"last":1.3440000000000001}]},{"osd":6,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.0209999999999999,"15min":1.024},"min":{"1min":0.79800000000000004,"5min":0.439,"15min":0.439},"max":{"1min":1.3080000000000001,"5min":1.395,"15min":1.395},"last":1.2010000000000001},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.0269999999999999,"15min":1.0569999999999999},"min":{"1min":0.74099999999999999,"5min":0.375,"15min":0.375},"max":{"1min":1.2709999999999999,"5min":1.4319999999999999,"15min":1.554},"last":1.046}]},{"osd":7,"last update":"Mon Nov 6 02:14:29 2023","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.1020000000000001,"15min":1.1220000000000001},"min":{"1min":0.68200000000000005,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.4410000000000001,"5min":1.5549999999999999,"15min":2.2829999999999999},"last":1.4430000000000001},{"interface":"front","average":{"1min":1.123,"5min":1.0980000000000001,"15min":1.1040000000000001},"min":{"1min":0.91100000000000003,"5min":0.497,"15min":0.497},"max":{"1min":1.3620000000000001,"5min":1.504,"15min":1.504},"last":1.3600000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574935,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32068,"kb_used_data":1020,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339772,"statfs":{"total":96636764160,"available":96603926528,"internally_reserved":0,"allocated":1044480,"data_stored":471568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:14:28 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.59099999999999997,"15min":0.52900000000000003},"min":{"1min":0.28499999999999998,"5min":0.252,"15min":0.252},"max":{"1min":0.90300000000000002,"5min":1.079,"15min":1.079},"last":0.94899999999999995},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.63400000000000001,"15min":0.56999999999999995},"min":{"1min":0.374,"5min":0.316,"15min":0.27300000000000002},"max":{"1min":0.86699999999999999,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":0.57199999999999995}]},{"osd":1,"last update":"Mon Nov 6 02:14:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.65000000000000002,"15min":0.58999999999999997},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.26400000000000001},"max":{"1min":0.97699999999999998,"5min":1.48,"15min":1.48},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.72799999999999998,"15min":0.621},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.26500000000000001},"max":{"1min":1.04,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":0.66200000000000003}]},{"osd":3,"last update":"Mon Nov 6 02:14:28 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.71399999999999997,"15min":0.63600000000000001},"min":{"1min":0.316,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.0580000000000001,"5min":1.121,"15min":1.155},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.76400000000000001,"15min":0.68200000000000005},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.069,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":1.127}]},{"osd":4,"last update":"Mon Nov 6 02:14:28 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.93100000000000005,"15min":0.84299999999999997},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.1539999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":1.05},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.92900000000000005,"15min":0.91900000000000004},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.234,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":1.0149999999999999}]},{"osd":5,"last update":"Mon Nov 6 02:14:28 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.96299999999999997,"15min":0.94899999999999995},"min":{"1min":0.56299999999999994,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.258,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":0.995},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.97299999999999998,"15min":0.94699999999999995},"min":{"1min":0.626,"5min":0.60999999999999999,"15min":0.52600000000000002},"max":{"1min":1.2310000000000001,"5min":1.615,"15min":1.7110000000000001},"last":1.1930000000000001}]},{"osd":6,"last update":"Mon Nov 6 02:14:28 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.96899999999999997,"15min":0.94799999999999995},"min":{"1min":0.60199999999999998,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.218,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":1.101},{"interface":"front","average":{"1min":0.93600000000000005,"5min":1.01,"15min":1.0009999999999999},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.274,"5min":1.675,"15min":1.677},"last":1.165}]},{"osd":7,"last update":"Mon Nov 6 02:14:28 2023","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":1.0309999999999999,"15min":0.99299999999999999},"min":{"1min":0.67500000000000004,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.2869999999999999,"5min":1.659,"15min":1.659},"last":1.2150000000000001},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.0760000000000001,"15min":1.0529999999999999},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.1879999999999999,"5min":1.4650000000000001,"15min":1.5980000000000001},"last":1.0800000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36164,"kb_used_data":1020,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335676,"statfs":{"total":96636764160,"available":96599732224,"internally_reserved":0,"allocated":1044480,"data_stored":471568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 02:14:28 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.628,"15min":0.626},"min":{"1min":0.29599999999999999,"5min":0.28699999999999998,"15min":0.224},"max":{"1min":0.95099999999999996,"5min":1.603,"15min":1.603},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.71999999999999997,"15min":0.67400000000000004},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.081,"5min":2.161,"15min":2.161},"last":0.76400000000000001}]},{"osd":2,"last update":"Mon Nov 6 02:14:28 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.64700000000000002,"15min":0.63},"min":{"1min":0.34399999999999997,"5min":0.253,"15min":0.253},"max":{"1min":1.036,"5min":1.649,"15min":1.649},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.74199999999999999,"15min":0.74399999999999999},"min":{"1min":0.373,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.91700000000000004,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":1.042}]},{"osd":3,"last update":"Mon Nov 6 02:14:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.80400000000000005,"15min":0.80300000000000005},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.131,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.79100000000000004,"15min":0.80400000000000005},"min":{"1min":0.35499999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.94199999999999995,"5min":1.5,"15min":1.8149999999999999},"last":0.98199999999999998}]},{"osd":4,"last update":"Mon Nov 6 02:14:28 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.96199999999999997,"15min":0.98199999999999998},"min":{"1min":0.55600000000000005,"5min":0.504,"15min":0.504},"max":{"1min":1.2110000000000001,"5min":2.2759999999999998,"15min":2.2759999999999998},"last":1.0269999999999999},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.95599999999999996,"15min":0.98099999999999998},"min":{"1min":0.63300000000000001,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.0700000000000001,"5min":2.9180000000000001,"15min":2.9180000000000001},"last":1.0149999999999999}]},{"osd":5,"last update":"Mon Nov 6 02:14:28 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.97199999999999998,"15min":0.96799999999999997},"min":{"1min":0.65800000000000003,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.177,"5min":2.7469999999999999,"15min":2.7469999999999999},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.98399999999999999,"5min":1.05,"15min":1.069},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.52100000000000002},"max":{"1min":1.3720000000000001,"5min":1.921,"15min":1.921},"last":1.141}]},{"osd":6,"last update":"Mon Nov 6 02:14:28 2023","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":1.0960000000000001,"15min":1.1000000000000001},"min":{"1min":0.61899999999999999,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.304,"5min":3.5710000000000002,"15min":3.5710000000000002},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.94999999999999996,"5min":1.0369999999999999,"15min":1.0629999999999999},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.268,"5min":2.843,"15min":2.843},"last":0.95199999999999996}]},{"osd":7,"last update":"Mon Nov 6 02:14:28 2023","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":1.0980000000000001,"15min":1.1140000000000001},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.4159999999999999,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":1.2190000000000001},{"interface":"front","average":{"1min":1.052,"5min":1.1419999999999999,"15min":1.1539999999999999},"min":{"1min":0.61799999999999999,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.3520000000000001,"5min":3.4790000000000001,"15min":3.4790000000000001},"last":1.085}]}]},{"osd":0,"up_from":13,"seq":55834574936,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32644,"kb_used_data":1596,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339196,"statfs":{"total":96636764160,"available":96603336704,"internally_reserved":0,"allocated":1634304,"data_stored":1060917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 02:14:27 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.62,"15min":0.60499999999999998},"min":{"1min":0.32500000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.94899999999999995,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.62,"5min":0.66600000000000004,"15min":0.64600000000000002},"min":{"1min":0.26100000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.96599999999999997,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.77700000000000002}]},{"osd":2,"last update":"Mon Nov 6 02:14:27 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.68799999999999994,"15min":0.67200000000000004},"min":{"1min":0.36599999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.0900000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":1.038},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.69699999999999995,"15min":0.68200000000000005},"min":{"1min":0.315,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.98199999999999998,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.99299999999999999}]},{"osd":3,"last update":"Mon Nov 6 02:14:27 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.69499999999999995,"15min":0.65100000000000002},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":2.1150000000000002,"5min":2.1150000000000002,"15min":2.1150000000000002},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.78500000000000003,"15min":0.749},"min":{"1min":0.39300000000000002,"5min":0.317,"15min":0.317},"max":{"1min":1.0860000000000001,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":1.224}]},{"osd":4,"last update":"Mon Nov 6 02:14:27 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.93000000000000005,"15min":0.86099999999999999},"min":{"1min":0.60099999999999998,"5min":0.55400000000000005,"15min":0.54300000000000004},"max":{"1min":2.3130000000000002,"5min":2.3130000000000002,"15min":2.3130000000000002},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.96299999999999997,"15min":0.94499999999999995},"min":{"1min":0.59599999999999997,"5min":0.5,"15min":0.5},"max":{"1min":1.3140000000000001,"5min":1.3839999999999999,"15min":1.387},"last":1.161}]},{"osd":5,"last update":"Mon Nov 6 02:14:27 2023","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.95999999999999996,"15min":0.95199999999999996},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":1.091},{"interface":"front","average":{"1min":1.006,"5min":0.96099999999999997,"15min":0.90100000000000002},"min":{"1min":0.53800000000000003,"5min":0.496,"15min":0.496},"max":{"1min":2.3530000000000002,"5min":2.3530000000000002,"15min":2.3530000000000002},"last":1.1200000000000001}]},{"osd":6,"last update":"Mon Nov 6 02:14:27 2023","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":1.0349999999999999,"15min":1.036},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.4550000000000001,"5min":1.581,"15min":1.589},"last":1.2909999999999999},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.0309999999999999,"15min":1.01},"min":{"1min":0.68100000000000005,"5min":0.65800000000000003,"15min":0.57099999999999995},"max":{"1min":1.4990000000000001,"5min":1.5529999999999999,"15min":1.6220000000000001},"last":1.3169999999999999}]},{"osd":7,"last update":"Mon Nov 6 02:14:27 2023","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":1.024,"15min":1.02},"min":{"1min":0.55300000000000005,"5min":0.504,"15min":0.504},"max":{"1min":1.357,"5min":1.504,"15min":1.504},"last":1.1970000000000001},{"interface":"front","average":{"1min":1.034,"5min":1.0029999999999999,"15min":1.0069999999999999},"min":{"1min":0.64700000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":2.387,"5min":2.387,"15min":2.387},"last":1.25}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T02:15:25.061 DEBUG:teuthology.orchestra.run.smithi049:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-11-06T02:15:25.182 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.185+0000 7f90bd01b640 1 Processor -- start 2023-11-06T02:15:25.183 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.185+0000 7f90bd01b640 1 -- start start 2023-11-06T02:15:25.183 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.185+0000 7f90bd01b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f90b8070870 0x7f90b80b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:25.183 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.185+0000 7f90bd01b640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f90b806d150 0x7f90b806d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:25.183 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.185+0000 7f90bd01b640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90b806db20 con 0x7f90b815a880 2023-11-06T02:15:25.184 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.185+0000 7f90bd01b640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90b80ae360 con 0x7f90b806d150 2023-11-06T02:15:25.184 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.185+0000 7f90bd01b640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90b80ae640 con 0x7f90b8070870 2023-11-06T02:15:25.184 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.185+0000 7f90b759e640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f90b806d150 0x7f90b806d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:25.184 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.185+0000 7f90b759e640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f90b806d150 0x7f90b806d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35444/0 (socket says 172.21.15.49:35444) 2023-11-06T02:15:25.184 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.185+0000 7f90b759e640 1 -- 172.21.15.49:0/3016601870 learned_addr learned my addr 172.21.15.49:0/3016601870 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:15:25.185 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.185+0000 7f90b6d9d640 1 --1- 172.21.15.49:0/3016601870 >> v1:172.21.15.49:6789/0 conn(0x7f90b815a880 0x7f90b8070160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:51490/0 (socket says 172.21.15.49:51490) 2023-11-06T02:15:25.185 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.185+0000 7f90b659c640 1 --2- 172.21.15.49:0/3016601870 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f90b8070870 0x7f90b80b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:25.185 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.185+0000 7f90b759e640 1 -- 172.21.15.49:0/3016601870 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f90b8070870 msgr2=0x7f90b80b15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:15:25.185 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.185+0000 7f90b759e640 1 --2- 172.21.15.49:0/3016601870 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f90b8070870 0x7f90b80b15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:25.186 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.185+0000 7f90b759e640 1 -- 172.21.15.49:0/3016601870 >> v1:172.21.15.49:6789/0 conn(0x7f90b815a880 legacy=0x7f90b8070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:25.186 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.185+0000 7f90b759e640 1 -- 172.21.15.49:0/3016601870 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90b80ae920 con 0x7f90b806d150 2023-11-06T02:15:25.186 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.185+0000 7f90b5d9b640 1 -- 172.21.15.49:0/3016601870 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 307160469 0 0) 0x7f90b806db20 con 0x7f90b815a880 2023-11-06T02:15:25.189 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.189+0000 7f90b759e640 1 --2- 172.21.15.49:0/3016601870 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90b806d150 0x7f90b806d5e0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f90b0009e50 tx=0x7f90b001ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65aac312453049bb server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:15:25.189 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.189+0000 7f90b5d9b640 1 -- 172.21.15.49:0/3016601870 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b00a57f0 con 0x7f90b806d150 2023-11-06T02:15:25.189 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.189+0000 7f90b5d9b640 1 -- 172.21.15.49:0/3016601870 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90b00a5950 con 0x7f90b806d150 2023-11-06T02:15:25.189 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.189+0000 7f90b5d9b640 1 -- 172.21.15.49:0/3016601870 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b00a5ce0 con 0x7f90b806d150 2023-11-06T02:15:25.190 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.189+0000 7f90bd01b640 1 -- 172.21.15.49:0/3016601870 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90b806d150 msgr2=0x7f90b806d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:25.190 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.189+0000 7f90bd01b640 1 --2- 172.21.15.49:0/3016601870 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90b806d150 0x7f90b806d5e0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f90b0009e50 tx=0x7f90b001ca10 comp rx=0 tx=0).stop 2023-11-06T02:15:25.190 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.189+0000 7f90bd01b640 1 -- 172.21.15.49:0/3016601870 shutdown_connections 2023-11-06T02:15:25.190 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.189+0000 7f90bd01b640 1 --2- 172.21.15.49:0/3016601870 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90b806d150 0x7f90b806d5e0 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:25.190 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.189+0000 7f90bd01b640 1 --2- 172.21.15.49:0/3016601870 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f90b8070870 0x7f90b80b15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:25.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.189+0000 7f90bd01b640 1 -- 172.21.15.49:0/3016601870 >> 172.21.15.49:0/3016601870 conn(0x7f90b806a370 msgr2=0x7f90b80ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:15:25.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.189+0000 7f90bd01b640 1 -- 172.21.15.49:0/3016601870 shutdown_connections 2023-11-06T02:15:25.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.189+0000 7f90bd01b640 1 -- 172.21.15.49:0/3016601870 wait complete. 2023-11-06T02:15:25.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.193+0000 7f90bd01b640 1 Processor -- start 2023-11-06T02:15:25.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.193+0000 7f90bd01b640 1 -- start start 2023-11-06T02:15:25.192 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.193+0000 7f90bd01b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90b806d150 0x7f90b8159ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:25.192 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.193+0000 7f90bd01b640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f90b8070870 0x7f90b80c7940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:25.192 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.193+0000 7f90bd01b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90b815a880 0x7f90b80c7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:25.192 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.193+0000 7f90bd01b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90b8159ff0 con 0x7f90b806d150 2023-11-06T02:15:25.192 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.193+0000 7f90bd01b640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90b80c98e0 con 0x7f90b815a880 2023-11-06T02:15:25.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.193+0000 7f90bd01b640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90b80c9bc0 con 0x7f90b8070870 2023-11-06T02:15:25.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.193+0000 7f90b6d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90b806d150 0x7f90b8159ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:25.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.193+0000 7f90b6d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90b806d150 0x7f90b8159ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35456/0 (socket says 172.21.15.49:35456) 2023-11-06T02:15:25.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.193+0000 7f90b6d9d640 1 -- 172.21.15.49:0/3396051921 learned_addr learned my addr 172.21.15.49:0/3396051921 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:15:25.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.193+0000 7f90b659c640 1 --2- 172.21.15.49:0/3396051921 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f90b8070870 0x7f90b80c7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:25.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.193+0000 7f90b6d9d640 1 -- 172.21.15.49:0/3396051921 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f90b8070870 msgr2=0x7f90b80c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:25.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.193+0000 7f90b6d9d640 1 --2- 172.21.15.49:0/3396051921 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f90b8070870 0x7f90b80c7940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:25.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.193+0000 7f90b759e640 1 --2- 172.21.15.49:0/3396051921 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90b815a880 0x7f90b80c7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:25.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.193+0000 7f90b6d9d640 1 -- 172.21.15.49:0/3396051921 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90b815a880 msgr2=0x7f90b80c7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:25.195 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.193+0000 7f90b6d9d640 1 --2- 172.21.15.49:0/3396051921 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90b815a880 0x7f90b80c7e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:25.195 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.193+0000 7f90b6d9d640 1 -- 172.21.15.49:0/3396051921 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90b0009b30 con 0x7f90b806d150 2023-11-06T02:15:25.195 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.193+0000 7f90b6d9d640 1 --2- 172.21.15.49:0/3396051921 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90b806d150 0x7f90b8159ab0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f90a00011f0 tx=0x7f90a000c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:15:25.195 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.193+0000 7f90af7fe640 1 -- 172.21.15.49:0/3396051921 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a00074e0 con 0x7f90b806d150 2023-11-06T02:15:25.196 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.193+0000 7f90af7fe640 1 -- 172.21.15.49:0/3396051921 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90a0007640 con 0x7f90b806d150 2023-11-06T02:15:25.196 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.193+0000 7f90bd01b640 1 -- 172.21.15.49:0/3396051921 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90b80c9ea0 con 0x7f90b806d150 2023-11-06T02:15:25.196 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.193+0000 7f90af7fe640 1 -- 172.21.15.49:0/3396051921 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a0004a10 con 0x7f90b806d150 2023-11-06T02:15:25.196 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.193+0000 7f90bd01b640 1 -- 172.21.15.49:0/3396051921 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90b80ca460 con 0x7f90b806d150 2023-11-06T02:15:25.197 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.193+0000 7f90bd01b640 1 -- 172.21.15.49:0/3396051921 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90b80ca680 con 0x7f90b806d150 2023-11-06T02:15:25.197 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.193+0000 7f90af7fe640 1 -- 172.21.15.49:0/3396051921 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f90a0004b70 con 0x7f90b806d150 2023-11-06T02:15:25.198 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.197+0000 7f90af7fe640 1 --2- 172.21.15.49:0/3396051921 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9098065a60 0x7f9098067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:25.198 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.197+0000 7f90af7fe640 1 -- 172.21.15.49:0/3396051921 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f90a00825c0 con 0x7f90b806d150 2023-11-06T02:15:25.198 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.197+0000 7f90b659c640 1 --2- 172.21.15.49:0/3396051921 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9098065a60 0x7f9098067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:25.198 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.197+0000 7f90b659c640 1 --2- 172.21.15.49:0/3396051921 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9098065a60 0x7f9098067f20 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f90b815bf70 tx=0x7f90a8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:15:25.198 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.201+0000 7f90af7fe640 1 -- 172.21.15.49:0/3396051921 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f90a0050070 con 0x7f90b806d150 2023-11-06T02:15:25.353 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.357+0000 7f90bd01b640 1 -- 172.21.15.49:0/3396051921 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f90b8074040 con 0x7f90b806d150 2023-11-06T02:15:25.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.369+0000 7f90af7fe640 1 -- 172.21.15.49:0/3396051921 <== mon.0 v2:172.21.15.49:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f90a0050070 con 0x7f90b806d150 2023-11-06T02:15:25.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.369+0000 7f90af7fe640 1 -- 172.21.15.49:0/3396051921 <== mon.0 v2:172.21.15.49:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v18)=0 v18) v1 ==== 123+0+0 (secure 0 0 0) 0x7f90a0053690 con 0x7f90b806d150 2023-11-06T02:15:25.367 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.369+0000 7f90bd01b640 1 -- 172.21.15.49:0/3396051921 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9098065a60 msgr2=0x7f9098067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:25.367 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.369+0000 7f90bd01b640 1 --2- 172.21.15.49:0/3396051921 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9098065a60 0x7f9098067f20 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f90b815bf70 tx=0x7f90a8008040 comp rx=0 tx=0).stop 2023-11-06T02:15:25.367 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.369+0000 7f90bd01b640 1 -- 172.21.15.49:0/3396051921 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90b806d150 msgr2=0x7f90b8159ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:25.367 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.369+0000 7f90bd01b640 1 --2- 172.21.15.49:0/3396051921 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90b806d150 0x7f90b8159ab0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f90a00011f0 tx=0x7f90a000c9b0 comp rx=0 tx=0).stop 2023-11-06T02:15:25.368 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.369+0000 7f90bd01b640 1 -- 172.21.15.49:0/3396051921 shutdown_connections 2023-11-06T02:15:25.368 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.369+0000 7f90bd01b640 1 --2- 172.21.15.49:0/3396051921 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f9098065a60 0x7f9098067f20 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:25.368 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.369+0000 7f90bd01b640 1 --2- 172.21.15.49:0/3396051921 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90b815a880 0x7f90b80c7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:25.368 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.369+0000 7f90bd01b640 1 --2- 172.21.15.49:0/3396051921 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f90b8070870 0x7f90b80c7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:25.369 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.369+0000 7f90bd01b640 1 --2- 172.21.15.49:0/3396051921 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90b806d150 0x7f90b8159ab0 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:25.369 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.373+0000 7f90bd01b640 1 -- 172.21.15.49:0/3396051921 >> 172.21.15.49:0/3396051921 conn(0x7f90b806a370 msgr2=0x7f90b80b3ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:15:25.369 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.373+0000 7f90bd01b640 1 -- 172.21.15.49:0/3396051921 shutdown_connections 2023-11-06T02:15:25.369 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.373+0000 7f90bd01b640 1 -- 172.21.15.49:0/3396051921 wait complete. 2023-11-06T02:15:25.386 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-06T02:15:25.386 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T02:15:25.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.509+0000 7f01ada3a640 1 Processor -- start 2023-11-06T02:15:25.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.509+0000 7f01ada3a640 1 -- start start 2023-11-06T02:15:25.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.509+0000 7f01ada3a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f01a80b1f90 0x7f01a80cda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:25.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.509+0000 7f01ada3a640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01a80d17c0 con 0x7f01a80714a0 2023-11-06T02:15:25.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.509+0000 7f01ada3a640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01a80ce6c0 con 0x7f01a8161650 2023-11-06T02:15:25.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.509+0000 7f01ada3a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01a8161ad0 con 0x7f01a80b1f90 2023-11-06T02:15:25.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.509+0000 7f01a77fe640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f01a80714a0 0x7f01a80b6060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:51504/0 (socket says 172.21.15.49:51504) 2023-11-06T02:15:25.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.509+0000 7f01a77fe640 1 -- 172.21.15.49:0/2317973526 learned_addr learned my addr 172.21.15.49:0/2317973526 (peer_addr_for_me v1:172.21.15.49:0/0) 2023-11-06T02:15:25.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.509+0000 7f01a6ffd640 1 --2- 172.21.15.49:0/2317973526 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f01a80b1f90 0x7f01a80cda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:25.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.509+0000 7f01a67fc640 1 -- 172.21.15.49:0/2317973526 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2709832717 0 0) 0x7f01a80d17c0 con 0x7f01a80714a0 2023-11-06T02:15:25.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.509+0000 7f01a67fc640 1 -- 172.21.15.49:0/2317973526 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0194003530 con 0x7f01a80714a0 2023-11-06T02:15:25.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.509+0000 7f01a67fc640 1 -- 172.21.15.49:0/2317973526 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1901335614 0 0) 0x7f0194003530 con 0x7f01a80714a0 2023-11-06T02:15:25.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.509+0000 7f01a67fc640 1 -- 172.21.15.49:0/2317973526 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f01a80b1f90 msgr2=0x7f01a80cda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:15:25.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.509+0000 7f01a67fc640 1 --2- 172.21.15.49:0/2317973526 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f01a80b1f90 0x7f01a80cda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:25.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.509+0000 7f01a67fc640 1 -- 172.21.15.49:0/2317973526 >> v1:172.21.15.150:6789/0 conn(0x7f01a8161650 legacy=0x7f01a80d1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:25.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.509+0000 7f01a67fc640 1 -- 172.21.15.49:0/2317973526 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01a80d2360 con 0x7f01a80714a0 2023-11-06T02:15:25.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.509+0000 7f01a67fc640 1 -- 172.21.15.49:0/2317973526 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1223735535 0 0) 0x7f01a0001a00 con 0x7f01a80714a0 2023-11-06T02:15:25.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.509+0000 7f01a67fc640 1 -- 172.21.15.49:0/2317973526 >> v1:172.21.15.49:6789/0 conn(0x7f01a80714a0 legacy=0x7f01a80b6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:25.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.509+0000 7f01a67fc640 1 --2- 172.21.15.49:0/2317973526 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f01940039b0 0x7f0194003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:25.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.509+0000 7f01a67fc640 1 --2- 172.21.15.49:0/2317973526 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f01a80b1f90 0x7f01940047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:25.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.509+0000 7f01a67fc640 1 --2- 172.21.15.49:0/2317973526 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0194004d10 0x7f0194005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:25.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.509+0000 7f01a67fc640 1 -- 172.21.15.49:0/2317973526 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a8161ad0 con 0x7f01940039b0 2023-11-06T02:15:25.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.509+0000 7f01a67fc640 1 -- 172.21.15.49:0/2317973526 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f019400ba40 con 0x7f0194004d10 2023-11-06T02:15:25.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.509+0000 7f01a67fc640 1 -- 172.21.15.49:0/2317973526 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f019400bc00 con 0x7f01a80b1f90 2023-11-06T02:15:25.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.509+0000 7f01a6ffd640 1 --2- 172.21.15.49:0/2317973526 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f01940039b0 0x7f0194003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:25.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.509+0000 7f019ffff640 1 --2- 172.21.15.49:0/2317973526 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f01a80b1f90 0x7f01940047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:25.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.509+0000 7f01a77fe640 1 --2- 172.21.15.49:0/2317973526 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0194004d10 0x7f0194005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:25.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.509+0000 7f01a6ffd640 1 -- 172.21.15.49:0/2317973526 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f01a80b1f90 msgr2=0x7f01940047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:15:25.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.509+0000 7f01a6ffd640 1 --2- 172.21.15.49:0/2317973526 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f01a80b1f90 0x7f01940047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:25.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.509+0000 7f01a6ffd640 1 -- 172.21.15.49:0/2317973526 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0194004d10 msgr2=0x7f0194005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T02:15:25.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.509+0000 7f01a6ffd640 1 --2- 172.21.15.49:0/2317973526 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0194004d10 0x7f0194005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:25.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.509+0000 7f01a6ffd640 1 -- 172.21.15.49:0/2317973526 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0194003530 con 0x7f01940039b0 2023-11-06T02:15:25.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.509+0000 7f01a77fe640 1 --2- 172.21.15.49:0/2317973526 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0194004d10 0x7f0194005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T02:15:25.516 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.513+0000 7f01a6ffd640 1 --2- 172.21.15.49:0/2317973526 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f01940039b0 0x7f0194003dd0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f0198009ab0 tx=0x7f019801ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4bfcc9545241840 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:15:25.517 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.513+0000 7f01a67fc640 1 -- 172.21.15.49:0/2317973526 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01980a64f0 con 0x7f01940039b0 2023-11-06T02:15:25.517 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.513+0000 7f01a67fc640 1 -- 172.21.15.49:0/2317973526 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f01980a6650 con 0x7f01940039b0 2023-11-06T02:15:25.517 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.513+0000 7f01a67fc640 1 -- 172.21.15.49:0/2317973526 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01980a6a20 con 0x7f01940039b0 2023-11-06T02:15:25.517 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.513+0000 7f01ada3a640 1 -- 172.21.15.49:0/2317973526 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f01940039b0 msgr2=0x7f0194003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:25.518 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.513+0000 7f01ada3a640 1 --2- 172.21.15.49:0/2317973526 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f01940039b0 0x7f0194003dd0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f0198009ab0 tx=0x7f019801ca00 comp rx=0 tx=0).stop 2023-11-06T02:15:25.518 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.513+0000 7f01a77fe640 1 -- 172.21.15.49:0/2317973526 reap_dead start 2023-11-06T02:15:25.518 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.513+0000 7f01ada3a640 1 -- 172.21.15.49:0/2317973526 shutdown_connections 2023-11-06T02:15:25.518 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.513+0000 7f01ada3a640 1 -- 172.21.15.49:0/2317973526 >> 172.21.15.49:0/2317973526 conn(0x7f01a80d2aa0 msgr2=0x7f01a80d2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:15:25.518 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.513+0000 7f01ada3a640 1 -- 172.21.15.49:0/2317973526 shutdown_connections 2023-11-06T02:15:25.519 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.513+0000 7f01ada3a640 1 -- 172.21.15.49:0/2317973526 wait complete. 2023-11-06T02:15:25.519 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.513+0000 7f01ada3a640 1 Processor -- start 2023-11-06T02:15:25.519 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.517+0000 7f01ada3a640 1 -- start start 2023-11-06T02:15:25.519 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.517+0000 7f01ada3a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01940039b0 0x7f01a8161f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:25.519 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.517+0000 7f01ada3a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f01a81624d0 0x7f01a8165d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:25.520 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.517+0000 7f01ada3a640 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f01a8159330 0x7f01a815b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:25.520 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.517+0000 7f01ada3a640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a815b0e0 con 0x7f01a81624d0 2023-11-06T02:15:25.520 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.517+0000 7f01ada3a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a8166750 con 0x7f01940039b0 2023-11-06T02:15:25.520 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.517+0000 7f01ada3a640 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01a815aa90 con 0x7f01a8159330 2023-11-06T02:15:25.520 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.517+0000 7f019ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f01a81624d0 0x7f01a8165d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:25.520 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.517+0000 7f019ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f01a81624d0 0x7f01a8165d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35470/0 (socket says 172.21.15.49:35470) 2023-11-06T02:15:25.521 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.517+0000 7f019ffff640 1 -- 172.21.15.49:0/1825482320 learned_addr learned my addr 172.21.15.49:0/1825482320 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-11-06T02:15:25.521 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.517+0000 7f019ffff640 1 -- 172.21.15.49:0/1825482320 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f01a8159330 msgr2=0x7f01a815b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:25.521 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.517+0000 7f01a77fe640 1 --2- 172.21.15.49:0/1825482320 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f01a8159330 0x7f01a815b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:25.521 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.517+0000 7f01a6ffd640 1 --2- 172.21.15.49:0/1825482320 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01940039b0 0x7f01a8161f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:25.522 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.517+0000 7f019ffff640 1 --2- 172.21.15.49:0/1825482320 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f01a8159330 0x7f01a815b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:25.522 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.517+0000 7f019ffff640 1 -- 172.21.15.49:0/1825482320 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01940039b0 msgr2=0x7f01a8161f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:25.522 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.517+0000 7f019ffff640 1 --2- 172.21.15.49:0/1825482320 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01940039b0 0x7f01a8161f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:25.522 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.517+0000 7f019ffff640 1 -- 172.21.15.49:0/1825482320 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01a815ad70 con 0x7f01a81624d0 2023-11-06T02:15:25.522 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.517+0000 7f019ffff640 1 --2- 172.21.15.49:0/1825482320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f01a81624d0 0x7f01a8165d20 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f019000e550 tx=0x7f019000ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:15:25.523 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.517+0000 7f01a4ff9640 1 -- 172.21.15.49:0/1825482320 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01900052e0 con 0x7f01a81624d0 2023-11-06T02:15:25.523 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.517+0000 7f01a4ff9640 1 -- 172.21.15.49:0/1825482320 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f0190005440 con 0x7f01a81624d0 2023-11-06T02:15:25.523 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.517+0000 7f01ada3a640 1 -- 172.21.15.49:0/1825482320 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01a8166a30 con 0x7f01a81624d0 2023-11-06T02:15:25.523 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.517+0000 7f01a4ff9640 1 -- 172.21.15.49:0/1825482320 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0190003840 con 0x7f01a81624d0 2023-11-06T02:15:25.523 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.517+0000 7f01ada3a640 1 -- 172.21.15.49:0/1825482320 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01a8166430 con 0x7f01a81624d0 2023-11-06T02:15:25.524 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.517+0000 7f01ada3a640 1 -- 172.21.15.49:0/1825482320 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01a815b250 con 0x7f01a81624d0 2023-11-06T02:15:25.524 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.517+0000 7f01a4ff9640 1 -- 172.21.15.49:0/1825482320 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f01900039a0 con 0x7f01a81624d0 2023-11-06T02:15:25.524 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.517+0000 7f01a4ff9640 1 --2- 172.21.15.49:0/1825482320 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0188065a60 0x7f0188067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T02:15:25.524 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.517+0000 7f01a4ff9640 1 -- 172.21.15.49:0/1825482320 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f0190082ee0 con 0x7f01a81624d0 2023-11-06T02:15:25.524 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.517+0000 7f01a6ffd640 1 --2- 172.21.15.49:0/1825482320 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0188065a60 0x7f0188067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T02:15:25.525 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.521+0000 7f01a6ffd640 1 --2- 172.21.15.49:0/1825482320 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0188065a60 0x7f0188067f20 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f01980af000 tx=0x7f01980046d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T02:15:25.525 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.525+0000 7f01a4ff9640 1 -- 172.21.15.49:0/1825482320 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f0190050990 con 0x7f01a81624d0 2023-11-06T02:15:25.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.713+0000 7f01ada3a640 1 -- 172.21.15.49:0/1825482320 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f01a8074040 con 0x7f01a81624d0 2023-11-06T02:15:25.709 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.713+0000 7f01a4ff9640 1 -- 172.21.15.49:0/1825482320 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+3529 (secure 0 0 0) 0x7f0190053fb0 con 0x7f01a81624d0 2023-11-06T02:15:25.709 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-11-06T02:15:25.710 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":61,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5714,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/315581318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":315581318},{"type":"v1","addr":"172.21.15.150:6837","nonce":315581318}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41},{"gid":6121,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/136800245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":136800245},{"type":"v1","addr":"172.21.15.150:6835","nonce":136800245}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41},{"gid":7110,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6833/4259579254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":4259579254},{"type":"v1","addr":"172.21.15.49:6833","nonce":4259579254}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41},{"gid":8001,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/3105033804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3105033804},{"type":"v1","addr":"172.21.15.49:6835","nonce":3105033804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61}],"filesystems":[]} 2023-11-06T02:15:25.710 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 61 2023-11-06T02:15:25.711 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.717+0000 7f01ada3a640 1 -- 172.21.15.49:0/1825482320 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0188065a60 msgr2=0x7f0188067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:25.711 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.717+0000 7f01ada3a640 1 --2- 172.21.15.49:0/1825482320 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0188065a60 0x7f0188067f20 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f01980af000 tx=0x7f01980046d0 comp rx=0 tx=0).stop 2023-11-06T02:15:25.712 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.717+0000 7f01ada3a640 1 -- 172.21.15.49:0/1825482320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f01a81624d0 msgr2=0x7f01a8165d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T02:15:25.712 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.717+0000 7f01ada3a640 1 --2- 172.21.15.49:0/1825482320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f01a81624d0 0x7f01a8165d20 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f019000e550 tx=0x7f019000ea10 comp rx=0 tx=0).stop 2023-11-06T02:15:25.712 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.717+0000 7f01ada3a640 1 -- 172.21.15.49:0/1825482320 shutdown_connections 2023-11-06T02:15:25.712 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.717+0000 7f01ada3a640 1 --2- 172.21.15.49:0/1825482320 >> [v2:172.21.15.150:6832/38104,v1:172.21.15.150:6833/38104] conn(0x7f0188065a60 0x7f0188067f20 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:25.713 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.717+0000 7f01ada3a640 1 --2- 172.21.15.49:0/1825482320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f01a81624d0 0x7f01a8165d20 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:25.713 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.717+0000 7f01ada3a640 1 --2- 172.21.15.49:0/1825482320 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f01a8159330 0x7f01a815b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:25.713 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.717+0000 7f01ada3a640 1 --2- 172.21.15.49:0/1825482320 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01940039b0 0x7f01a8161f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T02:15:25.713 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.717+0000 7f01ada3a640 1 -- 172.21.15.49:0/1825482320 >> 172.21.15.49:0/1825482320 conn(0x7f01a80d2aa0 msgr2=0x7f01a80b8180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T02:15:25.714 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.717+0000 7f01ada3a640 1 -- 172.21.15.49:0/1825482320 shutdown_connections 2023-11-06T02:15:25.714 INFO:teuthology.orchestra.run.smithi049.stderr:2023-11-06T02:15:25.717+0000 7f01ada3a640 1 -- 172.21.15.49:0/1825482320 wait complete. 2023-11-06T02:15:25.728 DEBUG:tasks.cephfs.filesystem:already dead... 2023-11-06T02:15:25.728 INFO:teuthology.misc:Shutting down mds daemons... 2023-11-06T02:15:25.728 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-06T02:15:25.729 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T02:15:25.737 INFO:tasks.ceph.mds.a:Stopped 2023-11-06T02:15:25.738 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-06T02:15:25.738 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T02:15:25.795 INFO:tasks.ceph.mds.c:Stopped 2023-11-06T02:15:25.795 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-06T02:15:25.795 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T02:15:25.814 INFO:tasks.ceph.mds.b:Stopped 2023-11-06T02:15:25.815 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-06T02:15:25.815 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T02:15:25.868 INFO:tasks.ceph.mds.d:Stopped 2023-11-06T02:15:25.869 INFO:teuthology.misc:Shutting down osd daemons... 2023-11-06T02:15:25.869 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-11-06T02:15:25.870 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T02:15:25.965 INFO:tasks.ceph.osd.0:Stopped 2023-11-06T02:15:25.965 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-11-06T02:15:25.966 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T02:15:29.891 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2023-11-06T02:15:32.068 INFO:tasks.ceph.osd.1:Stopped 2023-11-06T02:15:32.069 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-11-06T02:15:32.069 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T02:15:32.142 INFO:tasks.ceph.osd.2:Stopped 2023-11-06T02:15:32.143 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-11-06T02:15:32.143 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T02:15:32.210 INFO:tasks.ceph.osd.3:Stopped 2023-11-06T02:15:32.210 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-11-06T02:15:32.210 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T02:15:32.290 INFO:tasks.ceph.osd.4:Stopped 2023-11-06T02:15:32.291 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-11-06T02:15:32.291 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T02:15:32.371 INFO:tasks.ceph.osd.5:Stopped 2023-11-06T02:15:32.372 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-11-06T02:15:32.372 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T02:15:32.473 INFO:tasks.ceph.osd.6:Stopped 2023-11-06T02:15:32.474 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-11-06T02:15:32.474 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T02:15:35.396 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2023-11-06T02:15:35.397 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2023-11-06T02:15:38.576 INFO:tasks.ceph.osd.7:Stopped 2023-11-06T02:15:38.577 INFO:teuthology.misc:Shutting down mgr daemons... 2023-11-06T02:15:38.577 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-11-06T02:15:38.577 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T02:15:38.605 INFO:tasks.ceph.mgr.y:Stopped 2023-11-06T02:15:38.605 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-11-06T02:15:38.606 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T02:15:38.635 INFO:tasks.ceph.mgr.x:Stopped 2023-11-06T02:15:38.636 INFO:teuthology.misc:Shutting down mon daemons... 2023-11-06T02:15:38.636 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-11-06T02:15:38.636 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T02:15:38.666 INFO:tasks.ceph.mon.a:Stopped 2023-11-06T02:15:38.667 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-11-06T02:15:38.667 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T02:15:38.692 INFO:tasks.ceph.mon.b:Stopped 2023-11-06T02:15:38.693 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-11-06T02:15:38.693 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T02:15:38.758 INFO:tasks.ceph.mon.c:Stopped 2023-11-06T02:15:38.759 INFO:tasks.ceph:Checking cluster log for badness... 2023-11-06T02:15:38.759 DEBUG:teuthology.orchestra.run.smithi049:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'inode wrongly marked free' | egrep -v 'bad backtrace on inode' | egrep -v 'inode table repaired for inode' | egrep -v 'Scrub error on inode' | egrep -v 'Scrub error on dir' | egrep -v 'Metadata damage detected' | head -n 1 2023-11-06T02:15:38.777 INFO:teuthology.orchestra.run.smithi049.stdout:2023-11-06T02:08:26.039686+0000 mon.a (mon.0) 120 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-11-06T02:15:38.777 WARNING:tasks.ceph:Found errors (ERR|WRN|SEC) in cluster log 2023-11-06T02:15:38.778 DEBUG:teuthology.orchestra.run.smithi049:> sudo egrep '\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'inode wrongly marked free' | egrep -v 'bad backtrace on inode' | egrep -v 'inode table repaired for inode' | egrep -v 'Scrub error on inode' | egrep -v 'Scrub error on dir' | egrep -v 'Metadata damage detected' | head -n 1 2023-11-06T02:15:38.833 DEBUG:teuthology.orchestra.run.smithi049:> sudo egrep '\[ERR\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'inode wrongly marked free' | egrep -v 'bad backtrace on inode' | egrep -v 'inode table repaired for inode' | egrep -v 'Scrub error on inode' | egrep -v 'Scrub error on dir' | egrep -v 'Metadata damage detected' | head -n 1 2023-11-06T02:15:38.889 DEBUG:teuthology.orchestra.run.smithi049:> sudo egrep '\[WRN\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'inode wrongly marked free' | egrep -v 'bad backtrace on inode' | egrep -v 'inode table repaired for inode' | egrep -v 'Scrub error on inode' | egrep -v 'Scrub error on dir' | egrep -v 'Metadata damage detected' | head -n 1 2023-11-06T02:15:38.945 INFO:teuthology.orchestra.run.smithi049.stdout:2023-11-06T02:08:26.039686+0000 mon.a (mon.0) 120 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-11-06T02:15:38.946 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi049.front.sepia.ceph.com 2023-11-06T02:15:38.946 DEBUG:teuthology.orchestra.run.smithi049:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-11-06T02:15:39.181 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi049.front.sepia.ceph.com 2023-11-06T02:15:39.181 DEBUG:teuthology.orchestra.run.smithi049:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-11-06T02:15:39.370 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi049.front.sepia.ceph.com 2023-11-06T02:15:39.370 DEBUG:teuthology.orchestra.run.smithi049:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-11-06T02:15:39.545 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi049.front.sepia.ceph.com 2023-11-06T02:15:39.546 DEBUG:teuthology.orchestra.run.smithi049:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-11-06T02:15:39.713 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi150.front.sepia.ceph.com 2023-11-06T02:15:39.713 DEBUG:teuthology.orchestra.run.smithi150:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-11-06T02:15:39.961 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi150.front.sepia.ceph.com 2023-11-06T02:15:39.962 DEBUG:teuthology.orchestra.run.smithi150:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-11-06T02:15:40.147 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi150.front.sepia.ceph.com 2023-11-06T02:15:40.147 DEBUG:teuthology.orchestra.run.smithi150:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-11-06T02:15:40.291 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi150.front.sepia.ceph.com 2023-11-06T02:15:40.291 DEBUG:teuthology.orchestra.run.smithi150:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-11-06T02:15:40.397 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 has been restored 2023-11-06T02:15:40.431 INFO:tasks.ceph:Archiving mon data... 2023-11-06T02:15:40.433 DEBUG:teuthology.misc:Transferring archived files from smithi049:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/yuriw-2023-11-05_23:44:49-fs-reef-release-distro-default-smithi/7448684/data/mon.a.tgz 2023-11-06T02:15:40.433 DEBUG:teuthology.orchestra.run.smithi049:> mktemp 2023-11-06T02:15:40.440 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/tmp.X6VwUq3qSh 2023-11-06T02:15:40.440 DEBUG:teuthology.orchestra.run.smithi049:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.X6VwUq3qSh 2023-11-06T02:15:40.585 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod 0666 /tmp/tmp.X6VwUq3qSh 2023-11-06T02:15:40.652 DEBUG:teuthology.orchestra.remote:smithi049:/tmp/tmp.X6VwUq3qSh is 364KB 2023-11-06T02:15:40.712 DEBUG:teuthology.orchestra.run.smithi049:> rm -fr /tmp/tmp.X6VwUq3qSh 2023-11-06T02:15:40.718 DEBUG:teuthology.misc:Transferring archived files from smithi150:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/yuriw-2023-11-05_23:44:49-fs-reef-release-distro-default-smithi/7448684/data/mon.b.tgz 2023-11-06T02:15:40.718 DEBUG:teuthology.orchestra.run.smithi150:> mktemp 2023-11-06T02:15:40.724 INFO:teuthology.orchestra.run.smithi150.stdout:/tmp/tmp.WLfYqehlPF 2023-11-06T02:15:40.725 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.WLfYqehlPF 2023-11-06T02:15:40.865 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 0666 /tmp/tmp.WLfYqehlPF 2023-11-06T02:15:40.928 DEBUG:teuthology.orchestra.remote:smithi150:/tmp/tmp.WLfYqehlPF is 369KB 2023-11-06T02:15:40.990 DEBUG:teuthology.orchestra.run.smithi150:> rm -fr /tmp/tmp.WLfYqehlPF 2023-11-06T02:15:40.996 DEBUG:teuthology.misc:Transferring archived files from smithi150:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/yuriw-2023-11-05_23:44:49-fs-reef-release-distro-default-smithi/7448684/data/mon.c.tgz 2023-11-06T02:15:40.997 DEBUG:teuthology.orchestra.run.smithi150:> mktemp 2023-11-06T02:15:41.044 INFO:teuthology.orchestra.run.smithi150.stdout:/tmp/tmp.6q1uKehJ5w 2023-11-06T02:15:41.045 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.6q1uKehJ5w 2023-11-06T02:15:41.178 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 0666 /tmp/tmp.6q1uKehJ5w 2023-11-06T02:15:41.240 DEBUG:teuthology.orchestra.remote:smithi150:/tmp/tmp.6q1uKehJ5w is 369KB 2023-11-06T02:15:41.297 DEBUG:teuthology.orchestra.run.smithi150:> rm -fr /tmp/tmp.6q1uKehJ5w 2023-11-06T02:15:41.302 INFO:tasks.ceph:Cleaning ceph cluster... 2023-11-06T02:15:41.303 DEBUG:teuthology.orchestra.run.smithi049:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-11-06T02:15:41.305 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-11-06T02:15:41.356 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-11-06T02:15:41.357 DEBUG:teuthology.orchestra.run.smithi049:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-11-06T02:15:41.364 DEBUG:teuthology.orchestra.run.smithi150:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-11-06T02:15:41.388 INFO:teuthology.orchestra.run.smithi049.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-11-06T02:15:41.423 INFO:teuthology.orchestra.run.smithi150.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-11-06T02:15:41.427 INFO:tasks.ceph:Archiving crash dumps... 2023-11-06T02:15:41.432 DEBUG:teuthology.misc:Transferring archived files from smithi049:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-11-05_23:44:49-fs-reef-release-distro-default-smithi/7448684/remote/smithi049/crash 2023-11-06T02:15:41.433 DEBUG:teuthology.orchestra.run.smithi049:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-11-06T02:15:41.449 DEBUG:teuthology.misc:Transferring archived files from smithi150:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-11-05_23:44:49-fs-reef-release-distro-default-smithi/7448684/remote/smithi150/crash 2023-11-06T02:15:41.450 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-11-06T02:15:41.481 INFO:tasks.ceph:Shutting down logrotate 2023-11-06T02:15:41.481 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T02:15:41.500 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T02:15:41.571 DEBUG:teuthology.orchestra.run.smithi049:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-11-06T02:15:41.589 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-11-06T02:15:41.604 INFO:tasks.ceph:Compressing logs... 2023-11-06T02:15:41.604 DEBUG:teuthology.orchestra.run.smithi049:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-06T02:15:41.634 DEBUG:teuthology.orchestra.run.smithi150:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-06T02:15:44.247 INFO:tasks.ceph:Archiving logs... 2023-11-06T02:15:44.247 DEBUG:teuthology.misc:Transferring archived files from smithi049:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-11-05_23:44:49-fs-reef-release-distro-default-smithi/7448684/remote/smithi049/log 2023-11-06T02:15:44.248 DEBUG:teuthology.orchestra.run.smithi049:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-06T02:15:44.977 DEBUG:teuthology.misc:Transferring archived files from smithi150:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-11-05_23:44:49-fs-reef-release-distro-default-smithi/7448684/remote/smithi150/log 2023-11-06T02:15:44.978 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-06T02:15:45.239 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-11-06T02:15:45.252 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-11-06T02:15:45.252 DEBUG:teuthology.orchestra.run.smithi049:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-06T02:15:45.255 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-06T02:15:45.329 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-11-06T02:15:45.339 INFO:teuthology.task.clock:Checking final clock skew... 2023-11-06T02:15:45.339 DEBUG:teuthology.orchestra.run.smithi049:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-06T02:15:45.342 DEBUG:teuthology.orchestra.run.smithi150:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-06T02:15:45.353 INFO:teuthology.orchestra.run.smithi049.stdout: remote refid st t when poll reach delay offset jitter 2023-11-06T02:15:45.354 INFO:teuthology.orchestra.run.smithi049.stdout:============================================================================== 2023-11-06T02:15:45.354 INFO:teuthology.orchestra.run.smithi049.stdout:*hv01.front.sepi 44.4.53.2 2 u 4 64 377 0.108 -2.455 1.393 2023-11-06T02:15:45.354 INFO:teuthology.orchestra.run.smithi049.stdout:+hv02.front.sepi 63.231.80.2 3 u 2 64 377 0.073 -3.741 2.417 2023-11-06T02:15:45.354 INFO:teuthology.orchestra.run.smithi049.stdout:+hv03.front.sepi 74.6.168.72 3 u 2 64 377 0.093 -1.159 1.446 2023-11-06T02:15:45.355 INFO:teuthology.orchestra.run.smithi049.stdout: hv04.front.sepi .INIT. 16 u - 128 0 0.000 +0.000 0.000 2023-11-06T02:15:45.356 INFO:teuthology.orchestra.run.smithi150.stdout: remote refid st t when poll reach delay offset jitter 2023-11-06T02:15:45.356 INFO:teuthology.orchestra.run.smithi150.stdout:============================================================================== 2023-11-06T02:15:45.356 INFO:teuthology.orchestra.run.smithi150.stdout:*hv01.front.sepi 44.4.53.2 2 u 61 64 377 0.070 -1.501 0.961 2023-11-06T02:15:45.357 INFO:teuthology.orchestra.run.smithi150.stdout:+hv02.front.sepi 63.231.80.2 3 u 1 64 377 0.100 -0.403 0.942 2023-11-06T02:15:45.357 INFO:teuthology.orchestra.run.smithi150.stdout:+hv03.front.sepi 74.6.168.72 3 u 2 64 377 0.089 -2.067 1.505 2023-11-06T02:15:45.357 INFO:teuthology.orchestra.run.smithi150.stdout: hv04.front.sepi .INIT. 16 u - 128 0 0.000 +0.000 0.000 2023-11-06T02:15:45.357 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-11-06T02:15:45.369 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-11-06T02:15:45.370 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-11-06T02:15:45.384 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-11-06T02:15:45.399 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-11-06T02:15:45.414 INFO:teuthology.task.internal:Duration was 1133.238808 seconds 2023-11-06T02:15:45.414 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-11-06T02:15:45.429 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-11-06T02:15:45.429 DEBUG:teuthology.orchestra.run.smithi049:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-06T02:15:45.432 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-06T02:15:45.507 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-11-06T02:15:45.507 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi049.front.sepia.ceph.com 2023-11-06T02:15:45.508 DEBUG:teuthology.orchestra.run.smithi049:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-06T02:15:45.518 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi150.front.sepia.ceph.com 2023-11-06T02:15:45.518 DEBUG:teuthology.orchestra.run.smithi150:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-06T02:15:45.531 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-11-06T02:15:45.531 DEBUG:teuthology.orchestra.run.smithi049:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-06T02:15:45.564 DEBUG:teuthology.orchestra.run.smithi150:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-06T02:15:45.593 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-11-06T02:15:45.594 DEBUG:teuthology.orchestra.run.smithi049:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-06T02:15:45.632 DEBUG:teuthology.orchestra.run.smithi150:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-06T02:15:45.701 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-11-06T02:15:45.712 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-11-06T02:15:45.712 DEBUG:teuthology.orchestra.run.smithi049:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-06T02:15:45.726 DEBUG:teuthology.orchestra.run.smithi150:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-06T02:15:45.736 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-11-06T02:15:45.748 DEBUG:teuthology.orchestra.run.smithi049:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-06T02:15:45.768 DEBUG:teuthology.orchestra.run.smithi150:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-06T02:15:45.776 INFO:teuthology.orchestra.run.smithi049.stdout:kernel.core_pattern = core 2023-11-06T02:15:45.790 INFO:teuthology.orchestra.run.smithi150.stdout:kernel.core_pattern = core 2023-11-06T02:15:45.804 DEBUG:teuthology.orchestra.run.smithi049:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-06T02:15:45.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T02:15:45.835 DEBUG:teuthology.orchestra.run.smithi150:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-06T02:15:45.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T02:15:45.851 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-11-06T02:15:45.861 INFO:teuthology.task.internal:Transferring archived files... 2023-11-06T02:15:45.862 DEBUG:teuthology.misc:Transferring archived files from smithi049:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-11-05_23:44:49-fs-reef-release-distro-default-smithi/7448684/remote/smithi049 2023-11-06T02:15:45.863 DEBUG:teuthology.orchestra.run.smithi049:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-06T02:15:45.897 DEBUG:teuthology.misc:Transferring archived files from smithi150:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-11-05_23:44:49-fs-reef-release-distro-default-smithi/7448684/remote/smithi150 2023-11-06T02:15:45.898 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-06T02:15:45.917 INFO:teuthology.task.internal:Removing archive directory... 2023-11-06T02:15:45.917 DEBUG:teuthology.orchestra.run.smithi049:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-06T02:15:45.936 DEBUG:teuthology.orchestra.run.smithi150:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-06T02:15:45.960 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-11-06T02:15:45.973 INFO:teuthology.task.internal:Not uploading archives. 2023-11-06T02:15:45.974 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-11-06T02:15:45.987 INFO:teuthology.task.internal:Tidying up after the test... 2023-11-06T02:15:45.987 DEBUG:teuthology.orchestra.run.smithi049:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-06T02:15:45.990 DEBUG:teuthology.orchestra.run.smithi150:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-06T02:15:45.994 INFO:teuthology.orchestra.run.smithi049.stdout: 655472 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 6 02:15 /home/ubuntu/cephtest 2023-11-06T02:15:46.008 INFO:teuthology.orchestra.run.smithi150.stdout: 655472 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 6 02:15 /home/ubuntu/cephtest 2023-11-06T02:15:46.009 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-11-06T02:15:46.021 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-11-06T02:15:46.047 INFO:teuthology.nuke:Checking targets against current locks 2023-11-06T02:15:46.073 DEBUG:teuthology.nuke:shortname: smithi049 2023-11-06T02:15:46.073 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-06T02:15:46.095 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi049.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-11-05_23:44:49-fs-reef-release-distro-default-smithi/7448684', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-06 01:45:25.558695', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBExxTF5Kt7KZ8Ef/cLx/cqs3AWnVQ4ai7T1jEDiGZzy9Q5LtI0OPverOvitoCFdXpAojj0/ANPgMelc6AP0UzDA='} 2023-11-06T02:15:46.100 DEBUG:teuthology.nuke:shortname: smithi150 2023-11-06T02:15:46.101 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-06T02:15:46.122 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi150.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-11-05_23:44:49-fs-reef-release-distro-default-smithi/7448684', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-06 01:45:25.557230', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHAJnIp8mtEt40gfMIRsjg5lCglKz1ohJ2WB8MsZn64LZXl/9/UGBQ4oYaVUsfH3FRK1MXjvSQFDp7zwxCN2mjg='} 2023-11-06T02:15:46.143 INFO:teuthology.orchestra.console.smithi049:Power off 2023-11-06T02:15:46.144 DEBUG:teuthology.orchestra.console.smithi049:pexpect command: ipmitool -H smithi049.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-06T02:15:46.168 DEBUG:teuthology.orchestra.console.smithi049:power off output: Chassis Power Control: Down/Off 2023-11-06T02:15:46.168 DEBUG:teuthology.orchestra.console.smithi049:pexpect command: ipmitool -H smithi049.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-06T02:15:46.185 INFO:teuthology.orchestra.console.smithi150:Power off 2023-11-06T02:15:46.186 DEBUG:teuthology.orchestra.console.smithi150:pexpect command: ipmitool -H smithi150.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-06T02:15:46.204 DEBUG:teuthology.orchestra.console.smithi049:check power output: Chassis Power is on 2023-11-06T02:15:46.209 DEBUG:teuthology.orchestra.console.smithi150:power off output: Chassis Power Control: Down/Off 2023-11-06T02:15:46.209 DEBUG:teuthology.orchestra.console.smithi150:pexpect command: ipmitool -H smithi150.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-06T02:15:46.242 DEBUG:teuthology.orchestra.console.smithi150:check power output: Chassis Power is on 2023-11-06T02:15:50.206 DEBUG:teuthology.orchestra.console.smithi049:pexpect command: ipmitool -H smithi049.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-06T02:15:50.243 DEBUG:teuthology.orchestra.console.smithi150:pexpect command: ipmitool -H smithi150.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-06T02:15:50.317 DEBUG:teuthology.orchestra.console.smithi049:check power output: Chassis Power is on 2023-11-06T02:15:50.353 DEBUG:teuthology.orchestra.console.smithi150:check power output: Chassis Power is on 2023-11-06T02:15:54.322 DEBUG:teuthology.orchestra.console.smithi049:pexpect command: ipmitool -H smithi049.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-06T02:15:54.354 DEBUG:teuthology.orchestra.console.smithi150:pexpect command: ipmitool -H smithi150.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-06T02:15:54.434 DEBUG:teuthology.orchestra.console.smithi049:check power output: Chassis Power is on 2023-11-06T02:15:54.464 DEBUG:teuthology.orchestra.console.smithi150:check power output: Chassis Power is off 2023-11-06T02:15:54.565 INFO:teuthology.orchestra.console.smithi150:Power off completed 2023-11-06T02:15:58.438 DEBUG:teuthology.orchestra.console.smithi049:pexpect command: ipmitool -H smithi049.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-06T02:15:58.548 DEBUG:teuthology.orchestra.console.smithi049:check power output: Chassis Power is off 2023-11-06T02:15:58.649 INFO:teuthology.orchestra.console.smithi049:Power off completed 2023-11-06T02:15:58.753 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/forward-scrub} duration: 1133.2388079166412 failure_reason: '"2023-11-06T02:08:26.039686+0000 mon.a (mon.0) 120 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log' flavor: default owner: scheduled_yuriw@teuthology success: false 2023-11-06T02:15:58.754 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-06T02:15:58.914 INFO:teuthology.run:FAIL